Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
feature/Dry-run-as-temporal-activity (#982)
Louis Choquel5942376ac3d6
diff --git a/.claude/skills/temporal-e2e-validate/SKILL.md b/.claude/skills/temporal-e2e-validate/SKILL.mdindex f42ef170b..cbb561d78 100644--- a/.claude/skills/temporal-e2e-validate/SKILL.md+++ b/.claude/skills/temporal-e2e-validate/SKILL.md@@ -10,7 +10,7 @@ description: >concurrent isolation, image payload storage, cross-worker graphtracing with GraphSpec assembly, and cross-worker cost-report assembly(runner-side usage aggregated into a single submitter cost report, validated- cheaply via `--mock-inference`). Step 8 validates v1 per-activity routing.+ cheaply via the hidden `--dry-run --mock-usage` trigger). Step 8 validates v1 per-activity routing.Step 9 validates v2 per-queue submitter options (timeouts, retry,rate-limit), per-handle option overrides, named worker-runtime profilesselected via `--profile`, and the strict `--task-queue` CLI typo check@@ -19,17 +19,19 @@ description: >parity arm); Mode 2 Tiers 13-16 validate that a worker-side failure — from anLLM, extract, image-gen, or batched child-workflow activity — carriesa structured `ErrorReport` across the activity -> workflow -> submitter- boundary.+ boundary. Tier 17 validates that a `--temporal --dry-run` run dispatches the+ real inference activities and mocks INSIDE them (run mode orthogonal to+ backend — no API keys, no storage IO, zero-token suppressed usage).Use when the user says "validate temporal", "e2e temporal","temporal regression", "temporal validation", "3-process test", "full temporaltest", "validate phases", "image temporal", "queue options", "runtime profile","validate error handling", "error report", "error propagation",- "cost report", "distributed cost", "mock inference",+ "cost report", "distributed cost", "mock usage",or wants comprehensive verification that distributed execution worksend-to-end. Also use proactively after changes to LibraryCrate, deferredhydration, ClassRegistry scoping, graph tracing, image generation activities,content storage, queue options resolution, worker-runtime profiles,- usage / cost reporting, the `--costs` / `--mock-inference` flags,+ usage / cost reporting, the `--costs` / `--mock-usage` flags,error classification / `ErrorReport` / the Temporal error boundary, orTemporal workflow code.allowed-tools:@@ -305,8 +307,9 @@ loaded only for the work at hand. Read them as needed:- "validate temporal" / "full temporal test" / a broad regression check → `mode-2-setup.md`, then `mode-2-tiers.md`; offer the two batteries as opt-in extras.- "validate temporal error handling" / "error report" / "error propagation" → Mode 1 Step 2b above (the precise pytest assertions) **plus** `mode-2-setup.md` + `mode-2-tiers.md` Step 5f (Tiers 13–16, cross-process).-- "cost report" / "distributed cost" / "mock inference" → `mode-2-setup.md` (split workers), then `mode-2-tiers.md` Step 5b' (Tier 8b). This routing is **scope-aware**: once a request has landed here (via one of those three triggers), read the scope manifest at the top of Step 5b' and run the arms it selects:- - **default** (bare "cost report" / "distributed cost" / "mock inference") → the free, deterministic mock arms only (`--mock-inference`); no live spend.+- "dry run temporal" / "dry-run dispatch" / "leaf mock" → `mode-2-setup.md` + `mode-2-tiers.md` Step 5g (Tier 17: DRY honors the backend — dispatch + mock inside the activity, no keys needed), plus the Mode-1 pytest `tests/integration/pipelex/temporal/tracing/test_dry_run_dispatches_and_mocks.py`.+- "cost report" / "distributed cost" / "mock usage" → `mode-2-setup.md` (split workers), then `mode-2-tiers.md` Step 5b' (Tier 8b). This routing is **scope-aware**: once a request has landed here (via one of those three triggers), read the scope manifest at the top of Step 5b' and run the arms it selects:+ - **default** (bare "cost report" / "distributed cost" / "mock usage") → the free, deterministic mock arms only (`--dry-run --mock-usage`); no live spend.- **explicit spend opt-in** — the canonical token `full` (aliases `thorough`, `every`, `with-spend`) qualifying the cost request (e.g. "cost reporting full", "cost report — every arm") → **every** cost arm, real spend authorized: mock primary + cross-child + CSV cross-check + the `--no-costs` negative gate + live LLM + live img-gen + live extract. Run them all and report PASS/FAIL per arm. Do **not** treat bare "live" or "all" as a spend opt-in — they're too easily incidental ("live" also collides with the default mock arm, which already runs in LIVE mode); if that's the only signal, confirm with the user before spending real money.- "queue options" / "runtime profile" / "routing" → `mode-2-setup.md`, then `routing-battery.md` and/or `queue-options-battery.md`.- Anything image-related → `mode-2-setup.md` + `mode-2-tiers.md` Tiers 4–5, run **live** — dry-run cannot surface payload-size bugs.diff --git a/.claude/skills/temporal-e2e-validate/references/mode-2-tiers.md b/.claude/skills/temporal-e2e-validate/references/mode-2-tiers.mdindex 9fbc71832..aef712df1 100644--- a/.claude/skills/temporal-e2e-validate/references/mode-2-tiers.md+++ b/.claude/skills/temporal-e2e-validate/references/mode-2-tiers.md@@ -7,10 +7,10 @@## Contents-- **Step 3** — Sequential tests: Tier 1 (sequence), Tier 2 (hydration), Tier 2b (cross-process registry), Tier 2c (validate sweep stays in-process), Tier 3 (parallel), Tier 4 (image generation), Tier 5 (image flow)+- **Step 3** — Sequential tests: Tier 1 (sequence), Tier 2 (hydration), Tier 2b (cross-process registry), Tier 2c (validate sweep stays in-process), Tier 2d (dry-run+validate as one in-memory activity), Tier 3 (parallel), Tier 4 (image generation), Tier 5 (image flow)- **Step 4** — Verify graph output- **Step 5** — Concurrent isolation tests (concept / pipe / multi-concept)-- **Step 5b** — Tier 8: cross-worker usage emission; Tier 8b: cross-worker cost report assembly (`--mock-inference`, free)+- **Step 5b** — Tier 8: cross-worker usage emission; Tier 8b: cross-worker cost report assembly (`--dry-run --mock-usage`, free)- **Step 5c** — Tier 9: object generation cross-process- **Step 5d** — Tier 11: `make_extract_pages` two-activity cross-process- **Step 5e** — Tier 12: deeply-nested controller stack (CV batch screening)@@ -159,6 +159,126 @@ After this completes, tell the user:called). This Mode-2 scenario is the deployment-faithful demonstration across the real API↔workerprocess boundary.+**Tier 2d — Dry-run + validation runs as ONE in-process, in-memory activity.**++Sibling to Tier 2c: where 2c proves the *direct* `/validate` sweep doesn't leak to Temporal, 2d+proves the *Temporal-dispatched* path runs the whole sweep **+** graph dry-run inside a single+activity (`act_dry_validate`, dispatched via the one-step wrapper workflow `wf_dry_validate`), in+memory, returning `{status map, GraphSpec}` in one round-trip. The submitter script below is the+same dispatch shape the Temporal-enabled API `/validate` uses.++```bash+# Server + split workers up (Mode 2). Dispatch the wrapper-workflow→activity over the parallel+# controller bundle (interesting graph: sequence → parallel fan-out → 2 branches → summary).+timeout 120 .venv/bin/python .claude/skills/temporal-e2e-validate/scripts/submit_dry_validate.py \+ --bundle tests/integration/pipelex/temporal/library_crate/temporal_parallel.mthds \+ --pipe temporal_parallel_test.temporal_parallel_sequence > /tmp/tier2d-validate.log 2>&1; echo "EXIT=$?"+tail -20 /tmp/tier2d-validate.log+```++GREEN: `EXIT=0` · the STATUS MAP lists every pipe in the bundle as `SUCCESS` · `GRAPH:` line shows a+non-empty GraphSpec (nodes for the whole controller topology) and the JSON landed at+`/tmp/tier2d-graph-spec.json`.++**Strong check (the point):** during the run the worker ran the wrapper workflow + exactly one+`act_dry_validate` and **nothing else** — NO child `WfPipeRouter`/`WfPipeRun`, NO `act_llm_gen_*`,+NO `act_assemble_tracing`/`act_flush_trace_events`. Capture both worker sessions and grep — expect+only `wf_dry_validate` / `act_dry_validate` for this run (mirrors Tier 2c's worker-idle check, but+here the one activity is expected; what must be absent is everything *nested*):++```bash+tmux capture-pane -t temporal-worker-router -p -S -200 | grep -iE "WfPipeRouter|WfPipeRun|act_llm_gen|act_assemble_tracing|act_flush_trace_events" # expect: nothing new+tmux capture-pane -t temporal-worker-runner -p -S -200 | grep -iE "WfPipeRouter|WfPipeRun|act_llm_gen|act_assemble_tracing|act_flush_trace_events" # expect: nothing new+```++**In-memory tracing:** no new NDJSON partition appears under `.pipelex/traces/` for the activity's+internal graph dry-run, and no DynamoDB write — the `GraphSpec` rode back on the activity result,+assembled from the in-memory log. **No usage/cost:** no cost table, no `usage_report` events.++**Best-effort graph sub-case:** dispatch a bundle with no `main_pipe` and no `--pipe` (so the graph+arm has nothing to target):++```bash+timeout 120 .venv/bin/python .claude/skills/temporal-e2e-validate/scripts/submit_dry_validate.py \+ --bundle tests/integration/pipelex/temporal/library_crate/temporal_batch.mthds \+ --no-pipe > /tmp/tier2d-nograph.log 2>&1; echo "EXIT=$?"+tail -5 /tmp/tier2d-nograph.log+```++GREEN: `EXIT=0`, full STATUS MAP, and `GRAPH: None (best-effort — validation still succeeded)`.++**Concurrency:** launch two submitter invocations in parallel (e.g. `temporal_parallel.mthds` and+`temporal_batch.mthds --no-pipe` with `--graph-out` pointing at distinct files) and confirm both+exit 0 with distinct `graph_id`s / status maps — no shared or merged trace events.++RED (prove it bites) — the fix is committed, so neutralize it in the working tree. Either arm:++- **Drop the content-generator scope:** in `pipelex/pipe_run/dry_run_pipeline.py`+ (`dry_run_pipe_in_process`), remove `scoped_content_generator(content_generator)` from the+ `with` line. The DRY mock lives at the cogt leaf, so the leaf resolves the hub+ `ContentGeneratorInWorkflow` and dies with `_NotInWorkflowEventLoopError` / the strong check shows+ dispatch. (The CI-cheap Mode-1 companion is+ `test_dry_run_graph_in_process.py::test_leaf_level_mock_stays_in_process`.)+- **Drop the shared event log:** in the same function, remove `scoped_event_log(event_log)` from+ the `with` line — the two-instance regression: emit and assemble no longer share the instance, the+ assembly finds zero events, and `dry_run_pipe_in_process` raises `In-process dry-run of pipe '...'+ did not produce a graph spec`. In the activity that PipelexError degrades per the parity+ best-effort catch, so the submitter still exits 0 — the RED signal is `GRAPH: None` where the+ GREEN run produced a real graph (nodes/edges + the JSON file). The loud, CI-cheap guard for the+ same regression is the Mode-1 companion `test_dry_run_graph_in_process.py` (it calls+ `dry_run_pipe_in_process` directly and asserts a non-empty graph).++**Restore immediately:** `git checkout -- pipelex/pipe_run/dry_run_pipeline.py`.++**API arm (after Phase 4 — the real `/validate` route in `../pipelex-api`).** With the server ++split workers up, exercise the actual route code performing the real dispatch (the sibling repo's+venv must have this pipelex installed, e.g. `cd ../pipelex-api && uv pip install -e ../<this-worktree>`):++```bash+# From ../pipelex-api: boot PYTEST-mode with temporal_enabled=True, point the dispatch at the+# e2e queue, and POST a real bundle through TestClient (full route + handler chain, real worker).+cd ../pipelex-api && timeout 120 .venv/bin/python - <<'PY'+import os+os.environ.setdefault("COMPLETION_CALLBACK_SECRET", "tier2d-placeholder")+from fastapi import FastAPI+from fastapi.testclient import TestClient+from pipelex.config import get_config+from pipelex.pipelex import Pipelex+from pipelex.system.runtime import IntegrationMode++BUNDLE = open("../pipelex/tests/integration/pipelex/temporal/library_crate/native_text_sequence.mthds").read() # or any main_pipe bundle+Pipelex.make(IntegrationMode.PYTEST, temporal_enabled=True)+get_config().temporal.worker_config.default_task_queue = "temporal_task_queue"+from api.exception_handlers import register_exception_handlers+from api.routes import router as api_router+app = FastAPI(); app.include_router(api_router, prefix="/api/v1"); register_exception_handlers(app)+resp = TestClient(app).post("/api/v1/validate", json={"mthds_contents": [BUNDLE]})+print(resp.status_code, resp.json().get("graph_spec") is not None)+Pipelex.teardown_if_needed()+PY+```++GREEN: `200 True` (one round-trip returned the envelope with a worker-assembled `graph_spec`), the+worker shows only `wf_dry_validate` + one `act_dry_validate` (same strong check as above), and a+strict-mode signature bundle POSTed the same way returns **422** with+`error_type=ValidateBundleError`, `error_domain=input` — identical to the direct path. The failed+workflow must appear exactly ONCE in `temporal workflow list` (the dispatch pins+`RetryPolicy(maximum_attempts=1)` on the wrapper workflow — a deterministic validation failure+must not re-run). The bundle's graph is best-effort: a bundle with no `main_pipe` 422s on the+API-side precondition; the worker-side `graph_spec=None` arm is covered by the route's unit tests+(`tests/unit/test_validate_temporal_dispatch.py`).++After this completes, tell the user:++- PASS (GREEN exits 0 · status map all SUCCESS · non-empty GraphSpec · worker shows ONLY the+ wrapper workflow + one `act_dry_validate` · no NDJSON/DDB write · best-effort sub-case returns+ `GRAPH: None` with exit 0) / FAIL.+- The cheaper CI-automated companion is the Mode-1 pytest+ `tests/integration/pipelex/temporal/test_dry_validate_activity_in_memory.py` (real worker against+ the in-process server; zero-nested-dispatch asserted on the Temporal history; in-memory tracing+ with `make_event_log` forbidden; best-effort graph; structured `ErrorReport` on validation+ failure; concurrent isolation).+**Tier 3 — Do parallel branches execute as separate child workflows?**This sends a PipeParallel controller that fans out into two branches (tone analysis and@@ -402,18 +522,17 @@ stamped with a runner-process `writer_id` (`act_{pid}_{uuid8}`). It's the e2ecounterpart to the Phase 4 integration test`tests/integration/pipelex/temporal/tracing/test_split_worker_usage.py`.-**Important — dry-run does NOT exercise this path.** In dry-run mode, `PipeLLM`-(and friends) instantiate `ContentGeneratorDry()` directly inside the workflow-body — on the router process — and never dispatch `act_llm_gen_text` to the-runner. So a vanilla `pipelex run bundle --temporal --dry-run` against-router+runner workers will emit ALL `usage_report` events with-`writer_id="primary"`, never `act_*`.--**Cheap deterministic CLI way: `--mock-inference` (Tier 8b below).** To observe-runner-side `act_*` writer files from the CLI *without LLM spend*, use-`--mock-inference`: a LIVE run (operators dispatch `act_llm_gen_text` to the-runner exactly as a real run does) whose AI calls are faked at the inference leaf-with reportable synthetic usage. This is the deterministic, free path — prefer it+**Important — the default dry run does NOT exercise the *reportable*-usage path.**+`--temporal --dry-run` DOES dispatch `act_llm_gen_text`+to the runner (the leaf mocks inside the activity — that's Tier 17's subject),+but its default synthetic usage is **zero-token and suppressed by design**, so no+rendered cost report and no reportable usage events come back from it.++**Cheap deterministic CLI way: `--dry-run --mock-usage` (Tier 8b below).** To observe+runner-side `act_*` writer files from the CLI *without LLM spend*, add the hidden+`--mock-usage` flag to a dry run: the dry run dispatches `act_llm_gen_text` to the+runner exactly as a real run does, and the flag flips the dry leaf's reporting to+reportable non-zero synthetic usage. This is the deterministic, free path — prefer itover live mode for writer-id observation, and see **Tier 8b** for the fullcross-worker cost-report assertion. The two other paths below remain valid: thePhase 4 integration test (which substitutes the inference activity with a wrapper@@ -464,7 +583,7 @@ own `task_queue`) so that `act_llm_gen_text` resolves to a queue the runnerlistens on, and confirm both workers are running with the latest code(restart them if they predate the Phase 2 runner-side fallback commit).-### Step 5b': Tier 8b — Cross-worker cost report assembly (`--mock-inference`, free + deterministic)+### Step 5b': Tier 8b — Cross-worker cost report assembly (`--dry-run --mock-usage`, free + deterministic)#### Scope manifest — which arms to run@@ -474,15 +593,15 @@ Run each arm in listed order. After each, surface the asserted token totals (whe| # | Arm | Scope | Spend | Expected assertion ||---|---|---|---|---|-| A | **Mock primary** — `--mock-inference` `native_text_sequence` | default + full | free | 2 events / 200 input / 100 output, `--expected-model-type llm --require-fallback` |-| B | **Cross-child fan-out** — `--mock-inference` `temporal_parallel` | default + full | free | 3 events / 300 input / 150 output, `--expected-model-type llm --require-fallback` (the CLI assert checks the summed total, **not** workflow-span; the cross-child guarantee itself is enforced by the pytest counterpart `test_split_worker_cross_child_usage.py`) |+| A | **Mock primary** — `--dry-run --mock-usage` `native_text_sequence` | default + full | free | 2 events / 200 input / 100 output, `--expected-model-type llm --require-fallback` |+| B | **Cross-child fan-out** — `--dry-run --mock-usage` `temporal_parallel` | default + full | free | 3 events / 300 input / 150 output, `--expected-model-type llm --require-fallback` (the CLI assert checks the summed total, **not** workflow-span; the cross-child guarantee itself is enforced by the pytest counterpart `test_split_worker_cross_child_usage.py`) || C | **CSV un-truncated cross-check** — flip `reporting_config.is_generate_cost_report_file_enabled=true`, rerun arm A, confirm CSV totals == NDJSON totals, restore to `false` | full | free | `csv tokens` line matches NDJSON totals || D | **`--no-costs` negative gate** | full | free | no cost table, no `usage_report` events, `reactflow.html` still assembles |-| E | **Live LLM arm** — drop `--mock-inference` on `native_text_sequence` | full | **real** | non-zero total, real `model_names` (not `mock_inference`), `--require-fallback --require-nonzero` |+| E | **Live LLM arm** — drop `--dry-run --mock-usage` on `native_text_sequence` | full | **real** | non-zero total, real `model_names` (not `mock_usage`), `--require-fallback --require-nonzero` || F | **Live img-gen arm** — run a Tier 4/5 image bundle live with `--graph --costs` | full | **real** | `--expected-model-type img_gen --require-fallback --require-nonzero` || G | **Live extract arm** — run the extract bundle `pdf_extract_page_views.mthds` (`--pipe pdf_extract_with_page_views`) live against the plain split workers with `--graph --costs` | full | **real** | `--expected-model-type extract --require-fallback --require-nonzero` |-Arms F and G validate non-LLM usage, which `--mock-inference` cannot reach (the mock leaves raise `MockInferenceUnsupportedError`), so they are **live-only** — they are the sole way to prove img-gen / extract token usage crosses the runner fallback and aggregates into the cost report. Each arm's full command + assertion is detailed in the sub-sections below; arms A/C map to "Primary check", B to "Cross-child aggregation", D to "Negative check", E to "Live arm", F/G to "Non-LLM cross-worker cost". Arms F and G run their bundles on the **same plain router+runner split workers** as the other arms — they do **not** need the routing battery's multi-queue (`q_extract` / `q_image_gen`) setup. On plain split workers the img-gen / extract activity runs on the runner and emits via the `act_*` fallback, which is exactly what `--require-fallback` checks.+Arms F and G validate non-LLM usage, which the mock arms cannot reach (non-LLM dry leaves emit no synthetic usage), so they are **live-only** — they are the sole way to prove img-gen / extract token usage crosses the runner fallback and aggregates into the cost report. Each arm's full command + assertion is detailed in the sub-sections below; arms A/C map to "Primary check", B to "Cross-child aggregation", D to "Negative check", E to "Live arm", F/G to "Non-LLM cross-worker cost". Arms F and G run their bundles on the **same plain router+runner split workers** as the other arms — they do **not** need the routing battery's multi-queue (`q_extract` / `q_image_gen`) setup. On plain split workers the img-gen / extract activity runs on the runner and emits via the `act_*` fallback, which is exactly what `--require-fallback` checks.Tier 8 (Step 5b above — the writer-id landing pytest) is the precursor to all of these and should pass first under any scope.@@ -494,20 +613,19 @@ Tier 8b proves the next link end-to-end: those cross-worker usage events`assemble_usage=True`) and the submitter renders a **single** end-of-run costreport covering usage from all workers — with **no LLM spend**.-**Why `--mock-inference` and not `--dry-run`.** Dry-run instantiates-`ContentGeneratorDry` inside the workflow body *on the router* and never-dispatches `act_llm_gen_text` to the runner (Tier 8 note), and its usage is-zero-token → the cost report is *suppressed*. `--mock-inference` keeps-`run_mode=LIVE` so operators dispatch the real `act_llm_gen_text` to the runner-exactly like a paid run; only the inference *leaf* is faked, emitting reportable-non-zero synthetic usage (model `mock_inference`). So the runner emits `act_*`-usage events AND the report is non-suppressed — the one thing dry-run can't-validate cheaply.+**Why the `--mock-usage` sub-flag.** A dry run+dispatches `act_llm_gen_text` to the runner (the leaf mocks inside the+activity), but its default synthetic usage is zero-token → the cost report is+*suppressed* by design. The hidden `--mock-usage` flag (requires `--dry-run`)+keeps that exact dispatch and flips the dry leaf's reporting to reportable+non-zero synthetic usage (model `mock_usage`). So the runner emits `act_*`+usage events AND the report is non-suppressed — the one thing the default dry+run can't validate cheaply.**Requires split router+runner workers** (Step 2 in `mode-2-setup.md`). The pointis that the usage events originate on the *runner* process and must aggregate-into the report assembled for the *router*/submitter. `--mock-inference` is on the-main `run` subcommands (pipe/method/bundle) and is mutually exclusive with+into the report assembled for the *router*/submitter. `--mock-usage` is a hidden+option on the main `run` subcommands (pipe/method/bundle) and requires`--dry-run`.**Primary check — both artifacts assemble from one event read (D3):**@@ -516,15 +634,15 @@ main `run` subcommands (pipe/method/bundle) and is mutually exclusive withtimeout 180 .venv/bin/pipelex run bundle \tests/integration/pipelex/temporal/library_crate/native_text_sequence.mthds \--pipe native_text_sequence \- --temporal --mock-inference --no-logo --graph --costs 2>&1 | tail -30+ --temporal --dry-run --mock-usage --mock-inputs --no-logo --graph --costs 2>&1 | tail -30echo "EXIT=$?"```Expect, at the end of the submitter output:-- A Rich **cost table** with model `mock_inference`, non-zero input/output token+- A Rich **cost table** with model `mock_usage`, non-zero input/output tokencounts, and a run total (the synthetic per-call counts are- `MOCK_INFERENCE_NB_TOKENS_BY_CATEGORY = {INPUT: 100, OUTPUT: 50}`, so a+ `MOCK_USAGE_NB_TOKENS_BY_CATEGORY = {INPUT: 100, OUTPUT: 50}`, so a2-LLM-step sequence totals 200 input / 100 output).- A `reactflow.html` for the run (graph assembles from the *same* event read).- Exit 0.@@ -533,8 +651,8 @@ Then **assert the numbers** (don't eyeball — the terminal table truncates widecolumns to `0 … …`). The `assert_cross_worker_cost.py` helper sums input/outputtokens straight from the NDJSON usage events, counts them, checks a runner `act_*`writer engaged, and (if a CSV report exists) cross-checks the un-truncated CSV-totals. For mock-inference the per-call counts are fixed-(`MOCK_INFERENCE_NB_TOKENS_BY_CATEGORY = {INPUT: 100, OUTPUT: 50}`), so a+totals. For mock-usage the per-call counts are fixed+(`MOCK_USAGE_NB_TOKENS_BY_CATEGORY = {INPUT: 100, OUTPUT: 50}`), so a2-LLM-step sequence must total exactly 2 events / 200 input / 100 output:```bash@@ -570,7 +688,7 @@ from multiple child workflows aggregates into a single submitter report:timeout 180 .venv/bin/pipelex run bundle \tests/integration/pipelex/temporal/library_crate/temporal_parallel.mthds \--pipe temporal_parallel_sequence \- --temporal --mock-inference --no-logo --graph --costs 2>&1 | tail -30+ --temporal --dry-run --mock-usage --mock-inputs --no-logo --graph --costs 2>&1 | tail -30echo "EXIT=$?"```@@ -597,7 +715,7 @@ The pytest counterpart for this cross-child aggregation (no spend) istimeout 180 .venv/bin/pipelex run bundle \tests/integration/pipelex/temporal/library_crate/native_text_sequence.mthds \--pipe native_text_sequence \- --temporal --mock-inference --no-logo --graph --no-costs 2>&1 | tail -30+ --temporal --dry-run --mock-usage --mock-inputs --no-logo --graph --no-costs 2>&1 | tail -30echo "EXIT=$?"RUN_ID=$(ls -t .pipelex/traces/ | head -1)grep -l '"event_kind":"usage_report"' .pipelex/traces/$RUN_ID/*.ndjson || echo "No usage_report events (expected with --no-costs)"@@ -610,7 +728,7 @@ still assembles — proving `--costs` gates only the cost channel, independent o`--graph`.**Live arm (real spend, opt-in)** — mirrors Tier 8's live block for the-real-payload case. Drop `--mock-inference` so the real `act_llm_gen_text` runs on+real-payload case. Drop `--dry-run --mock-usage` so the real `act_llm_gen_text` runs onthe runner and bills real tokens:```bash@@ -632,15 +750,15 @@ RUN_ID=$(ls -t .pipelex/traces/ | head -1)```Expect `RESULT: PASS` with non-zero `total tokens` and a real `model_names` entry-(not `mock_inference`). The pytest counterpart for this real-inference cross-worker+(not `mock_usage`). The pytest counterpart for this real-inference cross-workerpath (gated, opt-in spend) is`tests/integration/pipelex/temporal/tracing/test_split_worker_real_inference_cost.py`(marked `inference`/`llm`; the no-spend counterpart is-`tests/integration/pipelex/temporal/tracing/test_mock_inference_temporal.py`).+`tests/integration/pipelex/temporal/tracing/test_mock_usage_temporal.py`).-**Non-LLM cross-worker cost (img-gen / extract — live only).** `--mock-inference`-cannot cover image generation or extraction: their mock leaves raise-`MockInferenceUnsupportedError`, so non-LLM usage only crosses the runner boundary+**Non-LLM cross-worker cost (img-gen / extract — live only).** `--mock-usage`+cannot cover image generation or extraction: non-LLM dry leaves emit no synthetic+usage, so non-LLM token usage only crosses the runner boundaryon a real run. The img-gen / extract live tiers (Tiers 4 / 5 / 10c) alreadyexercise those activities cross-process but never check the cost numbers. Afterrunning one of those live tiers with `--graph --costs` against split workers,@@ -1007,6 +1125,103 @@ If you continue to Step 6 or later tiers, restart the shared worker(s) per Step---+### Step 5g: Tier 17 — DRY honors the Temporal backend (leaf mock inside the activity)++Contract under test: `run_mode` is orthogonal to the backend — a `--temporal --dry-run` run goes+through the REAL distribution machinery (`WfPipeRouter` → child workflows → `act_*_gen_*`+activities on the worker) and the cogt **leaf inside each activity** mocks instead of calling a+provider (`run_mode=DRY` rides `CogtRunParams` on every assignment across the wire). A regression+to the old behavior — dry-run short-circuiting at the operator/router and dispatching nothing —+must turn this tier red. Purpose: test dispatch, scheduling, serialization, routing, and+cross-worker propagation at zero AI cost.++The Mode-1 (pytest) companion is+`tests/integration/pipelex/temporal/tracing/test_dry_run_dispatches_and_mocks.py` — it asserts the+dispatch via workflow history (`ActivityTaskScheduled` across the parent + child workflows), the+`DRY RUN:` leaf output, and the zero-token suppressed usage over the in-process server.++**Setup.** Split workers up (router + runner per Step 2 in `mode-2-setup.md`), Temporal-enabled.+The dry leaves never touch a provider or storage, so this tier needs NO API keys — the "dry works+with no inference configured" invariant must hold even though activities dispatch.++**GREEN arm — LLM bundle:**++```bash+timeout 600 .venv/bin/pipelex run bundle \+ tests/integration/pipelex/temporal/library_crate/native_text_sequence.mthds \+ --pipe native_text_sequence \+ --temporal --dry-run --mock-inputs --no-logo --graph > /tmp/tier17.log 2>&1+echo "EXIT=$?"+tail -15 /tmp/tier17.log+# The runner DID execute the LLM activity (pre-Part-B: nothing inference-related ran here):+tmux capture-pane -t temporal-worker-runner -p -S -300 | grep -c "act_llm_gen_text"+# The leaf mocked INSIDE the activity — no provider call, no storage write on the runner:+tmux capture-pane -t temporal-worker-runner -p -S -300 | grep -iE "openai|anthropic|provider|storing|uploaded" | head -5+```++Report to the user:++- **PASS** = exit 0 · the runner pane shows `act_llm_gen_text` executions (count ≥ 1) · no+ provider/storage lines on the runner · the output `main_stuff` text starts with `DRY RUN:` ·+ **no cost table rendered** (zero-token dry usage is suppressed by design — contrast Tier 8b) ·+ `reactflow.html` generated (cross-worker graph tracing assembled, LibraryCrate propagated).+- **FAIL** = non-zero exit; or the runner pane shows NO `act_llm_gen_text` (the dry run didn't+ dispatch — the pre-Part-B shortcut is back); or any provider/storage activity on the runner.++**GREEN arm — non-LLM leaves** (every leaf mocks under DRY — img/extract DRY branches sit at the+`*_and_store` layer, so also assert no storage writes):++```bash+# Extract (two activities: act_extract_gen_extract_pages + act_render_page_views):+timeout 600 .venv/bin/pipelex run bundle \+ tests/integration/pipelex/temporal/library_crate/pdf_extract_page_views.mthds \+ --pipe pdf_extract_with_page_views \+ --temporal --dry-run --mock-inputs --no-logo > /tmp/tier17-extract.log 2>&1+echo "EXIT=$?"+tmux capture-pane -t temporal-worker-runner -p -S -200 | grep -cE "act_extract_gen_extract_pages|act_render_page_views"++# Image generation:+timeout 600 .venv/bin/pipelex run bundle \+ tests/integration/pipelex/pipes/pipelines/test_image_out_in.mthds \+ --pipe generate_image \+ --temporal --dry-run --mock-inputs --no-logo > /tmp/tier17-img.log 2>&1+echo "EXIT=$?"+tmux capture-pane -t temporal-worker-runner -p -S -200 | grep -c "act_img_gen_images"++# Web search:+timeout 600 .venv/bin/pipelex run bundle \+ tests/integration/pipelex/temporal/library_crate/native_search.mthds \+ --pipe native_search \+ --temporal --dry-run --mock-inputs --no-logo > /tmp/tier17-search.log 2>&1+echo "EXIT=$?"+tmux capture-pane -t temporal-worker-runner -p -S -200 | grep -c "act_search_gen_sourced_answer"+```++- **PASS** per arm = exit 0 + the named activity count ≥ 1 on the runner + no storage writes+ (no bucket/object upload lines in the runner pane) + zero provider calls.++**No-keys GREEN arm — prove dry needs no inference configured.** The dispatched-and-mocked+behavior must hold on a worker with NO usable credentials. Tamper `.env` with invalid keys and+restart the runner using the same backup/restore method as Tiers 13–16 (Step 5f), then re-run the+GREEN LLM command:++- **PASS** = exit 0 with the keyless runner — the dry leaves never resolve a provider, so invalid+ credentials are never exercised. Keep this runner for the RED arm below.++**RED arm — prove the leaf branch bites.** There is no router-level mock to revert to,+so the RED mutation is at the leaf: temporarily disable the DRY branch in `llm_gen_text`+(`pipelex/cogt/content_generation/llm_generate.py` — comment out the+`if llm_assignment.cogt_run_params.run_mode.is_dry:` short-circuit), restart the **runner** worker+(it must boot WITHOUT inference credentials — same `.env` tamper method as Tiers 13–16 if your+`.env` has real keys), and re-run the GREEN LLM command:++- **RED-PASS** = the dispatched `act_llm_gen_text` attempts a real call and **fails loud** on+ missing/invalid keys — the dispatched + mocked-inside + no-spend + no-keys quartet flips.+- Restore the leaf branch (`git checkout -- pipelex/cogt/content_generation/llm_generate.py`),+ restore `.env` if tampered, restart the workers, and re-run GREEN to confirm recovery.++---+### Step 6: StoragePayloadCodec tests — does the codec work end-to-end?These tests validate Phase 5: large payloads are transparently offloaded to external@@ -1129,13 +1344,14 @@ ls results/*/reactflow.html| Tier 2: Hydration | Worker handles dynamic concepts it has never seen | PASS/FAIL | path | — || Tier 2b: Cross-process registry | `act_deliver` decodes hydrated `pipe_output` on runner (forced via `delivery_assignment`) | PASS/FAIL | — | — || Tier 2c: Validate sweep stays in-process | `validate bundle --temporal` over a standalone `PipeBatch` exits 0 **and** the worker received no `WfPipeRouter` dispatch (the sweep never leaks to Temporal) | PASS/FAIL | — | — |+| Tier 2d: Dry-run+validate as one in-memory activity | the Temporal-dispatched /validate runs the whole sweep + graph dry-run inside ONE in-process activity (zero nested dispatch), traces the graph in memory (no NDJSON/DDB), returns {status, GraphSpec}; best-effort graph → None on failure | PASS/FAIL | path | — || Tier 3: Parallel | Branches execute as concurrent child workflows | PASS/FAIL | path | — || Tier 4: ImgGen | Image generation pipeline works through Temporal | PASS/FAIL | path | yes/no || Tier 5: Image flow | Generated image flows as input to next pipe step | PASS/FAIL | path | yes/no || Tier 6: Codec transparency | Existing pipelines work unchanged with codec enabled | PASS/FAIL | path | — || Tier 7: Large payload | Multi-step pipeline with codec stress test | PASS/FAIL | path | — || Tier 8: Cross-worker usage | Runner-side `UsageReportEvent` lands in same NDJSON dir with `act_*` writer_id (live mode or integration test) | PASS/FAIL | — | — |-| Tier 8b: Cross-worker cost report | `--mock-inference` (free): runner-side usage assembles onto `PipeOutput.tokens_usages` and the submitter renders a single non-suppressed cost report (model `mock_inference`); `--no-costs` renders none while `--graph` still assembles. **Numeric assertion** via `scripts/assert_cross_worker_cost.py` (sums NDJSON usage tokens, checks count + `act_*` fallback + optional CSV cross-check) — mock: 2 events/200 input/100 output; parallel: 3/300/150; live arms `--require-nonzero`; img-gen/extract `--expected-model-type` (live only) | PASS/FAIL | path | — |+| Tier 8b: Cross-worker cost report | `--dry-run --mock-usage` (free): runner-side usage assembles onto `PipeOutput.tokens_usages` and the submitter renders a single non-suppressed cost report (model `mock_usage`); `--no-costs` renders none while `--graph` still assembles. **Numeric assertion** via `scripts/assert_cross_worker_cost.py` (sums NDJSON usage tokens, checks count + `act_*` fallback + optional CSV cross-check) — mock: 2 events/200 input/100 output; parallel: 3/300/150; live arms `--require-nonzero`; img-gen/extract `--expected-model-type` (live only) | PASS/FAIL | path | — || Tier 9: Object gen cross-process | `act_llm_gen_object` / `act_llm_gen_object_list` survive the JSON round-trip with nested fields intact | PASS/FAIL | path | — || Tier 10a: Multi-activity routing | `activity_queues.default` routes both `act_llm_gen_text` and `act_img_gen_images` to their dedicated worker pools; default runner sees 0 hits for either | PASS/FAIL/SKIPPED | — | — || Tier 10b: Per-handle routing | `activity_queues.by_handle` overrides the activity default per model handle — two distinct handles in one workflow land on two distinct workers | PASS/FAIL/SKIPPED | path | — |@@ -1146,6 +1362,7 @@ ls results/*/reactflow.html| Tier 14: Extract error propagation | Same for a non-LLM (`act_extract_gen_extract_pages`) activity failure | PASS/FAIL | — | — || Tier 15: Image-gen error propagation | Same for the image-generation (`act_img_gen_images`) activity failure | PASS/FAIL | — | — || Tier 16: Batch child-workflow error propagation | A per-branch failure inside a `PipeBatch` fan-out child workflow crosses the fan-in carrying its `ErrorReport`, not a generic batch wrapper | PASS/FAIL | — | — |+| Tier 17: DRY honors the backend | a `--temporal --dry-run` run dispatches `act_llm_gen_*` (+ extract/img-gen/search) to the worker and mocks INSIDE the activity — no real IO, no API keys needed, zero-token suppressed usage, cross-worker graph still assembles | PASS/FAIL | path | — || Scenario A: v2 multi-class routing | Specialized scopes + per-class runners cover LLM/img-gen/extract without cross-class leakage | PASS/FAIL/SKIPPED | — | — || Scenario B: per-queue timeout | `queue_options[X].start_to_close_timeout` overrides the worker_config baseline and flows into `ActivityTaskScheduled.start_to_close_timeout` | PASS/FAIL/SKIPPED | — | — || Scenario C: per-handle override | `handle_options[<handle>].start_to_close_timeout` wins over per-queue value for that one handle | PASS/FAIL/SKIPPED | — | — |diff --git a/.claude/skills/temporal-e2e-validate/references/routing-battery.md b/.claude/skills/temporal-e2e-validate/references/routing-battery.mdindex 7936fcbd1..94ac3f842 100644--- a/.claude/skills/temporal-e2e-validate/references/routing-battery.md+++ b/.claude/skills/temporal-e2e-validate/references/routing-battery.md@@ -15,11 +15,10 @@ proves the routing feature works as advertised when operators actually configureit: each activity (and, in Tier 10b, each model handle) lands on its dedicatedworker pool, never on the fallback runner.-Step 8 is **live-only**. The routing decision happens in the workflow regardless-of dry/live, but the only way to prove a routed activity wasn't picked up by the-wrong worker is to actually dispatch it and watch where it executes. Dry-run-short-circuits LLM calls inside the workflow process (`ContentGeneratorDry`),-so no `act_*` ever gets scheduled and the routing assertion is meaningless.+Step 8 is **live-only** for its spend-bearing assertions. Since Part B, dry-run+also dispatches `act_*` activities (the cogt leaf mocks inside them), so routing+CAN be observed dry — but this battery predates that and its arms assert on real+provider effects (model handles, spend), so run it live as written.**Step 8.0 — Preflight + setup**diff --git a/.claude/skills/temporal-e2e-validate/scripts/submit_dry_validate.py b/.claude/skills/temporal-e2e-validate/scripts/submit_dry_validate.pynew file mode 100644index 000000000..82ae00e41--- /dev/null+++ b/.claude/skills/temporal-e2e-validate/scripts/submit_dry_validate.py@@ -0,0 +1,93 @@+# ruff: noqa: INP001+r"""Tier-2d submitter: dispatch the dry-run+validation job as ONE Temporal activity.++Dispatches the one-step wrapper workflow ``WfDryValidate`` (which runs the single+``act_dry_validate`` activity fully in-process on a worker) and awaits the+``{status map, GraphSpec}`` result in one round-trip — the Temporal-enabled ``/validate``+dispatch shape. Prints the per-pipe status map and writes the GraphSpec JSON when produced.++Usage (with the Temporal server + split workers up — see mode-2-setup.md):++ .venv/bin/python .claude/skills/temporal-e2e-validate/scripts/submit_dry_validate.py \\+ --bundle tests/integration/pipelex/temporal/library_crate/temporal_parallel.mthds \\+ --pipe temporal_parallel_test.temporal_parallel_sequence++Exit codes: 0 = validation succeeded (graph may still be None — best-effort); 1 = validation+failed (the structured ErrorReport is printed).+"""++import argparse+import asyncio+import sys+from pathlib import Path++from pipelex.cli.cli_factory import make_pipelex_for_cli+from pipelex.cli.error_handlers import ErrorContext+from pipelex.pipelex import Pipelex+from pipelex.temporal.exceptions import WorkflowExecutionError+from pipelex.temporal.tprl_pipe.act_dry_validate import DryValidateArg+from pipelex.temporal.tprl_pipe.dry_validate_dispatch import dispatch_dry_validate++DEFAULT_BUNDLE = "tests/integration/pipelex/temporal/library_crate/temporal_parallel.mthds"+DEFAULT_PIPE = "temporal_parallel_test.temporal_parallel_sequence"+DEFAULT_GRAPH_OUT = "/tmp/tier2d-graph-spec.json" # noqa: S108+++async def submit(bundle: str, pipe_code: str | None, allow_signatures: bool, graph_out: str) -> int:+ bundle_content = Path(bundle).expanduser().read_text(encoding="utf-8")+ arg = DryValidateArg(+ mthds_contents=[bundle_content],+ allow_signatures=allow_signatures,+ pipe_code=pipe_code,+ )+ print(f"Dispatching wf_dry_validate for bundle='{bundle}' pipe_code={pipe_code!r} allow_signatures={allow_signatures}")+ try:+ result = await dispatch_dry_validate(arg)+ except WorkflowExecutionError as exc:+ print("VALIDATION FAILED — structured ErrorReport recovered from the workflow failure:")+ print(f" error_type: {exc.error_report.error_type if exc.error_report else 'n/a'}")+ print(f" error_domain: {exc.error_report.error_domain if exc.error_report else 'n/a'}")+ print(f" message: {exc.message}")+ return 1++ print(f"STATUS MAP ({len(result.dry_run_outputs)} pipes):")+ for pipe_ref, output in sorted(result.dry_run_outputs.items()):+ suffix = f" — {output.error_message}" if output.error_message else ""+ print(f" {output.status:<8} {pipe_ref}{suffix}")++ if result.graph_spec is None:+ print("GRAPH: None (best-effort — validation still succeeded)")+ else:+ print(f"GRAPH: graph_id={result.graph_spec.graph_id} nodes={len(result.graph_spec.nodes)} edges={len(result.graph_spec.edges)}")+ graph_path = Path(graph_out).expanduser()+ try:+ graph_path.parent.mkdir(parents=True, exist_ok=True)+ graph_path.write_text(result.graph_spec.model_dump_json(indent=2), encoding="utf-8")+ except OSError as exc:+ # The graph file is a best-effort artifact: an IO failure must not be reported as a validation failure (exit 1)+ print(f"GRAPH JSON write to {graph_path} failed: {exc}")+ else:+ print(f"GRAPH JSON written to {graph_path}")+ return 0+++def main() -> None:+ parser = argparse.ArgumentParser(description=__doc__)+ parser.add_argument("--bundle", default=DEFAULT_BUNDLE)+ parser.add_argument("--pipe", default=DEFAULT_PIPE, dest="pipe_code")+ parser.add_argument("--no-pipe", action="store_true", help="Do not pass a pipe_code (use the bundle's main_pipe, if any)")+ parser.add_argument("--allow-signatures", action="store_true")+ parser.add_argument("--graph-out", default=DEFAULT_GRAPH_OUT)+ args = parser.parse_args()++ pipe_code: str | None = None if args.no_pipe else args.pipe_code+ make_pipelex_for_cli(context=ErrorContext.VALIDATION_BEFORE_PIPE_RUN, library_dirs=None, temporal_enabled=True)+ try:+ exit_code = asyncio.run(submit(bundle=args.bundle, pipe_code=pipe_code, allow_signatures=args.allow_signatures, graph_out=args.graph_out))+ finally:+ Pipelex.teardown_if_needed()+ sys.exit(exit_code)+++if __name__ == "__main__":+ main()diff --git a/.github/workflows/tests-check.yml b/.github/workflows/tests-check.ymlindex 6bd591d55..e9e160466 100644--- a/.github/workflows/tests-check.yml+++ b/.github/workflows/tests-check.yml@@ -24,6 +24,7 @@ jobs:env:VIRTUAL_ENV: ${{ github.workspace }}/.venvENV: dev+ PIPELEX_HANG_DUMP_DIR: /tmp/pipelex-hang-dumps-${{ github.run_id }}-py${{ matrix.python-version }}steps:- uses: actions/checkout@v4@@ -36,6 +37,19 @@ jobs:- name: Run testsrun: make gha-tests+ - name: Show hang dumps+ if: failure()+ run: |+ if ls "$PIPELEX_HANG_DUMP_DIR"/* >/dev/null 2>&1; then+ for dump_file in "$PIPELEX_HANG_DUMP_DIR"/*; do+ echo "::group::$dump_file"+ cat "$dump_file"+ echo "::endgroup::"+ done+ else+ echo "No hang dumps were produced."+ fi+# --------------------------------------------------------------------------# 2. Aggregator job — the *single* required status check# --------------------------------------------------------------------------diff --git a/.pipelex/pipelex.toml b/.pipelex/pipelex.tomlindex e4a2ff815..de39d5c70 100644--- a/.pipelex/pipelex.toml+++ b/.pipelex/pipelex.toml@@ -275,6 +275,8 @@ backoff_coefficient = 2.0maximum_interval = "unlimited"maximum_attempts = 3non_retryable_error_types = [+ "DryRunMockBuildError",+ "DryRunObjectFidelityError","ExtractHandleNotFoundError","FileNotFoundError","ImgGenHandleNotFoundError",diff --git a/AGENTS.md b/AGENTS.mdindex 49a20efdb..a57c7b2a5 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -69,7 +69,7 @@.venv/bin/pipelex-dev generate-mthds-schema```- - **`generate-error-pages`**: Regenerate the per-class error reference pages under `docs/errors/` (one page per `PipelexError` subclass). Run after adding or renaming an error class. Also available as `make generate-error-pages` (alias `make gep`).+ - **`generate-error-pages`**: Regenerate the per-class error reference pages under `docs/errors/` — one Markdown page per `PipelexError` subclass, which is what each error's `type_uri` dereferences to. Run after adding or renaming an error class. Pages a maintainer claims with a `<!-- pipelex:authored -->` marker are preserved across runs. Also available as `make generate-error-pages` (alias `make gep`).```bash.venv/bin/pipelex-dev generate-error-pagesdiff --git a/CHANGELOG.md b/CHANGELOG.mdindex e832ed49b..4bc057732 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -1,5 +1,32 @@# Changelog+## [Unreleased]++### Changed++- **`--dry-run` now honors the configured backend — the mock moved from the operators to the cogt leaf.** `run_mode` rides a new `CogtRunParams` carrier (derived from `PipeRunParams.run_mode`, stamped on every cogt assignment, serialized across the Temporal wire), so a DRY run on a Temporal backend dispatches the REAL `act_llm_gen_*` / extract / img-gen / search / render activities and the leaf mocks **inside** the activity — testing dispatch, scheduling, serialization, routing, and cross-worker propagation at zero AI cost, with no API keys and no storage IO (the img/extract DRY branches sit above the store step). `ContentGeneratorDry` is deleted: operators no longer swap generators (the base operator's dry path simply reuses the live path), and the in-process validation scopes pass an inline `ContentGenerator`. Object mocks are now schema-built on every backend — identical mock on direct and Temporal; classes with exotic format constraints should declare `examples`/`mock_format` (a re-validation failure surfaces as a typed `DryRunObjectFidelityError`, and a deterministic mock-build failure as the new non-retryable `DryRunMockBuildError`). The acceptance gate is Tier 17 of the `temporal-e2e-validate` skill (GREEN on all leaf families + keyless worker, RED-proven) plus a Mode-1 pytest.+- **Unified dry run — `--mock-inference` removed (breaking).** There is now exactly one non-live run mode: `--dry-run`. The mock-inference mode is retired; its capability — non-zero synthetic usage so the cost report renders, used to validate cross-worker cost-report assembly cheaply — survives as an **internal** `is_mock_usage` sub-flag of DRY on `CogtRunParams` (replacing `is_mock_inference`; setting it on a LIVE run is a validation error). It is exposed on the Python surface (`PipelexRunner` / `execute_pipeline` / `prepare_pipe_job` via `is_mock_usage=...`) and as a hidden test-only CLI trigger, deliberately undocumented. Dry-run leaf coverage was already uniform across every operator, so `MockInferenceUnsupportedError` and its img-gen/extract/search guards are deleted; `MockInferenceObjectFidelityError` is renamed `DryRunObjectFidelityError` (mock-built objects are DRY-only now); the `mock_inference` usage sentinel is renamed `mock_usage`.+- **Keyless boot (`needs_inference=False`) now forces every run to DRY instead of installing a mock generator.** Generator selection is purely backend-keyed (a keyless Temporal submitter still dispatches and mocks inside activities); the forced-DRY flag is consumed at `PipeRunParamsFactory.make_run_params` — the single writer of `run_mode` — so every execution entry point is covered, including the runtime bridge.++### Removed++- **`dry_run_config.apply_to_jinja2_rendering`** config key: it was dead — PipeCompose renders templates directly and the jinja2 parse check survives in the templating leaf's DRY branch. Remove it from your `.pipelex/pipelex.toml` override if present (config is strict and will reject the unknown key).++### Fixed++- **`dry_run_pipeline` now owns its graph transport — graph generation no longer depends on the host's `tracing_config`.** The function requests a graph explicitly (`generate_graph=True`) but relied on the host's configured tracing backend as the emit/assemble channel, so a host with `tracing_config.is_enabled = false` (e.g. pipelex-api's `/validate` in direct mode, where the web-app's Dry Run button lands) always got "Pipeline execution did not produce a graph spec" and a response without `graph_spec`. The run now traces through a scoped `InMemoryEventLog` — the graph comes back under any tracing config, and validation dry-runs stop writing NDJSON/DynamoDB trace events as a side effect when tracing IS enabled. The no-graph contract violation is now a typed `DryRunGraphNotProducedError` (was a bare `PipelexError`, ambiguous in host logs that record only the exception type) — raised by both `dry_run_pipeline` and `dry_run_pipe_in_process`.+- **tz-aware payloads no longer hang Temporal callers on hosts without a timezone database — kajson bumped to 0.7.0.** kajson ≤ 0.6.0 decoded timezone-aware datetimes via `ZoneInfo(...)`, which needs the IANA tz database: system tz files or the `tzdata` package. On hosts with neither (slim containers, uv-managed standalone Pythons — including the CI test runner, where `tzdata` only arrived transitively via pandas on Python < 3.11), decoding any aware-datetime payload crossing the Temporal boundary (e.g. the `GraphSpec` on `act_dry_validate`'s result) raised `ZoneInfoNotFoundError` *inside the workflow task* — which Temporal retries forever, so the submitter hung instead of failing. Fixed upstream in kajson 0.7.0: its self-sufficient timezone wire format carries the UTC offset (decoding degrades gracefully to a fixed offset instead of raising, and plain UTC decodes with zero tz-database dependence), and kajson now declares `tzdata` as its own runtime dependency, so a tz database is always available for named zones.++### Added++- **Temporal test hang diagnostics (CI).** Setting `PIPELEX_HANG_DUMP_DIR` arms a per-test watchdog in the temporal integration suite that dumps every thread's stack to a file before pytest-timeout's thread-method kill destroys the process, and mirrors temporalio's failure logs (which carry the workflow-task traceback) to the same directory. The tests workflow prints these dumps when a job fails — pytest-xdist swallows worker stderr on a crash, so this is the only post-mortem signal for a hung test on a remote runner.++- **`act_dry_validate` — the whole `/validate` job as ONE in-process Temporal activity.** New activity (+ one-step wrapper workflow `wf_dry_validate` and submitter helper `dispatch_dry_validate`) that runs the validation sweep **and** the graph-producing dry-run in one round-trip: the library is loaded once, the sweep borrows it, the graph dry-run traces into an in-memory event log against the same library, and the result carries `{per-pipe status map, GraphSpec | None}`. Zero nested Temporal dispatch, no NDJSON/DynamoDB tracing I/O, no usage/cost reporting. The graph is best-effort (an expected dry-run failure returns `graph_spec=None` with validation still successful); the sweep half is `validate_bundle` itself, so validation failures cross the activity boundary as structured `ErrorReport`s carrying the same categorized `ValidateBundleError` identity the direct path surfaces (a strict-mode signature refusal keeps its offending-pipe and signature refs in the caller-facing message). Registered in the `pipe` task pack; verified end-to-end in a real 3-process deployment (Tier 2d of the `temporal-e2e-validate` skill).++- **`dry_run_pipe_in_process` — graph-producing dry-run that never dispatches.** New entry in `pipelex/pipe_run/dry_run_pipeline.py`: dry-runs a pipe against an already-open library fully in-process and returns its `GraphSpec`, even under a Temporal-enabled hub. Three ContextVar scopes pin the whole run in-process: `scoped_event_log` (in-memory trace transport), `scoped_pipe_router` (nested controller sub-pipes resolve a local router instead of the `TemporalPipeRouter`), and the new `pipelex.hub.scoped_content_generator` (inference leaves resolve an inline dry generator instead of `ContentGeneratorInWorkflow` — holds regardless of whether the DRY mock lives at the pipe level or the leaf). This is the graph half of running `/validate` as one in-process Temporal activity.++- **`scoped_event_log` — a shared, scope-pinned trace-event transport.** New `pipelex.hub.scoped_event_log(event_log)` context manager (with `get_event_log_override()` accessor, mirroring `scoped_pipe_router`): both the write side (tracer emission in `pipeline_run_setup`) and the read side (`assemble_tracing`) now prefer the scoped instance over building their own backend via `make_event_log`, so a run can trace into ONE shared `InMemoryEventLog` — no NDJSON file, no DynamoDB round-trip — and still assemble its `GraphSpec` from it. A set override implies tracing-enabled (honored even when `tracing_config.is_enabled` is off); the scoped instance's `close()` must be idempotent or a no-op (the write-side tracer closes its transport at teardown — `InMemoryEventLog`'s is a no-op). ContextVar-scoped, so concurrent runs with separate scopes stay isolated. This is the foundation for running the `/validate` graph dry-run fully in-process and in-memory inside a Temporal activity.+## [v0.32.1] - 2026-06-09### Addeddiff --git a/TODOS.md b/TODOS.mdindex 8395ab551..10218c19a 100644--- a/TODOS.md+++ b/TODOS.md@@ -1,137 +1,259 @@-# Fix: `/validate` dry-run leaked nested controller sub-pipes to Temporal+# Dry-run + validation as one in-process Temporal activity (in-memory tracing)-> **Status: DONE.** Bug fixed and all three layers of test coverage are in place. This doc is the PR-review guide — what the bug was, what changed, why it's correct, and how it's guarded.+> **Branch:** `feature/Dry-run-as-temporal-activity`. This file is the **executable plan + progress tracker** for **Mode 1** (in-memory activity). Update the checkboxes and the checkpoint handoff blocks as you go.>-> **Stacking:** this branch (`fix/Temporal-dry-run`) is stacked on `feature/PipeSearch-Temporal` (open PR [#974](https://github.com/Pipelex/pipelex/pull/974)). The `pipe_search.py` / `search_generate.py` / `content_generator_in_workflow.py` / `native_search.mthds` / `structured_search.mthds` / `test_workflow_search_*` files in the diff belong to **#974**, not to this change — review them there. The files owned by **this** change are listed under [Files in this change](#files-in-this-change) below.+> **Read first, in order:** (1) [`wip/dry-run-refactor/dry-run-modes-master-plan.md`](wip/dry-run-refactor/dry-run-modes-master-plan.md) — the big picture (both dry-run modes, what they share); (2) [`wip/dry-run-refactor/followup-temporal-validation-activity.md`](wip/dry-run-refactor/followup-temporal-validation-activity.md) — this mode's design / why; (3) the "Cold-start context" section just below. **The other mode (full-distribution, leaf-mocks) is out of scope here** — return to the master plan for it after Checkpoint F.+>+> **Goal:** when Temporal is enabled, the API dispatches the whole `/validate` job — validation sweep **+** graph-producing dry-run — to a worker as **one activity** that runs entirely **in-process** and traces the graph in an **in-memory** event log (no DynamoDB round-trip, no NDJSON files, no usage/cost reporting). Direct mode stays in-process and unchanged.++---++## Cold-start context (read this first in a fresh session)-**Where the fix lives:** `pipelex` core (the runtime), **not** `pipelex-api`. All paths are relative to the `pipelex` repo (locally the editable `_search` worktree). No change was needed in `pipelex-api`.+The five facts that shape the whole plan — verify by symbol before relying on them, but they were true at branch start:-## Symptom+1. **The validation sweep is already Temporal-safe and in-process.** `BundleValidator.validate_pipes` wraps its per-pipe loop in `scoped_pipe_router(self._pipe_router)` (PR #976) and runs a locally-constructed `PipeRun` — never `get_pipe_run()`. DRY mocks inline at the pipe level (`ContentGeneratorDry`). So the sweep half is ready to wrap; **Part B is NOT a prerequisite** (it's the opposite cell — a top-level DRY run that *should* hit the worker path).-With the API running Temporal-enabled (`.pipelex/pipelex_override.toml` → `[temporal] is_enabled = true`, task queue `pipelex_dev`) and a worker up:+2. **The graph dry-run is the real work.** `pipelex-api`'s `/validate` (`../pipelex-api/api/routes/pipelex/validate.py`) calls `validate_bundle()` (sweep) **and** `dry_run_pipeline()` (graph). `dry_run_pipeline` (`pipelex/pipe_run/dry_run_pipeline.py`) goes through `PipelexRunner.execute_pipeline` → `get_pipe_run()` → `TemporalPipeRun` under a Temporal hub → dispatches a top-level workflow to the worker, which traces to `temporal_dynamodb` and assembles the `GraphSpec`. That works today (single dispatch, no collision). **Do not move it in-process *in the API*** — the API runs tracing-off; that would break graph generation. The new design moves it in-process *inside the activity, on the worker*.-- `make bundle-validate BUNDLE=.../fashion_moodboard` → **HTTP 422**-- `make bundle-validate BUNDLE=.../joke_judge` → **OK**-- `make bundle-run BUNDLE=.../fashion_moodboard` (full run) → **OK**-- `pipelex validate bundle .../fashion_moodboard` (CLI) → **OK**+3. **The two-instance problem (the crux).** Emit and assemble build *separate* event logs from config: write side `pipeline_run_setup.py` (`make_event_log(tracing_config)`, ~line 200, handed to the tracer); read side `pipelex/pipe_run/tracing_assembly.py` `assemble_tracing` (`make_event_log(tracing_config)` again, ~line 95, then `read_events`), triggered from `PipeRun.run`'s `finally` (`pipelex/pipe_run/pipe_run.py` ~line 76). NDJSON/DynamoDB bridge the two instances via an external store; a plain in-memory log can't. **Fix = a shared instance scoped to the run.**-The 422 body:+4. **`InMemoryEventLog` already exists** (`pipelex/tracing/in_memory_event_log.py`, implements `EventLogProtocol`, used in unit tests). We don't write it — we *wire* it via a scope.-```-ValidateBundleError (error_domain=input), 422-detail: "Dry run failed with 1 unexpected pipe failure(s):- 'fashion_moodboard.create_moodboards': ... status=DryRunStatus.FAILURE- error_message=\"Dry run failed for pipe 'fashion_moodboard.create_moodboards':- Failed to execute workflow WfPipeRouter\""-```+5. **The scope pattern to mirror** lives in `pipelex/hub.py`: `scoped_current_library` (~line 518) and `scoped_pipe_router` (~line 635) — `ContextVar` + `@contextmanager` + a `get_*` accessor that prefers the override. The in-process "force DIRECT under a Temporal worker" primitive to reuse is `pipelex/runtime_bridge/bridge.py` `_run_direct` (~line 318) / `run_pipe_via_bridge` (~line 111, already accepts a `trace_context` honored in DIRECT mode).-The string **"Failed to execute workflow WfPipeRouter"** (no trailing detail) is produced only at `pipelex/temporal/tprl/workflow_caller.py:136`, the `(WorkflowAlreadyStartedError, RPCError)` branch of `WorkflowExecutor.execute_workflow` — i.e. a **top-level** Temporal workflow dispatch from the submitter side. A dry run should never reach that code.+### File map-## Root cause+| Concern | File |+|---|---|+| Event-log protocol / impls | `pipelex/tracing/{event_log_protocol,in_memory_event_log,ndjson_event_log,dynamodb_event_log,buffering_event_log}.py` |+| Event-log factory | `pipelex/tracing/event_log_factory.py` (`make_event_log`) |+| Scope pattern to mirror | `pipelex/hub.py` (`scoped_current_library`, `scoped_pipe_router`, `get_pipe_router`) |+| Write-side wiring | `pipelex/pipeline/pipeline_run_setup.py` (~200) |+| Read-side wiring | `pipelex/pipe_run/tracing_assembly.py` (~95), called from `pipelex/pipe_run/pipe_run.py` (~76) |+| Graph dry-run (to make in-process/in-memory) | `pipelex/pipe_run/dry_run_pipeline.py` |+| In-process run primitive to reuse | `pipelex/runtime_bridge/bridge.py` (`_run_direct`, `run_pipe_via_bridge`, `build_pipe_job_from_input`) |+| Validation sweep | `pipelex/pipeline/bundle_validator.py` |+| Tracer lifecycle | `pipelex/graph/graph_tracer_manager.py` (`open_tracer`, `close_tracer`) |+| Activity error boundary | `pipelex/temporal/tprl/activity_error_boundary.py` (`convert_pipelex_errors`) |+| Existing activities + registration | `pipelex/temporal/tprl_pipe/act_*.py`; `pipelex/temporal/tasks.py` (`PackName.PIPE` activity_list) |+| API route (cross-repo) | `../pipelex-api/api/routes/pipelex/validate.py` |-The `/validate` path runs an **in-process** dry-run sweep, but nested controller sub-pipes leaked out to the Temporal router.+----Flow: `api/routes/pipelex/validate.py` → `validate_bundle()` (`pipelex/pipeline/validate_bundle.py`) → `BundleValidator.validate_pipes()` (`pipelex/pipeline/bundle_validator.py`).+## Pre-flight — ALL DECIDED (2026-06-09)-`BundleValidator` deliberately builds a **local, in-process** execution primitive so the sweep never touches Temporal — its own module docstring says so. **But that local router was only used for the _top-level_ pipe.** Every controller dispatches its nested sub-pipes through the **hub's** `get_pipe_router()`, not the injected router. This is broad — it is **not** specific to `PipeBatch`:+All decisions are resolved up front so a cold-start session runs autonomously through Checkpoint F. There is **no remaining human gate** — the dispatch question that used to gate this was resolved in favor of the wrapper workflow (no `temporalio` bump).-```python-# pipelex/pipe_controllers/batch/pipe_batch.py (_live_run_controller_pipe)-return await get_pipe_router().run(pipe_job=PipeJobFactory.make_pipe_job(...))+- [x] **(DECIDED) In-memory exposure = scope-only.** Add `scoped_event_log` only; do **not** add a `TracingBackend.IN_MEMORY` config enum (insufficient alone — the two-instance problem — and it would change the global default instead of being opt-in per call).+- [x] ~~**(DECIDED) Graph dry-run shape = bridge DIRECT reuse.**~~ ⚠️ **REVISED BY ENG REVIEW (2026-06-09) — see GSTACK REVIEW REPORT, decision D-C1.** The bridge **cannot** carry `run_mode=DRY` + `mock_inputs`: `PipelexPipeRunInput` has no run-mode/mock-input field and `build_pipe_job_from_input` (`bridge.py:216`) calls `make_run_params()` → `PipeRunMode.LIVE` with no mock-input synthesis. Running the main pipe through `run_pipe_via_bridge` DIRECT would produce a **LIVE run with no inputs** that fails before reaching the graph. **New shape: use the `prepare_pipe_job` seam** (`execution_seams.py:199`) with `pipe_run_mode=PipeRunMode.DRY` + `execution_config(mock_inputs=True, generate_graph=True)` (what `dry_run_pipeline.py` already does), running a **local `PipeRun`** under `scoped_event_log` + `scoped_pipe_router` + `scoped_content_generator`. Still do **not** refactor `PipelexRunner`; still no bridge wire-contract change.+- [x] **(DECIDED) One activity (status + graph).** One activity returns `{status map, GraphSpec, signature_check_error}`; the graph dry-run is **best-effort inside the activity** — a graph failure still returns a successful validation with `graph=None`, preserving today's `/validate` semantics. (Not two separate activities.)+- [x] **(DECIDED) Dispatch = one-step wrapper workflow (no `temporalio` bump).** The API dispatches a one-step workflow that runs the single activity and returns — works on the current `temporalio` (`1.23.0`), no SDK bump, no Temporal Cloud/server verification, **no hard gate**. Functionally identical to the caller. A true standalone activity is a **later optional optimization** (Phase G0, deferred — not on this branch's critical path).+- [x] **(DECIDED: BUILD THE SCOPE NOW) Force the inline content generator.** Today the in-process sweep/dry-run is safe because DRY mocks at the **pipe** level (`ContentGeneratorDry` inline) and never calls `get_content_generator()`. But under a Temporal-enabled hub `get_content_generator()` is `ContentGeneratorInWorkflow` **globally** (boot-time, `pipelex.py:370-385`). The moment **Part B** ([`wip/dry-run-refactor/followup-leaf-run-mode-mock.md`](wip/dry-run-refactor/followup-leaf-run-mode-mock.md)) relocates the DRY mock to the leaf, this activity's leaf would dispatch `act_llm_gen_*` and break the in-process guarantee. **DECISION (2026-06-09): add the inline content-generator scope now** (`scoped_content_generator`, mirroring `scoped_pipe_router`) in Phases 2–3, so the activity is correct regardless of Part-B ordering and the shared seam is built once. Rejected the "rely on the pipe-level mock, defer the scope" alternative as fragile (would silently regress when Part B lands). The Phase-2/3 zero-dispatch test must simulate the leaf-level mock (not only today's pipe-level mock) so the guard isn't a no-op until then.+ - This scope is **orthogonal to** the req-1 "full-distribution leaf-mock" mode (Mode C) — that mode deliberately *does* dispatch and mocks inside the activity. The two coexist (proof: `is_mock_inference` already ships the LLM slice of Mode C alongside the DRY path). The scope is per-call/ContextVar, so it doesn't disturb Mode C. See [`wip/dry-run-refactor/dry-run-modes-master-plan.md`](wip/dry-run-refactor/dry-run-modes-master-plan.md).++> ### ✅ Former HUMAN GATE — RESOLVED (2026-06-09): wrapper workflow, no bump+>+> The `temporalio` standalone-activity question that used to gate dispatch is **resolved**: we use the **one-step wrapper workflow** on the current `temporalio` (`1.23.0`). No SDK bump, no Temporal Cloud/server verification, nothing to ask. The cold-start session proceeds through all phases without stopping for human input. (Standalone activity is a later optional optimization — Phase G0, deferred.)-# pipelex/pipe_controllers/sub_pipe.py (SubPipe.run_pipe — used by EVERY PipeSequence step;-# also the inline `batch_over` path and the PipeCondition path)-pipe_output = await get_pipe_router().run(pipe_job=PipeJobFactory.make_pipe_job(...))-```+----`get_pipe_router()` (`pipelex/hub.py`) returns a contextvar override **if one is set**, else the hub default. In the Temporal-enabled API process the hub default is the **`TemporalPipeRouter`** (wired in `pipelex/pipelex.py` when `get_config().temporal.is_enabled`). Called from the API process — i.e. **outside** any Temporal workflow (`is_in_temporal_workflow()` is False) — `TemporalPipeRouter._run_pipe_job` took the **top-level dispatch** branch → `executor.execute_workflow(WfPipeRouter, ...)` → failed → "Failed to execute workflow WfPipeRouter". The per-pipe dry run was marked `FAILURE`, `BundleValidator._aggregate` raised `DryRunError`, `validate_bundle` translated it to `ValidateBundleError`, and the global handler rendered 422.+## Status at a glance-`BundleValidator` built the in-process router but **never installed it as the `scoped_pipe_router(...)` override**, so the nested controllers fell through to the hub default (Temporal).+| Phase | Title | Status |+|---|---|---|+| 1 | `scoped_event_log` + shared in-memory tracing (in-repo, no Temporal) | ☑ done |+| | **⛔ CHECKPOINT 1 — in-memory tracing verified in direct mode** | ☑ done (suite green, committed) |+| 2 | In-process, in-memory graph dry-run safe under a Temporal hub (+ `scoped_content_generator`) | ☑ done |+| | **⛔ CHECKPOINT 2 — graph dry-run verified under a Temporal hub** | ☑ done (suite green, committed) |+| 3 | The `act_dry_validate` activity + wrapper-workflow dispatch + worker registration + isolation test | ☑ done |+| | **⛔ CHECKPOINT 3 — activity registered + isolation-tested** | ☑ done (suite green, Tier 2d GREEN+RED, committed) |+| 4 | API dispatch (cross-repo `pipelex-api`) | ☑ done |+| | **⛔ CHECKPOINT F — all requirements met** | ☑ done (both suites green, Tier 2d both arms GREEN, committed) |+| G0 | *(optional, deferred)* `temporalio` bump → true standalone activity | ☐ later |-## FAIL vs. FALSE-PASS — the leak happened always; only some shapes 422'd+Status legend: ☐ not started · ◐ in progress · ☑ done. **No human gate remains** (dispatch = wrapper workflow).-The leak (nested dispatch → `TemporalPipeRouter`) happened for **every** bundle with a controller. Whether the dry run **failed** depended on a Temporal **workflow-id collision**:+----- `make_top_workflow_id` (`pipelex/temporal/temporal_manager.py`) is `f"{prefix}{pipeline_run_id}"` — derived from `pipeline_run_id` **alone**. The whole sweep shares **one** `dry_run_pipeline_id`, so **every** top-level Temporal dispatch in a sweep computed the **same** workflow id.-- `WorkflowExecutor.execute_workflow` **waits for completion**, and no `id_reuse_policy` is set → Temporal's default `ALLOW_DUPLICATE` (reuse allowed once the prior run has *closed*). So **serial** dispatches reusing that id are fine; only **concurrent** same-id dispatches collide → `WorkflowAlreadyStartedError` → `WorkflowExecutionError("Failed to execute workflow WfPipeRouter")`. (`WorkflowExecutionError` is a `PipelexError`, so it records as a dry-run `FAILURE` → 422, not a 500.)+## Phase 1 — `scoped_event_log` + shared in-memory tracing-The deciding structural factor is **standalone batch/parallel vs. batch-reached-as-a-sub-pipe**:+Pure in-repo capability, no Temporal, no cross-repo. Fully testable in direct mode. **TDD: write the failing tests first.**-- A **standalone top-level `PipeBatch`/`PipeParallel`** the sweep dry-runs *directly* runs its top level on the local in-process router, but its **fan-out loop executes in the API process** (`_live_run_controller_pipe` → `gather_bounded`) and fires **N concurrent top-level Temporal dispatches** with the same id → collision → **FAILURE**. A mocked "multiple" list yields `nb_stuffs = 3`, so N≥2.-- A **batch reached as a sub-pipe** (inline `batch_over` step, or a batch nested under a sequence/parallel) is dispatched as a **single** top-level workflow — the fan-out into branches then happens **inside the worker** as child workflows (legal). Sequence steps run serially, so it's one dispatch at a time → **no concurrent collision** → it **passed** (still round-tripping Temporal, just not colliding — a false pass).+- [x] *Tests first* (`tests/unit/pipelex/tracing/test_scoped_event_log.py` + `tests/integration/pipelex/pipeline/test_scoped_in_memory_tracing.py`): assert that running a dry-run-with-graph under `with scoped_event_log(InMemoryEventLog())` (a) produces a non-empty, correct `GraphSpec`, (b) writes **no** NDJSON file and touches **no** configured backend, and (c) emit and assemble hit the **same** instance. Concurrency/nesting tests included (unit: ContextVar isolation + nesting + exception restore; integration: two concurrent scoped dry-runs stay isolated).+- [x] Added to `pipelex/hub.py` (mirroring `scoped_pipe_router`): `_event_log_override: ContextVar["EventLogProtocol | None"]`, `scoped_event_log(event_log)` `@contextmanager` (save/set/restore), and accessor `get_event_log_override()`.+- [x] Write side prefers the override: `pipeline_run_setup.py` (`event_log = get_event_log_override()`, factory only when `None` and `is_enabled`). D1: a set override implies tracing-enabled.+- [x] Read side prefers the override: `tracing_assembly.py::assemble_tracing` — the `is_enabled` early-return is bypassed when an override is set (D1); the override is read **without** `close()` (the scope owner keeps the instance's lifecycle).+- [x] `EventLogProtocol` surface confirmed sufficient — no protocol change.+- [x] `make agent-check` clean · new tracing tests green (7/7).-So `fashion_moodboard`'s `create_moodboards` (a standalone `PipeBatch`) 422'd, while a sequence-nested batch silently round-tripped Temporal and passed. The fix removes the Temporal round-trip for the whole sweep, fixing both the real failure **and** the silent false-pass leak.+> ### ⛔ CHECKPOINT 1 — after Phase 1 — **MANDATORY STOP**+>+> A self-contained, separately reviewable in-repo capability. Natural place to split sessions and/or land alone.+>+> **Verify:** `make agent-check` clean · `make agent-test` green · the new in-memory-tracing tests pass (same-instance emit+read, zero file/backend, concurrency-safe) · commit.+>+> **Handoff (filled in, 2026-06-09):**+>+> - **Scope API (`pipelex/hub.py`, right after `get_pipe_router`):** `scoped_event_log(event_log: EventLogProtocol)` — `@contextmanager`, ContextVar save/set/restore, mirrors `scoped_pipe_router`. Accessor: `get_event_log_override() -> EventLogProtocol | None`. ContextVar: `_event_log_override`.+> - **Write-side edit:** `pipeline_run_setup.py::pipeline_run_setup` — `event_log = get_event_log_override()`; falls back to `make_event_log(tracing_config)` only when no override AND `tracing_config.is_enabled`.+> - **Read-side edit:** `tracing_assembly.py::assemble_tracing` — early-return is now `if event_log_override is None and not tracing_config.is_enabled`; with an override, `read_events` is called on it directly and it is NOT `close()`d (scope owner keeps lifecycle; the no-override branch keeps its `make_event_log` + `close()` shape — D2 symmetric primitive).+> - **`is_enabled` vs override (D1):** a set override **implies tracing-enabled** at both guards; regression-tested (`test_override_implies_enabled_when_tracing_disabled`).+> - **`EventLogProtocol`:** unchanged.+> - **Other `make_event_log` call sites left alone (intentional):** `tracing/activity_event_log.py` (Temporal worker per-process usage log — Phase-3 activity never reaches it) and `runtime_bridge/primitives/trace_flush.py` (bridge DIRECT flush — off-path since D-C1 dropped the bridge).+>+> **Next: Phase 2.**++---++## Phase 2 — In-process, in-memory graph dry-run safe under a Temporal hub++Make the graph-producing dry-run run fully in-process even under a Temporal-enabled hub, tracing into the Phase-1 in-memory log. Reuse the bridge DIRECT primitive (it already forces in-process via `scoped_pipe_router` + a local `PipeRun`, and `run_pipe_via_bridge` already honors a `trace_context` in DIRECT mode).++- [x] *Tests first* (`tests/integration/pipelex/temporal/test_dry_run_graph_in_process.py`): under the suite's **Temporal-enabled** hub (real `TemporalPipeRouter` + `ContentGeneratorInWorkflow` as hub defaults, asserted as preconditions), the in-process graph dry-run (a) returns a correct `GraphSpec` covering the full `temporal_parallel` controller topology, (b) dispatches **zero** workflows (spy `WorkflowExecutor.execute_workflow`), and (c) touches no file/DDB transport (`make_event_log` forbidden via patch).+- [x] Added `dry_run_pipe_in_process(pipe, *, library_id)` in `pipelex/pipe_run/dry_run_pipeline.py`: opens a `GraphTracerManager` tracer against an `InMemoryEventLog`, runs via `prepare_pipe_job` (DRY + mock_inputs + generate_graph) + a local `PipeRun` under the three scopes (`scoped_event_log` + `scoped_pipe_router` + `scoped_content_generator`) — not the bridge, not `PipelexRunner` (D-C1). `GraphSpec` rides back on `PipeOutput`. Caller owns the library lifecycle (mirrors `validate_pipes`, ready for D4).+- [x] Added `scoped_content_generator(content_generator)` in `hub.py` (ContextVar + contextmanager + `get_content_generator()` prefers the override). Leaf-level-mock simulation test (`test_leaf_level_mock_stays_in_process`) forces DRY through `get_content_generator()` and stays zero-dispatch; **RED-proven**: with the scope removed, the leaf reached `ContentGeneratorInWorkflow` and failed with `_NotInWorkflowEventLoopError`.+- [x] Tracer-key alignment (D-C7): tracer opened at `graph_id=pipeline_run_id` and closed by `pipeline_run_id` in `PipeRun.run`'s finally — aligned by construction, no divergent `tracer_key` parameter exposed; pinned by the non-empty-graph assertions (a divergent key would assemble an empty graph).+- [x] `dry_run_pipeline`'s existing (worker-workflow + DynamoDB) path left intact — the API still uses it until Phase 4.+- [x] `make agent-check` clean · zero-dispatch / in-memory-graph tests green.++> ### ⛔ CHECKPOINT 2 — after Phase 2 — **MANDATORY STOP**+>+> **Verify:** under a Temporal-enabled hub the in-process graph dry-run yields a `GraphSpec` with **zero** nested dispatch and **zero** file/DDB I/O · `make agent-test` green · commit.+>+> **Handoff (filled in, 2026-06-09):**+>+> - **Entry point:** `dry_run_pipe_in_process(pipe: PipeAbstract, *, library_id: str) -> GraphSpec` in `pipelex/pipe_run/dry_run_pipeline.py`. Takes a resolved pipe + an already-open library (caller owns the library lifecycle — fits D4's load-once activity shape). Raises `PipelexError` when the run fails or yields no graph.+> - **Tracer lifecycle:** `open_tracer(graph_id=pipeline_run_id, event_log=InMemoryEventLog, workflow_id="direct", emit_graph_events=True, emit_usage_events=False)` before the run; `trace_context` threaded via `prepare_pipe_job(trace_context=...)` (no bridge involved — D-C1); closed by `pipeline_run_id` inside `PipeRun.run`'s `finally` (which also assembles from the scoped log onto `PipeOutput`); a safety-net `close_tracer(pipeline_run_id)` in the entry's own `finally` covers pre-run failures (idempotent pop).+> - **Tracer-key alignment:** `graph_id == pipeline_run_id == close key`, by construction; no `tracer_key` parameter exposed. A unique `dry_run_graph_<uuid>` run id is minted per call.+> - **Scopes:** `scoped_event_log(event_log)` + `scoped_pipe_router(PipeRouter(observer=ObserverNoOp()))` + `scoped_content_generator(ContentGeneratorDry())` wrap `prepare_pipe_job` + `pipe_run.run`. `scoped_content_generator` is new in `hub.py`; `get_content_generator()` now prefers the ContextVar override.+> - **Divergence from plan:** none of substance. No `PipelineManager.add_new_pipeline` registration and no report-delegate usage-event wiring (mirrors the sweep, not `pipeline_run_setup`) — deliberate: no usage/cost in this mode.+>+> **Next: Phase 3.**++---++## Phase 3 — The `act_dry_validate` activity + wrapper-workflow dispatch + registration + isolation test++Wrap the sweep + the in-memory graph dry-run in one activity, dispatched via a one-step wrapper workflow (DECIDED — no `temporalio` bump). No gate; runs straight through.++- [x] Activity defined (`pipelex/temporal/tprl_pipe/act_dry_validate.py`): body runs **`validate_bundle`** (the exact function the direct-mode route calls — sweep → status map AND the categorized `ValidateBundleError` error cascade, so both backends surface the identical 422 contract) **and** `dry_run_pipe_in_process` (→ `GraphSpec`, against the same library `validate_bundle` left loaded+current — D4), teardown once in the activity's `finally`. The scopes live in the callees (the Phase-2 entry carries all three; the sweep carries `scoped_pipe_router` + `scoped_content_generator` itself). **Graph best-effort per D5 (widened to cross-backend parity in the post-implementation review, R-D1):** catches `PipelexError` + `ValidationError`/`FactoryException` — the same catch `_classify_pipe` and the direct route use, and pipe resolution sits inside it — → `graph=None`; non-Pipelex bugs propagate. Decorated with `convert_pipelex_errors`.+- [x] Input `DryValidateArg` (`mthds_contents`/`allow_signatures`/`pipe_code` — no `bundle_uris` and, post-review R-D3, no `library_dirs`: no caller passes them and `library_dirs` was an unconstrained worker-filesystem read surface); output `DryValidateResult` (`dry_run_outputs: dict[str, DryRunOutput]` + `graph_spec: GraphSpec | None`). Per D3: no `signature_check_error` field — signature refusals raise (wrapped as `ValidateBundleError`, same as direct mode).+- [x] One-step wrapper workflow `wf_dry_validate.py` (`WfDryValidate`) runs the single activity with explicit `start_to_close_timeout=5min` + `RetryPolicy(maximum_attempts=2, non_retryable_error_types=["ValidateBundleError"])` (D-C5; post-review the list is just the one name that actually crosses — `validate_bundle` wraps the other deterministic failures into `ValidateBundleError` and the graph arm degrades instead of raising). Submitter helper `dry_validate_dispatch.py::dispatch_dry_validate` is the one-round-trip call the API (Phase 4) and the Tier-2d script share.+- [x] Registered both in `pipelex/temporal/tasks.py` (`PackName.PIPE`).+- [x] *Isolation tests* (`tests/integration/pipelex/temporal/test_dry_validate_activity_in_memory.py`): real worker against the in-process server; status map + GraphSpec correct in one round-trip; **zero nested dispatch asserted on the Temporal history** (exactly one `ActivityTaskScheduled`, zero child workflows — D6, stronger than the wrapper spy); in-memory tracing (`make_event_log` forbidden, no NDJSON); best-effort graph (`graph=None`, validation OK) + bug-propagates arm (D5); `SignaturesNotAllowedError` crosses as structured `ErrorReport` with offender + signature refs intact (T3) and lenient-mode counterpart; concurrent invocations isolated.+- [x] **Tier 2d added** to `.claude/skills/temporal-e2e-validate/` (scenario in `references/mode-2-tiers.md` after Tier 2c, submitter script `scripts/submit_dry_validate.py`, Step-7 master-table row) **and RUN against a real 3-process setup** (server + split router/runner workers): GREEN (exit 0, 5/5 SUCCESS, GraphSpec 5 nodes/6 edges, server history = 1 activity + 0 children, no NDJSON partition, activity ran on the runner process), best-effort sub-case (no main_pipe → `GRAPH: None`, exit 0), concurrency (2 parallel dispatches, distinct graph_ids, no cross-contamination), RED (dropping `scoped_event_log` → exit 1 `did not produce a graph spec` — the two-instance regression fails loudly; doc updated to the observed behavior; the content-generator RED arm is the Mode-1 leaf-mock test from Phase 2), restored + GREEN re-confirmed.+- [x] `make agent-check` clean · activity isolation tests green (7/7) · Tier 2d GREEN+RED.++> ### ⛔ CHECKPOINT 3 — after Phase 3 — **MANDATORY STOP**+>+> **Verify:** activity + wrapper workflow registered + isolation-tested (status map + GraphSpec correct, zero nested dispatch, in-memory tracing, best-effort graph, structured error on validation failure) · **Tier 2d (activity arm) GREEN and RED-proven** in `temporal-e2e-validate` · `make agent-test` green · Temporal e2e green · commit.+>+> **Handoff (filled in, 2026-06-09):**+>+> - **Activity:** `act_dry_validate(arg: DryValidateArg) -> DryValidateResult` in `pipelex/temporal/tprl_pipe/act_dry_validate.py`, `@activity.defn(name="act_dry_validate")` over `@convert_pipelex_errors`. `DryValidateArg{mthds_contents, allow_signatures, pipe_code}` (no `library_dirs` — dropped post-review, R-D3); `DryValidateResult{dry_run_outputs: dict[str, DryRunOutput], graph_spec: GraphSpec | None}`. **The sweep half is `validate_bundle` itself** (not `validate_current_library`) — chosen for cross-backend 422 parity: failures cross as the categorized `ValidateBundleError` (`error_domain=input`, refs in the caller-facing message), exactly what the direct route raises; the API problem document is built from these `ErrorReport` fields (the structured arrays on `ValidateBundleError` never reached the problem document even in direct mode — verified). Graph target = `arg.pipe_code or` the first blueprint's qualified `main_pipe`.+> - **Wrapper workflow:** `WfDryValidate` (`wf_dry_validate.py`, Temporal name `wf_dry_validate`) — one `workflow.execute_activity(act_dry_validate, start_to_close_timeout=5min, RetryPolicy(maximum_attempts=2, non_retryable_error_types=[ValidateBundleError]))` (post-review trim: the other deterministic names never cross — `validate_bundle` wraps them).+> - **Submitter helper (new, for Phase 4):** `dispatch_dry_validate(arg, *, task_queue=None, should_auto_connect_temporal=True)` in `tprl_pipe/dry_validate_dispatch.py` — mints the workflow id (`dry_validate_<uuid>` through `make_workflow_id`) and awaits the wrapper workflow; failures surface as `WorkflowExecutionError` carrying the recovered `ErrorReport`. The API route should call THIS.+> - **Registration:** `tasks.py` `PackName.PIPE` — `WfDryValidate` appended to `workflow_list`, `act_dry_validate` to `activity_list` (split workers pick them up by scope automatically — verified in the 3-process run: workflow on router, activity on runner).+> - **Divergences from plan:** (1) the activity body holds no scopes itself — `dry_run_pipe_in_process` owns its three, and `validate_pipes` now owns `scoped_pipe_router` + `scoped_content_generator` (the Part-B guard now also covers direct CLI sweeps, not just the activity); (2) graph "no main_pipe and no pipe_code" → `graph=None` silently (no error), matching the route's separate main-pipe precondition; (3) ~~the event-log-drop RED fails loudly (exit 1)~~ post-review (R-D1 parity) the event-log-drop regression degrades to `GRAPH: None` in the activity — the loud CI guard is the Mode-1 `test_dry_run_graph_in_process.py` direct-call assertions; Tier-2d doc updated.+>+> **Next: Phase 4.**++---++## Phase 4 — API dispatch (cross-repo `pipelex-api`)++- [x] `../pipelex-api/api/routes/pipelex/validate.py` split into `_validate_direct` (today's body, unchanged) + `_validate_via_temporal`, gated on `get_config().temporal.is_enabled`. Temporal branch: **dispatch FIRST** (before any API-side parsing, so every validation failure surfaces through the worker's `validate_bundle` cascade with the identical categorized `ValidateBundleError` 422 — `WorkflowExecutionError.to_error_report()` returns the recovered report and the existing global handler renders it; zero route-side catch), then re-parse blueprints for the envelope, check the `main_pipe` precondition, and build `pipe_structures` from a local **load-only** `acquire_library` (no sweep, no dry-run, no tracing API-side). Best-effort graph rides back as `graph_spec` on the activity result.+- [x] Both backends tested (`tests/unit/test_validate_temporal_dispatch.py`): Temporal-enabled returns the envelope from one round-trip (graph + blueprint + structures, dispatch awaited exactly once with the request's contents/allow_signatures), best-effort `graph_spec=null` → 200, validation failure renders the same 422 (`ValidateBundleError`/`input`, refs in `detail`), and direct mode **never** touches the dispatch helper. Existing direct-path tests (`test_allow_signatures` etc.) green against the refactored route.+- [x] **Tier 2d API arm RUN live** (real route + real dispatch via TestClient in the pipelex-api venv, against the running server + split workers): 200 with worker-assembled `graph_spec` (3 nodes/3 edges) in one round-trip; signature bundle → 422 `ValidateBundleError`/`input` through the real activity→workflow→submitter→handler chain; **found+fixed in the process:** the config-default workflow retry policy re-ran a deterministically-failed validation (same workflow id listed twice) — `dispatch_dry_validate` now pins `RetryPolicy(maximum_attempts=1)` on the wrapper workflow (D-C5 at the workflow tier), re-verified (failed workflow appears exactly once). API-arm procedure added to the Tier-2d skill scenario.+- [x] Docs + CHANGELOG `[Unreleased]` in both repos (pipelex: tracing doc + temporal-integration doc + CHANGELOG, done in Phases 1–3; pipelex-api: `docs/pipe-validate.md` "Execution Backends" section + CHANGELOG entry noting the pipelex version requirement).++> ### ⛔ CHECKPOINT F — after Phase 4 — **ALL REQUIREMENTS MET**+>+> req 2 (production dry-run + validation as one in-process Temporal activity, in-memory tracing) · req 3 (direct in-process unchanged). req 1 is delivered by Part B, not here.+>+> **Verify:** `pipelex` `make agent-test` green · Temporal e2e green · **Tier 2d (activity + API arms) GREEN** · `pipelex-api` tests green on **both** backends · commit.+>+> **Handoff (filled in, 2026-06-09) — as-built summary:**+>+> - **Phase 1** — `pipelex/hub.py`: `scoped_event_log` / `get_event_log_override` (ContextVar scope); write side `pipeline_run_setup.py` and read side `tracing_assembly.py::assemble_tracing` prefer the override (override implies tracing-enabled, read side doesn't `close()` it). The two-instance fix.+> - **Phase 2** — `pipelex/pipe_run/dry_run_pipeline.py::dry_run_pipe_in_process(pipe, *, library_id)`: graph dry-run with zero dispatch under the three scopes (`scoped_event_log` + `scoped_pipe_router` + new `hub.scoped_content_generator`, which `get_content_generator()` prefers); tracer keys aligned by construction.+> - **Phase 3** — `pipelex/temporal/tprl_pipe/act_dry_validate.py` (`DryValidateArg`/`DryValidateResult`, activity composes `validate_bundle` + `dry_run_pipe_in_process`, D4 load-once, D5 narrow best-effort graph), `wf_dry_validate.py::WfDryValidate` (one-step wrapper, D-C5 activity retry bounds), `dry_validate_dispatch.py::dispatch_dry_validate` (submitter helper, `maximum_attempts=1` at the workflow tier), registered in `tasks.py`; `BundleValidator.validate_pipes` also scopes the dry content generator. Tier 2d (skill scenario + `submit_dry_validate.py` + master-table row) GREEN+RED on a real 3-process stack.+> - **Phase 4** — `pipelex-api` route split (`_validate_direct` unchanged / `_validate_via_temporal` dispatch-first), unit tests for both backends, live API-arm verification, docs+CHANGELOG both repos.+> - **How the API dispatches now:** `dispatch_dry_validate(DryValidateArg(mthds_contents, allow_signatures))` → `wf_dry_validate` → one `act_dry_validate` on the worker → `{dry_run_outputs, graph_spec}` in one round-trip; failures cross as `WorkflowExecutionError` carrying the recovered `ValidateBundleError` report → existing 422 handler.+> - **Cross-repo shipping note:** the `pipelex-api` branch (`feature/Update-dry-run-api`) imports `pipelex.temporal.tprl_pipe.act_dry_validate` — it needs a pipelex release/rev **newer than v0.32.1** (bump the `pipelex` pin + `[tool.uv.sources]` rev) before its CI can pass; local verification used `uv pip install -e ../_dry`.+> - **Follow-ups (deferred):** retire the old worker-workflow graph path for `/validate` (now unused by the API; `dry_run_pipeline` still serves the direct CLI/API path) · Phase G0 standalone activity (`temporalio` bump) · D-plan §7 endpoint unification · consider moving `pipe_structures` into the activity result to drop the API-side load-only library acquisition (today it double-loads: worker + API).+>+> Folded into `wip/dry-run-refactor/consolidation-as-built.md` (§ "Part C as built") and the README workstream table.-## The fix as shipped+----### Part 1 — scope the in-process router for the whole sweep (commit `3377babb`)+## Phase G0 — *(optional, deferred — do AFTER Checkpoint F, only if wanted)* true standalone activity-The codebase had already solved this exact leak for DIRECT mode in `pipelex/runtime_bridge/bridge.py::_run_direct`, with a comment describing the precise failure mode. `BundleValidator` now applies the same guard around its dry-run sweep:+The wrapper workflow ships the feature with no SDK bump. Replacing it with a true standalone activity is a pure runtime optimization (one fewer workflow per call), not a requirement — do it later, on its own branch/PR, only if the round-trip cost matters. Independent regression surface (the SDK underpins the whole worker/runtime).-1. `__init__` keeps the router instance instead of discarding it inside `PipeRun(...)`:- ```python- self._pipe_router = PipeRouter(observer=ObserverNoOp())- self._pipe_run: PipeRunProtocol = PipeRun(pipe_router=self._pipe_router)- ```-2. `validate_pipes` wraps the **per-pipe dry-run sweep loop** (step 4) in `scoped_pipe_router`:- ```python- with scoped_pipe_router(self._pipe_router):- for pipe in sweepable_pipes:- results[pipe.pipe_ref] = await self._classify_pipe(...)- ```- Wrapping `validate_pipes` covers all entry points (`acquire_and_validate`, `validate_current_library`, and the direct `validate_bundle` path all funnel through it). Steps 1–3 (wiring check, signature pre-pass, telemetry) don't run pipes, so they don't need the scope.+- [ ] Confirm the target `temporalio` version (past `1.23.0`) and that our Temporal Cloud / server supports standalone-activity execution. *(This is the only step that needs infra input — and it's no longer blocking anything.)*+- [ ] Bump `temporalio` in `pyproject.toml` (+ `uv.lock` via `uv`); run the **full** Temporal e2e suite on the new SDK (`temporal-e2e-validate`) as a separately-reviewable step.+- [ ] Swap the API dispatch from the wrapper workflow to the standalone activity; keep the wrapper-workflow path as fallback if useful.-**Why it's safe:**+----- `scoped_pipe_router` is **contextvar-scoped** and restores the prior value on exit, so concurrent `/validate` requests don't leak into each other.-- `PipeBatch` fans branches out via `gather_bounded` → `asyncio.gather`, whose Tasks copy the current context at creation (and creation happens inside the `with` scope), so each branch task inherits the override and resolves the in-process router too.+## Distributed verification — `temporal-e2e-validate` (Mode 1 / Tier 2d)-### Part 2 — `--temporal/--no-temporal` flag on `validate` (commit `b8eadeda`)+Mode 1's contract must be proven in a **real distributed context**, not only unit/integration. Add a tier to the repo's own skill at `.claude/skills/temporal-e2e-validate/`, following the **Tier 2c precedent** (#976 added: a scenario in `references/mode-2-tiers.md` Step 3, a Mode-1 pytest, and a Step-7 master-table row). Build it as part of Phase 3 (activity arm) and Phase 4 (API arm).-A `--temporal/--no-temporal` flag was added to `pipelex validate bundle`, `pipe`, and `method` (and therefore `validate --all`, which routes to `validate pipe`), giving parity with `pipelex run`. It threads through `_validate_core.execute_validate(temporal=...)` → `make_pipelex_for_cli(temporal_enabled=...)`, overriding `temporal.is_enabled` **for the boot only**.+**Tier 2d — dry-run + validation runs as ONE in-process, in-memory activity.** Sibling to Tier 2c: where 2c proves the *direct* `/validate` sweep doesn't leak to Temporal, 2d proves the *Temporal-dispatched* path runs the whole sweep **+** graph dry-run inside a single activity, in memory, returning `{status map, GraphSpec}`.-**This flag is behavior-neutral for validation today.** The sweep always runs in-process regardless of the flag (Part 1 guarantees that). The flag does not change *what* validation does — it controls *how Pipelex boots*, which is the lever for exercising the "validation stays in-process even on a Temporal-enabled hub" contract without juggling a `pipelex_temporary_override.toml`. It is also forward-looking: once validation runs as a standalone Temporal activity ([`wip/dry-run-refactor/followup-temporal-validation-activity.md`](wip/dry-run-refactor/followup-temporal-validation-activity.md)), `--temporal` stops being a no-op and becomes the switch that dispatches the sweep through that activity.+**Mode 2 (3-process) GREEN** — boot Temporal-enabled, split workers up (`mode-2-setup.md`); dispatch the wrapper-workflow→`act_dry_validate` over a controller bundle (`temporal_parallel.mthds` — interesting graph):-## Reviewer note — the graph step was deliberately NOT changed+- exit 0; per-pipe status map present (all `SUCCESS`); non-empty `GraphSpec` → `reactflow.html` assembles.+- **Strong check (the point):** during the run the worker ran the wrapper workflow + exactly one `act_dry_validate` and **nothing else** — NO child `WfPipeRouter`/`WfPipeRun`, NO `act_llm_gen_*`, NO `act_assemble_tracing`/`act_flush_trace_events`. Capture both worker sessions and grep — expect none of those for this run. (Mirrors Tier 2c's worker-idle check, but here the activity itself is expected; what must be absent is everything *nested*.)+- **In-memory tracing:** no new NDJSON partition appears under `.pipelex/traces/` for the activity's internal graph dry-run, and no DynamoDB write — the `GraphSpec` rode back on the activity result, assembled from the in-memory log.+- **No usage/cost:** no cost table, no `usage_report` events.+- **Best-effort graph sub-case:** a bundle whose graph dry-run fails (a pipe needing un-mockable input) still returns exit 0 with the status map and `graph=None` (no `reactflow.html`); validation still OK.+- **Concurrency:** two concurrent dispatches return distinct `GraphSpec`s with no shared/merged trace events.-An earlier diagnosis claimed the best-effort **graph** step in `validate.py` — `dry_run_pipeline()` (`pipelex/pipe_run/dry_run_pipeline.py`) → `PipelexRunner(...).execute_pipeline(...)` — had "the same leak." **It does not**, and an attempt to "fix" it was **reverted**. The graph step does a **single** top-level dispatch of the main pipe — no concurrent same-id collision — so under Temporal it dispatches the one workflow to the worker, which (tracing enabled, `backend = "temporal_dynamodb"`) assembles the `GraphSpec` and returns it on `PipeOutput`. That is the intended distributed design: `pipelex-api` runs with tracing disabled (thin submitter); the **worker** owns tracing + graph assembly. Forcing it in-process broke graph generation (tracing off in the API → empty `GraphSpec` → graph dropped). **Do not "fix" the graph step to run in-process** — if a `/validate` graph is wanted without a worker, the lever is enabling tracing in the API, a separate deployment decision.+**Mode 2 RED (prove it bites):** in the activity body, drop `scoped_content_generator(inline)` → the leaf reaches the hub `ContentGeneratorInWorkflow` and the activity tries to dispatch `act_llm_gen_*` from inside an activity (illegal) / the strong check shows dispatch; **or** drop `scoped_event_log` → the `GraphSpec` comes back empty (the two-instance regression). Restore immediately.-## Test coverage — all three layers in place+**Mode 1 (pytest) companion (CI-cheap)** — `tests/integration/pipelex/temporal/test_dry_validate_activity_in_memory.py`: run the wrapper-workflow→activity against the in-process server; spy `WorkflowExecutor.execute_activity`/`execute_workflow` and assert no nested dispatch during the activity; assert the event-log backend received no writes (in-memory only); assert best-effort graph returns `graph=None` on a failing graph dry-run.-The contract — *the validation sweep never dispatches to Temporal* — is guarded at three complementary layers (none redundant):+**Step-7 master-table row to add:** `Tier 2d: Dry-run+validate as one in-memory activity | the Temporal-dispatched /validate runs the whole sweep + graph dry-run inside ONE in-process activity (zero nested dispatch), traces the graph in memory (no NDJSON/DDB), returns {status, GraphSpec}; best-effort graph → None on failure | PASS/FAIL | path | — `.-| Layer | Where it runs | CI-automated | What it adds | Status |-|---|---|---|---|---|-| **Sentinel regression test** (no Temporal) | `tests/integration/pipelex/pipeline/test_bundle_validator.py::TestBundleValidatorIntegration::test_standalone_batch_sweep_scopes_in_process_router` | ✅ | Installs a *raising* hub-default router; sweeps a standalone `PipeBatch`; asserts SUCCESS with the hub default never touched. Deterministic, zero Temporal infra. | exists |-| **Mode-1 pytest** (real Temporal router) | `tests/integration/pipelex/temporal/test_validate_sweep_stays_in_process.py::TestValidateSweepStaysInProcess` | ✅ | Resolves the **real** `TemporalPipeRouter` as the hub default (proving the config→hub-default *wiring* the sentinel bypasses); spies `WorkflowExecutor.execute_workflow` and asserts it's never called. | **new — this change** |-| **Mode-2 scenario** (3 separate processes) | `temporal-e2e-validate` skill, Tier 2c (agent-run) | ✗ | Production-faithful API↔worker boundary: `validate bundle --temporal` over a standalone `PipeBatch` exits 0 **and** the worker stays idle (no `WfPipeRouter` dispatch). Documents the GREEN/RED procedure. | **new — this change** |+## GSTACK REVIEW REPORT-The Mode-1 pytest is the cheaper automated catch (fast, deterministic, runs in CI); the Mode-2 scenario is the deployment-faithful demonstration and the only one that also exercises the genuinely cross-process surfaces (LibraryCrate propagation, serialization). Both were verified RED with the `scoped_pipe_router` scope removed and GREEN with it in place.+_Eng review (FULL) 2026-06-09 · commit 6434b6e09 · outside voice: Codex._-The repro bundle reuses an existing fixture (no new fixture): `tests/integration/pipelex/temporal/library_crate/temporal_batch.mthds` already declares a standalone `type = "PipeBatch"` pipe (`batch_temporal_describe_topics`) whose direct sweep fans out — the exact shape that 422'd.+| Review | Trigger | Why | Runs | Status | Findings |+|--------|---------|-----|------|--------|----------|+| CEO Review | `/plan-ceo-review` | Scope & strategy | 0 | — | — |+| Codex Review | `/codex review` | Independent 2nd opinion | 1 | issues_found | 4 substantive (C1/C2/C3 resolved, C5 build-now), C6/C7 folded to tests |+| Eng Review | `/plan-eng-review` | Architecture & tests (required) | 1 | issues_open | 7 issues, 1 critical-gap-class (T3) |+| Design Review | `/plan-design-review` | UI/UX gaps | 0 | — | — |+| DX Review | `/plan-devex-review` | Developer experience gaps | 0 | — | — |-## Files in this change+- **CODEX:** caught the Phase-2 `bridge DIRECT` flaw (cannot carry `run_mode=DRY` + `mock_inputs` — verified at `bridge.py:216`), the output-contract contradiction, and the library-lifecycle gap — all missed by the in-review reasoning, all folded as decisions.+- **CROSS-MODEL:** the review and Codex agree the plan's foundation (scoped in-memory event log, one-activity dispatch via wrapper workflow, best-effort graph) is sound. The one tension — Phase 2 seam — resolved in Codex's favor (`prepare_pipe_job`, not the bridge), verified in code.+- **VERDICT:** ENG CLEARED — findings folded as decisions; T1–T10 backlogged (tasks JSONL in gstack project dir). Scope accepted as-is (Step 0). Lake score 8/8. Implement with the decisions below.-Owned by **this** change (the rest of the diff belongs to PR #974 — see [Stacking](#fix-validate-dry-run-leaked-nested-controller-sub-pipes-to-temporal)):+**Decisions locked in this review (amend the phases accordingly):**-- `pipelex/pipeline/bundle_validator.py` — the fix (`__init__` keeps `self._pipe_router`; `validate_pipes` wraps the sweep in `scoped_pipe_router`).-- `pipelex/cli/commands/validate/{_validate_core,bundle_cmd,method_cmd,pipe_cmd}.py` — the `--temporal/--no-temporal` flag.-- `tests/integration/pipelex/pipeline/test_bundle_validator.py` — sentinel regression test (+ siblings).-- `tests/integration/pipelex/temporal/test_validate_sweep_stays_in_process.py` — **new** Mode-1 pytest.-- `tests/unit/pipelex/cli/test_validate_temporal_flag.py` — locks the CLI surface (each subcommand exposes `--temporal/--no-temporal`; `--help` short-circuits before boot).-- `.claude/skills/temporal-e2e-validate/references/mode-2-tiers.md` — **new** Tier 2c scenario (+ Contents + Step 7 row).-- `CHANGELOG.md` — entry under `[Unreleased]`.-- `TODOS.md`, `wip/distributed-execution/validate-sweep-temporal-leak-repro.md` — this guide + the repro brief.+- **D1 — Phase 1 `is_enabled`:** a set `scoped_event_log` override **implies tracing-enabled** at all three guards (`pipeline_run_setup.py:199`, `tracing_assembly.py:84` early-return, `:95`). Regression test: `is_enabled=False` + override → GraphSpec still assembles in-memory. (T2)+- **D2 — Phase 1 write side:** keep **both** `make_event_log` patches for a symmetric primitive (write side is off this feature's bridge path but works for any `execute_pipeline` caller); test the full path. (T7)+- **D-C1 — Phase 2 seam:** use **`prepare_pipe_job`** (`pipe_run_mode=DRY`, `mock_inputs=True`, `generate_graph=True`) + a **local `PipeRun`** under the three scopes — **not** the bridge (the bridge runs LIVE/no-mock). (T1)+- **D3 — Phase 3 output contract:** output = `{status map (success only), GraphSpec | None}`. Unexpected pipe failures + signature refusals **raise** → `ErrorReport` → API 422. Drop the `signature_check_error` output field. **Mandatory: verify the structured offending-signatures survive the `ErrorReport` crossing** (else 422 degrades to a plain message — the one critical-gap-class). (T3)+- **D4 — Phase 3 library lifecycle:** **load once**, run sweep **and** graph against the same `library_id` (borrow-the-open-library inner sweep, no per-step teardown), tear down once in the activity's `finally`. (T4)+- **D5 — Phase 3 best-effort catch:** **narrow** to the specific dry-run-failure `PipelexError` subclasses → `graph=None`; all other `PipelexError`s and every non-`PipelexError` bug **propagate** (mirrors `assemble_tracing:69`). Test the bug-propagates arm. (T5)+- **D-C5 — Phase 3 retry/timeout:** set explicit `start_to_close_timeout` + `non_retryable_error_types` on `act_dry_validate` so deterministic failures don't retry 3×. (T6)+- **D6 — tests:** harden the Mode-1 spy to assert at the `workflow.execute_activity` SDK site / Temporal history, not the `WorkflowExecutor` wrapper (T8); forbid divergent `tracer_key` (T9).+- **Dismissed:** activity-result payload-size guard — existing S3 payload offload already handles large results.-## How to verify (local setup)+NO UNRESOLVED DECISIONS-API on :8081 (`make run`), Temporal server + worker up (from `pipelex-worker`), both on the editable `pipelex` checkout.+### Post-implementation /review (2026-06-09, on the completed branch)-```bash-# was failing → now 200 with validated bundle-make bundle-validate BUNDLE=/Users/lchoquel/repos/Pipelex/pipelex-demos/mthds-wip/fashion_moodboard-# regression guard → still 200-make bundle-validate BUNDLE=/Users/lchoquel/repos/Pipelex/pipelex-demos/mthds-wip/joke_judge-```+Pre-landing review (structured pass + specialist subagents + Claude adversarial + Red Team + Codex adversarial + Codex structured review, all on commit 6ba8d8137). Codex structured: clean. Four decisions taken and applied on-branch:-CLI parity check (no override file needed): `pipelex validate bundle <bundle> --temporal` boots Temporal-enabled yet the sweep stays in-process — exits 0 with no workflow dispatched to the worker. See [`wip/distributed-execution/validate-sweep-temporal-leak-repro.md`](wip/distributed-execution/validate-sweep-temporal-leak-repro.md) for the full GREEN/RED procedure (now implemented as Tier 2c in the `temporal-e2e-validate` skill).+- **R-D1 — graph-arm cross-backend parity (supersedes D5's narrow catch).** The direct route catches base `PipelexError` around the graph dry-run (best-effort for ANY domain failure); the activity's narrow tuple made the same bundle 200 on direct and 422-after-retry on Temporal. Widened to `(PipelexError, ValidationError, FactoryException)` — the `_classify_pipe` catch — with pipe resolution moved inside (unknown `pipe_code` → `graph=None`). `WfDryValidate.non_retryable_error_types` trimmed to `["ValidateBundleError"]` (the only deterministic name that actually crosses). Consequence: the event-log-drop regression degrades to `GRAPH: None` in the activity; the loud CI guard is the Mode-1 direct-call test.+- **R-D2 — cheap hardening.** Activity `finally` mirrors `PipeRun.run`'s suppress-secondary pattern (a teardown raise no longer replaces the user's real error); `dispatch_dry_validate` pins `workflow_execution_timeout=12min` (interactive bound, not the 1-hour batch default); `scoped_event_log` lifecycle docs corrected (the write-side tracer DOES `close()` its log at teardown — a scoped log's `close()` must be idempotent/no-op); `dry_run_pipe_in_process` constructs its primitives before `open_tracer` (no tracer-entry leak window). Design-tradeoff items deferred to [`wip/dry-run-refactor/followup-dry-validate-hardening.md`](wip/dry-run-refactor/followup-dry-validate-hardening.md).+- **R-D3 — `DryValidateArg.library_dirs` dropped.** No caller passed it; it was an unconstrained worker-filesystem read surface. Re-add with allowlist confinement if a real caller appears.+- **R-D4 — test lakes added.** No-retry assertion (D-C5 pinned), no-main-pipe arm, unknown-pipe_code parity arm, `scoped_content_generator` unit tests, restore-prev-library direct-call test, `dispatch_dry_validate` retry/timeout unit test.diff --git a/docs/agents/debugging-hanging-pytest-runs.md b/docs/agents/debugging-hanging-pytest-runs.mdindex 737229e1f..3cf8c7ae4 100644--- a/docs/agents/debugging-hanging-pytest-runs.md+++ b/docs/agents/debugging-hanging-pytest-runs.md@@ -161,3 +161,4 @@ In all four cases, write up what you know in 3–4 sentences and surface to the- `boot_temporal` fixture teardown (in `tests/integration/pipelex/temporal/conftest.py`) is the known fixture-teardown hang point. If a hang traceback mentions `manager.teardown()` or `temporal_hub.reset()`, it's this.- xdist worker crashes on integration temporal tests often show as `[gw N] node down: Not properly terminated` followed by `replacing crashed worker`. Treat as infrastructure flake until you've proven otherwise serially.- `is_in_temporal_workflow()` is the right gate for code that might be invoked from inside a Temporal sandbox — `get_config()` is unsafe to read from inside, and many singletons return sandboxed-empty copies. This was the actual bug I hit; it would have been caught instantly by the playbook above if I'd run the integration tests serially on first iteration instead of trusting the xdist run.+- **A payload-conversion error inside a workflow task looks like a clean hang, not a failure.** When the data converter raises while decoding an activity result (or workflow argument), Temporal fails the *workflow task* and retries it indefinitely — the submitter's `execute_workflow` never returns, pytest-timeout's thread-method kill fires, and xdist reports `node down` with the real traceback destroyed along with the process. Historical trigger: kajson ≤ 0.6.0 decoded tz-aware datetimes via `ZoneInfo(...)`, which needs the IANA tz database — absent on slim containers and uv-managed standalone Pythons. Fixed upstream in kajson 0.7.0 (self-sufficient timezone wire format + `tzdata` declared as kajson's own dependency), but any other deterministic decode error (registry miss, schema drift) still presents as this same hang. To diagnose this class of hang on a remote runner, set `PIPELEX_HANG_DUMP_DIR=<dir>`: the temporal integration conftest arms a per-test watchdog that dumps all thread stacks to a file before the kill AND mirrors temporalio's failure logs (which carry the workflow-task traceback) into the same directory; the tests workflow prints the directory on job failure. To reproduce a missing-tz-database environment locally, run with `PYTHONTZPATH=/nonexistent` after uninstalling `tzdata`.diff --git a/docs/building-methods/pipes/executing-pipelines.md b/docs/building-methods/pipes/executing-pipelines.mdindex 5e2b1c23d..a7b0163ac 100644--- a/docs/building-methods/pipes/executing-pipelines.md+++ b/docs/building-methods/pipes/executing-pipelines.md@@ -232,3 +232,9 @@ Pipelex has a REST API for executing pipelines. See more about it [here](https:/<!-- start_pipeline exists on PipelexRunner as a protocol stub (raises NotImplementedError).The public entry point is execute_pipeline(). Keep this note so reviewers don't flag the stub as contradictory. -->The public Python entry point is `PipelexRunner.execute_pipeline()`. The older standalone `start_pipeline` function is no longer a documented entry point.++---++## Run Modes & Backends++Every execution method above accepts a run mode (live or dry run) and a backend (in-process or Temporal). See [Run Modes & Backends](run-modes-and-backends.md) for the full matrix and when to use each combination.diff --git a/docs/building-methods/pipes/run-modes-and-backends.md b/docs/building-methods/pipes/run-modes-and-backends.mdnew file mode 100644index 000000000..b7799a6c3--- /dev/null+++ b/docs/building-methods/pipes/run-modes-and-backends.md@@ -0,0 +1,86 @@+---+description: "All the ways to run a Pipelex pipeline: live or dry-run — in-process or distributed on Temporal. The run mode and the execution backend are independent axes."+---++# Run Modes & Backends++Pipelex separates **what the inference leaves do** (the run mode) from **where the work executes** (the backend). The two axes are independent: any run mode works on any backend, because the run mode travels with every inference job and is honored at the leaf — the last stop before a provider would actually be called.++## Axis 1: Run Mode++| Run mode | CLI trigger | Python trigger | AI providers | Usage & cost report |+|---|---|---|---|---|+| **Live** (default) | — | — | Real calls | Real usage, report rendered |+| **Dry run** | `--dry-run` | `pipe_run_mode=PipeRunMode.DRY` | Never called — leaves build mocks | Zero tokens, report suppressed |++**Live** runs call the configured AI providers for real: real cost, real latency, and storage IO for generated images and extracted pages.++**Dry run** exercises the full pipeline — controllers, working memory, data flow, execution-graph tracing — with no AI cost. Each inference leaf branches to a format-compliant mock instead of calling the provider (see [Dry Run Mock Generation](../../under-the-hood/dry-run-mock-generation.md)). A dry run needs no API keys, performs no storage IO (mocked images and pages never touch the storage provider), reports zero-token usage, and suppresses the end-of-run cost report. Add `--mock-inputs` to also synthesize any missing required inputs (it requires `--dry-run`).++## Axis 2: Backend++| Backend | Trigger | What runs where |+|---|---|---|+| **Direct** (default) | — | Everything in your process |+| **Temporal** | `--temporal`, or config `[temporal] is_enabled = true` | Controllers as child workflows, inference leaves as activities on workers |++The run mode rides across the wire with every inference job, so the backend never changes *what* a leaf does — only *where* it does it. In particular, when you run a **pipeline** in dry mode on Temporal, the real inference activities are still dispatched and mock **inside** them on the worker. That's by design: it lets you test the entire distribution machinery — dispatch, scheduling, serialization, task-queue routing, cross-worker graph tracing — with zero AI spend and no credentials.++One job type deliberately opts out of this fan-out: a **validation sweep** submitted through the runner API runs as a single in-process activity instead of dispatching nested activities — see [Validation Sweeps](#validation-sweeps) below for how that fits the model.++## The Matrix++All cells below describe **pipeline runs** — `pipelex run ...` or `PipelexRunner.execute_pipeline()`. Validation sweeps have their own distribution shape, covered in the next section.++| Run mode \ Backend | Direct (in-process) | Temporal |+|---|---|---|+| **Live** | Standard local run. | Durable distributed run; real inference executes on workers. |+| **Dry run** | Leaves mock inline. Validates pipeline logic and data flow at zero cost. | Activities are genuinely dispatched and mock inside the worker — proves out the distribution machinery with no API keys, no spend, no storage IO. |++```bash+# Live + direct (the default)+pipelex run bundle my_bundle.mthds++# Dry run + direct+pipelex run bundle my_bundle.mthds --dry-run --mock-inputs++# Live + Temporal+pipelex run bundle my_bundle.mthds --temporal++# Dry run + Temporal — dispatches real activities, mocks inside them+pipelex run bundle my_bundle.mthds --temporal --dry-run --mock-inputs+```++From Python, the same knobs live on the runner — the backend comes from config:++```python+from pipelex.pipe_run.pipe_run_mode import PipeRunMode+from pipelex.pipeline.runner import PipelexRunner++runner = PipelexRunner(+ bundle_uri="path/to/my_bundle.mthds",+ pipe_run_mode=PipeRunMode.DRY,+)+response = await runner.execute_pipeline()+```++## Validation Sweeps++`pipelex validate` (see the [validate commands](../../tools/cli/validate.md)) combines static validation with a dry run of every pipe. It relates to the matrix in two specific ways:++- **The CLI sweep always runs in-process**, even when Temporal is enabled. The `--temporal` flag on `pipelex validate` exists precisely to verify that: it flips the boot config so you can confirm a validation sweep does not dispatch anything under a Temporal-enabled setup.+- **The runner API's `/validate` route ships the sweep to a worker — as one activity**: with Temporal enabled, it offloads the *whole* sweep as a single activity (`act_dry_validate`, wrapped in the `WfDryValidate` workflow). Inside that activity, the pipe router and the content generator are scoped to their in-process variants, so nothing nested is dispatched: every dry run in the sweep behaves exactly like the dry × direct cell of the matrix, just hosted on a worker instead of your machine. The per-pipe statuses plus the execution graph come back in one round-trip. See [Temporal integration](../../under-the-hood/temporal-integration.md) for the mechanics.++This is not a contradiction of "the run mode never changes the backend" — it's the same leaf-mock foundation with a different *backend choice* for the nested work. A `--temporal --dry-run` pipeline run picks the Temporal backend all the way down, fanning out for real to test the distribution machinery. A Temporal validation sweep picks the Temporal backend for exactly one hop (get the sweep onto a worker) and the in-process backend for everything inside, because its goal is a cheap interactive answer, not distribution testing. Two deliberate opposites built from the same parts.++## Forced Dry Mode++When a process boots without inference configured — `Pipelex.make(needs_inference=False)`, e.g. no AI provider set up — every run that process initiates is forced to dry-run mode. This is a property of the run, not of the backend: under a Temporal-enabled boot the forced-dry run still dispatches activities and mocks inside them, like any other dry run. A warning logs whenever a live-requested run is forced dry.++## See Also++- [Validation & Dry Run](../../features/validation-dry-run.md) — feature overview+- [Dry Run Configuration](../../configuration/config-pipeline-validation/dry-run-config.md) — mock list sizes, allowed-to-fail pipes+- [Cost Tracking & Reporting](../../features/cost-tracking.md) — what the usage reports contain+- [Distributed Execution](../../distributed-execution/index.md) — running on Temporal+- [Dry Run Mock Generation](../../under-the-hood/dry-run-mock-generation.md) — how format-compliant mocks are builtdiff --git a/docs/configuration/config-pipeline-validation/dry-run-config.md b/docs/configuration/config-pipeline-validation/dry-run-config.mdindex 8d4625263..d6de31f9e 100644--- a/docs/configuration/config-pipeline-validation/dry-run-config.md+++ b/docs/configuration/config-pipeline-validation/dry-run-config.md@@ -10,7 +10,6 @@ The `DryRunConfig` class controls how Pipelex behaves during dry runs.```pythonclass DryRunConfig(ConfigModel):- apply_to_jinja2_rendering: booltext_gen_truncate_length: intnb_list_items: intnb_extract_pages: int@@ -20,7 +19,6 @@ class DryRunConfig(ConfigModel):### Fields-- `apply_to_jinja2_rendering`: When true, simulates Jinja2 template rendering during dry runs- `text_gen_truncate_length`: Maximum length of generated text during dry runs- `nb_list_items`: Number of items to generate for list content during dry runs- `nb_extract_pages`: Number of pages to simulate for OCR operations during dry runs@@ -31,7 +29,6 @@ class DryRunConfig(ConfigModel):```toml[pipelex.dry_run_config]-apply_to_jinja2_rendering = truetext_gen_truncate_length = 100nb_list_items = 3nb_extract_pages = 2@@ -41,14 +38,7 @@ allowed_to_fail_pipes = ["optional_pipe", "experimental_pipe"]## Dry Run Behavior-### Template Rendering--When `apply_to_jinja2_rendering` is true:--- Templates are processed but not actually rendered-- Variables are validated-- Template syntax is checked-- No actual content is generated+During a dry run, no AI provider is ever called and no storage IO is performed: each inference leaf (LLM text/object generation, image generation, document extraction, web search, templating) produces a synthetic mock instead. Jinja2 templates are still parse-checked, so a broken template fails loudly even in a dry run.### Text Generation@@ -82,7 +72,6 @@ The `text_gen_truncate_length` controls:- Use dry runs for testing before production- Set appropriate truncation lengths-- Enable template validation when testing templates- Review dry run logs for potential issues## Related Documentationdiff --git a/docs/errors/dry-run-graph-not-produced-error.md b/docs/errors/dry-run-graph-not-produced-error.mdnew file mode 100644index 000000000..c98acafea--- /dev/null+++ b/docs/errors/dry-run-graph-not-produced-error.md@@ -0,0 +1,21 @@+---+title: "Dry run graph not produced"+description: "Reference for the `DryRunGraphNotProducedError` Pipelex error class."+---++<!-- pipelex:generated -->++# Dry run graph not produced++Raised when a graph-producing dry run completes but no ``GraphSpec`` was assembled onto the pipe output.++| Field | Value |+|---|---|+| `error_type` | `DryRunGraphNotProducedError` |+| `title` | Dry run graph not produced |+| `type_uri` | `https://docs.pipelex.com/latest/errors/dry-run-graph-not-produced-error/` |+| `error_domain` | _(inherited from parent)_ |+| Defined in | `pipelex.pipe_run.exceptions` |+| Parent class | [`DryRunError`](dry-run-error.md) |++[Back to Error Reference](index.md)diff --git a/docs/errors/dry-run-mock-build-error.md b/docs/errors/dry-run-mock-build-error.mdnew file mode 100644index 000000000..0a5f5e700--- /dev/null+++ b/docs/errors/dry-run-mock-build-error.md@@ -0,0 +1,21 @@+---+title: "Dry run mock build"+description: "Reference for the `DryRunMockBuildError` Pipelex error class."+---++<!-- pipelex:generated -->++# Dry run mock build++Raised when building a leaf mock object fails (polyfactory build or model validation).++| Field | Value |+|---|---|+| `error_type` | `DryRunMockBuildError` |+| `title` | Dry run mock build |+| `type_uri` | `https://docs.pipelex.com/latest/errors/dry-run-mock-build-error/` |+| `error_domain` | `input` |+| Defined in | `pipelex.cogt.content_generation.exceptions` |+| Parent class | [`PipelexError`](pipelex-error.md) |++[Back to Error Reference](index.md)diff --git a/docs/errors/dry-run-object-fidelity-error.md b/docs/errors/dry-run-object-fidelity-error.mdnew file mode 100644index 000000000..1e8e3445e--- /dev/null+++ b/docs/errors/dry-run-object-fidelity-error.md@@ -0,0 +1,21 @@+---+title: "Dry run object fidelity"+description: "Reference for the `DryRunObjectFidelityError` Pipelex error class."+---++<!-- pipelex:generated -->++# Dry run object fidelity++Raised when a dry-run synthetic object fails re-validation against its original class.++| Field | Value |+|---|---|+| `error_type` | `DryRunObjectFidelityError` |+| `title` | Dry run object fidelity |+| `type_uri` | `https://docs.pipelex.com/latest/errors/dry-run-object-fidelity-error/` |+| `error_domain` | `input` |+| Defined in | `pipelex.cogt.content_generation.exceptions` |+| Parent class | [`PipelexError`](pipelex-error.md) |++[Back to Error Reference](index.md)diff --git a/docs/errors/execution-and-runtime.md b/docs/errors/execution-and-runtime.mdindex 5952b3081..2b181ca49 100644--- a/docs/errors/execution-and-runtime.md+++ b/docs/errors/execution-and-runtime.md@@ -16,6 +16,7 @@ own page. Classes are grouped by subsystem.- [`BatchParamsError`](batch-params-error.md) — Batch params- [`DeliveryError`](delivery-error.md) — Delivery- [`DryRunError`](dry-run-error.md) — Dry run+- [`DryRunGraphNotProducedError`](dry-run-graph-not-produced-error.md) — Dry run graph not produced- [`PipeJobError`](pipe-job-error.md) — Pipe job- [`PipeRouterError`](pipe-router-error.md) — Pipe router- [`PipeRunError`](pipe-run-error.md) — Pipe rundiff --git a/docs/errors/inference-and-providers.md b/docs/errors/inference-and-providers.mdindex 31f3b19d1..876905eea 100644--- a/docs/errors/inference-and-providers.md+++ b/docs/errors/inference-and-providers.md@@ -14,6 +14,8 @@ own page. Classes are grouped by subsystem.- [`CogtError`](cogt-error.md) — AI inference failed- [`CostRegistryError`](cost-registry-error.md) — Cost registry+- [`DryRunMockBuildError`](dry-run-mock-build-error.md) — Dry run mock build+- [`DryRunObjectFidelityError`](dry-run-object-fidelity-error.md) — Dry run object fidelity- [`ExtractCapabilityError`](extract-capability-error.md) — Extract capability- [`ExtractHandleNotFoundError`](extract-handle-not-found-error.md) — Extract handle not found- [`ExtractInputError`](extract-input-error.md) — Extract input@@ -46,8 +48,6 @@ own page. Classes are grouped by subsystem.- [`LLMPromptSpecError`](llm-prompt-spec-error.md) — LLM prompt spec- [`LLMPromptTemplateInputsError`](llm-prompt-template-inputs-error.md) — LLM prompt template inputs- [`LLMSettingsValidationError`](llm-settings-validation-error.md) — LLM settings validation-- [`MockInferenceObjectFidelityError`](mock-inference-object-fidelity-error.md) — Mock inference object fidelity-- [`MockInferenceUnsupportedError`](mock-inference-unsupported-error.md) — Mock inference unsupported- [`ModelChoiceNotFoundError`](model-choice-not-found-error.md) — Model choice not found- [`ModelDeckNotFoundError`](model-deck-not-found-error.md) — Model deck not found- [`ModelDeckPresetValidatonError`](model-deck-preset-validaton-error.md) — Model deck preset validatondiff --git a/docs/errors/mock-inference-object-fidelity-error.md b/docs/errors/mock-inference-object-fidelity-error.mddeleted file mode 100644index 39f0b7be9..000000000--- a/docs/errors/mock-inference-object-fidelity-error.md+++ /dev/null@@ -1,21 +0,0 @@-----title: "Mock inference object fidelity"-description: "Reference for the `MockInferenceObjectFidelityError` Pipelex error class."------<!-- pipelex:generated -->--# Mock inference object fidelity--Raised when a ``--mock-inference`` synthetic object fails re-validation against its original class.--| Field | Value |-|---|---|-| `error_type` | `MockInferenceObjectFidelityError` |-| `title` | Mock inference object fidelity |-| `type_uri` | `https://docs.pipelex.com/latest/errors/mock-inference-object-fidelity-error/` |-| `error_domain` | `input` |-| Defined in | `pipelex.cogt.content_generation.exceptions` |-| Parent class | [`PipelexError`](pipelex-error.md) |--[Back to Error Reference](index.md)diff --git a/docs/errors/mock-inference-unsupported-error.md b/docs/errors/mock-inference-unsupported-error.mddeleted file mode 100644index df279a42c..000000000--- a/docs/errors/mock-inference-unsupported-error.md+++ /dev/null@@ -1,21 +0,0 @@-----title: "Mock inference unsupported"-description: "Reference for the `MockInferenceUnsupportedError` Pipelex error class."------<!-- pipelex:generated -->--# Mock inference unsupported--Raised when ``--mock-inference`` reaches an inference operation that has no leaf-level mock.--| Field | Value |-|---|---|-| `error_type` | `MockInferenceUnsupportedError` |-| `title` | Mock inference unsupported |-| `type_uri` | `https://docs.pipelex.com/latest/errors/mock-inference-unsupported-error/` |-| `error_domain` | `input` |-| Defined in | `pipelex.cogt.content_generation.exceptions` |-| Parent class | [`PipelexError`](pipelex-error.md) |--[Back to Error Reference](index.md)diff --git a/docs/features/validation-dry-run.md b/docs/features/validation-dry-run.mdindex 184d28304..828daf97a 100644--- a/docs/features/validation-dry-run.md+++ b/docs/features/validation-dry-run.md@@ -38,3 +38,5 @@ List specific pipe codes in `pipelex.dry_run_config.allowed_to_fail_pipes` so dr- `pipelex run pipe my_pipe --dry-run --mock-inputs` — Generate synthetic inputsFor configuration details, see [Dry Run Configuration](../configuration/config-pipeline-validation/dry-run-config.md).++Dry run is one of two run modes (live, dry run), each of which works in-process or distributed on Temporal — see [Run Modes & Backends](../building-methods/pipes/run-modes-and-backends.md) for the full matrix.diff --git a/docs/under-the-hood/distributed-content-generation.md b/docs/under-the-hood/distributed-content-generation.mdindex 2b3005fb9..649963160 100644--- a/docs/under-the-hood/distributed-content-generation.md+++ b/docs/under-the-hood/distributed-content-generation.md@@ -10,7 +10,7 @@ This page is for contributors working on Pipelex internals. If you're deployingThis page covers the serialization mechanisms that enable content generation (LLM structured output, image generation, PDF extraction, web search) to run across Temporal worker processes. For the broader Temporal architecture — LibraryCrate propagation, deferred hydration, per-workflow scoping — see [Temporal Integration](./temporal-integration.md).!!! note "Every inference leaf goes through the same seam"- All inference operators dispatch their leaf call through the swappable `ContentGenerator` abstraction: direct inline (`ContentGenerator`), as a Temporal activity (`ContentGeneratorInWorkflow`), or as a dry mock (`ContentGeneratorDry`). Web search (`PipeSearch`) joined this seam last — its `act_search_gen_*` activities make search-on-Temporal replay-safe (the result is recorded in workflow history) and let search failures cross the workflow boundary as classified errors. An operator that runs its leaf inline instead would re-execute it on every replay and, on failure, hang the submitter with an unclassified workflow-task error.+ All inference operators dispatch their leaf call through the swappable `ContentGenerator` abstraction: direct inline (`ContentGenerator`) or as a Temporal activity (`ContentGeneratorInWorkflow`); the backend choice is independent of the run mode — under `run_mode=DRY` the chosen backend still dispatches and the cogt leaf mocks inside it. Web search (`PipeSearch`) joined this seam last — its `act_search_gen_*` activities make search-on-Temporal replay-safe (the result is recorded in workflow history) and let search failures cross the workflow boundary as classified errors. An operator that runs its leaf inline instead would re-execute it on every replay and, on failure, hang the submitter with an unclassified workflow-task error.---diff --git a/docs/under-the-hood/dry-run-mock-generation.md b/docs/under-the-hood/dry-run-mock-generation.mdindex b66004e8f..07c864dce 100644--- a/docs/under-the-hood/dry-run-mock-generation.md+++ b/docs/under-the-hood/dry-run-mock-generation.md@@ -38,7 +38,7 @@ flowchart TDC --> D[DryRunFactory.make_dry_run_factory]D --> E[Polyfactory with custom providers]- F[PipeLLM dry run] --> G[ContentGeneratorDry.make_object_direct]+ F[PipeLLM dry run] --> G[cogt leaf: dry_llm_gen_object in dry_mock.py]G --> H[DryRunFactory.make_dry_run_factory]H --> E@@ -46,10 +46,12 @@ flowchart TDJ --> K[model_validate with resolved values]```+A dry run does not swap in a special content generator: `run_mode=DRY` rides `CogtRunParams` on every cogt assignment, and each inference leaf (`llm_generate`, the `*_and_store` image/extract leaves, search, templating) branches to its `dry_*` mock helper in `dry_mock.py` — identically whether the leaf runs inline or inside a Temporal activity.+| Trigger | Entry Point | Mock Generation ||---------|-------------|-----------------|-| `pipelex validate` | `dry_run_pipe()` | `WorkingMemoryFactory.make_mock_inputs()` |-| `PipeLLM` output in dry mode | `ContentGeneratorDry.make_object_direct()` | `DryRunFactory` (auto-detects from field definitions) |+| `pipelex validate` | `BundleValidator.validate_pipes()` | `WorkingMemoryFactory.make_mock_inputs()` |+| `PipeLLM` output in dry mode | `dry_llm_gen_object()` / `dry_llm_gen_object_list()` (cogt leaf, `dry_mock.py`) | `DryRunFactory` via the schema-reconstructed class || `PipeFunc` output in dry mode | `WorkingMemoryFactory.make_mock_content()` | `DryRunFactory` (with explicit field constraints) || `PipeCompose` in dry mode | `StructuredContentComposer.compose()` | Uses resolved values from working memory (no mocks) |@@ -217,14 +219,14 @@ The `factory_use_construct=True` flag bypasses `field_validator` and `model_vali### LLM Output Generation-When `ContentGeneratorDry.make_object_direct()` generates mock LLM outputs:+When the dry LLM leaf (`dry_llm_gen_object` in `dry_mock.py`) generates mock outputs, it reconstructs the output class from the JSON schema carried on the assignment (the same class on any backend), then builds via `build_mock_object()`:```python-object_factory = DryRunFactory.make_dry_run_factory(object_class)-return object_factory.build(factory_use_construct=True)+item_class = SchemaToModelFactory.make_from_json_schema(schema=..., class_name=...)+return build_mock_object(item_class)```-No explicit field constraints are passed—the factory auto-detects `MockFormat` from field definitions.+No explicit field constraints are passed — the factory auto-detects `MockFormat` from field definitions. Note that `json_schema_extra` hints can be dropped by the schema round-trip: classes with exotic format constraints should declare `examples` / `mock_format`, otherwise re-validation against the original class raises a typed `DryRunObjectFidelityError`.### PipeCompose Resolution@@ -250,7 +252,7 @@ In dry run mode, the working memory already contains properly formatted mock val| Scenario | Format Constraints | Validators Bypassed ||----------|-------------------|---------------------|| `WorkingMemoryFactory.make_mock_content()` | Yes (from `json_schema_extra` + explicit params) | Yes (`factory_use_construct`) |-| `ContentGeneratorDry.make_object_direct()` | Yes (from `json_schema_extra` only) | Yes (`factory_use_construct`) |+| `dry_llm_gen_object()` (cogt leaf) | Yes (from `json_schema_extra` surviving the schema round-trip) | No (validators run; failures raise typed `DryRunMockBuildError`) || `StructuredContentComposer.compose()` | N/A (uses resolved values) | No (validates) |---@@ -304,10 +306,10 @@ class MySpec(StructuredContent):| File | Purpose ||------|---------|| `pipelex/cogt/content_generation/dry_run_factory.py` | `DryRunFactory` class with `MockFormat` enum and generators |-| `pipelex/cogt/content_generation/content_generator_dry.py` | `ContentGeneratorDry` for mock LLM/image outputs |+| `pipelex/cogt/content_generation/dry_mock.py` | Leaf-level dry/mock helpers (`dry_llm_gen_*`, `build_mock_object`, `stamp_mock_main_coordination`) || `pipelex/core/memory/working_memory_factory.py` | `WorkingMemoryFactory.make_mock_content()` with field constraints || `pipelex/pipe_operators/compose/structured_content_composer.py` | Composes `StructuredContent` from working memory (no mocks) |-| `pipelex/pipe_run/dry_run.py` | `dry_run_pipe()` orchestration |+| `pipelex/pipe_run/dry_run_pipeline.py` | `dry_run_pipeline()` / `dry_run_pipe_in_process()` orchestration |---diff --git a/docs/under-the-hood/execution-graph-tracing.md b/docs/under-the-hood/execution-graph-tracing.mdindex 2f83b1e76..9f849a260 100644--- a/docs/under-the-hood/execution-graph-tracing.md+++ b/docs/under-the-hood/execution-graph-tracing.md@@ -83,7 +83,7 @@ graph_spec = response.pipe_output.graph_spec```!!! tip "Dry run from MTHDS content"- To dry-run an entire bundle straight from MTHDS content and get back a `GraphSpec`, use `dry_run_pipeline(mthds_contents=...)` (`pipelex/pipe_run/dry_run_pipeline.py`) — the shared entrypoint behind the CLI graph commands and the API, which wires the same DRY-mode runner for you.+ To dry-run an entire bundle straight from MTHDS content and get back a `GraphSpec`, use `dry_run_pipeline(mthds_contents=...)` (`pipelex/pipe_run/dry_run_pipeline.py`) — the shared entrypoint behind the CLI graph commands and the API, which wires the same DRY-mode runner for you. It owns its graph transport (a scoped in-memory event log), so it produces the graph regardless of the host's `tracing_config` and never writes trace files as a side effect.### Outputs@@ -249,6 +249,28 @@ manager.on_pipe_end_success(graph_spec = manager.close_tracer(pipeline_run_id)```+### Event-Log Transport and the Scoped Override++Trace events travel through an `EventLogProtocol` backend (`pipelex/tracing/`): the tracer emits events into it during the run (write side, wired in `pipeline_run_setup`), and `assemble_tracing` reads them back after the run to build the `GraphSpec` and usage aggregates (read side, triggered from `PipeRun.run`). Both sides normally build their backend instance independently from `tracing_config` via `make_event_log` — NDJSON files or DynamoDB bridge the two instances through external storage.++For fully in-process runs, `pipelex.hub.scoped_event_log` pins one shared instance for both sides instead:++```python+from pipelex.hub import scoped_event_log+from pipelex.tracing.in_memory_event_log import InMemoryEventLog++with scoped_event_log(InMemoryEventLog()):+ response = await runner.execute_pipeline(...) # graph assembles in memory+```++Semantics:++- The override is ContextVar-scoped (mirrors `scoped_pipe_router`), so concurrent runs with separate scopes never cross-contaminate, and the prior value is restored on exit.+- A set override **implies tracing-enabled**: it is honored even when `tracing_config.is_enabled` is `False`, on both the write side and the read side's early-return.+- Lifecycle: the read side does not `close()` the scoped instance and the machinery never calls `cleanup()` on it — but the write-side tracer DOES call `close()` on its event log at teardown, before the read side assembles. A scoped event log's `close()` must therefore be idempotent or a no-op (as `InMemoryEventLog`'s is); scoping a backend whose `close()` releases a real resource would break its own assembly read.++This is what lets a graph-producing dry-run trace entirely in memory (no NDJSON file, no DynamoDB round-trip). Both dry-run entrypoints in `pipelex/pipe_run/dry_run_pipeline.py` rely on it: `dry_run_pipe_in_process` (validation and graph dry-run hosted inside a single Temporal activity) and `dry_run_pipeline` itself — these functions exist to produce a graph, so they install their own scoped `InMemoryEventLog` rather than depending on the host having tracing configured (a host with `tracing_config.is_enabled = false`, like pipelex-api's `/validate` in direct mode, still gets its graph). A run that nonetheless finishes without a graph raises the typed `DryRunGraphNotProducedError`.+### TraceContext PropagationTraceContext is a serializable context that flows through pipe execution:diff --git a/docs/under-the-hood/temporal-integration.md b/docs/under-the-hood/temporal-integration.mdindex 36bb881e4..ed026c7b6 100644--- a/docs/under-the-hood/temporal-integration.md+++ b/docs/under-the-hood/temporal-integration.md@@ -214,6 +214,21 @@ Both call `ContentGeneratorInWorkflow.make_templated_text` from inside `WfPipeRo---+## Validation Dispatch: One In-Process Activity++Pipeline *execution* fans out across workflows and activities as described above — but *validation* deliberately does the opposite. When Temporal is enabled, a `/validate` job (validation sweep + graph-producing dry-run) dispatches as the one-step wrapper workflow `wf_dry_validate`, which runs a single `act_dry_validate` activity and returns `{per-pipe status map, GraphSpec | None}` in one round-trip.++Inside the activity everything is in-process and in-memory:++- The library is loaded once: the sweep half is `validate_bundle` itself (the same function the direct-mode route calls, so both backends share the categorized `ValidateBundleError` contract), which leaves the library loaded; the graph dry-run (`dry_run_pipe_in_process`) runs against that same library; teardown happens once in the activity's `finally`.+- ContextVar scopes pin the run in-process: `scoped_pipe_router` (nested controller sub-pipes resolve a local router, not the `TemporalPipeRouter`), `scoped_content_generator` (inference leaves resolve an inline dry generator, not `ContentGeneratorInWorkflow`), and `scoped_event_log` (the graph traces into a shared `InMemoryEventLog` — no NDJSON file, no DynamoDB round-trip; the `GraphSpec` rides back on the activity result). No usage/cost events are emitted.+- The graph is best-effort: an expected dry-run failure (the run-failure wrappers, or a mock-input mint failure) returns `graph_spec=None` with validation still successful; any other error propagates and fails the activity.+- Validation failures cross the boundary as structured `ErrorReport`s via the activity error boundary — a strict-mode `SignaturesNotAllowedError` keeps its offending-pipe and signature refs, so the API can render a meaningful 422.++Submitters call `dispatch_dry_validate` (`pipelex/temporal/tprl_pipe/dry_validate_dispatch.py`) for the whole round-trip. The wrapper workflow exists so the dispatch works on the current `temporalio` SDK; replacing it with a true standalone activity is a deferred optimization.++---+## Known Limitation: StuffArtefact Serialization in Dry-RunIn dry-run mode, PipeLLM produces `StuffArtefact` debug objects as working memory content. When controllers dispatch internal templating activities that carry working memory contents through the Temporal data converter, Kajson fails with `TypeError: Type <class 'StuffArtefact'> is not JSON serializable`.diff --git a/mkdocs.yml b/mkdocs.ymlindex eb0941d87..46b6e589c 100644--- a/mkdocs.yml+++ b/mkdocs.yml@@ -239,6 +239,7 @@ plugins:- building-methods/libraries.md: "Libraries"- building-methods/packages.md: "Packages"- building-methods/pipes/executing-pipelines.md: "Executing Pipelines"+ - building-methods/pipes/run-modes-and-backends.md: "Run Modes & Backends"- building-methods/pipes/provide-inputs.md: "Providing Inputs to Pipelines"- building-methods/pipes/working-memory.md: "Working Memory"- building-methods/pipes/pipe-output.md: "Pipe Output"@@ -454,6 +455,7 @@ nav:- Libraries: building-methods/libraries.md- Packages: building-methods/packages.md- Executing Pipelines: building-methods/pipes/executing-pipelines.md+ - Run Modes & Backends: building-methods/pipes/run-modes-and-backends.md- Providing Inputs to Pipelines: building-methods/pipes/provide-inputs.md- Working Memory: building-methods/pipes/working-memory.md- Pipe Output: building-methods/pipes/pipe-output.mddiff --git a/pipelex/builder/bundle_header_spec.py b/pipelex/builder/bundle_header_spec.pyindex 89a3d7f7a..6e07005fe 100644--- a/pipelex/builder/bundle_header_spec.py+++ b/pipelex/builder/bundle_header_spec.py@@ -3,6 +3,7 @@ from rich.console import Groupfrom rich.text import Textfrom typing_extensions import override+from pipelex.cogt.content_generation.dry_mock import MOCK_MAIN_PIPE_CODEfrom pipelex.cogt.content_generation.dry_run_factory import MockFormatfrom pipelex.core.stuffs.structured_content import StructuredContentfrom pipelex.tools.misc.pretty import PrettyPrintable@@ -12,8 +13,9 @@ class BundleHeaderSpec(StructuredContent):domain_code: str = Field(description="Name of the domain of the knowledge work.", json_schema_extra={"mock_format": MockFormat.SNAKE_CASE})description: str = Field(description="Definition of the domain of the knowledge work.")system_prompt: str | None = Field(description="System prompt for the domain.")- # Use a specific mock value that matches DRY_RUN_MAIN_PIPE_CODE in working_memory_factory- main_pipe: str = Field(description="The main pipe of the domain.", examples=["mock_main"])+ # The example IS the dry-run coordination: mocked headers name MOCK_MAIN_PIPE_CODE as the+ # bundle's main pipe, and stamp_mock_main_coordination makes mocked pipe-spec lists answer to it.+ main_pipe: str = Field(description="The main pipe of the domain.", examples=[MOCK_MAIN_PIPE_CODE])@overridedef rendered_pretty(self, title: str | None = None, depth: int = 0) -> PrettyPrintable:diff --git a/pipelex/cli/commands/run/_run_core.py b/pipelex/cli/commands/run/_run_core.pyindex 97d7553d2..af6fe1af5 100644--- a/pipelex/cli/commands/run/_run_core.py+++ b/pipelex/cli/commands/run/_run_core.py@@ -48,16 +48,17 @@ if TYPE_CHECKING:COMMAND = "run"-def validate_run_flag_combination(*, dry_run: bool, mock_inference: bool, mock_inputs: bool) -> None:- """Reject illegal ``--dry-run`` / ``--mock-inference`` / ``--mock-inputs`` combinations.+def validate_run_flag_combination(*, dry_run: bool, mock_usage: bool, mock_inputs: bool) -> None:+ """Reject illegal ``--dry-run`` / ``--mock-usage`` / ``--mock-inputs`` combinations.Single owner of which run-flag combinations are legal, shared by the ``pipe`` / ``method`` / ``bundle``run subcommands so they can't drift:- - ``--mock-inputs`` requires ``--dry-run`` — it fills missing required inputs for the dry generator- that ``--dry-run`` swaps in pre-dispatch; without ``--dry-run`` there is nothing for it to feed.- - ``--mock-inference`` cannot be combined with ``--dry-run`` — ``--dry-run`` swaps the generator- pre-dispatch so the leaf is never reached, which would silently ignore ``--mock-inference``.+ - ``--mock-inputs`` requires ``--dry-run`` — it synthesizes missing required inputs for a dry+ run; a live run must be fed real inputs.+ - ``--mock-usage`` (hidden test trigger) requires ``--dry-run`` — it is a sub-flag of the dry+ run (non-zero synthetic usage so the cost report renders); on a live run it is a contract+ violation.Prints the offending combination to stderr and raises ``typer.Exit(1)``; returns ``None`` when thecombination is legal.@@ -65,8 +66,8 @@ def validate_run_flag_combination(*, dry_run: bool, mock_inference: bool, mock_iif mock_inputs and not dry_run:typer.secho("Failed to run: --mock-inputs requires --dry-run", fg=typer.colors.RED, err=True)raise typer.Exit(1)- if mock_inference and dry_run:- typer.secho("Failed to run: --mock-inference cannot be combined with --dry-run", fg=typer.colors.RED, err=True)+ if mock_usage and not dry_run:+ typer.secho("Failed to run: --mock-usage requires --dry-run", fg=typer.colors.RED, err=True)raise typer.Exit(1)@@ -82,7 +83,7 @@ async def _execute_run(graph_full_data: bool | None,output_dir: str,dry_run: bool,- mock_inference: bool,+ mock_usage: bool,mock_inputs: bool,library_dir: list[str] | None,costs: bool | None = None,@@ -177,7 +178,7 @@ async def _execute_run(runner = PipelexRunner(bundle_uris=[bundle_path] if bundle_path else None,pipe_run_mode=pipe_run_mode,- is_mock_inference=mock_inference,+ is_mock_usage=mock_usage,execution_config=execution_config,library_dirs=library_dir,)@@ -352,7 +353,7 @@ def execute_run(graph_full_data: bool | None,output_dir: str,dry_run: bool,- mock_inference: bool,+ mock_usage: bool,mock_inputs: bool,library_dir: list[str] | None,costs: bool | None = None,@@ -386,7 +387,7 @@ def execute_run(graph_full_data=graph_full_data,output_dir=output_dir,dry_run=dry_run,- mock_inference=mock_inference,+ mock_usage=mock_usage,mock_inputs=mock_inputs,library_dir=library_dir,costs=costs,diff --git a/pipelex/cli/commands/run/bundle_cmd.py b/pipelex/cli/commands/run/bundle_cmd.pyindex bc5bf39c5..f9697ec4b 100644--- a/pipelex/cli/commands/run/bundle_cmd.py+++ b/pipelex/cli/commands/run/bundle_cmd.py@@ -61,11 +61,12 @@ def run_bundle_cmd(bool,typer.Option("--dry-run", help="Run pipeline in dry mode (no actual inference calls)"),] = False,- mock_inference: Annotated[+ mock_usage: Annotated[bool,typer.Option(- "--mock-inference",- help="Live run that fakes AI calls at the inference leaf with reportable synthetic usage. Mutually exclusive with --dry-run.",+ "--mock-usage",+ hidden=True,+ help="Internal test trigger: dry run whose LLM leaf mocks report nonzero synthetic usage so the cost report renders. Requires --dry-run.",),] = False,mock_inputs: Annotated[@@ -112,7 +113,7 @@ def run_bundle_cmd(pipelex run bundle my_bundle.mthds --pipe my_pipe --inputs data.jsonpipelex run bundle pipeline_01/ --dry-run"""- validate_run_flag_combination(dry_run=dry_run, mock_inference=mock_inference, mock_inputs=mock_inputs)+ validate_run_flag_combination(dry_run=dry_run, mock_usage=mock_usage, mock_inputs=mock_inputs)pipe_code: str | None = pipebundle_path: str | None = None@@ -183,7 +184,7 @@ def run_bundle_cmd(graph_full_data=graph_full_data,output_dir=output_dir,dry_run=dry_run,- mock_inference=mock_inference,+ mock_usage=mock_usage,mock_inputs=mock_inputs,library_dir=library_dir,costs=costs,diff --git a/pipelex/cli/commands/run/method_cmd.py b/pipelex/cli/commands/run/method_cmd.pyindex a9f901e17..bb9ed0008 100644--- a/pipelex/cli/commands/run/method_cmd.py+++ b/pipelex/cli/commands/run/method_cmd.py@@ -60,11 +60,12 @@ def run_method_cmd(bool,typer.Option("--dry-run", help="Run pipeline in dry mode (no actual inference calls)"),] = False,- mock_inference: Annotated[+ mock_usage: Annotated[bool,typer.Option(- "--mock-inference",- help="Live run that fakes AI calls at the inference leaf with reportable synthetic usage. Mutually exclusive with --dry-run.",+ "--mock-usage",+ hidden=True,+ help="Internal test trigger: dry run whose LLM leaf mocks report nonzero synthetic usage so the cost report renders. Requires --dry-run.",),] = False,mock_inputs: Annotated[@@ -114,7 +115,7 @@ def run_method_cmd(pipelex run method my-method --inputs data.jsonpipelex run method my-method --dry-run"""- validate_run_flag_combination(dry_run=dry_run, mock_inference=mock_inference, mock_inputs=mock_inputs)+ validate_run_flag_combination(dry_run=dry_run, mock_usage=mock_usage, mock_inputs=mock_inputs)# Resolve method name to pipe_code and library dirspipe_code, method_library_dirs, _ = resolve_method_target(@@ -153,7 +154,7 @@ def run_method_cmd(graph_full_data=graph_full_data,output_dir=effective_output_dir,dry_run=dry_run,- mock_inference=mock_inference,+ mock_usage=mock_usage,mock_inputs=mock_inputs,library_dir=effective_library_dir,costs=costs,diff --git a/pipelex/cli/commands/run/pipe_cmd.py b/pipelex/cli/commands/run/pipe_cmd.pyindex db645f638..7e96fab46 100644--- a/pipelex/cli/commands/run/pipe_cmd.py+++ b/pipelex/cli/commands/run/pipe_cmd.py@@ -57,11 +57,12 @@ def run_pipe_cmd(bool,typer.Option("--dry-run", help="Run pipeline in dry mode (no actual inference calls)"),] = False,- mock_inference: Annotated[+ mock_usage: Annotated[bool,typer.Option(- "--mock-inference",- help="Live run that fakes AI calls at the inference leaf with reportable synthetic usage. Mutually exclusive with --dry-run.",+ "--mock-usage",+ hidden=True,+ help="Internal test trigger: dry run whose LLM leaf mocks report nonzero synthetic usage so the cost report renders. Requires --dry-run.",),] = False,mock_inputs: Annotated[@@ -118,7 +119,7 @@ def run_pipe_cmd()raise typer.Exit(1)- validate_run_flag_combination(dry_run=dry_run, mock_inference=mock_inference, mock_inputs=mock_inputs)+ validate_run_flag_combination(dry_run=dry_run, mock_usage=mock_usage, mock_inputs=mock_inputs)# Check installed methods' exports for additional library dirstry:@@ -148,7 +149,7 @@ def run_pipe_cmd(graph_full_data=graph_full_data,output_dir=output_dir,dry_run=dry_run,- mock_inference=mock_inference,+ mock_usage=mock_usage,mock_inputs=mock_inputs,library_dir=library_dir,costs=costs,diff --git a/pipelex/cogt/content_generation/assignment_models.py b/pipelex/cogt/content_generation/assignment_models.pyindex 73380b4ad..45100e33f 100644--- a/pipelex/cogt/content_generation/assignment_models.py+++ b/pipelex/cogt/content_generation/assignment_models.py@@ -1,8 +1,9 @@from typing import Any-from pydantic import BaseModel+from pydantic import BaseModel, Fieldfrom typing_extensions import override+from pipelex.cogt.content_generation.cogt_run_params import CogtRunParamsfrom pipelex.cogt.extract.extract_input import ExtractInputfrom pipelex.cogt.extract.extract_job_components import ExtractJobConfig, ExtractJobParamsfrom pipelex.cogt.img_gen.img_gen_job_components import ImgGenJobConfig, ImgGenJobParams@@ -18,6 +19,7 @@ from pipelex.pipeline.job_metadata import JobMetadataclass LLMAssignment(BaseModel):job_metadata: JobMetadata+ cogt_run_params: CogtRunParamsllm_setting: LLMSettingllm_prompt: LLMPrompt@@ -25,12 +27,14 @@ class LLMAssignment(BaseModel):def make_from_prompt(cls,job_metadata: JobMetadata,+ cogt_run_params: CogtRunParams,llm_setting: LLMSetting,llm_prompt: LLMPrompt,) -> "LLMAssignment":"""Factory method for creating LLMAssignment from existing prompt."""return cls(job_metadata=job_metadata,+ cogt_run_params=cogt_run_params,llm_setting=llm_setting,llm_prompt=llm_prompt,)@@ -38,6 +42,7 @@ class LLMAssignment(BaseModel):def clone_with_new_prompt(self, new_prompt: LLMPrompt) -> "LLMAssignment":return LLMAssignment(job_metadata=self.job_metadata,+ cogt_run_params=self.cogt_run_params,llm_setting=self.llm_setting,llm_prompt=new_prompt,)@@ -66,21 +71,34 @@ class ObjectAssignment(BaseModel):object_class_name: strobject_class_schema: dict[str, Any]llm_assignment_for_object: LLMAssignment+ # Fixed list size for object-list generation (None = leaf mock falls back to+ # `dry_run_config.nb_list_items`). Carried on the assignment so the leaf mock+ # preserves fixed-size list lengths on any backend (eng review D11). ge=0: a negative+ # count must fail loud at construction, not silently mock an empty list.+ nb_items: int | None = Field(default=None, ge=0)++ @property+ def cogt_run_params(self) -> CogtRunParams:+ """Delegates to the nested LLM assignment — single copy, no duplication."""+ return self.llm_assignment_for_object.cogt_run_params@staticmethoddef make_for_class(object_class: type[BaseModel],llm_assignment: LLMAssignment,+ nb_items: int | None = None,) -> "ObjectAssignment":return ObjectAssignment(object_class_name=object_class.__name__,object_class_schema=object_class.model_json_schema(),llm_assignment_for_object=llm_assignment,+ nb_items=nb_items,)class ImgGenAssignment(BaseModel):job_metadata: JobMetadata+ cogt_run_params: CogtRunParamsimg_gen_handle: strimg_gen_prompt: ImgGenPromptimg_gen_job_params: ImgGenJobParams@@ -90,6 +108,7 @@ class ImgGenAssignment(BaseModel):class TemplatingAssignment(BaseModel):job_metadata: JobMetadata+ cogt_run_params: CogtRunParamscontext: dict[str, Any]template: strtemplating_style: TemplatingStyle | None = None@@ -98,6 +117,7 @@ class TemplatingAssignment(BaseModel):class ExtractAssignment(BaseModel):job_metadata: JobMetadata+ cogt_run_params: CogtRunParamsextract_handle: strextract_input: ExtractInputextract_job_params: ExtractJobParams@@ -106,6 +126,7 @@ class ExtractAssignment(BaseModel):class RenderPageViewsAssignment(BaseModel):job_metadata: JobMetadata+ cogt_run_params: CogtRunParamsdocument_uri: strpage_views_dpi: int@@ -120,6 +141,7 @@ class SearchAssignment(BaseModel):"""job_metadata: JobMetadata+ cogt_run_params: CogtRunParamsquery: strsearch_setting: SearchSettinginclude_domains: list[str] | None = None@@ -144,6 +166,11 @@ class SearchObjectAssignment(BaseModel):output_class_schema: dict[str, Any]search_assignment: SearchAssignment+ @property+ def cogt_run_params(self) -> CogtRunParams:+ """Delegates to the nested search assignment — single copy, no duplication."""+ return self.search_assignment.cogt_run_params+@staticmethoddef make_for_class(output_class: type[BaseModel],diff --git a/pipelex/cogt/content_generation/cogt_run_params.py b/pipelex/cogt/content_generation/cogt_run_params.pynew file mode 100644index 000000000..fbee9ea75--- /dev/null+++ b/pipelex/cogt/content_generation/cogt_run_params.py@@ -0,0 +1,70 @@+"""Cogt-tier slice of the run params (D-plan Part B).++``CogtRunParams`` carries the execution-mode contract down to the cogt leaf. The+facts (``run_mode``, ``is_mock_usage``) are owned by ``PipeRunParams`` as direct+fields; its derived ``cogt_run_params`` property mints this carrier on demand, so+there is exactly one stored copy. Single writer of the facts:+``PipeRunParamsFactory.make_run_params`` (fed by ``prepare_pipe_job``).++Route to the leaf: operators slice ``pipe_run_params.cogt_run_params`` off and pass+it into the content-generator protocol methods; generators stamp it as a field on+every cogt assignment (``LLMAssignment``, ``ImgGenAssignment``, ...), which+serializes whole across the Temporal activity boundary — so the leaf sees the same+``run_mode`` whether it runs inline or inside an ``act_*_gen_*`` activity. That is+what makes ``run_mode`` orthogonal to the backend (D-plan §3.5).++Deliberately NOT ``PipeRunParams`` itself: threading the pipe-tier params into cogt+would drag pipe-only concepts (multiplicity, batch params, pipe stack) into a layer+that must stay pipe-agnostic. This class is the cogt slice and grows only+cogt-relevant flags.+"""++from pydantic import BaseModel, ConfigDict, model_validator++from pipelex.pipe_run.pipe_run_mode import PipeRunMode+from pipelex.types import Self+++def check_mock_usage_requires_dry(*, run_mode: PipeRunMode, is_mock_usage: bool) -> None:+ """Single home of the rule: ``is_mock_usage`` is a sub-flag of ``run_mode=DRY``.++ Raises ``ValueError`` on the illegal LIVE combination — called by the model validators on+ both ``CogtRunParams`` and ``PipeRunParams`` (each guards its own boundary) and by+ ``PipeRunParamsFactory.make_run_params`` (which must reject the REQUESTED mode before the+ keyless forced-DRY coercion can mask it).+ """+ if is_mock_usage and run_mode.is_live:+ msg = "is_mock_usage is a sub-flag of run_mode=DRY: it cannot be set on a LIVE run"+ raise ValueError(msg)+++class CogtRunParams(BaseModel):+ """Execution-mode contract carried on every cogt assignment (crosses the Temporal wire).++ There is exactly one non-live mode: ``run_mode=DRY`` (no provider calls, no storage IO, no+ user-code execution). ``is_mock_usage`` is a secondary, internal flag that only has meaning+ on a DRY run — setting it on a LIVE carrier is a contract violation rejected at validation.+ """++ # `extra="forbid"`: a stale or typo'd key on a wire payload must fail loud (mirrors+ # PipeRunParams). `frozen=True`: the carrier is an immutable value object — once minted from+ # `PipeRunParams.cogt_run_params` and stamped on an assignment, no leaf can mutate the+ # execution contract mid-run. `run_mode` is REQUIRED: a wire payload that omits it must fail+ # loud rather than silently default to the spending direction (LIVE) or the mock direction+ # (DRY).+ model_config = ConfigDict(extra="forbid", frozen=True)++ run_mode: PipeRunMode++ # Internal sub-flag of DRY — only CLI access is the hidden `--mock-usage` test trigger+ # (requires `--dry-run`, not shown in --help): when True, the dry LLM leaves report+ # *non-zero* synthetic usage (deterministic sentinel counts, $0 cost) so the end-of-run cost+ # report renders — the cheap, deterministic cross-worker cost-report validation affordance.+ # Default False keeps dry runs zero-token with the report suppressed. Single writer:+ # ``PipeRunParamsFactory.make_run_params`` (fed by ``prepare_pipe_job``).+ is_mock_usage: bool = False++ @model_validator(mode="after")+ def validate_mock_usage_requires_dry(self) -> Self:+ check_mock_usage_requires_dry(run_mode=self.run_mode, is_mock_usage=self.is_mock_usage)+ return selfdiff --git a/pipelex/cogt/content_generation/content_generator.py b/pipelex/cogt/content_generation/content_generator.pyindex 9265fbe19..d08e8eeac 100644--- a/pipelex/cogt/content_generation/content_generator.py+++ b/pipelex/cogt/content_generation/content_generator.py@@ -9,22 +9,23 @@ from pipelex.cogt.content_generation.assignment_models import (ImgGenAssignment,LLMAssignment,ObjectAssignment,+ RenderPageViewsAssignment,SearchAssignment,SearchObjectAssignment,TemplatingAssignment,)+from pipelex.cogt.content_generation.cogt_run_params import CogtRunParamsfrom pipelex.cogt.content_generation.content_generator_protocol import ContentGeneratorProtocol, update_job_metadata-from pipelex.cogt.content_generation.exceptions import MockInferenceObjectFidelityError-from pipelex.cogt.content_generation.extract_generate import extract_gen_pages+from pipelex.cogt.content_generation.exceptions import DryRunObjectFidelityError+from pipelex.cogt.content_generation.extract_generate import extract_gen_pages_and_storefrom pipelex.cogt.content_generation.generated_content_factory import GeneratedContentFactory-from pipelex.cogt.content_generation.img_gen_generate import img_gen_image_list, img_gen_single_image+from pipelex.cogt.content_generation.img_gen_generate import img_gen_image_list_and_store, img_gen_single_image_and_storefrom pipelex.cogt.content_generation.llm_generate import llm_gen_object, llm_gen_object_list, llm_gen_text+from pipelex.cogt.content_generation.render_generate import render_page_views_and_storefrom pipelex.cogt.content_generation.search_generate import search_gen_sourced_answer, search_gen_structuredfrom pipelex.cogt.content_generation.templating_generate import templating_gen_textfrom pipelex.cogt.extract.extract_input import ExtractInputfrom pipelex.cogt.extract.extract_job_components import ExtractJobConfig, ExtractJobParams-from pipelex.cogt.extract.extract_output import ExtractOutput-from pipelex.cogt.image.generated_image import GeneratedImageRawDetailsfrom pipelex.cogt.img_gen.img_gen_job_components import ImgGenJobConfig, ImgGenJobParamsfrom pipelex.cogt.img_gen.img_gen_prompt import ImgGenPromptfrom pipelex.cogt.llm.llm_prompt import LLMPrompt@@ -35,8 +36,8 @@ from pipelex.config import get_configfrom pipelex.core.stuffs.image_content import ImageContentfrom pipelex.core.stuffs.page_content import PageContentfrom pipelex.core.stuffs.search_result_content import SearchResultContent+from pipelex.hub import get_storage_providerfrom pipelex.pipeline.job_metadata import JobMetadata-from pipelex.tools.misc.image_utils import ImageFormatfrom pipelex.tools.typing.pydantic_utils import BaseModelTypeVar@@ -44,7 +45,7 @@ def _revalidate_against_object_class(raw_obj: BaseModel,object_class: type[BaseModelTypeVar],*,- is_mock_inference: bool,+ is_mock_built: bool,) -> BaseModelTypeVar:"""Re-validate a leaf-generated object's data against the original ``object_class``.@@ -52,31 +53,43 @@ def _revalidate_against_object_class(schema; re-validating their data against the original class makes the result the proper subtype (e.g.``StructuredContent``) the caller expects.- Under ``--mock-inference`` the object was built by polyfactory from the schema-reconstructed class, which- can drop invariants the original class enforces (custom validators, ``json_schema_extra`` format/pattern- hints datamodel-code-generator omits on round-trip). A re-validation failure there is the known object-mock- fidelity gap, so the ``ValidationError`` is re-raised as a clear typed- :class:`MockInferenceObjectFidelityError` that names the class and points at ``--dry-run``. The catch is- scoped to the mock path only — a LIVE provider's invalid output keeps its existing ``ValidationError``.+ Under the dry-run leaf mock (``run_mode=DRY``) the object was built by polyfactory from+ the schema-reconstructed class, which can drop invariants the original class enforces (custom+ validators, ``json_schema_extra`` format/pattern hints datamodel-code-generator omits on round-trip).+ A re-validation failure there is the known object-mock fidelity gap, so the ``ValidationError`` is+ re-raised as a clear typed :class:`DryRunObjectFidelityError` naming the class and the+ ``examples`` / ``mock_format`` remedy. The catch is scoped to the dry path only — a LIVE provider's+ invalid output keeps its existing ``ValidationError``."""raw_data = raw_obj.model_dump(serialize_as_any=True)- if not is_mock_inference:+ if not is_mock_built:return object_class.model_validate(raw_data)try:return object_class.model_validate(raw_data)except ValidationError as exc:- raise MockInferenceObjectFidelityError.for_object_class(object_class.__name__) from exc+ raise DryRunObjectFidelityError.for_object_class(object_class.__name__) from excclass ContentGenerator(ContentGeneratorProtocol):def __init__(self, generated_content_factory: GeneratedContentFactory) -> None:self._generated_content_factory = generated_content_factory+ @classmethod+ def make_inline(cls) -> "ContentGenerator":+ """Inline generator wired to the hub storage provider.++ The single home for the in-process scopes' generator (``bundle_validator``,+ ``dry_run_pipeline``): under DRY its leaves mock without dispatching and without+ touching storage.+ """+ return cls(generated_content_factory=GeneratedContentFactory(storage_provider=get_storage_provider()))+@override@update_job_metadataasync def make_llm_text(self,job_metadata: JobMetadata,+ cogt_run_params: CogtRunParams,llm_setting_main: LLMSetting,llm_prompt_for_text: LLMPrompt,) -> str:@@ -84,6 +97,7 @@ class ContentGenerator(ContentGeneratorProtocol):log.verbose(f"llm_setting_main: {llm_setting_main}")llm_assignment = LLMAssignment.make_from_prompt(job_metadata=job_metadata,+ cogt_run_params=cogt_run_params,llm_setting=llm_setting_main,llm_prompt=llm_prompt_for_text,)@@ -97,6 +111,7 @@ class ContentGenerator(ContentGeneratorProtocol):async def make_object(self,job_metadata: JobMetadata,+ cogt_run_params: CogtRunParams,object_class: type[BaseModelTypeVar],llm_setting_for_object: LLMSetting,llm_prompt_for_object: LLMPrompt,@@ -104,6 +119,7 @@ class ContentGenerator(ContentGeneratorProtocol):log.verbose(f"{self.__class__.__name__} make_object: {llm_prompt_for_object}")llm_assignment_for_object = LLMAssignment.make_from_prompt(job_metadata=job_metadata,+ cogt_run_params=cogt_run_params,llm_setting=llm_setting_for_object,llm_prompt=llm_prompt_for_object,)@@ -113,13 +129,14 @@ class ContentGenerator(ContentGeneratorProtocol):)raw_obj = await llm_gen_object(object_assignment=object_assignment)log.verbose(f"{self.__class__.__name__} generated object direct: {raw_obj}")- return _revalidate_against_object_class(raw_obj, object_class, is_mock_inference=job_metadata.is_mock_inference)+ return _revalidate_against_object_class(raw_obj, object_class, is_mock_built=cogt_run_params.run_mode.is_dry)@override@update_job_metadataasync def make_object_list(self,job_metadata: JobMetadata,+ cogt_run_params: CogtRunParams,object_class: type[BaseModelTypeVar],llm_setting_for_object_list: LLMSetting,llm_prompt_for_object_list: LLMPrompt,@@ -127,80 +144,53 @@ class ContentGenerator(ContentGeneratorProtocol):) -> list[BaseModelTypeVar]:llm_assignment_for_object = LLMAssignment.make_from_prompt(job_metadata=job_metadata,+ cogt_run_params=cogt_run_params,llm_setting=llm_setting_for_object_list,llm_prompt=llm_prompt_for_object_list,)object_assignment = ObjectAssignment.make_for_class(object_class=object_class,llm_assignment=llm_assignment_for_object,+ nb_items=nb_items,)raw_list = await llm_gen_object_list(object_assignment=object_assignment)log.verbose(f"{self.__class__.__name__} generated object list direct: {raw_list}")- return [_revalidate_against_object_class(raw_obj, object_class, is_mock_inference=job_metadata.is_mock_inference) for raw_obj in raw_list]-- @override- async def make_image_content(- self,- job_metadata: JobMetadata,- generated_image_raw_details: GeneratedImageRawDetails,- img_gen_prompt: ImgGenPrompt | None,- ) -> ImageContent:- image_content = await self._generated_content_factory.make_image_content(- primary_id=job_metadata.user_id,- secondary_id=job_metadata.pipeline_run_id,- raw_details=generated_image_raw_details,- )- if img_gen_prompt:- image_content.source_prompt = img_gen_prompt.positive_text- image_content.source_negative_prompt = img_gen_prompt.negative_text- return image_content-- @override- async def make_page_contents(- self,- job_metadata: JobMetadata,- extract_output: ExtractOutput,- ) -> list[PageContent]:- return await self._generated_content_factory.make_page_contents(- primary_id=job_metadata.user_id,- secondary_id=job_metadata.pipeline_run_id,- extract_output=extract_output,- )+ return [_revalidate_against_object_class(raw_obj, object_class, is_mock_built=cogt_run_params.run_mode.is_dry) for raw_obj in raw_list]@override@update_job_metadataasync def make_single_image(self,job_metadata: JobMetadata,+ cogt_run_params: CogtRunParams,img_gen_handle: str,img_gen_prompt: ImgGenPrompt,img_gen_job_params: ImgGenJobParams | None = None,img_gen_job_config: ImgGenJobConfig | None = None,) -> ImageContent:img_gen_config = get_config().cogt.img_gen_config- img_gen_job_params = img_gen_job_params or img_gen_config.make_default_img_gen_job_params()- img_gen_job_config = img_gen_job_config or img_gen_config.img_gen_job_configimg_gen_assignment = ImgGenAssignment(job_metadata=job_metadata,+ cogt_run_params=cogt_run_params,img_gen_handle=img_gen_handle,img_gen_prompt=img_gen_prompt,- img_gen_job_params=img_gen_job_params,- img_gen_job_config=img_gen_job_config,+ img_gen_job_params=img_gen_job_params or img_gen_config.make_default_img_gen_job_params(),+ img_gen_job_config=img_gen_job_config or img_gen_config.img_gen_job_config,nb_images=1,)- generated_image_raw_details = await img_gen_single_image(img_gen_assignment=img_gen_assignment)- log.verbose(f"{self.__class__.__name__} generated image raw details: {generated_image_raw_details}")- return await self.make_image_content(- job_metadata=job_metadata,- img_gen_prompt=img_gen_prompt,- generated_image_raw_details=generated_image_raw_details,+ image_content = await img_gen_single_image_and_store(+ img_gen_assignment=img_gen_assignment,+ generated_content_factory=self._generated_content_factory,)+ log.verbose(f"{self.__class__.__name__} generated image: {image_content}")+ return image_content@override@update_job_metadataasync def make_image_list(self,job_metadata: JobMetadata,+ cogt_run_params: CogtRunParams,img_gen_handle: str,img_gen_prompt: ImgGenPrompt,nb_images: int,@@ -210,27 +200,25 @@ class ContentGenerator(ContentGeneratorProtocol):img_gen_config = get_config().cogt.img_gen_configimg_gen_assignment = ImgGenAssignment(job_metadata=job_metadata,+ cogt_run_params=cogt_run_params,img_gen_handle=img_gen_handle,img_gen_prompt=img_gen_prompt,img_gen_job_params=img_gen_job_params or img_gen_config.make_default_img_gen_job_params(),img_gen_job_config=img_gen_job_config or img_gen_config.img_gen_job_config,nb_images=nb_images,)- generated_images_as_raw_details = await img_gen_image_list(img_gen_assignment=img_gen_assignment)- log.verbose(f"{self.__class__.__name__} generated image list: {generated_images_as_raw_details}")- return [- await self.make_image_content(- job_metadata=job_metadata,- generated_image_raw_details=raw_details,- img_gen_prompt=img_gen_prompt,- )- for raw_details in generated_images_as_raw_details- ]+ image_contents = await img_gen_image_list_and_store(+ img_gen_assignment=img_gen_assignment,+ generated_content_factory=self._generated_content_factory,+ )+ log.verbose(f"{self.__class__.__name__} generated image list: {image_contents}")+ return image_contents@overrideasync def make_templated_text(self,job_metadata: JobMetadata,+ cogt_run_params: CogtRunParams,context: dict[str, Any],template: str,templating_style: TemplatingStyle | None = None,@@ -238,6 +226,7 @@ class ContentGenerator(ContentGeneratorProtocol):) -> str:templating_assignment = TemplatingAssignment(job_metadata=job_metadata,+ cogt_run_params=cogt_run_params,context=context,template=template,templating_style=templating_style,@@ -250,6 +239,7 @@ class ContentGenerator(ContentGeneratorProtocol):async def make_render_page_views(self,job_metadata: JobMetadata,+ cogt_run_params: CogtRunParams,extract_input: ExtractInput,extract_handle: str,extract_job_params: ExtractJobParams | None = None,@@ -260,29 +250,23 @@ class ContentGenerator(ContentGeneratorProtocol):raise ValueError(msg)job_params = extract_job_params or ExtractJobParams.make_default_extract_job_params()page_views_dpi = job_params.page_views_dpi or get_config().cogt.extract_config.default_page_views_dpi- # Deferred import: avoid pulling heavy SDK at module-load time- from pipelex.tools.pdf.pypdfium2_renderer import pypdfium2_renderer # noqa: PLC0415-- page_view_images = await pypdfium2_renderer.render_pdf_pages_from_uri(pdf_uri=extract_input.document_uri, dpi=page_views_dpi)- page_view_images_resolved: list[ImageContent] = []- for page_view_image in page_view_images:- image_content = await self.make_image_content(- job_metadata=job_metadata,- generated_image_raw_details=GeneratedImageRawDetails.make_from_pil_image(- pil_image=page_view_image,- image_format=ImageFormat.PNG,- ),- img_gen_prompt=None,- )- page_view_images_resolved.append(image_content)-- return page_view_images_resolved+ render_assignment = RenderPageViewsAssignment(+ job_metadata=job_metadata,+ cogt_run_params=cogt_run_params,+ document_uri=extract_input.document_uri,+ page_views_dpi=page_views_dpi,+ )+ return await render_page_views_and_store(+ render_assignment=render_assignment,+ generated_content_factory=self._generated_content_factory,+ )@override@update_job_metadataasync def make_extract_pages(self,job_metadata: JobMetadata,+ cogt_run_params: CogtRunParams,extract_input: ExtractInput,extract_handle: str,extract_job_params: ExtractJobParams | None = None,@@ -292,15 +276,15 @@ class ContentGenerator(ContentGeneratorProtocol):extract_job_config = extract_job_config or ExtractJobConfig()extract_assignment = ExtractAssignment(job_metadata=job_metadata,+ cogt_run_params=cogt_run_params,extract_input=extract_input,extract_handle=extract_handle,extract_job_params=extract_job_params,extract_job_config=extract_job_config,)- extract_output = await extract_gen_pages(extract_assignment=extract_assignment)- page_contents = await self.make_page_contents(- job_metadata=job_metadata,- extract_output=extract_output,+ page_contents = await extract_gen_pages_and_store(+ extract_assignment=extract_assignment,+ generated_content_factory=self._generated_content_factory,)if extract_job_params and extract_job_params.should_include_page_views:page_view_contents: list[ImageContent] = []@@ -309,6 +293,7 @@ class ContentGenerator(ContentGeneratorProtocol):extract_input=extract_input,extract_handle=extract_handle,job_metadata=job_metadata,+ cogt_run_params=cogt_run_params,extract_job_params=extract_job_params,extract_job_config=extract_job_config,)@@ -341,4 +326,11 @@ class ContentGenerator(ContentGeneratorProtocol):search_assignment=search_assignment,),)- return output_structure_class.model_validate(result_dict)+ # Same fidelity guard as the object paths (D6): a mock-built dict that fails re-validation+ # against the original class is the known schema-round-trip gap, not a provider fault.+ if not search_assignment.cogt_run_params.run_mode.is_dry:+ return output_structure_class.model_validate(result_dict)+ try:+ return output_structure_class.model_validate(result_dict)+ except ValidationError as exc:+ raise DryRunObjectFidelityError.for_object_class(output_structure_class.__name__) from excdiff --git a/pipelex/cogt/content_generation/content_generator_dry.py b/pipelex/cogt/content_generation/content_generator_dry.pydeleted file mode 100644index 3b0b78a30..000000000--- a/pipelex/cogt/content_generation/content_generator_dry.py+++ /dev/null@@ -1,319 +0,0 @@-from typing import Any--from typing_extensions import override--from pipelex import log-from pipelex.cogt.content_generation.assignment_models import SearchAssignment-from pipelex.cogt.content_generation.content_generator_protocol import ContentGeneratorProtocol, update_job_metadata-from pipelex.cogt.content_generation.dry_mock import build_mock_object, report_dry_llm_job-from pipelex.cogt.content_generation.dry_run_factory import DryRunFactory-from pipelex.cogt.extract.extract_input import ExtractInput-from pipelex.cogt.extract.extract_job_components import ExtractJobConfig, ExtractJobParams-from pipelex.cogt.extract.extract_output import ExtractOutput, Page-from pipelex.cogt.image.generated_image import GeneratedImageRawDetails-from pipelex.cogt.image.image_size import ImageSize-from pipelex.cogt.img_gen.img_gen_job_components import ImgGenJobConfig, ImgGenJobParams-from pipelex.cogt.img_gen.img_gen_prompt import ImgGenPrompt-from pipelex.cogt.llm.llm_prompt import LLMPrompt-from pipelex.cogt.llm.llm_setting import LLMSetting-from pipelex.cogt.templating.template_category import TemplateCategory-from pipelex.cogt.templating.templating_style import TemplatingStyle-from pipelex.config import get_config-from pipelex.core.stuffs.document_content import DocumentContent-from pipelex.core.stuffs.image_content import ImageContent-from pipelex.core.stuffs.page_content import PageContent-from pipelex.core.stuffs.search_result_content import SearchResultContent-from pipelex.core.stuffs.text_and_images_content import TextAndImagesContent-from pipelex.core.stuffs.text_content import TextContent-from pipelex.pipeline.job_metadata import JobMetadata-from pipelex.tools.jinja2.jinja2_parsing import check_jinja2_parsing-from pipelex.tools.typing.pydantic_utils import BaseModelTypeVar-from pipelex.urls import URLs---class ContentGeneratorDry(ContentGeneratorProtocol):- """This class is used to generate mock content for testing purposes.- It does not use any inference.- """-- @property- def _text_gen_truncate_length(self) -> int:- return get_config().pipelex.dry_run_config.text_gen_truncate_length-- def _make_generated_image_fake(- self,- raw_details: GeneratedImageRawDetails,- ) -> ImageContent:- return ImageContent(- url=raw_details.actual_url or URLs.jpg_example_1,- public_url=raw_details.actual_url or URLs.jpg_example_1,- mime_type=raw_details.mime_type or "image/jpeg",- size=raw_details.size,- )-- @override- @update_job_metadata- async def make_llm_text(- self,- job_metadata: JobMetadata,- llm_setting_main: LLMSetting,- llm_prompt_for_text: LLMPrompt,- ) -> str:- func_name = "make_llm_text"- log.verbose(f"🤡 DRY RUN: {self.__class__.__name__}.{func_name}")- prompt_truncated = llm_prompt_for_text.desc(truncate_text_length=self._text_gen_truncate_length)- report_dry_llm_job(job_metadata=job_metadata, llm_setting=llm_setting_main, llm_prompt=llm_prompt_for_text)- return f"DRY RUN: {func_name} • llm_setting={llm_setting_main.desc()} • prompt={prompt_truncated}"-- @override- @update_job_metadata- async def make_object(- self,- job_metadata: JobMetadata,- object_class: type[BaseModelTypeVar],- llm_setting_for_object: LLMSetting,- llm_prompt_for_object: LLMPrompt,- ) -> BaseModelTypeVar:- report_dry_llm_job(job_metadata=job_metadata, llm_setting=llm_setting_for_object, llm_prompt=llm_prompt_for_object)- # build_mock_object runs validators so the mock is valid; fields with format constraints- # (snake_case, PascalCase, ...) should declare examples / mock_format in their Field().- return build_mock_object(object_class)-- @override- @update_job_metadata- async def make_object_list(- self,- job_metadata: JobMetadata,- object_class: type[BaseModelTypeVar],- llm_setting_for_object_list: LLMSetting,- llm_prompt_for_object_list: LLMPrompt,- nb_items: int | None = None,- ) -> list[BaseModelTypeVar]:- func_name = "make_object_list"- log.verbose(f"🤡 DRY RUN: {self.__class__.__name__}.{func_name}")- nb_list_items = nb_items or get_config().pipelex.dry_run_config.nb_list_items- items: list[BaseModelTypeVar] = []- for idx in range(nb_list_items):- item = await self.make_object(- job_metadata=job_metadata,- object_class=object_class,- llm_setting_for_object=llm_setting_for_object_list,- llm_prompt_for_object=llm_prompt_for_object_list,- )- # Set first item's pipe_code to "mock_main" to coordinate with BundleHeaderSpec.main_pipe- # which uses examples=["mock_main"] for dry run validation- if idx == 0 and hasattr(item, "pipe_code"):- item.pipe_code = "mock_main" # pyright: ignore[reportAttributeAccessIssue]- items.append(item)- return items-- @override- async def make_image_content(- self,- job_metadata: JobMetadata,- generated_image_raw_details: GeneratedImageRawDetails,- img_gen_prompt: ImgGenPrompt | None,- ) -> ImageContent:- image_content = self._make_generated_image_fake(raw_details=generated_image_raw_details)- if img_gen_prompt:- image_content.source_prompt = img_gen_prompt.positive_text- image_content.source_negative_prompt = img_gen_prompt.negative_text- return image_content-- @override- async def make_page_contents(- self,- job_metadata: JobMetadata,- extract_output: ExtractOutput,- ) -> list[PageContent]:- page_contents: list[PageContent] = []- for page_index in sorted(extract_output.pages.keys()):- page = extract_output.pages[page_index]- page_images: list[ImageContent] = []- for extracted_image in page.extracted_images:- image_content = await self.make_image_content(- job_metadata=job_metadata,- generated_image_raw_details=extracted_image,- img_gen_prompt=None,- )- page_images.append(image_content)- page_contents.append(- PageContent(- text_and_images=TextAndImagesContent(- text=TextContent(text=page.text) if page.text else None,- images=page_images,- raw_html=page.raw_html,- ),- )- )- return page_contents-- @override- @update_job_metadata- async def make_single_image(- self,- job_metadata: JobMetadata,- img_gen_handle: str,- img_gen_prompt: ImgGenPrompt,- img_gen_job_params: ImgGenJobParams | None = None,- img_gen_job_config: ImgGenJobConfig | None = None,- ) -> ImageContent:- func_name = "make_single_image"- log.verbose(f"🤡 DRY RUN: {self.__class__.__name__}.{func_name}")- image_urls = get_config().pipelex.dry_run_config.image_urls- image_url = image_urls[0]- return await self.make_image_content(- job_metadata=job_metadata,- generated_image_raw_details=GeneratedImageRawDetails(- actual_url=image_url,- size=ImageSize(width=1024, height=1024),- mime_type="image/jpeg",- ),- img_gen_prompt=img_gen_prompt,- )-- @override- @update_job_metadata- async def make_image_list(- self,- job_metadata: JobMetadata,- img_gen_handle: str,- img_gen_prompt: ImgGenPrompt,- nb_images: int,- img_gen_job_params: ImgGenJobParams | None = None,- img_gen_job_config: ImgGenJobConfig | None = None,- ) -> list[ImageContent]:- func_name = "make_image_list"- log.verbose(f"🤡 DRY RUN: {self.__class__.__name__}.{func_name}")- image_urls = get_config().pipelex.dry_run_config.image_urls- image_contents: list[ImageContent] = []- for image_index in range(nb_images):- image_content = await self.make_image_content(- job_metadata=job_metadata,- generated_image_raw_details=GeneratedImageRawDetails(- actual_url=image_urls[image_index % len(image_urls)],- size=ImageSize(width=1024, height=1024),- mime_type="image/jpeg",- ),- img_gen_prompt=img_gen_prompt,- )- image_contents.append(image_content)- return image_contents-- @override- async def make_templated_text(- self,- job_metadata: JobMetadata,- context: dict[str, Any],- template: str,- templating_style: TemplatingStyle | None = None,- template_category: TemplateCategory | None = None,- ) -> str:- check_jinja2_parsing(template_source=template, template_category=template_category or TemplateCategory.BASIC)- func_name = "make_templated_text"- log.verbose(f"🤡 DRY RUN: {self.__class__.__name__}.{func_name}")- jinja2_truncated = template[: self._text_gen_truncate_length]- return (- f"DRY RUN: {func_name} • context={context} • "- f"jinja2={jinja2_truncated} • templating_style={templating_style} • template_category={template_category}"- )-- @override- async def make_render_page_views(- self,- job_metadata: JobMetadata,- extract_input: ExtractInput,- extract_handle: str,- extract_job_params: ExtractJobParams | None = None,- extract_job_config: ExtractJobConfig | None = None,- ) -> list[ImageContent]:- if not extract_input.document_uri:- msg = "Document URI is required to render page views"- raise ValueError(msg)- nb_pages = get_config().pipelex.dry_run_config.nb_extract_pages- page_view_images_resolved: list[ImageContent] = []- for _ in range(1, nb_pages + 1):- page_view_image = self._make_generated_image_fake(- raw_details=GeneratedImageRawDetails(- actual_url=URLs.jpg_example_1,- size=ImageSize(width=1024, height=1024),- mime_type="image/jpeg",- ),- )- page_view_images_resolved.append(page_view_image)- return page_view_images_resolved-- @override- @update_job_metadata- async def make_extract_pages(- self,- job_metadata: JobMetadata,- extract_input: ExtractInput,- extract_handle: str,- extract_job_params: ExtractJobParams | None = None,- extract_job_config: ExtractJobConfig | None = None,- ) -> list[PageContent]:- func_name = "make_extract_pages"- log.verbose(f"🤡 DRY RUN: {self.__class__.__name__}.{func_name}")- nb_pages: int- if extract_input.image_uri:- nb_pages = 1- else:- nb_pages = get_config().pipelex.dry_run_config.nb_extract_pages- page_contents: list[PageContent] = []- for _ in range(1, nb_pages + 1):- page = Page(- text="DRY RUN: OCR text",- extracted_images=[],- )- page_contents.append(- PageContent(- text_and_images=TextAndImagesContent(- text=TextContent(text=page.text) if page.text else None,- images=[],- ),- page_view=None,- )- )-- if extract_job_params and extract_job_params.should_include_page_views:- page_view_contents: list[ImageContent] = []- if extract_input.document_uri:- page_view_contents = await self.make_render_page_views(- extract_input=extract_input,- extract_handle=extract_handle,- job_metadata=job_metadata,- extract_job_params=extract_job_params,- extract_job_config=extract_job_config,- )- elif extract_input.image_uri:- page_view_contents = [ImageContent(url=extract_input.image_uri)]- if len(page_view_contents) != len(page_contents):- msg = f"Number of page view contents ({len(page_view_contents)}) does not match number of page contents ({len(page_contents)})"- raise ValueError(msg)- for page_content in page_contents:- page_content.page_view = page_view_contents.pop(0)-- return page_contents-- @override- async def make_search_sourced_answer(- self,- search_assignment: SearchAssignment,- ) -> SearchResultContent:- log.verbose(f"🤡 DRY RUN: {self.__class__.__name__}.make_search_sourced_answer")- nb_sources = get_config().pipelex.dry_run_config.nb_list_items- doc_factory = DryRunFactory.make_dry_run_factory(DocumentContent)- mock_sources = [doc_factory.build() for _ in range(nb_sources)]- search_result_factory = DryRunFactory.make_dry_run_factory(SearchResultContent)- return search_result_factory.build(sources=mock_sources)-- @override- async def make_search_structured(- self,- output_structure_class: type[BaseModelTypeVar],- search_assignment: SearchAssignment,- ) -> BaseModelTypeVar:- log.verbose(f"🤡 DRY RUN: {self.__class__.__name__}.make_search_structured")- structured_factory = DryRunFactory.make_dry_run_factory(output_structure_class)- return structured_factory.build()diff --git a/pipelex/cogt/content_generation/content_generator_protocol.py b/pipelex/cogt/content_generation/content_generator_protocol.pyindex 7a9f01e60..e996acf86 100644--- a/pipelex/cogt/content_generation/content_generator_protocol.py+++ b/pipelex/cogt/content_generation/content_generator_protocol.py@@ -3,10 +3,9 @@ from functools import wrapsfrom typing import Any, ParamSpec, Protocol, TypeVarfrom pipelex.cogt.content_generation.assignment_models import SearchAssignment+from pipelex.cogt.content_generation.cogt_run_params import CogtRunParamsfrom pipelex.cogt.extract.extract_input import ExtractInputfrom pipelex.cogt.extract.extract_job_components import ExtractJobConfig, ExtractJobParams-from pipelex.cogt.extract.extract_output import ExtractOutput-from pipelex.cogt.image.generated_image import GeneratedImageRawDetailsfrom pipelex.cogt.img_gen.img_gen_job_components import ImgGenJobConfig, ImgGenJobParamsfrom pipelex.cogt.img_gen.img_gen_prompt import ImgGenPromptfrom pipelex.cogt.llm.llm_prompt import LLMPrompt@@ -45,9 +44,19 @@ def update_job_metadata(func: Callable[P, Coroutine[Any, Any, R]]) -> Callable[Pclass ContentGeneratorProtocol(Protocol):+ """Content-generation seam between operators and the cogt leaves.++ Every leaf-dispatching method takes ``cogt_run_params`` (the cogt slice of the run params,+ sliced off ``PipeRunParams`` by the operator). Implementations stamp it onto the assignment+ they build, so the leaf sees the same ``run_mode`` on any backend — inline or inside a+ Temporal activity. The two search methods take a pre-built assignment instead; the operator+ stamps ``cogt_run_params`` at assignment construction.+ """+def make_llm_text(self,job_metadata: JobMetadata,+ cogt_run_params: CogtRunParams,llm_setting_main: LLMSetting,llm_prompt_for_text: LLMPrompt,) -> Coroutine[Any, Any, str]: ...@@ -55,6 +64,7 @@ class ContentGeneratorProtocol(Protocol):def make_object(self,job_metadata: JobMetadata,+ cogt_run_params: CogtRunParams,object_class: type[BaseModelTypeVar],llm_setting_for_object: LLMSetting,llm_prompt_for_object: LLMPrompt,@@ -63,28 +73,17 @@ class ContentGeneratorProtocol(Protocol):def make_object_list(self,job_metadata: JobMetadata,+ cogt_run_params: CogtRunParams,object_class: type[BaseModelTypeVar],llm_setting_for_object_list: LLMSetting,llm_prompt_for_object_list: LLMPrompt,nb_items: int | None = None,) -> Coroutine[Any, Any, list[BaseModelTypeVar]]: ...- async def make_image_content(- self,- job_metadata: JobMetadata,- generated_image_raw_details: GeneratedImageRawDetails,- img_gen_prompt: ImgGenPrompt | None,- ) -> ImageContent: ...-- async def make_page_contents(- self,- job_metadata: JobMetadata,- extract_output: ExtractOutput,- ) -> list[PageContent]: ...-def make_single_image(self,job_metadata: JobMetadata,+ cogt_run_params: CogtRunParams,img_gen_handle: str,img_gen_prompt: ImgGenPrompt,img_gen_job_params: ImgGenJobParams | None = None,@@ -94,6 +93,7 @@ class ContentGeneratorProtocol(Protocol):def make_image_list(self,job_metadata: JobMetadata,+ cogt_run_params: CogtRunParams,img_gen_handle: str,img_gen_prompt: ImgGenPrompt,nb_images: int,@@ -104,6 +104,7 @@ class ContentGeneratorProtocol(Protocol):def make_templated_text(self,job_metadata: JobMetadata,+ cogt_run_params: CogtRunParams,context: dict[str, Any],template: str,templating_style: TemplatingStyle | None = None,@@ -113,6 +114,7 @@ class ContentGeneratorProtocol(Protocol):def make_render_page_views(self,job_metadata: JobMetadata,+ cogt_run_params: CogtRunParams,extract_input: ExtractInput,extract_handle: str,extract_job_params: ExtractJobParams | None = None,@@ -122,6 +124,7 @@ class ContentGeneratorProtocol(Protocol):def make_extract_pages(self,job_metadata: JobMetadata,+ cogt_run_params: CogtRunParams,extract_input: ExtractInput,extract_handle: str,extract_job_params: ExtractJobParams,diff --git a/pipelex/cogt/content_generation/dry_mock.py b/pipelex/cogt/content_generation/dry_mock.pyindex 512bb3050..e84423aa7 100644--- a/pipelex/cogt/content_generation/dry_mock.py+++ b/pipelex/cogt/content_generation/dry_mock.py@@ -1,38 +1,59 @@-"""Leaf-level inference mocking shared by ``--dry-run`` and ``--mock-inference``.--Two triggers, one mechanism. Both fake the AI call *at the cogt leaf* — the-lowest point where ``ContentGenerator`` (direct) and the Temporal activities-(``act_llm_gen_*``) converge — so a single branch covers both execution modes:--- ``--dry-run`` (``run_mode=DRY``): ``ContentGeneratorDry`` is swapped in- pre-dispatch by each operator; it reports a **zero-token** synthetic LLM job- via :func:`report_dry_llm_job`. Zero tokens ⇒ ``AggregatedCosts.has_reportable_usage``- is False ⇒ the end-of-run cost report is suppressed (correct: a dry run did no- real work).-- ``--mock-inference`` (``run_mode=LIVE`` + ``JobMetadata.is_mock_inference``):- operators dispatch normally and the leaf itself routes to the- :func:`mock_llm_gen_text` / :func:`mock_llm_gen_object` /- :func:`mock_llm_gen_object_list` helpers here, which report **non-zero**- synthetic usage via :func:`report_mock_inference_llm_job`. Non-zero tokens ⇒ a- cost report *renders*. This is the durable reason the two modes differ at the- reporting layer: only a non-zero mock can validate cross-worker cost-report- rendering cheaply and deterministically (no provider spend).--This module is the single home for "what a mocked inference produces". The-follow-up that makes ``run_mode=DRY`` honor the backend-(``wip/dry-run-refactor/followup-leaf-run-mode-mock.md``) re-keys the leaf branch-from ``is_mock_inference`` to ``run_mode`` and folds ``ContentGeneratorDry`` into-these helpers — so the helpers, not the call sites, are the load-bearing piece.+"""Leaf-level inference mocking for ``--dry-run`` (``run_mode=DRY``).++One non-live mode, one mechanism. The dry run fakes the AI call *at the cogt+leaf* — the lowest point where ``ContentGenerator`` (direct, inline) and the+Temporal activities (``act_llm_gen_*`` and friends) converge — so a single+branch covers both backends and ``run_mode`` stays orthogonal to backend choice+(D-plan §3.5). ``CogtRunParams.run_mode == DRY`` rides every assignment; the+leaf routes to the :func:`dry_llm_gen_text` / :func:`dry_llm_gen_object` / ...+helpers here. Non-LLM dry leaves (img-gen / extract / render / search /+templating) mint synthetic outputs without reporting usage. For img-gen and+extract, the DRY branch lives at the ``*_and_store`` layer — one step above the+raw provider leaf — so a dry run performs **no storage IO** (eng review D10).++Usage reporting is keyed on the internal ``CogtRunParams.is_mock_usage``+sub-flag (DRY-only; the only CLI access is the hidden ``--mock-usage`` test+trigger, which requires ``--dry-run`` and is not shown in --help):++- ``is_mock_usage=False`` (default): the LLM leaves report a **zero-token**+ synthetic job via :func:`report_dry_llm_job`. Zero tokens ⇒+ ``AggregatedCosts.has_reportable_usage`` is False ⇒ the end-of-run cost report+ is suppressed (correct: a dry run did no real work).+- ``is_mock_usage=True``: the LLM leaves report **non-zero** synthetic usage via+ :func:`report_mock_usage_llm_job`. Non-zero tokens ⇒ a cost report *renders*.+ Only a non-zero mock can validate cross-worker cost-report rendering cheaply+ and deterministically (no provider spend) — that is this flag's whole reason+ to exist. Non-LLM leaves keep their no-usage dry behavior either way.++This module is the single home for "what a mocked inference produces". Object+mocks are built from the **schema-reconstructed** class on both backends (one+code path, identical mock everywhere); exotic format constraints must declare+``examples`` / ``mock_format`` — see ``DryRunObjectFidelityError``."""+from collections.abc import Callable, Sequencefrom datetime import datetime+from typing import Any-from pydantic import BaseModel+from polyfactory.exceptions import FactoryException+from pydantic import BaseModel, ValidationErrorfrom pipelex import log-from pipelex.cogt.content_generation.assignment_models import LLMAssignment, ObjectAssignment+from pipelex.cogt.content_generation.assignment_models import (+ ExtractAssignment,+ ImgGenAssignment,+ LLMAssignment,+ ObjectAssignment,+ RenderPageViewsAssignment,+ SearchAssignment,+ SearchObjectAssignment,+ TemplatingAssignment,+)+from pipelex.cogt.content_generation.cogt_run_params import CogtRunParamsfrom pipelex.cogt.content_generation.dry_run_factory import DryRunFactory+from pipelex.cogt.content_generation.exceptions import DryRunMockBuildErrorfrom pipelex.cogt.content_generation.schema_to_model_factory import SchemaToModelFactory+from pipelex.cogt.image.image_size import ImageSizefrom pipelex.cogt.llm.llm_job import LLMJobfrom pipelex.cogt.llm.llm_job_components import LLMJobConfig, LLMJobReportfrom pipelex.cogt.llm.llm_prompt import LLMPrompt@@ -40,22 +61,33 @@ from pipelex.cogt.llm.llm_report import LLMTokensUsagefrom pipelex.cogt.llm.llm_setting import LLMSettingfrom pipelex.cogt.usage.token_category import NbTokensByCategoryDict, TokenCategoryfrom pipelex.config import get_config+from pipelex.core.stuffs.document_content import DocumentContent+from pipelex.core.stuffs.image_content import ImageContent+from pipelex.core.stuffs.page_content import PageContent+from pipelex.core.stuffs.search_result_content import SearchResultContent+from pipelex.core.stuffs.text_and_images_content import TextAndImagesContent+from pipelex.core.stuffs.text_content import TextContentfrom pipelex.hub import get_report_delegatefrom pipelex.pipeline.job_metadata import JobMetadata+from pipelex.tools.jinja2.jinja2_parsing import check_jinja2_parsingfrom pipelex.tools.typing.pydantic_utils import BaseModelTypeVar+# The pipe code every mocked bundle answers to — shared with BundleHeaderSpec.main_pipe's+# examples so bundle dry-validation's mocked header names a pipe that exists (D3).+MOCK_MAIN_PIPE_CODE = "mock_main"+# Sentinel model identifiers so a synthetic usage record is never confused with real inference.DRY_RUN_INFERENCE_MODEL_NAME = "dry_run"DRY_RUN_INFERENCE_MODEL_ID = "dry_run"-MOCK_INFERENCE_MODEL_NAME = "mock_inference"-MOCK_INFERENCE_MODEL_ID = "mock_inference"+MOCK_USAGE_MODEL_NAME = "mock_usage"+MOCK_USAGE_MODEL_ID = "mock_usage"-# Synthetic, deterministic, clearly non-real token counts for ``--mock-inference``. Non-zero so the+# Synthetic, deterministic, clearly non-real token counts for ``is_mock_usage=True``. Non-zero so the# assembled usage is reportable (``AggregatedCosts.has_reportable_usage`` True → the cost report renders),-# which is exactly what ``--dry-run``'s zero-token usage deliberately suppresses. Input/output differ so-# the rendered table distinguishes the two columns. Cost stays 0 (``unit_costs={}``): a mocked run has-# token usage but no real spend — the "free model" reporting case.-MOCK_INFERENCE_NB_TOKENS_BY_CATEGORY: NbTokensByCategoryDict = {+# which is exactly what the default dry run's zero-token usage deliberately suppresses. Input/output+# differ so the rendered table distinguishes the two columns. Cost stays 0 (``unit_costs={}``): a mocked+# run has token usage but no real spend — the "free model" reporting case.+MOCK_USAGE_NB_TOKENS_BY_CATEGORY: NbTokensByCategoryDict = {TokenCategory.INPUT: 100,TokenCategory.OUTPUT: 50,}@@ -73,7 +105,7 @@ def _report_synthetic_llm_job("""Build a synthetic ``LLMJob`` and report it through ``get_report_delegate()``.Shared core of :func:`report_dry_llm_job` (zero tokens) and- :func:`report_mock_inference_llm_job` (non-zero tokens). Reporting it makes the runner-side+ :func:`report_mock_usage_llm_job` (non-zero tokens). Reporting it makes the runner-sidecross-worker emission path observable without a real LLM call.The synthetic ``job_metadata`` copy gets ``completed_at`` set so ``report_inference_job`` can@@ -120,88 +152,273 @@ def report_dry_llm_job(job_metadata: JobMetadata, llm_setting: LLMSetting, llm_p)-def report_mock_inference_llm_job(job_metadata: JobMetadata, llm_setting: LLMSetting, llm_prompt: LLMPrompt) -> None:- """Report a non-zero synthetic LLM job for a ``--mock-inference`` call (cost report renders)."""+def report_mock_usage_llm_job(job_metadata: JobMetadata, llm_setting: LLMSetting, llm_prompt: LLMPrompt) -> None:+ """Report a non-zero synthetic LLM job for a dry run with ``is_mock_usage=True`` (cost report renders)."""_report_synthetic_llm_job(job_metadata=job_metadata,llm_setting=llm_setting,llm_prompt=llm_prompt,- inference_model_name=MOCK_INFERENCE_MODEL_NAME,- inference_model_id=MOCK_INFERENCE_MODEL_ID,- nb_tokens_by_category=dict(MOCK_INFERENCE_NB_TOKENS_BY_CATEGORY),+ inference_model_name=MOCK_USAGE_MODEL_NAME,+ inference_model_id=MOCK_USAGE_MODEL_ID,+ nb_tokens_by_category=dict(MOCK_USAGE_NB_TOKENS_BY_CATEGORY),)-def build_mock_object(model_class: type[BaseModelTypeVar]) -> BaseModelTypeVar:+def build_mock_object(model_class: type[BaseModelTypeVar], **field_values: Any) -> BaseModelTypeVar:"""Build one mock instance of ``model_class`` via the dry-run polyfactory.Runs validators so the mock is valid; fields with format constraints(snake_case, PascalCase, ...) should declare ``examples`` / ``mock_format`` so- polyfactory uses those instead of random strings.- """- return DryRunFactory.make_dry_run_factory(model_class).build()+ polyfactory uses those instead of random strings. ``field_values`` pin specific+ fields instead of generating them.+ A build failure is deterministic (retries can never succeed), so it is wrapped+ into the typed :class:`DryRunMockBuildError` naming the class and the remedy;+ the activity error boundary then makes it terminal (eng review D7, listed in+ ``non_retryable_error_types``).+ """+ try:+ return DryRunFactory.make_dry_run_factory(model_class).build(**field_values)+ except (ValidationError, FactoryException) as exc:+ raise DryRunMockBuildError.for_object_class(model_class.__name__) from exc-def _mock_text(*, llm_prompt: LLMPrompt, llm_setting: LLMSetting) -> str:- truncate_length = get_config().pipelex.dry_run_config.text_gen_truncate_length- prompt_truncated = llm_prompt.desc(truncate_text_length=truncate_length)- return f"MOCK INFERENCE • llm_setting={llm_setting.desc()} • prompt={prompt_truncated}"+def build_mock_objects(model_class: type[BaseModelTypeVar], count: int) -> list[BaseModelTypeVar]:+ """Build ``count`` mock instances with a single factory construction.-def mock_llm_gen_text(llm_assignment: LLMAssignment) -> str:- """Leaf mock for ``llm_gen_text``: synthetic text + reportable usage, no provider call."""- job_metadata = llm_assignment.job_metadata- log.verbose(f"🤡 MOCK INFERENCE: llm_gen_text for '{job_metadata.pipeline_run_id}'")- report_mock_inference_llm_job(- job_metadata=job_metadata,- llm_setting=llm_assignment.llm_setting,- llm_prompt=llm_assignment.llm_prompt,- )- return _mock_text(llm_prompt=llm_assignment.llm_prompt, llm_setting=llm_assignment.llm_setting)+ ``DryRunFactory.make_dry_run_factory`` recursively scans the model tree and mints dynamic+ factory classes — building it once per list (not once per item) keeps list mocks linear in+ item construction only. Same :class:`DryRunMockBuildError` wrap as :func:`build_mock_object`.+ """+ factory = DryRunFactory.make_dry_run_factory(model_class)+ try:+ return [factory.build() for _ in range(count)]+ except (ValidationError, FactoryException) as exc:+ raise DryRunMockBuildError.for_object_class(model_class.__name__) from exc-def _reconstruct_class_and_report(object_assignment: ObjectAssignment) -> type[BaseModel]:- """Reconstruct the object's model class from its JSON schema and report one synthetic usage event.+def _reconstruct_object_class(object_assignment: ObjectAssignment) -> type[BaseModel]:+ """Reconstruct the object's model class from its JSON schema.- Shared by both object mocks. The leaf carries only the JSON schema (not the original class), so the- class is rebuilt via :class:`SchemaToModelFactory`. Reports exactly once — the live leaf makes one- ``gen_object`` call (a list is one call against a list-wrapper schema), so a single ``UsageReportEvent``- matches the real one-call topology that cross-worker assertions count.+ The leaf carries only the JSON schema (not the original class), so the class is rebuilt via+ :class:`SchemaToModelFactory`. This is the single schema-based mock site: both backends build+ the same mock, and fidelity bugs surface in cheap local unit tests instead of only on a worker+ (pre-flight decision 2)."""- llm_assignment = object_assignment.llm_assignment_for_object- content_class = SchemaToModelFactory.make_from_json_schema(+ return SchemaToModelFactory.make_from_json_schema(schema=object_assignment.object_class_schema,class_name=object_assignment.object_class_name,)- report_mock_inference_llm_job(- job_metadata=llm_assignment.job_metadata,- llm_setting=llm_assignment.llm_setting,- llm_prompt=llm_assignment.llm_prompt,- )- return content_class-def mock_llm_gen_object(object_assignment: ObjectAssignment) -> BaseModel:- """Leaf mock for ``llm_gen_object``: a polyfactory-built instance of the schema model + reportable usage.+def stamp_mock_main_coordination(items: Sequence[Any]) -> None:+ """Set the first item's ``pipe_code`` to ``"mock_main"`` — the single home of this coordination (D3).++ WHY: bundle dry-validation mocks a ``BundleHeaderSpec`` whose ``main_pipe`` field declares+ ``examples=["mock_main"]`` (``pipelex/builder/bundle_header_spec.py``), so the polyfactory mock+ header names ``mock_main`` as the bundle's main pipe. Every mock that fabricates a *list of pipe+ specs* must therefore make its first item answer to that name, or the mocked bundle fails its own+ main-pipe check. Callers: the mock-input factory (``working_memory_factory``), the batch+ controller's dry aggregation (``pipe_batch``), and the dry object-list leaf mock+ (:func:`dry_llm_gen_object_list`). The stamp is a no-op for items without a ``pipe_code`` field.+ """+ if items and hasattr(items[0], "pipe_code"):+ items[0].pipe_code = MOCK_MAIN_PIPE_CODE+++def _nb_list_items(object_assignment: ObjectAssignment) -> int:+ """Resolve the object-list mock length: the assignment's fixed ``nb_items`` wins (D11), including 0."""+ if object_assignment.nb_items is not None:+ return object_assignment.nb_items+ return get_config().pipelex.dry_run_config.nb_list_items+++_ReportLLMJobFunc = Callable[[JobMetadata, LLMSetting, LLMPrompt], None]+++def _leaf_gen_object(object_assignment: ObjectAssignment, report_func: _ReportLLMJobFunc) -> BaseModel:+ """Shared object-mock pipeline: reconstruct the class from its schema, report once, build one mock.Built from the schema-reconstructed class (the leaf carries only the JSON schema, not the originalclass), so format hints encoded via ``json_schema_extra`` that datamodel-code-generator drops onround-trip are not honored — exotic-format schemas may yield mock data the original class would- reject. ``ContentGenerator.make_object`` re-validates against the original class and re-raises that- failure as ``MockInferenceObjectFidelityError`` (review F2); the direct ``ContentGeneratorDry``- dry-run path keeps full fidelity by building the original class. Durable fix:- followup-leaf-run-mode-mock.md §8.+ reject. The generator re-validates against the original class and re-raises that failure as+ ``DryRunObjectFidelityError`` (review F2); declare ``examples`` / ``mock_format`` on the+ constrained fields to fix it.++ Reports exactly once — the live leaf makes one ``gen_object`` call (a list is one call against a+ list-wrapper schema), so a single ``UsageReportEvent`` matches the real one-call topology that+ cross-worker assertions count.+ """+ item_class = _reconstruct_object_class(object_assignment)+ mock_object = build_mock_object(item_class)+ # Report only after a successful build — a failed mock must not leave a usage event behind.+ llm_assignment = object_assignment.llm_assignment_for_object+ report_func(llm_assignment.job_metadata, llm_assignment.llm_setting, llm_assignment.llm_prompt)+ return mock_object+++def _leaf_gen_object_list(object_assignment: ObjectAssignment, report_func: _ReportLLMJobFunc) -> list[BaseModel]:+ """List counterpart of :func:`_leaf_gen_object`: one report, ``nb_items`` builds (D11)."""+ item_class = _reconstruct_object_class(object_assignment)+ mock_objects = build_mock_objects(item_class, _nb_list_items(object_assignment))+ # Report only after a successful build — a failed mock must not leave a usage event behind.+ llm_assignment = object_assignment.llm_assignment_for_object+ report_func(llm_assignment.job_metadata, llm_assignment.llm_setting, llm_assignment.llm_prompt)+ return mock_objects+++# --- Dry leaf helpers (``run_mode == DRY``) -------------------------------------------------------+#+# Each helper mints the synthetic output for one leaf, so a dry run behaves identically inline+# (direct backend) and inside a Temporal activity (the backend dispatches normally; the leaf mocks).+++def _dry_report_func(cogt_run_params: CogtRunParams) -> _ReportLLMJobFunc:+ """Select the synthetic-job report func for a dry LLM leaf on the ``is_mock_usage`` sub-flag.++ Default: zero-token (cost report suppressed). ``is_mock_usage=True``: non-zero sentinel counts+ (cost report renders) — the cross-worker cost-report validation affordance.+ """+ if cogt_run_params.is_mock_usage:+ return report_mock_usage_llm_job+ return report_dry_llm_job+++def _dry_text_gen_truncate_length() -> int:+ return get_config().pipelex.dry_run_config.text_gen_truncate_length+++def dry_llm_gen_text(llm_assignment: LLMAssignment) -> str:+ """Dry leaf for ``llm_gen_text``: synthetic job report + a ``DRY RUN:`` marker string."""+ job_metadata = llm_assignment.job_metadata+ log.verbose(f"🤡 DRY RUN: llm_gen_text for '{job_metadata.pipeline_run_id}'")+ report_func = _dry_report_func(llm_assignment.cogt_run_params)+ report_func(job_metadata, llm_assignment.llm_setting, llm_assignment.llm_prompt)+ prompt_truncated = llm_assignment.llm_prompt.desc(truncate_text_length=_dry_text_gen_truncate_length())+ return f"DRY RUN: llm_gen_text • llm_setting={llm_assignment.llm_setting.desc()} • prompt={prompt_truncated}"+++def dry_llm_gen_object(object_assignment: ObjectAssignment) -> BaseModel:+ """Dry leaf for ``llm_gen_object``: schema-built mock instance + synthetic job report."""+ log.verbose(f"🤡 DRY RUN: llm_gen_object for '{object_assignment.object_class_name}'")+ return _leaf_gen_object(object_assignment, report_func=_dry_report_func(object_assignment.cogt_run_params))+++def dry_llm_gen_object_list(object_assignment: ObjectAssignment) -> list[BaseModel]:+ """Dry leaf for ``llm_gen_object_list``: ``nb_items`` schema-built mocks + one synthetic report.++ Applies :func:`stamp_mock_main_coordination` so bundle dry-validation's mocked+ ``BundleHeaderSpec.main_pipe`` check passes through the leaf mock (D3). The stamp is+ unconditional on ``is_mock_usage`` — it only matters to bundle dry-validation and is+ harmless elsewhere.+ """+ log.verbose(f"🤡 DRY RUN: llm_gen_object_list for '{object_assignment.object_class_name}'")+ items = _leaf_gen_object_list(object_assignment, report_func=_dry_report_func(object_assignment.cogt_run_params))+ stamp_mock_main_coordination(items)+ return items+++def dry_templating_gen_text(templating_assignment: TemplatingAssignment) -> str:+ """Dry leaf for ``templating_gen_text``: parse-check the template (a bad jinja2 template must+ still fail under DRY), then return a marker string instead of rendering."""- return build_mock_object(_reconstruct_class_and_report(object_assignment))---def mock_llm_gen_object_list(object_assignment: ObjectAssignment) -> list[BaseModel]:- """Leaf mock for ``llm_gen_object_list``: ``nb_list_items`` builds + one reportable usage event."""- item_class = _reconstruct_class_and_report(object_assignment)- nb_list_items = get_config().pipelex.dry_run_config.nb_list_items- # Unlike ``ContentGeneratorDry.make_object_list``, the first item's ``pipe_code`` is intentionally NOT- # set to ``"mock_main"``: that coordination satisfies ``BundleHeaderSpec.main_pipe`` during bundle- # dry-validation, which runs under ``run_mode=DRY`` (``ContentGeneratorDry``, never this leaf mock).- # ``--mock-inference`` is a LIVE run and never drives bundle dry-validation, so there is no main_pipe- # check to satisfy here.- return [build_mock_object(item_class) for _ in range(nb_list_items)]+ check_jinja2_parsing(+ template_source=templating_assignment.template,+ template_category=templating_assignment.category,+ )+ log.verbose("🤡 DRY RUN: templating_gen_text")+ jinja2_truncated = templating_assignment.template[: _dry_text_gen_truncate_length()]+ # Context KEYS only: the context is built from working memory, so dumping values would leak+ # inputs the real template never renders (and bloat the mock output).+ context_keys = sorted(templating_assignment.context.keys())+ return (+ f"DRY RUN: templating_gen_text • context_keys={context_keys} • "+ f"jinja2={jinja2_truncated} • templating_style={templating_assignment.templating_style} • "+ f"template_category={templating_assignment.category}"+ )+++def _dry_image_content(image_url: str, img_gen_assignment: ImgGenAssignment | None = None) -> ImageContent:+ image_content = ImageContent(+ url=image_url,+ public_url=image_url,+ mime_type="image/jpeg",+ size=ImageSize(width=1024, height=1024),+ )+ if img_gen_assignment:+ image_content.source_prompt = img_gen_assignment.img_gen_prompt.positive_text+ image_content.source_negative_prompt = img_gen_assignment.img_gen_prompt.negative_text+ return image_content+++def dry_img_gen_image_contents(img_gen_assignment: ImgGenAssignment) -> list[ImageContent]:+ """Dry leaf for image generation: URL-only ``ImageContent`` mocks, no provider, no storage IO.++ Sits at the ``*_and_store`` layer — one step above the raw provider leaf — so a dry run never+ touches the storage provider (eng review D10).+ """+ log.verbose(f"🤡 DRY RUN: img_gen for '{img_gen_assignment.img_gen_handle}'")+ image_urls = get_config().pipelex.dry_run_config.image_urls+ return [+ _dry_image_content(image_url=image_urls[image_index % len(image_urls)], img_gen_assignment=img_gen_assignment)+ for image_index in range(img_gen_assignment.nb_images)+ ]+++def dry_extract_page_contents(extract_assignment: ExtractAssignment) -> list[PageContent]:+ """Dry leaf for document extraction: synthetic ``PageContent`` mocks, no provider, no storage IO.++ Sits at the ``*_and_store`` layer (eng review D10). Page views are attached above the leaf by the+ generator-level page-view logic, exactly as in a LIVE run.+ """+ log.verbose(f"🤡 DRY RUN: extract_gen_pages for '{extract_assignment.extract_handle}'")+ nb_pages: int+ if extract_assignment.extract_input.image_uri:+ nb_pages = 1+ else:+ nb_pages = get_config().pipelex.dry_run_config.nb_extract_pages+ return [+ PageContent(+ text_and_images=TextAndImagesContent(+ text=TextContent(text="DRY RUN: OCR text"),+ images=[],+ ),+ page_view=None,+ )+ for _ in range(nb_pages)+ ]+++def dry_render_page_views(render_assignment: RenderPageViewsAssignment) -> list[ImageContent]:+ """Dry leaf for page-view rendering: URL-only page-view image mocks, no pdf rendering, no storage IO.++ Fake URLs come from ``dry_run_config.image_urls`` (validated non-empty) — the single configured+ source of truth for dry fake images, same as the img-gen mock.+ """+ log.verbose(f"🤡 DRY RUN: render_page_views for '{render_assignment.job_metadata.pipeline_run_id}'")+ nb_pages = get_config().pipelex.dry_run_config.nb_extract_pages+ image_urls = get_config().pipelex.dry_run_config.image_urls+ return [_dry_image_content(image_url=image_urls[page_index % len(image_urls)]) for page_index in range(nb_pages)]+++def dry_search_gen_sourced_answer(search_assignment: SearchAssignment) -> SearchResultContent:+ """Dry leaf for sourced-answer search: polyfactory-built result with mock sources, no provider."""+ log.verbose(f"🤡 DRY RUN: search_gen_sourced_answer for '{search_assignment.search_handle}'")+ nb_sources = get_config().pipelex.dry_run_config.nb_list_items+ mock_sources = build_mock_objects(DocumentContent, nb_sources)+ return build_mock_object(SearchResultContent, sources=mock_sources)+++def dry_search_gen_structured(search_object_assignment: SearchObjectAssignment) -> dict[str, Any]:+ """Dry leaf for structured search: schema-built mock dumped to a dict, no provider.++ Returns a raw dict (the leaf contract) which the submitter re-validates against the original+ output structure class, matching the live path.+ """+ log.verbose(f"🤡 DRY RUN: search_gen_structured for '{search_object_assignment.output_class_name}'")+ output_class = SchemaToModelFactory.make_from_json_schema(+ schema=search_object_assignment.output_class_schema,+ class_name=search_object_assignment.output_class_name,+ )+ return build_mock_object(output_class).model_dump(mode="json")diff --git a/pipelex/cogt/content_generation/exceptions.py b/pipelex/cogt/content_generation/exceptions.pyindex c20877ce5..8012cb2d2 100644--- a/pipelex/cogt/content_generation/exceptions.py+++ b/pipelex/cogt/content_generation/exceptions.py@@ -9,63 +9,61 @@ class UnsafeSchemaError(SecurityError):pass-class MockInferenceUnsupportedError(PipelexError):- """Raised when ``--mock-inference`` reaches an inference operation that has no leaf-level mock.+class DryRunObjectFidelityError(PipelexError):+ """Raised when a dry-run synthetic object fails re-validation against its original class.- ``--mock-inference`` is honored only at the LLM leaf, which returns synthetic text/objects with- reportable usage and never calls a provider. Image generation, document extraction, and web search- have no such leaf mock yet, so under ``--mock-inference`` they would dispatch to the **real** provider- and spend real money — the exact opposite of what the flag promises. This guard fails loud at those- leaves instead of silently spending. Full no-spend coverage of every operator is available today via- ``--dry-run``; per-operator leaf mocks land later (``wip/dry-run-refactor/followup-leaf-run-mode-mock.md``,- via ``run_mode=DRY``).+ The dry object leaf mock builds its instance from the **schema-reconstructed** class (the leaf+ carries only the JSON schema, not the original class). ``ContentGenerator.make_object`` then+ re-validates that data against the **original** class. Invariants the JSON-schema round-trip cannot+ capture — custom ``@field_validator`` / ``@model_validator`` logic, or format / pattern hints encoded+ via ``json_schema_extra`` that datamodel-code-generator drops — are absent from the reconstructed+ class, so polyfactory can fill a value the reconstructed class accepts but the original rejects. This+ turns an opaque mid-run ``pydantic.ValidationError`` into a clear, actionable message that names the+ class and the remedy. Scoped to the dry path only: a LIVE provider's invalid output keeps its existing+ ``ValidationError`` behavior."""- # The caller picked an unsupported flag/operation combination they can fix (drop --mock-inference- # for this pipe, or use --dry-run) — caller-fixable input, not a server-side fault.+ # The caller's output concept enforces an invariant the schema round-trip drops; they can fix it by+ # declaring `examples` / `mock_format` on the offending fields — caller-fixable input, not a+ # server-side fault.error_domain = ErrorDomain.INPUT- # The message is pure caller-facing guidance (names the operation, points at --dry-run) with no- # internal paths or secrets, so it should survive STRICT disclosure intact.+ # The message is pure caller-facing guidance (names the caller's own output concept class and the+ # field-level remedy) with no internal paths or secrets, so it should survive STRICT disclosure intact._authors_caller_facing_message = True@classmethod- def for_operation(cls, operation: str) -> "MockInferenceUnsupportedError":- """Build the guard error for a named inference operation (e.g. ``"image generation (PipeImgGen)"``)."""+ def for_object_class(cls, object_class_name: str) -> "DryRunObjectFidelityError":+ """Build the fidelity error for a named output object class (e.g. ``"InvoiceLine"``)."""return cls(- f"--mock-inference does not support {operation}: it has no leaf-level mock, so the run would "- "call the real provider and spend. Use --dry-run for full no-spend coverage."+ f"The dry-run leaf mock built a synthetic '{object_class_name}' that failed re-validation against "+ "the original class: the mock is generated from the JSON schema, which can drop invariants the class "+ "enforces (custom validators, json_schema_extra format/pattern hints). Declare `examples` or "+ "`mock_format` on the constrained fields so the mock factory produces valid values.")-class MockInferenceObjectFidelityError(PipelexError):- """Raised when a ``--mock-inference`` synthetic object fails re-validation against its original class.+class DryRunMockBuildError(PipelexError):+ """Raised when building a leaf mock object fails (polyfactory build or model validation).- The object leaf mock builds its instance from the **schema-reconstructed** class (the leaf carries- only the JSON schema, not the original class). ``ContentGenerator.make_object`` then re-validates that- data against the **original** class. Invariants the JSON-schema round-trip cannot capture — custom- ``@field_validator`` / ``@model_validator`` logic, or format / pattern hints encoded via- ``json_schema_extra`` that datamodel-code-generator drops — are absent from the reconstructed class, so- polyfactory can fill a value the reconstructed class accepts but the original rejects. This turns an- opaque mid-run ``pydantic.ValidationError`` into a clear, actionable message that names the class and- points at the no-spend alternative. Scoped to the mock path only: a LIVE provider's invalid output keeps- its existing ``ValidationError`` behavior. The durable fix (build against the original class in-process)- lands with the leaf run-mode follow-up (``wip/dry-run-refactor/followup-leaf-run-mode-mock.md`` §8).+ The leaf mock builds objects via polyfactory from the schema-reconstructed class. A build failure+ here is deterministic — retrying it (e.g. Temporal activity retries) can never succeed — so it is+ surfaced as a typed ``PipelexError``, which the activity error boundary converts to a terminal+ ``ApplicationError``. The message names the class and the remedy: declare ``examples`` or+ ``mock_format`` on fields whose constraints polyfactory cannot satisfy from the schema alone."""- # The caller picked --mock-inference for a pipe whose output concept enforces an invariant the schema- # round-trip drops; they can fix it by using --dry-run (full-fidelity mock) for this pipe — caller-fixable- # input, not a server-side fault.+ # The caller's output concept has field constraints the mock factory cannot satisfy; fixable by+ # declaring `examples` / `mock_format` on those fields — caller-fixable input, not a server fault.error_domain = ErrorDomain.INPUT- # The message is pure caller-facing guidance (names the caller's own output concept class, points at- # --dry-run) with no internal paths or secrets, so it should survive STRICT disclosure intact.+ # The message is pure caller-facing guidance (names the caller's own output concept class and the+ # field-level remedy) with no internal paths or secrets, so it should survive STRICT disclosure intact._authors_caller_facing_message = True@classmethod- def for_object_class(cls, object_class_name: str) -> "MockInferenceObjectFidelityError":- """Build the fidelity error for a named output object class (e.g. ``"InvoiceLine"``)."""+ def for_object_class(cls, object_class_name: str) -> "DryRunMockBuildError":+ """Build the mock-build error for a named output object class (e.g. ``"InvoiceLine"``)."""return cls(- f"--mock-inference built a synthetic '{object_class_name}' that failed re-validation against the "- "original class: the mock is generated from the JSON schema, which can drop invariants the class "- "enforces (custom validators, json_schema_extra format/pattern hints). Use --dry-run for "- "full-fidelity mock objects (it builds the original class directly)."+ f"Failed to build a mock '{object_class_name}' for the dry/mocked inference leaf: polyfactory "+ "could not produce an instance that satisfies the class constraints. Declare `examples` or "+ "`mock_format` on the constrained fields so the mock factory produces valid values.")diff --git a/pipelex/cogt/content_generation/extract_generate.py b/pipelex/cogt/content_generation/extract_generate.pyindex f230033a2..1250940d9 100644--- a/pipelex/cogt/content_generation/extract_generate.py+++ b/pipelex/cogt/content_generation/extract_generate.py@@ -1,5 +1,5 @@from pipelex.cogt.content_generation.assignment_models import ExtractAssignment-from pipelex.cogt.content_generation.exceptions import MockInferenceUnsupportedError+from pipelex.cogt.content_generation.dry_mock import dry_extract_page_contentsfrom pipelex.cogt.content_generation.generated_content_factory import GeneratedContentFactoryfrom pipelex.cogt.extract.extract_job_factory import ExtractJobFactoryfrom pipelex.cogt.extract.extract_output import ExtractOutput@@ -8,9 +8,6 @@ from pipelex.hub import get_extract_workerasync def extract_gen_pages(extract_assignment: ExtractAssignment) -> ExtractOutput:- if extract_assignment.job_metadata.is_mock_inference:- error = MockInferenceUnsupportedError.for_operation("document extraction (PipeExtract)")- raise errorextract_worker = get_extract_worker(extract_handle=extract_assignment.extract_handle)extract_job = ExtractJobFactory.make_extract_job(extract_input=extract_assignment.extract_input,@@ -25,7 +22,14 @@ async def extract_gen_pages_and_store(extract_assignment: ExtractAssignment,generated_content_factory: GeneratedContentFactory,) -> list[PageContent]:- """Extract pages and store extracted images, returning PageContent with URLs (no raw binary data)."""+ """Extract pages and store extracted images, returning PageContent with URLs (no raw binary data).++ The DRY branch sits at the ``*_and_store`` layer, above the raw provider leaf, so a dry run+ performs no storage IO — see the ``dry_mock`` module docstring (eng review D10). Do not+ "unify" it downward into the raw leaf.+ """+ if extract_assignment.cogt_run_params.run_mode.is_dry:+ return dry_extract_page_contents(extract_assignment)extract_output = await extract_gen_pages(extract_assignment)return await generated_content_factory.make_page_contents(primary_id=extract_assignment.job_metadata.user_id,diff --git a/pipelex/cogt/content_generation/img_gen_generate.py b/pipelex/cogt/content_generation/img_gen_generate.pyindex 192366d44..a92781884 100644--- a/pipelex/cogt/content_generation/img_gen_generate.py+++ b/pipelex/cogt/content_generation/img_gen_generate.py@@ -1,6 +1,6 @@from pipelex import logfrom pipelex.cogt.content_generation.assignment_models import ImgGenAssignment-from pipelex.cogt.content_generation.exceptions import MockInferenceUnsupportedError+from pipelex.cogt.content_generation.dry_mock import dry_img_gen_image_contentsfrom pipelex.cogt.content_generation.generated_content_factory import GeneratedContentFactoryfrom pipelex.cogt.image.generated_image import GeneratedImageRawDetailsfrom pipelex.cogt.img_gen.img_gen_job_factory import ImgGenJobFactory@@ -9,9 +9,6 @@ from pipelex.hub import get_img_gen_workerasync def img_gen_single_image(img_gen_assignment: ImgGenAssignment) -> GeneratedImageRawDetails:- if img_gen_assignment.job_metadata.is_mock_inference:- error = MockInferenceUnsupportedError.for_operation("image generation (PipeImgGen)")- raise errorimg_gen_worker = get_img_gen_worker(img_gen_handle=img_gen_assignment.img_gen_handle)img_gen_job = ImgGenJobFactory.make_img_gen_job_from_prompt(img_gen_prompt=img_gen_assignment.img_gen_prompt,@@ -25,9 +22,6 @@ async def img_gen_single_image(img_gen_assignment: ImgGenAssignment) -> Generateasync def img_gen_image_list(img_gen_assignment: ImgGenAssignment) -> list[GeneratedImageRawDetails]:- if img_gen_assignment.job_metadata.is_mock_inference:- error = MockInferenceUnsupportedError.for_operation("image generation (PipeImgGen)")- raise errorimg_gen_worker = get_img_gen_worker(img_gen_handle=img_gen_assignment.img_gen_handle)img_gen_job = ImgGenJobFactory.make_img_gen_job_from_prompt(img_gen_prompt=img_gen_assignment.img_gen_prompt,@@ -43,17 +37,20 @@ async def img_gen_image_list(img_gen_assignment: ImgGenAssignment) -> list[Generreturn generated_image_list-async def img_gen_image(img_gen_assignment: ImgGenAssignment) -> GeneratedImageRawDetails | list[GeneratedImageRawDetails]:- if img_gen_assignment.nb_images > 1:- return await img_gen_image_list(img_gen_assignment)- return await img_gen_single_image(img_gen_assignment)--async def img_gen_single_image_and_store(img_gen_assignment: ImgGenAssignment,generated_content_factory: GeneratedContentFactory,) -> ImageContent:- """Generate a single image and store it, returning an ImageContent with URLs (no raw binary data)."""+ """Generate a single image and store it, returning an ImageContent with URLs (no raw binary data).++ The DRY branch sits at the ``*_and_store`` layer, above the raw provider leaf, so a dry run+ performs no storage IO — see the ``dry_mock`` module docstring (eng review D10). Do not+ "unify" it downward into the raw leaf.+ """+ if img_gen_assignment.cogt_run_params.run_mode.is_dry:+ # Exactly one mock, matching the live single path's one-provider-call semantics+ # regardless of the assignment's nb_images.+ return dry_img_gen_image_contents(img_gen_assignment.model_copy(update={"nb_images": 1}))[0]generated_image = await img_gen_single_image(img_gen_assignment)image_content = await generated_content_factory.make_image_content(primary_id=img_gen_assignment.job_metadata.user_id,@@ -69,7 +66,12 @@ async def img_gen_image_list_and_store(img_gen_assignment: ImgGenAssignment,generated_content_factory: GeneratedContentFactory,) -> list[ImageContent]:- """Generate multiple images and store them, returning ImageContent list with URLs (no raw binary data)."""+ """Generate multiple images and store them, returning ImageContent list with URLs (no raw binary data).++ DRY branch at the ``*_and_store`` layer — see ``img_gen_single_image_and_store`` (D10).+ """+ if img_gen_assignment.cogt_run_params.run_mode.is_dry:+ return dry_img_gen_image_contents(img_gen_assignment)generated_image_list = await img_gen_image_list(img_gen_assignment)image_contents: list[ImageContent] = []for raw_details in generated_image_list:diff --git a/pipelex/cogt/content_generation/llm_generate.py b/pipelex/cogt/content_generation/llm_generate.pyindex 059e9e046..ad106922b 100644--- a/pipelex/cogt/content_generation/llm_generate.py+++ b/pipelex/cogt/content_generation/llm_generate.py@@ -4,15 +4,19 @@ from pydantic import BaseModelfrom pipelex import logfrom pipelex.cogt.content_generation.assignment_models import LLMAssignment, ObjectAssignment-from pipelex.cogt.content_generation.dry_mock import mock_llm_gen_object, mock_llm_gen_object_list, mock_llm_gen_text+from pipelex.cogt.content_generation.dry_mock import (+ dry_llm_gen_object,+ dry_llm_gen_object_list,+ dry_llm_gen_text,+)from pipelex.cogt.content_generation.schema_to_model_factory import SchemaToModelFactoryfrom pipelex.cogt.llm.llm_job_factory import LLMJobFactoryfrom pipelex.hub import get_llm_workerasync def llm_gen_text(llm_assignment: LLMAssignment) -> str:- if llm_assignment.job_metadata.is_mock_inference:- return mock_llm_gen_text(llm_assignment)+ if llm_assignment.cogt_run_params.run_mode.is_dry:+ return dry_llm_gen_text(llm_assignment)llm_worker = get_llm_worker(llm_handle=llm_assignment.llm_handle)llm_job = LLMJobFactory.make_llm_job(job_metadata=llm_assignment.job_metadata,@@ -26,8 +30,8 @@ async def llm_gen_text(llm_assignment: LLMAssignment) -> str:async def llm_gen_object(object_assignment: ObjectAssignment) -> BaseModel:llm_assignment = object_assignment.llm_assignment_for_object- if llm_assignment.job_metadata.is_mock_inference:- return mock_llm_gen_object(object_assignment)+ if object_assignment.cogt_run_params.run_mode.is_dry:+ return dry_llm_gen_object(object_assignment)llm_worker = get_llm_worker(llm_handle=llm_assignment.llm_handle)llm_job = LLMJobFactory.make_llm_job(job_metadata=llm_assignment.job_metadata,@@ -47,8 +51,8 @@ async def llm_gen_object(object_assignment: ObjectAssignment) -> BaseModel:async def llm_gen_object_list(object_assignment: ObjectAssignment) -> list[BaseModel]:llm_assignment = object_assignment.llm_assignment_for_object- if llm_assignment.job_metadata.is_mock_inference:- return mock_llm_gen_object_list(object_assignment)+ if object_assignment.cogt_run_params.run_mode.is_dry:+ return dry_llm_gen_object_list(object_assignment)log.verbose(f"llm_gen_object_list to generate a list of '{object_assignment.object_class_name}'")llm_worker = get_llm_worker(llm_handle=llm_assignment.llm_handle)llm_job = LLMJobFactory.make_llm_job(diff --git a/pipelex/cogt/content_generation/render_generate.py b/pipelex/cogt/content_generation/render_generate.pynew file mode 100644index 000000000..3b6b980cd--- /dev/null+++ b/pipelex/cogt/content_generation/render_generate.py@@ -0,0 +1,50 @@+"""Framework-agnostic page-view rendering leaf, sibling of ``img_gen_generate``.++Single home for "render a PDF's pages as images and store them": the direct+``ContentGenerator`` calls it inline; the Temporal ``act_render_page_views`` activity calls it+on a worker. Rendering and storage happen inside the leaf so large binary data never crosses a+Temporal workflow boundary — only URL-bearing ``ImageContent`` references are returned.+"""++from pipelex.cogt.content_generation.assignment_models import RenderPageViewsAssignment+from pipelex.cogt.content_generation.dry_mock import dry_render_page_views+from pipelex.cogt.content_generation.generated_content_factory import GeneratedContentFactory+from pipelex.cogt.image.generated_image import GeneratedImageRawDetails+from pipelex.core.stuffs.image_content import ImageContent+from pipelex.tools.misc.image_utils import ImageFormat+++async def render_page_views_and_store(+ render_assignment: RenderPageViewsAssignment,+ generated_content_factory: GeneratedContentFactory,+) -> list[ImageContent]:+ """Render PDF pages as images, store them, and return lightweight ImageContent references.++ The DRY branch sits here, above both the pypdfium2 rendering and the store step, so a dry run+ performs no rendering and no storage IO (eng review D10).+ """+ if render_assignment.cogt_run_params.run_mode.is_dry:+ return dry_render_page_views(render_assignment)+ # Deferred import: avoid pulling the pdf rendering SDK at module-load time+ from pipelex.tools.pdf.pypdfium2_renderer import pypdfium2_renderer # noqa: PLC0415++ page_view_images = await pypdfium2_renderer.render_pdf_pages_from_uri(+ pdf_uri=render_assignment.document_uri,+ dpi=render_assignment.page_views_dpi,+ )+ image_contents: list[ImageContent] = []+ for page_view_image in page_view_images:+ raw_details = GeneratedImageRawDetails.make_from_pil_image(+ pil_image=page_view_image,+ image_format=ImageFormat.PNG,+ )+ # Free the rendered page's pixel buffer as soon as its bytes are extracted — a large PDF+ # otherwise holds every page bitmap in memory until the whole store loop finishes.+ page_view_image.close()+ image_content = await generated_content_factory.make_image_content(+ primary_id=render_assignment.job_metadata.user_id,+ secondary_id=render_assignment.job_metadata.pipeline_run_id,+ raw_details=raw_details,+ )+ image_contents.append(image_content)+ return image_contentsdiff --git a/pipelex/cogt/content_generation/search_generate.py b/pipelex/cogt/content_generation/search_generate.pyindex d61f88900..649ddfe70 100644--- a/pipelex/cogt/content_generation/search_generate.py+++ b/pipelex/cogt/content_generation/search_generate.py@@ -11,7 +11,7 @@ loop, which left search failures hanging the submitter — see ``wip/`` brief).from typing import Anyfrom pipelex.cogt.content_generation.assignment_models import SearchAssignment, SearchObjectAssignment-from pipelex.cogt.content_generation.exceptions import MockInferenceUnsupportedError+from pipelex.cogt.content_generation.dry_mock import dry_search_gen_sourced_answer, dry_search_gen_structuredfrom pipelex.cogt.content_generation.schema_to_model_factory import SchemaToModelFactoryfrom pipelex.cogt.model_backends.model_type import ModelTypefrom pipelex.cogt.search.search_job import SearchJob@@ -22,14 +22,6 @@ from pipelex.core.stuffs.search_result_content import SearchResultContentfrom pipelex.hub import get_model_deck-def _guard_no_mock_inference(search_assignment: SearchAssignment) -> None:- # Web search has no leaf-level mock — under --mock-inference (run_mode stays LIVE) this live- # path runs, so fail loud rather than silently hit the real provider (use --dry-run instead).- if search_assignment.job_metadata.is_mock_inference:- error = MockInferenceUnsupportedError.for_operation("web search (PipeSearch)")- raise error--def _make_search_worker(search_assignment: SearchAssignment) -> SearchWorkerAbstract:model_deck = get_model_deck()inference_model = model_deck.get_required_inference_model(@@ -52,7 +44,8 @@ def _make_search_job(search_assignment: SearchAssignment) -> SearchJob:async def search_gen_sourced_answer(search_assignment: SearchAssignment) -> SearchResultContent:- _guard_no_mock_inference(search_assignment)+ if search_assignment.cogt_run_params.run_mode.is_dry:+ return dry_search_gen_sourced_answer(search_assignment)worker = _make_search_worker(search_assignment)search_job = _make_search_job(search_assignment)return await worker.search_sourced_answer(search_job=search_job)@@ -66,7 +59,8 @@ async def search_gen_structured(search_object_assignment: SearchObjectAssignmentwhich sidesteps shipping a dynamic output class across the Temporal boundary."""search_assignment = search_object_assignment.search_assignment- _guard_no_mock_inference(search_assignment)+ if search_assignment.cogt_run_params.run_mode.is_dry:+ return dry_search_gen_structured(search_object_assignment)worker = _make_search_worker(search_assignment)search_job = _make_search_job(search_assignment)output_class = SchemaToModelFactory.make_from_json_schema(diff --git a/pipelex/cogt/content_generation/templating_generate.py b/pipelex/cogt/content_generation/templating_generate.pyindex ddec36a4e..96f4ce7f7 100644--- a/pipelex/cogt/content_generation/templating_generate.py+++ b/pipelex/cogt/content_generation/templating_generate.py@@ -1,8 +1,11 @@from pipelex.cogt.content_generation.assignment_models import TemplatingAssignment+from pipelex.cogt.content_generation.dry_mock import dry_templating_gen_textfrom pipelex.cogt.templating.template_rendering import render_templateasync def templating_gen_text(templating_assignment: TemplatingAssignment) -> str:+ if templating_assignment.cogt_run_params.run_mode.is_dry:+ return dry_templating_gen_text(templating_assignment)templated_text: str = await render_template(template=templating_assignment.template,category=templating_assignment.category,diff --git a/pipelex/core/memory/working_memory_factory.py b/pipelex/core/memory/working_memory_factory.pyindex a68f39954..cbccf09f8 100644--- a/pipelex/core/memory/working_memory_factory.py+++ b/pipelex/core/memory/working_memory_factory.py@@ -4,6 +4,7 @@ from polyfactory.exceptions import FactoryExceptionfrom pydantic import BaseModel, ValidationErrorfrom pipelex import log+from pipelex.cogt.content_generation.dry_mock import stamp_mock_main_coordinationfrom pipelex.cogt.content_generation.dry_run_factory import DryRunFactoryfrom pipelex.core.memory.exceptions import WorkingMemoryFactoryErrorfrom pipelex.core.memory.working_memory import MAIN_STUFF_NAME, StuffDict, WorkingMemory@@ -233,14 +234,8 @@ class WorkingMemoryFactory(BaseModel):else:nb_stuffs = typed_named_stuff_spec.multiplicity- items: list[StuffContent] = []- for idx in range(nb_stuffs):- item_mock_content = cls.make_mock_content(typed_named_stuff_spec)- # For the first item in pipe specs, set pipe_code to "mock_main"- # to match the mock main_pipe in BundleHeaderSpec- if idx == 0 and hasattr(item_mock_content, "pipe_code"):- item_mock_content.pipe_code = "mock_main" # pyright: ignore[reportAttributeAccessIssue]- items.append(item_mock_content)+ items: list[StuffContent] = [cls.make_mock_content(typed_named_stuff_spec) for _ in range(nb_stuffs)]+ stamp_mock_main_coordination(items)mock_list_content = ListContent[StuffContent](items=items)return StuffFactory.make_stuff(diff --git a/pipelex/graph/graph_tracer_manager.py b/pipelex/graph/graph_tracer_manager.pyindex b45e24786..5f0afe54b 100644--- a/pipelex/graph/graph_tracer_manager.py+++ b/pipelex/graph/graph_tracer_manager.py@@ -135,7 +135,6 @@ class GraphTracerManager(metaclass=ABCSingletonMeta):raise ValueError(msg)tracer = GraphTracer()- self._tracers[key] = tracertrace_context = tracer.setup(graph_id=graph_id,@@ -151,6 +150,10 @@ class GraphTracerManager(metaclass=ABCSingletonMeta):# Set the tracer_key on the TraceContext so downstream lookups use the same keyif tracer_key is not None:trace_context = trace_context.model_copy(update={"tracer_key": tracer_key})++ # Register only once setup has fully succeeded: a failure above must not leave+ # an unusable tracer entry in the process-wide manager.+ self._tracers[key] = tracerreturn trace_contextdef close_tracer(self, tracer_key: str) -> GraphSpec | None:diff --git a/pipelex/hub.py b/pipelex/hub.pyindex 3331b8377..ca4cbda0e 100644--- a/pipelex/hub.py+++ b/pipelex/hub.py@@ -49,6 +49,7 @@ if TYPE_CHECKING:from pipelex.pipe_run.pipe_router_protocol import PipeRouterProtocolfrom pipelex.pipe_run.pipe_run_protocol import PipeRunProtocol+ from pipelex.tracing.event_log_protocol import EventLogProtocolclass PipelexHub:@@ -74,6 +75,10 @@ class PipelexHub:self._inference_manager: InferenceManagerProtocolself._report_delegate: ReportingProtocolself._content_generator: ContentGeneratorProtocol | None = None+ # Keyless boot (``Pipelex.make(needs_inference=False)``) forces every run to DRY (eng+ # review D4): the backend still picks inline vs in-workflow on its own; the leaf mocks.+ # Consumed by ``PipeRunParamsFactory.make_run_params`` (the single writer of run_mode).+ self._is_dry_run_forced: bool = False# pipelexself._library_manager: LibraryManagerAbstract | None = None@@ -182,6 +187,12 @@ class PipelexHub:def set_content_generator(self, content_generator: ContentGeneratorProtocol):self._content_generator = content_generator+ def set_dry_run_forced(self, is_forced: bool) -> None:+ self._is_dry_run_forced = is_forced++ def is_dry_run_forced(self) -> bool:+ return self._is_dry_run_forced+# pipelexdef set_domain_library(self, domain_library: DomainLibraryAbstract):@@ -463,7 +474,38 @@ def get_report_delegate() -> ReportingProtocol:return get_pipelex_hub().get_report_delegate()+_content_generator_override: ContextVar[ContentGeneratorProtocol | None] = ContextVar("content_generator_override", default=None)+++@contextmanager+def scoped_content_generator(content_generator: ContentGeneratorProtocol) -> Generator[None, None, None]:+ """Set ``content_generator`` as the active generator for the scope, then restore the prior value on exit.++ Inference operators (PipeLLM / PipeImgGen / PipeExtract / PipeSearch / PipeStructure) resolve+ :func:`get_content_generator`; under a Temporal-enabled hub that default is+ ``ContentGeneratorInWorkflow``, which dispatches activities. An in-process run (e.g. the+ dry-run/validation activity body) wraps itself in this scope with an inline generator so its+ leaves never dispatch — the DRY mock lives at the cogt leaf, so the inline generator's leaves+ mock without dispatching and without storage IO. ContextVar-scoped like+ :func:`scoped_pipe_router`, so concurrent runs don't cross-contaminate.+ """+ prev = _content_generator_override.get()+ _content_generator_override.set(content_generator)+ try:+ yield+ finally:+ _content_generator_override.set(prev)+++def is_dry_run_forced() -> bool:+ """True when the boot was keyless (``needs_inference=False``): every run is forced to DRY (D4)."""+ return get_pipelex_hub().is_dry_run_forced()++def get_content_generator() -> ContentGeneratorProtocol:+ override = _content_generator_override.get()+ if override is not None:+ return overridereturn get_pipelex_hub().get_required_content_generator()@@ -663,6 +705,41 @@ def get_pipe_router() -> "PipeRouterProtocol":return get_pipelex_hub().get_required_pipe_router()+_event_log_override: ContextVar["EventLogProtocol | None"] = ContextVar("event_log_override", default=None)+++@contextmanager+def scoped_event_log(event_log: "EventLogProtocol") -> Generator[None, None, None]:+ """Pin ``event_log`` as the trace-event transport for the scope, then restore the prior value on exit.++ Both the write side (tracer emission, set up in ``pipeline_run_setup``) and the read+ side (``tracing_assembly.assemble_tracing``) prefer this override over building a new+ backend via ``make_event_log``, so emit and assemble share the SAME instance — which+ is what makes a plain in-memory event log usable for graph assembly (no external+ store bridges the two sides). A set override implies tracing-enabled: it is honored+ even when ``tracing_config.is_enabled`` is False.++ Lifecycle: the machinery never calls ``cleanup`` on the instance and the read side does+ not ``close`` it — but the write-side tracer DOES call ``close()`` on its event log at+ teardown (``GraphTracer._reset``), which happens BEFORE the read side assembles. A scoped+ event log's ``close()`` must therefore be safe to call mid-lifecycle — idempotent or a+ no-op, as ``InMemoryEventLog``'s is. Scoping a backend whose ``close()`` releases a real+ resource (NDJSON file handle, DynamoDB client) would break its own assembly read. Mirrors+ :func:`scoped_pipe_router`.+ """+ prev = _event_log_override.get()+ _event_log_override.set(event_log)+ try:+ yield+ finally:+ _event_log_override.set(prev)+++def get_event_log_override() -> "EventLogProtocol | None":+ """Return the contextvar-scoped event-log override set by :func:`scoped_event_log`, or None."""+ return _event_log_override.get()++def get_pipe_run() -> "PipeRunProtocol":return get_pipelex_hub().get_required_pipe_run()diff --git a/pipelex/kit/agent_rules/codex_commands.md b/pipelex/kit/agent_rules/codex_commands.mdindex a3775d562..378ccd7e9 100644--- a/pipelex/kit/agent_rules/codex_commands.md+++ b/pipelex/kit/agent_rules/codex_commands.md@@ -67,6 +67,12 @@.venv/bin/pipelex-dev generate-mthds-schema```+ - **`generate-error-pages`**: Regenerate the per-class error reference pages under `docs/errors/` — one Markdown page per `PipelexError` subclass, which is what each error's `type_uri` dereferences to. Run after adding or renaming an error class. Pages a maintainer claims with a `<!-- pipelex:authored -->` marker are preserved across runs. Also available as `make generate-error-pages` (alias `make gep`).++ ```bash+ .venv/bin/pipelex-dev generate-error-pages+ ```+## Pipelex CLI CommandsTo run the Pipelex CLI commands without the logo, you can use the `--no-logo` flag, this will avoid useless tokens in the console output.diff --git a/pipelex/kit/agent_rules/commands.md b/pipelex/kit/agent_rules/commands.mdindex fcf3fb247..424017c2e 100644--- a/pipelex/kit/agent_rules/commands.md+++ b/pipelex/kit/agent_rules/commands.md@@ -133,6 +133,12 @@.venv/bin/pipelex-dev generate-mthds-schema```+ - **`generate-error-pages`**: Regenerate the per-class error reference pages under `docs/errors/` — one Markdown page per `PipelexError` subclass, which is what each error's `type_uri` dereferences to. Run after adding or renaming an error class. Pages a maintainer claims with a `<!-- pipelex:authored -->` marker are preserved across runs. Also available as `make generate-error-pages` (alias `make gep`).++ ```bash+ .venv/bin/pipelex-dev generate-error-pages+ ```+- **`refresh-graph-ui-sri`**: Refetch the pinned graph viewer assets from jsDelivr (`@pipelex/mthds-ui` standalone JS+CSS, `elkjs`) and rewrite `pipelex/graph/reactflow/standalone_assets.py` with new `sha384` Subresource Integrity hashes. Use when bumping the pinned mthds-ui or elkjs version.```bashdiff --git a/pipelex/kit/configs/pipelex.toml b/pipelex/kit/configs/pipelex.tomlindex e4a2ff815..de39d5c70 100644--- a/pipelex/kit/configs/pipelex.toml+++ b/pipelex/kit/configs/pipelex.toml@@ -275,6 +275,8 @@ backoff_coefficient = 2.0maximum_interval = "unlimited"maximum_attempts = 3non_retryable_error_types = [+ "DryRunMockBuildError",+ "DryRunObjectFidelityError","ExtractHandleNotFoundError","FileNotFoundError","ImgGenHandleNotFoundError",diff --git a/pipelex/pipe_controllers/batch/pipe_batch.py b/pipelex/pipe_controllers/batch/pipe_batch.pyindex 8d39a381b..77f1f8841 100644--- a/pipelex/pipe_controllers/batch/pipe_batch.py+++ b/pipelex/pipe_controllers/batch/pipe_batch.py@@ -4,6 +4,7 @@ from typing import TYPE_CHECKING, Any, Literal, castfrom typing_extensions import overridefrom pipelex import log+from pipelex.cogt.content_generation.dry_mock import stamp_mock_main_coordinationfrom pipelex.config import get_configfrom pipelex.core.memory.working_memory import WorkingMemoryfrom pipelex.core.pipes.inputs.input_stuff_specs import InputStuffSpecs@@ -163,7 +164,6 @@ class PipeBatch(PipeController):"output_multiplicity": None,},)- branch_pipe_run_params.run_mode = pipe_run_params.run_modereturn await get_pipe_router().run(pipe_job=PipeJobFactory.make_pipe_job(pipe=sub_pipe,@@ -271,16 +271,12 @@ class PipeBatch(PipeController):output_name=output_name,library_crate=library_crate,)- # For dry run coordination: set the first item's pipe_code to "mock_main"- # to match the BundleHeaderSpec.main_pipe examples=["mock_main"]+ # Dry-run coordination: see stamp_mock_main_coordination's docstring (single home, D3).main_stuff = pipe_output.main_stuffcontent = main_stuff.contentif isinstance(content, ListContent):list_content = cast("ListContent[StuffContent]", content)- if list_content.items:- first_item = list_content.items[0]- if hasattr(first_item, "pipe_code"):- first_item.pipe_code = "mock_main" # pyright: ignore[reportAttributeAccessIssue]+ stamp_mock_main_coordination(list_content.items)return pipe_output@overridediff --git a/pipelex/pipe_operators/compose/pipe_compose.py b/pipelex/pipe_operators/compose/pipe_compose.pyindex d66643614..e9089a9cf 100644--- a/pipelex/pipe_operators/compose/pipe_compose.py+++ b/pipelex/pipe_operators/compose/pipe_compose.py@@ -4,8 +4,6 @@ from pydantic import Fieldfrom typing_extensions import overridefrom pipelex import log-from pipelex.cogt.content_generation.content_generator_dry import ContentGeneratorDry-from pipelex.cogt.content_generation.content_generator_protocol import ContentGeneratorProtocolfrom pipelex.cogt.content_generation.dry_run_factory import DryRunFactoryfrom pipelex.cogt.templating.template_category import TemplateCategoryfrom pipelex.cogt.templating.template_rendering import render_template@@ -20,7 +18,7 @@ from pipelex.core.pipes.pipe_output import PipeOutputfrom pipelex.core.stuffs.html_content import HtmlContentfrom pipelex.core.stuffs.stuff_content import StuffContentfrom pipelex.core.stuffs.stuff_factory import StuffFactory-from pipelex.hub import get_class_registry, get_concept_library, get_content_generator, get_native_concept+from pipelex.hub import get_class_registry, get_concept_library, get_native_conceptfrom pipelex.pipe_operators.compose.construct_blueprint import ConstructBlueprintfrom pipelex.pipe_operators.compose.exceptions import PipeComposeError, StructuredContentComposerValueErrorfrom pipelex.pipe_operators.compose.structured_content_composer import StructuredContentComposer@@ -147,17 +145,13 @@ class PipeCompose(PipeOperator[PipeComposeOutput]):working_memory: WorkingMemory,pipe_run_params: PipeRunParams,output_name: str | None = None,- content_generator: ContentGeneratorProtocol | None = None,) -> PipeComposeOutput:- content_generator = content_generator or get_content_generator()-if self.is_construct_mode:return await self._run_construct_mode(job_metadata=job_metadata,working_memory=working_memory,pipe_run_params=pipe_run_params,output_name=output_name,- content_generator=content_generator,)else:return await self._run_template_mode(@@ -231,7 +225,6 @@ class PipeCompose(PipeOperator[PipeComposeOutput]):working_memory: WorkingMemory,pipe_run_params: PipeRunParams,output_name: str | None,- content_generator: ContentGeneratorProtocol,) -> PipeComposeOutput:"""Run PipeCompose in construct mode (produces StructuredContent output)."""if self.construct_blueprint is None:@@ -245,14 +238,12 @@ class PipeCompose(PipeOperator[PipeComposeOutput]):)# Create composer and compose the structured content- # Pass runtime params, extra context, and content generator for template fields (consistent with _run_template_mode)composer = StructuredContentComposer(construct_blueprint=self.construct_blueprint,working_memory=working_memory,output_class=output_class,runtime_params=pipe_run_params.params if pipe_run_params else None,extra_context=self.extra_context,- content_generator=content_generator,pipe_run_params=pipe_run_params,)try:@@ -288,14 +279,6 @@ class PipeCompose(PipeOperator[PipeComposeOutput]):pipe_run_params: PipeRunParams,output_name: str | None = None,) -> PipeComposeOutput:- content_generator_used: ContentGeneratorProtocol- if get_config().pipelex.dry_run_config.apply_to_jinja2_rendering:- log.verbose(f"PipeCompose: using dry run operator pipe for jinja2 rendering: {self.code}")- content_generator_used = ContentGeneratorDry()- else:- log.verbose(f"PipeCompose: using regular operator pipe for jinja2 rendering (dry run not applied to jinja2): {self.code}")- content_generator_used = get_content_generator()-if self.is_construct_mode:try:return await self._live_run_operator_pipe(@@ -303,7 +286,6 @@ class PipeCompose(PipeOperator[PipeComposeOutput]):working_memory=working_memory,pipe_run_params=pipe_run_params,output_name=output_name,- content_generator=content_generator_used,)except PipeComposeError as exc:# Construct mode can fail with mock data (e.g., description-only concepts@@ -327,7 +309,6 @@ class PipeCompose(PipeOperator[PipeComposeOutput]):working_memory=working_memory,pipe_run_params=pipe_run_params,output_name=output_name,- content_generator=content_generator_used,)def _make_mock_construct_output(diff --git a/pipelex/pipe_operators/compose/structured_content_composer.py b/pipelex/pipe_operators/compose/structured_content_composer.pyindex a117f7870..07be8c06d 100644--- a/pipelex/pipe_operators/compose/structured_content_composer.py+++ b/pipelex/pipe_operators/compose/structured_content_composer.py@@ -5,14 +5,12 @@ from typing import Any, Union, cast, get_args, get_originfrom pydantic import ValidationErrorfrom pipelex import log-from pipelex.cogt.content_generation.content_generator_protocol import ContentGeneratorProtocolfrom pipelex.cogt.templating.template_category import TemplateCategoryfrom pipelex.cogt.templating.template_rendering import render_templatefrom pipelex.core.memory.working_memory import WorkingMemoryfrom pipelex.core.stuffs.list_content import ListContentfrom pipelex.core.stuffs.stuff_content import StuffContentfrom pipelex.core.stuffs.text_content import TextContent-from pipelex.hub import get_content_generatorfrom pipelex.pipe_operators.compose.construct_blueprint import ConstructBlueprint, ConstructFieldBlueprint, ConstructFieldMethodfrom pipelex.pipe_operators.compose.exceptions import (StructuredContentComposerTypeError,@@ -39,7 +37,6 @@ class StructuredContentComposer:output_class: The StructuredContent subclass to instantiateruntime_params: Additional runtime parameters for template context (from PipeRunParams.params)extra_context: Extra context values for template rendering (from PipeCompose.extra_context)- content_generator: The content generator to use for template rendering (supports dry run mode)pipe_run_params: The pipe run parameters (used to check if we're in dry run mode)"""@@ -50,7 +47,6 @@ class StructuredContentComposer:output_class: type[StuffContent],runtime_params: dict[str, Any] | None = None,extra_context: dict[str, Any] | None = None,- content_generator: ContentGeneratorProtocol | None = None,pipe_run_params: PipeRunParams | None = None,):self.construct_blueprint = construct_blueprint@@ -58,7 +54,6 @@ class StructuredContentComposer:self.output_class = output_classself.runtime_params = runtime_params or {}self.extra_context = extra_context or {}- self.content_generator = content_generator or get_content_generator()self.pipe_run_params = pipe_run_params# Per-field record of how each field was built. Populated as fields resolve.# Shape per entry: {"method": ConstructFieldMethod, "rendered": str (templates only)}.@@ -688,7 +683,6 @@ class StructuredContentComposer:output_class=nested_class,runtime_params=self.runtime_params,extra_context=self.extra_context,- content_generator=self.content_generator,pipe_run_params=self.pipe_run_params,)diff --git a/pipelex/pipe_operators/extract/pipe_extract.py b/pipelex/pipe_operators/extract/pipe_extract.pyindex a1b93e8b8..d64a376f3 100644--- a/pipelex/pipe_operators/extract/pipe_extract.py+++ b/pipelex/pipe_operators/extract/pipe_extract.py@@ -3,8 +3,6 @@ from typing import TYPE_CHECKING, Any, Literalfrom pydantic import model_validatorfrom typing_extensions import Self, override-from pipelex.cogt.content_generation.content_generator_dry import ContentGeneratorDry-from pipelex.cogt.content_generation.content_generator_protocol import ContentGeneratorProtocolfrom pipelex.cogt.exceptions import ModelChoiceNotFoundErrorfrom pipelex.cogt.extract.extract_input import ExtractInputfrom pipelex.cogt.extract.extract_job_components import ExtractJobConfig, ExtractJobParams@@ -123,9 +121,8 @@ class PipeExtract(PipeOperator[PipeExtractOutput]):working_memory: WorkingMemory,pipe_run_params: PipeRunParams,output_name: str | None = None,- content_generator: ContentGeneratorProtocol | None = None,) -> PipeExtractOutput:- content_generator = content_generator or get_content_generator()+ content_generator = get_content_generator()image_uri: str | None = Nonepdf_uri: str | None = None@@ -157,6 +154,7 @@ class PipeExtract(PipeOperator[PipeExtractOutput]):)page_contents = await content_generator.make_extract_pages(extract_input=extract_input,+ cogt_run_params=pipe_run_params.cogt_run_params,extract_handle=extract_setting.model,job_metadata=job_metadata,extract_job_params=extract_job_params,@@ -190,22 +188,6 @@ class PipeExtract(PipeOperator[PipeExtractOutput]):pipeline_run_id=job_metadata.pipeline_run_id,)- @override- async def _dry_run_operator_pipe(- self,- job_metadata: JobMetadata,- working_memory: WorkingMemory,- pipe_run_params: PipeRunParams,- output_name: str | None = None,- ) -> PipeExtractOutput:- return await self._live_run_operator_pipe(- job_metadata=job_metadata,- working_memory=working_memory,- pipe_run_params=pipe_run_params,- output_name=output_name,- content_generator=ContentGeneratorDry(),- )-@overrideasync def _validate_before_run(self, job_metadata: JobMetadata, working_memory: WorkingMemory, pipe_run_params: PipeRunParams, output_name: str | None = Nonediff --git a/pipelex/pipe_operators/img_gen/pipe_img_gen.py b/pipelex/pipe_operators/img_gen/pipe_img_gen.pyindex d80a42a27..269f6453c 100644--- a/pipelex/pipe_operators/img_gen/pipe_img_gen.py+++ b/pipelex/pipe_operators/img_gen/pipe_img_gen.py@@ -4,8 +4,6 @@ from pydantic import Fieldfrom typing_extensions import overridefrom pipelex import log-from pipelex.cogt.content_generation.content_generator_dry import ContentGeneratorDry-from pipelex.cogt.content_generation.content_generator_protocol import ContentGeneratorProtocolfrom pipelex.cogt.img_gen.img_gen_job_components import AspectRatio, Background, ImgGenJobParamsfrom pipelex.cogt.img_gen.img_gen_param_support import ImgGenParamSupportfrom pipelex.cogt.img_gen.img_gen_setting import ImgGenModelChoice, ImgGenSetting, ImgGenSettingValueError@@ -142,9 +140,8 @@ class PipeImgGen(PipeOperator[PipeImgGenOutput]):working_memory: WorkingMemory,pipe_run_params: PipeRunParams,output_name: str | None = None,- content_generator: ContentGeneratorProtocol | None = None,) -> PipeImgGenOutput:- content_generator = content_generator or get_content_generator()+ content_generator = get_content_generator()multiplicity_resolution = output_multiplicity_to_apply(base_multiplicity=self.output_multiplicity or False,@@ -236,6 +233,7 @@ class PipeImgGen(PipeOperator[PipeImgGenOutput]):if nb_images > 1:image_content_list = await content_generator.make_image_list(job_metadata=job_metadata,+ cogt_run_params=pipe_run_params.cogt_run_params,img_gen_handle=img_gen_handle,img_gen_prompt=img_gen_prompt,nb_images=nb_images,@@ -253,6 +251,7 @@ class PipeImgGen(PipeOperator[PipeImgGenOutput]):else:image_content = await content_generator.make_single_image(job_metadata=job_metadata,+ cogt_run_params=pipe_run_params.cogt_run_params,img_gen_handle=img_gen_handle,img_gen_prompt=img_gen_prompt,img_gen_job_params=img_gen_job_params,@@ -288,22 +287,6 @@ class PipeImgGen(PipeOperator[PipeImgGenOutput]):pipeline_run_id=job_metadata.pipeline_run_id,)- @override- async def _dry_run_operator_pipe(- self,- job_metadata: JobMetadata,- working_memory: WorkingMemory,- pipe_run_params: PipeRunParams,- output_name: str | None = None,- ) -> PipeImgGenOutput:- return await self._live_run_operator_pipe(- job_metadata=job_metadata,- working_memory=working_memory,- pipe_run_params=pipe_run_params,- output_name=output_name,- content_generator=ContentGeneratorDry(),- )-@overrideasync def _validate_before_run(self, job_metadata: JobMetadata, working_memory: WorkingMemory, pipe_run_params: PipeRunParams, output_name: str | None = Nonediff --git a/pipelex/pipe_operators/llm/pipe_llm.py b/pipelex/pipe_operators/llm/pipe_llm.pyindex 3ec77b613..ddcc35dd6 100644--- a/pipelex/pipe_operators/llm/pipe_llm.py+++ b/pipelex/pipe_operators/llm/pipe_llm.py@@ -5,7 +5,6 @@ from typing_extensions import overridefrom pipelex import logfrom pipelex.base_exceptions import iter_cause_chain-from pipelex.cogt.content_generation.content_generator_dry import ContentGeneratorDryfrom pipelex.cogt.content_generation.content_generator_protocol import ContentGeneratorProtocolfrom pipelex.cogt.exceptions import LLMCompletionErrorfrom pipelex.cogt.llm.llm_prompt import LLMPrompt@@ -172,9 +171,8 @@ class PipeLLM(PipeOperator[PipeLLMOutput]):working_memory: WorkingMemory,pipe_run_params: PipeRunParams,output_name: str | None = None,- content_generator: ContentGeneratorProtocol | None = None,) -> PipeLLMOutput:- content_generator = content_generator or get_content_generator()+ content_generator = get_content_generator()# interpret / unwrap the argumentsoutput_stuff_spec = self.resolve_dynamic_output_stuff_spec(pipe_run_params=pipe_run_params)@@ -242,6 +240,7 @@ class PipeLLM(PipeOperator[PipeLLMOutput]):try:generated_text: str = await content_generator.make_llm_text(job_metadata=job_metadata,+ cogt_run_params=pipe_run_params.cogt_run_params,llm_prompt_for_text=llm_prompt_1_for_text,llm_setting_main=llm_setting_main,)@@ -349,6 +348,7 @@ class PipeLLM(PipeOperator[PipeLLMOutput]):try:generated_objects = await content_generator.make_object_list(job_metadata=job_metadata,+ cogt_run_params=pipe_run_params.cogt_run_params,object_class=content_class,llm_prompt_for_object_list=llm_prompt_for_object,llm_setting_for_object_list=llm_setting_for_object,@@ -367,6 +367,7 @@ class PipeLLM(PipeOperator[PipeLLMOutput]):try:return await content_generator.make_object(job_metadata=job_metadata,+ cogt_run_params=pipe_run_params.cogt_run_params,object_class=content_class,llm_prompt_for_object=llm_prompt_for_object,llm_setting_for_object=llm_setting_for_object,@@ -389,22 +390,6 @@ class PipeLLM(PipeOperator[PipeLLMOutput]):breakreturn f"{error_details}\nLLM settings: {settings}"- @override- async def _dry_run_operator_pipe(- self,- job_metadata: JobMetadata,- working_memory: WorkingMemory,- pipe_run_params: PipeRunParams,- output_name: str | None = None,- ) -> PipeLLMOutput:- return await self._live_run_operator_pipe(- job_metadata=job_metadata,- working_memory=working_memory,- pipe_run_params=pipe_run_params,- output_name=output_name,- content_generator=ContentGeneratorDry(),- )-@overrideasync def _validate_before_run(self, job_metadata: JobMetadata, working_memory: WorkingMemory, pipe_run_params: PipeRunParams, output_name: str | None = Nonediff --git a/pipelex/pipe_operators/pipe_operator.py b/pipelex/pipe_operators/pipe_operator.pyindex 7dc8ede41..9b7a4ee1c 100644--- a/pipelex/pipe_operators/pipe_operator.py+++ b/pipelex/pipe_operators/pipe_operator.py@@ -105,7 +105,6 @@ class PipeOperator(PipeAbstract, Generic[PipeOperatorOutputType]):) -> PipeOperatorOutputType:pass- @abstractmethodasync def _dry_run_operator_pipe(self,job_metadata: JobMetadata,@@ -113,4 +112,15 @@ class PipeOperator(PipeAbstract, Generic[PipeOperatorOutputType]):pipe_run_params: PipeRunParams,output_name: str | None = None,) -> PipeOperatorOutputType:- pass+ """Default DRY behavior: reuse the live path — run_mode=DRY rides+ ``pipe_run_params.cogt_run_params`` into every assignment and the cogt leaf mocks, on any+ backend (inline or inside a Temporal activity). Operators whose dry behavior genuinely+ differs (e.g. ``PipeFunc`` mocks its output instead of running user code, ``PipeCompose``+ adds a construct-mode fallback) override this.+ """+ return await self._live_run_operator_pipe(+ job_metadata=job_metadata,+ working_memory=working_memory,+ pipe_run_params=pipe_run_params,+ output_name=output_name,+ )diff --git a/pipelex/pipe_operators/search/pipe_search.py b/pipelex/pipe_operators/search/pipe_search.pyindex 534726e32..abcb8d813 100644--- a/pipelex/pipe_operators/search/pipe_search.py+++ b/pipelex/pipe_operators/search/pipe_search.py@@ -4,8 +4,6 @@ from typing_extensions import overridefrom pipelex import logfrom pipelex.cogt.content_generation.assignment_models import SearchAssignment-from pipelex.cogt.content_generation.content_generator_dry import ContentGeneratorDry-from pipelex.cogt.content_generation.content_generator_protocol import ContentGeneratorProtocolfrom pipelex.cogt.exceptions import ModelChoiceNotFoundErrorfrom pipelex.cogt.model_backends.model_type import ModelTypefrom pipelex.cogt.models.model_deck_check import check_search_choice_with_deck@@ -79,9 +77,8 @@ class PipeSearch(PipeOperator[PipeSearchOutput]):working_memory: WorkingMemory,pipe_run_params: PipeRunParams,output_name: str | None = None,- content_generator: ContentGeneratorProtocol | None = None,) -> PipeSearchOutput:- content_generator = content_generator or get_content_generator()+ content_generator = get_content_generator()# 0. Log the search runsearch_choice_desc = self.search_choice or "default"@@ -118,6 +115,7 @@ class PipeSearch(PipeOperator[PipeSearchOutput]):# and lets its failures cross the workflow boundary as classified errors instead of hanging.search_assignment = SearchAssignment(job_metadata=job_metadata,+ cogt_run_params=pipe_run_params.cogt_run_params,query=query_text,search_setting=search_setting,include_domains=self.include_domains,@@ -159,24 +157,6 @@ class PipeSearch(PipeOperator[PipeSearchOutput]):pipeline_run_id=job_metadata.pipeline_run_id,)- @override- async def _dry_run_operator_pipe(- self,- job_metadata: JobMetadata,- working_memory: WorkingMemory,- pipe_run_params: PipeRunParams,- output_name: str | None = None,- ) -> PipeSearchOutput:- # Dry run reuses the live path with the dry content generator — identical to PipeLLM — so the- # search mock now lives behind the same seam (ContentGeneratorDry.make_search_*) as every leaf.- return await self._live_run_operator_pipe(- job_metadata=job_metadata,- working_memory=working_memory,- pipe_run_params=pipe_run_params,- output_name=output_name,- content_generator=ContentGeneratorDry(),- )-@overrideasync def _validate_before_run(self, job_metadata: JobMetadata, working_memory: WorkingMemory, pipe_run_params: PipeRunParams, output_name: str | None = Nonediff --git a/pipelex/pipe_operators/structure/pipe_structure.py b/pipelex/pipe_operators/structure/pipe_structure.pyindex 6f9fd28a7..9db4cdfbe 100644--- a/pipelex/pipe_operators/structure/pipe_structure.py+++ b/pipelex/pipe_operators/structure/pipe_structure.py@@ -3,8 +3,6 @@ from typing import Any, Literalfrom typing_extensions import overridefrom pipelex import log-from pipelex.cogt.content_generation.content_generator_dry import ContentGeneratorDry-from pipelex.cogt.content_generation.content_generator_protocol import ContentGeneratorProtocolfrom pipelex.cogt.exceptions import LLMCompletionErrorfrom pipelex.cogt.llm.llm_prompt import LLMPromptfrom pipelex.cogt.llm.llm_setting import LLMModelChoice, LLMSetting@@ -107,9 +105,8 @@ class PipeStructure(PipeOperator[PipeStructureOutput]):working_memory: WorkingMemory,pipe_run_params: PipeRunParams,output_name: str | None = None,- content_generator: ContentGeneratorProtocol | None = None,) -> PipeStructureOutput:- content_generator = content_generator or get_content_generator()+ content_generator = get_content_generator()text_str = working_memory.get_stuff_as_str(name=self.text_input_name)@@ -155,6 +152,7 @@ class PipeStructure(PipeOperator[PipeStructureOutput]):try:generated_objects = await content_generator.make_object_list(job_metadata=job_metadata,+ cogt_run_params=pipe_run_params.cogt_run_params,object_class=content_class,llm_prompt_for_object_list=llm_prompt,llm_setting_for_object_list=llm_setting_for_object,@@ -169,6 +167,7 @@ class PipeStructure(PipeOperator[PipeStructureOutput]):try:the_content = await content_generator.make_object(job_metadata=job_metadata,+ cogt_run_params=pipe_run_params.cogt_run_params,object_class=content_class,llm_prompt_for_object=llm_prompt,llm_setting_for_object=llm_setting_for_object,@@ -204,22 +203,6 @@ class PipeStructure(PipeOperator[PipeStructureOutput]):def _format_error_location(self, pipe_run_params: PipeRunParams) -> str:return f"in pipe '{pipe_run_params.pipe_stack_str}'"- @override- async def _dry_run_operator_pipe(- self,- job_metadata: JobMetadata,- working_memory: WorkingMemory,- pipe_run_params: PipeRunParams,- output_name: str | None = None,- ) -> PipeStructureOutput:- return await self._live_run_operator_pipe(- job_metadata=job_metadata,- working_memory=working_memory,- pipe_run_params=pipe_run_params,- output_name=output_name,- content_generator=ContentGeneratorDry(),- )-@overrideasync def _validate_before_run(self, job_metadata: JobMetadata, working_memory: WorkingMemory, pipe_run_params: PipeRunParams, output_name: str | None = Nonediff --git a/pipelex/pipe_run/dry_run_pipeline.py b/pipelex/pipe_run/dry_run_pipeline.pyindex 2b378f0f9..f6e7e5b70 100644--- a/pipelex/pipe_run/dry_run_pipeline.py+++ b/pipelex/pipe_run/dry_run_pipeline.py@@ -4,14 +4,25 @@ Provides a single entrypoint to dry-run an entire pipeline from MTHDS content,producing a GraphSpec. Used by both the CLI graph commands and the API."""-from pipelex.base_exceptions import PipelexError+from pipelex.cogt.content_generation.content_generator import ContentGeneratorfrom pipelex.config import get_configfrom pipelex.core.interpreter.exceptions import PipelexInterpreterErrorfrom pipelex.core.interpreter.interpreter import PipelexInterpreter+from pipelex.core.pipes.pipe_abstract import PipeAbstractfrom pipelex.core.pipes.pipe_factory import PipeFactory+from pipelex.graph.graph_tracer_manager import GraphTracerManagerfrom pipelex.graph.graphspec import GraphSpec+from pipelex.hub import scoped_content_generator, scoped_event_log, scoped_pipe_router+from pipelex.observer.observer_protocol import ObserverNoOp+from pipelex.pipe_run.exceptions import DryRunGraphNotProducedError+from pipelex.pipe_run.pipe_router import PipeRouter+from pipelex.pipe_run.pipe_run import PipeRunfrom pipelex.pipe_run.pipe_run_mode import PipeRunMode+from pipelex.pipeline.execution_seams import prepare_pipe_job+from pipelex.pipeline.pipeline_factory import PipelineFactoryfrom pipelex.pipeline.runner import PipelexRunner+from pipelex.system.telemetry.otel_constants import OTelConstants+from pipelex.tracing.in_memory_event_log import InMemoryEventLogasync def dry_run_pipeline(@@ -37,7 +48,7 @@ async def dry_run_pipeline(Raises:PipelexInterpreterError: If content parsing fails or main_pipe is missing.- PipelexError: If pipeline execution does not produce a graph spec.+ DryRunGraphNotProducedError: If pipeline execution does not produce a graph spec.PipelineExecutionError: If dry-run execution fails."""if not mthds_contents:@@ -73,14 +84,108 @@ async def dry_run_pipeline(execution_config=execution_config,library_dirs=library_dirs or [],)- response = await runner.execute_pipeline(- pipe_code=pipe_code,- mthds_contents=mthds_contents,- )+ # This function's whole purpose is the graph, so it owns the graph transport: the scoped+ # in-memory event log is the run's emit AND assemble channel (see hub.scoped_event_log, D1),+ # making graph generation independent of the host's tracing_config — a host with tracing+ # disabled (e.g. pipelex-api's /validate in direct mode) still gets its graph, and a host+ # with a configured backend doesn't get trace files written as a side effect of validation.+ with scoped_event_log(InMemoryEventLog()):+ response = await runner.execute_pipeline(+ pipe_code=pipe_code,+ mthds_contents=mthds_contents,+ )pipe_output = response.pipe_outputif not pipe_output.graph_spec:msg = "Pipeline execution did not produce a graph spec"- raise PipelexError(msg)+ raise DryRunGraphNotProducedError(msg)return pipe_output.graph_spec, pipe_code+++async def dry_run_pipe_in_process(pipe: PipeAbstract, *, library_id: str) -> GraphSpec:+ """Dry-run ``pipe`` against an already-open library fully in-process, tracing the graph in memory.++ The in-process twin of :func:`dry_run_pipeline` for hosts where the hub is Temporal-enabled+ but the run must NOT dispatch anything — e.g. the body of the dry-run/validation Temporal+ activity. Three contextvar scopes pin the whole run (root pipe + nested controller sub-pipes+ + inference leaves) to in-process execution:++ - ``scoped_event_log(InMemoryEventLog())`` — emit and assemble share one in-memory transport+ (no NDJSON file, no DynamoDB round-trip); the ``GraphSpec`` rides back on ``PipeOutput``.+ - ``scoped_pipe_router(local PipeRouter)`` — nested controller sub-pipes resolve the local+ router instead of the hub's ``TemporalPipeRouter`` (mirrors ``BundleValidator``).+ - ``scoped_content_generator(inline ContentGenerator)`` — inference leaves resolve an inline+ generator instead of the hub's ``ContentGeneratorInWorkflow``, so nothing dispatches; the+ DRY mock lives at the cogt leaf (Part B), which also skips all storage IO.++ The tracer is opened at ``graph_id=pipeline_run_id`` and closed by ``pipeline_run_id`` (in+ ``PipeRun.run``'s ``finally``) — emit and assemble keys are aligned by construction (D-C7).+ Usage/cost events are not emitted (``emit_usage_events=False``): this is a validation-side+ dry-run, not a billed run.++ The caller owns the library lifecycle: ``library_id`` must be open and current, and is left+ untouched (mirrors ``BundleValidator.validate_pipes``).++ Args:+ pipe: The pipe to dry-run (resolved against the open library).+ library_id: The id of the already-open library to run against.++ Returns:+ The assembled GraphSpec.++ Raises:+ DryRunGraphNotProducedError: If the dry-run completes without producing a graph spec.+ """+ execution_config = get_config().pipelex.pipeline_execution_config.with_execution_overrides(+ generate_graph=True,+ mock_inputs=True,+ )+ pipeline_run_id = f"dry_run_graph_{PipelineFactory.make_pipeline_run_id()}"+ event_log = InMemoryEventLog()++ # Local, direct execution primitives — NOT get_pipe_run() (a Temporal hub would spawn a+ # workflow). Keep the router instance: the scope installs it for nested sub-pipes.+ # Constructed BEFORE the tracer opens so nothing can raise between open_tracer and the+ # try/finally that guarantees its close — an entry in the process-wide GraphTracerManager+ # must never outlive this call.+ pipe_router = PipeRouter(observer=ObserverNoOp())+ pipe_run = PipeRun(pipe_router=pipe_router)+ content_generator = ContentGenerator.make_inline()++ graph_tracer_manager = GraphTracerManager.get_or_create_instance()+ trace_context = graph_tracer_manager.open_tracer(+ graph_id=pipeline_run_id,+ data_inclusion=execution_config.graph_config.data_inclusion,+ pipeline_ref_domain=pipe.domain_code,+ pipeline_ref_main_pipe=pipe.code,+ event_log=event_log,+ workflow_id="direct",+ pipeline_run_id=pipeline_run_id,+ emit_graph_events=True,+ emit_usage_events=False,+ )+ try:+ with scoped_event_log(event_log), scoped_pipe_router(pipe_router), scoped_content_generator(content_generator):+ pipe_job = await prepare_pipe_job(+ pipe=pipe,+ library_id=library_id,+ execution_config=execution_config,+ pipe_run_mode=PipeRunMode.DRY,+ pipeline_run_id=pipeline_run_id,+ user_id=OTelConstants.DEFAULT_USER_ID,+ trace_context=trace_context,+ )+ # PipeRun.run's finally closes the tracer by pipeline_run_id and assembles the+ # GraphSpec from the scoped in-memory log onto pipe_output.+ pipe_output = await pipe_run.run(pipe_job)+ finally:+ # Safety net for failures before PipeRun.run owns the tracer (e.g. prepare_pipe_job+ # raising): close_tracer pops by key, so this is a no-op after a completed run.+ graph_tracer_manager.close_tracer(pipeline_run_id)++ if not pipe_output.graph_spec:+ msg = f"In-process dry-run of pipe '{pipe.pipe_ref}' did not produce a graph spec"+ raise DryRunGraphNotProducedError(msg)++ return pipe_output.graph_specdiff --git a/pipelex/pipe_run/exceptions.py b/pipelex/pipe_run/exceptions.pyindex 514bd064c..942453307 100644--- a/pipelex/pipe_run/exceptions.py+++ b/pipelex/pipe_run/exceptions.py@@ -73,6 +73,17 @@ class DryRunError(PipelexError):"""Raised when a dry run fails due to missing inputs or other validation issues."""+class DryRunGraphNotProducedError(DryRunError):+ """Raised when a graph-producing dry run completes but no ``GraphSpec`` was assembled onto the pipe output.++ The dry-run entrypoints (``dry_run_pipeline`` / ``dry_run_pipe_in_process``) exist to produce a+ graph, so a run that finishes without one is a contract violation on their side — not a+ validation failure of the bundle. Distinct from the bare ``PipelexError`` it replaces so that+ hosts logging only the exception type (e.g. pipelex-api's best-effort ``/validate`` graph step)+ get an unambiguous signal.+ """++class PipeRouterError(PipelexError):def __init__(self,diff --git a/pipelex/pipe_run/pipe_run_params.py b/pipelex/pipe_run/pipe_run_params.pyindex d565e1a26..2bf209b88 100644--- a/pipelex/pipe_run/pipe_run_params.py+++ b/pipelex/pipe_run/pipe_run_params.py@@ -2,12 +2,13 @@ from __future__ import annotationsfrom typing import Any-from pydantic import BaseModel, Field, field_validator+from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validatorfrom pipelex import log+from pipelex.cogt.content_generation.cogt_run_params import CogtRunParams, check_mock_usage_requires_dryfrom pipelex.core.memory.working_memory import BATCH_ITEM_STUFF_NAME, MAIN_STUFF_NAMEfrom pipelex.core.pipes.variable_multiplicity import VariableMultiplicity, VariableMultiplicityResolution-from pipelex.pipe_run.pipe_run_mode import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunMode # noqa: TC001 — pydantic resolves the field annotation at runtimefrom pipelex.pipeline.exceptions import PipeStackOverflowErrorfrom pipelex.types import Self, StrEnum@@ -135,7 +136,26 @@ class BatchParams(BaseModel):class PipeRunParams(BaseModel):- run_mode: PipeRunMode = PipeRunMode.LIVE+ # `extra="forbid"` so a stale `PipeRunParams(cogt_run_params=...)` constructor fails loudly+ # instead of silently building a LIVE-mode instance (cogt_run_params is derived now, not a field).+ model_config = ConfigDict(extra="forbid")++ # REQUIRED (no default): a payload missing the run mode must fail loud instead of silently+ # running LIVE (the spending direction) or DRY (the mock direction). `frozen=True` because a+ # post-construction DRY→LIVE flip would bypass the validator below and flow straight into+ # real provider spend via the derived carrier — written once at construction by+ # `PipeRunParamsFactory.make_run_params`; operators slice the derived `cogt_run_params` off+ # and thread it into the content-generator protocol so the cogt leaf sees the same mode on+ # any backend.+ run_mode: PipeRunMode = Field(frozen=True)++ # Internal sub-flag of DRY — only CLI access is the hidden `--mock-usage` test trigger+ # (requires `--dry-run`, not shown in --help): when True, the dry LLM leaves report+ # *non-zero* synthetic usage so the end-of-run cost report renders — the cheap, deterministic+ # cross-worker cost-report validation affordance. Rejected on a LIVE run (validator below);+ # frozen for the same reason as `run_mode`.+ is_mock_usage: bool = Field(default=False, frozen=True)+final_stuff_code: str | None = Noneoutput_multiplicity: VariableMultiplicity | None = Nonedynamic_output_concept_ref: str | None = None@@ -146,6 +166,21 @@ class PipeRunParams(BaseModel):pipe_stack: list[str] = Field(default_factory=list)pipe_layers: list[str] = Field(default_factory=list)+ @model_validator(mode="after")+ def validate_mock_usage_requires_dry(self) -> Self:+ check_mock_usage_requires_dry(run_mode=self.run_mode, is_mock_usage=self.is_mock_usage)+ return self++ @property+ def cogt_run_params(self) -> CogtRunParams:+ """Mint the cogt-tier slice of these params — what generators stamp on every assignment.++ Derived (not stored) from the run-mode fields above, so there is exactly one copy of the+ facts; the carrier exists because the cogt layer must stay pipe-agnostic and the assignment+ payloads cross the Temporal activity boundary.+ """+ return CogtRunParams(run_mode=self.run_mode, is_mock_usage=self.is_mock_usage)+@propertydef pipe_stack_str(self) -> str:return ".".join(self.pipe_stack)diff --git a/pipelex/pipe_run/pipe_run_params_factory.py b/pipelex/pipe_run/pipe_run_params_factory.pyindex 4819de965..c9b7cdf22 100644--- a/pipelex/pipe_run/pipe_run_params_factory.py+++ b/pipelex/pipe_run/pipe_run_params_factory.py@@ -1,24 +1,48 @@from typing import Any+from pipelex import log+from pipelex.cogt.content_generation.cogt_run_params import check_mock_usage_requires_dryfrom pipelex.config import get_configfrom pipelex.core.pipes.variable_multiplicity import VariableMultiplicity-from pipelex.pipe_run.pipe_run_params import BatchParams, PipeRunMode, PipeRunParams+from pipelex.hub import is_dry_run_forced+from pipelex.pipe_run.pipe_run_mode import PipeRunMode+from pipelex.pipe_run.pipe_run_params import BatchParams, PipeRunParamsclass PipeRunParamsFactory:@classmethoddef make_run_params(cls,+ *,pipe_run_mode: PipeRunMode = PipeRunMode.LIVE,+ is_mock_usage: bool = False,pipe_stack_limit: int | None = None,output_multiplicity: VariableMultiplicity | None = None,dynamic_output_concept_ref: str | None = None,batch_params: BatchParams | None = None,params: dict[str, Any] | None = None,) -> PipeRunParams:+ """Single writer of ``run_mode`` and ``is_mock_usage`` — direct fields on ``PipeRunParams``.++ The keyless-boot forced-DRY flag (eng review D4) is resolved HERE — at the single writer —+ so every execution entry point is covered (``prepare_pipe_job``, the runtime bridge,+ ``PipeJobFactory`` defaults), not just the pipeline-API path.++ The REQUESTED mode is validated before the forced-DRY coercion, so a contract violation+ (``is_mock_usage`` on a LIVE request) fails loud on every boot — the keyless coercion must+ not silently turn an illegal request into a legal one.+ """+ check_mock_usage_requires_dry(run_mode=pipe_run_mode, is_mock_usage=is_mock_usage)+ if is_dry_run_forced() and pipe_run_mode.is_live:+ log.warning(+ "LIVE run requested under a keyless boot (needs_inference=False): forcing run_mode to DRY — "+ "outputs will be synthetic mocks, not real inference."+ )+ pipe_run_mode = PipeRunMode.DRYpipe_stack_limit = pipe_stack_limit or get_config().pipelex.pipe_run_config.pipe_stack_limitreturn PipeRunParams(run_mode=pipe_run_mode,+ is_mock_usage=is_mock_usage,pipe_stack_limit=pipe_stack_limit,output_multiplicity=output_multiplicity,dynamic_output_concept_ref=dynamic_output_concept_ref,diff --git a/pipelex/pipe_run/tracing_assembly.py b/pipelex/pipe_run/tracing_assembly.pyindex 90adf1a37..b71f5521a 100644--- a/pipelex/pipe_run/tracing_assembly.py+++ b/pipelex/pipe_run/tracing_assembly.py@@ -19,6 +19,7 @@ from pipelex.base_exceptions import PipelexConfigErrorfrom pipelex.config import get_configfrom pipelex.core.pipes.pipe_output import PipeOutputfrom pipelex.graph.graphspec import GraphSpec, PipelineRef+from pipelex.hub import get_event_log_overridefrom pipelex.reporting.reporting_types import AnyTokensUsagefrom pipelex.system.exceptions import MissingDependencyErrorfrom pipelex.tracing.event_log_factory import make_event_log@@ -81,7 +82,10 @@ def assemble_tracing("""result = TracingAssembly()tracing_config = get_config().pipelex.tracing_config- if not tracing_config.is_enabled:+ # A scoped override (see hub.scoped_event_log) is the run's transport and implies+ # tracing-enabled (D1) — it must not be skipped by the is_enabled early-return.+ event_log_override = get_event_log_override()+ if event_log_override is None and not tracing_config.is_enabled:return resultif not (assemble_graph or assemble_usage):return result@@ -92,11 +96,15 @@ def assemble_tracing(# inside ``DynamoDBEventLog.read_events`` — so the assembly layer catches the EventLogError base and# never imports boto3 itself; NDJSON read failures surface as OSError / JSONDecodeError.try:- event_log = make_event_log(tracing_config)- try:- events = event_log.read_events(pipeline_run_id)- finally:- event_log.close()+ if event_log_override is not None:+ # The scope owner keeps the instance's lifecycle: read without close().+ events = event_log_override.read_events(pipeline_run_id)+ else:+ event_log = make_event_log(tracing_config)+ try:+ events = event_log.read_events(pipeline_run_id)+ finally:+ event_log.close()except (OSError, json.JSONDecodeError, ValidationError, PipelexConfigError, MissingDependencyError, EventLogError) as read_error:message = f"Tracing assembly failed to read events for pipeline_run_id={pipeline_run_id}: {read_error}"log.warning(message)diff --git a/pipelex/pipelex.py b/pipelex/pipelex.pyindex 6500f34db..e38360e18 100644--- a/pipelex/pipelex.py+++ b/pipelex/pipelex.py@@ -11,7 +11,6 @@ from pydantic import ValidationErrorfrom pipelex import logfrom pipelex.base_exceptions import PipelexConfigError, PipelexSetupErrorfrom pipelex.cogt.content_generation.content_generator import ContentGenerator-from pipelex.cogt.content_generation.content_generator_dry import ContentGeneratorDryfrom pipelex.cogt.content_generation.content_generator_protocol import (ContentGeneratorProtocol,)@@ -367,18 +366,19 @@ If you need help, drop by our Discord: we're happy to assist: {URLs.discord}.)raise PipelexSetupError(error_msg) from credentials_exc+ # Keyless boot forces every run to DRY — consumed at PipeRunParamsFactory.make_run_params,+ # the single writer of run_mode, covering every entry point; generator selection is+ # backend-keyed unconditionally (eng review D4) — a keyless Temporal submitter must still+ # dispatch activities and mock inside them, so `needs_inference` plays no role in picking the+ # generator. Its other boot roles (gateway/model setup, credentials, telemetry) are unchanged.+ self.pipelex_hub.set_dry_run_forced(not needs_inference)if content_generator is None:- if not needs_inference:- content_generator = ContentGeneratorDry()- elif get_config().temporal.is_enabled:+ if get_config().temporal.is_enabled:from pipelex.temporal.tprl_content_generation.content_generator_in_workflow_factory import ( # noqa: PLC0415ContentGeneratorInWorkflowFactory,)- generated_content_factory = GeneratedContentFactory(storage_provider=storage_provider)- content_generator = ContentGeneratorInWorkflowFactory.make_content_generator_in_workflow(- generated_content_factory=generated_content_factory,- )+ content_generator = ContentGeneratorInWorkflowFactory.make_content_generator_in_workflow()else:generated_content_factory = GeneratedContentFactory(storage_provider=storage_provider)content_generator = ContentGenerator(generated_content_factory=generated_content_factory)@@ -538,10 +538,13 @@ If you need help, drop by our Discord: we're happy to assist: {URLs.discord}.Args:integration_mode: Integration mode (CLI, FASTAPI, DOCKER, MCP, N8N, PYTHON, PYTEST)- needs_inference: When False, disables inference functionality by using a mock- content generator and loading backends leniently (skipping those with missing- credentials). This skips gateway terms check and model deck validation.- Useful for commands like validate/show that don't call inference APIs.+ needs_inference: When False, forces every run THIS process initiates to DRY mode+ (consumed at PipeRunParamsFactory.make_run_params, the single writer of run_mode:+ operators dispatch normally and the cogt leaf mocks) and loads backends leniently+ (skipping those with missing credentials). This skips gateway terms check and model+ deck validation. Useful for commands like validate/show that don't call inference+ APIs. Generator selection stays backend-keyed. Submitter-side contract only: it does+ not constrain work this process executes as a Temporal worker.temporal_enabled: When provided, overrides the temporal.is_enabled config value.True forces Temporal workflow execution, False forces direct execution.needs_model_specs: When True, load real model specs even if needs_inferencediff --git a/pipelex/pipelex.toml b/pipelex/pipelex.tomlindex 0c26868d3..5d6fef5d6 100644--- a/pipelex/pipelex.toml+++ b/pipelex/pipelex.toml@@ -368,7 +368,6 @@ region = ""####################################################################################################[pipelex.dry_run_config]-apply_to_jinja2_rendering = falsetext_gen_truncate_length = 256nb_list_items = 3nb_extract_pages = 4@@ -502,6 +501,8 @@ backoff_coefficient = 2.0maximum_interval = "unlimited"maximum_attempts = 3non_retryable_error_types = [+ "DryRunMockBuildError",+ "DryRunObjectFidelityError","ExtractHandleNotFoundError","FileNotFoundError","ImgGenHandleNotFoundError",diff --git a/pipelex/pipeline/bundle_validator.py b/pipelex/pipeline/bundle_validator.pyindex 5211cb2b8..91160f6e7 100644--- a/pipelex/pipeline/bundle_validator.py+++ b/pipelex/pipeline/bundle_validator.py@@ -34,6 +34,7 @@ from pydantic import BaseModel, ValidationErrorfrom pipelex import logfrom pipelex.base_exceptions import PipelexError+from pipelex.cogt.content_generation.content_generator import ContentGeneratorfrom pipelex.config import get_configfrom pipelex.core.pipes.pipe_abstract import PipeAbstractfrom pipelex.hub import (@@ -43,6 +44,7 @@ from pipelex.hub import (get_library_manager,get_pipe_library,get_telemetry_manager,+ scoped_content_generator,scoped_pipe_router,set_current_library,)@@ -250,7 +252,12 @@ class BundleValidator:# scope is contextvar-based, so concurrent /validate sweeps don't cross-contaminate, and the# asyncio tasks a batch fan-out spawns copy the context at creation (inside this scope) so they# inherit the override too.- with scoped_pipe_router(self._pipe_router):+ # scoped_content_generator: the sweep is always DRY and must stay in-process, so its+ # inference leaves resolve an inline ContentGenerator even under a Temporal-enabled hub+ # (where get_content_generator() is ContentGeneratorInWorkflow) — same rationale as the+ # router scope above. The DRY mock lives at the cogt leaf (Part B), so the inline+ # generator's leaves mock without dispatching and without touching storage.+ with scoped_pipe_router(self._pipe_router), scoped_content_generator(ContentGenerator.make_inline()):for pipe in sweepable_pipes:results[pipe.pipe_ref] = await self._classify_pipe(pipe=pipe, library_id=library_id, execution_config=execution_config, dry_run_pipeline_id=dry_run_pipeline_iddiff --git a/pipelex/pipeline/execution_seams.py b/pipelex/pipeline/execution_seams.pyindex 841702a8d..724ad3a62 100644--- a/pipelex/pipeline/execution_seams.py+++ b/pipelex/pipeline/execution_seams.py@@ -170,7 +170,7 @@ async def prepare_pipe_job(output_multiplicity: VariableMultiplicity | None = None,dynamic_output_concept_ref: str | None = None,request_id: str | None = None,- is_mock_inference: bool = False,+ is_mock_usage: bool = False,) -> PipeJob:"""Build a :class:`PipeJob` for ``pipe`` against an already-open library.@@ -179,10 +179,16 @@ async def prepare_pipe_job(params, job metadata, and the library crate. Performs no pipeline-managerregistration, no report-registry open, no telemetry, no graph-tracer open,and no library mutation. ``trace_context`` / ``otel_context`` are created by- the caller and threaded onto the job metadata. ``is_mock_inference`` (the- ``--mock-inference`` trigger) is the single-writer point onto- :attr:`JobMetadata.is_mock_inference` — a LIVE run whose LLM inference-leaf calls are- faked (non-LLM leaves — image-gen / extract / search — raise ``MockInferenceUnsupportedError``).+ the caller and threaded onto the job metadata. ``is_mock_usage`` is the single-writer+ point onto :attr:`CogtRunParams.is_mock_usage` — the internal DRY sub-flag that makes the+ dry LLM leaves report non-zero synthetic usage so the cost report renders (DRY-only;+ rejected on a LIVE run).++ A keyless boot (``Pipelex.make(needs_inference=False)``) forces the run to DRY (eng review+ D4): the backend still dispatches normally and the cogt leaf mocks, so a keyless Temporal+ submitter exercises the real distribution machinery at zero spend. The flag is resolved by+ ``PipeRunParamsFactory.make_run_params`` (the single writer of ``run_mode``), so it covers+ every entry point that builds run params, not just this one."""working_memory: WorkingMemory | None = None@@ -224,13 +230,13 @@ async def prepare_pipe_job(otel_context=otel_context,trace_context=trace_context,request_id=request_id,- is_mock_inference=is_mock_inference,)pipe_run_params = PipeRunParamsFactory.make_run_params(output_multiplicity=output_multiplicity,dynamic_output_concept_ref=dynamic_output_concept_ref,pipe_run_mode=pipe_run_mode,+ is_mock_usage=is_mock_usage,)# Build the library crate from all accumulated blueprints for Temporal dispatch.diff --git a/pipelex/pipeline/job_metadata.py b/pipelex/pipeline/job_metadata.pyindex 5cbeacb5b..3107490f3 100644--- a/pipelex/pipeline/job_metadata.py+++ b/pipelex/pipeline/job_metadata.py@@ -77,17 +77,6 @@ class JobMetadata(BaseModel):unit_job_id: UnitJobId | None = Nonejob_category: JobCategory | None = None- # Interim ``--mock-inference`` trigger: a LIVE run (``run_mode`` stays LIVE, so operators- # dispatch normally) whose AI calls are faked at the cogt leaf instead of hitting a provider.- # Lives here — not on ``PipeRunParams`` — because ``PipeRunParams.run_mode`` does not reach the- # leaf, whereas ``JobMetadata`` rides into every assignment and crosses the Temporal boundary, so- # the leaf branch (``llm_generate.py``) and the activity body both see it. Single-writer:- # ``prepare_pipe_job`` off the CLI flag. The leaf emits synthetic *non-zero* usage so a cost- # report renders (unlike ``run_mode=DRY`` whose zero-token usage is suppressed) — this is the- # cheap, deterministic cross-worker cost-report validation affordance. See- # ``cogt/content_generation/dry_mock.py`` and ``wip/dry-run-refactor/followup-leaf-run-mode-mock.md``.- is_mock_inference: bool = False-started_at: datetime | None = Field(default_factory=datetime.now)completed_at: datetime | None = Nonediff --git a/pipelex/pipeline/pipeline_run_setup.py b/pipelex/pipeline/pipeline_run_setup.pyindex 98828d501..26fa4cca6 100644--- a/pipelex/pipeline/pipeline_run_setup.py+++ b/pipelex/pipeline/pipeline_run_setup.py@@ -8,6 +8,7 @@ from pipelex.graph.graph_tracer_manager import GraphTracerManagerfrom pipelex.hub import (clear_current_library,get_current_library_id_or_none,+ get_event_log_override,get_library_manager,get_otel_tracer,get_pipeline_manager,@@ -50,7 +51,7 @@ async def pipeline_run_setup(output_multiplicity: VariableMultiplicity | None = None,dynamic_output_concept_ref: str | None = None,pipe_run_mode: PipeRunMode | None = None,- is_mock_inference: bool = False,+ is_mock_usage: bool = False,search_domain_codes: list[str] | None = None,user_id: str | None = None,pipeline_run_id: str | None = None,@@ -104,12 +105,13 @@ async def pipeline_run_setup(Pipe run mode: ``PipeRunMode.LIVE`` or ``PipeRunMode.DRY``. If not specified,inferred from the environment variable ``PIPELEX_FORCE_DRY_RUN_MODE``. Defaultsto ``PipeRunMode.LIVE`` if the environment variable is not set.- is_mock_inference:- The ``--mock-inference`` trigger: keep ``run_mode`` LIVE (operators dispatch- normally) but fake every AI call at the cogt leaf. Threaded onto- :attr:`JobMetadata.is_mock_inference`, which reaches the leaf in both direct and- Temporal modes. Distinct from ``--dry-run``: it exercises the live control flow and- emits reportable (non-zero) synthetic usage so a cost report renders.+ is_mock_usage:+ Internal sub-flag of ``run_mode=DRY`` — only CLI access is the hidden ``--mock-usage``+ test trigger (requires ``--dry-run``, not shown in --help): when True, the dry+ LLM leaves report *non-zero* synthetic usage so the end-of-run cost report renders —+ the cheap, deterministic cross-worker cost-report validation affordance. Threaded onto+ :attr:`CogtRunParams.is_mock_usage`, which rides every assignment to the leaf in both+ direct and Temporal modes. Requires a DRY run; setting it on a LIVE run fails validation.search_domain_codes:List of domain codes to search for pipes. The executed pipe's domain is automaticallyadded if not already present.@@ -196,7 +198,10 @@ async def pipeline_run_setup(# (node/edge) events and usage (cost) events.config = get_config()tracing_config = config.pipelex.tracing_config- if tracing_config.is_enabled:+ # A scoped override (see hub.scoped_event_log) is the run's transport and implies+ # tracing-enabled (D1); otherwise build the configured backend when tracing is on.+ event_log = get_event_log_override()+ if event_log is None and tracing_config.is_enabled:event_log = make_event_log(tracing_config)graph_tracer_manager = GraphTracerManager.get_or_create_instance()@@ -258,7 +263,7 @@ async def pipeline_run_setup(library_id=library_id,execution_config=execution_config,pipe_run_mode=pipe_run_mode,- is_mock_inference=is_mock_inference,+ is_mock_usage=is_mock_usage,pipeline_run_id=pipeline_run_id,user_id=user_id,inputs=inputs,diff --git a/pipelex/pipeline/runner.py b/pipelex/pipeline/runner.pyindex 17ef3791c..71a4a4175 100644--- a/pipelex/pipeline/runner.py+++ b/pipelex/pipeline/runner.py@@ -59,7 +59,7 @@ class PipelexRunner(RunnerProtocol["PipeOutput"]):library_dirs: list[str] | None = None,bundle_uris: list[str] | None = None,pipe_run_mode: PipeRunMode | None = None,- is_mock_inference: bool = False,+ is_mock_usage: bool = False,search_domain_codes: list[str] | None = None,user_id: str | None = None,execution_config: PipelineExecutionConfig | None = None,@@ -69,7 +69,7 @@ class PipelexRunner(RunnerProtocol["PipeOutput"]):self.library_dirs = library_dirsself.bundle_uris = bundle_urisself.pipe_run_mode = pipe_run_mode- self.is_mock_inference = is_mock_inference+ self.is_mock_usage = is_mock_usageself.search_domain_codes = search_domain_codesself.user_id = user_idself.execution_config = execution_config@@ -155,7 +155,7 @@ class PipelexRunner(RunnerProtocol["PipeOutput"]):output_multiplicity=output_multiplicity,dynamic_output_concept_ref=dynamic_output_concept_ref,pipe_run_mode=self.pipe_run_mode,- is_mock_inference=self.is_mock_inference,+ is_mock_usage=self.is_mock_usage,search_domain_codes=self.search_domain_codes,user_id=self.user_id,)diff --git a/pipelex/system/configuration/configs.py b/pipelex/system/configuration/configs.pyindex d0a0a6654..71eada30a 100644--- a/pipelex/system/configuration/configs.py+++ b/pipelex/system/configuration/configs.py@@ -61,7 +61,6 @@ class PipeRunConfig(ConfigModel):class DryRunConfig(ConfigModel):- apply_to_jinja2_rendering: booltext_gen_truncate_length: intnb_list_items: intnb_extract_pages: intdiff --git a/pipelex/temporal/tasks.py b/pipelex/temporal/tasks.pyindex 795f71e03..45f168e39 100644--- a/pipelex/temporal/tasks.py+++ b/pipelex/temporal/tasks.py@@ -9,7 +9,9 @@ from pipelex.temporal.tprl_content_generation.act_render_page_views import act_rfrom pipelex.temporal.tprl_content_generation.act_search_generate import act_search_gen_sourced_answer, act_search_gen_structuredfrom pipelex.temporal.tprl_pipe.act_assemble_tracing import act_assemble_tracingfrom pipelex.temporal.tprl_pipe.act_deliver import act_deliver+from pipelex.temporal.tprl_pipe.act_dry_validate import act_dry_validatefrom pipelex.temporal.tprl_pipe.act_flush_trace_events import act_flush_trace_events+from pipelex.temporal.tprl_pipe.wf_dry_validate import WfDryValidatefrom pipelex.temporal.tprl_pipe.wf_pipe_router import WfPipeRouterfrom pipelex.temporal.tprl_pipe.wf_pipe_run import WfPipeRunfrom pipelex.types import StrEnum@@ -40,10 +42,12 @@ class Tasks:workflow_list=[WfPipeRouter,WfPipeRun,+ WfDryValidate,],activity_list=[act_assemble_tracing,act_deliver,+ act_dry_validate,act_flush_trace_events,],),diff --git a/pipelex/temporal/test_extras/wf_test_content_generator_child.py b/pipelex/temporal/test_extras/wf_test_content_generator_child.pyindex 75ff9e738..a3987a3e8 100644--- a/pipelex/temporal/test_extras/wf_test_content_generator_child.py+++ b/pipelex/temporal/test_extras/wf_test_content_generator_child.py@@ -2,19 +2,18 @@ from temporalio import workflowwith workflow.unsafe.imports_passed_through():from pipelex import pretty_print- from pipelex.cogt.content_generation.content_generator_dry import ContentGeneratorDry+ from pipelex.cogt.content_generation.cogt_run_params import CogtRunParamsfrom pipelex.cogt.content_generation.content_generator_protocol import ContentGeneratorProtocol # noqa: TC001- from pipelex.cogt.content_generation.generated_content_factory import GeneratedContentFactoryfrom pipelex.cogt.extract.extract_input import ExtractInputfrom pipelex.cogt.extract.extract_job_components import ExtractJobConfig, ExtractJobParamsfrom pipelex.cogt.img_gen.img_gen_prompt import ImgGenPromptfrom pipelex.cogt.llm.llm_prompt import LLMPromptfrom pipelex.hub import get_model_deck+ from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipeline.job_metadata import JobMetadatafrom pipelex.temporal.log_temporal import workflow_logfrom pipelex.temporal.test_extras.temporal_registry_test_models import Personfrom pipelex.temporal.tprl_content_generation.content_generator_in_workflow_factory import ContentGeneratorInWorkflowFactory- from pipelex.tools.storage.in_memory_storage_provider import InMemoryStorageProviderfrom tests.integration.pipelex.temporal.test_data import PipeTestCases@@ -46,23 +45,21 @@ class WfTestContentGeneratorChild:@workflow.runasync def run(self, is_dry_run: bool = False):workflow_log.debug("Workflow start")- content_generator: ContentGeneratorProtocol- if is_dry_run:- content_generator = ContentGeneratorDry()- else:- generated_content_factory = GeneratedContentFactory(storage_provider=InMemoryStorageProvider())- content_generator = ContentGeneratorInWorkflowFactory.make_content_generator_in_workflow(- generated_content_factory=generated_content_factory,- )+ # One generator for both arms: under is_dry_run the activities are still dispatched and+ # the cogt leaf mocks inside them (run_mode=DRY rides cogt_run_params — Part B).+ content_generator: ContentGeneratorProtocol = ContentGeneratorInWorkflowFactory.make_content_generator_in_workflow()job_metadata = JobMetadata(user_id="temporal-test",pipeline_run_id=workflow.info().workflow_id,)+ run_mode = PipeRunMode.DRY if is_dry_run else PipeRunMode.LIVE+ cogt_run_params = CogtRunParams(run_mode=run_mode)llm_setting_for_text = get_model_deck().get_llm_setting(llm_choice="$testing-text")crafted_text = await content_generator.make_llm_text(job_metadata=job_metadata,+ cogt_run_params=cogt_run_params,llm_setting_main=llm_setting_for_text,llm_prompt_for_text=LLMPrompt(user_text=USER_TEXT_FOR_BASE),)@@ -71,6 +68,7 @@ class WfTestContentGeneratorChild:llm_setting_for_object = get_model_deck().get_llm_setting(llm_choice="$testing-structured")crafted_object_direct = await content_generator.make_object(job_metadata=job_metadata,+ cogt_run_params=cogt_run_params,object_class=Person,llm_setting_for_object=llm_setting_for_object,llm_prompt_for_object=LLMPrompt(user_text=USER_TEXT_FOR_SINGLE_PERSON),@@ -79,6 +77,7 @@ class WfTestContentGeneratorChild:crafted_object_list_direct = await content_generator.make_object_list(job_metadata=job_metadata,+ cogt_run_params=cogt_run_params,object_class=Person,llm_setting_for_object_list=llm_setting_for_object,llm_prompt_for_object_list=LLMPrompt(user_text=USER_TEXTS_FOR_PEOPLE_STR),@@ -88,6 +87,7 @@ class WfTestContentGeneratorChild:crafted_image = await content_generator.make_single_image(img_gen_handle="gpt-image-1-mini",job_metadata=job_metadata,+ cogt_run_params=cogt_run_params,img_gen_prompt=ImgGenPrompt(positive_text=POSITIVE_TEXT_FOR_IMAGE),)pretty_print(crafted_image, title="make_single_image")@@ -97,6 +97,7 @@ class WfTestContentGeneratorChild:}jinja2_text = await content_generator.make_templated_text(job_metadata=job_metadata,+ cogt_run_params=cogt_run_params,context=context,template="♦️♦️ {{ the_answer }} ♦️♦️",)@@ -106,6 +107,7 @@ class WfTestContentGeneratorChild:extract_input=ExtractInput(image_uri=PipeTestCases.IMG_EXPENSE_REPORT_1,),+ cogt_run_params=cogt_run_params,extract_handle="azure-document-intelligence",job_metadata=job_metadata,extract_job_params=ExtractJobParams.make_default_extract_job_params(),diff --git a/pipelex/temporal/test_extras/wf_test_content_generator_pdf_page_views.py b/pipelex/temporal/test_extras/wf_test_content_generator_pdf_page_views.pyindex 9a4f3b4fa..dd8aa33b3 100644--- a/pipelex/temporal/test_extras/wf_test_content_generator_pdf_page_views.py+++ b/pipelex/temporal/test_extras/wf_test_content_generator_pdf_page_views.py@@ -2,16 +2,15 @@ from temporalio import workflowwith workflow.unsafe.imports_passed_through():from pipelex import pretty_print- from pipelex.cogt.content_generation.content_generator_dry import ContentGeneratorDry+ from pipelex.cogt.content_generation.cogt_run_params import CogtRunParamsfrom pipelex.cogt.content_generation.content_generator_protocol import ContentGeneratorProtocol # noqa: TC001- from pipelex.cogt.content_generation.generated_content_factory import GeneratedContentFactoryfrom pipelex.cogt.extract.extract_input import ExtractInputfrom pipelex.cogt.extract.extract_job_components import ExtractJobConfig, ExtractJobParams+ from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipeline.job_metadata import JobMetadatafrom pipelex.temporal.exceptions import ContentGenerationErrorfrom pipelex.temporal.log_temporal import workflow_logfrom pipelex.temporal.tprl_content_generation.content_generator_in_workflow_factory import ContentGeneratorInWorkflowFactory- from pipelex.tools.storage.in_memory_storage_provider import InMemoryStorageProviderfrom tests.integration.pipelex.temporal.test_data import PipeTestCases@@ -29,24 +28,22 @@ class WfTestContentGeneratorPdfPageViews:@workflow.runasync def run(self, is_dry_run: bool = False) -> None:workflow_log.debug("Workflow start")- content_generator: ContentGeneratorProtocol- if is_dry_run:- content_generator = ContentGeneratorDry()- else:- generated_content_factory = GeneratedContentFactory(storage_provider=InMemoryStorageProvider())- content_generator = ContentGeneratorInWorkflowFactory.make_content_generator_in_workflow(- generated_content_factory=generated_content_factory,- )+ # One generator for both arms: under is_dry_run the activities are still dispatched and+ # the cogt leaf mocks inside them (run_mode=DRY rides cogt_run_params — Part B).+ content_generator: ContentGeneratorProtocol = ContentGeneratorInWorkflowFactory.make_content_generator_in_workflow()job_metadata = JobMetadata(user_id="temporal-test",pipeline_run_id=workflow.info().workflow_id,)+ run_mode = PipeRunMode.DRY if is_dry_run else PipeRunMode.LIVE+ cogt_run_params = CogtRunParams(run_mode=run_mode)page_contents = await content_generator.make_extract_pages(extract_input=ExtractInput(document_uri=PipeTestCases.JOB_OFFER_PDF_LOCAL,),+ cogt_run_params=cogt_run_params,extract_handle="azure-document-intelligence",job_metadata=job_metadata,extract_job_params=ExtractJobParams(diff --git a/pipelex/temporal/test_extras/wf_test_structured_output_cross_process.py b/pipelex/temporal/test_extras/wf_test_structured_output_cross_process.pyindex a867247f3..b805e2c47 100644--- a/pipelex/temporal/test_extras/wf_test_structured_output_cross_process.py+++ b/pipelex/temporal/test_extras/wf_test_structured_output_cross_process.py@@ -6,16 +6,16 @@ from temporalio import workflow# protocol is therefore deliberate — moving it under ``TYPE_CHECKING`` would# break replay because the symbol is type-annotated on a runtime variable.with workflow.unsafe.imports_passed_through():+ from pipelex.cogt.content_generation.cogt_run_params import CogtRunParamsfrom pipelex.cogt.content_generation.content_generator_protocol import ContentGeneratorProtocol # noqa: TC001- from pipelex.cogt.content_generation.generated_content_factory import GeneratedContentFactoryfrom pipelex.cogt.llm.llm_prompt import LLMPromptfrom pipelex.cogt.llm.llm_setting import LLMSetting+ from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipeline.job_metadata import JobMetadatafrom pipelex.temporal.exceptions import ContentGenerationErrorfrom pipelex.temporal.log_temporal import workflow_logfrom pipelex.temporal.test_extras.temporal_registry_test_models import FixtureCustomer, FixtureInvoice, FixtureLineItemfrom pipelex.temporal.tprl_content_generation.content_generator_in_workflow_factory import ContentGeneratorInWorkflowFactory- from pipelex.tools.storage.in_memory_storage_provider import InMemoryStorageProvider@workflow.defn(name="wf_test_structured_output_cross_process")@@ -33,18 +33,17 @@ class WfTestStructuredOutputCrossProcess:@workflow.runasync def run(self, is_list: bool) -> None:workflow_log.debug("Workflow start")- generated_content_factory = GeneratedContentFactory(storage_provider=InMemoryStorageProvider())- content_generator: ContentGeneratorProtocol = ContentGeneratorInWorkflowFactory.make_content_generator_in_workflow(- generated_content_factory=generated_content_factory,- )+ content_generator: ContentGeneratorProtocol = ContentGeneratorInWorkflowFactory.make_content_generator_in_workflow()job_metadata = JobMetadata(user_id="temporal-test", pipeline_run_id=workflow.info().workflow_id)+ cogt_run_params = CogtRunParams(run_mode=PipeRunMode.LIVE)llm_setting = LLMSetting(model="$testing-structured", temperature=0.0)prompt = LLMPrompt(user_text="round-trip nested structured output")if is_list:invoices = await content_generator.make_object_list(job_metadata=job_metadata,+ cogt_run_params=cogt_run_params,object_class=FixtureInvoice,llm_setting_for_object_list=llm_setting,llm_prompt_for_object_list=prompt,@@ -53,6 +52,7 @@ class WfTestStructuredOutputCrossProcess:else:invoice = await content_generator.make_object(job_metadata=job_metadata,+ cogt_run_params=cogt_run_params,object_class=FixtureInvoice,llm_setting_for_object=llm_setting,llm_prompt_for_object=prompt,diff --git a/pipelex/temporal/tprl_content_generation/act_render_page_views.py b/pipelex/temporal/tprl_content_generation/act_render_page_views.pyindex 0a12d6d4b..52bd08674 100644--- a/pipelex/temporal/tprl_content_generation/act_render_page_views.py+++ b/pipelex/temporal/tprl_content_generation/act_render_page_views.py@@ -3,12 +3,10 @@ from temporalio import activityfrom pipelex import logfrom pipelex.cogt.content_generation.assignment_models import RenderPageViewsAssignmentfrom pipelex.cogt.content_generation.generated_content_factory import GeneratedContentFactory-from pipelex.cogt.image.generated_image import GeneratedImageRawDetails+from pipelex.cogt.content_generation.render_generate import render_page_views_and_storefrom pipelex.core.stuffs.image_content import ImageContentfrom pipelex.hub import get_storage_providerfrom pipelex.temporal.tprl.activity_error_boundary import convert_pipelex_errors-from pipelex.tools.misc.image_utils import ImageFormat-from pipelex.tools.pdf.pypdfium2_renderer import pypdfium2_renderer@activity.defn@@ -16,26 +14,14 @@ from pipelex.tools.pdf.pypdfium2_renderer import pypdfium2_rendererasync def act_render_page_views(render_assignment: RenderPageViewsAssignment) -> list[ImageContent]:"""Render PDF pages as images, store them, and return lightweight ImageContent references.- Performs pypdfium2 rendering and S3 storage within the activity so that- large binary data never crosses a Temporal workflow boundary.+ Performs pypdfium2 rendering and storage within the activity so that large binary data+ never crosses a Temporal workflow boundary. The rendering + storing logic lives in the+ framework-agnostic ``render_page_views_and_store`` leaf, shared with the direct backend."""log.dev("act_render_page_views")- page_view_images = await pypdfium2_renderer.render_pdf_pages_from_uri(- pdf_uri=render_assignment.document_uri,- dpi=render_assignment.page_views_dpi,- )storage_provider = get_storage_provider()generated_content_factory = GeneratedContentFactory(storage_provider=storage_provider)- image_contents: list[ImageContent] = []- for page_view_image in page_view_images:- raw_details = GeneratedImageRawDetails.make_from_pil_image(- pil_image=page_view_image,- image_format=ImageFormat.PNG,- )- image_content = await generated_content_factory.make_image_content(- primary_id=render_assignment.job_metadata.user_id,- secondary_id=render_assignment.job_metadata.pipeline_run_id,- raw_details=raw_details,- )- image_contents.append(image_content)- return image_contents+ return await render_page_views_and_store(+ render_assignment=render_assignment,+ generated_content_factory=generated_content_factory,+ )diff --git a/pipelex/temporal/tprl_content_generation/content_generator_in_workflow.py b/pipelex/temporal/tprl_content_generation/content_generator_in_workflow.pyindex 01283983f..a46c5d5f7 100644--- a/pipelex/temporal/tprl_content_generation/content_generator_in_workflow.py+++ b/pipelex/temporal/tprl_content_generation/content_generator_in_workflow.py@@ -16,13 +16,11 @@ from pipelex.cogt.content_generation.assignment_models import (SearchObjectAssignment,TemplatingAssignment,)+from pipelex.cogt.content_generation.cogt_run_params import CogtRunParamsfrom pipelex.cogt.content_generation.content_generator_protocol import ContentGeneratorProtocol, update_job_metadata-from pipelex.cogt.content_generation.exceptions import MockInferenceObjectFidelityError-from pipelex.cogt.content_generation.generated_content_factory import GeneratedContentFactory+from pipelex.cogt.content_generation.exceptions import DryRunObjectFidelityErrorfrom pipelex.cogt.extract.extract_input import ExtractInputfrom pipelex.cogt.extract.extract_job_components import ExtractJobConfig, ExtractJobParams-from pipelex.cogt.extract.extract_output import ExtractOutput-from pipelex.cogt.image.generated_image import GeneratedImageRawDetailsfrom pipelex.cogt.img_gen.img_gen_job_components import ImgGenJobConfig, ImgGenJobParamsfrom pipelex.cogt.img_gen.img_gen_prompt import ImgGenPromptfrom pipelex.cogt.llm.llm_prompt import LLMPrompt@@ -54,7 +52,7 @@ def _revalidate_against_object_class(raw_obj: BaseModel,object_class: type[BaseModelTypeVar],*,- is_mock_inference: bool,+ is_mock_built: bool,) -> BaseModelTypeVar:"""Re-validate an activity-boundary object against the original ``object_class``.@@ -62,20 +60,20 @@ def _revalidate_against_object_class(activity-boundary ``BaseModel`` through json (``mode="json"`` is required for fields that need json-modeserialization to round-trip cleanly) into the caller's original concrete class (e.g. ``StructuredContent``).- Under ``--mock-inference`` the object was built from the schema-reconstructed class inside- ``act_llm_gen_object*``, which can drop invariants the original class enforces (custom validators,- ``json_schema_extra`` format/pattern hints datamodel-code-generator omits on round-trip). That- re-validation failure is re-raised as a clear :class:`MockInferenceObjectFidelityError` rather than an- opaque pydantic crash mid-workflow (review F2). Scoped to the mock path only — a LIVE provider's invalid- output keeps its existing ``ValidationError``.+ Under the dry-run leaf mock (``run_mode=DRY``) the object was built from the+ schema-reconstructed class inside ``act_llm_gen_object*``, which can drop invariants the original class+ enforces (custom validators, ``json_schema_extra`` format/pattern hints datamodel-code-generator omits+ on round-trip). That re-validation failure is re-raised as a clear+ :class:`DryRunObjectFidelityError` rather than an opaque pydantic crash mid-workflow (review F2).+ Scoped to the dry path only — a LIVE provider's invalid output keeps its existing ``ValidationError``."""raw_data = raw_obj.model_dump(mode="json", serialize_as_any=True)- if not is_mock_inference:+ if not is_mock_built:return object_class.model_validate(raw_data)try:return object_class.model_validate(raw_data)except ValidationError as exc:- raise MockInferenceObjectFidelityError.for_object_class(object_class.__name__) from exc+ raise DryRunObjectFidelityError.for_object_class(object_class.__name__) from excclass ContentGeneratorInWorkflow(ContentGeneratorProtocol):@@ -88,16 +86,17 @@ class ContentGeneratorInWorkflow(ContentGeneratorProtocol):Activity IDs are never customized — the Temporal SDK assigns deterministicsequential integers per workflow run, which both guarantees uniqueness andis replay-safe by construction. Per-call meaning is carried in ``summary=``.- """- def __init__(self, generated_content_factory: GeneratedContentFactory) -> None:- self._generated_content_factory = generated_content_factory+ Holds no storage factory: all storage happens inside the activities (the+ ``*_and_store`` leaves), never on the workflow side.+ """@override@update_job_metadataasync def make_llm_text(self,job_metadata: JobMetadata,+ cogt_run_params: CogtRunParams,llm_setting_main: LLMSetting,llm_prompt_for_text: LLMPrompt,) -> str:@@ -106,6 +105,7 @@ class ContentGeneratorInWorkflow(ContentGeneratorProtocol):worker_config = get_config().temporal.worker_configllm_assignment = LLMAssignment(job_metadata=job_metadata,+ cogt_run_params=cogt_run_params,llm_setting=llm_setting_main,llm_prompt=llm_prompt_for_text,)@@ -136,6 +136,7 @@ class ContentGeneratorInWorkflow(ContentGeneratorProtocol):async def make_object(self,job_metadata: JobMetadata,+ cogt_run_params: CogtRunParams,object_class: type[BaseModelTypeVar],llm_setting_for_object: LLMSetting,llm_prompt_for_object: LLMPrompt,@@ -144,6 +145,7 @@ class ContentGeneratorInWorkflow(ContentGeneratorProtocol):worker_config = get_config().temporal.worker_configllm_assignment_for_object = LLMAssignment(job_metadata=job_metadata,+ cogt_run_params=cogt_run_params,llm_setting=llm_setting_for_object,llm_prompt=llm_prompt_for_object,)@@ -170,13 +172,14 @@ class ContentGeneratorInWorkflow(ContentGeneratorProtocol):raise TemporalError.from_app_error(exc=exc.cause) from excraiselog.verbose(f"ContentGeneratorInWorkflow generated object direct: {obj}")- return _revalidate_against_object_class(obj, object_class, is_mock_inference=job_metadata.is_mock_inference)+ return _revalidate_against_object_class(obj, object_class, is_mock_built=cogt_run_params.run_mode.is_dry)@override@update_job_metadataasync def make_object_list(self,job_metadata: JobMetadata,+ cogt_run_params: CogtRunParams,object_class: type[BaseModelTypeVar],llm_setting_for_object_list: LLMSetting,llm_prompt_for_object_list: LLMPrompt,@@ -185,12 +188,14 @@ class ContentGeneratorInWorkflow(ContentGeneratorProtocol):worker_config = get_config().temporal.worker_configllm_assignment_for_object = LLMAssignment(job_metadata=job_metadata,+ cogt_run_params=cogt_run_params,llm_setting=llm_setting_for_object_list,llm_prompt=llm_prompt_for_object_list,)object_assignment = ObjectAssignment.make_for_class(object_class=object_class,llm_assignment=llm_assignment_for_object,+ nb_items=nb_items,)dispatch_kwargs = worker_config.resolve_dispatch(activity_name=act_llm_gen_object_list.__name__,@@ -211,42 +216,14 @@ class ContentGeneratorInWorkflow(ContentGeneratorProtocol):raise TemporalError.from_app_error(exc=exc.cause) from excraiselog.verbose(f"ContentGeneratorInWorkflow generated object list direct: {obj_list}")- return [_revalidate_against_object_class(raw_obj, object_class, is_mock_inference=job_metadata.is_mock_inference) for raw_obj in obj_list]-- @override- async def make_image_content(- self,- job_metadata: JobMetadata,- generated_image_raw_details: GeneratedImageRawDetails,- img_gen_prompt: ImgGenPrompt | None,- ) -> ImageContent:- image_content = await self._generated_content_factory.make_image_content(- primary_id=job_metadata.user_id,- secondary_id=job_metadata.pipeline_run_id,- raw_details=generated_image_raw_details,- )- if img_gen_prompt:- image_content.source_prompt = img_gen_prompt.positive_text- image_content.source_negative_prompt = img_gen_prompt.negative_text- return image_content-- @override- async def make_page_contents(- self,- job_metadata: JobMetadata,- extract_output: ExtractOutput,- ) -> list[PageContent]:- return await self._generated_content_factory.make_page_contents(- primary_id=job_metadata.user_id,- secondary_id=job_metadata.pipeline_run_id,- extract_output=extract_output,- )+ return [_revalidate_against_object_class(raw_obj, object_class, is_mock_built=cogt_run_params.run_mode.is_dry) for raw_obj in obj_list]@override@update_job_metadataasync def make_single_image(self,job_metadata: JobMetadata,+ cogt_run_params: CogtRunParams,img_gen_handle: str,img_gen_prompt: ImgGenPrompt,img_gen_job_params: ImgGenJobParams | None = None,@@ -256,6 +233,7 @@ class ContentGeneratorInWorkflow(ContentGeneratorProtocol):img_gen_config = get_config().cogt.img_gen_configimg_gen_assignment = ImgGenAssignment(job_metadata=job_metadata,+ cogt_run_params=cogt_run_params,img_gen_handle=img_gen_handle,img_gen_prompt=img_gen_prompt,img_gen_job_params=img_gen_job_params or img_gen_config.make_default_img_gen_job_params(),@@ -292,6 +270,7 @@ class ContentGeneratorInWorkflow(ContentGeneratorProtocol):async def make_image_list(self,job_metadata: JobMetadata,+ cogt_run_params: CogtRunParams,img_gen_handle: str,img_gen_prompt: ImgGenPrompt,nb_images: int,@@ -302,6 +281,7 @@ class ContentGeneratorInWorkflow(ContentGeneratorProtocol):img_gen_config = get_config().cogt.img_gen_configimg_gen_assignment = ImgGenAssignment(job_metadata=job_metadata,+ cogt_run_params=cogt_run_params,img_gen_handle=img_gen_handle,img_gen_prompt=img_gen_prompt,img_gen_job_params=img_gen_job_params or img_gen_config.make_default_img_gen_job_params(),@@ -334,6 +314,7 @@ class ContentGeneratorInWorkflow(ContentGeneratorProtocol):async def make_templated_text(self,job_metadata: JobMetadata,+ cogt_run_params: CogtRunParams,context: dict[str, Any],template: str,templating_style: TemplatingStyle | None = None,@@ -342,6 +323,7 @@ class ContentGeneratorInWorkflow(ContentGeneratorProtocol):worker_config = get_config().temporal.worker_configtemplating_assignment = TemplatingAssignment(job_metadata=job_metadata,+ cogt_run_params=cogt_run_params,context=context,template=template,templating_style=templating_style,@@ -372,6 +354,7 @@ class ContentGeneratorInWorkflow(ContentGeneratorProtocol):async def make_render_page_views(self,job_metadata: JobMetadata,+ cogt_run_params: CogtRunParams,extract_input: ExtractInput,extract_handle: str,extract_job_params: ExtractJobParams | None = None,@@ -385,6 +368,7 @@ class ContentGeneratorInWorkflow(ContentGeneratorProtocol):page_views_dpi = job_params.page_views_dpi or get_config().cogt.extract_config.default_page_views_dpirender_assignment = RenderPageViewsAssignment(job_metadata=job_metadata,+ cogt_run_params=cogt_run_params,document_uri=extract_input.document_uri,page_views_dpi=page_views_dpi,)@@ -413,6 +397,7 @@ class ContentGeneratorInWorkflow(ContentGeneratorProtocol):async def make_extract_pages(self,job_metadata: JobMetadata,+ cogt_run_params: CogtRunParams,extract_input: ExtractInput,extract_handle: str,extract_job_params: ExtractJobParams,@@ -421,6 +406,7 @@ class ContentGeneratorInWorkflow(ContentGeneratorProtocol):worker_config = get_config().temporal.worker_configextract_assignment = ExtractAssignment(job_metadata=job_metadata,+ cogt_run_params=cogt_run_params,extract_handle=extract_handle,extract_input=extract_input,extract_job_params=extract_job_params,@@ -452,6 +438,7 @@ class ContentGeneratorInWorkflow(ContentGeneratorProtocol):page_views_dpi = extract_job_params.page_views_dpi or get_config().cogt.extract_config.default_page_views_dpirender_assignment = RenderPageViewsAssignment(job_metadata=job_metadata,+ cogt_run_params=cogt_run_params,document_uri=extract_input.document_uri,page_views_dpi=page_views_dpi,)@@ -548,10 +535,13 @@ class ContentGeneratorInWorkflow(ContentGeneratorProtocol):# json.loads(...)), so a malformed structured response raises a bare ValidationError here in# workflow code. Left raw it is neither WorkflowExecutionError nor PipelexError, so Temporal# treats it as a workflow-task failure and retries forever, hanging the submitter — the exact- # failure mode this seam exists to prevent. Convert it to a terminal ContentGenerationError (a- # PipelexError) so the workflow fails and surfaces as an ErrorReport, matching the direct path.+ # failure mode this seam exists to prevent. Convert it to a typed PipelexError so the workflow+ # fails and surfaces as an ErrorReport, matching the direct path: under a leaf mock the failure+ # is the known schema-round-trip fidelity gap (D6), otherwise it is a malformed provider response.try:return output_structure_class.model_validate(result_dict)except ValidationError as exc:+ if search_assignment.cogt_run_params.run_mode.is_dry:+ raise DryRunObjectFidelityError.for_object_class(output_structure_class.__name__) from excmsg = f"Structured search result failed validation against {output_structure_class.__name__}: {exc}"raise ContentGenerationError(msg) from excdiff --git a/pipelex/temporal/tprl_content_generation/content_generator_in_workflow_factory.py b/pipelex/temporal/tprl_content_generation/content_generator_in_workflow_factory.pyindex 444d9410a..80d373979 100644--- a/pipelex/temporal/tprl_content_generation/content_generator_in_workflow_factory.py+++ b/pipelex/temporal/tprl_content_generation/content_generator_in_workflow_factory.py@@ -1,11 +1,7 @@-from pipelex.cogt.content_generation.generated_content_factory import GeneratedContentFactoryfrom pipelex.temporal.tprl_content_generation.content_generator_in_workflow import ContentGeneratorInWorkflowclass ContentGeneratorInWorkflowFactory:@classmethod- def make_content_generator_in_workflow(- cls,- generated_content_factory: GeneratedContentFactory,- ) -> ContentGeneratorInWorkflow:- return ContentGeneratorInWorkflow(generated_content_factory=generated_content_factory)+ def make_content_generator_in_workflow(cls) -> ContentGeneratorInWorkflow:+ return ContentGeneratorInWorkflow()diff --git a/pipelex/temporal/tprl_pipe/act_dry_validate.py b/pipelex/temporal/tprl_pipe/act_dry_validate.pynew file mode 100644index 000000000..931daa9c2--- /dev/null+++ b/pipelex/temporal/tprl_pipe/act_dry_validate.py@@ -0,0 +1,144 @@+"""Temporal activity running the whole /validate job — validation sweep + graph dry-run — in one process.++When Temporal is enabled, the API dispatches the whole ``/validate`` job to a worker as ONE+activity (via the one-step wrapper workflow ``WfDryValidate``) instead of running the sweep+API-side and the graph dry-run as a top-level worker workflow. The body is fully in-process:++- the **sweep** is :func:`pipelex.pipeline.validate_bundle.validate_bundle` — the exact function+ the direct-mode ``/validate`` route calls, so both backends surface the SAME error contract+ (the categorized ``ValidateBundleError`` cascade) and the same status map. It is already+ Temporal-safe: ``BundleValidator.validate_pipes`` scopes its own in-process router and dry+ content generator and runs a locally-constructed ``PipeRun``.+- the **graph dry-run** is ``dry_run_pipe_in_process`` — in-process by construction (local+ ``PipeRun`` under ``scoped_event_log`` / ``scoped_pipe_router`` / ``scoped_content_generator``),+ tracing into an in-memory event log; the ``GraphSpec`` rides back on the activity result. No+ NDJSON file, no DynamoDB round-trip, no usage/cost reporting.++Library lifecycle (D4): the library is loaded ONCE — ``validate_bundle`` loads it and leaves it+current on success (its D6 loaded-on-success contract; it owns teardown on failure), the graph+runs against the same library, and this activity tears it down once in its ``finally``.++Error contract (D3): the output carries the success-only status map + the best-effort+``GraphSpec``. Validation failures (blueprint/factory/wiring errors, unexpected pipe failures,+strict-mode signature refusals) RAISE out of ``validate_bundle`` as ``ValidateBundleError`` and+cross the boundary as structured ``ErrorReport``s via ``convert_pipelex_errors`` — the API+renders them as the same RFC 7807 422 the direct path produces (identical ``error_type``,+``error_domain=input``, caller-facing message carrying the offending refs).++Graph best-effort (D5, widened to cross-backend parity): any ``PipelexError`` raised while+resolving or dry-running the graph pipe — plus pydantic ``ValidationError`` / polyfactory+``FactoryException`` (the mock-input mint shapes) — degrades to ``graph_spec=None``. This is+the exact contract the direct-mode route applies around ``dry_run_pipeline`` and the same+catch ``BundleValidator._classify_pipe`` uses, so both backends answer identically for the+same bundle. Only non-Pipelex programming bugs propagate and fail the activity, mirroring+``assemble_tracing``'s bug-propagation policy.+"""++import sys++from polyfactory.exceptions import FactoryException+from pydantic import BaseModel, ValidationError+from temporalio import activity++from pipelex import log+from pipelex.base_exceptions import PipelexError+from pipelex.core.bundles.pipelex_bundle_blueprint import PipelexBundleBlueprint+from pipelex.core.pipes.pipe_factory import PipeFactory+from pipelex.graph.graphspec import GraphSpec+from pipelex.hub import (+ clear_current_library,+ get_current_library_id_or_none,+ get_library_manager,+ get_required_pipe,+ set_current_library,+)+from pipelex.pipe_run.dry_run_pipeline import dry_run_pipe_in_process+from pipelex.pipeline.bundle_validator import DryRunOutput+from pipelex.pipeline.validate_bundle import validate_bundle+from pipelex.temporal.tprl.activity_error_boundary import convert_pipelex_errors+++class DryValidateArg(BaseModel):+ """Input for the dry-run+validation activity (serializable across the Temporal boundary)."""++ mthds_contents: list[str] | None = None+ allow_signatures: bool = False+ # Optional explicit pipe selection for the graph arm; defaults to the bundle's declared+ # main_pipe (qualified from the first blueprint declaring one). The sweep always covers the+ # whole bundle; no graph is produced when neither this nor a main_pipe is set.+ pipe_code: str | None = None+++class DryValidateResult(BaseModel):+ """Output of the dry-run+validation activity: success-only status map + best-effort graph (D3)."""++ dry_run_outputs: dict[str, DryRunOutput]+ graph_spec: GraphSpec | None = None++[email protected](name="act_dry_validate")+@convert_pipelex_errors+async def act_dry_validate(arg: DryValidateArg) -> DryValidateResult:+ """Run the validation sweep + the in-memory graph dry-run against one library, in-process.++ Raises (crossing as structured ``ErrorReport``s via ``convert_pipelex_errors``):+ ValidateBundleError: any validation failure — blueprint/factory/wiring errors, unexpected+ dry-run pipe failures, strict-mode signature refusals (the same categorized cascade+ the direct-mode route surfaces).+ PipelexError: non-validation failures (config, library, tracing infra).+ """+ prev_library_id = get_current_library_id_or_none()+ # Failure → validate_bundle owns its teardown and the error propagates through the boundary.+ # Success → the library is left loaded + current (D6) and THIS activity owns the teardown.+ validate_result = await validate_bundle(+ mthds_contents=arg.mthds_contents,+ allow_signatures=arg.allow_signatures,+ )+ library_id = get_current_library_id_or_none()+ try:+ # Graph: best-effort (D5), against the SAME loaded library. Pipe resolution sits INSIDE+ # the catch on purpose: an unknown explicit pipe_code degrades to graph_spec=None just+ # like any other graph-arm domain failure — same answer the direct route gives.+ graph_spec: GraphSpec | None = None+ graph_pipe_ref = arg.pipe_code or _qualified_main_pipe(validate_result.blueprints)+ if graph_pipe_ref and library_id:+ try:+ main_pipe = get_required_pipe(pipe_code=graph_pipe_ref)+ graph_spec = await dry_run_pipe_in_process(pipe=main_pipe, library_id=library_id)+ except (PipelexError, ValidationError, FactoryException) as graph_error:+ log.warning(+ f"act_dry_validate: graph dry-run of '{graph_pipe_ref}' did not produce a graph "+ f"({type(graph_error).__name__}); returning validation result without graph_spec"+ )++ return DryValidateResult(dry_run_outputs=validate_result.dry_run_result, graph_spec=graph_spec)+ finally:+ # Restore the caller's outer current-library FIRST so the guarantee survives a teardown+ # raise, then tear the validated library down once — mirrors acquire_and_validate.+ primary_error = sys.exc_info()[1]+ if prev_library_id is not None and prev_library_id != library_id:+ set_current_library(library_id=prev_library_id)+ else:+ clear_current_library()+ if library_id is not None:+ try:+ get_library_manager().teardown(library_id=library_id)+ except PipelexError as teardown_error:+ # A teardown failure must not REPLACE the body's in-flight error (the caller's+ # 422 would name the teardown instead of the user's actual problem) — suppress+ # it and let the primary propagate; raise it only when the body succeeded.+ # Mirrors PipeRun.run's close_tracer handling.+ if primary_error is None:+ raise+ log.error(+ f"act_dry_validate: library teardown also failed after a body error; "+ f"raising the original error. Suppressed teardown error: {teardown_error}"+ )+++def _qualified_main_pipe(blueprints: list[PipelexBundleBlueprint]) -> str | None:+ """Return the domain-qualified main_pipe of the first blueprint declaring one, else None."""+ for blueprint in blueprints:+ if blueprint.main_pipe:+ return PipeFactory.make_pipe_ref_with_domain(domain_code=blueprint.domain, pipe_code=blueprint.main_pipe)+ return Nonediff --git a/pipelex/temporal/tprl_pipe/dry_validate_dispatch.py b/pipelex/temporal/tprl_pipe/dry_validate_dispatch.pynew file mode 100644index 000000000..bcb2d9776--- /dev/null+++ b/pipelex/temporal/tprl_pipe/dry_validate_dispatch.py@@ -0,0 +1,62 @@+"""Submitter-side dispatch for the dry-run+validation wrapper workflow.++The single round-trip a Temporal-enabled ``/validate`` caller awaits: dispatch+``WfDryValidate`` (which runs the one ``act_dry_validate`` activity in-process on a worker)+and get back the per-pipe status map + the best-effort ``GraphSpec``. Used by the API route+(cross-repo, ``pipelex-api``) and by the Tier-2d e2e submitter script.+"""++from datetime import timedelta++from temporalio.common import RetryPolicy++from pipelex.config import get_config+from pipelex.pipeline.pipeline_factory import PipelineFactory+from pipelex.temporal.tprl.workflow_caller import WorkflowExecutorFactory+from pipelex.temporal.tprl_pipe.act_dry_validate import DryValidateArg, DryValidateResult+from pipelex.temporal.tprl_pipe.wf_dry_validate import WfDryValidate+++async def dispatch_dry_validate(+ arg: DryValidateArg,+ *,+ task_queue: str | None = None,+ should_auto_connect_temporal: bool = True,+) -> DryValidateResult:+ """Dispatch ``WfDryValidate`` to a worker and await ``{status map, GraphSpec}``.++ A validation failure (strict-mode signature refusal, unexpected pipe failure) comes back+ as ``WorkflowExecutionError`` carrying the structured ``ErrorReport`` recovered by+ ``WorkflowExecutor.execute_workflow`` — the same error shape every other Temporal dispatch+ surfaces, which the API renders as an RFC 7807 422.++ Args:+ arg: The serializable activity input (mthds_contents, allow_signatures, pipe_code).+ task_queue: Optional task-queue override; defaults to the configured default queue.+ should_auto_connect_temporal: Whether the executor may auto-connect the Temporal client.++ Returns:+ The activity result: per-pipe status map + best-effort GraphSpec.+ """+ worker_config = get_config().temporal.worker_config+ executor = WorkflowExecutorFactory[DryValidateArg, DryValidateResult]().create_executor(+ task_queue=task_queue or worker_config.default_task_queue,+ should_auto_connect_temporal=should_auto_connect_temporal,+ # Interactive round-trip: bound the whole workflow to the activity retry budget+ # (2 × 5 min attempts + dispatch slack) instead of the config default — a batch-tuned+ # 1-hour window during which a worker outage would queue stale validation work that no+ # HTTP caller is still waiting for. With this bound, an outage fails the caller fast.+ workflow_execution_timeout=timedelta(minutes=12),+ # No workflow-level retry (D-C5): validation is deterministic, so re-running the whole+ # wrapper workflow on failure is pure waste — the config default policy would re-run a+ # failed validation up to 3 times (its non_retryable list doesn't know ValidateBundleError).+ # Transient infra failures are already retried at the ACTIVITY tier by WfDryValidate's own+ # bounded policy.+ retry_policy=RetryPolicy(maximum_attempts=1),+ )+ workflow_id = executor.make_workflow_id(pipeline_run_id=f"dry_validate_{PipelineFactory.make_pipeline_run_id()}")+ return await executor.execute_workflow(+ workflow_class=WfDryValidate,+ workflow_arg=arg,+ workflow_id=workflow_id,+ )diff --git a/pipelex/temporal/tprl_pipe/wf_dry_validate.py b/pipelex/temporal/tprl_pipe/wf_dry_validate.pynew file mode 100644index 000000000..7862512fa--- /dev/null+++ b/pipelex/temporal/tprl_pipe/wf_dry_validate.py@@ -0,0 +1,45 @@+"""One-step wrapper workflow dispatching the dry-run+validation activity.++The dispatch unit the API awaits for a Temporal-enabled ``/validate``: it runs the single+``act_dry_validate`` activity and returns its result — functionally identical to a standalone+activity call, but it works on the current ``temporalio`` SDK with no bump (the pre-flight+dispatch decision). Replacing it with a true standalone activity is the deferred Phase-G0+optimization.+"""++from datetime import timedelta++from temporalio import workflow+from temporalio.common import RetryPolicy+from typing_extensions import override++with workflow.unsafe.imports_passed_through():+ from pipelex.temporal.tprl.workflow_caller import WorkflowClass+ from pipelex.temporal.tprl_pipe.act_dry_validate import DryValidateArg, DryValidateResult, act_dry_validate++[email protected](name="wf_dry_validate")+class WfDryValidate(WorkflowClass[DryValidateArg, DryValidateResult]):+ """Runs ``act_dry_validate`` once and returns its result."""++ @override+ @workflow.run+ async def run(self, workflow_arg: DryValidateArg) -> DryValidateResult:+ # D-C5: explicit timeout + retry bounds, deterministic in workflow code (no config reads).+ # Validation is deterministic — its failures must NOT retry. The whole sweep cascade+ # crosses the boundary as ONE type: validate_bundle wraps SignaturesNotAllowedError /+ # DryRunError / PipelexInterpreterError into ValidateBundleError before they escape, and+ # the activity-boundary TemporalError carries the OUTER PipelexError class name as the+ # ApplicationError type (which is what non_retryable_error_types matches). The graph arm+ # never raises domain errors (it degrades to graph_spec=None), so ValidateBundleError is+ # the entire deterministic-failure surface. Everything else (worker crash, transient+ # infra) gets one retry.+ return await workflow.execute_activity(+ act_dry_validate,+ arg=workflow_arg,+ start_to_close_timeout=timedelta(minutes=5),+ retry_policy=RetryPolicy(+ maximum_attempts=2,+ non_retryable_error_types=["ValidateBundleError"],+ ),+ )diff --git a/pipelex/test_extras/shared_pytest_plugins.py b/pipelex/test_extras/shared_pytest_plugins.pyindex 99d5c4bf7..d38c671ef 100644--- a/pipelex/test_extras/shared_pytest_plugins.py+++ b/pipelex/test_extras/shared_pytest_plugins.py@@ -6,7 +6,7 @@ from rich.console import Consolefrom rich.panel import Panelfrom pipelex.hub import get_console-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.system.environment import is_env_var_set, is_env_var_truthy, set_envfrom pipelex.system.runtime import CODEX_CLOUD_ENV_VAR_KEY, RunMode, runtime_managerfrom pipelex.tools.misc.placeholder import make_placeholder_value, value_is_placeholder@@ -214,7 +214,7 @@ def pytest_collection_modifyitems(config: Config, items: list[pytest.Item]) -> N- Are NOT marked with @pytest.mark.dry_runnableTests marked with both inference and dry_runnable can still run because- they use the mock ContentGeneratorDry.+ the cogt leaf mocks under run_mode=DRY."""if not config.getoption("--disable-inference", default=False):returndiff --git a/pipelex/tools/tabular/csv_codec.py b/pipelex/tools/tabular/csv_codec.pyindex 48af67131..e6fa84e9f 100644--- a/pipelex/tools/tabular/csv_codec.py+++ b/pipelex/tools/tabular/csv_codec.py@@ -46,7 +46,7 @@ DEFAULT_ENCODING = "utf-8"DEFAULT_READ_ENCODING = "utf-8-sig"_NONE_TYPE = type(None)-_UNION_TYPE = getattr(types, "UnionType", None) # Py3.10+: types.UnionType (PEP 604 `X | None`)+_UnionType = getattr(types, "UnionType", None) # Py3.10+: types.UnionType (PEP 604 `X | None`)# Scalar python types a CSV cell can round-trip. `bool` precedes `int` semantically# (bool is an int subclass) but membership here is by identity, so both are accepted._FLAT_SCALAR_TYPES: frozenset[type] = frozenset({str, int, float, bool, date, datetime})@@ -65,7 +65,7 @@ def _is_flat_annotation(annotation: Any) -> bool:containers, nested models, and ``Any``."""origin = get_origin(annotation)- if origin in {Union, _UNION_TYPE}:+ if origin in {Union, _UnionType}:non_none_args = [arg for arg in get_args(annotation) if arg is not _NONE_TYPE]if len(non_none_args) != 1:# A genuine multi-type union (e.g. int | str) is not flat.@@ -85,7 +85,7 @@ def _is_flat_annotation(annotation: Any) -> bool:def _annotation_allows_none(annotation: Any) -> bool:"""Whether a field annotation accepts ``None`` (an ``Optional`` / ``... | None`` field)."""- if get_origin(annotation) in {Union, _UNION_TYPE}:+ if get_origin(annotation) in {Union, _UnionType}:return any(arg is _NONE_TYPE for arg in get_args(annotation))return annotation is _NONE_TYPEdiff --git a/pyproject.toml b/pyproject.tomlindex 0835cbbfb..3df4962d3 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -28,7 +28,7 @@ dependencies = ["instructor>=1.13", # 1.13+ exposes instructor.core; 1.11/1.12 had mypy typing errors"jinja2>=3.1.4","json2html>=1.3.0",- "kajson==0.6.0",+ "kajson==0.7.0","markdown>=3.6","mthds>=0.3.0","networkx>=3.4.2",diff --git a/tests/e2e/agent_cli/conftest.py b/tests/e2e/agent_cli/conftest.pyindex a80bae9e2..029ac3665 100644--- a/tests/e2e/agent_cli/conftest.py+++ b/tests/e2e/agent_cli/conftest.py@@ -52,7 +52,7 @@ UNREACHABLE_REMOTE_CONFIG_URL = "http://127.0.0.1:1/pipelex_remote_config.json"# gateway_img_gen (image), gateway_extract, gateway_search — i.e. a value the matching worker factory's# gateway branch accepts (an unrecognized sdk like "gateway_image" hits ``case _: raise NotImplementedError``).# The sdk is consumed only at worker creation, which the offline dry-run tests never reach-# (ContentGeneratorDry mocks generation), so the membership check is indifferent to it — but it must still+# (the cogt leaf mocks generation under DRY), so the membership check is indifferent to it — but it must still# be real so the fake cache stays faithful and any non-dry consumer of this helper can build a worker._GATEWAY_SPEC_TEMPLATE_BY_TYPE: dict[ModelType, dict[str, Any]] = {ModelType.LLM: {"sdk": "gateway_completions", "model_type": "llm", "inputs": ["text"], "outputs": ["text", "structured"]},diff --git a/tests/e2e/agent_cli/test_offline_run_dry.py b/tests/e2e/agent_cli/test_offline_run_dry.pyindex 5f4597934..29ff6de73 100644--- a/tests/e2e/agent_cli/test_offline_run_dry.py+++ b/tests/e2e/agent_cli/test_offline_run_dry.py@@ -251,7 +251,7 @@ class TestOfflineDryRun:cache covers img_gen handles too — not just LLMs — so the gateway-membership check passes foran image model and the offline dry-run completes.- Scope note: dry-run mocks image generation via ``ContentGeneratorDry``, so this does NOT build+ Scope note: dry-run mocks image generation at the cogt leaf, so this does NOT buildthe img-gen worker and therefore does not exercise the spec's ``sdk`` -> worker-factory mapping;it covers membership + handle resolution for image models. The sdk-vs-factory contract is aruntime (non-dry) concern.diff --git a/tests/e2e/pipelex/pipes/pipe_controller/pipe_condition/test_pipe_condition_1.py b/tests/e2e/pipelex/pipes/pipe_controller/pipe_condition/test_pipe_condition_1.pyindex 48b1288cd..728b90340 100644--- a/tests/e2e/pipelex/pipes/pipe_controller/pipe_condition/test_pipe_condition_1.py+++ b/tests/e2e/pipelex/pipes/pipe_controller/pipe_condition/test_pipe_condition_1.py@@ -9,7 +9,7 @@ from pipelex.core.memory.working_memory_factory import WorkingMemoryFactoryfrom pipelex.core.stuffs.stuff_factory import StuffFactoryfrom pipelex.hub import get_pipe_router, get_required_pipefrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatafrom tests.integration.pipelex.pipes.controller.pipe_condition.pipe_condition import CategoryInputdiff --git a/tests/e2e/pipelex/pipes/pipe_controller/pipe_condition/test_pipe_condition_2.py b/tests/e2e/pipelex/pipes/pipe_controller/pipe_condition/test_pipe_condition_2.pyindex 099554d92..c7c62eeb5 100644--- a/tests/e2e/pipelex/pipes/pipe_controller/pipe_condition/test_pipe_condition_2.py+++ b/tests/e2e/pipelex/pipes/pipe_controller/pipe_condition/test_pipe_condition_2.py@@ -9,7 +9,7 @@ from pipelex.core.memory.working_memory_factory import WorkingMemoryFactoryfrom pipelex.core.stuffs.stuff_factory import StuffFactoryfrom pipelex.hub import get_pipe_router, get_required_pipefrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatafrom tests.integration.pipelex.pipes.controller.pipe_condition.pipe_condition import CategoryInputdiff --git a/tests/e2e/pipelex/pipes/pipe_controller/pipe_condition/test_pipe_condition_simple.py b/tests/e2e/pipelex/pipes/pipe_controller/pipe_condition/test_pipe_condition_simple.pyindex 20dcbdb89..ee9aa6d61 100644--- a/tests/e2e/pipelex/pipes/pipe_controller/pipe_condition/test_pipe_condition_simple.py+++ b/tests/e2e/pipelex/pipes/pipe_controller/pipe_condition/test_pipe_condition_simple.py@@ -14,7 +14,7 @@ from pipelex.core.pipes.pipe_factory import PipeFactoryfrom pipelex.pipe_controllers.condition.pipe_condition import PipeConditionfrom pipelex.pipe_controllers.condition.pipe_condition_blueprint import PipeConditionBlueprintfrom pipelex.pipe_run.exceptions import PipeRunError-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatafrom tests.integration.pipelex.pipes.controller.pipe_condition.pipe_condition import CategoryInputdiff --git a/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_batching.py b/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_batching.pyindex a295a0a5e..eb407ea9c 100644--- a/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_batching.py+++ b/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_batching.py@@ -11,7 +11,7 @@ from pipelex.core.memory.working_memory_factory import WorkingMemoryFactoryfrom pipelex.core.pipes.inputs.input_stuff_specs import TypedNamedStuffSpecfrom pipelex.core.stuffs.stuff_factory import StuffFactoryfrom pipelex.hub import get_required_pipe-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatafrom pipelex.pipeline.runner import PipelexRunnerdiff --git a/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_multiplicity.py b/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_multiplicity.pyindex ab3b2d2e5..478c3c4c6 100644--- a/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_multiplicity.py+++ b/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_multiplicity.py@@ -6,7 +6,7 @@ from pipelex.core.concepts.concept_factory import ConceptFactoryfrom pipelex.core.memory.working_memory_factory import WorkingMemoryFactoryfrom pipelex.core.stuffs.stuff_factory import StuffFactoryfrom pipelex.core.stuffs.text_content import TextContent-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipeline.runner import PipelexRunnerdiff --git a/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_simple.py b/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_simple.pyindex 4d3067a21..d00e5cf67 100644--- a/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_simple.py+++ b/tests/e2e/pipelex/pipes/pipe_controller/pipe_sequence/test_pipe_sequence_simple.py@@ -11,7 +11,7 @@ from pipelex.core.memory.working_memory_factory import WorkingMemoryFactoryfrom pipelex.core.stuffs.stuff_factory import StuffFactoryfrom pipelex.core.stuffs.text_content import TextContentfrom pipelex.hub import get_required_pipe-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatadiff --git a/tests/integration/pipelex/cogt/content_generation/test_data.py b/tests/integration/pipelex/cogt/content_generation/test_data.pynew file mode 100644index 000000000..0a12bef6e--- /dev/null+++ b/tests/integration/pipelex/cogt/content_generation/test_data.py@@ -0,0 +1,25 @@+"""Shared fixture classes for the object-mock fidelity tests (dry-run fidelity arms)."""++from pydantic import field_validator++from pipelex.core.stuffs.structured_content import StructuredContent+++class ConstrainedName(StructuredContent):+ """Original class with an invariant the JSON-schema round-trip cannot capture."""++ name: str++ @field_validator("name")+ @classmethod+ def _require_prefix(cls, value: str) -> str:+ if not value.startswith("PFX_"):+ msg = "name must start with 'PFX_'"+ raise ValueError(msg)+ return value+++class PlainName(StructuredContent):+ """Control class whose only constraint (a plain string field) survives the round-trip."""++ name: strdiff --git a/tests/integration/pipelex/cogt/content_generation/test_mock_inference_object_fidelity.py b/tests/integration/pipelex/cogt/content_generation/test_dry_run_object_fidelity.pysimilarity index 59%rename from tests/integration/pipelex/cogt/content_generation/test_mock_inference_object_fidelity.pyrename to tests/integration/pipelex/cogt/content_generation/test_dry_run_object_fidelity.pyindex 23f73df03..c16ff1c59 100644--- a/tests/integration/pipelex/cogt/content_generation/test_mock_inference_object_fidelity.py+++ b/tests/integration/pipelex/cogt/content_generation/test_dry_run_object_fidelity.py@@ -1,11 +1,11 @@-"""Regression test for the ``--mock-inference`` object-mock fidelity gap (review F2).+"""Regression test for the dry-run object-mock fidelity gap (review F2).-A ``--mock-inference`` object mock is built from the schema-reconstructed class, then re-validated by+A dry-run object mock is built from the schema-reconstructed class, then re-validated by``ContentGenerator.make_object`` against the **original** class. An invariant the JSON-schema round-tripcannot capture — here a custom ``@field_validator`` — is absent from the reconstructed class, so the mockcarries a value the original class rejects. The contract under test: that failure surfaces as a clear typed-``MockInferenceObjectFidelityError`` (naming the class, pointing at ``--dry-run``), NOT an opaque-``pydantic.ValidationError`` mid-run. A class without such a hidden invariant round-trips fine and the mock+``DryRunObjectFidelityError`` (naming the class and the ``examples`` / ``mock_format`` remedy), NOT an+opaque ``pydantic.ValidationError`` mid-run. A class without such a hidden invariant round-trips fine and the mockvalidates — proving the guard is scoped to genuine fidelity failures, not every mock object.No provider is ever called (the leaf mock short-circuits before ``get_llm_worker``), so this needs no@@ -13,62 +13,45 @@ inference marker."""import pytest-from pydantic import field_validator+from pipelex.cogt.content_generation.cogt_run_params import CogtRunParamsfrom pipelex.cogt.content_generation.content_generator_protocol import ContentGeneratorProtocol-from pipelex.cogt.content_generation.exceptions import MockInferenceObjectFidelityError+from pipelex.cogt.content_generation.exceptions import DryRunObjectFidelityErrorfrom pipelex.cogt.llm.llm_prompt import LLMPromptfrom pipelex.cogt.llm.llm_setting import LLMSetting-from pipelex.core.stuffs.structured_content import StructuredContent+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipeline.job_metadata import JobMetadata---class ConstrainedName(StructuredContent):- """Original class with an invariant the JSON-schema round-trip cannot capture."""-- name: str-- @field_validator("name")- @classmethod- def _require_prefix(cls, value: str) -> str:- if not value.startswith("PFX_"):- msg = "name must start with 'PFX_'"- raise ValueError(msg)- return value---class PlainName(StructuredContent):- """Control class whose only constraint (a plain string field) survives the round-trip."""-- name: str+from tests.integration.pipelex.cogt.content_generation.test_data import ConstrainedName, PlainName@pytest.mark.asyncio(loop_scope="class")-class TestMockInferenceObjectFidelity:- def _mock_job_metadata(self, job_metadata: JobMetadata) -> JobMetadata:- return job_metadata.model_copy(update={"is_mock_inference": True})+class TestDryRunObjectFidelity:+ def _dry_cogt_run_params(self) -> CogtRunParams:+ return CogtRunParams(run_mode=PipeRunMode.DRY)async def test_make_object_raises_typed_error_on_fidelity_gap(self, job_metadata: JobMetadata, content_generator: ContentGeneratorProtocol) -> None:- """make_object re-raises the dropped-invariant ValidationError as MockInferenceObjectFidelityError."""- with pytest.raises(MockInferenceObjectFidelityError) as exc_info:+ """make_object re-raises the dropped-invariant ValidationError as DryRunObjectFidelityError."""+ with pytest.raises(DryRunObjectFidelityError) as exc_info:await content_generator.make_object(- job_metadata=self._mock_job_metadata(job_metadata),+ job_metadata=job_metadata,+ cogt_run_params=self._dry_cogt_run_params(),object_class=ConstrainedName,llm_prompt_for_object=LLMPrompt(user_text="make a prefixed name"),llm_setting_for_object=LLMSetting(model="gpt-4o", temperature=0.5),)assert ConstrainedName.__name__ in str(exc_info.value)- assert "--dry-run" in str(exc_info.value)+ assert "mock_format" in str(exc_info.value)async def test_make_object_list_raises_typed_error_on_fidelity_gap(self, job_metadata: JobMetadata, content_generator: ContentGeneratorProtocol) -> None:"""make_object_list re-raises the per-item fidelity failure as the same typed error."""- with pytest.raises(MockInferenceObjectFidelityError):+ with pytest.raises(DryRunObjectFidelityError):await content_generator.make_object_list(- job_metadata=self._mock_job_metadata(job_metadata),+ job_metadata=job_metadata,+ cogt_run_params=self._dry_cogt_run_params(),object_class=ConstrainedName,llm_prompt_for_object_list=LLMPrompt(user_text="make prefixed names"),llm_setting_for_object_list=LLMSetting(model="gpt-4o", temperature=0.5),@@ -79,7 +62,8 @@ class TestMockInferenceObjectFidelity:) -> None:"""A class whose constraints survive the round-trip mocks and re-validates without the guard firing."""result = await content_generator.make_object(- job_metadata=self._mock_job_metadata(job_metadata),+ job_metadata=job_metadata,+ cogt_run_params=self._dry_cogt_run_params(),object_class=PlainName,llm_prompt_for_object=LLMPrompt(user_text="make a plain name"),llm_setting_for_object=LLMSetting(model="gpt-4o", temperature=0.5),diff --git a/tests/integration/pipelex/cogt/content_generation/test_leaf_dry_object_mocks.py b/tests/integration/pipelex/cogt/content_generation/test_leaf_dry_object_mocks.pynew file mode 100644index 000000000..2956e7a95--- /dev/null+++ b/tests/integration/pipelex/cogt/content_generation/test_leaf_dry_object_mocks.py@@ -0,0 +1,185 @@+"""Integration tests for the ``run_mode == DRY`` object mocks at the cogt leaf.++The dry object mock is built from the **schema-reconstructed** class — the single schema-based+mock site shared by both backends (pre-flight decision 2), exercising real datamodel-code-generator+codegen. Contracts pinned here:++- a representative ``StructuredContent`` mocks into a valid instance of the original class+ (the object-mock fidelity pin mandated by pre-flight decision 2);+- ``nb_items`` carried on ``ObjectAssignment`` controls the dry list length, falling back to+ ``dry_run_config.nb_list_items`` (eng review D11);+- a hidden invariant the schema round-trip drops surfaces as ``DryRunObjectFidelityError`` (eng review D6);+- the structured-search dry leaf returns a dict that validates against the original class.++No provider is ever called (the leaves short-circuit before any worker), so no inference marker.+"""++import pytest++from pipelex.cogt.content_generation.assignment_models import LLMAssignment, ObjectAssignment, SearchAssignment, SearchObjectAssignment+from pipelex.cogt.content_generation.cogt_run_params import CogtRunParams+from pipelex.cogt.content_generation.content_generator_protocol import ContentGeneratorProtocol+from pipelex.cogt.content_generation.dry_mock import dry_llm_gen_object, dry_llm_gen_object_list+from pipelex.cogt.content_generation.exceptions import DryRunObjectFidelityError+from pipelex.cogt.content_generation.search_generate import search_gen_structured+from pipelex.cogt.llm.llm_prompt import LLMPrompt+from pipelex.cogt.llm.llm_setting import LLMSetting+from pipelex.cogt.search.search_setting import SearchSetting+from pipelex.config import get_config+from pipelex.core.stuffs.structured_content import StructuredContent+from pipelex.pipe_run.pipe_run_mode import PipeRunMode+from pipelex.pipeline.job_metadata import JobMetadata+from tests.integration.pipelex.cogt.content_generation.test_data import ConstrainedName+++class RepresentativeInvoiceLine(StructuredContent):+ """Representative structured output: mixed primitive types and an optional field."""++ label: str+ quantity: int+ unit_price: float+ discounted: bool+ note: str | None = None+++class StructuredAnswer(StructuredContent):+ """Output structure for the structured-search dry arm."""++ answer: str+ confidence: float+++class SpecWithPipeCode(StructuredContent):+ """Pipe-spec-shaped item: carries the ``pipe_code`` field the mock_main coordination targets."""++ pipe_code: str+ description: str+++def _dry_object_assignment(object_class: type[StructuredContent], nb_items: int | None = None) -> ObjectAssignment:+ llm_assignment = LLMAssignment(+ job_metadata=JobMetadata(user_id="u", pipeline_run_id="run_dry_objects"),+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.DRY),+ llm_setting=LLMSetting(model="gpt-4o", temperature=0.5),+ llm_prompt=LLMPrompt(user_text="make objects"),+ )+ return ObjectAssignment.make_for_class(object_class=object_class, llm_assignment=llm_assignment, nb_items=nb_items)++[email protected](loop_scope="class")+class TestLeafDryObjectMocks:+ async def test_dry_object_mock_fidelity_on_representative_structured_content(self) -> None:+ """The schema-built mock validates back into the original representative class — the fidelity pin."""+ raw_mock = dry_llm_gen_object(_dry_object_assignment(RepresentativeInvoiceLine))++ revalidated = RepresentativeInvoiceLine.model_validate(raw_mock.model_dump(serialize_as_any=True))+ assert isinstance(revalidated.label, str)+ assert isinstance(revalidated.quantity, int)+ assert isinstance(revalidated.unit_price, float)+ assert isinstance(revalidated.discounted, bool)++ @pytest.mark.parametrize("nb_items", [4, 0])+ async def test_dry_object_list_honors_fixed_nb_items(self, nb_items: int) -> None:+ """A fixed nb_items on the assignment controls the dry list length (D11) — including an explicit 0."""+ mocks = dry_llm_gen_object_list(_dry_object_assignment(RepresentativeInvoiceLine, nb_items=nb_items))++ assert len(mocks) == nb_items++ async def test_dry_object_list_stamps_mock_main_coordination(self) -> None:+ """The dry leaf stamps the first pipe-spec-shaped item with pipe_code='mock_main' (D3).++ This is what keeps builder-bundle dry-validation working through the leaf mock: the mocked+ ``BundleHeaderSpec.main_pipe`` (``examples=["mock_main"]``) must name an existing pipe.+ """+ mocks = dry_llm_gen_object_list(_dry_object_assignment(SpecWithPipeCode, nb_items=3))++ first_item_pipe_code = getattr(mocks[0], "pipe_code", None)+ assert first_item_pipe_code == "mock_main"++ async def test_dry_object_list_stamps_regardless_of_mock_usage(self) -> None:+ """The stamp is unconditional on the is_mock_usage sub-flag — it only changes reporting (D3)."""+ llm_assignment = LLMAssignment(+ job_metadata=JobMetadata(user_id="u", pipeline_run_id="run_mock_usage_stamp"),+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.DRY, is_mock_usage=True),+ llm_setting=LLMSetting(model="gpt-4o", temperature=0.5),+ llm_prompt=LLMPrompt(user_text="make specs"),+ )+ object_assignment = ObjectAssignment.make_for_class(object_class=SpecWithPipeCode, llm_assignment=llm_assignment, nb_items=3)++ mocks = dry_llm_gen_object_list(object_assignment)++ first_item_pipe_code = getattr(mocks[0], "pipe_code", None)+ assert first_item_pipe_code == "mock_main"++ async def test_dry_object_list_defaults_to_config_length(self) -> None:+ """Without nb_items the dry list falls back to dry_run_config.nb_list_items."""+ mocks = dry_llm_gen_object_list(_dry_object_assignment(RepresentativeInvoiceLine))++ assert len(mocks) == get_config().pipelex.dry_run_config.nb_list_items++ async def test_generator_dry_object_list_keeps_fixed_length(self, job_metadata: JobMetadata, content_generator: ContentGeneratorProtocol) -> None:+ """End-to-end through ContentGenerator: a fixed nb_items survives down to the dry leaf (D11 regression)."""+ result = await content_generator.make_object_list(+ job_metadata=job_metadata,+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.DRY),+ object_class=RepresentativeInvoiceLine,+ llm_setting_for_object_list=LLMSetting(model="gpt-4o", temperature=0.5),+ llm_prompt_for_object_list=LLMPrompt(user_text="make invoice lines"),+ nb_items=3,+ )++ assert len(result) == 3+ assert all(isinstance(item, RepresentativeInvoiceLine) for item in result)++ async def test_generator_dry_arm_raises_typed_fidelity_error(+ self, job_metadata: JobMetadata, content_generator: ContentGeneratorProtocol+ ) -> None:+ """The DRY arm of the fidelity wrapper fires (D6): a dropped invariant surfaces as the typed error."""+ with pytest.raises(DryRunObjectFidelityError) as exc_info:+ await content_generator.make_object(+ job_metadata=job_metadata,+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.DRY),+ object_class=ConstrainedName,+ llm_setting_for_object=LLMSetting(model="gpt-4o", temperature=0.5),+ llm_prompt_for_object=LLMPrompt(user_text="make a prefixed name"),+ )+ assert ConstrainedName.__name__ in str(exc_info.value)++ async def test_dry_search_structured_fidelity_gap_raises_typed_error(+ self, job_metadata: JobMetadata, content_generator: ContentGeneratorProtocol+ ) -> None:+ """The structured-search path carries the same D6 fidelity guard as the object paths:+ a schema-built dict that drops a hidden invariant surfaces as DryRunObjectFidelityError.+ """+ search_assignment = SearchAssignment(+ job_metadata=job_metadata,+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.DRY),+ query="what is pipelex?",+ search_setting=SearchSetting(model="mock-search-handle"),+ )++ with pytest.raises(DryRunObjectFidelityError) as exc_info:+ await content_generator.make_search_structured(+ output_structure_class=ConstrainedName,+ search_assignment=search_assignment,+ )+ assert ConstrainedName.__name__ in str(exc_info.value)++ async def test_dry_search_structured_dict_validates_against_original_class(self) -> None:+ """The structured-search dry leaf returns a dict the original output class accepts."""+ search_assignment = SearchAssignment(+ job_metadata=JobMetadata(user_id="u", pipeline_run_id="run_dry_search_structured"),+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.DRY),+ query="what is pipelex?",+ search_setting=SearchSetting(model="mock-search-handle"),+ )+ search_object_assignment = SearchObjectAssignment.make_for_class(+ output_class=StructuredAnswer,+ search_assignment=search_assignment,+ )++ result_dict = await search_gen_structured(search_object_assignment=search_object_assignment)++ validated = StructuredAnswer.model_validate(result_dict)+ assert isinstance(validated.answer, str)+ assert isinstance(validated.confidence, float)diff --git a/tests/integration/pipelex/cogt/test_content_generator.py b/tests/integration/pipelex/cogt/test_content_generator.pyindex ce5a98946..d632139d3 100644--- a/tests/integration/pipelex/cogt/test_content_generator.py+++ b/tests/integration/pipelex/cogt/test_content_generator.py@@ -1,6 +1,7 @@import pytestfrom pipelex import pretty_print+from pipelex.cogt.content_generation.cogt_run_params import CogtRunParamsfrom pipelex.cogt.content_generation.content_generator_protocol import ContentGeneratorProtocolfrom pipelex.cogt.exceptions import ModelNotFoundErrorfrom pipelex.cogt.extract.extract_input import ExtractInput@@ -11,6 +12,7 @@ from pipelex.cogt.llm.llm_setting import LLMSettingfrom pipelex.core.stuffs.image_content import ImageContentfrom pipelex.core.stuffs.page_content import PageContentfrom pipelex.hub import get_model_deck+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipeline.job_metadata import JobMetadatafrom tests.cases import DocumentTestCases, ImageTestCasesfrom tests.integration.pipelex.cogt.test_data import Employee@@ -61,6 +63,7 @@ class TestContentGenerator:text: str = await content_generator.make_llm_text(job_metadata=job_metadata,+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),llm_prompt_for_text=LLMPrompt(user_text=USER_TEXT_FOR_BASE),llm_setting_main=llm_setting_main,)@@ -75,6 +78,7 @@ class TestContentGenerator:person_direct: Employee = await content_generator.make_object(job_metadata=job_metadata,+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),object_class=Employee,llm_prompt_for_object=LLMPrompt(user_text=USER_TEXT_FOR_SINGLE_PERSON),llm_setting_for_object=llm_setting_for_object,@@ -90,6 +94,7 @@ class TestContentGenerator:person_list_direct: list[Employee] = await content_generator.make_object_list(job_metadata=job_metadata,+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),object_class=Employee,llm_prompt_for_object_list=LLMPrompt(user_text=USER_TEXTS_FOR_PEOPLE_STR),llm_setting_for_object_list=llm_setting_for_object,@@ -106,6 +111,7 @@ class TestContentGenerator:positive_text = "A dog with sunglasses coding on a laptop"image: ImageContent = await content_generator.make_single_image(job_metadata=job_metadata,+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),img_gen_handle=img_gen_handle,img_gen_prompt=ImgGenPrompt(positive_text=positive_text,@@ -121,6 +127,7 @@ class TestContentGenerator:jinja2_text: str = await content_generator.make_templated_text(job_metadata=job_metadata,+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),context=context,template="The answer is: {{ the_answer }}",)@@ -137,6 +144,7 @@ class TestContentGenerator:):page_contents = await content_generator.make_extract_pages(job_metadata=job_metadata,+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),extract_handle=extract_handle_from_image,extract_input=ExtractInput(image_uri=ImageTestCases.IMAGE_FILE_PATH_JPG_1),extract_job_params=ExtractJobParams.make_default_extract_job_params(),@@ -157,6 +165,7 @@ class TestContentGenerator:):page_contents = await content_generator.make_extract_pages(job_metadata=job_metadata,+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),extract_handle=extract_handle_from_pdf,extract_input=ExtractInput(document_uri=DocumentTestCases.PDF_FILE_PATH_1),extract_job_params=extract_job_params,@@ -173,6 +182,7 @@ class TestContentGenerator:with pytest.raises(ModelNotFoundError) as excinfo:await content_generator.make_llm_text(job_metadata=job_metadata,+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),llm_prompt_for_text=LLMPrompt(user_text=USER_TEXT_FOR_BASE),llm_setting_main=llm_setting_main,)diff --git a/tests/integration/pipelex/cogt/test_llm_generate_security.py b/tests/integration/pipelex/cogt/test_llm_generate_security.pyindex 91562733b..3b0277853 100644--- a/tests/integration/pipelex/cogt/test_llm_generate_security.py+++ b/tests/integration/pipelex/cogt/test_llm_generate_security.py@@ -10,16 +10,19 @@ import pytestfrom pytest_mock import MockerFixturefrom pipelex.cogt.content_generation.assignment_models import LLMAssignment, ObjectAssignment+from pipelex.cogt.content_generation.cogt_run_params import CogtRunParamsfrom pipelex.cogt.content_generation.exceptions import UnsafeSchemaErrorfrom pipelex.cogt.content_generation.llm_generate import llm_gen_object, llm_gen_object_listfrom pipelex.cogt.llm.llm_prompt import LLMPromptfrom pipelex.cogt.llm.llm_setting import LLMSetting+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipeline.job_metadata import JobMetadatadef _make_stub_llm_assignment() -> LLMAssignment:return LLMAssignment(job_metadata=JobMetadata(user_id="test-user", pipeline_run_id="test-run"),+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),llm_setting=LLMSetting(model="test-model", temperature=0.7),llm_prompt=LLMPrompt(user_text="test prompt"),)diff --git a/tests/integration/pipelex/pipe_signature/test_pipe_signature.py b/tests/integration/pipelex/pipe_signature/test_pipe_signature.pyindex 212bcbab2..3cdba6ee4 100644--- a/tests/integration/pipelex/pipe_signature/test_pipe_signature.py+++ b/tests/integration/pipelex/pipe_signature/test_pipe_signature.py@@ -7,7 +7,7 @@ from pipelex.core.pipes.pipe_blueprint import PipeCategory, PipeTypefrom pipelex.core.pipes.pipe_factory import PipeFactoryfrom pipelex.core.stuffs.list_content import ListContentfrom pipelex.hub import get_current_library-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipe_signature.exceptions import PipeSignatureNotExecutableErrorfrom pipelex.pipe_signature.pipe_signature import PipeSignaturediff --git a/tests/integration/pipelex/pipeline/test_dry_run_pipeline_graph.py b/tests/integration/pipelex/pipeline/test_dry_run_pipeline_graph.pynew file mode 100644index 000000000..54f49b89a--- /dev/null+++ b/tests/integration/pipelex/pipeline/test_dry_run_pipeline_graph.py@@ -0,0 +1,65 @@+"""Regression tests: ``dry_run_pipeline`` owns its graph transport.++``dry_run_pipeline`` explicitly requests a graph (``generate_graph=True``), so producing it must+not depend on the host's ``tracing_config`` — the validation host that surfaced this (pipelex-api+in direct mode) ships ``tracing_config.is_enabled = false``, which used to leave the tracer with+no transport and the run with ``graph_spec=None``. The function now installs a scoped in-memory+event log around the run, so:++- with tracing **disabled**, the graph still comes back (the regression);+- with tracing **enabled**, the scoped override takes priority over the configured backend, so a+ validation dry-run writes no trace files (``make_event_log`` is never called).+"""++import pytest+from pytest_mock import MockerFixture++from pipelex.config import get_config+from pipelex.pipe_run.dry_run_pipeline import dry_run_pipeline+from pipelex.system.configuration.configs import NdjsonTracingConfig, TracingBackend++_DRY_RUN_GRAPH_DOMAIN = "dry_run_pipeline_graph"+_DRY_RUN_GRAPH_MTHDS = f"""+domain = "{_DRY_RUN_GRAPH_DOMAIN}"+description = "Minimal bundle for dry_run_pipeline graph-transport tests"+main_pipe = "echo_subject"++[pipe.echo_subject]+type = "PipeLLM"+description = "Pipe used to exercise dry_run_pipeline graph generation"+output = "Text"+prompt = "Echo something"+"""++[email protected](loop_scope="class")+class TestDryRunPipelineGraphTransport:+ def _forbid_event_log_factory(self, mocker: MockerFixture) -> None:+ factory_error = AssertionError("make_event_log must not be called: dry_run_pipeline traces through its scoped in-memory log")+ mocker.patch("pipelex.pipeline.pipeline_run_setup.make_event_log", side_effect=factory_error)+ mocker.patch("pipelex.pipe_run.tracing_assembly.make_event_log", side_effect=factory_error)++ async def test_graph_produced_with_tracing_disabled(self, mocker: MockerFixture) -> None:+ """The pipelex-api regression: tracing off must not mean graph off."""+ mocker.patch.object(get_config().pipelex.tracing_config, "is_enabled", False)+ self._forbid_event_log_factory(mocker)++ graph_spec, pipe_code = await dry_run_pipeline(mthds_contents=[_DRY_RUN_GRAPH_MTHDS])++ assert pipe_code == f"{_DRY_RUN_GRAPH_DOMAIN}.echo_subject"+ traced_pipe_codes = {node.pipe_code for node in graph_spec.nodes if node.pipe_code}+ assert "echo_subject" in traced_pipe_codes++ async def test_scoped_log_takes_priority_over_enabled_backend(self, tmp_path_factory: pytest.TempPathFactory, mocker: MockerFixture) -> None:+ """With tracing enabled, the scoped in-memory log still owns the run — no trace files written."""+ cfg = get_config().pipelex.tracing_config+ traces_dir = tmp_path_factory.mktemp("dry_run_pipeline_traces")+ mocker.patch.object(cfg, "is_enabled", True)+ mocker.patch.object(cfg, "backend", TracingBackend.NDJSON)+ mocker.patch.object(cfg, "ndjson", NdjsonTracingConfig(traces_dir=str(traces_dir)))+ self._forbid_event_log_factory(mocker)++ graph_spec, _ = await dry_run_pipeline(mthds_contents=[_DRY_RUN_GRAPH_MTHDS])++ assert graph_spec.nodes+ assert not list(traces_dir.iterdir()), "the configured ndjson backend must not be touched"diff --git a/tests/integration/pipelex/pipeline/test_mock_inference_direct.py b/tests/integration/pipelex/pipeline/test_mock_usage_direct.pysimilarity index 69%rename from tests/integration/pipelex/pipeline/test_mock_inference_direct.pyrename to tests/integration/pipelex/pipeline/test_mock_usage_direct.pyindex 40897c954..50e2477fc 100644--- a/tests/integration/pipelex/pipeline/test_mock_inference_direct.py+++ b/tests/integration/pipelex/pipeline/test_mock_usage_direct.py@@ -1,16 +1,15 @@-"""Integration tests for ``--mock-inference`` in DIRECT mode (Phase 5).+"""Integration tests for the ``is_mock_usage`` dry sub-flag in DIRECT mode.-A ``--mock-inference`` run stays LIVE (real control flow, real templating) but fakes every AI call-at the cogt leaf, emitting *reportable* (non-zero) synthetic usage. These tests prove the end-to-end-contract without any provider spend:+A dry run with ``is_mock_usage=True`` keeps the dry contract (no provider, no spend, leaf mocks) but+emits *reportable* (non-zero) synthetic usage. These tests prove the end-to-end contract:- The leaf never reaches the LLM worker (``get_llm_worker`` is not called).-- Usage rides back on ``PipeOutput.tokens_usages`` under the ``mock_inference`` sentinel model, with+- Usage rides back on ``PipeOutput.tokens_usages`` under the ``mock_usage`` sentinel model, withnon-zero tokens — so ``AggregatedCosts.has_reportable_usage`` is True and the cost report renders- (unlike ``--dry-run``, whose zero-token usage Phase 3 deliberately suppresses).+ (unlike the default dry run, whose zero-token usage is deliberately suppressed).-Both the text path (``mock_llm_gen_text``) and the structured-object path (``mock_llm_gen_object``)-are covered. NDJSON tracing is enabled so the usage events assemble onto ``PipeOutput``.+Both the text path and the structured-object path are covered. NDJSON tracing is enabled so the+usage events assemble onto ``PipeOutput``."""import io@@ -19,18 +18,19 @@ import pytestfrom pytest_mock import MockerFixturefrom rich.console import Console-from pipelex.cogt.content_generation.dry_mock import MOCK_INFERENCE_MODEL_NAME+from pipelex.cogt.content_generation.dry_mock import MOCK_USAGE_MODEL_NAMEfrom pipelex.cogt.usage.cost_registry import CostRegistryfrom pipelex.config import get_configfrom pipelex.core.pipes.pipe_output import PipeOutput+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipeline.runner import PipelexRunnerfrom pipelex.reporting.cost_report_renderer import render_run_cost_reportfrom pipelex.system.configuration.configs import NdjsonTracingConfig, PipelineExecutionConfig, TracingBackend-_DOMAIN = "mock_inference_direct"+_DOMAIN = "mock_usage_direct"_MTHDS = f"""domain = "{_DOMAIN}"-description = "Minimal bundle for --mock-inference DIRECT tests"+description = "Minimal bundle for is_mock_usage DIRECT tests"[concept.Topic]description = "A topic"@@ -59,7 +59,7 @@ def _recording_console() -> Console:@pytest.mark.asyncio(loop_scope="class")-class TestMockInferenceDirect:+class TestMockUsageDirect:def _enable_ndjson_tracing(self, mocker: MockerFixture, traces_dir: str) -> None:cfg = get_config().pipelex.tracing_configmocker.patch.object(cfg, "is_enabled", True)@@ -67,30 +67,30 @@ class TestMockInferenceDirect:mocker.patch.object(cfg, "ndjson", NdjsonTracingConfig(traces_dir=traces_dir))def _config(self) -> PipelineExecutionConfig:- # Costs on (default) so usage assembles; mock_inputs fills the `subject` input for the LIVE run.+ # Costs on (default) so usage assembles; mock_inputs fills the `subject` input for the dry run.return get_config().pipelex.pipeline_execution_config.with_execution_overrides(generate_graph=False,generate_usage=True,mock_inputs=True,)- async def _run_mock_inference(self, pipe_code: str) -> PipeOutput:- # pipe_run_mode left as None -> LIVE; is_mock_inference fakes only the AI leaf.- runner = PipelexRunner(is_mock_inference=True, execution_config=self._config())+ async def _run_mock_usage(self, pipe_code: str) -> PipeOutput:+ # A DRY run; is_mock_usage switches the leaf reporting to non-zero synthetic usage.+ runner = PipelexRunner(pipe_run_mode=PipeRunMode.DRY, is_mock_usage=True, execution_config=self._config())response = await runner.execute_pipeline(pipe_code=pipe_code, mthds_contents=[_MTHDS])return response.pipe_outputasync def test_text_pipe_mocks_leaf_and_assembles_reportable_usage(self, tmp_path_factory: pytest.TempPathFactory, mocker: MockerFixture) -> None:- """A LIVE text pipe under --mock-inference never calls the worker, yet assembles non-zero usage."""+ """A dry text pipe under is_mock_usage never calls the worker, yet assembles non-zero usage."""self._enable_ndjson_tracing(mocker, str(tmp_path_factory.mktemp("traces_mock_text")))worker_spy = mocker.patch("pipelex.cogt.content_generation.llm_generate.get_llm_worker")- pipe_output = await self._run_mock_inference("write_text")+ pipe_output = await self._run_mock_usage("write_text")worker_spy.assert_not_called() # no provider call -> no spendassert pipe_output.tokens_usages is not Noneassert len(pipe_output.tokens_usages) >= 1- assert all(usage.inference_model_name == MOCK_INFERENCE_MODEL_NAME for usage in pipe_output.tokens_usages)+ assert all(usage.inference_model_name == MOCK_USAGE_MODEL_NAME for usage in pipe_output.tokens_usages)aggregated = CostRegistry.aggregate_costs(pipe_output.tokens_usages)assert aggregated.has_reportable_usage # non-zero tokens -> the report is NOT suppressed@@ -99,11 +99,11 @@ class TestMockInferenceDirect:async def test_object_pipe_mocks_leaf_and_assembles_reportable_usage(self, tmp_path_factory: pytest.TempPathFactory, mocker: MockerFixture) -> None:- """The structured-object path (mock_llm_gen_object) also fakes the leaf and reports non-zero usage."""+ """The structured-object path also fakes the leaf and reports non-zero usage."""self._enable_ndjson_tracing(mocker, str(tmp_path_factory.mktemp("traces_mock_object")))worker_spy = mocker.patch("pipelex.cogt.content_generation.llm_generate.get_llm_worker")- pipe_output = await self._run_mock_inference("write_topic")+ pipe_output = await self._run_mock_usage("write_topic")worker_spy.assert_not_called()assert pipe_output.tokens_usages is not None@@ -114,8 +114,8 @@ class TestMockInferenceDirect:main_stuff = pipe_output.working_memory.get_optional_main_stuff()assert main_stuff is not None- async def test_cost_report_renders_for_mock_inference(self, tmp_path_factory: pytest.TempPathFactory, mocker: MockerFixture) -> None:- """The end-of-run cost report RENDERS for a --mock-inference run (non-suppressed), under the sentinel model."""+ async def test_cost_report_renders_for_mock_usage(self, tmp_path_factory: pytest.TempPathFactory, mocker: MockerFixture) -> None:+ """The end-of-run cost report RENDERS for an is_mock_usage run (non-suppressed), under the sentinel model."""self._enable_ndjson_tracing(mocker, str(tmp_path_factory.mktemp("traces_mock_render")))mocker.patch("pipelex.cogt.content_generation.llm_generate.get_llm_worker")reporting_config = get_config().pipelex.reporting_config@@ -124,7 +124,7 @@ class TestMockInferenceDirect:console = _recording_console()mocker.patch("pipelex.cogt.usage.cost_registry.get_console", return_value=console)- pipe_output = await self._run_mock_inference("write_text")+ pipe_output = await self._run_mock_usage("write_text")render_run_cost_report(pipeline_run_id=pipe_output.pipeline_run_id,@@ -132,5 +132,5 @@ class TestMockInferenceDirect:is_generate_costs=True,)rendered = console.export_text()- assert MOCK_INFERENCE_MODEL_NAME in rendered # the table is shown (NOT suppressed like a dry run)+ assert MOCK_USAGE_MODEL_NAME in rendered # the table is shown (NOT suppressed like a default dry run)assert "Total" in rendereddiff --git a/tests/integration/pipelex/pipeline/test_scoped_in_memory_tracing.py b/tests/integration/pipelex/pipeline/test_scoped_in_memory_tracing.pynew file mode 100644index 000000000..0047dd1db--- /dev/null+++ b/tests/integration/pipelex/pipeline/test_scoped_in_memory_tracing.py@@ -0,0 +1,134 @@+"""Integration tests for in-memory graph tracing via `scoped_event_log` (direct mode).++A dry-run-with-graph executed under ``with scoped_event_log(InMemoryEventLog())`` must:++- produce a non-empty, correct ``GraphSpec`` on ``pipe_output``;+- write NO NDJSON file and touch NO configured backend (the factory is never called —+ the scoped instance is the single transport);+- emit and assemble against the SAME instance (the fix for the two-instance problem);+- assemble the graph even when ``tracing_config.is_enabled`` is False — a set override+ implies tracing-enabled (decision D1);+- keep concurrently-scoped runs isolated from each other.+"""++import asyncio+from pathlib import Path++import pytest+from pytest_mock import MockerFixture++from pipelex.config import get_config+from pipelex.core.pipes.pipe_output import PipeOutput+from pipelex.hub import scoped_event_log+from pipelex.pipe_run.pipe_run_mode import PipeRunMode+from pipelex.pipeline.runner import PipelexRunner+from pipelex.system.configuration.configs import NdjsonTracingConfig, TracingBackend+from pipelex.tracing.in_memory_event_log import InMemoryEventLog++_SCOPED_DOMAIN = "scoped_in_memory_tracing"+_SCOPED_MTHDS = f"""+domain = "{_SCOPED_DOMAIN}"+description = "Minimal bundle for scoped in-memory tracing tests"++[concept.Topic]+description = "A topic"++[concept.Topic.structure]+name = {{ type = "text", description = "Topic name" }}++[pipe.echo_topic]+type = "PipeLLM"+description = "Pipe used to exercise scoped in-memory tracing"+inputs = {{ subject = "Text" }}+output = "Topic"+prompt = "Echo the $subject as a topic"+"""++[email protected](loop_scope="class")+class TestScopedInMemoryTracing:+ def _forbid_event_log_factory(self, mocker: MockerFixture) -> None:+ """Make any call to make_event_log fail loudly — the scoped instance must be the only transport."""+ factory_error = AssertionError("make_event_log must not be called when a scoped event log is set")+ mocker.patch("pipelex.pipeline.pipeline_run_setup.make_event_log", side_effect=factory_error)+ mocker.patch("pipelex.pipe_run.tracing_assembly.make_event_log", side_effect=factory_error)++ async def _run_dry_with_graph(self) -> PipeOutput:+ execution_config = get_config().pipelex.pipeline_execution_config.with_execution_overrides(+ generate_graph=True,+ mock_inputs=True,+ )+ runner = PipelexRunner(pipe_run_mode=PipeRunMode.DRY, execution_config=execution_config)+ response = await runner.execute_pipeline(pipe_code="echo_topic", mthds_contents=[_SCOPED_MTHDS])+ return response.pipe_output++ async def test_in_memory_graph_same_instance_no_backend(self, tmp_path: Path, mocker: MockerFixture) -> None:+ """With NDJSON tracing configured, the scoped log wins: graph assembles in memory, zero file I/O."""+ traces_dir = tmp_path / "traces"+ traces_dir.mkdir()+ cfg = get_config().pipelex.tracing_config+ mocker.patch.object(cfg, "is_enabled", True)+ mocker.patch.object(cfg, "backend", TracingBackend.NDJSON)+ mocker.patch.object(cfg, "ndjson", NdjsonTracingConfig(traces_dir=str(traces_dir)))+ self._forbid_event_log_factory(mocker)++ event_log = InMemoryEventLog()+ emit_spy = mocker.spy(event_log, "emit")+ read_spy = mocker.spy(event_log, "read_events")++ with scoped_event_log(event_log):+ pipe_output = await self._run_dry_with_graph()++ # (a) non-empty, correct GraphSpec+ assert pipe_output.graph_spec is not None+ assert len(pipe_output.graph_spec.nodes) >= 1+ assert pipe_output.graph_spec.graph_id == pipe_output.pipeline_run_id+ assert pipe_output.graph_assembly_error is None++ # (b) no NDJSON file, no configured backend touched+ assert list(traces_dir.iterdir()) == []++ # (c) emit and assemble hit the SAME instance+ assert emit_spy.call_count >= 1+ assert read_spy.call_count >= 1+ assert event_log.read_events(pipe_output.pipeline_run_id)++ async def test_override_implies_enabled_when_tracing_disabled(self, mocker: MockerFixture) -> None:+ """D1 regression: is_enabled=False + scoped override → the GraphSpec still assembles in memory."""+ cfg = get_config().pipelex.tracing_config+ mocker.patch.object(cfg, "is_enabled", False)+ self._forbid_event_log_factory(mocker)++ event_log = InMemoryEventLog()+ with scoped_event_log(event_log):+ pipe_output = await self._run_dry_with_graph()++ assert pipe_output.graph_spec is not None+ assert len(pipe_output.graph_spec.nodes) >= 1+ assert event_log.read_events(pipe_output.pipeline_run_id)++ async def test_concurrent_scoped_runs_do_not_cross_contaminate(self, mocker: MockerFixture) -> None:+ """Two concurrently-scoped runs each trace into their own log, with no shared/merged events."""+ cfg = get_config().pipelex.tracing_config+ mocker.patch.object(cfg, "is_enabled", False)+ self._forbid_event_log_factory(mocker)++ log_alpha = InMemoryEventLog(writer_id="alpha")+ log_beta = InMemoryEventLog(writer_id="beta")++ async def scoped_run(event_log: InMemoryEventLog) -> PipeOutput:+ with scoped_event_log(event_log):+ return await self._run_dry_with_graph()++ output_alpha, output_beta = await asyncio.gather(scoped_run(log_alpha), scoped_run(log_beta))++ assert output_alpha.graph_spec is not None+ assert output_beta.graph_spec is not None+ assert output_alpha.pipeline_run_id != output_beta.pipeline_run_id+ assert output_alpha.graph_spec.graph_id != output_beta.graph_spec.graph_id++ # Each log holds only its own run's events.+ assert log_alpha.read_events(output_alpha.pipeline_run_id)+ assert log_alpha.read_events(output_beta.pipeline_run_id) == []+ assert log_beta.read_events(output_beta.pipeline_run_id)+ assert log_beta.read_events(output_alpha.pipeline_run_id) == []diff --git a/tests/integration/pipelex/pipes/controller/pipe_batch/test_pipe_batch_simple.py b/tests/integration/pipelex/pipes/controller/pipe_batch/test_pipe_batch_simple.pyindex ee089f9c7..b631d073a 100644--- a/tests/integration/pipelex/pipes/controller/pipe_batch/test_pipe_batch_simple.py+++ b/tests/integration/pipelex/pipes/controller/pipe_batch/test_pipe_batch_simple.py@@ -16,7 +16,7 @@ from pipelex.core.stuffs.text_content import TextContentfrom pipelex.hub import get_concept_libraryfrom pipelex.pipe_controllers.batch.pipe_batch import PipeBatchfrom pipelex.pipe_controllers.batch.pipe_batch_blueprint import PipeBatchBlueprint-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatadiff --git a/tests/integration/pipelex/pipes/controller/pipe_batch_dotted_path/test_dotted_batch.py b/tests/integration/pipelex/pipes/controller/pipe_batch_dotted_path/test_dotted_batch.pyindex 3d7af9079..46fba4200 100644--- a/tests/integration/pipelex/pipes/controller/pipe_batch_dotted_path/test_dotted_batch.py+++ b/tests/integration/pipelex/pipes/controller/pipe_batch_dotted_path/test_dotted_batch.py@@ -15,7 +15,8 @@ from pipelex.core.stuffs.stuff_factory import StuffFactoryfrom pipelex.core.stuffs.text_content import TextContentfrom pipelex.hub import get_required_pipefrom pipelex.pipe_controllers.sub_pipe import SubPipe-from pipelex.pipe_run.pipe_run_params import BatchParams, PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunMode+from pipelex.pipe_run.pipe_run_params import BatchParamsfrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatadiff --git a/tests/integration/pipelex/pipes/controller/pipe_condition/test_pipe_condition_complex.py b/tests/integration/pipelex/pipes/controller/pipe_condition/test_pipe_condition_complex.pyindex bda8f5081..894504013 100644--- a/tests/integration/pipelex/pipes/controller/pipe_condition/test_pipe_condition_complex.py+++ b/tests/integration/pipelex/pipes/controller/pipe_condition/test_pipe_condition_complex.py@@ -15,7 +15,7 @@ from pipelex.core.stuffs.text_content import TextContentfrom pipelex.hub import get_pipe_router, get_required_pipefrom pipelex.pipe_run.exceptions import PipeRouterErrorfrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatafrom tests.integration.pipelex.pipes.controller.pipe_condition.pipe_condition_complex import (diff --git a/tests/integration/pipelex/pipes/controller/pipe_condition/test_pipe_condition_continue_output_type.py b/tests/integration/pipelex/pipes/controller/pipe_condition/test_pipe_condition_continue_output_type.pyindex 69c6dcfd5..eb0b1e835 100644--- a/tests/integration/pipelex/pipes/controller/pipe_condition/test_pipe_condition_continue_output_type.py+++ b/tests/integration/pipelex/pipes/controller/pipe_condition/test_pipe_condition_continue_output_type.py@@ -17,7 +17,7 @@ from pipelex.core.stuffs.list_content import ListContentfrom pipelex.core.stuffs.stuff_factory import StuffFactoryfrom pipelex.hub import get_pipe_router, get_required_pipefrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatafrom tests.integration.pipelex.pipes.controller.pipe_condition.pipe_condition_continue_output_type import (diff --git a/tests/integration/pipelex/pipes/controller/pipe_condition/test_pipe_condition_simple.py b/tests/integration/pipelex/pipes/controller/pipe_condition/test_pipe_condition_simple.pyindex 5b50cf027..a72d1cbd4 100644--- a/tests/integration/pipelex/pipes/controller/pipe_condition/test_pipe_condition_simple.py+++ b/tests/integration/pipelex/pipes/controller/pipe_condition/test_pipe_condition_simple.py@@ -18,7 +18,7 @@ from pipelex.pipe_controllers.condition.pipe_condition_blueprint import PipeCondfrom pipelex.pipe_controllers.condition.special_outcome import SpecialOutcomefrom pipelex.pipe_run.exceptions import PipeRouterErrorfrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatafrom tests.integration.pipelex.pipes.controller.pipe_condition.pipe_condition import CategoryInputdiff --git a/tests/integration/pipelex/pipes/controller/pipe_parallel/test_pipe_parallel_simple.py b/tests/integration/pipelex/pipes/controller/pipe_parallel/test_pipe_parallel_simple.pyindex 2b9a9bbdf..f06da6405 100644--- a/tests/integration/pipelex/pipes/controller/pipe_parallel/test_pipe_parallel_simple.py+++ b/tests/integration/pipelex/pipes/controller/pipe_parallel/test_pipe_parallel_simple.py@@ -13,7 +13,7 @@ from pipelex.core.stuffs.text_content import TextContentfrom pipelex.pipe_controllers.parallel.pipe_parallel import PipeParallelfrom pipelex.pipe_controllers.parallel.pipe_parallel_blueprint import PipeParallelBlueprintfrom pipelex.pipe_controllers.sub_pipe_factory import SubPipeBlueprint-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatadiff --git a/tests/integration/pipelex/pipes/controller/pipe_sequence/test_pipe_sequence_dry_run.py b/tests/integration/pipelex/pipes/controller/pipe_sequence/test_pipe_sequence_dry_run.pyindex a020ea81f..8470b9aed 100644--- a/tests/integration/pipelex/pipes/controller/pipe_sequence/test_pipe_sequence_dry_run.py+++ b/tests/integration/pipelex/pipes/controller/pipe_sequence/test_pipe_sequence_dry_run.py@@ -11,7 +11,7 @@ from pipelex.core.stuffs.stuff import Stufffrom pipelex.core.stuffs.stuff_factory import StuffFactoryfrom pipelex.hub import get_pipe_router, get_required_pipefrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatafrom pipelex.tools.misc.json_utils import load_json_list_from_pathdiff --git a/tests/integration/pipelex/pipes/controller/pipe_sequence/test_pipe_sequence_simple.py b/tests/integration/pipelex/pipes/controller/pipe_sequence/test_pipe_sequence_simple.pyindex 0f59065bb..1d8ed299a 100644--- a/tests/integration/pipelex/pipes/controller/pipe_sequence/test_pipe_sequence_simple.py+++ b/tests/integration/pipelex/pipes/controller/pipe_sequence/test_pipe_sequence_simple.py@@ -15,7 +15,7 @@ from pipelex.hub import get_concept_library, get_native_conceptfrom pipelex.pipe_controllers.sequence.pipe_sequence import PipeSequencefrom pipelex.pipe_controllers.sequence.pipe_sequence_blueprint import PipeSequenceBlueprintfrom pipelex.pipe_controllers.sub_pipe_blueprint import SubPipeBlueprint-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatadiff --git a/tests/integration/pipelex/pipes/llm_prompt_inputs/test_image_inputs_inference.py b/tests/integration/pipelex/pipes/llm_prompt_inputs/test_image_inputs_inference.pyindex b633b1b48..d9761685c 100644--- a/tests/integration/pipelex/pipes/llm_prompt_inputs/test_image_inputs_inference.py+++ b/tests/integration/pipelex/pipes/llm_prompt_inputs/test_image_inputs_inference.py@@ -16,7 +16,7 @@ from pipelex.core.stuffs.text_and_images_content import TextAndImagesContentfrom pipelex.core.stuffs.text_content import TextContentfrom pipelex.hub import get_native_concept, get_pipe_router, get_required_pipefrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatafrom tests.cases import ImageTestCasesdiff --git a/tests/integration/pipelex/pipes/operator/pipe_compose_structured/test_pipe_compose_class_compatibility.py b/tests/integration/pipelex/pipes/operator/pipe_compose_structured/test_pipe_compose_class_compatibility.pyindex 056ac37e0..5b656f50b 100644--- a/tests/integration/pipelex/pipes/operator/pipe_compose_structured/test_pipe_compose_class_compatibility.py+++ b/tests/integration/pipelex/pipes/operator/pipe_compose_structured/test_pipe_compose_class_compatibility.py@@ -22,7 +22,7 @@ from pipelex.hub import get_native_concept, get_pipe_routerfrom pipelex.pipe_operators.compose.pipe_compose import PipeComposefrom pipelex.pipe_operators.compose.pipe_compose_blueprint import PipeComposeBlueprintfrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatafrom tests.integration.pipelex.pipes.operator.pipe_compose_structured.models_for_pipe_compose import (diff --git a/tests/integration/pipelex/pipes/operator/pipe_compose_structured/test_pipe_compose_content_conversions.py b/tests/integration/pipelex/pipes/operator/pipe_compose_structured/test_pipe_compose_content_conversions.pyindex beb94423a..9c12c1f9d 100644--- a/tests/integration/pipelex/pipes/operator/pipe_compose_structured/test_pipe_compose_content_conversions.py+++ b/tests/integration/pipelex/pipes/operator/pipe_compose_structured/test_pipe_compose_content_conversions.py@@ -24,7 +24,7 @@ from pipelex.hub import get_native_concept, get_pipe_routerfrom pipelex.pipe_operators.compose.pipe_compose import PipeComposefrom pipelex.pipe_operators.compose.pipe_compose_blueprint import PipeComposeBlueprintfrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatafrom tests.integration.pipelex.pipes.operator.pipe_compose_structured.models_for_pipe_compose import (diff --git a/tests/integration/pipelex/pipes/operator/pipe_compose_structured/test_pipe_compose_structured.py b/tests/integration/pipelex/pipes/operator/pipe_compose_structured/test_pipe_compose_structured.pyindex 2e84c0bce..7237e05aa 100644--- a/tests/integration/pipelex/pipes/operator/pipe_compose_structured/test_pipe_compose_structured.py+++ b/tests/integration/pipelex/pipes/operator/pipe_compose_structured/test_pipe_compose_structured.py@@ -20,7 +20,7 @@ from pipelex.hub import get_native_concept, get_pipe_routerfrom pipelex.pipe_operators.compose.pipe_compose import PipeComposefrom pipelex.pipe_operators.compose.pipe_compose_blueprint import PipeComposeBlueprintfrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatafrom tests.integration.pipelex.pipes.operator.pipe_compose_structured.models_for_pipe_compose import (diff --git a/tests/integration/pipelex/pipes/operator/pipe_compose_structured/test_pipe_compose_structured_compatibility.py b/tests/integration/pipelex/pipes/operator/pipe_compose_structured/test_pipe_compose_structured_compatibility.pyindex 7e5899460..2265c162d 100644--- a/tests/integration/pipelex/pipes/operator/pipe_compose_structured/test_pipe_compose_structured_compatibility.py+++ b/tests/integration/pipelex/pipes/operator/pipe_compose_structured/test_pipe_compose_structured_compatibility.py@@ -23,7 +23,7 @@ from pipelex.pipe_operators.compose.exceptions import PipeComposeErrorfrom pipelex.pipe_operators.compose.pipe_compose import PipeComposefrom pipelex.pipe_operators.compose.pipe_compose_blueprint import PipeComposeBlueprintfrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatafrom tests.integration.pipelex.pipes.operator.pipe_compose_structured.models_for_pipe_compose import (diff --git a/tests/integration/pipelex/pipes/operator/pipe_compose_structured/test_pipe_compose_stuff_content_subclasses.py b/tests/integration/pipelex/pipes/operator/pipe_compose_structured/test_pipe_compose_stuff_content_subclasses.pyindex 93941ac99..b8e941b79 100644--- a/tests/integration/pipelex/pipes/operator/pipe_compose_structured/test_pipe_compose_stuff_content_subclasses.py+++ b/tests/integration/pipelex/pipes/operator/pipe_compose_structured/test_pipe_compose_stuff_content_subclasses.py@@ -32,7 +32,7 @@ from pipelex.hub import get_native_concept, get_pipe_routerfrom pipelex.pipe_operators.compose.pipe_compose import PipeComposefrom pipelex.pipe_operators.compose.pipe_compose_blueprint import PipeComposeBlueprintfrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatafrom pipelex.urls import URLsdiff --git a/tests/integration/pipelex/pipes/operator/pipe_extract/test_pipe_extract.py b/tests/integration/pipelex/pipes/operator/pipe_extract/test_pipe_extract.pyindex 99ad56c3d..91431ab46 100644--- a/tests/integration/pipelex/pipes/operator/pipe_extract/test_pipe_extract.py+++ b/tests/integration/pipelex/pipes/operator/pipe_extract/test_pipe_extract.py@@ -16,7 +16,7 @@ from pipelex.hub import get_concept_library, get_pipe_routerfrom pipelex.pipe_operators.extract.pipe_extract import PipeExtractfrom pipelex.pipe_operators.extract.pipe_extract_blueprint import PipeExtractBlueprintfrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatafrom tests.integration.pipelex.test_data import PipeExtractTestCasesdiff --git a/tests/integration/pipelex/pipes/operator/pipe_extract/test_pipe_extract_model_not_found_reroute.py b/tests/integration/pipelex/pipes/operator/pipe_extract/test_pipe_extract_model_not_found_reroute.pyindex ac4f87584..00a4ca75a 100644--- a/tests/integration/pipelex/pipes/operator/pipe_extract/test_pipe_extract_model_not_found_reroute.py+++ b/tests/integration/pipelex/pipes/operator/pipe_extract/test_pipe_extract_model_not_found_reroute.py@@ -16,7 +16,7 @@ from pipelex.pipe_operators.exceptions import PipeOperatorModelAvailabilityErrorfrom pipelex.pipe_operators.extract.pipe_extract import PipeExtractfrom pipelex.pipe_operators.extract.pipe_extract_blueprint import PipeExtractBlueprintfrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatadiff --git a/tests/integration/pipelex/pipes/operator/pipe_func/test_pipe_func.py b/tests/integration/pipelex/pipes/operator/pipe_func/test_pipe_func.pyindex 8d37805ea..57be3214a 100644--- a/tests/integration/pipelex/pipes/operator/pipe_func/test_pipe_func.py+++ b/tests/integration/pipelex/pipes/operator/pipe_func/test_pipe_func.py@@ -13,7 +13,7 @@ from pipelex.hub import get_pipe_routerfrom pipelex.pipe_operators.func.pipe_func import PipeFuncfrom pipelex.pipe_operators.func.pipe_func_blueprint import PipeFuncBlueprintfrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatafrom pipelex.system.registries.func_registry import func_registrydiff --git a/tests/integration/pipelex/pipes/operator/pipe_img_gen/test_pipe_img_gen_model_not_found_reroute.py b/tests/integration/pipelex/pipes/operator/pipe_img_gen/test_pipe_img_gen_model_not_found_reroute.pyindex 01da8b7b0..d453b59cb 100644--- a/tests/integration/pipelex/pipes/operator/pipe_img_gen/test_pipe_img_gen_model_not_found_reroute.py+++ b/tests/integration/pipelex/pipes/operator/pipe_img_gen/test_pipe_img_gen_model_not_found_reroute.py@@ -12,7 +12,7 @@ from pipelex.pipe_operators.exceptions import PipeOperatorModelAvailabilityErrorfrom pipelex.pipe_operators.img_gen.pipe_img_gen import PipeImgGenfrom pipelex.pipe_operators.img_gen.pipe_img_gen_blueprint import PipeImgGenBlueprintfrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatadiff --git a/tests/integration/pipelex/pipes/operator/pipe_img_gen/test_pipe_img_gen_run.py b/tests/integration/pipelex/pipes/operator/pipe_img_gen/test_pipe_img_gen_run.pyindex 8b6859747..2b5803c79 100644--- a/tests/integration/pipelex/pipes/operator/pipe_img_gen/test_pipe_img_gen_run.py+++ b/tests/integration/pipelex/pipes/operator/pipe_img_gen/test_pipe_img_gen_run.py@@ -10,7 +10,7 @@ from pipelex.hub import get_pipe_routerfrom pipelex.pipe_operators.img_gen.pipe_img_gen import PipeImgGenfrom pipelex.pipe_operators.img_gen.pipe_img_gen_blueprint import PipeImgGenBlueprintfrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatafrom tests.integration.pipelex.fixtures.model_combo import ModelCombodiff --git a/tests/integration/pipelex/pipes/operator/pipe_llm/test_pipe_llm.py b/tests/integration/pipelex/pipes/operator/pipe_llm/test_pipe_llm.pyindex 1664ca9b4..7465f80a6 100644--- a/tests/integration/pipelex/pipes/operator/pipe_llm/test_pipe_llm.py+++ b/tests/integration/pipelex/pipes/operator/pipe_llm/test_pipe_llm.py@@ -14,7 +14,7 @@ from pipelex.hub import get_class_registry, get_native_concept, get_pipe_libraryfrom pipelex.pipe_operators.llm.pipe_llm import PipeLLMfrom pipelex.pipe_operators.llm.pipe_llm_blueprint import PipeLLMBlueprint, StructuringMethodfrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatafrom tests.integration.pipelex.test_data import BasicStructuredDataTestCases, PipeTestCasesdiff --git a/tests/integration/pipelex/pipes/operator/pipe_llm/test_pipe_llm_model_not_found_reroute.py b/tests/integration/pipelex/pipes/operator/pipe_llm/test_pipe_llm_model_not_found_reroute.pyindex e313dd25b..f0b5c840b 100644--- a/tests/integration/pipelex/pipes/operator/pipe_llm/test_pipe_llm_model_not_found_reroute.py+++ b/tests/integration/pipelex/pipes/operator/pipe_llm/test_pipe_llm_model_not_found_reroute.py@@ -13,7 +13,7 @@ from pipelex.pipe_operators.exceptions import PipeOperatorModelAvailabilityErrorfrom pipelex.pipe_operators.llm.pipe_llm import PipeLLMfrom pipelex.pipe_operators.llm.pipe_llm_blueprint import PipeLLMBlueprintfrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatadiff --git a/tests/integration/pipelex/pipes/operator/pipe_search/test_pipe_search_mock_inference_guard.py b/tests/integration/pipelex/pipes/operator/pipe_search/test_pipe_search_mock_inference_guard.pydeleted file mode 100644index 583895e68..000000000--- a/tests/integration/pipelex/pipes/operator/pipe_search/test_pipe_search_mock_inference_guard.py+++ /dev/null@@ -1,63 +0,0 @@-"""Integration test for the ``is_mock_inference`` hard guard for web search.--Web search has no leaf-level mock. It now goes through the same ``cogt/content_generation`` seam as-LLM/img-gen/extract: ``PipeSearch`` calls the content generator, whose direct impl runs the-framework-agnostic ``search_generate`` leaf. Under ``--mock-inference`` (``run_mode=LIVE`` +-``JobMetadata.is_mock_inference``) the live path runs, so the guard lives at the top of that leaf-(``search_gen_sourced_answer``) and must fail loud before the search worker is built.-``MockInferenceUnsupportedError`` is a plain ``PipelexError`` (not a model-availability error and not a-``CogtError``), so neither ``PipeOperator._live_run_pipe`` nor the router re-wraps it — it propagates-unchanged.-"""--from typing import Callable--import pytest-from pytest_mock import MockerFixture--from pipelex.cogt.content_generation.exceptions import MockInferenceUnsupportedError-from pipelex.core.concepts.native.concept_native import NativeConceptCode-from pipelex.core.pipes.pipe_factory import PipeFactory-from pipelex.hub import get_pipe_library, get_pipe_router-from pipelex.pipe_operators.search.pipe_search import PipeSearch-from pipelex.pipe_operators.search.pipe_search_blueprint import PipeSearchBlueprint-from pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode-from pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactory-from pipelex.pipeline.job_metadata import JobMetadata--[email protected](loop_scope="class")-class TestPipeSearchMockInferenceGuard:- async def test_mock_inference_raises_before_building_search_worker(- self,- job_metadata: JobMetadata,- load_empty_library: Callable[[], None],- mocker: MockerFixture,- ) -> None:- """A LIVE PipeSearch run with is_mock_inference=True raises the guard before any provider call."""- load_empty_library()-- worker_factory_spy = mocker.patch("pipelex.cogt.content_generation.search_generate.SearchWorkerFactory.make_search_worker")-- pipe = PipeFactory[PipeSearch].make_from_blueprint(- domain_code="generic",- pipe_code="adhoc_for_test_search_mock_inference_guard",- blueprint=PipeSearchBlueprint(- description="Search mock-inference guard test",- output=NativeConceptCode.SEARCH_RESULT,- prompt="Search for recent technology news.",- ),- )- get_pipe_library().add_new_pipe(pipe)-- pipe_job = PipeJobFactory.make_pipe_job(- pipe=pipe,- pipe_run_params=PipeRunParamsFactory.make_run_params(pipe_run_mode=PipeRunMode.LIVE),- job_metadata=job_metadata.model_copy(update={"is_mock_inference": True}),- )-- with pytest.raises(MockInferenceUnsupportedError):- await get_pipe_router().run(pipe_job=pipe_job)-- worker_factory_spy.assert_not_called() # no provider call -> no spenddiff --git a/tests/integration/pipelex/pipes/operator/pipe_search/test_pipe_search_model_not_found_reroute.py b/tests/integration/pipelex/pipes/operator/pipe_search/test_pipe_search_model_not_found_reroute.pyindex 46d2db530..b303d53d8 100644--- a/tests/integration/pipelex/pipes/operator/pipe_search/test_pipe_search_model_not_found_reroute.py+++ b/tests/integration/pipelex/pipes/operator/pipe_search/test_pipe_search_model_not_found_reroute.py@@ -12,7 +12,7 @@ from pipelex.pipe_operators.exceptions import PipeOperatorModelAvailabilityErrorfrom pipelex.pipe_operators.search.pipe_search import PipeSearchfrom pipelex.pipe_operators.search.pipe_search_blueprint import PipeSearchBlueprintfrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatadiff --git a/tests/integration/pipelex/pipes/operator/pipe_structure/test_pipe_structure.py b/tests/integration/pipelex/pipes/operator/pipe_structure/test_pipe_structure.pyindex 95e673bdf..94a65e205 100644--- a/tests/integration/pipelex/pipes/operator/pipe_structure/test_pipe_structure.py+++ b/tests/integration/pipelex/pipes/operator/pipe_structure/test_pipe_structure.py@@ -13,7 +13,7 @@ from pipelex.hub import get_native_concept, get_pipe_library, get_pipe_routerfrom pipelex.pipe_operators.structure.pipe_structure import PipeStructurefrom pipelex.pipe_operators.structure.pipe_structure_blueprint import PipeStructureBlueprintfrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatadiff --git a/tests/integration/pipelex/pipes/operator/pipe_structure/test_pipe_structure_in_batch.py b/tests/integration/pipelex/pipes/operator/pipe_structure/test_pipe_structure_in_batch.pyindex 57eb6d752..9b3e9a2a1 100644--- a/tests/integration/pipelex/pipes/operator/pipe_structure/test_pipe_structure_in_batch.py+++ b/tests/integration/pipelex/pipes/operator/pipe_structure/test_pipe_structure_in_batch.py@@ -17,7 +17,7 @@ from pipelex.pipe_controllers.batch.pipe_batch_blueprint import PipeBatchBlueprifrom pipelex.pipe_operators.structure.pipe_structure import PipeStructurefrom pipelex.pipe_operators.structure.pipe_structure_blueprint import PipeStructureBlueprintfrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatafrom tests.integration.pipelex.pipes.operator.pipe_structure.test_structures_basic import RestaurantReviewdiff --git a/tests/integration/pipelex/pipes/operator/pipe_structure/test_pipe_structure_in_sequence.py b/tests/integration/pipelex/pipes/operator/pipe_structure/test_pipe_structure_in_sequence.pyindex 06a4cabaf..272361e36 100644--- a/tests/integration/pipelex/pipes/operator/pipe_structure/test_pipe_structure_in_sequence.py+++ b/tests/integration/pipelex/pipes/operator/pipe_structure/test_pipe_structure_in_sequence.py@@ -16,7 +16,7 @@ from pipelex.pipe_controllers.sub_pipe_blueprint import SubPipeBlueprintfrom pipelex.pipe_operators.structure.pipe_structure import PipeStructurefrom pipelex.pipe_operators.structure.pipe_structure_blueprint import PipeStructureBlueprintfrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatafrom tests.integration.pipelex.pipes.operator.pipe_structure.test_structures_basic import RestaurantReviewdiff --git a/tests/integration/pipelex/pipes/operator/pipe_structure/test_preliminary_text_e2e.py b/tests/integration/pipelex/pipes/operator/pipe_structure/test_preliminary_text_e2e.pyindex 06d9b9af6..39c1cb954 100644--- a/tests/integration/pipelex/pipes/operator/pipe_structure/test_preliminary_text_e2e.py+++ b/tests/integration/pipelex/pipes/operator/pipe_structure/test_preliminary_text_e2e.py@@ -14,7 +14,7 @@ from pipelex.hub import get_native_concept, get_pipe_router, get_report_delegatefrom pipelex.pipe_controllers.sequence.pipe_sequence import PipeSequencefrom pipelex.pipe_operators.structure.pipe_structure import PipeStructurefrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatafrom tests.integration.pipelex.pipes.operator.pipe_structure.test_structures_basic import RestaurantReviewdiff --git a/tests/integration/pipelex/pipes/operator/pipe_structure/test_preliminary_text_inline_e2e.py b/tests/integration/pipelex/pipes/operator/pipe_structure/test_preliminary_text_inline_e2e.pyindex 05215d10b..57f7ce754 100644--- a/tests/integration/pipelex/pipes/operator/pipe_structure/test_preliminary_text_inline_e2e.py+++ b/tests/integration/pipelex/pipes/operator/pipe_structure/test_preliminary_text_inline_e2e.py@@ -15,7 +15,7 @@ from pipelex.hub import get_native_concept, get_pipe_router, get_report_delegatefrom pipelex.pipe_controllers.sequence.pipe_sequence import PipeSequencefrom pipelex.pipe_operators.structure.pipe_structure import PipeStructurefrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatadiff --git a/tests/integration/pipelex/pipes/test_image_out_in.py b/tests/integration/pipelex/pipes/test_image_out_in.pyindex 28b36fe47..756cf77e7 100644--- a/tests/integration/pipelex/pipes/test_image_out_in.py+++ b/tests/integration/pipelex/pipes/test_image_out_in.py@@ -5,7 +5,7 @@ import pytestfrom pipelex.hub import get_pipe_router, get_required_pipefrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatadiff --git a/tests/integration/pipelex/pipes/test_pipe_compose.py b/tests/integration/pipelex/pipes/test_pipe_compose.pyindex 28dd3a1fe..a8ebcf6a3 100644--- a/tests/integration/pipelex/pipes/test_pipe_compose.py+++ b/tests/integration/pipelex/pipes/test_pipe_compose.py@@ -16,7 +16,7 @@ from pipelex.hub import get_native_concept, get_pipe_routerfrom pipelex.pipe_operators.compose.pipe_compose import PipeCompose, PipeComposeOutputfrom pipelex.pipe_operators.compose.pipe_compose_blueprint import PipeComposeBlueprintfrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatafrom tests.cases import JINJA2TestCasesdiff --git a/tests/integration/pipelex/pipes/test_pipe_running_variants.py b/tests/integration/pipelex/pipes/test_pipe_running_variants.pyindex da833bf08..511fe0870 100644--- a/tests/integration/pipelex/pipes/test_pipe_running_variants.py+++ b/tests/integration/pipelex/pipes/test_pipe_running_variants.py@@ -9,7 +9,7 @@ from pipelex.core.pipes.variable_multiplicity import VariableMultiplicityfrom pipelex.core.stuffs.stuff import Stufffrom pipelex.hub import get_pipe_router, get_required_pipefrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.exceptions import PipeStackOverflowErrorfrom pipelex.pipeline.job_metadata import JobMetadatadiff --git a/tests/integration/pipelex/system/test_keyless_boot_forced_dry.py b/tests/integration/pipelex/system/test_keyless_boot_forced_dry.pynew file mode 100644index 000000000..8e12523c5--- /dev/null+++ b/tests/integration/pipelex/system/test_keyless_boot_forced_dry.py@@ -0,0 +1,97 @@+"""Pin the keyless-boot contract (eng review D4): force-DRY flag only, generator backend-keyed.++A keyless boot (``needs_inference=False``) must NOT change generator selection — the backend alone+picks inline vs in-workflow, so a keyless Temporal submitter still dispatches activities (and the+leaf mocks inside them — the Tier 17 no-keys arm depends on it). What keyless boot does instead is+set the forced-DRY flag, which ``PipeRunParamsFactory.make_run_params`` — the single writer of+``run_mode`` — consumes by overriding any requested run mode to DRY, covering every execution+entry point (pipeline API, runtime bridge, factory defaults).+"""++from collections.abc import Generator++import pytest++from pipelex.cogt.content_generation.content_generator import ContentGenerator+from pipelex.config import get_config+from pipelex.core.concepts.native.concept_native import NativeConceptCode+from pipelex.core.pipes.pipe_factory import PipeFactory+from pipelex.hub import get_content_generator, is_dry_run_forced+from pipelex.pipe_operators.search.pipe_search import PipeSearch+from pipelex.pipe_operators.search.pipe_search_blueprint import PipeSearchBlueprint+from pipelex.pipe_run.pipe_run_mode import PipeRunMode+from pipelex.pipelex import Pipelex+from pipelex.pipeline.execution_seams import load_libraries_and_activate, prepare_pipe_job+from pipelex.system.runtime import IntegrationMode, runtime_manager+from pipelex.temporal.tprl_content_generation.content_generator_in_workflow import ContentGeneratorInWorkflow++[email protected](scope="module", autouse=True)+def reset_pipelex_config_fixture() -> Generator[None, None, None]:+ """Override the global module fixture: this module sets up and tears down per test."""+ yield+ Pipelex.teardown_if_needed()+++def _test_integration_mode() -> IntegrationMode:+ """CI mode on CI runners (no terms acceptance), PYTEST locally — mirrors the global conftest boot."""+ return IntegrationMode.CI if runtime_manager.is_ci_testing else IntegrationMode.PYTEST+++class TestKeylessBootForcedDry:+ def _boot_keyless(self, *, temporal_enabled: bool) -> None:+ Pipelex.teardown_if_needed()+ Pipelex.make(+ integration_mode=_test_integration_mode(),+ needs_inference=False,+ temporal_enabled=temporal_enabled,+ )++ @pytest.mark.asyncio+ async def test_keyless_direct_boot_inline_generator_and_forced_dry(self) -> None:+ """Keyless + direct: inline generator, forced-DRY flag set, and a LIVE-requested job prepared as DRY."""+ try:+ self._boot_keyless(temporal_enabled=False)+ assert is_dry_run_forced()+ assert isinstance(get_content_generator(), ContentGenerator)++ # The flag is consumed at the single writer of run_mode: a LIVE-requested job comes out DRY.+ library_id = load_libraries_and_activate([])+ pipe = PipeFactory[PipeSearch].make_from_blueprint(+ domain_code="generic",+ pipe_code="adhoc_for_keyless_forced_dry",+ blueprint=PipeSearchBlueprint(+ description="Keyless forced-dry test",+ output=NativeConceptCode.SEARCH_RESULT,+ prompt="What is Pipelex?",+ ),+ )+ pipe_job = await prepare_pipe_job(+ pipe=pipe,+ library_id=library_id,+ execution_config=get_config().pipelex.pipeline_execution_config,+ pipe_run_mode=PipeRunMode.LIVE,+ pipeline_run_id="keyless_forced_dry_run",+ user_id="test-user",+ )+ assert pipe_job.pipe_run_params.run_mode.is_dry+ finally:+ Pipelex.teardown_if_needed()++ def test_keyless_temporal_boot_in_workflow_generator_and_forced_dry(self) -> None:+ """Keyless + Temporal-enabled: the in-workflow generator is still selected (backend-keyed, D4)."""+ try:+ self._boot_keyless(temporal_enabled=True)+ assert is_dry_run_forced()+ assert isinstance(get_content_generator(), ContentGeneratorInWorkflow)+ finally:+ Pipelex.teardown_if_needed()++ def test_keyed_boot_does_not_force_dry(self) -> None:+ """Control arm: a normal boot leaves the flag unset."""+ try:+ Pipelex.teardown_if_needed()+ Pipelex.make(integration_mode=_test_integration_mode())+ assert not is_dry_run_forced()+ finally:+ Pipelex.teardown_if_needed()diff --git a/tests/integration/pipelex/temporal/conftest.py b/tests/integration/pipelex/temporal/conftest.pyindex 0fa4f0e9f..880f2967e 100644--- a/tests/integration/pipelex/temporal/conftest.py+++ b/tests/integration/pipelex/temporal/conftest.py@@ -1,3 +1,7 @@+import faulthandler+import logging+import os+from pathlib import Pathfrom typing import AsyncGenerator, Generator, castimport pytest@@ -21,6 +25,67 @@ from pipelex.test_extras.shared_pytest_plugins import ClassRegistryModeTEMPORAL_SERVER_NONE = "none"TEMPORAL_SERVER_TIME_SKIPPING = "time-skipping"+HANG_DUMP_DIR_ENV_VAR = "PIPELEX_HANG_DUMP_DIR"+HANG_DUMP_DELAY_SECONDS = 150.0++[email protected](scope="session", autouse=True)+def temporal_failure_log_capture() -> Generator[None, None, None]:+ """Mirror temporalio WARNING+ logs (with tracebacks) to a file under the hang-dump dir.++ Enabled only when ``PIPELEX_HANG_DUMP_DIR`` is set (CI hang debugging). A payload-conversion+ error inside a workflow task makes Temporal retry the task forever — the test then hangs+ until pytest-timeout's thread-method kill, which destroys pytest's captured logs along with+ the process. temporalio logs each failed workflow activation WITH the offending traceback,+ so mirroring its logger to a file preserves the actual error across the kill.+ """+ dump_dir_value = os.environ.get(HANG_DUMP_DIR_ENV_VAR)+ if not dump_dir_value:+ yield+ return+ dump_dir = Path(dump_dir_value)+ dump_dir.mkdir(parents=True, exist_ok=True)+ handler = logging.FileHandler(dump_dir / f"temporalio-pid{os.getpid()}.log", encoding="utf-8")+ handler.setLevel(logging.WARNING)+ handler.setFormatter(logging.Formatter("%(asctime)s %(name)s %(levelname)s %(message)s"))+ temporalio_logger = logging.getLogger("temporalio")+ temporalio_logger.addHandler(handler)+ try:+ yield+ finally:+ temporalio_logger.removeHandler(handler)+ handler.close()++[email protected](autouse=True)+def temporal_hang_watchdog(request: FixtureRequest) -> Generator[None, None, None]:+ """Dump every thread's stack to a file when a temporal test exceeds the watchdog delay.++ Enabled only when the ``PIPELEX_HANG_DUMP_DIR`` env var is set (CI hang debugging).+ The delay sits below pytest-timeout's ``--timeout`` so the dump lands BEFORE the+ thread-method kill (``os._exit``) destroys the process — pytest-xdist swallows worker+ stderr on "node down", so a file under the dump dir is the only way to see where a+ hung test was stuck on a remote runner. One file per worker process; each test appends+ an armed/disarmed marker, so the hung test is the last armed entry without a disarm.+ """+ dump_dir_value = os.environ.get(HANG_DUMP_DIR_ENV_VAR)+ if not dump_dir_value:+ yield+ return+ test_id = cast("str", request.node.nodeid) # pyright: ignore[reportUnknownMemberType]+ dump_dir = Path(dump_dir_value)+ dump_dir.mkdir(parents=True, exist_ok=True)+ dump_path = dump_dir / f"hang-pid{os.getpid()}.txt"+ with dump_path.open("a", encoding="utf-8") as dump_file:+ dump_file.write(f"\n===== armed: {test_id} =====\n")+ dump_file.flush()+ faulthandler.dump_traceback_later(HANG_DUMP_DELAY_SECONDS, file=dump_file)+ try:+ yield+ finally:+ faulthandler.cancel_dump_traceback_later()+ dump_file.write(f"===== disarmed: {test_id} =====\n")+def pytest_addoption(parser: Parser) -> None:parser.addoption(@@ -66,8 +131,7 @@ def boot_temporal(reset_pipelex_config_fixture: None) -> Generator[None, None, N# 1. Sub-pipes dispatch as child workflows (not inline in the sandbox)# 2. Inference calls (LLM, img_gen, etc.) dispatch as activities (not inline)# This mirrors what a full Temporal-enabled Pipelex.make() would set up.- from pipelex.cogt.content_generation.generated_content_factory import GeneratedContentFactory # noqa: PLC0415- from pipelex.hub import get_pipelex_hub, get_storage_provider # noqa: PLC0415+ from pipelex.hub import get_pipelex_hub # noqa: PLC0415from pipelex.temporal.tprl_content_generation.content_generator_in_workflow_factory import ( # noqa: PLC0415ContentGeneratorInWorkflowFactory,)@@ -76,12 +140,7 @@ def boot_temporal(reset_pipelex_config_fixture: None) -> Generator[None, None, Npipelex_hub = get_pipelex_hub()pipelex_hub.set_pipe_router(make_temporal_pipe_router())- generated_content_factory = GeneratedContentFactory(storage_provider=get_storage_provider())- pipelex_hub.set_content_generator(- ContentGeneratorInWorkflowFactory.make_content_generator_in_workflow(- generated_content_factory=generated_content_factory,- )- )+ pipelex_hub.set_content_generator(ContentGeneratorInWorkflowFactory.make_content_generator_in_workflow())yieldmanager.teardown()diff --git a/tests/integration/pipelex/temporal/data_converter/test_data_conv_craft.py b/tests/integration/pipelex/temporal/data_converter/test_data_conv_craft.pyindex a4a9b278a..25935aeef 100644--- a/tests/integration/pipelex/temporal/data_converter/test_data_conv_craft.py+++ b/tests/integration/pipelex/temporal/data_converter/test_data_conv_craft.py@@ -2,8 +2,10 @@ import pytestfrom pipelex import pretty_printfrom pipelex.cogt.content_generation.assignment_models import LLMAssignment, ObjectAssignment+from pipelex.cogt.content_generation.cogt_run_params import CogtRunParamsfrom pipelex.cogt.llm.llm_prompt import LLMPromptfrom pipelex.hub import get_model_deck+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipeline.job_metadata import JobMetadatafrom pipelex.temporal.temporal_data_converter import BaseModelPayloadConverterfrom pipelex.temporal.test_extras.temporal_registry_test_models import Person@@ -22,6 +24,7 @@ class TestDataConverterForCrafting:llm_prompt_for_text = LLMPrompt(user_text=user_text)llm_assignment = LLMAssignment(job_metadata=JobMetadata(user_id="test", pipeline_run_id="test"),+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.DRY),llm_setting=llm_setting,llm_prompt=llm_prompt_for_text,)@@ -33,6 +36,8 @@ class TestDataConverterForCrafting:pretty_print(restored, title="restored LLMAssignment")assert restoredassert llm_assignment == restored+ # The run-mode contract must survive the wire: the leaf inside the activity keys on it.+ assert restored.cogt_run_params.run_mode.is_drydef test_data_converter_for_make_object(self,@@ -43,6 +48,7 @@ class TestDataConverterForCrafting:llm_prompt_for_object = LLMPrompt(user_text=user_text)llm_assignment_for_object = LLMAssignment(job_metadata=JobMetadata(user_id="test", pipeline_run_id="test"),+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),llm_setting=llm_setting_for_object,llm_prompt=llm_prompt_for_object,)diff --git a/tests/integration/pipelex/temporal/data_converter/test_data_conv_pipe_run_params.py b/tests/integration/pipelex/temporal/data_converter/test_data_conv_pipe_run_params.pynew file mode 100644index 000000000..cc353baf5--- /dev/null+++ b/tests/integration/pipelex/temporal/data_converter/test_data_conv_pipe_run_params.py@@ -0,0 +1,37 @@+"""Wire round-trip for ``PipeRunParams`` with the nested ``CogtRunParams`` carrier (eng review D2).++``WfPipeRouter`` receives ``pipe_run_params`` inside its workflow arg, so the nested carrier must+survive the Temporal payload converter — and ``run_mode`` (a delegating property, deliberately not+a serialized field) must read back correctly from the restored nested model.+"""++import pytest++from pipelex.cogt.content_generation.cogt_run_params import CogtRunParams+from pipelex.pipe_run.pipe_run_mode import PipeRunMode+from pipelex.pipe_run.pipe_run_params import PipeRunParams+from pipelex.temporal.temporal_data_converter import BaseModelPayloadConverter+++class TestDataConverterForPipeRunParams:+ def test_cogt_run_params_round_trip(+ self,+ payload_converter: BaseModelPayloadConverter,+ ):+ pipe_run_params = PipeRunParams(+ run_mode=PipeRunMode.DRY,+ is_mock_usage=True,+ pipe_stack_limit=20,+ pipe_stack=["root", "child"],+ )++ payload = payload_converter.to_payload(pipe_run_params)+ assert payload+ restored: PipeRunParams = payload_converter.from_payload(payload, type_hint=PipeRunParams)++ assert restored == pipe_run_params+ assert restored.run_mode.is_dry+ assert restored.is_mock_usage+ # The derived carrier reads the restored fields — what generators stamp on assignments.+ assert restored.cogt_run_params == CogtRunParams(run_mode=PipeRunMode.DRY, is_mock_usage=True)diff --git a/tests/integration/pipelex/temporal/test_activity_error_boundary.py b/tests/integration/pipelex/temporal/test_activity_error_boundary.pyindex 4cff5160a..cde348e3b 100644--- a/tests/integration/pipelex/temporal/test_activity_error_boundary.py+++ b/tests/integration/pipelex/temporal/test_activity_error_boundary.py@@ -31,11 +31,13 @@ from temporalio.worker import UnsandboxedWorkflowRunner, Workerfrom pipelex import logfrom pipelex.cogt.content_generation.assignment_models import ExtractAssignment, LLMAssignment+from pipelex.cogt.content_generation.cogt_run_params import CogtRunParamsfrom pipelex.cogt.exceptions import CogtError, InferenceErrorCategoryfrom pipelex.cogt.extract.extract_input import ExtractInputfrom pipelex.cogt.extract.extract_job_components import ExtractJobConfig, ExtractJobParamsfrom pipelex.cogt.llm.llm_prompt import LLMPromptfrom pipelex.hub import get_model_deck+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipeline.job_metadata import JobMetadatafrom pipelex.temporal.tprl.temporal_error import TemporalErrorfrom pipelex.temporal.tprl_content_generation.act_extract_generate import act_extract_gen_extract_pages@@ -153,6 +155,7 @@ class TestActivityErrorBoundary:llm_assignment = LLMAssignment(job_metadata=JobMetadata(user_id="test", pipeline_run_id="test"),+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),llm_setting=get_model_deck().get_llm_setting(llm_choice="$testing-text"),llm_prompt=LLMPrompt(user_text="never reaches the model — llm_gen_text is mocked"),)@@ -208,6 +211,7 @@ class TestActivityErrorBoundary:extract_assignment = ExtractAssignment(job_metadata=JobMetadata(user_id="test", pipeline_run_id="test"),+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),extract_handle="extract-handle-never-reached",extract_input=ExtractInput(document_uri="file://never-reached.pdf"),extract_job_params=ExtractJobParams.make_default_extract_job_params(),diff --git a/tests/integration/pipelex/temporal/test_dry_run_graph_in_process.py b/tests/integration/pipelex/temporal/test_dry_run_graph_in_process.pynew file mode 100644index 000000000..a884705ad--- /dev/null+++ b/tests/integration/pipelex/temporal/test_dry_run_graph_in_process.py@@ -0,0 +1,116 @@+"""Mode-1 Temporal guard: the in-process graph dry-run must stay in-process under a Temporal hub.++Phase-2 companion to ``test_validate_sweep_stays_in_process``: where that test proves the+*validation sweep* never dispatches to Temporal, this one proves the *graph-producing dry-run*+(``dry_run_pipe_in_process``) runs fully in-process and traces into an in-memory event log even+when the hub default router is the real ``TemporalPipeRouter`` and the hub default content+generator is ``ContentGeneratorInWorkflow`` (wired by this suite's ``boot_temporal`` fixture,+exactly as a Temporal-enabled worker/API process is wired).++Contract asserted (TODOS.md Phase 2):++- (a) a correct, non-empty ``GraphSpec`` comes back, covering the whole controller topology;+- (b) ZERO workflows/activities are dispatched (spy on ``WorkflowExecutor.execute_workflow``);+- (c) no file/DDB transport is touched (``make_event_log`` is forbidden — the scoped in-memory+ instance is the single transport);+- the zero-dispatch guarantee holds with the DRY mock at the LEAF (Part B): DRY routes through+ the hub-resolved content generator, so the scoped inline generator is what keeps the run+ in-process — this is what ``scoped_content_generator`` exists for;+- tracer-key alignment: emit and assemble share the ``pipeline_run_id`` partition by+ construction (a divergent key would yield an empty graph, so the non-empty assertion pins it).+"""++from pathlib import Path++import pytest+from pytest_mock import MockerFixture++from pipelex.cogt.content_generation.content_generator import ContentGenerator+from pipelex.graph.graphspec import GraphSpec+from pipelex.hub import get_library_manager, get_pipelex_hub, get_required_pipe, scoped_current_library+from pipelex.pipe_run.dry_run_pipeline import dry_run_pipe_in_process+from pipelex.pipeline.execution_seams import acquire_library+from pipelex.temporal.tprl.workflow_caller import WorkflowExecutor+from pipelex.temporal.tprl_content_generation.content_generator_in_workflow import ContentGeneratorInWorkflow+from pipelex.temporal.tprl_pipe.temporal_pipe_router import TemporalPipeRouter+from pipelex.tracing.in_memory_event_log import InMemoryEventLog+from pipelex.tracing.trace_events import UsageReportEvent+from tests.integration.pipelex.temporal.test_data import PipeParallelTemporalTestData++_PARALLEL_MAIN_PIPE_REF = f"{PipeParallelTemporalTestData.DOMAIN}.{PipeParallelTemporalTestData.PIPE_CODE}"++[email protected](loop_scope="class")+class TestDryRunGraphInProcess:+ def _assert_temporal_hub_preconditions(self) -> None:+ """The hub defaults must be the REAL Temporal ones — that's the environment being guarded against."""+ hub = get_pipelex_hub()+ assert isinstance(hub.get_required_pipe_router(), TemporalPipeRouter), (+ "Temporal suite precondition: the hub default router must be the real TemporalPipeRouter"+ )+ assert isinstance(hub.get_required_content_generator(), ContentGeneratorInWorkflow), (+ "Temporal suite precondition: the hub default content generator must be ContentGeneratorInWorkflow"+ )++ def _forbid_event_log_factory(self, mocker: MockerFixture) -> None:+ factory_error = AssertionError("make_event_log must not be called: the in-process graph dry-run traces in memory only")+ mocker.patch("pipelex.pipeline.pipeline_run_setup.make_event_log", side_effect=factory_error)+ mocker.patch("pipelex.pipe_run.tracing_assembly.make_event_log", side_effect=factory_error)++ async def _run_in_process_graph_dry_run(self, library_id: str) -> GraphSpec:+ mthds_content = Path(PipeParallelTemporalTestData.BUNDLE_FILE).read_text(encoding="utf-8")+ # scoped_current_library restores any outer current-library binding on exit instead of+ # clobbering it (acquire_library sets the same id internally; the scope owns the restore).+ with scoped_current_library(library_id):+ acquire_library(library_id=library_id, mthds_contents=[mthds_content])+ try:+ main_pipe = get_required_pipe(pipe_code=_PARALLEL_MAIN_PIPE_REF)+ return await dry_run_pipe_in_process(pipe=main_pipe, library_id=library_id)+ finally:+ get_library_manager().teardown(library_id=library_id)++ async def test_graph_dry_run_zero_dispatch_in_memory(self, mocker: MockerFixture) -> None:+ self._assert_temporal_hub_preconditions()+ execute_workflow_spy = mocker.spy(WorkflowExecutor, "execute_workflow")+ self._forbid_event_log_factory(mocker)+ emit_spy = mocker.spy(InMemoryEventLog, "emit")++ graph_spec = await self._run_in_process_graph_dry_run(library_id="dry_run_graph_in_process_lib")++ # (a) correct, non-empty GraphSpec covering the whole controller topology (sequence,+ # parallel fan-out, both branches, summary). Tracer-key alignment is pinned by the same+ # assertion: a divergent emit/assemble key would have produced an empty graph.+ traced_pipe_codes = {node.pipe_code for node in graph_spec.nodes if node.pipe_code}+ expected_pipe_codes = {pipe_ref.split(".")[-1] for pipe_ref in PipeParallelTemporalTestData.EXPECTED_PIPE_REFS}+ assert expected_pipe_codes <= traced_pipe_codes+ assert graph_spec.edges++ # (b) zero Temporal dispatch during the whole dry-run.+ execute_workflow_spy.assert_not_called()++ # (d) usage-event isolation (pre-flight decision 3): the dry leaves report synthetic+ # zero-token jobs, but the run's trace context has emit_usage_events=False, so no+ # UsageReportEvent ever reaches the (scoped, in-memory) transport — nothing leaks to any+ # ambient registry, and the events die with the run.+ emitted_events = [call.args[1] for call in emit_spy.call_args_list]+ assert emitted_events, "the dry-run must emit graph events through the scoped log"+ assert not [event for event in emitted_events if isinstance(event, UsageReportEvent)]++ async def test_leaf_level_mock_stays_in_process(self, mocker: MockerFixture) -> None:+ """Leaf-level DRY (Part B, now the real path): DRY routes through the hub-resolved+ generator and the cogt leaf mocks, so the scoped inline generator must keep the run+ in-process — without ``scoped_content_generator`` the leaf would reach the hub's+ ``ContentGeneratorInWorkflow`` and try to dispatch activities.+ """+ self._assert_temporal_hub_preconditions()+ execute_workflow_spy = mocker.spy(WorkflowExecutor, "execute_workflow")+ self._forbid_event_log_factory(mocker)+ # Prove the run actually used the scoped inline generator (not the hub default).+ inline_text_spy = mocker.spy(ContentGenerator, "make_llm_text")++ graph_spec = await self._run_in_process_graph_dry_run(library_id="dry_run_graph_leaf_mock_lib")++ assert graph_spec.nodes+ assert inline_text_spy.call_count >= 1+ execute_workflow_spy.assert_not_called()diff --git a/tests/integration/pipelex/temporal/test_dry_validate_activity_in_memory.py b/tests/integration/pipelex/temporal/test_dry_validate_activity_in_memory.pynew file mode 100644index 000000000..32699bb3e--- /dev/null+++ b/tests/integration/pipelex/temporal/test_dry_validate_activity_in_memory.py@@ -0,0 +1,333 @@+"""Mode-1 isolation tests for the dry-run+validation Temporal activity (``act_dry_validate``).++The whole ``/validate`` job — validation sweep **+** graph-producing dry-run — runs as ONE+in-process activity dispatched via the one-step wrapper workflow ``WfDryValidate``. A real worker+runs the wrapper-workflow→activity against the in-process Temporal server, and the tests pin the+Phase-3 contract (TODOS.md):++- the per-pipe status map and the ``GraphSpec`` are correct, in one round-trip;+- ZERO nested dispatch: the workflow history contains exactly one scheduled activity+ (``act_dry_validate``) and no child workflows (asserted at the Temporal-history level, per D6 —+ not on the ``WorkflowExecutor`` wrapper);+- tracing stays in memory: no NDJSON partition appears, ``make_event_log`` is never called;+- the graph is best-effort (D5): an expected dry-run failure yields ``graph_spec=None`` with+ validation still successful, while a non-Pipelex programming bug propagates and fails the run;+- a validation failure crosses back as a structured ``ErrorReport`` whose offending-signature+ data survives the crossing (D3/T3);+- concurrent invocations don't cross-contaminate the scoped overrides.+"""++import asyncio+import uuid+from pathlib import Path++import pytest+from pytest_mock import MockerFixture+from temporalio.api.enums.v1 import EventType+from temporalio.client import Client as TemporalClient+from temporalio.client import WorkflowFailureError+from temporalio.common import RetryPolicy+from temporalio.worker import UnsandboxedWorkflowRunner, Worker++from pipelex.config import get_config+from pipelex.hub import clear_current_library, get_current_library_id_or_none, get_library_manager+from pipelex.pipe_run.exceptions import DryRunError+from pipelex.pipeline.bundle_validator import DryRunStatus+from pipelex.pipeline.execution_seams import acquire_library+from pipelex.pipeline.validate_bundle import validate_bundle+from pipelex.system.configuration.configs import NdjsonTracingConfig, TracingBackend+from pipelex.temporal.tprl.temporal_error import recover_error_report+from pipelex.temporal.tprl_pipe.act_dry_validate import DryValidateArg, DryValidateResult, act_dry_validate+from pipelex.temporal.tprl_pipe.wf_dry_validate import WfDryValidate+from tests.integration.pipelex.temporal.test_data import PipeParallelTemporalTestData++_PARALLEL_MAIN_PIPE_REF = f"{PipeParallelTemporalTestData.DOMAIN}.{PipeParallelTemporalTestData.PIPE_CODE}"++_ALPHA_MTHDS = """+domain = "dry_validate_alpha"+description = "Alpha bundle for act_dry_validate isolation tests"+main_pipe = "alpha_sequence"++[pipe.alpha_sequence]+type = "PipeSequence"+description = "Two-step alpha sequence"+inputs = { subject = "Text" }+output = "Text"+steps = [+ { pipe = "alpha_first", result = "first_take" },+ { pipe = "alpha_second", result = "second_take" },+]++[pipe.alpha_first]+type = "PipeLLM"+description = "First alpha step"+inputs = { subject = "Text" }+output = "Text"+prompt = "Describe $subject"++[pipe.alpha_second]+type = "PipeLLM"+description = "Second alpha step"+inputs = { first_take = "Text" }+output = "Text"+prompt = "Refine $first_take"+"""++_BETA_MTHDS = """+domain = "dry_validate_beta"+description = "Beta bundle for act_dry_validate isolation tests"+main_pipe = "beta_single"++[pipe.beta_single]+type = "PipeLLM"+description = "Single beta pipe"+inputs = { topic = "Text" }+output = "Text"+prompt = "Expand on $topic"+"""++_NO_MAIN_MTHDS = """+domain = "dry_validate_no_main"+description = "Bundle with no main_pipe, for the no-graph arm"++[pipe.lone_pipe]+type = "PipeLLM"+description = "Single pipe, no main_pipe declared"+inputs = { topic = "Text" }+output = "Text"+prompt = "Expand on $topic"+"""++_SIGNATURE_MTHDS = """+domain = "dry_validate_sig"+description = "Bundle carrying an unimplemented PipeSignature"+main_pipe = "sig_caller"++[pipe.sig_caller]+type = "PipeSequence"+description = "Sequence that reaches a signature"+inputs = { doc = "Text" }+output = "Text"+steps = [+ { pipe = "unimplemented_sig", result = "sig_result" },+]++[pipe.unimplemented_sig]+type = "PipeSignature"+description = "A contract-only placeholder"+inputs = { doc = "Text" }+output = "Text"+"""++[email protected](loop_scope="class")+class TestDryValidateActivityInMemory:+ def _forbid_event_log_factory(self, mocker: MockerFixture) -> None:+ factory_error = AssertionError("make_event_log must not be called: act_dry_validate traces in memory only")+ mocker.patch("pipelex.pipeline.pipeline_run_setup.make_event_log", side_effect=factory_error)+ mocker.patch("pipelex.pipe_run.tracing_assembly.make_event_log", side_effect=factory_error)++ def _point_ndjson_tracing_at(self, mocker: MockerFixture, traces_dir: Path) -> None:+ cfg = get_config().pipelex.tracing_config+ mocker.patch.object(cfg, "is_enabled", True)+ mocker.patch.object(cfg, "backend", TracingBackend.NDJSON)+ mocker.patch.object(cfg, "ndjson", NdjsonTracingConfig(traces_dir=str(traces_dir)))++ async def _execute(+ self,+ temporal_client: TemporalClient,+ arg: DryValidateArg,+ *,+ workflow_id: str | None = None,+ task_queue: str | None = None,+ ) -> DryValidateResult:+ workflow_id = workflow_id or f"wf_dry_validate_{uuid.uuid4().hex[:8]}"+ task_queue = task_queue or f"q_dry_validate_{uuid.uuid4().hex[:8]}"+ async with Worker(+ temporal_client,+ task_queue=task_queue,+ workflows=[WfDryValidate],+ activities=[act_dry_validate],+ workflow_runner=UnsandboxedWorkflowRunner(),+ ):+ return await temporal_client.execute_workflow( # pyright: ignore[reportUnknownMemberType]+ workflow=WfDryValidate.run,+ arg=arg,+ id=workflow_id,+ task_queue=task_queue,+ retry_policy=RetryPolicy(maximum_attempts=1),+ )++ async def test_status_map_and_graph_in_one_round_trip_zero_nested_dispatch(+ self, temporal_client: TemporalClient, tmp_path: Path, mocker: MockerFixture+ ) -> None:+ traces_dir = tmp_path / "traces"+ traces_dir.mkdir()+ self._point_ndjson_tracing_at(mocker, traces_dir)+ self._forbid_event_log_factory(mocker)++ workflow_id = f"wf_dry_validate_{uuid.uuid4().hex[:8]}"+ mthds_content = Path(PipeParallelTemporalTestData.BUNDLE_FILE).read_text(encoding="utf-8")+ result = await self._execute(+ temporal_client,+ DryValidateArg(mthds_contents=[mthds_content], pipe_code=_PARALLEL_MAIN_PIPE_REF),+ workflow_id=workflow_id,+ )++ # Status map: every pipe in the bundle swept SUCCESS.+ for pipe_ref in PipeParallelTemporalTestData.EXPECTED_PIPE_REFS:+ assert result.dry_run_outputs[pipe_ref].status == DryRunStatus.SUCCESS++ # GraphSpec: non-empty, covering the whole controller topology.+ assert result.graph_spec is not None+ traced_pipe_codes = {node.pipe_code for node in result.graph_spec.nodes if node.pipe_code}+ expected_pipe_codes = {pipe_ref.split(".")[-1] for pipe_ref in PipeParallelTemporalTestData.EXPECTED_PIPE_REFS}+ assert expected_pipe_codes <= traced_pipe_codes++ # Zero nested dispatch, asserted on the Temporal history itself (D6): exactly one scheduled+ # activity — act_dry_validate — and no child workflows.+ history = await temporal_client.get_workflow_handle(workflow_id).fetch_history()+ scheduled_activities = [+ event.activity_task_scheduled_event_attributes.activity_type.name+ for event in history.events+ if event.event_type == EventType.EVENT_TYPE_ACTIVITY_TASK_SCHEDULED+ ]+ assert scheduled_activities == ["act_dry_validate"]+ child_workflow_initiations = [+ event for event in history.events if event.event_type == EventType.EVENT_TYPE_START_CHILD_WORKFLOW_EXECUTION_INITIATED+ ]+ assert child_workflow_initiations == []++ # In-memory tracing: no NDJSON partition appeared (make_event_log was also forbidden above).+ assert list(traces_dir.iterdir()) == []++ async def test_main_pipe_auto_resolution_from_bundle(self, temporal_client: TemporalClient, mocker: MockerFixture) -> None:+ """Without an explicit pipe_code, the graph arm targets the bundle's declared main_pipe."""+ self._forbid_event_log_factory(mocker)++ result = await self._execute(temporal_client, DryValidateArg(mthds_contents=[_ALPHA_MTHDS]))++ assert result.dry_run_outputs["dry_validate_alpha.alpha_sequence"].status == DryRunStatus.SUCCESS+ assert result.graph_spec is not None+ traced_pipe_codes = {node.pipe_code for node in result.graph_spec.nodes if node.pipe_code}+ assert {"alpha_sequence", "alpha_first", "alpha_second"} <= traced_pipe_codes++ async def test_graph_failure_is_best_effort(self, temporal_client: TemporalClient, mocker: MockerFixture) -> None:+ """D5: an expected dry-run failure in the graph arm yields graph_spec=None, validation still OK."""+ mocker.patch(+ "pipelex.temporal.tprl_pipe.act_dry_validate.dry_run_pipe_in_process",+ side_effect=DryRunError("simulated graph dry-run failure"),+ )++ result = await self._execute(temporal_client, DryValidateArg(mthds_contents=[_ALPHA_MTHDS]))++ assert result.graph_spec is None+ assert result.dry_run_outputs["dry_validate_alpha.alpha_sequence"].status == DryRunStatus.SUCCESS++ async def test_graph_bug_propagates(self, temporal_client: TemporalClient, mocker: MockerFixture) -> None:+ """D5 (bug-propagates arm): a non-Pipelex programming bug in the graph arm fails the run."""+ mocker.patch(+ "pipelex.temporal.tprl_pipe.act_dry_validate.dry_run_pipe_in_process",+ side_effect=KeyError("simulated programming bug"),+ )++ with pytest.raises(WorkflowFailureError):+ await self._execute(temporal_client, DryValidateArg(mthds_contents=[_ALPHA_MTHDS]))++ async def test_validation_failure_crosses_as_structured_error_report(self, temporal_client: TemporalClient) -> None:+ """D3/T3: a strict-mode signature refusal raises, and the structured offending-signature+ data survives the ErrorReport crossing (it must not degrade to a plain message).++ The sweep goes through ``validate_bundle``'s cascade, so the crossing report carries the+ SAME identity the direct-mode route's 422 carries — ``ValidateBundleError``,+ ``error_domain=input`` — with the offender and signature refs in the caller-facing+ message (the API problem document is built from exactly these ErrorReport fields).+ """+ with pytest.raises(WorkflowFailureError) as exc_info:+ await self._execute(temporal_client, DryValidateArg(mthds_contents=[_SIGNATURE_MTHDS], allow_signatures=False))++ error_report = recover_error_report(exc_info.value)+ assert error_report.error_type == "ValidateBundleError"+ assert error_report.error_domain == "input"+ # The structured signature data — offender and signature refs — survives in the report.+ assert "dry_validate_sig.sig_caller" in error_report.message+ assert "dry_validate_sig.unimplemented_sig" in error_report.message++ async def test_signatures_allowed_in_lenient_mode(self, temporal_client: TemporalClient) -> None:+ """The same signature bundle validates in lenient mode (allow_signatures=True)."""+ result = await self._execute(temporal_client, DryValidateArg(mthds_contents=[_SIGNATURE_MTHDS], allow_signatures=True))++ assert result.dry_run_outputs["dry_validate_sig.sig_caller"].status == DryRunStatus.SUCCESS+ assert result.graph_spec is not None++ async def test_validation_failure_does_not_retry_activity(self, temporal_client: TemporalClient, mocker: MockerFixture) -> None:+ """D-C5 regression: a deterministic validation failure must run the activity exactly once.+ ValidateBundleError is non-retryable at the activity tier via a string match on the+ ApplicationError type name — if that match drifts (class rename, boundary change), the+ whole sweep silently re-runs and only this assertion catches it.+ """+ sweep_spy = mocker.patch(+ "pipelex.temporal.tprl_pipe.act_dry_validate.validate_bundle",+ wraps=validate_bundle,+ )+ with pytest.raises(WorkflowFailureError):+ await self._execute(temporal_client, DryValidateArg(mthds_contents=[_SIGNATURE_MTHDS], allow_signatures=False))+ assert sweep_spy.call_count == 1++ async def test_no_main_pipe_and_no_pipe_code_yields_no_graph(self, temporal_client: TemporalClient) -> None:+ """Without a declared main_pipe and without an explicit pipe_code, the graph arm is+ skipped: validation succeeds with graph_spec=None.+ """+ result = await self._execute(temporal_client, DryValidateArg(mthds_contents=[_NO_MAIN_MTHDS]))++ assert result.dry_run_outputs["dry_validate_no_main.lone_pipe"].status == DryRunStatus.SUCCESS+ assert result.graph_spec is None++ async def test_unknown_explicit_pipe_code_degrades_to_no_graph(self, temporal_client: TemporalClient) -> None:+ """Graph-arm parity: an unknown explicit pipe_code is a graph-arm domain failure+ (PipeNotFoundError, a PipelexError) and degrades to graph_spec=None with validation+ still successful — the same answer the direct route gives, never a failed request.+ """+ result = await self._execute(temporal_client, DryValidateArg(mthds_contents=[_ALPHA_MTHDS], pipe_code="dry_validate_alpha.does_not_exist"))++ assert result.graph_spec is None+ assert result.dry_run_outputs["dry_validate_alpha.alpha_sequence"].status == DryRunStatus.SUCCESS++ async def test_direct_call_restores_outer_current_library(self) -> None:+ """The activity's finally restores the caller's outer current-library (the+ prev != validated arm) after tearing the validated library down. Exercised by calling+ the activity directly — through a worker, the activity runs in its own copied context+ and the test task could not observe the ContextVar restore.+ """+ outer_library_id = "outer_lib_dry_validate"+ acquire_library(library_id=outer_library_id, mthds_contents=[_BETA_MTHDS])+ try:+ result = await act_dry_validate(DryValidateArg(mthds_contents=[_ALPHA_MTHDS]))++ assert result.graph_spec is not None+ assert get_current_library_id_or_none() == outer_library_id+ finally:+ get_library_manager().teardown(library_id=outer_library_id)+ clear_current_library()++ async def test_concurrent_invocations_do_not_cross_contaminate(self, temporal_client: TemporalClient, mocker: MockerFixture) -> None:+ """Two concurrent dispatches return distinct GraphSpecs with no shared/merged trace events."""+ self._forbid_event_log_factory(mocker)++ result_alpha, result_beta = await asyncio.gather(+ self._execute(temporal_client, DryValidateArg(mthds_contents=[_ALPHA_MTHDS])),+ self._execute(temporal_client, DryValidateArg(mthds_contents=[_BETA_MTHDS])),+ )++ assert result_alpha.graph_spec is not None+ assert result_beta.graph_spec is not None+ assert result_alpha.graph_spec.graph_id != result_beta.graph_spec.graph_id+ alpha_codes = {node.pipe_code for node in result_alpha.graph_spec.nodes if node.pipe_code}+ beta_codes = {node.pipe_code for node in result_beta.graph_spec.nodes if node.pipe_code}+ assert "alpha_sequence" in alpha_codes+ assert "beta_single" not in alpha_codes+ assert "beta_single" in beta_codes+ assert "alpha_sequence" not in beta_codes+ assert "dry_validate_beta.beta_single" not in result_alpha.dry_run_outputs+ assert "dry_validate_alpha.alpha_sequence" not in result_beta.dry_run_outputsdiff --git a/tests/integration/pipelex/temporal/test_payload_codec_pipeline.py b/tests/integration/pipelex/temporal/test_payload_codec_pipeline.pyindex 91a004af8..6cbfbeab3 100644--- a/tests/integration/pipelex/temporal/test_payload_codec_pipeline.py+++ b/tests/integration/pipelex/temporal/test_payload_codec_pipeline.py@@ -69,8 +69,7 @@ def boot_temporal(reset_pipelex_config_fixture: None) -> Generator[None, None, N)manager.setup()- from pipelex.cogt.content_generation.generated_content_factory import GeneratedContentFactory # noqa: PLC0415- from pipelex.hub import get_pipelex_hub, get_storage_provider # noqa: PLC0415+ from pipelex.hub import get_pipelex_hub # noqa: PLC0415from pipelex.temporal.tprl_content_generation.content_generator_in_workflow_factory import ( # noqa: PLC0415ContentGeneratorInWorkflowFactory,)@@ -79,12 +78,7 @@ def boot_temporal(reset_pipelex_config_fixture: None) -> Generator[None, None, Npipelex_hub = get_pipelex_hub()pipelex_hub.set_pipe_router(make_temporal_pipe_router())- generated_content_factory = GeneratedContentFactory(storage_provider=get_storage_provider())- pipelex_hub.set_content_generator(- ContentGeneratorInWorkflowFactory.make_content_generator_in_workflow(- generated_content_factory=generated_content_factory,- )- )+ pipelex_hub.set_content_generator(ContentGeneratorInWorkflowFactory.make_content_generator_in_workflow())yieldmanager.teardown()diff --git a/tests/integration/pipelex/temporal/tracing/helpers.py b/tests/integration/pipelex/temporal/tracing/helpers.pyindex eb5dca3cd..3bd374799 100644--- a/tests/integration/pipelex/temporal/tracing/helpers.py+++ b/tests/integration/pipelex/temporal/tracing/helpers.py@@ -1,7 +1,7 @@"""Shared test helpers for Temporal graph tracing integration tests."""import uuid-from collections import defaultdict+from collections import Counter, defaultdictfrom collections.abc import AsyncGenerator, Awaitable, Callable, Generator, Iterablefrom contextlib import asynccontextmanager, contextmanagerfrom datetime import datetime@@ -209,13 +209,11 @@ async def _runner_isolated_act_llm_gen_text(llm_assignment: LLMAssignment) -> st- Returns a fixed string so the calling workflow can finish withoutattempting a real LLM call.- This avoids the LIVE/DRY-mode dilemma: the existing `pipe_job_from_bundle`- fixture defaults to DRY mode, where the LLM activity is normally bypassed- by `ContentGeneratorDry` reporting inline inside the workflow. Subbing- `act_llm_gen_text` with this version forces the activity to fire from the- workflow's `start_activity` call (because we route `act_llm_gen_text` to- `q_runner` via `worker_config.activity_queues`), exercising the- cross-worker path even in DRY mode and keeping the test hermetic.+ Note: since Part B (leaf-level dry mock), DRY mode also dispatches+ `act_llm_gen_text` (the leaf mocks inside the activity with zero-token,+ suppressed usage), so this substitute is what guarantees a *reportable*+ non-zero usage event lands on the runner regardless of the fixture's run+ mode — keeping the cross-worker assertion hermetic and deterministic."""delegate = get_report_delegate()if isinstance(delegate, ReportingManager):@@ -316,3 +314,30 @@ async def make_split_workers(),):yield+++async def collect_scheduled_activity_counts(temporal_client: TemporalClient, workflow_id: str) -> "Counter[str]":+ """Walk a workflow tree (parent + child workflows, pinned by run_id) and count every scheduled activity.++ Counts ``ActivityTaskScheduled`` events by activity-type name across the parent history and every+ child workflow it started — the leaf-dispatch evidence Tier 17 asserts on. Children are fetched by+ ``(workflow_id, run_id)`` so a retried/continued run under the same id cannot substitute a history+ the parent never observed.+ """+ scheduled_counts: Counter[str] = Counter()+ pending_runs: list[tuple[str, str | None]] = [(workflow_id, None)]+ visited_runs: set[tuple[str, str | None]] = set()+ while pending_runs:+ current_run = pending_runs.pop()+ if current_run in visited_runs:+ continue+ visited_runs.add(current_run)+ current_id, current_run_id = current_run+ history = await temporal_client.get_workflow_handle(current_id, run_id=current_run_id).fetch_history()+ for event in history.events:+ if event.HasField("activity_task_scheduled_event_attributes"):+ scheduled_counts[event.activity_task_scheduled_event_attributes.activity_type.name] += 1+ elif event.HasField("child_workflow_execution_started_event_attributes"):+ child_execution = event.child_workflow_execution_started_event_attributes.workflow_execution+ pending_runs.append((child_execution.workflow_id, child_execution.run_id))+ return scheduled_countsdiff --git a/tests/integration/pipelex/temporal/tracing/test_dry_run_dispatches_and_mocks.py b/tests/integration/pipelex/temporal/tracing/test_dry_run_dispatches_and_mocks.pynew file mode 100644index 000000000..f33313a0c--- /dev/null+++ b/tests/integration/pipelex/temporal/tracing/test_dry_run_dispatches_and_mocks.py@@ -0,0 +1,103 @@+"""Mode-1 companion of Tier 17: DRY honors the Temporal backend (Part B, req 1).++The default-dry analogue of ``test_mock_usage_temporal.py``. Pre-Part-B, a Temporal dry run never+dispatched ``act_llm_gen_*`` (operators swapped in a workflow-side dry generator); after Part B,+``run_mode=DRY`` rides ``CogtRunParams`` across the serialization boundary, the workflow dispatches+the REAL ``act_llm_gen_text`` activity, and the cogt leaf mocks INSIDE the activity — zero-token+synthetic usage (cost report suppressed by design), no provider call.++The dispatch assertion reads the workflow history (``ActivityTaskScheduled`` across the parent and+its child workflows), which is the same signal the Mode-2 / Tier 17 e2e arm checks against a real+Temporal server. Lives under ``tracing/`` so the directory conftest applies both the CI quarantine+(in-process WorkflowEnvironment + child workflows hang under CI xdist) and the tmp-dir trace+redirect (usage assembly must not write to the configured traces backend).+"""++import uuid+from collections.abc import Generator+from typing import TYPE_CHECKING++import pytest+from temporalio.client import Client as TemporalClient++from pipelex.cogt.content_generation.dry_mock import DRY_RUN_INFERENCE_MODEL_NAME+from pipelex.cogt.usage.cost_registry import CostRegistry+from pipelex.pipe_run.pipe_job import PipeJob+from pipelex.pipe_run.pipe_run_mode import PipeRunMode+from pipelex.runtime_bridge.primitives.pipe_run_arg import PipeRunArg+from pipelex.temporal.temporal_hub import get_task_manager+from pipelex.temporal.tprl_content_generation.act_llm_generate import act_llm_gen_text+from pipelex.temporal.tprl_pipe.wf_pipe_run import WfPipeRun+from tests.integration.pipelex.fixtures.pipe_job_helpers import pipe_job_from_bundle+from tests.integration.pipelex.temporal.library_crate.helpers import rehydrate_pipe_output+from tests.integration.pipelex.temporal.tracing.helpers import collect_scheduled_activity_counts, inject_trace_context+from tests.integration.pipelex.temporal.tracing.test_data import SequenceTracingTestData++if TYPE_CHECKING:+ from pipelex.core.pipes.pipe_output import PipeOutput++# native_text_sequence is a two-step PipeSequence of PipeLLMs: each step must dispatch its own+# act_llm_gen_text and report its own (zero-token) usage — a single dispatch would mean one step+# silently bypassed the distribution machinery.+EXPECTED_LLM_DISPATCHES = 2++[email protected](loop_scope="class")+class TestTemporalDryRunDispatchesAndMocks:+ @pytest.fixture+ def dry_sequence_job(self, is_class_registry_isolated: bool) -> Generator[PipeJob, None, None]:+ """A DRY native_text_sequence job — same bundle as the mock-usage arm, default dry reporting."""+ yield from pipe_job_from_bundle(+ bundle_file=SequenceTracingTestData.BUNDLE_FILE,+ pipe_code=SequenceTracingTestData.PIPE_CODE,+ pipe_run_mode=PipeRunMode.DRY,+ isolated_registry=is_class_registry_isolated,+ )++ async def test_dry_run_dispatches_activities_and_mocks_inside(+ self,+ dry_sequence_job: PipeJob,+ temporal_client: TemporalClient,+ ) -> None:+ """DRY dispatches the real act_llm_gen_* and the leaf mocks inside, zero-token suppressed."""+ execution_run_id = f"dry_dispatch_{uuid.uuid4().hex[:12]}"+ execution_job = inject_trace_context(+ dry_sequence_job,+ execution_run_id,+ emit_graph_events=False,+ emit_usage_events=True,+ )+ assert execution_job.pipe_run_params.run_mode.is_dry, "inject_trace_context must preserve run_mode"++ pipe_run_arg = PipeRunArg(pipe_job=execution_job).prepare_for_temporal()+ task_queue = str(uuid.uuid4())+ workflow_id = str(uuid.uuid4())++ async with get_task_manager().make_worker(temporal_client, task_queue=task_queue, is_not_sandboxed=True):+ workflow_handle = await temporal_client.start_workflow( # pyright: ignore[reportUnknownMemberType]+ workflow=WfPipeRun.run,+ arg=pipe_run_arg,+ id=workflow_id,+ task_queue=task_queue,+ )+ pipe_output: PipeOutput = await workflow_handle.result()+ # The LLM activity is scheduled by the controller's CHILD workflows (one per step), so+ # count ActivityTaskScheduled events across the whole workflow tree.+ scheduled_counts = await collect_scheduled_activity_counts(temporal_client, workflow_id)++ # THE req-1 flip: the dry run DID dispatch the LLM activity, once per sequence step+ # (pre-Part-B it dispatched nothing inference-related).+ assert scheduled_counts[act_llm_gen_text.__name__] == EXPECTED_LLM_DISPATCHES++ rehydrate_pipe_output(pipe_output, pipe_job=dry_sequence_job)++ # The leaf mocked INSIDE the activity: the output is the DRY mock, not provider text.+ assert "DRY RUN:" in pipe_output.main_stuff_as_text.text++ # One zero-token usage per step under the dry_run sentinel — assembled back cross-process,+ # but the cost report stays suppressed (the inverse of is_mock_usage's reportable payload).+ assert pipe_output.tokens_usages is not None+ assert len(pipe_output.tokens_usages) == EXPECTED_LLM_DISPATCHES+ assert all(usage.inference_model_name == DRY_RUN_INFERENCE_MODEL_NAME for usage in pipe_output.tokens_usages)+ assert not CostRegistry.aggregate_costs(pipe_output.tokens_usages).has_reportable_usagediff --git a/tests/integration/pipelex/temporal/tracing/test_mock_inference_temporal.py b/tests/integration/pipelex/temporal/tracing/test_mock_usage_temporal.pysimilarity index 54%rename from tests/integration/pipelex/temporal/tracing/test_mock_inference_temporal.pyrename to tests/integration/pipelex/temporal/tracing/test_mock_usage_temporal.pyindex 558530492..91f3abe24 100644--- a/tests/integration/pipelex/temporal/tracing/test_mock_inference_temporal.py+++ b/tests/integration/pipelex/temporal/tracing/test_mock_usage_temporal.py@@ -1,15 +1,15 @@-"""Temporal (cross-process) coverage for ``--mock-inference`` (Phase 5).+"""Temporal (cross-process) coverage for the ``is_mock_usage`` dry sub-flag.The DIRECT counterpart lives in-``tests/integration/pipelex/pipeline/test_mock_inference_direct.py``. This arm proves the part DIRECT-cannot: that ``JobMetadata.is_mock_inference`` survives the Temporal serialization boundary and the-**real** ``act_llm_gen_text`` activity honors it — faking the LLM at the cogt leaf inside the activity-body (no provider call), with the synthetic, reportable usage assembling back onto-``PipeOutput.tokens_usages`` via Step 2's ``act_assemble_tracing``.+``tests/integration/pipelex/pipeline/test_mock_usage_direct.py``. This arm proves the part DIRECT+cannot: that ``CogtRunParams.is_mock_usage`` survives the Temporal serialization boundary and the+**real** ``act_llm_gen_text`` activity honors it — the dry leaf inside the activity body selects the+non-zero reporting payload (no provider call), with the synthetic, reportable usage assembling back+onto ``PipeOutput.tokens_usages`` via Step 2's ``act_assemble_tracing``.-Run LIVE (``PipeRunMode.LIVE``) so the workflow actually dispatches ``act_llm_gen_text`` (DRY would-swap ``ContentGeneratorDry`` in pre-dispatch and never reach the activity). The only thing faked is the-inference leaf, keyed on the per-run flag.+Run DRY (the flag is a sub-flag of DRY): the workflow still dispatches the real activity tree+(run mode is orthogonal to backend — Tier 17), and ``is_mock_usage`` flips the leaf reporting from+the suppressed zero-token payload to the reportable non-zero one.Like its sibling tracing tests this is ``gha_disabled`` by the directory conftest (the in-processWorkflowEnvironment + concurrent pipe execution hangs under CI xdist); verify locally/serially.@@ -22,7 +22,7 @@ from typing import TYPE_CHECKINGimport pytestfrom temporalio.client import Client as TemporalClient-from pipelex.cogt.content_generation.dry_mock import MOCK_INFERENCE_MODEL_NAME+from pipelex.cogt.content_generation.dry_mock import MOCK_USAGE_MODEL_NAMEfrom pipelex.cogt.usage.cost_registry import CostRegistryfrom pipelex.pipe_run.pipe_job import PipeJobfrom pipelex.pipe_run.pipe_run_mode import PipeRunMode@@ -40,40 +40,39 @@ if TYPE_CHECKING:@pytest.mark.temporal@pytest.mark.asyncio(loop_scope="class")-class TestTemporalMockInference:+class TestTemporalMockUsage:@pytest.fixture- def mock_inference_sequence_job(self, is_class_registry_isolated: bool) -> Generator[PipeJob, None, None]:- """A LIVE native_text_sequence job carrying is_mock_inference=True.+ def mock_usage_sequence_job(self, is_class_registry_isolated: bool) -> Generator[PipeJob, None, None]:+ """A DRY native_text_sequence job carrying is_mock_usage=True.- LIVE forces the workflow to dispatch the real ``act_llm_gen_text``; the flag makes that- activity's leaf fake the LLM. The same bundle is run LIVE (with a fake activity substitute)- by ``test_split_worker_usage`` — here we run the *real* activity and let the flag do the mocking.+ DRY still dispatches the real ``act_llm_gen_text`` (run mode orthogonal to backend); the+ flag makes that activity's leaf report non-zero synthetic usage instead of the suppressed+ zero-token payload."""for pipe_job in pipe_job_from_bundle(bundle_file=SequenceTracingTestData.BUNDLE_FILE,pipe_code=SequenceTracingTestData.PIPE_CODE,- pipe_run_mode=PipeRunMode.LIVE,+ pipe_run_mode=PipeRunMode.DRY,isolated_registry=is_class_registry_isolated,):- yield pipe_job.model_copy(- update={"job_metadata": pipe_job.job_metadata.model_copy(update={"is_mock_inference": True})},- )+ mocked_run_params = pipe_job.pipe_run_params.model_copy(update={"is_mock_usage": True})+ yield pipe_job.model_copy(update={"pipe_run_params": mocked_run_params})- async def test_mock_inference_assembles_reportable_usage_cross_process(+ async def test_mock_usage_assembles_reportable_usage_cross_process(self,- mock_inference_sequence_job: PipeJob,+ mock_usage_sequence_job: PipeJob,temporal_client: TemporalClient,) -> None:- """The flag crosses the boundary, the real activity mocks the leaf, and reportable usage rides back."""- execution_run_id = f"mock_inf_{uuid.uuid4().hex[:12]}"+ """The flag crosses the boundary, the real activity honors it, and reportable usage rides back."""+ execution_run_id = f"mock_usage_{uuid.uuid4().hex[:12]}"# costs-only (no graph) mirrors the realistic --no-graph --costs default the skill validates.execution_job = inject_trace_context(- mock_inference_sequence_job,+ mock_usage_sequence_job,execution_run_id,emit_graph_events=False,emit_usage_events=True,)- assert execution_job.job_metadata.is_mock_inference, "inject_trace_context must preserve is_mock_inference"+ assert execution_job.pipe_run_params.cogt_run_params.is_mock_usage, "inject_trace_context must preserve is_mock_usage"pipe_run_arg = PipeRunArg(pipe_job=execution_job).prepare_for_temporal()task_queue = str(uuid.uuid4())@@ -86,11 +85,11 @@ class TestTemporalMockInference:id=workflow_id,task_queue=task_queue,)- rehydrate_pipe_output(pipe_output, pipe_job=mock_inference_sequence_job)+ rehydrate_pipe_output(pipe_output, pipe_job=mock_usage_sequence_job)assert pipe_output.tokens_usages is not Noneassert len(pipe_output.tokens_usages) >= 1# All usage came from the mocked leaf (the sentinel model) — proof no real provider ran.- assert all(usage.inference_model_name == MOCK_INFERENCE_MODEL_NAME for usage in pipe_output.tokens_usages)- # Non-zero synthetic usage -> a cost report would render (not suppressed like a dry run).+ assert all(usage.inference_model_name == MOCK_USAGE_MODEL_NAME for usage in pipe_output.tokens_usages)+ # Non-zero synthetic usage -> a cost report would render (not suppressed like a default dry run).assert CostRegistry.aggregate_costs(pipe_output.tokens_usages).has_reportable_usagediff --git a/tests/integration/pipelex/temporal/tracing/test_split_worker_usage.py b/tests/integration/pipelex/temporal/tracing/test_split_worker_usage.pyindex e9fe34235..1560f27f5 100644--- a/tests/integration/pipelex/temporal/tracing/test_split_worker_usage.py+++ b/tests/integration/pipelex/temporal/tracing/test_split_worker_usage.py@@ -69,12 +69,11 @@ class TestSplitWorkerUsageEmission:def live_sequence_tracing_job(self, is_class_registry_isolated: bool) -> Generator[PipeJob, None, None]:"""Like the class-scoped `sequence_tracing_job` but in LIVE mode.- DRY mode short-circuits the `act_llm_gen_text` activity dispatch- (`ContentGeneratorDry` reports inline inside the workflow), so the- cross-worker activity hop never fires. LIVE mode forces the workflow- to dispatch `act_llm_gen_text` directly via- `ContentGeneratorInWorkflow.make_llm_text`, which is what the- runner-side fallback pinned in this suite is supposed to exercise.+ Since Part B, DRY also dispatches `act_llm_gen_text` (the leaf mocks+ inside the activity) — but its synthetic usage is zero-token and+ suppressed by design, so the *reportable*-usage assertions here need+ LIVE mode, where the fake activity substitute synthesizes non-zero+ usage on the runner side.The fake `act_llm_gen_text` substitute installed by`make_split_workers` returns a stub string and synthesizes theusage report, so no real LLM call is made.diff --git a/tests/unit/pipelex/cli/test_mock_inference_cli_guard.py b/tests/unit/pipelex/cli/test_mock_usage_cli_guard.pysimilarity index 51%rename from tests/unit/pipelex/cli/test_mock_inference_cli_guard.pyrename to tests/unit/pipelex/cli/test_mock_usage_cli_guard.pyindex b292b23ef..03d3f0ac6 100644--- a/tests/unit/pipelex/cli/test_mock_inference_cli_guard.py+++ b/tests/unit/pipelex/cli/test_mock_usage_cli_guard.py@@ -1,8 +1,8 @@-"""Unit tests for the ``--mock-inference`` CLI surface on the run subcommands.+"""Unit tests for the hidden ``--mock-usage`` CLI surface on the run subcommands.-Two cheap, no-boot checks: every run subcommand declares the option, and each rejects combining-``--mock-inference`` with ``--dry-run`` (the two are mutually exclusive — dry-run swaps the generator-pre-dispatch, so the leaf mock would be silently ignored). The guard runs at the top of each command,+Two cheap, no-boot checks: every run subcommand declares the option (hidden — for the+``temporal-e2e-validate`` Mode-2 scripts, not for ``--help``), and each rejects ``--mock-usage``+without ``--dry-run`` (it is a sub-flag of the dry run). The guard runs at the top of each command,before any filesystem access or Pipelex boot, so calling the command function directly is enough."""@@ -17,10 +17,10 @@ from pipelex.cli.commands.run.method_cmd import run_method_cmdfrom pipelex.cli.commands.run.pipe_cmd import run_pipe_cmd-class TestMockInferenceCliGuard:+class TestMockUsageCliGuard:@pytest.mark.parametrize("command", [run_pipe_cmd, run_bundle_cmd, run_method_cmd])- def test_run_command_declares_mock_inference(self, command: Callable[..., None]) -> None:- assert "mock_inference" in signature(command).parameters, f"{command.__name__} must declare a --mock-inference option"+ def test_run_command_declares_mock_usage(self, command: Callable[..., None]) -> None:+ assert "mock_usage" in signature(command).parameters, f"{command.__name__} must declare a --mock-usage option"@pytest.mark.parametrize(("command", "required_kwargs"),@@ -30,6 +30,6 @@ class TestMockInferenceCliGuard:(run_method_cmd, {"name": "some_method"}),],)- def test_rejects_mock_inference_combined_with_dry_run(self, command: Callable[..., None], required_kwargs: dict[str, str]) -> None:+ def test_rejects_mock_usage_without_dry_run(self, command: Callable[..., None], required_kwargs: dict[str, str]) -> None:with pytest.raises(typer.Exit):- command(dry_run=True, mock_inference=True, **required_kwargs)+ command(dry_run=False, mock_usage=True, **required_kwargs)diff --git a/tests/unit/pipelex/cli/test_run_flag_combination.py b/tests/unit/pipelex/cli/test_run_flag_combination.pyindex f35c546d1..25b1d68e7 100644--- a/tests/unit/pipelex/cli/test_run_flag_combination.py+++ b/tests/unit/pipelex/cli/test_run_flag_combination.py@@ -1,8 +1,8 @@"""Unit tests for ``validate_run_flag_combination`` — the single owner of which run-flag combinations are legal.-The ``pipe`` / ``method`` / ``bundle`` run subcommands all delegate their ``--dry-run`` / ``--mock-inference`` /+The ``pipe`` / ``method`` / ``bundle`` run subcommands all delegate their ``--dry-run`` / ``--mock-usage`` /``--mock-inputs`` guarding to this one function, so the three can't drift (review F3). The CLI-level wiring —-that each subcommand actually calls it — is pinned separately in ``test_mock_inference_cli_guard.py``; here we+that each subcommand actually calls it — is pinned separately in ``test_mock_usage_cli_guard.py``; here wepin the full truth table of the validator itself."""@@ -15,21 +15,22 @@ from pipelex.cli.commands.run._run_core import validate_run_flag_combination #class TestRunFlagCombination:def test_mock_inputs_without_dry_run_is_rejected(self) -> None:with pytest.raises(typer.Exit):- validate_run_flag_combination(dry_run=False, mock_inference=False, mock_inputs=True)+ validate_run_flag_combination(dry_run=False, mock_usage=False, mock_inputs=True)- def test_mock_inference_with_dry_run_is_rejected(self) -> None:+ def test_mock_usage_without_dry_run_is_rejected(self) -> None:with pytest.raises(typer.Exit):- validate_run_flag_combination(dry_run=True, mock_inference=True, mock_inputs=False)+ validate_run_flag_combination(dry_run=False, mock_usage=True, mock_inputs=False)@pytest.mark.parametrize(- ("dry_run", "mock_inference", "mock_inputs"),+ ("dry_run", "mock_usage", "mock_inputs"),[(False, False, False), # plain live run(True, False, False), # plain dry run- (False, True, False), # mock-inference (live)+ (True, True, False), # dry run with reportable mock usage(True, False, True), # dry run with mock inputs+ (True, True, True), # dry run with both sub-flags],)- def test_legal_combinations_pass(self, dry_run: bool, mock_inference: bool, mock_inputs: bool) -> None:+ def test_legal_combinations_pass(self, dry_run: bool, mock_usage: bool, mock_inputs: bool) -> None:# No raise == legal (the function returns None and exits only on an illegal combination).- validate_run_flag_combination(dry_run=dry_run, mock_inference=mock_inference, mock_inputs=mock_inputs)+ validate_run_flag_combination(dry_run=dry_run, mock_usage=mock_usage, mock_inputs=mock_inputs)diff --git a/tests/unit/pipelex/cli/test_traceback_flag_run_core.py b/tests/unit/pipelex/cli/test_traceback_flag_run_core.pyindex 50f341666..a3a5bc29a 100644--- a/tests/unit/pipelex/cli/test_traceback_flag_run_core.py+++ b/tests/unit/pipelex/cli/test_traceback_flag_run_core.py@@ -83,7 +83,7 @@ class TestTracebackFlagInExecuteRunInnerExcepts:graph_full_data=None,output_dir=OUTPUT_DIR,dry_run=False,- mock_inference=False,+ mock_usage=False,mock_inputs=False,library_dir=None,)@@ -113,7 +113,7 @@ class TestTracebackFlagInExecuteRunInnerExcepts:graph_full_data=None,output_dir=OUTPUT_DIR,dry_run=False,- mock_inference=False,+ mock_usage=False,mock_inputs=False,library_dir=None,)@@ -142,7 +142,7 @@ class TestTracebackFlagInExecuteRunInnerExcepts:graph_full_data=None,output_dir=OUTPUT_DIR,dry_run=False,- mock_inference=False,+ mock_usage=False,mock_inputs=False,library_dir=None,)@@ -171,7 +171,7 @@ class TestTracebackFlagInExecuteRunInnerExcepts:graph_full_data=None,output_dir=OUTPUT_DIR,dry_run=False,- mock_inference=False,+ mock_usage=False,mock_inputs=False,library_dir=None,)diff --git a/tests/unit/pipelex/cogt/content_generation/test_assignment_models_schema.py b/tests/unit/pipelex/cogt/content_generation/test_assignment_models_schema.pyindex d422fbe94..0acb75055 100644--- a/tests/unit/pipelex/cogt/content_generation/test_assignment_models_schema.py+++ b/tests/unit/pipelex/cogt/content_generation/test_assignment_models_schema.py@@ -3,8 +3,10 @@from pydantic import BaseModelfrom pipelex.cogt.content_generation.assignment_models import LLMAssignment, ObjectAssignment+from pipelex.cogt.content_generation.cogt_run_params import CogtRunParamsfrom pipelex.cogt.llm.llm_prompt import LLMPromptfrom pipelex.cogt.llm.llm_setting import LLMSetting+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipeline.job_metadata import JobMetadata@@ -23,6 +25,7 @@ def _make_stub_job_metadata() -> JobMetadata:def _make_stub_llm_assignment() -> LLMAssignment:return LLMAssignment(job_metadata=_make_stub_job_metadata(),+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),llm_setting=LLMSetting(model="test-model", temperature=0.7),llm_prompt=LLMPrompt(user_text="test prompt"),)diff --git a/tests/unit/pipelex/cogt/content_generation/test_assignment_models_security.py b/tests/unit/pipelex/cogt/content_generation/test_assignment_models_security.pyindex 8589eaa0c..3cb0c3a84 100644--- a/tests/unit/pipelex/cogt/content_generation/test_assignment_models_security.py+++ b/tests/unit/pipelex/cogt/content_generation/test_assignment_models_security.py@@ -8,10 +8,12 @@ from typing import Anyimport pytestfrom pipelex.cogt.content_generation.assignment_models import LLMAssignment, ObjectAssignment+from pipelex.cogt.content_generation.cogt_run_params import CogtRunParamsfrom pipelex.cogt.content_generation.exceptions import UnsafeSchemaErrorfrom pipelex.cogt.content_generation.schema_to_model_factory import SchemaToModelFactoryfrom pipelex.cogt.llm.llm_prompt import LLMPromptfrom pipelex.cogt.llm.llm_setting import LLMSetting+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipeline.job_metadata import JobMetadata@@ -25,6 +27,7 @@ def _make_stub_job_metadata() -> JobMetadata:def _make_stub_llm_assignment() -> LLMAssignment:return LLMAssignment(job_metadata=_make_stub_job_metadata(),+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),llm_setting=LLMSetting(model="test-model", temperature=0.7),llm_prompt=LLMPrompt(user_text="test prompt"),)diff --git a/tests/unit/pipelex/cogt/content_generation/test_content_generator_dry_reporting.py b/tests/unit/pipelex/cogt/content_generation/test_content_generator_dry_reporting.pydeleted file mode 100644index f85466e01..000000000--- a/tests/unit/pipelex/cogt/content_generation/test_content_generator_dry_reporting.py+++ /dev/null@@ -1,57 +0,0 @@-"""Pin that ContentGeneratorDry invokes report_inference_job with a synthetic LLMJob.--This is a Phase-2 prerequisite for the Phase-6 e2e cross-worker emission tier:-without the dry generator emitting a UsageReportEvent, the Tier 8 assertions in-.claude/skills/temporal-e2e-validate/SKILL.md cannot run dry — they would have-to require live LLM calls to observe the runner-side fallback.-"""--import pytest-from pytest_mock import MockerFixture--from pipelex.cogt.content_generation.content_generator_dry import ContentGeneratorDry-from pipelex.cogt.llm.llm_job import LLMJob-from pipelex.cogt.llm.llm_prompt import LLMPrompt-from pipelex.cogt.llm.llm_setting import LLMSetting-from pipelex.pipeline.job_metadata import JobMetadata---class TestContentGeneratorDryReporting:- """Pin the ContentGeneratorDry → ReportingManager.report_inference_job hook."""-- @pytest.mark.asyncio- async def test_make_llm_text_invokes_report_inference_job(self, mocker: MockerFixture) -> None:- """make_llm_text builds a synthetic LLMJob and reports it through get_report_delegate()."""- captured_jobs: list[LLMJob] = []-- def _capture(inference_job: LLMJob) -> None:- captured_jobs.append(inference_job)-- mock_delegate = mocker.MagicMock()- mock_delegate.report_inference_job.side_effect = _capture- # ContentGeneratorDry now delegates synthetic-job reporting to dry_mock.report_dry_llm_job,- # so patch get_report_delegate where it is looked up at call time (dry_mock, not this module).- mocker.patch(- "pipelex.cogt.content_generation.dry_mock.get_report_delegate",- return_value=mock_delegate,- )-- generator = ContentGeneratorDry()- job_metadata = JobMetadata(- user_id="test_user",- pipeline_run_id="run_dry_001",- )- llm_setting = LLMSetting(model="gpt-4o", temperature=0.5)- llm_prompt = LLMPrompt()-- await generator.make_llm_text(- job_metadata=job_metadata,- llm_setting_main=llm_setting,- llm_prompt_for_text=llm_prompt,- )-- assert len(captured_jobs) == 1- reported_job = captured_jobs[0]- assert isinstance(reported_job, LLMJob)- assert reported_job.job_metadata.pipeline_run_id == "run_dry_001"- assert reported_job.job_report.llm_tokens_usage is not Nonediff --git a/tests/unit/pipelex/cogt/content_generation/test_dry_mock.py b/tests/unit/pipelex/cogt/content_generation/test_dry_mock.pyindex 46d294ea1..518c8a2fb 100644--- a/tests/unit/pipelex/cogt/content_generation/test_dry_mock.py+++ b/tests/unit/pipelex/cogt/content_generation/test_dry_mock.py@@ -1,29 +1,44 @@"""Unit tests for the shared leaf-level mock helpers (dry_mock).-The load-bearing contract: a ``--mock-inference`` synthetic LLM job carries *non-zero* tokens so it is-reportable (``AggregatedCosts.has_reportable_usage`` True → a cost report renders), while a ``--dry-run``-synthetic job carries zero tokens and stays non-reportable (the report is suppressed). This is the durable-reason the two modes diverge at the reporting layer.+The load-bearing contract: an ``is_mock_usage`` synthetic LLM job carries *non-zero* tokens so it is+reportable (``AggregatedCosts.has_reportable_usage`` True → a cost report renders), while the default+dry synthetic job carries zero tokens and stays non-reportable (the report is suppressed). This is the+durable reason the sub-flag exists at the reporting layer."""import pytestfrom pytest_mock import MockerFixture-from pipelex.cogt.content_generation.assignment_models import LLMAssignment+from pipelex.cogt.content_generation.assignment_models import LLMAssignment, ObjectAssignment+from pipelex.cogt.content_generation.cogt_run_params import CogtRunParamsfrom pipelex.cogt.content_generation.dry_mock import (DRY_RUN_INFERENCE_MODEL_NAME,- MOCK_INFERENCE_MODEL_NAME,- mock_llm_gen_text,+ MOCK_USAGE_MODEL_NAME,+ dry_llm_gen_object,+ dry_llm_gen_object_list,report_dry_llm_job,- report_mock_inference_llm_job,+ report_mock_usage_llm_job,)+from pipelex.cogt.content_generation.exceptions import DryRunMockBuildErrorfrom pipelex.cogt.llm.llm_job import LLMJobfrom pipelex.cogt.llm.llm_prompt import LLMPromptfrom pipelex.cogt.llm.llm_setting import LLMSettingfrom pipelex.cogt.usage.cost_registry import CostRegistry+from pipelex.core.stuffs.text_content import TextContent+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipeline.job_metadata import JobMetadata+def _dry_object_assignment() -> ObjectAssignment:+ llm_assignment = LLMAssignment(+ job_metadata=JobMetadata(user_id="u", pipeline_run_id="run_dry_mock"),+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.DRY),+ llm_setting=LLMSetting(model="gpt-4o", temperature=0.5),+ llm_prompt=LLMPrompt(),+ )+ return ObjectAssignment.make_for_class(object_class=TextContent, llm_assignment=llm_assignment)++class TestDryMock:def _capture_reported_job(self, mocker: MockerFixture) -> list[LLMJob]:"""Patch get_report_delegate in dry_mock and capture every reported LLMJob."""@@ -37,10 +52,10 @@ class TestDryMock:mocker.patch("pipelex.cogt.content_generation.dry_mock.get_report_delegate", return_value=mock_delegate)return captured- def test_mock_inference_job_is_reportable_non_zero(self, mocker: MockerFixture) -> None:- """report_mock_inference_llm_job emits non-zero usage under the mock_inference sentinel -> reportable."""+ def test_mock_usage_job_is_reportable_non_zero(self, mocker: MockerFixture) -> None:+ """report_mock_usage_llm_job emits non-zero usage under the mock_usage sentinel -> reportable."""captured = self._capture_reported_job(mocker)- report_mock_inference_llm_job(+ report_mock_usage_llm_job(job_metadata=JobMetadata(user_id="u", pipeline_run_id="run_mock"),llm_setting=LLMSetting(model="gpt-4o", temperature=0.5),llm_prompt=LLMPrompt(),@@ -49,7 +64,7 @@ class TestDryMock:assert len(captured) == 1usage = captured[0].job_report.llm_tokens_usageassert usage is not None- assert usage.inference_model_name == MOCK_INFERENCE_MODEL_NAME+ assert usage.inference_model_name == MOCK_USAGE_MODEL_NAMEaggregated = CostRegistry.aggregate_costs([usage])assert aggregated.has_reportable_usage # the whole point: a cost report will renderassert aggregated.total_nb_tokens > 0@@ -71,18 +86,30 @@ class TestDryMock:assert not aggregated.has_reportable_usage # dry runs are deliberately suppressedassert aggregated.total_nb_tokens == 0- @pytest.mark.asyncio- async def test_mock_llm_gen_text_returns_synthetic_text_and_reports(self, mocker: MockerFixture) -> None:- """mock_llm_gen_text returns a synthetic string and reports one reportable usage event."""+ def test_failed_dry_object_build_reports_no_usage(self, mocker: MockerFixture) -> None:+ """A failed mock build must not leave a usage event behind (report only after success)."""captured = self._capture_reported_job(mocker)- assignment = LLMAssignment(- job_metadata=JobMetadata(user_id="u", pipeline_run_id="run_text", is_mock_inference=True),- llm_setting=LLMSetting(model="gpt-4o", temperature=0.5),- llm_prompt=LLMPrompt(),+ mocker.patch("pipelex.cogt.content_generation.dry_mock._reconstruct_object_class", return_value=TextContent)+ mocker.patch(+ "pipelex.cogt.content_generation.dry_mock.build_mock_object",+ side_effect=DryRunMockBuildError.for_object_class("TextContent"),)- result = mock_llm_gen_text(assignment)+ with pytest.raises(DryRunMockBuildError):+ dry_llm_gen_object(_dry_object_assignment())- assert result.startswith("MOCK INFERENCE")- assert len(captured) == 1- assert captured[0].job_report.llm_tokens_usage is not None+ assert captured == []++ def test_failed_dry_object_list_build_reports_no_usage(self, mocker: MockerFixture) -> None:+ """The list leaf carries the same report-only-after-success contract."""+ captured = self._capture_reported_job(mocker)+ mocker.patch("pipelex.cogt.content_generation.dry_mock._reconstruct_object_class", return_value=TextContent)+ mocker.patch(+ "pipelex.cogt.content_generation.dry_mock.build_mock_objects",+ side_effect=DryRunMockBuildError.for_object_class("TextContent"),+ )++ with pytest.raises(DryRunMockBuildError):+ dry_llm_gen_object_list(_dry_object_assignment())++ assert captured == []diff --git a/tests/unit/pipelex/cogt/content_generation/test_content_generator_dry_tz.py b/tests/unit/pipelex/cogt/content_generation/test_dry_report_tz.pysimilarity index 71%rename from tests/unit/pipelex/cogt/content_generation/test_content_generator_dry_tz.pyrename to tests/unit/pipelex/cogt/content_generation/test_dry_report_tz.pyindex 9246c21d0..89f954d2b 100644--- a/tests/unit/pipelex/cogt/content_generation/test_content_generator_dry_tz.py+++ b/tests/unit/pipelex/cogt/content_generation/test_dry_report_tz.py@@ -1,4 +1,4 @@-"""Pin that the dry generator copes with both naive and aware ``started_at``.+"""Pin that the dry synthetic-job reporting copes with both naive and aware ``started_at``.The synthetic ``completed_at`` must share the tzinfo of the incoming``started_at`` so ``JobMetadata.duration`` can subtract them without crossing@@ -12,14 +12,14 @@ from datetime import datetime, timezoneimport pytestfrom pytest_mock import MockerFixture-from pipelex.cogt.content_generation.content_generator_dry import ContentGeneratorDry+from pipelex.cogt.content_generation.dry_mock import report_dry_llm_jobfrom pipelex.cogt.llm.llm_job import LLMJobfrom pipelex.cogt.llm.llm_prompt import LLMPromptfrom pipelex.cogt.llm.llm_setting import LLMSettingfrom pipelex.pipeline.job_metadata import JobMetadata-class TestContentGeneratorDryTimezone:+class TestDryReportTimezone:@pytest.mark.parametrize("started_at",[@@ -28,8 +28,7 @@ class TestContentGeneratorDryTimezone:pytest.param(None, id="none_uses_default_factory"),],)- @pytest.mark.asyncio- async def test_duration_is_finite_for_any_started_at_tz(+ def test_duration_is_finite_for_any_started_at_tz(self,started_at: datetime | None,mocker: MockerFixture,@@ -41,26 +40,22 @@ class TestContentGeneratorDryTimezone:mock_delegate = mocker.MagicMock()mock_delegate.report_inference_job.side_effect = _capture- # The tz-handling logic now lives in dry_mock; ContentGeneratorDry delegates to it, so patch- # get_report_delegate where it is looked up at call time (dry_mock, not this module).+ # Patch get_report_delegate where it is looked up at call time (dry_mock).mocker.patch("pipelex.cogt.content_generation.dry_mock.get_report_delegate",return_value=mock_delegate,)- generator = ContentGeneratorDry()job_metadata = JobMetadata(user_id="test_user",pipeline_run_id="run_dry_tz",started_at=started_at,)- llm_setting = LLMSetting(model="gpt-4o", temperature=0.5)- llm_prompt = LLMPrompt()- await generator.make_llm_text(+ report_dry_llm_job(job_metadata=job_metadata,- llm_setting_main=llm_setting,- llm_prompt_for_text=llm_prompt,+ llm_setting=LLMSetting(model="gpt-4o", temperature=0.5),+ llm_prompt=LLMPrompt(),)assert len(captured_jobs) == 1diff --git a/tests/unit/pipelex/cogt/content_generation/test_dry_run_mock_build_error.py b/tests/unit/pipelex/cogt/content_generation/test_dry_run_mock_build_error.pynew file mode 100644index 000000000..17998dc51--- /dev/null+++ b/tests/unit/pipelex/cogt/content_generation/test_dry_run_mock_build_error.py@@ -0,0 +1,58 @@+"""Pin the typed terminal mock-build error (eng review D7).++``build_mock_object`` wraps deterministic polyfactory/validation failures into+``DryRunMockBuildError`` (a ``PipelexError``), so the activity error boundary converts it to a+terminal failure instead of letting Temporal retry a failure that can never succeed. The message+must name the class and the ``examples`` / ``mock_format`` remedy.+"""++import pytest+from pydantic import field_validator++from pipelex.base_exceptions import PipelexError+from pipelex.cogt.content_generation.dry_mock import build_mock_object, build_mock_objects+from pipelex.cogt.content_generation.exceptions import DryRunMockBuildError+from pipelex.core.stuffs.structured_content import StructuredContent+++class UnbuildableName(StructuredContent):+ """polyfactory cannot guess a value passing this validator without `examples` / `mock_format`."""++ name: str++ @field_validator("name")+ @classmethod+ def _require_exact(cls, value: str) -> str:+ if value != "the-one-true-name":+ msg = "name must be exactly 'the-one-true-name'"+ raise ValueError(msg)+ return value+++class BuildableName(StructuredContent):+ """Control: a plain string field builds fine."""++ name: str+++class TestDryRunMockBuildError:+ def test_unbuildable_class_raises_typed_terminal_error(self) -> None:+ """A deterministic build failure surfaces as DryRunMockBuildError naming class and remedy."""+ with pytest.raises(DryRunMockBuildError) as exc_info:+ build_mock_object(UnbuildableName)++ assert isinstance(exc_info.value, PipelexError)+ assert UnbuildableName.__name__ in str(exc_info.value)+ assert "mock_format" in str(exc_info.value)++ def test_unbuildable_class_list_variant_raises_same_typed_error(self) -> None:+ """The list builder carries the same terminal-error contract as the single builder."""+ with pytest.raises(DryRunMockBuildError):+ build_mock_objects(UnbuildableName, 2)++ def test_buildable_class_builds(self) -> None:+ """Control arm: an unconstrained class builds a valid instance."""+ mock_object = build_mock_object(BuildableName)++ assert isinstance(mock_object, BuildableName)+ assert isinstance(mock_object.name, str)diff --git a/tests/unit/pipelex/cogt/content_generation/test_extract_generate_dry_branch.py b/tests/unit/pipelex/cogt/content_generation/test_extract_generate_dry_branch.pynew file mode 100644index 000000000..d5bd39597--- /dev/null+++ b/tests/unit/pipelex/cogt/content_generation/test_extract_generate_dry_branch.py@@ -0,0 +1,81 @@+"""Unit tests for the ``run_mode == DRY`` branch in the extract leaf (extract_generate).++The DRY branch sits at the ``*_and_store`` layer, ABOVE the raw provider leaf (eng review D10):+a dry run must perform no provider call and no storage IO. Contract: under DRY the worker and the+``GeneratedContentFactory`` are never touched, a document mocks ``nb_extract_pages`` synthetic+pages while an image mocks exactly one, and LIVE keeps the real extract-and-store path.+"""++import pytest+from pytest_mock import MockerFixture++from pipelex.cogt.content_generation.assignment_models import ExtractAssignment+from pipelex.cogt.content_generation.cogt_run_params import CogtRunParams+from pipelex.cogt.content_generation.extract_generate import extract_gen_pages_and_store+from pipelex.cogt.extract.extract_input import ExtractInput+from pipelex.cogt.extract.extract_job_components import ExtractJobConfig, ExtractJobParams+from pipelex.config import get_config+from pipelex.pipe_run.pipe_run_mode import PipeRunMode+from pipelex.pipeline.job_metadata import JobMetadata+++class TestExtractGenerateDryBranch:+ def _assignment(self, *, run_mode: PipeRunMode, extract_input: ExtractInput) -> ExtractAssignment:+ return ExtractAssignment(+ job_metadata=JobMetadata(user_id="u", pipeline_run_id="run_extract_dry"),+ cogt_run_params=CogtRunParams(run_mode=run_mode),+ extract_handle="mock-extract-handle",+ extract_input=extract_input,+ extract_job_params=ExtractJobParams.make_default_extract_job_params(),+ extract_job_config=ExtractJobConfig(),+ )++ @pytest.mark.asyncio+ async def test_dry_document_mocks_pages_without_io(self, mocker: MockerFixture) -> None:+ """DRY on a document: nb_extract_pages synthetic pages, no provider call, no storage IO."""+ worker_spy = mocker.patch("pipelex.cogt.content_generation.extract_generate.get_extract_worker")+ factory = mocker.MagicMock()++ page_contents = await extract_gen_pages_and_store(+ extract_assignment=self._assignment(run_mode=PipeRunMode.DRY, extract_input=ExtractInput(document_uri="file:///tmp/doc.pdf")),+ generated_content_factory=factory,+ )++ worker_spy.assert_not_called()+ factory.make_page_contents.assert_not_called()+ assert len(page_contents) == get_config().pipelex.dry_run_config.nb_extract_pages+ for page_content in page_contents:+ assert page_content.text_and_images.text is not None+ assert page_content.text_and_images.text.text.startswith("DRY RUN:")++ @pytest.mark.asyncio+ async def test_dry_image_mocks_single_page(self, mocker: MockerFixture) -> None:+ """DRY on an image input mocks exactly one page."""+ mocker.patch("pipelex.cogt.content_generation.extract_generate.get_extract_worker")+ factory = mocker.MagicMock()++ page_contents = await extract_gen_pages_and_store(+ extract_assignment=self._assignment(run_mode=PipeRunMode.DRY, extract_input=ExtractInput(image_uri="file:///tmp/scan.png")),+ generated_content_factory=factory,+ )++ assert len(page_contents) == 1++ @pytest.mark.asyncio+ async def test_live_runs_provider_and_stores(self, mocker: MockerFixture) -> None:+ """LIVE keeps the real path: provider extracts, factory stores."""+ sentinel_output = mocker.MagicMock()+ worker = mocker.MagicMock()+ worker.extract_pages = mocker.AsyncMock(return_value=sentinel_output)+ mocker.patch("pipelex.cogt.content_generation.extract_generate.get_extract_worker", return_value=worker)+ mocker.patch("pipelex.cogt.content_generation.extract_generate.ExtractJobFactory.make_extract_job", return_value=mocker.MagicMock())+ factory = mocker.MagicMock()+ factory.make_page_contents = mocker.AsyncMock(return_value=[])++ await extract_gen_pages_and_store(+ extract_assignment=self._assignment(run_mode=PipeRunMode.LIVE, extract_input=ExtractInput(document_uri="file:///tmp/doc.pdf")),+ generated_content_factory=factory,+ )++ worker.extract_pages.assert_awaited_once()+ factory.make_page_contents.assert_awaited_once()diff --git a/tests/unit/pipelex/cogt/content_generation/test_extract_generate_mock_guard.py b/tests/unit/pipelex/cogt/content_generation/test_extract_generate_mock_guard.pydeleted file mode 100644index 4b6be31b4..000000000--- a/tests/unit/pipelex/cogt/content_generation/test_extract_generate_mock_guard.py+++ /dev/null@@ -1,58 +0,0 @@-"""Unit tests for the ``is_mock_inference`` hard guard in the extract leaf (extract_generate).--``--mock-inference`` has no leaf-level mock for document extraction, so reaching the extract leaf under-the flag would dispatch to the real provider and spend. The leaf must fail loud instead: raise-``MockInferenceUnsupportedError`` before ``get_extract_worker`` is ever called. With the flag off, the-real worker path runs unchanged — proving the guard is keyed strictly on ``job_metadata.is_mock_inference``.-"""--import pytest-from pytest_mock import MockerFixture--from pipelex.cogt.content_generation.assignment_models import ExtractAssignment-from pipelex.cogt.content_generation.exceptions import MockInferenceUnsupportedError-from pipelex.cogt.content_generation.extract_generate import extract_gen_pages-from pipelex.cogt.extract.extract_input import ExtractInput-from pipelex.cogt.extract.extract_job_components import ExtractJobConfig, ExtractJobParams-from pipelex.pipeline.job_metadata import JobMetadata---class TestExtractGenerateMockGuard:- def _assignment(self, *, is_mock_inference: bool) -> ExtractAssignment:- return ExtractAssignment(- job_metadata=JobMetadata(user_id="u", pipeline_run_id="run_extract_guard", is_mock_inference=is_mock_inference),- extract_handle="mock-extract-handle",- extract_input=ExtractInput(document_uri="file:///tmp/doc.pdf"),- extract_job_params=ExtractJobParams(- max_nb_images=None,- image_min_size=None,- should_caption_images=False,- should_include_page_views=False,- page_views_dpi=None,- ),- extract_job_config=ExtractJobConfig(),- )-- @pytest.mark.asyncio- async def test_mock_flag_raises_and_skips_worker(self, mocker: MockerFixture) -> None:- """is_mock_inference=True -> the leaf raises before any provider call (get_extract_worker untouched)."""- worker_spy = mocker.patch("pipelex.cogt.content_generation.extract_generate.get_extract_worker")-- with pytest.raises(MockInferenceUnsupportedError):- await extract_gen_pages(self._assignment(is_mock_inference=True))-- worker_spy.assert_not_called() # no provider call -> no spend-- @pytest.mark.asyncio- async def test_no_flag_uses_real_worker(self, mocker: MockerFixture) -> None:- """is_mock_inference=False -> the real worker path runs (get_extract_worker is called)."""- sentinel = mocker.MagicMock()- worker = mocker.MagicMock()- worker.extract_pages = mocker.AsyncMock(return_value=sentinel)- worker_spy = mocker.patch("pipelex.cogt.content_generation.extract_generate.get_extract_worker", return_value=worker)- mocker.patch("pipelex.cogt.content_generation.extract_generate.ExtractJobFactory.make_extract_job", return_value=mocker.MagicMock())-- result = await extract_gen_pages(self._assignment(is_mock_inference=False))-- worker_spy.assert_called_once()- assert result is sentineldiff --git a/tests/unit/pipelex/cogt/content_generation/test_img_gen_generate_dry_branch.py b/tests/unit/pipelex/cogt/content_generation/test_img_gen_generate_dry_branch.pynew file mode 100644index 000000000..d30251099--- /dev/null+++ b/tests/unit/pipelex/cogt/content_generation/test_img_gen_generate_dry_branch.py@@ -0,0 +1,101 @@+"""Unit tests for the ``run_mode == DRY`` branch in the img-gen leaf (img_gen_generate).++The DRY branch sits at the ``*_and_store`` layer, ABOVE the raw provider leaf (eng review D10):+a dry run must perform no provider call and no storage IO. Contract: under DRY the worker and the+``GeneratedContentFactory`` are never touched, and the returned ``ImageContent`` are URL-only mocks+carrying the prompt stamps; LIVE keeps the real generate-and-store path.+"""++import pytest+from pytest_mock import MockerFixture++from pipelex.cogt.content_generation.assignment_models import ImgGenAssignment+from pipelex.cogt.content_generation.cogt_run_params import CogtRunParams+from pipelex.cogt.content_generation.img_gen_generate import img_gen_image_list_and_store, img_gen_single_image_and_store+from pipelex.cogt.img_gen.img_gen_job_components import AspectRatio, Background, ImgGenJobConfig, ImgGenJobParams+from pipelex.cogt.img_gen.img_gen_prompt import ImgGenPrompt+from pipelex.pipe_run.pipe_run_mode import PipeRunMode+from pipelex.pipeline.job_metadata import JobMetadata+++class TestImgGenGenerateDryBranch:+ def _assignment(self, *, run_mode: PipeRunMode, nb_images: int = 1) -> ImgGenAssignment:+ return ImgGenAssignment(+ job_metadata=JobMetadata(user_id="u", pipeline_run_id="run_img_dry"),+ cogt_run_params=CogtRunParams(run_mode=run_mode),+ img_gen_handle="mock-img-handle",+ img_gen_prompt=ImgGenPrompt(positive_text="a red apple", negative_text="no worms"),+ img_gen_job_params=ImgGenJobParams(aspect_ratio=AspectRatio.SQUARE, background=Background.AUTO),+ img_gen_job_config=ImgGenJobConfig(is_sync_mode=True),+ nb_images=nb_images,+ )++ @pytest.mark.asyncio+ async def test_dry_single_skips_provider_and_storage(self, mocker: MockerFixture) -> None:+ """DRY single image: no worker call, no storage IO, URL-only mock with prompt stamps."""+ worker_spy = mocker.patch("pipelex.cogt.content_generation.img_gen_generate.get_img_gen_worker")+ factory = mocker.MagicMock()++ image_content = await img_gen_single_image_and_store(+ img_gen_assignment=self._assignment(run_mode=PipeRunMode.DRY),+ generated_content_factory=factory,+ )++ worker_spy.assert_not_called()+ factory.make_image_content.assert_not_called()+ assert image_content.url+ assert image_content.source_prompt == "a red apple"+ assert image_content.source_negative_prompt == "no worms"++ @pytest.mark.asyncio+ async def test_dry_single_yields_one_mock_even_with_nb_images_above_one(self, mocker: MockerFixture) -> None:+ """The single path mints exactly ONE mock regardless of the assignment's nb_images —+ matching the live single path's one-provider-call semantics — without mutating the+ assignment.+ """+ mocker.patch("pipelex.cogt.content_generation.img_gen_generate.get_img_gen_worker")+ assignment = self._assignment(run_mode=PipeRunMode.DRY, nb_images=3)++ image_content = await img_gen_single_image_and_store(+ img_gen_assignment=assignment,+ generated_content_factory=mocker.MagicMock(),+ )++ assert image_content.url+ assert assignment.nb_images == 3 # the normalization copies, never mutates++ @pytest.mark.asyncio+ async def test_dry_list_returns_nb_images_mocks_without_io(self, mocker: MockerFixture) -> None:+ """DRY image list: exactly nb_images URL-only mocks, no provider, no storage."""+ worker_spy = mocker.patch("pipelex.cogt.content_generation.img_gen_generate.get_img_gen_worker")+ factory = mocker.MagicMock()++ image_contents = await img_gen_image_list_and_store(+ img_gen_assignment=self._assignment(run_mode=PipeRunMode.DRY, nb_images=3),+ generated_content_factory=factory,+ )++ worker_spy.assert_not_called()+ factory.make_image_content.assert_not_called()+ assert len(image_contents) == 3+ assert all(image_content.url for image_content in image_contents)++ @pytest.mark.asyncio+ async def test_live_single_runs_provider_and_stores(self, mocker: MockerFixture) -> None:+ """LIVE keeps the real path: provider generates, factory stores."""+ worker = mocker.MagicMock()+ worker.gen_image = mocker.AsyncMock(return_value=mocker.MagicMock())+ mocker.patch("pipelex.cogt.content_generation.img_gen_generate.get_img_gen_worker", return_value=worker)+ mocker.patch(+ "pipelex.cogt.content_generation.img_gen_generate.ImgGenJobFactory.make_img_gen_job_from_prompt", return_value=mocker.MagicMock()+ )+ factory = mocker.MagicMock()+ factory.make_image_content = mocker.AsyncMock(return_value=mocker.MagicMock())++ await img_gen_single_image_and_store(+ img_gen_assignment=self._assignment(run_mode=PipeRunMode.LIVE),+ generated_content_factory=factory,+ )++ worker.gen_image.assert_awaited_once()+ factory.make_image_content.assert_awaited_once()diff --git a/tests/unit/pipelex/cogt/content_generation/test_img_gen_generate_mock_guard.py b/tests/unit/pipelex/cogt/content_generation/test_img_gen_generate_mock_guard.pydeleted file mode 100644index ff58502dd..000000000--- a/tests/unit/pipelex/cogt/content_generation/test_img_gen_generate_mock_guard.py+++ /dev/null@@ -1,58 +0,0 @@-"""Unit tests for the ``is_mock_inference`` hard guard in the img-gen leaf (img_gen_generate).--``--mock-inference`` has no leaf-level mock for image generation, so reaching the img-gen leaf under-the flag would dispatch to the real provider and spend. The leaf must fail loud instead: raise-``MockInferenceUnsupportedError`` before ``get_img_gen_worker`` is ever called. With the flag off, the-real worker path runs unchanged — proving the guard is keyed strictly on ``job_metadata.is_mock_inference``.-"""--from collections.abc import Awaitable, Callable--import pytest-from pytest_mock import MockerFixture--from pipelex.cogt.content_generation.assignment_models import ImgGenAssignment-from pipelex.cogt.content_generation.exceptions import MockInferenceUnsupportedError-from pipelex.cogt.content_generation.img_gen_generate import img_gen_image_list, img_gen_single_image-from pipelex.cogt.img_gen.img_gen_job_components import AspectRatio, Background, ImgGenJobConfig, ImgGenJobParams-from pipelex.cogt.img_gen.img_gen_prompt import ImgGenPrompt-from pipelex.pipeline.job_metadata import JobMetadata---class TestImgGenGenerateMockGuard:- def _assignment(self, *, is_mock_inference: bool) -> ImgGenAssignment:- return ImgGenAssignment(- job_metadata=JobMetadata(user_id="u", pipeline_run_id="run_img_guard", is_mock_inference=is_mock_inference),- img_gen_handle="mock-img-handle",- img_gen_prompt=ImgGenPrompt(positive_text="a red apple"),- img_gen_job_params=ImgGenJobParams(aspect_ratio=AspectRatio.SQUARE, background=Background.AUTO),- img_gen_job_config=ImgGenJobConfig(is_sync_mode=True),- nb_images=1,- )-- @pytest.mark.parametrize("leaf", [img_gen_single_image, img_gen_image_list])- @pytest.mark.asyncio- async def test_mock_flag_raises_and_skips_worker(self, mocker: MockerFixture, leaf: Callable[[ImgGenAssignment], Awaitable[object]]) -> None:- """is_mock_inference=True -> the leaf raises before any provider call (get_img_gen_worker untouched)."""- worker_spy = mocker.patch("pipelex.cogt.content_generation.img_gen_generate.get_img_gen_worker")-- with pytest.raises(MockInferenceUnsupportedError):- await leaf(self._assignment(is_mock_inference=True))-- worker_spy.assert_not_called() # no provider call -> no spend-- @pytest.mark.asyncio- async def test_no_flag_uses_real_worker(self, mocker: MockerFixture) -> None:- """is_mock_inference=False -> the real worker path runs (get_img_gen_worker is called)."""- sentinel = mocker.MagicMock()- worker = mocker.MagicMock()- worker.gen_image = mocker.AsyncMock(return_value=sentinel)- worker_spy = mocker.patch("pipelex.cogt.content_generation.img_gen_generate.get_img_gen_worker", return_value=worker)- mocker.patch(- "pipelex.cogt.content_generation.img_gen_generate.ImgGenJobFactory.make_img_gen_job_from_prompt", return_value=mocker.MagicMock()- )-- result = await img_gen_single_image(self._assignment(is_mock_inference=False))-- worker_spy.assert_called_once()- assert result is sentineldiff --git a/tests/unit/pipelex/cogt/content_generation/test_llm_generate_dry_branch.py b/tests/unit/pipelex/cogt/content_generation/test_llm_generate_dry_branch.pynew file mode 100644index 000000000..0c3ccb9df--- /dev/null+++ b/tests/unit/pipelex/cogt/content_generation/test_llm_generate_dry_branch.py@@ -0,0 +1,104 @@+"""Unit tests for the ``run_mode == DRY`` branch in the LLM text leaf (llm_generate).++The leaf is the single point where direct execution and the Temporal activities converge, so a DRY+branch there mocks identically on both backends. Contract: under DRY the worker is never touched+(no provider, no spend), the result carries the ``DRY RUN:`` marker, and exactly one synthetic+usage event is reported. The internal ``is_mock_usage`` sub-flag selects the reporting payload:+zero-token under the ``dry_run`` sentinel by default (cost report suppressed), non-zero under the+``mock_usage`` sentinel when set (cost report renders).+"""++import pytest+from pytest_mock import MockerFixture++from pipelex.cogt.content_generation.assignment_models import LLMAssignment+from pipelex.cogt.content_generation.cogt_run_params import CogtRunParams+from pipelex.cogt.content_generation.dry_mock import (+ DRY_RUN_INFERENCE_MODEL_NAME,+ MOCK_USAGE_MODEL_NAME,+ MOCK_USAGE_NB_TOKENS_BY_CATEGORY,+)+from pipelex.cogt.content_generation.llm_generate import llm_gen_text+from pipelex.cogt.llm.llm_prompt import LLMPrompt+from pipelex.cogt.llm.llm_setting import LLMSetting+from pipelex.cogt.usage.cost_registry import CostRegistry+from pipelex.pipe_run.pipe_run_mode import PipeRunMode+from pipelex.pipeline.job_metadata import JobMetadata+++class TestLlmGenerateDryBranch:+ def _assignment(self, *, run_mode: PipeRunMode, is_mock_usage: bool = False) -> LLMAssignment:+ return LLMAssignment(+ job_metadata=JobMetadata(user_id="u", pipeline_run_id="run_dry_branch"),+ cogt_run_params=CogtRunParams(run_mode=run_mode, is_mock_usage=is_mock_usage),+ llm_setting=LLMSetting(model="gpt-4o", temperature=0.5),+ llm_prompt=LLMPrompt(),+ )++ @pytest.mark.asyncio+ @pytest.mark.parametrize("is_mock_usage", [False, True])+ async def test_dry_mode_routes_to_dry_mock_and_skips_worker(self, mocker: MockerFixture, is_mock_usage: bool) -> None:+ """run_mode=DRY -> the dry helper answers with the marker string; get_llm_worker is never+ called — regardless of the is_mock_usage sub-flag (it only changes reporting).+ """+ worker_spy = mocker.patch("pipelex.cogt.content_generation.llm_generate.get_llm_worker")+ mock_delegate = mocker.MagicMock()+ mocker.patch("pipelex.cogt.content_generation.dry_mock.get_report_delegate", return_value=mock_delegate)++ result = await llm_gen_text(self._assignment(run_mode=PipeRunMode.DRY, is_mock_usage=is_mock_usage))++ worker_spy.assert_not_called()+ assert result.startswith("DRY RUN:")++ @pytest.mark.asyncio+ async def test_dry_mode_reports_one_zero_token_job(self, mocker: MockerFixture) -> None:+ """Default dry leaf reports exactly one synthetic job, zero-token under the dry_run sentinel (suppressed)."""+ mocker.patch("pipelex.cogt.content_generation.llm_generate.get_llm_worker")+ mock_delegate = mocker.MagicMock()+ mocker.patch("pipelex.cogt.content_generation.dry_mock.get_report_delegate", return_value=mock_delegate)++ await llm_gen_text(self._assignment(run_mode=PipeRunMode.DRY))++ assert mock_delegate.report_inference_job.call_count == 1+ reported_job = mock_delegate.report_inference_job.call_args.kwargs["inference_job"]+ usage = reported_job.job_report.llm_tokens_usage+ assert usage is not None+ assert usage.inference_model_name == DRY_RUN_INFERENCE_MODEL_NAME+ aggregated = CostRegistry.aggregate_costs([usage])+ assert not aggregated.has_reportable_usage+ assert aggregated.total_nb_tokens == 0++ @pytest.mark.asyncio+ async def test_mock_usage_reports_one_non_zero_job(self, mocker: MockerFixture) -> None:+ """is_mock_usage=True -> exactly one synthetic job, non-zero sentinel counts under the+ mock_usage sentinel, so the aggregated usage is reportable (cost report renders).+ """+ worker_spy = mocker.patch("pipelex.cogt.content_generation.llm_generate.get_llm_worker")+ mock_delegate = mocker.MagicMock()+ mocker.patch("pipelex.cogt.content_generation.dry_mock.get_report_delegate", return_value=mock_delegate)++ await llm_gen_text(self._assignment(run_mode=PipeRunMode.DRY, is_mock_usage=True))++ worker_spy.assert_not_called()+ assert mock_delegate.report_inference_job.call_count == 1+ reported_job = mock_delegate.report_inference_job.call_args.kwargs["inference_job"]+ usage = reported_job.job_report.llm_tokens_usage+ assert usage is not None+ assert usage.inference_model_name == MOCK_USAGE_MODEL_NAME+ aggregated = CostRegistry.aggregate_costs([usage])+ assert aggregated.has_reportable_usage+ assert aggregated.total_nb_tokens == sum(MOCK_USAGE_NB_TOKENS_BY_CATEGORY.values())++ @pytest.mark.asyncio+ async def test_live_mode_uses_real_worker(self, mocker: MockerFixture) -> None:+ """run_mode=LIVE -> the real worker path runs and no synthetic job is reported."""+ worker = mocker.MagicMock()+ worker.gen_text = mocker.AsyncMock(return_value="real generated text")+ worker_spy = mocker.patch("pipelex.cogt.content_generation.llm_generate.get_llm_worker", return_value=worker)+ report_spy = mocker.patch("pipelex.cogt.content_generation.dry_mock.get_report_delegate", return_value=mocker.MagicMock())++ result = await llm_gen_text(self._assignment(run_mode=PipeRunMode.LIVE))++ worker_spy.assert_called_once()+ assert result == "real generated text"+ report_spy.assert_not_called()diff --git a/tests/unit/pipelex/cogt/content_generation/test_llm_generate_mock_branch.py b/tests/unit/pipelex/cogt/content_generation/test_llm_generate_mock_branch.pydeleted file mode 100644index c2ed7d2bd..000000000--- a/tests/unit/pipelex/cogt/content_generation/test_llm_generate_mock_branch.py+++ /dev/null@@ -1,50 +0,0 @@-"""Unit tests for the ``is_mock_inference`` branch in the LLM leaf (llm_generate).--The leaf is the single point where direct execution and the Temporal activities converge, so branching-there covers both modes. These tests pin that the branch is keyed strictly on-``job_metadata.is_mock_inference``: when set, the worker is never touched and the mock helper answers;-when unset, the real worker path runs.-"""--import pytest-from pytest_mock import MockerFixture--from pipelex.cogt.content_generation.assignment_models import LLMAssignment-from pipelex.cogt.content_generation.llm_generate import llm_gen_text-from pipelex.cogt.llm.llm_prompt import LLMPrompt-from pipelex.cogt.llm.llm_setting import LLMSetting-from pipelex.pipeline.job_metadata import JobMetadata---class TestLlmGenerateMockBranch:- def _assignment(self, *, is_mock_inference: bool) -> LLMAssignment:- return LLMAssignment(- job_metadata=JobMetadata(user_id="u", pipeline_run_id="run_branch", is_mock_inference=is_mock_inference),- llm_setting=LLMSetting(model="gpt-4o", temperature=0.5),- llm_prompt=LLMPrompt(),- )-- @pytest.mark.asyncio- async def test_mock_flag_routes_to_mock_and_skips_worker(self, mocker: MockerFixture) -> None:- """is_mock_inference=True -> the mock helper answers and get_llm_worker is never called (no spend)."""- worker_spy = mocker.patch("pipelex.cogt.content_generation.llm_generate.get_llm_worker")- mocker.patch("pipelex.cogt.content_generation.dry_mock.get_report_delegate", return_value=mocker.MagicMock())-- result = await llm_gen_text(self._assignment(is_mock_inference=True))-- worker_spy.assert_not_called()- assert result.startswith("MOCK INFERENCE")-- @pytest.mark.asyncio- async def test_no_flag_uses_real_worker(self, mocker: MockerFixture) -> None:- """is_mock_inference=False -> the real worker path runs (get_llm_worker is called)."""- worker = mocker.MagicMock()- worker.gen_text = mocker.AsyncMock(return_value="real generated text")- worker_spy = mocker.patch("pipelex.cogt.content_generation.llm_generate.get_llm_worker", return_value=worker)- report_spy = mocker.patch("pipelex.cogt.content_generation.dry_mock.get_report_delegate", return_value=mocker.MagicMock())-- result = await llm_gen_text(self._assignment(is_mock_inference=False))-- worker_spy.assert_called_once()- assert result == "real generated text"- report_spy.assert_not_called() # the mock reporting path is not taken on a real rundiff --git a/tests/unit/pipelex/cogt/content_generation/test_render_generate_dry_branch.py b/tests/unit/pipelex/cogt/content_generation/test_render_generate_dry_branch.pynew file mode 100644index 000000000..622e6578c--- /dev/null+++ b/tests/unit/pipelex/cogt/content_generation/test_render_generate_dry_branch.py@@ -0,0 +1,62 @@+"""Unit tests for the ``run_mode == DRY`` branch in the page-view render leaf (render_generate).++The DRY branch sits above both the pypdfium2 rendering and the store step (eng review D10): a dry+run renders nothing and stores nothing, returning ``nb_extract_pages`` URL-only page-view mocks.+"""++import pytest+from pytest_mock import MockerFixture++from pipelex.cogt.content_generation.assignment_models import RenderPageViewsAssignment+from pipelex.cogt.content_generation.cogt_run_params import CogtRunParams+from pipelex.cogt.content_generation.render_generate import render_page_views_and_store+from pipelex.config import get_config+from pipelex.pipe_run.pipe_run_mode import PipeRunMode+from pipelex.pipeline.job_metadata import JobMetadata+++class TestRenderGenerateDryBranch:+ def _assignment(self, *, run_mode: PipeRunMode) -> RenderPageViewsAssignment:+ return RenderPageViewsAssignment(+ job_metadata=JobMetadata(user_id="u", pipeline_run_id="run_render_dry"),+ cogt_run_params=CogtRunParams(run_mode=run_mode),+ document_uri="file:///tmp/doc.pdf",+ page_views_dpi=72,+ )++ @pytest.mark.asyncio+ async def test_dry_mocks_page_views_without_rendering_or_storage(self, mocker: MockerFixture) -> None:+ """DRY: nb_extract_pages URL-only page-view mocks, no pdf rendering, no storage IO."""+ factory = mocker.MagicMock()++ page_views = await render_page_views_and_store(+ render_assignment=self._assignment(run_mode=PipeRunMode.DRY),+ generated_content_factory=factory,+ )++ factory.make_image_content.assert_not_called()+ assert len(page_views) == get_config().pipelex.dry_run_config.nb_extract_pages+ assert all(page_view.url for page_view in page_views)++ @pytest.mark.asyncio+ async def test_live_renders_and_stores(self, mocker: MockerFixture) -> None:+ """LIVE keeps the real path: pypdfium2 renders, factory stores each page view."""+ pil_image = mocker.MagicMock()+ mocker.patch(+ "pipelex.tools.pdf.pypdfium2_renderer.pypdfium2_renderer.render_pdf_pages_from_uri",+ new=mocker.AsyncMock(return_value=[pil_image, pil_image]),+ )+ mocker.patch(+ "pipelex.cogt.content_generation.render_generate.GeneratedImageRawDetails.make_from_pil_image",+ return_value=mocker.MagicMock(),+ )+ factory = mocker.MagicMock()+ factory.make_image_content = mocker.AsyncMock(return_value=mocker.MagicMock())++ page_views = await render_page_views_and_store(+ render_assignment=self._assignment(run_mode=PipeRunMode.LIVE),+ generated_content_factory=factory,+ )++ assert len(page_views) == 2+ assert factory.make_image_content.await_count == 2diff --git a/tests/unit/pipelex/cogt/content_generation/test_scoped_content_generator.py b/tests/unit/pipelex/cogt/content_generation/test_scoped_content_generator.pynew file mode 100644index 000000000..0d62e8574--- /dev/null+++ b/tests/unit/pipelex/cogt/content_generation/test_scoped_content_generator.py@@ -0,0 +1,81 @@+"""Unit tests for the `scoped_content_generator` ContextVar scope in `pipelex.hub`.++The scope lets an in-process run (the validation sweep, the in-process graph dry-run)+pin an inline content generator so its inference leaves resolve it through+`get_content_generator()` instead of the hub default — which, under a Temporal-enabled+hub, is `ContentGeneratorInWorkflow` and would dispatch activities. Mirrors the+`scoped_event_log` unit tests: set/restore, nesting, restore-on-exception, and+concurrent ContextVar isolation.+"""++import asyncio++import pytest++from pipelex.cogt.content_generation.content_generator import ContentGenerator+from pipelex.cogt.content_generation.content_generator_protocol import ContentGeneratorProtocol+from pipelex.cogt.content_generation.generated_content_factory import GeneratedContentFactory+from pipelex.hub import get_content_generator, scoped_content_generator+from pipelex.tools.storage.in_memory_storage_provider import InMemoryStorageProvider+++def _make_inline_generator() -> ContentGenerator:+ return ContentGenerator(generated_content_factory=GeneratedContentFactory(storage_provider=InMemoryStorageProvider()))+++class TestScopedContentGenerator:+ def test_override_set_and_restored(self):+ """Inside the scope the resolved generator is the override; outside it is the hub default."""+ generator = _make_inline_generator()+ assert get_content_generator() is not generator+ with scoped_content_generator(generator):+ assert get_content_generator() is generator+ assert get_content_generator() is not generator++ def test_nesting_restores_outer_override(self):+ """An inner scope shadows the outer one and restores it on exit."""+ outer_generator = _make_inline_generator()+ inner_generator = _make_inline_generator()+ with scoped_content_generator(outer_generator):+ assert get_content_generator() is outer_generator+ with scoped_content_generator(inner_generator):+ assert get_content_generator() is inner_generator+ assert get_content_generator() is outer_generator+ assert get_content_generator() is not outer_generator++ def test_override_restored_on_exception(self):+ """The override is restored even when the scoped block raises."""+ generator = _make_inline_generator()++ def raise_inside_scope() -> None:+ with scoped_content_generator(generator):+ assert get_content_generator() is generator+ msg = "boom"+ raise RuntimeError(msg)++ with pytest.raises(RuntimeError, match="boom"):+ raise_inside_scope()+ assert get_content_generator() is not generator++ @pytest.mark.asyncio+ async def test_concurrent_scopes_do_not_cross_contaminate(self):+ """Two concurrently-scoped tasks each see their own override (ContextVar isolation)."""+ generator_alpha = _make_inline_generator()+ generator_beta = _make_inline_generator()+ observed: dict[str, ContentGeneratorProtocol] = {}++ async def scope_and_observe(content_generator: ContentGeneratorProtocol, key: str) -> None:+ with scoped_content_generator(content_generator):+ await asyncio.sleep(0.01)+ observed[key] = get_content_generator()+ await asyncio.sleep(0.01)++ await asyncio.gather(+ scope_and_observe(generator_alpha, "alpha"),+ scope_and_observe(generator_beta, "beta"),+ )++ assert observed["alpha"] is generator_alpha+ assert observed["beta"] is generator_beta+ assert get_content_generator() is not generator_alpha+ assert get_content_generator() is not generator_betadiff --git a/tests/unit/pipelex/cogt/content_generation/test_search_generate_dry_branch.py b/tests/unit/pipelex/cogt/content_generation/test_search_generate_dry_branch.pynew file mode 100644index 000000000..1888cbab4--- /dev/null+++ b/tests/unit/pipelex/cogt/content_generation/test_search_generate_dry_branch.py@@ -0,0 +1,54 @@+"""Unit tests for the ``run_mode == DRY`` branch in the search leaf (search_generate).++Contract: under DRY the sourced-answer leaf mints a full synthetic ``SearchResultContent`` (with+mock sources) without resolving any worker or touching the model deck; LIVE keeps the real path.+The structured-search DRY arm does schema codegen, so it is covered by the integration tests in+``tests/integration/pipelex/cogt/content_generation/test_leaf_dry_object_mocks.py``.+"""++import pytest+from pytest_mock import MockerFixture++from pipelex.cogt.content_generation.assignment_models import SearchAssignment+from pipelex.cogt.content_generation.cogt_run_params import CogtRunParams+from pipelex.cogt.content_generation.search_generate import search_gen_sourced_answer+from pipelex.cogt.search.search_setting import SearchSetting+from pipelex.config import get_config+from pipelex.core.stuffs.search_result_content import SearchResultContent+from pipelex.pipe_run.pipe_run_mode import PipeRunMode+from pipelex.pipeline.job_metadata import JobMetadata+++class TestSearchGenerateDryBranch:+ def _assignment(self, *, run_mode: PipeRunMode) -> SearchAssignment:+ return SearchAssignment(+ job_metadata=JobMetadata(user_id="u", pipeline_run_id="run_search_dry"),+ cogt_run_params=CogtRunParams(run_mode=run_mode),+ query="what is pipelex?",+ search_setting=SearchSetting(model="mock-search-handle"),+ )++ @pytest.mark.asyncio+ async def test_dry_mints_result_without_worker(self, mocker: MockerFixture) -> None:+ """DRY: a synthetic SearchResultContent with mock sources; the model deck is never resolved."""+ deck_spy = mocker.patch("pipelex.cogt.content_generation.search_generate.get_model_deck")++ result = await search_gen_sourced_answer(self._assignment(run_mode=PipeRunMode.DRY))++ deck_spy.assert_not_called()+ assert isinstance(result, SearchResultContent)+ assert len(result.sources) == get_config().pipelex.dry_run_config.nb_list_items++ @pytest.mark.asyncio+ async def test_live_resolves_worker(self, mocker: MockerFixture) -> None:+ """LIVE keeps the real path: the worker is resolved from the deck and runs the search."""+ sentinel = mocker.MagicMock()+ worker = mocker.MagicMock()+ worker.search_sourced_answer = mocker.AsyncMock(return_value=sentinel)+ mocker.patch("pipelex.cogt.content_generation.search_generate._make_search_worker", return_value=worker)+ mocker.patch("pipelex.cogt.content_generation.search_generate._make_search_job", return_value=mocker.MagicMock())++ result = await search_gen_sourced_answer(self._assignment(run_mode=PipeRunMode.LIVE))++ worker.search_sourced_answer.assert_awaited_once()+ assert result is sentineldiff --git a/tests/unit/pipelex/cogt/content_generation/test_templating_generate_dry_branch.py b/tests/unit/pipelex/cogt/content_generation/test_templating_generate_dry_branch.pynew file mode 100644index 000000000..ca38e6655--- /dev/null+++ b/tests/unit/pipelex/cogt/content_generation/test_templating_generate_dry_branch.py@@ -0,0 +1,51 @@+"""Unit tests for the ``run_mode == DRY`` branch in the templating leaf (templating_generate).++Contract: under DRY the template is never rendered (a marker string is returned instead), but a+syntactically broken jinja2 template must STILL fail loudly — dry-run validation relies on the+parse check surviving the mock.+"""++import pytest+from pytest_mock import MockerFixture++from pipelex.cogt.content_generation.assignment_models import TemplatingAssignment+from pipelex.cogt.content_generation.cogt_run_params import CogtRunParams+from pipelex.cogt.content_generation.templating_generate import templating_gen_text+from pipelex.cogt.templating.template_category import TemplateCategory+from pipelex.pipe_run.pipe_run_mode import PipeRunMode+from pipelex.pipeline.job_metadata import JobMetadata+from pipelex.tools.jinja2.exceptions import Jinja2TemplateSyntaxError+++class TestTemplatingGenerateDryBranch:+ def _assignment(self, *, run_mode: PipeRunMode, template: str) -> TemplatingAssignment:+ return TemplatingAssignment(+ job_metadata=JobMetadata(user_id="u", pipeline_run_id="run_templating_dry"),+ cogt_run_params=CogtRunParams(run_mode=run_mode),+ context={"the_answer": "42"},+ template=template,+ category=TemplateCategory.BASIC,+ )++ @pytest.mark.asyncio+ async def test_dry_returns_marker_without_rendering(self, mocker: MockerFixture) -> None:+ """DRY: marker string returned; render_template is never invoked."""+ render_spy = mocker.patch("pipelex.cogt.content_generation.templating_generate.render_template")++ result = await templating_gen_text(self._assignment(run_mode=PipeRunMode.DRY, template="{{ the_answer }}"))++ render_spy.assert_not_called()+ assert result.startswith("DRY RUN:")++ @pytest.mark.asyncio+ async def test_dry_still_raises_on_broken_template(self) -> None:+ """DRY preserves the jinja2 parse check: a broken template fails loudly instead of being mocked over."""+ with pytest.raises(Jinja2TemplateSyntaxError):+ await templating_gen_text(self._assignment(run_mode=PipeRunMode.DRY, template="{% if unclosed %}"))++ @pytest.mark.asyncio+ async def test_live_renders_template(self) -> None:+ """LIVE renders for real."""+ result = await templating_gen_text(self._assignment(run_mode=PipeRunMode.LIVE, template="answer={{ the_answer }}"))++ assert result == "answer=42"diff --git a/tests/unit/pipelex/core/concepts/test_special_args_concept.py b/tests/unit/pipelex/core/concepts/test_special_args_concept.pyindex f390c62c5..71e1d0672 100644--- a/tests/unit/pipelex/core/concepts/test_special_args_concept.py+++ b/tests/unit/pipelex/core/concepts/test_special_args_concept.py@@ -16,7 +16,7 @@ from pipelex.hub import get_concept_library, get_pipe_routerfrom pipelex.pipe_operators.llm.pipe_llm import PipeLLMfrom pipelex.pipe_operators.llm.pipe_llm_blueprint import PipeLLMBlueprintfrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import PipeRunMode+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactoryfrom pipelex.pipeline.job_metadata import JobMetadatadiff --git a/tests/unit/pipelex/graph/test_graph_tracer_manager.py b/tests/unit/pipelex/graph/test_graph_tracer_manager.pynew file mode 100644index 000000000..568542bcc--- /dev/null+++ b/tests/unit/pipelex/graph/test_graph_tracer_manager.py@@ -0,0 +1,48 @@+"""Unit tests for GraphTracerManager's per-run tracer lifecycle (open/close registration)."""++import pytest+from pytest_mock import MockerFixture++from pipelex.graph.graph_config import DataInclusionConfig+from pipelex.graph.graph_tracer import GraphTracer+from pipelex.graph.graph_tracer_manager import GraphTracerManager+++def _make_data_inclusion() -> DataInclusionConfig:+ return DataInclusionConfig(+ stuff_json_content=False,+ stuff_text_content=False,+ stuff_html_content=False,+ error_stack_traces=False,+ pipe_and_concept_registry=False,+ )+++class TestGraphTracerManagerOpenTracer:+ def test_open_tracer_registers_then_close_pops(self) -> None:+ """Happy path: open_tracer registers the tracer under graph_id; close_tracer pops it."""+ GraphTracerManager.clear_instance()+ manager = GraphTracerManager.get_or_create_instance()+ try:+ trace_context = manager.open_tracer(graph_id="lifecycle_graph", data_inclusion=_make_data_inclusion())+ assert trace_context.graph_id == "lifecycle_graph"+ assert manager.get_tracer("lifecycle_graph") is not None++ manager.close_tracer("lifecycle_graph")+ assert manager.get_tracer("lifecycle_graph") is None+ finally:+ GraphTracerManager.clear_instance()++ def test_setup_failure_does_not_leak_registration(self, mocker: MockerFixture) -> None:+ """A failure inside GraphTracer.setup must not leave a tracer registered in the manager."""+ GraphTracerManager.clear_instance()+ manager = GraphTracerManager.get_or_create_instance()+ try:+ mocker.patch.object(GraphTracer, "setup", side_effect=ValueError("setup blew up"))++ with pytest.raises(ValueError, match="setup blew up"):+ manager.open_tracer(graph_id="leak_check_graph", data_inclusion=_make_data_inclusion())++ assert manager.get_tracer("leak_check_graph") is None+ finally:+ GraphTracerManager.clear_instance()diff --git a/tests/unit/pipelex/pipe_run/test_cogt_run_params_carrier.py b/tests/unit/pipelex/pipe_run/test_cogt_run_params_carrier.pynew file mode 100644index 000000000..c8394cc78--- /dev/null+++ b/tests/unit/pipelex/pipe_run/test_cogt_run_params_carrier.py@@ -0,0 +1,104 @@+"""Pin the run-mode ownership on ``PipeRunParams`` and the derived ``CogtRunParams`` carrier.++``run_mode`` / ``is_mock_usage`` are direct fields of ``PipeRunParams`` (single stored copy); the+``cogt_run_params`` property mints the cogt-tier carrier from them on demand. The factory is the+single writer, and a stale ``cogt_run_params=`` kwarg must fail loudly instead of silently building+a LIVE-mode instance.+"""++import pytest+from pydantic import ValidationError+from pytest_mock import MockerFixture++from pipelex.cogt.content_generation.cogt_run_params import CogtRunParams+from pipelex.pipe_run.pipe_run_mode import PipeRunMode+from pipelex.pipe_run.pipe_run_params import PipeRunParams+from pipelex.pipe_run.pipe_run_params_factory import PipeRunParamsFactory+++class TestCogtRunParamsCarrier:+ def test_cogt_run_params_derives_from_run_mode_fields(self) -> None:+ """The carrier is minted from the stored fields — one copy of the facts."""+ run_params = PipeRunParams(run_mode=PipeRunMode.DRY, is_mock_usage=True, pipe_stack_limit=20)++ cogt_run_params = run_params.cogt_run_params+ assert cogt_run_params.run_mode.is_dry+ assert cogt_run_params.is_mock_usage+ assert cogt_run_params.run_mode is run_params.run_mode++ def test_factory_is_single_writer(self) -> None:+ """make_run_params resolves pipe_run_mode into the run_mode field."""+ run_params = PipeRunParamsFactory.make_run_params(pipe_run_mode=PipeRunMode.DRY)++ assert run_params.run_mode.is_dry+ assert run_params.cogt_run_params.run_mode.is_dry++ def test_forced_dry_coerces_live_and_warns(self, mocker: MockerFixture) -> None:+ """Keyless boot: a LIVE request is coerced to DRY with a warning."""+ mocker.patch("pipelex.pipe_run.pipe_run_params_factory.is_dry_run_forced", return_value=True)+ warning_spy = mocker.patch("pipelex.pipe_run.pipe_run_params_factory.log.warning")++ run_params = PipeRunParamsFactory.make_run_params(pipe_run_mode=PipeRunMode.LIVE)++ assert run_params.run_mode.is_dry+ warning_spy.assert_called_once()++ def test_forced_dry_does_not_mask_mock_usage_violation(self, mocker: MockerFixture) -> None:+ """Keyless boot: LIVE + is_mock_usage still fails loud — the factory validates the REQUESTED+ mode first, so the coercion cannot silently turn an illegal request into a reportable dry run.+ """+ mocker.patch("pipelex.pipe_run.pipe_run_params_factory.is_dry_run_forced", return_value=True)++ with pytest.raises(ValueError, match="is_mock_usage"):+ PipeRunParamsFactory.make_run_params(pipe_run_mode=PipeRunMode.LIVE, is_mock_usage=True)++ def test_forced_dry_keeps_mock_usage_on_dry_request(self, mocker: MockerFixture) -> None:+ """Keyless boot: an explicit DRY + is_mock_usage request is legal and passes through untouched."""+ mocker.patch("pipelex.pipe_run.pipe_run_params_factory.is_dry_run_forced", return_value=True)++ run_params = PipeRunParamsFactory.make_run_params(pipe_run_mode=PipeRunMode.DRY, is_mock_usage=True)++ assert run_params.run_mode.is_dry+ assert run_params.is_mock_usage++ def test_stale_cogt_run_params_kwarg_fails_loudly(self) -> None:+ """cogt_run_params is a derived property, not a field: passing it as a kwarg must raise.++ run_mode is supplied so the ONLY thing that can raise is the forbidden extra — without+ it the test would pass on the missing-required-field error even if extra="forbid" were+ dropped.+ """+ with pytest.raises(ValidationError, match="cogt_run_params"):+ PipeRunParams(+ run_mode=PipeRunMode.DRY,+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.DRY), # type: ignore[call-arg] # pyright: ignore[reportCallIssue]+ pipe_stack_limit=20,+ )++ def test_run_mode_fields_are_frozen(self) -> None:+ """A post-construction DRY→LIVE flip (or mock-usage flip) must raise — it would bypass+ the construction validator and flow into real provider spend via the derived carrier.+ """+ run_params = PipeRunParams(run_mode=PipeRunMode.DRY, pipe_stack_limit=20)++ with pytest.raises(ValidationError, match="frozen"):+ run_params.run_mode = PipeRunMode.LIVE # type: ignore[misc] # the static read-only error is the runtime contract under test+ with pytest.raises(ValidationError, match="frozen"):+ run_params.is_mock_usage = True # type: ignore[misc] # the static read-only error is the runtime contract under test++ def test_mock_usage_requires_dry_on_pipe_run_params(self) -> None:+ """is_mock_usage is a sub-flag of DRY: setting it on a LIVE PipeRunParams is a contract violation."""+ with pytest.raises(ValidationError, match="is_mock_usage"):+ PipeRunParams(run_mode=PipeRunMode.LIVE, is_mock_usage=True, pipe_stack_limit=20)++ def test_mock_usage_requires_dry_on_carrier(self) -> None:+ """The wire carrier enforces the same rule at its own boundary (assignments cross the wire)."""+ with pytest.raises(ValidationError, match="is_mock_usage"):+ CogtRunParams(run_mode=PipeRunMode.LIVE, is_mock_usage=True)++ def test_mock_usage_rides_dry_run_mode(self) -> None:+ """The legal combination: DRY + is_mock_usage builds fine on both models."""+ cogt_run_params = CogtRunParams(run_mode=PipeRunMode.DRY, is_mock_usage=True)++ assert cogt_run_params.run_mode.is_dry+ assert cogt_run_params.is_mock_usagediff --git a/tests/unit/pipelex/temporal/test_content_generator_in_workflow.py b/tests/unit/pipelex/temporal/test_content_generator_in_workflow.pyindex fc0672c2f..f50e9ea45 100644--- a/tests/unit/pipelex/temporal/test_content_generator_in_workflow.py+++ b/tests/unit/pipelex/temporal/test_content_generator_in_workflow.py@@ -18,7 +18,7 @@ from pydantic import BaseModelfrom pytest_mock import MockerFixturefrom temporalio.exceptions import ActivityError, ApplicationError-from pipelex.cogt.content_generation.generated_content_factory import GeneratedContentFactory+from pipelex.cogt.content_generation.cogt_run_params import CogtRunParamsfrom pipelex.cogt.extract.extract_input import ExtractInputfrom pipelex.cogt.extract.extract_job_components import ExtractJobConfig, ExtractJobParamsfrom pipelex.cogt.img_gen.img_gen_prompt import ImgGenPrompt@@ -27,9 +27,9 @@ from pipelex.cogt.llm.llm_setting import LLMSettingfrom pipelex.core.stuffs.image_content import ImageContentfrom pipelex.core.stuffs.page_content import PageContentfrom pipelex.core.stuffs.text_and_images_content import TextAndImagesContent+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipeline.job_metadata import JobMetadatafrom pipelex.temporal.tprl_content_generation.content_generator_in_workflow import ContentGeneratorInWorkflow-from pipelex.tools.storage.in_memory_storage_provider import InMemoryStorageProviderclass _Person(BaseModel):@@ -42,8 +42,7 @@ def _make_job_metadata() -> JobMetadata:def _make_generator() -> ContentGeneratorInWorkflow:- factory = GeneratedContentFactory(storage_provider=InMemoryStorageProvider())- return ContentGeneratorInWorkflow(generated_content_factory=factory)+ return ContentGeneratorInWorkflow()def _make_llm_setting() -> LLMSetting:@@ -93,6 +92,7 @@ class TestContentGeneratorInWorkflow:await generator.make_llm_text(job_metadata=_make_job_metadata(),+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),llm_setting_main=_make_llm_setting(),llm_prompt_for_text=LLMPrompt(user_text="hello"),)@@ -142,6 +142,7 @@ class TestContentGeneratorInWorkflow:await generator.make_extract_pages(job_metadata=_make_job_metadata(),+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),extract_input=ExtractInput(image_uri="test://image.png"),extract_handle="test-handle",extract_job_params=ExtractJobParams.make_default_extract_job_params(),@@ -166,6 +167,7 @@ class TestContentGeneratorInWorkflow:params.should_include_page_views = Trueawait generator.make_extract_pages(job_metadata=_make_job_metadata(),+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),extract_input=ExtractInput(image_uri="test://image.png"),extract_handle="test-handle",extract_job_params=params,@@ -198,6 +200,7 @@ class TestContentGeneratorInWorkflow:params.should_include_page_views = Trueresult = await generator.make_extract_pages(job_metadata=_make_job_metadata(),+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),extract_input=ExtractInput(document_uri="test://doc.pdf"),extract_handle="test-handle",extract_job_params=params,@@ -226,6 +229,7 @@ class TestContentGeneratorInWorkflow:await generator.make_templated_text(job_metadata=job_metadata,+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),context={"key": "value"},template="{{ key }}",)@@ -248,6 +252,7 @@ class TestContentGeneratorInWorkflow:with pytest.raises(TemporalError) as exc_info:await generator.make_llm_text(job_metadata=_make_job_metadata(),+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),llm_setting_main=_make_llm_setting(),llm_prompt_for_text=LLMPrompt(user_text="hello"),)@@ -270,6 +275,7 @@ class TestContentGeneratorInWorkflow:with pytest.raises(ActivityError) as exc_info:await generator.make_llm_text(job_metadata=_make_job_metadata(),+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),llm_setting_main=_make_llm_setting(),llm_prompt_for_text=LLMPrompt(user_text="hello"),)@@ -301,6 +307,7 @@ class TestContentGeneratorInWorkflow:if caller == "make_object":return await generator.make_object(job_metadata=job_metadata,+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),object_class=_Person,llm_setting_for_object=_make_llm_setting(),llm_prompt_for_object=LLMPrompt(user_text="hello"),@@ -308,6 +315,7 @@ class TestContentGeneratorInWorkflow:if caller == "make_object_list":return await generator.make_object_list(job_metadata=job_metadata,+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),object_class=_Person,llm_setting_for_object_list=_make_llm_setting(),llm_prompt_for_object_list=LLMPrompt(user_text="hello"),@@ -315,12 +323,14 @@ class TestContentGeneratorInWorkflow:if caller == "make_single_image":return await generator.make_single_image(job_metadata=job_metadata,+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),img_gen_handle="test-handle",img_gen_prompt=ImgGenPrompt(positive_text="prompt"),)if caller == "make_image_list":return await generator.make_image_list(job_metadata=job_metadata,+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),img_gen_handle="test-handle",img_gen_prompt=ImgGenPrompt(positive_text="prompt"),nb_images=2,@@ -328,12 +338,14 @@ class TestContentGeneratorInWorkflow:if caller == "make_templated_text":return await generator.make_templated_text(job_metadata=job_metadata,+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),context={"key": "value"},template="{{ key }}",)if caller == "make_render_page_views":return await generator.make_render_page_views(job_metadata=job_metadata,+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),extract_input=ExtractInput(document_uri="test://doc.pdf"),extract_handle="test-handle",)diff --git a/tests/unit/pipelex/temporal/test_content_generator_in_workflow_object_fidelity.py b/tests/unit/pipelex/temporal/test_content_generator_in_workflow_object_fidelity.pyindex d2903d4ee..39a006c74 100644--- a/tests/unit/pipelex/temporal/test_content_generator_in_workflow_object_fidelity.py+++ b/tests/unit/pipelex/temporal/test_content_generator_in_workflow_object_fidelity.py@@ -1,25 +1,25 @@-"""Unit tests for the ``--mock-inference`` object-mock fidelity guard on the Temporal arm (review F2).+"""Unit tests for the dry-run object-mock fidelity guard on the Temporal arm (review F2).The activity (``act_llm_gen_object*``) builds the mock from the schema-reconstructed class, then``ContentGeneratorInWorkflow.make_object`` re-validates it against the original class. When the originalclass enforces an invariant the JSON-schema round-trip cannot capture (here a custom ``@field_validator``),-that re-validation fails — and must surface as a clear typed ``MockInferenceObjectFidelityError`` rather than-an opaque ``pydantic.ValidationError`` mid-workflow. The guard is scoped to ``is_mock_inference`` only: a LIVE-provider's invalid output keeps its existing ``ValidationError``. ``execute_activity`` is mocked to return the-synthetic object, so no Temporal server (or real provider) is involved.+that re-validation fails — and must surface as a clear typed ``DryRunObjectFidelityError`` rather than+an opaque ``pydantic.ValidationError`` mid-workflow. The guard is armed by ``run_mode=DRY`` only: a LIVE+provider's invalid output keeps its existing ``ValidationError``. ``execute_activity`` is mocked to return+the synthetic object, so no Temporal server (or real provider) is involved."""import pytestfrom pydantic import BaseModel, ValidationError, field_validatorfrom pytest_mock import MockerFixture-from pipelex.cogt.content_generation.exceptions import MockInferenceObjectFidelityError-from pipelex.cogt.content_generation.generated_content_factory import GeneratedContentFactory+from pipelex.cogt.content_generation.cogt_run_params import CogtRunParams+from pipelex.cogt.content_generation.exceptions import DryRunObjectFidelityErrorfrom pipelex.cogt.llm.llm_prompt import LLMPromptfrom pipelex.cogt.llm.llm_setting import LLMSetting+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipeline.job_metadata import JobMetadatafrom pipelex.temporal.tprl_content_generation.content_generator_in_workflow import ContentGeneratorInWorkflow-from pipelex.tools.storage.in_memory_storage_provider import InMemoryStorageProviderclass ConstrainedName(BaseModel):@@ -43,11 +43,11 @@ class RawName(BaseModel):def _make_generator() -> ContentGeneratorInWorkflow:- return ContentGeneratorInWorkflow(generated_content_factory=GeneratedContentFactory(storage_provider=InMemoryStorageProvider()))+ return ContentGeneratorInWorkflow()-def _mock_job_metadata(*, is_mock_inference: bool) -> JobMetadata:- return JobMetadata(user_id="u", pipeline_run_id="run", pipe_code="my_pipe", is_mock_inference=is_mock_inference)+def _job_metadata() -> JobMetadata:+ return JobMetadata(user_id="u", pipeline_run_id="run", pipe_code="my_pipe")@pytest.mark.asyncio(loop_scope="class")@@ -58,12 +58,13 @@ class TestContentGeneratorInWorkflowObjectFidelity:mocker.patch("temporalio.workflow.unsafe.is_replaying_history_events", return_value=False)async def test_make_object_raises_typed_error_on_fidelity_gap(self, mocker: MockerFixture) -> None:- """A mock-inference object whose re-validation fails surfaces MockInferenceObjectFidelityError."""+ """A dry-built object whose re-validation fails surfaces DryRunObjectFidelityError."""mocker.patch("temporalio.workflow.execute_activity", new_callable=mocker.AsyncMock, return_value=RawName(name="bad"))- with pytest.raises(MockInferenceObjectFidelityError) as exc_info:+ with pytest.raises(DryRunObjectFidelityError) as exc_info:await _make_generator().make_object(- job_metadata=_mock_job_metadata(is_mock_inference=True),+ job_metadata=_job_metadata(),+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.DRY),object_class=ConstrainedName,llm_setting_for_object=LLMSetting(model="test-llm", temperature=0.5),llm_prompt_for_object=LLMPrompt(user_text="hello"),@@ -74,21 +75,23 @@ class TestContentGeneratorInWorkflowObjectFidelity:"""The list path applies the same per-item guard."""mocker.patch("temporalio.workflow.execute_activity", new_callable=mocker.AsyncMock, return_value=[RawName(name="bad")])- with pytest.raises(MockInferenceObjectFidelityError):+ with pytest.raises(DryRunObjectFidelityError):await _make_generator().make_object_list(- job_metadata=_mock_job_metadata(is_mock_inference=True),+ job_metadata=_job_metadata(),+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.DRY),object_class=ConstrainedName,llm_setting_for_object_list=LLMSetting(model="test-llm", temperature=0.5),llm_prompt_for_object_list=LLMPrompt(user_text="hello"),)- async def test_non_mock_run_keeps_raw_validation_error(self, mocker: MockerFixture) -> None:- """Outside --mock-inference the guard is inert: a re-validation failure stays a raw ValidationError."""+ async def test_live_run_keeps_raw_validation_error(self, mocker: MockerFixture) -> None:+ """On a LIVE run the guard is inert: a re-validation failure stays a raw ValidationError."""mocker.patch("temporalio.workflow.execute_activity", new_callable=mocker.AsyncMock, return_value=RawName(name="bad"))with pytest.raises(ValidationError):await _make_generator().make_object(- job_metadata=_mock_job_metadata(is_mock_inference=False),+ job_metadata=_job_metadata(),+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),object_class=ConstrainedName,llm_setting_for_object=LLMSetting(model="test-llm", temperature=0.5),llm_prompt_for_object=LLMPrompt(user_text="hello"),diff --git a/tests/unit/pipelex/temporal/test_default_activity_id_collision_bug.py b/tests/unit/pipelex/temporal/test_default_activity_id_collision_bug.pyindex 768713592..0c32834f0 100644--- a/tests/unit/pipelex/temporal/test_default_activity_id_collision_bug.py+++ b/tests/unit/pipelex/temporal/test_default_activity_id_collision_bug.py@@ -17,12 +17,12 @@ produce unique-per-run ids by construction.import pytestfrom pytest_mock import MockerFixture-from pipelex.cogt.content_generation.generated_content_factory import GeneratedContentFactory+from pipelex.cogt.content_generation.cogt_run_params import CogtRunParamsfrom pipelex.cogt.llm.llm_prompt import LLMPromptfrom pipelex.cogt.llm.llm_setting import LLMSetting+from pipelex.pipe_run.pipe_run_mode import PipeRunModefrom pipelex.pipeline.job_metadata import JobMetadatafrom pipelex.temporal.tprl_content_generation.content_generator_in_workflow import ContentGeneratorInWorkflow-from pipelex.tools.storage.in_memory_storage_provider import InMemoryStorageProviderdef _make_job_metadata() -> JobMetadata:@@ -30,8 +30,7 @@ def _make_job_metadata() -> JobMetadata:def _make_generator() -> ContentGeneratorInWorkflow:- factory = GeneratedContentFactory(storage_provider=InMemoryStorageProvider())- return ContentGeneratorInWorkflow(generated_content_factory=factory)+ return ContentGeneratorInWorkflow()def _make_llm_setting() -> LLMSetting:@@ -58,11 +57,13 @@ class TestDefaultActivityIdCollisionBug:await generator.make_llm_text(job_metadata=_make_job_metadata(),+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),llm_setting_main=_make_llm_setting(),llm_prompt_for_text=LLMPrompt(user_text="hello"),)await generator.make_llm_text(job_metadata=_make_job_metadata(),+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),llm_setting_main=_make_llm_setting(),llm_prompt_for_text=LLMPrompt(user_text="hello again"),)diff --git a/tests/unit/pipelex/temporal/test_dry_validate_dispatch.py b/tests/unit/pipelex/temporal/test_dry_validate_dispatch.pynew file mode 100644index 000000000..72b7131f3--- /dev/null+++ b/tests/unit/pipelex/temporal/test_dry_validate_dispatch.py@@ -0,0 +1,42 @@+"""Unit tests for ``dispatch_dry_validate`` — the submitter-side round-trip the API route calls.++The helper owns two behavior decisions nothing else pins:++- D-C5 at the workflow tier: ``RetryPolicy(maximum_attempts=1)`` so a deterministically-failed+ validation workflow is never re-run by the server (the config default policy would).+- The interactive execution bound: an explicit ``workflow_execution_timeout`` matching the+ activity retry budget instead of the batch-tuned 1-hour config default, so a worker outage+ fails the HTTP caller fast instead of queueing stale validation work.+"""++from datetime import timedelta++import pytest+from pytest_mock import MockerFixture++from pipelex.temporal.tprl.workflow_caller import WorkflowExecutorFactory+from pipelex.temporal.tprl_pipe.act_dry_validate import DryValidateArg, DryValidateResult+from pipelex.temporal.tprl_pipe.dry_validate_dispatch import dispatch_dry_validate+from pipelex.temporal.tprl_pipe.wf_dry_validate import WfDryValidate+++class TestDryValidateDispatch:+ @pytest.mark.asyncio+ async def test_dispatch_pins_no_workflow_retry_and_interactive_timeout(self, mocker: MockerFixture) -> None:+ expected_result = DryValidateResult(dry_run_outputs={})+ fake_executor = mocker.Mock()+ fake_executor.make_workflow_id.return_value = "wf_dry_validate_test"+ fake_executor.execute_workflow = mocker.AsyncMock(return_value=expected_result)+ create_executor = mocker.patch.object(WorkflowExecutorFactory, "create_executor", return_value=fake_executor)++ arg = DryValidateArg(mthds_contents=['domain = "unit_dispatch"'])+ result = await dispatch_dry_validate(arg)++ assert result is expected_result+ create_kwargs = create_executor.call_args.kwargs+ assert create_kwargs["retry_policy"].maximum_attempts == 1+ assert create_kwargs["workflow_execution_timeout"] == timedelta(minutes=12)+ execute_kwargs = fake_executor.execute_workflow.call_args.kwargs+ assert execute_kwargs["workflow_class"] is WfDryValidate+ assert execute_kwargs["workflow_arg"] is arg+ assert execute_kwargs["workflow_id"] == "wf_dry_validate_test"diff --git a/tests/unit/pipelex/temporal/test_non_retryable_baseline_pins.py b/tests/unit/pipelex/temporal/test_non_retryable_baseline_pins.pynew file mode 100644index 000000000..f9e39c7ea--- /dev/null+++ b/tests/unit/pipelex/temporal/test_non_retryable_baseline_pins.py@@ -0,0 +1,24 @@+"""Pin the deterministic dry-run error classes into the baseline ``non_retryable_error_types``.++The terminality contract of the dry-run mock errors rests on a *string* linkage: the class name+literal in ``pipelex/pipelex.toml`` must match ``cls.__name__``. A rename on either side silently+makes a deterministic failure retryable again — the retry-forever / wasted-retry family this+project has already been bitten by — with the whole suite staying green. This test makes that+drift loud.+"""++from pathlib import Path++import tomli++from pipelex.cogt.content_generation.exceptions import DryRunMockBuildError, DryRunObjectFidelityError+++class TestNonRetryableBaselinePins:+ def test_deterministic_dry_errors_are_pinned_non_retryable(self) -> None:+ toml_path = Path(__file__).parents[4] / "pipelex" / "pipelex.toml"+ config = tomli.loads(toml_path.read_text(encoding="utf-8"))+ baseline = config["temporal"]["worker_config"]["retry_policy_config"]["non_retryable_error_types"]++ assert DryRunMockBuildError.__name__ in baseline+ assert DryRunObjectFidelityError.__name__ in baselinediff --git a/tests/unit/pipelex/tracing/test_scoped_event_log.py b/tests/unit/pipelex/tracing/test_scoped_event_log.pynew file mode 100644index 000000000..39b3ffa6d--- /dev/null+++ b/tests/unit/pipelex/tracing/test_scoped_event_log.py@@ -0,0 +1,72 @@+"""Unit tests for the `scoped_event_log` ContextVar scope in `pipelex.hub`.++The scope lets a caller pin a specific `EventLogProtocol` instance for the duration+of a run so the write side (tracer emission) and the read side (tracing assembly)+share the SAME instance — the fix for the two-instance problem that an in-memory+event log cannot bridge via an external store.+"""++import asyncio++import pytest++from pipelex.hub import get_event_log_override, scoped_event_log+from pipelex.tracing.event_log_protocol import EventLogProtocol+from pipelex.tracing.in_memory_event_log import InMemoryEventLog+++class TestScopedEventLog:+ def test_override_set_and_restored(self):+ """Inside the scope the override is the given instance; outside it is None."""+ assert get_event_log_override() is None+ event_log = InMemoryEventLog()+ with scoped_event_log(event_log):+ assert get_event_log_override() is event_log+ assert get_event_log_override() is None++ def test_nesting_restores_outer_override(self):+ """An inner scope shadows the outer one and restores it on exit."""+ outer_log = InMemoryEventLog(writer_id="outer")+ inner_log = InMemoryEventLog(writer_id="inner")+ with scoped_event_log(outer_log):+ assert get_event_log_override() is outer_log+ with scoped_event_log(inner_log):+ assert get_event_log_override() is inner_log+ assert get_event_log_override() is outer_log+ assert get_event_log_override() is None++ def test_override_restored_on_exception(self):+ """The override is restored even when the scoped block raises."""+ event_log = InMemoryEventLog()++ def raise_inside_scope() -> None:+ with scoped_event_log(event_log):+ assert get_event_log_override() is event_log+ msg = "boom"+ raise RuntimeError(msg)++ with pytest.raises(RuntimeError, match="boom"):+ raise_inside_scope()+ assert get_event_log_override() is None++ @pytest.mark.asyncio+ async def test_concurrent_scopes_do_not_cross_contaminate(self):+ """Two concurrently-scoped tasks each see their own override (ContextVar isolation)."""+ log_alpha = InMemoryEventLog(writer_id="alpha")+ log_beta = InMemoryEventLog(writer_id="beta")+ observed: dict[str, EventLogProtocol | None] = {}++ async def scope_and_observe(event_log: EventLogProtocol, key: str) -> None:+ with scoped_event_log(event_log):+ await asyncio.sleep(0.01)+ observed[key] = get_event_log_override()+ await asyncio.sleep(0.01)++ await asyncio.gather(+ scope_and_observe(log_alpha, "alpha"),+ scope_and_observe(log_beta, "beta"),+ )++ assert observed["alpha"] is log_alpha+ assert observed["beta"] is log_beta+ assert get_event_log_override() is Nonediff --git a/uv.lock b/uv.lockindex e1c776b08..1840507ba 100644--- a/uv.lock+++ b/uv.lock@@ -1991,14 +1991,15 @@ wheels = [[[package]]name = "kajson"-version = "0.6.0"+version = "0.7.0"source = { registry = "https://pypi.org/simple" }dependencies = [{ name = "pydantic" },+ { name = "tzdata" },]-sdist = { url = "https://files.pythonhosted.org/packages/8d/54/83c3b00f270c39037b699398e42e16837ef1109c8af47ac668fe3c14a817/kajson-0.6.0.tar.gz", hash = "sha256:7544a544e6f135f129015c9475ff2cc0a7fc8e76d6eea1d8d37f2e438ab4d936", size = 22567, upload-time = "2026-05-30T19:06:08.084Z" }+sdist = { url = "https://files.pythonhosted.org/packages/d2/a4/da91af6dac42516e45a9897ba901d8ec2ee2b13b135caf6ce120867aac2b/kajson-0.7.0.tar.gz", hash = "sha256:1c8f80d157f980c9adb9fda0e77c1cc0078f50547d84ef548a5e19b5a84d8d37", size = 25071, upload-time = "2026-06-10T10:00:56.097Z" }wheels = [- { url = "https://files.pythonhosted.org/packages/e9/fb/e325ef3bd84f2d40286a09fe78447a0a20e3a6ec12ce28e0d4ecc8bce620/kajson-0.6.0-py3-none-any.whl", hash = "sha256:77ddb0e12b274479ced061d84da091a250fdeb3f4dd7444ae0c6ca929ba73431", size = 29024, upload-time = "2026-05-30T19:06:06.645Z" },+ { url = "https://files.pythonhosted.org/packages/f0/1c/8251c0dadd1843e020edf4ef162aaa7dd0e98cbd2e1fe33d843807cd7ce9/kajson-0.7.0-py3-none-any.whl", hash = "sha256:d9903327bc7f182ec481ffafd074765347365600592e5f9f67866d5b0ae4f939", size = 31524, upload-time = "2026-06-10T10:00:54.952Z" },][[package]]@@ -3850,7 +3851,7 @@ requires-dist = [{ name = "jinja2", specifier = ">=3.1.4" },{ name = "json2html", specifier = ">=1.3.0" },{ name = "jsonschema", marker = "extra == 'dev'", specifier = ">=4.20.0" },- { name = "kajson", specifier = "==0.6.0" },+ { name = "kajson", specifier = "==0.7.0" },{ name = "linkup-sdk", marker = "extra == 'linkup'", specifier = ">=0.12.0" },{ name = "lxml", marker = "extra == 'docling'", specifier = ">=6.1.0" },{ name = "markdown", specifier = ">=3.6" },diff --git a/wip/distributed-execution/temporal-e2e-validate-prompt-menu.md b/wip/distributed-execution/temporal-e2e-validate-prompt-menu.mdindex 7b18c6635..5cb668ece 100644--- a/wip/distributed-execution/temporal-e2e-validate-prompt-menu.md+++ b/wip/distributed-execution/temporal-e2e-validate-prompt-menu.md@@ -9,13 +9,13 @@ Convention below: `$` marks prompts that spend real money (live inference / live### Complete sweep, real spend authorized ($)```text-/temporal-e2e-validate run the COMPLETE sweep with real spend authorized — do NOT skip the opt-in batteries or the live arms. Mode 1 pytest dry then live. Mode 2 every tier: 1/2/2b/3 core, 4 & 5 live (image payload), Step 4 graph tracing, Step 5 concurrent isolation, Tier 8 usage emission, Tier 8b cost reporting FULL (arms A–G), Tier 9 object-gen, Tier 11 extract, Tier 12 CV batch live, Tiers 13–16 error propagation, Tiers 6/7 codec. Then Step 8 routing battery (10a–10c) and Step 9 queue-options + worker-runtime-profile battery (Scenarios A–F). Finish with the master results table — PASS/FAIL per tier/scenario — and stop-and-report on any failure.+/temporal-e2e-validate run the COMPLETE sweep with real spend authorized — do NOT skip the opt-in batteries or the live arms. Mode 1 pytest dry then live. Mode 2 every tier: 1/2/2b/2c/2d/3 core, 4 & 5 live (image payload), Step 4 graph tracing, Step 5 concurrent isolation, Tier 8 usage emission, Tier 8b cost reporting FULL (arms A–G), Tier 9 object-gen, Tier 11 extract, Tier 12 CV batch live, Tiers 13–16 error propagation, Tiers 6/7 codec. Then Step 8 routing battery (10a–10c) and Step 9 queue-options + worker-runtime-profile battery (Scenarios A–F). Finish with the master results table — PASS/FAIL per tier/scenario — and stop-and-report on any failure.```### Complete sweep, no spend (full structural pass)```text-/temporal-e2e-validate run the full sweep DRY/mock only — no real spend. Mode 1 dry, Mode 2 all tiers in dry/mock (Tier 8b cost reporting via --mock-inference, arms A–B), Step 5 isolation, Tiers 6/7 codec, plus Scenario C (pytest) and Scenario F (CLI typo). Skip the live-only arms: image payload (4/5), CV batch live (12), routing battery (10a–10c), queue-options live scenarios (A/B/D/E), and the live cost arms (E–G). Master results table at the end.+/temporal-e2e-validate run the full sweep DRY/mock only — no real spend. Mode 1 dry, Mode 2 all tiers in dry/mock (incl. 2c and 2d — both free; Tier 8b cost reporting via --mock-inference, arms A–B), Step 5 isolation, Tiers 6/7 codec, plus Scenario C (pytest) and Scenario F (CLI typo). Skip the live-only arms: image payload (4/5), CV batch live (12), routing battery (10a–10c), queue-options live scenarios (A/B/D/E), and the live cost arms (E–G). Master results table at the end.```Recommended order: run the **no-spend** sweep first to confirm the plumbing cheaply, then the **real-spend** sweep once it's green. The all-in-one is long and runs tiers sequentially (reporting after each); if a live backend isn't configured, that arm stops-and-reports rather than silently passing — see Prerequisites.@@ -62,6 +62,23 @@ Guards the `/validate` dry-run leak (nested controller sub-pipes dispatching to- **Mode-2 Tier 2c** (deployment-faithful: `validate bundle --temporal` over a standalone `PipeBatch` → exit 0 **and** worker idle / no `WfPipeRouter` dispatch). Reached via the prompt above; full GREEN/RED procedure in `references/mode-2-tiers.md` Step 3, Tier 2c.+### Dry-run + validation as ONE in-memory activity — Tier 2d (free)++Sibling of Tier 2c: where 2c proves the *direct* `/validate` sweep never dispatches, 2d proves the *Temporal-dispatched* `/validate` runs the whole sweep **+** graph dry-run inside a single in-process activity (`wf_dry_validate` → `act_dry_validate`) — zero nested dispatch on the workflow history, in-memory tracing (no NDJSON/DDB), `{status map, GraphSpec}` in one round-trip, best-effort graph (`GRAPH: None` on a graph-arm failure, validation still OK). Needs split workers up (workflow lands on the router worker, the activity on the runner).++```text+/temporal-e2e-validate validate the Temporal-dispatched /validate runs as one in-process in-memory activity # Mode 2 Tier 2d+```++- **Submitter:** `.claude/skills/temporal-e2e-validate/scripts/submit_dry_validate.py` (dispatches the wrapper workflow, prints the status map, writes the GraphSpec JSON). Full GREEN/RED procedure + the API arm (real `../pipelex-api` route via TestClient) in `references/mode-2-tiers.md` Step 3, Tier 2d. RED note: the event-log-drop arm signals `GRAPH: None` where GREEN produced a real graph (it no longer exits 1 — the activity's parity best-effort catch degrades it); the content-generator arm's deterministic RED is the Mode-1 leaf-mock test below.+- **Mode-1 pytest companions** (also outside `library_crate/`, so run them by path — the generic "Mode 1 pytest suite" prompt does not pick them up):++ ```text+ timeout 180 .venv/bin/pytest tests/integration/pipelex/temporal/test_dry_validate_activity_in_memory.py tests/integration/pipelex/temporal/test_dry_run_graph_in_process.py -m temporal --timeout=120+ ```++ (Self-contained against the in-process server — `--temporal-server none` default works; covers zero-nested-dispatch on the Temporal history, in-memory tracing, best-effort + bug-propagates arms, no-retry pin, ErrorReport crossing, concurrency.)+### Graph tracing (free)```textdiff --git a/wip/distributed-execution/temporal-e2e-validate-skill.md b/wip/distributed-execution/temporal-e2e-validate-skill.mdindex 82a6b61ff..eae0fedc0 100644--- a/wip/distributed-execution/temporal-e2e-validate-skill.md+++ b/wip/distributed-execution/temporal-e2e-validate-skill.md@@ -9,7 +9,7 @@ For ready-to-paste invocation prompts (per-area + run-it-all-in-one-go), see the`/temporal-e2e-validate` validates that Pipelex pipelines execute correctly across separate Temporal worker processes. Two modes:- **Mode 1** — pytest against a real Temporal dev server with an in-process worker (`SKILL.md`). Fast regression path; folds in the error-handling suite (activity error boundary, workflow error-report full chain, local parity arm).-- **Mode 2** — true 3-process setup (server + separate worker process(es) + submitter). Reference files under `references/`: `mode-2-setup.md` (server + workers), `mode-2-tiers.md` (Tiers 1–16), `routing-battery.md` (Step 8, v1 `activity_queues` routing), `queue-options-battery.md` (Step 9, v2 queue options + worker-runtime profiles).+- **Mode 2** — true 3-process setup (server + separate worker process(es) + submitter). Reference files under `references/`: `mode-2-setup.md` (server + workers), `mode-2-tiers.md` (Tiers 1–16 plus the later-added 2c — direct `/validate` sweep stays in-process, PR #976 — and 2d — Temporal-dispatched `/validate` as one in-memory activity, added with `act_dry_validate`), `routing-battery.md` (Step 8, v1 `activity_queues` routing), `queue-options-battery.md` (Step 9, v2 queue options + worker-runtime profiles).## Full-run result (2026-06-02)diff --git a/wip/dry-run-refactor/README.md b/wip/dry-run-refactor/README.mdindex e4bdeea75..e43b0fe45 100644--- a/wip/dry-run-refactor/README.md+++ b/wip/dry-run-refactor/README.md@@ -16,10 +16,13 @@ Everything below is **not yet done**. Each deferred workstream has its own track### Deferred workstreams — own branches+> **Parts B and C are the two dry-run modes** (full-distribution-with-leaf-mocks, and one-in-process-activity-in-memory). How they relate, what they share, and who builds the common `scoped_content_generator` seam → [`dry-run-modes-master-plan.md`](./dry-run-modes-master-plan.md).+| Workstream | What it delivers | Status / gate | Tracker ||---|---|---|---|-| **Part B — leaf-level run-mode mock** | Move the LIVE/DRY decision down to the cogt leaf so DRY *honors the backend* (DRY-on-Temporal dispatches `act_llm_gen_*` and mocks inside the activity). Retires the "DRY → local" shortcut. (req 1) | Not started. Has **pre-flight human-input items** (run_mode carrier, object-mock fidelity, synthetic-report disposition) — resolve before phase B1. | [`followup-leaf-run-mode-mock.md`](./followup-leaf-run-mode-mock.md) |-| **Part C — distributed validation as a Temporal activity** | The API dispatches the whole validation sweep to a worker as one standalone activity; worker runs it in-process and returns the status map. (req 2) | Not started. **HARD-GATED** on a `temporalio` bump past 1.23.0 + Temporal Cloud/server standalone-activity support — needs human input. Depends on Part A **and** Part B. | [`followup-temporal-validation-activity.md`](./followup-temporal-validation-activity.md) |+| **Part B — leaf-level run-mode mock** | Move the LIVE/DRY decision down to the cogt leaf so DRY *honors the backend* (DRY-on-Temporal dispatches `act_llm_gen_*` and mocks inside the activity). Retires the "DRY → local" shortcut. (req 1) | **COMPLETE (2026-06-10)** on `feature/Mock-activities` (PR #983) — all phases done, Checkpoint E reached, Tier 17 GREEN+RED. | [`followup-leaf-run-mode-mock.md`](./followup-leaf-run-mode-mock.md) |+| **Unified dry run — retire `is_mock_inference` as a mode** | One non-live mode: **dry run** ("no effects": every leaf mocked, no PipeFunc execution, no storage IO). `is_mock_inference` becomes `CogtRunParams.is_mock_usage`, an **internal** sub-flag of DRY (no public CLI) that flips usage reporting from zero-token/suppressed to non-zero/rendered — what the cost-assembly tests (incl. `temporal-e2e-validate` Tier 8b) consume. **Supersedes eng-review decision D8.** | **IMPLEMENTED (2026-06-10)** on `feature/Mock-activities` — plan + as-built: [`plan-unified-dry-run.md`](./plan-unified-dry-run.md). | [`design-unified-dry-run.md`](./design-unified-dry-run.md) |+| **Part C — dry-run + validation as one in-process Temporal activity** | The API dispatches the whole job — validation sweep **+** graph-producing dry-run — to a worker as one activity that runs in-process and traces the graph **in memory** (no DynamoDB round-trip, no NDJSON, no usage/cost). Returns the status map **+** `GraphSpec`. (req 2) | **Built** on `feature/Dry-run-as-temporal-activity` (+ `pipelex-api` `feature/Update-dry-run-api`) — all phases through Checkpoint F (phase record in that branch's `TODOS.md`); verified by Mode-1 isolation tests and Tier 2d of `temporal-e2e-validate` (activity + API arms, GREEN+RED on a real 3-process stack). As-built record: [`consolidation-as-built.md` § Part C](./consolidation-as-built.md). The `pipelex-api` branch needs a pipelex release/rev bump (> v0.32.1) before merge. Deferred: Phase G0 standalone activity (`temporalio` bump); retiring the old worker-workflow graph path; moving `pipe_structures` into the activity result. | [`followup-temporal-validation-activity.md`](./followup-temporal-validation-activity.md) || **Run-outcome seam** *(optional)* | Have the run primitive *return* its classified outcome so `BundleValidator` consumes a typed outcome instead of re-catching + chain-walking the re-raised exception (D-plan D6). | Not started. **Optional tightening, not a bug** — gated on "is it worth the cross-backend churn?" Default: leave it. | [`followup-run-outcome-seam.md`](./followup-run-outcome-seam.md) |### Cross-repo@@ -32,12 +35,14 @@ Everything below is **not yet done**. Each deferred workstream has its own track### Hardening & cleanups — anytime, small+Part-C operational hardening (heartbeat/cancellation, CPU-bound activity body, worker-vs-API config drift, double dry-run of the main pipe, `_pipe_source_map` scoping, selected-pipe wire field, per-queue dispatch tuning) has its own tracker: [`followup-dry-validate-hardening.md`](./followup-dry-validate-hardening.md).+| Item | Where | Note ||---|---|---|| **Per-sweep unique `pipeline_run_id`** | `bundle_validator.validate_pipes` | The report registry + per-pipe graph-trace read are keyed by the **constant** `SpecialPipelineId.DRY_RUN_UNTITLED`, so two concurrent in-process sweeps collide (2nd `open_registry` raises "already exists"; interleaved `close_registry` can drop the other's registry). Mint a unique id per sweep and thread it through `open_registry` + `prepare_pipe_job` + `close_registry` — one fix also scopes the per-pipe trace read to an empty dir. **Rides with Part C** (no concurrent sweeping exists in-process today). `close_registry` is already hardened to `pop(..., None)`. || **Shared `acquired_library(...)` helper** | ~6 sites | The "restore outer current-library first, then teardown" `finally` is copied across `validate_bundle.py` (×4), `execution_seams.acquire_library`, and `bundle_validator.acquire_and_validate` (now via `clear_current_library`). Extract one context manager / helper so the load-bearing ordering lives in one place. || **`acquire_and_validate` self-teardown guard** | `bundle_validator.acquire_and_validate` | If a caller passes an explicit `library_id` equal to the already-current library, the `finally` restores-then-tears-down the same library (dangling current pointer). Guard `prev_library_id != acquired_id`, or document that `library_id` must not alias the caller's current library. Default `library_id=""` is safe, so latent. |-| **`__init__` content-generator injection seam** | `bundle_validator.BundleValidator.__init__` | Hardcodes `PipeRun(PipeRouter(ObserverNoOp()))` (correct layer, deliberately not `get_pipe_run()`). Part C's `scoped_content_generator` will need a seam to force the inline generator — revisit when Part C lands. |+| ~~**`__init__` content-generator injection seam**~~ | `bundle_validator.BundleValidator.__init__` | **Resolved by Part C** — no constructor seam was needed: `validate_pipes` wraps its sweep loop in `scoped_content_generator(ContentGenerator.make_inline())` (ContextVar scope alongside `scoped_pipe_router`), forcing the inline generator under any hub. || **Real-config FAILURE / `allowed_to_fail` integration test** | `tests/integration/.../test_bundle_validator.py` | Integration coverage is SUCCESS-path + real cross-package SKIPPED. The namespaced `allowed_to_fail` match is pinned only at unit level (mocked config). Add a real-config FAILURE-match test (e.g. load `failing_pipelines.mthds`). || **Success-path report-registry leak** *(pre-existing)* | `runner.execute_pipeline` finally | Production never closes the report registry on the **success** path either (the `finally` tears down tracer / event-log / library but not the registry). Predates this work; a registry-ownership change spanning setup→run→report. Out of this consolidation's scope, but worth tracking. || **Test-style nits** *(very low)* | `pipelex-api` route tests + pipelex `test_agent_output` | Opportunistic: share the duplicated `_build_client()` across the two route-test modules; fold the overlapping body asserts in the runner-with-signatures test; trim the cosmetic domain assertion. None blocking. |diff --git a/wip/dry-run-refactor/consolidation-as-built.md b/wip/dry-run-refactor/consolidation-as-built.mdindex 0b05f520d..723868f10 100644--- a/wip/dry-run-refactor/consolidation-as-built.md+++ b/wip/dry-run-refactor/consolidation-as-built.md@@ -55,6 +55,17 @@ Every `validate` surface reports each pipe by its **namespaced `pipe_ref`** (`do- **Kept:** `pipe_run/dry_run_pipeline.py` — a still-live "dry-run a whole bundle from MTHDS content → GraphSpec" helper (callers: `pipelex/graph/graph_rendering.py` and `pipelex-api`'s validate route). It already uses the unified DRY-mode `PipelexRunner` path. Not inlined into `graph_rendering.py` (that would break the cross-repo import).- The two relocated mock helpers (`convert_to_working_memory_format`, `convert_stuff_spec_to_typed_named`) now live as classmethods on `WorkingMemoryFactory`.+## Part C as built — dry-run + validation as ONE in-process Temporal activity (Mode 1)++Shipped on `feature/Dry-run-as-temporal-activity` (+ the `pipelex-api` branch `feature/Update-dry-run-api`). Full phase-by-phase record in the branch's `TODOS.md`; design in [`followup-temporal-validation-activity.md`](./followup-temporal-validation-activity.md).++- **The two-instance fix:** `pipelex.hub.scoped_event_log(event_log)` + `get_event_log_override()` — both the tracer write side (`pipeline_run_setup.py`) and the assembly read side (`tracing_assembly.py`) prefer the scoped instance over `make_event_log`, so a run traces into ONE shared `InMemoryEventLog`. A set override implies tracing-enabled; the scope owner keeps the instance's lifecycle.+- **The in-process graph dry-run:** `dry_run_pipe_in_process(pipe, *, library_id)` in `pipe_run/dry_run_pipeline.py` — `prepare_pipe_job` (DRY + mock_inputs + generate_graph) + a local `PipeRun` under `scoped_event_log` + `scoped_pipe_router` + the new `hub.scoped_content_generator` (inference leaves resolve an inline `ContentGeneratorDry` instead of `ContentGeneratorInWorkflow`; `get_content_generator()` prefers the override). `BundleValidator.validate_pipes` also carries the content-generator scope, so the sweep stays in-process post-Part-B everywhere.+- **The activity:** `temporal/tprl_pipe/act_dry_validate.py` — composes `validate_bundle` (the SAME function the direct route calls, so both backends share the categorized `ValidateBundleError` 422 contract) + `dry_run_pipe_in_process` against the same once-loaded library; graph is best-effort behind a narrow expected-failure catch (`DryRunError`/`PipeRunError`/`PipeRouterError`/`ValidationError`/`FactoryException` → `graph_spec=None`; everything else propagates). Dispatched via the one-step wrapper `wf_dry_validate.py::WfDryValidate` (explicit activity timeout + non-retryable validation error types) through the submitter helper `dry_validate_dispatch.py::dispatch_dry_validate` (workflow-tier `maximum_attempts=1` — a deterministic validation failure must not re-run). Registered in `temporal/tasks.py` (`PackName.PIPE`).+- **The API route** (`pipelex-api/api/routes/pipelex/validate.py`): Temporal-enabled `/validate` dispatches FIRST (one round-trip, identical 422 via the recovered `ErrorReport`), then re-parses blueprints and builds `pipe_structures` from a load-only `acquire_library`. Direct mode unchanged.+- **Verification:** Mode-1 isolation tests (`tests/integration/pipelex/temporal/test_dry_validate_activity_in_memory.py` — zero nested dispatch pinned on the Temporal history) · Tier 2d in the `temporal-e2e-validate` skill (activity + API arms, GREEN + RED-proven on a real 3-process stack) · `pipelex-api/tests/unit/test_validate_temporal_dispatch.py` (both backends).+- **Deferred:** Phase G0 standalone activity (`temporalio` bump) · retiring the old worker-workflow graph path for `/validate` · moving `pipe_structures` into the activity result (drops the API-side load-only acquisition).+## Where the code lives- `pipelex/pipeline/execution_seams.py` — `acquire_library`, `prepare_pipe_job`.diff --git a/wip/dry-run-refactor/deferred-unified-dry-run-review.md b/wip/dry-run-refactor/deferred-unified-dry-run-review.mdnew file mode 100644index 000000000..5dcc84c0f--- /dev/null+++ b/wip/dry-run-refactor/deferred-unified-dry-run-review.md@@ -0,0 +1,27 @@+# Deferred items — unified dry run review rounds (PR #986)++> Findings from the PR #986 review rounds (greptile/cubic bots + the `/review` specialist/adversarial passes, 2026-06-10) that were **verified real but deliberately NOT fixed** — each is a design tradeoff or a guard for a currently-unreachable path, not a silent bug in shipped behavior. Fixed-on-the-spot findings are not listed (see the PR's commits).++## Run-mode safety & contracts++- **Typed-error wrap of the `cogt_run_params` mint + factory contract check.** The derived property and `PipeRunParamsFactory` raise pydantic `ValidationError` / `ValueError` on the LIVE+`is_mock_usage` violation. Inside Temporal *workflow* code an unconverted exception retries forever (the known submitter-hang class) — but post-hardening the illegal state requires deliberately bypassing validators (`model_copy(update=...)` on exactly those fields; attribute writes are blocked by `Field(frozen=True)`). Guarding that is guarding a near-impossible path; revisit only if a real producer of the illegal state appears. Same family: the factory's `ValueError` surfacing as an untyped 500 through `pipelex-api` once that repo adopts `is_mock_usage` — decide a typed INPUT-domain error at that adoption point, not before.+- **Keyless-boot trust boundary narrowed (D4 consequence).** Pre-Part-B, a keyless boot installed a mock generator globally, so even direct `ContentGeneratorProtocol` callers got mocks. Now generator selection is backend-keyed only and forced-DRY lives at `make_run_params` — an embedder calling the protocol directly with a hand-built LIVE carrier under a keyless boot whose lenient backend load found credentials *can* reach a real provider. This is the submitter-side-contract-only design decided in eng review D4; flagging the contract change here for visibility. A `is_dry_run_forced()` guard at `get_content_generator()` would restore the old blanket but re-couples generator selection to run mode — rejected for now.+- **Forced-DRY silently downgrades an explicit LIVE request (warning only).** A misconfigured production boot (keyless by accident) returns synthetic structured outputs to clients with only a server-side `log.warning`. Options: fail loud when the LIVE request was *explicit* (coerce only defaulted modes), or stamp `run_mode` into client-visible output metadata. Interacts with the already-deferred `PIPELEX_FORCE_DRY_RUN_MODE` env-var cleanup (see `wip/dry-run-refactor/README.md` open table) — decide together.+- **`hub` forced-DRY flag is process-global, not ContextVar-scoped.** Consistent with the other hub singletons, but it guards spend; if embedded re-boot-while-running ever becomes a supported pattern, move it to the ContextVar style of `scoped_content_generator`.++## Dry-leaf behavior++- **`stamp_mock_main_coordination` can corrupt a user item class that happens to have a constrained `pipe_code` field** (direct attribute write, no validation), and the resulting `DryRunObjectFidelityError` remedy ("declare `examples`/`mock_format`") cannot fix a framework-injected value. Niche (requires a user output class with a constrained `pipe_code`), and the stamp is load-bearing for bundle dry-validation (D3). Candidate fix: pin the field through `build_mock_object(pipe_code="mock_main")` for the first item (runs validators) or scope the stamp to the builder's spec classes.+- **Ambient `dry_run_config.nb_extract_pages` read at two leaves** (extract + render-page-views) — under Temporal these run in different activities; heterogeneous worker configs would mismatch counts and trip the generator's length check. Already tracked since the B1 review round (`followup-leaf-run-mode-mock.md`); the D11-style fix (carry the resolved count on the assignments) stands. Note: the "both `image_uri` and `document_uri`" variant of this finding is impossible — `ExtractInput` validates exactly-one.+- **Inline page-view count mismatch raises bare `ValueError`; the in-workflow twin raises `ContentGenerationError`.** Cross-backend error-type inconsistency on the same fault (pre-existing). Aligning changes error classification on a live path (`ValueError` is in `non_retryable_error_types`); do it together with the nb_extract_pages fix above.+- **Dry image mocks hardcode `mime_type="image/jpeg"` + 1024×1024** while `dry_run_config.image_urls` includes a `.png` — mock metadata can contradict the URL it points at (pre-existing, carried from `ContentGeneratorDry`). Derive mime from the URL suffix if a consumer ever keys on it.++## Reporting & surfaces++- **Synthetic usage is discriminated only by sentinel model-name strings** (`dry_run` / `mock_usage`, `unit_costs={}`). Cost stays $0 by construction, but a downstream aggregator summing `nb_tokens_by_category` without filtering the sentinels would mix synthetic tokens into real usage metrics. A structural `is_synthetic` marker on the usage record is the durable fix — wire-format change, do it when a real aggregator consumer appears (Temporal not in prod).+- **Validation sweeps now reach the `exec()` codegen path.** `BundleValidator` / `dry_run_pipe_in_process` swapped `ContentGeneratorDry` (original-class mocks, no codegen) for the inline `ContentGenerator`, whose dry object leaves run schema-reconstruction (datamodel-code-generator → `exec()`). The hosted `/validate` surface — keyless, typically less gated than execution — now exercises the codegen pipeline on user-bundle-derived schemas. The factory's own docs say its containment is defense-in-depth, not a sandbox; its existing hardening TODO (programmatic `create_model()` or subprocess sandbox) gains priority from this surface expansion.++## Test consolidation candidates (nice-to-have)++- **Fidelity re-validation pattern exists in 4 hand-rolled copies** (`_revalidate_against_object_class` ×2 + the inline search-structured variants in both generators, which already disagree on live-arm behavior — a deliberate boundary decision recorded in `followup-leaf-run-mode-mock.md` deferred items). Consolidating crosses the cogt/temporal layering; revisit if a fifth copy appears.+- **DRY + `should_include_page_views` combined path untested** — both leaves currently agree on length only because they read the same config key; rides with the nb_extract_pages assignment-carried-count fix.diff --git a/wip/dry-run-refactor/design-unified-dry-run.md b/wip/dry-run-refactor/design-unified-dry-run.mdnew file mode 100644index 000000000..3c5a6f61b--- /dev/null+++ b/wip/dry-run-refactor/design-unified-dry-run.md@@ -0,0 +1,113 @@+# Design — Unified dry run (retire `is_mock_inference` as a mode; add internal `is_mock_usage` flag)++> **Status: IMPLEMENTED (2026-06-10) — see [`plan-unified-dry-run.md`](./plan-unified-dry-run.md) for the plan and as-built record.** Design agreed with the user (2026-06-10). This design **supersedes eng-review decision D8** of [`followup-leaf-run-mode-mock.md`](./followup-leaf-run-mode-mock.md) ("is_mock_inference KEPT as thin reportable-mock"): the *mode* is retired; the *capability* (non-zero synthetic usage) survives as a secondary flag on dry run. It also resolves the "fate of `is_mock_inference`" item tracked in `wip/registry/deferred-followups.md`.+>+> **Baseline:** the as-built state of `feature/Mock-activities` (PR #983, Part B complete through Checkpoint E) + the Part C validation activity (`feature/Dry-run-as-temporal-activity`). Read [`followup-leaf-run-mode-mock.md`](./followup-leaf-run-mode-mock.md) and [`followup-temporal-validation-activity.md`](./followup-temporal-validation-activity.md) for how that baseline works.++## 1. How we got here (why two modes exist today)++- **The old dry run** predated the leaf-mock work: `--dry-run` swapped in `ContentGeneratorDry` *pre-dispatch*, so a dry run never entered Temporal activities. That mode is dead — Part B deleted `ContentGeneratorDry` and moved the mock to the cogt leaf.+- **`--mock-inference`** was added as an interim trigger (registry branch, `fix/For-API-update` Phase 5) because tests needed a *dispatching* fake: a LIVE run whose LLM leaves fake the call with non-zero synthetic usage, so the cross-worker cost-report assembly could be validated cheaply. At the time it was the only non-live mode that reached the activities.+- **Part B (this branch) unified the foundation**: `run_mode` now rides `CogtRunParams` on every assignment, every inference leaf branches at the leaf, and `--dry-run` on Temporal genuinely dispatches and mocks inside activities (Tier 17). Result: **two modes that both dispatch and both mock at the leaf**, separated only by usage reporting, leaf coverage (mock-inference is LLM-only), and a few pipe-tier dry behaviors. Two names for nearly one thing — actively confusing, as the run-modes doc work surfaced.++## 2. The design++### 2.1 One non-live mode: **dry run**++There is exactly one non-live run mode for pipeline runs: `run_mode=DRY` (`--dry-run`). Its defining contract is **no effects**:++- no AI provider calls, no API cost, no API keys needed;+- no storage IO (mocks built above the `*_and_store` step — unchanged D10);+- no user-code execution: `PipeFunc` does not execute the user's Python; `PipeCompose` keeps its dry arm (exact compose-dry details deliberately deferred — see §5);+- zero-token synthetic usage, cost report suppressed — *by default* (see §2.3).++The name is "dry run" (not "mock inference") precisely because the contract is broader than inference mocking: it is the no-effects mode.++### 2.2 Dry run is orthogonal to backend (unchanged)++Part B's deliverable holds as-is:++- **DRY × direct**: leaves mock inline in-process.+- **DRY × Temporal**: the run dispatches the *real* workflow/activity tree (child workflows, `act_*_gen_*` on workers) and the leaf mocks inside each activity. This is the mode that exercises the distribution machinery (dispatch, serialization, routing, cross-worker graph tracing) at zero cost with zero credentials.+- **The validation sweep** (`validate_bundle` / `act_dry_validate`) is *not* a third mode: it is dry run with the backend pinned in-process by the existing ContextVar scopes (`scoped_pipe_router`, `scoped_content_generator`, `scoped_event_log`), batched over all pipes, with mock inputs always on and the graph traced in memory. Unchanged by this design.++### 2.3 `is_mock_usage` — a secondary, internal flag on dry run++`CogtRunParams.is_mock_inference` is replaced by `CogtRunParams.is_mock_usage: bool = False`, with new semantics:++- It is a **sub-flag of DRY**, not a mode: it only has meaning when `run_mode=DRY`. Setting it with `run_mode=LIVE` is a contract violation (proposed: model-validator rejection — fail loud; confirm at plan time).+- `is_mock_usage=False` (default): leaves report **zero-token** synthetic jobs → `AggregatedCosts.has_reportable_usage` is False → cost report suppressed. Exactly today's dry-run reporting.+- `is_mock_usage=True`: leaves report **non-zero** synthetic usage (the deterministic sentinel counts, e.g. INPUT 100 / OUTPUT 50, `unit_costs={}` so cost is $0) → the cost report **renders**. This is what cross-worker cost-report assembly tests consume — they assert on non-zero aggregated values, not just on report presence.+- **Internal only — no public CLI flag.** Exposed through the Python surface (`PipelexRunner` / `execute_pipeline` / `PipeRunParamsFactory.make_run_params`) for tests and embedders. No documented CLI option; CLI users get `--dry-run` and nothing else.+- **Tests must still trigger it end-to-end**, including the `temporal-e2e-validate` skill's Mode-2 (3-process, CLI-driven) tiers, with non-zero values asserted. Since Mode 2 drives the real CLI, the design proposes a **hidden CLI option** (typer `hidden=True`, e.g. `--mock-usage`, requires `--dry-run`) as the test trigger: present for scripts, absent from `--help` and docs. Alternative if hidden flags are unwanted: a tiny Python submitter under `temporal/test_extras`. Decide at plan time; the hidden flag is the lowest-friction proposal.++### 2.4 Leaf coverage becomes uniform++Mock-inference's LLM-only restriction dies with the mode. Dry run already mocks **every** leaf (LLM text/object/list, img-gen, extract, render-page-views, search, templating). Consequently:++- `MockInferenceUnsupportedError` and its fail-loud guards at the img/extract/search leaves are **deleted** — their entire reason was mock-inference's partial coverage. With `is_mock_usage` riding on DRY, a "reportable" run covers every operator a dry run covers.+- Non-LLM leaves keep their current dry behavior (synthetic outputs, no usage reporting). Whether `is_mock_usage=True` should ALSO emit synthetic usage for non-LLM leaves (img-gen/extract jobs have their own usage records in live runs) is left to plan time; the minimal design keeps reportable synthetic usage LLM-only, matching what the cost-assembly tests actually assert today.++## 3. Delta inventory — what changes relative to the as-built baseline++### 3.1 Carrier and factory++- `CogtRunParams`: `is_mock_inference` → `is_mock_usage` (semantics per §2.3; frozen/extra-forbid/required-run_mode unchanged). The `is_mock_built` property collapses — mock-built is now simply `run_mode.is_dry`; keep or inline it (the fidelity guards key off it).+- `PipeRunParamsFactory.make_run_params`: parameter re-keyed; stays the single writer. The boot forced-DRY path simplifies — the "forced-DRY swallows an explicit `--mock-inference`" warning disappears (there is nothing to swallow anymore).+- `PipelexRunner` / `execute_pipeline` / `prepare_pipe_job`: `is_mock_inference` param → `is_mock_usage`.++### 3.2 Leaves (`pipelex/cogt/content_generation/`)++- Each LLM leaf's **dual branch collapses to one**: `if cogt_run_params.run_mode.is_dry → dry helper`; the dry helper selects its report func on `is_mock_usage` (zero-token `report_dry_llm_job` vs non-zero `report_mock_usage_llm_job`, renamed from `report_mock_inference_llm_job`). The object/list variants are already parameterized via `_leaf_gen_object(report_func=...)`; the text variant adopts the same shape.+- **Delete** `mock_llm_gen_text` / `mock_llm_gen_object` / `mock_llm_gen_object_list` / `_mock_text` and the `MOCK INFERENCE` marker string. One marker family remains (`DRY RUN: …`). Tests grepping the mock marker must re-key.+- Sentinel identifiers: `dry_run` model name stays for zero-token; `mock_inference` model name/id → `mock_usage` (or fold to a single sentinel with two token payloads — plan-time detail; tests that count/filter by sentinel follow).+- `stamp_mock_main_coordination`: the "mock-inference deliberately does NOT stamp" arm dies with the mode; the dry object-list leaf stamps regardless of `is_mock_usage` (harmless — the stamp only matters to bundle dry-validation, and its docstring sheds the mock-inference caveat).+- `dry_mock.py` module docstring rewritten for the one-mode world.++### 3.3 Errors++- `MockInferenceUnsupportedError`: **deleted** (guards at img/extract/search leaves removed). Error page removed on regeneration.+- `MockInferenceObjectFidelityError`: fires on mock-built objects, which is now DRY-only — **rename candidate** (e.g. `DryRunObjectFidelityError`); message/remedy text already generalized in B1. Decide at plan time; regenerate error pages either way.+- `DryRunMockBuildError`: unchanged.++### 3.4 CLI++- `--mock-inference` removed from `run pipe` / `run bundle` / `run method`.+- `validate_run_flag_combination` loses the mutual-exclusion arm; only "`--mock-inputs` requires `--dry-run`" remains (plus, if the hidden-flag proposal is taken, "`--mock-usage` requires `--dry-run`").++### 3.5 Tests++- Leaf unit tests: dual-arm (dry vs mock) tests merge into dry tests parameterized over `is_mock_usage`; reporting assertions cover both token payloads (zero suppressed / non-zero rendered).+- `test_mock_inference_temporal.py` (wire-survival of the flag): re-key to `is_mock_usage` on the carrier.+- Fidelity tests (D6 arms): re-key from `is_mock_built`'s two triggers to DRY-only.+- Keyless-boot tests: drop the swallow-warning arm.+- **`temporal-e2e-validate` skill**: Tier 8b (the `--mock-inference` cost-assembly arms — cross-worker usage aggregated into one rendered submitter report) re-points to the dry-run + `is_mock_usage` trigger (hidden flag or submitter script per §2.3) and keeps asserting **non-zero** aggregated values. Tier 17 prose drops the "distinct from `--mock-inference`" contrast and instead states the `is_mock_usage` sub-flag. Routing battery / scope manifests mentioning `--mock-inference` updated.++### 3.6 Docs++- `docs/building-methods/pipes/run-modes-and-backends.md`: the 3×2 matrix collapses to **one mode axis** (live / dry) × two backends, plus the validation-sweep section; mock inference disappears as a user-facing concept (the internal flag is deliberately undocumented, or mentioned only as an internal testing affordance).+- `docs/under-the-hood/dry-run-mock-generation.md`, `docs/features/validation-dry-run.md`, `docs/features/cost-tracking.md`, `docs/tools/cli/run.md`: purge `--mock-inference`.+- Error pages regenerated (`make gep`).+- CHANGELOG entry under `[Unreleased]` (breaking: `--mock-inference` removed; no deprecation period per our principles).++### 3.7 Non-concerns++- **Wire/backward compatibility: none.** The Temporal integration has never shipped to prod; the `CogtRunParams` field rename needs no migration shims.+- **`pipelex-api` / platform repos**: grep for `mock_inference` at plan time; the public runner API params carry `is_mock_inference` today via `_run_core` → runner — the public surface change must ride a pipelex version bump like any other.++## 4. What does NOT change++- The leaf-mock foundation: `CogtRunParams` carrier (frozen, single-writer, stamped on every assignment, serializes across the activity boundary), the `dry_*` helpers, schema-built object mocks (`SchemaToModelFactory` + `DryRunFactory`), `DryRunMockBuildError`, the fidelity re-validation.+- Backend orthogonality and Tier 17's meaning (DRY-on-Temporal dispatches and mocks inside activities).+- The validation sweep and `act_dry_validate` (Part C) — shape, scopes, contracts all untouched.+- The forced-DRY keyless boot (`Pipelex.make(needs_inference=False)` → every run forced DRY) — it only loses the swallow-warning edge case.+- `--dry-run` and `--mock-inputs` CLI surface.++## 5. Deliberately deferred to plan time++- **PipeCompose dry-arm specifics** — the user explicitly parked this ("we'll see about those details later"). Current behavior (construct-mode `PipeComposeError` fallback, direct template rendering) stands until then.+- The exact e2e trigger mechanism for `is_mock_usage` (hidden CLI flag vs test submitter script) — §2.3 proposal: hidden flag.+- Whether `is_mock_usage=True` + `run_mode=LIVE` is a validation error (proposed) or silently ignored.+- `MockInferenceObjectFidelityError` rename.+- Sentinel-model naming consolidation (§3.2).+- Whether non-LLM leaves emit synthetic usage under `is_mock_usage=True` (§2.4 — minimal design says no).diff --git a/wip/dry-run-refactor/dry-run-modes-master-plan.md b/wip/dry-run-refactor/dry-run-modes-master-plan.mdnew file mode 100644index 000000000..f830dfacb--- /dev/null+++ b/wip/dry-run-refactor/dry-run-modes-master-plan.md@@ -0,0 +1,69 @@+# Master plan — two dry-run modes on one foundation++> **What this is:** the overview tying together the two dry-run modes we want, how each is achieved, what they share, and who builds what. Neither per-mode tracker owns the *relationship* — this doc does. The per-mode plans:+>+> - **Mode 1 (in-memory activity)** → design [`followup-temporal-validation-activity.md`](./followup-temporal-validation-activity.md) · executable plan [`../../TODOS.md`](../../TODOS.md) · branch `feature/Dry-run-as-temporal-activity` (active).+> - **Mode 2 (full distribution, leaves mocked)** → [`followup-leaf-run-mode-mock.md`](./followup-leaf-run-mode-mock.md) (done — landed on this branch via PR #986).+> - **Design source:** [`D-plan.md`](./D-plan.md) §3.5 (run-mode ⟂ backend) + §4.8 (leaf-level mock) + §4.9 (validation activity).++## The two modes, in one sentence each++1. **In-memory activity (req 2).** The whole dry-run + validation runs **in-process inside a single Temporal activity** on a worker — nothing nested is dispatched — tracing the graph **in memory**. Purpose: let the hosted API offload "validate this bundle + give me its graph" to a worker, cheaply, with no DynamoDB round-trip and no AI calls.++2. **Full distribution, leaves mocked (req 1).** A dry-run goes through the **real** Temporal path — `TemporalPipeRun` → `WfPipeRouter` → child workflows → `act_llm_gen_*` / extract / img-gen activities — and the **leaf inside each dispatched activity** mocks instead of calling the model. Purpose: exercise the distribution machinery (dispatch, scheduling, serialization, routing, cross-worker propagation, graph assembly) without AI cost or latency.++## Why they don't conflict — one foundation, two backends++Both are the **same `run_mode=DRY`** run; they differ only in **backend** (where the leaf executes), which is the D-plan §3.5 thesis: *run mode is orthogonal to execution backend*. The foundation that makes both clean is the **leaf-level mock** (Part B / D-plan §4.8): mock at the cogt leaf keyed on `run_mode`; the content generator is run-mode-agnostic and only decides *where* the leaf runs.++| | Controllers (nested sub-pipes) | Cogt leaf (LLM/extract/img-gen) | Graph tracing | Result |+|---|---|---|---|---|+| **Mode 1 — in-memory activity** | forced **in-process** via `scoped_pipe_router` | forced **inline** via `scoped_content_generator` → mocks in-process | `scoped_event_log(InMemoryEventLog())` → in memory | one activity, zero nested dispatch, zero external I/O |+| **Mode 2 — full distribution** | hub default = **Temporal router** → child workflows | hub default = **`ContentGeneratorInWorkflow`** → dispatches `act_llm_gen_*`, mocks **inside** the activity | hub default = `temporal_dynamodb` (cross-worker assembly) | real distributed path, no AI spend |++The **only divergence is which content generator / router the run uses** — and both are selected by **per-call ContextVar overrides** (coroutine-local, restore-on-exit), so concurrent runs of different modes on one worker never cross-contaminate. Mode 1 *sets* the overrides; Mode 2 *leaves the hub defaults*. They are additive, not exclusive.++**Proof they already coexist:** the LLM slice of Mode 2 first shipped as `is_mock_inference` (registry branch, #967) — `run_mode` stayed LIVE so operators dispatched `act_llm_gen_*` normally while the leaf faked the call. The unified dry run (#986) has since retired that flag into the internal `is_mock_usage` DRY sub-flag; the coexistence holds, now keyed on `run_mode=DRY` end to end.++## The shared seam — `scoped_content_generator` (build once)++The single place the two modes touch. Under a Temporal-enabled hub, `get_content_generator()` returns `ContentGeneratorInWorkflow` **globally** (boot-time selection, `pipelex.py:370-385` — not contextual). So:++- **Mode 2 wants that default** — the in-workflow generator *is* how it dispatches leaf activities.+- **Mode 1 must override it to inline** — otherwise, once Part B routes the DRY mock through `get_content_generator()` at the leaf, Mode 1's in-process activity would dispatch `act_llm_gen_*` and break.++So `scoped_content_generator` (a hub ContextVar + context manager, mirroring `scoped_pipe_router` in `hub.py`, with `get_content_generator()` preferring the override) is needed by Mode 1 and is the seam Part C originally specced. **Decision (2026-06-09): build it now, in Mode 1's branch**, so Mode 1 is correct regardless of when Part B lands and the seam exists once for both.++### The ContextVar scope family (`pipelex/hub.py`)++| Scope | Exists? | Used by | Forces |+|---|---|---|---|+| `scoped_current_library` | yes (`hub.py`) | both / general | which library is current |+| `scoped_pipe_router` | yes (`hub.py`, PR #976) | Mode 1 | nested controllers run in-process, not via the Temporal router |+| `scoped_content_generator` | yes (`hub.py`, this branch) | Mode 1 | the leaf uses the inline generator, not the in-workflow one |+| `scoped_event_log` | yes (`hub.py`, this branch) | Mode 1 | graph trace events share one in-memory log across emit + assemble |++## Build order — who builds what++**Mode 1 — branch `feature/Dry-run-as-temporal-activity` (active now).** Full phase plan + checkpoints in [`../../TODOS.md`](../../TODOS.md):++- Phase 1 — **done.** `scoped_event_log` + shared in-memory tracing (fixed the two-instance emit/assemble problem: `pipeline_run_setup.py` write vs `tracing_assembly.py` read each called `make_event_log` separately; the in-memory log now shares one instance).+- Phase 2 — **done.** In-process graph dry-run safe under a Temporal hub (`dry_run_pipe_in_process` in `pipe_run/dry_run_pipeline.py`); built `scoped_content_generator` and wraps the run in it + `scoped_event_log` + `scoped_pipe_router`.+- Phase 3 — **done.** The `act_dry_validate` activity (sweep + best-effort in-memory graph dry-run) + the `WfDryValidate` one-step wrapper workflow + `dispatch_dry_validate` + worker registration (`temporal/tasks.py`) + isolation test + **`temporal-e2e-validate` Tier 2d** (3-process distributed proof).+- Phase 4 — **pending.** API dispatch (cross-repo `pipelex-api` `/validate`), gated on a pipelex release bump.+- Phase G0 *(optional, deferred)* — `temporalio` bump → true standalone activity (a runtime optimization over the wrapper workflow). **All decisions resolved 2026-06-09; no human gate remains** (dispatch = wrapper workflow on current `temporalio`).++**Mode 2 — done (landed on this branch via PR #986)** ([`followup-leaf-run-mode-mock.md`](./followup-leaf-run-mode-mock.md)):++- B1 — **done.** Generalized the leaf mock from `is_mock_inference` (LLM-only) to `run_mode=DRY` across all leaves (LLM, extract, img-gen, templating); shared `cogt/content_generation/dry_mock.py`.+- B2 — **done.** Collapsed each operator's `_dry_run_pipe` so DRY routes through the hub content generator (no more hardcoded `ContentGeneratorDry()`); `is_mock_inference` retired into the internal `is_mock_usage` DRY sub-flag. **Consumes `scoped_content_generator`** (built once by Mode 1).+- B3 — **done.** Temporal + DRY verified e2e via **`temporal-e2e-validate` Tier 17**: activities dispatched, leaves mock inside them, no real IO, no API keys (the req-1 gate).++**Net:** Mode 1 builds the two new scopes and lands the in-memory activity; Mode 2 builds the all-leaf mock and verifies the distributed-DRY path, reusing the content-generator scope. Neither blocks the other; the shared seam is built once.++## Acceptance — each mode gated by a specific distributed `temporal-e2e-validate` tier++Both modes must be proven in the real 3-process topology (server + split workers + submitter), not only in unit/integration. Each adds a tier to the repo's `temporal-e2e-validate` skill following the Tier 2c precedent (a `references/mode-2-tiers.md` scenario with GREEN/RED + worker-log strong-check, a Mode-1 pytest companion, a Step-7 master-table row).++- **Mode 1 (req 2) — Tier 2d.** A Temporal-dispatched `/validate` runs the whole sweep + graph dry-run inside **one in-process activity** (zero nested dispatch — strong worker-idle check), traces the graph **in memory** (no NDJSON/DDB), and returns `{status map, GraphSpec}` in one round-trip; best-effort graph → `None` on failure; direct mode unchanged (req 3). Spec in [`../../TODOS.md`](../../TODOS.md) § Distributed verification.+- **Mode 2 (req 1) — Tier 17.** A `--temporal --dry-run` run **dispatches** `act_llm_gen_*` (+ extract/img-gen) to the worker and **mocks inside** each activity — no real IO, no API keys, zero-token suppressed usage, cross-worker graph still assembles. Flips today's Tier 8 "dry-run never dispatches" note. Spec in [`followup-leaf-run-mode-mock.md`](./followup-leaf-run-mode-mock.md) § Distributed verification.diff --git a/wip/dry-run-refactor/followup-dry-validate-hardening.md b/wip/dry-run-refactor/followup-dry-validate-hardening.mdnew file mode 100644index 000000000..182b47ebb--- /dev/null+++ b/wip/dry-run-refactor/followup-dry-validate-hardening.md@@ -0,0 +1,47 @@+# Follow-up: operational hardening of the dry-validate activity path++Deferred items from the post-implementation review of `feature/Dry-run-as-temporal-activity` (Part C, `act_dry_validate` + `wf_dry_validate` + `dispatch_dry_validate`). Each was verified against the code at review time; none blocks shipping — they are design tradeoffs or pre-existing shapes that deserve a deliberate pass, not a reflexive patch. Cheap fixes from the same review (teardown suppress-secondary, `workflow_execution_timeout=12min`, close-contract docs, tracer-open ordering, graph-arm parity, dropping `library_dirs`) were applied on the branch and are NOT listed here.++## 1. No heartbeat: a hung validation cannot be cancelled and outlives its timeout++`act_dry_validate` never calls `activity.heartbeat()` and `WfDryValidate` sets no `heartbeat_timeout`. In the Temporal Python SDK, cancellation is only delivered to an async activity through heartbeating — so when the 5-minute `start_to_close_timeout` fires, the *workflow* gives up but the worker coroutine keeps running to completion: the activity slot, the loaded library, and the `GraphTracerManager` entry are held until the run finishes on its own (or forever, if a pathological bundle spins), and the retry attempt starts a second copy of the same work on top. Validation input is user-authored pipe graphs; with recursive design in flight, whether a cyclic controller graph can spin indefinitely in DRY mode needs confirmation.++**Direction:** periodic `activity.heartbeat()` pulses (between the sweep and the graph arm, and inside `BundleValidator.validate_pipes`' per-pipe loop) + a `heartbeat_timeout` on the `execute_activity` call. Alternatively wrap the body in `asyncio.wait_for` matched to `start_to_close`.++## 2. CPU-bound activity body on the worker event loop++The activity body is almost entirely CPU work (TOML interpretation, pydantic construction, polyfactory mock minting, DRY sweep + graph trace) with essentially no real awaits — `validate_bundle` yields exactly once (`await asyncio.sleep(0)`). On a worker that co-hosts inference activities, a large bundle blocks the event loop for long stretches: co-resident activities' heartbeat/start-to-close timers keep ticking, risking spurious timeouts that re-run real, billed LLM calls.++**Direction:** run the validation core off the loop (`asyncio.to_thread`, or a sync `@activity.defn` using the worker's activity executor), or at minimum add a per-pipe `await asyncio.sleep(0)` in the sweep loop. Interacts with item 1 (heartbeats need the loop free to fire).++## 3. Validation verdict depends on worker-side config (drift surface)++`BundleValidator._aggregate` reads `dry_run_config.allowed_to_fail_pipes` and `dry_run_pipe_in_process` reads `pipeline_execution_config` / `graph_config.data_inclusion` from the process-local config. Direct mode resolves these from `pipelex-api-deploy`'s `.pipelex` toml; Temporal mode from `pipelex-worker`'s separately-pinned one. Config drift between the two deploy repos silently changes which pipes are allowed to fail (same bundle passes on one backend, 422s on the other) and how much data the returned `GraphSpec` carries.++**Direction:** either carry the decision-affecting knobs in `DryValidateArg` so the submitter's config is authoritative, or declare worker config the single source of truth for `/validate` and add a deploy-time drift check between the two repos' configs.++## 4. The main pipe is dry-run twice per /validate++The sweep dry-runs every sweepable pipe (untraced, `generate_graph=False`), then `dry_run_pipe_in_process` re-runs the main pipe's whole subtree to capture the `GraphSpec`. Inherited from the direct-mode shape (sweep and graph were always two runs), but now that both arms share one process and one library, a single traced run of the main pipe could feed both its `DryRunOutput` classification and the graph.++**Direction:** fold with the existing D-plan §7 endpoint-unification follow-up; exclude the main pipe from the untraced sweep loop and derive its classification from the traced run.++## 5. `LibraryManager._pipe_source_map` is process-global, keyed by bare `pipe_ref` (pre-existing)++One dict for all libraries: load overwrites entries, teardown pops by `pipe_ref` regardless of owner. Two concurrent `act_dry_validate` invocations whose bundles share a `pipe_ref` overwrite each other's entries, and the first teardown deletes the survivor's. Consumers are currently low-stakes (`get_pipe_source` → `which` CLI), but this feature makes concurrent multi-library validation the norm on workers.++**Direction:** scope the map per-library (key by `(library_id, pipe_ref)` or move it onto `Library`).++## 6. No selected-pipe validation field on the wire++`validate_bundle` supports `dry_run_pipe_codes` (used by `validate_pipe_in_bundle` for builder flows that validate one implemented pipe while siblings may fail); `DryValidateArg` doesn't carry it, so the Temporal path can't preserve that behavior if the API ever exposes pipe selection. Not needed today — the API validates whole bundles — but it's the first field to add when the builder flows move server-side.++## 7. Per-queue dispatch tuning doesn't reach `act_dry_validate`++`WfDryValidate` hardcodes `start_to_close_timeout=5min` / `maximum_attempts=2` in workflow code (per D-C5, precedented by `WfPipeRun`'s own hardcoded utility-activity bounds), so the v2 `resolve_dispatch` overlays (`queue_options`, per-handle options) cannot tune the validation activity. Fine while one bound fits all deployments; revisit if a deployment needs a larger validation budget (big bundles WILL hit the 5-minute cap — see item 1 for what happens then).++## 8. Payload-converter exceptions in workflow tasks hang the submitter (retry-forever), and aware datetimes were the live trigger++Root-caused 2026-06-10 (CI py3.11–3.14 hang on the three GraphSpec-crossing tests): an exception raised by the data converter while decoding an activity result fails the *workflow task*, which Temporal retries indefinitely — the submitter's `execute_workflow` never returns. The trigger was environmental (`ZoneInfoNotFoundError`: no IANA tzdata on the runner for py3.11+), and kajson 0.7.0 structurally eliminated that trigger class: its self-sufficient timezone wire format carries the UTC offset, so decoding degrades gracefully to a fixed offset instead of raising, and kajson declares `tzdata` itself. But the failure *shape* is general: any deterministic decode bug (kajson registry miss, schema drift, future enum rename) will present as an infinite hang on every host, not a failed run. `dispatch_dry_validate`'s 12-minute `workflow_execution_timeout` bounds the API path; direct `execute_workflow` callers (tests, scripts) have no bound.++**Direction (design tradeoff, do not reflexively patch):** consider making deterministic conversion errors fail-fast — e.g. catch `KajsonDecoderError` in `BaseModelPayloadConverter.from_payload` and re-raise as a non-retryable shape, or document that every submitter must set `workflow_execution_timeout`. Temporal's retry-forever default exists so a code redeploy can heal a bad task; trading that away deserves a deliberate decision.diff --git a/wip/dry-run-refactor/followup-leaf-run-mode-mock.md b/wip/dry-run-refactor/followup-leaf-run-mode-mock.mdindex d00eb2988..dbb057f38 100644--- a/wip/dry-run-refactor/followup-leaf-run-mode-mock.md+++ b/wip/dry-run-refactor/followup-leaf-run-mode-mock.md@@ -1,74 +1,190 @@# Follow-up — Leaf-level run-mode mock (DRY honors the backend)-> **Status: deferred — own branch, not yet started.** Implements **D-plan Part B / D4 / req 1**. Split out of [`consolidation-as-built.md`](./consolidation-as-built.md) on 2026-06-01 (eng-review D1) so the in-process consolidation ships alone.+> **⚠ Decision D8 superseded (2026-06-10, post-completion):** `is_mock_inference` will NOT survive as a thin reportable-mock mode — it becomes an internal `is_mock_usage` sub-flag of DRY. See [`design-unified-dry-run.md`](./design-unified-dry-run.md). Everything else in this file stands as the as-built record.+>+> **Status: COMPLETE (2026-06-10) — built on `feature/Mock-activities`, all phases done, Checkpoint E reached.** Implements **D-plan Part B / D4 / req 1**. Split out of [`consolidation-as-built.md`](./consolidation-as-built.md) on 2026-06-01 (eng-review D1) so the in-process consolidation ships alone.>> **Design rationale:** [`D-plan.md`](./D-plan.md) §3.5 (run mode ⟂ backend) and §4.8 (leaf-level mock). **Risks:** D-plan §8 (object-mock fidelity, req-1 fidelity regressions).>-> **Depends on:** the consolidation ([`consolidation-as-built.md`](./consolidation-as-built.md)) is *not* a hard prerequisite (this is a separable cogt/operator refactor), but the Temporal-validation follow-up builds on **both**, so sequence this after the consolidation lands. **Branch off the same D-plan.**+> **Depends on:** the consolidation ([`consolidation-as-built.md`](./consolidation-as-built.md)) is *not* a hard prerequisite (this is a separable cogt/operator refactor). **Branch off the same D-plan.**+>+> **Relationship to the in-memory activity follow-up (corrected 2026-06-09):** this follow-up (req 1) is **orthogonal** to [`followup-temporal-validation-activity.md`](./followup-temporal-validation-activity.md) (req 2), **not** a prerequisite for it — see "Still relevant — a distinct dry-run mode" below.>> **⚠️ Coordinate with the registry branch's Phase 5 (`fix/For-API-update`, decided 2026-06-06).** That branch ships `--mock-inference` / `is_mock_inference` as an *interim* trigger, deliberately built **leaf-first so it lands B1's core ahead of time**: a shared `cogt/content_generation/dry_mock.py` + a per-leaf dry branch keyed on a per-run flag carried on `JobMetadata`. So when this follow-up runs, **B1 collapses to "re-key that helper from `is_mock_inference` → `run_mode`"** (carrier + helper already exist — verify before building from scratch), and **B2 settles the fate of `is_mock_inference`** (retire vs keep a thin reportable-mock — see B2). One distinction to preserve when re-keying: `--mock-inference` emits *non-zero* synthetic usage so a cost report renders; `--dry-run` stays zero-token and its report is *suppressed*. See the registry feature's [`../registry/deferred-followups.md`](../registry/deferred-followups.md) (`--mock-inference` coverage + the `is_mock_inference` fate decision).-Goal: move the LIVE/DRY decision **down to the cogt leaf** so DRY honors the configured backend — DRY-on-Temporal dispatches `act_llm_gen_*` and mocks **inside** the activity, retiring the "DRY → local in-process" shortcut. Separable cogt/operator refactor (§4.8). **Resolve the Pre-flight items before starting.**+Goal: move the LIVE/DRY decision **down to the cogt leaf** so DRY honors the configured backend — DRY-on-Temporal dispatches `act_llm_gen_*` and mocks **inside** the activity, retiring the "DRY → local in-process" shortcut. Separable cogt/operator refactor (§4.8). Pre-flight items are decided; eng-review decisions (2026-06-10, D2–D11) are folded into the phase checklists below.++The threading route after Part B:++```+PipeRunParams.cogt_run_params (run_mode lives ONLY here; .run_mode = delegating property)+ └─ built once at PipeRunParams construction (prepare_pipe_job resolves boot force-DRY)+ └─ operator (one path, no Dry-swap) passes cogt_run_params into generator methods+ └─ hub ContentGenerator | ContentGeneratorInWorkflow ← backend choice ONLY+ └─ assignment.cogt_run_params ← serializes across the wire+ └─ [direct: inline] | [temporal: act_*_gen_* on worker]+ └─ leaf: run_mode==DRY → dry_mock (no provider, no storage)+ LIVE → real call (+ store for img/extract)+```++## Still relevant — a distinct dry-run mode (do NOT drop this file)++This describes a **different** dry-run mode from the in-memory in-process activity ([`followup-temporal-validation-activity.md`](./followup-temporal-validation-activity.md)). Both are wanted; they're orthogonal `run_mode × backend` combinations of one foundation (mock at the leaf; the backend decides where the leaf runs — D-plan §3.5):++- **This file (req 1) — full distribution, leaf-only mocks.** A DRY run goes through the **real** Temporal path (`WfPipeRouter` → child workflows → `act_llm_gen_*` activities) and the **leaf inside each activity** mocks instead of calling the model. Purpose: **test the distribution machinery** (dispatch, scheduling, serialization, routing, cross-worker propagation) without AI cost or latency.+- **The other file (req 2) — one in-process activity, in-memory tracing.** The whole dry-run + validation runs **in-process inside a single activity** (nothing nested is dispatched), tracing the graph in memory. Purpose: offload validate+graph to a worker cheaply.++**Already partly shipped:** `is_mock_inference` (the registry branch's interim trigger) **is** the LLM slice of this mode — `run_mode` stays LIVE so operators dispatch `act_llm_gen_*` normally, but the leaf fakes the call (`JobMetadata.is_mock_inference` → the `llm_generate.py` leaf branch). So req-1 behavior already coexists, today, alongside the pipe-level DRY path — concrete proof the two modes don't conflict. B1/B2 generalize it from LLM-only + `is_mock_inference` to all leaves + `run_mode=DRY`.++**Shared seam with the in-memory activity — `scoped_content_generator` (RESOLVED: it exists, consume it).** B2 routes the DRY mock through `get_content_generator()` at the leaf. Under a Temporal-enabled hub that returns `ContentGeneratorInWorkflow` **globally** (boot-time, `pipelex.py:370-385`). The in-process activity (req 2) landed first and **built the inline-content-generator scope** — `scoped_content_generator` is live (`bundle_validator.py:260` uses it, see `TODOS.md` Phase 2). This follow-up consumes it: B2's Mode-1 scopes (`validate_pipes`, `dry_run_pipe_in_process`) swap `ContentGeneratorDry()` for an inline `ContentGenerator` inside the existing scope.## Status at a glance| Phase | Title | Status | Commit ||---|---|---|---|-| B1 | Thread `run_mode` to cogt leaf + leaf DRY branch | ☐ not started | |-| B2 | Collapse operator dry path | ☐ | |-| | **⛔ CHECKPOINT D** | | |-| B3 | Verify Temporal + DRY e2e (**req-1 gate**) | ☐ | |-| | **⛔ CHECKPOINT E — follow-up complete** | | |+| B1 | Thread `run_mode` to cogt leaf + leaf DRY branch | ☑ done | `feat(dry-run): thread run_mode to the cogt leaf via CogtRunParams` (2026-06-10, review-fixed) |+| B2 | Collapse operator dry path | ☑ done | `feat(dry-run): collapse the operator dry path onto the leaf mock` (2026-06-10) |+| | **⛔ CHECKPOINT D — cleared 2026-06-10** | | |+| B3 | Verify Temporal + DRY e2e (**req-1 gate**) | ☑ done | `test(temporal): Tier 17 — DRY honors the Temporal backend` (2026-06-10) |+| | **⛔ CHECKPOINT E — follow-up complete — REACHED 2026-06-10** | | |Status legend: ☐ not started · ◐ in progress · ☑ done.-## Pre-flight — needs human input--Resolve **before** the phase that depends on it. Record the answer in the relevant checkpoint Handoff.+## Pre-flight — ALL DECIDED (2026-06-10, with the user)-- [ ] **(before B1) `run_mode` carrier.** Confirm: carry `run_mode` on `JobMetadata` (recommended, lowest-churn — §4.8) vs an explicit field on each cogt assignment. Default if unanswered: `JobMetadata`.-- [ ] **(before B1) object-mock-from-schema fidelity.** Single schema-based mock site (default) vs two-site (class-based direct + schema-based activity). §4.8 / §8. Default if unanswered: single site + fidelity test.-- [ ] **(before B1/B2) synthetic dry LLM report.** Keep `_report_dry_llm_job` firing during the validation sweep, or gate it off so validation is report-silent (runner-emission e2e keeps it on)? §8. Default if unanswered: open one throwaway per-sweep registry, keep the report.-- [ ] **(before B2) `ContentGeneratorDry` disposition.** Delete outright, or keep as a thin force-`run_mode=DRY` facade for the boot `not needs_inference` fallback? §4.8.+- [x] **(DECIDED, refined by eng review D2 2026-06-10) `run_mode` carrier = a new `CogtRunParams` class, nested INSIDE `PipeRunParams`, holding the only copy of `run_mode`.** The cogt-tier params class `CogtRunParams` (`pipelex/cogt/content_generation/cogt_run_params.py`, a `BaseModel` — it crosses the Temporal wire) becomes a field `PipeRunParams.cogt_run_params`. **`run_mode` lives ONLY in `CogtRunParams` — zero duplication**: `PipeRunParams.run_mode` becomes a delegating property (`return self.cogt_run_params.run_mode`) so existing pipe-tier readers (`_run_pipe_traced` match, factories) keep working unchanged. Single writer: wherever `PipeRunParams` is constructed (`make_run_params` factory / `prepare_pipe_job`, which also resolves the boot force-DRY flag into it). Route to the leaf: operators slice `pipe_run_params.cogt_run_params` off and pass it into the content-generator protocol methods (**12-method signature change** across `ContentGeneratorProtocol` + every implementation — `ContentGenerator`, `ContentGeneratorInWorkflow`, `ContentGeneratorDry` until B2 deletes it, the `temporal/test_extras` workflow callers, and test doubles); generators stamp it as a field on each cogt assignment (`LLMAssignment`, `ObjectAssignment`, img-gen/extract/templating/search/render); the leaf keys on `cogt_run_params.run_mode == DRY`. Temporal converter work is ~zero (activities take the assignment models whole; a new pydantic field rides free). Rationale: type-explicit on the execution contract (can't be missed when building an assignment), keeps `JobMetadata` pure tracing/reporting identity, and gives `is_mock_inference` a natural home to migrate into (one flag-transport style, not two) — the class starts with one attribute by design; it is the cogt slice of the run params and will grow. **Rejected:** the `JobMetadata` mirror (the doc's old default — drifts metadata toward a behavior-flag grab-bag), threading `PipeRunParams` into cogt (layer violation — drags pipe-tier concepts like multiplicity/batch params into a deliberately pipe-agnostic layer), and a stored duplicate `run_mode` on both `PipeRunParams` and `CogtRunParams` (copy-drift risk). Consequence for B1: not a pure re-key — B1 introduces `CogtRunParams`, the `PipeRunParams` nesting + property, the protocol/call-site threading, and the assignment field, then re-keys the leaf branch; `is_mock_inference` migrates into `CogtRunParams` in B2 (fate decided: see B2).+- [x] **(DECIDED) Object mock = single schema-based site + fidelity test.** Both direct and dispatched leaves build via `SchemaToModelFactory` from the JSON schema — one code path, identical mock on both backends (the backend-parity this follow-up exists to deliver); fidelity bugs surface in cheap local unit tests instead of only on a worker. Accepted cost: direct mode gives up the in-hand original class — exotic format constraints (`json_schema_extra` hints dropped on round-trip) must declare `examples`/`mock_format`; the mandated fidelity test on a representative `StructuredContent` pins this.+- [x] **(DECIDED) Synthetic dry LLM report = keep, into a throwaway per-sweep registry.** The leaf stays unconditional (no "am I validating?" flag threading into cogt); the validation sweep opens one throwaway `UsageRegistry` so events have a home and die with it. Zero-token suppression already guarantees no rendered report; the runner-emission e2e keeps its cheap observable signal.+- [x] **(DECIDED) `ContentGeneratorDry` = delete outright in B2.** One mock mechanism remains: `run_mode=DRY` at the leaf. The boot `not needs_inference` fallback (`pipelex.py:371`) is re-expressed as a forced-DRY flag consumed at `prepare_pipe_job` (B2 mandated this anyway); Mode-1's scopes (`validate_pipes`, `dry_run_pipe_in_process`) pass the inline `ContentGenerator` instead (their callers already run DRY); tests using it as a double switch to the run_mode-threaded equivalent.## Phase B1 — Thread `run_mode` to the leaf + add the leaf DRY branch (§4.8)> **B1's core LANDED on the registry branch's Phase 5 (`fix/For-API-update`).** Already built: `JobMetadata.is_mock_inference`, the shared `cogt/content_generation/dry_mock.py` (synthetic-job reporting parameterized zero-vs-non-zero, plus the leaf mocks + `build_mock_object`), and the per-leaf branch in `llm_generate.py` keyed on `is_mock_inference`. So B1 reduces to: (1) carry `run_mode` on `JobMetadata` alongside `is_mock_inference` (same single-writer, `prepare_pipe_job`), and (2) re-key the leaf branch from `is_mock_inference` → `run_mode==DRY`. **Don't rebuild the helper.** Two caveats to finish what Phase 5 deliberately scoped out: it covers the **LLM leaf only** — `img_gen_*` / `extract_gen_pages` / `templating_gen_text` leaf branches are still TODO here (Phase 5 deferred them because their output is stored above the leaf; see that branch's §7); and `ContentGeneratorDry` still owns its method bodies (only its *reporting* delegates to `dry_mock`), so B2's collapse is still real work.-> **Interim hard guard already landed (F1, registry branch).** `img_gen_single_image` / `img_gen_image_list` / `extract_gen_pages`, plus `PipeSearch._live_run_operator_pipe` (web search has no `content_generation` leaf — it spends in the operator's live path), now **raise `MockInferenceUnsupportedError`** under `is_mock_inference` instead of silently calling the real provider. So when B1 adds the real synthetic-output leaf branch for img-gen/extract here, it is **replacing a fail-loud guard, not patching a silent-spend path** — delete the guard at that leaf as you add the DRY branch. Search has no leaf to thread `run_mode` into; decide whether it gains a real mock or keeps the guard when B1/B2 settle the operator dry paths. See `wip/registry/deferred-followups.md` (`--mock-inference` coverage).+> **Interim hard guard already landed (F1, registry branch).** `img_gen_single_image` / `img_gen_image_list` / `extract_gen_pages`, plus the search leaf, now **raise `MockInferenceUnsupportedError`** under `is_mock_inference` instead of silently calling the real provider. So when B1 adds the real synthetic-output leaf branch for img-gen/extract/search here, it is **replacing a fail-loud guard, not patching a silent-spend path** — delete the guard at that leaf as you add the DRY branch (under decision 8B the guard's `is_mock_inference` arm stays wherever no reportable-mock is built). See `wip/registry/deferred-followups.md` (`--mock-inference` coverage).+>+> **Stale-premise correction (eng review D5, 2026-06-10): search HAS a cogt leaf now.** `search_generate.py` landed as a framework-agnostic leaf (sibling of `llm_generate`, with `act_search_gen_sourced_answer` / `act_search_gen_structured` activities), so the old "web search has no `content_generation` leaf" claim is outdated. **Decision: full uniform treatment** — the search leaf and the `render_page_views` leaf both get `run_mode` DRY branches in B1 (lift `make_search_sourced_answer` / `make_search_structured` / `make_render_page_views` mock bodies from `ContentGeneratorDry:299-319,222-244` into `dry_mock.py`), `PipeSearch` joins B2's swap-removal list (`pipe_search.py:177`), and Tier 17 extends to the search activities.+>+> **Mock typing is leaf-shaped, not generator-shaped (outside-voice fold, 2026-06-10).** The img/extract leaves return `GeneratedImageRawDetails` / `ExtractOutput`, not `ImageContent` / `PageContent` — "lift bodies from `ContentGeneratorDry`" means lift the *semantics* and build mocks at each leaf's own return type; the generator-level conversion (`make_image_content` / `make_page_contents`) stays where it is.+>+> **DRY short-circuits storage (eng review D10, 2026-06-10).** The img/extract Temporal activities store outputs via `GeneratedContentFactory` + `get_storage_provider()` (`act_img_gen_generate.py`, `*_and_store` functions). The DRY branch sits **above the store step**: mock-and-return URL-only `ImageContent`/`PageContent` (what `ContentGeneratorDry`'s fakes already are) without touching the storage provider — dry means dry: no storage IO, no storage credentials, no mock artifacts in real buckets. Name this asymmetry explicitly (the DRY branch for img/extract lives at the `*_and_store` layer, one step above the raw provider leaf) — do not "unify" it downward. Tier 17 asserts no storage writes.++- [x] Confirm the funnel by grep: `ContentGenerator.make_llm_text` and `ContentGeneratorInWorkflow.make_llm_text` both build `LLMAssignment` and converge on `llm_gen_text(assignment)` (inline vs via `act_llm_gen_text`). ✓ confirmed.+- [x] **Introduce `CogtRunParams`** (`pipelex/cogt/content_generation/cogt_run_params.py`, `BaseModel`) nested as `PipeRunParams.cogt_run_params`; `run_mode` lives ONLY there; `PipeRunParams.run_mode` becomes a delegating property so existing readers are untouched. Single writer at `PipeRunParams` construction (`make_run_params` / `prepare_pipe_job`). *As built:* `PipeRunParams` got `extra="forbid"` so a stale `run_mode=` kwarg fails loudly; `PipeRunMode` importers were repointed from the `pipe_run_params` re-export to `pipe_run_mode` directly; the redundant `branch_pipe_run_params.run_mode = …` writer in `pipe_batch.py` (a no-op after `model_copy(deep=True)`) was deleted.+- [x] **Thread it through the protocol**: leaf-dispatching `ContentGeneratorProtocol` methods gain a `cogt_run_params` parameter; all implementations + operator call sites updated; generators stamp it as a field on every assignment model. *As built (deviation from the "12-method" framing):* 8 raw-param methods gained the parameter; the two search methods take a pre-built assignment, so `PipeSearch` stamps `cogt_run_params` at `SearchAssignment` construction (no redundant param); `make_image_content` / `make_page_contents` are post-leaf conversion steps with no leaf dispatch and were deliberately left un-threaded. `ObjectAssignment` / `SearchObjectAssignment` expose `cogt_run_params` as delegating properties over their nested assignment (zero duplication).+- [x] **Carry `nb_items: int | None` on `ObjectAssignment`** (eng review D11): both generators pass it through; the dry AND mock leaf list mocks honor `nb_items or config.nb_list_items`.+- [x] *Tests:* per-leaf DRY-branch unit tests (`test_*_dry_branch.py` per leaf module); object-mock-from-schema fidelity pin on a representative `StructuredContent` + generator-level D6 DRY-arm tests (`test_leaf_dry_object_mocks.py`); fixed `nb_items` regression (leaf + through `ContentGenerator`); templating DRY still raises on a bad jinja2 template; `LLMAssignment` + `PipeRunParams` wire round-trip with `cogt_run_params` through the Temporal payload converter. *Deferred to B2:* the builder-bundle dry-validation `mock_main` pin (it lands with B2's `stamp_mock_main_coordination()` helper).+- [x] Add the DRY branch to each leaf, keyed on `cogt_run_params.run_mode.is_dry` — `llm_gen_text` / `llm_gen_object` / `llm_gen_object_list` (zero-token `report_dry_llm_job` + schema-built mocks), `img_gen_*_and_store` / `extract_gen_pages_and_store` (**above the store step** — D10), `render_page_views_and_store` (new shared leaf `render_generate.py`, now also used by `act_render_page_views` AND direct `ContentGenerator` — direct img/extract/render converge on the same `*_and_store` leaves as the activities), `templating_gen_text` (preserves `check_jinja2_parsing`), `search_gen_*` (lifted mocks — D5). Semantics lifted from `ContentGeneratorDry` into `dry_mock.py` `dry_*` helpers. *Note:* the `MockInferenceUnsupportedError` guards stay as-is — they are keyed on `is_mock_inference` only (which still has no non-LLM reportable mock per D8), so there was nothing to delete at the DRY arm.+- [x] **Typed terminal mock-build errors** (eng review D7): `build_mock_object` wraps polyfactory/validation failures into `DryRunMockBuildError` naming the class and the remedy; unit test pins the type (`test_dry_run_mock_build_error.py`). Error pages regenerated.+- [x] **Re-key the object-fidelity wrapper** (eng review D6): both `_revalidate_against_object_class` copies (direct + in-workflow) fire on `is_mock_built = run_mode.is_dry or is_mock_inference`; `MockInferenceObjectFidelityError` docs/message generalized (remedy = `examples`/`mock_format`, no longer "use --dry-run"); DRY-arm unit + integration tests added.+- [x] **Comment/doc maintenance**: `dry_mock.py` module docstring rewritten for the post-B1 world; `JobMetadata.is_mock_inference` comment updated (D8 migration noted); stale "search has no leaf mock" comment fixed.+- [x] Operators still call `ContentGeneratorDry` for now — **no behavior change yet**. `make agent-check` + `make agent-test` green (2026-06-10).++## Deferred items from the B1 code review (2026-06-10)++Findings from the post-B1 review round that were verified real but deliberately NOT fixed in B1 — each is a design tradeoff or a later-phase concern, not a silent bug in shipped behavior:++- **Storing activities resolve `get_storage_provider()` before the leaf's DRY branch** (`act_img_gen_generate.py`, `act_extract_generate.py`, `act_render_page_views.py`): D10's "no storage credentials under DRY" holds only because boot always constructs a provider from config (provider construction needs no credentials; only real IO does). If B3's Tier 17 no-keys arm ever boots a worker without a storage provider, move acquisition inside the leaf's LIVE arm. Verify in B3.+- **Sequential page-view storage uploads** in `render_page_views_and_store` (pre-existing behavior, relocated): each rendered page is stored one `await` at a time; a large PDF pays serialized upload latency inside the activity. Candidate `asyncio.gather` with a bound — perf-only, touch when a real workload hurts.+- **Ambient `dry_run_config.nb_extract_pages` read at two leaves** (`dry_extract_page_contents`, `dry_render_page_views`): under Temporal these run in different activities, so heterogeneous worker configs could mismatch page counts and trip the generator's length check on a dry run. D11-style fix = carry the resolved count on the assignments. Revisit in B3 (Tier 17 would surface it).+- **No cross-call memoization of `DryRunFactory.make_dry_run_factory`**: list mocks now build the factory once per call (`build_mock_objects`), but repeated leaf calls for the same class still rebuild it. An lru-cache keyed by class adds lifetime/invalidation surface for dynamically-created classes — skipped as over-engineering until dry-run profiling says otherwise.+- **Schema-reconstruction failures stay untyped** (`_reconstruct_object_class` → `SchemaToModelFactory`): D7 wrapped the polyfactory build; codegen failures (ValueError / datamodel-code-generator errors) share their surface with the LIVE leaf path (same function, same schema), so typing them only on the mock path would add asymmetry. Belongs to the broader "retry-forever on unconverted exceptions" hardening family already tracked.++Additional deferred items from the B2 code review (2026-06-10):++- **`PIPELEX_FORCE_DRY_RUN_MODE` env var vs the hub forced-DRY flag**: two run-mode-forcing mechanisms now coexist — the env var (resolved in `pipeline_run_setup`, only a *default* despite its name: an explicit run mode wins, per its own `# TODO: rethink this, it's not forcing`) and the hub flag (genuinely forcing, resolved at `make_run_params`). Nothing in the repo sets the env var; folding it into the hub flag (or deleting it) is a user-facing-surface decision — defer.+- **Mock-input list factory not hoisted** (`working_memory_factory.make_mock_inputs` builds a `DryRunFactory` per item): NOT a pure hoist — `_get_mockable_class` picks a *random concrete subclass per item*, so hoisting would change mock diversity. Left as is.+- **`ContentGeneratorInWorkflowFactory` is now a param-less wrapper**: kept as the deliberate lazy-import construction seam (boot imports it deferred); revisit if it never grows params.+- **Mock-inference guard not consolidated**: the `MockInferenceUnsupportedError` guards at the img/extract/search leaves stay inline (3 small sites with distinct operation labels); consolidation would save little.++Deferred items from the PR #983 review-bot round (2026-06-10):-- [ ] Confirm the funnel by grep: `ContentGenerator.make_llm_text` and `ContentGeneratorInWorkflow.make_llm_text` both build `LLMAssignment` and converge on `llm_gen_text(assignment)` (inline vs via `act_llm_gen_text`).-- [ ] Carry `run_mode` on `JobMetadata` (or the confirmed carrier); single-writer from `prepare_pipe_job` / `PipeRunParams`.-- [ ] *Tests first:* per-leaf DRY-branch unit tests; the object-mock-from-schema fidelity test on a representative `StructuredContent`.-- [ ] Add the DRY branch to each leaf — `llm_gen_text` (the `"DRY RUN: …"` string + `_report_dry_llm_job`), `llm_gen_object` / `llm_gen_object_list` (`DryRunFactory` build from `ObjectAssignment.object_class_schema` + report), `img_gen_*` (fake `ImageContent`), `extract_gen_pages` (mock pages), `templating_gen_text` (**preserve `check_jinja2_parsing`** then mock). Lift bodies from `ContentGeneratorDry` (consider a shared `cogt/content_generation/dry_mock.py`).-- [ ] Operators still call `ContentGeneratorDry` for now — **no behavior change yet**. `make agent-check` + `make agent-test` green.+- **Direct structured-search live validation stays a raw `ValidationError`** (greptile P1 / cubic P2): the Temporal arm wraps it in `ContentGenerationError` because an unconverted exception in workflow code retries forever and hangs the submitter — a boundary-specific failure mode the direct path doesn't have (the raw error propagates and is reported normally). Unifying would require moving/duplicating a temporal-tier error class into cogt; not worth the layering churn for a cosmetic consistency gain. Mock-built failures ARE unified (typed `MockInferenceObjectFidelityError` on both arms, D6).+- **Raw img/extract leaves carry no DRY fail-loud guard** (flagged independently by two adversarial passes): only the `*_and_store` wrappers branch on DRY; a future direct caller of `img_gen_single_image`/`img_gen_image_list`/`extract_gen_pages` with a DRY assignment would hit the real provider. Today no such caller exists (grep-verified) and the D10 docstrings forbid unifying downward — kept documentary rather than adding guards for an unreachable path; revisit if a new raw-leaf caller appears.+- **Per-call schema hashing + global codegen cache lock** (perf, /review final gate): every dry object mock pays `json.dumps`+`sha256` of the full schema for the cache lookup, and `SchemaToModelFactory.make_from_json_schema` holds one process-global lock even on cache hits (documented tradeoff in that module) — now on the DRY path too. Candidates: precomputed schema hash on `ObjectAssignment`, double-checked locking. Touch when dry-sweep profiling says so.+- **Assignment models are extra-ignore while the nested carrier is extra-forbid**: stale keys on `LLMAssignment` etc. are silently dropped. Broader wire-hygiene sweep deferred (belongs with the decode-error hardening family).+- **`Pipelex.make()` failure poisons the test worker** (observed in CI): a boot that raises mid-`setup()` leaves the singleton registered with `LogConfig` set, and `teardown()` of the half-initialized instance raises before `reset_config()` — every later module-scoped boot on that xdist worker then fails with "LogConfig is already set". Pre-existing test-infra hardening item (make `teardown()` tolerate a half-setup instance, or have `make()` clean up on failure); surfaced by the keyless-boot test's control arm booting PYTEST mode in CI (fixed by mirroring the conftest's CI-mode selection).## Phase B2 — Collapse the operator dry path (§4.8)-- [ ] Remove the pipe-level `ContentGeneratorDry()` swap from each operator's `_dry_run_operator_pipe` (`PipeLLM`, `PipeCompose`, `PipeImgGen`, `PipeExtract`, `PipeStructure` — grep for `ContentGeneratorDry()`). Route DRY through the hub content generator with `run_mode` threaded.-- [ ] Re-express the boot `not needs_inference` fallback (`pipelex.py:368`) as force-`run_mode=DRY`; apply the confirmed `ContentGeneratorDry` disposition.-- [ ] **Settle `is_mock_inference` / `--mock-inference`** (the registry branch's interim trigger). Now that `run_mode=DRY` honors the backend, **either** (a) *fully retire* the flag, its `JobMetadata` field, its CLI option, and the leaf branch's `is_mock_inference` arm — `run_mode=DRY` becomes the sole trigger — and re-key the registry branch's `temporal-e2e-validate` Tier 8b to assert on assembled `tokens_usages` (plain DRY emits zero-token usage, which Phase 3 *suppresses*, so the rendered table can't be asserted under DRY); **or** (b) *keep a thin reportable-mock* (the non-zero-synthetic-usage arm) precisely so Tier 8b can still validate the rendered cost report cheaply. Pick based on whether cheap rendered-report validation is worth one surviving flag. (Cross-ref: registry feature's [`../registry/deferred-followups.md`](../registry/deferred-followups.md).)-- [ ] `PipeSignature._dry_run_pipe` and controller dry behavior unchanged (no leaf op).-- [ ] Verify **direct + DRY** outcomes unchanged: full `make agent-test` green + spot-check a dry `validate --all` and a dry single-pipe run.+- [x] Remove the pipe-level `ContentGeneratorDry()` swap from each operator's `_dry_run_operator_pipe` (`PipeLLM`, `PipeCompose`, `PipeImgGen`, `PipeExtract`, `PipeStructure`, `PipeSearch` — D5). DRY now routes through the hub content generator; `cogt_run_params` (carrying DRY) rides into every assignment and the leaf mocks. *As built (post-review):* the delegation is the base-class default (`PipeOperator._dry_run_operator_pipe`); only `PipeFunc` and `PipeCompose` override; the dead `content_generator` parameter was removed from the operators' `_live_run_operator_pipe` and from `StructuredContentComposer` (the scoped contextvar is the only generator-override seam); the in-process scopes use `ContentGenerator.make_inline()`; `MOCK_MAIN_PIPE_CODE` is a shared constant imported by `bundle_header_spec`. Also: `PipeCompose`'s dry path kept its construct-mode `PipeComposeError` fallback but dropped the generator selection entirely (the composer renders templates directly and never calls the generator), which made `dry_run_config.apply_to_jinja2_rendering` fully dead — the config key was removed (configs.py + pipelex.toml).+- [x] **Consolidate the `mock_main` coordination into one shared helper** (eng review D3): `stamp_mock_main_coordination()` lives in `dry_mock.py` with the one WHY docstring (coordination with `bundle_header_spec.py` `examples=["mock_main"]`); called by `working_memory_factory`, `pipe_batch`, and the dry object-list leaf (`dry_llm_gen_object_list`) — `--mock-inference` keeps not stamping. Regression tests pin both arms (`test_leaf_dry_object_mocks.py`).+- [x] Re-express the boot `not needs_inference` fallback as force-`run_mode=DRY`: `PipelexHub.set_dry_run_forced(...)` set at boot, consumed at `PipeRunParamsFactory.make_run_params` — the single writer of `run_mode` — so EVERY entry point that builds run params is covered (pipeline API, runtime bridge, `PipeJobFactory` defaults); a warning logs when forced-DRY swallows an explicit `--mock-inference`. **Generator selection backend-keyed unconditionally** (D4). Regression tests in `test_keyless_boot_forced_dry.py`: keyless+temporal ⇒ in-workflow generator + forced DRY; keyless direct ⇒ inline + forced DRY; control arm; LIVE-requested job under keyless boot prepared as DRY. `ContentGeneratorDry` **deleted**; `bundle_validator` + `dry_run_pipeline` scopes pass an inline `ContentGenerator`; `temporal/test_extras` workflows always use the in-workflow generator (the dry arm now dispatches and mocks inside activities — leaf parity in action); its remaining test doubles repointed (`test_scoped_content_generator`, `test_dry_report_tz` (renamed), `test_dry_run_graph_in_process` simplified to real post-Part-B behavior); redundant `test_content_generator_dry_reporting.py` deleted (covered by the leaf dry-branch tests).+- [x] **`is_mock_inference` migrated from `JobMetadata` into `CogtRunParams`** (D8): single writer `PipeRunParamsFactory.make_run_params` (fed by `prepare_pipe_job` off the CLI flag); all leaf branches/guards and both fidelity wrappers re-keyed; `JobMetadata` field deleted; tests migrated (the temporal mock-inference test now injects the flag via `pipe_run_params.cogt_run_params`, pinning wire survival on the new carrier). The CLI surface (`--mock-inference`, runner/pipeline params) is unchanged, so Tier 8b's CLI-level arms keep working as written; its prose gets the Tier-8-note update in B3.+- [x] **Usage isolation** (pre-flight decision 3, resolved as-built): there is no ambient `UsageRegistry` anymore — usage events are dropped unless a trace context with `emit_usage_events=True` is registered, and validation sweeps never register one. Pinned by the new isolation assertion in `test_dry_run_graph_in_process` (dry run emits graph events but zero `UsageReportEvent`s through the scoped in-memory transport).+- [x] `PipeSignature._dry_run_pipe` and controller dry behavior unchanged (no leaf op) — untouched.+- [x] Verify **direct + DRY** outcomes: full `make agent-test` green (2026-06-10) + spot-checked `pipelex validate --all` and a dry `run bundle native_text_sequence --dry-run --mock-inputs` (output carries the leaf marker `DRY RUN: llm_gen_text`, proving the leaf mock served the run); fixed-`nb_items` pinned by `test_generator_dry_object_list_keeps_fixed_length`; builder-bundle dry-validation covered by the suite's builder tests + the `mock_main` leaf pins.-> ### ⛔ CHECKPOINT D — after Phase B2 — **MANDATORY STOP**+> ### ⛔ CHECKPOINT D — after Phase B2 — **REACHED 2026-06-10**>> Direct DRY re-expressed through the leaf with identical outcomes; Temporal cell not yet verified.>-> **Verify:** `make agent-check` clean · `make agent-test` green · commit.+> **Verified:** `make agent-check` clean · full `make agent-test` green · dry `validate --all` + dry bundle run spot-checked (leaf marker confirmed in output) · committed.>-> **Handoff (fill in):** record final assignment/`run_mode` carrier shape, `ContentGeneratorDry` disposition, object-mock decision. **Next entry point: Phase B3 — Temporal+DRY e2e** (needs a Temporal server).+> **Handoff:**+>+> - **Carrier shape (final):** `CogtRunParams { run_mode: PipeRunMode = LIVE, is_mock_inference: bool = False }` in `pipelex/cogt/content_generation/cogt_run_params.py`. Nested as `PipeRunParams.cogt_run_params` (only copy; `PipeRunParams.run_mode` is a read-only delegating property; `extra="forbid"`). Single writer: `PipeRunParamsFactory.make_run_params(pipe_run_mode=…, is_mock_inference=…)` fed by `prepare_pipe_job`; the factory itself consumes the hub's forced-DRY flag (`PipelexHub.set_dry_run_forced` set at boot when `needs_inference=False`), covering every entry point incl. the runtime bridge. `CogtRunParams.is_mock_built` is the derived fidelity-guard predicate (DRY wins over `is_mock_inference` at every leaf). Every cogt assignment carries `cogt_run_params` (required field; `ObjectAssignment`/`SearchObjectAssignment` delegate via property to their nested assignment).+> - **`ContentGeneratorDry`: deleted.** One mock mechanism remains: the leaf branches keyed on `cogt_run_params` (`run_mode.is_dry` for DRY; `is_mock_inference` for the thin reportable LLM mock). The in-process scopes (`bundle_validator`, `dry_run_pipeline`) pass an inline `ContentGenerator`; under DRY its leaves mock without storage IO.+> - **Object-mock decision (held):** single schema-based site — both backends build via `SchemaToModelFactory` + polyfactory (`_leaf_gen_object(_list)` in `dry_mock.py`); fidelity failures surface as `MockInferenceObjectFidelityError` on both the mock and DRY arms; deterministic build failures as `DryRunMockBuildError` (non-retryable on Temporal via `non_retryable_error_types`).+> - **Next entry point: Phase B3 — Temporal+DRY e2e (Tier 17)** — needs a real Temporal server (3-process Mode 2); use the `temporal-e2e-validate` skill. Also update the skill's Tier 8 note + Tier 8b prose there.## Phase B3 — Verify Temporal + DRY end-to-end (**req-1 acceptance gate**)-- [ ] With a Temporal server (`temporal-e2e-validate` topology), run a pipeline `run_mode=DRY` and assert: `act_llm_gen_*` + extract/img-gen activities **are dispatched** and **mock inside the activity**; LibraryCrate propagation, cross-process serialization, graph tracing behave as LIVE; **no real LLM/IO** occurs.-- [ ] Add a DRY arm to the Temporal e2e suite.-- [ ] `make agent-check` + `make agent-test` green; Temporal e2e green.+The acceptance gate is a **specific distributed scenario** in the repo's `temporal-e2e-validate` skill — **Tier 17** — built to the Tier 2c/2d precedent (a 3-process scenario in `references/mode-2-tiers.md`, a Mode-1 pytest, a Step-7 master-table row). Full spec below.++- [x] **Add `temporal-e2e-validate` Tier 17 — "DRY honors the Temporal backend (leaf mock inside the activity)".** Added as Step 5g in `mode-2-tiers.md` (GREEN LLM + extract/img-gen/search arms + no-keys arm + RED mutate-the-leaf procedure) + Step-7 master-table row + SKILL.md routing entry; the Tier 8 / Tier 8b / routing-battery stale notes were already corrected in B2's review round. Mode-1 pytest: `tests/integration/pipelex/temporal/tracing/test_dry_run_dispatches_and_mocks.py` (under `tracing/` so the directory conftest applies the CI-xdist quarantine and the tmp-dir trace redirect) — asserts dispatch via workflow history across the parent + child workflow tree (the LLM activity is scheduled by the controller's child workflows, not the root), the `DRY RUN:` leaf output, and zero-token suppressed usage under the `dry_run` sentinel.+- [x] `make agent-check` + `make agent-test` green; **Temporal e2e RUN AND GREEN (2026-06-10, real `temporal server start-dev` + split router/runner workers)**: GREEN LLM arm (exit 0, `act_llm_gen_text` ×2 executed on the runner, zero provider/storage lines, `DRY RUN: llm_gen_text` in output, no cost table, `reactflow.html` assembled) · GREEN extract (`act_extract_gen_extract_pages` + `act_render_page_views` dispatched) · GREEN img-gen (`act_img_gen_images`) · GREEN search (`act_search_gen_sourced_answer`) · **no-keys GREEN** (runner booted with invalid credentials — dry run still exit 0) · **RED-proven** (DRY branch disabled in `llm_gen_text` + keyless runner → exit 1, dispatched activity failed loud with HTTP 401 carried back to the submitter) · recovery confirmed (leaf restored → exit 0).++## Distributed verification — `temporal-e2e-validate` (Mode 2 / Tier 17)++This tier **flips the current Tier 8 note** ("dry-run instantiates `ContentGeneratorDry` on the router and never dispatches `act_llm_gen_text`"): after Part B, DRY honors the backend and DOES dispatch, mocking inside the activity. That behavior change is the req-1 deliverable, so its proof is a new tier.++**Tier 17 — DRY honors the Temporal backend (leaf mock inside the activity).** New sequential tier (Step 3 family).++**Mode 2 (3-process) GREEN** — split workers up (`mode-2-setup.md`), Temporal-enabled; run a multi-step LLM bundle DRY:++```+pipelex run bundle .../library_crate/native_text_sequence.mthds --pipe native_text_sequence --temporal --dry-run --mock-inputs --no-logo --graph+```++- exit 0; **the worker DID dispatch `act_llm_gen_text`** (grep the runner session / `WorkflowHandle.fetch_history()` shows `ActivityTaskScheduled` for it) — the runner actually ran the activity. *(Pre-Part-B this dispatches nothing — that's the behavior we're changing.)*+- **The leaf mocked inside the activity:** no real provider call, **runs with NO API keys** (the "dry works with no inference configured" invariant must hold even though the activity dispatches), zero real spend, output is the DRY mock (`"DRY RUN: …"` / minted object), and usage is **zero-token / report suppressed** — distinct from `--mock-inference`, which keeps non-zero synthetic usage for a rendered cost report (kept per decision 8B).+- **No storage writes** (eng review D10): the DRY img/extract path never touches the storage provider — assert no storage IO occurred (no bucket/object writes in the worker session).+- cross-worker graph tracing assembled (`reactflow.html`); LibraryCrate propagation + serialization behaved as LIVE.+- **Extend to non-LLM leaves once B1 adds them:** rerun against extract (`pdf_extract_page_views`), img-gen (`generate_image`), and **search** (eng review D5) and assert `act_extract_gen_extract_pages` / `act_img_gen_images` / `act_search_gen_*` are dispatched and mocked inside — this **replaces** the `MockInferenceUnsupportedError` fail-loud guard at those leaves.++**Mode 2 RED (prove it bites):** temporarily key the leaf branch to LIVE (mutate the `run_mode` check in `llm_generate.py`) → the dispatched activity attempts a real call and **fails loud on missing keys / would spend** — the **dispatched + mocked-inside + no-spend + no-keys** quartet flips. Restore. *(Outside-voice fold: post-B2 there is no router-level mock to "revert to", so the RED arm is mutate-the-leaf, not revert-the-re-key.)*++**Mode 1 (pytest) companion** — `tests/integration/pipelex/temporal/tracing/test_dry_run_dispatches_and_mocks.py` (the DRY analogue of the existing `test_mock_inference_temporal.py`): assert `run_mode=DRY` over the in-process server dispatches `act_llm_gen_*` and the leaf mints the DRY mock with **zero-token, suppressed** usage, no real inference.-> ### ⛔ CHECKPOINT E — after Phase B3 — **Follow-up complete** — **MANDATORY STOP**+**Coordination with Tier 8b.** The existing `--mock-inference` cost arms (Tier 8b) are the LLM **cost-rendering** slice of this same mode (LIVE run mode + leaf mock, non-zero usage). **Decided (D8): the thin reportable-mock survives**, migrating into `CogtRunParams` — Tier 8b stays rendered-report-based, re-pointed at the migrated flag; update its scope manifest in B2. Also **update the Tier 8 note** in the `temporal-e2e-validate` skill as part of B3 — Tier 17 flips its meaning ("dry-run never dispatches" becomes pre-Part-B history).++**Step-7 master-table row to add:** `Tier 17: DRY honors the backend | a --temporal --dry-run run dispatches act_llm_gen_* (+ extract/img-gen) to the worker and mocks INSIDE the activity — no real IO, no API keys needed, zero-token suppressed usage, cross-worker graph still assembles | PASS/FAIL | path | — `.++> ### ⛔ CHECKPOINT E — after Phase B3 — **Follow-up complete** — **REACHED 2026-06-10**>> Run mode is now orthogonal to backend across all four cells. **Req 1 satisfied.** Foundation for the Temporal-validation follow-up is in place.>-> **Verify:** Temporal+DRY e2e green · full `make agent-test` green · commit.+> **Verified:** `temporal-e2e-validate` **Tier 17 GREEN and RED-proven** (all arms, incl. no-keys; see B3 checklist for the run log summary) · full `make agent-test` green · committed.>-> **Handoff (fill in):** (template) — **Next:** [`followup-temporal-validation-activity.md`](./followup-temporal-validation-activity.md) (HARD GATE — get the `temporalio` answer first).+> **Handoff:** the four `run_mode × backend` cells all behave: LIVE/direct (unchanged), LIVE/temporal (unchanged), DRY/direct (leaf mock inline, no storage), DRY/temporal (real dispatch, leaf mock inside the activity — Tier 17). The `--mock-inference` reportable-mock coexists on the LIVE row (Tier 8b unchanged at the CLI surface, flag carried by `CogtRunParams`). Branch `feature/Mock-activities` → PR against `feature/Dry-run-as-temporal-activity`. — **Next:** [`followup-temporal-validation-activity.md`](./followup-temporal-validation-activity.md) (HARD GATE — get the `temporalio` answer first).++## GSTACK REVIEW REPORT++| Review | Trigger | Why | Runs | Status | Findings |+|--------|---------|-----|------|--------|----------|+| CEO Review | `/plan-ceo-review` | Scope & strategy | 0 | — | — |+| Codex Review | `/codex review` | Independent 2nd opinion | 1 | issues→folded | 12 points: 5 cross-model agreements, 3 precision folds, 2 new adopted (DRY storage short-circuit, nb_items carrier), 1 verification note, 1 dismissed |+| Eng Review | `/plan-eng-review` | Architecture & tests (required) | 1 | CLEAR (clean) | 16 issues, 0 critical gaps open |+| Design Review | `/plan-design-review` | UI/UX gaps | 0 | — | — |+| DX Review | `/plan-devex-review` | Developer experience gaps | 0 | — | — |++Decisions folded into the phases above (2026-06-10): **D1** full CogtRunParams scope confirmed · **D2** CogtRunParams nests inside PipeRunParams holding the ONLY run_mode copy (delegating property; 12-method protocol threading named) · **D3** mock_main consolidates into one shared helper, leaf stamp conditional on DRY · **D4** boot generator selection backend-keyed only · **D5** search + render_page_views get full leaf DRY branches · **D6** fidelity wrapper re-keys to DRY-or-mock · **D7** typed terminal DryRunMockBuildError · **D8** is_mock_inference KEPT as thin reportable-mock, migrating into CogtRunParams · **D10** DRY short-circuits storage (mock above the *_and_store step) · **D11** nb_items rides ObjectAssignment.++**CODEX:** outside voice confirmed the search/render/mock_main/carrier-consistency findings independently and surfaced the two storage/nb_items issues this review had missed; its `make agent-test` complaint was wrong (that target is the repo standard) and was dismissed.++**CROSS-MODEL:** agreement on 5 of 8 in-review findings; zero unresolved tensions — both new Codex findings were verified in code and adopted (10A, 11A).++**VERDICT:** ENG CLEARED — ready to implement (B1 entry point; Tier 17 GREEN+RED is the req-1 acceptance gate).++NO UNRESOLVED DECISIONSdiff --git a/wip/dry-run-refactor/followup-temporal-validation-activity.md b/wip/dry-run-refactor/followup-temporal-validation-activity.mdindex 97d9b516d..4191c605a 100644--- a/wip/dry-run-refactor/followup-temporal-validation-activity.md+++ b/wip/dry-run-refactor/followup-temporal-validation-activity.md@@ -1,61 +1,70 @@-# Follow-up — Distributed validation as a Temporal activity+# Follow-up — Distributed dry-run + validation as one in-process Temporal activity-> **Status: deferred — own branch, not yet started, HARD-GATED.** Implements **D-plan Part C / D5 / req 2**. Split out of [`consolidation-as-built.md`](./consolidation-as-built.md) on 2026-06-01 (eng-review D1).+> **Status: in progress — branch `feature/Dry-run-as-temporal-activity`.** Implements **D-plan Part C / D5 / req 2**, expanded with the **in-memory tracing** requirement surfaced 2026-06-09. The executable phase-by-phase plan with checkboxes and checkpoints lives in [`../../TODOS.md`](../../TODOS.md) (repo root) — this doc is the durable design reference (the *why* + the decisions).>> **Design rationale:** [`D-plan.md`](./D-plan.md) §4.9 (distributed validation activity). **Risks:** D-plan §8 (nested-dispatch from the sweep, `temporalio` bump regression surface).>-> **Depends on:** the consolidation (`BundleValidator`, [`consolidation-as-built.md`](./consolidation-as-built.md)) **and** the leaf run-mode follow-up ([`followup-leaf-run-mode-mock.md`](./followup-leaf-run-mode-mock.md)). **Branch off the same D-plan.**->-> **Scope note (D2):** the consolidation already migrated `pipelex-api/build/runner.py` to call `BundleValidator` in-process. So Phase C2 here only adds the **Temporal-dispatch** half on the validate route — not the in-process consumer.+> **Depends on:** the consolidation (`BundleValidator`, [`consolidation-as-built.md`](./consolidation-as-built.md)) — **shipped (#956)**. Part B ([`followup-leaf-run-mode-mock.md`](./followup-leaf-run-mode-mock.md)) is **no longer a hard prerequisite** — see "Dependency picture, corrected" below.-Production flow (req 2): the web app calls the Pipelex API to validate a bundle; the API dispatches the validation to a Temporal worker as a **single standalone activity**; the worker runs the whole sweep in-process and returns the per-pipe status map.+## What this delivers (req 2)-## Status at a glance+Production flow: the web app calls the Pipelex API to validate a bundle; when Temporal is enabled the API dispatches the whole job — **validation sweep + graph-producing dry-run** — to a worker as **one activity**; the worker runs it all **in-process, in memory**, and returns the per-pipe status map **and** the `GraphSpec`. No usage/cost reporting (dry runs are free). In direct mode the API keeps running it in-process unchanged (req 3).-| Phase | Title | Status | Commit |-|---|---|---|---|-| | **⛔ HUMAN GATE — pre-flight** | | |-| C0 | Bump `temporalio` + verify standalone-activity support | ☐ not started | |-| | **⛔ CHECKPOINT (SDK bump, separately reviewable)** | | |-| C1 | `scoped_content_generator` + `act_validate_bundle` | ☐ | |-| C2 | API dispatch (cross-repo `pipelex-api`) | ☐ | |-| | **⛔ CHECKPOINT F — all reqs met** | | |+## Dependency picture, corrected (2026-06-09)-Status legend: ☐ not started · ◐ in progress · ☑ done.+The original brief said this work "depends on Part B (leaf-level run-mode mock)" and hinged on a new `scoped_content_generator` ContextVar. Both have been overtaken by what shipped since:-## Pre-flight — needs human input (HARD GATE)+- **The sweep already stays in-process under a Temporal hub.** PR #976 wrapped `BundleValidator`'s per-pipe loop in `scoped_pipe_router(self._pipe_router)` (`bundle_validator.py`), so nested controller sub-pipes resolve the local in-process router instead of leaking to the Temporal router. The nested-dispatch risk D-plan §8 worried about — the load-bearing reason for `scoped_content_generator` — is **already handled** for the sweep.+- **DRY already mocks inline at the pipe level.** Pre-Part-B, each operator's dry path uses `ContentGeneratorDry()` inline, so a DRY sweep never dispatches `act_llm_gen_*`. That is exactly what we want *inside* a single activity (everything in-process). Part B is about the **opposite** cell (a top-level DRY run that *should* exercise the worker path and mock inside the activity) — it does not gate hosting the sweep in one activity.+- **The usage registry is gone.** Post-#967 (distributed cost reporting) usage rides on `PipeOutput`, not a process-global report registry. The sweep accumulates no per-run reporting state (`bundle_validator.py` step-4 comment), so overlapping sweeps can't collide on it. The "open one throwaway per-sweep registry" worry from earlier drafts is moot.-- [ ] **(before C0 — HARD GATE) `temporalio` standalone activities.** Which SDK version to bump to, and does our Temporal Cloud / server version support standalone-activity execution? Needs infra knowledge/credentials. If unsupported, fall back to the one-step wrapper-workflow dispatch (§4.9). **Agent cannot resolve alone — must ask.**+**Caveat — `scoped_content_generator` is deferred, not dead.** "Largely superseded" holds *only pre-Part-B*. Today a DRY sweep mocks at the **pipe** level (`ContentGeneratorDry` inline) and never touches `get_content_generator()`, so the hub's global generator is irrelevant and `scoped_pipe_router` alone keeps the sweep in-process. But under a Temporal-enabled hub `get_content_generator()` returns `ContentGeneratorInWorkflow` **globally** (set once at boot — `pipelex.py:370-385`, not contextual). So once **Part B** relocates the DRY mock *down to the leaf* (routing it through `get_content_generator()`), this in-process activity would reach `ContentGeneratorInWorkflow` and try to dispatch `act_llm_gen_*` — breaking the in-process guarantee. **Part B therefore reintroduces the need for the inline content-generator scope here.** This is the same `scoped_content_generator` seam Part C originally specced and the same one Shape C ([`followup-leaf-run-mode-mock.md`](./followup-leaf-run-mode-mock.md)) needs — whichever follow-up lands first should build it; the other consumes it. **Recommendation: add the inline content-generator scope to this activity now**, so it is correct regardless of Part-B ordering (robust over quick).-> ### ⛔ HUMAN GATE — before C0 — **STOP, get answers**->-> Do not start C0 until the **`temporalio`** Pre-flight item is answered (target version + Temporal Cloud/server standalone-activity support). Ask the user. Record the decision (and fallback choice if standalone is unsupported) here before proceeding.+**Net:** the validation-sweep half is essentially ready to wrap. The real new work is the **graph-producing dry-run half**, specifically **in-memory graph tracing** (so it runs inside one activity without the heavyweight cross-worker DynamoDB round-trip) plus forcing the inline content generator so the leaf never dispatches.-## Phase C0 — Bump `temporalio` + verify standalone-activity support+## The graph step today vs. the new design-- [ ] Bump `temporalio` past `1.23.0` in `pyproject.toml` (+ `uv.lock` via uv) to the confirmed version; confirm server/Cloud support.-- [ ] Run the **full** Temporal e2e suite (the SDK underpins the whole worker/runtime). Treat as self-contained + separately reviewable.-- [ ] If blocked: implement the one-step wrapper-workflow fallback (§4.9) instead; note it in the Handoff.+**Today (the thin-submitter design — see the #976 reviewer note).** `pipelex-api`'s `/validate` route does two things: `validate_bundle()` (the in-process sweep → status map) and, best-effort, `dry_run_pipeline()` → a `GraphSpec`. `dry_run_pipeline` goes through `PipelexRunner.execute_pipeline` → the hub's `get_pipe_run()`, which under a Temporal hub is `TemporalPipeRun` — so it dispatches **one top-level workflow** to the worker. The worker runs the pipeline, traces to the `temporal_dynamodb` event-log backend (`BufferingEventLog` in the workflow → `act_flush_trace_events` → DynamoDB), and `act_assemble_tracing` reads the events back and assembles the `GraphSpec`. The API itself runs with tracing **off** (thin submitter); the worker owns tracing + assembly. This works because the graph step is a *single* dispatch (no concurrent same-id collision — unlike the sweep's fan-out, which #976 fixed). **Do not move this step in-process in the API** — that breaks graph generation (tracing off in the API → empty graph).-> ### ⛔ CHECKPOINT (after C0) — **MANDATORY STOP**->-> **Verify:** full Temporal e2e green on the new SDK · `make agent-test` green · commit (own PR/review for the bump). **Next entry point: Phase C1.**+**New (this follow-up).** Host the dry-run + validation in **one activity on the worker** that runs entirely in-process and traces the graph in an **in-memory** event log:-## Phase C1 — `scoped_content_generator` + `act_validate_bundle` (§4.9)+- No per-pipe child workflows / activities (the whole pipeline dry-runs in-process inside the one activity — like the sweep does).+- No `temporal_dynamodb` round-trip and no NDJSON files for the graph — the trace events live and die in memory inside the activity, just long enough to assemble the `GraphSpec`.+- No usage/cost events at all (dry run).-- [ ] *Tests first:* assert zero activity/workflow dispatches during a sweep run under a Temporal-enabled hub; assert concurrent invocations don't cross-contaminate the override.-- [ ] Add `_content_generator_override: ContextVar` + `scoped_content_generator(...)` mirroring `_library_id` / `scoped_current_library` in `hub.py`; make `get_content_generator()` prefer the override.-- [ ] Wrap `BundleValidator`'s sweep in `with scoped_content_generator(inline_dry_generator):` (forces the inline leaf; never dispatches nested activities).-- [ ] Add `tprl_pipe/act_validate_bundle.py` (thin wrapper over `BundleValidator.validate`) with the `convert_pipelex_errors` boundary; serializable in (`mthds_contents`/dirs, `allow_signatures`, `--pipe` selection) and out (`{pipe_ref: DryRunOutput}` + signature-check error).-- [ ] Register the activity on the worker; integration-test it in isolation. `make agent-check` + `make agent-test` green.+This is lighter than the current top-level-workflow + DynamoDB path and is the natural shape for "validate this bundle and show me its graph" — the webapp's actual call.-## Phase C2 — API dispatch (cross-repo `pipelex-api`)+## The core technical problem — emit and assemble use *two* event-log instances-- [ ] In `../pipelex-api`: switch the **validate route** to dispatch `act_validate_bundle` (standalone) when Temporal enabled, else call `BundleValidator` in-process. *(The `build/runner.py` consumer is already migrated in the consolidation — D2.)*-- [ ] Test both backends against the API.+The blocker that makes "just add an in-memory backend" insufficient: the write side and the read side each construct their **own** event log from config, and only an **external store** bridges them.-> ### ⛔ CHECKPOINT F — after Phase C2 — **ALL REQUIREMENTS MET**->-> req 1 (distributed DRY testing, activity-level mocks) · req 2 (production validation as a standalone Temporal activity) · req 3 (direct in-process).->-> **Verify:** `pipelex` `make agent-test` green · Temporal e2e green · `pipelex-api` tests green both backends · commit. **Handoff:** final state + any follow-ups (e.g. §7 API endpoint unification).+- **Write side:** `pipeline_run_setup` (`pipeline_run_setup.py`) calls `make_event_log(tracing_config)` and hands it to the graph tracer, which emits `PipeStart`/`PipeEnd`/edge events into it during the run.+- **Read side:** after the run, `PipeRun.run`'s `finally` calls `assemble_tracing_on_output` → `assemble_tracing` (`tracing_assembly.py`), which calls `make_event_log(tracing_config)` **again** — a brand-new instance — and `read_events(pipeline_run_id)` to rebuild the `GraphSpec`.++For NDJSON the two instances connect through the file on disk; for DynamoDB through the table. For a plain `InMemoryEventLog` they would be **two empty lists** — the reader sees nothing.++`InMemoryEventLog` already exists (`pipelex/tracing/in_memory_event_log.py`, fully implements `EventLogProtocol`, used in unit tests). It is not the missing piece. **The missing piece is a way to share one instance across emit and assemble for the duration of a run.**++### The fix — a scoped event-log override (mirror the existing scope pattern)++Add a hub-level `_event_log_override: ContextVar[EventLogProtocol | None]` + a `scoped_event_log(event_log)` context manager, exactly mirroring `scoped_current_library` and `scoped_pipe_router` (`hub.py`). Make **both** `make_event_log` call sites (`pipeline_run_setup` write, `assemble_tracing` read) prefer the override when set, else fall back to `make_event_log(tracing_config)`. The activity wraps its dry-run in `with scoped_event_log(InMemoryEventLog()):` so emit and assemble hit the **same** instance — fully in memory, no file, no DynamoDB. ContextVar-scoped ⇒ coroutine-local ⇒ concurrent activities on one worker don't cross-contaminate (same property that makes `scoped_pipe_router` concurrency-safe).++**Decision (default): scope-only; no new `TracingBackend.IN_MEMORY` config enum.** A config-selectable in-memory backend is *insufficient on its own* (the two-instance problem) unless the factory caches a per-run singleton keyed by run id — uglier than the scope, and it would change the global default rather than being opt-in per call. The scope is the clean mechanism. (Revisit only if a use case wants in-memory tracing selected purely by config, with no caller scope.)++## The in-process graph dry-run must be safe under a Temporal hub++`dry_run_pipeline` → `PipelexRunner.execute_pipeline` → `get_pipe_run()` → `TemporalPipeRun` under a Temporal hub → dispatches a workflow. Inside an activity that is illegal, and it defeats the in-process goal. The runtime bridge already solved this exact "force in-process even inside a Temporal worker" problem: `runtime_bridge.bridge._run_direct` scopes a local `PipeRouter` via `scoped_pipe_router` + runs a local `PipeRun`, and `run_pipe_via_bridge` already accepts a `trace_context` that is honored in DIRECT mode. So the in-process graph dry-run = open a `GraphTracerManager` tracer against an `InMemoryEventLog` (under `scoped_event_log`), build the `trace_context`, run the main pipe through the **bridge DIRECT path** (not `PipelexRunner`/`get_pipe_run()`), and the `GraphSpec` rides back on `PipeOutput` — assembled from the same in-memory log. Net: GraphSpec produced, zero nested dispatch, zero files/DDB.++## The activity + dispatch++- **`act_dry_validate`** — a thin activity wrapping the validation sweep (`BundleValidator`) **and** the in-memory graph dry-run, wrapped in `scoped_event_log(InMemoryEventLog())` + `scoped_content_generator(inline)` and the existing `convert_pipelex_errors` boundary (`tprl/activity_error_boundary.py`) so failures cross back as structured `ErrorReport`s. The graph dry-run is **best-effort inside the activity** (a graph failure returns `graph=None` with validation still successful — today's `/validate` contract). Inputs (serializable): `mthds_contents` / `library_dirs` / `bundle_uris`, `allow_signatures`, optional `--pipe` selection. Output (serializable): `{pipe_ref: DryRunOutput}` map + `GraphSpec | None` + the aggregated signature-check error. Registered in `pipelex/temporal/tasks.py` (the `PackName.PIPE` lists).+- **Dispatch — DECIDED (2026-06-09): one-step wrapper workflow.** A one-step workflow runs the single activity and returns — works on the current `temporalio` (`1.23.0`), **no SDK bump, no Temporal Cloud/server verification, no hard gate**. Functionally identical to the caller. A true standalone activity (no workflow) is a **later optional optimization** only — deferred, off the critical path.+- **API wiring (cross-repo `pipelex-api`).** When Temporal enabled, `/validate` dispatches the wrapper workflow and awaits `{status map, GraphSpec}`; in direct mode it keeps the current in-process `validate_bundle` + `dry_run_pipeline`. This *replaces* today's "graph as a top-level workflow to the worker + DynamoDB" path with the single-activity in-memory path.++## ✅ Former HUMAN GATE — RESOLVED (2026-06-09)++The `temporalio` standalone-activity question is **resolved**: dispatch uses the **one-step wrapper workflow** on the current `temporalio`. No bump, no infra verification, nothing left to ask — the whole plan (Phases 1→4 in [`../../TODOS.md`](../../TODOS.md)) runs without stopping for human input. Switching to a true standalone activity later is a pure runtime optimization (Phase G0, deferred); only *that* would need the version + Cloud/server-support facts.++## Acceptance (Checkpoint F)++req 1 (distributed DRY testing, activity-level mocks — delivered by Part B, not here) · **req 2 (production dry-run + validation as one in-process Temporal activity, in-memory tracing) — this follow-up** · req 3 (direct in-process unchanged).diff --git a/wip/dry-run-refactor/plan-unified-dry-run.md b/wip/dry-run-refactor/plan-unified-dry-run.mdnew file mode 100644index 000000000..68a040418--- /dev/null+++ b/wip/dry-run-refactor/plan-unified-dry-run.md@@ -0,0 +1,77 @@+# Plan — Unified dry run (retire `is_mock_inference`; add internal `is_mock_usage` flag)++> **Status: COMPLETE (2026-06-10) — all phases done, `make agent-check` + full `make agent-test` green.** Implements [`design-unified-dry-run.md`](./design-unified-dry-run.md) (design agreed with the user). Baseline: `feature/Mock-activities` as-built (Part B complete + Part C validation activity merged in).++## Plan-time decisions (resolving design §5)++The design deferred these to plan time; resolved here, taking the design's proposed options:++- **`is_mock_usage=True` + `run_mode=LIVE` is a validation error** — a pydantic model-validator on `CogtRunParams` rejects it (fail loud, per the design's proposal). The CLI never produces the combination (the hidden flag requires `--dry-run`), so the validator only fires on a buggy embedder call.+- **E2E trigger = hidden CLI flag `--mock-usage`** (typer `hidden=True` on `run pipe` / `run bundle` / `run method`, requires `--dry-run`). Present for the `temporal-e2e-validate` Mode-2 scripts, absent from `--help` and docs.+- **`MockInferenceObjectFidelityError` → renamed `DryRunObjectFidelityError`** — mock-built objects are now DRY-only, the old name is a lie; no backward compat per project principles. Error pages regenerated.+- **Sentinel naming**: two sentinels stay — `dry_run` (zero-token) unchanged; `mock_inference` → `mock_usage`. No fold into a single sentinel: tests filter usage events by sentinel name, and two names keep the two reporting payloads grep-able.+- **Non-LLM leaves do NOT emit synthetic usage under `is_mock_usage=True`** (minimal design — matches what the cost-assembly tests assert).+- **`CogtRunParams.is_mock_built` property deleted** (not kept as alias): mock-built is now exactly `run_mode.is_dry`; keeping a second name for one predicate is the disease this refactor cures. Fidelity-guard call sites read `run_mode.is_dry`.+- **PipeCompose dry arm: untouched** (explicitly parked by the user).++## Phases++### Phase 1 — Carrier + factory + entry points++- [x] `CogtRunParams`: `is_mock_inference` → `is_mock_usage` with new semantics docstring; add model-validator rejecting `is_mock_usage=True` with `run_mode.is_live`; delete `is_mock_built` property. Frozen/extra-forbid/required-run_mode unchanged.+- [x] `PipeRunParamsFactory.make_run_params`: param re-keyed `is_mock_inference` → `is_mock_usage`; forced-DRY path simplified (single warning; no swallow arm — `is_mock_usage` rides DRY fine).+- [x] Entry-point threading re-keyed: `PipelexRunner.__init__` / `execute_pipeline`, `pipeline_run_setup`, `prepare_pipe_job` (`execution_seams.py`). Docstrings rewritten for the sub-flag semantics.+- [x] Fidelity guard call sites (`content_generator.py` ×3, `content_generator_in_workflow.py` ×3) re-keyed from `is_mock_built` to `run_mode.is_dry`.++### Phase 2 — Leaves++- [x] `llm_generate.py`: dual branch collapses to one `if run_mode.is_dry` per leaf; dry helpers select report func on `is_mock_usage`.+- [x] `dry_mock.py`: delete `mock_llm_gen_text` / `mock_llm_gen_object` / `mock_llm_gen_object_list` / `_mock_text` and the `MOCK INFERENCE` marker; rename `report_mock_inference_llm_job` → `report_mock_usage_llm_job`, `MOCK_INFERENCE_MODEL_NAME/ID` → `MOCK_USAGE_*` (= `"mock_usage"`), `MOCK_INFERENCE_NB_TOKENS_BY_CATEGORY` → `MOCK_USAGE_NB_TOKENS_BY_CATEGORY`; dry helpers pick report func via a shared `_dry_report_func(cogt_run_params)`; `stamp_mock_main_coordination` docstring sheds the mock-inference caveat (the dry object-list leaf stamps regardless of `is_mock_usage`); module docstring rewritten for the one-mode world.+- [x] Delete the `MockInferenceUnsupportedError` guards: `img_gen_generate.py` (×2), `extract_generate.py`, `search_generate.py` (`_guard_no_mock_inference`).++### Phase 3 — Errors + config lists++- [x] Delete `MockInferenceUnsupportedError` from `cogt/content_generation/exceptions.py`; remove it from `non_retryable_error_types` in all three toml copies (`pipelex/pipelex.toml`, `pipelex/kit/configs/pipelex.toml`, `.pipelex/pipelex.toml`).+- [x] Rename `MockInferenceObjectFidelityError` → `DryRunObjectFidelityError`; message/docstring re-scoped to DRY-only.+- [x] Regenerate error pages (`make gep`): `mock-inference-unsupported-error.md` removed, fidelity page renamed.++### ⛔ CHECKPOINT 1 — core re-key compiles and unit tests pass — **CLEARED 2026-06-10** (targeted cogt/cli/pipe_run/temporal unit + cogt integration runs green)++### Phase 4 — CLI++- [x] Remove `--mock-inference` from `pipe_cmd.py` / `bundle_cmd.py` / `method_cmd.py`; add hidden `--mock-usage` (typer `hidden=True`).+- [x] `validate_run_flag_combination`: mutual-exclusion arm replaced by "`--mock-usage` requires `--dry-run`"; `--mock-inputs` rule unchanged.+- [x] `_run_core.py`: `mock_inference` params → `mock_usage`, threaded to `PipelexRunner(is_mock_usage=...)`.++### Phase 5 — Tests (TDD where behavior is new: validator + CLI flag rule written red-first)++- [x] `test_cogt_run_params_carrier.py`: re-key; add the LIVE+`is_mock_usage` rejection test (red first).+- [x] `test_run_flag_combination.py`: new `--mock-usage`-requires-`--dry-run` arm (red first); exclusion arm dropped.+- [x] Merge `test_llm_generate_mock_branch.py` into `test_llm_generate_dry_branch.py` parameterized over `is_mock_usage` (zero-token suppressed vs non-zero rendered payloads asserted); delete the mock-branch file.+- [x] Delete guard tests: `test_extract_generate_mock_guard.py`, `test_img_gen_generate_mock_guard.py`, `test_pipe_search_mock_inference_guard.py`, `test_mock_inference_cli_guard.py` (folded into flag-combination tests).+- [x] Re-key + rename: `test_mock_inference_temporal.py` → `test_mock_usage_temporal.py` (wire survival on the carrier, DRY + `is_mock_usage`), `test_mock_inference_direct.py` → `test_mock_usage_direct.py`, `test_mock_inference_object_fidelity.py` → `test_dry_run_object_fidelity.py`, `test_content_generator_in_workflow_object_fidelity.py` (error rename).+- [x] Fidelity tests (D6 arms): re-key from two triggers to DRY-only; `test_leaf_dry_object_mocks.py` stamp test now asserts stamping regardless of `is_mock_usage`.+- [x] Keyless-boot tests: drop the swallow-warning arm.+- [x] `test_dry_mock.py`, `test_traceback_flag_run_core.py`, tracing `test_data.py`/`helpers.py`: mechanical re-keys.++### ⛔ CHECKPOINT 2 — full suite green (`make agent-check` + `make agent-test`) — **CLEARED 2026-06-10** (0 lint/type errors, full suite passed)++### Phase 6 — Skill + docs + changelog++- [x] `temporal-e2e-validate` skill: Tier 8b arms re-pointed to `--dry-run --mock-usage` (still asserting non-zero aggregated values); Tier 17 prose drops the "distinct from `--mock-inference`" contrast and states the sub-flag; SKILL.md description + routing battery + scope manifests updated.+- [x] `docs/building-methods/pipes/run-modes-and-backends.md`: matrix collapses to one mode axis × two backends; mock inference disappears as a user-facing concept.+- [x] Purge `--mock-inference` from `docs/features/validation-dry-run.md`, `docs/under-the-hood/dry-run-mock-generation.md`, `docs/building-methods/pipes/executing-pipelines.md`; error pages already regenerated in Phase 3.+- [x] CHANGELOG `[Unreleased]`: breaking — `--mock-inference` removed; `CogtRunParams.is_mock_inference` → internal `is_mock_usage` DRY sub-flag.+- [x] wip docs: this plan's checkboxes updated; `wip/registry/deferred-followups.md` "fate of is_mock_inference" item closed; `README.md` updated.++### Phase 7 — Final verification++- [x] `make agent-check` + `make agent-test` green; commit.++## Post-review change (PR #986 round 2, human review)++Reviewer + user direction: `run_mode` belongs on `PipeRunParams` directly. Implemented as an inversion of the D2 carrier shape — `run_mode` and `is_mock_usage` are direct fields of `PipeRunParams` (single stored copy, model-validator enforces the DRY-only sub-flag rule); `cogt_run_params` became a **derived property** minting the frozen cogt-tier carrier on demand. The wire contract is unchanged (`CogtRunParams` still stamped on every assignment, still validates at its own boundary); operators' `pipe_run_params.cogt_run_params` reads work as before. The factory validates the REQUESTED mode before the keyless forced-DRY coercion so the contract violation cannot be masked (greptile round-1 finding, kept).++## Cross-repo note (design §3.7)++`pipelex-api` carries `is_mock_inference` on its public runner API params (fed from `_run_core` → runner). That change rides a pipelex version bump like any other public-surface change — grep `mock_inference` in `pipelex-api/` when bumping the pin there; NOT part of this branch.diff --git a/wip/registry/README.md b/wip/registry/README.mdindex 1ee07a0f8..6b7af21a8 100644--- a/wip/registry/README.md+++ b/wip/registry/README.md@@ -1,6 +1,6 @@# `UsageRegistry` leak fix + distributed cost reporting-Cost reporting was reworked from a leaky per-run, in-process `UsageRegistry` buffer into an **event-sourced artifact that rides back on `PipeOutput`**. The success-path registry leak is gone by removal; a multi-worker (Temporal) run now aggregates usage from every worker into a single end-of-run cost report. A default-on `--costs` switch decouples cost collection from `--graph` over a shared event-log transport, and `--mock-inference` validates the distributed path cheaply.+Cost reporting was reworked from a leaky per-run, in-process `UsageRegistry` buffer into an **event-sourced artifact that rides back on `PipeOutput`**. The success-path registry leak is gone by removal; a multi-worker (Temporal) run now aggregates usage from every worker into a single end-of-run cost report. A default-on `--costs` switch decouples cost collection from `--graph` over a shared event-log transport, and a cheap deterministic mock validates the distributed path (originally `--mock-inference`; since the unified dry run it is the internal `--dry-run --mock-usage` trigger — see `../dry-run-refactor/plan-unified-dry-run.md`).Shipped in **PR #967** on `fix/For-API-update`, with the companion rename **#968** (`GraphContext → TraceContext`, `is_generate_costs → is_generate_usage`).diff --git a/wip/registry/deferred-followups.md b/wip/registry/deferred-followups.mdindex 2b047b83c..113c89165 100644--- a/wip/registry/deferred-followups.md+++ b/wip/registry/deferred-followups.md@@ -32,10 +32,12 @@ These were found by the gstack `/review` adversarial pass (Codex + Claude). They- **DIRECT vs TEMPORAL object re-validation use different dump modes.** `ContentGenerator._revalidate_against_object_class` (DIRECT) dumps with `model_dump(serialize_as_any=True)` (python mode) while `ContentGeneratorInWorkflow` (Temporal) uses `model_dump(mode="json", serialize_as_any=True)`. Same logical re-validation, two serialization modes — so a `--mock-inference` synthetic object (or any object) can round-trip cleanly in one execution mode and raise `MockInferenceObjectFidelityError` in the other. The divergence predates this PR (each arm kept its existing behavior), but the shared-helper framing makes it load-bearing. Aligning the two on one canonical mode is the durable fix; which mode is canonical is the judgment call.-## `--mock-inference` coverage+## `--mock-inference` coverage — RESOLVED (2026-06-10, unified dry run)++> **All items below are closed by the unified dry run** (`wip/dry-run-refactor/design-unified-dry-run.md` + `plan-unified-dry-run.md`): `--mock-inference` is removed as a mode; the reportable-usage capability is the internal `CogtRunParams.is_mock_usage` sub-flag of DRY; `MockInferenceUnsupportedError` and the per-leaf guards are deleted (dry run covers every leaf). Kept as the historical record.- **LLM leaf only.** `--mock-inference` mocks the LLM inference leaf (`llm_gen_text`/`object`/`object_list`). Image-gen / extract / search **fail loud** with `MockInferenceUnsupportedError` (their output is stored *above* the leaf by `generated_content_factory`, so a leaf-level mock would push synthetic data through the storage path — whereas `ContentGeneratorDry` short-circuits storage). Full per-operator synthetic-output coverage comes via [`../dry-run-refactor/followup-leaf-run-mode-mock.md`](../dry-run-refactor/followup-leaf-run-mode-mock.md) B2, which replaces each hard guard with a real synthetic-output branch.- **Main `run` subcommands only.** The flag is on `pipelex run pipe|method|bundle`, not the agent CLI. The mechanism (`JobMetadata.is_mock_inference` + leaf branch + `PipelexRunner`) is surface-agnostic, so the agent CLI can adopt it trivially when needed.-- **`is_mock_inference` fate after the dry-run-refactor follow-up.** Decided 2026-06-06: build the interim flag now (Phase 5 is the follow-up's B1 core — a reusable leaf-level dry helper), settle retirement at B2. Open sub-decision: *fully retire* `--mock-inference` (DRY-on-Temporal covers cross-worker dispatch + `tokens_usages` assembly once it dispatches) vs *keep a thin reportable-mock* (DRY emits zero-token usage → suppressed report, so the cheap *rendered* cost-report validation — Tier 8b — is unique to a non-zero mock). Decide at B2 time. See [`../dry-run-refactor/followup-leaf-run-mode-mock.md`](../dry-run-refactor/followup-leaf-run-mode-mock.md).+- **`is_mock_inference` fate after the dry-run-refactor follow-up — SETTLED 2026-06-10: fully retired as a mode, capability kept as the internal `is_mock_usage` DRY sub-flag (supersedes the interim D8 keep).** Original framing — decided 2026-06-06: build the interim flag now (Phase 5 is the follow-up's B1 core — a reusable leaf-level dry helper), settle retirement at B2. Open sub-decision: *fully retire* `--mock-inference` (DRY-on-Temporal covers cross-worker dispatch + `tokens_usages` assembly once it dispatches) vs *keep a thin reportable-mock* (DRY emits zero-token usage → suppressed report, so the cheap *rendered* cost-report validation — Tier 8b — is unique to a non-zero mock). Decide at B2 time. See [`../dry-run-refactor/followup-leaf-run-mode-mock.md`](../dry-run-refactor/followup-leaf-run-mode-mock.md).