Skip to content

Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.

Test coverage grind: offline unit tests for CLI, Temporal, inference plumbing, core runtime, and tools — plus the source bugs they surfaced (#989)

Louis Choquel3e69eb947e2d
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 41df16599..260611da4 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,5 +1,24 @@
# Changelog
+## [Unreleased]
+
+### Added
+
+- **Unit tests for CLI internal logic.** New offline test coverage for the pipelex-internal logic behind the CLI commands (the spec'd CLI *interface* remains owned by the conformance suite): the doctor diagnostic checks (config files, telemetry, backend credentials/files, deck sync, model checks, fix mode, health-report rendering), the `pipelex run` core execution and its sync wrapper (bundle/main_pipe resolution, inputs loading, output/graph/working-memory/CSV saving, error-handler dispatch), the `build structures|inputs|output|runner` codegen cores, the readiness gate, the `show`/`which` report logic, and the gateway/telemetry/signature error handlers.
+- **Unit tests for inference plumbing, core runtime, and tools/config internals.** Offline coverage for the layers around inference and execution that can break independently of any provider: the structured-output ↔ instructor mode mapping, backend-credentials error messages, model-deck reference checks (presets/aliases/waterfalls/handles with fuzzy suggestions), the per-provider image-gen argument factory (Flux/Qwen/SDXL/GPT-image taxonomies), the LLM and image-gen worker routing factories (SDK dispatch, client caching, missing-dependency errors), gateway completions request-shaping and extract-output parsing (Azure/Mistral/Deepseek/Linkup), and the Mistral factory (messages, OCR extract output, document prep/upload). Core runtime: the local observer JSONL sink, the output renderer's Anything-concept resolution, the bundle-level graph rendering dispatch, the builder bundle-spec validation/`to_blueprint()`/pretty rendering and inputs operations, and the pipeline runner's error paths and MTHDS protocol surfaces (`validate`/`models`/`version`). Tools: the TOML config-sync engine (structure/comment preservation, dry-run, idempotency), document/image format enums, PIL image conversion, and the storage config validators.
+- **Unit tests for the Temporal deploy-critical entry points.** Offline coverage for the operational surfaces that previously only failed inside a live cluster: the worker CLI (project resolution from pyproject, fast-fail task-queue validation before library load, worker-base library loading, forced `temporal.is_enabled`, full Typer arg wiring into the worker loop), the codec HTTP server for the Temporal Web UI (encode/decode round-trip over real HTTP, CORS preflight and origin gating, content-type/malformed-payload guards, storage-failure → HTTP status mapping), and the server connection logic (API-key resolution per secret method, TLS/RPC-metadata wiring, payload-codec converter selection, named server-config selection, SDK error wrapping).
+
+### Fixed
+
+- **OpenAI image moderation mapping was inverted.** `is_moderated = true` sent OpenAI's gpt-image `moderation="low"` (the LESS restrictive setting) and `false` sent `"auto"` (standard filtering) — the exact opposite of the documented "enable content moderation" intent. The mapping now matches the flag: enabled → `"auto"`, disabled → `"low"`. Alongside the mapping fix, `ImgGenSetting.is_moderated` now defaults to `None` ("no explicit choice") instead of `False`: workers omit the moderation/safety-checker parameter so the provider's own default applies (OpenAI's documented default is `auto`). Without this, model-deck alias/handle resolution — which builds settings without the field — would have started sending the less restrictive `low` by default once the mapping was corrected. For FAL models this also means `enable_safety_checker` is no longer explicitly disabled by default; the provider default applies there too.
+- **Storage config validation now covers every provider and requires a real `{hash}` replacement field in `uri_format`.** GCP's `lazy_validate` accepted any uri_format containing the bare substring `hash` (e.g. `assets/myhash/file`), which has no substitution slot and would render the same URI for every stored object — silent storage-wide overwrites; the local and in-memory configs had no validation at all, leaving the same misconfiguration uncaught. The check parses the format string: every placeholder must be a plain `{name}` (escaped `{{hash}}`, format specs like `{hash:.0}` or `{primary_id:.0}`, conversions, and attribute/index access are all rejected — on any field they render constant, colliding, or oversized keys, so the rule is enforced uniformly rather than only on `{hash}`), placeholders outside the supported set (`{primary_id}`, `{secondary_id}`, `{hash}`, `{extension}`) are rejected at config time instead of crashing with `KeyError` at content-store time, a `{hash}` placeholder is required, a test rendering catches anything parsing can't, and bucket configs must set a positive `signed_urls_lifespan_seconds` (or `"disabled"`) so signed URLs can't be born dead. Operator note: this validation runs when the storage provider is constructed at Pipelex boot — a misconfigured `uri_format` (including for the default local method) now fails fast at startup rather than at the first content store. The per-provider checks and the operator-facing error message now live on a shared config base (so the providers can no longer drift apart), the storage provider factory validates local/in-memory configs the same way it already did S3/GCP, and `StorageProviderConfig.storage_path` dispatches on the configured method instead of blaming a missing local config when the method is S3/GCP.
+- **Mistral chat requests now send the system message before the user message.** `MistralFactory.make_simple_messages` appended the system message after the user message, contradicting its own docstring and the OpenAI-typed sibling method.
+- **String concept values in bundle specs are now constructible and load correctly.** Validating a string against the `ConceptSpec | str` union crashed in a `mode="before"` validator that assumed dict input, and `to_blueprint()` mapped the string into the blueprint's `structure` field, where the concept loader rejects anything that isn't a registered structure class name. A string concept value now validates cleanly and passes through to the blueprint as the concept's description.
+- **LocalObserver JSONL records always carry the true lifecycle event name.** A payload carrying its own `event_type` key could silently overwrite the event name in the written record, breaking event-type-based filtering for JSONL consumers; the event name now wins the merge.
+- **Temporal worker startup no longer logs a misleading project name.** When the worker is started for the current project (no explicit project argument), the startup log said `Starting worker for current project 'None'`; it now omits the name instead of interpolating the unresolved value.
+- **Tests under `tests/**/build/` directories were silently skipped.** pytest's default `norecursedirs` includes `build`, so the whole `tests/unit/pipelex/cli/commands/build/` directory (mirroring the source layout) was never collected in full runs — including a pre-existing cross-package refines regression test. The pytest config now overrides `norecursedirs` to the defaults minus `build`, and sets `testpaths = ["tests"]` so bare `pytest` runs never scan outside the test tree.
+- **`Anything`-output option numbering is now deterministic.** The output renderer iterated a pipe-dependency set when collecting a `PipeCondition`'s possible outputs, so the user-facing `output_option_N` / `schema_option_N` numbering could vary between runs; dependencies are now sorted by pipe code.
+
## [v0.33.0] - 2026-06-11
### Breaking Changes
diff --git a/TODOS.md b/TODOS.md
index d91bd6a86..ee01eb54d 100644
--- a/TODOS.md
+++ b/TODOS.md
@@ -1,55 +1,50 @@
-# Burn down the nondeterminism review follow-ups
+# Reviewer's guide — `feature/Add-tests`
-> **Cold-start brief for a new session.** Mission: go through every item in [`wip/distributed-execution/nondeterminism-fix-review-follow-ups.md`](wip/distributed-execution/nondeterminism-fix-review-follow-ups.md), re-verify each against the current code, **fix the obvious ones**, **dismiss the wrong ones** (record the dismissal + reason in the follow-ups doc, don't silently delete), and **surface the judgment-call ones** in a new `.md` for discussion with the user. Do not start new feature work.
+This branch is a test-coverage grind over the offline-testable layers of pipelex, plus the source bugs that writing those tests surfaced. The diff is almost entirely new unit tests; the source changes are a small, deliberate set of bug fixes, each pinned by a test in the same branch. Overall coverage went from ~72% to ~78% (per-module before/after numbers live in [wip/tests/missing-tests-menu.md](wip/tests/missing-tests-menu.md)).
-## Context (read first, in this order)
+## How to review this PR
-1. `wip/distributed-execution/workflow-nondeterminism-audit.md` — the verified audit. H1, H2, M1 are FIXED; the MEDIUM/LOW drift table is open but **out of scope for this session** (it has its own fix-order, steps 3–5).
-2. `wip/distributed-execution/nondeterminism-fix-review-follow-ups.md` — the work list. Items came from two recall-biased multi-agent review rounds: round 1 reviewed the committed H1/M1 fixes, round 2 reviewed the H2 fix. Every item was verified once by a review agent, but **re-verify before acting** — line numbers drift and some verdicts may not survive contact with the current code.
+Start with the source changes (next section) — they are the only behavior changes. Then sample the test modules with the conventions section in mind: several patterns that look unusual are deliberate and documented below, so check there before flagging.
-Branch state when this brief was written: `fix/Config`, H1/M1 fixes committed; the H2 fix (new `LibraryManager.open_fresh_library` + eviction-safe `finally` reorder in `WfPipeRouter`) was uncommitted in the working tree. **Check `git status` first** — the user may have committed it since. Full suite was green (`make agent-test`) and `make agent-check` clean at that point.
+## Source changes (the behavior diff)
-## Triage protocol
+Every fix below was found by writing a pinning test first, judging the pinned behavior wrong in a review pass, then fixing source and flipping the test. The investigation record is [wip/tests/deferred-source-bugs-pinned-by-tests.md](wip/tests/deferred-source-bugs-pinned-by-tests.md); user-facing wording is in `CHANGELOG.md` under `[Unreleased]`.
-For each item: re-verify → then one of:
+- `pipelex/plugins/openai/openai_img_gen_factory.py` + `pipelex/cogt/img_gen/img_gen_setting.py` — the image-moderation mapping was inverted: `is_moderated=true` sent OpenAI's *less* restrictive `"low"` and `false` sent `"auto"`. Now enabled → `"auto"`, disabled → `"low"`. To keep the corrected mapping from silently downgrading default runs (deck alias/handle resolution builds settings without the field, and the old `False` default would have started sending `"low"`), `ImgGenSetting.is_moderated` now defaults to `None` — workers omit the parameter and the provider default applies.
+- `pipelex/tools/storage/storage_config.py` + `storage_provider_factory.py` — the biggest source diff. GCP's `lazy_validate` accepted any `uri_format` containing the bare substring `hash` (no substitution slot → every object renders the same URI → silent storage-wide overwrites), and the local/in-memory configs had no validation at all. The per-provider checks and error-message assembly now live on a shared `StorageMethodConfig` base that parses the format string: every placeholder must be a plain `{name}` (a format spec, conversion, or attribute/index access is rejected on *any* supported field, not just `{hash}` — `{primary_id:.0}` truncates to a colliding key and `{primary_id:1000000000}` would balloon the test render, same hazard class as `{hash:.0}`), a `{hash}` placeholder is required, unknown placeholders are rejected against the supported set, a test rendering as backstop, positive signed-URL lifespans on bucket configs; the factory validates every method; `storage_path` dispatches on the configured method instead of blaming a missing local config.
+- `pipelex/builder/bundle_spec.py` + `pipelex/builder/concept/concept_spec.py` + `pipelex/core/concepts/concept_blueprint.py` — string concept values in bundle specs were unconstructible (a `mode="before"` validator assumed dict input and crashed on the `ConceptSpec | str` union) and `to_blueprint()` mapped the string into `structure`, where the loader rejects anything that isn't a registered structure class. Both before-validators (`ConceptSpec` and `ConceptBlueprint`) now fall through untouched on non-dict input, and the string passes through as the concept's description (matching `ConceptFactory.make_from_blueprint`, which treats a bare string as `ConceptDeclarationType.STRING`).
+- `pipelex/plugins/mistral/mistral_factory.py` — `make_simple_messages` appended the system message *after* the user message, contradicting its own docstring and the OpenAI-typed sibling. System now comes first.
+- `pipelex/observer/local_observer.py` — a payload carrying its own `event_type` key silently overwrote the lifecycle event name in the JSONL record; merge order flipped so the event name wins.
+- `pipelex/temporal/worker_cli.py` — startup log said `Starting worker for current project 'None'` when no project was given; it now omits the name.
+- `pipelex/core/pipes/output/output_renderer.py` — a `PipeCondition`'s possible outputs were collected by iterating a dependency *set*, so the user-facing `output_option_N` numbering was nondeterministic with two or more mapped pipes; now sorted by pipe code.
+- `pyproject.toml` — **pytest's default `norecursedirs` includes `build`**, so everything under `tests/unit/pipelex/cli/commands/build/` (mirroring the source layout) was silently never collected — including a pre-existing regression test. The config now overrides `norecursedirs` to the defaults minus `build`, plus `testpaths = ["tests"]` so bare runs never scan outside the test tree. This is load-bearing for the whole branch: without it a chunk of the new tests wouldn't run in CI.
-- **FIX** — obvious, low-risk, verdict holds: implement, with tests where the repo's TDD norm applies (red first for behavior changes; no tests for trivial comment/message fixes).
-- **DISMISS** — wrong, stale, or unreachable: edit the follow-ups doc to mark it dismissed with the evidence (file:line), so it doesn't get re-chased.
-- **SURFACE** — real but a judgment call (design tradeoff, behavior change with ripple, multiple defensible shapes): write it up in a new `wip/distributed-execution/nondeterminism-follow-ups-decisions-needed.md` with the options and a recommendation, and leave the code alone.
+## Test layout
-## The items, with expected triage (verify anyway)
+Tests mirror the source tree under `tests/unit/pipelex/`. The grind covered five areas:
-Check the box when the item reaches its terminal state (FIXED / DISMISSED / SURFACED), and append the outcome to the line — e.g. `→ FIXED (commit abc123)`, `→ DISMISSED (unreachable, see follow-ups doc)`, `→ SURFACED (decisions doc §2)`.
+- **CLI internals** — `tests/unit/pipelex/cli/` and `tests/unit/pipelex/cli/commands/build/`: doctor checks, run core + wrapper, build codegen cores, readiness gate, show/which, error handlers.
+- **Temporal entry points** — `tests/unit/pipelex/temporal/`: worker CLI, codec HTTP server (driven over real HTTP via `aiohttp.test_utils`), server connection logic. These are deploy-critical surfaces whose bugs previously only showed inside a live cluster.
+- **Inference plumbing** — `tests/unit/pipelex/cogt/{llm,img_gen,models,model_backends}/` and `tests/unit/pipelex/plugins/{gateway,mistral}/`: structured-output mode mapping, credentials messages, model-deck reference checks, the per-provider img-gen args factory, both worker-routing factories, gateway request shaping + extract-output parsing, the Mistral factory. None of these tests touch a provider.
+- **Core runtime** — `tests/unit/pipelex/{observer,core/pipes/output,graph,builder,pipeline}/`: local observer JSONL sink, output renderer Anything-resolution, bundle-level graph dispatch, bundle-spec validation/`to_blueprint()`/pretty rendering, inputs ops, pipeline runner error paths and MTHDS protocol surfaces.
+- **Tools/config** — `tests/unit/pipelex/tools/{misc,storage}/`: the TOML config-sync engine (rewrites user config in place — tests pin the destroy-config guards: never creates keys, preserves comments/structure, dry-run is byte-identical, idempotent), format enums, PIL conversion, storage config validators.
-- [x] **0 — PRIORITY: rekey per-run worker-local state by `run_id`, not `workflow_id`** → FIXED. `wf_run_id` keys library/tracer/event-log context (and event/node-id stamps — one identity); `open_fresh_library` made pop-based (atomic, concurrent-teardown tolerant). New RED-first guard `test_wf_pipe_router_run_scoped_state_keying.py`; eviction-leak test converted to start-then-install under `wf_{run_id}`; `temporal-integration.md` keying story updated.
-- [x] **0bis — cancellation test pinning the `finally`-ordering invariant** → FIXED, premise corrected: cancellation at an activity await surfaces as `ActivityError` (an `Exception`, swallowed by the best-effort except) — it CANNOT abort the finally. The guard instead forces eviction via worker shutdown: `test_wf_pipe_router_eviction_cleanup_ordering.py`, verified RED against a reordered finally.
-- [x] **1 — stale runner-fallback warning message** → FIXED. Renamed `log_once_runner_fallback_engaged`, INFO level, message states the post-H1 contract; module docstring + the two unit tests updated.
-- [x] **2 — dead `_event_log_contexts.clear()`** → FIXED. Deleted in `tracing/helpers.py` AND the same dead clear in `test_split_worker_real_inference_cost.py`; docstrings rewritten.
-- [x] **3 — test-scaffolding dedup** → FIXED (two sub-points DISMISSED with evidence in the follow-ups doc: the enable-tracing fixture has only one copy; the extract-pages scan needs `(name, activity_id)` pairs). Shared: `make_synthetic_usage_llm_job`, `act_flush_noop`, `scheduled_activity_names` in `tracing/helpers.py`; `make_prepared_greeting_job` in `library_crate/helpers.py`.
-- [x] **4 — `open_tracer` stale-eviction → delegate to `close_tracer`** → FIXED with the membership-gated warning.
-- [x] **5 — skip the guaranteed-empty flush activity in costs-only LIVE** → FIXED. Determinism argument re-derived (gate inputs payload-pure; costs-only LIVE buffer provably empty; Temporal unshipped). `schedule_flush` computed in setup, consumed in finally; costs-only test arm now asserts flush ABSENT + clean replay.
-- [x] **6 — trim redundant null conjuncts in the `finally`** → FIXED (folded into 5/9 rewrite; sentinels kept).
-- [x] **8 — document `set_event_log` overwrite as load-bearing** → FIXED (contract comment naming the three idioms).
-- [x] **9 — single-block `finally` simplification** → FIXED (sentinel + TYPE_CHECKING import gone; temporal suite green).
-- [x] **10 — shared scoped-library helper for the two uuid-keyed `runtime_bridge` copies** → FIXED. Sync `scoped_library_for_crate` in `runtime_bridge/primitives/scoped_library.py`; `wf_pipe_router` left bespoke.
-- [x] **11 — `LibraryManager.teardown`: forget the entry even if `library.teardown()` raises** → FIXED (`_pop_and_teardown_library`, RED-first unit tests).
-- [x] **7 — graph events lack an H1-style guard** (design note) → SURFACED (decisions doc §1, with options + recommendation; ties into T3).
+## Conventions the tests follow (read before flagging)
-## Constraints and gotchas
+- **One test class per module** (house pytest standard) — that's why e.g. doctor coverage is split across several `test_doctor_*.py` files. pytest-mock (`MockerFixture`) only, never `unittest.mock` imports; strong value asserts; parametrization over copy-paste.
+- **The CLI *interface* is deliberately untested here.** Arg parsing, `--help`, Typer wrappers, agent JSON shapes are owned by the sibling `conformance` repo's spec suite. This branch tests the `do_*`/`_core` functions beneath them. Low coverage remaining on `*_cmd.py` wrapper lines is by design.
+- **Module-namespace patching, source-module patching for call-time imports.** Collaborators imported at a module's top are patched at the consuming module's namespace; classes imported *inside* functions (the deferred-import pattern in worker factories) are patched at their source module. Worker-factory tests use a fresh real `PluginSdkRegistry()` per test so the booted singleton is never mutated.
+- **Real collaborators where cheap, mocks where not:** real TOML files in `tmp_path`, real PIL images, real pydantic SDK models (Mistral OCR, Portkey `GenericResponse`), a real aiohttp server for the codec; AsyncMocks for inference/network/Temporal SDK boundaries.
+- **Recorded Rich consoles** (`Console(record=True, color_system=None)` + `export_text()`) for output-rendering asserts.
+- **Remaining uncovered lines are deliberate:** `TYPE_CHECKING` blocks, interface-layer wrappers (conformance-owned), lines already pinned by the integration suite (e.g. pipeline runner happy path), and live-call worker paths.
-- **Do not modify** `tests/integration/pipelex/temporal/test_wf_pipe_router_eviction_library_leak.py` to make anything pass — it's the H2 regression guard. Extending it (item 0's successor-direction case) is fine.
-- Repo error-handling rules apply (`.claude/rules/python-standards.md`): no new bare `except Exception`, `try/finally` for required cleanup.
-- The `finally` block in `wf_pipe_router.py` is the determinism boundary: any edit there must keep ALL synchronous worker-local cleanup before the single flush await, and the flush gate payload-pure. Re-read the block's comments before touching it.
-- Items 5, 6, 9 (and 0 partially) all edit that same block — do them as one coherent change, not three diffs.
-- `worker_scopes` live only in base `pipelex.toml` — a recurring false-positive bot finding; don't "fix" it.
+## Known deferred items (not in this PR)
-## Acceptance
+[wip/tests/deferred-source-bugs-pinned-by-tests.md](wip/tests/deferred-source-bugs-pinned-by-tests.md) keeps two intentionally-deferred lists: a design tradeoff (LocalObserver's flat record namespace — nesting the payload is a breaking JSONL-shape change to decide deliberately) and test-quality cleanups (a Pipelex-boot opt-out conftest for pure test dirs, hoisting a few duplicated test scaffolds, minor in-file dedup). Don't treat their absence as gaps.
-- [x] Every checklist item above is checked with its outcome appended (**FIXED** / **DISMISSED (reason)** / **SURFACED → decisions doc**), and the follow-ups doc reflects the same status per item.
-- [x] `wip/distributed-execution/nondeterminism-follow-ups-decisions-needed.md` exists iff anything was surfaced, and the track `README.md` links it.
-- [x] Targeted suites green along the way (temporal integration + unit libraries/reporting/tracing/graph + runtime_bridge).
-- [x] `make agent-check` clean at the end.
-- [x] Full `make agent-test` green at the end.
-- [x] CHANGELOG `[Unreleased]` updated for any behavior change; `docs/under-the-hood/temporal-integration.md` updated if item 0 changes the keying story.
+## Verifying locally
-Update THIS file as you go (checkpoint style): check boxes and append outcomes as each item lands, so a further session can resume cold.
+```bash
+make agent-check # lint + typecheck gate
+make agent-test # full offline suite (the new build/ dirs collect thanks to the norecursedirs fix)
+```
diff --git a/pipelex/builder/bundle_spec.py b/pipelex/builder/bundle_spec.py
index 4dc565355..6cf218181 100644
--- a/pipelex/builder/bundle_spec.py
+++ b/pipelex/builder/bundle_spec.py
@@ -1,3 +1,5 @@
+from typing import TYPE_CHECKING
+
from pydantic import ConfigDict, Field, ValidationError, field_validator, model_validator
from rich.console import Group
from rich.table import Table
@@ -9,13 +11,15 @@ from pipelex.builder.exceptions import PipelexBundleSpecBlueprintError
from pipelex.builder.pipe.pipe_spec_union import PipeSpecUnion
from pipelex.core.bundles.pipe_sorter import sort_pipes_by_dependencies
from pipelex.core.bundles.pipelex_bundle_blueprint import PipeBlueprintUnion, PipelexBundleBlueprint
-from pipelex.core.concepts.concept_blueprint import ConceptBlueprint
from pipelex.core.domains.exceptions import DomainCodeError
from pipelex.core.domains.validation import validate_domain_code
from pipelex.core.stuffs.structured_content import StructuredContent
from pipelex.tools.misc.pretty import PrettyPrintable
from pipelex.tools.typing.pydantic_utils import format_pydantic_validation_error
+if TYPE_CHECKING:
+ from pipelex.core.concepts.concept_blueprint import ConceptBlueprint
+
class PipelexBundleSpec(StructuredContent):
"""Complete spec of a Pipelex bundle TOML definition.
@@ -32,8 +36,8 @@ class PipelexBundleSpec(StructuredContent):
unless overridden at the pipe level.
main_pipe: The main pipe of the bundle.
concept: Dictionary of concept definitions used in this domain. Keys are concept
- codes in PascalCase format, values are ConceptBlueprint instances or
- string references to existing concepts.
+ codes in PascalCase format, values are ConceptSpec instances or plain
+ strings used as the concept's description.
pipe: Dictionary of pipe definitions for data transformation. Keys are pipe
codes in snake_case format, values are specific pipe spec types
(PipeLLM, PipeImgGen, PipeSequence, etc.).
@@ -84,7 +88,8 @@ class PipelexBundleSpec(StructuredContent):
if isinstance(concept_spec_or_name, ConceptSpec):
concept[concept_code] = concept_spec_or_name.to_blueprint()
else:
- concept[concept_code] = ConceptBlueprint(description=concept_code, structure=concept_spec_or_name)
+ # A bare string is a concept description — the blueprint layer accepts it as-is
+ concept[concept_code] = concept_spec_or_name
pipe: dict[str, PipeBlueprintUnion] | None = None
if self.pipe:
@@ -142,7 +147,7 @@ class PipelexBundleSpec(StructuredContent):
concept_rendered = concept_spec_or_name.rendered_pretty()
concepts_table.add_row(concept_rendered)
else:
- # Simple string concept reference
+ # Plain string concept value: the string is the concept's description
concepts_table.add_row(Text.from_markup(f"[green]{concept_code}[/green]: {concept_spec_or_name}"))
bundle_group.renderables.append(concepts_table)
diff --git a/pipelex/builder/concept/concept_spec.py b/pipelex/builder/concept/concept_spec.py
index befb3ba23..bfe6419af 100644
--- a/pipelex/builder/concept/concept_spec.py
+++ b/pipelex/builder/concept/concept_spec.py
@@ -1,5 +1,5 @@
from datetime import datetime
-from typing import Any
+from typing import Any, cast
from pydantic import ConfigDict, Field, field_validator, model_validator
from rich.console import Group
@@ -311,14 +311,19 @@ class ConceptSpec(StructuredContent):
@model_validator(mode="before")
@classmethod
- def model_validate_spec(cls, values: dict[str, Any]) -> dict[str, Any]:
- if values.get("refines") and values.get("structure"):
+ def model_validate_spec(cls, values: Any) -> Any:
+ # mode="before" receives raw, unvalidated input — a non-dict (e.g. a plain string in a
+ # `ConceptSpec | str` union) must fall through untouched rather than crash on dict access
+ if not isinstance(values, dict):
+ return values
+ fields = cast("dict[str, Any]", values)
+ if fields.get("refines") and fields.get("structure"):
msg = (
- f"Forbidden to have refines and structure at the same time: `{values.get('refines')}` "
- f"and `{values.get('structure')}` for concept that has the description `{values.get('description')}`"
+ f"Forbidden to have refines and structure at the same time: `{fields.get('refines')}` "
+ f"and `{fields.get('structure')}` for concept that has the description `{fields.get('description')}`"
)
raise ConceptSpecError(msg)
- return values
+ return fields
def to_blueprint(self) -> ConceptBlueprint:
"""Convert this ConceptBlueprint to the original core ConceptBlueprint."""
diff --git a/pipelex/cogt/img_gen/img_gen_setting.py b/pipelex/cogt/img_gen/img_gen_setting.py
index f9989e008..c3ba2e3b2 100644
--- a/pipelex/cogt/img_gen/img_gen_setting.py
+++ b/pipelex/cogt/img_gen/img_gen_setting.py
@@ -17,7 +17,9 @@ class ImgGenSetting(ConfigModel):
quality: Quality | None = Field(default=None, strict=False)
nb_steps: int | None = Field(default=None, gt=0)
guidance_scale: float | None = Field(default=None, gt=0)
- is_moderated: bool = False
+ # None means "no explicit choice": workers omit the moderation/safety-checker param so the provider's
+ # own default applies (OpenAI gpt-image defaults to "auto", i.e. standard filtering)
+ is_moderated: bool | None = None
safety_tolerance: int | None = Field(default=None, ge=1, le=6)
description: str | None = None
diff --git a/pipelex/core/concepts/concept_blueprint.py b/pipelex/core/concepts/concept_blueprint.py
index 27a788bf9..71632d722 100644
--- a/pipelex/core/concepts/concept_blueprint.py
+++ b/pipelex/core/concepts/concept_blueprint.py
@@ -1,4 +1,4 @@
-from typing import Any
+from typing import Any, cast
from pydantic import BaseModel, ConfigDict, field_validator, model_validator
@@ -75,13 +75,18 @@ class ConceptBlueprint(BaseModel):
@model_validator(mode="before")
@classmethod
- def validate_mutually_exclusive_fields(cls, values: dict[str, Any] | str) -> dict[str, Any] | str:
+ def validate_mutually_exclusive_fields(cls, values: Any) -> Any:
"""Validate that refines and structure are not both set."""
- if isinstance(values, dict) and values.get("refines") and values.get("structure"):
+ # mode="before" receives raw, unvalidated input — a non-dict (e.g. a plain string in a
+ # `ConceptBlueprint | str` union) must fall through untouched rather than crash on dict access
+ if not isinstance(values, dict):
+ return values
+ fields = cast("dict[str, Any]", values)
+ if fields.get("refines") and fields.get("structure"):
msg = (
f"A concept cannot have both 'refines' and 'structure'. "
- f"Got refines='{values.get('refines')}' and structure='{values.get('structure')}'. "
- f"Concept description: '{values.get('description')}'"
+ f"Got refines='{fields.get('refines')}' and structure='{fields.get('structure')}'. "
+ f"Concept description: '{fields.get('description')}'"
)
raise ValueError(msg)
- return values
+ return fields
diff --git a/pipelex/core/pipes/output/output_renderer.py b/pipelex/core/pipes/output/output_renderer.py
index 5a00dca66..c133dfc83 100644
--- a/pipelex/core/pipes/output/output_renderer.py
+++ b/pipelex/core/pipes/output/output_renderer.py
@@ -41,7 +41,8 @@ def _collect_possible_outputs(
return []
possible_outputs: list[dict[str, Any]] = []
- for mapped_pipe_code in mapped_pipe_codes:
+ # pipe_dependencies() is a set — sort so the user-facing output_option_N numbering is deterministic
+ for mapped_pipe_code in sorted(mapped_pipe_codes):
mapped_pipe = get_required_pipe(pipe_code=mapped_pipe_code)
try:
output_dict = mapped_pipe.output.render_stuff_spec(output_format)
diff --git a/pipelex/observer/local_observer.py b/pipelex/observer/local_observer.py
index 0d40f6896..69cac749c 100644
--- a/pipelex/observer/local_observer.py
+++ b/pipelex/observer/local_observer.py
@@ -20,9 +20,10 @@ class LocalObserver(ObserverProtocol):
self.storage_dir.mkdir(parents=True, exist_ok=True)
def _write_to_jsonl(self, event_type: str, payload: PayloadType) -> None:
+ # event_type goes last so a payload key cannot overwrite the lifecycle event name
payload = {
- "event_type": event_type,
**payload,
+ "event_type": event_type,
}
file_path = self.storage_dir / f"{event_type}.jsonl"
diff --git a/pipelex/plugins/mistral/mistral_factory.py b/pipelex/plugins/mistral/mistral_factory.py
index 0cc44ed11..dc2c10e25 100644
--- a/pipelex/plugins/mistral/mistral_factory.py
+++ b/pipelex/plugins/mistral/mistral_factory.py
@@ -73,6 +73,9 @@ class MistralFactory:
async def make_simple_messages(self, llm_job: LLMJob) -> list[Messages]:
"""Makes a list of messages with a system message (if provided) and followed by a user message."""
messages: list[Messages] = []
+ if system_text := llm_job.llm_prompt.system_text:
+ messages.append(SystemMessage(content=system_text))
+
user_content: list[ContentChunk] = []
if user_text := llm_job.llm_prompt.user_text:
user_content.append(TextChunk(text=user_text))
@@ -85,9 +88,6 @@ class MistralFactory:
if user_content:
messages.append(UserMessage(content=user_content))
- if system_text := llm_job.llm_prompt.system_text:
- messages.append(SystemMessage(content=system_text))
-
return messages
async def make_mistral_image_url(self, prompt_image: PromptImage) -> ImageURLChunk:
diff --git a/pipelex/plugins/openai/openai_img_gen_factory.py b/pipelex/plugins/openai/openai_img_gen_factory.py
index f2b1dedec..b6e58c79f 100644
--- a/pipelex/plugins/openai/openai_img_gen_factory.py
+++ b/pipelex/plugins/openai/openai_img_gen_factory.py
@@ -119,11 +119,12 @@ class OpenAIImgGenFactory:
@classmethod
def moderation_for_openai_image(cls, is_moderated: bool | None) -> OpenAIImageModerationType:
+ """Map the is_moderated flag to OpenAI's moderation parameter: "auto" is standard filtering, "low" is less restrictive."""
if is_moderated is None:
return omit
if is_moderated:
- return "low"
- return "auto"
+ return "auto"
+ return "low"
@classmethod
def input_fidelity_for_openai_image(cls, input_fidelity: InputFidelity) -> OpenAIImageInputFidelityType:
diff --git a/pipelex/temporal/worker_cli.py b/pipelex/temporal/worker_cli.py
index 2debb304a..7326db74c 100644
--- a/pipelex/temporal/worker_cli.py
+++ b/pipelex/temporal/worker_cli.py
@@ -31,7 +31,7 @@ async def run_worker(
profile_name: str | None = None,
):
if project is None:
- log.info(f"Starting worker for current project '{project}', from {os.path.relpath(__file__)}")
+ log.info(f"Starting worker for current project, from {os.path.relpath(__file__)}")
else:
log.info(f"Starting worker for chosen project '{project}', from {os.path.relpath(__file__)}")
await get_task_manager().run_worker(
diff --git a/pipelex/tools/storage/storage_config.py b/pipelex/tools/storage/storage_config.py
index 9f3a20e28..98ddc61f5 100644
--- a/pipelex/tools/storage/storage_config.py
+++ b/pipelex/tools/storage/storage_config.py
@@ -1,6 +1,8 @@
-from typing import Literal
+from string import Formatter
+from typing import ClassVar, Literal
from pydantic import Field, model_validator
+from typing_extensions import override
from pipelex.system.configuration.config_model import ConfigModel
from pipelex.tools.storage.exceptions import StorageConfigError
@@ -15,19 +17,108 @@ class StorageMethod(StrEnum):
GCP = "gcp"
-class StorageLocalConfig(ConfigModel):
+class StorageMethodConfig(ConfigModel):
+ """Base for per-method storage configs: shared uri_format rules and operator-facing error reporting.
+
+ Subclasses declare `provider_label` and extend `_collect_error_msgs()` with their own checks.
+ """
+
+ # The exact keyword arguments GeneratedContentFactory._build_storage_key supplies to uri_format.format();
+ # any other placeholder would raise KeyError/IndexError at content-store time, so reject it at config time.
+ URI_FORMAT_SUPPORTED_FIELDS: ClassVar[frozenset[str]] = frozenset({"primary_id", "secondary_id", "hash", "extension"})
+
+ provider_label: ClassVar[str]
+
uri_format: str
+
+ def _collect_uri_format_error_msgs(self) -> list[str]:
+ if self.uri_format == "":
+ return ["- set a value for uri_format"]
+ try:
+ # Parse instead of substring-checking: the escaped literal "{{hash}}" renders as the constant
+ # "{hash}" (same URI for every object), and a format spec like "{hash:.0}" truncates the digest
+ # to a constant/colliding key. Only a plain {hash} replacement field guarantees unique keys.
+ parsed_fields = [
+ (field_name, format_spec, conversion)
+ for _, field_name, format_spec, conversion in Formatter().parse(self.uri_format)
+ if field_name is not None
+ ]
+ except ValueError as exc:
+ return [f"- uri_format is not a valid format string ({exc})"]
+ error_msgs: list[str] = []
+ unsupported_names: set[str] = set()
+ nonplain_names: set[str] = set()
+ hash_field_found = False
+ plain_hash_found = False
+ for field_name, format_spec, conversion in parsed_fields:
+ base_name = field_name.split(".")[0].split("[")[0]
+ if base_name not in self.URI_FORMAT_SUPPORTED_FIELDS:
+ unsupported_names.add(field_name)
+ continue
+ if base_name == "hash":
+ hash_field_found = True
+ if field_name == "hash" and not format_spec and not conversion:
+ plain_hash_found = True
+ elif field_name != base_name or format_spec or conversion:
+ # Every supported field is a str path component, so a format spec, conversion, or
+ # attribute/index access renders a constant, colliding, or oversized key ({primary_id:.0}
+ # truncates to "" → every object shares one URI; {primary_id:1000000000} would allocate a
+ # ~1GB string at render time) — the same silent-overwrite/OOM hazard we reject on {hash}.
+ nonplain_names.add(field_name)
+ supported_list = ", ".join("{" + name + "}" for name in sorted(self.URI_FORMAT_SUPPORTED_FIELDS))
+ for unsupported_name in sorted(unsupported_names):
+ error_msgs.append(f"- uri_format placeholder '{{{unsupported_name}}}' is not supported (supported: {supported_list})")
+ for nonplain_name in sorted(nonplain_names):
+ error_msgs.append(f"- uri_format placeholder '{{{nonplain_name}}}' must be plain: no format spec, conversion, or attribute/index access")
+ if not hash_field_found:
+ error_msgs.append("- uri_format must contain a {hash} placeholder")
+ elif not plain_hash_found:
+ error_msgs.append("- the {hash} placeholder must be plain: no format spec, conversion, or attribute/index access")
+ if not error_msgs:
+ # Defensive backstop: only plain {supported} placeholders reach here, so format() with the real
+ # keyword set is cheap and cannot allocate — it just proves nothing slipped past the parse checks.
+ try:
+ self.uri_format.format(**dict.fromkeys(self.URI_FORMAT_SUPPORTED_FIELDS, "test"))
+ except (KeyError, IndexError, ValueError, TypeError, AttributeError) as exc:
+ # The full set str.format raises — the backstop must never let a raw formatting error escape
+ error_msgs.append(f"- uri_format failed a test rendering ({exc})")
+ return error_msgs
+
+ def _collect_error_msgs(self) -> list[str]:
+ return self._collect_uri_format_error_msgs()
+
+ def lazy_validate(self) -> None:
+ error_msgs = self._collect_error_msgs()
+ if not error_msgs:
+ return
+ msg = f"You have enabled storage on {self.provider_label} so you need a proper {self.provider_label} config.\n\n"
+ msg += f"To fix your {self.provider_label} config:\n"
+ msg += "\n".join(error_msgs)
+ msg += f"\nThis can be done in the .pipelex/pipelex.toml file. More details can be found in the documentation: {URLs.documentation}"
+ raise StorageConfigError(msg)
+
+
+class StorageLocalConfig(StorageMethodConfig):
+ provider_label: ClassVar[str] = "local"
+
local_storage_path: str
+ @override
+ def _collect_error_msgs(self) -> list[str]:
+ error_msgs = super()._collect_error_msgs()
+ if self.local_storage_path == "":
+ error_msgs.append("- set a value for local_storage_path")
+ return error_msgs
-class StorageInMemoryConfig(ConfigModel):
- uri_format: str
+class StorageInMemoryConfig(StorageMethodConfig):
+ provider_label: ClassVar[str] = "in_memory"
+
+
+class StorageBucketConfig(StorageMethodConfig):
+ """Shared shape for bucket-based remote providers: bucket naming rules and signed-URL lifespan."""
-class StorageS3Config(ConfigModel):
- uri_format: str
bucket_name: str
- region: str
signed_urls_lifespan_seconds: int | Literal["disabled"]
@property
@@ -37,66 +128,44 @@ class StorageS3Config(ConfigModel):
return None
return self.signed_urls_lifespan_seconds
- def lazy_validate(self):
- error_msgs: list[str] = []
-
- if self.uri_format == "":
- error_msgs.append("- set a value for uri_format")
- elif "{hash}" not in self.uri_format:
- error_msgs.append("- uri_format must contain a {hash} placeholder")
-
+ @override
+ def _collect_error_msgs(self) -> list[str]:
+ error_msgs = super()._collect_error_msgs()
if self.bucket_name == "":
error_msgs.append("- set a value for bucket_name")
elif "." in self.bucket_name:
error_msgs.append("- bucket_name cannot contain a dot")
elif "/" in self.bucket_name:
error_msgs.append("- bucket_name cannot contain a slash")
+ if isinstance(self.signed_urls_lifespan_seconds, int) and self.signed_urls_lifespan_seconds <= 0:
+ error_msgs.append("- signed_urls_lifespan_seconds must be a positive number of seconds, or 'disabled'")
+ return error_msgs
- if self.region == "":
- error_msgs.append("- set a value for region")
- if error_msgs:
- msg = "You have enabled storage on S3 so you need a proper S3 config.\n\nTo fix you S3 config:\n"
- msg += "\n".join(error_msgs)
- msg += f"\nThis can be done in the .pipelex/pipelex.toml file. More details can be found in the documentation: {URLs.documentation}"
- raise StorageConfigError(msg)
+class StorageS3Config(StorageBucketConfig):
+ provider_label: ClassVar[str] = "S3"
+ region: str
-class StorageGcpConfig(ConfigModel):
- uri_format: str
- bucket_name: str
- project_id: str
- signed_urls_lifespan_seconds: int | Literal["disabled"]
-
- @property
- def signed_urls_lifespan(self) -> int | None:
- """Return signed URL lifespan in seconds, or None if disabled."""
- if self.signed_urls_lifespan_seconds == "disabled":
- return None
- return self.signed_urls_lifespan_seconds
+ @override
+ def _collect_error_msgs(self) -> list[str]:
+ error_msgs = super()._collect_error_msgs()
+ if self.region == "":
+ error_msgs.append("- set a value for region")
+ return error_msgs
- def lazy_validate(self):
- error_msgs: list[str] = []
- if self.uri_format == "":
- error_msgs.append("set a value for uri_format")
- elif "hash" not in self.uri_format:
- error_msgs.append("uri_format must contain a {hash} placeholder")
+class StorageGcpConfig(StorageBucketConfig):
+ provider_label: ClassVar[str] = "GCP"
- if self.bucket_name == "":
- error_msgs.append("set a value for bucket_name")
- elif "." in self.bucket_name:
- error_msgs.append("bucket_name cannot contain a dot")
- elif "/" in self.bucket_name:
- error_msgs.append("bucket_name cannot contain a slash")
+ project_id: str
+ @override
+ def _collect_error_msgs(self) -> list[str]:
+ error_msgs = super()._collect_error_msgs()
if self.project_id == "":
- error_msgs.append("set a value for project_id")
-
- if error_msgs:
- msg = "You have enabled storage on GCP so you need a proper GCP config:\n"
- msg += "\n".join(error_msgs)
- raise StorageConfigError(msg)
+ error_msgs.append("- set a value for project_id")
+ return error_msgs
class StorageProviderConfig(ConfigModel):
@@ -131,10 +200,15 @@ class StorageProviderConfig(ConfigModel):
@property
def storage_path(self) -> str:
- if not self.local:
- msg = "local config is required when method is local"
- raise StorageConfigError(msg)
- return self.local.local_storage_path
+ match self.method:
+ case StorageMethod.LOCAL:
+ if not self.local:
+ msg = "local config is required to access storage_path"
+ raise StorageConfigError(msg)
+ return self.local.local_storage_path
+ case StorageMethod.IN_MEMORY | StorageMethod.S3 | StorageMethod.GCP:
+ msg = f"storage_path is only available when method is local, but method is '{self.method}'"
+ raise StorageConfigError(msg)
@property
def uri_format(self) -> str:
diff --git a/pipelex/tools/storage/storage_provider_factory.py b/pipelex/tools/storage/storage_provider_factory.py
index 917d19c13..4fff5088f 100644
--- a/pipelex/tools/storage/storage_provider_factory.py
+++ b/pipelex/tools/storage/storage_provider_factory.py
@@ -21,13 +21,21 @@ def make_storage_provider_from_config(storage_provider_config: StorageProviderCo
A configured storage provider instance.
Raises:
- PipelexConfigError: If required provider-specific config is missing.
+ StorageConfigError: If the required provider-specific config is missing or fails its lazy_validate checks.
"""
match storage_provider_config.method:
case StorageMethod.LOCAL:
- log.verbose(f"Using local storage at: {storage_provider_config.storage_path}")
- return LocalStorageProvider(root_path=Path(storage_provider_config.storage_path))
+ if storage_provider_config.local is None:
+ msg = "local config is required when method is local"
+ raise StorageConfigError(msg)
+ storage_provider_config.local.lazy_validate()
+ log.verbose(f"Using local storage at: {storage_provider_config.local.local_storage_path}")
+ return LocalStorageProvider(root_path=Path(storage_provider_config.local.local_storage_path))
case StorageMethod.IN_MEMORY:
+ if storage_provider_config.in_memory is None:
+ msg = "in_memory config is required when method is in_memory"
+ raise StorageConfigError(msg)
+ storage_provider_config.in_memory.lazy_validate()
log.verbose("Using in-memory storage")
return InMemoryStorageProvider()
case StorageMethod.S3:
diff --git a/pyproject.toml b/pyproject.toml
index 4e3d3836e..5a3469ade 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -268,6 +268,21 @@ typeCheckingMode = "strict"
[tool.pytest]
minversion = "9.0"
+# All tests live under tests/ — this keeps bare `pytest` runs from scanning the whole repo (and from
+# ever collecting a stray root-level `build/` artifact tree, since `build` is removed from norecursedirs below).
+testpaths = ["tests"]
+# pytest's default norecursedirs includes `build`, which silently skipped every test under
+# tests/unit/pipelex/cli/commands/build/ (mirroring the source layout). Override = defaults minus `build`.
+norecursedirs = [
+ "*.egg",
+ ".*",
+ "_darcs",
+ "CVS",
+ "dist",
+ "node_modules",
+ "venv",
+ "{arch}",
+]
addopts = [
"--import-mode=importlib",
"-ra", # Show all test outcomes (including skips)
diff --git a/tests/unit/pipelex/builder/operations/test_inputs_ops.py b/tests/unit/pipelex/builder/operations/test_inputs_ops.py
new file mode 100644
index 000000000..a6f1a0ade
--- /dev/null
+++ b/tests/unit/pipelex/builder/operations/test_inputs_ops.py
@@ -0,0 +1,151 @@
+"""Unit tests for build_inputs_for_pipe in builder operations (inputs_ops)."""
+
+from __future__ import annotations
+
+import asyncio
+from pathlib import Path
+from types import SimpleNamespace
+from typing import TYPE_CHECKING, Any
+
+import pytest
+
+from pipelex.builder.operations.inputs_ops import build_inputs_for_pipe
+
+if TYPE_CHECKING:
+ from pytest_mock import MockerFixture
+
+MODULE = "pipelex.builder.operations.inputs_ops"
+
+
+class TestBuildInputsForPipe:
+ @pytest.fixture
+ def ops_mocks(self, mocker: MockerFixture) -> dict[str, Any]:
+ """Stub the library plumbing, bundle validation, pipe lookup and input rendering."""
+ library_manager = mocker.MagicMock()
+ library_manager.open_library.return_value = ("lib_id", mocker.MagicMock())
+ validate_result = SimpleNamespace(blueprints=[SimpleNamespace(main_pipe="bundle_main", domain="demo_domain")])
+ return {
+ "library_manager": library_manager,
+ "get_library_manager": mocker.patch(f"{MODULE}.get_library_manager", return_value=library_manager),
+ "set_current_library": mocker.patch(f"{MODULE}.set_current_library"),
+ "resolve_library_dirs": mocker.patch(f"{MODULE}.resolve_library_dirs", return_value=([], "defaults")),
+ "validate_bundle": mocker.patch(f"{MODULE}.validate_bundle", new=mocker.AsyncMock(return_value=validate_result)),
+ "get_required_pipe": mocker.patch(f"{MODULE}.get_required_pipe", return_value=SimpleNamespace(code="bundle_main")),
+ "render_inputs": mocker.patch(f"{MODULE}.render_inputs", return_value='{\n "topic": "your topic"\n}'),
+ }
+
+ def test_mthds_contents_first_main_pipe_wins_domain_qualified(self, ops_mocks: dict[str, Any]) -> None:
+ """Without a pipe code, the first blueprint declaring a main_pipe wins, domain-qualified."""
+ ops_mocks["validate_bundle"].return_value = SimpleNamespace(
+ blueprints=[
+ SimpleNamespace(main_pipe=None, domain="other_domain"),
+ SimpleNamespace(main_pipe="main", domain="domain"),
+ SimpleNamespace(main_pipe="later_main", domain="later_domain"),
+ ]
+ )
+
+ result = asyncio.run(build_inputs_for_pipe(mthds_contents=["mthds content"]))
+
+ ops_mocks["get_required_pipe"].assert_called_once_with(pipe_code="domain.main")
+ assert result["pipe_code"] == "domain.main"
+
+ def test_mthds_contents_validates_with_allow_signatures(self, ops_mocks: dict[str, Any]) -> None:
+ """The mthds_contents branch validates with allow_signatures=True — placeholders are tolerated on purpose."""
+ library_dirs = [Path("/some/lib")]
+
+ asyncio.run(build_inputs_for_pipe(mthds_contents=["mthds content"], library_dirs=library_dirs))
+
+ ops_mocks["validate_bundle"].assert_awaited_once_with(
+ mthds_contents=["mthds content"],
+ library_dirs=library_dirs,
+ allow_signatures=True,
+ )
+
+ def test_mthds_contents_without_any_main_pipe_raises(self, ops_mocks: dict[str, Any]) -> None:
+ """When no blueprint declares a main_pipe and no pipe code is given, a ValueError is raised."""
+ ops_mocks["validate_bundle"].return_value = SimpleNamespace(
+ blueprints=[
+ SimpleNamespace(main_pipe=None, domain="alpha_domain"),
+ SimpleNamespace(main_pipe=None, domain="beta_domain"),
+ ]
+ )
+
+ with pytest.raises(ValueError, match="Bundle does not declare a main_pipe"):
+ asyncio.run(build_inputs_for_pipe(mthds_contents=["mthds content"]))
+
+ def test_mthds_contents_explicit_pipe_code_skips_scan(self, ops_mocks: dict[str, Any]) -> None:
+ """An explicit pipe code is used verbatim, ignoring the blueprints' main_pipe declarations."""
+ result = asyncio.run(build_inputs_for_pipe(pipe_code="explicit_pipe", mthds_contents=["mthds content"]))
+
+ ops_mocks["get_required_pipe"].assert_called_once_with(pipe_code="explicit_pipe")
+ assert result["pipe_code"] == "explicit_pipe"
+
+ def test_bundle_path_validates_file_and_qualifies_main_pipe(self, ops_mocks: dict[str, Any], tmp_path: Path) -> None:
+ """The bundle_path branch validates by file path and domain-qualifies the bundle's main_pipe."""
+ bundle_path = tmp_path / "demo.mthds"
+
+ result = asyncio.run(build_inputs_for_pipe(bundle_path=bundle_path))
+
+ ops_mocks["validate_bundle"].assert_awaited_once_with(
+ mthds_file_path=bundle_path,
+ library_dirs=None,
+ allow_signatures=True,
+ )
+ ops_mocks["get_required_pipe"].assert_called_once_with(pipe_code="demo_domain.bundle_main")
+ assert result["pipe_code"] == "demo_domain.bundle_main"
+
+ def test_bundle_path_without_main_pipe_raises_with_path(self, ops_mocks: dict[str, Any], tmp_path: Path) -> None:
+ """A bundle file without a main_pipe raises a ValueError naming the bundle path."""
+ bundle_path = tmp_path / "demo.mthds"
+ ops_mocks["validate_bundle"].return_value = SimpleNamespace(blueprints=[SimpleNamespace(main_pipe=None, domain="demo_domain")])
+
+ with pytest.raises(ValueError, match="does not declare a main_pipe") as exc_info:
+ asyncio.run(build_inputs_for_pipe(bundle_path=bundle_path))
+
+ assert str(bundle_path) in str(exc_info.value)
+
+ def test_bundle_path_explicit_pipe_code_wins(self, ops_mocks: dict[str, Any], tmp_path: Path) -> None:
+ """With a bundle path, an explicit pipe code overrides the bundle's main_pipe."""
+ result = asyncio.run(build_inputs_for_pipe(pipe_code="explicit_pipe", bundle_path=tmp_path / "demo.mthds"))
+
+ ops_mocks["get_required_pipe"].assert_called_once_with(pipe_code="explicit_pipe")
+ assert result["pipe_code"] == "explicit_pipe"
+
+ def test_no_bundle_loads_libraries_from_resolved_dirs(self, ops_mocks: dict[str, Any]) -> None:
+ """Without a bundle, the library is opened, set current, and non-empty resolved dirs are loaded."""
+ resolved_dirs = [Path("/resolved/dir")]
+ ops_mocks["resolve_library_dirs"].return_value = (resolved_dirs, "explicit")
+
+ asyncio.run(build_inputs_for_pipe(pipe_code="my_pipe", library_dirs=resolved_dirs))
+
+ ops_mocks["library_manager"].open_library.assert_called_once_with()
+ ops_mocks["set_current_library"].assert_called_once_with(library_id="lib_id")
+ ops_mocks["resolve_library_dirs"].assert_called_once_with(resolved_dirs)
+ ops_mocks["library_manager"].load_libraries.assert_called_once_with(library_id="lib_id", library_dirs=resolved_dirs)
+ ops_mocks["validate_bundle"].assert_not_awaited()
+
+ def test_no_bundle_empty_resolved_dirs_skips_loading(self, ops_mocks: dict[str, Any]) -> None:
+ """Without a bundle, empty resolved dirs mean no library loading at all."""
+ asyncio.run(build_inputs_for_pipe(pipe_code="my_pipe"))
+
+ ops_mocks["library_manager"].load_libraries.assert_not_called()
+ ops_mocks["set_current_library"].assert_called_once_with(library_id="lib_id")
+
+ def test_no_bundle_and_no_pipe_code_raises(self, ops_mocks: dict[str, Any]) -> None:
+ """Neither a bundle nor a pipe code is an immediate error."""
+ with pytest.raises(ValueError, match="No pipe code specified"):
+ asyncio.run(build_inputs_for_pipe())
+
+ ops_mocks["get_required_pipe"].assert_not_called()
+
+ def test_happy_path_return_shape(self, ops_mocks: dict[str, Any]) -> None:
+ """The result carries success, the resolved pipe code, and the parsed inputs dict from render_inputs."""
+ result = asyncio.run(build_inputs_for_pipe(pipe_code="my_pipe"))
+
+ the_pipe = ops_mocks["get_required_pipe"].return_value
+ ops_mocks["render_inputs"].assert_called_once_with(the_pipe, indent=2)
+ assert result == {
+ "success": True,
+ "pipe_code": "my_pipe",
+ "inputs": {"topic": "your topic"},
+ }
diff --git a/tests/unit/pipelex/builder/test_bundle_spec_rendered_pretty.py b/tests/unit/pipelex/builder/test_bundle_spec_rendered_pretty.py
new file mode 100644
index 000000000..93a7f2cc2
--- /dev/null
+++ b/tests/unit/pipelex/builder/test_bundle_spec_rendered_pretty.py
@@ -0,0 +1,101 @@
+"""Unit tests for PipelexBundleSpec.rendered_pretty rich rendering."""
+
+from rich.console import Console
+
+from pipelex.builder.bundle_spec import PipelexBundleSpec
+from pipelex.builder.concept.concept_spec import ConceptSpec
+from pipelex.builder.pipe.pipe_llm_spec import PipeLLMSpec
+from pipelex.builder.pipe.pipe_sequence_spec import PipeSequenceSpec
+from pipelex.builder.pipe.sub_pipe_spec import SubPipeSpec
+
+
+def make_llm_spec(pipe_code: str) -> PipeLLMSpec:
+ return PipeLLMSpec(
+ pipe_code=pipe_code,
+ description=f"Generate text for {pipe_code}",
+ inputs={"topic": "Text"},
+ output="Text",
+ prompt="Write about $topic",
+ model="$writing-creative",
+ )
+
+
+def make_full_bundle_spec() -> PipelexBundleSpec:
+ return PipelexBundleSpec(
+ domain="test_domain",
+ description="A bundle that writes articles",
+ system_prompt="You are a concise writer",
+ main_pipe="main_seq",
+ concept={
+ "Article": ConceptSpec(concept_code="Article", description="A written article", refines="Text"),
+ "Summary": "A short summary of a document",
+ },
+ pipe={
+ "main_seq": PipeSequenceSpec(
+ pipe_code="main_seq",
+ description="Sequence main_seq",
+ inputs={"topic": "Text"},
+ output="Text",
+ steps=[SubPipeSpec(pipe_code="step_one", result="step_one_result")],
+ ),
+ "step_one": make_llm_spec("step_one"),
+ },
+ )
+
+
+def render_to_text(bundle_spec: PipelexBundleSpec, title: str | None = None) -> str:
+ console = Console(record=True, color_system=None, width=300)
+ console.print(bundle_spec.rendered_pretty(title=title))
+ return console.export_text()
+
+
+class TestPipelexBundleSpecRenderedPretty:
+ def test_full_render_with_title(self) -> None:
+ """A full bundle renders title, header info, system prompt and both tables."""
+ rendered = render_to_text(make_full_bundle_spec(), title="Bundle Overview")
+
+ assert "Bundle Overview" in rendered
+ assert "Domain: test_domain" in rendered
+ assert "Description: A bundle that writes articles" in rendered
+ assert "Main Pipe: main_seq" in rendered
+ assert "System Prompt: You are a concise writer" in rendered
+
+ def test_concepts_table_renders_spec_and_string_rows(self) -> None:
+ """The Concepts table shows a full ConceptSpec row and a string-reference row."""
+ rendered = render_to_text(make_full_bundle_spec())
+
+ assert "Concepts" in rendered
+ assert "Concept: Article" in rendered
+ assert "Refines: Text" in rendered
+ assert "Description: A written article" in rendered
+ assert "Summary: A short summary of a document" in rendered
+
+ def test_pipes_table_lists_each_pipe(self) -> None:
+ """The Pipes table lists every pipe with its type, inputs and output concept codes."""
+ rendered = render_to_text(make_full_bundle_spec())
+
+ assert "Pipes" in rendered
+ assert "Pipe: main_seq" in rendered
+ assert "Type: PipeSequence (PipeController)" in rendered
+ assert "Sequence Steps:" in rendered
+ assert "Pipe: step_one" in rendered
+ assert "Type: PipeLLM (PipeOperator)" in rendered
+ assert "Input: topic (Text)" in rendered
+ assert "Output: Text" in rendered
+
+ def test_minimal_render_omits_optional_sections(self) -> None:
+ """Without title, description and system_prompt, those sections are absent from the output."""
+ bundle_spec = PipelexBundleSpec(
+ domain="test_domain",
+ main_pipe="step_one",
+ pipe={"step_one": make_llm_spec("step_one")},
+ )
+
+ rendered = render_to_text(bundle_spec)
+
+ assert "Domain: test_domain" in rendered
+ assert "Main Pipe: step_one" in rendered
+ assert "Description: A bundle" not in rendered
+ assert "System Prompt:" not in rendered
+ assert "Concepts" not in rendered
+ assert "Pipes" in rendered
diff --git a/tests/unit/pipelex/builder/test_bundle_spec_to_blueprint.py b/tests/unit/pipelex/builder/test_bundle_spec_to_blueprint.py
new file mode 100644
index 000000000..1e299419f
--- /dev/null
+++ b/tests/unit/pipelex/builder/test_bundle_spec_to_blueprint.py
@@ -0,0 +1,178 @@
+"""Unit tests for PipelexBundleSpec.to_blueprint conversion: concepts, pipe ordering, and error wrapping."""
+
+from typing import TYPE_CHECKING, Any
+
+import pytest
+from pydantic import ValidationError
+
+from pipelex.builder.bundle_spec import PipelexBundleSpec
+from pipelex.builder.concept.concept_spec import ConceptSpec, ConceptStructureSpec, ConceptStructureSpecFieldType
+from pipelex.builder.exceptions import PipelexBundleSpecBlueprintError
+from pipelex.builder.pipe.pipe_llm_spec import PipeLLMSpec
+from pipelex.builder.pipe.pipe_sequence_spec import PipeSequenceSpec
+from pipelex.builder.pipe.sub_pipe_spec import SubPipeSpec
+from pipelex.core.concepts.concept_blueprint import ConceptBlueprint
+from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprint, ConceptStructureBlueprintFieldType
+
+if TYPE_CHECKING:
+ from pytest_mock import MockerFixture
+
+
+def make_llm_spec(pipe_code: str) -> PipeLLMSpec:
+ return PipeLLMSpec(
+ pipe_code=pipe_code,
+ description=f"Generate text for {pipe_code}",
+ inputs={"topic": "Text"},
+ output="Text",
+ prompt="Write about $topic",
+ model="$writing-creative",
+ )
+
+
+def make_sequence_spec(pipe_code: str, step_codes: list[str]) -> PipeSequenceSpec:
+ steps = [SubPipeSpec(pipe_code=step_code, result=f"{step_code}_result") for step_code in step_codes]
+ return PipeSequenceSpec(
+ pipe_code=pipe_code,
+ description=f"Sequence {pipe_code}",
+ inputs={"topic": "Text"},
+ output="Text",
+ steps=steps,
+ )
+
+
+def make_captured_validation_error() -> ValidationError:
+ """Capture a real pydantic ValidationError from an intentionally invalid model construction."""
+ try:
+ ConceptBlueprint.model_validate({})
+ except ValidationError as exc:
+ return exc
+ pytest.fail("ConceptBlueprint.model_validate({}) should have raised a ValidationError")
+
+
+class TestPipelexBundleSpecToBlueprint:
+ def test_concept_spec_value_converted_to_concept_blueprint(self) -> None:
+ """A ConceptSpec value converts to its own blueprint, preserving description, structure and refines."""
+ bundle_spec = PipelexBundleSpec(
+ domain="test_domain",
+ main_pipe="write_text",
+ concept={
+ "Article": ConceptSpec(concept_code="Article", description="A written article", refines="Text"),
+ "Person": ConceptSpec(
+ concept_code="Person",
+ description="A person",
+ structure={
+ "full_name": ConceptStructureSpec(
+ the_field_name="full_name",
+ description="Full name of the person",
+ type=ConceptStructureSpecFieldType.TEXT,
+ required=True,
+ ),
+ },
+ ),
+ },
+ pipe={"write_text": make_llm_spec("write_text")},
+ )
+
+ blueprint = bundle_spec.to_blueprint()
+
+ assert blueprint.concept is not None
+ assert blueprint.concept["Article"] == ConceptBlueprint(description="A written article", structure=None, refines="Text")
+ expected_field = ConceptStructureBlueprint(
+ description="Full name of the person",
+ type=ConceptStructureBlueprintFieldType.TEXT,
+ required=True,
+ )
+ assert blueprint.concept["Person"] == ConceptBlueprint(description="A person", structure={"full_name": expected_field}, refines=None)
+
+ def test_string_concept_value_passes_through_as_description(self) -> None:
+ """A string concept value passes through to the blueprint dict unchanged, where it means the concept's description."""
+ bundle_spec = PipelexBundleSpec(
+ domain="test_domain",
+ main_pipe="write_text",
+ concept={"Summary": "A short summary of a document"},
+ pipe={"write_text": make_llm_spec("write_text")},
+ )
+
+ blueprint = bundle_spec.to_blueprint()
+
+ assert blueprint.concept is not None
+ assert blueprint.concept["Summary"] == "A short summary of a document"
+
+ def test_string_concept_value_survives_normal_validation(self) -> None:
+ """model_validate on a dict payload keeps a string concept value as a plain str via the `ConceptSpec | str` union."""
+ bundle_spec = PipelexBundleSpec.model_validate(
+ {
+ "domain": "test_domain",
+ "main_pipe": "write_text",
+ "concept": {"Summary": "A short summary of a document"},
+ "pipe": {"write_text": make_llm_spec("write_text").model_dump()},
+ }
+ )
+
+ assert bundle_spec.concept is not None
+ assert bundle_spec.concept["Summary"] == "A short summary of a document"
+
+ @pytest.mark.parametrize("empty_concepts", [None, {}])
+ def test_no_concepts_yields_none_concept_on_blueprint(self, empty_concepts: dict[str, Any] | None) -> None:
+ """Concept set to None or left empty produces a blueprint with concept=None."""
+ bundle_spec = PipelexBundleSpec(
+ domain="test_domain",
+ main_pipe="write_text",
+ concept=empty_concepts,
+ pipe={"write_text": make_llm_spec("write_text")},
+ )
+
+ blueprint = bundle_spec.to_blueprint()
+
+ assert blueprint.concept is None
+
+ def test_pipes_sorted_controller_first_then_step_order(self) -> None:
+ """Pipes fed in scrambled dict order come out controller-first, then in step order, with header fields preserved."""
+ bundle_spec = PipelexBundleSpec(
+ domain="test_domain",
+ description="A sequenced bundle",
+ system_prompt="Be precise",
+ main_pipe="main_seq",
+ pipe={
+ "step_two": make_llm_spec("step_two"),
+ "step_one": make_llm_spec("step_one"),
+ "main_seq": make_sequence_spec("main_seq", step_codes=["step_one", "step_two"]),
+ },
+ )
+
+ blueprint = bundle_spec.to_blueprint()
+
+ assert blueprint.domain == "test_domain"
+ assert blueprint.description == "A sequenced bundle"
+ assert blueprint.system_prompt == "Be precise"
+ assert blueprint.main_pipe == "main_seq"
+ assert blueprint.pipe is not None
+ assert list(blueprint.pipe.keys()) == ["main_seq", "step_one", "step_two"]
+ assert blueprint.pipe["main_seq"].type == "PipeSequence"
+ assert blueprint.pipe["step_one"].type == "PipeLLM"
+ assert blueprint.pipe["step_two"].type == "PipeLLM"
+
+ def test_pipe_spec_validation_error_wrapped_with_pipe_code(self, mocker: "MockerFixture") -> None:
+ """A ValidationError from a pipe spec's to_blueprint is wrapped, naming the failing pipe code."""
+ captured_error = make_captured_validation_error()
+ mocker.patch.object(PipeLLMSpec, "to_blueprint", side_effect=captured_error)
+ bundle_spec = PipelexBundleSpec(
+ domain="test_domain",
+ main_pipe="write_text",
+ pipe={"write_text": make_llm_spec("write_text")},
+ )
+
+ with pytest.raises(PipelexBundleSpecBlueprintError, match="Failed to create pipe blueprint from spec for pipe code write_text"):
+ bundle_spec.to_blueprint()
+
+ def test_bundle_blueprint_construction_failure_wrapped(self) -> None:
+ """A spec that yields an invalid bundle blueprint (native concept shadowing) raises the wrapped error."""
+ bundle_spec = PipelexBundleSpec(
+ domain="test_domain",
+ main_pipe="write_text",
+ concept={"Text": "Shadowing a native concept"},
+ pipe={"write_text": make_llm_spec("write_text")},
+ )
+
+ with pytest.raises(PipelexBundleSpecBlueprintError, match="Failed to create pipelex bundle blueprint"):
+ bundle_spec.to_blueprint()
diff --git a/tests/unit/pipelex/builder/test_bundle_spec_validation.py b/tests/unit/pipelex/builder/test_bundle_spec_validation.py
new file mode 100644
index 000000000..fe1095799
--- /dev/null
+++ b/tests/unit/pipelex/builder/test_bundle_spec_validation.py
@@ -0,0 +1,58 @@
+"""Unit tests for PipelexBundleSpec pydantic validation rules."""
+
+from typing import Any
+
+import pytest
+from pydantic import ValidationError
+
+from pipelex.builder.bundle_spec import PipelexBundleSpec
+from pipelex.builder.pipe.pipe_llm_spec import PipeLLMSpec
+
+
+def make_llm_spec(pipe_code: str = "write_text") -> PipeLLMSpec:
+ return PipeLLMSpec(
+ pipe_code=pipe_code,
+ description=f"Generate text for {pipe_code}",
+ inputs={"topic": "Text"},
+ output="Text",
+ prompt="Write about $topic",
+ model="$writing-creative",
+ )
+
+
+class TestPipelexBundleSpecValidation:
+ @pytest.mark.parametrize("bad_domain", ["Bad Domain", "UpperCase", "has-dashes", "1starts_with_digit"])
+ def test_invalid_domain_code_rejected(self, bad_domain: str) -> None:
+ """An invalid domain code is rejected with an explicit error message."""
+ with pytest.raises(ValidationError, match="not a valid domain code"):
+ PipelexBundleSpec(domain=bad_domain, main_pipe="write_text", pipe={"write_text": make_llm_spec()})
+
+ def test_main_pipe_absent_from_pipe_dict_rejected(self) -> None:
+ """A main_pipe that is not a key of the pipe dict is rejected."""
+ with pytest.raises(ValidationError, match="Main pipe 'missing_pipe' could not be found in bundle spec"):
+ PipelexBundleSpec(domain="test_domain", main_pipe="missing_pipe", pipe={"write_text": make_llm_spec()})
+
+ @pytest.mark.parametrize("empty_pipes", [{}, None])
+ def test_main_pipe_with_no_pipes_rejected(self, empty_pipes: dict[str, Any] | None) -> None:
+ """A main_pipe declared while the pipe dict is empty or None is rejected."""
+ with pytest.raises(ValidationError, match="Main pipe 'write_text' could not be found in bundle spec"):
+ PipelexBundleSpec(domain="test_domain", main_pipe="write_text", pipe=empty_pipes)
+
+ def test_valid_spec_constructs(self) -> None:
+ """A well-formed spec constructs and exposes its declared fields."""
+ bundle_spec = PipelexBundleSpec(
+ domain="test_domain",
+ description="A test bundle",
+ system_prompt="You are concise",
+ main_pipe="write_text",
+ pipe={"write_text": make_llm_spec()},
+ )
+
+ assert bundle_spec.domain == "test_domain"
+ assert bundle_spec.description == "A test bundle"
+ assert bundle_spec.system_prompt == "You are concise"
+ assert bundle_spec.main_pipe == "write_text"
+ assert bundle_spec.pipe is not None
+ assert list(bundle_spec.pipe.keys()) == ["write_text"]
+ assert bundle_spec.pipe["write_text"].description == "Generate text for write_text"
+ assert bundle_spec.concept == {}
diff --git a/tests/unit/pipelex/cli/commands/build/test_inputs_core.py b/tests/unit/pipelex/cli/commands/build/test_inputs_core.py
new file mode 100644
index 000000000..7f9a72313
--- /dev/null
+++ b/tests/unit/pipelex/cli/commands/build/test_inputs_core.py
@@ -0,0 +1,109 @@
+"""Unit tests for the `pipelex build inputs` core logic (_generate_inputs_core)."""
+
+from __future__ import annotations
+
+import asyncio
+from types import SimpleNamespace
+from typing import TYPE_CHECKING, Any
+
+import pytest
+import typer
+
+from pipelex.cli.commands.build.inputs._inputs_core import _generate_inputs_core # noqa: PLC2701 # pyright: ignore[reportPrivateUsage]
+from pipelex.core.pipes.inputs.exceptions import NoInputsRequiredError
+from pipelex.pipeline.exceptions import ValidateBundleError
+
+if TYPE_CHECKING:
+ from pathlib import Path
+
+ from pytest_mock import MockerFixture
+
+MODULE = "pipelex.cli.commands.build.inputs._inputs_core"
+
+
+class TestGenerateInputsCore:
+ @pytest.fixture
+ def core_mocks(self, mocker: MockerFixture) -> dict[str, Any]:
+ """Stub the library plumbing, bundle validation, pipe lookup and input rendering."""
+ library_manager = mocker.MagicMock()
+ library_manager.open_library.return_value = ("lib_id", mocker.MagicMock())
+ mocker.patch(f"{MODULE}.get_library_manager", return_value=library_manager)
+ mocker.patch(f"{MODULE}.set_current_library")
+ mocker.patch(f"{MODULE}.resolve_library_dirs", return_value=([], "defaults"))
+ validate_result = SimpleNamespace(blueprints=[SimpleNamespace(main_pipe="bundle_main")])
+ return {
+ "validate_bundle": mocker.patch(f"{MODULE}.validate_bundle", new=mocker.AsyncMock(return_value=validate_result)),
+ "get_required_pipe": mocker.patch(f"{MODULE}.get_required_pipe", return_value=SimpleNamespace(code="bundle_main")),
+ "render_inputs": mocker.patch(f"{MODULE}.render_inputs", return_value='{\n "topic": "your topic"\n}'),
+ }
+
+ def test_bundle_main_pipe_writes_inputs_next_to_bundle(self, core_mocks: dict[str, Any], tmp_path: Path) -> None:
+ """Without --pipe, the bundle's main_pipe is used and inputs.json lands next to the bundle."""
+ bundle_path = tmp_path / "demo.mthds"
+
+ asyncio.run(_generate_inputs_core(pipe_code=None, bundle_path=bundle_path))
+
+ core_mocks["get_required_pipe"].assert_called_once_with(pipe_code="bundle_main")
+ inputs_file = tmp_path / "inputs.json"
+ assert inputs_file.read_text(encoding="utf-8") == '{\n "topic": "your topic"\n}'
+
+ def test_explicit_pipe_code_wins_over_main_pipe(self, core_mocks: dict[str, Any], tmp_path: Path) -> None:
+ """With --pipe, the explicit pipe code is used even when the bundle has a main_pipe."""
+ bundle_path = tmp_path / "demo.mthds"
+
+ asyncio.run(_generate_inputs_core(pipe_code="my_pipe", bundle_path=bundle_path))
+
+ core_mocks["get_required_pipe"].assert_called_once_with(pipe_code="my_pipe")
+
+ def test_bundle_without_main_pipe_exits(self, core_mocks: dict[str, Any], tmp_path: Path) -> None:
+ """A bundle without main_pipe and no --pipe is an error."""
+ core_mocks["validate_bundle"].return_value = SimpleNamespace(blueprints=[SimpleNamespace(main_pipe=None)])
+
+ with pytest.raises(typer.Exit) as exc_info:
+ asyncio.run(_generate_inputs_core(pipe_code=None, bundle_path=tmp_path / "demo.mthds"))
+
+ assert exc_info.value.exit_code == 1
+
+ @pytest.mark.usefixtures("core_mocks")
+ def test_no_bundle_and_no_pipe_exits(self) -> None:
+ """Neither a bundle nor a pipe code is an immediate error."""
+ with pytest.raises(typer.Exit) as exc_info:
+ asyncio.run(_generate_inputs_core(pipe_code=None, bundle_path=None))
+
+ assert exc_info.value.exit_code == 1
+
+ def test_validate_bundle_error_exits(self, core_mocks: dict[str, Any], tmp_path: Path) -> None:
+ """A bundle validation failure exits 1."""
+ core_mocks["validate_bundle"].side_effect = ValidateBundleError("broken bundle")
+
+ with pytest.raises(typer.Exit) as exc_info:
+ asyncio.run(_generate_inputs_core(pipe_code=None, bundle_path=tmp_path / "demo.mthds"))
+
+ assert exc_info.value.exit_code == 1
+
+ def test_no_inputs_required_exits_zero(self, core_mocks: dict[str, Any], tmp_path: Path) -> None:
+ """A pipe needing no inputs exits 0 with a friendly message, not an error."""
+ core_mocks["render_inputs"].side_effect = NoInputsRequiredError("pipe has no inputs")
+
+ with pytest.raises(typer.Exit) as exc_info:
+ asyncio.run(_generate_inputs_core(pipe_code="my_pipe", bundle_path=tmp_path / "demo.mthds"))
+
+ assert exc_info.value.exit_code == 0
+
+ @pytest.mark.usefixtures("core_mocks")
+ def test_explicit_output_path_wins(self, tmp_path: Path) -> None:
+ """An explicit output path is used verbatim, parents created."""
+ target_path = tmp_path / "deep" / "custom_inputs.json"
+
+ asyncio.run(_generate_inputs_core(pipe_code="my_pipe", bundle_path=tmp_path / "demo.mthds", output_path=target_path))
+
+ assert target_path.read_text(encoding="utf-8") == '{\n "topic": "your topic"\n}'
+
+ @pytest.mark.usefixtures("core_mocks")
+ def test_no_bundle_defaults_to_results_dir(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ """With only a pipe code, the file goes to results/inputs.json under the cwd."""
+ monkeypatch.chdir(tmp_path)
+
+ asyncio.run(_generate_inputs_core(pipe_code="my_pipe", bundle_path=None))
+
+ assert (tmp_path / "results" / "inputs.json").exists()
diff --git a/tests/unit/pipelex/cli/commands/build/test_output_core.py b/tests/unit/pipelex/cli/commands/build/test_output_core.py
new file mode 100644
index 000000000..7a2291cc6
--- /dev/null
+++ b/tests/unit/pipelex/cli/commands/build/test_output_core.py
@@ -0,0 +1,120 @@
+"""Unit tests for the `pipelex build output` core logic (_generate_output_core)."""
+
+from __future__ import annotations
+
+import asyncio
+from types import SimpleNamespace
+from typing import TYPE_CHECKING, Any
+
+import pytest
+import typer
+
+from pipelex.cli.commands.build.output._output_core import _generate_output_core # noqa: PLC2701 # pyright: ignore[reportPrivateUsage]
+from pipelex.core.concepts.concept_representation_generator import ConceptRepresentationFormat
+from pipelex.pipeline.exceptions import ValidateBundleError
+
+if TYPE_CHECKING:
+ from pathlib import Path
+
+ from pytest_mock import MockerFixture
+
+MODULE = "pipelex.cli.commands.build.output._output_core"
+
+
+class TestGenerateOutputCore:
+ @pytest.fixture
+ def core_mocks(self, mocker: MockerFixture) -> dict[str, Any]:
+ """Stub the library plumbing, bundle validation, pipe lookup and output rendering."""
+ library_manager = mocker.MagicMock()
+ library_manager.open_library.return_value = ("lib_id", mocker.MagicMock())
+ mocker.patch(f"{MODULE}.get_library_manager", return_value=library_manager)
+ mocker.patch(f"{MODULE}.set_current_library")
+ mocker.patch(f"{MODULE}.resolve_library_dirs", return_value=([], "defaults"))
+ validate_result = SimpleNamespace(blueprints=[SimpleNamespace(main_pipe="bundle_main")])
+ return {
+ "validate_bundle": mocker.patch(f"{MODULE}.validate_bundle", new=mocker.AsyncMock(return_value=validate_result)),
+ "get_required_pipe": mocker.patch(f"{MODULE}.get_required_pipe", return_value=SimpleNamespace(code="bundle_main")),
+ "render_output": mocker.patch(f"{MODULE}.render_output", return_value='{"answer": 42}'),
+ }
+
+ def test_bundle_main_pipe_writes_json_next_to_bundle(self, core_mocks: dict[str, Any], tmp_path: Path) -> None:
+ """Without --pipe, the bundle's main_pipe is used and output.json lands next to the bundle."""
+ asyncio.run(_generate_output_core(pipe_code=None, bundle_path=tmp_path / "demo.mthds"))
+
+ core_mocks["get_required_pipe"].assert_called_once_with(pipe_code="bundle_main")
+ assert (tmp_path / "output.json").read_text(encoding="utf-8") == '{"answer": 42}'
+
+ @pytest.mark.usefixtures("core_mocks")
+ def test_python_format_defaults_to_output_py(self, tmp_path: Path) -> None:
+ """The PYTHON format defaults to output.py next to the bundle."""
+ asyncio.run(
+ _generate_output_core(
+ pipe_code="my_pipe",
+ bundle_path=tmp_path / "demo.mthds",
+ output_format=ConceptRepresentationFormat.PYTHON,
+ )
+ )
+
+ assert (tmp_path / "output.py").exists()
+
+ @pytest.mark.usefixtures("core_mocks")
+ def test_schema_format_defaults_to_results_dir_without_bundle(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ """Without a bundle, the SCHEMA format goes to results/output_schema.json."""
+ monkeypatch.chdir(tmp_path)
+
+ asyncio.run(_generate_output_core(pipe_code="my_pipe", output_format=ConceptRepresentationFormat.SCHEMA))
+
+ assert (tmp_path / "results" / "output_schema.json").exists()
+
+ @pytest.mark.usefixtures("core_mocks")
+ def test_explicit_output_path_wins(self, tmp_path: Path) -> None:
+ """An explicit output path is used verbatim."""
+ target_path = tmp_path / "deep" / "custom_output.json"
+
+ asyncio.run(_generate_output_core(pipe_code="my_pipe", bundle_path=tmp_path / "demo.mthds", output_path=target_path))
+
+ assert target_path.read_text(encoding="utf-8") == '{"answer": 42}'
+
+ def test_bundle_without_main_pipe_exits(self, core_mocks: dict[str, Any], tmp_path: Path) -> None:
+ """A bundle without main_pipe and no --pipe is an error."""
+ core_mocks["validate_bundle"].return_value = SimpleNamespace(blueprints=[SimpleNamespace(main_pipe=None)])
+
+ with pytest.raises(typer.Exit) as exc_info:
+ asyncio.run(_generate_output_core(pipe_code=None, bundle_path=tmp_path / "demo.mthds"))
+
+ assert exc_info.value.exit_code == 1
+
+ @pytest.mark.usefixtures("core_mocks")
+ def test_no_bundle_and_no_pipe_exits(self) -> None:
+ """Neither a bundle nor a pipe code is an immediate error."""
+ with pytest.raises(typer.Exit) as exc_info:
+ asyncio.run(_generate_output_core(pipe_code=None, bundle_path=None))
+
+ assert exc_info.value.exit_code == 1
+
+ def test_validate_bundle_error_exits(self, core_mocks: dict[str, Any], tmp_path: Path) -> None:
+ """A bundle validation failure exits 1."""
+ core_mocks["validate_bundle"].side_effect = ValidateBundleError("broken bundle")
+
+ with pytest.raises(typer.Exit) as exc_info:
+ asyncio.run(_generate_output_core(pipe_code=None, bundle_path=tmp_path / "demo.mthds"))
+
+ assert exc_info.value.exit_code == 1
+
+ def test_render_value_error_exits_zero(self, core_mocks: dict[str, Any], tmp_path: Path) -> None:
+ """A ValueError from rendering (e.g. nothing to represent) exits 0, not 1."""
+ core_mocks["render_output"].side_effect = ValueError("nothing to render")
+
+ with pytest.raises(typer.Exit) as exc_info:
+ asyncio.run(_generate_output_core(pipe_code="my_pipe", bundle_path=tmp_path / "demo.mthds"))
+
+ assert exc_info.value.exit_code == 0
+
+ def test_render_unexpected_error_exits_one(self, core_mocks: dict[str, Any], tmp_path: Path) -> None:
+ """An unexpected rendering failure exits 1."""
+ core_mocks["render_output"].side_effect = RuntimeError("render exploded")
+
+ with pytest.raises(typer.Exit) as exc_info:
+ asyncio.run(_generate_output_core(pipe_code="my_pipe", bundle_path=tmp_path / "demo.mthds"))
+
+ assert exc_info.value.exit_code == 1
diff --git a/tests/unit/pipelex/cli/commands/build/test_runner_core.py b/tests/unit/pipelex/cli/commands/build/test_runner_core.py
new file mode 100644
index 000000000..b3136c0ba
--- /dev/null
+++ b/tests/unit/pipelex/cli/commands/build/test_runner_core.py
@@ -0,0 +1,124 @@
+"""Unit tests for the `pipelex build runner` core logic (_prepare_runner_core).
+
+The registry getters are mocked at the module namespace so the real class/func
+registries of the module-scoped Pipelex are never torn down by these tests.
+"""
+
+from __future__ import annotations
+
+import asyncio
+from types import SimpleNamespace
+from typing import TYPE_CHECKING, Any
+
+import pytest
+import typer
+
+from pipelex.cli.commands.build.runner._runner_core import _prepare_runner_core # noqa: PLC2701 # pyright: ignore[reportPrivateUsage]
+from pipelex.pipeline.exceptions import ValidateBundleError
+
+if TYPE_CHECKING:
+ from pathlib import Path
+
+ from pytest_mock import MockerFixture
+
+MODULE = "pipelex.cli.commands.build.runner._runner_core"
+
+
+class TestPrepareRunnerCore:
+ @pytest.fixture
+ def core_mocks(self, mocker: MockerFixture) -> dict[str, Any]:
+ """Stub library plumbing, registries, structure generation and runner codegen."""
+ library_manager = mocker.MagicMock()
+ library_manager.open_library.return_value = ("lib_id", mocker.MagicMock())
+ mocker.patch(f"{MODULE}.get_library_manager", return_value=library_manager)
+ mocker.patch(f"{MODULE}.set_current_library")
+ mocker.patch(f"{MODULE}.resolve_library_dirs", return_value=([], "defaults"))
+ mocker.patch(f"{MODULE}.get_class_registry", return_value=mocker.MagicMock())
+ mocker.patch(f"{MODULE}.get_func_registry", return_value=mocker.MagicMock())
+ mocker.patch(f"{MODULE}.ClassRegistryUtils")
+ blueprint = SimpleNamespace(main_pipe="bundle_main", pipe=None)
+ validate_result = SimpleNamespace(blueprints=[blueprint])
+ return {
+ "validate_bundle": mocker.patch(f"{MODULE}.validate_bundle", new=mocker.AsyncMock(return_value=validate_result)),
+ "get_required_pipe": mocker.patch(f"{MODULE}.get_required_pipe", return_value=SimpleNamespace(code="bundle_main")),
+ "generate_structures": mocker.patch(f"{MODULE}.generate_structures_from_blueprints", return_value=[]),
+ "generate_runner_code": mocker.patch(f"{MODULE}.generate_runner_code", return_value="# runner code\n"),
+ }
+
+ def test_bundle_main_pipe_writes_runner_next_to_bundle(self, core_mocks: dict[str, Any], tmp_path: Path) -> None:
+ """Without --pipe, the bundle's main_pipe names the default run_<pipe>.py file."""
+ asyncio.run(_prepare_runner_core(pipe_code=None, bundle_path=tmp_path / "demo.mthds"))
+
+ core_mocks["get_required_pipe"].assert_called_once_with(pipe_code="bundle_main")
+ runner_file = tmp_path / "run_bundle_main.py"
+ assert runner_file.read_text(encoding="utf-8") == "# runner code\n"
+ codegen_kwargs = core_mocks["generate_runner_code"].call_args.kwargs
+ assert codegen_kwargs["output_multiplicity"] is False
+ assert codegen_kwargs["library_dir"] == str(tmp_path.resolve())
+
+ def test_structures_generated_into_output_dir(self, core_mocks: dict[str, Any], tmp_path: Path) -> None:
+ """Structure files are generated into a structures/ dir next to the runner."""
+ core_mocks["generate_structures"].return_value = [("demo", "Invoice")]
+
+ asyncio.run(_prepare_runner_core(pipe_code=None, bundle_path=tmp_path / "demo.mthds"))
+
+ structures_kwargs = core_mocks["generate_structures"].call_args.kwargs
+ assert structures_kwargs["output_directory"] == tmp_path / "structures"
+ assert structures_kwargs["target_path"] == tmp_path
+
+ @pytest.mark.usefixtures("core_mocks")
+ def test_explicit_output_path_wins(self, tmp_path: Path) -> None:
+ """An explicit output path is used verbatim for the runner file."""
+ target_path = tmp_path / "generated" / "my_runner.py"
+
+ asyncio.run(_prepare_runner_core(pipe_code="my_pipe", bundle_path=tmp_path / "demo.mthds", output_path=target_path))
+
+ assert target_path.read_text(encoding="utf-8") == "# runner code\n"
+
+ def test_output_multiplicity_detected_from_pipe_blueprint(self, core_mocks: dict[str, Any], tmp_path: Path) -> None:
+ """A pipe whose output declares a multiplicity generates a list-aware runner."""
+ blueprint = SimpleNamespace(
+ main_pipe="bundle_main",
+ pipe={"bundle_main": SimpleNamespace(output="Invoice[]")},
+ )
+ core_mocks["validate_bundle"].return_value = SimpleNamespace(blueprints=[blueprint])
+
+ asyncio.run(_prepare_runner_core(pipe_code=None, bundle_path=tmp_path / "demo.mthds"))
+
+ codegen_kwargs = core_mocks["generate_runner_code"].call_args.kwargs
+ assert codegen_kwargs["output_multiplicity"] is True
+
+ @pytest.mark.usefixtures("core_mocks")
+ def test_no_bundle_exits(self) -> None:
+ """The runner build requires a bundle file."""
+ with pytest.raises(typer.Exit) as exc_info:
+ asyncio.run(_prepare_runner_core(pipe_code="my_pipe", bundle_path=None))
+
+ assert exc_info.value.exit_code == 1
+
+ def test_bundle_without_main_pipe_exits(self, core_mocks: dict[str, Any], tmp_path: Path) -> None:
+ """A bundle without main_pipe and no --pipe is an error."""
+ core_mocks["validate_bundle"].return_value = SimpleNamespace(blueprints=[SimpleNamespace(main_pipe=None, pipe=None)])
+
+ with pytest.raises(typer.Exit) as exc_info:
+ asyncio.run(_prepare_runner_core(pipe_code=None, bundle_path=tmp_path / "demo.mthds"))
+
+ assert exc_info.value.exit_code == 1
+
+ def test_validate_bundle_error_exits(self, core_mocks: dict[str, Any], tmp_path: Path) -> None:
+ """A bundle validation failure exits 1."""
+ core_mocks["validate_bundle"].side_effect = ValidateBundleError("broken bundle")
+
+ with pytest.raises(typer.Exit) as exc_info:
+ asyncio.run(_prepare_runner_core(pipe_code=None, bundle_path=tmp_path / "demo.mthds"))
+
+ assert exc_info.value.exit_code == 1
+
+ def test_runner_codegen_failure_exits(self, core_mocks: dict[str, Any], tmp_path: Path) -> None:
+ """A failure while generating runner code exits 1."""
+ core_mocks["generate_runner_code"].side_effect = RuntimeError("codegen exploded")
+
+ with pytest.raises(typer.Exit) as exc_info:
+ asyncio.run(_prepare_runner_core(pipe_code=None, bundle_path=tmp_path / "demo.mthds"))
+
+ assert exc_info.value.exit_code == 1
diff --git a/tests/unit/pipelex/cli/commands/build/test_structures_cmd.py b/tests/unit/pipelex/cli/commands/build/test_structures_cmd.py
new file mode 100644
index 000000000..479483443
--- /dev/null
+++ b/tests/unit/pipelex/cli/commands/build/test_structures_cmd.py
@@ -0,0 +1,217 @@
+"""Unit tests for the structure-codegen logic in build/structures_cmd.py.
+
+Drives generate_structures_from_blueprints and its helpers directly with real
+blueprints and a tmp output dir — the same pattern as the cross-package refines
+regression test next door.
+"""
+
+from __future__ import annotations
+
+from pathlib import Path
+from typing import TYPE_CHECKING
+
+from pipelex.cli.commands.build.structures_cmd import (
+ _build_concept_ref_to_class_info, # noqa: PLC2701 # pyright: ignore[reportPrivateUsage]
+ _compute_relative_path_from_output_dir, # noqa: PLC2701 # pyright: ignore[reportPrivateUsage]
+ generate_structures_from_blueprints,
+)
+from pipelex.core.bundles.pipelex_bundle_blueprint import PipelexBundleBlueprint
+from pipelex.core.concepts.concept_blueprint import ConceptBlueprint
+from pipelex.core.stuffs.text_content import TextContent
+from pipelex.hub import get_class_registry
+
+if TYPE_CHECKING:
+ import pytest
+ from pytest_mock import MockerFixture
+
+
+class TestStructuresCmd:
+ def test_relative_path_inside_cwd(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ """An output dir under the cwd yields its relative path."""
+ monkeypatch.chdir(tmp_path)
+ output_directory = tmp_path / "generated" / "structures"
+ output_directory.mkdir(parents=True)
+
+ assert _compute_relative_path_from_output_dir(output_directory) == Path("generated/structures")
+
+ def test_relative_path_outside_cwd(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ """An output dir outside the cwd yields None."""
+ inside_dir = tmp_path / "inside"
+ inside_dir.mkdir()
+ outside_dir = tmp_path / "outside"
+ outside_dir.mkdir()
+ monkeypatch.chdir(inside_dir)
+
+ assert _compute_relative_path_from_output_dir(outside_dir) is None
+
+ def test_relative_path_equal_to_cwd(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ """An output dir equal to the cwd yields the empty relative path."""
+ monkeypatch.chdir(tmp_path)
+
+ assert _compute_relative_path_from_output_dir(tmp_path) == Path()
+
+ def test_concept_ref_mapping_skips_native_and_qualifies_names(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ """The concept_ref map skips the native domain, qualifies bare names and builds module paths."""
+ monkeypatch.chdir(tmp_path)
+ output_directory = tmp_path / "structures"
+ output_directory.mkdir()
+ blueprints = [
+ PipelexBundleBlueprint(domain="native", description="native stub", concept={"NativeStub": "native stub concept"}),
+ PipelexBundleBlueprint(
+ domain="billing",
+ description="billing concepts",
+ concept={
+ "Invoice": ConceptBlueprint(description="An invoice", structure={"total": "the total"}),
+ "CustomName": ConceptBlueprint(description="Custom structure ref", structure="MyCustomClass"),
+ },
+ ),
+ ]
+
+ mapping = _build_concept_ref_to_class_info(blueprints, output_directory)
+
+ assert set(mapping.keys()) == {"billing.Invoice", "billing.CustomName"}
+ assert mapping["billing.Invoice"].class_name == "billing__Invoice"
+ assert mapping["billing.Invoice"].module_path == "structures.billing__invoice"
+ assert mapping["billing.CustomName"].class_name == "MyCustomClass"
+ assert mapping["billing.CustomName"].module_path == "structures.billing__my_custom_class"
+
+ def test_concept_ref_mapping_without_relative_path(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ """When the output dir is outside the cwd, module paths are None."""
+ inside_dir = tmp_path / "inside"
+ inside_dir.mkdir()
+ outside_dir = tmp_path / "outside"
+ outside_dir.mkdir()
+ monkeypatch.chdir(inside_dir)
+ blueprints = [
+ PipelexBundleBlueprint(
+ domain="billing",
+ description="billing concepts",
+ concept={"Invoice": ConceptBlueprint(description="An invoice")},
+ ),
+ ]
+
+ mapping = _build_concept_ref_to_class_info(blueprints, outside_dir)
+
+ assert mapping["billing.Invoice"].module_path is None
+
+ def test_generate_string_concept_defaults_to_text_content(self, tmp_path: Path) -> None:
+ """A bare string concept generates a TextContent subclass file."""
+ blueprints = [
+ PipelexBundleBlueprint(domain="notes", description="notes", concept={"QuickNote": "A short note"}),
+ ]
+
+ generated = generate_structures_from_blueprints(blueprints, tmp_path, skip_existing_check=True, quiet=True)
+
+ assert generated == [("notes", "QuickNote")]
+ generated_code = (tmp_path / "notes__quick_note.py").read_text(encoding="utf-8")
+ assert "class notes__QuickNote(TextContent):" in generated_code
+ assert (tmp_path / "__init__.py").exists()
+
+ def test_generate_explicit_structure(self, tmp_path: Path) -> None:
+ """An explicit structure dict generates a structured class with its fields."""
+ blueprints = [
+ PipelexBundleBlueprint(
+ domain="billing",
+ description="billing",
+ concept={
+ "Invoice": ConceptBlueprint(
+ description="An invoice",
+ structure={"total": "the invoice total", "customer_name": "the customer"},
+ ),
+ },
+ ),
+ ]
+
+ generated = generate_structures_from_blueprints(blueprints, tmp_path, skip_existing_check=True, quiet=True)
+
+ assert generated == [("billing", "Invoice")]
+ generated_code = (tmp_path / "billing__invoice.py").read_text(encoding="utf-8")
+ assert "class billing__Invoice" in generated_code
+ assert "total" in generated_code
+ assert "customer_name" in generated_code
+
+ def test_generate_refines_native_concept(self, tmp_path: Path) -> None:
+ """A concept refining a native concept inherits from its Content class."""
+ blueprints = [
+ PipelexBundleBlueprint(
+ domain="notes",
+ description="notes",
+ concept={"FancyText": ConceptBlueprint(description="Fancy text", refines="Text")},
+ ),
+ ]
+
+ generated = generate_structures_from_blueprints(blueprints, tmp_path, skip_existing_check=True, quiet=True)
+
+ assert generated == [("notes", "FancyText")]
+ generated_code = (tmp_path / "notes__fancy_text.py").read_text(encoding="utf-8")
+ assert "class notes__FancyText(TextContent):" in generated_code
+
+ def test_generate_default_concept_blueprint(self, tmp_path: Path) -> None:
+ """A blueprint with neither structure nor refines defaults to TextContent."""
+ blueprints = [
+ PipelexBundleBlueprint(
+ domain="notes",
+ description="notes",
+ concept={"PlainThing": ConceptBlueprint(description="Just a thing")},
+ ),
+ ]
+
+ generated = generate_structures_from_blueprints(blueprints, tmp_path, skip_existing_check=True, quiet=True)
+
+ assert generated == [("notes", "PlainThing")]
+ generated_code = (tmp_path / "notes__plain_thing.py").read_text(encoding="utf-8")
+ assert "class notes__PlainThing(TextContent):" in generated_code
+
+ def test_generate_skips_native_domain_and_writes_no_init(self, tmp_path: Path) -> None:
+ """Native-domain blueprints generate nothing — not even an __init__.py."""
+ blueprints = [
+ PipelexBundleBlueprint(domain="native", description="native stub", concept={"NativeStub": "native stub concept"}),
+ ]
+
+ generated = generate_structures_from_blueprints(blueprints, tmp_path, skip_existing_check=True, quiet=True)
+
+ assert generated == []
+ assert not (tmp_path / "__init__.py").exists()
+ assert list(tmp_path.iterdir()) == []
+
+ def test_generate_skips_manually_created_class(self, tmp_path: Path) -> None:
+ """With existing-class checking on, a registered class suppresses generation."""
+ manual_class: type = type("ManualDoctorTestConcept", (TextContent,), {})
+ get_class_registry().register_class(manual_class, name="ManualDoctorTestConcept", should_warn_if_already_registered=False)
+ try:
+ blueprints = [
+ PipelexBundleBlueprint(
+ domain="notes",
+ description="notes",
+ concept={"ManualDoctorTestConcept": ConceptBlueprint(description="Manually owned")},
+ ),
+ ]
+
+ generated = generate_structures_from_blueprints(
+ blueprints,
+ tmp_path,
+ target_path=tmp_path,
+ skip_existing_check=False,
+ quiet=True,
+ )
+
+ assert generated == []
+ assert not (tmp_path / "notes__manual_doctor_test_concept.py").exists()
+ finally:
+ # The class registry is process-global — leaving the class registered would make later tests order-dependent
+ get_class_registry().unregister_class_by_name("ManualDoctorTestConcept")
+
+ def test_generate_non_quiet_echoes_progress(self, tmp_path: Path, mocker: MockerFixture) -> None:
+ """Without quiet, progress is echoed through typer."""
+ echo_mock = mocker.patch("pipelex.cli.commands.build.structures_cmd.typer.echo")
+ secho_mock = mocker.patch("pipelex.cli.commands.build.structures_cmd.typer.secho")
+ blueprints = [
+ PipelexBundleBlueprint(domain="notes", description="notes", concept={"LoudNote": "A loud note"}),
+ ]
+
+ generate_structures_from_blueprints(blueprints, tmp_path, skip_existing_check=True, quiet=False)
+
+ echo_mock.assert_called_once()
+ secho_calls = [str(call.args[0]) for call in secho_mock.call_args_list]
+ assert any("notes__loud_note.py" in message for message in secho_calls)
+ assert any("__init__.py" in message for message in secho_calls)
diff --git a/tests/unit/pipelex/cli/test_doctor_backend_checks.py b/tests/unit/pipelex/cli/test_doctor_backend_checks.py
new file mode 100644
index 000000000..87f9ad7b7
--- /dev/null
+++ b/tests/unit/pipelex/cli/test_doctor_backend_checks.py
@@ -0,0 +1,212 @@
+"""Unit tests for doctor's backend credential / backend file / kit template checks."""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
+from pipelex.cli.commands.doctor_cmd import (
+ check_backend_credentials,
+ check_backend_files,
+ check_kit_template_exists,
+ replace_backend_file,
+)
+from pipelex.cogt.exceptions import InferenceBackendLibraryError
+
+if TYPE_CHECKING:
+ from pathlib import Path
+
+ import pytest
+ from pytest_mock import MockerFixture
+
+BACKENDS_TOML = """
+[internal]
+enabled = true
+
+[openai]
+enabled = true
+api_key = "${TEST_DOCTOR_OPENAI_KEY}"
+
+[azure]
+enabled = false
+api_key = "${TEST_DOCTOR_AZURE_KEY}"
+"""
+
+
+def _write_backends_toml(config_dir: Path, content: str = BACKENDS_TOML) -> Path:
+ inference_dir = config_dir / "inference"
+ inference_dir.mkdir(parents=True, exist_ok=True)
+ backends_toml = inference_dir / "backends.toml"
+ backends_toml.write_text(content, encoding="utf-8")
+ return backends_toml
+
+
+class TestDoctorBackendChecks:
+ def test_credentials_backends_toml_missing(self, tmp_path: Path) -> None:
+ """No backends.toml means the credentials check fails with a clear message."""
+ healthy, reports, message = check_backend_credentials(config_dir=tmp_path)
+
+ assert healthy is False
+ assert reports == {}
+ assert message == "Backend configuration file not found"
+
+ def test_credentials_all_valid(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ """All env vars set with real values is healthy; internal and disabled backends are skipped."""
+ _write_backends_toml(tmp_path)
+ monkeypatch.setenv("TEST_DOCTOR_OPENAI_KEY", "sk-real-value")
+ monkeypatch.delenv("TEST_DOCTOR_AZURE_KEY", raising=False)
+
+ healthy, reports, message = check_backend_credentials(config_dir=tmp_path)
+
+ assert healthy is True
+ assert set(reports.keys()) == {"openai"}
+ assert reports["openai"].all_credentials_valid is True
+ assert reports["openai"].required_vars == ["TEST_DOCTOR_OPENAI_KEY"]
+ assert message == "All 1 enabled backend(s) have valid credentials"
+
+ def test_credentials_missing_env_var(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ """An unset env var is reported as missing for its backend."""
+ _write_backends_toml(tmp_path)
+ monkeypatch.delenv("TEST_DOCTOR_OPENAI_KEY", raising=False)
+
+ healthy, reports, message = check_backend_credentials(config_dir=tmp_path)
+
+ assert healthy is False
+ assert reports["openai"].missing_vars == ["TEST_DOCTOR_OPENAI_KEY"]
+ assert reports["openai"].all_credentials_valid is False
+ assert message == "1 backend(s) have missing or invalid credentials"
+
+ def test_credentials_placeholder_value(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ """An env var still holding a placeholder value is reported separately."""
+ _write_backends_toml(tmp_path)
+ monkeypatch.setenv("TEST_DOCTOR_OPENAI_KEY", "placeholder-for-TEST_DOCTOR_OPENAI_KEY")
+
+ healthy, reports, _ = check_backend_credentials(config_dir=tmp_path)
+
+ assert healthy is False
+ assert reports["openai"].placeholder_vars == ["TEST_DOCTOR_OPENAI_KEY"]
+ assert reports["openai"].missing_vars == []
+
+ def test_credentials_broad_failure_reported(self, tmp_path: Path, mocker: MockerFixture) -> None:
+ """Any unexpected failure during the scan becomes a finding, not a crash."""
+ _write_backends_toml(tmp_path)
+ mocker.patch("pipelex.cli.commands.doctor_cmd.load_toml_from_path", side_effect=RuntimeError("kaboom"))
+
+ healthy, reports, message = check_backend_credentials(config_dir=tmp_path)
+
+ assert healthy is False
+ assert reports == {}
+ assert message == "Error checking backend credentials: kaboom"
+
+ def test_backend_files_no_backends_dir(self, tmp_path: Path) -> None:
+ """A missing inference/backends directory is healthy (nothing to check)."""
+ healthy, reports, message = check_backend_files(config_dir=tmp_path)
+
+ assert healthy is True
+ assert reports == {}
+ assert message == "No backend files to check"
+
+ def test_backend_files_no_backends_toml(self, tmp_path: Path) -> None:
+ """A backends dir without backends.toml is healthy (nothing to check)."""
+ (tmp_path / "inference" / "backends").mkdir(parents=True)
+
+ healthy, reports, message = check_backend_files(config_dir=tmp_path)
+
+ assert healthy is True
+ assert reports == {}
+ assert message == "No backends.toml to check"
+
+ def test_backend_files_toml_error(self, tmp_path: Path) -> None:
+ """An unparseable backends.toml fails the check with a load error."""
+ _write_backends_toml(tmp_path, content="not [ valid toml")
+ (tmp_path / "inference" / "backends").mkdir(parents=True)
+
+ healthy, reports, message = check_backend_files(config_dir=tmp_path)
+
+ assert healthy is False
+ assert reports == {}
+ assert message.startswith("Error loading backends.toml:")
+
+ def _setup_backend_file(self, tmp_path: Path) -> Path:
+ _write_backends_toml(tmp_path)
+ backends_dir = tmp_path / "inference" / "backends"
+ backends_dir.mkdir(parents=True)
+ backend_file = backends_dir / "openai.toml"
+ backend_file.write_text("# openai backend specs\n", encoding="utf-8")
+ return backend_file
+
+ def test_backend_files_valid(self, tmp_path: Path, mocker: MockerFixture) -> None:
+ """A backend file that loads cleanly is reported valid, with kit-template info."""
+ self._setup_backend_file(tmp_path)
+ library_mock = mocker.Mock()
+ mocker.patch("pipelex.cli.commands.doctor_cmd.InferenceBackendLibrary.make_empty", return_value=library_mock)
+
+ healthy, reports, message = check_backend_files(config_dir=tmp_path)
+
+ assert healthy is True
+ assert message == "All backend files are valid"
+ assert reports["openai"].is_valid is True
+ assert reports["openai"].error_message is None
+ # openai ships in the kit, so the template is found by the real probe
+ assert reports["openai"].has_kit_template is True
+
+ def test_backend_files_library_error_naming_backend(self, tmp_path: Path, mocker: MockerFixture) -> None:
+ """A library error naming the backend marks that file invalid."""
+ self._setup_backend_file(tmp_path)
+ library_mock = mocker.Mock()
+ library_mock.load.side_effect = InferenceBackendLibraryError("openai: invalid model spec")
+ mocker.patch("pipelex.cli.commands.doctor_cmd.InferenceBackendLibrary.make_empty", return_value=library_mock)
+
+ healthy, reports, message = check_backend_files(config_dir=tmp_path)
+
+ assert healthy is False
+ assert message == "1 backend file(s) have validation errors"
+ assert reports["openai"].is_valid is False
+ assert reports["openai"].error_message == "openai: invalid model spec"
+
+ def test_backend_files_library_error_not_naming_backend(self, tmp_path: Path, mocker: MockerFixture) -> None:
+ """A library error that names neither the backend nor its file leaves it valid."""
+ self._setup_backend_file(tmp_path)
+ library_mock = mocker.Mock()
+ library_mock.load.side_effect = InferenceBackendLibraryError("some unrelated failure")
+ mocker.patch("pipelex.cli.commands.doctor_cmd.InferenceBackendLibrary.make_empty", return_value=library_mock)
+
+ healthy, reports, _ = check_backend_files(config_dir=tmp_path)
+
+ assert healthy is True
+ assert reports["openai"].is_valid is True
+
+ def test_kit_template_exists_for_shipped_backend(self) -> None:
+ """The kit ships an openai backend template."""
+ assert check_kit_template_exists("openai") is True
+
+ def test_kit_template_missing_for_unknown_backend(self) -> None:
+ """An unknown backend has no kit template."""
+ assert check_kit_template_exists("definitely_not_a_backend") is False
+
+ def test_kit_template_lookup_failure_returns_false(self, mocker: MockerFixture) -> None:
+ """A failing kit lookup degrades to 'no template'."""
+ mocker.patch("pipelex.cli.commands.doctor_cmd.get_kit_configs_dir", side_effect=RuntimeError("no kit"))
+
+ assert check_kit_template_exists("openai") is False
+
+ def test_replace_backend_file_writes_template(self, tmp_path: Path) -> None:
+ """Replacing a backend file copies the kit template into the config dir."""
+ success = replace_backend_file("openai", dry_run=False, config_dir=tmp_path)
+
+ assert success is True
+ target_file = tmp_path / "inference" / "backends" / "openai.toml"
+ assert target_file.is_file()
+ assert target_file.read_text(encoding="utf-8") != ""
+
+ def test_replace_backend_file_dry_run_does_not_write(self, tmp_path: Path) -> None:
+ """Dry-run reports success without touching the filesystem."""
+ success = replace_backend_file("openai", dry_run=True, config_dir=tmp_path)
+
+ assert success is True
+ assert not (tmp_path / "inference" / "backends" / "openai.toml").exists()
+
+ def test_replace_backend_file_missing_template(self, tmp_path: Path) -> None:
+ """A backend without a kit template cannot be replaced."""
+ success = replace_backend_file("definitely_not_a_backend", dry_run=False, config_dir=tmp_path)
+
+ assert success is False
diff --git a/tests/unit/pipelex/cli/test_doctor_check_models.py b/tests/unit/pipelex/cli/test_doctor_check_models.py
new file mode 100644
index 000000000..ea739fbd1
--- /dev/null
+++ b/tests/unit/pipelex/cli/test_doctor_check_models.py
@@ -0,0 +1,164 @@
+"""Unit tests for doctor's check_models — gateway gating and model deck validation paths.
+
+All inference-adjacent collaborators (backend files probe, gateway config fetch,
+ModelManager) are mocked at the doctor module namespace: these tests cover the
+decision tree, not the model loading itself.
+"""
+
+from __future__ import annotations
+
+from types import SimpleNamespace
+from typing import TYPE_CHECKING, Any
+
+import pytest
+
+from pipelex.cli.commands.doctor_cmd import BackendFileReport, check_models
+from pipelex.cogt.exceptions import InferenceBackendLibraryError, ModelDeckValidationError
+from pipelex.system.pipelex_service.exceptions import RemoteConfigUnavailableError
+from pipelex.system.pipelex_service.types import RemoteConfigSource
+
+if TYPE_CHECKING:
+ from pytest_mock import MockerFixture
+
+
+class TestCheckModels:
+ @pytest.fixture
+ def healthy_backend_files(self, mocker: MockerFixture) -> None:
+ mocker.patch(
+ "pipelex.cli.commands.doctor_cmd.check_backend_files",
+ return_value=(True, {}, "All backend files are valid"),
+ )
+
+ @pytest.fixture
+ def gateway_disabled(self, mocker: MockerFixture) -> None:
+ mocker.patch("pipelex.cli.commands.doctor_cmd.is_pipelex_gateway_enabled", return_value=False)
+
+ @pytest.fixture
+ def models_manager(self, mocker: MockerFixture) -> Any:
+ manager_mock = mocker.Mock()
+ mocker.patch("pipelex.cli.commands.doctor_cmd.ModelManager", return_value=manager_mock)
+ return manager_mock
+
+ def test_backend_files_unhealthy_short_circuits(self, mocker: MockerFixture) -> None:
+ """Invalid backend files stop the check before any model loading."""
+ bad_report = BackendFileReport(
+ backend_name="openai",
+ file_path="/cfg/inference/backends/openai.toml",
+ is_valid=False,
+ error_message="openai: bad spec",
+ )
+ mocker.patch(
+ "pipelex.cli.commands.doctor_cmd.check_backend_files",
+ return_value=(False, {"openai": bad_report}, "1 backend file(s) have validation errors"),
+ )
+ manager_class_mock = mocker.patch("pipelex.cli.commands.doctor_cmd.ModelManager")
+
+ healthy, message, reports = check_models()
+
+ assert healthy is False
+ assert message == "Backend configuration error: openai: bad spec"
+ assert reports == {"openai": bad_report}
+ manager_class_mock.assert_not_called()
+
+ @pytest.mark.usefixtures("healthy_backend_files", "gateway_disabled")
+ def test_gateway_disabled_happy_path(self, models_manager: Any) -> None:
+ """With the gateway disabled and a valid deck, models are healthy."""
+ healthy, message, reports = check_models()
+
+ assert healthy is True
+ assert message == "Models are valid"
+ assert reports == {}
+ models_manager.setup.assert_called_once()
+ assert models_manager.setup.call_args.kwargs["gateway_config"] is None
+ models_manager.validate_model_deck.assert_called_once()
+
+ @pytest.mark.usefixtures("healthy_backend_files")
+ def test_gateway_enabled_missing_service_config(self, mocker: MockerFixture) -> None:
+ """Gateway enabled without a service config is unhealthy."""
+ mocker.patch("pipelex.cli.commands.doctor_cmd.is_pipelex_gateway_enabled", return_value=True)
+ mocker.patch("pipelex.cli.commands.doctor_cmd.load_pipelex_service_config_if_exists", return_value=None)
+
+ healthy, message, _ = check_models()
+
+ assert healthy is False
+ assert message == "Pipelex Gateway is enabled but service configuration is missing"
+
+ @pytest.mark.usefixtures("healthy_backend_files")
+ def test_gateway_enabled_terms_not_accepted(self, mocker: MockerFixture) -> None:
+ """Gateway enabled with unaccepted terms is unhealthy."""
+ mocker.patch("pipelex.cli.commands.doctor_cmd.is_pipelex_gateway_enabled", return_value=True)
+ service_config = SimpleNamespace(agreement=SimpleNamespace(terms_accepted=False))
+ mocker.patch("pipelex.cli.commands.doctor_cmd.load_pipelex_service_config_if_exists", return_value=service_config)
+
+ healthy, message, _ = check_models()
+
+ assert healthy is False
+ assert message == "Pipelex Gateway is enabled but terms have not been accepted"
+
+ @pytest.mark.usefixtures("healthy_backend_files")
+ def test_gateway_enabled_remote_fetch_failure(self, mocker: MockerFixture) -> None:
+ """A failed remote-config fetch is unhealthy with the fetch error in the message."""
+ mocker.patch("pipelex.cli.commands.doctor_cmd.is_pipelex_gateway_enabled", return_value=True)
+ service_config = SimpleNamespace(agreement=SimpleNamespace(terms_accepted=True))
+ mocker.patch("pipelex.cli.commands.doctor_cmd.load_pipelex_service_config_if_exists", return_value=service_config)
+ mocker.patch(
+ "pipelex.cli.commands.doctor_cmd.RemoteConfigFetcher.fetch_remote_config",
+ side_effect=RemoteConfigUnavailableError("offline, cold cache"),
+ )
+
+ healthy, message, _ = check_models()
+
+ assert healthy is False
+ assert message == "Failed to fetch Pipelex Gateway remote configuration: offline, cold cache"
+
+ @pytest.mark.usefixtures("healthy_backend_files")
+ def test_gateway_enabled_passes_gateway_config_to_setup(self, mocker: MockerFixture, models_manager: Any) -> None:
+ """A successful fetch builds a GatewayConfig and threads it into the model setup."""
+ mocker.patch("pipelex.cli.commands.doctor_cmd.is_pipelex_gateway_enabled", return_value=True)
+ service_config = SimpleNamespace(agreement=SimpleNamespace(terms_accepted=True))
+ mocker.patch("pipelex.cli.commands.doctor_cmd.load_pipelex_service_config_if_exists", return_value=service_config)
+ fetch_result = SimpleNamespace(
+ config=SimpleNamespace(backend_model_specs={}, aws_region="eu-west-3"),
+ source=RemoteConfigSource.FRESH,
+ )
+ mocker.patch("pipelex.cli.commands.doctor_cmd.RemoteConfigFetcher.fetch_remote_config", return_value=fetch_result)
+
+ healthy, message, _ = check_models()
+
+ assert healthy is True
+ assert message == "Models are valid"
+ setup_kwargs = models_manager.setup.call_args.kwargs
+ assert setup_kwargs["gateway_config"] is not None
+ assert setup_kwargs["gateway_config"].aws_region == "eu-west-3"
+ assert setup_kwargs["gateway_config_source"] == RemoteConfigSource.FRESH
+
+ @pytest.mark.usefixtures("healthy_backend_files", "gateway_disabled")
+ def test_model_deck_validation_error(self, models_manager: Any) -> None:
+ """A deck validation failure is reported as a models error."""
+ models_manager.validate_model_deck.side_effect = ModelDeckValidationError("preset broken")
+
+ healthy, message, _ = check_models()
+
+ assert healthy is False
+ assert message == "Error checking models: preset broken"
+
+ @pytest.mark.usefixtures("gateway_disabled")
+ def test_backend_library_error_updates_named_backend_report(self, mocker: MockerFixture, models_manager: Any) -> None:
+ """A library error naming a known backend flips that backend's report to invalid."""
+ openai_report = BackendFileReport(
+ backend_name="openai",
+ file_path="/cfg/inference/backends/openai.toml",
+ is_valid=True,
+ )
+ mocker.patch(
+ "pipelex.cli.commands.doctor_cmd.check_backend_files",
+ return_value=(True, {"openai": openai_report}, "All backend files are valid"),
+ )
+ models_manager.setup.side_effect = InferenceBackendLibraryError("openai: cannot resolve model")
+
+ healthy, message, reports = check_models()
+
+ assert healthy is False
+ assert message == "Error checking models: openai: cannot resolve model"
+ assert reports["openai"].is_valid is False
+ assert reports["openai"].error_message == "openai: cannot resolve model"
diff --git a/tests/unit/pipelex/cli/test_doctor_config_checks.py b/tests/unit/pipelex/cli/test_doctor_config_checks.py
new file mode 100644
index 000000000..9fbd1ed29
--- /dev/null
+++ b/tests/unit/pipelex/cli/test_doctor_config_checks.py
@@ -0,0 +1,198 @@
+"""Unit tests for doctor's config-location, config-files and telemetry checks."""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
+from pydantic import BaseModel, ValidationError
+
+from pipelex.cli.commands.doctor_cmd import (
+ check_config_files,
+ check_telemetry_config,
+ gather_config_location,
+)
+from pipelex.cli.exceptions import PipelexCLIError
+from pipelex.system.configuration.config_loader import ConfigLoader, config_manager
+from pipelex.system.configuration.configs import PipelexConfig
+from pipelex.system.exceptions import ConfigValidationError
+from pipelex.system.telemetry.telemetry_config import TELEMETRY_CONFIG_FILE_NAME
+from pipelex.tools.misc.exceptions import TomlError
+
+if TYPE_CHECKING:
+ from pathlib import Path
+
+ from pytest_mock import MockerFixture
+
+
+class _TinyModel(BaseModel):
+ value: int
+
+
+def _make_validation_error() -> ValidationError:
+ try:
+ _TinyModel.model_validate({"value": "not-an-int"})
+ except ValidationError as exc:
+ return exc
+ msg = "expected validation to fail"
+ raise AssertionError(msg)
+
+
+class TestDoctorConfigChecks:
+ def test_gather_config_location_project_local(self, mocker: MockerFixture, tmp_path: Path) -> None:
+ """With a project .pipelex/ resolved, the location is reported as project-local."""
+ project_dir = tmp_path / "project" / ".pipelex"
+ global_dir = tmp_path / "global"
+ mocker.patch.object(ConfigLoader, "project_config_dir", new_callable=mocker.PropertyMock, return_value=project_dir)
+ mocker.patch.object(ConfigLoader, "project_root", new_callable=mocker.PropertyMock, return_value=tmp_path / "project")
+ mocker.patch.object(ConfigLoader, "global_config_dir", new_callable=mocker.PropertyMock, return_value=global_dir)
+ mocker.patch.object(ConfigLoader, "pipelex_config_dir", new_callable=mocker.PropertyMock, return_value=project_dir)
+
+ location = gather_config_location()
+
+ assert location.is_project_local is True
+ assert location.config_dir == str(project_dir)
+ assert location.project_root == str(tmp_path / "project")
+ assert location.global_config_dir == str(global_dir)
+
+ def test_gather_config_location_global(self, mocker: MockerFixture, tmp_path: Path) -> None:
+ """Without a project .pipelex/, the location is the global config dir."""
+ global_dir = tmp_path / "global"
+ mocker.patch.object(ConfigLoader, "project_config_dir", new_callable=mocker.PropertyMock, return_value=None)
+ mocker.patch.object(ConfigLoader, "project_root", new_callable=mocker.PropertyMock, return_value=None)
+ mocker.patch.object(ConfigLoader, "global_config_dir", new_callable=mocker.PropertyMock, return_value=global_dir)
+ mocker.patch.object(ConfigLoader, "pipelex_config_dir", new_callable=mocker.PropertyMock, return_value=global_dir)
+
+ location = gather_config_location()
+
+ assert location.is_project_local is False
+ assert location.project_root is None
+ assert location.config_dir == str(global_dir)
+
+ def test_check_config_files_all_good(self, mocker: MockerFixture, tmp_path: Path) -> None:
+ """No missing files and no pipelex.toml to validate is healthy."""
+ mocker.patch("pipelex.cli.commands.doctor_cmd.init_config", return_value=0)
+
+ healthy, missing_count, message = check_config_files(config_dir=tmp_path)
+
+ assert healthy is True
+ assert missing_count == 0
+ assert message == "All configuration files present and valid"
+
+ def test_check_config_files_missing_files_counted(self, mocker: MockerFixture, tmp_path: Path) -> None:
+ """Missing config files are counted in the unhealthy message."""
+ mocker.patch("pipelex.cli.commands.doctor_cmd.init_config", return_value=3)
+
+ healthy, missing_count, message = check_config_files(config_dir=tmp_path)
+
+ assert healthy is False
+ assert missing_count == 3
+ assert message == "3 configuration file(s) missing"
+
+ def test_check_config_files_init_error(self, mocker: MockerFixture, tmp_path: Path) -> None:
+ """A failure while probing for missing files is reported as a finding."""
+ mocker.patch("pipelex.cli.commands.doctor_cmd.init_config", side_effect=PipelexCLIError("probe failed"))
+
+ healthy, missing_count, message = check_config_files(config_dir=tmp_path)
+
+ assert healthy is False
+ assert missing_count == 0
+ assert "Error checking config files: probe failed" in message
+
+ def test_check_config_files_validation_error(self, mocker: MockerFixture, tmp_path: Path) -> None:
+ """A pydantic ValidationError on the merged config produces the migration-aware report."""
+ (tmp_path / "pipelex.toml").write_text("[pipelex]\n", encoding="utf-8")
+ mocker.patch("pipelex.cli.commands.doctor_cmd.init_config", return_value=0)
+ mocker.patch.object(config_manager, "load_config", return_value={})
+ mocker.patch.object(PipelexConfig, "model_validate", side_effect=_make_validation_error())
+
+ healthy, missing_count, message = check_config_files(config_dir=tmp_path)
+
+ assert healthy is False
+ assert missing_count == 0
+ assert message.startswith("Configuration validation failed:")
+
+ def test_check_config_files_config_validation_error_with_cause(self, mocker: MockerFixture, tmp_path: Path) -> None:
+ """A ConfigValidationError wrapping a ValidationError recovers the original report."""
+ (tmp_path / "pipelex.toml").write_text("[pipelex]\n", encoding="utf-8")
+ mocker.patch("pipelex.cli.commands.doctor_cmd.init_config", return_value=0)
+ wrapped_error = ConfigValidationError("wrapped")
+ wrapped_error.__cause__ = _make_validation_error()
+ mocker.patch.object(config_manager, "load_config", side_effect=wrapped_error)
+
+ healthy, _, message = check_config_files(config_dir=tmp_path)
+
+ assert healthy is False
+ assert message.startswith("Configuration validation failed:")
+ assert "wrapped" not in message
+
+ def test_check_config_files_config_validation_error_without_cause(self, mocker: MockerFixture, tmp_path: Path) -> None:
+ """A ConfigValidationError without an underlying ValidationError uses its own message."""
+ (tmp_path / "pipelex.toml").write_text("[pipelex]\n", encoding="utf-8")
+ mocker.patch("pipelex.cli.commands.doctor_cmd.init_config", return_value=0)
+ mocker.patch.object(config_manager, "load_config", side_effect=ConfigValidationError("plain failure"))
+
+ healthy, _, message = check_config_files(config_dir=tmp_path)
+
+ assert healthy is False
+ assert message == "Configuration validation failed: plain failure"
+
+ def test_check_config_files_toml_error(self, mocker: MockerFixture, tmp_path: Path) -> None:
+ """A TOML parse failure on pipelex.toml is reported as a load error."""
+ (tmp_path / "pipelex.toml").write_text("not valid toml [", encoding="utf-8")
+ mocker.patch("pipelex.cli.commands.doctor_cmd.init_config", return_value=0)
+ mocker.patch.object(
+ config_manager,
+ "load_config",
+ side_effect=TomlError(message="unterminated table", doc="", pos=0, lineno=1, colno=1),
+ )
+
+ healthy, _, message = check_config_files(config_dir=tmp_path)
+
+ assert healthy is False
+ assert "Error loading pipelex.toml:" in message
+ assert "unterminated table" in message
+
+ def test_check_config_files_os_error(self, mocker: MockerFixture, tmp_path: Path) -> None:
+ """An OSError while loading the config is reported as a load error."""
+ (tmp_path / "pipelex.toml").write_text("[pipelex]\n", encoding="utf-8")
+ mocker.patch("pipelex.cli.commands.doctor_cmd.init_config", return_value=0)
+ mocker.patch.object(config_manager, "load_config", side_effect=OSError("permission denied"))
+
+ healthy, _, message = check_config_files(config_dir=tmp_path)
+
+ assert healthy is False
+ assert "Error loading pipelex.toml: permission denied" in message
+
+ def test_check_telemetry_config_file_not_found(self, tmp_path: Path) -> None:
+ """A missing telemetry.toml is unhealthy with a clear message."""
+ healthy, message = check_telemetry_config(config_dir=tmp_path)
+
+ assert healthy is False
+ assert message == "Telemetry configuration file not found"
+
+ def test_check_telemetry_config_toml_syntax_error(self, tmp_path: Path) -> None:
+ """Invalid TOML in telemetry.toml is reported as a syntax error."""
+ (tmp_path / TELEMETRY_CONFIG_FILE_NAME).write_text("not [ valid toml", encoding="utf-8")
+
+ healthy, message = check_telemetry_config(config_dir=tmp_path)
+
+ assert healthy is False
+ assert message.startswith("TOML syntax error:")
+
+ def test_check_telemetry_config_old_format_detected(self, tmp_path: Path) -> None:
+ """The legacy flat format gets the dedicated migration message."""
+ (tmp_path / TELEMETRY_CONFIG_FILE_NAME).write_text('telemetry_mode = "off"\nproject_api_key = "key"\n', encoding="utf-8")
+
+ healthy, message = check_telemetry_config(config_dir=tmp_path)
+
+ assert healthy is False
+ assert message == "Config format has changed - run 'pipelex init telemetry' to update"
+
+ def test_check_telemetry_config_invalid_new_format(self, tmp_path: Path) -> None:
+ """A new-format file that fails validation gets the generic fix message."""
+ (tmp_path / TELEMETRY_CONFIG_FILE_NAME).write_text('[custom_posthog]\nmode = "no-such-mode"\n', encoding="utf-8")
+
+ healthy, message = check_telemetry_config(config_dir=tmp_path)
+
+ assert healthy is False
+ assert message == "Invalid configuration - run 'pipelex init telemetry' to fix"
diff --git a/tests/unit/pipelex/cli/test_doctor_deck_sync.py b/tests/unit/pipelex/cli/test_doctor_deck_sync.py
new file mode 100644
index 000000000..0f98372bf
--- /dev/null
+++ b/tests/unit/pipelex/cli/test_doctor_deck_sync.py
@@ -0,0 +1,98 @@
+"""Unit tests for doctor's deck sync check."""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
+from pipelex.cli.commands.doctor_cmd import check_deck_sync
+from pipelex.cogt.models.deck_manifest import DeckFileStatus, DeckSyncReport
+
+if TYPE_CHECKING:
+ from pathlib import Path
+
+ from pytest_mock import MockerFixture
+
+
+class TestCheckDeckSync:
+ def _make_deck_dir(self, tmp_path: Path) -> Path:
+ deck_dir = tmp_path / "inference" / "deck"
+ deck_dir.mkdir(parents=True)
+ return deck_dir
+
+ def test_deck_dir_missing_is_healthy(self, tmp_path: Path) -> None:
+ """A missing deck dir defers to the config checks and reports healthy."""
+ healthy, report, message = check_deck_sync(config_dir=tmp_path)
+
+ assert healthy is True
+ assert report.manifest_present is False
+ assert message == "Deck directory not present"
+
+ def test_clean_deck(self, tmp_path: Path, mocker: MockerFixture) -> None:
+ """A clean deck is healthy and names the kit version."""
+ self._make_deck_dir(tmp_path)
+ clean_report = DeckSyncReport(
+ kit_version="1.2.0",
+ installed_kit_version="1.2.0",
+ manifest_present=True,
+ files={"deck.toml": DeckFileStatus.UP_TO_DATE},
+ )
+ mocker.patch("pipelex.cli.commands.doctor_cmd.compute_deck_sync_report", return_value=clean_report)
+
+ healthy, report, message = check_deck_sync(config_dir=tmp_path)
+
+ assert healthy is True
+ assert report is clean_report
+ assert message == "Deck is up to date with pipelex 1.2.0"
+
+ def test_manifest_missing(self, tmp_path: Path, mocker: MockerFixture) -> None:
+ """A deck without a manifest points the user at `pipelex update`."""
+ self._make_deck_dir(tmp_path)
+ dirty_report = DeckSyncReport(
+ kit_version="1.2.0",
+ installed_kit_version=None,
+ manifest_present=False,
+ files={"deck.toml": DeckFileStatus.LOCALLY_MODIFIED},
+ )
+ mocker.patch("pipelex.cli.commands.doctor_cmd.compute_deck_sync_report", return_value=dirty_report)
+
+ healthy, _, message = check_deck_sync(config_dir=tmp_path)
+
+ assert healthy is False
+ assert "Deck manifest missing" in message
+ assert "pipelex update" in message
+
+ def test_kit_version_mismatch(self, tmp_path: Path, mocker: MockerFixture) -> None:
+ """A version-skewed deck reports both versions and the actionable file count."""
+ self._make_deck_dir(tmp_path)
+ dirty_report = DeckSyncReport(
+ kit_version="1.3.0",
+ installed_kit_version="1.2.0",
+ manifest_present=True,
+ files={
+ "deck.toml": DeckFileStatus.CLEAN_BEHIND,
+ "extra.toml": DeckFileStatus.KIT_ADDED,
+ "ok.toml": DeckFileStatus.UP_TO_DATE,
+ },
+ )
+ mocker.patch("pipelex.cli.commands.doctor_cmd.compute_deck_sync_report", return_value=dirty_report)
+
+ healthy, _, message = check_deck_sync(config_dir=tmp_path)
+
+ assert healthy is False
+ assert message == "Deck installed for pipelex 1.2.0, current is 1.3.0 (2 file(s) need action)"
+
+ def test_same_version_with_actionable_files(self, tmp_path: Path, mocker: MockerFixture) -> None:
+ """A same-version deck with modified files reports the actionable count."""
+ self._make_deck_dir(tmp_path)
+ dirty_report = DeckSyncReport(
+ kit_version="1.2.0",
+ installed_kit_version="1.2.0",
+ manifest_present=True,
+ files={"deck.toml": DeckFileStatus.LOCALLY_MODIFIED},
+ )
+ mocker.patch("pipelex.cli.commands.doctor_cmd.compute_deck_sync_report", return_value=dirty_report)
+
+ healthy, _, message = check_deck_sync(config_dir=tmp_path)
+
+ assert healthy is False
+ assert message == "1 deck file(s) need action"
diff --git a/tests/unit/pipelex/cli/test_doctor_display_report.py b/tests/unit/pipelex/cli/test_doctor_display_report.py
new file mode 100644
index 000000000..938eba55a
--- /dev/null
+++ b/tests/unit/pipelex/cli/test_doctor_display_report.py
@@ -0,0 +1,242 @@
+"""Unit tests for doctor's display_health_report rendering and the doctor_cmd error guard."""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any
+
+import pytest
+from rich.console import Console
+
+from pipelex.cli.commands.doctor_cmd import (
+ BackendFileReport,
+ ConfigLocationInfo,
+ display_health_report,
+ doctor_cmd,
+)
+from pipelex.cogt.model_backends.backend_library import BackendCredentialsReport
+from pipelex.cogt.models.deck_manifest import DeckFileStatus, DeckSyncReport
+
+if TYPE_CHECKING:
+ from pytest_mock import MockerFixture
+
+CLEAN_DECK = DeckSyncReport(kit_version="1.2.0", installed_kit_version="1.2.0", manifest_present=True, files={})
+
+PROJECT_LOCATION = ConfigLocationInfo(
+ config_dir="/work/project/.pipelex",
+ is_project_local=True,
+ project_root="/work/project",
+ global_config_dir="/home/user/.pipelex",
+)
+
+GLOBAL_LOCATION = ConfigLocationInfo(
+ config_dir="/home/user/.pipelex",
+ is_project_local=False,
+ project_root=None,
+ global_config_dir="/home/user/.pipelex",
+)
+
+
+def _healthy_report_kwargs() -> dict[str, Any]:
+ return {
+ "config_healthy": True,
+ "config_message": "All configuration files present and valid",
+ "config_missing_count": 0,
+ "telemetry_healthy": True,
+ "telemetry_message": "Telemetry configured (mode: off)",
+ "backends_healthy": True,
+ "backends_message": "All 1 enabled backend(s) have valid credentials",
+ "backend_credential_reports": {},
+ "models_healthy": True,
+ "models_message": "Models are valid",
+ "backend_file_reports": {},
+ "deck_healthy": True,
+ "deck_message": "Deck is up to date with pipelex 1.2.0",
+ "deck_report": CLEAN_DECK,
+ "config_location": PROJECT_LOCATION,
+ }
+
+
+class TestDoctorDisplayReport:
+ @pytest.fixture
+ def console(self, mocker: MockerFixture) -> Console:
+ recorded_console = Console(width=200, record=True, color_system=None)
+ mocker.patch("pipelex.cli.commands.doctor_cmd.get_console", return_value=recorded_console)
+ return recorded_console
+
+ def test_all_healthy_report(self, console: Console) -> None:
+ """A fully healthy system renders the green banner and project-local location."""
+ display_health_report(**_healthy_report_kwargs())
+
+ output = console.export_text()
+ assert "Overall Status: ✅ All systems healthy" in output
+ assert "Using project config: /work/project/.pipelex" in output
+ assert "Project root: /work/project" in output
+ assert "Configuration Files" in output
+ assert "Telemetry Configuration" in output
+ assert "Backend Credentials" in output
+ assert "Models are valid" in output
+ assert "Deck is up to date with pipelex 1.2.0" in output
+ assert "Possible Solutions" not in output
+
+ def test_global_location_rendered(self, console: Console) -> None:
+ """Without a project .pipelex/, the global location line is rendered."""
+ kwargs = _healthy_report_kwargs()
+ kwargs["config_location"] = GLOBAL_LOCATION
+
+ display_health_report(**kwargs)
+
+ output = console.export_text()
+ assert "Using global config: /home/user/.pipelex" in output
+ assert "No project .pipelex/ directory found" in output
+
+ def test_unhealthy_config_suggests_init(self, console: Console) -> None:
+ """Missing config files render the failure row and the init-config solution."""
+ kwargs = _healthy_report_kwargs()
+ kwargs["config_healthy"] = False
+ kwargs["config_message"] = "2 configuration file(s) missing"
+ kwargs["config_missing_count"] = 2
+
+ display_health_report(**kwargs)
+
+ output = console.export_text()
+ assert "Overall Status: ⚠️ Issues Found" in output
+ assert "✗ 2 configuration file(s) missing" in output
+ assert "Possible Solutions" in output
+ assert "pipelex init config" in output
+ assert "pipelex doctor --fix" in output
+
+ def test_credential_details_rendered_per_backend(self, console: Console) -> None:
+ """Backend credential issues list missing and placeholder vars per backend."""
+ kwargs = _healthy_report_kwargs()
+ kwargs["backends_healthy"] = False
+ kwargs["backends_message"] = "2 backend(s) have missing or invalid credentials"
+ kwargs["backend_credential_reports"] = {
+ "openai": BackendCredentialsReport(
+ backend_name="openai",
+ required_vars=["OPENAI_API_KEY"],
+ missing_vars=["OPENAI_API_KEY"],
+ placeholder_vars=[],
+ all_credentials_valid=False,
+ ),
+ "mistral": BackendCredentialsReport(
+ backend_name="mistral",
+ required_vars=["MISTRAL_API_KEY"],
+ missing_vars=[],
+ placeholder_vars=["MISTRAL_API_KEY"],
+ all_credentials_valid=False,
+ ),
+ "anthropic": BackendCredentialsReport(
+ backend_name="anthropic",
+ required_vars=["ANTHROPIC_API_KEY"],
+ missing_vars=[],
+ placeholder_vars=[],
+ all_credentials_valid=True,
+ ),
+ }
+
+ display_health_report(**kwargs)
+
+ output = console.export_text()
+ assert "Missing: OPENAI_API_KEY" in output
+ assert "Placeholders: MISTRAL_API_KEY" in output
+ assert "All credentials set" in output
+ assert "Set the following environment variables:" in output
+ assert "Replace placeholder values for:" in output
+
+ def test_models_skipped_advisory(self, console: Console) -> None:
+ """A skipped models check renders as an advisory, not a failure."""
+ kwargs = _healthy_report_kwargs()
+ kwargs["models_healthy"] = False
+ kwargs["models_message"] = "skipped — fix configuration errors first"
+ kwargs["models_skipped"] = True
+
+ display_health_report(**kwargs)
+
+ output = console.export_text()
+ assert "Models check deferred until config errors are fixed." in output
+
+ def test_backend_file_issue_details(self, console: Console) -> None:
+ """Invalid backend files show kit-template vs custom-backend guidance."""
+ kwargs = _healthy_report_kwargs()
+ kwargs["models_healthy"] = False
+ kwargs["models_message"] = "Backend configuration error: openai: bad spec"
+ kwargs["backend_file_reports"] = {
+ "openai": BackendFileReport(
+ backend_name="openai",
+ file_path="/cfg/inference/backends/openai.toml",
+ is_valid=False,
+ error_message="openai: bad spec\nmore detail",
+ has_kit_template=True,
+ ),
+ "custom_llm": BackendFileReport(
+ backend_name="custom_llm",
+ file_path="/cfg/inference/backends/custom_llm.toml",
+ is_valid=False,
+ error_message="custom_llm: unknown field",
+ has_kit_template=False,
+ ),
+ }
+
+ display_health_report(**kwargs)
+
+ output = console.export_text()
+ assert "Backend configuration format may be outdated" in output
+ assert "Template available for replacement" in output
+ assert "This appears to be a custom backend - manual fix required" in output
+ assert "openai: bad spec" in output
+ assert "more detail" not in output
+ assert "pipelex doctor --fix" in output
+ assert "Manually fix backend configuration in" in output
+ assert "custom_llm.toml" in output
+
+ def test_deck_drift_per_file_detail(self, console: Console) -> None:
+ """Deck drift lists each actionable file with its status label."""
+ kwargs = _healthy_report_kwargs()
+ kwargs["deck_healthy"] = False
+ kwargs["deck_message"] = "1 deck file(s) need action"
+ kwargs["deck_report"] = DeckSyncReport(
+ kit_version="1.2.0",
+ installed_kit_version="1.2.0",
+ manifest_present=True,
+ files={"deck.toml": DeckFileStatus.LOCALLY_MODIFIED, "ok.toml": DeckFileStatus.UP_TO_DATE},
+ )
+
+ display_health_report(**kwargs)
+
+ output = console.export_text()
+ assert "1 deck file(s) need action" in output
+ assert "deck.toml" in output
+ assert "pipelex update" in output
+
+ def test_fix_mode_announces_interactive_fixes(self, console: Console) -> None:
+ """In fix mode, auto-fixable backend issues announce the upcoming prompts."""
+ kwargs = _healthy_report_kwargs()
+ kwargs["models_healthy"] = False
+ kwargs["models_message"] = "Backend configuration error"
+ kwargs["backend_file_reports"] = {
+ "openai": BackendFileReport(
+ backend_name="openai",
+ file_path="/cfg/inference/backends/openai.toml",
+ is_valid=False,
+ has_kit_template=True,
+ ),
+ }
+ kwargs["fix_mode"] = True
+
+ display_health_report(**kwargs)
+
+ output = console.export_text()
+ assert "Interactive fixes for outdated backend configurations will be offered below" in output
+ assert "Run pipelex doctor --fix" not in output
+
+ def test_doctor_cmd_catches_unexpected_errors(self, console: Console, mocker: MockerFixture) -> None:
+ """The doctor entry point converts unexpected errors into a friendly exit 1."""
+ mocker.patch("pipelex.cli.commands.doctor_cmd.do_doctor_cmd", side_effect=RuntimeError("totally unexpected"))
+
+ with pytest.raises(SystemExit) as exc_info:
+ doctor_cmd()
+
+ assert exc_info.value.code == 1
+ output = console.export_text()
+ assert "✗ Unexpected error: totally unexpected" in output
+ assert "https://go.pipelex.com/discord" in output
diff --git a/tests/unit/pipelex/cli/test_doctor_fix_mode.py b/tests/unit/pipelex/cli/test_doctor_fix_mode.py
new file mode 100644
index 000000000..97a38c1c1
--- /dev/null
+++ b/tests/unit/pipelex/cli/test_doctor_fix_mode.py
@@ -0,0 +1,162 @@
+"""Unit tests for doctor's interactive fix mode and manual-fix guidance in do_doctor_cmd."""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any
+
+import pytest
+from rich.console import Console
+
+from pipelex.cli.commands.doctor_cmd import BackendFileReport, do_doctor_cmd
+from pipelex.cli.commands.init.ui.types import InitFocus
+from pipelex.cogt.model_backends.backend_library import BackendCredentialsReport
+from pipelex.cogt.models.deck_manifest import DeckFileStatus, DeckSyncReport
+
+if TYPE_CHECKING:
+ from pathlib import Path
+
+ from pytest_mock import MockerFixture
+
+CLEAN_DECK = DeckSyncReport(kit_version="1.2.0", installed_kit_version="1.2.0", manifest_present=True, files={})
+
+
+class TestDoctorFixMode:
+ @pytest.fixture
+ def doctor_mocks(self, mocker: MockerFixture) -> dict[str, Any]:
+ """Stub every check healthy plus the display/bootstrap; tests flip what they need."""
+ mocks: dict[str, Any] = {
+ "setup": mocker.patch("pipelex.cli.commands.doctor_cmd.setup_doctor_runtime"),
+ "config": mocker.patch("pipelex.cli.commands.doctor_cmd.check_config_files", return_value=(True, 0, "OK")),
+ "telemetry": mocker.patch("pipelex.cli.commands.doctor_cmd.check_telemetry_config", return_value=(True, "OK")),
+ "backends": mocker.patch("pipelex.cli.commands.doctor_cmd.check_backend_credentials", return_value=(True, {}, "OK")),
+ "models": mocker.patch("pipelex.cli.commands.doctor_cmd.check_models", return_value=(True, "OK", {})),
+ "deck": mocker.patch("pipelex.cli.commands.doctor_cmd.check_deck_sync", return_value=(True, CLEAN_DECK, "OK")),
+ "display": mocker.patch("pipelex.cli.commands.doctor_cmd.display_health_report"),
+ "init_cmd": mocker.patch("pipelex.cli.commands.doctor_cmd.init_cmd"),
+ "update_cmd": mocker.patch("pipelex.cli.commands.doctor_cmd.update_cmd"),
+ "replace": mocker.patch("pipelex.cli.commands.doctor_cmd.replace_backend_file", return_value=True),
+ "confirm": mocker.patch("pipelex.cli.commands.doctor_cmd.Confirm.ask", return_value=True),
+ }
+ recorded_console = Console(width=200, record=True, color_system=None)
+ mocker.patch("pipelex.cli.commands.doctor_cmd.get_console", return_value=recorded_console)
+ mocks["console"] = recorded_console
+ return mocks
+
+ def _run_doctor_expecting_exit_one(self) -> None:
+ with pytest.raises(SystemExit) as exc_info:
+ do_doctor_cmd(fix=True)
+ assert exc_info.value.code == 1
+
+ def test_fix_missing_config_files_runs_init_config(self, doctor_mocks: dict[str, Any]) -> None:
+ """Accepting the config fix runs init focused on config files."""
+ doctor_mocks["config"].return_value = (False, 2, "2 configuration file(s) missing")
+
+ self._run_doctor_expecting_exit_one()
+
+ doctor_mocks["init_cmd"].assert_called_once_with(focus=InitFocus.CONFIG, skip_confirmation=True)
+ output = doctor_mocks["console"].export_text()
+ assert "Interactive Fix Mode" in output
+ assert "Configuration files installed" in output
+
+ def test_fix_telemetry_format_change_runs_init_telemetry(self, doctor_mocks: dict[str, Any]) -> None:
+ """Accepting the telemetry fix runs init focused on telemetry."""
+ doctor_mocks["telemetry"].return_value = (False, "Config format has changed - run 'pipelex init telemetry' to update")
+
+ self._run_doctor_expecting_exit_one()
+
+ doctor_mocks["init_cmd"].assert_called_once_with(focus=InitFocus.TELEMETRY, skip_confirmation=True)
+ assert "Telemetry configured" in doctor_mocks["console"].export_text()
+
+ def test_fix_outdated_deck_runs_update(self, doctor_mocks: dict[str, Any]) -> None:
+ """Accepting the deck fix runs `pipelex update --yes`."""
+ dirty_deck = DeckSyncReport(
+ kit_version="1.3.0",
+ installed_kit_version="1.2.0",
+ manifest_present=True,
+ files={"deck.toml": DeckFileStatus.CLEAN_BEHIND},
+ )
+ doctor_mocks["deck"].return_value = (False, dirty_deck, "deck out of date")
+
+ self._run_doctor_expecting_exit_one()
+
+ doctor_mocks["update_cmd"].assert_called_once_with(yes=True)
+ assert "Model deck updated" in doctor_mocks["console"].export_text()
+
+ def _make_invalid_backend_report(self, tmp_path: Path) -> BackendFileReport:
+ return BackendFileReport(
+ backend_name="openai",
+ file_path=str(tmp_path / "inference" / "backends" / "openai.toml"),
+ is_valid=False,
+ error_message="openai: bad spec",
+ has_kit_template=True,
+ )
+
+ def test_fix_backend_file_replacement_accepted(self, doctor_mocks: dict[str, Any], tmp_path: Path) -> None:
+ """Accepting the backend replace copies the kit template into the right config dir."""
+ bad_report = self._make_invalid_backend_report(tmp_path)
+ doctor_mocks["models"].return_value = (False, "Backend configuration error: openai: bad spec", {"openai": bad_report})
+
+ self._run_doctor_expecting_exit_one()
+
+ doctor_mocks["replace"].assert_called_once_with("openai", dry_run=False, config_dir=tmp_path)
+ assert "Replaced openai backend configuration" in doctor_mocks["console"].export_text()
+
+ def test_fix_backend_file_replacement_declined(self, doctor_mocks: dict[str, Any], tmp_path: Path) -> None:
+ """Declining the backend replace leaves the file alone and reports the skip."""
+ bad_report = self._make_invalid_backend_report(tmp_path)
+ doctor_mocks["models"].return_value = (False, "Backend configuration error: openai: bad spec", {"openai": bad_report})
+ doctor_mocks["confirm"].return_value = False
+
+ self._run_doctor_expecting_exit_one()
+
+ doctor_mocks["replace"].assert_not_called()
+ assert "Skipped openai" in doctor_mocks["console"].export_text()
+
+ def test_fix_backend_replacement_failure_reported(self, doctor_mocks: dict[str, Any], tmp_path: Path) -> None:
+ """A failed template copy is reported without aborting the doctor run."""
+ bad_report = self._make_invalid_backend_report(tmp_path)
+ doctor_mocks["models"].return_value = (False, "Backend configuration error: openai: bad spec", {"openai": bad_report})
+ doctor_mocks["replace"].return_value = False
+
+ self._run_doctor_expecting_exit_one()
+
+ assert "Failed to replace openai" in doctor_mocks["console"].export_text()
+
+ def test_manual_fixes_for_config_validation_error(self, doctor_mocks: dict[str, Any]) -> None:
+ """A config validation error (nothing missing) lands in the manual-fixes section."""
+ doctor_mocks["config"].return_value = (False, 0, "Configuration validation failed: bad field")
+
+ self._run_doctor_expecting_exit_one()
+
+ doctor_mocks["init_cmd"].assert_not_called()
+ output = doctor_mocks["console"].export_text()
+ assert "Manual Fixes Required" in output
+ assert "Configuration validation error:" in output
+ assert "pipelex init config" in output
+
+ def test_manual_fixes_for_missing_credentials(self, doctor_mocks: dict[str, Any]) -> None:
+ """Missing backend credentials produce env-var setup guidance for every platform."""
+ credentials_report = BackendCredentialsReport(
+ backend_name="openai",
+ required_vars=["OPENAI_API_KEY"],
+ missing_vars=["OPENAI_API_KEY"],
+ placeholder_vars=[],
+ all_credentials_valid=False,
+ )
+ doctor_mocks["backends"].return_value = (False, {"openai": credentials_report}, "1 backend(s) have missing or invalid credentials")
+
+ self._run_doctor_expecting_exit_one()
+
+ output = doctor_mocks["console"].export_text()
+ assert "Manual Fixes Required" in output
+ assert "Backend credentials:" in output
+ assert "OPENAI_API_KEY=your_value_here" in output
+ assert "export OPENAI_API_KEY=your_value_here" in output
+
+ def test_fix_mode_with_all_healthy_exits_zero(self, doctor_mocks: dict[str, Any]) -> None:
+ """With everything healthy, fix mode exits 0 without prompting."""
+ with pytest.raises(SystemExit) as exc_info:
+ do_doctor_cmd(fix=True)
+
+ assert exc_info.value.code == 0
+ doctor_mocks["confirm"].assert_not_called()
diff --git a/tests/unit/pipelex/cli/test_error_handlers_more.py b/tests/unit/pipelex/cli/test_error_handlers_more.py
new file mode 100644
index 000000000..3df19de42
--- /dev/null
+++ b/tests/unit/pipelex/cli/test_error_handlers_more.py
@@ -0,0 +1,331 @@
+"""Unit tests for the CLI error handlers not covered by test_error_handlers.py.
+
+Covers the shared panel renderer, the gateway/telemetry/signature handlers, and the
+detailed sections of bundle-validation error display. Assertions are on the recorded
+plain-text console output plus the exit code.
+"""
+
+from __future__ import annotations
+
+from pathlib import Path
+from typing import TYPE_CHECKING
+
+import pytest
+import typer
+from rich.console import Console
+
+from pipelex.cli.error_handlers import (
+ ErrorContext,
+ display_error_panel,
+ handle_gateway_api_key_missing_error,
+ handle_gateway_do_not_track_conflict_error,
+ handle_gateway_terms_not_accepted_error,
+ handle_gateway_unknown_model_error,
+ handle_model_deck_preset_error,
+ handle_remote_config_unavailable_error,
+ handle_remote_config_validation_error,
+ handle_signatures_not_allowed_error,
+ handle_telemetry_config_validation_error,
+ handle_validate_bundle_error,
+)
+from pipelex.cogt.exceptions import GatewayUnknownModelError, ModelDeckPresetValidatonError
+from pipelex.cogt.inference.error_classification import UserAction, UserActionKind
+from pipelex.cogt.model_backends.model_type import ModelType
+from pipelex.core.bundles.exceptions import PipelexBundleBlueprintValidationErrorData
+from pipelex.core.exceptions import PipesAndConceptValidationErrorData
+from pipelex.core.pipes.exceptions import PipeValidationErrorType
+from pipelex.pipe_signature.exceptions import SignaturesNotAllowedError
+from pipelex.pipeline.exceptions import ValidateBundleError
+from pipelex.system.pipelex_service.exceptions import (
+ GatewayApiKeyMissingError,
+ GatewayDoNotTrackConflictError,
+ GatewayTermsNotAcceptedError,
+ RemoteConfigUnavailableError,
+ RemoteConfigValidationError,
+)
+from pipelex.system.pipelex_service.types import RemoteConfigSource
+from pipelex.system.telemetry.exceptions import TelemetryConfigValidationError
+
+if TYPE_CHECKING:
+ from pytest_mock import MockerFixture
+
+
+class TestErrorHandlersExtended:
+ @pytest.fixture
+ def console(self, mocker: MockerFixture) -> Console:
+ """Recorded console patched into the error handlers module."""
+ recorded_console = Console(width=120, record=True, color_system=None)
+ mocker.patch("pipelex.cli.error_handlers.get_console", return_value=recorded_console)
+ return recorded_console
+
+ def test_display_error_panel_full_layout(self) -> None:
+ """The panel renders title, aligned fields, error, tip and links in order."""
+ console = Console(width=120, record=True, color_system=None)
+ display_error_panel(
+ console,
+ title="Something failed",
+ fields=[("Pipe", "'my_pipe'"), ("Model Choice", "'gpt-5'")],
+ error_message="model not found",
+ tip="try another model",
+ links=[("Docs", "https://docs.example.com")],
+ )
+ output = console.export_text()
+ assert "❌ Something failed" in output
+ assert "Pipe: 'my_pipe'" in output
+ assert "Model Choice: 'gpt-5'" in output
+ assert "Error: model not found" in output
+ assert "💡 Tip: try another model" in output
+ assert "Docs: https://docs.example.com" in output
+
+ def test_display_error_panel_omits_error_block_when_none(self) -> None:
+ """A None error_message omits the Error block entirely."""
+ console = Console(width=120, record=True, color_system=None)
+ display_error_panel(
+ console,
+ title="Soft failure",
+ fields=[],
+ error_message=None,
+ tip="a tip",
+ links=[],
+ )
+ output = console.export_text()
+ assert "Error:" not in output
+ assert "💡 Tip: a tip" in output
+
+ def test_display_error_panel_multi_line_tip(self) -> None:
+ """Multi-line tips keep their line structure."""
+ console = Console(width=120, record=True, color_system=None)
+ display_error_panel(
+ console,
+ title="Failure",
+ fields=[],
+ error_message=None,
+ tip="first line\nsecond line",
+ links=[],
+ )
+ output = console.export_text()
+ assert "first line" in output
+ assert "second line" in output
+
+ def _make_deck_preset_error(self, enabled_backends: set[str] | None) -> ModelDeckPresetValidatonError:
+ return ModelDeckPresetValidatonError(
+ message="preset references unavailable model",
+ model_type=ModelType.LLM,
+ preset_id="default_text",
+ model_handle="gpt-5",
+ enabled_backends=enabled_backends,
+ )
+
+ def test_handle_model_deck_preset_error_builds_solutions_tip(self, console: Console) -> None:
+ """Without user-action advice, the handler builds the 'Possible solutions' tip."""
+ exc = self._make_deck_preset_error(enabled_backends={"openai", "anthropic"})
+
+ with pytest.raises(typer.Exit) as exc_info:
+ handle_model_deck_preset_error(exc, context=ErrorContext.VALIDATION)
+
+ assert exc_info.value.exit_code == 1
+ output = console.export_text()
+ assert "Preset ID:" in output
+ assert "'default_text'" in output
+ assert "Enabled Backends:" in output
+ assert "anthropic, openai" in output
+ assert "Possible solutions:" in output
+ assert "Configure model 'gpt-5' in one of your enabled backends" in output
+
+ def test_handle_model_deck_preset_error_uses_user_action_detail(self, console: Console) -> None:
+ """When the error carries user-action advice, it becomes the tip verbatim."""
+ exc = self._make_deck_preset_error(enabled_backends=None)
+ exc.user_action = UserAction(kind=UserActionKind.CHANGE_MODEL, detail="switch to gpt-4o in the preset")
+
+ with pytest.raises(typer.Exit):
+ handle_model_deck_preset_error(exc, context=ErrorContext.VALIDATION)
+
+ output = console.export_text()
+ assert "switch to gpt-4o in the preset" in output
+ assert "Possible solutions:" not in output
+ assert "Enabled Backends:" not in output
+
+ def test_handle_validate_bundle_error_renders_all_sections(self, console: Console) -> None:
+ """Blueprint, pipe, dry-run and signature sections all render with their details."""
+ blueprint_error = PipelexBundleBlueprintValidationErrorData(
+ error_type=PipeValidationErrorType.MISSING_INPUT_VARIABLE,
+ domain_code="demo",
+ source="bundle.mthds",
+ pipe_code="my_pipe",
+ message="input variable 'topic' is missing",
+ variable_names=["topic"],
+ )
+ pipe_error = PipesAndConceptValidationErrorData(
+ domain_code="demo",
+ pipe_code="other_pipe",
+ concept_code="Report",
+ field_name="output",
+ error_type=PipeValidationErrorType.INADEQUATE_OUTPUT_CONCEPT,
+ message="output concept mismatch",
+ field_path="pipes.other_pipe.output",
+ variable_names=["report"],
+ )
+ signature_error = SignaturesNotAllowedError(
+ offending_pipe_refs={"stub_pipe"},
+ signature_refs={"stub_pipe"},
+ dep_paths={"stub_pipe": ["main_pipe", "stub_pipe"]},
+ )
+ exc = ValidateBundleError(
+ message="validation failed",
+ pipelex_bundle_blueprint_validation_errors=[blueprint_error],
+ pipe_validation_errors=[pipe_error],
+ dry_run_error_message="dry run exploded",
+ signature_check_error=signature_error,
+ )
+
+ with pytest.raises(typer.Exit) as exc_info:
+ handle_validate_bundle_error(exc, bundle_path=Path("methods/demo.mthds"))
+
+ assert exc_info.value.exit_code == 1
+ output = console.export_text()
+ assert "❌ Bundle validation failed" in output
+ assert "Bundle: methods/demo.mthds" in output
+ assert "Blueprint Validation Errors:" in output
+ assert "Missing Input Variable" in output
+ assert "input variable 'topic' is missing" in output
+ assert "└─ Source: bundle.mthds" in output
+ assert "Pipe Validation Errors:" in output
+ assert "Inadequate Output Concept" in output
+ assert "Field: output" in output
+ assert "└─ Path: pipes.other_pipe.output" in output
+ assert "Dry Run Error:" in output
+ assert "dry run exploded" in output
+ assert "Unimplemented Signatures:" in output
+
+ def test_handle_validate_bundle_error_minimal_skips_sections(self, console: Console) -> None:
+ """With no detail lists and no bundle path, only the banner and tip render."""
+ exc = ValidateBundleError(message="validation failed")
+
+ with pytest.raises(typer.Exit):
+ handle_validate_bundle_error(exc)
+
+ output = console.export_text()
+ assert "❌ Bundle validation failed" in output
+ assert "Bundle:" not in output
+ assert "Blueprint Validation Errors:" not in output
+ assert "Pipe Validation Errors:" not in output
+ assert "Dry Run Error:" not in output
+ assert "Unimplemented Signatures:" not in output
+ assert "💡 Tip:" in output
+
+ def test_handle_signatures_not_allowed_error(self, console: Console) -> None:
+ """The signature handler explains the strict-mode refusal and the opt-out flag."""
+ exc = SignaturesNotAllowedError(
+ offending_pipe_refs={"stub_pipe"},
+ signature_refs={"stub_pipe"},
+ dep_paths={"stub_pipe": ["main_pipe", "stub_pipe"]},
+ )
+
+ with pytest.raises(typer.Exit) as exc_info:
+ handle_signatures_not_allowed_error(exc, context=ErrorContext.VALIDATION)
+
+ assert exc_info.value.exit_code == 1
+ output = console.export_text()
+ assert "unimplemented PipeSignature placeholders" in output
+ assert "--allow-signatures" in output
+
+ def test_handle_telemetry_config_validation_error(self, console: Console) -> None:
+ """The telemetry handler explains the format migration."""
+ exc = TelemetryConfigValidationError("old flat format")
+
+ with pytest.raises(typer.Exit) as exc_info:
+ handle_telemetry_config_validation_error(exc)
+
+ assert exc_info.value.exit_code == 1
+ output = console.export_text()
+ assert "Telemetry configuration format has changed" in output
+ assert "pipelex init telemetry" in output
+
+ def test_handle_gateway_terms_not_accepted_error(self, console: Console) -> None:
+ """The terms handler points at init config and the BYOK alternative."""
+ with pytest.raises(typer.Exit) as exc_info:
+ handle_gateway_terms_not_accepted_error(GatewayTermsNotAcceptedError())
+
+ assert exc_info.value.exit_code == 1
+ output = console.export_text()
+ assert "Pipelex Gateway terms not accepted" in output
+ assert "pipelex init config" in output
+ assert "Disable pipelex_gateway" in output
+
+ def test_handle_gateway_api_key_missing_error(self, console: Console) -> None:
+ """The API-key handler names the env var to set."""
+ with pytest.raises(typer.Exit) as exc_info:
+ handle_gateway_api_key_missing_error(GatewayApiKeyMissingError())
+
+ assert exc_info.value.exit_code == 1
+ output = console.export_text()
+ assert "Pipelex Gateway API key not set" in output
+ assert "PIPELEX_GATEWAY_API_KEY" in output
+
+ def test_handle_gateway_do_not_track_conflict_error(self, console: Console) -> None:
+ """The DNT handler offers both resolution options."""
+ with pytest.raises(typer.Exit) as exc_info:
+ handle_gateway_do_not_track_conflict_error(GatewayDoNotTrackConflictError(dnt_env_var="DO_NOT_TRACK"))
+
+ assert exc_info.value.exit_code == 1
+ output = console.export_text()
+ assert "Pipelex Gateway requires telemetry" in output
+ assert "Unset" in output
+ assert "disable pipelex_gateway" in output
+
+ def test_handle_remote_config_validation_error(self, console: Console) -> None:
+ """A malformed gateway config is flagged as a server-side bug to report."""
+ with pytest.raises(typer.Exit) as exc_info:
+ handle_remote_config_validation_error(RemoteConfigValidationError("missing key 'models'"))
+
+ assert exc_info.value.exit_code == 1
+ output = console.export_text()
+ assert "Pipelex Gateway configuration is invalid" in output
+ assert "missing key 'models'" in output
+ assert "Please report this!" in output
+
+ def test_handle_remote_config_unavailable_error(self, console: Console) -> None:
+ """Offline with a cold cache yields reconnect-or-BYOK guidance."""
+ with pytest.raises(typer.Exit) as exc_info:
+ handle_remote_config_unavailable_error(RemoteConfigUnavailableError("no network, no cache"))
+
+ assert exc_info.value.exit_code == 1
+ output = console.export_text()
+ assert "unreachable and no cached config" in output
+ assert "no network, no cache" in output
+ assert "pipelex init" in output
+
+ @pytest.mark.parametrize(
+ ("config_source", "expected_phrase"),
+ [
+ (RemoteConfigSource.FRESH, "Check the model handle for typos"),
+ (RemoteConfigSource.CACHED, "cache may be stale"),
+ ],
+ )
+ def test_handle_gateway_unknown_model_error_branches_on_source(
+ self,
+ console: Console,
+ config_source: RemoteConfigSource,
+ expected_phrase: str,
+ ) -> None:
+ """The unknown-model handler branches its remediation on config provenance."""
+ exc = GatewayUnknownModelError(model_name="mystery-model", source=config_source)
+
+ with pytest.raises(typer.Exit) as exc_info:
+ handle_gateway_unknown_model_error(exc)
+
+ assert exc_info.value.exit_code == 1
+ output = console.export_text()
+ assert "Unknown gateway model handle" in output
+ assert "'mystery-model'" in output
+ assert expected_phrase in output
+
+ def test_handlers_escape_rich_markup_in_messages(self, console: Console) -> None:
+ """Square brackets in exception text must render literally, not as markup."""
+ exc = RemoteConfigValidationError("payload has [red]markup[/red] inside")
+
+ with pytest.raises(typer.Exit):
+ handle_remote_config_validation_error(exc)
+
+ output = console.export_text()
+ assert "payload has [red]markup[/red] inside" in output
diff --git a/tests/unit/pipelex/cli/test_readiness.py b/tests/unit/pipelex/cli/test_readiness.py
new file mode 100644
index 000000000..880f95841
--- /dev/null
+++ b/tests/unit/pipelex/cli/test_readiness.py
@@ -0,0 +1,164 @@
+"""Unit tests for the CLI readiness gate (pipelex/cli/readiness.py)."""
+
+from __future__ import annotations
+
+import sys
+from pathlib import Path
+from typing import TYPE_CHECKING
+
+import pytest
+from rich.console import Console
+
+import pipelex
+from pipelex.cli.exceptions import ReadinessCheckError
+from pipelex.cli.readiness import (
+ _find_venv_directories, # noqa: PLC2701 # pyright: ignore[reportPrivateUsage]
+ _is_development_install, # noqa: PLC2701 # pyright: ignore[reportPrivateUsage]
+ _is_in_virtual_environment, # noqa: PLC2701 # pyright: ignore[reportPrivateUsage]
+ check_readiness,
+)
+
+if TYPE_CHECKING:
+ from pytest_mock import MockerFixture
+
+
+def _make_recorded_console() -> Console:
+ return Console(width=100, record=True, color_system=None)
+
+
+class TestReadiness:
+ @pytest.fixture
+ def no_env_markers(self, monkeypatch: pytest.MonkeyPatch) -> None:
+ """Strip every signal that we are inside a virtual environment."""
+ monkeypatch.delenv("CONDA_DEFAULT_ENV", raising=False)
+ monkeypatch.delenv("VIRTUAL_ENV", raising=False)
+ monkeypatch.setattr(sys, "prefix", "/usr")
+ monkeypatch.setattr(sys, "base_prefix", "/usr")
+
+ @pytest.mark.usefixtures("no_env_markers")
+ def test_is_in_virtual_environment_via_sys_prefix(self, monkeypatch: pytest.MonkeyPatch) -> None:
+ """A venv is detected when sys.prefix differs from sys.base_prefix."""
+ monkeypatch.setattr(sys, "prefix", "/some/project/.venv")
+ assert _is_in_virtual_environment() is True
+
+ @pytest.mark.usefixtures("no_env_markers")
+ def test_is_in_virtual_environment_via_conda(self, monkeypatch: pytest.MonkeyPatch) -> None:
+ """A conda env is detected via CONDA_DEFAULT_ENV."""
+ monkeypatch.setenv("CONDA_DEFAULT_ENV", "base")
+ assert _is_in_virtual_environment() is True
+
+ @pytest.mark.usefixtures("no_env_markers")
+ def test_is_in_virtual_environment_via_virtual_env_var(self, monkeypatch: pytest.MonkeyPatch) -> None:
+ """A virtualenv is detected via VIRTUAL_ENV."""
+ monkeypatch.setenv("VIRTUAL_ENV", "/some/project/.venv")
+ assert _is_in_virtual_environment() is True
+
+ @pytest.mark.usefixtures("no_env_markers")
+ def test_is_in_virtual_environment_false_without_markers(self) -> None:
+ """No markers at all means no virtual environment."""
+ assert _is_in_virtual_environment() is False
+
+ def test_find_venv_directories_in_cwd(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ """A .venv with bin/python in the cwd is found."""
+ venv_python = tmp_path / ".venv" / "bin" / "python"
+ venv_python.parent.mkdir(parents=True)
+ venv_python.touch()
+ monkeypatch.chdir(tmp_path)
+ assert _find_venv_directories() == [".venv"]
+
+ def test_find_venv_directories_in_parent(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ """A venv two levels up (within the 3-level walk) is found."""
+ venv_python = tmp_path / "venv" / "bin" / "python"
+ venv_python.parent.mkdir(parents=True)
+ venv_python.touch()
+ nested_dir = tmp_path / "sub" / "deeper"
+ nested_dir.mkdir(parents=True)
+ monkeypatch.chdir(nested_dir)
+ assert _find_venv_directories() == ["venv"]
+
+ def test_find_venv_directories_ignores_dir_without_python(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ """A venv-named directory without bin/python is not reported."""
+ (tmp_path / ".venv").mkdir()
+ monkeypatch.chdir(tmp_path)
+ assert _find_venv_directories() == []
+
+ def test_find_venv_directories_empty(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ """No venv directories at all yields an empty list."""
+ monkeypatch.chdir(tmp_path)
+ assert _find_venv_directories() == []
+
+ def test_is_development_install_with_git_dir(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ """A .git directory above the pipelex package marks a dev install."""
+ repo_root = tmp_path / "repo"
+ package_init = repo_root / "pipelex" / "__init__.py"
+ package_init.parent.mkdir(parents=True)
+ package_init.touch()
+ (repo_root / ".git").mkdir()
+ monkeypatch.setattr(pipelex, "__file__", str(package_init))
+ assert _is_development_install() is True
+
+ def test_is_development_install_without_git_dir(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
+ """No .git in the parent chain means a production install."""
+ package_init = tmp_path / "site-packages" / "pipelex" / "__init__.py"
+ package_init.parent.mkdir(parents=True)
+ package_init.touch()
+ monkeypatch.setattr(pipelex, "__file__", str(package_init))
+ assert _is_development_install() is False
+
+ def test_is_development_install_handles_missing_module_file(self, monkeypatch: pytest.MonkeyPatch) -> None:
+ """A missing pipelex.__file__ attribute degrades to 'not a dev install'."""
+ monkeypatch.delattr(pipelex, "__file__")
+ assert _is_development_install() is False
+
+ def test_is_development_install_handles_os_error(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch, mocker: MockerFixture) -> None:
+ """An OSError during path probing degrades to 'not a dev install'."""
+ package_init = tmp_path / "pipelex" / "__init__.py"
+ package_init.parent.mkdir(parents=True)
+ package_init.touch()
+ monkeypatch.setattr(pipelex, "__file__", str(package_init))
+ mocker.patch.object(Path, "exists", side_effect=OSError("probe failed"))
+ assert _is_development_install() is False
+
+ def test_check_readiness_passes_for_dev_install_in_venv(self, mocker: MockerFixture) -> None:
+ """Dev install with an active venv passes silently."""
+ mocker.patch("pipelex.cli.readiness._is_development_install", return_value=True)
+ mocker.patch("pipelex.cli.readiness._is_in_virtual_environment", return_value=True)
+ check_readiness()
+
+ def test_check_readiness_passes_for_production_install(self, mocker: MockerFixture) -> None:
+ """Production installs are never gated on venv activation."""
+ mocker.patch("pipelex.cli.readiness._is_development_install", return_value=False)
+ mocker.patch("pipelex.cli.readiness._is_in_virtual_environment", return_value=False)
+ check_readiness()
+
+ def test_check_readiness_raises_with_activation_instructions(self, mocker: MockerFixture) -> None:
+ """Dev install without venv but with discoverable venvs suggests activation."""
+ mocker.patch("pipelex.cli.readiness._is_development_install", return_value=True)
+ mocker.patch("pipelex.cli.readiness._is_in_virtual_environment", return_value=False)
+ mocker.patch("pipelex.cli.readiness._find_venv_directories", return_value=[".venv", "venv"])
+ console = _make_recorded_console()
+ mocker.patch("pipelex.cli.readiness.get_console", return_value=console)
+
+ with pytest.raises(ReadinessCheckError, match="no virtual environment is active"):
+ check_readiness()
+
+ output = console.export_text()
+ assert "Virtual Environment Required" in output
+ assert "Found virtual environment(s) in your project:" in output
+ assert ".venv" in output
+ assert "source .venv/bin/activate" in output
+
+ def test_check_readiness_raises_with_creation_instructions(self, mocker: MockerFixture) -> None:
+ """Dev install without venv and none found suggests creating one."""
+ mocker.patch("pipelex.cli.readiness._is_development_install", return_value=True)
+ mocker.patch("pipelex.cli.readiness._is_in_virtual_environment", return_value=False)
+ mocker.patch("pipelex.cli.readiness._find_venv_directories", return_value=[])
+ console = _make_recorded_console()
+ mocker.patch("pipelex.cli.readiness.get_console", return_value=console)
+
+ with pytest.raises(ReadinessCheckError):
+ check_readiness()
+
+ output = console.export_text()
+ assert "To create and activate a virtual environment:" in output
+ assert "python -m venv .venv" in output
diff --git a/tests/unit/pipelex/cli/test_run_core_execution.py b/tests/unit/pipelex/cli/test_run_core_execution.py
new file mode 100644
index 000000000..6fa6fdc01
--- /dev/null
+++ b/tests/unit/pipelex/cli/test_run_core_execution.py
@@ -0,0 +1,337 @@
+"""Unit tests for the core `pipelex run` execution logic (_execute_run).
+
+The runner (PipelexMTHDSProtocol) and the rendering collaborators are mocked at the
+module namespace; file outputs are asserted on real tmp_path files. The traceback
+behavior of the inner except blocks is covered by test_traceback_flag_run_core.py.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import json
+from types import SimpleNamespace
+from typing import TYPE_CHECKING, Any
+
+import pytest
+import typer
+from rich.console import Console
+
+from pipelex.cli.commands.run._run_core import _execute_run # noqa: PLC2701 # pyright: ignore[reportPrivateUsage]
+from pipelex.core.concepts.exceptions import ConceptValueError
+from pipelex.core.stuffs.list_content import ListContent
+from pipelex.core.stuffs.text_content import TextContent
+
+if TYPE_CHECKING:
+ from collections.abc import Coroutine
+ from pathlib import Path
+
+ from pytest_mock import MockerFixture
+
+
+def _run_async(coro: Coroutine[Any, Any, Any]) -> Any:
+ return asyncio.run(coro)
+
+
+def _call_execute_run(**overrides: Any) -> Coroutine[Any, Any, None]:
+ kwargs: dict[str, Any] = {
+ "pipe_code": "test_pipe",
+ "bundle_path": None,
+ "inputs": None,
+ "save_working_memory": False,
+ "working_memory_path": None,
+ "save_main_stuff": False,
+ "no_pretty_print": True,
+ "graph": None,
+ "graph_full_data": None,
+ "output_dir": "temp/test_outputs",
+ "dry_run": False,
+ "mock_usage": False,
+ "mock_inputs": False,
+ "library_dir": None,
+ }
+ kwargs.update(overrides)
+ return _execute_run(**kwargs)
+
+
+class TestRunCoreExecution:
+ @pytest.fixture
+ def console(self, mocker: MockerFixture) -> Console:
+ recorded_console = Console(width=200, record=True, color_system=None)
+ mocker.patch("pipelex.cli.commands.run._run_core.get_console", return_value=recorded_console)
+ return recorded_console
+
+ @pytest.fixture
+ def config_mock(self, mocker: MockerFixture) -> Any:
+ fake_config = mocker.MagicMock()
+ fake_config.pipelex.pipeline_execution_config.with_execution_overrides.return_value = mocker.MagicMock()
+ mocker.patch("pipelex.cli.commands.run._run_core.get_config", return_value=fake_config)
+ return fake_config
+
+ def _mock_runner(self, mocker: MockerFixture, pipe_output: Any) -> Any:
+ """Patch the protocol runner so execute() resolves to a response with pipe_output."""
+ runner_mock = mocker.MagicMock()
+ runner_mock.execute = mocker.AsyncMock(return_value=SimpleNamespace(pipe_output=pipe_output))
+ runner_class_mock = mocker.patch("pipelex.cli.commands.run._run_core.PipelexMTHDSProtocol", return_value=runner_mock)
+ mocker.patch("pipelex.cli.commands.run._run_core.render_cost_report_for_output")
+ return runner_class_mock
+
+ def _make_pipe_output(
+ self,
+ mocker: MockerFixture,
+ main_stuff: Any | None = None,
+ graph_spec: Any | None = None,
+ ) -> Any:
+ working_memory = mocker.MagicMock()
+ working_memory.get_optional_main_stuff.return_value = main_stuff
+ working_memory.smart_dump.return_value = {"stuff": "dump"}
+ return SimpleNamespace(
+ optional_main_stuff=main_stuff,
+ graph_spec=graph_spec,
+ working_memory=working_memory,
+ )
+
+ @pytest.mark.usefixtures("config_mock")
+ def test_happy_path_prints_recap(self, mocker: MockerFixture, console: Console) -> None:
+ """A successful run prints the completion recap and passes the pipe code to the runner."""
+ pipe_output = self._make_pipe_output(mocker)
+ self._mock_runner(mocker, pipe_output)
+
+ _run_async(_call_execute_run())
+
+ output = console.export_text()
+ assert "Pipeline execution completed successfully" in output
+ assert "Output saved to" not in output
+
+ @pytest.mark.usefixtures("config_mock")
+ def test_dry_run_recap(self, mocker: MockerFixture, console: Console) -> None:
+ """A dry run prints the dry-run recap."""
+ pipe_output = self._make_pipe_output(mocker)
+ self._mock_runner(mocker, pipe_output)
+
+ _run_async(_call_execute_run(dry_run=True))
+
+ assert "Dry run completed successfully" in console.export_text()
+
+ @pytest.mark.usefixtures("config_mock", "console")
+ def test_bundle_main_pipe_used_when_no_pipe_code(self, mocker: MockerFixture, tmp_path: Path) -> None:
+ """Without --pipe, the bundle's main_pipe is resolved and executed."""
+ bundle_file = tmp_path / "demo.mthds"
+ bundle_file.write_text("# bundle content\n", encoding="utf-8")
+ mocker.patch(
+ "pipelex.cli.commands.run._run_core.PipelexInterpreter.make_pipelex_bundle_blueprint",
+ return_value=SimpleNamespace(main_pipe="bundle_main"),
+ )
+ pipe_output = self._make_pipe_output(mocker)
+ runner_class_mock = self._mock_runner(mocker, pipe_output)
+
+ _run_async(_call_execute_run(pipe_code=None, bundle_path=str(bundle_file)))
+
+ execute_kwargs = runner_class_mock.return_value.execute.call_args.kwargs
+ assert execute_kwargs["pipe_code"] == "bundle_main"
+ assert execute_kwargs["mthds_contents"] == ["# bundle content\n"]
+
+ @pytest.mark.usefixtures("config_mock", "console")
+ def test_bundle_without_main_pipe_exits(self, mocker: MockerFixture, tmp_path: Path) -> None:
+ """A bundle with no main_pipe and no --pipe is an error."""
+ bundle_file = tmp_path / "demo.mthds"
+ bundle_file.write_text("# bundle content\n", encoding="utf-8")
+ mocker.patch(
+ "pipelex.cli.commands.run._run_core.PipelexInterpreter.make_pipelex_bundle_blueprint",
+ return_value=SimpleNamespace(main_pipe=None),
+ )
+
+ with pytest.raises(typer.Exit) as exc_info:
+ _run_async(_call_execute_run(pipe_code=None, bundle_path=str(bundle_file)))
+
+ assert exc_info.value.exit_code == 1
+
+ def test_no_pipe_code_and_no_bundle_exits(self) -> None:
+ """Neither a pipe code nor a bundle is an immediate error."""
+ with pytest.raises(typer.Exit) as exc_info:
+ _run_async(_call_execute_run(pipe_code=None, bundle_path=None))
+
+ assert exc_info.value.exit_code == 1
+
+ @pytest.mark.usefixtures("config_mock", "console")
+ def test_inline_json_inputs_passed_to_runner(self, mocker: MockerFixture) -> None:
+ """Inline JSON inputs are parsed and forwarded to the runner."""
+ pipe_output = self._make_pipe_output(mocker)
+ runner_class_mock = self._mock_runner(mocker, pipe_output)
+
+ _run_async(_call_execute_run(inputs='{"topic": "cats"}'))
+
+ execute_kwargs = runner_class_mock.return_value.execute.call_args.kwargs
+ assert execute_kwargs["inputs"] == {"topic": "cats"}
+
+ @pytest.mark.usefixtures("config_mock", "console")
+ def test_file_inputs_loaded_and_resolved(self, mocker: MockerFixture, tmp_path: Path) -> None:
+ """File inputs are loaded and run through the path resolver."""
+ inputs_file = tmp_path / "inputs.json"
+ inputs_file.write_text(json.dumps({"topic": "dogs"}), encoding="utf-8")
+ resolver_mock = mocker.patch(
+ "pipelex.cli.commands.run._run_core.resolve_inputs_paths",
+ return_value={"topic": "dogs", "resolved": True},
+ )
+ pipe_output = self._make_pipe_output(mocker)
+ runner_class_mock = self._mock_runner(mocker, pipe_output)
+
+ _run_async(_call_execute_run(inputs=str(inputs_file)))
+
+ resolver_mock.assert_called_once_with({"topic": "dogs"}, tmp_path.resolve())
+ execute_kwargs = runner_class_mock.return_value.execute.call_args.kwargs
+ assert execute_kwargs["inputs"] == {"topic": "dogs", "resolved": True}
+
+ @pytest.mark.usefixtures("console")
+ def test_non_dict_input_file_exits(self, tmp_path: Path) -> None:
+ """An input file holding a JSON list (not a dict) is rejected."""
+ inputs_file = tmp_path / "inputs.json"
+ inputs_file.write_text('["not", "a", "dict"]', encoding="utf-8")
+
+ with pytest.raises(typer.Exit) as exc_info:
+ _run_async(_call_execute_run(inputs=str(inputs_file)))
+
+ assert exc_info.value.exit_code == 1
+
+ @pytest.mark.usefixtures("config_mock")
+ def test_graph_outputs_saved(self, mocker: MockerFixture, console: Console, tmp_path: Path) -> None:
+ """A graph spec on the output triggers graph generation into the output dir."""
+ pipe_output = self._make_pipe_output(mocker, graph_spec=mocker.MagicMock())
+ self._mock_runner(mocker, pipe_output)
+ mocker.patch("pipelex.cli.commands.run._run_core.generate_graph_outputs", new=mocker.AsyncMock(return_value=mocker.MagicMock()))
+ save_graphs_mock = mocker.patch(
+ "pipelex.cli.commands.run._run_core.save_graph_outputs_to_dir",
+ return_value=["mermaidflow_html", "reactflow_html"],
+ )
+
+ _run_async(_call_execute_run(output_dir=str(tmp_path)))
+
+ save_graphs_mock.assert_called_once()
+ graph_output_dir = save_graphs_mock.call_args.kwargs["output_dir"]
+ assert str(graph_output_dir).startswith(str(tmp_path))
+ output = console.export_text()
+ assert "graphs: mermaidflow, reactflow" in output
+
+ @pytest.mark.usefixtures("config_mock")
+ def test_main_stuff_saved_in_all_formats(self, mocker: MockerFixture, console: Console, tmp_path: Path) -> None:
+ """--save-main-stuff writes JSON, Markdown, HTML and the viewer file."""
+ main_stuff = mocker.MagicMock()
+ main_stuff.content.rendered_json_async = mocker.AsyncMock(return_value='{"answer": 42}')
+ main_stuff.content.rendered_markdown_async = mocker.AsyncMock(return_value="# answer 42")
+ main_stuff.content.rendered_html_async = mocker.AsyncMock(return_value="<p>42</p>")
+ mocker.patch("pipelex.cli.commands.run._run_core.render_stuff_viewer", new=mocker.AsyncMock(return_value="<html>viewer</html>"))
+ pipe_output = self._make_pipe_output(mocker, main_stuff=main_stuff)
+ self._mock_runner(mocker, pipe_output)
+
+ _run_async(_call_execute_run(save_main_stuff=True, output_dir=str(tmp_path)))
+
+ output_dirs = list(tmp_path.iterdir())
+ assert len(output_dirs) == 1
+ saved_dir = output_dirs[0]
+ assert (saved_dir / "main_stuff.json").read_text(encoding="utf-8") == '{"answer": 42}'
+ assert (saved_dir / "main_stuff.md").read_text(encoding="utf-8") == "# answer 42"
+ assert (saved_dir / "main_stuff.html").read_text(encoding="utf-8") == "<p>42</p>"
+ assert (saved_dir / "main_stuff_viewer.html").read_text(encoding="utf-8") == "<html>viewer</html>"
+ assert "main_stuff: json, md, html, html_viewer" in console.export_text()
+
+ @pytest.mark.usefixtures("config_mock")
+ def test_working_memory_saved(self, mocker: MockerFixture, console: Console, tmp_path: Path) -> None:
+ """--save-working-memory dumps the working memory as JSON in the output dir."""
+ pipe_output = self._make_pipe_output(mocker)
+ self._mock_runner(mocker, pipe_output)
+
+ _run_async(_call_execute_run(save_working_memory=True, output_dir=str(tmp_path)))
+
+ output_dirs = list(tmp_path.iterdir())
+ assert len(output_dirs) == 1
+ memory_file = output_dirs[0] / "working_memory.json"
+ assert json.loads(memory_file.read_text(encoding="utf-8")) == {"stuff": "dump"}
+ assert "working_memory.json" in console.export_text()
+
+ @pytest.mark.usefixtures("config_mock")
+ def test_working_memory_saved_to_explicit_path(self, mocker: MockerFixture, console: Console, tmp_path: Path) -> None:
+ """An explicit --working-memory-path wins over the output dir default."""
+ pipe_output = self._make_pipe_output(mocker)
+ self._mock_runner(mocker, pipe_output)
+ explicit_path = tmp_path / "custom_memory.json"
+
+ _run_async(
+ _call_execute_run(
+ save_working_memory=True,
+ working_memory_path=str(explicit_path),
+ output_dir=str(tmp_path / "outputs"),
+ )
+ )
+
+ assert json.loads(explicit_path.read_text(encoding="utf-8")) == {"stuff": "dump"}
+ assert f"working_memory: {explicit_path}" in console.export_text()
+
+ def test_save_csv_empty_path_fails_fast(self) -> None:
+ """An empty --save-csv path fails before anything runs."""
+ with pytest.raises(typer.Exit) as exc_info:
+ _run_async(_call_execute_run(save_csv=" "))
+
+ assert exc_info.value.exit_code == 1
+
+ def test_save_csv_bad_suffix_fails_fast(self) -> None:
+ """An unsupported table suffix fails before anything runs."""
+ with pytest.raises(typer.Exit) as exc_info:
+ _run_async(_call_execute_run(save_csv="out.xlsx"))
+
+ assert exc_info.value.exit_code == 1
+
+ @pytest.mark.usefixtures("config_mock", "console")
+ def test_save_csv_no_main_stuff_exits(self, mocker: MockerFixture, tmp_path: Path) -> None:
+ """--save-csv with no main stuff produced is an error after the run."""
+ pipe_output = self._make_pipe_output(mocker, main_stuff=None)
+ self._mock_runner(mocker, pipe_output)
+
+ with pytest.raises(typer.Exit) as exc_info:
+ _run_async(_call_execute_run(save_csv=str(tmp_path / "out.csv")))
+
+ assert exc_info.value.exit_code == 1
+
+ @pytest.mark.usefixtures("config_mock", "console")
+ def test_save_csv_non_list_output_exits(self, mocker: MockerFixture, tmp_path: Path) -> None:
+ """--save-csv requires the main stuff to be a list."""
+ main_stuff = mocker.MagicMock()
+ main_stuff.content = TextContent(text="not a list")
+ pipe_output = self._make_pipe_output(mocker, main_stuff=main_stuff)
+ self._mock_runner(mocker, pipe_output)
+
+ with pytest.raises(typer.Exit) as exc_info:
+ _run_async(_call_execute_run(save_csv=str(tmp_path / "out.csv")))
+
+ assert exc_info.value.exit_code == 1
+
+ @pytest.mark.usefixtures("config_mock")
+ def test_save_csv_happy_path(self, mocker: MockerFixture, console: Console, tmp_path: Path) -> None:
+ """A flat list output is written through the CSV codec to the literal path."""
+ main_stuff = mocker.MagicMock()
+ main_stuff.content = ListContent(items=[TextContent(text="row one")])
+ main_stuff.concept.get_structure_class.return_value = TextContent
+ pipe_output = self._make_pipe_output(mocker, main_stuff=main_stuff)
+ self._mock_runner(mocker, pipe_output)
+ mocker.patch("pipelex.cli.commands.run._run_core.flat_field_names", return_value=["text"])
+ csv_codec_mock = mocker.patch("pipelex.cli.commands.run._run_core.csv_from_list_content")
+ csv_target = tmp_path / "reports" / "out.csv"
+
+ _run_async(_call_execute_run(save_csv=str(csv_target)))
+
+ csv_codec_mock.assert_called_once()
+ assert csv_codec_mock.call_args.kwargs["path"] == csv_target
+ assert csv_target.parent.is_dir()
+ assert f"CSV saved to {csv_target}" in console.export_text()
+
+ @pytest.mark.usefixtures("config_mock", "console")
+ def test_save_csv_concept_error_framed_as_csv_failure(self, mocker: MockerFixture, tmp_path: Path) -> None:
+ """A missing structure class on the output concept is a --save-csv failure."""
+ main_stuff = mocker.MagicMock()
+ main_stuff.content = ListContent(items=[TextContent(text="row one")])
+ main_stuff.concept.get_structure_class.side_effect = ConceptValueError("no structure class")
+ pipe_output = self._make_pipe_output(mocker, main_stuff=main_stuff)
+ self._mock_runner(mocker, pipe_output)
+
+ with pytest.raises(typer.Exit) as exc_info:
+ _run_async(_call_execute_run(save_csv=str(tmp_path / "out.csv")))
+
+ assert exc_info.value.exit_code == 1
diff --git a/tests/unit/pipelex/cli/test_run_core_wrapper.py b/tests/unit/pipelex/cli/test_run_core_wrapper.py
new file mode 100644
index 000000000..8fd9cb10b
--- /dev/null
+++ b/tests/unit/pipelex/cli/test_run_core_wrapper.py
@@ -0,0 +1,141 @@
+"""Unit tests for execute_run — the sync wrapper around _execute_run.
+
+Covers setup/teardown wiring and the error-handler dispatch at the CLI boundary.
+"""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any
+
+import pytest
+import typer
+from rich.console import Console
+
+from pipelex.base_exceptions import PipelexError
+from pipelex.cli.commands.run._run_core import execute_run # noqa: PLC2701
+from pipelex.cli.error_handlers import ErrorContext
+from pipelex.cogt.model_backends.model_type import ModelType
+from pipelex.core.pipes.exceptions import PipeOperatorModelChoiceError
+from pipelex.pipe_operators.exceptions import PipeOperatorModelAvailabilityError
+from pipelex.pipe_run.pipe_run_mode import PipeRunMode
+
+if TYPE_CHECKING:
+ from pytest_mock import MockerFixture
+
+
+def _call_execute_run(**overrides: Any) -> None:
+ kwargs: dict[str, Any] = {
+ "pipe_code": "test_pipe",
+ "bundle_path": None,
+ "inputs": None,
+ "save_working_memory": False,
+ "working_memory_path": None,
+ "save_main_stuff": False,
+ "no_pretty_print": True,
+ "graph": None,
+ "graph_full_data": None,
+ "output_dir": "temp/test_outputs",
+ "dry_run": False,
+ "mock_usage": False,
+ "mock_inputs": False,
+ "library_dir": None,
+ }
+ kwargs.update(overrides)
+ execute_run(**kwargs)
+
+
+class TestExecuteRunWrapper:
+ @pytest.fixture
+ def wrapper_mocks(self, mocker: MockerFixture) -> dict[str, Any]:
+ """Stub the Pipelex bootstrap, telemetry context, teardown and the async core."""
+ telemetry_manager = mocker.MagicMock()
+ mocks: dict[str, Any] = {
+ "make_pipelex": mocker.patch("pipelex.cli.commands.run._run_core.make_pipelex_for_cli"),
+ "telemetry": mocker.patch("pipelex.cli.commands.run._run_core.get_telemetry_manager", return_value=telemetry_manager),
+ "tag": mocker.patch("pipelex.cli.commands.run._run_core.tag"),
+ "teardown": mocker.patch("pipelex.cli.commands.run._run_core.Pipelex.teardown_if_needed"),
+ "core": mocker.patch("pipelex.cli.commands.run._run_core._execute_run", new=mocker.AsyncMock(return_value=None)),
+ }
+ recorded_console = Console(width=200, record=True, color_system=None)
+ mocker.patch("pipelex.cli.commands.run._run_core.get_console", return_value=recorded_console)
+ mocks["console"] = recorded_console
+ return mocks
+
+ def test_happy_path_setup_and_teardown(self, wrapper_mocks: dict[str, Any]) -> None:
+ """The wrapper boots Pipelex for the CLI, runs the core, and always tears down."""
+ _call_execute_run(library_dir=["./libs"])
+
+ make_kwargs = wrapper_mocks["make_pipelex"].call_args.kwargs
+ assert make_kwargs["context"] == ErrorContext.VALIDATION_BEFORE_PIPE_RUN
+ assert make_kwargs["library_dirs"] == ["./libs"]
+ wrapper_mocks["core"].assert_awaited_once()
+ wrapper_mocks["teardown"].assert_called_once()
+
+ def test_model_choice_error_dispatched_to_handler(self, wrapper_mocks: dict[str, Any], mocker: MockerFixture) -> None:
+ """A model-choice error is routed to its dedicated handler."""
+ choice_error = PipeOperatorModelChoiceError(
+ message="model 'gpt-5' not found",
+ pipe_type="llm_text",
+ pipe_code="test_pipe",
+ model_type=ModelType.LLM,
+ model_choice="gpt-5",
+ )
+ wrapper_mocks["core"].side_effect = choice_error
+ handler_mock = mocker.patch("pipelex.cli.commands.run._run_core.handle_model_choice_error")
+
+ _call_execute_run()
+
+ handler_mock.assert_called_once_with(choice_error, context=ErrorContext.PIPE_RUN)
+ wrapper_mocks["teardown"].assert_called_once()
+
+ def test_model_availability_error_dispatched_to_handler(self, wrapper_mocks: dict[str, Any], mocker: MockerFixture) -> None:
+ """A model-availability error is routed to its dedicated handler."""
+ availability_error = PipeOperatorModelAvailabilityError(
+ message="model not available",
+ run_mode=PipeRunMode.LIVE,
+ pipe_type="llm_text",
+ pipe_code="test_pipe",
+ pipe_stack=["test_pipe"],
+ model_handle="gpt-4o",
+ fallback_list=[],
+ )
+ wrapper_mocks["core"].side_effect = availability_error
+ handler_mock = mocker.patch("pipelex.cli.commands.run._run_core.handle_model_availability_error")
+
+ _call_execute_run()
+
+ handler_mock.assert_called_once_with(availability_error, context=ErrorContext.PIPE_RUN)
+
+ def test_typer_exit_passes_through(self, wrapper_mocks: dict[str, Any]) -> None:
+ """A typer.Exit from the core propagates untouched (already user-reported)."""
+ wrapper_mocks["core"].side_effect = typer.Exit(1)
+
+ with pytest.raises(typer.Exit) as exc_info:
+ _call_execute_run()
+
+ assert exc_info.value.exit_code == 1
+ wrapper_mocks["teardown"].assert_called_once()
+
+ def test_pipelex_error_prints_message_and_exits(self, wrapper_mocks: dict[str, Any]) -> None:
+ """A PipelexError prints the friendly failure message and exits 1."""
+ wrapper_mocks["core"].side_effect = PipelexError("the pipe blew up")
+
+ with pytest.raises(typer.Exit) as exc_info:
+ _call_execute_run()
+
+ assert exc_info.value.exit_code == 1
+ output = wrapper_mocks["console"].export_text()
+ assert "Failed to execute pipeline" in output
+ assert "the pipe blew up" in output
+
+ def test_unexpected_error_prints_exception_and_exits(self, wrapper_mocks: dict[str, Any]) -> None:
+ """An unexpected exception prints the rich traceback and exits 1."""
+ wrapper_mocks["core"].side_effect = RuntimeError("totally unexpected")
+
+ with pytest.raises(typer.Exit) as exc_info:
+ _call_execute_run()
+
+ assert exc_info.value.exit_code == 1
+ output = wrapper_mocks["console"].export_text()
+ assert "Failed to execute pipeline" in output
+ assert "totally unexpected" in output
diff --git a/tests/unit/pipelex/cli/test_show_cmd.py b/tests/unit/pipelex/cli/test_show_cmd.py
new file mode 100644
index 000000000..ab8069e49
--- /dev/null
+++ b/tests/unit/pipelex/cli/test_show_cmd.py
@@ -0,0 +1,212 @@
+"""Unit tests for the `pipelex show` report logic (pipelex/cli/commands/show_cmd.py).
+
+Covers the do_* functions directly with mocked hub getters — the Typer wrappers
+(arg parsing, --help) are owned by the conformance suite.
+"""
+
+from __future__ import annotations
+
+from types import SimpleNamespace
+from typing import TYPE_CHECKING, Any
+
+import pytest
+from rich.console import Console
+
+from pipelex.base_exceptions import PipelexConfigError
+from pipelex.cli.commands.show_cmd import do_list_pipes, do_show_backends, do_show_config, do_show_pipe
+from pipelex.cli.exceptions import PipelexCLIError
+from pipelex.tools.misc.exceptions import TomlError
+
+if TYPE_CHECKING:
+ from pathlib import Path
+
+ from pytest_mock import MockerFixture
+
+
+def _make_backend(name: str, enabled: bool, endpoint: str | None = None, model_count: int = 0) -> SimpleNamespace:
+ return SimpleNamespace(
+ name=name,
+ enabled=enabled,
+ endpoint=endpoint,
+ model_specs={f"model_{model_index}": object() for model_index in range(model_count)},
+ )
+
+
+def _make_routing_profile(
+ name: str = "standard",
+ description: str | None = "The standard profile",
+ default: str | None = "openai",
+ routes: dict[str, str] | None = None,
+) -> SimpleNamespace:
+ return SimpleNamespace(name=name, description=description, default=default, routes=routes or {})
+
+
+class TestShowCmd:
+ @pytest.fixture
+ def console(self, mocker: MockerFixture) -> Console:
+ recorded_console = Console(width=120, record=True, color_system=None)
+ mocker.patch("pipelex.cli.commands.show_cmd.get_console", return_value=recorded_console)
+ return recorded_console
+
+ @pytest.fixture
+ def telemetry(self, mocker: MockerFixture) -> Any:
+ telemetry_manager = mocker.Mock()
+ mocker.patch("pipelex.cli.commands.show_cmd.get_telemetry_manager", return_value=telemetry_manager)
+ return telemetry_manager
+
+ def _mock_backend_setup(
+ self,
+ mocker: MockerFixture,
+ backends: list[SimpleNamespace],
+ routing_profile: SimpleNamespace,
+ ) -> None:
+ mocker.patch("pipelex.cli.commands.show_cmd.get_secrets_provider")
+ models_manager = SimpleNamespace(
+ inference_backend_library=SimpleNamespace(root={backend.name: backend for backend in backends}),
+ routing_profile=routing_profile,
+ )
+ mocker.patch("pipelex.cli.commands.show_cmd.get_models_manager", return_value=models_manager)
+
+ def test_do_show_config_pretty_prints_loaded_config(self, mocker: MockerFixture) -> None:
+ """The loaded config is pretty-printed under the expected title."""
+ fake_config = {"pipelex": "config"}
+ mocked_manager = mocker.patch("pipelex.cli.commands.show_cmd.config_manager")
+ mocked_manager.load_config.return_value = fake_config
+ pretty_print_mock = mocker.patch("pipelex.cli.commands.show_cmd.pretty_print")
+
+ do_show_config()
+
+ pretty_print_mock.assert_called_once_with(fake_config, title="Pipelex configuration")
+
+ def test_do_show_config_wraps_toml_error(self, mocker: MockerFixture) -> None:
+ """A TOML load failure surfaces as PipelexConfigError with context."""
+ mocked_manager = mocker.patch("pipelex.cli.commands.show_cmd.config_manager")
+ mocked_manager.load_config.side_effect = TomlError(message="bad toml", doc="", pos=0, lineno=1, colno=1)
+
+ with pytest.raises(PipelexConfigError, match="Error loading configuration: bad toml"):
+ do_show_config()
+
+ def test_do_list_pipes_tracks_pipe_count(self, mocker: MockerFixture, telemetry: Any) -> None:
+ """Listing pipes delegates to the library and reports the count to telemetry."""
+ pipe_library = mocker.Mock()
+ pipe_library.pretty_list_pipes.return_value = 7
+ mocker.patch("pipelex.cli.commands.show_cmd.get_pipe_library", return_value=pipe_library)
+
+ do_list_pipes()
+
+ pipe_library.pretty_list_pipes.assert_called_once()
+ telemetry.track_event.assert_called_once()
+ _, call_kwargs = telemetry.track_event.call_args
+ assert call_kwargs["properties"] == {"nb_pipes": 7}
+
+ def test_do_show_pipe_pretty_prints_pipe(self, mocker: MockerFixture, telemetry: Any) -> None:
+ """Showing a pipe fetches it from the hub and pretty-prints it."""
+ fake_pipe = SimpleNamespace(type="PipeLLM")
+ get_required_pipe_mock = mocker.patch("pipelex.cli.commands.show_cmd.get_required_pipe", return_value=fake_pipe)
+ pretty_print_mock = mocker.patch("pipelex.cli.commands.show_cmd.pretty_print")
+
+ do_show_pipe(pipe_code="my_pipe")
+
+ get_required_pipe_mock.assert_called_once_with(pipe_code="my_pipe")
+ pretty_print_mock.assert_called_once_with(fake_pipe, title="Pipe 'my_pipe'")
+ telemetry.track_event.assert_called_once()
+
+ def test_do_show_backends_no_backends(self, mocker: MockerFixture, console: Console, telemetry: Any) -> None:
+ """An empty backend library prints a warning and stops."""
+ self._mock_backend_setup(mocker, backends=[], routing_profile=_make_routing_profile())
+
+ do_show_backends()
+
+ output = console.export_text()
+ assert "No backends configured." in output
+ telemetry.track_event.assert_not_called()
+
+ def test_do_show_backends_enabled_only_hides_disabled(self, mocker: MockerFixture, console: Console, telemetry: Any) -> None:
+ """Default mode lists only enabled backends and hints at the hidden ones."""
+ backends = [
+ _make_backend("openai", enabled=True, endpoint="https://api.openai.com", model_count=2),
+ _make_backend("anthropic", enabled=False, endpoint="https://api.anthropic.com", model_count=1),
+ ]
+ self._mock_backend_setup(mocker, backends=backends, routing_profile=_make_routing_profile(routes={"gpt-*": "openai"}))
+
+ do_show_backends(show_all=False)
+
+ output = console.export_text()
+ assert "Enabled Backends" in output
+ assert "openai" in output
+ assert "https://api.openai.com" in output
+ assert "anthropic" not in output.split("Active Routing Profile")[0]
+ assert "1 disabled backend(s) hidden" in output
+ assert "pipelex show backends --all" in output
+ assert "Active Routing Profile: standard" in output
+ assert "Default Backend: openai" in output
+ assert "Routing Rules:" in output
+ assert "gpt-*" in output
+ telemetry.track_event.assert_called_once()
+ _, call_kwargs = telemetry.track_event.call_args
+ assert call_kwargs["properties"] == {"nb_backends": 2}
+
+ @pytest.mark.usefixtures("telemetry")
+ def test_do_show_backends_show_all_includes_status_column(self, mocker: MockerFixture, console: Console, tmp_path: Path) -> None:
+ """--all loads the library leniently and shows Enabled/Disabled status."""
+ backends = [
+ _make_backend("openai", enabled=True, model_count=1),
+ _make_backend("anthropic", enabled=False, model_count=1),
+ ]
+ mocker.patch("pipelex.cli.commands.show_cmd.get_secrets_provider")
+ models_manager = SimpleNamespace(
+ inference_backend_library=SimpleNamespace(root={}),
+ routing_profile=_make_routing_profile(routes={}),
+ )
+ mocker.patch("pipelex.cli.commands.show_cmd.get_models_manager", return_value=models_manager)
+ mocked_config_manager = mocker.patch("pipelex.cli.commands.show_cmd.config_manager")
+ mocked_config_manager.backends_file_path = tmp_path / "backends.toml"
+ mocked_config_manager.backends_dir_path = tmp_path / "backends"
+ lenient_library = SimpleNamespace(root={backend.name: backend for backend in backends}, load=mocker.Mock())
+ library_class_mock = mocker.patch("pipelex.cli.commands.show_cmd.InferenceBackendLibrary", return_value=lenient_library)
+
+ do_show_backends(show_all=True)
+
+ library_class_mock.assert_called_once()
+ load_kwargs = lenient_library.load.call_args.kwargs
+ assert load_kwargs["include_disabled"] is True
+ assert load_kwargs["lenient"] is True
+ output = console.export_text()
+ assert "All Configured Backends" in output
+ assert "Status" in output
+ assert "Enabled" in output
+ assert "Disabled" in output
+ assert "anthropic" in output
+ assert "disabled backend(s) hidden" not in output
+
+ @pytest.mark.usefixtures("telemetry")
+ def test_do_show_backends_no_routes_prints_placeholder(self, mocker: MockerFixture, console: Console) -> None:
+ """A routing profile without rules prints the no-rules placeholder."""
+ backends = [_make_backend("openai", enabled=True)]
+ self._mock_backend_setup(mocker, backends=backends, routing_profile=_make_routing_profile(description=None, default=None, routes={}))
+
+ do_show_backends()
+
+ output = console.export_text()
+ assert "No specific routing rules defined." in output
+ assert "Default Backend:" not in output
+
+ @pytest.mark.usefixtures("telemetry")
+ def test_do_show_backends_markup_error_in_routing_profile(self, mocker: MockerFixture, console: Console) -> None:
+ """Bad markup in routing profile data degrades to a warning, not a crash."""
+ backends = [_make_backend("openai", enabled=True)]
+ bad_profile = _make_routing_profile(name="bad[/bold]name", description=None, default=None, routes={})
+ self._mock_backend_setup(mocker, backends=backends, routing_profile=bad_profile)
+
+ do_show_backends()
+
+ output = console.export_text()
+ assert "Warning: Could not display routing profile information:" in output
+
+ @pytest.mark.usefixtures("console")
+ def test_do_show_backends_wraps_access_errors(self, mocker: MockerFixture) -> None:
+ """Failures while accessing backend config surface as PipelexCLIError."""
+ mocker.patch("pipelex.cli.commands.show_cmd.get_secrets_provider", side_effect=OSError("denied"))
+
+ with pytest.raises(PipelexCLIError, match="Error accessing backend or routing configuration: denied"):
+ do_show_backends()
diff --git a/tests/unit/pipelex/cli/test_which_cmd.py b/tests/unit/pipelex/cli/test_which_cmd.py
new file mode 100644
index 000000000..5c38da323
--- /dev/null
+++ b/tests/unit/pipelex/cli/test_which_cmd.py
@@ -0,0 +1,86 @@
+"""Unit tests for `pipelex which` pipe-location logic (pipelex/cli/commands/which_cmd.py)."""
+
+from __future__ import annotations
+
+from types import SimpleNamespace
+from typing import TYPE_CHECKING
+
+import pytest
+from rich.console import Console
+
+from pipelex.cli.commands.which_cmd import do_which_pipe
+
+if TYPE_CHECKING:
+ from pathlib import Path
+
+ from pytest_mock import MockerFixture
+
+
+class TestWhichCmd:
+ @pytest.fixture
+ def console(self, mocker: MockerFixture) -> Console:
+ recorded_console = Console(width=300, record=True, color_system=None)
+ mocker.patch("pipelex.cli.commands.which_cmd.get_console", return_value=recorded_console)
+ return recorded_console
+
+ def test_pipe_found_with_source(self, mocker: MockerFixture, console: Console, tmp_path: Path) -> None:
+ """A found pipe reports type, domain and source, and returns True."""
+ fake_pipe = SimpleNamespace(pipe_type="PipeLLM", domain_code="demo")
+ mocker.patch("pipelex.cli.commands.which_cmd.get_optional_pipe", return_value=fake_pipe)
+ mocker.patch("pipelex.cli.commands.which_cmd.get_pipe_source", return_value=str(tmp_path / "demo.mthds"))
+
+ found = do_which_pipe(pipe_code="demo.my_pipe", library_dirs=[tmp_path], source_label="--library-dir")
+
+ assert found is True
+ output = console.export_text()
+ assert "Search path for 'demo.my_pipe':" in output
+ assert "✓" in output
+ assert "(--library-dir)" in output
+ assert "Found: demo.my_pipe" in output
+ assert "Type: PipeLLM" in output
+ assert "Domain: demo" in output
+ assert "Source:" in output
+ assert "demo.mthds" in output
+
+ def test_pipe_found_without_source(self, mocker: MockerFixture, console: Console, tmp_path: Path) -> None:
+ """A found pipe with no known source path omits the Source line."""
+ fake_pipe = SimpleNamespace(pipe_type="PipeLLM", domain_code="demo")
+ mocker.patch("pipelex.cli.commands.which_cmd.get_optional_pipe", return_value=fake_pipe)
+ mocker.patch("pipelex.cli.commands.which_cmd.get_pipe_source", return_value=None)
+
+ found = do_which_pipe(pipe_code="demo.my_pipe", library_dirs=[tmp_path], source_label="PIPELEXPATH")
+
+ assert found is True
+ assert "Source:" not in console.export_text()
+
+ def test_pipe_not_found(self, mocker: MockerFixture, console: Console, tmp_path: Path) -> None:
+ """A missing pipe returns False and prints the search tip."""
+ mocker.patch("pipelex.cli.commands.which_cmd.get_optional_pipe", return_value=None)
+
+ found = do_which_pipe(pipe_code="ghost_pipe", library_dirs=[tmp_path], source_label="PIPELEXPATH")
+
+ assert found is False
+ output = console.export_text()
+ assert "Not found: ghost_pipe" in output
+ assert "PIPELEXPATH or passed via --library-dir" in output
+
+ def test_empty_library_dirs(self, mocker: MockerFixture, console: Console) -> None:
+ """An empty search path is reported explicitly."""
+ mocker.patch("pipelex.cli.commands.which_cmd.get_optional_pipe", return_value=None)
+
+ found = do_which_pipe(pipe_code="any_pipe", library_dirs=[], source_label="defaults")
+
+ assert found is False
+ assert "(no directories configured)" in console.export_text()
+
+ def test_nonexistent_directory_marked(self, mocker: MockerFixture, console: Console, tmp_path: Path) -> None:
+ """Directories missing on disk are marked with a red cross but still listed."""
+ mocker.patch("pipelex.cli.commands.which_cmd.get_optional_pipe", return_value=None)
+ missing_dir = tmp_path / "does-not-exist"
+
+ do_which_pipe(pipe_code="any_pipe", library_dirs=[tmp_path, missing_dir], source_label="defaults")
+
+ output = console.export_text()
+ assert "✓" in output
+ assert "✗" in output
+ assert str(missing_dir) in output
diff --git a/tests/unit/pipelex/cogt/img_gen/conftest.py b/tests/unit/pipelex/cogt/img_gen/conftest.py
new file mode 100644
index 000000000..bada8632c
--- /dev/null
+++ b/tests/unit/pipelex/cogt/img_gen/conftest.py
@@ -0,0 +1,66 @@
+"""Shared builders for img_gen unit tests."""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
+from pipelex.cogt.img_gen.img_gen_job import ImgGenJob
+from pipelex.cogt.img_gen.img_gen_job_components import (
+ AspectRatio,
+ Background,
+ ImgGenJobConfig,
+ ImgGenJobParams,
+ ImgGenJobReport,
+ InputFidelity,
+ Quality,
+)
+from pipelex.cogt.img_gen.img_gen_prompt import ImgGenPrompt
+from pipelex.pipeline.job_metadata import JobMetadata
+from pipelex.tools.misc.image_utils import ImageFormat
+
+if TYPE_CHECKING:
+ from pipelex.cogt.image.image_size import ImageSize
+ from pipelex.cogt.image.prompt_image import PromptImage
+
+
+def make_img_gen_job(
+ *,
+ positive_text: str = "A test prompt",
+ negative_text: str | None = None,
+ input_images: list[PromptImage] | None = None,
+ aspect_ratio: AspectRatio = AspectRatio.SQUARE,
+ size: ImageSize | None = None,
+ background: Background = Background.OPAQUE,
+ quality: Quality | None = None,
+ input_fidelity: InputFidelity | None = None,
+ nb_steps: int | None = None,
+ guidance_scale: float | None = None,
+ is_moderated: bool | None = None,
+ safety_tolerance: int | None = None,
+ is_raw: bool | None = None,
+ output_format: ImageFormat | None = ImageFormat.PNG,
+) -> ImgGenJob:
+ """Create a test ImgGenJob with configurable prompt and job parameters."""
+ return ImgGenJob(
+ img_gen_prompt=ImgGenPrompt(
+ positive_text=positive_text,
+ negative_text=negative_text,
+ input_images=input_images,
+ ),
+ job_params=ImgGenJobParams(
+ aspect_ratio=aspect_ratio,
+ size=size,
+ background=background,
+ quality=quality,
+ input_fidelity=input_fidelity,
+ nb_steps=nb_steps,
+ guidance_scale=guidance_scale,
+ is_moderated=is_moderated,
+ safety_tolerance=safety_tolerance,
+ is_raw=is_raw,
+ output_format=output_format,
+ ),
+ job_config=ImgGenJobConfig(is_sync_mode=False),
+ job_report=ImgGenJobReport(),
+ job_metadata=JobMetadata(user_id="test-user", pipeline_run_id="test-run"),
+ )
diff --git a/tests/unit/pipelex/cogt/img_gen/test_img_gen_args_aspect_ratio.py b/tests/unit/pipelex/cogt/img_gen/test_img_gen_args_aspect_ratio.py
new file mode 100644
index 000000000..4ecf69b3e
--- /dev/null
+++ b/tests/unit/pipelex/cogt/img_gen/test_img_gen_args_aspect_ratio.py
@@ -0,0 +1,154 @@
+"""Tests for ImgGenArgsFactory.make_args_from_aspect_ratio across the FAL/Flux/Qwen taxonomies."""
+
+from __future__ import annotations
+
+import pytest
+
+from pipelex.cogt.exceptions import ImgGenParameterError
+from pipelex.cogt.img_gen.img_gen_args_factory import ImgGenArgsFactory
+from pipelex.cogt.img_gen.img_gen_job_components import AspectRatio
+from pipelex.cogt.img_gen.img_gen_model_rules import AspectRatioTaxonomy
+
+
+class TestImgGenArgsAspectRatio:
+ @pytest.mark.parametrize(
+ ("aspect_ratio", "expected_image_size"),
+ [
+ (AspectRatio.SQUARE, "square_hd"),
+ (AspectRatio.LANDSCAPE_4_3, "landscape_4_3"),
+ (AspectRatio.LANDSCAPE_16_9, "landscape_16_9"),
+ (AspectRatio.LANDSCAPE_21_9, "landscape_21_9"),
+ (AspectRatio.PORTRAIT_3_4, "portrait_4_3"),
+ (AspectRatio.PORTRAIT_9_16, "portrait_16_9"),
+ (AspectRatio.PORTRAIT_9_21, "portrait_21_9"),
+ ],
+ )
+ def test_flux_maps_supported_ratios_to_image_size(self, aspect_ratio: AspectRatio, expected_image_size: str) -> None:
+ """Flux taxonomy maps each supported aspect ratio to the FAL `image_size` preset string."""
+ result = ImgGenArgsFactory.make_args_from_aspect_ratio(
+ aspect_ratio_taxonomy=AspectRatioTaxonomy.FLUX,
+ aspect_ratio=aspect_ratio,
+ size=None,
+ model_name="flux-dev",
+ )
+
+ assert result == {"image_size": expected_image_size}
+
+ @pytest.mark.parametrize(
+ "aspect_ratio",
+ [
+ AspectRatio.LANDSCAPE_3_2,
+ AspectRatio.PORTRAIT_2_3,
+ ],
+ )
+ def test_flux_rejects_unsupported_ratios(self, aspect_ratio: AspectRatio) -> None:
+ """Flux taxonomy raises a parameter error naming Flux for unsupported ratios."""
+ with pytest.raises(ImgGenParameterError) as exc_info:
+ ImgGenArgsFactory.make_args_from_aspect_ratio(
+ aspect_ratio_taxonomy=AspectRatioTaxonomy.FLUX,
+ aspect_ratio=aspect_ratio,
+ size=None,
+ model_name="flux-dev",
+ )
+
+ error_message = str(exc_info.value)
+ assert aspect_ratio in error_message
+ assert "Flux image generation model" in error_message
+
+ @pytest.mark.parametrize(
+ ("aspect_ratio", "expected_ratio_string"),
+ [
+ (AspectRatio.SQUARE, "1:1"),
+ (AspectRatio.LANDSCAPE_4_3, "4:3"),
+ (AspectRatio.LANDSCAPE_16_9, "16:9"),
+ (AspectRatio.LANDSCAPE_21_9, "21:9"),
+ (AspectRatio.PORTRAIT_3_4, "3:4"),
+ (AspectRatio.PORTRAIT_9_16, "9:16"),
+ (AspectRatio.PORTRAIT_9_21, "9:21"),
+ ],
+ )
+ def test_flux_11_ultra_maps_supported_ratios_to_aspect_ratio(self, aspect_ratio: AspectRatio, expected_ratio_string: str) -> None:
+ """Flux-1.1 Ultra taxonomy maps each supported aspect ratio to a `aspect_ratio` ratio string."""
+ result = ImgGenArgsFactory.make_args_from_aspect_ratio(
+ aspect_ratio_taxonomy=AspectRatioTaxonomy.FLUX_11_ULTRA,
+ aspect_ratio=aspect_ratio,
+ size=None,
+ model_name="flux-1.1-ultra",
+ )
+
+ assert result == {"aspect_ratio": expected_ratio_string}
+
+ @pytest.mark.parametrize(
+ "aspect_ratio",
+ [
+ AspectRatio.LANDSCAPE_3_2,
+ AspectRatio.PORTRAIT_2_3,
+ ],
+ )
+ def test_flux_11_ultra_rejects_unsupported_ratios(self, aspect_ratio: AspectRatio) -> None:
+ """Flux-1.1 Ultra taxonomy raises a parameter error naming Flux-1.1 Ultra for unsupported ratios."""
+ with pytest.raises(ImgGenParameterError) as exc_info:
+ ImgGenArgsFactory.make_args_from_aspect_ratio(
+ aspect_ratio_taxonomy=AspectRatioTaxonomy.FLUX_11_ULTRA,
+ aspect_ratio=aspect_ratio,
+ size=None,
+ model_name="flux-1.1-ultra",
+ )
+
+ error_message = str(exc_info.value)
+ assert aspect_ratio in error_message
+ assert "Flux-1.1 Ultra" in error_message
+
+ @pytest.mark.parametrize(
+ ("aspect_ratio", "expected_width", "expected_height", "expected_ratio_string"),
+ [
+ (AspectRatio.SQUARE, 1328, 1328, "1:1"),
+ (AspectRatio.LANDSCAPE_16_9, 1664, 928, "16:9"),
+ (AspectRatio.PORTRAIT_9_16, 928, 1664, "9:16"),
+ (AspectRatio.LANDSCAPE_4_3, 1472, 1140, "4:3"),
+ (AspectRatio.PORTRAIT_3_4, 1140, 1472, "3:4"),
+ (AspectRatio.LANDSCAPE_3_2, 1584, 1056, "3:2"),
+ (AspectRatio.PORTRAIT_2_3, 1056, 1584, "2:3"),
+ ],
+ )
+ def test_qwen_image_maps_supported_ratios_to_pixel_dimensions(
+ self,
+ aspect_ratio: AspectRatio,
+ expected_width: int,
+ expected_height: int,
+ expected_ratio_string: str,
+ ) -> None:
+ """Qwen taxonomy maps each supported aspect ratio to exact width/height pixels plus a ratio string."""
+ result = ImgGenArgsFactory.make_args_from_aspect_ratio(
+ aspect_ratio_taxonomy=AspectRatioTaxonomy.QWEN_IMAGE,
+ aspect_ratio=aspect_ratio,
+ size=None,
+ model_name="qwen-image",
+ )
+
+ assert result == {
+ "width": expected_width,
+ "height": expected_height,
+ "aspect_ratio": expected_ratio_string,
+ }
+
+ @pytest.mark.parametrize(
+ "aspect_ratio",
+ [
+ AspectRatio.LANDSCAPE_21_9,
+ AspectRatio.PORTRAIT_9_21,
+ ],
+ )
+ def test_qwen_image_rejects_unsupported_ratios(self, aspect_ratio: AspectRatio) -> None:
+ """Qwen taxonomy raises a parameter error for ratios outside its supported set."""
+ with pytest.raises(ImgGenParameterError) as exc_info:
+ ImgGenArgsFactory.make_args_from_aspect_ratio(
+ aspect_ratio_taxonomy=AspectRatioTaxonomy.QWEN_IMAGE,
+ aspect_ratio=aspect_ratio,
+ size=None,
+ model_name="qwen-image",
+ )
+
+ error_message = str(exc_info.value)
+ assert aspect_ratio in error_message
+ assert "not supported" in error_message
diff --git a/tests/unit/pipelex/cogt/img_gen/test_img_gen_args_inference_safety.py b/tests/unit/pipelex/cogt/img_gen/test_img_gen_args_inference_safety.py
new file mode 100644
index 000000000..48c854318
--- /dev/null
+++ b/tests/unit/pipelex/cogt/img_gen/test_img_gen_args_inference_safety.py
@@ -0,0 +1,258 @@
+"""Tests for ImgGenArgsFactory inference-steps and safety-checker argument mappings."""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any
+
+import pytest
+
+from pipelex.cogt.img_gen.img_gen_args_factory import ImgGenArgsFactory
+from pipelex.cogt.img_gen.img_gen_job_components import Quality
+from pipelex.cogt.img_gen.img_gen_model_rules import (
+ ImgGenArgTopic,
+ ImgGenModelRules,
+ InferenceTaxonomy,
+ SafetyCheckerTaxonomy,
+)
+from tests.unit.pipelex.cogt.img_gen.conftest import make_img_gen_job
+
+if TYPE_CHECKING:
+ from pytest_mock import MockerFixture
+
+
+class TestImgGenArgsInferenceSafety:
+ @pytest.mark.parametrize("nb_steps", [1, 2, 4, 8])
+ def test_sdxl_lightning_accepts_valid_steps_without_config_lookup(self, mocker: MockerFixture, nb_steps: int) -> None:
+ """SDXL Lightning passes valid step counts through without consulting the config."""
+ mock_config = mocker.MagicMock()
+ mocker.patch("pipelex.cogt.img_gen.img_gen_args_factory.get_config", return_value=mock_config)
+
+ result = ImgGenArgsFactory.make_args_from_inference(
+ inference_taxonomy=InferenceTaxonomy.SDXL_LIGHTNING,
+ num_inference_steps=nb_steps,
+ quality=None,
+ guidance_scale=None,
+ is_raw=None,
+ )
+
+ assert result == {"num_inference_steps": nb_steps}
+ mock_config.cogt.img_gen_config.get_num_inference_steps.assert_not_called()
+
+ @pytest.mark.parametrize("nb_steps", [3, 5, 16])
+ def test_sdxl_lightning_coerces_invalid_steps_to_four(self, mocker: MockerFixture, nb_steps: int) -> None:
+ """SDXL Lightning coerces step counts outside its accepted set to its safe default with a warning."""
+ mock_log = mocker.patch("pipelex.cogt.img_gen.img_gen_args_factory.log")
+
+ result = ImgGenArgsFactory.make_args_from_inference(
+ inference_taxonomy=InferenceTaxonomy.SDXL_LIGHTNING,
+ num_inference_steps=nb_steps,
+ quality=None,
+ guidance_scale=None,
+ is_raw=None,
+ )
+
+ assert result == {"num_inference_steps": 4}
+ mock_log.warning.assert_called_once()
+
+ @pytest.mark.parametrize(
+ ("quality", "expected_quality"),
+ [
+ (None, Quality.MEDIUM),
+ (Quality.HIGH, Quality.HIGH),
+ ],
+ )
+ def test_sdxl_lightning_none_steps_uses_config_lookup(self, mocker: MockerFixture, quality: Quality | None, expected_quality: Quality) -> None:
+ """SDXL Lightning derives missing steps from the config, defaulting quality to MEDIUM."""
+ mock_config = mocker.MagicMock()
+ mock_config.cogt.img_gen_config.get_num_inference_steps.return_value = 8
+ mocker.patch("pipelex.cogt.img_gen.img_gen_args_factory.get_config", return_value=mock_config)
+
+ result = ImgGenArgsFactory.make_args_from_inference(
+ inference_taxonomy=InferenceTaxonomy.SDXL_LIGHTNING,
+ num_inference_steps=None,
+ quality=quality,
+ guidance_scale=None,
+ is_raw=None,
+ )
+
+ assert result == {"num_inference_steps": 8}
+ mock_config.cogt.img_gen_config.get_num_inference_steps.assert_called_once_with(
+ model_name="sdxl_lightning",
+ quality=expected_quality,
+ )
+
+ @pytest.mark.parametrize(
+ ("inference_taxonomy", "guidance_scale", "expected_args"),
+ [
+ (InferenceTaxonomy.FLUX, 3.5, {"num_inference_steps": 28, "guidance_scale": 3.5}),
+ (InferenceTaxonomy.FLUX, None, {"num_inference_steps": 28}),
+ (InferenceTaxonomy.FLUX, 0.0, {"num_inference_steps": 28}),
+ (InferenceTaxonomy.QWEN_IMAGE, 4.0, {"num_inference_steps": 28, "guidance_scale": 4.0}),
+ (InferenceTaxonomy.QWEN_IMAGE, None, {"num_inference_steps": 28}),
+ ],
+ )
+ def test_flux_and_qwen_explicit_steps_win_and_guidance_only_when_truthy(
+ self,
+ mocker: MockerFixture,
+ inference_taxonomy: InferenceTaxonomy,
+ guidance_scale: float | None,
+ expected_args: dict[str, Any],
+ ) -> None:
+ """Flux and Qwen forward explicit steps without config lookup; guidance_scale appears only when truthy."""
+ mock_config = mocker.MagicMock()
+ mocker.patch("pipelex.cogt.img_gen.img_gen_args_factory.get_config", return_value=mock_config)
+
+ result = ImgGenArgsFactory.make_args_from_inference(
+ inference_taxonomy=inference_taxonomy,
+ num_inference_steps=28,
+ quality=None,
+ guidance_scale=guidance_scale,
+ is_raw=None,
+ )
+
+ assert result == expected_args
+ mock_config.cogt.img_gen_config.get_num_inference_steps.assert_not_called()
+
+ @pytest.mark.parametrize(
+ ("inference_taxonomy", "expected_model_name"),
+ [
+ (InferenceTaxonomy.FLUX, "flux"),
+ (InferenceTaxonomy.QWEN_IMAGE, "qwen_image"),
+ ],
+ )
+ def test_flux_and_qwen_none_steps_use_config_lookup(
+ self,
+ mocker: MockerFixture,
+ inference_taxonomy: InferenceTaxonomy,
+ expected_model_name: str,
+ ) -> None:
+ """Flux and Qwen derive missing steps from the config using their own model_name key."""
+ mock_config = mocker.MagicMock()
+ mock_config.cogt.img_gen_config.get_num_inference_steps.return_value = 12
+ mocker.patch("pipelex.cogt.img_gen.img_gen_args_factory.get_config", return_value=mock_config)
+
+ result = ImgGenArgsFactory.make_args_from_inference(
+ inference_taxonomy=inference_taxonomy,
+ num_inference_steps=None,
+ quality=Quality.LOW,
+ guidance_scale=None,
+ is_raw=None,
+ )
+
+ assert result == {"num_inference_steps": 12}
+ mock_config.cogt.img_gen_config.get_num_inference_steps.assert_called_once_with(
+ model_name=expected_model_name,
+ quality=Quality.LOW,
+ )
+
+ @pytest.mark.parametrize(
+ ("is_raw", "expected_args"),
+ [
+ (True, {"raw": True}),
+ (False, {}),
+ (None, {}),
+ ],
+ )
+ def test_flux_11_ultra_emits_raw_only_when_true(self, is_raw: bool | None, expected_args: dict[str, Any]) -> None:
+ """Flux-1.1 Ultra emits the `raw` flag only when is_raw is set to True."""
+ result = ImgGenArgsFactory.make_args_from_inference(
+ inference_taxonomy=InferenceTaxonomy.FLUX_11_ULTRA,
+ num_inference_steps=None,
+ quality=None,
+ guidance_scale=None,
+ is_raw=is_raw,
+ )
+
+ assert result == expected_args
+
+ @pytest.mark.parametrize(
+ ("quality", "expected_value"),
+ [
+ (Quality.LOW, "low"),
+ (Quality.HIGH, "high"),
+ ],
+ )
+ def test_gpt_image_forwards_explicit_quality(self, quality: Quality, expected_value: str) -> None:
+ """GPT Image taxonomy forwards explicit quality as its string value."""
+ result = ImgGenArgsFactory.make_args_from_inference(
+ inference_taxonomy=InferenceTaxonomy.GPT_IMAGE,
+ num_inference_steps=None,
+ quality=quality,
+ guidance_scale=None,
+ is_raw=None,
+ )
+
+ assert result == {"quality": expected_value}
+
+ @pytest.mark.parametrize(
+ ("is_moderated", "safety_tolerance", "expected_args"),
+ [
+ (True, None, {"enable_safety_checker": True}),
+ (False, None, {"enable_safety_checker": False}),
+ (None, 3, {"safety_tolerance": 3}),
+ (True, 5, {"enable_safety_checker": True, "safety_tolerance": 5}),
+ (None, None, {}),
+ ],
+ )
+ def test_safety_available_emits_each_parameter_independently(
+ self,
+ is_moderated: bool | None,
+ safety_tolerance: int | None,
+ expected_args: dict[str, Any],
+ ) -> None:
+ """AVAILABLE safety taxonomy emits enable_safety_checker and safety_tolerance independently when set."""
+ result = ImgGenArgsFactory.make_args_from_safety_checker(
+ safety_checker_taxonomy=SafetyCheckerTaxonomy.AVAILABLE,
+ is_moderated=is_moderated,
+ safety_tolerance=safety_tolerance,
+ )
+
+ assert result == expected_args
+
+ def test_safety_unavailable_emits_nothing_even_with_values(self) -> None:
+ """UNAVAILABLE safety taxonomy skips all safety parameters even when values are provided."""
+ result = ImgGenArgsFactory.make_args_from_safety_checker(
+ safety_checker_taxonomy=SafetyCheckerTaxonomy.UNAVAILABLE,
+ is_moderated=True,
+ safety_tolerance=4,
+ )
+
+ assert result == {}
+
+ @pytest.mark.parametrize(
+ ("is_moderated", "expected_args"),
+ [
+ (None, {}),
+ (True, {"moderation": "auto"}),
+ (False, {"moderation": "low"}),
+ ],
+ )
+ def test_safety_openai_moderation_mapping(self, is_moderated: bool | None, expected_args: dict[str, Any]) -> None:
+ """OPENAI_MODERATION maps moderation enabled to standard filtering ("auto") and disabled to less restrictive ("low"),
+ skipping the omit sentinel when unset.
+ """
+ result = ImgGenArgsFactory.make_args_from_safety_checker(
+ safety_checker_taxonomy=SafetyCheckerTaxonomy.OPENAI_MODERATION,
+ is_moderated=is_moderated,
+ safety_tolerance=None,
+ )
+
+ assert result == expected_args
+
+ @pytest.mark.asyncio
+ async def test_full_flux_11_ultra_job_emits_raw_flag(self) -> None:
+ """A full job with Flux-1.1 Ultra inference rules dispatches is_raw through to the args dict."""
+ model_rules: ImgGenModelRules = {
+ ImgGenArgTopic.INFERENCE: InferenceTaxonomy.FLUX_11_ULTRA,
+ }
+ img_gen_job = make_img_gen_job(is_raw=True)
+
+ result = await ImgGenArgsFactory.make_args_for_model(
+ model_rules=model_rules,
+ img_gen_job=img_gen_job,
+ nb_images=1,
+ model_id="fal-ai/flux-pro/v1.1-ultra",
+ model_name="flux-1.1-ultra",
+ )
+
+ assert result == {"raw": True}
diff --git a/tests/unit/pipelex/cogt/img_gen/test_img_gen_args_input_images.py b/tests/unit/pipelex/cogt/img_gen/test_img_gen_args_input_images.py
new file mode 100644
index 000000000..f5952c715
--- /dev/null
+++ b/tests/unit/pipelex/cogt/img_gen/test_img_gen_args_input_images.py
@@ -0,0 +1,264 @@
+"""Tests for ImgGenArgsFactory input-image and input-fidelity argument mappings."""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, cast
+
+import pytest
+
+from pipelex.cogt.exceptions import ImgGenParameterError
+from pipelex.cogt.image.prompt_image import PromptImage, PromptImageUri
+from pipelex.cogt.img_gen.img_gen_args_factory import ImgGenArgsFactory
+from pipelex.cogt.img_gen.img_gen_job_components import InputFidelity
+from pipelex.cogt.img_gen.img_gen_model_rules import (
+ ImgGenArgTopic,
+ ImgGenModelRules,
+ InputFidelityTaxonomy,
+ InputImagesTaxonomy,
+ NumImagesTaxonomy,
+ PromptTaxonomy,
+)
+from pipelex.tools.misc.filetype_utils import FileType
+from pipelex.tools.uri.prepared_file import PreparedFile, PreparedFileBase64, PreparedFileHttpUrl, PreparedFileLocalPath
+from tests.unit.pipelex.cogt.img_gen.conftest import make_img_gen_job
+
+if TYPE_CHECKING:
+ from pytest_mock import MockerFixture
+
+PNG_FILE_TYPE = FileType(extension="png", mime="image/png")
+
+
+def make_prepped_base64(base64_data: str) -> PreparedFileBase64:
+ """Create a PreparedFileBase64 with PNG file type for testing."""
+ return PreparedFileBase64(base64_data=base64_data, file_type=PNG_FILE_TYPE)
+
+
+def make_input_images(nb_images: int) -> list[PromptImage]:
+ """Create a list of URI prompt images for testing."""
+ return cast("list[PromptImage]", [PromptImageUri(uri=f"https://example.com/image_{index_image}.png") for index_image in range(nb_images)])
+
+
+class TestImgGenArgsInputImages:
+ @pytest.mark.asyncio
+ async def test_bfl_flux_2_mixes_base64_and_http_url_prepped_files(self, mocker: MockerFixture) -> None:
+ """BFL Flux 2 numbers keys input_image, input_image_2, ... and accepts both data URLs and raw HTTP URLs."""
+ input_images = make_input_images(3)
+ prepped_files: list[PreparedFile] = [
+ make_prepped_base64("Zmlyc3Q="),
+ PreparedFileHttpUrl(url="https://example.com/hosted.png"),
+ make_prepped_base64("dGhpcmQ="),
+ ]
+ prep_mock = mocker.patch(
+ "pipelex.cogt.img_gen.img_gen_args_factory.prep_prompt_images",
+ new_callable=mocker.AsyncMock,
+ return_value=prepped_files,
+ )
+
+ result = await ImgGenArgsFactory.make_args_from_input_images(
+ input_images_taxonomy=InputImagesTaxonomy.BFL_FLUX_2,
+ input_images=input_images,
+ )
+
+ assert result == {
+ "input_image": "data:image/png;base64,Zmlyc3Q=",
+ "input_image_2": "https://example.com/hosted.png",
+ "input_image_3": "data:image/png;base64,dGhpcmQ=",
+ }
+ prep_mock.assert_awaited_once_with(prompt_images=input_images, is_http_url_enabled=True)
+
+ @pytest.mark.asyncio
+ async def test_bfl_flux_2_caps_prepped_images_at_eight(self, mocker: MockerFixture) -> None:
+ """BFL Flux 2 keeps at most eight prepped images, dropping any extras."""
+ input_images = make_input_images(10)
+ prepped_files: list[PreparedFile] = [PreparedFileHttpUrl(url=f"https://example.com/img_{index_file}.png") for index_file in range(10)]
+ mocker.patch(
+ "pipelex.cogt.img_gen.img_gen_args_factory.prep_prompt_images",
+ new_callable=mocker.AsyncMock,
+ return_value=prepped_files,
+ )
+
+ result = await ImgGenArgsFactory.make_args_from_input_images(
+ input_images_taxonomy=InputImagesTaxonomy.BFL_FLUX_2,
+ input_images=input_images,
+ )
+
+ expected_keys = ["input_image"] + [f"input_image_{index_key}" for index_key in range(2, 9)]
+ assert sorted(result.keys()) == sorted(expected_keys)
+ assert result["input_image"] == "https://example.com/img_0.png"
+ assert result["input_image_8"] == "https://example.com/img_7.png"
+ assert "input_image_9" not in result
+
+ @pytest.mark.asyncio
+ async def test_gpt_image_collects_base64_data_urls(self, mocker: MockerFixture) -> None:
+ """GPT Image returns all prepped images as a list of base64 data URLs under the `image` key."""
+ input_images = make_input_images(2)
+ prepped_files: list[PreparedFile] = [
+ make_prepped_base64("Zmlyc3Q="),
+ make_prepped_base64("c2Vjb25k"),
+ ]
+ prep_mock = mocker.patch(
+ "pipelex.cogt.img_gen.img_gen_args_factory.prep_prompt_images",
+ new_callable=mocker.AsyncMock,
+ return_value=prepped_files,
+ )
+
+ result = await ImgGenArgsFactory.make_args_from_input_images(
+ input_images_taxonomy=InputImagesTaxonomy.GPT_IMAGE,
+ input_images=input_images,
+ )
+
+ assert result == {
+ "image": [
+ "data:image/png;base64,Zmlyc3Q=",
+ "data:image/png;base64,c2Vjb25k",
+ ],
+ }
+ prep_mock.assert_awaited_once_with(prompt_images=input_images, is_http_url_enabled=False)
+
+ @pytest.mark.asyncio
+ async def test_gpt_image_rejects_http_url_prepped_files(self, mocker: MockerFixture) -> None:
+ """GPT Image raises a parameter error when a prepped file is an HTTP URL instead of base64."""
+ input_images = make_input_images(1)
+ prepped_files: list[PreparedFile] = [PreparedFileHttpUrl(url="https://example.com/hosted.png")]
+ mocker.patch(
+ "pipelex.cogt.img_gen.img_gen_args_factory.prep_prompt_images",
+ new_callable=mocker.AsyncMock,
+ return_value=prepped_files,
+ )
+
+ with pytest.raises(ImgGenParameterError) as exc_info:
+ await ImgGenArgsFactory.make_args_from_input_images(
+ input_images_taxonomy=InputImagesTaxonomy.GPT_IMAGE,
+ input_images=input_images,
+ )
+
+ assert "requires base64 data URLs" in str(exc_info.value)
+
+ @pytest.mark.asyncio
+ async def test_none_taxonomy_with_images_raises(self) -> None:
+ """The NONE taxonomy rejects any provided input images."""
+ with pytest.raises(ImgGenParameterError) as exc_info:
+ await ImgGenArgsFactory.make_args_from_input_images(
+ input_images_taxonomy=InputImagesTaxonomy.NONE,
+ input_images=make_input_images(1),
+ )
+
+ assert "does not support image inputs" in str(exc_info.value)
+
+ @pytest.mark.asyncio
+ @pytest.mark.parametrize(
+ "input_images_taxonomy",
+ [
+ InputImagesTaxonomy.GPT_IMAGE,
+ InputImagesTaxonomy.BFL_FLUX_2,
+ InputImagesTaxonomy.NONE,
+ ],
+ )
+ @pytest.mark.parametrize("input_images", [None, []])
+ async def test_no_input_images_returns_empty_without_prepping(
+ self,
+ mocker: MockerFixture,
+ input_images_taxonomy: InputImagesTaxonomy,
+ input_images: list[PromptImage] | None,
+ ) -> None:
+ """None or empty input images yield an empty dict and never trigger image preparation."""
+ prep_mock = mocker.patch(
+ "pipelex.cogt.img_gen.img_gen_args_factory.prep_prompt_images",
+ new_callable=mocker.AsyncMock,
+ )
+
+ result = await ImgGenArgsFactory.make_args_from_input_images(
+ input_images_taxonomy=input_images_taxonomy,
+ input_images=input_images,
+ )
+
+ assert result == {}
+ prep_mock.assert_not_awaited()
+
+ @pytest.mark.asyncio
+ @pytest.mark.parametrize(
+ ("input_images_taxonomy", "expected_provider_name"),
+ [
+ (InputImagesTaxonomy.GPT_IMAGE, "GPT Image"),
+ (InputImagesTaxonomy.BFL_FLUX_2, "Flux 2"),
+ ],
+ )
+ async def test_unexpected_prepped_file_type_raises_naming_provider(
+ self,
+ mocker: MockerFixture,
+ input_images_taxonomy: InputImagesTaxonomy,
+ expected_provider_name: str,
+ ) -> None:
+ """A prepped file outside the expected shapes raises a parameter error naming the provider and the type."""
+ prepped_files: list[PreparedFile] = [PreparedFileLocalPath(path="some_dir/local_image.png")]
+ mocker.patch(
+ "pipelex.cogt.img_gen.img_gen_args_factory.prep_prompt_images",
+ new_callable=mocker.AsyncMock,
+ return_value=prepped_files,
+ )
+
+ with pytest.raises(ImgGenParameterError) as exc_info:
+ await ImgGenArgsFactory.make_args_from_input_images(
+ input_images_taxonomy=input_images_taxonomy,
+ input_images=make_input_images(1),
+ )
+
+ error_message = str(exc_info.value)
+ assert expected_provider_name in error_message
+ assert "PreparedFileLocalPath" in error_message
+
+ @pytest.mark.parametrize(
+ ("input_fidelity", "expected_value"),
+ [
+ (InputFidelity.LOW, "low"),
+ (InputFidelity.HIGH, "high"),
+ ],
+ )
+ def test_input_fidelity_gpt_image_legacy_emits_value(self, input_fidelity: InputFidelity, expected_value: str) -> None:
+ """Legacy GPT Image taxonomy maps input fidelity through the OpenAI factory to its string value."""
+ result = ImgGenArgsFactory.make_args_from_input_fidelity(
+ input_fidelity_taxonomy=InputFidelityTaxonomy.GPT_IMAGE_LEGACY,
+ input_fidelity=input_fidelity,
+ model_name="gpt-image-1",
+ )
+
+ assert result == {"input_fidelity": expected_value}
+
+ @pytest.mark.parametrize(
+ "input_fidelity_taxonomy",
+ [
+ InputFidelityTaxonomy.GPT_IMAGE_LEGACY,
+ InputFidelityTaxonomy.UNAVAILABLE,
+ ],
+ )
+ def test_input_fidelity_none_returns_empty_for_all_taxonomies(self, input_fidelity_taxonomy: InputFidelityTaxonomy) -> None:
+ """A missing input fidelity yields an empty dict regardless of taxonomy, even UNAVAILABLE."""
+ result = ImgGenArgsFactory.make_args_from_input_fidelity(
+ input_fidelity_taxonomy=input_fidelity_taxonomy,
+ input_fidelity=None,
+ model_name="any-model",
+ )
+
+ assert result == {}
+
+ @pytest.mark.asyncio
+ async def test_input_fidelity_without_rule_raises_with_model_name(self) -> None:
+ """A job carrying input_fidelity fails when the model rules omit the input_fidelity topic entirely."""
+ model_rules: ImgGenModelRules = {
+ ImgGenArgTopic.PROMPT: PromptTaxonomy.POSITIVE_ONLY,
+ ImgGenArgTopic.NUM_IMAGES: NumImagesTaxonomy.FAL,
+ }
+ img_gen_job = make_img_gen_job(input_fidelity=InputFidelity.HIGH)
+
+ with pytest.raises(ImgGenParameterError) as exc_info:
+ await ImgGenArgsFactory.make_args_for_model(
+ model_rules=model_rules,
+ img_gen_job=img_gen_job,
+ nb_images=1,
+ model_id="fal-ai/flux/dev",
+ model_name="flux-dev",
+ )
+
+ error_message = str(exc_info.value)
+ assert "flux-dev" in error_message
+ assert "does not support input_fidelity" in error_message
diff --git a/tests/unit/pipelex/cogt/img_gen/test_img_gen_args_prompt_and_output.py b/tests/unit/pipelex/cogt/img_gen/test_img_gen_args_prompt_and_output.py
new file mode 100644
index 000000000..54f9a6c69
--- /dev/null
+++ b/tests/unit/pipelex/cogt/img_gen/test_img_gen_args_prompt_and_output.py
@@ -0,0 +1,272 @@
+"""Tests for ImgGenArgsFactory prompt, num_images, specific, background, and output_format mappings."""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any
+
+import pytest
+
+from pipelex.cogt.exceptions import ImgGenParameterError
+from pipelex.cogt.img_gen.img_gen_args_factory import ImgGenArgsFactory
+from pipelex.cogt.img_gen.img_gen_job_components import AspectRatio, Background
+from pipelex.cogt.img_gen.img_gen_model_rules import (
+ AspectRatioTaxonomy,
+ BackgroundTaxonomy,
+ ImgGenArgTopic,
+ ImgGenModelRules,
+ InferenceTaxonomy,
+ NumImagesTaxonomy,
+ OutputCompressionTaxonomy,
+ OutputFormatTaxonomy,
+ PromptTaxonomy,
+ SafetyCheckerTaxonomy,
+ SpecificTaxonomy,
+)
+from pipelex.tools.misc.image_utils import ImageFormat
+from tests.unit.pipelex.cogt.img_gen.conftest import make_img_gen_job
+
+if TYPE_CHECKING:
+ from pytest_mock import MockerFixture
+
+
+class TestImgGenArgsPromptAndOutput:
+ def test_positive_only_drops_negative_prompt_with_warning(self, mocker: MockerFixture) -> None:
+ """POSITIVE_ONLY taxonomy silently drops the negative prompt and logs a warning."""
+ mock_log = mocker.patch("pipelex.cogt.img_gen.img_gen_args_factory.log")
+
+ result = ImgGenArgsFactory.make_args_from_prompt(
+ prompt_taxonomy=PromptTaxonomy.POSITIVE_ONLY,
+ positive_text="a red bicycle",
+ negative_text="blurry, low quality",
+ )
+
+ assert result == {"prompt": "a red bicycle"}
+ mock_log.warning.assert_called_once()
+ warning_message = mock_log.warning.call_args.args[0]
+ assert "negative prompt" in warning_message.lower()
+
+ def test_positive_only_without_negative_does_not_warn(self, mocker: MockerFixture) -> None:
+ """POSITIVE_ONLY taxonomy emits no warning when no negative prompt is provided."""
+ mock_log = mocker.patch("pipelex.cogt.img_gen.img_gen_args_factory.log")
+
+ result = ImgGenArgsFactory.make_args_from_prompt(
+ prompt_taxonomy=PromptTaxonomy.POSITIVE_ONLY,
+ positive_text="a red bicycle",
+ negative_text=None,
+ )
+
+ assert result == {"prompt": "a red bicycle"}
+ mock_log.warning.assert_not_called()
+
+ @pytest.mark.parametrize(
+ ("negative_text", "expected_args"),
+ [
+ (None, {"prompt": "a red bicycle"}),
+ ("", {"prompt": "a red bicycle"}),
+ ("blurry", {"prompt": "a red bicycle", "negative_prompt": "blurry"}),
+ ],
+ )
+ def test_with_negative_emits_negative_prompt_only_when_set(self, negative_text: str | None, expected_args: dict[str, Any]) -> None:
+ """WITH_NEGATIVE taxonomy includes `negative_prompt` only when negative text is non-empty."""
+ result = ImgGenArgsFactory.make_args_from_prompt(
+ prompt_taxonomy=PromptTaxonomy.WITH_NEGATIVE,
+ positive_text="a red bicycle",
+ negative_text=negative_text,
+ )
+
+ assert result == expected_args
+
+ @pytest.mark.parametrize(
+ ("num_images_taxonomy", "expected_args"),
+ [
+ (NumImagesTaxonomy.FAL, {"num_images": 3}),
+ (NumImagesTaxonomy.GPT_IMAGE, {"n": 3}),
+ ],
+ )
+ def test_num_images_maps_to_provider_parameter_name(self, num_images_taxonomy: NumImagesTaxonomy, expected_args: dict[str, Any]) -> None:
+ """FAL uses `num_images` while GPT Image uses `n` for the image count."""
+ result = ImgGenArgsFactory.make_args_from_num_images(
+ num_images_taxonomy=num_images_taxonomy,
+ nb_images=3,
+ )
+
+ assert result == expected_args
+
+ def test_specific_fal_disables_sync_mode(self) -> None:
+ """FAL specific taxonomy always emits `sync_mode = False`."""
+ result = ImgGenArgsFactory.make_args_from_specific(specific_taxonomy=SpecificTaxonomy.FAL)
+
+ assert result == {"sync_mode": False}
+
+ @pytest.mark.parametrize(
+ ("background", "expected_value"),
+ [
+ (Background.TRANSPARENT, "transparent"),
+ (Background.OPAQUE, "opaque"),
+ (Background.AUTO, "auto"),
+ ],
+ )
+ def test_background_available_emits_background_value(self, background: Background, expected_value: str) -> None:
+ """AVAILABLE background taxonomy forwards the background value as-is."""
+ result = ImgGenArgsFactory.make_args_from_background(
+ background_taxonomy=BackgroundTaxonomy.AVAILABLE,
+ background=background,
+ model_name="gpt-image-1",
+ )
+
+ assert result == {"background": expected_value}
+
+ def test_background_unavailable_transparent_raises_with_model_name(self) -> None:
+ """UNAVAILABLE background taxonomy rejects transparent backgrounds and names the model."""
+ with pytest.raises(ImgGenParameterError) as exc_info:
+ ImgGenArgsFactory.make_args_from_background(
+ background_taxonomy=BackgroundTaxonomy.UNAVAILABLE,
+ background=Background.TRANSPARENT,
+ model_name="flux-dev",
+ )
+
+ error_message = str(exc_info.value)
+ assert "flux-dev" in error_message
+ assert "transparent background" in error_message
+
+ @pytest.mark.parametrize(
+ "background",
+ [
+ Background.OPAQUE,
+ Background.AUTO,
+ ],
+ )
+ def test_background_unavailable_non_transparent_emits_nothing(self, background: Background) -> None:
+ """UNAVAILABLE background taxonomy skips the parameter for non-transparent backgrounds."""
+ result = ImgGenArgsFactory.make_args_from_background(
+ background_taxonomy=BackgroundTaxonomy.UNAVAILABLE,
+ background=background,
+ model_name="flux-dev",
+ )
+
+ assert result == {}
+
+ @pytest.mark.parametrize(
+ ("output_format", "expected_value"),
+ [
+ (ImageFormat.PNG, "png"),
+ (ImageFormat.JPEG, "jpeg"),
+ ],
+ )
+ def test_output_format_sdxl_uses_format_key(self, output_format: ImageFormat, expected_value: str) -> None:
+ """SDXL taxonomy emits png/jpeg under the `format` key."""
+ result = ImgGenArgsFactory.make_args_from_output_format(
+ output_format_taxonomy=OutputFormatTaxonomy.SDXL,
+ output_format=output_format,
+ )
+
+ assert result == {"format": expected_value}
+
+ def test_output_format_sdxl_rejects_webp(self) -> None:
+ """SDXL taxonomy rejects WebP output and names SDXL in the error."""
+ with pytest.raises(ImgGenParameterError) as exc_info:
+ ImgGenArgsFactory.make_args_from_output_format(
+ output_format_taxonomy=OutputFormatTaxonomy.SDXL,
+ output_format=ImageFormat.WEBP,
+ )
+
+ assert "SDXL" in str(exc_info.value)
+
+ @pytest.mark.parametrize(
+ ("output_format", "expected_value"),
+ [
+ (ImageFormat.PNG, "png"),
+ (ImageFormat.JPEG, "jpeg"),
+ ],
+ )
+ def test_output_format_flux_1_uses_output_format_key(self, output_format: ImageFormat, expected_value: str) -> None:
+ """Flux 1 taxonomy emits png/jpeg under the `output_format` key."""
+ result = ImgGenArgsFactory.make_args_from_output_format(
+ output_format_taxonomy=OutputFormatTaxonomy.FLUX_1,
+ output_format=output_format,
+ )
+
+ assert result == {"output_format": expected_value}
+
+ def test_output_format_flux_1_rejects_webp(self) -> None:
+ """Flux 1 taxonomy rejects WebP output and names Flux 1 in the error."""
+ with pytest.raises(ImgGenParameterError) as exc_info:
+ ImgGenArgsFactory.make_args_from_output_format(
+ output_format_taxonomy=OutputFormatTaxonomy.FLUX_1,
+ output_format=ImageFormat.WEBP,
+ )
+
+ assert "Flux 1" in str(exc_info.value)
+
+ @pytest.mark.parametrize(
+ "output_format_taxonomy",
+ [
+ OutputFormatTaxonomy.FLUX_2,
+ OutputFormatTaxonomy.GPT_IMAGE_LEGACY,
+ ],
+ )
+ @pytest.mark.parametrize(
+ "output_format",
+ [
+ ImageFormat.PNG,
+ ImageFormat.JPEG,
+ ImageFormat.WEBP,
+ ],
+ )
+ def test_output_format_passthrough_taxonomies_accept_all_formats(
+ self,
+ output_format_taxonomy: OutputFormatTaxonomy,
+ output_format: ImageFormat,
+ ) -> None:
+ """Flux 2 and legacy GPT Image taxonomies pass every image format through, including WebP."""
+ result = ImgGenArgsFactory.make_args_from_output_format(
+ output_format_taxonomy=output_format_taxonomy,
+ output_format=output_format,
+ )
+
+ assert result == {"output_format": output_format.value}
+
+ @pytest.mark.asyncio
+ async def test_full_fal_flux_job_builds_complete_args_dict(self) -> None:
+ """A full FAL Flux rule set dispatches every topic and assembles the complete args dict."""
+ model_rules: ImgGenModelRules = {
+ ImgGenArgTopic.PROMPT: PromptTaxonomy.WITH_NEGATIVE,
+ ImgGenArgTopic.NUM_IMAGES: NumImagesTaxonomy.FAL,
+ ImgGenArgTopic.ASPECT_RATIO: AspectRatioTaxonomy.FLUX,
+ ImgGenArgTopic.BACKGROUND: BackgroundTaxonomy.UNAVAILABLE,
+ ImgGenArgTopic.INFERENCE: InferenceTaxonomy.FLUX,
+ ImgGenArgTopic.SAFETY_CHECKER: SafetyCheckerTaxonomy.AVAILABLE,
+ ImgGenArgTopic.OUTPUT_FORMAT: OutputFormatTaxonomy.FLUX_1,
+ ImgGenArgTopic.OUTPUT_COMPRESSION: OutputCompressionTaxonomy.UNAVAILABLE,
+ ImgGenArgTopic.SPECIFIC: SpecificTaxonomy.FAL,
+ }
+ img_gen_job = make_img_gen_job(
+ negative_text="blurry",
+ aspect_ratio=AspectRatio.LANDSCAPE_16_9,
+ nb_steps=20,
+ guidance_scale=3.5,
+ is_moderated=False,
+ safety_tolerance=2,
+ output_format=ImageFormat.JPEG,
+ )
+
+ result = await ImgGenArgsFactory.make_args_for_model(
+ model_rules=model_rules,
+ img_gen_job=img_gen_job,
+ nb_images=2,
+ model_id="fal-ai/flux/dev",
+ model_name="flux-dev",
+ )
+
+ assert result == {
+ "prompt": "A test prompt",
+ "negative_prompt": "blurry",
+ "num_images": 2,
+ "image_size": "landscape_16_9",
+ "num_inference_steps": 20,
+ "guidance_scale": 3.5,
+ "enable_safety_checker": False,
+ "safety_tolerance": 2,
+ "output_format": "jpeg",
+ "sync_mode": False,
+ }
diff --git a/tests/unit/pipelex/cogt/img_gen/test_img_gen_setting_defaults.py b/tests/unit/pipelex/cogt/img_gen/test_img_gen_setting_defaults.py
new file mode 100644
index 000000000..34b014bc1
--- /dev/null
+++ b/tests/unit/pipelex/cogt/img_gen/test_img_gen_setting_defaults.py
@@ -0,0 +1,18 @@
+from pipelex.cogt.img_gen.img_gen_setting import ImgGenSetting
+
+
+class TestImgGenSettingDefaults:
+ def test_is_moderated_defaults_to_none(self):
+ """is_moderated must default to None ("no explicit choice"), so workers omit the moderation/safety-checker
+ param and the provider's own default applies (OpenAI gpt-image: "auto", i.e. standard filtering).
+
+ A concrete False default would explicitly request reduced moderation for every deck alias/handle
+ resolution that constructs a setting without the field — a silent safety downgrade.
+ """
+ setting = ImgGenSetting(model="some-img-gen-model")
+ assert setting.is_moderated is None
+
+ def test_explicit_is_moderated_round_trips(self):
+ """An explicit choice is preserved as-is for both values."""
+ assert ImgGenSetting(model="some-img-gen-model", is_moderated=True).is_moderated is True
+ assert ImgGenSetting(model="some-img-gen-model", is_moderated=False).is_moderated is False
diff --git a/tests/unit/pipelex/cogt/img_gen/test_img_gen_worker_factory.py b/tests/unit/pipelex/cogt/img_gen/test_img_gen_worker_factory.py
new file mode 100644
index 000000000..62ae74d05
--- /dev/null
+++ b/tests/unit/pipelex/cogt/img_gen/test_img_gen_worker_factory.py
@@ -0,0 +1,262 @@
+"""Routing tests for ImgGenWorkerFactory: each plugin SDK string must build the right image
+generation worker with the right SDK client, completions factory, and reporting delegate,
+while caching the SDK instance in the plugin SDK registry.
+"""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
+import pytest
+
+from pipelex.cogt.img_gen.img_gen_worker_factory import ImgGenWorkerFactory
+from pipelex.cogt.llm.thinking_mode import ThinkingMode
+from pipelex.cogt.model_backends.backend import InferenceBackend
+from pipelex.cogt.model_backends.model_spec import InferenceModelSpec
+from pipelex.cogt.model_backends.model_type import ModelType
+from pipelex.cogt.usage.cost_category import CostCategory
+from pipelex.exceptions import MissingDependencyError
+from pipelex.plugins.blackboxai.blackboxai_completions_factory import BlackboxaiCompletionsFactory
+from pipelex.plugins.gateway.gateway_completions_factory import GatewayCompletionsFactory
+from pipelex.plugins.openai.openai_completions_factory import OpenAICompletionsFactory
+from pipelex.plugins.openrouter.openrouter_completions_factory import OpenRouterCompletionsFactory
+from pipelex.plugins.plugin_sdk_registry import Plugin, PluginSdkRegistry
+
+if TYPE_CHECKING:
+ from pytest_mock import MockerFixture
+
+FACTORY_MODULE = "pipelex.cogt.img_gen.img_gen_worker_factory"
+
+GATEWAY_CLIENT = "pipelex.plugins.gateway.gateway_factory.GatewayFactory.make_portkey_client"
+GATEWAY_COMPLETIONS_CLIENT = (
+ "pipelex.plugins.gateway.gateway_completions_factory.GatewayCompletionsFactory.make_portkey_openai_client_for_completions"
+)
+OPENAI_CLIENT = "pipelex.plugins.openai.openai_client_factory.OpenAIClientFactory.make_openai_client"
+GOOGLE_CLIENT = "pipelex.plugins.google.google_factory.GoogleFactory.make_google_client"
+HUGGINGFACE_PROVIDER = "pipelex.plugins.huggingface.huggingface_factory.HuggingFaceFactory.make_huggingface_inference_provider"
+
+GATEWAY_WORKER = "pipelex.plugins.gateway.gateway_img_gen_worker.GatewayImgGenWorker"
+FAL_WORKER = "pipelex.plugins.fal.fal_img_gen_worker.FalImgGenWorker"
+HUGGINGFACE_WORKER = "pipelex.plugins.huggingface.huggingface_img_gen_worker.HuggingFaceImgGenWorker"
+OPENAI_WORKER = "pipelex.plugins.openai.openai_img_gen_worker.OpenAIImgGenWorker"
+COMPLETIONS_WORKER = "pipelex.plugins.openai.openai_completions_img_gen_worker.OpenAICompletionsImgGenWorker"
+AZURE_WORKER = "pipelex.plugins.azure_rest.azure_img_gen_worker.AzureImgGenWorker"
+GOOGLE_WORKER = "pipelex.plugins.google.google_img_gen_worker.GoogleImgGenWorker"
+
+
+def make_img_gen_model_spec(sdk: str, backend_name: str = "test_backend", variant: str | None = None) -> InferenceModelSpec:
+ return InferenceModelSpec(
+ backend_name=backend_name,
+ name="test-img-model",
+ sdk=sdk,
+ variant=variant,
+ model_type=ModelType.IMG_GEN,
+ model_id="test-img-model-id",
+ inputs=["text"],
+ outputs=["image"],
+ costs={CostCategory.INPUT: 1.0, CostCategory.OUTPUT: 2.0},
+ thinking_mode=ThinkingMode.NONE,
+ max_tokens=None,
+ max_prompt_images=None,
+ )
+
+
+def make_backend(name: str = "test_backend") -> InferenceBackend:
+ return InferenceBackend(
+ name=name,
+ api_key="test-key",
+ extra_config={"api_version": "2024-02-01"},
+ )
+
+
+def patch_hub_getters(
+ mocker: MockerFixture,
+ backend: InferenceBackend,
+) -> PluginSdkRegistry:
+ """Patch hub getters at the worker-factory namespace; return the fresh SDK registry."""
+ registry = PluginSdkRegistry()
+ models_manager = mocker.MagicMock()
+ models_manager.get_required_inference_backend.return_value = backend
+ mocker.patch(f"{FACTORY_MODULE}.get_models_manager", return_value=models_manager)
+ plugin_manager = mocker.MagicMock()
+ plugin_manager.plugin_sdk_registry = registry
+ mocker.patch(f"{FACTORY_MODULE}.get_plugin_manager", return_value=plugin_manager)
+ return registry
+
+
+class TestImgGenWorkerFactory:
+ @pytest.mark.parametrize(
+ ("sdk", "client_target", "worker_target", "factory_cls", "http_flag", "passes_plugin"),
+ [
+ pytest.param("gateway_img_gen", GATEWAY_CLIENT, GATEWAY_WORKER, None, None, False, id="gateway_img_gen"),
+ pytest.param("openai_img_gen", OPENAI_CLIENT, OPENAI_WORKER, None, None, True, id="openai_img_gen"),
+ pytest.param("blackboxai_img_gen", OPENAI_CLIENT, COMPLETIONS_WORKER, BlackboxaiCompletionsFactory, True, True, id="blackboxai_img_gen"),
+ pytest.param("openrouter_img_gen", OPENAI_CLIENT, COMPLETIONS_WORKER, OpenRouterCompletionsFactory, True, True, id="openrouter_img_gen"),
+ pytest.param(
+ "gateway_completions",
+ GATEWAY_COMPLETIONS_CLIENT,
+ COMPLETIONS_WORKER,
+ GatewayCompletionsFactory,
+ False,
+ True,
+ id="gateway_completions",
+ ),
+ pytest.param("google", GOOGLE_CLIENT, GOOGLE_WORKER, None, None, False, id="google"),
+ ],
+ )
+ def test_routing_builds_expected_worker(
+ self,
+ mocker: MockerFixture,
+ sdk: str,
+ client_target: str,
+ worker_target: str,
+ factory_cls: type | None,
+ http_flag: bool | None,
+ passes_plugin: bool,
+ ) -> None:
+ """Each SDK string routes to its image generation worker with the SDK client wired in."""
+ backend = make_backend()
+ patch_hub_getters(mocker, backend=backend)
+ inference_model = make_img_gen_model_spec(sdk=sdk)
+ sdk_client = mocker.MagicMock(name="sdk_client")
+ client_factory_mock = mocker.patch(client_target, return_value=sdk_client)
+ worker_cls_mock = mocker.patch(worker_target)
+ reporting_delegate = mocker.MagicMock(name="reporting_delegate")
+
+ worker = ImgGenWorkerFactory.make_img_gen_worker(inference_model=inference_model, reporting_delegate=reporting_delegate)
+
+ client_factory_mock.assert_called_once()
+ client_kwargs = client_factory_mock.call_args.kwargs
+ assert client_kwargs["backend"] is backend
+ if passes_plugin:
+ assert client_kwargs["plugin"] == Plugin(sdk=sdk, backend="test_backend", variant=None)
+
+ worker_cls_mock.assert_called_once()
+ worker_kwargs = worker_cls_mock.call_args.kwargs
+ assert worker_kwargs["sdk_instance"] is sdk_client
+ assert worker_kwargs["inference_model"] is inference_model
+ assert worker_kwargs["reporting_delegate"] is reporting_delegate
+ assert worker is worker_cls_mock.return_value
+
+ if factory_cls is not None:
+ built_factory = worker_kwargs["openai_completions_factory"]
+ assert type(built_factory) is factory_cls
+ assert isinstance(built_factory, OpenAICompletionsFactory)
+ assert built_factory.is_http_url_enabled is http_flag
+
+ def test_fal_builds_async_client_with_api_key(self, mocker: MockerFixture) -> None:
+ """The fal SDK string builds a FalImgGenWorker around an AsyncClient keyed with the backend api_key."""
+ backend = make_backend()
+ patch_hub_getters(mocker, backend=backend)
+ inference_model = make_img_gen_model_spec(sdk="fal")
+ fal_client_mock = mocker.patch("fal_client.AsyncClient")
+ worker_cls_mock = mocker.patch(FAL_WORKER)
+
+ worker = ImgGenWorkerFactory.make_img_gen_worker(inference_model=inference_model)
+
+ fal_client_mock.assert_called_once_with(key="test-key")
+ worker_cls_mock.assert_called_once()
+ worker_kwargs = worker_cls_mock.call_args.kwargs
+ assert worker_kwargs["sdk_instance"] is fal_client_mock.return_value
+ assert worker_kwargs["inference_model"] is inference_model
+ assert worker_kwargs["reporting_delegate"] is None
+ assert worker is worker_cls_mock.return_value
+
+ def test_huggingface_with_variant_resolves_provider(self, mocker: MockerFixture) -> None:
+ """A huggingface variant is mapped to a provider via HuggingFaceFactory and passed to the client."""
+ backend = make_backend()
+ patch_hub_getters(mocker, backend=backend)
+ inference_model = make_img_gen_model_spec(sdk="huggingface_img_gen", variant="fal-ai")
+ provider_mock = mocker.patch(HUGGINGFACE_PROVIDER, return_value="fal-ai")
+ hf_client_mock = mocker.patch("huggingface_hub.AsyncInferenceClient")
+ worker_cls_mock = mocker.patch(HUGGINGFACE_WORKER)
+
+ ImgGenWorkerFactory.make_img_gen_worker(inference_model=inference_model)
+
+ provider_mock.assert_called_once_with(provider_str="fal-ai")
+ hf_client_mock.assert_called_once_with(provider="fal-ai", token="test-key")
+ worker_cls_mock.assert_called_once()
+ assert worker_cls_mock.call_args.kwargs["sdk_instance"] is hf_client_mock.return_value
+
+ def test_huggingface_without_variant_uses_auto_provider(self, mocker: MockerFixture) -> None:
+ """Without a variant, the huggingface client gets the auto provider policy."""
+ backend = make_backend()
+ patch_hub_getters(mocker, backend=backend)
+ inference_model = make_img_gen_model_spec(sdk="huggingface_img_gen", variant=None)
+ provider_mock = mocker.patch(HUGGINGFACE_PROVIDER)
+ hf_client_mock = mocker.patch("huggingface_hub.AsyncInferenceClient")
+ worker_cls_mock = mocker.patch(HUGGINGFACE_WORKER)
+
+ ImgGenWorkerFactory.make_img_gen_worker(inference_model=inference_model)
+
+ provider_mock.assert_not_called()
+ hf_client_mock.assert_called_once_with(provider="auto", token="test-key")
+ worker_cls_mock.assert_called_once()
+ assert worker_cls_mock.call_args.kwargs["sdk_instance"] is hf_client_mock.return_value
+
+ def test_azure_rest_builds_worker_without_registry(self, mocker: MockerFixture) -> None:
+ """The azure_rest SDK constructs the worker directly: no SDK instance is registered."""
+ backend = make_backend()
+ registry = patch_hub_getters(mocker, backend=backend)
+ inference_model = make_img_gen_model_spec(sdk="azure_rest_img_gen")
+ worker_cls_mock = mocker.patch(AZURE_WORKER)
+ reporting_delegate = mocker.MagicMock(name="reporting_delegate")
+
+ worker = ImgGenWorkerFactory.make_img_gen_worker(inference_model=inference_model, reporting_delegate=reporting_delegate)
+
+ worker_cls_mock.assert_called_once_with(
+ plugin=Plugin(sdk="azure_rest_img_gen", backend="test_backend", variant=None),
+ inference_model=inference_model,
+ reporting_delegate=reporting_delegate,
+ )
+ assert worker is worker_cls_mock.return_value
+ assert registry.root == {}
+
+ def test_cached_sdk_instance_skips_client_factory(self, mocker: MockerFixture) -> None:
+ """A pre-seeded registry entry is reused: the client factory must not be called."""
+ backend = make_backend()
+ registry = patch_hub_getters(mocker, backend=backend)
+ inference_model = make_img_gen_model_spec(sdk="gateway_img_gen")
+ plugin = Plugin.make_for_inference_model(inference_model=inference_model)
+ cached_client = mocker.MagicMock(name="cached_client")
+ registry.set_sdk_instance(plugin=plugin, sdk_instance=cached_client)
+ client_factory_mock = mocker.patch(GATEWAY_CLIENT)
+ worker_cls_mock = mocker.patch(GATEWAY_WORKER)
+
+ ImgGenWorkerFactory.make_img_gen_worker(inference_model=inference_model)
+
+ client_factory_mock.assert_not_called()
+ worker_cls_mock.assert_called_once()
+ assert worker_cls_mock.call_args.kwargs["sdk_instance"] is cached_client
+
+ @pytest.mark.parametrize(
+ ("sdk", "expected_extra"),
+ [
+ pytest.param("fal", "fal", id="fal"),
+ pytest.param("google", "google", id="google"),
+ ],
+ )
+ def test_missing_dependency_raises(self, mocker: MockerFixture, sdk: str, expected_extra: str) -> None:
+ """When the optional SDK is absent, the factory raises MissingDependencyError naming the extra."""
+ backend = make_backend()
+ patch_hub_getters(mocker, backend=backend)
+ inference_model = make_img_gen_model_spec(sdk=sdk)
+ mocker.patch("importlib.util.find_spec", return_value=None)
+
+ with pytest.raises(MissingDependencyError) as exc_info:
+ ImgGenWorkerFactory.make_img_gen_worker(inference_model=inference_model)
+
+ assert exc_info.value.extra_name == expected_extra
+ assert f"pipelex[{expected_extra}]" in str(exc_info.value)
+
+ def test_unknown_sdk_raises_not_implemented(self, mocker: MockerFixture) -> None:
+ """An unrecognized SDK string raises NotImplementedError naming the plugin."""
+ backend = make_backend()
+ patch_hub_getters(mocker, backend=backend)
+ inference_model = make_img_gen_model_spec(sdk="definitely_not_an_sdk")
+
+ with pytest.raises(NotImplementedError) as exc_info:
+ ImgGenWorkerFactory.make_img_gen_worker(inference_model=inference_model)
+
+ assert "definitely_not_an_sdk" in str(exc_info.value)
+ assert "not supported for image generation" in str(exc_info.value)
diff --git a/tests/unit/pipelex/cogt/llm/test_llm_worker_factory.py b/tests/unit/pipelex/cogt/llm/test_llm_worker_factory.py
new file mode 100644
index 000000000..f1298f83b
--- /dev/null
+++ b/tests/unit/pipelex/cogt/llm/test_llm_worker_factory.py
@@ -0,0 +1,311 @@
+"""Routing tests for LLMWorkerFactory: each plugin SDK string must build the right worker
+with the right SDK client, completions/responses factory, and reporting delegate, while
+caching the SDK instance in the plugin SDK registry.
+"""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
+import pytest
+
+from pipelex.cogt.llm.llm_worker_factory import LLMWorkerFactory
+from pipelex.cogt.llm.thinking_mode import ThinkingMode
+from pipelex.cogt.model_backends.backend import InferenceBackend
+from pipelex.cogt.model_backends.model_spec import InferenceModelSpec
+from pipelex.cogt.model_backends.model_type import ModelType
+from pipelex.cogt.usage.cost_category import CostCategory
+from pipelex.exceptions import MissingDependencyError
+from pipelex.plugins.gateway.gateway_completions_factory import GatewayCompletionsFactory
+from pipelex.plugins.gateway.gateway_responses_factory import GatewayResponsesFactory
+from pipelex.plugins.mistral.mistral_factory import MistralFactory
+from pipelex.plugins.openai.openai_completions_factory import OpenAICompletionsFactory
+from pipelex.plugins.openai.openai_responses_factory import OpenAIResponsesFactory
+from pipelex.plugins.plugin_sdk_registry import Plugin, PluginSdkRegistry
+from pipelex.plugins.portkey.portkey_completions_factory import PortkeyCompletionsFactory
+from pipelex.plugins.portkey.portkey_responses_factory import PortkeyResponsesFactory
+
+if TYPE_CHECKING:
+ from pytest_mock import MockerFixture
+
+FACTORY_MODULE = "pipelex.cogt.llm.llm_worker_factory"
+
+GATEWAY_COMPLETIONS_CLIENT = (
+ "pipelex.plugins.gateway.gateway_completions_factory.GatewayCompletionsFactory.make_portkey_openai_client_for_completions"
+)
+GATEWAY_RESPONSES_CLIENT = "pipelex.plugins.gateway.gateway_responses_factory.GatewayResponsesFactory.make_portkey_openai_client_for_responses"
+PORTKEY_COMPLETIONS_CLIENT = (
+ "pipelex.plugins.portkey.portkey_completions_factory.PortkeyCompletionsFactory.make_portkey_openai_client_for_completions"
+)
+PORTKEY_RESPONSES_CLIENT = "pipelex.plugins.portkey.portkey_responses_factory.PortkeyResponsesFactory.make_portkey_openai_client_for_responses"
+OPENAI_CLIENT = "pipelex.plugins.openai.openai_client_factory.OpenAIClientFactory.make_openai_client"
+ANTHROPIC_CLIENT = "pipelex.plugins.anthropic.anthropic_factory.AnthropicFactory.make_anthropic_client"
+MISTRAL_CLIENT = "pipelex.plugins.mistral.mistral_factory.MistralFactory.make_mistral_client"
+BEDROCK_CLIENT = "pipelex.plugins.bedrock.bedrock_factory.BedrockFactory.make_bedrock_client"
+GOOGLE_CLIENT = "pipelex.plugins.google.google_factory.GoogleFactory.make_google_client"
+
+COMPLETIONS_WORKER = "pipelex.plugins.openai.openai_completions_llm_worker.OpenAICompletionsLLMWorker"
+RESPONSES_WORKER = "pipelex.plugins.openai.openai_responses_llm_worker.OpenAIResponsesLLMWorker"
+ANTHROPIC_WORKER = "pipelex.plugins.anthropic.anthropic_llm_worker.AnthropicLLMWorker"
+MISTRAL_WORKER = "pipelex.plugins.mistral.mistral_llm_worker.MistralLLMWorker"
+BEDROCK_WORKER = "pipelex.plugins.bedrock.bedrock_llm_worker.BedrockLLMWorker"
+GOOGLE_WORKER = "pipelex.plugins.google.google_llm_worker.GoogleLLMWorker"
+
+
+def make_llm_model_spec(sdk: str, backend_name: str = "test_backend") -> InferenceModelSpec:
+ return InferenceModelSpec(
+ backend_name=backend_name,
+ name="test-model",
+ sdk=sdk,
+ model_type=ModelType.LLM,
+ model_id="test-model-id",
+ inputs=["text"],
+ outputs=["text"],
+ costs={CostCategory.INPUT: 1.0, CostCategory.OUTPUT: 2.0},
+ thinking_mode=ThinkingMode.NONE,
+ max_tokens=None,
+ max_prompt_images=None,
+ )
+
+
+def make_backend(name: str = "test_backend") -> InferenceBackend:
+ return InferenceBackend(
+ name=name,
+ api_key="test-key",
+ extra_config={"region_name": "us-east-1"},
+ )
+
+
+def patch_hub_getters(
+ mocker: MockerFixture,
+ backend: InferenceBackend,
+) -> PluginSdkRegistry:
+ """Patch hub getters at the worker-factory namespace; return the fresh SDK registry."""
+ registry = PluginSdkRegistry()
+ models_manager = mocker.MagicMock()
+ models_manager.get_required_inference_backend.return_value = backend
+ mocker.patch(f"{FACTORY_MODULE}.get_models_manager", return_value=models_manager)
+ plugin_manager = mocker.MagicMock()
+ plugin_manager.plugin_sdk_registry = registry
+ mocker.patch(f"{FACTORY_MODULE}.get_plugin_manager", return_value=plugin_manager)
+ return registry
+
+
+class TestLLMWorkerFactory:
+ @pytest.mark.parametrize(
+ ("sdk", "client_target", "worker_target", "factory_field", "factory_cls", "http_flag", "passes_plugin", "expects_extra_config"),
+ [
+ pytest.param(
+ "gateway_completions",
+ GATEWAY_COMPLETIONS_CLIENT,
+ COMPLETIONS_WORKER,
+ "openai_completions_factory",
+ GatewayCompletionsFactory,
+ False,
+ True,
+ False,
+ id="gateway_completions",
+ ),
+ pytest.param(
+ "gateway_responses",
+ GATEWAY_RESPONSES_CLIENT,
+ RESPONSES_WORKER,
+ "openai_responses_factory",
+ GatewayResponsesFactory,
+ False,
+ True,
+ False,
+ id="gateway_responses",
+ ),
+ pytest.param(
+ "portkey_completions",
+ PORTKEY_COMPLETIONS_CLIENT,
+ COMPLETIONS_WORKER,
+ "openai_completions_factory",
+ PortkeyCompletionsFactory,
+ False,
+ True,
+ False,
+ id="portkey_completions",
+ ),
+ pytest.param(
+ "portkey_responses",
+ PORTKEY_RESPONSES_CLIENT,
+ RESPONSES_WORKER,
+ "openai_responses_factory",
+ PortkeyResponsesFactory,
+ False,
+ True,
+ False,
+ id="portkey_responses",
+ ),
+ pytest.param(
+ "openai",
+ OPENAI_CLIENT,
+ COMPLETIONS_WORKER,
+ "openai_completions_factory",
+ OpenAICompletionsFactory,
+ True,
+ True,
+ False,
+ id="openai",
+ ),
+ pytest.param(
+ "azure_openai",
+ OPENAI_CLIENT,
+ COMPLETIONS_WORKER,
+ "openai_completions_factory",
+ OpenAICompletionsFactory,
+ True,
+ True,
+ False,
+ id="azure_openai",
+ ),
+ pytest.param(
+ "openai_responses",
+ OPENAI_CLIENT,
+ RESPONSES_WORKER,
+ "openai_responses_factory",
+ OpenAIResponsesFactory,
+ True,
+ True,
+ False,
+ id="openai_responses",
+ ),
+ pytest.param(
+ "azure_openai_responses",
+ OPENAI_CLIENT,
+ RESPONSES_WORKER,
+ "openai_responses_factory",
+ OpenAIResponsesFactory,
+ True,
+ True,
+ False,
+ id="azure_openai_responses",
+ ),
+ pytest.param("anthropic", ANTHROPIC_CLIENT, ANTHROPIC_WORKER, None, None, None, True, True, id="anthropic"),
+ pytest.param("bedrock_anthropic", ANTHROPIC_CLIENT, ANTHROPIC_WORKER, None, None, None, True, True, id="bedrock_anthropic"),
+ pytest.param("mistral", MISTRAL_CLIENT, MISTRAL_WORKER, "mistral_factory", MistralFactory, None, False, False, id="mistral"),
+ pytest.param("bedrock_boto3", BEDROCK_CLIENT, BEDROCK_WORKER, None, None, None, True, False, id="bedrock_boto3"),
+ pytest.param("bedrock_aioboto3", BEDROCK_CLIENT, BEDROCK_WORKER, None, None, None, True, False, id="bedrock_aioboto3"),
+ pytest.param("google", GOOGLE_CLIENT, GOOGLE_WORKER, None, None, None, False, False, id="google"),
+ ],
+ )
+ def test_routing_builds_expected_worker(
+ self,
+ mocker: MockerFixture,
+ sdk: str,
+ client_target: str,
+ worker_target: str,
+ factory_field: str | None,
+ factory_cls: type | None,
+ http_flag: bool | None,
+ passes_plugin: bool,
+ expects_extra_config: bool,
+ ) -> None:
+ """Each SDK string routes to its worker class with the SDK client and factory wired in."""
+ backend = make_backend()
+ patch_hub_getters(mocker, backend=backend)
+ inference_model = make_llm_model_spec(sdk=sdk)
+ sdk_client = mocker.MagicMock(name="sdk_client")
+ client_factory_mock = mocker.patch(client_target, return_value=sdk_client)
+ worker_cls_mock = mocker.patch(worker_target)
+ reporting_delegate = mocker.MagicMock(name="reporting_delegate")
+
+ worker = LLMWorkerFactory.make_llm_worker(inference_model=inference_model, reporting_delegate=reporting_delegate)
+
+ client_factory_mock.assert_called_once()
+ client_kwargs = client_factory_mock.call_args.kwargs
+ assert client_kwargs["backend"] is backend
+ if passes_plugin:
+ assert client_kwargs["plugin"] == Plugin(sdk=sdk, backend="test_backend", variant=None)
+
+ worker_cls_mock.assert_called_once()
+ worker_kwargs = worker_cls_mock.call_args.kwargs
+ assert worker_kwargs["sdk_instance"] is sdk_client
+ assert worker_kwargs["inference_model"] is inference_model
+ assert worker_kwargs["reporting_delegate"] is reporting_delegate
+ assert worker is worker_cls_mock.return_value
+
+ if factory_field is not None:
+ assert factory_cls is not None
+ built_factory = worker_kwargs[factory_field]
+ assert type(built_factory) is factory_cls
+ if http_flag is not None:
+ assert isinstance(built_factory, (OpenAICompletionsFactory, OpenAIResponsesFactory))
+ assert built_factory.is_http_url_enabled is http_flag
+
+ if expects_extra_config:
+ assert worker_kwargs["extra_config"] is backend.extra_config
+
+ def test_cached_sdk_instance_skips_client_factory(self, mocker: MockerFixture) -> None:
+ """A pre-seeded registry entry is reused: the client factory must not be called."""
+ backend = make_backend()
+ registry = patch_hub_getters(mocker, backend=backend)
+ inference_model = make_llm_model_spec(sdk="openai")
+ plugin = Plugin.make_for_inference_model(inference_model=inference_model)
+ cached_client = mocker.MagicMock(name="cached_client")
+ registry.set_sdk_instance(plugin=plugin, sdk_instance=cached_client)
+ client_factory_mock = mocker.patch(OPENAI_CLIENT)
+ worker_cls_mock = mocker.patch(COMPLETIONS_WORKER)
+
+ LLMWorkerFactory.make_llm_worker(inference_model=inference_model)
+
+ client_factory_mock.assert_not_called()
+ worker_cls_mock.assert_called_once()
+ assert worker_cls_mock.call_args.kwargs["sdk_instance"] is cached_client
+
+ def test_cold_registry_creates_then_reuses_sdk_instance(self, mocker: MockerFixture) -> None:
+ """A cold registry triggers one client build; a second call reuses the cached instance."""
+ backend = make_backend()
+ registry = patch_hub_getters(mocker, backend=backend)
+ inference_model = make_llm_model_spec(sdk="openai")
+ plugin = Plugin.make_for_inference_model(inference_model=inference_model)
+ sdk_client = mocker.MagicMock(name="sdk_client")
+ client_factory_mock = mocker.patch(OPENAI_CLIENT, return_value=sdk_client)
+ worker_cls_mock = mocker.patch(COMPLETIONS_WORKER)
+
+ LLMWorkerFactory.make_llm_worker(inference_model=inference_model)
+ LLMWorkerFactory.make_llm_worker(inference_model=inference_model)
+
+ client_factory_mock.assert_called_once()
+ assert registry.get_sdk_instance(plugin=plugin) is sdk_client
+ first_kwargs, second_kwargs = (call.kwargs for call in worker_cls_mock.call_args_list)
+ assert first_kwargs["sdk_instance"] is sdk_client
+ assert second_kwargs["sdk_instance"] is sdk_client
+
+ @pytest.mark.parametrize(
+ ("sdk", "expected_extra"),
+ [
+ pytest.param("anthropic", "anthropic", id="anthropic"),
+ pytest.param("bedrock_anthropic", "anthropic", id="bedrock_anthropic"),
+ pytest.param("mistral", "mistral", id="mistral"),
+ pytest.param("bedrock_boto3", "bedrock", id="bedrock_boto3"),
+ pytest.param("bedrock_aioboto3", "bedrock", id="bedrock_aioboto3"),
+ pytest.param("google", "google", id="google"),
+ ],
+ )
+ def test_missing_dependency_raises(self, mocker: MockerFixture, sdk: str, expected_extra: str) -> None:
+ """When the optional SDK is absent, the factory raises MissingDependencyError naming the extra."""
+ backend = make_backend()
+ patch_hub_getters(mocker, backend=backend)
+ inference_model = make_llm_model_spec(sdk=sdk)
+ mocker.patch("importlib.util.find_spec", return_value=None)
+
+ with pytest.raises(MissingDependencyError) as exc_info:
+ LLMWorkerFactory.make_llm_worker(inference_model=inference_model)
+
+ assert exc_info.value.extra_name == expected_extra
+ assert f"pipelex[{expected_extra}]" in str(exc_info.value)
+
+ def test_unknown_sdk_raises_not_implemented(self, mocker: MockerFixture) -> None:
+ """An unrecognized SDK string raises NotImplementedError naming the plugin."""
+ backend = make_backend()
+ patch_hub_getters(mocker, backend=backend)
+ inference_model = make_llm_model_spec(sdk="definitely_not_an_sdk")
+
+ with pytest.raises(NotImplementedError) as exc_info:
+ LLMWorkerFactory.make_llm_worker(inference_model=inference_model)
+
+ assert "definitely_not_an_sdk" in str(exc_info.value)
+ assert "is not supported" in str(exc_info.value)
diff --git a/tests/unit/pipelex/cogt/llm/test_structured_output.py b/tests/unit/pipelex/cogt/llm/test_structured_output.py
new file mode 100644
index 000000000..368b01822
--- /dev/null
+++ b/tests/unit/pipelex/cogt/llm/test_structured_output.py
@@ -0,0 +1,62 @@
+import pytest
+from instructor import Mode as InstructorMode
+
+from pipelex.cogt.llm.structured_output import StructureMethod
+
+EXPECTED_INSTRUCTOR_MODES: dict[StructureMethod, InstructorMode] = {
+ # generic
+ StructureMethod.INSTRUCTOR_JSON: InstructorMode.JSON,
+ StructureMethod.INSTRUCTOR_MD_JSON: InstructorMode.MD_JSON,
+ StructureMethod.INSTRUCTOR_JSON_SCHEMA: InstructorMode.JSON_SCHEMA,
+ # openai
+ StructureMethod.INSTRUCTOR_OPENAI_PARALLEL_TOOLS: InstructorMode.PARALLEL_TOOLS,
+ StructureMethod.INSTRUCTOR_OPENAI_TOOLS: InstructorMode.TOOLS,
+ StructureMethod.INSTRUCTOR_OPENAI_STRUCTURED_OUTPUTS: InstructorMode.TOOLS_STRICT,
+ StructureMethod.INSTRUCTOR_OPENAI_JSON_O1: InstructorMode.JSON_O1,
+ StructureMethod.INSTRUCTOR_OPENAI_RESPONSES_TOOLS: InstructorMode.RESPONSES_TOOLS,
+ StructureMethod.INSTRUCTOR_OPENAI_RESPONSES_TOOLS_WITH_INBUILT_TOOLS: InstructorMode.RESPONSES_TOOLS_WITH_INBUILT_TOOLS,
+ # anthropic
+ StructureMethod.INSTRUCTOR_ANTHROPIC_TOOLS: InstructorMode.ANTHROPIC_TOOLS,
+ StructureMethod.INSTRUCTOR_ANTHROPIC_REASONING_TOOLS: InstructorMode.ANTHROPIC_REASONING_TOOLS,
+ StructureMethod.INSTRUCTOR_ANTHROPIC_JSON: InstructorMode.ANTHROPIC_JSON,
+ # mistral
+ StructureMethod.INSTRUCTOR_MISTRAL_TOOLS: InstructorMode.MISTRAL_TOOLS,
+ StructureMethod.INSTRUCTOR_MISTRAL_STRUCTURED_OUTPUTS: InstructorMode.MISTRAL_STRUCTURED_OUTPUTS,
+ # vertexai & google
+ StructureMethod.INSTRUCTOR_VERTEXAI_TOOLS: InstructorMode.VERTEXAI_TOOLS,
+ StructureMethod.INSTRUCTOR_VERTEXAI_JSON: InstructorMode.VERTEXAI_JSON,
+ StructureMethod.INSTRUCTOR_VERTEXAI_PARALLEL_TOOLS: InstructorMode.VERTEXAI_PARALLEL_TOOLS,
+ StructureMethod.INSTRUCTOR_GENAI_TOOLS: InstructorMode.GENAI_TOOLS,
+ StructureMethod.INSTRUCTOR_GENAI_STRUCTURED_OUTPUTS: InstructorMode.GENAI_STRUCTURED_OUTPUTS,
+ # cohere
+ StructureMethod.INSTRUCTOR_COHERE_TOOLS: InstructorMode.COHERE_TOOLS,
+ StructureMethod.INSTRUCTOR_COHERE_JSON_SCHEMA: InstructorMode.COHERE_JSON_SCHEMA,
+ # cerebras
+ StructureMethod.INSTRUCTOR_CEREBRAS_TOOLS: InstructorMode.CEREBRAS_TOOLS,
+ StructureMethod.INSTRUCTOR_CEREBRAS_JSON: InstructorMode.CEREBRAS_JSON,
+ # fireworks
+ StructureMethod.INSTRUCTOR_FIREWORKS_TOOLS: InstructorMode.FIREWORKS_TOOLS,
+ StructureMethod.INSTRUCTOR_FIREWORKS_JSON: InstructorMode.FIREWORKS_JSON,
+ # bedrock
+ StructureMethod.INSTRUCTOR_BEDROCK_TOOLS: InstructorMode.BEDROCK_TOOLS,
+ StructureMethod.INSTRUCTOR_BEDROCK_JSON: InstructorMode.BEDROCK_JSON,
+ # other providers
+ StructureMethod.INSTRUCTOR_WRITER_TOOLS: InstructorMode.WRITER_TOOLS,
+ StructureMethod.INSTRUCTOR_PERPLEXITY_JSON: InstructorMode.PERPLEXITY_JSON,
+ StructureMethod.INSTRUCTOR_OPENROUTER_STRUCTURED_OUTPUTS: InstructorMode.OPENROUTER_STRUCTURED_OUTPUTS,
+}
+
+
+class TestStructureMethod:
+ def test_expected_modes_cover_every_member(self):
+ """Completeness guard: a new StructureMethod member must be added to the expected mapping."""
+ assert set(EXPECTED_INSTRUCTOR_MODES) == set(StructureMethod)
+
+ @pytest.mark.parametrize(
+ ("structure_method", "expected_mode"),
+ list(EXPECTED_INSTRUCTOR_MODES.items()),
+ ids=[str(member) for member in EXPECTED_INSTRUCTOR_MODES],
+ )
+ def test_as_instructor_mode(self, structure_method: StructureMethod, expected_mode: InstructorMode):
+ """Each structure method maps to its exact instructor Mode."""
+ assert structure_method.as_instructor_mode() is expected_mode
diff --git a/tests/unit/pipelex/cogt/model_backends/test_backend_credentials.py b/tests/unit/pipelex/cogt/model_backends/test_backend_credentials.py
new file mode 100644
index 000000000..efabb18de
--- /dev/null
+++ b/tests/unit/pipelex/cogt/model_backends/test_backend_credentials.py
@@ -0,0 +1,200 @@
+import pytest
+from pytest_mock import MockerFixture
+
+from pipelex.cogt.model_backends.backend_credentials import (
+ BackendCredentialsErrorMsgFactory,
+ BackendCredentialsReport,
+ CredentialsValidationReport,
+)
+from pipelex.tools.secrets.env_secrets_provider import EnvSecretsProvider
+from pipelex.tools.secrets.secrets_provider_abstract import SecretsProviderAbstract
+
+GATEWAY_PITCH = (
+ "\n💡 Tip: Get a free Pipelex Gateway API key!\n"
+ " With Pipelex Gateway, you get unified access to multiple AI providers\n"
+ " (OpenAI, Anthropic, Google, Mistral, etc.) with a single API key.\n"
+ " Check the project's 'README.md' for details on obtaining your key.\n"
+ "\n🔑 Or bring your own keys:\n"
+ " Set your own provider keys (OPENAI_API_KEY, ANTHROPIC_API_KEY, GOOGLE_API_KEY,\n"
+ " MISTRAL_API_KEY, AZURE_API_KEY, etc.) and enable the corresponding backends\n"
+ " in '.pipelex/inference/backends.toml'.\n"
+)
+
+
+def make_report(
+ backend_name: str,
+ missing_vars: list[str],
+ placeholder_vars: list[str],
+) -> BackendCredentialsReport:
+ return BackendCredentialsReport(
+ backend_name=backend_name,
+ required_vars=sorted(set(missing_vars) | set(placeholder_vars)),
+ missing_vars=missing_vars,
+ placeholder_vars=placeholder_vars,
+ all_credentials_valid=not (missing_vars or placeholder_vars),
+ )
+
+
+class TestBackendCredentials:
+ def test_backend_credentials_report_round_trip(self):
+ """BackendCredentialsReport holds its constructed field values."""
+ report = BackendCredentialsReport(
+ backend_name="openai",
+ required_vars=["OPENAI_API_KEY", "OPENAI_ORG_ID"],
+ missing_vars=["OPENAI_API_KEY"],
+ placeholder_vars=["OPENAI_ORG_ID"],
+ all_credentials_valid=False,
+ )
+ assert report.backend_name == "openai"
+ assert report.required_vars == ["OPENAI_API_KEY", "OPENAI_ORG_ID"]
+ assert report.missing_vars == ["OPENAI_API_KEY"]
+ assert report.placeholder_vars == ["OPENAI_ORG_ID"]
+ assert report.all_credentials_valid is False
+
+ def test_credentials_validation_report_round_trip(self):
+ """CredentialsValidationReport holds its constructed field values."""
+ valid_report = make_report(backend_name="anthropic", missing_vars=[], placeholder_vars=[])
+ invalid_report = make_report(backend_name="openai", missing_vars=["OPENAI_API_KEY"], placeholder_vars=[])
+ validation_report = CredentialsValidationReport(
+ backend_reports={"anthropic": valid_report, "openai": invalid_report},
+ all_backends_valid=False,
+ )
+ assert validation_report.backend_reports["anthropic"] == valid_report
+ assert validation_report.backend_reports["openai"] == invalid_report
+ assert validation_report.backend_reports["anthropic"].all_credentials_valid is True
+ assert validation_report.all_backends_valid is False
+
+ def test_one_variable_missing_env_provider(self):
+ """The env-provider branch tells the user to add the variable to the environment or .env file."""
+ error_msg = BackendCredentialsErrorMsgFactory.make_one_variable_missing_error_msg(
+ secrets_provider=EnvSecretsProvider(),
+ backend_name="openai",
+ var_name="OPENAI_API_KEY",
+ )
+ expected_msg = (
+ "Could not get credentials for inference backend 'openai':\n\n"
+ "Credential issue:\n • 'openai': missing 'OPENAI_API_KEY'\n\n"
+ "You have two options:\n\n"
+ "1. Add the missing environment variable\n"
+ " Add the variable to your environment or .env file:\n"
+ " - 'OPENAI_API_KEY'=<your_api_key>\n"
+ "\n2. Disable this backend\n"
+ " Add 'enabled = false' under '\\[openai]' in '.pipelex/inference/backends.toml'\n" + GATEWAY_PITCH
+ )
+ assert error_msg == expected_msg
+
+ def test_one_variable_missing_generic_provider(self, mocker: MockerFixture):
+ """A non-env secrets provider gets the 'secrets provider' wording instead of .env guidance."""
+ generic_provider = mocker.MagicMock(spec=SecretsProviderAbstract)
+ error_msg = BackendCredentialsErrorMsgFactory.make_one_variable_missing_error_msg(
+ secrets_provider=generic_provider,
+ backend_name="anthropic",
+ var_name="ANTHROPIC_API_KEY",
+ )
+ expected_msg = (
+ "Could not get credentials for inference backend 'anthropic':\n\n"
+ "Credential issue:\n • 'anthropic': missing 'ANTHROPIC_API_KEY'\n\n"
+ "You have two options:\n\n"
+ "1. Provide the missing secret\n"
+ " Make sure 'ANTHROPIC_API_KEY' is available from your secrets provider.\n"
+ "\n2. Disable this backend\n"
+ " Add 'enabled = false' under '\\[anthropic]' in '.pipelex/inference/backends.toml'\n" + GATEWAY_PITCH
+ )
+ assert error_msg == expected_msg
+ assert ".env file" not in error_msg
+
+ def test_comprehensive_env_missing_vars_only(self):
+ """Env branch with only missing vars lists them sorted with the api-key hint and no placeholder note."""
+ reports = {"openai": make_report(backend_name="openai", missing_vars=["OPENAI_API_KEY", "AZURE_API_KEY"], placeholder_vars=[])}
+ error_msg = BackendCredentialsErrorMsgFactory.make_comprehensive_error_msg(
+ backend_credential_reports=reports,
+ secrets_provider=EnvSecretsProvider(),
+ )
+ expected_msg = (
+ "Could not get credentials for inference backend(s): 'openai'\n\n"
+ "Credential issues:\n • 'openai': missing: 'OPENAI_API_KEY', 'AZURE_API_KEY'\n\n"
+ "You have two options:\n\n"
+ "1. Add the missing environment variables\n"
+ " Add the missing variables to your environment or .env file:\n"
+ " - 'AZURE_API_KEY'=<your_api_key>\n"
+ " - 'OPENAI_API_KEY'=<your_api_key>\n"
+ "\n2. Disable unused backends\n"
+ " Disable backends you don't need in '.pipelex/inference/backends.toml':\n"
+ " - Add 'enabled = false' under '\\[openai]'\n" + GATEWAY_PITCH
+ )
+ assert error_msg == expected_msg
+
+ def test_comprehensive_env_placeholder_vars_only(self):
+ """Env branch with only placeholder vars flags unresolved placeholders and adds the replacement hint."""
+ reports = {"mistral": make_report(backend_name="mistral", missing_vars=[], placeholder_vars=["MISTRAL_API_KEY"])}
+ error_msg = BackendCredentialsErrorMsgFactory.make_comprehensive_error_msg(
+ backend_credential_reports=reports,
+ secrets_provider=EnvSecretsProvider(),
+ )
+ assert " • 'mistral': unresolved placeholders: 'MISTRAL_API_KEY'" in error_msg
+ assert " (Also replace placeholder values like '${VAR}' with actual keys)\n" in error_msg
+ assert "=<your_api_key>" not in error_msg
+ assert " - Add 'enabled = false' under '\\[mistral]'\n" in error_msg
+
+ def test_comprehensive_env_both_kinds_one_backend(self):
+ """Missing and placeholder issues on one backend are joined with a semicolon on its detail line."""
+ reports = {"openai": make_report(backend_name="openai", missing_vars=["VAR_AAA"], placeholder_vars=["VAR_BBB"])}
+ error_msg = BackendCredentialsErrorMsgFactory.make_comprehensive_error_msg(
+ backend_credential_reports=reports,
+ secrets_provider=EnvSecretsProvider(),
+ )
+ assert " • 'openai': missing: 'VAR_AAA'; unresolved placeholders: 'VAR_BBB'" in error_msg
+ assert " - 'VAR_AAA'=<your_api_key>\n" in error_msg
+ assert " (Also replace placeholder values like '${VAR}' with actual keys)\n" in error_msg
+
+ def test_comprehensive_env_multiple_backends_dedupes_and_sorts(self):
+ """Each backend gets a disable line; duplicate missing vars across backends are deduped and sorted."""
+ reports = {
+ "openai": make_report(backend_name="openai", missing_vars=["SHARED_KEY", "OPENAI_API_KEY"], placeholder_vars=[]),
+ "azure": make_report(backend_name="azure", missing_vars=["SHARED_KEY", "AZURE_API_KEY"], placeholder_vars=[]),
+ }
+ error_msg = BackendCredentialsErrorMsgFactory.make_comprehensive_error_msg(
+ backend_credential_reports=reports,
+ secrets_provider=EnvSecretsProvider(),
+ )
+ assert "Could not get credentials for inference backend(s): 'openai', 'azure'" in error_msg
+ assert error_msg.count(" - 'SHARED_KEY'=<your_api_key>\n") == 1
+ azure_var_pos = error_msg.index(" - 'AZURE_API_KEY'=<your_api_key>\n")
+ openai_var_pos = error_msg.index(" - 'OPENAI_API_KEY'=<your_api_key>\n")
+ shared_var_pos = error_msg.index(" - 'SHARED_KEY'=<your_api_key>\n")
+ assert azure_var_pos < openai_var_pos < shared_var_pos
+ assert " - Add 'enabled = false' under '\\[openai]'\n" in error_msg
+ assert " - Add 'enabled = false' under '\\[azure]'\n" in error_msg
+
+ @pytest.mark.parametrize("provider_kind", ["none", "generic_mock"])
+ def test_comprehensive_non_env_provider(self, mocker: MockerFixture, provider_kind: str):
+ """Non-env branch asks to provide secrets and merges missing + placeholder vars into one sorted list."""
+ secrets_provider: SecretsProviderAbstract | None
+ if provider_kind == "none":
+ secrets_provider = None
+ else:
+ secrets_provider = mocker.MagicMock(spec=SecretsProviderAbstract)
+ reports = {
+ "anthropic": make_report(backend_name="anthropic", missing_vars=["ANTHROPIC_API_KEY"], placeholder_vars=[]),
+ "bedrock": make_report(backend_name="bedrock", missing_vars=[], placeholder_vars=["AWS_REGION"]),
+ }
+ error_msg = BackendCredentialsErrorMsgFactory.make_comprehensive_error_msg(
+ backend_credential_reports=reports,
+ secrets_provider=secrets_provider,
+ )
+ expected_msg = (
+ "Could not get credentials for inference backend(s): 'anthropic', 'bedrock'\n\n"
+ "Credential issues:\n"
+ " • 'anthropic': missing: 'ANTHROPIC_API_KEY'\n"
+ " • 'bedrock': unresolved placeholders: 'AWS_REGION'\n\n"
+ "You have two options:\n\n"
+ "1. Provide the missing secrets\n"
+ " Make sure the following secrets are available from your secrets provider:\n"
+ " - 'ANTHROPIC_API_KEY'\n"
+ " - 'AWS_REGION'\n"
+ "\n2. Disable unused backends\n"
+ " Disable backends you don't need in '.pipelex/inference/backends.toml':\n"
+ " - Add 'enabled = false' under '\\[anthropic]'\n"
+ " - Add 'enabled = false' under '\\[bedrock]'\n" + GATEWAY_PITCH
+ )
+ assert error_msg == expected_msg
diff --git a/tests/unit/pipelex/cogt/models/test_model_deck_check.py b/tests/unit/pipelex/cogt/models/test_model_deck_check.py
new file mode 100644
index 000000000..f694d84c2
--- /dev/null
+++ b/tests/unit/pipelex/cogt/models/test_model_deck_check.py
@@ -0,0 +1,298 @@
+from typing import Any, Callable
+
+import pytest
+from pytest_mock import MockerFixture
+
+from pipelex.cogt.config_cogt import ModelDeckConfig
+from pipelex.cogt.exceptions import ModelChoiceNotFoundError
+from pipelex.cogt.extract.extract_setting import ExtractSetting
+from pipelex.cogt.img_gen.img_gen_job_components import Quality
+from pipelex.cogt.img_gen.img_gen_setting import ImgGenSetting
+from pipelex.cogt.llm.llm_setting import LLMSetting, LLMSettingChoicesDefaults
+from pipelex.cogt.llm.thinking_mode import ThinkingMode
+from pipelex.cogt.model_backends.model_spec import InferenceModelSpec
+from pipelex.cogt.model_backends.model_type import ModelType
+from pipelex.cogt.models.model_deck import ModelDeck
+from pipelex.cogt.models.model_deck_check import (
+ check_extract_choice_with_deck,
+ check_img_gen_choice_with_deck,
+ check_llm_choice_with_deck,
+ check_search_choice_with_deck,
+)
+from pipelex.cogt.models.model_reference import ModelReference, ModelReferenceKind
+from pipelex.cogt.search.search_setting import SearchSetting
+from pipelex.cogt.usage.cost_category import CostCategory
+from pipelex.system.runtime import ProblemReaction
+
+CheckFunction = Callable[[Any], None]
+
+GET_MODEL_DECK_TARGET = "pipelex.cogt.models.model_deck_check.get_model_deck"
+
+
+class TestModelDeckCheck:
+ def _create_model_spec(self, name: str, model_type: ModelType) -> InferenceModelSpec:
+ return InferenceModelSpec(
+ backend_name="test_backend",
+ name=name,
+ sdk="test_sdk",
+ model_type=model_type,
+ model_id=f"test_model_{name}",
+ costs={CostCategory.INPUT: 0.001, CostCategory.OUTPUT: 0.002},
+ thinking_mode=ThinkingMode.NONE,
+ max_tokens=1000,
+ max_prompt_images=None,
+ )
+
+ def _create_test_model_deck(self) -> ModelDeck:
+ """Build a deck with one preset, alias, waterfall and handle per model type."""
+ return ModelDeck(
+ inference_models={
+ "gpt-4o-mini": self._create_model_spec("gpt-4o-mini", ModelType.LLM),
+ "extract-engine": self._create_model_spec("extract-engine", ModelType.TEXT_EXTRACTOR),
+ "img-painter": self._create_model_spec("img-painter", ModelType.IMG_GEN),
+ "web-searcher": self._create_model_spec("web-searcher", ModelType.SEARCH),
+ },
+ # LLM-specific
+ llm_default_temperature=0.7,
+ llm_aliases={"best-gpt": "gpt-4o-mini"},
+ llm_waterfalls={"small-llm": ["gpt-4o-mini"]},
+ llm_presets={"cheap-llm": LLMSetting(model="gpt-4o-mini", temperature=0.5)},
+ llm_choice_defaults=LLMSettingChoicesDefaults(
+ default_temperature=0.7,
+ for_text=LLMSetting(model="default_text", temperature=0.7, max_tokens=1000),
+ for_object=LLMSetting(model="default_object", temperature=0.1, max_tokens=1000),
+ ),
+ # Extract-specific
+ extract_aliases={"best-extract": "extract-engine"},
+ extract_waterfalls={"fallback-extract": ["extract-engine"]},
+ extract_presets={"cheap-extract": ExtractSetting(model="extract-engine")},
+ extract_choice_default="default-extract-document",
+ # ImgGen-specific
+ img_gen_default_quality=Quality.MEDIUM,
+ img_gen_aliases={"best-img": "img-painter"},
+ img_gen_waterfalls={"fallback-img": ["img-painter"]},
+ img_gen_presets={"cheap-img": ImgGenSetting(model="img-painter")},
+ img_gen_choice_default="gen_image_basic",
+ # Search-specific
+ search_aliases={"best-search": "web-searcher"},
+ search_waterfalls={"fallback-search": ["web-searcher"]},
+ search_presets={"cheap-search": SearchSetting(model="web-searcher")},
+ search_choice_default="@default-search",
+ model_deck_config=ModelDeckConfig(is_model_fallback_enabled=False, missing_presets_reaction=ProblemReaction.NONE),
+ )
+
+ @pytest.mark.parametrize(
+ ("check_fn", "setting_instance"),
+ [
+ pytest.param(check_llm_choice_with_deck, LLMSetting(model="gpt-4o-mini", temperature=0.5), id="llm"),
+ pytest.param(check_extract_choice_with_deck, ExtractSetting(model="extract-engine"), id="extract"),
+ pytest.param(check_search_choice_with_deck, SearchSetting(model="web-searcher"), id="search"),
+ pytest.param(check_img_gen_choice_with_deck, ImgGenSetting(model="img-painter"), id="img_gen"),
+ ],
+ )
+ def test_setting_instance_short_circuits_without_deck_lookup(
+ self,
+ mocker: MockerFixture,
+ check_fn: CheckFunction,
+ setting_instance: Any,
+ ) -> None:
+ """A concrete Setting instance is accepted as-is without ever consulting the deck."""
+ get_deck_mock = mocker.patch(GET_MODEL_DECK_TARGET)
+
+ check_fn(setting_instance)
+
+ get_deck_mock.assert_not_called()
+
+ @pytest.mark.parametrize(
+ ("check_fn", "model_choice"),
+ [
+ pytest.param(check_llm_choice_with_deck, "$cheap-llm", id="llm-preset"),
+ pytest.param(check_llm_choice_with_deck, "@best-gpt", id="llm-alias"),
+ pytest.param(check_llm_choice_with_deck, "~small-llm", id="llm-waterfall"),
+ pytest.param(check_llm_choice_with_deck, "gpt-4o-mini", id="llm-handle"),
+ pytest.param(check_extract_choice_with_deck, "$cheap-extract", id="extract-preset"),
+ pytest.param(check_extract_choice_with_deck, "@best-extract", id="extract-alias"),
+ pytest.param(check_extract_choice_with_deck, "~fallback-extract", id="extract-waterfall"),
+ pytest.param(check_extract_choice_with_deck, "extract-engine", id="extract-handle"),
+ pytest.param(check_search_choice_with_deck, "$cheap-search", id="search-preset"),
+ pytest.param(check_search_choice_with_deck, "@best-search", id="search-alias"),
+ pytest.param(check_search_choice_with_deck, "~fallback-search", id="search-waterfall"),
+ pytest.param(check_search_choice_with_deck, "web-searcher", id="search-handle"),
+ pytest.param(check_img_gen_choice_with_deck, "$cheap-img", id="img_gen-preset"),
+ pytest.param(check_img_gen_choice_with_deck, "@best-img", id="img_gen-alias"),
+ pytest.param(check_img_gen_choice_with_deck, "~fallback-img", id="img_gen-waterfall"),
+ pytest.param(check_img_gen_choice_with_deck, "img-painter", id="img_gen-handle"),
+ ],
+ )
+ def test_found_choice_returns_none(
+ self,
+ mocker: MockerFixture,
+ check_fn: CheckFunction,
+ model_choice: str,
+ ) -> None:
+ """Each reference kind resolves successfully (no exception) when the name exists in the right deck collection."""
+ model_deck = self._create_test_model_deck()
+ mocker.patch(GET_MODEL_DECK_TARGET, return_value=model_deck)
+
+ check_fn(model_choice)
+
+ @pytest.mark.parametrize(
+ ("check_fn", "model_choice", "expected_kind", "expected_model_type", "options_attr"),
+ [
+ pytest.param(check_llm_choice_with_deck, "$missing-preset", ModelReferenceKind.PRESET, ModelType.LLM, "llm_presets", id="llm-preset"),
+ pytest.param(check_llm_choice_with_deck, "@missing-alias", ModelReferenceKind.ALIAS, ModelType.LLM, "llm_aliases", id="llm-alias"),
+ pytest.param(
+ check_llm_choice_with_deck, "~missing-waterfall", ModelReferenceKind.WATERFALL, ModelType.LLM, "llm_waterfalls", id="llm-waterfall"
+ ),
+ pytest.param(check_llm_choice_with_deck, "missing-handle", ModelReferenceKind.HANDLE, ModelType.LLM, "inference_models", id="llm-handle"),
+ pytest.param(
+ check_extract_choice_with_deck,
+ "$missing-preset",
+ ModelReferenceKind.PRESET,
+ ModelType.TEXT_EXTRACTOR,
+ "extract_presets",
+ id="extract-preset",
+ ),
+ pytest.param(
+ check_extract_choice_with_deck,
+ "@missing-alias",
+ ModelReferenceKind.ALIAS,
+ ModelType.TEXT_EXTRACTOR,
+ "extract_aliases",
+ id="extract-alias",
+ ),
+ pytest.param(
+ check_extract_choice_with_deck,
+ "~missing-waterfall",
+ ModelReferenceKind.WATERFALL,
+ ModelType.TEXT_EXTRACTOR,
+ "extract_waterfalls",
+ id="extract-waterfall",
+ ),
+ pytest.param(
+ check_extract_choice_with_deck,
+ "missing-handle",
+ ModelReferenceKind.HANDLE,
+ ModelType.TEXT_EXTRACTOR,
+ "inference_models",
+ id="extract-handle",
+ ),
+ pytest.param(
+ check_search_choice_with_deck,
+ "$missing-preset",
+ ModelReferenceKind.PRESET,
+ ModelType.SEARCH,
+ "search_presets",
+ id="search-preset",
+ ),
+ pytest.param(
+ check_search_choice_with_deck, "@missing-alias", ModelReferenceKind.ALIAS, ModelType.SEARCH, "search_aliases", id="search-alias"
+ ),
+ pytest.param(
+ check_search_choice_with_deck,
+ "~missing-waterfall",
+ ModelReferenceKind.WATERFALL,
+ ModelType.SEARCH,
+ "search_waterfalls",
+ id="search-waterfall",
+ ),
+ pytest.param(
+ check_search_choice_with_deck,
+ "missing-handle",
+ ModelReferenceKind.HANDLE,
+ ModelType.SEARCH,
+ "inference_models",
+ id="search-handle",
+ ),
+ pytest.param(
+ check_img_gen_choice_with_deck,
+ "$missing-preset",
+ ModelReferenceKind.PRESET,
+ ModelType.IMG_GEN,
+ "img_gen_presets",
+ id="img_gen-preset",
+ ),
+ pytest.param(
+ check_img_gen_choice_with_deck, "@missing-alias", ModelReferenceKind.ALIAS, ModelType.IMG_GEN, "img_gen_aliases", id="img_gen-alias"
+ ),
+ pytest.param(
+ check_img_gen_choice_with_deck,
+ "~missing-waterfall",
+ ModelReferenceKind.WATERFALL,
+ ModelType.IMG_GEN,
+ "img_gen_waterfalls",
+ id="img_gen-waterfall",
+ ),
+ pytest.param(
+ check_img_gen_choice_with_deck,
+ "missing-handle",
+ ModelReferenceKind.HANDLE,
+ ModelType.IMG_GEN,
+ "inference_models",
+ id="img_gen-handle",
+ ),
+ ],
+ )
+ def test_not_found_choice_raises_with_context(
+ self,
+ mocker: MockerFixture,
+ check_fn: CheckFunction,
+ model_choice: str,
+ expected_kind: ModelReferenceKind,
+ expected_model_type: ModelType,
+ options_attr: str,
+ ) -> None:
+ """A missing name raises ModelChoiceNotFoundError carrying the sigil-form choice, kind, type and the right deck collection."""
+ model_deck = self._create_test_model_deck()
+ mocker.patch(GET_MODEL_DECK_TARGET, return_value=model_deck)
+
+ with pytest.raises(ModelChoiceNotFoundError) as exc_info:
+ check_fn(model_choice)
+
+ error = exc_info.value
+ assert error.model_choice == model_choice
+ assert error.reference_kind == expected_kind
+ assert error.model_type == expected_model_type
+ expected_options: dict[str, Any] = getattr(model_deck, options_attr)
+ assert error.available_options == list(expected_options.keys())
+
+ def test_near_miss_handle_yields_fuzzy_suggestion(self, mocker: MockerFixture) -> None:
+ """A typo'd handle gets a fuzzy suggestion pointing at the close-by real handle."""
+ model_deck = self._create_test_model_deck()
+ mocker.patch(GET_MODEL_DECK_TARGET, return_value=model_deck)
+
+ with pytest.raises(ModelChoiceNotFoundError) as exc_info:
+ check_llm_choice_with_deck("gpt-4o-mimi")
+
+ error = exc_info.value
+ assert "gpt-4o-mini" in error.suggestions
+ assert "gpt-4o-mini" in str(error)
+
+ def test_alias_referenced_with_preset_sigil_yields_wrong_sigil_hint(self, mocker: MockerFixture) -> None:
+ """A name that exists as an alias but is referenced with the preset sigil gets a wrong-sigil hint."""
+ model_deck = self._create_test_model_deck()
+ mocker.patch(GET_MODEL_DECK_TARGET, return_value=model_deck)
+
+ with pytest.raises(ModelChoiceNotFoundError) as exc_info:
+ check_llm_choice_with_deck("$best-gpt")
+
+ error = exc_info.value
+ assert error.wrong_sigil_hints
+ assert "best-gpt exists as @best-gpt (alias)" in error.wrong_sigil_hints
+
+ @pytest.mark.parametrize(
+ "model_choice",
+ [
+ pytest.param("$cheap-llm", id="preset"),
+ pytest.param("@best-gpt", id="alias"),
+ pytest.param("~small-llm", id="waterfall"),
+ pytest.param("gpt-4o-mini", id="handle"),
+ ],
+ )
+ def test_string_and_model_reference_inputs_are_equivalent(self, mocker: MockerFixture, model_choice: str) -> None:
+ """Raw strings and pre-parsed ModelReference objects are both accepted (no exception) for every kind."""
+ model_deck = self._create_test_model_deck()
+ mocker.patch(GET_MODEL_DECK_TARGET, return_value=model_deck)
+
+ check_llm_choice_with_deck(model_choice)
+ check_llm_choice_with_deck(ModelReference.parse(model_choice))
diff --git a/tests/unit/pipelex/core/pipes/output/test_output_renderer_anything_formats.py b/tests/unit/pipelex/core/pipes/output/test_output_renderer_anything_formats.py
new file mode 100644
index 000000000..e146a3e26
--- /dev/null
+++ b/tests/unit/pipelex/core/pipes/output/test_output_renderer_anything_formats.py
@@ -0,0 +1,106 @@
+import json
+from typing import Any
+
+import pytest
+from pytest_mock import MockerFixture
+
+from pipelex.core.concepts.concept_representation_generator import ConceptRepresentationFormat
+from pipelex.core.pipes.output.output_renderer import render_output
+
+GET_REQUIRED_PIPE_TARGET = "pipelex.core.pipes.output.output_renderer.get_required_pipe"
+
+
+def _make_anything_condition_pipe(mocker: MockerFixture, mapped_pipe_code: str) -> Any:
+ condition_pipe = mocker.MagicMock()
+ condition_pipe.type = "PipeCondition"
+ condition_pipe.output.concept.code = "Anything"
+ condition_pipe.pipe_dependencies.return_value = {mapped_pipe_code}
+ return condition_pipe
+
+
+def _make_mapped_pipe(mocker: MockerFixture, concept_ref: str, rendered: dict[str, Any]) -> Any:
+ mapped_pipe = mocker.MagicMock()
+ mapped_pipe.output.concept.concept_ref = concept_ref
+ mapped_pipe.output.render_stuff_spec.return_value = rendered
+ return mapped_pipe
+
+
+class TestRenderOutputAnythingFormats:
+ @pytest.mark.parametrize(
+ "output_format",
+ [
+ ConceptRepresentationFormat.JSON,
+ ConceptRepresentationFormat.PYTHON,
+ ConceptRepresentationFormat.SCHEMA,
+ ],
+ )
+ def test_anything_without_possible_outputs_raises(self, mocker: MockerFixture, output_format: ConceptRepresentationFormat) -> None:
+ """An Anything output with no determinable possible outputs raises a ValueError naming native.Anything."""
+ anything_pipe = mocker.MagicMock()
+ anything_pipe.type = "PipeLLM"
+ anything_pipe.output.concept.code = "Anything"
+
+ with pytest.raises(ValueError, match=r"native\.Anything"):
+ render_output(anything_pipe, output_format=output_format)
+
+ def test_anything_json_renders_output_options(self, mocker: MockerFixture) -> None:
+ """JSON format wraps each possible output under an output_option_N key with concept and content."""
+ condition_pipe = _make_anything_condition_pipe(mocker, "make_summary")
+ mapped_pipe = _make_mapped_pipe(
+ mocker,
+ "test.Summary",
+ rendered={"concept": "test.Summary", "content": {"text": "summary text"}},
+ )
+ mocker.patch(GET_REQUIRED_PIPE_TARGET, return_value=mapped_pipe)
+
+ result = render_output(condition_pipe, output_format=ConceptRepresentationFormat.JSON)
+
+ parsed = json.loads(result)
+ assert parsed == {
+ "output_option_1": {
+ "concept": "test.Summary",
+ "content": {"text": "summary text"},
+ }
+ }
+ mapped_pipe.output.render_stuff_spec.assert_called_once_with(ConceptRepresentationFormat.JSON)
+
+ def test_anything_python_renders_option_lines(self, mocker: MockerFixture) -> None:
+ """PYTHON format lists each possible output as a commented option with an output_N assignment."""
+ condition_pipe = _make_anything_condition_pipe(mocker, "make_summary")
+ mapped_pipe = _make_mapped_pipe(
+ mocker,
+ "test.Summary",
+ rendered={"concept": "test.Summary", "content": 'SummaryContent(text="hello")'},
+ )
+ mocker.patch(GET_REQUIRED_PIPE_TARGET, return_value=mapped_pipe)
+
+ result = render_output(condition_pipe, output_format=ConceptRepresentationFormat.PYTHON)
+
+ result_lines = result.splitlines()
+ assert result_lines[0] == "# Multiple possible output types"
+ assert result_lines[1] == "# The actual output will be one of the following:"
+ assert "# Option 1: test.Summary" in result_lines
+ assert 'output_1 = SummaryContent(text="hello")' in result_lines
+ mapped_pipe.output.render_stuff_spec.assert_called_once_with(ConceptRepresentationFormat.PYTHON)
+
+ def test_anything_schema_renders_schema_options(self, mocker: MockerFixture) -> None:
+ """SCHEMA format wraps each possible output under a schema_option_N key with concept and content."""
+ condition_pipe = _make_anything_condition_pipe(mocker, "build_schema")
+ schema_content = {"type": "object", "properties": {"text": {"type": "string"}}}
+ mapped_pipe = _make_mapped_pipe(
+ mocker,
+ "test.Schema",
+ rendered={"concept": "test.Schema", "content": schema_content},
+ )
+ mocker.patch(GET_REQUIRED_PIPE_TARGET, return_value=mapped_pipe)
+
+ result = render_output(condition_pipe, output_format=ConceptRepresentationFormat.SCHEMA)
+
+ parsed = json.loads(result)
+ assert parsed == {
+ "schema_option_1": {
+ "concept": "test.Schema",
+ "content": schema_content,
+ }
+ }
+ mapped_pipe.output.render_stuff_spec.assert_called_once_with(ConceptRepresentationFormat.SCHEMA)
diff --git a/tests/unit/pipelex/core/pipes/output/test_output_renderer_collect_condition.py b/tests/unit/pipelex/core/pipes/output/test_output_renderer_collect_condition.py
new file mode 100644
index 000000000..5298ec27a
--- /dev/null
+++ b/tests/unit/pipelex/core/pipes/output/test_output_renderer_collect_condition.py
@@ -0,0 +1,112 @@
+from typing import Any
+
+import pytest
+from pytest_mock import MockerFixture
+
+from pipelex.core.concepts.concept_representation_generator import ConceptRepresentationFormat
+from pipelex.core.concepts.exceptions import ConceptError
+from pipelex.core.pipes.output.output_renderer import _collect_possible_outputs # noqa: PLC2701 # pyright: ignore[reportPrivateUsage]
+
+GET_REQUIRED_PIPE_TARGET = "pipelex.core.pipes.output.output_renderer.get_required_pipe"
+
+
+def _make_condition_pipe(mocker: MockerFixture, dependencies: set[str]) -> Any:
+ condition_pipe = mocker.MagicMock()
+ condition_pipe.type = "PipeCondition"
+ condition_pipe.pipe_dependencies.return_value = dependencies
+ return condition_pipe
+
+
+def _make_mapped_pipe(mocker: MockerFixture, concept_ref: str) -> Any:
+ mapped_pipe = mocker.MagicMock()
+ mapped_pipe.output.concept.concept_ref = concept_ref
+ return mapped_pipe
+
+
+class TestCollectPossibleOutputsCondition:
+ def test_empty_dependencies_yield_no_outputs(self, mocker: MockerFixture) -> None:
+ """A PipeCondition with no mapped pipes yields no possible outputs and never resolves pipes."""
+ condition_pipe = _make_condition_pipe(mocker, set())
+ get_pipe_mock = mocker.patch(GET_REQUIRED_PIPE_TARGET)
+
+ result = _collect_possible_outputs(condition_pipe)
+
+ assert result == []
+ get_pipe_mock.assert_not_called()
+
+ def test_single_mapped_pipe_uses_content_key(self, mocker: MockerFixture) -> None:
+ """A single mapped pipe yields one entry whose content comes from the rendered dict's 'content' key."""
+ condition_pipe = _make_condition_pipe(mocker, {"make_summary"})
+ mapped_pipe = _make_mapped_pipe(mocker, "test.Summary")
+ mapped_pipe.output.render_stuff_spec.return_value = {
+ "concept": "test.Summary",
+ "content": {"text": "summary text"},
+ }
+ get_pipe_mock = mocker.patch(GET_REQUIRED_PIPE_TARGET, return_value=mapped_pipe)
+
+ result = _collect_possible_outputs(condition_pipe, ConceptRepresentationFormat.JSON)
+
+ assert result == [{"concept_ref": "test.Summary", "content": {"text": "summary text"}}]
+ get_pipe_mock.assert_called_once_with(pipe_code="make_summary")
+ mapped_pipe.output.render_stuff_spec.assert_called_once_with(ConceptRepresentationFormat.JSON)
+
+ def test_multiple_mapped_pipes_are_ordered_deterministically(self, mocker: MockerFixture) -> None:
+ """With several mapped pipes, outputs come back sorted by pipe code regardless of set iteration order."""
+ condition_pipe = _make_condition_pipe(mocker, {"zeta_branch", "alpha_branch", "mid_branch"})
+ mapped_pipes: dict[str, Any] = {}
+ for pipe_code in ("alpha_branch", "mid_branch", "zeta_branch"):
+ mapped_pipe = _make_mapped_pipe(mocker, f"test.{pipe_code.title().replace('_', '')}")
+ mapped_pipe.output.render_stuff_spec.return_value = {"content": {"from": pipe_code}}
+ mapped_pipes[pipe_code] = mapped_pipe
+
+ def resolve_pipe(pipe_code: str) -> Any:
+ return mapped_pipes[pipe_code]
+
+ mocker.patch(GET_REQUIRED_PIPE_TARGET, side_effect=resolve_pipe)
+
+ result = _collect_possible_outputs(condition_pipe, ConceptRepresentationFormat.JSON)
+
+ assert [entry["content"]["from"] for entry in result] == ["alpha_branch", "mid_branch", "zeta_branch"]
+
+ def test_single_mapped_pipe_falls_back_to_whole_dict(self, mocker: MockerFixture) -> None:
+ """When the rendered dict has no 'content' key, the whole dict is used as the content."""
+ condition_pipe = _make_condition_pipe(mocker, {"build_schema"})
+ rendered_dict = {"type": "object", "properties": {"name": {"type": "string"}}}
+ mapped_pipe = _make_mapped_pipe(mocker, "test.Schema")
+ mapped_pipe.output.render_stuff_spec.return_value = rendered_dict
+ mocker.patch(GET_REQUIRED_PIPE_TARGET, return_value=mapped_pipe)
+
+ result = _collect_possible_outputs(condition_pipe, ConceptRepresentationFormat.SCHEMA)
+
+ assert result == [{"concept_ref": "test.Schema", "content": rendered_dict}]
+
+ @pytest.mark.parametrize(
+ "render_error",
+ [
+ ValueError("unsupported shape"),
+ ConceptError("unresolved structure class"),
+ ],
+ ids=["value_error", "pipelex_error"],
+ )
+ def test_render_failure_yields_placeholder(self, mocker: MockerFixture, render_error: Exception) -> None:
+ """A mapped pipe whose output cannot be rendered yields a placeholder entry instead of propagating."""
+ condition_pipe = _make_condition_pipe(mocker, {"dynamic_pipe"})
+ mapped_pipe = _make_mapped_pipe(mocker, "test.Dynamic")
+ mapped_pipe.output.render_stuff_spec.side_effect = render_error
+ mocker.patch(GET_REQUIRED_PIPE_TARGET, return_value=mapped_pipe)
+
+ result = _collect_possible_outputs(condition_pipe)
+
+ assert result == [{"concept_ref": "test.Dynamic", "content": "<unable to render>"}]
+
+ @pytest.mark.parametrize("pipe_type", ["PipeLLM", "PipeImgGen"])
+ def test_operator_pipe_types_yield_no_outputs(self, mocker: MockerFixture, pipe_type: str) -> None:
+ """Operator pipe types have no mapped outputs to collect."""
+ operator_pipe = mocker.MagicMock()
+ operator_pipe.type = pipe_type
+ get_pipe_mock = mocker.patch(GET_REQUIRED_PIPE_TARGET)
+
+ result = _collect_possible_outputs(operator_pipe)
+
+ assert result == []
+ get_pipe_mock.assert_not_called()
diff --git a/tests/unit/pipelex/core/pipes/output/test_output_renderer_collect_sequence.py b/tests/unit/pipelex/core/pipes/output/test_output_renderer_collect_sequence.py
new file mode 100644
index 000000000..6140239ad
--- /dev/null
+++ b/tests/unit/pipelex/core/pipes/output/test_output_renderer_collect_sequence.py
@@ -0,0 +1,112 @@
+from typing import Any
+
+import pytest
+from pytest_mock import MockerFixture
+
+from pipelex.core.concepts.concept_representation_generator import ConceptRepresentationFormat
+from pipelex.core.concepts.exceptions import ConceptError
+from pipelex.core.pipes.output.output_renderer import _collect_possible_outputs # noqa: PLC2701 # pyright: ignore[reportPrivateUsage]
+
+GET_REQUIRED_PIPE_TARGET = "pipelex.core.pipes.output.output_renderer.get_required_pipe"
+
+
+def _make_sequence_pipe(mocker: MockerFixture, sub_pipe_codes: list[str]) -> Any:
+ sequence_pipe = mocker.MagicMock()
+ sequence_pipe.type = "PipeSequence"
+ sub_pipes: list[Any] = []
+ for sub_pipe_code in sub_pipe_codes:
+ sub_pipe = mocker.MagicMock()
+ sub_pipe.pipe_code = sub_pipe_code
+ sub_pipes.append(sub_pipe)
+ sequence_pipe.sequential_sub_pipes = sub_pipes
+ return sequence_pipe
+
+
+def _make_concrete_pipe(mocker: MockerFixture, concept_ref: str) -> Any:
+ concrete_pipe = mocker.MagicMock()
+ concrete_pipe.type = "PipeLLM"
+ concrete_pipe.output.concept.code = "Text"
+ concrete_pipe.output.concept.concept_ref = concept_ref
+ return concrete_pipe
+
+
+class TestCollectPossibleOutputsSequence:
+ def test_empty_sub_pipes_yield_no_outputs(self, mocker: MockerFixture) -> None:
+ """A PipeSequence with no sub-pipes yields no possible outputs and never resolves pipes."""
+ sequence_pipe = _make_sequence_pipe(mocker, [])
+ get_pipe_mock = mocker.patch(GET_REQUIRED_PIPE_TARGET)
+
+ result = _collect_possible_outputs(sequence_pipe)
+
+ assert result == []
+ get_pipe_mock.assert_not_called()
+
+ def test_last_sub_pipe_with_empty_code_yields_no_outputs(self, mocker: MockerFixture) -> None:
+ """A last sub-pipe with an empty pipe_code yields no possible outputs."""
+ sequence_pipe = _make_sequence_pipe(mocker, ["first_step", ""])
+ get_pipe_mock = mocker.patch(GET_REQUIRED_PIPE_TARGET)
+
+ result = _collect_possible_outputs(sequence_pipe)
+
+ assert result == []
+ get_pipe_mock.assert_not_called()
+
+ def test_concrete_last_pipe_yields_single_entry(self, mocker: MockerFixture) -> None:
+ """Only the last sub-pipe is resolved and its rendered output is the single possible output."""
+ sequence_pipe = _make_sequence_pipe(mocker, ["first_step", "final_step"])
+ final_pipe = _make_concrete_pipe(mocker, "test.Final")
+ final_pipe.output.render_stuff_spec.return_value = {
+ "concept": "test.Final",
+ "content": {"text": "final output"},
+ }
+ get_pipe_mock = mocker.patch(GET_REQUIRED_PIPE_TARGET, return_value=final_pipe)
+
+ result = _collect_possible_outputs(sequence_pipe, ConceptRepresentationFormat.JSON)
+
+ assert result == [{"concept_ref": "test.Final", "content": {"text": "final output"}}]
+ get_pipe_mock.assert_called_once_with(pipe_code="final_step")
+ final_pipe.output.render_stuff_spec.assert_called_once_with(ConceptRepresentationFormat.JSON)
+
+ @pytest.mark.parametrize(
+ "render_error",
+ [
+ ValueError("unsupported shape"),
+ ConceptError("unresolved structure class"),
+ ],
+ ids=["value_error", "pipelex_error"],
+ )
+ def test_last_pipe_render_failure_yields_no_outputs(self, mocker: MockerFixture, render_error: Exception) -> None:
+ """A last pipe whose output cannot be rendered yields no possible outputs instead of propagating."""
+ sequence_pipe = _make_sequence_pipe(mocker, ["final_step"])
+ final_pipe = _make_concrete_pipe(mocker, "test.Dynamic")
+ final_pipe.output.render_stuff_spec.side_effect = render_error
+ mocker.patch(GET_REQUIRED_PIPE_TARGET, return_value=final_pipe)
+
+ result = _collect_possible_outputs(sequence_pipe)
+
+ assert result == []
+
+ def test_anything_last_pipe_recurses_into_it(self, mocker: MockerFixture) -> None:
+ """When the last pipe itself outputs Anything, collection recurses into it down to the concrete leaf."""
+ outer_pipe = _make_sequence_pipe(mocker, ["step_one", "mid_sequence"])
+ mid_pipe = _make_sequence_pipe(mocker, ["leaf_step"])
+ mid_pipe.output.concept.code = "Anything"
+ leaf_pipe = _make_concrete_pipe(mocker, "test.Leaf")
+ leaf_pipe.output.render_stuff_spec.return_value = {
+ "concept": "test.Leaf",
+ "content": {"text": "leaf output"},
+ }
+ pipes_by_code = {"mid_sequence": mid_pipe, "leaf_step": leaf_pipe}
+
+ def fake_get_required_pipe(pipe_code: str) -> Any:
+ return pipes_by_code[pipe_code]
+
+ get_pipe_mock = mocker.patch(GET_REQUIRED_PIPE_TARGET, side_effect=fake_get_required_pipe)
+
+ result = _collect_possible_outputs(outer_pipe, ConceptRepresentationFormat.JSON)
+
+ assert result == [{"concept_ref": "test.Leaf", "content": {"text": "leaf output"}}]
+ assert get_pipe_mock.call_count == 2
+ get_pipe_mock.assert_any_call(pipe_code="mid_sequence")
+ get_pipe_mock.assert_any_call(pipe_code="leaf_step")
+ leaf_pipe.output.render_stuff_spec.assert_called_once_with(ConceptRepresentationFormat.JSON)
diff --git a/tests/unit/pipelex/graph/test_graph_rendering_dry_run_bundle.py b/tests/unit/pipelex/graph/test_graph_rendering_dry_run_bundle.py
new file mode 100644
index 000000000..61e54bac7
--- /dev/null
+++ b/tests/unit/pipelex/graph/test_graph_rendering_dry_run_bundle.py
@@ -0,0 +1,92 @@
+"""Unit tests for the bundle dry-run helper in graph rendering."""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
+import pytest
+
+from pipelex.graph.graph_rendering import (
+ _dry_run_bundle, # noqa: PLC2701 # pyright: ignore[reportPrivateUsage]
+)
+
+if TYPE_CHECKING:
+ from pathlib import Path
+
+ from pytest_mock import MockerFixture
+
+GRAPH_RENDERING_MODULE = "pipelex.graph.graph_rendering"
+
+BUNDLE_CONTENT = 'domain = "test_domain"\nmain_pipe = "test_pipe"\n'
+
+
[email protected](loop_scope="class")
+class TestDryRunBundle:
+ """Tests for _dry_run_bundle library-dirs handling and dry_run_pipeline delegation."""
+
+ async def test_library_dirs_none_uses_bundle_parent(
+ self,
+ mocker: MockerFixture,
+ tmp_path: Path,
+ ) -> None:
+ """With no library_dirs, the resolved bundle parent dir should be the only library dir."""
+ bundle_path = tmp_path / "bundle.mthds"
+ bundle_path.write_text(BUNDLE_CONTENT, encoding="utf-8")
+
+ graph_spec_mock = mocker.MagicMock()
+ mock_dry_run = mocker.patch(
+ f"{GRAPH_RENDERING_MODULE}.dry_run_pipeline",
+ new_callable=mocker.AsyncMock,
+ return_value=(graph_spec_mock, "pipe_code"),
+ )
+
+ result = await _dry_run_bundle(bundle_path, library_dirs=None)
+
+ assert result == (graph_spec_mock, "pipe_code")
+ call_kwargs = mock_dry_run.call_args.kwargs
+ assert call_kwargs["mthds_contents"] == [BUNDLE_CONTENT]
+ assert call_kwargs["bundle_uris"] == [str(bundle_path)]
+ assert call_kwargs["library_dirs"] == [str(tmp_path.resolve())]
+
+ async def test_library_dirs_missing_parent_gets_parent_appended(
+ self,
+ mocker: MockerFixture,
+ tmp_path: Path,
+ ) -> None:
+ """When the bundle parent is missing from library_dirs, it is appended without mutating the original list."""
+ bundle_path = tmp_path / "bundle.mthds"
+ bundle_path.write_text(BUNDLE_CONTENT, encoding="utf-8")
+
+ mock_dry_run = mocker.patch(
+ f"{GRAPH_RENDERING_MODULE}.dry_run_pipeline",
+ new_callable=mocker.AsyncMock,
+ return_value=(mocker.MagicMock(), "pipe_code"),
+ )
+
+ provided_dirs = ["/somewhere/else"]
+ await _dry_run_bundle(bundle_path, library_dirs=provided_dirs)
+
+ call_kwargs = mock_dry_run.call_args.kwargs
+ assert call_kwargs["library_dirs"] == ["/somewhere/else", str(tmp_path.resolve())]
+ assert provided_dirs == ["/somewhere/else"]
+
+ async def test_library_dirs_already_containing_parent_passed_through(
+ self,
+ mocker: MockerFixture,
+ tmp_path: Path,
+ ) -> None:
+ """When the resolved bundle parent is already present, library_dirs pass through unchanged."""
+ bundle_path = tmp_path / "bundle.mthds"
+ bundle_path.write_text(BUNDLE_CONTENT, encoding="utf-8")
+
+ mock_dry_run = mocker.patch(
+ f"{GRAPH_RENDERING_MODULE}.dry_run_pipeline",
+ new_callable=mocker.AsyncMock,
+ return_value=(mocker.MagicMock(), "pipe_code"),
+ )
+
+ provided_dirs = ["/somewhere/else", str(tmp_path.resolve())]
+ await _dry_run_bundle(bundle_path, library_dirs=provided_dirs)
+
+ call_kwargs = mock_dry_run.call_args.kwargs
+ assert call_kwargs["library_dirs"] == provided_dirs
diff --git a/tests/unit/pipelex/graph/test_graph_rendering_generate_graph.py b/tests/unit/pipelex/graph/test_graph_rendering_generate_graph.py
new file mode 100644
index 000000000..f612639a7
--- /dev/null
+++ b/tests/unit/pipelex/graph/test_graph_rendering_generate_graph.py
@@ -0,0 +1,165 @@
+"""Unit tests for bundle-level graph generation dispatch."""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any
+
+import pytest
+
+from pipelex.graph.graph_rendering import GraphFormat, generate_graph_for_bundle
+from pipelex.tools.misc.chart_utils import FlowchartDirection
+
+if TYPE_CHECKING:
+ from pathlib import Path
+
+ from pytest_mock import MockerFixture
+
+GRAPH_RENDERING_MODULE = "pipelex.graph.graph_rendering"
+
+BUNDLE_CONTENT = 'domain = "test_domain"\nmain_pipe = "test_pipe"\n'
+
+
[email protected](loop_scope="class")
+class TestGenerateGraphForBundle:
+ """Tests for generate_graph_for_bundle format dispatch, rename branch, and return shape."""
+
+ def _setup_mocks(
+ self,
+ mocker: MockerFixture,
+ tmp_path: Path,
+ saved_files: dict[str, Path],
+ ) -> dict[str, Any]:
+ """Create the bundle file and patch the graph_rendering collaborators."""
+ bundle_path = tmp_path / "bundle.mthds"
+ bundle_path.write_text(BUNDLE_CONTENT, encoding="utf-8")
+
+ graph_spec_mock = mocker.MagicMock()
+ mock_dry_run = mocker.patch(
+ f"{GRAPH_RENDERING_MODULE}.dry_run_pipeline",
+ new_callable=mocker.AsyncMock,
+ return_value=(graph_spec_mock, "pipe_code"),
+ )
+
+ execution_config_mock = mocker.MagicMock()
+ config_mock = mocker.MagicMock()
+ config_mock.pipelex.pipeline_execution_config.with_execution_overrides.return_value = execution_config_mock
+ mocker.patch(f"{GRAPH_RENDERING_MODULE}.get_config", return_value=config_mock)
+
+ mock_generate = mocker.patch(
+ f"{GRAPH_RENDERING_MODULE}.generate_graph_outputs",
+ new_callable=mocker.AsyncMock,
+ return_value=mocker.MagicMock(),
+ )
+ mock_save = mocker.patch(
+ f"{GRAPH_RENDERING_MODULE}.save_graph_outputs_to_dir",
+ return_value=saved_files,
+ )
+
+ return {
+ "bundle_path": bundle_path,
+ "graph_spec_mock": graph_spec_mock,
+ "mock_dry_run": mock_dry_run,
+ "config_mock": config_mock,
+ "execution_config_mock": execution_config_mock,
+ "mock_generate": mock_generate,
+ "mock_save": mock_save,
+ }
+
+ @pytest.mark.parametrize(
+ ("graph_format", "expected_mermaidflow", "expected_reactflow"),
+ [
+ (GraphFormat.MERMAIDFLOW, True, False),
+ (GraphFormat.REACTFLOW, False, True),
+ (GraphFormat.BOTH, True, True),
+ ],
+ )
+ async def test_format_dispatch_sets_inclusion_flags(
+ self,
+ mocker: MockerFixture,
+ tmp_path: Path,
+ graph_format: GraphFormat,
+ expected_mermaidflow: bool,
+ expected_reactflow: bool,
+ ) -> None:
+ """Each graph format should map to the matching mermaidflow/reactflow inclusion flags."""
+ mocks = self._setup_mocks(mocker, tmp_path, saved_files={})
+
+ await generate_graph_for_bundle(
+ bundle_path=mocks["bundle_path"],
+ graph_format=graph_format,
+ )
+
+ execution_config_mock = mocks["execution_config_mock"]
+ graphs_inclusion_update = execution_config_mock.graph_config.graphs_inclusion.model_copy.call_args.kwargs["update"]
+ assert graphs_inclusion_update["mermaidflow_html"] is expected_mermaidflow
+ assert graphs_inclusion_update["reactflow_html"] is expected_reactflow
+ overrides_kwargs = mocks["config_mock"].pipelex.pipeline_execution_config.with_execution_overrides.call_args.kwargs
+ assert overrides_kwargs == {"generate_graph": True, "mock_inputs": True}
+
+ async def test_rename_branch_sanitizes_traversal_name(
+ self,
+ mocker: MockerFixture,
+ tmp_path: Path,
+ ) -> None:
+ """A traversal-laden graph_name should be renamed to a safe filename in the same dir."""
+ original_path = tmp_path / "reactflow.html"
+ original_path.write_text("<html></html>", encoding="utf-8")
+ mocks = self._setup_mocks(mocker, tmp_path, saved_files={"reactflow_html": original_path})
+
+ result = await generate_graph_for_bundle(
+ bundle_path=mocks["bundle_path"],
+ graph_format=GraphFormat.REACTFLOW,
+ graph_name="../../evil.html",
+ )
+
+ renamed_path = tmp_path / "evil.html"
+ assert renamed_path.is_file()
+ assert not original_path.exists()
+ assert result["graph_files"] == {"reactflow_html": str(renamed_path)}
+
+ async def test_no_reactflow_key_skips_rename(
+ self,
+ mocker: MockerFixture,
+ tmp_path: Path,
+ ) -> None:
+ """Without a reactflow_html entry, saved files pass through as strings with no rename."""
+ mermaid_path = tmp_path / "mermaid.html"
+ mermaid_path.write_text("<html></html>", encoding="utf-8")
+ mocks = self._setup_mocks(mocker, tmp_path, saved_files={"mermaidflow_html": mermaid_path})
+
+ result = await generate_graph_for_bundle(
+ bundle_path=mocks["bundle_path"],
+ graph_format=GraphFormat.MERMAIDFLOW,
+ graph_name="renamed.html",
+ )
+
+ assert mermaid_path.is_file()
+ assert not (tmp_path / "renamed.html").exists()
+ assert result["graph_files"] == {"mermaidflow_html": str(mermaid_path)}
+
+ @pytest.mark.parametrize(
+ ("direction", "expected_direction"),
+ [
+ (None, None),
+ (FlowchartDirection.LEFT_TO_RIGHT, str(FlowchartDirection.LEFT_TO_RIGHT)),
+ ],
+ )
+ async def test_return_shape(
+ self,
+ mocker: MockerFixture,
+ tmp_path: Path,
+ direction: FlowchartDirection | None,
+ expected_direction: str | None,
+ ) -> None:
+ """The result should carry the bundle parent dir, dry-run pipe_code, and stringified direction."""
+ mocks = self._setup_mocks(mocker, tmp_path, saved_files={})
+
+ result = await generate_graph_for_bundle(
+ bundle_path=mocks["bundle_path"],
+ graph_format=GraphFormat.BOTH,
+ direction=direction,
+ )
+
+ assert result["graph_output_dir"] == str(tmp_path)
+ assert result["pipe_code"] == "pipe_code"
+ assert result["direction"] == expected_direction
diff --git a/tests/unit/pipelex/graph/test_graph_rendering_generate_view.py b/tests/unit/pipelex/graph/test_graph_rendering_generate_view.py
new file mode 100644
index 000000000..0bfcc5376
--- /dev/null
+++ b/tests/unit/pipelex/graph/test_graph_rendering_generate_view.py
@@ -0,0 +1,87 @@
+"""Unit tests for bundle-level graph view (GraphSpec JSON) generation."""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any
+
+import pytest
+
+from pipelex.graph.graph_rendering import generate_view_for_bundle
+from pipelex.tools.misc.chart_utils import FlowchartDirection
+
+if TYPE_CHECKING:
+ from pathlib import Path
+
+ from pytest_mock import MockerFixture
+
+GRAPH_RENDERING_MODULE = "pipelex.graph.graph_rendering"
+
+BUNDLE_CONTENT = 'domain = "test_domain"\nmain_pipe = "test_pipe"\n'
+
+
[email protected](loop_scope="class")
+class TestGenerateViewForBundle:
+ """Tests for generate_view_for_bundle graphspec payload and direction precedence."""
+
+ def _setup_mocks(
+ self,
+ mocker: MockerFixture,
+ tmp_path: Path,
+ config_layout_direction: FlowchartDirection | None,
+ ) -> tuple[Path, Any]:
+ """Create the bundle file, patch collaborators, and return the bundle path and graph spec mock."""
+ bundle_path = tmp_path / "bundle.mthds"
+ bundle_path.write_text(BUNDLE_CONTENT, encoding="utf-8")
+
+ graph_spec_mock = mocker.MagicMock()
+ graph_spec_mock.model_dump.return_value = {"nodes": "node_payload", "edges": "edge_payload"}
+ mocker.patch(
+ f"{GRAPH_RENDERING_MODULE}.dry_run_pipeline",
+ new_callable=mocker.AsyncMock,
+ return_value=(graph_spec_mock, "pipe_code"),
+ )
+
+ execution_config_mock = mocker.MagicMock()
+ execution_config_mock.graph_config.reactflow_config.layout_direction = config_layout_direction
+ config_mock = mocker.MagicMock()
+ config_mock.pipelex.pipeline_execution_config.with_execution_overrides.return_value = execution_config_mock
+ mocker.patch(f"{GRAPH_RENDERING_MODULE}.get_config", return_value=config_mock)
+
+ return bundle_path, graph_spec_mock
+
+ async def test_returns_graphspec_json_dump(
+ self,
+ mocker: MockerFixture,
+ tmp_path: Path,
+ ) -> None:
+ """The graphspec key should carry the by-alias JSON model dump of the dry-run GraphSpec."""
+ bundle_path, graph_spec_mock = self._setup_mocks(mocker, tmp_path, config_layout_direction=None)
+
+ result = await generate_view_for_bundle(bundle_path=bundle_path)
+
+ assert result["graphspec"] == {"nodes": "node_payload", "edges": "edge_payload"}
+ assert result["pipe_code"] == "pipe_code"
+ graph_spec_mock.model_dump.assert_called_once_with(mode="json", by_alias=True)
+
+ @pytest.mark.parametrize(
+ ("direction_arg", "config_layout_direction", "expected_direction"),
+ [
+ (FlowchartDirection.TOP_DOWN, FlowchartDirection.LEFT_TO_RIGHT, str(FlowchartDirection.TOP_DOWN)),
+ (None, FlowchartDirection.LEFT_TO_RIGHT, str(FlowchartDirection.LEFT_TO_RIGHT)),
+ (None, None, None),
+ ],
+ )
+ async def test_direction_precedence(
+ self,
+ mocker: MockerFixture,
+ tmp_path: Path,
+ direction_arg: FlowchartDirection | None,
+ config_layout_direction: FlowchartDirection | None,
+ expected_direction: str | None,
+ ) -> None:
+ """An explicit direction wins, else the reactflow config layout direction, else None."""
+ bundle_path, _graph_spec_mock = self._setup_mocks(mocker, tmp_path, config_layout_direction=config_layout_direction)
+
+ result = await generate_view_for_bundle(bundle_path=bundle_path, direction=direction_arg)
+
+ assert result["direction"] == expected_direction
diff --git a/tests/unit/pipelex/observer/test_local_observer.py b/tests/unit/pipelex/observer/test_local_observer.py
new file mode 100644
index 000000000..bdc6379cf
--- /dev/null
+++ b/tests/unit/pipelex/observer/test_local_observer.py
@@ -0,0 +1,106 @@
+"""Unit tests for LocalObserver: JSONL persistence of run lifecycle events."""
+
+from __future__ import annotations
+
+import json
+from types import SimpleNamespace
+from typing import TYPE_CHECKING, Any
+
+import pytest
+
+from pipelex.observer.local_observer import LocalObserver, LocalObserverEventType
+
+if TYPE_CHECKING:
+ from pathlib import Path
+
+ from pytest_mock import MockerFixture
+
+MODULE = "pipelex.observer.local_observer"
+
+
+def _read_jsonl_lines(file_path: Path) -> list[dict[str, Any]]:
+ """Parse each line of a JSONL file into a dict."""
+ lines = file_path.read_text(encoding="utf-8").splitlines()
+ return [json.loads(line) for line in lines]
+
+
[email protected](loop_scope="class")
+class TestLocalObserver:
+ async def test_constructor_with_explicit_str_dir_creates_directory(self, tmp_path: Path) -> None:
+ """An explicit storage_dir given as str is created on construction, parents included."""
+ storage_dir = tmp_path / "nested" / "obs"
+
+ observer = LocalObserver(storage_dir=str(storage_dir))
+
+ assert observer.storage_dir == storage_dir
+ assert storage_dir.is_dir()
+
+ async def test_constructor_with_explicit_path_dir_creates_directory(self, tmp_path: Path) -> None:
+ """An explicit storage_dir given as Path is created on construction, parents included."""
+ storage_dir = tmp_path / "nested" / "obs"
+
+ observer = LocalObserver(storage_dir=storage_dir)
+
+ assert observer.storage_dir == storage_dir
+ assert storage_dir.is_dir()
+
+ async def test_constructor_default_dir_comes_from_config(self, mocker: MockerFixture, tmp_path: Path) -> None:
+ """Without an explicit storage_dir, the directory comes from the observer config."""
+ default_dir = tmp_path / "configured" / "observer"
+ config_stub = SimpleNamespace(pipelex=SimpleNamespace(observer_config=SimpleNamespace(observer_dir=str(default_dir))))
+ get_config_mock = mocker.patch(f"{MODULE}.get_config", return_value=config_stub)
+
+ observer = LocalObserver()
+
+ get_config_mock.assert_called_once_with()
+ assert observer.storage_dir == default_dir
+ assert default_dir.is_dir()
+
+ @pytest.mark.parametrize(
+ ("method_name", "event_type"),
+ [
+ ("observe_before_run", LocalObserverEventType.BEFORE_RUN),
+ ("observe_after_successful_run", LocalObserverEventType.AFTER_SUCCESSFUL_RUN),
+ ("observe_after_failing_run", LocalObserverEventType.AFTER_FAILING_RUN),
+ ],
+ )
+ async def test_observe_writes_one_jsonl_line_with_event_type(
+ self,
+ tmp_path: Path,
+ method_name: str,
+ event_type: LocalObserverEventType,
+ ) -> None:
+ """Each observe method appends exactly one line to {event_type}.jsonl merging event_type with the payload."""
+ observer = LocalObserver(storage_dir=tmp_path / "obs")
+ payload: dict[str, Any] = {"pipeline_run_id": "run-42", "step_count": 3}
+
+ await getattr(observer, method_name)(payload)
+
+ jsonl_path = tmp_path / "obs" / f"{event_type}.jsonl"
+ records = _read_jsonl_lines(jsonl_path)
+ assert records == [{"event_type": str(event_type), "pipeline_run_id": "run-42", "step_count": 3}]
+
+ async def test_two_observe_calls_append_two_lines_in_order(self, tmp_path: Path) -> None:
+ """Two calls on the same event type append two independently parseable lines, in call order."""
+ observer = LocalObserver(storage_dir=tmp_path / "obs")
+
+ await observer.observe_before_run({"pipeline_run_id": "run-1"})
+ await observer.observe_before_run({"pipeline_run_id": "run-2"})
+
+ jsonl_path = tmp_path / "obs" / f"{LocalObserverEventType.BEFORE_RUN}.jsonl"
+ records = _read_jsonl_lines(jsonl_path)
+ assert records == [
+ {"event_type": str(LocalObserverEventType.BEFORE_RUN), "pipeline_run_id": "run-1"},
+ {"event_type": str(LocalObserverEventType.BEFORE_RUN), "pipeline_run_id": "run-2"},
+ ]
+
+ async def test_event_name_wins_over_payload_event_type_key(self, tmp_path: Path) -> None:
+ """A payload carrying its own event_type cannot overwrite the lifecycle event name in the written record."""
+ observer = LocalObserver(storage_dir=tmp_path / "obs")
+ payload: dict[str, Any] = {"event_type": "custom_override", "pipeline_run_id": "run-9"}
+
+ await observer.observe_after_failing_run(payload)
+
+ jsonl_path = tmp_path / "obs" / f"{LocalObserverEventType.AFTER_FAILING_RUN}.jsonl"
+ records = _read_jsonl_lines(jsonl_path)
+ assert records == [{"event_type": str(LocalObserverEventType.AFTER_FAILING_RUN), "pipeline_run_id": "run-9"}]
diff --git a/tests/unit/pipelex/pipeline/test_runner_execute_error_paths.py b/tests/unit/pipelex/pipeline/test_runner_execute_error_paths.py
new file mode 100644
index 000000000..20dbb3c4d
--- /dev/null
+++ b/tests/unit/pipelex/pipeline/test_runner_execute_error_paths.py
@@ -0,0 +1,159 @@
+"""Error-path coverage for :meth:`PipelexMTHDSProtocol.execute`.
+
+Pins the ``except PipelexError`` arms (wrap-with-job vs propagate-unwrapped-without-job)
+and the ``except ValidationError`` arm. The happy path, the ``PipeRouterError`` wrap, and
+the finally-block library-restore matrix are pinned by the integration suite
+(``tests/integration/pipelex/pipeline/``) and are deliberately not re-tested here.
+"""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, NamedTuple
+
+import pytest
+from pydantic import BaseModel, ValidationError
+from typing_extensions import override
+
+from pipelex.pipe_run.pipe_run_mode import PipeRunMode
+from pipelex.pipe_run.pipe_run_protocol import PipeRunProtocol
+from pipelex.pipeline.exceptions import PipeExecutionError, PipelineExecutionError
+from pipelex.pipeline.runner import PipelexMTHDSProtocol
+from pipelex.system.telemetry.events import EventName, EventProperty, Outcome
+from pipelex.tools.typing.pydantic_utils import format_pydantic_validation_error
+
+if TYPE_CHECKING:
+ from pytest_mock import MockerFixture, MockType
+
+ from pipelex.core.pipes.pipe_output import PipeOutput
+ from pipelex.pipe_run.delivery_assignment import DeliveryAssignment
+ from pipelex.pipe_run.pipe_job import PipeJob
+
+
+class _MiniPayload(BaseModel):
+ """Minimal model used to capture a genuine pydantic ValidationError."""
+
+ count: int
+
+
+class _RaisingPipeRun(PipeRunProtocol):
+ """A PipeRun whose ``run`` always raises the injected exception."""
+
+ def __init__(self, exc: Exception):
+ self._exc = exc
+
+ @override
+ async def run(self, pipe_job: PipeJob, delivery_assignment: DeliveryAssignment | None = None) -> PipeOutput:
+ raise self._exc
+
+
+class _ExecuteEnv(NamedTuple):
+ setup_mock: MockType
+ telemetry_manager: MockType
+ pipe_job: MockType
+ execution_config: MockType
+
+
+_RUN_ID = "run-id-123"
+_LIB_ID = "lib-id-456"
+
+
[email protected](loop_scope="class")
+class TestRunnerExecuteErrorPaths:
+ def _patch_env(self, mocker: MockerFixture) -> _ExecuteEnv:
+ """Patch the hub getters and ``pipeline_run_setup`` at the runner namespace.
+
+ The execution_config stub disables graph and usage generation so the
+ finally block stays inert apart from the (mocked) registry/library cleanup.
+ """
+ pipe_job = mocker.MagicMock(name="pipe_job")
+ pipe_job.pipe.code = "echo_pipe"
+ pipe_job.pipe.pipe_type = "PipeLLM"
+ pipe_job.pipe_run_params.run_mode = PipeRunMode.DRY
+ pipe_job.pipe_run_params.pipe_stack = ["root_pipe", "echo_pipe"]
+ pipe_job.output_name = "topic_out"
+
+ setup_mock = mocker.patch(
+ "pipelex.pipeline.runner.pipeline_run_setup",
+ new=mocker.AsyncMock(return_value=(pipe_job, _RUN_ID, _LIB_ID)),
+ )
+ telemetry_manager = mocker.patch("pipelex.pipeline.runner.get_telemetry_manager").return_value
+ mocker.patch("pipelex.pipeline.runner.get_report_delegate")
+ mocker.patch("pipelex.pipeline.runner.get_pipeline_manager")
+ mocker.patch("pipelex.pipeline.runner.get_library_manager")
+ mocker.patch("pipelex.pipeline.runner.set_current_library")
+ mocker.patch("pipelex.pipeline.runner.clear_current_library")
+ mocker.patch("pipelex.pipeline.runner.get_current_library_id_or_none", return_value=None)
+
+ execution_config = mocker.MagicMock(name="execution_config")
+ execution_config.is_generate_graph = False
+ execution_config.is_generate_usage = False
+ return _ExecuteEnv(setup_mock=setup_mock, telemetry_manager=telemetry_manager, pipe_job=pipe_job, execution_config=execution_config)
+
+ async def test_pipelex_error_with_resolved_job_wraps_into_pipeline_execution_error(self, mocker: MockerFixture) -> None:
+ """A PipelexError raised by the pipe run AFTER setup resolved the job is wrapped
+ into a PipelineExecutionError carrying the job's identity, and a FAILURE
+ telemetry event fires.
+ """
+ env = self._patch_env(mocker)
+ original = PipeExecutionError("boom from pipe")
+ runner = PipelexMTHDSProtocol(
+ execution_config=env.execution_config,
+ pipe_run=_RaisingPipeRun(original),
+ )
+
+ with pytest.raises(PipelineExecutionError) as exc_info:
+ await runner.execute(pipe_code="echo_pipe")
+
+ wrapped = exc_info.value
+ assert wrapped.message == "boom from pipe"
+ assert wrapped.pipe_code == "echo_pipe"
+ assert wrapped.output_name == "topic_out"
+ assert wrapped.run_mode == PipeRunMode.DRY
+ assert wrapped.pipe_stack == ["root_pipe", "echo_pipe"]
+ assert wrapped.__cause__ is original
+ env.telemetry_manager.track_event.assert_called_once_with(
+ event_name=EventName.PIPELINE_COMPLETE,
+ properties={
+ EventProperty.PIPELINE_RUN_ID: _RUN_ID,
+ EventProperty.PIPE_TYPE: "PipeLLM",
+ EventProperty.PIPELINE_OUTCOME: Outcome.FAILURE,
+ },
+ )
+
+ async def test_pipelex_error_during_setup_propagates_unwrapped(self, mocker: MockerFixture) -> None:
+ """When pipeline_run_setup itself raises (pipe_job is None), the SAME exception
+ instance propagates unwrapped and no failure telemetry event fires.
+ """
+ env = self._patch_env(mocker)
+ original = PipeExecutionError("setup exploded")
+ env.setup_mock.side_effect = original
+ runner = PipelexMTHDSProtocol(execution_config=env.execution_config)
+
+ with pytest.raises(PipeExecutionError) as exc_info:
+ await runner.execute(pipe_code="echo_pipe")
+
+ assert exc_info.value is original
+ assert exc_info.value.__cause__ is None
+ env.telemetry_manager.track_event.assert_not_called()
+
+ async def test_validation_error_wraps_into_pipe_execution_error(self, mocker: MockerFixture) -> None:
+ """A pydantic ValidationError raised by the pipe run becomes a PipeExecutionError
+ whose message names the failing model and embeds the formatted error.
+ """
+ env = self._patch_env(mocker)
+ with pytest.raises(ValidationError) as validation_exc_info:
+ _MiniPayload(count="not-a-number") # pyright: ignore[reportArgumentType]
+ validation_error = validation_exc_info.value
+ runner = PipelexMTHDSProtocol(
+ execution_config=env.execution_config,
+ pipe_run=_RaisingPipeRun(validation_error),
+ )
+
+ with pytest.raises(PipeExecutionError) as exc_info:
+ await runner.execute(pipe_code="echo_pipe")
+
+ wrapped = exc_info.value
+ assert wrapped.__cause__ is validation_error
+ assert "Input validation failed for '_MiniPayload'" in wrapped.message
+ assert format_pydantic_validation_error(validation_error) in wrapped.message
+ env.telemetry_manager.track_event.assert_not_called()
diff --git a/tests/unit/pipelex/pipeline/test_runner_protocol_surfaces.py b/tests/unit/pipelex/pipeline/test_runner_protocol_surfaces.py
new file mode 100644
index 000000000..ff6d9c731
--- /dev/null
+++ b/tests/unit/pipelex/pipeline/test_runner_protocol_surfaces.py
@@ -0,0 +1,135 @@
+"""Coverage for the small protocol surfaces of :class:`PipelexMTHDSProtocol`.
+
+Pins the ``extra`` rejection guard on ``execute``, the ``start`` not-implemented
+contract, the ``models`` deck shaping (presets to ModelInfo, aliases/waterfalls merged
+across categories, category filter passthrough), and the ``version`` handshake
+including the missing-distribution fallback.
+"""
+
+from __future__ import annotations
+
+from importlib import metadata
+from typing import TYPE_CHECKING, Any
+
+import pytest
+from mthds.protocol.exceptions import PipelineRequestError
+from mthds.protocol.models import ModelCategory as MthdsModelCategory
+
+from pipelex.builder.operations.models_ops import ModelCategory
+from pipelex.pipeline.runner import (
+ MTHDS_PROTOCOL_VERSION,
+ PipelexModelDeck,
+ PipelexMTHDSProtocol,
+ PipelexVersionInfo,
+)
+
+if TYPE_CHECKING:
+ from pytest_mock import MockerFixture
+
+_MODELS_PAYLOAD: dict[str, Any] = {
+ "presets": {
+ "llm": [{"name": "smart_llm"}, {"name": "cheap_llm"}],
+ "extract": [{"name": "doc_extractor"}],
+ },
+ "aliases": {
+ "llm": {"best": "smart_llm"},
+ "extract": {"ocr": "doc_extractor"},
+ },
+ "waterfalls": {
+ "llm": {"main_chain": ["smart_llm", "cheap_llm"]},
+ "extract": {},
+ },
+}
+
+
[email protected](loop_scope="class")
+class TestRunnerProtocolSurfaces:
+ async def test_execute_rejects_extra_args_listing_keys_sorted(self) -> None:
+ """The extra guard fires before any setup — no mocking needed — and names
+ the offending keys in sorted order.
+ """
+ runner = PipelexMTHDSProtocol()
+
+ with pytest.raises(PipelineRequestError) as exc_info:
+ await runner.execute(pipe_code="any_pipe", extra={"zed": 1, "abc": 2})
+
+ assert str(exc_info.value) == "The local runtime defines no extension args; got ['abc', 'zed']."
+
+ async def test_start_raises_not_implemented_pointing_at_execute(self) -> None:
+ """The local runtime does not implement async start; the message redirects to execute."""
+ runner = PipelexMTHDSProtocol()
+
+ with pytest.raises(NotImplementedError) as exc_info:
+ await runner.start(pipe_code="any_pipe")
+
+ assert "execute" in str(exc_info.value)
+ assert "not implemented by the local runtime" in str(exc_info.value)
+
+ async def test_models_shapes_deck_and_merges_aliases_and_waterfalls(self, mocker: MockerFixture) -> None:
+ """Each preset becomes one ModelInfo typed by its category; aliases and
+ waterfalls are merged across categories; no filter = categories None.
+ """
+ list_models_mock = mocker.patch("pipelex.pipeline.runner.list_models", return_value=_MODELS_PAYLOAD)
+ runner = PipelexMTHDSProtocol()
+
+ deck = await runner.models()
+
+ assert isinstance(deck, PipelexModelDeck)
+ model_entries = {(model_info.name, model_info.type) for model_info in deck.models}
+ assert model_entries == {
+ ("smart_llm", MthdsModelCategory.LLM),
+ ("cheap_llm", MthdsModelCategory.LLM),
+ ("doc_extractor", MthdsModelCategory.EXTRACT),
+ }
+ assert deck.aliases == {"best": "smart_llm", "ocr": "doc_extractor"}
+ assert deck.waterfalls == {"main_chain": ["smart_llm", "cheap_llm"]}
+ list_models_mock.assert_called_once_with(categories=None)
+
+ async def test_models_category_filter_translates_to_builder_enum(self, mocker: MockerFixture) -> None:
+ """A protocol-level category filter reaches list_models as the builder's enum."""
+ llm_only_payload: dict[str, Any] = {
+ "presets": {"llm": [{"name": "smart_llm"}]},
+ "aliases": {"llm": {"best": "smart_llm"}},
+ "waterfalls": {"llm": {}},
+ }
+ list_models_mock = mocker.patch("pipelex.pipeline.runner.list_models", return_value=llm_only_payload)
+ runner = PipelexMTHDSProtocol()
+
+ deck = await runner.models(category=MthdsModelCategory.LLM)
+
+ list_models_mock.assert_called_once_with(categories=[ModelCategory.LLM])
+ assert isinstance(deck, PipelexModelDeck)
+ assert [(model_info.name, model_info.type) for model_info in deck.models] == [("smart_llm", MthdsModelCategory.LLM)]
+ assert deck.aliases == {"best": "smart_llm"}
+ assert deck.waterfalls == {}
+
+ async def test_version_reports_installed_distribution(self, mocker: MockerFixture) -> None:
+ """All three implementation version fields carry the installed pipelex version,
+ alongside the protocol version and implementation name.
+ """
+ version_mock = mocker.patch("pipelex.pipeline.runner.metadata.version", return_value="9.8.7")
+ runner = PipelexMTHDSProtocol()
+
+ version_info = await runner.version()
+
+ assert isinstance(version_info, PipelexVersionInfo)
+ assert version_info.protocol_version == MTHDS_PROTOCOL_VERSION
+ assert version_info.runner_version == "9.8.7"
+ assert version_info.implementation == "pipelex"
+ assert version_info.implementation_version == "9.8.7"
+ assert version_info.runtime_version == "9.8.7"
+ version_mock.assert_called_once_with("pipelex")
+
+ async def test_version_falls_back_to_unknown_when_distribution_missing(self, mocker: MockerFixture) -> None:
+ """A source checkout without an installed distribution must not break version()."""
+ mocker.patch("pipelex.pipeline.runner.metadata.version", side_effect=metadata.PackageNotFoundError("pipelex"))
+ runner = PipelexMTHDSProtocol()
+
+ version_info = await runner.version()
+
+ assert isinstance(version_info, PipelexVersionInfo)
+ assert version_info.protocol_version == MTHDS_PROTOCOL_VERSION
+ assert version_info.runner_version == "unknown"
+ assert version_info.implementation == "pipelex"
+ assert version_info.implementation_version == "unknown"
+ assert version_info.runtime_version == "unknown"
diff --git a/tests/unit/pipelex/pipeline/test_runner_validate_method.py b/tests/unit/pipelex/pipeline/test_runner_validate_method.py
new file mode 100644
index 000000000..0a737d6e2
--- /dev/null
+++ b/tests/unit/pipelex/pipeline/test_runner_validate_method.py
@@ -0,0 +1,201 @@
+"""Coverage for :meth:`PipelexMTHDSProtocol.validate` — report shaping and library restore.
+
+Pins how the protocol wrapper maps a ``validate_bundle`` result onto
+``PipelexValidationReport`` (single blueprint = dict, multiple = list; pipe structures
+keyed by pipe code), the ``library_dirs``/``allow_signatures`` passthrough, and the
+finally-block matrix that restores the caller's current-library and tears the
+validation library down.
+"""
+
+from __future__ import annotations
+
+from pathlib import Path
+from typing import TYPE_CHECKING, NamedTuple
+
+import pytest
+
+from pipelex.pipeline.runner import PipelexMTHDSProtocol, PipelexValidationReport
+
+if TYPE_CHECKING:
+ from pytest_mock import MockerFixture, MockType
+
+_BLUEPRINT_DUMP_ONE = {"domain": "demo", "description": "first bundle"}
+_BLUEPRINT_DUMP_TWO = {"domain": "extra", "description": "second bundle"}
+_PIPE_DUMP_ONE = {"code": "first_pipe", "type": "PipeLLM"}
+_PIPE_DUMP_TWO = {"code": "second_pipe", "type": "PipeFunc"}
+
+
+class _ValidateEnv(NamedTuple):
+ validate_bundle_mock: MockType
+ current_library_mock: MockType
+ set_current_library_mock: MockType
+ clear_current_library_mock: MockType
+ library_manager: MockType
+
+
[email protected](loop_scope="class")
+class TestRunnerValidateMethod:
+ def _make_blueprint(self, mocker: MockerFixture, dump: dict[str, str]) -> MockType:
+ blueprint: MockType = mocker.MagicMock(name=f"blueprint_{dump['domain']}")
+ blueprint.model_dump.return_value = dump
+ return blueprint
+
+ def _make_pipe(self, mocker: MockerFixture, dump: dict[str, str]) -> MockType:
+ pipe: MockType = mocker.MagicMock(name=f"pipe_{dump['code']}")
+ pipe.code = dump["code"]
+ pipe.model_dump.return_value = dump
+ return pipe
+
+ def _patch_env(
+ self,
+ mocker: MockerFixture,
+ *,
+ blueprint_dumps: list[dict[str, str]],
+ pipe_dumps: list[dict[str, str]],
+ library_ids: list[str | None],
+ ) -> _ValidateEnv:
+ """Patch ``validate_bundle`` and the library hub getters at the runner namespace.
+
+ ``library_ids`` feeds ``get_current_library_id_or_none`` as a side_effect pair:
+ first call = the caller's prev library, second call = the validation library left
+ current by ``validate_bundle``.
+ """
+ result = mocker.MagicMock(name="validate_bundle_result")
+ result.blueprints = [self._make_blueprint(mocker, dump) for dump in blueprint_dumps]
+ result.pipes = [self._make_pipe(mocker, dump) for dump in pipe_dumps]
+ validate_bundle_mock = mocker.patch(
+ "pipelex.pipeline.runner.validate_bundle",
+ new=mocker.AsyncMock(return_value=result),
+ )
+ current_library_mock = mocker.patch(
+ "pipelex.pipeline.runner.get_current_library_id_or_none",
+ side_effect=library_ids,
+ )
+ set_current_library_mock = mocker.patch("pipelex.pipeline.runner.set_current_library")
+ clear_current_library_mock = mocker.patch("pipelex.pipeline.runner.clear_current_library")
+ library_manager = mocker.patch("pipelex.pipeline.runner.get_library_manager").return_value
+ return _ValidateEnv(
+ validate_bundle_mock=validate_bundle_mock,
+ current_library_mock=current_library_mock,
+ set_current_library_mock=set_current_library_mock,
+ clear_current_library_mock=clear_current_library_mock,
+ library_manager=library_manager,
+ )
+
+ async def test_single_blueprint_reported_as_dict(self, mocker: MockerFixture) -> None:
+ """One blueprint = a bare dict (not a one-item list), with pipe structures keyed by code."""
+ env = self._patch_env(
+ mocker,
+ blueprint_dumps=[_BLUEPRINT_DUMP_ONE],
+ pipe_dumps=[_PIPE_DUMP_ONE, _PIPE_DUMP_TWO],
+ library_ids=[None, "val-lib"],
+ )
+ runner = PipelexMTHDSProtocol()
+
+ report = await runner.validate(mthds_contents=["bundle-content"])
+
+ assert isinstance(report, PipelexValidationReport)
+ assert report.blueprint == _BLUEPRINT_DUMP_ONE
+ assert report.graph_spec is None
+ assert report.pipe_structures == {"first_pipe": _PIPE_DUMP_ONE, "second_pipe": _PIPE_DUMP_TWO}
+ for blueprint in env.validate_bundle_mock.return_value.blueprints:
+ blueprint.model_dump.assert_called_once_with(mode="json")
+ for pipe in env.validate_bundle_mock.return_value.pipes:
+ pipe.model_dump.assert_called_once_with(mode="json")
+
+ async def test_multiple_blueprints_reported_as_list(self, mocker: MockerFixture) -> None:
+ """Several blueprints = a list of dicts in declaration order."""
+ self._patch_env(
+ mocker,
+ blueprint_dumps=[_BLUEPRINT_DUMP_ONE, _BLUEPRINT_DUMP_TWO],
+ pipe_dumps=[_PIPE_DUMP_ONE],
+ library_ids=[None, "val-lib"],
+ )
+ runner = PipelexMTHDSProtocol()
+
+ report = await runner.validate(mthds_contents=["content-one", "content-two"])
+
+ assert isinstance(report, PipelexValidationReport)
+ assert report.blueprint == [_BLUEPRINT_DUMP_ONE, _BLUEPRINT_DUMP_TWO]
+ assert report.pipe_structures == {"first_pipe": _PIPE_DUMP_ONE}
+
+ async def test_library_dirs_converted_to_paths_and_allow_signatures_passthrough(self, mocker: MockerFixture) -> None:
+ """Constructor library_dirs strings reach validate_bundle as Path objects;
+ allow_signatures is forwarded verbatim.
+ """
+ env = self._patch_env(
+ mocker,
+ blueprint_dumps=[_BLUEPRINT_DUMP_ONE],
+ pipe_dumps=[],
+ library_ids=[None, "val-lib"],
+ )
+ runner = PipelexMTHDSProtocol(library_dirs=["dir_alpha", "nested/dir_beta"])
+
+ await runner.validate(mthds_contents=["bundle-content"], allow_signatures=True)
+
+ env.validate_bundle_mock.assert_awaited_once_with(
+ mthds_contents=["bundle-content"],
+ library_dirs=[Path("dir_alpha"), Path("nested/dir_beta")],
+ allow_signatures=True,
+ )
+
+ async def test_no_library_dirs_passes_none(self, mocker: MockerFixture) -> None:
+ """Without constructor library_dirs, validate_bundle receives None (strict default)."""
+ env = self._patch_env(
+ mocker,
+ blueprint_dumps=[_BLUEPRINT_DUMP_ONE],
+ pipe_dumps=[],
+ library_ids=[None, "val-lib"],
+ )
+ runner = PipelexMTHDSProtocol()
+
+ await runner.validate(mthds_contents=["bundle-content"])
+
+ env.validate_bundle_mock.assert_awaited_once_with(
+ mthds_contents=["bundle-content"],
+ library_dirs=None,
+ allow_signatures=False,
+ )
+
+ @pytest.mark.parametrize(
+ ("prev_library_id", "validation_library_id", "expect_set_prev", "expect_clear", "expect_teardown"),
+ [
+ pytest.param("outer-lib", "val-lib", True, False, True, id="prev_set_restores_then_teardown"),
+ pytest.param(None, "val-lib", False, True, True, id="prev_none_clears_then_teardown"),
+ pytest.param("same-lib", "same-lib", False, False, False, id="validation_is_prev_no_restore_no_teardown"),
+ ],
+ )
+ async def test_finally_restores_caller_library_matrix(
+ self,
+ mocker: MockerFixture,
+ prev_library_id: str | None,
+ validation_library_id: str,
+ expect_set_prev: bool,
+ expect_clear: bool,
+ expect_teardown: bool,
+ ) -> None:
+ """The finally restores the caller's current-library and tears the validation
+ library down — unless the validation library IS the caller's library.
+ """
+ env = self._patch_env(
+ mocker,
+ blueprint_dumps=[_BLUEPRINT_DUMP_ONE],
+ pipe_dumps=[],
+ library_ids=[prev_library_id, validation_library_id],
+ )
+ runner = PipelexMTHDSProtocol()
+
+ await runner.validate(mthds_contents=["bundle-content"])
+
+ if expect_set_prev:
+ env.set_current_library_mock.assert_called_once_with(library_id=prev_library_id)
+ else:
+ env.set_current_library_mock.assert_not_called()
+ if expect_clear:
+ env.clear_current_library_mock.assert_called_once_with()
+ else:
+ env.clear_current_library_mock.assert_not_called()
+ if expect_teardown:
+ env.library_manager.teardown.assert_called_once_with(library_id=validation_library_id)
+ else:
+ env.library_manager.teardown.assert_not_called()
diff --git a/tests/unit/pipelex/plugins/gateway/test_gateway_completions_client.py b/tests/unit/pipelex/plugins/gateway/test_gateway_completions_client.py
new file mode 100644
index 000000000..527a5972d
--- /dev/null
+++ b/tests/unit/pipelex/plugins/gateway/test_gateway_completions_client.py
@@ -0,0 +1,102 @@
+"""Construction tests for the GatewayCompletionsFactory OpenAI client and extras delegation.
+
+These patch the GatewayFactory accessors and the SDK client constructor — they never touch
+the network. Transport retry wiring (``max_retries``) is pinned separately in
+``tests/unit/pipelex/plugins/test_transport_retry_wiring.py``.
+"""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any
+
+import pytest
+
+from pipelex.plugins.gateway.gateway_completions_factory import GatewayCompletionsFactory
+from pipelex.plugins.gateway.gateway_exceptions import GatewayFactoryError
+
+if TYPE_CHECKING:
+ from pytest_mock import MockerFixture
+
+_FACTORY_NAMESPACE = "pipelex.plugins.gateway.gateway_completions_factory"
+
+
+def _patch_gateway_accessors(
+ mocker: MockerFixture,
+ is_debug_enabled: bool,
+ endpoint: str,
+ api_key: str,
+) -> None:
+ mocker.patch(f"{_FACTORY_NAMESPACE}.GatewayFactory.is_debug_enabled", return_value=is_debug_enabled)
+ mocker.patch(f"{_FACTORY_NAMESPACE}.GatewayFactory.get_endpoint", return_value=endpoint)
+ mocker.patch(f"{_FACTORY_NAMESPACE}.GatewayFactory.get_api_key", return_value=api_key)
+ config = mocker.MagicMock()
+ config.cogt.transport_max_retries = 2
+ mocker.patch(f"{_FACTORY_NAMESPACE}.get_config", return_value=config)
+
+
+class TestGatewayCompletionsClient:
+ """Client construction validates the SDK variant and wires Portkey headers."""
+
+ def test_wrong_sdk_variant_raises_factory_error(self, mocker: MockerFixture) -> None:
+ """A plugin whose sdk is not the completions variant is rejected before any client is built."""
+ _patch_gateway_accessors(mocker, is_debug_enabled=False, endpoint="https://gateway.test", api_key="gw-test")
+ mock_client = mocker.patch("openai.AsyncOpenAI")
+ plugin = mocker.MagicMock()
+ plugin.sdk = "gateway_responses"
+
+ with pytest.raises(GatewayFactoryError, match="is not supported by 'GatewayCompletionsFactory'"):
+ GatewayCompletionsFactory.make_portkey_openai_client_for_completions(plugin=plugin, backend=mocker.MagicMock())
+
+ mock_client.assert_not_called()
+
+ @pytest.mark.parametrize(
+ ("is_debug_enabled", "expected_debug_header"),
+ [
+ (True, "true"),
+ (False, "false"),
+ ],
+ )
+ def test_happy_path_wires_endpoint_placeholder_key_and_portkey_headers(
+ self,
+ mocker: MockerFixture,
+ is_debug_enabled: bool,
+ expected_debug_header: str,
+ ) -> None:
+ """The client gets the gateway endpoint as base_url, a non-empty placeholder api_key,
+ and default headers built by createHeaders carrying the Portkey api key and debug flag.
+ """
+ _patch_gateway_accessors(mocker, is_debug_enabled=is_debug_enabled, endpoint="https://gateway.example/v1", api_key="pk-secret")
+ mock_client = mocker.patch("openai.AsyncOpenAI")
+ plugin = mocker.MagicMock()
+ plugin.sdk = "gateway_completions"
+
+ client = GatewayCompletionsFactory.make_portkey_openai_client_for_completions(plugin=plugin, backend=mocker.MagicMock())
+
+ assert client is mock_client.return_value
+ mock_client.assert_called_once()
+ call_kwargs = mock_client.call_args.kwargs
+ assert call_kwargs["base_url"] == "https://gateway.example/v1"
+ # Auth goes through Portkey headers; the OpenAI SDK still requires a non-empty api_key.
+ assert call_kwargs["api_key"] == "unused-auth-via-portkey-headers"
+ default_headers = call_kwargs["default_headers"]
+ assert default_headers["x-portkey-api-key"] == "pk-secret"
+ assert default_headers["x-portkey-debug"] == expected_debug_header
+ assert default_headers["x-portkey-strict-open-ai-compliance"] == "false"
+
+ def test_make_extras_delegates_to_gateway_factory(self, mocker: MockerFixture) -> None:
+ """The make_extras override passes all three args through to GatewayFactory.make_extras
+ and returns its result unchanged.
+ """
+ expected_extras: tuple[dict[str, str], dict[str, Any]] = ({"x-portkey-provider": "test-backend"}, {"seed": 42})
+ mock_make_extras = mocker.patch(
+ f"{_FACTORY_NAMESPACE}.GatewayFactory.make_extras",
+ return_value=expected_extras,
+ )
+ factory = GatewayCompletionsFactory(is_http_url_enabled=True)
+ inference_model = mocker.MagicMock()
+ inference_job = mocker.MagicMock()
+
+ result = factory.make_extras(inference_model=inference_model, inference_job=inference_job, output_desc="text")
+
+ assert result is expected_extras
+ mock_make_extras.assert_called_once_with(inference_model=inference_model, inference_job=inference_job, output_desc="text")
diff --git a/tests/unit/pipelex/plugins/gateway/test_gateway_completions_extract_output.py b/tests/unit/pipelex/plugins/gateway/test_gateway_completions_extract_output.py
new file mode 100644
index 000000000..c25cc7832
--- /dev/null
+++ b/tests/unit/pipelex/plugins/gateway/test_gateway_completions_extract_output.py
@@ -0,0 +1,374 @@
+"""Tests for GatewayCompletionsFactory extract-output parsing.
+
+Responses are built with ``GenericResponse.model_validate({...})``: the model allows extras,
+so custom top-level fields like ``pages`` become attributes — exactly what the
+``hasattr(response, "pages")`` branches need. Omitting ``pages`` exercises the
+``choices[0].message.content`` fallback that survives Portkey proxying.
+"""
+
+from __future__ import annotations
+
+import json
+from typing import TYPE_CHECKING, Any
+
+import pytest
+from portkey_ai.api_resources.utils import GenericResponse
+
+from pipelex.cogt.extract.extract_output import ExtractOutput
+from pipelex.plugins.gateway.gateway_completions_factory import GatewayCompletionsFactory
+from pipelex.plugins.gateway.gateway_exceptions import GatewayExtractResponseError
+
+if TYPE_CHECKING:
+ from pytest_mock import MockerFixture
+
+_FACTORY_NAMESPACE = "pipelex.plugins.gateway.gateway_completions_factory"
+
+_FALSY_CHOICES_PAYLOADS: list[dict[str, Any]] = [
+ {},
+ {"choices": []},
+ {"choices": "nope"},
+ {"choices": [{}]},
+ {"choices": [{"message": "nope"}]},
+ {"choices": [{"message": {"content": 42}}]},
+ {"choices": [{"message": {"content": '{"key": 1}'}}]},
+ {"choices": [{"message": {"content": "{not json"}}]},
+]
+
+
+def _response(payload: dict[str, Any]) -> GenericResponse:
+ return GenericResponse.model_validate(payload)
+
+
+def _choices_response(content: Any) -> GenericResponse:
+ """Build a response whose pages/content live in choices[0].message.content."""
+ return GenericResponse.model_validate({"choices": [{"message": {"content": content}}]})
+
+
+class TestGatewayCompletionsExtractOutput:
+ """Extract-protocol dispatch and per-protocol response parsing."""
+
+ # ---- make_extract_output_from_response dispatch ----
+
+ @pytest.mark.parametrize(
+ ("model_handle", "target_method"),
+ [
+ ("mistral-document-ai-2505", "_make_extract_output_from_response_mistral"),
+ ("azure-document-intelligence", "_make_extract_output_from_response_azure"),
+ ("deepseek-ocr", "_make_extract_output_from_response_deepseek"),
+ ("linkup-fetch", "_make_extract_output_from_response_linkup_fetch"),
+ ],
+ )
+ def test_dispatch_routes_each_handle_to_its_parser(
+ self,
+ mocker: MockerFixture,
+ model_handle: str,
+ target_method: str,
+ ) -> None:
+ """Each known model handle dispatches to the matching protocol-specific parser."""
+ sentinel_output = ExtractOutput(pages={})
+ mock_parser = mocker.patch.object(GatewayCompletionsFactory, target_method, return_value=sentinel_output)
+ inference_model = mocker.MagicMock()
+ inference_model.name = model_handle
+ response = _response({})
+
+ result = GatewayCompletionsFactory.make_extract_output_from_response(inference_model=inference_model, response=response)
+
+ assert result is sentinel_output
+ mock_parser.assert_called_once_with(response=response)
+
+ def test_dispatch_unknown_handle_raises_value_error(self, mocker: MockerFixture) -> None:
+ """An unknown model handle is rejected by GatewayExtractProtocol.make_from_model_handle."""
+ inference_model = mocker.MagicMock()
+ inference_model.name = "gpt-4o"
+
+ with pytest.raises(ValueError, match="Invalid model ID: gpt-4o"):
+ GatewayCompletionsFactory.make_extract_output_from_response(inference_model=inference_model, response=_response({}))
+
+ # ---- _extract_pages_from_choices_content ----
+
+ def test_extract_pages_from_choices_content_happy_path(self) -> None:
+ """A JSON list in choices[0].message.content is parsed and returned as page dicts."""
+ page_dicts = [{"index": 0, "markdown": "hello"}, {"index": 1, "markdown": "world"}]
+ response = _choices_response(json.dumps(page_dicts))
+
+ result = GatewayCompletionsFactory._extract_pages_from_choices_content(response=response) # noqa: SLF001 # pyright: ignore[reportPrivateUsage]
+
+ assert result == page_dicts
+
+ @pytest.mark.parametrize(
+ "payload",
+ _FALSY_CHOICES_PAYLOADS,
+ ids=[
+ "no-choices",
+ "empty-choices",
+ "choices-not-a-list",
+ "message-missing",
+ "message-not-a-dict",
+ "content-not-a-str",
+ "content-json-not-a-list",
+ "content-malformed-json",
+ ],
+ )
+ def test_extract_pages_from_choices_content_falsy_paths_return_none(self, payload: dict[str, Any]) -> None:
+ """Every malformed or missing choices/message/content shape returns None, never raises."""
+ response = _response(payload)
+
+ result = GatewayCompletionsFactory._extract_pages_from_choices_content(response=response) # noqa: SLF001 # pyright: ignore[reportPrivateUsage]
+
+ assert result is None
+
+ # ---- Azure Doc Intel ----
+
+ def test_azure_top_level_pages_with_images(self) -> None:
+ """Top-level pages parse into Pages, mapping image fields into ExtractedImageFromPage."""
+ response = _response(
+ {
+ "pages": [
+ {
+ "index": 2,
+ "markdown": "# Invoice page",
+ "images": [
+ {
+ "base64_str": "QUJD",
+ "mime_type": "image/png",
+ "caption": "Company logo",
+ "bounding_box": {
+ "top_left_x": 1.0,
+ "top_left_y": 2.0,
+ "top_right_x": 30.0,
+ "top_right_y": 2.0,
+ "bottom_right_x": 30.0,
+ "bottom_right_y": 40.0,
+ "bottom_left_x": 1.0,
+ "bottom_left_y": 40.0,
+ },
+ }
+ ],
+ }
+ ],
+ },
+ )
+
+ extract_output = GatewayCompletionsFactory._make_extract_output_from_response_azure(response=response) # noqa: SLF001 # pyright: ignore[reportPrivateUsage]
+
+ assert list(extract_output.pages.keys()) == [2]
+ page = extract_output.pages[2]
+ assert page.text == "# Invoice page"
+ assert len(page.extracted_images) == 1
+ extracted_image = page.extracted_images[0]
+ assert extracted_image.size is None
+ assert extracted_image.base64_str == "QUJD"
+ assert extracted_image.mime_type == "image/png"
+ assert extracted_image.caption == "Company logo"
+ assert extracted_image.bounding_box is not None
+ assert extracted_image.bounding_box.top_left_x == 1.0
+ assert extracted_image.bounding_box.bottom_right_y == 40.0
+
+ def test_azure_falls_back_to_choices_content(self) -> None:
+ """Without a top-level pages field, pages are recovered from choices[0].message.content."""
+ page_dicts: list[dict[str, Any]] = [{"index": 0, "markdown": "Fallback page", "images": []}]
+ response = _choices_response(json.dumps(page_dicts))
+
+ extract_output = GatewayCompletionsFactory._make_extract_output_from_response_azure(response=response) # noqa: SLF001 # pyright: ignore[reportPrivateUsage]
+
+ assert list(extract_output.pages.keys()) == [0]
+ assert extract_output.pages[0].text == "Fallback page"
+ assert extract_output.pages[0].extracted_images == []
+
+ def test_azure_without_pages_anywhere_raises(self) -> None:
+ """Neither a top-level pages field nor a choices fallback means the response is unusable."""
+ with pytest.raises(GatewayExtractResponseError, match="does not have pages"):
+ GatewayCompletionsFactory._make_extract_output_from_response_azure(response=_response({"success": True})) # noqa: SLF001 # pyright: ignore[reportPrivateUsage]
+
+ def test_azure_page_schema_violation_is_wrapped(self) -> None:
+ """A page dict failing GatewayExtractPageAzure validation is wrapped in GatewayExtractResponseError."""
+ response = _response({"pages": [{"index": 0, "images": []}]})
+
+ with pytest.raises(GatewayExtractResponseError, match="Azure schema"):
+ GatewayCompletionsFactory._make_extract_output_from_response_azure(response=response) # noqa: SLF001 # pyright: ignore[reportPrivateUsage]
+
+ # ---- Mistral Doc AI ----
+
+ def test_mistral_without_pages_raises(self) -> None:
+ """A response with no pages attribute is rejected (Mistral has no choices fallback)."""
+ with pytest.raises(GatewayExtractResponseError, match="does not have pages"):
+ GatewayCompletionsFactory._make_extract_output_from_response_mistral(response=_response({"success": True})) # noqa: SLF001 # pyright: ignore[reportPrivateUsage]
+
+ def test_mistral_pages_with_images_corner_handling(self) -> None:
+ """Images without base64 are skipped; full corner coords build a BoundingBox; partial coords leave it None."""
+ response = _response(
+ {
+ "pages": [
+ {
+ "index": 0,
+ "markdown": "# Report",
+ "images": [
+ {"image_base64": None, "image_annotation": "ghost"},
+ {
+ "image_base64": "data:image/png;base64,QUJD",
+ "top_left_x": 10,
+ "top_left_y": 20,
+ "bottom_right_x": 110,
+ "bottom_right_y": 220,
+ "image_annotation": "figure 1",
+ },
+ {"image_base64": "data:image/png;base64,REVG", "top_left_x": 5},
+ ],
+ }
+ ],
+ },
+ )
+
+ extract_output = GatewayCompletionsFactory._make_extract_output_from_response_mistral(response=response) # noqa: SLF001 # pyright: ignore[reportPrivateUsage]
+
+ assert list(extract_output.pages.keys()) == [0]
+ page = extract_output.pages[0]
+ assert page.text == "# Report"
+ # The image without image_base64 is skipped entirely
+ assert len(page.extracted_images) == 2
+ full_corners_image = page.extracted_images[0]
+ assert full_corners_image.actual_url_or_prefixed_base64 == "data:image/png;base64,QUJD"
+ assert full_corners_image.base64_str == "QUJD"
+ assert full_corners_image.mime_type == "image/png"
+ assert full_corners_image.caption == "figure 1"
+ assert full_corners_image.bounding_box is not None
+ assert full_corners_image.bounding_box.top_left_x == 10.0
+ assert full_corners_image.bounding_box.top_left_y == 20.0
+ assert full_corners_image.bounding_box.top_right_x == 110.0
+ assert full_corners_image.bounding_box.top_right_y == 20.0
+ assert full_corners_image.bounding_box.bottom_right_x == 110.0
+ assert full_corners_image.bounding_box.bottom_right_y == 220.0
+ assert full_corners_image.bounding_box.bottom_left_x == 10.0
+ assert full_corners_image.bounding_box.bottom_left_y == 220.0
+ partial_corners_image = page.extracted_images[1]
+ assert partial_corners_image.bounding_box is None
+
+ def test_mistral_page_schema_violation_is_wrapped(self) -> None:
+ """A page dict failing GatewayExtractPageMistral validation is wrapped in GatewayExtractResponseError."""
+ response = _response({"pages": [{"index": 0, "markdown": "missing images field"}]})
+
+ with pytest.raises(GatewayExtractResponseError, match="Mistral schema"):
+ GatewayCompletionsFactory._make_extract_output_from_response_mistral(response=response) # noqa: SLF001 # pyright: ignore[reportPrivateUsage]
+
+ # ---- DeepSeek OCR ----
+
+ def test_deepseek_top_level_pages_happy_path(self) -> None:
+ """Top-level pages parse into text-only Pages with no extracted images."""
+ response = _response({"pages": [{"index": 1, "markdown": "## Page two"}]})
+
+ extract_output = GatewayCompletionsFactory._make_extract_output_from_response_deepseek(response=response) # noqa: SLF001 # pyright: ignore[reportPrivateUsage]
+
+ assert list(extract_output.pages.keys()) == [1]
+ assert extract_output.pages[1].text == "## Page two"
+ assert extract_output.pages[1].extracted_images == []
+
+ def test_deepseek_scaled_down_image_logs_warning_and_parses(self, mocker: MockerFixture) -> None:
+ """source_image_info.scaled_down triggers the warning branch without breaking the parse."""
+ mock_log = mocker.patch(f"{_FACTORY_NAMESPACE}.log")
+ response = _response(
+ {
+ "pages": [
+ {
+ "index": 0,
+ "markdown": "Scaled page",
+ "source_image_info": {
+ "original": {"width": 2000, "height": 1000, "bytes": 204800},
+ "processed": {"width": 1000, "height": 500, "bytes": 51200},
+ "scaled_down": True,
+ },
+ }
+ ],
+ },
+ )
+
+ extract_output = GatewayCompletionsFactory._make_extract_output_from_response_deepseek(response=response) # noqa: SLF001 # pyright: ignore[reportPrivateUsage]
+
+ assert extract_output.pages[0].text == "Scaled page"
+ mock_log.warning.assert_called_once()
+ warning_message = mock_log.warning.call_args.args[0]
+ assert "scaled down from 2000x1000" in warning_message
+ assert "to 1000x500" in warning_message
+
+ def test_deepseek_falls_back_to_choices_content(self) -> None:
+ """Without a top-level pages field, pages are recovered from choices[0].message.content."""
+ page_dicts = [{"index": 0, "markdown": "Fallback md"}]
+ response = _choices_response(json.dumps(page_dicts))
+
+ extract_output = GatewayCompletionsFactory._make_extract_output_from_response_deepseek(response=response) # noqa: SLF001 # pyright: ignore[reportPrivateUsage]
+
+ assert list(extract_output.pages.keys()) == [0]
+ assert extract_output.pages[0].text == "Fallback md"
+
+ def test_deepseek_without_pages_anywhere_raises(self) -> None:
+ """Neither a top-level pages field nor a choices fallback means the response is unusable."""
+ with pytest.raises(GatewayExtractResponseError, match="does not have pages"):
+ GatewayCompletionsFactory._make_extract_output_from_response_deepseek(response=_response({"success": True})) # noqa: SLF001 # pyright: ignore[reportPrivateUsage]
+
+ def test_deepseek_page_schema_violation_is_wrapped(self) -> None:
+ """A page dict failing GatewayExtractPageDeepseek validation is wrapped in GatewayExtractResponseError."""
+ response = _response({"pages": [{"markdown": "missing index"}]})
+
+ with pytest.raises(GatewayExtractResponseError, match="Deepseek schema"):
+ GatewayCompletionsFactory._make_extract_output_from_response_deepseek(response=response) # noqa: SLF001 # pyright: ignore[reportPrivateUsage]
+
+ # ---- Linkup fetch ----
+
+ def test_linkup_fetch_happy_path(self) -> None:
+ """A valid fetch result in choices content becomes a single page at index 0,
+ with markdown, raw_html, and images whose empty alt maps to caption None.
+ """
+ fetch_payload = {
+ "markdown": "# Fetched",
+ "raw_html": "<html><body>hi</body></html>",
+ "images": [
+ {"url": "https://site.example/a.png", "alt": ""},
+ {"url": "https://site.example/b.png", "alt": "Logo"},
+ ],
+ }
+ response = _choices_response(json.dumps(fetch_payload))
+
+ extract_output = GatewayCompletionsFactory._make_extract_output_from_response_linkup_fetch(response=response) # noqa: SLF001 # pyright: ignore[reportPrivateUsage]
+
+ assert list(extract_output.pages.keys()) == [0]
+ page = extract_output.pages[0]
+ assert page.text == "# Fetched"
+ assert page.raw_html == "<html><body>hi</body></html>"
+ assert len(page.extracted_images) == 2
+ first_image = page.extracted_images[0]
+ assert first_image.actual_url == "https://site.example/a.png"
+ assert first_image.caption is None
+ assert first_image.mime_type is None
+ second_image = page.extracted_images[1]
+ assert second_image.actual_url == "https://site.example/b.png"
+ assert second_image.caption == "Logo"
+
+ @pytest.mark.parametrize(
+ "payload",
+ [
+ {"choices": [{"message": "nope"}]},
+ {"choices": [{"message": {"content": 42}}]},
+ ],
+ ids=["message-not-a-dict", "content-not-a-str"],
+ )
+ def test_extract_content_string_falsy_paths_return_none(self, payload: dict[str, Any]) -> None:
+ """Malformed message/content shapes yield None from the content-string extractor."""
+ result = GatewayCompletionsFactory._extract_content_string_from_response(response=_response(payload)) # noqa: SLF001 # pyright: ignore[reportPrivateUsage]
+
+ assert result is None
+
+ def test_linkup_fetch_without_content_raises(self) -> None:
+ """A response with no usable choices content cannot be parsed as a fetch result."""
+ with pytest.raises(GatewayExtractResponseError, match="does not contain content"):
+ GatewayCompletionsFactory._make_extract_output_from_response_linkup_fetch(response=_response({"success": True})) # noqa: SLF001 # pyright: ignore[reportPrivateUsage]
+
+ @pytest.mark.parametrize(
+ "invalid_content",
+ ["[1, 2]", "{not valid json"],
+ ids=["json-but-not-a-fetch-result", "malformed-json"],
+ )
+ def test_linkup_fetch_invalid_fetch_result_is_wrapped(self, invalid_content: str) -> None:
+ """Content that is present but not a valid GatewayFetchResultResponse is wrapped."""
+ response = _choices_response(invalid_content)
+
+ with pytest.raises(GatewayExtractResponseError, match="Error parsing Gateway fetch response"):
+ GatewayCompletionsFactory._make_extract_output_from_response_linkup_fetch(response=response) # noqa: SLF001 # pyright: ignore[reportPrivateUsage]
diff --git a/tests/unit/pipelex/plugins/gateway/test_gateway_completions_messages.py b/tests/unit/pipelex/plugins/gateway/test_gateway_completions_messages.py
new file mode 100644
index 000000000..bca8291fe
--- /dev/null
+++ b/tests/unit/pipelex/plugins/gateway/test_gateway_completions_messages.py
@@ -0,0 +1,165 @@
+"""Tests for GatewayCompletionsFactory.make_simple_messages — the override that sends
+documents as ``image_url`` parts (which Portkey/Gateway translates per provider).
+
+File preparation is patched at the factory namespace; no filesystem or network access.
+"""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any
+
+import pytest
+
+from pipelex.cogt.image.prompt_image import PromptImageDetail
+from pipelex.plugins.gateway.gateway_completions_factory import GatewayCompletionsFactory
+from pipelex.tools.misc.filetype_utils import FileType
+from pipelex.tools.uri.prepared_file import PreparedFileBase64, PreparedFileHttpUrl, PreparedFileLocalPath
+
+if TYPE_CHECKING:
+ from pytest_mock import MockerFixture
+
+_FACTORY_NAMESPACE = "pipelex.plugins.gateway.gateway_completions_factory"
+
+_PNG_FILE_TYPE = FileType(extension="png", mime="image/png")
+_PDF_FILE_TYPE = FileType(extension="pdf", mime="application/pdf")
+
+
+def _make_llm_job(
+ mocker: MockerFixture,
+ system_text: str | None = None,
+ user_text: str | None = None,
+ has_images: bool = False,
+ has_documents: bool = False,
+ image_detail: PromptImageDetail | None = None,
+) -> Any:
+ """Create a mock LLM job carrying just the prompt fields make_simple_messages reads."""
+ job = mocker.MagicMock()
+ job.llm_prompt.system_text = system_text
+ job.llm_prompt.user_text = user_text
+ job.llm_prompt.user_images = [mocker.MagicMock()] if has_images else []
+ job.llm_prompt.user_documents = [mocker.MagicMock()] if has_documents else []
+ job.job_params.image_detail = image_detail
+ return job
+
+
[email protected](loop_scope="class")
+class TestGatewayCompletionsMessages:
+ """make_simple_messages builds system + user messages with text, image and document parts."""
+
+ async def test_system_and_user_text(self, mocker: MockerFixture) -> None:
+ """System text becomes the first message; user text becomes a text content part."""
+ factory = GatewayCompletionsFactory(is_http_url_enabled=True)
+ llm_job = _make_llm_job(mocker, system_text="You are concise.", user_text="Summarize this.")
+
+ messages = await factory.make_simple_messages(llm_job=llm_job)
+
+ assert messages == [
+ {"role": "system", "content": "You are concise."},
+ {"role": "user", "content": [{"type": "text", "text": "Summarize this."}]},
+ ]
+
+ @pytest.mark.parametrize(
+ ("image_detail", "expected_detail"),
+ [
+ (None, "auto"),
+ (PromptImageDetail.HIGH, "high"),
+ (PromptImageDetail.LOW, "low"),
+ ],
+ )
+ async def test_images_become_image_url_parts(
+ self,
+ mocker: MockerFixture,
+ image_detail: PromptImageDetail | None,
+ expected_detail: str,
+ ) -> None:
+ """Base64 images become data-URL image_url parts, HTTP images keep their raw URL,
+ and detail comes from job_params.image_detail (defaulting to AUTO).
+ """
+ factory = GatewayCompletionsFactory(is_http_url_enabled=True)
+ llm_job = _make_llm_job(mocker, user_text="Look at these.", has_images=True, image_detail=image_detail)
+ prepped_images = [
+ PreparedFileBase64(base64_data="QUJD", file_type=_PNG_FILE_TYPE),
+ PreparedFileHttpUrl(url="https://img.example/pic.png"),
+ ]
+ mock_prep_images = mocker.patch(
+ f"{_FACTORY_NAMESPACE}.prep_prompt_images",
+ new_callable=mocker.AsyncMock,
+ return_value=prepped_images,
+ )
+
+ messages = await factory.make_simple_messages(llm_job=llm_job)
+
+ mock_prep_images.assert_awaited_once_with(prompt_images=llm_job.llm_prompt.user_images, is_http_url_enabled=True)
+ assert len(messages) == 1
+ user_message = messages[0]
+ assert user_message["role"] == "user"
+ user_contents = user_message["content"]
+ assert user_contents == [
+ {"type": "text", "text": "Look at these."},
+ {"type": "image_url", "image_url": {"url": "data:image/png;base64,QUJD", "detail": expected_detail}},
+ {"type": "image_url", "image_url": {"url": "https://img.example/pic.png", "detail": expected_detail}},
+ ]
+
+ async def test_local_path_image_raises_type_error(self, mocker: MockerFixture) -> None:
+ """A PreparedFileLocalPath image is rejected — it should have been converted to base64."""
+ factory = GatewayCompletionsFactory(is_http_url_enabled=True)
+ llm_job = _make_llm_job(mocker, user_text="text", has_images=True)
+ mocker.patch(
+ f"{_FACTORY_NAMESPACE}.prep_prompt_images",
+ new_callable=mocker.AsyncMock,
+ return_value=[PreparedFileLocalPath(path="some/local/pic.png")],
+ )
+
+ with pytest.raises(TypeError, match="PreparedFileLocalPath is not supported for images"):
+ await factory.make_simple_messages(llm_job=llm_job)
+
+ async def test_base64_document_becomes_image_url_part_with_auto_detail(self, mocker: MockerFixture) -> None:
+ """Base64 documents are sent as image_url parts with a data URL and detail='auto',
+ and document preparation always disables HTTP URLs regardless of the factory flag.
+ """
+ factory = GatewayCompletionsFactory(is_http_url_enabled=True)
+ llm_job = _make_llm_job(mocker, user_text="Read this doc.", has_documents=True)
+ mock_prep_documents = mocker.patch(
+ f"{_FACTORY_NAMESPACE}.prep_prompt_documents",
+ new_callable=mocker.AsyncMock,
+ return_value=[PreparedFileBase64(base64_data="UERG", file_type=_PDF_FILE_TYPE)],
+ )
+
+ messages = await factory.make_simple_messages(llm_job=llm_job)
+
+ mock_prep_documents.assert_awaited_once_with(prompt_documents=llm_job.llm_prompt.user_documents, is_http_url_enabled=False)
+ assert messages == [
+ {
+ "role": "user",
+ "content": [
+ {"type": "text", "text": "Read this doc."},
+ {"type": "image_url", "image_url": {"url": "data:application/pdf;base64,UERG", "detail": "auto"}},
+ ],
+ },
+ ]
+
+ @pytest.mark.parametrize(
+ ("prepped_document", "expected_match"),
+ [
+ (PreparedFileHttpUrl(url="https://docs.example/file.pdf"), "PreparedFileHttpUrl is not supported for documents"),
+ (PreparedFileLocalPath(path="some/local/file.pdf"), "PreparedFileLocalPath is not supported for documents"),
+ ],
+ ids=["http-url-document", "local-path-document"],
+ )
+ async def test_non_base64_document_raises_type_error(
+ self,
+ mocker: MockerFixture,
+ prepped_document: PreparedFileHttpUrl | PreparedFileLocalPath,
+ expected_match: str,
+ ) -> None:
+ """HTTP-URL and local-path documents are rejected — only base64 is supported here."""
+ factory = GatewayCompletionsFactory(is_http_url_enabled=True)
+ llm_job = _make_llm_job(mocker, user_text="text", has_documents=True)
+ mocker.patch(
+ f"{_FACTORY_NAMESPACE}.prep_prompt_documents",
+ new_callable=mocker.AsyncMock,
+ return_value=[prepped_document],
+ )
+
+ with pytest.raises(TypeError, match=expected_match):
+ await factory.make_simple_messages(llm_job=llm_job)
diff --git a/tests/unit/pipelex/plugins/mistral/test_mistral_factory_document_prep.py b/tests/unit/pipelex/plugins/mistral/test_mistral_factory_document_prep.py
new file mode 100644
index 000000000..ffd43dda0
--- /dev/null
+++ b/tests/unit/pipelex/plugins/mistral/test_mistral_factory_document_prep.py
@@ -0,0 +1,180 @@
+"""Tests for MistralFactory OCR document preparation: URI-to-chunk conversion and file upload to Mistral."""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any, cast
+
+import pytest
+
+from pipelex.plugins.mistral.mistral_factory import MistralFactory
+from pipelex.tools.misc.filetype_utils import FileType
+from pipelex.tools.uri.prepared_file import PreparedFileBase64, PreparedFileHttpUrl, PreparedFileLocalPath
+
+if TYPE_CHECKING:
+ from pathlib import Path
+
+ from mistralai import Mistral
+ from pytest_mock import MockerFixture
+
+PNG_FILE_TYPE = FileType(extension="png", mime="image/png")
+PDF_FILE_TYPE = FileType(extension="pdf", mime="application/pdf")
+
+
+def _make_mistral_client(mocker: MockerFixture, *, uploaded_file_id: str = "file-abc", signed_url: str = "https://signed.example.com/doc") -> Any:
+ """Create a mock Mistral client with async files endpoints."""
+ mock_client = mocker.MagicMock(name="mistral_client")
+ uploaded_file = mocker.MagicMock(name="uploaded_file")
+ uploaded_file.id = uploaded_file_id
+ mock_client.files.upload_async = mocker.AsyncMock(return_value=uploaded_file)
+ signed_url_response = mocker.MagicMock(name="signed_url_response")
+ signed_url_response.url = signed_url
+ mock_client.files.get_signed_url_async = mocker.AsyncMock(return_value=signed_url_response)
+ return mock_client
+
+
[email protected](loop_scope="class")
+class TestMistralFactoryDocumentPrep:
+ # ---- make_mistral_image_url_chunk_from_uri ----
+
+ @pytest.mark.parametrize(
+ ("_topic", "prepared", "expected_url"),
+ [
+ (
+ "http_url_kept_as_is",
+ PreparedFileHttpUrl(url="https://example.com/pic.png"),
+ "https://example.com/pic.png",
+ ),
+ (
+ "base64_becomes_data_url",
+ PreparedFileBase64(base64_data="QUJD", file_type=PNG_FILE_TYPE),
+ "data:image/png;base64,QUJD",
+ ),
+ ],
+ )
+ async def test_make_mistral_image_url_chunk_from_uri(
+ self,
+ mocker: MockerFixture,
+ _topic: str,
+ prepared: Any,
+ expected_url: str,
+ ) -> None:
+ """HTTP URLs are kept as-is and base64 files become data URLs; local paths are not requested from preparation."""
+ prepare_mock = mocker.patch(
+ "pipelex.plugins.mistral.mistral_factory.prepare_file_from_uri",
+ return_value=prepared,
+ )
+
+ chunk = await MistralFactory.make_mistral_image_url_chunk_from_uri(uri="some://uri")
+
+ assert chunk == {"type": "image_url", "image_url": expected_url}
+ prepare_mock.assert_awaited_once_with(uri="some://uri", keep_http_url=True, keep_local_path=False)
+
+ async def test_make_mistral_image_url_chunk_local_path_raises(self, mocker: MockerFixture) -> None:
+ """An unexpected PreparedFileLocalPath (despite keep_local_path=False) raises a TypeError."""
+ mocker.patch(
+ "pipelex.plugins.mistral.mistral_factory.prepare_file_from_uri",
+ return_value=PreparedFileLocalPath(path="/fake/dir/pic.png"),
+ )
+
+ with pytest.raises(TypeError, match="Unexpected PreparedFileLocalPath"):
+ await MistralFactory.make_mistral_image_url_chunk_from_uri(uri="/fake/dir/pic.png")
+
+ # ---- make_mistral_document_url_chunk_from_uri ----
+
+ async def test_make_mistral_document_url_chunk_http_kept(self, mocker: MockerFixture) -> None:
+ """An HTTP document URL is kept as-is and the client is never used."""
+ prepare_mock = mocker.patch(
+ "pipelex.plugins.mistral.mistral_factory.prepare_file_from_uri",
+ return_value=PreparedFileHttpUrl(url="https://example.com/file.pdf"),
+ )
+ mock_client = _make_mistral_client(mocker)
+
+ chunk = await MistralFactory.make_mistral_document_url_chunk_from_uri(
+ mistral_client=cast("Mistral", mock_client),
+ uri="https://example.com/file.pdf",
+ )
+
+ assert chunk == {"type": "document_url", "document_url": "https://example.com/file.pdf"}
+ prepare_mock.assert_awaited_once_with(uri="https://example.com/file.pdf", keep_http_url=True, keep_local_path=True)
+ mock_client.files.upload_async.assert_not_awaited()
+ mock_client.files.get_signed_url_async.assert_not_awaited()
+
+ async def test_make_mistral_document_url_chunk_local_path_uploads(self, mocker: MockerFixture, tmp_path: Path) -> None:
+ """A local document is uploaded to Mistral for OCR and the chunk carries the signed URL of the uploaded file."""
+ local_file = tmp_path / "scan.pdf"
+ file_content = b"%PDF-1.7 fake pdf bytes"
+ local_file.write_bytes(file_content)
+ mocker.patch(
+ "pipelex.plugins.mistral.mistral_factory.prepare_file_from_uri",
+ return_value=PreparedFileLocalPath(path=str(local_file)),
+ )
+ mock_client = _make_mistral_client(mocker, uploaded_file_id="file-xyz", signed_url="https://signed.example.com/scan.pdf")
+
+ chunk = await MistralFactory.make_mistral_document_url_chunk_from_uri(
+ mistral_client=cast("Mistral", mock_client),
+ uri=str(local_file),
+ )
+
+ assert chunk == {"type": "document_url", "document_url": "https://signed.example.com/scan.pdf"}
+ mock_client.files.upload_async.assert_awaited_once_with(
+ file={"file_name": "scan.pdf", "content": file_content},
+ purpose="ocr",
+ )
+ mock_client.files.get_signed_url_async.assert_awaited_once_with(file_id="file-xyz")
+
+ async def test_make_mistral_document_url_chunk_base64_two_pass(self, mocker: MockerFixture) -> None:
+ """A base64 first-pass result triggers a second preparation pass with both keep flags False, yielding a data URL."""
+ first_pass = PreparedFileBase64(base64_data="Rmly", file_type=PDF_FILE_TYPE)
+ second_pass = PreparedFileBase64(base64_data="U2Vj", file_type=PDF_FILE_TYPE)
+ prepare_mock = mocker.patch(
+ "pipelex.plugins.mistral.mistral_factory.prepare_file_from_uri",
+ side_effect=[first_pass, second_pass],
+ )
+ mock_client = _make_mistral_client(mocker)
+
+ chunk = await MistralFactory.make_mistral_document_url_chunk_from_uri(
+ mistral_client=cast("Mistral", mock_client),
+ uri="data:application/pdf;base64,Rmly",
+ )
+
+ assert chunk == {"type": "document_url", "document_url": "data:application/pdf;base64,U2Vj"}
+ assert prepare_mock.await_count == 2
+ prepare_mock.assert_awaited_with(uri="data:application/pdf;base64,Rmly", keep_http_url=False, keep_local_path=False)
+ mock_client.files.upload_async.assert_not_awaited()
+
+ async def test_make_mistral_document_url_chunk_base64_second_pass_failure_raises(self, mocker: MockerFixture) -> None:
+ """If the second preparation pass does not yield a base64 file, a TypeError is raised."""
+ mocker.patch(
+ "pipelex.plugins.mistral.mistral_factory.prepare_file_from_uri",
+ side_effect=[
+ PreparedFileBase64(base64_data="Rmly", file_type=PDF_FILE_TYPE),
+ PreparedFileHttpUrl(url="https://example.com/file.pdf"),
+ ],
+ )
+ mock_client = _make_mistral_client(mocker)
+
+ with pytest.raises(TypeError, match="Failed to convert URI to base64"):
+ await MistralFactory.make_mistral_document_url_chunk_from_uri(
+ mistral_client=cast("Mistral", mock_client),
+ uri="data:application/pdf;base64,Rmly",
+ )
+
+ # ---- upload_file_to_mistral_for_ocr ----
+
+ async def test_upload_file_to_mistral_for_ocr(self, mocker: MockerFixture, tmp_path: Path) -> None:
+ """The file is read from disk and uploaded with its name, bytes content, and the ocr purpose; the uploaded id is returned."""
+ local_file = tmp_path / "report.pdf"
+ file_content = b"%PDF-1.7 report bytes"
+ local_file.write_bytes(file_content)
+ mock_client = _make_mistral_client(mocker, uploaded_file_id="file-123")
+
+ uploaded_file_id = await MistralFactory.upload_file_to_mistral_for_ocr(
+ mistral_client=cast("Mistral", mock_client),
+ file_path=local_file,
+ )
+
+ assert uploaded_file_id == "file-123"
+ mock_client.files.upload_async.assert_awaited_once_with(
+ file={"file_name": "report.pdf", "content": file_content},
+ purpose="ocr",
+ )
diff --git a/tests/unit/pipelex/plugins/mistral/test_mistral_factory_extract_output.py b/tests/unit/pipelex/plugins/mistral/test_mistral_factory_extract_output.py
new file mode 100644
index 000000000..522a91916
--- /dev/null
+++ b/tests/unit/pipelex/plugins/mistral/test_mistral_factory_extract_output.py
@@ -0,0 +1,202 @@
+"""Tests for MistralFactory OCR response conversion: base64 cleaning, extracted image building, and ExtractOutput assembly."""
+
+from __future__ import annotations
+
+import base64
+
+import pytest
+from mistralai.models import OCRImageObject, OCRPageObject, OCRResponse, OCRUsageInfo
+
+from pipelex.plugins.mistral.mistral_exceptions import MistralExtractResponseError
+from pipelex.plugins.mistral.mistral_factory import MistralFactory
+
+JPEG_BODY = b"\xff\xd8\xff\xe0\x00\x10JFIF-jpeg-payload-bytes"
+PNG_BODY = b"\x89PNG\r\n\x1a\npng-payload-bytes"
+
+
+def _encode(raw_bytes: bytes) -> str:
+ return base64.b64encode(raw_bytes).decode("ascii")
+
+
+def _make_ocr_image_obj(
+ *,
+ image_base64: str | None,
+ top_left_x: int | None = None,
+ top_left_y: int | None = None,
+ bottom_right_x: int | None = None,
+ bottom_right_y: int | None = None,
+) -> OCRImageObject:
+ return OCRImageObject(
+ id="img-0.jpeg",
+ top_left_x=top_left_x,
+ top_left_y=top_left_y,
+ bottom_right_x=bottom_right_x,
+ bottom_right_y=bottom_right_y,
+ image_base64=image_base64,
+ )
+
+
+class TestMistralFactoryExtractOutput:
+ # ---- _clean_mistral_image_base64 ----
+
+ @pytest.mark.parametrize(
+ ("_topic", "raw_bytes"),
+ [
+ ("already_jpeg", JPEG_BODY),
+ ("already_png", PNG_BODY),
+ ],
+ )
+ def test_clean_base64_already_clean_returned_unchanged(self, _topic: str, raw_bytes: bytes) -> None:
+ """Data already starting with a JPEG or PNG magic number is returned as the exact same base64 string."""
+ base64_str = _encode(raw_bytes)
+ cleaned = MistralFactory._clean_mistral_image_base64(base64_str) # noqa: SLF001 # pyright: ignore[reportPrivateUsage]
+ assert cleaned == base64_str
+
+ @pytest.mark.parametrize(
+ ("_topic", "image_body"),
+ [
+ ("jpeg_with_metadata", JPEG_BODY),
+ ("png_with_metadata", PNG_BODY),
+ ],
+ )
+ def test_clean_base64_strips_prepended_metadata(self, _topic: str, image_body: bytes) -> None:
+ """Metadata bytes prepended before the image magic number are stripped, leaving exactly the image bytes."""
+ base64_str = _encode(b"METADATA" + image_body)
+ cleaned = MistralFactory._clean_mistral_image_base64(base64_str) # noqa: SLF001 # pyright: ignore[reportPrivateUsage]
+ assert base64.b64decode(cleaned) == image_body
+
+ @pytest.mark.parametrize(
+ ("_topic", "raw_bytes"),
+ [
+ ("no_magic_at_all", b"\x00\x01\x02\x03" * 10),
+ ("magic_beyond_scan_window", b"\x00" * 33 + JPEG_BODY),
+ ],
+ )
+ def test_clean_base64_no_magic_in_window_returns_original(self, _topic: str, raw_bytes: bytes) -> None:
+ """When no image magic number appears in the first 32 bytes, the original string is returned untouched."""
+ base64_str = _encode(raw_bytes)
+ cleaned = MistralFactory._clean_mistral_image_base64(base64_str) # noqa: SLF001 # pyright: ignore[reportPrivateUsage]
+ assert cleaned == base64_str
+
+ def test_clean_base64_invalid_input_returns_original(self) -> None:
+ """An input that fails base64 decoding is returned unmodified (ValueError branch)."""
+ invalid_base64 = "not-valid-base64!!!"
+ cleaned = MistralFactory._clean_mistral_image_base64(invalid_base64) # noqa: SLF001 # pyright: ignore[reportPrivateUsage]
+ assert cleaned == invalid_base64
+
+ # ---- make_extracted_image_from_page_from_mistral_ocr_image_obj ----
+
+ def test_make_extracted_image_missing_base64_raises(self) -> None:
+ """An OCR image object without base64 data raises MistralExtractResponseError."""
+ ocr_image_obj = _make_ocr_image_obj(image_base64=None, top_left_x=0, top_left_y=0, bottom_right_x=10, bottom_right_y=10)
+ with pytest.raises(MistralExtractResponseError, match="does not have an image base64"):
+ MistralFactory.make_extracted_image_from_page_from_mistral_ocr_image_obj(ocr_image_obj)
+
+ def test_make_extracted_image_full_coords(self) -> None:
+ """Full corner coordinates produce an ImageSize (bottom-right minus top-left) and a four-corner BoundingBox; mime type is pinned to JPEG."""
+ jpeg_base64 = _encode(JPEG_BODY)
+ ocr_image_obj = _make_ocr_image_obj(
+ image_base64=jpeg_base64,
+ top_left_x=10,
+ top_left_y=20,
+ bottom_right_x=110,
+ bottom_right_y=220,
+ )
+
+ extracted_image = MistralFactory.make_extracted_image_from_page_from_mistral_ocr_image_obj(ocr_image_obj)
+
+ assert extracted_image.base64_str == jpeg_base64
+ assert extracted_image.mime_type == "image/jpeg"
+ assert extracted_image.size is not None
+ assert extracted_image.size.width == 100
+ assert extracted_image.size.height == 200
+ bounding_box = extracted_image.bounding_box
+ assert bounding_box is not None
+ assert bounding_box.top_left_x == 10
+ assert bounding_box.top_left_y == 20
+ assert bounding_box.top_right_x == 110
+ assert bounding_box.top_right_y == 20
+ assert bounding_box.bottom_right_x == 110
+ assert bounding_box.bottom_right_y == 220
+ assert bounding_box.bottom_left_x == 10
+ assert bounding_box.bottom_left_y == 220
+
+ @pytest.mark.parametrize(
+ ("_topic", "top_left_x", "top_left_y", "bottom_right_x", "bottom_right_y"),
+ [
+ ("all_coords_missing", None, None, None, None),
+ ("only_top_left", 10, 20, None, None),
+ ("only_x_pair", 10, None, 110, None),
+ ],
+ )
+ def test_make_extracted_image_partial_coords(
+ self,
+ _topic: str,
+ top_left_x: int | None,
+ top_left_y: int | None,
+ bottom_right_x: int | None,
+ bottom_right_y: int | None,
+ ) -> None:
+ """Partial corner coordinates yield no size and no bounding box, but the image itself is still extracted."""
+ jpeg_base64 = _encode(JPEG_BODY)
+ ocr_image_obj = _make_ocr_image_obj(
+ image_base64=jpeg_base64,
+ top_left_x=top_left_x,
+ top_left_y=top_left_y,
+ bottom_right_x=bottom_right_x,
+ bottom_right_y=bottom_right_y,
+ )
+
+ extracted_image = MistralFactory.make_extracted_image_from_page_from_mistral_ocr_image_obj(ocr_image_obj)
+
+ assert extracted_image.size is None
+ assert extracted_image.bounding_box is None
+ assert extracted_image.base64_str == jpeg_base64
+ assert extracted_image.mime_type == "image/jpeg"
+
+ # ---- make_extract_output_from_mistral_response ----
+
+ @pytest.mark.asyncio
+ async def test_make_extract_output_from_mistral_response(self) -> None:
+ """An OCR response with a page carrying an image and a page without is converted to ExtractOutput keyed by page index."""
+ jpeg_base64 = _encode(JPEG_BODY)
+ page_with_image = OCRPageObject(
+ index=0,
+ markdown="# Page with image",
+ images=[
+ _make_ocr_image_obj(
+ image_base64=jpeg_base64,
+ top_left_x=5,
+ top_left_y=6,
+ bottom_right_x=15,
+ bottom_right_y=26,
+ )
+ ],
+ dimensions=None,
+ )
+ page_without_image = OCRPageObject(
+ index=2,
+ markdown="# Page without image",
+ images=[],
+ dimensions=None,
+ )
+ mistral_response = OCRResponse(
+ pages=[page_with_image, page_without_image],
+ model="mistral-ocr-latest",
+ usage_info=OCRUsageInfo(pages_processed=2),
+ )
+
+ extract_output = await MistralFactory.make_extract_output_from_mistral_response(mistral_extract_response=mistral_response)
+
+ assert set(extract_output.pages.keys()) == {0, 2}
+ first_page = extract_output.pages[0]
+ assert first_page.text == "# Page with image"
+ assert len(first_page.extracted_images) == 1
+ extracted_image = first_page.extracted_images[0]
+ assert extracted_image.base64_str == jpeg_base64
+ assert extracted_image.size is not None
+ assert extracted_image.size.width == 10
+ assert extracted_image.size.height == 20
+ second_page = extract_output.pages[2]
+ assert second_page.text == "# Page without image"
+ assert second_page.extracted_images == []
diff --git a/tests/unit/pipelex/plugins/mistral/test_mistral_factory_messages.py b/tests/unit/pipelex/plugins/mistral/test_mistral_factory_messages.py
new file mode 100644
index 000000000..ad6e9cabd
--- /dev/null
+++ b/tests/unit/pipelex/plugins/mistral/test_mistral_factory_messages.py
@@ -0,0 +1,335 @@
+"""Tests for MistralFactory message construction: simple messages, chunk conversion, OpenAI-typed messages, and token usage mapping."""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any, cast
+
+import pytest
+from mistralai.models import DocumentURLChunk, ImageURLChunk, SystemMessage, TextChunk, UsageInfo, UserMessage
+
+from pipelex.cogt.image.prompt_image import PromptImageDetail
+from pipelex.cogt.usage.token_category import TokenCategory
+from pipelex.plugins.mistral.mistral_factory import MistralFactory
+from pipelex.tools.misc.filetype_utils import FileType
+from pipelex.tools.uri.prepared_file import PreparedFileBase64, PreparedFileHttpUrl, PreparedFileLocalPath
+
+if TYPE_CHECKING:
+ from pytest_mock import MockerFixture
+
+PNG_FILE_TYPE = FileType(extension="png", mime="image/png")
+PDF_FILE_TYPE = FileType(extension="pdf", mime="application/pdf")
+
+
+def _make_llm_job(
+ mocker: MockerFixture,
+ *,
+ user_text: str | None = None,
+ user_images: list[Any] | None = None,
+ user_documents: list[Any] | None = None,
+ system_text: str | None = None,
+ image_detail: PromptImageDetail | None = None,
+) -> Any:
+ """Create a mock LLM job exposing only the prompt fields read by MistralFactory."""
+ llm_job = mocker.MagicMock()
+ llm_job.llm_prompt.user_text = user_text
+ llm_job.llm_prompt.user_images = user_images
+ llm_job.llm_prompt.user_documents = user_documents
+ llm_job.llm_prompt.system_text = system_text
+ llm_job.job_params.image_detail = image_detail
+ return llm_job
+
+
[email protected](loop_scope="class")
+class TestMistralFactoryMessages:
+ # ---- make_simple_messages ----
+
+ async def test_make_simple_messages_text_only(self, mocker: MockerFixture) -> None:
+ """A text-only prompt yields a single UserMessage holding one TextChunk with the exact text."""
+ factory = MistralFactory()
+ messages = await factory.make_simple_messages(llm_job=_make_llm_job(mocker, user_text="Describe the picture"))
+
+ assert len(messages) == 1
+ user_message = messages[0]
+ assert isinstance(user_message, UserMessage)
+ assert isinstance(user_message.content, list)
+ assert len(user_message.content) == 1
+ text_chunk = user_message.content[0]
+ assert isinstance(text_chunk, TextChunk)
+ assert text_chunk.text == "Describe the picture"
+
+ async def test_make_simple_messages_system_first_then_user(self, mocker: MockerFixture) -> None:
+ """The SystemMessage comes FIRST, followed by the UserMessage, matching the docstring and the OpenAI-typed sibling."""
+ factory = MistralFactory()
+ messages = await factory.make_simple_messages(llm_job=_make_llm_job(mocker, user_text="user words", system_text="system words"))
+
+ assert len(messages) == 2
+ system_message = messages[0]
+ assert isinstance(system_message, SystemMessage)
+ assert system_message.content == "system words"
+ assert isinstance(messages[1], UserMessage)
+
+ @pytest.mark.parametrize(
+ ("system_text", "expected_count"),
+ [
+ ("only the system speaks", 1),
+ (None, 0),
+ ],
+ )
+ async def test_make_simple_messages_without_user_content(
+ self,
+ mocker: MockerFixture,
+ system_text: str | None,
+ expected_count: int,
+ ) -> None:
+ """With no user content, only the system message (if any) is emitted; otherwise the list is empty."""
+ factory = MistralFactory()
+ messages = await factory.make_simple_messages(llm_job=_make_llm_job(mocker, system_text=system_text))
+
+ assert len(messages) == expected_count
+ if system_text is not None:
+ system_message = messages[0]
+ assert isinstance(system_message, SystemMessage)
+ assert system_message.content == system_text
+
+ async def test_make_simple_messages_with_images_preserves_order(self, mocker: MockerFixture) -> None:
+ """Image chunks are appended after the text chunk, in the same order as the prompt images."""
+ image_one = mocker.MagicMock(name="image_one")
+ image_two = mocker.MagicMock(name="image_two")
+ prepared_by_image_id: dict[int, Any] = {
+ id(image_one): PreparedFileHttpUrl(url="https://example.com/one.png"),
+ id(image_two): PreparedFileBase64(base64_data="QUJD", file_type=PNG_FILE_TYPE),
+ }
+
+ def fake_prepare_prompt_image(prompt_image: Any, is_http_url_enabled: bool) -> Any:
+ assert is_http_url_enabled is True
+ return prepared_by_image_id[id(prompt_image)]
+
+ mocker.patch(
+ "pipelex.plugins.mistral.mistral_factory.prepare_prompt_image",
+ side_effect=fake_prepare_prompt_image,
+ )
+
+ factory = MistralFactory()
+ messages = await factory.make_simple_messages(
+ llm_job=_make_llm_job(mocker, user_text="look at these", user_images=[image_one, image_two]),
+ )
+
+ assert len(messages) == 1
+ user_message = messages[0]
+ assert isinstance(user_message, UserMessage)
+ assert isinstance(user_message.content, list)
+ assert len(user_message.content) == 3
+ assert isinstance(user_message.content[0], TextChunk)
+ chunk_one = user_message.content[1]
+ chunk_two = user_message.content[2]
+ assert isinstance(chunk_one, ImageURLChunk)
+ assert isinstance(chunk_two, ImageURLChunk)
+ assert chunk_one.image_url == "https://example.com/one.png"
+ assert chunk_two.image_url == "data:image/png;base64,QUJD"
+
+ async def test_make_simple_messages_with_documents(self, mocker: MockerFixture) -> None:
+ """Document chunks are appended in prompt order, each prepared via prep_prompt_documents with HTTP URLs enabled."""
+ document_one = mocker.MagicMock(name="document_one")
+ document_two = mocker.MagicMock(name="document_two")
+ prepared_by_document_id: dict[int, Any] = {
+ id(document_one): PreparedFileHttpUrl(url="https://example.com/one.pdf"),
+ id(document_two): PreparedFileBase64(base64_data="UERG", file_type=PDF_FILE_TYPE),
+ }
+
+ def fake_prep_prompt_documents(prompt_documents: list[Any], is_http_url_enabled: bool) -> list[Any]:
+ assert is_http_url_enabled is True
+ assert len(prompt_documents) == 1
+ return [prepared_by_document_id[id(prompt_documents[0])]]
+
+ mocker.patch(
+ "pipelex.plugins.mistral.mistral_factory.prep_prompt_documents",
+ side_effect=fake_prep_prompt_documents,
+ )
+
+ factory = MistralFactory()
+ messages = await factory.make_simple_messages(
+ llm_job=_make_llm_job(mocker, user_documents=[document_one, document_two]),
+ )
+
+ assert len(messages) == 1
+ user_message = messages[0]
+ assert isinstance(user_message, UserMessage)
+ assert isinstance(user_message.content, list)
+ assert len(user_message.content) == 2
+ chunk_one = user_message.content[0]
+ chunk_two = user_message.content[1]
+ assert isinstance(chunk_one, DocumentURLChunk)
+ assert isinstance(chunk_two, DocumentURLChunk)
+ assert chunk_one.document_url == "https://example.com/one.pdf"
+ assert chunk_two.document_url == "data:application/pdf;base64,UERG"
+
+ # ---- make_mistral_image_url ----
+
+ @pytest.mark.parametrize(
+ ("prepared", "expected_url"),
+ [
+ (PreparedFileBase64(base64_data="QUJD", file_type=PNG_FILE_TYPE), "data:image/png;base64,QUJD"),
+ (PreparedFileHttpUrl(url="https://example.com/pic.png"), "https://example.com/pic.png"),
+ ],
+ )
+ async def test_make_mistral_image_url(
+ self,
+ mocker: MockerFixture,
+ prepared: Any,
+ expected_url: str,
+ ) -> None:
+ """Base64 files become data URLs and HTTP URLs are passed through; HTTP URLs are enabled at preparation."""
+ prepare_mock = mocker.patch(
+ "pipelex.plugins.mistral.mistral_factory.prepare_prompt_image",
+ return_value=prepared,
+ )
+ prompt_image = mocker.MagicMock(name="prompt_image")
+
+ factory = MistralFactory()
+ chunk = await factory.make_mistral_image_url(prompt_image=prompt_image)
+
+ assert chunk.image_url == expected_url
+ prepare_mock.assert_awaited_once_with(prompt_image=prompt_image, is_http_url_enabled=True)
+
+ async def test_make_mistral_image_url_local_path_raises(self, mocker: MockerFixture) -> None:
+ """A PreparedFileLocalPath is rejected with a TypeError for images."""
+ mocker.patch(
+ "pipelex.plugins.mistral.mistral_factory.prepare_prompt_image",
+ return_value=PreparedFileLocalPath(path="/fake/dir/pic.png"),
+ )
+
+ factory = MistralFactory()
+ with pytest.raises(TypeError, match="PreparedFileLocalPath is not supported for images"):
+ await factory.make_mistral_image_url(prompt_image=mocker.MagicMock(name="prompt_image"))
+
+ # ---- make_mistral_document_url ----
+
+ @pytest.mark.parametrize(
+ ("prepared", "expected_url"),
+ [
+ (PreparedFileBase64(base64_data="UERG", file_type=PDF_FILE_TYPE), "data:application/pdf;base64,UERG"),
+ (PreparedFileHttpUrl(url="https://example.com/file.pdf"), "https://example.com/file.pdf"),
+ ],
+ )
+ async def test_make_mistral_document_url(
+ self,
+ mocker: MockerFixture,
+ prepared: Any,
+ expected_url: str,
+ ) -> None:
+ """Base64 documents become data URLs and HTTP URLs are passed through; HTTP URLs are enabled at preparation."""
+ prep_mock = mocker.patch(
+ "pipelex.plugins.mistral.mistral_factory.prep_prompt_documents",
+ return_value=[prepared],
+ )
+ prompt_document = mocker.MagicMock(name="prompt_document")
+
+ factory = MistralFactory()
+ chunk = await factory.make_mistral_document_url(prompt_document=prompt_document)
+
+ assert chunk.document_url == expected_url
+ prep_mock.assert_awaited_once_with(prompt_documents=[prompt_document], is_http_url_enabled=True)
+
+ async def test_make_mistral_document_url_local_path_raises(self, mocker: MockerFixture) -> None:
+ """A PreparedFileLocalPath is rejected with a TypeError for documents."""
+ mocker.patch(
+ "pipelex.plugins.mistral.mistral_factory.prep_prompt_documents",
+ return_value=[PreparedFileLocalPath(path="/fake/dir/file.pdf")],
+ )
+
+ factory = MistralFactory()
+ with pytest.raises(TypeError, match="PreparedFileLocalPath is not supported for documents"):
+ await factory.make_mistral_document_url(prompt_document=mocker.MagicMock(name="prompt_document"))
+
+ # ---- make_simple_messages_openai_typed ----
+
+ @pytest.mark.parametrize(
+ ("image_detail", "expected_detail"),
+ [
+ (None, "auto"),
+ (PromptImageDetail.HIGH, "high"),
+ (PromptImageDetail.LOW, "low"),
+ ],
+ )
+ async def test_make_simple_messages_openai_typed(
+ self,
+ mocker: MockerFixture,
+ image_detail: PromptImageDetail | None,
+ expected_detail: str,
+ ) -> None:
+ """OpenAI-typed messages put the system message FIRST, then a user message with text and image parts; a missing image_detail maps to AUTO."""
+ prep_mock = mocker.patch(
+ "pipelex.plugins.mistral.mistral_factory.prep_prompt_images",
+ return_value=[
+ PreparedFileBase64(base64_data="QUJD", file_type=PNG_FILE_TYPE),
+ PreparedFileHttpUrl(url="https://example.com/pic.png"),
+ ],
+ )
+ user_images: list[Any] = [mocker.MagicMock(name="image_one"), mocker.MagicMock(name="image_two")]
+ llm_job = _make_llm_job(
+ mocker,
+ user_text="user words",
+ user_images=user_images,
+ system_text="system words",
+ image_detail=image_detail,
+ )
+
+ factory = MistralFactory()
+ messages = await factory.make_simple_messages_openai_typed(llm_job=llm_job)
+
+ assert len(messages) == 2
+ assert messages[0] == {"role": "system", "content": "system words"}
+ user_message = cast("dict[str, Any]", messages[1])
+ assert user_message["role"] == "user"
+ content_parts = cast("list[dict[str, Any]]", user_message["content"])
+ assert len(content_parts) == 3
+ assert content_parts[0] == {"text": "user words", "type": "text"}
+ assert content_parts[1] == {
+ "image_url": {"url": "data:image/png;base64,QUJD", "detail": expected_detail},
+ "type": "image_url",
+ }
+ assert content_parts[2] == {
+ "image_url": {"url": "https://example.com/pic.png", "detail": expected_detail},
+ "type": "image_url",
+ }
+ prep_mock.assert_awaited_once_with(prompt_images=user_images, is_http_url_enabled=True)
+
+ async def test_make_simple_messages_openai_typed_local_path_raises(self, mocker: MockerFixture) -> None:
+ """A PreparedFileLocalPath image is rejected with a TypeError on the OpenAI-typed path."""
+ mocker.patch(
+ "pipelex.plugins.mistral.mistral_factory.prep_prompt_images",
+ return_value=[PreparedFileLocalPath(path="/fake/dir/pic.png")],
+ )
+ llm_job = _make_llm_job(mocker, user_images=[mocker.MagicMock(name="image_one")])
+
+ factory = MistralFactory()
+ with pytest.raises(TypeError, match="PreparedFileLocalPath is not supported for images"):
+ await factory.make_simple_messages_openai_typed(llm_job=llm_job)
+
+ # ---- make_nb_tokens_by_category ----
+
+ @pytest.mark.parametrize(
+ ("prompt_tokens", "completion_tokens", "expected_input", "expected_output"),
+ [
+ (120, 45, 120, 45),
+ (None, None, 0, 0),
+ (None, 7, 0, 7),
+ ],
+ )
+ async def test_make_nb_tokens_by_category(
+ self,
+ prompt_tokens: int | None,
+ completion_tokens: int | None,
+ expected_input: int,
+ expected_output: int,
+ ) -> None:
+ """Token usage maps prompt/completion tokens to INPUT/OUTPUT, falling back to 0 when the SDK reports None."""
+ usage = UsageInfo(prompt_tokens=prompt_tokens, completion_tokens=completion_tokens)
+
+ factory = MistralFactory()
+ nb_tokens_by_category = factory.make_nb_tokens_by_category(usage=usage)
+
+ assert nb_tokens_by_category == {
+ TokenCategory.INPUT: expected_input,
+ TokenCategory.OUTPUT: expected_output,
+ }
diff --git a/tests/unit/pipelex/temporal/test_codec_server.py b/tests/unit/pipelex/temporal/test_codec_server.py
new file mode 100644
index 000000000..886eb4f3e
--- /dev/null
+++ b/tests/unit/pipelex/temporal/test_codec_server.py
@@ -0,0 +1,187 @@
+"""Unit tests for the Temporal codec HTTP server (encode/decode endpoints + CORS)."""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, cast
+
+import pytest
+from aiohttp.test_utils import TestClient, TestServer
+from google.protobuf import json_format
+from temporalio.api.common.v1 import Payload, Payloads
+
+from pipelex.temporal.codec.codec_server import build_codec_server
+from pipelex.temporal.codec.storage_payload_codec import STORAGE_REF_ENCODING, StoragePayloadCodec
+from pipelex.tools.storage.exceptions import StorageFileNotFoundError
+from pipelex.tools.storage.in_memory_storage_provider import InMemoryStorageProvider
+
+if TYPE_CHECKING:
+ from collections.abc import Sequence
+
+ALLOWED_ORIGIN = "http://localhost:8233"
+OTHER_ORIGIN = "http://evil.example.com"
+SIZE_THRESHOLD = 64
+STORAGE_PREFIX = "test-codec/"
+JSON_HEADERS = {"Content-Type": "application/json"}
+
+
+class _RaisingCodec:
+ """Stub codec whose encode/decode always raise the configured exception."""
+
+ def __init__(self, exc: Exception) -> None:
+ self._exc = exc
+
+ async def encode(self, _payloads: Sequence[Payload]) -> list[Payload]:
+ raise self._exc
+
+ async def decode(self, _payloads: Sequence[Payload]) -> list[Payload]:
+ raise self._exc
+
+
+def _make_real_codec(storage: InMemoryStorageProvider) -> StoragePayloadCodec:
+ return StoragePayloadCodec(
+ storage_provider=storage,
+ size_threshold=SIZE_THRESHOLD,
+ storage_prefix=STORAGE_PREFIX,
+ )
+
+
+def _payloads_json(*payloads: Payload) -> bytes:
+ """Serialize payloads to the Payloads-proto JSON the Temporal Web UI sends."""
+ return json_format.MessageToJson(Payloads(payloads=list(payloads))).encode()
+
+
+def _parse_payloads(body: bytes) -> Payloads:
+ return json_format.Parse(body, Payloads())
+
+
+def _make_payload(size: int) -> Payload:
+ return Payload(metadata={"encoding": b"binary/plain"}, data=b"x" * size)
+
+
[email protected](loop_scope="class")
+class TestCodecServer:
+ @pytest.mark.parametrize("endpoint", ["/encode", "/decode"])
+ async def test_cors_preflight_allowed_origin(self, endpoint: str) -> None:
+ """OPTIONS preflight from an allowed origin gets the full CORS header set."""
+ app = build_codec_server(codec=_make_real_codec(InMemoryStorageProvider()), cors_origins=[ALLOWED_ORIGIN])
+ async with TestClient(TestServer(app)) as client:
+ response = await client.options(endpoint, headers={"Origin": ALLOWED_ORIGIN})
+
+ assert response.status == 200
+ assert response.headers["Access-Control-Allow-Origin"] == ALLOWED_ORIGIN
+ assert response.headers["Access-Control-Allow-Methods"] == "POST"
+ assert response.headers["Access-Control-Allow-Headers"] == "content-type,x-namespace"
+
+ @pytest.mark.parametrize(
+ ("topic", "origin_headers"),
+ [
+ ("disallowed_origin", {"Origin": OTHER_ORIGIN}),
+ ("missing_origin", {}),
+ ],
+ )
+ async def test_cors_preflight_rejected_origin(self, topic: str, origin_headers: dict[str, str]) -> None:
+ """OPTIONS preflight from a disallowed or missing origin gets no CORS headers."""
+ app = build_codec_server(codec=_make_real_codec(InMemoryStorageProvider()), cors_origins=[ALLOWED_ORIGIN])
+ async with TestClient(TestServer(app)) as client:
+ response = await client.options("/encode", headers=origin_headers)
+
+ assert response.status == 200
+ assert "Access-Control-Allow-Origin" not in response.headers, f"CORS leaked for {topic}"
+
+ async def test_non_json_content_type_rejected_with_cors(self) -> None:
+ """A POST without application/json content type returns 415, still carrying CORS headers."""
+ app = build_codec_server(codec=_make_real_codec(InMemoryStorageProvider()), cors_origins=[ALLOWED_ORIGIN])
+ async with TestClient(TestServer(app)) as client:
+ response = await client.post(
+ "/encode",
+ data=b"whatever",
+ headers={"Content-Type": "text/plain", "Origin": ALLOWED_ORIGIN},
+ )
+
+ assert response.status == 415
+ assert await response.text() == "Expected application/json"
+ assert response.headers["Access-Control-Allow-Origin"] == ALLOWED_ORIGIN
+
+ async def test_malformed_payloads_json_rejected(self) -> None:
+ """A POST body that is not valid Payloads JSON returns 400."""
+ app = build_codec_server(codec=_make_real_codec(InMemoryStorageProvider()), cors_origins=[ALLOWED_ORIGIN])
+ async with TestClient(TestServer(app)) as client:
+ response = await client.post("/encode", data=b"this is not json", headers=JSON_HEADERS)
+
+ assert response.status == 400
+ assert await response.text() == "Malformed Payloads JSON"
+
+ async def test_encode_decode_round_trip_over_http(self) -> None:
+ """A large payload is offloaded by /encode and fully restored by /decode."""
+ storage = InMemoryStorageProvider()
+ app = build_codec_server(codec=_make_real_codec(storage), cors_origins=[ALLOWED_ORIGIN])
+ original = _make_payload(SIZE_THRESHOLD + 100)
+ original_bytes = original.SerializeToString()
+
+ async with TestClient(TestServer(app)) as client:
+ encode_response = await client.post(
+ "/encode",
+ data=_payloads_json(original),
+ headers={**JSON_HEADERS, "Origin": ALLOWED_ORIGIN},
+ )
+
+ assert encode_response.status == 200
+ assert encode_response.content_type == "application/json"
+ assert encode_response.headers["Access-Control-Allow-Origin"] == ALLOWED_ORIGIN
+
+ encoded_body = await encode_response.read()
+ encoded = _parse_payloads(encoded_body)
+ assert len(encoded.payloads) == 1
+ assert encoded.payloads[0].metadata["encoding"] == STORAGE_REF_ENCODING
+ assert len(storage.root) == 1
+
+ decode_response = await client.post("/decode", data=encoded_body, headers=JSON_HEADERS)
+
+ assert decode_response.status == 200
+ decoded = _parse_payloads(await decode_response.read())
+ assert len(decoded.payloads) == 1
+ assert decoded.payloads[0].SerializeToString() == original_bytes
+
+ async def test_encode_small_payload_passes_through(self) -> None:
+ """A payload below the threshold comes back from /encode unchanged."""
+ storage = InMemoryStorageProvider()
+ app = build_codec_server(codec=_make_real_codec(storage), cors_origins=[ALLOWED_ORIGIN])
+ original = _make_payload(10)
+
+ async with TestClient(TestServer(app)) as client:
+ response = await client.post("/encode", data=_payloads_json(original), headers=JSON_HEADERS)
+
+ assert response.status == 200
+ encoded = _parse_payloads(await response.read())
+ assert encoded.payloads[0].SerializeToString() == original.SerializeToString()
+ assert len(storage.root) == 0
+
+ @pytest.mark.parametrize(
+ ("topic", "codec_exc", "expected_status", "expected_text"),
+ [
+ ("storage_object_missing", StorageFileNotFoundError("blob gone"), 404, "Storage object not found"),
+ ("storage_io_error", OSError("disk on fire"), 502, "Storage I/O error"),
+ ("file_not_found_is_io_error", FileNotFoundError("no such file"), 502, "Storage I/O error"),
+ ],
+ )
+ async def test_codec_failures_map_to_http_errors(
+ self,
+ topic: str,
+ codec_exc: Exception,
+ expected_status: int,
+ expected_text: str,
+ ) -> None:
+ """Storage lookup and I/O failures inside the codec map to diagnostic HTTP errors with CORS."""
+ raising_codec = cast("StoragePayloadCodec", _RaisingCodec(codec_exc))
+ app = build_codec_server(codec=raising_codec, cors_origins=[ALLOWED_ORIGIN])
+
+ async with TestClient(TestServer(app)) as client:
+ response = await client.post(
+ "/decode",
+ data=_payloads_json(_make_payload(10)),
+ headers={**JSON_HEADERS, "Origin": ALLOWED_ORIGIN},
+ )
+
+ assert response.status == expected_status, f"Wrong status for {topic}"
+ assert await response.text() == expected_text
+ assert response.headers["Access-Control-Allow-Origin"] == ALLOWED_ORIGIN
diff --git a/tests/unit/pipelex/temporal/test_temporal_connect.py b/tests/unit/pipelex/temporal/test_temporal_connect.py
new file mode 100644
index 000000000..d136452a4
--- /dev/null
+++ b/tests/unit/pipelex/temporal/test_temporal_connect.py
@@ -0,0 +1,159 @@
+"""Unit tests for temporal_connect — API-key resolution, TLS wiring, codec wiring, server selection."""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any
+
+import pytest
+
+from pipelex.temporal import temporal_connect
+from pipelex.temporal.config_temporal import SecretMethod, TemporalServerConfig
+from pipelex.temporal.exceptions import TemporalConfigError, TemporalServerError
+
+if TYPE_CHECKING:
+ from pytest_mock import MockerFixture
+
+TEST_NAMESPACE = "test-ns"
+TEST_TARGET_HOST = "temporal.example.com:7233"
+
+
+def _make_server_config(api_key_method: SecretMethod, api_key_id: str = "") -> TemporalServerConfig:
+ return TemporalServerConfig(
+ description="Test server",
+ target_host=TEST_TARGET_HOST,
+ namespace=TEST_NAMESPACE,
+ api_key_method=api_key_method,
+ api_key_id=api_key_id,
+ )
+
+
[email protected](loop_scope="class")
+class TestTemporalConnect:
+ @pytest.fixture
+ def connect_mocks(self, mocker: MockerFixture) -> dict[str, Any]:
+ """Stub the config, the Temporal SDK client, and the secret/env resolvers."""
+ config = mocker.MagicMock()
+ config.temporal.payload_codec_config.is_enabled = False
+ client_cls = mocker.patch.object(temporal_connect, "TemporalClient")
+ client_cls.connect = mocker.AsyncMock(return_value=mocker.sentinel.client)
+ return {
+ "config": config,
+ "get_config": mocker.patch.object(temporal_connect, "get_config", return_value=config),
+ "connect": client_cls.connect,
+ "get_required_env": mocker.patch.object(temporal_connect, "get_required_env", return_value="env-api-key"),
+ "get_secret": mocker.patch.object(temporal_connect, "get_secret", return_value="provider-api-key"),
+ }
+
+ async def test_no_api_key_connects_without_tls(self, connect_mocks: dict[str, Any]) -> None:
+ """api_key_method=none → plain connection: no key, no TLS, no RPC metadata, default converter."""
+ server_config = _make_server_config(SecretMethod.NONE)
+
+ client = await temporal_connect.connect_to_temporal_server(server_config=server_config)
+
+ assert client is connect_mocks["connect"].return_value
+ connect_kwargs = connect_mocks["connect"].call_args.kwargs
+ assert connect_kwargs["target_host"] == TEST_TARGET_HOST
+ assert connect_kwargs["namespace"] == TEST_NAMESPACE
+ assert connect_kwargs["api_key"] is None
+ assert connect_kwargs["tls"] is False
+ assert connect_kwargs["rpc_metadata"] == {}
+ assert connect_kwargs["data_converter"] is temporal_connect.data_converter
+ connect_mocks["get_required_env"].assert_not_called()
+ connect_mocks["get_secret"].assert_not_called()
+
+ async def test_env_var_api_key_enables_tls_and_namespace_metadata(self, connect_mocks: dict[str, Any]) -> None:
+ """api_key_method=env_var resolves the key from the env and switches on TLS + namespace metadata."""
+ server_config = _make_server_config(SecretMethod.ENV_VAR, api_key_id="TEMPORAL_API_KEY")
+
+ await temporal_connect.connect_to_temporal_server(server_config=server_config)
+
+ connect_mocks["get_required_env"].assert_called_once_with("TEMPORAL_API_KEY")
+ connect_kwargs = connect_mocks["connect"].call_args.kwargs
+ assert connect_kwargs["api_key"] == "env-api-key"
+ assert connect_kwargs["tls"] is True
+ assert connect_kwargs["rpc_metadata"] == {"temporal-namespace": TEST_NAMESPACE}
+
+ async def test_secret_provider_api_key_resolved_via_get_secret(self, connect_mocks: dict[str, Any]) -> None:
+ """api_key_method=secret_provider resolves the key from the secret provider."""
+ server_config = _make_server_config(SecretMethod.SECRET_PROVIDER, api_key_id="temporal-key-id")
+
+ await temporal_connect.connect_to_temporal_server(server_config=server_config)
+
+ connect_mocks["get_secret"].assert_called_once_with(secret_id="temporal-key-id")
+ connect_kwargs = connect_mocks["connect"].call_args.kwargs
+ assert connect_kwargs["api_key"] == "provider-api-key"
+ assert connect_kwargs["tls"] is True
+
+ @pytest.mark.parametrize("api_key_method", [SecretMethod.ENV_VAR, SecretMethod.SECRET_PROVIDER])
+ async def test_missing_api_key_id_raises_config_error(
+ self,
+ connect_mocks: dict[str, Any],
+ api_key_method: SecretMethod,
+ ) -> None:
+ """A key-bearing method without an api_key_id is a config error, raised before any connect attempt."""
+ server_config = _make_server_config(api_key_method, api_key_id="")
+
+ with pytest.raises(TemporalConfigError, match="api_key_id is required"):
+ await temporal_connect.connect_to_temporal_server(server_config=server_config)
+
+ connect_mocks["connect"].assert_not_called()
+
+ async def test_payload_codec_enabled_builds_custom_converter(
+ self,
+ connect_mocks: dict[str, Any],
+ mocker: MockerFixture,
+ ) -> None:
+ """When the payload codec is enabled, the converter is built from the configured codec."""
+ connect_mocks["config"].temporal.payload_codec_config.is_enabled = True
+ make_codec = mocker.patch.object(temporal_connect, "make_codec_from_config", return_value=mocker.sentinel.codec)
+ make_converter = mocker.patch.object(temporal_connect, "make_data_converter", return_value=mocker.sentinel.converter)
+
+ await temporal_connect.connect_to_temporal_server(server_config=_make_server_config(SecretMethod.NONE))
+
+ make_codec.assert_called_once_with()
+ make_converter.assert_called_once_with(payload_codec=mocker.sentinel.codec)
+ connect_kwargs = connect_mocks["connect"].call_args.kwargs
+ assert connect_kwargs["data_converter"] is mocker.sentinel.converter
+
+ async def test_sdk_runtime_error_wrapped_in_server_error(self, connect_mocks: dict[str, Any]) -> None:
+ """A RuntimeError from the SDK connect is wrapped in TemporalServerError with the server description."""
+ connect_mocks["connect"].side_effect = RuntimeError("connection refused")
+ server_config = _make_server_config(SecretMethod.NONE)
+
+ with pytest.raises(TemporalServerError, match="Failed to connect to Temporal server") as exc_info:
+ await temporal_connect.connect_to_temporal_server(server_config=server_config)
+
+ assert server_config.full_description in str(exc_info.value)
+ assert "connection refused" in str(exc_info.value)
+
+ async def test_selected_server_connects_with_named_config(self, connect_mocks: dict[str, Any]) -> None:
+ """connect_to_temporal_selected_server looks up the named config and connects with it."""
+ server_config = _make_server_config(SecretMethod.NONE)
+ connect_mocks["config"].temporal.temporal_config.temporal_server_configs = {"local": server_config}
+
+ client = await temporal_connect.connect_to_temporal_selected_server(selected_server_config="local")
+
+ assert client is connect_mocks["connect"].return_value
+ connect_kwargs = connect_mocks["connect"].call_args.kwargs
+ assert connect_kwargs["target_host"] == TEST_TARGET_HOST
+
+ async def test_selected_server_unknown_name_raises_config_error(self, connect_mocks: dict[str, Any]) -> None:
+ """An unknown server-config name is a config error, raised before any connect attempt."""
+ connect_mocks["config"].temporal.temporal_config.temporal_server_configs = {"local": _make_server_config(SecretMethod.NONE)}
+
+ with pytest.raises(TemporalConfigError, match="Server config not found for selected server: 'staging'"):
+ await temporal_connect.connect_to_temporal_selected_server(selected_server_config="staging")
+
+ connect_mocks["connect"].assert_not_called()
+
+ async def test_connect_to_temporal_uses_configured_selected_server(self, connect_mocks: dict[str, Any]) -> None:
+ """connect_to_temporal picks the server named by temporal_config.selected_server."""
+ server_config = _make_server_config(SecretMethod.NONE)
+ connect_mocks["config"].temporal.temporal_config.selected_server = "chosen"
+ connect_mocks["config"].temporal.temporal_config.temporal_server_configs = {"chosen": server_config}
+
+ client = await temporal_connect.connect_to_temporal()
+
+ assert client is connect_mocks["connect"].return_value
+ connect_kwargs = connect_mocks["connect"].call_args.kwargs
+ assert connect_kwargs["namespace"] == TEST_NAMESPACE
diff --git a/tests/unit/pipelex/temporal/test_worker_cli.py b/tests/unit/pipelex/temporal/test_worker_cli.py
new file mode 100644
index 000000000..b2bb0afaa
--- /dev/null
+++ b/tests/unit/pipelex/temporal/test_worker_cli.py
@@ -0,0 +1,209 @@
+"""Unit tests for the Temporal worker CLI entry point — project resolution, fast-fail queue check, library load, arg wiring."""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any
+
+import pytest
+from typer.testing import CliRunner
+
+from pipelex.system.runtime import RunMode
+from pipelex.temporal import worker_cli
+from pipelex.temporal.exceptions import WorkerTaskQueueUnknownError
+
+if TYPE_CHECKING:
+ from pytest_mock import MockerFixture
+
+DEFAULT_TASK_QUEUE = "default_queue"
+PYPROJECT_WITH_PROJECT_NAME = {"project": {"name": "my-project"}}
+
+
+class TestWorkerCli:
+ @pytest.fixture
+ def boot_mocks(self, mocker: MockerFixture) -> dict[str, Any]:
+ """Stub everything configure() touches: pyproject load, Pipelex boot, config, hub library getters, and the worker loop."""
+ config = mocker.MagicMock()
+ config.temporal.worker_config.default_task_queue = DEFAULT_TASK_QUEUE
+ config.temporal.is_enabled = True
+ library_manager = mocker.MagicMock()
+ return {
+ "config": config,
+ "load_toml": mocker.patch.object(worker_cli, "load_toml_from_path", return_value=PYPROJECT_WITH_PROJECT_NAME),
+ "pipelex_make": mocker.patch.object(worker_cli.Pipelex, "make"),
+ "get_config": mocker.patch.object(worker_cli, "get_config", return_value=config),
+ "set_run_mode": mocker.patch.object(type(worker_cli.runtime_manager), "set_run_mode"),
+ "library_manager": library_manager,
+ "get_library_manager": mocker.patch("pipelex.hub.get_library_manager", return_value=library_manager),
+ "set_current_library": mocker.patch("pipelex.hub.set_current_library"),
+ "resolve_library_dirs": mocker.patch("pipelex.hub.resolve_library_dirs", return_value=([], "PIPELEXPATH")),
+ "run_worker": mocker.patch.object(worker_cli, "run_worker", new=mocker.AsyncMock(return_value=None)),
+ }
+
+ @pytest.mark.parametrize("project", [None, "chosen-project"])
+ @pytest.mark.asyncio
+ async def test_run_worker_forwards_kwargs_to_task_manager(self, mocker: MockerFixture, project: str | None) -> None:
+ """run_worker hands all worker options to the task manager, for both implicit and explicit project."""
+ task_manager = mocker.MagicMock()
+ task_manager.run_worker = mocker.AsyncMock(return_value=None)
+ mocker.patch.object(worker_cli, "get_task_manager", return_value=task_manager)
+
+ await worker_cli.run_worker(
+ project=project,
+ is_not_sandboxed=True,
+ is_unit_testing=False,
+ task_queue="some_queue",
+ scope_name="router",
+ profile_name="anthropic-tier4",
+ )
+
+ task_manager.run_worker.assert_awaited_once_with(
+ is_not_sandboxed=True,
+ is_unit_testing=False,
+ task_queue="some_queue",
+ scope_name="router",
+ profile_name="anthropic-tier4",
+ )
+
+ def test_explicit_project_skips_pyproject(self, boot_mocks: dict[str, Any]) -> None:
+ """An explicit project argument bypasses pyproject.toml entirely."""
+ worker_cli.configure(project="explicit-project")
+
+ boot_mocks["load_toml"].assert_not_called()
+ boot_mocks["pipelex_make"].assert_called_once_with(temporal_enabled=True)
+ assert boot_mocks["run_worker"].call_args.args[0] == "explicit-project"
+
+ def test_project_resolved_from_pyproject_project_name(self, boot_mocks: dict[str, Any]) -> None:
+ """With no argument, the project name comes from [project].name in pyproject.toml."""
+ worker_cli.configure()
+
+ boot_mocks["load_toml"].assert_called_once_with(path="pyproject.toml")
+ assert boot_mocks["run_worker"].call_args.args[0] == "my-project"
+
+ def test_project_falls_back_to_poetry_name(self, boot_mocks: dict[str, Any]) -> None:
+ """When [project].name is absent, [tool.poetry].name is used."""
+ boot_mocks["load_toml"].return_value = {"tool": {"poetry": {"name": "poetry-project"}}}
+
+ worker_cli.configure()
+
+ assert boot_mocks["run_worker"].call_args.args[0] == "poetry-project"
+
+ def test_missing_project_name_raises(self, boot_mocks: dict[str, Any]) -> None:
+ """A pyproject.toml without any project name aborts before booting Pipelex."""
+ boot_mocks["load_toml"].return_value = {"tool": {"poetry": {}}}
+
+ with pytest.raises(ValueError, match=r"Project name not found in pyproject\.toml"):
+ worker_cli.configure()
+
+ boot_mocks["pipelex_make"].assert_not_called()
+
+ def test_unit_testing_flag_sets_run_mode(self, boot_mocks: dict[str, Any]) -> None:
+ """--is-unit-testing switches the runtime manager to UNIT_TEST mode."""
+ worker_cli.configure(is_unit_testing=True)
+
+ boot_mocks["set_run_mode"].assert_called_once_with(RunMode.UNIT_TEST)
+
+ def test_no_unit_testing_flag_leaves_run_mode_alone(self, boot_mocks: dict[str, Any]) -> None:
+ """Without the flag, the run mode is never touched."""
+ worker_cli.configure()
+
+ boot_mocks["set_run_mode"].assert_not_called()
+
+ @pytest.mark.parametrize(
+ ("topic", "task_queue", "expected_validated_queue"),
+ [
+ ("explicit_queue", "custom_queue", "custom_queue"),
+ ("default_queue_fallback", None, DEFAULT_TASK_QUEUE),
+ ],
+ )
+ def test_task_queue_validated_before_library_load(
+ self,
+ boot_mocks: dict[str, Any],
+ mocker: MockerFixture,
+ topic: str,
+ task_queue: str | None,
+ expected_validated_queue: str,
+ ) -> None:
+ """The effective task queue (explicit or config default) is validated against known queues."""
+ worker_cli.configure(task_queue=task_queue)
+
+ validate_mock = boot_mocks["config"].temporal.validate_task_queue_known
+ assert validate_mock.call_args_list == [mocker.call(expected_validated_queue)], f"Wrong validated queue for {topic}"
+
+ def test_unknown_task_queue_fast_fails_before_library_load(self, boot_mocks: dict[str, Any]) -> None:
+ """A typo'd task queue aborts before any library loading or worker start."""
+ boot_mocks["config"].temporal.validate_task_queue_known.side_effect = WorkerTaskQueueUnknownError("unknown queue 'tyop'")
+
+ with pytest.raises(WorkerTaskQueueUnknownError, match="unknown queue 'tyop'"):
+ worker_cli.configure(task_queue="tyop")
+
+ boot_mocks["get_library_manager"].assert_not_called()
+ boot_mocks["run_worker"].assert_not_called()
+
+ def test_library_loaded_from_resolved_dirs(self, boot_mocks: dict[str, Any]) -> None:
+ """Resolved library dirs are loaded into the worker_base library."""
+ boot_mocks["resolve_library_dirs"].return_value = (["libs_a", "libs_b"], "PIPELEXPATH")
+
+ worker_cli.configure()
+
+ boot_mocks["library_manager"].open_library.assert_called_once_with(library_id="worker_base")
+ boot_mocks["set_current_library"].assert_called_once_with(library_id="worker_base")
+ boot_mocks["resolve_library_dirs"].assert_called_once_with(library_dirs=None)
+ boot_mocks["library_manager"].load_libraries.assert_called_once_with(library_id="worker_base", library_dirs=["libs_a", "libs_b"])
+
+ def test_no_library_dirs_skips_load(self, boot_mocks: dict[str, Any]) -> None:
+ """With no resolved dirs (PIPELEXPATH unset), the library load is skipped but the library is still opened."""
+ worker_cli.configure()
+
+ boot_mocks["library_manager"].open_library.assert_called_once_with(library_id="worker_base")
+ boot_mocks["library_manager"].load_libraries.assert_not_called()
+
+ def test_temporal_disabled_in_config_is_forced_on(self, boot_mocks: dict[str, Any], mocker: MockerFixture) -> None:
+ """When temporal.is_enabled is false, worker mode forces it on via a config copy."""
+ temporal_mock = boot_mocks["config"].temporal
+ temporal_mock.is_enabled = False
+ temporal_mock.model_copy.return_value = mocker.sentinel.updated_temporal
+
+ worker_cli.configure()
+
+ temporal_mock.model_copy.assert_called_once_with(update={"is_enabled": True})
+ assert boot_mocks["config"].temporal is mocker.sentinel.updated_temporal
+
+ def test_temporal_enabled_in_config_is_left_alone(self, boot_mocks: dict[str, Any]) -> None:
+ """When temporal.is_enabled is already true, the config is not copied or reassigned."""
+ temporal_mock = boot_mocks["config"].temporal
+
+ worker_cli.configure()
+
+ temporal_mock.model_copy.assert_not_called()
+ assert boot_mocks["config"].temporal is temporal_mock
+
+ def test_cli_arg_wiring_reaches_run_worker(self, boot_mocks: dict[str, Any]) -> None:
+ """All CLI options travel through Typer parsing into run_worker, positionally correct."""
+ runner = CliRunner()
+
+ result = runner.invoke(
+ worker_cli.app,
+ [
+ "cli-project",
+ "--is-not-sandboxed",
+ "--is-unit-testing",
+ "--task-queue",
+ "queue_from_cli",
+ "--scope",
+ "router",
+ "--profile",
+ "anthropic-tier4",
+ ],
+ )
+
+ assert result.exit_code == 0, result.output
+ boot_mocks["run_worker"].assert_called_once_with("cli-project", True, True, "queue_from_cli", "router", "anthropic-tier4")
+
+ def test_cli_defaults_reach_run_worker(self, boot_mocks: dict[str, Any]) -> None:
+ """Invoking with no arguments resolves the project from pyproject and passes default options."""
+ runner = CliRunner()
+
+ result = runner.invoke(worker_cli.app, [])
+
+ assert result.exit_code == 0, result.output
+ boot_mocks["run_worker"].assert_called_once_with("my-project", False, False, None, None, None)
diff --git a/tests/unit/pipelex/tools/misc/test_document_utils.py b/tests/unit/pipelex/tools/misc/test_document_utils.py
new file mode 100644
index 000000000..318ee79de
--- /dev/null
+++ b/tests/unit/pipelex/tools/misc/test_document_utils.py
@@ -0,0 +1,111 @@
+import pytest
+
+from pipelex.tools.misc.document_utils import DocumentFormat
+
+
+class TestDocumentFormat:
+ @pytest.mark.parametrize(
+ ("document_format", "expected_is_pdf", "expected_is_docx", "expected_is_pptx"),
+ [
+ (DocumentFormat.PDF, True, False, False),
+ (DocumentFormat.DOCX, False, True, False),
+ (DocumentFormat.PPTX, False, False, True),
+ ],
+ )
+ def test_format_predicates(
+ self,
+ document_format: DocumentFormat,
+ expected_is_pdf: bool,
+ expected_is_docx: bool,
+ expected_is_pptx: bool,
+ ):
+ """Each member answers the full is_pdf/is_docx/is_pptx predicate matrix."""
+ assert document_format.is_pdf is expected_is_pdf
+ assert document_format.is_docx is expected_is_docx
+ assert document_format.is_pptx is expected_is_pptx
+
+ @pytest.mark.parametrize(
+ ("document_format", "expected_extension"),
+ [
+ (DocumentFormat.PDF, "pdf"),
+ (DocumentFormat.DOCX, "docx"),
+ (DocumentFormat.PPTX, "pptx"),
+ ],
+ )
+ def test_as_file_extension(self, document_format: DocumentFormat, expected_extension: str):
+ """as_file_extension returns the exact extension string for each member."""
+ assert document_format.as_file_extension == expected_extension
+
+ @pytest.mark.parametrize(
+ ("document_format", "expected_mime_type"),
+ [
+ (DocumentFormat.PDF, "application/pdf"),
+ (DocumentFormat.DOCX, "application/vnd.openxmlformats-officedocument.wordprocessingml.document"),
+ (DocumentFormat.PPTX, "application/vnd.openxmlformats-officedocument.presentationml.presentation"),
+ ],
+ )
+ def test_as_mime_type(self, document_format: DocumentFormat, expected_mime_type: str):
+ """as_mime_type returns the exact MIME type string for each member."""
+ assert document_format.as_mime_type == expected_mime_type
+
+ def test_get_supported_mime_types(self):
+ """get_supported_mime_types returns the exact frozenset of document MIME types."""
+ supported = DocumentFormat.get_supported_mime_types()
+ assert isinstance(supported, frozenset)
+ assert supported == frozenset(
+ {
+ "application/pdf",
+ "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
+ "application/vnd.openxmlformats-officedocument.presentationml.presentation",
+ }
+ )
+
+ @pytest.mark.parametrize("document_format", list(DocumentFormat))
+ def test_is_supported_mime_type_true_for_members(self, document_format: DocumentFormat):
+ """Every member's MIME type is reported as supported."""
+ assert DocumentFormat.is_supported_mime_type(document_format.as_mime_type) is True
+
+ @pytest.mark.parametrize(
+ "mime_type",
+ [
+ "image/png",
+ "not-a-mime-type",
+ "",
+ "application/PDF",
+ ],
+ )
+ def test_is_supported_mime_type_false_for_unsupported(self, mime_type: str):
+ """Non-document MIME types and garbage are reported as unsupported."""
+ assert DocumentFormat.is_supported_mime_type(mime_type) is False
+
+ @pytest.mark.parametrize("document_format", list(DocumentFormat))
+ def test_raise_if_unsupported_mime_type_passes_on_valid(self, document_format: DocumentFormat):
+ """Supported MIME types do not raise."""
+ DocumentFormat.raise_if_unsupported_mime_type(document_format.as_mime_type)
+
+ def test_raise_if_unsupported_mime_type_raises_with_supported_list(self):
+ """Unsupported MIME types raise ValueError naming the offender and listing supported types."""
+ with pytest.raises(ValueError, match="Unsupported document MIME type: image/png") as exc_info:
+ DocumentFormat.raise_if_unsupported_mime_type("image/png")
+ error_message = str(exc_info.value)
+ assert "application/pdf" in error_message
+ assert "application/vnd.openxmlformats-officedocument.wordprocessingml.document" in error_message
+ assert "application/vnd.openxmlformats-officedocument.presentationml.presentation" in error_message
+
+ @pytest.mark.parametrize("document_format", list(DocumentFormat))
+ def test_from_mime_type_round_trip(self, document_format: DocumentFormat):
+ """from_mime_type returns the exact member for its own MIME type."""
+ assert DocumentFormat.from_mime_type(document_format.as_mime_type) is document_format
+
+ @pytest.mark.parametrize(
+ "mime_type",
+ [
+ "image/png",
+ "application/msword",
+ "garbage",
+ ],
+ )
+ def test_from_mime_type_raises_on_unknown(self, mime_type: str):
+ """from_mime_type raises ValueError for unknown MIME types."""
+ with pytest.raises(ValueError, match=f"Unsupported document MIME type: {mime_type}"):
+ DocumentFormat.from_mime_type(mime_type)
diff --git a/tests/unit/pipelex/tools/misc/test_image_format.py b/tests/unit/pipelex/tools/misc/test_image_format.py
new file mode 100644
index 000000000..4b4a199bb
--- /dev/null
+++ b/tests/unit/pipelex/tools/misc/test_image_format.py
@@ -0,0 +1,107 @@
+import pytest
+
+from pipelex.tools.misc.image_utils import ImageFormat
+
+
+class TestImageFormat:
+ @pytest.mark.parametrize(
+ ("image_format", "expected_transparent", "expected_is_png", "expected_is_jpeg"),
+ [
+ (ImageFormat.PNG, True, True, False),
+ (ImageFormat.JPEG, False, False, True),
+ (ImageFormat.WEBP, False, False, False),
+ ],
+ )
+ def test_format_predicates(
+ self,
+ image_format: ImageFormat,
+ expected_transparent: bool,
+ expected_is_png: bool,
+ expected_is_jpeg: bool,
+ ):
+ """Each member answers the is_transparent_compatible/is_png/is_jpeg predicate matrix."""
+ assert image_format.is_transparent_compatible is expected_transparent
+ assert image_format.is_png is expected_is_png
+ assert image_format.is_jpeg is expected_is_jpeg
+
+ @pytest.mark.parametrize(
+ ("image_format", "expected_extension"),
+ [
+ (ImageFormat.PNG, "png"),
+ (ImageFormat.JPEG, "jpg"),
+ (ImageFormat.WEBP, "webp"),
+ ],
+ )
+ def test_as_file_extension(self, image_format: ImageFormat, expected_extension: str):
+ """as_file_extension returns the exact extension string; note JPEG maps to 'jpg'."""
+ assert image_format.as_file_extension == expected_extension
+
+ @pytest.mark.parametrize(
+ ("image_format", "expected_mime_type"),
+ [
+ (ImageFormat.PNG, "image/png"),
+ (ImageFormat.JPEG, "image/jpeg"),
+ (ImageFormat.WEBP, "image/webp"),
+ ],
+ )
+ def test_as_mime_type(self, image_format: ImageFormat, expected_mime_type: str):
+ """as_mime_type returns the exact MIME type string for each member."""
+ assert image_format.as_mime_type == expected_mime_type
+
+ @pytest.mark.parametrize("image_format", list(ImageFormat))
+ def test_from_mime_type_round_trip(self, image_format: ImageFormat):
+ """from_mime_type returns the exact member for its own MIME type."""
+ assert ImageFormat.from_mime_type(image_format.as_mime_type) is image_format
+
+ def test_from_mime_type_raises_on_unknown(self):
+ """from_mime_type raises ValueError for an unknown image MIME type."""
+ with pytest.raises(ValueError, match="Unsupported image MIME type: image/tiff"):
+ ImageFormat.from_mime_type("image/tiff")
+
+ def test_get_supported_mime_types(self):
+ """get_supported_mime_types returns the exact frozenset of image MIME types."""
+ supported = ImageFormat.get_supported_mime_types()
+ assert isinstance(supported, frozenset)
+ assert supported == frozenset({"image/png", "image/jpeg", "image/webp"})
+
+ @pytest.mark.parametrize("image_format", list(ImageFormat))
+ def test_is_supported_mime_type_true_for_members(self, image_format: ImageFormat):
+ """Every member's MIME type is reported as supported."""
+ assert ImageFormat.is_supported_mime_type(image_format.as_mime_type) is True
+
+ @pytest.mark.parametrize(
+ "mime_type",
+ [
+ "image/tiff",
+ "application/pdf",
+ "garbage",
+ "",
+ ],
+ )
+ def test_is_supported_mime_type_false_for_unsupported(self, mime_type: str):
+ """Non-image MIME types and garbage are reported as unsupported."""
+ assert ImageFormat.is_supported_mime_type(mime_type) is False
+
+ @pytest.mark.parametrize("image_format", list(ImageFormat))
+ def test_raise_if_unsupported_mime_type_passes_on_valid(self, image_format: ImageFormat):
+ """Supported MIME types do not raise."""
+ ImageFormat.raise_if_unsupported_mime_type(image_format.as_mime_type)
+
+ def test_raise_if_unsupported_mime_type_image_prefix_branch(self):
+ """An image/* MIME type that is not supported hits the 'Unsupported image MIME type' message."""
+ with pytest.raises(ValueError, match="Unsupported image MIME type: image/tiff") as exc_info:
+ ImageFormat.raise_if_unsupported_mime_type("image/tiff")
+ error_message = str(exc_info.value)
+ assert "image/jpeg" in error_message
+ assert "image/png" in error_message
+ assert "image/webp" in error_message
+
+ def test_raise_if_unsupported_mime_type_non_image_branch(self):
+ """A non-image MIME type hits the 'Invalid image MIME type' message with the expected-format hint."""
+ with pytest.raises(ValueError, match="Invalid image MIME type: application/pdf") as exc_info:
+ ImageFormat.raise_if_unsupported_mime_type("application/pdf")
+ error_message = str(exc_info.value)
+ assert "Expected format 'image/<subtype>'" in error_message
+ assert "image/jpeg" in error_message
+ assert "image/png" in error_message
+ assert "image/webp" in error_message
diff --git a/tests/unit/pipelex/tools/misc/test_pil_image_to_bytes.py b/tests/unit/pipelex/tools/misc/test_pil_image_to_bytes.py
new file mode 100644
index 000000000..15d660073
--- /dev/null
+++ b/tests/unit/pipelex/tools/misc/test_pil_image_to_bytes.py
@@ -0,0 +1,57 @@
+import io
+
+import pytest
+from PIL import Image
+
+from pipelex.tools.misc.image_utils import ImageFormat, pil_image_to_bytes
+
+
+class TestPilImageToBytes:
+ @pytest.mark.parametrize(
+ ("image_format", "expected_pil_format", "expected_magic_bytes"),
+ [
+ (ImageFormat.PNG, "PNG", b"\x89PNG"),
+ (ImageFormat.JPEG, "JPEG", b"\xff\xd8"),
+ (ImageFormat.WEBP, "WEBP", b"RIFF"),
+ ],
+ )
+ def test_converts_to_requested_format(
+ self,
+ image_format: ImageFormat,
+ expected_pil_format: str,
+ expected_magic_bytes: bytes,
+ ):
+ """The output bytes reopen as the requested format and start with its magic bytes."""
+ pil_image = Image.new("RGB", (4, 4), color=(10, 20, 30))
+
+ image_bytes = pil_image_to_bytes(pil_image, image_format=image_format)
+
+ assert isinstance(image_bytes, bytes)
+ assert image_bytes.startswith(expected_magic_bytes)
+ reopened = Image.open(io.BytesIO(image_bytes))
+ assert reopened.format == expected_pil_format
+ assert reopened.size == (4, 4)
+
+ def test_none_format_defaults_to_png(self):
+ """Passing image_format=None encodes as PNG."""
+ pil_image = Image.new("RGB", (4, 4), color=(200, 100, 50))
+
+ image_bytes = pil_image_to_bytes(pil_image, image_format=None)
+
+ assert image_bytes.startswith(b"\x89PNG")
+ reopened = Image.open(io.BytesIO(image_bytes))
+ assert reopened.format == "PNG"
+
+ def test_png_round_trip_preserves_pixels(self):
+ """PNG is lossless: pixel values survive the encode/decode round trip."""
+ pil_image = Image.new("RGB", (4, 4), color=(123, 45, 67))
+ pil_image.putpixel((0, 0), (255, 0, 0))
+ pil_image.putpixel((3, 3), (0, 0, 255))
+
+ image_bytes = pil_image_to_bytes(pil_image, image_format=ImageFormat.PNG)
+
+ reopened = Image.open(io.BytesIO(image_bytes))
+ assert reopened.getpixel((0, 0)) == (255, 0, 0)
+ assert reopened.getpixel((3, 3)) == (0, 0, 255)
+ assert reopened.getpixel((1, 1)) == (123, 45, 67)
+ assert reopened.getpixel((2, 2)) == (123, 45, 67)
diff --git a/tests/unit/pipelex/tools/misc/test_toml_sync_nested_access.py b/tests/unit/pipelex/tools/misc/test_toml_sync_nested_access.py
new file mode 100644
index 000000000..aab89c1f6
--- /dev/null
+++ b/tests/unit/pipelex/tools/misc/test_toml_sync_nested_access.py
@@ -0,0 +1,89 @@
+from typing import Any
+
+import pytest
+import tomlkit
+from tomlkit import TOMLDocument
+
+from pipelex.tools.misc.toml_sync import collect_leaf_key_paths, get_nested_value
+
+NESTED_TOML_CONTENT = """title = "hello"
+
+[section]
+key = 42
+
+[section.sub]
+deep = "x"
+"""
+
+
+def make_doc(doc_kind: str) -> TOMLDocument | dict[str, Any]:
+ """Build the same nested structure as a tomlkit document or a plain dict."""
+ if doc_kind == "tomlkit":
+ return tomlkit.parse(NESTED_TOML_CONTENT)
+ return {"title": "hello", "section": {"key": 42, "sub": {"deep": "x"}}}
+
+
+class TestTomlSyncNestedAccess:
+ @pytest.mark.parametrize("doc_kind", ["tomlkit", "plain_dict"])
+ @pytest.mark.parametrize(
+ ("key_path", "expected_value"),
+ [
+ ("title", "hello"),
+ ("section.key", 42),
+ ("section.sub.deep", "x"),
+ ],
+ )
+ def test_get_nested_value_found(self, doc_kind: str, key_path: str, expected_value: Any) -> None:
+ """Existing top-level, 2-deep and 3-deep dotted paths are found with the right value."""
+ doc = make_doc(doc_kind)
+ found, value = get_nested_value(doc, key_path)
+ assert found is True
+ assert value == expected_value
+
+ @pytest.mark.parametrize("doc_kind", ["tomlkit", "plain_dict"])
+ @pytest.mark.parametrize(
+ "key_path",
+ [
+ "missing",
+ "section.missing",
+ "absent_table.key",
+ "title.sub",
+ "section.key.sub",
+ ],
+ )
+ def test_get_nested_value_not_found(self, doc_kind: str, key_path: str) -> None:
+ """Missing leaves, missing intermediate tables and traversal through scalars all yield (False, None)."""
+ doc = make_doc(doc_kind)
+ found, value = get_nested_value(doc, key_path)
+ assert found is False
+ assert value is None
+
+ def test_collect_leaf_key_paths_flat_doc(self) -> None:
+ """A flat document yields its top-level keys as leaf paths."""
+ doc = tomlkit.parse('alpha = 1\nbeta = "two"\ngamma = true\n')
+ assert collect_leaf_key_paths(doc) == ["alpha", "beta", "gamma"]
+
+ def test_collect_leaf_key_paths_nested_tables(self) -> None:
+ """Nested [section.sub] tables produce dotted leaf paths."""
+ doc = tomlkit.parse(NESTED_TOML_CONTENT)
+ assert collect_leaf_key_paths(doc) == ["title", "section.key", "section.sub.deep"]
+
+ def test_collect_leaf_key_paths_empty_doc(self) -> None:
+ """An empty document has no leaf paths."""
+ doc = tomlkit.parse("")
+ assert collect_leaf_key_paths(doc) == []
+
+ def test_collect_leaf_key_paths_array_is_single_leaf(self) -> None:
+ """A plain array value is a single leaf, not recursed into."""
+ doc = tomlkit.parse("nums = [1, 2, 3]\n")
+ assert collect_leaf_key_paths(doc) == ["nums"]
+
+ def test_collect_leaf_key_paths_aot_is_single_leaf(self) -> None:
+ """An [[array-of-tables]] is not a dict so the whole array is one leaf (synced wholesale)."""
+ doc = tomlkit.parse('[[servers]]\nname = "one"\n\n[[servers]]\nname = "two"\n')
+ assert collect_leaf_key_paths(doc) == ["servers"]
+
+ def test_collect_leaf_key_paths_inline_table_recurses(self) -> None:
+ """An inline table is a dict subclass so its keys become dotted leaf paths."""
+ doc = tomlkit.parse('inline = {alpha = 1, beta = "two"}\n')
+ assert collect_leaf_key_paths(doc) == ["inline.alpha", "inline.beta"]
diff --git a/tests/unit/pipelex/tools/misc/test_toml_sync_set_value.py b/tests/unit/pipelex/tools/misc/test_toml_sync_set_value.py
new file mode 100644
index 000000000..e3ed165b7
--- /dev/null
+++ b/tests/unit/pipelex/tools/misc/test_toml_sync_set_value.py
@@ -0,0 +1,74 @@
+from typing import Any
+
+import pytest
+import tomlkit
+from tomlkit import TOMLDocument
+
+from pipelex.tools.misc.toml_sync import set_nested_value
+
+
+def dump_doc(doc: TOMLDocument) -> str:
+ """Serialize a tomlkit document back to TOML text."""
+ return tomlkit.dumps(doc) # pyright: ignore[reportUnknownMemberType]
+
+
+NESTED_TOML_CONTENT = """title = "hello"
+
+[section]
+key = 42
+
+[section.sub]
+deep = "x"
+"""
+
+
+class TestTomlSyncSetValue:
+ @pytest.mark.parametrize(
+ ("key_path", "new_value"),
+ [
+ ("title", "world"),
+ ("section.key", 99),
+ ("section.sub.deep", "y"),
+ ],
+ )
+ def test_set_existing_key(self, key_path: str, new_value: str | int) -> None:
+ """Setting an existing top-level or nested key returns True and changes the value in the doc."""
+ doc = tomlkit.parse(NESTED_TOML_CONTENT)
+ result = set_nested_value(doc, key_path, new_value)
+ assert result is True
+ current: Any = doc
+ for key in key_path.split("."):
+ current = current[key]
+ assert current == new_value
+
+ @pytest.mark.parametrize(
+ "key_path",
+ [
+ "brand_new",
+ "section.brand_new",
+ "absent_table.key",
+ "title.sub",
+ "title.sub.deeper",
+ "section.key.sub",
+ ],
+ )
+ def test_missing_key_returns_false_and_doc_untouched(self, key_path: str) -> None:
+ """Missing final key, missing intermediate table or traversal through a scalar returns False and never creates the key."""
+ doc = tomlkit.parse(NESTED_TOML_CONTENT)
+ result = set_nested_value(doc, key_path, "intruder")
+ assert result is False
+ assert dump_doc(doc) == NESTED_TOML_CONTENT
+
+ def test_set_value_preserves_inline_comment(self) -> None:
+ """Replacing a value keeps the existing inline comment via trivia restoration."""
+ doc = tomlkit.parse("timeout = 30 # seconds\n")
+ result = set_nested_value(doc, "timeout", 60)
+ assert result is True
+ assert dump_doc(doc) == "timeout = 60 # seconds\n"
+
+ def test_set_value_preserves_inline_comment_on_nested_key(self) -> None:
+ """Trivia restoration also applies to keys inside a [section] table."""
+ doc = tomlkit.parse('[section]\nname = "old" # keep me\n')
+ result = set_nested_value(doc, "section.name", "new")
+ assert result is True
+ assert dump_doc(doc) == '[section]\nname = "new" # keep me\n'
diff --git a/tests/unit/pipelex/tools/misc/test_toml_sync_values.py b/tests/unit/pipelex/tools/misc/test_toml_sync_values.py
new file mode 100644
index 000000000..ebe5e9ebc
--- /dev/null
+++ b/tests/unit/pipelex/tools/misc/test_toml_sync_values.py
@@ -0,0 +1,184 @@
+from pathlib import Path
+
+import pytest
+
+from pipelex.tools.misc.toml_sync import sync_toml_values
+
+TARGET_CONTENT = """# Pipelex target config
+# header comments must survive sync
+
+title = "old-title"
+timeout = 30 # seconds
+
+[section]
+key = 1
+
+[section.sub]
+deep = "old" # nested comment
+"""
+
+SOURCE_CONTENT = """title = "new-title"
+timeout = 30
+
+[section]
+key = 1
+
+[section.sub]
+deep = "new"
+"""
+
+SYNCED_TARGET_CONTENT = """# Pipelex target config
+# header comments must survive sync
+
+title = "new-title"
+timeout = 30 # seconds
+
+[section]
+key = 1
+
+[section.sub]
+deep = "new" # nested comment
+"""
+
+
+def write_pair(tmp_path: Path, source_content: str, target_content: str) -> tuple[Path, Path]:
+ """Write source and target TOML files into tmp_path and return their paths."""
+ source_path = tmp_path / "source.toml"
+ target_path = tmp_path / "target.toml"
+ source_path.write_text(source_content)
+ target_path.write_text(target_content)
+ return source_path, target_path
+
+
+class TestSyncTomlValues:
+ def test_happy_path_updates_values_and_preserves_comments(self, tmp_path: Path) -> None:
+ """Differing values are updated in place while header comments, inline comments and blank lines all survive."""
+ source_path, target_path = write_pair(tmp_path, SOURCE_CONTENT, TARGET_CONTENT)
+
+ result = sync_toml_values(source_path, target_path)
+
+ assert target_path.read_text() == SYNCED_TARGET_CONTENT
+ assert result.updated_keys == ["title", "section.sub.deep"]
+ assert result.unchanged_keys == ["timeout", "section.key"]
+ assert result.updated_count == 2
+ assert result.unchanged_count == 2
+ assert [(change.key_path, change.old_value, change.new_value) for change in result.changes] == [
+ ("title", "old-title", "new-title"),
+ ("section.sub.deep", "old", "new"),
+ ]
+
+ def test_key_only_in_source_is_never_added_to_target(self, tmp_path: Path) -> None:
+ """A key present in source but absent from target must not be created in the target."""
+ source_path, target_path = write_pair(
+ tmp_path,
+ 'shared = "from-source"\nsource_only = "intruder"\n',
+ 'shared = "from-target"\n',
+ )
+
+ result = sync_toml_values(source_path, target_path)
+
+ target_text = target_path.read_text()
+ assert "source_only" not in target_text
+ assert target_text == 'shared = "from-source"\n'
+ assert result.updated_keys == ["shared"]
+
+ def test_key_only_in_target_stays_untouched_and_unchanged(self, tmp_path: Path) -> None:
+ """A key present in target but absent from source keeps its value and lands in unchanged_keys."""
+ source_path, target_path = write_pair(
+ tmp_path,
+ 'shared = "same"\n',
+ 'shared = "same"\ntarget_only = "precious" # do not lose\n',
+ )
+
+ result = sync_toml_values(source_path, target_path)
+
+ assert target_path.read_text() == 'shared = "same"\ntarget_only = "precious" # do not lose\n'
+ assert result.updated_keys == []
+ assert sorted(result.unchanged_keys) == ["shared", "target_only"]
+ assert result.changes == []
+
+ def test_dry_run_reports_changes_without_writing(self, tmp_path: Path) -> None:
+ """With dry_run=True the result lists the would-be changes but the target bytes are untouched."""
+ source_path, target_path = write_pair(tmp_path, SOURCE_CONTENT, TARGET_CONTENT)
+ bytes_before = target_path.read_bytes()
+
+ result = sync_toml_values(source_path, target_path, dry_run=True)
+
+ assert target_path.read_bytes() == bytes_before
+ assert result.updated_keys == ["title", "section.sub.deep"]
+ assert [(change.key_path, change.old_value, change.new_value) for change in result.changes] == [
+ ("title", "old-title", "new-title"),
+ ("section.sub.deep", "old", "new"),
+ ]
+
+ def test_no_changes_means_no_rewrite(self, tmp_path: Path) -> None:
+ """When every value already matches, the target file is not rewritten so formatting quirks survive byte-for-byte."""
+ quirky_target = 'answer = 42 # extra spacing\n\n\n[section]\nkey="tight"'
+ source_path, target_path = write_pair(tmp_path, 'answer = 42\n\n[section]\nkey = "tight"\n', quirky_target)
+ bytes_before = target_path.read_bytes()
+
+ result = sync_toml_values(source_path, target_path)
+
+ assert target_path.read_bytes() == bytes_before
+ assert result.updated_keys == []
+ assert sorted(result.unchanged_keys) == ["answer", "section.key"]
+
+ def test_sync_is_idempotent(self, tmp_path: Path) -> None:
+ """A second sync after a first one that applied changes finds nothing to update and leaves the bytes identical."""
+ source_path, target_path = write_pair(tmp_path, SOURCE_CONTENT, TARGET_CONTENT)
+
+ first_result = sync_toml_values(source_path, target_path)
+ bytes_after_first = target_path.read_bytes()
+
+ second_result = sync_toml_values(source_path, target_path)
+
+ assert first_result.updated_keys == ["title", "section.sub.deep"]
+ assert second_result.updated_keys == []
+ assert second_result.changes == []
+ assert second_result.unchanged_keys == ["title", "timeout", "section.key", "section.sub.deep"]
+ assert target_path.read_bytes() == bytes_after_first
+
+ def test_nested_section_and_type_changing_sync(self, tmp_path: Path) -> None:
+ """A nested [section.sub] key syncs via its dotted path and an int-to-string type change applies while keeping the inline comment."""
+ source_path, target_path = write_pair(
+ tmp_path,
+ 'count = "thirty"\n\n[outer.inner]\nflag = true\n',
+ "count = 30 # inline note\n\n[outer.inner]\nflag = false\n",
+ )
+
+ result = sync_toml_values(source_path, target_path)
+
+ assert target_path.read_text() == 'count = "thirty" # inline note\n\n[outer.inner]\nflag = true\n'
+ assert result.updated_keys == ["count", "outer.inner.flag"]
+ assert [(change.key_path, change.old_value, change.new_value) for change in result.changes] == [
+ ("count", 30, "thirty"),
+ ("outer.inner.flag", False, True),
+ ]
+
+ def test_array_of_tables_synced_wholesale(self, tmp_path: Path) -> None:
+ """An [[array-of-tables]] is one leaf, so a differing array is replaced wholesale by the source's."""
+ source_path, target_path = write_pair(
+ tmp_path,
+ '[[servers]]\nname = "alpha"\nport = 9000\n',
+ '[[servers]]\nname = "beta"\nport = 8000\n\n[[servers]]\nname = "gamma"\nport = 8001\n',
+ )
+
+ result = sync_toml_values(source_path, target_path)
+
+ assert target_path.read_text() == '[[servers]]\nname = "alpha"\nport = 9000\n'
+ assert result.updated_keys == ["servers"]
+
+ @pytest.mark.parametrize("missing_file", ["source", "target"])
+ def test_missing_file_raises_file_not_found(self, tmp_path: Path, missing_file: str) -> None:
+ """There is no existence guard: a missing source or target raises a plain FileNotFoundError."""
+ existing_path = tmp_path / "existing.toml"
+ existing_path.write_text('key = "value"\n')
+ absent_path = tmp_path / "absent.toml"
+
+ if missing_file == "source":
+ source_path, target_path = absent_path, existing_path
+ else:
+ source_path, target_path = existing_path, absent_path
+
+ with pytest.raises(FileNotFoundError):
+ sync_toml_values(source_path, target_path)
diff --git a/tests/unit/pipelex/tools/storage/test_storage_gcp_config.py b/tests/unit/pipelex/tools/storage/test_storage_gcp_config.py
new file mode 100644
index 000000000..948f11bb9
--- /dev/null
+++ b/tests/unit/pipelex/tools/storage/test_storage_gcp_config.py
@@ -0,0 +1,96 @@
+from typing import Literal
+
+import pytest
+
+from pipelex.tools.storage.exceptions import StorageConfigError
+from pipelex.tools.storage.storage_config import StorageGcpConfig
+from pipelex.urls import URLs
+
+
+def make_gcp_config(
+ uri_format: str = "gcp-assets/{hash}",
+ bucket_name: str = "my-bucket",
+ project_id: str = "my-project",
+ signed_urls_lifespan_seconds: int | Literal["disabled"] = 3600,
+) -> StorageGcpConfig:
+ return StorageGcpConfig(
+ uri_format=uri_format,
+ bucket_name=bucket_name,
+ project_id=project_id,
+ signed_urls_lifespan_seconds=signed_urls_lifespan_seconds,
+ )
+
+
+class TestStorageGcpConfig:
+ def test_lazy_validate_valid_config_passes_silently(self):
+ """A fully valid GCP config must pass lazy_validate without raising."""
+ config = make_gcp_config()
+ config.lazy_validate()
+
+ @pytest.mark.parametrize(
+ ("uri_format", "bucket_name", "project_id", "expected_fragment"),
+ [
+ pytest.param("", "my-bucket", "my-project", "- set a value for uri_format", id="empty-uri-format"),
+ pytest.param("assets/no-placeholder/file", "my-bucket", "my-project", "- uri_format must contain a {hash} placeholder", id="uri-no-hash"),
+ pytest.param("assets/myhash/file", "my-bucket", "my-project", "- uri_format must contain a {hash} placeholder", id="uri-bare-hash"),
+ pytest.param("gcp-assets/{hash}", "", "my-project", "- set a value for bucket_name", id="empty-bucket-name"),
+ pytest.param("gcp-assets/{hash}", "my.bucket", "my-project", "- bucket_name cannot contain a dot", id="bucket-with-dot"),
+ pytest.param("gcp-assets/{hash}", "my/bucket", "my-project", "- bucket_name cannot contain a slash", id="bucket-with-slash"),
+ pytest.param("gcp-assets/{hash}", "my-bucket", "", "- set a value for project_id", id="empty-project-id"),
+ ],
+ )
+ def test_lazy_validate_single_fault(
+ self,
+ uri_format: str,
+ bucket_name: str,
+ project_id: str,
+ expected_fragment: str,
+ ):
+ """Each individual config fault must raise a StorageConfigError naming the fault and pointing to the docs.
+
+ The bare-hash row pins that a 'hash' substring without braces is rejected: only the literal '{hash}'
+ placeholder gives uri_format.format() a substitution slot, matching the S3 validation.
+ """
+ config = make_gcp_config(uri_format=uri_format, bucket_name=bucket_name, project_id=project_id)
+ with pytest.raises(StorageConfigError) as exc_info:
+ config.lazy_validate()
+ message = str(exc_info.value)
+ assert "You have enabled storage on GCP" in message
+ assert expected_fragment in message
+ assert URLs.documentation in message
+
+ def test_lazy_validate_multiple_faults_raise_one_error_listing_all(self):
+ """All faults must be aggregated into a single StorageConfigError that lists every fix line and the docs URL."""
+ config = make_gcp_config(uri_format="", bucket_name="", project_id="")
+ with pytest.raises(StorageConfigError) as exc_info:
+ config.lazy_validate()
+ message = str(exc_info.value)
+ assert "You have enabled storage on GCP" in message
+ assert "- set a value for uri_format" in message
+ assert "- set a value for bucket_name" in message
+ assert "- set a value for project_id" in message
+ assert URLs.documentation in message
+
+ @pytest.mark.parametrize(
+ ("lifespan_setting", "expected_lifespan"),
+ [
+ pytest.param(3600, 3600, id="int-passthrough"),
+ pytest.param("disabled", None, id="disabled-maps-to-none"),
+ ],
+ )
+ def test_signed_urls_lifespan(
+ self,
+ lifespan_setting: int | Literal["disabled"],
+ expected_lifespan: int | None,
+ ):
+ """signed_urls_lifespan passes integers through and maps the 'disabled' literal to None."""
+ config = make_gcp_config(signed_urls_lifespan_seconds=lifespan_setting)
+ assert config.signed_urls_lifespan == expected_lifespan
+
+ @pytest.mark.parametrize("lifespan_setting", [pytest.param(0, id="zero"), pytest.param(-300, id="negative")])
+ def test_lazy_validate_rejects_non_positive_signed_urls_lifespan(self, lifespan_setting: int):
+ """A zero or negative lifespan produces immediately-dead signed URLs, so lazy_validate must reject it."""
+ config = make_gcp_config(signed_urls_lifespan_seconds=lifespan_setting)
+ with pytest.raises(StorageConfigError) as exc_info:
+ config.lazy_validate()
+ assert "- signed_urls_lifespan_seconds must be a positive number of seconds, or 'disabled'" in str(exc_info.value)
diff --git a/tests/unit/pipelex/tools/storage/test_storage_in_memory_config.py b/tests/unit/pipelex/tools/storage/test_storage_in_memory_config.py
new file mode 100644
index 000000000..6f9925116
--- /dev/null
+++ b/tests/unit/pipelex/tools/storage/test_storage_in_memory_config.py
@@ -0,0 +1,47 @@
+import pytest
+
+from pipelex.tools.storage.exceptions import StorageConfigError
+from pipelex.tools.storage.storage_config import StorageInMemoryConfig
+from pipelex.urls import URLs
+
+
+class TestStorageInMemoryConfig:
+ def test_lazy_validate_valid_config_passes_silently(self):
+ """A valid in-memory config must pass lazy_validate without raising."""
+ config = StorageInMemoryConfig(uri_format="memory-assets/{hash}")
+ config.lazy_validate()
+
+ @pytest.mark.parametrize(
+ ("uri_format", "expected_fragment"),
+ [
+ pytest.param("", "- set a value for uri_format", id="empty-uri-format"),
+ pytest.param("assets/no-placeholder/file", "- uri_format must contain a {hash} placeholder", id="uri-no-hash"),
+ pytest.param("assets/myhash/file", "- uri_format must contain a {hash} placeholder", id="uri-bare-hash"),
+ pytest.param("assets/{{hash}}/file", "- uri_format must contain a {hash} placeholder", id="uri-escaped-hash"),
+ pytest.param("assets/{hash/file", "- uri_format is not a valid format string", id="uri-malformed-format"),
+ pytest.param("assets/{hash}/{tenant}", "- uri_format placeholder '{tenant}' is not supported", id="uri-unsupported-placeholder"),
+ pytest.param("assets/{}/{hash}", "- uri_format placeholder '{}' is not supported", id="uri-positional-placeholder"),
+ pytest.param("assets/{hash:.0}", "- the {hash} placeholder must be plain", id="uri-hash-format-spec"),
+ pytest.param("assets/{hash[0]}", "- the {hash} placeholder must be plain", id="uri-hash-index-access"),
+ # A format spec, conversion, or attribute/index access on any supported field is rejected uniformly
+ # as "must be plain" (a str path component carrying one of these renders a constant/colliding key).
+ pytest.param("assets/{primary_id!x}/{hash}", "- uri_format placeholder '{primary_id}' must be plain", id="uri-conversion-on-field"),
+ pytest.param(
+ "assets/{primary_id.foo}/{hash}", "- uri_format placeholder '{primary_id.foo}' must be plain", id="uri-attr-access-on-field"
+ ),
+ pytest.param("assets/{extension[0]}/{hash}", "- uri_format placeholder '{extension[0]}' must be plain", id="uri-index-access-on-field"),
+ ],
+ )
+ def test_lazy_validate_single_fault(
+ self,
+ uri_format: str,
+ expected_fragment: str,
+ ):
+ """Each uri_format fault must raise a StorageConfigError naming the fault and pointing to the docs."""
+ config = StorageInMemoryConfig(uri_format=uri_format)
+ with pytest.raises(StorageConfigError) as exc_info:
+ config.lazy_validate()
+ message = str(exc_info.value)
+ assert "You have enabled storage on in_memory" in message
+ assert expected_fragment in message
+ assert URLs.documentation in message
diff --git a/tests/unit/pipelex/tools/storage/test_storage_local_config.py b/tests/unit/pipelex/tools/storage/test_storage_local_config.py
new file mode 100644
index 000000000..91a6097f2
--- /dev/null
+++ b/tests/unit/pipelex/tools/storage/test_storage_local_config.py
@@ -0,0 +1,54 @@
+import pytest
+
+from pipelex.tools.storage.exceptions import StorageConfigError
+from pipelex.tools.storage.storage_config import StorageLocalConfig
+from pipelex.urls import URLs
+
+
+def make_local_config(
+ uri_format: str = "local-assets/{hash}",
+ local_storage_path: str = ".pipelex/storage",
+) -> StorageLocalConfig:
+ return StorageLocalConfig(uri_format=uri_format, local_storage_path=local_storage_path)
+
+
+class TestStorageLocalConfig:
+ def test_lazy_validate_valid_config_passes_silently(self):
+ """A fully valid local config must pass lazy_validate without raising."""
+ config = make_local_config()
+ config.lazy_validate()
+
+ @pytest.mark.parametrize(
+ ("uri_format", "local_storage_path", "expected_fragment"),
+ [
+ pytest.param("", ".pipelex/storage", "- set a value for uri_format", id="empty-uri-format"),
+ pytest.param("assets/no-placeholder/file", ".pipelex/storage", "- uri_format must contain a {hash} placeholder", id="uri-no-hash"),
+ pytest.param("assets/myhash/file", ".pipelex/storage", "- uri_format must contain a {hash} placeholder", id="uri-bare-hash"),
+ pytest.param("local-assets/{hash}", "", "- set a value for local_storage_path", id="empty-storage-path"),
+ ],
+ )
+ def test_lazy_validate_single_fault(
+ self,
+ uri_format: str,
+ local_storage_path: str,
+ expected_fragment: str,
+ ):
+ """Each individual config fault must raise a StorageConfigError naming the fault and pointing to the docs."""
+ config = make_local_config(uri_format=uri_format, local_storage_path=local_storage_path)
+ with pytest.raises(StorageConfigError) as exc_info:
+ config.lazy_validate()
+ message = str(exc_info.value)
+ assert "You have enabled storage on local" in message
+ assert expected_fragment in message
+ assert URLs.documentation in message
+
+ def test_lazy_validate_multiple_faults_raise_one_error_listing_all(self):
+ """All faults must be aggregated into a single StorageConfigError that lists every fix line and the docs URL."""
+ config = make_local_config(uri_format="", local_storage_path="")
+ with pytest.raises(StorageConfigError) as exc_info:
+ config.lazy_validate()
+ message = str(exc_info.value)
+ assert "You have enabled storage on local" in message
+ assert "- set a value for uri_format" in message
+ assert "- set a value for local_storage_path" in message
+ assert URLs.documentation in message
diff --git a/tests/unit/pipelex/tools/storage/test_storage_provider_config.py b/tests/unit/pipelex/tools/storage/test_storage_provider_config.py
new file mode 100644
index 000000000..37fda6344
--- /dev/null
+++ b/tests/unit/pipelex/tools/storage/test_storage_provider_config.py
@@ -0,0 +1,184 @@
+from typing import Any, Callable
+
+import pytest
+from pydantic import BaseModel, ValidationError
+
+from pipelex.tools.storage.exceptions import StorageConfigError
+from pipelex.tools.storage.storage_config import (
+ StorageConfig,
+ StorageGcpConfig,
+ StorageInMemoryConfig,
+ StorageLocalConfig,
+ StorageMethod,
+ StorageProviderConfig,
+ StorageS3Config,
+)
+
+LOCAL_URI_FORMAT = "local-assets/{hash}"
+IN_MEMORY_URI_FORMAT = "memory-assets/{hash}"
+S3_URI_FORMAT = "s3-assets/{hash}"
+GCP_URI_FORMAT = "gcp-assets/{hash}"
+LOCAL_STORAGE_PATH = "/opt/pipelex-test-storage"
+
+
+def make_local_config() -> StorageLocalConfig:
+ return StorageLocalConfig(uri_format=LOCAL_URI_FORMAT, local_storage_path=LOCAL_STORAGE_PATH)
+
+
+def make_in_memory_config() -> StorageInMemoryConfig:
+ return StorageInMemoryConfig(uri_format=IN_MEMORY_URI_FORMAT)
+
+
+def make_s3_config() -> StorageS3Config:
+ return StorageS3Config(
+ uri_format=S3_URI_FORMAT,
+ bucket_name="my-bucket",
+ region="eu-west-1",
+ signed_urls_lifespan_seconds=3600,
+ )
+
+
+def make_gcp_config() -> StorageGcpConfig:
+ return StorageGcpConfig(
+ uri_format=GCP_URI_FORMAT,
+ bucket_name="my-bucket",
+ project_id="my-project",
+ signed_urls_lifespan_seconds=3600,
+ )
+
+
+class TestStorageProviderConfig:
+ @pytest.mark.parametrize(
+ ("method", "field_name", "sub_config_factory"),
+ [
+ pytest.param(StorageMethod.LOCAL, "local", make_local_config, id="local"),
+ pytest.param(StorageMethod.IN_MEMORY, "in_memory", make_in_memory_config, id="in-memory"),
+ pytest.param(StorageMethod.S3, "s3", make_s3_config, id="s3"),
+ pytest.param(StorageMethod.GCP, "gcp", make_gcp_config, id="gcp"),
+ ],
+ )
+ def test_validator_accepts_method_with_matching_sub_config(
+ self,
+ method: StorageMethod,
+ field_name: str,
+ sub_config_factory: Callable[[], BaseModel],
+ ):
+ """Each storage method constructs fine when its matching sub-config is provided."""
+ kwargs: dict[str, Any] = {"method": method, field_name: sub_config_factory()}
+ provider_config = StorageProviderConfig(**kwargs)
+ assert provider_config.method == method
+ assert getattr(provider_config, field_name) is kwargs[field_name]
+
+ @pytest.mark.parametrize(
+ ("method", "expected_message"),
+ [
+ pytest.param(StorageMethod.LOCAL, "local config is required when method is local", id="local"),
+ pytest.param(StorageMethod.IN_MEMORY, "in_memory config is required when method is in_memory", id="in-memory"),
+ pytest.param(StorageMethod.S3, "s3 config is required when method is s3", id="s3"),
+ pytest.param(StorageMethod.GCP, "gcp config is required when method is gcp", id="gcp"),
+ ],
+ )
+ def test_validator_rejects_method_without_matching_sub_config(
+ self,
+ method: StorageMethod,
+ expected_message: str,
+ ):
+ """Construction raises a StorageConfigError (unwrapped by pydantic) naming the required section."""
+ with pytest.raises(StorageConfigError) as exc_info:
+ StorageProviderConfig(method=method)
+ assert expected_message in str(exc_info.value)
+
+ def test_method_string_coercion_despite_strict_mode(self):
+ """The method field is Field(strict=False), so a plain string coerces to StorageMethod under strict config."""
+ provider_config = StorageProviderConfig.model_validate({"method": "local", "local": make_local_config()})
+ assert isinstance(provider_config.method, StorageMethod)
+ assert provider_config.method == StorageMethod.LOCAL
+
+ @pytest.mark.parametrize(
+ ("method", "field_name", "sub_config_factory", "expected_uri_format"),
+ [
+ pytest.param(StorageMethod.LOCAL, "local", make_local_config, LOCAL_URI_FORMAT, id="local"),
+ pytest.param(StorageMethod.IN_MEMORY, "in_memory", make_in_memory_config, IN_MEMORY_URI_FORMAT, id="in-memory"),
+ pytest.param(StorageMethod.S3, "s3", make_s3_config, S3_URI_FORMAT, id="s3"),
+ pytest.param(StorageMethod.GCP, "gcp", make_gcp_config, GCP_URI_FORMAT, id="gcp"),
+ ],
+ )
+ def test_uri_format_returns_matching_sub_config_format(
+ self,
+ method: StorageMethod,
+ field_name: str,
+ sub_config_factory: Callable[[], BaseModel],
+ expected_uri_format: str,
+ ):
+ """The uri_format property dispatches on method and returns the matching sub-config's uri_format."""
+ kwargs: dict[str, Any] = {"method": method, field_name: sub_config_factory()}
+ provider_config = StorageProviderConfig(**kwargs)
+ assert provider_config.uri_format == expected_uri_format
+
+ @pytest.mark.parametrize(
+ ("method", "expected_message"),
+ [
+ pytest.param(StorageMethod.LOCAL, "local config is required to access uri_format", id="local"),
+ pytest.param(StorageMethod.IN_MEMORY, "in_memory config is required to access uri_format", id="in-memory"),
+ pytest.param(StorageMethod.S3, "s3 config is required to access uri_format", id="s3"),
+ pytest.param(StorageMethod.GCP, "gcp config is required to access uri_format", id="gcp"),
+ ],
+ )
+ def test_uri_format_raises_when_matching_sub_config_is_none(
+ self,
+ method: StorageMethod,
+ expected_message: str,
+ ):
+ """The uri_format error branches are normally unreachable (the model validator guards construction),
+ so we build via model_construct to bypass validation and pin each per-method error message.
+ """
+ provider_config = StorageProviderConfig.model_construct(method=method)
+ with pytest.raises(StorageConfigError) as exc_info:
+ _ = provider_config.uri_format
+ assert expected_message in str(exc_info.value)
+
+ def test_storage_path_returns_local_storage_path(self):
+ """storage_path returns the local sub-config's local_storage_path."""
+ provider_config = StorageProviderConfig(method=StorageMethod.LOCAL, local=make_local_config())
+ assert provider_config.storage_path == LOCAL_STORAGE_PATH
+
+ def test_storage_path_raises_for_non_local_method_even_with_local_config(self):
+ """storage_path dispatches on method: a non-local method raises even when a local sub-config is set."""
+ provider_config = StorageProviderConfig(
+ method=StorageMethod.S3,
+ s3=make_s3_config(),
+ local=make_local_config(),
+ )
+ with pytest.raises(StorageConfigError) as exc_info:
+ _ = provider_config.storage_path
+ assert "storage_path is only available when method is local" in str(exc_info.value)
+
+ def test_storage_path_raises_when_local_config_is_none(self):
+ """The missing-local-config branch is normally unreachable (the model validator guards construction),
+ so we build via model_construct to bypass validation and pin its error message.
+ """
+ provider_config = StorageProviderConfig.model_construct(method=StorageMethod.LOCAL)
+ with pytest.raises(StorageConfigError) as exc_info:
+ _ = provider_config.storage_path
+ assert "local config is required to access storage_path" in str(exc_info.value)
+
+ def test_storage_config_boolean_flags_are_required(self):
+ """StorageConfig requires both boolean flags: omitting them fails validation with 'missing' errors."""
+ with pytest.raises(ValidationError) as exc_info:
+ StorageConfig.model_validate({"method": "in_memory", "in_memory": make_in_memory_config()})
+ missing_fields = {error["loc"][0] for error in exc_info.value.errors() if error["type"] == "missing"}
+ assert missing_fields == {"is_fetch_remote_content_enabled", "is_upload_local_content_enabled"}
+
+ def test_storage_config_boolean_flags_round_trip_through_model_dump(self):
+ """The boolean flags survive a model_dump round-trip with their exact values."""
+ storage_config = StorageConfig(
+ method=StorageMethod.IN_MEMORY,
+ in_memory=make_in_memory_config(),
+ is_fetch_remote_content_enabled=True,
+ is_upload_local_content_enabled=False,
+ )
+ dumped = storage_config.model_dump()
+ assert dumped["is_fetch_remote_content_enabled"] is True
+ assert dumped["is_upload_local_content_enabled"] is False
+ rebuilt = StorageConfig.model_validate(dumped, strict=False)
+ assert rebuilt == storage_config
diff --git a/tests/unit/pipelex/tools/storage/test_storage_provider_factory.py b/tests/unit/pipelex/tools/storage/test_storage_provider_factory.py
new file mode 100644
index 000000000..051abc73f
--- /dev/null
+++ b/tests/unit/pipelex/tools/storage/test_storage_provider_factory.py
@@ -0,0 +1,52 @@
+from pathlib import Path
+
+import pytest
+
+from pipelex.tools.storage.exceptions import StorageConfigError
+from pipelex.tools.storage.in_memory_storage_provider import InMemoryStorageProvider
+from pipelex.tools.storage.local_storage_provider import LocalStorageProvider
+from pipelex.tools.storage.storage_config import (
+ StorageInMemoryConfig,
+ StorageLocalConfig,
+ StorageMethod,
+ StorageProviderConfig,
+)
+from pipelex.tools.storage.storage_provider_factory import make_storage_provider_from_config
+
+
+class TestStorageProviderFactory:
+ def test_local_method_returns_local_provider(self, tmp_path: Path):
+ """A valid local config yields a LocalStorageProvider."""
+ provider_config = StorageProviderConfig(
+ method=StorageMethod.LOCAL,
+ local=StorageLocalConfig(uri_format="assets/{hash}", local_storage_path=str(tmp_path / "store")),
+ )
+ provider = make_storage_provider_from_config(provider_config)
+ assert isinstance(provider, LocalStorageProvider)
+
+ def test_in_memory_method_returns_in_memory_provider(self):
+ """A valid in-memory config yields an InMemoryStorageProvider."""
+ provider_config = StorageProviderConfig(
+ method=StorageMethod.IN_MEMORY,
+ in_memory=StorageInMemoryConfig(uri_format="assets/{hash}"),
+ )
+ provider = make_storage_provider_from_config(provider_config)
+ assert isinstance(provider, InMemoryStorageProvider)
+
+ def test_local_config_is_validated_before_provider_construction(self, tmp_path: Path):
+ """A local uri_format without a {hash} placeholder is rejected when the provider is built."""
+ provider_config = StorageProviderConfig(
+ method=StorageMethod.LOCAL,
+ local=StorageLocalConfig(uri_format="assets/constant", local_storage_path=str(tmp_path / "store")),
+ )
+ with pytest.raises(StorageConfigError, match="uri_format must contain a"):
+ make_storage_provider_from_config(provider_config)
+
+ def test_in_memory_config_is_validated_before_provider_construction(self):
+ """An in-memory uri_format without a {hash} placeholder is rejected when the provider is built."""
+ provider_config = StorageProviderConfig(
+ method=StorageMethod.IN_MEMORY,
+ in_memory=StorageInMemoryConfig(uri_format="assets/constant"),
+ )
+ with pytest.raises(StorageConfigError, match="uri_format must contain a"):
+ make_storage_provider_from_config(provider_config)
diff --git a/tests/unit/pipelex/tools/storage/test_storage_s3_config.py b/tests/unit/pipelex/tools/storage/test_storage_s3_config.py
new file mode 100644
index 000000000..e48bd1495
--- /dev/null
+++ b/tests/unit/pipelex/tools/storage/test_storage_s3_config.py
@@ -0,0 +1,134 @@
+from typing import Literal
+
+import pytest
+
+from pipelex.tools.storage.exceptions import StorageConfigError
+from pipelex.tools.storage.storage_config import StorageS3Config
+from pipelex.urls import URLs
+
+
+def make_s3_config(
+ uri_format: str = "s3-assets/{hash}",
+ bucket_name: str = "my-bucket",
+ region: str = "eu-west-1",
+ signed_urls_lifespan_seconds: int | Literal["disabled"] = 3600,
+) -> StorageS3Config:
+ return StorageS3Config(
+ uri_format=uri_format,
+ bucket_name=bucket_name,
+ region=region,
+ signed_urls_lifespan_seconds=signed_urls_lifespan_seconds,
+ )
+
+
+class TestStorageS3Config:
+ def test_lazy_validate_valid_config_passes_silently(self):
+ """A fully valid S3 config must pass lazy_validate without raising."""
+ config = make_s3_config()
+ config.lazy_validate()
+
+ @pytest.mark.parametrize(
+ ("uri_format", "bucket_name", "region", "expected_fragment"),
+ [
+ pytest.param("", "my-bucket", "eu-west-1", "- set a value for uri_format", id="empty-uri-format"),
+ pytest.param("assets/myhash/file", "my-bucket", "eu-west-1", "- uri_format must contain a {hash} placeholder", id="uri-no-braced-hash"),
+ pytest.param("assets/{{hash}}", "my-bucket", "eu-west-1", "- uri_format must contain a {hash} placeholder", id="uri-escaped-hash"),
+ pytest.param("assets/{hash", "my-bucket", "eu-west-1", "- uri_format is not a valid format string", id="uri-malformed-format"),
+ pytest.param(
+ "assets/{hash}/{tenant}", "my-bucket", "eu-west-1", "- uri_format placeholder '{tenant}' is not supported", id="uri-unsupported"
+ ),
+ pytest.param("assets/{hash:.0}", "my-bucket", "eu-west-1", "- the {hash} placeholder must be plain", id="uri-hash-format-spec"),
+ # A format spec on a non-hash field is just as dangerous: {primary_id:.0} truncates to "" (collisions),
+ # and a huge width like {primary_id:1000000000} would allocate a ~1GB string if it ever reached a
+ # test rendering — so it must be rejected at parse time, before any format() call.
+ pytest.param(
+ "assets/{hash}/{primary_id:.0}",
+ "my-bucket",
+ "eu-west-1",
+ "- uri_format placeholder '{primary_id}' must be plain",
+ id="uri-field-truncate-spec",
+ ),
+ pytest.param(
+ "assets/{hash}/{primary_id:1000000000}",
+ "my-bucket",
+ "eu-west-1",
+ "- uri_format placeholder '{primary_id}' must be plain",
+ id="uri-field-huge-width",
+ ),
+ pytest.param(
+ "assets/{hash}/{secondary_id!r}",
+ "my-bucket",
+ "eu-west-1",
+ "- uri_format placeholder '{secondary_id}' must be plain",
+ id="uri-field-conversion",
+ ),
+ pytest.param(
+ "assets/{hash}/{primary_id.foo}",
+ "my-bucket",
+ "eu-west-1",
+ "- uri_format placeholder '{primary_id.foo}' must be plain",
+ id="uri-field-attr-access",
+ ),
+ pytest.param(
+ "assets/{hash}/{extension[0]}",
+ "my-bucket",
+ "eu-west-1",
+ "- uri_format placeholder '{extension[0]}' must be plain",
+ id="uri-field-index-access",
+ ),
+ pytest.param("s3-assets/{hash}", "", "eu-west-1", "- set a value for bucket_name", id="empty-bucket-name"),
+ pytest.param("s3-assets/{hash}", "my.bucket", "eu-west-1", "- bucket_name cannot contain a dot", id="bucket-with-dot"),
+ pytest.param("s3-assets/{hash}", "my/bucket", "eu-west-1", "- bucket_name cannot contain a slash", id="bucket-with-slash"),
+ pytest.param("s3-assets/{hash}", "my-bucket", "", "- set a value for region", id="empty-region"),
+ ],
+ )
+ def test_lazy_validate_single_fault(
+ self,
+ uri_format: str,
+ bucket_name: str,
+ region: str,
+ expected_fragment: str,
+ ):
+ """Each individual config fault must raise a StorageConfigError naming the fault and pointing to the docs."""
+ config = make_s3_config(uri_format=uri_format, bucket_name=bucket_name, region=region)
+ with pytest.raises(StorageConfigError) as exc_info:
+ config.lazy_validate()
+ message = str(exc_info.value)
+ assert expected_fragment in message
+ assert URLs.documentation in message
+
+ def test_lazy_validate_multiple_faults_raise_one_error_listing_all(self):
+ """All faults must be aggregated into a single StorageConfigError that lists every fix line and the docs URL."""
+ config = make_s3_config(uri_format="", bucket_name="", region="")
+ with pytest.raises(StorageConfigError) as exc_info:
+ config.lazy_validate()
+ message = str(exc_info.value)
+ assert "You have enabled storage on S3" in message
+ assert "- set a value for uri_format" in message
+ assert "- set a value for bucket_name" in message
+ assert "- set a value for region" in message
+ assert URLs.documentation in message
+
+ @pytest.mark.parametrize(
+ ("lifespan_setting", "expected_lifespan"),
+ [
+ pytest.param(3600, 3600, id="int-passthrough"),
+ pytest.param("disabled", None, id="disabled-maps-to-none"),
+ ],
+ )
+ def test_signed_urls_lifespan(
+ self,
+ lifespan_setting: int | Literal["disabled"],
+ expected_lifespan: int | None,
+ ):
+ """signed_urls_lifespan passes integers through and maps the 'disabled' literal to None."""
+ config = make_s3_config(signed_urls_lifespan_seconds=lifespan_setting)
+ assert config.signed_urls_lifespan == expected_lifespan
+
+ @pytest.mark.parametrize("lifespan_setting", [pytest.param(0, id="zero"), pytest.param(-300, id="negative")])
+ def test_lazy_validate_rejects_non_positive_signed_urls_lifespan(self, lifespan_setting: int):
+ """A zero or negative lifespan produces immediately-dead signed URLs, so lazy_validate must reject it."""
+ config = make_s3_config(signed_urls_lifespan_seconds=lifespan_setting)
+ with pytest.raises(StorageConfigError) as exc_info:
+ config.lazy_validate()
+ assert "- signed_urls_lifespan_seconds must be a positive number of seconds, or 'disabled'" in str(exc_info.value)
diff --git a/wip/pydantic-before-validator-typing.md b/wip/pydantic-before-validator-typing.md
new file mode 100644
index 000000000..c1ff269db
--- /dev/null
+++ b/wip/pydantic-before-validator-typing.md
@@ -0,0 +1,57 @@
+# Typing `mode="before"` validators (Pydantic v2) — a short brief
+
+## The core fact
+
+A `@model_validator(mode="before")` (and `@field_validator(..., mode="before")`) runs **before Pydantic has coerced or validated anything**. It receives the *raw* input as handed to the model. For a model that input is usually a `dict`, but it can be anything the caller passed: a `str`, an `int`, `None`, an already-constructed instance, or — crucially — a value being trial-validated as one arm of a `Union`.
+
+So the input type is genuinely unconstrained. Pydantic's own type stubs annotate the before-validator value as `Any`.
+
+## What's pythonic for the annotation
+
+Two honest options, in order of preference:
+
+1. **`Any` in, `Any` out.** Truthful about the contract (raw, pre-validation input), and decoupled from any particular caller. The validator's job is to *normalize toward the dict shape*, not to assert it.
+2. **An explicit `dict[str, Any] | T` union** — only when the model deliberately participates in a known `T`-vs-other union and you want the two expected shapes documented at the signature. This reads as self-documenting but is *circumstantial*: the non-dict arm names a sibling type from one consumer's union, so it leaks that consumer into the model. Avoid widening it past what you actually branch on.
+
+Anti-pattern: annotating the value as `dict[str, Any]` when the validator can in fact receive a non-dict. That's a comfortable lie — it makes `values.get(...)` typecheck while the code crashes at runtime on a `str`/`None`.
+
+## The load-bearing rule: guard on shape, inside the validator
+
+```python
+@model_validator(mode="before")
+@classmethod
+def normalize(cls, data: Any) -> Any:
+ if not isinstance(data, dict):
+ return data # let a later arm / the model machinery handle it
+ # ... dict-only logic here ...
+ return data
+```
+
+The `isinstance` guard — not the annotation — is what makes the validator robust. A before-validator should never assume its input shape.
+
+## Why this matters for unions (the real trap)
+
+Pydantic converts only `ValueError` and `AssertionError` raised inside a validator into a `ValidationError`. **Any other exception** (`AttributeError`, `TypeError`, `KeyError`, …) propagates unchanged.
+
+Inside `Union` validation, Pydantic tries members and catches each member's `ValidationError` to move on to the next. A *non-`ValidationError`* exception is not caught — it aborts the whole union. So an unguarded before-validator that does `data.get(...)` on a `str` raises `AttributeError`, which escapes the union instead of cleanly rejecting that arm. The symptom looks like "this valid input doesn't match the union," but the cause is a validator that wasn't shape-safe.
+
+## Don't fix it by reordering the union
+
+Putting the simpler type first (`str | Model` instead of `Model | str`) can make the symptom vanish — Pydantic's default *smart* mode prefers an exact-type match, so a `str` input may hit the `str` arm and never invoke the model's before-validator. But this is masking at the wrong layer:
+
+- it only helps the one call site whose ordering you changed;
+- it doesn't protect `Model.model_validate(<non-dict>)` called directly, or any other union the model appears in;
+- it depends on union-resolution behavior (smart vs `left_to_right`) that's easy to perturb later.
+
+Fix the validator (guard on shape); treat union ordering as a readability choice, not a correctness mechanism.
+
+## Smart vs left-to-right, in one line
+
+Default `union_mode="smart"` picks the best-matching member regardless of declaration order (exact type matches win); `union_mode="left_to_right"` tries members top-to-bottom and takes the first that validates. Neither rescues a before-validator that raises a non-`ValidationError` on the arm it *does* try.
+
+## Takeaways
+
+- A before-validator receives `Any`; type it `Any` (or a deliberately-narrow union you actually branch on) — never a shape you merely hope for.
+- Guard with `isinstance(...)` and return non-matching input untouched.
+- Raise `ValueError`/`AssertionError` for real validation failures so Pydantic wraps them; let unexpected types fall through rather than crash.
+- Robustness belongs on the model's validator, not on the order of a consumer's `Union`.
diff --git a/wip/tests/add-tests-branch-overview.html b/wip/tests/add-tests-branch-overview.html
new file mode 100644
index 000000000..e3df974b4
--- /dev/null
+++ b/wip/tests/add-tests-branch-overview.html
@@ -0,0 +1,325 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<title>feature/Add-tests — branch overview</title>
+<script src="https://cdn.jsdelivr.net/npm/mermaid@11/dist/mermaid.min.js"></script>
+<style>
+ :root {
+ --bg: #0f1419; --panel: #1a2027; --panel2: #232b35; --ink: #e6edf3;
+ --muted: #9aa7b4; --line: #2d3742; --accent: #4cc2ff; --accent2: #7ee787;
+ --add-bg: #12261a; --add-ink: #7ee787; --del-bg: #2a1416; --del-ink: #ff9492;
+ --warn: #f0b429; --chip: #2b3744;
+ }
+ * { box-sizing: border-box; }
+ body {
+ margin: 0; background: var(--bg); color: var(--ink);
+ font: 16px/1.65 -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif;
+ }
+ .wrap { max-width: 980px; margin: 0 auto; padding: 32px 22px 96px; }
+ h1 { font-size: 30px; margin: 0 0 4px; letter-spacing: -0.5px; }
+ h2 { font-size: 22px; margin: 44px 0 14px; padding-bottom: 8px; border-bottom: 1px solid var(--line); }
+ h3 { font-size: 17px; margin: 26px 0 8px; color: var(--accent); }
+ p { margin: 10px 0; }
+ a { color: var(--accent); }
+ code { background: var(--panel2); padding: 1px 6px; border-radius: 5px; font-size: 13.5px;
+ font-family: "SF Mono", ui-monospace, Menlo, Consolas, monospace; }
+ .sub { color: var(--muted); font-size: 14px; margin-top: 0; }
+ .chips { display: flex; flex-wrap: wrap; gap: 8px; margin: 16px 0 4px; }
+ .chip { background: var(--chip); color: var(--ink); border: 1px solid var(--line);
+ padding: 3px 11px; border-radius: 999px; font-size: 12.5px; }
+ .chip b { color: var(--accent2); }
+ .tldr { background: linear-gradient(180deg, #16202b, #131a22); border: 1px solid #2c4257;
+ border-left: 4px solid var(--accent); border-radius: 10px; padding: 18px 20px; margin: 22px 0; }
+ .tldr h2 { border: 0; margin: 0 0 8px; font-size: 18px; color: var(--accent); }
+ .tldr ul { margin: 8px 0 0; padding-left: 20px; }
+ .tldr li { margin: 6px 0; }
+ .grid { display: grid; grid-template-columns: 1fr 1fr; gap: 12px; margin: 14px 0; }
+ @media (max-width: 720px) { .grid { grid-template-columns: 1fr; } }
+ .card { background: var(--panel); border: 1px solid var(--line); border-radius: 10px; padding: 14px 16px; }
+ .card .big { font-size: 26px; font-weight: 700; color: var(--accent2); }
+ .card .lbl { color: var(--muted); font-size: 13px; }
+ pre { background: #0b0f14; border: 1px solid var(--line); border-radius: 9px; padding: 14px 16px;
+ overflow-x: auto; font-size: 13px; line-height: 1.5;
+ font-family: "SF Mono", ui-monospace, Menlo, Consolas, monospace; }
+ pre.diff .add { color: var(--add-ink); background: var(--add-bg); display: block; }
+ pre.diff .del { color: var(--del-ink); background: var(--del-bg); display: block; }
+ pre.diff .ctx { color: var(--muted); display: block; }
+ pre.diff .hd { color: var(--accent); display: block; }
+ details { background: var(--panel); border: 1px solid var(--line); border-radius: 9px;
+ padding: 0 16px; margin: 12px 0; }
+ details[open] { padding-bottom: 14px; }
+ summary { cursor: pointer; padding: 13px 0; font-weight: 600; list-style: none; }
+ summary::-webkit-details-marker { display: none; }
+ summary::before { content: "▸ "; color: var(--accent); }
+ details[open] summary::before { content: "▾ "; }
+ .tabs { margin: 16px 0; }
+ .tabbar { display: flex; flex-wrap: wrap; gap: 6px; border-bottom: 1px solid var(--line); }
+ .tabbtn { background: transparent; border: 0; color: var(--muted); padding: 9px 13px;
+ cursor: pointer; font-size: 13.5px; border-bottom: 2px solid transparent; }
+ .tabbtn:hover { color: var(--ink); }
+ .tabbtn.active { color: var(--accent); border-bottom-color: var(--accent); font-weight: 600; }
+ .tabpane { display: none; padding-top: 14px; }
+ .tabpane.active { display: block; }
+ .mermaid { background: var(--panel); border: 1px solid var(--line); border-radius: 10px;
+ padding: 18px; margin: 16px 0; text-align: center; }
+ table { border-collapse: collapse; width: 100%; margin: 14px 0; font-size: 14px; }
+ th, td { border: 1px solid var(--line); padding: 8px 11px; text-align: left; vertical-align: top; }
+ th { background: var(--panel2); }
+ .ok { color: var(--accent2); } .no { color: var(--del-ink); } .mute { color: var(--muted); }
+ .note { border-left: 3px solid var(--warn); background: #211c10; padding: 10px 14px;
+ border-radius: 6px; margin: 14px 0; font-size: 14.5px; }
+ .foot { color: var(--muted); font-size: 13px; margin-top: 60px; border-top: 1px solid var(--line); padding-top: 16px; }
+ kbd { background: var(--panel2); border: 1px solid var(--line); border-bottom-width: 2px;
+ border-radius: 5px; padding: 1px 6px; font-size: 12px; font-family: ui-monospace, monospace; }
+</style>
+</head>
+<body>
+<div class="wrap">
+
+ <h1>feature/Add-tests</h1>
+ <p class="sub">Offline test-coverage grind for pipelex — plus the source bugs writing those tests surfaced. PR&nbsp;#989 → <code>dev</code>.</p>
+
+ <div class="chips">
+ <span class="chip">coverage <b>~72% → ~78%</b></span>
+ <span class="chip">source files changed <b>12</b></span>
+ <span class="chip">net new lines ~<b>10.3k</b> (almost all tests)</span>
+ <span class="chip">behavior diff <b>196 ins / 85 del</b></span>
+ </div>
+
+ <div class="tldr">
+ <h2>TL;DR</h2>
+ <ul>
+ <li>This branch is a <b>unit-test grind</b> over the offline-testable layers of pipelex (CLI internals, Temporal entry points, inference plumbing, core runtime, tools/config). The diff is <b>almost entirely new tests</b>.</li>
+ <li>Writing those tests surfaced a <b>small, deliberate set of real source bugs</b>. Each was pinned by a test first, judged wrong in review, then fixed with the test flipped. <b>That's the only behavior change in the PR.</b></li>
+ <li>The headline fix is <b>storage <code>uri_format</code> validation</b>: a misconfigured format string used to silently render the same URI for every object (storage-wide overwrites). It now fails fast at boot with an operator-facing message — and the rule is enforced uniformly on <i>every</i> placeholder.</li>
+ <li>One load-bearing infra fix: pytest's default <code>norecursedirs</code> includes <code>build</code>, so an entire <code>tests/.../build/</code> tree (including a pre-existing regression test) <b>was never being collected</b>. Now fixed in <code>pyproject.toml</code>.</li>
+ <li><b>How to review:</b> read the source changes below (they're the only behavior diff), then sample the tests with the conventions section in mind. Remaining uncovered lines are deliberate (interface layer is owned by the <code>conformance</code> repo).</li>
+ </ul>
+ </div>
+
+ <h2>1 · What's in the branch</h2>
+ <p>The grind covered five areas. Tests mirror the source tree under <code>tests/unit/pipelex/</code>. None of them touch a live provider, network, or cluster — inference, network, and Temporal SDK boundaries are mocked; real collaborators are used where cheap (real TOML in <code>tmp_path</code>, real PIL images, real pydantic SDK models, a real aiohttp server for the codec).</p>
+
+ <div class="mermaid">
+flowchart LR
+ G["feature/Add-tests<br/>coverage grind"] --> A["CLI internals<br/><i>doctor, run, build codegen,<br/>readiness, show/which, error handlers</i>"]
+ G --> B["Temporal entry points<br/><i>worker CLI, codec HTTP server,<br/>server connection</i>"]
+ G --> C["Inference plumbing<br/><i>structured-output modes, creds,<br/>model-deck, img-gen args,<br/>worker factories, gateway, mistral</i>"]
+ G --> D["Core runtime<br/><i>observer, output renderer,<br/>graph dispatch, bundle-spec,<br/>pipeline runner</i>"]
+ G --> E["Tools / config<br/><i>TOML config-sync, format enums,<br/>PIL conversion, storage validators</i>"]
+ </div>
+
+ <div class="grid">
+ <div class="card"><div class="big">21→92%</div><div class="lbl"><code>doctor_cmd.py</code> — all diagnostic checks, fix mode</div></div>
+ <div class="card"><div class="big">32→94%</div><div class="lbl"><code>run/_run_core.py</code> — resolution, output/graph/CSV saving</div></div>
+ <div class="card"><div class="big">51→99%</div><div class="lbl"><code>cli/error_handlers.py</code> — every handler + panels</div></div>
+ <div class="card"><div class="big">20→97%</div><div class="lbl"><code>cli/readiness.py</code> — venv / dev-install / gate</div></div>
+ </div>
+
+ <h2>2 · The source bugs (the behavior diff)</h2>
+ <p>Each tab is one fix. They are independent. Minimal diffs only — the surrounding code and tests are in the branch.</p>
+
+ <div class="tabs" id="bugtabs">
+ <div class="tabbar">
+ <button class="tabbtn active" data-tab="t-storage">storage uri_format ★</button>
+ <button class="tabbtn" data-tab="t-mod">img moderation</button>
+ <button class="tabbtn" data-tab="t-concept">concept string</button>
+ <button class="tabbtn" data-tab="t-observer">observer</button>
+ <button class="tabbtn" data-tab="t-mistral">mistral order</button>
+ <button class="tabbtn" data-tab="t-render">output renderer</button>
+ <button class="tabbtn" data-tab="t-pytest">pytest collect ★</button>
+ </div>
+
+ <div class="tabpane active" id="t-storage">
+ <h3>Storage <code>uri_format</code> validation — the biggest fix</h3>
+ <p><b>The bug.</b> GCP's <code>lazy_validate</code> accepted any format string containing the bare substring <code>hash</code> (e.g. <code>assets/myhash/file</code>) — no substitution slot, so every stored object renders the <i>same</i> URI: silent storage-wide overwrites. Local and in-memory configs had <b>no validation at all</b>. The check now parses the format string and requires a real, plain <code>{hash}</code> replacement field.</p>
+ <p>The per-provider checks moved onto a shared <code>StorageMethodConfig</code> base so the providers can't drift apart:</p>
+ <div class="mermaid">
+classDiagram
+ ConfigModel <|-- StorageMethodConfig
+ StorageMethodConfig <|-- StorageLocalConfig
+ StorageMethodConfig <|-- StorageInMemoryConfig
+ StorageMethodConfig <|-- StorageBucketConfig
+ StorageBucketConfig <|-- StorageS3Config
+ StorageBucketConfig <|-- StorageGcpConfig
+ class StorageMethodConfig {
+ +uri_format str
+ +URI_FORMAT_SUPPORTED_FIELDS
+ +lazy_validate()
+ +_collect_uri_format_error_msgs()
+ }
+ class StorageBucketConfig {
+ +bucket_name str
+ +signed_urls_lifespan_seconds
+ }
+ </div>
+ <p>Validation happens at <b>boot</b>, when the provider is constructed. The flow for one placeholder:</p>
+ <div class="mermaid">
+flowchart TD
+ S["uri_format"] --> P{"empty?"}
+ P -- yes --> EM["set a value for uri_format"]
+ P -- no --> F["string.Formatter().parse()"]
+ F -- ValueError --> IV["not a valid format string"]
+ F -- ok --> L["for each placeholder"]
+ L --> U{"base name in<br/>supported set?"}
+ U -- no --> UN["'{x}' is not supported"]
+ U -- yes --> PL{"plain {name}?<br/>no spec / conversion /<br/>attr / index"}
+ PL -- no --> NP["'{name}' must be plain"]
+ PL -- yes --> H{"a {hash} present<br/>somewhere?"}
+ H -- no --> NH["must contain a {hash} placeholder"]
+ H -- yes --> BK["backstop: render with test values"]
+ BK --> OK["valid ✓"]
+ </div>
+ <p><b>Core of the parser</b> (substring check → parse the format string):</p>
+<pre class="diff"><span class="hd">pipelex/tools/storage/storage_config.py</span><span class="del">- elif "{hash}" not in self.uri_format: # GCP: "hash" substring only!</span>
+<span class="del">- error_msgs.append("- uri_format must contain a {hash} placeholder")</span>
+<span class="add">+ URI_FORMAT_SUPPORTED_FIELDS = frozenset({"primary_id", "secondary_id", "hash", "extension"})</span>
+<span class="add">+ parsed = [(name, spec, conv) for _, name, spec, conv in</span>
+<span class="add">+ Formatter().parse(self.uri_format) if name is not None]</span>
+<span class="add">+ for name, spec, conv in parsed:</span>
+<span class="add">+ base = name.split(".")[0].split("[")[0]</span>
+<span class="add">+ if base not in URI_FORMAT_SUPPORTED_FIELDS: unsupported.add(name)</span>
+<span class="add">+ elif name != base or spec or conv: nonplain.add(name) # see note</span>
+<span class="add">+ if base == "hash": hash_found = True</span></pre>
+ <div class="note"><b>Round-2 finding (Codex adversarial pass).</b> The first version enforced "must be plain" only on <code>{hash}</code>. But a format spec on any other field is just as dangerous: <code>{primary_id:.0}</code> truncates to <code>""</code> (collisions again), and <code>{primary_id:1000000000}</code> would balloon the backstop's test-render into a ~1&nbsp;GB allocation at boot. The <code>elif name != base or spec or conv</code> above generalizes the rule to <b>every</b> supported field, and rejects it at parse time <i>before</i> any <code>format()</code> call. Pinned by new S3 regression rows.</div>
+ <p>Also on this fix: bucket configs must set a positive <code>signed_urls_lifespan_seconds</code> (or <code>"disabled"</code>) so signed URLs can't be born dead; the factory now <code>lazy_validate</code>s local/in-memory like it already did S3/GCP; and <code>storage_path</code> dispatches on the configured method instead of blaming a missing local config.</p>
+ </div>
+
+ <div class="tabpane" id="t-mod">
+ <h3>OpenAI image moderation mapping was inverted</h3>
+ <p><code>is_moderated=true</code> sent gpt-image's <i>less</i> restrictive <code>"low"</code>, and <code>false</code> sent <code>"auto"</code> — the exact opposite of "enable moderation".</p>
+<pre class="diff"><span class="hd">pipelex/plugins/openai/openai_img_gen_factory.py</span><span class="ctx"> def moderation_for_openai_image(cls, is_moderated: bool | None):</span>
+<span class="ctx"> if is_moderated is None:</span>
+<span class="ctx"> return omit</span>
+<span class="ctx"> if is_moderated:</span>
+<span class="del">- return "low" # enabled → LESS restrictive (wrong)</span>
+<span class="add">+ return "auto" # enabled → standard filtering</span>
+<span class="del">- return "auto"</span>
+<span class="add">+ return "low" # disabled → less restrictive</span></pre>
+<pre class="diff"><span class="hd">pipelex/cogt/img_gen/img_gen_setting.py</span><span class="del">- is_moderated: bool = False</span>
+<span class="add">+ is_moderated: bool | None = None # None → omit param → provider default ("auto")</span></pre>
+ <p>The default became <code>None</code> so the corrected mapping doesn't silently downgrade default runs: deck alias/handle resolution builds settings <i>without</i> the field, and the old <code>False</code> default would have started sending <code>"low"</code>. With <code>None</code>, workers omit the parameter and the provider's own default applies.</p>
+ </div>
+
+ <div class="tabpane" id="t-concept">
+ <h3>String concept values in bundle specs were unconstructible</h3>
+ <p>Two problems: a <code>mode="before"</code> validator assumed dict input and crashed on the <code>ConceptSpec | str</code> union; and <code>to_blueprint()</code> shoved the string into <code>structure</code>, where the loader only accepts registered structure class names.</p>
+<pre class="diff"><span class="hd">pipelex/core/concepts/concept_blueprint.py (and concept_spec.py — same guard)</span><span class="ctx"> @model_validator(mode="before")</span>
+<span class="del">- def validate(cls, values: dict[str, Any] | str) -> ...:</span>
+<span class="del">- if isinstance(values, dict) and values.get("refines") and values.get("structure"):</span>
+<span class="add">+ def validate(cls, values: Any) -> Any:</span>
+<span class="add">+ # before-validators get RAW input; a non-dict (str in a union) must fall through</span>
+<span class="add">+ if not isinstance(values, dict):</span>
+<span class="add">+ return values</span>
+<span class="add">+ fields = cast("dict[str, Any]", values)</span>
+<span class="add">+ if fields.get("refines") and fields.get("structure"):</span>
+<span class="ctx"> raise ValueError(...)</span></pre>
+<pre class="diff"><span class="hd">pipelex/builder/bundle_spec.py</span><span class="del">- concept[code] = ConceptBlueprint(description=code, structure=string_value)</span>
+<span class="add">+ # a bare string is the concept's description — the blueprint layer accepts it as-is</span>
+<span class="add">+ concept[code] = string_value</span></pre>
+ <p>This matches <code>ConceptFactory.make_from_blueprint</code>, which treats a bare string as <code>ConceptDeclarationType.STRING</code> (a description), not a class reference.</p>
+ </div>
+
+ <div class="tabpane" id="t-observer">
+ <h3>LocalObserver let a payload key overwrite the event name</h3>
+ <p>A payload carrying its own <code>event_type</code> key silently overwrote the lifecycle event name in the JSONL record (which also names the output file). Merge order flipped so the event name wins.</p>
+<pre class="diff"><span class="hd">pipelex/observer/local_observer.py</span><span class="ctx"> payload = {</span>
+<span class="del">- "event_type": event_type,</span>
+<span class="ctx"> **payload,</span>
+<span class="add">+ "event_type": event_type, # last, so a payload key cannot clobber it</span>
+<span class="ctx"> }</span></pre>
+ </div>
+
+ <div class="tabpane" id="t-mistral">
+ <h3>Mistral sent the system message after the user message</h3>
+ <p><code>make_simple_messages</code> appended the system message <i>after</i> the user message — contradicting its own docstring and the OpenAI-typed sibling. System now comes first.</p>
+<pre class="diff"><span class="hd">pipelex/plugins/mistral/mistral_factory.py</span><span class="add">+ if system_text := llm_job.llm_prompt.system_text:</span>
+<span class="add">+ messages.append(SystemMessage(content=system_text))</span>
+<span class="ctx"> ... build user_content ...</span>
+<span class="ctx"> if user_content:</span>
+<span class="ctx"> messages.append(UserMessage(content=user_content))</span>
+<span class="del">- if system_text := llm_job.llm_prompt.system_text: # was appended LAST</span>
+<span class="del">- messages.append(SystemMessage(content=system_text))</span></pre>
+ </div>
+
+ <div class="tabpane" id="t-render">
+ <h3>Output-option numbering was nondeterministic</h3>
+ <p>A <code>PipeCondition</code>'s possible outputs were collected by iterating a dependency <i>set</i>, so the user-facing <code>output_option_N</code> numbering varied between runs with two or more mapped pipes. Now sorted by pipe code.</p>
+<pre class="diff"><span class="hd">pipelex/core/pipes/output/output_renderer.py</span><span class="del">- for mapped_pipe_code in mapped_pipe_codes: # set → nondeterministic</span>
+<span class="add">+ for mapped_pipe_code in sorted(mapped_pipe_codes): # stable output_option_N</span></pre>
+ <p>(Also in this group: <code>temporal/worker_cli.py</code> no longer logs <code>Starting worker for current project 'None'</code> when no project is given.)</p>
+ </div>
+
+ <div class="tabpane" id="t-pytest">
+ <h3>An entire test tree was silently never collected</h3>
+ <p>pytest's default <code>norecursedirs</code> includes <code>build</code>. So everything under <code>tests/unit/pipelex/cli/commands/build/</code> (mirroring the source layout) was <b>never collected</b> in full runs — including a pre-existing cross-package refines regression test that had never run. This is load-bearing for the whole branch: without it, a chunk of the new tests wouldn't run in CI.</p>
+<pre class="diff"><span class="hd">pyproject.toml — [tool.pytest]</span><span class="add">+ testpaths = ["tests"] # bare `pytest` never scans outside tests/</span>
+<span class="add">+ norecursedirs = [ # = pytest defaults MINUS "build"</span>
+<span class="add">+ "*.egg", ".*", "_darcs", "CVS", "dist", "node_modules", "venv", "{arch}",</span>
+<span class="add">+ ]</span></pre>
+ <div class="note"><b>Verified empirically.</b> pytest 9.0.3 reads the <code>[tool.pytest]</code> table natively (promoted to canonical in pytest 9). Recursive discovery from the repo root now finds the <code>build/</code> tests (collected via recursion, not by naming the dir), and the <code>addopts</code> marker filter still applies. The default local config clears the new storage validation at boot.</div>
+ </div>
+ </div>
+
+ <h2>3 · Test conventions (read before flagging)</h2>
+ <p>Several patterns that look unusual are deliberate:</p>
+ <table>
+ <tr><th>Convention</th><th>Why</th></tr>
+ <tr><td>One <code>TestClass</code> per module</td><td>House pytest standard — that's why e.g. doctor coverage is split across several <code>test_doctor_*.py</code> files.</td></tr>
+ <tr><td>pytest-mock (<code>MockerFixture</code>) only</td><td>Never <code>unittest.mock</code> imports. Strong value asserts; parametrization over copy-paste.</td></tr>
+ <tr><td>CLI <i>interface</i> deliberately untested</td><td>Arg parsing, <code>--help</code>, Typer wrappers, agent JSON shapes are owned by the sibling <code>conformance</code> repo. This branch tests the <code>do_*</code> / <code>_core</code> functions beneath them. Low coverage on <code>*_cmd.py</code> wrapper lines is by design.</td></tr>
+ <tr><td>Namespace vs source-module patching</td><td>Top-level imports patched at the consuming module; classes imported <i>inside</i> functions (deferred-import in worker factories) patched at their source module. Worker-factory tests use a fresh real <code>PluginSdkRegistry()</code> per test so the booted singleton is never mutated.</td></tr>
+ <tr><td>Real collaborators where cheap</td><td>Real TOML in <code>tmp_path</code>, real PIL images, real pydantic SDK models, a real aiohttp server for the codec. AsyncMocks only at inference / network / Temporal SDK boundaries.</td></tr>
+ <tr><td>Remaining uncovered lines are deliberate</td><td><code>TYPE_CHECKING</code> blocks, interface-layer wrappers (conformance-owned), lines pinned by the integration suite, live-call worker paths.</td></tr>
+ </table>
+
+ <h2>4 · How it was reviewed</h2>
+ <p>The behavior diff went through CI (tests py3.10–3.14, typecheck, lint, uv-lock), Greptile (multiple rounds, all threads resolved), and a local two-model adversarial pass (Claude + Codex). Findings were verified against the code, deduplicated, and only fixed when a clear win:</p>
+ <ul>
+ <li><b>Fixed:</b> the generalized <code>uri_format</code> plain-placeholder rule (Codex High — see the storage tab note).</li>
+ <li><b>Deferred</b> (recorded in <code>wip/tests/</code>): a pre-existing truthiness-vs-presence edge case in the concept mutual-exclusion check, plus a handful of test-scaffold DRY cleanups. Contrived or pre-existing; not introduced here.</li>
+ <li><b>No-op:</b> the "bare string is now a description" change <i>is</i> the intended fix, confirmed against the blueprint contract.</li>
+ </ul>
+ <p class="mute">Cubic could not run this branch — the workspace hit its free monthly review-line limit (resumes July 1). External constraint, not a code issue.</p>
+
+ <details>
+ <summary>Deferred items (not in this PR)</summary>
+ <ul>
+ <li><b>LocalObserver flat record namespace</b> — the merge-order fix guarantees the event name wins, but lifecycle metadata and payload keys still share one flat dict. Nesting the payload is a breaking JSONL-shape change — decide deliberately.</li>
+ <li><b>uri_format required on configs that never consume it</b> — the Temporal codec / in-memory paths build their own keys, yet the factory now validates a <code>uri_format</code> they ignore. Possible config-shape split.</li>
+ <li><b>Concept mutual-exclusion is truthiness-based</b> — <code>refines</code> + empty-but-present <code>structure</code> slips the guard. Pre-existing; switching to <code>is not None</code> changes behavior for legitimately-empty structures.</li>
+ <li><b><code>img_gen_param_defaults</code> partially dead config</b> — <code>is_moderated</code> / <code>safety_tolerance</code> there are never consumed; whether they should feed setting resolution is an open design call.</li>
+ <li><b>Test-quality cleanups</b> — a Pipelex-boot opt-out conftest for pure test dirs, hoisting duplicated scaffolds, minor in-file dedup.</li>
+ </ul>
+ <p class="mute">Full detail: <code>wip/tests/deferred-source-bugs-pinned-by-tests.md</code>.</p>
+ </details>
+
+ <h2>5 · Verifying locally</h2>
+<pre>make agent-check # lint + typecheck gate
+make agent-test # full offline suite (build/ dirs now collect)</pre>
+
+ <p class="foot">
+ Generated for the <code>feature/Add-tests</code> reviewers. Branch reviewer's guide lives in <code>TODOS.md</code>; user-facing wording in <code>CHANGELOG.md</code> under <code>[Unreleased]</code>.<br>
+ Tip: this page has <kbd>▸</kbd> foldable sections and tabbed diffs — click around.
+ </p>
+
+</div>
+
+<script>
+ // Tabs
+ document.querySelectorAll('.tabbtn').forEach(function (btn) {
+ btn.addEventListener('click', function () {
+ var pane = btn.getAttribute('data-tab');
+ document.querySelectorAll('#bugtabs .tabbtn').forEach(function (b) { b.classList.remove('active'); });
+ document.querySelectorAll('#bugtabs .tabpane').forEach(function (p) { p.classList.remove('active'); });
+ btn.classList.add('active');
+ document.getElementById(pane).classList.add('active');
+ });
+ });
+ // Mermaid
+ mermaid.initialize({ startOnLoad: true, theme: 'dark', themeVariables: { fontSize: '14px' } });
+</script>
+</body>
+</html>
diff --git a/wip/tests/deferred-source-bugs-pinned-by-tests.md b/wip/tests/deferred-source-bugs-pinned-by-tests.md
new file mode 100644
index 000000000..115e76831
--- /dev/null
+++ b/wip/tests/deferred-source-bugs-pinned-by-tests.md
@@ -0,0 +1,29 @@
+# Source bugs pinned by the coverage-grind tests — RESOLVED (cleanups remaining)
+
+Outcome of the post-grind code review (2026-06-12, branch `feature/Add-tests`). The coverage grind (TODOS.md Phases C/D/E) deliberately wrote tests that pin **current** behavior — including behavior the review then judged to be wrong at the source. The defects were initially deferred here; they have since been **verified and fixed on this same branch** (2026-06-12), with each pinning test flipped alongside its source fix and the fixes recorded in `CHANGELOG.md` under `[Unreleased]`.
+
+Applied fixes (see CHANGELOG for user-facing wording):
+
+1. **OpenAI image moderation mapping inverted** — `moderation_for_openai_image` now maps enabled → `"auto"`, disabled → `"low"`; the OPENAI_MODERATION parametrize rows flipped with it.
+2. **GCP storage `lazy_validate` accepted uri_format without a real `{hash}` placeholder** — now checks the literal `"{hash}"` like S3, and the GCP error message gained the `- ` fix-line prefixes and docs URL; the bare-substring acceptance test became a rejection parametrize row. A post-fix review pass then extended this: the per-provider checks and message assembly were consolidated onto a shared `StorageMethodConfig` base (S3/GCP can no longer drift apart), and the local/in-memory configs — which had NO validation at all and were exposed to the same constant-URI overwrite — got `lazy_validate` too, wired into the storage provider factory like S3/GCP and pinned by their own test modules.
+3. **Bundle-spec string-concept arm mapped the string into `structure`** — the string now passes through to the blueprint dict, where the loader treats it as the concept's description.
+4. **`ConceptSpec.model_validate_spec` crashed on non-dict input** — gained the same `isinstance(values, dict)` guard as `ConceptBlueprint.validate_mutually_exclusive_fields`, so the `ConceptSpec | str` union falls through to `str` normally. All `model_construct` workaround sites in the bundle-spec tests were swept back to normal construction, and a test pins the normal-validation path for string concepts.
+5. **Mistral `make_simple_messages` sent the system message AFTER the user message** — system now comes first, matching the docstring and the OpenAI-typed sibling; the ordering test flipped.
+6. **`StorageProviderConfig.storage_path` ignored `method` and raised a misleading message** — the property now dispatches on method (mirroring `uri_format`), raising "storage_path is only available when method is local" for non-local methods; both pinning tests flipped.
+7. **LocalObserver let a payload's own `event_type` overwrite the event name** — merge order flipped so the lifecycle event name wins; the pinning test flipped with it.
+
+## Deferred design tradeoffs (from the post-fix review pass and the pre-landing /review army)
+
+- **LocalObserver flat record namespace:** the merge-order fix guarantees the lifecycle `event_type` wins, but lifecycle metadata and arbitrary payload keys still share one flat dict in the JSONL record. Any future metadata field (timestamp, run id) re-opens the same collision class with whatever payload key shares its name. The deeper fix is nesting the payload under its own key (e.g. `{"event_type": ..., "payload": {...}}`), which is a breaking change to the JSONL record shape for downstream consumers — decide deliberately, don't slip it into an unrelated change.
+- **`uri_format` is required on storage configs that never consume it:** the Temporal payload codec builds its own keys (`storage_prefix + hash_hex` in `storage_payload_codec.py`) and `InMemoryStorageProvider()` takes no config, yet the provider factory now `lazy_validate`s those sections, so a codec storage config must carry a ritual `uri_format = "{hash}"` and the error message tells operators to fix a value that has no effect on that path. Decide whether the uri_format checks belong only to the asset-storage path (e.g. split the codec/in-memory config shape from the asset one).
+- **`validate_storage_provider_config` raises `StorageConfigError` from a pydantic `model_validator`:** pre-existing behavior, now pinned by a test. House style says validators raise `ValueError`/`TypeError` so pydantic aggregates them into the config tree's `ValidationError` with proper field locations; the unwrapped custom error aborts whole-config validation instead. Changing it changes the exception contract for any caller catching `StorageConfigError` on construction — decide deliberately.
+- **`rendered_pretty` interpolates LLM-authored strings into Rich markup unescaped:** `PipelexBundleSpec.rendered_pretty` (and sibling spec renderers) pass authored strings through `Text.from_markup`, so unbalanced markup like `[/]` raises `rich.errors.MarkupError` and tags restyle output. Console-display-only severity, pre-existing pattern across several fields (the string-concept arm is just newly reachable). Sweep once with `rich.markup.escape` rather than patching one arm.
+- **Concept mutual-exclusion check is truthiness-based, not presence-based:** both `ConceptBlueprint.validate_mutually_exclusive_fields` and `ConceptSpec.model_validate_spec` reject "refines and structure both set" with `fields.get("refines") and fields.get("structure")`. That's a *truthiness* test, so a contrived config with `refines = "Base"` and an empty-but-present `structure` (`""`/`{}`) slips past the guard, and `ConceptFactory.make_from_blueprint` then prioritizes `structure is not None` and silently takes the structure path. Pre-existing (this branch only added the non-dict `isinstance` guard, not the truthiness test) and only reachable with a self-inflicted empty structure. Switching to `... is not None` would change behavior for any legitimately-empty-but-present structure value, so decide deliberately rather than slipping it into the test branch. Surfaced by the pre-landing Codex pass.
+- **`img_gen_param_defaults` is partially dead config:** `is_moderated` and `safety_tolerance` in `[cogt.img_gen_config.img_gen_param_defaults]` are never consumed — `pipe_img_gen` takes those values from the resolved `ImgGenSetting` with no fallback to the param defaults (unlike `aspect_ratio`/`background`/`is_raw`). The safety consequence for moderation was neutralized by defaulting `ImgGenSetting.is_moderated` to `None` (provider default applies); whether param defaults should feed setting resolution — and what that means for `safety_tolerance = 5` suddenly becoming effective on FAL — is an open design call.
+
+## Test-quality cleanups (no source impact, batch when convenient) — STILL DEFERRED
+
+- **Pipelex-boot opt-out for pure test dirs:** many of the new pure-logic modules (tools/misc, tools/storage, plugins factories, model_deck_check, backend_credentials) pay the autouse module-scoped `Pipelex.make()` from `tests/conftest.py` without using it. Precedent exists: `tests/unit/pipelex/system/pipelex_service/conftest.py` overrides `reset_pipelex_config_fixture` with a no-op. A one-file conftest per pure directory shaves seconds off every suite run.
+- **Hoist duplicated scaffolds where a shared home already exists:** the three `graph/test_graph_rendering_*.py` files rebuild the same bundle-write + `dry_run_pipeline` AsyncMock scaffold (and `graph/conftest.py` already exports shared helpers); `make_llm_spec` is copied across the three `builder/test_bundle_spec_*.py` files (builder subpackages use `test_data.py` modules for exactly this); the storage trio re-implements the S3/GCP config builders in `test_storage_provider_config.py`.
+- **Minor in-file dedup:** `test_storage_provider_config.py` carries the same 4-row method table twice; `test_local_observer.py`'s str/Path constructor twins could be one parametrized test; `test_model_deck_check.py` rebuilds the full ModelDeck inside every parametrized case (class-scoped fixture would do); `NESTED_TOML_CONTENT` is duplicated across two toml_sync files; `test_inputs_ops.py`'s fixture dict carries an unused `get_library_manager` key; the mistral messages tests use id()-keyed side_effect closures with asserts inside mocks where positional `side_effect=[...]` + post-hoc assertions would fail more readably. (Done already: `test_storage_s3_config.py`'s standalone braced-hash test that duplicated a parametrize row was removed during the fix pass.)
+- **More scaffold hoists found by the pre-landing review army:** the three `test_output_renderer_*.py` modules re-declare the same patch-target constant and near-identical mock-pipe builders (a `tests/unit/pipelex/core/pipes/output/conftest.py` would hold both); the `core_mocks`-style fixture stub block is repeated across `test_inputs_core.py`/`test_output_core.py`/`test_runner_core.py` and `builder/operations/test_inputs_ops.py`; `make_backend`/`patch_hub_getters` are duplicated between `test_llm_worker_factory.py` and `test_img_gen_worker_factory.py` (they encode the booted-singleton-protection convention, so drift would silently weaken one factory's isolation); the PNG/PDF `FileType` constants re-declared across the mistral/gateway message tests belong in the existing `test_data.py`.
diff --git a/wip/tests/missing-tests-menu.md b/wip/tests/missing-tests-menu.md
new file mode 100644
index 000000000..ea1a94933
--- /dev/null
+++ b/wip/tests/missing-tests-menu.md
@@ -0,0 +1,64 @@
+# Missing Tests Menu
+
+Source: full non-inference suite run with `--cov=pipelex` (overall ~78% line coverage after sections A–E were ground down — was ~72%) plus a structural sweep of `pipelex/` vs `tests/`. This is a curated menu of the gaps that matter, not an exhaustive list. Caveat: plugin inference workers' live-call paths are exercised by `inference`-marked tests that don't run in this measurement, so their numbers understate real coverage — only their offline-testable logic (factories, arg builders, error mapping) is listed here.
+
+## A. CLI internal logic — DONE (2026-06-12, see TODOS.md for the as-built notes)
+
+Scope note: the spec'd CLI *interface* (arg parsing, `--help` surfaces, `init` behavior, `validate --all`, agent JSON output shapes) is owned by `../conformance` (paired with `docs/specs/`) — not duplicated here. This section is only the pipelex-internal logic those commands run, which conformance's subprocess-level tests never reach (and which doesn't count toward conformance coverage anyway). All testable offline with Typer's `CliRunner` or direct calls into the `_core` modules.
+
+- [x] `cli/commands/doctor_cmd.py` (21% → **92%**, ~600 stmts) — all diagnostic checks, fix mode, health-report rendering covered; remainder ≈ `setup_doctor_runtime` (once-per-process log.configure side effects, deliberately untested).
+- [x] `cli/commands/run/_run_core.py` (32% → **94%**) — happy path, bundle/inputs resolution, output saving (graphs/main_stuff/working-memory/CSV), wrapper error dispatch.
+- [x] `cli/commands/show_cmd.py` (27% → **73%**) and `which_cmd.py` (27% → **66%**) — the `do_*` report logic is fully covered; the remainder is the Typer wrappers that boot Pipelex (interface layer, conformance-owned).
+- [x] `cli/commands/build/*` internals (`_output_core` 41→**74%**, `_runner_core` 21→**76%**, `_inputs_core` 21→**72%**, `structures_cmd` 41→**69%**) — codegen cores covered; remainder is the Typer command wrappers. NB: these tests live in `tests/unit/pipelex/cli/commands/build/`, which pytest's default `norecursedirs` silently skipped until the pyproject override (defaults minus `build`) — the pre-existing cross-package refines regression test there had never been running.
+- [x] `cli/readiness.py` (20% → **97%**) — venv detection, dev-install detection, and the readiness gate.
+- [x] `cli/error_handlers.py` (51% → **99%**, ~290 stmts) — every handler incl. gateway/telemetry/signature ones, panel rendering, validation-error detail sections.
+
+## B. Temporal distributed execution — DONE (2026-06-12, see TODOS.md Phase B for the as-built notes)
+
+Never shipped to prod yet, but these are the deploy-critical entry points; bugs here only surface inside a live cluster.
+
+- [x] `temporal/worker_cli.py` (0% → **100%**) — project resolution from pyproject, fast-fail task-queue validation before library load, worker-base library loading, forced `is_enabled`, full Typer arg wiring into the worker loop.
+- [x] `temporal/codec/codec_server.py` (0% → **96%**) — encode/decode round-trip over real HTTP, CORS preflight/origin gating, content-type and malformed-payload guards, storage-failure → HTTP status mapping; remainder = the `TYPE_CHECKING` import block. NB: `codec_server_cli.py` stays at 0% — it's the thin arg-parse + `run_app` wrapper around `build_codec_server`, same interface-layer category as the Typer wrappers left out in section A.
+- [x] `temporal/temporal_connect.py` (24% → **98%**) — API-key resolution per secret method, TLS/RPC-metadata wiring, payload-codec converter selection, named server-config selection, SDK error wrapping; remainder = the `TYPE_CHECKING` import block.
+
+## C. Inference plumbing (offline-testable parts) — DONE (2026-06-12, see TODOS.md Phase C for the as-built notes)
+
+Not the live calls — the factories, arg-builders, and config logic around them, which can break independently of any provider.
+
+- [x] `cogt/img_gen/img_gen_args_factory.py` (58% → **100%**, ~320 stmts) — all provider taxonomies (Flux, Flux-1.1 Ultra, Qwen, SDXL/Lightning, GPT-image, BFL Flux 2) across aspect-ratio, prompt, inference, safety, output-format, input-images, and input-fidelity arg-building.
+- [x] `cogt/llm/llm_worker_factory.py` (28% → **100%**) and `img_gen/img_gen_worker_factory.py` (40% → **99%**) — full SDK routing matrices, client-instance caching, missing-dependency errors; remainder = a `TYPE_CHECKING` import.
+- [x] `cogt/llm/structured_output.py` (55% → **99%**) — every `StructureMethod` → instructor mode pinned with a completeness guard; remainder = a `TYPE_CHECKING` import.
+- [x] `cogt/model_backends/backend_credentials.py` (30% → **100%**) — env vs generic-provider error messages, missing/placeholder var aggregation, report models.
+- [x] `cogt/models/model_deck_check.py` (42% → **100%**) — every check function x every reference kind (found/not-found), fuzzy suggestions, wrong-sigil hints, setting short-circuits.
+- [x] `plugins/gateway/gateway_completions_factory.py` (21% → **96%**) and `mistral_factory.py` (27% → **99%**) — client construction, message shaping, and the extract-output parsers (Azure/Mistral/Deepseek/Linkup; OCR responses, image cleanup, document prep/upload). Remainders = `TYPE_CHECKING` blocks, a defensive `except` reachable only if `model_dump` itself throws, and mistral's `make_mistral_client` retry wiring (already pinned by `test_transport_retry_wiring.py`).
+
+## D. Core runtime odds and ends — DONE (2026-06-12, see TODOS.md Phase D for the as-built notes)
+
+- [x] `observer/local_observer.py` (0% → **100%**) — constructor dir resolution, every observe method, JSONL append semantics, event-type collision behavior (originally pinned as payload-wins; the source was then fixed on this same branch so the lifecycle event name wins, and the pinning test flipped with it).
+- [x] `core/pipes/output/output_renderer.py` (33% → **98%**) — Anything-output resolution through PipeCondition/PipeSequence (incl. recursion), operator arms, every Anything render format; remainder = the `TYPE_CHECKING` import block.
+- [x] `graph/graph_rendering.py` (40% → **100%**) — `_dry_run_bundle` library-dirs matrix, format dispatch, the sanitized-rename branch, `generate_view_for_bundle` direction precedence.
+- [x] `builder/bundle_spec.py` (34% → **100%**) and `builder/operations/inputs_ops.py` (20% → **98%**) — spec validation, `to_blueprint()` ordering/error wrapping, pretty rendering; every `build_inputs_for_pipe` branch. Remainder = a `TYPE_CHECKING` import. NB: the test phase surfaced a real source bug here (`ConceptSpec.model_validate_spec` crashed on plain-string concept values); it was fixed later on this same branch with the pinning tests flipped — see `deferred-source-bugs-pinned-by-tests.md` and the CHANGELOG `[Unreleased]` entry.
+- [x] `pipeline/runner.py` (65% → **93%**) — `extra` rejection, both `except PipelexError` arms, `except ValidationError`, `start()`, and the protocol surfaces `validate()` (incl. the finally restore matrix), `models()`, `version()`. Remainder = the `TYPE_CHECKING` block plus the happy-path/`PipeRouterError`/tracer-close lines already pinned by the integration suite.
+
+## E. Tools / config — DONE (2026-06-12, see TODOS.md Phase E for the as-built notes)
+
+- [x] `tools/misc/toml_sync.py` (0% → **98%**) and `tools/misc/document_utils.py` (0% → **97%**) — toml_sync's full read/set/sync surface incl. the destroy-config guards (never creates keys, comment/structure preservation, dry-run, no-op write guard, idempotency); document format enum matrix. Remainders = a `TYPE_CHECKING` import, an unreachable defensive `continue`, and the type-checker-appeasement trailing `raise`.
+- [x] `tools/storage/storage_config.py` (45% → **100%**) — per-provider `lazy_validate` fault matrices, provider-config validator, `uri_format`/`storage_path` properties. (The originally-pinned S3-vs-GCP `{hash}` asymmetry was then eliminated on this same branch: a shared config base validates every provider, local/in-memory included, each pinned by its own test module.)
+- [x] `tools/misc/image_utils.py` (52% → **98%**) — format enum matrix, both unsupported-MIME message branches, PIL conversion round-trips; remainder = the unreachable trailing `raise`.
+
+## Deliberately not on the menu
+
+- CLI interface surfaces owned by `../conformance`: `init` (dedicated modules there), `validate` command wrappers, `--help` smoke tests, agent JSON output shapes. Rule of thumb: cross-repo contract behavior → conformance test paired with a spec section; pipelex-internal logic → here.
+- `cli/dev_cli/*` (mostly 0%) — internal dev tooling, not shipped; low value per test.
+- `plugins/*_list.py`, `huggingface_factory` (0%) — model-listing scripts hitting live APIs; belongs to inference-marked tests if anything.
+- `pipelex.py` / `hub.py` (84%/83%) — already decent; remaining lines are teardown/edge accessors.
+- Templating, input normalizer, URI resolver, reporting — explorer flagged them as thin on dedicated tests, but coverage shows they're well exercised (88–100%) via integration; dedicated unit tests would be redundant.
+
+## Suggested grind order
+
+1. ~~**A (CLI)**~~ — DONE 2026-06-12 (branch `feature/Add-tests`; plan + as-built notes in TODOS.md).
+2. ~~**B (Temporal)**~~ — DONE 2026-06-12 (user-prioritized ahead of C; same branch, TODOS.md Phase B).
+3. ~~**C (inference plumbing)**~~ — DONE 2026-06-12 (same branch, TODOS.md Phase C).
+4. ~~**D then E**~~ — DONE 2026-06-12 (same branch, TODOS.md Phases D and E).
+
+All menu sections are complete. Remaining gaps are deliberate (see "Deliberately not on the menu") plus the per-module remainders noted inline above (TYPE_CHECKING blocks, interface-layer wrappers, integration-pinned lines).