Skip to content

Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.

Feature/more plugins (#1015)

Louis Choqueld6e5dd690307
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 6f47b107f..3ab3e68ae 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -9,7 +9,9 @@
- **The main-stuff invariant is now enforced at every post-run boundary (breaking):** With every pipe stamping a main output, the defensive "maybe there is no main stuff" branches for completed runs are gone. Wire models make the field required (`main_stuff_name: str` on `PipelexPipeRunOutput` and on the `/execute` response extension), result delivery always writes the `main_stuff.*` artifact files (a completed run with no main stuff now fails delivery loudly instead of storing an empty envelope), and the graph tracer and OTel/Langfuse output capture read the main stuff directly. `PipeOutput.optional_main_stuff` is deleted — use `PipeOutput.main_stuff`; `WorkingMemory.get_optional_main_stuff()` remains for pre-run/empty memories.
- **Orchestrator SPI: the delivery axis is now two methods (breaking):** `OrchestratorProtocol.run` no longer takes a `delivery` parameter — it is the BLOCKING arm and returns the completed-run `PipelexPipeRunOutput` (which therefore requires `main_stuff_name` with no "not finished yet" escape hatch). The FIRE_AND_FORGET arm is the new `OrchestratorProtocol.start`, returning the new ids-only `PipelexPipeDispatchAck` (`pipeline_run_id` + `workflow_id`). `PipelexPipeRunOutput.is_completed` is deleted (it is always a completed output now); `DeliveryMode` remains the wire-input vocabulary that picks which method the endpoint calls. Orchestrators that cannot do genuine async (core's DIRECT) implement `start` by raising, unreachable behind the `supports_fire_and_forget` gate. Migration: orchestrator plugins split their `match delivery` bodies into the two methods; `/start`-serving runners call `orchestrator.start(...)` and read the ack's `workflow_id`.
- **`PipeParallel` honors `final_stuff_code`:** A requested final stuff code (e.g. pre-allocated by a `PipeBatch` for its branch outputs) is now stamped on the parallel's combined stuff instead of being discarded, so the combined output keeps its graph identity when a parallel runs as a batch branch.
-
+- **`LLMWorkerInternalAbstract` folded into `LLMWorkerAbstract` (breaking):** The LLM worker family now matches the other three (img_gen, extract, search): the abstract base holds the `InferenceModelSpec` (`__init__(inference_model, reporting_delegate=None)`) and owns the whole job lifecycle — `llm_job_before_start` (started_at stamp + usage-report init), spec-driven capability checks (vision, documents), constraint application, and spec-based OTel naming. Previously those lived on the intermediate `LLMWorkerInternalAbstract` (a remnant of the pre-plugin-system "external plugin" era), so an out-of-tree worker subclassing the documented `LLMWorkerAbstract` silently inherited a template method with a hole in it: forgetting to call `llm_job_before_start` crashed reporting on a `None` duration. That state is now unrepresentable — a plugin worker implements `_gen_text`/`_gen_object` and nothing else. `LLMWorkerInternalAbstract` is gone; subclasses now extend `LLMWorkerAbstract` directly. The prompt/response debug dumps moved from the worker base to the `llm_generate` funnel (the dump helpers in `llm_utils` are now config-gated internally), keeping the worker ABC free of `pipelex.config` (the hub imports the ABC at module level, and config imports the hub — a config import from the ABC would be an import cycle).
+- **Legacy `set_llm_worker_from_external_plugin` removed (breaking):** This pre-plugin-system path on the inference manager registered a spec-less worker class by handle — the old meaning of "external plugin". The real seam (a `pipelex.plugins` entry point registering `(family, sdk)` backends) fully replaces it.
+- **Docs — cookbook inference-plugins page rewritten for the plugin system:** `docs/cookbook/using-inference-plugins.md` now documents the real cookbook example (an installable `pipelex.plugins` entry-point package serving the zero-key `hello-1` model) instead of the retired pre-plugin-system "plugin = config entry" walkthrough that referenced a model handle defined nowhere. Also fixed the stale claim that model config lives in `.pipelex/pipelex.toml` (it lives in `.pipelex/inference/`), and corrected the lookup-miss error named in `docs/under-the-hood/inference-backend-plugins.md` (`InferenceBackendNotFoundError`, not `NotImplementedError`).
- **Dry-run mock lists shrink from 3 to 2 items:** When a pipe input is declared as a list without a fixed size, the dry-run mock now fabricates 2 items instead of 3. Cuts dry-run work on list inputs without changing correctness.
- **Repo lint validates `.mthds` files against the locally generated MTHDS schema:** `plxt lint` in this repo now uses `derived/mthds_schema.json` (regenerated automatically by the lint make targets) instead of the released schema bundled with plxt, so language-surface additions like `size` are lintable here before they propagate to a plxt release.
@@ -29,6 +31,7 @@
- **Stale main stuff after an `add_each_output`-only parallel:** A pipeline ending in an `add_each_output`-only `PipeParallel` used to silently report the *previous* step's output (or even an input) as its main result, and such a parallel as the top-level pipe on the API path completed with no main stuff at all. The parallel now always stamps its combined output, in live and dry runs alike.
- **Img2img support check keyed on declared inputs:** `ImgGenParamSupport.check_job_params` now takes the full model spec and checks image-to-image support against the model's declared `inputs` (the universal capability field) instead of requiring an `input_images` rule. The rule is an args-factory formatting concern that Gemini-family models — whose request path never reads it — legitimately don't carry, so the old rules-keyed check wrongly reported the whole family as not supporting img2img.
- **Google img-gen worker no longer hardcodes 1K:** The native Google worker and the gateway path now send the requested `image_size` to the Gemini API (omitted entirely when `size` is unset); previously the worker always sent `"1K"` regardless of intent, making 2K/4K generation unreachable.
+- **Routing-profile `optional_routes` were silently dropped:** `RoutingProfileFactory.make_routing_profile` parsed and validated the `optional_routes` table from `routing_profiles.toml` but never passed it to the built `RoutingProfile`, so optional routes declared in config had no effect — models relying on one fell through to the profile's default backend and could silently vanish from the model deck. The factory now carries `optional_routes` through, with unit coverage for the pass-through and for the enabled/disabled-backend gating.
- **Unknown boot orchestrator now fails loud:** Requesting a boot orchestrator no installed plugin provides — via `--orchestrator <name>` or `Pipelex.make(boot_orchestrator=...)` — now raises `UnknownBootOrchestratorError` at boot instead of silently falling back to in-process execution. A typo or a missing orchestrator plugin (e.g. `--orchestrator temporal` without the Temporal plugin installed) is reported rather than quietly running the job on the wrong runtime. The requested name is matched against registered plugin names, the same namespace the boot gate uses.
- **Failed boot no longer leaks process-global state:** A `Pipelex.make` that raises during setup now releases the process-global singletons a partial boot acquired (config, logging, the kajson class registry, template registries), mirroring `teardown`. Previously a failed boot could poison a subsequent boot in the same process — surfacing as a "LogConfig is already set" error or a stale, half-populated class registry.
diff --git a/TODOS.md b/TODOS.md
index d7b14d2a4..7c0c670f2 100644
--- a/TODOS.md
+++ b/TODOS.md
@@ -1,190 +1,113 @@
-# TODOS — Required main stuff (PipeParallel always combines)
+# Cookbook "hello plugin" → real inference-backend plugin (Option B)
-**Branch:** `feature/Required-main-stuff` · **Design:** `wip/required-main-stuff.md` (full) · `wip/required-main-stuff-brief.html` (brief)
+Status: **CHECKPOINT B CLEARED — TRACK COMPLETE (2026-07-03).** Phases 0–4 done and ALL COMMITTED. Pipelex (`feature/More-plugins`): `2f9b7e1d9` (optional_routes fix), `dee230f46` (LLM worker fold, breaking), plus the docs+tracker commit carrying this file. Cookbook (`dev`): `ecb231c` (the whole example; root `pyproject.toml`/`uv.lock` editable pin deliberately left uncommitted — flip to `pipelex==X.Y.Z` at release, see caveat below). Only remaining action = release ordering (step 3 below). This tracker is retired.
-**Scope:** implement the design **excluding §9 (companion track — `/execute` vs `/start`→delivery result-shape unification)**, which is designed and tackled separately. Everything else is in scope: always-combine `PipeParallel`, delete `combined_output`, new `native.Composite` concept, static branch-name validation, invariant enforcement across in-repo surfaces, then the gated cross-repo sweep.
+## NEXT SESSION — start here
-## The rule being implemented
+1. **Commit slicing first** (user to approve slices). This worktree (`_plugins`, branch `feature/More-plugins`) holds four separable concerns — do NOT mix them:
+ - `optional_routes` factory fix: `pipelex/cogt/model_routing/routing_profile_factory.py` + `tests/unit/pipelex/cogt/model_routing/test_routing_profile_optional_routes.py` + its CHANGELOG "Fixed" entry.
+ - LLM worker fold (breaking): `llm_worker_abstract.py` (folded), `llm_worker_internal_abstract.py` (deleted), `llm_utils.py` + `llm_generate.py` (dump gating moved), `inference_manager{,_protocol}.py` (legacy setter removed), 6 re-parented workers under `pipelex/plugins/`, reworked tests (`test_worker_error_enrichment`, `test_external_plugin`, `test_llm_gen_text`, `test_setup_inference_workers`, factory/manager types) + the two CHANGELOG "Changed" entries.
+ - Orchestrator-plugins docs from a PRIOR session (unrelated to this track — `docs/under-the-hood/orchestrator-plugins.md` etc.).
+ - `TODOS.md` (this tracker).
+ Cookbook side = one coherent commit (example package, `.pipelex/inference/` config, `.mthds`, README×2, CHANGELOG, `test_bundles.py`, mypy exclude) — but the `[tool.uv.sources]` editable pin in `pyproject.toml`/`uv.lock` must NOT ship (see editable-pin caveat below).
+2. **Phase 4 — docs rewrite** (checkboxes below). Also re-check `docs/under-the-hood/inference-backend-plugins.md`'s worker section against the folded base (constructor now takes `inference_model`; no lifecycle gotchas left to document).
+3. **Release ordering**: pipelex release > 0.36.0 (breaking changelog) → flip cookbook pin to `pipelex==X.Y.Z` → cookbook release. Before the pipelex release, grep `pipelex-temporal` + `pipelex-mistralai-workflows` for LLM worker subclasses / `set_llm_worker_from_external_plugin` (expected: none — both are orchestrator-only).
-> A pipe run always delivers a main stuff. Consequently, a pipeline always delivers a main stuff.
+All gates were green at session end (2026-07-03): pipelex `make agent-check` + `make tb` + FULL `make agent-test`; cookbook `make agent-check` + `make agent-test` + hello-plugin real run end-to-end.
-Enforced at the one non-stamping leaf: `PipeParallel` always combines its branch outputs into its declared `output` concept and stamps that as main stuff. `combined_output` is deleted (breaking, no back-compat per house policy). `native.Composite` is the untyped combination vehicle.
+## Goal
-## Working conventions for this track
+Replace the legacy cookbook example `pipelex-cookbook/examples/c_advanced/using_inference_plugins/` with a genuine entry-point plugin that demonstrates the real plugin system. Today that example is a single `.mthds` file whose `model` field references the handle `llm_plugin_example_using_openai` — a name **defined nowhere** (not in the cookbook's `.pipelex/inference/`, not in the core kit). "Plugin" in its name is pre-plugin-system vocabulary meaning "custom model config entry"; the example has only ever run as DRY RUN (see `pipelex-cookbook/.pipelex/traces/`). Decision taken with the user (2026-07-03): **Option B** — make it an actual `pipelex.plugins` entry-point plugin, not a config-only reframe.
-- **TDD:** write the red tests first for each behavioral change, then implement to green.
-- **Gates:** `make agent-check` after code changes; targeted pytest during development (`.venv/bin/pytest -x -q <path>`); full `make agent-test` before each checkpoint.
-- **Commits:** one commit per phase, landed at its checkpoint, so each review sub-agent gets a clean SHA range.
-- **Checkpoint protocol (MANDATORY STOP):** at each checkpoint the agent must stop forward progress and, in order:
- 1. **Verify** — run the phase's full gates (`make agent-check` + `make agent-test`) and confirm the phase's acceptance criteria below.
- 2. **Update this file** — check off done items, record decisions taken, deviations from the design, and current code state in the "Checkpoint log" section, so a brand-new session can cold-start from this file alone. Update `wip/required-main-stuff.md` status line too.
- 3. **Fan out review** — spawn a **Sonnet-5 sub-agent with NO inherited context** to run the `/code-review` skill on the phase's changes. Hand it *only* a pointer to the changes (the phase's commit SHA range, e.g. `git diff <base>..HEAD`, or the working-tree diff) — never the plan, the design doc, the rationale, or your own conclusions. Target: clean solid software, not over-engineering.
- 4. **Triage findings** — fix real defects before proceeding; findings that are design tradeoffs (not silent bugs) go to a deferred-items doc under `wip/required-main-stuff/`, not reflexively applied.
+## Background: the plugin system (cold-start summary)
-## Phase 1 — Runtime + language surface (this repo)
+The pipelex repo (this worktree, `_plugins`) has a discovery-based plugin system. Everything below is verified against the current tree:
-### 1a. Red tests first
+- **Contract** (`pipelex/plugins/contract.py`): a plugin is any object satisfying the `@runtime_checkable` `PipelexPlugin` protocol — `name: str`, `targets_api: int` (must equal `PLUGIN_API_VERSION`, currently **2**), and `register(self, registrar) -> None`. `register` is **side-effect-free**: it may only call registrar menu methods — no I/O, no SDK import, no client construction. Heavy work goes inside the `make_worker` closures.
+- **Discovery** (`pipelex/plugins/discovery.py`): `build_registrar` iterates `BUILTIN_PLUGINS` then external entry points in group **`pipelex.plugins`**. An entry point may resolve to a plugin instance or a zero-arg factory returning one. Denylist via `plugins.disabled` config; fail-loud on duplicates/version mismatch/broken plugin.
+- **Inference seam** (`pipelex/plugins/inference_backend_registry.py`): `registrar.add_inference_backend(family=InferenceFamily.LLM, sdk="<token>", make_worker=...)`. A model's `sdk` field selects the backend factory; worker factories hold no match over SDK strings. `MakeWorkerFn` is called as `make_worker(*, inference_model: InferenceModelSpec, backend: InferenceBackend, sdk_clients: SdkClientRegistry, reporting_delegate: ReportingProtocol | None) -> InferenceWorkerAbstract`. Use `require_sdk(...)` inside `make_worker` for optional-dependency guards.
+- **Minimal LLM worker**: subclass `LLMWorkerAbstract` (`pipelex/cogt/llm/llm_worker_abstract.py`); since the fold (see follow-up below) the abstract surface is just `_gen_text` + `_gen_object` — the base takes `inference_model` in `__init__`, owns the job lifecycle, and derives capability flags from the spec.
+- **Optional companion**: `registrar.add_model_lister(sdk="<token>", lister=...)` — powers `pipelex show models`.
+- **Reference implementations**: smallest builtin = `pipelex/plugins/blackboxai/blackboxai_plugin.py` (one `add_inference_backend`, lazy imports inside `_make_..._worker`). External entry-point precedents = `pipelex-temporal` (`pipelex_temporal/temporal_plugin.py`) and `pipelex-mistralai-workflows`.
+- **Canonical authoring doc**: `docs/under-the-hood/inference-backend-plugins.md` (the "acme" walkthrough — the cookbook example should be its living counterpart).
+- **Verification command**: `pipelex plugins list` prints every discovered plugin (origin builtin/external, status, API version, contributions).
-- [x] Regression test for the **stale-stamp bug**: a `PipeSequence` ending in an `add_each`-only `PipeParallel` must deliver the combined composite as main stuff, not the previous step's output (today it silently delivers step N-1's stuff).
-- [x] **Terminal-parallel delivers main stuff**: an `add_each`-only parallel as top-level `main_pipe` with memory from `make_from_pipeline_inputs` (the API path) completes with a stamped main stuff (today `main_stuff_name` is `None` and `PipeOutput.main_stuff` raises).
-- [x] **Static validation accept/reject cases**: `output = "Composite"` accepted; structured concept with fields matching branch `result` names accepted (required fields ⊆ result names, result names ⊆ declared fields); native non-composite (`Text`, `Image`, …), `Dynamic`, `Anything`, and multiplicity suffixes (`Foo[]`, `Foo[N]`) rejected at bundle/library validation time.
-- [x] **`CompositeContent` round-trips**: `smart_dump`, kajson transport round-trip (`dump_for_transport` → rehydrate), `rendered_markdown` / `rendered_html` with named sub-contents surfaced as top-level fields (no wrapper key).
-- [x] **Dry-run parity**: dry run of a parallel stamps the same combined main stuff shape as live run.
+Model config layer (how a `.mthds` `model` handle reaches the plugin): the cookbook's `.pipelex/inference/backends.toml` declares backends keyed by name (`[hello]`, `enabled`, `api_key`...); per-backend model files `backends/<name>.toml` declare model sections with `[defaults]` carrying `sdk = "<token>"` — the section header is the model handle referenced from `.mthds`. The `add-model` skill (in this repo's `.claude/skills/`) documents the full add-a-model procedure including routing profiles.
-### 1b. `native.Composite` concept
+## Decisions
-- [x] Add `COMPOSITE = "Composite"` to `NativeConceptCode` in `pipelex/core/concepts/native/concept_native.py`.
-- [x] New `CompositeContent(StuffContent)` in `pipelex/core/stuffs/composite_content.py` — pydantic `extra="allow"` so branch names are top-level serialized fields; full content surface (`smart_dump`, kajson round-trip, `rendered_markdown`/`rendered_html`).
-- [x] Register in the class registry + concept factory wherever native concepts are wired.
+- **D1 — worker shape**: a deterministic, zero-key "hello" echo worker (returns a canned/derived completion) so the example runs everywhere with no credentials; the rewritten docs page points to `inference-backend-plugins.md` for wrapping a real SDK. Rationale: cookbook examples must be runnable; the seam being demonstrated is discovery/registration, not OpenAI usage. (Override here if we'd rather wrap the OpenAI SDK.)
+- **D2 — package location**: the plugin package lives inside the example dir, e.g. `examples/c_advanced/using_inference_plugins/hello_inference_plugin/` with its own `pyproject.toml`, installed with `uv pip install -e` (cookbook already uses uv). It is NOT a dependency of the cookbook's root `pyproject.toml` — installing it is the demonstrated step.
+- **D3 — naming**: retire the dangling handle `llm_plugin_example_using_openai`. New names: plugin `hello-inference` (entry-point name `hello_inference`), sdk token `hello`, backend `[hello]`, model handle e.g. `hello-1`. No `pipelex_` prefix anywhere user-facing that belongs to the example.
-### 1c. Always-combine runtime
+## Phases
-- [x] `pipelex/pipe_controllers/parallel/pipe_parallel.py`: unconditional combine in `_live_run_controller_pipe` **and** `_dry_run_controller_pipe` via a shared helper (an existing TODO already asks for this); combination concept taken from `self.output`; `set_new_main_stuff` always called; parallel-combine graph edges always registered.
-- [x] Investigate `final_stuff_code` while in this file (the parallel currently clears it with a copy-pasted `PipeBatch` log line): confirm what it drives (graph/trace identity) — final wiring decision is a Phase 2 item, but capture findings now.
+### Phase 0 — Recon in the cookbook repo
-### 1d. Delete `combined_output` across the language surface
+- [x] Verify how a custom backend + model file wires end-to-end in the cookbook's current `.pipelex/` layout: add a scratch `[hello]` backend + `backends/hello.toml` model with `sdk = "hello"` and confirm the failure mode is `InferenceBackendNotFoundError` for `(llm, hello)` (proves config resolves and the missing piece is exactly the plugin). **DONE** — exact friendly message confirmed: "No inference backend registered for sdk 'hello' in the llm family. Is its plugin installed and enabled?".
+- [x] Confirm whether routing profiles (`.pipelex/inference/routing_profiles.toml`) need an entry for a directly-referenced model handle, or whether `model = { model = "hello-1" }` in `.mthds` bypasses routing. **ANSWERED: routing is NOT bypassed.** `ModelManager.build_deck` routes every known model through the active profile; a DEFAULT match whose backend lacks the model spec **silently drops the model from the deck** (only the `internal` backend is tried as fallback). So the example needs an exact route. We use `optional_routes = { "hello-1" = "hello" }` in the active `all_pipelex_gateway` profile — optional routes only apply when the target backend is enabled, so the entry is inert if `[hello]` is disabled. **This surfaced a real core bug (fixed here, see below).**
+- [x] ~~Check the cookbook's pinned `pipelex` version supports the plugin system~~ — resolved 2026-07-03: the cookbook now sources `pipelex` **editable from this worktree** (`pipelex-cookbook/pyproject.toml` `[tool.uv.sources] pipelex = { path = "../_plugins", editable = true }`), so it runs against this tree's tip. Any pipelex-side change needed for the example is made HERE and is live in the cookbook immediately.
-- [x] `pipelex/pipe_controllers/parallel/pipe_parallel_blueprint.py`: delete field + the one-of-two (`add_each_output`/`combined_output`) validator.
-- [x] `pipelex/pipe_controllers/parallel/pipe_parallel_factory.py`: stop resolving the combined concept separately.
-- [x] `pipelex/builder/pipe/pipe_parallel_spec.py`: delete field + validators; simplify `to_blueprint()`; update pretty-render.
-- [x] `pipelex/builder/operations/pipe_ops.py` + `pipelex/cli/agent_cli/commands/pipe_cmd.py`: remove `combined_output` display/handling code.
-- [x] `pipelex/core/bundles/pipelex_bundle_blueprint.py`: drop the `combined_output` concept-ref collection; **add the new static structure-compatibility check** (design §3).
+### Phase 1 — The plugin package (in `pipelex-cookbook`) — DONE
-### 1e. Fixtures, schema, docs
+- [x] Create `examples/c_advanced/using_inference_plugins/hello_inference_plugin/` with `pyproject.toml`: distribution name `hello-inference-plugin`, entry point `hello_inference = "hello_inference_plugin.hello_plugin:HelloInferencePlugin"` (module path deviates from the plan's `hello_inference_plugin:...` — no-re-exports rule forbids defining the class in `__init__.py`; layout mirrors the builtins' `<name>_plugin.py`), dependency on `pipelex`, hatchling build, `py.typed` marker (needed by the cookbook's mypy).
+- [x] Implement `HelloInferencePlugin` in `hello_inference_plugin/hello_plugin.py`: `name = "hello_inference"`, `targets_api = PLUGIN_API_VERSION`, side-effect-free `register`, worker import deferred into the `_make_hello_llm_worker` closure.
+- [x] Implement `HelloLLMWorker(LLMWorkerAbstract)` in `hello_llm_worker.py`: deterministic `_gen_text` (canned haiku + word-count token usage), `_gen_object` raises `LLMCapabilityError`; capability flags come from the model spec via the base class. (An earlier gotcha — workers had to call `llm_job.llm_job_before_start` themselves or reporting crashed on a `None` duration — was eliminated by folding `LLMWorkerInternalAbstract` into `LLMWorkerAbstract` in core, see below.)
+- [x] Bonus TAKEN: `registrar.add_model_lister(sdk="hello", lister=...)` in `hello_list.py` — reads the models from the backend config (no remote API), lights up `pipelex show models hello`.
-- [x] Migrate test fixtures and data: `tests/integration/pipelex/pipes/controller/pipe_parallel/pipe_parallel_1.mthds`, `tests/e2e/pipelex/pipes/pipe_controller/pipe_parallel/parallel_graph_*.mthds`, `tests/unit/pipelex/pipe_controllers/parallel/{data.py,test_pipe_parallel_blueprint.py}`, `tests/unit/pipelex/builder/pipe/pipe_controller/pipe_parallel/test_data.py`, `tests/unit/pipelex/builder/operations/test_pipe_spec_to_toml.py`, `tests/unit/pipelex/graph/test_mermaidflow.py`, `tests/integration/pipelex/pipeline/test_bundle_validator.py`, `tests/integration/pipelex/cli/test_agent_validate_pipe_in_bundle.py`, `tests/integration/pipelex/pipes/controller/pipe_parallel/test_pipe_parallel_validation.py` (and any others `grep -rln combined_output` surfaces).
-- [x] Regenerate the MTHDS JSON Schema: `.venv/bin/pipelex-dev generate-mthds-schema`.
-- [x] Rewrite `docs/building-methods/pipes/pipe-controllers/PipeParallel.md` to the new semantics — current reality only, no historical narrative.
-- [x] Changelog entry under `[Unreleased]`, marked breaking (delete `combined_output`; `output` is the combination concept; new `native.Composite`).
-- [x] Gates: `make agent-check` green, targeted parallel/builder/graph tests green, full `make agent-test` green.
+### Phase 2 — Config + method files (in `pipelex-cookbook`) — DONE
-### ⛔ CHECKPOINT 1 — MANDATORY STOP
+- [x] `[hello]` backend added to `.pipelex/inference/backends.toml` (enabled, no key), `backends/hello.toml` declares `hello-1` with `sdk = "hello"`, and `optional_routes = { "hello-1" = "hello" }` added to the active `all_pipelex_gateway` profile in `routing_profiles.toml`.
+- [x] `hello_plugin.mthds` references `hello-1`; dangling handle `llm_plugin_example_using_openai` retired.
-Acceptance: parallel always combines (live + dry), `combined_output` fully gone from source and fixtures, `native.Composite` round-trips, static validation catches bad `output` declarations at `/validate` time, all gates green.
+### Phase 3 — End-to-end verification (in `pipelex-cookbook`) — DONE
-- [x] Verify acceptance criteria + full gates (`make agent-check` + full `make agent-test` green).
-- [x] Commit Phase 1 — `757146324` on top of `1b26b3fae`.
-- [x] Update this file (checkpoint log below) + design doc status for cold start.
-- [x] Fan out context-free Sonnet-5 `/code-review` sub-agent on the Phase 1 diff (pointer only).
-- [x] Triage findings: review found NO correctness bugs; duplicated native-concept match extracted to `NativeConceptCode.is_composite` (fixed); double-parse finding deferred → `wip/required-main-stuff/deferred-phase-1-review.md`.
+- [x] `uv pip install -e examples/c_advanced/using_inference_plugins/hello_inference_plugin` (left installed in the cookbook venv).
+- [x] `pipelex plugins list` shows `hello_inference | external | registered | 2 | inference backend llm:hello + model lister hello`.
+- [x] Dry run AND real run green — real run outputs the deterministic haiku, zero keys. `pipelex show models hello` lists `hello-1`.
+- [x] Negative check: after uninstall, run fails with the friendly "No inference backend registered for sdk 'hello' in the llm family. Is its plugin installed and enabled?" — and the DRY RUN still passes without the plugin (worker creation is lazy), so cookbook CI needs nothing installed.
+- [x] Housekeeping: example README written (install → discover → run → failure mode → docs link); cookbook CHANGELOG `[Unreleased]` entry; root README bullet under "Advanced Methods"; `tests/e2e/test_bundles.py` stale special-cases removed (`NEEDS_OPENAI_KEY` and `GHA_DISABLED` both emptied — the example no longer needs a key nor a GHA skip); cookbook `pyproject.toml` mypy `exclude` gains the nested plugin dir (module-name clash: resolve as the installed package, not via `examples/` traversal).
+- Gates: cookbook `make agent-check` + `make agent-test` green; all `tests/e2e/test_bundles.py` dry-run cases pass (hello_plugin auto-discovered, no special-casing). Pipelex-side `make agent-check` green + targeted cogt tests pass.
-## Phase 2 — Enforce the invariant across in-repo surfaces
+**CHECKPOINT A — CLEARED 2026-07-03.** Working end-to-end in the cookbook. NOT committed yet in either repo (user to arbitrate commit slicing; this worktree also has unrelated uncommitted orchestrator-plugins docs).
-Once every pipe stamps, "a completed run has a main stuff" is guaranteed; tighten the defensive branches from optional to direct.
+#### Follow-up DONE (in this worktree): LLM worker family symmetry (fold `LLMWorkerInternalAbstract` → `LLMWorkerAbstract`)
-- [x] Graph tracer output-spec branch in `pipelex/core/pipes/pipe_abstract.py` — remove the defensive branch and its apologetic comment ("main_stuff may not exist for pipes like PipeParallel…").
-- [x] Delivery executor `pipelex/pipe_run/delivery_executor.py`: always write the `main_stuff.*` artifact files for completed runs (kills the "empty envelope" failure mode).
-- [x] CLI run cores: `pipelex/cli/commands/run/_run_core.py` + `pipelex/cli/agent_cli/commands/run/_run_core.py` — drop the "no main output produced" branches for live runs.
-- [x] OTel telemetry attribute extraction: `pipelex/system/telemetry/otel_factory.py`.
-- [x] Wire models go non-optional (`main_stuff_name: str`): `pipelex/runtime_bridge/payloads.py`, `pipelex/runtime_bridge/serialization.py` (`resolve_main_stuff_root_key`), `pipelex/pipeline/pipeline_response.py` (`PipelexRunResultExecute`), `pipelex/cli/agent_cli/commands/run/_run_core_api.py`. Temporal payload history is not a concern — never shipped to prod.
-- [x] Audit every remaining `get_optional_main_stuff` call site: `WorkingMemory.get_optional_main_stuff` itself **stays** (pre-run/empty memory legitimately has none); each post-run boundary call site is either tightened or justified in the checkpoint log.
-- [x] Resolve the **`final_stuff_code`** open question (design §7) using the Phase 1c findings: presumably honor it on the combined stuff like operators do.
-- [x] Gates: `make agent-check` + full `make agent-test` green.
+Decided with the user right after Checkpoint A: `LLMWorkerInternalAbstract` was a remnant of the pre-plugin-system "fake plugin" era, and the base `LLMWorkerAbstract` had a hole in its template method (external workers had to call `llm_job_before_start` themselves or reporting crashed). Fixed structurally, matching the other three families:
-### ⛔ CHECKPOINT 2 — MANDATORY STOP
+- `LLMWorkerAbstract.__init__` now takes `inference_model`; the base owns the lifecycle (`llm_job_before_start`, spec-driven capability checks, constraints, spec-based OTel names). `LLMWorkerInternalAbstract` deleted; all builtin LLM workers re-parented.
+- Legacy `set_llm_worker_from_external_plugin` (manager + protocol) removed — pre-plugin-system path registering spec-less worker classes by handle; its test reworked to drive an out-of-tree `LLMWorkerAbstract` subclass with a real spec.
+- Import-cycle lesson (user cares): `pipelex/hub.py` imports the worker ABCs at module level, and `pipelex.config.get_config` imports the hub — so **nothing in hub's import closure may import `pipelex.config` at module level**. The dump gating that caused this moved to the `llm_generate` funnel; `dump_prompt`/`dump_response_from_text_gen` in `llm_utils` are config-gated internally (llm_utils left hub's closure when the ABC dropped it). NO lazy imports. Pre-existing deeper inversion flagged, not fixed: `config.py → hub` and `configs.py → aws_config.py → hub` mean the config layer sits ON TOP of the hub — a future refactor could move the config singleton below the hub and dissolve this class of cycle for good.
+- Cookbook side: `HelloLLMWorker` shrank to `_gen_text` + `_gen_object` only (the teaching outcome we wanted); breaking changelog entries added in pipelex CHANGELOG `[Unreleased]`.
-Acceptance: no defensive main-stuff branch left for completed live runs; wire `main_stuff_name` non-optional; call-site audit documented; all gates green. Cross-repo work starts fresh from here — this checkpoint's TODOS update is the cold-start context for that session.
+#### Core bug found & fixed during Phase 0 (in this worktree)
-- [x] Verify acceptance criteria + full gates.
-- [x] Commit Phase 2 — `3c3cf4570` on top of `9acbd7a86`.
-- [x] Update this file (checkpoint log: final call-site audit results, `final_stuff_code` decision) + design doc status.
-- [x] Fan out context-free Sonnet-5 `/code-review` sub-agent on the Phase 2 diff (pointer only).
-- [x] Triage findings: ONE confirmed bug (reproduced by the reviewer) — `PipeCondition`'s `continue` outcome doesn't stamp a main stuff, so the tightened tracer epilogue crashed with a raw `WorkingMemoryStuffNotFoundError` whenever a condition continued with no pre-existing main stuff under active tracing. FIXED at the semantic site (see Checkpoint 2 log "Review triage"). All other tightened surfaces judged internally consistent by the review; no simplification findings.
+`RoutingProfileFactory.make_routing_profile` (`pipelex/cogt/model_routing/routing_profile_factory.py`) parsed + validated `optional_routes` from `routing_profiles.toml` but never passed it to the built `RoutingProfile` — TOML optional routes silently did nothing (nothing in the tree used them; zero test coverage). Fixed (one-line pass-through), new test module `tests/unit/pipelex/cogt/model_routing/test_routing_profile_optional_routes.py` (factory pass-through + enabled/disabled gating), CHANGELOG `[Unreleased]` entry added. **Consequence for the cookbook:** its `[Unreleased]` note says the example requires `pipelex` > 0.36.0 — the cookbook-side release must wait for (or pin past) the pipelex release carrying this fix. Related pre-existing wart spotted, NOT fixed: `RoutingProfile.get_backend_match_for_model` mutates `self.routes` in place when merging optional routes (`possible_routes = self.routes` without copy) — harmless today (idempotent merge), flag if it ever bites.
-## Phase 3 — Cross-repo sweep (GATED)
+### Phase 4 — Docs rewrite (in the pipelex repo, this worktree) — DONE 2026-07-03
-**Gate: do not start before Phases 1–2 are merged to `main` and a pipelex version is cut.** Each repo below is its own git repo — commit/PR per its own conventions.
+- [x] Rewrote `docs/cookbook/using-inference-plugins.md`: what a plugin is (entry point + `PipelexPlugin` + registrar), package layout, model-config side (`.pipelex/inference/`, optional route), install + `pipelex plugins list` + run walkthrough, failure mode, links to `under-the-hood/inference-backend-plugins.md`. Stale `.pipelex/pipelex.toml` claim gone; GitHub badge now points at the example dir. Also updated the stale blurb in `docs/cookbook/index.md`.
+- [x] Audited `docs/features/llm-integration.md` — contains no plugin terminology at all (nothing to fix); the stale "plugins" mention was the cookbook page's own link text, now rewritten. Workspace-wide grep: no other legacy "plugin = config entry" language in docs (remaining "plugin" hits are the unrelated Claude Code skills plugin page).
+- [x] Re-checked `docs/under-the-hood/inference-backend-plugins.md` worker section against the folded base: acme example already passes `inference_model` to the worker, no lifecycle gotchas documented — one stale bit found & fixed: lookup-miss error is `InferenceBackendNotFoundError`, not `NotImplementedError` (the doc's own fail-loud table already had it right).
+- [x] `make docs-check` (strict mkdocs build) green.
+- [x] Changelog entry under `[Unreleased]` (brief docs entry under "Changed").
-- [ ] `mthds/` — spec: `mthds-format.md`, `pipes-controllers.md`, `validation-rules.md`, `namespace-resolution.md`, `cross-package-references.md`, `docs/mthds_schema.json` — remove `combined_output`, define always-combine + `native.Composite`.
-- [ ] `vscode-pipelex/` — pinned `mthds_schema.json` in taplo-common; `plxt` lint/format accepts the new shape and rejects `combined_output`.
-- [ ] `conformance/` — `tests/pipelex/test_validate_subcommands.py` fixture updates; keep spec ↔ test links green (`make check-spec-links`).
-- [ ] `mthds-plugins/` — skills references: `mthds-reference.md`, `build-phases.md`, `recursive-cheat-sheet.md`, `mthds-run/SKILL.md` across plugin variants.
-- [ ] `pipelex-app/` — `src/types/core/pipes/pipe_controllers/pipe-parallel.ts`.
-- [ ] `mthds-ui/` — `src/graph/types.ts`: `combined_output_concept` exec-data key (kept as-is in Phase 1 for compat; decide rename here per design §7) + graph-spec data fixtures.
-- [ ] `cocode/` — migrate `ai_instruction_update.mthds`, `swe_docs.mthds`.
-- [ ] `pipelex-website/` — `docs/mthds-doc.md`.
-- [ ] Demo/workshop repos — `illustration_generator/bundle.mthds` copies.
-- [ ] SDKs (`pipelex-sdk-js/`, `mthds-python/`) + `pipelex-starter-python/` — tighten `main_stuff` handling for completed runs; check `PipeOutputAbstract` (mthds-python) for encoded main-stuff optionality (design §7). Full `find_main_content` cleanup also needs the companion track — do only the invariant-side tightening here.
-- [ ] `pipelex-temporal/` (private) — adapt to the split orchestrator SPI: move the `FIRE_AND_FORGET` arm out of `match delivery` into `start(...)` returning `PipelexPipeDispatchAck`; `run(...)` keeps only the blocking arm (no `delivery` param, no `is_completed`). Same check for `pipelex-mistralai-workflows/` if it registers an orchestrator.
-- [ ] `pipelex-api/` — `/execute` calls `orchestrator.run(...)` without `delivery`; `/start` calls `orchestrator.start(...)` and reads the ack's `workflow_id` (today it reads `run_output.workflow_id` off a degenerate `PipelexPipeRunOutput`).
+**CHECKPOINT B (final)** — Phase 4 work all done; **only the commits remain** (user to approve the slices from "NEXT SESSION" step 1, plus these Phase 4 docs edits which ride with the tracker/docs concern or their own docs slice). Once committed in both repos, note the commit SHAs here and retire/archive this tracker. Release ordering reminder (step 3 above) still applies before shipping either side.
-### ⛔ CHECKPOINT 3 — MANDATORY STOP
+## Cross-repo map
-Acceptance: every consumer repo aligned with the released pipelex version; conformance + spec-links green; no repo still references `combined_output`.
+| Repo | Work |
+|---|---|
+| `pipelex-cookbook` | plugin package, backend/model config, `.mthds`, run verification (Phases 0–3) |
+| `pipelex` (this worktree `_plugins`) | docs page rewrite + terminology audit (Phase 4), plus any core change the example surfaces |
-- [ ] Verify per-repo test suites / checks.
-- [ ] Update this file; close out or fold residual items into follow-up trackers.
-- [ ] Fan out context-free Sonnet-5 `/code-review` sub-agents per changed repo (pointer only — the repo's diff/SHA range).
-- [ ] Triage findings; then close this track (companion track remains open, separate design).
+**Editable-pin caveat**: the cookbook's `[tool.uv.sources]` editable pin on `../_plugins` is a local dev convenience (set by the user 2026-07-03) and must NOT ship: before merging/releasing the cookbook side, flip back to a PyPI `pipelex==X.Y.Z` pin carrying whatever core changes this work needed (same playbook as the pipelex-api editable-pin precedent — editable local paths break CI). If the example needs no core change, the flip-back is to the current release.
-## Open questions (from design §7 — resolve during implementation)
+## Gotchas for a cold start
-- [x] `final_stuff_code`: RESOLVED in Phase 2 — the parallel honors it on the combined stuff (see Checkpoint 2 log).
-- [ ] Graph `execution_data` key `combined_output_concept`: keep name in Phase 1 for `mthds-ui` compat; rename decision in Phase 3 (`mthds-ui` row).
-- [ ] `add_each_output` naming: recommendation is **keep** (pure churn otherwise); revisit only if MTHDS spec editors want it (Phase 3, `mthds/` row).
-- [ ] `PipeOutputAbstract` (mthds-python): check for encoded optionality (Phase 3, SDKs row).
-
-## Checkpoint log
-
-*(Filled in at each checkpoint — decisions taken, deviations from design, current code state, audit results. This section is the cold-start context for a fresh session.)*
-
-### Checkpoint 1 — reached (Phase 1 complete)
-
-**Code state:** Phase 1 fully implemented on `feature/Required-main-stuff`; one commit carries the whole phase (see git log). `make agent-check` green; full `make agent-test` green.
-
-**What landed:**
-
-- `native.Composite`: `NativeConceptCode.COMPOSITE` + `CompositeContent(StuffContent)` (`pipelex/core/stuffs/composite_content.py`, pydantic `extra="allow"`), wired into `ConceptFactory.make_native_concept`, `NativeConceptCode.structure_class`, and `CoreRegistryModels.STUFF`. Custom `rendered_markdown`/`rendered_html` iterate the named components; a `components` property exposes `model_extra`.
-- Always-combine runtime: `PipeParallel._run_branches_and_combine` is the shared live/dry helper (resolves the old TODO); combine concept comes from `self.output`; `set_new_main_stuff` and PARALLEL_COMBINE graph edges are unconditional. `combined_output` field deleted from runtime model, blueprint, factory, spec, pipe_ops/pipe_cmd display, and bundle-blueprint concept-ref collection.
-- Static validation, two layers: (1) parse-time in `PipeParallelBlueprint.validate_output` — rejects multiplicity suffixes and native non-`Composite`/`Dynamic`/`Anything` outputs; mirrored on the runtime model in `PipeParallel.validate_output_static` for direct constructions; (2) library-time in `PipeParallel.validate_output_with_library` — structure-compatibility check (required fields ⊆ result names; result names ⊆ declared fields; `CompositeContent` subclasses skip). Both feed `/validate` (bundle parse + `Library.validate...` → `pipe.validate_with_libraries()`).
-
-**Decisions / deviations from the plan:**
-
-- The TODOS 1d line said to put the structure-compatibility check in `pipelex_bundle_blueprint.py`; it lives on `PipeParallel.validate_output_with_library` instead, because the concept's structure class (registered Python class, generated class, refines chain) is only resolvable against the loaded library — the bundle blueprint cannot see class-backed structures. Design §3 says "bundle/library validation time", which this satisfies.
-- kajson round-trip needed a `__json_encode__` hook on `CompositeContent`: pydantic stores extras outside `__dict__`, so kajson's default `__dict__` fallback silently dropped every component. The hook hands kajson the live components so nested contents keep their class metadata (`# noqa: PLW3201` — kajson protocol name).
-- A description-only concept as a parallel output is rejected by the field-compat check naturally (its generated structure class is TextContent-derived, requiring `text`), with a message suggesting `Composite` — test pins this.
-- Graph `execution_data` key `combined_output_concept` kept (now always set = declared output concept ref) for `mthds-ui` compat, per design §7; rename decision deferred to Phase 3.
-- `tests/e2e/.../cv_job_match.mthds` had two parallels with now-invalid outputs (`Page[]`, `Text`) — migrated to `Composite` (was not on the plan's fixture list).
-- Mermaidflow unit test's IOSpec name `combined_output` renamed `combined_result` (terminology hygiene only — it was never the field).
-
-**`final_stuff_code` findings (1c investigation, wiring decision = Phase 2):**
-
-- Producer: only `PipeBatch` sets it — a pre-allocated per-branch stuff code (`branch_output_item_code`) so the batch's aggregated `ListContent` item identity matches the branch's final output stuff in the graph.
-- Consumers: `PipeLLM` and `PipeStructure` pass it as `code=` to `StuffFactory.make_stuff` for their output stuff. Other operators do not honor it (pre-existing gap worth noting in Phase 2).
-- `PipeSequence` threads it to the last step only (clears for earlier steps); `PipeBatch` and `PipeParallel` clear it on entry.
-- Implication: a `PipeParallel` as a `PipeBatch` branch drops the requested code, so the batch item's graph identity doesn't match the parallel's combined stuff. Now that the parallel always stamps, Phase 2 should honor `final_stuff_code` on the combined stuff (pass `code=` through `combine_stuffs` → `make_stuff`). A `TODO(Phase 2)` marks the spot in `_live_run_controller_pipe`.
-
-**New/updated tests:** `tests/unit/pipelex/core/stuffs/test_composite_content.py` (round-trips incl. transport dump→hydrate), `tests/unit/pipelex/pipe_controllers/parallel/` (blueprint accept/reject + data.py), `tests/integration/.../pipe_parallel/test_pipe_parallel_always_combines.py` (+ fixture `parallel_always_combine.mthds`; stale-stamp regression + terminal-parallel main stuff; dry parity via `pipe_run_mode` parametrization), `tests/integration/.../pipe_parallel/test_pipe_parallel_output_validation.py` (library-level accept/reject via `acquire_library`). e2e graph expectations updated: the add_each-only graph now expects `parallel_combine: 2` edges.
-
-### Checkpoint 2 — reached (Phase 2 complete)
-
-**Code state:** Phase 2 fully implemented on `feature/Required-main-stuff`; commit `3c3cf4570` on top of `9acbd7a86`. `make agent-check` green; full `make agent-test` green (all tests passing). TDD followed: red tests written and confirmed failing before each behavioral change.
-
-**What landed (invariant enforcement, per-surface):**
-
-- **Graph tracer epilogue** (`pipe_abstract.py` `_run_pipe_traced` success path): `get_main_stuff()` direct read; `output_spec` built unconditionally; the `main_stuff is not None` gate on `output_concept_data` removed. The apologetic comment is gone.
-- **Delivery executor** (`generate_result_files`): typed path reads `get_main_stuff()` (raises `WorkingMemoryStuffNotFoundError` when absent); raw path raises `PipeJobError` when the raw dump lacks a main stuff. The `try_local_hydrate_stuff` → raw-render fallback is unchanged (it's about local class availability, not missing main stuff). A missing main stuff surfaces as `StorageDeliveryError` at the `_store_results` boundary — the same handling path as any storage failure, so the caller's failure-webhook behavior is unchanged.
-- **CLI run cores:** human core pretty-prints/saves/CSV-exports via `main_stuff` directly (the "--save-csv: no main stuff produced" exit branch is deleted); agent core builds `main_stuff_json` + `compact_result` unconditionally; agent API-runner core raises `PipeExecutionError` when a completed response has no main stuff under the announced key (the `main_stuff_name`-extension *name* fallback to `MAIN_STUFF_NAME` is kept — that's protocol-extension handling, not invariant softening).
-- **OTel** (`make_output_json`): direct read; the `"{}"` fallback is gone (called on the success path only).
-- **Wire models:** `PipelexPipeRunOutput.main_stuff_name: str` (required) and `PipelexRunResultExecute.main_stuff_name: str`. `resolve_main_stuff_root_key` returns `str` and raises `PipeJobError` on a memory with no main stuff; `PipelexRunResultExecute.from_pipe_output` now uses it (previously `aliases.get(MAIN_STUFF_NAME, MAIN_STUFF_NAME)` — which could announce a key that wasn't actually in `root`).
-- **`PipeOutput.optional_main_stuff` DELETED** (breaking, in changelog). Workspace grep: only external consumer is `pipelex-demo-vibe` examples → Phase 3 sweep list.
-
-**`final_stuff_code` resolution (design §7):** the parallel now honors it on the combined stuff. Implementation: `_run_branches_and_combine` captures `pipe_run_params.final_stuff_code` and clears it *before* branch fan-out (branches deep-copy the params, so they must not inherit the parent's requested code), then passes it as the new `code=` param on `StuffFactory.combine_stuffs` → `make_stuff`. Live/dry parity for free (shared helper). Pinned by `test_pipe_parallel_final_stuff_code.py` (combined stuff carries the code; branch outputs don't). **Noted, not addressed (pre-existing gaps, out of scope):** only `PipeLLM`/`PipeStructure` honor `final_stuff_code` among operators; `PipeBatch` still clears it on entry rather than stamping its aggregate list with it.
-
-**Call-site audit (final state):** `grep get_optional_main_stuff` across `pipelex/` + `tests/` → the definition in `working_memory.py` (kept: pre-run/empty memory legitimately has none; public API for SDK-side pre-run checks) plus ONE justified in-source caller added by the review triage: `PipeCondition`'s pre-delivery checks (a mid-run "is there something to pass through / did anything get delivered" probe — legitimately optional at that point, converted into a loud `PipeRunError` when absent). Every post-run boundary call site was tightened.
-
-**Review triage (Checkpoint 2):** The context-free review found and reproduced one real bug: the Phase 1 premise "every pipe stamps" was FALSE for `PipeCondition`'s `continue` outcome (live path returned the memory untouched) and for the dry run of an all-special-outcomes condition — so the tightened tracer epilogue (which fires on EVERY pipe node) crashed with a raw `WorkingMemoryStuffNotFoundError` on any continue-with-no-prior-main-stuff under tracing. Resolution (design-consistent, not a defensive revert): `continue` IS pass-through delivery — the delivered main stuff is the one already in memory; with nothing to pass through the pipe now raises a clear actionable `PipeRunError` at the semantic site ("map the outcome to a pipe, or place this condition after a producing step"), in both live and dry paths. Pinned by `tests/integration/.../pipe_condition/test_pipe_condition_continue_delivery.py` (pass-through identity + both fail-loud cases; run params constructed directly so keyless forced-DRY coercion can't swap the exercised path). Suite gap noted by the reviewer (nothing drove continue-with-no-main-stuff under tracing) is now covered. The reviewer confirmed the parallel `final_stuff_code` capture/clear mirrors `PipeBatch`'s existing pattern, and `combine_stuffs(code=)` has no empty-string/double-stamp edge. Related known-shaky area: `test_pipe_condition_continue_output_type.py` stays xfail (separate pre-existing bug, batch-over-continue aggregation semantics — untouched by this fix since batch branches always have a main stuff stamped).
-
-**Test changes:** new `tests/integration/.../pipe_parallel/test_pipe_parallel_final_stuff_code.py`; `test_output_serialization.py` extended (aliased/direct root-key resolution + missing-main-stuff raises); `test_input_models.py` pins `main_stuff_name` required; `test_delivery_executor.py` mocks now carry a real main stuff + two new missing-main-stuff raise tests + `main_stuff.*` files asserted stored; `test_run_core_execution.py` obsolete `test_save_csv_no_main_stuff_exits` deleted; `test_run_pipe_tracer_metadata.py` + `test_run_cost_report.py` mock outputs made invariant-compliant; `test_mock_usage_direct.py` tightened.
-
-**Cross-repo consequences flagged for Phase 3:** `main_stuff_name` is now required on the `/execute` response extension and `PipelexPipeRunOutput` — the closed `pipelex-transport` library and any consumer parsing these payloads must be checked; `pipelex-demo-vibe` uses the deleted `optional_main_stuff`.
-
-### Between checkpoints — PR opened toward the Phase 3 gate
-
-- Merged `origin/dev` into the branch (`fc96c1d96`; only conflict = both sides' `[Unreleased]` changelog sections, combined into unified Added/Changed/Fixed). Post-merge `make agent-check` green; full `make agent-test` re-run on the merged tree.
-- PR **#1014** opened against `dev`. Phase 3 stays gated until the PR is merged and a pipelex version is cut.
-- **Review round 1 (codex bot, CONFIRMED REAL):** requiring `main_stuff_name` on `PipelexPipeRunOutput` broke the fire-and-forget arm — `pipelex-temporal`'s `FIRE_AND_FORGET` branch constructs the payload with `main_stuff_name=None, is_completed=False` (its only not-completed constructor anywhere). Resolution chosen by the user (NOT the "make it optional again" convenient fix, which would re-spread null-guards): **split the delivery axis into the type system.** `OrchestratorProtocol` now has `run(...) -> PipelexPipeRunOutput` (blocking, completed, `main_stuff_name` stays required) and `start(...) -> PipelexPipeDispatchAck` (new ids-only ack model: `pipeline_run_id` + `workflow_id`); the `delivery` param is gone from the SPI (`DeliveryMode` stays as wire-input vocab picking the method); `PipelexPipeRunOutput.is_completed` DELETED — nothing anywhere read it. DIRECT implements `start` by raising `PipelexBridgeDispatchError` (unreachable behind the `supports_fire_and_forget` gate). Docs (`orchestrator-plugins.md`) + changelog updated; new Phase 3 rows added for `pipelex-temporal` and `pipelex-api`.
-- **Review round 2 (`4ee873fe4`, pushed):** all remaining bot threads (codex ×2, greptile ×1, cubic ×5) confirmed real, fixed, replied and resolved in-thread — 0 unresolved threads on the PR. (1) `PipeParallelSpec.add_each_output` defaults to `false` like the blueprint (always-combine specs can omit it). (2) `validate_output_with_library` now also checks branch output **types** against structured field content classes — compatibility is conceptual (branch structure class IS the field class or a subclass; refinement passes since refining concepts generate inheriting classes — user's explicit call), matching exactly what pydantic accepts in `combine_stuffs`; conservative skips: non-content-class annotations (primitive fields from `.mthds` structures), list-producing branches. Gotcha pinned in-code: the `type: Literal[...]` field shadows the builtin in the class body, so the method signature needs `builtins.type[StuffContent]`. (3) **Greptile P1 (real):** `dump_for_transport()` lost Composite component types (plain dicts after hydration; the existing round-trip test only compared `smart_dump()`, hence the miss) — components are now marker-stamped (`__pipelex_class__`/`__pipelex_module__`, ListContent components as plain lists of stamped item dicts, mirroring the top-level convention) and `hydrate_content` rebuilds typed components; regression tests assert component *types* incl. a ListContent component. (4) HTML value rendering deduped into shared `pipelex/core/stuffs/html_rendering.render_value_html()` used by `StructuredContent` + `CompositeContent`. (5) wip §4 reworded implemented-not-proposal. Changelog + `PipeParallel.md` doc updated; `make agent-check` + full `make agent-test` green.
-- **Review round 3 (`/review` sweep, own critical pass + testing/maintainability/adversarial sub-agents):** scope CLEAN, no bug bots missed on the shipped path. **Applied in-PR:** `DirectOrchestrator.start()` raise-path test (`tests/unit/pipelex/runtime_bridge/test_direct_orchestrator.py` — `supports_fire_and_forget is False` + `start()` raises `PipelexBridgeDispatchError`); gates green. **Deferred to `wip/required-main-stuff/deferred-review-round-3.md`** (per user, keep PR frozen — none reachable in prod): **F1** nested-`CompositeContent` transport is decode-asymmetric (inner composite components decay to raw dicts and leak `__pipelex_*` markers on the Temporal round-trip — real silent corruption but Temporal not shipped; fix before hosted-Temporal rollout: recurse in `_hydrate_list_item` for CompositeContent subclasses + nested test); **F2** `ConceptValueError` (bare `ValueError`, not `PipelexError`) from `get_structure_class()` in `validate_output_with_library` escapes the validate sweep's catches → whole-bundle abort vs per-pipe FAILURE (low reach; fix: re-raise as `PipeValidationError`); **M1** `rendered_html` table-wrapper + empty-literal still copy-pasted across Composite/Structured (extract `render_named_values_table`). Minor noted: `combined_output_concept` key rename stays a Phase-3 mthds-ui-coordinated item; unguarded `get_required_pipe` consistency nit; `from_pipe_output` non-COMPLETED latent; duplicate-result-name stays a runtime guard.
-
-### Checkpoint 3 — not reached
+- `register` must stay side-effect-free and import-light; anything heavy goes inside `make_worker`. Discovery runs `build_registrar` at boot AND in `pipelex plugins list`.
+- `targets_api` must equal `PLUGIN_API_VERSION` (2) or discovery fails loud with `PluginApiVersionMismatchError`.
+- The keyword-only-arguments convention is enforced on `pipelex/` source only, but the example package should follow it anyway — it's teaching material.
+- Worker subclass signatures must match `LLMWorkerAbstract` exactly (use `@override`); the reporting/telemetry plumbing is inherited, don't reimplement it.
+- Don't edit `docs/errors/` pages by hand (generated); nothing here should need new error classes anyway.
+- Stale-terminology precedent: on 2026-07-03 we already fixed "CLI-build command harvest" leftovers in `contract.py`, `discovery.py`, and `inference-backend-plugins.md` — same spirit applies to any old "LLM plugin" config-speak found during Phase 4.
diff --git a/docs/cookbook/index.md b/docs/cookbook/index.md
index 4747c9dbb..dbf4970a2 100644
--- a/docs/cookbook/index.md
+++ b/docs/cookbook/index.md
@@ -38,7 +38,7 @@ We strongly encourage you to clone the [Pipelex Cookbook repository](https://git
- [**Synthetic Data**](./gen-synthetic-data.md) — Generate diverse synthetic data samples from a description.
- [**Expense Data with Receipts**](./gen-expense-data.md) — Generate complete expense datasets with receipt images and fraud scenarios.
-- [**Using Inference Plugins**](./using-inference-plugins.md) — Route model calls through custom provider configurations.
+- [**Using Inference Plugins**](./using-inference-plugins.md) — Install an inference-backend plugin and run a method against the model it serves, with zero API keys.
## Work in Progress
diff --git a/docs/cookbook/using-inference-plugins.md b/docs/cookbook/using-inference-plugins.md
index 0f3093130..779fae655 100644
--- a/docs/cookbook/using-inference-plugins.md
+++ b/docs/cookbook/using-inference-plugins.md
@@ -1,26 +1,73 @@
---
title: "Using Inference Plugins Example"
-description: "Use custom LLM inference plugins in Pipelex to route model calls through your own provider configuration."
+description: "Install a Pipelex inference-backend plugin, watch discovery register it, and run a method against the model it serves — no API key needed."
---
# Example: Using Inference Plugins
-This example shows how to use an LLM inference plugin to route model calls through a custom provider configuration instead of the default Pipelex gateway.
+This example shows how Pipelex discovers **inference-backend plugins**: installable Python packages that teach Pipelex how to serve models for a given `sdk` token. The cookbook ships a complete, minimal plugin — a deterministic "LLM" that always answers with the same haiku — so the whole flow runs anywhere with no API key and no network.
## Get the code
-[![GitHub](https://img.shields.io/badge/View_on_GitHub-5a0dad?logo=github&logoColor=white&style=flat)](https://github.com/Pipelex/pipelex-cookbook/blob/main/examples/c_advanced/using_inference_plugins/hello_plugin.mthds)
+[![GitHub](https://img.shields.io/badge/View_on_GitHub-5a0dad?logo=github&logoColor=white&style=flat)](https://github.com/Pipelex/pipelex-cookbook/tree/main/examples/c_advanced/using_inference_plugins)
## What it demonstrates
-- Referencing an LLM plugin by name in the `model` field
-- How plugins allow you to use your own API keys and provider configuration
-- Minimal change from a standard `PipeLLM` pipe
+- What a plugin **is**: a package exposing a `pipelex.plugins` entry point that resolves to a `PipelexPlugin` — an object with a `name`, a `targets_api` version, and a side-effect-free `register(registrar)` method
+- Registration through the registrar: `register` contributes an inference backend for `(family="llm", sdk="hello")`, plus an optional model lister
+- Discovery by presence: installing the package is the whole integration — no enable-list, no config switch
+- The model-config side: how a model handle in a `.mthds` file resolves to the plugin's worker through `.pipelex/inference/`
+- The failure mode when the plugin is missing — a loud, friendly error, not a silent fallback
+
+## The plugin package
+
+The plugin lives inside the example directory as its own installable package:
+
+```
+examples/c_advanced/using_inference_plugins/
+├── hello_plugin.mthds # the method that uses the plugin-served model
+└── hello_inference_plugin/ # the plugin package
+ ├── pyproject.toml # declares the `pipelex.plugins` entry point
+ └── hello_inference_plugin/
+ ├── hello_plugin.py # HelloInferencePlugin: name, targets_api, register()
+ ├── hello_llm_worker.py # HelloLLMWorker(LLMWorkerAbstract): the actual "model"
+ └── hello_list.py # optional lister behind `pipelex show models hello`
+```
+
+The entry point in `pyproject.toml` is the single line that wires everything:
+
+```toml
+[project.entry-points."pipelex.plugins"]
+hello_inference = "hello_inference_plugin.hello_plugin:HelloInferencePlugin"
+```
+
+The plugin class registers what it serves — and nothing else happens at registration time (no I/O, no client construction; heavy work belongs inside the worker factory):
+
+```python
+class HelloInferencePlugin:
+ name = "hello_inference"
+ targets_api = PLUGIN_API_VERSION
+
+ def register(self, registrar: PluginRegistrar) -> None:
+ registrar.add_inference_backend(family=InferenceFamily.LLM, sdk="hello", make_worker=_make_hello_llm_worker)
+ registrar.add_model_lister(sdk="hello", lister=_list_hello_models)
+```
+
+The worker subclasses `LLMWorkerAbstract` and only implements the generation itself — the base class owns the job lifecycle, capability checks, usage reporting, and telemetry. Here it deterministically returns a haiku; a real plugin would call a remote model at this exact spot.
+
+## The model-config side
+
+Model configuration lives in the project's `.pipelex/inference/` directory (not in `.pipelex/pipelex.toml`). Three pieces connect a `.mthds` model handle to the plugin:
+
+- `backends.toml` declares the `[hello]` backend (enabled, no API key)
+- `backends/hello.toml` declares model `hello-1` with `sdk = "hello"` — the token that selects the plugin's worker factory
+- `routing_profiles.toml` routes `hello-1` to the `hello` backend via an `optional_routes` entry, which only applies while the target backend is enabled
## The Method: `hello_plugin.mthds`
```toml
domain = "hello_plugin"
+description = "Using a model served by an inference-backend plugin"
main_pipe = "hello_plugin"
[pipe]
@@ -28,26 +75,50 @@ main_pipe = "hello_plugin"
type = "PipeLLM"
description = "Write text about Hello World."
output = "Text"
-model = { model = "llm_plugin_example_using_openai", temperature = 0.5 }
+model = { model = "hello-1", temperature = 0.5 }
prompt = """
Write a haiku about Hello World.
"""
```
-The key difference from the standard [Hello World](./hello-world.md) example is the `model` field: instead of using a default gateway model, it references `llm_plugin_example_using_openai` — a plugin defined in your project's configuration.
+The only difference from the standard [Hello World](./hello-world.md) example is the `model` field: `hello-1` is served by the plugin instead of a gateway backend.
## How to run
-1. Configure the LLM plugin in your project's `.pipelex/pipelex.toml` (see the [Configuration](../features/configuration.md) docs for details).
+1. Install the plugin package (this is the step being demonstrated):
+
+ ```bash
+ uv pip install -e examples/c_advanced/using_inference_plugins/hello_inference_plugin
+ ```
-2. Run the example:
+2. Check that Pipelex discovered it:
+
+ ```bash
+ pipelex plugins list # hello_inference | external | registered
+ pipelex show models hello # lists hello-1
+ ```
+
+3. Run the method — no credentials needed:
```bash
pipelex run bundle examples/c_advanced/using_inference_plugins/hello_plugin.mthds
```
+ The output is the plugin's deterministic haiku.
+
+## The failure mode
+
+Uninstall the package and run again:
+
+```bash
+uv pip uninstall hello-inference-plugin
+pipelex run bundle examples/c_advanced/using_inference_plugins/hello_plugin.mthds
+```
+
+Pipelex fails loud at worker-creation time: `No inference backend registered for sdk 'hello' in the llm family. Is its plugin installed and enabled?` — the model config still resolves; only the worker factory is missing. A dry run still passes without the plugin, because worker creation is lazy.
+
## Related Documentation
-- [LLM Integration](../features/llm-integration.md) - Overview of LLM integration and plugins
-- [Configuration System](../features/configuration.md) - How to configure providers and plugins
+- [Inference Backend Plugins](../under-the-hood/inference-backend-plugins.md) - The plugin seam and the Inference SPI, including how to wrap a real SDK (dependency guards, client memoization)
+- [Configuration System](../features/configuration.md) - How `.pipelex/` project configuration works
- [Pipelex Gateway & Model Access](../features/gateway.md) - Default model access through the gateway
diff --git a/docs/under-the-hood/inference-backend-plugins.md b/docs/under-the-hood/inference-backend-plugins.md
index 77074607c..0e641a325 100644
--- a/docs/under-the-hood/inference-backend-plugins.md
+++ b/docs/under-the-hood/inference-backend-plugins.md
@@ -28,7 +28,7 @@ run time (e.g. LLMWorkerFactory.make_llm_worker)
└─ worker = make_worker(inference_model=…, backend=…, sdk_clients=…, reporting_delegate=…)
```
-The worker factories (`LLMWorkerFactory`, `ImgGenWorkerFactory`, `ExtractWorkerFactory`, `SearchWorkerFactory`) hold **no** `match` over SDK strings. They build a `ModelHandle`, resolve the `InferenceBackend` config, look up the backend's `make_worker` by `(family, sdk)`, and call it. A lookup miss raises a friendly `NotImplementedError` ("… Is its plugin installed?").
+The worker factories (`LLMWorkerFactory`, `ImgGenWorkerFactory`, `ExtractWorkerFactory`, `SearchWorkerFactory`) hold **no** `match` over SDK strings. They build a `ModelHandle`, resolve the `InferenceBackend` config, look up the backend's `make_worker` by `(family, sdk)`, and call it. A lookup miss raises a friendly `InferenceBackendNotFoundError` ("… Is its plugin installed and enabled?").
---
@@ -43,7 +43,7 @@ class PipelexPlugin(Protocol):
def register(self, registrar: PluginRegistrar) -> None: ...
```
-`register` is the **only** method core calls, and it is **side-effect-free**: it may call the registrar's menu methods and nothing else — no hub access, no I/O, no client/SDK construction. This is what makes `build_registrar` safe to run more than once (it runs at boot, and again at CLI-build to harvest plugin-contributed commands).
+`register` is the **only** method core calls, and it is **side-effect-free**: it may call the registrar's menu methods and nothing else — no hub access, no I/O, no client/SDK construction. This is what makes `build_registrar` safe to run more than once (it runs at boot, and again whenever the `pipelex plugins list` diagnostic command re-discovers plugins to print what each contributed).
`targets_api` is checked against `PLUGIN_API_VERSION`. A mismatch fails loud with `PluginApiVersionMismatchError` — a single coarse integer gate, not semver-range matching.
diff --git a/docs/under-the-hood/orchestrator-plugins.md b/docs/under-the-hood/orchestrator-plugins.md
index 248c2acf5..879beb20f 100644
--- a/docs/under-the-hood/orchestrator-plugins.md
+++ b/docs/under-the-hood/orchestrator-plugins.md
@@ -60,6 +60,44 @@ A token with no registered orchestrator (its plugin is not installed) raises `Mi
---
+## Bundle validators: the `/validate` counterpart
+
+`/validate` rides the same open-token seam as `run`. A **bundle validator** produces a validation verdict for one orchestration mode, the way an orchestrator runs a pipe for one mode: the core `direct` plugin contributes the in-process validator, and an external orchestrator plugin contributes a worker-dispatched validator under its own token — which core never names. A host runtime resolves the requested mode and dispatches through the `BundleValidatorRegistry` on the hub instead of branching on a backend.
+
+A validator satisfies `BundleValidatorProtocol` (`pipelex/plugins/bundle_validator_registry.py`):
+
+```python
+class BundleValidatorProtocol(Protocol):
+ async def validate_bundles(
+ self,
+ *,
+ mthds_contents: list[str],
+ mthds_sources: list[str] | None,
+ allow_signatures: bool,
+ library_dirs: "Sequence[Path] | None",
+ ) -> BundleValidationVerdict: ...
+```
+
+The `library_dirs` annotation is a quoted forward reference because `Sequence` and `Path` are imported under `TYPE_CHECKING` in `bundle_validator_registry.py`; a plugin that imports them at module scope can use the unquoted form.
+
+Two contract points distinguish it from `OrchestratorProtocol.run`:
+
+- **No fire-and-forget arm.** Validation is inherently blocking, so — unlike orchestration's `run`/`start` split — there is no `start` counterpart and no `supports_fire_and_forget` capability flag.
+- **Verdict-as-value, not raise.** `validate_bundles` *returns* the verdict — `BundleValidationVerdict` is the union of the valid arm (a `ValidationReport`) and the invalid arm (an `ErrorReport` carrying `validation_errors`) — and raises only for a no-verdict infra fault, which a host runtime maps to a 5xx. This is the same valid/invalid pair the API maps onto its 200-always `/validate` wire, so the verdict contract is backend-independent.
+
+The seam is deliberately typed at the MTHDS-protocol level (`ValidationReport` from `mthds.protocol`), not the concrete `PipelexValidationReport` envelope: the concrete report's module reaches the hub, so naming it from this hub-reachable seam would close an import cycle, and the seam is generic across orchestrators (language-standard altitude), so it speaks the protocol report — the Pipelex-runtime envelope is the concrete `ValidationReport` subtype the validators actually produce, and the API recovers that precise type at its edge. `library_dirs` is host context the in-process arm needs to load the method library; a worker-dispatched arm ignores it — its worker loads its own library.
+
+A plugin contributes one per token, right next to its orchestrator:
+
+```python
+registrar.add_orchestrator(mode=DIRECT_ORCHESTRATION_MODE, orchestrator=DirectOrchestrator())
+registrar.add_bundle_validator(mode=DIRECT_ORCHESTRATION_MODE, validator=DirectBundleValidator())
+```
+
+The failure modes mirror the orchestrator seam exactly: a duplicate token fails loud at discovery (`DuplicateBundleValidatorError`, naming both plugins), and a lookup miss raises `MissingBundleValidatorError` — the same generic, plugin-decoupled message shape as `MissingOrchestratorError` (*"is its plugin installed?"*), with the same `"direct"` special case reporting a boot/discovery fault.
+
+---
+
## HTTP error mappers: rendering an orchestrator's transport faults
An orchestrator's *runtime* (a Temporal client, a Mistral workflow runner) raises SDK-specific transport faults — a server unreachable, a workflow timeout — that a host runtime serving HTTP must turn into a proper error response, not a catch-all 500. But core names no web framework and the SDK lives only in the plugin, so the host (`pipelex-api`) cannot itself know how to classify `temporalio.TemporalError`.
@@ -97,11 +135,16 @@ if registrar.config.plugins.boot_orchestrator == self.name:
registrar.claim_task_manager(_setup_temporal_task_manager)
registrar.claim_pipe_router(_make_temporal_pipe_router)
registrar.claim_pipe_run(_make_temporal_pipe_run)
+ registrar.claim_isolated_execution_probe(_temporal_isolated_execution_probe)
registrar.add_teardown(_teardown_temporal)
```
Each `claim_*` takes a **thunk** (a zero-arg factory), never a constructed instance. The thunk runs only at the boot apply-point, so `register` itself imports no `temporalio` — even on a worker. This is the deferred-thunk rule that keeps the import-light invariant intact at boot.
+### The isolated-execution-probe slot
+
+The first four slots swap in the runtime's implementations of core execution services. The fifth, `claim_isolated_execution_probe`, is different in kind: its thunk resolves not a service but an **ambient predicate** (`Callable[[], bool]`) reporting whether the current call runs inside an isolated sub-execution — a Temporal activity — whose side-effecting emissions must not be written into the parent run's replay-deterministic buffer. `ReportingManager` consults it (`hub.is_in_isolated_execution()`) to route an activity-side usage emission to the per-process log instead of the workflow's registered buffer. It exists as a hub slot for the same reason the others do: only a boot-orchestrator plugin whose runtime has a replay/activity split knows how to answer, and core names no such runtime. Unclaimed (any in-process boot), the hub's core default answers "never isolated" — the in-process orchestrator has no such split.
+
Because the gate is a name-match, `plugins.boot_orchestrator` must name a plugin that actually registered. After discovery, `Pipelex.setup` rejects a `boot_orchestrator` that no registered plugin carries — a typo or a missing plugin (e.g. `--orchestrator temporal` without `pipelex-temporal` installed) raises `UnknownBootOrchestratorError` instead of silently running in-process: nothing would claim the hub slots, so the process would otherwise fall through to the core defaults and execute on the wrong runtime. The check matches against **plugin names** (the same namespace the gate uses), not the `orchestration_mode` registry — a plugin's name and the token(s) it serves are separate namespaces that *may* differ, even where a shipped plugin keeps them identical (`pipelex-temporal` is named `temporal` and serves `temporal`). The error names no specific plugin, keeping core decoupled.
### Injection precedence
@@ -112,7 +155,7 @@ At each ordered hub slot, `Pipelex.setup` resolves in this precedence:
2. a plugin slot-claim thunk;
3. the core default.
-A slot claim must never silently override an explicit injection. Teardown runs the plugin-registered teardown callbacks **LIFO**, before core teardown, so a worker's in-flight runtime resources release first.
+A slot claim must never silently override an explicit injection. Two slots take no explicit `setup()` parameter and so skip step 1: the task manager (no core default either — unclaimed means no task manager wiring at all) and the isolated-execution probe (whose core default is the always-False predicate above). Teardown runs the plugin-registered teardown callbacks **LIFO**, before core teardown, so a worker's in-flight runtime resources release first.
---
@@ -140,7 +183,7 @@ What an out-of-tree orchestrator imports *is* a contract. The SPI is a documente
| Boundary serialization + boot | `pipelex.runtime_bridge.serialization` (`serialize_pipe_output`, `serialize_completed_output`, `PIPE_DISPATCH_ERRORS`), `pipelex.runtime_bridge.payloads` (`PipelexPipeRunInput`, `PipelexPipeRunOutput`, `PipelexPipeDispatchAck`), `pipelex.runtime_bridge.bootstrap` (`ensure_pipelex_booted`) |
| Mode + delivery + errors | `pipelex.runtime_bridge.orchestration_mode` (`OrchestrationMode`, `DIRECT_ORCHESTRATION_MODE`), `pipelex.runtime_bridge.delivery_mode` (`DeliveryMode`), `pipelex.runtime_bridge.exceptions` (`MissingOrchestratorError`, `PipelexBridgeDispatchError`) |
| Working-memory hydration | `pipelex.runtime_bridge.primitives.hydration` (re-hydrate `working_memory_raw` → typed `WorkingMemory`; stayed open because it is host-agnostic — used by core delivery and the open `pipelex-api` runner, and re-used across the boundary by `pipelex-transport`) |
-| Plugin contract | `pipelex.plugins.contract` (`PipelexPlugin`, `PLUGIN_API_VERSION`), `pipelex.plugins.registrar` (`PluginRegistrar` menu: `add_orchestrator`, `add_http_error_mapper`, `claim_*`, `add_teardown`; read accessor: `get_http_error_mappers`), `pipelex.plugins.orchestrator_registry` (`OrchestratorProtocol`) |
+| Plugin contract | `pipelex.plugins.contract` (`PipelexPlugin`, `PLUGIN_API_VERSION`), `pipelex.plugins.registrar` (`PluginRegistrar` menu: `add_orchestrator`, `add_bundle_validator`, `add_http_error_mapper`, `claim_*`, `add_teardown`; read accessor: `get_http_error_mappers`), `pipelex.plugins.orchestrator_registry` (`OrchestratorProtocol`), `pipelex.plugins.bundle_validator_registry` (`BundleValidatorProtocol`, `BundleValidationVerdict`) |
| Execution protocols | `PipeRouterProtocol`, `PipeRunProtocol`, `ContentGeneratorProtocol`, the task-manager protocol |
| Payload / core types | `PipeJob`, `PipeOutput`, `DeliveryAssignment`, `WorkingMemory` (+ factory, `dump_for_transport`), `JobMetadata`, `LibraryCrate` |
| Library + hub scoping | `set_current_library` / `get_current_library`, `scoped_pipe_router`, `get_class_registry` (per-call library hydration via `library_crate_dump`) |
@@ -155,8 +198,8 @@ What an out-of-tree orchestrator imports *is* a contract. The SPI is a documente
`pipelex_temporal/temporal_plugin.py` (in the external `pipelex-temporal` distribution) is the reference orchestrator plugin. Its `register`:
-- **always** (regardless of the boot gate): contributes a single `TemporalOrchestrator` registered once under the `"temporal"` token (import-light; `temporalio` is pulled lazily inside `run`/`start`). It advertises `supports_fire_and_forget = True`; its `run` awaits completion and reports `make_workflow_id(...)`, its `start` enqueues the workflow and returns a `PipelexPipeDispatchAck` carrying the workflow id;
-- **only when `plugins.boot_orchestrator == "temporal"`**: claims the content-generator / task-manager / pipe-router / pipe-run hub slots with thunks and registers the teardown callback — booting this process as a Temporal-default runtime.
+- **always** (regardless of the boot gate): contributes a single `TemporalOrchestrator` registered once under the `"temporal"` token (import-light; `temporalio` is pulled lazily inside `run`/`start`), the matching `TemporalBundleValidator` under the same token (the worker-dispatched `/validate` arm), and an HTTP error mapper classifying Temporal transport faults. The orchestrator advertises `supports_fire_and_forget = True`; its `run` awaits completion and reports `make_workflow_id(...)`, its `start` enqueues the workflow and returns a `PipelexPipeDispatchAck` carrying the workflow id;
+- **only when `plugins.boot_orchestrator == "temporal"`**: claims the content-generator / task-manager / pipe-router / pipe-run / isolated-execution-probe hub slots with thunks and registers the teardown callback — booting this process as a Temporal-default runtime.
The orchestrator itself (`pipelex_temporal/temporal_orchestrators.py`) keeps the `WorkflowExecutionError` catch and the `make_workflow_id` recompute in the blocking `run` arm. It serializes its `PipeOutput` through `pipelex.runtime_bridge.serialization`, shared with the core DIRECT orchestrator so the boundary shape cannot drift.
diff --git a/pipelex/cogt/content_generation/llm_generate.py b/pipelex/cogt/content_generation/llm_generate.py
index ad106922b..a160b44e9 100644
--- a/pipelex/cogt/content_generation/llm_generate.py
+++ b/pipelex/cogt/content_generation/llm_generate.py
@@ -11,6 +11,7 @@ from pipelex.cogt.content_generation.dry_mock import (
)
from pipelex.cogt.content_generation.schema_to_model_factory import SchemaToModelFactory
from pipelex.cogt.llm.llm_job_factory import LLMJobFactory
+from pipelex.cogt.llm.llm_utils import dump_prompt, dump_response_from_text_gen
from pipelex.hub import get_llm_worker
@@ -23,7 +24,9 @@ async def llm_gen_text(llm_assignment: LLMAssignment) -> str:
llm_prompt=llm_assignment.llm_prompt,
llm_job_params=llm_assignment.llm_job_params,
)
+ dump_prompt(llm_prompt=llm_job.llm_prompt)
generated_text = await llm_worker.gen_text(llm_job=llm_job)
+ dump_response_from_text_gen(response=generated_text)
log.verbose(generated_text, title="llm_gen_text")
return generated_text
@@ -42,10 +45,12 @@ async def llm_gen_object(object_assignment: ObjectAssignment) -> BaseModel:
schema=object_assignment.object_class_schema,
class_name=object_assignment.object_class_name,
)
+ dump_prompt(llm_prompt=llm_job.llm_prompt)
generated_object: BaseModel = await llm_worker.gen_object(
llm_job=llm_job,
schema=content_class,
)
+ dump_response_from_text_gen(response=generated_object)
return generated_object
@@ -76,9 +81,11 @@ async def llm_gen_object_list(object_assignment: ObjectAssignment) -> list[BaseM
else:
ListSchema.__doc__ = f"A list of {item_class_name}."
+ dump_prompt(llm_prompt=llm_job.llm_prompt)
wrapped_list: ListSchema = await llm_worker.gen_object(
llm_job=llm_job,
schema=ListSchema,
)
+ dump_response_from_text_gen(response=wrapped_list)
generated_list: list[BaseModel] = cast("list[BaseModel]", wrapped_list.items) # pyright: ignore[reportUnknownMemberType]
return generated_list
diff --git a/pipelex/cogt/inference/inference_manager.py b/pipelex/cogt/inference/inference_manager.py
index af9fada0d..df04a1985 100644
--- a/pipelex/cogt/inference/inference_manager.py
+++ b/pipelex/cogt/inference/inference_manager.py
@@ -8,7 +8,6 @@ from pipelex.cogt.img_gen.img_gen_worker_factory import ImgGenWorkerFactory
from pipelex.cogt.inference.inference_manager_protocol import InferenceManagerProtocol
from pipelex.cogt.llm.llm_worker_abstract import LLMWorkerAbstract
from pipelex.cogt.llm.llm_worker_factory import LLMWorkerFactory
-from pipelex.cogt.llm.llm_worker_internal_abstract import LLMWorkerInternalAbstract
from pipelex.cogt.model_backends.model_type import ModelType
from pipelex.hub import get_models_manager, get_report_delegate
@@ -53,7 +52,7 @@ class InferenceManager(InferenceManagerProtocol):
def _setup_one_internal_llm_worker(
self,
llm_handle: str,
- ) -> LLMWorkerInternalAbstract:
+ ) -> LLMWorkerAbstract:
inference_model = get_models_manager().get_inference_model(model_handle=llm_handle, model_type=ModelType.LLM)
llm_worker = LLMWorkerFactory.make_llm_worker(
inference_model=inference_model,
@@ -69,18 +68,6 @@ class InferenceManager(InferenceManagerProtocol):
llm_worker = self._setup_one_internal_llm_worker(llm_handle=llm_handle)
return llm_worker
- @override
- def set_llm_worker_from_external_plugin(
- self,
- llm_handle: str,
- *,
- llm_worker_class: type[LLMWorkerAbstract],
- should_warn_if_already_registered: bool = True,
- ):
- if llm_handle in self.llm_workers and should_warn_if_already_registered:
- log.warning(f"LLM worker for '{llm_handle}' already registered, skipping")
- self.llm_workers[llm_handle] = llm_worker_class(reporting_delegate=get_report_delegate())
-
####################################################################################################
# Manage ImageGen Workers
####################################################################################################
diff --git a/pipelex/cogt/inference/inference_manager_protocol.py b/pipelex/cogt/inference/inference_manager_protocol.py
index d5c916072..d839b0ddd 100644
--- a/pipelex/cogt/inference/inference_manager_protocol.py
+++ b/pipelex/cogt/inference/inference_manager_protocol.py
@@ -18,14 +18,6 @@ class InferenceManagerProtocol(Protocol):
def get_llm_worker(self, llm_handle: str) -> LLMWorkerAbstract: ...
- def set_llm_worker_from_external_plugin(
- self,
- llm_handle: str,
- *,
- llm_worker_class: type[LLMWorkerAbstract],
- should_warn_if_already_registered: bool = True,
- ): ...
-
####################################################################################################
# IMG Generation Workers
####################################################################################################
diff --git a/pipelex/cogt/llm/llm_utils.py b/pipelex/cogt/llm/llm_utils.py
index fb6c3fe59..dfb58532b 100644
--- a/pipelex/cogt/llm/llm_utils.py
+++ b/pipelex/cogt/llm/llm_utils.py
@@ -2,9 +2,13 @@ from typing import Any
from pipelex import pretty_print
from pipelex.cogt.llm.llm_prompt import LLMPrompt
+from pipelex.config import get_config
def dump_prompt(llm_prompt: LLMPrompt) -> None:
+ """Print the prompt for debugging. No-op unless `cogt.llm_config.is_dump_text_prompts_enabled`."""
+ if not get_config().cogt.llm_config.is_dump_text_prompts_enabled:
+ return
prompt_dump = ""
if user_text := llm_prompt.user_text:
prompt_dump += f"\n# User text:\n{user_text}\n"
@@ -20,6 +24,9 @@ def dump_prompt(llm_prompt: LLMPrompt) -> None:
def dump_response_from_text_gen(response: Any) -> None:
+ """Print the LLM response for debugging. No-op unless `cogt.llm_config.is_dump_response_text_enabled`."""
+ if not get_config().cogt.llm_config.is_dump_response_text_enabled:
+ return
pretty_print(response, title="Response from LLM provider")
diff --git a/pipelex/cogt/llm/llm_worker_abstract.py b/pipelex/cogt/llm/llm_worker_abstract.py
index a56fbcde3..ef235658c 100644
--- a/pipelex/cogt/llm/llm_worker_abstract.py
+++ b/pipelex/cogt/llm/llm_worker_abstract.py
@@ -9,9 +9,10 @@ from opentelemetry.trace import NonRecordingSpan, Span, SpanContext, SpanKind, S
from typing_extensions import override
from pipelex import log
-from pipelex.cogt.exceptions import CogtError
+from pipelex.cogt.exceptions import CogtError, LLMCapabilityError
from pipelex.cogt.inference.inference_constants import InferenceOutputType
from pipelex.cogt.inference.inference_worker_abstract import InferenceWorkerAbstract
+from pipelex.cogt.model_backends.constraints import ListedConstraint, ValuedConstraint
from pipelex.cogt.usage.token_category import TokenCategory
from pipelex.pipeline.exceptions import JobMetadataError
from pipelex.pipeline.job_metadata import UnitJobId
@@ -24,6 +25,7 @@ from pipelex.system.telemetry.otel_constants import (
)
from pipelex.system.telemetry.otel_factory import OtelFactory
from pipelex.system.telemetry.telemetry_manager_abstract import TelemetryManagerAbstract
+from pipelex.tools.misc.filetype_utils import UNKNOWN_FILE_TYPE
from pipelex.tools.misc.package_utils import get_package_version
if TYPE_CHECKING:
@@ -31,6 +33,8 @@ if TYPE_CHECKING:
from pydantic import BaseModel
from pipelex.cogt.llm.llm_job import LLMJob
+ from pipelex.cogt.llm.llm_job_components import LLMJobParams
+ from pipelex.cogt.model_backends.model_spec import InferenceModelSpec
from pipelex.reporting.reporting_protocol import ReportingProtocol
from pipelex.tools.typing.pydantic_utils import BaseModelTypeVar
@@ -38,15 +42,18 @@ if TYPE_CHECKING:
class LLMWorkerAbstract(InferenceWorkerAbstract, ABC):
def __init__(
self,
+ inference_model: InferenceModelSpec,
reporting_delegate: ReportingProtocol | None = None,
):
"""Initialize the LLMWorker.
Args:
+ inference_model (InferenceModelSpec): The inference model to be used by the worker.
reporting_delegate (ReportingProtocol | None): An optional report delegate for reporting unit jobs.
"""
InferenceWorkerAbstract.__init__(self, reporting_delegate=reporting_delegate)
+ self.inference_model = inference_model
#########################################################
# Instance methods
@@ -55,33 +62,31 @@ class LLMWorkerAbstract(InferenceWorkerAbstract, ABC):
@property
@override
def desc(self) -> str:
- return "If you're using an external plugin, override this method to describe your llm worker"
+ return self.inference_model.tag
@property
- @abstractmethod
def is_gen_object_supported(self) -> bool:
- return False
+ return self.inference_model.is_gen_object_supported
@property
- @abstractmethod
def is_vision_supported(self) -> bool:
- return False
+ return self.inference_model.is_vision_supported
#########################################################
- # OTel helper methods - override in subclasses with model info
+ # OTel helper methods
#########################################################
def _get_provider_name(self) -> str:
- """Get the GenAI provider name (e.g., 'openai', 'anthropic'). Override in subclass."""
- return "unknown"
+ """Get the GenAI provider name from the inference model backend."""
+ return self.inference_model.backend_name
def _get_request_model_name(self) -> str:
- """Get the request model name. Override in subclass."""
- return "unknown"
+ """Get the model name from the inference model."""
+ return self.inference_model.name
def _get_response_model_name(self) -> str:
- """Get the response model name. Override in subclass."""
- return "unknown"
+ """Get the response model name from the inference model."""
+ return self.inference_model.model_id
def _start_otel_span_llm(self, llm_job: LLMJob, *, output_type: InferenceOutputType, output_class_name: str | None = None) -> Span | None:
"""Start an OTel span for the LLM job and return it.
@@ -318,12 +323,52 @@ class LLMWorkerAbstract(InferenceWorkerAbstract, ABC):
self,
llm_job: LLMJob,
):
+ log.dev(f"✨ {self.desc} ✨")
+
# Verify that the job is valid
llm_job.validate_before_execution()
# Verify feasibility
self._check_can_perform_job(llm_job=llm_job)
+ # Start the job lifecycle: stamps started_at and initializes the token-usage report
+ llm_job.llm_job_before_start(inference_model=self.inference_model)
+ llm_job.applied_job_params = self._apply_constraints(llm_job=llm_job)
+
+ def _apply_constraints(self, llm_job: LLMJob) -> LLMJobParams | None:
+ """Apply constraints from the inference model to job params.
+
+ Args:
+ llm_job: The LLM job containing the original job params
+
+ Returns:
+ A copy of job_params with constraints applied, or None if no changes were needed
+
+ """
+ original_params = llm_job.job_params
+ new_temperature = original_params.temperature
+ max_tokens = original_params.max_tokens or self.inference_model.max_tokens
+ new_max_tokens = max_tokens
+ has_changes = False
+
+ # Temperature constraints
+ if ListedConstraint.TEMPERATURE_MUST_BE_MULTIPLIED_BY_2 in self.inference_model.listed_constraints:
+ new_temperature *= 2
+ has_changes = True
+ fixed_temperature = self.inference_model.valued_constraints.get(ValuedConstraint.FIXED_TEMPERATURE)
+ if fixed_temperature is not None and new_temperature != fixed_temperature:
+ log.warning(
+ f"Model {self.inference_model.desc} used with temperature {new_temperature}, "
+ f"but it must be {fixed_temperature} for this model so we forced it to {fixed_temperature}"
+ )
+ new_temperature = fixed_temperature
+ has_changes = True
+
+ if not has_changes:
+ return None
+
+ return original_params.model_copy(update={"temperature": new_temperature, "max_tokens": new_max_tokens})
+
async def _after_text_job(
self,
*,
@@ -356,7 +401,44 @@ class LLMWorkerAbstract(InferenceWorkerAbstract, ABC):
def _check_can_perform_job(self, llm_job: LLMJob):
# This can be overridden by subclasses for specific checks
- pass
+ self._check_vision_support(llm_job=llm_job)
+ self._check_document_support(llm_job=llm_job)
+
+ def _validate_no_reasoning_for_structured_gen(self, job_params: LLMJobParams):
+ if job_params.reasoning_effort is not None or job_params.reasoning_budget is not None:
+ msg = f"Model '{self.inference_model.desc}' does not support reasoning parameters for structured generation"
+ raise LLMCapabilityError(msg)
+
+ def _check_vision_support(self, llm_job: LLMJob):
+ if llm_job.llm_prompt.user_images:
+ if not self.inference_model.is_vision_supported:
+ msg = f"LLM Engine '{self.inference_model.tag}' does not support vision."
+ raise LLMCapabilityError(msg)
+
+ nb_images = len(llm_job.llm_prompt.user_images)
+ max_prompt_images = self.inference_model.max_prompt_images or 5000
+ if nb_images > max_prompt_images:
+ msg = f"LLM Engine '{self.inference_model.tag}' does not accept that many images: {nb_images}."
+ raise LLMCapabilityError(msg)
+
+ def _check_document_support(self, llm_job: LLMJob):
+ if not llm_job.llm_prompt.user_documents:
+ return
+
+ if not self.inference_model.is_document_supported:
+ msg = f"LLM Engine '{self.inference_model.tag}' does not support documents."
+ raise LLMCapabilityError(msg)
+
+ # Check each document's type is supported
+ supported = self.inference_model.supported_document_types
+ for doc in llm_job.llm_prompt.user_documents:
+ doc_type = doc.get_document_type()
+ # Skip validation for unknown types - let the provider handle it
+ if doc_type == UNKNOWN_FILE_TYPE:
+ continue
+ if doc_type not in supported:
+ msg = f"LLM Engine '{self.inference_model.tag}' does not support {doc_type} documents."
+ raise LLMCapabilityError(msg)
async def gen_text(
self,
diff --git a/pipelex/cogt/llm/llm_worker_factory.py b/pipelex/cogt/llm/llm_worker_factory.py
index a3eb86fda..2a4754619 100644
--- a/pipelex/cogt/llm/llm_worker_factory.py
+++ b/pipelex/cogt/llm/llm_worker_factory.py
@@ -1,6 +1,6 @@
from typing import cast
-from pipelex.cogt.llm.llm_worker_internal_abstract import LLMWorkerInternalAbstract
+from pipelex.cogt.llm.llm_worker_abstract import LLMWorkerAbstract
from pipelex.cogt.model_backends.model_spec import InferenceModelSpec
from pipelex.hub import get_inference_backend_registry, get_models_manager, get_sdk_client_manager
from pipelex.plugins.inference_backend_registry import InferenceFamily
@@ -14,7 +14,7 @@ class LLMWorkerFactory:
inference_model: InferenceModelSpec,
*,
reporting_delegate: ReportingProtocol | None = None,
- ) -> LLMWorkerInternalAbstract:
+ ) -> LLMWorkerAbstract:
model_handle = ModelHandle.make_for_inference_model(inference_model=inference_model)
backend = get_models_manager().get_required_inference_backend(inference_model.backend_name)
make_worker = get_inference_backend_registry().lookup(family=InferenceFamily.LLM, sdk=model_handle.sdk)
@@ -27,4 +27,4 @@ class LLMWorkerFactory:
# The LLM family registry only ever holds LLM workers (the (family, sdk)
# key guarantees it); the uniform MakeWorkerFn return type is widened to
# InferenceWorkerAbstract, so narrow it back here.
- return cast("LLMWorkerInternalAbstract", worker)
+ return cast("LLMWorkerAbstract", worker)
diff --git a/pipelex/cogt/llm/llm_worker_internal_abstract.py b/pipelex/cogt/llm/llm_worker_internal_abstract.py
deleted file mode 100644
index 34d2924a7..000000000
--- a/pipelex/cogt/llm/llm_worker_internal_abstract.py
+++ /dev/null
@@ -1,186 +0,0 @@
-from opentelemetry.trace import Span
-from pydantic import BaseModel
-from typing_extensions import override
-
-from pipelex import log
-from pipelex.cogt.exceptions import LLMCapabilityError
-from pipelex.cogt.llm.llm_job import LLMJob
-from pipelex.cogt.llm.llm_job_components import LLMJobParams
-from pipelex.cogt.llm.llm_utils import dump_prompt, dump_response_from_text_gen
-from pipelex.cogt.llm.llm_worker_abstract import LLMWorkerAbstract
-from pipelex.cogt.model_backends.constraints import ListedConstraint, ValuedConstraint
-from pipelex.cogt.model_backends.model_spec import InferenceModelSpec
-from pipelex.config import get_config
-from pipelex.reporting.reporting_protocol import ReportingProtocol
-from pipelex.tools.misc.filetype_utils import UNKNOWN_FILE_TYPE
-
-
-class LLMWorkerInternalAbstract(LLMWorkerAbstract):
- def __init__(
- self,
- inference_model: InferenceModelSpec,
- reporting_delegate: ReportingProtocol | None = None,
- ):
- """Initialize the LLMWorker.
-
- Args:
- inference_model (InferenceModelSpec): The inference model to be used by the worker.
- reporting_delegate (ReportingProtocol | None): An optional report delegate for reporting unit jobs.
-
- """
- LLMWorkerAbstract.__init__(self, reporting_delegate=reporting_delegate)
- self.inference_model = inference_model
-
- #########################################################
- # Instance methods
- #########################################################
-
- @property
- @override
- def desc(self) -> str:
- return self.inference_model.tag
-
- @property
- @override
- def is_gen_object_supported(self) -> bool:
- return self.inference_model.is_gen_object_supported
-
- @property
- @override
- def is_vision_supported(self) -> bool:
- return self.inference_model.is_vision_supported
-
- #########################################################
- # OTel helper method overrides
- #########################################################
-
- @override
- def _get_provider_name(self) -> str:
- """Get the GenAI provider name from the inference model SDK."""
- return self.inference_model.backend_name
-
- @override
- def _get_request_model_name(self) -> str:
- """Get the model name from the inference model."""
- return self.inference_model.name
-
- @override
- def _get_response_model_name(self) -> str:
- """Get the response model name from the inference model."""
- return self.inference_model.model_id
-
- #########################################################
- # Job lifecycle overrides
- #########################################################
-
- @override
- async def _before_job(
- self,
- llm_job: LLMJob,
- ):
- log.dev(f"✨ {self.desc} ✨")
- await super()._before_job(llm_job=llm_job)
- llm_job.llm_job_before_start(inference_model=self.inference_model)
- llm_job.applied_job_params = self._apply_constraints(llm_job=llm_job)
- if get_config().cogt.llm_config.is_dump_text_prompts_enabled:
- dump_prompt(llm_prompt=llm_job.llm_prompt)
-
- def _apply_constraints(self, llm_job: LLMJob) -> LLMJobParams | None:
- """Apply constraints from the inference model to job params.
-
- Args:
- llm_job: The LLM job containing the original job params
-
- Returns:
- A copy of job_params with constraints applied, or None if no changes were needed
-
- """
- original_params = llm_job.job_params
- new_temperature = original_params.temperature
- max_tokens = original_params.max_tokens or self.inference_model.max_tokens
- new_max_tokens = max_tokens
- has_changes = False
-
- # Temperature constraints
- if ListedConstraint.TEMPERATURE_MUST_BE_MULTIPLIED_BY_2 in self.inference_model.listed_constraints:
- new_temperature *= 2
- has_changes = True
- fixed_temperature = self.inference_model.valued_constraints.get(ValuedConstraint.FIXED_TEMPERATURE)
- if fixed_temperature is not None and new_temperature != fixed_temperature:
- log.warning(
- f"Model {self.inference_model.desc} used with temperature {new_temperature}, "
- f"but it must be {fixed_temperature} for this model so we forced it to {fixed_temperature}"
- )
- new_temperature = fixed_temperature
- has_changes = True
-
- if not has_changes:
- return None
-
- return original_params.model_copy(update={"temperature": new_temperature, "max_tokens": new_max_tokens})
-
- @override
- async def _after_text_job(
- self,
- *,
- span: Span | None,
- llm_job: LLMJob,
- result_text: str,
- ):
- if get_config().cogt.llm_config.is_dump_response_text_enabled:
- dump_response_from_text_gen(response=result_text)
- await super()._after_text_job(span=span, llm_job=llm_job, result_text=result_text)
-
- @override
- async def _after_object_job(
- self,
- *,
- span: Span | None,
- llm_job: LLMJob,
- result_object: BaseModel,
- ):
- if get_config().cogt.llm_config.is_dump_response_text_enabled:
- dump_response_from_text_gen(response=result_object)
- await super()._after_object_job(span=span, llm_job=llm_job, result_object=result_object)
-
- @override
- def _check_can_perform_job(self, llm_job: LLMJob):
- # This can be overridden by subclasses for specific checks
- self._check_vision_support(llm_job=llm_job)
- self._check_document_support(llm_job=llm_job)
-
- def _validate_no_reasoning_for_structured_gen(self, job_params: LLMJobParams):
- if job_params.reasoning_effort is not None or job_params.reasoning_budget is not None:
- msg = f"Model '{self.inference_model.desc}' does not support reasoning parameters for structured generation"
- raise LLMCapabilityError(msg)
-
- def _check_vision_support(self, llm_job: LLMJob):
- if llm_job.llm_prompt.user_images:
- if not self.inference_model.is_vision_supported:
- msg = f"LLM Engine '{self.inference_model.tag}' does not support vision."
- raise LLMCapabilityError(msg)
-
- nb_images = len(llm_job.llm_prompt.user_images)
- max_prompt_images = self.inference_model.max_prompt_images or 5000
- if nb_images > max_prompt_images:
- msg = f"LLM Engine '{self.inference_model.tag}' does not accept that many images: {nb_images}."
- raise LLMCapabilityError(msg)
-
- def _check_document_support(self, llm_job: LLMJob):
- if not llm_job.llm_prompt.user_documents:
- return
-
- if not self.inference_model.is_document_supported:
- msg = f"LLM Engine '{self.inference_model.tag}' does not support documents."
- raise LLMCapabilityError(msg)
-
- # Check each document's type is supported
- supported = self.inference_model.supported_document_types
- for doc in llm_job.llm_prompt.user_documents:
- doc_type = doc.get_document_type()
- # Skip validation for unknown types - let the provider handle it
- if doc_type == UNKNOWN_FILE_TYPE:
- continue
- if doc_type not in supported:
- msg = f"LLM Engine '{self.inference_model.tag}' does not support {doc_type} documents."
- raise LLMCapabilityError(msg)
diff --git a/pipelex/cogt/model_routing/routing_profile_factory.py b/pipelex/cogt/model_routing/routing_profile_factory.py
index 4f5a9349a..4d096a652 100644
--- a/pipelex/cogt/model_routing/routing_profile_factory.py
+++ b/pipelex/cogt/model_routing/routing_profile_factory.py
@@ -56,5 +56,6 @@ class RoutingProfileFactory:
description=blueprint.description,
default=blueprint.default,
routes=blueprint.routes,
+ optional_routes=blueprint.optional_routes,
fallback_order=blueprint.fallback_order,
)
diff --git a/pipelex/plugins/anthropic/anthropic_llm_worker.py b/pipelex/plugins/anthropic/anthropic_llm_worker.py
index ee533c732..40fc39568 100644
--- a/pipelex/plugins/anthropic/anthropic_llm_worker.py
+++ b/pipelex/plugins/anthropic/anthropic_llm_worker.py
@@ -29,7 +29,7 @@ from pipelex.cogt.llm.llm_utils import (
dump_kwargs,
dump_response_from_structured_gen,
)
-from pipelex.cogt.llm.llm_worker_internal_abstract import LLMWorkerInternalAbstract
+from pipelex.cogt.llm.llm_worker_abstract import LLMWorkerAbstract
from pipelex.cogt.llm.thinking_mode import ThinkingMode
from pipelex.cogt.model_backends.constraints import ListedConstraint
from pipelex.cogt.model_backends.model_spec import InferenceModelSpec
@@ -57,7 +57,7 @@ class _ThinkingParams:
suppress_temperature: bool
-class AnthropicLLMWorker(LLMWorkerInternalAbstract):
+class AnthropicLLMWorker(LLMWorkerAbstract):
def __init__(
self,
sdk_instance: Any,
@@ -65,7 +65,7 @@ class AnthropicLLMWorker(LLMWorkerInternalAbstract):
inference_model: InferenceModelSpec,
reporting_delegate: ReportingProtocol | None = None,
):
- LLMWorkerInternalAbstract.__init__(
+ LLMWorkerAbstract.__init__(
self,
inference_model=inference_model,
reporting_delegate=reporting_delegate,
diff --git a/pipelex/plugins/bedrock/bedrock_llm_worker.py b/pipelex/plugins/bedrock/bedrock_llm_worker.py
index 0d83cbbd9..68c3cc5fb 100644
--- a/pipelex/plugins/bedrock/bedrock_llm_worker.py
+++ b/pipelex/plugins/bedrock/bedrock_llm_worker.py
@@ -10,7 +10,7 @@ from pipelex.cogt.inference.error_classify import classify_inference_error
from pipelex.cogt.inference.error_render import InferenceErrorFamily, render_inference_error
from pipelex.cogt.llm.llm_job import LLMJob
from pipelex.cogt.llm.llm_job_components import LLMJobParams
-from pipelex.cogt.llm.llm_worker_internal_abstract import LLMWorkerInternalAbstract
+from pipelex.cogt.llm.llm_worker_abstract import LLMWorkerAbstract
from pipelex.cogt.model_backends.model_spec import InferenceModelSpec
from pipelex.plugins.bedrock.bedrock_client_protocol import BedrockClientProtocol
from pipelex.plugins.bedrock.bedrock_exceptions import BedrockWorkerConfigurationError
@@ -19,14 +19,14 @@ from pipelex.reporting.reporting_protocol import ReportingProtocol
from pipelex.tools.typing.pydantic_utils import BaseModelTypeVar
-class BedrockLLMWorker(LLMWorkerInternalAbstract):
+class BedrockLLMWorker(LLMWorkerAbstract):
def __init__(
self,
sdk_instance: Any,
inference_model: InferenceModelSpec,
reporting_delegate: ReportingProtocol | None = None,
):
- LLMWorkerInternalAbstract.__init__(
+ LLMWorkerAbstract.__init__(
self,
inference_model=inference_model,
reporting_delegate=reporting_delegate,
diff --git a/pipelex/plugins/contract.py b/pipelex/plugins/contract.py
index 6fdec6553..2ea89c730 100644
--- a/pipelex/plugins/contract.py
+++ b/pipelex/plugins/contract.py
@@ -25,7 +25,7 @@ class PipelexPlugin(Protocol):
**Invariant — ``register`` is side-effect-free.** It may *only* call
registrar menu methods: no hub access, no I/O, no SDK/client construction.
This is what makes ``build_registrar`` safe to run more than once (it runs at
- CLI-build to harvest commands *and* again at boot). Anything heavy — importing
+ boot *and* again in the ``pipelex plugins list`` diagnostic command). Anything heavy — importing
a backend SDK, constructing a client, importing ``temporalio`` — happens lazily
inside the ``make_worker`` closures and the hub-slot-claim thunks, never in
``register`` itself.
diff --git a/pipelex/plugins/discovery.py b/pipelex/plugins/discovery.py
index cef78cd30..5241dfea9 100644
--- a/pipelex/plugins/discovery.py
+++ b/pipelex/plugins/discovery.py
@@ -25,8 +25,8 @@ def build_registrar(*, config: "PipelexConfig") -> PluginRegistrar:
"""Discover every plugin and let each register itself into a fresh registrar.
A **pure function**: it touches no global state and constructs no client/SDK,
- so it is safe to call more than once (D3 runs it at CLI-build to harvest
- commands and again at boot). Iterates ``BUILTIN_PLUGINS`` first, then external
+ so it is safe to call more than once (it runs at boot and again in the
+ ``pipelex plugins list`` diagnostic command). Iterates ``BUILTIN_PLUGINS`` first, then external
``pipelex.plugins`` entry points; version-checks each; skips (and logs) any
plugin named in ``config.plugins.disabled`` — externals are denylisted by their
entry-point name *before* ``load()`` so a broken installed plugin can still be
diff --git a/pipelex/plugins/google/google_llm_worker.py b/pipelex/plugins/google/google_llm_worker.py
index d408e9ac8..6032f6501 100644
--- a/pipelex/plugins/google/google_llm_worker.py
+++ b/pipelex/plugins/google/google_llm_worker.py
@@ -21,7 +21,7 @@ from pipelex.cogt.llm.instructor_retry import make_instructor_schema_retrying
from pipelex.cogt.llm.llm_job import LLMJob
from pipelex.cogt.llm.llm_job_components import LLMJobParams, ReasoningEffort
from pipelex.cogt.llm.llm_utils import dump_error, dump_kwargs, dump_response_from_structured_gen
-from pipelex.cogt.llm.llm_worker_internal_abstract import LLMWorkerInternalAbstract
+from pipelex.cogt.llm.llm_worker_abstract import LLMWorkerAbstract
from pipelex.cogt.llm.thinking_mode import ThinkingMode
from pipelex.cogt.model_backends.model_spec import InferenceModelSpec
from pipelex.cogt.usage.token_category import NbTokensByCategoryDict, TokenCategory
@@ -35,7 +35,7 @@ if TYPE_CHECKING:
from openai.types.chat import ChatCompletionMessageParam
-class GoogleLLMWorker(LLMWorkerInternalAbstract):
+class GoogleLLMWorker(LLMWorkerAbstract):
def __init__(
self,
sdk_instance: GoogleGenAiClient,
diff --git a/pipelex/plugins/mistral/mistral_llm_worker.py b/pipelex/plugins/mistral/mistral_llm_worker.py
index b1c6ab892..3105037e6 100644
--- a/pipelex/plugins/mistral/mistral_llm_worker.py
+++ b/pipelex/plugins/mistral/mistral_llm_worker.py
@@ -19,7 +19,7 @@ from pipelex.cogt.inference.error_render import InferenceErrorFamily, render_inf
from pipelex.cogt.llm.instructor_retry import make_instructor_schema_retrying
from pipelex.cogt.llm.llm_job import LLMJob
from pipelex.cogt.llm.llm_job_components import LLMJobParams
-from pipelex.cogt.llm.llm_worker_internal_abstract import LLMWorkerInternalAbstract
+from pipelex.cogt.llm.llm_worker_abstract import LLMWorkerAbstract
from pipelex.cogt.llm.thinking_mode import ThinkingMode
from pipelex.cogt.model_backends.model_spec import InferenceModelSpec
from pipelex.config import get_config
@@ -33,7 +33,7 @@ if TYPE_CHECKING:
from mistralai.types import OptionalNullable
-class MistralLLMWorker(LLMWorkerInternalAbstract):
+class MistralLLMWorker(LLMWorkerAbstract):
def __init__(
self,
mistral_factory: MistralFactory,
@@ -41,7 +41,7 @@ class MistralLLMWorker(LLMWorkerInternalAbstract):
inference_model: InferenceModelSpec,
reporting_delegate: ReportingProtocol | None = None,
):
- LLMWorkerInternalAbstract.__init__(
+ LLMWorkerAbstract.__init__(
self,
inference_model=inference_model,
reporting_delegate=reporting_delegate,
diff --git a/pipelex/plugins/openai/openai_completions_llm_worker.py b/pipelex/plugins/openai/openai_completions_llm_worker.py
index f8b2b83dc..6fbdbd9fb 100644
--- a/pipelex/plugins/openai/openai_completions_llm_worker.py
+++ b/pipelex/plugins/openai/openai_completions_llm_worker.py
@@ -22,7 +22,7 @@ from pipelex.cogt.llm.instructor_retry import make_instructor_schema_retrying
from pipelex.cogt.llm.llm_job import LLMJob
from pipelex.cogt.llm.llm_job_components import LLMJobParams
from pipelex.cogt.llm.llm_utils import dump_error, dump_kwargs, dump_response_from_structured_gen
-from pipelex.cogt.llm.llm_worker_internal_abstract import LLMWorkerInternalAbstract
+from pipelex.cogt.llm.llm_worker_abstract import LLMWorkerAbstract
from pipelex.cogt.llm.thinking_mode import ThinkingMode
from pipelex.cogt.model_backends.constraints import ListedConstraint
from pipelex.cogt.model_backends.model_spec import InferenceModelSpec
@@ -36,7 +36,7 @@ if TYPE_CHECKING:
from openai.types.chat import ChatCompletionMessage
-class OpenAICompletionsLLMWorker(LLMWorkerInternalAbstract):
+class OpenAICompletionsLLMWorker(LLMWorkerAbstract):
def __init__(
self,
openai_completions_factory: OpenAICompletionsFactory,
@@ -44,7 +44,7 @@ class OpenAICompletionsLLMWorker(LLMWorkerInternalAbstract):
inference_model: InferenceModelSpec,
reporting_delegate: ReportingProtocol | None = None,
):
- LLMWorkerInternalAbstract.__init__(
+ LLMWorkerAbstract.__init__(
self,
inference_model=inference_model,
reporting_delegate=reporting_delegate,
diff --git a/pipelex/plugins/openai/openai_responses_llm_worker.py b/pipelex/plugins/openai/openai_responses_llm_worker.py
index 9410896a9..cc8d660b8 100644
--- a/pipelex/plugins/openai/openai_responses_llm_worker.py
+++ b/pipelex/plugins/openai/openai_responses_llm_worker.py
@@ -27,7 +27,7 @@ from pipelex.cogt.inference.error_classify import classify_inference_error
from pipelex.cogt.inference.error_render import InferenceErrorFamily, render_inference_error
from pipelex.cogt.llm.instructor_retry import make_instructor_schema_retrying
from pipelex.cogt.llm.llm_utils import dump_error, dump_kwargs, dump_response_from_structured_gen
-from pipelex.cogt.llm.llm_worker_internal_abstract import LLMWorkerInternalAbstract
+from pipelex.cogt.llm.llm_worker_abstract import LLMWorkerAbstract
from pipelex.cogt.llm.thinking_mode import ThinkingMode
from pipelex.config import get_config
from pipelex.system.telemetry.otel_constants import InferenceOutputType
@@ -43,7 +43,7 @@ if TYPE_CHECKING:
from pipelex.tools.typing.pydantic_utils import BaseModelTypeVar
-class OpenAIResponsesLLMWorker(LLMWorkerInternalAbstract):
+class OpenAIResponsesLLMWorker(LLMWorkerAbstract):
def __init__(
self,
openai_responses_factory: OpenAIResponsesFactory,
diff --git a/tests/integration/pipelex/cogt/test_llm_gen_text.py b/tests/integration/pipelex/cogt/test_llm_gen_text.py
index fa1827cf5..8f64cea7d 100644
--- a/tests/integration/pipelex/cogt/test_llm_gen_text.py
+++ b/tests/integration/pipelex/cogt/test_llm_gen_text.py
@@ -8,7 +8,6 @@ from pipelex.cogt.llm.llm_job_components import LLMJobConfig, LLMJobParams
from pipelex.cogt.llm.llm_job_factory import LLMJobFactory
from pipelex.cogt.llm.llm_prompt import LLMPrompt
from pipelex.cogt.llm.llm_worker_abstract import LLMWorkerAbstract
-from pipelex.cogt.llm.llm_worker_internal_abstract import LLMWorkerInternalAbstract
from pipelex.cogt.model_backends.constraints import ListedConstraint, ValuedConstraint
from pipelex.hub import get_llm_worker, get_model_deck
from pipelex.pipeline.job_metadata import JobMetadata
@@ -84,15 +83,14 @@ class TestLLMGenText:
if temperature > 1:
break
llm_job.job_params = job_params_base.model_copy(update={"max_tokens": max_tokens, "temperature": temperature})
- if isinstance(llm_worker, LLMWorkerInternalAbstract):
- fixed_temp = llm_worker.inference_model.valued_constraints.get(ValuedConstraint.FIXED_TEMPERATURE)
- if fixed_temp is not None:
- log.warning(f"ValuedConstraint FIXED_TEMPERATURE={fixed_temp}, forcing temperature")
- llm_job.job_params = job_params_base.model_copy(update={"temperature": fixed_temp})
- if ListedConstraint.MAX_TOKENS_MUST_BE_HIGH_ENOUGH in llm_worker.inference_model.listed_constraints:
- log.warning("ListedConstraint MAX_TOKENS_MUST_BE_HIGH_ENOUGH, forcing max tokens to at least2000")
- completion_max_tokens = max(max_tokens, 2000)
- llm_job.job_params = job_params_base.model_copy(update={"max_tokens": completion_max_tokens})
+ fixed_temp = llm_worker.inference_model.valued_constraints.get(ValuedConstraint.FIXED_TEMPERATURE)
+ if fixed_temp is not None:
+ log.warning(f"ValuedConstraint FIXED_TEMPERATURE={fixed_temp}, forcing temperature")
+ llm_job.job_params = job_params_base.model_copy(update={"temperature": fixed_temp})
+ if ListedConstraint.MAX_TOKENS_MUST_BE_HIGH_ENOUGH in llm_worker.inference_model.listed_constraints:
+ log.warning("ListedConstraint MAX_TOKENS_MUST_BE_HIGH_ENOUGH, forcing max tokens to at least2000")
+ completion_max_tokens = max(max_tokens, 2000)
+ llm_job.job_params = job_params_base.model_copy(update={"max_tokens": completion_max_tokens})
task: asyncio.Task[str] = asyncio.create_task(llm_worker.gen_text(llm_job=llm_job))
tasks.append(task)
diff --git a/tests/integration/pipelex/cogt/test_setup_inference_workers.py b/tests/integration/pipelex/cogt/test_setup_inference_workers.py
index de8a85060..72e36abf4 100644
--- a/tests/integration/pipelex/cogt/test_setup_inference_workers.py
+++ b/tests/integration/pipelex/cogt/test_setup_inference_workers.py
@@ -1,4 +1,3 @@
-from pipelex.cogt.llm.llm_worker_internal_abstract import LLMWorkerInternalAbstract
from pipelex.cogt.model_backends.model_type import ModelType
from pipelex.hub import get_inference_manager, get_model_deck
@@ -10,8 +9,7 @@ class TestSetupInferenceWorkers:
match inference_model.model_type:
case ModelType.LLM:
llm_worker = inference_manager.get_llm_worker(llm_handle=model_handle)
- if isinstance(llm_worker, LLMWorkerInternalAbstract):
- assert inference_model == llm_worker.inference_model
+ assert inference_model == llm_worker.inference_model
case ModelType.TEXT_EXTRACTOR:
_ = inference_manager.get_extract_worker(extract_handle=model_handle)
case ModelType.IMG_GEN:
diff --git a/tests/integration/pipelex/plugins/test_external_plugin.py b/tests/integration/pipelex/plugins/test_external_plugin.py
index 3f04f9d3c..cf7e28cf0 100644
--- a/tests/integration/pipelex/plugins/test_external_plugin.py
+++ b/tests/integration/pipelex/plugins/test_external_plugin.py
@@ -4,26 +4,38 @@ import pytest
from polyfactory.factories.pydantic_factory import ModelFactory
from typing_extensions import override
-from pipelex import log, pretty_print
+from pipelex import pretty_print
from pipelex.cogt.llm.llm_job import LLMJob
from pipelex.cogt.llm.llm_job_components import LLMJobParams
from pipelex.cogt.llm.llm_job_factory import LLMJobFactory
from pipelex.cogt.llm.llm_prompt import LLMPrompt
-from pipelex.cogt.llm.llm_setting import LLMSetting
from pipelex.cogt.llm.llm_worker_abstract import LLMWorkerAbstract
+from pipelex.cogt.llm.thinking_mode import ThinkingMode
+from pipelex.cogt.model_backends.model_spec import InferenceModelSpec
+from pipelex.cogt.model_backends.model_type import ModelType
from pipelex.cogt.usage.token_category import NbTokensByCategoryDict, TokenCategory
-from pipelex.core.concepts.native.concept_native import NativeConceptCode
-from pipelex.core.pipes.pipe_factory import PipeFactory
-from pipelex.hub import get_inference_manager, get_pipe_router, get_report_delegate
-from pipelex.pipe_operators.llm.pipe_llm import PipeLLM
-from pipelex.pipe_operators.llm.pipe_llm_blueprint import PipeLLMBlueprint
-from pipelex.pipe_run.pipe_job_factory import PipeJobFactory
+from pipelex.hub import get_report_delegate
from pipelex.pipeline.job_metadata import JobMetadata
from pipelex.tools.typing.pydantic_utils import BaseModelTypeVar
from tests.integration.pipelex.cogt.test_data import LLMTestConstants, Person
-from tests.integration.pipelex.test_data import PipeTestCases
-EXTERNAL_PLUGIN_NAME = "mock_external_llm"
+EXTERNAL_MODEL_NAME = "mock_external_llm"
+
+
+def _make_mock_inference_model() -> InferenceModelSpec:
+ return InferenceModelSpec(
+ backend_name="mock_backend",
+ name=EXTERNAL_MODEL_NAME,
+ sdk="mock_sdk",
+ model_type=ModelType.LLM,
+ model_id="mock-external-llm-1",
+ inputs=["text", "images"],
+ outputs=["text", "structured"],
+ costs={},
+ thinking_mode=ThinkingMode.NONE,
+ max_tokens=None,
+ max_prompt_images=None,
+ )
class MockExternalLLMWorker(LLMWorkerAbstract):
@@ -71,8 +83,12 @@ class MockExternalLLMWorker(LLMWorkerAbstract):
@pytest.mark.asyncio(loop_scope="class")
class TestExternalPlugin:
async def test_external_llm_worker(self, job_metadata: JobMetadata, load_empty_library: Callable[[], None]):
+ """An out-of-tree LLMWorkerAbstract subclass runs the full gen_text/gen_object template with just the two _gen impls."""
load_empty_library()
- llm_worker = MockExternalLLMWorker(reporting_delegate=get_report_delegate())
+ llm_worker = MockExternalLLMWorker(
+ inference_model=_make_mock_inference_model(),
+ reporting_delegate=get_report_delegate(),
+ )
llm_job = LLMJobFactory.make_llm_job(
llm_prompt=LLMPrompt(
system_text=None,
@@ -92,39 +108,3 @@ class TestExternalPlugin:
generated_object = await llm_worker.gen_object(llm_job=llm_job, schema=Person)
assert generated_object
pretty_print(generated_object)
-
- async def test_pipe_llm_with_external_llm_handle(self, job_metadata: JobMetadata, load_empty_library: Callable[[], None]):
- load_empty_library()
- llm_handle = EXTERNAL_PLUGIN_NAME
- get_inference_manager().set_llm_worker_from_external_plugin(
- llm_handle=llm_handle,
- llm_worker_class=MockExternalLLMWorker,
- )
-
- pipe_llm_blueprint = PipeLLMBlueprint(
- description="LLM test with external plugin",
- output=NativeConceptCode.TEXT,
- system_prompt=PipeTestCases.SYSTEM_PROMPT,
- prompt=PipeTestCases.USER_PROMPT,
- model=LLMSetting(
- model=llm_handle,
- temperature=0.5,
- max_tokens=None,
- ),
- )
-
- pipe_job = PipeJobFactory.make_pipe_job(
- pipe=PipeFactory[PipeLLM].make_from_blueprint(
- domain_code="generic",
- pipe_code="adhoc_for_test_pipe_llm_with_external_llm_handle",
- blueprint=pipe_llm_blueprint,
- ),
- job_metadata=job_metadata,
- )
- pipe_llm_output = await get_pipe_router().run(
- pipe_job=pipe_job,
- )
-
- log.verbose(pipe_llm_output, title="stuff")
- llm_generated_text = pipe_llm_output.main_stuff_as_text
- pretty_print(llm_generated_text, title="llm_generated_text")
diff --git a/tests/unit/pipelex/cogt/model_routing/test_routing_profile_optional_routes.py b/tests/unit/pipelex/cogt/model_routing/test_routing_profile_optional_routes.py
new file mode 100644
index 000000000..3c9d3ea6d
--- /dev/null
+++ b/tests/unit/pipelex/cogt/model_routing/test_routing_profile_optional_routes.py
@@ -0,0 +1,67 @@
+from pipelex.cogt.model_routing.routing_models import BackendMatchingMethod
+from pipelex.cogt.model_routing.routing_profile import RoutingProfile
+from pipelex.cogt.model_routing.routing_profile_factory import RoutingProfileBlueprint, RoutingProfileFactory
+
+
+class TestRoutingProfileOptionalRoutes:
+ """Tests for optional_routes: factory pass-through and enabled-backend gating."""
+
+ def test_factory_carries_optional_routes_from_blueprint(self):
+ """Regression: the factory used to drop optional_routes parsed from the blueprint."""
+ # Arrange
+ blueprint = RoutingProfileBlueprint(
+ description="test profile",
+ default="openai",
+ optional_routes={"hello-1": "hello"},
+ )
+
+ # Act
+ routing_profile = RoutingProfileFactory.make_routing_profile(
+ name="test_profile",
+ blueprint=blueprint,
+ )
+
+ # Assert
+ assert routing_profile.optional_routes == {"hello-1": "hello"}
+
+ def test_optional_route_applies_when_backend_enabled(self):
+ """An optional route wins as an exact match when its target backend is enabled."""
+ # Arrange
+ routing_profile = RoutingProfile(
+ name="test_profile",
+ default="openai",
+ optional_routes={"hello-1": "hello"},
+ )
+ enabled_backends = ["openai", "hello"]
+
+ # Act
+ result = routing_profile.get_backend_match_for_model(
+ enabled_backends=enabled_backends,
+ model_name="hello-1",
+ )
+
+ # Assert
+ assert result is not None
+ assert result.backend_name == "hello"
+ assert result.matching_method == BackendMatchingMethod.EXACT_MATCH
+
+ def test_optional_route_inert_when_backend_disabled(self):
+ """An optional route whose target backend is disabled falls through to the default."""
+ # Arrange
+ routing_profile = RoutingProfile(
+ name="test_profile",
+ default="openai",
+ optional_routes={"hello-1": "hello"},
+ )
+ enabled_backends = ["openai"]
+
+ # Act
+ result = routing_profile.get_backend_match_for_model(
+ enabled_backends=enabled_backends,
+ model_name="hello-1",
+ )
+
+ # Assert
+ assert result is not None
+ assert result.backend_name == "openai"
+ assert result.matching_method == BackendMatchingMethod.DEFAULT
diff --git a/tests/unit/pipelex/cogt/test_worker_error_enrichment.py b/tests/unit/pipelex/cogt/test_worker_error_enrichment.py
index e4ecde1b4..4f9caa5bd 100644
--- a/tests/unit/pipelex/cogt/test_worker_error_enrichment.py
+++ b/tests/unit/pipelex/cogt/test_worker_error_enrichment.py
@@ -43,10 +43,8 @@ class _StubSchema(BaseModel):
class _StubLLMWorker(LLMWorkerAbstract):
"""LLM worker whose abstract impl always raises the supplied error."""
- def __init__(self, model_handle: str, provider_name: str, error: Exception) -> None:
- LLMWorkerAbstract.__init__(self, reporting_delegate=None)
- self._model_handle = model_handle
- self._provider_name = provider_name
+ def __init__(self, inference_model: Any, error: Exception) -> None:
+ LLMWorkerAbstract.__init__(self, inference_model=inference_model, reporting_delegate=None)
self._error = error
@property
@@ -59,14 +57,6 @@ class _StubLLMWorker(LLMWorkerAbstract):
def is_vision_supported(self) -> bool:
return True
- @override
- def _get_request_model_name(self) -> str:
- return self._model_handle
-
- @override
- def _get_provider_name(self) -> str:
- return self._provider_name
-
@override
async def _gen_text(self, llm_job: Any) -> str:
raise self._error
@@ -121,9 +111,11 @@ class _StubSearchWorker(SearchWorkerAbstract):
def _make_llm_job(mocker: MockerFixture) -> Any:
- """LLM job with telemetry disabled so the OTel span is skipped."""
+ """LLM job with telemetry disabled so the OTel span is skipped, and a prompt that no-ops the capability checks."""
job = mocker.MagicMock()
job.job_metadata.otel_context = None
+ job.llm_prompt.user_images = []
+ job.llm_prompt.user_documents = []
return job
@@ -136,9 +128,20 @@ def _make_extract_job(mocker: MockerFixture) -> Any:
return job
-def _make_inference_model() -> SimpleNamespace:
+def _make_inference_model(backend_name: str = WORKER_PROVIDER) -> SimpleNamespace:
"""Stub inference model exposing the fields the workers read."""
- return SimpleNamespace(name=WORKER_MODEL, backend_name=WORKER_PROVIDER, tag="stub-tag", desc="stub-desc", is_img2img_supported=True)
+ return SimpleNamespace(
+ name=WORKER_MODEL,
+ backend_name=backend_name,
+ model_id=WORKER_MODEL,
+ tag="stub-tag",
+ desc="stub-desc",
+ max_tokens=None,
+ max_prompt_images=None,
+ listed_constraints=[],
+ valued_constraints={},
+ is_img2img_supported=True,
+ )
@pytest.mark.asyncio(loop_scope="class")
@@ -149,7 +152,7 @@ class TestWorkerErrorEnrichment:
async def test_llm_worker_enriches_error(self, mocker: MockerFixture, use_object: bool) -> None:
"""gen_text / gen_object stamp the worker's model and provider onto a bare LLMCompletionError."""
error = LLMCompletionError("LLM worker failed", error_category=InferenceErrorCategory.TRANSIENT)
- worker = _StubLLMWorker(model_handle=WORKER_MODEL, provider_name=WORKER_PROVIDER, error=error)
+ worker = _StubLLMWorker(inference_model=_make_inference_model(), error=error)
job = _make_llm_job(mocker)
invocation = worker.gen_object(llm_job=job, schema=_StubSchema) if use_object else worker.gen_text(llm_job=job)
@@ -164,7 +167,7 @@ class TestWorkerErrorEnrichment:
"""An inner error that already set model_handle keeps its value; only the missing provider is filled."""
inner_model = "preset-only-model"
error = LLMModelNotFoundError("model not found", model_handle=inner_model)
- worker = _StubLLMWorker(model_handle=WORKER_MODEL, provider_name=WORKER_PROVIDER, error=error)
+ worker = _StubLLMWorker(inference_model=_make_inference_model(), error=error)
with pytest.raises(LLMModelNotFoundError) as exc_info:
await worker.gen_text(llm_job=_make_llm_job(mocker))
@@ -176,7 +179,7 @@ class TestWorkerErrorEnrichment:
async def test_llm_worker_skips_unknown_provider(self, mocker: MockerFixture) -> None:
"""A worker that does not know its provider ('unknown' default) leaves provider unset."""
error = LLMCompletionError("LLM worker failed", error_category=InferenceErrorCategory.TRANSIENT)
- worker = _StubLLMWorker(model_handle=WORKER_MODEL, provider_name="unknown", error=error)
+ worker = _StubLLMWorker(inference_model=_make_inference_model(backend_name="unknown"), error=error)
with pytest.raises(LLMCompletionError) as exc_info:
await worker.gen_text(llm_job=_make_llm_job(mocker))
@@ -194,7 +197,7 @@ class TestWorkerErrorEnrichment:
)
async def test_llm_worker_closes_span_on_failure(self, mocker: MockerFixture, raised_error: Exception) -> None:
"""gen_text always closes the OTel span on failure — for a CogtError and for an unexpected non-CogtError alike."""
- worker = _StubLLMWorker(model_handle=WORKER_MODEL, provider_name=WORKER_PROVIDER, error=raised_error)
+ worker = _StubLLMWorker(inference_model=_make_inference_model(), error=raised_error)
fake_span = mocker.MagicMock()
fake_span.is_recording.return_value = True
mocker.patch.object(worker, "_start_otel_span_llm", return_value=fake_span)