Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
2/3 Pipelex kernel — Phase 2: the remaining operators (#1082)
Louis Choquel40ca9d247c0b
diff --git a/.test_durations b/.test_durationsindex 51d60f2e6..c0e0b79a5 100644--- a/.test_durations+++ b/.test_durations@@ -6245,11 +6245,11 @@"tests/unit/pipelex/pipe_operators/pipe_img_gen/test_pipe_img_gen_input.py::TestPipeImgGenValidateInputs::test_validate_inputs_valid_cases[valid_with_output_format-blueprint7]": 0.12428837500920054,"tests/unit/pipelex/pipe_operators/pipe_img_gen/test_pipe_img_gen_input.py::TestPipeImgGenValidateInputs::test_validate_inputs_valid_cases[valid_with_seed-blueprint4]": 0.11954637499002274,"tests/unit/pipelex/pipe_operators/pipe_img_gen/test_pipe_img_gen_input.py::TestPipeImgGenValidateInputs::test_validate_inputs_valid_cases[valid_with_seed_auto-blueprint5]": 0.1384919580013957,- "tests/unit/pipelex/pipe_operators/pipe_img_gen/test_pipe_img_gen_size_default.py::TestResolveDefaultSize::test_resolve_default_size[exact-default-applies-without-ratio]": 0.0012770010071108118,- "tests/unit/pipelex/pipe_operators/pipe_img_gen/test_pipe_img_gen_size_default.py::TestResolveDefaultSize::test_resolve_default_size[explicit-ratio-suppresses-exact-default]": 1.1081438330147648,- "tests/unit/pipelex/pipe_operators/pipe_img_gen/test_pipe_img_gen_size_default.py::TestResolveDefaultSize::test_resolve_default_size[no-default-stays-none]": 0.15100087500468362,- "tests/unit/pipelex/pipe_operators/pipe_img_gen/test_pipe_img_gen_size_default.py::TestResolveDefaultSize::test_resolve_default_size[tier-default-applies-without-ratio]": 0.0010004179930547252,- "tests/unit/pipelex/pipe_operators/pipe_img_gen/test_pipe_img_gen_size_default.py::TestResolveDefaultSize::test_resolve_default_size[tier-default-composes-with-explicit-ratio]": 0.0004727919877041131,+ "tests/unit/pipelex/kernel/test_img_gen_size_default.py::TestResolveDefaultSize::test_resolve_default_size[exact-default-applies-without-ratio]": 0.0012770010071108118,+ "tests/unit/pipelex/kernel/test_img_gen_size_default.py::TestResolveDefaultSize::test_resolve_default_size[explicit-ratio-suppresses-exact-default]": 1.1081438330147648,+ "tests/unit/pipelex/kernel/test_img_gen_size_default.py::TestResolveDefaultSize::test_resolve_default_size[no-default-stays-none]": 0.15100087500468362,+ "tests/unit/pipelex/kernel/test_img_gen_size_default.py::TestResolveDefaultSize::test_resolve_default_size[tier-default-applies-without-ratio]": 0.0010004179930547252,+ "tests/unit/pipelex/kernel/test_img_gen_size_default.py::TestResolveDefaultSize::test_resolve_default_size[tier-default-composes-with-explicit-ratio]": 0.0004727919877041131,"tests/unit/pipelex/pipe_operators/pipe_llm/test_llm_prompt_blueprint.py::TestLLMPromptBlueprintRequiredVariables::test_required_variables_deduplicates_same_root": 0.001239625024027191,"tests/unit/pipelex/pipe_operators/pipe_llm/test_llm_prompt_blueprint.py::TestLLMPromptBlueprintRequiredVariables::test_required_variables_excludes_internal_variables": 0.0012199170014355332,"tests/unit/pipelex/pipe_operators/pipe_llm/test_llm_prompt_blueprint.py::TestLLMPromptBlueprintRequiredVariables::test_required_variables_excludes_place_holder": 0.13840670901117846,diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 849290876..4af662e46 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -4,12 +4,14 @@### Added-- **Pipelex kernel (`pipelex.kernel`)**: A new public subpackage holding operator-execution semantics as importable functions, so what a `PipeLLM` step *does* — deck resolution, templating-style derivation, prompt assembly, generation, memory write-back — is no longer reachable only through a fully booted interpreter with a loaded library. Every kernel call is servable on a runtime-only boot with zero `.mthds` loaded, which is pinned by a permanent test. This first release covers the LLM operator; the remaining operators follow.+- **Pipelex kernel (`pipelex.kernel`)**: A new public subpackage holding operator-execution semantics as importable functions, so what a step *does* — deck resolution, templating-style derivation, prompt assembly, generation, memory write-back — is no longer reachable only through a fully booted interpreter with a loaded library. Every operator is covered: LLM and structuring, extraction, image generation, search, template composition, and Python function calls. Every one of those calls is servable on a runtime-only boot with zero `.mthds` loaded, and a permanent test pins it by calling each of them on such a boot and then checking that no interpreter module was loaded. Two arms stay with the interpreter, because their inputs are language or deployment artifacts rather than execution semantics — `PipeCompose`'s construct mode, which is driven by an MTHDS blueprint, and `PipeFunc`'s pluggable executor, which selects *where* a function runs.### Changed- **Prompt reference types moved (Breaking)**: `ImageReference` and `DocumentReference` (with their `*Kind` enums) moved from `pipelex.pipe_operators.shared.image_reference` / `pipelex.pipe_operators.llm.document_reference` to `pipelex.kernel.prompt_references`. They describe how a prompt resolves an image or document out of working memory, which is execution semantics rather than a language artifact.-- **`LLMPromptBlueprintValueError` replaced (Breaking)**: The error raised when a prompt's image or document reference cannot be resolved is now `pipelex.kernel.exceptions.PromptContentError`, moved with the code that raises it. `LLMPromptBlueprint` keeps its fields, its validation and its `make_llm_prompt` signature.+- **`LLMPromptBlueprintValueError` and `ImgGenPromptBlueprintValueError` replaced (Breaking)**: The error raised when a prompt's image or document reference cannot be resolved is now `pipelex.kernel.exceptions.PromptContentError`, moved with the code that raises it. Both blueprints keep their fields, their validation, and their `make_*_prompt` signatures.+- **Image-generation prompts are buildable without the interpreter**: `pipelex.kernel.img_gen_prompt.assemble_img_gen_prompt` is the image counterpart of the kernel's LLM prompt assembly — image-reference resolution, `[Image N]` placeholder numbering, and template rendering. `run_img_gen` takes a ready `ImgGenPrompt`, and until now the only way to build one was an interpreter-layer blueprint, which made image generation the single operator a runtime-only caller could not actually reach. `ImgGenPromptBlueprint` now delegates to it, keeping its parse-and-validate role and the `max_prompt_images` limit whose breach is a pipe-level error.+- **`StructuredContentComposer` no longer takes `pipe_run_params` (Breaking)**: The constructor parameter and the attribute behind it are gone. They were dead — stored, threaded down to nested composers, and never read — and the docstring's claim that they gated dry-run mode was stale. Dropping them is what leaves the composer importing nothing from `pipelex.pipe_run`.- **Keyless-boot forced-DRY rule has one home**: `pipelex.runtime_hub.resolve_run_mode_for_boot` now owns the coercion that a keyless boot (`needs_inference=False`) applies to a requested run mode. Both run-params factories call it — the pipe tier's `PipeRunParamsFactory.make_run_params`, whose behavior is unchanged, and the kernel tier's `PipelexKernel.make`, which previously minted a LIVE `CogtRunParams` on an offline boot. A programmatic kernel caller on a keyless boot now gets the same forced DRY and the same warning as a pipe run.### Fixeddiff --git a/docs/under-the-hood/architecture-overview.md b/docs/under-the-hood/architecture-overview.mdindex ee4b4f269..4ef5528cc 100644--- a/docs/under-the-hood/architecture-overview.md+++ b/docs/under-the-hood/architecture-overview.md@@ -46,6 +46,8 @@ Operators perform concrete actions:- **PipeCompose** - Compose content from templates or construct structured objects- **PipeSearch** - Search the web for information+What an operator *does* — model-deck resolution, prompt assembly, generation, memory write-back — lives one layer down in [`pipelex/kernel/`](https://github.com/Pipelex/pipelex/tree/main/pipelex/kernel), as plain functions that take everything method-specific as an explicit argument and read only the runtime services a boot stands up, such as the model deck and the content generator. The operator classes above hold what is specific to being a step in a method: blueprint resolution, input validation, error context, and execution-graph tracing. The split exists so the same semantics can be called without a loaded method, and so there is one implementation rather than two that drift.+### Core DomainLocated in [`pipelex/core/`](https://github.com/Pipelex/pipelex/tree/main/pipelex/core)diff --git a/pipelex/kernel/__init__.py b/pipelex/kernel/__init__.pyindex e7cc464b3..e513f4dda 100644--- a/pipelex/kernel/__init__.py+++ b/pipelex/kernel/__init__.py@@ -1,8 +1,8 @@"""The Pipelex kernel: operator-execution semantics as importable functions.-What a `PipeLLM` step actually *does* — deck resolution, templating-style derivation, prompt-assembly, generation, memory write-back — used to be reachable only through a fully booted-interpreter with a loaded library. This package holds that semantics once, so it has **one+What an operator step actually *does* — deck resolution, prompt or job-params assembly, generation,+memory write-back — used to be reachable only through a fully booted interpreter with a loaded+library. This package holds that semantics once, so it has **oneimplementation with multiple callers**: the interpreter's operator classes, and any programmaticcaller invoking the same functions on a ``RuntimeBoot``-only process with zero ``.mthds`` loaded.Single-sourcing is the point — two callers with two copies drift.@@ -60,8 +60,16 @@ and strictly only when it is obviously the subject. Nothing here clears that barargument.**Boot contract.** Every kernel call must be servable on ``RuntimeBoot.make()``-(``pipelex/runtime_boot.py``) with no interpreter constructed and no library loaded. The-import-closure test guards that structurally; a booted-runtime test proves it dynamically.+(``pipelex/runtime_boot.py``) with no interpreter constructed and no library loaded.+``tests/unit/pipelex/kernel/test_kernel_boot_contract.py`` is what proves it: it boots keyless,+**calls every entry point**, and sweeps ``sys.modules`` afterwards. Treat that as the gate rather+than as one of two — ``tests/unit/pipelex/test_runtime_layer_import_closure.py`` covers only what+its ``pipelex.kernel.pipelex_kernel`` entry point imports, and the façade is LLM-only, so most of+this package sits outside its closure. **Every new entry point here gets an arm in that+subprocess.** The rule is not ceremony: the sweep is the only gate that can see a function-local+interpreter import (both static guards read module-level imports, and the blind spot is+*per-function*), and nothing tells you when it has fallen behind — Phase 2 added five ops and every+gate stayed green while covering none of them.Per the repo-wide rule, this module holds **no re-exports** — import each symbol from the modulethat defines it.diff --git a/pipelex/kernel/compose_ops.py b/pipelex/kernel/compose_ops.pynew file mode 100644index 000000000..28e80623d--- /dev/null+++ b/pipelex/kernel/compose_ops.py@@ -0,0 +1,91 @@+"""Compose operator semantics: context assembly, template rendering, content construction, write-back.++These are the functions the interpreter's `PipeCompose` calls on its template path, and the ones a+programmatic caller invokes on a `RuntimeBoot`-only process. Nothing here reaches a hub: composing a+text is pure rendering over the memory it is handed.++**Construct mode is deliberately not here**, and that is a placement verdict rather than an oversight.+Its semantics are `StructuredContentComposer` over a `ConstructBlueprint`, and the blueprint is a+language artifact that `.mthds` parses into — moving both into the kernel would relocate the MTHDS+field-composition model and its error family for a caller shape that has no programmatic use (a caller+holding real Python would build the object, not describe it in a blueprint). See+`wip/kernel/deferred-follow-ups.md`.+"""++from typing import Any++from pipelex.cogt.templating.template_rendering import render_template+from pipelex.core.concepts.concept import Concept+from pipelex.core.memory.working_memory import WorkingMemory+from pipelex.core.stuffs.html_content import HtmlContent+from pipelex.core.stuffs.stuff_content import StuffContent+from pipelex.kernel.compose_results import ComposeResult+from pipelex.kernel.memory_ops import store_result+from pipelex.tools.jinja2.template_category import TemplateCategory+from pipelex.tools.templating.templating_style import TemplatingStyle+++def build_compose_context(+ *,+ memory: WorkingMemory,+ runtime_params: dict[str, Any] | None = None,+ extra_context: dict[str, Any] | None = None,+) -> dict[str, Any]:+ """Layer the three context sources a composed template renders against, least specific first.++ Memory's stuffs, then the per-run params, then the step's own `extra_context` — so a step-authored+ value wins over a run-supplied one, which wins over a stuff of the same name. Single-sourced here+ because the construct path's TEMPLATE fields layer the same three in the same order, and a second+ copy of the order is how the two would drift.+ """+ context: dict[str, Any] = memory.generate_context()+ if runtime_params:+ context.update(**runtime_params)+ if extra_context:+ context.update(**extra_context)+ return context+++def build_composed_content(*, output_class: type[StuffContent], rendered_text: str) -> StuffContent:+ """Wrap a rendered text in the step's declared output class.++ An HTML-shaped class takes the text as its inner markup with no wrapper class; anything else takes+ it as plain text. That fork is on the class rather than on the concept, which is why the kernel can+ answer it without a library.++ `model_validate` rather than a keyword construction, for the reason `run_llm_text` states: the class+ arrives typed as `type[StuffContent]`, whose declared fields include neither `text` nor `inner_html`+ — the subclasses a Text- or Html-compatible concept resolves to are what carry them. Validation, and+ the `ValidationError` a mismatch raises, are identical either way.+ """+ if issubclass(output_class, HtmlContent):+ return output_class.model_validate({"inner_html": rendered_text, "css_class": ""})+ return output_class.model_validate({"text": rendered_text})+++async def run_compose_template(+ *,+ memory: WorkingMemory,+ template: str,+ category: TemplateCategory,+ concept: Concept,+ output_class: type[StuffContent],+ templating_style: TemplatingStyle | None = None,+ runtime_params: dict[str, Any] | None = None,+ extra_context: dict[str, Any] | None = None,+ result_name: str | None = None,+ result_code: str | None = None,+) -> ComposeResult:+ """A whole compose step over a template: build the context, render, wrap, store, report."""+ rendered_text = await render_template(+ template=template,+ category=category,+ context=build_compose_context(memory=memory, runtime_params=runtime_params, extra_context=extra_context),+ templating_style=templating_style,+ )+ content = build_composed_content(output_class=output_class, rendered_text=rendered_text)+ return ComposeResult(+ memory=store_result(memory=memory, concept=concept, content=content, result_name=result_name, result_code=result_code),+ content=content,+ rendered_text=rendered_text,+ )diff --git a/pipelex/kernel/compose_results.py b/pipelex/kernel/compose_results.pynew file mode 100644index 000000000..a837ae968--- /dev/null+++ b/pipelex/kernel/compose_results.py@@ -0,0 +1,24 @@+"""What a kernel compose call hands back.++The rendered text rides along because the kernel is where it is produced and the interpreter's+execution-graph tracer records it. The memory contract holds as everywhere else in the kernel:+**the returned `memory` is the result**, and no caller may rely on the argument having been mutated.++The serialization note on `pipelex.kernel.llm_results` applies here too — `content` is annotated with+the base `StuffContent`, so use `kajson` rather than a plain `model_dump()`.+"""++from pydantic import BaseModel, ConfigDict++from pipelex.core.memory.working_memory import WorkingMemory+from pipelex.core.stuffs.stuff_content import StuffContent+++class ComposeResult(BaseModel):+ """The outcome of a kernel compose call over a template."""++ model_config = ConfigDict(frozen=True)++ memory: WorkingMemory+ content: StuffContent+ rendered_text: strdiff --git a/pipelex/kernel/extract_ops.py b/pipelex/kernel/extract_ops.pynew file mode 100644index 000000000..cd596c03e--- /dev/null+++ b/pipelex/kernel/extract_ops.py@@ -0,0 +1,96 @@+"""Extract operator semantics: deck resolution, job-params assembly, extraction, memory write-back.++These are the functions the interpreter's `PipeExtract` calls, and the ones a programmatic caller+invokes on a `RuntimeBoot`-only process. They read the runtime hub for the services a runtime boot+stands up (the model deck, the content generator) and take everything else as an explicit argument.++One seam is deliberately *not* here: deciding **what** to extract. Turning a step's declared input into+an image or a document URI is a question about the memory's stuffs and the concepts they carry, which+is the caller's business; the kernel takes the `ExtractInput` already built.+"""++from typing import TYPE_CHECKING++from pipelex.cogt.content_generation.cogt_run_params import CogtRunParams+from pipelex.cogt.extract.extract_input import ExtractInput+from pipelex.cogt.extract.extract_job_components import ExtractJobConfig, ExtractJobParams+from pipelex.cogt.extract.extract_setting import ExtractModelChoice, ExtractSetting+from pipelex.core.concepts.concept import Concept+from pipelex.core.memory.working_memory import WorkingMemory+from pipelex.core.stuffs.list_content import ListContent+from pipelex.kernel.extract_results import ExtractResult+from pipelex.kernel.memory_ops import store_result+from pipelex.runtime_hub import get_content_generator, get_model_deck+from pipelex.system.job_metadata import JobMetadata++if TYPE_CHECKING:+ from pipelex.core.stuffs.page_content import PageContent+++def resolve_extract_setting(*, extract_choice: ExtractModelChoice | None = None) -> ExtractSetting:+ """The deck chain for an extraction: the step's own choice, then the deck's default."""+ model_deck = get_model_deck()+ resolved_choice = extract_choice or model_deck.extract_choice_default+ return model_deck.get_extract_setting(extract_choice=resolved_choice)+++def build_extract_job_params(+ *,+ extract_setting: ExtractSetting,+ should_caption_images: bool = False,+ should_include_page_views: bool = False,+ page_views_dpi: int | None = None,+ max_page_images: int | None = None,+ render_js: bool | None = None,+ include_raw_html: bool | None = None,+) -> ExtractJobParams:+ """Compose the per-job parameters from the resolved setting and the step's own one-time values.++ `max_page_images` is the one that composes rather than simply overriding-or-defaulting: a step+ that names it wins over the setting's `max_nb_images`, and `0` is a meaningful value (extract no+ images), so the precedence is written against `None` and not against falsiness. `image_min_size`+ has no step-level counterpart and always comes from the setting.+ """+ return ExtractJobParams(+ should_caption_images=should_caption_images,+ should_include_page_views=should_include_page_views,+ page_views_dpi=page_views_dpi,+ max_nb_images=max_page_images if max_page_images is not None else extract_setting.max_nb_images,+ image_min_size=extract_setting.image_min_size,+ render_js=render_js,+ include_raw_html=include_raw_html,+ )+++async def run_extract(+ *,+ memory: WorkingMemory,+ extract_input: ExtractInput,+ extract_setting: ExtractSetting,+ extract_job_params: ExtractJobParams,+ concept: Concept,+ job_metadata: JobMetadata,+ cogt_run_params: CogtRunParams,+ result_name: str | None = None,+ result_code: str | None = None,+) -> ExtractResult:+ """A whole extract step: extract the pages, store them as a list, report.++ The pages always land in memory as a `ListContent` — an extraction produces a page sequence even+ when the source is a single image — so there is no multiplicity fork here as there is on the LLM+ and image-generation paths.+ """+ page_contents = await get_content_generator().make_extract_pages(+ extract_input=extract_input,+ cogt_run_params=cogt_run_params,+ extract_handle=extract_setting.model,+ job_metadata=job_metadata,+ extract_job_params=extract_job_params,+ extract_job_config=ExtractJobConfig(),+ )+ content: ListContent[PageContent] = ListContent(items=page_contents)+ return ExtractResult(+ memory=store_result(memory=memory, concept=concept, content=content, result_name=result_name, result_code=result_code),+ content=content,+ extract_setting=extract_setting,+ )diff --git a/pipelex/kernel/extract_results.py b/pipelex/kernel/extract_results.pynew file mode 100644index 000000000..79405a567--- /dev/null+++ b/pipelex/kernel/extract_results.py@@ -0,0 +1,36 @@+"""What a kernel extract call hands back.++The produced pages plus the one intermediate the caller cannot recompute without redoing the deck+chain: the resolved `ExtractSetting`, whose `model` is what the interpreter's execution-graph tracer+records. The memory contract holds as everywhere else in the kernel — **the returned `memory` is the+result**, and no caller may rely on the argument having been mutated.++The serialization note on `pipelex.kernel.llm_results` applies here too, for the same reason: `pages`+holds concrete `PageContent` instances, so use `kajson` rather than a plain `model_dump()` if one of+these ever crosses a boundary. Nothing serializes them today.++`content` is the one envelope field in the kernel annotated below `StuffContent`, because an+extraction genuinely always produces a page sequence and a caller should get `.items` typed without a+cast. The narrowing has one visible consequence worth knowing: `run_extract` builds a plain+`ListContent`, so pydantic revalidates it into a fresh `ListContent[PageContent]` here, and+`result.content` is therefore **not the same object** as the content the returned memory holds — the+only kernel op where those two diverge. Nothing is lost by it (the `PageContent` items pass through+by identity, only the container is new), but do not write an identity check across the two.+"""++from pydantic import BaseModel, ConfigDict++from pipelex.cogt.extract.extract_setting import ExtractSetting+from pipelex.core.memory.working_memory import WorkingMemory+from pipelex.core.stuffs.list_content import ListContent+from pipelex.core.stuffs.page_content import PageContent+++class ExtractResult(BaseModel):+ """The outcome of a kernel extract call."""++ model_config = ConfigDict(frozen=True)++ memory: WorkingMemory+ content: ListContent[PageContent]+ extract_setting: ExtractSettingdiff --git a/pipelex/kernel/func_ops.py b/pipelex/kernel/func_ops.pynew file mode 100644index 000000000..8718fb7d0--- /dev/null+++ b/pipelex/kernel/func_ops.py@@ -0,0 +1,90 @@+"""Function-call operator semantics: registry lookup, invocation, return coercion, memory write-back.++These are the functions the interpreter's default `PipeFunc` executor calls, and the ones a+programmatic caller invokes on a `RuntimeBoot`-only process. Unlike every other operator's ops module+this one reads no hub at all: running a registered function touches no inference service, only the+process-global function registry.++`call_registered_function` is where the semantics actually live — async-vs-sync dispatch and the+return coercion that lets a function return a bare `str` or a `list` and still produce one+`StuffContent` shape downstream. It is deliberately separate from `run_func`, because the+interpreter's out-of-process executors (a sandbox, a Temporal activity) replace *the call* and keep+the write-back; `run_func` is the in-process composition of the two, and the entry point a+programmatic caller wants.++One seam is deliberately *not* here: **choosing where the function runs.** Executor selection is+configured deployment machinery, and its protocol is typed on interpreter models — see+`wip/kernel/deferred-follow-ups.md`.+"""++import asyncio+import inspect+from typing import cast++from pipelex.core.concepts.concept import Concept+from pipelex.core.memory.working_memory import WorkingMemory+from pipelex.core.stuffs.list_content import ListContent+from pipelex.core.stuffs.stuff_content import StuffContent+from pipelex.core.stuffs.text_content import TextContent+from pipelex.kernel.func_results import FuncCallResult, FuncResult+from pipelex.kernel.memory_ops import store_result+from pipelex.system.registries.func_registry import func_registry+++async def call_registered_function(*, function_name: str, memory: WorkingMemory) -> FuncCallResult:+ """Resolve a registered function by name, run it against the memory, and coerce what it returns.++ The function is arbitrary user code and its failure surface is not enumerable, so nothing is+ caught here: a caller that can add context to a failure (the interpreter's operator decorates it+ with the step's inputs and declared output) wraps this call instead.++ A sync function runs on a worker thread so it cannot block the event loop. The coercion accepts+ the three shapes the language allows an author to return — a `StuffContent`, a list of them, or a+ bare `str` — and normalizes them so every caller downstream sees one shape.+ """+ function = func_registry.get_required_function(function_name)++ if inspect.iscoroutinefunction(function):+ func_output_object = await function(working_memory=memory)+ else:+ func_output_object = await asyncio.to_thread(function, working_memory=memory)++ content: StuffContent+ if isinstance(func_output_object, StuffContent):+ content = func_output_object+ elif isinstance(func_output_object, list):+ func_result_list = cast("list[StuffContent]", func_output_object)+ content = ListContent(items=func_result_list)+ elif isinstance(func_output_object, str):+ content = TextContent(text=func_output_object)+ else:+ msg = f"Function '{function_name}' must return a StuffContent or a list, got {type(func_output_object)}"+ raise TypeError(msg)++ return FuncCallResult(+ content=content,+ function_module=getattr(function, "__module__", None),+ function_qualname=getattr(function, "__qualname__", function_name),+ )+++async def run_func(+ *,+ memory: WorkingMemory,+ function_name: str,+ concept: Concept,+ result_name: str | None = None,+ result_code: str | None = None,+) -> FuncResult:+ """A whole func step, run in this process: call the registered function and store what it returned.++ Takes the output `concept` rather than deriving one, for the reason every other op does: turning+ a concept into anything is a library's business, and the kernel is handed the resolved value.+ """+ call_result = await call_registered_function(function_name=function_name, memory=memory)+ return FuncResult(+ memory=store_result(memory=memory, concept=concept, content=call_result.content, result_name=result_name, result_code=result_code),+ content=call_result.content,+ function_module=call_result.function_module,+ function_qualname=call_result.function_qualname,+ )diff --git a/pipelex/kernel/func_results.py b/pipelex/kernel/func_results.pynew file mode 100644index 000000000..a7425a0dd--- /dev/null+++ b/pipelex/kernel/func_results.py@@ -0,0 +1,44 @@+"""What a kernel function call hands back.++Two envelopes rather than one, because two different things end: `FuncCallResult` is the outcome of+running the customer's function itself, and `FuncResult` is the outcome of a whole func step — the+call plus the memory write-back. The split is not decorative: the interpreter's pluggable executors+run the *call* somewhere else entirely (a sandbox, a Temporal activity) and hand back exactly the+first shape, so it has to exist on its own.++Both carry the function's module and qualified name. They are not cosmetic either: the execution+tracer records them, and a caller that dispatched the call out-of-process no longer holds the+function object to read them off — so they ride back on the result or they are lost.++The serialization note on `pipelex.kernel.llm_results` applies here too — `content` is annotated with+the base `StuffContent`, so use `kajson` rather than a plain `model_dump()`.+"""++from pydantic import BaseModel, ConfigDict++from pipelex.core.memory.working_memory import WorkingMemory+from pipelex.core.stuffs.stuff_content import StuffContent+++class FuncCallResult(BaseModel):+ """The outcome of running one registered function, with its return value already coerced."""++ model_config = ConfigDict(frozen=True)++ content: StuffContent+ function_module: str | None = None+ function_qualname: str | None = None+++class FuncResult(BaseModel):+ """The outcome of a kernel func call: the call's result, plus the memory it was written into.++ The memory contract holds as everywhere else: **the returned `memory` is the result.**+ """++ model_config = ConfigDict(frozen=True)++ memory: WorkingMemory+ content: StuffContent+ function_module: str | None = None+ function_qualname: str | None = Nonediff --git a/pipelex/kernel/img_gen_ops.py b/pipelex/kernel/img_gen_ops.pynew file mode 100644index 000000000..b22869bee--- /dev/null+++ b/pipelex/kernel/img_gen_ops.py@@ -0,0 +1,152 @@+"""Image-generation operator semantics: deck resolution, job-params assembly, generation, write-back.++These are the functions the interpreter's `PipeImgGen` calls, and the ones a programmatic caller+invokes on a `RuntimeBoot`-only process. They read the runtime hub and the config for what a runtime+boot stands up (the model deck, the content generator, the image-generation defaults) and take+everything else as an explicit argument.++Two seams are deliberately *not* here, both for the reason `llm_ops` states about text-vs-object+dispatch — answering them needs something the kernel does not have:++- **How many images to generate.** Multiplicity resolution reads the step's declared output against a+ per-run override, which is run-params machinery the interpreter owns; the kernel takes `nb_images`.+- **Prompt assembly.** An image prompt is built from a language-side blueprint over memory-borne image+ references, so the caller hands over a ready `ImgGenPrompt` — the same division `run_llm_text` draws+ when it takes an already-mapped `LlmPromptContent`.+"""++from typing import TYPE_CHECKING, Literal++from pipelex.cogt.content_generation.cogt_run_params import CogtRunParams+from pipelex.cogt.image.image_size import ImageSize+from pipelex.cogt.img_gen.img_gen_job_components import AspectRatio, Background, ImgGenJobParams, ImgGenSize+from pipelex.cogt.img_gen.img_gen_prompt import ImgGenPrompt+from pipelex.cogt.img_gen.img_gen_setting import ImgGenModelChoice, ImgGenSetting+from pipelex.config import get_config+from pipelex.core.concepts.concept import Concept+from pipelex.core.memory.working_memory import WorkingMemory+from pipelex.core.stuffs.image_content import ImageContent+from pipelex.core.stuffs.list_content import ListContent+from pipelex.kernel.img_gen_results import ImgGenResult+from pipelex.kernel.memory_ops import store_result+from pipelex.runtime_hub import get_content_generator, get_model_deck+from pipelex.system.job_metadata import JobMetadata+from pipelex.tools.misc.image_utils import ImageFormat++if TYPE_CHECKING:+ from pipelex.core.stuffs.stuff_content import StuffContent+++def resolve_img_gen_setting(*, img_gen_choice: ImgGenModelChoice | None = None) -> ImgGenSetting:+ """The deck chain for an image generation: the step's own choice, then the deck's default."""+ model_deck = get_model_deck()+ if img_gen_choice is not None:+ return model_deck.get_img_gen_setting(img_gen_choice)+ return model_deck.get_img_gen_setting(model_deck.img_gen_choice_default)+++def resolve_default_size(*, explicit_aspect_ratio: AspectRatio | None, default_size: ImgGenSize | None) -> ImgGenSize | None:+ """Resolve the config-level size default applicable to a step, honoring exact-size/aspect-ratio exclusivity.++ An exact size implies its own aspect ratio (the blueprint forbids setting both on a pipe), so an+ exact-size deck default does not apply to a step that explicitly sets `aspect_ratio` — the more+ specific step-level field wins. A tier default composes with any ratio and always applies.+ """+ if explicit_aspect_ratio is not None and isinstance(default_size, ImageSize):+ return None+ return default_size+++def build_img_gen_job_params(+ *,+ img_gen_setting: ImgGenSetting,+ aspect_ratio: AspectRatio | None = None,+ size: ImgGenSize | None = None,+ is_raw: bool | None = None,+ seed: int | Literal["auto"] | None = None,+ background: Background | None = None,+ output_format: ImageFormat | None = None,+) -> ImgGenJobParams:+ """Compose the per-job parameters from the resolved setting, the step's one-time values and config defaults.++ Three provenances meet here and the precedence between them is the semantics worth single-sourcing:+ quality, step count, guidance, moderation and safety come from the resolved *setting*; aspect ratio,+ size, background, rawness and seed come from the step when it names them and from the configured+ `img_gen_param_defaults` when it does not; and `output_format` is step-only — it has no configured+ default, so leaving it unset means "let the backend decide".++ `seed` accepts the literal `"auto"` that the config and the language both allow, and turns it into+ `None` — the backend reads an absent seed as "pick one". `0` is a legal seed (`ImgGenJobParams`+ constrains it with `ge=0`), so it is read as a value rather than as an absence.+ """+ img_gen_param_defaults = get_config().cogt.img_gen_config.img_gen_param_defaults+ seed_setting = seed if seed is not None else img_gen_param_defaults.seed+ resolved_seed: int | None+ if isinstance(seed_setting, str):+ resolved_seed = None+ else:+ resolved_seed = seed_setting+ return ImgGenJobParams(+ aspect_ratio=aspect_ratio or img_gen_param_defaults.aspect_ratio,+ size=size or resolve_default_size(explicit_aspect_ratio=aspect_ratio, default_size=img_gen_param_defaults.size),+ background=background or img_gen_param_defaults.background,+ quality=img_gen_setting.quality,+ nb_steps=img_gen_setting.nb_steps,+ guidance_scale=img_gen_setting.guidance_scale,+ is_moderated=img_gen_setting.is_moderated,+ safety_tolerance=img_gen_setting.safety_tolerance,+ is_raw=is_raw if is_raw is not None else img_gen_param_defaults.is_raw,+ output_format=output_format,+ seed=resolved_seed,+ )+++async def run_img_gen(+ *,+ memory: WorkingMemory,+ img_gen_prompt: ImgGenPrompt,+ img_gen_setting: ImgGenSetting,+ img_gen_job_params: ImgGenJobParams,+ concept: Concept,+ output_class: type[ImageContent],+ job_metadata: JobMetadata,+ cogt_run_params: CogtRunParams,+ nb_images: int = 1,+ result_name: str | None = None,+ result_code: str | None = None,+) -> ImgGenResult:+ """A whole image-generation step: generate, re-validate onto the output class, store, report.++ Takes the concrete `ImageContent` subclass rather than a name, for the reason `run_llm_object`+ takes `output_class`: turning a concept into a class is a library's business. Every generated+ image is re-validated onto it, because the content generator returns a plain `ImageContent` and+ the declared output concept may refine it.+ """+ img_gen_job_config = get_config().cogt.img_gen_config.img_gen_job_config+ content_generator = get_content_generator()+ content: StuffContent+ if nb_images > 1:+ image_contents = await content_generator.make_image_list(+ job_metadata=job_metadata,+ cogt_run_params=cogt_run_params,+ img_gen_handle=img_gen_setting.model,+ img_gen_prompt=img_gen_prompt,+ nb_images=nb_images,+ img_gen_job_params=img_gen_job_params,+ img_gen_job_config=img_gen_job_config,+ )+ content = ListContent(items=[output_class.model_validate(image_content.smart_dump()) for image_content in image_contents])+ else:+ image_content = await content_generator.make_single_image(+ job_metadata=job_metadata,+ cogt_run_params=cogt_run_params,+ img_gen_handle=img_gen_setting.model,+ img_gen_prompt=img_gen_prompt,+ img_gen_job_params=img_gen_job_params,+ img_gen_job_config=img_gen_job_config,+ )+ content = output_class.model_validate(image_content.smart_dump())+ return ImgGenResult(+ memory=store_result(memory=memory, concept=concept, content=content, result_name=result_name, result_code=result_code),+ content=content,+ )diff --git a/pipelex/kernel/img_gen_prompt.py b/pipelex/kernel/img_gen_prompt.pynew file mode 100644index 000000000..90c33eae9--- /dev/null+++ b/pipelex/kernel/img_gen_prompt.py@@ -0,0 +1,251 @@+"""Assemble an `ImgGenPrompt` — the image counterpart of `llm_prompt_content.assemble_llm_prompt`.++`run_img_gen` takes a *ready* `ImgGenPrompt`, by deliberate analogy with `run_llm_text` taking a ready+`LlmPromptContent`. The analogy is only honest if the kernel also ships the constructor: without this+module a `RuntimeBoot`-only process could call every other operator op and not the image one, because+the sole builder lived in `pipe_operators/` — an interpreter-layer module the kernel may not import.+`kernel/prompt_references.py` already hosts `ImageReference`, under a docstring asserting the kernel+resolves them; this is where that becomes true.++What lives here is the part a caller must not have to re-derive: the image registry, the `[Image N]`+placeholder tokens, and the correspondence between them. The registry is the single source of truth+for ordering — tokens are numbered from registry indices and `input_images` is read back from the same+registry — because a token/`input_images` order mismatch silently mislabels which image the prompt is+talking about, and nothing downstream can detect it.++What deliberately stays with the caller is `max_prompt_images`. That limit is a property of the model+a caller has chosen, its breach raises an interpreter-layer error, and checking `len(input_images)` is+a line of code — not the kind of subtlety this module exists to centralise.+"""++from typing import TYPE_CHECKING, Any, cast++from pipelex import log+from pipelex.cogt.image.prompt_image_factory import PromptImageFactory+from pipelex.cogt.img_gen.img_gen_prompt import ImgGenPrompt+from pipelex.cogt.templating.template_blueprint import TemplateBlueprint+from pipelex.cogt.templating.template_rendering import render_template+from pipelex.core.stuffs.image_content import ImageContent+from pipelex.kernel.exceptions import PromptContentError+from pipelex.kernel.prompt_references import ImageReference, ImageReferenceKind+from pipelex.tools.jinja2.image_registry import ImageRegistry+from pipelex.tools.jinja2.jinja2_models import Jinja2ContextKey+from pipelex.tools.misc.context_provider_abstract import ContextProviderAbstract+from pipelex.tools.misc.dict_utils import substitute_nested_in_context+from pipelex.tools.misc.exceptions import ContextProviderError++if TYPE_CHECKING:+ from pipelex.cogt.image.prompt_image import PromptImage+++async def assemble_img_gen_prompt(+ *,+ context_provider: ContextProviderAbstract,+ prompt_blueprint: TemplateBlueprint | None = None,+ negative_prompt_blueprint: TemplateBlueprint | None = None,+ image_references: list[ImageReference] | None = None,+ extra_params: dict[str, Any] | None = None,+) -> ImgGenPrompt:+ """Build an `ImgGenPrompt`: resolve image references, render the templates, collect the images.++ `context_provider` is typically the `WorkingMemory` the step runs against. Raises+ `PromptContentError` when an image reference names something the context does not hold, or holds+ as the wrong type.+ """+ image_registry = ImageRegistry()+ # Maps image variable name to its 0-based registry index (for placeholder generation)+ image_registry_indices: dict[str, int] = {}+ # Track which variable paths are lists, so we can substitute the whole list+ list_image_refs: list[ImageReference] = []++ if image_references:+ for image_ref in image_references:+ match image_ref.kind:+ case ImageReferenceKind.DIRECT:+ _extract_direct_image(+ image_ref=image_ref,+ context_provider=context_provider,+ image_registry=image_registry,+ image_registry_indices=image_registry_indices,+ )+ case ImageReferenceKind.DIRECT_LIST:+ _extract_direct_list_images(+ image_ref=image_ref,+ context_provider=context_provider,+ image_registry=image_registry,+ image_registry_indices=image_registry_indices,+ )+ list_image_refs.append(image_ref)+ case ImageReferenceKind.NESTED:+ # Nested images will be extracted by the | with_images filter+ # during template rendering - the registry is passed in context+ pass++ extra_params = _with_image_placeholders(+ extra_params=extra_params,+ image_registry_indices=image_registry_indices,+ list_image_refs=list_image_refs,+ )++ positive_text: str = ""+ if prompt_blueprint:+ positive_text = await _render_text(+ context_provider=context_provider,+ template_blueprint=prompt_blueprint,+ extra_params=extra_params,+ image_registry=image_registry,+ )++ negative_text: str | None = None+ if negative_prompt_blueprint:+ negative_text = await _render_text(+ context_provider=context_provider,+ template_blueprint=negative_prompt_blueprint,+ extra_params=extra_params,+ image_registry=image_registry,+ )++ # The registry contains all images (direct + nested) in the correct order, already deduplicated+ # by URL. This ensures [Image N] tokens match positions.+ input_images: list[PromptImage] | None = None+ if image_registry.images:+ input_images = [PromptImageFactory.make_prompt_image(uri=registry_image.url) for registry_image in image_registry.images]++ log.verbose(f"ImgGenPrompt: {len(input_images or [])} input images")++ return ImgGenPrompt(+ positive_text=positive_text,+ negative_text=negative_text,+ input_images=input_images,+ )+++def _with_image_placeholders(+ *,+ extra_params: dict[str, Any] | None,+ image_registry_indices: dict[str, int],+ list_image_refs: list[ImageReference],+) -> dict[str, Any]:+ """Add the `[Image N]` tokens the templates substitute, numbered from the registry."""+ params = dict(extra_params) if extra_params else {}+ if not image_registry_indices:+ return params++ # Collect list variable paths for exclusion from direct substitution+ list_variable_paths = {list_ref.variable_path for list_ref in list_image_refs}++ for image_name, registry_index in image_registry_indices.items():+ # Skip list item references (e.g., "images[1]")+ if "[" in image_name:+ continue+ # Skip dotted paths (e.g., "page.page_view") - handled by finalize/filter at render time+ if "." in image_name:+ continue+ # Skip list variable references (handled separately below)+ if image_name in list_variable_paths:+ continue+ params[image_name] = f"[Image {registry_index + 1}]"++ # For list image references, substitute the list variable itself with a string containing all+ # the [Image N] tokens for items in that list+ for list_ref in list_image_refs:+ list_tokens: list[str] = []+ for image_name, registry_index in image_registry_indices.items():+ # Check if this image belongs to this list (e.g., "collection_a[1]" belongs to "collection_a")+ if image_name.startswith(f"{list_ref.variable_path}["):+ list_tokens.append(f"[Image {registry_index + 1}]")+ if list_tokens:+ params[list_ref.variable_path] = ", ".join(list_tokens)++ return params+++def _extract_direct_image(+ *,+ image_ref: ImageReference,+ context_provider: ContextProviderAbstract,+ image_registry: ImageRegistry,+ image_registry_indices: dict[str, int],+) -> None:+ """Extract a single ImageContent from context and register it."""+ log.verbose(f"Getting direct image '{image_ref.variable_path}' from context")+ try:+ prompt_image_content = context_provider.get_typed_object_or_attribute(+ name=image_ref.variable_path,+ wanted_type=ImageContent,+ accept_list=False,+ )+ if isinstance(prompt_image_content, ImageContent):+ registry_index = image_registry.register_image(prompt_image_content)+ image_registry_indices[image_ref.variable_path] = registry_index+ else:+ msg = f"Image reference '{image_ref.variable_path}' is of type '{type(prompt_image_content).__name__}', expected ImageContent"+ raise PromptContentError(msg)+ except ContextProviderError as exc:+ msg = f"Could not find image '{image_ref.variable_path}' in context: {exc}"+ raise PromptContentError(msg) from exc+++def _extract_direct_list_images(+ *,+ image_ref: ImageReference,+ context_provider: ContextProviderAbstract,+ image_registry: ImageRegistry,+ image_registry_indices: dict[str, int],+) -> None:+ """Extract a list of ImageContent from context and register them."""+ log.verbose(f"Getting image list '{image_ref.variable_path}' from context")+ try:+ prompt_image_content = context_provider.get_typed_object_or_attribute(+ name=image_ref.variable_path,+ wanted_type=ImageContent,+ accept_list=True,+ )+ if isinstance(prompt_image_content, (list, tuple)):+ items = cast("list[Any] | tuple[Any, ...]", prompt_image_content)+ for list_position, image_item in enumerate(items, start=1):+ if not isinstance(image_item, ImageContent):+ msg = f"Item of '{image_ref.variable_path}' is of type '{type(image_item).__name__}', expected ImageContent"+ raise PromptContentError(msg)+ registry_index = image_registry.register_image(image_item)+ # Use list position (1-based) for variable name, registry index for image number+ image_item_name = f"{image_ref.variable_path}[{list_position}]"+ image_registry_indices[image_item_name] = registry_index+ else:+ msg = (+ f"Image list reference '{image_ref.variable_path}' is of type '{type(prompt_image_content).__name__}', "+ "expected list or tuple of ImageContent"+ )+ raise PromptContentError(msg)+ except ContextProviderError as exc:+ msg = f"Could not find image list '{image_ref.variable_path}' in context: {exc}"+ raise PromptContentError(msg) from exc+++async def _render_text(+ *,+ context_provider: ContextProviderAbstract,+ template_blueprint: TemplateBlueprint,+ extra_params: dict[str, Any] | None = None,+ image_registry: ImageRegistry | None = None,+) -> str:+ """Render a template with context and optional image registry."""+ context: dict[str, Any] = context_provider.generate_context()+ if extra_params:+ context = substitute_nested_in_context(context=context, extra_params=extra_params)+ if template_blueprint.extra_context:+ context.update(**template_blueprint.extra_context)++ # Add image registry to context for | with_images and | format filters+ if image_registry is not None:+ context[Jinja2ContextKey.IMAGE_REGISTRY] = image_registry++ finalize = image_registry.make_finalize() if image_registry and image_registry.images else None++ return await render_template(+ template=template_blueprint.template,+ category=template_blueprint.category,+ context=context,+ templating_style=template_blueprint.templating_style,+ finalize=finalize,+ )diff --git a/pipelex/kernel/img_gen_results.py b/pipelex/kernel/img_gen_results.pynew file mode 100644index 000000000..46b4ca11c--- /dev/null+++ b/pipelex/kernel/img_gen_results.py@@ -0,0 +1,28 @@+"""What a kernel image-generation call hands back.++Thinner than the LLM envelope on purpose: everything the interpreter's execution-graph tracer records+for an image step — the resolved model, the rendered prompts, the aspect ratio, the image count — is+either an input the caller already holds or something it resolved through the ops functions before+calling. So this carries the produced content and, above all, the memory: **the returned `memory` is+the result**, and no caller may rely on the argument having been mutated.++The serialization note on `pipelex.kernel.llm_results` applies here too — `content` is annotated with+the base `StuffContent`, so use `kajson` rather than a plain `model_dump()`.+"""++from pydantic import BaseModel, ConfigDict++from pipelex.core.memory.working_memory import WorkingMemory+from pipelex.core.stuffs.stuff_content import StuffContent+++class ImgGenResult(BaseModel):+ """The outcome of a kernel image-generation call.++ `content` is the generated image, or a `ListContent` of them when several were requested.+ """++ model_config = ConfigDict(frozen=True)++ memory: WorkingMemory+ content: StuffContentdiff --git a/pipelex/kernel/llm_ops.py b/pipelex/kernel/llm_ops.pyindex da797eb87..36a2e2f3a 100644--- a/pipelex/kernel/llm_ops.py+++ b/pipelex/kernel/llm_ops.py@@ -29,9 +29,9 @@ from pipelex.core.concepts.concept import Conceptfrom pipelex.core.memory.working_memory import WorkingMemoryfrom pipelex.core.stuffs.list_content import ListContentfrom pipelex.core.stuffs.stuff_content import StuffContent-from pipelex.core.stuffs.stuff_factory import StuffFactoryfrom pipelex.kernel.llm_prompt_content import LlmPromptContent, assemble_llm_promptfrom pipelex.kernel.llm_results import LlmObjectResult, LlmTextResult, StructuringPath+from pipelex.kernel.memory_ops import store_resultfrom pipelex.runtime_hub import get_content_generator, get_model_deckfrom pipelex.system.job_metadata import JobMetadatafrom pipelex.tools.jinja2.template_category import TemplateCategory@@ -135,25 +135,6 @@ async def generate_object_content()-def store_result(- *,- memory: WorkingMemory,- concept: Concept,- content: StuffContent,- result_name: str | None = None,- result_code: str | None = None,-) -> WorkingMemory:- """Write a produced content into memory as the new main stuff, and return the memory.-- Returning it is the contract, not a convenience: the argument is mutated today because inline- execution aliases the two, and a caller that relies on that aliasing breaks the moment a- serialization boundary sits between them.- """- stuff = StuffFactory.make_stuff(concept=concept, content=content, name=result_name, code=result_code)- memory.set_new_main_stuff(stuff=stuff, name=result_name)- return memory--async def run_llm_text(*,memory: WorkingMemory,diff --git a/pipelex/kernel/memory_ops.py b/pipelex/kernel/memory_ops.pynew file mode 100644index 000000000..18af07a64--- /dev/null+++ b/pipelex/kernel/memory_ops.py@@ -0,0 +1,31 @@+"""Memory write-back, shared by every operator's kernel ops.++Each operator ends the same way — wrap the produced content in a `Stuff` and make it the memory's new+main stuff — so that ending lives here once rather than at each domain's ops module. It is also where+the memory contract is actually implemented, which is why a domain module never inlines the two calls:+the day the contract stops aliasing argument and return, exactly one function changes.+"""++from pipelex.core.concepts.concept import Concept+from pipelex.core.memory.working_memory import WorkingMemory+from pipelex.core.stuffs.stuff_content import StuffContent+from pipelex.core.stuffs.stuff_factory import StuffFactory+++def store_result(+ *,+ memory: WorkingMemory,+ concept: Concept,+ content: StuffContent,+ result_name: str | None = None,+ result_code: str | None = None,+) -> WorkingMemory:+ """Write a produced content into memory as the new main stuff, and return the memory.++ Returning it is the contract, not a convenience: the argument is mutated today because inline+ execution aliases the two, and a caller that relies on that aliasing breaks the moment a+ serialization boundary sits between them.+ """+ stuff = StuffFactory.make_stuff(concept=concept, content=content, name=result_name, code=result_code)+ memory.set_new_main_stuff(stuff=stuff, name=result_name)+ return memorydiff --git a/pipelex/kernel/search_ops.py b/pipelex/kernel/search_ops.pynew file mode 100644index 000000000..01be4bef2--- /dev/null+++ b/pipelex/kernel/search_ops.py@@ -0,0 +1,112 @@+"""Search operator semantics: deck resolution, query rendering, search, memory write-back.++These are the functions the interpreter's `PipeSearch` calls, and the ones a programmatic caller+invokes on a `RuntimeBoot`-only process. They read the runtime hub for the services a runtime boot+stands up (the model deck, the content generator) and take everything else as an explicit argument.++The sourced-answer / structured fork is the caller's, made explicit the way `llm_ops` makes the+text-vs-object fork explicit: pass `output_structure_class` and the search comes back structured onto+it, leave it out and it comes back as a sourced answer. Resolving a concept to that class is a+library's business, so the kernel never asks for one.+"""++from pipelex.cogt.content_generation.assignment_models import SearchAssignment+from pipelex.cogt.content_generation.cogt_run_params import CogtRunParams+from pipelex.cogt.model_backends.model_type import ModelType+from pipelex.cogt.search.search_setting import SearchModelChoice, SearchSetting+from pipelex.cogt.templating.template_rendering import render_template+from pipelex.core.concepts.concept import Concept+from pipelex.core.memory.working_memory import WorkingMemory+from pipelex.core.stuffs.stuff_content import StuffContent+from pipelex.kernel.memory_ops import store_result+from pipelex.kernel.search_results import SearchResult+from pipelex.runtime_hub import get_content_generator, get_model_deck+from pipelex.system.job_metadata import JobMetadata+from pipelex.tools.jinja2.template_category import TemplateCategory+++def resolve_search_setting(+ *,+ search_choice: SearchModelChoice | None = None,+ include_images_override: bool | None = None,+ max_results_override: int | None = None,+) -> SearchSetting:+ """The deck chain for a search, then handle resolution, then the step's own overrides.++ The middle step is the one worth naming: the deck's setting may name a waterfall or an alias, and+ the returned setting is pinned to the *resolved* handle so it doubles as a distributed run's+ routing key. A caller that skipped it would route on a name no worker recognises.+ """+ model_deck = get_model_deck()+ resolved_choice = search_choice or model_deck.search_choice_default+ search_setting = model_deck.get_search_setting(search_choice=resolved_choice)++ inference_model = model_deck.get_required_inference_model(model_handle=search_setting.model, model_type=ModelType.SEARCH)+ if inference_model.name != search_setting.model:+ search_setting = search_setting.model_copy(update={"model": inference_model.name})++ if include_images_override is not None:+ search_setting = search_setting.model_copy(update={"include_images": include_images_override})+ if max_results_override is not None:+ search_setting = search_setting.model_copy(update={"max_results": max_results_override})+ return search_setting+++async def run_search(+ *,+ memory: WorkingMemory,+ template: str,+ category: TemplateCategory,+ search_setting: SearchSetting,+ concept: Concept,+ job_metadata: JobMetadata,+ cogt_run_params: CogtRunParams,+ output_structure_class: type[StuffContent] | None = None,+ include_domains: list[str] | None = None,+ exclude_domains: list[str] | None = None,+ from_date: str | None = None,+ to_date: str | None = None,+ result_name: str | None = None,+ result_code: str | None = None,+) -> SearchResult:+ """A whole search step: render the query, search, store, report.++ The query template arrives as source plus category rather than as a `TemplateBlueprint`, the way+ `run_compose_template` takes it: a search renders against memory alone, with no style and no extra+ context, so a blueprint would hand over two fields this function has nothing to do with.++ The search itself goes through the same content-generation seam as the LLM, image-generation and+ extract leaves — direct inline, an activity when in-workflow, or a dry mock — which is what makes+ it replay-safe under a distributed orchestrator and lets its failures cross a workflow boundary as+ classified errors instead of hanging.+ """+ query_text = await render_template(+ template=template,+ category=category,+ context=memory.generate_context(),+ )+ search_assignment = SearchAssignment(+ job_metadata=job_metadata,+ cogt_run_params=cogt_run_params,+ query=query_text,+ search_setting=search_setting,+ include_domains=include_domains,+ exclude_domains=exclude_domains,+ from_date=from_date,+ to_date=to_date,+ )+ content_generator = get_content_generator()+ content: StuffContent+ if output_structure_class is None:+ content = await content_generator.make_search_sourced_answer(search_assignment=search_assignment)+ else:+ content = await content_generator.make_search_structured(+ output_structure_class=output_structure_class,+ search_assignment=search_assignment,+ )+ return SearchResult(+ memory=store_result(memory=memory, concept=concept, content=content, result_name=result_name, result_code=result_code),+ content=content,+ rendered_query=query_text,+ search_setting=search_setting,+ )diff --git a/pipelex/kernel/search_results.py b/pipelex/kernel/search_results.pynew file mode 100644index 000000000..101b70dc9--- /dev/null+++ b/pipelex/kernel/search_results.py@@ -0,0 +1,31 @@+"""What a kernel search call hands back.++Two intermediates ride along because the kernel is where they are produced and the interpreter's+execution-graph tracer records them: the rendered query, and the `SearchSetting` *after* handle+resolution and pipe-level overrides — which is not the one the caller passed in, so it has to come+back. The memory contract holds as everywhere else: **the returned `memory` is the result**.++The serialization note on `pipelex.kernel.llm_results` applies here too — `content` is annotated with+the base `StuffContent`, so use `kajson` rather than a plain `model_dump()`.+"""++from pydantic import BaseModel, ConfigDict++from pipelex.cogt.search.search_setting import SearchSetting+from pipelex.core.memory.working_memory import WorkingMemory+from pipelex.core.stuffs.stuff_content import StuffContent+++class SearchResult(BaseModel):+ """The outcome of a kernel search call.++ `content` is a sourced answer, or an instance of the requested structure class when one was asked+ for.+ """++ model_config = ConfigDict(frozen=True)++ memory: WorkingMemory+ content: StuffContent+ rendered_query: str+ search_setting: SearchSettingdiff --git a/pipelex/pipe_operators/compose/pipe_compose.py b/pipelex/pipe_operators/compose/pipe_compose.pyindex cdd90d02f..b81ffe652 100644--- a/pipelex/pipe_operators/compose/pipe_compose.py+++ b/pipelex/pipe_operators/compose/pipe_compose.py@@ -6,7 +6,6 @@ from typing_extensions import overridefrom pipelex import logfrom pipelex.cogt.content_generation.dry_run_factory import DryRunFactoryfrom pipelex.cogt.templating.template_preprocessor import rewrite_template_sigils-from pipelex.cogt.templating.template_rendering import render_templatefrom pipelex.config import get_configfrom pipelex.core.concepts.native.concept_native import NativeConceptCodefrom pipelex.core.memory.working_memory import WorkingMemory@@ -14,10 +13,10 @@ from pipelex.core.pipes.exceptions import PipeValidationError, PipeValidationErrfrom pipelex.core.pipes.inputs.input_stuff_specs import InputStuffSpecsfrom pipelex.core.pipes.inputs.input_stuff_specs_factory import InputStuffSpecsFactoryfrom pipelex.core.pipes.pipe_output import PipeOutput-from pipelex.core.stuffs.html_content import HtmlContentfrom pipelex.core.stuffs.stuff_content import StuffContentfrom pipelex.core.stuffs.stuff_factory import StuffFactoryfrom pipelex.interpreter_hub import get_concept_library, get_native_concept+from pipelex.kernel.compose_ops import run_compose_templatefrom pipelex.pipe_machinery.template_guard_lint import lint_optional_input_guardsfrom pipelex.pipe_operators.compose.construct_blueprint import ConstructBlueprintfrom pipelex.pipe_operators.compose.exceptions import PipeComposeError, StructuredContentComposerValueError@@ -192,44 +191,30 @@ class PipeCompose(PipeOperator[PipeComposeOutput]):msg = "Template is required for template mode"raise ValueError(msg)- context: dict[str, Any] = working_memory.generate_context()- if pipe_run_params:- context.update(**pipe_run_params.params)- if self.extra_context:- context.update(**self.extra_context)-- rendered_template_text = await render_template(- template=self.template,- category=self.category,- context=context,- templating_style=self.templating_style,- )- log.verbose(f"Template rendered text:\n{rendered_template_text}")- assert isinstance(rendered_template_text, str)-- # Get the structure class from the registry (might be a subclass of TextContent or HtmlContent)+ # Resolved here rather than inside the kernel: turning a concept into a class is a library's+ # business, and handing the class over is what spares the kernel a registry read.structure_class = get_class_registry().get_required_subclass(name=self.output.concept.structure_class_name,base_class=StuffContent,)-- # Construct content based on the structure class type- if issubclass(structure_class, HtmlContent):- the_content = structure_class(inner_html=rendered_template_text, css_class="")- else:- the_content = structure_class(text=rendered_template_text)-- output_stuff = StuffFactory.make_stuff(concept=self.output.concept, content=the_content, name=output_name)-- working_memory.set_new_main_stuff(- stuff=output_stuff,- name=output_name,+ compose_result = await run_compose_template(+ memory=working_memory,+ template=self.template,+ category=self.category,+ concept=self.output.concept,+ output_class=structure_class,+ templating_style=self.templating_style,+ runtime_params=pipe_run_params.params,+ extra_context=self.extra_context,+ result_name=output_name,)+ working_memory = compose_result.memory+ log.verbose(f"Template rendered text:\n{compose_result.rendered_text}")# Capture execution data for the graph tracerexecution_data_dict: dict[str, Any] = {"compose_mode": "template",- "rendered_text": rendered_template_text,+ "rendered_text": compose_result.rendered_text,}self._register_execution_data(job_metadata=job_metadata, execution_data=execution_data_dict)@@ -264,7 +249,6 @@ class PipeCompose(PipeOperator[PipeComposeOutput]):output_class=output_class,runtime_params=pipe_run_params.params if pipe_run_params else None,extra_context=self.extra_context,- pipe_run_params=pipe_run_params,)try:the_content = await composer.compose()diff --git a/pipelex/pipe_operators/compose/structured_content_composer.py b/pipelex/pipe_operators/compose/structured_content_composer.pyindex 8ea104b31..f065bc47d 100644--- a/pipelex/pipe_operators/compose/structured_content_composer.py+++ b/pipelex/pipe_operators/compose/structured_content_composer.py@@ -13,13 +13,13 @@ from pipelex.core.stuffs.number_content import NumberContentfrom pipelex.core.stuffs.stuff_content import StuffContentfrom pipelex.core.stuffs.text_content import TextContentfrom pipelex.core.stuffs.yes_no_content import YesNoContent+from pipelex.kernel.compose_ops import build_compose_contextfrom pipelex.pipe_operators.compose.construct_blueprint import ConstructBlueprint, ConstructFieldBlueprint, ConstructFieldMethodfrom pipelex.pipe_operators.compose.exceptions import (StructuredContentComposerTypeError,StructuredContentComposerValidationError,StructuredContentComposerValueError,)-from pipelex.pipe_run.pipe_run_params import PipeRunParamsfrom pipelex.tools.jinja2.template_category import TemplateCategoryfrom pipelex.tools.typing.annotation_utils import unwrap_optionalfrom pipelex.tools.typing.class_utils import are_classes_equivalent@@ -59,7 +59,6 @@ class StructuredContentComposer:output_class: The StructuredContent subclass to instantiateruntime_params: Additional runtime parameters for template context (from PipeRunParams.params)extra_context: Extra context values for template rendering (from PipeCompose.extra_context)- pipe_run_params: The pipe run parameters (used to check if we're in dry run mode)"""def __init__(@@ -69,14 +68,12 @@ class StructuredContentComposer:output_class: type[StuffContent],runtime_params: dict[str, Any] | None = None,extra_context: dict[str, Any] | None = None,- pipe_run_params: PipeRunParams | None = None,):self.construct_blueprint = construct_blueprintself.working_memory = working_memoryself.output_class = output_classself.runtime_params = runtime_params or {}self.extra_context = extra_context or {}- self.pipe_run_params = pipe_run_params# Per-field record of how each field was built. Populated as fields resolve.# Shape per entry: {"method": ConstructFieldMethod, "rendered": str (templates only)}.# Nested fields record only their method; their sub-fields are not surfaced.@@ -776,21 +773,17 @@ class StructuredContentComposer:msg = "template is required for TEMPLATE method"raise StructuredContentComposerValueError(msg)- # Build context consistently with _run_template_mode in PipeCompose:- # 1. Working memory context (stuffs as variables)- context: dict[str, Any] = self.working_memory.generate_context()- # 2. Runtime params (from PipeRunParams.params)- if self.runtime_params:- context.update(**self.runtime_params)- # 3. Extra context (from PipeCompose.extra_context)- if self.extra_context:- context.update(**self.extra_context)-+ # The same three-layer context PipeCompose's template mode renders against, built by the same+ # kernel function so the layering order cannot fork between the two paths.# TODO: dry-run templating is being removed — this direct render_template call is intentionalreturn await render_template(template=field_blueprint.template,category=TemplateCategory.BASIC,- context=context,+ context=build_compose_context(+ memory=self.working_memory,+ runtime_params=self.runtime_params,+ extra_context=self.extra_context,+ ),)async def _resolve_nested(self, field_blueprint: ConstructFieldBlueprint, *, field_name: str) -> StuffContent:@@ -817,7 +810,6 @@ class StructuredContentComposer:output_class=nested_class,runtime_params=self.runtime_params,extra_context=self.extra_context,- pipe_run_params=self.pipe_run_params,)return await nested_composer.compose()diff --git a/pipelex/pipe_operators/extract/pipe_extract.py b/pipelex/pipe_operators/extract/pipe_extract.pyindex 35bddbd2c..7333d35c6 100644--- a/pipelex/pipe_operators/extract/pipe_extract.py+++ b/pipelex/pipe_operators/extract/pipe_extract.py@@ -1,29 +1,23 @@-from typing import TYPE_CHECKING, Any, Literal+from typing import Any, Literalfrom pydantic import model_validatorfrom typing_extensions import Self, overridefrom pipelex.cogt.exceptions import ModelChoiceNotFoundErrorfrom pipelex.cogt.extract.extract_input import ExtractInput-from pipelex.cogt.extract.extract_job_components import ExtractJobConfig, ExtractJobParams-from pipelex.cogt.extract.extract_setting import ExtractModelChoice, ExtractSetting+from pipelex.cogt.extract.extract_setting import ExtractModelChoicefrom pipelex.cogt.models.model_deck_check import check_extract_choice_with_deckfrom pipelex.core.concepts.native.concept_native import NativeConceptCodefrom pipelex.core.memory.working_memory import WorkingMemoryfrom pipelex.core.pipes.exceptions import PipeValidationError, PipeValidationErrorTypefrom pipelex.core.pipes.inputs.input_stuff_specs import InputStuffSpecsfrom pipelex.core.pipes.pipe_output import PipeOutput-from pipelex.core.stuffs.list_content import ListContent-from pipelex.core.stuffs.stuff_factory import StuffFactoryfrom pipelex.interpreter_hub import get_concept_library, get_native_concept+from pipelex.kernel.extract_ops import build_extract_job_params, resolve_extract_setting, run_extractfrom pipelex.pipe_operators.pipe_operator import PipeOperatorfrom pipelex.pipe_run.pipe_run_params import PipeRunParams-from pipelex.runtime_hub import get_content_generator, get_model_deckfrom pipelex.system.job_metadata import JobMetadata-if TYPE_CHECKING:- from pipelex.core.stuffs.page_content import PageContent-class PipeExtractOutput(PipeOutput):pass@@ -119,8 +113,6 @@ class PipeExtract(PipeOperator[PipeExtractOutput]):pipe_run_params: PipeRunParams,output_name: str | None = None,) -> PipeExtractOutput:- content_generator = get_content_generator()-image_uri: str | None = Nonepdf_uri: str | None = Noneif self.image_stuff_name:@@ -130,46 +122,30 @@ class PipeExtract(PipeOperator[PipeExtractOutput]):document_stuff = working_memory.get_stuff_as_document(name=self.document_stuff_name)pdf_uri = document_stuff.url- extract_choice: ExtractModelChoice = self.extract_choice or get_model_deck().extract_choice_default- extract_setting: ExtractSetting = get_model_deck().get_extract_setting(extract_choice=extract_choice)-- # MTHDS-level max_page_images takes precedence if set, otherwise use ExtractSetting- max_nb_images = self.max_page_images if self.max_page_images is not None else extract_setting.max_nb_images-- extract_job_params = ExtractJobParams(+ # The deck chain and the job-params composition are kernel semantics; the setting is resolved+ # per run into a local and never cached onto `self`, for the reason `pipe_llm.py` states about+ # its own settings — the pipe instance is the one the library holds and hands out.+ extract_setting = resolve_extract_setting(extract_choice=self.extract_choice)+ extract_job_params = build_extract_job_params(+ extract_setting=extract_setting,should_caption_images=self.should_caption_images,should_include_page_views=self.should_include_page_views,page_views_dpi=self.page_views_dpi,- max_nb_images=max_nb_images,- image_min_size=extract_setting.image_min_size,+ max_page_images=self.max_page_images,render_js=self.render_js,include_raw_html=self.include_raw_html,)- extract_input = ExtractInput(- image_uri=image_uri,- document_uri=pdf_uri,- )- page_contents = await content_generator.make_extract_pages(- extract_input=extract_input,- cogt_run_params=pipe_run_params.cogt_run_params,- extract_handle=extract_setting.model,- job_metadata=job_metadata,+ extract_result = await run_extract(+ memory=working_memory,+ extract_input=ExtractInput(image_uri=image_uri, document_uri=pdf_uri),+ extract_setting=extract_setting,extract_job_params=extract_job_params,- extract_job_config=ExtractJobConfig(),- )-- content: ListContent[PageContent] = ListContent(items=page_contents)-- output_stuff = StuffFactory.make_stuff(- name=output_name,concept=self.output.concept,- content=content,- )-- working_memory.set_new_main_stuff(- stuff=output_stuff,- name=output_name,+ job_metadata=job_metadata,+ cogt_run_params=pipe_run_params.cogt_run_params,+ result_name=output_name,)+ working_memory = extract_result.memory# Capture execution data for the graph tracerexecution_data_dict: dict[str, Any] = {diff --git a/pipelex/pipe_operators/func/direct_pipe_func_executor.py b/pipelex/pipe_operators/func/direct_pipe_func_executor.pyindex fbb78c7cd..0d10dadd8 100644--- a/pipelex/pipe_operators/func/direct_pipe_func_executor.py+++ b/pipelex/pipe_operators/func/direct_pipe_func_executor.py@@ -1,10 +1,8 @@import asyncio-import inspectimport shutilimport sysimport tempfilefrom pathlib import Path-from typing import castfrom typing_extensions import override@@ -12,12 +10,10 @@ from pipelex import logfrom pipelex.codegen.emitters.python_structures import emit_python_structuresfrom pipelex.codegen.resolved_concepts import resolve_concepts_from_cratefrom pipelex.core.memory.working_memory import WorkingMemory-from pipelex.core.stuffs.list_content import ListContentfrom pipelex.core.stuffs.structured_content import StructuredContent-from pipelex.core.stuffs.stuff_content import StuffContentfrom pipelex.core.stuffs.stuff_factory import StuffFactory-from pipelex.core.stuffs.text_content import TextContentfrom pipelex.interpreter_hub import get_library_manager+from pipelex.kernel.func_ops import call_registered_functionfrom pipelex.pipe_operators.func.exceptions import PipeFuncExecutionErrorfrom pipelex.pipe_operators.func.pipe_func_execution_dtos import PipeFuncExecutionRequest, PipeFuncExecutionResponsefrom pipelex.pipe_operators.func.pipe_func_executor_protocol import PipeFuncExecutionResult, PipeFuncExecutorProtocol@@ -25,7 +21,6 @@ from pipelex.pipe_run.pipe_run_params import PipeRunParamsfrom pipelex.runtime_bridge.primitives.rehydration import rehydrate_library_and_memoryfrom pipelex.system.job_metadata import JobMetadatafrom pipelex.system.registries.class_registry_utils import ClassRegistryUtils-from pipelex.system.registries.func_registry import func_registryfrom pipelex.system.registries.func_registry_utils import FuncRegistryUtils# The library id the transported run rebuilds under. Internal to one process (a fresh sandbox box or a@@ -37,10 +32,14 @@ class DirectPipeFuncExecutor(PipeFuncExecutorProtocol):"""The ``direct`` execution mode: resolve the function from the process-global registry and run it here.The free, in-process, trusted mode core owns (mirrors ``DirectOrchestrator`` on the orchestration- axis). This is the pre-existing PipeFunc behavior, extracted verbatim from ``_live_run_operator_pipe``:- async vs sync dispatch, and str/list -> StuffContent coercion. It is used for local runs and inside- a sandbox box (where the real function IS registered); a sandbox-dispatching executor from the- out-of-tree plugin runs it out-of-process instead when ``execution_mode`` names a sandbox backend.+ axis). It is used for local runs and inside a sandbox box (where the real function IS registered);+ a sandbox-dispatching executor from the out-of-tree plugin runs it out-of-process instead when+ ``execution_mode`` names a sandbox backend.++ What running a function *means* — the registry lookup, async-vs-sync dispatch, and the+ str/list -> StuffContent coercion — is the kernel's ``call_registered_function``, so this executor+ is what is left once that is factored out: the in-process wiring of the live arm, and the whole+ transported arm below."""@override@@ -54,32 +53,16 @@ class DirectPipeFuncExecutor(PipeFuncExecutorProtocol):pipe_run_params: PipeRunParams,) -> PipeFuncExecutionResult:log.verbose(f"Running PipeFunc in-process with function '{function_name}'")- function = func_registry.get_required_function(function_name)-- # The function is arbitrary user code; its failure surface is not enumerable. We do NOT catch- # here: the operator wraps this call and decorates the failure with the pipe's inputs/output- # for a diagnostic PipeRunError (preserving the pre-refactor behavior).- if inspect.iscoroutinefunction(function):- func_output_object = await function(working_memory=working_memory)- else:- func_output_object = await asyncio.to_thread(function, working_memory=working_memory)-- the_content: StuffContent- if isinstance(func_output_object, StuffContent):- the_content = func_output_object- elif isinstance(func_output_object, list):- func_result_list = cast("list[StuffContent]", func_output_object)- the_content = ListContent(items=func_result_list)- elif isinstance(func_output_object, str):- the_content = TextContent(text=func_output_object)- else:- msg = f"Function '{function_name}' must return a StuffContent or a list, got {type(func_output_object)}"- raise TypeError(msg)-+ # The call itself — registry lookup, async/sync dispatch, return coercion — is the kernel's,+ # so an out-of-process executor and this one share one definition of what running a function+ # means. Nothing is caught here: the function is arbitrary user code whose failure surface is+ # not enumerable, and the operator wraps this call to decorate the failure with the pipe's+ # inputs/output for a diagnostic PipeRunError.+ call_result = await call_registered_function(function_name=function_name, memory=working_memory)return PipeFuncExecutionResult(- content=the_content,- function_module=getattr(function, "__module__", None),- function_qualname=getattr(function, "__qualname__", function_name),+ content=call_result.content,+ function_module=call_result.function_module,+ function_qualname=call_result.function_qualname,)@overridediff --git a/pipelex/pipe_operators/func/pipe_func.py b/pipelex/pipe_operators/func/pipe_func.pyindex 709492db8..a907039c5 100644--- a/pipelex/pipe_operators/func/pipe_func.py+++ b/pipelex/pipe_operators/func/pipe_func.py@@ -16,6 +16,7 @@ from pipelex.core.stuffs.list_content import ListContentfrom pipelex.core.stuffs.stuff_content import StuffContentfrom pipelex.core.stuffs.stuff_factory import StuffFactoryfrom pipelex.interpreter_hub import get_pipe_func_executor+from pipelex.kernel.memory_ops import store_resultfrom pipelex.pipe_operators.pipe_operator import PipeOperatorfrom pipelex.pipe_run.pipe_run_params import PipeRunParamsfrom pipelex.runtime_hub import get_class_registry@@ -225,15 +226,11 @@ class PipeFunc(PipeOperator[PipeFuncOutput]):the_content = execution_result.content- output_stuff = StuffFactory.make_stuff(- name=output_name,+ working_memory = store_result(+ memory=working_memory,concept=self.output.concept,content=the_content,- )-- working_memory.set_new_main_stuff(- stuff=output_stuff,- name=output_name,+ result_name=output_name,)# Capture execution data for the graph tracer. PipeFunc has no prompts ordiff --git a/pipelex/pipe_operators/img_gen/img_gen_prompt_blueprint.py b/pipelex/pipe_operators/img_gen/img_gen_prompt_blueprint.pyindex e975f882d..5bf13acd5 100644--- a/pipelex/pipe_operators/img_gen/img_gen_prompt_blueprint.py+++ b/pipelex/pipe_operators/img_gen/img_gen_prompt_blueprint.py@@ -1,46 +1,39 @@-"""Blueprint for building image generation prompts with image extraction support.+"""Blueprint for image-generation prompts: the authored shape, and its input contract.-This module provides the ImgGenPromptBlueprint class which handles:-- Template rendering for positive/negative prompts-- Image reference extraction (direct, list, and nested)-- Placeholder generation for image references+Parse-and-validate lives here; the *assembly* — image registry, `[Image N]` placeholders, template+rendering — moved to `pipelex.kernel.img_gen_prompt`, so a caller with no interpreter can build an+`ImgGenPrompt` too. This class keeps what is genuinely blueprint-shaped: the authored fields, the+declared input variables, and the `max_prompt_images` limit, whose breach is an interpreter-layer+error about the model the pipe chose."""-from typing import TYPE_CHECKING, Any, cast+from typing import Anyfrom pydantic import BaseModel-from pipelex import log-from pipelex.cogt.image.prompt_image_factory import PromptImageFactoryfrom pipelex.cogt.img_gen.img_gen_prompt import ImgGenPromptfrom pipelex.cogt.templating.template_blueprint import TemplateBlueprint-from pipelex.cogt.templating.template_rendering import render_template-from pipelex.core.stuffs.image_content import ImageContent-from pipelex.kernel.prompt_references import ImageReference, ImageReferenceKind+from pipelex.kernel.img_gen_prompt import assemble_img_gen_prompt+from pipelex.kernel.prompt_references import ImageReferencefrom pipelex.pipe_operators.img_gen.exceptions import PipeImgGenFactoryError-from pipelex.tools.jinja2.image_registry import ImageRegistry-from pipelex.tools.jinja2.jinja2_models import Jinja2ContextKeyfrom pipelex.tools.misc.context_provider_abstract import ContextProviderAbstract-from pipelex.tools.misc.dict_utils import substitute_nested_in_context-from pipelex.tools.misc.exceptions import ContextProviderErrorfrom pipelex.tools.misc.string_utils import get_root_from_dotted_path-if TYPE_CHECKING:- from pipelex.cogt.image.prompt_image import PromptImage+class ImgGenPromptBlueprint(BaseModel):+ """The authored shape of an image-generation prompt, and its input contract.-class ImgGenPromptBlueprintValueError(ValueError):- """Raised when there's an issue extracting values for img_gen prompt blueprint."""-+ This class no longer assembles anything. Template rendering, image-reference extraction,+ `[Image N]` placeholder generation and `input_images` collection all live in+ `pipelex.kernel.img_gen_prompt.assemble_img_gen_prompt`, so a caller with no interpreter can+ build an `ImgGenPrompt` too. What stays here is what is genuinely blueprint-shaped:-class ImgGenPromptBlueprint(BaseModel):- """Blueprint for building ImgGenPrompt with optional image inputs.+ - the authored fields (`prompt_blueprint`, `negative_prompt_blueprint`, `image_references`)+ - the input variables they declare (`required_variables`)+ - the `max_prompt_images` limit, whose breach is an interpreter-layer error about the model+ the pipe chose, and which is therefore applied here rather than in the kernel- This class handles:- - Template rendering for positive and negative prompts- - Image reference extraction (direct images, image lists, nested images)- - [Image N] placeholder generation in prompt text- - Collecting all images for the ImgGenPrompt.input_images field+ Add assembly behavior to the kernel module, not to this class."""prompt_blueprint: TemplateBlueprint | None = None@@ -69,228 +62,27 @@ class ImgGenPromptBlueprint(BaseModel):extra_params: dict[str, Any] | None = None,max_prompt_images: int | None = None,) -> ImgGenPrompt:- """Build an ImgGenPrompt from the blueprint.-- Args:- context_provider: Provider for variable values (typically WorkingMemory)- extra_params: Additional parameters for template substitution- max_prompt_images: Maximum number of images allowed (for validation)-- Returns:- ImgGenPrompt with rendered text and extracted input images+ """Build an `ImgGenPrompt` from the blueprint, delegating the assembly to the kernel.- Raises:- ImgGenPromptBlueprintValueError: If image extraction fails- PipeImgGenFactoryError: If max_prompt_images is exceeded+ Raises `PromptContentError` (from the kernel) if an image reference cannot be resolved, and+ `PipeImgGenFactoryError` if more images were collected than the chosen model accepts."""- ############################################################- # Image Registry and Direct Image Extraction- ############################################################- image_registry = ImageRegistry()- # Maps image variable name to its 0-based registry index (for placeholder generation)- image_registry_indices: dict[str, int] = {}- # Track which variable paths are lists, so we can substitute the whole list- list_image_refs: list[ImageReference] = []-- # Process image references (DIRECT and DIRECT_LIST kinds)- if self.image_references:- for image_ref in self.image_references:- match image_ref.kind:- case ImageReferenceKind.DIRECT:- self._extract_direct_image(- image_ref=image_ref,- context_provider=context_provider,- image_registry=image_registry,- image_registry_indices=image_registry_indices,- )- case ImageReferenceKind.DIRECT_LIST:- self._extract_direct_list_images(- image_ref=image_ref,- context_provider=context_provider,- image_registry=image_registry,- image_registry_indices=image_registry_indices,- )- list_image_refs.append(image_ref)- case ImageReferenceKind.NESTED:- # Nested images will be extracted by the | with_images filter- # during template rendering - the registry is passed in context- pass-- ############################################################- # Prepare extra_params with image placeholders- ############################################################- extra_params = dict(extra_params) if extra_params else {}- if image_registry_indices:- # Collect list variable paths for exclusion from direct substitution- list_variable_paths = {list_ref.variable_path for list_ref in list_image_refs}-- # Add placeholders for direct (non-dotted, non-list) images- for image_name, registry_index in image_registry_indices.items():- # Skip list item references (e.g., "images[1]")- if "[" in image_name:- continue- # Skip dotted paths (e.g., "page.page_view") - handled by finalize/filter at render time- if "." in image_name:- continue- # Skip list variable references (handled separately below)- if image_name in list_variable_paths:- continue- # Add direct image placeholder- extra_params[image_name] = f"[Image {registry_index + 1}]"-- # For list image references, substitute the list variable itself- # with a string containing all the [Image N] tokens for items in that list- for list_ref in list_image_refs:- list_tokens: list[str] = []- for image_name, registry_index in image_registry_indices.items():- # Check if this image belongs to this list (e.g., "collection_a[1]" belongs to "collection_a")- if image_name.startswith(f"{list_ref.variable_path}["):- list_tokens.append(f"[Image {registry_index + 1}]")- if list_tokens:- extra_params[list_ref.variable_path] = ", ".join(list_tokens)-- ############################################################- # Render prompt text- ############################################################- positive_text: str = ""- if self.prompt_blueprint:- positive_text = await self._render_text(- context_provider=context_provider,- template_blueprint=self.prompt_blueprint,- extra_params=extra_params,- image_registry=image_registry,- )-- negative_text: str | None = None- if self.negative_prompt_blueprint:- negative_text = await self._render_text(- context_provider=context_provider,- template_blueprint=self.negative_prompt_blueprint,- extra_params=extra_params,- image_registry=image_registry,- )-- ############################################################- # Collect all images from registry (single source of truth)- ############################################################- # The registry contains all images (direct + nested) in the correct order,- # already deduplicated by URL. This ensures [Image N] tokens match positions.- input_images: list[PromptImage] | None = None- if image_registry.images:- input_images = [PromptImageFactory.make_prompt_image(uri=registry_image.url) for registry_image in image_registry.images]-- # Validate against max_prompt_images if specified- if max_prompt_images is not None and len(input_images) > max_prompt_images:- msg = (- f"Too many input images: got {len(input_images)}, but model only supports {max_prompt_images}. "- f"Reduce the number of images in your template."- )- raise PipeImgGenFactoryError(msg)-- log.verbose(f"ImgGenPrompt: {len(input_images or [])} input images")-- return ImgGenPrompt(- positive_text=positive_text,- negative_text=negative_text,- input_images=input_images,+ img_gen_prompt = await assemble_img_gen_prompt(+ context_provider=context_provider,+ prompt_blueprint=self.prompt_blueprint,+ negative_prompt_blueprint=self.negative_prompt_blueprint,+ image_references=self.image_references,+ extra_params=extra_params,)- def _extract_direct_image(- self,- image_ref: ImageReference,- *,- context_provider: ContextProviderAbstract,- image_registry: ImageRegistry,- image_registry_indices: dict[str, int],- ) -> None:- """Extract a single ImageContent from context and register it."""- log.verbose(f"Getting direct image '{image_ref.variable_path}' from context")- try:- prompt_image_content = context_provider.get_typed_object_or_attribute(- name=image_ref.variable_path,- wanted_type=ImageContent,- accept_list=False,- )- if isinstance(prompt_image_content, ImageContent):- registry_index = image_registry.register_image(prompt_image_content)- image_registry_indices[image_ref.variable_path] = registry_index- else:- msg = f"Image reference '{image_ref.variable_path}' is of type '{type(prompt_image_content).__name__}', expected ImageContent"- raise ImgGenPromptBlueprintValueError(msg)- except ContextProviderError as exc:- msg = f"Could not find image '{image_ref.variable_path}' in context: {exc}"- raise ImgGenPromptBlueprintValueError(msg) from exc-- def _extract_direct_list_images(- self,- image_ref: ImageReference,- *,- context_provider: ContextProviderAbstract,- image_registry: ImageRegistry,- image_registry_indices: dict[str, int],- ) -> None:- """Extract a list of ImageContent from context and register them."""- log.verbose(f"Getting image list '{image_ref.variable_path}' from context")- try:- prompt_image_content = context_provider.get_typed_object_or_attribute(- name=image_ref.variable_path,- wanted_type=ImageContent,- accept_list=True,+ # Enforced here rather than in the kernel: the limit is a property of the model this pipe+ # chose, and its breach raises an interpreter-layer error.+ input_images = img_gen_prompt.input_images+ if max_prompt_images is not None and input_images is not None and len(input_images) > max_prompt_images:+ msg = (+ f"Too many input images: got {len(input_images)}, but model only supports {max_prompt_images}. "+ f"Reduce the number of images in your template.")- if isinstance(prompt_image_content, list):- prompt_image_content = cast("list[Any]", prompt_image_content)- for list_position, image_item in enumerate(prompt_image_content, start=1):- if not isinstance(image_item, ImageContent):- msg = f"Item of '{image_ref.variable_path}' is of type '{type(image_item).__name__}', expected ImageContent"- raise ImgGenPromptBlueprintValueError(msg)- registry_index = image_registry.register_image(image_item)- # Use list position (1-based) for variable name, registry index for image number- image_item_name = f"{image_ref.variable_path}[{list_position}]"- image_registry_indices[image_item_name] = registry_index- elif isinstance(prompt_image_content, tuple):- content_tuple: tuple[Any, ...] = cast("tuple[Any, ...]", prompt_image_content)- for list_position, image_item in enumerate(content_tuple, start=1):- if not isinstance(image_item, ImageContent):- msg = f"Item of '{image_ref.variable_path}' is of type '{type(image_item).__name__}', expected ImageContent"- raise ImgGenPromptBlueprintValueError(msg)- registry_index = image_registry.register_image(image_item)- image_item_name = f"{image_ref.variable_path}[{list_position}]"- image_registry_indices[image_item_name] = registry_index- else:- msg = (- f"Image list reference '{image_ref.variable_path}' is of type '{type(prompt_image_content).__name__}', "- "expected list or tuple of ImageContent"- )- raise ImgGenPromptBlueprintValueError(msg)- except ContextProviderError as exc:- msg = f"Could not find image list '{image_ref.variable_path}' in context: {exc}"- raise ImgGenPromptBlueprintValueError(msg) from exc+ raise PipeImgGenFactoryError(msg)- async def _render_text(- self,- *,- context_provider: ContextProviderAbstract,- template_blueprint: TemplateBlueprint,- extra_params: dict[str, Any] | None = None,- image_registry: ImageRegistry | None = None,- ) -> str:- """Render a template with context and optional image registry."""- context: dict[str, Any] = context_provider.generate_context()- if extra_params:- context = substitute_nested_in_context(context=context, extra_params=extra_params)- if template_blueprint.extra_context:- context.update(**template_blueprint.extra_context)-- # Add image registry to context for | with_images and | format filters- if image_registry is not None:- context[Jinja2ContextKey.IMAGE_REGISTRY] = image_registry-- finalize = image_registry.make_finalize() if image_registry and image_registry.images else None-- return await render_template(- template=template_blueprint.template,- category=template_blueprint.category,- context=context,- templating_style=template_blueprint.templating_style,- finalize=finalize,- )+ return img_gen_promptdiff --git a/pipelex/pipe_operators/img_gen/pipe_img_gen.py b/pipelex/pipe_operators/img_gen/pipe_img_gen.pyindex d1abe8e58..e579692d0 100644--- a/pipelex/pipe_operators/img_gen/pipe_img_gen.py+++ b/pipelex/pipe_operators/img_gen/pipe_img_gen.py@@ -1,16 +1,14 @@-from typing import TYPE_CHECKING, Any, Literal+from typing import Any, Literalfrom pydantic import Fieldfrom typing_extensions import overridefrom pipelex import log-from pipelex.cogt.image.image_size import ImageSize-from pipelex.cogt.img_gen.img_gen_job_components import AspectRatio, Background, ImgGenJobParams, ImgGenSize+from pipelex.cogt.img_gen.img_gen_job_components import AspectRatio, Background, ImgGenSizefrom pipelex.cogt.img_gen.img_gen_param_support import ImgGenParamSupportfrom pipelex.cogt.img_gen.img_gen_setting import ImgGenModelChoice, ImgGenSetting, ImgGenSettingValueErrorfrom pipelex.cogt.model_backends.model_type import ModelTypefrom pipelex.cogt.models.model_deck_check import check_img_gen_choice_with_deck-from pipelex.config import get_configfrom pipelex.core.concepts.native.concept_native import NativeConceptCodefrom pipelex.core.memory.exceptions import WorkingMemoryStuffNotFoundErrorfrom pipelex.core.memory.working_memory import WorkingMemory@@ -20,34 +18,19 @@ from pipelex.core.pipes.pipe_output import PipeOutputfrom pipelex.core.pipes.variable_multiplicity import VariableMultiplicityfrom pipelex.core.stuffs.exceptions import StuffContentTypeErrorfrom pipelex.core.stuffs.image_content import ImageContent-from pipelex.core.stuffs.list_content import ListContent-from pipelex.core.stuffs.stuff_factory import StuffFactoryfrom pipelex.interpreter_hub import get_concept_library, get_native_concept+from pipelex.kernel.exceptions import PromptContentError+from pipelex.kernel.img_gen_ops import build_img_gen_job_params, resolve_img_gen_setting, run_img_genfrom pipelex.pipe_machinery.template_guard_lint import lint_optional_input_guardsfrom pipelex.pipe_operators.img_gen.exceptions import PipeImgGenFactoryError, PipeImgGenRunError-from pipelex.pipe_operators.img_gen.img_gen_prompt_blueprint import ImgGenPromptBlueprint, ImgGenPromptBlueprintValueError+from pipelex.pipe_operators.img_gen.img_gen_prompt_blueprint import ImgGenPromptBlueprintfrom pipelex.pipe_operators.pipe_operator import PipeOperatorfrom pipelex.pipe_run.exceptions import PipeRunParamsErrorfrom pipelex.pipe_run.pipe_run_params import PipeRunParams, output_multiplicity_to_apply-from pipelex.runtime_hub import get_class_registry, get_content_generator, get_model_deck+from pipelex.runtime_hub import get_class_registry, get_model_deckfrom pipelex.system.job_metadata import JobMetadatafrom pipelex.tools.misc.image_utils import ImageFormat-if TYPE_CHECKING:- from pipelex.core.stuffs.stuff_content import StuffContent---def resolve_default_size(*, explicit_aspect_ratio: AspectRatio | None, default_size: ImgGenSize | None) -> ImgGenSize | None:- """Resolve the config-level size default applicable to a pipe, honoring exact-size/aspect-ratio exclusivity.-- An exact size implies its own aspect ratio (the blueprint forbids setting both on a pipe), so an- exact-size deck default does not apply to a pipe that explicitly sets `aspect_ratio` — the more- specific pipe field wins. A tier default composes with any ratio and always applies.- """- if explicit_aspect_ratio is not None and isinstance(default_size, ImageSize):- return None- return default_size-class PipeImgGenOutput(PipeOutput):pass@@ -175,29 +158,18 @@ class PipeImgGen(PipeOperator[PipeImgGenOutput]):pipe_run_params: PipeRunParams,output_name: str | None = None,) -> PipeImgGenOutput:- content_generator = get_content_generator()-multiplicity_resolution = output_multiplicity_to_apply(base_multiplicity=self.output_multiplicity or False,override_multiplicity=pipe_run_params.output_multiplicity,)applied_output_multiplicity = multiplicity_resolution.resolved_multiplicity- img_gen_config = get_config().cogt.img_gen_config- img_gen_param_defaults = img_gen_config.img_gen_param_defaults- model_deck = get_model_deck()-- # Get ImgGenSetting either from img_gen choice or legacy settings- img_gen_setting: ImgGenSetting- if self.img_gen_choice is not None:- # New pattern: use img_gen choice (preset or inline setting)- img_gen_setting = model_deck.get_img_gen_setting(self.img_gen_choice)- else:- # Use default from model deck- img_gen_setting = model_deck.get_img_gen_setting(model_deck.img_gen_choice_default)+ # The deck chain is kernel semantics; the setting is resolved per run into a local and never+ # cached onto `self`, for the reason `pipe_llm.py` states about its own settings.+ img_gen_setting: ImgGenSetting = resolve_img_gen_setting(img_gen_choice=self.img_gen_choice)# Get max_prompt_images from model spec for validation- model_spec = model_deck.get_optional_inference_model(model_handle=img_gen_setting.model, model_type=ModelType.IMG_GEN)+ model_spec = get_model_deck().get_optional_inference_model(model_handle=img_gen_setting.model, model_type=ModelType.IMG_GEN)max_prompt_images = model_spec.max_prompt_images if model_spec else Nonetry:@@ -212,40 +184,25 @@ class PipeImgGen(PipeOperator[PipeImgGenOutput]):except StuffContentTypeError as stuff_content_type_error:msg = f"While runnning the PipeImgGen '{self.code}' some inputs are not of the right type: {stuff_content_type_error}"raise PipeImgGenRunError(message=msg) from stuff_content_type_error- except ImgGenPromptBlueprintValueError as blueprint_error:+ except PromptContentError as blueprint_error:msg = f"While running the PipeImgGen '{self.code}' image extraction failed: {blueprint_error}"raise PipeImgGenRunError(message=msg) from blueprint_errorexcept PipeImgGenFactoryError as factory_error:msg = f"While running the PipeImgGen '{self.code}' prompt construction failed: {factory_error}"raise PipeImgGenRunError(message=msg) from factory_error- # Process one-time settings- seed_setting = self.seed or img_gen_param_defaults.seed- seed: int | None- if isinstance(seed_setting, str) and seed_setting == "auto":- seed = None- else:- seed = seed_setting-- # Build ImgGenJobParams from ImgGenSetting + one-time settings- img_gen_job_params = ImgGenJobParams(- aspect_ratio=self.aspect_ratio or img_gen_param_defaults.aspect_ratio,- size=self.size or resolve_default_size(explicit_aspect_ratio=self.aspect_ratio, default_size=img_gen_param_defaults.size),- background=self.background or img_gen_param_defaults.background,- quality=img_gen_setting.quality,- nb_steps=img_gen_setting.nb_steps,- guidance_scale=img_gen_setting.guidance_scale,- is_moderated=img_gen_setting.is_moderated,- safety_tolerance=img_gen_setting.safety_tolerance,- is_raw=self.is_raw if self.is_raw is not None else img_gen_param_defaults.is_raw,+ # The three-provenance composition (setting / step field / configured default) is kernel semantics.+ img_gen_job_params = build_img_gen_job_params(+ img_gen_setting=img_gen_setting,+ aspect_ratio=self.aspect_ratio,+ size=self.size,+ is_raw=self.is_raw,+ seed=self.seed,+ background=self.background,output_format=self.output_format,- seed=seed,)- # Get the image generation handle- img_gen_handle = img_gen_setting.model- log.verbose(f"Using img_gen handle: {img_gen_handle}")+ log.verbose(f"Using img_gen handle: {img_gen_setting.model}")- the_content: StuffContentnb_images: intif isinstance(applied_output_multiplicity, bool):if self.output_multiplicity is True:@@ -260,52 +217,26 @@ class PipeImgGen(PipeOperator[PipeImgGenOutput]):else:nb_images = 1- # Get the structure class from the registry (must be a subclass of ImageContent)+ # Resolved here rather than inside the kernel: turning a concept into a class is a library's+ # business, and handing the class over is what spares the kernel a registry read.image_content_subclass = get_class_registry().get_required_subclass(name=self.output.concept.structure_class_name,base_class=ImageContent,)- if nb_images > 1:- image_content_list = await content_generator.make_image_list(- job_metadata=job_metadata,- cogt_run_params=pipe_run_params.cogt_run_params,- img_gen_handle=img_gen_handle,- img_gen_prompt=img_gen_prompt,- nb_images=nb_images,- img_gen_job_params=img_gen_job_params,- img_gen_job_config=img_gen_config.img_gen_job_config,- )- subclass_content_items: list[ImageContent] = []- for image_content in image_content_list:- subclass_content = image_content_subclass.model_validate(image_content.smart_dump())- subclass_content_items.append(subclass_content)- the_content = ListContent(- items=subclass_content_items,- )- log.verbose(the_content, title="List of image contents")- else:- image_content = await content_generator.make_single_image(- job_metadata=job_metadata,- cogt_run_params=pipe_run_params.cogt_run_params,- img_gen_handle=img_gen_handle,- img_gen_prompt=img_gen_prompt,- img_gen_job_params=img_gen_job_params,- img_gen_job_config=img_gen_config.img_gen_job_config,- )-- the_content = image_content_subclass.model_validate(image_content.smart_dump())- log.verbose(the_content, title=f"output stuff content of PipeImg {self.code}")-- output_stuff = StuffFactory.make_stuff(- name=output_name,+ img_gen_result = await run_img_gen(+ memory=working_memory,+ img_gen_prompt=img_gen_prompt,+ img_gen_setting=img_gen_setting,+ img_gen_job_params=img_gen_job_params,concept=self.output.concept,- content=the_content,- )-- working_memory.set_new_main_stuff(- stuff=output_stuff,- name=output_name,+ output_class=image_content_subclass,+ job_metadata=job_metadata,+ cogt_run_params=pipe_run_params.cogt_run_params,+ nb_images=nb_images,+ result_name=output_name,)+ working_memory = img_gen_result.memory+ log.verbose(img_gen_result.content, title=f"output stuff content of PipeImg {self.code}")# Capture execution data for the graph tracerexecution_data_dict: dict[str, Any] = {diff --git a/pipelex/pipe_operators/search/pipe_search.py b/pipelex/pipe_operators/search/pipe_search.pyindex 43a75cb2c..f1882d76d 100644--- a/pipelex/pipe_operators/search/pipe_search.py+++ b/pipelex/pipe_operators/search/pipe_search.py@@ -3,22 +3,18 @@ from typing import TYPE_CHECKING, Any, Literalfrom typing_extensions import overridefrom pipelex import log-from pipelex.cogt.content_generation.assignment_models import SearchAssignmentfrom pipelex.cogt.exceptions import ModelChoiceNotFoundError-from pipelex.cogt.model_backends.model_type import ModelTypefrom pipelex.cogt.models.model_deck_check import check_search_choice_with_deck-from pipelex.cogt.search.search_setting import SearchModelChoice, SearchSetting+from pipelex.cogt.search.search_setting import SearchModelChoicefrom pipelex.cogt.templating.template_blueprint import TemplateBlueprint-from pipelex.cogt.templating.template_rendering import render_templatefrom pipelex.core.memory.working_memory import WorkingMemoryfrom pipelex.core.pipes.inputs.input_stuff_specs import InputStuffSpecsfrom pipelex.core.pipes.pipe_output import PipeOutput-from pipelex.core.stuffs.stuff_factory import StuffFactoryfrom pipelex.interpreter_hub import get_concept_library+from pipelex.kernel.search_ops import resolve_search_setting, run_searchfrom pipelex.pipe_machinery.template_guard_lint import lint_optional_input_guardsfrom pipelex.pipe_operators.pipe_operator import PipeOperatorfrom pipelex.pipe_run.pipe_run_params import PipeRunParams-from pipelex.runtime_hub import get_content_generator, get_model_deckfrom pipelex.system.job_metadata import JobMetadatafrom pipelex.tools.misc.string_utils import get_root_from_dotted_path@@ -91,76 +87,46 @@ class PipeSearch(PipeOperator[PipeSearchOutput]):pipe_run_params: PipeRunParams,output_name: str | None = None,) -> PipeSearchOutput:- content_generator = get_content_generator()-- # 0. Log the search runsearch_choice_desc = self.search_choice or "default"log.dev(f"✨ PipeSearch '{self.code}' running with search choice '{search_choice_desc}' ✨")- # 1. Render the prompt template against working memory context- query_text = await render_template(- template=self.prompt_blueprint.template,- category=self.prompt_blueprint.category,- context=working_memory.generate_context(),+ # The deck chain, the handle resolution and the override application are kernel semantics; the+ # setting is resolved per run into a local and never cached onto `self`, for the reason+ # `pipe_llm.py` states about its own settings.+ search_setting = resolve_search_setting(+ search_choice=self.search_choice,+ include_images_override=self.include_images_override,+ max_results_override=self.max_results_override,)- # 2. Resolve SearchSetting from deck- model_deck = get_model_deck()- search_choice: SearchModelChoice = self.search_choice or model_deck.search_choice_default- search_setting: SearchSetting = model_deck.get_search_setting(search_choice=search_choice)-- # 3. Resolve the model handle (waterfalls/aliases → actual provider/variant handle). Pin- # search_setting.model to the resolved handle so it doubles as the Temporal routing key.- inference_model = model_deck.get_required_inference_model(model_handle=search_setting.model, model_type=ModelType.SEARCH)- resolved_model_handle = inference_model.name- if resolved_model_handle != search_setting.model:- search_setting = search_setting.model_copy(update={"model": resolved_model_handle})-- # 4. Apply pipe-level overrides- if self.include_images_override is not None:- search_setting = search_setting.model_copy(update={"include_images": self.include_images_override})- if self.max_results_override is not None:- search_setting = search_setting.model_copy(update={"max_results": self.max_results_override})-- # 5. Build the serializable assignment and run the search behind the content-generation seam.- # The leaf goes through the same swappable generator as LLM/img-gen/extract: direct inline, a- # Temporal activity when in-workflow, or a dry mock. That makes search-on-Temporal replay-safe- # and lets its failures cross the workflow boundary as classified errors instead of hanging.- search_assignment = SearchAssignment(+ # The sourced-answer / structured fork stays here: it is declared on the pipe, and resolving the+ # output concept to a structure class is what a loaded library is for. Handing the class over+ # is what spares the kernel a library read.+ output_structure_class: type[StuffContent] | None = None+ if self.is_structured_output:+ output_structure_class = get_concept_library().get_structure_class(concept=self.output.concept)++ search_result = await run_search(+ memory=working_memory,+ template=self.prompt_blueprint.template,+ category=self.prompt_blueprint.category,+ search_setting=search_setting,+ concept=self.output.concept,job_metadata=job_metadata,cogt_run_params=pipe_run_params.cogt_run_params,- query=query_text,- search_setting=search_setting,+ output_structure_class=output_structure_class,include_domains=self.include_domains,exclude_domains=self.exclude_domains,from_date=self.from_date,to_date=self.to_date,+ result_name=output_name,)+ working_memory = search_result.memory- content: StuffContent- if not self.is_structured_output:- content = await content_generator.make_search_sourced_answer(search_assignment=search_assignment)- else:- output_structure_class = get_concept_library().get_structure_class(concept=self.output.concept)- content = await content_generator.make_search_structured(- output_structure_class=output_structure_class,- search_assignment=search_assignment,- )-- # 6. Create Stuff, set in working memory, and return output- output_stuff = StuffFactory.make_stuff(- name=output_name,- concept=self.output.concept,- content=content,- )- working_memory.set_new_main_stuff(- stuff=output_stuff,- name=output_name,- )# Capture execution data for the graph tracerexecution_data_dict: dict[str, Any] = {- "rendered_query": query_text,- "resolved_model": search_setting.model,+ "rendered_query": search_result.rendered_query,+ "resolved_model": search_result.search_setting.model,"is_structured_output": self.is_structured_output,}diff --git a/pipelex/pipe_operators/structure/pipe_structure.py b/pipelex/pipe_operators/structure/pipe_structure.pyindex e1157e587..25efd740e 100644--- a/pipelex/pipe_operators/structure/pipe_structure.py+++ b/pipelex/pipe_operators/structure/pipe_structure.py@@ -18,7 +18,8 @@ from pipelex.core.pipes.pipe_output import PipeOutputfrom pipelex.core.pipes.variable_multiplicity import VariableMultiplicityfrom pipelex.core.stuffs.stuff_content import StuffContentfrom pipelex.interpreter_hub import get_concept_library, get_native_concept-from pipelex.kernel.llm_ops import derive_structure_prompt, generate_object_content, resolve_llm_setting_for_object, store_result+from pipelex.kernel.llm_ops import derive_structure_prompt, generate_object_content, resolve_llm_setting_for_object+from pipelex.kernel.memory_ops import store_resultfrom pipelex.pipe_operators.pipe_operator import PipeOperatorfrom pipelex.pipe_run.pipe_run_params import PipeRunParams, output_multiplicity_to_applyfrom pipelex.runtime_hub import get_class_registrydiff --git a/subject_grants.toml b/subject_grants.tomlindex 16f65d1a5..efe0dd944 100644--- a/subject_grants.toml+++ b/subject_grants.toml@@ -4200,14 +4200,6 @@ rationale = "Verb-object: validates the item; expectations stay keyword."param = "response"rationale = "Converter: rebinds a transported response into a result; single self-labelling operand."-["pipelex/pipe_operators/img_gen/img_gen_prompt_blueprint.py::ImgGenPromptBlueprint._extract_direct_image"]-param = "image_ref"-rationale = "Verb-object: extracts from the image ref; context stays keyword."--["pipelex/pipe_operators/img_gen/img_gen_prompt_blueprint.py::ImgGenPromptBlueprint._extract_direct_list_images"]-param = "image_ref"-rationale = "Verb-object: extracts from the image ref; context stays keyword."-["pipelex/pipe_operators/llm/pipe_llm.py::PipeLLM._format_error_location"]param = "pipe_run_params"rationale = "Derivation from the run params; single operand."diff --git a/tests/integration/pipelex/pipes/operator/pipe_extract/test_pipe_extract_model_not_found_reroute.py b/tests/integration/pipelex/pipes/operator/pipe_extract/test_pipe_extract_model_not_found_reroute.pyindex 34815cc24..b54725cc7 100644--- a/tests/integration/pipelex/pipes/operator/pipe_extract/test_pipe_extract_model_not_found_reroute.py+++ b/tests/integration/pipelex/pipes/operator/pipe_extract/test_pipe_extract_model_not_found_reroute.py@@ -43,7 +43,7 @@ class TestPipeExtractModelNotFoundReroute:)fake_content_generator = mocker.MagicMock()fake_content_generator.make_extract_pages = mocker.AsyncMock(side_effect=not_found_error)- mocker.patch("pipelex.pipe_operators.extract.pipe_extract.get_content_generator", return_value=fake_content_generator)+ mocker.patch("pipelex.kernel.extract_ops.get_content_generator", return_value=fake_content_generator)pipe = PipeFactory[PipeExtract].make_from_blueprint(domain_code="generic",diff --git a/tests/integration/pipelex/pipes/operator/pipe_img_gen/test_pipe_img_gen_model_not_found_reroute.py b/tests/integration/pipelex/pipes/operator/pipe_img_gen/test_pipe_img_gen_model_not_found_reroute.pyindex 99dd431ed..5bd731431 100644--- a/tests/integration/pipelex/pipes/operator/pipe_img_gen/test_pipe_img_gen_model_not_found_reroute.py+++ b/tests/integration/pipelex/pipes/operator/pipe_img_gen/test_pipe_img_gen_model_not_found_reroute.py@@ -39,7 +39,7 @@ class TestPipeImgGenModelNotFoundReroute:)fake_content_generator = mocker.MagicMock()fake_content_generator.make_single_image = mocker.AsyncMock(side_effect=not_found_error)- mocker.patch("pipelex.pipe_operators.img_gen.pipe_img_gen.get_content_generator", return_value=fake_content_generator)+ mocker.patch("pipelex.kernel.img_gen_ops.get_content_generator", return_value=fake_content_generator)pipe = PipeFactory[PipeImgGen].make_from_blueprint(domain_code="generic",diff --git a/tests/unit/pipelex/kernel/test_func_ops.py b/tests/unit/pipelex/kernel/test_func_ops.pynew file mode 100644index 000000000..4ae173141--- /dev/null+++ b/tests/unit/pipelex/kernel/test_func_ops.py@@ -0,0 +1,96 @@+"""The func path's kernel-only arms.++`run_func` is the whole reason these tests exist: no interpreter test reaches it. `PipeFunc` rides+the *pluggable* executor — which may run the function in a sandbox or a Temporal activity — and then+stores the result, so it exercises `call_registered_function` (through the direct executor) and+`store_result` separately, never their composition. A programmatic caller on a `RuntimeBoot`-only+process rides exactly that composition, so it is tested here or nowhere.++The sync arm is the second one worth pinning: a plain `def` function must not be awaited directly,+it has to reach a worker thread, and a rewrite that dropped the `iscoroutinefunction` branch would+still pass every async-function test.+"""++import pytest++from pipelex.core.concepts.concept_factory import ConceptFactory+from pipelex.core.concepts.native.concept_native import NativeConceptCode+from pipelex.core.memory.working_memory import WorkingMemory+from pipelex.core.memory.working_memory_factory import WorkingMemoryFactory+from pipelex.core.stuffs.text_content import TextContent+from pipelex.kernel.func_ops import call_registered_function, run_func+from pipelex.system.registries.func_registry import func_registry++ASYNC_FUNCTION_NAME = "kernel_unit_test_async_func"++SYNC_FUNCTION_NAME = "kernel_unit_test_sync_func"++ASYNC_TEXT = "produced by the async function"++SYNC_TEXT = "produced by the sync function"++RESULT_NAME = "func_output"+++# Being a coroutine function with nothing to await is the point: it is what selects the awaited arm.+async def _async_producer(working_memory: WorkingMemory) -> TextContent: # noqa: ARG001, RUF029+ return TextContent(text=ASYNC_TEXT)+++def _sync_producer(working_memory: WorkingMemory) -> TextContent: # noqa: ARG001+ return TextContent(text=SYNC_TEXT)+++def registered_producers():+ func_registry.register_function(_async_producer, name=ASYNC_FUNCTION_NAME)+ func_registry.register_function(_sync_producer, name=SYNC_FUNCTION_NAME)+ try:+ yield+ finally:+ func_registry.unregister_function_by_name(ASYNC_FUNCTION_NAME)+ func_registry.unregister_function_by_name(SYNC_FUNCTION_NAME)++[email protected]("registered_producers")[email protected](loop_scope="class")+class TestFuncOps:+ @pytest.mark.parametrize(+ ("function_name", "expected_text", "expected_qualname"),+ [+ pytest.param(ASYNC_FUNCTION_NAME, ASYNC_TEXT, _async_producer.__qualname__, id="async-function"),+ pytest.param(SYNC_FUNCTION_NAME, SYNC_TEXT, _sync_producer.__qualname__, id="sync-function-on-a-worker-thread"),+ ],+ )+ async def test_call_registered_function_runs_both_dispatch_arms(+ self,+ function_name: str,+ expected_text: str,+ expected_qualname: str,+ ) -> None:+ call_result = await call_registered_function(function_name=function_name, memory=WorkingMemoryFactory.make_empty())++ assert isinstance(call_result.content, TextContent)+ assert call_result.content.text == expected_text+ assert call_result.function_module == __name__+ assert call_result.function_qualname == expected_qualname, (+ "the module and qualname ride back on the result because a caller that dispatched the call "+ "out-of-process no longer holds the function object to read them off"+ )++ async def test_run_func_returns_the_memory_holding_the_result(self) -> None:+ memory = WorkingMemoryFactory.make_empty()++ result = await run_func(+ memory=memory,+ function_name=ASYNC_FUNCTION_NAME,+ concept=ConceptFactory.make_native_concept(native_concept_code=NativeConceptCode.TEXT),+ result_name=RESULT_NAME,+ )++ main_stuff = result.memory.get_main_stuff()+ assert main_stuff.stuff_name == RESULT_NAME+ assert isinstance(main_stuff.content, TextContent)+ assert main_stuff.content.text == ASYNC_TEXT+ assert result.content is main_stuff.content+ assert result.memory.get_stuff(name=RESULT_NAME).content is main_stuff.contentdiff --git a/tests/unit/pipelex/kernel/test_img_gen_prompt_kernel.py b/tests/unit/pipelex/kernel/test_img_gen_prompt_kernel.pynew file mode 100644index 000000000..d4d83f927--- /dev/null+++ b/tests/unit/pipelex/kernel/test_img_gen_prompt_kernel.py@@ -0,0 +1,101 @@+"""The kernel can build an `ImgGenPrompt`, and builds the same one the blueprint does.++`run_img_gen` takes a ready `ImgGenPrompt`, so before this constructor existed a `RuntimeBoot`-only+process could call every other operator op and not the image one — the sole builder was an+interpreter-layer blueprint. That made the package's "callable without the interpreter" guarantee+false for exactly one operator, and a guarantee at less than 100% is worth nothing.++The gate is equality, not merely "it runs": the blueprint now delegates here, so the two must agree+by construction — and the assertion is what will catch it if someone later reintroduces assembly on+the interpreter side. What is actually easy to get wrong, and what these cases pin, is the+correspondence between the `[Image N]` tokens in the text and the order of `input_images`: the+registry is the single source of truth for both, and a mismatch mislabels which image the prompt is+describing with nothing downstream able to notice.+"""++import pytest++from pipelex.cogt.image.prompt_image import PromptImageUri+from pipelex.cogt.templating.template_blueprint import TemplateBlueprint+from pipelex.core.concepts.concept_factory import ConceptFactory+from pipelex.core.concepts.native.concept_native import NativeConceptCode+from pipelex.core.memory.working_memory import WorkingMemory+from pipelex.core.memory.working_memory_factory import WorkingMemoryFactory+from pipelex.core.stuffs.image_content import ImageContent+from pipelex.core.stuffs.stuff_factory import StuffFactory+from pipelex.kernel.exceptions import PromptContentError+from pipelex.kernel.img_gen_prompt import assemble_img_gen_prompt+from pipelex.kernel.prompt_references import ImageReference, ImageReferenceKind+from pipelex.pipe_operators.img_gen.img_gen_prompt_blueprint import ImgGenPromptBlueprint+from pipelex.tools.jinja2.template_category import TemplateCategory++IMAGE_URL_1 = "https://example.com/first.png"+IMAGE_URL_2 = "https://example.com/second.png"+++def _memory_with_images() -> WorkingMemory:+ image_concept = ConceptFactory.make_native_concept(native_concept_code=NativeConceptCode.IMAGE)+ return WorkingMemoryFactory.make_from_multiple_stuffs(+ [+ StuffFactory.make_stuff(concept=image_concept, content=ImageContent(url=IMAGE_URL_1), name="hero"),+ StuffFactory.make_stuff(concept=image_concept, content=ImageContent(url=IMAGE_URL_2), name="sidekick"),+ ]+ )+++def _prompt_template() -> TemplateBlueprint:+ return TemplateBlueprint(template="Draw {{ hero }} next to {{ sidekick }}.", category=TemplateCategory.IMG_GEN_PROMPT)+++def _image_references() -> list[ImageReference]:+ return [+ ImageReference(variable_path="hero", kind=ImageReferenceKind.DIRECT),+ ImageReference(variable_path="sidekick", kind=ImageReferenceKind.DIRECT),+ ]+++class TestImgGenPromptKernel:+ @pytest.mark.asyncio(loop_scope="class")+ async def test_tokens_are_numbered_from_the_registry_that_orders_input_images(self) -> None:+ prompt = await assemble_img_gen_prompt(+ context_provider=_memory_with_images(),+ prompt_blueprint=_prompt_template(),+ image_references=_image_references(),+ )++ assert prompt.input_images is not None+ assert all(isinstance(image, PromptImageUri) for image in prompt.input_images)+ uris = [image.uri for image in prompt.input_images if isinstance(image, PromptImageUri)]+ assert uris == [IMAGE_URL_1, IMAGE_URL_2]+ assert "[Image 1]" in prompt.positive_text+ assert "[Image 2]" in prompt.positive_text+ assert prompt.positive_text.index("[Image 1]") < prompt.positive_text.index("[Image 2]"), (+ "token order must follow registry order, which is the order of `input_images` — if these "+ "diverge the prompt names the wrong image and nothing downstream can tell."+ )+ assert IMAGE_URL_1 not in prompt.positive_text, "a raw URL in the text means the placeholder substitution was skipped"++ @pytest.mark.asyncio(loop_scope="class")+ async def test_the_kernel_and_the_blueprint_build_the_same_prompt(self) -> None:+ """Both readers, same answer — the property the blueprint's delegation has to preserve."""+ from_kernel = await assemble_img_gen_prompt(+ context_provider=_memory_with_images(),+ prompt_blueprint=_prompt_template(),+ image_references=_image_references(),+ )+ from_blueprint = await ImgGenPromptBlueprint(+ prompt_blueprint=_prompt_template(),+ image_references=_image_references(),+ ).make_img_gen_prompt(context_provider=_memory_with_images())++ assert from_kernel == from_blueprint++ @pytest.mark.asyncio(loop_scope="class")+ async def test_an_unresolvable_reference_raises_the_kernel_error(self) -> None:+ """The error moved into the kernel with the code that raises it, so it is the kernel's."""+ with pytest.raises(PromptContentError):+ await assemble_img_gen_prompt(+ context_provider=_memory_with_images(),+ prompt_blueprint=TemplateBlueprint(template="Draw {{ nobody }}.", category=TemplateCategory.IMG_GEN_PROMPT),+ image_references=[ImageReference(variable_path="nobody", kind=ImageReferenceKind.DIRECT)],+ )diff --git a/tests/unit/pipelex/pipe_operators/pipe_img_gen/test_pipe_img_gen_size_default.py b/tests/unit/pipelex/kernel/test_img_gen_size_default.pysimilarity index 92%rename from tests/unit/pipelex/pipe_operators/pipe_img_gen/test_pipe_img_gen_size_default.pyrename to tests/unit/pipelex/kernel/test_img_gen_size_default.pyindex ab602c3f2..9f2ec867c 100644--- a/tests/unit/pipelex/pipe_operators/pipe_img_gen/test_pipe_img_gen_size_default.py+++ b/tests/unit/pipelex/kernel/test_img_gen_size_default.py@@ -1,10 +1,10 @@-"""Tests for the config-level size default resolution against a pipe's explicit aspect_ratio."""+"""Tests for the config-level size default resolution against a step's explicit aspect_ratio."""import pytestfrom pipelex.cogt.image.image_size import ImageSizefrom pipelex.cogt.img_gen.img_gen_job_components import AspectRatio, SizeTier-from pipelex.pipe_operators.img_gen.pipe_img_gen import resolve_default_size+from pipelex.kernel.img_gen_ops import resolve_default_sizeclass TestResolveDefaultSize:diff --git a/tests/unit/pipelex/kernel/test_kernel_boot_contract.py b/tests/unit/pipelex/kernel/test_kernel_boot_contract.pyindex 0a32fa52c..b98bd7d49 100644--- a/tests/unit/pipelex/kernel/test_kernel_boot_contract.py+++ b/tests/unit/pipelex/kernel/test_kernel_boot_contract.py@@ -15,13 +15,22 @@ interpreter through a function-local import, which is invisible to the static grimport-closure test *at once*. The sweep below therefore runs **after** the kernel call, which iswhat makes this test strictly stronger than the import-time one it complements.+**Every operator's entry point is called, not a representative one.** The blind spot this test exists+for is per-function, so covering one op says nothing about the next: a function-local import inside+`run_search` is caught only by calling `run_search`. That also makes this the only gate the ops+modules other than `llm_ops` have — `PipelexKernel` is an LLM-era façade that does not import them, so+`test_runtime_layer_import_closure.py`'s `pipelex.kernel.pipelex_kernel` entry point does not reach+them even at import time.+Run in a **subprocess** for the reason its two siblings state: a suite-level boot already owns theprocess singletons, and both hubs are sticky class attributes that `teardown` deliberately does notclear — so an in-process check would answer from a stale `Pipelex` and pass vacuously.`needs_inference=False` keeps the boot offline (no gateway terms gate, no model-deck validation), and-`PipeRunMode.DRY` keeps the call offline: the cogt leaf mocks before any worker is looked up, so the-`LLMSetting` names a model the deck never has to resolve.+`PipeRunMode.DRY` keeps the call offline: every cogt leaf mocks before a worker is looked up, so each+setting below names a model the deck never has to resolve. The settings are built literally rather+than through the `resolve_*_setting` helpers for that same reason — those read the deck, which is a+separate question from the one this test asks.**On the deferred orchestrator question.** `runtime_boot.py`'s orchestrator-rejection comment defersthe external-interpreter-orchestrator half-application hole to "the first caller of a runtime-only@@ -43,28 +52,49 @@ from tests.unit.pipelex.test_runtime_layer_import_closure import INTERPRETER_PAC#: (`docs/agents/debugging-hanging-pytest-runs.md`).SUBPROCESS_TIMEOUT_SECONDS = 300-#: Boot the runtime layer in a fresh interpreter, run **both** façade calls through the kernel, assert+#: Boot the runtime layer in a fresh interpreter, run **every** operator's kernel entry point, assert#: the typed results, and only then sweep `sys.modules`. The order is the point: a function-local#: interpreter import inside a kernel op would land in `sys.modules` during the call and nowhere else.-#: Both arms run because they share almost nothing below the façade — `resolve_llm_setting_for_text`+#: Both LLM arms run because they share almost nothing below the façade — `resolve_llm_setting_for_text`#: vs `_for_object`, and `dry_llm_gen_text` vs `dry_llm_gen_object` — so covering one leaves the#: other's closure unproven, and the contract is stated over *a kernel call*, not over one of them.+#: The five operator ops below are called as module-level functions rather than through `PipelexKernel`+#: for the plain reason that the façade does not expose them._KERNEL_CALL_SCRIPT = textwrap.dedent("""import asyncioimport sys+ from pipelex.cogt.extract.extract_input import ExtractInput+ from pipelex.cogt.extract.extract_setting import ExtractSetting+ from pipelex.cogt.img_gen.img_gen_setting import ImgGenSettingfrom pipelex.cogt.llm.llm_setting import LLMSetting+ from pipelex.cogt.search.search_setting import SearchSetting+ from pipelex.cogt.templating.template_blueprint import TemplateBlueprintfrom pipelex.core.concepts.concept_factory import ConceptFactoryfrom pipelex.core.concepts.native.concept_native import NativeConceptCode+ from pipelex.core.memory.working_memory import WorkingMemoryfrom pipelex.core.memory.working_memory_factory import WorkingMemoryFactory+ from pipelex.core.stuffs.image_content import ImageContent+ from pipelex.core.stuffs.list_content import ListContentfrom pipelex.core.stuffs.number_content import NumberContent+ from pipelex.core.stuffs.search_result_content import SearchResultContent+ from pipelex.core.stuffs.stuff_factory import StuffFactoryfrom pipelex.core.stuffs.text_content import TextContent+ from pipelex.kernel.compose_ops import run_compose_template+ from pipelex.kernel.extract_ops import build_extract_job_params, run_extract+ from pipelex.kernel.func_ops import run_func+ from pipelex.kernel.img_gen_ops import build_img_gen_job_params, run_img_gen+ from pipelex.kernel.img_gen_prompt import assemble_img_gen_promptfrom pipelex.kernel.llm_results import LlmObjectResult, LlmTextResultfrom pipelex.kernel.pipelex_kernel import PipelexKernel+ from pipelex.kernel.prompt_references import ImageReference, ImageReferenceKind+ from pipelex.kernel.search_ops import run_searchfrom pipelex.runtime_boot import RuntimeBootfrom pipelex.system.pipe_run_mode import PipeRunMode+ from pipelex.system.registries.func_registry import func_registryfrom pipelex.system.runtime import IntegrationMode+ from pipelex.tools.jinja2.template_category import TemplateCategoryinterpreter_packages = frozenset(sys.argv[1:])# An empty set would make the sweep flag nothing and the test pass vacuously — the same guard both@@ -73,10 +103,25 @@ _KERNEL_CALL_SCRIPT = textwrap.dedent(print("no interpreter packages passed — the sweep would match nothing")raise SystemExit(2)+ def fail(message):+ print(message)+ raise SystemExit(1)++ def check_stored(result, label, expected_type):+ # Every op ends by writing what it produced into memory as the new main stuff, and that+ # write-back is the half a runtime-only caller most depends on — so each arm checks it landed.+ # Checked by type rather than by identity because the two are not universally the same object:+ # `ExtractResult` narrows its `content` field below `StuffContent`, so pydantic revalidates it+ # into a fresh container (see `pipelex/kernel/extract_results.py`).+ stored = result.memory.get_main_stuff().content+ if not isinstance(stored, expected_type):+ fail(f"the memory returned by {label} carries {type(stored).__name__}, not the {expected_type.__name__} it produced")+RuntimeBoot.make(integration_mode=IntegrationMode.PYTEST, needs_inference=False)kernel = PipelexKernel.make(run_mode=PipeRunMode.DRY, user_id="kernel-boot-contract")model = LLMSetting(model="kernel-boot-contract-model", temperature=0.5)+ text_concept = ConceptFactory.make_native_concept(native_concept_code=NativeConceptCode.TEXT)object_result = asyncio.run(kernel.llm_object(@@ -90,14 +135,11 @@ _KERNEL_CALL_SCRIPT = textwrap.dedent()if not isinstance(object_result, LlmObjectResult):- print(f"llm_object returned {type(object_result).__name__}, not an LlmObjectResult")- raise SystemExit(1)+ fail(f"llm_object returned {type(object_result).__name__}, not an LlmObjectResult")if not isinstance(object_result.content, NumberContent):- print(f"llm_object produced {type(object_result.content).__name__}, not the NumberContent it was given")- raise SystemExit(1)+ fail(f"llm_object produced {type(object_result.content).__name__}, not the NumberContent it was given")if object_result.memory.get_main_stuff().content is not object_result.content:- print("the memory returned by llm_object does not carry the produced content as its main stuff")- raise SystemExit(1)+ fail("the memory returned by llm_object does not carry the produced content as its main stuff")text_result = asyncio.run(kernel.llm_text(@@ -109,17 +151,143 @@ _KERNEL_CALL_SCRIPT = textwrap.dedent()if not isinstance(text_result, LlmTextResult):- print(f"llm_text returned {type(text_result).__name__}, not an LlmTextResult")- raise SystemExit(1)+ fail(f"llm_text returned {type(text_result).__name__}, not an LlmTextResult")main_text_content = text_result.memory.get_main_stuff().contentif not isinstance(main_text_content, TextContent):- print(f"llm_text landed {type(main_text_content).__name__} in memory, not a TextContent")- raise SystemExit(1)+ fail(f"llm_text landed {type(main_text_content).__name__} in memory, not a TextContent")if main_text_content.text != text_result.text:- print("the memory returned by llm_text does not carry the text the result reports")- raise SystemExit(1)+ fail("the memory returned by llm_text does not carry the text the result reports")++ extract_setting = ExtractSetting(model="kernel-boot-contract-extract-model")+ extract_result = asyncio.run(+ run_extract(+ memory=WorkingMemoryFactory.make_empty(),+ extract_input=ExtractInput(document_uri="https://example.com/kernel-boot-contract.pdf"),+ extract_setting=extract_setting,+ extract_job_params=build_extract_job_params(extract_setting=extract_setting),+ concept=ConceptFactory.make_native_concept(native_concept_code=NativeConceptCode.PAGE),+ job_metadata=kernel.make_step_metadata(),+ cogt_run_params=kernel.cogt_run_params,+ result_name="pages",+ )+ )++ # An extraction always lands a page sequence, even from a single image — there is no multiplicity+ # fork on this path, so the ListContent is the shape and not one of two.+ if not isinstance(extract_result.content, ListContent):+ fail(f"run_extract produced {type(extract_result.content).__name__}, not the ListContent of pages it owes")+ check_stored(extract_result, "run_extract", ListContent)++ # The image arm runs the pair, assemble then generate, rather than handing `run_img_gen` a prompt+ # built in the test. It could not do otherwise before `assemble_img_gen_prompt` existed, because+ # the only builder was an interpreter-layer blueprint — so this arm's coverage was incomplete in+ # exactly the way the kernel surface was. Assembly is also the half that reaches into memory to+ # resolve references, which is where a function-local interpreter import would hide, and this+ # subprocess is the only gate that can see one.+ img_gen_memory = WorkingMemoryFactory.make_from_single_stuff(+ StuffFactory.make_stuff(+ concept=ConceptFactory.make_native_concept(native_concept_code=NativeConceptCode.IMAGE),+ content=ImageContent(url="https://example.com/kernel-boot-contract-reference.png"),+ name="reference",+ )+ )+ img_gen_prompt = asyncio.run(+ assemble_img_gen_prompt(+ context_provider=img_gen_memory,+ prompt_blueprint=TemplateBlueprint(+ template="Draw something like {{ reference }}.",+ category=TemplateCategory.IMG_GEN_PROMPT,+ ),+ image_references=[ImageReference(variable_path="reference", kind=ImageReferenceKind.DIRECT)],+ )+ )+ # The token and the input image are two readings of one registry; asserting only that the call+ # returned would leave the half that actually reaches into memory — reference resolution —+ # unproven on a boot that has no library.+ if "[Image 1]" not in img_gen_prompt.positive_text:+ fail(f"assemble_img_gen_prompt left the referenced image unsubstituted: {img_gen_prompt.positive_text!r}")+ if not img_gen_prompt.input_images or len(img_gen_prompt.input_images) != 1:+ fail("assemble_img_gen_prompt did not carry the referenced image through to input_images")++ img_gen_setting = ImgGenSetting(model="kernel-boot-contract-img-gen-model")+ img_gen_result = asyncio.run(+ run_img_gen(+ memory=img_gen_memory,+ img_gen_prompt=img_gen_prompt,+ img_gen_setting=img_gen_setting,+ img_gen_job_params=build_img_gen_job_params(img_gen_setting=img_gen_setting),+ concept=ConceptFactory.make_native_concept(native_concept_code=NativeConceptCode.IMAGE),+ output_class=ImageContent,+ job_metadata=kernel.make_step_metadata(),+ cogt_run_params=kernel.cogt_run_params,+ result_name="picture",+ )+ )++ if not isinstance(img_gen_result.content, ImageContent):+ fail(f"run_img_gen produced {type(img_gen_result.content).__name__}, not the ImageContent class it was given")+ check_stored(img_gen_result, "run_img_gen", ImageContent)++ search_query = "What runs on a runtime-only boot?"+ search_result = asyncio.run(+ run_search(+ memory=WorkingMemoryFactory.make_empty(),+ template=search_query,+ category=TemplateCategory.LLM_PROMPT,+ search_setting=SearchSetting(model="kernel-boot-contract-search-model"),+ concept=text_concept,+ job_metadata=kernel.make_step_metadata(),+ cogt_run_params=kernel.cogt_run_params,+ result_name="findings",+ )+ )++ # The rendered query is what proves the templating ran here rather than being skipped: it is the+ # one part of a search step that is not the provider's.+ if search_result.rendered_query != search_query:+ fail(f"run_search rendered {search_result.rendered_query!r}, not the query it was given")+ check_stored(search_result, "run_search", SearchResultContent)++ composed_text = "Composed on a runtime-only boot."+ compose_result = asyncio.run(+ run_compose_template(+ memory=WorkingMemoryFactory.make_empty(),+ template=composed_text,+ category=TemplateCategory.LLM_PROMPT,+ concept=text_concept,+ output_class=TextContent,+ result_name="composed",+ )+ )++ if compose_result.rendered_text != composed_text:+ fail(f"run_compose_template rendered {compose_result.rendered_text!r}, not the template it was given")+ check_stored(compose_result, "run_compose_template", TextContent)++ # Registered here rather than imported, because the func path's whole point is that the function is+ # the caller's: a runtime-only process has no library to have registered one for it.+ FUNCTION_NAME = "kernel_boot_contract_function"+ FUNCTION_TEXT = "produced by a registered function"++ def produce_text(working_memory: WorkingMemory) -> TextContent:+ return TextContent(text=FUNCTION_TEXT)++ func_registry.register_function(produce_text, name=FUNCTION_NAME)++ func_result = asyncio.run(+ run_func(+ memory=WorkingMemoryFactory.make_empty(),+ function_name=FUNCTION_NAME,+ concept=text_concept,+ result_name="called",+ )+ )++ if not isinstance(func_result.content, TextContent) or func_result.content.text != FUNCTION_TEXT:+ fail("run_func did not return what the registered function produced")+ check_stored(func_result, "run_func", TextContent)- # Swept last, so a function-local interpreter import taken during either call is caught too.+ # Swept last, so a function-local interpreter import taken during any of the calls is caught too.offenders = sorted(namefor name in sys.modulesdiff --git a/tests/unit/pipelex/test_runtime_layer_import_closure.py b/tests/unit/pipelex/test_runtime_layer_import_closure.pyindex f8ac5daa6..fdf3eb6ab 100644--- a/tests/unit/pipelex/test_runtime_layer_import_closure.py+++ b/tests/unit/pipelex/test_runtime_layer_import_closure.py@@ -64,14 +64,20 @@ import pytest#: sibling declaration already claimed was clean. The import now points at the definition site, per this#: repo's "direct full-path imports everywhere" rule, and this entry point is what keeps it there.#:-#: `kernel.pipelex_kernel` is the Pipelex kernel's widest module-level closure: the façade imports the-#: ops modules that hold the operator-execution semantics, so one entry point covers the package.-#: It earns a place here for a reason none of the others share — the kernel's *contract* is that a-#: programmatic caller can invoke these functions on a `RuntimeBoot`-only process with zero `.mthds`-#: loaded, which is this very property rather than a consequence of it. Note the blind spot the-#: kernel's own doctrine restates: this test sees module-level imports only, so a function-local-#: import inside a kernel module is invisible to it *and* to the static graph at once, and reviews-#: have to catch those by hand.+#: `kernel.pipelex_kernel` earns a place here for a reason none of the others share — the kernel's+#: *contract* is that a programmatic caller can invoke its functions on a `RuntimeBoot`-only process+#: with zero `.mthds` loaded, which is this very property rather than a consequence of it.+#:+#: It does **not** cover the kernel package, and no entry point here does: the façade is an LLM-era+#: one that imports `llm_ops` alone, so the per-operator ops modules are outside every closure this+#: tuple pins. They are covered instead by `tests/unit/pipelex/kernel/test_kernel_boot_contract.py`,+#: which imports *and calls* each of them and sweeps afterwards — strictly stronger than this test on+#: those modules, so listing them here as well would buy only a sharper failure message. Adding a+#: kernel module to this tuple is therefore worth it only if it is *not* reached by that call path.+#:+#: Note the blind spot the kernel's own doctrine restates: this test sees module-level imports only,+#: so a function-local import inside a kernel module is invisible to it *and* to the static graph at+#: once. That is precisely the hole the boot-contract test's post-call sweep exists to close.RUNTIME_LAYER_ENTRY_POINTS = ["pipelex.cogt.content_generation.content_generator","pipelex.runtime_hub",diff --git a/wip/kernel/deferred-follow-ups.md b/wip/kernel/deferred-follow-ups.mdindex 8ca93a1bf..df42cc9fa 100644--- a/wip/kernel/deferred-follow-ups.md+++ b/wip/kernel/deferred-follow-ups.md@@ -62,6 +62,78 @@ So the kernel result is behaving exactly like the type it mirrors, and the proje**Why it is deferred rather than fixed here.** The right fix depends on a question Phase 2 answers, not this PR: is `"structure"` a *fourth kernel structuring path*, or a pipe-level label that merely shares a dict key? Today it is the latter — `PipeStructure` calls `generate_object_content` directly and builds its own execution data, so the kernel never produces `"structure"` and adding the member now would put a value into a "what the kernel op did" type that no kernel op can return. When Phase 2 re-points `PipeStructure` fully, the answer falls out: either it starts flowing through a kernel entry point that legitimately returns a fourth path (add the member, drop the literal), or it stays a caller-side label (narrow the enum's docstring to say it names the *LLM* paths, not the whole key). Cheap either way; wrong to guess now.+## KF-7 — `PipeCompose`'s construct mode is not kernel-covered, and covering it means relocating an MTHDS blueprint++**Surfaced by** task 2.4 itself. The task read "kernel templating + structured-composition ops"; only the templating half landed.++**Why the construct half did not.** Its semantics are `StructuredContentComposer` over a `ConstructBlueprint`, both under `pipelex/pipe_operators/compose/` — an interpreter package by the closure test's own `INTERPRETER_PACKAGES` predicate, so the kernel cannot import either. There is no adapter shape that avoids this: the composer's *input* is the blueprint. Covering construct mode therefore means moving `construct_blueprint.py` (315 lines of MTHDS field-composition model, with its serializers and validators) and `structured_content_composer.py` (840 lines) into `pipelex/kernel/`, plus the `PipeComposeError` family they raise — which is a `PipelexError` subtree, so it drags `make generate-error-pages` and the `tests/data/errors/error_identity.txt` snapshot with it.++Phase 1's `LLMPromptBlueprint` precedent looks like it points the other way, and does not: there the kernel needed a *small* runtime-layer input model (`LlmPromptContent`) that the blueprint maps down onto. The equivalent here is a near-verbatim second copy of a 315-line blueprint, which is duplication, not layering.++**What the caller loses.** Close to nothing. A programmatic caller holds real Python — it builds its structured object directly rather than describing the construction in a blueprint. The blueprint exists so that `.mthds` can express composition declaratively, which is a language concern with a language-side consumer.++**What was done instead.** The one thing genuinely shared between the two paths — the three-layer context ordering (memory stuffs → run params → step `extra_context`) that both the template path and the construct path's TEMPLATE fields render against — is now single-sourced in `build_compose_context` and called from both. That is the drift this extraction exists to kill; the rest of construct mode was never at risk of forking, because it has exactly one caller.++**When to revisit.** If a second caller of construct-mode composition ever appears, or if `pipelex/kernel/` is packaged as a standalone distribution (a stated non-goal today) and the MTHDS blueprint models get a layer verdict of their own. Note the precedent that would support the move when it comes: `TemplateBlueprint` is also a `.mthds`-parsed artifact and already lives in the runtime layer, under `pipelex/cogt/templating/`.++## KF-8 — `PipeFunc`'s executor seam is not kernel-carried, so a kernel `run_func` always runs in this process++**Surfaced by** task 2.5 itself. The task read "kernel function-call op over the `PipeFuncExecutorProtocol` seam ... the kernel op must carry **both arms**"; the call and the write-back landed, the seam did not.++**Why the seam did not.** Neither arm's type survives the layer boundary. `PipeFuncExecutorProtocol.run_pipe_func` declares `pipe_run_params: PipeRunParams` and `run_pipe_func_transported` is typed on `PipeFuncExecutionRequest`, which carries a `LibraryCrate`. `pipe_run` and `libraries` are both interpreter packages, and the task's own analysis rules out moving those two models as far outside Phase 2. So the kernel cannot name the protocol it was told to take as an argument.++**The two ways to satisfy the letter, and why neither was taken.**++- *Re-type the protocol off `PipeRunParams`.* Cheap in this tree — nothing in it reads the parameter, including `DirectPipeFuncExecutor` — but it is not a cleanup, it is a wire-format change: both out-of-tree implementers (our Daytona sandbox plugin and our Temporal plugin) thread `pipe_run_params` straight onto `PipeFuncExecutionRequest`, which crosses a process boundary. Two repos this tree's CI cannot see would break at the same time, and the honest first question — *does anything downstream ever read it?* — is not answerable from here.+- *A kernel-side narrow protocol plus an interpreter-side adapter.* Buys a second protocol for one seam, an adapter class binding `pipe_run_params`, and a result-model conversion at the boundary — to compose two calls. Against a "no over-engineering" bar that is the more expensive of the two, not the safer one.++**What was done instead.** The split follows what is actually shared. `call_registered_function` holds what running a function *means* — registry lookup, async-vs-sync dispatch, the str/list→`StuffContent` coercion — and both the in-process executor and the kernel's own `run_func` ride it, so the executor and a programmatic caller cannot fork on it. `store_result` holds the write-back, which `PipeFunc` now rides too. What stays interpreter-side is *where the function runs*, which is configured deployment machinery rather than operator semantics.++**What the caller loses, stated plainly.** A programmatic caller cannot ask the kernel to run its function in a sandbox: `run_func` is the in-process path, full stop. And `run_func` has no in-tree caller — `PipeFunc` rides the pieces, not the composition — which is why it carries its own kernel unit tests rather than leaning on the zero-behavior-change suite.++**When to revisit.** Whenever `pipe_run_params` on the executor seam is settled on its own merits — the question is whether a serialized PipeFunc request needs the run params at all, and it wants an answer coordinated across the three repos, not a drive-by narrowing here. Phase 3's `JobMetadata` task (3.3) is the natural moment: it is already the task that asks what run-scoped state a kernel call carries, and this is the same question asked at the one seam that serializes it.++## KF-9 — Two constructors build `ImgGenJobParams` from the same config defaults with different derivation rules++**Surfaced by** cubic on PR #1082, against `build_img_gen_job_params` in `pipelex/kernel/img_gen_ops.py`: a transparent background with no step-level `output_format` builds params that fail `ImgGenJobParams.validate_background_vs_output_format`.++**The asymmetry is real.** `ImgGenJobParamsDefaults.make_img_gen_job_params` (`pipelex/cogt/img_gen/img_gen_job_components.py`) derives `output_format = ImageFormat.PNG` when `self.background.is_certainly_transparent`; the kernel's `build_img_gen_job_params` passes `output_format` straight through. Both read the same `img_gen_param_defaults`, and both are live — the defaults constructor is the fallback in `content_generator.py` and `img_gen_job_factory.py`.++**Why it is not a regression.** The pre-refactor `PipeImgGen._live_run_operator_pipe` constructed `ImgGenJobParams(...)` inline with the identical `output_format=self.output_format`; the operator path never went through `make_img_gen_job_params`. Verified against `git show refactor/Kernel:pipelex/pipe_operators/img_gen/pipe_img_gen.py`. The behavior moved unchanged.++**Reachability.** The shipped default is `background = "auto"`, so it does not fire out of the box. It fires under a `.pipelex/pipelex.toml` override of `[cogt.img_gen_config.img_gen_param_defaults] background = "transparent"`, which would then break every `PipeImgGen` that omits `output_format`. Note the same failure is already reachable today with no config override at all, via a step-level `background = "transparent"` with no `output_format` — so a fix confined to the config path would not close the case.++**Why it is deferred rather than fixed here.** Re-deriving PNG inside `build_img_gen_job_params` would make a currently-failing configuration start working — a behavior change inside a zero-behavior-change PR — and it is the wrong-shaped fix. The defect is that *two* constructors of the same type read the same defaults with divergent rules, which is precisely the duplication this extraction exists to kill. Single-source them; do not patch one side. Failure today is loud and well-worded, not silent corruption, so nothing is at risk in the meantime.++## KF-10 — `max_results` reaches a provider unvalidated because the constraint sits on the wrong model++**Surfaced by** cubic on PR #1082, against `resolve_search_setting` in `pipelex/kernel/search_ops.py`: `search_setting.model_copy(update={"max_results": max_results_override})` skips validation, because pydantic v2's `model_copy` does no validation by design.++**The chain is genuinely unconstrained end to end.** `PipeSearchBlueprint.max_results` is `int | None = None` with no `ge`; `PipeSearchFactory.make` passes it through to `PipeSearch.max_results_override`, also unconstrained; only `SearchSetting.max_results` carries `Field(default=None, ge=1)`, and the `model_copy` is what bypasses it. So `max_results = 0` in a `.mthds` reaches a search provider.++**Why it is not a regression.** Identical on the base branch, step 4 of `PipeSearch._live_run_operator_pipe`. Verified against `git show refactor/Kernel:pipelex/pipe_operators/search/pipe_search.py`.++**Why it is deferred rather than fixed here, and why the obvious fix is the wrong one.** Revalidating the copy (the bot's suggestion) would raise a pydantic `ValidationError` deep inside the search run path, with no `.mthds` locator to tell the author which step is wrong — trading an unvalidated value for a bad diagnostic. The constraint belongs on `PipeSearchBlueprint.max_results` as `Field(default=None, ge=1)`, where it is caught at validation time with a proper error. That changes the MTHDS JSON Schema and drags the committed downstream copies (`mthds/`, `vscode-pipelex/`, `mthds-ui/`) with it, which is unambiguously outside a zero-behavior-change refactor. Do it as a standalone language-surface change with the schema regeneration and the cross-repo sweep in the same PR.++## KF-11 — `PipeImgGen`'s `seed` is documented language surface that no image-generation provider reads++**Surfaced by** verifying cubic's P2 against `build_img_gen_job_params`, which reported that `seed or img_gen_param_defaults.seed` discarded an explicit `0`. That part was true and is **fixed in PR #1082** — the line now reads `seed if seed is not None else …`, matching the `is_raw` line below it. The larger fact came out of checking the claimed impact.++**The seed is inert.** `seed` is a declared `PipeImgGen` blueprint field (therefore in the published MTHDS JSON Schema), it is validated (`ImgGenJobParams.seed`, `ge=0`), and it is composed into `ImgGenJobParams` on every image-generation run — and no img-gen worker reads it. Grepping `job_params.seed` across every worker under `pipelex/providers/*/` finds exactly one image-generation reference, and it is commented out: `pipelex/providers/google/google_img_gen_worker.py`. (The live hits are all `LLMJobParams.seed`, a different field on a different job type.) Which is why the `0` fix is observationally a no-op and does not dent the PR's zero-behavior-change claim.++**The failure a user hits.** They write `seed = 12345` in a `PipeImgGen` step to reproduce an image, run it twice, and get two different images — no error, no warning, nothing indicating the parameter was discarded.++**Why it is deferred rather than fixed here.** It is pre-existing, entirely outside the kernel extraction, and the fix is per-provider plumbing (each SDK spells the seed differently, and not all of them accept one) plus a decision about what to do for backends that cannot honor it — reject at validation, or document the field as best-effort. Both are product calls. Recorded here because this is where it was found; it belongs in the img-gen backlog rather than the kernel one.++## KF-12 — `run_img_gen` treats a non-positive `nb_images` as one image, and what it *should* do is a Phase-3 question++**Surfaced by** Codex on PR #1082 (P2): `nb_images=0` or a negative count falls through the `if nb_images > 1:` fork to `make_single_image`, so an invalid request generates one billable image instead of being rejected.++**Why it is not a regression.** The fork is byte-identical to the base branch — `git show refactor/Kernel:pipelex/pipe_operators/img_gen/pipe_img_gen.py` has the same `if nb_images > 1:` at line 268, with the same `make_image_list` / `make_single_image` arms. The multiplicity derivation above it also moved unchanged: `nb_images = applied_output_multiplicity` whenever that is an `int`, with no lower bound anywhere on the way in.++**Why it is deferred rather than fixed here, and this one is genuine doubt rather than scope.** There is no zero-behavior-change fix available. Unlike the `run_search` signature narrowing in this PR — where the ignorable fields could simply be removed from the signature — `nb_images` is an `int` and plain Python cannot express `int >= 1` at the boundary, so any fix *adds an error path* to a PR whose contract is that it adds none. And the fix's shape is not obvious: `nb_images=0` could defensibly mean "reject this request" or "produce an empty `ListContent`", and picking between them from a bot comment rather than from the caller experience is exactly the mistake KF-2 records. Phase 3's task 3.1 (`shape_inputs`) is the task that decides what a kernel entry point does with a malformed argument; settle it there, for all the ops at once, rather than one `int` at a time.++**Worth noting on the way.** The interpreter path can reach it too — nothing between a `.mthds` `nb_output` and `nb_images` constrains the value — so if the answer turns out to be "reject", the constraint most likely belongs upstream on the blueprint, the same conclusion KF-10 reaches about `max_results`. The two should be settled together.## KF-16 — The model-derived prompting style is obsolete and should become an explicit choice**Numbered 16, not 7, deliberately.** This item lands on the *first* PR of a three-PR stack, but the stacked Phase 2 and Phase 3 PRs already add KF-7 through KF-15. Taking the next free number instead of the next sequential one keeps every one of those entries — and the cross-references to them in the plan — exactly as written, so folding this in churns neither of the PRs above it. The gap below is theirs to fill.diff --git a/wip/kernel/kernel-extraction-plan.md b/wip/kernel/kernel-extraction-plan.mdindex 640042ad8..e46fa2c8c 100644--- a/wip/kernel/kernel-extraction-plan.md+++ b/wip/kernel/kernel-extraction-plan.md@@ -2,12 +2,15 @@## Status block (update at every checkpoint)-- **Current phase:** **Phase 1 COMPLETE — CHECKPOINT A passed. PR #1081 is open against `dev` and must NOT be merged** (the series is stacked: 1/3 here, Phase 2 as 2/3, Phase 3 as 3/3). Branch `refactor/Kernel`, off `dev` with `dev` merged in at `221b8ee0b`. Phase 0 is `23276cecd`, Phase 1 tasks 1.1–1.6 are `3a11d113a`, task 1.7 is `1e6bb1ef2`; three review-driven commits follow (`b7c15aec8`, `9a446e9c5`, `c0815cf7f`). Phase 2 is untouched.-- **Gates at head (`c0815cf7f`):** `make agent-check` **green**, full `make agent-test` **green**, `make drift-check` **green** (`hub-layering-convention` acked twice — the fix below reopened it). CI green on every check. Two gate failures surfaced at this checkpoint and both are fixed — see "What Phase 1 taught Phase 2" below. Note for the record: this doc once claimed `make agent-check` was green at `3a11d113a`. It was not; the keyword-only gate was red there.+- **Current phase:** **Phase 2 COMPLETE — CHECKPOINT B reached. Two PRs are open against `dev` and NEITHER may be merged** (the series is stacked: 1/3 = PR #1081 on `refactor/Kernel`, 2/3 = this phase on `refactor/Kernel-phase2` based on it, Phase 3 to follow as 3/3). **Commits are named by subject here, not by SHA.** This branch is rebased every time `refactor/Kernel` moves, which rewrites every hash on it — so a SHA written into this block goes stale the next time that happens, and this block has already been caught carrying stale ones. Phase 1's record, on `refactor/Kernel`: the Phase 0 scaffolding, tasks 1.1–1.6, task 1.7, then three review-driven commits (the forced-DRY fix below among them). Phase 2, in order: the extract / img-gen / search / compose operator extraction (tasks 2.1–2.4); the function-call semantics plus the per-operator boot contract (task 2.5, with the extension the cold review forced); the bot-triage fixes; then the parity-gaps fold — `pipelex/kernel/img_gen_prompt.py` for that track's gap 2.3, and the boot-contract arm it turned out to owe — and the doc corrections from the bot round after it. Phase 3 is untouched.+- **Gates at Phase 2 head:** `make agent-check` **green**, full `make agent-test` **green**, `make drift-check` **green** — last full-suite run at the boot-contract-arm commit, everything after it being documentation only (not reopened — Phase 2 touched no hub module and no guard file, because the executor seam stayed interpreter-side; see the 2.5 deviation). CI green on every check on PR #1082.+- **Gates at Phase 1 head (`c0815cf7f`), kept for the record:** all three green, `hub-layering-convention` acked twice — the forced-DRY fix reopened it. CI green on every check. Two gate failures surfaced at Checkpoint A and both are fixed — see "What Phase 1 taught Phase 2" below. Note for the record: this doc once claimed `make agent-check` was green at `3a11d113a`. It was not; the keyword-only gate was red there.- **One real bug was found by review and fixed — the PR's only behavior change.** `PipelexKernel.make` minted a LIVE `CogtRunParams` on a keyless boot (`needs_inference=False`), bypassing the forced-DRY coercion `PipeRunParamsFactory` applies. Reachable, not hypothetical: `run_mode` alone gates mocking, so the call reached a real provider, and `needs_inference=False` means backends load *leniently* rather than credential-free. The hazard was the **default** — `PipelexKernel.make(user_id=…)` names no mode. Root cause is this branch's own doing: the factory's docstring called itself "the single writer" of `run_mode`, which stopped being true when the kernel became a second run-params factory. The rule now lives once in `runtime_hub.resolve_run_mode_for_boot`, called by both factories, pinned by a third arm on the D4 contract module. **Phase 2 lesson: any new kernel-side factory of run-scoped state owes the same audit — ask which process-wide invariants the pipe tier applies that the kernel tier now skips.**-- **Review record:** a cold `/code-review` (no inherited context) found no correctness bug, no layering violation and no stale reference, and confirmed the three past bug fixes in this code (templating-style mutation, the multi-image/document dict-key collision, LLM-error cause unwrapping) preserved verbatim. Bots: Greptile 5/5 on head; cubic found the forced-DRY bug above; Codex raised one item (polymorphic serialization of `LlmObjectResult.content`) four times, independently verified and rejected twice — see KF-5.-- **Next action:** Phase 2, starting at task 2.1. **Read "What Phase 1 taught Phase 2" below first** — it lists four move-consequence classes and two traps that are not obvious from the task list, one of which (2.5) makes that task not a move at all.-- **Open decisions:** none blocking. The kernel construction shape (0.2), structure-prompt provenance and the deferred runtime-boot orchestrator question are settled under Decisions; seven deviations from the plan's letter are recorded there. Deferred items are KF-1..KF-6 in [`deferred-follow-ups.md`](deferred-follow-ups.md).+- **Review record — Phase 1 (PR #1081):** a cold `/code-review` (no inherited context) found no correctness bug, no layering violation and no stale reference, and confirmed the three past bug fixes in this code (templating-style mutation, the multi-image/document dict-key collision, LLM-error cause unwrapping) preserved verbatim. Bots: Greptile 5/5 on head; cubic found the forced-DRY bug above; Codex raised one item (polymorphic serialization of `LlmObjectResult.content`) four times, independently verified and rejected twice — see KF-5.+- **Review record — Phase 2 (PR #1082):** the cold `/code-review` found the boot-contract gap (see "What the cold review changed" below), which was the one substantive finding. Bots across two rounds: Greptile pass 5/5 both times with no actionable defects; cubic found four items and came back "no issues" after the fixes; Codex found three. **Three were fixed**, in *honour an explicit seed of 0 and stop run_search taking fields it ignores* — named by subject rather than hash, because this branch is rebased whenever its parent moves and the hash this line used to carry now resolves from no branch at all: the img-gen seed read with `or`, swallowing an explicit `0` that `ImgGenJobParams` accepts under its own `ge=0`; `run_search` taking a `TemplateBlueprint` and using two of its four fields; and an architecture-overview sentence *this branch introduced* claiming kernel ops do "no hub lookups" (they read `runtime_hub` freely — what they never touch is the interpreter layer). **Four were deferred** as KF-9..KF-12, each verified pre-existing against `git show refactor/Kernel:<path>` first. **One was rejected**: a P1 asking for a single-consumer test fixture to move to a `conftest.py` that does not exist, against a rule whose own text says "for easy sharing". Two of the deferred four are worth carrying into Phase 3 as one item, not two — KF-10 (`max_results`) and KF-12 (`nb_images`) are the same defect, a value flowing from blueprint to provider with no bound anywhere on the path, and in both cases the constraint belongs upstream on the blueprint rather than where the bot pointed.+- **Next action:** Phase 3, starting at task 3.1. Two things Phase 2 leaves on its doorstep: `pipelex/kernel/memory_ops.py` already exists and is where 3.1 (`shape_inputs`) and 3.2 (result extraction) belong; and 3.3's run-scoped-state question is the natural moment to settle KF-8, since the PipeFunc executor seam is the one place run params actually serialize. "What Phase 1 taught Phase 2" below still applies verbatim to Phase 3 — the four move-consequence classes are properties of moving code, not of Phase 2.+- **What Phase 2 turned out to be, in one line:** four operators were genuine extractions (2.1–2.4 moved deck resolution, job-params assembly, generation and write-back into `pipelex/kernel/`); the fifth (2.5) was not a move and did not become one — its seam is typed on interpreter models, so what moved was the *call*, not the pluggable indirection.+- **Open decisions:** none blocking. The kernel construction shape (0.2), structure-prompt provenance and the deferred runtime-boot orchestrator question are settled under Decisions, alongside every deviation from the plan's letter taken in Phases 1 and 2. Deferred items are KF-1..KF-12 in [`deferred-follow-ups.md`](deferred-follow-ups.md).### What Phase 1 taught Phase 2@@ -20,8 +23,8 @@ Read this before starting 2.1. **Moving code has consequences no compiler report**Silent (nothing tells you):**-3. **Under-the-hood docs naming moved files.** Phase 1 broke a file path in `docs/under-the-hood/image-handling-in-llm-prompts.md` and nothing noticed — no drift contract has a trigger glob over `pipelex/pipe_operators/**` or `pipelex/kernel/**`, and `mkdocs build --strict` does not validate inline code spans in a table. Grep `docs/` for every path you move. Already known for Phase 2: `docs/under-the-hood/dry-run-mock-generation.md` names `pipe_operators/compose/structured_content_composer.py`, a task-2.4 target, and `docs/under-the-hood/architecture-overview.md` still presents `pipe_operators/` as the sole home of operator behavior.-4. **`docs/errors/` pages carry the defining module path.** The generator emits ``| Defined in | `<module>` |`` onto each page, so moving a `PipelexError` subclass makes its committed page stale — and `make generate-error-pages` runs in neither `make check` nor CI. Phase 1 escaped this only by luck: `LLMPromptBlueprintValueError` and its replacement `PromptContentError` are bare `ValueError`s, so `tests/data/errors/error_identity.txt` never moved. Phase 2 moves error classes that *are* `PipelexError`s (search, img-gen, compose, extract). Two chores follow: regenerate the pages, and note that the generator's `_SUBSYSTEM_SECTIONS` has no `kernel` key, so a moved error currently falls through to the `platform-and-tooling` fallback under a humanized "Kernel" heading instead of "Execution & runtime".+3. **Under-the-hood docs naming moved files.** Phase 1 broke a file path in `docs/under-the-hood/image-handling-in-llm-prompts.md` and nothing noticed — no drift contract has a trigger glob over `pipelex/pipe_operators/**` or `pipelex/kernel/**`, and `mkdocs build --strict` does not validate inline code spans in a table. Grep `docs/` for every path you move. Already known for Phase 2: `docs/under-the-hood/dry-run-mock-generation.md` names `pipe_operators/compose/structured_content_composer.py`, a task-2.4 target, and `docs/under-the-hood/architecture-overview.md` still presents `pipe_operators/` as the sole home of operator behavior. Both settled in Phase 2: the composer did not move, so its reference is still accurate, and the architecture overview now states the split. The Phase-3 equivalent is task 3.4's kernel page, which should be written against these two rather than beside them.+4. **`docs/errors/` pages carry the defining module path.** The generator emits ``| Defined in | `<module>` |`` onto each page, so moving a `PipelexError` subclass makes its committed page stale — and `make generate-error-pages` runs in neither `make check` nor CI. Phase 1 escaped this only by luck: `LLMPromptBlueprintValueError` and its replacement `PromptContentError` are bare `ValueError`s, so `tests/data/errors/error_identity.txt` never moved. Phase 2 was predicted to move error classes that *are* `PipelexError`s (search, img-gen, compose, extract). **It moved none** — each operator's error family stayed with the interpreter-side code that raises it, because the kernel ops raise the same cogt-level errors the moved code raised and the operators rewrap as they always did. So the two chores this item names went unexercised and are still owed by whoever first moves a `PipelexError` into `pipelex/kernel/`: regenerate the pages, and note that the generator's `_SUBSYSTEM_SECTIONS` has no `kernel` key, so a moved error falls through to the `platform-and-tooling` fallback under a humanized "Kernel" heading instead of "Execution & runtime".**Two traps that are not move-consequences:**@@ -39,9 +42,17 @@ Read this before starting 2.1. **Moving code has consequences no compiler report| `prompt_references.py` | `ImageReference(Kind)` and `DocumentReference(Kind)`, **moved** out of `pipe_operators/` (they could not stay there: a runtime-layer prompt-content model may not import an interpreter-layer type). All importers re-pointed; the two unit-test modules moved to `tests/unit/pipelex/kernel/`. || `llm_prompt_content.py` | `LlmPromptContent` (+ `make_from_text`) and `assemble_llm_prompt` — the whole former `LLMPromptBlueprint.make_llm_prompt` body, images/documents/registry/placeholders included. || `llm_results.py` | `StructuringPath` StrEnum, `LlmTextResult`, `LlmObjectResult`. |-| `llm_ops.py` | `resolve_llm_setting_for_text`, `resolve_llm_setting_for_object`, `derive_templating_style`, `derive_structure_prompt`, `generate_object_content`, `store_result`, `run_llm_text`, `run_llm_object`. |+| `llm_ops.py` | `resolve_llm_setting_for_text`, `resolve_llm_setting_for_object`, `derive_templating_style`, `derive_structure_prompt`, `generate_object_content`, `run_llm_text`, `run_llm_object`. |+| `memory_ops.py` | `store_result` — the write-back every operator ends with. **Phase 2 moved it out of `llm_ops.py`** (see Deviations); this is also where Phase 3's 3.1 and 3.2 belong. |+| `extract_ops.py` / `extract_results.py` | `resolve_extract_setting`, `build_extract_job_params`, `run_extract`; `ExtractResult`. |+| `img_gen_ops.py` / `img_gen_results.py` | `resolve_img_gen_setting`, `resolve_default_size`, `build_img_gen_job_params`, `run_img_gen`; `ImgGenResult`. |+| `search_ops.py` / `search_results.py` | `resolve_search_setting`, `run_search`; `SearchResult`. |+| `compose_ops.py` / `compose_results.py` | `build_compose_context`, `build_composed_content`, `run_compose_template`; `ComposeResult`. Construct mode stayed interpreter-side — see Deviations. |+| `func_ops.py` / `func_results.py` | `call_registered_function`, `run_func`; `FuncCallResult`, `FuncResult`. The pluggable executor seam stayed interpreter-side — see Deviations. || `exceptions.py` | `PromptContentError(ValueError)` — the former `LLMPromptBlueprintValueError`, moved with the code that raises it. |+`PipelexKernel` is **LLM-only and stayed that way through Phase 2** — the five operator ops above are module-level functions with no façade method, and the interpreter calls them directly. See Decisions for why, and note the test consequence: `test_runtime_layer_import_closure.py`'s `pipelex.kernel.pipelex_kernel` entry point does not reach them, so `test_kernel_boot_contract.py` is their only gate.+Interpreter side: `LLMPromptBlueprint` keeps its fields, `required_variables()` and its `make_llm_prompt` **signature** (its many test call sites depend on it) but now maps down through a new `to_prompt_content()` and delegates; `PipeLLM._live_run_operator_pipe` calls `run_llm_text` / `run_llm_object` and its `_llm_gen_object_stuff_content` is gone; `PipeStructure` calls `resolve_llm_setting_for_object`, `derive_structure_prompt`, `generate_object_content` and `store_result` (not `run_llm_object` — see Decisions); `pipe_operators/llm/helpers.py` (`get_output_structure_prompt`) is deleted.### Verified in Phase 0 (do not redo)@@ -79,7 +90,7 @@ This completes the runtime/interpreter layering arc: the hub split (#1062/#1064)## Target caller experience-The sketch below is the API shape to preserve while the internals get extracted for real out of `pipe_llm.py` (which is where the deck chain, style derivation, and prompt assembly currently live). Every parameter is keyword-only, including `memory`. Two contracts the sketch elides for brevity, stated here because they are load-bearing: `user` and `system` are prompt *templates* rendered against `memory` inside the kernel — template references are how images and documents enter the prompt (task 1.3's assembly functions do the rendering) — and the return value is a small typed result carrying the updated memory plus the rendered prompts, the resolved model setting, and the structuring path taken, because the interpreter's execution-graph tracer consumes those intermediates after generation and recomputing them outside the kernel would mean a second assembly path. Expect the signatures to grow explicit keyword parameters for setting overrides and output multiplicity as tasks 1.1–1.4 extract the real thing; the shape that must survive is fully-named arguments over explicit state. One limit the sketch does not show, now that Phase 1 has landed: **the façade methods carry no image or document references at all** — both hardcode `LlmPromptContent.make_from_text(...)`, and a caller needing references drops to `run_llm_text` / `run_llm_object` with an `LlmPromptContent` it builds itself. Phase 2 brings image-bearing operators (2.1 `PipeExtract`, 2.2 `PipeImgGen`), so decide there whether `PipelexKernel` grows reference-carrying parameters or stays a deliberately text-only convenience. The sketch above already reflects one growth the plan did not anticipate: folding the parity-gaps track in widened `llm_text` to take an optional `concept` and `output_class`, and made `model` optional too, so the façade can express what the op beneath it always could — see the 2.1 entry in that track and the `llm_text` docstring for why each of the three has to be reachable.+The sketch below is the API shape to preserve while the internals get extracted for real out of `pipe_llm.py` (which is where the deck chain, style derivation, and prompt assembly currently live). Every parameter is keyword-only, including `memory`. Two contracts the sketch elides for brevity, stated here because they are load-bearing: `user` and `system` are prompt *templates* rendered against `memory` inside the kernel — template references are how images and documents enter the prompt (task 1.3's assembly functions do the rendering) — and the return value is a small typed result carrying the updated memory plus the rendered prompts, the resolved model setting, and the structuring path taken, because the interpreter's execution-graph tracer consumes those intermediates after generation and recomputing them outside the kernel would mean a second assembly path. Expect the signatures to grow explicit keyword parameters for setting overrides and output multiplicity as tasks 1.1–1.4 extract the real thing; the shape that must survive is fully-named arguments over explicit state. One limit the sketch does not show, now that Phase 1 has landed: **the façade methods carry no image or document references at all** — both hardcode `LlmPromptContent.make_from_text(...)`, and a caller needing references drops to `run_llm_text` / `run_llm_object` with an `LlmPromptContent` it builds itself. Phase 2 brings image-bearing operators (2.1 `PipeExtract`, 2.2 `PipeImgGen`), so decide there whether `PipelexKernel` grows reference-carrying parameters or stays a deliberately text-only convenience. **Decided in Phase 2 — it stays LLM-only; see Decisions.** The sketch above already reflects one growth the plan did not anticipate: folding the parity-gaps track in widened `llm_text` to take an optional `concept` and `output_class`, and made `model` optional too, so the façade can express what the op beneath it always could — see the 2.1 entry in that track and the `llm_text` docstring for why each of the three has to be reachable.```pythonclass PipelexKernel:@@ -159,11 +170,13 @@ Each item is a fragment of `PipeLLM` to extract into a kernel function for real:Same treatment, one operator at a time, interpreter re-pointed as each lands. Each task includes kernel unit tests for that operator's kernel-only arms — the paths the re-pointed interpreter suite never takes are exactly where the two callers could drift undetected.-- [ ] 2.1 `PipeExtract` → kernel extract ops over the `make_extract_pages` seam, including the page-view handling (`should_include_page_views`, `page_views_dpi`, `max_page_images`).-- [ ] 2.2 `PipeImgGen` → kernel image ops (single / list).-- [ ] 2.3 `PipeSearch` → kernel search ops (sourced answer / structured).-- [ ] 2.4 `PipeCompose` → kernel templating + structured-composition ops.-- [ ] 2.5 `PipeFunc` → kernel function-call op over the **`PipeFuncExecutorProtocol` seam**, not a bare registry lookup. The executor is pluggable — `run_pipe_func` (live objects) and `run_pipe_func_transported` (serialized request/response via `pipe_func_execution_dtos`) are its two arms — and the kernel op must carry **both arms**. But the kernel never reads `HubSlot.PIPE_FUNC_EXECUTOR`: that slot is applied in `Pipelex.setup`, so it does not exist on a runtime-only boot, and an ambient read would break the boot contract for this one op. Instead the executor enters as an explicit protocol-typed argument — the interpreter passes its hub-resolved one (selected by `pipe_func_config.execution_mode`, `direct` by default), a programmatic caller passes the direct executor or its own.+- [x] 2.1 `PipeExtract` → kernel extract ops over the `make_extract_pages` seam, including the page-view handling (`should_include_page_views`, `page_views_dpi`, `max_page_images`). Landed as `resolve_extract_setting` / `build_extract_job_params` / `run_extract` in `kernel/extract_ops.py`, with `ExtractResult` in `kernel/extract_results.py`.+- [x] 2.2 `PipeImgGen` → kernel image ops (single / list). Landed as `resolve_img_gen_setting` / `resolve_default_size` / `build_img_gen_job_params` / `run_img_gen` in `kernel/img_gen_ops.py`, with `ImgGenResult` in `kernel/img_gen_results.py`. `resolve_default_size` **moved** out of `pipe_img_gen.py` (it resolves a *config* default, which is kernel business); its unit test followed it to `tests/unit/pipelex/kernel/test_img_gen_size_default.py` and `.test_durations` was path-rewritten accordingly.+- [x] 2.3 `PipeSearch` → kernel search ops (sourced answer / structured). Landed as `resolve_search_setting` / `run_search` in `kernel/search_ops.py`, with `SearchResult` in `kernel/search_results.py`. The kernel renders the query template and hands the rendered text back for the tracer. It takes the template as `template: str` + `category: TemplateCategory` rather than as the `TemplateBlueprint` it first landed with: a search renders against memory alone, so a blueprint handed over `templating_style` and `extra_context` that `run_search` silently ignored — harmless on the language path (`PipeSearchFactory` hardcodes the blueprint) but a trap on a caller-facing surface. Narrowing the signature, rather than merging the two fields in, keeps the zero-behavior-change contract and makes the op symmetric with `run_compose_template`.+- [x] 2.4 `PipeCompose` → kernel templating ops. Landed as `build_compose_context` / `build_composed_content` / `run_compose_template` in `kernel/compose_ops.py`, with `ComposeResult` in `kernel/compose_results.py`. **Construct mode stays interpreter-side — a deliberate deviation, see Decisions.** `StructuredContentComposer` now builds its TEMPLATE-field context through the kernel's `build_compose_context`, so the two paths' three-layer ordering is single-sourced, and its dead `pipe_run_params` field is gone (stored, threaded to nested composers, never read — its docstring's "used to check if we're in dry run mode" was stale). That removal is what leaves the composer importing nothing from `pipe_run`.+- [x] 2.5 `PipeFunc` → kernel function-call op. Landed as `call_registered_function` / `run_func` in `kernel/func_ops.py`, with `FuncCallResult` / `FuncResult` in `kernel/func_results.py`. `call_registered_function` **moved** out of `DirectPipeFuncExecutor.run_pipe_func` — registry lookup, async-vs-sync dispatch and the str/list→`StuffContent` coercion are what running a function *means*, and they are now defined once for both the in-process executor and a programmatic caller. `PipeFunc` keeps its hub-resolved executor call and rides `store_result` for the write-back. **The pluggable executor seam stays interpreter-side — a deliberate deviation, see Decisions.** Kernel unit tests cover `run_func` (which no interpreter test reaches) and both dispatch arms.++ The original task text, kept because its analysis of the seam is what the deviation rests on: `PipeFunc` → kernel function-call op over the **`PipeFuncExecutorProtocol` seam**, not a bare registry lookup. The executor is pluggable — `run_pipe_func` (live objects) and `run_pipe_func_transported` (serialized request/response via `pipe_func_execution_dtos`) are its two arms — and the kernel op must carry **both arms**. But the kernel never reads `HubSlot.PIPE_FUNC_EXECUTOR`: that slot is applied in `Pipelex.setup`, so it does not exist on a runtime-only boot, and an ambient read would break the boot contract for this one op. Instead the executor enters as an explicit protocol-typed argument — the interpreter passes its hub-resolved one (selected by `pipe_func_config.execution_mode`, `direct` by default), a programmatic caller passes the direct executor or its own.⚠ **This is the highest-risk task in Phase 2, and it is not a move.** An earlier draft of this plan said the protocol "needs a runtime-layer home, since it currently lives in `pipe_operators/func/`", framing it as placement. It is not: the protocol and its DTOs are *typed on interpreter-layer models*. `pipe_func_executor_protocol.py` imports `PipeRunParams`; `pipe_func_execution_dtos.py` imports `PipeRunParams` **and** `LibraryCrate`; `pipe_func_execution_transport.py` imports `get_current_library` / `get_library_manager` straight from `interpreter_hub`, which is a flat layering violation the instant it lands in `pipelex/kernel/`. Both `pipe_run` and `libraries` are interpreter packages. The kernel is clean today *because* it uses `CogtRunParams` and never `PipeRunParams`, and the kernel's own doctrine forbids the usual escape hatch ("if a function-local import seems necessary, the placement is wrong and the type should move instead"). So the real task is **re-typing the protocol off `PipeRunParams`/`LibraryCrate`, or keeping an interpreter-side adapter** — not relocating a file. Moving those two models is far outside Phase 2's scope.@@ -175,10 +188,27 @@ Same treatment, one operator at a time, interpreter re-pointed as each lands. EaAlso relevant: 2.5 must touch `pipelex/interpreter_hub.py`, which reopens the `hub-layering-convention` drift contract — and this time the review target is genuinely falsified, because `docs/contribute/hub-layering.md` states in several places that the pipe-func accessor is interpreter-layer *because* its protocol lives in `pipe_operators/func/`.-**CHECKPOINT B** — all operators riding the kernel; gates green; this doc updated.+### What the cold review changed — the boot contract was claimed for six operators and tested for one++A cold `/code-review` of the whole Phase-2 diff returned one finding, and it was right: the boot contract was being *asserted* for every operator and *tested* for the LLM arms alone. Worth recording in full, because the gap was invisible from inside the phase and the same shape will recur in Phase 3.++`test_kernel_boot_contract.py` is the only gate that catches a **function-local interpreter import** inside a kernel function — the static hub-layering guard and the import-closure test both see module-level imports only, which the kernel's own doctrine says out loud. Phase 1 built that gate for `llm_object`/`llm_text` and Phase 2 added five operator ops without extending it. The blind spot is per-function: a function-local import inside `run_search` is caught only by calling `run_search`.++Worse, the *import-time* gate did not reach them either. `test_runtime_layer_import_closure.py` lists `pipelex.kernel.pipelex_kernel` as its kernel entry point on the strength of a comment claiming "the façade imports the ops modules … so one entry point covers the package". That was true in Phase 1 and Phase 2 silently falsified it: `PipelexKernel` is an LLM-era façade that imports `llm_ops` alone. So `extract_ops`, `img_gen_ops`, `search_ops`, `compose_ops` and `func_ops` were policed by **neither** test.++Fixed by extending the boot-contract subprocess to call all seven entry points on the same keyless `RuntimeBoot` and sweep afterwards. All five were drivable in DRY exactly as the LLM arms are — every cogt leaf mocks before a worker is looked up — so the settings are built literally rather than through the `resolve_*_setting` helpers, which read the deck and would be asking a different question. The stale closure-test comment is corrected to say what actually covers those modules and when adding one there would still be worth it. **Verified by mutation, not by assertion:** a function-local `pipe_operators` import planted in `run_search` makes the extended test fail with 72 offending modules, and passes without it.++Two things surfaced while writing it, both recorded where they belong rather than here:++- `ExtractResult.content` is the one kernel envelope field annotated below `StuffContent` (`ListContent[PageContent]`), so pydantic revalidates the plain `ListContent` that `run_extract` builds into a fresh container — making `result.content` a different object from the one the returned memory holds, uniquely among the ops. Nothing is lost (the `PageContent` items pass through by identity) and the interpreter is unaffected, since `PipeExtract` reads only `.memory`. The narrowing is worth keeping — a caller gets `.items` typed without a cast — so it is documented in `extract_results.py` instead of being flattened for uniformity.+- The per-op check in the boot-contract test is therefore by type, not by identity. Asserting identity would have been asserting a coincidence.++**CHECKPOINT B — PASSED.** All operators riding the kernel; `make agent-check`, full `make agent-test` and `make drift-check` green; this doc updated. Scope honestly stated: "riding the kernel" means every operator's produced content now reaches memory through `store_result`, and every operator's *resolvable* semantics (deck chains, job params, generation, templating, function invocation) are kernel functions with the interpreter as one caller. Two arms deliberately did not follow — `PipeCompose`'s construct mode and `PipeFunc`'s executor indirection — both recorded under Decisions and in `deferred-follow-ups.md`. The boot contract is now pinned for every operator rather than claimed for them.## Phase 3 — memory boundary and run-scoped state parity+**Standing rule, learned the hard way in Phase 2:** every new kernel entry point gets called in `test_kernel_boot_contract.py`'s subprocess. It is the only gate that sees a function-local interpreter import, it is per-function, and neither static guard will tell you when it has fallen behind — Phase 2 added five ops and both gates stayed green while covering none of them. Adding the call is a few lines; noticing it was never added took a cold review.+- [ ] 3.1 Boundary shaping: kernel `shape_inputs` over the existing `InputShaper` (per-signature specialization is out of scope).- [ ] 3.2 Result extraction helpers (main-stuff and named-slot, typed).- [ ] 3.3 Per-step `JobMetadata` semantics (per-step `pipe_run_id` minted from the instance's run-level metadata via `copy_with_update`, per the 0.2 constraint) as a kernel duty; optional trace-context wiring so cost/usage reporting reaches parity when a caller provides one. This task must also answer the lifecycle-ownership question it implies: the interpreter's run machinery is what opens and closes usage reporting today, so state who owns that lifecycle for a kernel-driven run. **Parity bar, measured not asserted:** a test runs an equivalent step through the re-pointed interpreter and through direct kernel calls (mock usage) and asserts the two produced `TokensUsageRecord` lists ([`pipelex/reporting/usage_records.py`](../../pipelex/reporting/usage_records.py)) match in shape and count — the list `/execute` returns on `pipe_output.tokens_usages` and durable runs persist as `tokens_usages.json`.@@ -192,6 +222,16 @@ Same treatment, one operator at a time, interpreter re-pointed as each lands. Ea- **The deferred runtime-boot orchestrator question — SETTLED: recorded, not guarded.** `runtime_boot.py` deferred the external-interpreter-orchestrator half-application hole to "the first caller of a runtime-only boot", which is now the kernel's boot-contract test. That caller does not trigger it, and not by luck: the gate is only reached when `plugins.boot_orchestrator` is non-`None`, and the test names none — as no kernel caller will, since the kernel's doctrine forbids it from reading `HubSlot` at all and Phase 2's `PipeFunc` op takes its executor as an explicit argument for that reason. The remedy stays unbuilt: both candidates are real machinery (a `HubSlot.is_interpreter_slot` property plus a `honours_interpreter_slots` class attribute, or a widened `build_registrar` signature) bought for a path with still no production caller. Reassess when a runtime-only boot is offered to real callers — that is when naming an orchestrator becomes reachable. Written up in [`wip/boot-split/runtime-boot-external-interpreter-orchestrator.md`](../boot-split/runtime-boot-external-interpreter-orchestrator.md), and `runtime_boot.py`'s comment now points at the settlement rather than at an open question.- **Kernel construction shape** (task 0.2) — **SETTLED as implemented.** `PipelexKernel` holds exactly `job_metadata` (run-level, fresh `uuid4` `pipeline_run_id`) and `cogt_run_params`. `make_step_metadata()` mints the per-step copy with a fresh `pipe_run_id` and an explicit `otel_context=None`. Nothing config- or deck-derived is cached.+### Deviations from the plan's letter, taken during Phase 2++- **`PipeFunc`'s pluggable executor seam stays interpreter-side (task 2.5).** The task asked the kernel op to take the executor as an explicit protocol-typed argument, carrying both arms. Neither arm's type survives the layer boundary: `PipeFuncExecutorProtocol.run_pipe_func` declares `pipe_run_params: PipeRunParams` and `run_pipe_func_transported` is typed on `LibraryCrate`, both interpreter models the task itself rules out moving. That leaves two ways to satisfy the letter, and both were rejected on cost. *Re-typing the protocol* breaks two out-of-tree implementers (our Daytona and Temporal plugins) which thread `pipe_run_params` onto the transported request — for a parameter nothing in either repo actually reads, which makes it a wire-format question, not a cleanup. *A kernel-side protocol plus an interpreter adapter* buys a second protocol for one seam, a wrapper class, and a model conversion at the boundary — to compose two calls. So the composition (`run_func`) is the kernel's in-process path, and what is genuinely shared — the call itself and the write-back — is single-sourced in `call_registered_function` and `store_result`, which is what the interpreter's default executor and operator now ride. Consequence to hold onto: a programmatic caller cannot dispatch a kernel `run_func` to a sandbox, and `run_func` has no in-tree caller, which is why it is unit-tested directly. Recorded in [`deferred-follow-ups.md`](deferred-follow-ups.md) as KF-8.+- **`PipeCompose`'s construct mode stays interpreter-side (task 2.4).** Its semantics are `StructuredContentComposer` (840 lines) over a `ConstructBlueprint` (315 lines), and the kernel cannot import either: `pipe_operators` is an interpreter package by the closure test's own predicate. So covering construct mode means *relocating both into `pipelex/kernel/`* — an MTHDS field-composition blueprint, its composer, and the `PipeComposeError` family they raise (which drags the `docs/errors/` regeneration and the `error_identity.txt` snapshot). Phase 1's `LLMPromptBlueprint` precedent points the other way but does not apply: there the kernel needed a *small* input model (`LlmPromptContent`) and the blueprint mapped down; the equivalent here is a near-verbatim copy of a 315-line model. Weighed against that cost, the caller-side gain is close to nil — a programmatic caller holding real Python builds the object directly rather than describing it in a blueprint. The template path, which *is* pure rendering over memory, is fully extracted, and the one thing both paths share (the three-layer context ordering) is now single-sourced in `build_compose_context`. Recorded in [`deferred-follow-ups.md`](deferred-follow-ups.md) as KF-7.+- **`PipelexKernel` stays an LLM-only façade — the reference-carrying question the plan assigned to Phase 2 is answered "no".** The task at "Target caller experience" was to decide, once image-bearing operators landed, whether the façade grows reference-carrying parameters. It did not grow at all: the five Phase-2 ops are module-level functions with no façade method. Two reasons. The façade's job is holding *per-run state* a caller would otherwise thread through every call (`job_metadata`, `cogt_run_params`), and that state is identical for every op — a façade method per operator would add a naming surface without adding a service. And the ops' remaining arguments are the operator-specific ones (`ExtractInput`, `ImgGenPrompt`, `output_class`, a template and its category), which a façade would have to enumerate verbatim; wrapping them buys nothing and forks the signature the moment an op grows a parameter. So a programmatic caller mints a `PipelexKernel` for the run state and calls the module-level functions with it, which is what the boot-contract test does. **Consequence worth carrying into Phase 3:** `test_runtime_layer_import_closure.py`'s only kernel entry point is `pipelex.kernel.pipelex_kernel`, so an LLM-only façade means that test's closure does not reach the other ten kernel modules — the standing rule at the head of Phase 3 exists for exactly this reason.+- **The kernel ops modules gained a shared `memory_ops.store_result`.** Phase 1 put it in `llm_ops.py`; every Phase-2 operator ends the same way, and importing it *from `llm_ops`* would have implied an LLM dependency that does not exist. It moved to `pipelex/kernel/memory_ops.py`, which is also where Phase 3's boundary shaping (3.1) and result extraction (3.2) belong.+- **Not every operator earned a rich result envelope, and that was decided per operator** (the plan's trap #6). Each envelope carries exactly the intermediates the kernel *produces* and the caller cannot recompute: `SearchResult` carries the rendered query and the post-resolution `SearchSetting` (which is not the one passed in); `ComposeResult` the rendered text; `ExtractResult` the resolved `ExtractSetting`. `ImgGenResult` carries only memory + content, because everything the img-gen tracer records is either a caller input or something the caller resolved through an ops function before calling. The memory contract is why even the thin one exists.+- **The kernel gained an image-prompt constructor the plan never scheduled (`pipelex/kernel/img_gen_prompt.py`).** Folded in from the parity-gaps track (its gap 2.3), on the same reasoning as Phase 1's `llm_text` widening: it is a defect *this* PR would otherwise introduce. `run_img_gen` takes a *ready* `ImgGenPrompt` by deliberate analogy with `run_llm_text` taking a ready `LlmPromptContent`, but the kernel shipped the LLM prompt's constructor and no image counterpart — so a `RuntimeBoot`-only process could call every operator op except the image one, and `kernel/prompt_references.py` hosted `ImageReference` under a docstring asserting the kernel resolves them while the interpreter-layer blueprint did the resolving. A "callable without the interpreter" guarantee at less than 100% is worth nothing. `ImgGenPromptBlueprint` drops 296 → 82 lines and delegates, keeping parse-and-validate; `ImgGenPromptBlueprintValueError` becomes `PromptContentError`, the same swap Phase 1 made on the LLM side. **The predicted cost did not materialise:** the parity plan expected the move to drag `ImageRegistry`, `PromptImageFactory`, `substitute_nested_in_context` and `render_template` across a layer boundary and force a scoping decision — all four are already `cogt`/`tools` citizens, and the blueprint held exactly one interpreter-layer import. Nothing moved layers. `max_prompt_images` deliberately stayed with the blueprint: it is a property of the model the *pipe* chose, its breach is a pipe-level error, and it is a `len()` check rather than the subtlety the kernel module exists to centralise. Two now-dead subject grants were removed rather than re-granted, per this package's zero-grants rule — exactly the sweep this plan warns about for every operator moved.+- **The new entry point took the boot-contract arm the standing rule owes it, and the arm replaced a hand-built prompt.** `assemble_llm_prompt` never needed one — `run_llm_text` and `run_llm_object` both call it, so the LLM arms carry its closure. `run_img_gen` takes a ready prompt and calls nothing, so the img-gen arm had been *constructing* its `ImgGenPrompt` in the test, for the plain reason that nothing in the kernel could build one. That made the arm's coverage incomplete in precisely the way the kernel surface was. It now assembles from a memory holding a referenced image and generates from the result, asserting on the `[Image N]` token *and* on `input_images` — the two readings of one registry, whose disagreement would mislabel which image the prompt describes with nothing downstream able to notice.+### Deviations from the plan's letter, taken during Phase 1- **The deck chain is two functions, not one `resolve_llm_settings`.** `resolve_llm_setting_for_text` and `resolve_llm_setting_for_object` (the latter taking the optional `llm_choice_for_text` that makes its chain one rung longer). A single function returning both settings would have made `PipeStructure` — which resolves only the object setting today — newly resolve the text one too, which is a behavior change on the one gate that matters. Two functions keep each caller resolving exactly what it resolved before, and the chain is still single-sourced.