Skip to content

Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.

fix: stop the runtime silently substituting a caller's class and a caller's function (#1076)

Louis Choquel8d00d5db916a
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 53080b0ee..8f7c5293f 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,5 +1,16 @@
# Changelog
+## [Unreleased]
+
+### Fixed
+
+- **Structured Output No Longer Loses Your Class's Invariants:** In-process structured generation handed the provider a class rebuilt from the output structure's JSON schema instead of the class the caller passed — so a model with a custom `@field_validator` or `json_schema_extra` format hints reached the provider as a weaker contract than the one you wrote, because `datamodel-code-generator` cannot express those on round-trip. The output structure's own description (its class docstring) was dropped too, so the provider never saw it. The caller's live class now travels down to the leaf and is used as-is, on both the live and the dry path. The rebuild remains exactly as before for the case it exists for: a distributed worker holding only the serialized `ObjectAssignment`, where the class genuinely cannot travel. `ObjectAssignment`'s wire shape is unchanged.
+
+### Changed
+
+- **Dry-Run Object Mocks Are Built From Your Real Class:** a consequence of the structured-output fix above, called out separately because it can require action. The dry-run leaf mock now builds from your output class rather than a schema rebuild of it, so constraints the rebuild used to erase are enforced at mock-build time. A field whose type polyfactory cannot satisfy — an opaque custom type with a `__get_pydantic_core_schema__`, or a bare generic `ListContent` — can now fail `pipelex validate` with `DryRunMockBuildError` where it previously passed. Declare `examples` or `mock_format` on the offending field to fix it. The failure is loud and names the class and the remedy; it is not a silent behavior change. (Breaking)
+- **PipeFunc Name Collisions Are Now Loud:** Registering two different functions under the same PipeFunc name raises `FuncRegistryError` naming both origins, instead of logging at debug level and letting the last scan win. Registration names are unqualified and every scanned library directory feeds one process-wide registry, so a silent overwrite made which function a `.mthds` step actually ran depend on filesystem scan order — and could differ between two machines running the same code. Re-registering the *same* function object stays a no-op, so overlapping scan roots are unaffected. See [PipeFunc](building-methods/pipes/pipe-operators/PipeFunc.md). (Breaking)
+
## [v0.41.0] - 2026-07-30
### Highlights
diff --git a/TODOS.md b/TODOS.md
index f38e4172f..52b2bdc10 100644
--- a/TODOS.md
+++ b/TODOS.md
@@ -1,119 +1,157 @@
-# Determinism: prompt rendering must not mutate library-held pipe objects
+# Two places where the runtime silently substituted something for what the caller gave it
-**Branch:** `refactor/Determinism`
+Branch `refactor/Exec`, off `dev` at v0.41.0. Two independent fixes, batched because they share a shape and nothing else: in both, the runtime accepted a caller's object, quietly replaced it with something almost-equivalent, and never said so. One replaced a Pydantic class with a lossy rebuild of it; the other replaced a registered function with a later one of the same name.
-**Status: COMPLETE.** All three phases done. Every design claim was verified against the code before implementing (see [Verification log](#verification-log)). Gates green (`make agent-check`), full suite green (`make agent-test`). The MTHDS schema came out byte-identical and never mentioned `LLMPromptBlueprint` at all. No doc changes were needed. Deviations from the design are recorded at the bottom.
+This file is the review guide. It carries the reasoning, the measurements, and — importantly — the two places where doing the work **corrected the plan that motivated it**. Read those first if you are short on time: they are where a reviewer's attention is worth the most.
-## Problem
+- **Commit `ba9b0750c`** — `FuncRegistry.register_function` silently overwrote (item 2 below).
+- **Commit `bbf06b707`** — the schema→`exec()` round trip was paid in-process, and it was lossy (item 1 below).
-Rendering an LLM prompt writes a derived value back onto objects owned by the pipe library. `PipeLibrary.get_required_pipe` (`pipelex/libraries/pipe/pipe_library.py:111`) returns the stored instance — nothing copies it — so these write-backs mutate shared, long-lived state as a side effect of running a pipe. Two sites:
+Each commit is independently green on `make agent-check`, `make drift-check`, and the full `make agent-test`.
-1. `pipelex/pipe_operators/llm/pipe_llm.py:232` — inside `_live_run_operator_pipe`, the operator caches the config-derived prompting style onto its own spec: `self.llm_prompt_spec.templating_style = get_config().pipelex.prompting_config.get_prompting_style(...)`. (Thirty lines above, `resolve_dynamic_output_stuff_spec`'s docstring advertises "Pure: never mutates `self`" — that purity is local to that one method; the run method itself is not pure.)
-2. `pipelex/pipe_operators/llm/llm_prompt_blueprint.py:358-359` — `_unravel_text` writes the spec-level style into the caller's `TemplateBlueprint`: `jinja2_blueprint.templating_style = templating_style`. The written value is never even read on this path — the `render_template` call at line 374-379 passes `self.templating_style`, not the blueprint's field — so the write is pure pollution of a library-held object.
+---
-Executed proof (three lines: boot dry, build an `LLMPromptBlueprint` with a `TemplateBlueprint`, set `templating_style` on the spec, `await make_llm_prompt(...)`, read `template_blueprint.templating_style`):
+## Item 1 — the schema→`exec()` round trip was paid in-process, and it was lossy
+### What was wrong
+
+Structured generation handed a concrete Pydantic class to the content generator, which threw the class away, rebuilt an approximation of it from its JSON schema through code generation plus `exec()`, and gave the **rebuild** to the provider as the structured-output schema — while the original class was still sitting in the caller's frame.
+
+The path: `content_generator.make_object` / `make_object_list` received `object_class: type[BaseModelTypeVar]`; `ObjectAssignment.make_for_class` kept only `__name__` + `model_json_schema()` and dropped the class; `llm_generate.py` rebuilt it via `SchemaToModelFactory.make_from_json_schema` and passed the rebuild to `llm_worker.gen_object(schema=...)`; `dry_mock.py` rebuilt it on the dry path too. Then `content_generator` re-validated the result against the class it had never stopped holding.
+
+### What the rebuild drops — verified, not assumed
+
+- custom `@field_validator` / `@model_validator` logic
+- `json_schema_extra` format/pattern hints
+- **the output structure's own description** — its class docstring lands in `model_json_schema()["description"]`, and `datamodel-code-generator` does not re-emit it, so the rebuilt class has `__doc__ = None` and its schema has no top-level `description`. Field-level descriptions do survive.
+
+That last one the original plan did not name, and it is the most directly user-visible: every in-process structured call was sending the provider a schema stripped of the concept's own description — prompt-relevant signal, silently gone. Reproduce in three lines:
+
+```python
+class Line(StructuredContent):
+ """A billed line."""
+ label: str
+
+rebuilt = SchemaToModelFactory.make_from_json_schema(schema=Line.model_json_schema(), class_name="Line")
+assert rebuilt.__doc__ is None # and "description" is absent from rebuilt.model_json_schema()
```
-BEFORE spec.templating_style=None prompt_blueprint.templating_style=None
-AFTER spec.templating_style=<set> prompt_blueprint.templating_style=<set>
-SAME OBJECT as spec.prompt_blueprint: True
-```
-**Consequences:**
+Read from the user's side: someone who writes a model with a validator got a structured-output schema that had silently lost it, so the provider was constrained by a weaker contract than the one they wrote. The re-validation caught the bad result *afterwards*; it did not stop the weaker schema from being sent.
+
+### The constraint that shaped the fix
+
+`ObjectAssignment` is a serializable DTO **by design**. It is the payload a distributed orchestrator — our Temporal plugin, hooked in via `pipelex/plugins/orchestrator_registry.py` — sends to a worker that has no access to the caller's class object. Across that boundary the class genuinely cannot travel, and rebuilding from schema is the right answer. So the fix is not "remove the reconstruction". It is "stop paying for it when nothing crossed a boundary".
+
+### What changed
+
+- **`ObjectAssignment` is untouched.** No live-class field was added. It is a wire model; a `type[BaseModel]` field makes it unserializable and breaks the boundary path outright. This is the tempting wrong fix — the class is *right there* in `make_for_class` — and it was rejected deliberately, not overlooked.
+- **The class is threaded as an explicit keyword-only parameter beside the assignment**, on the in-process path only: `llm_gen_object(object_assignment, *, object_class=None)` and the same on `llm_gen_object_list` and the two dry leaves. `None` means "no class in hand, rebuild from the schema" — which is exactly what a worker entering from the boundary passes, so **the boundary path keeps today's behaviour by construction, with no flag to set**.
+- **One resolution helper serves both run modes** — `object_class_resolution.resolve_object_class`, a new leaf module so neither `llm_generate` nor `dry_mock` has to import the other. Live and dry both rebuilt before and both take the fast path now. Had only the live leaf been fixed, the dry mock would be built against a *weaker* class than the one the provider is constrained by — the mock would be more faithful than the thing it mocks, which is worse than the state we started from. `test_dry_leaf_resolves_the_class_the_same_way_as_the_live_leaf` asserts this directly rather than leaving it inferred from two separate passing tests.
+- Keyword-only matters mechanically here: every function on this path already carries a subject grant in `subject_grants.toml`, and a grant covers one positional subject. A second bare positional would be a violation with or without it. No grant was added; the grant for the deleted `dry_mock._reconstruct_object_class` was removed.
+
+### The measurement, and the correction it forces
+
+The plan asserted "a code generation plus an `exec()` per structured call". **That is wrong**, and the number says so: `SchemaToModelFactory.make_from_json_schema` memoizes on a sha256 of the schema, so the cost is per *distinct structure per process*, not per call. Measured on a representative `StructuredContent` (mixed primitives, an optional, a nested `$defs` list):
+
+| | before | after |
+|---|---|---|
+| first structure in a process (one-time `datamodel-code-generator` warmup + codegen + `exec()`) | ~81 ms | 0 |
+| each additional distinct structure | ~6–7 ms | 0 |
+| repeat call on an already-seen structure | ~0.01 ms (cache hit) | 0 |
+
+So the throughput win is real but small and front-loaded: ~81 ms + ~7 ms × (N−1) per process for a method with N distinct structured outputs, and effectively nothing per call after warm-up. **The fidelity fix is the reason to do this; the saved `exec()` perimeter and the codegen are the bonus.** Please weigh it that way — this is not a performance change.
+
+`schema_to_model_factory.py`'s module docstring now opens by saying when its perimeter is actually walked: only when no live class is in hand, which is also the only case where the schema is attacker-influenceable. Its Layer 2 restricted builtins are explicitly *not* a sandbox (`().__class__.__base__.__subclasses__()` stays reachable), so not walking it where nothing crossed a boundary is worth having on its own.
+
+### The three "verify rather than assume" items
+
+- **Kajson's `__kajson_class_source__`.** Nothing regresses. The attribute exists so kajson can rebuild the reconstructed class on the far side of a process boundary. In-process, the leaf's return value is immediately converted by `_revalidate_against_object_class` into an instance of the caller's real class and never serialized as the rebuild; in-tree the attribute is read only by `SchemaToModelFactory`'s own tests. The boundary path — the only one that ships the rebuilt class anywhere — is untouched.
+- **`_revalidate_against_object_class` on the fast path.** ⚠ **The plan got this one wrong, and both PR review bots caught it.** The plan said to keep the call and framed short-circuiting it as declining a *performance* saving. It is not a performance question: with the live class at the leaf, the provider (instructor's `response_model=`) already constructs **and validates** an instance of the caller's class, so re-validating the dump ran the caller's validators a **second** time, on data they had already normalized. A validator that transforms rather than rejects — `return f"INV-{value}"` — produced `INV-INV-…`; one that asserts its input is not yet normalized would reject valid provider output. Same defect on the list path and on the dry path. That is a regression this branch introduced, and it hit exactly the population the fix exists to serve. The fix is subtractive: `if isinstance(raw_obj, object_class): return raw_obj`. **`isinstance`, not `type(...) is`** — and not for defensiveness: instructor never returns the class you gave it, it returns `create_model(cls.__name__, __base__=(cls, OpenAISchema))`, a subclass. A `type(...) is` check would miss every real provider call and silently reinstate the double validation. `test_instructor_style_subclass_is_returned_without_revalidation` pins that. The call is still the boundary-path conversion (a schema-rebuilt class is never a subclass of `object_class`, so it falls through) and still the site that raises the dry fidelity error. The measured cost of the removed round trip is ~0.008 ms/object — nil, which is the point: the reason to short-circuit is semantic, not throughput.
+- **`DryRunObjectFidelityError` reachability.** It stops firing on the in-process object path and stays reachable on the boundary path and on the structured-search path, which always rebuilds. Not deleted; its tests were re-scoped rather than removed.
+
+### The behaviour change a reviewer should actually look at
+
+On the in-process path the mock is now built from the real class, so an invariant the schema round-trip used to drop is present at *build* time. The constrained fixture therefore fails earlier and with a different class: `DryRunMockBuildError` out of `build_mock_object`, instead of `DryRunObjectFidelityError` out of the re-validation. Both are `ErrorDomain.INPUT`, both are caller-facing, and both name the same `examples` / `mock_format` remedy — but it is a wire-visible `error_type` change on that path and it should be seen, not skimmed.
+
+`test_dry_run_object_fidelity.py` was rewritten around exactly this split: it asserts the fidelity error against the boundary composition (leaf with no class in hand → re-validate, the same comprehension shape `make_object_list` uses), and asserts the build error in-process.
+
+### Tests
+
+- **Identity, not equality** — `test_object_class_passthrough.py` asserts the provider receives *the same class object* (`schema is HintedName`), never a structurally-equal one. Structural equality is precisely what the round trip already preserved, so an equality assertion would have passed against the bug.
+- **A validator that transforms, asserted to run exactly once** (`NormalizedReference`, `INV-` prefix). The original test suite used only a *rejecting* validator, which is idempotent and therefore structurally blind to double execution — which is why the double-validation regression above got past it. Both new tests were confirmed red without the `isinstance` short-circuit and green with it.
+- A model carrying a custom validator and `json_schema_extra` hints reaches the provider with both intact, asserted observably (the received class rejects the invalid value; its schema still carries the hint).
+- A **control** test proves the loss is real: with no class in hand, the rebuilt class *accepts* data the author's class rejects.
+- The list path asserts the wrapper's item annotation is the caller's class, on both arms.
+- Both run modes take the same path, asserted directly.
+
+---
+
+## Item 2 — `FuncRegistry.register_function` silently overwrote
+
+### What was wrong
-- A pipe's `model_dump()` — used by `_make_pipe_data_for_registry` for the graph registry and by the crate payload that travels to a Temporal worker — differs before and after its first run in a process. Serialized output depends on run order.
-- The cached value can go stale: a deck reconfigured in-process, or an external-plugin model with no `InferenceModel`, means the first run's cached style silently governs every later run. First run and subsequent runs of the same instance are not equivalent.
-- Any harness that compares two executions in one process is measuring a mutated object on the second pass.
+On a key collision, `register_function` logged and then overwrote anyway. `log()` is `self._logger.debug(...)`, so under any normal configuration the collision produced no visible output at all — and the assignment ran on both branches, so the `if` distinguished only the wording.
-## Design
+Three facts made it reachable rather than theoretical: the registry is a flat, process-global `dict[str, Callable]`, so there is one key space for every source that ever registers; registration names are unqualified (the `@pipe_func` custom name if given, otherwise the bare `func.__name__` — no module, no package, no library prefix); and registration is driven by scanning the configured library directories.
-Compute the style into a local and pass it down — the value stays run-scoped, never written onto `self` or the blueprint.
+**The asymmetry gives it away.** `unregister_function` and `unregister_function_by_name` *raise* `FuncRegistryError` when a name is absent. Removing something that was never there was a hard error; replacing something that is there was a debug line.
-**Key facts established by code reading (verify cheaply before relying on them):**
+### The failure
-- `LLMPromptBlueprint.templating_style` has exactly one writer in the tree: the run-time cache at `pipe_llm.py:232`. `PipeLLMFactory.make` (`pipe_llm_factory.py:118-125`) never sets it; no test, doc, or schema references it. The field *is* the cache — remove it (no backward compat, per workspace policy).
-- `TemplateBlueprint.templating_style` stays: it is a declared field that the compose and img-gen paths legitimately read (`pipe_compose_blueprint.py:82`, `img_gen_prompt_blueprint.py:294`). On the LLM path the factory never sets it, so it is `None` for every LLM prompt blueprint today.
-- The only reader of the config-derived style is `render_template(templating_style=...)`; `get_prompting_style` (`system/configuration/configs.py:84`) is a cheap pure lookup — recomputing per run instead of caching is free and is the *correct* behavior (a deck/config change in-process takes effect on the next run instead of being shadowed by the first run's cache).
+Two library directories — or a library plus an installed package — each defining a `@pipe_func` named `summarize` produced a registry containing one of them, chosen by scan order. Every `PipeFunc` step naming `summarize` then ran the winner, including steps authored against the loser, with no warning at author time, boot time, or run time. Scan order comes from the filesystem and `pkgutil`, so which one wins could differ between two machines running the same code.
-**Changes:**
+### What changed
-1. `pipe_llm.py::_live_run_operator_pipe`: replace the cache-guard block (lines 226-234) with a local:
+A collision now raises `FuncRegistryError` naming the key, both origins, and the remedy (`@pipe_func(name=...)`). The registry already raised on the symmetric case, so this is consistency, not a new posture.
- ```python
- templating_style: TemplatingStyle | None = None
- if inference_model := model_deck.get_optional_inference_model(model_handle=llm_setting_main.model, model_type=ModelType.LLM):
- prompting_target = llm_setting_main.prompting_target or inference_model.prompting_target
- templating_style = get_config().pipelex.prompting_config.get_prompting_style(prompting_target=prompting_target)
- ```
+### The two open questions, answered by reading the code
- Pass `templating_style=templating_style` to **both** `make_llm_prompt` calls (text branch line ~252, object branch line ~297). Keep the existing TODO comment about external LLM plugins having no inference model.
-2. `llm_prompt_blueprint.py::make_llm_prompt`: add keyword-only param `templating_style: TemplatingStyle | None = None`; thread it to both `_unravel_text` calls (system prompt line ~230, user prompt line ~242).
-3. `llm_prompt_blueprint.py::_unravel_text`: add the same param; delete the write-back (lines 358-360); compute the effective style into a local and hand it to `render_template`:
+- **Is re-registering the *same* function object a collision?** No — it is a no-op. This is the shape the double-scan actually produces: `import_module_from_file` keys `sys.modules` by a name mangled from the file's **absolute path**, so a folder scanned twice returns the cached module and therefore the *same* function object. Two distinct objects under one key is the only shape that indicates a real clash. Pinned by `test_rescanning_the_same_folder_is_idempotent`.
+- **Does any first-party code deliberately overwrite?** No — `pipelex/` defines no `@pipe_func` of its own, so nothing registers a builtin expecting a user function to replace it. **No `replace=True` parameter was added**; with no caller for it, it would have been a speculative surface.
- ```python
- effective_style = jinja2_blueprint.templating_style or templating_style
- ```
+### ⚠ The one place this narrows a path that used to work
- Blueprint-declared style wins over the run-derived one — that is the evident intent of the old guard (`and not jinja2_blueprint.templating_style`), and it fixes the latent inconsistency where the blueprint's own style was respected by the write-back guard but then ignored by the render call. Behavior-identical today on the LLM path (the factory never sets the blueprint field), but now coherent.
-4. `llm_prompt_blueprint.py`: remove the `templating_style` field from `LLMPromptBlueprint` (line 30). It has no remaining reader or writer after 1-3.
-5. Default param stays `None` so the many existing `make_llm_prompt` call sites in `tests/integration/pipelex/pipes/llm_prompt_inputs/` keep working unchanged (they exercise image/document extraction, not styling).
+`LibraryManager.teardown(library_id=…)` does **not** clear `func_registry` — only `RuntimeBoot.teardown` does. So in a long-lived process that loads bundle A, tears it down, then loads bundle B, a registration name shared by both now collides with A's **stale** entry and B fails to load until the process restarts. Previously the overwrite was silent and B won, which for the sequential case was right.
-**Non-goals:**
+Tracing the hosts shows **no first-party process can reach this today**: a direct-mode `pipelex-api`'s library dirs are process-constant (per-request `ApiRunner()` passes `library_dirs=None`, resolving to boot-time defaults or `PIPELEXPATH`, and inline `.mthds` text registers no functions), the local MCP workshop is a TypeScript client that runs no pipelex in-process, and the hosted runner is immune (it takes the source-capture branch and never registers in-process). The exposed population is a third-party embedder varying `library_dirs` per call in one long-lived process — a public surface, so the narrowing is real, but the failure there is loud and typed, not silent. Two loads of the *same* bundle are fine — the module is cached by absolute path, so the same object re-registers and the idempotent no-op absorbs it.
-- `TemplateBlueprint` model and the compose / img-gen styling paths — untouched.
-- `WorkingMemory` mutation semantics (`set_new_main_stuff`) — separate concern, out of scope.
-- Other library-held-object mutation hazards, if any turn up — file them, don't fix them here.
+Not fixed here, and not because it is unimportant: there is no small correct fix. `func_registry` is a flat process-global dict with **no ownership dimension**, so per-library eviction has nothing to key on — note that each `Library` already owns its own `ClassRegistry`, and functions never got the same treatment. The remedy (ownership side table, id threaded through registration, eviction on teardown, rollback on a failed scan) is a design change that deserves its own review, not a late addition to this branch. Written up in `wip/refactoring/func-registry-entries-outlive-their-library.md`. **Ruled 2026-07-31: non-blocking.** With no first-party path reachable (see above) and the third-party failure mode loud and typed, this PR ships with the gap documented and the ownership design lands as its own follow-up PR.
-## Phase 0 — Red tests (TDD)
+### One reachability check worth recording
-- [x] New test module, e.g. `tests/unit/pipelex/pipe_operators/pipe_llm/test_prompt_rendering_purity.py`:
- - [x] **Blueprint-level purity:** build an `LLMPromptBlueprint` with a `prompt_blueprint` `TemplateBlueprint`, snapshot `spec.model_dump()`, `await make_llm_prompt(...)` with a style passed in (and a minimal context provider — reuse whatever the `llm_prompt_inputs` integration tests use), assert `template_blueprint.templating_style is None` afterward and `spec.model_dump()` is unchanged. This is the executed proof from the problem statement, inverted into a regression test.
- - [x] **Effective-style precedence:** a `TemplateBlueprint` with an explicit `templating_style` renders with its own style even when a different one is passed in; a blueprint without one renders with the passed style. Assert via the rendered output or by monkeypatching/spying `render_template` — whichever the existing test idiom supports (check `tests/unit/pipelex/tools/test_template_content_rendering.py` for the established pattern).
- - [x] **Pipe-level purity (site 1):** run a `PipeLLM` in dry mode, snapshot `pipe.model_dump()` before and after, assert identical. Dry run boots without inference; follow the boot pattern used by existing dry-run tests.
-- [x] Run the new module only (`.venv/bin/pytest -x -q <module>`), confirm the purity tests are **red** against the current code (the precedence test may need the new param to exist first — it can start as red-on-TypeError).
+The transported PipeFunc path (`DirectPipeFuncExecutor.run_pipe_func_transported`) materializes customer sources into a fresh `mkdtemp` workdir, which *would* yield a different function object under the same name on a second run in one process. It stays safe because that path is one-shot per process: `pipe_func_transported_entrypoint` runs exactly one request and then calls `Pipelex.teardown_if_needed()`, which clears the registry. Recorded here so a reviewer does not have to re-derive it.
-## Phase 1 — Implement
+### Tests
-- [x] Changes 1-4 above, in one pass (they are interlocking; the field removal is what makes the compiler/type-checker find any reader the grep missed).
-- [x] Re-grep `templating_style` across `pipelex/` to confirm the only remaining occurrences are `TemplateBlueprint`, the compose/img-gen paths, jinja2 plumbing, and config.
-- [x] New params are keyword-only (both methods already have `*`-sections — add the param after `*`).
-- [x] Run the Phase 0 module: all green.
+Two distinct functions under one name raise, and the message names both sources; the same function object registered twice is a no-op; an ineligible function still routes to `register_ineligible_function` and does not participate in collision detection; and end to end, two library directories defining the same function name fail the scan with the typed error instead of silently picking one.
-## Phase 2 — Gates, docs, changelog
+---
-- [x] `make agent-check` — includes the mthds-schema regen; inspect the `derived/mthds_schema.json` diff. Expected: no change (`LLMPromptBlueprint` is factory-built, not part of the parse surface). If it *does* change, stop and reassess — that would mean the field was on a documented surface after all.
-- [x] `git diff` review: no stray edits, no leftover guard at `pipe_llm.py:226`.
-- [x] Grep `docs/` for `templating_style` / prompting-style caching claims; update anything that describes the old cache-on-first-run behavior.
-- [x] `CHANGELOG.md` under `[Unreleased]`:
- - Fixed: rendering an LLM prompt no longer mutates the library-held pipe (`templating_style` write-backs removed); a pipe's serialized form is now identical before and after runs, and in-process config/deck changes take effect on the next run instead of being shadowed by a first-run cache.
- - Changed (breaking): `LLMPromptBlueprint` no longer has a `templating_style` field; `make_llm_prompt` takes the style as a parameter.
-- [x] `make agent-test` — full suite.
+## Deliberately not done
-## Checkpoint — wrap-up
+Each of these is a plausible-looking scope expansion that was declined on purpose.
-- [x] Update this file: mark phases done, record any deviations from the design (especially if the schema diff or the doc grep turned something up).
-- [x] Commit on `refactor/Determinism`. Do not push or open a PR without explicit go-ahead.
+- **Not replacing `exec()` with `pydantic.create_model()`.** That is the standing TODO at `schema_to_model_factory.py` and a larger change with its own compatibility surface (`__kajson_class_source__` exists because the generated source is shipped verbatim). Item 1 makes that TODO matter *less* by not walking the path in-process; it does not close it, and the docstring does not imply otherwise.
+- **Not namespacing the function registry by module.** Qualified names would remove the collision class entirely, but `function_name` is authored in `.mthds` files, so that is a language decision, not a registry fix.
+- **Not changing `ObjectAssignment`'s wire shape.** Anything that alters the payload an orchestrator plugin sends belongs in its own change, argued on its own terms.
+- **Not extending the fix to the in-process structured-*search* path.** `make_search_structured` holds `output_structure_class` and does not thread it down, so `PipeSearch` with a structured output still loses the docstring, validators and `json_schema_extra` hints to the rebuild. It is the same bug in the same shape, roughly four lines away — declined here rather than in scope creep, because it changes a *second* error-class surface the same way the object path's did (fidelity error → build error), which needs its own test re-scoping and its own review pass rather than riding in on a branch that is already bot-clean. Deferred to `wip/refactoring/structured-search-still-rebuilds-in-process.md`.
+- **Not auditing the *boundary* arm of `_revalidate_against_object_class`.** Its `model_dump(serialize_as_any=True)` → `model_validate` round trip is not identity-preserving in general (subclass erasure; dumped-by-field-name data that only revalidates thanks to `populate_by_name=True` and a compensating before-validator elsewhere). Pre-existing, unchanged here, must stay — it *is* the boundary conversion. Deferred to `wip/refactoring/boundary-revalidation-round-trip-is-unaudited.md`.
-## Verification log
+## Cross-repo
-Each "key fact" the design said to verify cheaply, and how it actually checked out:
+⚠ **An earlier revision of this file claimed "checked, nothing to change". That claim was false, and the reason is worth knowing: a workspace-rooted `grep -r … .` silently returned zero matches while iterating the sibling repo directories explicitly returned 71.** The sweep below was redone by enumerating `*/` one directory at a time. Never trust a single `.`-rooted recursive grep for a cross-repo sweep here.
-- **One writer, no stray readers.** A tree-wide grep for `templating_style` (py/toml/mthds/json/md) confirms `LLMPromptBlueprint.templating_style` had exactly one writer (`pipe_llm.py:232`) and two readers, both inside `_unravel_text`. Zero references in tests, docs, or the schema. Confirmed removable.
-- **The factory never sets it.** `pipe_llm_factory.py` contains no occurrence of `templating_style` at all; it builds `LLMPromptBlueprint` without the field and its `TemplateBlueprint`s without a style.
-- **Nothing on the LLM path can declare a blueprint-level style.** `PipeLLMBlueprint.prompt` and `system_prompt` are plain `str | None` — no nested template section, unlike PipeCompose's `template`. So the new precedence rule (`jinja2_blueprint.templating_style or templating_style`) is unreachable-by-declaration on the LLM path today, and the change is a coherence fix rather than a live behavior change. Worth re-checking if PipeLLM ever gains a rich prompt section.
-- **The dry path really does reach the mutating code.** `PipeLLM` does not override `_dry_run_operator_pipe`, and the `PipeOperator` default delegates to `_live_run_operator_pipe`. This is what makes the dry-run purity test a valid guard on site 1 rather than a vacuous pass.
-- **The mutation is observable with the default deck.** Probed before writing the test: the default text model resolves to `claude-4.6-sonnet`, whose `LLMSetting.prompting_target` is `None` but whose `InferenceModelSpec.prompting_target` is `anthropic`, deriving a real `xml/plain` style. Had both been `None` the write-back would have been `None`-over-`None` and the pipe-level test would have passed against the unfixed code. Confirmed red for the right reason before implementing (the failure diff named `llm_prompt_spec`).
-- **Schema untouched.** `derived/mthds_schema.json` is gitignored, so a snapshot/compare was used instead of `git diff`: byte-identical across the regen, and `LLMPromptBlueprint` appears in it **zero** times — it is factory-built and was never on the parse surface, exactly as the design predicted.
-- **Docs clean.** No page describes the cache-on-first-run behavior. The prompting-style page documents only `PromptingConfig` and `get_prompting_style` (both unchanged); `features/llm-integration.md` says provider-specific formatting is applied "automatically", which stays true and is now more accurate. No doc edit needed.
+**`cocode` breaks on upgrade — release-gated, needs a follow-up PR in that repo.** `cocode/cocode/pipelines/doc_proofread/file_utils.py` carries `@pipe_func()` *and* a redundant module-level `func_registry.register_function(read_file_content, name="read_file_content")`. `cocode/cocode/swe/swe_cmd.py:18` dotted-imports that module at CLI startup, registering object **A**; `PIPELINE_LIBRARY_DIRS` then makes the same directory a library dir, so the boot scan re-imports the file under a path-mangled `sys.modules` key, the module-level call fires again with object **B**, and the new collision check raises. Every `cocode` command would die at first `load_libraries`. Shielded only by cocode's `pipelex==0.41.0` pin. The fix belongs in cocode and is a deletion: the `@pipe_func()` decorator already covers the registration, so the explicit call is redundant belt-and-braces. `cocode/cocode/pipelines/text_utils.py:81` is the same shape but currently latent — it has no `@pipe_func`, so the scan's AST pre-check skips the file entirely.
-## Deviations from the design
+Checked and clean:
-- **Precedence asserted through rendered output, not a spy.** The design left the choice open. Observing the output is stronger and needs no mocking: the XML tag style wraps an `@variable` in `<name>…</name>` while NO_TAG inlines it bare, so the rendered text names the style that actually governed. The test is parametrized over three cases, including a negative one (passed NO_TAG really applies) so the two positive cases can't both pass by accident.
-- **`TemplatingStyle` in `pipe_llm.py` sits in a `TYPE_CHECKING` block.** It is used only in a local-variable annotation, so ruff's `TC001` requires it; the module had no `TYPE_CHECKING` block before and gained one (with `TYPE_CHECKING` added to the `typing` import). Local annotations aren't evaluated at runtime, so this is safe. `llm_prompt_blueprint.py` keeps its runtime import — there the name appears in method signatures on a `BaseModel`.
-- **The `log.verbose` line changed meaning.** It was `"Setting prompting style to …"`, emitted only when the cache was written; it is now `"Rendering with prompting style …"`, emitted on every render (including when the effective style is `None`). Intentional: with the cache gone there is no "setting" event to report, and the per-render line is the more useful trace.
-- **One tuple-vs-string lint nit.** `pytest.mark.parametrize` takes a tuple of names in this repo (ruff `PT006`), not the comma-separated string shown in `.claude/rules/pytest-standards.md`.
+- The private Temporal plugin's `act_llm_gen_object` / `act_llm_gen_object_list` call `llm_gen_object(object_assignment=...)` with no class, so they take the boundary path unchanged. Its `content_generator_in_workflow.py` comment describing the reconstruction stays accurate for that path.
+- `pipelex-transport` and `pipelex-mistralai-workflows` bridge conftests register module-level function objects with teardown — the idempotent case.
+- `pipelex-demo-mistral` / `pipelex-demo-vibe` register `flaky_echo` explicitly, and it carries no `@pipe_func` decorator, so the folder scan's AST pre-check never imports the file. One registration per process.
+- The only `llm_gen_object` mention in `docs/specs/` is the `UnitJobId` string value, which did not change.
-## Follow-ups (not done here, per the non-goals)
+## Docs touched
-- `WorkingMemory` mutation semantics (`set_new_main_stuff`) — untouched, separate concern.
-- No other library-held-object mutation hazard surfaced during the grep sweep. The `resolve_dynamic_output_stuff_spec` purity docstring noted in the problem statement is now accurate for the whole run method, not just that one helper.
+- `docs/building-methods/pipes/pipe-operators/PipeFunc.md` — new "Function names share one flat name space" section.
+- `docs/under-the-hood/dry-run-mock-generation.md` — the dry object leaf resolves its class the same way the live leaf does; which typed error you get on which path.
+- `docs/under-the-hood/distributed-content-generation.md` — reconstruction is the boundary answer and only that; the type bridge is kept in-process on purpose.
diff --git a/docs/building-methods/pipes/pipe-operators/PipeFunc.md b/docs/building-methods/pipes/pipe-operators/PipeFunc.md
index 18619c96c..e65ec7acb 100644
--- a/docs/building-methods/pipes/pipe-operators/PipeFunc.md
+++ b/docs/building-methods/pipes/pipe-operators/PipeFunc.md
@@ -87,6 +87,23 @@ async def concatenate_texts(working_memory: WorkingMemory) -> TextContent:
Then use `function_name = "custom_concat"` in your `.mthds` file.
+### Function names share one flat name space
+
+Registration names are unqualified — there is no module or package prefix — and every library directory Pipelex scans registers into the same process-wide registry. Two functions registered under the same name is therefore an error: Pipelex raises a `FuncRegistryError` naming both origins instead of letting scan order decide which one your `.mthds` files actually call.
+
+Registering the *same* function again — a directory scanned twice, a module reached by two paths — is a no-op, so overlapping scan roots are fine.
+
+If you hit a collision, give one of the two functions a distinct registration name:
+
+```python
+@pipe_func(name="invoices_summarize")
+async def summarize(working_memory: WorkingMemory) -> TextContent:
+ # Implementation...
+ pass
+```
+
+Prefixing names with the domain they serve is the simplest way to keep them distinct as a project grows.
+
## Configuration
Once the function is registered, you can use it in your `.mthds` file.
diff --git a/docs/under-the-hood/distributed-content-generation.md b/docs/under-the-hood/distributed-content-generation.md
index 5e3710dfa..b4ec5fb10 100644
--- a/docs/under-the-hood/distributed-content-generation.md
+++ b/docs/under-the-hood/distributed-content-generation.md
@@ -51,6 +51,8 @@ The schema is a plain `dict` — fully serializable, inspectable on the wire, an
### Schema-to-model reconstruction
+Reconstruction is the *boundary* answer, and only that. An in-process run never walks it: `ContentGenerator` passes the caller's live class down beside the assignment, and `resolve_object_class` (`pipelex/cogt/content_generation/object_class_resolution.py`) uses it as-is. That matters because the rebuild is lossy — custom validators, `json_schema_extra` hints, and the output structure's own description do not survive the round trip — so in-process the provider would otherwise be constrained by a weaker contract than the author wrote, for no reason. On a worker there is no choice: the class genuinely cannot travel, so the rebuild below is the right answer there.
+
On the worker, `SchemaToModelFactory.make_from_json_schema(schema, *, class_name)` (`pipelex/cogt/content_generation/schema_to_model_factory.py`) rebuilds a live Pydantic class from the embedded schema:
```python
@@ -103,11 +105,11 @@ Re-attaching after `kajson.loads()` ensures the source survives if the object cr
The class reconstructed on the worker is a structural match to the original, but a different Python class in memory — so `isinstance()` checks would fail. The caller bridges the gap with a `model_validate` round-trip (`pipelex/cogt/content_generation/content_generator.py`):
```python
-raw_obj = await llm_gen_object(object_assignment=object_assignment)
+raw_obj = await llm_gen_object(object_assignment=object_assignment, object_class=object_class)
return object_class.model_validate(raw_obj.model_dump(serialize_as_any=True))
```
-`model_dump(serialize_as_any=True)` produces a plain dict from the reconstructed object, and `model_validate()` rebuilds it as a proper instance of the original class, restoring type safety for downstream code.
+`model_dump(serialize_as_any=True)` produces a plain dict from the reconstructed object, and `model_validate()` rebuilds it as a proper instance of the original class, restoring type safety for downstream code. The bridge is kept on the in-process path too, where it becomes a same-type conversion: it is still the site that raises the typed dry-run fidelity error, and it is what makes the two paths return the same thing.
### Structured web search reuses the same mechanism
diff --git a/docs/under-the-hood/dry-run-mock-generation.md b/docs/under-the-hood/dry-run-mock-generation.md
index 23829a997..33af6a396 100644
--- a/docs/under-the-hood/dry-run-mock-generation.md
+++ b/docs/under-the-hood/dry-run-mock-generation.md
@@ -51,7 +51,7 @@ A dry run does not swap in a special content generator: `run_mode=DRY` rides `Co
| Trigger | Entry Point | Mock Generation |
|---------|-------------|-----------------|
| `pipelex validate` | `BundleValidator.validate_pipes()` | `WorkingMemoryFactory.make_mock_inputs()` |
-| `PipeLLM` output in dry mode | `dry_llm_gen_object()` / `dry_llm_gen_object_list()` (cogt leaf, `dry_mock.py`) | `DryRunFactory` via the schema-reconstructed class |
+| `PipeLLM` output in dry mode | `dry_llm_gen_object()` / `dry_llm_gen_object_list()` (cogt leaf, `dry_mock.py`) | `DryRunFactory` via the caller's class in-process, via the schema-reconstructed class on a worker |
| `PipeFunc` output in dry mode | `WorkingMemoryFactory.make_mock_content()` | `DryRunFactory` (with explicit field constraints) |
| `PipeCompose` in dry mode | `StructuredContentComposer.compose()` | Uses resolved values from working memory; falls back to a `DryRunFactory` mock of the output class only when a value error hits under mock inputs |
@@ -221,14 +221,16 @@ The `factory_use_construct=True` flag bypasses `field_validator` and `model_vali
### LLM Output Generation
-When the dry LLM leaf (`dry_llm_gen_object` in `dry_mock.py`) generates mock outputs, it reconstructs the output class from the JSON schema carried on the assignment (the same class on any backend), then builds via `build_mock_object()`:
+When the dry LLM leaf (`dry_llm_gen_object` in `dry_mock.py`) generates mock outputs, it resolves the output class exactly the way the live leaf does — via `resolve_object_class` — then builds via `build_mock_object()`:
```python
-item_class = SchemaToModelFactory.make_from_json_schema(schema=..., class_name=...)
+item_class = resolve_object_class(object_assignment=..., object_class=...)
return build_mock_object(item_class)
```
-No explicit field constraints are passed — the factory auto-detects `MockFormat` from field definitions. Note that `json_schema_extra` hints can be dropped by the schema round-trip: classes with exotic format constraints should declare `examples` / `mock_format`, otherwise re-validation against the original class raises a typed `DryRunObjectFidelityError`.
+`object_class` is the caller's live class when there is one (the in-process case), and `None` when only the serialized assignment is in hand (the distributed-worker case), in which case the class is rebuilt from the assignment's JSON schema. Keeping the two run modes on one resolution matters: if only the live leaf used the real class, the mock would be built against a *weaker* class than the one the provider is constrained by — the mock would be less faithful than the thing it mocks.
+
+No explicit field constraints are passed — the factory auto-detects `MockFormat` from field definitions. On the rebuilt-class path, `json_schema_extra` hints and custom validators are dropped by the schema round-trip: classes with exotic format constraints should declare `examples` / `mock_format`, otherwise re-validation against the original class raises a typed `DryRunObjectFidelityError`. On the live-class path those invariants are present at build time instead, so an unsatisfiable one surfaces as a typed `DryRunMockBuildError` naming the same remedy.
### PipeCompose Resolution
diff --git a/pipelex/cogt/content_generation/content_generator.py b/pipelex/cogt/content_generation/content_generator.py
index 428fd5495..8f0f57f7c 100644
--- a/pipelex/cogt/content_generation/content_generator.py
+++ b/pipelex/cogt/content_generation/content_generator.py
@@ -47,20 +47,35 @@ def _revalidate_against_object_class(
object_class: type[BaseModelTypeVar],
is_mock_built: bool,
) -> BaseModelTypeVar:
- """Re-validate a leaf-generated object's data against the original ``object_class``.
+ """Convert a leaf-generated object into ``object_class``, unless it already is one.
- ``llm_gen_object`` / ``llm_gen_object_list`` return plain ``BaseModel``s reconstructed from the JSON
- schema; re-validating their data against the original class makes the result the proper subtype (e.g.
+ On the in-process path the leaf worked from ``object_class``, so the object is already one — note
+ ``isinstance``, not ``type(...) is``: instructor does not hand back the class you gave it, it hands
+ back ``create_model(cls.__name__, __base__=(cls, OpenAISchema))``, a *subclass*. Either way it needs
+ no conversion — and re-validating it would run the caller's validators a **second** time, on data
+ they already normalized. A validator that transforms rather than merely rejects (``f"INV-{value}"``, a
+ list that gets a default appended) would corrupt its own output, and one that asserts its input is
+ not yet normalized would reject valid provider output. Returning it untouched is what makes the
+ caller's validator constrain the provider exactly once, which is the whole point of handing the live
+ class down.
+
+ The conversion is still needed at the *boundary*: a leaf that ran on a worker holding only the
+ serialized assignment returns a plain ``BaseModel`` reconstructed from the JSON schema — never a
+ subclass of ``object_class`` — so re-validating its data makes the result the proper subtype (e.g.
``StructuredContent``) the caller expects.
- Under the dry-run leaf mock (``run_mode=DRY``) the object was built by polyfactory from
- the schema-reconstructed class, which can drop invariants the original class enforces (custom
- validators, ``json_schema_extra`` format/pattern hints datamodel-code-generator omits on round-trip).
- A re-validation failure there is the known object-mock fidelity gap, so the ``ValidationError`` is
+ Under the dry-run leaf mock (``run_mode=DRY``) that reconstruction is also what polyfactory built the
+ mock from, and it can drop invariants the original class enforces (custom validators,
+ ``json_schema_extra`` format/pattern hints datamodel-code-generator omits on round-trip). A
+ re-validation failure there is the known object-mock fidelity gap, so the ``ValidationError`` is
re-raised as a clear typed :class:`DryRunObjectFidelityError` naming the class and the
- ``examples`` / ``mock_format`` remedy. The catch is scoped to the dry path only — a LIVE provider's
- invalid output keeps its existing ``ValidationError``.
+ ``examples`` / ``mock_format`` remedy. That gap cannot occur in-process — the mock is built from the
+ real class, so an unsatisfiable invariant fails earlier and louder, as ``DryRunMockBuildError`` out of
+ ``build_mock_object``, and never reaches this function. The catch is scoped to the dry path only — a
+ LIVE provider's invalid output keeps its existing ``ValidationError``.
"""
+ if isinstance(raw_obj, object_class):
+ return raw_obj
raw_data = raw_obj.model_dump(serialize_as_any=True)
if not is_mock_built:
return object_class.model_validate(raw_data)
@@ -129,7 +144,7 @@ class ContentGenerator(ContentGeneratorProtocol):
object_class=object_class,
llm_assignment=llm_assignment_for_object,
)
- raw_obj = await llm_gen_object(object_assignment=object_assignment)
+ raw_obj = await llm_gen_object(object_assignment=object_assignment, object_class=object_class)
log.verbose(f"{self.__class__.__name__} generated object direct: {raw_obj}")
return _revalidate_against_object_class(raw_obj, object_class=object_class, is_mock_built=cogt_run_params.run_mode.is_dry)
@@ -156,7 +171,7 @@ class ContentGenerator(ContentGeneratorProtocol):
llm_assignment=llm_assignment_for_object,
nb_items=nb_items,
)
- raw_list = await llm_gen_object_list(object_assignment=object_assignment)
+ raw_list = await llm_gen_object_list(object_assignment=object_assignment, object_class=object_class)
log.verbose(f"{self.__class__.__name__} generated object list direct: {raw_list}")
return [
_revalidate_against_object_class(raw_obj, object_class=object_class, is_mock_built=cogt_run_params.run_mode.is_dry)
diff --git a/pipelex/cogt/content_generation/dry_mock.py b/pipelex/cogt/content_generation/dry_mock.py
index a5e2a0668..2a44d85e7 100644
--- a/pipelex/cogt/content_generation/dry_mock.py
+++ b/pipelex/cogt/content_generation/dry_mock.py
@@ -26,9 +26,11 @@ trigger, which requires ``--dry-run`` and is not shown in --help):
to exist. Non-LLM leaves keep their no-usage dry behavior either way.
This module is the single home for "what a mocked inference produces". Object
-mocks are built from the **schema-reconstructed** class on both backends (one
-code path, identical mock everywhere); exotic format constraints must declare
-``examples`` / ``mock_format`` — see ``DryRunObjectFidelityError``.
+mocks resolve their class exactly the way the live leaf does (see
+``object_class_resolution``): from the caller's real class when one is in hand,
+from the JSON schema otherwise — so the mock is never built against a *weaker*
+class than the one the provider is constrained by. Exotic format constraints
+must declare ``examples`` / ``mock_format`` — see ``DryRunObjectFidelityError``.
"""
from collections.abc import Sequence
@@ -52,6 +54,7 @@ from pipelex.cogt.content_generation.assignment_models import (
from pipelex.cogt.content_generation.cogt_run_params import CogtRunParams
from pipelex.cogt.content_generation.dry_run_factory import DryRunFactory
from pipelex.cogt.content_generation.exceptions import DryRunMockBuildError
+from pipelex.cogt.content_generation.object_class_resolution import resolve_object_class
from pipelex.cogt.content_generation.schema_to_model_factory import SchemaToModelFactory
from pipelex.cogt.llm.llm_job import LLMJob
from pipelex.cogt.llm.llm_job_components import LLMJobConfig, LLMJobReport
@@ -196,20 +199,6 @@ def build_mock_objects(model_class: type[BaseModelTypeVar], *, count: int) -> li
raise DryRunMockBuildError.for_object_class(model_class.__name__) from exc
-def _reconstruct_object_class(object_assignment: ObjectAssignment) -> type[BaseModel]:
- """Reconstruct the object's model class from its JSON schema.
-
- The leaf carries only the JSON schema (not the original class), so the class is rebuilt via
- :class:`SchemaToModelFactory`. This is the single schema-based mock site: both backends build
- the same mock, and fidelity bugs surface in cheap local unit tests instead of only on a worker
- (pre-flight decision 2).
- """
- return SchemaToModelFactory.make_from_json_schema(
- schema=object_assignment.object_class_schema,
- class_name=object_assignment.object_class_name,
- )
-
-
def stamp_mock_main_coordination(items: Sequence[Any]) -> None:
"""Set the first item's ``pipe_code`` to ``"mock_main"`` — the single home of this coordination (D3).
@@ -220,9 +209,17 @@ def stamp_mock_main_coordination(items: Sequence[Any]) -> None:
main-pipe check. Callers: the mock-input factory (``working_memory_factory``), the batch
controller's dry aggregation (``pipe_batch``), and the dry object-list leaf mock
(:func:`dry_llm_gen_object_list`). The stamp is a no-op for items without a ``pipe_code`` field.
+
+ The item is now the *caller's own* class, not a throwaway schema rebuild, so the assignment can hit
+ a model config the caller chose — ``frozen=True`` or ``validate_assignment`` — and raise. Surface
+ that as the same typed :class:`DryRunMockBuildError` the surrounding mock build uses, rather than
+ letting a raw ``ValidationError`` escape from a mutation the caller never asked for.
"""
if items and hasattr(items[0], "pipe_code"):
- items[0].pipe_code = MOCK_MAIN_PIPE_CODE
+ try:
+ items[0].pipe_code = MOCK_MAIN_PIPE_CODE
+ except ValidationError as exc:
+ raise DryRunMockBuildError.for_object_class(type(items[0]).__name__) from exc
def _nb_list_items(object_assignment: ObjectAssignment) -> int:
@@ -236,21 +233,22 @@ class _ReportLLMJobFunc(Protocol):
def __call__(self, job_metadata: JobMetadata, *, llm_setting: LLMSetting, llm_prompt: LLMPrompt) -> None: ...
-def _leaf_gen_object(object_assignment: ObjectAssignment, *, report_func: _ReportLLMJobFunc) -> BaseModel:
- """Shared object-mock pipeline: reconstruct the class from its schema, report once, build one mock.
+def _leaf_gen_object(object_assignment: ObjectAssignment, *, report_func: _ReportLLMJobFunc, object_class: type[BaseModel] | None) -> BaseModel:
+ """Shared object-mock pipeline: resolve the class to mock, report once, build one mock.
- Built from the schema-reconstructed class (the leaf carries only the JSON schema, not the original
- class), so format hints encoded via ``json_schema_extra`` that datamodel-code-generator drops on
- round-trip are not honored — exotic-format schemas may yield mock data the original class would
- reject. The generator re-validates against the original class and re-raises that failure as
- ``DryRunObjectFidelityError`` (review F2); declare ``examples`` / ``mock_format`` on the
- constrained fields to fix it.
+ With the caller's class in hand the mock is built from it directly, so every invariant it enforces
+ is honored. Without it (the boundary case) the class is rebuilt from the JSON schema, which drops
+ custom validators and the format hints encoded via ``json_schema_extra`` that
+ datamodel-code-generator omits on round-trip — so exotic-format schemas may yield mock data the
+ original class would reject. The generator re-validates against the original class and re-raises
+ that failure as ``DryRunObjectFidelityError`` (review F2); declare ``examples`` / ``mock_format``
+ on the constrained fields to fix it.
Reports exactly once — the live leaf makes one ``gen_object`` call (a list is one call against a
list-wrapper schema), so a single ``UsageReportEvent`` matches the real one-call topology that
cross-worker assertions count.
"""
- item_class = _reconstruct_object_class(object_assignment)
+ item_class = resolve_object_class(object_assignment=object_assignment, object_class=object_class)
mock_object = build_mock_object(item_class)
# Report only after a successful build — a failed mock must not leave a usage event behind.
llm_assignment = object_assignment.llm_assignment_for_object
@@ -258,9 +256,11 @@ def _leaf_gen_object(object_assignment: ObjectAssignment, *, report_func: _Repor
return mock_object
-def _leaf_gen_object_list(object_assignment: ObjectAssignment, *, report_func: _ReportLLMJobFunc) -> list[BaseModel]:
+def _leaf_gen_object_list(
+ object_assignment: ObjectAssignment, *, report_func: _ReportLLMJobFunc, object_class: type[BaseModel] | None
+) -> list[BaseModel]:
"""List counterpart of :func:`_leaf_gen_object`: one report, ``nb_items`` builds (D11)."""
- item_class = _reconstruct_object_class(object_assignment)
+ item_class = resolve_object_class(object_assignment=object_assignment, object_class=object_class)
mock_objects = build_mock_objects(item_class, count=_nb_list_items(object_assignment))
# Report only after a successful build — a failed mock must not leave a usage event behind.
llm_assignment = object_assignment.llm_assignment_for_object
@@ -299,14 +299,18 @@ def dry_llm_gen_text(llm_assignment: LLMAssignment) -> str:
return f"DRY RUN: llm_gen_text • llm_setting={llm_assignment.llm_setting.desc()} • prompt={prompt_truncated}"
-def dry_llm_gen_object(object_assignment: ObjectAssignment) -> BaseModel:
- """Dry leaf for ``llm_gen_object``: schema-built mock instance + synthetic job report."""
+def dry_llm_gen_object(object_assignment: ObjectAssignment, *, object_class: type[BaseModel] | None = None) -> BaseModel:
+ """Dry leaf for ``llm_gen_object``: mock instance + synthetic job report.
+
+ Mirrors the live leaf's class resolution so the two run modes cannot drift: with the caller's class
+ in hand the mock is built from it, without it the class is rebuilt from the schema.
+ """
log.verbose(f"🤡 DRY RUN: llm_gen_object for '{object_assignment.object_class_name}'")
- return _leaf_gen_object(object_assignment, report_func=_dry_report_func(object_assignment.cogt_run_params))
+ return _leaf_gen_object(object_assignment, report_func=_dry_report_func(object_assignment.cogt_run_params), object_class=object_class)
-def dry_llm_gen_object_list(object_assignment: ObjectAssignment) -> list[BaseModel]:
- """Dry leaf for ``llm_gen_object_list``: ``nb_items`` schema-built mocks + one synthetic report.
+def dry_llm_gen_object_list(object_assignment: ObjectAssignment, *, object_class: type[BaseModel] | None = None) -> list[BaseModel]:
+ """Dry leaf for ``llm_gen_object_list``: ``nb_items`` mocks + one synthetic report.
Applies :func:`stamp_mock_main_coordination` so bundle dry-validation's mocked
``BundleHeaderSpec.main_pipe`` check passes through the leaf mock (D3). The stamp is
@@ -314,7 +318,7 @@ def dry_llm_gen_object_list(object_assignment: ObjectAssignment) -> list[BaseMod
harmless elsewhere.
"""
log.verbose(f"🤡 DRY RUN: llm_gen_object_list for '{object_assignment.object_class_name}'")
- items = _leaf_gen_object_list(object_assignment, report_func=_dry_report_func(object_assignment.cogt_run_params))
+ items = _leaf_gen_object_list(object_assignment, report_func=_dry_report_func(object_assignment.cogt_run_params), object_class=object_class)
stamp_mock_main_coordination(items)
return items
diff --git a/pipelex/cogt/content_generation/exceptions.py b/pipelex/cogt/content_generation/exceptions.py
index 8012cb2d2..eccaf65e7 100644
--- a/pipelex/cogt/content_generation/exceptions.py
+++ b/pipelex/cogt/content_generation/exceptions.py
@@ -45,7 +45,11 @@ class DryRunObjectFidelityError(PipelexError):
class DryRunMockBuildError(PipelexError):
"""Raised when building a leaf mock object fails (polyfactory build or model validation).
- The leaf mock builds objects via polyfactory from the schema-reconstructed class. A build failure
+ The leaf mock builds objects via polyfactory from whichever class the leaf resolved: the caller's
+ real class in-process, the schema-reconstructed one on a worker. In-process is the common trigger,
+ because the real class carries every invariant the schema round-trip would have dropped — so a
+ constraint polyfactory cannot satisfy fails here rather than surviving into
+ :class:`DryRunObjectFidelityError`. A build failure
here is deterministic — retrying it (e.g. Temporal activity retries) can never succeed — so it is
surfaced as a typed ``PipelexError``, which the activity error boundary converts to a terminal
``ApplicationError``. The message names the class and the remedy: declare ``examples`` or
diff --git a/pipelex/cogt/content_generation/llm_generate.py b/pipelex/cogt/content_generation/llm_generate.py
index 6619df46a..060e03326 100644
--- a/pipelex/cogt/content_generation/llm_generate.py
+++ b/pipelex/cogt/content_generation/llm_generate.py
@@ -9,7 +9,7 @@ from pipelex.cogt.content_generation.dry_mock import (
dry_llm_gen_object_list,
dry_llm_gen_text,
)
-from pipelex.cogt.content_generation.schema_to_model_factory import SchemaToModelFactory
+from pipelex.cogt.content_generation.object_class_resolution import resolve_object_class
from pipelex.cogt.llm.llm_job_factory import LLMJobFactory
from pipelex.cogt.llm.llm_utils import dump_prompt, dump_response_from_text_gen
from pipelex.runtime_hub import get_llm_worker
@@ -31,20 +31,22 @@ async def llm_gen_text(llm_assignment: LLMAssignment) -> str:
return generated_text
-async def llm_gen_object(object_assignment: ObjectAssignment) -> BaseModel:
+async def llm_gen_object(object_assignment: ObjectAssignment, *, object_class: type[BaseModel] | None = None) -> BaseModel:
+ """Generate one structured object. ``object_class`` is the caller's live class when there is one.
+
+ Omit it (the boundary case: a worker that received only the serialized assignment) to rebuild the
+ class from the assignment's JSON schema instead — see :mod:`.object_class_resolution`.
+ """
llm_assignment = object_assignment.llm_assignment_for_object
if object_assignment.cogt_run_params.run_mode.is_dry:
- return dry_llm_gen_object(object_assignment)
+ return dry_llm_gen_object(object_assignment, object_class=object_class)
llm_worker = get_llm_worker(llm_handle=llm_assignment.llm_handle)
llm_job = LLMJobFactory.make_llm_job(
job_metadata=llm_assignment.job_metadata,
llm_prompt=llm_assignment.llm_prompt,
llm_job_params=llm_assignment.llm_job_params,
)
- content_class = SchemaToModelFactory.make_from_json_schema(
- schema=object_assignment.object_class_schema,
- class_name=object_assignment.object_class_name,
- )
+ content_class = resolve_object_class(object_assignment=object_assignment, object_class=object_class)
dump_prompt(llm_prompt=llm_job.llm_prompt)
generated_object: BaseModel = await llm_worker.gen_object(
llm_job=llm_job,
@@ -54,10 +56,15 @@ async def llm_gen_object(object_assignment: ObjectAssignment) -> BaseModel:
return generated_object
-async def llm_gen_object_list(object_assignment: ObjectAssignment) -> list[BaseModel]:
+async def llm_gen_object_list(object_assignment: ObjectAssignment, *, object_class: type[BaseModel] | None = None) -> list[BaseModel]:
+ """Generate a list of structured objects. ``object_class`` is the caller's live item class when there is one.
+
+ Omit it (the boundary case: a worker that received only the serialized assignment) to rebuild the
+ item class from the assignment's JSON schema instead — see :mod:`.object_class_resolution`.
+ """
llm_assignment = object_assignment.llm_assignment_for_object
if object_assignment.cogt_run_params.run_mode.is_dry:
- return dry_llm_gen_object_list(object_assignment)
+ return dry_llm_gen_object_list(object_assignment, object_class=object_class)
log.verbose(f"llm_gen_object_list to generate a list of '{object_assignment.object_class_name}'")
llm_worker = get_llm_worker(llm_handle=llm_assignment.llm_handle)
llm_job = LLMJobFactory.make_llm_job(
@@ -66,10 +73,7 @@ async def llm_gen_object_list(object_assignment: ObjectAssignment) -> list[BaseM
llm_job_params=llm_assignment.llm_job_params,
)
item_class_name = object_assignment.object_class_name
- item_class = SchemaToModelFactory.make_from_json_schema(
- schema=object_assignment.object_class_schema,
- class_name=item_class_name,
- )
+ item_class = resolve_object_class(object_assignment=object_assignment, object_class=object_class)
class ListSchema(BaseModel):
items: list[item_class] # type: ignore[valid-type] # pyright: ignore[reportInvalidTypeForm]
diff --git a/pipelex/cogt/content_generation/object_class_resolution.py b/pipelex/cogt/content_generation/object_class_resolution.py
new file mode 100644
index 000000000..904a10cd6
--- /dev/null
+++ b/pipelex/cogt/content_generation/object_class_resolution.py
@@ -0,0 +1,29 @@
+"""Resolve which Pydantic class a structured-generation leaf hands to the provider.
+
+``ObjectAssignment`` is a wire model: it carries the output structure's JSON *schema*, never the live
+class, because a distributed orchestrator ships it to a worker process that cannot receive a ``type``.
+A worker entering from that boundary has to rebuild an approximation of the class from the schema —
+which is lossy (``datamodel-code-generator`` cannot express a custom validator, and drops
+``json_schema_extra`` format/pattern hints) and costs a code generation plus an ``exec()`` on the first
+call for each distinct schema.
+
+In-process there is no boundary: the caller's real class is still on the stack, so it travels down
+beside the assignment and is used as-is. ``None`` means "no class in hand", which is exactly what a
+worker entering from the boundary passes — so the distributed path keeps today's behaviour by
+construction, with no flag to set.
+"""
+
+from pydantic import BaseModel
+
+from pipelex.cogt.content_generation.assignment_models import ObjectAssignment
+from pipelex.cogt.content_generation.schema_to_model_factory import SchemaToModelFactory
+
+
+def resolve_object_class(*, object_assignment: ObjectAssignment, object_class: type[BaseModel] | None) -> type[BaseModel]:
+ """Return the caller's class when it is in hand, otherwise rebuild it from the assignment's schema."""
+ if object_class is not None:
+ return object_class
+ return SchemaToModelFactory.make_from_json_schema(
+ schema=object_assignment.object_class_schema,
+ class_name=object_assignment.object_class_name,
+ )
diff --git a/pipelex/cogt/content_generation/schema_to_model_factory.py b/pipelex/cogt/content_generation/schema_to_model_factory.py
index 9b920f849..9ac512ec4 100644
--- a/pipelex/cogt/content_generation/schema_to_model_factory.py
+++ b/pipelex/cogt/content_generation/schema_to_model_factory.py
@@ -5,6 +5,13 @@ BaseModel class that can be used as a structured output schema for LLM generatio
The generated class carries its own source code as __kajson_class_source__, enabling
kajson to deserialize it across process boundaries without a class registry.
+When this perimeter is actually walked: only when the caller has no live class in hand. Structured
+generation in-process passes its real class down beside the assignment (see
+`object_class_resolution.resolve_object_class`) and never reaches this module — the class never left
+the interpreter, so there is nothing to rebuild. The reconstruction is for the boundary case: a worker
+that received a serialized `ObjectAssignment` (or `SearchObjectAssignment`) carrying only the JSON
+schema, which is exactly the case where the schema is attacker-influenceable.
+
Security model: this module exec()'s code generated from an attacker-influenceable JSON
schema when schemas cross a process boundary (e.g. Temporal payloads). Two layers
narrow the attack surface, but Layer 2 is defense-in-depth, NOT a sandbox:
diff --git a/pipelex/pipeline/validate_bundle.py b/pipelex/pipeline/validate_bundle.py
index 2b106dfc2..5ac8445d2 100644
--- a/pipelex/pipeline/validate_bundle.py
+++ b/pipelex/pipeline/validate_bundle.py
@@ -37,6 +37,7 @@ from pipelex.pipe_machinery.pipe_abstract import PipeAbstract
from pipelex.pipe_run.exceptions import DryRunError
from pipelex.pipeline.bundle_validator import BundleValidator, DryRunOutput, DryRunStatus
from pipelex.pipeline.exceptions import ValidateBundleError
+from pipelex.system.registries.exceptions import FuncRegistryError
class ValidateBundleResult(BaseModel):
@@ -184,6 +185,13 @@ def translate_to_validate_bundle_error() -> Generator[None, None, None]:
pipelex_bundle_blueprint_validation_errors=blueprint_validation_errors,
pipe_validation_errors=pipe_concept_validation_errors,
) from library_error
+ except FuncRegistryError as func_registry_error:
+ # A duplicate @pipe_func name across the scanned library dirs. Raised while loading the
+ # library, so it lands here rather than on the PipeFunc field validator's ValueError path.
+ # It is caller-fixable input (rename one with @pipe_func(name=...)), so it must produce a
+ # verdict — `is_valid: false` — not the "no verdict could be produced" exit 2 / 5xx an
+ # untranslated error would give.
+ raise ValidateBundleError(message=func_registry_error.message) from func_registry_error
except PipeRunError as pipe_run_error:
raise ValidateBundleError(
message=pipe_run_error.message,
diff --git a/pipelex/system/registries/func_registry.py b/pipelex/system/registries/func_registry.py
index 956f2fd9c..5fd26c974 100644
--- a/pipelex/system/registries/func_registry.py
+++ b/pipelex/system/registries/func_registry.py
@@ -19,6 +19,17 @@ FuncRegistryDict = dict[str, Callable[..., Any]]
PIPE_FUNC_MARKER = "_is_pipe_func"
+def _describe_function_origin(*, func: Callable[..., Any]) -> str:
+ """Renders where a registered function came from, for collision diagnostics.
+
+ Modules imported by file path get a name mangled from their absolute path, so the module name
+ alone is enough to point the author at the offending file.
+ """
+ module_name = getattr(func, "__module__", None) or "<unknown module>"
+ qualified_name = getattr(func, "__qualname__", None) or getattr(func, "__name__", "<unknown function>")
+ return f"'{qualified_name}' from module '{module_name}'"
+
+
def pipe_func(name: str | None = None) -> Callable[[T], T]:
"""Decorator to mark a function for automatic registration in the func_registry.
@@ -102,15 +113,33 @@ class FuncRegistry(RootModel[FuncRegistryDict]):
*,
name: str | None = None,
) -> None:
- """Registers a function in the registry with a name if it meets eligibility criteria."""
+ """Registers a function in the registry with a name if it meets eligibility criteria.
+
+ Raises:
+ FuncRegistryError: If a *different* function is already registered under the same name.
+ Registering the same function object again is a no-op, so a folder scanned twice
+ (the same module object is reused from ``sys.modules``) stays idempotent.
+
+ """
if not self.is_eligible_function(func):
return
key = name or func.__name__
- if key in self.root:
- self.log(f"Function '{key}' already exists in registry")
- else:
- self.log(f"Registered new single function '{key}' in registry")
+ already_registered = self.root.get(key)
+ if already_registered is not None:
+ if already_registered is func:
+ self.log(f"Function '{key}' is already registered in registry with the same function object")
+ return
+ msg = (
+ f"Function name '{key}' is already registered by a different function: "
+ f"{_describe_function_origin(func=already_registered)} would be replaced by {_describe_function_origin(func=func)}. "
+ f"PipeFunc names share a single flat, process-wide name space, so which one wins would depend on "
+ f'scan order. Give one of them a distinct name with @pipe_func(name="..."). '
+ f"See: {URLs.pipe_func_docs}"
+ )
+ raise FuncRegistryError(msg)
+
+ self.log(f"Registered new single function '{key}' in registry")
self.root[key] = func
def unregister_function(self, func: Callable[..., Any]) -> None:
diff --git a/subject_grants.toml b/subject_grants.toml
index 993083c9f..2f8d7c1e1 100644
--- a/subject_grants.toml
+++ b/subject_grants.toml
@@ -1296,10 +1296,6 @@ rationale = "Consumes the object_assignment; single operand."
param = "object_assignment"
rationale = "Consumes the object_assignment; single operand."
-["pipelex/cogt/content_generation/dry_mock.py::_reconstruct_object_class"]
-param = "object_assignment"
-rationale = "Consumes the object_assignment; single operand."
-
["pipelex/cogt/content_generation/dry_mock.py::build_mock_object"]
param = "model_class"
rationale = "Factory from source: builds the result from the model_class; single operand."
diff --git a/tests/integration/pipelex/cogt/content_generation/test_dry_run_object_fidelity.py b/tests/integration/pipelex/cogt/content_generation/test_dry_run_object_fidelity.py
index 1682fb425..e9c1e8b18 100644
--- a/tests/integration/pipelex/cogt/content_generation/test_dry_run_object_fidelity.py
+++ b/tests/integration/pipelex/cogt/content_generation/test_dry_run_object_fidelity.py
@@ -1,12 +1,16 @@
-"""Regression test for the dry-run object-mock fidelity gap (review F2).
+"""Where the dry-run object-mock fidelity gap still lives, now that the in-process path is fixed.
-A dry-run object mock is built from the schema-reconstructed class, then re-validated by
-``ContentGenerator.make_object`` against the **original** class. An invariant the JSON-schema round-trip
-cannot capture — here a custom ``@field_validator`` — is absent from the reconstructed class, so the mock
-carries a value the original class rejects. The contract under test: that failure surfaces as a clear typed
-``DryRunObjectFidelityError`` (naming the class and the ``examples`` / ``mock_format`` remedy), NOT an
-opaque ``pydantic.ValidationError`` mid-run. A class without such a hidden invariant round-trips fine and the mock
-validates — proving the guard is scoped to genuine fidelity failures, not every mock object.
+The gap only exists when the mock is built from a class *rebuilt from JSON schema*: an invariant the
+round trip cannot capture — here a custom ``@field_validator`` — is absent from the rebuild, so
+polyfactory fills a value the original class rejects. That is the boundary case (a worker holding only
+a serialized ``ObjectAssignment``), and there the failure must still surface as a clear typed
+``DryRunObjectFidelityError`` naming the class and the ``examples`` / ``mock_format`` remedy, not an
+opaque ``pydantic.ValidationError`` mid-run.
+
+In-process the caller's real class travels down to the leaf, so the mock is built from it and the gap
+cannot occur — the constrained class instead fails at *build* time with the equally typed
+``DryRunMockBuildError``, which names the same remedy. A class with no hidden invariant mocks and
+validates cleanly either way.
No provider is ever called (the leaf mock short-circuits before ``get_llm_worker``), so this needs no
inference marker.
@@ -14,9 +18,12 @@ inference marker.
import pytest
+from pipelex.cogt.content_generation.assignment_models import LLMAssignment, ObjectAssignment
from pipelex.cogt.content_generation.cogt_run_params import CogtRunParams
+from pipelex.cogt.content_generation.content_generator import _revalidate_against_object_class # noqa: PLC2701 # pyright: ignore[reportPrivateUsage]
from pipelex.cogt.content_generation.content_generator_protocol import ContentGeneratorProtocol
-from pipelex.cogt.content_generation.exceptions import DryRunObjectFidelityError
+from pipelex.cogt.content_generation.dry_mock import dry_llm_gen_object, dry_llm_gen_object_list
+from pipelex.cogt.content_generation.exceptions import DryRunMockBuildError, DryRunObjectFidelityError
from pipelex.cogt.llm.llm_prompt import LLMPrompt
from pipelex.cogt.llm.llm_setting import LLMSetting
from pipelex.system.job_metadata import JobMetadata
@@ -24,16 +31,47 @@ from pipelex.system.pipe_run_mode import PipeRunMode
from tests.integration.pipelex.cogt.content_generation.test_data import ConstrainedName, PlainName
+def _boundary_assignment(job_metadata: JobMetadata) -> ObjectAssignment:
+ """The payload a worker receives across the boundary: schema only, no live class."""
+ return ObjectAssignment.make_for_class(
+ object_class=ConstrainedName,
+ llm_assignment=LLMAssignment(
+ job_metadata=job_metadata,
+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.DRY),
+ llm_setting=LLMSetting(model="gpt-4o", temperature=0.5),
+ llm_prompt=LLMPrompt(user_text="make a prefixed name"),
+ ),
+ nb_items=2,
+ )
+
+
@pytest.mark.asyncio(loop_scope="class")
class TestDryRunObjectFidelity:
def _dry_cogt_run_params(self) -> CogtRunParams:
return CogtRunParams(run_mode=PipeRunMode.DRY)
- async def test_make_object_raises_typed_error_on_fidelity_gap(
+ async def test_boundary_object_mock_raises_typed_fidelity_error(self, job_metadata: JobMetadata) -> None:
+ """No class in hand: the schema-built mock fails re-validation as the typed fidelity error."""
+ raw_mock = dry_llm_gen_object(_boundary_assignment(job_metadata))
+
+ with pytest.raises(DryRunObjectFidelityError) as exc_info:
+ _revalidate_against_object_class(raw_mock, object_class=ConstrainedName, is_mock_built=True)
+ assert ConstrainedName.__name__ in str(exc_info.value)
+ assert "mock_format" in str(exc_info.value)
+
+ async def test_boundary_object_list_mock_raises_typed_fidelity_error(self, job_metadata: JobMetadata) -> None:
+ """The list leaf carries the same boundary-path fidelity contract, per item."""
+ raw_mocks = dry_llm_gen_object_list(_boundary_assignment(job_metadata))
+
+ # Same comprehension shape as ContentGenerator.make_object_list, so this is the real composition.
+ with pytest.raises(DryRunObjectFidelityError):
+ _ = [_revalidate_against_object_class(raw_mock, object_class=ConstrainedName, is_mock_built=True) for raw_mock in raw_mocks]
+
+ async def test_make_object_builds_the_mock_from_the_caller_class(
self, job_metadata: JobMetadata, content_generator: ContentGeneratorProtocol
) -> None:
- """make_object re-raises the dropped-invariant ValidationError as DryRunObjectFidelityError."""
- with pytest.raises(DryRunObjectFidelityError) as exc_info:
+ """In-process the invariant is present at build time, so the failure is a build error, not a fidelity gap."""
+ with pytest.raises(DryRunMockBuildError) as exc_info:
await content_generator.make_object(
job_metadata=job_metadata,
cogt_run_params=self._dry_cogt_run_params(),
@@ -44,11 +82,11 @@ class TestDryRunObjectFidelity:
assert ConstrainedName.__name__ in str(exc_info.value)
assert "mock_format" in str(exc_info.value)
- async def test_make_object_list_raises_typed_error_on_fidelity_gap(
+ async def test_make_object_list_builds_the_mock_from_the_caller_class(
self, job_metadata: JobMetadata, content_generator: ContentGeneratorProtocol
) -> None:
- """make_object_list re-raises the per-item fidelity failure as the same typed error."""
- with pytest.raises(DryRunObjectFidelityError):
+ """List counterpart: same in-process resolution, same typed build error."""
+ with pytest.raises(DryRunMockBuildError):
await content_generator.make_object_list(
job_metadata=job_metadata,
cogt_run_params=self._dry_cogt_run_params(),
@@ -60,7 +98,7 @@ class TestDryRunObjectFidelity:
async def test_make_object_succeeds_when_no_hidden_invariant(
self, job_metadata: JobMetadata, content_generator: ContentGeneratorProtocol
) -> None:
- """A class whose constraints survive the round-trip mocks and re-validates without the guard firing."""
+ """A class whose constraints survive the round-trip mocks and re-validates without any guard firing."""
result = await content_generator.make_object(
job_metadata=job_metadata,
cogt_run_params=self._dry_cogt_run_params(),
diff --git a/tests/integration/pipelex/cogt/content_generation/test_leaf_dry_object_mocks.py b/tests/integration/pipelex/cogt/content_generation/test_leaf_dry_object_mocks.py
index 63b7b1950..7d45a086a 100644
--- a/tests/integration/pipelex/cogt/content_generation/test_leaf_dry_object_mocks.py
+++ b/tests/integration/pipelex/cogt/content_generation/test_leaf_dry_object_mocks.py
@@ -1,15 +1,18 @@
"""Integration tests for the ``run_mode == DRY`` object mocks at the cogt leaf.
-The dry object mock is built from the **schema-reconstructed** class — the single schema-based
-mock site shared by both backends (pre-flight decision 2), exercising real datamodel-code-generator
-codegen. Contracts pinned here:
+These call the leaves with no live class in hand, so the mock is built from the **schema-reconstructed**
+class (pre-flight decision 2), exercising real datamodel-code-generator codegen. Contracts pinned here:
- a representative ``StructuredContent`` mocks into a valid instance of the original class
(the object-mock fidelity pin mandated by pre-flight decision 2);
- ``nb_items`` carried on ``ObjectAssignment`` controls the dry list length, falling back to
``dry_run_config.nb_list_items`` (eng review D11);
-- a hidden invariant the schema round-trip drops surfaces as ``DryRunObjectFidelityError`` (eng review D6);
-- the structured-search dry leaf returns a dict that validates against the original class.
+- the structured-search dry leaf returns a dict that validates against the original class, and a hidden
+ invariant the schema round-trip drops there surfaces as ``DryRunObjectFidelityError`` (eng review D6).
+
+The object leaf's own fidelity arms live in ``test_dry_run_object_fidelity.py``, which separates the
+boundary case (still schema-rebuilt, still the fidelity error) from the in-process case (built from the
+caller's real class).
No provider is ever called (the leaves short-circuit before any worker), so no inference marker.
"""
@@ -131,20 +134,6 @@ class TestLeafDryObjectMocks:
assert len(result) == 3
assert all(isinstance(item, RepresentativeInvoiceLine) for item in result)
- async def test_generator_dry_arm_raises_typed_fidelity_error(
- self, job_metadata: JobMetadata, content_generator: ContentGeneratorProtocol
- ) -> None:
- """The DRY arm of the fidelity wrapper fires (D6): a dropped invariant surfaces as the typed error."""
- with pytest.raises(DryRunObjectFidelityError) as exc_info:
- await content_generator.make_object(
- job_metadata=job_metadata,
- cogt_run_params=CogtRunParams(run_mode=PipeRunMode.DRY),
- object_class=ConstrainedName,
- llm_setting_for_object=LLMSetting(model="gpt-4o", temperature=0.5),
- llm_prompt_for_object=LLMPrompt(user_text="make a prefixed name"),
- )
- assert ConstrainedName.__name__ in str(exc_info.value)
-
async def test_dry_search_structured_fidelity_gap_raises_typed_error(
self, job_metadata: JobMetadata, content_generator: ContentGeneratorProtocol
) -> None:
diff --git a/tests/integration/pipelex/cogt/content_generation/test_object_class_passthrough.py b/tests/integration/pipelex/cogt/content_generation/test_object_class_passthrough.py
new file mode 100644
index 000000000..4ae98a2ae
--- /dev/null
+++ b/tests/integration/pipelex/cogt/content_generation/test_object_class_passthrough.py
@@ -0,0 +1,237 @@
+"""The caller's live class must reach the provider unchanged when there is one.
+
+Structured generation used to hand the provider a class rebuilt from the caller's JSON schema even
+when the caller's real class was still on the stack. The rebuild is lossy — custom validators and
+``json_schema_extra`` hints do not survive the round trip — so the provider was constrained by a
+weaker contract than the one the author wrote.
+
+These tests pin both arms:
+
+- in-process (a class is in hand): the provider receives *the same class object*, so every invariant
+ is intact. Asserted with ``is``: a structural-equality assertion would pass against the bug, since
+ structural equality is exactly what the round trip already preserved.
+- at the boundary (only the serialized assignment): the class is still rebuilt from the schema, and
+ the loss is still real — which is what makes the first arm worth having.
+
+The LLM worker is mocked, so no inference marker.
+"""
+
+from typing import Any, get_args
+
+import pytest
+from pydantic import BaseModel, Field, ValidationError, create_model, field_validator
+from pytest_mock import MockerFixture
+
+from pipelex.cogt.content_generation.assignment_models import LLMAssignment, ObjectAssignment
+from pipelex.cogt.content_generation.cogt_run_params import CogtRunParams
+from pipelex.cogt.content_generation.content_generator import _revalidate_against_object_class # noqa: PLC2701 # pyright: ignore[reportPrivateUsage]
+from pipelex.cogt.content_generation.content_generator_protocol import ContentGeneratorProtocol
+from pipelex.cogt.content_generation.dry_mock import dry_llm_gen_object
+from pipelex.cogt.content_generation.llm_generate import llm_gen_object, llm_gen_object_list
+from pipelex.cogt.llm.llm_prompt import LLMPrompt
+from pipelex.cogt.llm.llm_setting import LLMSetting
+from pipelex.core.stuffs.structured_content import StructuredContent
+from pipelex.system.job_metadata import JobMetadata
+from pipelex.system.pipe_run_mode import PipeRunMode
+
+
+class HintedName(StructuredContent):
+ """Carries both kinds of invariant the JSON-schema round trip drops."""
+
+ name: str = Field(json_schema_extra={"mock_format": "name"})
+
+ @field_validator("name")
+ @classmethod
+ def _require_prefix(cls, value: str) -> str:
+ if not value.startswith("PFX_"):
+ msg = "name must start with 'PFX_'"
+ raise ValueError(msg)
+ return value
+
+
+class SimpleName(StructuredContent):
+ """No invariant the round trip drops, so its mock builds under either class resolution."""
+
+ name: str
+
+
+class NormalizedReference(StructuredContent):
+ """A validator that *transforms* rather than rejects — the shape that exposes double validation.
+
+ A rejecting validator is idempotent and cannot tell one execution from two, which is why the
+ identity assertions above are not enough on their own.
+ """
+
+ reference: str
+
+ @field_validator("reference")
+ @classmethod
+ def _normalize(cls, value: str) -> str:
+ return f"INV-{value}"
+
+
+class _StubListResponse(BaseModel):
+ """Stand-in for the list wrapper the live leaf builds, so the mocked worker can return something."""
+
+ items: list[HintedName]
+
+
+def _live_object_assignment() -> ObjectAssignment:
+ llm_assignment = LLMAssignment(
+ job_metadata=JobMetadata(user_id="u", pipeline_run_id="run_object_class_passthrough"),
+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),
+ llm_setting=LLMSetting(model="test-model", temperature=0.5),
+ llm_prompt=LLMPrompt(user_text="make a name"),
+ )
+ return ObjectAssignment.make_for_class(object_class=HintedName, llm_assignment=llm_assignment)
+
+
+def _dry_object_assignment() -> ObjectAssignment:
+ llm_assignment = LLMAssignment(
+ job_metadata=JobMetadata(user_id="u", pipeline_run_id="run_object_class_passthrough_dry"),
+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.DRY),
+ llm_setting=LLMSetting(model="test-model", temperature=0.5),
+ llm_prompt=LLMPrompt(user_text="make a name"),
+ )
+ return ObjectAssignment.make_for_class(object_class=SimpleName, llm_assignment=llm_assignment)
+
+
+def _patch_worker(mocker: MockerFixture, *, gen_object_result: BaseModel) -> Any:
+ mock_worker = mocker.MagicMock()
+ mock_worker.gen_object = mocker.AsyncMock(return_value=gen_object_result)
+ mocker.patch("pipelex.cogt.content_generation.llm_generate.get_llm_worker", return_value=mock_worker)
+ return mock_worker
+
+
[email protected](loop_scope="class")
+class TestObjectClassPassthrough:
+ async def test_live_leaf_hands_the_provider_the_caller_class_itself(self, mocker: MockerFixture) -> None:
+ """Identity, not equality: the provider gets the very class object the caller passed."""
+ mock_worker = _patch_worker(mocker, gen_object_result=HintedName(name="PFX_ok"))
+
+ await llm_gen_object(_live_object_assignment(), object_class=HintedName)
+
+ assert mock_worker.gen_object.await_args.kwargs["schema"] is HintedName
+
+ async def test_live_leaf_preserves_validator_and_schema_hints(self, mocker: MockerFixture) -> None:
+ """The class the provider receives still enforces the custom validator and carries its hints."""
+ mock_worker = _patch_worker(mocker, gen_object_result=HintedName(name="PFX_ok"))
+
+ await llm_gen_object(_live_object_assignment(), object_class=HintedName)
+
+ received_class = mock_worker.gen_object.await_args.kwargs["schema"]
+ with pytest.raises(ValidationError):
+ received_class.model_validate({"name": "no_prefix"})
+ assert received_class.model_json_schema()["properties"]["name"]["mock_format"] == "name"
+
+ async def test_boundary_leaf_still_rebuilds_from_the_schema_and_loses_the_invariants(self, mocker: MockerFixture) -> None:
+ """Without a class in hand the leaf rebuilds — unchanged behavior, and the loss is real.
+
+ This is the control for the two tests above: it shows the reconstructed class accepts data the
+ author's class rejects, which is exactly what passing the live class stops.
+ """
+ mock_worker = _patch_worker(mocker, gen_object_result=HintedName(name="PFX_ok"))
+
+ await llm_gen_object(_live_object_assignment())
+
+ rebuilt_class = mock_worker.gen_object.await_args.kwargs["schema"]
+ assert rebuilt_class is not HintedName
+ assert rebuilt_class.__name__ == HintedName.__name__
+ # The custom validator did not survive the round trip.
+ assert rebuilt_class.model_validate({"name": "no_prefix"}).name == "no_prefix"
+
+ async def test_live_list_leaf_wraps_the_caller_class_itself(self, mocker: MockerFixture) -> None:
+ """The list wrapper's item type is the caller's class, not a rebuild of it."""
+ mock_worker = _patch_worker(mocker, gen_object_result=_StubListResponse(items=[HintedName(name="PFX_ok")]))
+
+ await llm_gen_object_list(_live_object_assignment(), object_class=HintedName)
+
+ list_schema = mock_worker.gen_object.await_args.kwargs["schema"]
+ item_annotation = list_schema.model_fields["items"].annotation
+ assert get_args(item_annotation)[0] is HintedName
+
+ async def test_boundary_list_leaf_still_wraps_a_rebuilt_class(self, mocker: MockerFixture) -> None:
+ """List counterpart of the boundary control: no class in hand still means a rebuilt item class."""
+ mock_worker = _patch_worker(mocker, gen_object_result=_StubListResponse(items=[HintedName(name="PFX_ok")]))
+
+ await llm_gen_object_list(_live_object_assignment())
+
+ list_schema = mock_worker.gen_object.await_args.kwargs["schema"]
+ item_annotation = list_schema.model_fields["items"].annotation
+ assert get_args(item_annotation)[0] is not HintedName
+
+ async def test_caller_validators_run_exactly_once(self, job_metadata: JobMetadata, content_generator: ContentGeneratorProtocol) -> None:
+ """Handing the live class down must make the caller's validator constrain the result once, not twice.
+
+ The leaf now builds from the caller's class, so its validators already ran there. Re-validating
+ the result against the same class would run them again on data they had already normalized —
+ ``INV-INV-…`` — which is the opposite of honoring the invariant this change exists to preserve.
+ """
+ result = await content_generator.make_object(
+ job_metadata=job_metadata,
+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.DRY),
+ object_class=NormalizedReference,
+ llm_prompt_for_object=LLMPrompt(user_text="make a reference"),
+ llm_setting_for_object=LLMSetting(model="test-model", temperature=0.5),
+ )
+
+ assert result.reference.startswith("INV-")
+ assert not result.reference.startswith("INV-INV-")
+
+ async def test_caller_validators_run_exactly_once_per_list_item(
+ self, job_metadata: JobMetadata, content_generator: ContentGeneratorProtocol
+ ) -> None:
+ """The list path revalidates per item, so it carries the same once-only contract."""
+ results = await content_generator.make_object_list(
+ job_metadata=job_metadata,
+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.DRY),
+ object_class=NormalizedReference,
+ llm_prompt_for_object_list=LLMPrompt(user_text="make references"),
+ llm_setting_for_object_list=LLMSetting(model="test-model", temperature=0.5),
+ nb_items=3,
+ )
+
+ assert len(results) == 3
+ # Both directions: `not INV-INV-` alone would also pass if the validator never ran at all.
+ assert all(item.reference.startswith("INV-") for item in results)
+ assert all(not item.reference.startswith("INV-INV-") for item in results)
+
+ async def test_instructor_style_subclass_is_returned_without_revalidation(self, mocker: MockerFixture) -> None:
+ """Instructor returns a *subclass* of the response model, not the class itself.
+
+ `create_model(cls.__name__, __base__=(cls, OpenAISchema))` is what actually comes back, which is
+ why the conversion short-circuits on `isinstance` and not `type(...) is`. A `type(...) is` check
+ would miss it and reinstate the double validation on every real provider call.
+ """
+ wrapped_class = create_model("NormalizedReference", __base__=NormalizedReference)
+ _patch_worker(mocker, gen_object_result=wrapped_class(reference="123"))
+
+ raw_obj = await llm_gen_object(
+ ObjectAssignment.make_for_class(
+ object_class=NormalizedReference,
+ llm_assignment=LLMAssignment(
+ job_metadata=JobMetadata(user_id="u", pipeline_run_id="run_wrapped"),
+ cogt_run_params=CogtRunParams(run_mode=PipeRunMode.LIVE),
+ llm_setting=LLMSetting(model="test-model", temperature=0.5),
+ llm_prompt=LLMPrompt(user_text="make a reference"),
+ ),
+ ),
+ object_class=NormalizedReference,
+ )
+
+ assert type(raw_obj) is not NormalizedReference
+ assert isinstance(raw_obj, NormalizedReference)
+ result = _revalidate_against_object_class(raw_obj, object_class=NormalizedReference, is_mock_built=False)
+ assert result.reference == "INV-123"
+
+ async def test_dry_leaf_resolves_the_class_the_same_way_as_the_live_leaf(self) -> None:
+ """Both run modes take the same path — asserted directly, not inferred from two passing tests.
+
+ If only the live leaf took the fast path, the dry mock would be built from a *weaker* class than
+ the one the provider is constrained by, making the mock less faithful than the thing it mocks.
+ """
+ with_class = dry_llm_gen_object(_dry_object_assignment(), object_class=SimpleName)
+ assert isinstance(with_class, SimpleName)
+
+ without_class = dry_llm_gen_object(_dry_object_assignment())
+ assert not isinstance(without_class, SimpleName)
diff --git a/tests/unit/pipelex/cogt/content_generation/test_dry_mock.py b/tests/unit/pipelex/cogt/content_generation/test_dry_mock.py
index 10dfebf6c..9e92731f4 100644
--- a/tests/unit/pipelex/cogt/content_generation/test_dry_mock.py
+++ b/tests/unit/pipelex/cogt/content_generation/test_dry_mock.py
@@ -89,27 +89,25 @@ class TestDryMock:
def test_failed_dry_object_build_reports_no_usage(self, mocker: MockerFixture) -> None:
"""A failed mock build must not leave a usage event behind (report only after success)."""
captured = self._capture_reported_job(mocker)
- mocker.patch("pipelex.cogt.content_generation.dry_mock._reconstruct_object_class", return_value=TextContent)
mocker.patch(
"pipelex.cogt.content_generation.dry_mock.build_mock_object",
side_effect=DryRunMockBuildError.for_object_class("TextContent"),
)
with pytest.raises(DryRunMockBuildError):
- dry_llm_gen_object(_dry_object_assignment())
+ dry_llm_gen_object(_dry_object_assignment(), object_class=TextContent)
assert captured == []
def test_failed_dry_object_list_build_reports_no_usage(self, mocker: MockerFixture) -> None:
"""The list leaf carries the same report-only-after-success contract."""
captured = self._capture_reported_job(mocker)
- mocker.patch("pipelex.cogt.content_generation.dry_mock._reconstruct_object_class", return_value=TextContent)
mocker.patch(
"pipelex.cogt.content_generation.dry_mock.build_mock_objects",
side_effect=DryRunMockBuildError.for_object_class("TextContent"),
)
with pytest.raises(DryRunMockBuildError):
- dry_llm_gen_object_list(_dry_object_assignment())
+ dry_llm_gen_object_list(_dry_object_assignment(), object_class=TextContent)
assert captured == []
diff --git a/tests/unit/pipelex/tools/test_func_registry.py b/tests/unit/pipelex/tools/test_func_registry.py
index 316ad48b6..fc92058d8 100644
--- a/tests/unit/pipelex/tools/test_func_registry.py
+++ b/tests/unit/pipelex/tools/test_func_registry.py
@@ -150,11 +150,32 @@ class TestFuncRegistry:
registry.register_functions([])
assert len(registry.root) == 0
- def test_register_existing_function_with_warning(self, registry: FuncRegistry, caplog: LogCaptureFixture):
+ def test_register_same_function_twice_is_idempotent(self, registry: FuncRegistry):
+ """Re-registering the very same function object is a no-op: a folder scanned twice must not fail."""
registry.register_function(valid_function)
- with caplog.at_level("DEBUG"):
- registry.register_function(valid_function)
- assert "already exists in registry" in caplog.text
+ registry.register_function(valid_function)
+ assert registry.get_function("valid_function") is valid_function
+
+ def test_register_colliding_function_raises_and_names_both_sources(self, registry: FuncRegistry):
+ """Two distinct functions under one name is an error, and the message must name both origins."""
+ registry.register_function(valid_function, name="shared_name")
+ with pytest.raises(FuncRegistryError) as exc_info:
+ registry.register_function(valid_async_function, name="shared_name")
+
+ message = str(exc_info.value)
+ assert "shared_name" in message
+ assert "valid_function" in message
+ assert "valid_async_function" in message
+ assert valid_function.__module__ in message
+ assert '@pipe_func(name="..."' in message
+ # The first registration wins: the collision is refused, not applied.
+ assert registry.get_function("shared_name") is valid_function
+
+ def test_ineligible_function_does_not_collide(self, registry: FuncRegistry):
+ """Ineligible functions never enter the registry, so they cannot trigger collision detection."""
+ registry.register_function(valid_function, name="shared_name")
+ registry.register_function(sample_function, name="shared_name")
+ assert registry.get_function("shared_name") is valid_function
def test_teardown(self, registry: FuncRegistry):
registry.register_function(valid_function)
diff --git a/tests/unit/pipelex/tools/test_func_registry_utils.py b/tests/unit/pipelex/tools/test_func_registry_utils.py
index 15ab541e5..7261e05b2 100644
--- a/tests/unit/pipelex/tools/test_func_registry_utils.py
+++ b/tests/unit/pipelex/tools/test_func_registry_utils.py
@@ -8,6 +8,7 @@ from pydantic import Field
from pipelex.core.memory.working_memory import WorkingMemory
from pipelex.core.stuffs.structured_content import StructuredContent
from pipelex.core.stuffs.text_content import TextContent
+from pipelex.system.registries.exceptions import FuncRegistryError
from pipelex.system.registries.func_registry import func_registry
from pipelex.system.registries.func_registry_utils import FuncRegistryUtils
@@ -227,6 +228,47 @@ async def nested_function(working_memory: WorkingMemory) -> TextContent:
assert func_registry.has_function("root_function"), "root_function should be registered"
assert not func_registry.has_function("nested_function"), "nested_function should NOT be registered"
+ def test_same_function_name_in_two_folders_raises(self):
+ """Two library dirs defining the same @pipe_func name must fail loudly, not silently pick a winner."""
+ source_template = """
+from pipelex.core.memory.working_memory import WorkingMemory
+from pipelex.core.stuffs.text_content import TextContent
+from pipelex.system.registries.func_registry import pipe_func
+
+@pipe_func()
+async def summarize(working_memory: WorkingMemory) -> TextContent:
+ return TextContent(text="{flavor}")
+"""
+ with tempfile.TemporaryDirectory() as first_dir, tempfile.TemporaryDirectory() as second_dir:
+ (Path(first_dir) / "first_funcs.py").write_text(source_template.format(flavor="first"))
+ (Path(second_dir) / "second_funcs.py").write_text(source_template.format(flavor="second"))
+
+ func_registry.teardown()
+ FuncRegistryUtils.register_funcs_in_folder(folder_path=Path(first_dir), is_recursive=False)
+ with pytest.raises(FuncRegistryError, match="summarize"):
+ FuncRegistryUtils.register_funcs_in_folder(folder_path=Path(second_dir), is_recursive=False)
+ func_registry.teardown()
+
+ def test_rescanning_the_same_folder_is_idempotent(self):
+ """A folder scanned twice reuses the cached module, so the same function object re-registers cleanly."""
+ with tempfile.TemporaryDirectory() as temp_dir:
+ (Path(temp_dir) / "stable_funcs.py").write_text("""
+from pipelex.core.memory.working_memory import WorkingMemory
+from pipelex.core.stuffs.text_content import TextContent
+from pipelex.system.registries.func_registry import pipe_func
+
+@pipe_func()
+async def stable_function(working_memory: WorkingMemory) -> TextContent:
+ return TextContent(text="stable")
+""")
+
+ func_registry.teardown()
+ FuncRegistryUtils.register_funcs_in_folder(folder_path=Path(temp_dir), is_recursive=False)
+ first_registration = func_registry.get_function("stable_function")
+ FuncRegistryUtils.register_funcs_in_folder(folder_path=Path(temp_dir), is_recursive=False)
+ assert func_registry.get_function("stable_function") is first_registration
+ func_registry.teardown()
+
def test_eligibility_check_directly(self):
# Valid async function
async def valid_async_function(working_memory: WorkingMemory) -> TextContent: # noqa: ARG001,RUF029 # pyright: ignore[reportUnknownParameterType,reportMissingParameterType, reportUnusedParameter]
diff --git a/wip/refactoring/boundary-revalidation-round-trip-is-unaudited.md b/wip/refactoring/boundary-revalidation-round-trip-is-unaudited.md
new file mode 100644
index 000000000..e7d752f5a
--- /dev/null
+++ b/wip/refactoring/boundary-revalidation-round-trip-is-unaudited.md
@@ -0,0 +1,31 @@
+# The boundary re-validation round trip is lossy in ways nobody has audited
+
+Deferred out of PR #1076 (`refactor/Exec`). **Pre-existing, unchanged by that PR, and it must stay** — it is the boundary conversion. Recorded so the next person to touch it knows what has and has not been checked.
+
+## What it is
+
+`_revalidate_against_object_class` (`pipelex/cogt/content_generation/content_generator.py`) converts a leaf result into the caller's class with:
+
+```python
+object_class.model_validate(raw_obj.model_dump(serialize_as_any=True))
+```
+
+After #1076 this only runs on the **boundary** path — a worker that held only the serialized `ObjectAssignment` returns an instance of a class rebuilt from the JSON schema, which is never a subclass of `object_class`, so the `isinstance` short-circuit does not catch it. In-process the object is returned untouched.
+
+## Why it is worth a look eventually
+
+The dump→validate round trip is not identity-preserving in general, and the ways it can lose are not covered by any test:
+
+- **Subclass erasure.** A dumped subclass instance revalidates as the declared class, silently dropping the subclass's extra fields and its type.
+- **Dumped-by-field-name data has to revalidate cleanly.** This currently happens to work only because of compensating configuration elsewhere: `PipeSearchSpec` / `PipeComposeSpec` set `populate_by_name=True`, and `PipeComposeSpec` carries a `normalize_construct_field` before-validator. A new aliased model on this path would not automatically inherit that luck.
+- **Validators run on the dumped form**, not the constructed form. For a model whose validators are not total over their own serialized output, that is a different input than the one they were written against.
+
+None of these is currently reachable in-process (the short-circuit sees to that) and none has a known live trigger on the boundary path either — which is exactly why this is a note and not a change.
+
+## What would close it
+
+An audit of what actually crosses this conversion on the boundary path, plus a round-trip property test over the models that do. If the audit finds a real loss, the fix is probably to have the boundary return raw data rather than a reconstructed instance, so there is one validation instead of a dump of one.
+
+## Related
+
+- The in-process arm of this same function was a genuine double-validation regression, found by the PR bots on #1076 and fixed there with an `isinstance` short-circuit. See that PR's `TODOS.md` for the reasoning.
diff --git a/wip/refactoring/func-registry-entries-outlive-their-library.md b/wip/refactoring/func-registry-entries-outlive-their-library.md
new file mode 100644
index 000000000..58f651eb0
--- /dev/null
+++ b/wip/refactoring/func-registry-entries-outlive-their-library.md
@@ -0,0 +1,49 @@
+# PipeFunc registrations outlive the library that registered them
+
+Deferred out of PR #1076 (`refactor/Exec`), raised as a P1 by the Codex reviewer and independently by the pre-landing review pass. **This is a genuine narrowing introduced by that PR, not a pre-existing quirk** — recorded here rather than fixed because the only correct fix is a design change, and it is worth deciding on its own terms.
+
+## The scenario
+
+A long-lived process that loads bundles sequentially:
+
+1. `open_library()` → `load_libraries(library_id=A, library_dirs=[…])` — registers `summarize` from project A.
+2. `LibraryManager.teardown(library_id=A)` — tears down A's pipe/concept/domain libraries and its own `ClassRegistry`.
+3. `open_library()` → `load_libraries(library_id=B, …)` — project B also defines `summarize`, at a different path.
+
+Step 3 now raises `FuncRegistryError`. A's registration is still in the registry: `LibraryManager.teardown` does not touch `func_registry`, and only `RuntimeBoot.teardown` clears it. So the "collision" is with a **stale** entry from a library nobody can reach any more, not with a concurrently-live one.
+
+Before #1076 the same sequence silently overwrote, and B won — which for the sequential case was the right outcome. So this path regressed from *works* to *fails until the process restarts*.
+
+## Who this hits
+
+**No first-party host can reach this today** — an earlier revision of this note named two victims, and tracing the code shows both are immune:
+
+- A self-hosted, direct-mode `pipelex-api` cannot vary its PipeFunc directories within one process. `ApiRunner()` is constructed per request with `library_dirs=None`, which `resolve_library_dirs` resolves to instance defaults fixed at boot or `PIPELEXPATH` — both process-constant. The per-request variation is inline `.mthds` text, which registers no functions. Re-scanning the same constant dirs returns `sys.modules`-cached modules, so the same function objects re-register and the idempotent no-op absorbs them.
+- The local MCP workshop (`pipelex-mcp`) is a TypeScript stdio *client* of the Pipelex API — it runs no pipelex in-process, and `.mthds` contents travel as text with no Python. It can only hit whatever server backs it, which is the previous (immune) case or the hosted runner.
+
+Not the hosted runner either: `is_pipe_func_sandbox_hosted()` takes the source-capture branch and never registers in-process. The Temporal worker ignores `library_dirs` outright, and the transported-PipeFunc path is one-shot per process.
+
+The exposed population is a **third-party embedder**: a long-lived Python process passing *different* `library_dirs` per call through the public surface (`PipelexMTHDSProtocol(library_dirs=…)`, `validate_bundles_in_process(library_dirs=…)`, `acquire_library(library_dirs=…)`), where two projects' Python files share a registration name. That is a legitimate use of a public parameter, so the narrowing is real — but when it fires, the failure is loud and typed (`FuncRegistryError` naming both origins and the `@pipe_func(name=...)` remedy), not silent corruption. This is why the narrowing was ruled non-blocking for PR #1076 (decision 2026-07-31): ship with the gap documented, fix ownership in its own PR.
+
+Two loads of the **same** bundle are fine — `import_module_from_file` returns the module cached under its absolute-path key, so the second scan re-registers the identical object and the idempotent no-op absorbs it. The failure needs two *different* files that share a registration name.
+
+## Why it was not fixed in #1076
+
+There is no small correct fix. `func_registry` is a flat, process-global `dict[str, Callable]` with **no ownership dimension** — nothing records which library registered a name, so per-library eviction has nothing to key on. Note the asymmetry that makes this stand out: each `Library` already owns its own `ClassRegistry` (`Library.get_class_registry`, torn down with the library). Functions never got the same treatment.
+
+The reviewer's proposed remedy — "registrations need library ownership and removal on teardown, including rollback after a failed scan" — is correct in substance and is a design change: a new ownership side-table, a `library_id` threaded through `FuncRegistryUtils` into `register_function`, eviction wired into `_pop_and_teardown_library`, and transactional rollback when a scan fails partway. That is not something to slip into a PR at bot-clean state.
+
+## What doing it looks like
+
+Give functions the ownership `ClassRegistry` already has:
+
+1. Thread the owning library id into registration (`FuncRegistryUtils.register_funcs_in_folder` already runs inside `scoped_current_library`, but `func_registry` is runtime-layer and must not import the interpreter's `current_library` — pass the id in as a plain `str`).
+2. Keep a `name -> library_id` side table; evict that library's names in `_pop_and_teardown_library`.
+3. Roll back the names a scan registered when that scan raises partway, so a failed load does not leave half a library behind.
+4. Then the collision check keys on live entries only, and the stale case disappears rather than being papered over.
+
+Do **not** "fix" it by weakening the collision check back to a silent overwrite — that is the bug #1076 exists to remove.
+
+## Related
+
+- `wip/refactoring/structured-search-still-rebuilds-in-process.md`, `wip/refactoring/boundary-revalidation-round-trip-is-unaudited.md` — the other deferrals out of the same PR.
diff --git a/wip/refactoring/revalidate-against-object-class-is-duplicated-three-ways.md b/wip/refactoring/revalidate-against-object-class-is-duplicated-three-ways.md
new file mode 100644
index 000000000..7682d099f
--- /dev/null
+++ b/wip/refactoring/revalidate-against-object-class-is-duplicated-three-ways.md
@@ -0,0 +1,39 @@
+# `_revalidate_against_object_class` exists three times, in two repos
+
+Surfaced while reviewing PR #1076 (`refactor/Exec`). Not caused by that PR — but that PR made the copies **diverge**, which is what turns tolerable duplication into a trap.
+
+## The three sites
+
+| where | shape | dump |
+|---|---|---|
+| `pipelex/cogt/content_generation/content_generator.py` — `_revalidate_against_object_class` | `BaseModel` in | `model_dump(serialize_as_any=True)` |
+| `pipelex-temporal` — `tprl_content_generation/content_generator_in_workflow.py:52` | `BaseModel` in | `model_dump(mode="json", serialize_as_any=True)` |
+| `pipelex/cogt/content_generation/content_generator.py` — inlined in `make_search_structured` | `dict` in | n/a (already a dict) |
+
+All three implement the same contract: validate the leaf's data against the caller's original class, and on the dry path re-raise a `ValidationError` as `DryRunObjectFidelityError` naming the class and the `examples` / `mock_format` remedy. The dry/live split, the `try`/`except`, and most of the explanatory docstring are duplicated verbatim.
+
+## Why the copies exist
+
+The first two are the two implementations of `ContentGeneratorProtocol` — the direct/inline backend and the workflow arm. The plugin could not reuse core's version for one reason only: it is `_`-private. It already imports `assignment_models`, `content_generator_protocol` and `exceptions` from that same package, so nothing else was in the way.
+
+The one real difference is `mode="json"`, needed because the Temporal object crossed a payload boundary and some fields only round-trip cleanly in json mode. That is a parameter, not a reason for a second function.
+
+## What #1076 changed, and why it matters now
+
+Core's copy gained an `isinstance(raw_obj, object_class)` short-circuit, to stop the caller's validators running twice when the leaf already built from the real class. The Temporal copy did **not** get it.
+
+That is correct today: across `workflow.execute_activity` the object is always an instance of the class rebuilt from `__kajson_class_source__`, never of `object_class`, so the conversion is always needed. But it is now a latent trap — if that path ever hands back the caller's real class (a local activity, an in-process shortcut, a converter change), it will double-validate exactly the way core's did before #1076, and nothing links the two files to make anyone notice. The reasoning about instructor returning a *subclass*, which is why the check is `isinstance` and not `type(...) is`, lives only in core's docstring.
+
+## What doing it looks like
+
+1. Promote one public helper into `pipelex.cogt.content_generation` — the plugin already imports from that package, so this needs no new boundary.
+2. Give it the dump mode as a parameter rather than forking the function. **Do not** unify blindly on `mode="json"` for the in-process path: it coerces values (datetime → str and back) and the round-trip's edge cases are unaudited — see `wip/refactoring/boundary-revalidation-round-trip-is-unaudited.md`, which is about exactly this conversion.
+3. Fold the `make_search_structured` inline into it, or give the dict-in case its own thin entry point on the same helper, so the fidelity-error contract has one home. The dict arm has no `isinstance` short-circuit (a dict is never an instance of the class), so its validation is the single one — and that shapes the search-path fix: see the dry-arm double-validation trap recorded in `wip/refactoring/structured-search-still-rebuilds-in-process.md`, which the helper's contract must accommodate.
+4. Then delete the plugin's copy and have it call the shared one.
+
+The prize is not line count — it is that the fidelity-error contract, and the `isinstance`-not-`type` reasoning, stop being things you have to already know to get right.
+
+## Related
+
+- `wip/refactoring/boundary-revalidation-round-trip-is-unaudited.md` — what the shared helper's boundary arm actually does, and what has not been checked about it.
+- `wip/refactoring/structured-search-still-rebuilds-in-process.md` — the search path's other half of the same story.
diff --git a/wip/refactoring/structured-search-still-rebuilds-in-process.md b/wip/refactoring/structured-search-still-rebuilds-in-process.md
new file mode 100644
index 000000000..81f1a18b4
--- /dev/null
+++ b/wip/refactoring/structured-search-still-rebuilds-in-process.md
@@ -0,0 +1,44 @@
+# The in-process structured-search path still rebuilds the output class from its schema
+
+Deferred out of PR #1076 (`refactor/Exec`). That PR fixed the same bug on the LLM object paths; this is the one instance it deliberately left.
+
+## What it is
+
+`ContentGenerator.make_search_structured` (`pipelex/cogt/content_generation/content_generator.py`) holds the caller's `output_structure_class` and never threads it down. `search_generate.search_gen_structured` and `dry_mock.dry_search_gen_structured` both still call `SchemaToModelFactory.make_from_json_schema` unconditionally.
+
+So a `PipeSearch` with a structured (non-text) output still hands the provider a class rebuilt from JSON schema, which loses exactly what #1076 established the rebuild loses:
+
+- custom `@field_validator` / `@model_validator` logic
+- `json_schema_extra` format/pattern hints
+- the output structure's own description (its class docstring)
+
+It is the same defect, in the same shape, roughly four lines from being fixed.
+
+## Why it was not done in #1076
+
+Not because it is hard — because of what it drags with it. Threading the class down changes a *second* error-class surface the same way the object path's changed: the dry mock would then be built from the real class, so `test_dry_search_structured_fidelity_gap_raises_typed_error` stops producing `DryRunObjectFidelityError` and starts producing `DryRunMockBuildError`. That needs its own test re-scoping and its own review pass. #1076 was already at bot-clean state when this surfaced, and quietly widening it there would have landed an unreviewed behavior change on a second surface.
+
+## ⚠ "Mirror the object path exactly" is not quite enough — the dry arm has a double-validation trap
+
+The search leaf is **dict-out by contract** (`search_gen_structured -> dict[str, Any]`; that is what keeps the dynamic class off the Temporal wire), and `make_search_structured`'s inline re-validation of that dict is the *single* validation on the live path. That difference from the object path matters when threading the class down:
+
+- **Dry arm (confirmed in code):** `dry_search_gen_structured` builds the mock and *dumps it to a dict*; the submitter then re-validates the dict. Hand the real class to `build_mock_object` naively and the caller's validators run at mock build **and again** on the dumped data — a transforming validator produces `INV-INV-…` in dry mode, the exact defect class #1076 fixed on the object path with the `isinstance` short-circuit. That short-circuit cannot apply as-is to a dict. So the fix needs one extra decision: either the in-process dry arm returns the mock *instance* (the leaf's in-process return widens to instance-or-dict and the shared helper's model arm short-circuits), or the submitter skips re-validation when the class was in hand at the leaf.
+- **Live arm (unverified):** the workers pass the schema class into the provider SDK (`linkup_search_worker` hands it to `async_search(structured_output_schema=…)`). Check whether the SDK internally instantiates that class before returning the dict — if it does, threading the real class in makes the caller's validators run inside the SDK and again in the submitter, the same double-run.
+
+This interacts with the shared-helper design in `wip/refactoring/revalidate-against-object-class-is-duplicated-three-ways.md` (the helper needs a dict arm without the short-circuit), which is one more reason to do that consolidation first.
+
+## What doing it looks like
+
+Mirror the object path — adjusted for the dict-out contract above:
+
+1. `search_gen_structured(search_object_assignment, *, output_class: type[BaseModel] | None = None)`, same keyword-only shape, `None` = boundary.
+2. Same for `dry_search_gen_structured`.
+3. Resolve through `object_class_resolution.resolve_object_class`'s sibling (it currently takes an `ObjectAssignment`; `SearchObjectAssignment` carries the same two fields under different names, so either generalize the helper over the two field names or add a thin second entry point — do **not** duplicate the `SchemaToModelFactory` call).
+4. `make_search_structured` passes `output_class=output_structure_class`.
+5. Re-scope `test_dry_search_structured_fidelity_gap_raises_typed_error` to the boundary composition, exactly as `test_dry_run_object_fidelity.py` was.
+
+`SearchObjectAssignment`'s wire shape must not change, for the same reason `ObjectAssignment`'s did not.
+
+## Related
+
+- `wip/refactoring/boundary-revalidation-round-trip-is-unaudited.md` — the other deferral out of the same PR.