Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
fix(parity): make three readers agree with the runtime about one authored fact (#1085)
Louis Choquel3ea6a705952f
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex daa6d644c..d781f2a1b 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -4,6 +4,10 @@### Fixed+- **Cross-domain pipe refs in normalized crates (Breaking)**: A bare pipe ref calling a pipe declared in another domain was normalized by prefixing the *caller's* domain, naming a pipe the crate does not hold — while the live library resolves the same ref by searching every domain. The crate fingerprinted the wrong content and a crate round-trip broke the method at run time with `PipeNotFoundError`, with nothing raised on the way in. Normalization now resolves a bare pipe ref against the whole crate exactly as `PipeLibrary` does, and raises `CrateNormalizationError` when the ref is ambiguous or matches no pipe instead of emitting a dangling ref.+- **Structureless concepts in the `python-structures` projection (Breaking)**: A concept declaring only a description was projected as a `StructuredContent` subclass while the runtime promotes the same declaration to a refinement of native `Text`. Since `TextContent` and `StructuredContent` are siblings, the interpreter's text-vs-object dispatch answered differently depending on which class it was handed — and the mismatch raised nothing, because the read-back validated against both. The projection now emits `TextContent` as the base, matching the runtime. A Python-class-backed concept (`structure = "<ClassName>"`) still projects onto the root base: its real shape is not visible to the crate.+- **Format-stable import blocks in generated Python**: A generated import line whose merged names crossed 88 columns was emitted flat, so the first `ruff format` in a consumer's tree rewrote it — changing the body bytes and making `pipelex codegen check` report the file as hand-edited. The import block now pre-explodes past the threshold like every other emitted construct.+- **Unimportable `python-structures` module for a concept refining `Anything`**: `Anything` is the one native with no content class of its own, so a concept refining it fell back to the root `StructuredContent` — emitting the class name without registering its import. The generated module then raised `NameError` on the consumer's first import. The fallback now goes through the renderer that registers the import.- **Input shaping honors its injected concept provider (Breaking)**: `InputShaper` took a `ConceptProviderAbstract` and consulted it for concept resolution and compatibility, then resolved each value's structure class from the process-global class registry anyway — the ambient lookup the provider parameter exists to replace. It now resolves the class through the provider and hands the resolved type down, so a caller that supplies its own provider can shape inputs without a loaded library. Breaking on one failure path: a declared structure class that does not resolve now raises `ConceptStructureClassNotFoundError` instead of arriving as a `StructureValidationError` about the value, which pointed the author at their input rather than at the concept.## [v0.42.0] - 2026-08-01diff --git a/docs/errors/crate-normalization-error.md b/docs/errors/crate-normalization-error.mdindex 9ebddf2f5..b67919791 100644--- a/docs/errors/crate-normalization-error.md+++ b/docs/errors/crate-normalization-error.md@@ -7,7 +7,7 @@ description: "Reference for the `CrateNormalizationError` Pipelex error class."# Crate normalization error-Raised when a library crate cannot be normalized (e.g. a crate key that is not domain-qualified).+Raised when a library crate cannot be normalized (e.g. a non-domain-qualified key, or a pipe ref matching no crate pipe, or several).| Field | Value ||---|---|diff --git a/docs/under-the-hood/codegen-projections.md b/docs/under-the-hood/codegen-projections.mdindex da2e87a5a..f14834fe4 100644--- a/docs/under-the-hood/codegen-projections.md+++ b/docs/under-the-hood/codegen-projections.md@@ -17,7 +17,7 @@ An emitter then walks the `ResolvedLibrary` and renders text. Emitters live in `The `types` projection ranges over the crate's concept set (one type per qualified concept). Three targets ship today:-- **`python-structures`** — Pipelex runtime `StructuredContent` subclasses (the runtime idiom, the successor to `pipelex build structures`). Native references map to the runtime content classes (`TextContent`, …); native concepts themselves are not re-emitted.+- **`python-structures`** — Pipelex runtime `StuffContent` subclasses (the runtime idiom, the successor to `pipelex build structures`). Native references map to the runtime content classes (`TextContent`, …); native concepts themselves are not re-emitted. Each concept lands on the same content class the runtime would resolve it to: a structured concept on `StructuredContent`, a concept refining a native on that native's content class, a structureless one on `TextContent`.- **`python-pydantic`** — plain `pydantic.BaseModel` types with no Pipelex imports. Every concept is emitted uniformly, including the materialized natives, so the module depends only on `pydantic` and the standard library.- **`ts-zod`** — a pure TypeScript + Zod types file (`types.ts`, imports only `zod`) plus a `binder.ts` companion. `types.ts` holds Zod schemas and their inferred types; type names are the concept codes; **field keys are the crate's snake_case wire names verbatim**. Concept references use `z.lazy(() => XSchema)` so declaration order is irrelevant and cycles are handled.@@ -91,7 +91,7 @@ The verdict rides the exit code (mirroring `resolve` / `validate`): `0` current,A stamp's `content_hash` is a raw SHA-256 over the body bytes. So if the emitted code is not already what your formatter wants, the first `ruff check --fix` / `ruff format` (or `prettier --write`) over your tree rewrites those bytes and `pipelex codegen check` reports the file as **hand-edited** — accusing you of the one thing you did not do.-The emitters therefore emit exactly what the formatter would write. Generated Python uses builtin generics (`list[str]`, `dict[str, int]`), `X | None` over `Optional[X]`, double-quoted `Literal` members, and isort-grouped imports; generated TypeScript matches Prettier's blank-line and import-wrapping behavior. Running your formatter over a generated tree is a no-op, and the stamp stays valid.+The emitters therefore emit exactly what the formatter would write. Generated Python uses builtin generics (`list[str]`, `dict[str, int]`), `X | None` over `Optional[X]`, double-quoted `Literal` members, and isort-grouped imports — merged, sorted, and, past the width threshold below, already wrapped into the parenthesized magic-trailing-comma form ruff itself would produce; generated TypeScript matches Prettier's blank-line and import-wrapping behavior. Running your formatter over a generated tree is a no-op, and the stamp stays valid.**You should not need to exclude generated paths from your linter.** If you carry such an exclusion from an older version, drop it.@@ -162,10 +162,12 @@ A deterministic tool that guesses is a liability. Where a resolved type carriesTwo concept-level cases are surfaced the same honest way:-- A **structureless** concept (no structure, no refinement) projects as an opaque type (an empty model / `z.unknown()`) with the imprecision stated in its docstring.-- A **Python-class-backed** concept (`structure = "<ClassName>"`, whose shape lives only in hand-written Python, not in MTHDS) is surfaced as opaque — the bare class name is never silently emitted into a portable crate.+- A **structureless** concept (no structure, no refinement) projects as an opaque type — an empty model for `python-pydantic`, `z.unknown()` for ts-zod — with the imprecision stated in its docstring. Its *base class* is not a guess, though: `python-structures` emits it on `TextContent`, because that is what the runtime resolves the same declaration to — "describe it in prose and get text back" is what a structureless concept means to an author, and the interpreter's text-vs-object dispatch reads the base class to decide which call to make. A projection that emitted the root base instead would answer that question differently from the runtime for the same authored concept, silently. That inherited `text` field is what makes this target's structureless class the one exception to the pass-through rule below.+- A **Python-class-backed** concept (`structure = "<ClassName>"`, whose shape lives only in hand-written Python, not in MTHDS) is surfaced as opaque — the bare class name is never silently emitted into a portable crate. This is the one opaque shape that keeps the root base: its content class genuinely is not visible to the crate, so promoting it to `TextContent` would be the guess the rule above forbids.-Opaque is always **pass-through, never lossy**: the ts-zod `z.unknown()` hands the wire object through verbatim, and the Python emitters set `model_config = ConfigDict(extra="allow")` on opaque classes so `model_validate` keeps every field (pydantic's default `extra="ignore"` would silently strip the content). The owner of a Python-class-backed concept recovers the typed object by validating with their own class (`MyLegacyClass.model_validate(payload)`); every other consumer gets the honest untyped object.+Opaque is **pass-through, never lossy**: the ts-zod `z.unknown()` hands the wire object through verbatim, and the Python emitters set `model_config = ConfigDict(extra="allow")` on opaque classes so `model_validate` keeps every field (pydantic's default `extra="ignore"` would silently strip the content). The owner of a Python-class-backed concept recovers the typed object by validating with their own class (`MyLegacyClass.model_validate(payload)`); every other consumer gets the honest untyped object.++The one exception is the shape described above: a **structureless** concept on `python-structures` inherits `TextContent.text`, which is required, so an object-shaped payload raises instead of landing in `extra`. That is not a regression against the runtime — the class `ConceptFactory` builds for the same declaration is a `TextContent` subclass too, so it rejects the same payload. Matching the runtime is the point; a projection that accepted a payload the runtime refuses would be the more expensive lie. Consumers who want the old permissive behaviour are describing a concept that is not structureless, and should give it a structure.## The extension-file storydiff --git a/pipelex/codegen/emitters/python_common.py b/pipelex/codegen/emitters/python_common.pyindex 8a5e18269..aef7506df 100644--- a/pipelex/codegen/emitters/python_common.py+++ b/pipelex/codegen/emitters/python_common.py@@ -255,9 +255,10 @@ def literal_annotation(*, choices: list[str] | None, imports: set[str]) -> str:def render_import_block(imports: set[str]) -> str:"""Render the collected imports the way isort would, so the artifact is lint-clean on arrival.- A flat `sorted()` over the raw statements is not isort order. Two things it misses: `from X import a`- and `from X import b` have to merge into one line, and stdlib has to be separated from third-party by- a blank line.+ A flat `sorted()` over the raw statements is not isort order. Three things it misses: `from X import a`+ and `from X import b` have to merge into one line, stdlib has to be separated from third-party by+ a blank line, and a merged line past `PY_EXPLODE_WIDTH` has to arrive already exploded (see+ `_import_statement`) or the consumer's formatter explodes it for us.Grouping targets the **consumer's** tree, where `pipelex` is an installed dependency and thereforethird-party. In this repo `pipelex` is first-party, so ruff here wants the opposite order — but no@@ -273,13 +274,29 @@ def render_import_block(imports: set[str]) -> str:stdlib: list[str] = []third_party: list[str] = []for module in sorted(names_by_module):- line = f"from {module} import {', '.join(sorted(names_by_module[module]))}"+ statement = _import_statement(module=module, names=sorted(names_by_module[module]))group = stdlib if module.partition(".")[0] in sys.stdlib_module_names else third_party- group.append(line)+ group.append(statement)return "\n\n".join("\n".join(group) for group in (stdlib, third_party) if group)+def _import_statement(*, module: str, names: list[str]) -> str:+ """One `from … import …` statement, pre-exploded past `PY_EXPLODE_WIDTH` like its sibling renderers.++ A flat import line is a formatter's business exactly like a flat `Field(...)` call: past the+ consumer's `line-length`, `ruff format` rewrites it into this parenthesized, magic-trailing-comma+ form — which changes the body bytes and makes `codegen check` report the file as hand-edited.+ Emitting the exploded form ourselves makes the artifact a fixed point at any width >= the+ threshold. Ruff wraps a single overlong name the same way, so the rule needs no name-count arm.+ """+ flat = f"from {module} import {', '.join(names)}"+ if len(flat) <= PY_EXPLODE_WIDTH:+ return flat+ rendered = "".join(f"{_PY_INDENT}{name},\n" for name in names)+ return f"from {module} import (\n{rendered})"++def python_module_body(*, header: str, imports: set[str], blocks: list[str]) -> str:"""Assemble a generated module; a projection with nothing to emit is its header alone.diff --git a/pipelex/codegen/emitters/python_structures.py b/pipelex/codegen/emitters/python_structures.pyindex 0bd1b5602..7f454654c 100644--- a/pipelex/codegen/emitters/python_structures.py+++ b/pipelex/codegen/emitters/python_structures.py@@ -1,9 +1,10 @@-"""python-structures emitter: project a crate's concept set as Pipelex `StructuredContent` classes.+"""python-structures emitter: project a crate's concept set as Pipelex `StuffContent` classes.The runtime idiom (today's `StructureGenerator` presentation) over the shared resolved layers. Each-non-native concept becomes a `StructuredContent` subclass; native *references* map to the runtime-content classes (`TextContent`, ...); a concept that still refines a native inherits that content-class. Native concepts themselves are not re-emitted — they already exist in the runtime.+non-native concept becomes a `StuffContent` subclass; native *references* map to the runtime content+classes (`TextContent`, ...); a concept that still refines a native inherits that content class, and+a structureless one inherits `TextContent` because that is what the runtime promotes it to (see+`_base_class`). Native concepts themselves are not re-emitted — they already exist in the runtime.Field annotations use `from __future__ import annotations`, so concept references are plain forwardnames resolved from the module namespace (no explicit string quoting, no ordering constraint beyond@@ -34,7 +35,7 @@ _FILENAME = "structures.py"def emit_python_structures(library: ResolvedLibrary) -> list[EmittedFile]:- """Emit the `structures.py` module: one `StructuredContent` subclass per non-native concept."""+ """Emit the `structures.py` module: one `StuffContent` subclass per non-native concept."""by_ref = library.by_ref()emitted = [concept for concept in library.concepts if not concept.is_native]in_module = {concept.concept_ref for concept in emitted}@@ -57,8 +58,14 @@ def _render_class(concept: ResolvedConcept, *, by_ref: dict[str, ResolvedConceptdocstring = class_docstring(concept.description, extra_line=caveat)header = f"class {class_name}({base}):"if concept.structureless:- # Opaque = pass-through, never lossy (B1-1): the runtime base inherits pydantic's default- # extra="ignore", which would silently strip every field on model_validate.+ # Opaque carries no declared field, so without this every field would hit pydantic's default+ # extra="ignore" and be silently stripped on model_validate (B1-1). It stays load-bearing on+ # all three structureless bases, `TextContent` included: there it preserves everything+ # *alongside* `text`. What it cannot do is make the promoted arm accept an object payload with+ # no `text` at all — that field is required, so pass-through holds for the fields around it,+ # not for a payload shaped like something else. The runtime's class for the same declaration is+ # a `TextContent` subclass too and refuses the same payload; see the exception noted in+ # docs/under-the-hood/codegen-projections.md.imports.add("from pydantic import ConfigDict")return f'{header}\n{docstring}\n\n model_config = ConfigDict(extra="allow")'if not concept.fields:@@ -69,6 +76,16 @@ def _render_class(concept: ResolvedConcept, *, by_ref: dict[str, ResolvedConceptdef _base_class(concept: ResolvedConcept, *, by_ref: dict[str, ResolvedConcept], imports: set[str]) -> str:if concept.base_ref is None:+ if concept.structureless and concept.opaque_python_class is None:+ # A concept declaring only a description is promoted by the runtime to a refinement of+ # native Text (`ConceptFactory._handle_basic_blueprint` generates it on a `TextContent`+ # base), so the projection has to land on the same content class. `TextContent` and+ # `StructuredContent` are siblings: emitting the latter makes the interpreter's+ # text-vs-object dispatch answer differently for the same authored concept, and the text+ # path's `model_validate({"text": ...})` succeeds on both, so nothing raises.+ # A Python-class-backed concept is the one structureless shape that keeps the root base:+ # its real shape lives in hand-written Python and is genuinely unknown here (B1-1 floor).+ return _native_class(NativeConceptCode.TEXT.concept_ref, imports=imports)return _structured_content(imports=imports)if NativeConceptCode.is_native_concept_ref_or_code(concept_ref_or_code=concept.base_ref):return _native_class(concept.base_ref, imports=imports)@@ -80,7 +97,7 @@ def _base_class(concept: ResolvedConcept, *, by_ref: dict[str, ResolvedConcept],def _structured_content(*, imports: set[str]) -> str:- """The runtime root base, registering its import — every concept refining a native uses `_native_class` instead."""+ """The runtime root base, registering its import — a concept whose content class is known uses `_native_class` instead."""imports.add("from pipelex.core.stuffs.structured_content import StructuredContent")return "StructuredContent"@@ -141,6 +158,8 @@ def _native_class(concept_ref: str, *, imports: set[str]) -> str:native_code = NativeConceptCode(QualifiedRef.parse(concept_ref).local_code)structure_class = native_code.structure_classif structure_class is None:- return "StructuredContent"+ # `Anything` alone has no content class of its own, so it falls back to the runtime root — via+ # the renderer that registers the import, never a bare name the module would not import.+ return _structured_content(imports=imports)imports.add(f"from {structure_class.__module__} import {structure_class.__name__}")return structure_class.__name__diff --git a/pipelex/libraries/crate_normalization.py b/pipelex/libraries/crate_normalization.pyindex e929aebc1..7e18534a3 100644--- a/pipelex/libraries/crate_normalization.py+++ b/pipelex/libraries/crate_normalization.py@@ -6,7 +6,10 @@ is closed, canonical, and explicit:1. merge — already done upstream (keys are `domain.Code` / `domain.code`);2. fully qualify every *in-body* reference — pipe `inputs` / `output` concept refs, pipe-step / branch- / outcome / batch pipe refs, concept `refines`, and structure-field `concept_ref` / `item_concept_ref`;+ / outcome / batch pipe refs, concept `refines`, and structure-field `concept_ref` / `item_concept_ref`.+ A bare *pipe* ref resolves crate-wide (`_qualify_pipe_ref`), the way `PipeLibrary.get_optional_pipe`+ does, so the normalized ref names the pipe an interpreted run would call — including when the callee+ is declared in a sibling domain;3. flatten refinement — a refining concept adopts its **in-crate structured** base's effectivestructure (the `refines` link is dropped once the structure is materialized). A concept whoserefinement chain bottoms out at a **native** keeps its `refines` link: the native is materialized@@ -31,6 +34,7 @@ are not yet applied here, mirroring the spec's own "Specification Status" callou[Library Crate Format]: mthds/docs/spec/library-crate.md"""+from collections import defaultdictfrom typing import NamedTuplefrom pipelex.codegen.native_expansion import collect_native_refs_from_structure, materialize_native_concept@@ -71,8 +75,10 @@ def normalize_crate(crate: LibraryCrate, *, mthds_version: str) -> LibraryCrate:}_flatten_refinement(concepts)+ pipe_refs_by_code = _index_pipe_refs_by_code(pipes=crate.pipes)pipes: dict[str, PipeBlueprintUnion] = {- pipe_ref: _normalize_pipe(blueprint, owner_domain=_domain_of(pipe_ref)) for pipe_ref, blueprint in crate.pipes.items()+ pipe_ref: _normalize_pipe(blueprint, owner_domain=_domain_of(pipe_ref), pipe_refs_by_code=pipe_refs_by_code)+ for pipe_ref, blueprint in crate.pipes.items()}_expand_natives(concepts=concepts, pipes=pipes)@@ -220,28 +226,43 @@ def _native_effective_structure(native_ref: str) -> dict[str, ConceptStructureBl# ---------------------------------------------------------------------------------------------------def _normalize_pipe(blueprint: PipeBlueprintUnion, *, owner_domain: str) -> PipeBlueprintUnion:+def _index_pipe_refs_by_code(*, pipes: dict[str, PipeBlueprintUnion]) -> dict[str, list[str]]:+ """Group the crate's own pipe refs by bare code — the index a bare in-body ref resolves against.++ Cross-package entries are excluded, mirroring `PipeLibrary.get_optional_pipe`, whose bare-code+ fallback searches only the entries that are not `alias->…` prefixed.+ """+ by_code: dict[str, list[str]] = defaultdict(list)+ for pipe_ref in pipes:+ if QualifiedRef.has_cross_package_prefix(pipe_ref):+ continue+ by_code[QualifiedRef.parse(pipe_ref).local_code].append(pipe_ref)+ return by_code+++def _normalize_pipe(blueprint: PipeBlueprintUnion, *, owner_domain: str, pipe_refs_by_code: dict[str, list[str]]) -> PipeBlueprintUnion:updates: dict[str, object] = {}if blueprint.inputs:updates["inputs"] = {name: _qualify_io_ref(ref, domain=owner_domain) for name, ref in blueprint.inputs.items()}if blueprint.output:updates["output"] = _qualify_io_ref(blueprint.output, domain=owner_domain)+ def qualify(*, pipe_ref: str) -> str:+ return _qualify_pipe_ref(pipe_ref, owner_domain=owner_domain, pipe_refs_by_code=pipe_refs_by_code)+match blueprint:case PipeSequenceBlueprint():- updates["steps"] = [step.model_copy(update={"pipe": _qualify_pipe_ref(step.pipe, domain=owner_domain)}) for step in blueprint.steps]+ updates["steps"] = [step.model_copy(update={"pipe": qualify(pipe_ref=step.pipe)}) for step in blueprint.steps]case PipeParallelBlueprint():- updates["branches"] = [- branch.model_copy(update={"pipe": _qualify_pipe_ref(branch.pipe, domain=owner_domain)}) for branch in blueprint.branches- ]+ updates["branches"] = [branch.model_copy(update={"pipe": qualify(pipe_ref=branch.pipe)}) for branch in blueprint.branches]case PipeConditionBlueprint():if blueprint.outcomes:- updates["outcomes"] = {key: _qualify_pipe_ref(pipe_ref, domain=owner_domain) for key, pipe_ref in blueprint.outcomes.items()}+ updates["outcomes"] = {key: qualify(pipe_ref=pipe_ref) for key, pipe_ref in blueprint.outcomes.items()}if blueprint.default_outcome:- updates["default_outcome"] = _qualify_pipe_ref(blueprint.default_outcome, domain=owner_domain)+ updates["default_outcome"] = qualify(pipe_ref=blueprint.default_outcome)case PipeBatchBlueprint():if blueprint.branch_pipe_code:- updates["branch_pipe_code"] = _qualify_pipe_ref(blueprint.branch_pipe_code, domain=owner_domain)+ updates["branch_pipe_code"] = qualify(pipe_ref=blueprint.branch_pipe_code)case _:pass@@ -266,14 +287,31 @@ def _render_ref_with_markers(concept_ref: str, *, parsed: MultiplicityParseResulreturn f"{concept_ref}{suffix}{parsed.presence.symbol}"-def _qualify_pipe_ref(pipe_ref: str, *, domain: str) -> str:+def _qualify_pipe_ref(pipe_ref: str, *, owner_domain: str, pipe_refs_by_code: dict[str, list[str]]) -> str:+ """Qualify one in-body pipe ref against the crate, the way the live library resolves it.++ A bare code is **not** the owner domain's by assumption: `PipeLibrary.get_optional_pipe` resolves+ it by searching every domain, so a pipe legitimately calls a sibling domain's pipe by bare code.+ Prefixing the owner domain here would name a pipe absent from `crate.pipes` — a crate that+ fingerprints the wrong content and dies with `PipeNotFoundError` on round-trip, with nothing+ raised on the way in. The crate-wide search is the same lookup, so the two readers agree; on+ ambiguity the library raises and so does this, rather than silently picking a winner.+ """if QualifiedRef.has_cross_package_prefix(pipe_ref):return pipe_ref # cross-package deferredif pipe_ref in SpecialOutcome.value_list():return pipe_refif "." in pipe_ref:return pipe_ref- return f"{domain}.{pipe_ref}"++ matches = pipe_refs_by_code.get(pipe_ref, [])+ if len(matches) == 1:+ return matches[0]+ if not matches:+ msg = f"Pipe ref '{pipe_ref}' (in domain '{owner_domain}') resolves to no pipe in the crate."+ raise CrateNormalizationError(msg)+ msg = f"Ambiguous pipe ref '{pipe_ref}' (in domain '{owner_domain}') matches {sorted(matches)} in the crate. Use a domain-qualified ref."+ raise CrateNormalizationError(msg)# --------------------------------------------------------------------------------------------------diff --git a/pipelex/libraries/exceptions.py b/pipelex/libraries/exceptions.pyindex 82a579bc8..e0fca00fb 100644--- a/pipelex/libraries/exceptions.py+++ b/pipelex/libraries/exceptions.py@@ -7,7 +7,7 @@ class LibraryError(PipelexError):class CrateNormalizationError(LibraryError):- """Raised when a library crate cannot be normalized (e.g. a crate key that is not domain-qualified)."""+ """Raised when a library crate cannot be normalized (e.g. a non-domain-qualified key, or a pipe ref matching no crate pipe, or several)."""_declared_title = "Crate normalization error"diff --git a/tests/unit/pipelex/codegen/conftest.py b/tests/unit/pipelex/codegen/conftest.pyindex 47ccbd55a..05a740587 100644--- a/tests/unit/pipelex/codegen/conftest.py+++ b/tests/unit/pipelex/codegen/conftest.py@@ -314,6 +314,22 @@ def refines_native_only_crate() -> LibraryCrate:return normalize_crate(authored, mthds_version=CRATE_TEST_VERSION)+def refines_anything_crate() -> LibraryCrate:+ """The lone concept refines `Anything` — the one native with no dedicated content class.++ Every other native maps to a runtime content class the emitter both names *and* imports. `Anything`+ is the single base that sends `python-structures` back to the root `StructuredContent`, which the+ native renderer therefore has to import itself. Nothing else in this crate reaches the root, so a+ missing import is not masked by a sibling concept — the module names a class it never imported.+ """+ authored = LibraryCrate(+ concepts={"wildcard.Loose": ConceptBlueprint(description="A refinement of the wildcard native", refines="Anything")},+ domains={"wildcard": DomainBlueprint(code="wildcard", description="Wildcard domain")},+ )+ return normalize_crate(authored, mthds_version=CRATE_TEST_VERSION)++def load_generated_module(content: str, *, tmp_path: Path, name: str) -> Any:"""Write generated Python to disk, import it, and return it (proves compile + exec).diff --git a/tests/unit/pipelex/codegen/test_emitted_artifacts_are_lint_clean.py b/tests/unit/pipelex/codegen/test_emitted_artifacts_are_lint_clean.pyindex a640f3d2e..716a105e1 100644--- a/tests/unit/pipelex/codegen/test_emitted_artifacts_are_lint_clean.py+++ b/tests/unit/pipelex/codegen/test_emitted_artifacts_are_lint_clean.py@@ -36,7 +36,7 @@ from typing import castimport pytest-from pipelex.codegen.emitters.python_common import python_header+from pipelex.codegen.emitters.python_common import PY_EXPLODE_WIDTH, python_header, render_import_blockfrom pipelex.codegen.emitters.python_pydantic import emit_python_pydanticfrom pipelex.codegen.emitters.python_structures import emit_python_structuresfrom pipelex.codegen.emitters.target import EmittedFile@@ -80,8 +80,8 @@ def _collect_kinds(resolved_type: ResolvedType) -> set[ResolvedTypeKind]:return kinds-def _assert_ruff_clean(*, emitted: list[EmittedFile], tmp_path: Path, target: str, line_length: int | None = None) -> None:- """Write the artifacts out and require both `ruff check` and `ruff format --check` to find nothing."""+def _write_artifacts(*, emitted: list[EmittedFile], tmp_path: Path) -> None:+ """Write the artifacts out, skipping the test when the current interpreter has no ruff."""try:subprocess.run([sys.executable, "-m", "ruff", "--version"], check=True, capture_output=True) # noqa: S603except (subprocess.CalledProcessError, FileNotFoundError):@@ -90,24 +90,41 @@ def _assert_ruff_clean(*, emitted: list[EmittedFile], tmp_path: Path, target: stfor emitted_file in emitted:(tmp_path / emitted_file.filename).write_text(emitted_file.content, encoding="utf-8")- width_config = ["--config", f"line-length={line_length}"] if line_length is not None else []- label = f"[{target}]" if line_length is None else f"[{target} @ line-length={line_length}]"- check = subprocess.run( # noqa: S603- [sys.executable, "-m", "ruff", "check", str(tmp_path), *_CONSUMER_LINT_CONFIG, *width_config],+def _width_config(line_length: int | None) -> list[str]:+ return ["--config", f"line-length={line_length}"] if line_length is not None else []+++def _label(*, target: str, line_length: int | None) -> str:+ return f"[{target}]" if line_length is None else f"[{target} @ line-length={line_length}]"+++def _assert_ruff_format_stable(*, tmp_path: Path, target: str, line_length: int | None = None) -> None:+ """Require `ruff format --check` to find nothing — the property byte-reproducibility rests on."""+ formatted = subprocess.run( # noqa: S603+ [sys.executable, "-m", "ruff", "format", "--check", str(tmp_path), "--config", str(_PYPROJECT), *_width_config(line_length)],capture_output=True,text=True,check=False,)- assert check.returncode == 0, f"{label} emitted artifact is not lint-clean:\n{check.stdout}{check.stderr}"+ label = _label(target=target, line_length=line_length)+ assert formatted.returncode == 0, f"{label} emitted artifact is not `ruff format` clean:\n{formatted.stdout}{formatted.stderr}"- formatted = subprocess.run( # noqa: S603- [sys.executable, "-m", "ruff", "format", "--check", str(tmp_path), "--config", str(_PYPROJECT), *width_config],++def _assert_ruff_clean(*, emitted: list[EmittedFile], tmp_path: Path, target: str, line_length: int | None = None) -> None:+ """Write the artifacts out and require both `ruff check` and `ruff format --check` to find nothing."""+ _write_artifacts(emitted=emitted, tmp_path=tmp_path)++ check = subprocess.run( # noqa: S603+ [sys.executable, "-m", "ruff", "check", str(tmp_path), *_CONSUMER_LINT_CONFIG, *_width_config(line_length)],capture_output=True,text=True,check=False,)- assert formatted.returncode == 0, f"{label} emitted artifact is not `ruff format` clean:\n{formatted.stdout}{formatted.stderr}"+ label = _label(target=target, line_length=line_length)+ assert check.returncode == 0, f"{label} emitted artifact is not lint-clean:\n{check.stdout}{check.stderr}"++ _assert_ruff_format_stable(tmp_path=tmp_path, target=target, line_length=line_length)class TestEmittedArtifactsAreLintClean:@@ -189,6 +206,36 @@ class TestEmittedArtifactsAreLintClean:crate = cast("LibraryCrate", request.getfixturevalue(crate_fixture))_assert_ruff_clean(emitted=emit(resolve_concepts_from_crate(crate)), tmp_path=tmp_path, target=target)+ @pytest.mark.parametrize(+ "imports",+ [+ pytest.param(+ {f"from a.deeply.nested.consumer.module import Alpha{index}Content" for index in range(6)},+ id="merged-names-cross-the-width",+ ),+ pytest.param(+ {"from a.deeply.nested.consumer.module import ASingleNameLongEnoughToCrossTheThresholdOnItsOwn"},+ id="one-name-crosses-the-width",+ ),+ ],+ )+ def test_import_block_is_format_stable_past_the_explode_width(self, imports: set[str], tmp_path: Path):+ """The import block is emitted bytes like any other, so it has to be a formatter fixed point too.++ Every *other* renderer pre-explodes past `PY_EXPLODE_WIDTH`; a flat import line that crosses it is+ rewritten into the parenthesized form by the consumer's first `ruff format`, which changes the body+ bytes and makes `codegen check` report the file as hand-edited. No crate shape reaches the width+ today — each native content class lives in its own module, so nothing merges far enough — which is+ exactly why the property needs a direct test rather than a crate fixture that happens to cover it.++ Format-stable only, not lint-clean: an import block on its own has nothing to *use* the names it+ imports, so `F401` fires on a shape no emitter ever writes. The whole-artifact tests above cover+ lint-cleanliness; what is under test here is the formatter fixed point.+ """+ emitted = [EmittedFile(filename="structures.py", content=f"{python_header(target='python-structures')}{render_import_block(imports)}\n")]+ _write_artifacts(emitted=emitted, tmp_path=tmp_path)+ _assert_ruff_format_stable(tmp_path=tmp_path, target="import-block", line_length=PY_EXPLODE_WIDTH)+def test_natives_only_crate_emits_a_lint_clean_structures_module(self, natives_only_crate: LibraryCrate, tmp_path: Path):"""The reachable route to an empty projection, and the reason the case is not academic.diff --git a/tests/unit/pipelex/codegen/test_projection_agrees_with_runtime_base.py b/tests/unit/pipelex/codegen/test_projection_agrees_with_runtime_base.pynew file mode 100644index 000000000..ca889e29d--- /dev/null+++ b/tests/unit/pipelex/codegen/test_projection_agrees_with_runtime_base.py@@ -0,0 +1,122 @@+"""The `python-structures` projection and the runtime concept factory must read one declaration alike.++Both are faithful readings of the same authored `ConceptBlueprint`: `ConceptFactory` builds the class+an interpreted run stores in memory, `emit_python_structures` writes the class a consumer imports and+hands back to that same runtime. When they disagree about the **content class** a concept lands on,+nothing raises — the interpreter's text-vs-object dispatch just answers differently for the same+concept, and the text path's `model_validate({"text": ...})` succeeds on either class. So the gate is+the agreement itself: same nearest `StuffContent` ancestor, for every shape a concept can be authored in.++There are exactly four such shapes **among concepts the crate can resolve on its own**, and that+enumeration is closed because MTHDS forbids `refines` and `structure` together+(`ConceptBlueprint.validate_mutually_exclusive_fields`) — the one combination where a divergence could+hide behind a structured concept refining a structureless one cannot be authored at all.++Two shapes sit outside it, both because the crate genuinely cannot see what they resolve to (the B1-1+floor): a concept backed by a hand-written Python class, and one refining a target the crate does not+carry (a cross-package base leaves `structureless=True` *with* `base_ref` set, so it lands on the root+rather than on this module's structureless arm). Neither is gated here, and neither is a defect this+module can settle — see `wip/parity/structureless-concept-with-registered-class.md`.+"""++from pathlib import Path+from typing import Any++import pytest++from pipelex.codegen.emitters.python_structures import emit_python_structures+from pipelex.codegen.resolved_concepts import resolve_concepts_from_crate+from pipelex.core.concepts.concept_blueprint import ConceptBlueprint+from pipelex.core.concepts.concept_factory import ConceptFactory+from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprint, ConceptStructureBlueprintFieldType+from pipelex.core.domains.domain_blueprint import DomainBlueprint+from pipelex.core.stuffs.stuff_content import StuffContent+from pipelex.libraries.crate_normalization import normalize_crate+from pipelex.libraries.library_crate import LibraryCrate+from pipelex.system.registries.class_registry_access import get_class_registry+from tests.unit.pipelex.codegen.conftest import CRATE_TEST_VERSION, load_generated_module++_DOMAIN = "parity"++# One authored source, read by both sides. Declaration order matters for the runtime reader only:+# `Refined` needs `Structured`'s generated class registered before it can refine it.+_AUTHORED: dict[str, ConceptBlueprint] = {+ # structureless — the shape the runtime promotes to a refinement of native Text.+ "Opaque": ConceptBlueprint(description="A concept described in prose and nothing else"),+ # structured+ "Structured": ConceptBlueprint(+ description="A concept with an explicit structure",+ structure={"note": ConceptStructureBlueprint(description="a note", type=ConceptStructureBlueprintFieldType.TEXT, required=True)},+ ),+ # refines a native+ "RefinesNative": ConceptBlueprint(description="A small image", refines="Image"),+ # refines an in-crate concept+ "Refined": ConceptBlueprint(description="A narrower structured concept", refines="Structured"),+}++_RUNTIME_CONTENT_PACKAGE = "pipelex.core.stuffs."+++def _nearest_content_ancestor(concept_class: type[Any]) -> str:+ """The nearest ancestor that is a runtime content class — the vocabulary both readers share.++ Generated and projected classes are both anonymous to each other, so comparing them directly says+ nothing; what has to match is the runtime class each one bottoms out at.+ """+ for ancestor in concept_class.__mro__:+ if ancestor.__module__.startswith(_RUNTIME_CONTENT_PACKAGE):+ return ancestor.__name__+ msg = f"{concept_class.__name__} has no runtime content class in its MRO: {[base.__name__ for base in concept_class.__mro__]}"+ raise AssertionError(msg)+++class TestProjectionAgreesWithRuntimeBase:+ @pytest.fixture+ def projected_module(self, tmp_path: Path) -> Any:+ authored = LibraryCrate(+ concepts={f"{_DOMAIN}.{code}": blueprint for code, blueprint in _AUTHORED.items()},+ domains={_DOMAIN: DomainBlueprint(code=_DOMAIN, description="Parity domain")},+ )+ crate = normalize_crate(authored, mthds_version=CRATE_TEST_VERSION)+ content = emit_python_structures(resolve_concepts_from_crate(crate))[0].content+ return load_generated_module(content, tmp_path=tmp_path, name="gen_structures_parity")++ @pytest.fixture+ def runtime_classes(self, load_empty_library: Any) -> dict[str, type[Any]]:+ load_empty_library()+ registry = get_class_registry()+ classes: dict[str, type[Any]] = {}+ for code, blueprint in _AUTHORED.items():+ concept = ConceptFactory.make_from_blueprint(domain_code=_DOMAIN, concept_code=code, blueprint_or_string_description=blueprint)+ classes[code] = registry.get_required_subclass(name=concept.structure_class_name, base_class=StuffContent)+ return classes++ @pytest.mark.parametrize("code", list(_AUTHORED))+ def test_both_readers_land_on_the_same_content_class(self, code: str, projected_module: Any, runtime_classes: dict[str, type[Any]]):+ projected: type[Any] = getattr(projected_module, code)+ assert _nearest_content_ancestor(projected) == _nearest_content_ancestor(runtime_classes[code])++ def test_structureless_lands_on_text_content(self, projected_module: Any, runtime_classes: dict[str, type[Any]]):+ """Name the answer, not just the agreement — the two could agree by both being wrong.++ `TextContent` is the runtime's reading: "describe it in prose and get text back" is what a+ structureless declaration means to an author today, and the interpreter's text-vs-object+ dispatch is what reads it back.+ """+ assert _nearest_content_ancestor(runtime_classes["Opaque"]) == "TextContent"+ assert _nearest_content_ancestor(projected_module.Opaque) == "TextContent"++ def test_python_class_backed_concept_keeps_the_root_base(self, tmp_path: Path):+ """The one structureless shape the projection must NOT promote.++ `structure = "<ClassName>"` names a hand-written Python class the crate cannot see, so its+ content class is genuinely unknown here — guessing `TextContent` would be the same defect in+ the other direction. The honest floor is the structurally valid root (B1-1).+ """+ authored = LibraryCrate(+ mthds_version=CRATE_TEST_VERSION,+ concepts={f"{_DOMAIN}.Legacy": ConceptBlueprint(description="python-backed", structure="MyLegacyClass")},+ )+ content = emit_python_structures(resolve_concepts_from_crate(authored))[0].content+ module = load_generated_module(content, tmp_path=tmp_path, name="gen_structures_parity_opaque_class")+ assert _nearest_content_ancestor(module.Legacy) == "StructuredContent"diff --git a/tests/unit/pipelex/codegen/test_python_structures_emitter.py b/tests/unit/pipelex/codegen/test_python_structures_emitter.pyindex aa3ba8c7c..3c6a24186 100644--- a/tests/unit/pipelex/codegen/test_python_structures_emitter.py+++ b/tests/unit/pipelex/codegen/test_python_structures_emitter.py@@ -50,6 +50,18 @@ class TestPythonStructuresEmitter:content = emit_python_structures(resolve_concepts_from_crate(refines_crate))[0].contentassert "class Thumbnail(ImageContent):" in content+ def test_refines_anything_imports_the_root_base_it_names(self, refines_anything_crate: LibraryCrate, tmp_path: Path):+ """Naming a base is not enough — the renderer that writes the name has to register its import.++ `Anything` is the only native with no content class of its own, so it is the only reference that+ falls back to the root `StructuredContent`. Emitting the bare name without the import produces a+ module that raises `NameError` on the consumer's first import, so this loads it rather than+ grepping for the line.+ """+ content = emit_python_structures(resolve_concepts_from_crate(refines_anything_crate))[0].content+ module = load_generated_module(content, tmp_path=tmp_path, name="gen_structures_anything")+ assert module.Loose.__mro__[1].__name__ == "StructuredContent"+def test_collision_qualifies_class_names(self, edge_crate: LibraryCrate):content = emit_python_structures(resolve_concepts_from_crate(edge_crate))[0].contentassert "class alpha__Result(" in contentdiff --git a/tests/unit/pipelex/libraries/test_crate_normalization.py b/tests/unit/pipelex/libraries/test_crate_normalization.pyindex 677f786d8..75e97b996 100644--- a/tests/unit/pipelex/libraries/test_crate_normalization.py+++ b/tests/unit/pipelex/libraries/test_crate_normalization.py@@ -1,3 +1,5 @@+from collections.abc import Callable+import pytestfrom pytest_mock import MockerFixture@@ -8,6 +10,10 @@ from pipelex.core.domains.domain_blueprint import DomainBlueprintfrom pipelex.libraries.crate_normalization import normalize_cratefrom pipelex.libraries.exceptions import CrateNormalizationErrorfrom pipelex.libraries.library_crate import LibraryCrate+from pipelex.pipe_controllers.batch.pipe_batch_blueprint import PipeBatchBlueprint+from pipelex.pipe_controllers.condition.pipe_condition_blueprint import PipeConditionBlueprint+from pipelex.pipe_controllers.condition.special_outcome import SpecialOutcome+from pipelex.pipe_controllers.parallel.pipe_parallel_blueprint import PipeParallelBlueprintfrom pipelex.pipe_controllers.sequence.pipe_sequence_blueprint import PipeSequenceBlueprintfrom pipelex.pipe_controllers.sub_pipe_blueprint import SubPipeBlueprintfrom pipelex.pipe_operators.llm.pipe_llm_blueprint import PipeLLMBlueprint@@ -82,6 +88,73 @@ def _authored_crate() -> LibraryCrate:)+def _cross_domain_crate() -> LibraryCrate:+ """A two-domain crate whose every controller kind calls a *sibling* domain's pipe by bare code.++ The authored shape of `pipelex-cookbook/examples/wip/advisory_board/`: an orchestrator domain+ driving pipes declared in a presentation domain. The library loads and runs, because+ `PipeLibrary.get_optional_pipe` resolves a bare code across every domain.+ """+ return LibraryCrate(+ pipes={+ "orchestrator.run_all": PipeSequenceBlueprint(+ description="Run the advisory board",+ output="Text",+ steps=[SubPipeBlueprint(pipe="present_as_markdown")],+ ),+ "orchestrator.fan_out": PipeParallelBlueprint(+ description="Render both presentations at once",+ inputs={"data": "Text"},+ output="Composite",+ branches=[+ SubPipeBlueprint(pipe="present_as_markdown", result="markdown"),+ SubPipeBlueprint(pipe="render_html", result="html"),+ ],+ ),+ "orchestrator.route": PipeConditionBlueprint(+ description="Pick a presentation",+ inputs={"data": "Text"},+ output="Text",+ expression="format",+ outcomes={"markdown": "present_as_markdown"},+ default_outcome="render_html",+ ),+ "orchestrator.route_to_special": PipeConditionBlueprint(+ description="Route to the special outcomes",+ inputs={"data": "Text"},+ output="Text",+ expression="format",+ outcomes={"stop": SpecialOutcome.FAIL},+ default_outcome=SpecialOutcome.CONTINUE,+ ),+ "orchestrator.batch_all": PipeBatchBlueprint(+ description="Present every document",+ inputs={"docs": "Text[]"},+ output="Text[]",+ branch_pipe_code="present_as_markdown",+ input_list_name="docs",+ input_item_name="doc",+ ),+ "presentation.present_as_markdown": PipeLLMBlueprint(+ description="Present as markdown",+ inputs={"data": "Text"},+ output="Text",+ prompt="Present $data as markdown",+ ),+ "presentation.render_html": PipeLLMBlueprint(+ description="Render as HTML",+ inputs={"data": "Text"},+ output="Text",+ prompt="Render $data as HTML",+ ),+ },+ domains={+ "orchestrator": DomainBlueprint(code="orchestrator", description="Orchestrator domain"),+ "presentation": DomainBlueprint(code="presentation", description="Presentation domain"),+ },+ )++def _reverse_refinement_chain_crate(size: int) -> LibraryCrate:concepts: dict[str, ConceptBlueprint | str] = {f"scale.Node{index:04d}": ConceptBlueprint(description=f"Node {index}", refines=f"Node{index + 1:04d}") for index in range(size - 1)@@ -124,12 +197,95 @@ class TestCrateNormalization:assert listed.output == "scoring.WeightedScore[]"def test_sequence_step_pipe_refs_qualified(self):- """A sequence step's bare pipe ref is qualified with the pipe's owner domain."""+ """A sequence step's bare pipe ref is qualified with the domain that declares the callee."""result = normalize_crate(_authored_crate(), mthds_version=MTHDS_TEST_VERSION)pipeline = result.pipes["scoring.pipeline"]assert isinstance(pipeline, PipeSequenceBlueprint)assert pipeline.steps[0].pipe == "scoring.compute_score"+ @pytest.mark.parametrize("crate_factory", [_authored_crate, _cross_domain_crate])+ def test_normalized_crate_is_closed_over_its_pipe_refs(self, crate_factory: Callable[[], LibraryCrate]):+ """Every in-body pipe ref of a normalized crate names a pipe the crate actually holds.++ Driven off `pipe_dependencies` rather than a hand-walk of the controller kinds, so a new kind —+ or a new `_qualify_pipe_ref` call site on an existing one — is covered the day it lands. An open+ ref is silent: the crate hashes content nobody can resolve, and a round-trip through+ `load_from_crate` dies with `PipeNotFoundError` at run time instead of at normalization.+ """+ result = normalize_crate(crate_factory(), mthds_version=MTHDS_TEST_VERSION)+ dangling = {+ f"{pipe_ref} -> {dependency}"+ for pipe_ref, blueprint in result.pipes.items()+ for dependency in blueprint.pipe_dependencies+ if dependency not in result.pipes+ }+ assert not dangling++ def test_bare_cross_domain_pipe_refs_resolve_to_the_declaring_domain(self):+ """A bare ref to a sibling domain's pipe is the shape the live `PipeLibrary` resolves.++ `get_optional_pipe` falls back to a search across every domain, so this bundle runs. Prefixing+ the owner domain would name `orchestrator.present_as_markdown`, which does not exist.+ """+ result = normalize_crate(_cross_domain_crate(), mthds_version=MTHDS_TEST_VERSION)+ sequence = result.pipes["orchestrator.run_all"]+ assert isinstance(sequence, PipeSequenceBlueprint)+ assert sequence.steps[0].pipe == "presentation.present_as_markdown"++ parallel = result.pipes["orchestrator.fan_out"]+ assert isinstance(parallel, PipeParallelBlueprint)+ assert [branch.pipe for branch in parallel.branches] == ["presentation.present_as_markdown", "presentation.render_html"]++ condition = result.pipes["orchestrator.route"]+ assert isinstance(condition, PipeConditionBlueprint)+ assert condition.outcomes == {"markdown": "presentation.present_as_markdown"}+ assert condition.default_outcome == "presentation.render_html"++ batch = result.pipes["orchestrator.batch_all"]+ assert isinstance(batch, PipeBatchBlueprint)+ assert batch.branch_pipe_code == "presentation.present_as_markdown"++ def test_special_outcomes_are_left_alone(self):+ """`fail` / `continue` are outcomes, not pipe refs — they must survive un-qualified."""+ result = normalize_crate(_cross_domain_crate(), mthds_version=MTHDS_TEST_VERSION)+ condition = result.pipes["orchestrator.route_to_special"]+ assert isinstance(condition, PipeConditionBlueprint)+ assert condition.outcomes == {"stop": SpecialOutcome.FAIL}+ assert condition.default_outcome == SpecialOutcome.CONTINUE++ def test_ambiguous_bare_pipe_ref_raises(self):+ """Two domains declaring the same code: the live library raises, so normalization raises too.++ Silently preferring the owner domain would produce a crate that resolves while an interpreted+ run of the same library dies on the ambiguity — a new disagreement, not a fix.+ """+ crate = _cross_domain_crate()+ crate.pipes["duplicates.present_as_markdown"] = PipeLLMBlueprint(+ description="A second pipe sharing the callee's code",+ inputs={"data": "Text"},+ output="Text",+ prompt="Present $data",+ )+ crate.domains["duplicates"] = DomainBlueprint(code="duplicates", description="Duplicates domain")++ with pytest.raises(CrateNormalizationError, match="Ambiguous pipe ref 'present_as_markdown'"):+ normalize_crate(crate, mthds_version=MTHDS_TEST_VERSION)++ def test_unresolvable_bare_pipe_ref_raises(self):+ """A bare ref matching no pipe in the crate is surfaced, not silently qualified into a dangling ref."""+ crate = LibraryCrate(+ pipes={+ "orchestrator.run_all": PipeSequenceBlueprint(+ description="Calls a pipe that is not in the crate",+ output="Text",+ steps=[SubPipeBlueprint(pipe="nowhere_to_be_found")],+ ),+ },+ domains={"orchestrator": DomainBlueprint(code="orchestrator", description="Orchestrator domain")},+ )+ with pytest.raises(CrateNormalizationError, match="resolves to no pipe in the crate"):+ normalize_crate(crate, mthds_version=MTHDS_TEST_VERSION)+def test_refinement_with_structured_base_is_flattened(self):"""Refining a concept with a structure adopts that (qualified) structure and drops `refines`."""result = normalize_crate(_authored_crate(), mthds_version=MTHDS_TEST_VERSION)diff --git a/wip/parity/README.md b/wip/parity/README.mdnew file mode 100644index 000000000..a869ff929--- /dev/null+++ b/wip/parity/README.md@@ -0,0 +1,37 @@+# Parity gaps — one authored fact, two readers that disagree++This track fixes a family of defects that share one shape: a single authored fact (a pipe ref, a concept declaration, a model choice, an emitted line of code) is read by two pipelex implementations that disagree about what it means. The **Phase 1 trio** are silent in the strict sense — both readers succeed, produce plausible output, and diverge with no error attached anywhere — which is why they are batched as one sweep rather than fixed opportunistically. The **Phase 2 kernel items fail differently**, and the difference is worth keeping straight: 2.1 was a *contract* gap whose divergence surfaces one step later as a `StuffContentTypeError` on read-back, 2.3 an *expressiveness* gap where one reader cannot state the fact at all, and 2.2 turned out on re-verification not to be a defect.++**Branch:** `fix/Parity-gaps` (this worktree, `_gaps/`). **PR:** [#1085](https://github.com/Pipelex/pipelex/pull/1085) → `dev`. **Plan:** [`parity-gaps-plan.md`](parity-gaps-plan.md).++## The gaps at a glance++| Gap | The two readers | Buildable |+| --- | --- | --- |+| Bare cross-domain pipe refs | `crate_normalization` qualifies with the owner domain; the live `PipeLibrary` searches every domain | now |+| Structureless concept base class | runtime promotes to a `TextContent` refinement; the `python-structures` projection emits `StructuredContent` + `extra="allow"` | now |+| Import-block format stability | `render_import_block` never wraps; a consumer's `ruff format` wraps past 88 columns | now |+| `MethodKernel.llm_text` narrowness | the façade hardcodes concept/class and requires a model; `run_llm_text` and the interpreter take all three | ✅ fixed **in** #1081 |+| `MethodKernel.llm_object` prompting style | the façade derives style from the object setting and drops the `for_text` rung; the interpreter derives style from the text setting | ⚠️ withdrawn — not a live defect |+| Img-gen prompt constructor | `kernel/prompt_references.py` says the kernel resolves image references; only the interpreter-layer blueprint can actually build an `ImgGenPrompt` | ✅ fixed **in** #1082 |++## Phasing++- **Phase 1 — the dev-buildable trio** (crate qualification, structureless base class, import wrapping). All targets exist on this branch today; each lands with its regression gate.+- **Phase 2 — the kernel trio.** Originally gated on the kernel-extraction PRs (#1081 façade, #1082 operators) merging to `dev`, to avoid destabilizing finalized PRs. **Louis overrode that gate:** the two live ones (2.1 and 2.3 — 2.2 was withdrawn) are defects *those PRs introduce*, so deferring meant knowingly merging a package whose stated contract is false and re-opening the same files to repair it. Phase 2 therefore shipped **inside** #1081 and #1082, not on this branch.++## Status++- **Phase 0 (planning): ✅ done 2026-08-03.** Plan written, targets verified on this tree, decisions D-1..D-3 posed with recommendations.+- **Phase 1: ✅ done 2026-08-03.** All three fixes landed with their gates, red-green verified; Checkpoint A recorded in the plan. D-1 settled as crate-wide resolution (not own-domain-first — see [`d1-domain-hint-deferred.md`](d1-domain-hint-deferred.md)); D-2 settled as recommended.+- **Phase 2: ✅ done 2026-08-04, folded into the kernel stack.** 2.1 fixed in #1081; 2.3 fixed in #1082, with the boot-contract arm it turned out to owe in the same PR and the doc review on #1083 — commits named by subject in [`parity-gaps-plan.md`](parity-gaps-plan.md), because the stack is rebased and hashes do not survive it. 2.2 **withdrawn** on re-verification — the façade's single `model` wins the object resolution's first rung, so the style it derives already matches an interpreted run for every call the façade can express; the surviving narrowness is deferred as KF-16, because the whole model-derived-style mechanism is slated to become an explicit user choice (design in `wip/prompting-style/README.md`, which ships on the kernel stack — not on this branch). D-3 settled as recommended, (a), and its predicted layering cost did not materialise.++## Deferred, not dropped++- [`d1-domain-hint-deferred.md`](d1-domain-hint-deferred.md) — whether `PipeLibrary` should prefer the caller's domain on an ambiguous bare pipe ref. A language call; the normalizer follows for free once settled.+- [`structureless-concept-with-registered-class.md`](structureless-concept-with-registered-class.md) — a description-only concept bound to a same-named hand-written class through a channel the crate structurally cannot see. Raised on PR #1085, verified pre-existing; the correct fix is a language call.+- [`deferred-review-observations.md`](deferred-review-observations.md) — four follow-ups from the #1085 finalization review: the cross-package `refines` shape, qualified-ref closedness, a latent test-order flake, and the `mthds/` spec's missing bare-**pipe**-ref rule (a cross-repo edit, ordered *after* D-1).++## Cold start++**Start with [`SESSION-HANDOFF.md`](SESSION-HANDOFF.md)** — where the work stands, what runs next, and the traps. Then read [`parity-gaps-plan.md`](parity-gaps-plan.md) top to bottom — it carries the evidence, the chosen fixes, and the gates. Verify the grounding claims against the tree before building on them (each carries its file:line). Gates for every phase: `make agent-check`, full `make agent-test`, `make drift-check`, and an `[Unreleased]` changelog entry per user-visible fix.diff --git a/wip/parity/SESSION-HANDOFF.md b/wip/parity/SESSION-HANDOFF.mdnew file mode 100644index 000000000..f5a15a464--- /dev/null+++ b/wip/parity/SESSION-HANDOFF.md@@ -0,0 +1,137 @@+# Session handoff — parity gaps, both phases built++**Branch:** `fix/Parity-gaps` in the `_gaps/` worktree, off `dev` at `221b8ee0b`. **PR:** [#1085](https://github.com/Pipelex/pipelex/pull/1085) → `dev`, open.++Read this first, then [`README.md`](README.md) and [`parity-gaps-plan.md`](parity-gaps-plan.md) (Checkpoint A holds the full record of what landed).++## Resume here (round 3 fully answered 2026-08-04, ~11:20 local / 04:20 UTC)++**Everything is pushed. Nothing is half-done.** Round 3 ran as four sub-rounds — 3a through 3d, each one cubic re-reviewing the head the previous fix produced — and all of them are arbitrated and answered. The next action is to read whatever comes back on *these* heads, then merge.++**One consistent labelling scheme, since the doc uses it throughout:** rounds 1–2 were the pre-2026-08-04 passes; **round 3** is this session, split into **3a** (the first cubic pass on the round-2 heads), **3b** (the KF-3 holder fix, which rewrote the whole stack), **3c** (the KF range), and **3d** (the spec divergence and the stale-status fixes). Anything below labelled "rounds 3–4" belongs to the *older* #1085-only numbering and is left as written, because it is a historical record of that PR's own cycle.++| PR | Branch / worktree | Round 3 |+| --- | --- | --- |+| [#1081](https://github.com/Pipelex/pipelex/pull/1081) (1/3) | `refactor/Kernel` — `_kernel/` | KF-3's wrong holder fixed (3b); cubic clean on head |+| [#1082](https://github.com/Pipelex/pipelex/pull/1082) (2/3) | `refactor/Kernel-phase2` — `_kernel/` | dead-hash fix (3a); rebased (3b); cubic clean on head |+| [#1083](https://github.com/Pipelex/pipelex/pull/1083) (3/3) | `refactor/Kernel-phase3` — `_kernel/` | fixes in 3a/3b/3c/3d; rebased twice; cubic clean on head |+| [#1085](https://github.com/Pipelex/pipelex/pull/1085) | `fix/Parity-gaps` — `_gaps/` | fixes in 3a/3d; **3d's answer is the one to read**; re-pinged |++**Read heads live** — `for p in 1081 1082 1083 1085; do gh pr view $p --json number,headRefOid; done`. Deliberately not written down: the kernel branches are rebased whenever a parent moves, so any head recorded here is wrong within one push.++**What round 3 produced, in order of what matters.**++1. **A real spec divergence, found in 3d and *not* fixed.** The MTHDS spec says bare pipe references "do NOT fall through to other domains"; the runtime searches across domains and Phase 1's 1.1 mirrored it. Both readers agree with each other and both disagree with the standard — in *two* rows, and in opposite directions. Written up in [`bare-pipe-ref-spec-divergence.md`](bare-pipe-ref-spec-divergence.md); it subsumes D-1, so settle the two together and the spec question first.+2. **KF-3 was pointed at the wrong operator (3b), and the argument rested on it.** `pipe_structure.py` has never referenced `LLMPromptBlueprint` — on any branch, including `dev`. `PipeLLM` holds it as `llm_prompt_spec`. The corrected reading changes the item's shape: **what is production-dead is `make_llm_prompt`, one method — not the model**, which stays. Fixed at the source, hence the stack rewrite.+3. **Round 2's arbitration had a hole (3a).** Recorded below under the rejected findings — the general lesson is there and it is the one worth carrying forward.+4. **Three self-staling constructs were converted, not patched**: commit hashes → subjects, a `KF-1..KF-15` range → the file, and a hardcoded helper count → no count. Each had already gone stale at least once; two produced review findings this round.++Backups to roll back to: `backup/parity-pre-round3` and `backup/kernel-{,phase2-,phase3-}pre-round3b`.++**Greptile and Codex have gone quiet, and the dates are the honest way to say it** — neither has posted anything since **2026-08-02** on the three kernel PRs and **2026-08-03** on #1085, across every push and every ping since. (Dates rather than a round count: the count needs editing on each new round and was already inconsistent with itself inside this file.) Cubic is the only live external signal. Before reading "the bots are clean" as coverage, decide deliberately whether to chase them — the finalization review already demonstrated once that a defect can survive every bot plus a cold `/code-review`.++**This table deliberately carries no head SHAs.** It used to, and they were wrong within one push — the three kernel branches are rebased whenever a parent moves, and every branch moves when a review round lands. Read the live heads instead: `for p in 1081 1082 1083 1085; do gh pr view $p --json number,headRefOid; done`.++**What round 2 produced** (a record of that round, not the current state — see the dates above). Cubic came back on all four heads; **Greptile and Codex did not, despite pings** — cubic was already the only fresh external signal by then, so treat "the bots are happy" with corresponding caution. Its findings were mostly doc-accuracy, and the arbitration is recorded in a comment on each PR. The substantive ones:++- **#1081 — the one that mattered.** `wip/prompting-style/README.md` described the prompting style as derived "in two places that must agree" and rested its first argument on exactly that. False since the extraction: `pipe_llm.py:223` calls the kernel's `derive_templating_style`, so there is one site. Left alone, KF-16's design basis would have carried a false premise.+- **#1082.** `ImgGenPromptBlueprint`'s class docstring still advertised the four things it had just delegated to the kernel, contradicting its own module docstring.+- **Deferred as KF-17**, not fixed: the resolvers select the model choice by truthiness, so `model=""` falls through to the deck default. The chain is verbatim from `dev`'s `pipe_llm.py`, the interpreter calls these same functions so no divergence exists, and pydantic already rejects `""` on every authored path. The honest fix is `LLMModelChoice` rejecting `""` at the type boundary once — a deliberate pass, not a spot edit.++**One finding was rejected on verification — do not re-litigate it:** "the changelog gate conflicts with the Phase 2 record" — the line already reads "per user-visible fix", which is the scoping the finding asked for.++**And one rejection was itself half-wrong, which is the lesson worth keeping.** Round 2 reported that "`model` was already optional on `llm_text`". Checked against the code that was false — pre-fix the façade declared `model: LLMModelChoice` with no default — so it was rejected and the round moved on. Round 3 came back with the same subject from the other end: §2.1's own "✅ Done" record *asserted* `model` was already optional. The bot had been echoing this plan, not misreading the code, and rejecting the code claim left the false sentence sitting in the doc to be re-reported. **When a finding is wrong about the code, check whether it is right about a doc that says so** — a claim has to be traced to its source before "verified false" means anything.++**Next actions, in order:**++1. Read the bot feedback on all four — **review bodies and inline comments, not check states** (see the cubic trap below). Greptile posts as *issue comments*; cubic posts *reviews* + issue comments; Codex posts reviews. If Greptile and Codex stay silent again, decide whether to chase them or proceed on cubic alone.+2. Fan out an **Opus 5** sub-agent to dedupe, verify each item, and arbitrate — fixing **only clear wins**, no over-engineering, no guarding impossible scenarios. Anything doubtful becomes a `.md` in `wip/parity/` (or `wip/kernel/` for stack items). Tell the sub-agent it is **STRICTLY READ-ONLY**.+3. Merge in order **#1081 → #1082 → #1083**. #1085 is independent of the stack and can merge whenever Louis chooses.++**A trap this round taught, now written into the kernel plan:** a doc that lives *on* a stacked branch must name commits **by subject, never by SHA**. Every rebase of the parent rewrites every hash on the child, so the SHAs are dead before anyone reads them — including the "corrected" ones a reviewer suggests. Cubic's proposed replacements were themselves already stale when the fix was written.++**Local gates.** `make agent-check` and `make drift-check` green on every branch. Full `make agent-test` green on `refactor/Kernel-phase3`, which is the stack tip and therefore carries all three PRs' content — the only source change in this round is a docstring, so it was not re-run on each branch separately. CI green on all four. Nothing is owed locally.++Each rebase in this round was verified the same way: tag the tip, rebase with `--onto`, then `git diff <pre-rebase-tag>..<new-tip>` and confirm it shows **exactly** the intended change and nothing else. Both rebases conflicted in one file only — the plan's status block, which both sides genuinely edited. That is the legitimate shape; an unexpected conflicted *source* file means the command was wrong, and the right move is to abort rather than resolve.++Backup tags on `origin`, in case a rebase needs undoing: `backup/kernel-pre-fold`, `backup/kernel-phase2-pre-fold`, `backup/kernel-phase3-pre-fold`, `backup/kernel-phase3-pre-2.3`, `backup/kernel-phase3-pre-resplit`, plus this round's `backup/kernel-pre-botround2`, `backup/kernel-phase2-pre-botround2`, `backup/kernel-phase3-pre-botround2`.++## Where the work stands++**Phase 1 is complete and committed.** All three dev-buildable fixes landed with their gates, each red-green verified (gate written first, run against the unfixed tree, observed to fail for the stated reason). Nothing is left to build in Phase 1.++Gates all run and green at `d9efffbfb`:++- `make agent-check` ✅+- full `make agent-test` ✅ (~19 min)+- `make drift-check` ✅ — re-run **after** `git add`, because it reads the git index, not the working tree+- real-world confirmation of 1.1 against `../pipelex-cookbook/examples/wip/advisory_board/`: the dangling ref `advisory_orchestrator.master_advisory_orchestrator -> advisory_orchestrator.present_as_markdown` is gone+- `[Unreleased]` changelog entry for each of the three fixes++**Phase 2 is built too — but not on this branch.** The plan gated it on #1081/#1082 merging; Louis overrode that gate, because each of these is a defect *those PRs introduce*, so deferring meant merging a package whose stated contract is false and then re-opening the same files to repair it. Phase 2 therefore shipped **inside** the kernel stack, in the `_kernel/` worktree:++| Gap | Outcome | Commit (by subject — the stack is rebased, so hashes die) |+| --- | --- | --- |+| 2.1 `llm_text` narrowness | fixed | *make llm_text as wide as the op beneath it* — `refactor/Kernel` (#1081) |+| 2.2 `llm_object` prompting style | **withdrawn** — not a live defect | silenced by the KF-16 numbering commit on #1081, which added the `llm_object` docstring recording why its single `model` is correct (its subject line is truncated in git — *"…to keep the stack's"* — so match on `KF-16`); deferred as KF-16 |+| 2.3 kernel cannot build an `ImgGenPrompt` | fixed | *let a runtime-only caller build the image prompt it has to pass in* — `refactor/Kernel-phase2` (#1082) |+| — its boot-contract arm | added | *give the image prompt assembler the boot-contract arm the rule owes it* — `refactor/Kernel-phase2` (#1082), same PR as the entry point |+| — the kernel doc's drift review | recorded | *record the image prompt assembler on the page that specifies the kernel* — `refactor/Kernel-phase3` (#1083), where the doc and its contract live |++**Nothing about Phase 2 lands on `fix/Parity-gaps`.** This branch's only Phase 2 artifact is the plan/README/handoff record you are reading.++## The goal this session was executing++Verbatim, so the new session picks up the same contract:++> execute `wip/parity/parity-gaps-plan.md` until it's all done. But do stop at each required checkpoint and, after recording the progress and running `make check` and `make test`, stage the changes and fan out a **Sonnet-5** sub-agent to run the skill `/code-review`, then sort through the findings with a strict "no over-engineering" judgement.+>+> Then stack the PRs on `dev`, poll CI and the PR review bots, then fan out an **Opus 5** sub-agent to check their feedback, deduplicate, verify each item, arbitrate to solve it ONLY if it's a clear win (avoid over-engineering, don't guard impossible scenarios). Resolve the threads, push the fixes, rinse and repeat. Pinging `@greptileai` and `@cubic-dev-ai` in a comment re-triggers them.+>+> When the PR review bots are all happy, fan out an **Opus 5** sub-agent to run gstack's `/review wip/parity/parity-gaps-plan.md` and finalize the PR.+>+> **No over-engineering.** In case of any doubt, defer the issue for later as a `.md` in `./wip/parity/` — include that instruction in every sub-agent's context.+>+> **Fan-out convention for `/code-review`:** spawn each review sub-agent with **no inherited context** — hand it only a pointer to the changes under review (the phase's commit SHA, `git diff <base>..HEAD`, or the working-tree files), never the plan, the rationale, or your own conclusions.++## Next actions, in order++1. ~~Checkpoint A's code review (Sonnet-5 `/code-review`, no inherited context, pointed at `d9efffbfb`).~~ Fanned out.+2. ~~Open the PR.~~ [#1085](https://github.com/Pipelex/pipelex/pull/1085).+3. ~~Poll CI + the review bots, fan out an Opus 5 sub-agent over their feedback.~~ Done — CI green, Greptile 5/5 clean, cubic clean, Sonnet-5 `/code-review` clean, one Codex P2 verified pre-existing and deferred. Both bot threads answered inline and resolved.+4. ~~Finalize with gstack's `/review`.~~ Done — verdict **land**. It caught one real defect the other four missed (the `Anything` import omission) plus three doc inaccuracies, all fixed; the rest deferred to [`deferred-review-observations.md`](deferred-review-observations.md).+5. ~~Rounds 3–4: cubic follow-ups.~~ Done. Round 3 = a fair catch on a stale code comment (fixed, `c4788e47b`); round 4 = a P1 duplicate of Codex's round-1 P2, answered with measurements (`e0f2d2362`).+6. **Merge decision is Louis'.** Every prescribed review step in the goal has run. That was the state **as of `e0f2d2362`, before round 3** — 23 checks success, 6 skipped, 0 failures, 0 unresolved threads. It is a record of that moment, not of the current head: round 3 (3a–3d) has landed since. The top of this file is the current summary, and heads are read live rather than written down. Check the head before reading "nothing is pending".+7. ~~**Phase 2** only after #1081 / #1082 merge.~~ Superseded — folded into the kernel stack instead (see above). What remains there is the stack's own review cycle: push the three branches, poll CI and the bots, and merge in order #1081 → #1082 → #1083.++**One open question wants a human ruling** (it does not block Phase 1): whether a hand-written class should have to be *named* in the `.mthds` to bind, or whether the bare-name auto-detect at `concept_factory.py:389` keeps working. Two independent reviewers converged on it — see [`structureless-concept-with-registered-class.md`](structureless-concept-with-registered-class.md), which now carries the measurements.++## What changed, file by file++| File | What |+| --- | --- |+| `pipelex/libraries/crate_normalization.py` | `_index_pipe_refs_by_code` + `_qualify_pipe_ref` resolving crate-wide, raising on ambiguity/absence |+| `pipelex/libraries/exceptions.py` | `CrateNormalizationError` docstring names the new raise reasons (the docstring is the generator source for `docs/errors/crate-normalization-error.md`; it must fit 150 columns on one line) |+| `pipelex/codegen/emitters/python_structures.py` | `_base_class` emits `TextContent` for a structureless concept, root base for a Python-class-backed one; `_native_class` routes the `Anything` fallback through `_structured_content` so the root base it names is actually imported |+| `pipelex/codegen/emitters/python_common.py` | `_import_statement` pre-explodes past `PY_EXPLODE_WIDTH` |+| `tests/unit/pipelex/libraries/test_crate_normalization.py` | `_cross_domain_crate` fixture + closedness / cross-domain / special-outcome / ambiguity / absence tests |+| `tests/unit/pipelex/codegen/test_projection_agrees_with_runtime_base.py` | new — the four-shape ancestor-agreement gate |+| `tests/unit/pipelex/codegen/conftest.py`, `test_python_structures_emitter.py` | `refines_anything_crate` fixture + the load-the-module gate for the `Anything` fallback |+| `tests/unit/pipelex/codegen/test_emitted_artifacts_are_lint_clean.py` | `_assert_ruff_clean` split so the new import-block format-stability test reuses the format half |+| `docs/under-the-hood/codegen-projections.md`, `docs/errors/crate-normalization-error.md` | doc updates (the errors page is generated — regenerate with `make generate-error-pages`, never hand-edit) |+| `CHANGELOG.md` | three `[Unreleased]` → `Fixed` entries |+| `wip/parity/*` | plan (Checkpoint A recorded), README status, D-1 deferral note, this file |++## Things worth knowing before touching this again++- **The D-1 deviation is deliberate and argued.** The plan recommended "own domain first, then crate-wide"; what landed is crate-wide only. `PipeLibrary.get_optional_pipe` does *not* prefer the caller's domain — it raises on an ambiguous bare code — so the own-domain rung would have created a fresh disagreement in the exact case the two resolutions differ. Full table in [`d1-domain-hint-deferred.md`](d1-domain-hint-deferred.md). If a reviewer pushes back, that doc is the answer, not a reason to change the code.+- **⚠ Crate-wide resolution is a *runtime-parity* choice, not the spec's rule — and the MTHDS standard says the opposite.** The spec sentence is *"Bare pipe references do NOT fall through to other domains or other packages"*, from § *Resolution Order for Bare Pipe References* in the sibling **`mthds/` repo** at the workspace root — **not in this repo**, so it will not resolve from a `pipelex` checkout. The runtime's `get_optional_pipe` searches across domains anyway, and 1.1 mirrored it on purpose. The two readers now agree with each other and **both disagree with the standard, in two places and in opposite directions** — looser on one row, stricter on the other. That is the right stopping point for a parity change and the wrong place to leave the story untold — written up with the full table in [`bare-pipe-ref-spec-divergence.md`](bare-pipe-ref-spec-divergence.md), which also explains why it subsumes D-1. Do not "fix" either reader alone.+- **One plan grounding claim was stale** and is corrected in Checkpoint A: the lint-clean test already ran `ruff format --check`, parametrized down to line-length 88. The real hole was coverage — no crate shape produces an import line past 88, since each native content class lives in its own module — so 1.3's gate is a direct test on `render_import_block`, not another crate fixture.+- **`extra="allow"` stayed** on the structureless arm, per the plan, and it is still load-bearing — an earlier version of this note claimed otherwise and was wrong. On a `TextContent` base it preserves every field *alongside* `text`; drop it and pydantic's default `extra="ignore"` strips them silently. What it does not do is let the promoted arm accept an object payload carrying no `text`, because that field is required — and the runtime's class for the same declaration refuses that payload too, which is the agreement 1.2 exists to hold. Verify before re-arguing this: `class X(TextContent): model_config = ConfigDict(extra="allow")` keeps `{"text": …, "source": …}` whole; without it you get `{"text": …}`.+- **The keyword-only hook fires on every edit** under `pipelex/`. Record a subject grant *before* running checks if a subject should stay positional — `make agent-check` runs the auto-fixer, which will silently keyword-only an ungranted one.+- **A green cubic *check* does not mean cubic found nothing.** Its check bucket went `pass` on the same run whose review body said "1 issue found". Read `gh api repos/.../pulls/N/reviews` bodies, not just `gh pr checks`. (Cubic also *edits* an earlier review body in place once threads are resolved, so re-fetching an old review can show a different summary than when it was posted.)+- **The local clock in this worktree is UTC+7.** Filtering the GitHub API with a local-time string silently returns nothing and looks exactly like "the bots have not replied". Always build timestamp filters from `date -u`.+- **`make agent-test` takes ~19 minutes.** Run it in the background and use the targeted paths from `tests/CLAUDE.md` while iterating.+- **Stacked rebases need `--onto`.** Folding Phase 2 into the stack meant rebasing #1082 and #1083 onto rewritten parents. `git rebase <new-base> <branch>` replays the branch's *whole* history including the parent commits already rewritten, which conflicts in every file the parent touched; `git rebase --onto <new-base> <old-base> <branch>` is the correct form, with `<old-base>` the parent tip the branch was actually built on. Verify each one by `git diff <pre-rebase-tag>..<new-tip>` — it must show *exactly* the folded change and nothing else.+- **Never redirect a `git` command's output to `/dev/null`.** `git stash push -- <paths>` refuses when the paths include an untracked file; with the error silenced, the following `git stash pop` applied a *different, unrelated* stash into the tree. It survived only because a conflicted pop does not drop the stash. Use `mv` to set a file aside for a red-green check — `git stash` is the wrong tool whenever untracked files are involved.+- **A fixture that already registers an import cannot gate a missing one.** The `Anything` defect was invisible to `every_type_kind_crate` because its structured concepts import `StructuredContent` anyway. Gating an *omission* needs a crate where nothing else supplies the thing — hence the single-concept `refines_anything_crate`. Worth remembering before adding a shape to a rich fixture and calling it covered.diff --git a/wip/parity/bare-pipe-ref-spec-divergence.md b/wip/parity/bare-pipe-ref-spec-divergence.mdnew file mode 100644index 000000000..0005511c4--- /dev/null+++ b/wip/parity/bare-pipe-ref-spec-divergence.md@@ -0,0 +1,50 @@+# Bare pipe references resolve across domains, and the MTHDS spec says they must not++**Status:** deferred, deliberately. Surfaced by a review bot against the D-1 note on PR #1085; verified and recorded here rather than fixed. This is a **language decision with cross-repo consequences**, not a parity fix.++## The divergence++The MTHDS standard is explicit. **The spec is not in this repository** — it lives in the sibling `mthds/` repo at the workspace root (the open standard, published to mthds.ai), so a fresh `pipelex` clone cannot resolve the path below. The normative sentences are quoted verbatim here so the claim stays checkable without it; to re-verify against the source, read `docs/spec/namespace-resolution.md` § *Resolution Order for Bare Pipe References* in `mthds/`.++> 1. **Current bundle** — check pipes declared in the same `.mthds` file.+> 2. **Same domain, other bundles** — if the bundle is part of a package, check pipes in other bundles that declare the same domain.+> 3. **Error** — if not found, the reference is invalid.+>+> Bare pipe references do NOT fall through to other domains or other packages.++The runtime does fall through. `PipeLibrary.get_optional_pipe` (`pipelex/libraries/pipe/pipe_library.py`), step 3, is commented *"Bare code fallback — search across domains"* and matches on `val.code == pipe_code` across every non-cross-package entry, ignoring the caller's domain entirely:++| bare ref `foo` used from domain `A` | spec | runtime today | |+| --- | --- | --- | --- |+| only `A.foo` exists | `A.foo` | `A.foo` | agree |+| only `B.foo` exists | **error** — no fall-through | `B.foo` | **differ** |+| both exist | `A.foo` — found in the own domain at step 1/2, so the search stops before `B` | **raises** `PipeLibraryError` (ambiguous) | **differ** |+| neither | error | `None` → `PipeNotFoundError` | agree |++**Two rows diverge, and both diverge in outcome.** Row 2 is the fall-through the spec forbids. Row 3 is subtler and easy to get backwards: the spec never reaches an ambiguity at all, because it finds `A.foo` in the caller's own domain and stops — so where the spec quietly succeeds, the runtime raises. A working bundle under the spec is a hard error today.++Row 3 also settles the relationship to D-1: the answer the spec gives there — prefer the own domain — is exactly what a `domain_hint` would produce. Spec compliance therefore delivers D-1's outcome for the ambiguous case *and* closes row 2, which `domain_hint` alone would leave open.++## Why the parity fix mirrored the runtime rather than the spec++Deliberately, and it would be a mistake to "correct" it in isolation. Phase 1's 1.1 made the crate normalizer resolve bare pipe refs the way `get_optional_pipe` does — crate-wide — because the track's entire purpose is that two readers of one authored fact agree. Making the *normalizer* spec-compliant while the runtime stayed permissive would have manufactured a fresh disagreement of exactly the kind being removed: a library that normalizes to an error but runs fine, or vice versa.++So the landed state is coherent: **both readers agree with each other, and both disagree with the spec in the same two places.** That is a strictly better position than where the track started, and it is the correct stopping point for a parity change.++Note the disagreement does not run one way. On row 2 the runtime is *more permissive* than the spec (it resolves where the spec errors); on row 3 it is *stricter* (it raises where the spec quietly returns `A.foo`). "The runtime is looser than the standard" is the tempting one-line summary and it is wrong — worth knowing before anyone reasons about which direction a fix travels.++See [`d1-domain-hint-deferred.md`](d1-domain-hint-deferred.md) for the narrower question this raises (whether `get_optional_pipe` should grow a `domain_hint` and prefer the caller's domain on ambiguity). **Spec compliance subsumes it**: if bare refs never leave their own domain, cross-domain ambiguity cannot arise, and row 3 resolves to `A.foo` — the same answer `domain_hint` was invented to produce. Settle them together, and settle the spec question first, because it decides whether `domain_hint` has anything left to do.++## Why this is not being fixed here++Three reasons, and the third is decisive.++1. **It is a behavior change to the language, not to a reader.** Every `.mthds` in the wild that leans on cross-domain bare resolution stops loading. That is the opposite of a zero-behavior-change parity fix.+2. **It is cross-repo.** The rule lives in `mthds/` (the standard) and is implemented in `pipelex/`; `conformance/` would need the case, and any bundle in `pipelex-cookbook/`, the demo repos or the hub that relies on the fall-through would need qualifying. A one-line resolver change does not scope it.+3. **The direction of the fix is genuinely open, and it is not this track's call.** Either the runtime tightens to the spec, or the spec loosens to describe what every implementation actually does. The second is not obviously wrong — cross-domain bare resolution inside a single crate is convenient and has evidently been relied on. Picking one is a language-owner decision.++## What to do with it++Raise it with whoever owns the MTHDS spec, together with D-1. Whichever way it settles, the change lands in one PR carrying: the resolver, the crate normalizer (they must move together — that is this track's whole lesson), the spec page, a `conformance/` case, and the sweep of any bundle that relied on the old behavior.++**Do not fix one reader and not the other.** That is the defect class this entire track exists to remove, and it would be an unusually ironic way to reintroduce it.diff --git a/wip/parity/d1-domain-hint-deferred.md b/wip/parity/d1-domain-hint-deferred.mdnew file mode 100644index 000000000..d5d75278b--- /dev/null+++ b/wip/parity/d1-domain-hint-deferred.md@@ -0,0 +1,36 @@+# D-1 follow-up — the `domain_hint` conversation, deferred++**Status:** deferred, deliberately. Raised at 1.1 (Phase 1), not taken.++## What 1.1 actually landed, and how it differs from the plan's recommendation++The plan's D-1 recommendation (a) reads: *"Qualify against the crate, the way the library resolves: **own domain first**, then a crate-wide unique match, raising `CrateNormalizationError` on ambiguity or absence."*++What landed drops the "own domain first" rung: `_qualify_pipe_ref` resolves a bare pipe ref by a **crate-wide** search only — unique match wins, ambiguity raises, absence raises.++The two differ in exactly one case, and in that case "own domain first" would have *created* a new disagreement rather than closing one:++| bare ref `foo` from domain `A` | live `PipeLibrary.get_optional_pipe` | own-domain-first | crate-wide (landed) |+| --- | --- | --- | --- |+| only `A.foo` exists | `A.foo` | `A.foo` | `A.foo` |+| only `B.foo` exists | `B.foo` | `B.foo` | `B.foo` |+| both `A.foo` and `B.foo` exist | **raises** `PipeLibraryError` (ambiguous) | `A.foo` ✗ | **raises** `CrateNormalizationError` |+| neither exists | `None` → `PipeNotFoundError` at run | raises | raises |++`get_optional_pipe`'s bare-code fallback (`pipelex/libraries/pipe/pipe_library.py`) searches every non-cross-package entry and raises on more than one match — it does **not** prefer the caller's domain. Its own `TODO` says so in as many words: *"add domain_hint parameter so controllers can prefer their own domain when bare code is ambiguous."* The preference is a thing the runtime is asked to grow, not a thing it has.++So preferring the owner domain in the normalizer would mean a library whose bare ref is ambiguous normalizes happily into a crate that resolves, while an interpreted run of that same library dies on the ambiguity. That is the shape of defect this whole track exists to remove. The plan's own standing rule — *"each fix lands with the test that pins the agreement"* — settles it: mirror the reader that ships.++## The conversation that is deferred++Whether `PipeLibrary.get_optional_pipe` should grow a `domain_hint` and prefer the caller's domain on an ambiguous bare code. This is a **language/runtime decision** (it changes which pipe a runnable bundle calls), not a normalization one, and it belongs with the same person who owns D-1 option (b).++If the answer is yes, the normalizer follows in the same change: `_qualify_pipe_ref` already has `owner_domain` in hand — it is the caller that most obviously knows the hint — and adding the preference rung there is a two-line change plus the ambiguity test flipping from "raises" to "picks own domain". Until then, the two readers agree, which is the property worth holding.++## Why the strictness costs nothing today++An ambiguous or dangling bare pipe ref cannot survive in a library that loads. `Library.validate_library` (`pipelex/libraries/library.py:128`) gates on `isinstance(pipe, PipeController)` — so **every** controller kind, Sequence, Condition, Parallel and Batch alike, not just the two that motivated the check — and puts each of its `pipe_dependencies()` through `get_required_pipe`. An unresolvable one surfaces as a structured `LibraryLoadingError(UNRESOLVED_PIPE_DEPENDENCY)` naming both the referencing controller and the missing ref.++Note the ordering, because it is the opposite of what "validation gates the crate" would suggest: the crate is **built first** (`LibraryCrateFactory.make_from_blueprints`, `library_manager.py:528`) and validated after, inside `load_from_crate` (`:492`). What runs after validation is *normalization* — `get_crate` → `normalize_crate` — and that is the step this fix guards. So the guarantee is not "validation runs first"; it is that no library reaching a normalized crate through the loader has an unresolvable controller dependency, because `validate_library` has already rejected it with a better error.++Raising in the normalizer therefore changes no behavior for any bundle that loads — it only turns a hand-built or transported crate's silent dangling ref into a diagnostic, on the path where nothing else would catch it.diff --git a/wip/parity/deferred-review-observations.md b/wip/parity/deferred-review-observations.mdnew file mode 100644index 000000000..864737e3b--- /dev/null+++ b/wip/parity/deferred-review-observations.md@@ -0,0 +1,37 @@+# Deferred — observations from the PR #1085 finalization review++Raised by the gstack `/review` pass over Phase 1 (2026-08-03), verified against the tree at `c36feb85b`. Each is real. None is a clear win as a code change *in this PR*, so each is recorded here rather than fixed under the standing "no over-engineering, defer when in doubt" bar. The review's verdict was **land**; these are follow-ups, not blockers.++The two findings that *were* clear wins — the `Anything` import omission and the doc/citation corrections — landed in this PR and are recorded in the plan's Checkpoint A instead.++## 1. A cross-package `refines` target is a fifth structureless shape, ungated on both sides++`test_projection_agrees_with_runtime_base.py` enumerates four authorable concept shapes and claims the enumeration is closed. It is closed only over concepts the crate can resolve on its own. A concept whose `refines` target is absent from the crate — a cross-package base — gets `structureless=True` **with** `base_ref` set (`pipelex/codegen/resolved_concepts.py:127-131`, the `elif value.refines:` branch), so it skips 1.2's new structureless arm and falls to `python_structures.py:89-90`, landing on `StructuredContent`.++That is the same choice this PR just ruled wrong for the in-crate structureless shape, taken in the sibling branch, untested on both sides.++**Why it is not fixed here.** It sits on the same B1-1 floor as the Python-class-backed case: the base is *genuinely* not visible to the crate, so there is nothing to agree with — the runtime resolves it through a package the crate does not carry. Promoting it to `TextContent` would be a guess, and keeping the root base is the honest answer to "we cannot see this". The projection is not silent about it either: that same branch sets `imprecision_reason = "refinement base '<ref>' is not available in this crate"`, which the emitter writes into the class docstring. So the reader is told, which is the bar the "surfacing imprecision, never guessing" rule actually sets. The test docstring has been scoped to say so rather than overclaim closure.++**What would settle it.** Decide whether a cross-package base should be resolved at normalization time (which would make the crate non-self-contained, a contract question) or stay deferred. Until then the root base is correct-by-honesty, not by accident. Related: [`structureless-concept-with-registered-class.md`](structureless-concept-with-registered-class.md), which is the same "the crate cannot see it" floor reached by a different channel.++## 2. Crate closedness is fixture-true, not property-true++`_qualify_pipe_ref` returns any ref containing a `.` unchanged, with **no existence check**. So a typo'd *qualified* ref (`presentation.nope`) still produces a dangling ref in the normalized crate and a run-time `PipeNotFoundError` — exactly what the closedness test's docstring warns against. The test passes because its fixtures contain no such typo.++**Why it is not fixed here.** Pre-existing, and out of Phase 1's scope: 1.1 was about *bare* refs, which had no resolution at all. Adding existence validation for qualified refs is a separate behavioural change with its own blast radius — a hand-built or transported crate legitimately carries refs to pipes it does not hold yet in some load orders, and the loader's `validate_library` already rejects the reachable cases with a better, structured error.++**Trap for whoever extends the test.** The closedness docstring invites adding a cross-package fixture to its `parametrize`. Doing that will fail the test for a legitimate reason — cross-package deps are deliberately deferred, not resolved — unless the comprehension filters them out, mirroring `_index_pipe_refs_by_code`'s own `has_cross_package_prefix` skip.++## 3. Generic concept codes in the parity test are a latent test-order flake++`test_projection_agrees_with_runtime_base.py` names its four shapes `Opaque`, `Structured`, `Refined`, `RefinesNative`. `ConceptFactory._handle_basic_blueprint` looks the **bare** code up in the process-global class registry before falling back to the generated `TextContent`-based class. No collision exists today, but any future `class Opaque(StructuredContent)` registered anywhere in the suite would silently flip `test_structureless_lands_on_text_content` — the test would fail, or worse pass, for a reason unrelated to what it gates.++**Why it is not fixed here.** Speculative today; the fix (namespacing the codes) is free but guards a scenario that does not exist. Worth doing the next time that file is touched for another reason. Recorded so the failure is diagnosable in one read if it ever happens, rather than costing an afternoon.++## 4. Cross-repo — the `mthds/` spec states the bare-ref rule for concepts only++`mthds/docs/spec/library-crate.md` §2 describes bare-reference resolution purely in *concept* terms ("resolve to `domain_path.ConceptCode`"). The bare-**pipe**-ref rule this PR settled — resolve crate-wide, unique match required, error on ambiguity or absence — is unstated there, and that file is the one `crate_normalization.py`'s own header cites as `[Library Crate Format]`.++So the implementation now has a documented behaviour its governing spec does not describe. That is a spec edit in the `mthds/` repo, not this one.++**Note the ordering dependency:** if D-1's `domain_hint` question is answered "yes, prefer the caller's domain" (see [`d1-domain-hint-deferred.md`](d1-domain-hint-deferred.md)), the rule changes and the spec text would have to be written twice. Settle D-1 first, then write the spec paragraph once.diff --git a/wip/parity/parity-gaps-plan.md b/wip/parity/parity-gaps-plan.mdnew file mode 100644index 000000000..61c4f80e9--- /dev/null+++ b/wip/parity/parity-gaps-plan.md@@ -0,0 +1,151 @@+# Parity-gaps plan — fix the places where two readers disagree about one authored fact++**Thesis.** pipelex has several surfaces that each claim to be a faithful reading of the same authored source: the live library vs the normalized crate, the runtime concept factory vs the `python-structures` projection, the emitted bytes vs the consumer's formatter, and (once the kernel extraction merges) the `MethodKernel` façade vs the ops the interpreter actually calls. Where two of them disagree, the result is a *wrong value with no error attached* — both sides succeed, and only downstream behavior reveals the split. This plan fixes every such disagreement currently known, each with the regression gate that would have caught it.++**Standing rules for this branch:**++- Every claim below carries its instrument (file:line or a repro) and its date. Re-verify before building on any of them — code moves.+- Each fix lands with the test that pins the *agreement*, not just the fixed behavior: the natural gate is "both readers, same answer", asserted mechanically.+- No backward compatibility (repo principle): where a fix changes observable behavior, it changes it, with a changelog entry — no deprecation shims.+- Changelog entries go under `[Unreleased]` on this work branch; a versioned heading only at release time.++## Grounding — verified on this tree (`fix/Parity-gaps` at `221b8ee0b`), 2026-08-03++- `pipelex/libraries/crate_normalization.py` — `_qualify_pipe_ref` applied to sequence steps, condition branches, and outcomes (lines 232–239); `CrateNormalizationError` already exists (`pipelex/libraries/exceptions.py`), so the fix needs no new error class.+- `pipelex/codegen/emitters/python_structures.py:59–63` — the structureless arm emits `ConfigDict(extra="allow")` on a `StructuredContent` base; `pipelex/core/concepts/concept_factory.py:376` — `_handle_basic_blueprint` is the runtime's opposite reading.+- `pipelex/codegen/emitters/python_common.py:255` — `render_import_block`, no wrapping at any width; `PY_EXPLODE_WIDTH = 88` at line 24; the lint-clean gate is `tests/unit/pipelex/codegen/test_emitted_artifacts_are_lint_clean.py` and it runs `ruff check`, not `ruff format --diff`.+- `pipelex/pipe_operators/llm/pipe_llm.py:219–237` — the interpreter resolves `llm_setting_main` (text) and `llm_setting_for_object` separately and derives `templating_style` from `llm_setting_main`.+- `pipelex/kernel/` does **not** exist on this branch. The façade and operator ops arrive with the open kernel-extraction PRs: #1081 (package + LLM slice), #1082 (remaining operators), #1083 (memory boundary). Phase 2 claims are written against those PRs' code and must be re-verified on the merged result. — **Superseded:** Phase 2 shipped inside those PRs instead, so it was re-verified and built on their branches; nothing in Phase 2 lands on this one.++## Phase 1 — the dev-buildable trio++### 1.1 Crate normalization mis-qualifies a bare cross-domain pipe ref++**The defect.** `_qualify_pipe_ref` prefixes a bare pipe ref with the **owner domain** unconditionally, while the live `PipeLibrary.get_optional_pipe` resolves a bare code by **searching every domain** (its bare-code fallback is gated on `"." not in pipe_code`). So a bundle calling a sibling domain's pipe runs correctly through the interpreter and normalizes into a crate whose step names a pipe that does not exist.++**Repro** (measured 2026-08-03): load `pipelex-cookbook/examples/wip/advisory_board/` (domain `advisory_orchestrator` calling `present_as_markdown`, declared in domain `presentation`), normalize, and query both sides: the normalized step ref `advisory_orchestrator.present_as_markdown` is absent from `crate.pipes`, while the live lookup of the bare code succeeds. The library loads and validates green; the crate is silently wrong.++**Why it matters.** (1) The normalized fingerprint hashes the wrong content — two libraries differing only in which domain declares the callee hash as if the caller pointed at its own domain. (2) A crate round-trip breaks the method at run time: decode → `load_from_crate` reinstates the *dotted* ref, which never reaches the bare-code fallback, so the run dies with `PipeNotFoundError`. That is the transport path (sandbox-hosted load, distributed execution payloads), not a hypothetical. (3) Every consumer of `crate.pipes` inherits it, including the per-pipe codegen kinds the spec reserves.++**The fix — decision D-1, recommendation (a).** Qualify against the crate, the way the library resolves: own domain first, then a crate-wide unique match, raising `CrateNormalizationError` on ambiguity or absence. Normalization already sees all domains at once, so this is a lookup change, not a restructuring. The alternative (b) — reject the authored bare cross-domain form as a validation error — is a language decision that needs Louis, breaks authored bundles, and contradicts `get_optional_pipe`'s deliberate fallback (whose own `TODO` asks for a `domain_hint`, i.e. the fallback is intended to stay). Whoever lands (a) should also decide whether that `domain_hint` TODO is the same conversation — normalization is exactly the caller that knows the hint.++**The gate.** Normalize a two-domain crate whose step crosses domains and assert every in-body pipe ref resolves to a key present in `crate.pipes`. Five paths qualify a ref today — a sequence's `steps`, a parallel's `branches`, a condition's `outcomes` **and** its `default_outcome`, and a batch's `branch_pipe_code` — so drive the assertion off `pipe_dependencies` rather than hand-walking the controller kinds, and the sixth arrives already covered. A crate-closedness assertion of that shape would have caught this the day the fallback was written; make it a standing test, not a one-off.++### 1.2 The types projection and the runtime disagree on a structureless concept's base class++**The defect.** A concept declaring only a description — no `structure`, no `refines` — resolves to two different types. The runtime promotes it to a refinement of native `Text`: `ConceptFactory._handle_basic_blueprint` generates its class on a `TextContent` base. The `python-structures` projection emits it as `class X(StructuredContent)` with `ConfigDict(extra="allow")`. `TextContent` and `StructuredContent` are siblings, so the same authored concept is a text type on one side and an object type on the other.++**Why it is worse than the (deliberate) naming difference.** Two things branch on the base class. First, the interpreter's text-vs-object dispatch asks whether the output concept is strictly compatible with native `Text` — `True` with the runtime's class, `False` with the projected one, so the same concept triggers two different LLM calls with different prompts and result shapes. Second, the text path's read-back (`model_validate({"text": ...})`) succeeds on **both** classes with identical dumps — `extra="allow"` swallows the mismatch — so the divergence produces no exception anywhere. Any consumer that generates `structures.py` and hands those classes back to the runtime (the projection's entire point) inherits this. Cookbook bundles authoring structureless output concepts (e.g. `examples/wip/blog_article_generator/`, `examples/wip/write_tweet/tech_tweet.mthds`) are live instances of the shape.++**The fix — decision D-2, recommendation (a).** Make the projection follow the runtime: emit `TextContent` as the base for a structureless concept (one arm in `python_structures`), so the projected class satisfies the same `isinstance` checks the runtime's does and carries a real `text` field — `extra="allow"` stops doing load-bearing work for this shape (it stays, for pass-through of unknown fields, which was its actual job). The alternative (b) — stop the runtime's promotion and make structureless genuinely shapeless — is a language decision: "describe it in prose and get text back" is what a structureless concept currently *means* to authors, and changing that is not a codegen call.++**The gate.** Assert that the projected class and `ConceptFactory`'s generated class for the same blueprint share the same `StuffContent` ancestor, across all four shapes the crate can resolve on its own: structureless, structured, refines-native, refines-in-crate. A cross-package `refines` is a fifth shape and stays deliberately outside this gate — the crate cannot resolve the target without its dependency loaded, so the comparison has nothing to compare; it is written up in [`deferred-review-observations.md`](deferred-review-observations.md). Within the four, one language invariant keeps the sweep closed and should be stated in the test: MTHDS forbids `refines` + `structure` together, so a structured concept refining a structureless one — the shape where a divergence could hide — cannot be authored at all.++### 1.3 `render_import_block` emits lines a consumer's formatter rewrites++**The defect.** Every renderer in `python_common.py` respects `PY_EXPLODE_WIDTH = 88` and pre-explodes past it with a magic trailing comma — the formatter-clean discipline #1070 established, precisely so a consumer's `ruff format` cannot rewrite emitted bytes. `render_import_block` is the one hole: it merges and sorts, but never wraps, at any width. A merged import whose names cross 88 columns renders flat and gets rewritten into the parenthesized form by the first `ruff format` in the consumer's tree. Today's emitted artifacts stay under the threshold, so the defect is latent rather than absent — one more name sharing a module and it fires, and the existing lint-clean test would not notice, since `ruff check` has no opinion on a long import line.++**The fix.** Wrap past `PY_EXPLODE_WIDTH` in the parenthesized form with a magic trailing comma — the same shape the sibling renderers already use and exactly what ruff itself produces, so the emitted bytes become a fixed point of the consumer's formatter at any width ≥ 88.++**The gate.** Extend the lint-clean test to assert `ruff format --diff --line-length 88` is empty for each emitted artifact, not just that `ruff check` passes — and keep #1070's sweep shape (input shapes × targets × widths), because example-based versions of this exact claim have been disproved before. Lint-clean and format-stable are different properties, and only the second is what byte-reproducibility in a consumer's tree rests on.++**🔶 CHECKPOINT A — the dev-buildable trio landed.** ✅ **Reached 2026-08-03.**++All three fixes landed on `fix/Parity-gaps`, each red-green verified against the tree before it (the gate was written first, run against the unfixed code, and observed to fail for the stated reason).++| Fix | What landed | Gate |+| --- | --- | --- |+| 1.1 | `_qualify_pipe_ref` resolves a bare pipe ref crate-wide against an index of the crate's own pipe refs, raising `CrateNormalizationError` on ambiguity or absence | `test_crate_normalization.py`: crate-closedness over `pipe_dependencies` (parametrized across both crate shapes), cross-domain resolution at all five call sites, special outcomes untouched, ambiguity, absence |+| 1.2 | `python_structures._base_class` emits `TextContent` for a structureless concept; a Python-class-backed one keeps the root base | `test_projection_agrees_with_runtime_base.py`: same nearest `StuffContent` ancestor as `ConceptFactory`'s generated class, across all four authorable shapes, plus the named answer and the opaque-class carve-out |+| 1.3 | `render_import_block` pre-explodes past `PY_EXPLODE_WIDTH` via `_import_statement` | `test_emitted_artifacts_are_lint_clean.py`: `ruff format --check` at `PY_EXPLODE_WIDTH` over a merged block and a single overlong name |++**Real-world confirmation of 1.1.** The plan's repro was run end-to-end against `pipelex-cookbook/examples/wip/advisory_board/`: before the fix, `advisory_orchestrator.master_advisory_orchestrator -> advisory_orchestrator.present_as_markdown` is dangling in the normalized crate; after, the ref reads `presentation.present_as_markdown` and the crate is closed.++**Deviations from the recommendations.**++- **D-1: the "own domain first" rung was dropped.** Recommendation (a) had own-domain-first, then crate-wide. `get_optional_pipe` does not prefer the caller's domain — it raises on an ambiguous bare code — so that rung would have *created* a disagreement in the one case the two differ. What landed mirrors the shipping reader exactly. Written up in [`d1-domain-hint-deferred.md`](d1-domain-hint-deferred.md).+- **D-1's `domain_hint` conversation: deferred**, not opened. It is a runtime/language decision (which pipe a runnable bundle calls), and the normalizer follows it for free once it is settled — same doc.+- **1.3's grounding claim was stale.** The plan recorded that the lint-clean gate "runs `ruff check`, not `ruff format --diff`". It already ran `ruff format --check`, including parametrized over consumer line-lengths down to 88. The real hole was coverage, not the assertion: no crate shape produces an import line past 88 (each native content class lives in its own module, so nothing merges far enough), so the gate is a direct test on `render_import_block` rather than another crate fixture. `_assert_ruff_clean` was split so the new test reuses the format-stability half — an import block alone has nothing to *use* its names, so `ruff check` would flag `F401` on a shape no emitter writes.+- **§1.1's call-site count above was wrong.** It says "steps, branches, outcomes — all three `_qualify_pipe_ref` call sites". There are **five**, across all four controller kinds: `steps` (Sequence), `branches` (Parallel), `outcomes` *and* `default_outcome` (Condition), `branch_pipe_code` (Batch). What shipped covers all five and the gate exercises all five; only the plan's pre-implementation count was short.++**Gates run:** `make agent-check` ✅ · full `make agent-test` ✅ · `make drift-check` ✅ · `[Unreleased]` changelog entry for each of the three ✅.++**Review rounds.** CI green (all shards, lint, typecheck, keyword-only, doc-check). Greptile 5/5 clean; cubic clean across the diff; a Sonnet-5 `/code-review` fan-out (no inherited context) found nothing. Codex raised one P2, verified real but **pre-existing** and deferred with a written argument ([`structureless-concept-with-registered-class.md`](structureless-concept-with-registered-class.md)). A final gstack `/review` returned **land**, and surfaced one genuine defect none of the four had caught, fixed here:++- **`_native_class` named the root base without importing it.** `Anything` is the only native with `structure_class is None`, so it was the one reference that fell through to a bare `"StructuredContent"` string — a module that raises `NameError` on the consumer's first import (`ruff` flags it `F821`). Now routed through `_structured_content`, which registers the import like every other emission in the file. Gated by `test_refines_anything_imports_the_root_base_it_names`, which *loads* the module rather than grepping it. Note the `every_type_kind_crate` fixture cannot gate this — its structured concepts already register the import, masking the omission — hence a dedicated single-concept crate.++That review also corrected three documentation inaccuracies, all fixed: the doc's "opaque is always pass-through, never lossy" claim (false for this target since 1.2 — `TextContent.text` is required, so an object payload now raises), a citation of a `ConceptBlueprint` validator that does not exist, and the mechanism sentence in the D-1 note (the guarantee comes from `validate_library`'s `isinstance(pipe, PipeController)` gate covering *all four* controller kinds, and the crate is built **before** validation, not after). Its remaining observations were judged not-clear-wins and deferred to [`deferred-review-observations.md`](deferred-review-observations.md).++This is the PR boundary: Phase 1 ships on its own, as PR #1085 against `dev`. Phase 2 did not wait behind it — see below.++## Phase 2 — the kernel trio ✅ **RESOLVED, by folding into #1081 / #1082 rather than waiting for the gate**++These three are written against the open kernel-extraction PRs and share a theme: the `pipelex.kernel` package's stated contract — a runtime layer callable without the interpreter, whose façade is a faithful convenience over the ops — is not yet true in three places.++**How this phase actually shipped.** The plan's instruction was to re-verify against the *merged* code and not land these as review feedback on the finalized stacked PRs. That was the right default and it was overridden deliberately, by Louis' call, once the alternative was priced honestly: each of these is a defect *the kernel PRs themselves introduce*, so landing them afterwards would mean knowingly merging a package whose contract is false and then repairing it — and the repair PR would have to re-open the same files. Two of the three folded in; the third dissolved on re-verification.++| Gap | Outcome | Where |+|---|---|---|+| 2.1 `llm_text` narrower than its op | Fixed | *make llm_text as wide as the op beneath it*, on `refactor/Kernel` (#1081) |+| 2.2 `llm_object` prompting style | **Not a live defect** — claim overstated; the surviving question is deferred | *number the prompting-style follow-up KF-16 to keep the stack's*, on `refactor/Kernel` (#1081); KF-16 → `wip/prompting-style/README.md` |+| 2.3 kernel cannot build an `ImgGenPrompt` | Fixed | *let a runtime-only caller build the image prompt it has to pass in* + arm *give the image prompt assembler the boot-contract arm the rule owes it*, on `refactor/Kernel-phase2` (#1082); doc review *record the image prompt assembler on the page that specifies the kernel* (#1083) |++**Commits are named by subject, not by SHA, and deliberately.** These fixes live on a three-deep stacked series that is rebased whenever a parent moves, which rewrites every hash on it. An earlier revision of this table named SHAs; the rebase that folded in the review round orphaned three of them, so they resolved from no branch at all. A subject survives the rewrite — `git log --grep` finds it.++The re-verification the plan asked for happened against each PR's own branch rather than against a merged result, which is stricter in one way (the code is exactly what will merge) and weaker in another (it cannot see interactions with a later PR in the stack). Nothing in the stack touches these paths after #1082, so the weakness is not live here.++### 2.1 `MethodKernel.llm_text` is narrower than the op beneath it++**The defect** (against #1081's code). The façade's text method hardcodes what the op parameterizes, in three ways: output concept (op takes `concept: Concept`; façade hardcodes native `Text`), output class (op takes `output_class`; façade hardcodes `TextContent`), and model (op's resolution accepts `None` and defers to the deck; façade requires a `model`). Its sibling `llm_object` has none of these limits — the asymmetry is between the façade's own two methods. The interpreter passes all three through, with the model choice routinely `None`.++**Why it matters.** A programmatic caller producing a concept that refines native `Text` stores `native.Text`/`TextContent` where an interpreted run stores the declared concept and its generated class — a silent memory divergence, and a hard `StuffContentTypeError` the moment anything reads the result back expecting the declared class. And a model-less call — the normal authored case, since omitting the model defers to the deck exactly like a `$preset` does — is inexpressible through the façade for no semantic reason.++**The fix.** Widen `llm_text` to match `llm_object`: `concept` and `output_class` optional, defaulting to today's hardcoded pair; `model` optional, passed through to the resolution unchanged. Additive — every existing call site keeps working because the defaults are today's behavior.++**✅ Done — *make llm_text as wide as the op beneath it*, on `refactor/Kernel` (#1081).** Re-verified against that branch and held exactly as written: all three widenings were needed. Pre-fix the façade declared `model: LLMModelChoice` with no default — required — and it now declares `model: LLMModelChoice | None = None` alongside the newly optional `concept` and `output_class`. Gated by `tests/unit/pipelex/kernel/test_llm_text_kernel.py`, which asserts on what lands in the returned memory (`type(stuff.content) is MeetingSummary`, the stuff's `concept_ref`) rather than on the signature — a signature assertion would pass on a façade that accepted the arguments and dropped them. No changelog entry: the package is new and unreleased, so there is no shipped behavior to note.++### 2.2 `MethodKernel.llm_object` renders under the wrong model's prompting style++**The defect** (against #1081's code). A `PipeLLM` carries two model choices (`model` → for_text, `model_to_structure` → for_object). The interpreter resolves **two** settings and derives the templating style from the **text** one (`pipe_llm.py:219–237` on dev — the pipe's main model governs how the prompt is written; the object path differs only in how the answer comes back). The façade collapses to one choice, drops the object chain's `for_text` fallback rung, and derives the style from the **object** setting. Unlike 2.1 this is not narrowness visible in a signature — it is a wrong value.++**When it bites.** A pipe declaring both choices, resolving to models with different prompting targets: the prompt is assembled under the structuring model's style while an interpreted run assembles it under the main model's. Both calls succeed; the prompt text differs. Also, "text model X, object model default" is expressible in MTHDS and inexpressible through the façade.++**The fix.** Give `llm_object` the second choice (`model_for_object`), forward the `for_text` rung into the object resolution, and derive the style from the text setting — mirroring the interpreter line for line. Existing single-choice callers keep working and their style derivation *changes*, which is the point: today it is wrong whenever the two settings differ. Pair with a test asserting the two-choice case — no current test covers a pipe declaring both.++**⚠️ Not built — the claim above is overstated, and the feature it defends is slated for removal.** Two independent reasons, either sufficient:++*The defect is not live.* The façade's single explicit `model` wins `resolve_llm_setting_for_object`'s **first** rung, so the style is derived from that very setting — which is also what an interpreted run derives for a pipe naming only a text model. The two agree for every call the façade can express. Divergence needs a pipe naming *both* choices, and the façade cannot express one. So what remains is narrowness, not a wrong value — the same category as 2.1, and a much weaker instance of it.++*The mechanism is obsolete.* Model-derived prompting style dates from when models were genuinely sensitive to prompt markup. Louis' ruling: rather than widen the derivation, the style becomes an explicit user choice defaulting to XML, and the derivation goes away. Building `model_for_object` here would mean constructing the two-setting derivation that refactor deletes.++Deferred as **KF-16** in `wip/kernel/deferred-follow-ups.md`, with the design in `wip/prompting-style/README.md` — both ship on [PR #1081](https://github.com/Pipelex/pipelex/pull/1081), not on this branch, hence plain paths rather than links (they resolve once that PR merges to `dev`). Silenced in #1081 by *number the prompting-style follow-up KF-16 to keep the stack's* — a docstring on `llm_object` recording why its single `model` is correct, carrying the trap explicitly: *do not "fix" this by deriving the style from an object-only resolution, which would introduce the divergence rather than close it.*++### 2.3 Nothing in the kernel can build an `ImgGenPrompt`++**The defect** (against #1082's code). `kernel.img_gen_ops.run_img_gen` takes a ready `ImgGenPrompt`, by deliberate analogy with `run_llm_text` taking a ready `LlmPromptContent`. The analogy is where it breaks: the kernel ships the LLM prompt's constructor (`llm_prompt_content.make_from_text`) but no image counterpart — the only builder is `ImgGenPromptBlueprint.make_img_gen_prompt`, an interpreter-layer module. So a `RuntimeBoot`-only process can call the text, object, and compose ops and **cannot call the image op at all**. Sharper: `kernel/prompt_references.py` hosts `ImageReference`/`ImageReferenceKind` under a docstring asserting the kernel is what resolves them — and it is not; the blueprint imports those names upward and does the resolving. The vocabulary migrated and the semantics did not, which makes the layer's "callable without the interpreter" guarantee false for one operator — and that guarantee is the kind worth nothing at less than 100%.++**The fix — decision D-3, recommendation (a).** A `pipelex/kernel/img_gen_prompt.py` mirroring `llm_prompt_content.py`: takes memory (as a context provider), the template source/category/style, an optional negative template, and the kernel's own `ImageReference` list; returns an `ImgGenPrompt`. The blueprint keeps its parse-and-validate role and delegates — completing the move its own docstring describes as already done. The honest cost: the assembly pulls `ImageRegistry`, `PromptImageFactory`, `substitute_nested_in_context` and `render_template` with it, so scoping must decide whether those are runtime-layer or tools-layer citizens (most already are). The alternative (b) — export just the render step and let callers assemble — is smaller but leaves the image-registry and `[Image N]` placeholder logic (the genuinely subtle part, where a token-order/`input_images`-order mismatch silently mislabels images) to be re-derived by every caller; acceptable only if explicitly framed as a first step.++**The gate.** A test that builds an `ImgGenPrompt` through the new kernel constructor and through the blueprint from the same authored inputs and asserts they are equal — the same "both readers, same answer" shape as everything above — plus a boot-scoped test that the constructor is importable and usable under `RuntimeBoot` alone.++**✅ Done — *let a runtime-only caller build the image prompt it has to pass in*, on `refactor/Kernel-phase2` (#1082), recommendation (a).** `pipelex/kernel/img_gen_prompt.py` holds `assemble_img_gen_prompt`; `ImgGenPromptBlueprint` drops 296 → 82 lines and delegates, keeping parse-and-validate. `ImgGenPromptBlueprintValueError` becomes the kernel's existing `PromptContentError`, whose docstring already described exactly this failure — the same swap Phase 1 made on the LLM side.++**D-3's scoping question, answered: it cost nothing.** The plan priced the move as dragging `ImageRegistry`, `PromptImageFactory`, `substitute_nested_in_context` and `render_template` across a layer boundary. All four are already `cogt`/`tools` citizens, and the blueprint held exactly **one** interpreter-layer import (`PipeImgGenFactoryError`). Nothing moved layers. `max_prompt_images` deliberately stayed with the blueprint: it is a property of the model the *pipe* chose, its breach is a pipe-level error, and it is a `len()` check rather than the subtlety the kernel module exists to centralise.++**Both gates built, and the boot-scoped one found a real gap.** The equality test is `tests/unit/pipelex/kernel/test_img_gen_prompt_kernel.py`. The boot-scoped half revealed that `assemble_img_gen_prompt` had no arm in `test_kernel_boot_contract.py` — which the kernel's own standing rule requires, because that subprocess is per-function and is the only gate that can see a function-local interpreter import. `assemble_llm_prompt` needs none (both `run_llm_*` call it, so the LLM arms carry its closure); `run_img_gen` takes a *ready* prompt and calls nothing, so the img-gen arm's coverage was incomplete in precisely the way the kernel surface was — it hand-built its `ImgGenPrompt` because nothing in the kernel could build one. Fixed by *give the image prompt assembler the boot-contract arm the rule owes it*, on **#1082**, deliberately not on #1083: an entry point and its only gate belong in the same PR, or #1082 merges to `dev` in violation of the rule its own package docstring states. The arm now assembles then generates, asserting on the `[Image N]` token *and* `input_images`, the two readings of one registry. Only the *doc* half went to #1083, as *record the image prompt assembler on the page that specifies the kernel*, because that is where the kernel page and its drift contract live.++**🔶 CHECKPOINT B — plan complete.** Record: merged-code re-verification results for 2.1–2.3 (what moved, what held), each fix's commit and gates, D-3's scoping outcome (which helpers moved to which layer), and changelog entries. Run the full gate set. Update the README status block and close the track, or spin out whatever the re-verification surfaced.++**Checkpoint B record.** Re-verification: 2.1 held as written in all three of its arms, `model` included; 2.2 did **not** hold and was withdrawn (above); 2.3 held, and its predicted layering cost did not materialise. Commits and gates are recorded per-gap above. D-3 settled on (a) with no helper moving layers. Changelog: one entry extended (`LLMPromptBlueprintValueError` → *and `ImgGenPromptBlueprintValueError`*) plus one new entry for `assemble_img_gen_prompt`, both on #1082; 2.1 needed none. Gate set green on both branches — `make agent-check`, `make drift-check`, and a full `make agent-test`. Spun out: KF-16 / `wip/prompting-style/README.md`.++## Decisions++- **D-1 — bare cross-domain qualification: (a) qualify against the crate (recommended) vs (b) reject the authored form.** (a) matches the runtime the language already ships and fixes fingerprint + round-trip at once; (b) is a breaking language ruling for Louis. Settle at 1.1; note the adjacent `domain_hint` TODO either way.+- **D-2 — structureless base class: (a) projection follows the runtime (`TextContent` base, recommended) vs (b) runtime stops promoting.** (a) is one arm in the projection and honors what the declaration means to authors today; (b) is a language change. Settle at 1.2.+- **D-3 — img-gen prompt assembly: (a) kernel constructor module mirroring `llm_prompt_content` (recommended) vs (b) export the bare render step.** (a) completes the layering the kernel already claims in writing; (b) re-distributes the subtle half to every caller. Settle at 2.3, with the helper-scoping question answered explicitly rather than by drift. — **✅ Settled: (a).** The scoping question answered itself: no helper changed layers, because all four were already `cogt`/`tools`. See 2.3.++## Out of scope++- The runtime's structureless *promotion* semantics (D-2 option (b)) and the bare-ref *language rule* (D-1 option (b)) — both are author-facing language decisions, flagged for Louis, not taken here.+- `MethodKernel` growing façade methods for the non-LLM operators — a design conversation for after the kernel PRs settle, not a parity defect.+- Anything in the open kernel PRs themselves — they are finalized and stacked; this branch follows them, it does not amend them.diff --git a/wip/parity/structureless-concept-with-registered-class.md b/wip/parity/structureless-concept-with-registered-class.mdnew file mode 100644index 000000000..95036cdf7--- /dev/null+++ b/wip/parity/structureless-concept-with-registered-class.md@@ -0,0 +1,53 @@+# Deferred — a structureless concept bound to a registered class through a channel the crate cannot see++**Raised by:** the Codex reviewer on PR #1085 (P2), then **independently re-raised by cubic at P1** two rounds later against the same line. **Verified 2026-08-03**, against the tree at `d9efffbfb`, re-checked and quantified at `c4788e47b`. **Status: deferred** — no fix available at the emitter layer that is a clear win, and the correct fix is a language decision for a separate change.++⚠ **Two independent reviewers converged on this**, which raises its priority above an ordinary deferral. It is the one open question on this PR that a human should rule on. Nothing about the ruling blocks Phase 1 — but it should not sit indefinitely either.++## The shape++`ConceptFactory._handle_basic_blueprint` (`pipelex/core/concepts/concept_factory.py:386-392`) looks the **bare concept code** up in the class registry *before* falling back to the `TextContent`-based generated class. On a hit it binds the concept to that class outright — no promotion, no `refines`. What fills the registry from a user's library directory is `ClassRegistryUtils.import_modules_in_folder(base_class_names=["StructuredContent"])` + `auto_register_all_subclasses(base_class=StructuredContent)` (`pipelex/libraries/library_manager.py:338`, `:372`), so any hand-written `StructuredContent` subclass in `sys.modules` is registered under its bare name.++So a description-only concept `CustomerReview = "..."` sitting beside a hand-written `class CustomerReview(StructuredContent)` resolves, at run time, to the hand-written class.++This is not theoretical. It is **the documented endpoint of a migration guide** — `docs/building-methods/concepts/python-classes.md:162-224` teaches exactly this move ("Structure section removed — now defined in Python"), and it teaches the *bare-name* form rather than the crate-visible `structure = "<ClassName>"` one.++**Measured 2026-08-03**, and the split is what decides the trade:++| Tree | Hand-written content classes | Description-only concepts | Colliding (the shape) |+| --- | --- | --- | --- |+| `pipelex/` own test fixtures | — | — | **93** |+| `pipelex-cookbook/` | 40 | 13 | **0** |+| `pipelex-starter-python/` | 0 | 0 | **0** |++So the shape is pervasive in this repo's fixtures (e.g. `tests/integration/pipelex/pipes/controller/pipe_sequence/pipe_sequence_2.mthds`'s `CustomerReview = "A single customer review text"` beside `pipe_sequence.py`'s three-field `CustomerReview`) — and **those fixtures are interpreted, never codegen'd**. The trees that *are* the codegen-facing examples hold none: the cookbook's structure classes all use the domain-qualified generated form, which hits the *second* lookup (`concept_factory.py:391`) and lands on the same `TextContent`-based generated class, so no divergence there.++That asymmetry is the argument for what landed. 1.2 makes the projection agree with the runtime for the plain description-only concept — common in every tree, including the 13 in the cookbook — and disagree for the registered-class shape, which does not occur in any tree that runs codegen. Reverting inverts both.++## What PR #1085 changed about it, honestly++Before, the projection emitted `StructuredContent` for every concept with no base. Because the only class reachable through the sanctioned loader path is a `StructuredContent` subclass, the old output happened to match the runtime's *nearest content ancestor* for this shape. After, it emits `TextContent`, which does not.++But the projection was never the runtime class either way: it emits a **fieldless shell with `extra="allow"`**, before and after. The runtime class carries the author's real fields and validators. Keeping the base would not have made the projection usable as a stand-in — that needs the whole class body, which the crate structurally cannot supply.++The ledger, stated in full rather than in the direction that flatters the PR:++- **Worse.** The declared base no longer matches, and `TextContent.text` is required (`pipelex/core/stuffs/text_content.py:17`), so `Projected.model_validate({"title": …})` now *fails* where the old shell absorbed the payload into `extra`.+- **Better.** The old projected `X(StructuredContent)`, if `structures.py` lands in the library directory, is itself auto-registered and can **win the registry over the author's real class** (`auto_register_all_subclasses` is first-wins, `pipelex/system/registries/class_registry_utils.py:167-170`) — silently substituting an empty shell for a three-field class. The projected `TextContent` subclass is never registered at all, so that hazard is gone.+- **Unchanged.** The interpreted library. Nothing in the runtime consumes the projected class, so "the projection silently changes object content into text content" overstates the blast radius.++Net: the PR **moves an already-divergent case from "right base, empty body" to "wrong base, empty body"**, and removes a collision hazard on the way. It does not introduce the divergence.++## Why it is not fixed here++Three candidates, none a clear win:++1. **Revert 1.2 to `StructuredContent`.** Re-breaks the common case — every description-only concept with no hand-written class, which is the majority everywhere and the entirety of the cookbook — and reinstates the registry-clobber hazard. Strictly worse.+2. **Teach the crate the auto-detected class.** The crate builder would have to consult the live class registry — `get_class_registry()` returns the active library's ContextVar-scoped registry when one is set and the process-global one otherwise — making the crate *and its fingerprint* depend on what happened to be importable in the emitting process. (The scoping isolates concurrent runs from each other; it does nothing to make the registry's *contents* reproducible, which is the property at stake here.) That destroys the two properties the crate exists for: portability (it travels to a sandbox as `python_sources`) and a semantic fingerprint the stamp/lock trust chain rests on. A contract break, not a layering nit.+3. **Drop the bare-name auto-detect at `concept_factory.py:389`, requiring the explicit `structure = "<ClassName>"` form.** This is the *actually correct* resolution: it makes the runtime's binding crate-visible, collapses the case into the already-handled `opaque_python_class` branch, and removes an environment-dependent name binding. Breaking changes are fine in this repo. But it is a language/runtime decision that contradicts a published migration guide and touches many fixtures — far outside "small and obviously correct", and outside this PR's scope.++## What settling it will need++- A ruling on (3): should a hand-written class have to be *named* in the `.mthds` to bind, or should a bare-name match keep working? Same family of question as D-1's `domain_hint` — a language call, not a codegen one.+- Either way, `docs/building-methods/concepts/python-classes.md:218-221` currently teaches the form that is invisible to codegen. That doc/feature tension needs resolving whichever way the ruling goes.+- **No test covers this shape today**, for either reader. `test_projection_agrees_with_runtime_base.py` builds its four shapes against an *empty* library, so `concept_factory.py:389` is never taken. Whoever settles this should add the missing coverage as part of it — the branch that binds a concept to a registered class is currently unguarded on both sides.