Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
fix: reflect class-backed concepts field by field in the input-form descriptor (#1166)
Louis Choquel4def57b50239
diff --git a/.badges/tests.json b/.badges/tests.jsonindex 79c245966..fdd00b0fa 100644--- a/.badges/tests.json+++ b/.badges/tests.json@@ -1,7 +1,7 @@{"schemaVersion": 1,"label": "tests",- "message": "11096",+ "message": "11140","color": "blue","cacheSeconds": 300}\ No newline at end of filediff --git a/CHANGELOG.md b/CHANGELOG.mdindex 486c8f353..9c55c3e0e 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -8,6 +8,7 @@### Fixed+- **Class-backed concepts reflect field by field in the input-form descriptor**: a concept declaring `structure = "SomeClass"` used to collapse to `kind: "unknown"` the moment one field annotation could not be mapped, taking every sibling field with it — and a nested non-native pydantic model was reported as an opaque payload rather than walked into. Every `document` and `image` position beneath such a concept disappeared from the descriptor, so a consumer preparing inputs from it passed a local file path at one of those positions through un-uploaded and the run failed on the unresolved path. Reflection is now partial: each annotation is mapped on its own, an unmappable one is `unknown` at that field alone, a nested model is an `object` the reflection walks into, a `list[X]` reports its element node, and a class that holds itself is cut at the revisit instead of recursing. A `RootModel` is reported as its root annotation's node wherever it appears — as a field of a reflected class, and as the registered class backing the concept itself — because the value it accepts is the root value, so the descriptor no longer states an object over a synthetic `root` key that contradicts the `json_schema` derived beside it. The native consistency probe keeps its faithful-or-absent rule — it is compared against a normative pinned blueprint, where a plausible-looking partial answer would be the failure — and the two reflections now share one annotation vocabulary (`pipelex/core/concepts/annotation_shapes.py`) instead of one calling the other. See [Input-Form Descriptor](under-the-hood/input-form-descriptor.md).- **Generated-image storage key extension follows the actual mime type**: a generated image could be stored under a `.png` key while its bytes and reported `mime_type` were `image/jpeg`, because `GeneratedContentFactory` picked the key's extension from the *requested* image format while `mime_type` came from the provider's *actual* answer — and a provider may return a different format than requested. The factory now resolves the effective mime type once (provider-reported, then base64-extracted, then the requested format, then the `image/jpeg` default) and derives the storage-key extension from that same value, on every path (stored bytes and fetched remote content alike), so the key and the `mime_type` can no longer diverge and anything inferring a type from the extension sees the truth.## [v0.55.0] - 2026-08-29diff --git a/docs/under-the-hood/input-form-descriptor.md b/docs/under-the-hood/input-form-descriptor.mdindex 85afcefe1..609810699 100644--- a/docs/under-the-hood/input-form-descriptor.md+++ b/docs/under-the-hood/input-form-descriptor.md@@ -28,7 +28,7 @@ The derivation is total. A node it cannot map honestly reports `kind: "unknown"`Kinds are decided by chain membership and declared types — never by sniffing a schema shape. Per node, in precedence order:1. A concept with an authored structure *table* anywhere along its refinement chain is an `object`; its fields are the merged structures along the chain, base fields first, a refining concept overriding its parents'. The table decides even when it is empty — the engine backs an empty `[concept.X.structure]` with a field-less structured model, not with `TextContent`, so the descriptor reports `object` with an empty `fields` list rather than falling through to rule 4.-2. Otherwise, the first `structure = "ClassName"` on the chain decides: a native class name (`TextContent`, `ImageContent`, …) maps by identity to that native's row; any other registered class is reflected field by field into an `object`; an unregistered or unmappable class is `unknown`.+2. Otherwise, the first `structure = "ClassName"` on the chain decides: a native class name (`TextContent`, `ImageContent`, …) maps by identity to that native's row; any other registered class is reflected field by field into an `object`; a class the registry does not hold — or one that is not a pydantic model — is `unknown`.3. Otherwise, a chain bottoming at a native concept takes that native's row, keeping the concept's own `concept_ref`, description and `refines`.4. Otherwise — a description-only or string-described concept — `prose`, with `refines` carrying the authored chain alone (absent when nothing was authored): a producer states only the links it holds, so no `native.Text` link is reconstructed — text-valuedness reaches the wire as the `prose` kind itself.@@ -49,6 +49,8 @@ The table and rules above ARE the no-hint kind assignment — stated rules, notNested structure fields map by their declared type: `text` → `text`; `integer` → `number` with `integer: true`; `number` → `number`; `boolean` → `boolean`; `date` → `date`; `datetime` → `date` with `datetime: true`; `time` → `text` with `format: "time"`; a field with `choices` → `enum` (choices win over `type`, matching the structure generator); `concept` → the concept's node, carrying its namespaced `concept_ref`; `list` → `list` whose `item` comes from `item_type` / `item_concept_ref` (a nested list's inner item is inexpressible and reports `unknown`; the `item` node carries no `name` member — a list's item has no authored name, and the index labels entries); `dict` → `unknown`. The shorthand `field = "description"` form is a required `text`. Nested fields take the blueprint field's description and `required` over the concept's; a scalar flattened at the top level keeps the concept's description.+A reflected class's fields map by their annotation, and each annotation is mapped **on its own**: a scalar takes its kind from the type, a native content class (`ImageContent`, `DocumentContent`, …) takes that native's row by identity, a `list[X]` is a `list` whose `item` is `X` reflected one layer down, a nested non-native pydantic model is an `object` the reflection walks into, a `RootModel` is its root annotation's node (the value it accepts is the root value itself, never an object over a `root` key), and an annotation with no honest node — a union that is neither `X | None` nor `int | float`, a `dict`, a class already on the path — is `unknown` at **that field**, leaving its siblings stated. Partiality is the point: collapsing the whole payload the moment one annotation could not be mapped hid every `document` and `image` position under the class, and a consumer preparing inputs from the descriptor then passed a local file path through un-uploaded. A class declaring no field is an `object` with an empty `fields` list — a payload that demands nothing, the class-backed twin of an empty authored structure table. A registered class that IS a `RootModel` is read the same way at the top: the concept's node is its root annotation's, carrying the concept's own `concept_ref`, `refines` and description — so a root-valued concept agrees with the `json_schema` derived from that same class beside it, instead of describing an object over a `root` key that `model_validate` would reject. This is deliberately unlike the reflection the native consistency probe performs (`pipelex/codegen/native_expansion.py`), which stays faithful-or-absent because it is compared against a normative pinned blueprint, where a plausible-looking partial answer would be the failure.+Constraint slots (`minimum`, `exclusive_minimum`, `min_length`, `pattern`, …) come only from reflected classes: the language cannot author constraint keys (an authored `minimum = 0` is rejected as an unknown structure-field key), whereas a registered class's `Field(gt=0, max_length=8, pattern=...)` is read from the pydantic field metadata and stamped on the matching `number` or `text` node. A pydantic default on a reflected class is an authored fact: `field_info.is_required()` decides the node's `required` and a non-`None` default is reported as its `default_value` — a defaulted field is never required, the same invariant blueprint validation enforces by rejecting the `required = true` + `default_value` pair. (Authored `hints` are a parsed blueprint field with its own leniency rules; see below.)## Hints@@ -67,4 +69,4 @@ Inapplicable slots are absent, never JSON `null`: the report's valid arm is dump## Seeing it-`pipelex-dev trace-input-semantics` captures the descriptors as `hop5_input_form.json` beside `hop5_pipe_io_contracts.json`, so an authored fact can be checked on both projections at once — see [Tracing Input Semantics](../contribute/trace-input-semantics.md). The assignment table is pinned by `tests/integration/pipelex/pipeline/test_input_form.py` over the committed probe bundle (and the hints behavior over its hinted sibling, `hinted_bundle.mthds`), the emitted wire's null-free shape by `tests/unit/pipelex/pipeline/test_input_form_models.py`, the hint merges and kind-feeding by `tests/unit/pipelex/pipeline/test_input_form_hints.py`, and the escape hatches the library loader keeps unreachable (concept cycles, unregistered classes) by `tests/unit/pipelex/pipeline/test_input_form_deriver.py`.+`pipelex-dev trace-input-semantics` captures the descriptors as `hop5_input_form.json` beside `hop5_pipe_io_contracts.json`, so an authored fact can be checked on both projections at once — see [Tracing Input Semantics](../contribute/trace-input-semantics.md). The assignment table is pinned by `tests/integration/pipelex/pipeline/test_input_form.py` over the committed probe bundle (and the hints behavior over its hinted sibling, `hinted_bundle.mthds`), the emitted wire's null-free shape by `tests/unit/pipelex/pipeline/test_input_form_models.py`, the hint merges and kind-feeding by `tests/unit/pipelex/pipeline/test_input_form_hints.py`, and the escape hatches the library loader keeps unreachable (concept cycles, unregistered classes, a structure class that holds itself) by `tests/unit/pipelex/pipeline/test_input_form_deriver.py`.diff --git a/pipelex/codegen/native_expansion.py b/pipelex/codegen/native_expansion.pyindex 5bea43011..5b583e97c 100644--- a/pipelex/codegen/native_expansion.py+++ b/pipelex/codegen/native_expansion.py@@ -19,19 +19,22 @@ annotation has no honest blueprint form at all (e.g. a non-Optional union) makesreflected structure absent."""-import datetime-from types import UnionType-from typing import Any, Union, cast, get_args, get_origin+from typing import Any, cast, get_originfrom pydantic import BaseModel+from pipelex.core.concepts.annotation_shapes import (+ is_number_union,+ is_union,+ list_item_annotation,+ native_code_for_content_class,+ scalar_field_type,+ strip_optional,+)from pipelex.core.concepts.concept_blueprint import ConceptBlueprint, ConceptStructureBlueprintTypefrom pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprint, ConceptStructureBlueprintFieldTypefrom pipelex.core.concepts.native.concept_native import NativeConceptCodefrom pipelex.core.concepts.native.pinned_blueprints import make_pinned_native_blueprint-from pipelex.core.stuffs.stuff_content import StuffContent--_NATIVE_CLASS_NAME_TO_CODE: dict[str, NativeConceptCode] = {native_code.structure_class_name: native_code for native_code in NativeConceptCode}class _UnmappableAnnotationError(Exception):@@ -58,16 +61,18 @@ def reflect_native_structure(native_code: NativeConceptCode) -> dict[str, Concepstructure_class = native_code.structure_classif structure_class is None:return None- return reflect_structure_class(structure_class=cast("type[BaseModel]", structure_class))+ return _reflect_structure_class(structure_class=cast("type[BaseModel]", structure_class))-def reflect_structure_class(*, structure_class: type[BaseModel]) -> dict[str, ConceptStructureBlueprintType] | None:- """Reflect a registered structure class into blueprint form, faithful-or-absent.+def _reflect_structure_class(*, structure_class: type[BaseModel]) -> dict[str, ConceptStructureBlueprintType] | None:+ """Reflect a content class into blueprint form, faithful-or-absent.- Same rules as native reflection: every field must map unambiguously to a blueprint field, or- the whole reflected structure is absent (`None`) — never a guessed shape. The input-form- deriver uses this for class-backed concepts (`structure = "ClassName"`), where the class is- the only statement of the payload's fields.+ Every field must map unambiguously to a blueprint field, or the whole reflected structure is+ absent (`None`) — never a guessed shape, and never a partial one. That severity is what makes+ this a probe worth trusting: the pinned blueprint it is compared against is the normative form,+ so a structure reflected from a class that drifted must not look plausible. The input-form+ deriver reflects a class-backed concept for a different purpose and answers differently — see+ `InputFormDeriver._reflected_class_fields`."""model_fields = structure_class.model_fieldsif not model_fields:@@ -83,11 +88,11 @@ def reflect_structure_class(*, structure_class: type[BaseModel]) -> dict[str, Codef _annotation_to_blueprint(annotation: Any, *, description: str) -> ConceptStructureBlueprint:- inner, required = _strip_optional(annotation)+ inner, required = strip_optional(annotation=annotation)- if _is_number_union(inner):+ if is_number_union(annotation=inner):return ConceptStructureBlueprint(description=description, type=ConceptStructureBlueprintFieldType.NUMBER, required=required)- if get_origin(inner) in {Union, UnionType}:+ if is_union(annotation=inner):# A union that is not simply Optional[X] nor a number union has no single blueprint shape.raise _UnmappableAnnotationError@@ -99,11 +104,11 @@ def _annotation_to_blueprint(annotation: Any, *, description: str) -> ConceptStrif origin is list:return _list_blueprint(inner, description=description, required=required)- scalar_type = _scalar_field_type(inner)+ scalar_type = scalar_field_type(annotation=inner)if scalar_type is not None:return ConceptStructureBlueprint(description=description, type=scalar_type, required=required)- native_code = _native_code_for_content_class(inner)+ native_code = native_code_for_content_class(annotation=inner)if native_code is not None:return ConceptStructureBlueprint(description=description, type=ConceptStructureBlueprintFieldType.CONCEPT, concept_ref=native_code.concept_ref, required=required@@ -121,12 +126,11 @@ def _annotation_to_blueprint(annotation: Any, *, description: str) -> ConceptStrdef _list_blueprint(list_annotation: Any, *, description: str, required: bool) -> ConceptStructureBlueprint:- args = get_args(list_annotation)- if not args:+ item_inner = list_item_annotation(annotation=list_annotation)+ if item_inner is None:raise _UnmappableAnnotationError- item_inner, _ = _strip_optional(args[0])- native_code = _native_code_for_content_class(item_inner)+ native_code = native_code_for_content_class(annotation=item_inner)if native_code is not None:return ConceptStructureBlueprint(description=description,@@ -135,7 +139,7 @@ def _list_blueprint(list_annotation: Any, *, description: str, required: bool) -item_concept_ref=native_code.concept_ref,required=required,)- scalar_type = _scalar_field_type(item_inner)+ scalar_type = scalar_field_type(annotation=item_inner)if scalar_type is not None:return ConceptStructureBlueprint(description=description, type=ConceptStructureBlueprintFieldType.LIST, item_type=scalar_type, required=required@@ -143,47 +147,6 @@ def _list_blueprint(list_annotation: Any, *, description: str, required: bool) -raise _UnmappableAnnotationError-def _strip_optional(annotation: Any) -> tuple[Any, bool]:- """Return (inner, required): peel a single `X | None` into (X, False); otherwise (annotation, True)."""- if get_origin(annotation) in {Union, UnionType}:- args = get_args(annotation)- non_none = [arg for arg in args if arg is not type(None)]- if len(non_none) == 1 and len(non_none) != len(args):- return non_none[0], False- return annotation, True---def _is_number_union(annotation: Any) -> bool:- if get_origin(annotation) not in {Union, UnionType}:- return False- return set(get_args(annotation)) == {int, float}---def _scalar_field_type(annotation: Any) -> ConceptStructureBlueprintFieldType | None:- # Order matters: bool is an int subclass, datetime is a date subclass — check the narrower first.- if annotation is bool:- return ConceptStructureBlueprintFieldType.BOOLEAN- if annotation is int:- return ConceptStructureBlueprintFieldType.INTEGER- if annotation is float:- return ConceptStructureBlueprintFieldType.NUMBER- if annotation is str:- return ConceptStructureBlueprintFieldType.TEXT- if annotation is datetime.datetime:- return ConceptStructureBlueprintFieldType.DATETIME- if annotation is datetime.date:- return ConceptStructureBlueprintFieldType.DATE- if annotation is datetime.time:- return ConceptStructureBlueprintFieldType.TIME- return None---def _native_code_for_content_class(annotation: Any) -> NativeConceptCode | None:- if not isinstance(annotation, type) or not issubclass(annotation, StuffContent):- return None- return _NATIVE_CLASS_NAME_TO_CODE.get(annotation.__name__)--def _is_nested_model(annotation: Any) -> bool:"""A pydantic model that is not itself a native content class — an object on the wire."""return isinstance(annotation, type) and issubclass(annotation, BaseModel)diff --git a/pipelex/core/concepts/annotation_shapes.py b/pipelex/core/concepts/annotation_shapes.pynew file mode 100644index 000000000..5c3282a19--- /dev/null+++ b/pipelex/core/concepts/annotation_shapes.py@@ -0,0 +1,78 @@+"""How a Python type annotation maps onto the MTHDS shape vocabulary.++Two reflections in this engine read registered pydantic classes and ask an annotation the same+questions: the native consistency probe (`pipelex/codegen/native_expansion.py`), which reflects a+native's content class into blueprint form, and the input-form deriver+(`pipelex/pipeline/input_form.py`), which reflects a class-backed concept into descriptor nodes.+What they build differs, and so does what they do with an annotation neither can map — the probe+answers absent, the deriver emits an `unknown` node — but the questions themselves live here, so+the two reflections cannot come to different answers about the same annotation.+"""++import datetime+from types import UnionType+from typing import Any, Union, get_args, get_origin++from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprintFieldType+from pipelex.core.concepts.native.concept_native import NativeConceptCode+from pipelex.core.stuffs.stuff_content import StuffContent++_NATIVE_CLASS_NAME_TO_CODE: dict[str, NativeConceptCode] = {native_code.structure_class_name: native_code for native_code in NativeConceptCode}+++def strip_optional(*, annotation: Any) -> tuple[Any, bool]:+ """Return (inner, required): peel a single `X | None` into (X, False); otherwise (annotation, True)."""+ if get_origin(annotation) in {Union, UnionType}:+ args = get_args(annotation)+ non_none = [arg for arg in args if arg is not type(None)]+ if len(non_none) == 1 and len(non_none) != len(args):+ return non_none[0], False+ return annotation, True+++def is_union(*, annotation: Any) -> bool:+ """Whether the annotation is a union — asked after `strip_optional`, so `X | None` is already gone."""+ return get_origin(annotation) in {Union, UnionType}+++def is_number_union(*, annotation: Any) -> bool:+ """Whether the annotation is exactly `int | float`, the one union with a single MTHDS shape."""+ if not is_union(annotation=annotation):+ return False+ return set(get_args(annotation)) == {int, float}+++def scalar_field_type(*, annotation: Any) -> ConceptStructureBlueprintFieldType | None:+ """The scalar MTHDS field type of the annotation, or `None` when it is not a scalar."""+ # Order matters: bool is an int subclass, datetime is a date subclass — check the narrower first.+ if annotation is bool:+ return ConceptStructureBlueprintFieldType.BOOLEAN+ if annotation is int:+ return ConceptStructureBlueprintFieldType.INTEGER+ if annotation is float:+ return ConceptStructureBlueprintFieldType.NUMBER+ if annotation is str:+ return ConceptStructureBlueprintFieldType.TEXT+ if annotation is datetime.datetime:+ return ConceptStructureBlueprintFieldType.DATETIME+ if annotation is datetime.date:+ return ConceptStructureBlueprintFieldType.DATE+ if annotation is datetime.time:+ return ConceptStructureBlueprintFieldType.TIME+ return None+++def native_code_for_content_class(*, annotation: Any) -> NativeConceptCode | None:+ """The native concept a content class stands for, decided by identity — never by shape."""+ if not isinstance(annotation, type) or not issubclass(annotation, StuffContent):+ return None+ return _NATIVE_CLASS_NAME_TO_CODE.get(annotation.__name__)+++def list_item_annotation(*, annotation: Any) -> Any:+ """The element annotation of a list, already peeled of an `X | None` wrapper; `None` when unparameterized."""+ args = get_args(annotation)+ if not args:+ return None+ inner, _ = strip_optional(annotation=args[0])+ return innerdiff --git a/pipelex/pipeline/input_form.py b/pipelex/pipeline/input_form.pyindex fcd925688..9c04337af 100644--- a/pipelex/pipeline/input_form.py+++ b/pipelex/pipeline/input_form.py@@ -40,7 +40,7 @@ registry, and bundle-defined classes are only reliably current while their libra"""from collections.abc import Mapping, Sequence-from typing import Any+from typing import Any, get_originfrom annotated_types import Ge, Gt, Le, Lt, MaxLen, MinLenfrom mthds.protocol.input_form import (@@ -78,7 +78,14 @@ from pydantic import BaseModel, TypeAdapterfrom pydantic.fields import FieldInfofrom pydantic_core import PydanticUndefined-from pipelex.codegen.native_expansion import reflect_structure_class+from pipelex.core.concepts.annotation_shapes import (+ is_number_union,+ is_union,+ list_item_annotation,+ native_code_for_content_class,+ scalar_field_type,+ strip_optional,+)from pipelex.core.concepts.concept_blueprint import ConceptBlueprint, ConceptStructureBlueprintTypefrom pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprint, ConceptStructureBlueprintFieldTypefrom pipelex.core.concepts.native.concept_native import NativeConceptCode@@ -355,30 +362,134 @@ class InputFormDeriver:refines: list[str] | None,seen: frozenset[str],) -> InputFormField:- """A concept whose payload is stated by a registered class: a native class maps by identity, any other is reflected."""+ """A concept whose payload is stated by a registered class.++ A native class maps by identity, a `RootModel` IS its root value, any other class is reflected.+ """native_code = next((code for code in NativeConceptCode if code.structure_class_name == class_name), None)if native_code is not None:return self._native_node(name=name, native_code=native_code, concept_ref=concept_ref, description=description, refines=refines, seen=seen)structure_class = get_class_registry().get_class(name=class_name)if not (isinstance(structure_class, type) and issubclass(structure_class, BaseModel)):return _unknown_node(name=name, concept_ref=concept_ref, description=description, refines=refines)- reflected = reflect_structure_class(structure_class=structure_class)- if reflected is None:- if structure_class.model_fields:- # A field this reflection could not map: the payload's shape is genuinely opaque here.- return _unknown_node(name=name, concept_ref=concept_ref, description=description, refines=refines)- # A class declaring no field at all is not opaque — it states a payload that demands- # nothing, the class-backed twin of an empty authored structure table, and the form says- # so. (`reflect_structure_class` collapses both into `None` because its other consumer,- # the native consistency probe, reads `None` as "structureless by design".)- reflected = {}- fields = [+ if structure_class.__pydantic_root_model__:+ # A `RootModel` IS its root value on the wire at the top of a class-backed concept exactly+ # as it is one field down (`_reflected_node`): an `object` over the synthetic `root` field+ # would state a shape `model_validate` rejects, and would contradict the `json_schema`+ # derived from the same class beside it. The concept's own identity is stamped back on the+ # node it yields — including on a `list` root, whose `concept_ref` names this concept+ # rather than an element concept, there being none: a top-level field states the concept+ # it carries, and a root-valued concept IS the whole value.+ root_node = self._reflected_node(+ name=name,+ annotation=_root_annotation_of(root_model_class=structure_class),+ seen=seen,+ classes_seen=frozenset({structure_class}),+ )+ return root_node.model_copy(update={"concept_ref": concept_ref, "refines": refines, "description": description or root_node.description})+ return ObjectField(+ name=name,+ concept_ref=concept_ref,+ refines=refines,+ description=description,+ required=True,+ fields=self._reflected_class_fields(structure_class=structure_class, seen=seen, classes_seen=frozenset()),+ )++ # ---- Class reflection -------------------------------------------------------------------------++ def _reflected_class_fields(+ self, *, structure_class: type[BaseModel], seen: frozenset[str], classes_seen: frozenset[type[BaseModel]]+ ) -> list[InputFormField]:+ """One node per field the class declares, every annotation mapped on its own.++ Reflection here is **partial**, deliberately unlike the native consistency probe's+ faithful-or-absent rule (`native_expansion._reflect_structure_class`): an annotation with no+ honest node makes THAT field `unknown` and leaves its siblings stated. The two answers differ+ because the purposes do — the probe is compared against a normative pinned form,+ where a plausible-looking partial answer would be the failure, whereas the descriptor is+ total by contract. Collapsing the whole payload here hid every `document` and `image`+ position under a class that one sibling annotation happened to defeat, and a consumer+ preparing inputs from the descriptor would then pass a local file path through un-uploaded.++ A class declaring no field yields no nodes, and the `object` node built over them says so+ with an empty `fields` list: a payload that demands nothing — the class-backed twin of an+ empty authored structure table, not an opaque payload.+ """+ classes_seen |= {structure_class}+ return [_with_reflected_constraints(- node=self._structure_field(name=field_name, field=field, seen=seen), field_info=structure_class.model_fields[field_name]+ node=self._reflected_field(+ name=field_name,+ annotation=field_info.annotation,+ description=field_info.description or field_name,+ seen=seen,+ classes_seen=classes_seen,+ ),+ field_info=field_info,)- for field_name, field in reflected.items()+ for field_name, field_info in structure_class.model_fields.items()]- return ObjectField(name=name, concept_ref=concept_ref, refines=refines, description=description, required=True, fields=fields)++ def _reflected_field(+ self, *, name: str, annotation: Any, description: str, seen: frozenset[str], classes_seen: frozenset[type[BaseModel]]+ ) -> InputFormField:+ """One reflected field: the annotation's node, carrying the field's own description and optionality."""+ inner, required = strip_optional(annotation=annotation)+ node = self._reflected_node(name=name, annotation=inner, seen=seen, classes_seen=classes_seen)+ return node.model_copy(update={"description": description, "required": required})++ def _reflected_node(self, *, name: str, annotation: Any, seen: frozenset[str], classes_seen: frozenset[type[BaseModel]]) -> InputFormField:+ """The node one annotation maps to, before the field's own facts are stamped on it.++ The shape questions are the blueprint reflection's own (`core/concepts/annotation_shapes.py`),+ answered in descriptor terms: a native content class is that native's node — routed through+ `_concept_node`, so the concept cycle guard covers a pinned native's own fields — a nested+ non-native model is an `object` whose fields are reflected in turn, which is what keeps a+ file-bearing field one level down visible, a `RootModel` is its root annotation's node+ because the value it accepts is the root value itself, and an annotation with no honest node+ is `unknown`.+ """+ if is_number_union(annotation=annotation):+ return NumberField(name=name, required=True, integer=False)+ if is_union(annotation=annotation):+ # A union that is neither `X | None` (already peeled) nor a number union has no single node shape.+ return _unknown_node(name=name)+ origin = get_origin(annotation)+ if origin is list:+ return self._reflected_list_node(name=name, annotation=annotation, seen=seen, classes_seen=classes_seen)+ if origin is dict:+ # A mapping with unspecified value types, exactly as a `dict` structure field reports.+ return _unknown_node(name=name)+ scalar_type = scalar_field_type(annotation=annotation)+ if scalar_type is not None:+ return _scalar_field(field_type=scalar_type, name=name, required=True)+ native_code = native_code_for_content_class(annotation=annotation)+ if native_code is not None:+ return self._concept_node(name=name, concept_ref=native_code.concept_ref, seen=seen)+ if isinstance(annotation, type) and issubclass(annotation, BaseModel) and annotation not in classes_seen:+ # A nested model already on the path would recurse forever; the revisit is `unknown`,+ # the same answer the concept walk gives a concept ref it has already seen.+ nested_class: type[BaseModel] = annotation+ deeper: frozenset[type[BaseModel]] = classes_seen | {nested_class}+ if nested_class.__pydantic_root_model__:+ # A `RootModel` IS its root value on the wire, so its node is the root annotation's:+ # an `object` over the synthetic `root` field would state a shape `model_validate` rejects.+ return self._reflected_node(name=name, annotation=_root_annotation_of(root_model_class=nested_class), seen=seen, classes_seen=deeper)+ return ObjectField(+ name=name, required=True, fields=self._reflected_class_fields(structure_class=nested_class, seen=seen, classes_seen=classes_seen)+ )+ return _unknown_node(name=name)++ def _reflected_list_node(self, *, name: str, annotation: Any, seen: frozenset[str], classes_seen: frozenset[type[BaseModel]]) -> InputFormField:+ """A reflected `list[X]`: the element node derived one layer down and carried in `item`."""+ item_annotation = list_item_annotation(annotation=annotation)+ item = (+ _unknown_node(name=name)+ if item_annotation is None+ else self._reflected_node(name=name, annotation=item_annotation, seen=seen, classes_seen=classes_seen)+ )+ return ListField(name=name, concept_ref=item.concept_ref, refines=item.refines, required=True, item=_as_list_item(node=item))# ---- Crate walks ------------------------------------------------------------------------------@@ -528,6 +639,16 @@ def _as_list_item(*, node: InputFormField) -> InputFormItem:return _LIST_ITEM_ADAPTER.validate_python({slot: value for slot, value in node if slot != "name"})+def _root_annotation_of(*, root_model_class: type[BaseModel]) -> Any:+ """The annotation a `RootModel`'s value actually takes: its `root` field's, optionality peeled.++ Callers test `__pydantic_root_model__` — pydantic's own marker — rather than `issubclass(…, RootModel)`,+ which narrows to an unparameterized generic the type checker then reports as partially unknown.+ """+ annotation, _ = strip_optional(annotation=root_model_class.model_fields["root"].annotation)+ return annotation++def _with_effective_hints(*, node: InputFormField, hints: dict[str, str] | None) -> InputFormField:"""Stamp a node's FINAL effective hints; an applicable `intent` word feeds `kind`, never competes.diff --git a/subject_grants.toml b/subject_grants.tomlindex 006f6f1ac..3f561d48a 100644--- a/subject_grants.toml+++ b/subject_grants.toml@@ -1152,26 +1152,10 @@ rationale = "Single-operand conversion of the annotation; description stays keywparam = "annotation"rationale = "Predicate on the annotation; single operand."-["pipelex/codegen/native_expansion.py::_is_number_union"]-param = "annotation"-rationale = "Predicate on the annotation; single operand."-["pipelex/codegen/native_expansion.py::_list_blueprint"]param = "list_annotation"rationale = "Noun-named derivation from the list annotation; the rest stays keyword."-["pipelex/codegen/native_expansion.py::_native_code_for_content_class"]-param = "annotation"-rationale = "Noun-named derivation: inspects the annotation for its native code; single operand."--["pipelex/codegen/native_expansion.py::_scalar_field_type"]-param = "annotation"-rationale = "Noun-named derivation: inspects the annotation for its scalar type; single operand."--["pipelex/codegen/native_expansion.py::_strip_optional"]-param = "annotation"-rationale = "Single-operand transformation of the annotation."-["pipelex/codegen/native_expansion.py::collect_native_refs_from_structure"]param = "structure"rationale = "Single-operand analysis over the structure mapping."diff --git a/tests/integration/pipelex/pipeline/test_input_form.py b/tests/integration/pipelex/pipeline/test_input_form.pyindex a52c67dd8..4ecf6a7f9 100644--- a/tests/integration/pipelex/pipeline/test_input_form.py+++ b/tests/integration/pipelex/pipeline/test_input_form.py@@ -27,6 +27,8 @@ from pydantic import Fieldfrom pipelex.core.memory.exceptions import ListWhereSingularErrorfrom pipelex.core.memory.input_shaper import InputShaper, PipelineInputs+from pipelex.core.stuffs.document_content import DocumentContent+from pipelex.core.stuffs.image_content import ImageContentfrom pipelex.core.stuffs.list_content import ListContentfrom pipelex.core.stuffs.structured_content import StructuredContentfrom pipelex.interpreter_hub import (@@ -40,7 +42,7 @@ from pipelex.pipeline.input_form import FieldKind, InputFormField, build_input_ffrom pipelex.pipeline.pipe_io_contracts import IOMultiplicity, build_pipe_io_contractsfrom pipelex.pipeline.validate_bundle import validate_bundlefrom pipelex.system.registries.class_registry_access import get_class_registry-from tests.helpers.input_form import fields_by_name+from tests.helpers.input_form import as_list, fields_by_nameif TYPE_CHECKING:from collections.abc import Callable@@ -484,10 +486,20 @@ class InputFormFieldLessPayload(StructuredContent):"""A registered structure class declaring no field — a payload that demands nothing."""-class InputFormUnmappablePayload(StructuredContent):- """A structure class whose field has no single blueprint shape — reflection must stay absent, never guess."""+class InputFormAttachment(StructuredContent):+ """A nested non-native model, carrying a file-bearing field one level below the concept."""++ caption: str = Field(description="PROBE_desc_reflected_caption")+ picture: ImageContent = Field(description="PROBE_desc_reflected_picture")+++class InputFormPartlyMappablePayload(StructuredContent):+ """A structure class one annotation defeats — every other field, the file-bearing ones included, must still be stated."""payload: str | int = Field(description="PROBE_desc_reflected_payload")+ label: str = Field(description="PROBE_desc_reflected_label")+ attachment: InputFormAttachment = Field(description="PROBE_desc_reflected_attachment")+ scans: list[DocumentContent] = Field(description="PROBE_desc_reflected_scans")_KIND_TABLE_MTHDS = """@@ -498,9 +510,9 @@ description = "Bundle pinning the rest of the kind-assignment table"description = "PROBE_desc_concept_Constrained: class-backed by a hand-written constrained class"structure = "InputFormConstrainedPayload"-[concept.Unmappable]-description = "PROBE_desc_concept_Unmappable: class-backed by a class reflection cannot map"-structure = "InputFormUnmappablePayload"+[concept.PartlyMappable]+description = "PROBE_desc_concept_PartlyMappable: class-backed by a class one annotation defeats"+structure = "InputFormPartlyMappablePayload"[concept.FieldLess]description = "PROBE_desc_concept_FieldLess: class-backed by a class declaring no field"@@ -538,12 +550,12 @@ template = "$empty_in $refining_in"[pipe.edges]type = "PipeLLM"description = "Fixed-count-of-one multiplicity and class-backed reflection"-inputs = { one = "Gadget[1]", constrained = "Constrained", unmappable = "Unmappable", field_less = "FieldLess" }+inputs = { one = "Gadget[1]", constrained = "Constrained", partly_mappable = "PartlyMappable", field_less = "FieldLess" }output = "Text"prompt = \"\"\"@one@constrained-@unmappable+@partly_mappable@field_less\"\"\""""@@ -577,7 +589,7 @@ class TestKindAssignmentTable:try:registry = get_class_registry()registry.register_class(InputFormConstrainedPayload)- registry.register_class(InputFormUnmappablePayload)+ registry.register_class(InputFormPartlyMappablePayload)registry.register_class(InputFormFieldLessPayload)result = await validate_bundle(mthds_contents=[_KIND_TABLE_MTHDS])return build_input_form(result.pipes)@@ -736,12 +748,42 @@ class TestKindAssignmentTable:assert field_less.required is Trueassert field_less.gating is True- async def test_unmappable_class_falls_back_to_unknown(self, load_empty_library: Callable[[], str]) -> None:- """Reflection is faithful-or-absent: a class with an unmappable field yields `unknown`, with identity kept."""+ async def test_an_unmappable_annotation_is_unknown_on_that_field_alone(self, load_empty_library: Callable[[], str]) -> None:+ """Class reflection is partial: the field no annotation maps is `unknown`, its siblings stay stated."""input_form = await self._derive_kind_table(load_empty_library)- unmappable = _field_by_name(input_form["input_form_kinds.edges"], "unmappable")+ partly_mappable = _field_by_name(input_form["input_form_kinds.edges"], "partly_mappable")++ assert partly_mappable.kind == FieldKind.OBJECT, "One defeated annotation no longer collapses the whole payload"+ assert partly_mappable.concept_ref == "input_form_kinds.PartlyMappable"+ assert partly_mappable.description is not None+ assert "PROBE_desc_concept_PartlyMappable" in partly_mappable.description+ by_name = fields_by_name(partly_mappable)- assert unmappable.kind == FieldKind.UNKNOWN- assert unmappable.concept_ref == "input_form_kinds.Unmappable"- assert unmappable.description is not None- assert "PROBE_desc_concept_Unmappable" in unmappable.description+ payload = by_name["payload"]+ assert payload.kind == FieldKind.UNKNOWN, "A union that is neither optional nor numeric has no honest node"+ assert payload.description == "PROBE_desc_reflected_payload", "The unmappable field keeps its identity and helper text"+ assert by_name["label"].kind == FieldKind.TEXT, "The mappable sibling is stated, not lost with it"++ async def test_file_positions_below_a_defeated_sibling_stay_visible(self, load_empty_library: Callable[[], str]) -> None:+ """What partial reflection is for: a consumer preparing inputs must still see every file position.++ The collapse hid these — an `image` nested in a plain model, a list of `document` — behind the+ one sibling annotation the reflection could not map, and a local path at such a position went+ through un-uploaded.+ """+ input_form = await self._derive_kind_table(load_empty_library)+ by_name = fields_by_name(_field_by_name(input_form["input_form_kinds.edges"], "partly_mappable"))++ attachment = by_name["attachment"]+ assert attachment.kind == FieldKind.OBJECT, "A nested non-native model is walked into, not flattened to an opaque dict"+ assert attachment.description == "PROBE_desc_reflected_attachment"+ nested = fields_by_name(attachment)+ assert nested["caption"].kind == FieldKind.TEXT+ assert nested["picture"].kind == FieldKind.IMAGE, "The file position one level down is reported"+ assert nested["picture"].concept_ref == "native.Image", "A content class maps to its native by identity"+ assert nested["picture"].description == "PROBE_desc_reflected_picture", "The field's own description wins over the native's"++ scans = as_list(by_name["scans"])+ assert scans.concept_ref == "native.Document", "A list of a content class names the element concept"+ assert scans.item.kind == FieldKind.DOCUMENT+ assert not hasattr(scans.item, "name"), "A list item has no authored name"diff --git a/tests/unit/pipelex/pipeline/test_input_form_deriver.py b/tests/unit/pipelex/pipeline/test_input_form_deriver.pyindex 545a25aaf..bcc8888ff 100644--- a/tests/unit/pipelex/pipeline/test_input_form_deriver.py+++ b/tests/unit/pipelex/pipeline/test_input_form_deriver.py@@ -1,16 +1,49 @@"""Unit-pin the deriver's escape hatches that the library loader keeps out of reach.-The loader rejects concept cycles (`LibraryLoadingError`) and a `structure = "ClassName"` naming an-unregistered class before a bundle ever reaches `build_input_form`, so those branches are exercised-directly on a hand-built qualified crate: the derivation must stay total and finite on any crate.+The loader rejects concept cycles (`LibraryLoadingError`), a `structure = "ClassName"` naming an+unregistered class, and a registered class that holds itself (which it reads as a concept cycle),+before a bundle ever reaches `build_input_form` — so those branches are exercised directly on a+hand-built qualified crate: the derivation must stay total and finite on any crate."""from __future__ import annotations+from pydantic import Field, RootModel+from pipelex.core.concepts.concept_blueprint import ConceptBlueprintfrom pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprint, ConceptStructureBlueprintFieldType+from pipelex.core.stuffs.structured_content import StructuredContentfrom pipelex.pipeline.input_form import FieldKind, InputFormDeriver-from tests.helpers.input_form import as_object+from pipelex.system.registries.class_registry_access import get_class_registry+from tests.helpers.input_form import as_list, as_object, fields_by_name+++class SelfReferentialPayload(StructuredContent):+ """A registered structure class that holds itself — reflection must cut the path, not recurse forever."""++ label: str = Field(description="The label")+ child: "SelfReferentialPayload | None" = Field(default=None, description="The nested twin")+++class RootBackedTags(RootModel[list[str]]):+ """A `RootModel` field: the value on the wire IS the root list, never an object over a `root` key."""+++class RootBackedPayload(StructuredContent):+ """A registered structure class holding a `RootModel` field."""++ label: str = Field(description="The label")+ tags: RootBackedTags = Field(description="The tags")+++# mypy reads the two bases' `model_construct` overloads as incompatible; pydantic builds these classes+# fine and pyright accepts them, and a class-backed concept over a `RootModel` is what these tests pin.+class RootBackedSlug(StructuredContent, RootModel[str]): # type: ignore[misc]+ """A registered structure class that IS a `RootModel` — the concept's whole payload is the root string."""+++class RootBackedTagList(StructuredContent, RootModel[list[str]]): # type: ignore[misc]+ """A registered structure class that IS a `RootModel` over a list — the payload is the list itself."""def _concept_field(*, concept_ref: str) -> ConceptStructureBlueprint:@@ -72,3 +105,72 @@ class TestInputFormDeriverEscapeHatches:assert node.kind == FieldKind.UNKNOWNassert node.refines == ["dep->other.Score"]assert node.description == "Refines a dependency's score"++ def test_a_self_referential_class_is_cut_at_the_revisit(self) -> None:+ """Class reflection terminates: the position that revisits the class is `unknown`, its siblings stated."""+ concepts: dict[str, ConceptBlueprint | str] = {+ "demo.Recursive": ConceptBlueprint(description="Backed by a class that holds itself", structure="SelfReferentialPayload"),+ }+ registry = get_class_registry()+ registry.register_class(SelfReferentialPayload)+ try:+ node = InputFormDeriver(concepts=concepts).derive_concept(name="recursive", concept_ref="demo.Recursive")+ finally:+ registry.unregister_class(SelfReferentialPayload)++ assert node.kind == FieldKind.OBJECT+ by_name = fields_by_name(node)+ assert by_name["label"].kind == FieldKind.TEXT+ assert by_name["child"].kind == FieldKind.UNKNOWN+ assert by_name["child"].required is False, "The self-reference is optional, and cutting the path does not change that"++ def test_a_root_model_field_reports_its_root_annotation(self) -> None:+ """A `RootModel` is its root value on the wire, so the node is the root's — never an `object` over `root`."""+ concepts: dict[str, ConceptBlueprint | str] = {+ "demo.RootBacked": ConceptBlueprint(description="Backed by a class holding a RootModel", structure="RootBackedPayload"),+ }+ registry = get_class_registry()+ registry.register_class(RootBackedPayload)+ try:+ node = InputFormDeriver(concepts=concepts).derive_concept(name="root_backed", concept_ref="demo.RootBacked")+ finally:+ registry.unregister_class(RootBackedPayload)++ by_name = fields_by_name(node)+ assert by_name["label"].kind == FieldKind.TEXT+ tags = as_list(by_name["tags"])+ assert tags.item.kind == FieldKind.TEXT+ assert tags.description == "The tags", "The field's own description survives the root reflection"++ def test_a_root_model_backed_concept_reports_its_root_annotation(self) -> None:+ """A class-backed concept whose class IS a `RootModel` states the root's node, never an `object` over `root`."""+ concepts: dict[str, ConceptBlueprint | str] = {+ "demo.Slug": ConceptBlueprint(description="A slug", structure="RootBackedSlug"),+ }+ registry = get_class_registry()+ registry.register_class(RootBackedSlug)+ try:+ node = InputFormDeriver(concepts=concepts).derive_concept(name="slug", concept_ref="demo.Slug")+ finally:+ registry.unregister_class(RootBackedSlug)++ assert node.kind == FieldKind.TEXT, "The payload `model_validate` accepts is the root string itself"+ assert node.concept_ref == "demo.Slug", "A top-level field states the concept it carries"+ assert node.description == "A slug"++ def test_a_root_model_backed_concept_over_a_list_is_a_list_node(self) -> None:+ """The root annotation decides the kind: a list root is a `list` node carrying the concept's own ref."""+ concepts: dict[str, ConceptBlueprint | str] = {+ "demo.TagList": ConceptBlueprint(description="The tags", structure="RootBackedTagList"),+ }+ registry = get_class_registry()+ registry.register_class(RootBackedTagList)+ try:+ node = InputFormDeriver(concepts=concepts).derive_concept(name="tag_list", concept_ref="demo.TagList")+ finally:+ registry.unregister_class(RootBackedTagList)++ tag_list = as_list(node)+ assert tag_list.item.kind == FieldKind.TEXT+ assert tag_list.concept_ref == "demo.TagList", "A root-valued concept IS the whole value, so its ref rides the list node"+ assert tag_list.description == "The tags"