Skip to content

Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.

Stop four wrap serializers from erasing their serialization JSON Schema

Louis Choquel4d23517f4b22
diff --git a/AGENTS.md b/AGENTS.md
index ecee6cb0b..4fe056dae 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -210,6 +210,8 @@
- Use Field(default_factory=...) for mutable defaults
- Use `# pyright: ignore[specificError]` or `# type: ignore` only as a last resort. In particular, if you are sure about the type, you often solve issues by using cast() or creating a new typed variable.
+One scoped exception to "every function return must be typed": a pydantic `@model_serializer` / `@field_serializer` return must stay **unannotated** unless the annotation is the exact type the model publishes. Pydantic turns a serializer's return annotation into the model's serialization JSON Schema and prefers it over the schema it would generate, so a `-> dict[str, Any]` publishes the whole model as an opaque `{"type": "object", "additionalProperties": true}` — no properties, no `required`, no closed shape — for every consumer that generates response schemas, which FastAPI always does for its responses. A precise annotation is fine and desirable (`-> str` on a model that serializes to its raw string); a `dict`/`Any` one erases the model, and nothing in a normal test suite notices because validation-mode schemas and `model_dump()` both stay correct.
+
#### Data-holder shapes (NamedTuple, dataclass, BaseModel)
The deciding question is **does this value ever get serialized** — sent across a process or network boundary, persisted to disk or a database, or round-tripped through (de)serialization? If yes, it must be one of the two serialization-safe forms (a `BaseModel` or a pydantic dataclass): a `NamedTuple` or stdlib `@dataclass` loses its type on the way through and silently corrupts the data across the boundary. (A distributed executor like Temporal is the canonical case — its payloads cross the wire between worker processes — but the same applies to any cache, queue, API response, or on-disk record.) Stdlib `@dataclass` is banned outright; pick one of these four forms:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index f6d14b915..6986ee052 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,5 +1,11 @@
# Changelog
+## [Unreleased]
+
+### Fixed
+
+- **Blueprint serialization schemas**: `ConceptBlueprint`, `ConceptStructureBlueprint`, `InputSlotBlueprint` and `ConstructBlueprint` publish their real shape again in serialization mode. Each carries a `@model_serializer(mode="wrap")` annotated `-> dict[str, Any]`, and pydantic turns a serializer's return annotation into the model's serialization JSON Schema in preference to the one it would generate — so all four came back as an opaque `{"type": "object", "additionalProperties": true}` with no properties, no `required` and no closed shape. This is the same annotation `mthds` 0.11.1 dropped for the input-form field descriptors. Nothing here noticed, because validation-mode schemas and `model_dump()` were both correct throughout; the damage landed in consumers that generate response-model schemas, which FastAPI always does, so a downstream OpenAPI artifact regains its field-level detail on its next `pipelex` bump. A new sweep test holds every Pipelex model to the rule, and the agent rules now scope "every function return must be typed" so a serializer's return stays unannotated.
+
## [v0.54.0] - 2026-08-28
### Added
diff --git a/pipelex/core/concepts/concept_blueprint.py b/pipelex/core/concepts/concept_blueprint.py
index 2ba1c9bd8..707fbbc79 100644
--- a/pipelex/core/concepts/concept_blueprint.py
+++ b/pipelex/core/concepts/concept_blueprint.py
@@ -31,11 +31,17 @@ class ConceptBlueprint(BaseModel):
return dict(sorted(hints.items())) if hints else None
@model_serializer(mode="wrap")
- def serialize_without_absent_hints(self, handler: SerializerFunctionWrapHandler) -> dict[str, Any]:
+ def serialize_without_absent_hints(self, handler: SerializerFunctionWrapHandler):
"""Absent hints are an absent member — never `null` (spec rule; keeps hint-free crate
fingerprints byte-identical to before hints existed). Deliberately NOT a blanket
`exclude_none`: the crate spec's general absent-members canonicalization is a separate
forward contract whose adoption would migrate every existing fingerprint.
+
+ The return is deliberately unannotated, and stays that way. Pydantic turns a serializer's
+ return annotation into the model's serialization JSON Schema and prefers it over the schema
+ it would generate, so a `-> dict[str, Any]` here publishes this model as an opaque
+ `{"type": "object", "additionalProperties": true}` — no properties, no `required`, no
+ closed shape — for every consumer that generates response schemas (FastAPI always does).
"""
dumped: dict[str, Any] = handler(self)
if dumped.get("hints") is None:
diff --git a/pipelex/core/concepts/concept_structure_blueprint.py b/pipelex/core/concepts/concept_structure_blueprint.py
index b2f3f1df0..7638ac1b6 100644
--- a/pipelex/core/concepts/concept_structure_blueprint.py
+++ b/pipelex/core/concepts/concept_structure_blueprint.py
@@ -82,9 +82,12 @@ class ConceptStructureBlueprint(BaseModel):
return dict(sorted(hints.items())) if hints else None
@model_serializer(mode="wrap")
- def serialize_without_absent_hints(self, handler: SerializerFunctionWrapHandler) -> dict[str, Any]:
+ def serialize_without_absent_hints(self, handler: SerializerFunctionWrapHandler):
"""Absent hints are an absent member — never `null` (spec rule; keeps hint-free crate
fingerprints byte-identical to before hints existed).
+
+ The return is deliberately unannotated — see `ConceptBlueprint.serialize_without_absent_hints`:
+ an annotation here becomes the model's serialization JSON Schema and erases its shape.
"""
dumped: dict[str, Any] = handler(self)
if dumped.get("hints") is None:
diff --git a/pipelex/kit/agent_rules/python_standards.md b/pipelex/kit/agent_rules/python_standards.md
index a173c81c6..12e040f67 100644
--- a/pipelex/kit/agent_rules/python_standards.md
+++ b/pipelex/kit/agent_rules/python_standards.md
@@ -95,6 +95,8 @@
- Use Field(default_factory=...) for mutable defaults
- Use `# pyright: ignore[specificError]` or `# type: ignore` only as a last resort. In particular, if you are sure about the type, you often solve issues by using cast() or creating a new typed variable.
+One scoped exception to "every function return must be typed": a pydantic `@model_serializer` / `@field_serializer` return must stay **unannotated** unless the annotation is the exact type the model publishes. Pydantic turns a serializer's return annotation into the model's serialization JSON Schema and prefers it over the schema it would generate, so a `-> dict[str, Any]` publishes the whole model as an opaque `{"type": "object", "additionalProperties": true}` — no properties, no `required`, no closed shape — for every consumer that generates response schemas, which FastAPI always does for its responses. A precise annotation is fine and desirable (`-> str` on a model that serializes to its raw string); a `dict`/`Any` one erases the model, and nothing in a normal test suite notices because validation-mode schemas and `model_dump()` both stay correct.
+
### Data-holder shapes (NamedTuple, dataclass, BaseModel)
The deciding question is **does this value ever get serialized** — sent across a process or network boundary, persisted to disk or a database, or round-tripped through (de)serialization? If yes, it must be one of the two serialization-safe forms (a `BaseModel` or a pydantic dataclass): a `NamedTuple` or stdlib `@dataclass` loses its type on the way through and silently corrupts the data across the boundary. (A distributed executor like Temporal is the canonical case — its payloads cross the wire between worker processes — but the same applies to any cache, queue, API response, or on-disk record.) Stdlib `@dataclass` is banned outright; pick one of these four forms:
diff --git a/pipelex/pipe_machinery/pipe_blueprint.py b/pipelex/pipe_machinery/pipe_blueprint.py
index 8238b8ba9..30483c2ff 100644
--- a/pipelex/pipe_machinery/pipe_blueprint.py
+++ b/pipelex/pipe_machinery/pipe_blueprint.py
@@ -205,10 +205,13 @@ class InputSlotBlueprint(BaseModel):
return dict(sorted(hints.items())) if hints else None
@model_serializer(mode="wrap")
- def serialize_without_absent_hints(self, handler: SerializerFunctionWrapHandler) -> dict[str, Any]:
+ def serialize_without_absent_hints(self, handler: SerializerFunctionWrapHandler):
"""Absent hints are an absent member — never `null` (spec rule). Unreachable for a
hint-free slot in practice (the parse-time collapse leaves it a plain string), kept so any
directly-constructed instance still serializes canonically.
+
+ The return is deliberately unannotated — see `ConceptBlueprint.serialize_without_absent_hints`:
+ an annotation here becomes the model's serialization JSON Schema and erases its shape.
"""
dumped: dict[str, Any] = handler(self)
if dumped.get("hints") is None:
diff --git a/pipelex/pipe_operators/compose/construct_blueprint.py b/pipelex/pipe_operators/compose/construct_blueprint.py
index b5b9e643e..b0d63a745 100644
--- a/pipelex/pipe_operators/compose/construct_blueprint.py
+++ b/pipelex/pipe_operators/compose/construct_blueprint.py
@@ -279,11 +279,14 @@ class ConstructBlueprint(BaseModel):
return {field_name: field_bp.to_mthds_dict() for field_name, field_bp in self.fields.items()}
@model_serializer(mode="wrap")
- def serialize_with_context(self, handler: SerializerFunctionWrapHandler, info: SerializationInfo) -> dict[str, Any]:
+ def serialize_with_context(self, handler: SerializerFunctionWrapHandler, info: SerializationInfo):
"""Serialize with format-aware context.
When context contains {"format": "mthds"}, outputs MTHDS-format dict.
Otherwise, uses default Pydantic serialization.
+
+ The return is deliberately unannotated — see `ConceptBlueprint.serialize_without_absent_hints`:
+ an annotation here becomes the model's serialization JSON Schema and erases its shape.
"""
if info.context and info.context.get("format") == "mthds":
return self.to_mthds_dict()
diff --git a/tests/unit/pipelex/test_serialization_json_schema.py b/tests/unit/pipelex/test_serialization_json_schema.py
new file mode 100644
index 000000000..8fd7dd31e
--- /dev/null
+++ b/tests/unit/pipelex/test_serialization_json_schema.py
@@ -0,0 +1,197 @@
+"""No Pipelex model publishes an opaque serialization JSON Schema.
+
+Pydantic derives a model's *serialization* schema from its `@model_serializer`'s return
+annotation and prefers it over the schema it would generate. A `-> dict[str, Any]` therefore
+replaces the real shape with `{"type": "object", "additionalProperties": true}` — no properties,
+no `required`, no closed shape — while validation-mode schemas stay complete and every runtime
+`model_dump()` keeps working. Nothing in this repo notices: the suite is green, and no call to
+`model_json_schema()` in `pipelex/` asks for serialization mode (the MTHDS schema generator asks
+for `mode="validation"` explicitly). The damage lands one repo away, in a consumer that generates
+response-model schemas — FastAPI always does — where a whole blueprint reaches clients as an
+opaque object and a client generator produces nothing useful from it.
+
+The bug class already recurred: it hit four models here and the input-form field descriptors
+upstream in `mthds`. Hence a sweep rather than four hand-written assertions — the next model to
+grow a wrap serializer is caught the day it is written, not the next time a consumer regenerates
+an artifact.
+"""
+
+from __future__ import annotations
+
+import importlib
+import pkgutil
+from typing import Any, cast
+
+import pytest
+from pydantic import BaseModel
+from pydantic.errors import PydanticInvalidForJsonSchema
+
+import pipelex
+from pipelex.core.concepts.concept_blueprint import ConceptBlueprint
+from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprint
+from pipelex.pipe_machinery.pipe_blueprint import InputSlotBlueprint
+from pipelex.pipe_operators.compose.construct_blueprint import ConstructBlueprint
+
+# The models that carry a `@model_serializer(mode="wrap")` and once published nothing because of it.
+WRAP_SERIALIZER_MODELS = [
+ ConceptBlueprint,
+ ConceptStructureBlueprint,
+ InputSlotBlueprint,
+ ConstructBlueprint,
+]
+
+# `FuncRegistry` holds live callables, which have no JSON Schema at all — pydantic refuses to build
+# one. Pinned by name rather than skipped silently: a model that newly stops being describable drops
+# out of the sweep, and this is what makes that visible instead of quietly shrinking the guard.
+MODELS_WITHOUT_A_JSON_SCHEMA = {
+ "pipelex.system.registries.func_registry.FuncRegistry",
+}
+
+REMEDY = (
+ "Drop the return annotation from the model's `@model_serializer` — pydantic turns it into the "
+ "model's serialization JSON Schema and prefers it over the generated one."
+)
+
+
+def _import_every_pipelex_module() -> tuple[int, list[str]]:
+ """Import every `pipelex.*` module, returning how many loaded and the failures.
+
+ Failures are returned rather than swallowed: a sweep that quietly skips what it cannot import
+ is a guard that has gone quiet exactly where the tree got interesting.
+ """
+ imported = 0
+ failures: list[str] = []
+ for module_info in pkgutil.walk_packages(pipelex.__path__, prefix=f"{pipelex.__name__}."):
+ try:
+ importlib.import_module(module_info.name)
+ except Exception as exc:
+ failures.append(f"{module_info.name}: {type(exc).__name__}: {exc}")
+ else:
+ imported += 1
+ return imported, failures
+
+
+def _discover_pipelex_models() -> list[type[BaseModel]]:
+ """Every `BaseModel` subclass declared under `pipelex.`, after loading the whole package."""
+ _import_every_pipelex_module()
+
+ seen: set[type[BaseModel]] = set()
+
+ def walk(cls: type[BaseModel]) -> None:
+ for sub in cls.__subclasses__():
+ if sub not in seen:
+ seen.add(sub)
+ walk(sub)
+
+ walk(BaseModel)
+ models = [cls for cls in seen if cls.__module__.startswith(f"{pipelex.__name__}.")]
+ return sorted(models, key=lambda cls: f"{cls.__module__}.{cls.__qualname__}")
+
+
+def _resolve_root(schema: dict[str, Any]) -> dict[str, Any]:
+ """Follow a top-level `$ref` into `$defs`.
+
+ A self-referential model (`ConstructBlueprint` nests itself) publishes `{"$defs": …, "$ref": …}`
+ at the top, so reading `properties` off the raw schema would judge every recursive model opaque.
+ """
+ ref = schema.get("$ref")
+ if not isinstance(ref, str):
+ return schema
+ definition_name = ref.rsplit("/", maxsplit=1)[-1]
+ defs: dict[str, Any] = schema.get("$defs") or {}
+ resolved: Any = defs.get(definition_name)
+ if isinstance(resolved, dict):
+ return cast("dict[str, Any]", resolved)
+ return schema
+
+
+def _is_opaque_object(schema: dict[str, Any]) -> bool:
+ """The exact shape an erasing return annotation produces: an object that says nothing."""
+ root = _resolve_root(schema)
+ return root.get("type") == "object" and not root.get("properties") and root.get("additionalProperties") is True
+
+
+def _erases_its_shape(model: type[BaseModel]) -> bool:
+ """The model is opaque when serialized but not when validated — the erasure signature.
+
+ Asymmetry is what makes this a defect rather than a fact. A model that is honestly open —
+ `extra="allow"` with no declared fields (`CompositeContent`), or a `RootModel[dict[str, Any]]`
+ (`SdkClientRegistry`) — is opaque in *both* modes, and an open dict really is its shape. Testing
+ the asymmetry instead of the shape is what lets this sweep run without an allowlist to maintain,
+ and an allowlist is the wrong instrument here anyway: it would have to be extended by the very
+ person adding the next erasing serializer.
+ """
+ return _is_opaque_object(model.model_json_schema(mode="serialization")) and not _is_opaque_object(model.model_json_schema(mode="validation"))
+
+
+def _sweep_serialization_shapes() -> dict[str, bool]:
+ """Every describable Pipelex model, mapped to whether it erases its shape when serialized."""
+ verdicts: dict[str, bool] = {}
+ for model in _discover_pipelex_models():
+ qualified_name = f"{model.__module__}.{model.__qualname__}"
+ if qualified_name in MODELS_WITHOUT_A_JSON_SCHEMA:
+ continue
+ verdicts[qualified_name] = _erases_its_shape(model)
+ return verdicts
+
+
+def _models_without_a_json_schema() -> set[str]:
+ """The models pydantic refuses to describe at all, discovered rather than assumed."""
+ undescribable: set[str] = set()
+ for model in _discover_pipelex_models():
+ try:
+ model.model_json_schema(mode="serialization")
+ except PydanticInvalidForJsonSchema:
+ undescribable.add(f"{model.__module__}.{model.__qualname__}")
+ return undescribable
+
+
+class TestSerializationJsonSchema:
+ def test_discovery_is_not_vacuous(self) -> None:
+ """Anti-vacuity: the walk really loaded the package and really found models.
+
+ Deliberately not parametrized — pytest reports `got empty parameter set` and exits 0 on an
+ empty list, so the guard would be unreachable exactly when it matters.
+ """
+ imported, failures = _import_every_pipelex_module()
+ assert not failures, "Modules failed to import, so the sweep below is silently narrower than it looks:\n" + "\n".join(failures)
+ assert imported, "No pipelex module imported — the sweep would compare two empty sets and pass vacuously."
+
+ models = _discover_pipelex_models()
+ assert models, "No pipelex BaseModel discovered — the sweep would pass vacuously."
+
+ discovered_names = {cls.__qualname__ for cls in models}
+ assert "ConceptBlueprint" in discovered_names, "The blueprint layer is missing from the sweep — discovery is not reaching the wire surface."
+
+ assert _models_without_a_json_schema() == MODELS_WITHOUT_A_JSON_SCHEMA, (
+ "The set of models pydantic cannot describe has moved. A model that newly stops being describable silently "
+ "leaves the sweep below, so update MODELS_WITHOUT_A_JSON_SCHEMA deliberately — after checking the model "
+ f"really has no wire form.\n expected: {sorted(MODELS_WITHOUT_A_JSON_SCHEMA)}\n actual: {sorted(_models_without_a_json_schema())}"
+ )
+
+ def test_no_pipelex_model_publishes_an_opaque_serialization_schema(self) -> None:
+ """A model whose serialization schema says nothing is invisible to every client generator."""
+ offenders = [name for name, erased in _sweep_serialization_shapes().items() if erased]
+
+ assert not offenders, (
+ "These models describe themselves when validated but publish an opaque object when serialized:\n"
+ + "\n".join(f" - {name}" for name in offenders)
+ + f"\n\n{REMEDY}"
+ )
+
+ @pytest.mark.parametrize("model", WRAP_SERIALIZER_MODELS, ids=lambda model: model.__qualname__)
+ def test_wrap_serializer_models_keep_their_property_lists(self, model: type[BaseModel]) -> None:
+ """Serialization mode describes the same fields as validation mode.
+
+ Sharper than the opaque-shape sweep: it catches a *partial* erasure, where a schema keeps a
+ property or two and loses the rest, which the sweep above would wave through.
+ """
+ serialization_properties = sorted(_resolve_root(model.model_json_schema(mode="serialization")).get("properties", {}))
+ validation_properties = sorted(_resolve_root(model.model_json_schema(mode="validation")).get("properties", {}))
+
+ assert serialization_properties, f"{model.__qualname__} publishes no properties in serialization mode. {REMEDY}"
+ assert serialization_properties == validation_properties, (
+ f"{model.__qualname__} describes different fields depending on the schema mode.\n"
+ f" serialization: {serialization_properties}\n"
+ f" validation: {validation_properties}\n\n{REMEDY}"
+ )