Skip to content

Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.

refactor: import the standard's wire models for the input-form descriptor and the pipe I/O contracts (#1154)

Louis Choquel6354c7df2dbb
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 637de90fd..a710c2dfc 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,5 +1,13 @@
# Changelog
+## [Unreleased]
+
+### Changed
+
+- **The input-form descriptor and the pipe I/O contracts are typed by the standard now (Breaking for importers)**: The wire models of both artifacts are declared once, by the MTHDS standard's Python client, and Pipelex imports them instead of restating them. `pipelex/pipeline/pipe_io_contracts.py` re-exports `PipeInputContract`, `PipeOutputContract`, `PipeIOContract` and `IOMultiplicity` from `mthds.protocol.pipe_io_contracts`; `pipelex/pipeline/input_form.py` re-exports `FieldKind`, `PipeInputFormDescriptor` and the field models from `mthds.protocol.input_form`; and `pipelex/core/pipes/variable_multiplicity.py` re-exports `PresenceMarker` from the contracts module, since the marker is part of the contract's wire vocabulary. **The wire is unchanged** — every field name, every value, and every absent-versus-null rule is the one the engine already emitted, and the derivation is the same derivation. What moved is which package declares the shape. Importers are affected in three ways: a field descriptor is now one model per kind (`TextField`, `ProseField`, `DateField`, `NumberField`, `BooleanField`, `EnumField`, `DocumentField`, `ImageField`, `ObjectField`, `ListField`, `UnknownField`) under an `InputFormField` union discriminated on `kind`, so a node is narrowed with `isinstance` or a `match` on its class rather than read off a flat model; `FieldKind.is_list` and `FieldKind.is_object` are gone with that narrowing; and the `date` kind's Python attribute is now `datetime`, the name it always had on the wire, rather than `datetime_flag`. Because the engine constructs the standard's own closed shapes, a drift between what it emits and what the standard defines now fails at derivation instead of silently on the wire. See [Input-form descriptor](under-the-hood/input-form-descriptor.md) and [Pipe I/O contracts](under-the-hood/pipe-io-contracts.md).
+- **`PresenceMarker`'s symbol grammar moved off the enum (Breaking)**: The marker enum is the standard's, so the three helpers that are this engine's `.mthds` suffix grammar rather than wire vocabulary became module-level functions in `pipelex/core/pipes/variable_multiplicity.py`: `PresenceMarker.from_symbol(...)` is `presence_from_symbol(symbol=...)`, `marker.symbol` is `presence_symbol(presence=marker)`, and `marker.is_force` is `is_force_presence(presence=marker)`, all keyword-only per this repo's convention. `is_optional` and `is_plain` stay properties on the enum, which the standard declares.
+- **Minimum `mthds` version**: The floor moves to `mthds>=0.9.0`, the release that carries `mthds/protocol/pipe_io_contracts.py` and `mthds/protocol/input_form.py`.
+
## [v0.53.0] - 2026-08-25
### Added
diff --git a/docs/under-the-hood/input-form-descriptor.md b/docs/under-the-hood/input-form-descriptor.md
index adb77cba7..7133cd762 100644
--- a/docs/under-the-hood/input-form-descriptor.md
+++ b/docs/under-the-hood/input-form-descriptor.md
@@ -6,7 +6,8 @@ The descriptor exists because the emitted `json_schema` is a *payload* contract,
## Where it lives
-- `pipelex/pipeline/input_form.py` — the wire models (`FieldKind`, `InputFormField`, `PipeInputFormDescriptor`), the one public derivation `build_input_form(pipes)`, and the `InputFormDeriver` behind it.
+- `mthds.protocol.input_form` — **the wire models**, declared by the MTHDS standard's Python client and mirroring its `input-form-descriptor` page: `FieldKind`, one model per kind (`TextField`, `ProseField`, `DateField`, `NumberField`, `BooleanField`, `EnumField`, `DocumentField`, `ImageField`, `ObjectField`, `ListField`, `UnknownField`), their `InputFormField` union discriminated on `kind`, and `PipeInputFormDescriptor`. Pipelex does not declare a second copy of them.
+- `pipelex/pipeline/input_form.py` — the one public derivation `build_input_form(pipes)` and the `InputFormDeriver` behind it, which import and re-export those models. A node's kind IS its model, so the deriver constructs the per-kind model and a consumer narrows a node with `isinstance` or a `match` on its class.
- `PipelexValidationReport.input_form` (`pipelex/pipeline/validation_report.py`) — a **required** field, keyed exactly like `pipe_io_contracts`. `build_validation_report` requires it as a keyword: the shared-assembly rule says a report field is populated on every backend or none, so a backend that forgets it fails loudly instead of shipping an empty view.
- `pipelex/pipeline/validate_in_process.py` — the in-process assembly derives it inside the validation library's window, right beside `build_pipe_io_contracts`.
- `pipelex/codegen/native_expansion.py` — `reflect_structure_class`, the faithful-or-absent reflection of a registered structure class into blueprint form, shared with the native consistency probe.
@@ -63,8 +64,8 @@ On a top-level field, `required` is `presence != "optional"`, and `gating` — w
## Wire shape
-Inapplicable slots are absent, never JSON `null`: the report's valid arm is dumped without `exclude_none`, so `InputFormField` owns its wire shape through a serializer that drops `None` values and writes the `date` kind's flag under its spec name `datetime`. Applicable falsy values (`required: false`, `integer: false`, `gating: false`) are kept. Per-kind validators keep a node honest at construction (`enum` needs `choices`, `object` needs `fields`, `list` needs `item`, `number` needs `integer`, `date` needs `datetime`).
+Inapplicable slots are absent, never JSON `null`: the report's valid arm is dumped without `exclude_none`, so a field model owns its wire shape through a serializer that drops `None` values. Applicable falsy values (`required: false`, `integer: false`, `gating: false`) are kept. Every shape is closed (`extra="forbid"`), and a kind's own slots are simply that model's required fields — an `enum` node carries `choices`, an `object` node `fields`, a `list` node `item`, a `number` node `integer`, a `date` node `datetime` — so a node that does not belong to its kind cannot be constructed at all. Two further invariants are checked at construction: a field never carries both `required: true` and a `default_value`, and `presence` and `gating` are stated on every top-level field and on nothing below it.
## Seeing it
-`pipelex-dev trace-input-semantics` captures the descriptors as `hop5_input_form.json` beside `hop5_pipe_io_contracts.json`, so an authored fact can be checked on both projections at once — see [Tracing Input Semantics](../contribute/trace-input-semantics.md). The assignment table is pinned by `tests/integration/pipelex/pipeline/test_input_form.py` over the committed probe bundle (and the hints behavior over its hinted sibling, `hinted_bundle.mthds`), the wire models by `tests/unit/pipelex/pipeline/test_input_form_models.py`, the hint merges and kind-feeding by `tests/unit/pipelex/pipeline/test_input_form_hints.py`, and the escape hatches the library loader keeps unreachable (concept cycles, unregistered classes) by `tests/unit/pipelex/pipeline/test_input_form_deriver.py`.
+`pipelex-dev trace-input-semantics` captures the descriptors as `hop5_input_form.json` beside `hop5_pipe_io_contracts.json`, so an authored fact can be checked on both projections at once — see [Tracing Input Semantics](../contribute/trace-input-semantics.md). The assignment table is pinned by `tests/integration/pipelex/pipeline/test_input_form.py` over the committed probe bundle (and the hints behavior over its hinted sibling, `hinted_bundle.mthds`), the emitted wire's null-free shape by `tests/unit/pipelex/pipeline/test_input_form_models.py`, the hint merges and kind-feeding by `tests/unit/pipelex/pipeline/test_input_form_hints.py`, and the escape hatches the library loader keeps unreachable (concept cycles, unregistered classes) by `tests/unit/pipelex/pipeline/test_input_form_deriver.py`.
diff --git a/docs/under-the-hood/pipe-io-contracts.md b/docs/under-the-hood/pipe-io-contracts.md
index 122eed0bb..3d45050e0 100644
--- a/docs/under-the-hood/pipe-io-contracts.md
+++ b/docs/under-the-hood/pipe-io-contracts.md
@@ -6,7 +6,8 @@ Reach for `pipe_io_contracts` when you need the schema and the declared contract
## Where it lives
-- `pipelex/pipeline/pipe_io_contracts.py` — the wire models (`PipeInputContract`, `PipeOutputContract`, `IOMultiplicity`) and the one public derivation, `build_pipe_io_contracts(pipes)`.
+- `mthds.protocol.pipe_io_contracts` — **the wire models** (`PipeInputContract`, `PipeOutputContract`, `PipeIOContract`, `IOMultiplicity`, `PresenceMarker`), declared by the MTHDS standard's Python client and mirroring its `pipe-io-contracts` page. Pipelex does not declare a second copy of them.
+- `pipelex/pipeline/pipe_io_contracts.py` — the one public derivation, `build_pipe_io_contracts(pipes)`, which imports and re-exports those models. Because it constructs the standard's own closed shapes, a projection that drifted from the standard fails at derivation rather than on the wire.
- `PipelexValidationReport.pipe_io_contracts` (`pipelex/pipeline/validation_report.py`) — where the artifact travels.
- `pipelex/pipeline/validate_in_process.py` — the in-process assembly derives it inside the validation library's window, right beside `build_input_form`. Iterating the same loaded pipes is what makes the two key sets equal by construction.
diff --git a/pipelex/builder/pipe/pipe_spec.py b/pipelex/builder/pipe/pipe_spec.py
index e7e25df7c..aca457f09 100644
--- a/pipelex/builder/pipe/pipe_spec.py
+++ b/pipelex/builder/pipe/pipe_spec.py
@@ -11,7 +11,13 @@ from pipelex import log
from pipelex.cogt.content_generation.dry_run_factory import MockFormat
from pipelex.core.concepts.exceptions import ConceptStringError
from pipelex.core.concepts.validation import validate_concept_ref_or_code
-from pipelex.core.pipes.variable_multiplicity import MULTIPLICITY_PATTERN, PresenceMarker, parse_concept_with_multiplicity
+from pipelex.core.pipes.variable_multiplicity import (
+ MULTIPLICITY_PATTERN,
+ is_force_presence,
+ parse_concept_with_multiplicity,
+ presence_from_symbol,
+ presence_symbol,
+)
from pipelex.core.stuffs.structured_content import StructuredContent
from pipelex.pipe_machinery.pipe_blueprint import PipeBlueprint, PipeCategory, PipeType, valid_pipe_type_tags
from pipelex.tools.misc.pretty import PrettyPrintable
@@ -101,7 +107,7 @@ class PipeSpec(StructuredContent):
# Extract concept without multiplicity/presence markers for validation
parse_result = parse_concept_with_multiplicity(output)
# Mirror the blueprint grammar rules (D1, D4): `!` never on outputs, `?` never with multiplicity
- if parse_result.presence.is_force:
+ if is_force_presence(presence=parse_result.presence):
msg = (
f"Invalid output: '{output}'. The force marker '!' is not allowed on outputs — "
f"it is a use-site assertion for inputs. To declare that this pipe may produce no value, use '?'."
@@ -146,11 +152,11 @@ class PipeSpec(StructuredContent):
# Mirror the blueprint grammar rule (D1, D4): presence markers never combine with multiplicity
bracket_content = match.group(2)
- presence = PresenceMarker.from_symbol(match.group(3))
+ presence = presence_from_symbol(symbol=match.group(3))
if not presence.is_plain and bracket_content is not None:
msg = (
f"Invalid input '{input_name}': '{concept_spec}'. "
- f"The presence marker '{presence.symbol}' cannot be combined with multiplicity: "
+ f"The presence marker '{presence_symbol(presence=presence)}' cannot be combined with multiplicity: "
f"a plural slot is never absent — when nothing is found, it is the empty list."
)
raise ValueError(msg)
diff --git a/pipelex/core/pipes/inputs/input_stuff_specs_factory.py b/pipelex/core/pipes/inputs/input_stuff_specs_factory.py
index 9b6e60cce..b080a4a81 100644
--- a/pipelex/core/pipes/inputs/input_stuff_specs_factory.py
+++ b/pipelex/core/pipes/inputs/input_stuff_specs_factory.py
@@ -8,7 +8,7 @@ from pipelex.core.concepts.validation import validate_concept_ref_or_code
from pipelex.core.pipes.inputs.exceptions import InputStuffSpecsFactoryError
from pipelex.core.pipes.inputs.input_stuff_specs import InputStuffSpecs, PipeInputsRoot
from pipelex.core.pipes.stuff_spec.stuff_spec import StuffSpec
-from pipelex.core.pipes.variable_multiplicity import PresenceMarker
+from pipelex.core.pipes.variable_multiplicity import presence_from_symbol
if TYPE_CHECKING:
from pipelex.core.pipes.variable_multiplicity import VariableMultiplicity
@@ -106,4 +106,4 @@ class InputStuffSpecsFactory:
# else: No brackets, multiplicity stays None
concept = concept_provider.get_required_concept(concept_ref=concept_ref_with_domain)
- return StuffSpec(concept=concept, multiplicity=multiplicity, presence=PresenceMarker.from_symbol(marker_symbol))
+ return StuffSpec(concept=concept, multiplicity=multiplicity, presence=presence_from_symbol(symbol=marker_symbol))
diff --git a/pipelex/core/pipes/variable_multiplicity.py b/pipelex/core/pipes/variable_multiplicity.py
index 0f313e5a1..a30f3f429 100644
--- a/pipelex/core/pipes/variable_multiplicity.py
+++ b/pipelex/core/pipes/variable_multiplicity.py
@@ -1,12 +1,29 @@
from __future__ import annotations
import re
-from enum import StrEnum
+from mthds.protocol.pipe_io_contracts import PresenceMarker
from pydantic import BaseModel, Field
from pipelex.core.pipes.exceptions import PipeVariableMultiplicityError
+__all__ = [
+ "MULTIPLICITY_PATTERN",
+ "MultiplicityParseResult",
+ "PresenceMarker",
+ "VariableMultiplicity",
+ "VariableMultiplicityResolution",
+ "fixed_item_count",
+ "format_concept_with_multiplicity",
+ "is_force_presence",
+ "is_multiple_multiplicity",
+ "is_multiplicity_compatible",
+ "make_variable_multiplicity",
+ "parse_concept_with_multiplicity",
+ "presence_from_symbol",
+ "presence_symbol",
+]
+
VariableMultiplicity = bool | int
# The io-ref suffix grammar: an optional multiplicity suffix ("[]" or "[N]") followed by an
@@ -16,61 +33,43 @@ VariableMultiplicity = bool | int
MULTIPLICITY_PATTERN = r"^([a-zA-Z_][a-zA-Z0-9_]*(?:\.[a-zA-Z_][a-zA-Z0-9_]*)*)(?:\[(\d*)\])?([?!])?$"
-class PresenceMarker(StrEnum):
- """Presence of a value in a declared slot: plain (always present), optional (`?` — the slot may
- legitimately hold no value), or force (`!` — a use-site assertion that the value must be present).
+def presence_from_symbol(*, symbol: str | None) -> PresenceMarker:
+ """The presence marker an io-ref suffix symbol denotes: nothing is `plain`, `?` optional, `!` force.
+
+ The symbol grammar is this engine's parser, not the standard's wire vocabulary, so it lives here
+ beside `MULTIPLICITY_PATTERN` rather than on the enum the standard declares.
"""
+ match symbol:
+ case None | "":
+ return PresenceMarker.PLAIN
+ case "?":
+ return PresenceMarker.OPTIONAL
+ case "!":
+ return PresenceMarker.FORCE
+ case _:
+ msg = f"Invalid presence marker symbol: '{symbol}'. Expected '?', '!', or nothing."
+ raise PipeVariableMultiplicityError(msg)
+
- PLAIN = "plain"
- OPTIONAL = "optional"
- FORCE = "force"
-
- @classmethod
- def from_symbol(cls, symbol: str | None) -> PresenceMarker:
- match symbol:
- case None | "":
- return PresenceMarker.PLAIN
- case "?":
- return PresenceMarker.OPTIONAL
- case "!":
- return PresenceMarker.FORCE
- case _:
- msg = f"Invalid presence marker symbol: '{symbol}'. Expected '?', '!', or nothing."
- raise PipeVariableMultiplicityError(msg)
-
- @property
- def symbol(self) -> str:
- match self:
- case PresenceMarker.PLAIN:
- return ""
- case PresenceMarker.OPTIONAL:
- return "?"
- case PresenceMarker.FORCE:
- return "!"
-
- @property
- def is_optional(self) -> bool:
- match self:
- case PresenceMarker.OPTIONAL:
- return True
- case PresenceMarker.PLAIN | PresenceMarker.FORCE:
- return False
-
- @property
- def is_force(self) -> bool:
- match self:
- case PresenceMarker.FORCE:
- return True
- case PresenceMarker.PLAIN | PresenceMarker.OPTIONAL:
- return False
-
- @property
- def is_plain(self) -> bool:
- match self:
- case PresenceMarker.PLAIN:
- return True
- case PresenceMarker.OPTIONAL | PresenceMarker.FORCE:
- return False
+def presence_symbol(*, presence: PresenceMarker) -> str:
+ """The io-ref suffix symbol a presence marker renders as — the inverse of `presence_from_symbol`."""
+ match presence:
+ case PresenceMarker.PLAIN:
+ return ""
+ case PresenceMarker.OPTIONAL:
+ return "?"
+ case PresenceMarker.FORCE:
+ return "!"
+
+
+def is_force_presence(*, presence: PresenceMarker | None) -> bool:
+ """Whether a marker is the force assertion (`!`), an unstated marker included as not forced.
+
+ The standard's enum answers `is_optional` and `is_plain`, the two questions a wire consumer asks.
+ The force assertion is the authored claim this engine's lints and blueprint checks read, so the
+ third question is answered here.
+ """
+ return presence is PresenceMarker.FORCE
class VariableMultiplicityResolution(BaseModel):
@@ -201,7 +200,7 @@ def parse_concept_with_multiplicity(concept_ref_or_code: str) -> MultiplicityPar
return MultiplicityParseResult(
concept_ref_or_code=extracted_concept,
multiplicity=multiplicity,
- presence=PresenceMarker.from_symbol(marker_symbol),
+ presence=presence_from_symbol(symbol=marker_symbol),
)
@@ -305,4 +304,4 @@ def format_concept_with_multiplicity(
else:
# Fixed-length list - brackets with number
multiplicity_suffix = f"[{multiplicity}]"
- return f"{concept_code_or_string}{multiplicity_suffix}{presence.symbol}"
+ return f"{concept_code_or_string}{multiplicity_suffix}{presence_symbol(presence=presence)}"
diff --git a/pipelex/libraries/crate_qualification.py b/pipelex/libraries/crate_qualification.py
index 639dce54e..1738a7a03 100644
--- a/pipelex/libraries/crate_qualification.py
+++ b/pipelex/libraries/crate_qualification.py
@@ -40,7 +40,7 @@ from typing import TYPE_CHECKING, NamedTuple
from pipelex.core.concepts.concept_blueprint import ConceptBlueprint, ConceptStructureBlueprintType
from pipelex.core.concepts.concept_factory import ConceptFactory
from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprint
-from pipelex.core.pipes.variable_multiplicity import MultiplicityParseResult, parse_concept_with_multiplicity
+from pipelex.core.pipes.variable_multiplicity import MultiplicityParseResult, parse_concept_with_multiplicity, presence_symbol
from pipelex.core.qualified_ref import QualifiedRef
from pipelex.libraries.exceptions import CrateNormalizationError
from pipelex.libraries.library_crate import LibraryCrate
@@ -207,7 +207,7 @@ def _render_ref_with_markers(concept_ref: str, *, parsed: MultiplicityParseResul
suffix = f"[{parsed.multiplicity}]"
else:
suffix = ""
- return f"{concept_ref}{suffix}{parsed.presence.symbol}"
+ return f"{concept_ref}{suffix}{presence_symbol(presence=parsed.presence)}"
def _qualify_pipe_ref(pipe_ref: str, *, owner_domain: str) -> str:
diff --git a/pipelex/pipe_machinery/pipe_blueprint.py b/pipelex/pipe_machinery/pipe_blueprint.py
index 396a5ff84..47fefe4a0 100644
--- a/pipelex/pipe_machinery/pipe_blueprint.py
+++ b/pipelex/pipe_machinery/pipe_blueprint.py
@@ -12,8 +12,10 @@ from pipelex.core.pipes.exceptions import PipeValidationError
from pipelex.core.pipes.variable_multiplicity import (
MULTIPLICITY_PATTERN,
PipeVariableMultiplicityError,
- PresenceMarker,
+ is_force_presence,
parse_concept_with_multiplicity,
+ presence_from_symbol,
+ presence_symbol,
)
from pipelex.pipe_machinery.validation import validate_input_name
from pipelex.validation_error_types import PipeValidationErrorType
@@ -370,11 +372,11 @@ class PipeBlueprint(ABC, BaseModel):
# D1/D4: presence markers are mutually exclusive with multiplicity — a plural slot's
# "nothing" is the empty list, so there is nothing for `?` or `!` to say.
bracket_content = match.group(2)
- presence = PresenceMarker.from_symbol(match.group(3))
+ presence = presence_from_symbol(symbol=match.group(3))
if not presence.is_plain and bracket_content is not None:
msg = (
f"Invalid input '{input_name}': '{concept_spec}'. "
- f"The presence marker '{presence.symbol}' cannot be combined with multiplicity: "
+ f"The presence marker '{presence_symbol(presence=presence)}' cannot be combined with multiplicity: "
f"a plural slot is never absent — when nothing is found, it is the empty list."
)
raise PipeValidationError(
@@ -403,7 +405,7 @@ class PipeBlueprint(ABC, BaseModel):
raise ValueError(msg) from exc
# D1: `!` is a use-site assertion — it is meaningless on outputs (a producer doesn't unwrap).
- if output_parse_result.presence.is_force:
+ if is_force_presence(presence=output_parse_result.presence):
msg = (
f"Invalid output: '{self.output}'. The force marker '!' is not allowed on outputs — "
f"it is a use-site assertion for inputs. To declare that this pipe may produce no value, use '?'."
diff --git a/pipelex/pipeline/advisory_warnings.py b/pipelex/pipeline/advisory_warnings.py
index dde8af257..a74337b6c 100644
--- a/pipelex/pipeline/advisory_warnings.py
+++ b/pipelex/pipeline/advisory_warnings.py
@@ -34,7 +34,7 @@ from pipelex.pipe_machinery.pipe_abstract import PipeAbstract
from pipelex.pipeline.blueprint_selection import collect_entry_pipe_refs
from pipelex.pipeline.controller_taint import ControllerTaintAnalysis, collect_controller_taint_analyses
from pipelex.pipeline.hint_warnings import build_hint_warnings
-from pipelex.pipeline.input_form import PipeInputFormDescriptor, build_input_form, qualify_current_library_crate
+from pipelex.pipeline.input_form import InputForm, build_input_form, qualify_current_library_crate
from pipelex.pipeline.optionality_warnings import build_optionality_warnings
from pipelex.pipeline.vacuous_presence_warnings import build_vacuous_presence_warnings
@@ -42,7 +42,7 @@ from pipelex.pipeline.vacuous_presence_warnings import build_vacuous_presence_wa
def build_advisory_warnings(
*,
taint_analyses: Sequence[ControllerTaintAnalysis],
- input_form: dict[str, PipeInputFormDescriptor],
+ input_form: InputForm,
entry_pipe_refs: Iterable[str],
qualified_crate: QualifiedCrateContent,
) -> list[ValidationErrorItem]:
diff --git a/pipelex/pipeline/input_form.py b/pipelex/pipeline/input_form.py
index e633c14f3..1c2d6786a 100644
--- a/pipelex/pipeline/input_form.py
+++ b/pipelex/pipeline/input_form.py
@@ -7,6 +7,17 @@ concept tables, and no description matching. It is keyed by the same namespaced
`pipe_io_contracts` — both are built from the same loaded pipes, so the key spaces are equal by
construction.
+**The wire shapes belong to the standard, not to this engine.** `FieldKind`, the per-kind field
+models, their `InputFormField` union and `PipeInputFormDescriptor` are the models of
+`mthds.protocol.input_form`, mirroring the standard's `input-form-descriptor` page; they are
+imported and re-exported here so this module keeps its callers, and this engine holds no second
+declaration of them. What stays here is the derivation: how an authored library becomes a
+descriptor. Because a node's kind IS its model, the deriver constructs the per-kind model rather
+than passing a `kind`, and the models' own parse-time invariants — the closed shapes, the rule
+that `presence` and `gating` are stated on every top-level field and nowhere below it, the rule
+that a fixed `item_count` is at least two — gate this emission at derivation time rather than on
+the wire.
+
The descriptor is derived from **authored facts, never from the emitted JSON Schema**. Two fact
sources feed it:
@@ -27,14 +38,31 @@ registry, and bundle-defined classes are only reliably current while their libra
"""
from collections.abc import Mapping, Sequence
-from enum import StrEnum
from typing import Any
from annotated_types import Ge, Gt, Le, Lt, MaxLen, MinLen
-from pydantic import BaseModel, Field, SerializerFunctionWrapHandler, model_serializer, model_validator
+from mthds.protocol.input_form import (
+ BooleanField,
+ DateField,
+ DocumentField,
+ EnumField,
+ FieldKind,
+ ImageField,
+ InputForm,
+ InputFormField,
+ InputFormFieldBase,
+ ListField,
+ NumberField,
+ ObjectField,
+ PipeInputFormDescriptor,
+ ProseField,
+ TextField,
+ TextValuedFieldBase,
+ UnknownField,
+)
+from pydantic import BaseModel
from pydantic.fields import FieldInfo
from pydantic_core import PydanticUndefined
-from typing_extensions import Self
from pipelex.codegen.native_expansion import reflect_structure_class
from pipelex.core.concepts.concept_blueprint import ConceptBlueprint, ConceptStructureBlueprintType
@@ -42,7 +70,7 @@ from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBl
from pipelex.core.concepts.native.concept_native import NativeConceptCode
from pipelex.core.concepts.native.pinned_blueprints import make_pinned_native_blueprint
from pipelex.core.pipes.stuff_spec.stuff_spec import StuffSpec
-from pipelex.core.pipes.variable_multiplicity import PresenceMarker, fixed_item_count
+from pipelex.core.pipes.variable_multiplicity import fixed_item_count
from pipelex.interpreter_hub import get_current_library, get_library_manager
from pipelex.language.intent_hints import HintSiteValueKind, IntentWord, applicable_intent, merge_hints
from pipelex.libraries.crate_qualification import QualifiedCrateContent, qualify_crate
@@ -51,178 +79,31 @@ from pipelex.pipe_machinery.pipe_abstract import PipeAbstract
from pipelex.pipe_machinery.pipe_blueprint import InputSlotBlueprint
from pipelex.system.registries.class_registry_access import get_class_registry
-
-class FieldKind(StrEnum):
- """The closed `kind` union of the input-form descriptor — field intents, never widget names."""
-
- TEXT = "text"
- PROSE = "prose"
- DATE = "date"
- NUMBER = "number"
- BOOLEAN = "boolean"
- ENUM = "enum"
- DOCUMENT = "document"
- IMAGE = "image"
- OBJECT = "object"
- LIST = "list"
- UNKNOWN = "unknown"
-
- @property
- def is_list(self) -> bool:
- match self:
- case FieldKind.LIST:
- return True
- case (
- FieldKind.TEXT
- | FieldKind.PROSE
- | FieldKind.DATE
- | FieldKind.NUMBER
- | FieldKind.BOOLEAN
- | FieldKind.ENUM
- | FieldKind.DOCUMENT
- | FieldKind.IMAGE
- | FieldKind.OBJECT
- | FieldKind.UNKNOWN
- ):
- return False
-
- @property
- def is_object(self) -> bool:
- match self:
- case FieldKind.OBJECT:
- return True
- case (
- FieldKind.TEXT
- | FieldKind.PROSE
- | FieldKind.NUMBER
- | FieldKind.BOOLEAN
- | FieldKind.DATE
- | FieldKind.ENUM
- | FieldKind.DOCUMENT
- | FieldKind.IMAGE
- | FieldKind.LIST
- | FieldKind.UNKNOWN
- ):
- return False
-
-
-class InputFormField(BaseModel):
- """One field descriptor: a recursive node discriminated on `kind`.
-
- Every wire name is the spec's snake_case slot name. Inapplicable slots are dropped at
- serialization rather than emitted as JSON `null` — the report's valid arm is dumped without
- `exclude_none`, so the model owns its own wire shape. Applicable falsy values (`required: false`,
- `integer: false`, ...) are kept.
- """
-
- kind: FieldKind = Field(strict=False)
- name: str
- """The identifier as authored: the input slot name on a top-level field, the structure field
- name on a nested one. Unused on a `list`'s `item` (the index labels items)."""
-
- title: str | None = None
- """Human label; a renderer falls back to `name`. Never the generated class name."""
-
- concept_ref: str | None = None
- """The namespaced concept ref on every concept-typed node (`native.Document`, `demo.Invoice`).
- On a `list` node it names the ELEMENT concept."""
-
- refines: list[str] | None = None
- """The concept's refinement chain, immediate parent first, walked to its end. Absent when the
- concept refines nothing."""
-
- description: str | None = None
- required: bool
- """Top-level: the caller must supply the slot (`presence != "optional"`). Nested: the field must
- be present within the concept's payload. The two levels never interact."""
-
- presence: PresenceMarker | None = Field(default=None, strict=False)
- """The authored presence marker of the pipe's input slot, three-valued so `!` is not flattened.
- Top-level fields only."""
-
- gating: bool | None = None
- """Top-level fields only: the run cannot start until this slot has content. Stated rather than
- re-derived from `required` — a variable-multiplicity list never gates (`[]` is its legitimate
- value) while a fixed-count one does."""
-
- default_value: Any | None = None
- """The value applied when the caller omits the field — present only when a default was authored,
- never the emission's `null`-for-optional artifact. Always beside `required: false`: the blueprint
- rejects `required = true` with a default, and a reflected default makes the field not required."""
-
- examples: list[Any] | None = None
- hints: dict[str, str] | None = None
- """The node's effective MTHDS intent hints (spec: intent-hints.md): the key-by-key merge of the
- concept's refinement chain and the site's own hints, nearer/site layer winning. Flat
- string-to-string by contract. Everything well-formed rides here, unknown entries included; an
- applicable `intent` word additionally feeds `kind` (never competes with it)."""
-
- # `text` / `prose` constraint slots.
- min_length: int | None = None
- max_length: int | None = None
- pattern: str | None = None
- format: str | None = None
- # `date`: the `datetime` wire slot (the attribute name avoids shadowing the stdlib module).
- datetime_flag: bool | None = None
- # `number` slots.
- integer: bool | None = None
- minimum: float | None = None
- maximum: float | None = None
- exclusive_minimum: float | None = None
- exclusive_maximum: float | None = None
- # `enum`.
- choices: list[Any] | None = None
- # `object`.
- fields: list["InputFormField"] | None = None
- # `list`.
- item: "InputFormField | None" = None
- item_count: int | None = None
- """Present exactly when the slot was authored with a fixed `[N]` multiplicity."""
-
- @model_validator(mode="after")
- def validate_per_kind_slots(self) -> Self:
- match self.kind:
- case FieldKind.ENUM:
- if not self.choices:
- msg = f"Field '{self.name}' of kind 'enum' must carry a non-empty 'choices' list"
- raise ValueError(msg)
- case FieldKind.OBJECT:
- if self.fields is None:
- msg = f"Field '{self.name}' of kind 'object' must carry 'fields'"
- raise ValueError(msg)
- case FieldKind.LIST:
- if self.item is None:
- msg = f"Field '{self.name}' of kind 'list' must carry 'item'"
- raise ValueError(msg)
- case FieldKind.NUMBER:
- if self.integer is None:
- msg = f"Field '{self.name}' of kind 'number' must state 'integer'"
- raise ValueError(msg)
- case FieldKind.DATE:
- if self.datetime_flag is None:
- msg = f"Field '{self.name}' of kind 'date' must state 'datetime'"
- raise ValueError(msg)
- case FieldKind.TEXT | FieldKind.PROSE | FieldKind.BOOLEAN | FieldKind.DOCUMENT | FieldKind.IMAGE | FieldKind.UNKNOWN:
- pass
- return self
-
- @model_serializer(mode="wrap")
- def serialize_without_inapplicable_slots(self, handler: SerializerFunctionWrapHandler) -> dict[str, Any]:
- dumped: dict[str, Any] = handler(self)
- wire = {slot: value for slot, value in dumped.items() if value is not None}
- if "datetime_flag" in wire:
- wire["datetime"] = wire.pop("datetime_flag")
- return wire
-
-
-class PipeInputFormDescriptor(BaseModel):
- """The input form of one pipe — an `input_form` entry."""
-
- fields: list[InputFormField]
- """One field descriptor per input slot, in authored input order. Empty for a pipe with no inputs."""
-
-
-def build_input_form(pipes: Sequence[PipeAbstract], *, qualified_crate: QualifiedCrateContent | None = None) -> dict[str, PipeInputFormDescriptor]:
+__all__ = [
+ "BooleanField",
+ "DateField",
+ "DocumentField",
+ "EnumField",
+ "FieldKind",
+ "ImageField",
+ "InputForm",
+ "InputFormDeriver",
+ "InputFormField",
+ "InputFormFieldBase",
+ "ListField",
+ "NumberField",
+ "ObjectField",
+ "PipeInputFormDescriptor",
+ "ProseField",
+ "TextField",
+ "TextValuedFieldBase",
+ "UnknownField",
+ "build_input_form",
+ "qualify_current_library_crate",
+]
+
+
+def build_input_form(pipes: Sequence[PipeAbstract], *, qualified_crate: QualifiedCrateContent | None = None) -> InputForm:
"""Derive the `input_form` descriptors of loaded pipes from the authored blueprints of the current library.
Works on any loaded `PipeAbstract`, `PipeSignature` placeholders included, iterating exactly
@@ -243,7 +124,7 @@ def build_input_form(pipes: Sequence[PipeAbstract], *, qualified_crate: Qualifie
"""
qualified = qualified_crate if qualified_crate is not None else qualify_current_library_crate()
deriver = InputFormDeriver(concepts=qualified.concepts)
- input_form: dict[str, PipeInputFormDescriptor] = {}
+ input_form: InputForm = {}
for pipe in pipes:
# Slot hints come from the qualified blueprint, not the runtime spec: `StuffSpec` stays
# hint-free by design (structural non-normativity). A pipe with no blueprint in the crate
@@ -296,8 +177,7 @@ class InputFormDeriver:
# A plural slot's merged hints ride the `list` node AND its `item` (the `concept_ref`
# duplication precedent): applicability is judged per item, and a renderer reading
# either node finds the same answer.
- node = InputFormField(
- kind=FieldKind.LIST,
+ node = ListField(
name=name,
concept_ref=node.concept_ref,
refines=node.refines,
@@ -311,7 +191,7 @@ class InputFormDeriver:
node = _with_effective_hints(node=node, hints=effective_hints)
presence = stuff_spec.presence
required = not presence.is_optional
- gating = required and not (node.kind.is_list and node.item_count is None)
+ gating = required and not (isinstance(node, ListField) and node.item_count is None)
return node.model_copy(update={"presence": presence, "required": required, "gating": gating})
# ---- Concept nodes ----------------------------------------------------------------------------
@@ -363,8 +243,7 @@ class InputFormDeriver:
merged_structure = self._merged_structure(concept_ref=concept_ref, chain=chain)
if merged_structure is not None:
fields = [self._structure_field(name=field_name, field=field, seen=seen) for field_name, field in merged_structure.items()]
- return InputFormField(
- kind=FieldKind.OBJECT,
+ return ObjectField(
name=name,
concept_ref=concept_ref,
refines=refines,
@@ -406,47 +285,35 @@ class InputFormDeriver:
) -> InputFormField:
"""The node of a native concept, or of a concept whose chain bottoms at one (identity-decided, never shape-sniffed)."""
pinned = make_pinned_native_blueprint(native_code)
- kind: FieldKind
- integer: bool | None = None
- datetime_flag: bool | None = None
- text_format: str | None = None
- fields: list[InputFormField] | None = None
+ node_ref = concept_ref or native_code.concept_ref
+ text = description or pinned.description
match native_code:
case NativeConceptCode.TEXT | NativeConceptCode.HTML:
- kind = FieldKind.PROSE
+ return ProseField(name=name, concept_ref=node_ref, refines=refines, description=text, required=True)
case NativeConceptCode.NUMBER:
- kind = FieldKind.NUMBER
- integer = False
+ return NumberField(name=name, concept_ref=node_ref, refines=refines, description=text, required=True, integer=False)
case NativeConceptCode.YES_NO:
- kind = FieldKind.BOOLEAN
+ return BooleanField(name=name, concept_ref=node_ref, refines=refines, description=text, required=True)
case NativeConceptCode.DATE:
- kind = FieldKind.DATE
- datetime_flag = False
+ return DateField(name=name, concept_ref=node_ref, refines=refines, description=text, required=True, datetime=False)
case NativeConceptCode.TIME:
- kind = FieldKind.TEXT
- text_format = "time"
+ return TextField(name=name, concept_ref=node_ref, refines=refines, description=text, required=True, format="time")
case NativeConceptCode.DOCUMENT:
- kind = FieldKind.DOCUMENT
+ return DocumentField(name=name, concept_ref=node_ref, refines=refines, description=text, required=True)
case NativeConceptCode.IMAGE:
- kind = FieldKind.IMAGE
+ return ImageField(name=name, concept_ref=node_ref, refines=refines, description=text, required=True)
case NativeConceptCode.PAGE | NativeConceptCode.TEXT_AND_IMAGES | NativeConceptCode.SEARCH_RESULT:
- kind = FieldKind.OBJECT
pinned_structure = pinned.structure if isinstance(pinned.structure, dict) else {}
- fields = [self._structure_field(name=field_name, field=field, seen=seen) for field_name, field in pinned_structure.items()]
+ return ObjectField(
+ name=name,
+ concept_ref=node_ref,
+ refines=refines,
+ description=text,
+ required=True,
+ fields=[self._structure_field(name=field_name, field=field, seen=seen) for field_name, field in pinned_structure.items()],
+ )
case NativeConceptCode.DYNAMIC | NativeConceptCode.ANYTHING | NativeConceptCode.JSON | NativeConceptCode.COMPOSITE:
- kind = FieldKind.UNKNOWN
- return InputFormField(
- kind=kind,
- name=name,
- concept_ref=concept_ref or native_code.concept_ref,
- refines=refines,
- description=description or pinned.description,
- required=True,
- integer=integer,
- datetime_flag=datetime_flag,
- format=text_format,
- fields=fields,
- )
+ return UnknownField(name=name, concept_ref=node_ref, refines=refines, description=text, required=True)
def _class_backed_node(
self,
@@ -481,9 +348,7 @@ class InputFormDeriver:
)
for field_name, field in reflected.items()
]
- return InputFormField(
- kind=FieldKind.OBJECT, name=name, concept_ref=concept_ref, refines=refines, description=description, required=True, fields=fields
- )
+ return ObjectField(name=name, concept_ref=concept_ref, refines=refines, description=description, required=True, fields=fields)
# ---- Crate walks ------------------------------------------------------------------------------
@@ -537,10 +402,9 @@ class InputFormDeriver:
"""
if isinstance(field, str):
# The shorthand `field = "description"` form declares a required text field.
- return InputFormField(kind=FieldKind.TEXT, name=name, description=field, required=True)
+ return TextField(name=name, description=field, required=True)
if field.choices:
- enum_node = InputFormField(
- kind=FieldKind.ENUM,
+ enum_node = EnumField(
name=name,
description=field.description,
required=field.required,
@@ -560,8 +424,7 @@ class InputFormDeriver:
# The merged hints ride the `list` node AND its `item` (the `concept_ref`
# duplication precedent); a concept item already carries its concept layer.
effective_hints = merge_hints([item.hints, field.hints])
- return InputFormField(
- kind=FieldKind.LIST,
+ return ListField(
name=name,
concept_ref=item.concept_ref,
refines=item.refines,
@@ -618,22 +481,52 @@ def _with_effective_hints(*, node: InputFormField, hints: dict[str, str] | None)
"""Stamp a node's FINAL effective hints; an applicable `intent` word feeds `kind`, never competes.
Call exactly once per node, at the terminal where the full merge is known (slot, field, or bare
- concept). `node.kind` is still the no-hint default there, so an absent, unknown, or inapplicable
- intent leaves it untouched — and `rating` / `quantity` never change `kind` at all (both map to
- `number`; the union has no finer kind). Inapplicable and unknown content still rides the slot as
- preserved content (the advisory lint already warned).
+ concept). The node is still on its no-hint default kind there, so an absent, unknown, or
+ inapplicable intent leaves it untouched — and `rating` / `quantity` never change the kind at all
+ (both map to `number`; the union has no finer kind). Inapplicable and unknown content still rides
+ the slot as preserved content (the advisory lint already warned).
"""
if not hints:
return node
- updates: dict[str, Any] = {"hints": hints}
match applicable_intent(hints, site_kind=_node_site_kind(node)):
case IntentWord.PROSE:
- updates["kind"] = FieldKind.PROSE
+ return _recast_text_kind(node=node, hints=hints, to_prose=True)
case IntentWord.LABEL:
- updates["kind"] = FieldKind.TEXT
+ return _recast_text_kind(node=node, hints=hints, to_prose=False)
case IntentWord.RATING | IntentWord.QUANTITY | None:
- pass
- return node.model_copy(update=updates)
+ return node.model_copy(update={"hints": hints})
+
+
+def _recast_text_kind(*, node: InputFormField, hints: dict[str, str], to_prose: bool) -> InputFormField:
+ """Rebuild a text-valued node as the kind an intent word asks for, carrying its slots across.
+
+ A node's kind IS its model since the shapes came from the standard, so flipping `text` to `prose`
+ (or back) is a rebuild rather than a field update. Only a text-valued site reaches here — `prose`
+ and `label` apply nowhere else — so the source is always a `text` or `prose` node and its own
+ extra slots are the shared text constraints, which carry over unchanged.
+ """
+ if not isinstance(node, TextValuedFieldBase):
+ # Unreachable while `_node_site_kind` reports text-valued for the text kinds alone; stated
+ # rather than asserted, so a future kind that reports text-valued degrades to a hint stamp.
+ return node.model_copy(update={"hints": hints})
+ kind_model = ProseField if to_prose else TextField
+ return kind_model(
+ name=node.name,
+ title=node.title,
+ concept_ref=node.concept_ref,
+ refines=node.refines,
+ description=node.description,
+ required=node.required,
+ presence=node.presence,
+ gating=node.gating,
+ default_value=node.default_value,
+ examples=node.examples,
+ hints=hints,
+ min_length=node.min_length,
+ max_length=node.max_length,
+ pattern=node.pattern,
+ format=node.format,
+ )
def _node_site_kind(node: InputFormField) -> HintSiteValueKind:
@@ -645,26 +538,17 @@ def _node_site_kind(node: InputFormField) -> HintSiteValueKind:
text-valued EXCEPT a time-formatted text (`type = "time"` is neither) and an `Html`-backed node
(`prose` presentation, but the chain reaches `native.Html`, not `native.Text`).
"""
- match node.kind:
- case FieldKind.NUMBER:
+ match node:
+ case NumberField():
return HintSiteValueKind.NUMBER_VALUED
- case FieldKind.TEXT | FieldKind.PROSE:
+ case TextField() | ProseField():
if node.format is not None:
return HintSiteValueKind.OTHER
html_ref = NativeConceptCode.HTML.concept_ref
if node.concept_ref == html_ref or (node.refines is not None and html_ref in node.refines):
return HintSiteValueKind.OTHER
return HintSiteValueKind.TEXT_VALUED
- case (
- FieldKind.DATE
- | FieldKind.BOOLEAN
- | FieldKind.ENUM
- | FieldKind.DOCUMENT
- | FieldKind.IMAGE
- | FieldKind.OBJECT
- | FieldKind.LIST
- | FieldKind.UNKNOWN
- ):
+ case DateField() | BooleanField() | EnumField() | DocumentField() | ImageField() | ObjectField() | ListField() | UnknownField():
return HintSiteValueKind.OTHER
@@ -676,43 +560,24 @@ def _scalar_field(
description: str | None = None,
default_value: Any | None = None,
) -> InputFormField:
- """The node of a scalar blueprint type: its kind and per-kind flags (a compound type maps to `unknown`)."""
- kind: FieldKind
- integer: bool | None = None
- datetime_flag: bool | None = None
- text_format: str | None = None
+ """The node of a scalar blueprint type: its kind and per-kind slots (a compound type maps to `unknown`)."""
match field_type:
case ConceptStructureBlueprintFieldType.TEXT:
- kind = FieldKind.TEXT
+ return TextField(name=name, description=description, required=required, default_value=default_value)
case ConceptStructureBlueprintFieldType.INTEGER:
- kind = FieldKind.NUMBER
- integer = True
+ return NumberField(name=name, description=description, required=required, default_value=default_value, integer=True)
case ConceptStructureBlueprintFieldType.NUMBER:
- kind = FieldKind.NUMBER
- integer = False
+ return NumberField(name=name, description=description, required=required, default_value=default_value, integer=False)
case ConceptStructureBlueprintFieldType.BOOLEAN:
- kind = FieldKind.BOOLEAN
+ return BooleanField(name=name, description=description, required=required, default_value=default_value)
case ConceptStructureBlueprintFieldType.DATE:
- kind = FieldKind.DATE
- datetime_flag = False
+ return DateField(name=name, description=description, required=required, default_value=default_value, datetime=False)
case ConceptStructureBlueprintFieldType.DATETIME:
- kind = FieldKind.DATE
- datetime_flag = True
+ return DateField(name=name, description=description, required=required, default_value=default_value, datetime=True)
case ConceptStructureBlueprintFieldType.TIME:
- kind = FieldKind.TEXT
- text_format = "time"
+ return TextField(name=name, description=description, required=required, default_value=default_value, format="time")
case ConceptStructureBlueprintFieldType.LIST | ConceptStructureBlueprintFieldType.DICT | ConceptStructureBlueprintFieldType.CONCEPT:
- kind = FieldKind.UNKNOWN
- return InputFormField(
- kind=kind,
- name=name,
- description=description,
- required=required,
- default_value=default_value,
- integer=integer,
- datetime_flag=datetime_flag,
- format=text_format,
- )
+ return UnknownField(name=name, description=description, required=required, default_value=default_value)
def _with_reflected_constraints(*, node: InputFormField, field_info: FieldInfo) -> InputFormField:
@@ -732,8 +597,8 @@ def _with_reflected_constraints(*, node: InputFormField, field_info: FieldInfo)
constraints["required"] = field_info.is_required()
if field_info.default is not PydanticUndefined and field_info.default is not None:
constraints["default_value"] = field_info.default
- match node.kind:
- case FieldKind.NUMBER:
+ match node:
+ case NumberField():
for constraint in field_info.metadata:
match constraint:
case Gt(gt=bound):
@@ -746,7 +611,7 @@ def _with_reflected_constraints(*, node: InputFormField, field_info: FieldInfo)
constraints["maximum"] = bound
case _:
pass
- case FieldKind.TEXT:
+ case TextField():
for constraint in field_info.metadata:
match constraint:
case MinLen(min_length=length):
@@ -759,15 +624,7 @@ def _with_reflected_constraints(*, node: InputFormField, field_info: FieldInfo)
if isinstance(pattern, str):
constraints["pattern"] = pattern
case (
- FieldKind.PROSE
- | FieldKind.DATE
- | FieldKind.BOOLEAN
- | FieldKind.ENUM
- | FieldKind.DOCUMENT
- | FieldKind.IMAGE
- | FieldKind.OBJECT
- | FieldKind.LIST
- | FieldKind.UNKNOWN
+ ProseField() | DateField() | BooleanField() | EnumField() | DocumentField() | ImageField() | ObjectField() | ListField() | UnknownField()
):
pass
return node.model_copy(update=constraints) if constraints else node
@@ -775,8 +632,7 @@ def _with_reflected_constraints(*, node: InputFormField, field_info: FieldInfo)
def _prose_promoted_node(*, name: str, concept_ref: str, description: str, chain: list[str]) -> InputFormField:
"""A description-only or string-described concept: this engine backs it with a `TextContent` subclass."""
- return InputFormField(
- kind=FieldKind.PROSE,
+ return ProseField(
name=name,
concept_ref=concept_ref,
refines=[*chain, NativeConceptCode.TEXT.concept_ref],
@@ -794,8 +650,7 @@ def _unknown_node(
required: bool = True,
default_value: Any | None = None,
) -> InputFormField:
- return InputFormField(
- kind=FieldKind.UNKNOWN,
+ return UnknownField(
name=name,
concept_ref=concept_ref,
refines=refines,
diff --git a/pipelex/pipeline/pipe_io_contracts.py b/pipelex/pipeline/pipe_io_contracts.py
index 12511aaa6..52969f576 100644
--- a/pipelex/pipeline/pipe_io_contracts.py
+++ b/pipelex/pipeline/pipe_io_contracts.py
@@ -1,12 +1,21 @@
"""Per-pipe input/output contracts (`pipe_io_contracts`) for the validate surfaces.
-This is the canonical builder for the `pipe_io_contracts` artifact reported by the MTHDS
+This is the reference builder for the `pipe_io_contracts` artifact reported by the MTHDS
Protocol `validate` operation (local runtime and hosted API alike). It projects loaded
pipes into typed `PipeIOContract` entries — for each pipe, the JSON-Schema view of its
declared inputs and the concept/multiplicity of its output — keyed by the namespaced
`pipe_ref` (`domain.code`), the one identity convention shared by every validate artifact
(`validated_pipes`, `pending_signatures`).
+**The wire shapes belong to the standard, not to this engine.** `IOMultiplicity`,
+`PipeInputContract`, `PipeOutputContract` and `PipeIOContract` are the models of
+`mthds.protocol.pipe_io_contracts`, mirroring the standard's `pipe-io-contracts` page;
+they are imported and re-exported here so this module keeps its callers, and this engine
+holds no second declaration of them. What stays here is the projection: how a loaded pipe
+becomes a contract. The models are closed shapes that check their own cross-field
+invariants at construction, so a projection that ever drifted from the standard fails
+here, at the derivation, rather than silently on the wire.
+
Callers must invoke the builder against loaded pipes INSIDE the validation library's
window, before teardown — the builder now says so itself rather than degrading. It asks
the interpreter hub for the loaded method's concept library up front, so a post-teardown
@@ -19,70 +28,27 @@ the library that defined them. They still do — what changed is that this build
reaches them behind the library's back. (Registry teardown hygiene is tracked separately.)
"""
-from enum import StrEnum
from typing import Any, Sequence
-from pydantic import BaseModel, Field, PydanticUndefinedAnnotation, PydanticUserError
+from mthds.protocol.pipe_io_contracts import IOMultiplicity, PipeInputContract, PipeIOContract, PipeIOContracts, PipeOutputContract
+from pydantic import PydanticUndefinedAnnotation, PydanticUserError
from pipelex.core.concepts.concept_representation_generator import ConceptRepresentationFormat
from pipelex.core.concepts.exceptions import ConceptValueError
-from pipelex.core.pipes.variable_multiplicity import PresenceMarker, VariableMultiplicity, fixed_item_count, is_multiple_multiplicity
+from pipelex.core.pipes.variable_multiplicity import VariableMultiplicity, fixed_item_count, is_multiple_multiplicity
from pipelex.interpreter_hub import get_concept_library
from pipelex.pipe_machinery.pipe_abstract import PipeAbstract
from pipelex.pipeline.exceptions import PipeIOContractError
-
-class IOMultiplicity(StrEnum):
- """Wire value for a slot's multiplicity: one item, a variable-length list, or a fixed count.
-
- A fixed-count slot (`Concept[N]`) reports `fixed` and carries the exact count in the
- contract's `item_count`; a variable-length slot (`Concept[]`) reports `variable` with no
- count. `Concept[1]` is single — no list framing.
- """
-
- SINGLE = "single"
- VARIABLE = "variable"
- FIXED = "fixed"
-
-
-class PipeInputContract(BaseModel):
- """One declared input: the concept it expects and the JSON Schema of its content."""
-
- concept_ref: str
- presence: PresenceMarker = PresenceMarker.PLAIN
- """The declared presence marker, verbatim: `optional` (`?`) means the caller may omit the
- input and the pipe handles absence itself; `plain` and `force` (`!`) both must be provided —
- the distinction is the authored assertion, which lint and graph surfaces read."""
-
- multiplicity: IOMultiplicity = IOMultiplicity.SINGLE
- item_count: int | None = None
- """The exact item count, non-null exactly when `multiplicity` is `fixed`. The slot is always
- on the wire — `null` off the fixed arm, per the protocol spec (the input-form descriptor makes
- the opposite choice and omits it; the two artifacts deliberately differ)."""
-
- json_schema: dict[str, Any] = Field(default_factory=dict)
-
-
-class PipeOutputContract(BaseModel):
- """The pipe's output: the concept it produces and how many items it resolves to."""
-
- concept_ref: str
- multiplicity: IOMultiplicity
- item_count: int | None = None
- """The exact item count, non-null exactly when `multiplicity` is `fixed`. Always on the wire,
- `null` off the fixed arm — see `PipeInputContract.item_count`."""
-
- optional: bool = False
- """`True` when the output is declared optional (`?`): the pipe may resolve it as a recorded
- absence instead of a value — a successful run with an absent result. Genuinely two-valued:
- `!` is rejected on outputs."""
-
-
-class PipeIOContract(BaseModel):
- """The input/output contract of one pipe — a `pipe_io_contracts` entry."""
-
- inputs: dict[str, PipeInputContract] = Field(default_factory=dict)
- output: PipeOutputContract
+__all__ = [
+ "IOMultiplicity",
+ "PipeIOContract",
+ "PipeIOContracts",
+ "PipeInputContract",
+ "PipeOutputContract",
+ "build_pipe_io_contracts",
+ "make_io_multiplicity",
+]
def make_io_multiplicity(*, multiplicity: VariableMultiplicity | None) -> tuple[IOMultiplicity, int | None]:
@@ -99,7 +65,7 @@ def make_io_multiplicity(*, multiplicity: VariableMultiplicity | None) -> tuple[
return IOMultiplicity.SINGLE, None
-def build_pipe_io_contracts(pipes: Sequence[PipeAbstract]) -> dict[str, PipeIOContract]:
+def build_pipe_io_contracts(pipes: Sequence[PipeAbstract]) -> PipeIOContracts:
"""Project loaded pipes into `pipe_io_contracts` entries keyed by namespaced `pipe_ref`.
Works on any loaded `PipeAbstract` — including `PipeSignature` placeholders, whose
@@ -129,7 +95,7 @@ def build_pipe_io_contracts(pipes: Sequence[PipeAbstract]) -> dict[str, PipeIOCo
# `Concept[1]` and serve one's schema for the other.
schema_memo: dict[tuple[str, bool, int | None], dict[str, Any]] = {}
concept_provider = get_concept_library()
- io_contracts: dict[str, PipeIOContract] = {}
+ io_contracts: PipeIOContracts = {}
for pipe in pipes:
pipe_inputs: dict[str, PipeInputContract] = {}
for var_name, stuff_spec in pipe.inputs.root.items():
diff --git a/pipelex/pipeline/vacuous_presence_warnings.py b/pipelex/pipeline/vacuous_presence_warnings.py
index f13ef9130..d5fc28425 100644
--- a/pipelex/pipeline/vacuous_presence_warnings.py
+++ b/pipelex/pipeline/vacuous_presence_warnings.py
@@ -28,8 +28,9 @@ a lint whose boundary is arguable is a lint that gets ignored).
from collections.abc import Iterable, Mapping
from pipelex.base_exceptions import ValidationErrorCategory, ValidationErrorItem
+from pipelex.core.pipes.variable_multiplicity import is_force_presence
from pipelex.core.qualified_ref import QualifiedRef
-from pipelex.pipeline.input_form import InputFormField, PipeInputFormDescriptor
+from pipelex.pipeline.input_form import InputFormField, ObjectField, PipeInputFormDescriptor
from pipelex.validation_error_types import PipeValidationErrorType
@@ -64,15 +65,22 @@ def build_vacuous_presence_warnings(
def _warning_for_slot(*, pipe_ref: str, slot: InputFormField) -> ValidationErrorItem | None:
"""The item a top-level slot earns, or `None` when the slot is silent."""
- if not slot.gating or not slot.kind.is_object:
+ if not slot.gating:
return None
- if any(field.required for field in slot.fields or []):
+ if not isinstance(slot, ObjectField):
+ # The lint's claim is about what a concept's payload declares, which only an object node has.
+ return None
+ if any(field.required for field in slot.fields):
return None
concept_ref = slot.concept_ref
if concept_ref is None:
# No concept to name, hence no honest message to state: the lint's whole claim is about
# what a named concept declares.
return None
+ if slot.name is None:
+ # A top-level field always states its name — only a list's item omits one — but the wire
+ # model keeps the slot optional, and a finding with nothing to point at would not be one.
+ return None
# Split on the LAST dot: a domain is a dotted path ('legal.contracts'), so a leading-dot split
# would hand the locator a truncated domain and a pipe code carrying the rest of it.
locator = QualifiedRef.parse(pipe_ref)
@@ -86,13 +94,13 @@ def _warning_for_slot(*, pipe_ref: str, slot: InputFormField) -> ValidationError
)
-def _message(*, pipe_ref: str, slot: InputFormField, concept_ref: str) -> str:
+def _message(*, pipe_ref: str, slot: ObjectField, concept_ref: str) -> str:
"""The finding's prose: what was declared, why it enforces nothing, and the two remedies.
Carries no authored free text (no descriptions), so it needs no eliding — unlike the hint lint,
which interpolates authored content.
"""
- marker_desc = "declared with a force marker '!'" if slot.presence is not None and slot.presence.is_force else "declared without '?'"
+ marker_desc = "declared with a force marker '!'" if is_force_presence(presence=slot.presence) else "declared without '?'"
if slot.fields:
defect = f"concept '{concept_ref}' declares no required field — an empty object satisfies it"
second_remedy = f"make at least one field of '{concept_ref}' required"
diff --git a/pipelex/pipeline/validate_in_process.py b/pipelex/pipeline/validate_in_process.py
index 9118f54bc..0c90d635b 100644
--- a/pipelex/pipeline/validate_in_process.py
+++ b/pipelex/pipeline/validate_in_process.py
@@ -30,9 +30,9 @@ from pipelex.pipe_run.dry_run_in_process import best_effort_graph_spec
from pipelex.pipeline.advisory_warnings import build_advisory_warnings
from pipelex.pipeline.blueprint_selection import collect_entry_pipe_refs, select_primary_blueprint
from pipelex.pipeline.controller_taint import collect_controller_taint_analyses
-from pipelex.pipeline.input_form import PipeInputFormDescriptor, build_input_form, qualify_current_library_crate
+from pipelex.pipeline.input_form import InputForm, build_input_form, qualify_current_library_crate
from pipelex.pipeline.liftable_pipes import LiftablePipeEntry, build_liftable_pipes
-from pipelex.pipeline.pipe_io_contracts import PipeIOContract, build_pipe_io_contracts
+from pipelex.pipeline.pipe_io_contracts import PipeIOContracts, build_pipe_io_contracts
from pipelex.pipeline.validate_bundle import validate_bundle
from pipelex.pipeline.validation_report import PipelexValidationReport, build_validation_report
@@ -106,10 +106,10 @@ async def validate_bundles_in_process(
# current — the graph arm and the finally must target the SAME library even if
# something inside the window later moves the contextvar.
validation_library_id = get_current_library_id_or_none()
- pipe_io_contracts: dict[str, PipeIOContract] = build_pipe_io_contracts(result.pipes)
+ pipe_io_contracts: PipeIOContracts = build_pipe_io_contracts(result.pipes)
# One crate qualification per validate pass — the descriptors and the hint lint read the same one.
qualified_crate = qualify_current_library_crate()
- input_form: dict[str, PipeInputFormDescriptor] = build_input_form(result.pipes, qualified_crate=qualified_crate)
+ input_form: InputForm = build_input_form(result.pipes, qualified_crate=qualified_crate)
# One taint walk per validate pass — both report projections read the same analyses.
taint_analyses = collect_controller_taint_analyses(result.pipes)
liftable_pipes: list[LiftablePipeEntry] = build_liftable_pipes(taint_analyses)
diff --git a/pipelex/pipeline/validation_report.py b/pipelex/pipeline/validation_report.py
index 9538b36a8..f94a4d909 100644
--- a/pipelex/pipeline/validation_report.py
+++ b/pipelex/pipeline/validation_report.py
@@ -22,9 +22,9 @@ from pipelex.graph.graphspec import GraphSpec
from pipelex.mthds_parsing.pipelex_bundle_blueprint import PipelexBundleBlueprint
from pipelex.pipeline.blueprint_selection import select_primary_blueprint
from pipelex.pipeline.bundle_validator import DryRunOutput
-from pipelex.pipeline.input_form import PipeInputFormDescriptor
+from pipelex.pipeline.input_form import InputForm
from pipelex.pipeline.liftable_pipes import LiftablePipeEntry
-from pipelex.pipeline.pipe_io_contracts import PipeIOContract
+from pipelex.pipeline.pipe_io_contracts import PipeIOContracts
from pipelex.pipeline.validate_bundle import ValidatedPipeEntry, build_validated_pipes
from pipelex.tools.typing.pydantic_utils import empty_list_factory_of
@@ -46,10 +46,10 @@ class PipelexValidationReport(ValidationReport):
bundle_blueprint: PipelexBundleBlueprint
"""The batch's primary blueprint: first declaring `main_pipe`, else first."""
- pipe_io_contracts: dict[str, PipeIOContract] = Field(default_factory=dict)
+ pipe_io_contracts: PipeIOContracts = Field(default_factory=dict)
"""Per-pipe input/output contracts, keyed by namespaced `pipe_ref` (`domain.code`)."""
- input_form: dict[str, PipeInputFormDescriptor]
+ input_form: InputForm
"""Per-pipe input-form descriptors (the MTHDS input-form descriptor, derived from authored
facts — never from `json_schema`), keyed exactly like `pipe_io_contracts`. Required, no
default: the shared-assembly rule says a report field is populated on every backend or none,
@@ -82,8 +82,8 @@ class PipelexValidationReport(ValidationReport):
def build_validation_report(
*,
blueprints: Sequence[PipelexBundleBlueprint],
- pipe_io_contracts: dict[str, PipeIOContract],
- input_form: dict[str, PipeInputFormDescriptor],
+ pipe_io_contracts: PipeIOContracts,
+ input_form: InputForm,
dry_run_result: dict[str, DryRunOutput],
pending_signatures: list[str],
graph_spec: GraphSpec | None = None,
diff --git a/pyproject.toml b/pyproject.toml
index 9b222d925..36f965c03 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -31,7 +31,7 @@ dependencies = [
"json2html>=1.3.0",
"kajson==0.7.0",
"markdown>=3.6",
- "mthds>=0.8.2",
+ "mthds>=0.9.0",
"networkx>=3.4.2",
"openai>=2.0.0",
"opentelemetry-api",
diff --git a/subject_grants.toml b/subject_grants.toml
index 3a43ce4ae..8ab39e0d3 100644
--- a/subject_grants.toml
+++ b/subject_grants.toml
@@ -2928,10 +2928,6 @@ rationale = "Type-directed accessor: the target item type; single operand."
param = "item_type"
rationale = "Type-directed accessor: the target item type; single operand."
-["pipelex/core/pipes/variable_multiplicity.py::PresenceMarker.from_symbol"]
-param = "symbol"
-rationale = "Factory from source: parses the symbol; single operand."
-
["pipelex/core/pipes/variable_multiplicity.py::format_concept_with_multiplicity"]
param = "concept_code_or_string"
rationale = "Transforms the concept string; multiplicity stays keyword."
diff --git a/tests/helpers/input_form.py b/tests/helpers/input_form.py
new file mode 100644
index 000000000..5d21962cb
--- /dev/null
+++ b/tests/helpers/input_form.py
@@ -0,0 +1,38 @@
+"""Narrowing accessors for the input-form descriptor's discriminated union.
+
+The descriptor's field models come from `mthds.protocol.input_form`, where a node's kind IS its
+model and `InputFormField` is their union discriminated on `kind`. A test that reads a per-kind
+slot — `fields` on an object node, `item` on a list node, `integer` on a number node — must narrow
+to that model first, and a failed narrowing is exactly the assertion the test meant to make. These
+helpers do both in one gesture, so a test that expected one kind and got another fails saying which
+kind arrived.
+"""
+
+from __future__ import annotations
+
+from typing import TypeVar
+
+from pipelex.pipeline.input_form import InputFormField, InputFormFieldBase, ListField, ObjectField
+
+FieldModelType = TypeVar("FieldModelType", bound=InputFormFieldBase)
+
+
+def as_kind(node: InputFormField, kind_model: type[FieldModelType]) -> FieldModelType:
+ """The node, narrowed to one per-kind model — asserting the kind on the way through."""
+ assert isinstance(node, kind_model), f"Expected a '{kind_model.__name__}' node, got '{node.kind}'"
+ return node
+
+
+def as_object(node: InputFormField) -> ObjectField:
+ """The node, narrowed to an `object`."""
+ return as_kind(node, ObjectField)
+
+
+def as_list(node: InputFormField) -> ListField:
+ """The node, narrowed to a `list`."""
+ return as_kind(node, ListField)
+
+
+def fields_by_name(node: InputFormField) -> dict[str, InputFormField]:
+ """An object node's payload fields keyed by their authored name."""
+ return {field.name: field for field in as_object(node).fields if field.name is not None}
diff --git a/tests/integration/pipelex/pipeline/test_input_form.py b/tests/integration/pipelex/pipeline/test_input_form.py
index 06eb7b68b..75d267a1b 100644
--- a/tests/integration/pipelex/pipeline/test_input_form.py
+++ b/tests/integration/pipelex/pipeline/test_input_form.py
@@ -31,6 +31,7 @@ from pipelex.pipeline.input_form import FieldKind, InputFormField, build_input_f
from pipelex.pipeline.pipe_io_contracts import build_pipe_io_contracts
from pipelex.pipeline.validate_bundle import validate_bundle
from pipelex.system.registries.class_registry_access import get_class_registry
+from tests.helpers.input_form import fields_by_name
if TYPE_CHECKING:
from collections.abc import Callable
@@ -104,7 +105,7 @@ prompt = "Write a haiku."
def _field_by_name(descriptor: PipeInputFormDescriptor, name: str) -> InputFormField:
- by_name = {field.name: field for field in descriptor.fields}
+ by_name = {field.name: field for field in descriptor.fields if field.name is not None}
assert name in by_name, f"Expected a field named {name!r}, got {sorted(by_name)}"
return by_name[name]
@@ -241,9 +242,9 @@ class TestBuildInputForm:
assert by_name["enabled"].kind == FieldKind.BOOLEAN
assert by_name["released_on"].kind == FieldKind.DATE
- assert by_name["released_on"].datetime_flag is False
+ assert by_name["released_on"].datetime is False
assert by_name["launched_at"].kind == FieldKind.DATE
- assert by_name["launched_at"].datetime_flag is True
+ assert by_name["launched_at"].datetime is True
daily_at = by_name["daily_at"]
assert daily_at.kind == FieldKind.TEXT
assert daily_at.format == "time"
@@ -301,7 +302,7 @@ class TestBuildInputForm:
assert number_in.integer is False
date_in = _field_by_name(natives, "date_in")
assert date_in.kind == FieldKind.DATE
- assert date_in.datetime_flag is False
+ assert date_in.datetime is False
time_in = _field_by_name(natives, "time_in")
assert time_in.kind == FieldKind.TEXT
assert time_in.format == "time"
@@ -391,8 +392,8 @@ class TestBuildInputForm:
finally:
_teardown_validation_library(outer_library_id)
- review = {field.name: field for field in input_form["input_semantics_hinted.hinted_slots"].fields}["hinted"]
- review_fields = {field.name: field for field in review.fields or []}
+ review = _field_by_name(input_form["input_semantics_hinted.hinted_slots"], "hinted")
+ review_fields = fields_by_name(review)
assert review_fields["headline"].hints == {"intent": "label"}
assert review_fields["headline"].kind is FieldKind.TEXT
assert review_fields["body"].hints == {"intent": "prose"}
@@ -560,10 +561,8 @@ class TestKindAssignmentTable:
input_form = await self._derive_kind_table(load_empty_library)
one = _field_by_name(input_form["input_form_kinds.edges"], "one")
- assert one.kind == FieldKind.OBJECT
+ assert one.kind == FieldKind.OBJECT, "A `[1]` slot is the concept's own node, never a list wrapper"
assert one.concept_ref == "input_form_kinds.Gadget"
- assert one.item is None
- assert one.item_count is None
assert one.required is True
assert one.gating is True
@@ -660,4 +659,3 @@ class TestKindAssignmentTable:
assert unmappable.concept_ref == "input_form_kinds.Unmappable"
assert unmappable.description is not None
assert "PROBE_desc_concept_Unmappable" in unmappable.description
- assert unmappable.fields is None
diff --git a/tests/unit/pipelex/pipeline/test_input_form_deriver.py b/tests/unit/pipelex/pipeline/test_input_form_deriver.py
index ce2f8e1f2..545a25aaf 100644
--- a/tests/unit/pipelex/pipeline/test_input_form_deriver.py
+++ b/tests/unit/pipelex/pipeline/test_input_form_deriver.py
@@ -10,6 +10,7 @@ from __future__ import annotations
from pipelex.core.concepts.concept_blueprint import ConceptBlueprint
from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprint, ConceptStructureBlueprintFieldType
from pipelex.pipeline.input_form import FieldKind, InputFormDeriver
+from tests.helpers.input_form import as_object
def _concept_field(*, concept_ref: str) -> ConceptStructureBlueprint:
@@ -27,9 +28,7 @@ class TestInputFormDeriverEscapeHatches:
}
node = InputFormDeriver(concepts=concepts).derive_concept(name="root", concept_ref="demo.Node")
- assert node.kind == FieldKind.OBJECT
- assert node.fields is not None
- child = node.fields[1]
+ child = as_object(node).fields[1]
assert child.name == "child"
assert child.kind == FieldKind.UNKNOWN
assert child.concept_ref == "demo.Node"
@@ -41,10 +40,7 @@ class TestInputFormDeriverEscapeHatches:
}
left = InputFormDeriver(concepts=concepts).derive_concept(name="left", concept_ref="demo.Left")
- assert left.fields is not None
- right = left.fields[0]
- assert right.kind == FieldKind.OBJECT
- assert right.fields is not None
+ right = as_object(as_object(left).fields[0])
assert right.fields[0].kind == FieldKind.UNKNOWN
assert right.fields[0].concept_ref == "demo.Left"
diff --git a/tests/unit/pipelex/pipeline/test_input_form_hints.py b/tests/unit/pipelex/pipeline/test_input_form_hints.py
index ee51d1bb5..e4674b743 100644
--- a/tests/unit/pipelex/pipeline/test_input_form_hints.py
+++ b/tests/unit/pipelex/pipeline/test_input_form_hints.py
@@ -5,7 +5,8 @@ duplication, and preserved-content riding.
from pipelex.core.concepts.concept_blueprint import ConceptBlueprint
from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprint, ConceptStructureBlueprintFieldType
-from pipelex.pipeline.input_form import FieldKind, InputFormDeriver
+from pipelex.pipeline.input_form import FieldKind, InputFormDeriver, NumberField
+from tests.helpers.input_form import as_kind, as_list, fields_by_name
_CONCEPTS: dict[str, ConceptBlueprint | str] = {
# Description-only concept: text-valued, no-hint kind is `prose`.
@@ -76,7 +77,7 @@ class TestConceptEffectiveHints:
class TestStructureFieldHints:
def test_field_hints_stamped_and_feed_kind(self):
card = _deriver().derive_concept(name="card", concept_ref="docs.Card")
- fields = {field.name: field for field in card.fields or []}
+ fields = fields_by_name(card)
assert fields["title"].kind is FieldKind.TEXT
assert fields["title"].hints == {"intent": "label"}
assert fields["body"].hints is None
@@ -84,25 +85,23 @@ class TestStructureFieldHints:
def test_rating_on_integer_field_rides_without_changing_kind(self):
card = _deriver().derive_concept(name="card", concept_ref="docs.Card")
- fields = {field.name: field for field in card.fields or []}
- assert fields["stars"].kind is FieldKind.NUMBER
- assert fields["stars"].integer is True
- assert fields["stars"].hints == {"intent": "rating"}
+ fields = fields_by_name(card)
+ stars = as_kind(fields["stars"], NumberField)
+ assert stars.integer is True
+ assert stars.hints == {"intent": "rating"}
def test_concept_typed_field_site_wins_over_concept_layer(self):
card = _deriver().derive_concept(name="card", concept_ref="docs.Card")
- fields = {field.name: field for field in card.fields or []}
+ fields = fields_by_name(card)
# Essay's concept layer says prose; the field site says label — the site wins.
assert fields["essay"].hints == {"intent": "label"}
assert fields["essay"].kind is FieldKind.TEXT
def test_list_field_hints_ride_list_and_item(self):
card = _deriver().derive_concept(name="card", concept_ref="docs.Card")
- fields = {field.name: field for field in card.fields or []}
- tags = fields["tags"]
- assert tags.kind is FieldKind.LIST
+ fields = fields_by_name(card)
+ tags = as_list(fields["tags"])
assert tags.hints == {"intent": "label"}
- assert tags.item is not None
assert tags.item.hints == {"intent": "label"}
# The item is the text-valued site: the label intent flips ITS kind.
assert tags.item.kind is FieldKind.TEXT
diff --git a/tests/unit/pipelex/pipeline/test_input_form_models.py b/tests/unit/pipelex/pipeline/test_input_form_models.py
index 92a64d993..f1ec5c98d 100644
--- a/tests/unit/pipelex/pipeline/test_input_form_models.py
+++ b/tests/unit/pipelex/pipeline/test_input_form_models.py
@@ -1,49 +1,32 @@
-"""Unit-pin the input-form wire models: per-kind slot validators and null-free serialization.
-
-The report's valid arm is dumped WITHOUT `exclude_none` on the HTTP surface, so the field model
-owns its own wire shape: inapplicable slots are dropped at serialization (never emitted as JSON
-null), applicable falsy values (`required: false`, `gating: false`, `integer: false`,
-`item_count` on a fixed list) are kept, and the `datetime` wire slot serializes under its spec
-name regardless of the Python attribute that carries it.
+"""Characterize the wire this engine emits through the standard's input-form models.
+
+The descriptor's shapes are declared by `mthds.protocol.input_form` and re-exported from
+`pipelex.pipeline.input_form`, so the per-kind slot rules and the closed shapes are the standard's
+own and are pinned in the package that declares them. What this module pins is the engine's side of
+the bargain: the report's valid arm is dumped WITHOUT `exclude_none` on the HTTP surface, so a node
+must render its own wire — inapplicable slots absent rather than JSON null, applicable falsy values
+(`required: false`, `gating: false`, `integer: false`, `item_count` on a fixed list) kept, the
+`datetime` slot under its spec name, and the serializer recursing through a descriptor.
"""
from __future__ import annotations
-import pytest
-from pydantic import ValidationError
-
from pipelex.core.pipes.variable_multiplicity import PresenceMarker
-from pipelex.pipeline.input_form import FieldKind, InputFormField, PipeInputFormDescriptor
+from pipelex.pipeline.input_form import (
+ DateField,
+ EnumField,
+ InputFormField,
+ ListField,
+ NumberField,
+ ObjectField,
+ PipeInputFormDescriptor,
+ ProseField,
+ TextField,
+)
def _prose_field(name: str = "brief") -> InputFormField:
- return InputFormField(kind=FieldKind.PROSE, name=name, concept_ref="demo.Brief", required=True, presence=PresenceMarker.PLAIN, gating=True)
-
-
-class TestInputFormFieldValidators:
- def test_enum_requires_choices(self) -> None:
- with pytest.raises(ValidationError):
- InputFormField(kind=FieldKind.ENUM, name="tone", required=False)
-
- def test_object_requires_fields(self) -> None:
- with pytest.raises(ValidationError):
- InputFormField(kind=FieldKind.OBJECT, name="widget", required=True)
-
- def test_list_requires_item(self) -> None:
- with pytest.raises(ValidationError):
- InputFormField(kind=FieldKind.LIST, name="notes", required=True)
-
- def test_number_requires_integer_flag(self) -> None:
- with pytest.raises(ValidationError):
- InputFormField(kind=FieldKind.NUMBER, name="count", required=True)
-
- def test_date_requires_datetime_flag(self) -> None:
- with pytest.raises(ValidationError):
- InputFormField(kind=FieldKind.DATE, name="released_on", required=True)
-
- def test_scalar_kinds_accept_minimal_slots(self) -> None:
- field = InputFormField(kind=FieldKind.TEXT, name="title", required=True)
- assert field.kind == FieldKind.TEXT
+ return ProseField(name=name, concept_ref="demo.Brief", required=True, presence=PresenceMarker.PLAIN, gating=True)
class TestInputFormSerialization:
@@ -55,13 +38,12 @@ class TestInputFormSerialization:
assert dumped["required"] is True
assert dumped["presence"] == "plain"
assert dumped["gating"] is True
- for absent in ("title", "refines", "description", "default_value", "examples", "hints", "fields", "item", "item_count", "choices"):
+ for absent in ("title", "refines", "description", "default_value", "examples", "hints", "min_length", "max_length", "pattern", "format"):
assert absent not in dumped, f"Inapplicable slot {absent!r} must be absent, not null"
assert None not in dumped.values()
def test_applicable_falsy_values_are_kept(self) -> None:
- field = InputFormField(
- kind=FieldKind.NUMBER,
+ field = NumberField(
name="count",
concept_ref="demo.Count",
required=False,
@@ -77,32 +59,29 @@ class TestInputFormSerialization:
assert dumped["exclusive_minimum"] == 0
def test_datetime_slot_serializes_under_its_wire_name(self) -> None:
- field = InputFormField(kind=FieldKind.DATE, name="released_on", required=True, datetime_flag=False)
+ field = DateField(name="released_on", required=True, datetime=False)
dumped = field.model_dump(mode="json")
assert dumped["datetime"] is False
- assert "datetime_flag" not in dumped
def test_single_member_choices_stay_a_list(self) -> None:
- field = InputFormField(kind=FieldKind.ENUM, name="only", required=False, choices=["single"])
+ field = EnumField(name="only", required=False, choices=["single"])
assert field.model_dump(mode="json")["choices"] == ["single"]
def test_descriptor_dump_recurses_the_serializer(self) -> None:
descriptor = PipeInputFormDescriptor(
fields=[
- InputFormField(
- kind=FieldKind.LIST,
+ ListField(
name="gadgets",
concept_ref="demo.Gadget",
required=True,
presence=PresenceMarker.PLAIN,
gating=True,
item_count=2,
- item=InputFormField(
- kind=FieldKind.OBJECT,
+ item=ObjectField(
name="gadgets",
concept_ref="demo.Gadget",
required=True,
- fields=[InputFormField(kind=FieldKind.TEXT, name="label", required=True)],
+ fields=[TextField(name="label", required=True)],
),
)
]
diff --git a/tests/unit/pipelex/pipeline/test_vacuous_presence_warnings.py b/tests/unit/pipelex/pipeline/test_vacuous_presence_warnings.py
index 97d6e7190..27082cfbe 100644
--- a/tests/unit/pipelex/pipeline/test_vacuous_presence_warnings.py
+++ b/tests/unit/pipelex/pipeline/test_vacuous_presence_warnings.py
@@ -10,7 +10,21 @@ import pytest
from pipelex.base_exceptions import ValidationErrorCategory, ValidationErrorItem
from pipelex.core.pipes.variable_multiplicity import PresenceMarker
-from pipelex.pipeline.input_form import FieldKind, InputFormField, PipeInputFormDescriptor
+from pipelex.pipeline.input_form import (
+ BooleanField,
+ DateField,
+ DocumentField,
+ EnumField,
+ ImageField,
+ InputFormField,
+ ListField,
+ NumberField,
+ ObjectField,
+ PipeInputFormDescriptor,
+ ProseField,
+ TextField,
+ UnknownField,
+)
from pipelex.pipeline.vacuous_presence_warnings import build_vacuous_presence_warnings
ENTRY_PIPE_REF = "demo.run"
@@ -19,7 +33,7 @@ OPTIONS_CONCEPT_REF = "demo.RunOptions"
def _nested(*, name: str, required: bool) -> InputFormField:
"""A structure field inside an object node (no slot facts: no presence, no gating)."""
- return InputFormField(kind=FieldKind.TEXT, name=name, required=required)
+ return TextField(name=name, required=required)
def _object_slot(
@@ -32,8 +46,7 @@ def _object_slot(
concept_ref: str | None = OPTIONS_CONCEPT_REF,
) -> InputFormField:
"""A top-level object slot, defaulting to the all-optional shape the lint fires on."""
- return InputFormField(
- kind=FieldKind.OBJECT,
+ return ObjectField(
name=name,
concept_ref=concept_ref,
required=required,
@@ -45,8 +58,7 @@ def _object_slot(
def _object_item() -> InputFormField:
"""The all-optional object as a list's `item` node: a concept node, so no slot facts."""
- return InputFormField(
- kind=FieldKind.OBJECT,
+ return ObjectField(
name="opts",
concept_ref=OPTIONS_CONCEPT_REF,
required=True,
@@ -123,8 +135,7 @@ class TestVacuousPresenceWarnings:
def test_variable_multiplicity_list_is_silent(self):
"""`Concept[]` never gates by the descriptor's own rule — `[]` is its legitimate value."""
- slot = InputFormField(
- kind=FieldKind.LIST,
+ slot = ListField(
name="opts",
concept_ref=OPTIONS_CONCEPT_REF,
required=True,
@@ -136,8 +147,7 @@ class TestVacuousPresenceWarnings:
def test_fixed_count_list_is_silent(self):
"""`Concept[N]` gates, but the vacuity question is per item — deferred (design §7)."""
- slot = InputFormField(
- kind=FieldKind.LIST,
+ slot = ListField(
name="opts",
concept_ref=OPTIONS_CONCEPT_REF,
required=True,
@@ -151,15 +161,15 @@ class TestVacuousPresenceWarnings:
@pytest.mark.parametrize(
"slot",
[
- InputFormField(kind=FieldKind.TEXT, name="opts", required=True, presence=PresenceMarker.PLAIN, gating=True),
- InputFormField(kind=FieldKind.PROSE, name="opts", required=True, presence=PresenceMarker.PLAIN, gating=True),
- InputFormField(kind=FieldKind.NUMBER, name="opts", required=True, presence=PresenceMarker.PLAIN, gating=True, integer=False),
- InputFormField(kind=FieldKind.BOOLEAN, name="opts", required=True, presence=PresenceMarker.PLAIN, gating=True),
- InputFormField(kind=FieldKind.DATE, name="opts", required=True, presence=PresenceMarker.PLAIN, gating=True, datetime_flag=False),
- InputFormField(kind=FieldKind.ENUM, name="opts", required=True, presence=PresenceMarker.PLAIN, gating=True, choices=["a", "b"]),
- InputFormField(kind=FieldKind.DOCUMENT, name="opts", required=True, presence=PresenceMarker.PLAIN, gating=True),
- InputFormField(kind=FieldKind.IMAGE, name="opts", required=True, presence=PresenceMarker.PLAIN, gating=True),
- InputFormField(kind=FieldKind.UNKNOWN, name="opts", required=True, presence=PresenceMarker.PLAIN, gating=True),
+ TextField(name="opts", required=True, presence=PresenceMarker.PLAIN, gating=True),
+ ProseField(name="opts", required=True, presence=PresenceMarker.PLAIN, gating=True),
+ NumberField(name="opts", required=True, presence=PresenceMarker.PLAIN, gating=True, integer=False),
+ BooleanField(name="opts", required=True, presence=PresenceMarker.PLAIN, gating=True),
+ DateField(name="opts", required=True, presence=PresenceMarker.PLAIN, gating=True, datetime=False),
+ EnumField(name="opts", required=True, presence=PresenceMarker.PLAIN, gating=True, choices=["a", "b"]),
+ DocumentField(name="opts", required=True, presence=PresenceMarker.PLAIN, gating=True),
+ ImageField(name="opts", required=True, presence=PresenceMarker.PLAIN, gating=True),
+ UnknownField(name="opts", required=True, presence=PresenceMarker.PLAIN, gating=True),
],
ids=["text", "prose", "number", "boolean", "date", "enum", "document", "image", "unknown"],
)
diff --git a/tests/unit/pipelex/pipeline/test_validation_report.py b/tests/unit/pipelex/pipeline/test_validation_report.py
index eff839334..8829ef661 100644
--- a/tests/unit/pipelex/pipeline/test_validation_report.py
+++ b/tests/unit/pipelex/pipeline/test_validation_report.py
@@ -47,7 +47,7 @@ class TestBuildValidationReport:
pipe_io_contracts = {
"beta.do_it": PipeIOContract(
inputs={},
- output=PipeOutputContract(concept_ref="native.Text", multiplicity=IOMultiplicity.SINGLE),
+ output=PipeOutputContract(concept_ref="native.Text", multiplicity=IOMultiplicity.SINGLE, item_count=None, optional=False),
),
}
dry_run_result: dict[str, DryRunOutput] = {
diff --git a/uv.lock b/uv.lock
index ff35ceb03..40a24bedd 100644
--- a/uv.lock
+++ b/uv.lock
@@ -2552,7 +2552,7 @@ wheels = [
[[package]]
name = "mthds"
-version = "0.8.2"
+version = "0.9.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "httpx" },
@@ -2561,9 +2561,9 @@ dependencies = [
{ name = "tomlkit" },
{ name = "typing-extensions" },
]
-sdist = { url = "https://files.pythonhosted.org/packages/ed/87/9fefbf352f2f2794336ae6fa1f72ea7d7a74ce78524e3bce734bea83aa02/mthds-0.8.2.tar.gz", hash = "sha256:1ac24a415f5a4942e93309066ed6b65a553ca379578e2da93f492bc63342b0b6", size = 130890, upload-time = "2026-08-21T10:49:37.05Z" }
+sdist = { url = "https://files.pythonhosted.org/packages/52/f3/e585dd38f4be879347702d2792c01b571fac607fdc5c520d8ed4d4592b90/mthds-0.9.0.tar.gz", hash = "sha256:2386f52d4bf40e13dbe793cbc9d45ab16981d3de8f846b42ad49f4a7412ea55a", size = 159117, upload-time = "2026-08-26T22:40:50.699Z" }
wheels = [
- { url = "https://files.pythonhosted.org/packages/b7/85/051bf842ead495c9126e04a940cbfd3fada2474599b89be27b6d73e73158/mthds-0.8.2-py3-none-any.whl", hash = "sha256:13d314523afc5f774b7f19f65b94ce32e753a617298aae8d6d11315448bbea9f", size = 58167, upload-time = "2026-08-21T10:49:35.804Z" },
+ { url = "https://files.pythonhosted.org/packages/ec/17/6163b570259974bca8dacf8e0b4f0e83649eebae1f5dbbe16603a5e5d749/mthds-0.9.0-py3-none-any.whl", hash = "sha256:226410499aef67847519d994e4cf0fb1ac9b9621b96e34bc17bfa719dd856013", size = 68740, upload-time = "2026-08-26T22:40:49.33Z" },
]
[[package]]
@@ -3472,7 +3472,7 @@ requires-dist = [
{ name = "mkdocs-meta-manager", marker = "extra == 'docs'", specifier = ">=1.1.0" },
{ name = "mkdocs-redirects", marker = "extra == 'docs'", specifier = "==1.2.2" },
{ name = "moto", extras = ["dynamodb", "s3"], marker = "extra == 'dev'", specifier = ">=5.0.0" },
- { name = "mthds", specifier = ">=0.8.2" },
+ { name = "mthds", specifier = ">=0.9.0" },
{ name = "mypy", marker = "extra == 'dev'", specifier = "==1.19.1" },
{ name = "networkx", specifier = ">=3.4.2" },
{ name = "openai", specifier = ">=2.0.0" },
diff --git a/wip/input-form/import-protocol-models.md b/wip/input-form/import-protocol-models.md
new file mode 100644
index 000000000..01b9f8880
--- /dev/null
+++ b/wip/input-form/import-protocol-models.md
@@ -0,0 +1,37 @@
+# Import the standard's wire models for the input-form descriptor and the pipe I/O contracts
+
+This is the `pipelex` half of item 3.1 of the workspace input-form program (`../../../wip/input-form/plan.md` at the workspace root, ledger item `L-260826-db8dd5`). The program's decision D-1 gives the wire types of the input-form descriptor and of `pipe_io_contracts` to the standard's clients — `mthds/protocol` in TypeScript, `mthds.protocol` in Python — and D-3 says the engines import those models for what they emit rather than restating them. This document records what that meant here, and the calls taken along the way.
+
+## What moved
+
+`pipelex/pipeline/pipe_io_contracts.py` and `pipelex/pipeline/input_form.py` used to declare the wire shapes themselves. They no longer do: both modules now import the models from `mthds.protocol.pipe_io_contracts` and `mthds.protocol.input_form`, re-export them for the callers that already read them from here, and keep only what is genuinely this engine's — the builders, the deriver, and the exception conversion. The `mthds` floor moves to the release that carries those modules.
+
+The wire is unchanged. Every field name, every value and every absent-versus-null rule is the one the engine already emitted; the derivation is the same derivation. What changed is which package declares the shape.
+
+## The three shape differences the import forced
+
+The standard's models are not a transcription of what this engine declared, so importing them was not a delete-and-import. Three differences did real work.
+
+**The descriptor is a discriminated union, not one model with optional per-kind slots.** The standard declares `TextField`, `ProseField`, `DateField`, `NumberField`, `BooleanField`, `EnumField`, `DocumentField`, `ImageField`, `ObjectField`, `ListField` and `UnknownField`, each closed, with `InputFormField` the union discriminated on `kind`. That is strictly better than what was here — a `text` node can no longer carry `choices`, and a kind's required slots are simply required fields — but it means a node's kind is now its class. Two consequences ran through the deriver: constructing a node picks a class instead of passing a `kind=`, and the place that used to flip an existing node's kind by `model_copy(update={"kind": ...})` now rebuilds the node as the target class. That place is the intent-hint stamp, and the rebuild it needs is narrower than it first looks: `prose` and `label` are the only intent words that feed the kind, and both apply to text-valued sites alone, so the flip is always `text` ⟷ `prose` — two models sharing a base and therefore sharing every slot. `_recast_text_kind` is that rebuild, and it carries the shared text constraints across unchanged. It also states rather than asserts the invariant: a node that somehow reached it without being text-valued degrades to a plain hint stamp instead of raising.
+
+**`FieldKind.is_list` and `is_object` are gone, and are not missed.** They existed to ask a flat model which arm it was on. With the union the question is `isinstance(node, ListField)` / `isinstance(node, ObjectField)`, which also narrows the type, so the callers that used to reach for `slot.fields` after an `is_object` check now do it with the type checker's agreement instead of against it.
+
+**`datetime_flag` is `datetime`.** The old model spelled the slot `datetime_flag` in Python and renamed it in a serializer, to keep the attribute from shadowing the stdlib module name. The standard's `DateField` simply declares `datetime`, which is legal as a pydantic field and never shadows anything at module scope, so the serializer rename went with the declaration.
+
+## `PresenceMarker` is now the standard's
+
+The standard declares `PresenceMarker` in `mthds.protocol.pipe_io_contracts`, because the marker is part of the contract's wire vocabulary. This engine declared its own in `pipelex/core/pipes/variable_multiplicity.py`, with the same three members and the same meaning. Keeping both was not an option once the engine constructs the standard's models: the two enums are unrelated classes to a type checker, so every construction site would have needed a conversion, and the workspace would have kept two declarations of one wire enum — the thing D-1 exists to end.
+
+So `variable_multiplicity.py` re-exports the standard's `PresenceMarker`. The three helpers it used to carry that are grammar rather than wire — parsing a marker symbol, rendering one, and asking whether a marker is the force assertion — became module-level functions in the same module (`presence_from_symbol`, `presence_symbol`, `is_force_presence`), so nothing about the `.mthds` suffix grammar left this repo. `is_optional` and `is_plain` come from the standard's enum, which declares both.
+
+## How the unchanged wire was proved, not asserted
+
+`mthds-python` commits one real payload pair captured from this engine's own derivation, byte-for-byte, as its Stage 2.3 parity fixture: `tests/fixtures/protocol/input_form.json` and `pipe_io_contracts.json`, produced by `pipelex-dev trace-input-semantics tests/data/input_semantics/hinted_bundle.mthds tests/data/input_semantics/probe_bundle.mthds` on `pipelex` 0.53.0 at checkout `bc97dad0b` — which is this change's merge base. Re-running that same command after the import and diffing the two hop-5 outputs against the committed fixture reports no difference on either artifact. The probe bundles exercise every construct the language accepts, so that is the whole emission, not a sample of it.
+
+## The parse-time gates this turned on
+
+The standard's models are closed shapes (`extra="forbid"`) and enforce cross-field invariants at the parse. Constructing them makes those invariants live gates on what this engine emits, which is the point: a divergence between the engine and the standard now fails loudly at derivation instead of silently on the wire. The gates that bear on the emission are the `item_count`/`multiplicity` pairing on both contract sides, the rule that a presence marker is never combined with multiplicity, the rule that a fixed count is at least two, the rule that a field never carries both `required: true` and a `default_value`, and the placement rule that `presence` and `gating` are stated on every top-level descriptor field and absent everywhere below it.
+
+## Deliberately out of scope
+
+The program's item 4.5 collects the engine-emission corrections the standard's pages made necessary — the fabricated `refines: ["native.Text"]` on description-only concepts and the `name` member on a list's `item` (`L-260826-0ed8dd`), `native.Date` emitted as `kind: "date"` and `native.Html` as `prose` where the standard's ordered table puts both on `object` (`L-260826-236839`), and `css_class` becoming optional on `native.Html` (`L-260826-3cea94`). None of them is a shape question, so none of them blocks this import, and none is folded in here. They are their own change.