Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
feat(codegen): normalized library crate + deterministic projections (types, inputs, trust chain, resolve/codegen CLI + agent CLI) (#1039)
Louis Choquel048c87971d3d
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 649daac2b..178e1c006 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -3,7 +3,8 @@## [Unreleased]### Added-+- **`pipelex resolve` and the `pipelex codegen` family:** `pipelex resolve` assembles a bundle closure and emits the **normalized library crate** — a flat, fully-qualified, natives-expanded, self-contained, fingerprinted snapshot of a resolved library — as JSON or TOML. `pipelex codegen types --target ts-zod|python-pydantic|python-structures` projects the crate's concept set into typed artifacts, and `pipelex codegen inputs --pipe <ref>` renders a runnable inputs template for a pipe (defaulting to the closure's `main_pipe`). The `ts-zod` target emits a pure `types.ts` (imports only `zod`, with wire-native snake_case keys so a schema validates a wire payload directly) plus a `binder.ts` exposing a typed `parse`/`serialize` pair per concept; `python-pydantic` emits self-contained `BaseModel`s and `python-structures` emits runtime `StructuredContent` classes. Generated files are never edited — each carries an `AUTOGENERATED` header pointing at the sibling extension-file mechanism.+- **Codegen trust chain (stamps · `codegen.lock` · offline `codegen check`):** Every file `pipelex codegen types` writes now carries a self-describing **stamp** header (source crate fingerprint, engine version, projection, options, and a content hash of the body below it), and the artifact set is recorded in a sibling **`codegen.lock`**. `pipelex codegen check [DIR]` verifies generated artifacts are current **offline** — pure hashing, no engine, network, or API key — reporting drift by category (missing, modified, hand-edited, orphan) with the verdict on the exit code (`0` current · `1` drift · `2` no lock). Emission is idempotent (write-if-changed — unchanged output leaves files untouched) and prunes previously-generated files that drop out of the set, so a deleted concept never lingers as a stale generated file. Input templates (`codegen inputs`) are deliberately not stamped or locked — they are user-editable scaffolds, not tracked generated code.- **Agent bundle fixer:** Added `pipelex-agent fix bundle`, a deterministic in-place fixer over the shared validation fix loop. The command supports `--format`/`--error-format`, repeatable `--select`/`--ignore` fix-rule filters, `--allow-signatures`, `--max-iterations`, and the same bundle-file or directory resolution as `pipelex-agent validate bundle`. Successful fix results report `pending_signatures` and `is_runnable`; by default the command exits non-zero when a fixed bundle remains valid-but-not-runnable.- **Human bundle fixer:** Added `pipelex fix bundle`, the human counterpart of the agent fixer over the same fix loop, with Rich-rendered output naming every change made (fix descriptions, files written, iteration count), the same `--select`/`--ignore`/`--allow-signatures`/`--max-iterations` options, and the same bundle-file or directory resolution as `pipelex validate bundle`. A new `--diff` flag previews the changes as a unified diff without writing anything — the real fix loop runs against a temp-copy sandbox, and exit codes keep the verdict semantics so `--diff` answers "would it converge?".- **Suggested fixes surfaced in `pipelex validate`:** When a validation error has a deterministic safe fix, the human error output now shows a per-error `💡 Suggested fix:` line and ends with an actionable footer naming the exact `pipelex fix bundle` command (echoing the invocation's `-L` dirs) in place of the generic tip.@@ -15,11 +16,17 @@- Declared structured lists (e.g., `Person[]`) accept a direct path to a `.csv` file by signature, without requiring an envelope.- **Native `YesNo` concept:** Built-in concept (backed by `YesNoContent`) for boolean judgments. Requires a strict boolean, renders as `yes`/`no` in downstream prompts, and can be read via `pipe_output.main_stuff_as_yes_no.yes_no`.- **Native `Date` concept:** Built-in concept (backed by `DateContent`) for calendar dates. Requires a `datetime.date` and accepts an optional `datetime.time`. Preserves source precision (never forcing the LLM to fabricate a time of day) and retains UTC offsets verbatim.+- **Native `Time` concept:** Built-in concept (backed by `TimeContent`) for a time of day, optionally with a UTC offset — never attached to an invented date. A top-level TOML time-of-day literal in an inputs file now maps to `Time` (breaking: `Time` joins the reserved native concept codes, and the former bare-time rejection — `InputsTimeOnlyNotSupportedError` — is gone).+- **Structure-field `type = "time"`:** The concept structure language gains a `time` field type (a time of day, optionally with a UTC offset), completing the temporal triple beside `date` and `datetime`. Generates a `datetime.time` field; emitted by every codegen target.+- **Pinned normative native definitions:** Native concepts now materialize into the normalized crate by **lookup into the standard's pinned per-version definitions** (`pipelex/core/concepts/native/pinned_blueprints.py`, mirroring the mthds spec's new `native-concepts.md`), never by reflection over the runtime content classes — so crate fingerprints over materialized natives byte-agree across implementations. The old reflection is retained solely as a consistency test proving each runtime content class still matches its pinned blueprint.+- **Agent-CLI codegen mirror (`pipelex-agent codegen types|check`):** the codegen family is now available to machine consumers through the agent CLI's two-stream envelopes (`--format markdown|json` success stream, `--error-format` error stream). `codegen types` runs the same engine and write-if-changed emission as the bare CLI; `codegen check` presents the offline drift verdict structurally — drift is a `CodegenDriftError` envelope enumerating `drifts[]` by category (exit `1`), a missing or unreadable `codegen.lock` is a no-verdict error (exit `2`). With the pipelex runner, `mthds-agent codegen …` forwards to these commands.+- **Host-facing resolve/codegen engine cores (what the HTTP routes ride):** `pipelex.pipeline.resolve_bundle.resolve_crate_from_contents` resolves **in-memory** MTHDS contents into the normalized crate — same engine, same `ValidateBundleError` verdict vocabulary, and the same loaded-on-success library contract as `validate_bundle` — so a host (the `pipelex-api` `/v1/resolve` + `/v1/codegen` routes) serves resolution without touching the filesystem. `pipelex.codegen.emission.build_stamped_projection` is the new **pure** stamping core (stamped files + `codegen.lock` content, no disk writes) that `write_stamped_projection` now rides, so artifacts served over HTTP are byte-identical to locally written ones and pass the offline `codegen check` verbatim. The shared bundle-loading error cascade `translate_to_validate_bundle_error` is now public (renamed from its underscore-private form) since it gained a third entry point.- **Comprehensive input error handling:** A suite of typed errors for input shaping failures (`WrongScalarKindError`, `ListWhereSingularError`, `MultiplicityCountMismatchError`, `StructureValidationError`, `UnknownInputNameError`) that provide clear hints and render the expected JSON shape.- **CLI `--explicit` flag:** Added `--explicit` to `pipelex build inputs` and `pipelex-agent inputs` to generate the legacy `{"concept", "content"}` envelope templates.### Changed-+- (Breaking) **`pipelex build structures` is now an alias of `codegen types --target python-structures`:** the legacy always-qualified per-file generator is deleted. The alias resolves the target's whole directory closure and emits one stamped `structures.py` module plus a `codegen.lock` (bare-when-unique class names, declared imprecision instead of the old silent `TextContent` guess for structureless concepts). The `--force` flag is gone (emission is write-if-changed), a `.mthds` file target now resolves its parent directory's closure, and concepts-only loading of invalid-pipe bundles is no longer supported (the crate requires a valid library). The concepts-only loading API (`load_concepts_only`, `load_concepts_only_from_directory`, and the library-manager methods behind them) is removed with it.+- (Breaking) **`pipelex build runner` emits its structures through the codegen engine:** the scaffolded `structures/` directory now contains the stamped single-module projection (`structures.py` + `codegen.lock`) instead of per-concept files, and the generated runner script imports and instantiates the emitted class spellings (`from structures.structures import Invoice`) — fixing dropped imports for nested custom concepts along the way. A user class backing a `structure = "<ClassName>"` concept is imported from its real module.- **Human validation error rendering routed through the shared structured items:** `pipelex validate`'s error details now render from the same `ValidationErrorItem`s the agent CLI and API emit, which surfaces information the old renderer silently dropped: pipe-factory errors (e.g. an unknown concept) get their own section, a parse-level failure (e.g. a TOML syntax error) now shows its message instead of no detail at all, and pipe validation errors name their source file. The dry-run message remains visible alongside categorized errors.- **Validation error messages now speak MTHDS author syntax:** Pipe validation errors state the problem and the fix the way an author writes them — a sequence output-multiplicity mismatch now reads "declares its output as 'StoryIdea', but its last step yields 'StoryIdea[]'. Update the sequence's output to 'StoryIdea[]'", and an input-type mismatch reads "input 'dish' is declared as 'Number' but its step needs 'Text'. Update the input to 'Text'" — instead of leaking Python internals (`multiplicity=None`, `Concept(...)`/`PresenceMarker` reprs). The misleading `(required: […], provided: …)` suffix is suppressed for multiplicity errors (where both sides are identical by definition) and rendered as joined author-syntax refs — never Python list-repr brackets — elsewhere.- **Agent `validate` / `fix` output renders prose, not a JSON dump:** `pipelex-agent validate bundle` and `pipelex-agent fix bundle` markdown output now renders validation errors as category-grouped prose (a humanized title, identity fields, the message, and a `💡 Suggested fix:` line) followed by a fix-aware footer naming the exact `pipelex-agent fix bundle` command when a safe fix exists — mirroring the human panel. The machine-facing JSON stream (`--format json`) is unchanged, so software consumers keep branching on the same structured fields.@@ -28,11 +35,17 @@- (Breaking) **Structure-field `type = "date"` behavior:** A `date` field now generates a `datetime.date` (JSON schema `format: date`), preventing LLMs from hallucinating times for calendar dates. A new `type = "datetime"` field handles full timestamps (`format: date-time`), and default-value validation now rejects a `datetime` default on a `date` field.- (Breaking) **Strict input validation:** Undeclared input names are now a hard error (previously silently ignored), and top-level `null` values are rejected—absence must be expressed by omitting the key.- (Breaking) **Explicit envelope validation:** The `{"concept", "content"}` escape hatch is now compatibility-checked against the pipe's declared signature; incompatible concepts raise a typed error.-- (Breaking) **TOML date/time handling:** TOML date and datetime literals are natively supported and map directly to the `Date` concept. Only bare time-of-day literals are rejected (via the renamed `InputsTimeOnlyNotSupportedError`), since a time alone has no date to attach to.+- (Breaking) **TOML date/time handling:** TOML date and datetime literals are natively supported and map directly to the `Date` concept; a bare time-of-day literal maps to the new `Time` concept (it never silently becomes a `Date` — shaping a `Time` into a `Date` slot fails the compatibility check).+- (Breaking) **`ImageContent` flattens its pixel dimensions:** `size: ImageSize | None` is replaced by paired optional `width` / `height` integer fields (both-or-neither, enforced by a validator) — a named nested shape in the crate is a concept, and the size grouping did not earn that citizenship. `ImageSize` remains as the internal img-gen parameter type. Image wire payloads change shape accordingly.+- (Breaking) **`native.Date` declares real structure:** materialized `Date` is no longer structureless — its pinned definition carries a required `date` field plus an optional `time` field (the new `time` field type), so generated clients get real Date types. `DateContent`'s wire form is unchanged.- **Documentation overhaul:** Updated `provide-inputs.md`, `executing-pipelines.md`, and `native-concepts.md` for the Smart Inputs paradigm, leading with the bare-value approach and demoting the envelope format to an escape hatch.### Fixed-+- **`resolve` / `codegen` rejected bundles that pin a model:** the resolve and codegen commands booted without model specs, so library validation could not check pipe model pins (`model = "gpt-4o-mini"`) against the deck and rejected any bundle carrying one. They now boot like `validate` — offline, but with model specs loaded.+- **Native `Image` no longer materializes structureless:** every native now materializes from its pinned normative definition, so generated clients for image-producing pipes expose `url`, `public_url`, `caption`, `width`, `height`, etc. (The brief interim dict-with-imprecision mapping for the nested size model is superseded by the `ImageContent` flatten above.)+- **Deterministic input-template placeholders:** mock URL placeholders in generated inputs templates (`codegen inputs` / `build inputs`) are now deterministic (`https://mock.invalid/<field>`), so committed templates no longer churn on every regeneration.+- **`codegen inputs` is write-if-changed:** like `codegen types`, an already-current inputs template is left untouched (no mtime churn) and the console reports `Unchanged` instead of claiming `Generated`.+- **Opaque generated classes no longer strip content:** a structureless or Python-class-backed concept's generated Python class now carries `model_config = ConfigDict(extra="allow")`, so `model_validate` keeps the payload verbatim instead of pydantic's default `extra="ignore"` silently dropping every field. Opaque is pass-through, never lossy — matching the ts-zod `z.unknown()` behavior.- **Human CLI `~` expansion:** `pipelex validate bundle` (and the new `pipelex fix bundle`) now expand `~` in the bundle path and `-L/--library-dir` values, matching the agent CLI.- **Validate telemetry label:** the `CLI_COMMAND` telemetry tag for `pipelex validate` subcommands no longer double-suffixes (previously "validate bundle bundle").- **Silent mistyping of inputs:** Providing a bare string for a refined text concept (e.g., `legal.Question`) no longer silently degrades the type to a generic `native.Text`; the declared concept type is now retained.diff --git a/TODOS.md b/TODOS.mdnew file mode 100644index 000000000..fa9cd195a--- /dev/null+++ b/TODOS.md@@ -0,0 +1,54 @@+# PR recap — DevEx codegen: the normalized library crate and its projections++This file is a reviewer's guide to this PR (`feature/Devex-codegen` → `dev`). It summarizes what the branch delivers, the design decisions behind shapes a reviewer might otherwise flag, and the verification already performed. The full execution history (checkpoints, review fan-outs, triage records) lives in this file's git history on the branch; the deferred continuation of the track is documented at the workspace root under `wip/codegen/`.++## What this PR delivers++**One authority, many projections.** A resolved, validated MTHDS library is snapshotted into a **normalized library crate** — flat, fully qualified, natives materialized, refinement flattened, fingerprinted — and every developer-facing artifact is a deterministic projection off that crate: typed models (Python/TS), runnable input templates, runner scaffolds, and (served from `pipelex-api`, in a companion branch) HTTP responses. Generated files carry a self-describing stamp, are tracked in a sibling `codegen.lock`, and are verifiable **offline** by pure hashing.++The crate format is spec'd in the MTHDS standard (`mthds` repo: `docs/spec/library-crate.md`); the pipelex surfaces are spec'd at the workspace root (`docs/specs/pipelex-codegen.md`) with conformance skeletons/tests in the `conformance` repo. Contract-first: the specs and skeletons were written and reviewed before the engine code.++## Component map (where to look in the diff)++- **`pipelex/codegen/`** — the engine. `resolved_fields.py` / `resolved_concepts.py` (neutral resolved layers every emitter consumes; imprecision is explicit, never guessed), `emitters/` (`python_structures`, `python_pydantic`, `ts_zod` + `naming`, `target`, `types_emitter` dispatch), `crate_encoding.py` (canonical JSON/TOML), `stamp.py` / `lock.py` / `check.py` / `emission.py` (the trust chain: stamped headers, `codegen.lock`, offline drift check, idempotent write-if-changed emission with stale-file pruning), `native_expansion.py` (retained as a consistency probe only — see pinned natives below).+- **`pipelex/libraries/crate_normalization.py`** + `LibraryCrate.compute_normalized_fingerprint` — the normalization pass (post-validation, D6) and the widened fingerprint (concepts + pipes + domain metadata, provenance excluded; canonical JSON ≈ RFC 8785). The loader skips `native.*` crate entries so a normalized crate round-trips through `load_from_crate`.+- **`pipelex/core/concepts/native/pinned_blueprints.py`** — natives materialize by lookup into the standard's pinned per-version definitions, never by runtime reflection (cross-implementation fingerprint byte-agreement). A parametrized test proves each pinned blueprint still matches its runtime content class.+- **New native surface** — `native.Time` (`TimeContent`), the `time` structure-field type through the whole chain (blueprint enum → resolved fields → all three emitters → `StructureGenerator`), `ImageContent` flattened to paired `width`/`height` (breaking wire), `native.Date` pinned with real structure (wire unchanged).+- **CLI** — `pipelex resolve` (crate to stdout, JSON/TOML), the `pipelex codegen types|inputs|check` family, shared `crate_loading.py` (one resolve/validate exit-code contract: 0 resolved · 1 invalid library · 2 no-verdict), and the D9 re-point: `build structures` is a thin alias of `codegen types --target python-structures`, `build runner` scaffolds through the same projection with the emitted class spellings. The legacy per-file structures generator and the concepts-only loading family are deleted.+- **Agent CLI** — `pipelex-agent codegen types|check` mirrors the family through the two-stream envelopes (markdown/JSON); drift is a structured `CodegenDriftError` with `drifts[]` (exit 1), a missing/unreadable lock is no-verdict (exit 2). Deliberately no `codegen inputs` mirror (`pipelex-agent inputs` already surfaces that projection).+- **`pipelex/pipeline/resolve_bundle.py`** + `emission.build_stamped_projection` — host-facing engine cores for the HTTP routes (`pipelex-api` companion branch): in-memory resolve with `validate_bundle`'s verdict vocabulary and loaded-on-success lifecycle; a pure stamping core so HTTP-served artifacts are byte-identical to locally written ones (pinned by a direct parity test).+- **`StructureGenerator`** — refactored to consume the resolved-field layer; runtime materialization via `concept_factory` is byte-for-byte unchanged (the existing suite was the harness). Dead legacy dict-based paths deleted.+- **Docs** — `docs/under-the-hood/codegen-projections.md` (engine, trust chain, extension-file story, HTTP serving), plus native/inputs doc updates. `CHANGELOG.md` `[Unreleased]` covers everything user-visible.++## Breaking changes (intentional, changelogged — no-backward-compat policy)++- `pipelex build structures` output shape: one stamped `structures.py` + `codegen.lock`, bare-when-unique class names, declared imprecision instead of the old silent `TextContent` guess; `--force` gone; concepts-only loading API removed.+- `pipelex build runner` structures directory rides the same projection; the script imports emitted class spellings.+- `ImageContent` wire shape: `size` → paired `width`/`height` (both-or-neither validator).+- `Time` joins the reserved native codes; the former blanket bare-time input rejection (`InputsTimeOnlyNotSupportedError`) is removed — a top-level TOML time maps to `TimeContent`.++## Design decisions a reviewer might otherwise flag++- **Imprecision over guessing (D5/B1-3):** a structureless or Python-class-backed concept emits an explicit opaque/imprecise marker (`# imprecise:` / `@imprecise`, `extra="allow"` pass-through classes), never a fabricated shape. This is the spec'd behavior, not missing handling.+- **ts-zod keys are wire-native snake_case (D10):** a cold review empirically confirmed that a generic deep snake↔camel remap silently corrupts caller data keys inside `z.record()`/`z.unknown()` values, so the schema validates the wire directly and the remap layer does not exist. camelCase ergonomics, if ever wanted, must be a schema-aware transform.+- **Sibling `codegen.lock`, not an extension of `methods.lock` (D4):** different owner (Pipelex vs standard), location, content, and lifecycle — consumer projects often aren't MTHDS packages at all.+- **Verdicts are structural, exit codes/HTTP statuses are presentation** (workspace meta-rule): machine consumers branch on `is_valid`/error envelopes, never on exit codes.+- **`pipe_ref` is plumbed through the stamp but unused today:** the stamp format is spec'd to carry it for the per-pipe projection kinds that arrive in later phases; it is cheap and round-trip tested — not speculative dead code.+- **Loaded-on-success contract:** `resolve_crate_from_contents` (like `validate_bundle`) leaves the library loaded + current on success and the host owns teardown — the HTTP routes read live pipes from it before their `finally`. Teardown conservation is pinned by tests on both repos.+- **`native_expansion.py` reflection retained:** consistency probe only (pinned blueprint ↔ runtime class); materialization is the pinned lookup.++## Verification record++- Gates at every checkpoint and after the final dev merge: `make agent-check` (ruff, pyright 0, mypy 0, kw-only, plxt) + **full** `make agent-test`.+- Emitted-code quality gates in-tree: `pyright --strict` over generated Python (D7), serialize→parse round-trips, resolve→emit e2e over a multi-bundle closure fixture feeding every emitter; golden tests assert compile-and-parse behavior, not byte equality.+- Live binary proofs at each stage: stamp/lock/check drift loop (hand-edit → exit 1 → regen heals → byte-identical), alias re-point, agent-CLI envelopes and exit codes.+- **Proof by starter** (`pipelex-starter-python`, companion branch): hand-written models deleted, committed generated clients, offline smoke tests — the conversion caught and fixed real engine bugs (model-spec-less boot, structureless `Image`, non-deterministic template placeholders).+- Independent cold `/code-review` fan-outs (one reviewer per touched repo, diff-pointer only, no plan context) ran at every checkpoint; all confirmed findings were fixed in-branch (e.g. the ts-binder key-corruption Critical, the non-UTF-8 `codegen check` crash, the `time` exec-globals `NameError`) and design tradeoffs were recorded as deferred docs rather than reflexively applied.++## Cross-repo companions (separate branches, not in this PR)++`pipelex-api` `feature/Codegen` (routes `POST /v1/resolve` + `POST /v1/codegen`, breaking `/v1/build/*` migration — deploy gated on this release), `mthds` `feature/Codegen` (library-crate + native-concepts specs), `conformance` `feature/Codegen` (route conformance live; CLI skeletons release-gated), workspace-root `feature/Follow-ups` (`docs/specs/pipelex-codegen.md`), `mthds-js` `feature/Codegen` (agent passthrough stubs), `pipelex-starter-python` `feature/Codegen` (the converted starter). Post-release coordination lives in `wip/codegen/release-wave.md` at the workspace root.++## Deferred (documented, not in this PR)++The remaining Phase 3 stretches (MCP projection; webapp form + n8n), Phase 4 (methods as products), and Phase 5 (frontier) are deferred by decision (2026-07-11) and documented one-file-per-concern under the workspace root's `wip/codegen/`.diff --git a/docs/building-methods/concepts/inline-structures.md b/docs/building-methods/concepts/inline-structures.mdindex b6e73409e..24185a1ec 100644--- a/docs/building-methods/concepts/inline-structures.md+++ b/docs/building-methods/concepts/inline-structures.md@@ -61,6 +61,8 @@ Each field can specify:- **item_type**: For `list` fields, the type of list items- **key_type** and **value_type**: For `dict` fields, the types of keys and values+Field names must be valid Python identifiers and cannot be Python keywords; generated Python projections use them as model attributes.+## Supported Field TypesInline structures support these field types:@@ -131,8 +133,17 @@ A timestamp — a date *and* a time of day. Generates a `datetime.datetime` fielcreated_at = { type = "datetime", description = "Creation timestamp" }```-!!! note "Field type vs the native `Date` concept"- A structure field's `date` / `datetime` type is a plain machine type — you pick one up front. If a field should keep *document fidelity* (a time only when the source states one), model it as `type = "concept", concept_ref = "Date"` instead: the native `Date` concept carries a required date plus an optional time.+### time++A time of day, optionally with a UTC offset — no date attached. Generates a `datetime.time` field (JSON schema `format: time`). Use it for opening hours, schedules, recurring times — anything you'd write as `09:00:00` or `15:40:00+02:00`.++```toml+[concept.OpeningHours.structure]+opens_at = { type = "time", description = "Daily opening time" }+```++!!! note "Field type vs the native `Date` / `Time` concepts"+ A structure field's `date` / `datetime` / `time` type is a plain machine type — you pick one up front. If a field should keep *document fidelity* (a time only when the source states one), model it as `type = "concept", concept_ref = "Date"` instead: the native `Date` concept carries a required date plus an optional `time` field.### list@@ -147,7 +158,7 @@ scores = { type = "list", item_type = "number", description = "Test scores" }### dict-Dictionaries/maps with key-value pairs. **Must specify `key_type` and `value_type`**.+Dictionaries/maps with key-value pairs. **Must specify `key_type` and `value_type`**. Dictionary keys are text on the JSON wire, so `key_type` must be `"text"` (`"str"` remains accepted as a legacy spelling).```toml[concept.Configuration.structure]@@ -273,27 +284,21 @@ pipelex build structures ./my_pipelines/ -o ./generated/### Example Output-Given the Invoice example from above, Pipelex generates:+Given the Invoice example from above, Pipelex generates one stamped `structures.py` module (plus a `codegen.lock` for the offline drift check) containing every concept of the closure:```python-# generated/finance__invoice.py-from datetime import date-from pydantic import Field-from pipelex.core.stuffs.structured_content import StructuredContent-from .finance__customer import Customer-from .finance__line_item import LineItem-+# generated/structures.py (below the codegen stamp header)class Invoice(StructuredContent):"""A commercial document issued by a seller to a buyer"""- invoice_number: str = Field(description="The unique invoice identifier")+ invoice_number: str = Field(..., description="The unique invoice identifier")issue_date: date = Field(..., description="The date the invoice was issued")customer: Customer = Field(..., description="The customer for this invoice")- line_items: list[LineItem] = Field(..., description="List of line items")+ line_items: List[LineItem] = Field(..., description="List of line items")total_amount: float = Field(..., description="The total invoice amount")```-You can then import and use these classes with full type safety.+You can then import and use these classes with full type safety. Generated files are never edited in place — to customize a class, subclass it in a sibling module (the generated header shows how).## When to Use Inline Structures@@ -323,4 +328,3 @@ See [Python StructuredContent Classes](python-classes.md) for advanced features.- [Define Your Concepts](define_your_concepts.md) - Learn about concept semantics and naming- [Python StructuredContent Classes](python-classes.md) - Advanced features with Python- [MTHDS Language Tutorial](../../get-started/mthds-language-tutorial.md) - Get started with structured outputs-diff --git a/docs/building-methods/concepts/native-concepts.md b/docs/building-methods/concepts/native-concepts.mdindex bac1a4df1..d5888f3dc 100644--- a/docs/building-methods/concepts/native-concepts.md+++ b/docs/building-methods/concepts/native-concepts.md@@ -32,6 +32,7 @@ Here are all the native concepts you can use out of the box:| `Number` | A number | `NumberContent` || `YesNo` | The answer to a yes/no question | `YesNoContent` || `Date` | A calendar date, optionally with a time of day | `DateContent` |+| `Time` | A time of day, optionally with a UTC offset | `TimeContent` || `Page` | A document page with text, images, and optional page view | `PageContent` || `Dynamic` | A dynamic concept that adapts to context | `DynamicContent` || `JSON` | A JSON object | `JSONContent` |@@ -62,16 +63,25 @@ Represents an image with optional metadata:```pythonclass ImageContent(StuffContent):url: str- source_prompt: Optional[str] = None- caption: Optional[str] = None- base_64: Optional[str] = None+ public_url: str | None = None+ source_prompt: str | None = None+ source_negative_prompt: str | None = None+ caption: str | None = None+ mime_type: str | None = None+ width: int | None = None+ height: int | None = None+ filename: str | None = None```**Fields:**-- `url`: Location of the image (file path or URL)-- `source_prompt`: The prompt used to generate the image (if applicable)++- `url`: Location of the image (a storage URI, an HTTP(S) URL, or a base64 data URL)+- `public_url`: Optional public-facing URL (when `url` is a private/internal reference)+- `source_prompt` / `source_negative_prompt`: The prompts used to generate the image (if applicable)- `caption`: Descriptive text for the image-- `base_64`: Base64-encoded image data (alternative to URL)+- `mime_type`: Optional MIME type of the image+- `width` / `height`: Pixel dimensions — present together or not at all+- `filename`: Optional original filename**Use for:** Photos, generated images, diagrams, screenshots.@@ -158,6 +168,19 @@ prompt = "Extract the scheduled departure from this ticket: $ticket"As a pipeline input, a top-level TOML date or datetime literal maps to `Date` directly (`departure = 2026-07-07T15:40:00+02:00`); read it from a Python caller via `pipe_output.main_stuff_as_date`.+### TimeContent++Represents a time of day, as precise as its source states — never attached to an invented date:++```python+class TimeContent(StuffContent):+ time: datetime.time+```++The UTC offset is kept on the `time` when the source states one, exactly as `Date` handles its optional time. It renders as ISO 8601 (`15:40:00`, or `15:40:00+02:00`) when injected into a prompt. As a pipeline input, a top-level TOML time-of-day literal maps to `Time` directly (`opening = 09:00:00`).++**Use for:** Opening hours, schedules, times of day with no specific date.+**Use for:** Issue dates, due dates, dates of birth, departures, effective/termination dates — any date found on a document. For a date value without any date attached (a bare time of day), degrade to `Text`.### TextAndImagesContentdiff --git a/docs/building-methods/concepts/python-classes.md b/docs/building-methods/concepts/python-classes.mdindex 23e83fa01..0ff8b2348 100644--- a/docs/building-methods/concepts/python-classes.md+++ b/docs/building-methods/concepts/python-classes.md@@ -100,13 +100,13 @@ The easiest way to get Python classes is to use `pipelex build structures`:pipelex build structures ./my_pipelines/```-This generates Python classes from your inline definitions, giving you type hints and IDE autocomplete without writing boilerplate.+This generates a single stamped `structures.py` module from your inline definitions, giving you type hints and IDE autocomplete without writing boilerplate. Generated files are never edited in place — they are overwritten on every regeneration, and `pipelex codegen check` flags hand edits as drift.### When You Need Custom LogicIf you need custom validation or computed properties, you have two options:-1. **Generate then customize**: Run `pipelex build structures`, then add your custom logic to the generated files+1. **Generate then copy**: Run `pipelex build structures`, then copy the generated class into your own module and customize it there2. **Write from scratch**: Create a Python class manually with your custom logic### Example: Adding Custom Validation@@ -131,7 +131,7 @@ pipelex build structures ./my_pipeline.mthds -o ./structures/**Step 3: Add custom validation**-Edit the generated file to add your validation:+Copy the generated class into your own module (never edit the generated file — it is overwritten on regeneration) and add your validation:```pythonfrom pipelex.core.stuffs.structured_content import StructuredContent@@ -168,7 +168,7 @@ The Python class is automatically discovered and registered.### From Inline Structure to Python Class-The easiest migration path is to use `pipelex build structures` to generate the Python class, then customize it:+The easiest migration path is to use `pipelex build structures` to generate the Python class, then copy it into your own module and customize it there:**1. You have this inline structure:**@@ -191,7 +191,7 @@ in_stock = { type = "boolean", description = "Stock availability", default_valuepipelex build structures ./ecommerce.mthds -o ./structures/```-**3. Add your custom logic** to the generated file:+**3. Copy the generated class into your own module** and add your custom logic:```pythonfrom pipelex.core.stuffs.structured_content import StructuredContentdiff --git a/docs/errors/authoring-and-language.md b/docs/errors/authoring-and-language.mdindex 76405856c..c1834fdb8 100644--- a/docs/errors/authoring-and-language.md+++ b/docs/errors/authoring-and-language.md@@ -103,6 +103,7 @@ own page. Classes are grouped by subsystem.- [`ConceptLibraryError`](concept-library-error.md) — Concept library- [`ConceptLoadingError`](concept-loading-error.md) — Concept loading+- [`CrateNormalizationError`](crate-normalization-error.md) — Crate normalization error- [`DomainLibraryError`](domain-library-error.md) — Domain library- [`DomainLoadingError`](domain-loading-error.md) — Domain loading- [`LibraryError`](library-error.md) — Library errordiff --git a/docs/errors/codegen-error.md b/docs/errors/codegen-error.mdnew file mode 100644index 000000000..f930e5459--- /dev/null+++ b/docs/errors/codegen-error.md@@ -0,0 +1,21 @@+---+title: "Codegen error"+description: "Reference for the `CodegenError` Pipelex error class."+---++<!-- pipelex:generated -->++# Codegen error++Base error for the codegen projection engine (stamps, lock, check, emission).++| Field | Value |+|---|---|+| `error_type` | `CodegenError` |+| `title` | Codegen error |+| `type_uri` | `https://docs.pipelex.com/latest/errors/codegen-error/` |+| `error_domain` | _(inherited from parent)_ |+| Defined in | `pipelex.codegen.exceptions` |+| Parent class | [`PipelexError`](pipelex-error.md) |++[Back to Error Reference](index.md)diff --git a/docs/errors/codegen-lock-error.md b/docs/errors/codegen-lock-error.mdnew file mode 100644index 000000000..49398134c--- /dev/null+++ b/docs/errors/codegen-lock-error.md@@ -0,0 +1,21 @@+---+title: "Codegen lock error"+description: "Reference for the `CodegenLockError` Pipelex error class."+---++<!-- pipelex:generated -->++# Codegen lock error++Raised when a `codegen.lock` cannot be read or parsed.++| Field | Value |+|---|---|+| `error_type` | `CodegenLockError` |+| `title` | Codegen lock error |+| `type_uri` | `https://docs.pipelex.com/latest/errors/codegen-lock-error/` |+| `error_domain` | _(inherited from parent)_ |+| Defined in | `pipelex.codegen.exceptions` |+| Parent class | [`CodegenError`](codegen-error.md) |++[Back to Error Reference](index.md)diff --git a/docs/errors/codegen-stamp-error.md b/docs/errors/codegen-stamp-error.mdnew file mode 100644index 000000000..20d1b5d92--- /dev/null+++ b/docs/errors/codegen-stamp-error.md@@ -0,0 +1,21 @@+---+title: "Codegen stamp error"+description: "Reference for the `CodegenStampError` Pipelex error class."+---++<!-- pipelex:generated -->++# Codegen stamp error++Raised when a generated file cannot be stamped (e.g. an unstampable file type).++| Field | Value |+|---|---|+| `error_type` | `CodegenStampError` |+| `title` | Codegen stamp error |+| `type_uri` | `https://docs.pipelex.com/latest/errors/codegen-stamp-error/` |+| `error_domain` | _(inherited from parent)_ |+| Defined in | `pipelex.codegen.exceptions` |+| Parent class | [`CodegenError`](codegen-error.md) |++[Back to Error Reference](index.md)diff --git a/docs/errors/crate-normalization-error.md b/docs/errors/crate-normalization-error.mdnew file mode 100644index 000000000..9ebddf2f5--- /dev/null+++ b/docs/errors/crate-normalization-error.md@@ -0,0 +1,21 @@+---+title: "Crate normalization error"+description: "Reference for the `CrateNormalizationError` Pipelex error class."+---++<!-- pipelex:generated -->++# Crate normalization error++Raised when a library crate cannot be normalized (e.g. a crate key that is not domain-qualified).++| Field | Value |+|---|---|+| `error_type` | `CrateNormalizationError` |+| `title` | Crate normalization error |+| `type_uri` | `https://docs.pipelex.com/latest/errors/crate-normalization-error/` |+| `error_domain` | _(inherited from parent)_ |+| Defined in | `pipelex.libraries.exceptions` |+| Parent class | [`LibraryError`](library-error.md) |++[Back to Error Reference](index.md)diff --git a/docs/errors/index.md b/docs/errors/index.mdindex bd7ba0ab1..cce7a6f29 100644--- a/docs/errors/index.md+++ b/docs/errors/index.md@@ -18,7 +18,7 @@ the error came from:- [Authoring & language](authoring-and-language.md) — Core language, Pipe operators, Pipe controllers, Pipe signatures, Builder, Libraries.- [Execution & runtime](execution-and-runtime.md) — Pipe execution, Pipeline execution, Runtime bridge, Graph, Tracing.- [Inference & providers](inference-and-providers.md) — Inference (Cogt), Provider plugins.-- [Platform & tooling](platform-and-tooling.md) — Base & root errors, Tools, Kit, System & configuration, CLI.+- [Platform & tooling](platform-and-tooling.md) — Base & root errors, Tools, Kit, System & configuration, CLI, Codegen.See [Error Model](../under-the-hood/error-model.md) for the underlying contractand classification rules.diff --git a/docs/errors/inputs-time-only-not-supported-error.md b/docs/errors/inputs-time-only-not-supported-error.mddeleted file mode 100644index 6bb72a31f..000000000--- a/docs/errors/inputs-time-only-not-supported-error.md+++ /dev/null@@ -1,22 +0,0 @@-----title: "Inputs time only not supported"-description: "Reference for the `InputsTimeOnlyNotSupportedError` Pipelex error class."------<!-- pipelex:generated -->--# Inputs time only not supported--Raised when a loaded inputs file carries a bare TOML time-of-day value.--| Field | Value |-|---|---|-| `error_type` | `InputsTimeOnlyNotSupportedError` |-| `title` | Inputs time only not supported |-| `type_uri` | `https://docs.pipelex.com/latest/errors/inputs-time-only-not-supported-error/` |-| `error_domain` | `input` |-| Defined in | `pipelex.cli.commands.run.exceptions` |-| Parent class | [`PipelexError`](pipelex-error.md) |-| `user_action` | `change_input` — A time of day alone has no date to attach to. Include the date (e.g. 2026-07-06T12:00:00), or quote the value as a string |--[Back to Error Reference](index.md)diff --git a/docs/errors/platform-and-tooling.md b/docs/errors/platform-and-tooling.mdindex 87827cdd0..15361dd15 100644--- a/docs/errors/platform-and-tooling.md+++ b/docs/errors/platform-and-tooling.md@@ -88,8 +88,13 @@ own page. Classes are grouped by subsystem.## CLI- [`AmbiguousInputsFilesError`](ambiguous-inputs-files-error.md) — Ambiguous inputs files-- [`InputsTimeOnlyNotSupportedError`](inputs-time-only-not-supported-error.md) — Inputs time only not supported- [`PipelexCLIError`](pipelex-cli-error.md) — Pipelex CLI- [`ReadinessCheckError`](readiness-check-error.md) — Readiness check+## Codegen++- [`CodegenError`](codegen-error.md) — Codegen error+- [`CodegenLockError`](codegen-lock-error.md) — Codegen lock error+- [`CodegenStampError`](codegen-stamp-error.md) — Codegen stamp error+[Back to Error Reference](index.md)diff --git a/docs/tools/cli/build/runner.md b/docs/tools/cli/build/runner.mdindex 9032d27e5..69652759a 100644--- a/docs/tools/cli/build/runner.md+++ b/docs/tools/cli/build/runner.md@@ -76,8 +76,8 @@ pipelex build runner bundle my_bundle.mthds --output custom_runner.pyThe generated Python file includes:-1. **All necessary imports** - Imports for Pipelex execution, content types, and any custom structures-2. **Generated Pydantic structures** - Models for your concepts (also generated separately)+1. **All necessary imports** - Imports for Pipelex execution, content types, and any custom structures (`from structures.structures import …`)+2. **Generated Pydantic structures** - A stamped `structures/structures.py` module (plus its `codegen.lock`) emitted by the codegen engine, same as `pipelex build structures`3. **Input memory setup** - Example input values based on the pipe's input types4. **Pipeline execution** - Async function that executes the pipeline5. **Output handling** - Code to extract and display the resultsdiff --git a/docs/tools/cli/build/structures.md b/docs/tools/cli/build/structures.mdindex f435dace2..382ff302f 100644--- a/docs/tools/cli/build/structures.md+++ b/docs/tools/cli/build/structures.md@@ -6,6 +6,9 @@ description: "Generate Pydantic models from concept definitions in your .mthds fGenerate Pydantic models from your concept definitions.+!!! note "Alias of `codegen types`"+ `pipelex build structures` is an alias of [`pipelex codegen types --target python-structures`](../../../under-the-hood/codegen-projections.md): it resolves your bundles into a normalized library crate and emits a single stamped `structures.py` module plus a `codegen.lock`. Use `pipelex codegen check` to verify the generated files are current without regenerating.+## Usage```bash@@ -14,11 +17,12 @@ pipelex build structures <TARGET> [OPTIONS]**Arguments:**-- `TARGET` - Either a library directory containing `.mthds` files, or a specific `.mthds` file+- `TARGET` - Either a library directory containing `.mthds` files, or a specific `.mthds` file (its directory is used as the closure)**Options:**-- `-o`, `--output-dir` - Output directory for generated Python files (defaults to `structures/` in target's directory)+- `-o`, `--output-dir` - Output directory for the generated module (defaults to `structures/` in target's directory)+- `-L`, `--library-dir` - Additional directory of `.mthds` bundles to load (repeatable)## Examples@@ -42,20 +46,21 @@ pipelex build structures ./my_pipelines/ -o ./generated/## What Gets Generated-For each concept in your library, the command generates:+One `structures.py` module for the whole closure, plus a `codegen.lock`:-- **Pydantic model classes** - Full type definitions for your concepts-- **Field definitions** - Proper types and constraints-- **Validation** - Based on concept constraints+- **Pydantic model classes** - One `StructuredContent` subclass per concept, with types, constraints, and descriptions+- **Bare-when-unique names** - A concept `demo.Report` becomes `class Report`; only a code that collides across domains gets the domain-qualified spelling+- **Stamp + lock** - Each file carries a self-describing codegen stamp (crate fingerprint, engine version, content hash) and is tracked in `codegen.lock`, so `pipelex codegen check` can detect drift offline+- **Declared imprecision** - A concept whose shape cannot be fully derived is surfaced with an explicit caveat, never a silent guess-## Why Use Generated Structures?+Generated files are never meant to be edited: to customize a type, subclass it in a sibling module (the generated header shows how) — subclasses survive regeneration.-Now you have your structures as Python code:+## Why Use Generated Structures?-- **Iterate on them** - Modify and extend the generated classes-- **Add custom validation** - Add Pydantic validators for business logic- **Use as type hints** - Get IDE autocompletion and type checking in your code+- **Add custom logic** - Subclass the generated classes in your own module- **Integrate in your app** - Use the models directly in your application+- **Trust the drift check** - `pipelex codegen check` tells you when the models no longer match the method## Example Output@@ -72,25 +77,22 @@ education = { type = "text", description = "Educational background", required =work_history = { type = "text", description = "Summary of previous roles", required = true }```-You get a Pydantic model:+You get (inside the stamped `structures.py`):```python-from pipelex.core.stuffs.structured_content import StructuredContent-from pydantic import Field-class CandidateProfile(StructuredContent):- """Generated CandidateProfile class"""+ """A structured representation of a candidate's professional background."""skills: str = Field(..., description="Technical and soft skills")- years_of_experience: float = Field(..., description="Total years of experience")+ years_of_experience: Optional[float] = Field(default=None, description="Total years of experience")education: str = Field(..., description="Educational background")work_history: str = Field(..., description="Summary of previous roles")```## Related Documentation+- [Codegen projections](../../../under-the-hood/codegen-projections.md) - The engine behind this command (`codegen types`, stamps, lock, offline check)- [Build Runner](runner.md) - Generate Python runner scripts- [Build Inputs](inputs.md) - Generate example input JSON for a pipe- [Build Output](output.md) - Generate example output JSON for a pipe- [Concepts](../../../building-methods/concepts/define_your_concepts.md) - Understanding concept definitions-diff --git a/docs/tools/cli/fix.md b/docs/tools/cli/fix.mdindex 65fa46b06..ee000c4fa 100644--- a/docs/tools/cli/fix.md+++ b/docs/tools/cli/fix.md@@ -65,3 +65,4 @@ Only fixes classified as SAFE are applied — deterministic corrections derived## Related Documentation- [Validate Commands](validate.md) - Where suggested fixes are surfaced+- [Error Model — suggested fixes](../../under-the-hood/error-model.md#suggested_fix-structured-deterministic-fixes) - The `SuggestedFix` wire model, the fix planner, and the patch-op vocabularydiff --git a/docs/tools/cli/run.md b/docs/tools/cli/run.mdindex edab93618..0c278db12 100644--- a/docs/tools/cli/run.md+++ b/docs/tools/cli/run.md@@ -198,8 +198,8 @@ name = "Acme Corp"country = "France"```-!!! tip "TOML date and datetime literals are native `Date` inputs"- A top-level TOML date or datetime literal (e.g. `hearing = 2026-09-01` or `departure = 2026-07-07T15:40:00+02:00`) maps directly to the native [`Date`](../../building-methods/concepts/native-concepts.md) concept — the offset is kept when stated. A bare *time-of-day* literal (`opening = 09:00:00`) is still rejected: a time alone has no date to attach to, so include the date or quote it as a string.+!!! tip "TOML temporal literals are native `Date` / `Time` inputs"+ A top-level TOML date or datetime literal (e.g. `hearing = 2026-09-01` or `departure = 2026-07-07T15:40:00+02:00`) maps directly to the native [`Date`](../../building-methods/concepts/native-concepts.md) concept — the offset is kept when stated. A bare *time-of-day* literal (`opening = 09:00:00`) maps to the native `Time` concept; it never silently becomes a `Date` (a time alone has no date to attach to, so shaping a `Time` into a `Date` slot fails the compatibility check).### Auto-detection and ambiguitydiff --git a/docs/under-the-hood/codegen-projections.md b/docs/under-the-hood/codegen-projections.mdnew file mode 100644index 000000000..e49195894--- /dev/null+++ b/docs/under-the-hood/codegen-projections.md@@ -0,0 +1,127 @@+# Codegen Projections++Codegen turns one **normalized library crate** — the flat, fully-qualified, self-contained, fingerprinted snapshot of a resolved library (see [`pipelex resolve`](../tools/cli/build/structures.md) and the [Library Crate Format](https://mthds.ai)) — into typed, documented artifacts for each consumer. The crate is the single authority: codegen never re-derives meaning from raw bundles and never calls a model.++This page describes the projection engine in `pipelex/codegen/`. The formal contract lives in `docs/specs/pipelex-codegen.md` (workspace root).++## Two resolved layers, then emitters++The engine reads the crate through two neutral, language-agnostic layers, so every emitter consumes the same resolved shape rather than re-deriving the semantic mapping:++- **Resolved fields** (`pipelex/codegen/resolved_fields.py`) — one structure field becomes a `ResolvedType` tree (`text`/`number`/`concept`/`list`/`dict`/`literal`/…). Inline `choices` become a literal; a bare concept ref is promoted to its domain; natives are flagged. Where the source under-specifies a shape — a `list` with no `item_type`, a `concept` field with no ref — the resolved type is `ANY` carrying an explicit **imprecision marker** (never a guess).+- **Resolved concepts** (`pipelex/codegen/resolved_concepts.py`) — one crate concept becomes a `ResolvedConcept`: its type name inputs (domain, code, collision flag), its description, its refinement base, and either its resolved fields, or a structureless/opaque marker. Class naming is decided here once: a concept code that is unique across the crate stays bare (`Report`); a code that collides across domains is domain-qualified so a definition and every reference agree.++An emitter then walks the `ResolvedLibrary` and renders text. Emitters live in `pipelex/codegen/emitters/`, dispatched by `emit_types(crate, target=...)`.++## Targets++The `types` projection ranges over the crate's concept set (one type per qualified concept). Three targets ship today:++- **`python-structures`** — Pipelex runtime `StructuredContent` subclasses (the runtime idiom, the successor to `pipelex build structures`). Native references map to the runtime content classes (`TextContent`, …); native concepts themselves are not re-emitted.+- **`python-pydantic`** — plain `pydantic.BaseModel` types with no Pipelex imports. Every concept is emitted uniformly, including the materialized natives, so the module depends only on `pydantic` and the standard library.+- **`ts-zod`** — a pure TypeScript + Zod types file (`types.ts`, imports only `zod`) plus a `binder.ts` companion. `types.ts` holds Zod schemas and their inferred types; type names are the concept codes; **field keys are the crate's snake_case wire names verbatim**. Concept references use `z.lazy(() => XSchema)` so declaration order is irrelevant and cycles are handled.++### The ts-zod types file and its binder++`types.ts` stays dependency-free and portable (only `zod`). Its object keys are **wire-native** (snake_case), so a schema validates a wire payload *directly*. `binder.ts` is the thin companion exposing one typed `parse<Name>` / `serialize<Name>` pair per concept — each a direct `Schema.parse`, with no key-remapping layer. A pipe's IO types are concepts, so a pipe's output parser / input serializer is just the binder pair for those concept types — the binder is the concept-set-wide realization of the per-pipe parse/serialize helpers.++Wire-native keys (D10) are a deliberate correctness choice: a camelCase-keyed schema would need a binder that remaps keys, and a *generic* deep remap cannot tell a schema-declared field key from arbitrary data inside a `z.record()` / `z.unknown()` value (e.g. `native.JSON`'s `json_obj` map) — so it would silently rename the caller's actual data. Keeping keys wire-native removes that hazard entirely; if camelCase ergonomics are wanted later, they must ride a *schema-aware* transform, not a blind key remap.++For **`python-pydantic`**, no key mapping is ever needed either: wire names are already snake_case Python names, so parse/serialize are the native `Model.model_validate(data)` / `model.model_dump(mode="json")`.++### Refinement and native bases++A concept that refines another keeps its `refines` link when the base is **native-backed** (the refinement chain bottoms out at a native such as `Text` or `Number`), because the native is materialized into the crate and the base carries real runtime behavior; the emitter then renders inheritance (`class Summary(TextContent)` / `class Summary(Text)` / `z.lazy(() => TextSchema)`), which round-trips to the correct base class. A concept that refines an **in-crate structured** base has that base's effective structure flattened in during normalization.++Native materialization itself is faithful-or-absent per native: a content-class field maps when it has an unambiguous blueprint form — primitives, dicts, lists, references to other natives, and nested non-native models, which serialize as JSON objects and therefore map to a `dict` blueprint with unspecified value types (declared imprecision the emitters surface, e.g. `native.JSON`'s `json_obj` map). A field with no honest blueprint form at all (for example, a non-optional union) leaves the whole native structureless rather than guessing a partial shape.++## The CLI surface++Two command families drive the engine (formal contract: `docs/specs/pipelex-codegen.md`, workspace root):++- **`pipelex resolve [PATH]… [-f json|toml] [-L DIR]…`** — assembles the closure (working bundles + the local `.mthds/methods/` cache), requires it to be **valid**, and emits the normalized crate to stdout. The verdict rides the exit code, mirroring the bare `validate` group: `0` resolved, `1` the library is invalid (a negative verdict — no crate), `2` no verdict (empty closure / not found).+- **`pipelex codegen <kind> …`** — the two-axis projection family (`kind` × `--target`):+ - `pipelex codegen types --target ts-zod|python-pydantic|python-structures [-o DIR] [PATH]… [-L DIR]…` — projects the crate's concept set for a target and writes each emitted file under the output directory.+ - `pipelex codegen inputs [--pipe <ref>] [-f json|toml] [--explicit] [-o FILE] [PATH]… [-L DIR]…` — projects a runnable inputs template for one pipe (Smart Inputs light shape by default, `--explicit` for the envelope), selected by qualified `--pipe` and defaulting to the closure's declared `main_pipe`.++Both codegen commands load and normalize the crate through the same shared helper the resolver uses (`pipelex/cli/commands/crate_loading.py`), so they share the resolve/validate exit-code contract.++The **agent CLI mirrors** the family for machine consumers: `pipelex-agent codegen types --target <flavor>` runs the same engine and emission layer but presents through the agent CLI's two-stream envelopes (`--format markdown|json` on stdout for success, `--error-format` on stderr for errors — markdown default), and `pipelex-agent codegen check` is the same offline drift check with the verdict as a structured envelope: up-to-date is a success envelope, drift is a `CodegenDriftError` error envelope enumerating `drifts[]` by category (exit `1`), and a missing/unreadable lock is a no-verdict error (exit `2`). There is no `pipelex-agent codegen inputs` — the existing `pipelex-agent inputs` group already surfaces that projection. The legacy `build` family rides the same engine: `pipelex build inputs` renders through the same `input_renderer` engine as `codegen inputs` (they cannot diverge), and `pipelex build structures` is a thin alias of `codegen types --target python-structures` (the old always-qualified per-file generator is gone — the alias emits the single stamped `structures.py` + `codegen.lock`, bare-when-unique names). `pipelex build runner` emits its `structures/` scaffold through the same projection and spells the script's imports and example inputs with the emitted class names (`from structures.structures import …`), so the script and the module always agree.++## The trust chain: stamps, lock, and the offline check++Drift — generated code that no longer matches the method it was generated from — is the tax every codegen system eventually charges. The engine refuses to pay it. Every file `pipelex codegen types` writes carries a machine-parseable **stamp** header, the artifact set is recorded in a sibling **`codegen.lock`**, and `pipelex codegen check` verifies both — entirely offline.++### Stamps++`pipelex/codegen/stamp.py` prepends a fenced comment block (in the target's comment syntax — `#` for Python, `//` for TypeScript) to each generated file:++```python+# >>> pipelex-codegen-stamp >>>+# crate_fingerprint: 1336e999…293f46+# engine_version: 0.38.0+# projection: types / python-pydantic+# options: {}+# content_hash: 1d074a49…6f42e4+# <<< pipelex-codegen-stamp <<<+```++The `crate_fingerprint` is the crate's semantic hash, so reformatting or commenting a `.mthds` file never changes a stamp — only a change to the method's effective type surface does. The `content_hash` is a SHA-256 over the body **below** the stamp, so a lone file can testify that it has not been hand-edited, without the engine, the network, or the lock.++### Lock++`pipelex/codegen/lock.py` writes `codegen.lock` (human-diffable TOML) recording each generated artifact's path and body hash, plus the crate fingerprint and engine version the set was built against. The lock catches the one drift class per-file stamps cannot: a deleted concept whose stale generated file lingers on disk. It is a Pipelex-owned artifact, distinct from the standard's `methods.lock` (which pins remote dependency versions).++### Offline check++`pipelex codegen check [DIR]` (`pipelex/codegen/check.py`) is pure hashing — **no engine, no network, no API key** — so any client (this CLI, an SDK, a short CI script) implements it identically. It reports drift by category:++- **missing** — an artifact in the lock is absent on disk;+- **modified** — a file's body no longer matches its locked hash;+- **hand-edited** — a file's stamp is stripped or its recorded content hash no longer matches the body;+- **orphan** — a stamped generated file on disk that the lock does not track.++The verdict rides the exit code (mirroring `resolve` / `validate`): `0` current, `1` drift present, `2` no lock found. Regeneration against the engine is a **dev action**; the offline check is the **CI action** — so template improvements never redden a consumer's CI.++### Idempotent emission++`pipelex/codegen/emission.py` ties it together in two layers. `build_stamped_projection` is the **pure** core: it stamps each emitted body and assembles the matching `codegen.lock` content — no filesystem access — so it is the single source of truth for what a projection *is* on disk. `write_stamped_projection` rides it to materialize locally: it writes each file only when its content changed (write-if-changed — no mtime churn, clean diffs, watch-mode friendly), removes any previously-tracked stamped file that dropped out of the set, and rewrites the lock. Only files the tool itself stamped are ever removed, so a hand-authored file sharing the output directory is never touched. `codegen inputs` applies the same write-if-changed rule to its single (unstamped) template file, so a full regeneration pass over a committed consumer is a true no-op when everything is current.++## Serving the engine over HTTP++The same engine backs the `pipelex-api` routes (`POST /v1/resolve`, `POST /v1/codegen`, and the re-pointed `/v1/build/*` — see `docs/specs/pipelex-codegen.md` → "Route envelopes" at the workspace root). Two host-facing cores make that possible without any CLI plumbing:++- **`pipelex.pipeline.resolve_bundle.resolve_crate_from_contents`** resolves **in-memory** MTHDS contents (strings, with optional per-content sources) into the normalized crate. It mirrors `validate_bundle`'s in-memory arm — the same `translate_to_validate_bundle_error` cascade, so an invalid library raises the one shared `ValidateBundleError` and a resolve verdict cannot drift from a validate verdict — and the same **loaded-on-success contract**: the library is left loaded and current for the host to read live pipes from, and the host owns its teardown. Resolution is static (no dry-run sweep), matching `pipelex resolve`.+- **`build_stamped_projection`** (above) gives the host the stamped artifact set plus the lock as pure content. A client that writes the served files and lock verbatim reproduces a local run byte-for-byte — the offline `codegen check` passes on the written tree exactly as it would locally. There is deliberately no server-side check route: the check is offline by design.++## Surfacing imprecision, never guessing++A deterministic tool that guesses is a liability. Where a resolved type carries an imprecision marker, the emitter surfaces it rather than inventing a shape:++- Python emitters append an inline `# imprecise: <reason>` comment on the field.+- The ts-zod emitter emits `z.unknown()` plus a JSDoc `@imprecise <reason>` tag.++Two concept-level cases are surfaced the same honest way:++- A **structureless** concept (no structure, no refinement) projects as an opaque type (an empty model / `z.unknown()`) with the imprecision stated in its docstring.+- A **Python-class-backed** concept (`structure = "<ClassName>"`, whose shape lives only in hand-written Python, not in MTHDS) is surfaced as opaque — the bare class name is never silently emitted into a portable crate.++Opaque is always **pass-through, never lossy**: the ts-zod `z.unknown()` hands the wire object through verbatim, and the Python emitters set `model_config = ConfigDict(extra="allow")` on opaque classes so `model_validate` keeps every field (pydantic's default `extra="ignore"` would silently strip the content). The owner of a Python-class-backed concept recovers the typed object by validating with their own class (`MyLegacyClass.model_validate(payload)`); every other consumer gets the honest untyped object.++## The extension-file story++Generated code is never edited — hand edits are overwritten on regeneration, and the trust chain treats them as drift. Customization lives in **sibling extension files** that survive regeneration, and each generated file's header says so:++- **Python** — subclass the generated type from a sibling module:++ ```python+ # my_types_ext.py+ from .structures import Report++ class MyReport(Report):+ ...+ ```++- **TypeScript** — augment the generated type from a sibling module via declaration merging.++Every generated file carries an `AUTOGENERATED — DO NOT EDIT` header naming its projection and pointing at this mechanism.diff --git a/docs/under-the-hood/distributed-content-generation.md b/docs/under-the-hood/distributed-content-generation.mdindex 644e954d7..01e2e9434 100644--- a/docs/under-the-hood/distributed-content-generation.md+++ b/docs/under-the-hood/distributed-content-generation.md@@ -137,7 +137,7 @@ async def generate_and_store_images(img_gen_assignment):```!!! info "What crosses the boundary"- `ImageContent` carries `url` (storage URI), `public_url`, `mime_type`, `size`, `caption` — but never raw bytes. The `url` can be an S3 URI, HTTP URL, or local file path depending on storage configuration.+ `ImageContent` carries `url` (storage URI), `public_url`, `mime_type`, paired `width`/`height`, and `caption` — but never raw bytes. The `url` can be an S3 URI, HTTP URL, or local file path depending on storage configuration.What gets stored vs. what crosses the boundary, by content type:diff --git a/docs/under-the-hood/error-model.md b/docs/under-the-hood/error-model.mdindex d429417ba..d803ba21c 100644--- a/docs/under-the-hood/error-model.md+++ b/docs/under-the-hood/error-model.md@@ -75,11 +75,11 @@ A bundle-validation failure (`ValidateBundleError`) aggregates per-error data acTogether the two residuals make the **structured-info invariant total**: every invalid verdict carries a non-empty `validation_errors[]`, never a bare message. The builder tries the channels in order — categorized data, then the `dry_run` residual (the more specific channel), then the `blueprint_validation` fallback — and emits exactly one residual only when no earlier channel produced an item.-Besides `category` and `message`, each item carries whatever identity fields its stage produced — `error_type`, `pipe_code`, `concept_code`, `domain_code`, `field_path`, `field_name`, `variable_names`, `missing_concept_code`, `declared_concepts`, and a `source` (the declaring file path, or the per-content source the in-memory load path was given) that hands a consumer the owning file for cross-file diagnostic placement.+Besides `category` and `message`, each item carries whatever identity fields its stage produced — `error_type`, `pipe_code`, `concept_code`, `domain_code`, `field_path`, `field_name`, `variable_names`, `missing_concept_code`, `declared_concepts`, and a `source` (the declaring file path, or the per-content source the in-memory load path was given) that hands a consumer the owning file for cross-file diagnostic placement. When the error has a deterministic remedy, the item also carries a [`suggested_fix`](#suggested_fix-structured-deterministic-fixes).**Signatures are never an error.** An unimplemented `PipeSignature` reached during validation is a *runnability fact*, not a validation failure: the validator no longer raises on it. The assembled library's outstanding signatures ride the validation report's `pending_signatures`, and `is_runnable = not pending_signatures`. `allow_signatures` is a sweep-mechanics flag only (whether signature pipes are mock-run and listed in `validated_pipes`) — it does not change the verdict, so strict ≡ lenient in the report body. The "is this a failure?" decision moves to the consumer: the CLI exits non-zero on `not is_runnable` unless `--allow-signatures`; the HTTP caller reads `is_runnable`. (The **execute/run** path is different: running a stub still raises `PipeSignatureNotExecutableError`.)-**Host-wiring guards are programmer errors, not content verdicts.** `validate_bundle` / `load_concepts_only`'s "provide exactly one of `mthds_contents` / `mthds_file_path`" and the `mthds_sources`-length-mismatch guards raise `PipelexUnexpectedError` (→ 500, redacted under STRICT), not `ValidateBundleError` — a caller wiring bug must not be reported as if the submitted bundle were invalid. The empty-`mthds_contents` guard stays caller-facing (it can legitimately reflect an end user submitting no bundles).+**Host-wiring guards are programmer errors, not content verdicts.** `validate_bundle`'s "provide exactly one of `mthds_contents` / `mthds_file_path`" guard and `resolve_crate_from_contents`'s `mthds_sources`-length-mismatch guard raise `PipelexUnexpectedError` (→ 500, redacted under STRICT), not `ValidateBundleError` — a caller wiring bug must not be reported as if the submitted bundle were invalid. The empty-`mthds_contents` guard stays caller-facing (it can legitimately reflect an end user submitting no bundles).`ValidationErrorItem` and the builder are the single source of truth across surfaces: `build_validation_error_items()` (`pipelex/pipeline/validation_errors.py`) is called by both `ValidateBundleError.to_error_report()` (the API path) and the agent CLI's `extract_validation_errors()` (the CLI JSON envelope), so the two structured shapes cannot drift. The item lives in `pipelex/base_exceptions.py` alongside `ErrorReport` — not next to the source error-data models — because `ErrorReport` references it as a typed field and the root exceptions module must not import the `pipelex.core` error modules.@@ -95,6 +95,22 @@ report.http_status # 422 / 429 / 500 — for HTTP adapters!!! warning "`ErrorReport` is `extra="forbid"`"`from_dict()` rejects unknown keys, so it is the strict inverse of `to_dict()`. A report dict that crosses a serialization boundary and fails validation on the way back is an internal contract bug — the writer and the reader share the schema within one deploy. A cross-boundary recovery helper that rebuilds a report (e.g. a distributed-worker bridge) is expected to catch that `ValidationError` and synthesize a fallback report so failure-webhook delivery stays intact while keeping the contract bug visible; any other caller of `from_dict()` should treat the validation failure as a bug to fix.+### `suggested_fix` — structured deterministic fixes++When a validation error has a deterministic remedy, its `ValidationErrorItem` carries a `suggested_fix` — a `SuggestedFix` (`pipelex/suggested_fix.py`, deliberately stdlib+pydantic-only so `pipelex.base_exceptions` can import it without a cycle; naming is brand-neutral, fixes are a language-level concept):++- `fix_code` — the kebab-case rule id (e.g. `match-sequence-output`). The planner's `KNOWN_FIX_CODES` set is the validation set for user-facing rule filters (`--select` / `--ignore`); an unknown code is rejected loudly, never lenient-ignored, because a typo'd filter selects *behavior*.+- `description` — human-readable statement of the change.+- `safety` — `safe` fixes may be auto-applied; `unsafe` ones require explicit opt-in.+- `source` — the file the ops target, when known (multi-file libraries). An applier must only apply ops to the file they target.+- `ops[]` — the fix itself, as **semantic TOML patch ops** addressed by table path (`FixOpKind`: `set_key`, `ensure_table`, `delete_key`, `delete_table`, `rename_table_key`; each op's `table_path` follows the same conventions as the items' `field_path`). The ops are the machine contract; any rendered diff or `💡 Suggested fix:` line is presentation.++The **fix planner** (`pipelex/pipeline/fixes/planner.py`) translates enriched typed error data into `SuggestedFix` payloads — pure functions keyed strictly on `error_type` + structured fields, never on message strings. Each rule fires only when its enrichment is present (set only at the raise sites that know the correct value), so the same error type raised elsewhere without enrichment is structurally suppressed. The planner runs inside `build_validation_error_items()`, so every consumer of the validation report — CLI, API, MCP — sees fixes with zero extra plumbing.++Applying fixes is the runtime's job too: the **applier** (`pipelex/pipeline/fixes/applier.py`) mutates a tomlkit DOM in place per op (guarded — an op whose target table is absent is skipped and reported, never raised) and then reflows the whole file to canonical MTHDS style, and the **convergence loop** (`pipelex/pipeline/fixes/fix_loop.py`) runs validate → apply SAFE fixes → re-validate to a fixed point, reporting non-convergence loudly. The user-facing surface is [`pipelex fix bundle`](../tools/cli/fix.md).++On the hosted API the same payload rides the wire verbatim as `validation_errors[].suggested_fix`; how it appears in HTTP error responses is documented on the API side, in `pipelex-api`'s `docs/error-responses.md` → "Suggested fixes".+---## Classification Enumsdiff --git a/mkdocs.yml b/mkdocs.ymlindex c7ced8fff..7ce5022db 100644--- a/mkdocs.yml+++ b/mkdocs.yml@@ -308,6 +308,7 @@ plugins:- under-the-hood/index.md: "Under the Hood Overview"- under-the-hood/architecture-overview.md: "Architecture Overview"- under-the-hood/build-time-elaboration.md: "Build-time Elaboration"+ - under-the-hood/codegen-projections.md: "Codegen Projections"- under-the-hood/dry-run-mock-generation.md: "Dry Run Mock Generation"- under-the-hood/execution-graph-tracing.md: "Execution Graph Tracing"- under-the-hood/image-handling-in-llm-prompts.md: "Image Handling in LLM Prompts"diff --git a/pipelex/builder/runner_code.py b/pipelex/builder/runner_code.pyindex 1fbdcfa04..c77765720 100644--- a/pipelex/builder/runner_code.py+++ b/pipelex/builder/runner_code.py@@ -5,44 +5,27 @@ This module provides functions to generate executable Python code for Pipelex piMain Functions:--------------- generate_runner_code(pipe): Generate complete executable Python script++Custom structure classes are imported from the codegen engine's single-module projection+(`structures/structures.py`, emitted by `codegen types --target python-structures` — see+`pipelex/codegen/`). A class that lives in a real importable module (a user-registered class backing+a `structure = "<ClassName>"` concept) is imported from that module instead."""from typing import Any-from pydantic import BaseModel, ConfigDict--from pipelex.core.concepts.concept import Conceptfrom pipelex.core.concepts.concept_representation_generator import (ConceptRepresentationFormat,ConceptRepresentationGenerator,)from pipelex.core.concepts.native.concept_native import NativeConceptCode-from pipelex.core.domains.domain import SpecialDomainfrom pipelex.core.pipes.inputs.input_stuff_specs import InputStuffSpecsfrom pipelex.core.pipes.pipe_abstract import PipeAbstractfrom pipelex.core.pipes.variable_multiplicity import VariableMultiplicity-from pipelex.tools.misc.string_utils import pascal_case_to_snake_case---class CustomClassInfo(BaseModel):- """Information about a custom structure class for import generation."""-- model_config = ConfigDict(frozen=True)-- class_name: str- domain_code: str- concept_code: str-- @property- def module_name(self) -> str:- """Get the module name (filename without .py) for this class."""- concept_snake_case = pascal_case_to_snake_case(self.concept_code)- return f"{self.domain_code}__{concept_snake_case}"+from pipelex.hub import get_class_registry- @property- def import_statement(self) -> str:- """Get the import statement for this class (uses sys.path from script directory)."""- return f"from structures.{self.module_name} import {self.class_name}"+STRUCTURES_MODULE = "structures.structures"+"""Module path of the emitted types projection, relative to the generated script's directory."""def _is_multiple(multiplicity: VariableMultiplicity | None) -> bool:@@ -101,61 +84,59 @@ def _get_structure_class_import(class_name: str) -> str | None:return f"from {cls.__module__} import {cls.__name__}"-def _collect_concept_info(concept: Concept) -> CustomClassInfo | None:- """Collect information about a concept for import generation.+def _custom_import_statement(class_name: str, *, emitted_names: set[str]) -> str:+ """Build the import statement for a custom (non-native) structure class.- Args:- concept: The concept to collect info for-- Returns:- CustomClassInfo if it's a custom concept, None if native+ A name the codegen engine emitted comes from the single-module projection. Otherwise, a class+ registered from a real importable module (a user-authored class backing a+ `structure = "<ClassName>"` concept) is imported from that module; runtime-materialized classes+ (exec'd, no importable module) fall back to the projection module, which carries their emitted+ counterparts."""- if SpecialDomain.is_native(concept.domain_code):- return None-- # For custom concepts, use the concept code as the class name- # The structure class name should match the concept code- return CustomClassInfo(- class_name=concept.structure_class_name,- domain_code=concept.domain_code,- concept_code=concept.code,- )+ if class_name not in emitted_names:+ registered = get_class_registry().get_class(name=class_name)+ if registered is not None and registered.__module__ not in {"__main__", "builtins"}:+ return f"from {registered.__module__} import {class_name}"+ return f"from {STRUCTURES_MODULE} import {class_name}"-def _collect_imports_for_inputs(inputs: InputStuffSpecs) -> tuple[set[str], dict[str, CustomClassInfo]]:- """Collect all imports needed for a pipe's inputs.+def _collect_imports_for_inputs(inputs: InputStuffSpecs, *, class_name_overrides: dict[str, str]) -> tuple[set[str], set[str]]:+ """Collect all class names needed for a pipe's inputs.Args:inputs: The pipe inputs+ class_name_overrides: Runtime-class-name -> emitted-name mapping applied to the rendered codeReturns:- Tuple of (set of native class names, dict mapping class name to CustomClassInfo)+ Tuple of (native class names, custom class names) — custom names are post-override spellings"""native_classes: set[str] = set()- custom_classes: dict[str, CustomClassInfo] = {}+ custom_classes: set[str] = set()for input_req in inputs.root.values():concept = input_req.conceptstructure_class = concept.get_structure_class()- # Get imports from the representation generator- generator = ConceptRepresentationGenerator(ConceptRepresentationFormat.PYTHON)+ # Get imports from the representation generator (includes nested custom classes)+ generator = ConceptRepresentationGenerator(ConceptRepresentationFormat.PYTHON, class_name_overrides=class_name_overrides)generator.generate_representation(concept.concept_ref, structure_class=structure_class)for class_name in generator.imports_needed:if NativeConceptCode.is_native_structure_class(class_name):native_classes.add(class_name)else:- # For custom classes, we need the concept info- # The class name should be the structure class name which is the concept code- custom_info = _collect_concept_info(concept)- if custom_info and custom_info.class_name == class_name:- custom_classes[class_name] = custom_info+ custom_classes.add(class_name)return native_classes, custom_classes-def generate_runner_code(pipe: PipeAbstract, *, output_multiplicity: bool = False, library_dir: str | None = None) -> str:+def generate_runner_code(+ pipe: PipeAbstract,+ *,+ output_multiplicity: bool = False,+ library_dir: str | None = None,+ class_name_overrides: dict[str, str] | None = None,+) -> str:"""Generate the complete Python runner code for a pipe.This generates a runnable Python script with:@@ -168,17 +149,22 @@ def generate_runner_code(pipe: PipeAbstract, *, output_multiplicity: bool = Falspipe: The pipe to generate runner code foroutput_multiplicity: Whether the output is a list (e.g., Text[])library_dir: Directory containing the MTHDS bundles to load+ class_name_overrides: Runtime-class-name -> emitted-name mapping so the script spells custom+ classes the way the accompanying types projection defines them (bare-when-unique)"""- # Get output information- structure_class_name = pipe.output.concept.structure_class_name+ overrides = class_name_overrides or {}+ emitted_names = set(overrides.values())++ # Get output information (spelled per the emitted projection when a mapping is provided)+ runtime_output_class_name = pipe.output.concept.structure_class_name+ structure_class_name = overrides.get(runtime_output_class_name, runtime_output_class_name)is_native = NativeConceptCode.is_native_structure_class(structure_class_name)- custom_info = None if is_native else _collect_concept_info(pipe.output.concept)# Check if output is the special "Anything" concept (no specific content type)is_anything_output = pipe.output.concept.code == NativeConceptCode.ANYTHING# Collect all imports needed for inputs- native_classes, custom_classes = _collect_imports_for_inputs(pipe.inputs)+ native_classes, custom_classes = _collect_imports_for_inputs(pipe.inputs, class_name_overrides=overrides)# Track if we need to import Any from typing (for Anything output)needs_any_import = is_anything_output@@ -187,8 +173,8 @@ def generate_runner_code(pipe: PipeAbstract, *, output_multiplicity: bool = Falsif not is_anything_output:if is_native:native_classes.add(structure_class_name)- elif custom_info:- custom_classes[structure_class_name] = custom_info+ else:+ custom_classes.add(structure_class_name)# Build import sectionimport_lines: list[str] = []@@ -221,12 +207,11 @@ def generate_runner_code(pipe: PipeAbstract, *, output_multiplicity: bool = Falsnative_imports.append(import_stmt)import_lines.extend(native_imports)- # Add custom structure class imports from structures folder+ # Add custom structure class imports (from the types projection or their real module)if custom_classes:import_lines.append("")- for class_name in sorted(custom_classes.keys()):- custom_info = custom_classes[class_name]- import_lines.append(custom_info.import_statement)+ for class_name in sorted(custom_classes):+ import_lines.append(_custom_import_statement(class_name, emitted_names=emitted_names))import_lines.extend([@@ -244,6 +229,7 @@ def generate_runner_code(pipe: PipeAbstract, *, output_multiplicity: bool = Falsresult, _ = input_req.concept.render_concept_representation(output_format=ConceptRepresentationFormat.PYTHON,is_multiple=is_multiple,+ class_name_overrides=overrides,)python_code = _format_representation_as_python(result, is_multiple=is_multiple)input_entries.append(f' "{var_name}": {python_code},')diff --git a/pipelex/cli/_cli.py b/pipelex/cli/_cli.pyindex 9786aa925..cc50dcae9 100644--- a/pipelex/cli/_cli.py+++ b/pipelex/cli/_cli.py@@ -6,6 +6,7 @@ from typer.core import TyperGroupfrom typing_extensions import overridefrom pipelex.cli.commands.build.app import build_app+from pipelex.cli.commands.codegen.app import codegen_appfrom pipelex.cli.commands.doctor_cmd import doctor_cmdfrom pipelex.cli.commands.fix.app import fix_appfrom pipelex.cli.commands.graph_cmd import graph_app@@ -13,6 +14,7 @@ from pipelex.cli.commands.init.command import init_cmdfrom pipelex.cli.commands.init.ui.types import InitFocusfrom pipelex.cli.commands.login.command import login_cmdfrom pipelex.cli.commands.plugins_cmd import plugins_app+from pipelex.cli.commands.resolve_cmd import resolve_cmdfrom pipelex.cli.commands.run.app import run_appfrom pipelex.cli.commands.show_cmd import show_appfrom pipelex.cli.commands.update_cmd import update_cmd@@ -33,6 +35,8 @@ _CORE_COMMAND_ORDER: list[str] = ["build","validate","fix",+ "resolve",+ "codegen","run","graph","show",@@ -234,6 +238,8 @@ app.add_typer(name="fix",help="Apply deterministic safe fixes to a bundle (.mthds) and re-validate until valid",)+app.command(name="resolve", help="Resolve a bundle closure into the normalized library crate (JSON or TOML) on stdout")(resolve_cmd)+app.add_typer(codegen_app, name="codegen", help="Project the resolved crate into typed, runnable artifacts (types, inputs) and check for drift")app.add_typer(run_app, name="run", help="Run a method or pipe, optionally providing a specific bundle file (.mthds)")app.add_typer(graph_app, name="graph", help="Generate and render execution graphs")app.add_typer(show_app, name="show", help="Show configuration, pipes, and list AI models")diff --git a/pipelex/cli/agent_cli/CLAUDE.md b/pipelex/cli/agent_cli/CLAUDE.mdindex 28f3c9b70..c6e17d167 100644--- a/pipelex/cli/agent_cli/CLAUDE.md+++ b/pipelex/cli/agent_cli/CLAUDE.md@@ -6,7 +6,7 @@ Machine-first CLI for running and validating Pipelex method bundles (`.mthds` fiTwo independent options control the two output streams:-- `--format markdown|json` — **success/useful output**. Defaults to markdown. Accepted by `run`, `validate`, `fix`, `init`, `models`, `check-model`, `doctor`. Goes to stdout. Threaded explicitly to `agent_success_formatted()` from each command function — no hidden state.+- `--format markdown|json` — **success/useful output**. Defaults to markdown. Accepted by `run`, `validate`, `fix`, `init`, `models`, `check-model`, `doctor`, `codegen types`, `codegen check`. Goes to stdout. Threaded explicitly to `agent_success_formatted()` from each command function — no hidden state.- `--error-format markdown|json` — **error reporting** (stderr). Optional. When omitted, **inherits the value of `--format`**, so `--format json` still flips both as it did historically. Accepted by the same commands as `--format`.Only the error format is backed by a module-level `ContextVar` in `agent_output.py` (`_agent_cli_error_format`). The reason: `agent_error()` is called from sites that don't see the Typer option — factory init failures (`agent_cli_factory.py`), the unknown-command handler in `PipelexAgentCLI.get_command`, log-level/runner validation in the app callback, and any future site in shared library/runtime code. The ContextVar lets all of them honor `--error-format` (or `--format`'s inherited value) for free. JSON is the default so errors raised before any command opts in stay machine-parseable.@@ -66,6 +66,10 @@ commands/bundle_cmd.py # inputs bundle — inputs from bundle file/directorymethod_cmd.py # inputs method — inputs for installed method_inputs_core.py # Shared inputs logic+ codegen/ # codegen — crate projections + offline drift check+ app.py # codegen_app Typer, subcommand registration+ types_cmd.py # codegen types — project the crate's concept set (stamped files + codegen.lock)+ check_cmd.py # codegen check — offline drift check (pure hashing, no Pipelex boot)fmt_cmd.py # fmt — format file via plxt passthroughlint_cmd.py # lint — lint file via plxt passthroughplxt_passthrough.py # Shared helper for plxt subprocess delegation@@ -88,6 +92,7 @@ commands/| `fmt` | Formats a .mthds/.toml/.plx file in-place (delegates to plxt) || `lint` | Lints a .mthds/.toml/.plx file for errors (delegates to plxt) || `inputs` | Generates an example inputs template for a pipe/bundle/method (pipe\|bundle\|method subcommands). `--format json\|toml` (template serialization, default: json — NOT the markdown\|json pair); `toml` prints raw TOML to stdout |+| `codegen` | Agent mirror of the bare `pipelex codegen` family (types\|check subcommands). `types --target <flavor>` resolves the closure into the normalized crate and writes stamped typed artifacts + `codegen.lock` (write-if-changed); `check` is the offline drift check (pure hashing, no Pipelex boot — exit 0 current, 1 drift as a structured `CodegenDriftError` with `drifts[]`, 2 no/unreadable lock). Both: `--format markdown\|json` (success, default: markdown) + `--error-format markdown\|json` (errors, defaults to `--format`'s value). || `concept` | Converts a JSON concept spec into raw TOML (stdout) || `pipe` | Converts a JSON pipe spec into raw TOML (stdout). A spec with a `type` (via `--type` or a `type` key) is that concrete pipe; a **typeless** spec (no `type`) is a signature and renders a `[pipe.x]` section with no type line. An explicit `type = "PipeSignature"` is rejected with a migration error — `PipeSignature` is not a type. || `models` | Lists available model presets, aliases, and waterfalls. `--format markdown\|json` (success, default: markdown) + `--error-format markdown\|json` (errors, defaults to `--format`'s value) |diff --git a/pipelex/cli/agent_cli/_agent_cli.py b/pipelex/cli/agent_cli/_agent_cli.pyindex 9e11a6305..209f2c577 100644--- a/pipelex/cli/agent_cli/_agent_cli.py+++ b/pipelex/cli/agent_cli/_agent_cli.py@@ -12,6 +12,7 @@ from pipelex.cli.agent_cli.commands.accept_gateway_terms_cmd import agent_acceptfrom pipelex.cli.agent_cli.commands.agent_cli_factory import silence_logging_for_agent_clifrom pipelex.cli.agent_cli.commands.agent_output import CliOutputFormat, agent_error, set_agent_cli_error_formatfrom pipelex.cli.agent_cli.commands.check_model_cmd import agent_check_model_cmd+from pipelex.cli.agent_cli.commands.codegen.app import codegen_appfrom pipelex.cli.agent_cli.commands.concept_cmd import concept_cmdfrom pipelex.cli.agent_cli.commands.doctor_cmd import agent_doctor_cmdfrom pipelex.cli.agent_cli.commands.fix.app import fix_app@@ -40,6 +41,7 @@ class PipelexAgentCLI(TyperGroup):"fmt","lint","inputs",+ "codegen","concept","pipe","models",@@ -139,6 +141,7 @@ app.add_typer(fix_app, name="fix", help="Fix a bundle file or pipeline directoryapp.command(name="fmt", help="Format a .mthds, .toml, or .plx file in-place")(fmt_cmd)app.command(name="lint", help="Lint a .mthds, .toml, or .plx file")(lint_cmd)app.add_typer(inputs_app, name="inputs", help="Generate example input JSON for a pipe")+app.add_typer(codegen_app, name="codegen", help="Project the crate into typed artifacts (types) and check drift offline (check)")app.command(name="concept", help="Structure a concept from JSON spec and output TOML")(concept_cmd)app.command(name="pipe", help="Structure a pipe from JSON spec and output TOML")(pipe_cmd)app.command(name="models", help="List available model presets, aliases, and waterfalls")(agent_models_cmd)diff --git a/pipelex/cli/agent_cli/commands/agent_output.py b/pipelex/cli/agent_cli/commands/agent_output.pyindex 12e8cc5b4..3c9fd5d78 100644--- a/pipelex/cli/agent_cli/commands/agent_output.py+++ b/pipelex/cli/agent_cli/commands/agent_output.py@@ -147,6 +147,13 @@ AGENT_ERROR_HINTS: dict[str, str] = {"ValueError": "Check that the provided value is valid for the parameter (e.g., --type must be a valid pipe type)",# Init errors"InitConfigError": "Check the --config JSON and ensure backend keys match available backends in the template",+ # Codegen errors+ "CodegenDriftError": (+ "Regenerate with 'pipelex-agent codegen types --target <flavor> -o <dir>' (a dev action). "+ "Check 'drifts' in this error response for the drifting artifacts and their categories."+ ),+ "CodegenLockNotFoundError": "Pass the directory that holds codegen.lock, or generate first with 'pipelex-agent codegen types'",+ "CodegenLockError": ("The codegen.lock file is unreadable or malformed — remove it, then regenerate with 'pipelex-agent codegen types'"),# Unknown command"UnknownCommandError": "Check 'valid_commands' in this error response for available commands",}@@ -173,6 +180,9 @@ AGENT_ERROR_DOMAINS: dict[str, str] = {"PipelineRequestError": "input","GraphSpecParseError": "input","UnknownCommandError": "input",+ "CodegenDriftError": "input",+ "CodegenLockNotFoundError": "input",+ "CodegenLockError": "input",# config = environment/config changes needed"ClientAuthenticationError": "config","PipeOperatorModelChoiceError": "config",diff --git a/pipelex/cli/agent_cli/commands/codegen/__init__.py b/pipelex/cli/agent_cli/commands/codegen/__init__.pynew file mode 100644index 000000000..e69de29bbdiff --git a/pipelex/cli/agent_cli/commands/codegen/app.py b/pipelex/cli/agent_cli/commands/codegen/app.pynew file mode 100644index 000000000..0f3daa4a3--- /dev/null+++ b/pipelex/cli/agent_cli/commands/codegen/app.py@@ -0,0 +1,22 @@+"""Agent CLI codegen subcommand group.++The agent-CLI mirror of the bare `pipelex codegen` family (see `docs/specs/pipelex-codegen.md` →+"CLI: codegen"): the same engine, presented through the agent CLI's two-stream envelopes+(`--format` / `--error-format`, markdown default). `inputs` has no mirror here — the existing+`pipelex-agent inputs` group already surfaces that projection.+"""++from __future__ import annotations++import typer++from pipelex.cli.agent_cli.commands.codegen.check_cmd import agent_codegen_check_cmd+from pipelex.cli.agent_cli.commands.codegen.types_cmd import agent_codegen_types_cmd++codegen_app = typer.Typer(+ add_completion=False,+ no_args_is_help=True,+)++codegen_app.command(name="types", help="Project the crate's concept set into typed artifacts for a target flavor")(agent_codegen_types_cmd)+codegen_app.command(name="check", help="Verify generated artifacts are current — offline, no engine, no API key")(agent_codegen_check_cmd)diff --git a/pipelex/cli/agent_cli/commands/codegen/check_cmd.py b/pipelex/cli/agent_cli/commands/codegen/check_cmd.pynew file mode 100644index 000000000..bacfda014--- /dev/null+++ b/pipelex/cli/agent_cli/commands/codegen/check_cmd.py@@ -0,0 +1,101 @@+"""Agent CLI codegen check command — the offline drift check.++Same pure-hashing core as the bare `pipelex codegen check` (no Pipelex boot, no network, no API+key), with the agent CLI's presentation: an up-to-date verdict is a structured success envelope on+stdout; drift is a produced **negative verdict** — a structured error envelope on stderr carrying+the drifting artifacts, exit `1` (mirroring the validate surface); a missing or unreadable+`codegen.lock` is a no-verdict condition, exit `2`.+"""++from __future__ import annotations++from pathlib import Path+from typing import Annotated, Any++import typer++from pipelex.cli.agent_cli.commands.agent_output import (+ CliOutputFormat,+ agent_error,+ agent_success_formatted,+ set_agent_cli_error_format,+)+from pipelex.codegen.check import run_codegen_check+from pipelex.codegen.exceptions import CodegenLockError+from pipelex.codegen.lock import CODEGEN_LOCK_FILENAME+++def agent_codegen_check_cmd(+ root: Annotated[+ str,+ typer.Argument(help=f"Directory holding the {CODEGEN_LOCK_FILENAME} and generated artifacts (default: current directory)"),+ ] = ".",+ output_format: Annotated[+ CliOutputFormat,+ typer.Option("--format", help="Success output format: markdown (default) or json (structured)"),+ ] = CliOutputFormat.MARKDOWN,+ error_format: Annotated[+ CliOutputFormat | None,+ typer.Option("--error-format", help="Error output format (defaults to --format value): markdown or json"),+ ] = None,+) -> None:+ """Verify generated artifacts are current, offline — no engine, no network, no API key.++ Examples:+ pipelex-agent codegen check ./generated/+ pipelex-agent codegen check ./generated/ --format json+ """+ set_agent_cli_error_format(error_format or output_format)+ root_path = Path(root).expanduser()++ try:+ report = run_codegen_check(root=root_path)++ if not report.lock_found:+ agent_error(+ f"No {CODEGEN_LOCK_FILENAME} found in '{root_path}' — nothing to check.",+ error_type="CodegenLockNotFoundError",+ exit_code=2,+ root=str(root_path),+ )++ if not report.is_current:+ # Produced negative verdict: the artifacts have drifted (exit 1, like an invalid validate).+ drifts = [drift.model_dump(mode="json") for drift in report.drifts]+ agent_error(+ f"Generated artifacts in '{root_path}' have drifted.",+ error_type="CodegenDriftError",+ exit_code=1,+ root=str(root_path),+ is_current=False,+ drifts=drifts,+ )++ result: dict[str, Any] = {+ "success": True,+ "is_current": True,+ "root": str(root_path),+ }+ agent_success_formatted(result, markdown_renderer=_format_codegen_check_markdown, output_format=output_format)++ except typer.Exit:+ raise++ except CodegenLockError as exc:+ # No verdict: the lock exists but cannot be read, so nothing can be checked.+ agent_error(exc.message, error_type=type(exc).__name__, cause=exc, exit_code=2, root=str(root_path))++ except Exception as exc: # noqa: BLE001+ # Agent CLI command boundary: agent_error() (NoReturn) converts any unexpected failure into the structured error payload.+ agent_error(str(exc), error_type=type(exc).__name__, cause=exc)+++def _format_codegen_check_markdown(result: dict[str, Any]) -> str:+ """Render the codegen-check success envelope as agent-readable markdown."""+ return "\n".join(+ [+ "# Generated artifacts up to date",+ "",+ f"Generated artifacts in `{result['root']}` match their `{CODEGEN_LOCK_FILENAME}` — no drift.",+ ]+ )diff --git a/pipelex/cli/agent_cli/commands/codegen/types_cmd.py b/pipelex/cli/agent_cli/commands/codegen/types_cmd.pynew file mode 100644index 000000000..256d428cf--- /dev/null+++ b/pipelex/cli/agent_cli/commands/codegen/types_cmd.py@@ -0,0 +1,145 @@+"""Agent CLI codegen types command — project the crate's concept set into typed artifacts.++Same engine as the bare `pipelex codegen types` (resolve the closure into a normalized crate, emit+the types projection, write stamped files + `codegen.lock` through the write-if-changed layer), with+the agent CLI's presentation: a structured success envelope (markdown default, `--format json`) on+stdout and the structured error envelope on stderr. The resolve verdict maps to the 0/1/2 exit+policy — `1` invalid library (negative verdict), `2` closure not assembled (no verdict).+"""++from __future__ import annotations++from pathlib import Path+from typing import TYPE_CHECKING, Annotated, Any++import typer++from pipelex.cli.agent_cli.commands.agent_cli_factory import make_pipelex_for_agent_cli+from pipelex.cli.agent_cli.commands.agent_output import (+ CliOutputFormat,+ agent_error,+ agent_success_formatted,+ set_agent_cli_error_format,+)+from pipelex.cli.commands.crate_loading import load_normalized_crate+from pipelex.codegen.emission import write_stamped_projection+from pipelex.codegen.emitters.target import CodegenKind, CodegenTarget+from pipelex.codegen.emitters.types_emitter import emit_types+from pipelex.libraries.exceptions import LibraryLoadingError+from pipelex.libraries.pipe.exceptions import PipeLibraryError+from pipelex.pipelex import Pipelex+from pipelex.tools.misc.package_utils import get_package_version++if TYPE_CHECKING:+ from pipelex.libraries.library_crate import LibraryCrate+++def agent_codegen_types_cmd(+ target: Annotated[+ CodegenTarget,+ typer.Option("--target", "-t", help="Codegen target flavor (ts-zod, python-pydantic, python-structures)"),+ ],+ paths: Annotated[+ list[Path] | None,+ typer.Argument(help="Directories of .mthds bundles to resolve into the closure (added to --library-dir)"),+ ] = None,+ output_dir: Annotated[+ str,+ typer.Option("--output", "-o", help="Directory the generated files are written into (default: current directory)"),+ ] = ".",+ library_dir: Annotated[+ list[str] | None,+ typer.Option("--library-dir", "-L", help="Directory of .mthds bundles to load (repeatable)"),+ ] = None,+ output_format: Annotated[+ CliOutputFormat,+ typer.Option("--format", help="Success output format: markdown (default) or json (structured)"),+ ] = CliOutputFormat.MARKDOWN,+ error_format: Annotated[+ CliOutputFormat | None,+ typer.Option("--error-format", help="Error output format (defaults to --format value): markdown or json"),+ ] = None,+) -> None:+ """Project the crate's concept set into typed artifacts for the chosen target.++ Resolves the bundle closure into a normalized library crate, emits the types projection, and+ writes stamped files plus `codegen.lock` (write-if-changed; stale stamped files are pruned).++ Examples:+ pipelex-agent codegen types ./my_pipes/ --target python-pydantic+ pipelex-agent codegen types ./my_pipes/ --target ts-zod -o ./generated/+ pipelex-agent codegen types --target python-structures -L ./shared_pipes/ --format json+ """+ set_agent_cli_error_format(error_format or output_format)+ output_root = Path(output_dir).expanduser()+ # needs_model_specs=True (like `validate`): library validation checks pipe model pins+ # against the deck, so the specs must be loaded even though codegen needs no inference.+ make_pipelex_for_agent_cli(needs_inference=False, needs_model_specs=True)++ try:+ combined_dirs: list[Path] = [*(paths or []), *(Path(lib_dir) for lib_dir in library_dir or [])]++ crate: LibraryCrate+ try:+ crate = load_normalized_crate(library_dirs=combined_dirs or None)+ except (LibraryLoadingError, PipeLibraryError) as exc:+ # Negative verdict: the library is structurally invalid, so no crate can be produced.+ agent_error(f"Cannot resolve — the library is invalid: {exc}", error_type=type(exc).__name__, cause=exc, exit_code=1)+ except FileNotFoundError as exc:+ # No verdict: the closure could not even be assembled.+ agent_error(f"Cannot resolve — {exc}", error_type="FileNotFoundError", cause=exc, exit_code=2)++ emitted = emit_types(crate, target=target)+ report = write_stamped_projection(+ emitted,+ output_dir=output_root,+ crate_fingerprint=crate.fingerprint,+ engine_version=get_package_version(),+ kind=CodegenKind.TYPES,+ target=target,+ )+ result: dict[str, Any] = {+ "success": True,+ "kind": str(CodegenKind.TYPES),+ "target": str(target),+ "output_dir": str(output_root),+ "crate_fingerprint": crate.fingerprint,+ "engine_version": get_package_version(),+ "written": report.written,+ "unchanged": report.unchanged,+ "removed": report.removed,+ "lock_file": str(output_root / report.lock_path),+ }+ agent_success_formatted(result, markdown_renderer=_format_codegen_types_markdown, output_format=output_format)++ except typer.Exit:+ raise++ except Exception as exc: # noqa: BLE001+ # Agent CLI command boundary: agent_error() (NoReturn) converts any unexpected failure into the structured error payload.+ agent_error(str(exc), error_type=type(exc).__name__, cause=exc)++ finally:+ Pipelex.teardown_if_needed()+++def _format_codegen_types_markdown(result: dict[str, Any]) -> str:+ """Render the codegen-types success envelope as agent-readable markdown."""+ lines: list[str] = [+ "# Codegen complete",+ "",+ f"- **Projection:** {result['kind']} → {result['target']}",+ f"- **Output directory:** {result['output_dir']}",+ f"- **Crate fingerprint:** `{result['crate_fingerprint']}`",+ "",+ "## Files",+ "",+ ]+ for filename in result["written"]:+ lines.append(f"- Generated `{filename}`")+ for filename in result["unchanged"]:+ lines.append(f"- Unchanged `{filename}`")+ for filename in result["removed"]:+ lines.append(f"- Removed stale `{filename}`")+ lines.append(f"- Locked `{result['lock_file']}`")+ return "\n".join(lines)diff --git a/pipelex/cli/agent_cli/commands/run/stdin_resolver.py b/pipelex/cli/agent_cli/commands/run/stdin_resolver.pyindex 0b0f7724a..698b331ea 100644--- a/pipelex/cli/agent_cli/commands/run/stdin_resolver.py+++ b/pipelex/cli/agent_cli/commands/run/stdin_resolver.py@@ -10,7 +10,7 @@ from typing import Any, NamedTuple, castfrom pipelex.cli.agent_cli.commands.agent_output import agent_errorfrom pipelex.cli.commands.run._inputs_file_loader import find_default_inputs_file, load_inputs_dict_from_pathfrom pipelex.cli.commands.run._inputs_path_resolver import resolve_inputs_paths-from pipelex.cli.commands.run.exceptions import AmbiguousInputsFilesError, InputsTimeOnlyNotSupportedError+from pipelex.cli.commands.run.exceptions import AmbiguousInputsFilesErrorfrom pipelex.tools.misc.exceptions import JsonTypeError, TomlErrorWORKING_MEMORY_KEY = "working_memory"@@ -216,8 +216,6 @@ def _parse_inputs_arg(inputs_arg: str) -> ParsedCliInputs:agent_error(f"Input file must be a valid JSON dictionary: {inputs_arg}", error_type="JsonTypeError", cause=exc)except TomlError as exc:agent_error(exc.message, error_type="TomlError", cause=exc)- except InputsTimeOnlyNotSupportedError as exc:- agent_error(exc.message, error_type="InputsTimeOnlyNotSupportedError", cause=exc)def _read_stdin_inputs() -> dict[str, Any] | None:diff --git a/pipelex/cli/commands/build/runner/_runner_core.py b/pipelex/cli/commands/build/runner/_runner_core.pyindex 24eac0587..d71301b30 100644--- a/pipelex/cli/commands/build/runner/_runner_core.py+++ b/pipelex/cli/commands/build/runner/_runner_core.py@@ -4,35 +4,35 @@ import asynciofrom typing import TYPE_CHECKINGimport typer+from mthds.package.manifest.schema import MTHDS_STANDARD_VERSIONfrom posthog import tagfrom pipelex.builder.runner_code import generate_runner_codefrom pipelex.cli.cli_factory import make_pipelex_for_cli-from pipelex.cli.commands.build.structures_cmd import generate_structures_from_blueprintsfrom pipelex.cli.error_handlers import (ErrorContext,handle_model_availability_error,handle_model_choice_error,)+from pipelex.codegen.emission import write_stamped_projection+from pipelex.codegen.emitters.naming import runtime_to_emitted_class_names+from pipelex.codegen.emitters.target import CodegenKind, CodegenTarget+from pipelex.codegen.emitters.types_emitter import emit_types+from pipelex.codegen.resolved_concepts import resolve_concepts_from_cratefrom pipelex.core.pipes.exceptions import PipeOperatorModelChoiceErrorfrom pipelex.core.pipes.inputs.exceptions import PipeInputErrorfrom pipelex.core.pipes.variable_multiplicity import parse_concept_with_multiplicity-from pipelex.core.registry_models import CoreRegistryModels-from pipelex.core.stuffs.stuff_content import StuffContentfrom pipelex.hub import (- get_class_registry,- get_func_registry,+ get_current_library_id_or_none,get_library_manager,get_required_pipe,get_telemetry_manager,- resolve_library_dirs,- set_current_library,)+from pipelex.libraries.crate_normalization import normalize_cratefrom pipelex.pipe_operators.exceptions import PipeOperatorModelAvailabilityErrorfrom pipelex.pipelex import PACKAGE_VERSIONfrom pipelex.pipeline.exceptions import ValidateBundleErrorfrom pipelex.pipeline.validate_bundle import validate_bundle-from pipelex.system.registries.class_registry_utils import ClassRegistryUtilsfrom pipelex.system.runtime import IntegrationModefrom pipelex.system.telemetry.events import EventPropertyfrom pipelex.tools.misc.file_utils import (@@ -57,14 +57,6 @@ async def _prepare_runner_core(library_dirs: list[Path] | None = None,) -> None:"""Core logic for generating a Python runner file."""- # Set up library so pipes can be found- lib_manager = get_library_manager()- lib_id, _ = lib_manager.open_library()- set_current_library(library_id=lib_id)- effective_dirs, _ = resolve_library_dirs([str(lib_dir) for lib_dir in library_dirs] if library_dirs else None)- if effective_dirs:- lib_manager.load_libraries(library_id=lib_id, library_dirs=effective_dirs)-all_blueprints: list[PipelexBundleBlueprint] = []if bundle_path:@@ -112,33 +104,31 @@ async def _prepare_runner_core(final_output_path = bundle_dir / f"run_{pipe_code}.py"output_dir = final_output_path.parent- # Generate structures folder FIRST+ # Emit the types projection (stamped structures.py + codegen.lock) into structures/ via the+ # codegen engine. validate_bundle left its loaded-and-validated library open and current, so the+ # crate is read from there (D6: the normalized crate is built from a valid library).structures_output_dir = output_dir / "structures"- if all_blueprints:- get_class_registry().teardown()- get_func_registry().teardown()- get_class_registry().register_classes(CoreRegistryModels.get_all_models())- generated_structures = generate_structures_from_blueprints(- blueprints=all_blueprints,- output_directory=structures_output_dir,- target_path=output_dir,- )- if generated_structures:- typer.secho(f"Generated {len(generated_structures)} structure(s) in: {structures_output_dir}", fg=typer.colors.GREEN)-- if structures_output_dir.exists():- ClassRegistryUtils.register_classes_in_folder(- folder_path=structures_output_dir,- base_class=StuffContent,- is_recursive=True,+ class_name_overrides: dict[str, str] = {}+ library_id = get_current_library_id_or_none()+ crate = get_library_manager().get_crate(library_id) if library_id else None+ if crate is not None:+ normalized_crate = normalize_crate(crate, mthds_version=MTHDS_STANDARD_VERSION)+ emitted = emit_types(normalized_crate, target=CodegenTarget.PYTHON_STRUCTURES)+ report = write_stamped_projection(+ emitted,+ output_dir=structures_output_dir,+ crate_fingerprint=normalized_crate.fingerprint,+ engine_version=PACKAGE_VERSION,+ kind=CodegenKind.TYPES,+ target=CodegenTarget.PYTHON_STRUCTURES,)- ClassRegistryUtils.register_classes_in_folder(- folder_path=output_dir,- base_class=StuffContent,- is_recursive=True,- force_exclude_dirs=[structures_output_dir.resolve()] if structures_output_dir.exists() else None,- )- get_class_registry().register_classes(CoreRegistryModels.get_all_models())+ for filename in report.written:+ typer.secho(f"Generated {structures_output_dir / filename}", fg=typer.colors.GREEN)+ for filename in report.unchanged:+ typer.secho(f"Unchanged {structures_output_dir / filename}", fg=typer.colors.BLUE)+ for filename in report.removed:+ typer.secho(f"Removed stale {structures_output_dir / filename}", fg=typer.colors.YELLOW)+ class_name_overrides = runtime_to_emitted_class_names(resolve_concepts_from_crate(normalized_crate))output_is_list = Falsefor blueprint in all_blueprints:@@ -150,13 +140,16 @@ async def _prepare_runner_core(if library_dirs:pipelex_library_dir = str(library_dirs[0].resolve())- elif bundle_path:- pipelex_library_dir = str(bundle_path.parent.resolve())else:- pipelex_library_dir = None+ pipelex_library_dir = str(bundle_path.parent.resolve())try:- runner_code = generate_runner_code(pipe=the_pipe, output_multiplicity=output_is_list, library_dir=pipelex_library_dir)+ runner_code = generate_runner_code(+ pipe=the_pipe,+ output_multiplicity=output_is_list,+ library_dir=pipelex_library_dir,+ class_name_overrides=class_name_overrides,+ )except Exception as exc:# CLI command boundary: any failure generating the runner code is reported to the user and exits via typer.Exit.typer.secho(f"Error generating runner code: {exc}", fg=typer.colors.RED)diff --git a/pipelex/cli/commands/build/structures_cmd.py b/pipelex/cli/commands/build/structures_cmd.pyindex f5cc6c0df..3c0cf372e 100644--- a/pipelex/cli/commands/build/structures_cmd.py+++ b/pipelex/cli/commands/build/structures_cmd.py@@ -1,321 +1,30 @@-import inspect+"""CLI command `pipelex build structures`: alias of `pipelex codegen types --target python-structures`.++The legacy per-file generator (one always-qualified class file per concept) was deleted when the+codegen engine landed (D9): this command now delegates verbatim to the engine, which emits a single+stamped `structures.py` (bare-when-unique class names, declared imprecision, `codegen.lock` beside+it). See `docs/specs/pipelex-codegen.md` -> "CLI: codegen".+"""+from pathlib import Path-from typing import TYPE_CHECKING, Annotated+from typing import Annotatedimport typer-from pipelex import log-from pipelex.base_exceptions import PipelexError-from pipelex.cli.cli_factory import make_pipelex_for_cli-from pipelex.cli.error_handlers import ErrorContext-from pipelex.core.concepts.concept_factory import ConceptFactory-from pipelex.core.concepts.helpers import (- get_structure_class_name_from_blueprint,- make_qualified_structure_class_name,- normalize_structure_blueprint,-)-from pipelex.core.concepts.native.concept_native import NativeConceptCode-from pipelex.core.concepts.structure_generation.exceptions import ConceptStructureGeneratorError-from pipelex.core.concepts.structure_generation.generator import ConceptClassInfo, StructureGenerator-from pipelex.core.interpreter.helpers import is_pipelex_file-from pipelex.core.qualified_ref import QualifiedRef-from pipelex.core.registry_models import CoreRegistryModels-from pipelex.core.stuffs.text_content import TextContent-from pipelex.hub import get_class_registry, get_func_registry, resolve_library_dirs-from pipelex.pipeline.validate_bundle import load_concepts_only, load_concepts_only_from_directory-from pipelex.tools.misc.string_utils import pascal_case_to_snake_case--if TYPE_CHECKING:- from pipelex.core.bundles.pipelex_bundle_blueprint import PipelexBundleBlueprint+from pipelex.cli.commands.codegen.types_cmd import codegen_types_cmd+from pipelex.codegen.emitters.target import CodegenTargetSUB_COMMAND_STRUCTURES = "structures"-def _compute_relative_path_from_output_dir(output_directory: Path) -> Path | None:- """Compute the relative path from the output directory to the current working directory.-- Args:- output_directory: The directory where structure files will be generated-- Returns:- Relative path or None if cannot be determined- """- try:- cwd = Path.cwd()- return output_directory.resolve().relative_to(cwd)- except ValueError:- return None---def _build_concept_ref_to_class_info(- blueprints: list["PipelexBundleBlueprint"],- *,- output_directory: Path,-) -> dict[str, ConceptClassInfo]:- """Build a mapping from concept refs to their class info including module paths.-- Args:- blueprints: List of PipelexBundleBlueprint containing concept definitions- output_directory: Directory where structure files will be generated-- Returns:- Mapping from concept_ref to ConceptClassInfo with module paths- """- concept_ref_to_class_info: dict[str, ConceptClassInfo] = {}- base_relative_path = _compute_relative_path_from_output_dir(output_directory)-- for blueprint in blueprints:- if blueprint.domain == "native":- continue-- if not blueprint.concept:- continue-- for concept_code, concept_blueprint in blueprint.concept.items():- # Build the concept ref (domain.ConceptCode)- concept_ref = f"{blueprint.domain}.{concept_code}"-- # Get the class name from the blueprint. When the helper returns the bare- # concept_code (i.e. no user-supplied structure class reference), qualify- # it with the domain so cross-class imports/types match the qualified- # class definitions emitted below.- raw_class_name = get_structure_class_name_from_blueprint(concept_blueprint, concept_ref_or_code=concept_code)- if raw_class_name == concept_code:- class_name = make_qualified_structure_class_name(domain_code=blueprint.domain, concept_code=concept_code)- file_stem_snake_case = pascal_case_to_snake_case(concept_code)- else:- class_name = raw_class_name- file_stem_snake_case = pascal_case_to_snake_case(raw_class_name)-- # Build the module path for this concept's structure file- if base_relative_path:- base_module_path = ".".join(base_relative_path.parts)- module_path = f"{base_module_path}.{blueprint.domain}__{file_stem_snake_case}"- else:- module_path = None-- concept_ref_to_class_info[concept_ref] = ConceptClassInfo(- class_name=class_name,- module_path=module_path,- )-- return concept_ref_to_class_info---def generate_structures_from_blueprints(- blueprints: list["PipelexBundleBlueprint"],- *,- output_directory: Path,- target_path: Path | None = None,- skip_existing_check: bool = False,- quiet: bool = False,-) -> list[tuple[str, str]]:- """Generate Python structure files from blueprint concept definitions.-- Args:- blueprints: List of PipelexBundleBlueprint containing concept definitions- output_directory: Directory where structure files will be generated- target_path: Optional path to scan for manually-created structure classes- skip_existing_check: If True, always generate structures without checking if they exist- quiet: If True, suppress progress output (use log.verbose instead of typer.echo/secho)-- Returns:- List of (domain, concept_code) tuples for generated files- """- output_directory.mkdir(parents=True, exist_ok=True)-- # Build concept_ref_to_class_info mapping for all concepts- concept_ref_to_class_info = _build_concept_ref_to_class_info(blueprints, output_directory=output_directory)- class_registry = get_class_registry()-- # Only check for existing classes if we're not skipping and have a target path- check_existing = not skip_existing_check and target_path is not None-- generated_files: list[tuple[str, str]] = []-- if quiet:- log.verbose(f"Generating structures in: {output_directory}")- else:- typer.echo(f"\n📝 Generating structures in: {output_directory}")-- for blueprint in blueprints:- if blueprint.domain == "native":- continue-- if not blueprint.concept:- continue-- for concept_code, concept_blueprint in blueprint.concept.items():- # Check if structure class was manually created (only when check_existing is enabled)- if check_existing and class_registry.has_class(name=concept_code):- existing_class = class_registry.get_class(name=concept_code)- if existing_class:- try:- source_file = inspect.getfile(existing_class)- log.warning(- f"Skipping Generation for '{concept_code}' (domain '{blueprint.domain}'): "- f"manually-created class exists at '{source_file}'"- )- except (TypeError, OSError):- module_name = existing_class.__module__ if hasattr(existing_class, "__module__") else "unknown"- log.warning(- f"Skipping '{concept_code}' (domain '{blueprint.domain}'): manually-created class exists in module '{module_name}'"- )- continue-- # Handle simple string concept definitions (description only, refines Text by default)- if isinstance(concept_blueprint, str):- try:- generated_code, _ = StructureGenerator(- concept_ref_to_class_info=concept_ref_to_class_info, local_domain=blueprint.domain- ).generate_from_structure_blueprint(- class_name=make_qualified_structure_class_name(domain_code=blueprint.domain, concept_code=concept_code),- structure_blueprint={},- base_class_name=TextContent.__name__,- description=concept_blueprint,- )- except ConceptStructureGeneratorError as exc:- msg = f"Error generating structure class for concept '{concept_code}' in domain '{blueprint.domain}': {exc}"- raise PipelexError(msg) from exc-- concept_snake_case = pascal_case_to_snake_case(concept_code)- output_file = output_directory / f"{blueprint.domain}__{concept_snake_case}.py"- output_file.write_text(generated_code)- generated_files.append((blueprint.domain, concept_code))- if not quiet:- typer.secho(f" ✅ Generated {output_file.name}", fg=typer.colors.GREEN)- else:- log.verbose(f"Generated {output_file.name}")- continue-- # Handle concepts with explicit structure definition- if concept_blueprint.structure:- if isinstance(concept_blueprint.structure, str):- continue- normalized_structure = normalize_structure_blueprint(concept_blueprint.structure)-- try:- generated_code, the_generated_class = StructureGenerator(- concept_ref_to_class_info=concept_ref_to_class_info,- local_domain=blueprint.domain,- ).generate_from_structure_blueprint(- class_name=make_qualified_structure_class_name(domain_code=blueprint.domain, concept_code=concept_code),- structure_blueprint=normalized_structure,- description=concept_blueprint.description,- )- except ConceptStructureGeneratorError as exc:- msg = f"Error generating python code for structure class of concept '{concept_code}' in domain '{blueprint.domain}': {exc}"- raise PipelexError(msg) from exc-- # Register the generated class so it can be used as a base class for refined concepts- get_class_registry().register_class(the_generated_class)-- concept_snake_case = pascal_case_to_snake_case(concept_code)- output_file = output_directory / f"{blueprint.domain}__{concept_snake_case}.py"- output_file.write_text(generated_code)- generated_files.append((blueprint.domain, concept_code))- if not quiet:- typer.secho(f" ✅ Generated {output_file.name}", fg=typer.colors.GREEN)- else:- log.verbose(f"Generated {output_file.name}")-- # Handle concepts with refines- elif concept_blueprint.refines:- current_refine = ConceptFactory.make_refine(refine=concept_blueprint.refines, domain_code=blueprint.domain)-- # For native concepts, the structure class name is "ConceptCode" + "Content" (e.g., TextContent)- # For custom concepts, the structure class name is the domain-qualified name- # (e.g., other_domain__Customer) so it matches what ConceptFactory registers.- if current_refine:- refined_ref = QualifiedRef.parse_stripping_cross_package(current_refine)- refined_concept_code = refined_ref.local_code- if NativeConceptCode.is_native_concept_ref_or_code(concept_ref_or_code=current_refine):- refined_structure_class_name = refined_concept_code + "Content"- else:- refined_domain_code = refined_ref.domain_path or blueprint.domain- refined_structure_class_name = make_qualified_structure_class_name(- domain_code=refined_domain_code, concept_code=refined_concept_code- )- else:- refined_structure_class_name = TextContent.__name__-- try:- generated_code, the_generated_class = StructureGenerator(- concept_ref_to_class_info=concept_ref_to_class_info,- local_domain=blueprint.domain,- ).generate_from_structure_blueprint(- class_name=make_qualified_structure_class_name(domain_code=blueprint.domain, concept_code=concept_code),- structure_blueprint={},- base_class_name=refined_structure_class_name,- description=concept_blueprint.description,- )- except ConceptStructureGeneratorError as exc:- msg = (- f"Error generating python code for structure class of concept '{concept_code}' "- f"refining '{refined_structure_class_name}' in domain '{blueprint.domain}': {exc}"- )- raise PipelexError(msg) from exc-- # Register the generated class so it can be used as a base class for other refined concepts- get_class_registry().register_class(the_generated_class)-- concept_snake_case = pascal_case_to_snake_case(concept_code)- output_file = output_directory / f"{blueprint.domain}__{concept_snake_case}.py"- output_file.write_text(generated_code)- generated_files.append((blueprint.domain, concept_code))- if not quiet:- typer.secho(f" ✅ Generated {output_file.name}", fg=typer.colors.GREEN)- else:- log.verbose(f"Generated {output_file.name}")-- # Handle concepts with neither structure nor refines - defaults to TextContent- else:- try:- generated_code, the_generated_class = StructureGenerator(- concept_ref_to_class_info=concept_ref_to_class_info,- local_domain=blueprint.domain,- ).generate_from_structure_blueprint(- class_name=make_qualified_structure_class_name(domain_code=blueprint.domain, concept_code=concept_code),- structure_blueprint={},- base_class_name=TextContent.__name__,- description=concept_blueprint.description,- )- except ConceptStructureGeneratorError as exc:- msg = f"Error generating structure class for concept '{concept_code}' in domain '{blueprint.domain}': {exc}"- raise PipelexError(msg) from exc-- # Register the generated class so it can be used as a base class for refined concepts- get_class_registry().register_class(the_generated_class)-- concept_snake_case = pascal_case_to_snake_case(concept_code)- output_file = output_directory / f"{blueprint.domain}__{concept_snake_case}.py"- output_file.write_text(generated_code)- generated_files.append((blueprint.domain, concept_code))- if not quiet:- typer.secho(f" ✅ Generated {output_file.name}", fg=typer.colors.GREEN)- else:- log.verbose(f"Generated {output_file.name}")-- # Generate empty __init__.py to make structures importable- if generated_files:- init_file = output_directory / "__init__.py"- init_file.write_text("")- if not quiet:- typer.secho(" ✅ Generated __init__.py", fg=typer.colors.GREEN)- else:- log.verbose("Generated __init__.py")-- return generated_files--def build_structures_command(target: Annotated[str,- typer.Argument(help="Target directory to scan for .mthds files, or a specific .mthds file"),+ typer.Argument(help="Directory of .mthds bundles to resolve, or a .mthds file (its directory is used)"),],output_dir: Annotated[str | None,- typer.Option("--output-dir", "-o", help="Output directory for generated structures (default: structures/ in target's directory)"),+ typer.Option("--output-dir", "-o", help="Output directory for the generated module (default: structures/ in the target's directory)"),] = None,library_dir: Annotated[list[str] | None,@@ -325,98 +34,27 @@ def build_structures_command(help="Directory to search for pipe definitions (.mthds files). Can be specified multiple times.",),] = None,- force: Annotated[- bool,- typer.Option(- "--force",- "-f",- help="Force regeneration of all structures, overwriting existing files without checking if classes already exist.",- ),- ] = False,) -> None:"""Generate Python structure classes from concept definitions in .mthds files.+ Alias of `pipelex codegen types --target python-structures`: emits a single stamped+ `structures.py` plus its `codegen.lock`.+Examples:- pipelex build structures my_bundle.mthdspipelex build structures ./my_pipes/+ pipelex build structures my_bundle.mthdspipelex build structures my_bundle.mthds -o ./generated/pipelex build structures my_bundle.mthds -L ./shared_pipes/- pipelex build structures my_bundle.mthds --force"""-- def _build_structures_cmd():- target_path = Path(target).resolve()-- if not target_path.exists():- typer.secho(f"❌ Target does not exist: {target_path}", fg=typer.colors.RED, err=True)- raise typer.Exit(1)-- # Resolve library directories using the standard 3-tier priority- library_dirs_paths, _ = resolve_library_dirs(library_dir)-- # Determine if target is a file or directory- is_mthds_file = target_path.is_file() and is_pipelex_file(target_path)- pipelex_instance = make_pipelex_for_cli(context=ErrorContext.BUILD, library_dirs=library_dir, needs_inference=False)-- try:- if is_mthds_file:- # Single MTHDS file: output to parent directory- base_dir = target_path.parent- output_directory = Path(output_dir) if output_dir else base_dir / "structures"-- typer.echo(f"🔍 Loading concepts from bundle: {target_path}")-- # Load concepts only (no pipes)- load_result = load_concepts_only(mthds_file_path=target_path, library_dirs=library_dirs_paths)- # THIS IS A HACK, while waiting class/func registries to be in libraries.- get_class_registry().teardown()- get_func_registry().teardown()- get_class_registry().register_classes(CoreRegistryModels.get_all_models())-- all_blueprints: list[PipelexBundleBlueprint] = load_result.blueprints-- typer.echo(f"✅ Loaded {len(all_blueprints)} blueprint(s)")-- # Generate structures using the helper function- generated_files = generate_structures_from_blueprints(- blueprints=all_blueprints,- output_directory=output_directory,- target_path=base_dir,- skip_existing_check=force,- )- else:- # Directory: scan for all MTHDS files- if not target_path.is_dir():- typer.secho(f"❌ Target is not a directory or .mthds file: {target_path}", fg=typer.colors.RED, err=True)- raise typer.Exit(1)-- output_directory = Path(output_dir) if output_dir else target_path / "structures"-- typer.echo(f"🔍 Loading concepts from bundles in: {target_path}")-- # Load concepts only from directory (no pipes)- load_result = load_concepts_only_from_directory(directory=target_path)- # THIS IS A HACK, while waiting class/func registries to be in libraries.- get_class_registry().teardown()- get_func_registry().teardown()- get_class_registry().register_classes(CoreRegistryModels.get_all_models())-- typer.echo(f"✅ Loaded {len(load_result.blueprints)} blueprint(s)")-- # Generate structures using the helper function- generated_files = generate_structures_from_blueprints(- blueprints=load_result.blueprints,- output_directory=output_directory,- target_path=target_path,- skip_existing_check=force,- )-- if generated_files:- typer.secho(f"\n✨ Done! Generated {len(generated_files)} structure(s) in: {output_directory}", fg=typer.colors.GREEN)- else:- typer.secho("\n✨ Done! No structures to generate.", fg=typer.colors.GREEN)-- finally:- pipelex_instance.teardown()-- _build_structures_cmd()+ target_path = Path(target)+ if not target_path.exists():+ typer.secho(f"Cannot resolve — target does not exist: {target_path}", fg=typer.colors.RED, err=True)+ raise typer.Exit(2)+ closure_dir = target_path.parent if target_path.is_file() else target_path++ codegen_types_cmd(+ target=CodegenTarget.PYTHON_STRUCTURES,+ paths=[closure_dir],+ output_dir=output_dir or str(closure_dir / "structures"),+ library_dir=[Path(lib_dir) for lib_dir in library_dir] if library_dir else None,+ )diff --git a/pipelex/cli/commands/codegen/__init__.py b/pipelex/cli/commands/codegen/__init__.pynew file mode 100644index 000000000..e69de29bbdiff --git a/pipelex/cli/commands/codegen/app.py b/pipelex/cli/commands/codegen/app.pynew file mode 100644index 000000000..9158084a8--- /dev/null+++ b/pipelex/cli/commands/codegen/app.py@@ -0,0 +1,14 @@+import typer++from pipelex.cli.commands.codegen.check_cmd import codegen_check_cmd+from pipelex.cli.commands.codegen.inputs_cmd import codegen_inputs_cmd+from pipelex.cli.commands.codegen.types_cmd import codegen_types_cmd++codegen_app = typer.Typer(+ help="Project the normalized library crate into typed, runnable artifacts (two axes: kind + --target)",+ no_args_is_help=True,+)++codegen_app.command("types", help="Project the crate's concept set into typed artifacts (--target)")(codegen_types_cmd)+codegen_app.command("inputs", help="Project a runnable inputs template for a pipe (--pipe, defaults to main_pipe)")(codegen_inputs_cmd)+codegen_app.command("check", help="Verify generated artifacts are current — offline, no engine or network")(codegen_check_cmd)diff --git a/pipelex/cli/commands/codegen/check_cmd.py b/pipelex/cli/commands/codegen/check_cmd.pynew file mode 100644index 000000000..d73d0c73e--- /dev/null+++ b/pipelex/cli/commands/codegen/check_cmd.py@@ -0,0 +1,50 @@+"""CLI command `pipelex codegen check`: the offline drift check.++Pure hashing over `codegen.lock` and the files on disk — it does **not** boot Pipelex, load the crate,+touch the network, or need an API key (the whole point: CI checks offline, regeneration is a dev+action). The verdict rides the structured `CodegenCheckReport`; the exit code mirrors the bare+`validate`/`resolve` group — `0` current, `1` drift present (a negative verdict), `2` no lock found (no+verdict).++See `docs/specs/pipelex-codegen.md` → "Offline check algorithm".+"""++from pathlib import Path+from typing import Annotated++import typer++from pipelex.codegen.check import CodegenCheckReport, run_codegen_check+from pipelex.codegen.exceptions import CodegenLockError+from pipelex.codegen.lock import CODEGEN_LOCK_FILENAME+++def codegen_check_cmd(+ root: Annotated[+ str,+ typer.Argument(help=f"Directory holding the {CODEGEN_LOCK_FILENAME} and generated artifacts (default: current directory)."),+ ] = ".",+) -> None:+ """Verify generated artifacts are current, offline — no engine, no network, no API key."""+ root_path = Path(root).expanduser()+ try:+ report = run_codegen_check(root=root_path)+ except CodegenLockError as exc:+ typer.secho(exc.message, fg=typer.colors.RED, err=True)+ raise typer.Exit(2) from exc+ if not report.lock_found:+ typer.secho(f"No {CODEGEN_LOCK_FILENAME} found in '{root_path}' — nothing to check.", fg=typer.colors.YELLOW, err=True)+ raise typer.Exit(2)++ _print_report(report, root=root_path)+ if not report.is_current:+ raise typer.Exit(1)+++def _print_report(report: CodegenCheckReport, *, root: Path) -> None:+ if report.is_current:+ typer.secho(f"Generated artifacts in '{root}' are up to date.", fg=typer.colors.GREEN)+ return+ typer.secho(f"Generated artifacts in '{root}' have drifted:", fg=typer.colors.RED, err=True)+ for drift in report.drifts:+ typer.secho(f" [{drift.category}] {drift.path} — {drift.detail}", fg=typer.colors.RED, err=True)diff --git a/pipelex/cli/commands/codegen/inputs_cmd.py b/pipelex/cli/commands/codegen/inputs_cmd.pynew file mode 100644index 000000000..5beb3cd66--- /dev/null+++ b/pipelex/cli/commands/codegen/inputs_cmd.py@@ -0,0 +1,121 @@+"""CLI command `pipelex codegen inputs`: project a runnable inputs template for a selected pipe.++Re-surfaces the input renderer (`input_renderer`) over one pipe of the resolved closure: the Smart+Inputs light shape by default, `--explicit` for the ceremonial `{concept, content}` envelope. The pipe+is selected by qualified `--pipe` ref and defaults to the closure's declared `main_pipe`. The crate+loader leaves the library open and current, so the live pipe is rendered directly.++See `docs/specs/pipelex-codegen.md` -> "CLI: codegen".+"""++from pathlib import Path+from typing import Annotated++import typer+from posthog import tag++from pipelex.builder.conventions import DEFAULT_INPUTS_FILE_NAME, DEFAULT_INPUTS_TOML_FILE_NAME+from pipelex.cli.cli_factory import make_pipelex_for_cli+from pipelex.cli.commands.crate_loading import load_normalized_crate_or_exit+from pipelex.cli.error_handlers import ErrorContext+from pipelex.core.pipes.inputs.input_renderer import InputsTemplateFormat, NoInputsRequiredError, render_inputs, render_inputs_toml+from pipelex.core.pipes.pipe_abstract import PipeAbstract+from pipelex.hub import get_required_pipe, get_telemetry_manager+from pipelex.libraries.library_crate import LibraryCrate+from pipelex.libraries.pipe.exceptions import PipeLibraryError+from pipelex.pipelex import Pipelex+from pipelex.system.runtime import IntegrationMode+from pipelex.system.telemetry.events import EventProperty+from pipelex.tools.misc.file_utils import ensure_directory_for_file_path, save_text_to_path+from pipelex.tools.misc.package_utils import get_package_version++COMMAND = "codegen"+SUB_COMMAND = "inputs"+++def codegen_inputs_cmd(+ pipe: Annotated[+ str | None,+ typer.Option("--pipe", "-p", help="Qualified pipe ref (domain.pipe_code). Defaults to the closure's main_pipe."),+ ] = None,+ paths: Annotated[+ list[Path] | None,+ typer.Argument(help="Directories of .mthds bundles to resolve into the closure (added to --library-dir)."),+ ] = None,+ template_format: Annotated[+ InputsTemplateFormat,+ typer.Option("--format", "-f", help="Inputs template encoding (json or toml)."),+ ] = InputsTemplateFormat.JSON,+ explicit: Annotated[+ bool,+ typer.Option("--explicit", help="Emit the ceremonial {concept, content} envelope instead of the light values."),+ ] = False,+ output: Annotated[+ str | None,+ typer.Option("--output", "-o", help="Path to write the inputs template (defaults to results/)."),+ ] = None,+ library_dir: Annotated[+ list[Path] | None,+ typer.Option("--library-dir", "-L", help="Directory of .mthds bundles to load (repeatable)."),+ ] = None,+) -> None:+ """Project a runnable inputs template for the selected (or main) pipe of the closure."""+ # needs_model_specs=True (like `validate`): library validation checks pipe model pins+ # against the deck, so the specs must be loaded even though codegen needs no inference.+ make_pipelex_for_cli(context=ErrorContext.VALIDATION, needs_inference=False, needs_model_specs=True)++ try:+ with get_telemetry_manager().telemetry_context():+ tag(name=EventProperty.INTEGRATION, value=IntegrationMode.CLI)+ tag(name=EventProperty.PIPELEX_VERSION, value=get_package_version())+ tag(name=EventProperty.CLI_COMMAND, value=f"{COMMAND} {SUB_COMMAND}")++ combined_dirs: list[Path] = [*(paths or []), *(library_dir or [])]+ crate = load_normalized_crate_or_exit(library_dirs=combined_dirs or None)++ pipe_ref = pipe or _default_main_pipe_ref(crate)+ try:+ the_pipe = get_required_pipe(pipe_code=pipe_ref)+ except PipeLibraryError as exc:+ typer.secho(f"Cannot project inputs — pipe '{pipe_ref}' not found:\n{exc}", fg=typer.colors.RED, err=True)+ raise typer.Exit(1) from exc++ try:+ content, default_file_name = _render(the_pipe=the_pipe, template_format=template_format, explicit=explicit)+ except NoInputsRequiredError as exc:+ typer.secho(str(exc), fg=typer.colors.YELLOW)+ raise typer.Exit(0) from exc++ destination = Path(output).expanduser() if output else Path("results") / default_file_name+ # Write-if-changed, like `codegen types`: no mtime churn and a truthful console verdict+ # when the committed template is already current.+ if destination.is_file() and destination.read_text(encoding="utf-8") == content:+ typer.secho(f"Unchanged {destination}", fg=typer.colors.BLUE)+ else:+ ensure_directory_for_file_path(file_path=destination)+ save_text_to_path(text=content, path=destination)+ typer.secho(f"Generated {destination}", fg=typer.colors.GREEN)+ finally:+ Pipelex.teardown_if_needed()+++def _render(*, the_pipe: PipeAbstract, template_format: InputsTemplateFormat, explicit: bool) -> tuple[str, str]:+ """Render the inputs template string and return it with the format's default file name."""+ match template_format:+ case InputsTemplateFormat.JSON:+ return render_inputs(the_pipe, explicit=explicit), DEFAULT_INPUTS_FILE_NAME+ case InputsTemplateFormat.TOML:+ return render_inputs_toml(the_pipe, explicit=explicit), DEFAULT_INPUTS_TOML_FILE_NAME+++def _default_main_pipe_ref(crate: LibraryCrate) -> str:+ """The closure's single declared `main_pipe` (qualified), or exit 1 if none / ambiguous."""+ candidates = [f"{domain_code}.{domain.main_pipe}" for domain_code, domain in crate.domains.items() if domain.main_pipe]+ if not candidates:+ typer.secho("Cannot project inputs — no main_pipe is declared in the closure; pass --pipe.", fg=typer.colors.RED, err=True)+ raise typer.Exit(1)+ if len(candidates) > 1:+ joined = ", ".join(sorted(candidates))+ typer.secho(f"Cannot project inputs — the closure declares several main_pipes ({joined}); pass --pipe.", fg=typer.colors.RED, err=True)+ raise typer.Exit(1)+ return candidates[0]diff --git a/pipelex/cli/commands/codegen/types_cmd.py b/pipelex/cli/commands/codegen/types_cmd.pynew file mode 100644index 000000000..944541cdf--- /dev/null+++ b/pipelex/cli/commands/codegen/types_cmd.py@@ -0,0 +1,83 @@+"""CLI command `pipelex codegen types`: project the crate's concept set into typed artifacts.++Thin wiring over the `emit_types` engine (`pipelex/codegen/emitters/types_emitter.py`): load and+normalize the closure into a crate, project its concept set for the chosen `--target`, and write each+emitted file under the output root. The verdict (resolved / invalid library) rides the resolve exit+codes via `load_normalized_crate_or_exit`; the success stream is the list of written files.++See `docs/specs/pipelex-codegen.md` -> "CLI: codegen" and "Two axes".+"""++from pathlib import Path+from typing import Annotated++import typer+from posthog import tag++from pipelex.cli.cli_factory import make_pipelex_for_cli+from pipelex.cli.commands.crate_loading import load_normalized_crate_or_exit+from pipelex.cli.error_handlers import ErrorContext+from pipelex.codegen.emission import write_stamped_projection+from pipelex.codegen.emitters.target import CodegenKind, CodegenTarget+from pipelex.codegen.emitters.types_emitter import emit_types+from pipelex.hub import get_telemetry_manager+from pipelex.pipelex import Pipelex+from pipelex.system.runtime import IntegrationMode+from pipelex.system.telemetry.events import EventProperty+from pipelex.tools.misc.package_utils import get_package_version++COMMAND = "codegen"+SUB_COMMAND = "types"+++def codegen_types_cmd(+ target: Annotated[+ CodegenTarget,+ typer.Option("--target", "-t", help="Codegen target flavor (ts-zod, python-pydantic, python-structures)."),+ ],+ paths: Annotated[+ list[Path] | None,+ typer.Argument(help="Directories of .mthds bundles to resolve into the closure (added to --library-dir)."),+ ] = None,+ output_dir: Annotated[+ str,+ typer.Option("--output", "-o", help="Directory the generated files are written into (default: current directory)."),+ ] = ".",+ library_dir: Annotated[+ list[Path] | None,+ typer.Option("--library-dir", "-L", help="Directory of .mthds bundles to load (repeatable)."),+ ] = None,+) -> None:+ """Project the crate's concept set into typed artifacts for the chosen target."""+ # needs_model_specs=True (like `validate`): library validation checks pipe model pins+ # against the deck, so the specs must be loaded even though codegen needs no inference.+ make_pipelex_for_cli(context=ErrorContext.VALIDATION, needs_inference=False, needs_model_specs=True)+ output_root = Path(output_dir).expanduser()++ try:+ with get_telemetry_manager().telemetry_context():+ tag(name=EventProperty.INTEGRATION, value=IntegrationMode.CLI)+ tag(name=EventProperty.PIPELEX_VERSION, value=get_package_version())+ tag(name=EventProperty.CLI_COMMAND, value=f"{COMMAND} {SUB_COMMAND}")++ combined_dirs: list[Path] = [*(paths or []), *(library_dir or [])]+ crate = load_normalized_crate_or_exit(library_dirs=combined_dirs or None)++ emitted = emit_types(crate, target=target)+ report = write_stamped_projection(+ emitted,+ output_dir=output_root,+ crate_fingerprint=crate.fingerprint,+ engine_version=get_package_version(),+ kind=CodegenKind.TYPES,+ target=target,+ )+ for filename in report.written:+ typer.secho(f"Generated {output_root / filename}", fg=typer.colors.GREEN)+ for filename in report.unchanged:+ typer.secho(f"Unchanged {output_root / filename}", fg=typer.colors.BLUE)+ for filename in report.removed:+ typer.secho(f"Removed stale {output_root / filename}", fg=typer.colors.YELLOW)+ typer.secho(f"Locked {output_root / report.lock_path}", fg=typer.colors.GREEN)+ finally:+ Pipelex.teardown_if_needed()diff --git a/pipelex/cli/commands/crate_loading.py b/pipelex/cli/commands/crate_loading.pynew file mode 100644index 000000000..1b62ace3a--- /dev/null+++ b/pipelex/cli/commands/crate_loading.py@@ -0,0 +1,62 @@+"""Shared CLI helper: assemble a bundle closure and emit the normalized library crate.++Both `pipelex resolve` and the `pipelex codegen` family need the same thing — load the closure,+require it to be **valid**, and normalize it into the flat, self-contained crate the projections read.+Centralizing it here keeps the resolve/validate verdict contract in one place: a structurally invalid+library is a negative verdict, a closure that could not be assembled is no verdict, and an internal+invariant failure (`CrateNormalizationError`) is deliberately left to surface rather than masked as a+verdict. `load_normalized_crate` is the presentation-free core (typed exceptions); the bare-CLI+wrapper `load_normalized_crate_or_exit` maps them to the 1/2 exit codes, and the agent CLI maps the+same exceptions to its structured error envelope.++See `docs/specs/pipelex-codegen.md` → "CLI: resolve" and the standard's `library-crate.md`.+"""++from pathlib import Path++import typer+from mthds.package.manifest.schema import MTHDS_STANDARD_VERSION++from pipelex.hub import get_library_manager+from pipelex.libraries.crate_normalization import normalize_crate+from pipelex.libraries.exceptions import LibraryLoadingError+from pipelex.libraries.library_crate import LibraryCrate+from pipelex.libraries.pipe.exceptions import PipeLibraryError+from pipelex.pipeline.execution_seams import load_libraries_and_activate+++def load_normalized_crate(*, library_dirs: list[Path] | None) -> LibraryCrate:+ """Load, validate, and normalize the crate for a bundle closure.++ Raises:+ LibraryLoadingError: The library is structurally invalid (a negative verdict).+ PipeLibraryError: A pipe conflict across bundles (a negative verdict).+ FileNotFoundError: The closure could not be assembled, or it holds no .mthds bundles (no verdict).+ """+ expanded_library_dirs = [library_dir.expanduser() for library_dir in library_dirs] if library_dirs is not None else None+ library_id = load_libraries_and_activate(expanded_library_dirs)+ crate = get_library_manager().get_crate(library_id)+ if crate is None:+ msg = "no .mthds bundles found in the closure."+ raise FileNotFoundError(msg)+ # CrateNormalizationError (an internal invariant) is deliberately not caught — it must surface.+ return normalize_crate(crate, mthds_version=MTHDS_STANDARD_VERSION)+++def load_normalized_crate_or_exit(*, library_dirs: list[Path] | None) -> LibraryCrate:+ """Load, validate, and normalize the crate for a bundle closure, or raise `typer.Exit` per policy.++ Exit codes mirror the bare `validate` group: `1` when the library is structurally invalid (a+ negative verdict — failed to load, or a pipe conflict across bundles), `2` when the closure could+ not be assembled at all (no verdict — nothing found / load failure).+ """+ try:+ return load_normalized_crate(library_dirs=library_dirs)+ except (LibraryLoadingError, PipeLibraryError) as exc:+ # Negative verdict: the library is structurally invalid, so no crate can be produced.+ typer.secho(f"Cannot resolve — the library is invalid:\n{exc}", fg=typer.colors.RED, err=True)+ raise typer.Exit(1) from exc+ except FileNotFoundError as exc:+ # No verdict: the closure could not even be assembled.+ typer.secho(f"Cannot resolve — {exc}", fg=typer.colors.RED, err=True)+ raise typer.Exit(2) from excdiff --git a/pipelex/cli/commands/resolve_cmd.py b/pipelex/cli/commands/resolve_cmd.pynew file mode 100644index 000000000..7b3f61ee7--- /dev/null+++ b/pipelex/cli/commands/resolve_cmd.py@@ -0,0 +1,60 @@+"""CLI command `pipelex resolve`: emit the normalized library crate for a bundle closure.++Resolution is a first-class language operation alongside validation: it assembles the library closure,+requires it to be **valid** (a resolution over an invalid library produces a verdict, not a crate), and+emits the normalized crate to stdout in the selected encoding. The success stream is the crate; the+verdict rides the exit code, mirroring the bare `validate` group — `0` resolved, `1` produced a negative+verdict (the library is invalid), `2` no verdict could be produced (nothing to resolve / load failure).++See `docs/specs/pipelex-codegen.md` → "CLI: resolve" and the standard's `library-crate.md`.+"""++from pathlib import Path+from typing import Annotated++import typer+from posthog import tag++from pipelex.cli.cli_factory import make_pipelex_for_cli+from pipelex.cli.commands.crate_loading import load_normalized_crate_or_exit+from pipelex.cli.error_handlers import ErrorContext+from pipelex.codegen.crate_encoding import CrateEncoding, encode_crate+from pipelex.hub import get_telemetry_manager+from pipelex.pipelex import Pipelex+from pipelex.system.runtime import IntegrationMode+from pipelex.system.telemetry.events import EventProperty+from pipelex.tools.misc.package_utils import get_package_version++COMMAND = "resolve"+++def resolve_cmd(+ paths: Annotated[+ list[Path] | None,+ typer.Argument(help="Directories of .mthds bundles to resolve into the closure (added to --library-dir)."),+ ] = None,+ output_format: Annotated[+ CrateEncoding,+ typer.Option("--format", "-f", help="Crate encoding. Both carry the same fingerprint."),+ ] = CrateEncoding.JSON,+ library_dir: Annotated[+ list[Path] | None,+ typer.Option("--library-dir", "-L", help="Directory of .mthds bundles to load (repeatable)."),+ ] = None,+) -> None:+ """Emit the normalized library crate for a bundle closure to stdout."""+ # needs_model_specs=True (like `validate`): library validation checks pipe model pins+ # against the deck, so the specs must be loaded even though resolving needs no inference.+ make_pipelex_for_cli(context=ErrorContext.VALIDATION, needs_inference=False, needs_model_specs=True)++ try:+ with get_telemetry_manager().telemetry_context():+ tag(name=EventProperty.INTEGRATION, value=IntegrationMode.CLI)+ tag(name=EventProperty.PIPELEX_VERSION, value=get_package_version())+ tag(name=EventProperty.CLI_COMMAND, value=COMMAND)++ combined_dirs: list[Path] = [*(paths or []), *(library_dir or [])]+ normalized = load_normalized_crate_or_exit(library_dirs=combined_dirs or None)+ typer.echo(encode_crate(normalized, encoding=output_format), nl=False)+ finally:+ Pipelex.teardown_if_needed()diff --git a/pipelex/cli/commands/run/_inputs_file_loader.py b/pipelex/cli/commands/run/_inputs_file_loader.pyindex b4e4edf5c..eb39cf900 100644--- a/pipelex/cli/commands/run/_inputs_file_loader.py+++ b/pipelex/cli/commands/run/_inputs_file_loader.py@@ -12,11 +12,12 @@ from __future__ import annotationsimport datetimefrom pathlib import Path-from typing import Any, cast+from typing import Anyfrom pipelex.builder.conventions import DEFAULT_INPUTS_FILE_NAME, DEFAULT_INPUTS_TOML_FILE_NAME-from pipelex.cli.commands.run.exceptions import AmbiguousInputsFilesError, InputsTimeOnlyNotSupportedError+from pipelex.cli.commands.run.exceptions import AmbiguousInputsFilesErrorfrom pipelex.core.stuffs.date_content import DateContent+from pipelex.core.stuffs.time_content import TimeContentfrom pipelex.tools.misc.json_utils import load_json_dict_from_pathfrom pipelex.tools.misc.toml_utils import load_toml_from_path@@ -32,24 +33,23 @@ def load_inputs_dict_from_path(path: Path) -> dict[str, Any]:Returns:The loaded inputs dictionary. Any top-level TOML date/datetime literal is- converted to a ``DateContent`` (which enters the pipeline-inputs seam as a- bare content instance); nested date/datetime values are left in place for- the envelope/structure factory arms to consume.+ converted to a ``DateContent`` and any top-level TOML time-of-day literal to+ a ``TimeContent`` (each enters the pipeline-inputs seam as a bare content+ instance); nested temporal values are left in place for the+ envelope/structure factory arms to consume.Raises:FileNotFoundError: If the file does not exist.TomlError: If a ``.toml`` file has invalid TOML syntax.json.JSONDecodeError: If a JSON file has invalid JSON syntax.JsonTypeError: If a JSON file does not hold a dictionary.- InputsTimeOnlyNotSupportedError: If a loaded value is a bare TOML- time-of-day (which has no date to attach to and no native concept)."""inputs_dict: dict[str, Any]if path.suffix.lower() == TOML_SUFFIX:inputs_dict = load_toml_from_path(path)else:inputs_dict = load_json_dict_from_path(path)- return _convert_temporal_inputs(inputs_dict, source_path=path)+ return _convert_temporal_inputs(inputs_dict)def resolve_inputs_arg_against_dir(inputs_arg: str | None, *, base_dir: Path) -> str | None:@@ -111,8 +111,8 @@ def find_default_inputs_file(directory: Path) -> Path | None:return None-def _convert_temporal_inputs(inputs_dict: dict[str, Any], *, source_path: Path) -> dict[str, Any]:- """Map TOML temporal literals to native ``Date`` inputs, rejecting bare times.+def _convert_temporal_inputs(inputs_dict: dict[str, Any]) -> dict[str, Any]:+ """Map top-level TOML temporal literals to native ``Date`` / ``Time`` inputs.Only TOML can produce temporal Python objects (JSON has no temporal type), but thewalk is format-agnostic and harmless on JSON (which yields none).@@ -120,59 +120,26 @@ def _convert_temporal_inputs(inputs_dict: dict[str, Any], *, source_path: Path)- A **top-level** ``datetime.datetime`` / ``datetime.date`` becomes a ``DateContent``(a datetime keeps its time and offset; a bare date has ``time=None``). It then entersthe pipeline-inputs seam as a bare content instance, inferred as ``native.Date``.- - A ``datetime.time`` **anywhere** in the tree is rejected: a time of day alone has no- date to attach to and no native concept.- - A **nested** date/datetime (inside an envelope's ``content`` or a structured dict) is+ - A **top-level** ``datetime.time`` becomes a ``TimeContent`` (offset kept when stated),+ inferred as ``native.Time``. A time never silently becomes a ``Date`` — shaping a+ ``Time`` into a ``Date`` slot fails the concept-compatibility check downstream.+ - A **nested** temporal value (inside an envelope's ``content`` or a structured dict) isleft in place for the factory arms and pydantic validation to consume.Args:inputs_dict: The loaded top-level inputs dictionary.- source_path: The inputs file the values were loaded from, for the error message.Returns:A new dict with top-level temporal literals converted.-- Raises:- InputsTimeOnlyNotSupportedError: On the first bare time-of-day value found."""converted: dict[str, Any] = {}for key, value in inputs_dict.items():- _reject_time_of_day(value, source_path=source_path, key_path=str(key))if isinstance(value, datetime.datetime):converted[key] = DateContent(date=value.date(), time=value.timetz())elif isinstance(value, datetime.date):converted[key] = DateContent(date=value)+ elif isinstance(value, datetime.time):+ converted[key] = TimeContent(time=value)else:converted[key] = valuereturn converted---def _reject_time_of_day(value: Any, *, source_path: Path, key_path: str) -> None:- """Recursively reject bare ``datetime.time`` values; leave date/datetime alone.-- Args:- value: The loaded value to inspect (dict, list, scalar, ...).- source_path: The inputs file the value was loaded from, for the error message.- key_path: Dotted/indexed path to ``value`` inside the inputs dict.-- Raises:- InputsTimeOnlyNotSupportedError: On the first ``datetime.time`` value found.- """- # datetime is a subclass of date and neither is a time, so this only fires on a bare time-of-day.- if isinstance(value, datetime.time):- located = f" at '{key_path}'" if key_path else ""- msg = (- f"Inputs file '{source_path}' holds a bare TOML time-of-day{located} ({value!r}). "- "A time of day alone has no date to attach to: include the date (e.g. 2026-07-06T12:00:00) "- "or quote the value as a string."- )- raise InputsTimeOnlyNotSupportedError(msg)- if isinstance(value, dict):- value_dict = cast("dict[str, Any]", value)- for key, sub_value in value_dict.items():- sub_key_path = f"{key_path}.{key}" if key_path else str(key)- _reject_time_of_day(sub_value, source_path=source_path, key_path=sub_key_path)- elif isinstance(value, list):- value_list = cast("list[Any]", value)- for index_item, sub_value in enumerate(value_list):- _reject_time_of_day(sub_value, source_path=source_path, key_path=f"{key_path}[{index_item}]")diff --git a/pipelex/cli/commands/run/_run_core.py b/pipelex/cli/commands/run/_run_core.pyindex cad9e14d5..d19fcd70c 100644--- a/pipelex/cli/commands/run/_run_core.py+++ b/pipelex/cli/commands/run/_run_core.py@@ -13,7 +13,6 @@ from pipelex.base_exceptions import PipelexErrorfrom pipelex.cli.cli_factory import make_pipelex_for_clifrom pipelex.cli.commands.run._inputs_file_loader import load_inputs_dict_from_pathfrom pipelex.cli.commands.run._inputs_path_resolver import resolve_inputs_paths-from pipelex.cli.commands.run.exceptions import InputsTimeOnlyNotSupportedErrorfrom pipelex.cli.error_handlers import (ErrorContext,handle_model_availability_error,@@ -182,10 +181,6 @@ async def _execute_run(print_traceback_if_requested(get_console())typer.secho(f"Failed to parse input file: {toml_error_exc.message}", fg=typer.colors.RED, err=True)raise typer.Exit(1) from toml_error_exc- except InputsTimeOnlyNotSupportedError as time_only_exc:- print_traceback_if_requested(get_console())- typer.secho(f"Failed to load input file: {time_only_exc.message}", fg=typer.colors.RED, err=True)- raise typer.Exit(1) from time_only_exc# Determine pipe run modepipe_run_mode = PipeRunMode.DRY if dry_run else Nonediff --git a/pipelex/cli/commands/run/exceptions.py b/pipelex/cli/commands/run/exceptions.pyindex 86ad92a7c..e9552226e 100644--- a/pipelex/cli/commands/run/exceptions.py+++ b/pipelex/cli/commands/run/exceptions.py@@ -4,25 +4,6 @@ from pipelex.base_exceptions import ErrorDomain, PipelexErrorfrom pipelex.cogt.inference.error_classification import UserAction, UserActionKind-class InputsTimeOnlyNotSupportedError(PipelexError):- """Raised when a loaded inputs file carries a bare TOML time-of-day value.-- TOML date and datetime literals now map to the native ``Date`` concept, but a- time of day alone has no date to attach to and no native concept, so the inputs- loader rejects it. (Date/datetime values are accepted; only bare times are not.)- """-- error_domain = ErrorDomain.INPUT- user_action = UserAction(- kind=UserActionKind.CHANGE_INPUT,- # The example is an unquoted TOML datetime literal (which maps to Date); quoting it would make it- # a string (Text), i.e. the "or quote as a string" fallback — the opposite of "include the date".- detail="A time of day alone has no date to attach to. Include the date (e.g. 2026-07-06T12:00:00), or quote the value as a string",- )- # The message describes a fault in the caller's own inputs file — caller-facing copy.- _authors_caller_facing_message = True--class AmbiguousInputsFilesError(PipelexError):"""Raised when a bundle directory holds both default inputs files on auto-detect.diff --git a/pipelex/codegen/__init__.py b/pipelex/codegen/__init__.pynew file mode 100644index 000000000..e69de29bbdiff --git a/pipelex/codegen/check.py b/pipelex/codegen/check.pynew file mode 100644index 000000000..7c4107f6f--- /dev/null+++ b/pipelex/codegen/check.py@@ -0,0 +1,163 @@+"""The offline drift check: pure hashing, no engine, no network, no API key.++`codegen check` verifies that generated artifacts still match what the method resolves to — using only+`codegen.lock`, the files on disk, and a hash function. Because it never touches the engine, any client+(this CLI, an SDK, a short CI script) implements it identically, and template improvements never redden+a consumer's CI: regeneration is a dev action, the check is the CI action.++The algorithm (see `docs/specs/pipelex-codegen.md` → "Offline check algorithm"):++1. For each artifact in the lock, locate the file and recompute its body content hash; a mismatch or a+ missing file is a drift.+2. For each generated file present, parse its stamp and recompute the hash below it; a self-inconsistent+ stamp (or a stripped stamp on a locked file) is a hand-edit drift.+3. Detect locked files with no counterpart on disk (deleted) and stamped files not in the lock (orphan).++The verdict rides the structured `CodegenCheckReport`, never the exit code alone — the drifting+artifacts are enumerated by category.+"""++from collections.abc import Iterator+from enum import StrEnum+from pathlib import Path++from pydantic import BaseModel, ConfigDict, Field++from pipelex.codegen.exceptions import CodegenError, CodegenLockError+from pipelex.codegen.lock import CODEGEN_LOCK_FILENAME, CodegenLock, load_lock, resolve_artifact_path, resolve_output_path+from pipelex.codegen.stamp import STAMPABLE_SUFFIXES, comment_prefix_for, compute_content_hash, has_stamp, parse_stamped+from pipelex.tools.misc.file_utils import load_text_from_path+from pipelex.tools.typing.pydantic_utils import empty_list_factory_of++# Vendor / VCS / cache directories that never hold generated output — pruned from the orphan scan so a+# `codegen check` at a project root does not read (and choke on) unrelated files under them.+_SKIP_DIRS = frozenset(+ {".git", ".hg", ".svn", ".venv", "venv", "node_modules", "__pycache__", "dist", "build", ".next", ".mypy_cache", ".pytest_cache", ".ruff_cache"}+)+++class DriftCategory(StrEnum):+ """The kind of drift found for one artifact."""++ MISSING = "missing"+ """Listed in the lock, absent on disk (a deleted-artifact drift)."""++ MODIFIED = "modified"+ """Present, but its body hash no longer matches the locked hash."""++ HAND_EDITED = "hand-edited"+ """Present, but its stamp is missing or self-inconsistent (edited below the stamp)."""++ ORPHAN = "orphan"+ """A stamped generated file on disk that the lock does not track (a stale lingering artifact)."""+++class CodegenDrift(BaseModel):+ """One drifting artifact: its path (relative to the lock), the drift category, and a human detail."""++ model_config = ConfigDict(frozen=True)++ path: str+ category: DriftCategory+ detail: str+++class CodegenCheckReport(BaseModel):+ """The structured verdict of an offline check over one output root."""++ model_config = ConfigDict(frozen=True)++ lock_found: bool+ drifts: list[CodegenDrift] = Field(default_factory=empty_list_factory_of(CodegenDrift))++ @property+ def is_current(self) -> bool:+ """Whether generated artifacts are in sync (a lock was found and no drift was detected)."""+ return self.lock_found and not self.drifts+++def run_codegen_check(*, root: Path) -> CodegenCheckReport:+ """Run the offline drift check over `root` (the directory holding `codegen.lock`)."""+ try:+ lock_path = resolve_output_path(root, relative_path=Path(CODEGEN_LOCK_FILENAME))+ safe_root = lock_path.parent+ lock = load_lock(lock_path)+ if lock is None:+ return CodegenCheckReport(lock_found=False)++ drifts: list[CodegenDrift] = []+ drifts += _check_locked_artifacts(root=safe_root, lock=lock)+ drifts += _find_orphans(root=safe_root, lock=lock)+ return CodegenCheckReport(lock_found=True, drifts=drifts)+ except CodegenLockError:+ raise+ except CodegenError as exc:+ msg = f"Unsafe codegen artifact tree at '{root}': {exc}"+ raise CodegenLockError(msg) from exc+++def _check_locked_artifacts(*, root: Path, lock: CodegenLock) -> list[CodegenDrift]:+ drifts: list[CodegenDrift] = []+ for path, locked_hash in sorted(lock.hash_by_path().items()):+ file_path = resolve_artifact_path(root, artifact_path=path)+ if not file_path.is_file():+ drifts.append(CodegenDrift(path=path, category=DriftCategory.MISSING, detail="Locked artifact is absent on disk."))+ continue+ drift = _check_present_artifact(path=path, file_path=file_path, locked_hash=locked_hash)+ if drift is not None:+ drifts.append(drift)+ return drifts+++def _check_present_artifact(*, path: str, file_path: Path, locked_hash: str) -> CodegenDrift | None:+ text = _read_text_or_none(file_path)+ if text is None:+ return CodegenDrift(path=path, category=DriftCategory.HAND_EDITED, detail="File is not valid UTF-8 — not generated output.")+ parsed = parse_stamped(text, comment_prefix=comment_prefix_for(path))+ if parsed is None:+ return CodegenDrift(path=path, category=DriftCategory.HAND_EDITED, detail="Stamp header is missing or unparseable.")+ body_hash = compute_content_hash(parsed.body)+ if parsed.stamp.content_hash != body_hash:+ return CodegenDrift(path=path, category=DriftCategory.HAND_EDITED, detail="Body was edited below the stamp (stamp hash no longer matches).")+ if body_hash != locked_hash:+ return CodegenDrift(path=path, category=DriftCategory.MODIFIED, detail="Body no longer matches the locked hash — regenerate.")+ return None+++def _find_orphans(*, root: Path, lock: CodegenLock) -> list[CodegenDrift]:+ tracked = lock.paths()+ orphans: list[CodegenDrift] = []+ for file_path in _iter_stampable_files(root):+ relative = file_path.relative_to(root).as_posix()+ if relative in tracked:+ continue+ text = _read_text_or_none(file_path)+ if text is not None and has_stamp(text, comment_prefix=comment_prefix_for(relative)):+ orphans.append(+ CodegenDrift(+ path=relative,+ category=DriftCategory.ORPHAN,+ detail="Stamped generated file not tracked by the lock — stale; remove or regenerate.",+ )+ )+ return orphans+++def _iter_stampable_files(directory: Path) -> Iterator[Path]:+ """Yield stampable files under `directory`, pre-order and deterministic, pruning vendor/VCS dirs."""+ for entry in sorted(directory.iterdir()):+ if entry.is_symlink():+ continue+ if entry.is_dir():+ if entry.name not in _SKIP_DIRS:+ yield from _iter_stampable_files(entry)+ elif entry.suffix in STAMPABLE_SUFFIXES:+ yield entry+++def _read_text_or_none(path: Path) -> str | None:+ """Read UTF-8 text, or `None` if the file is not valid UTF-8 (so it cannot be our generated output)."""+ try:+ return load_text_from_path(path)+ except UnicodeDecodeError:+ return Nonediff --git a/pipelex/codegen/crate_encoding.py b/pipelex/codegen/crate_encoding.pynew file mode 100644index 000000000..a4b952db7--- /dev/null+++ b/pipelex/codegen/crate_encoding.py@@ -0,0 +1,79 @@+"""Canonical JSON and TOML encodings of a normalized library crate.++Both encodings serialize the SAME logical crate (per the standard's [Library Crate Format]) and carry+the same `fingerprint` — the digest is a property of the logical crate, not of a particular encoding's+bytes. Emission is deterministic: the top-level maps (`concepts`, `pipes`, `domains`, `source_map`) are+sorted by qualified key so version-control diffs stay minimal and independent producers agree, while+nested object key order is preserved because structure-field order is semantic (a downstream emitter+declares fields in that order).++Provenance-only inline `source` fields and unset (`None`) members are dropped: they carry no meaning,+TOML has no null, and the fingerprint already excludes them. The top-level `source_map` remains the+single provenance trace.++[Library Crate Format]: mthds/docs/spec/library-crate.md+"""+# tomlkit is not fully typed (`tomlkit.dumps`), so its member access reads as unknown here.+# pyright: reportUnknownMemberType=false++import json+from enum import StrEnum+from typing import Any++import tomlkit+from pydantic import BaseModel++from pipelex.libraries.library_crate import LibraryCrate+++class CrateEncoding(StrEnum):+ """The canonical wire encodings a normalized crate can be emitted in."""++ JSON = "json"+ TOML = "toml"+++def _dump_object(obj: BaseModel) -> dict[str, Any]:+ """Dump a crate object for emission.++ Prunes `None` members, the provenance-only inline `source`, and the internal `pipe_category`+ discriminator (the union routes on `type`; `pipe_category` is a non-user-facing technical field+ the base marks `exclude=True`, and every subclass defaults it, so dropping it round-trips).+ """+ return obj.model_dump(mode="json", exclude_none=True, exclude={"source", "pipe_category"})+++def _encoding_payload(crate: LibraryCrate) -> dict[str, Any]:+ """Build the ordered, key-sorted payload shared by both encodings."""+ concepts: dict[str, Any] = {}+ for concept_ref, value in sorted(crate.concepts.items()):+ concepts[concept_ref] = value if isinstance(value, str) else _dump_object(value)+ pipes = {pipe_ref: _dump_object(blueprint) for pipe_ref, blueprint in sorted(crate.pipes.items())}+ domains = {domain_code: _dump_object(domain) for domain_code, domain in sorted(crate.domains.items())}+ return {+ "mthds_version": crate.mthds_version,+ "concepts": concepts,+ "pipes": pipes,+ "domains": domains,+ "source_map": dict(sorted(crate.source_map.items())),+ "fingerprint": crate.fingerprint,+ }+++def encode_crate_json(crate: LibraryCrate) -> str:+ """Encode a normalized crate as canonical JSON (fixed member order, top-level maps key-sorted)."""+ return json.dumps(_encoding_payload(crate), indent=2, ensure_ascii=False) + "\n"+++def encode_crate_toml(crate: LibraryCrate) -> str:+ """Encode a normalized crate as canonical TOML — dotted qualified refs are quoted single keys."""+ return tomlkit.dumps(_encoding_payload(crate))+++def encode_crate(crate: LibraryCrate, *, encoding: CrateEncoding) -> str:+ """Encode a normalized crate in the selected canonical encoding."""+ match encoding:+ case CrateEncoding.JSON:+ return encode_crate_json(crate)+ case CrateEncoding.TOML:+ return encode_crate_toml(crate)diff --git a/pipelex/codegen/emission.py b/pipelex/codegen/emission.pynew file mode 100644index 000000000..4c81c5a98--- /dev/null+++ b/pipelex/codegen/emission.py@@ -0,0 +1,199 @@+"""Write a stamped, locked projection to disk — idempotently.++The writing layer sits between an emitter (which produces pure `EmittedFile` bodies) and the filesystem.+For one projection run it: stamps each body (so every file self-describes), writes each file only when+its content changed (no mtime churn — clean diffs, watch-mode friendly), removes any previously-tracked+file that dropped out of the set (so a deleted concept never lingers as a stale generated file), and+rewrites `codegen.lock` with the new artifact set.++Only files the tool itself stamped are ever removed, so a hand-authored file sharing the output+directory is never touched. Input templates (`codegen inputs`) are deliberately *not* stamped or locked:+they are user-editable scaffolds, not tracked generated code.+"""++from pathlib import Path++from pydantic import BaseModel, ConfigDict, Field++from pipelex.codegen.emitters.target import CodegenKind, CodegenTarget, EmittedFile+from pipelex.codegen.exceptions import CodegenError, CodegenLockError+from pipelex.codegen.lock import (+ CODEGEN_LOCK_FILENAME,+ CodegenLock,+ build_lock,+ encode_lock,+ load_lock,+ resolve_artifact_path,+ resolve_output_path,+ validate_artifact_paths,+)+from pipelex.codegen.stamp import apply_stamp, comment_prefix_for, compute_content_hash, has_stamp, parse_stamped+from pipelex.tools.misc.file_utils import ensure_directory_for_file_path, failable_load_text_from_path, remove_file, save_text_to_path+from pipelex.tools.typing.pydantic_utils import empty_list_factory_of+++class StampedProjection(BaseModel):+ """One projection's stamped artifact set plus its lock — pure content, no filesystem.++ This is the single source of truth for what a projection *is* on disk: `write_stamped_projection`+ materializes it locally, and a serving host (the HTTP codegen route) returns it over the wire —+ so a client that writes `files` and `lock_content` verbatim produces byte-identical artifacts+ and passes the offline `codegen check` exactly as a local run would.+ """++ model_config = ConfigDict(frozen=True)++ files: list[EmittedFile] = Field(default_factory=empty_list_factory_of(EmittedFile))+ """The emitted files with their stamp headers applied (same filenames, stamped contents)."""++ lock: CodegenLock+ """The artifact-set lock (paths + body hashes + crate fingerprint + engine version)."""++ lock_content: str+ """The lock encoded as canonical TOML — written verbatim as `codegen.lock`."""+++def build_stamped_projection(+ emitted: list[EmittedFile],+ *,+ crate_fingerprint: str,+ engine_version: str,+ kind: CodegenKind,+ target: CodegenTarget,+ pipe_ref: str | None = None,+ options: dict[str, str] | None = None,+) -> StampedProjection:+ """Stamp each emitted body and assemble the matching lock — pure, no filesystem access."""+ validate_artifact_paths(emitted_file.filename for emitted_file in emitted)+ resolved_options = options or {}+ stamped_files: list[EmittedFile] = []+ artifact_hashes: dict[str, str] = {}+ for emitted_file in emitted:+ stamped = apply_stamp(+ emitted_file.content,+ crate_fingerprint=crate_fingerprint,+ engine_version=engine_version,+ kind=kind,+ target=target,+ pipe_ref=pipe_ref,+ options=resolved_options,+ comment_prefix=comment_prefix_for(emitted_file.filename),+ )+ stamped_files.append(EmittedFile(filename=emitted_file.filename, content=stamped))+ artifact_hashes[emitted_file.filename] = compute_content_hash(emitted_file.content)+ lock = build_lock(crate_fingerprint=crate_fingerprint, engine_version=engine_version, artifacts=artifact_hashes)+ return StampedProjection(files=stamped_files, lock=lock, lock_content=encode_lock(lock))+++class WriteReport(BaseModel):+ """What one projection write did to the output tree — for the CLI to report to the user."""++ model_config = ConfigDict(frozen=True)++ written: list[str] = Field(default_factory=list)+ """Artifacts whose content changed and were (re)written."""++ unchanged: list[str] = Field(default_factory=list)+ """Artifacts whose content was already current (skipped — no mtime churn)."""++ removed: list[str] = Field(default_factory=list)+ """Previously-tracked stamped artifacts that dropped out of the set and were deleted."""++ lock_path: str = ""+ """The `codegen.lock` path (relative to the output root)."""+++def write_stamped_projection(+ emitted: list[EmittedFile],+ *,+ output_dir: Path,+ crate_fingerprint: str,+ engine_version: str,+ kind: CodegenKind,+ target: CodegenTarget,+ pipe_ref: str | None = None,+ options: dict[str, str] | None = None,+) -> WriteReport:+ """Stamp, write-if-changed, prune de-listed files, and rewrite the lock for one projection run."""+ lock_path = resolve_output_path(output_dir, relative_path=Path(CODEGEN_LOCK_FILENAME))+ output_root = lock_path.parent+ previous_paths = _previous_tracked_paths(lock_path)++ projection = build_stamped_projection(+ emitted,+ crate_fingerprint=crate_fingerprint,+ engine_version=engine_version,+ kind=kind,+ target=target,+ pipe_ref=pipe_ref,+ options=options,+ )+ _preflight_destinations(output_root=output_root, projection=projection, previous_paths=previous_paths)++ written: list[str] = []+ unchanged: list[str] = []+ for stamped_file in projection.files:+ destination = resolve_artifact_path(output_root, artifact_path=stamped_file.filename)+ if _write_if_changed(destination, content=stamped_file.content):+ written.append(stamped_file.filename)+ else:+ unchanged.append(stamped_file.filename)++ removed = _prune_delisted(output_dir=output_root, previous_paths=previous_paths, current_paths=projection.lock.paths())++ _write_if_changed(lock_path, content=projection.lock_content)++ return WriteReport(written=written, unchanged=unchanged, removed=removed, lock_path=CODEGEN_LOCK_FILENAME)+++def _preflight_destinations(*, output_root: Path, projection: StampedProjection, previous_paths: set[str]) -> None:+ """Reject unowned destination collisions before any projection file is written."""+ for stamped_file in projection.files:+ destination = resolve_artifact_path(output_root, artifact_path=stamped_file.filename)+ existing_content = failable_load_text_from_path(destination)+ if existing_content is None or existing_content == stamped_file.content:+ continue+ if stamped_file.filename in previous_paths:+ continue+ comment_prefix = comment_prefix_for(stamped_file.filename)+ if parse_stamped(existing_content, comment_prefix=comment_prefix) is not None:+ continue+ msg = f"Refusing to overwrite unowned file '{destination}'. Move it or choose a different codegen output directory."+ raise CodegenError(msg)+++def _previous_tracked_paths(lock_path: Path) -> set[str]:+ try:+ lock = load_lock(lock_path)+ except CodegenLockError as exc:+ if isinstance(exc.__cause__, CodegenError):+ # Unsafe tracked paths are not corrupt state to overwrite: they are a security+ # violation, and suppressing them would weaken the containment boundary.+ raise+ # A new projection is authoritative and can replace a corrupt prior lock. Without a+ # trustworthy artifact set there is nothing safe to prune, so recover with an empty set.+ return set()+ return lock.paths() if lock is not None else set()+++def _write_if_changed(path: Path, *, content: str) -> bool:+ """Write `content` to `path` only if it differs from what is already there. Returns whether it wrote."""+ if failable_load_text_from_path(path) == content:+ return False+ ensure_directory_for_file_path(file_path=path)+ save_text_to_path(text=content, path=path)+ return True+++def _prune_delisted(*, output_dir: Path, previous_paths: set[str], current_paths: set[str]) -> list[str]:+ """Delete files that were tracked before but are no longer produced — only if they still carry our stamp."""+ removed: list[str] = []+ for relative in sorted(previous_paths - current_paths):+ stale_path = resolve_artifact_path(output_dir, artifact_path=relative)+ content = failable_load_text_from_path(stale_path)+ if content is None:+ continue+ if has_stamp(content, comment_prefix=comment_prefix_for(relative)):+ remove_file(stale_path)+ removed.append(relative)+ return removeddiff --git a/pipelex/codegen/emitters/__init__.py b/pipelex/codegen/emitters/__init__.pynew file mode 100644index 000000000..e69de29bbdiff --git a/pipelex/codegen/emitters/naming.py b/pipelex/codegen/emitters/naming.pynew file mode 100644index 000000000..954efa68f--- /dev/null+++ b/pipelex/codegen/emitters/naming.py@@ -0,0 +1,102 @@+"""Name-derivation rules shared by every types emitter (see `docs/specs/pipelex-codegen.md`).++A qualified concept `domain.Code` yields a type named from `Code` (already PascalCase) when that+code is unique across the crate; when the code collides across domains, the domain disambiguates.+The disambiguated spelling differs by target because identifier grammars differ:++- Python allows the interpunct `·` (U+00B7, an `Other_ID_Continue` code point), so it reuses the+ runtime seed `make_qualified_structure_class_name` (`legal.contracts` + `Result` -> `legal·contracts__Result`);+- TypeScript does not allow `·` in identifiers, so a colliding TS type PascalCases and joins the+ domain segments before the code (`legal.contracts` + `Result` -> `LegalContractsResult`). Because+ that spelling is not injective, the TypeScript emitter allocates the final names crate-wide and+ adds deterministic numeric suffixes when distinct concepts still produce the same candidate.++Field names: the crate's snake_case is the wire contract, and every target keeps it verbatim (D10:+wire-native keys — a TS schema validates the wire payload directly, no key remapping layer).+"""++from pipelex.codegen.resolved_concepts import ResolvedConcept, ResolvedLibrary+from pipelex.core.concepts.helpers import make_qualified_structure_class_name+++def snake_to_pascal(name: str) -> str:+ """`snake_case` -> `PascalCase` (empty segments from stray underscores are dropped)."""+ return "".join(part[:1].upper() + part[1:] for part in name.split("_") if part)+++def python_class_name(*, domain: str, code: str, needs_qualification: bool) -> str:+ """The Python class name for a concept: bare `Code`, or the runtime domain-qualified spelling."""+ if not needs_qualification:+ return code+ return make_qualified_structure_class_name(domain_code=domain, concept_code=code)+++def ts_type_name(*, domain: str, code: str, needs_qualification: bool) -> str:+ """Derive a TypeScript name candidate; use `allocate_ts_type_names` for final emitted names."""+ if not needs_qualification:+ return code+ domain_pascal = "".join(snake_to_pascal(segment) for segment in domain.split("."))+ return f"{domain_pascal}{code}"+++def allocate_ts_type_names(library: ResolvedLibrary) -> dict[str, str]:+ """Allocate one unique, deterministic TypeScript type name per concept reference.++ A bare concept keeps its authored code when it collides with a generated qualified candidate.+ Other ties are settled by qualified ref, independent of input order. Numeric suffix allocation+ reserves every unsuffixed candidate first, so a disambiguated name cannot steal another+ concept's natural candidate. Each allocated type also reserves its derived `<Type>Schema`+ identifier because both names are imported together by the binder.+ """+ concepts_by_candidate: dict[str, list[ResolvedConcept]] = {}+ for concept in library.concepts:+ candidate = ts_type_name(domain=concept.domain, code=concept.code, needs_qualification=concept.needs_qualification)+ concepts_by_candidate.setdefault(candidate, []).append(concept)++ reserved_names = set(concepts_by_candidate)+ used_symbols: set[str] = set()+ allocated_by_ref: dict[str, str] = {}+ for candidate, concepts in sorted(concepts_by_candidate.items()):+ ordered_concepts = sorted(concepts, key=lambda concept: (concept.needs_qualification, concept.concept_ref))+ suffix = 2+ for concept_index, concept in enumerate(ordered_concepts):+ if concept_index == 0 and _ts_symbols_are_free(candidate, used_symbols=used_symbols):+ allocated_name = candidate+ else:+ allocated_name = f"{candidate}{suffix}"+ while (+ not _ts_symbols_are_free(allocated_name, used_symbols=used_symbols)+ or allocated_name in reserved_names+ or f"{allocated_name}Schema" in reserved_names+ ):+ suffix += 1+ allocated_name = f"{candidate}{suffix}"+ suffix += 1+ allocated_by_ref[concept.concept_ref] = allocated_name+ used_symbols.add(allocated_name)+ used_symbols.add(f"{allocated_name}Schema")++ return allocated_by_ref+++def _ts_symbols_are_free(type_name: str, *, used_symbols: set[str]) -> bool:+ """Whether a type and its derived schema identifier are both unclaimed."""+ return type_name not in used_symbols and f"{type_name}Schema" not in used_symbols+++def runtime_to_emitted_class_names(library: ResolvedLibrary) -> dict[str, str]:+ """Map each concept's runtime structure-class name to its emitted Python class name.++ The runtime materializes every non-native concept under the domain-qualified spelling+ (`make_qualified_structure_class_name`), while the Python emitters name bare-when-unique — this+ mapping lets a consumer (the runner-script generator) spell classes the way the emitted+ projection defines them. Opaque Python-backed concepts (`structure = "<ClassName>"`) keep the+ user's real class at runtime, so they are deliberately not remapped.+ """+ mapping: dict[str, str] = {}+ for concept in library.concepts:+ if concept.is_native or concept.opaque_python_class:+ continue+ runtime_name = make_qualified_structure_class_name(domain_code=concept.domain, concept_code=concept.code)+ mapping[runtime_name] = python_class_name(domain=concept.domain, code=concept.code, needs_qualification=concept.needs_qualification)+ return mappingdiff --git a/pipelex/codegen/emitters/python_common.py b/pipelex/codegen/emitters/python_common.pynew file mode 100644index 000000000..b41450197--- /dev/null+++ b/pipelex/codegen/emitters/python_common.py@@ -0,0 +1,126 @@+"""Rendering helpers shared by the two Python types emitters (structures + plain pydantic).++Only the target-agnostic pieces live here: string / default-value escaping, the generated-file+header (carrying the extension-file subclassing story), the class docstring, and the `= Field(...)`+assignment. Each Python emitter keeps its own resolved-type -> annotation walker, because the+container spelling (`typing.List`/`Optional` for the runtime idiom vs. builtin generics + `| None`+for plain pydantic) and the concept spelling (runtime content classes vs. generated models) differ.+"""++from datetime import date, datetime, time+from typing import Any, cast++from pipelex.codegen.resolved_concepts import ResolvedConcept+from pipelex.codegen.resolved_fields import ResolvedField, iter_imprecision_reasons+++def python_header(*, target: str) -> str:+ """The generated-file header: an AUTOGENERATED marker plus the extension-file (subclassing) story."""+ return (+ "# ---------------------------------------------------------------------------\n"+ "# AUTOGENERATED by Pipelex codegen — DO NOT EDIT.\n"+ "#\n"+ "# This file is a projection of a normalized MTHDS library crate. It is\n"+ "# regenerated from the method; any hand edit here is overwritten.\n"+ "#\n"+ "# To customize a generated type, do NOT edit this file. Create a sibling\n"+ "# module and subclass — subclasses survive regeneration:\n"+ "#\n"+ "# # my_types_ext.py\n"+ "# from .structures import Report\n"+ "#\n"+ "# class MyReport(Report):\n"+ "# ...\n"+ "#\n"+ f"# projection: types / {target}\n"+ "# ---------------------------------------------------------------------------\n"+ )+++def escape_py_string(value: str) -> str:+ """Escape a string for a double-quoted Python literal (handles quotes, backslashes, newlines)."""+ # repr() escapes everything correctly and returns a quoted literal; prefer double quotes when repr+ # chose single quotes (for consistency with the rest of the generated code).+ escaped = repr(value)+ if escaped.startswith("'") and escaped.endswith("'") and not escaped.startswith("'''"):+ inner = escaped[1:-1].replace("\\'", "'").replace('"', '\\"')+ return f'"{inner}"'+ return escaped+++def format_default_value(value: Any) -> str:+ """Format a default value as Python source. Temporal values rebuild via `fromisoformat` so they+ use the bare `date` / `datetime` names the generated module imports (repr module-qualifies them).+ """+ if isinstance(value, str):+ return escape_py_string(value)+ if isinstance(value, datetime):+ return f'datetime.fromisoformat("{value.isoformat()}")'+ if isinstance(value, date):+ return f'date.fromisoformat("{value.isoformat()}")'+ if isinstance(value, time):+ return f'time.fromisoformat("{value.isoformat()}")'+ if isinstance(value, list):+ items = cast("list[Any]", value)+ return f"[{', '.join(format_default_value(item) for item in items)}]"+ if isinstance(value, dict):+ pairs = cast("dict[str, Any]", value)+ rendered = [f"{format_default_value(key)}: {format_default_value(pairs[key])}" for key in sorted(pairs)]+ return f"{{{', '.join(rendered)}}}"+ return repr(value)+++def class_docstring(description: str, *, extra_line: str | None = None, indent: str = " ") -> str:+ """Render a class docstring; append `extra_line` (e.g. an imprecision caveat) on its own line."""+ text = description or ""+ if extra_line:+ body = f"{text}\n\n{indent}{extra_line}" if text else extra_line+ return f"{indent}{escape_py_string(body)}"+ return f"{indent}{escape_py_string(text)}"+++def field_line(field: ResolvedField, *, annotation: str) -> str:+ """Render one `name: <annotation> = Field(...)` line, surfacing any type-tree imprecision inline.++ `annotation` must already carry optionality in the emitter's own idiom (`Optional[...]` or `... | None`).+ """+ params: list[str]+ if field.required and field.default_value is None:+ params = ["..."]+ elif field.default_value is not None:+ params = [f"default={format_default_value(field.default_value)}"]+ else:+ params = ["default=None"]+ params.append(f"description={escape_py_string(field.description)}")++ line = f" {field.name}: {annotation} = Field({', '.join(params)})"+ reasons = list(dict.fromkeys(iter_imprecision_reasons(field.resolved_type)))+ if reasons:+ line += f" # imprecise: {'; '.join(reasons)}"+ return line+++def any_annotation(*, imports: set[str]) -> str:+ """The `Any` annotation, registering its import — the honest spelling for an imprecise type."""+ imports.add("from typing import Any")+ return "Any"+++def order_by_base(concepts: list[ResolvedConcept], *, in_module: set[str]) -> list[ResolvedConcept]:+ """Order concepts so an in-module refinement base is defined before its subclass (inheritance is eager)."""+ by_ref = {concept.concept_ref: concept for concept in concepts}+ ordered: list[ResolvedConcept] = []+ seen: set[str] = set()++ def visit(concept: ResolvedConcept) -> None:+ if concept.concept_ref in seen:+ return+ seen.add(concept.concept_ref)+ base_ref = concept.base_ref+ if base_ref is not None and base_ref in in_module and base_ref in by_ref:+ visit(by_ref[base_ref])+ ordered.append(concept)++ for concept in concepts:+ visit(concept)+ return ordereddiff --git a/pipelex/codegen/emitters/python_pydantic.py b/pipelex/codegen/emitters/python_pydantic.pynew file mode 100644index 000000000..fb871d9f9--- /dev/null+++ b/pipelex/codegen/emitters/python_pydantic.py@@ -0,0 +1,111 @@+"""python-pydantic emitter: project a crate's concept set as plain `pydantic.BaseModel` types.++The neutral type projection — the same shapes as python-structures but with no Pipelex imports and+no `StructuredContent` base, so the generated module depends only on `pydantic` (and stdlib). Every+concept in the crate is emitted uniformly, including the materialized natives (`native.Text` becomes+a `Text` model), so the file is self-contained: a consumer needs the crate and pydantic, nothing else.++Modern typing throughout: builtin generics (`list` / `dict`), `X | None` for optionals, and+`from __future__ import annotations` so concept references resolve as forward names from the module.+"""++from pipelex.codegen.emitters.naming import python_class_name+from pipelex.codegen.emitters.python_common import any_annotation, class_docstring, field_line, order_by_base, python_header+from pipelex.codegen.emitters.target import EmittedFile+from pipelex.codegen.resolved_concepts import ResolvedConcept, ResolvedLibrary+from pipelex.codegen.resolved_fields import ResolvedField, ResolvedType, ResolvedTypeKind++_FILENAME = "models.py"+++def emit_python_pydantic(library: ResolvedLibrary) -> list[EmittedFile]:+ """Emit the `models.py` module: one plain `BaseModel` per concept (natives included)."""+ by_ref = library.by_ref()+ in_module = {concept.concept_ref for concept in library.concepts}+ ordered = order_by_base(library.concepts, in_module=in_module)++ has_opaque = any(concept.structureless for concept in library.concepts)+ imports: set[str] = {"from pydantic import BaseModel, ConfigDict, Field"} if has_opaque else {"from pydantic import BaseModel, Field"}+ blocks = [_render_class(concept, by_ref=by_ref, imports=imports) for concept in ordered]++ header = python_header(target="python-pydantic")+ import_block = "\n".join(sorted(imports))+ body = f"{header}from __future__ import annotations\n\n{import_block}\n\n\n" + "\n\n\n".join(blocks) + "\n"+ return [EmittedFile(filename=_FILENAME, content=body)]+++def _render_class(concept: ResolvedConcept, *, by_ref: dict[str, ResolvedConcept], imports: set[str]) -> str:+ class_name = python_class_name(domain=concept.domain, code=concept.code, needs_qualification=concept.needs_qualification)+ base = _base_class(concept, by_ref=by_ref)+ caveat = f"Imprecise: {concept.imprecision_reason}." if concept.structureless and concept.imprecision_reason else None+ docstring = class_docstring(concept.description, extra_line=caveat)+ header = f"class {class_name}({base}):"+ if concept.structureless:+ # Opaque = pass-through, never lossy (B1-1): pydantic's default extra="ignore" would+ # silently strip every field on model_validate, so the unknown shape is kept verbatim.+ return f'{header}\n{docstring}\n\n model_config = ConfigDict(extra="allow")'+ if not concept.fields:+ return f"{header}\n{docstring}"+ lines = [_render_field(concept_field, by_ref=by_ref, imports=imports) for concept_field in concept.fields]+ return f"{header}\n{docstring}\n\n" + "\n".join(lines)+++def _base_class(concept: ResolvedConcept, *, by_ref: dict[str, ResolvedConcept]) -> str:+ if concept.base_ref is None:+ return "BaseModel"+ base = by_ref.get(concept.base_ref)+ if base is not None:+ return python_class_name(domain=base.domain, code=base.code, needs_qualification=base.needs_qualification)+ # Cross-package / unknown base is not resolvable in-crate: fall back to a structurally valid root.+ return "BaseModel"+++def _render_field(concept_field: ResolvedField, *, by_ref: dict[str, ResolvedConcept], imports: set[str]) -> str:+ annotation = _annotation(concept_field.resolved_type, by_ref=by_ref, imports=imports)+ if not concept_field.required:+ annotation = f"{annotation} | None"+ return field_line(concept_field, annotation=annotation)+++def _annotation(resolved_type: ResolvedType, *, by_ref: dict[str, ResolvedConcept], imports: set[str]) -> str:+ match resolved_type.kind:+ case ResolvedTypeKind.TEXT:+ return "str"+ case ResolvedTypeKind.NUMBER:+ return "float"+ case ResolvedTypeKind.INTEGER:+ return "int"+ case ResolvedTypeKind.BOOLEAN:+ return "bool"+ case ResolvedTypeKind.DATE:+ imports.add("from datetime import date")+ return "date"+ case ResolvedTypeKind.DATETIME:+ imports.add("from datetime import datetime")+ return "datetime"+ case ResolvedTypeKind.TIME:+ imports.add("from datetime import time")+ return "time"+ case ResolvedTypeKind.LITERAL:+ imports.add("from typing import Literal")+ return f"Literal[{', '.join(repr(choice) for choice in resolved_type.choices or [])}]"+ case ResolvedTypeKind.CONCEPT:+ return _concept_annotation(resolved_type, by_ref=by_ref, imports=imports)+ case ResolvedTypeKind.LIST:+ item = _annotation(resolved_type.item, by_ref=by_ref, imports=imports) if resolved_type.item else any_annotation(imports=imports)+ return f"list[{item}]"+ case ResolvedTypeKind.DICT:+ value = _annotation(resolved_type.value, by_ref=by_ref, imports=imports) if resolved_type.value else any_annotation(imports=imports)+ return f"dict[str, {value}]"+ case ResolvedTypeKind.ANY:+ return any_annotation(imports=imports)+++def _concept_annotation(resolved_type: ResolvedType, *, by_ref: dict[str, ResolvedConcept], imports: set[str]) -> str:+ concept_ref = resolved_type.concept_ref+ if concept_ref is None:+ return any_annotation(imports=imports)+ concept = by_ref.get(concept_ref)+ if concept is not None:+ return python_class_name(domain=concept.domain, code=concept.code, needs_qualification=concept.needs_qualification)+ return any_annotation(imports=imports)diff --git a/pipelex/codegen/emitters/python_structures.py b/pipelex/codegen/emitters/python_structures.pynew file mode 100644index 000000000..7a1572668--- /dev/null+++ b/pipelex/codegen/emitters/python_structures.py@@ -0,0 +1,132 @@+"""python-structures emitter: project a crate's concept set as Pipelex `StructuredContent` classes.++The runtime idiom (today's `StructureGenerator` presentation) over the shared resolved layers. Each+non-native concept becomes a `StructuredContent` subclass; native *references* map to the runtime+content classes (`TextContent`, ...); a concept that still refines a native inherits that content+class. Native concepts themselves are not re-emitted — they already exist in the runtime.++Field annotations use `from __future__ import annotations`, so concept references are plain forward+names resolved from the module namespace (no explicit string quoting, no ordering constraint beyond+class inheritance).+"""++from pipelex.codegen.emitters.naming import python_class_name+from pipelex.codegen.emitters.python_common import any_annotation, class_docstring, field_line, order_by_base, python_header+from pipelex.codegen.emitters.target import EmittedFile+from pipelex.codegen.resolved_concepts import ResolvedConcept, ResolvedLibrary+from pipelex.codegen.resolved_fields import ResolvedField, ResolvedType, ResolvedTypeKind+from pipelex.core.concepts.native.concept_native import NativeConceptCode+from pipelex.core.qualified_ref import QualifiedRef++_FILENAME = "structures.py"+++def emit_python_structures(library: ResolvedLibrary) -> list[EmittedFile]:+ """Emit the `structures.py` module: one `StructuredContent` subclass per non-native concept."""+ by_ref = library.by_ref()+ emitted = [concept for concept in library.concepts if not concept.is_native]+ in_module = {concept.concept_ref for concept in emitted}+ ordered = order_by_base(emitted, in_module=in_module)++ has_opaque = any(concept.structureless for concept in emitted)+ pydantic_import = "from pydantic import ConfigDict, Field" if has_opaque else "from pydantic import Field"+ imports: set[str] = {pydantic_import, "from pipelex.core.stuffs.structured_content import StructuredContent"}+ blocks = [_render_class(concept, by_ref=by_ref, imports=imports) for concept in ordered]++ header = python_header(target="python-structures")+ import_block = "\n".join(sorted(imports))+ body = f"{header}from __future__ import annotations\n\n{import_block}\n\n\n" + "\n\n\n".join(blocks) + "\n"+ return [EmittedFile(filename=_FILENAME, content=body)]+++def _render_class(concept: ResolvedConcept, *, by_ref: dict[str, ResolvedConcept], imports: set[str]) -> str:+ class_name = python_class_name(domain=concept.domain, code=concept.code, needs_qualification=concept.needs_qualification)+ base = _base_class(concept, by_ref=by_ref, imports=imports)+ caveat = f"Imprecise: {concept.imprecision_reason}." if concept.structureless and concept.imprecision_reason else None+ docstring = class_docstring(concept.description, extra_line=caveat)+ header = f"class {class_name}({base}):"+ if concept.structureless:+ # Opaque = pass-through, never lossy (B1-1): the runtime base inherits pydantic's default+ # extra="ignore", which would silently strip every field on model_validate.+ return f'{header}\n{docstring}\n\n model_config = ConfigDict(extra="allow")'+ if not concept.fields:+ return f"{header}\n{docstring}"+ lines = [_render_field(concept_field, by_ref=by_ref, imports=imports) for concept_field in concept.fields]+ return f"{header}\n{docstring}\n\n" + "\n".join(lines)+++def _base_class(concept: ResolvedConcept, *, by_ref: dict[str, ResolvedConcept], imports: set[str]) -> str:+ if concept.base_ref is None:+ return "StructuredContent"+ if NativeConceptCode.is_native_concept_ref_or_code(concept_ref_or_code=concept.base_ref):+ return _native_class(concept.base_ref, imports=imports)+ base = by_ref.get(concept.base_ref)+ if base is not None:+ return python_class_name(domain=base.domain, code=base.code, needs_qualification=base.needs_qualification)+ # Cross-package / unknown base is not resolvable in-crate: fall back to a structurally valid root.+ return "StructuredContent"+++def _render_field(concept_field: ResolvedField, *, by_ref: dict[str, ResolvedConcept], imports: set[str]) -> str:+ annotation = _annotation(concept_field.resolved_type, by_ref=by_ref, imports=imports)+ if not concept_field.required:+ imports.add("from typing import Optional")+ annotation = f"Optional[{annotation}]"+ return field_line(concept_field, annotation=annotation)+++def _annotation(resolved_type: ResolvedType, *, by_ref: dict[str, ResolvedConcept], imports: set[str]) -> str:+ match resolved_type.kind:+ case ResolvedTypeKind.TEXT:+ return "str"+ case ResolvedTypeKind.NUMBER:+ return "float"+ case ResolvedTypeKind.INTEGER:+ return "int"+ case ResolvedTypeKind.BOOLEAN:+ return "bool"+ case ResolvedTypeKind.DATE:+ imports.add("from datetime import date")+ return "date"+ case ResolvedTypeKind.DATETIME:+ imports.add("from datetime import datetime")+ return "datetime"+ case ResolvedTypeKind.TIME:+ imports.add("from datetime import time")+ return "time"+ case ResolvedTypeKind.LITERAL:+ imports.add("from typing import Literal")+ return f"Literal[{', '.join(repr(choice) for choice in resolved_type.choices or [])}]"+ case ResolvedTypeKind.CONCEPT:+ return _concept_annotation(resolved_type, by_ref=by_ref, imports=imports)+ case ResolvedTypeKind.LIST:+ imports.add("from typing import List")+ item = _annotation(resolved_type.item, by_ref=by_ref, imports=imports) if resolved_type.item else any_annotation(imports=imports)+ return f"List[{item}]"+ case ResolvedTypeKind.DICT:+ imports.add("from typing import Dict")+ value = _annotation(resolved_type.value, by_ref=by_ref, imports=imports) if resolved_type.value else any_annotation(imports=imports)+ return f"Dict[str, {value}]"+ case ResolvedTypeKind.ANY:+ return any_annotation(imports=imports)+++def _concept_annotation(resolved_type: ResolvedType, *, by_ref: dict[str, ResolvedConcept], imports: set[str]) -> str:+ concept_ref = resolved_type.concept_ref+ if concept_ref is None:+ return any_annotation(imports=imports)+ if resolved_type.is_native:+ return _native_class(concept_ref, imports=imports)+ concept = by_ref.get(concept_ref)+ if concept is not None:+ return python_class_name(domain=concept.domain, code=concept.code, needs_qualification=concept.needs_qualification)+ return any_annotation(imports=imports)+++def _native_class(concept_ref: str, *, imports: set[str]) -> str:+ native_code = NativeConceptCode(QualifiedRef.parse(concept_ref).local_code)+ structure_class = native_code.structure_class+ if structure_class is None:+ return "StructuredContent"+ imports.add(f"from {structure_class.__module__} import {structure_class.__name__}")+ return structure_class.__name__diff --git a/pipelex/codegen/emitters/target.py b/pipelex/codegen/emitters/target.pynew file mode 100644index 000000000..61c5e5888--- /dev/null+++ b/pipelex/codegen/emitters/target.py@@ -0,0 +1,43 @@+"""The two codegen axes (`kind` and `target`) and the emitted-file unit shared by every emitter.++Codegen has exactly two explicit axes (see `docs/specs/pipelex-codegen.md` → "Two axes"): `kind` is+*what* to project (`types` over the concept set, `inputs` per pipe, …), `target` is *for whom* (a+language / idiom flavor). An emitter returns one or more `EmittedFile`s — a filename relative to the+output root plus its content — so a single projection can span more than one file (the ts-zod purity+split: a pure types file plus a thin binder file). The `kind`/`target` pair is what a generated file's+stamp records as its projection.+"""++from enum import StrEnum++from pydantic import BaseModel, ConfigDict+++class CodegenKind(StrEnum):+ """A codegen projection `kind` — the *what* axis. Only the kinds that emit tracked artifacts today+ are listed: `types` (over the concept set) and `inputs` (per pipe). Recorded in a file's stamp.+ """++ TYPES = "types"+ INPUTS = "inputs"+++class CodegenTarget(StrEnum):+ """A codegen target flavor. All targets are Pipelex projections — the MTHDS standard specifies no+ type projection (see `docs/specs/pipelex-codegen.md` → "Ownership"). They differ in audience:+ `ts-zod` and `python-pydantic` emit idiom-neutral types any consumer can use; `python-structures`+ emits StructuredContent classes for a Pipelex runtime host.+ """++ PYTHON_STRUCTURES = "python-structures"+ PYTHON_PYDANTIC = "python-pydantic"+ TS_ZOD = "ts-zod"+++class EmittedFile(BaseModel):+ """One generated file: a filename relative to the output root and its full content."""++ model_config = ConfigDict(frozen=True)++ filename: str+ content: strdiff --git a/pipelex/codegen/emitters/ts_zod.py b/pipelex/codegen/emitters/ts_zod.pynew file mode 100644index 000000000..9a0a37a54--- /dev/null+++ b/pipelex/codegen/emitters/ts_zod.py@@ -0,0 +1,333 @@+"""ts-zod emitter: project a crate's concept set as a pure TypeScript + Zod types file plus a binder.++`types.ts` imports **only** `zod` — Zod schemas plus their inferred types — so it stays dependency-free+and portable. `binder.ts` is the thin companion that exposes a typed `parse<Name>` / `serialize<Name>`+pair per concept over those schemas. A pipe's IO types are concepts, so a pipe's output parser / input+serializer is just the binder pair for those concept types — the binder is the concept-set-wide+realization of the per-pipe parse/serialize helpers.++**Wire-native field keys (D10).** Object keys are the crate's snake_case field names *verbatim*, so a+schema validates a wire payload directly — no key remapping layer, and therefore no way to silently+corrupt arbitrary keys inside a `z.record()` / `z.unknown()` value (a camelCase remapping binder can't+tell a schema-declared field key from arbitrary data — e.g. `native.JSON`'s `json_obj` map). The+inferred `z.infer` types therefore carry snake_case keys. (This departs from a camelCase-TS convention;+if camelCase ergonomics are later wanted they must ride a *schema-aware* transform, not a blind deep+key remap — see D10 in `_codegen/TODOS.md`.) Concept references use `z.lazy(() => XSchema)` so schema+declaration order is irrelevant and reference cycles are handled.++TypeScript customization is by declaration merging (augment the generated `interface`/module from a+sibling file), the TS analog of the Python subclassing story; the generated files are never edited.+"""++import json+from typing import Any++from pipelex.codegen.emitters.naming import allocate_ts_type_names+from pipelex.codegen.emitters.target import EmittedFile+from pipelex.codegen.resolved_concepts import ResolvedConcept, ResolvedLibrary+from pipelex.codegen.resolved_fields import ResolvedField, ResolvedType, ResolvedTypeKind, iter_imprecision_reasons+from pipelex.tools.misc.json_utils import clean_json_content++_FILENAME = "types.ts"+_BINDER_FILENAME = "binder.ts"+++def emit_ts_zod(library: ResolvedLibrary) -> list[EmittedFile]:+ """Emit the pure `types.ts` file and its `binder.ts` companion (wire<->domain parse/serialize)."""+ concepts = sorted(library.concepts, key=lambda concept: concept.concept_ref)+ by_ref = library.by_ref()+ type_name_by_ref = allocate_ts_type_names(library)+ recursive_refs = _find_recursive_concept_refs(concepts)+ header = _ts_header()+ blocks = [_render_schema(concept, by_ref=by_ref, type_name_by_ref=type_name_by_ref, recursive_refs=recursive_refs) for concept in concepts]+ body = f'{header}import {{ z }} from "zod";\n\n\n' + "\n\n\n".join(blocks) + "\n"+ return [EmittedFile(filename=_FILENAME, content=body), _emit_binder(concepts, type_name_by_ref=type_name_by_ref)]+++def _ts_header() -> str:+ return (+ "// ---------------------------------------------------------------------------\n"+ "// AUTOGENERATED by Pipelex codegen — DO NOT EDIT.\n"+ "//\n"+ "// This file is a projection of a normalized MTHDS library crate. It is\n"+ "// regenerated from the method; any hand edit here is overwritten.\n"+ "//\n"+ "// To customize a generated type, do NOT edit this file. Augment it from a\n"+ "// sibling module via declaration merging — augmentations survive regeneration.\n"+ "//\n"+ "// projection: types / ts-zod\n"+ "// ---------------------------------------------------------------------------\n"+ )+++def _render_schema(+ concept: ResolvedConcept,+ *,+ by_ref: dict[str, ResolvedConcept],+ type_name_by_ref: dict[str, str],+ recursive_refs: set[str],+) -> str:+ type_name = type_name_by_ref[concept.concept_ref]+ doc = _jsdoc(concept.description, imprecise=concept.imprecision_reason if concept.structureless else None)++ schema_expr = _schema_expr(concept, by_ref=by_ref, type_name_by_ref=type_name_by_ref)+ if concept.concept_ref in recursive_refs:+ type_expr = _concept_type_expr(concept, by_ref=by_ref, type_name_by_ref=type_name_by_ref)+ return f"{doc}export type {type_name} = {type_expr};\nexport const {type_name}Schema: z.ZodType<{type_name}> = {schema_expr};"+ return f"{doc}export const {type_name}Schema = {schema_expr};\nexport type {type_name} = z.infer<typeof {type_name}Schema>;"+++def _schema_expr(+ concept: ResolvedConcept,+ *,+ by_ref: dict[str, ResolvedConcept],+ type_name_by_ref: dict[str, str],+) -> str:+ if concept.base_ref is not None and not concept.fields:+ base = by_ref.get(concept.base_ref)+ if base is not None:+ base_name = type_name_by_ref[base.concept_ref]+ return f"z.lazy(() => {base_name}Schema)"+ return "z.unknown()"+ if not concept.fields:+ if concept.structureless:+ # Structureless / opaque concept: unknown shape, surfaced (never guessed).+ return "z.unknown()"+ return "z.object({})"+ field_lines = "\n".join(_render_field(concept_field, by_ref=by_ref, type_name_by_ref=type_name_by_ref) for concept_field in concept.fields)+ return f"z.object({{\n{field_lines}\n}})"+++def _render_field(+ concept_field: ResolvedField,+ *,+ by_ref: dict[str, ResolvedConcept],+ type_name_by_ref: dict[str, str],+) -> str:+ # Wire-native key: the crate's snake_case field name verbatim (D10), so the schema validates the+ # wire payload directly and no key transform can corrupt nested record/opaque data.+ key = concept_field.name+ reasons = list(dict.fromkeys(iter_imprecision_reasons(concept_field.resolved_type)))+ doc = _jsdoc(concept_field.description, imprecise="; ".join(reasons) if reasons else None, indent=" ")+ expr = _zod_type(concept_field.resolved_type, by_ref=by_ref, type_name_by_ref=type_name_by_ref)+ if concept_field.default_value is not None:+ expr = f"{expr}.default({_format_default_value(concept_field.default_value)})"+ elif not concept_field.required:+ expr = f"{expr}.optional()"+ return f"{doc} {key}: {expr},"+++def _format_default_value(value: Any) -> str:+ """Render a JSON-safe default canonically so mapping insertion order cannot affect output."""+ return json.dumps(clean_json_content(value), sort_keys=True)+++def _concept_type_expr(+ concept: ResolvedConcept,+ *,+ by_ref: dict[str, ResolvedConcept],+ type_name_by_ref: dict[str, str],+) -> str:+ """Render the explicit TypeScript type required to break recursive schema inference."""+ if concept.base_ref is not None and not concept.fields:+ base = by_ref.get(concept.base_ref)+ return type_name_by_ref[base.concept_ref] if base is not None else "unknown"+ if concept.structureless:+ return "unknown"+ field_lines = "\n".join(_render_type_field(field, by_ref=by_ref, type_name_by_ref=type_name_by_ref) for field in concept.fields)+ return f"{{\n{field_lines}\n}}"+++def _render_type_field(+ concept_field: ResolvedField,+ *,+ by_ref: dict[str, ResolvedConcept],+ type_name_by_ref: dict[str, str],+) -> str:+ optional_marker = "?" if not concept_field.required and concept_field.default_value is None else ""+ type_expr = _ts_type(concept_field.resolved_type, by_ref=by_ref, type_name_by_ref=type_name_by_ref)+ return f" {concept_field.name}{optional_marker}: {type_expr};"+++def _ts_type(+ resolved_type: ResolvedType,+ *,+ by_ref: dict[str, ResolvedConcept],+ type_name_by_ref: dict[str, str],+) -> str:+ """Render one neutral resolved type as an explicit TypeScript type expression."""+ match resolved_type.kind:+ case ResolvedTypeKind.TEXT | ResolvedTypeKind.DATE | ResolvedTypeKind.DATETIME | ResolvedTypeKind.TIME:+ return "string"+ case ResolvedTypeKind.NUMBER | ResolvedTypeKind.INTEGER:+ return "number"+ case ResolvedTypeKind.BOOLEAN:+ return "boolean"+ case ResolvedTypeKind.LITERAL:+ return " | ".join(json.dumps(choice) for choice in resolved_type.choices or [])+ case ResolvedTypeKind.CONCEPT:+ return _concept_type(resolved_type, by_ref=by_ref, type_name_by_ref=type_name_by_ref)+ case ResolvedTypeKind.LIST:+ item = _ts_type(resolved_type.item, by_ref=by_ref, type_name_by_ref=type_name_by_ref) if resolved_type.item else "unknown"+ return f"Array<{item}>"+ case ResolvedTypeKind.DICT:+ value = _ts_type(resolved_type.value, by_ref=by_ref, type_name_by_ref=type_name_by_ref) if resolved_type.value else "unknown"+ return f"Record<string, {value}>"+ case ResolvedTypeKind.ANY:+ return "unknown"+++def _concept_type(+ resolved_type: ResolvedType,+ *,+ by_ref: dict[str, ResolvedConcept],+ type_name_by_ref: dict[str, str],+) -> str:+ concept_ref = resolved_type.concept_ref+ if concept_ref is None:+ return "unknown"+ concept = by_ref.get(concept_ref)+ return type_name_by_ref[concept.concept_ref] if concept is not None else "unknown"+++def _find_recursive_concept_refs(concepts: list[ResolvedConcept]) -> set[str]:+ """Find concepts that can reach themselves through an in-crate type reference."""+ graph = {concept.concept_ref: _concept_dependencies(concept) for concept in concepts}+ recursive_refs: set[str] = set()+ for origin_ref, dependencies in graph.items():+ pending_refs = list(dependencies)+ visited_refs: set[str] = set()+ while pending_refs:+ current_ref = pending_refs.pop()+ if current_ref == origin_ref:+ recursive_refs.add(origin_ref)+ break+ if current_ref in visited_refs:+ continue+ visited_refs.add(current_ref)+ pending_refs.extend(graph.get(current_ref, set()))+ return recursive_refs+++def _concept_dependencies(concept: ResolvedConcept) -> set[str]:+ dependencies: set[str] = {concept.base_ref} if concept.base_ref is not None else set()+ for concept_field in concept.fields:+ dependencies.update(_resolved_type_concept_refs(concept_field.resolved_type))+ return dependencies+++def _resolved_type_concept_refs(resolved_type: ResolvedType) -> set[str]:+ concept_refs: set[str] = {resolved_type.concept_ref} if resolved_type.concept_ref is not None else set()+ for child_type in (resolved_type.item, resolved_type.key, resolved_type.value):+ if child_type is not None:+ concept_refs.update(_resolved_type_concept_refs(child_type))+ return concept_refs+++def _zod_type(+ resolved_type: ResolvedType,+ *,+ by_ref: dict[str, ResolvedConcept],+ type_name_by_ref: dict[str, str],+) -> str:+ match resolved_type.kind:+ case ResolvedTypeKind.TEXT:+ return "z.string()"+ case ResolvedTypeKind.NUMBER:+ return "z.number()"+ case ResolvedTypeKind.INTEGER:+ return "z.number().int()"+ case ResolvedTypeKind.BOOLEAN:+ return "z.boolean()"+ case ResolvedTypeKind.DATE | ResolvedTypeKind.DATETIME | ResolvedTypeKind.TIME:+ # ISO date / datetime / time string on the wire (the `@wire` field keeps the shape documented).+ return "z.string()"+ case ResolvedTypeKind.LITERAL:+ choices = ", ".join(json.dumps(choice) for choice in resolved_type.choices or [])+ return f"z.enum([{choices}])"+ case ResolvedTypeKind.CONCEPT:+ return _concept_schema(resolved_type, by_ref=by_ref, type_name_by_ref=type_name_by_ref)+ case ResolvedTypeKind.LIST:+ item = _zod_type(resolved_type.item, by_ref=by_ref, type_name_by_ref=type_name_by_ref) if resolved_type.item else "z.unknown()"+ return f"z.array({item})"+ case ResolvedTypeKind.DICT:+ value = _zod_type(resolved_type.value, by_ref=by_ref, type_name_by_ref=type_name_by_ref) if resolved_type.value else "z.unknown()"+ return f"z.record(z.string(), {value})"+ case ResolvedTypeKind.ANY:+ return "z.unknown()"+++def _concept_schema(+ resolved_type: ResolvedType,+ *,+ by_ref: dict[str, ResolvedConcept],+ type_name_by_ref: dict[str, str],+) -> str:+ concept_ref = resolved_type.concept_ref+ if concept_ref is None:+ return "z.unknown()"+ concept = by_ref.get(concept_ref)+ if concept is not None:+ name = type_name_by_ref[concept.concept_ref]+ return f"z.lazy(() => {name}Schema)"+ return "z.unknown()"+++def _emit_binder(concepts: list[ResolvedConcept], *, type_name_by_ref: dict[str, str]) -> EmittedFile:+ """Emit `binder.ts`: typed `parse<Name>` / `serialize<Name>` helpers over the `types.ts` schemas.++ Keys are wire-native (D10), so parse/serialize are a direct `Schema.parse` — no key remapping, and+ thus no risk of corrupting arbitrary keys inside a `z.record()` / `z.unknown()` value.+ """+ type_names = [type_name_by_ref[concept.concept_ref] for concept in concepts]+ import_lines = "\n".join(f" {name}Schema,\n type {name}," for name in type_names)+ pairs = "\n\n".join(_binder_pair(name) for name in type_names)+ prelude = f'import {{\n{import_lines}\n}} from "./types";'+ return EmittedFile(filename=_BINDER_FILENAME, content=f"{_binder_header()}{prelude}\n\n\n{pairs}\n")+++def _binder_pair(type_name: str) -> str:+ """The `parse<Name>` / `serialize<Name>` pair for one concept type (wire-native, so `Schema.parse`)."""+ return (+ f"/** Parse a wire payload into a validated `{type_name}`. */\n"+ f"export function parse{type_name}(wire: unknown): {type_name} {{\n"+ f" return {type_name}Schema.parse(wire);\n"+ f"}}\n\n"+ f"/** Validate a `{type_name}` for the wire (keys are already wire-native). */\n"+ f"export function serialize{type_name}(value: {type_name}): {type_name} {{\n"+ f" return {type_name}Schema.parse(value);\n"+ f"}}"+ )+++def _binder_header() -> str:+ return (+ "// ---------------------------------------------------------------------------\n"+ "// AUTOGENERATED by Pipelex codegen — DO NOT EDIT.\n"+ "//\n"+ "// Typed parse/serialize helpers over the pure `types.ts` schemas. Field keys are\n"+ "// wire-native (snake_case), so parse/serialize validate the wire payload directly.\n"+ "//\n"+ "// projection: types / ts-zod (binder)\n"+ "// ---------------------------------------------------------------------------\n"+ )+++def _jsdoc(description: str, *, imprecise: str | None, indent: str = "") -> str:+ parts = _sanitize_jsdoc_text(description).split("\n") if description else []+ if imprecise:+ imprecise_lines = _sanitize_jsdoc_text(imprecise).split("\n")+ parts.append(f"@imprecise {imprecise_lines[0]}")+ parts.extend(imprecise_lines[1:])+ if not parts:+ return ""+ if len(parts) == 1 and imprecise is None:+ return f"{indent}/** {parts[0]} */\n"+ body = "".join(f"{indent} * {part}\n" for part in parts)+ return f"{indent}/**\n{body}{indent} */\n"+++def _sanitize_jsdoc_text(value: str) -> str:+ """Normalize line endings and neutralize comment terminators in authored JSDoc text."""+ normalized = value.replace("\r\n", "\n").replace("\r", "\n")+ return normalized.replace("*/", "* /")diff --git a/pipelex/codegen/emitters/types_emitter.py b/pipelex/codegen/emitters/types_emitter.pynew file mode 100644index 000000000..528c6da59--- /dev/null+++ b/pipelex/codegen/emitters/types_emitter.py@@ -0,0 +1,25 @@+"""The `codegen types` projection: a crate's concept set -> typed artifacts, selected by target.++`emit_types` is the single entry point the CLI (and route) call. It resolves the crate once into the+neutral `ResolvedLibrary` and dispatches to the target's emitter. The result is one or more+`EmittedFile`s; the caller decides where they land on disk.+"""++from pipelex.codegen.emitters.python_pydantic import emit_python_pydantic+from pipelex.codegen.emitters.python_structures import emit_python_structures+from pipelex.codegen.emitters.target import CodegenTarget, EmittedFile+from pipelex.codegen.emitters.ts_zod import emit_ts_zod+from pipelex.codegen.resolved_concepts import resolve_concepts_from_crate+from pipelex.libraries.library_crate import LibraryCrate+++def emit_types(crate: LibraryCrate, *, target: CodegenTarget) -> list[EmittedFile]:+ """Project the concept set of a normalized `crate` into typed artifacts for `target`."""+ library = resolve_concepts_from_crate(crate)+ match target:+ case CodegenTarget.PYTHON_STRUCTURES:+ return emit_python_structures(library)+ case CodegenTarget.PYTHON_PYDANTIC:+ return emit_python_pydantic(library)+ case CodegenTarget.TS_ZOD:+ return emit_ts_zod(library)diff --git a/pipelex/codegen/exceptions.py b/pipelex/codegen/exceptions.pynew file mode 100644index 000000000..00c9b5de0--- /dev/null+++ b/pipelex/codegen/exceptions.py@@ -0,0 +1,19 @@+from pipelex.base_exceptions import PipelexError+++class CodegenError(PipelexError):+ """Base error for the codegen projection engine (stamps, lock, check, emission)."""++ _declared_title = "Codegen error"+++class CodegenStampError(CodegenError):+ """Raised when a generated file cannot be stamped (e.g. an unstampable file type)."""++ _declared_title = "Codegen stamp error"+++class CodegenLockError(CodegenError):+ """Raised when a `codegen.lock` cannot be read or parsed."""++ _declared_title = "Codegen lock error"diff --git a/pipelex/codegen/lock.py b/pipelex/codegen/lock.pynew file mode 100644index 000000000..1299da8b2--- /dev/null+++ b/pipelex/codegen/lock.py@@ -0,0 +1,177 @@+"""`codegen.lock`: the set-level manifest of generated artifacts (a Pipelex-codegen artifact).++Per-file stamps let a lone file testify about itself; the lock catches the one drift class a stamp+cannot — a **deleted concept whose stale generated file lingers**. It records the generated artifact+*set* (each artifact's path + body content hash) plus the source crate fingerprint and engine version+the set was generated against, so the offline check can spot a file on disk that no longer belongs+(orphan) or a locked file that has vanished (deleted).++`codegen.lock` is Pipelex-owned and distinct from the standard's `methods.lock` (which pins remote+dependency versions): different owner, location, content, and lifecycle — see+`docs/specs/pipelex-codegen.md` → "Lock format". It is encoded as human-diffable TOML, artifacts+sorted by path so version-control diffs stay minimal.+"""+# tomlkit is not fully typed (`tomlkit.dumps`), so its member access reads as unknown here.+# pyright: reportUnknownMemberType=false++import os+from collections.abc import Iterable+from pathlib import Path, PurePosixPath, PureWindowsPath+from typing import NoReturn+from unicodedata import category++import tomlkit+from pydantic import BaseModel, ConfigDict, Field++from pipelex.codegen.exceptions import CodegenError, CodegenLockError+from pipelex.codegen.stamp import STAMPABLE_SUFFIXES+from pipelex.tools.misc.exceptions import TomlError+from pipelex.tools.misc.file_utils import load_text_from_path+from pipelex.tools.misc.toml_utils import load_toml_from_content+from pipelex.tools.typing.pydantic_utils import empty_list_factory_of++CODEGEN_LOCK_FILENAME = "codegen.lock"++_LOCK_HEADER = "# codegen.lock — generated artifact set (Pipelex codegen). Do not edit by hand.\n\n"+++class CodegenLockEntry(BaseModel):+ """One tracked artifact: its path relative to the lock, and its body content hash."""++ model_config = ConfigDict(frozen=True, extra="forbid")++ path: str+ content_hash: str+++class CodegenLock(BaseModel):+ """The generated artifact set for one output root, keyed to the crate + engine it was built against."""++ model_config = ConfigDict(frozen=True, extra="forbid")++ crate_fingerprint: str+ engine_version: str+ artifacts: list[CodegenLockEntry] = Field(default_factory=empty_list_factory_of(CodegenLockEntry))++ def paths(self) -> set[str]:+ """The set of tracked artifact paths (relative to the lock)."""+ validate_artifact_paths(entry.path for entry in self.artifacts)+ return {entry.path for entry in self.artifacts}++ def hash_by_path(self) -> dict[str, str]:+ """Map each tracked artifact path to its locked body content hash."""+ validate_artifact_paths(entry.path for entry in self.artifacts)+ return {entry.path: entry.content_hash for entry in self.artifacts}+++def validate_artifact_path(path: str) -> Path:+ """Validate one canonical artifact path and return its relative filesystem form."""+ if not path:+ _raise_path_error(path, reason="path is empty")+ if "\\" in path:+ _raise_path_error(path, reason="backslashes are not allowed; use forward slashes")+ if any(category(character).startswith("C") for character in path):+ _raise_path_error(path, reason="control characters are not allowed")++ posix_path = PurePosixPath(path)+ windows_path = PureWindowsPath(path)+ if posix_path.is_absolute() or windows_path.is_absolute() or windows_path.drive or windows_path.root:+ _raise_path_error(path, reason="absolute paths and drive prefixes are not allowed")++ parts = path.split("/")+ if any(part in {"", ".", ".."} for part in parts):+ _raise_path_error(path, reason="empty, '.', and '..' path components are not allowed")++ relative_path = Path(*parts)+ if relative_path.suffix not in STAMPABLE_SUFFIXES:+ expected = ", ".join(sorted(STAMPABLE_SUFFIXES))+ _raise_path_error(path, reason=f"unsupported artifact suffix (expected one of: {expected})")+ return relative_path+++def validate_artifact_paths(paths: Iterable[str]) -> dict[str, Path]:+ """Validate an artifact collection and reject duplicate canonical paths."""+ validated: dict[str, Path] = {}+ for path in paths:+ if path in validated:+ _raise_path_error(path, reason="Duplicate artifact path")+ validated[path] = validate_artifact_path(path)+ return validated+++def resolve_artifact_path(root: Path, *, artifact_path: str) -> Path:+ """Resolve a validated artifact beneath `root` without following symbolic links."""+ relative_path = validate_artifact_path(artifact_path)+ return resolve_output_path(root, relative_path=relative_path)+++def resolve_output_path(root: Path, *, relative_path: Path) -> Path:+ """Resolve one internal output file beneath `root`, rejecting all symlink components."""+ if relative_path.is_absolute() or not relative_path.parts or any(part in {"", ".", ".."} for part in relative_path.parts):+ _raise_path_error(str(relative_path), reason="internal output path must be a canonical relative path")++ requested_root = Path(os.path.normpath(root.absolute()))+ if requested_root.is_symlink():+ _raise_path_error(str(requested_root), reason="output root must not be a symbolic link")+ normalized_root = requested_root.resolve(strict=False)+ if normalized_root.exists() and not normalized_root.is_dir():+ _raise_path_error(str(normalized_root), reason="output root exists but is not a directory")++ destination = normalized_root / relative_path+ _reject_symlink_components(normalized_root, relative_path=relative_path)+ resolved_root = normalized_root.resolve(strict=False)+ resolved_destination = destination.resolve(strict=False)+ if not resolved_destination.is_relative_to(resolved_root):+ _raise_path_error(str(destination), reason=f"resolved path escapes output root '{resolved_root}'")+ if destination.exists() and not destination.is_file():+ _raise_path_error(str(destination), reason="output destination exists but is not a regular file")+ return destination+++def _reject_symlink_components(root: Path, *, relative_path: Path) -> None:+ current = root+ for part in relative_path.parts:+ current /= part+ if current.is_symlink():+ _raise_path_error(str(root / relative_path), reason=f"symbolic link component is not allowed: '{current}'")+++def _raise_path_error(path: str, *, reason: str) -> NoReturn:+ msg = f"Unsafe codegen artifact path '{path}': {reason}."+ raise CodegenError(msg)+++def build_lock(*, crate_fingerprint: str, engine_version: str, artifacts: dict[str, str]) -> CodegenLock:+ """Assemble a lock from a `{path: content_hash}` map, artifacts sorted by path for a stable diff."""+ validate_artifact_paths(artifacts)+ entries = [CodegenLockEntry(path=path, content_hash=content_hash) for path, content_hash in sorted(artifacts.items())]+ return CodegenLock(crate_fingerprint=crate_fingerprint, engine_version=engine_version, artifacts=entries)+++def encode_lock(lock: CodegenLock) -> str:+ """Encode the lock as canonical, human-diffable TOML (artifacts already sorted by path)."""+ validate_artifact_paths(entry.path for entry in lock.artifacts)+ payload = {+ "crate_fingerprint": lock.crate_fingerprint,+ "engine_version": lock.engine_version,+ "artifacts": [{"path": entry.path, "content_hash": entry.content_hash} for entry in lock.artifacts],+ }+ return _LOCK_HEADER + tomlkit.dumps(payload)+++def load_lock(lock_path: Path) -> CodegenLock | None:+ """Read and parse a `codegen.lock`, or `None` if it does not exist. Raises on a malformed lock."""+ if not lock_path.is_file():+ return None+ try:+ content = load_text_from_path(lock_path)+ data = load_toml_from_content(content)+ lock = CodegenLock.model_validate(data)+ validate_artifact_paths(entry.path for entry in lock.artifacts)+ return lock+ except (CodegenError, TomlError, OSError, ValueError, TypeError) as exc:+ # TomlError = malformed TOML; UnicodeDecodeError/ValueError = non-UTF-8 bytes or a pydantic+ # ValidationError on a shape mismatch (UnicodeDecodeError is a ValueError subclass);+ # OSError = the lock exists but cannot be read.+ msg = f"Malformed or unsafe codegen lock at '{lock_path}': {exc}"+ raise CodegenLockError(msg) from excdiff --git a/pipelex/codegen/native_expansion.py b/pipelex/codegen/native_expansion.pynew file mode 100644index 000000000..154fb75c3--- /dev/null+++ b/pipelex/codegen/native_expansion.py@@ -0,0 +1,178 @@+"""Materialize native concepts into explicit concept-object form for the normalized crate.++Native concepts (`native.Text`, `native.Image`, ...) are the standard's built-in vocabulary. To+make a crate self-contained (a consumer needs no hardcoded native table), every referenced native+is materialized into a `ConceptBlueprint`. Materialization is a **lookup, not a computation**: the+standard pins every native's normative blueprint form per MTHDS version+(`mthds/docs/spec/native-concepts.md`), and `pipelex/core/concepts/native/pinned_blueprints.py` is+the runtime copy of that pinned set. Deriving the definitions from the runtime content classes by+reflection would make this implementation's quirks the de-facto standard and break cross-+implementation fingerprint byte-agreement — so the reflection below is retained **only as the+consistency probe**: the unit test compares each runtime content class's reflected shape against+its pinned blueprint, proving the two can never drift silently.++Reflection is faithful-or-absent: a native's structure is reflected only when *every* field of its+content class maps unambiguously to a blueprint field — primitive, dict, list, a reference to+another native, or a nested non-native model (whose wire form is a JSON object, so it maps honestly+to a `dict` blueprint with unspecified value types — declared imprecision). A field whose+annotation has no honest blueprint form at all (e.g. a non-Optional union) makes the whole+reflected structure absent.+"""++import datetime+from types import UnionType+from typing import Any, Union, cast, get_args, get_origin++from pydantic import BaseModel++from pipelex.core.concepts.concept_blueprint import ConceptBlueprint, ConceptStructureBlueprintType+from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprint, ConceptStructureBlueprintFieldType+from pipelex.core.concepts.native.concept_native import NativeConceptCode+from pipelex.core.concepts.native.pinned_blueprints import make_pinned_native_blueprint+from pipelex.core.stuffs.stuff_content import StuffContent++_NATIVE_CLASS_NAME_TO_CODE: dict[str, NativeConceptCode] = {native_code.structure_class_name: native_code for native_code in NativeConceptCode}+++class _UnmappableAnnotationError(Exception):+ """Internal signal that a native field's annotation has no clean blueprint representation."""+++def materialize_native_concept(native_code: NativeConceptCode) -> ConceptBlueprint:+ """Return the pinned normative `ConceptBlueprint` for a native concept (a lookup, never reflection)."""+ return make_pinned_native_blueprint(native_code)+++def collect_native_refs_from_structure(structure: dict[str, ConceptStructureBlueprint]) -> set[str]:+ """Return the `native.<Code>` refs a materialized native structure itself references (for transitive expansion)."""+ refs: set[str] = set()+ for field_blueprint in structure.values():+ for ref in (field_blueprint.concept_ref, field_blueprint.item_concept_ref):+ if ref and NativeConceptCode.is_native_concept_ref_or_code(concept_ref_or_code=ref):+ refs.add(NativeConceptCode.get_validated_native_concept_ref(concept_ref_or_code=ref))+ return refs+++def reflect_native_structure(native_code: NativeConceptCode) -> dict[str, ConceptStructureBlueprintType] | None:+ """Reflect a native's runtime content class into blueprint form — the consistency probe, not the authority."""+ structure_class = native_code.structure_class+ if structure_class is None:+ return None+ model_fields = cast("type[StuffContent]", structure_class).model_fields+ if not model_fields:+ return None+ try:+ structure: dict[str, ConceptStructureBlueprintType] = {+ field_name: _annotation_to_blueprint(field_info.annotation, description=field_info.description or field_name)+ for field_name, field_info in model_fields.items()+ }+ except _UnmappableAnnotationError:+ return None+ return structure+++def _annotation_to_blueprint(annotation: Any, *, description: str) -> ConceptStructureBlueprint:+ inner, required = _strip_optional(annotation)++ if _is_number_union(inner):+ return ConceptStructureBlueprint(description=description, type=ConceptStructureBlueprintFieldType.NUMBER, required=required)+ if get_origin(inner) in {Union, UnionType}:+ # A union that is not simply Optional[X] nor a number union has no single blueprint shape.+ raise _UnmappableAnnotationError++ origin = get_origin(inner)+ if origin is dict:+ return ConceptStructureBlueprint(+ description=description, type=ConceptStructureBlueprintFieldType.DICT, key_type="text", value_type="Any", required=required+ )+ if origin is list:+ return _list_blueprint(inner, description=description, required=required)++ scalar_type = _scalar_field_type(inner)+ if scalar_type is not None:+ return ConceptStructureBlueprint(description=description, type=scalar_type, required=required)++ native_code = _native_code_for_content_class(inner)+ if native_code is not None:+ return ConceptStructureBlueprint(+ description=description, type=ConceptStructureBlueprintFieldType.CONCEPT, concept_ref=native_code.concept_ref, required=required+ )++ if _is_nested_model(inner):+ # A nested non-native model serializes as a JSON object; its honest blueprint form is a dict+ # with unspecified value types — declared imprecision, not a guess. (No pinned native uses+ # this today — a hit here means a runtime class drifted from its pinned blueprint.)+ return ConceptStructureBlueprint(+ description=description, type=ConceptStructureBlueprintFieldType.DICT, key_type="text", value_type="Any", required=required+ )++ raise _UnmappableAnnotationError+++def _list_blueprint(list_annotation: Any, *, description: str, required: bool) -> ConceptStructureBlueprint:+ args = get_args(list_annotation)+ if not args:+ raise _UnmappableAnnotationError+ item_inner, _ = _strip_optional(args[0])++ native_code = _native_code_for_content_class(item_inner)+ if native_code is not None:+ return ConceptStructureBlueprint(+ description=description,+ type=ConceptStructureBlueprintFieldType.LIST,+ item_type="concept",+ item_concept_ref=native_code.concept_ref,+ required=required,+ )+ scalar_type = _scalar_field_type(item_inner)+ if scalar_type is not None:+ return ConceptStructureBlueprint(+ description=description, type=ConceptStructureBlueprintFieldType.LIST, item_type=scalar_type, required=required+ )+ raise _UnmappableAnnotationError+++def _strip_optional(annotation: Any) -> tuple[Any, bool]:+ """Return (inner, required): peel a single `X | None` into (X, False); otherwise (annotation, True)."""+ if get_origin(annotation) in {Union, UnionType}:+ args = get_args(annotation)+ non_none = [arg for arg in args if arg is not type(None)]+ if len(non_none) == 1 and len(non_none) != len(args):+ return non_none[0], False+ return annotation, True+++def _is_number_union(annotation: Any) -> bool:+ if get_origin(annotation) not in {Union, UnionType}:+ return False+ return set(get_args(annotation)) == {int, float}+++def _scalar_field_type(annotation: Any) -> ConceptStructureBlueprintFieldType | None:+ # Order matters: bool is an int subclass, datetime is a date subclass — check the narrower first.+ if annotation is bool:+ return ConceptStructureBlueprintFieldType.BOOLEAN+ if annotation is int:+ return ConceptStructureBlueprintFieldType.INTEGER+ if annotation is float:+ return ConceptStructureBlueprintFieldType.NUMBER+ if annotation is str:+ return ConceptStructureBlueprintFieldType.TEXT+ if annotation is datetime.datetime:+ return ConceptStructureBlueprintFieldType.DATETIME+ if annotation is datetime.date:+ return ConceptStructureBlueprintFieldType.DATE+ if annotation is datetime.time:+ return ConceptStructureBlueprintFieldType.TIME+ return None+++def _native_code_for_content_class(annotation: Any) -> NativeConceptCode | None:+ if not isinstance(annotation, type) or not issubclass(annotation, StuffContent):+ return None+ return _NATIVE_CLASS_NAME_TO_CODE.get(annotation.__name__)+++def _is_nested_model(annotation: Any) -> bool:+ """A pydantic model that is not itself a native content class — an object on the wire."""+ return isinstance(annotation, type) and issubclass(annotation, BaseModel)diff --git a/pipelex/codegen/resolved_concepts.py b/pipelex/codegen/resolved_concepts.pynew file mode 100644index 000000000..a9da926fa--- /dev/null+++ b/pipelex/codegen/resolved_concepts.py@@ -0,0 +1,151 @@+"""Neutral resolved-concept layer over a normalized library crate.++`resolve_concepts_from_crate` turns each concept in a normalized `LibraryCrate` into a+`ResolvedConcept` — the language-agnostic unit every types emitter (python-structures,+python-pydantic, ts-zod) consumes. Class naming (bare vs domain-qualified), the refinement base,+native flagging, structureless/imprecision detection, and field resolution (via the shared+`resolved_fields` layer) all live here, so no emitter re-derives them.++The crate is the single authority: this layer reads only the normalized crate — never the loader,+the class registry, or the raw bundles.+"""++from collections import Counter++from pydantic import BaseModel, ConfigDict++from pipelex.codegen.resolved_fields import ResolvedField, resolve_structure_fields+from pipelex.core.concepts.concept_blueprint import ConceptBlueprint+from pipelex.core.concepts.helpers import normalize_structure_blueprint+from pipelex.core.concepts.native.concept_native import NativeConceptCode+from pipelex.core.qualified_ref import QualifiedRef+from pipelex.libraries.library_crate import LibraryCrate+++class ResolvedConcept(BaseModel):+ """One crate concept resolved into the neutral shape a types emitter renders.++ Exactly one of these shapes holds: `fields` non-empty (a structured concept), `base_ref` set (a+ concept that still refines a structureless / cross-package base), or `structureless` (an opaque+ concept — string-described, Python-class-backed, or genuinely shape-less). `imprecision_reason`+ is set whenever the concept is opaque so the emitter surfaces a caveat instead of guessing a shape.+ """++ model_config = ConfigDict(frozen=True)++ concept_ref: str+ """Qualified ref: `domain.Code` or `native.Code`."""++ domain: str+ code: str+ """The local concept code (PascalCase), the seed of every target's type name."""++ description: str+ is_native: bool+ needs_qualification: bool+ """True when this code collides across domains, so every target must domain-qualify its type name."""++ base_ref: str | None+ """A refinement base still present in the normalized crate (qualified), else None."""++ fields: list[ResolvedField]+ structureless: bool+ imprecision_reason: str | None+ opaque_python_class: str | None+ """The Python class name for a `structure = "<ClassName>"` concept — opaque to a portable crate (B1-1 floor)."""+++class ResolvedLibrary(BaseModel):+ """Every concept of one normalized crate, resolved and ordered by ref — the emitter input unit."""++ model_config = ConfigDict(frozen=True)++ mthds_version: str+ concepts: list[ResolvedConcept]++ def by_ref(self) -> dict[str, ResolvedConcept]:+ """Index the resolved concepts by their qualified ref (for base / field-reference lookups)."""+ return {concept.concept_ref: concept for concept in self.concepts}+++def resolve_concepts_from_crate(crate: LibraryCrate) -> ResolvedLibrary:+ """Resolve every concept of a normalized crate into the neutral emitter input, sorted by ref.++ Collision detection is crate-wide: a bare code that appears in more than one domain forces+ domain-qualified naming for *every* concept carrying that code, in every target — so the+ definition and every reference agree on the spelling.+ """+ code_counts = Counter(QualifiedRef.parse(concept_ref).local_code for concept_ref in crate.concepts)+ available_refs = set(crate.concepts)+ concepts = [+ _resolve_concept(+ concept_ref,+ value=value,+ needs_qualification=code_counts[QualifiedRef.parse(concept_ref).local_code] > 1,+ available_refs=available_refs,+ )+ for concept_ref, value in sorted(crate.concepts.items())+ ]+ return ResolvedLibrary(mthds_version=crate.mthds_version, concepts=concepts)+++def _resolve_concept(+ concept_ref: str,+ *,+ value: ConceptBlueprint | str,+ needs_qualification: bool,+ available_refs: set[str],+) -> ResolvedConcept:+ parsed = QualifiedRef.parse(concept_ref)+ domain = parsed.domain_path or ""+ code = parsed.local_code+ is_native = NativeConceptCode.is_native_concept_ref_or_code(concept_ref_or_code=concept_ref)++ description: str+ base_ref: str | None = None+ fields: list[ResolvedField] = []+ structureless: bool+ imprecision_reason: str | None = None+ opaque_python_class: str | None = None++ if isinstance(value, str):+ # A normalized crate promotes string-described concepts to blueprints; handle the raw form+ # defensively so a hand-built crate still resolves.+ description = value+ structureless = True+ imprecision_reason = "concept is described by text only and declares no structure"+ elif isinstance(value.structure, str):+ # `structure = "<ClassName>"`: the shape lives only in hand-written Python, not in MTHDS, so it+ # is opaque to a portable crate. Surface it (never emit the bare class name silently) — B1-1 floor.+ description = value.description+ structureless = True+ opaque_python_class = value.structure+ imprecision_reason = f"concept is backed by the Python class '{value.structure}', which has no MTHDS structure to project"+ elif isinstance(value.structure, dict):+ description = value.description+ structureless = False+ fields = resolve_structure_fields(normalize_structure_blueprint(value.structure), local_domain=domain)+ elif value.refines:+ description = value.description+ base_ref = value.refines+ structureless = not (NativeConceptCode.is_native_concept_ref_or_code(concept_ref_or_code=value.refines) or value.refines in available_refs)+ if structureless:+ imprecision_reason = f"refinement base '{value.refines}' is not available in this crate"+ else:+ description = value.description+ structureless = True+ imprecision_reason = "concept declares no structure"++ return ResolvedConcept(+ concept_ref=concept_ref,+ domain=domain,+ code=code,+ description=description,+ is_native=is_native,+ needs_qualification=needs_qualification,+ base_ref=base_ref,+ fields=fields,+ structureless=structureless,+ imprecision_reason=imprecision_reason,+ opaque_python_class=opaque_python_class,+ )diff --git a/pipelex/codegen/resolved_fields.py b/pipelex/codegen/resolved_fields.pynew file mode 100644index 000000000..f576dfe21--- /dev/null+++ b/pipelex/codegen/resolved_fields.py@@ -0,0 +1,224 @@+"""Neutral, language-agnostic resolved-field layer.++Every codegen emitter (Python structures, plain pydantic, TS/Zod, ...) consumes the same+resolved shape rather than re-deriving the semantic mapping from `ConceptStructureBlueprint`.+This module owns that mapping — the one place that turns an authored structure field into a+resolved type tree:++- inline `choices` -> a literal type,+- primitive field types -> their neutral kind,+- `concept` / list-of-`concept` -> a concept reference (native-flagged, bare refs promoted to the+ local domain), leaving the language-specific class spelling to each emitter,+- `list` / `dict` -> recursive item / value types.++Where the source is imprecise — a `list` with no `item_type`, a `dict` with no resolvable value+type — the resolved type is `ANY` carrying an explicit imprecision marker, so an emitter surfaces+the imprecision (a TODO / caveat) instead of guessing a shape. The layer never emits a language;+it produces neutral fields an emitter walks.+"""++from enum import StrEnum+from typing import Any++from pydantic import BaseModel, ConfigDict++from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprint, ConceptStructureBlueprintFieldType+from pipelex.core.concepts.native.concept_native import NativeConceptCode+from pipelex.core.qualified_ref import QualifiedRef+++class ResolvedTypeKind(StrEnum):+ """The neutral type kinds an emitter maps to its own language."""++ TEXT = "text"+ NUMBER = "number"+ INTEGER = "integer"+ BOOLEAN = "boolean"+ DATE = "date"+ DATETIME = "datetime"+ TIME = "time"+ LITERAL = "literal"+ CONCEPT = "concept"+ LIST = "list"+ DICT = "dict"+ ANY = "any"+++class ResolvedType(BaseModel):+ """A resolved, language-neutral type tree node.++ Only the members relevant to `kind` are populated: `choices` for LITERAL, `concept_ref` /+ `is_native` for CONCEPT, `item` for LIST, `key` / `value` for DICT. `imprecise` (with an+ optional reason) is set on ANY nodes that stand in for a source the author left unspecified.+ """++ model_config = ConfigDict(frozen=True)++ kind: ResolvedTypeKind+ choices: list[str] | None = None+ concept_ref: str | None = None+ is_native: bool = False+ item: "ResolvedType | None" = None+ key: "ResolvedType | None" = None+ value: "ResolvedType | None" = None+ imprecise: bool = False+ imprecision_reason: str | None = None+++class ResolvedField(BaseModel):+ """One resolved structure field: its name, docs, presence, default, and neutral type."""++ model_config = ConfigDict(frozen=True)++ name: str+ description: str+ required: bool+ default_value: Any | None = None+ resolved_type: ResolvedType+++def resolve_structure_fields(+ structure_blueprint: dict[str, ConceptStructureBlueprint],+ *,+ local_domain: str | None = None,+) -> list[ResolvedField]:+ """Resolve every field of a structure blueprint into the neutral form, preserving field order."""+ return [+ resolve_field(field_name, blueprint=field_blueprint, local_domain=local_domain) for field_name, field_blueprint in structure_blueprint.items()+ ]+++def iter_imprecision_reasons(resolved_type: ResolvedType) -> list[str]:+ """Collect every declared-imprecision reason in a resolved-type tree, depth-first.++ Imprecision markers are set (never guessed) wherever the source under-specifies a shape — a+ `list` with no `item_type`, a `concept` field with no ref, an unrecognized nested type name.+ They are write-only until an emitter surfaces them: this walker is how an emitter turns the+ markers on a field's whole type tree into caveats (a `# imprecise:` comment / JSDoc note)+ instead of emitting a silent `Any` / `z.any()`.+ """+ reasons: list[str] = []+ if resolved_type.imprecise and resolved_type.imprecision_reason:+ reasons.append(resolved_type.imprecision_reason)+ for child in (resolved_type.item, resolved_type.key, resolved_type.value):+ if child is not None:+ reasons.extend(iter_imprecision_reasons(child))+ return reasons+++def resolve_field(+ field_name: str,+ *,+ blueprint: ConceptStructureBlueprint,+ local_domain: str | None = None,+) -> ResolvedField:+ """Resolve a single structure-blueprint field into a neutral `ResolvedField`."""+ return ResolvedField(+ name=field_name,+ description=blueprint.description,+ required=blueprint.required,+ default_value=blueprint.default_value,+ resolved_type=_resolve_type(blueprint, local_domain=local_domain),+ )+++def _resolve_type(blueprint: ConceptStructureBlueprint, *, local_domain: str | None) -> ResolvedType:+ # Inline choices win over `type` (a typed-choices field is still a literal set).+ if blueprint.choices:+ return ResolvedType(kind=ResolvedTypeKind.LITERAL, choices=blueprint.choices)+ if blueprint.type is None:+ # Validation guarantees `type is None` only alongside choices (handled above); this is a+ # defensive fallback matching the historical "assume text" behavior.+ return ResolvedType(kind=ResolvedTypeKind.TEXT)++ match blueprint.type:+ case ConceptStructureBlueprintFieldType.TEXT:+ return ResolvedType(kind=ResolvedTypeKind.TEXT)+ case ConceptStructureBlueprintFieldType.NUMBER:+ return ResolvedType(kind=ResolvedTypeKind.NUMBER)+ case ConceptStructureBlueprintFieldType.INTEGER:+ return ResolvedType(kind=ResolvedTypeKind.INTEGER)+ case ConceptStructureBlueprintFieldType.BOOLEAN:+ return ResolvedType(kind=ResolvedTypeKind.BOOLEAN)+ case ConceptStructureBlueprintFieldType.DATE:+ return ResolvedType(kind=ResolvedTypeKind.DATE)+ case ConceptStructureBlueprintFieldType.DATETIME:+ return ResolvedType(kind=ResolvedTypeKind.DATETIME)+ case ConceptStructureBlueprintFieldType.TIME:+ return ResolvedType(kind=ResolvedTypeKind.TIME)+ case ConceptStructureBlueprintFieldType.CONCEPT:+ return _resolve_concept(blueprint.concept_ref, local_domain=local_domain)+ case ConceptStructureBlueprintFieldType.LIST:+ return ResolvedType(kind=ResolvedTypeKind.LIST, item=_resolve_list_item(blueprint, local_domain=local_domain))+ case ConceptStructureBlueprintFieldType.DICT:+ return ResolvedType(+ kind=ResolvedTypeKind.DICT,+ key=ResolvedType(kind=ResolvedTypeKind.TEXT),+ value=_resolve_scalar_type_name(blueprint.value_type, local_domain=local_domain, context="dict value"),+ )+++def _resolve_concept(concept_ref: str | None, *, local_domain: str | None) -> ResolvedType:+ if not concept_ref:+ return ResolvedType(kind=ResolvedTypeKind.ANY, imprecise=True, imprecision_reason="concept field without concept_ref")+ if NativeConceptCode.is_native_concept_ref_or_code(concept_ref_or_code=concept_ref):+ return ResolvedType(kind=ResolvedTypeKind.CONCEPT, concept_ref=concept_ref, is_native=True)+ # A bare concept ref (no domain) is local to the current bundle's domain — promote it so the+ # resolved ref is unambiguous for every downstream emitter.+ parsed_ref = QualifiedRef.parse_stripping_cross_package(concept_ref)+ if not parsed_ref.domain_path and local_domain:+ concept_ref = f"{local_domain}.{concept_ref}"+ return ResolvedType(kind=ResolvedTypeKind.CONCEPT, concept_ref=concept_ref, is_native=False)+++def _resolve_list_item(blueprint: ConceptStructureBlueprint, *, local_domain: str | None) -> ResolvedType:+ if blueprint.item_type == "concept" and blueprint.item_concept_ref:+ return _resolve_concept(blueprint.item_concept_ref, local_domain=local_domain)+ return _resolve_scalar_type_name(blueprint.item_type, local_domain=local_domain, context="list item")+++def _resolve_scalar_type_name(type_name: str | None, *, local_domain: str | None, context: str) -> ResolvedType:+ """Resolve a nested type named by a string (a list `item_type` / dict `value_type`).++ These positions carry a `ConceptStructureBlueprintFieldType` value as a string, never a concept+ ref (concept nesting uses the dedicated `concept` / `item_concept_ref` path). An unset or+ unrecognized name is genuine source imprecision, surfaced as ANY rather than guessed.+ """+ if type_name is None or type_name == "Any":+ # "Any" is the explicit unspecified-value sentinel (dict blueprints require a value_type,+ # so native expansion writes "Any" where the values are genuinely untyped).+ return ResolvedType(kind=ResolvedTypeKind.ANY, imprecise=True, imprecision_reason=f"{context} type unspecified")+ try:+ field_type = ConceptStructureBlueprintFieldType(type_name)+ except ValueError:+ return ResolvedType(+ kind=ResolvedTypeKind.ANY, imprecise=True, imprecision_reason=f"{context} type '{type_name}' is not a recognized field type"+ )++ match field_type:+ case (+ ConceptStructureBlueprintFieldType.TEXT+ | ConceptStructureBlueprintFieldType.NUMBER+ | ConceptStructureBlueprintFieldType.INTEGER+ | ConceptStructureBlueprintFieldType.BOOLEAN+ | ConceptStructureBlueprintFieldType.DATE+ | ConceptStructureBlueprintFieldType.DATETIME+ | ConceptStructureBlueprintFieldType.TIME+ ):+ return _resolve_type(ConceptStructureBlueprint(description="", type=field_type), local_domain=local_domain)+ case ConceptStructureBlueprintFieldType.LIST:+ # Nested list without its own item type: the inner item is unspecified.+ return ResolvedType(+ kind=ResolvedTypeKind.LIST,+ item=ResolvedType(kind=ResolvedTypeKind.ANY, imprecise=True, imprecision_reason=f"nested {context} list item type unspecified"),+ )+ case ConceptStructureBlueprintFieldType.DICT:+ return ResolvedType(+ kind=ResolvedTypeKind.DICT,+ key=ResolvedType(kind=ResolvedTypeKind.TEXT),+ value=ResolvedType(kind=ResolvedTypeKind.ANY, imprecise=True, imprecision_reason=f"nested {context} dict value type unspecified"),+ )+ case ConceptStructureBlueprintFieldType.CONCEPT:+ # `concept` in a scalar-type-name position carries no ref (a concept nesting would use the+ # dedicated concept path), so its shape is unspecified.+ return ResolvedType(kind=ResolvedTypeKind.ANY, imprecise=True, imprecision_reason=f"{context} concept type carries no concept_ref")diff --git a/pipelex/codegen/stamp.py b/pipelex/codegen/stamp.pynew file mode 100644index 000000000..cb4ee1b02--- /dev/null+++ b/pipelex/codegen/stamp.py@@ -0,0 +1,207 @@+"""Stamp headers: every generated file self-describes so a lone file can testify about itself.++A stamp is a small, machine-parseable comment block prepended to a generated file (see the Stamp+header format in `docs/specs/pipelex-codegen.md`). It records the source crate fingerprint, the engine+version that produced the file, the projection (`kind` / `target`, plus `pipe_ref` for per-pipe+artifacts), any output-affecting options, and a **content hash of the body below the stamp** — so a+hand edit anywhere under the stamp is detectable without the engine, the network, or the lock.++The stamp hashes the crate fingerprint (the semantic hash), not raw authored bytes: reformatting or+commenting a `.mthds` file never changes a stamp; changing a method's effective type surface always+does. The block is emitted in the target language's comment syntax (`#` for Python, `//` for+TypeScript), delimited by `>>> … >>>` / `<<< … <<<` fences so the offline check can split stamp from+body byte-exactly and recompute the hash.+"""++import hashlib+import json+from pathlib import Path+from typing import cast++from pydantic import BaseModel, ConfigDict++from pipelex.codegen.emitters.target import CodegenKind, CodegenTarget+from pipelex.codegen.exceptions import CodegenStampError++_BEGIN_MARKER = ">>> pipelex-codegen-stamp >>>"+_END_MARKER = "<<< pipelex-codegen-stamp <<<"++_COMMENT_PREFIX_BY_SUFFIX = {".py": "#", ".ts": "//"}++# The file suffixes codegen stamps — the single source of truth (the offline check derives from this).+STAMPABLE_SUFFIXES = frozenset(_COMMENT_PREFIX_BY_SUFFIX)+++class CodegenStamp(BaseModel):+ """The self-describing header of one generated file (everything but the body it protects)."""++ model_config = ConfigDict(frozen=True, extra="forbid")++ crate_fingerprint: str+ """The semantic hash of the crate the file was generated from (the standard's crate `fingerprint`)."""++ engine_version: str+ """The `pipelex` codegen engine version that produced the file."""++ kind: CodegenKind+ """The projection `kind` axis (`types`, `inputs`, …)."""++ target: CodegenTarget+ """The projection `target` axis (the language / idiom flavor)."""++ pipe_ref: str | None = None+ """The qualified pipe ref for a per-pipe projection; `None` for a concept-set projection."""++ options: dict[str, str]+ """Output-affecting projection options (empty when none apply)."""++ content_hash: str+ """SHA-256 hex digest of the body below the stamp (recomputed by the offline check)."""+++class ParsedStamp(BaseModel):+ """A stamp parsed back off disk, paired with the body text it protects (below the stamp block)."""++ model_config = ConfigDict(frozen=True)++ stamp: CodegenStamp+ body: str+++def comment_prefix_for(filename: str) -> str:+ """The line-comment prefix for a generated file, by suffix (`.py` → `#`, `.ts` → `//`)."""+ prefix = _COMMENT_PREFIX_BY_SUFFIX.get(Path(filename).suffix)+ if prefix is None:+ msg = f"Cannot stamp '{filename}': no known comment syntax for its file type (expected .py or .ts)."+ raise CodegenStampError(msg)+ return prefix+++def compute_content_hash(body: str) -> str:+ """The canonical content hash of a generated body: lowercase SHA-256 hex over its UTF-8 bytes."""+ return hashlib.sha256(body.encode("utf-8")).hexdigest()+++def _projection_line(*, kind: CodegenKind, target: CodegenTarget, pipe_ref: str | None) -> str:+ parts = [f"{kind} / {target}"]+ if pipe_ref is not None:+ parts.append(pipe_ref)+ return " / ".join(parts)+++def apply_stamp(+ body: str,+ *,+ crate_fingerprint: str,+ engine_version: str,+ kind: CodegenKind,+ target: CodegenTarget,+ pipe_ref: str | None,+ options: dict[str, str],+ comment_prefix: str,+) -> str:+ """Prepend a stamp block to `body`, hashing the body so tampering below the stamp is detectable.++ The returned text is `<stamp block>` + `body`, so a producer that regenerates the same body against+ the same crate and engine writes byte-identical output (the enabler for write-if-changed).+ """+ content_hash = compute_content_hash(body)+ projection = _projection_line(kind=kind, target=target, pipe_ref=pipe_ref)+ fields = {+ "crate_fingerprint": crate_fingerprint,+ "engine_version": engine_version,+ "projection": projection,+ "options": json.dumps(options, sort_keys=True, ensure_ascii=False, separators=(",", ":")),+ "content_hash": content_hash,+ }+ lines = [f"{comment_prefix} {_BEGIN_MARKER}"]+ lines += [f"{comment_prefix} {key}: {value}" for key, value in fields.items()]+ lines.append(f"{comment_prefix} {_END_MARKER}")+ return "\n".join(lines) + "\n" + body+++def parse_stamped(text: str, *, comment_prefix: str) -> ParsedStamp | None:+ """Split a stamped file back into its stamp and the body below it, or `None` if no stamp is present.++ The body is everything after the end-marker line (byte-exact), so recomputing its hash reproduces+ the value the stamp recorded.+ """+ begin_line = f"{comment_prefix} {_BEGIN_MARKER}"+ end_line = f"{comment_prefix} {_END_MARKER}"+ if not text.startswith(begin_line):+ return None+ end_index = text.find(f"\n{end_line}\n")+ if end_index == -1:+ return None+ header_region = text[len(begin_line) + 1 : end_index]+ body = text[end_index + len(end_line) + 2 :]++ fields = _parse_fields(header_region, comment_prefix=comment_prefix)+ projection = fields.get("projection")+ if projection is None:+ return None+ parsed_projection = _parse_projection(projection)+ if parsed_projection is None:+ return None+ kind, target, pipe_ref = parsed_projection++ options_raw = fields.get("options", "{}")+ options = _parse_options(options_raw)+ if options is None:+ return None+ stamp = CodegenStamp(+ crate_fingerprint=fields.get("crate_fingerprint", ""),+ engine_version=fields.get("engine_version", ""),+ kind=kind,+ target=target,+ pipe_ref=pipe_ref,+ options=options,+ content_hash=fields.get("content_hash", ""),+ )+ return ParsedStamp(stamp=stamp, body=body)+++def has_stamp(text: str, *, comment_prefix: str) -> bool:+ """Whether `text` opens with a codegen stamp block for the given comment syntax."""+ return text.startswith(f"{comment_prefix} {_BEGIN_MARKER}")+++def _parse_fields(header_region: str, *, comment_prefix: str) -> dict[str, str]:+ fields: dict[str, str] = {}+ for raw_line in header_region.splitlines():+ stripped = raw_line[len(comment_prefix) :].strip() if raw_line.startswith(comment_prefix) else raw_line.strip()+ key, separator, value = stripped.partition(":")+ if separator:+ fields[key.strip()] = value.strip()+ return fields+++def _parse_projection(projection: str) -> tuple[CodegenKind, CodegenTarget, str | None] | None:+ parts = [segment.strip() for segment in projection.split("/")]+ if len(parts) < 2:+ return None+ kind = _kind_from_value(parts[0])+ target = _target_from_value(parts[1])+ if kind is None or target is None:+ return None+ pipe_ref = parts[2] if len(parts) > 2 else None+ return kind, target, pipe_ref+++def _kind_from_value(value: str) -> CodegenKind | None:+ return next((member for member in CodegenKind if member == value), None)+++def _target_from_value(value: str) -> CodegenTarget | None:+ return next((member for member in CodegenTarget if member == value), None)+++def _parse_options(options_raw: str) -> dict[str, str] | None:+ try:+ loaded = json.loads(options_raw)+ except json.JSONDecodeError:+ return None+ if not isinstance(loaded, dict):+ return None+ typed = cast("dict[str, object]", loaded)+ return {str(key): str(value) for key, value in typed.items()}diff --git a/pipelex/cogt/content_generation/dry_mock.py b/pipelex/cogt/content_generation/dry_mock.pyindex 1c3834603..1f664b070 100644--- a/pipelex/cogt/content_generation/dry_mock.py+++ b/pipelex/cogt/content_generation/dry_mock.py@@ -53,7 +53,6 @@ from pipelex.cogt.content_generation.cogt_run_params import CogtRunParamsfrom pipelex.cogt.content_generation.dry_run_factory import DryRunFactoryfrom pipelex.cogt.content_generation.exceptions import DryRunMockBuildErrorfrom pipelex.cogt.content_generation.schema_to_model_factory import SchemaToModelFactory-from pipelex.cogt.image.image_size import ImageSizefrom pipelex.cogt.llm.llm_job import LLMJobfrom pipelex.cogt.llm.llm_job_components import LLMJobConfig, LLMJobReportfrom pipelex.cogt.llm.llm_prompt import LLMPrompt@@ -345,7 +344,8 @@ def _dry_image_content(image_url: str, *, img_gen_assignment: ImgGenAssignment |url=image_url,public_url=image_url,mime_type="image/jpeg",- size=ImageSize(width=1024, height=1024),+ width=1024,+ height=1024,)if img_gen_assignment:image_content.source_prompt = img_gen_assignment.img_gen_prompt.positive_textdiff --git a/pipelex/cogt/content_generation/generated_content_factory.py b/pipelex/cogt/content_generation/generated_content_factory.pyindex cb28c8c83..d530bbee1 100644--- a/pipelex/cogt/content_generation/generated_content_factory.py+++ b/pipelex/cogt/content_generation/generated_content_factory.py@@ -153,7 +153,8 @@ class GeneratedContentFactory:return ImageContent(url=url,public_url=public_url,- size=raw_details.size,+ width=raw_details.size.width if raw_details.size else None,+ height=raw_details.size.height if raw_details.size else None,mime_type=mime_type,caption=raw_details.caption,)diff --git a/pipelex/core/concepts/concept.py b/pipelex/core/concepts/concept.pyindex a8c3d0f69..31f5f448c 100644--- a/pipelex/core/concepts/concept.py+++ b/pipelex/core/concepts/concept.py@@ -196,12 +196,15 @@ class Concept(ConceptAbstract):*,output_format: ConceptRepresentationFormat,is_multiple: bool = False,+ class_name_overrides: dict[str, str] | None = None,) -> tuple[dict[str, Any], set[str]]:"""Render a representation for this concept.Args:output_format: The format to generate (JSON, PYTHON, or SCHEMA)is_multiple: If True, wrap content in a list/array schema+ class_name_overrides: Optional runtime-class-name -> rendered-name mapping applied to+ Python instantiation code and imports (see ConceptRepresentationGenerator)Returns:Tuple of (representation dict, imports_needed set)@@ -213,7 +216,7 @@ class Concept(ConceptAbstract):case ConceptRepresentationFormat.SCHEMA:return self._render_schema_representation(is_multiple=is_multiple)case ConceptRepresentationFormat.JSON | ConceptRepresentationFormat.PYTHON:- generator = ConceptRepresentationGenerator(output_format)+ generator = ConceptRepresentationGenerator(output_format, class_name_overrides=class_name_overrides)# For inputs, we only want required fields (not optional ones)result = generator.generate_representation(self.concept_ref, structure_class=self.get_structure_class(), include_optional=False)diff --git a/pipelex/core/concepts/concept_blueprint.py b/pipelex/core/concepts/concept_blueprint.pyindex 71632d722..a67efda34 100644--- a/pipelex/core/concepts/concept_blueprint.py+++ b/pipelex/core/concepts/concept_blueprint.py@@ -1,3 +1,4 @@+import keywordfrom typing import Any, castfrom pydantic import BaseModel, ConfigDict, field_validator, model_validator@@ -47,6 +48,15 @@ class ConceptBlueprint(BaseModel):cls, structure: str | dict[str, ConceptStructureBlueprintType] | None) -> str | dict[str, ConceptStructureBlueprintType] | None:if isinstance(structure, dict):+ invalid_identifier_fields = [field_name for field_name in structure if not field_name.isidentifier() or keyword.iskeyword(field_name)]+ if invalid_identifier_fields:+ invalid_list = ", ".join(f"'{name}'" for name in sorted(invalid_identifier_fields))+ msg = (+ "Concept structure field names must be valid Python identifiers and cannot be Python keywords. "+ f"Problematic fields: {invalid_list}."+ )+ raise ValueError(msg)+# Check for reserved field namesreserved_fields_used = [field_name for field_name in structure if field_name in RESERVED_FIELD_NAMES]if reserved_fields_used:diff --git a/pipelex/core/concepts/concept_factory.py b/pipelex/core/concepts/concept_factory.pyindex a4f73d1cd..63b8bf3f1 100644--- a/pipelex/core/concepts/concept_factory.py+++ b/pipelex/core/concepts/concept_factory.py@@ -161,6 +161,13 @@ class ConceptFactory:description="A calendar date, optionally with a time of day — as precise as its source states.",structure_class_name=structure_class_name,)+ case NativeConceptCode.TIME:+ return Concept(+ code=native_concept_code,+ domain_code=SpecialDomain.NATIVE,+ description="A time of day, optionally with a UTC offset — as precise as its source states.",+ structure_class_name=structure_class_name,+ )case NativeConceptCode.PAGE:return Concept(code=native_concept_code,diff --git a/pipelex/core/concepts/concept_representation_generator.py b/pipelex/core/concepts/concept_representation_generator.pyindex 361cb0ad6..86e983d3d 100644--- a/pipelex/core/concepts/concept_representation_generator.py+++ b/pipelex/core/concepts/concept_representation_generator.py@@ -9,7 +9,6 @@ import inspectimport randomimport typesimport typing-import uuidfrom enum import StrEnumfrom typing import Any, Union, cast, get_args, get_origin@@ -39,8 +38,19 @@ class ConceptRepresentationGenerator:{"concept": "domain.ConceptCode", "content": "MyClass(field1='value1', ...)"}"""- def __init__(self, output_format: ConceptRepresentationFormat):+ def __init__(self, output_format: ConceptRepresentationFormat, *, class_name_overrides: dict[str, str] | None = None):+ """Initialize the generator.++ Args:+ output_format: The representation format to generate.+ class_name_overrides: Optional runtime-class-name -> rendered-name mapping. Python+ representations spell class names as the runtime classes are named; a caller that+ pairs the representation with codegen-emitted classes (whose names may differ, e.g.+ bare-when-unique vs domain-qualified) passes the mapping so instantiation code and+ ``imports_needed`` use the emitted spellings.+ """self.output_format = output_format+ self._class_name_overrides = class_name_overrides or {}self._imports_needed: set[str] = set()@property@@ -91,7 +101,7 @@ class ConceptRepresentationGenerator:Returns:Dict (JSON) or string (Python) representing the class"""- class_name = content_class.__name__+ class_name = self._class_name_overrides.get(content_class.__name__, content_class.__name__)self._imports_needed.add(class_name)fields_dict = self._generate_fields_dict(content_class, include_optional=include_optional)@@ -273,7 +283,7 @@ class ConceptRepresentationGenerator:Returns:Dict (JSON) or string (Python)"""- class_name = model_class.__name__+ class_name = self._class_name_overrides.get(model_class.__name__, model_class.__name__)self._imports_needed.add(class_name)fields_dict: dict[str, Any] = {}@@ -303,7 +313,10 @@ class ConceptRepresentationGenerator:"""if actual_type is str:if field_name == "url" or field_name.endswith("_url"):- return f"https://mock-{uuid.uuid4().hex[:8]}.invalid/{uuid.uuid4()}"+ # Deterministic on purpose: these placeholders land in committed inputs templates+ # (`codegen inputs` / `build inputs`), where a random URL would churn on every regen.+ # `.invalid` is a reserved TLD, so the URL can never resolve.+ return f"https://mock.invalid/{field_name}"return f"{field_name}_value"elif actual_type is int:return 0diff --git a/pipelex/core/concepts/concept_structure_blueprint.py b/pipelex/core/concepts/concept_structure_blueprint.pyindex 224f456d0..54130b1b8 100644--- a/pipelex/core/concepts/concept_structure_blueprint.py+++ b/pipelex/core/concepts/concept_structure_blueprint.py@@ -1,4 +1,4 @@-from datetime import date, datetime+from datetime import date, datetime, timefrom enum import StrEnumfrom typing import Any, Self@@ -43,6 +43,7 @@ class ConceptStructureBlueprintFieldType(StrEnum):NUMBER = "number"DATE = "date"DATETIME = "datetime"+ TIME = "time"CONCEPT = "concept"@@ -94,6 +95,9 @@ class ConceptStructureBlueprint(BaseModel):if not self.key_type:msg = f"When type is 'dict', key_type must not be empty. Actual key_type: {self.key_type}"raise ValueError(msg)+ if self.key_type not in {"text", "str"}:+ msg = f"When type is 'dict', key_type must be 'text'. Actual key_type: {self.key_type}"+ raise ValueError(msg)if not self.value_type:msg = f"When type is 'dict', value_type must not be empty. Actual value_type: {self.value_type}"raise ValueError(msg)@@ -125,6 +129,7 @@ class ConceptStructureBlueprint(BaseModel):| ConceptStructureBlueprintFieldType.NUMBER| ConceptStructureBlueprintFieldType.DATE| ConceptStructureBlueprintFieldType.DATETIME+ | ConceptStructureBlueprintFieldType.TIME| None):pass@@ -185,6 +190,9 @@ class ConceptStructureBlueprint(BaseModel):case ConceptStructureBlueprintFieldType.DATETIME:if not isinstance(self.default_value, datetime):self._raise_type_mismatch_error("datetime", actual_type_name=type(self.default_value).__name__)+ case ConceptStructureBlueprintFieldType.TIME:+ if not isinstance(self.default_value, time):+ self._raise_type_mismatch_error("time", actual_type_name=type(self.default_value).__name__)case ConceptStructureBlueprintFieldType.CONCEPT:# CONCEPT type cannot have default values, this is already validated in validate_structure_blueprint# This case is here for exhaustivenessdiff --git a/pipelex/core/concepts/native/concept_native.py b/pipelex/core/concepts/native/concept_native.pyindex 338b2f193..2d0967e12 100644--- a/pipelex/core/concepts/native/concept_native.py+++ b/pipelex/core/concepts/native/concept_native.py@@ -16,6 +16,7 @@ from pipelex.core.stuffs.page_content import PageContentfrom pipelex.core.stuffs.search_result_content import SearchResultContentfrom pipelex.core.stuffs.text_and_images_content import TextAndImagesContentfrom pipelex.core.stuffs.text_content import TextContent+from pipelex.core.stuffs.time_content import TimeContentfrom pipelex.core.stuffs.yes_no_content import YesNoContent@@ -29,6 +30,7 @@ class NativeConceptCode(StrEnum):NUMBER = "Number"YES_NO = "YesNo"DATE = "Date"+ TIME = "Time"PAGE = "Page"JSON = "JSON"SEARCH_RESULT = "SearchResult"@@ -55,6 +57,7 @@ class NativeConceptCode(StrEnum):| NativeConceptCode.NUMBER| NativeConceptCode.YES_NO| NativeConceptCode.DATE+ | NativeConceptCode.TIME| NativeConceptCode.PAGE| NativeConceptCode.JSON| NativeConceptCode.SEARCH_RESULT@@ -96,6 +99,8 @@ class NativeConceptCode(StrEnum):return YesNoContentcase NativeConceptCode.DATE:return DateContent+ case NativeConceptCode.TIME:+ return TimeContentcase NativeConceptCode.PAGE:return PageContentcase NativeConceptCode.JSON:@@ -157,6 +162,7 @@ class NativeConceptCode(StrEnum):| NativeConceptCode.NUMBER| NativeConceptCode.YES_NO| NativeConceptCode.DATE+ | NativeConceptCode.TIME| NativeConceptCode.PAGE| NativeConceptCode.ANYTHING| NativeConceptCode.JSON@@ -182,6 +188,7 @@ class NativeConceptCode(StrEnum):| NativeConceptCode.NUMBER| NativeConceptCode.YES_NO| NativeConceptCode.DATE+ | NativeConceptCode.TIME| NativeConceptCode.PAGE| NativeConceptCode.ANYTHING| NativeConceptCode.JSONdiff --git a/pipelex/core/concepts/native/pinned_blueprints.py b/pipelex/core/concepts/native/pinned_blueprints.pynew file mode 100644index 000000000..f9f1c0f3e--- /dev/null+++ b/pipelex/core/concepts/native/pinned_blueprints.py@@ -0,0 +1,190 @@+"""Pinned normative blueprints of the native concepts, per MTHDS standard version.++The MTHDS standard pins every native concept's exact blueprint form — fields, types, and+descriptions — in `mthds/docs/spec/native-concepts.md`. This module is the runtime copy of that+pinned set: crate materialization looks definitions up here instead of reflecting over the runtime+content classes, so two independent implementations byte-agree on materialized natives (and+therefore on crate fingerprints). Any edit here is a standard change and must land in the spec+page first; the consistency test in `tests/unit/pipelex/codegen/` proves each runtime content+class still matches its pinned blueprint, so the two can never drift silently.++The set below is the MTHDS 1.0.0 pinned set. Version-keyed lookup can come when a second+standard version exists; today `mthds_version` on the crate records which set was used.+"""++from pipelex.core.concepts.concept_blueprint import ConceptBlueprint, ConceptStructureBlueprintType+from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprint, ConceptStructureBlueprintFieldType+from pipelex.core.concepts.native.concept_native import NativeConceptCode++PINNED_NATIVES_MTHDS_VERSION = "1.0.0"+++def make_pinned_native_blueprint(native_code: NativeConceptCode) -> ConceptBlueprint:+ """Build the pinned normative `ConceptBlueprint` for a native concept (fresh instance per call)."""+ return ConceptBlueprint(+ source=None,+ description=_pinned_description(native_code),+ structure=_pinned_structure(native_code),+ )+++def _pinned_description(native_code: NativeConceptCode) -> str:+ match native_code:+ case NativeConceptCode.DYNAMIC:+ return "A dynamic concept"+ case NativeConceptCode.TEXT:+ return "A text"+ case NativeConceptCode.IMAGE:+ return "An image"+ case NativeConceptCode.DOCUMENT:+ return "A document"+ case NativeConceptCode.HTML:+ return "HTML content"+ case NativeConceptCode.TEXT_AND_IMAGES:+ return "A text and an image"+ case NativeConceptCode.NUMBER:+ return "A number"+ case NativeConceptCode.YES_NO:+ return "The answer to a yes/no question"+ case NativeConceptCode.DATE:+ return "A calendar date, optionally with a time of day — as precise as its source states."+ case NativeConceptCode.TIME:+ return "A time of day, optionally with a UTC offset — as precise as its source states."+ case NativeConceptCode.PAGE:+ return "The content of a page of a document, comprising text and linked images and an optional page view image"+ case NativeConceptCode.JSON:+ return "A JSON object"+ case NativeConceptCode.SEARCH_RESULT:+ return "A search result with answer and sources"+ case NativeConceptCode.ANYTHING:+ return "Anything"+ case NativeConceptCode.COMPOSITE:+ return "A named composition of contents"+++def _pinned_structure(native_code: NativeConceptCode) -> dict[str, ConceptStructureBlueprintType] | None:+ match native_code:+ case NativeConceptCode.DYNAMIC | NativeConceptCode.ANYTHING | NativeConceptCode.COMPOSITE:+ # Structureless by design: their shape is intentionally open, surfaced as such per the+ # crate sufficiency guarantee — never a guessed structure.+ return None+ case NativeConceptCode.TEXT:+ return {+ "text": _text_field("The text", required=True),+ }+ case NativeConceptCode.IMAGE:+ return {+ "url": _text_field("The image URL: a storage URI, an HTTP(S) URL, or a base64 data URL", required=True),+ "public_url": _text_field("The public URL of the image"),+ "source_prompt": _text_field("The source prompt of the image"),+ "source_negative_prompt": _text_field("The source negative prompt of the image"),+ "caption": _text_field("The caption of the image"),+ "mime_type": _text_field("The MIME type of the image"),+ "width": ConceptStructureBlueprint(description="The width of the image, in pixels", type=ConceptStructureBlueprintFieldType.INTEGER),+ "height": ConceptStructureBlueprint(+ description="The height of the image, in pixels", type=ConceptStructureBlueprintFieldType.INTEGER+ ),+ "filename": _text_field("The original filename of the image"),+ }+ case NativeConceptCode.DOCUMENT:+ return {+ "url": _text_field("The document URL: a storage URI, an HTTP(S) URL, or a base64 data URL", required=True),+ "public_url": _text_field("The public HTTPS URL of the document"),+ "mime_type": _text_field("The MIME type of the document"),+ "filename": _text_field("The original filename of the document"),+ "title": _text_field("The title of the document or source"),+ "snippet": _text_field("A text snippet or excerpt from the document"),+ }+ case NativeConceptCode.HTML:+ return {+ "inner_html": _text_field("The inner HTML of the content", required=True),+ "css_class": _text_field("The CSS class of the content", required=True),+ }+ case NativeConceptCode.TEXT_AND_IMAGES:+ return {+ "text": ConceptStructureBlueprint(+ description="A text content",+ type=ConceptStructureBlueprintFieldType.CONCEPT,+ concept_ref=NativeConceptCode.TEXT.concept_ref,+ ),+ "images": ConceptStructureBlueprint(+ description="A list of images that were extracted from the text",+ type=ConceptStructureBlueprintFieldType.LIST,+ item_type="concept",+ item_concept_ref=NativeConceptCode.IMAGE.concept_ref,+ ),+ "raw_html": _text_field("The raw HTML of the fetched page, if requested"),+ }+ case NativeConceptCode.NUMBER:+ return {+ "number": ConceptStructureBlueprint(description="The number", type=ConceptStructureBlueprintFieldType.NUMBER, required=True),+ }+ case NativeConceptCode.YES_NO:+ return {+ "yes_no": ConceptStructureBlueprint(+ description="Whether the answer is yes (true) or no (false).", type=ConceptStructureBlueprintFieldType.BOOLEAN, required=True+ ),+ }+ case NativeConceptCode.DATE:+ return {+ "date": ConceptStructureBlueprint(+ description="The calendar date, in ISO 8601 (e.g. 2026-07-07). Always required.",+ type=ConceptStructureBlueprintFieldType.DATE,+ required=True,+ ),+ "time": ConceptStructureBlueprint(+ description=(+ "The time of day, in ISO 8601 (e.g. 15:40:00, or 15:40:00+02:00 with a UTC offset). "+ "Include it only when the source states a time — never invent a time. "+ "Keep the UTC offset exactly when the source states one."+ ),+ type=ConceptStructureBlueprintFieldType.TIME,+ ),+ }+ case NativeConceptCode.TIME:+ return {+ "time": ConceptStructureBlueprint(+ description="The time of day, in ISO 8601 (e.g. 15:40:00, or 15:40:00+02:00 with a UTC offset).",+ type=ConceptStructureBlueprintFieldType.TIME,+ required=True,+ ),+ }+ case NativeConceptCode.PAGE:+ return {+ "text_and_images": ConceptStructureBlueprint(+ description="The text and images content extracted from the page",+ type=ConceptStructureBlueprintFieldType.CONCEPT,+ concept_ref=NativeConceptCode.TEXT_AND_IMAGES.concept_ref,+ required=True,+ ),+ "page_view": ConceptStructureBlueprint(+ description="The screenshot of the page",+ type=ConceptStructureBlueprintFieldType.CONCEPT,+ concept_ref=NativeConceptCode.IMAGE.concept_ref,+ ),+ }+ case NativeConceptCode.JSON:+ return {+ "json_obj": ConceptStructureBlueprint(+ description="The JSON object",+ type=ConceptStructureBlueprintFieldType.DICT,+ key_type="text",+ value_type="Any",+ required=True,+ ),+ }+ case NativeConceptCode.SEARCH_RESULT:+ return {+ "answer": _text_field("The answer to the search query", required=True),+ "sources": ConceptStructureBlueprint(+ description="The source documents supporting the answer",+ type=ConceptStructureBlueprintFieldType.LIST,+ item_type="concept",+ item_concept_ref=NativeConceptCode.DOCUMENT.concept_ref,+ required=True,+ ),+ }+++def _text_field(description: str, *, required: bool = False) -> ConceptStructureBlueprint:+ return ConceptStructureBlueprint(description=description, type=ConceptStructureBlueprintFieldType.TEXT, required=required)diff --git a/pipelex/core/concepts/structure_generation/generator.py b/pipelex/core/concepts/structure_generation/generator.pyindex 87dfa0dfa..76061e9e4 100644--- a/pipelex/core/concepts/structure_generation/generator.py+++ b/pipelex/core/concepts/structure_generation/generator.py@@ -1,18 +1,18 @@import astimport textwrap-from datetime import date, datetime+from datetime import date, datetime, timefrom enum import Enumfrom typing import Any, List, Literal, Optional, castfrom kajson.class_registry_abstract import ClassRegistryAbstractfrom pydantic import Field-from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprint, ConceptStructureBlueprintFieldType+from pipelex.codegen.resolved_fields import ResolvedField, ResolvedType, ResolvedTypeKind, resolve_structure_fields+from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprintfrom pipelex.core.concepts.helpers import extract_concept_code_from_concept_ref_or_code, make_qualified_structure_class_namefrom pipelex.core.concepts.native.concept_native import NativeConceptCodefrom pipelex.core.concepts.structure_generation.exceptions import ConceptStructureGeneratorError, ConceptStructureValidationError, SyntaxErrorDatafrom pipelex.core.qualified_ref import QualifiedRef-from pipelex.core.stuffs.structured_content import StructuredContentfrom pipelex.core.stuffs.stuff_content import StuffContent# Target line length for emitted code (matches the project's ruff config).@@ -29,36 +29,13 @@ def _get_class_registry() -> ClassRegistryAbstract:return hub.get_class_registry() # type: ignore[no-any-return]-class ConceptClassInfo:- """Information about a concept's structure class for import generation."""-- def __init__(self, class_name: str, module_path: str | None = None):- """Initialize ConceptClassInfo.-- Args:- class_name: The name of the structure class (e.g., "Skill")- module_path: Optional module path where the class is defined (e.g., "myapp.structures.skill").- If None, a forward reference (string) will be used.- """- self.class_name = class_name- self.module_path = module_path--class StructureGenerator:"""Generate Pydantic BaseModel classes from concept structure blueprints."""- # TODO: use StrEnum instead of Enum- def __init__(- self,- concept_ref_to_class_info: dict[str, ConceptClassInfo] | None = None,- local_domain: str | None = None,- ):+ def __init__(self, local_domain: str | None = None):"""Initialize the StructureGenerator.Args:- concept_ref_to_class_info: Optional mapping from concept refs to ConceptClassInfo.- When module_path is provided, proper import statements will be generated.- When module_path is None, forward references (strings) will be used.local_domain: Optional domain to use when resolving bare concept refs(refs without a domain prefix, e.g. "ColorPalette") to theirdomain-qualified structure class name.@@ -69,11 +46,7 @@ class StructureGenerator:"from pipelex.core.stuffs.structured_content import StructuredContent","from pydantic import Field",}- self.enum_definitions: dict[str, dict[str, Any]] = {} # Store enum definitions- self.concept_ref_to_class_info = concept_ref_to_class_info or {}self.local_domain = local_domain- # Track concept classes that need to be mocked during validation (for file generation)- self._concept_classes_to_mock: set[str] = set()def generate_from_structure_blueprint(self,@@ -214,6 +187,8 @@ class StructureGenerator:return f'datetime.fromisoformat("{value.isoformat()}")'if isinstance(value, date):return f'date.fromisoformat("{value.isoformat()}")'+ if isinstance(value, time):+ return f'time.fromisoformat("{value.isoformat()}")'if isinstance(value, list):value_list = cast("list[Any]", value)formatted_items = [self._format_default_value(item) for item in value_list]@@ -300,29 +275,18 @@ class StructureGenerator:raise ConceptStructureGeneratorError(msg)self.imports.add(f"from {cls.__module__} import {cls.__name__}")elif base_class != "StructuredContent":- # Check if we have module info for this custom base class in concept_ref_to_class_info- base_module_path: str | None = None- for class_info in self.concept_ref_to_class_info.values():- if class_info.class_name == base_class and class_info.module_path:- base_module_path = class_info.module_path- break- if base_module_path is None:- # Cross-package refines: base class lives in another already-installed package — recover its module via the class registry.- registered_cls = _get_class_registry().get_class(name=base_class)- if registered_cls is not None and registered_cls.__name__ == base_class and registered_cls.__module__ not in {"__main__", "builtins"}:- base_module_path = registered_cls.__module__- if base_module_path:- self.imports.add(f"from {base_module_path} import {base_class}")+ # Cross-package refines: base class lives in another already-installed package — recover its module via the class registry.+ registered_cls = _get_class_registry().get_class(name=base_class)+ if registered_cls is not None and registered_cls.__name__ == base_class and registered_cls.__module__ not in {"__main__", "builtins"}:+ self.imports.add(f"from {registered_cls.__module__} import {base_class}")# Generate class header with docstring (use class name if no description provided)docstring = description or f"Generated {class_name} class"class_header = f"class {class_name}({base_class}):\n{self._format_class_docstring(docstring)}\n"- # Generate fields- field_definitions: list[str] = []- for field_name, field_blueprint in structure_blueprint.items():- field_code = self._generate_field_from_blueprint(field_name, field_blueprint=field_blueprint)- field_definitions.append(field_code)+ # Generate fields through the neutral resolved-field layer shared by every emitter.+ resolved_fields = resolve_structure_fields(structure_blueprint, local_domain=self.local_domain)+ field_definitions = [self._generate_field_line(resolved_field) for resolved_field in resolved_fields]if not field_definitions:# Empty class - docstring is sufficient, no need for pass@@ -331,316 +295,93 @@ class StructureGenerator:fields_code = "\n".join(field_definitions)return class_header + "\n" + fields_code- def _generate_field_from_blueprint(self, field_name: str, *, field_blueprint: ConceptStructureBlueprint) -> str:- """Generate a field definition from ConceptStructureBlueprint.-- Args:- field_name: Name of the field- field_blueprint: ConceptStructureBlueprint instance-- Returns:- Generated field code-- """- # Determine Python type- if field_blueprint.choices:- # Inline choices - use Literal type- python_type = f"Literal[{', '.join(repr(c) for c in field_blueprint.choices)}]"- else:- # Handle complex types- python_type = self._get_python_type_from_blueprint(field_blueprint)+ def _generate_field_line(self, resolved_field: ResolvedField) -> str:+ """Render one Python field definition from a neutral resolved field."""+ python_type = self._python_type_from_resolved(resolved_field.resolved_type)# Make optional if not required- if not field_blueprint.required:+ if not resolved_field.required:python_type = f"Optional[{python_type}]"- # Generate Field parameters- field_params = [self._format_field_description(field_blueprint.description)]-- if field_blueprint.required:- if field_blueprint.default_value is not None:- field_params.insert(0, f"default={self._format_default_value(field_blueprint.default_value)}")- else:- field_params.insert(0, "...")- elif field_blueprint.default_value is not None:- field_params.insert(0, f"default={self._format_default_value(field_blueprint.default_value)}")+ # Generate Field parameters (default/... first, then description)+ field_params = [self._format_field_description(resolved_field.description)]+ default_value = resolved_field.default_value+ if resolved_field.required and default_value is None:+ field_params.insert(0, "...")+ elif default_value is not None:+ field_params.insert(0, f"default={self._format_default_value(default_value)}")else:field_params.insert(0, "default=None")field_call = f"Field({', '.join(field_params)})"+ return f" {resolved_field.name}: {python_type} = {field_call}"- return f" {field_name}: {python_type} = {field_call}"-- def _get_python_type_from_blueprint(self, field_blueprint: ConceptStructureBlueprint) -> str:- """Convert ConceptStructureBlueprint to Python type annotation.-- Args:- field_blueprint: ConceptStructureBlueprint instance-- Returns:- Python type annotation string-- """- if field_blueprint.type is None:- # This should not happen based on validation, but handle gracefully- return "str"-- # Use match/case for type handling- match field_blueprint.type:- case ConceptStructureBlueprintFieldType.TEXT:+ def _python_type_from_resolved(self, resolved_type: ResolvedType) -> str:+ """Map a neutral resolved type to a Python type annotation, accumulating the imports it needs."""+ match resolved_type.kind:+ case ResolvedTypeKind.TEXT:return "str"- case ConceptStructureBlueprintFieldType.NUMBER:+ case ResolvedTypeKind.NUMBER:return "float"- case ConceptStructureBlueprintFieldType.INTEGER:+ case ResolvedTypeKind.INTEGER:return "int"- case ConceptStructureBlueprintFieldType.BOOLEAN:+ case ResolvedTypeKind.BOOLEAN:return "bool"- case ConceptStructureBlueprintFieldType.DATE:+ case ResolvedTypeKind.DATE:self.imports.add("from datetime import date")return "date"- case ConceptStructureBlueprintFieldType.DATETIME:+ case ResolvedTypeKind.DATETIME:self.imports.add("from datetime import datetime")return "datetime"- case ConceptStructureBlueprintFieldType.CONCEPT:- return self._resolve_concept_ref_to_type(field_blueprint.concept_ref)- case ConceptStructureBlueprintFieldType.LIST:- item_type = field_blueprint.item_type or "Any"- # Handle list of concepts- if item_type == "concept" and field_blueprint.item_concept_ref:- resolved_type = self._resolve_concept_ref_to_type(field_blueprint.item_concept_ref)- return f"List[{resolved_type}]"- # Recursively handle item types if they're FieldType enums- try:- item_type_enum = ConceptStructureBlueprintFieldType(item_type)- match item_type_enum:- case ConceptStructureBlueprintFieldType.DICT:- item_blueprint = ConceptStructureBlueprint(- description="lorem ipsum", type=item_type_enum, key_type="str", value_type="Any"- )- item_type = self._get_python_type_from_blueprint(item_blueprint)- case ConceptStructureBlueprintFieldType.CONCEPT:- # This case is handled above with item_concept_ref- pass- case (- ConceptStructureBlueprintFieldType.TEXT- | ConceptStructureBlueprintFieldType.NUMBER- | ConceptStructureBlueprintFieldType.INTEGER- | ConceptStructureBlueprintFieldType.BOOLEAN- | ConceptStructureBlueprintFieldType.DATE- | ConceptStructureBlueprintFieldType.DATETIME- | ConceptStructureBlueprintFieldType.LIST- ):- # Create a temporary blueprint for the item type- item_blueprint = ConceptStructureBlueprint(description="lorem ipsum", type=item_type_enum)- item_type = self._get_python_type_from_blueprint(item_blueprint)- except ValueError:- # Keep as string if not a known FieldType- pass+ case ResolvedTypeKind.TIME:+ self.imports.add("from datetime import time")+ return "time"+ case ResolvedTypeKind.LITERAL:+ choices = resolved_type.choices or []+ return f"Literal[{', '.join(repr(choice) for choice in choices)}]"+ case ResolvedTypeKind.CONCEPT:+ return self._python_type_for_concept(resolved_type)+ case ResolvedTypeKind.LIST:+ item_type = self._python_type_from_resolved(resolved_type.item) if resolved_type.item else "Any"return f"List[{item_type}]"- case ConceptStructureBlueprintFieldType.DICT:- key_type = "str"- value_type = field_blueprint.value_type or "Any"- try:- value_type_enum = ConceptStructureBlueprintFieldType(value_type)- item_blueprint = ConceptStructureBlueprint(description="lorem ipsum", type=value_type_enum)- value_type = self._get_python_type_from_blueprint(item_blueprint)- except ValueError:- pass+ case ResolvedTypeKind.DICT:+ key_type = self._python_type_from_resolved(resolved_type.key) if resolved_type.key else "str"+ value_type = self._python_type_from_resolved(resolved_type.value) if resolved_type.value else "Any"return f"Dict[{key_type}, {value_type}]"+ case ResolvedTypeKind.ANY:+ return "Any"- def _resolve_concept_ref_to_type(self, concept_ref: str | None) -> str:- """Resolve a concept reference to a Python type annotation.-- Args:- concept_ref: The concept reference (e.g., "myapp.Customer" or "native.Html")+ def _python_type_for_concept(self, resolved_type: ResolvedType) -> str:+ """Resolve a CONCEPT node to a Python type annotation.- Returns:- The Python type annotation (structure class name).- - For native concepts: generates import and returns class name (e.g., HtmlContent)- - If module_path is available in concept_ref_to_class_info: generates an import,- tracks for mocking, returns class name directly (for file generation)- - Otherwise: returns a forward reference string (for runtime - resolved via model_rebuild)+ The concept ref is already normalized by the resolved-field layer (natives flagged, bare+ refs promoted to the local domain). Emits:+ - for native concepts: an import of the native content class and its class name;+ - otherwise: a domain-qualified forward reference (resolved via model_rebuild at runtime)."""+ concept_ref = resolved_type.concept_refif not concept_ref:return "Any"- # Handle native concepts (e.g., "native.Html" -> HtmlContent)- if NativeConceptCode.is_native_concept_ref_or_code(concept_ref):- # Extract the concept code (e.g., "Html" from "native.Html")- concept_code = extract_concept_code_from_concept_ref_or_code(concept_ref)+ if resolved_type.is_native:+ native_concept_code = extract_concept_code_from_concept_ref_or_code(concept_ref)try:- native_code = NativeConceptCode(concept_code)+ native_code = NativeConceptCode(native_concept_code)+ except ValueError:+ native_code = None+ if native_code is not None:structure_class = native_code.structure_classif structure_class:self.imports.add(f"from {structure_class.__module__} import {structure_class.__name__}")return structure_class.__name__- except ValueError:- pass- # Bare concept_refs (e.g. "ColorPalette") in field defs are local to the- # current bundle's domain. Promote them to a qualified ref so the lookup- # below — and the qualified-name fallback after it — find the right class.- parsed_ref = QualifiedRef.parse_stripping_cross_package(concept_ref)- if not parsed_ref.domain_path and self.local_domain:- concept_ref = f"{self.local_domain}.{concept_ref}"- parsed_ref = QualifiedRef.parse_stripping_cross_package(concept_ref)-- # Check concept_ref_to_class_info for the concept reference- if concept_ref in self.concept_ref_to_class_info:- class_info = self.concept_ref_to_class_info[concept_ref]- if class_info.module_path:- # File generation: generate import and mock during validation- self.imports.add(f"from {class_info.module_path} import {class_info.class_name}")- self._concept_classes_to_mock.add(class_info.class_name)- return class_info.class_name- # No module path: use forward reference (resolved via model_rebuild at runtime)- return f'"{class_info.class_name}"'-- # Default: use domain-qualified forward reference when domain is available- # e.g., "myapp.Customer" -> '"myapp__Customer"'concept_code = extract_concept_code_from_concept_ref_or_code(concept_ref)- if parsed_ref.domain_path and not NativeConceptCode.is_native_concept_ref_or_code(concept_ref):+ parsed_ref = QualifiedRef.parse_stripping_cross_package(concept_ref)+ if parsed_ref.domain_path and not NativeConceptCode.is_native_concept_ref_or_code(concept_ref_or_code=concept_ref):qualified_name = make_qualified_structure_class_name(domain_code=parsed_ref.domain_path, concept_code=concept_code)return f'"{qualified_name}"'return f'"{concept_code}"'- def _generate_field(self, field_name: str, *, field_def: dict[str, Any] | str) -> str:- """Generate a single field definition.-- Args:- field_name: Name of the field- field_def: Field definition (dict or string for simple types)-- Returns:- Generated field code-- """- # Handle simple string definitions (just the definition text)- if isinstance(field_def, str):- field_def = {"type": ConceptStructureBlueprintFieldType.TEXT, "description": field_def}-- field_type = field_def.get("type", ConceptStructureBlueprintFieldType.TEXT)- description = field_def.get("description", f"{field_name} field")- required = field_def.get("required", False)- default_value = field_def.get("default")- choices = field_def.get("choices") # For inline enum-like choices-- # Determine Python type- if choices:- # Inline choices - use Literal type- python_type = f"Literal[{', '.join(repr(c) for c in choices)}]"- else:- # Handle complex types or enum references- python_type = self._get_python_type(field_type, field_def=field_def)-- # Make optional if not required- if not required:- python_type = f"Optional[{python_type}]"-- # Generate Field parameters- field_params = [self._format_field_description(description)]-- if required:- if default_value is not None:- field_params.insert(0, f"default={self._format_default_value(default_value)}")- else:- field_params.insert(0, "...")- elif default_value is not None:- field_params.insert(0, f"default={self._format_default_value(default_value)}")- else:- field_params.insert(0, "default=None")-- field_call = f"Field({', '.join(field_params)})"-- return f" {field_name}: {python_type} = {field_call}"-- def _get_python_type(self, field_type: Any, *, field_def: dict[str, Any]) -> str:- """Convert high-level type to Python type annotation.-- Args:- field_type: High-level type name or FieldType enum- field_def: Complete field definition-- Returns:- Python type annotation string-- """- # Check if it's a reference to a defined enum- if isinstance(field_type, str) and field_type in self.enum_definitions:- return field_type-- # Convert string to FieldType if needed- if isinstance(field_type, str):- try:- field_type_enum = ConceptStructureBlueprintFieldType(field_type)- except ValueError:- # Unknown type, assume it's a custom type or class reference- return field_type- field_type = field_type_enum-- # Use match/case for type handling- # Note: field_type is already a ConceptStructureBlueprintFieldType at this point- # (converted from string above if needed)- if not isinstance(field_type, ConceptStructureBlueprintFieldType):- return str(field_type)-- match field_type:- case ConceptStructureBlueprintFieldType.TEXT:- return "str"- case ConceptStructureBlueprintFieldType.NUMBER:- return "float"- case ConceptStructureBlueprintFieldType.INTEGER:- return "int"- case ConceptStructureBlueprintFieldType.BOOLEAN:- return "bool"- case ConceptStructureBlueprintFieldType.DATE:- self.imports.add("from datetime import date")- return "date"- case ConceptStructureBlueprintFieldType.DATETIME:- self.imports.add("from datetime import datetime")- return "datetime"- case ConceptStructureBlueprintFieldType.CONCEPT:- # Handle concept type - resolve using concept_ref from field_def- concept_ref = field_def.get("concept_ref")- return self._resolve_concept_ref_to_type(concept_ref)- case ConceptStructureBlueprintFieldType.LIST:- item_type = field_def.get("item_type", "Any")- # Handle list of concepts- if item_type == "concept":- item_concept_ref = field_def.get("item_concept_ref")- if item_concept_ref:- resolved_type = self._resolve_concept_ref_to_type(item_concept_ref)- return f"List[{resolved_type}]"- # Check if item_type is an enum reference- if isinstance(item_type, str) and item_type in self.enum_definitions:- return f"List[{item_type}]"- # Recursively handle item types- if isinstance(item_type, str):- try:- item_type_enum = ConceptStructureBlueprintFieldType(item_type)- item_type = self._get_python_type(item_type_enum, field_def={})- except ValueError:- # Keep as string if not a known FieldType- pass- return f"List[{item_type}]"- case ConceptStructureBlueprintFieldType.DICT:- key_type = field_def.get("key_type", "str")- value_type = field_def.get("value_type", "Any")- # Recursively handle key and value types- if isinstance(key_type, str):- try:- key_type_enum = ConceptStructureBlueprintFieldType(key_type)- key_type = self._get_python_type(key_type_enum, field_def={})- except ValueError:- pass- if isinstance(value_type, str):- try:- value_type_enum = ConceptStructureBlueprintFieldType(value_type)- value_type = self._get_python_type(value_type_enum, field_def={})- except ValueError:- pass- return f"Dict[{key_type}, {value_type}]"-def _validate_execution(self,python_code: str,@@ -656,6 +397,7 @@ class StructureGenerator:# Provide these directly as they're used in generated code but not imported from pipelex"date": date,"datetime": datetime,+ "time": time,"Enum": Enum,"Optional": Optional,"List": List, # noqa: UP006@@ -675,20 +417,6 @@ class StructureGenerator:validation_code = python_code- # For file generation: create mock classes for concept references that would fail import- # These classes don't exist yet or their modules aren't in the path- for mock_class_name in self._concept_classes_to_mock:- # Create a mock class that inherits from StructuredContent- mock_class = type(mock_class_name, (StructuredContent,), {})- exec_globals[mock_class_name] = mock_class- # Remove the import line for this class from the validation code- # (the actual generated code will still have it)- lines = validation_code.split("\n")- filtered_lines = [- line for line in lines if not (line.strip().startswith("from ") and f"import {mock_class_name}" in line and "pipelex" not in line)- ]- validation_code = "\n".join(filtered_lines)-# If a custom base class is specified that's not a native class, get it from the registryif base_class_name:if not NativeConceptCode.is_native_structure_class(base_class_name):diff --git a/pipelex/core/memory/input_shaper.py b/pipelex/core/memory/input_shaper.pyindex 5158e03b0..9bb3c3a72 100644--- a/pipelex/core/memory/input_shaper.py+++ b/pipelex/core/memory/input_shaper.py@@ -65,6 +65,7 @@ class InputKind(StrEnum):NUMBER = "number"YES_NO = "yes_no"DATE = "date"+ TIME = "time"IMAGE = "image"DOCUMENT = "document"STRUCTURED = "structured"@@ -76,7 +77,16 @@ class InputKind(StrEnum):match self:case InputKind.STRUCTURED:return True- case InputKind.TEXT | InputKind.NUMBER | InputKind.YES_NO | InputKind.DATE | InputKind.IMAGE | InputKind.DOCUMENT | InputKind.DYNAMIC:+ case (+ InputKind.TEXT+ | InputKind.NUMBER+ | InputKind.YES_NO+ | InputKind.DATE+ | InputKind.TIME+ | InputKind.IMAGE+ | InputKind.DOCUMENT+ | InputKind.DYNAMIC+ ):return False@@ -173,7 +183,16 @@ class InputShaper:name=variable_name,search_domain_codes=search_domain_codes,)- case InputKind.TEXT | InputKind.NUMBER | InputKind.YES_NO | InputKind.DATE | InputKind.IMAGE | InputKind.DOCUMENT | InputKind.STRUCTURED:+ case (+ InputKind.TEXT+ | InputKind.NUMBER+ | InputKind.YES_NO+ | InputKind.DATE+ | InputKind.TIME+ | InputKind.IMAGE+ | InputKind.DOCUMENT+ | InputKind.STRUCTURED+ ):content = cls._shape_with_multiplicity(value,stuff_spec=stuff_spec,@@ -198,6 +217,7 @@ class InputShaper:ordered_natives: list[tuple[NativeConceptCode, InputKind]] = [(NativeConceptCode.YES_NO, InputKind.YES_NO),(NativeConceptCode.DATE, InputKind.DATE),+ (NativeConceptCode.TIME, InputKind.TIME),(NativeConceptCode.NUMBER, InputKind.NUMBER),(NativeConceptCode.IMAGE, InputKind.IMAGE),(NativeConceptCode.DOCUMENT, InputKind.DOCUMENT),@@ -425,6 +445,15 @@ class InputShaper:value=value,)return cls._make_content(concept, value=value, stuff_spec=stuff_spec, variable_name=variable_name)+ case InputKind.TIME:+ if not isinstance(value, (datetime.time, str)):+ raise cls._wrong_kind(+ stuff_spec=stuff_spec,+ variable_name=variable_name,+ expected_kind="an ISO time-of-day string or a time object",+ value=value,+ )+ return cls._make_content(concept, value=value, stuff_spec=stuff_spec, variable_name=variable_name)case InputKind.IMAGE | InputKind.DOCUMENT:canonical: dict[str, Any]if isinstance(value, str):@@ -458,7 +487,7 @@ class InputShaper:cls,concept: Concept,*,- value: dict[str, Any] | str | bool | datetime.date,+ value: dict[str, Any] | str | bool | datetime.date | datetime.time,stuff_spec: StuffSpec,variable_name: str,) -> StuffContent:diff --git a/pipelex/core/pipes/inputs/input_renderer.py b/pipelex/core/pipes/inputs/input_renderer.pyindex 1bf055fd2..fe0a54127 100644--- a/pipelex/core/pipes/inputs/input_renderer.py+++ b/pipelex/core/pipes/inputs/input_renderer.py@@ -147,7 +147,7 @@ def _delighten_entry(entry: dict[str, Any], *, kind: InputKind) -> Any:return entrycase InputKind.STRUCTURED:return entry["content"]- case InputKind.TEXT | InputKind.NUMBER | InputKind.YES_NO | InputKind.DATE | InputKind.IMAGE | InputKind.DOCUMENT:+ case InputKind.TEXT | InputKind.NUMBER | InputKind.YES_NO | InputKind.DATE | InputKind.TIME | InputKind.IMAGE | InputKind.DOCUMENT:return _unwrap_scalar_content(entry["content"], envelope=entry)diff --git a/pipelex/core/registry_models.py b/pipelex/core/registry_models.pyindex 382820321..fbfcc9da6 100644--- a/pipelex/core/registry_models.py+++ b/pipelex/core/registry_models.py@@ -18,6 +18,7 @@ from pipelex.core.stuffs.stuff import Stufffrom pipelex.core.stuffs.stuff_content import StuffContentfrom pipelex.core.stuffs.text_and_images_content import TextAndImagesContentfrom pipelex.core.stuffs.text_content import TextContent+from pipelex.core.stuffs.time_content import TimeContentfrom pipelex.core.stuffs.yes_no_content import YesNoContentfrom pipelex.pipe_controllers.batch.pipe_batch import PipeBatchfrom pipelex.pipe_controllers.batch.pipe_batch_factory import PipeBatchFactory@@ -96,6 +97,7 @@ class CoreRegistryModels(RegistryModels):NumberContent,YesNoContent,DateContent,+ TimeContent,ImageContent,Stuff,StuffContent,diff --git a/pipelex/core/stuffs/document_content.py b/pipelex/core/stuffs/document_content.pyindex 1db4ae3c6..bebfa101f 100644--- a/pipelex/core/stuffs/document_content.py+++ b/pipelex/core/stuffs/document_content.py@@ -13,7 +13,7 @@ from pipelex.tools.uri.uri_resolver import extract_filename_from_uri, resolve_urclass DocumentContent(StuffContent):- url: str = Field(..., description="The document URL: pipelex storage URL, HTTP/HTTPS URL, or base64 data URL")+ url: str = Field(..., description="The document URL: a storage URI, an HTTP(S) URL, or a base64 data URL")public_url: str | None = Field(default=None, description="The public HTTPS URL of the document")mime_type: str | None = Field(default=None, description="The MIME type of the document")diff --git a/pipelex/core/stuffs/image_content.py b/pipelex/core/stuffs/image_content.pyindex d073f9ee1..57f9b070e 100644--- a/pipelex/core/stuffs/image_content.py+++ b/pipelex/core/stuffs/image_content.py@@ -6,7 +6,6 @@ from rich.markdown import Markdownfrom rich.text import Textfrom typing_extensions import override-from pipelex.cogt.image.image_size import ImageSizefrom pipelex.cogt.templating.template_category import TemplateCategoryfrom pipelex.cogt.templating.text_format import TextFormatfrom pipelex.core.stuffs.stuff_content import StuffContent@@ -18,16 +17,25 @@ from pipelex.tools.uri.uri_resolver import describe_uri, extract_filename_from_uclass ImageContent(StuffContent):- url: str = Field(..., description="The image URL: pipelex storage URL, HTTP/HTTPS URL, or base64 data URL")+ url: str = Field(..., description="The image URL: a storage URI, an HTTP(S) URL, or a base64 data URL")public_url: str | None = Field(default=None, description="The public URL of the image")source_prompt: str | None = Field(default=None, description="The source prompt of the image")source_negative_prompt: str | None = Field(default=None, description="The source negative prompt of the image")caption: str | None = Field(default=None, description="The caption of the image")mime_type: str | None = Field(default=None, description="The MIME type of the image")- size: ImageSize | None = Field(default=None, description="The size in pixels (width and height) of the image")+ width: int | None = Field(default=None, gt=0, description="The width of the image, in pixels")+ height: int | None = Field(default=None, gt=0, description="The height of the image, in pixels")filename: str | None = Field(default=None, description="The original filename of the image")+ @model_validator(mode="after")+ def _validate_paired_dimensions(self) -> Self:+ """Pixel dimensions come from measuring the image, which yields both — a lone one is a bug."""+ if (self.width is None) != (self.height is None):+ msg = "An Image carries both width and height, or neither — a lone pixel dimension is not a valid state."+ raise ValueError(msg)+ return self+@model_validator(mode="after")def _auto_populate_filename(self) -> Self:"""Auto-populate filename from url when it is a local file path."""@@ -107,10 +115,10 @@ class ImageContent(StuffContent):group.renderables.append(caption_text)# Size if present- if self.size:+ if self.width is not None and self.height is not None:size_text = Text()size_text.append("Size: ", style="bold")- size_text.append(f"{self.size.width}x{self.size.height}", style="dim")+ size_text.append(f"{self.width}x{self.height}", style="dim")group.renderables.append(size_text)# MIME type if presentdiff --git a/pipelex/core/stuffs/number_content.py b/pipelex/core/stuffs/number_content.pyindex a9ed6f1e5..dc66f6454 100644--- a/pipelex/core/stuffs/number_content.py+++ b/pipelex/core/stuffs/number_content.py@@ -1,12 +1,13 @@import json+from pydantic import Fieldfrom typing_extensions import overridefrom pipelex.core.stuffs.stuff_content import StuffContentclass NumberContent(StuffContent):- number: int | float+ number: int | float = Field(description="The number")@property@overridediff --git a/pipelex/core/stuffs/search_result_content.py b/pipelex/core/stuffs/search_result_content.pyindex c304535e3..949ef5fa8 100644--- a/pipelex/core/stuffs/search_result_content.py+++ b/pipelex/core/stuffs/search_result_content.py@@ -15,8 +15,10 @@ from pipelex.tools.typing.pydantic_utils import empty_list_factory_ofclass SearchResultContent(StuffContent):"""Represents the result of a search query with an answer and list of sources."""- answer: str- sources: list[DocumentContent] = Field(default_factory=empty_list_factory_of(DocumentContent))+ answer: str = Field(description="The answer to the search query")+ sources: list[DocumentContent] = Field(+ default_factory=empty_list_factory_of(DocumentContent), description="The source documents supporting the answer"+ )@property@overridediff --git a/pipelex/core/stuffs/stuff_content_factory.py b/pipelex/core/stuffs/stuff_content_factory.pyindex 5c0ca5d03..313d8a7da 100644--- a/pipelex/core/stuffs/stuff_content_factory.py+++ b/pipelex/core/stuffs/stuff_content_factory.py@@ -7,6 +7,7 @@ from pipelex.core.stuffs.date_content import DateContentfrom pipelex.core.stuffs.exceptions import StuffContentFactoryErrorfrom pipelex.core.stuffs.stuff_content import StuffContentfrom pipelex.core.stuffs.text_content import TextContent+from pipelex.core.stuffs.time_content import TimeContentfrom pipelex.core.stuffs.yes_no_content import YesNoContentfrom pipelex.hub import get_class_registry@@ -14,7 +15,7 @@ from pipelex.hub import get_class_registryclass StuffContentFactory:@classmethoddef make_content_from_value(- cls, stuff_content_subclass: type[StuffContent], *, value: dict[str, Any] | str | bool | datetime.date+ cls, stuff_content_subclass: type[StuffContent], *, value: dict[str, Any] | str | bool | datetime.date | datetime.time) -> StuffContent:# bool must be handled ahead of any future int handling (bool is a subclass of int) and before model_validate,# which rejects a bare bool. Covers native YesNoContent and the subclasses generated for concepts refining YesNo.@@ -26,6 +27,10 @@ class StuffContentFactory:# a dict ({"date","time"}) still rides model_validate below. Covers native Date and concepts refining it.if issubclass(stuff_content_subclass, DateContent) and isinstance(value, (datetime.date, str)):return cls._make_date_content(stuff_content_subclass, value=value)+ # A time object or an ISO time string under a Time-family class builds via the split constructor;+ # a dict ({"time": ...}) still rides model_validate below. Covers native Time and concepts refining it.+ if issubclass(stuff_content_subclass, TimeContent) and isinstance(value, (datetime.time, str)):+ return cls._make_time_content(stuff_content_subclass, value=value)return stuff_content_subclass.model_validate(obj=value)@classmethod@@ -61,7 +66,27 @@ class StuffContentFactory:return parsed.date(), parsed.timetz()@classmethod- def make_stuff_content_from_concept_required(cls, concept: Concept, *, value: dict[str, Any] | str | bool | datetime.date) -> StuffContent:+ def _make_time_content(cls, time_subclass: type[TimeContent], *, value: datetime.time | str) -> TimeContent:+ """Build a TimeContent (or refining subclass) from a time object or a strict ISO time string."""+ if isinstance(value, datetime.time):+ return time_subclass(time=value)+ # Require the extended HH:MM[:SS] form (optionally with fractional seconds and a UTC offset).+ # time.fromisoformat (3.11+) also accepts the basic "HHMMSS" form, which would let an all-digit+ # string slip past TimeContent's own no-number guard — pin extended-only, mirroring Date.+ if not re.fullmatch(r"\d{2}:\d{2}(:\d{2}(\.\d+)?)?([Zz]|[+-]\d{2}(:?\d{2})?)?", value.strip()):+ msg = f"Time input '{value}' is not an extended ISO 8601 time of day (e.g. '15:40:00' or '15:40:00+02:00')."+ raise StuffContentFactoryError(msg)+ try:+ parsed = datetime.time.fromisoformat(value.strip())+ except ValueError as exc:+ msg = f"Time input '{value}' is not an ISO 8601 time of day (e.g. '15:40:00' or '15:40:00+02:00')."+ raise StuffContentFactoryError(msg) from exc+ return time_subclass(time=parsed)++ @classmethod+ def make_stuff_content_from_concept_required(+ cls, concept: Concept, *, value: dict[str, Any] | str | bool | datetime.date | datetime.time+ ) -> StuffContent:"""Create StuffContent from concept code, requiring the concept to be linked to a class in the registry.Raises StuffContentFactoryError if no registry class is found."""diff --git a/pipelex/core/stuffs/stuff_factory.py b/pipelex/core/stuffs/stuff_factory.pyindex 113c4d5a7..92bda09e0 100644--- a/pipelex/core/stuffs/stuff_factory.py+++ b/pipelex/core/stuffs/stuff_factory.py@@ -270,6 +270,7 @@ class StuffFactory:2.1d: {"concept": "YesNo"/"domain.Concept", "content": bool} → YesNoContent (if YesNo-compatible)2.1e: {"concept": "Date"/"domain.Concept", "content": date/datetime obj} → DateContent (if Date-compatible)2.1f: {"concept": "Date"/"domain.Concept", "content": ISO str} → DateContent (if Date-compatible, checked after Text)+ 2.1g: {"concept": "Time"/"domain.Concept", "content": time obj or ISO str} → TimeContent (if Time-compatible)2.2/2.2b: {"concept": "...", "content": list[str]} → ListContent[TextContent]2.3: {"concept": "...", "content": StuffContent} → Use the StuffContent2.4: {"concept": "...", "content": list[StuffContent]} → ListContent[StuffContent]@@ -525,15 +526,23 @@ class StuffFactory:name=name,code=code,)+ # Case 2.1g: a strict-ISO time string under a Time-compatible concept builds a TimeContent.+ time_concept = get_native_concept(native_concept=NativeConceptCode.TIME)+ if concept_library.is_compatible(tested_concept=concept, wanted_concept=time_concept, strict=True):+ return cls.make_stuff(+ concept=concept,+ content=StuffContentFactory.make_stuff_content_from_concept_required(concept=concept, value=content),+ name=name,+ code=code,+ )msg = (f"Trying to create a Stuff '{name}' in the inputs of your pipe, from a dict that should represent a StuffContentOrData "- f"but the concept of name '{concept_ref}' is not compatible with native concept 'native.Text' or 'native.Date'"+ f"but the concept of name '{concept_ref}' is not compatible with native concept 'native.Text', 'native.Date', or 'native.Time'")raise StuffFactoryError(msg)# Case 2.1e: content is a bare date/datetime object (a TOML temporal literal used as envelope content)- # → DateContent for a Date-compatible concept. isinstance(date) covers datetime (its subclass); a bare- # time never reaches here — the inputs loader rejects it, and a time is not a date anyway.+ # → DateContent for a Date-compatible concept. isinstance(date) covers datetime (its subclass).if isinstance(content, datetime.date):date_concept = get_native_concept(native_concept=NativeConceptCode.DATE)if concept_library.is_compatible(tested_concept=concept, wanted_concept=date_concept, strict=True):@@ -549,6 +558,22 @@ class StuffFactory:)raise StuffFactoryError(msg)+ # Case 2.1g: content is a bare time object (a TOML time literal used as envelope content).+ if isinstance(content, datetime.time):+ time_concept = get_native_concept(native_concept=NativeConceptCode.TIME)+ if concept_library.is_compatible(tested_concept=concept, wanted_concept=time_concept, strict=True):+ return cls.make_stuff(+ concept=concept,+ content=StuffContentFactory.make_stuff_content_from_concept_required(concept=concept, value=content),+ name=name,+ code=code,+ )+ msg = (+ f"Trying to create a Stuff '{name}' in the inputs of your pipe, from a dict that should represent a StuffContentOrData "+ f"but the concept of name '{concept_ref}' is not compatible with native concept 'native.Time' (the content is a time)"+ )+ raise StuffFactoryError(msg)+# Case 2.3: content is a StuffContent object (includes both native and StructuredContent)if isinstance(content, StuffContent):if concept.structure_class_name != content.__class__.__name__:diff --git a/pipelex/core/stuffs/text_content.py b/pipelex/core/stuffs/text_content.pyindex 014771cd4..e05abcc3b 100644--- a/pipelex/core/stuffs/text_content.py+++ b/pipelex/core/stuffs/text_content.py@@ -2,6 +2,7 @@ import htmlimport jsonimport re+from pydantic import Fieldfrom rich.markdown import Markdownfrom rich.syntax import Syntaxfrom typing_extensions import override@@ -13,7 +14,7 @@ HTML_PATTERN = re.compile(r"^\s*<(!DOCTYPE|!--|[a-zA-Z])", re.IGNORECASE)class TextContent(StuffContent):- text: str+ text: str = Field(description="The text")@property@overridediff --git a/pipelex/core/stuffs/time_content.py b/pipelex/core/stuffs/time_content.pynew file mode 100644index 000000000..df1e6be29--- /dev/null+++ b/pipelex/core/stuffs/time_content.py@@ -0,0 +1,62 @@+import datetime+import json+from typing import Any++from pydantic import Field, field_validator+from rich.json import JSON+from typing_extensions import override++from pipelex.core.stuffs.stuff_content import StuffContent+from pipelex.tools.misc.pretty import PrettyPrintable+++class TimeContent(StuffContent):+ """A time of day, optionally with a UTC offset — as precise as its source states.++ The ``time`` carries the UTC offset on its ``tzinfo`` when the source states one+ (fidelity, not normalization). A time of day is not an instant: it has no date to+ attach to, so it never converts to a datetime on its own.+ """++ time: datetime.time = Field(description="The time of day, in ISO 8601 (e.g. 15:40:00, or 15:40:00+02:00 with a UTC offset).")++ @field_validator("time", mode="before")+ @classmethod+ def _reject_lax_temporal(cls, value: Any) -> Any:+ # Close pydantic's lax-mode coercion of a bare number into seconds-since-midnight, which+ # would silently produce wrong temporal data — a real ISO time always carries a ':' separator.+ # Raise ValueError so pydantic wraps it into a ValidationError the input/factory path catches.+ if isinstance(value, (int, float)): # bool is an int subclass+ msg = "A Time must be an ISO 8601 string or a time object, never a number (no seconds-since-midnight)."+ raise ValueError(msg) # noqa: TRY004 — must be ValueError so pydantic wraps it into a ValidationError+ if isinstance(value, datetime.datetime):+ msg = "A Time takes a time of day alone, not a datetime; use Date for a date with a time."+ raise ValueError(msg) # noqa: TRY004 — must be ValueError so pydantic wraps it into a ValidationError+ return value++ @property+ @override+ def short_desc(self) -> str:+ return f"a time of day ({self.time.isoformat()})"++ @override+ def rendered_plain(self) -> str:+ return self.time.isoformat()++ @override+ def rendered_html(self) -> str:+ return self.time.isoformat()++ @override+ def rendered_markdown(self, *, level: int = 1, is_pretty: bool = False) -> str:+ return self.time.isoformat()++ @override+ def rendered_json(self) -> str:+ return json.dumps({"time": self.time.isoformat()})++ @override+ def rendered_pretty(self, *, title: str | None = None, depth: int = 0) -> PrettyPrintable:+ # The base renders smart_dump() (a real time object, which stdlib json can't serialize);+ # render the ISO JSON instead so pretty_print of a Time does not crash.+ return JSON(self.rendered_json())diff --git a/pipelex/libraries/concept_reference_validation.py b/pipelex/libraries/concept_reference_validation.pyindex 30fe7d06c..5b847d686 100644--- a/pipelex/libraries/concept_reference_validation.py+++ b/pipelex/libraries/concept_reference_validation.py@@ -138,7 +138,7 @@ def validate_concept_references_in_blueprints(+ f"\nNative concepts: {sorted(native_codes)}.")# ConceptLibraryError now extends LibraryLoadingError, so the structured per-reference items- # ride the existing `except LibraryError` forwarding in _translate_to_validate_bundle_error and+ # ride the existing `except LibraryError` forwarding in translate_to_validate_bundle_error and# surface as categorized items (not a bare residual): pipe-owned refs as `pipe_validation`,# concept-owned refs as `blueprint_validation`.raise ConceptLibraryError(diff --git a/pipelex/libraries/crate_normalization.py b/pipelex/libraries/crate_normalization.pynew file mode 100644index 000000000..0ee2e9e94--- /dev/null+++ b/pipelex/libraries/crate_normalization.py@@ -0,0 +1,327 @@+"""Normalization pass: authored library crate -> normalized library crate.++Given a merged, key-qualified crate (as produced by `LibraryCrateFactory.make_from_blueprints` over a+loaded, validated library), this pass applies the [Library Crate Format] normalization so the result+is closed, canonical, and explicit:++1. merge — already done upstream (keys are `domain.Code` / `domain.code`);+2. fully qualify every *in-body* reference — pipe `inputs` / `output` concept refs, pipe-step / branch+ / outcome / batch pipe refs, concept `refines`, and structure-field `concept_ref` / `item_concept_ref`;+3. flatten refinement — a refining concept adopts its **in-crate structured** base's effective+ structure (the `refines` link is dropped once the structure is materialized). A concept whose+ refinement chain bottoms out at a **native** keeps its `refines` link: the native is materialized+ into `concepts` by step 4, so the base is resolvable in-crate, and keeping the link preserves the+ native content class on crate round-trip (`refines: native.Text` reloads as a `TextContent`+ subclass rather than a bare `StructuredContent` that lost `TextContent`'s specialized rendering);+4. expand natives — every referenced native is materialized into `concepts` as a `native.<Code>` entry+ (see `native_expansion`), transitively, and the crate is stamped with `mthds_version`;+5. materialize string-described concepts — a `Foo = "text"` shorthand becomes a description-only+ `ConceptBlueprint`; string structure fields become explicit `text` fields.++Deferred normalization steps (the spec's [Library Crate Format] "Normalization Pass" lists these; they+are not yet applied here, mirroring the spec's own "Specification Status" callout):++- materialize defaults and multiplicity (spec step 5): multiplicity markers already present on a ref are+ preserved (`_render_ref_with_markers`), but a bare singular ref is not rewritten to an explicit form,+ and default values are not yet materialized;+- cross-package (`alias->…`) references: the referenced dependency content is not folded into this crate+ yet, so those refs are left intact rather than rewritten to a canonical ref that would dangle. A+ single-package multi-bundle closure normalizes to a fully self-contained crate.++[Library Crate Format]: mthds/docs/spec/library-crate.md+"""++from typing import NamedTuple++from pipelex.codegen.native_expansion import collect_native_refs_from_structure, materialize_native_concept+from pipelex.core.bundles.pipelex_bundle_blueprint import PipeBlueprintUnion+from pipelex.core.concepts.concept_blueprint import ConceptBlueprint+from pipelex.core.concepts.concept_factory import ConceptFactory+from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprint+from pipelex.core.concepts.helpers import normalize_structure_blueprint+from pipelex.core.concepts.native.concept_native import NativeConceptCode+from pipelex.core.pipes.variable_multiplicity import MultiplicityParseResult, parse_concept_with_multiplicity+from pipelex.core.qualified_ref import QualifiedRef+from pipelex.libraries.exceptions import CrateNormalizationError+from pipelex.libraries.library_crate import LibraryCrate+from pipelex.pipe_controllers.batch.pipe_batch_blueprint import PipeBatchBlueprint+from pipelex.pipe_controllers.condition.pipe_condition_blueprint import PipeConditionBlueprint+from pipelex.pipe_controllers.condition.special_outcome import SpecialOutcome+from pipelex.pipe_controllers.parallel.pipe_parallel_blueprint import PipeParallelBlueprint+from pipelex.pipe_controllers.sequence.pipe_sequence_blueprint import PipeSequenceBlueprint++_ConceptEntry = ConceptBlueprint | str+++class _RefinementResolution(NamedTuple):+ is_native_backed: bool+ effective_structure: dict[str, ConceptStructureBlueprint] | None+++def normalize_crate(crate: LibraryCrate, *, mthds_version: str) -> LibraryCrate:+ """Produce a normalized library crate from a merged, key-qualified crate.++ The input crate must already be assembled and key-qualified (e.g. from+ `LibraryCrateFactory.make_from_blueprints` over a validated library, per D6). The output carries+ the normalized `fingerprint` (D2 scope) and the `mthds_version` stamp.+ """+ concepts: dict[str, _ConceptEntry] = {+ concept_ref: _normalize_concept(value, owner_domain=_domain_of(concept_ref), source=crate.source_map.get(concept_ref))+ for concept_ref, value in crate.concepts.items()+ }+ _flatten_refinement(concepts)++ pipes: dict[str, PipeBlueprintUnion] = {+ pipe_ref: _normalize_pipe(blueprint, owner_domain=_domain_of(pipe_ref)) for pipe_ref, blueprint in crate.pipes.items()+ }++ _expand_natives(concepts=concepts, pipes=pipes)++ fingerprint = LibraryCrate.compute_normalized_fingerprint(concepts=concepts, pipes=pipes, domains=crate.domains)+ return LibraryCrate(+ mthds_version=mthds_version,+ concepts=concepts,+ pipes=pipes,+ domains=crate.domains,+ source_map=crate.source_map,+ fingerprint=fingerprint,+ )+++def _domain_of(qualified_ref: str) -> str:+ domain_path = QualifiedRef.parse(qualified_ref).domain_path+ if domain_path is None:+ msg = f"Crate key '{qualified_ref}' is not domain-qualified; the normalizer expects a merged, key-qualified crate."+ raise CrateNormalizationError(msg)+ return domain_path+++# --------------------------------------------------------------------------------------------------+# Concepts+# --------------------------------------------------------------------------------------------------+++def _normalize_concept(value: _ConceptEntry, *, owner_domain: str, source: str | None) -> _ConceptEntry:+ # String-described concept -> explicit description-only ConceptBlueprint (structureless).+ if isinstance(value, str):+ return ConceptBlueprint(source=source, description=value)++ updates: dict[str, object] = {}+ if value.refines:+ updates["refines"] = ConceptFactory.make_refine(value.refines, domain_code=owner_domain)+ if isinstance(value.structure, dict):+ normalized_structure = normalize_structure_blueprint(value.structure)+ updates["structure"] = {+ field_name: _qualify_structure_field(field_blueprint, domain=owner_domain) for field_name, field_blueprint in normalized_structure.items()+ }+ return value.model_copy(update=updates) if updates else value+++def _qualify_structure_field(field_blueprint: ConceptStructureBlueprint, *, domain: str) -> ConceptStructureBlueprint:+ field_updates: dict[str, object] = {}+ if field_blueprint.concept_ref:+ field_updates["concept_ref"] = _qualify_concept_ref(field_blueprint.concept_ref, domain=domain)+ if field_blueprint.item_concept_ref:+ field_updates["item_concept_ref"] = _qualify_concept_ref(field_blueprint.item_concept_ref, domain=domain)+ return field_blueprint.model_copy(update=field_updates) if field_updates else field_blueprint+++def _qualify_concept_ref(concept_ref_or_code: str, *, domain: str) -> str:+ # make_concept_ref_with_domain_from_concept_ref_or_code resolves natives to `native.<Code>`,+ # bare refs to `domain.<Code>`, and preserves cross-package `alias->domain.Code` as-is (deferred).+ return ConceptFactory.make_concept_ref_with_domain_from_concept_ref_or_code(domain_code=domain, concept_ref_or_code=concept_ref_or_code)+++def _flatten_refinement(concepts: dict[str, _ConceptEntry]) -> None:+ """Materialize each refining concept's effective structure in place, dropping the `refines` link+ once a non-empty structure is materialized. Two kinds of base keep their `refines` link untouched:++ - a **native-backed** base (the refinement chain bottoms out at a native): the native is+ materialized into `concepts` by native expansion, so the base is resolvable in-crate; keeping+ the link is what preserves the native content class on round-trip (B1-2) — flattening would drop+ `refines: native.Text` and reload the concept as a bare `StructuredContent`, silently losing+ `TextContent`'s specialized rendering;+ - a structureless or cross-package base (nothing to materialize / not resolvable in-crate).+ """+ resolution_cache: dict[str, _RefinementResolution] = {}+ for concept_ref, value in list(concepts.items()):+ if not isinstance(value, ConceptBlueprint) or not value.refines or isinstance(value.structure, dict):+ continue+ resolution = _resolve_refinement(value.refines, concepts=concepts, cache=resolution_cache)+ if resolution.is_native_backed:+ continue+ if resolution.effective_structure:+ concepts[concept_ref] = value.model_copy(update={"refines": None, "structure": dict(resolution.effective_structure)})+++def _resolve_refinement(+ concept_ref: str,+ *,+ concepts: dict[str, _ConceptEntry],+ cache: dict[str, _RefinementResolution],+) -> _RefinementResolution:+ """Resolve native ancestry and effective structure iteratively, caching each visited ref once."""+ if concept_ref in cache:+ return cache[concept_ref]++ path: list[str] = []+ path_index_by_ref: dict[str, int] = {}+ current_ref = concept_ref+ resolution: _RefinementResolution+ while current_ref not in cache:+ if current_ref in path_index_by_ref:+ cycle = path[path_index_by_ref[current_ref] :]+ first_ref = min(cycle)+ first_index = cycle.index(first_ref)+ ordered_cycle = cycle[first_index:] + cycle[:first_index] + [first_ref]+ msg = f"Refinement cycle detected: {' -> '.join(ordered_cycle)}"+ raise CrateNormalizationError(msg)++ path_index_by_ref[current_ref] = len(path)+ path.append(current_ref)++ if NativeConceptCode.is_native_concept_ref_or_code(concept_ref_or_code=current_ref):+ native_ref = NativeConceptCode.get_validated_native_concept_ref(concept_ref_or_code=current_ref)+ resolution = _RefinementResolution(is_native_backed=True, effective_structure=_native_effective_structure(native_ref))+ break++ value = concepts.get(current_ref)+ if not isinstance(value, ConceptBlueprint):+ # Cross-package or unknown base: not resolvable in-crate.+ resolution = _RefinementResolution(is_native_backed=False, effective_structure=None)+ break+ if isinstance(value.structure, dict):+ effective_structure = {field_name: field for field_name, field in value.structure.items() if isinstance(field, ConceptStructureBlueprint)}+ resolution = _RefinementResolution(is_native_backed=False, effective_structure=effective_structure)+ break+ if value.refines:+ current_ref = value.refines+ continue+ resolution = _RefinementResolution(is_native_backed=False, effective_structure=None)+ break+ else:+ resolution = cache[current_ref]++ for visited_ref in reversed(path):+ cache[visited_ref] = resolution+ return resolution+++def _native_effective_structure(native_ref: str) -> dict[str, ConceptStructureBlueprint] | None:+ native_code = NativeConceptCode(QualifiedRef.parse(native_ref).local_code)+ materialized = materialize_native_concept(native_code)+ if not isinstance(materialized.structure, dict):+ return None+ return {field_name: field for field_name, field in materialized.structure.items() if isinstance(field, ConceptStructureBlueprint)}+++# --------------------------------------------------------------------------------------------------+# Pipes+# --------------------------------------------------------------------------------------------------+++def _normalize_pipe(blueprint: PipeBlueprintUnion, *, owner_domain: str) -> PipeBlueprintUnion:+ updates: dict[str, object] = {}+ if blueprint.inputs:+ updates["inputs"] = {name: _qualify_io_ref(ref, domain=owner_domain) for name, ref in blueprint.inputs.items()}+ if blueprint.output:+ updates["output"] = _qualify_io_ref(blueprint.output, domain=owner_domain)++ match blueprint:+ case PipeSequenceBlueprint():+ updates["steps"] = [step.model_copy(update={"pipe": _qualify_pipe_ref(step.pipe, domain=owner_domain)}) for step in blueprint.steps]+ case PipeParallelBlueprint():+ updates["branches"] = [+ branch.model_copy(update={"pipe": _qualify_pipe_ref(branch.pipe, domain=owner_domain)}) for branch in blueprint.branches+ ]+ case PipeConditionBlueprint():+ if blueprint.outcomes:+ updates["outcomes"] = {key: _qualify_pipe_ref(pipe_ref, domain=owner_domain) for key, pipe_ref in blueprint.outcomes.items()}+ if blueprint.default_outcome:+ updates["default_outcome"] = _qualify_pipe_ref(blueprint.default_outcome, domain=owner_domain)+ case PipeBatchBlueprint():+ if blueprint.branch_pipe_code:+ updates["branch_pipe_code"] = _qualify_pipe_ref(blueprint.branch_pipe_code, domain=owner_domain)+ case _:+ pass++ return blueprint.model_copy(update=updates)+++def _qualify_io_ref(io_ref: str, *, domain: str) -> str:+ if QualifiedRef.has_cross_package_prefix(io_ref):+ return io_ref # cross-package deferred+ parsed = parse_concept_with_multiplicity(io_ref)+ qualified = _qualify_concept_ref(parsed.concept_ref_or_code, domain=domain)+ return _render_ref_with_markers(qualified, parsed=parsed)+++def _render_ref_with_markers(concept_ref: str, *, parsed: MultiplicityParseResult) -> str:+ if parsed.multiplicity is True:+ suffix = "[]"+ elif isinstance(parsed.multiplicity, int):+ suffix = f"[{parsed.multiplicity}]"+ else:+ suffix = ""+ return f"{concept_ref}{suffix}{parsed.presence.symbol}"+++def _qualify_pipe_ref(pipe_ref: str, *, domain: str) -> str:+ if QualifiedRef.has_cross_package_prefix(pipe_ref):+ return pipe_ref # cross-package deferred+ if pipe_ref in SpecialOutcome.value_list():+ return pipe_ref+ if "." in pipe_ref:+ return pipe_ref+ return f"{domain}.{pipe_ref}"+++# --------------------------------------------------------------------------------------------------+# Native expansion+# --------------------------------------------------------------------------------------------------+++def _expand_natives(*, concepts: dict[str, _ConceptEntry], pipes: dict[str, PipeBlueprintUnion]) -> None:+ referenced = _collect_referenced_natives(concepts=concepts, pipes=pipes)+ materialized: dict[str, ConceptBlueprint] = {}+ worklist = list(referenced)+ while worklist:+ native_ref = worklist.pop()+ if native_ref in materialized:+ continue+ native_code = NativeConceptCode(QualifiedRef.parse(native_ref).local_code)+ blueprint = materialize_native_concept(native_code)+ materialized[native_ref] = blueprint+ if isinstance(blueprint.structure, dict):+ structure = {field_name: field for field_name, field in blueprint.structure.items() if isinstance(field, ConceptStructureBlueprint)}+ worklist.extend(ref for ref in collect_native_refs_from_structure(structure) if ref not in materialized)+ concepts.update(materialized)+++def _collect_referenced_natives(*, concepts: dict[str, _ConceptEntry], pipes: dict[str, PipeBlueprintUnion]) -> set[str]:+ referenced: set[str] = set()+ for value in concepts.values():+ if not isinstance(value, ConceptBlueprint):+ continue+ if value.refines:+ _add_native_ref(referenced, concept_ref_or_code=value.refines)+ if isinstance(value.structure, dict):+ for field in value.structure.values():+ if isinstance(field, ConceptStructureBlueprint):+ _add_native_ref(referenced, concept_ref_or_code=field.concept_ref)+ _add_native_ref(referenced, concept_ref_or_code=field.item_concept_ref)+ for blueprint in pipes.values():+ for io_ref in [*(blueprint.inputs or {}).values(), blueprint.output]:+ _add_native_ref(referenced, concept_ref_or_code=_io_ref_concept(io_ref))+ return referenced+++def _io_ref_concept(io_ref: str) -> str | None:+ if not io_ref or QualifiedRef.has_cross_package_prefix(io_ref):+ return None+ return parse_concept_with_multiplicity(io_ref).concept_ref_or_code+++def _add_native_ref(referenced: set[str], *, concept_ref_or_code: str | None) -> None:+ if concept_ref_or_code and NativeConceptCode.is_native_concept_ref_or_code(concept_ref_or_code=concept_ref_or_code):+ referenced.add(NativeConceptCode.get_validated_native_concept_ref(concept_ref_or_code=concept_ref_or_code))diff --git a/pipelex/libraries/exceptions.py b/pipelex/libraries/exceptions.pyindex b23349cc6..db94066b1 100644--- a/pipelex/libraries/exceptions.py+++ b/pipelex/libraries/exceptions.py@@ -7,6 +7,12 @@ class LibraryError(PipelexError):_declared_title = "Library error"+class CrateNormalizationError(LibraryError):+ """Raised when a library crate cannot be normalized (e.g. a crate key that is not domain-qualified)."""++ _declared_title = "Crate normalization error"++class LibraryLoadingError(LibraryError):"""Error raised when loading library components fails.diff --git a/pipelex/libraries/library.py b/pipelex/libraries/library.pyindex 6683f6be1..6be577b6b 100644--- a/pipelex/libraries/library.py+++ b/pipelex/libraries/library.py@@ -143,7 +143,7 @@ class Library(BaseModel):# `pipe_validation` item: `pipe_code` is the referencing controller and# `missing_pipe_code` is the dependency that does not resolve, so a machine# consumer can read both without parsing the message. LibraryLoadingError rides- # the existing `except LibraryError` forwarding in _translate_to_validate_bundle_error.+ # the existing `except LibraryError` forwarding in translate_to_validate_bundle_error.dependency_error_data = PipesAndConceptValidationErrorData(error_type=PipeValidationErrorType.UNRESOLVED_PIPE_DEPENDENCY,domain_code=pipe.domain_code,diff --git a/pipelex/libraries/library_crate.py b/pipelex/libraries/library_crate.pyindex 19c6fa152..36d61bd1c 100644--- a/pipelex/libraries/library_crate.py+++ b/pipelex/libraries/library_crate.py@@ -20,6 +20,9 @@ class LibraryCrate(BaseModel):model_config = ConfigDict(extra="forbid")+ mthds_version: str = ""+ """MTHDS standard version the crate was normalized against (empty for a non-normalized transport crate)."""+concepts: dict[str, ConceptBlueprint | str] = Field(default_factory=dict)"""concept_ref (domain.ConceptCode) -> ConceptBlueprint or string description"""@@ -33,7 +36,7 @@ class LibraryCrate(BaseModel):"""concept_ref or pipe_ref -> source file path (for error reporting)"""fingerprint: str = ""- """SHA-256 hex digest of the serialized concepts + pipes content."""+ """SHA-256 hex digest of the crate content (see compute_fingerprint_from_content / compute_normalized_fingerprint)."""@staticmethoddef compute_fingerprint_from_content(@@ -57,3 +60,38 @@ class LibraryCrate(BaseModel):def compute_fingerprint(self) -> str:"""Compute SHA-256 fingerprint from this crate's concepts and pipes."""return self.compute_fingerprint_from_content(concepts=self.concepts, pipes=self.pipes)++ @staticmethod+ def compute_normalized_fingerprint(+ *,+ concepts: dict[str, "ConceptBlueprint | str"],+ pipes: dict[str, PipeBlueprintUnion],+ domains: dict[str, DomainBlueprint],+ ) -> str:+ """Compute the semantic fingerprint of a normalized crate (D2 scope: concepts + pipes + domains).++ The hashed payload is `{concepts, pipes, domains}` with each object's provenance `source`+ removed (a change of file location must not change the digest); `source_map`, `mthds_version`,+ and the `fingerprint` member itself are excluded. Serialization is canonical — keys sorted at+ every level, no inter-token whitespace, non-ASCII as literal UTF-8 — so two producers agree+ byte-for-byte. This matches RFC 8785 (JSON Canonicalization Scheme) for the object / array /+ string / bool / integer payload a crate contains; full JCS number canonicalization is the+ forward contract for the rare float default value.+ """+ concepts_json: dict[str, object] = {}+ for ref, value in sorted(concepts.items()):+ concepts_json[ref] = value if isinstance(value, str) else _strip_source(value.model_dump(mode="json"))+ pipes_json = {ref: _strip_source(blueprint.model_dump(mode="json")) for ref, blueprint in sorted(pipes.items())}+ domains_json = {code: _strip_source(domain.model_dump(mode="json")) for code, domain in sorted(domains.items())}+ payload = {"concepts": concepts_json, "pipes": pipes_json, "domains": domains_json}+ serialized = json.dumps(payload, sort_keys=True, ensure_ascii=False, separators=(",", ":"))+ return hashlib.sha256(serialized.encode("utf-8")).hexdigest()++ def compute_normalized(self) -> str:+ """Compute the normalized (D2-scope) fingerprint from this crate's concepts, pipes, and domains."""+ return self.compute_normalized_fingerprint(concepts=self.concepts, pipes=self.pipes, domains=self.domains)+++def _strip_source(dumped: dict[str, object]) -> dict[str, object]:+ """Drop the provenance `source` member from a dumped blueprint before hashing (it is not semantic)."""+ return {key: value for key, value in dumped.items() if key != "source"}diff --git a/pipelex/libraries/library_crate_factory.py b/pipelex/libraries/library_crate_factory.pyindex 6cf8d16aa..94fdfa847 100644--- a/pipelex/libraries/library_crate_factory.py+++ b/pipelex/libraries/library_crate_factory.py@@ -72,8 +72,9 @@ class LibraryCrateFactory:else:# Fold this file's domain metadata into the established blueprint: an omitted field# defers to whichever same-domain file declared it, so the root's header wins over- # membership-only siblings regardless of load order. `main_pipe` is intentionally not- # merged — it is dropped when the runtime Domain is built, so first-write-wins is inert.+ # membership-only siblings regardless of load order. `main_pipe` is crate metadata+ # too: codegen consumers use it to select the default pipe, even though the runtime+ # Domain currently drops it.existing = domains[domain_code]existing.description = (merge_domain_metadata_field(@@ -92,6 +93,13 @@ class LibraryCrateFactory:incoming=blueprint.system_prompt,show_values_on_conflict=False,)+ existing.main_pipe = merge_domain_metadata_field(+ domain_code=domain_code,+ field_label="main_pipe",+ established=existing.main_pipe,+ incoming=blueprint.main_pipe,+ show_values_on_conflict=True,+ )# Conceptsif blueprint.concept is not None:diff --git a/pipelex/libraries/library_manager.py b/pipelex/libraries/library_manager.pyindex dd11d4265..7bf17e78c 100644--- a/pipelex/libraries/library_manager.py+++ b/pipelex/libraries/library_manager.py@@ -336,95 +336,6 @@ class LibraryManager(LibraryManagerAbstract):log.verbose(f"Loading MTHDS files from: {[str(p) for p in valid_mthds_paths]}")return self._load_mthds_files_into_library(library_id=library_id, valid_mthds_paths=valid_mthds_paths)- @override- def load_libraries_concepts_only(- self,- library_id: str,- *,- library_dirs: list[Path] | None = None,- library_file_paths: list[Path] | None = None,- ) -> list["Concept"]:- """Load only domains and concepts from library directories, skipping pipes.-- This is a lightweight alternative to load_libraries() that only processes- domains and concepts. It does not load pipes, does not perform pipe validation,- and does not run library.validate_library().-- Args:- library_id: The ID of the library to load into- library_dirs: List of directories containing MTHDS files- library_file_paths: List of specific MTHDS file paths to load-- Returns:- List of all concepts that were loaded- """- # Ensure libraries exist for this library_id- if library_id not in self._libraries:- msg = f"Trying to load a library that does not exist: '{library_id}'"- raise LibraryError(msg)-- if not library_dirs:- library_dirs = []-- all_dirs: list[Path] = []- all_mthds_paths: list[Path] = []- all_dirs.extend(library_dirs)- all_mthds_paths.extend(get_pipelex_mthds_files_from_dirs(set(library_dirs)))-- if library_file_paths:- all_mthds_paths.extend(library_file_paths)-- # Combine and deduplicate- seen_absolute_paths: set[str] = set()- valid_mthds_paths: list[Path] = []- for mthds_path in all_mthds_paths:- try:- absolute_path = str(mthds_path.resolve())- except (OSError, RuntimeError):- # For paths that can't be resolved (e.g., in zipped packages), use string representation- absolute_path = str(mthds_path)-- if absolute_path not in seen_absolute_paths:- valid_mthds_paths.append(mthds_path)- seen_absolute_paths.add(absolute_path)-- # Import modules and register in global registries- # Import from user directories - still needed for StructuredContent classes- for library_dir in all_dirs:- # Only import files that contain StructuredContent subclasses (uses AST pre-check)- ClassRegistryUtils.import_modules_in_folder(- folder_path=library_dir,- base_class_names=[StructuredContent.__name__],- force_include_dirs=[Path(builder_pkg.__file__).parent],- )- # NOTE: We skip FuncRegistryUtils.register_funcs_in_folder() since we're not loading pipes-- # Auto-discover and register all StructuredContent classes from sys.modules- num_registered = ClassRegistryUtils.auto_register_all_subclasses(- base_class=StructuredContent,- )- log.debug(f"Auto-registered {num_registered} StructuredContent classes from loaded modules")-- # Load MTHDS files as concepts only (no pipes)- log.debug(f"Loading concepts only from MTHDS files: {[str(p) for p in valid_mthds_paths]}")- library = self.get_library(library_id=library_id)- all_blueprints: list[PipelexBundleBlueprint] = []- for mthds_path in valid_mthds_paths:- # Track loaded path (resolve if possible)- try:- resolved_path = mthds_path.resolve()- except (OSError, RuntimeError):- resolved_path = mthds_path- library.loaded_mthds_paths.append(resolved_path)-- all_blueprints.append(PipelexInterpreter.make_pipelex_bundle_blueprint(bundle_path=mthds_path))-- # Load all sibling files as a single batch so same-domain concept references resolve against- # the merged set regardless of directory iteration order — mirroring load_concepts_only_from_directory.- # A one-file-at-a-time loop would make a valid additive concepts library fail based on file order- # (e.g. a file whose structure references a concept declared in a sibling that loads later).- return self.load_concepts_only_from_blueprints(library_id=library_id, blueprints=all_blueprints)-@overridedef load_from_crate(self, library_id: str, *, crate: LibraryCrate) -> list[PipeAbstract]:"""Load a LibraryCrate into a live Library.@@ -560,68 +471,6 @@ class LibraryManager(LibraryManagerAbstract):# Load from crate (domains, concepts, pipes, validation)return self.load_from_crate(library_id=library_id, crate=crate)- @override- def load_concepts_only_from_blueprints(- self,- library_id: str,- *,- blueprints: list[PipelexBundleBlueprint],- ) -> list["Concept"]:- """Load only domains and concepts from blueprints, skipping pipes.-- This is a lightweight alternative to load_from_blueprints() that only processes- domains and concepts. It does not load pipes, does not perform pipe validation,- and does not run library.validate_library().-- Args:- library_id: The ID of the library to load into- blueprints: List of parsed MTHDS blueprints to load-- Returns:- List of all concepts that were loaded- """- library = self.get_library(library_id=library_id)-- # Load all domains first- all_domains: list[Domain] = []- for blueprint in blueprints:- domain = DomainFactory.make_from_blueprint(- blueprint=DomainBlueprint(- source=blueprint.source,- code=blueprint.domain,- description=blueprint.description or "",- system_prompt=blueprint.system_prompt,- main_pipe=blueprint.main_pipe,- ),- )- all_domains.append(domain)- library.domain_library.add_domains(domains=all_domains)-- # Validate same-domain concept references against the live library before adding this batch's- # concepts — mirrors load_from_blueprints so the lightweight concepts-only path rejects a- # dangling concept_ref / item_concept_ref instead of leaving an invalid schema behind. The- # batch's own concepts come from `blueprints` (not yet in the library); concepts loaded by- # prior batches resolve via already_loaded_concept_refs.- validate_concept_references_in_blueprints(- blueprints=blueprints,- already_loaded_concept_refs=set(library.concept_library.root.keys()),- )-- # Load concepts (forward references resolved after all are loaded)- all_concepts = self._load_concepts_from_blueprints(blueprints)- library.concept_library.add_concepts(concepts=all_concepts)-- # Resolve forward references in dynamically generated structure classes- self._rebuild_models_with_forward_refs(all_concepts)-- # Detect cycles in concept references (A -> B -> A is forbidden)- self._detect_concept_cycles(all_concepts)-- library.validate_domain_library_with_libraries()- library.validate_concept_library_with_libraries()-- return all_concepts-def _load_concepts_from_blueprints(self,blueprints: list[PipelexBundleBlueprint],@@ -689,6 +538,13 @@ class LibraryManager(LibraryManagerAbstract):"""ref_to_entry: dict[str, tuple[str, str, ConceptBlueprint | str]] = {}for concept_ref, concept_blueprint in concepts.items():+ # Native concepts are auto-registered in every library (make_empty_with_native_concepts).+ # A normalized crate materializes them as explicit `native.<Code>` entries so external+ # consumers are self-contained; re-adding them here would collide with the pre-registered+ # natives. Skipping them makes a normalized crate directly loadable (round-trip), and is a+ # no-op for a transport crate (which never carries native entries).+ if NativeConceptCode.is_native_concept_ref_or_code(concept_ref_or_code=concept_ref):+ continueparsed = QualifiedRef.parse_concept_ref(raw=concept_ref)if parsed.domain_path is None:msg = f"Crate concept_ref '{concept_ref}' must be domain-qualified"@@ -836,7 +692,7 @@ class LibraryManager(LibraryManagerAbstract):# contract mismatches, duplicate refs) carry no structured validation payload — unlike# LibraryLoadingError, which must propagate intact so its aggregated errors survive.# Add the batch's file list as context, then let the outer- # _translate_to_validate_bundle_error LibraryError arm shape it into a ValidateBundleError.+ # translate_to_validate_bundle_error LibraryError arm shape it into a ValidateBundleError.msg = f"Could not load blueprints from {[str(pth) for pth in valid_mthds_paths]} because of: {ref_error}"raise type(ref_error)(msg) from ref_errordiff --git a/pipelex/libraries/library_manager_abstract.py b/pipelex/libraries/library_manager_abstract.pyindex ad8d1addd..866477907 100644--- a/pipelex/libraries/library_manager_abstract.py+++ b/pipelex/libraries/library_manager_abstract.py@@ -9,7 +9,6 @@ from pipelex.core.pipes.pipe_abstract import PipeAbstractfrom pipelex.libraries.library_crate import LibraryCrateif TYPE_CHECKING:- from pipelex.core.concepts.concept import Conceptfrom pipelex.libraries.library import Library@@ -97,22 +96,6 @@ class LibraryManagerAbstract(ABC):def load_from_blueprints(self, library_id: str, *, blueprints: list[PipelexBundleBlueprint]) -> list[PipeAbstract]:pass- @abstractmethod- def load_concepts_only_from_blueprints(self, library_id: str, *, blueprints: list[PipelexBundleBlueprint]) -> list["Concept"]:- """Load only domains and concepts from blueprints, skipping pipes.-- This is a lightweight alternative to load_from_blueprints() that only processes- domains and concepts. It does not load pipes, does not perform pipe validation,- and does not run library.validate_library().-- Args:- library_id: The ID of the library to load into- blueprints: List of parsed MTHDS blueprints to load-- Returns:- List of all concepts that were loaded- """-@abstractmethoddef _remove_from_blueprint(self, library_id: str, *, blueprint: PipelexBundleBlueprint) -> None:pass@@ -130,26 +113,3 @@ class LibraryManagerAbstract(ABC):library_file_paths: list[Path] | None = None,) -> list[PipeAbstract]:pass-- @abstractmethod- def load_libraries_concepts_only(- self,- library_id: str,- *,- library_dirs: list[Path] | None = None,- library_file_paths: list[Path] | None = None,- ) -> list["Concept"]:- """Load only domains and concepts from library directories, skipping pipes.-- This is a lightweight alternative to load_libraries() that only processes- domains and concepts. It does not load pipes, does not perform pipe validation,- and does not run library.validate_library().-- Args:- library_id: The ID of the library to load into- library_dirs: List of directories containing MTHDS files- library_file_paths: List of specific MTHDS file paths to load-- Returns:- List of all concepts that were loaded- """diff --git a/pipelex/pipeline/resolve_bundle.py b/pipelex/pipeline/resolve_bundle.pynew file mode 100644index 000000000..a9f9b4343--- /dev/null+++ b/pipelex/pipeline/resolve_bundle.py@@ -0,0 +1,117 @@+"""Resolve in-memory MTHDS contents into a normalized library crate (presentation-free core).++This is the contents-based sibling of the CLI's directory-based crate loader+(`pipelex.cli.commands.crate_loading.load_normalized_crate`): a host that already holds the+bundle contents as strings — an HTTP API, an SDK — resolves them here without touching the+filesystem. Same engine, same verdict vocabulary: an invalid library raises the one shared+``ValidateBundleError`` (carrying the structured ``validation_errors`` items every surface+renders), so a resolve verdict cannot drift from a validate verdict.++Resolution is static (parse → load → ``validate_library()`` → normalize): it deliberately runs+no dry-run sweep, matching `pipelex resolve` — the crate is emitted from a loaded **and+validated** library (D6), and runnability is `validate`'s vocabulary, not resolution's.+"""++from typing import Sequence++from mthds.package.manifest.schema import MTHDS_STANDARD_VERSION++from pipelex.base_exceptions import PipelexUnexpectedError+from pipelex.core.bundles.pipelex_bundle_blueprint import PipelexBundleBlueprint+from pipelex.core.interpreter.interpreter import PipelexInterpreter+from pipelex.core.qualified_ref import QualifiedRef+from pipelex.hub import (+ clear_current_library,+ get_current_library_id_or_none,+ get_library_manager,+ set_current_library,+)+from pipelex.libraries.crate_normalization import normalize_crate+from pipelex.libraries.library_crate import LibraryCrate+from pipelex.libraries.pipe.exceptions import PipeLibraryError+from pipelex.pipeline.validate_bundle import translate_to_validate_bundle_error+++def resolve_crate_from_contents(*, mthds_contents: list[str], mthds_sources: Sequence[str | None] | None = None) -> LibraryCrate:+ """Assemble, validate, and normalize a library crate from in-memory MTHDS contents.++ Mirrors ``validate_bundle``'s in-memory arm and its **loaded-on-success contract**: on+ success the freshly opened library is left loaded and current so the host can read live+ pipes from it (input templates, runner scaffolds) — the host then owns its teardown+ (``clear_current_library()`` + ``library_manager.teardown(library_id=...)``). On failure the+ library is torn down here and the caller's prior current-library is restored.++ Args:+ mthds_contents: MTHDS bundle contents (at least one) forming the closure to resolve.+ mthds_sources: Optional per-content logical sources (e.g. relative file paths), threaded+ onto each blueprint so diagnostics carry a real ``source``; a ``None`` entry leaves+ that content source-less. Must match ``mthds_contents`` in length when provided — a+ mismatch is a host-wiring bug.++ Returns:+ The normalized library crate (fully qualified, natives materialized, fingerprint set).++ Raises:+ ValidateBundleError: the produced negative verdict — the library could not be parsed,+ loaded, or validated; carries the structured validation-error items.+ PipelexUnexpectedError: host-wiring bug (empty contents, sources length mismatch).+ CrateNormalizationError: internal invariant violation in the normalization pass —+ deliberately not translated, it must surface as a server fault.+ """+ if not mthds_contents:+ msg = "mthds_contents must not be empty: provide at least one MTHDS content to resolve"+ raise PipelexUnexpectedError(msg)+ if mthds_sources is not None and len(mthds_sources) != len(mthds_contents):+ msg = "mthds_sources, when provided, must be a per-item source list matching mthds_contents in length"+ raise PipelexUnexpectedError(msg)++ library_manager = get_library_manager()+ library_id, _ = library_manager.open_library()+ success = False+ prev_library_id = get_current_library_id_or_none()+ try:+ set_current_library(library_id=library_id)+ content_sources: list[str | None] = list(mthds_sources) if mthds_sources is not None else [None] * len(mthds_contents)+ with translate_to_validate_bundle_error():+ blueprints = [+ PipelexInterpreter.make_pipelex_bundle_blueprint(mthds_content=content, mthds_source=source)+ for content, source in zip(mthds_contents, content_sources, strict=True)+ ]+ _reject_address_based_dependencies(blueprints=blueprints)+ # load_from_blueprints ends in load_from_crate, whose last step is+ # library.validate_library() — so a crate read after it satisfies the D6+ # "built only from a valid library" contract.+ library_manager.load_from_blueprints(library_id=library_id, blueprints=blueprints)+ crate = library_manager.get_crate(library_id)+ if crate is None:+ # Unreachable with non-empty contents (load_from_blueprints accumulated them), so a+ # None crate is an internal invariant break, not a caller-facing verdict.+ msg = "library crate unavailable after a successful bundle load"+ raise PipelexUnexpectedError(msg)+ normalized = normalize_crate(crate, mthds_version=MTHDS_STANDARD_VERSION)+ success = True+ return normalized+ finally:+ if not success:+ # Restore the caller's outer current-library FIRST so the safety guarantee holds+ # even when teardown raises (same ordering rationale as validate_bundle).+ if prev_library_id is not None:+ set_current_library(library_id=prev_library_id)+ else:+ clear_current_library()+ library_manager.teardown(library_id=library_id)+++def _reject_address_based_dependencies(*, blueprints: list[PipelexBundleBlueprint]) -> None:+ """Keep contents-only resolution independent from installed methods on the host filesystem."""+ for blueprint in blueprints:+ for pipe_ref, _context in blueprint.collect_pipe_references():+ if not QualifiedRef.has_cross_package_prefix(pipe_ref):+ continue+ alias, _remainder = QualifiedRef.split_cross_package_ref(pipe_ref)+ if QualifiedRef.is_address_based_alias(alias):+ msg = (+ f"In-memory resolve cannot load address-based dependency '{alias}' from the host filesystem; "+ "provide the dependency contents in the request or use directory-based resolution."+ )+ raise PipeLibraryError(msg)diff --git a/pipelex/pipeline/validate_bundle.py b/pipelex/pipeline/validate_bundle.pyindex 68d9008f2..b5e56c475 100644--- a/pipelex/pipeline/validate_bundle.py+++ b/pipelex/pipeline/validate_bundle.py@@ -2,18 +2,17 @@ import asynciofrom collections.abc import Generatorfrom contextlib import contextmanagerfrom pathlib import Path-from typing import Literal, Sequence+from typing import Sequencefrom pydantic import BaseModel, ValidationError# TypedDict from typing_extensions, not typing: pydantic rejects typing.TypedDict as a model# field on Python < 3.12, and ValidatedPipeEntry is a field of PipelexValidationReport.-from typing_extensions import TypedDict, assert_never+from typing_extensions import TypedDictfrom pipelex import logfrom pipelex.base_exceptions import PipelexUnexpectedErrorfrom pipelex.core.bundles.pipelex_bundle_blueprint import PipelexBundleBlueprint-from pipelex.core.concepts.concept import Conceptfrom pipelex.core.interpreter.exceptions import PipelexInterpreterErrorfrom pipelex.core.interpreter.interpreter import PipelexInterpreterfrom pipelex.core.pipes.exceptions import PipeFactoryError, PipeValidationError@@ -116,27 +115,16 @@ def _backfill_pipe_error_source(pipe_error: PipeValidationError) -> None:@contextmanager-def _translate_to_validate_bundle_error(category: Literal["pipe", "concept"]) -> Generator[None, None, None]:+def translate_to_validate_bundle_error() -> Generator[None, None, None]:"""Translate the bundle-loading exception surface into a single ``ValidateBundleError``.- Single source of truth for the bundle-loading error cascade, used by all- four entry points: ``validate_bundle`` / ``validate_bundles_from_directory``- (full pipe + dry-run path, ``category="pipe"``) and ``load_concepts_only`` /- ``load_concepts_only_from_directory`` (concepts-only path,- ``category="concept"``). A ``PipelexInterpreterError`` becomes a- ``ValidateBundleError`` carrying the blueprint validation errors, a- ``PipeFactoryError`` carries the categorized factory error, etc. Sharing one- source of truth means a new handler only needs to be added once. The- pipe-loading / dry-run handlers (``PipeFactoryError``, ``PipeValidationError``,- ``PipeRunError``, ``DryRunError``) are dead code in the concepts-only paths- (those functions never instantiate pipes or run dry runs), but they are- harmless there — they simply never fire.-- ``category`` controls the user-facing framing for the one branch that fires- from both paths: the ``except ValidationError`` arm catches pydantic- validation errors raised during model construction. A concepts-only path- that fails ``Concept`` validation must not be framed as a pipe-validation- error — pass ``category="concept"`` to surface concept-aware copy.+ Single source of truth for the bundle-loading error cascade, shared by the+ bundle-loading entry points: ``validate_bundle``, ``validate_bundles_from_directory``,+ and ``pipelex.pipeline.resolve_bundle.resolve_crate_from_contents``.+ A ``PipelexInterpreterError`` becomes a ``ValidateBundleError`` carrying the+ blueprint validation errors, a ``PipeFactoryError`` carries the categorized+ factory error, etc. Sharing one source of truth means a new handler only+ needs to be added once."""try:yield@@ -166,17 +154,7 @@ def _translate_to_validate_bundle_error(category: Literal["pipe", "concept"]) ->except ValidationError as validation_error:pipe_validation_errors = categorize_pipe_validation_error(validation_error=validation_error)validation_error_msg = report_validation_error(category="mthds", validation_error=validation_error)- match category:- case "pipe":- msg = f"Could not load blueprints because of: {validation_error_msg}"- case "concept":- msg = f"Could not load concepts because of: {validation_error_msg}"- case _ as unreachable:- # ``category`` is ``Literal["pipe", "concept"]`` — pyright catches a- # bad call site statically, but the runtime can still receive an- # unexpected value via ``# type: ignore`` or dynamic dispatch. Loud- # ``AssertionError`` beats silent ``UnboundLocalError`` on ``msg``.- assert_never(unreachable)+ msg = f"Could not load blueprints because of: {validation_error_msg}"raise ValidateBundleError(message=msg,pipe_validation_errors=pipe_validation_errors,@@ -299,7 +277,7 @@ async def validate_bundle(loaded_pipes: list[PipeAbstract] | None = Noneloaded_blueprints: list[PipelexBundleBlueprint] | None = Noneawait asyncio.sleep(0) # Yield to event loop (keeps function async-compatible)- with _translate_to_validate_bundle_error(category="pipe"):+ with translate_to_validate_bundle_error():if effective_dirs:log.verbose(f"Loading libraries from {len(effective_dirs)} directory(ies) ({source_label}) for validation")library_manager.load_libraries(@@ -378,7 +356,7 @@ async def validate_bundles_from_directory(directory: Path, *, allow_signatures:prev_library_id = get_current_library_id_or_none()try:set_current_library(library_id=library_id)- with _translate_to_validate_bundle_error(category="pipe"):+ with translate_to_validate_bundle_error():for mthds_file in mthds_files:blueprint = PipelexInterpreter.make_pipelex_bundle_blueprint(bundle_path=mthds_file)all_blueprints.append(blueprint)@@ -402,142 +380,3 @@ async def validate_bundles_from_directory(directory: Path, *, allow_signatures:else:clear_current_library()library_manager.teardown(library_id=library_id)---class LoadConceptsOnlyResult(BaseModel):- """Result of loading MTHDS files with concepts only (no pipes)."""-- blueprints: list[PipelexBundleBlueprint]- concepts: list[Concept]---def load_concepts_only(- mthds_file_path: Path | None = None,- *,- mthds_contents: list[str] | None = None,- library_dirs: Sequence[Path] | None = None,-) -> LoadConceptsOnlyResult:- """Load MTHDS files processing only domains and concepts, skipping pipes.-- This is a lightweight alternative to validate_bundle() that only processes- domains and concepts. It does not load pipes, does not perform pipe validation,- and does not run dry runs.-- Args:- mthds_file_path: Path to a single MTHDS file to load (mutually exclusive with mthds_contents)- mthds_contents: List of MTHDS content strings to load (mutually exclusive with mthds_file_path)- library_dirs: Optional directories containing additional MTHDS library files-- Returns:- LoadConceptsOnlyResult with blueprints and loaded concepts-- Raises:- ValidateBundleError: If loading fails due to interpreter or validation errors- """- provided_params = sum([mthds_contents is not None, mthds_file_path is not None])- if provided_params == 0:- # Programmer error (see validate_bundle's twin guard): the caller must wire exactly one- # input. Not a content verdict → PipelexUnexpectedError (→ 500), never ValidateBundleError.- msg = "At least one of mthds_contents or mthds_file_path must be provided to load_concepts_only"- raise PipelexUnexpectedError(msg)- if provided_params > 1:- msg = "Only one of mthds_contents or mthds_file_path can be provided to load_concepts_only, not both"- raise PipelexUnexpectedError(msg)-- library_manager = get_library_manager()- library_id, library = library_manager.open_library()- success = False- prev_library_id = get_current_library_id_or_none()- try:- set_current_library(library_id=library_id)-- # Load libraries from resolved directories before loading the bundle- effective_dirs, source_label = resolve_library_dirs(library_dirs)-- loaded_concepts: list[Concept] | None = None- loaded_blueprints: list[PipelexBundleBlueprint] | None = None- with _translate_to_validate_bundle_error(category="concept"):- if effective_dirs:- log.verbose(f"Loading concepts only from {len(effective_dirs)} library directory(ies) ({source_label})")- library_manager.load_libraries_concepts_only(- library_id=library_id,- library_dirs=effective_dirs,- )- else:- log.verbose(f"No library directories to load ({source_label})")-- if mthds_contents is not None:- loaded_blueprints = [PipelexInterpreter.make_pipelex_bundle_blueprint(mthds_content=content) for content in mthds_contents]- loaded_concepts = library_manager.load_concepts_only_from_blueprints(library_id=library_id, blueprints=loaded_blueprints)- result = LoadConceptsOnlyResult(blueprints=loaded_blueprints, concepts=loaded_concepts)-- else:- assert mthds_file_path is not None- blueprint = PipelexInterpreter.make_pipelex_bundle_blueprint(bundle_path=mthds_file_path)- loaded_blueprints = [blueprint]-- if mthds_file_path.resolve() not in library.loaded_mthds_paths:- # File not yet loaded - load it from the blueprint- loaded_concepts = library_manager.load_concepts_only_from_blueprints(library_id=library_id, blueprints=[blueprint])- else:- # File already loaded - get existing concepts from library- # For concepts-only loading, we just return empty list since concepts are already in library- loaded_concepts = []-- result = LoadConceptsOnlyResult(blueprints=loaded_blueprints, concepts=loaded_concepts)- success = True- return result- finally:- if not success:- # See ``validate_bundle``: restore the outer current-library before- # ``library_manager.teardown`` so the guarantee survives a teardown raise.- if prev_library_id is not None:- set_current_library(library_id=prev_library_id)- else:- clear_current_library()- library_manager.teardown(library_id=library_id)---def load_concepts_only_from_directory(directory: Path) -> LoadConceptsOnlyResult:- """Load MTHDS files from a directory, processing only domains and concepts, skipping pipes.-- This is a lightweight alternative to validate_bundles_from_directory() that only- processes domains and concepts. It does not load pipes, does not perform pipe- validation, and does not run dry runs.-- Args:- directory: Directory containing MTHDS files to load-- Returns:- LoadConceptsOnlyResult with blueprints and loaded concepts-- Raises:- ValidateBundleError: If loading fails due to interpreter or validation errors- """- mthds_files = get_pipelex_mthds_files_from_dirs(dirs={directory})- all_blueprints: list[PipelexBundleBlueprint] = []-- library_manager = get_library_manager()- library_id, _ = library_manager.open_library()- success = False- prev_library_id = get_current_library_id_or_none()- try:- set_current_library(library_id=library_id)- with _translate_to_validate_bundle_error(category="concept"):- for mthds_file in mthds_files:- blueprint = PipelexInterpreter.make_pipelex_bundle_blueprint(bundle_path=mthds_file)- all_blueprints.append(blueprint)-- loaded_concepts = library_manager.load_concepts_only_from_blueprints(library_id=library_id, blueprints=all_blueprints)- result = LoadConceptsOnlyResult(blueprints=all_blueprints, concepts=loaded_concepts)- success = True- return result- finally:- if not success:- # See ``validate_bundle``: restore the outer current-library before- # ``library_manager.teardown`` so the guarantee survives a teardown raise.- if prev_library_id is not None:- set_current_library(library_id=prev_library_id)- else:- clear_current_library()- library_manager.teardown(library_id=library_id)diff --git a/tests/e2e/pipelex/concepts/nested_concepts/test_structure_generator_cli.py b/tests/e2e/pipelex/concepts/nested_concepts/test_structure_generator_cli.pyindex 6a95acde1..d5683e418 100644--- a/tests/e2e/pipelex/concepts/nested_concepts/test_structure_generator_cli.py+++ b/tests/e2e/pipelex/concepts/nested_concepts/test_structure_generator_cli.py@@ -1,13 +1,14 @@# pyright: reportUnknownMemberType=false# pyright: reportUnknownVariableType=false# pyright: reportUnknownArgumentType=false-"""E2E test for the structure generator CLI with nested concepts.+"""E2E test for structures codegen (the `build structures` engine) with nested concepts.This test verifies that:-1. `pipelex build structures` generates valid Python files for nested concepts-2. The generated files are importable+1. The codegen engine (`codegen types --target python-structures`, which `pipelex build structures`+ aliases) emits a single importable module for nested concepts+2. The generated module is importable as-is (no manual forward-ref plumbing)3. The generated classes can be instantiated and used-4. Forward references between nested concepts are properly handled+4. Concept-to-concept references (nested and list-nested) resolve correctlyNote: pyright checks are disabled for this file because we dynamically importand instantiate generated classes at runtime.@@ -19,147 +20,105 @@ import tempfilefrom pathlib import Pathfrom typing import Any-import pytest+from mthds.package.manifest.schema import MTHDS_STANDARD_VERSION-from pipelex.cli.commands.build.structures_cmd import generate_structures_from_blueprints-from pipelex.pipeline.validate_bundle import validate_bundle+from pipelex.codegen.emission import write_stamped_projection+from pipelex.codegen.emitters.target import CodegenKind, CodegenTarget+from pipelex.codegen.emitters.types_emitter import emit_types+from pipelex.codegen.lock import CODEGEN_LOCK_FILENAME+from pipelex.hub import clear_current_library, get_current_library_id_or_none, get_library_manager, set_current_library+from pipelex.libraries.crate_normalization import normalize_crate+from pipelex.pipeline.execution_seams import load_libraries_and_activateclass TestStructureGeneratorCLI:- """E2E tests for structure generation CLI with nested concepts."""+ """E2E: the structures projection over a real bundle with nested concept references."""- async def test_generate_and_import_nested_concept_structures(self):- """Test that generated structure files for nested concepts are importable and usable.+ def test_generate_and_import_nested_concept_structures(self):+ """The emitted structures.py imports cleanly and its nested classes compose.This test:- 1. Uses the existing nested_concepts.mthds file with concept-to-concept references- 2. Generates Python structure files via the CLI helper function- 3. Dynamically imports the generated modules- 4. Instantiates the generated classes- 5. Verifies nested concept references work correctly+ 1. Resolves the existing nested_concepts.mthds bundle into a normalized crate+ 2. Emits the python-structures projection through the stamped write path+ 3. Dynamically imports the generated module+ 4. Instantiates the generated classes, nesting Customer and LineItem inside Invoice"""- # Path to the MTHDS file with nested concepts- mthds_file_path = Path("tests/e2e/pipelex/concepts/nested_concepts/nested_concepts.mthds").resolve()- assert mthds_file_path.exists(), f"MTHDS file not found: {mthds_file_path}"+ bundle_dir = Path("tests/e2e/pipelex/concepts/nested_concepts").resolve()+ assert (bundle_dir / "nested_concepts.mthds").exists()++ library_manager = get_library_manager()+ previous_library_id = get_current_library_id_or_none()+ resolve_library_id = load_libraries_and_activate([bundle_dir])+ try:+ crate = library_manager.get_crate(resolve_library_id)+ assert crate is not None+ normalized = normalize_crate(crate, mthds_version=MTHDS_STANDARD_VERSION)+ finally:+ if previous_library_id is not None:+ set_current_library(library_id=previous_library_id)+ else:+ clear_current_library()+ library_manager.teardown(library_id=resolve_library_id)- # Create a temporary directory for generated structureswith tempfile.TemporaryDirectory() as temp_dir:output_directory = Path(temp_dir)-- # Validate the MTHDS file to get blueprints- validate_result = await validate_bundle(mthds_file_path=mthds_file_path)- blueprints = validate_result.blueprints-- # Generate structure files- generated_files = generate_structures_from_blueprints(- blueprints=blueprints,- output_directory=output_directory,- skip_existing_check=True,- )-- # Verify files were generated- assert len(generated_files) >= 3, f"Expected at least 3 generated files, got {len(generated_files)}"-- # Find the generated files- line_item_file = output_directory / "nested_concepts_test__line_item.py"- customer_file = output_directory / "nested_concepts_test__customer.py"- invoice_file = output_directory / "nested_concepts_test__invoice.py"-- assert line_item_file.exists(), f"LineItem structure file not generated: {line_item_file}"- assert customer_file.exists(), f"Customer structure file not generated: {customer_file}"- assert invoice_file.exists(), f"Invoice structure file not generated: {invoice_file}"-- # Read and verify the generated code contains proper structure.- # Class names are domain-qualified (e.g. "nested_concepts_test__LineItem") so- # they match what ConceptFactory registers in the class registry.- line_item_code = line_item_file.read_text()- customer_code = customer_file.read_text()- invoice_code = invoice_file.read_text()-- line_item_class_name = "nested_concepts_test__LineItem"- customer_class_name = "nested_concepts_test__Customer"- invoice_class_name = "nested_concepts_test__Invoice"-- assert f"class {line_item_class_name}(StructuredContent):" in line_item_code- assert f"class {customer_class_name}(StructuredContent):" in customer_code- assert f"class {invoice_class_name}(StructuredContent):" in invoice_code-- # Verify Invoice has forward references to the qualified Customer and LineItem names- assert customer_class_name in invoice_code- assert line_item_class_name in invoice_code-- # Dynamically import the generated modules- # Import order matters: dependencies first- line_item_class = self._import_class_from_file(line_item_file, line_item_class_name)- customer_class = self._import_class_from_file(customer_file, customer_class_name)-- # Now import Invoice - it has forward references to LineItem and Customer- # We need to add LineItem and Customer to the namespace for forward reference resolution- invoice_spec = importlib.util.spec_from_file_location("invoice_module", invoice_file)- assert invoice_spec is not None- assert invoice_spec.loader is not None- invoice_module = importlib.util.module_from_spec(invoice_spec)-- # Add the dependencies to the module's globals for forward reference resolution- invoice_module.__dict__[line_item_class_name] = line_item_class- invoice_module.__dict__[customer_class_name] = customer_class-- sys.modules["invoice_module"] = invoice_module- invoice_spec.loader.exec_module(invoice_module)- invoice_class: Any = getattr(invoice_module, invoice_class_name)-- # Rebuild the model to resolve forward references- invoice_class.model_rebuild(- _types_namespace={- line_item_class_name: line_item_class,- customer_class_name: customer_class,- }- )-- # Instantiate the classes to verify they work- line_item = line_item_class(- product_name="Widget",- quantity=3,- unit_price=10.0,- )- assert line_item.product_name == "Widget"- assert line_item.quantity == 3- assert line_item.unit_price == 10.0-- customer = customer_class(- name="John Smith",- email="[email protected]",+ emitted = emit_types(normalized, target=CodegenTarget.PYTHON_STRUCTURES)+ report = write_stamped_projection(+ emitted,+ output_dir=output_directory,+ crate_fingerprint=normalized.fingerprint,+ engine_version="test",+ kind=CodegenKind.TYPES,+ target=CodegenTarget.PYTHON_STRUCTURES,)- assert customer.name == "John Smith"- assert customer.email == "[email protected]"-- # Create Invoice with nested concepts- invoice = invoice_class(- invoice_number="INV-001",- customer=customer,- line_items=[line_item],- total_amount=30.0,- )- assert invoice.invoice_number == "INV-001"- assert invoice.customer.name == "John Smith"- assert len(invoice.line_items) == 1- assert invoice.line_items[0].product_name == "Widget"- assert invoice.total_amount == 30.0-- # Clean up imported modules- sys.modules.pop("invoice_module", None)-- def _import_class_from_file(self, file_path: Path, class_name: str) -> Any:- """Dynamically import a class from a Python file.-- Args:- file_path: Path to the Python file- class_name: Name of the class to import-- Returns:- The imported class- """+ assert report.written == ["structures.py"]+ assert (output_directory / CODEGEN_LOCK_FILENAME).is_file()++ structures_file = output_directory / "structures.py"+ structures_code = structures_file.read_text(encoding="utf-8")++ # Bare-when-unique class names (no cross-domain collision in this closure)+ assert "class LineItem(StructuredContent):" in structures_code+ assert "class Customer(StructuredContent):" in structures_code+ assert "class Invoice(StructuredContent):" in structures_code++ # The single module imports as-is: forward references resolve within it.+ # It must stay in sys.modules while in use — pydantic resolves the+ # `from __future__ import annotations` forward refs lazily at first instantiation.+ module = self._import_module_from_file(structures_file)+ try:+ self._exercise_generated_classes(module)+ finally:+ sys.modules.pop(module.__name__, None)++ def _exercise_generated_classes(self, module: Any) -> None:+ line_item_class: Any = module.LineItem+ customer_class: Any = module.Customer+ invoice_class: Any = module.Invoice++ line_item = line_item_class(product_name="Widget", quantity=3, unit_price=10.0)+ assert line_item.product_name == "Widget"+ assert line_item.quantity == 3+ assert line_item.unit_price == 10.0++ customer = customer_class(name="John Smith", email="[email protected]")+ assert customer.name == "John Smith"+ assert customer.email == "[email protected]"++ invoice = invoice_class(+ invoice_number="INV-001",+ customer=customer,+ line_items=[line_item],+ total_amount=30.0,+ )+ assert invoice.invoice_number == "INV-001"+ assert invoice.customer.name == "John Smith"+ assert len(invoice.line_items) == 1+ assert invoice.line_items[0].product_name == "Widget"+ assert invoice.total_amount == 30.0++ def _import_module_from_file(self, file_path: Path) -> Any:+ """Dynamically import a module from a Python file (left in sys.modules — caller cleans up)."""module_name = f"test_module_{file_path.stem}"spec = importlib.util.spec_from_file_location(module_name, file_path)assert spec is not None, f"Could not load spec for {file_path}"@@ -167,7 +126,9 @@ class TestStructureGeneratorCLI:module = importlib.util.module_from_spec(spec)sys.modules[module_name] = module- spec.loader.exec_module(module)-- the_class: Any = getattr(module, class_name)- return the_class+ try:+ spec.loader.exec_module(module)+ except BaseException:+ sys.modules.pop(module_name, None)+ raise+ return modulediff --git a/tests/integration/pipelex/cli/commands/test_build_runner_flow.py b/tests/integration/pipelex/cli/commands/test_build_runner_flow.pynew file mode 100644index 000000000..5bfc5aa2f--- /dev/null+++ b/tests/integration/pipelex/cli/commands/test_build_runner_flow.py@@ -0,0 +1,81 @@+"""Integration: `pipelex build runner` core flow over a real bundle (validate -> emit -> script).++Pins the D9 re-point: the runner scaffold's structures/ output comes from the codegen engine (one+stamped structures.py + codegen.lock), and the generated script spells custom classes the way the+emitted module defines them (bare-when-unique), importing them from `structures.structures`.+"""++import ast+import tempfile+from pathlib import Path++import pytest++from pipelex.cli.commands.build.runner._runner_core import _prepare_runner_core # noqa: PLC2701 # pyright: ignore[reportPrivateUsage]+from pipelex.codegen.lock import CODEGEN_LOCK_FILENAME+from pipelex.hub import clear_current_library, get_current_library_id_or_none, get_library_manager++_BUNDLE_MTHDS = """\+domain = "runnerflow"+description = "Runner-flow test domain"+main_pipe = "answer_query"++[concept.Query]+description = "A user query"+structure.question = { description = "the question", type = "text", required = true }++[concept.Report]+description = "A report"+structure.summary = { description = "the summary", type = "text", required = true }++[pipe.answer_query]+type = "PipeLLM"+description = "Answer a query with a report"+inputs = { query = "Query" }+output = "Report"+model = "$quick-reasoning"+prompt = "Answer $query"+"""++[email protected](loop_scope="class")+class TestBuildRunnerFlow:+ async def test_runner_and_structures_projection_agree_on_names(self):+ """The emitted structures.py defines bare-when-unique classes and the runner script imports+ and instantiates exactly those spellings (no runtime-qualified names leak into the script).+ """+ with tempfile.TemporaryDirectory() as tmp_dir:+ bundle_dir = Path(tmp_dir)+ bundle_path = bundle_dir / "bundle.mthds"+ bundle_path.write_text(_BUNDLE_MTHDS, encoding="utf-8")++ try:+ await _prepare_runner_core(pipe_code=None, bundle_path=bundle_path, library_dirs=[bundle_dir])+ finally:+ # validate_bundle leaves its library open and current (loaded-on-success); the CLI+ # wrapper tears the whole Pipelex down, so the test cleans up the library itself.+ library_id = get_current_library_id_or_none()+ if library_id is not None:+ clear_current_library()+ get_library_manager().teardown(library_id=library_id)++ structures_file = bundle_dir / "structures" / "structures.py"+ assert structures_file.is_file()+ assert (bundle_dir / "structures" / CODEGEN_LOCK_FILENAME).is_file()+ structures_code = structures_file.read_text(encoding="utf-8")+ assert "class Query(StructuredContent):" in structures_code+ assert "class Report(StructuredContent):" in structures_code++ runner_file = bundle_dir / "run_answer_query.py"+ assert runner_file.is_file()+ runner_code = runner_file.read_text(encoding="utf-8")+ ast.parse(runner_code)++ # Imports come from the single emitted module, spelled bare-when-unique.+ assert "from structures.structures import Query" in runner_code+ assert "from structures.structures import Report" in runner_code+ # The example input instantiates the emitted class; the output cast uses it too.+ assert "Query(question=" in runner_code+ assert "pipe_output.main_stuff_as(content_type=Report)" in runner_code+ # No runtime-qualified spelling leaks into the script.+ assert "runnerflow__" not in runner_codediff --git a/tests/integration/pipelex/codegen/test_emitted_python_typechecks.py b/tests/integration/pipelex/codegen/test_emitted_python_typechecks.pynew file mode 100644index 000000000..b3de11940--- /dev/null+++ b/tests/integration/pipelex/codegen/test_emitted_python_typechecks.py@@ -0,0 +1,121 @@+"""The D7 quality gate: emitted Python passes strict pyright and execs into real classes.++Per decision D7 (see `_codegen/TODOS.md`), the pyright gate on emitted Python lives in pipelex (the+toolchain is already present); the `tsc --strict` gate on emitted TypeScript lives in the+`conformance/` cross-repo harness. This module projects a rich crate — one exercising every+`ResolvedType` kind, a literal-with-default, an optional, a native reference, and a refines-native+concept — into `python-structures` and `python-pydantic`, then (1) runs strict pyright over the+generated files and asserts zero errors, and (2) imports them so the classes are actually built.+"""++import importlib.util+import json+import subprocess # noqa: S404 -- runs the venv's own pyright on generated files with a fixed, trusted argv+import sys+from pathlib import Path++import pytest++from pipelex.codegen.emitters.python_pydantic import emit_python_pydantic+from pipelex.codegen.emitters.python_structures import emit_python_structures+from pipelex.codegen.resolved_concepts import resolve_concepts_from_crate+from pipelex.core.concepts.concept_blueprint import ConceptBlueprint+from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprint, ConceptStructureBlueprintFieldType+from pipelex.core.domains.domain_blueprint import DomainBlueprint+from pipelex.libraries.crate_normalization import normalize_crate+from pipelex.libraries.library_crate import LibraryCrate++_CRATE_VERSION = "1.0.0-test"+++def _rich_crate() -> LibraryCrate:+ """A crate exercising every resolved-type kind the Python emitters render."""+ field = ConceptStructureBlueprint+ kind = ConceptStructureBlueprintFieldType+ authored = LibraryCrate(+ concepts={+ "report.Report": ConceptBlueprint(+ description="A rich report",+ structure={+ "title": field(description="the title", type=kind.TEXT),+ "score": field(description="the score", type=kind.NUMBER),+ "count": field(description="a count", type=kind.INTEGER),+ "final": field(description="is final", type=kind.BOOLEAN),+ "due": field(description="due date", type=kind.DATE, required=False),+ "status": field(description="status", choices=["draft", "final"], default_value="draft"),+ "tags": field(description="tags", type=kind.LIST, item_type="text"),+ "meta": field(description="meta", type=kind.DICT, key_type="str", value_type="Any"),+ "summary": field(description="the summary", type=kind.CONCEPT, concept_ref="Summary"),+ "body": field(description="the body text", type=kind.CONCEPT, concept_ref="Text"),+ },+ ),+ "report.Summary": ConceptBlueprint(description="A short refined summary", refines="Text"),+ },+ domains={"report": DomainBlueprint(code="report", description="Report domain", main_pipe=None)},+ )+ return normalize_crate(authored, mthds_version=_CRATE_VERSION)+++def _run_strict_pyright(*, directory: Path) -> tuple[int, str]:+ """Run strict pyright over `directory` using the test venv, returning (error_count, raw_json).++ Pyright is a node subprocess whose piped stdout has been observed truncated under a fully+ loaded CI run, so an unparseable report gets one bounded retry; if it happens again, fail+ with the raw process output so the recurrence is diagnosable.+ """+ (directory / "pyrightconfig.json").write_text(json.dumps({"typeCheckingMode": "strict", "reportMissingModuleSource": False}), encoding="utf-8")+ pyright = Path(sys.executable).parent / "pyright"+ failures: list[str] = []+ for attempt_index in range(2):+ completed = subprocess.run( # noqa: S603 -- fixed, trusted argv (the venv's pyright over a tmp dir)+ [str(pyright), "--pythonpath", sys.executable, "--project", str(directory), "--outputjson"],+ capture_output=True,+ text=True,+ timeout=180,+ check=False,+ )+ try:+ report = json.loads(completed.stdout)+ except json.JSONDecodeError as decode_error:+ failures.append(+ f"attempt {attempt_index + 1}: pyright stdout was not parseable JSON ({decode_error}); "+ f"returncode={completed.returncode}\nstdout: {completed.stdout!r}\nstderr: {completed.stderr!r}"+ )+ continue+ return report["summary"]["errorCount"], completed.stdout+ msg = "pyright did not produce a parseable JSON report:\n" + "\n".join(failures)+ pytest.fail(msg)+++class TestEmittedPythonTypechecks:+ """Strict-pyright gate (D7) plus a runtime import of the generated Python modules."""++ def test_emitted_python_passes_strict_pyright(self, tmp_path: Path) -> None:+ pyright = Path(sys.executable).parent / "pyright"+ if not pyright.exists():+ pytest.skip("pyright not available in this environment")+ crate = _rich_crate()+ library = resolve_concepts_from_crate(crate)+ files = [*emit_python_structures(library), *emit_python_pydantic(library)]+ for emitted in files:+ (tmp_path / emitted.filename).write_text(emitted.content, encoding="utf-8")++ error_count, raw = _run_strict_pyright(directory=tmp_path)+ assert error_count == 0, f"emitted Python failed strict pyright:\n{raw}"++ def test_emitted_python_execs_into_real_classes(self, tmp_path: Path) -> None:+ crate = _rich_crate()+ library = resolve_concepts_from_crate(crate)+ for emitted in [*emit_python_structures(library), *emit_python_pydantic(library)]:+ module_name = emitted.filename.removesuffix(".py")+ module_path = tmp_path / emitted.filename+ module_path.write_text(emitted.content, encoding="utf-8")+ spec = importlib.util.spec_from_file_location(module_name, module_path)+ assert spec is not None+ assert spec.loader is not None+ module = importlib.util.module_from_spec(spec)+ sys.modules[module_name] = module+ spec.loader.exec_module(module)+ # The Summary concept refines the native Text — the emitted class must carry that base.+ summary_bases = [base.__name__ for base in module.Summary.__mro__]+ assert "TextContent" in summary_bases or "Text" in summary_basesdiff --git a/tests/integration/pipelex/codegen/test_resolve_flow.py b/tests/integration/pipelex/codegen/test_resolve_flow.pynew file mode 100644index 000000000..59d980a50--- /dev/null+++ b/tests/integration/pipelex/codegen/test_resolve_flow.py@@ -0,0 +1,152 @@+import importlib.util+import json+import sys+import tempfile+from collections.abc import Callable+from pathlib import Path++import tomli+from mthds.package.manifest.schema import MTHDS_STANDARD_VERSION++from pipelex.codegen.crate_encoding import encode_crate_json, encode_crate_toml+from pipelex.codegen.emitters.target import CodegenTarget+from pipelex.codegen.emitters.types_emitter import emit_types+from pipelex.core.concepts.concept_blueprint import ConceptBlueprint+from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprint+from pipelex.hub import get_library_manager+from pipelex.libraries.crate_normalization import normalize_crate+from pipelex.libraries.library_crate import LibraryCrate+from pipelex.pipe_controllers.sequence.pipe_sequence_blueprint import PipeSequenceBlueprint+from pipelex.pipeline.execution_seams import load_libraries_and_activate++# A single-package, multi-bundle closure (domain `pipeline` split across two sibling files, no+# cross-package `alias->` refs). `main.mthds` references a concept (`Score`) and a sub-pipe+# (`compute_score`) defined in `steps.mthds`, plus the native `Text` — so the resolve flow must merge,+# qualify cross-file concept/pipe refs, and materialize the native. This is the primary B1 target shape.+MAIN_MTHDS = """\+domain = "pipeline"+description = "Pipeline domain"++[concept.Report]+description = "A report"+structure.score = { description = "the score", type = "concept", concept_ref = "Score" }+structure.label = { description = "a label", type = "concept", concept_ref = "Text" }++[pipe.run_pipeline]+type = "PipeSequence"+description = "Run the pipeline"+inputs = { doc = "Text" }+output = "Score"+steps = [{ pipe = "compute_score", result = "score" }]+"""++STEPS_MTHDS = """\+domain = "pipeline"++[concept.Score]+description = "A score"+structure = { value = { description = "the value", type = "number" } }++[pipe.compute_score]+type = "PipeLLM"+description = "Compute a score"+inputs = { doc = "Text" }+output = "Score"+model = "$quick-reasoning"+prompt = "Compute a score from $doc"+"""+++def _structure_field(concept: ConceptBlueprint | str, field_name: str) -> ConceptStructureBlueprint:+ """Narrow a concept's structure field to a `ConceptStructureBlueprint` for typed assertions."""+ assert isinstance(concept, ConceptBlueprint)+ assert isinstance(concept.structure, dict)+ field = concept.structure[field_name]+ assert isinstance(field, ConceptStructureBlueprint)+ return field+++class TestResolveFlow:+ """Integration: the resolve pipeline (closure → crate → normalize → encode → round-trip)."""++ def test_multi_bundle_closure_normalizes_and_encodes(self, load_empty_library: Callable[[], str]):+ """A multi-bundle closure resolves to a flat, fully-qualified, natives-expanded crate; both+ encodings carry the same fingerprint; and the TOML form is directly runnable (round-trip load).+ """+ library_manager = get_library_manager()++ with tempfile.TemporaryDirectory() as tmp_dir:+ closure_dir = Path(tmp_dir)+ (closure_dir / "main.mthds").write_text(MAIN_MTHDS, encoding="utf-8")+ (closure_dir / "steps.mthds").write_text(STEPS_MTHDS, encoding="utf-8")++ resolve_library_id = load_libraries_and_activate([closure_dir])+ crate = library_manager.get_crate(resolve_library_id)+ assert crate is not None+ normalized = normalize_crate(crate, mthds_version=MTHDS_STANDARD_VERSION)+ library_manager.teardown(library_id=resolve_library_id)++ # Flat, fully-qualified keyspace across both bundles.+ assert {"pipeline.Report", "pipeline.Score", "pipeline.compute_score", "pipeline.run_pipeline"} <= set(+ normalized.concepts.keys() | normalized.pipes.keys()+ )+ # Native materialized (referenced by a field + pipe io).+ assert "native.Text" in normalized.concepts++ # Cross-file concept ref qualified against the owner domain.+ report = normalized.concepts["pipeline.Report"]+ assert _structure_field(report, "score").concept_ref == "pipeline.Score"+ assert _structure_field(report, "label").concept_ref == "native.Text"++ # Cross-file pipe ref inside the sequence controller qualified against the owner domain.+ run_pipeline = normalized.pipes["pipeline.run_pipeline"]+ assert isinstance(run_pipeline, PipeSequenceBlueprint)+ assert run_pipeline.steps[0].pipe == "pipeline.compute_score"++ # Both encodings agree on the fingerprint.+ json_doc = json.loads(encode_crate_json(normalized))+ toml_doc = tomli.loads(encode_crate_toml(normalized))+ assert json_doc["fingerprint"] == toml_doc["fingerprint"] == normalized.fingerprint++ # Directly runnable: the TOML form parses back into a crate that loads into a live library.+ from_toml = LibraryCrate.model_validate(toml_doc)+ assert from_toml.compute_normalized() == normalized.fingerprint+ reload_library_id = load_empty_library()+ library_manager.load_from_crate(library_id=reload_library_id, crate=from_toml)+ reloaded = library_manager.get_library(library_id=reload_library_id)+ assert "pipeline.run_pipeline" in reloaded.pipe_library.root+ assert "pipeline.Report" in reloaded.concept_library.root++ def test_multi_bundle_closure_feeds_every_emitter(self, tmp_path: Path):+ """The resolved multi-bundle crate projects through every types target, and the emitted Python+ execs into real classes — the resolve -> emit chain end to end (fixture 2).+ """+ library_manager = get_library_manager()+ with tempfile.TemporaryDirectory() as tmp_dir:+ closure_dir = Path(tmp_dir)+ (closure_dir / "main.mthds").write_text(MAIN_MTHDS, encoding="utf-8")+ (closure_dir / "steps.mthds").write_text(STEPS_MTHDS, encoding="utf-8")+ resolve_library_id = load_libraries_and_activate([closure_dir])+ crate = library_manager.get_crate(resolve_library_id)+ assert crate is not None+ normalized = normalize_crate(crate, mthds_version=MTHDS_STANDARD_VERSION)+ library_manager.teardown(library_id=resolve_library_id)++ # ts-zod projects a pure types file and its binder; each Python target projects one module.+ assert {file.filename for file in emit_types(normalized, target=CodegenTarget.TS_ZOD)} == {"types.ts", "binder.ts"}+ for target in (CodegenTarget.PYTHON_STRUCTURES, CodegenTarget.PYTHON_PYDANTIC):+ emitted = emit_types(normalized, target=target)+ assert len(emitted) == 1+ module_name = emitted[0].filename.removesuffix(".py")+ module_path = tmp_path / f"{target.name.lower()}_{emitted[0].filename}"+ module_path.write_text(emitted[0].content, encoding="utf-8")+ spec = importlib.util.spec_from_file_location(module_name, module_path)+ assert spec is not None+ assert spec.loader is not None+ module = importlib.util.module_from_spec(spec)+ sys.modules[module_name] = module+ spec.loader.exec_module(module)+ # Report cross-references Score (defined in the sibling bundle) — both classes build, and+ # Score, a leaf, instantiates. Report's fields survive the projection.+ assert module.Score(value=1.0).value == 1.0+ assert {"score", "label"} <= set(module.Report.model_fields)diff --git a/tests/integration/pipelex/core/concepts/test_concept_representation_generator.py b/tests/integration/pipelex/core/concepts/test_concept_representation_generator.pyindex 1ce54b5d2..47f2fab18 100644--- a/tests/integration/pipelex/core/concepts/test_concept_representation_generator.py+++ b/tests/integration/pipelex/core/concepts/test_concept_representation_generator.py@@ -78,7 +78,7 @@ class TestComplexNestedStructuresJson:result = generator.generate_representation("test.Attachment", structure_class=Attachment)assert result["concept"] == "test.Attachment"assert result["content"]["name"] == "name_value"- assert result["content"]["url"].startswith("https://mock-")+ assert result["content"]["url"].startswith("https://mock.invalid/")def test_embed_json(self) -> None:"""Test Embed with multiple string fields generates correct JSON."""@@ -104,7 +104,7 @@ class TestComplexNestedStructuresJson:assert content["link"] == "link_value"assert len(content["attachments"]) == 1assert content["attachments"][0]["name"] == "name_value"- assert content["attachments"][0]["url"].startswith("https://mock-")+ assert content["attachments"][0]["url"].startswith("https://mock.invalid/")assert content["embeds"] == [{"title": "title_value", "description": "description_value", "embed_type": "embed_type_value"}]def test_channel_json_deeply_nested(self) -> None:@@ -118,7 +118,7 @@ class TestComplexNestedStructuresJson:message = content["messages"][0]assert message["author"] == "author_value"assert len(message["attachments"]) == 1- assert message["attachments"][0]["url"].startswith("https://mock-")+ assert message["attachments"][0]["url"].startswith("https://mock.invalid/")def test_newsletter_json_multiple_lists(self) -> None:"""Test Newsletter with multiple list fields generates correct JSON."""@@ -148,7 +148,7 @@ class TestComplexNestedStructuresPython:generator = ConceptRepresentationGenerator(ConceptRepresentationFormat.PYTHON)result = generator.generate_representation("test.Attachment", structure_class=Attachment)assert result["concept"] == "test.Attachment"- assert 'Attachment(name="name_value", url="https://mock-' in result["content"]+ assert 'Attachment(name="name_value", url="https://mock.invalid/' in result["content"]def test_message_python_recursive(self) -> None:"""Test Message with nested structures generates correct Python."""@@ -156,7 +156,7 @@ class TestComplexNestedStructuresPython:result = generator.generate_representation("test.Message", structure_class=Message)content = result["content"]assert content.startswith(- 'Message(author="author_value", content="content_value", attachments=[Attachment(name="name_value", url="https://mock-'+ 'Message(author="author_value", content="content_value", attachments=[Attachment(name="name_value", url="https://mock.invalid/')assert 'embeds=[Embed(title="title_value"' in contentassert 'link="link_value")' in content@@ -167,7 +167,7 @@ class TestComplexNestedStructuresPython:result = generator.generate_representation("test.Channel", structure_class=Channel)content = result["content"]assert content.startswith('Channel(name="name_value", position=0, messages=[Message(')- assert 'Attachment(name="name_value", url="https://mock-' in content+ assert 'Attachment(name="name_value", url="https://mock.invalid/' in contentclass TestImportsTrackingIntegration:@@ -200,7 +200,7 @@ class TestConvenienceFunctionsIntegration:assert "attachments" in result["content"]assert len(result["content"]["attachments"]) == 1assert result["content"]["attachments"][0]["name"] == "name_value"- assert result["content"]["attachments"][0]["url"].startswith("https://mock-")+ assert result["content"]["attachments"][0]["url"].startswith("https://mock.invalid/")def test_generate_python_representation_complex(self) -> None:"""Test generate_python_representation with complex structure."""diff --git a/tests/integration/pipelex/libraries/test_crate_normalization_round_trip.py b/tests/integration/pipelex/libraries/test_crate_normalization_round_trip.pynew file mode 100644index 000000000..33c735fa2--- /dev/null+++ b/tests/integration/pipelex/libraries/test_crate_normalization_round_trip.py@@ -0,0 +1,67 @@+import tempfile+from collections.abc import Callable+from pathlib import Path++from pipelex.core.interpreter.interpreter import PipelexInterpreter+from pipelex.hub import get_library_manager+from pipelex.libraries.crate_normalization import normalize_crate+from pipelex.libraries.library_crate_factory import LibraryCrateFactory++MTHDS_TEST_VERSION = "0.0.0-test"++REPORT_MTHDS = """\+domain = "report"+description = "Report domain"++[concept.Score]+description = "A weighted score"+structure = { value = "the numeric value", note = { description = "a free-text note", type = "concept", concept_ref = "Text" } }++[concept.Snapshot]+description = "A domain-specific image"+refines = "Image"++[pipe.make_score]+type = "PipeLLM"+description = "Compute a score"+inputs = { data = "Text" }+output = "Score"+model = "$quick-reasoning"+prompt = "Compute a score from $data"+"""+++class TestCrateNormalizationRoundTrip:+ """Integration: a normalized crate (natives materialized) loads back into a live library."""++ def test_normalized_crate_loads_and_validates(self, load_empty_library: Callable[[], str]):+ """normalize_crate materializes `native.*` entries; load_from_crate must skip them and still+ load the authored concepts/pipes, then pass validate_library — proving the round-trip (D6).+ """+ library_manager = get_library_manager()++ with tempfile.TemporaryDirectory() as tmp_dir:+ report_path = Path(tmp_dir) / "report.mthds"+ report_path.write_text(REPORT_MTHDS, encoding="utf-8")+ blueprints = [PipelexInterpreter.make_pipelex_bundle_blueprint(bundle_path=report_path)]++ crate = LibraryCrateFactory.make_from_blueprints(blueprints=blueprints)+ normalized = normalize_crate(crate, mthds_version=MTHDS_TEST_VERSION)++ # The normalized crate must carry materialized native entries (the very thing the loader+ # has to tolerate): `native.Text` (referenced by a field + pipe io) and `native.Image`+ # (referenced by a `refines`).+ assert "native.Text" in normalized.concepts+ assert "native.Image" in normalized.concepts++ library_id = load_empty_library()+ # Would raise ConceptLibraryError("Concept 'native.Text' already exists") without the skip.+ library_manager.load_from_crate(library_id=library_id, crate=normalized)+ library = library_manager.get_library(library_id=library_id)++ # Authored content is present with fully-qualified refs.+ assert "report.Score" in library.concept_library.root+ assert "report.Snapshot" in library.concept_library.root+ assert "report.make_score" in library.pipe_library.root+ # Natives remain singly-registered (pre-registered natives, not the crate's copies).+ assert "native.Text" in library.concept_library.rootdiff --git a/tests/integration/pipelex/pipeline/test_load_concepts_only.py b/tests/integration/pipelex/pipeline/test_load_concepts_only.pydeleted file mode 100644index 6e9dd8fe9..000000000--- a/tests/integration/pipelex/pipeline/test_load_concepts_only.py+++ /dev/null@@ -1,417 +0,0 @@-"""Integration tests for load_concepts_only functions from MTHDS files."""--import tempfile-from collections.abc import Callable-from pathlib import Path--import pytest--from pipelex.hub import get_library_manager-from pipelex.pipeline.exceptions import ValidateBundleError-from pipelex.pipeline.validate_bundle import (- LoadConceptsOnlyResult,- load_concepts_only,- load_concepts_only_from_directory,-)---class TestLoadConceptsOnly:- """Integration tests for loading concepts only (no pipes) from MTHDS files."""-- def test_load_concepts_only_single_file(self, load_empty_library: Callable[[], str]):- """Test loading concepts from a single MTHDS file."""- load_empty_library()- mthds_content = """-domain = "testapp"-description = "Test domain"--[concept.Customer]-description = "A customer"--[concept.Customer.structure]-name = { type = "text", description = "Customer name" }-email = { type = "text", description = "Customer email" }-"""-- with tempfile.TemporaryDirectory() as tmp_dir:- mthds_path = Path(tmp_dir) / "test.mthds"- mthds_path.write_text(mthds_content, encoding="utf-8")-- result = load_concepts_only(mthds_file_path=mthds_path)-- assert isinstance(result, LoadConceptsOnlyResult)- assert len(result.blueprints) == 1- assert len(result.concepts) == 1- assert result.concepts[0].code == "Customer"-- def test_load_concepts_only_skips_pipes(self, load_empty_library: Callable[[], str]):- """Test that pipes are skipped when loading concepts only."""- load_empty_library()- mthds_content = """-domain = "testapp"-description = "Test domain with pipe"--[concept.Topic]-description = "A topic"--[concept.Topic.structure]-name = { type = "text", description = "Topic name" }--[pipe.generate_topic]-type = "PipeLLM"-description = "Generate a topic"-inputs = { subject = "Text" }-output = "Topic"-prompt = "Generate a topic about $subject"-"""-- with tempfile.TemporaryDirectory() as tmp_dir:- mthds_path = Path(tmp_dir) / "test.mthds"- mthds_path.write_text(mthds_content, encoding="utf-8")-- result = load_concepts_only(mthds_file_path=mthds_path)-- # Concepts should be loaded- assert len(result.concepts) == 1- assert result.concepts[0].code == "Topic"-- # Verify pipes were NOT loaded by checking the library- library_manager = get_library_manager()- library = library_manager.get_current_library()-- # The pipe library should not have the pipe loaded- assert len(library.pipe_library.root) == 0-- def test_load_concepts_only_from_directory(self, load_empty_library: Callable[[], str]):- """Test loading concepts from a directory with multiple MTHDS files."""- load_empty_library()- mthds_content_1 = """-domain = "crm"-description = "CRM domain"--[concept.Customer]-description = "A customer"--[concept.Customer.structure]-name = { type = "text", description = "Customer name" }-"""-- mthds_content_2 = """-domain = "accounting"-description = "Accounting domain"--[concept.Invoice]-description = "An invoice"--[concept.Invoice.structure]-amount = { type = "number", description = "Invoice amount" }-"""-- with tempfile.TemporaryDirectory() as tmp_dir:- (Path(tmp_dir) / "crm.mthds").write_text(mthds_content_1, encoding="utf-8")- (Path(tmp_dir) / "accounting.mthds").write_text(mthds_content_2, encoding="utf-8")-- result = load_concepts_only_from_directory(directory=Path(tmp_dir))-- assert len(result.blueprints) == 2- assert len(result.concepts) == 2-- concept_codes = [concept.code for concept in result.concepts]- assert "Customer" in concept_codes- assert "Invoice" in concept_codes-- def test_load_concepts_only_with_concept_references(self, load_empty_library: Callable[[], str]):- """Test loading concepts that reference other concepts."""- load_empty_library()- mthds_content = """-domain = "testapp"-description = "Test domain with concept references"--[concept.Customer]-description = "A customer"--[concept.Customer.structure]-name = { type = "text", description = "Customer name" }--[concept.Invoice]-description = "An invoice"--[concept.Invoice.structure]-customer = { type = "concept", concept_ref = "testapp.Customer", description = "The customer" }-total = { type = "number", description = "Invoice total" }-"""-- with tempfile.TemporaryDirectory() as tmp_dir:- mthds_path = Path(tmp_dir) / "test.mthds"- mthds_path.write_text(mthds_content, encoding="utf-8")-- result = load_concepts_only(mthds_file_path=mthds_path)-- assert len(result.concepts) == 2-- # Verify the library has the concepts loaded- library_manager = get_library_manager()- library = library_manager.get_current_library()-- customer = library.concept_library.get_required_concept("testapp.Customer")- invoice = library.concept_library.get_required_concept("testapp.Invoice")-- assert customer is not None- assert invoice is not None-- def test_load_concepts_only_detects_cycles(self, load_empty_library: Callable[[], str]):- """Test that cycle detection still works when loading concepts only."""- load_empty_library()- mthds_content = """-domain = "testapp"-description = "Test domain with cycles"--[concept.ConceptA]-description = "A"--[concept.ConceptA.structure]-b_ref = { type = "concept", concept_ref = "testapp.ConceptB", description = "Ref to B" }--[concept.ConceptB]-description = "B"--[concept.ConceptB.structure]-a_ref = { type = "concept", concept_ref = "testapp.ConceptA", description = "Ref to A" }-"""-- with tempfile.TemporaryDirectory() as tmp_dir:- mthds_path = Path(tmp_dir) / "test.mthds"- mthds_path.write_text(mthds_content, encoding="utf-8")-- with pytest.raises(Exception, match=r"[Cc]ycle"):- load_concepts_only(mthds_file_path=mthds_path)-- def test_load_concepts_only_with_library_dirs(self, load_empty_library: Callable[[], str]):- """Test loading concepts with library dependencies."""- load_empty_library()- # Library MTHDS with shared concepts- library_mthds = """-domain = "shared"-description = "Shared library"--[concept.Address]-description = "An address"--[concept.Address.structure]-street = { type = "text", description = "Street" }-city = { type = "text", description = "City" }-"""-- # Main MTHDS that references the library concept- main_mthds = """-domain = "main"-description = "Main domain"--[concept.Customer]-description = "A customer"--[concept.Customer.structure]-name = { type = "text", description = "Name" }-address = { type = "concept", concept_ref = "shared.Address", description = "The address" }-"""-- with tempfile.TemporaryDirectory() as lib_dir, tempfile.TemporaryDirectory() as main_dir:- (Path(lib_dir) / "shared.mthds").write_text(library_mthds, encoding="utf-8")- main_mthds_path = Path(main_dir) / "main.mthds"- main_mthds_path.write_text(main_mthds, encoding="utf-8")-- result = load_concepts_only(- mthds_file_path=main_mthds_path,- library_dirs=[Path(lib_dir)],- )-- # Main concepts should be loaded- assert len(result.concepts) == 1- assert result.concepts[0].code == "Customer"-- # Verify both shared and main concepts are in the library- library_manager = get_library_manager()- library = library_manager.get_current_library()-- address = library.concept_library.get_required_concept("shared.Address")- customer = library.concept_library.get_required_concept("main.Customer")-- assert address is not None- assert customer is not None-- def test_load_concepts_only_library_dir_sibling_cross_reference(self, load_empty_library: Callable[[], str]):- """Sibling files in a library dir resolve same-domain concept refs regardless of load order.-- The referencing file (``a_refers.mthds``) sorts before the declaring file (``b_declares.mthds``),- so loading each file as its own one-file batch would validate the reference before its- declaration is present and fail. Batching the directory's files together resolves it.- """- load_empty_library()- refers_mthds = """-domain = "libdom"-description = "Library domain"--[concept.Order]-description = "An order"--[concept.Order.structure]-line_item = { type = "concept", concept_ref = "libdom.LineItem", description = "A line item" }-"""- declares_mthds = """-domain = "libdom"-description = "Library domain"--[concept.LineItem]-description = "A line item"--[concept.LineItem.structure]-sku = { type = "text", description = "SKU" }-"""- main_mthds = """-domain = "main"-description = "Main domain"--[concept.Customer]-description = "A customer"--[concept.Customer.structure]-name = { type = "text", description = "Name" }-"""-- with tempfile.TemporaryDirectory() as lib_dir, tempfile.TemporaryDirectory() as main_dir:- # a_refers sorts before b_declares, so a one-file-batch loader would load the- # reference before its declaration and raise.- (Path(lib_dir) / "a_refers.mthds").write_text(refers_mthds, encoding="utf-8")- (Path(lib_dir) / "b_declares.mthds").write_text(declares_mthds, encoding="utf-8")- main_mthds_path = Path(main_dir) / "main.mthds"- main_mthds_path.write_text(main_mthds, encoding="utf-8")-- result = load_concepts_only(mthds_file_path=main_mthds_path, library_dirs=[Path(lib_dir)])-- assert result.concepts[0].code == "Customer"-- library_manager = get_library_manager()- library = library_manager.get_current_library()-- assert library.concept_library.get_required_concept("libdom.Order") is not None- assert library.concept_library.get_required_concept("libdom.LineItem") is not None-- def test_load_concepts_only_with_mthds_content(self, load_empty_library: Callable[[], str]):- """Test loading concepts from MTHDS content string."""- load_empty_library()- mthds_content = """-domain = "testapp"-description = "Test domain"--[concept.MyItem]-description = "An item"--[concept.MyItem.structure]-name = { type = "text", description = "Item name" }-"""-- result = load_concepts_only(mthds_contents=[mthds_content])-- assert len(result.blueprints) == 1- assert len(result.concepts) == 1- assert result.concepts[0].code == "MyItem"-- def test_load_concepts_only_with_refines(self, load_empty_library: Callable[[], str]):- """Test loading concepts with refines relationships."""- load_empty_library()- mthds_content = """-domain = "testapp"-description = "Test domain with refines"--[concept.Customer]-description = "A customer"--[concept.Customer.structure]-name = { type = "text", description = "Customer name" }--[concept.VIPCustomer]-description = "A VIP customer - inherits from Customer"-refines = "Customer"-"""-- with tempfile.TemporaryDirectory() as tmp_dir:- mthds_path = Path(tmp_dir) / "test.mthds"- mthds_path.write_text(mthds_content, encoding="utf-8")-- result = load_concepts_only(mthds_file_path=mthds_path)-- assert len(result.concepts) == 2-- concept_codes = [concept.code for concept in result.concepts]- assert "Customer" in concept_codes- assert "VIPCustomer" in concept_codes-- def test_load_concepts_only_rejects_dangling_concept_ref(self, load_empty_library: Callable[[], str]):- """A dangling concept_ref in a structure field is rejected, matching the full-load path."""- load_empty_library()- mthds_content = """-domain = "testapp"-description = "Test domain with a dangling concept_ref"--[concept.Invoice]-description = "An invoice"--[concept.Invoice.structure]-customer = { type = "concept", concept_ref = "testapp.MissingCustomer", description = "The customer" }-"""-- with pytest.raises(ValidateBundleError, match="MissingCustomer"):- load_concepts_only(mthds_contents=[mthds_content])-- def test_load_concepts_only_rejects_dangling_item_concept_ref(self, load_empty_library: Callable[[], str]):- """A dangling item_concept_ref in a list structure field is rejected, matching the full-load path."""- load_empty_library()- mthds_content = """-domain = "testapp"-description = "Test domain with a dangling item_concept_ref"--[concept.Order]-description = "An order"--[concept.Order.structure]-items = { type = "list", item_type = "concept", item_concept_ref = "testapp.MissingLineItem", description = "Line items" }-"""-- with pytest.raises(ValidateBundleError, match="MissingLineItem"):- load_concepts_only(mthds_contents=[mthds_content])-- def test_load_concepts_only_directory_skips_pipes(self, load_empty_library: Callable[[], str]):- """Test that pipes are skipped when loading from directory."""- load_empty_library()- mthds_content = """-domain = "testapp"-description = "Test domain with pipe"--[concept.Result]-description = "A result"--[concept.Result.structure]-value = { type = "text", description = "Result value" }--[pipe.my_pipe]-type = "PipeLLM"-description = "A pipe that should be skipped"-inputs = { subject = "Text" }-output = "Result"-prompt = "Generate a result about $subject"-"""-- with tempfile.TemporaryDirectory() as tmp_dir:- (Path(tmp_dir) / "test.mthds").write_text(mthds_content, encoding="utf-8")-- result = load_concepts_only_from_directory(directory=Path(tmp_dir))-- # Concepts should be loaded- assert len(result.concepts) == 1- assert result.concepts[0].code == "Result"-- # Verify pipes were NOT loaded- library_manager = get_library_manager()- library = library_manager.get_current_library()-- assert len(library.pipe_library.root) == 0diff --git a/tests/integration/pipelex/pipeline/test_resolve_bundle.py b/tests/integration/pipelex/pipeline/test_resolve_bundle.pynew file mode 100644index 000000000..1b39d9694--- /dev/null+++ b/tests/integration/pipelex/pipeline/test_resolve_bundle.py@@ -0,0 +1,149 @@+"""Integration: ``resolve_crate_from_contents`` — the in-memory (host-facing) resolve core.++Pins the contract the HTTP resolve/codegen routes build on: in-memory contents resolve to a+normalized crate through the same engine as the CLI, an invalid library raises the one shared+``ValidateBundleError``, and the library lifecycle honors the loaded-on-success /+torn-down-on-failure contract of ``validate_bundle``.+"""++from collections.abc import Callable++import pytest+from mthds.package.manifest.schema import MTHDS_STANDARD_VERSION+from pytest_mock import MockerFixture++from pipelex.base_exceptions import PipelexUnexpectedError+from pipelex.hub import clear_current_library, get_current_library_id_or_none, get_library_manager+from pipelex.pipeline.exceptions import ValidateBundleError+from pipelex.pipeline.resolve_bundle import resolve_crate_from_contents++MAIN_MTHDS = """\+domain = "pipeline"+description = "Pipeline domain"++[concept.Report]+description = "A report"+structure.score = { description = "the score", type = "concept", concept_ref = "Score" }+structure.label = { description = "a label", type = "concept", concept_ref = "Text" }++[pipe.run_pipeline]+type = "PipeSequence"+description = "Run the pipeline"+inputs = { doc = "Text" }+output = "Score"+steps = [{ pipe = "compute_score", result = "score" }]+"""++STEPS_MTHDS = """\+domain = "pipeline"++[concept.Score]+description = "A score"+structure = { value = { description = "the value", type = "number" } }++[pipe.compute_score]+type = "PipeLLM"+description = "Compute a score"+inputs = { doc = "Text" }+output = "Score"+model = "$quick-reasoning"+prompt = "Compute a score from $doc"+"""++# The pipe references a concept that no bundle declares — a load-stage library error, translated+# by the shared cascade into a ValidateBundleError carrying structured validation errors.+INVALID_MTHDS = """\+domain = "broken"+description = "Broken domain"++[pipe.bad_pipe]+type = "PipeLLM"+description = "References an undeclared concept"+inputs = { doc = "NoSuchConcept" }+output = "Text"+prompt = "Do something with $doc"+"""++ADDRESS_DEPENDENCY_MTHDS = """\+domain = "host"++[pipe.run_host]+type = "PipeSequence"+description = "Run an installed dependency"+inputs = { doc = "Text" }+output = "Text"+steps = [{ pipe = "github.com/org/pkg->remote.process", result = "result" }]+"""+++class TestResolveCrateFromContents:+ def test_multi_bundle_contents_resolve_to_normalized_crate(self, load_empty_library: Callable[[], str]):+ """A multi-bundle in-memory closure resolves to a flat, fully-qualified, natives-materialized crate,+ and the library is left loaded + current for the host (which then owns teardown).+ """+ load_empty_library()+ library_manager = get_library_manager()+ crate = resolve_crate_from_contents(+ mthds_contents=[MAIN_MTHDS, STEPS_MTHDS],+ mthds_sources=["main.mthds", "steps.mthds"],+ )+ library_id = get_current_library_id_or_none()+ try:+ assert crate.fingerprint+ assert crate.mthds_version == MTHDS_STANDARD_VERSION+ assert "pipeline.Report" in crate.concepts+ assert "pipeline.Score" in crate.concepts+ assert "native.Text" in crate.concepts, "referenced natives must be materialized into the normalized crate"+ assert "pipeline.run_pipeline" in crate.pipes+ assert "pipeline.compute_score" in crate.pipes+ # Sources threaded onto the blueprints surface in the crate's provenance map.+ assert crate.source_map.get("pipeline.run_pipeline") == "main.mthds"+ assert crate.source_map.get("pipeline.compute_score") == "steps.mthds"+ # Loaded-on-success: the freshly opened library is current, so the host can read live pipes.+ assert library_id is not None+ finally:+ clear_current_library()+ if library_id is not None:+ library_manager.teardown(library_id=library_id)++ def test_invalid_contents_raise_shared_verdict_and_tear_down(self, load_empty_library: Callable[[], str], mocker: MockerFixture):+ """An invalid library raises the one shared ValidateBundleError (structured items present),+ the opened library is torn down, and the caller's current-library state is restored.+ """+ load_empty_library()+ library_manager = get_library_manager()+ open_library_spy = mocker.spy(library_manager, "open_library")+ teardown_spy = mocker.spy(library_manager, "teardown")+ current_before = get_current_library_id_or_none()++ teardown_calls_before = teardown_spy.call_count+ with pytest.raises(ValidateBundleError) as exc_info:+ resolve_crate_from_contents(mthds_contents=[INVALID_MTHDS], mthds_sources=["broken.mthds"])++ error_report = exc_info.value.to_error_report()+ assert error_report.validation_errors, "the shared builder must produce structured validation-error items"++ assert teardown_spy.call_count == teardown_calls_before + 1+ opened_library_id, _ = open_library_spy.spy_return+ assert teardown_spy.call_args_list[-1].kwargs["library_id"] == opened_library_id+ assert get_current_library_id_or_none() == current_before++ def test_empty_contents_are_a_host_wiring_error(self):+ with pytest.raises(PipelexUnexpectedError):+ resolve_crate_from_contents(mthds_contents=[])++ def test_sources_length_mismatch_is_a_host_wiring_error(self):+ with pytest.raises(PipelexUnexpectedError):+ resolve_crate_from_contents(mthds_contents=[MAIN_MTHDS], mthds_sources=["a.mthds", "b.mthds"])++ def test_address_dependency_is_rejected_without_filesystem_discovery(self, load_empty_library: Callable[[], str], mocker: MockerFixture) -> None:+ load_empty_library()+ discovery = mocker.patch(+ "pipelex.libraries.library_manager.find_method_by_full_address",+ side_effect=AssertionError("in-memory resolve must not discover installed methods"),+ )++ with pytest.raises(ValidateBundleError, match="address-based dependency"):+ resolve_crate_from_contents(mthds_contents=[ADDRESS_DEPENDENCY_MTHDS], mthds_sources=["host.mthds"])++ discovery.assert_not_called()diff --git a/tests/integration/pipelex/pipeline/test_validate_bundle_category_framing.py b/tests/integration/pipelex/pipeline/test_validate_bundle_clean_summary.pysimilarity index 50%rename from tests/integration/pipelex/pipeline/test_validate_bundle_category_framing.pyrename to tests/integration/pipelex/pipeline/test_validate_bundle_clean_summary.pyindex 3e508ab8a..fc687ac11 100644--- a/tests/integration/pipelex/pipeline/test_validate_bundle_category_framing.py+++ b/tests/integration/pipelex/pipeline/test_validate_bundle_clean_summary.py@@ -1,20 +1,19 @@-"""Pin: a pydantic ValidationError surfacing through the helper produces a clean summary (disease C).--The shared helper ``_translate_to_validate_bundle_error`` is invoked from four entry points —-``validate_bundle*`` (``category="pipe"``) and ``load_concepts_only*`` (``category="concept"``).-A pydantic ``ValidationError`` raised during model construction surfaces as a single-``ValidateBundleError`` whose top-line ``message`` must NOT leak the pydantic repr-(``Value errors: '<field>': Value error, …``) nor the old ``Could not load blueprints/concepts-because of:`` framing prefix that rode on it. Instead the message is the clean, author-facing-summary the constructor derives from the structured error items (``ValidateBundleError.__init__``-→ ``_summarize_bundle_validation_message``) — the same text a consumer reads off the first-``validation_errors[]`` item. This is verified identically for all four entry points because the-clean-summary invariant is category-independent.+"""Pin: a pydantic ValidationError surfacing through the shared helper produces a clean summary.++The shared bundle-loading cascade ``translate_to_validate_bundle_error`` serves every entry point+(``validate_bundle*``, ``resolve_crate_from_contents``). A pydantic ``ValidationError`` raised+during model construction surfaces as a single ``ValidateBundleError`` whose top-line ``message``+must NOT leak the pydantic repr (``Value errors: '<field>': Value error, …``) nor the old+``Could not load blueprints/concepts because of:`` framing prefix that rode on it. Instead the+message is the clean, author-facing summary the constructor derives from the structured error+items (``ValidateBundleError.__init__`` → ``_summarize_bundle_validation_message``) — the same+text a consumer reads off the first ``validation_errors[]`` item. The invariant lives on the+constructor, so it is entry-point-independent."""import tempfile-from collections.abc import Callablefrom pathlib import Path+from typing import Callableimport pytestfrom pydantic import BaseModel@@ -22,12 +21,7 @@ from pytest_mock import MockerFixturefrom pipelex.pipeline import validate_bundle as validate_bundle_modulefrom pipelex.pipeline.exceptions import ValidateBundleError-from pipelex.pipeline.validate_bundle import (- load_concepts_only,- load_concepts_only_from_directory,- validate_bundle,- validate_bundles_from_directory,-)+from pipelex.pipeline.validate_bundle import validate_bundle, validate_bundles_from_directoryclass _BrokenResult(BaseModel):@@ -50,7 +44,6 @@ name = { type = "text", description = "Customer name" }def _assert_clean_summary(exc: ValidateBundleError) -> None:"""The top-line message is the clean item summary — no pydantic-repr / framing leak."""- # Disease C: none of the pre-summary leak fragments survive on the top-line.assert "Value error" not in exc.messageassert "Validation error(s)" not in exc.messageassert "Could not load blueprints because of" not in exc.message@@ -62,7 +55,7 @@ def _assert_clean_summary(exc: ValidateBundleError) -> None:@pytest.mark.asyncio(loop_scope="class")-class TestValidateBundleCategoryFraming:+class TestValidateBundleCleanSummary:async def test_validate_bundle_message_is_clean_summary(self,load_empty_library: Callable[[], str],@@ -88,29 +81,3 @@ class TestValidateBundleCategoryFraming:with pytest.raises(ValidateBundleError) as exc_info:await validate_bundles_from_directory(directory=Path(tmp_dir))_assert_clean_summary(exc_info.value)-- async def test_load_concepts_only_message_is_clean_summary(- self,- load_empty_library: Callable[[], str],- mocker: MockerFixture,- ) -> None:- load_empty_library()- mocker.patch.object(validate_bundle_module, "LoadConceptsOnlyResult", _BrokenResult)-- with pytest.raises(ValidateBundleError) as exc_info:- load_concepts_only(mthds_contents=[_VALID_MTHDS])- _assert_clean_summary(exc_info.value)-- async def test_load_concepts_only_from_directory_message_is_clean_summary(- self,- load_empty_library: Callable[[], str],- mocker: MockerFixture,- ) -> None:- load_empty_library()- mocker.patch.object(validate_bundle_module, "LoadConceptsOnlyResult", _BrokenResult)-- with tempfile.TemporaryDirectory() as tmp_dir:- (Path(tmp_dir) / "test.mthds").write_text(_VALID_MTHDS, encoding="utf-8")- with pytest.raises(ValidateBundleError) as exc_info:- load_concepts_only_from_directory(directory=Path(tmp_dir))- _assert_clean_summary(exc_info.value)diff --git a/tests/integration/pipelex/pipeline/test_validate_bundle_library_lifecycle.py b/tests/integration/pipelex/pipeline/test_validate_bundle_library_lifecycle.pyindex 993044bc8..2d42d5e24 100644--- a/tests/integration/pipelex/pipeline/test_validate_bundle_library_lifecycle.py+++ b/tests/integration/pipelex/pipeline/test_validate_bundle_library_lifecycle.py@@ -1,10 +1,9 @@"""Pin: an opened library is torn down when bundle-validation fails.-Every bundle-loading entry point (``validate_bundle``,-``validate_bundles_from_directory``, ``load_concepts_only``,-``load_concepts_only_from_directory``) calls ``library_manager.open_library()``+Both bundle-loading entry points (``validate_bundle`` and+``validate_bundles_from_directory``) call ``library_manager.open_library()``before doing any work. When the body raises (including a translated-``ValidateBundleError`` from ``_translate_to_validate_bundle_error``), the+``ValidateBundleError`` from ``translate_to_validate_bundle_error``), theopened library must be torn down so the process does not accumulate oneun-torn-down ``Library`` per failed validation.@@ -35,8 +34,6 @@ from pipelex.hub import clear_current_library, get_current_library, get_library_from pipelex.pipeline import validate_bundle as validate_bundle_modulefrom pipelex.pipeline.exceptions import ValidateBundleErrorfrom pipelex.pipeline.validate_bundle import (- load_concepts_only,- load_concepts_only_from_directory,validate_bundle,validate_bundles_from_directory,)@@ -109,46 +106,6 @@ class TestValidateBundleLibraryLifecycle:opened_library_id, _ = open_library_spy.spy_returnassert latest_call.kwargs["library_id"] == opened_library_id- async def test_load_concepts_only_tears_down_library_on_translated_error(- self,- load_empty_library: Callable[[], str],- mocker: MockerFixture,- ) -> None:- load_empty_library()- library_manager = get_library_manager()- open_library_spy = mocker.spy(library_manager, "open_library")- teardown_spy = mocker.spy(library_manager, "teardown")- mocker.patch.object(validate_bundle_module, "LoadConceptsOnlyResult", _BrokenResult)-- teardown_calls_before = teardown_spy.call_count- with pytest.raises(ValidateBundleError):- load_concepts_only(mthds_contents=[_VALID_MTHDS])- assert teardown_spy.call_count == teardown_calls_before + 1- latest_call = teardown_spy.call_args_list[-1]- opened_library_id, _ = open_library_spy.spy_return- assert latest_call.kwargs["library_id"] == opened_library_id-- async def test_load_concepts_only_from_directory_tears_down_library_on_translated_error(- self,- load_empty_library: Callable[[], str],- mocker: MockerFixture,- ) -> None:- load_empty_library()- library_manager = get_library_manager()- open_library_spy = mocker.spy(library_manager, "open_library")- teardown_spy = mocker.spy(library_manager, "teardown")- mocker.patch.object(validate_bundle_module, "LoadConceptsOnlyResult", _BrokenResult)-- with tempfile.TemporaryDirectory() as tmp_dir:- (Path(tmp_dir) / "test.mthds").write_text(_VALID_MTHDS, encoding="utf-8")- teardown_calls_before = teardown_spy.call_count- with pytest.raises(ValidateBundleError):- load_concepts_only_from_directory(directory=Path(tmp_dir))- assert teardown_spy.call_count == teardown_calls_before + 1- latest_call = teardown_spy.call_args_list[-1]- opened_library_id, _ = open_library_spy.spy_return- assert latest_call.kwargs["library_id"] == opened_library_id-async def test_validate_bundle_tears_down_on_base_exception_in_pre_try_window(self,load_empty_library: Callable[[], str],@@ -219,36 +176,6 @@ class TestValidateBundleLibraryLifecycle:opened_library_id, _ = open_library_spy.spy_returnassert latest_call.kwargs["library_id"] == opened_library_id- async def test_load_concepts_only_tears_down_on_resolve_library_dirs_failure(- self,- load_empty_library: Callable[[], str],- mocker: MockerFixture,- ) -> None:- """Pin: same pre-try-leak guarantee for the concepts-only entry point.-- The concepts-only path also calls ``resolve_library_dirs`` between- ``open_library`` and the helper ``with`` block. The fix applies- symmetrically to both entry points that consume ``library_dirs``.- """- load_empty_library()- library_manager = get_library_manager()- open_library_spy = mocker.spy(library_manager, "open_library")- teardown_spy = mocker.spy(library_manager, "teardown")- mocker.patch.object(- validate_bundle_module,- "resolve_library_dirs",- side_effect=TypeError("simulated invalid library_dirs element"),- )-- teardown_calls_before = teardown_spy.call_count- with pytest.raises(TypeError):- load_concepts_only(mthds_contents=[_VALID_MTHDS])-- assert teardown_spy.call_count == teardown_calls_before + 1- latest_call = teardown_spy.call_args_list[-1]- opened_library_id, _ = open_library_spy.spy_return- assert latest_call.kwargs["library_id"] == opened_library_id-@pytest.mark.asyncio(loop_scope="class")class TestValidateBundleRestoresOuterLibraryOnFailure:@@ -261,9 +188,8 @@ class TestValidateBundleRestoresOuterLibraryOnFailure:set — clearing it strands every subsequent ``get_current_library()`` inthe same async context with ``RuntimeError: No current library set``.- Covers all four entry points that touch ``_library_id``:- ``validate_bundle``, ``validate_bundles_from_directory``,- ``load_concepts_only``, ``load_concepts_only_from_directory``.+ Covers both entry points that touch ``_library_id``:+ ``validate_bundle`` and ``validate_bundles_from_directory``."""async def test_validate_bundle_restores_previous_current_library(@@ -297,35 +223,3 @@ class TestValidateBundleRestoresOuterLibraryOnFailure:assert get_current_library() == outer_library_idfinally:clear_current_library()-- async def test_load_concepts_only_restores_previous_current_library(- self,- load_empty_library: Callable[[], str],- mocker: MockerFixture,- ) -> None:- outer_library_id = load_empty_library()- set_current_library(library_id=outer_library_id)- try:- mocker.patch.object(validate_bundle_module, "LoadConceptsOnlyResult", _BrokenResult)- with pytest.raises(ValidateBundleError):- load_concepts_only(mthds_contents=[_VALID_MTHDS])- assert get_current_library() == outer_library_id- finally:- clear_current_library()-- async def test_load_concepts_only_from_directory_restores_previous_current_library(- self,- load_empty_library: Callable[[], str],- mocker: MockerFixture,- ) -> None:- outer_library_id = load_empty_library()- set_current_library(library_id=outer_library_id)- try:- mocker.patch.object(validate_bundle_module, "LoadConceptsOnlyResult", _BrokenResult)- with tempfile.TemporaryDirectory() as tmp_dir:- (Path(tmp_dir) / "test.mthds").write_text(_VALID_MTHDS, encoding="utf-8")- with pytest.raises(ValidateBundleError):- load_concepts_only_from_directory(directory=Path(tmp_dir))- assert get_current_library() == outer_library_id- finally:- clear_current_library()diff --git a/tests/integration/pipelex/pipeline/test_validate_bundle_return_placement.py b/tests/integration/pipelex/pipeline/test_validate_bundle_return_placement.pyindex b5ce4b97c..816c1b6de 100644--- a/tests/integration/pipelex/pipeline/test_validate_bundle_return_placement.py+++ b/tests/integration/pipelex/pipeline/test_validate_bundle_return_placement.py@@ -1,16 +1,15 @@"""Pin: result-class construction errors are translated to ``ValidateBundleError`` uniformly.-All four bundle-loading entry points (``validate_bundle``,-``validate_bundles_from_directory``, ``load_concepts_only``,-``load_concepts_only_from_directory``) must place their final ``return`` inside-the ``with _translate_to_validate_bundle_error()`` block. A+Both bundle-loading entry points (``validate_bundle`` and+``validate_bundles_from_directory``) must place their final ``return`` inside+the ``with translate_to_validate_bundle_error()`` block. A``pydantic.ValidationError`` raised from result-class construction thereforesurfaces as a single ``ValidateBundleError`` envelope regardless of entry point.-Without this guarantee, the four entry points split into two shapes — two-returning inside the ``with`` (translated) and two outside (raw-``pydantic.ValidationError`` propagates) — and downstream handlers that only-``except ValidateBundleError`` would miss the latter two.+Without this guarantee, the entry points split into two shapes — one returning+inside the ``with`` (translated) and one outside (raw ``pydantic.ValidationError``+propagates) — and downstream handlers that only ``except ValidateBundleError``+would miss the latter."""import tempfile@@ -24,8 +23,6 @@ from pytest_mock import MockerFixturefrom pipelex.pipeline import validate_bundle as validate_bundle_modulefrom pipelex.pipeline.exceptions import ValidateBundleErrorfrom pipelex.pipeline.validate_bundle import (- load_concepts_only,- load_concepts_only_from_directory,validate_bundle,validate_bundles_from_directory,)@@ -79,27 +76,3 @@ class TestValidateBundleResultConstructionErrorTranslation:(Path(tmp_dir) / "test.mthds").write_text(_VALID_MTHDS, encoding="utf-8")with pytest.raises(ValidateBundleError):await validate_bundles_from_directory(directory=Path(tmp_dir))-- async def test_load_concepts_only_translates_result_construction_error(- self,- load_empty_library: Callable[[], str],- mocker: MockerFixture,- ) -> None:- load_empty_library()- mocker.patch.object(validate_bundle_module, "LoadConceptsOnlyResult", _BrokenResult)-- with pytest.raises(ValidateBundleError):- load_concepts_only(mthds_contents=[_VALID_MTHDS])-- async def test_load_concepts_only_from_directory_translates_result_construction_error(- self,- load_empty_library: Callable[[], str],- mocker: MockerFixture,- ) -> None:- load_empty_library()- mocker.patch.object(validate_bundle_module, "LoadConceptsOnlyResult", _BrokenResult)-- with tempfile.TemporaryDirectory() as tmp_dir:- (Path(tmp_dir) / "test.mthds").write_text(_VALID_MTHDS, encoding="utf-8")- with pytest.raises(ValidateBundleError):- load_concepts_only_from_directory(directory=Path(tmp_dir))diff --git a/tests/unit/pipelex/builder/test_runner_generator.py b/tests/unit/pipelex/builder/test_runner_generator.pyindex 9f1202df8..17d1fa5fa 100644--- a/tests/unit/pipelex/builder/test_runner_generator.py+++ b/tests/unit/pipelex/builder/test_runner_generator.py@@ -266,7 +266,7 @@ class TestGenerateRunnerCode:assert "[DocumentContent(" in runner_codedef test_runner_code_custom_class_import_format(self) -> None:- """Test that custom class imports use domain_conceptCode format for standalone scripts."""+ """Custom classes import from the single-module types projection (structures/structures.py)."""# Create a custom concept (non-native)custom_concept = ConceptFactory.make(domain_code="test_domain",@@ -282,10 +282,31 @@ class TestGenerateRunnerCode:mock_pipe.inputs = InputStuffSpecs(root={"document": StuffSpec(concept=document_concept)})runner_code = generate_runner_code(mock_pipe)- # Custom imports should NOT use relative import (no leading dot) for standalone scripts- assert "from structures.test_domain__custom_output import CustomOutput" in runner_code+ # Custom imports come from the emitted single module; NOT a relative import (standalone script)+ assert "from structures.structures import CustomOutput" in runner_codeassert "from .structures" not in runner_code+ def test_runner_code_class_name_overrides_spell_emitted_names(self) -> None:+ """The overrides map re-spells runtime-qualified class names to the emitted (bare) names."""+ custom_concept = ConceptFactory.make(+ domain_code="test_domain",+ concept_code="CustomOutput",+ description="A custom output",+ structure_class_name="test_domain__CustomOutput",+ )+ text_concept = ConceptFactory.make_native_concept(NativeConceptCode.TEXT)++ mock_pipe = MagicMock()+ mock_pipe.code = "custom_pipe"+ mock_pipe.output = StuffSpec(concept=custom_concept)+ mock_pipe.inputs = InputStuffSpecs(root={"message": StuffSpec(concept=text_concept)})++ runner_code = generate_runner_code(mock_pipe, class_name_overrides={"test_domain__CustomOutput": "CustomOutput"})+ assert "from structures.structures import CustomOutput" in runner_code+ assert "async def run_custom_pipe() -> CustomOutput:" in runner_code+ assert "pipe_output.main_stuff_as(content_type=CustomOutput)" in runner_code+ assert "test_domain__CustomOutput" not in runner_code+def test_runner_code_anything_output_uses_any_return_type(self) -> None:"""Test that Anything output concept uses Any return type and main_stuff instead of main_stuff_as."""anything_concept = ConceptFactory.make_native_concept(NativeConceptCode.ANYTHING)diff --git a/tests/unit/pipelex/cli/commands/build/test_runner_core.py b/tests/unit/pipelex/cli/commands/build/test_runner_core.pyindex b3136c0ba..0d89d8c30 100644--- a/tests/unit/pipelex/cli/commands/build/test_runner_core.py+++ b/tests/unit/pipelex/cli/commands/build/test_runner_core.py@@ -1,7 +1,8 @@"""Unit tests for the `pipelex build runner` core logic (_prepare_runner_core).-The registry getters are mocked at the module namespace so the real class/func-registries of the module-scoped Pipelex are never torn down by these tests.+The library plumbing and the codegen engine seams are mocked at the module namespace, so these+tests pin the runner-core wiring (pipe selection, output paths, the structures projection hand-off,+and the exit codes), not the engine itself (covered by the codegen unit tests)."""from __future__ import annotations@@ -27,21 +28,25 @@ MODULE = "pipelex.cli.commands.build.runner._runner_core"class TestPrepareRunnerCore:@pytest.fixturedef core_mocks(self, mocker: MockerFixture) -> dict[str, Any]:- """Stub library plumbing, registries, structure generation and runner codegen."""+ """Stub library plumbing, the codegen engine seams, and runner codegen."""library_manager = mocker.MagicMock()- library_manager.open_library.return_value = ("lib_id", mocker.MagicMock())+ library_manager.get_crate.return_value = None # default: no crate -> no structures projectionmocker.patch(f"{MODULE}.get_library_manager", return_value=library_manager)- mocker.patch(f"{MODULE}.set_current_library")- mocker.patch(f"{MODULE}.resolve_library_dirs", return_value=([], "defaults"))- mocker.patch(f"{MODULE}.get_class_registry", return_value=mocker.MagicMock())- mocker.patch(f"{MODULE}.get_func_registry", return_value=mocker.MagicMock())- mocker.patch(f"{MODULE}.ClassRegistryUtils")+ mocker.patch(f"{MODULE}.get_current_library_id_or_none", return_value="lib_id")blueprint = SimpleNamespace(main_pipe="bundle_main", pipe=None)validate_result = SimpleNamespace(blueprints=[blueprint])return {+ "library_manager": library_manager,"validate_bundle": mocker.patch(f"{MODULE}.validate_bundle", new=mocker.AsyncMock(return_value=validate_result)),"get_required_pipe": mocker.patch(f"{MODULE}.get_required_pipe", return_value=SimpleNamespace(code="bundle_main")),- "generate_structures": mocker.patch(f"{MODULE}.generate_structures_from_blueprints", return_value=[]),+ "normalize_crate": mocker.patch(f"{MODULE}.normalize_crate"),+ "emit_types": mocker.patch(f"{MODULE}.emit_types", return_value=[]),+ "write_stamped_projection": mocker.patch(+ f"{MODULE}.write_stamped_projection",+ return_value=SimpleNamespace(written=["structures.py"], unchanged=[], removed=[]),+ ),+ "resolve_concepts": mocker.patch(f"{MODULE}.resolve_concepts_from_crate"),+ "runtime_to_emitted": mocker.patch(f"{MODULE}.runtime_to_emitted_class_names", return_value={"demo__Invoice": "Invoice"}),"generate_runner_code": mocker.patch(f"{MODULE}.generate_runner_code", return_value="# runner code\n"),}@@ -56,15 +61,26 @@ class TestPrepareRunnerCore:assert codegen_kwargs["output_multiplicity"] is Falseassert codegen_kwargs["library_dir"] == str(tmp_path.resolve())- def test_structures_generated_into_output_dir(self, core_mocks: dict[str, Any], tmp_path: Path) -> None:- """Structure files are generated into a structures/ dir next to the runner."""- core_mocks["generate_structures"].return_value = [("demo", "Invoice")]+ def test_structures_projection_emitted_into_structures_dir(self, core_mocks: dict[str, Any], tmp_path: Path) -> None:+ """With a crate available, the types projection is written into structures/ next to the+ runner and the emitted-name mapping is handed to the runner-code generator.+ """+ core_mocks["library_manager"].get_crate.return_value = SimpleNamespace()asyncio.run(_prepare_runner_core(pipe_code=None, bundle_path=tmp_path / "demo.mthds"))- structures_kwargs = core_mocks["generate_structures"].call_args.kwargs- assert structures_kwargs["output_directory"] == tmp_path / "structures"- assert structures_kwargs["target_path"] == tmp_path+ write_kwargs = core_mocks["write_stamped_projection"].call_args.kwargs+ assert write_kwargs["output_dir"] == tmp_path / "structures"+ codegen_kwargs = core_mocks["generate_runner_code"].call_args.kwargs+ assert codegen_kwargs["class_name_overrides"] == {"demo__Invoice": "Invoice"}++ def test_no_crate_skips_structures_projection(self, core_mocks: dict[str, Any], tmp_path: Path) -> None:+ """Without a crate (nothing loaded), the runner is still generated with no overrides."""+ asyncio.run(_prepare_runner_core(pipe_code=None, bundle_path=tmp_path / "demo.mthds"))++ core_mocks["write_stamped_projection"].assert_not_called()+ codegen_kwargs = core_mocks["generate_runner_code"].call_args.kwargs+ assert codegen_kwargs["class_name_overrides"] == {}@pytest.mark.usefixtures("core_mocks")def test_explicit_output_path_wins(self, tmp_path: Path) -> None:diff --git a/tests/unit/pipelex/cli/commands/build/test_structures_alias.py b/tests/unit/pipelex/cli/commands/build/test_structures_alias.pynew file mode 100644index 000000000..5ad81daec--- /dev/null+++ b/tests/unit/pipelex/cli/commands/build/test_structures_alias.py@@ -0,0 +1,74 @@+"""The `pipelex build structures` alias: thin delegation to `codegen types --target python-structures`.++The engine itself (crate loading, emission, stamping) is covered by the codegen unit tests; these+pin the alias wiring only — target mapping (file -> parent directory, directory -> itself), the+default output directory, and the missing-target exit code.+"""++from __future__ import annotations++from typing import TYPE_CHECKING++import pytest+import typer++from pipelex.cli.commands.build.structures_cmd import build_structures_command+from pipelex.codegen.emitters.target import CodegenTarget++if TYPE_CHECKING:+ from pathlib import Path++ from pytest_mock import MockerFixture++MODULE = "pipelex.cli.commands.build.structures_cmd"+++class TestBuildStructuresAlias:+ def test_directory_target_delegates_with_default_output(self, mocker: MockerFixture, tmp_path: Path) -> None:+ delegate = mocker.patch(f"{MODULE}.codegen_types_cmd")++ build_structures_command(target=str(tmp_path), output_dir=None, library_dir=None)++ delegate.assert_called_once_with(+ target=CodegenTarget.PYTHON_STRUCTURES,+ paths=[tmp_path],+ output_dir=str(tmp_path / "structures"),+ library_dir=None,+ )++ def test_file_target_resolves_to_parent_directory(self, mocker: MockerFixture, tmp_path: Path) -> None:+ delegate = mocker.patch(f"{MODULE}.codegen_types_cmd")+ bundle_file = tmp_path / "demo.mthds"+ bundle_file.write_text('domain = "demo"\n', encoding="utf-8")++ build_structures_command(target=str(bundle_file), output_dir=None, library_dir=None)++ delegate.assert_called_once_with(+ target=CodegenTarget.PYTHON_STRUCTURES,+ paths=[tmp_path],+ output_dir=str(tmp_path / "structures"),+ library_dir=None,+ )++ def test_explicit_output_dir_and_library_dirs_pass_through(self, mocker: MockerFixture, tmp_path: Path) -> None:+ delegate = mocker.patch(f"{MODULE}.codegen_types_cmd")+ shared_dir = tmp_path / "shared"+ shared_dir.mkdir()++ build_structures_command(target=str(tmp_path), output_dir=str(tmp_path / "generated"), library_dir=[str(shared_dir)])++ delegate.assert_called_once_with(+ target=CodegenTarget.PYTHON_STRUCTURES,+ paths=[tmp_path],+ output_dir=str(tmp_path / "generated"),+ library_dir=[shared_dir],+ )++ def test_missing_target_is_no_verdict_exit_2(self, mocker: MockerFixture, tmp_path: Path) -> None:+ delegate = mocker.patch(f"{MODULE}.codegen_types_cmd")++ with pytest.raises(typer.Exit) as exc_info:+ build_structures_command(target=str(tmp_path / "does_not_exist"), output_dir=None, library_dir=None)++ assert exc_info.value.exit_code == 2+ delegate.assert_not_called()diff --git a/tests/unit/pipelex/cli/commands/build/test_structures_cmd.py b/tests/unit/pipelex/cli/commands/build/test_structures_cmd.pydeleted file mode 100644index 5862ba6b4..000000000--- a/tests/unit/pipelex/cli/commands/build/test_structures_cmd.py+++ /dev/null@@ -1,217 +0,0 @@-"""Unit tests for the structure-codegen logic in build/structures_cmd.py.--Drives generate_structures_from_blueprints and its helpers directly with real-blueprints and a tmp output dir — the same pattern as the cross-package refines-regression test next door.-"""--from __future__ import annotations--from pathlib import Path-from typing import TYPE_CHECKING--from pipelex.cli.commands.build.structures_cmd import (- _build_concept_ref_to_class_info, # noqa: PLC2701 # pyright: ignore[reportPrivateUsage]- _compute_relative_path_from_output_dir, # noqa: PLC2701 # pyright: ignore[reportPrivateUsage]- generate_structures_from_blueprints,-)-from pipelex.core.bundles.pipelex_bundle_blueprint import PipelexBundleBlueprint-from pipelex.core.concepts.concept_blueprint import ConceptBlueprint-from pipelex.core.stuffs.text_content import TextContent-from pipelex.hub import get_class_registry--if TYPE_CHECKING:- import pytest- from pytest_mock import MockerFixture---class TestStructuresCmd:- def test_relative_path_inside_cwd(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:- """An output dir under the cwd yields its relative path."""- monkeypatch.chdir(tmp_path)- output_directory = tmp_path / "generated" / "structures"- output_directory.mkdir(parents=True)-- assert _compute_relative_path_from_output_dir(output_directory) == Path("generated/structures")-- def test_relative_path_outside_cwd(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:- """An output dir outside the cwd yields None."""- inside_dir = tmp_path / "inside"- inside_dir.mkdir()- outside_dir = tmp_path / "outside"- outside_dir.mkdir()- monkeypatch.chdir(inside_dir)-- assert _compute_relative_path_from_output_dir(outside_dir) is None-- def test_relative_path_equal_to_cwd(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:- """An output dir equal to the cwd yields the empty relative path."""- monkeypatch.chdir(tmp_path)-- assert _compute_relative_path_from_output_dir(tmp_path) == Path()-- def test_concept_ref_mapping_skips_native_and_qualifies_names(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:- """The concept_ref map skips the native domain, qualifies bare names and builds module paths."""- monkeypatch.chdir(tmp_path)- output_directory = tmp_path / "structures"- output_directory.mkdir()- blueprints = [- PipelexBundleBlueprint(domain="native", description="native stub", concept={"NativeStub": "native stub concept"}),- PipelexBundleBlueprint(- domain="billing",- description="billing concepts",- concept={- "Invoice": ConceptBlueprint(description="An invoice", structure={"total": "the total"}),- "CustomName": ConceptBlueprint(description="Custom structure ref", structure="MyCustomClass"),- },- ),- ]-- mapping = _build_concept_ref_to_class_info(blueprints, output_directory=output_directory)-- assert set(mapping.keys()) == {"billing.Invoice", "billing.CustomName"}- assert mapping["billing.Invoice"].class_name == "billing__Invoice"- assert mapping["billing.Invoice"].module_path == "structures.billing__invoice"- assert mapping["billing.CustomName"].class_name == "MyCustomClass"- assert mapping["billing.CustomName"].module_path == "structures.billing__my_custom_class"-- def test_concept_ref_mapping_without_relative_path(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:- """When the output dir is outside the cwd, module paths are None."""- inside_dir = tmp_path / "inside"- inside_dir.mkdir()- outside_dir = tmp_path / "outside"- outside_dir.mkdir()- monkeypatch.chdir(inside_dir)- blueprints = [- PipelexBundleBlueprint(- domain="billing",- description="billing concepts",- concept={"Invoice": ConceptBlueprint(description="An invoice")},- ),- ]-- mapping = _build_concept_ref_to_class_info(blueprints, output_directory=outside_dir)-- assert mapping["billing.Invoice"].module_path is None-- def test_generate_string_concept_defaults_to_text_content(self, tmp_path: Path) -> None:- """A bare string concept generates a TextContent subclass file."""- blueprints = [- PipelexBundleBlueprint(domain="notes", description="notes", concept={"QuickNote": "A short note"}),- ]-- generated = generate_structures_from_blueprints(blueprints, output_directory=tmp_path, skip_existing_check=True, quiet=True)-- assert generated == [("notes", "QuickNote")]- generated_code = (tmp_path / "notes__quick_note.py").read_text(encoding="utf-8")- assert "class notes__QuickNote(TextContent):" in generated_code- assert (tmp_path / "__init__.py").exists()-- def test_generate_explicit_structure(self, tmp_path: Path) -> None:- """An explicit structure dict generates a structured class with its fields."""- blueprints = [- PipelexBundleBlueprint(- domain="billing",- description="billing",- concept={- "Invoice": ConceptBlueprint(- description="An invoice",- structure={"total": "the invoice total", "customer_name": "the customer"},- ),- },- ),- ]-- generated = generate_structures_from_blueprints(blueprints, output_directory=tmp_path, skip_existing_check=True, quiet=True)-- assert generated == [("billing", "Invoice")]- generated_code = (tmp_path / "billing__invoice.py").read_text(encoding="utf-8")- assert "class billing__Invoice" in generated_code- assert "total" in generated_code- assert "customer_name" in generated_code-- def test_generate_refines_native_concept(self, tmp_path: Path) -> None:- """A concept refining a native concept inherits from its Content class."""- blueprints = [- PipelexBundleBlueprint(- domain="notes",- description="notes",- concept={"FancyText": ConceptBlueprint(description="Fancy text", refines="Text")},- ),- ]-- generated = generate_structures_from_blueprints(blueprints, output_directory=tmp_path, skip_existing_check=True, quiet=True)-- assert generated == [("notes", "FancyText")]- generated_code = (tmp_path / "notes__fancy_text.py").read_text(encoding="utf-8")- assert "class notes__FancyText(TextContent):" in generated_code-- def test_generate_default_concept_blueprint(self, tmp_path: Path) -> None:- """A blueprint with neither structure nor refines defaults to TextContent."""- blueprints = [- PipelexBundleBlueprint(- domain="notes",- description="notes",- concept={"PlainThing": ConceptBlueprint(description="Just a thing")},- ),- ]-- generated = generate_structures_from_blueprints(blueprints, output_directory=tmp_path, skip_existing_check=True, quiet=True)-- assert generated == [("notes", "PlainThing")]- generated_code = (tmp_path / "notes__plain_thing.py").read_text(encoding="utf-8")- assert "class notes__PlainThing(TextContent):" in generated_code-- def test_generate_skips_native_domain_and_writes_no_init(self, tmp_path: Path) -> None:- """Native-domain blueprints generate nothing — not even an __init__.py."""- blueprints = [- PipelexBundleBlueprint(domain="native", description="native stub", concept={"NativeStub": "native stub concept"}),- ]-- generated = generate_structures_from_blueprints(blueprints, output_directory=tmp_path, skip_existing_check=True, quiet=True)-- assert generated == []- assert not (tmp_path / "__init__.py").exists()- assert list(tmp_path.iterdir()) == []-- def test_generate_skips_manually_created_class(self, tmp_path: Path) -> None:- """With existing-class checking on, a registered class suppresses generation."""- manual_class: type = type("ManualDoctorTestConcept", (TextContent,), {})- get_class_registry().register_class(manual_class, name="ManualDoctorTestConcept", should_warn_if_already_registered=False)- try:- blueprints = [- PipelexBundleBlueprint(- domain="notes",- description="notes",- concept={"ManualDoctorTestConcept": ConceptBlueprint(description="Manually owned")},- ),- ]-- generated = generate_structures_from_blueprints(- blueprints,- output_directory=tmp_path,- target_path=tmp_path,- skip_existing_check=False,- quiet=True,- )-- assert generated == []- assert not (tmp_path / "notes__manual_doctor_test_concept.py").exists()- finally:- # The class registry is process-global — leaving the class registered would make later tests order-dependent- get_class_registry().unregister_class_by_name("ManualDoctorTestConcept")-- def test_generate_non_quiet_echoes_progress(self, tmp_path: Path, mocker: MockerFixture) -> None:- """Without quiet, progress is echoed through typer."""- echo_mock = mocker.patch("pipelex.cli.commands.build.structures_cmd.typer.echo")- secho_mock = mocker.patch("pipelex.cli.commands.build.structures_cmd.typer.secho")- blueprints = [- PipelexBundleBlueprint(domain="notes", description="notes", concept={"LoudNote": "A loud note"}),- ]-- generate_structures_from_blueprints(blueprints, output_directory=tmp_path, skip_existing_check=True, quiet=False)-- echo_mock.assert_called_once()- secho_calls = [str(call.args[0]) for call in secho_mock.call_args_list]- assert any("notes__loud_note.py" in message for message in secho_calls)- assert any("__init__.py" in message for message in secho_calls)diff --git a/tests/unit/pipelex/cli/commands/build/test_structures_cmd_cross_package_refines.py b/tests/unit/pipelex/cli/commands/build/test_structures_cmd_cross_package_refines.pydeleted file mode 100644index c347de78d..000000000--- a/tests/unit/pipelex/cli/commands/build/test_structures_cmd_cross_package_refines.py+++ /dev/null@@ -1,101 +0,0 @@-"""Regression test for cross-package refines in structures_cmd.--Ensures that when a concept refines a cross-package reference like-``alias->other_domain.OtherConcept``, the generated structure file uses a-valid Python identifier as its base class. Previously, ``QualifiedRef.parse``-left the ``alias->`` prefix in ``domain_path``, which the-``make_qualified_structure_class_name`` helper does not sanitise — the-generated module then failed to import because the base class name was not a-legal identifier.-"""--import re-import sys-import tempfile-from pathlib import Path-from types import ModuleType--from pipelex.cli.commands.build.structures_cmd import generate_structures_from_blueprints-from pipelex.core.bundles.pipelex_bundle_blueprint import PipelexBundleBlueprint-from pipelex.core.concepts.concept_blueprint import ConceptBlueprint-from pipelex.core.stuffs.structured_content import StructuredContent-from pipelex.hub import get_class_registry--_FAKE_BASE_MODULE_PATH = "tests_pr898_other_pkg_stub.structures.other_domain__other_concept"---class TestStructuresCmdCrossPackageRefines:- @classmethod- def _register_cross_package_stub(cls) -> type:- """Register a stub mimicking what another package's `pipelex build structures` would have- produced: a class whose `__name__` equals the registry key and that lives at a stable- importable module path, so the generator can recover the import path via `__module__`.- """- stub_module = ModuleType(_FAKE_BASE_MODULE_PATH)- stub_class: type = type("other_domain__OtherConcept", (StructuredContent,), {"__module__": _FAKE_BASE_MODULE_PATH})- stub_module.other_domain__OtherConcept = stub_class # type: ignore[attr-defined]- sys.modules[_FAKE_BASE_MODULE_PATH] = stub_module- get_class_registry().register_class(- stub_class,- name="other_domain__OtherConcept",- should_warn_if_already_registered=False,- )- return stub_class-- def test_cross_package_refine_produces_valid_identifier(self):- """A cross-package refine must produce a clean qualified base class name.-- The generated code must contain the domain-qualified class name- (e.g. ``other_domain__OtherConcept``) and must NOT leak the- cross-package alias separator ``->`` into the Python source. The- generated file must also import the base class so it is loadable- outside the generator's validation harness.- """- self._register_cross_package_stub()-- bundle = PipelexBundleBlueprint(- domain="local_domain",- description="Bundle that refines a cross-package concept.",- concept={- "RefiningConcept": ConceptBlueprint(- description="Refines a concept defined in another package.",- refines="some_alias->other_domain.OtherConcept",- ),- },- )-- try:- with tempfile.TemporaryDirectory() as temp_dir:- output_directory = Path(temp_dir)-- generated_files = generate_structures_from_blueprints(- blueprints=[bundle],- output_directory=output_directory,- skip_existing_check=True,- quiet=True,- )-- assert generated_files == [("local_domain", "RefiningConcept")]-- generated_file = output_directory / "local_domain__refining_concept.py"- assert generated_file.exists(), f"Generated file not found: {generated_file}"-- generated_code = generated_file.read_text()-- assert "->" not in generated_code, (- "Generated code leaked the cross-package alias separator '->' (produces an invalid Python identifier):\n" + generated_code- )- assert "some_alias" not in generated_code, (- "Generated code leaked the cross-package alias name into the class definition:\n" + generated_code- )- assert "class local_domain__RefiningConcept(other_domain__OtherConcept):" in generated_code, (- "Expected generated class to inherit from the qualified cross-package base class. Got:\n" + generated_code- )- import_pattern = re.compile(r"^from\s+\S+\s+import\s+other_domain__OtherConcept(\s|$)", re.MULTILINE)- assert import_pattern.search(generated_code), (- "Generated code is missing an import for the cross-package base class "- "`other_domain__OtherConcept`. Without it, importing the generated module fails with "- "`NameError: name 'other_domain__OtherConcept' is not defined`. Got:\n" + generated_code- )- finally:- sys.modules.pop(_FAKE_BASE_MODULE_PATH, None)diff --git a/tests/unit/pipelex/cli/commands/run/test_inputs_file_loader.py b/tests/unit/pipelex/cli/commands/run/test_inputs_file_loader.pyindex 6445648f8..80ebbf076 100644--- a/tests/unit/pipelex/cli/commands/run/test_inputs_file_loader.py+++ b/tests/unit/pipelex/cli/commands/run/test_inputs_file_loader.py@@ -9,8 +9,8 @@ from typing import TYPE_CHECKINGimport pytestfrom pipelex.cli.commands.run._inputs_file_loader import load_inputs_dict_from_path # noqa: PLC2701 # pyright: ignore[reportPrivateUsage]-from pipelex.cli.commands.run.exceptions import InputsTimeOnlyNotSupportedErrorfrom pipelex.core.stuffs.date_content import DateContent+from pipelex.core.stuffs.time_content import TimeContentfrom pipelex.tools.misc.exceptions import JsonTypeError, TomlErrorif TYPE_CHECKING:@@ -119,27 +119,27 @@ class TestInputsFileLoader:assert content.time is not Noneassert content.time.utcoffset() == datetime.timedelta(hours=2)- def test_top_level_local_time_rejected(self, tmp_path: Path) -> None:- """A bare TOML time-of-day is still rejected — it has no date to attach to."""+ def test_top_level_local_time_becomes_time_content(self, tmp_path: Path) -> None:+ """A top-level bare TOML time-of-day maps to a TimeContent (native Time), never a Date."""inputs_file = tmp_path / "inputs.toml"inputs_file.write_text("opening = 09:00:00\n", encoding="utf-8")- with pytest.raises(InputsTimeOnlyNotSupportedError) as exc_info:- load_inputs_dict_from_path(inputs_file)- assert "opening" in exc_info.value.message- assert "time of day alone" in exc_info.value.message+ loaded = load_inputs_dict_from_path(inputs_file)++ assert loaded["opening"] == TimeContent(time=datetime.time(9, 0))+ assert loaded["opening"].time.tzinfo is None- def test_nested_time_rejected_with_key_path(self, tmp_path: Path) -> None:- """A time-of-day nested in an envelope's content is rejected, with its key path in the message."""+ def test_nested_time_left_in_place(self, tmp_path: Path) -> None:+ """A time-of-day nested in an envelope's content is left for the factory/pydantic to consume."""inputs_file = tmp_path / "inputs.toml"inputs_file.write_text('[record]\nconcept = "Event"\n[[record.entries]]\nlabel = "kickoff"\nat = 09:00:00\n',encoding="utf-8",)- with pytest.raises(InputsTimeOnlyNotSupportedError) as exc_info:- load_inputs_dict_from_path(inputs_file)- assert "record.entries[0].at" in exc_info.value.message+ loaded = load_inputs_dict_from_path(inputs_file)++ assert loaded["record"]["entries"][0]["at"] == datetime.time(9, 0)def test_nested_datetime_left_in_place(self, tmp_path: Path) -> None:"""A date/datetime nested in an envelope's content is left for the factory/pydantic to consume."""diff --git a/tests/unit/pipelex/cli/test_agent_codegen_check_cmd.py b/tests/unit/pipelex/cli/test_agent_codegen_check_cmd.pynew file mode 100644index 000000000..380a3112b--- /dev/null+++ b/tests/unit/pipelex/cli/test_agent_codegen_check_cmd.py@@ -0,0 +1,94 @@+"""The `pipelex-agent codegen check` command: envelopes and the 0/1/2 verdict exit codes.++The offline check core is mocked out (covered by its own unit tests) so these tests pin the+agent-CLI wiring: an up-to-date verdict is a success envelope, drift is a structured+`CodegenDriftError` on stderr with exit 1, and a missing/unreadable lock is a no-verdict exit 2.+"""++from __future__ import annotations++import json+from pathlib import Path+from typing import TYPE_CHECKING++import pytest+import typer++from pipelex.cli.agent_cli.commands.agent_output import CliOutputFormat+from pipelex.cli.agent_cli.commands.codegen.check_cmd import agent_codegen_check_cmd+from pipelex.codegen.check import CodegenCheckReport, CodegenDrift, DriftCategory+from pipelex.codegen.exceptions import CodegenLockError++if TYPE_CHECKING:+ from pytest_mock import MockerFixture++CMD_MODULE = "pipelex.cli.agent_cli.commands.codegen.check_cmd"+++class TestAgentCodegenCheckCmd:+ """The agent-CLI offline drift check, with the pure-hashing core mocked out."""++ def test_current_is_json_success_envelope(self, mocker: MockerFixture, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None:+ mocker.patch(f"{CMD_MODULE}.run_codegen_check", return_value=CodegenCheckReport(lock_found=True))++ agent_codegen_check_cmd(root=str(tmp_path), output_format=CliOutputFormat.JSON, error_format=None)++ envelope = json.loads(capsys.readouterr().out)+ assert envelope["success"] is True+ assert envelope["is_current"] is True+ assert envelope["root"] == str(tmp_path)++ def test_current_markdown_is_the_default(self, mocker: MockerFixture, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None:+ mocker.patch(f"{CMD_MODULE}.run_codegen_check", return_value=CodegenCheckReport(lock_found=True))++ agent_codegen_check_cmd(root=str(tmp_path), output_format=CliOutputFormat.MARKDOWN, error_format=None)++ stdout = capsys.readouterr().out+ assert stdout.startswith("# Generated artifacts up to date")++ def test_expands_home_relative_root(self, mocker: MockerFixture, capsys: pytest.CaptureFixture[str]) -> None:+ check = mocker.patch(f"{CMD_MODULE}.run_codegen_check", return_value=CodegenCheckReport(lock_found=True))++ agent_codegen_check_cmd(root="~/generated", output_format=CliOutputFormat.JSON, error_format=None)++ assert check.call_args.kwargs["root"] == Path.home() / "generated"+ capsys.readouterr()++ def test_drift_is_structured_negative_verdict_exit_1(self, mocker: MockerFixture, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None:+ """Drift is a produced negative verdict: exit 1 with the drifting artifacts enumerated in the envelope."""+ report = CodegenCheckReport(+ lock_found=True,+ drifts=[CodegenDrift(path="models.py", category=DriftCategory.HAND_EDITED, detail="Body was edited below the stamp.")],+ )+ mocker.patch(f"{CMD_MODULE}.run_codegen_check", return_value=report)++ with pytest.raises(typer.Exit) as exc_info:+ agent_codegen_check_cmd(root=str(tmp_path), output_format=CliOutputFormat.JSON, error_format=None)++ assert exc_info.value.exit_code == 1+ error = json.loads(capsys.readouterr().err)+ assert error["error_type"] == "CodegenDriftError"+ assert error["is_current"] is False+ assert error["drifts"] == [{"path": "models.py", "category": "hand-edited", "detail": "Body was edited below the stamp."}]++ def test_no_lock_is_no_verdict_exit_2(self, mocker: MockerFixture, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None:+ mocker.patch(f"{CMD_MODULE}.run_codegen_check", return_value=CodegenCheckReport(lock_found=False))++ with pytest.raises(typer.Exit) as exc_info:+ agent_codegen_check_cmd(root=str(tmp_path), output_format=CliOutputFormat.JSON, error_format=None)++ assert exc_info.value.exit_code == 2+ error = json.loads(capsys.readouterr().err)+ assert error["error_type"] == "CodegenLockNotFoundError"++ def test_unreadable_lock_is_no_verdict_exit_2(self, mocker: MockerFixture, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None:+ mocker.patch(f"{CMD_MODULE}.run_codegen_check", side_effect=CodegenLockError("codegen.lock is not valid TOML"))++ with pytest.raises(typer.Exit) as exc_info:+ agent_codegen_check_cmd(root=str(tmp_path), output_format=CliOutputFormat.JSON, error_format=None)++ assert exc_info.value.exit_code == 2+ error = json.loads(capsys.readouterr().err)+ assert error["error_type"] == "CodegenLockError"+ assert "not valid TOML" in error["message"]+ assert "remove it, then regenerate" in error["hint"]diff --git a/tests/unit/pipelex/cli/test_agent_codegen_types_cmd.py b/tests/unit/pipelex/cli/test_agent_codegen_types_cmd.pynew file mode 100644index 000000000..525e2656c--- /dev/null+++ b/tests/unit/pipelex/cli/test_agent_codegen_types_cmd.py@@ -0,0 +1,166 @@+"""The `pipelex-agent codegen types` command: envelopes, exit codes, and boot kwargs.++Boot/teardown and the crate loader are mocked out so no real Pipelex is made; the emitter is stubbed+so these tests pin the agent-CLI wiring (the two-stream success/error envelopes and the resolve+verdict exit codes), not the projection engine (covered by its own unit tests). The stamped emission+layer runs for real against tmp_path.+"""++from __future__ import annotations++import json+from pathlib import Path+from typing import TYPE_CHECKING++import pytest+import typer++from pipelex.cli.agent_cli.commands.agent_output import CliOutputFormat+from pipelex.cli.agent_cli.commands.codegen.types_cmd import agent_codegen_types_cmd+from pipelex.codegen.emitters.target import CodegenTarget, EmittedFile+from pipelex.codegen.lock import CODEGEN_LOCK_FILENAME+from pipelex.libraries.exceptions import LibraryLoadingError+from pipelex.libraries.library_crate import LibraryCrate++if TYPE_CHECKING:+ from pytest_mock import MockerFixture++CMD_MODULE = "pipelex.cli.agent_cli.commands.codegen.types_cmd"+++class TestAgentCodegenTypesCmd:+ """The agent-CLI types projection command, with boot and the crate loader mocked out."""++ def _neutralize_boot(self, mocker: MockerFixture) -> None:+ mocker.patch(f"{CMD_MODULE}.make_pipelex_for_agent_cli")+ mocker.patch(f"{CMD_MODULE}.Pipelex.teardown_if_needed")++ def test_json_success_envelope_and_stamped_files(self, mocker: MockerFixture, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None:+ """A successful projection emits the structured JSON envelope on stdout and stamped files + lock on disk."""+ self._neutralize_boot(mocker)+ mocker.patch(f"{CMD_MODULE}.load_normalized_crate", return_value=LibraryCrate(fingerprint="deadbeef"))+ mocker.patch(f"{CMD_MODULE}.emit_types", return_value=[EmittedFile(filename="models.py", content="# models\n")])++ agent_codegen_types_cmd(+ target=CodegenTarget.PYTHON_PYDANTIC,+ paths=None,+ output_dir=str(tmp_path),+ library_dir=None,+ output_format=CliOutputFormat.JSON,+ error_format=None,+ )++ envelope = json.loads(capsys.readouterr().out)+ assert envelope["success"] is True+ assert envelope["kind"] == "types"+ assert envelope["target"] == "python-pydantic"+ assert envelope["crate_fingerprint"] == "deadbeef"+ assert envelope["written"] == ["models.py"]+ assert envelope["unchanged"] == []+ assert envelope["lock_file"].endswith(CODEGEN_LOCK_FILENAME)+ models_text = (tmp_path / "models.py").read_text(encoding="utf-8")+ assert models_text.startswith("# >>> pipelex-codegen-stamp >>>")+ assert models_text.endswith("# models\n")+ assert (tmp_path / CODEGEN_LOCK_FILENAME).is_file()++ def test_markdown_success_is_the_default(self, mocker: MockerFixture, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None:+ self._neutralize_boot(mocker)+ mocker.patch(f"{CMD_MODULE}.load_normalized_crate", return_value=LibraryCrate(fingerprint="deadbeef"))+ mocker.patch(f"{CMD_MODULE}.emit_types", return_value=[EmittedFile(filename="models.py", content="# models\n")])++ agent_codegen_types_cmd(+ target=CodegenTarget.PYTHON_PYDANTIC,+ paths=None,+ output_dir=str(tmp_path),+ library_dir=None,+ output_format=CliOutputFormat.MARKDOWN,+ error_format=None,+ )++ stdout = capsys.readouterr().out+ assert stdout.startswith("# Codegen complete")+ assert "- Generated `models.py`" in stdout+ assert "`deadbeef`" in stdout++ def test_invalid_library_is_structured_error_exit_1(self, mocker: MockerFixture, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None:+ """An invalid library is a negative verdict: exit 1 with the structured error envelope on stderr."""+ self._neutralize_boot(mocker)+ mocker.patch(f"{CMD_MODULE}.load_normalized_crate", side_effect=LibraryLoadingError("duplicate pipe"))++ with pytest.raises(typer.Exit) as exc_info:+ agent_codegen_types_cmd(+ target=CodegenTarget.PYTHON_PYDANTIC,+ paths=None,+ output_dir=str(tmp_path),+ library_dir=None,+ output_format=CliOutputFormat.JSON,+ error_format=None,+ )++ assert exc_info.value.exit_code == 1+ error = json.loads(capsys.readouterr().err)+ assert error["error"] is True+ assert error["error_type"] == "LibraryLoadingError"+ assert "duplicate pipe" in error["message"]++ def test_unassembled_closure_is_no_verdict_exit_2(self, mocker: MockerFixture, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None:+ self._neutralize_boot(mocker)+ mocker.patch(f"{CMD_MODULE}.load_normalized_crate", side_effect=FileNotFoundError("no .mthds bundles found in the closure."))++ with pytest.raises(typer.Exit) as exc_info:+ agent_codegen_types_cmd(+ target=CodegenTarget.PYTHON_PYDANTIC,+ paths=None,+ output_dir=str(tmp_path),+ library_dir=None,+ output_format=CliOutputFormat.JSON,+ error_format=None,+ )++ assert exc_info.value.exit_code == 2+ error = json.loads(capsys.readouterr().err)+ assert error["error_type"] == "FileNotFoundError"++ def test_boot_loads_model_specs_for_offline_validation(self, mocker: MockerFixture, tmp_path: Path) -> None:+ """Like the bare CLI, the agent mirror boots offline but WITH model specs: library validation+ checks pipe model pins against the deck, so a spec-less boot would reject any bundle pinning+ a model — the drift this test guards against.+ """+ boot = mocker.patch(f"{CMD_MODULE}.make_pipelex_for_agent_cli")+ mocker.patch(f"{CMD_MODULE}.Pipelex.teardown_if_needed")+ mocker.patch(f"{CMD_MODULE}.load_normalized_crate", return_value=LibraryCrate(fingerprint="deadbeef"))+ mocker.patch(f"{CMD_MODULE}.emit_types", return_value=[EmittedFile(filename="models.py", content="# models\n")])++ agent_codegen_types_cmd(+ target=CodegenTarget.PYTHON_PYDANTIC,+ paths=None,+ output_dir=str(tmp_path),+ library_dir=None,+ output_format=CliOutputFormat.JSON,+ error_format=None,+ )++ assert boot.call_args.kwargs["needs_inference"] is False+ assert boot.call_args.kwargs["needs_model_specs"] is True++ def test_expands_home_relative_output(self, mocker: MockerFixture, capsys: pytest.CaptureFixture[str]) -> None:+ self._neutralize_boot(mocker)+ mocker.patch(f"{CMD_MODULE}.load_normalized_crate", return_value=LibraryCrate(fingerprint="deadbeef"))+ mocker.patch(f"{CMD_MODULE}.emit_types", return_value=[])+ write_projection = mocker.patch(f"{CMD_MODULE}.write_stamped_projection")+ write_projection.return_value.written = []+ write_projection.return_value.unchanged = []+ write_projection.return_value.removed = []+ write_projection.return_value.lock_path = CODEGEN_LOCK_FILENAME++ agent_codegen_types_cmd(+ target=CodegenTarget.PYTHON_PYDANTIC,+ paths=None,+ output_dir="~/generated",+ library_dir=None,+ output_format=CliOutputFormat.JSON,+ error_format=None,+ )++ assert write_projection.call_args.kwargs["output_dir"] == Path.home() / "generated"+ capsys.readouterr()diff --git a/tests/unit/pipelex/cli/test_agent_output_drift.py b/tests/unit/pipelex/cli/test_agent_output_drift.pyindex 6a63e217a..9af237b78 100644--- a/tests/unit/pipelex/cli/test_agent_output_drift.py+++ b/tests/unit/pipelex/cli/test_agent_output_drift.py@@ -15,6 +15,7 @@ from pipelex.cli.agent_cli.commands.agent_output import AGENT_ERROR_DOMAINS, AGE# Modules defining every PipelexError subclass referenced by the lookup dicts.# Imported so PipelexError.__subclasses__() sees them when the registry is built._EXCEPTION_MODULES: tuple[str, ...] = (+ "pipelex.codegen.exceptions","pipelex.cogt.exceptions","pipelex.core.interpreter.exceptions","pipelex.core.pipes.exceptions",@@ -45,6 +46,8 @@ _NON_PIPELEX_ERROR_KEYS: frozenset[str] = frozenset("FixBundleError", # synthetic error_type label"GraphSpecParseError", # synthetic error_type label"BundleError", # synthetic error_type label+ "CodegenDriftError", # synthetic error_type label (a drift verdict, not an exception)+ "CodegenLockNotFoundError", # synthetic error_type label"InitConfigError", # synthetic error_type label"UnknownCommandError", # synthetic error_type label}diff --git a/tests/unit/pipelex/cli/test_codegen_check_cli.py b/tests/unit/pipelex/cli/test_codegen_check_cli.pynew file mode 100644index 000000000..1fa7dfb1d--- /dev/null+++ b/tests/unit/pipelex/cli/test_codegen_check_cli.py@@ -0,0 +1,69 @@+"""The `pipelex codegen check` command: the 0/1/2 offline-verdict exit codes (no engine boot).++The check is offline by design — it never boots Pipelex — so these drive the command directly over a+real generated tree and assert the exit-code policy: 0 current, 1 drift present, 2 no lock (no verdict).+"""++from __future__ import annotations++from pathlib import Path+from typing import TYPE_CHECKING++import pytest+import typer++from pipelex.cli.commands.codegen.check_cmd import codegen_check_cmd+from pipelex.codegen.emission import write_stamped_projection+from pipelex.codegen.emitters.target import CodegenKind, CodegenTarget, EmittedFile++if TYPE_CHECKING:+ from pytest_mock import MockerFixture++_FILES = [EmittedFile(filename="models.py", content="# h\nclass A:\n pass\n")]+++class TestCodegenCheckCli:+ def _generate(self, root: Path) -> None:+ write_stamped_projection(+ _FILES,+ output_dir=root,+ crate_fingerprint="fp1",+ engine_version="0.1.0",+ kind=CodegenKind.TYPES,+ target=CodegenTarget.PYTHON_PYDANTIC,+ )++ def test_no_lock_is_exit_2(self, tmp_path: Path) -> None:+ with pytest.raises(typer.Exit) as exc_info:+ codegen_check_cmd(root=str(tmp_path))+ assert exc_info.value.exit_code == 2++ def test_malformed_lock_is_exit_2(self, tmp_path: Path) -> None:+ (tmp_path / "codegen.lock").write_text("not = valid = toml [[", encoding="utf-8")++ with pytest.raises(typer.Exit) as exc_info:+ codegen_check_cmd(root=str(tmp_path))++ assert exc_info.value.exit_code == 2++ def test_expands_home_relative_root(self, mocker: MockerFixture) -> None:+ check = mocker.patch(+ "pipelex.cli.commands.codegen.check_cmd.run_codegen_check", return_value=mocker.MagicMock(lock_found=True, is_current=True)+ )++ codegen_check_cmd(root="~/generated")++ assert check.call_args.kwargs["root"] == Path.home() / "generated"++ def test_current_tree_is_exit_0(self, tmp_path: Path) -> None:+ self._generate(tmp_path)+ # A current tree does not raise (implicit exit 0).+ codegen_check_cmd(root=str(tmp_path))++ def test_drifted_tree_is_exit_1(self, tmp_path: Path) -> None:+ self._generate(tmp_path)+ target = tmp_path / "models.py"+ target.write_text(target.read_text(encoding="utf-8") + "drift = 1\n", encoding="utf-8")+ with pytest.raises(typer.Exit) as exc_info:+ codegen_check_cmd(root=str(tmp_path))+ assert exc_info.value.exit_code == 1diff --git a/tests/unit/pipelex/cli/test_codegen_cli.py b/tests/unit/pipelex/cli/test_codegen_cli.pynew file mode 100644index 000000000..05dab7727--- /dev/null+++ b/tests/unit/pipelex/cli/test_codegen_cli.py@@ -0,0 +1,172 @@+"""The `pipelex codegen types` / `codegen inputs` CLI: file writing, pipe selection, and exit codes.++Boot/teardown and the crate loader are mocked out so no real Pipelex is made; the emitter and the+input renderer are stubbed so these tests pin the CLI wiring (targets, output paths, main_pipe+defaulting, and the resolve-verdict exit codes), not the projection engines (covered by their own+unit tests). End-to-end behavior against the binary is pinned by the conformance suite.+"""++from __future__ import annotations++import contextlib+from pathlib import Path+from typing import TYPE_CHECKING++import pytest+import typer++from pipelex.cli.commands.codegen.inputs_cmd import codegen_inputs_cmd+from pipelex.cli.commands.codegen.types_cmd import codegen_types_cmd+from pipelex.codegen.emitters.target import CodegenTarget, EmittedFile+from pipelex.codegen.lock import CODEGEN_LOCK_FILENAME+from pipelex.core.domains.domain_blueprint import DomainBlueprint+from pipelex.core.pipes.inputs.input_renderer import InputsTemplateFormat+from pipelex.libraries.library_crate import LibraryCrate+from pipelex.libraries.pipe.exceptions import PipeLibraryError++if TYPE_CHECKING:+ from unittest.mock import MagicMock++ from pytest_mock import MockerFixture++TYPES = "pipelex.cli.commands.codegen.types_cmd"+INPUTS = "pipelex.cli.commands.codegen.inputs_cmd"+++class TestCodegenCli:+ """The two Phase-1 codegen commands, with boot/teardown and the crate loader mocked out."""++ def _neutralize_boot(self, mocker: MockerFixture, *, module: str) -> MagicMock:+ boot = mocker.patch(f"{module}.make_pipelex_for_cli")+ mocker.patch(f"{module}.Pipelex.teardown_if_needed")+ mocker.patch(f"{module}.tag")+ telemetry_manager = mocker.patch(f"{module}.get_telemetry_manager").return_value+ telemetry_manager.telemetry_context.return_value = contextlib.nullcontext()+ return boot++ def test_types_writes_every_emitted_stamped_and_locked_exit_0(self, mocker: MockerFixture, tmp_path: Path) -> None:+ self._neutralize_boot(mocker, module=TYPES)+ crate = LibraryCrate(fingerprint="deadbeef")+ mocker.patch(f"{TYPES}.load_normalized_crate_or_exit", return_value=crate)+ mocker.patch(+ f"{TYPES}.emit_types",+ return_value=[EmittedFile(filename="types.ts", content="// types\n"), EmittedFile(filename="binder.ts", content="// binder\n")],+ )+ codegen_types_cmd(target=CodegenTarget.TS_ZOD, paths=None, output_dir=str(tmp_path), library_dir=None)+ # Each file is stamped (self-describing) with the body preserved below the stamp, and locked.+ types_text = (tmp_path / "types.ts").read_text(encoding="utf-8")+ assert types_text.startswith("// >>> pipelex-codegen-stamp >>>")+ assert "crate_fingerprint: deadbeef" in types_text+ assert types_text.endswith("// types\n")+ assert (tmp_path / "binder.ts").read_text(encoding="utf-8").endswith("// binder\n")+ assert (tmp_path / CODEGEN_LOCK_FILENAME).is_file()++ @pytest.mark.parametrize("module", [TYPES, INPUTS])+ def test_boot_loads_model_specs_for_offline_validation(self, mocker: MockerFixture, tmp_path: Path, module: str) -> None:+ """Codegen boots offline but WITH model specs (like `validate`): library validation checks+ pipe model pins (`model = "gpt-4o-mini"`) against the deck, so a spec-less boot would reject+ any bundle that pins a model — the drift this test guards against.+ """+ boot = self._neutralize_boot(mocker, module=module)+ if module == TYPES:+ mocker.patch(f"{TYPES}.load_normalized_crate_or_exit", return_value=LibraryCrate(fingerprint="deadbeef"))+ mocker.patch(f"{TYPES}.emit_types", return_value=[EmittedFile(filename="types.ts", content="// types\n")])+ codegen_types_cmd(target=CodegenTarget.TS_ZOD, paths=None, output_dir=str(tmp_path), library_dir=None)+ else:+ crate = LibraryCrate(domains={"scoring": DomainBlueprint(code="scoring", description="d", main_pipe="run_scoring")})+ mocker.patch(f"{INPUTS}.load_normalized_crate_or_exit", return_value=crate)+ mocker.patch(f"{INPUTS}.get_required_pipe")+ mocker.patch(f"{INPUTS}.render_inputs", return_value="{}")+ codegen_inputs_cmd(+ pipe=None,+ paths=None,+ template_format=InputsTemplateFormat.JSON,+ explicit=False,+ output=str(tmp_path / "inputs.json"),+ library_dir=None,+ )+ assert boot.call_args.kwargs["needs_inference"] is False+ assert boot.call_args.kwargs["needs_model_specs"] is True++ def test_types_invalid_library_verdict_propagates(self, mocker: MockerFixture, tmp_path: Path) -> None:+ self._neutralize_boot(mocker, module=TYPES)+ mocker.patch(f"{TYPES}.load_normalized_crate_or_exit", side_effect=typer.Exit(1))+ with pytest.raises(typer.Exit) as exc_info:+ codegen_types_cmd(target=CodegenTarget.TS_ZOD, paths=None, output_dir=str(tmp_path), library_dir=None)+ assert exc_info.value.exit_code == 1++ def test_types_expands_home_relative_output(self, mocker: MockerFixture) -> None:+ self._neutralize_boot(mocker, module=TYPES)+ mocker.patch(f"{TYPES}.load_normalized_crate_or_exit", return_value=LibraryCrate(fingerprint="deadbeef"))+ mocker.patch(f"{TYPES}.emit_types", return_value=[])+ write_projection = mocker.patch(f"{TYPES}.write_stamped_projection")+ write_projection.return_value.written = []+ write_projection.return_value.unchanged = []+ write_projection.return_value.removed = []+ write_projection.return_value.lock_path = CODEGEN_LOCK_FILENAME++ codegen_types_cmd(target=CodegenTarget.TS_ZOD, paths=None, output_dir="~/generated", library_dir=None)++ assert write_projection.call_args.kwargs["output_dir"] == Path.home() / "generated"++ def test_inputs_defaults_to_the_single_main_pipe(self, mocker: MockerFixture, tmp_path: Path) -> None:+ self._neutralize_boot(mocker, module=INPUTS)+ crate = LibraryCrate(domains={"scoring": DomainBlueprint(code="scoring", description="d", main_pipe="run_scoring")})+ mocker.patch(f"{INPUTS}.load_normalized_crate_or_exit", return_value=crate)+ get_required_pipe = mocker.patch(f"{INPUTS}.get_required_pipe")+ mocker.patch(f"{INPUTS}.render_inputs", return_value='{"topic": "x"}')+ destination = tmp_path / "inputs.json"+ codegen_inputs_cmd(+ pipe=None, paths=None, template_format=InputsTemplateFormat.JSON, explicit=False, output=str(destination), library_dir=None+ )+ # The single declared main_pipe was selected (qualified) and rendered to the output file.+ assert get_required_pipe.call_args.kwargs["pipe_code"] == "scoring.run_scoring"+ assert destination.read_text(encoding="utf-8") == '{"topic": "x"}'++ def test_inputs_write_if_changed_leaves_identical_file_untouched(self, mocker: MockerFixture, tmp_path: Path) -> None:+ """Like `codegen types`, `codegen inputs` is write-if-changed: an already-current template is+ not rewritten (no mtime churn), so re-running codegen in a committed consumer stays a no-op.+ """+ self._neutralize_boot(mocker, module=INPUTS)+ crate = LibraryCrate(domains={"scoring": DomainBlueprint(code="scoring", description="d", main_pipe="run_scoring")})+ mocker.patch(f"{INPUTS}.load_normalized_crate_or_exit", return_value=crate)+ mocker.patch(f"{INPUTS}.get_required_pipe")+ mocker.patch(f"{INPUTS}.render_inputs", return_value='{"topic": "x"}')+ destination = tmp_path / "inputs.json"+ destination.write_text('{"topic": "x"}', encoding="utf-8")+ save = mocker.patch(f"{INPUTS}.save_text_to_path")+ codegen_inputs_cmd(+ pipe=None, paths=None, template_format=InputsTemplateFormat.JSON, explicit=False, output=str(destination), library_dir=None+ )+ save.assert_not_called()++ def test_inputs_no_main_pipe_is_exit_1(self, mocker: MockerFixture) -> None:+ self._neutralize_boot(mocker, module=INPUTS)+ crate = LibraryCrate(domains={"scoring": DomainBlueprint(code="scoring", description="d", main_pipe=None)})+ mocker.patch(f"{INPUTS}.load_normalized_crate_or_exit", return_value=crate)+ with pytest.raises(typer.Exit) as exc_info:+ codegen_inputs_cmd(pipe=None, paths=None, template_format=InputsTemplateFormat.JSON, explicit=False, output=None, library_dir=None)+ assert exc_info.value.exit_code == 1++ def test_inputs_ambiguous_main_pipe_is_exit_1(self, mocker: MockerFixture) -> None:+ self._neutralize_boot(mocker, module=INPUTS)+ crate = LibraryCrate(+ domains={+ "a": DomainBlueprint(code="a", description="d", main_pipe="run_a"),+ "b": DomainBlueprint(code="b", description="d", main_pipe="run_b"),+ }+ )+ mocker.patch(f"{INPUTS}.load_normalized_crate_or_exit", return_value=crate)+ with pytest.raises(typer.Exit) as exc_info:+ codegen_inputs_cmd(pipe=None, paths=None, template_format=InputsTemplateFormat.JSON, explicit=False, output=None, library_dir=None)+ assert exc_info.value.exit_code == 1++ def test_inputs_unknown_explicit_pipe_is_exit_1(self, mocker: MockerFixture) -> None:+ self._neutralize_boot(mocker, module=INPUTS)+ mocker.patch(f"{INPUTS}.load_normalized_crate_or_exit", return_value=LibraryCrate())+ mocker.patch(f"{INPUTS}.get_required_pipe", side_effect=PipeLibraryError("no such pipe"))+ with pytest.raises(typer.Exit) as exc_info:+ codegen_inputs_cmd(+ pipe="scoring.missing", paths=None, template_format=InputsTemplateFormat.JSON, explicit=False, output=None, library_dir=None+ )+ assert exc_info.value.exit_code == 1diff --git a/tests/unit/pipelex/cli/test_resolve_exit_codes.py b/tests/unit/pipelex/cli/test_resolve_exit_codes.pynew file mode 100644index 000000000..1bcf30ab3--- /dev/null+++ b/tests/unit/pipelex/cli/test_resolve_exit_codes.py@@ -0,0 +1,112 @@+"""The `pipelex resolve` 0/1/2 exit-code policy (mirrors the bare `validate` group).++Policy: exit 0 = resolved (crate emitted); exit 1 = a produced NEGATIVE VERDICT (the library is+invalid, so no crate can be produced); exit 2 = NO VERDICT (empty closure, file-not-found). The+verdict cases mock the resolve internals — boot/teardown are neutralized so no real Pipelex is made.+The real end-to-end exit codes against the binary are pinned by the conformance suite.+"""++from __future__ import annotations++import contextlib+from pathlib import Path+from typing import TYPE_CHECKING++import pytest+import typer++from pipelex.cli.commands.crate_loading import load_normalized_crate+from pipelex.cli.commands.resolve_cmd import resolve_cmd+from pipelex.codegen.crate_encoding import CrateEncoding+from pipelex.libraries.exceptions import LibraryLoadingError+from pipelex.libraries.pipe.exceptions import PipeLibraryError++if TYPE_CHECKING:+ from unittest.mock import MagicMock++ from pytest_mock import MockerFixture++MODULE = "pipelex.cli.commands.resolve_cmd"+# The load/normalize/library-manager seams now live in the shared crate-loading helper, so the+# verdict-path patches target that module; boot/teardown/encode still live on resolve_cmd.+CRATE_LOADING = "pipelex.cli.commands.crate_loading"+++class TestResolveExitCodes:+ """The resolve 0/1/2 exit-code policy, with boot/teardown mocked out."""++ def _neutralize_boot(self, mocker: MockerFixture) -> MagicMock:+ boot = mocker.patch(f"{MODULE}.make_pipelex_for_cli")+ mocker.patch(f"{MODULE}.Pipelex.teardown_if_needed")+ mocker.patch(f"{MODULE}.tag")+ telemetry_manager = mocker.patch(f"{MODULE}.get_telemetry_manager").return_value+ # A real null context manager — a bare MagicMock would suppress the exception under test.+ telemetry_manager.telemetry_context.return_value = contextlib.nullcontext()+ return boot++ def test_invalid_library_is_negative_verdict_exit_1(self, mocker: MockerFixture) -> None:+ self._neutralize_boot(mocker)+ mocker.patch(f"{CRATE_LOADING}.load_libraries_and_activate", side_effect=LibraryLoadingError("invalid library"))+ with pytest.raises(typer.Exit) as exc_info:+ resolve_cmd(paths=None, output_format=CrateEncoding.JSON, library_dir=None)+ assert exc_info.value.exit_code == 1++ def test_pipe_conflict_is_negative_verdict_exit_1(self, mocker: MockerFixture) -> None:+ # A duplicate-pipe-across-bundles conflict surfaces as PipeLibraryError (a LibraryError sibling+ # of LibraryLoadingError, not a subclass): still an invalid-library negative verdict -> exit 1.+ self._neutralize_boot(mocker)+ mocker.patch(f"{CRATE_LOADING}.load_libraries_and_activate", side_effect=PipeLibraryError("duplicate pipe across bundles"))+ with pytest.raises(typer.Exit) as exc_info:+ resolve_cmd(paths=None, output_format=CrateEncoding.JSON, library_dir=None)+ assert exc_info.value.exit_code == 1++ def test_empty_closure_is_no_verdict_exit_2(self, mocker: MockerFixture) -> None:+ self._neutralize_boot(mocker)+ mocker.patch(f"{CRATE_LOADING}.load_libraries_and_activate", return_value="lib-1")+ library_manager = mocker.patch(f"{CRATE_LOADING}.get_library_manager").return_value+ library_manager.get_crate.return_value = None+ with pytest.raises(typer.Exit) as exc_info:+ resolve_cmd(paths=None, output_format=CrateEncoding.JSON, library_dir=None)+ assert exc_info.value.exit_code == 2++ def test_file_not_found_is_no_verdict_exit_2(self, mocker: MockerFixture) -> None:+ self._neutralize_boot(mocker)+ mocker.patch(f"{CRATE_LOADING}.load_libraries_and_activate", side_effect=FileNotFoundError("no such directory"))+ with pytest.raises(typer.Exit) as exc_info:+ resolve_cmd(paths=None, output_format=CrateEncoding.JSON, library_dir=None)+ assert exc_info.value.exit_code == 2++ def test_valid_library_emits_crate_exit_0(self, mocker: MockerFixture, capsys: pytest.CaptureFixture[str]) -> None:+ self._neutralize_boot(mocker)+ mocker.patch(f"{CRATE_LOADING}.load_libraries_and_activate", return_value="lib-1")+ library_manager = mocker.patch(f"{CRATE_LOADING}.get_library_manager").return_value+ library_manager.get_crate.return_value = mocker.MagicMock()+ mocker.patch(f"{CRATE_LOADING}.normalize_crate", return_value=mocker.MagicMock())+ mocker.patch(f"{MODULE}.encode_crate", return_value="<<crate-body>>")+ # A resolved library emits the crate to stdout and does not raise (implicit exit 0).+ resolve_cmd(paths=None, output_format=CrateEncoding.JSON, library_dir=None)+ assert "<<crate-body>>" in capsys.readouterr().out++ def test_boot_loads_model_specs_for_offline_validation(self, mocker: MockerFixture) -> None:+ """Resolve boots offline but WITH model specs (like `validate`): library validation checks+ pipe model pins against the deck, so a spec-less boot would reject any bundle that pins a model.+ """+ boot = self._neutralize_boot(mocker)+ mocker.patch(f"{CRATE_LOADING}.load_libraries_and_activate", return_value="lib-1")+ library_manager = mocker.patch(f"{CRATE_LOADING}.get_library_manager").return_value+ library_manager.get_crate.return_value = mocker.MagicMock()+ mocker.patch(f"{CRATE_LOADING}.normalize_crate", return_value=mocker.MagicMock())+ mocker.patch(f"{MODULE}.encode_crate", return_value="{}")+ resolve_cmd(paths=None, output_format=CrateEncoding.JSON, library_dir=None)+ assert boot.call_args.kwargs["needs_inference"] is False+ assert boot.call_args.kwargs["needs_model_specs"] is True++ def test_home_relative_library_dirs_are_expanded(self, mocker: MockerFixture) -> None:+ activate = mocker.patch(f"{CRATE_LOADING}.load_libraries_and_activate", return_value="lib-1")+ library_manager = mocker.patch(f"{CRATE_LOADING}.get_library_manager").return_value+ library_manager.get_crate.return_value = mocker.MagicMock()+ mocker.patch(f"{CRATE_LOADING}.normalize_crate", return_value=mocker.MagicMock())++ load_normalized_crate(library_dirs=[Path("~/bundles")])++ activate.assert_called_once_with([Path.home() / "bundles"])diff --git a/tests/unit/pipelex/cli/test_run_core_execution.py b/tests/unit/pipelex/cli/test_run_core_execution.pyindex 4e2b3f973..0e60c5fa0 100644--- a/tests/unit/pipelex/cli/test_run_core_execution.py+++ b/tests/unit/pipelex/cli/test_run_core_execution.py@@ -294,24 +294,6 @@ class TestRunCoreExecution:assert exc_info.value.exit_code == 1- @pytest.mark.usefixtures("console")- def test_toml_time_only_input_file_exits(self, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None:- """A .toml inputs file holding a bare TOML time-of-day is rejected during loading.-- Top-level date/datetime literals are now accepted (converted to a native Date); only a bare- time-of-day has no date to attach to, so the loader raises InputsTimeOnlyNotSupportedError,- surfaced as a clean exit 1 before any runner is constructed. Asserting the "Failed to load- input file" framing pins that the exit comes from that catch site, not a downstream failure.- """- inputs_file = tmp_path / "inputs.toml"- inputs_file.write_text("deadline = 12:00:00\n", encoding="utf-8")-- with pytest.raises(typer.Exit) as exc_info:- _run_async(_call_execute_run(inputs=str(inputs_file)))-- assert exc_info.value.exit_code == 1- assert "Failed to load input file" in capsys.readouterr().err-@pytest.mark.usefixtures("console")def test_invalid_json_input_file_exits(self, tmp_path: Path) -> None:"""A .json inputs file with invalid JSON syntax is rejected cleanly."""diff --git a/tests/unit/pipelex/cli/test_stdin_resolver.py b/tests/unit/pipelex/cli/test_stdin_resolver.pyindex 0fb1f21be..1c494a5b4 100644--- a/tests/unit/pipelex/cli/test_stdin_resolver.py+++ b/tests/unit/pipelex/cli/test_stdin_resolver.py@@ -2,6 +2,7 @@from __future__ import annotations+import datetimeimport ioimport jsonfrom typing import TYPE_CHECKING, Any@@ -11,6 +12,7 @@ import typerfrom pipelex.cli.agent_cli.commands.agent_output import CliOutputFormat, set_agent_cli_error_formatfrom pipelex.cli.agent_cli.commands.run.stdin_resolver import parse_cli_inputs, resolve_stdin_inputs+from pipelex.core.stuffs.time_content import TimeContentif TYPE_CHECKING:from pathlib import Path@@ -365,21 +367,15 @@ class TestStdinResolver:assert envelope["error_domain"] == "input"assert "hint" in envelope- def test_inputs_toml_time_only_error_envelope(self, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None:- """A bare TOML time-of-day input emits the time-only-not-supported envelope with the include-a-date hint."""- set_agent_cli_error_format(CliOutputFormat.JSON)+ def test_inputs_toml_time_only_becomes_time_content(self, tmp_path: Path) -> None:+ """A bare TOML time-of-day input maps to a native Time content (no rejection since native.Time exists)."""toml_file = tmp_path / "inputs.toml"toml_file.write_text("opening = 09:00:00\n", encoding="utf-8")- with pytest.raises(typer.Exit) as exc_info:- parse_cli_inputs(inputs_arg=str(toml_file))+ parsed = parse_cli_inputs(inputs_arg=str(toml_file))- assert exc_info.value.exit_code == 1- envelope = json.loads(capsys.readouterr().err)- assert envelope["error_type"] == "InputsTimeOnlyNotSupportedError"- assert envelope["error_domain"] == "input"- assert "opening" in envelope["message"]- assert "no date to attach to" in envelope["hint"]+ assert parsed.pipeline_inputs is not None+ assert parsed.pipeline_inputs["opening"] == TimeContent(time=datetime.time(9, 0))def test_inputs_json_syntax_error_envelope(self, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None:"""Invalid JSON in a .json inputs file emits a JSONDecodeError envelope (not a raw traceback)."""diff --git a/tests/unit/pipelex/codegen/conftest.py b/tests/unit/pipelex/codegen/conftest.pynew file mode 100644index 000000000..0c3b286a2--- /dev/null+++ b/tests/unit/pipelex/codegen/conftest.py@@ -0,0 +1,185 @@+import importlib.util+import sys+from datetime import date, datetime, time+from pathlib import Path+from typing import Any++import pytest++from pipelex.core.concepts.concept_blueprint import ConceptBlueprint+from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprint, ConceptStructureBlueprintFieldType+from pipelex.core.domains.domain_blueprint import DomainBlueprint+from pipelex.libraries.crate_normalization import normalize_crate+from pipelex.libraries.library_crate import LibraryCrate++CRATE_TEST_VERSION = "1.0.0-test"+++def pipeline_crate() -> LibraryCrate:+ """A realistic normalized crate: cross-concept ref, native ref, list, literal-with-default, refines-native."""+ authored = LibraryCrate(+ concepts={+ "pipeline.Report": ConceptBlueprint(+ description="A report with a score and a label",+ structure={+ "score": ConceptStructureBlueprint(description="the score", type=ConceptStructureBlueprintFieldType.CONCEPT, concept_ref="Score"),+ "label": ConceptStructureBlueprint(description="a label", type=ConceptStructureBlueprintFieldType.CONCEPT, concept_ref="Text"),+ "tags": ConceptStructureBlueprint(description="free-form tags", type=ConceptStructureBlueprintFieldType.LIST, item_type="text"),+ "status": ConceptStructureBlueprint(description="review status", choices=["draft", "final"], default_value="draft"),+ },+ ),+ "pipeline.Score": ConceptBlueprint(+ description="A score",+ structure={+ "value": ConceptStructureBlueprint(description="the value", type=ConceptStructureBlueprintFieldType.NUMBER),+ "rationale": ConceptStructureBlueprint(description="why", type=ConceptStructureBlueprintFieldType.TEXT, required=False),+ },+ ),+ "pipeline.Summary": ConceptBlueprint(description="A short summary of a report", refines="Text"),+ },+ domains={"pipeline": DomainBlueprint(code="pipeline", description="Pipeline domain")},+ )+ return normalize_crate(authored, mthds_version=CRATE_TEST_VERSION)+++def temporal_defaults_crate() -> LibraryCrate:+ """A normalized crate carrying valid date, datetime, and time defaults."""+ authored = LibraryCrate(+ concepts={+ "schedule.Event": ConceptBlueprint(+ description="A scheduled event",+ structure={+ "starts_on": ConceptStructureBlueprint(+ description="Start date", type=ConceptStructureBlueprintFieldType.DATE, default_value=date(2026, 7, 11)+ ),+ "recorded_at": ConceptStructureBlueprint(+ description="Recorded timestamp",+ type=ConceptStructureBlueprintFieldType.DATETIME,+ default_value=datetime(2026, 7, 11, 9, 30),+ ),+ "starts_at": ConceptStructureBlueprint(+ description="Start time", type=ConceptStructureBlueprintFieldType.TIME, default_value=time(9, 30)+ ),+ },+ ),+ },+ domains={"schedule": DomainBlueprint(code="schedule", description="Schedule domain")},+ )+ return normalize_crate(authored, mthds_version=CRATE_TEST_VERSION)+++def reordered_dict_default_crates() -> tuple[LibraryCrate, LibraryCrate]:+ """Equivalent crates whose dictionary defaults differ only by insertion order."""++ def make_crate(default_value: dict[str, str]) -> LibraryCrate:+ return LibraryCrate(+ mthds_version=CRATE_TEST_VERSION,+ concepts={+ "settings.Options": ConceptBlueprint(+ description="Options",+ structure={+ "labels": ConceptStructureBlueprint(+ description="Labels by key",+ type=ConceptStructureBlueprintFieldType.DICT,+ key_type="text",+ value_type="text",+ default_value=default_value,+ )+ },+ )+ },+ )++ return make_crate({"zeta": "last", "alpha": "first"}), make_crate({"alpha": "first", "zeta": "last"})+++def edge_crate() -> LibraryCrate:+ """A crate exercising the honest edges: cross-domain code collision, an imprecise (untyped) list, a+ multi-word field (snake<->camel), a structureless concept, and a Python-class-backed opaque concept.+ """+ return LibraryCrate(+ mthds_version=CRATE_TEST_VERSION,+ concepts={+ "alpha.Result": ConceptBlueprint(+ description="alpha result",+ structure={+ "items": ConceptStructureBlueprint(description="untyped list", type=ConceptStructureBlueprintFieldType.LIST),+ "item_count": ConceptStructureBlueprint(description="how many", type=ConceptStructureBlueprintFieldType.INTEGER, required=True),+ },+ ),+ "beta.Result": ConceptBlueprint(+ description="beta result",+ structure={"n": ConceptStructureBlueprint(description="a count", type=ConceptStructureBlueprintFieldType.INTEGER, required=True)},+ ),+ "alpha.Blob": ConceptBlueprint(description="an opaque thing"),+ "alpha.Legacy": ConceptBlueprint(description="python-backed", structure="MyLegacyClass"),+ },+ )+++def materialized_image_crate() -> LibraryCrate:+ """A normalized crate that references the `Image` native, so normalization actually materializes it+ (from the pinned definitions — flat `width`/`height`), plus authored dict fields covering the DICT+ path for every emitter: a typed dict and an unspecified-values dict (the `Any` sentinel, surfaced+ as declared imprecision).+ """+ authored = LibraryCrate(+ concepts={+ "media.Photo": ConceptBlueprint(description="A photo", refines="Image"),+ "media.Gallery": ConceptBlueprint(+ description="A gallery",+ structure={+ "captions": ConceptStructureBlueprint(+ description="caption per photo code",+ type=ConceptStructureBlueprintFieldType.DICT,+ key_type="str",+ value_type="text",+ required=True,+ ),+ "metadata": ConceptStructureBlueprint(+ description="free-form metadata",+ type=ConceptStructureBlueprintFieldType.DICT,+ key_type="str",+ value_type="Any",+ required=False,+ ),+ },+ ),+ },+ domains={"media": DomainBlueprint(code="media", description="Media domain")},+ )+ return normalize_crate(authored, mthds_version=CRATE_TEST_VERSION)+++def refines_crate() -> LibraryCrate:+ """A concept refining a structureless native keeps its refines link, so the emitter renders inheritance."""+ return LibraryCrate(+ mthds_version=CRATE_TEST_VERSION,+ concepts={+ "native.Image": ConceptBlueprint(description="An image"),+ "media.Thumbnail": ConceptBlueprint(description="A small image", refines="native.Image"),+ },+ )+++def load_generated_module(content: str, *, tmp_path: Path, name: str) -> Any:+ """Write generated Python to disk, import it, and return it (proves compile + exec).++ Returns `Any`: the generated classes exist only at runtime, so their members are deliberately+ opaque to the type checker.+ """+ module_path = tmp_path / f"{name}.py"+ module_path.write_text(content, encoding="utf-8")+ spec = importlib.util.spec_from_file_location(name, module_path)+ assert spec is not None+ assert spec.loader is not None+ module = importlib.util.module_from_spec(spec)+ sys.modules[name] = module+ spec.loader.exec_module(module)+ return modulediff --git a/tests/unit/pipelex/codegen/test_artifact_paths.py b/tests/unit/pipelex/codegen/test_artifact_paths.pynew file mode 100644index 000000000..adbc9e1e7--- /dev/null+++ b/tests/unit/pipelex/codegen/test_artifact_paths.py@@ -0,0 +1,161 @@+from pathlib import Path++import pytest++from pipelex.codegen.check import run_codegen_check+from pipelex.codegen.emission import StampedProjection, WriteReport, build_stamped_projection, write_stamped_projection+from pipelex.codegen.emitters.target import CodegenKind, CodegenTarget, EmittedFile+from pipelex.codegen.exceptions import CodegenError, CodegenLockError+from pipelex.codegen.lock import CODEGEN_LOCK_FILENAME, validate_artifact_path+from tests.unit.pipelex.codegen.test_data import ArtifactPathCases++_FILES = [EmittedFile(filename="models.py", content="class Model:\n pass\n")]+++class TestArtifactPaths:+ @pytest.mark.parametrize("artifact_path", ArtifactPathCases.INVALID_PATHS)+ def test_invalid_artifact_paths_are_rejected(self, artifact_path: str) -> None:+ with pytest.raises(CodegenError):+ validate_artifact_path(artifact_path)++ def test_valid_nested_artifact_stays_beneath_root(self, tmp_path: Path) -> None:+ nested_file = EmittedFile(filename="generated/models.py", content="class Model:\n pass\n")++ self._write(tmp_path, files=[nested_file])++ assert (tmp_path / "generated" / "models.py").is_file()+ assert run_codegen_check(root=tmp_path).is_current++ def test_duplicate_emitted_paths_are_rejected(self) -> None:+ duplicate = EmittedFile(filename="models.py", content="different = True\n")++ with pytest.raises(CodegenError, match="Duplicate"):+ self._build([_FILES[0], duplicate])++ def test_traversal_cannot_write_outside_output_root(self, tmp_path: Path) -> None:+ output_root = tmp_path / "output"+ traversal = EmittedFile(filename="../escaped.py", content="escaped = True\n")++ with pytest.raises(CodegenError):+ self._write(output_root, files=[traversal])++ assert not (tmp_path / "escaped.py").exists()++ def test_malicious_lock_cannot_delete_outside_output_root(self, tmp_path: Path) -> None:+ output_root = tmp_path / "output"+ output_root.mkdir()+ victim = tmp_path / "victim.py"+ victim.write_text("# generated-looking victim\n", encoding="utf-8")+ self._write_raw_lock(output_root, paths=("../victim.py",))++ with pytest.raises(CodegenLockError):+ self._write(output_root, files=[])++ assert victim.read_text(encoding="utf-8") == "# generated-looking victim\n"++ def test_duplicate_lock_paths_are_rejected(self, tmp_path: Path) -> None:+ self._write_raw_lock(tmp_path, paths=("models.py", "models.py"))++ with pytest.raises(CodegenLockError, match="Duplicate"):+ run_codegen_check(root=tmp_path)++ def test_symlink_destination_cannot_redirect_a_write(self, tmp_path: Path) -> None:+ output_root = tmp_path / "output"+ output_root.mkdir()+ target = tmp_path / "target.py"+ target.write_text("keep = True\n", encoding="utf-8")+ self._make_symlink(output_root / "models.py", target=target)++ with pytest.raises(CodegenError, match="symbolic link"):+ self._write(output_root)++ assert target.read_text(encoding="utf-8") == "keep = True\n"++ def test_symlinked_parent_cannot_redirect_a_nested_write(self, tmp_path: Path) -> None:+ output_root = tmp_path / "output"+ output_root.mkdir()+ outside = tmp_path / "outside"+ outside.mkdir()+ self._make_symlink(output_root / "generated", target=outside, target_is_directory=True)+ nested_file = EmittedFile(filename="generated/models.py", content="escaped = True\n")++ with pytest.raises(CodegenError, match="symbolic link"):+ self._write(output_root, files=[nested_file])++ assert not (outside / "models.py").exists()++ def test_symlinked_output_root_is_rejected(self, tmp_path: Path) -> None:+ actual_root = tmp_path / "actual"+ actual_root.mkdir()+ output_root = tmp_path / "output"+ self._make_symlink(output_root, target=actual_root, target_is_directory=True)++ with pytest.raises(CodegenError, match="symbolic link"):+ self._write(output_root)++ assert not (actual_root / "models.py").exists()++ def test_symlinked_ancestor_is_canonicalized_above_the_output_boundary(self, tmp_path: Path) -> None:+ actual_parent = tmp_path / "actual"+ actual_parent.mkdir()+ alias_parent = tmp_path / "alias"+ self._make_symlink(alias_parent, target=actual_parent, target_is_directory=True)+ output_root = alias_parent / "output"++ self._write(output_root)++ assert (actual_parent / "output" / "models.py").is_file()++ def test_codegen_check_does_not_read_a_symlinked_locked_artifact(self, tmp_path: Path) -> None:+ self._write(tmp_path)+ (tmp_path / "models.py").unlink()+ outside = tmp_path / "outside.py"+ outside.write_text("outside = True\n", encoding="utf-8")+ self._make_symlink(tmp_path / "models.py", target=outside)++ with pytest.raises(CodegenLockError, match="symbolic link"):+ run_codegen_check(root=tmp_path)++ def test_orphan_scan_does_not_descend_through_directory_symlinks(self, tmp_path: Path) -> None:+ output_root = tmp_path / "output"+ self._write(output_root)+ outside = tmp_path / "outside"+ outside.mkdir()+ (outside / "stray.py").write_text((output_root / "models.py").read_text(encoding="utf-8"), encoding="utf-8")+ self._make_symlink(output_root / "linked", target=outside, target_is_directory=True)++ assert run_codegen_check(root=output_root).is_current++ @staticmethod+ def _make_symlink(link_path: Path, *, target: Path, target_is_directory: bool = False) -> None:+ try:+ link_path.symlink_to(target, target_is_directory=target_is_directory)+ except OSError as exc:+ pytest.skip(f"Symbolic links are unavailable: {exc}")++ @staticmethod+ def _write_raw_lock(root: Path, *, paths: tuple[str, ...]) -> None:+ artifacts = "\n".join(f'[[artifacts]]\npath = "{path}"\ncontent_hash = "hash-{index}"' for index, path in enumerate(paths))+ content = f'crate_fingerprint = "fp"\nengine_version = "1"\n\n{artifacts}\n'+ (root / CODEGEN_LOCK_FILENAME).write_text(content, encoding="utf-8")++ @staticmethod+ def _build(files: list[EmittedFile]) -> StampedProjection:+ return build_stamped_projection(+ files,+ crate_fingerprint="fp",+ engine_version="1",+ kind=CodegenKind.TYPES,+ target=CodegenTarget.PYTHON_PYDANTIC,+ )++ @staticmethod+ def _write(root: Path, *, files: list[EmittedFile] | None = None) -> WriteReport:+ return write_stamped_projection(+ _FILES if files is None else files,+ output_dir=root,+ crate_fingerprint="fp",+ engine_version="1",+ kind=CodegenKind.TYPES,+ target=CodegenTarget.PYTHON_PYDANTIC,+ )diff --git a/tests/unit/pipelex/codegen/test_check.py b/tests/unit/pipelex/codegen/test_check.pynew file mode 100644index 000000000..b2e0966cb--- /dev/null+++ b/tests/unit/pipelex/codegen/test_check.py@@ -0,0 +1,110 @@+"""The offline drift check: clean, missing, modified, hand-edited, and orphan verdicts, plus no-lock."""++from __future__ import annotations++from typing import TYPE_CHECKING++from pipelex.codegen.check import DriftCategory, run_codegen_check+from pipelex.codegen.emission import write_stamped_projection+from pipelex.codegen.emitters.target import CodegenKind, CodegenTarget, EmittedFile++if TYPE_CHECKING:+ from pathlib import Path++_FILES = [+ EmittedFile(filename="models.py", content="# h\nclass A:\n pass\n"),+ EmittedFile(filename="types.ts", content="// h\nexport type A = number;\n"),+]+++class TestCheck:+ def _generate(self, root: Path, *, fingerprint: str = "fp1", files: list[EmittedFile] | None = None) -> None:+ write_stamped_projection(+ files if files is not None else _FILES,+ output_dir=root,+ crate_fingerprint=fingerprint,+ engine_version="0.1.0",+ kind=CodegenKind.TYPES,+ target=CodegenTarget.PYTHON_PYDANTIC,+ )++ def test_no_lock_is_no_verdict(self, tmp_path: Path) -> None:+ report = run_codegen_check(root=tmp_path)+ assert report.lock_found is False+ assert report.is_current is False++ def test_freshly_generated_is_current(self, tmp_path: Path) -> None:+ self._generate(tmp_path)+ report = run_codegen_check(root=tmp_path)+ assert report.is_current+ assert report.drifts == []++ def test_deleted_file_is_missing_drift(self, tmp_path: Path) -> None:+ self._generate(tmp_path)+ (tmp_path / "types.ts").unlink()+ report = run_codegen_check(root=tmp_path)+ assert not report.is_current+ assert [(d.path, d.category) for d in report.drifts] == [("types.ts", DriftCategory.MISSING)]++ def test_hand_edit_below_stamp_is_hand_edited_drift(self, tmp_path: Path) -> None:+ self._generate(tmp_path)+ target = tmp_path / "models.py"+ target.write_text(target.read_text(encoding="utf-8") + "sneaky = 1\n", encoding="utf-8")+ report = run_codegen_check(root=tmp_path)+ assert any(d.path == "models.py" and d.category == DriftCategory.HAND_EDITED for d in report.drifts)++ def test_stripped_stamp_is_hand_edited_drift(self, tmp_path: Path) -> None:+ self._generate(tmp_path)+ target = tmp_path / "models.py"+ target.write_text("class A:\n pass\n", encoding="utf-8") # stamp header removed entirely+ report = run_codegen_check(root=tmp_path)+ assert any(d.path == "models.py" and d.category == DriftCategory.HAND_EDITED for d in report.drifts)++ def test_malformed_stamp_options_are_hand_edited_drift(self, tmp_path: Path) -> None:+ self._generate(tmp_path)+ target = tmp_path / "models.py"+ target.write_text(target.read_text(encoding="utf-8").replace("# options: {}", "# options: {bad"), encoding="utf-8")+ report = run_codegen_check(root=tmp_path)+ assert any(d.path == "models.py" and d.category == DriftCategory.HAND_EDITED for d in report.drifts)++ def test_stale_stamped_file_not_in_lock_is_orphan(self, tmp_path: Path) -> None:+ self._generate(tmp_path)+ # Copy a stamped file under a new name: it carries a stamp but the lock does not track it.+ (tmp_path / "stray.py").write_text((tmp_path / "models.py").read_text(encoding="utf-8"), encoding="utf-8")+ report = run_codegen_check(root=tmp_path)+ assert any(d.path == "stray.py" and d.category == DriftCategory.ORPHAN for d in report.drifts)++ def test_unstamped_sibling_file_is_ignored(self, tmp_path: Path) -> None:+ self._generate(tmp_path)+ (tmp_path / "hand_written.py").write_text("x = 1\n", encoding="utf-8") # no stamp -> not our artifact+ report = run_codegen_check(root=tmp_path)+ assert report.is_current++ def test_non_utf8_tracked_file_is_hand_edited_not_a_crash(self, tmp_path: Path) -> None:+ self._generate(tmp_path)+ (tmp_path / "models.py").write_bytes(b"\xff\xfe not valid utf-8")+ report = run_codegen_check(root=tmp_path) # must not raise+ assert any(d.path == "models.py" and d.category == DriftCategory.HAND_EDITED for d in report.drifts)++ def test_non_utf8_stray_file_is_ignored_not_a_crash(self, tmp_path: Path) -> None:+ self._generate(tmp_path)+ (tmp_path / "legacy.py").write_bytes(b"# caf\xe9\n") # latin-1 bytes, unrelated file+ report = run_codegen_check(root=tmp_path) # orphan scan must not choke on it+ assert report.is_current++ def test_orphan_scan_prunes_vendor_directories(self, tmp_path: Path) -> None:+ self._generate(tmp_path)+ # A stamped file buried in a vendor dir must not be flagged as an orphan (the scan prunes it).+ vendor = tmp_path / "node_modules" / "pkg"+ vendor.mkdir(parents=True)+ (vendor / "types.ts").write_text((tmp_path / "models.py").read_text(encoding="utf-8"), encoding="utf-8")+ report = run_codegen_check(root=tmp_path)+ assert report.is_current++ def test_nested_stamped_orphan_is_detected(self, tmp_path: Path) -> None:+ self._generate(tmp_path)+ nested = tmp_path / "sub" / "deep"+ nested.mkdir(parents=True)+ (nested / "stray.py").write_text((tmp_path / "models.py").read_text(encoding="utf-8"), encoding="utf-8")+ report = run_codegen_check(root=tmp_path)+ assert any(d.path == "sub/deep/stray.py" and d.category == DriftCategory.ORPHAN for d in report.drifts)diff --git a/tests/unit/pipelex/codegen/test_crate_encoding.py b/tests/unit/pipelex/codegen/test_crate_encoding.pynew file mode 100644index 000000000..4ac2a62c9--- /dev/null+++ b/tests/unit/pipelex/codegen/test_crate_encoding.py@@ -0,0 +1,85 @@+import json++import tomli++from pipelex.codegen.crate_encoding import CrateEncoding, encode_crate, encode_crate_json, encode_crate_toml+from pipelex.core.concepts.concept_blueprint import ConceptBlueprint+from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprint, ConceptStructureBlueprintFieldType+from pipelex.core.domains.domain_blueprint import DomainBlueprint+from pipelex.libraries.crate_normalization import normalize_crate+from pipelex.libraries.library_crate import LibraryCrate+from pipelex.pipe_operators.llm.pipe_llm_blueprint import PipeLLMBlueprint++MTHDS_TEST_VERSION = "0.1.0-test"+++def _normalized_crate() -> LibraryCrate:+ """A small normalized crate (natives expanded, refs qualified) to encode."""+ crate = LibraryCrate(+ concepts={+ "scoring.Score": ConceptBlueprint(+ description="A weighted score",+ structure={+ "value": ConceptStructureBlueprint(description="the value", type=ConceptStructureBlueprintFieldType.NUMBER),+ "note": ConceptStructureBlueprint(description="a note", type=ConceptStructureBlueprintFieldType.CONCEPT, concept_ref="Text"),+ },+ ),+ },+ pipes={+ "scoring.compute": PipeLLMBlueprint(description="Compute", inputs={"data": "Text"}, output="Score", prompt="from $data"),+ },+ domains={"scoring": DomainBlueprint(code="scoring", description="Scoring domain")},+ source_map={"scoring.Score": "/fake/scoring.mthds"},+ )+ return normalize_crate(crate, mthds_version=MTHDS_TEST_VERSION)+++class TestCrateEncoding:+ """Unit tests for the JSON and TOML crate encodings."""++ def test_both_encodings_carry_the_crate_fingerprint(self):+ """The fingerprint is a property of the logical crate; both encodings emit the same one."""+ crate = _normalized_crate()+ json_doc = json.loads(encode_crate_json(crate))+ toml_doc = tomli.loads(encode_crate_toml(crate))+ assert json_doc["fingerprint"] == crate.fingerprint+ assert toml_doc["fingerprint"] == crate.fingerprint++ def test_both_encodings_round_trip_to_the_same_logical_crate(self):+ """Parsing either encoding back into a LibraryCrate recomputes the same fingerprint and content."""+ crate = _normalized_crate()+ from_json = LibraryCrate.model_validate(json.loads(encode_crate_json(crate)))+ from_toml = LibraryCrate.model_validate(tomli.loads(encode_crate_toml(crate)))+ assert from_json.compute_normalized() == crate.fingerprint+ assert from_toml.compute_normalized() == crate.fingerprint+ assert from_json.concepts == from_toml.concepts+ assert from_json.pipes == from_toml.pipes+ assert from_json.domains == from_toml.domains++ def test_toml_quotes_dotted_qualified_keys(self):+ """Dotted qualified refs are emitted as quoted single keys, never unquoted nested tables."""+ toml_text = encode_crate_toml(_normalized_crate())+ assert '[concepts."scoring.Score"]' in toml_text+ assert '[concepts."native.Text"]' in toml_text+ assert '[pipes."scoring.compute"]' in toml_text+ # An unquoted dotted header would be silently parsed as nested tables — must never appear.+ assert "[concepts.scoring.Score]" not in toml_text++ def test_provenance_source_and_pipe_category_are_excluded(self):+ """Inline `source` and the internal `pipe_category` are dropped; top-level `source_map` stays."""+ payload = json.loads(encode_crate_json(_normalized_crate()))+ assert "source" not in payload["concepts"]["scoring.Score"]+ assert "source" not in payload["pipes"]["scoring.compute"]+ assert "pipe_category" not in payload["pipes"]["scoring.compute"]+ assert payload["source_map"] == {"scoring.Score": "/fake/scoring.mthds"}++ def test_top_level_maps_are_key_sorted(self):+ """Concepts/pipes/domains are emitted in sorted key order for minimal, stable diffs."""+ payload = json.loads(encode_crate_json(_normalized_crate()))+ assert list(payload["concepts"].keys()) == sorted(payload["concepts"].keys())++ def test_encode_crate_dispatch_matches_direct_encoders(self):+ """`encode_crate` dispatches to the encoding-specific encoder for each `CrateEncoding`."""+ crate = _normalized_crate()+ assert encode_crate(crate, encoding=CrateEncoding.JSON) == encode_crate_json(crate)+ assert encode_crate(crate, encoding=CrateEncoding.TOML) == encode_crate_toml(crate)diff --git a/tests/unit/pipelex/codegen/test_data.py b/tests/unit/pipelex/codegen/test_data.pynew file mode 100644index 000000000..a9f12f3a1--- /dev/null+++ b/tests/unit/pipelex/codegen/test_data.py@@ -0,0 +1,22 @@+from typing import ClassVar+++class DescriptionEscapingCases:+ MALICIOUS_DESCRIPTION: ClassVar[str] = (+ 'Safe opening"""\n injected = True\n """ignored */\nexport const injected = true;\n/**\r\nbackslash \\ path\rUnicode café'+ )+++class ArtifactPathCases:+ INVALID_PATHS: ClassVar[tuple[str, ...]] = (+ "",+ ".",+ "../escaped.py",+ "nested/../escaped.py",+ "/absolute.py",+ "C:/absolute.py",+ "nested\\windows.py",+ "nested//empty.py",+ "control\n.py",+ "unsupported.txt",+ )diff --git a/tests/unit/pipelex/codegen/test_description_escaping.py b/tests/unit/pipelex/codegen/test_description_escaping.pynew file mode 100644index 000000000..0b0eaf19a--- /dev/null+++ b/tests/unit/pipelex/codegen/test_description_escaping.py@@ -0,0 +1,74 @@+from collections.abc import Callable+from pathlib import Path++import pytest++from pipelex.codegen.emitters.python_pydantic import emit_python_pydantic+from pipelex.codegen.emitters.python_structures import emit_python_structures+from pipelex.codegen.emitters.target import EmittedFile+from pipelex.codegen.emitters.ts_zod import emit_ts_zod+from pipelex.codegen.resolved_concepts import ResolvedLibrary, resolve_concepts_from_crate+from pipelex.core.concepts.concept_blueprint import ConceptBlueprint+from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprint, ConceptStructureBlueprintFieldType+from pipelex.libraries.library_crate import LibraryCrate+from tests.unit.pipelex.codegen.conftest import load_generated_module+from tests.unit.pipelex.codegen.test_data import DescriptionEscapingCases+++class TestDescriptionEscaping:+ @pytest.mark.parametrize(+ ("emitter", "module_name"),+ [+ pytest.param(emit_python_pydantic, "generated_pydantic", id="python-pydantic"),+ pytest.param(emit_python_structures, "generated_structures", id="python-structures"),+ ],+ )+ def test_python_descriptions_cannot_escape_docstrings(+ self,+ emitter: Callable[[ResolvedLibrary], list[EmittedFile]],+ module_name: str,+ tmp_path: Path,+ ):+ description = DescriptionEscapingCases.MALICIOUS_DESCRIPTION+ library = resolve_concepts_from_crate(self._make_crate(description=description))++ content = emitter(library)[0].content+ module = load_generated_module(content, tmp_path=tmp_path, name=module_name)++ assert module.Payload.__doc__ == description+ assert not hasattr(module.Payload, "injected")+ assert module.Payload.model_fields["value"].description == description++ def test_typescript_descriptions_cannot_escape_jsdoc(self):+ description = DescriptionEscapingCases.MALICIOUS_DESCRIPTION+ library = resolve_concepts_from_crate(self._make_crate(description=description))++ content = emit_ts_zod(library)[0].content++ assert ' * Safe opening"""' in content+ assert " * injected = True" in content+ assert ' * """ignored * /' in content+ assert " * export const injected = true;" in content+ assert " * /**" in content+ assert " * backslash \\ path" in content+ assert " * Unicode café" in content+ assert "\nexport const injected = true;\n" not in content+ assert "\r" not in content++ @staticmethod+ def _make_crate(*, description: str) -> LibraryCrate:+ return LibraryCrate(+ mthds_version="1.0.0-test",+ concepts={+ "security.Payload": ConceptBlueprint(+ description=description,+ structure={+ "value": ConceptStructureBlueprint(+ description=description,+ type=ConceptStructureBlueprintFieldType.TEXT,+ required=True,+ )+ },+ )+ },+ )diff --git a/tests/unit/pipelex/codegen/test_emission.py b/tests/unit/pipelex/codegen/test_emission.pynew file mode 100644index 000000000..beddb0283--- /dev/null+++ b/tests/unit/pipelex/codegen/test_emission.py@@ -0,0 +1,111 @@+"""The stamped-projection writer: stamping, write-if-changed idempotence, de-listed pruning, and locking."""++from __future__ import annotations++from typing import TYPE_CHECKING++import pytest++from pipelex.codegen.emission import WriteReport, build_stamped_projection, write_stamped_projection+from pipelex.codegen.emitters.target import CodegenKind, CodegenTarget, EmittedFile+from pipelex.codegen.exceptions import CodegenError+from pipelex.codegen.lock import CODEGEN_LOCK_FILENAME, load_lock+from pipelex.codegen.stamp import has_stamp++if TYPE_CHECKING:+ from pathlib import Path++_FILES = [+ EmittedFile(filename="models.py", content="# h\nclass A:\n pass\n"),+ EmittedFile(filename="types.ts", content="// h\nexport type A = number;\n"),+]+++class TestEmission:+ def _write(self, root: Path, *, fingerprint: str = "fp1", files: list[EmittedFile] | None = None) -> WriteReport:+ return write_stamped_projection(+ files if files is not None else _FILES,+ output_dir=root,+ crate_fingerprint=fingerprint,+ engine_version="0.1.0",+ kind=CodegenKind.TYPES,+ target=CodegenTarget.PYTHON_PYDANTIC,+ )++ def test_first_write_stamps_every_file_and_writes_the_lock(self, tmp_path: Path) -> None:+ report = self._write(tmp_path)+ assert set(report.written) == {"models.py", "types.ts"}+ assert report.unchanged == []+ assert has_stamp((tmp_path / "models.py").read_text(encoding="utf-8"), comment_prefix="#")+ lock = load_lock(tmp_path / CODEGEN_LOCK_FILENAME)+ assert lock is not None+ assert lock.crate_fingerprint == "fp1"+ assert lock.paths() == {"models.py", "types.ts"}++ def test_first_write_refuses_unowned_destination_without_partial_writes(self, tmp_path: Path) -> None:+ hand_authored = "// hand-authored module\n"+ (tmp_path / "types.ts").write_text(hand_authored, encoding="utf-8")++ with pytest.raises(CodegenError, match="Refusing to overwrite unowned file"):+ self._write(tmp_path)++ assert (tmp_path / "types.ts").read_text(encoding="utf-8") == hand_authored+ assert not (tmp_path / "models.py").exists()+ assert not (tmp_path / CODEGEN_LOCK_FILENAME).exists()++ def test_body_is_preserved_below_the_stamp(self, tmp_path: Path) -> None:+ self._write(tmp_path)+ assert (tmp_path / "models.py").read_text(encoding="utf-8").endswith("# h\nclass A:\n pass\n")++ def test_rewriting_identical_input_is_all_unchanged(self, tmp_path: Path) -> None:+ self._write(tmp_path)+ report = self._write(tmp_path)+ assert report.written == []+ assert set(report.unchanged) == {"models.py", "types.ts"}++ def test_delisted_stamped_file_is_pruned_and_delocked(self, tmp_path: Path) -> None:+ self._write(tmp_path)+ # Regenerate producing only models.py: types.ts drops out and is removed, lock no longer tracks it.+ report = self._write(tmp_path, fingerprint="fp2", files=[_FILES[0]])+ assert report.removed == ["types.ts"]+ assert not (tmp_path / "types.ts").exists()+ lock = load_lock(tmp_path / CODEGEN_LOCK_FILENAME)+ assert lock is not None+ assert lock.paths() == {"models.py"}+ assert lock.crate_fingerprint == "fp2"++ def test_pure_build_and_disk_write_agree_byte_for_byte(self, tmp_path: Path) -> None:+ """The pure core and the writer must never drift: a host serving `build_stamped_projection`+ over the wire (the HTTP codegen route) hands out exactly the bytes a local+ `write_stamped_projection` run puts on disk — stamped artifacts and lock alike.+ """+ projection = build_stamped_projection(+ _FILES,+ crate_fingerprint="fp1",+ engine_version="0.1.0",+ kind=CodegenKind.TYPES,+ target=CodegenTarget.PYTHON_PYDANTIC,+ )+ self._write(tmp_path)+ for stamped_file in projection.files:+ assert (tmp_path / stamped_file.filename).read_text(encoding="utf-8") == stamped_file.content+ assert (tmp_path / CODEGEN_LOCK_FILENAME).read_text(encoding="utf-8") == projection.lock_content++ def test_pruning_never_touches_an_unstamped_file(self, tmp_path: Path) -> None:+ self._write(tmp_path)+ # A hand-authored file sharing a tracked name is replaced by a stamped one only via regeneration;+ # a file the lock tracks but that lost its stamp must NOT be deleted by the pruner.+ (tmp_path / "types.ts").write_text("// hand authored, no stamp\n", encoding="utf-8")+ report = self._write(tmp_path, fingerprint="fp2", files=[_FILES[0]])+ assert report.removed == []+ assert (tmp_path / "types.ts").exists()++ def test_regeneration_recovers_from_a_malformed_previous_lock(self, tmp_path: Path) -> None:+ (tmp_path / CODEGEN_LOCK_FILENAME).write_text("not = valid = toml [[", encoding="utf-8")++ report = self._write(tmp_path)++ assert set(report.written) == {"models.py", "types.ts"}+ lock = load_lock(tmp_path / CODEGEN_LOCK_FILENAME)+ assert lock is not None+ assert lock.paths() == {"models.py", "types.ts"}diff --git a/tests/unit/pipelex/codegen/test_lock.py b/tests/unit/pipelex/codegen/test_lock.pynew file mode 100644index 000000000..6468e8867--- /dev/null+++ b/tests/unit/pipelex/codegen/test_lock.py@@ -0,0 +1,56 @@+"""The codegen lock: build/encode/load round-trip, deterministic ordering, and malformed-lock handling."""++from __future__ import annotations++from typing import TYPE_CHECKING++import pytest++from pipelex.codegen.exceptions import CodegenLockError+from pipelex.codegen.lock import CODEGEN_LOCK_FILENAME, build_lock, encode_lock, load_lock++if TYPE_CHECKING:+ from pathlib import Path++ from pytest_mock import MockerFixture+++class TestLock:+ def test_build_sorts_artifacts_by_path(self) -> None:+ lock = build_lock(crate_fingerprint="fp", engine_version="1.0.0", artifacts={"z.ts": "hz", "a.py": "ha"})+ assert [entry.path for entry in lock.artifacts] == ["a.py", "z.ts"]+ assert lock.hash_by_path() == {"a.py": "ha", "z.ts": "hz"}+ assert lock.paths() == {"a.py", "z.ts"}++ def test_encode_then_load_round_trips(self, tmp_path: Path) -> None:+ lock = build_lock(crate_fingerprint="fp-abc", engine_version="2.1.0", artifacts={"models.py": "h1", "types.ts": "h2"})+ lock_path = tmp_path / CODEGEN_LOCK_FILENAME+ lock_path.write_text(encode_lock(lock), encoding="utf-8")+ loaded = load_lock(lock_path)+ assert loaded is not None+ assert loaded.crate_fingerprint == "fp-abc"+ assert loaded.engine_version == "2.1.0"+ assert loaded.hash_by_path() == {"models.py": "h1", "types.ts": "h2"}++ def test_load_absent_lock_is_none(self, tmp_path: Path) -> None:+ assert load_lock(tmp_path / CODEGEN_LOCK_FILENAME) is None++ def test_malformed_lock_raises(self, tmp_path: Path) -> None:+ lock_path = tmp_path / CODEGEN_LOCK_FILENAME+ lock_path.write_text("this is not = valid = toml [[", encoding="utf-8")+ with pytest.raises(CodegenLockError):+ load_lock(lock_path)++ def test_non_utf8_lock_raises_clean_error_not_a_crash(self, tmp_path: Path) -> None:+ lock_path = tmp_path / CODEGEN_LOCK_FILENAME+ lock_path.write_bytes(b"\xff\xfe not valid utf-8")+ with pytest.raises(CodegenLockError):+ load_lock(lock_path)++ def test_unreadable_lock_raises_clean_error(self, mocker: MockerFixture, tmp_path: Path) -> None:+ lock_path = tmp_path / CODEGEN_LOCK_FILENAME+ lock_path.write_text("crate_fingerprint = 'fp'", encoding="utf-8")+ mocker.patch("pipelex.codegen.lock.load_text_from_path", side_effect=PermissionError("permission denied"))++ with pytest.raises(CodegenLockError, match="permission denied"):+ load_lock(lock_path)diff --git a/tests/unit/pipelex/codegen/test_naming.py b/tests/unit/pipelex/codegen/test_naming.pynew file mode 100644index 000000000..ef36f023f--- /dev/null+++ b/tests/unit/pipelex/codegen/test_naming.py@@ -0,0 +1,126 @@+from pipelex.codegen.emitters.naming import allocate_ts_type_names, python_class_name, runtime_to_emitted_class_names, snake_to_pascal, ts_type_name+from pipelex.codegen.resolved_concepts import ResolvedConcept, ResolvedLibrary+++def _resolved_concept(+ *,+ domain: str,+ code: str,+ is_native: bool = False,+ needs_qualification: bool = False,+ opaque_python_class: str | None = None,+) -> ResolvedConcept:+ return ResolvedConcept(+ concept_ref=f"{domain}.{code}",+ domain=domain,+ code=code,+ description=f"{code} concept",+ is_native=is_native,+ needs_qualification=needs_qualification,+ base_ref=None,+ fields=[],+ structureless=opaque_python_class is not None,+ imprecision_reason=None,+ opaque_python_class=opaque_python_class,+ )+++class TestNaming:+ """Unit tests for the shared name-derivation rules (see docs/specs/pipelex-codegen.md)."""++ def test_snake_to_pascal(self):+ assert snake_to_pascal("value") == "Value"+ assert snake_to_pascal("legal_contracts") == "LegalContracts"++ def test_python_class_name_bare_when_unique(self):+ assert python_class_name(domain="pipeline", code="Report", needs_qualification=False) == "Report"++ def test_python_class_name_qualified_on_collision(self):+ # The runtime seed uses a double underscore, with dotted domains interpuncted (U+00B7).+ assert python_class_name(domain="alpha", code="Result", needs_qualification=True) == "alpha__Result"+ assert python_class_name(domain="legal.contracts", code="Result", needs_qualification=True) == "legal·contracts__Result"++ def test_ts_type_name_bare_when_unique(self):+ assert ts_type_name(domain="pipeline", code="Report", needs_qualification=False) == "Report"++ def test_ts_type_name_qualified_on_collision(self):+ # TS cannot use the interpunct; a colliding type PascalCases and joins the domain segments.+ assert ts_type_name(domain="alpha", code="Result", needs_qualification=True) == "AlphaResult"+ assert ts_type_name(domain="legal.contracts", code="Result", needs_qualification=True) == "LegalContractsResult"++ def test_ts_name_allocation_disambiguates_non_injective_domains(self):+ library = ResolvedLibrary(+ mthds_version="0.1.0",+ concepts=[+ _resolved_concept(domain="foo.bar", code="Result", needs_qualification=True),+ _resolved_concept(domain="foo_bar", code="Result", needs_qualification=True),+ ],+ )++ assert allocate_ts_type_names(library) == {+ "foo.bar.Result": "FooBarResult",+ "foo_bar.Result": "FooBarResult2",+ }++ def test_ts_name_allocation_preserves_bare_name_over_qualified_collision(self):+ library = ResolvedLibrary(+ mthds_version="0.1.0",+ concepts=[+ _resolved_concept(domain="alpha", code="Result", needs_qualification=True),+ _resolved_concept(domain="other", code="AlphaResult"),+ _resolved_concept(domain="other", code="AlphaResult2"),+ ],+ )++ assert allocate_ts_type_names(library) == {+ "alpha.Result": "AlphaResult3",+ "other.AlphaResult": "AlphaResult",+ "other.AlphaResult2": "AlphaResult2",+ }++ def test_ts_name_allocation_is_independent_of_input_order(self):+ concepts = [+ _resolved_concept(domain="foo.bar", code="Result", needs_qualification=True),+ _resolved_concept(domain="foo_bar", code="Result", needs_qualification=True),+ _resolved_concept(domain="alpha", code="Result", needs_qualification=True),+ _resolved_concept(domain="other", code="AlphaResult"),+ ]++ forward = ResolvedLibrary(mthds_version="0.1.0", concepts=concepts)+ reversed_library = ResolvedLibrary(mthds_version="0.1.0", concepts=list(reversed(concepts)))++ assert allocate_ts_type_names(forward) == allocate_ts_type_names(reversed_library)++ def test_ts_name_allocation_reserves_derived_schema_symbols(self):+ library = ResolvedLibrary(+ mthds_version="0.1.0",+ concepts=[+ _resolved_concept(domain="demo", code="Foo"),+ _resolved_concept(domain="demo", code="FooSchema"),+ ],+ )++ assert allocate_ts_type_names(library) == {+ "demo.Foo": "Foo",+ "demo.FooSchema": "FooSchema2",+ }++ def test_runtime_to_emitted_class_names(self):+ """Runtime-qualified spellings map to the emitted names; natives and opaque classes are skipped."""+ library = ResolvedLibrary(+ mthds_version="0.1.0",+ concepts=[+ _resolved_concept(domain="pipeline", code="Report"),+ _resolved_concept(domain="alpha", code="Result", needs_qualification=True),+ _resolved_concept(domain="native", code="Text", is_native=True),+ _resolved_concept(domain="pipeline", code="Wrapped", opaque_python_class="UserClass"),+ ],+ )++ mapping = runtime_to_emitted_class_names(library)++ # Unique code: runtime-qualified -> bare; collision: identity; native + opaque: absent.+ assert mapping == {+ "pipeline__Report": "Report",+ "alpha__Result": "alpha__Result",+ }diff --git a/tests/unit/pipelex/codegen/test_native_expansion.py b/tests/unit/pipelex/codegen/test_native_expansion.pynew file mode 100644index 000000000..3a7f4684f--- /dev/null+++ b/tests/unit/pipelex/codegen/test_native_expansion.py@@ -0,0 +1,86 @@+import pytest++from pipelex.codegen.native_expansion import materialize_native_concept, reflect_native_structure+from pipelex.core.concepts.concept_factory import ConceptFactory+from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprint, ConceptStructureBlueprintFieldType+from pipelex.core.concepts.native.concept_native import NativeConceptCode++STRUCTURELESS_BY_DESIGN = {NativeConceptCode.DYNAMIC, NativeConceptCode.ANYTHING, NativeConceptCode.COMPOSITE}+++class TestNativeExpansion:+ """Materialization is a lookup into the pinned normative definitions; reflection is the consistency probe."""++ @pytest.mark.parametrize("native_code", list(NativeConceptCode))+ def test_pinned_blueprint_matches_runtime_content_class(self, native_code: NativeConceptCode):+ """The consistency guarantee: each runtime content class still matches its pinned blueprint.++ The pinned definitions (mthds spec `native-concepts.md`) are the authority for crate+ materialization; this probe reflects the runtime class and compares, so a runtime change+ that drifts from the standard fails loudly here.+ """+ pinned = materialize_native_concept(native_code)+ reflected = reflect_native_structure(native_code)+ if native_code in STRUCTURELESS_BY_DESIGN:+ assert pinned.structure is None+ assert reflected is None+ else:+ assert pinned.structure is not None, f"{native_code} must carry a pinned structure"+ assert reflected == pinned.structure, f"{native_code}: runtime content class drifted from its pinned blueprint"++ @pytest.mark.parametrize("native_code", list(NativeConceptCode))+ def test_pinned_description_matches_concept_factory(self, native_code: NativeConceptCode):+ """The pinned description and the runtime native concept's description are the same string (both are hashed surfaces)."""+ pinned = materialize_native_concept(native_code)+ runtime_description = ConceptFactory.make_native_concept(native_concept_code=native_code).description+ assert pinned.description == runtime_description++ def test_image_is_flat_with_paired_pixel_dimensions(self):+ """`Image` pins flat `width`/`height` integer fields — the nested size object is gone (D11 resolution)."""+ blueprint = materialize_native_concept(NativeConceptCode.IMAGE)+ assert isinstance(blueprint.structure, dict)+ assert "size" not in blueprint.structure+ assert {"url", "public_url", "mime_type", "caption", "width", "height", "filename"} <= set(blueprint.structure.keys())+ url_field = blueprint.structure["url"]+ assert isinstance(url_field, ConceptStructureBlueprint)+ assert url_field.type == ConceptStructureBlueprintFieldType.TEXT+ assert url_field.required is True+ for dimension in ("width", "height"):+ dimension_field = blueprint.structure[dimension]+ assert isinstance(dimension_field, ConceptStructureBlueprint)+ assert dimension_field.type == ConceptStructureBlueprintFieldType.INTEGER+ assert dimension_field.required is False++ def test_date_pins_real_structure_with_time_field(self):+ """`Date` is no longer structureless: `date` is a required date, `time` an optional `time` field."""+ blueprint = materialize_native_concept(NativeConceptCode.DATE)+ assert isinstance(blueprint.structure, dict)+ assert set(blueprint.structure.keys()) == {"date", "time"}+ date_field = blueprint.structure["date"]+ assert isinstance(date_field, ConceptStructureBlueprint)+ assert date_field.type == ConceptStructureBlueprintFieldType.DATE+ assert date_field.required is True+ time_field = blueprint.structure["time"]+ assert isinstance(time_field, ConceptStructureBlueprint)+ assert time_field.type == ConceptStructureBlueprintFieldType.TIME+ assert time_field.required is False++ def test_time_pins_single_required_time_field(self):+ blueprint = materialize_native_concept(NativeConceptCode.TIME)+ assert isinstance(blueprint.structure, dict)+ assert set(blueprint.structure.keys()) == {"time"}+ time_field = blueprint.structure["time"]+ assert isinstance(time_field, ConceptStructureBlueprint)+ assert time_field.type == ConceptStructureBlueprintFieldType.TIME+ assert time_field.required is True++ def test_json_dict_uses_spec_vocabulary_spellings(self):+ """The pinned JSON dict uses the spec vocabulary key type (`text`) and the reserved `Any` value marker."""+ blueprint = materialize_native_concept(NativeConceptCode.JSON)+ assert isinstance(blueprint.structure, dict)+ json_field = blueprint.structure["json_obj"]+ assert isinstance(json_field, ConceptStructureBlueprint)+ assert json_field.type == ConceptStructureBlueprintFieldType.DICT+ assert json_field.key_type == "text"+ assert json_field.value_type == "Any"+ assert json_field.required is Truediff --git a/tests/unit/pipelex/codegen/test_python_pydantic_emitter.py b/tests/unit/pipelex/codegen/test_python_pydantic_emitter.pynew file mode 100644index 000000000..385e31c15--- /dev/null+++ b/tests/unit/pipelex/codegen/test_python_pydantic_emitter.py@@ -0,0 +1,117 @@+from datetime import date, datetime, time+from pathlib import Path++from pipelex.codegen.emitters.python_pydantic import emit_python_pydantic+from pipelex.codegen.resolved_concepts import resolve_concepts_from_crate+from pipelex.core.concepts.concept_blueprint import ConceptBlueprint+from pipelex.libraries.library_crate import LibraryCrate+from tests.unit.pipelex.codegen.conftest import load_generated_module+++class TestPythonPydanticEmitter:+ """Unit tests for the plain-pydantic types emitter (no Pipelex dependency)."""++ def test_emits_a_single_models_module(self, pipeline_crate: LibraryCrate):+ files = emit_python_pydantic(resolve_concepts_from_crate(pipeline_crate))+ assert [file.filename for file in files] == ["models.py"]++ def test_generated_module_is_pipelex_free_and_uses_modern_typing(self, pipeline_crate: LibraryCrate):+ content = emit_python_pydantic(resolve_concepts_from_crate(pipeline_crate))[0].content+ assert "pipelex" not in content+ assert "StructuredContent" not in content+ # Modern typing: builtin generics and `| None`, not typing.List / typing.Optional.+ assert "list[str] | None" in content+ assert "from typing import Optional" not in content+ assert "from typing import List" not in content++ def test_generated_module_compiles_and_validates(self, pipeline_crate: LibraryCrate, tmp_path: Path):+ content = emit_python_pydantic(resolve_concepts_from_crate(pipeline_crate))[0].content+ module = load_generated_module(content, tmp_path=tmp_path, name="gen_models")+ # Forward-referenced nested model resolves; defaults apply.+ report = module.Report(score={"value": 4.0}, label={"text": "x"})+ assert report.score.value == 4.0+ assert report.status == "draft"++ def test_temporal_defaults_generate_importable_models(self, temporal_defaults_crate: LibraryCrate, tmp_path: Path):+ content = emit_python_pydantic(resolve_concepts_from_crate(temporal_defaults_crate))[0].content+ module = load_generated_module(content, tmp_path=tmp_path, name="gen_models_temporal_defaults")+ event = module.Event()+ assert event.starts_on == date(2026, 7, 11)+ assert event.recorded_at == datetime(2026, 7, 11, 9, 30)+ assert event.starts_at == time(9, 30)++ def test_dict_defaults_are_canonical(self, reordered_dict_default_crates: tuple[LibraryCrate, LibraryCrate]):+ first_crate, second_crate = reordered_dict_default_crates++ first_output = emit_python_pydantic(resolve_concepts_from_crate(first_crate))+ second_output = emit_python_pydantic(resolve_concepts_from_crate(second_crate))++ assert first_output == second_output+ assert 'default={"alpha": "first", "zeta": "last"}' in first_output[0].content++ def test_serialize_parse_round_trip(self, pipeline_crate: LibraryCrate, tmp_path: Path):+ """The generated pydantic helpers round-trip: model_dump (serialize) then model_validate (parse)+ reproduce the value. Wire names are already snake_case Python names, so no binder is needed.+ """+ module = load_generated_module(+ emit_python_pydantic(resolve_concepts_from_crate(pipeline_crate))[0].content, tmp_path=tmp_path, name="gen_models_roundtrip"+ )+ report = module.Report(score={"value": 4.0, "rationale": "high"}, label={"text": "x"}, tags=["a", "b"])+ wire = report.model_dump(mode="json")+ assert module.Report.model_validate(wire) == report++ def test_native_concept_is_emitted_as_a_model(self, pipeline_crate: LibraryCrate):+ content = emit_python_pydantic(resolve_concepts_from_crate(pipeline_crate))[0].content+ # Uniform treatment: the materialized native becomes a plain model (self-contained crate).+ assert "class Text(BaseModel):" in content++ def test_refines_native_extends_the_emitted_base(self, refines_crate: LibraryCrate, tmp_path: Path):+ content = emit_python_pydantic(resolve_concepts_from_crate(refines_crate))[0].content+ assert "class Thumbnail(Image):" in content+ # The base is defined before the subclass (inheritance is eager) and the module imports cleanly.+ load_generated_module(content, tmp_path=tmp_path, name="gen_models_refines")++ def test_dict_fields_render_honestly_and_round_trip(self, materialized_image_crate: LibraryCrate, tmp_path: Path):+ """The DICT path — both an authored unspecified-values dict (imprecision surfaced) and an+ authored typed dict — renders honest annotations in a module that compiles and round-trips,+ and the pinned `Image` materializes flat `width`/`height` integers (D11 resolution).+ """+ content = emit_python_pydantic(resolve_concepts_from_crate(materialized_image_crate))[0].content+ assert "metadata: dict[str, Any] | None" in content+ assert "imprecise: dict value type unspecified" in content+ assert "captions: dict[str, str]" in content+ assert "width: int | None" in content+ assert "height: int | None" in content+ assert "size" not in content+ module = load_generated_module(content, tmp_path=tmp_path, name="gen_models_dict")+ image = module.Image(url="pipelex-storage://runs/abc/image.png", width=512, height=256)+ assert module.Image.model_validate(image.model_dump(mode="json")) == image++ def test_opaque_concepts_pass_content_through(self, edge_crate: LibraryCrate, tmp_path: Path):+ """Opaque = pass-through, never lossy (B1-1): a structureless / Python-class-backed concept+ validates any payload and keeps every field verbatim (extra="allow"), instead of pydantic's+ default extra="ignore" silently stripping the content.+ """+ content = emit_python_pydantic(resolve_concepts_from_crate(edge_crate))[0].content+ assert 'model_config = ConfigDict(extra="allow")' in content+ module = load_generated_module(content, tmp_path=tmp_path, name="gen_models_opaque")+ payload = {"kind": "legacy", "nested": {"x": 1}}+ legacy = module.Legacy.model_validate(payload)+ assert legacy.model_dump() == payload+ blob = module.Blob.model_validate(payload)+ assert blob.model_dump() == payload++ def test_unresolved_refinement_passes_content_through(self, tmp_path: Path):+ crate = LibraryCrate(concepts={"consumer.ExternalReport": ConceptBlueprint(description="External report", refines="vendor->reports.Report")})+ content = emit_python_pydantic(resolve_concepts_from_crate(crate))[0].content+ module = load_generated_module(content, tmp_path=tmp_path, name="gen_models_external_refinement")+ payload = {"kind": "external", "nested": {"score": 3}}++ report = module.ExternalReport.model_validate(payload)++ assert report.model_dump() == payload++ def test_collision_qualifies_class_names(self, edge_crate: LibraryCrate):+ content = emit_python_pydantic(resolve_concepts_from_crate(edge_crate))[0].content+ assert "class alpha__Result(BaseModel):" in content+ assert "class beta__Result(BaseModel):" in contentdiff --git a/tests/unit/pipelex/codegen/test_python_structures_emitter.py b/tests/unit/pipelex/codegen/test_python_structures_emitter.pynew file mode 100644index 000000000..9031fd93e--- /dev/null+++ b/tests/unit/pipelex/codegen/test_python_structures_emitter.py@@ -0,0 +1,84 @@+from datetime import date, datetime, time+from pathlib import Path+from typing import Any++from pipelex.codegen.emitters.python_structures import emit_python_structures+from pipelex.codegen.resolved_concepts import resolve_concepts_from_crate+from pipelex.libraries.library_crate import LibraryCrate+from tests.unit.pipelex.codegen.conftest import load_generated_module+++class TestPythonStructuresEmitter:+ """Unit tests for the python-structures (runtime StructuredContent) types emitter."""++ def test_emits_a_single_structures_module(self, pipeline_crate: LibraryCrate):+ files = emit_python_structures(resolve_concepts_from_crate(pipeline_crate))+ assert [file.filename for file in files] == ["structures.py"]++ def test_header_carries_the_extension_file_story(self, pipeline_crate: LibraryCrate):+ content = emit_python_structures(resolve_concepts_from_crate(pipeline_crate))[0].content+ assert "DO NOT EDIT" in content+ assert "subclass" in content+ assert "projection: types / python-structures" in content++ def test_generated_module_compiles_and_builds_runtime_classes(self, pipeline_crate: LibraryCrate, tmp_path: Path):+ content = emit_python_structures(resolve_concepts_from_crate(pipeline_crate))[0].content+ module = load_generated_module(content, tmp_path=tmp_path, name="gen_structures")+ report_cls: Any = module.Report+ # MRO check by name (avoids narrowing the runtime-only class away from its generated params).+ assert "StructuredContent" in {base.__name__ for base in report_cls.__mro__}+ # Cross-concept forward ref (Score) and native mapping (label -> TextContent) both resolve.+ report: Any = report_cls(score={"value": 2.0}, label={"text": "hi"}, status="draft")+ assert report.score.value == 2.0+ assert type(report.label).__name__ == "TextContent"++ def test_temporal_defaults_generate_importable_structures(self, temporal_defaults_crate: LibraryCrate, tmp_path: Path):+ content = emit_python_structures(resolve_concepts_from_crate(temporal_defaults_crate))[0].content+ module = load_generated_module(content, tmp_path=tmp_path, name="gen_structures_temporal_defaults")+ event: Any = module.Event()+ assert event.starts_on == date(2026, 7, 11)+ assert event.recorded_at == datetime(2026, 7, 11, 9, 30)+ assert event.starts_at == time(9, 30)++ def test_native_concept_is_not_re_emitted(self, pipeline_crate: LibraryCrate):+ content = emit_python_structures(resolve_concepts_from_crate(pipeline_crate))[0].content+ # native.Text maps to the runtime TextContent (imported), never a re-declared class.+ assert "class Text(" not in content+ assert "from pipelex.core.stuffs.text_content import TextContent" in content++ def test_refines_native_becomes_content_base_class(self, refines_crate: LibraryCrate):+ content = emit_python_structures(resolve_concepts_from_crate(refines_crate))[0].content+ assert "class Thumbnail(ImageContent):" in content++ def test_collision_qualifies_class_names(self, edge_crate: LibraryCrate):+ content = emit_python_structures(resolve_concepts_from_crate(edge_crate))[0].content+ assert "class alpha__Result(" in content+ assert "class beta__Result(" in content++ def test_typed_dict_field_maps_to_dict_annotation(self, materialized_image_crate: LibraryCrate, tmp_path: Path):+ """An authored typed dict renders the runtime Dict annotation (natives are skipped by this+ emitter, so the authored concept is the DICT path's only route here) and builds at runtime.+ """+ content = emit_python_structures(resolve_concepts_from_crate(materialized_image_crate))[0].content+ assert "captions: Dict[str, str]" in content+ module = load_generated_module(content, tmp_path=tmp_path, name="gen_structures_dict")+ gallery: Any = module.Gallery(captions={"p1": "sunset"})+ assert gallery.captions == {"p1": "sunset"}++ def test_imprecision_and_opaque_are_surfaced(self, edge_crate: LibraryCrate, tmp_path: Path):+ content = emit_python_structures(resolve_concepts_from_crate(edge_crate))[0].content+ assert "# imprecise: list item type unspecified" in content+ assert "Imprecise: concept is backed by the Python class 'MyLegacyClass'" in content+ # Still valid, importable Python despite the opaque concepts.+ load_generated_module(content, tmp_path=tmp_path, name="gen_structures_edge")++ def test_opaque_concepts_pass_content_through(self, edge_crate: LibraryCrate, tmp_path: Path):+ """Opaque = pass-through, never lossy (B1-1): the runtime base inherits pydantic's default+ extra="ignore", so without extra="allow" an opaque class would silently strip every field.+ """+ content = emit_python_structures(resolve_concepts_from_crate(edge_crate))[0].content+ assert 'model_config = ConfigDict(extra="allow")' in content+ module = load_generated_module(content, tmp_path=tmp_path, name="gen_structures_opaque")+ payload = {"kind": "legacy", "nested": {"x": 1}}+ legacy: Any = module.Legacy.model_validate(payload)+ assert legacy.model_dump() == payloaddiff --git a/tests/unit/pipelex/codegen/test_resolved_concepts.py b/tests/unit/pipelex/codegen/test_resolved_concepts.pynew file mode 100644index 000000000..95e9d380c--- /dev/null+++ b/tests/unit/pipelex/codegen/test_resolved_concepts.py@@ -0,0 +1,75 @@+from pipelex.codegen.resolved_concepts import resolve_concepts_from_crate+from pipelex.codegen.resolved_fields import ResolvedTypeKind, iter_imprecision_reasons+from pipelex.core.concepts.concept_blueprint import ConceptBlueprint+from pipelex.libraries.library_crate import LibraryCrate+++class TestResolvedConcepts:+ """Unit tests for resolving a normalized crate into the neutral emitter input."""++ def test_pipeline_concepts_resolve_with_native_and_qualified_refs(self, pipeline_crate: LibraryCrate):+ library = resolve_concepts_from_crate(pipeline_crate)+ by_ref = library.by_ref()++ assert library.mthds_version == "1.0.0-test"+ # The referenced native was materialized into the crate and is flagged native.+ assert by_ref["native.Text"].is_native is True+ # Unique codes need no qualification.+ assert by_ref["pipeline.Report"].needs_qualification is False++ report = by_ref["pipeline.Report"]+ assert report.structureless is False+ fields = {field.name: field for field in report.fields}+ # In-body concept ref is qualified; the native ref is flagged native.+ assert fields["score"].resolved_type.kind is ResolvedTypeKind.CONCEPT+ assert fields["score"].resolved_type.concept_ref == "pipeline.Score"+ assert fields["label"].resolved_type.concept_ref == "native.Text"+ assert fields["label"].resolved_type.is_native is True+ # The literal-with-default keeps its default.+ assert fields["status"].resolved_type.kind is ResolvedTypeKind.LITERAL+ assert fields["status"].default_value == "draft"++ def test_cross_domain_collision_flags_qualification(self, edge_crate: LibraryCrate):+ library = resolve_concepts_from_crate(edge_crate)+ by_ref = library.by_ref()+ # A code shared across two domains forces qualification on both.+ assert by_ref["alpha.Result"].needs_qualification is True+ assert by_ref["beta.Result"].needs_qualification is True+ # A code unique to one domain does not.+ assert by_ref["alpha.Blob"].needs_qualification is False++ def test_imprecise_field_carries_a_reason(self, edge_crate: LibraryCrate):+ result = resolve_concepts_from_crate(edge_crate).by_ref()["alpha.Result"]+ items_field = next(field for field in result.fields if field.name == "items")+ reasons = iter_imprecision_reasons(items_field.resolved_type)+ assert reasons+ assert "unspecified" in reasons[0]++ def test_structureless_concept_is_opaque_with_a_reason(self, edge_crate: LibraryCrate):+ blob = resolve_concepts_from_crate(edge_crate).by_ref()["alpha.Blob"]+ assert blob.structureless is True+ assert blob.fields == []+ assert blob.opaque_python_class is None+ assert blob.imprecision_reason is not None++ def test_python_backed_concept_is_surfaced_not_silently_emitted(self, edge_crate: LibraryCrate):+ legacy = resolve_concepts_from_crate(edge_crate).by_ref()["alpha.Legacy"]+ assert legacy.structureless is True+ assert legacy.opaque_python_class == "MyLegacyClass"+ assert "MyLegacyClass" in (legacy.imprecision_reason or "")++ def test_refines_native_keeps_base_ref(self, refines_crate: LibraryCrate):+ thumbnail = resolve_concepts_from_crate(refines_crate).by_ref()["media.Thumbnail"]+ assert thumbnail.base_ref == "native.Image"+ assert thumbnail.structureless is False+ assert thumbnail.fields == []++ def test_unresolved_refinement_is_opaque(self):+ crate = LibraryCrate(concepts={"consumer.ExternalReport": ConceptBlueprint(description="External report", refines="vendor->reports.Report")})++ report = resolve_concepts_from_crate(crate).by_ref()["consumer.ExternalReport"]++ assert report.base_ref == "vendor->reports.Report"+ assert report.structureless is True+ assert report.fields == []+ assert "not available in this crate" in (report.imprecision_reason or "")diff --git a/tests/unit/pipelex/codegen/test_stamp.py b/tests/unit/pipelex/codegen/test_stamp.pynew file mode 100644index 000000000..98f079571--- /dev/null+++ b/tests/unit/pipelex/codegen/test_stamp.py@@ -0,0 +1,75 @@+"""The stamp writer/parser: round-trip, tamper detection, comment syntax, and options/pipe_ref carry."""++from __future__ import annotations++import pytest++from pipelex.codegen.emitters.target import CodegenKind, CodegenTarget+from pipelex.codegen.exceptions import CodegenStampError+from pipelex.codegen.stamp import apply_stamp, comment_prefix_for, compute_content_hash, has_stamp, parse_stamped+++class TestStamp:+ def _stamp(self, body: str, *, comment_prefix: str, pipe_ref: str | None = None, options: dict[str, str] | None = None) -> str:+ return apply_stamp(+ body,+ crate_fingerprint="fp-123",+ engine_version="0.42.0",+ kind=CodegenKind.TYPES,+ target=CodegenTarget.PYTHON_PYDANTIC,+ pipe_ref=pipe_ref,+ options=options or {},+ comment_prefix=comment_prefix,+ )++ def test_round_trip_preserves_body_and_fields(self) -> None:+ body = "# header\nfrom __future__ import annotations\n\nclass Foo:\n pass\n"+ stamped = self._stamp(body, comment_prefix="#")+ parsed = parse_stamped(stamped, comment_prefix="#")+ assert parsed is not None+ assert parsed.body == body # byte-exact body below the stamp+ assert parsed.stamp.crate_fingerprint == "fp-123"+ assert parsed.stamp.engine_version == "0.42.0"+ assert parsed.stamp.kind == CodegenKind.TYPES+ assert parsed.stamp.target == CodegenTarget.PYTHON_PYDANTIC+ assert parsed.stamp.content_hash == compute_content_hash(body)++ def test_content_hash_covers_only_the_body_not_the_stamp(self) -> None:+ body = "// x\nexport type A = number;\n"+ stamped = self._stamp(body, comment_prefix="//")+ parsed = parse_stamped(stamped, comment_prefix="//")+ assert parsed is not None+ # Recomputing over the recovered body reproduces the recorded hash — the self-describing property.+ assert compute_content_hash(parsed.body) == parsed.stamp.content_hash++ def test_tampering_below_the_stamp_breaks_the_recomputed_hash(self) -> None:+ body = "# a\nclass A:\n pass\n"+ stamped = self._stamp(body, comment_prefix="#")+ tampered = stamped + "x = 1\n"+ parsed = parse_stamped(tampered, comment_prefix="#")+ assert parsed is not None+ assert compute_content_hash(parsed.body) != parsed.stamp.content_hash++ def test_pipe_ref_and_options_survive_the_round_trip(self) -> None:+ stamped = self._stamp("// b\n", comment_prefix="//", pipe_ref="documents.extract", options={"explicit": "true"})+ parsed = parse_stamped(stamped, comment_prefix="//")+ assert parsed is not None+ assert parsed.stamp.pipe_ref == "documents.extract"+ assert parsed.stamp.options == {"explicit": "true"}++ def test_ts_uses_double_slash_comment_prefix(self) -> None:+ stamped = self._stamp("// b\n", comment_prefix="//")+ assert stamped.startswith("// >>> pipelex-codegen-stamp >>>")+ assert has_stamp(stamped, comment_prefix="//")+ assert not has_stamp(stamped, comment_prefix="#")++ def test_unstamped_text_parses_to_none(self) -> None:+ assert parse_stamped("class A: pass\n", comment_prefix="#") is None++ def test_comment_prefix_by_suffix(self) -> None:+ assert comment_prefix_for("models.py") == "#"+ assert comment_prefix_for("types.ts") == "//"++ def test_unstampable_file_type_raises(self) -> None:+ with pytest.raises(CodegenStampError):+ comment_prefix_for("data.json")diff --git a/tests/unit/pipelex/codegen/test_ts_zod_emitter.py b/tests/unit/pipelex/codegen/test_ts_zod_emitter.pynew file mode 100644index 000000000..15e072e1d--- /dev/null+++ b/tests/unit/pipelex/codegen/test_ts_zod_emitter.py@@ -0,0 +1,264 @@+from pipelex.codegen.emitters.ts_zod import emit_ts_zod+from pipelex.codegen.resolved_concepts import ResolvedConcept, ResolvedLibrary, resolve_concepts_from_crate+from pipelex.core.concepts.concept_blueprint import ConceptBlueprint+from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprint, ConceptStructureBlueprintFieldType+from pipelex.libraries.library_crate import LibraryCrate+++class TestTsZodEmitter:+ """Unit tests for the ts-zod pure types-file emitter.++ The `tsc --strict` compile gate on emitted TS lives in the `conformance/` cross-repo harness (D7);+ here we assert the structural contract of the pure types file.+ """++ def test_emits_a_pure_types_file_and_a_binder(self, pipeline_crate: LibraryCrate):+ files = emit_ts_zod(resolve_concepts_from_crate(pipeline_crate))+ assert [file.filename for file in files] == ["types.ts", "binder.ts"]+ content = files[0].content+ # types.ts is pure: imports only zod, nothing Pipelex.+ assert 'import { z } from "zod";' in content+ assert "pipelex" not in content+ assert "projection: types / ts-zod" in content++ def test_binder_validates_per_concept_over_the_types_file(self, pipeline_crate: LibraryCrate):+ binder = emit_ts_zod(resolve_concepts_from_crate(pipeline_crate))[1].content+ # The binder depends on the pure types file, not on zod directly.+ assert 'from "./types";' in binder+ assert "ReportSchema,\n type Report," in binder+ # Keys are wire-native, so parse/serialize are a direct Schema.parse — no key remapping helper.+ assert "mapKeysDeep" not in binder+ assert "toCamel" not in binder+ assert "export function parseReport(wire: unknown): Report {" in binder+ assert "return ReportSchema.parse(wire);" in binder+ assert "export function serializeReport(value: Report): Report {" in binder++ def test_schema_and_inferred_type_per_concept(self, pipeline_crate: LibraryCrate):+ content = emit_ts_zod(resolve_concepts_from_crate(pipeline_crate))[0].content+ assert "export const ReportSchema = z.object({" in content+ assert "export type Report = z.infer<typeof ReportSchema>;" in content++ def test_concept_refs_use_lazy_and_literals_keep_defaults(self, pipeline_crate: LibraryCrate):+ content = emit_ts_zod(resolve_concepts_from_crate(pipeline_crate))[0].content+ # A concept reference is a forward-safe lazy schema; the literal-with-default is a defaulted enum.+ assert "score: z.lazy(() => ScoreSchema).optional()" in content+ assert 'status: z.enum(["draft", "final"]).default("draft")' in content++ def test_temporal_defaults_emit_iso_wire_strings(self, temporal_defaults_crate: LibraryCrate):+ content = emit_ts_zod(resolve_concepts_from_crate(temporal_defaults_crate))[0].content+ assert 'starts_on: z.string().default("2026-07-11")' in content+ assert 'recorded_at: z.string().default("2026-07-11T09:30:00")' in content+ assert 'starts_at: z.string().default("09:30:00")' in content++ def test_dict_defaults_are_canonical(self, reordered_dict_default_crates: tuple[LibraryCrate, LibraryCrate]):+ first_crate, second_crate = reordered_dict_default_crates++ first_output = emit_ts_zod(resolve_concepts_from_crate(first_crate))+ second_output = emit_ts_zod(resolve_concepts_from_crate(second_crate))++ assert first_output == second_output+ assert 'default({"alpha": "first", "zeta": "last"})' in first_output[0].content++ def test_explicitly_empty_structure_emits_an_empty_object_schema(self):+ crate = LibraryCrate(+ concepts={+ "demo.Marker": ConceptBlueprint(description="An empty marker", structure={}),+ "demo.Opaque": ConceptBlueprint(description="An opaque payload"),+ }+ )++ content = emit_ts_zod(resolve_concepts_from_crate(crate))[0].content++ assert "export const MarkerSchema = z.object({});" in content+ assert "export const OpaqueSchema = z.unknown();" in content++ def test_self_recursive_concept_uses_an_explicit_type_and_annotated_schema(self):+ crate = LibraryCrate(+ concepts={+ "graph.Node": ConceptBlueprint(+ description="A recursive node",+ structure={+ "next": ConceptStructureBlueprint(+ description="Next node",+ type=ConceptStructureBlueprintFieldType.CONCEPT,+ concept_ref="graph.Node",+ required=False,+ )+ },+ )+ }+ )++ content = emit_ts_zod(resolve_concepts_from_crate(crate))[0].content++ assert "export type Node = {\n next?: Node;\n};" in content+ assert "export const NodeSchema: z.ZodType<Node> = z.object({" in content++ def test_mutually_recursive_concepts_use_explicit_types_and_annotated_schemas(self):+ crate = LibraryCrate(+ concepts={+ "graph.Left": ConceptBlueprint(+ description="Left node",+ structure={+ "right": ConceptStructureBlueprint(+ description="Right node",+ type=ConceptStructureBlueprintFieldType.CONCEPT,+ concept_ref="graph.Right",+ required=True,+ )+ },+ ),+ "graph.Right": ConceptBlueprint(+ description="Right node",+ structure={+ "left": ConceptStructureBlueprint(+ description="Left node",+ type=ConceptStructureBlueprintFieldType.CONCEPT,+ concept_ref="graph.Left",+ required=True,+ )+ },+ ),+ }+ )++ content = emit_ts_zod(resolve_concepts_from_crate(crate))[0].content++ assert "export type Left = {\n right: Right;\n};" in content+ assert "export const LeftSchema: z.ZodType<Left> = z.object({" in content+ assert "export type Right = {\n left: Left;\n};" in content+ assert "export const RightSchema: z.ZodType<Right> = z.object({" in content++ def test_field_keys_are_wire_native_snake_case(self, edge_crate: LibraryCrate):+ # Keys are the crate's snake_case field names verbatim (D10) — the schema validates the wire+ # directly, with no camelCase remapping layer that could corrupt nested record/opaque data.+ content = emit_ts_zod(resolve_concepts_from_crate(edge_crate))[0].content+ assert "item_count: z.number().int()" in content+ assert "itemCount" not in content+ assert "@wire" not in content++ def test_refines_native_renders_a_lazy_base_schema(self, pipeline_crate: LibraryCrate):+ # Summary refines native.Text (kept, not flattened) — a forward-safe lazy ref to the native.+ content = emit_ts_zod(resolve_concepts_from_crate(pipeline_crate))[0].content+ assert "export const SummarySchema = z.lazy(() => TextSchema);" in content++ def test_collision_qualifies_type_names(self, edge_crate: LibraryCrate):+ content = emit_ts_zod(resolve_concepts_from_crate(edge_crate))[0].content+ assert "export const AlphaResultSchema = z.object({" in content+ assert "export const BetaResultSchema = z.object({" in content++ def test_collision_safe_names_are_shared_by_definitions_references_and_binders(self):+ concepts = [+ ResolvedConcept(+ concept_ref="foo.bar.Result",+ domain="foo.bar",+ code="Result",+ description="Hierarchical result",+ is_native=False,+ needs_qualification=True,+ base_ref="foo_bar.Result",+ fields=[],+ structureless=False,+ imprecision_reason=None,+ opaque_python_class=None,+ ),+ ResolvedConcept(+ concept_ref="foo_bar.Result",+ domain="foo_bar",+ code="Result",+ description="Underscored result",+ is_native=False,+ needs_qualification=True,+ base_ref=None,+ fields=[],+ structureless=True,+ imprecision_reason="concept declares no structure",+ opaque_python_class=None,+ ),+ ResolvedConcept(+ concept_ref="alpha.Result",+ domain="alpha",+ code="Result",+ description="Qualified result",+ is_native=False,+ needs_qualification=True,+ base_ref=None,+ fields=[],+ structureless=True,+ imprecision_reason="concept declares no structure",+ opaque_python_class=None,+ ),+ ResolvedConcept(+ concept_ref="other.AlphaResult",+ domain="other",+ code="AlphaResult",+ description="Bare alpha result",+ is_native=False,+ needs_qualification=False,+ base_ref=None,+ fields=[],+ structureless=True,+ imprecision_reason="concept declares no structure",+ opaque_python_class=None,+ ),+ ]+ forward = emit_ts_zod(ResolvedLibrary(mthds_version="0.1.0", concepts=concepts))+ reversed_output = emit_ts_zod(ResolvedLibrary(mthds_version="0.1.0", concepts=list(reversed(concepts))))++ assert forward == reversed_output+ types_content = forward[0].content+ binder_content = forward[1].content+ assert "export const FooBarResultSchema = z.lazy(() => FooBarResult2Schema);" in types_content+ assert types_content.count("export const FooBarResultSchema") == 1+ assert types_content.count("export const FooBarResult2Schema") == 1+ assert types_content.count("export const AlphaResultSchema") == 1+ assert types_content.count("export const AlphaResult2Schema") == 1+ assert binder_content.count("export function parseFooBarResult(") == 1+ assert binder_content.count("export function parseFooBarResult2(") == 1+ assert binder_content.count("export function parseAlphaResult(") == 1+ assert binder_content.count("export function parseAlphaResult2(") == 1++ def test_schema_derived_names_do_not_collide_with_authored_type_names(self):+ concepts = [+ ResolvedConcept(+ concept_ref=f"demo.{code}",+ domain="demo",+ code=code,+ description=f"{code} concept",+ is_native=False,+ needs_qualification=False,+ base_ref=None,+ fields=[],+ structureless=True,+ imprecision_reason="concept declares no structure",+ opaque_python_class=None,+ )+ for code in ("Foo", "FooSchema")+ ]++ types_file, binder_file = emit_ts_zod(ResolvedLibrary(mthds_version="0.1.0", concepts=concepts))++ assert "export const FooSchema = z.unknown();" in types_file.content+ assert "export type FooSchema2 = z.infer<typeof FooSchema2Schema>;" in types_file.content+ assert " FooSchema,\n type Foo," in binder_file.content+ assert " FooSchema2Schema,\n type FooSchema2," in binder_file.content++ def test_dict_fields_render_records_honestly(self, materialized_image_crate: LibraryCrate):+ """The DICT path — an authored unspecified-values dict and an authored typed dict — renders+ honest z.record schemas with the imprecision surfaced, never a guessed shape; the pinned+ `Image` materializes flat pixel dimensions.+ """+ content = emit_ts_zod(resolve_concepts_from_crate(materialized_image_crate))[0].content+ assert "metadata: z.record(z.string(), z.unknown()).optional()" in content+ assert "@imprecise dict value type unspecified" in content+ assert "captions: z.record(z.string(), z.string())" in content+ assert "width: z.number().int().optional()" in content++ def test_imprecision_and_opaque_are_surfaced(self, edge_crate: LibraryCrate):+ content = emit_ts_zod(resolve_concepts_from_crate(edge_crate))[0].content+ # An untyped list is an honest z.unknown() item plus a JSDoc caveat, never a guessed shape.+ assert "items: z.array(z.unknown())" in content+ assert "@imprecise list item type unspecified" in content+ # A structureless / Python-backed concept is opaque unknown, surfaced.+ assert "export const BlobSchema = z.unknown();" in content+ assert "export const LegacySchema = z.unknown();" in contentdiff --git a/tests/unit/pipelex/core/concepts/concept_blueprint/test_concept_blueprint.py b/tests/unit/pipelex/core/concepts/concept_blueprint/test_concept_blueprint.pyindex 77bb6fef3..f709d9ef7 100644--- a/tests/unit/pipelex/core/concepts/concept_blueprint/test_concept_blueprint.py+++ b/tests/unit/pipelex/core/concepts/concept_blueprint/test_concept_blueprint.py@@ -5,7 +5,7 @@ from datetime import date, datetimeimport pytestfrom pydantic import ValidationError-from pipelex.core.concepts.concept_blueprint import ConceptStructureBlueprint+from pipelex.core.concepts.concept_blueprint import ConceptBlueprint, ConceptStructureBlueprintfrom pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprintFieldType@@ -158,6 +158,21 @@ class TestConceptStructureBlueprintValidation:with pytest.raises(ValidationError, match="value_type must not be empty"):ConceptStructureBlueprint(description="Dict field without value_type", type=ConceptStructureBlueprintFieldType.DICT, key_type="text")+ @pytest.mark.parametrize("key_type", ["integer", "boolean", "date"])+ def test_dict_rejects_non_text_key_type(self, key_type: str):+ with pytest.raises(ValidationError, match="key_type must be 'text'"):+ ConceptStructureBlueprint(+ description="Unsupported map key",+ type=ConceptStructureBlueprintFieldType.DICT,+ key_type=key_type,+ value_type="text",+ )++ @pytest.mark.parametrize("field_name", ["not-valid", "2fast", "class"])+ def test_concept_rejects_field_names_that_are_not_python_identifiers(self, field_name: str):+ with pytest.raises(ValidationError, match="valid Python identifiers"):+ ConceptBlueprint(description="Invalid field name", structure={field_name: "A value"})+def test_edge_cases(self):"""Test edge cases for validation."""# Valid field without default_valuediff --git a/tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator.py b/tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator.pyindex 947f2f67d..818f772ca 100644--- a/tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator.py+++ b/tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator.py@@ -1,4 +1,4 @@-from datetime import date, datetime, timedelta, timezone+from datetime import date, datetime, time, timedelta, timezonefrom typing import castimport pytest@@ -414,6 +414,24 @@ class TypeMappingTest(StructuredContent):assert 'recorded_at: datetime = Field(..., description="Record timestamp")' in resultassert cast("type[StructuredContent]", generated_class).model_json_schema()["properties"]["recorded_at"]["format"] == "date-time"+ def test_time_field_maps_to_time_of_day(self):+ """`type = "time"` generates a `datetime.time` field that survives exec-validation.++ Regression: `_validate_execution`'s exec_globals seeded `date`/`datetime` but not `time`,+ so any bundle with a `time` field crashed library loading with `NameError: name 'time'`.+ """+ structure_blueprint = {+ "issued_at": ConceptStructureBlueprint(description="Issue time", type=ConceptStructureBlueprintFieldType.TIME, required=True),+ }++ result, generated_class = StructureGenerator().generate_from_structure_blueprint(+ class_name="TimeFieldTest", structure_blueprint=structure_blueprint+ )++ assert "from datetime import time" in result+ assert 'issued_at: time = Field(..., description="Issue time")' in result+ assert cast("type[StructuredContent]", generated_class).model_json_schema()["properties"]["issued_at"]["format"] == "time"+def test_date_field_with_default_round_trips(self):"""A `type = "date"` field with a date default generates code that evaluates.@@ -449,6 +467,22 @@ class TypeMappingTest(StructuredContent):assert "datetime.datetime(" not in resultassert cast("type[StructuredContent]", generated_class).model_fields["recorded_at"].default == offset_default+ def test_time_field_with_default_round_trips(self):+ """A `type = "time"` default evaluates through the same bare-class import as its annotation."""+ time_default = time(9, 30, 15)+ structure_blueprint = {+ "starts_at": ConceptStructureBlueprint(+ description="Start time", type=ConceptStructureBlueprintFieldType.TIME, default_value=time_default+ ),+ }++ result, generated_class = StructureGenerator().generate_from_structure_blueprint(+ class_name="TimeDefaultTest", structure_blueprint=structure_blueprint+ )++ assert "datetime.time(" not in result+ assert cast("type[StructuredContent]", generated_class).model_fields["starts_at"].default == time_default+def test_list_of_date_field_maps_to_list_of_calendar_date(self):"""`type = "list", item_type = "date"` generates a `List[date]` field with the bare-class import, and exec-validates."""structure_blueprint = {diff --git a/tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator_concept_refs.py b/tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator_concept_refs.pyindex 738f9125b..419d40d9b 100644--- a/tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator_concept_refs.py+++ b/tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator_concept_refs.py@@ -5,7 +5,7 @@ referencing other concepts (type = "concept" with concept_ref)."""from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprint, ConceptStructureBlueprintFieldType-from pipelex.core.concepts.structure_generation.generator import ConceptClassInfo, StructureGenerator+from pipelex.core.concepts.structure_generation.generator import StructureGeneratorfrom pipelex.core.stuffs.structured_content import StructuredContentAUTOGEN_HEADER = '''\@@ -360,177 +360,3 @@ class ComplexEntity(StructuredContent):assert "from pipelex.core.stuffs.image_content import ImageContent" in generated_codeassert "images: List[ImageContent] = Field(...," in generated_codeassert issubclass(generated_class, StructuredContent)-- def test_concept_ref_with_module_path_generates_import(self):- """Test that concept_ref with module_path generates proper import statements."""- structure_blueprint = {- "skill": ConceptStructureBlueprint(- description="The skill being evaluated",- type=ConceptStructureBlueprintFieldType.CONCEPT,- concept_ref="cv_tech_screening.Skill",- required=True,- ),- "matches_requirement": ConceptStructureBlueprint(- description="Whether this skill matches a job requirement",- type=ConceptStructureBlueprintFieldType.BOOLEAN,- required=True,- ),- }-- concept_ref_to_class_info = {- "cv_tech_screening.Skill": ConceptClassInfo(- class_name="Skill",- module_path="pipeline_01.structures.cv_tech_screening__skill",- ),- }-- generator = StructureGenerator(concept_ref_to_class_info=concept_ref_to_class_info)- class_code = generator._generate_class_source_code_from_blueprint("SkillMatchResult", structure_blueprint=structure_blueprint) # noqa: SLF001 # pyright: ignore[reportPrivateUsage]-- expected_class_code = '''\-class SkillMatchResult(StructuredContent):- """Generated SkillMatchResult class"""-- skill: Skill = Field(..., description="The skill being evaluated")- matches_requirement: bool = Field(..., description="Whether this skill matches a job requirement")'''-- expected_imports = {- "from enum import Enum",- "from pipelex.core.stuffs.structured_content import StructuredContent",- "from pipeline_01.structures.cv_tech_screening__skill import Skill",- "from pydantic import Field",- "from typing import Optional, List, Dict, Any, Literal",- }-- assert class_code == expected_class_code- assert generator.imports == expected_imports-- def test_list_of_concepts_with_module_path_generates_import(self):- """Test that list of concepts with module_path generates proper import statements."""- structure_blueprint = {- "skill_matches": ConceptStructureBlueprint(- description="List of skill match results",- type=ConceptStructureBlueprintFieldType.LIST,- item_type="concept",- item_concept_ref="cv_tech_screening.SkillMatchResult",- required=True,- ),- "overall_score": ConceptStructureBlueprint(- description="Overall match score",- type=ConceptStructureBlueprintFieldType.INTEGER,- required=True,- ),- }-- concept_ref_to_class_info = {- "cv_tech_screening.SkillMatchResult": ConceptClassInfo(- class_name="SkillMatchResult",- module_path="pipeline_01.structures.cv_tech_screening__skill_match_result",- ),- }-- generator = StructureGenerator(concept_ref_to_class_info=concept_ref_to_class_info)- class_code = generator._generate_class_source_code_from_blueprint("TechAnalysis", structure_blueprint=structure_blueprint) # noqa: SLF001 # pyright: ignore[reportPrivateUsage]-- expected_class_code = '''\-class TechAnalysis(StructuredContent):- """Generated TechAnalysis class"""-- skill_matches: List[SkillMatchResult] = Field(..., description="List of skill match results")- overall_score: int = Field(..., description="Overall match score")'''-- expected_imports = {- "from enum import Enum",- "from pipelex.core.stuffs.structured_content import StructuredContent",- "from pipeline_01.structures.cv_tech_screening__skill_match_result import SkillMatchResult",- "from pydantic import Field",- "from typing import Optional, List, Dict, Any, Literal",- }-- assert class_code == expected_class_code- assert generator.imports == expected_imports-- def test_class_info_without_module_path_uses_forward_reference(self):- """Test that ConceptClassInfo without module_path uses forward reference."""- structure_blueprint = {- "item": ConceptStructureBlueprint(- description="An item reference",- type=ConceptStructureBlueprintFieldType.CONCEPT,- concept_ref="myapp.SomeItem",- required=True,- ),- }-- concept_ref_to_class_info = {- "myapp.SomeItem": ConceptClassInfo(- class_name="SomeItem",- module_path=None,- ),- }-- generator = StructureGenerator(concept_ref_to_class_info=concept_ref_to_class_info)- generated_code, generated_class = generator.generate_from_structure_blueprint(class_name="Container", structure_blueprint=structure_blueprint)-- expected_code = (- AUTOGEN_HEADER- + STANDARD_IMPORTS- + '''--class Container(StructuredContent):- """Generated Container class"""-- item: "SomeItem" = Field(..., description="An item reference")-'''- )- assert generated_code == expected_code- assert issubclass(generated_class, StructuredContent)-- def test_multiple_concept_refs_with_same_class_name(self):- """Test generation with multiple concept refs that have the same structure class name."""- structure_blueprint = {- "billing_customer": ConceptStructureBlueprint(- description="The billing customer",- type=ConceptStructureBlueprintFieldType.CONCEPT,- concept_ref="billing.Customer",- required=True,- ),- "shipping_customer": ConceptStructureBlueprint(- description="The shipping customer",- type=ConceptStructureBlueprintFieldType.CONCEPT,- concept_ref="shipping.Customer",- required=False,- ),- }-- concept_ref_to_class_info = {- "billing.Customer": ConceptClassInfo(- class_name="BillingCustomer",- module_path="billing.structures.customer",- ),- "shipping.Customer": ConceptClassInfo(- class_name="ShippingCustomer",- module_path="shipping.structures.customer",- ),- }-- generator = StructureGenerator(concept_ref_to_class_info=concept_ref_to_class_info)- class_code = generator._generate_class_source_code_from_blueprint("Order", structure_blueprint=structure_blueprint) # noqa: SLF001 # pyright: ignore[reportPrivateUsage]-- expected_class_code = '''\-class Order(StructuredContent):- """Generated Order class"""-- billing_customer: BillingCustomer = Field(..., description="The billing customer")- shipping_customer: Optional[ShippingCustomer] = Field(default=None, description="The shipping customer")'''-- expected_imports = {- "from enum import Enum",- "from pipelex.core.stuffs.structured_content import StructuredContent",- "from billing.structures.customer import BillingCustomer",- "from shipping.structures.customer import ShippingCustomer",- "from pydantic import Field",- "from typing import Optional, List, Dict, Any, Literal",- }-- assert class_code == expected_class_code- assert generator.imports == expected_importsdiff --git a/tests/unit/pipelex/core/concepts/test_concept.py b/tests/unit/pipelex/core/concepts/test_concept.pyindex 7cf2e01a6..e78fc1884 100644--- a/tests/unit/pipelex/core/concepts/test_concept.py+++ b/tests/unit/pipelex/core/concepts/test_concept.py@@ -1,8 +1,7 @@import pytestfrom kajson.kajson_manager import KajsonManager-from pydantic import ValidationError+from pydantic import Field, ValidationError-from pipelex.cogt.image.image_size import ImageSizefrom pipelex.core.concepts.concept import Conceptfrom pipelex.core.concepts.concept_blueprint import ConceptBlueprintfrom pipelex.core.concepts.concept_factory import ConceptFactory@@ -25,7 +24,8 @@ class NestedImageInfo(StructuredContent):source_negative_prompt: str | None = Nonecaption: str | None = Nonemime_type: str | None = None- size: ImageSize | None = None+ width: int | None = Field(default=None, gt=0)+ height: int | None = Field(default=None, gt=0)filename: str | None = Nonediff --git a/tests/unit/pipelex/core/concepts/test_concept_representation_generator.py b/tests/unit/pipelex/core/concepts/test_concept_representation_generator.pyindex bc3ab54d9..fa2cfc100 100644--- a/tests/unit/pipelex/core/concepts/test_concept_representation_generator.py+++ b/tests/unit/pipelex/core/concepts/test_concept_representation_generator.py@@ -454,7 +454,7 @@ class TestIncludeOptionalParameter:result = generator.generate_class_representation(ContentWithRequiredAndOptional, include_optional=False)assert isinstance(result, dict)assert "url" in result- assert result["url"].startswith("https://mock-")+ assert result["url"] == "https://mock.invalid/url"# Verify optional fields are NOT presentassert "source_prompt" not in resultassert "caption" not in result@@ -465,7 +465,7 @@ class TestIncludeOptionalParameter:generator = ConceptRepresentationGenerator(ConceptRepresentationFormat.PYTHON)result = generator.generate_class_representation(ContentWithRequiredAndOptional, include_optional=False)assert isinstance(result, str)- assert result.startswith('ContentWithRequiredAndOptional(url="https://mock-')+ assert result.startswith('ContentWithRequiredAndOptional(url="https://mock.invalid/url"')assert result.endswith('")')def test_includes_optional_fields_by_default(self) -> None:@@ -500,7 +500,7 @@ class TestIncludeOptionalParameter:result = generator.generate_class_representation(ImageContent, include_optional=False)assert isinstance(result, dict)assert "url" in result- assert result["url"].startswith("https://mock-")+ assert result["url"] == "https://mock.invalid/url"# Verify optional fields are NOT presentassert "source_prompt" not in resultassert "caption" not in resultdiff --git a/tests/unit/pipelex/core/memory/input_shaper/data.py b/tests/unit/pipelex/core/memory/input_shaper/data.pyindex e45bc794b..e5f4250e0 100644--- a/tests/unit/pipelex/core/memory/input_shaper/data.py+++ b/tests/unit/pipelex/core/memory/input_shaper/data.py@@ -16,6 +16,7 @@ from pipelex.core.stuffs.image_content import ImageContentfrom pipelex.core.stuffs.number_content import NumberContentfrom pipelex.core.stuffs.structured_content import StructuredContentfrom pipelex.core.stuffs.text_content import TextContent+from pipelex.core.stuffs.time_content import TimeContentfrom pipelex.core.stuffs.yes_no_content import YesNoContentSHAPER_TEST_DOMAIN = "shaper_test"@@ -37,6 +38,10 @@ class Deadline(DateContent):"""Refines native Date."""+class OpeningTime(TimeContent):+ """Refines native Time."""++class Photo(ImageContent):"""Refines native Image."""@@ -68,7 +73,7 @@ class ShaperWeird(StructuredContent):# Non-StructuredContent refinements — registered explicitly (the refinement machinery would register# a subclass like these for a `refines = "<native>"` concept).-REFINING_CLASSES = [Question, Priority, Verdict, Deadline, Photo, Exhibit]+REFINING_CLASSES = [Question, Priority, Verdict, Deadline, OpeningTime, Photo, Exhibit]# (concept_code, structure_class_name, refines) for every test concept.CONCEPT_DEFS: list[tuple[str, str, str | None]] = [@@ -76,6 +81,7 @@ CONCEPT_DEFS: list[tuple[str, str, str | None]] = [("Priority", "Priority", "native.Number"),("Verdict", "Verdict", "native.YesNo"),("Deadline", "Deadline", "native.Date"),+ ("OpeningTime", "OpeningTime", "native.Time"),("Photo", "Photo", "native.Image"),("Exhibit", "Exhibit", "native.Document"),("ShaperInvoice", "ShaperInvoice", None),diff --git a/tests/unit/pipelex/core/memory/input_shaper/test_explicit_forms.py b/tests/unit/pipelex/core/memory/input_shaper/test_explicit_forms.pyindex b5f4de506..c1eedaf73 100644--- a/tests/unit/pipelex/core/memory/input_shaper/test_explicit_forms.py+++ b/tests/unit/pipelex/core/memory/input_shaper/test_explicit_forms.py@@ -1,3 +1,5 @@+import datetime+import pytestfrom pipelex import log, pretty_print@@ -9,7 +11,8 @@ from pipelex.core.memory.exceptions import (from pipelex.core.memory.input_shaper import InputShaperfrom pipelex.core.stuffs.list_content import ListContentfrom pipelex.core.stuffs.text_content import TextContent-from tests.unit.pipelex.core.memory.input_shaper.data import Question, ShaperInvoice, ShaperWeird, build_input_specs+from pipelex.core.stuffs.time_content import TimeContent+from tests.unit.pipelex.core.memory.input_shaper.data import OpeningTime, Question, ShaperInvoice, ShaperWeird, build_input_specsclass TestInputShaperExplicitForms:@@ -38,6 +41,29 @@ class TestInputShaperExplicitForms:assert stuff.concept.concept_ref == "shaper_test.Question"assert stuff.content == TextContent(text="What are the fees?")+ @pytest.mark.parametrize(+ ("concept_ref", "expected_type"),+ [("native.Time", TimeContent), ("shaper_test.OpeningTime", OpeningTime)],+ )+ def test_time_envelope_accepts_iso_string(self, concept_ref: str, expected_type: type[TimeContent]) -> None:+ """Explicit envelopes use the shared temporal factory for native Time and refinements."""+ input_specs = build_input_specs([("opening", "native.Time", None)])+ provided = {"concept": concept_ref, "content": "15:40:00+02:00"}++ working_memory = InputShaper.shape({"opening": provided}, input_specs=input_specs)++ stuff = working_memory.root["opening"]+ assert stuff.concept.concept_ref == concept_ref+ assert stuff.content == expected_type(time=datetime.time(15, 40, tzinfo=datetime.timezone(datetime.timedelta(hours=2))))++ def test_time_envelope_accepts_time_object(self) -> None:+ input_specs = build_input_specs([("opening", "native.Time", None)])+ provided = {"concept": "native.Time", "content": datetime.time(15, 40)}++ working_memory = InputShaper.shape({"opening": provided}, input_specs=input_specs)++ assert working_memory.root["opening"].content == TimeContent(time=datetime.time(15, 40))+def test_prebuilt_stuff_content_object(self) -> None:"""D6: a directly-provided StuffContent keeps today's behavior, plus the compat check."""input_specs = build_input_specs([("invoice", "shaper_test.ShaperInvoice", None)])diff --git a/tests/unit/pipelex/core/stuffs/image_content/test_data.py b/tests/unit/pipelex/core/stuffs/image_content/test_data.pyindex b5905baa0..8e531c6dd 100644--- a/tests/unit/pipelex/core/stuffs/image_content/test_data.py+++ b/tests/unit/pipelex/core/stuffs/image_content/test_data.py@@ -19,7 +19,8 @@ class TestData:"source_negative_prompt": None,"caption": None,"mime_type": None,- "size": None,+ "width": None,+ "height": None,"filename": None,}@@ -31,7 +32,8 @@ class TestData:"source_negative_prompt": None,"caption": "A beautiful sunset","mime_type": "image/png",- "size": None,+ "width": None,+ "height": None,"filename": None,}@@ -47,7 +49,8 @@ class TestData:"source_negative_prompt": null,"caption": null,"mime_type": null,- "size": null,+ "width": null,+ "height": null,"filename": null}}"""EXPECTED_RENDERED_JSON_WITH_PROMPT = f"""{{@@ -57,7 +60,8 @@ class TestData:"source_negative_prompt": null,"caption": null,"mime_type": null,- "size": null,+ "width": null,+ "height": null,"filename": null}}"""# rendered_for_prompt returns just the URL for imagesdiff --git a/tests/unit/pipelex/core/stuffs/image_content/test_image_content_dimensions.py b/tests/unit/pipelex/core/stuffs/image_content/test_image_content_dimensions.pynew file mode 100644index 000000000..5ae7a522f--- /dev/null+++ b/tests/unit/pipelex/core/stuffs/image_content/test_image_content_dimensions.py@@ -0,0 +1,32 @@+import pytest+from pydantic import ValidationError++from pipelex.core.stuffs.image_content import ImageContent+from tests.unit.pipelex.core.stuffs.image_content.test_data import TestData+++class TestImageContentDimensions:+ @pytest.mark.parametrize(+ ("width", "height"),+ [+ pytest.param(640, None, id="width-only"),+ pytest.param(None, 480, id="height-only"),+ pytest.param(0, 480, id="zero-width"),+ pytest.param(640, -1, id="negative-height"),+ ],+ )+ def test_rejects_unpaired_or_non_positive_dimensions(self, width: int | None, height: int | None) -> None:+ with pytest.raises(ValidationError):+ ImageContent(url=TestData.SAMPLE_URL, width=width, height=height)++ @pytest.mark.parametrize(+ ("width", "height"),+ [+ pytest.param(None, None, id="dimensions-unknown"),+ pytest.param(640, 480, id="dimensions-known"),+ ],+ )+ def test_accepts_both_dimensions_or_neither(self, width: int | None, height: int | None) -> None:+ image = ImageContent(url=TestData.SAMPLE_URL, width=width, height=height)+ assert image.width == width+ assert image.height == heightdiff --git a/tests/unit/pipelex/core/stuffs/page_content/test_data.py b/tests/unit/pipelex/core/stuffs/page_content/test_data.pyindex 851d969b0..5a61b5f72 100644--- a/tests/unit/pipelex/core/stuffs/page_content/test_data.py+++ b/tests/unit/pipelex/core/stuffs/page_content/test_data.py@@ -38,7 +38,8 @@ class TestData:"source_negative_prompt": None,"caption": None,"mime_type": None,- "size": None,+ "width": None,+ "height": None,"filename": None,},}diff --git a/tests/unit/pipelex/core/stuffs/text_and_images_content/test_data.py b/tests/unit/pipelex/core/stuffs/text_and_images_content/test_data.pyindex 6a04c8e5f..5eb8bd230 100644--- a/tests/unit/pipelex/core/stuffs/text_and_images_content/test_data.py+++ b/tests/unit/pipelex/core/stuffs/text_and_images_content/test_data.py@@ -31,7 +31,8 @@ class TestData:"source_negative_prompt": None,"caption": None,"mime_type": None,- "size": None,+ "width": None,+ "height": None,"filename": None,},{@@ -41,7 +42,8 @@ class TestData:"source_negative_prompt": None,"caption": None,"mime_type": None,- "size": None,+ "width": None,+ "height": None,"filename": None,},],diff --git a/tests/unit/pipelex/core/stuffs/time_content/test_time_content.py b/tests/unit/pipelex/core/stuffs/time_content/test_time_content.pynew file mode 100644index 000000000..031a91019--- /dev/null+++ b/tests/unit/pipelex/core/stuffs/time_content/test_time_content.py@@ -0,0 +1,38 @@+import datetime++import pytest+from pydantic import ValidationError++from pipelex.core.stuffs.time_content import TimeContent+++class TestTimeContent:+ """TimeContent fidelity rules: ISO time of day, offset kept when stated, never a number."""++ def test_iso_string_round_trip(self):+ content = TimeContent(time="15:40:00") # pyright: ignore[reportArgumentType] — pydantic lax ISO parsing is the point+ assert content.time == datetime.time(15, 40)+ assert content.rendered_plain() == "15:40:00"+ assert TimeContent.model_validate(content.model_dump(mode="json")) == content++ def test_offset_is_preserved(self):+ content = TimeContent(time="15:40:00+02:00") # pyright: ignore[reportArgumentType]+ assert content.time.utcoffset() == datetime.timedelta(hours=2)+ assert content.rendered_plain() == "15:40:00+02:00"+ assert TimeContent.model_validate(content.model_dump(mode="json")) == content++ def test_number_is_rejected(self):+ """A bare number must never be read as seconds-since-midnight."""+ with pytest.raises(ValidationError):+ TimeContent.model_validate({"time": 56400})++ @pytest.mark.parametrize("value", ["56400", "+56400", "56400.0", "5.64e4"])+ def test_numeric_string_is_rejected(self, value: str):+ """Pydantic must not reinterpret number-shaped strings as seconds-since-midnight."""+ with pytest.raises(ValidationError):+ TimeContent.model_validate({"time": value})++ def test_datetime_is_rejected(self):+ """A datetime carries a date — that belongs to Date, not Time."""+ with pytest.raises(ValidationError):+ TimeContent.model_validate({"time": datetime.datetime(2026, 7, 10, 15, 40)})diff --git a/tests/unit/pipelex/libraries/test_crate_normalization.py b/tests/unit/pipelex/libraries/test_crate_normalization.pynew file mode 100644index 000000000..677f786d8--- /dev/null+++ b/tests/unit/pipelex/libraries/test_crate_normalization.py@@ -0,0 +1,249 @@+import pytest+from pytest_mock import MockerFixture++from pipelex.core.concepts.concept_blueprint import ConceptBlueprint+from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprint, ConceptStructureBlueprintFieldType+from pipelex.core.concepts.native.concept_native import NativeConceptCode+from pipelex.core.domains.domain_blueprint import DomainBlueprint+from pipelex.libraries.crate_normalization import normalize_crate+from pipelex.libraries.exceptions import CrateNormalizationError+from pipelex.libraries.library_crate import LibraryCrate+from pipelex.pipe_controllers.sequence.pipe_sequence_blueprint import PipeSequenceBlueprint+from pipelex.pipe_controllers.sub_pipe_blueprint import SubPipeBlueprint+from pipelex.pipe_operators.llm.pipe_llm_blueprint import PipeLLMBlueprint++MTHDS_TEST_VERSION = "0.1.0-test"+++def _structure_field(concept: ConceptBlueprint | str, field_name: str) -> ConceptStructureBlueprint:+ """Narrow a concept's structure field to a `ConceptStructureBlueprint` for typed assertions."""+ assert isinstance(concept, ConceptBlueprint)+ assert isinstance(concept.structure, dict)+ field = concept.structure[field_name]+ assert isinstance(field, ConceptStructureBlueprint)+ return field+++def _authored_crate() -> LibraryCrate:+ """A merged, key-qualified single-package crate exercising every normalization step.++ All keys are domain-qualified (`scoring.*`) as `LibraryCrateFactory` would leave them; in-body+ references are authored bare (same-domain codes and native codes) so the normalizer has work to do.+ """+ return LibraryCrate(+ concepts={+ # String-described concept -> promoted to a description-only ConceptBlueprint.+ "scoring.Category": "A category label",+ # Structure with a bare same-domain concept_ref and a bare native concept_ref.+ "scoring.WeightedScore": ConceptBlueprint(+ description="A weighted score",+ structure={+ "value": ConceptStructureBlueprint(description="the numeric value", type=ConceptStructureBlueprintFieldType.NUMBER),+ "label": ConceptStructureBlueprint(+ description="the category", type=ConceptStructureBlueprintFieldType.CONCEPT, concept_ref="Category"+ ),+ "note": ConceptStructureBlueprint(+ description="a free-text note", type=ConceptStructureBlueprintFieldType.CONCEPT, concept_ref="Text"+ ),+ },+ ),+ # Refines a same-package concept that HAS a structure -> flattened (refines dropped).+ "scoring.DetailedScore": ConceptBlueprint(description="a detailed score", refines="WeightedScore"),+ # Refines a STRUCTURELESS native (Date, whose content class has no mappable structure) ->+ # refines kept, no structure materialized.+ "scoring.CustomDate": ConceptBlueprint(description="a domain-specific date", refines="Date"),+ # Refines a STRUCTURED native (Text, whose materialized structure has a `text` field) ->+ # refines kept (B1-2): flattening would drop the native base on round-trip.+ "scoring.Summary": ConceptBlueprint(description="a short summary", refines="Text"),+ # Multi-hop: refines a concept that itself bottoms out at a native -> refines kept.+ "scoring.LongSummary": ConceptBlueprint(description="a longer summary", refines="Summary"),+ },+ pipes={+ "scoring.compute_score": PipeLLMBlueprint(+ description="Compute a weighted score",+ inputs={"data": "Text"},+ output="WeightedScore",+ prompt="Compute from $data",+ ),+ "scoring.list_scores": PipeLLMBlueprint(+ description="Compute several weighted scores",+ inputs={"docs": "Text[]"},+ output="WeightedScore[]",+ prompt="List from $docs",+ ),+ "scoring.pipeline": PipeSequenceBlueprint(+ description="Run the scoring pipeline",+ output="WeightedScore",+ steps=[SubPipeBlueprint(pipe="compute_score")],+ ),+ },+ domains={"scoring": DomainBlueprint(code="scoring", description="Scoring domain")},+ source_map={"scoring.WeightedScore": "/fake/scoring.mthds"},+ )+++def _reverse_refinement_chain_crate(size: int) -> LibraryCrate:+ concepts: dict[str, ConceptBlueprint | str] = {+ f"scale.Node{index:04d}": ConceptBlueprint(description=f"Node {index}", refines=f"Node{index + 1:04d}") for index in range(size - 1)+ }+ concepts[f"scale.Node{size - 1:04d}"] = ConceptBlueprint(+ description="Structured base",+ structure={+ "value": ConceptStructureBlueprint(description="Value", type=ConceptStructureBlueprintFieldType.TEXT),+ },+ )+ return LibraryCrate(concepts=concepts)+++class TestCrateNormalization:+ """Unit tests for `normalize_crate` over a hand-built, key-qualified crate."""++ def test_string_concept_promoted_to_blueprint(self):+ """A string-described concept becomes an explicit description-only ConceptBlueprint."""+ result = normalize_crate(_authored_crate(), mthds_version=MTHDS_TEST_VERSION)+ category = result.concepts["scoring.Category"]+ assert isinstance(category, ConceptBlueprint)+ assert category.description == "A category label"+ assert category.structure is None++ def test_in_body_concept_refs_fully_qualified(self):+ """Bare same-domain and bare native concept refs in a structure become fully qualified."""+ result = normalize_crate(_authored_crate(), mthds_version=MTHDS_TEST_VERSION)+ weighted = result.concepts["scoring.WeightedScore"]+ assert _structure_field(weighted, "label").concept_ref == "scoring.Category"+ assert _structure_field(weighted, "note").concept_ref == "native.Text"++ def test_pipe_io_refs_qualified_with_multiplicity_preserved(self):+ """Pipe input/output concept refs qualify; multiplicity markers survive the rewrite."""+ result = normalize_crate(_authored_crate(), mthds_version=MTHDS_TEST_VERSION)+ compute = result.pipes["scoring.compute_score"]+ assert compute.inputs == {"data": "native.Text"}+ assert compute.output == "scoring.WeightedScore"+ listed = result.pipes["scoring.list_scores"]+ assert listed.inputs == {"docs": "native.Text[]"}+ assert listed.output == "scoring.WeightedScore[]"++ def test_sequence_step_pipe_refs_qualified(self):+ """A sequence step's bare pipe ref is qualified with the pipe's owner domain."""+ result = normalize_crate(_authored_crate(), mthds_version=MTHDS_TEST_VERSION)+ pipeline = result.pipes["scoring.pipeline"]+ assert isinstance(pipeline, PipeSequenceBlueprint)+ assert pipeline.steps[0].pipe == "scoring.compute_score"++ def test_refinement_with_structured_base_is_flattened(self):+ """Refining a concept with a structure adopts that (qualified) structure and drops `refines`."""+ result = normalize_crate(_authored_crate(), mthds_version=MTHDS_TEST_VERSION)+ detailed = result.concepts["scoring.DetailedScore"]+ assert isinstance(detailed, ConceptBlueprint)+ assert detailed.refines is None+ assert isinstance(detailed.structure, dict)+ assert set(detailed.structure.keys()) == {"value", "label", "note"}+ # The adopted structure carries the base's already-qualified refs.+ assert _structure_field(detailed, "label").concept_ref == "scoring.Category"+ assert _structure_field(detailed, "note").concept_ref == "native.Text"++ def test_refinement_with_structureless_native_base_keeps_refines(self):+ """Refining a structureless native (Date) keeps a qualified `refines` and stays structureless."""+ result = normalize_crate(_authored_crate(), mthds_version=MTHDS_TEST_VERSION)+ custom_date = result.concepts["scoring.CustomDate"]+ assert isinstance(custom_date, ConceptBlueprint)+ assert custom_date.refines == "native.Date"+ assert custom_date.structure is None++ def test_refinement_with_structured_native_base_keeps_refines(self):+ """Refining a STRUCTURED native (Text) keeps the qualified `refines` and is NOT flattened (B1-2).++ The old behavior inlined native.Text's `text` field and dropped `refines`, which loses the+ native content class on round-trip. The native is materialized separately (step 4), so keeping+ the link is both sufficient and identity-preserving.+ """+ result = normalize_crate(_authored_crate(), mthds_version=MTHDS_TEST_VERSION)+ summary = result.concepts["scoring.Summary"]+ assert isinstance(summary, ConceptBlueprint)+ assert summary.refines == "native.Text"+ assert summary.structure is None+ # The native base it points at is materialized in the crate (so the link resolves).+ assert "native.Text" in result.concepts++ def test_multi_hop_native_backed_chain_keeps_refines(self):+ """A concept whose refinement chain reaches a native only through an intermediate keeps `refines`."""+ result = normalize_crate(_authored_crate(), mthds_version=MTHDS_TEST_VERSION)+ long_summary = result.concepts["scoring.LongSummary"]+ assert isinstance(long_summary, ConceptBlueprint)+ # Not flattened: the chain LongSummary -> Summary -> native.Text bottoms at a native.+ assert long_summary.refines == "scoring.Summary"+ assert long_summary.structure is None++ def test_referenced_natives_are_materialized(self):+ """Every referenced native is materialized as a `native.<Code>` concept entry."""+ result = normalize_crate(_authored_crate(), mthds_version=MTHDS_TEST_VERSION)+ # native.Text is referenced by a structure field (and pipe io); it materializes WITH a structure.+ text_native = result.concepts["native.Text"]+ assert isinstance(text_native, ConceptBlueprint)+ assert isinstance(text_native.structure, dict)+ assert "text" in text_native.structure+ # native.Date is referenced via a `refines`; it materializes from the pinned definitions+ # with its declared structure (a required `date` plus an optional `time` field).+ date_native = result.concepts["native.Date"]+ assert isinstance(date_native, ConceptBlueprint)+ assert isinstance(date_native.structure, dict)+ assert set(date_native.structure.keys()) == {"date", "time"}++ def test_unreferenced_natives_absent(self):+ """Natives that nothing references are not materialized into the crate."""+ result = normalize_crate(_authored_crate(), mthds_version=MTHDS_TEST_VERSION)+ assert "native.SearchResult" not in result.concepts+ assert "native.Page" not in result.concepts++ def test_fingerprint_recomputed_and_version_stamped(self):+ """The normalized crate carries the D2-scope fingerprint and the mthds_version stamp."""+ result = normalize_crate(_authored_crate(), mthds_version=MTHDS_TEST_VERSION)+ assert result.mthds_version == MTHDS_TEST_VERSION+ assert result.fingerprint != ""+ assert result.fingerprint == result.compute_normalized()++ def test_normalization_is_idempotent(self):+ """Normalizing an already-normalized crate is a fixed point (content and fingerprint)."""+ once = normalize_crate(_authored_crate(), mthds_version=MTHDS_TEST_VERSION)+ twice = normalize_crate(once, mthds_version=MTHDS_TEST_VERSION)+ assert twice.fingerprint == once.fingerprint+ assert twice.concepts == once.concepts+ assert twice.pipes == once.pipes+ assert twice.domains == once.domains++ def test_deep_reverse_refinement_chain_flattens_without_recursion(self):+ result = normalize_crate(_reverse_refinement_chain_crate(1_500), mthds_version=MTHDS_TEST_VERSION)++ first = result.concepts["scale.Node0000"]+ assert isinstance(first, ConceptBlueprint)+ assert first.refines is None+ assert isinstance(first.structure, dict)+ assert set(first.structure) == {"value"}++ def test_refinement_resolution_work_is_linear_in_chain_length(self, mocker: MockerFixture):+ chain_size = 400+ native_check = mocker.spy(NativeConceptCode, "is_native_concept_ref_or_code")++ normalize_crate(_reverse_refinement_chain_crate(chain_size), mthds_version=MTHDS_TEST_VERSION)++ assert native_check.call_count < chain_size * 10++ def test_refinement_cycle_has_explicit_diagnostic(self):+ crate = LibraryCrate(+ concepts={+ "cycle.A": ConceptBlueprint(description="A", refines="B"),+ "cycle.B": ConceptBlueprint(description="B", refines="C"),+ "cycle.C": ConceptBlueprint(description="C", refines="A"),+ },+ )++ with pytest.raises(CrateNormalizationError) as exc_info:+ normalize_crate(crate, mthds_version=MTHDS_TEST_VERSION)++ assert str(exc_info.value) == "Refinement cycle detected: cycle.A -> cycle.B -> cycle.C -> cycle.A"++ def test_non_domain_qualified_key_raises(self):+ """A crate key that is not domain-qualified is a contract violation, surfaced explicitly."""+ crate = LibraryCrate(concepts={"BareConcept": "a concept with no domain in its key"})+ with pytest.raises(CrateNormalizationError, match="not domain-qualified"):+ normalize_crate(crate, mthds_version=MTHDS_TEST_VERSION)diff --git a/tests/unit/pipelex/libraries/test_library_crate.py b/tests/unit/pipelex/libraries/test_library_crate.pyindex a4a1d2e40..50325d424 100644--- a/tests/unit/pipelex/libraries/test_library_crate.py+++ b/tests/unit/pipelex/libraries/test_library_crate.py@@ -39,6 +39,27 @@ class TestLibraryCrate:assert "scoring" in crate.domainsassert crate.domains["scoring"].description == "Scoring domain"+ @pytest.mark.parametrize(+ "blueprints",+ [+ [BlueprintSamples.META_MEMBER_BUNDLE, BlueprintSamples.SCORING_BUNDLE],+ [BlueprintSamples.SCORING_BUNDLE, BlueprintSamples.META_MEMBER_BUNDLE],+ ],+ )+ def test_main_pipe_merges_from_same_domain_sibling(self, blueprints: list[PipelexBundleBlueprint]):+ """A membership-only file loaded first must not hide a later same-domain main_pipe."""+ pipeline_bundle = PipelexBundleBlueprint(+ source="/fake/meta_pipeline.mthds",+ domain="meta",+ main_pipe="compute_score",+ pipe=BlueprintSamples.SCORING_BUNDLE.pipe,+ )+ ordered = [pipeline_bundle if blueprint is BlueprintSamples.SCORING_BUNDLE else blueprint for blueprint in blueprints]++ crate = LibraryCrateFactory.make_from_blueprints(blueprints=ordered)++ assert crate.domains["meta"].main_pipe == "compute_score"+def test_merge_across_domains(self):"""Bundles with different domains produce refs qualified with respective domains."""crate = LibraryCrateFactory.make_from_blueprints(diff --git a/tests/unit/pipelex/pipeline/test_validate_bundle_helper.py b/tests/unit/pipelex/pipeline/test_validate_bundle_helper.pyindex 02f529ea9..5a8c5d52e 100644--- a/tests/unit/pipelex/pipeline/test_validate_bundle_helper.py+++ b/tests/unit/pipelex/pipeline/test_validate_bundle_helper.py@@ -1,44 +1,21 @@-"""Pin properties of ``_translate_to_validate_bundle_error``'s runtime contract.--- ``PipeValidationError`` and ``pydantic.ValidationError`` are siblings, not- parent-child — the helper's ``except`` cascade orders- ``except PipeValidationError`` BEFORE ``except ValidationError`` and relies- on ``PipeValidationError(ValueError)`` NOT being a subclass of- ``pydantic.ValidationError``. A future refactor that unifies the hierarchy- would silently route pydantic errors into the wrong arm.-- The ``match category:`` in the ``except ValidationError`` arm is exhaustive- over ``Literal["pipe", "concept"]`` and now ends with- ``case _ as unreachable: assert_never(unreachable)``. A bad runtime value- (passed via ``# type: ignore`` or dynamic dispatch) must raise a loud- ``AssertionError`` rather than fall through to an ``UnboundLocalError`` on- ``msg``.+"""Pin properties of ``translate_to_validate_bundle_error``'s runtime contract.++``PipeValidationError`` and ``pydantic.ValidationError`` are siblings, not+parent-child — the helper's ``except`` cascade orders+``except PipeValidationError`` BEFORE ``except ValidationError`` and relies+on ``PipeValidationError(ValueError)`` NOT being a subclass of+``pydantic.ValidationError``. A future refactor that unifies the hierarchy+would silently route pydantic errors into the wrong arm."""-from typing import cast--import pytest-from pydantic import BaseModel, ValidationError+from pydantic import ValidationErrorfrom pipelex.core.pipes.exceptions import PipeValidationError-from pipelex.pipeline.validate_bundle import (- _translate_to_validate_bundle_error, # noqa: PLC2701 # pyright: ignore[reportPrivateUsage]-)---class _ForceValidationError(BaseModel):- """Pydantic model whose ``model_validate({})`` raises ``ValidationError``.-- Used to drive the helper's ``except ValidationError`` arm without- constructing a ``pydantic.ValidationError`` directly (the class forbids- direct instantiation outside pydantic internals).- """-- required_field: intclass TestTranslateToValidateBundleErrorContract:def test_pipe_validation_error_is_not_a_pydantic_validation_error_subclass(self) -> None:- """Cascade ordering in ``_translate_to_validate_bundle_error`` depends on this.+ """Cascade ordering in ``translate_to_validate_bundle_error`` depends on this.If a future refactor makes ``PipeValidationError`` inherit from``pydantic.ValidationError``, the helper's ``except PipeValidationError``@@ -47,24 +24,3 @@ class TestTranslateToValidateBundleErrorContract:``categorize_pipe_validation_error``) would become dead code."""assert not issubclass(PipeValidationError, ValidationError)-- def test_unknown_category_raises_assertion_error_at_runtime(self) -> None:- """An unexpected ``category`` value reaches ``assert_never`` and raises.-- The ``category`` parameter is ``Literal["pipe", "concept"]`` — pyright- catches a bad call site statically. The runtime guard exists because- ``# type: ignore`` or dynamic dispatch can still slip an unexpected- value past the static check. The helper must then raise a loud- ``AssertionError`` rather than fall through to an ``UnboundLocalError``- on the unbound ``msg`` variable — the latter would silently break the- helper's translation contract.- """- # ``cast("str", ...)`` strips the Literal narrowing so the runtime value- # reaches the helper as a plain ``str`` — mirrors a real-world bypass- # via ``# type: ignore`` or dynamic dispatch.- bad_category = cast("str", "concepts")- with (- pytest.raises(AssertionError),- _translate_to_validate_bundle_error(category=bad_category), # type: ignore[arg-type] # pyright: ignore[reportArgumentType]- ):- _ForceValidationError.model_validate({})