Skip to content

Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.

fix(codegen): emit lint-clean artifacts so a consumer's formatter cannot break the stamp (#1070)

Louis Choqueld96d1cf662b4
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 3e4b7731d..a40fe0227 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -31,6 +31,26 @@
### Fixed
+- **Breaking: generated codegen artifacts were not lint-clean, so running your own formatter over them broke their stamp and got you accused of hand-editing.** A stamp's `content_hash` is a raw SHA-256 over the body bytes, and the emitters were not writing what a formatter wants — so the first `ruff check --fix` / `ruff format` (or `prettier --write`) over a generated tree rewrote the bytes and `pipelex codegen check` reported the file as `[hand-edited]`, which is the one thing the user did not do. It was silent until someone ran `codegen check`, and our own `pipelex-starter-python` template shipped a `piper/generated` lint exclusion to work around it — the tell that the emitter, not the consumer, was wrong.
+
+ **Emitted bytes change for all three targets, so a committed projection reports drift until it is regenerated** — `pipelex codegen types --target <target> -o <output_dir> <method_dir>`, or `pipelex build structures <dir>` for the python-structures alias. Both Python targets now emit isort-grouped imports (merged per module, stdlib separated from third-party) and double-quoted `Literal` members; `python-structures` additionally moves to builtin generics (`list[str]`, `dict[str, int]`) and `X | None`, dropping its `typing.List` / `Dict` / `Optional` imports. `ts-zod` drops the Python two-blank-line idiom Prettier always collapses, and wraps the binder import only when it exceeds Prettier's print width. Ruff's `TC003` is the one rule that cannot be answered in the bytes — moving `from datetime import date` into an `if TYPE_CHECKING:` block would break the generated models, since pydantic resolves annotations at runtime — so consumers set `runtime-evaluated-base-classes` instead; **the lint exclusion on generated paths is no longer needed and should be dropped.** Documented in [Codegen Projections](under-the-hood/codegen-projections.md). This repo now sets that option too, which retired two hand-written `# noqa: TC001` / `# noqa: TC003` suppressions that were working around the same rule one file at a time.
+
+ Two further emitter bugs surfaced while pinning the property. The imprecision caveat on an opaque concept was emitted as a literal `\n` escape rather than a paragraph break, so the docstring rendered as one line with visible escapes. And a description containing a double quote — `The "primary" thing` — was backslash-escaped into the docstring, which ruff's `D301` then auto-rewrites to an `r` prefix: byte-changing *and* text-corrupting, since the escapes start rendering literally.
+
+ **A concept description is now rendered as an idiomatic docstring, which changes `__doc__`.** Summary on the first line, continuation lines indented to the class body, closing delimiter on its own line — the only shape `ruff format` and the pydocstyle rules both leave alone. Rendered any flatter, a multi-line description trips `D207`/`D209` and a description padded with edge whitespace trips `D210`, every one of them an automatically-applied fix that rewrites the bytes. So `__doc__` now carries the class-body indentation that every hand-written docstring carries; what holds exactly is `inspect.getdoc(cls) == inspect.cleandoc(description)`, the value consumers actually read. Exact bytes survive wherever they are consumed programmatically rather than as prose: `Field(description=...)` and the crate itself are unchanged. A description that cannot sit between `"""` at all now falls back to `'''` rather than backslash-escaping, which is what keeps it `D301`-clean.
+
+ **Long lines are emitted already wrapped, with a trailing comma, because the consumer's `line-length` is theirs and not ours.** A description or choice list is authored text with no length bound, so a generated line can exceed any width — and the formatter would wrap the `Field(...)` call or the `Literal[...]` annotation, rewriting the bytes. Black and ruff both read a trailing comma as a deliberate choice to keep a construct exploded and will not rejoin it at *any* width, so anything past 88 columns — ruff's own default, and therefore the tightest width a consumer is likely to lint at — is emitted in that form. Short lines are untouched, so ordinary artifacts read as before. Linting Python at fewer than 88 columns can still rewrap short generated lines; that limit is documented rather than papered over.
+
+ **Imports are now registered by whoever writes the name, never seeded up front.** Two ordinary crate shapes used fewer imports than the emitters assumed and were left with an unused one — an `F401`, which `ruff check --fix` deletes: a method declaring only structureless concepts (`Question` → `Answer`) emits no `Field(...)` at all, and one whose concepts all refine a native never reaches the runtime root base, leaving `StructuredContent` unused too.
+
+ **`ts-zod` is emitted to match Prettier's defaults**, which now includes pre-wrapping the pieces a long concept code or choice list pushes over the 80-column print width: the `z.infer` type alias, the binder's `parse` / `serialize` signatures, and `z.enum([...])` (in place, or as a broken member chain when a `.optional()` / `.default(...)` is attached). A Prettier config that changes `printWidth`, `singleQuote` or `semi` will still rewrite the output; that assumption is now stated in the docs.
+
+ A projection with **nothing to emit** was the same bug in miniature, and it needed no unusual method to hit: `python-structures` skips natives, so a method that declares no concepts of its own — a `Text -> Text` pipe — left that target with no class to write, and it emitted an import block nothing used (an unused-import finding `ruff check --fix` deletes) above a trailing blank-line run (`ruff format` collapses it). The same shape reached the other three artifacts from a bundle declaring only a domain, where the binder degenerated to `import { } from "./types";`. An empty projection is now its header alone, which no formatter touches.
+
+ Guarded by `tests/unit/pipelex/codegen/test_emitted_artifacts_are_lint_clean.py`, which emits a projection covering every `ResolvedTypeKind` — the fixture is asserted to reach every enum member, so a new kind nobody wires in fails loudly — and runs `ruff check` plus `ruff format --check` over it for each Python target, across a range of consumer `line-length` settings, plus the two crate shapes that use no seeded import. Only rules whose fix ruff never applies automatically are suppressed in the invocation, since only an applied fix can rewrite bytes. The TypeScript side asserts the config-independent invariants always — no collapsible blank runs, no code line past the print width — and runs `prettier --check` when prettier is on PATH, which CI has no toolchain for.
+
+- **The runtime `StructureGenerator` emitted `Optional[X]` / `List` / `Dict` where the codegen emitters emit modern typing.** Its output is `exec()`d rather than written to disk, so no linter ever saw it, but the two paths render the same resolved-type tree and the emitter's own docstring justified its spelling by mirroring it. Both now agree on builtin generics, `X | None`, and double-quoted `Literal` members. One shape needed care: this module has no `from __future__ import annotations`, so a concept reference renders as a *quoted* forward reference, and `"Foo" | None` is a `TypeError` at runtime — `str` does not implement `|`. The union is folded inside the quotes (`"Foo | None"`) so the annotation stays one deferred expression for `model_rebuild`; a bare quoted ref is the only affected shape, as `list["Foo"] | None` is a real `GenericAlias`.
+
- **The hub-layering guard was only one hop deep, and four modules of the declared runtime-layer `pipelex.plugins` package breached the boundary through it.** `plugins/direct/direct_plugin.py`, `plugins/pipe_func/pipe_func_plugin.py` and their two aggregators (`builtins.py`, `discovery.py`) reached `pipelex.interpreter_hub` — and 57–67 interpreter modules — through `runtime_bridge`, `pipeline` and `pipe_operators`, while both `make check-hub-layering` and the import-closure test stayed green. The headline zero-interpreter-modules property was never damaged (nothing on the inference-layer or `runtime_hub` path reached those four), but the declaration claimed something the measurement contradicted. Fixed by relocating the two interpreter-touching plugins (above) rather than by declaring an exception, and the guard now has a **transitive rule**: it resolves the module-level import graph of `pipelex/` and reports any runtime-layer module that *reaches* the interpreter hub, with the shortest import chain and the line of its first hop. The built-in adapters' manifest also joins the import-closure test's entry points, since static analysis cannot see a dynamic import; it ships at `pipelex.providers.builtins`.
- **`hub.set_observer()` was documented as the way to register a custom observer, and never worked.** The value was written to the hub and never read — the live path is `Pipelex.make(observers={...})` → `MultiObserver` → `PipeRouter`, so anyone following that page got a silently ignored observer. The write-only state is deleted and both `docs/advanced/observer-provider-injection.md` and `docs/advanced/index.md` are rewritten to the mechanism that actually works, with a warning against hand-constructing a hub.
- **A duplicate `set_library_manager` pair in `Pipelex.setup` built, registered, then silently discarded a first `LibraryManager()`** when the caller passed none. Removed.
diff --git a/docs/under-the-hood/codegen-projections.md b/docs/under-the-hood/codegen-projections.md
index 982c6b97c..da2e87a5a 100644
--- a/docs/under-the-hood/codegen-projections.md
+++ b/docs/under-the-hood/codegen-projections.md
@@ -87,6 +87,65 @@ The verdict rides the exit code (mirroring `resolve` / `validate`): `0` current,
`pipelex/codegen/emission.py` ties it together in two layers. `build_stamped_projection` is the **pure** core: it stamps each emitted body and assembles the matching `codegen.lock` content — no filesystem access — so it is the single source of truth for what a projection *is* on disk. `write_stamped_projection` rides it to materialize locally: it writes each file only when its content changed (write-if-changed — no mtime churn, clean diffs, watch-mode friendly), removes any previously-tracked stamped file that dropped out of the set, and rewrites the lock. Only files the tool itself stamped are ever removed, so a hand-authored file sharing the output directory is never touched. `codegen inputs` applies the same write-if-changed rule to its single (unstamped) template file, so a full regeneration pass over a committed consumer is a true no-op when everything is current.
+### Lint-clean by construction
+
+A stamp's `content_hash` is a raw SHA-256 over the body bytes. So if the emitted code is not already what your formatter wants, the first `ruff check --fix` / `ruff format` (or `prettier --write`) over your tree rewrites those bytes and `pipelex codegen check` reports the file as **hand-edited** — accusing you of the one thing you did not do.
+
+The emitters therefore emit exactly what the formatter would write. Generated Python uses builtin generics (`list[str]`, `dict[str, int]`), `X | None` over `Optional[X]`, double-quoted `Literal` members, and isort-grouped imports; generated TypeScript matches Prettier's blank-line and import-wrapping behavior. Running your formatter over a generated tree is a no-op, and the stamp stays valid.
+
+**You should not need to exclude generated paths from your linter.** If you carry such an exclusion from an older version, drop it.
+
+Every import is registered by whoever writes the name it imports, never seeded up front. That matters for crate shapes which use fewer of them than you would expect: a method declaring only structureless concepts emits no `Field(...)` at all, and one whose concepts all refine a native never reaches the runtime root base. An import left unused is an `F401` — a fix ruff applies automatically, so the line would simply vanish and take the stamp with it.
+
+A projection with nothing to emit for a target is its **header alone** — no imports, no trailing blank lines. That case is ordinary rather than degenerate: `python-structures` skips native concepts, since they already exist in the runtime, so a method that declares no concepts of its own leaves that target with no class to write. The header is inert under any formatter, whereas an import block with nothing left to use it is an unused-import finding sitting above a collapsible blank-line run — both of which a formatter would rewrite, breaking the stamp.
+
+#### Line width, which is your setting and not ours
+
+A description or choice list is authored text with no length bound, so a generated line can exceed whatever `line-length` you lint at — and your formatter would then wrap the call, rewriting the bytes.
+
+Because we cannot know your width, anything too long is emitted **already wrapped, with a trailing comma**. That comma is load-bearing: Black and ruff both read it as a deliberate choice to keep the construct exploded, and will not rejoin it at any width. Short lines stay on one line, so ordinary artifacts read exactly as before.
+
+The threshold is 88 columns — ruff's own default, chosen because it is the *tightest* width you are likely to use, and only the tightest threshold is safe. If you lint Python at **fewer than 88 columns**, short generated lines can still be rewrapped; either raise `line-length` for the generated path or keep an exclusion there.
+
+`E501` is the one finding that can survive on a generated file, on a line holding a single long string literal — an authored description or choice value. There is no wrapped form: breaking a string would alter the author's text. It has no automatic fix, so it never changes bytes and never breaks a stamp, and it is not in ruff's default rule set, so most consumers never see it.
+
+#### Docstrings
+
+A concept description becomes the generated class's docstring, rendered the way a human writes one: summary on the first line, continuation lines indented to the class body, closing delimiter on its own line. That shape is not cosmetic — it is the only one that `ruff format` and the pydocstyle rules both leave alone. Rendered flat, a multi-line description trips `D207`/`D209` and edge whitespace trips `D210`, all of which ruff fixes automatically, rewriting the bytes.
+
+So `__doc__` carries the class-body indentation, exactly as a hand-written docstring does. What holds exactly is the value you actually read:
+
+```python
+inspect.getdoc(GeneratedClass) == inspect.cleandoc(authored_description)
+```
+
+Exact bytes survive wherever they are consumed programmatically rather than read as prose — `Field(description=...)` keeps the description verbatim, and so does the crate.
+
+#### TypeScript assumes Prettier's defaults
+
+The `ts-zod` target is emitted to match Prettier's **default** configuration: 80-column print width, double quotes, semicolons. A long concept name or choice list is pre-wrapped the way Prettier would wrap it, for the same reason the Python side pre-explodes long calls.
+
+If your Prettier config changes `printWidth`, `singleQuote`, or `semi`, the generated files will not match it, and `prettier --write` will rewrite them. Either run codegen output through your own Prettier before stamping, or keep an exclusion for the generated path.
+
+One ruff setting is required, because it cannot be fixed in the emitted bytes:
+
+```toml
+[tool.ruff.lint.flake8-type-checking]
+runtime-evaluated-base-classes = [
+ "pydantic.BaseModel",
+ "pipelex.core.stuffs.structured_content.StructuredContent",
+]
+```
+
+Without it, `TC003` asks you to move `from datetime import date` into an `if TYPE_CHECKING:` block. Applying that **breaks** the generated models — pydantic resolves annotations at runtime to build validators, so the import must stay at runtime. The setting tells ruff these classes are runtime-evaluated, and the finding goes away.
+
+The remaining findings are artifacts of *how* you lint, not of file content. None of them has an automatically-applied fix, so none can change bytes:
+
+- `INP001` ("implicit namespace package") fires when you point ruff at a bare directory with no `__init__.py`. Add one, or lint the package that contains it.
+- `E501` on a single long string literal, as described above.
+- `D301` ("use `r\"\"\"`") on the rare description that carries both `\"\"\"` and `'''`, or a control character — the only inputs with no verbatim rendering, which therefore have to be escaped. Ruff classes that fix as *unsafe*, so `ruff check --fix` leaves it alone.
+- Import grouping for the `python-structures` target assumes `pipelex` is a third-party dependency — which it is in your tree. The Pipelex repo itself, where `pipelex` is first-party, is the one place ruff wants the opposite order; no generated artifact is committed there, so the consumer's grouping is the one the emitter targets.
+
## Serving the engine over HTTP
The same engine backs the `pipelex-api` routes (`POST /v1/resolve`, `POST /v1/codegen`, and the re-pointed `/v1/build/*` — the route envelopes are documented in `pipelex-api`'s `docs/codegen.md`). Two host-facing cores make that possible without any CLI plumbing:
diff --git a/pipelex/codegen/emitters/python_common.py b/pipelex/codegen/emitters/python_common.py
index 46b3a322f..8a5e18269 100644
--- a/pipelex/codegen/emitters/python_common.py
+++ b/pipelex/codegen/emitters/python_common.py
@@ -1,18 +1,31 @@
"""Rendering helpers shared by the two Python types emitters (structures + plain pydantic).
Only the target-agnostic pieces live here: string / default-value escaping, the generated-file
-header (carrying the extension-file subclassing story), the class docstring, and the `= Field(...)`
-assignment. Each Python emitter keeps its own resolved-type -> annotation walker, because the
-container spelling (`typing.List`/`Optional` for the runtime idiom vs. builtin generics + `| None`
-for plain pydantic) and the concept spelling (runtime content classes vs. generated models) differ.
+header (carrying the extension-file subclassing story), the class docstring, the `= Field(...)`
+assignment, the `Literal[...]` annotation, and the import block. Each Python emitter keeps its own
+resolved-type -> annotation walker, because the concept spelling (runtime content classes vs.
+generated models) differs.
"""
+import sys
+from collections import defaultdict
from datetime import date, datetime, time
+from inspect import cleandoc
from typing import Any, cast
from pipelex.codegen.resolved_concepts import ResolvedConcept
from pipelex.core.concepts.resolved_fields import ResolvedField, iter_imprecision_reasons
+# Ruff's own default `line-length`, and deliberately not this repo's 150. It is the *tightest* width a
+# consumer is likely to lint at, and the threshold has to be the tightest one to be safe: anything we
+# leave flat must fit under every config we claim to support. Past it we emit an already-exploded form
+# with a magic trailing comma, which the formatter pins at any width — so the only shapes still at the
+# mercy of a consumer's setting are flat lines under 88 columns, which a narrower config would rewrap.
+PY_EXPLODE_WIDTH = 88
+
+# One level of the generated class body. Emitted code is always 4-space indented.
+_PY_INDENT = " "
+
def python_header(*, target: str) -> str:
"""The generated-file header: an AUTOGENERATED marker plus the extension-file (subclassing) story."""
@@ -70,24 +83,85 @@ def format_default_value(value: Any) -> str:
return repr(value)
+def normalized_docstring_text(text: str) -> str:
+ """The authored text as it reads back out of `inspect.getdoc` once rendered as a docstring.
+
+ Two normalizations compose, neither of them chosen. Per-line trailing whitespace goes because
+ `ruff format` strips it inside docstrings (and `D210` trims the edges), both as auto-applied fixes
+ that would otherwise rewrite the stamped bytes. Then `cleandoc` — the same call `inspect.getdoc`
+ makes — expands tabs, drops the common indentation, and trims blank edges. Composing them here is
+ what makes the contract exact rather than approximate: the emitter indents continuation lines to the
+ class body, and `cleandoc` takes that indentation straight back off.
+
+ This is the emitter's half of the fidelity contract, and the exact expression tests assert against:
+ `inspect.getdoc(cls) == normalized_docstring_text(description)`.
+ """
+ return cleandoc("\n".join(line.rstrip() for line in text.splitlines()))
+
+
+def _docstring_delimiter(text: str) -> str | None:
+ """The triple-quote delimiter `text` can sit inside verbatim, or `None` when it must be escaped.
+
+ Falling back to `'''` for a description containing `\"\"\"` is what keeps the docstring **backslash-free**,
+ and that is the whole point: any backslash trips ruff's `D301` ("use `r\"\"\"`"), and the `r` prefix it asks
+ for would then render the escapes literally, corrupting the text. `ruff format` rewrites `'''` to `\"\"\"`
+ only when that needs no escaping, so a `'''` chosen here is left alone.
+
+ `None` means no verbatim rendering exists — the text carries both triple-quote styles, a control
+ character, or a trailing backslash that would glue onto the closing delimiter.
+ """
+ if text.endswith("\\") or any(char not in {"\n", "\t"} and not char.isprintable() for char in text):
+ return None
+ return next((delimiter for delimiter in ('"""', "'''") if delimiter not in text and not text.endswith(delimiter[0])), None)
+
+
def class_docstring(description: str, *, extra_line: str | None = None, indent: str = " ") -> str:
- """Render a triple-quoted class docstring; append `extra_line` (e.g. an imprecision caveat) on its own line.
+ """Render a class docstring; append `extra_line` (e.g. an imprecision caveat) as a second paragraph.
+
+ Shaped the way a human writes one — summary on the first line, continuation lines indented to the class
+ body, closing delimiter on its own line — because that is the only shape the pydocstyle rules and
+ `ruff format` both leave untouched. Every rewrite they would otherwise apply (`D207` under-indented,
+ `D209` closing quotes, `D210` edge whitespace) is an auto-applied fix, and each one changes the body
+ bytes the codegen stamp hashes, which is precisely the drift this module exists to prevent.
- The body goes through `escape_py_string`, which escapes every `"` (and backslashes, newlines, control
- chars), so a hostile description can never form a bare `\"\"\"` and break out of the docstring.
+ So `__doc__` is deliberately *not* byte-identical to the authored description: it carries the
+ indentation every hand-written docstring carries. The invariant that does hold is the one consumers
+ read through — `inspect.getdoc(cls) == normalized_docstring_text(description)`. Exact bytes survive
+ where they are consumed programmatically: `Field(description=...)`, and the crate itself.
"""
- text = description or ""
- if extra_line:
- text = f"{text}\n\n{indent}{extra_line}" if text else extra_line
- quoted = escape_py_string(text)
- return f'{indent}"""{quoted[1:-1]}"""'
+ parts = [normalized_docstring_text(part) for part in (description or "", extra_line or "") if part.strip()]
+ if not parts:
+ return f'{indent}""""""'
+
+ body = "\n\n".join(parts)
+ delimiter = _docstring_delimiter(body)
+ if delimiter is None:
+ # No verbatim rendering exists, so escape. This trips `D301`, but ruff classes that fix as *unsafe*,
+ # so a consumer's `ruff check --fix` still leaves the bytes — and the stamp — alone.
+ # Each rendered break carries the body indent, exactly as the verbatim branches do, so `cleandoc`
+ # takes the same margin back off either way and the fidelity contract holds across both.
+ escaped = f"\\n\\n{indent}".join(f"\\n{indent}".join(escape_py_string(line)[1:-1] for line in part.split("\n")) for part in parts)
+ return f'{indent}"""{escaped}"""'
+ prefix = "r" if "\\" in body else ""
+ lines = body.split("\n")
+ if len(lines) == 1:
+ return f"{indent}{prefix}{delimiter}{lines[0]}{delimiter}"
+ continuation = "\n".join(f"{indent}{line}" if line else "" for line in lines[1:])
+ return f"{indent}{prefix}{delimiter}{lines[0]}\n{continuation}\n{indent}{delimiter}"
-def field_line(field: ResolvedField, *, annotation: str) -> str:
+
+def field_line(field: ResolvedField, *, annotation: str, imports: set[str]) -> str:
"""Render one `name: <annotation> = Field(...)` line, surfacing any type-tree imprecision inline.
`annotation` must already carry optionality in the emitter's own idiom (`Optional[...]` or `... | None`).
+
+ Registers its own `Field` import, like every other renderer here: an import nobody uses is an `F401`
+ that `ruff check --fix` deletes, which rewrites the body bytes and breaks the stamp. Demand-driven
+ is the only way that stays true for every crate shape — a crate of only opaque concepts, or only
+ concepts refining a native, emits no field line at all.
"""
+ imports.add("from pydantic import Field")
params: list[str]
if field.required and field.default_value is None:
params = ["..."]
@@ -97,11 +171,72 @@ def field_line(field: ResolvedField, *, annotation: str) -> str:
params = ["default=None"]
params.append(f"description={escape_py_string(field.description)}")
- line = f" {field.name}: {annotation} = Field({', '.join(params)})"
reasons = list(dict.fromkeys(iter_imprecision_reasons(field.resolved_type)))
- if reasons:
- line += f" # imprecise: {'; '.join(reasons)}"
- return line
+ comment = f" # imprecise: {'; '.join(reasons)}" if reasons else ""
+
+ flat = f" {field.name}: {annotation} = Field({', '.join(params)})"
+ # The trailing comment counts toward the width: ruff splits the call to fit `) # imprecise: …` too.
+ if len(flat) + len(comment) <= PY_EXPLODE_WIDTH:
+ return flat + comment
+
+ # Too long to be left alone: hand the formatter an already-exploded call with a magic trailing comma,
+ # which pins the shape at *any* line-length (verified 60..200). A long authored description or choice
+ # list has no width-independent flat form, so this is the only rendering a consumer's `ruff format`
+ # cannot rewrite. The annotation gets the same treatment when the head line alone is still too long.
+ opening = f" {field.name}: "
+ head_overhead = len(opening) + len(" = Field(")
+ head = f"{opening}{explode_bracket(annotation, budget=PY_EXPLODE_WIDTH - head_overhead)} = Field("
+ body = "".join(f" {param},\n" for param in params)
+ return f"{head}\n{body} ){comment}"
+
+
+def _split_top_level(inner: str) -> list[str]:
+ """Split `a, b[c, d], e` on its top-level commas only."""
+ members: list[str] = []
+ depth = 0
+ current = ""
+ for char in inner:
+ if char == "," and depth == 0:
+ members.append(current.strip())
+ current = ""
+ continue
+ depth += {"[": 1, "]": -1}.get(char, 0)
+ current += char
+ members.append(current.strip())
+ return members
+
+
+def explode_bracket(text: str, *, budget: int) -> str:
+ """Split a `Name[a, b, c]` annotation (optionally `| None`) across lines, if it exceeds `budget`.
+
+ Only the outermost bracket, and only when splitting it actually helps — that mirrors `ruff format`,
+ which breaks at the outermost bracket first and stops as soon as the result fits, so the emitted shape
+ is the one the formatter would have produced and therefore the one it leaves alone. The magic trailing
+ comma is what pins it: with it, no line-length setting can rejoin the members.
+
+ An optional annotation needs the parenthesized union form ruff wraps it in, one indent level deeper —
+ `X[…] | None` is not a bracket at its tail, so the flat spelling would otherwise be left too long.
+ """
+ if len(text) <= budget:
+ return text
+ optional_suffix = " | None"
+ base, is_optional = (text[: -len(optional_suffix)], True) if text.endswith(optional_suffix) else (text, False)
+ if not base.endswith("]"):
+ return text
+ name, _, inner = base.partition("[")
+ if not name or not inner:
+ return text
+
+ members = _split_top_level(inner[:-1])
+ if len(members) < 2:
+ return text
+
+ depth = 3 if is_optional else 2
+ rendered = "".join(f"{_PY_INDENT * depth}{member},\n" for member in members)
+ exploded = f"{name}[\n{rendered}{_PY_INDENT * (depth - 1)}]"
+ if not is_optional:
+ return exploded
+ return f"(\n{_PY_INDENT * 2}{exploded}\n{_PY_INDENT * 2}| None\n{_PY_INDENT})"
def any_annotation(*, imports: set[str]) -> str:
@@ -110,6 +245,58 @@ def any_annotation(*, imports: set[str]) -> str:
return "Any"
+def literal_annotation(*, choices: list[str] | None, imports: set[str]) -> str:
+ """The `Literal[...]` annotation, with double-quoted members (`repr()` would pick single quotes)."""
+ imports.add("from typing import Literal")
+ members = ", ".join(escape_py_string(choice) for choice in choices or [])
+ return f"Literal[{members}]"
+
+
+def render_import_block(imports: set[str]) -> str:
+ """Render the collected imports the way isort would, so the artifact is lint-clean on arrival.
+
+ A flat `sorted()` over the raw statements is not isort order. Two things it misses: `from X import a`
+ and `from X import b` have to merge into one line, and stdlib has to be separated from third-party by
+ a blank line.
+
+ Grouping targets the **consumer's** tree, where `pipelex` is an installed dependency and therefore
+ third-party. In this repo `pipelex` is first-party, so ruff here wants the opposite order — but no
+ generated artifact is ever committed to this repo, and the consumer is the only one who lints these
+ files. `tests/unit/pipelex/codegen/test_emitted_artifacts_are_lint_clean.py` lints them as a consumer
+ would, for that reason.
+ """
+ names_by_module: dict[str, set[str]] = defaultdict(set)
+ for statement in imports:
+ module, _, names = statement.removeprefix("from ").partition(" import ")
+ names_by_module[module].update(name.strip() for name in names.split(","))
+
+ stdlib: list[str] = []
+ third_party: list[str] = []
+ for module in sorted(names_by_module):
+ line = f"from {module} import {', '.join(sorted(names_by_module[module]))}"
+ group = stdlib if module.partition(".")[0] in sys.stdlib_module_names else third_party
+ group.append(line)
+
+ return "\n\n".join("\n".join(group) for group in (stdlib, third_party) if group)
+
+
+def python_module_body(*, header: str, imports: set[str], blocks: list[str]) -> str:
+ """Assemble a generated module; a projection with nothing to emit is its header alone.
+
+ The empty artifact needs an explicit canonical form, because the populated one degenerates into
+ something no formatter leaves alone: the import block has no class left to use it (`F401`, which
+ `ruff check --fix` deletes) and the block separator becomes a trailing blank-line run (which
+ `ruff format` collapses). Either rewrite changes the body bytes the stamp hashes, so `codegen
+ check` reports the file as hand-edited. A header is inert under both.
+
+ This is reachable without a degenerate crate: `python-structures` skips natives, so an ordinary
+ method that declares no concepts of its own leaves that emitter with nothing to write.
+ """
+ if not blocks:
+ return header
+ return f"{header}from __future__ import annotations\n\n{render_import_block(imports)}\n\n\n" + "\n\n\n".join(blocks) + "\n"
+
+
def order_by_base(concepts: list[ResolvedConcept], *, in_module: set[str]) -> list[ResolvedConcept]:
"""Order concepts so an in-module refinement base is defined before its subclass (inheritance is eager)."""
by_ref = {concept.concept_ref: concept for concept in concepts}
diff --git a/pipelex/codegen/emitters/python_pydantic.py b/pipelex/codegen/emitters/python_pydantic.py
index a9221aa36..adf9031f9 100644
--- a/pipelex/codegen/emitters/python_pydantic.py
+++ b/pipelex/codegen/emitters/python_pydantic.py
@@ -10,7 +10,15 @@ Modern typing throughout: builtin generics (`list` / `dict`), `X | None` for opt
"""
from pipelex.codegen.emitters.naming import python_class_name
-from pipelex.codegen.emitters.python_common import any_annotation, class_docstring, field_line, order_by_base, python_header
+from pipelex.codegen.emitters.python_common import (
+ any_annotation,
+ class_docstring,
+ field_line,
+ literal_annotation,
+ order_by_base,
+ python_header,
+ python_module_body,
+)
from pipelex.codegen.emitters.target import EmittedFile
from pipelex.codegen.resolved_concepts import ResolvedConcept, ResolvedLibrary
from pipelex.core.concepts.resolved_fields import ResolvedField, ResolvedType, ResolvedTypeKind
@@ -24,25 +32,27 @@ def emit_python_pydantic(library: ResolvedLibrary) -> list[EmittedFile]:
in_module = {concept.concept_ref for concept in library.concepts}
ordered = order_by_base(library.concepts, in_module=in_module)
- has_opaque = any(concept.structureless for concept in library.concepts)
- imports: set[str] = {"from pydantic import BaseModel, ConfigDict, Field"} if has_opaque else {"from pydantic import BaseModel, Field"}
+ # Every import is demand-driven — registered by the renderer that writes the name it imports.
+ # Seeding the set up front would emit an unused import for the crate shapes that use neither
+ # (all-opaque emits no field line; a fully-refining crate never reaches the `BaseModel` root),
+ # and `ruff check --fix` deletes those, rewriting the body bytes and breaking the stamp.
+ imports: set[str] = set()
blocks = [_render_class(concept, by_ref=by_ref, imports=imports) for concept in ordered]
- header = python_header(target="python-pydantic")
- import_block = "\n".join(sorted(imports))
- body = f"{header}from __future__ import annotations\n\n{import_block}\n\n\n" + "\n\n\n".join(blocks) + "\n"
+ body = python_module_body(header=python_header(target="python-pydantic"), imports=imports, blocks=blocks)
return [EmittedFile(filename=_FILENAME, content=body)]
def _render_class(concept: ResolvedConcept, *, by_ref: dict[str, ResolvedConcept], imports: set[str]) -> str:
class_name = python_class_name(domain=concept.domain, code=concept.code, needs_qualification=concept.needs_qualification)
- base = _base_class(concept, by_ref=by_ref)
+ base = _base_class(concept, by_ref=by_ref, imports=imports)
caveat = f"Imprecise: {concept.imprecision_reason}." if concept.structureless and concept.imprecision_reason else None
docstring = class_docstring(concept.description, extra_line=caveat)
header = f"class {class_name}({base}):"
if concept.structureless:
# Opaque = pass-through, never lossy (B1-1): pydantic's default extra="ignore" would
# silently strip every field on model_validate, so the unknown shape is kept verbatim.
+ imports.add("from pydantic import ConfigDict")
return f'{header}\n{docstring}\n\n model_config = ConfigDict(extra="allow")'
if not concept.fields:
return f"{header}\n{docstring}"
@@ -50,13 +60,19 @@ def _render_class(concept: ResolvedConcept, *, by_ref: dict[str, ResolvedConcept
return f"{header}\n{docstring}\n\n" + "\n".join(lines)
-def _base_class(concept: ResolvedConcept, *, by_ref: dict[str, ResolvedConcept]) -> str:
+def _base_class(concept: ResolvedConcept, *, by_ref: dict[str, ResolvedConcept], imports: set[str]) -> str:
if concept.base_ref is None:
- return "BaseModel"
+ return _base_model(imports=imports)
base = by_ref.get(concept.base_ref)
if base is not None:
return python_class_name(domain=base.domain, code=base.code, needs_qualification=base.needs_qualification)
# Cross-package / unknown base is not resolvable in-crate: fall back to a structurally valid root.
+ return _base_model(imports=imports)
+
+
+def _base_model(*, imports: set[str]) -> str:
+ """The pydantic root base, registering its import — a concept refining an in-module concept never reaches it."""
+ imports.add("from pydantic import BaseModel")
return "BaseModel"
@@ -64,7 +80,7 @@ def _render_field(concept_field: ResolvedField, *, by_ref: dict[str, ResolvedCon
annotation = _annotation(concept_field.resolved_type, by_ref=by_ref, imports=imports)
if not concept_field.required:
annotation = f"{annotation} | None"
- return field_line(concept_field, annotation=annotation)
+ return field_line(concept_field, annotation=annotation, imports=imports)
def _annotation(resolved_type: ResolvedType, *, by_ref: dict[str, ResolvedConcept], imports: set[str]) -> str:
@@ -87,8 +103,7 @@ def _annotation(resolved_type: ResolvedType, *, by_ref: dict[str, ResolvedConcep
imports.add("from datetime import time")
return "time"
case ResolvedTypeKind.LITERAL:
- imports.add("from typing import Literal")
- return f"Literal[{', '.join(repr(choice) for choice in resolved_type.choices or [])}]"
+ return literal_annotation(choices=resolved_type.choices, imports=imports)
case ResolvedTypeKind.CONCEPT:
return _concept_annotation(resolved_type, by_ref=by_ref, imports=imports)
case ResolvedTypeKind.LIST:
diff --git a/pipelex/codegen/emitters/python_structures.py b/pipelex/codegen/emitters/python_structures.py
index b7bad2b94..0bd1b5602 100644
--- a/pipelex/codegen/emitters/python_structures.py
+++ b/pipelex/codegen/emitters/python_structures.py
@@ -8,10 +8,22 @@ class. Native concepts themselves are not re-emitted — they already exist in t
Field annotations use `from __future__ import annotations`, so concept references are plain forward
names resolved from the module namespace (no explicit string quoting, no ordering constraint beyond
class inheritance).
+
+Modern typing throughout — builtin generics (`list` / `dict`) and `X | None` — matching both the
+runtime `StructureGenerator` and the repo's Python standards. The emitted bytes are lint-clean on
+arrival so a consumer's `ruff` run cannot invalidate the codegen stamp; see `render_import_block`.
"""
from pipelex.codegen.emitters.naming import python_class_name
-from pipelex.codegen.emitters.python_common import any_annotation, class_docstring, field_line, order_by_base, python_header
+from pipelex.codegen.emitters.python_common import (
+ any_annotation,
+ class_docstring,
+ field_line,
+ literal_annotation,
+ order_by_base,
+ python_header,
+ python_module_body,
+)
from pipelex.codegen.emitters.target import EmittedFile
from pipelex.codegen.resolved_concepts import ResolvedConcept, ResolvedLibrary
from pipelex.core.concepts.native.concept_native import NativeConceptCode
@@ -28,14 +40,13 @@ def emit_python_structures(library: ResolvedLibrary) -> list[EmittedFile]:
in_module = {concept.concept_ref for concept in emitted}
ordered = order_by_base(emitted, in_module=in_module)
- has_opaque = any(concept.structureless for concept in emitted)
- pydantic_import = "from pydantic import ConfigDict, Field" if has_opaque else "from pydantic import Field"
- imports: set[str] = {pydantic_import, "from pipelex.core.stuffs.structured_content import StructuredContent"}
+ # Every import is demand-driven — registered by the renderer that writes the name it imports.
+ # Seeding the set up front would emit an unused import for the crate shapes that use neither
+ # (all-opaque, or every concept refining a native), and `ruff check --fix` deletes those.
+ imports: set[str] = set()
blocks = [_render_class(concept, by_ref=by_ref, imports=imports) for concept in ordered]
- header = python_header(target="python-structures")
- import_block = "\n".join(sorted(imports))
- body = f"{header}from __future__ import annotations\n\n{import_block}\n\n\n" + "\n\n\n".join(blocks) + "\n"
+ body = python_module_body(header=python_header(target="python-structures"), imports=imports, blocks=blocks)
return [EmittedFile(filename=_FILENAME, content=body)]
@@ -48,6 +59,7 @@ def _render_class(concept: ResolvedConcept, *, by_ref: dict[str, ResolvedConcept
if concept.structureless:
# Opaque = pass-through, never lossy (B1-1): the runtime base inherits pydantic's default
# extra="ignore", which would silently strip every field on model_validate.
+ imports.add("from pydantic import ConfigDict")
return f'{header}\n{docstring}\n\n model_config = ConfigDict(extra="allow")'
if not concept.fields:
return f"{header}\n{docstring}"
@@ -57,22 +69,27 @@ def _render_class(concept: ResolvedConcept, *, by_ref: dict[str, ResolvedConcept
def _base_class(concept: ResolvedConcept, *, by_ref: dict[str, ResolvedConcept], imports: set[str]) -> str:
if concept.base_ref is None:
- return "StructuredContent"
+ return _structured_content(imports=imports)
if NativeConceptCode.is_native_concept_ref_or_code(concept_ref_or_code=concept.base_ref):
return _native_class(concept.base_ref, imports=imports)
base = by_ref.get(concept.base_ref)
if base is not None:
return python_class_name(domain=base.domain, code=base.code, needs_qualification=base.needs_qualification)
# Cross-package / unknown base is not resolvable in-crate: fall back to a structurally valid root.
+ return _structured_content(imports=imports)
+
+
+def _structured_content(*, imports: set[str]) -> str:
+ """The runtime root base, registering its import — every concept refining a native uses `_native_class` instead."""
+ imports.add("from pipelex.core.stuffs.structured_content import StructuredContent")
return "StructuredContent"
def _render_field(concept_field: ResolvedField, *, by_ref: dict[str, ResolvedConcept], imports: set[str]) -> str:
annotation = _annotation(concept_field.resolved_type, by_ref=by_ref, imports=imports)
if not concept_field.required:
- imports.add("from typing import Optional")
- annotation = f"Optional[{annotation}]"
- return field_line(concept_field, annotation=annotation)
+ annotation = f"{annotation} | None"
+ return field_line(concept_field, annotation=annotation, imports=imports)
def _annotation(resolved_type: ResolvedType, *, by_ref: dict[str, ResolvedConcept], imports: set[str]) -> str:
@@ -95,18 +112,15 @@ def _annotation(resolved_type: ResolvedType, *, by_ref: dict[str, ResolvedConcep
imports.add("from datetime import time")
return "time"
case ResolvedTypeKind.LITERAL:
- imports.add("from typing import Literal")
- return f"Literal[{', '.join(repr(choice) for choice in resolved_type.choices or [])}]"
+ return literal_annotation(choices=resolved_type.choices, imports=imports)
case ResolvedTypeKind.CONCEPT:
return _concept_annotation(resolved_type, by_ref=by_ref, imports=imports)
case ResolvedTypeKind.LIST:
- imports.add("from typing import List")
item = _annotation(resolved_type.item, by_ref=by_ref, imports=imports) if resolved_type.item else any_annotation(imports=imports)
- return f"List[{item}]"
+ return f"list[{item}]"
case ResolvedTypeKind.DICT:
- imports.add("from typing import Dict")
value = _annotation(resolved_type.value, by_ref=by_ref, imports=imports) if resolved_type.value else any_annotation(imports=imports)
- return f"Dict[str, {value}]"
+ return f"dict[str, {value}]"
case ResolvedTypeKind.ANY:
return any_annotation(imports=imports)
diff --git a/pipelex/codegen/emitters/ts_zod.py b/pipelex/codegen/emitters/ts_zod.py
index 0acebc842..122ee2017 100644
--- a/pipelex/codegen/emitters/ts_zod.py
+++ b/pipelex/codegen/emitters/ts_zod.py
@@ -31,6 +31,12 @@ from pipelex.tools.misc.json_utils import clean_json_content
_FILENAME = "types.ts"
_BINDER_FILENAME = "binder.ts"
+# Prettier's default `printWidth`. The emitted files must be prettier-clean on arrival, or a consumer's
+# formatter run rewrites the bytes and invalidates the codegen stamp (`pipelex codegen check` would then
+# report the file as hand-edited). Two prettier behaviors drive the emitted shape: it collapses runs of
+# blank lines to a single one, and it keeps an import on one line while it fits within this width.
+TS_PRINT_WIDTH = 80
+
def emit_ts_zod(library: ResolvedLibrary) -> list[EmittedFile]:
"""Emit the pure `types.ts` file and its `binder.ts` companion (wire<->domain parse/serialize)."""
@@ -40,7 +46,12 @@ def emit_ts_zod(library: ResolvedLibrary) -> list[EmittedFile]:
recursive_refs = _find_recursive_concept_refs(concepts)
header = _ts_header()
blocks = [_render_schema(concept, by_ref=by_ref, type_name_by_ref=type_name_by_ref, recursive_refs=recursive_refs) for concept in concepts]
- body = f'{header}import {{ z }} from "zod";\n\n\n' + "\n\n\n".join(blocks) + "\n"
+ if blocks:
+ body = f'{header}import {{ z }} from "zod";\n\n' + "\n\n".join(blocks) + "\n"
+ else:
+ # Nothing to project: the header alone. Keeping the import would leave `z` unused above a trailing
+ # blank-line run prettier collapses — a reformat, and a broken stamp. See `_emit_binder`.
+ body = header
return [EmittedFile(filename=_FILENAME, content=body), _emit_binder(concepts, type_name_by_ref=type_name_by_ref)]
@@ -74,7 +85,13 @@ def _render_schema(
if concept.concept_ref in recursive_refs:
type_expr = _concept_type_expr(concept, by_ref=by_ref, type_name_by_ref=type_name_by_ref)
return f"{doc}export type {type_name} = {type_expr};\nexport const {type_name}Schema: z.ZodType<{type_name}> = {schema_expr};"
- return f"{doc}export const {type_name}Schema = {schema_expr};\nexport type {type_name} = z.infer<typeof {type_name}Schema>;"
+
+ # A long enough concept name pushes the alias past prettier's print width, and prettier then breaks
+ # the generic — rewriting the bytes and invalidating the stamp. Emit the broken form ourselves.
+ alias = f"export type {type_name} = z.infer<typeof {type_name}Schema>;"
+ if len(alias) > TS_PRINT_WIDTH:
+ alias = f"export type {type_name} = z.infer<\n typeof {type_name}Schema\n>;"
+ return f"{doc}export const {type_name}Schema = {schema_expr};\n{alias}"
def _schema_expr(
@@ -110,11 +127,39 @@ def _render_field(
reasons = list(dict.fromkeys(iter_imprecision_reasons(concept_field.resolved_type)))
doc = _jsdoc(concept_field.description, imprecise="; ".join(reasons) if reasons else None, indent=" ")
expr = _zod_type(concept_field.resolved_type, by_ref=by_ref, type_name_by_ref=type_name_by_ref)
+ modifier = ""
if concept_field.default_value is not None:
- expr = f"{expr}.default({_format_default_value(concept_field.default_value)})"
+ modifier = f".default({_format_default_value(concept_field.default_value)})"
elif not concept_field.required:
- expr = f"{expr}.optional()"
- return f"{doc} {key}: {expr},"
+ modifier = ".optional()"
+
+ flat = f" {key}: {expr}{modifier},"
+ if len(flat) <= TS_PRINT_WIDTH or "\n" in expr:
+ return f"{doc}{flat}"
+ return f"{doc} {key}: {_break_zod_expr(expr, modifier=modifier)},"
+
+
+def _break_zod_expr(expr: str, *, modifier: str) -> str:
+ """Prettier's rendering of a field expression too long to sit on one line.
+
+ Only `z.enum([...])` can genuinely overflow — its members are authored choices with no length bound,
+ while every other emitted expression is a short fixed form or a `z.lazy(() => XSchema)` whose width is
+ the concept name's. Two shapes, and prettier picks between them: bare, it breaks the array in place;
+ with a modifier attached it breaks the whole member chain, pulling `.enum(` onto its own line.
+
+ Anything we do not model stays flat — `test_emitted_ts_lines_fit_the_print_width` is the guard that
+ turns an unmodelled overflow into a failing test rather than a silently broken stamp.
+ """
+ prefix, _, members_text = expr.partition("([")
+ if prefix != "z.enum" or not members_text.endswith("])"):
+ return f"{expr}{modifier}"
+ members = [member.strip() for member in members_text[:-2].split(",")]
+
+ if not modifier:
+ rendered = "".join(f" {member},\n" for member in members)
+ return f"z.enum([\n{rendered} ])"
+ rendered = "".join(f" {member},\n" for member in members)
+ return f"z\n .enum([\n{rendered} ])\n {modifier}"
def _format_default_value(value: Any) -> str:
@@ -280,26 +325,52 @@ def _emit_binder(concepts: list[ResolvedConcept], *, type_name_by_ref: dict[str,
thus no risk of corrupting arbitrary keys inside a `z.record()` / `z.unknown()` value.
"""
type_names = [type_name_by_ref[concept.concept_ref] for concept in concepts]
- import_lines = "\n".join(f" {name}Schema,\n type {name}," for name in type_names)
+ if not type_names:
+ # No pair to bind: the header alone is the canonical empty binder. The populated shape degenerates
+ # into `import { } from "./types";` (prettier rewrites the empty brace pair) above a trailing
+ # blank-line run (prettier collapses it) — either reformat breaks the stamped body hash.
+ return EmittedFile(filename=_BINDER_FILENAME, content=_binder_header())
+
+ specifiers = [specifier for name in type_names for specifier in (f"{name}Schema", f"type {name}")]
pairs = "\n\n".join(_binder_pair(name) for name in type_names)
- prelude = f'import {{\n{import_lines}\n}} from "./types";'
- return EmittedFile(filename=_BINDER_FILENAME, content=f"{_binder_header()}{prelude}\n\n\n{pairs}\n")
+
+ # Match prettier: one line while it fits, otherwise one specifier per line.
+ single_line = f'import {{ {", ".join(specifiers)} }} from "./types";'
+ if len(single_line) <= TS_PRINT_WIDTH:
+ prelude = single_line
+ else:
+ import_lines = "\n".join(f" {specifier}," for specifier in specifiers)
+ prelude = f'import {{\n{import_lines}\n}} from "./types";'
+
+ return EmittedFile(filename=_BINDER_FILENAME, content=f"{_binder_header()}{prelude}\n\n{pairs}\n")
def _binder_pair(type_name: str) -> str:
"""The `parse<Name>` / `serialize<Name>` pair for one concept type (wire-native, so `Schema.parse`)."""
return (
f"/** Parse a wire payload into a validated `{type_name}`. */\n"
- f"export function parse{type_name}(wire: unknown): {type_name} {{\n"
+ f"{_ts_signature(f'parse{type_name}', param='wire: unknown', returns=type_name)}\n"
f" return {type_name}Schema.parse(wire);\n"
f"}}\n\n"
f"/** Validate a `{type_name}` for the wire (keys are already wire-native). */\n"
- f"export function serialize{type_name}(value: {type_name}): {type_name} {{\n"
+ f"{_ts_signature(f'serialize{type_name}', param=f'value: {type_name}', returns=type_name)}\n"
f" return {type_name}Schema.parse(value);\n"
f"}}"
)
+def _ts_signature(name: str, *, param: str, returns: str) -> str:
+ """A `export function name(param): Returns {` header, wrapped the way prettier wraps it past the print width.
+
+ A concept code long enough to push the signature over the width is perfectly valid MTHDS, and prettier
+ breaks the parameter list onto its own line when it does not fit — a byte rewrite that breaks the stamp.
+ """
+ flat = f"export function {name}({param}): {returns} {{"
+ if len(flat) <= TS_PRINT_WIDTH:
+ return flat
+ return f"export function {name}(\n {param},\n): {returns} {{"
+
+
def _binder_header() -> str:
return (
"// ---------------------------------------------------------------------------\n"
diff --git a/pipelex/core/concepts/structure_generation/generator.py b/pipelex/core/concepts/structure_generation/generator.py
index 1f2ec8077..4768a376c 100644
--- a/pipelex/core/concepts/structure_generation/generator.py
+++ b/pipelex/core/concepts/structure_generation/generator.py
@@ -2,7 +2,7 @@ import ast
import textwrap
from datetime import date, datetime, time
from enum import Enum
-from typing import Any, List, Literal, Optional, cast
+from typing import Any, Literal, cast
from pydantic import Field
@@ -33,7 +33,7 @@ class StructureGenerator:
domain-qualified structure class name.
"""
self.imports = {
- "from typing import Optional, List, Dict, Any, Literal",
+ "from typing import Any, Literal",
"from enum import Enum",
"from pipelex.core.stuffs.structured_content import StructuredContent",
"from pydantic import Field",
@@ -291,9 +291,16 @@ class StructureGenerator:
"""Render one Python field definition from a neutral resolved field."""
python_type = self._python_type_from_resolved(resolved_field.resolved_type)
- # Make optional if not required
+ # Make optional if not required. A concept ref renders as a *quoted* forward reference (this
+ # module, unlike the codegen emitters, has no `from __future__ import annotations`), and
+ # `"Foo" | None` is a TypeError at runtime — str doesn't implement `|`. Fold the union inside
+ # the quotes so the whole annotation stays one deferred expression for model_rebuild to eval.
+ # Only a *bare* quoted ref needs this: `list["Foo"] | None` is a real GenericAlias and works.
if not resolved_field.required:
- python_type = f"Optional[{python_type}]"
+ if python_type.startswith('"') and python_type.endswith('"'):
+ python_type = f'"{python_type[1:-1]} | None"'
+ else:
+ python_type = f"{python_type} | None"
# Generate Field parameters (default/... first, then description)
field_params = [self._format_field_description(resolved_field.description)]
@@ -330,16 +337,16 @@ class StructureGenerator:
return "time"
case ResolvedTypeKind.LITERAL:
choices = resolved_type.choices or []
- return f"Literal[{', '.join(repr(choice) for choice in choices)}]"
+ return f"Literal[{', '.join(self._escape_string_for_python(choice) for choice in choices)}]"
case ResolvedTypeKind.CONCEPT:
return self._python_type_for_concept(resolved_type)
case ResolvedTypeKind.LIST:
item_type = self._python_type_from_resolved(resolved_type.item) if resolved_type.item else "Any"
- return f"List[{item_type}]"
+ return f"list[{item_type}]"
case ResolvedTypeKind.DICT:
key_type = self._python_type_from_resolved(resolved_type.key) if resolved_type.key else "str"
value_type = self._python_type_from_resolved(resolved_type.value) if resolved_type.value else "Any"
- return f"Dict[{key_type}, {value_type}]"
+ return f"dict[{key_type}, {value_type}]"
case ResolvedTypeKind.ANY:
return "Any"
@@ -391,9 +398,6 @@ class StructureGenerator:
"datetime": datetime,
"time": time,
"Enum": Enum,
- "Optional": Optional,
- "List": List, # noqa: UP006
- "Dict": dict,
"Any": Any,
"Literal": Literal,
"Field": Field,
diff --git a/pipelex/libraries/library_manager.py b/pipelex/libraries/library_manager.py
index 09ea92dd2..f27d0efa8 100644
--- a/pipelex/libraries/library_manager.py
+++ b/pipelex/libraries/library_manager.py
@@ -1246,8 +1246,9 @@ class LibraryManager(LibraryManagerAbstract):
concepts: List of concepts to check for cycles
"""
# TODO: Refactor to inspect ConceptStructureBlueprint directly (concept_ref and item_concept_ref fields)
- # instead of the generated Python types. This would be more direct and wouldn't depend on
- # how types are generated (e.g., Optional wrappers for non-required fields).
+ # instead of the generated Python types. This would be more direct and wouldn't depend on how types
+ # are generated — a coupling that has already bitten once: respelling the generator's optional fields
+ # from `Optional[X]` to `X | None` silently disabled cycle detection entirely.
class_registry = get_class_registry()
# Build mappings from class names to concept refs
@@ -1269,19 +1270,21 @@ class LibraryManager(LibraryManagerAbstract):
return []
type_names: list[str] = []
- origin = getattr(annotation, "__origin__", None)
+ # Branch on `__args__`, not `__origin__`: a PEP 604 union (`X | None`, a `types.UnionType`)
+ # carries its members in `__args__` but has **no** `__origin__`, unlike `typing.Optional[X]`.
+ # Keying off the origin sent every `X | None` annotation down the simple-type branch, where
+ # it has no `__name__` either — so the reference vanished and no cycle was ever found.
args = getattr(annotation, "__args__", ())
- if origin is None:
+ if args:
+ # Generic or union (Optional[X], X | None, list[X], Union[X, Y], ...): recurse into members.
+ for arg in args:
+ type_names.extend(extract_type_names(arg))
+ else:
# Simple type, get its name
type_name = getattr(annotation, "__name__", None)
if type_name:
type_names.append(type_name)
- else:
- # Generic type (like Optional[X], list[X], Union[X, Y], etc.)
- # Recursively extract from all type arguments
- for arg in args:
- type_names.extend(extract_type_names(arg))
return type_names
diff --git a/pipelex/pipe_run/pipe_run_params.py b/pipelex/pipe_run/pipe_run_params.py
index 1682ad8af..ae47d9149 100644
--- a/pipelex/pipe_run/pipe_run_params.py
+++ b/pipelex/pipe_run/pipe_run_params.py
@@ -9,7 +9,7 @@ from pipelex import log
from pipelex.cogt.content_generation.cogt_run_params import CogtRunParams, check_mock_usage_requires_dry
from pipelex.core.memory.working_memory import BATCH_ITEM_STUFF_NAME, MAIN_STUFF_NAME
from pipelex.core.pipes.variable_multiplicity import VariableMultiplicity, VariableMultiplicityResolution
-from pipelex.pipe_run.pipe_run_mode import PipeRunMode # noqa: TC001 — pydantic resolves the field annotation at runtime
+from pipelex.pipe_run.pipe_run_mode import PipeRunMode
from pipelex.pipeline.exceptions import PipeStackOverflowError
diff --git a/pipelex/system/pipelex_service/remote_config_fetcher.py b/pipelex/system/pipelex_service/remote_config_fetcher.py
index 3110da4c2..3e0c9e711 100644
--- a/pipelex/system/pipelex_service/remote_config_fetcher.py
+++ b/pipelex/system/pipelex_service/remote_config_fetcher.py
@@ -30,7 +30,7 @@ Behaviour:
from __future__ import annotations
import json
-from datetime import datetime # noqa: TC003 — Pydantic v2 resolves this annotation at runtime
+from datetime import datetime
from typing import Any
import httpx
diff --git a/pyproject.toml b/pyproject.toml
index bb352f4d9..173a0df5d 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -464,6 +464,16 @@ ignore = [
[tool.ruff.lint.pydocstyle]
convention = "google"
+[tool.ruff.lint.flake8-type-checking]
+# Pydantic resolves annotations at runtime to build validators, so an annotation-only import that
+# TC00x moves into an `if TYPE_CHECKING:` block breaks the model. These bases mark the classes whose
+# annotations are runtime-evaluated. This is also the setting consumers of `pipelex codegen types`
+# need for the generated artifacts — see docs/under-the-hood/codegen-projections.md.
+runtime-evaluated-base-classes = [
+ "pydantic.BaseModel",
+ "pipelex.core.stuffs.structured_content.StructuredContent",
+]
+
[tool.ruff.lint.per-file-ignores]
"tests/**/*.py" = [
"INP001", # Allow test files to not have __init__.py in their directories (avoids namespace collisions)
diff --git a/subject_grants.toml b/subject_grants.toml
index 39d832375..6ad9c908f 100644
--- a/subject_grants.toml
+++ b/subject_grants.toml
@@ -952,6 +952,14 @@ rationale = "Single-operand transformation: derives the class-name mapping over
param = "name"
rationale = "Single-operand conversion of the name."
+["pipelex/codegen/emitters/python_common.py::_docstring_delimiter"]
+param = "text"
+rationale = "Single-operand derivation: picks the delimiter that fits the text."
+
+["pipelex/codegen/emitters/python_common.py::_split_top_level"]
+param = "inner"
+rationale = "Verb-object: splits the bracket inner text; single operand."
+
["pipelex/codegen/emitters/python_common.py::class_docstring"]
param = "description"
rationale = "Noun-named derivation: builds the docstring from the description; extras stay keyword."
@@ -960,6 +968,10 @@ rationale = "Noun-named derivation: builds the docstring from the description; e
param = "value"
rationale = "Single-operand conversion of the value."
+["pipelex/codegen/emitters/python_common.py::explode_bracket"]
+param = "text"
+rationale = "Verb-object: explodes the bracketed annotation text; the budget is context."
+
["pipelex/codegen/emitters/python_common.py::field_line"]
param = "field"
rationale = "Noun-named derivation: the source line for the field; annotation stays keyword."
@@ -968,6 +980,10 @@ rationale = "Noun-named derivation: the source line for the field; annotation st
param = "value"
rationale = "Single-operand formatting of the value."
+["pipelex/codegen/emitters/python_common.py::normalized_docstring_text"]
+param = "text"
+rationale = "Single-operand normalization of the text into its docstring-canonical form."
+
["pipelex/codegen/emitters/python_common.py::order_by_base"]
param = "concepts"
rationale = "Verb-object: orders the concepts; in_module is context."
@@ -976,6 +992,10 @@ rationale = "Verb-object: orders the concepts; in_module is context."
param = "concept"
rationale = "Visitor: visits the concept; single operand."
+["pipelex/codegen/emitters/python_common.py::render_import_block"]
+param = "imports"
+rationale = "Verb-object: renders the collected imports into a source block; single operand."
+
["pipelex/codegen/emitters/python_pydantic.py::_annotation"]
param = "resolved_type"
rationale = "Noun-named derivation: the annotation for the resolved type; context stays keyword."
@@ -1032,6 +1052,10 @@ rationale = "Verb-object: emits from the resolved library; single operand."
param = "type_name"
rationale = "Noun-named derivation: encodes the type name into its parse/serialize pair."
+["pipelex/codegen/emitters/ts_zod.py::_break_zod_expr"]
+param = "expr"
+rationale = "Verb-object: breaks the zod expression across lines; the modifier is context."
+
["pipelex/codegen/emitters/ts_zod.py::_concept_dependencies"]
param = "concept"
rationale = "Noun-named derivation: the dependency refs of the concept; single operand."
@@ -1088,6 +1112,10 @@ rationale = "Single-operand sanitization of the text."
param = "concept"
rationale = "Noun-named derivation: the schema expr for the concept; context stays keyword."
+["pipelex/codegen/emitters/ts_zod.py::_ts_signature"]
+param = "name"
+rationale = "Verb-object: renders the signature for the named function; param and return type are context."
+
["pipelex/codegen/emitters/ts_zod.py::_ts_type"]
param = "resolved_type"
rationale = "Noun-named derivation: the TS type for the resolved type; context stays keyword."
diff --git a/tests/unit/pipelex/codegen/conftest.py b/tests/unit/pipelex/codegen/conftest.py
index 0c3b286a2..47ccbd55a 100644
--- a/tests/unit/pipelex/codegen/conftest.py
+++ b/tests/unit/pipelex/codegen/conftest.py
@@ -156,6 +156,18 @@ def materialized_image_crate() -> LibraryCrate:
return normalize_crate(authored, mthds_version=CRATE_TEST_VERSION)
+def natives_only_crate() -> LibraryCrate:
+ """A crate holding nothing but natives — what an ordinary method that declares no concepts of its
+ own normalizes to (a `Text -> Text` pipe materializes `native.Text` and nothing else).
+
+ `python-structures` skips natives, so this is the *reachable* route to an empty projection: the
+ library is non-empty, yet that emitter has no class to write.
+ """
+ authored = LibraryCrate(concepts={"native.Text": ConceptBlueprint(description="A text")})
+ return normalize_crate(authored, mthds_version=CRATE_TEST_VERSION)
+
+
@pytest.fixture
def refines_crate() -> LibraryCrate:
"""A concept refining a structureless native keeps its refines link, so the emitter renders inheritance."""
@@ -168,6 +180,140 @@ def refines_crate() -> LibraryCrate:
)
+def every_type_kind_crate() -> LibraryCrate:
+ """A normalized crate whose fields cover **every** `ResolvedTypeKind`, plus an optional field and a
+ structureless (opaque) concept.
+
+ This is the single source of type-kind coverage for the lint-clean regression test:
+ `test_emitted_artifacts_are_lint_clean` asserts the resolved trees reach every enum member, so a
+ newly added `ResolvedTypeKind` that nobody wires in here fails loudly instead of going unlinted.
+ """
+ authored = LibraryCrate(
+ concepts={
+ "lintcheck.Detail": ConceptBlueprint(
+ description="A nested detail",
+ structure={"note": ConceptStructureBlueprint(description="A note", type=ConceptStructureBlueprintFieldType.TEXT, required=True)},
+ ),
+ # A structureless concept: emits the two-paragraph docstring (description + imprecision caveat).
+ "lintcheck.Opaque": ConceptBlueprint(description="A structureless concept"),
+ # Descriptions carrying characters the docstring renderer must not backslash-escape: a
+ # double quote and a backslash both put a `\` in the docstring under naive escaping, which
+ # ruff's D301 then auto-rewrites to an `r` prefix — changing the bytes and breaking the stamp.
+ "lintcheck.Quoted": ConceptBlueprint(
+ description='The "primary" thing',
+ structure={
+ "pattern": ConceptStructureBlueprint(
+ description=r"Matches the \d regex", type=ConceptStructureBlueprintFieldType.TEXT, required=True
+ )
+ },
+ ),
+ "lintcheck.Record": ConceptBlueprint(
+ description="A record touching every resolved type kind",
+ structure={
+ # scalars
+ "title": ConceptStructureBlueprint(description="Title", type=ConceptStructureBlueprintFieldType.TEXT, required=True),
+ "ratio": ConceptStructureBlueprint(description="Ratio", type=ConceptStructureBlueprintFieldType.NUMBER, required=True),
+ "count": ConceptStructureBlueprint(description="Count", type=ConceptStructureBlueprintFieldType.INTEGER, required=True),
+ "is_active": ConceptStructureBlueprint(description="Active", type=ConceptStructureBlueprintFieldType.BOOLEAN, required=True),
+ "published_on": ConceptStructureBlueprint(description="Date", type=ConceptStructureBlueprintFieldType.DATE, required=True),
+ "recorded_at": ConceptStructureBlueprint(
+ description="Timestamp", type=ConceptStructureBlueprintFieldType.DATETIME, required=True
+ ),
+ "starts_at": ConceptStructureBlueprint(description="Start time", type=ConceptStructureBlueprintFieldType.TIME, required=True),
+ # literal — the double-quoting path
+ "status": ConceptStructureBlueprint(description="Status", choices=["draft", "final"], required=True),
+ # concept refs: one in-module, one native (exercises the native content-class import)
+ "detail": ConceptStructureBlueprint(
+ description="Detail", type=ConceptStructureBlueprintFieldType.CONCEPT, concept_ref="Detail", required=True
+ ),
+ "label": ConceptStructureBlueprint(
+ description="Label", type=ConceptStructureBlueprintFieldType.CONCEPT, concept_ref="Text", required=True
+ ),
+ # containers
+ "tags": ConceptStructureBlueprint(
+ description="Tags", type=ConceptStructureBlueprintFieldType.LIST, item_type="text", required=True
+ ),
+ "counts": ConceptStructureBlueprint(
+ description="Counts", type=ConceptStructureBlueprintFieldType.DICT, key_type="text", value_type="integer", required=True
+ ),
+ # ANY — only reachable nested, from genuine source imprecision. An untyped list also
+ # emits the trailing `# imprecise:` comment, so that shape gets linted too.
+ "items": ConceptStructureBlueprint(description="Untyped items", type=ConceptStructureBlueprintFieldType.LIST, required=True),
+ # an optional field, so the `X | None` path is emitted too
+ "note": ConceptStructureBlueprint(description="Optional note", type=ConceptStructureBlueprintFieldType.TEXT, required=False),
+ # A description with no width bound. Emitted flat it exceeds any line-length, and
+ # `ruff format` then wraps the `Field(...)` call — a byte rewrite that breaks the stamp.
+ "narrative": ConceptStructureBlueprint(
+ description=(
+ "A thoroughly explained field whose description the method author wrote at length "
+ "because the domain genuinely requires that much nuance to be unambiguous"
+ ),
+ type=ConceptStructureBlueprintFieldType.TEXT,
+ required=True,
+ ),
+ # A choice list with no width bound, in both the required and optional spellings —
+ # the optional one is the `Literal[...] | None` union, which ruff wraps in parentheses.
+ "workflow_state": ConceptStructureBlueprint(
+ description="Workflow state",
+ choices=["awaiting_triage", "in_progress_with_owner", "blocked_on_dependency", "ready_for_review"],
+ required=True,
+ ),
+ "prior_state": ConceptStructureBlueprint(
+ description="Prior workflow state",
+ choices=["awaiting_triage", "in_progress_with_owner", "blocked_on_dependency", "ready_for_review"],
+ required=False,
+ ),
+ },
+ ),
+ # A multi-line description, and one padded with edge whitespace. Rendered naively the first
+ # trips D207/D209 and the second D210 — all three auto-applied fixes that rewrite the bytes.
+ "lintcheck.Spanning": ConceptBlueprint(
+ description="A description that spans lines.\n\nAs a TOML multi-line string naturally does.",
+ structure={
+ "padded": ConceptStructureBlueprint(
+ description=" Padded with edge whitespace ", type=ConceptStructureBlueprintFieldType.TEXT, required=True
+ )
+ },
+ ),
+ # A description that cannot sit between `\"\"\"` at all, so the renderer has to fall back to
+ # `'''` — the only delimiter swap that keeps the docstring backslash-free and thus D301-clean.
+ "lintcheck.TripleQuoted": ConceptBlueprint(description='A description containing """ inside it'),
+ },
+ domains={"lintcheck": DomainBlueprint(code="lintcheck", description="Lint check domain")},
+ )
+ return normalize_crate(authored, mthds_version=CRATE_TEST_VERSION)
+
+
+def all_opaque_crate() -> LibraryCrate:
+ """Every concept structureless — the shape of an ordinary `Question -> Answer` method.
+
+ No field line is emitted, so a `Field` import seeded up front would be unused: an `F401` that
+ `ruff check --fix` deletes, rewriting the body and breaking the stamp.
+ """
+ authored = LibraryCrate(
+ concepts={
+ "qa.Question": ConceptBlueprint(description="A question asked by the user"),
+ "qa.Answer": ConceptBlueprint(description="An answer to the question"),
+ },
+ domains={"qa": DomainBlueprint(code="qa", description="Question answering")},
+ )
+ return normalize_crate(authored, mthds_version=CRATE_TEST_VERSION)
+
+
+def refines_native_only_crate() -> LibraryCrate:
+ """Every concept refines a native, so `python-structures` reaches its runtime root base for none of
+ them — a seeded `StructuredContent` import would be unused, and so would `Field`.
+ """
+ authored = LibraryCrate(
+ concepts={"digest.Summary": ConceptBlueprint(description="A summary", refines="Text")},
+ domains={"digest": DomainBlueprint(code="digest", description="Digesting")},
+ )
+ return normalize_crate(authored, mthds_version=CRATE_TEST_VERSION)
+
+
def load_generated_module(content: str, *, tmp_path: Path, name: str) -> Any:
"""Write generated Python to disk, import it, and return it (proves compile + exec).
diff --git a/tests/unit/pipelex/codegen/test_description_escaping.py b/tests/unit/pipelex/codegen/test_description_escaping.py
index 0b0eaf19a..7038b7f32 100644
--- a/tests/unit/pipelex/codegen/test_description_escaping.py
+++ b/tests/unit/pipelex/codegen/test_description_escaping.py
@@ -1,8 +1,10 @@
from collections.abc import Callable
+from inspect import getdoc
from pathlib import Path
import pytest
+from pipelex.codegen.emitters.python_common import normalized_docstring_text
from pipelex.codegen.emitters.python_pydantic import emit_python_pydantic
from pipelex.codegen.emitters.python_structures import emit_python_structures
from pipelex.codegen.emitters.target import EmittedFile
@@ -35,8 +37,14 @@ class TestDescriptionEscaping:
content = emitter(library)[0].content
module = load_generated_module(content, tmp_path=tmp_path, name=module_name)
- assert module.Payload.__doc__ == description
+ # The docstring is rendered the way a human writes one, so `__doc__` carries the class-body
+ # indentation — the shape that survives `ruff format` and the pydocstyle rules, and therefore the
+ # only shape that keeps the stamp valid. `inspect.getdoc` takes that indentation straight back
+ # off, which is what `normalized_docstring_text` describes and what consumers actually read.
+ assert getdoc(module.Payload) == normalized_docstring_text(description)
assert not hasattr(module.Payload, "injected")
+ # The field description is a plain single-line literal, so it stays byte-identical — exact bytes
+ # survive wherever they are consumed programmatically rather than read as documentation.
assert module.Payload.model_fields["value"].description == description
def test_typescript_descriptions_cannot_escape_jsdoc(self):
diff --git a/tests/unit/pipelex/codegen/test_emitted_artifacts_are_lint_clean.py b/tests/unit/pipelex/codegen/test_emitted_artifacts_are_lint_clean.py
new file mode 100644
index 000000000..c70960594
--- /dev/null
+++ b/tests/unit/pipelex/codegen/test_emitted_artifacts_are_lint_clean.py
@@ -0,0 +1,262 @@
+"""The emitted Python artifacts must be lint-clean on arrival.
+
+A stamped artifact's hash is a raw SHA-256 over its body bytes, so any reformatting a consumer's own
+`ruff check --fix` / `ruff format` would apply changes the bytes and makes `pipelex codegen check`
+report the file as hand-edited — accusing the user of the one thing they did not do. The emitters are
+therefore required to emit exactly what ruff wants; this test is the guard on that property.
+
+Three rules are deliberately suppressed *in the invocation* rather than satisfied in the emitted bytes.
+The test for whether a rule may be suppressed here is whether ruff can **auto-apply** its fix: only an
+applied fix rewrites bytes, and only rewritten bytes break a stamp. A rule that merely reports is a
+config-level caveat for the consumer to make, not a defect in what we emit.
+
+- `INP001` is an artifact of linting a loose directory (no `__init__.py`), not of file content.
+- `E501` fires on a single long string literal — an authored description or choice value that has no
+ wrappable form, because you cannot break a string without altering the author's text. It has no fix,
+ and it is not in ruff's default rule set, so it neither breaks a stamp nor affects most consumers.
+ The *surrounding* call is a different matter, and is wrapped: see `PY_EXPLODE_WIDTH`.
+- `TC003` wants `from datetime import date` moved into an `if TYPE_CHECKING:` block, which would
+ **break** the generated code — pydantic resolves annotations at runtime to build validators. Ruff's
+ `runtime-evaluated-base-classes` exists for exactly this, so the test passes it, exercising the
+ configuration we tell consumers to use.
+
+Imports are linted as a **consumer** would: in their tree `pipelex` is an installed dependency, so it
+belongs in the third-party group. In this repo it is first-party and ruff wants the opposite order —
+but no generated artifact is ever committed here, so the consumer's grouping is the one that matters.
+See `render_import_block` in `pipelex/codegen/emitters/python_common.py`.
+"""
+
+import re
+import shutil
+import subprocess # noqa: S404
+import sys
+from collections.abc import Callable
+from pathlib import Path
+from typing import cast
+
+import pytest
+
+from pipelex.codegen.emitters.python_common import python_header
+from pipelex.codegen.emitters.python_pydantic import emit_python_pydantic
+from pipelex.codegen.emitters.python_structures import emit_python_structures
+from pipelex.codegen.emitters.target import EmittedFile
+from pipelex.codegen.emitters.ts_zod import TS_PRINT_WIDTH, emit_ts_zod
+from pipelex.codegen.resolved_concepts import ResolvedLibrary, resolve_concepts_from_crate
+from pipelex.core.concepts.resolved_fields import ResolvedType, ResolvedTypeKind
+from pipelex.libraries.library_crate import LibraryCrate
+
+_REPO_ROOT = Path(__file__).parents[4]
+_PYPROJECT = _REPO_ROOT / "pyproject.toml"
+
+_RUNTIME_EVALUATED_BASES = "['pydantic.BaseModel','pipelex.core.stuffs.structured_content.StructuredContent']"
+
+_CONSUMER_LINT_CONFIG = [
+ "--config",
+ str(_PYPROJECT),
+ "--config",
+ "lint.isort.known-third-party=['pipelex']",
+ "--config",
+ "lint.per-file-ignores={'*'=['INP001','E501']}",
+ "--config",
+ f"lint.flake8-type-checking.runtime-evaluated-base-classes={_RUNTIME_EVALUATED_BASES}",
+]
+
+_EMITTERS: list[tuple[str, Callable[[ResolvedLibrary], list[EmittedFile]]]] = [
+ ("python-structures", emit_python_structures),
+ ("python-pydantic", emit_python_pydantic),
+]
+
+
+_EMPTY_LIBRARY = ResolvedLibrary(mthds_version="1.0.0-test", concepts=[])
+"""A crate with no concepts at all — reachable from a bundle that declares only a domain."""
+
+
+def _collect_kinds(resolved_type: ResolvedType) -> set[ResolvedTypeKind]:
+ """Every kind reachable in a resolved-type tree, depth-first."""
+ kinds = {resolved_type.kind}
+ for child in (resolved_type.item, resolved_type.key, resolved_type.value):
+ if child is not None:
+ kinds |= _collect_kinds(child)
+ return kinds
+
+
+def _assert_ruff_clean(*, emitted: list[EmittedFile], tmp_path: Path, target: str, line_length: int | None = None) -> None:
+ """Write the artifacts out and require both `ruff check` and `ruff format --check` to find nothing."""
+ try:
+ subprocess.run([sys.executable, "-m", "ruff", "--version"], check=True, capture_output=True) # noqa: S603
+ except (subprocess.CalledProcessError, FileNotFoundError):
+ pytest.skip("ruff not available in current interpreter")
+
+ for emitted_file in emitted:
+ (tmp_path / emitted_file.filename).write_text(emitted_file.content, encoding="utf-8")
+
+ width_config = ["--config", f"line-length={line_length}"] if line_length is not None else []
+ label = f"[{target}]" if line_length is None else f"[{target} @ line-length={line_length}]"
+
+ check = subprocess.run( # noqa: S603
+ [sys.executable, "-m", "ruff", "check", str(tmp_path), *_CONSUMER_LINT_CONFIG, *width_config],
+ capture_output=True,
+ text=True,
+ check=False,
+ )
+ assert check.returncode == 0, f"{label} emitted artifact is not lint-clean:\n{check.stdout}{check.stderr}"
+
+ formatted = subprocess.run( # noqa: S603
+ [sys.executable, "-m", "ruff", "format", "--check", str(tmp_path), "--config", str(_PYPROJECT), *width_config],
+ capture_output=True,
+ text=True,
+ check=False,
+ )
+ assert formatted.returncode == 0, f"{label} emitted artifact is not `ruff format` clean:\n{formatted.stdout}{formatted.stderr}"
+
+
+class TestEmittedArtifactsAreLintClean:
+ def test_fixture_covers_every_resolved_type_kind(self, every_type_kind_crate: LibraryCrate):
+ """The fixture is the single source of type-kind coverage — a new kind must be wired into it."""
+ library = resolve_concepts_from_crate(every_type_kind_crate)
+ covered: set[ResolvedTypeKind] = set()
+ for concept in library.concepts:
+ for concept_field in concept.fields:
+ covered |= _collect_kinds(concept_field.resolved_type)
+
+ missing = set(ResolvedTypeKind) - covered
+ assert not missing, f"every_type_kind_crate no longer covers: {sorted(missing)} — wire them into the fixture"
+
+ @pytest.mark.parametrize(("target", "emit"), _EMITTERS)
+ def test_emitted_artifact_passes_ruff(
+ self,
+ every_type_kind_crate: LibraryCrate,
+ tmp_path: Path,
+ target: str,
+ emit: Callable[[ResolvedLibrary], list[EmittedFile]],
+ ):
+ """`ruff check` and `ruff format --check` must both find nothing to change."""
+ _assert_ruff_clean(emitted=emit(resolve_concepts_from_crate(every_type_kind_crate)), tmp_path=tmp_path, target=target)
+
+ @pytest.mark.parametrize(("target", "emit"), _EMITTERS)
+ def test_empty_projection_is_lint_clean(
+ self,
+ tmp_path: Path,
+ target: str,
+ emit: Callable[[ResolvedLibrary], list[EmittedFile]],
+ ):
+ """A projection with nothing to emit still has to arrive lint-clean.
+
+ Emitting the import block with no class to use it is an `F401`, and the block separator leaves a
+ trailing blank-line run `ruff format` collapses — either one rewrites the body bytes and breaks
+ the stamp, which is exactly the accusation this module exists to prevent.
+ """
+ _assert_ruff_clean(emitted=emit(_EMPTY_LIBRARY), tmp_path=tmp_path, target=target)
+
+ @pytest.mark.parametrize("line_length", [88, 100, 120, 150, 200])
+ @pytest.mark.parametrize(("target", "emit"), _EMITTERS)
+ def test_emitted_artifact_is_stable_at_every_consumer_line_length(
+ self,
+ every_type_kind_crate: LibraryCrate,
+ tmp_path: Path,
+ target: str,
+ emit: Callable[[ResolvedLibrary], list[EmittedFile]],
+ line_length: int,
+ ):
+ """The consumer's `line-length` is theirs, not ours, and it decides what their formatter rewrites.
+
+ A long authored description or choice list has no flat form that survives every width, so the
+ emitter hands over an already-exploded call carrying a magic trailing comma — the one construct
+ Black and ruff both refuse to rejoin at any width. `PY_EXPLODE_WIDTH` is the floor of the range
+ guarded here: ruff's own default, and so the tightest setting a consumer is likely to lint at.
+ """
+ _assert_ruff_clean(
+ emitted=emit(resolve_concepts_from_crate(every_type_kind_crate)), tmp_path=tmp_path, target=target, line_length=line_length
+ )
+
+ @pytest.mark.parametrize("crate_fixture", ["all_opaque_crate", "refines_native_only_crate"])
+ @pytest.mark.parametrize(("target", "emit"), _EMITTERS)
+ def test_crates_that_use_no_seeded_import_are_lint_clean(
+ self,
+ request: pytest.FixtureRequest,
+ tmp_path: Path,
+ target: str,
+ emit: Callable[[ResolvedLibrary], list[EmittedFile]],
+ crate_fixture: str,
+ ):
+ """Every import has to be registered by whoever writes the name, not seeded up front.
+
+ Both crate shapes are ordinary — a `Question -> Answer` method declares only structureless
+ concepts, and a summarizer may declare only refinements of a native — and each leaves one of the
+ formerly-seeded imports unused. `F401` is a *safe* fix, so `ruff check --fix` deletes the line,
+ rewrites the body, and `codegen check` then reports a file nobody touched as hand-edited.
+ """
+ crate = cast("LibraryCrate", request.getfixturevalue(crate_fixture))
+ _assert_ruff_clean(emitted=emit(resolve_concepts_from_crate(crate)), tmp_path=tmp_path, target=target)
+
+ def test_natives_only_crate_emits_a_lint_clean_structures_module(self, natives_only_crate: LibraryCrate, tmp_path: Path):
+ """The reachable route to an empty projection, and the reason the case is not academic.
+
+ `python-structures` skips natives (they already exist in the runtime), so an ordinary method that
+ declares no concepts of its own — a `Text -> Text` pipe — leaves that emitter with nothing to
+ write, even though the library itself is not empty.
+ """
+ emitted = emit_python_structures(resolve_concepts_from_crate(natives_only_crate))
+ assert [emitted_file.content for emitted_file in emitted] == [python_header(target="python-structures")]
+ _assert_ruff_clean(emitted=emitted, tmp_path=tmp_path, target="python-structures")
+
+ def test_empty_ts_projection_is_a_bare_header(self):
+ """Both TypeScript artifacts collapse to their header alone when there is nothing to project.
+
+ The alternative — keeping the import line — emits `import { } from "./types";` (which prettier
+ rewrites to `import {} …`) above a trailing blank-line run (which it collapses). Needs no prettier
+ binary, so like the invariant above it always runs.
+ """
+ for emitted_file in emit_ts_zod(_EMPTY_LIBRARY):
+ body = emitted_file.content
+ # Comments and nothing else: no import to go unused, and no blank line to collapse (a blank
+ # line does not start with `//`, so this rules out the trailing run too).
+ assert all(line.startswith("//") for line in body.splitlines()), f"{emitted_file.filename} is not a bare header:\n{body}"
+ assert body.endswith("\n")
+
+ def test_emitted_ts_has_no_collapsible_blank_runs(self, every_type_kind_crate: LibraryCrate):
+ """Prettier collapses a run of blank lines to one under *every* config — so emitting two (the
+ Python idiom) guarantees a reformat, and a broken stamp, for any TypeScript consumer.
+
+ This invariant needs no prettier binary, so unlike `test_emitted_ts_is_prettier_clean` it always runs.
+ """
+ for emitted_file in emit_ts_zod(resolve_concepts_from_crate(every_type_kind_crate)):
+ offenders = re.findall(r"\n[ \t]*\n[ \t]*\n", emitted_file.content)
+ assert not offenders, f"{emitted_file.filename} contains {len(offenders)} collapsible blank-line run(s)"
+
+ def test_emitted_ts_lines_fit_the_print_width(self, every_type_kind_crate: LibraryCrate):
+ """No emitted code line may exceed prettier's print width — it would be wrapped, breaking the stamp.
+
+ This is the guard that actually holds the TypeScript line in CI, where there is no node toolchain
+ for `test_emitted_ts_is_prettier_clean` to use. It needs no prettier binary, and it is what turns
+ an unmodelled overflow (a long concept name, a long choice list) into a failing test.
+
+ Comment lines are exempt on purpose: prettier reflows code, never the contents of a `//` line or a
+ `/** … */` block, so a long JSDoc line is stable at any width.
+ """
+ for emitted_file in emit_ts_zod(resolve_concepts_from_crate(every_type_kind_crate)):
+ overlong = [
+ line for line in emitted_file.content.splitlines() if len(line) > TS_PRINT_WIDTH and not line.lstrip().startswith(("//", "/*", "*"))
+ ]
+ assert not overlong, f"{emitted_file.filename} has lines past prettier's print width:\n" + "\n".join(overlong)
+
+ def test_emitted_ts_is_prettier_clean(self, every_type_kind_crate: LibraryCrate, tmp_path: Path):
+ """`prettier --check` must find nothing to change in the emitted TypeScript.
+
+ Skipped when prettier is not on PATH — this is a Python repo, so CI has no node toolchain. The
+ always-on structural guard above is what holds the line there.
+ """
+ prettier = shutil.which("prettier")
+ if prettier is None:
+ pytest.skip("prettier not on PATH")
+
+ for emitted_file in emit_ts_zod(resolve_concepts_from_crate(every_type_kind_crate)):
+ (tmp_path / emitted_file.filename).write_text(emitted_file.content, encoding="utf-8")
+
+ checked = subprocess.run( # noqa: S603
+ [prettier, "--check", str(tmp_path / "*.ts")],
+ capture_output=True,
+ text=True,
+ check=False,
+ )
+ assert checked.returncode == 0, f"emitted ts-zod artifact is not prettier-clean:\n{checked.stdout}{checked.stderr}"
diff --git a/tests/unit/pipelex/codegen/test_python_structures_emitter.py b/tests/unit/pipelex/codegen/test_python_structures_emitter.py
index 9031fd93e..aa3ba8c7c 100644
--- a/tests/unit/pipelex/codegen/test_python_structures_emitter.py
+++ b/tests/unit/pipelex/codegen/test_python_structures_emitter.py
@@ -56,11 +56,11 @@ class TestPythonStructuresEmitter:
assert "class beta__Result(" in content
def test_typed_dict_field_maps_to_dict_annotation(self, materialized_image_crate: LibraryCrate, tmp_path: Path):
- """An authored typed dict renders the runtime Dict annotation (natives are skipped by this
+ """An authored typed dict renders the builtin `dict` annotation (natives are skipped by this
emitter, so the authored concept is the DICT path's only route here) and builds at runtime.
"""
content = emit_python_structures(resolve_concepts_from_crate(materialized_image_crate))[0].content
- assert "captions: Dict[str, str]" in content
+ assert "captions: dict[str, str]" in content
module = load_generated_module(content, tmp_path=tmp_path, name="gen_structures_dict")
gallery: Any = module.Gallery(captions={"p1": "sunset"})
assert gallery.captions == {"p1": "sunset"}
diff --git a/tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator.py b/tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator.py
index 818f772ca..b8e7ac9f5 100644
--- a/tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator.py
+++ b/tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator.py
@@ -44,15 +44,15 @@ To regenerate: pipelex build structures <target_directory>
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class TestModel(StructuredContent):
"""Generated TestModel class"""
name: str = Field(..., description="Name field")
- age: Optional[int] = Field(default=None, description="Age field")
- active: Optional[bool] = Field(default=True, description="Active status")
+ age: int | None = Field(default=None, description="Age field")
+ active: bool | None = Field(default=True, description="Active status")
'''
assert result == expected_code
@@ -102,15 +102,15 @@ To regenerate: pipelex build structures <target_directory>
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class ComplexModel(StructuredContent):
"""Generated ComplexModel class"""
- tags: Optional[List[str]] = Field(default=None, description="List of tags")
- metadata: Optional[Dict[str, str]] = Field(default=None, description="Metadata dictionary")
- scores: List[float] = Field(..., description="List of scores")
+ tags: list[str] | None = Field(default=None, description="List of tags")
+ metadata: dict[str, str] | None = Field(default=None, description="Metadata dictionary")
+ scores: list[float] = Field(..., description="List of scores")
'''
assert result == expected_code
@@ -146,15 +146,15 @@ To regenerate: pipelex build structures <target_directory>
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class Product(StructuredContent):
"""Generated Product class"""
- name: Optional[str] = Field(default=None, description="Product name")
- category: Literal['electronics', 'clothing', 'food', 'books'] = Field(..., description="Product category")
- size: Optional[Literal['XS', 'S', 'M', 'L', 'XL']] = Field(default=None, description="Size of the product")
+ name: str | None = Field(default=None, description="Product name")
+ category: Literal["electronics", "clothing", "food", "books"] = Field(..., description="Product category")
+ size: Literal["XS", "S", "M", "L", "XL"] | None = Field(default=None, description="Size of the product")
'''
assert result == expected_code
@@ -200,15 +200,15 @@ To regenerate: pipelex build structures <target_directory>
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class Order(StructuredContent):
"""Generated Order class"""
- status: Literal['pending', 'processing', 'shipped', 'delivered'] = Field(..., description="Order status")
- story_points: Literal['1', '3', '5', '8', '13'] = Field(..., description="Fibonacci story points")
- price_tier: Optional[Literal['0.25', '0.5', '1', '2', '5']] = Field(default=None, description="Price tier multiplier")
+ status: Literal["pending", "processing", "shipped", "delivered"] = Field(..., description="Order status")
+ story_points: Literal["1", "3", "5", "8", "13"] = Field(..., description="Fibonacci story points")
+ price_tier: Literal["0.25", "0.5", "1", "2", "5"] | None = Field(default=None, description="Price tier multiplier")
'''
assert result == expected_code
@@ -238,7 +238,7 @@ To regenerate: pipelex build structures <target_directory>
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class EmptyModel(StructuredContent):
@@ -276,14 +276,14 @@ To regenerate: pipelex build structures <target_directory>
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class DocumentInfo(StructuredContent):
"""Generated DocumentInfo class"""
title: str = Field(..., description="Document title")
- page_count: Optional[int] = Field(default=None, description="Number of pages")
+ page_count: int | None = Field(default=None, description="Number of pages")
'''
assert result == expected_code
@@ -315,7 +315,7 @@ To regenerate: pipelex build structures <target_directory>
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class ConvenienceTest(StructuredContent):
@@ -369,18 +369,18 @@ To regenerate: pipelex build structures <target_directory>
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class TypeMappingTest(StructuredContent):
"""Generated TypeMappingTest class"""
- text_field: Optional[str] = Field(default=None, description="Text field")
- number_field: Optional[float] = Field(default=None, description="Number field")
- integer_field: Optional[int] = Field(default=None, description="Integer field")
- boolean_field: Optional[bool] = Field(default=None, description="Boolean field")
- list_field: Optional[List[str]] = Field(default=None, description="List field")
- dict_field: Optional[Dict[str, int]] = Field(default=None, description="Dict field")
+ text_field: str | None = Field(default=None, description="Text field")
+ number_field: float | None = Field(default=None, description="Number field")
+ integer_field: int | None = Field(default=None, description="Integer field")
+ boolean_field: bool | None = Field(default=None, description="Boolean field")
+ list_field: list[str] | None = Field(default=None, description="List field")
+ dict_field: dict[str, int] | None = Field(default=None, description="Dict field")
'''
assert result == expected_code
@@ -484,7 +484,7 @@ class TypeMappingTest(StructuredContent):
assert cast("type[StructuredContent]", generated_class).model_fields["starts_at"].default == time_default
def test_list_of_date_field_maps_to_list_of_calendar_date(self):
- """`type = "list", item_type = "date"` generates a `List[date]` field with the bare-class import, and exec-validates."""
+ """`type = "list", item_type = "date"` generates a `list[date]` field with the bare-class import, and exec-validates."""
structure_blueprint = {
"deadlines": ConceptStructureBlueprint(
description="Deadlines", type=ConceptStructureBlueprintFieldType.LIST, item_type="date", required=True
@@ -496,11 +496,11 @@ class TypeMappingTest(StructuredContent):
)
assert "from datetime import date" in result
- assert 'deadlines: List[date] = Field(..., description="Deadlines")' in result
+ assert 'deadlines: list[date] = Field(..., description="Deadlines")' in result
assert issubclass(cast("type[StructuredContent]", generated_class), StructuredContent)
def test_list_of_datetime_field_maps_to_list_of_timestamp(self):
- """`type = "list", item_type = "datetime"` generates a `List[datetime]` field with the bare-class import, and exec-validates."""
+ """`type = "list", item_type = "datetime"` generates a `list[datetime]` field with the bare-class import, and exec-validates."""
structure_blueprint = {
"logged_at": ConceptStructureBlueprint(
description="Log timestamps", type=ConceptStructureBlueprintFieldType.LIST, item_type="datetime", required=True
@@ -512,7 +512,7 @@ class TypeMappingTest(StructuredContent):
)
assert "from datetime import datetime" in result
- assert 'logged_at: List[datetime] = Field(..., description="Log timestamps")' in result
+ assert 'logged_at: list[datetime] = Field(..., description="Log timestamps")' in result
assert issubclass(cast("type[StructuredContent]", generated_class), StructuredContent)
def test_list_of_datetime_field_with_default_round_trips(self):
@@ -581,14 +581,14 @@ To regenerate: pipelex build structures <target_directory>
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class RequiredFieldsModel(StructuredContent):
"""Generated RequiredFieldsModel class"""
title: str = Field(..., description="Required title")
- optional_field: Optional[str] = Field(default=None, description="Optional field")
+ optional_field: str | None = Field(default=None, description="Optional field")
'''
assert result == expected_code
@@ -637,15 +637,15 @@ To regenerate: pipelex build structures <target_directory>
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class PersonWithDefaults(StructuredContent):
"""Generated PersonWithDefaults class"""
- name: Optional[str] = Field(default="Anonymous", description="Person name")
- age: Optional[int] = Field(default=0, description="Person age")
- active: Optional[bool] = Field(default=True, description="Is active")
+ name: str | None = Field(default="Anonymous", description="Person name")
+ age: int | None = Field(default=0, description="Person age")
+ active: bool | None = Field(default=True, description="Is active")
'''
assert result == expected_code
@@ -694,15 +694,15 @@ To regenerate: pipelex build structures <target_directory>
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class ListTypesModel(StructuredContent):
"""Generated ListTypesModel class"""
- text_list: Optional[List[str]] = Field(default=None, description="List of text items")
- number_list: List[float] = Field(..., description="List of numbers")
- integer_list: Optional[List[int]] = Field(default=None, description="List of integers")
+ text_list: list[str] | None = Field(default=None, description="List of text items")
+ number_list: list[float] = Field(..., description="List of numbers")
+ integer_list: list[int] | None = Field(default=None, description="List of integers")
'''
assert result == expected_code
@@ -754,15 +754,15 @@ To regenerate: pipelex build structures <target_directory>
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class DictTypesModel(StructuredContent):
"""Generated DictTypesModel class"""
- string_to_string: Optional[Dict[str, str]] = Field(default=None, description="String to string mapping")
- string_to_number: Dict[str, float] = Field(..., description="String to number mapping")
- string_to_integer: Optional[Dict[str, int]] = Field(default=None, description="String to integer mapping")
+ string_to_string: dict[str, str] | None = Field(default=None, description="String to string mapping")
+ string_to_number: dict[str, float] = Field(..., description="String to number mapping")
+ string_to_integer: dict[str, int] | None = Field(default=None, description="String to integer mapping")
'''
assert result == expected_code
@@ -820,7 +820,7 @@ To regenerate: pipelex build structures <target_directory>
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class ComplexItem(StructuredContent):
@@ -828,10 +828,10 @@ class ComplexItem(StructuredContent):
id: int = Field(..., description="Unique identifier")
name: str = Field(..., description="Display name")
- tags: Optional[List[str]] = Field(default=None, description="Associated tags")
- metadata: Optional[Dict[str, str]] = Field(default=None, description="Additional metadata")
- active: Optional[bool] = Field(default=True, description="Whether item is active")
- priority: Optional[Literal['low', 'medium', 'high', 'urgent']] = Field(default="medium", description="Priority level")
+ tags: list[str] | None = Field(default=None, description="Associated tags")
+ metadata: dict[str, str] | None = Field(default=None, description="Additional metadata")
+ active: bool | None = Field(default=True, description="Whether item is active")
+ priority: Literal["low", "medium", "high", "urgent"] | None = Field(default="medium", description="Priority level")
'''
assert result == expected_code
@@ -873,7 +873,7 @@ from datetime import date
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class PersonInfo(StructuredContent):
@@ -918,7 +918,7 @@ To regenerate: pipelex build structures <target_directory>
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class ValidTestModel(StructuredContent):
@@ -996,13 +996,13 @@ from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pipelex.core.stuffs.text_content import TextContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class Question(TextContent):
"""Generated Question class"""
- author: Optional[str] = Field(default=None, description="The author of the question")
+ author: str | None = Field(default=None, description="The author of the question")
'''
assert generated_code == expected_code
@@ -1058,14 +1058,14 @@ from enum import Enum
from pipelex.core.stuffs.image_content import ImageContent
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class TableScreenshot(ImageContent):
"""Generated TableScreenshot class"""
- table_name: Optional[str] = Field(default=None, description="Name of the table in the screenshot")
- row_count: Optional[int] = Field(default=None, description="Number of rows in the table")
+ table_name: str | None = Field(default=None, description="Name of the table in the screenshot")
+ row_count: int | None = Field(default=None, description="Number of rows in the table")
'''
assert generated_code == expected_code
@@ -1112,13 +1112,13 @@ from enum import Enum
from pipelex.core.stuffs.number_content import NumberContent
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class Temperature(NumberContent):
"""Generated Temperature class"""
- unit: Optional[str] = Field(default=None, description="Unit of measurement")
+ unit: str | None = Field(default=None, description="Unit of measurement")
'''
assert generated_code == expected_code
@@ -1165,13 +1165,13 @@ from enum import Enum
from pipelex.core.stuffs.json_content import JSONContent
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class ConfigData(JSONContent):
"""Generated ConfigData class"""
- schema_version: Optional[str] = Field(default=None, description="Version of the schema")
+ schema_version: str | None = Field(default=None, description="Version of the schema")
'''
assert generated_code == expected_code
@@ -1212,7 +1212,7 @@ from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pipelex.core.stuffs.text_content import TextContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class EnhancedText(TextContent):
@@ -1268,13 +1268,13 @@ from enum import Enum
from pipelex.core.stuffs.document_content import DocumentContent
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class Invoice(DocumentContent):
"""Generated Invoice class"""
- document_type: Optional[str] = Field(default=None, description="Type of document")
+ document_type: str | None = Field(default=None, description="Type of document")
'''
assert generated_code == expected_code
diff --git a/tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator_concept_refs.py b/tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator_concept_refs.py
index 419d40d9b..d2f61da34 100644
--- a/tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator_concept_refs.py
+++ b/tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator_concept_refs.py
@@ -27,7 +27,7 @@ STANDARD_IMPORTS = """\
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
"""
@@ -96,7 +96,7 @@ class Invoice(StructuredContent):
class Invoice(StructuredContent):
"""Generated Invoice class"""
- line_items: List["myapp__LineItem"] = Field(..., description="List of line items")
+ line_items: list["myapp__LineItem"] = Field(..., description="List of line items")
total: float = Field(..., description="Invoice total")
'''
)
@@ -132,7 +132,7 @@ class Invoice(StructuredContent):
class CategoryNode(StructuredContent):
"""Generated CategoryNode class"""
- parent: Optional["myapp__Category"] = Field(default=None, description="Optional parent reference")
+ parent: "myapp__Category | None" = Field(default=None, description="Optional parent reference")
name: str = Field(..., description="Category name")
'''
)
@@ -162,7 +162,7 @@ class CategoryNode(StructuredContent):
class Parent(StructuredContent):
"""Generated Parent class"""
- children: Optional[List["myapp__Category"]] = Field(default=None, description="Optional child categories")
+ children: list["myapp__Category"] | None = Field(default=None, description="Optional child categories")
'''
)
assert generated_code == expected_code
@@ -198,7 +198,7 @@ class Order(StructuredContent):
"""Generated Order class"""
customer: "crm__Customer" = Field(..., description="Customer from CRM domain")
- products: List["inventory__Product"] = Field(..., description="Products from inventory domain")
+ products: list["inventory__Product"] = Field(..., description="Products from inventory domain")
'''
)
assert generated_code == expected_code
@@ -288,10 +288,10 @@ class ComplexEntity(StructuredContent):
id: int = Field(..., description="Unique identifier")
name: str = Field(..., description="Name")
- owner: Optional["myapp__User"] = Field(default=None, description="Owner reference")
- tags: Optional[List[str]] = Field(default=None, description="Tags")
- related_items: Optional[List["myapp__Item"]] = Field(default=None, description="Related items")
- metadata: Optional[Dict[str, str]] = Field(default=None, description="Metadata")
+ owner: "myapp__User | None" = Field(default=None, description="Owner reference")
+ tags: list[str] | None = Field(default=None, description="Tags")
+ related_items: list["myapp__Item"] | None = Field(default=None, description="Related items")
+ metadata: dict[str, str] | None = Field(default=None, description="Metadata")
'''
)
assert generated_code == expected_code
@@ -358,5 +358,5 @@ class ComplexEntity(StructuredContent):
# Native concepts in lists should also generate imports
assert "from pipelex.core.stuffs.image_content import ImageContent" in generated_code
- assert "images: List[ImageContent] = Field(...," in generated_code
+ assert "images: list[ImageContent] = Field(...," in generated_code
assert issubclass(generated_class, StructuredContent)
diff --git a/tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator_escaping.py b/tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator_escaping.py
index bad97aa3a..f5b2caad7 100644
--- a/tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator_escaping.py
+++ b/tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator_escaping.py
@@ -48,7 +48,7 @@ To regenerate: pipelex build structures <target_directory>
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class Animal(StructuredContent):
@@ -93,7 +93,7 @@ To regenerate: pipelex build structures <target_directory>
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class TestClass(StructuredContent):
@@ -138,7 +138,7 @@ To regenerate: pipelex build structures <target_directory>
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class Message(StructuredContent):
@@ -183,7 +183,7 @@ To regenerate: pipelex build structures <target_directory>
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class FilePath(StructuredContent):
@@ -228,7 +228,7 @@ To regenerate: pipelex build structures <target_directory>
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class MultiLine(StructuredContent):
@@ -273,7 +273,7 @@ To regenerate: pipelex build structures <target_directory>
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class Columns(StructuredContent):
@@ -319,7 +319,7 @@ To regenerate: pipelex build structures <target_directory>
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class ComplexType(StructuredContent):
@@ -365,13 +365,13 @@ To regenerate: pipelex build structures <target_directory>
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class WithDefault(StructuredContent):
"""Generated WithDefault class"""
- field_with_default: Optional[str] = Field(default="Example: \\"quoted text\\"", description="A field with quoted default")
+ field_with_default: str | None = Field(default="Example: \\"quoted text\\"", description="A field with quoted default")
'''
assert generated_code == expected_code
@@ -413,13 +413,13 @@ To regenerate: pipelex build structures <target_directory>
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class DefaultPath(StructuredContent):
"""Generated DefaultPath class"""
- path: Optional[str] = Field(default="C:\\\\Program Files\\\\App", description="Default path")
+ path: str | None = Field(default="C:\\\\Program Files\\\\App", description="Default path")
'''
assert generated_code == expected_code
@@ -458,7 +458,7 @@ To regenerate: pipelex build structures <target_directory>
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class EmptyDesc(StructuredContent):
@@ -509,7 +509,7 @@ To regenerate: pipelex build structures <target_directory>
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class LongDesc(StructuredContent):
@@ -558,7 +558,7 @@ To regenerate: pipelex build structures <target_directory>
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class UnicodeTest(StructuredContent):
@@ -603,7 +603,7 @@ To regenerate: pipelex build structures <target_directory>
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class CRLF(StructuredContent):
@@ -662,16 +662,16 @@ To regenerate: pipelex build structures <target_directory>
from enum import Enum
from pipelex.core.stuffs.structured_content import StructuredContent
from pydantic import Field
-from typing import Optional, List, Dict, Any, Literal
+from typing import Any, Literal
class AnimalRecord(StructuredContent):
"""Generated AnimalRecord class"""
animal_type: str = Field(..., description="The species or type of animal (e.g., \\"fox\\", \\"penguin\\", \\"octopus\\")")
- owner_name: Optional[str] = Field(default=None, description="The owner\'s full name")
- file_path: Optional[str] = Field(default="C:\\\\default\\\\path", description="Location on disk (e.g., C:\\\\Animals\\\\data.txt)")
- notes: Optional[str] = Field(default=None, description="Notes with \\"quotes\\" and\\nnewlines")
+ owner_name: str | None = Field(default=None, description="The owner\'s full name")
+ file_path: str | None = Field(default="C:\\\\default\\\\path", description="Location on disk (e.g., C:\\\\Animals\\\\data.txt)")
+ notes: str | None = Field(default=None, description="Notes with \\"quotes\\" and\\nnewlines")
'''
assert generated_code == expected_code
diff --git a/wip/codegen-lint-clean-artifacts.md b/wip/codegen-lint-clean-artifacts.md
new file mode 100644
index 000000000..8b7b0617f
--- /dev/null
+++ b/wip/codegen-lint-clean-artifacts.md
@@ -0,0 +1,281 @@
+# Emit lint-clean codegen artifacts
+
+**Status: PR [#1070](https://github.com/Pipelex/pipelex/pull/1070) → `dev`, OPEN.** Branch `fix/Codegen-lint-clean` off `dev` at `8c0b99b3a`. Phases 1–5 done except the release-gated cross-repo sweep (5.4). Local `make agent-check` + full `make agent-test` + `make drift-check` green.
+
+⚠ **Round 2 (2026-07-29) found the lint-clean claim did not hold, and the fixes are not yet committed.** Five input shapes still broke the stamp, one of them a regression this branch introduced. All are fixed and guarded; see [Review rounds](#review-rounds). The two claims that changed shape as a result — `__doc__` is no longer byte-identical to the description, and long lines are now pre-exploded — are ruled decisions (D2, D3 below), not incidental.
+
+## Review guide
+
+Everything above the `---` divider is the record of what shipped. The original plan below it is kept for its reasoning and rejected alternatives — read it for *why the approach is what it is*, not for current state.
+
+### What is in the diff, and why each file belongs
+
+| Area | Files | What changed |
+| --- | --- | --- |
+| **The fix** | `codegen/emitters/python_common.py`, `python_pydantic.py`, `python_structures.py`, `ts_zod.py` | Emit what the formatter wants: builtin generics, `X \| None`, double-quoted `Literal` members, isort-grouped imports (`render_import_block`), mode-aware docstring escaping, prettier-shaped blank lines and import wrapping, and the header-only empty projection (`python_module_body`). |
+| **Runtime generator (D1)** | `core/concepts/structure_generation/generator.py` | Same respelling, so the two paths that render the same resolved-type tree agree. Its output is `exec()`d, never written to disk, so no linter ever saw it — this is a consistency change, not a lint fix. |
+| **A real bug the respelling exposed** | `libraries/library_manager.py` | Concept cycle detection branched on `__origin__`, which a PEP 604 union does not have. **Not scope creep** — see finding 5. |
+| **Repo-side lint config** | `pyproject.toml`, `pipe_run/pipe_run_params.py`, `system/pipelex_service/remote_config_fetcher.py` | Sets `runtime-evaluated-base-classes` (phase 5.1), which retires two hand-written `# noqa: TC001` / `TC003` that were working around the same rule one file at a time. |
+| **Convention gate** | `subject_grants.toml` | Positional-subject grants for the new `_breaks_out_of_docstring` and `render_import_block`. Mechanically required by `make check-keyword-only`. |
+| **Tests** | `tests/unit/pipelex/codegen/*`, `.../structure_generation/test_structure_generator*.py` | The new guard plus the expectation churn from the respelling. |
+| **Docs** | `CHANGELOG.md`, `docs/under-the-hood/codegen-projections.md` | The breaking-bytes notice + the consumer contract. |
+
+### Deliberate — do not flag these
+
+1. **The emitted `python-structures` import order fails *this repo's* own ruff (`I001`) — and running this repo's `ruff --fix` over a generated `structures.py` therefore still breaks its stamp.** That is expected, not a surviving bug. It imports both `pydantic` and `pipelex`, and isort wants opposite orders depending on whether `pipelex` is first-party (here) or a dependency (a consumer's tree) — the two cannot both be satisfied. The emitter targets the consumer, who is the only one who lints these files; this repo commits no generated artifact. The test passes `lint.isort.known-third-party=['pipelex']` for exactly this reason, and so must any manual check — see [Verify it locally](#verify-it-locally). `python-pydantic` imports no `pipelex` and so is config-independent. Full reasoning in finding 1.
+2. **`TC003` is answered by consumer configuration, not by emitted bytes.** Moving `from datetime import date` into `if TYPE_CHECKING:` would break the generated models — pydantic resolves annotations at runtime. See ["`TC003` must not be satisfied by the emitter"](#tc003-must-not-be-satisfied-by-the-emitter).
+3. **`INP001` is suppressed in the test's ruff invocation, not in the bytes.** It is an artifact of pointing ruff at a bare directory, not of file content.
+4. **`class_docstring`'s fully-escaped branch is now reached only by genuinely pathological input** — a description carrying *both* `"""` and `'''`, a control character, or a trailing backslash. A description containing only `"""` falls back to `'''` instead, which keeps it backslash-free and therefore `D301`-clean (see [D2](#d2--docstring-rendering)). The escaped branch does trip `D301`, and that is accepted: ruff classes the fix as unsafe, so `ruff check --fix` never applies it and the stamp holds.
+5. **The empty projection carries no imports at all**, not even `from __future__ import annotations`. A header is the only shape inert under every formatter. See finding 7.
+6. **Emitted bytes change for all three targets — that is the point, and it is breaking.** Anyone holding a committed projection sees drift until they regenerate; the changelog says so and gives the command. See [Blast radius](#blast-radius).
+
+### Where each property is guarded
+
+| Property | Guard |
+| --- | --- |
+| Both Python targets are ruff-clean (`check` + `format --check`) | `test_emitted_artifacts_are_lint_clean.py::test_emitted_artifact_passes_ruff` |
+| The fixture reaches **every** `ResolvedTypeKind`, so a new kind cannot go unlinted | `…::test_fixture_covers_every_resolved_type_kind` |
+| An **empty** projection is ruff-clean for both Python targets | `…::test_empty_projection_is_lint_clean` |
+| The *reachable* empty case — natives-only crate through `python-structures` — is a bare header | `…::test_natives_only_crate_emits_a_lint_clean_structures_module` |
+| TS has no collapsible blank runs (config-independent, always runs) | `…::test_emitted_ts_has_no_collapsible_blank_runs` |
+| An empty TS projection is a bare header (config-independent, always runs) | `…::test_empty_ts_projection_is_a_bare_header` |
+| No emitted TS code line exceeds the print width (config-independent, always runs) | `…::test_emitted_ts_lines_fit_the_print_width` — the guard that actually holds the TS width line in CI. Comment lines are exempt: prettier reflows code, never `//` or `/** … */` contents. |
+| TS is genuinely prettier-clean | `…::test_emitted_ts_is_prettier_clean` — **skipped unless `prettier` is on PATH, which CI does not have** (Python repo, no node toolchain). The three always-on invariants above are what actually hold the TS line in CI; the prettier assertion is a local-run confirmation. Worth knowing before trusting the TS side. |
+| Python stays formatter-stable across the consumer's `line-length`, not just ours | `…::test_emitted_artifact_is_stable_at_every_consumer_line_length` (88–200) |
+| Crate shapes that use no seeded import stay lint-clean | `…::test_crates_that_use_no_seeded_import_are_lint_clean` (all-opaque, refines-native-only) |
+| `inspect.getdoc(cls)` round-trips the authored description | `test_description_escaping.py` |
+| Concept cycle detection still sees optional refs | `test_concept_to_concept_references.py` (this is what caught finding 5) |
+| The runtime generator's new spelling | `tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator*.py` |
+
+### Review rounds
+
+- **Round 1 — greptile, resolved in `c9a7238e4`.** One finding, on `ts_zod.py`: the empty binder stayed formatter-unstable. Confirmed, and wider than reported — all four artifacts degenerate the same way, and the case is reachable from an ordinary method rather than only a degenerate crate. Written up as finding 7. Thread answered and resolved; no other thread was open.
+
+- **Round 2 — `/review` (Claude) + `codex exec`, 2026-07-29.** The central claim did not hold. Reproduced end to end from an ordinary `.mthds` file: `codegen check` reported `[hand-edited]` on a projection nobody had touched. Five shapes, each verified against the real `ruff` / `prettier` binaries rather than argued:
+
+ 1. **A multi-line description — a regression this branch introduced.** `_breaks_out_of_docstring` treated a real newline as safe, which is true of the Python *parser* and false of the *formatter*. One-part docstrings tripped `D209`, two-part `D207`; both are safe fixes, and `ruff format` rewrote them too. The previous escaping put the description on one physical line, which trips only `D301` — an *unsafe* fix ruff never applies. So the bytes now changed where they previously survived.
+ 2. **A description with edge whitespace** → `D210`, a safe fix. Pre-existing.
+ 3. **An all-opaque or refines-native-only crate** → an unused `Field` / `StructuredContent` import, `F401`, a safe fix. Both are ordinary method shapes. (Found by codex; the fixture mixed opaque with fielded concepts, hiding it.)
+ 4. **A long description or choice list** → exceeds any `line-length`, and `ruff format` wraps the call. Fires even at this repo's generous 150 columns. (Found by codex.)
+ 5. **A long concept code in `ts-zod`** → prettier wraps the `z.infer` alias and the binder signatures; a long choice list expands `z.enum([...])`. Unguarded in CI, since the prettier test skips without a node toolchain.
+
+ Fixed on all five, with the rulings recorded as [D2](#d2--docstring-rendering) and [D3](#d3--line-width) below. Verified by sweep rather than by example: 800 Python combinations (crate shape × description shape × target × `line-length` 88–200) and 44 TypeScript files against the real prettier, all clean; the docstring fidelity contract holds across every description shape including the malicious fixture, carriage returns and trailing backslashes.
+
+ The two codex bot threads on the PR (`python_common.py:116` P1, `ts_zod.py:311` P2) are both confirmed-and-fixed by this round and still need answering.
+
+### D2 — docstring rendering
+
+For a description with a real newline you can have any two of {exact `__doc__` bytes, no `D301`, stable under `ruff format`}, never all three. **Ruled: render an idiomatic indented docstring**, and move the fidelity contract to the value consumers actually read:
+
+```python
+inspect.getdoc(cls) == inspect.cleandoc(description)
+```
+
+`__doc__` therefore carries the class-body indentation every hand-written docstring carries. Exact bytes still survive in `Field(description=...)` and the crate. A description that cannot sit between `"""` falls back to `'''` rather than backslash-escaping — that is what keeps it `D301`-clean, and `ruff format` rewrites `'''` to `"""` only when doing so needs no escaping, so the choice is stable. Only genuinely pathological input (both triple-quote styles, a control character, a trailing backslash) still escapes, and `D301`'s fix being unsafe means the stamp holds anyway.
+
+### D3 — line width
+
+The consumer's `line-length` is theirs, and we cannot know it. **Ruled: emit anything past 88 columns already exploded, with a magic trailing comma** — the one construct Black and ruff refuse to rejoin at any width (verified 60–200). 88 is ruff's default and therefore the tightest width a consumer is likely to use, and only the tightest threshold is safe. Short lines stay flat, so ordinary artifacts read unchanged and the "ejectable, human-maintainable" bar the plan argues for is preserved.
+
+Below 88 columns a consumer can still rewrap short lines. That is the accepted, documented limit. `E501` on a single long string literal is unavoidable — you cannot wrap a string without altering the author's text — but it has no fix and is not in ruff's default rule set, so it can never break a stamp.
+
+### Verify it locally
+
+Build the rich method from the [reproduction](#reproduce-it-2-minutes-from-this-worktree) below, then lint it **the way a consumer would** — that isort override is not optional, see the trap right after:
+
+```bash
+CONSUMER=(--config pyproject.toml --config "lint.isort.known-third-party=['pipelex']" --config "lint.per-file-ignores={'*'=['INP001']}")
+for t in python-structures python-pydantic; do
+ .venv/bin/pipelex codegen types --target $t -o /tmp/lintcheck/$t /tmp/lintcheck/method
+ .venv/bin/ruff check /tmp/lintcheck/$t "${CONSUMER[@]}" # → All checks passed!
+ .venv/bin/ruff format --check /tmp/lintcheck/$t --config pyproject.toml # → already formatted
+ .venv/bin/ruff check /tmp/lintcheck/$t --fix "${CONSUMER[@]}" && .venv/bin/ruff format /tmp/lintcheck/$t --config pyproject.toml
+ .venv/bin/pipelex codegen check /tmp/lintcheck/$t # → up to date (the stamp survived)
+done
+```
+
+⚠ **Do not drop the `known-third-party` override when checking `python-structures`.** Under *this repo's* bare config `pipelex` is first-party, so ruff reorders the generated imports, the bytes change, and `codegen check` reports `[hand-edited]` — which looks exactly like the bug still being live. It is not: it is the both-trees isort tension of deliberate item 1, and the emitter deliberately targets the consumer. Verified both ways. `python-pydantic` imports no `pipelex`, so it is config-independent and its stamp survives even under the bare config.
+
+For the empty case, point it at any bundle that declares no concepts of its own:
+
+```bash
+.venv/bin/pipelex codegen types --target python-structures -o /tmp/emptycheck tests/data/packages/standalone_bundle
+cat /tmp/emptycheck/structures.py # stamp + header, nothing else
+.venv/bin/ruff check /tmp/emptycheck --fix --config pyproject.toml && .venv/bin/ruff format /tmp/emptycheck --config pyproject.toml
+.venv/bin/pipelex codegen check /tmp/emptycheck # → up to date
+```
+
+A header-only artifact has no imports for isort to reorder, so this one is clean under any config. All commands above were run against `c9a7238e4` and produce the stated output.
+
+## What the plan did not anticipate (found while implementing)
+
+1. **The isort grouping for `python-structures` cannot satisfy both contexts.** It imports from *both* `pydantic` and `pipelex`. In a consumer's tree `pipelex` is an installed dependency (third-party — merged group, `pipelex` before `pydantic`); in this repo it is first-party (separate group, after `pydantic`). Verified empirically both ways. **Resolution: emit for the consumer** — they are the only ones who lint generated artifacts, and this repo commits none. The regression test lints with `lint.isort.known-third-party=['pipelex']` for that reason, and `render_import_block` carries the rationale.
+2. **`D301` is auto-fixed and byte-changing, and a plain double quote in a description triggered it.** `escape_py_string` backslash-escaped `"` into the docstring, so a description as ordinary as `The "primary" thing` produced `"""The \"primary\" thing"""` → `ruff check --fix` rewrites to `r"""` → stamp broken *and* text corrupted. Fixed with minimal, mode-aware docstring escaping. ⚠ **Superseded by [D2](#d2--docstring-rendering) in round 2** — the fidelity claim originally recorded here (exact description↔`__doc__` bytes) turned out to be incompatible with formatter stability for a multi-line description, and the contract is now `inspect.getdoc(cls) == inspect.cleandoc(description)`.
+3. **The imprecision caveat was emitted as a literal `\n`**, not a paragraph break — the multi-line docstring intent was defeated.
+4. **`"Foo" | None` is a `TypeError`** in the runtime generator: it has no `from __future__ import annotations`, so concept refs are *quoted* forward references and `str` has no `|`. The union is folded inside the quotes. Only a bare quoted ref is affected; `list["Foo"] | None` is a real `GenericAlias`.
+5. **The respelling silently disabled concept cycle detection.** `_detect_concept_cycles` walks the generated model's annotations and branched on `__origin__`, which a PEP 604 `X | None` (`types.UnionType`) does not have — so every optional concept reference vanished from the graph. The pre-existing TODO on that function warned about exactly this coupling. Fixed by branching on `__args__`; the 6 integration tests in `test_concept_to_concept_references.py` are what caught it.
+6. **`ts-zod` does have the analogous collision** (Phase 4's open question): prettier reformatted both emitted files. Two causes — the Python two-blank-line idiom (prettier always collapses, config-independent) and the always-wrapped binder import (prettier keeps it inline while it fits its print width). Both fixed.
+7. **The empty projection was the same bug in miniature** (raised in review, fixed on this branch). Every artifact assembles as `header + imports + separator + blocks`, which degenerates when `blocks` is empty: the import block has nothing left to use it (unused-import, which `ruff check --fix` deletes) sitting above a trailing blank-line run (`ruff format` collapses it), and `binder.ts` bottoms out at `import { } from "./types";`. **It is not a degenerate-input-only case:** `python-structures` filters natives, so an ordinary method declaring no concepts of its own — a `Text -> Text` pipe — reaches it. Resolution: an empty projection is its **header alone**, assembled in one place for Python (`python_module_body`, which also absorbed the body-assembly line both emitters duplicated) and guarded inline at the two `ts_zod` call sites.
+
+Also: adding `runtime-evaluated-base-classes` to this repo retired two hand-written `# noqa: TC001` / `# noqa: TC003` suppressions that were working around the same rule file by file.
+
+---
+
+**Original plan below, kept for the reasoning and the rejected alternatives.**
+
+**Where to work:** this worktree (`pipelex/`, on `dev`). Branch off `dev` — e.g. `fix/Codegen-lint-clean` — and open a normal PR back to `dev`. Nothing here is specific to any feature track.
+
+**Decision already taken (Louis, 2026-07-29):** emit what our linter wants, and code that matches our Python standards. The two turn out to be the same thing for every rule in play. The alternatives (hash a normalized body; tell consumers to exclude generated paths) are **rejected** — see [Why not the alternatives](#why-not-the-alternatives).
+
+---
+
+## The problem in one paragraph
+
+`pipelex codegen types` writes a stamped artifact whose stamp is a raw SHA-256 over the body's UTF-8 bytes (`compute_content_hash`, `pipelex/codegen/stamp.py:80`). The emitted body is not what `ruff` wants, so the moment a consumer runs `ruff check --fix` or `ruff format` over their tree, the bytes change, the hash no longer matches, and `pipelex codegen check` reports the file as **hand-edited** — accusing the user of the one thing they did not do. It is silent until someone runs `codegen check`. Our own `pipelex-starter-python` already papers over this with an exclusion (see [Blast radius](#blast-radius)), which is the tell that the emitter, not the consumer, is wrong.
+
+## Reproduce it (2 minutes, from this worktree)
+
+```bash
+mkdir -p /tmp/lintcheck/method && cat > /tmp/lintcheck/method/rich.mthds <<'EOF'
+domain = "lintcheck"
+description = "Exercises every emitted annotation kind"
+main_pipe = "make_record"
+
+[concept.Record]
+description = "A record with many field kinds"
+
+[concept.Record.structure]
+name = { type = "text", description = "Name", required = true }
+published = { type = "date", description = "Publication date", required = true }
+tags = { type = "list", item_type = "text", description = "Tags", required = true }
+counts = { type = "dict", key_type = "text", value_type = "integer", description = "Counts", required = true }
+status = { choices = ["draft", "final"], description = "Status", required = true }
+note = { type = "text", description = "Optional note" }
+
+[pipe.make_record]
+type = "PipeLLM"
+description = "Build a record"
+inputs = { topic = "Text" }
+output = "Record"
+model = "$writing-factual-cheap"
+prompt = "Make a record about $topic"
+EOF
+
+for t in python-structures python-pydantic; do
+ .venv/bin/pipelex codegen types --target $t -o /tmp/lintcheck/$t /tmp/lintcheck/method
+ echo "--- $t ---"
+ .venv/bin/ruff check /tmp/lintcheck/$t --config pyproject.toml
+ .venv/bin/ruff format --check /tmp/lintcheck/$t --config pyproject.toml
+done
+```
+
+Then confirm the stamp actually dies:
+
+```bash
+.venv/bin/ruff check /tmp/lintcheck/python-structures --fix --config pyproject.toml
+.venv/bin/pipelex codegen check /tmp/lintcheck/python-structures
+# → [hand-edited] structures.py — Body was edited below the stamp (stamp hash no longer matches).
+```
+
+⚠ **Use a method with varied field kinds, like the one above.** A method whose fields are all `text` makes `python-pydantic` look clean and makes `ruff format` look harmless. Both impressions are wrong, and both were believed at first.
+
+## What actually fires — measured, not assumed
+
+| rule | what it does | `python-structures` | `python-pydantic` | fix belongs in |
+| --- | --- | --- | --- | --- |
+| `UP006` | `List[X]`/`Dict[K,V]` → `list[X]`/`dict[K,V]` | ✅ fires | — (already builtin) | **emitter** |
+| `UP045` | `Optional[X]` → `X \| None` | ✅ fires | — (already `\| None`) | **emitter** |
+| `Q000` | `Literal['a','b']` → `Literal["a","b"]` | ✅ fires | ✅ fires | **emitter** |
+| `I001` | import block regrouped (stdlib / third-party / first-party, blank lines between) | ✅ fires | ✅ fires | **emitter** |
+| `ruff format` | would reformat | ✅ reformats | ✅ reformats | **emitter** |
+| `TC003` | move `from datetime import date` into an `if TYPE_CHECKING:` block | ✅ fires | ✅ fires | **NOT the emitter — see below** |
+| `INP001` | "implicit namespace package, add `__init__.py`" | fires in a bare dir | fires in a bare dir | neither — artifact of linting a loose directory, not of file content |
+
+**Both Python targets are affected.** `python-pydantic` is *closer* — it already uses builtin generics and `X | None` — but it is not clean.
+
+### `TC003` must not be satisfied by the emitter
+
+Moving the import into `if TYPE_CHECKING:` would **break the generated code**: pydantic resolves annotations at runtime to build validators, so `date` must be a real runtime import. Ruff has a first-class setting for exactly this, and it silences `TC003` for both targets — verified:
+
+```toml
+[tool.ruff.lint.flake8-type-checking]
+runtime-evaluated-base-classes = [
+ "pydantic.BaseModel",
+ "pipelex.core.stuffs.structured_content.StructuredContent",
+]
+```
+
+This is **consumer/repo configuration**, not emitted content. It goes in our docs and our starter template, and this repo should set it too (it currently does not).
+
+## Why the two asks coincide
+
+Every emitter-side rule above moves the output *toward* `.claude/rules/python-standards.md`: lowercase builtin generics (`list[]`, `dict[]`), `X | None` over `Optional[X]`, double quotes, sorted-and-grouped imports. There is no tension between "what ruff wants" and "our standards" here — satisfying one satisfies the other.
+
+## Decisions to settle before coding
+
+- **D1 — SETTLED (Louis, 2026-07-29): YES, change it too.** Does the runtime `StructureGenerator` change spelling too? `pipelex/core/concepts/structure_generation/generator.py:296` emits `Optional[{python_type}]`, and `python_structures.py`'s module docstring justifies its own spelling as mirroring that "runtime idiom". **The runtime generator's output is never written to disk** — `concept_factory.py` is its only caller and it `exec()`s the source — so no linter ever sees it and it is under no pressure to change.
+ **Recommended: change it too.** It costs a handful of test updates (`tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator*.py` pin `Optional[`), it keeps the two paths spelled alike so the emitter's docstring stays true, and it brings the runtime generator in line with the repo's own standards. If D1 goes the other way, the `python_structures.py` docstring **must** be rewritten to say the AOT projection deliberately diverges because only it is linted — do not leave a false rationale in place.
+- **D2 — one PR or two?** Recommended: **one**. The emitter change and the runtime-generator change are independently correct but share the test-update surface and the changelog entry.
+
+## Phases
+
+### Phase 1 — make the emitters emit it
+
+- [ ] 1.1 `pipelex/codegen/emitters/python_structures.py`: in `_annotation`, `List[...]` → `list[...]` and `Dict[str, ...]` → `dict[str, ...]`, dropping the `from typing import List` / `Dict` additions; in `_render_field`, `Optional[X]` → `X | None`, dropping the `from typing import Optional` addition.
+- [ ] 1.2 `Literal` choices in **both** Python emitters: emit double-quoted members (`repr()` yields single quotes today — use an explicit double-quoting helper, and put it in `python_common.py` since both targets need it).
+- [ ] 1.3 Import block in **both** emitters: `"\n".join(sorted(imports))` sorts raw strings, which is not isort order. Emit isort-grouped blocks — stdlib, third-party, first-party — separated by a blank line. Consider a small shared `render_import_block()` in `python_common.py`; both call sites are identical today.
+- [ ] 1.4 Re-run the reproduction. `ruff check` must report only `INP001` (the loose-directory artifact) and `ruff format --check` must report "already formatted", for both targets, on the rich method.
+- [ ] 1.5 Update the `python_structures.py` module docstring — the "runtime idiom" sentence is now either satisfied by D1 or falsified by it.
+
+### Phase 2 — the runtime generator (gated on D1 = yes)
+
+- [ ] 2.1 `structure_generation/generator.py`: same three spellings (`Optional` → `| None`, `List`/`Dict` → builtins), and drop the now-unused `typing` imports from its own import line.
+- [ ] 2.2 Update `tests/unit/pipelex/core/concepts/structure_generation/test_structure_generator*.py` for the new expected source text.
+
+**🛑 CHECKPOINT 1** — emitters and generator produce the new bytes, and every existing test is updated to match. Gates: `make agent-check` + full `make agent-test`. Do not start phase 3 in the same session; the test-expectation churn is the bulk of the work and is worth landing on its own.
+
+### Phase 3 — the regression test that would have caught this
+
+- [ ] 3.1 Add `tests/unit/pipelex/codegen/test_emitted_artifacts_are_lint_clean.py`: generate a projection covering **every** `ResolvedTypeKind` (text, number, integer, boolean, date, datetime, time, literal, concept, list, dict, any) plus an optional field and an opaque/structureless concept, for **each** Python target, then run `ruff check` and `ruff format --check` over it with this repo's own config and assert zero findings.
+- [ ] 3.2 Suppress the two non-content rules in the test's invocation rather than in the emitted bytes: `INP001` (loose directory) and `TC003` (needs `runtime-evaluated-base-classes`, which the test should pass so it exercises the configuration we tell consumers to use).
+- [ ] 3.3 Make the fixture method the **single source** for the type-kind coverage, so a new `ResolvedTypeKind` that nobody adds to it is visible. A test asserting the fixture covers every enum member is cheap and worth it.
+
+### Phase 4 — the TypeScript target
+
+- [ ] 4.1 Answer the open question: does `ts-zod` have the analogous collision with prettier / eslint? Nothing has checked. If it does, fix it the same way and extend the phase-3 test.
+
+**🛑 CHECKPOINT 2** — the property is guarded for every target, not just repaired once.
+
+### Phase 5 — consumers and docs
+
+- [ ] 5.1 Set `runtime-evaluated-base-classes` in **this** repo's `pyproject.toml` (`[tool.ruff.lint.flake8-type-checking]`) — we do not set it today.
+- [ ] 5.2 Document the consumer contract: generated artifacts are lint-clean by construction, here is the one ruff setting you need, and you should **no longer** need to exclude generated paths.
+- [ ] 5.3 CHANGELOG: emitted bytes change for both Python targets. Breaking for anyone with a committed projection — their next `codegen check` reports drift until they regenerate. Say so plainly and give the one-line regeneration command.
+- [ ] 5.4 Cross-repo sweep, **release-gated** (needs the new `pipelex` released first):
+ - `pipelex-starter-python/piper/generated/` — three committed `python-pydantic` projections (`summarize_pdf`, `extract_entities`, `generate_image`), all stamped `engine_version = "0.38.0"`. Regenerate, then **delete the `piper/generated` exclusion** from its `[tool.ruff] exclude` — that line, comment and all, is the bug's monument and should not survive this change.
+ - `pipelex-cookbook/` — grep for `codegen.lock`; none as of 2026-07-29, but check again at sweep time.
+
+**🛑 CHECKPOINT 3 = done** — no consumer needs a lint exclusion to keep a stamp valid.
+
+## Blast radius
+
+- **Emitted bytes change for both Python targets.** Anyone holding a committed projection sees `codegen check` report drift until they regenerate. That is the intended, one-time cost.
+- **`test_pure_build_and_disk_write_agree_byte_for_byte` does NOT need re-baselining.** It is a self-consistency check (pure build vs. disk write) over synthetic fixture files, not a golden-file test. Only tests that pin *expected emitted text* need updating — `tests/unit/pipelex/codegen/test_python_structures_emitter.py`, the `Literal`-quoting assertions in the pydantic/ts emitter tests, and (under D1) the runtime-generator tests.
+- **`pipelex-starter-python` is a live instance of the bug.** Its `pyproject.toml` carries `"piper/generated", # generated by \`pipelex codegen\` — reformatting would trip the codegen.lock drift check`. We shipped the workaround in our own template rather than fixing the emitter.
+- **Not affected:** the crate fingerprint, the lock format, `codegen check`'s logic, and the HTTP codegen route's contract. This changes only what the emitters write.
+
+## Why not the alternatives
+
+- **Hash a normalized body.** Weakens the stamp from "byte-identical" to "semantically unchanged", and the normalizer becomes its own compatibility surface that has to agree across versions and languages. It also leaves the emitted code below the ejectability bar — code the project's own linter wants to rewrite on sight is not code we can call idiomatic and human-maintainable.
+- **Tell consumers to exclude generated paths.** This is today's de-facto answer and it is why the bug survived: the exclusion is invisible, it must be copied into every consumer repo, and anyone who misses it gets an error message accusing them of hand-editing. It also does nothing for the quality bar.
+
+## Cold-start brief
+
+Read this file top to bottom, then:
+
+1. `cd` to this worktree (`pipelex/`, on `dev`), `git pull`, `make install` if the venv is stale.
+2. Run the [reproduction](#reproduce-it-2-minutes-from-this-worktree) to see the failure with your own eyes and confirm the rule table still matches.
+3. Settle **D1** with Louis if it is not already recorded here, then start at Phase 1.
+4. Branch off `dev` first — do not work on `dev` directly for this.
+
+The two emitters are near-identical twins (`pipelex/codegen/emitters/python_structures.py` and `python_pydantic.py`, ~110 lines each) sharing `python_common.py` and the neutral `resolve_structure_fields` layer. Read all three before editing; most of Phase 1 is deciding what belongs in the shared module.