Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
Say which layer catches a fault first, and lint the entries that answer runtime (#1143)
Louis Choquelb0f27622fe82
diff --git a/.pipelex/plxt.toml b/.pipelex/plxt.tomlindex 7a53ce931..2b8425c74 100644--- a/.pipelex/plxt.toml+++ b/.pipelex/plxt.toml@@ -24,14 +24,24 @@ exclude = ["node_modules/**",".git/**","*.lock",- # The MTHDS Test Corpus's deliberately invalid entries — the one .mthds population this repo does- # not lint, because each one exists to be rejected. Their fault is sometimes structural (a typeless- # pipe section, an unknown pipe `type`), which the schema catches as the error the entry was written- # to trigger. What guards them instead is strictly stronger: the corpus's entry-validation gate runs- # the real validation engine over every entry in CI and is red both when an invalid entry fails- # differently and when it accidentally validates. Excluded rather than schema-exempted because a- # per-rule `exclude` does not lift the schema, only narrows which rule applies.- "**/mthds_corpus/entries/invalid_*/*.mthds",+ # The MTHDS Test Corpus's schema-fault entries, and ONLY those. Every other deliberately invalid+ # entry is linted like any other .mthds file in this repo: its fault needs the runtime to notice,+ # so the schema has nothing to say about it and a clean lint is the correct outcome. The two below+ # are the exceptions — their fault is structural, so the schema rejects them as the very error they+ # were written to trigger, and linting them would report a defect that is the entry's whole point.+ #+ # This list is not hand-maintained: it is gated against the corpus vocabulary's `fails_at` signal by+ # tests/unit/pipelex/test_extras/test_mthds_corpus_plxt_exclusions.py, which fails if the two+ # disagree. That is also what makes pipelex the signal's first consumer, and it closes the loop in+ # the direction that matters — an error type mis-declared `runtime` leaves its entry linted, and+ # `make plxt-lint` goes red naming it.+ #+ # Excluded rather than schema-exempted because a per-rule `exclude` does not lift the schema, only+ # narrows which rule applies. What guards these two instead is strictly stronger: the corpus's+ # entry-validation gate runs the real validation engine over every entry in CI and is red both when+ # an invalid entry fails differently and when it accidentally validates.+ "**/mthds_corpus/entries/invalid_missing_pipe_type/*.mthds",+ "**/mthds_corpus/entries/invalid_unknown_pipe_type/*.mthds",] # Glob patterns for files to ignore.# These are evaluated relative to the config file location.diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 77aa3662f..c192ddd2c 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -1,5 +1,25 @@# Changelog+## [Unreleased]++### Added++- **Every `error.*` tag in the MTHDS Test Corpus vocabulary now says which layer of checking catches its fault first.** A non-excluded tag carries `fails_at = "schema"` when a pass over the raw document's shape already rejects a bundle carrying the fault — a pipe section with no `type` key, a `type` outside the closed set of pipe kinds — and `fails_at = "runtime"` when the document has to be interpreted to notice, which covers every other fault the corpus exercises. The consumer rule is one sentence: **a structural sweep expects a diagnostic on an entry exactly when its tag says `schema`, and silence on every other entry.**++ This closes a hole that had every consumer re-deriving the runtime's own knowledge. A JSON-Schema sweep, an editor diagnostic, or a linter run over the corpus previously had to hardcode which faults it believed were structural — a second, downstream reading of this repo's validation-error registry, guaranteed to drift from it. The corpus being swept now carries the answer, and it ships in the wheel, so a consumer that vendors `vocabulary.toml` gets it with no code of its own.++ The signal lives on the vocabulary tag rather than on each `entry.toml`, because it is a property of the *error type*, not of the entry: per-entry restatement would let two entries covering one fault disagree, with nothing to catch it. A `schema` fault is still rejected by the runtime — the field names where a fault is caught first, not who may catch it — which is why a schema-fault entry declares the same `expected_error` as any other invalid entry. The value `schema` is also `plxt`'s own `error[schema]` diagnostic category, so a consumer branching on the field and a human reading a linter's output use one word for one thing.++- **The exhaustivity gate now requires every `error.*` tag that is owed an entry to declare a `fails_at`**, so a new validation error type cannot land carrying a fault a structural consumer has no way to sweep for. Excluded tags carry no value, deliberately: an excluded tag has no entry, so there is nothing to have measured on.++### Changed++- **The corpus's deliberately invalid entries are linted again — all but the two whose fault the schema is supposed to reject.** `.pipelex/plxt.toml` excluded the whole `invalid_*` population behind one blanket glob, because some of those entries carry a structural fault and the config had no way to say which. `fails_at` is that way, so the exclusion now names exactly the schema-fault entries and every other invalid bundle is linted like any ordinary `.mthds` file in the tree.++ That also makes this repo the first consumer of its own signal, and it is what keeps the signal *measured* rather than merely asserted: the exclusion list is gated against the vocabulary by `test_mthds_corpus_plxt_exclusions.py`, so declaring a fault `runtime` when the schema in fact rejects it leaves its entry linted and turns `make plxt-lint` red naming it. The config is gated rather than generated because `plxt` is a static binary reading a static file, with nothing to hook a generation step onto.++- **Documentation**: `docs/contribute/mthds-test-corpus.md` gains a `fails_at` section covering the consumer rule, why the field sits on the tag, and how the linter config closes the measurement loop. Its invalid-entry authoring warning is corrected to match the narrowed exclusion: only a schema-fault bundle has to be laid out by hand now, since `make lint` and `make format` cover every other invalid entry.+## [v0.50.0] - 2026-08-20### Changeddiff --git a/docs/contribute/mthds-test-corpus.md b/docs/contribute/mthds-test-corpus.mdindex 337d8bc15..bb55c04a3 100644--- a/docs/contribute/mthds-test-corpus.md+++ b/docs/contribute/mthds-test-corpus.md@@ -70,7 +70,7 @@ Two rules shape an invalid entry beyond that. Its **`covers` list is its `error.The `pipelex` plugin's `PostToolUse` hook lints, formats and **blocks on an invalid verdict** for every `Write`/`Edit` of a `.mthds` file, and has no skip mechanism. Author valid entries with the editing tools to collect the free lint; write an invalid entry's bundle through a shell heredoc, which the hook's matcher does not intercept.- An invalid entry's bundle is outside `make lint` and `make format` as well — `.pipelex/plxt.toml` excludes the whole `invalid_*` population, for the reason recorded beside that glob. So nothing formats it for you: match the layout of the entry next door by hand. Its `entry.toml` is still linted normally.+ `make lint` and `make format` cover an invalid entry's bundle like any other `.mthds` file in the tree, with one carve-out: `.pipelex/plxt.toml` excludes the corpus's **schema-fault** entries — those whose tag says `fails_at = "schema"` — because a linter reporting them would be reporting the very defect the entry exists to carry, and that list is gated against the vocabulary rather than hand-maintained (see [`fails_at`](#fails_at-which-layer-catches-a-fault-first)). Nothing formats those, so match the layout of the entry next door by hand; every other invalid bundle is formatted for you. Its `entry.toml` is linted normally either way.## The tag vocabulary@@ -94,12 +94,25 @@ No native concept is excluded — every one of them turned out to support a real**Measure an exclusion, do not reason your way to one.** Every `error.*` exclusion reason above was established by writing the bundle that ought to trigger it and reading what the runtime actually said — which is how the first two turned out to be shadowed rather than merely awkward, and how the two the registry made look unreachable (`missing_pipe_type`, `native_concept_redeclaration`) turned out to have entries after all.+### `fails_at` — which layer catches a fault first++An `error.*` tag that is *not* excluded carries one more field: `fails_at`, either `schema` or `runtime`. It names the earliest layer of checking that rejects a bundle carrying the fault. `schema` means a pass over the raw document's shape already catches it — a section missing a required key, a value outside a closed set — so a JSON-Schema validator, an editor diagnostic, or `plxt lint` reports it without ever interpreting the document. `runtime` means the document has to be *interpreted* to notice: an unresolved reference, an input the flow never supplies, an output whose concept does not fit.++**The consumer rule is one sentence: a structural sweep expects a diagnostic on an entry exactly when its tag says `fails_at = "schema"`, and expects silence on every other entry.** That is what the field exists for. Before it, a consumer running a schema checker over the corpus had to hardcode which faults it thought were structural — a second, downstream reading of `pipelex`'s own registry, guaranteed to drift from it. Now the corpus being swept carries the answer.++A `schema` fault is still rejected by the runtime; the field names where a fault is caught *first*, not who is allowed to catch it. That is why a schema-fault entry declares the same `expected_error` as any other invalid entry — the runtime's diagnostic is what the entry pins. The two spellings line up on purpose, too: `schema` is also `plxt`'s own `error[schema]` diagnostic category, so a consumer branching on the field and a human reading a linter's output use one word for one thing.++Only non-excluded tags carry it, for the same reason exclusion reasons are measured: an excluded tag has no entry, so there is nothing to have measured on, and a value invented for one would be an argument dressed as a measurement.++**And it stays measured, because this repo consumes it first.** `.pipelex/plxt.toml` excludes the corpus's schema-fault entries from `plxt lint` — they *should* produce a diagnostic, which is the entry's whole point — and lints every other invalid entry like any ordinary `.mthds` file. That exclusion list is not hand-maintained: `tests/unit/pipelex/test_extras/test_mthds_corpus_plxt_exclusions.py` fails when it disagrees with the vocabulary. The config is gated rather than generated because `plxt` is a static binary reading a static file, with nothing to hook a generation step onto — but the loop still closes in the direction that matters. Declare a fault `runtime` when the schema in fact rejects it, and its entry stays linted, so `make plxt-lint` goes red naming it.+## The gates, and what a red one means| Gate | Where | Red means ||---|---|---|| Vocabulary drift | `tests/unit/pipelex/test_extras/test_mthds_corpus_vocabulary.py` | A registry changed and the committed vocabulary was not regenerated. Run `make generate-corpus-vocabulary`. |-| Exhaustivity | `tests/unit/pipelex/test_extras/test_mthds_corpus_exhaustivity.py` | A vocabulary tag has no `focused` entry covering it — write the entry, or record an exclusion with a reason. Also fires when an entry covers a tag that is not in the vocabulary, when an invalid entry's `covers` is not exactly the tag its `expected_error` names, and when a valid entry covers an `error.*` tag it cannot produce. |+| Exhaustivity | `tests/unit/pipelex/test_extras/test_mthds_corpus_exhaustivity.py` | A vocabulary tag has no `focused` entry covering it — write the entry, or record an exclusion with a reason. Also fires when an entry covers a tag that is not in the vocabulary, when an invalid entry's `covers` is not exactly the tag its `expected_error` names, when a valid entry covers an `error.*` tag it cannot produce, and when a required `error.*` tag declares no `fails_at`. |+| Linter exclusions | `tests/unit/pipelex/test_extras/test_mthds_corpus_plxt_exclusions.py` | `.pipelex/plxt.toml`'s corpus exclusions disagree with the vocabulary's `fails_at` signal — an entry whose fault the schema rejects must be excluded, every other invalid entry must be linted. || Manifest | `tests/unit/pipelex/test_extras/test_mthds_corpus_manifest.py` | The strict `entry.toml` model rejected something. || Layout and filters | `tests/unit/pipelex/test_extras/test_mthds_corpus_loader.py` | An entry's name does not match its directory, its bundle does not resolve, or the loader's filter semantics changed. || Entry validation | `tests/integration/pipelex/test_extras/test_mthds_corpus_entries.py` | A valid entry stopped validating, or an invalid one stopped failing with exactly its declared error. Runs over every entry regardless of tier, so an `inference`-tier entry that broke is caught without spending a token. |diff --git a/pipelex/cli/dev_cli/commands/generate_corpus_vocabulary_cmd.py b/pipelex/cli/dev_cli/commands/generate_corpus_vocabulary_cmd.pyindex c71b56787..915688767 100644--- a/pipelex/cli/dev_cli/commands/generate_corpus_vocabulary_cmd.py+++ b/pipelex/cli/dev_cli/commands/generate_corpus_vocabulary_cmd.py@@ -20,7 +20,7 @@ from rich.panel import Panelfrom pipelex.core.concepts.native.concept_native import NativeConceptCodefrom pipelex.pipe_machinery.pipe_blueprint import PipeCategory, PipeTypefrom pipelex.runtime_hub import get_console-from pipelex.test_extras.mthds_corpus.vocabulary import vocabulary_path+from pipelex.test_extras.mthds_corpus.vocabulary import ValidationLayer, vocabulary_pathfrom pipelex.validation_error_types import VALIDATION_ERROR_TYPES, PipeFactoryErrorType, PipeValidationErrorType, ValidationErrorType_ACRONYM_BOUNDARY_RE = re.compile(r"(?<=[A-Z0-9])(?=[A-Z][a-z])")@@ -33,6 +33,11 @@ _HEADER = """# The MTHDS Test Corpus tag vocabulary — the closed set an entry'# `feature.*` is hand-maintained in the generator because no registry describes a language feature,# and every exclusion reason is declared in the generator's source alongside them.#+# An `error.*` tag that is not excluded also carries `fails_at`, the earliest layer of checking that+# rejects a bundle carrying the fault: `schema` when a check of the document's shape alone already+# catches it, `runtime` when the document has to be interpreted to notice. A structural consumer+# expects a diagnostic on exactly the `schema` entries and none on the others.+## This file ships in the pipelex wheel, so it names a document its readers can reach:# https://docs.pipelex.com/contribute/mthds-test-corpus/ — which in turn points at the# cross-repo contract, `docs/specs/mthds-test-corpus.md` in the Pipelex workspace repo.@@ -156,6 +161,41 @@ _ERROR_TYPE_EXCLUSIONS: dict[ValidationErrorType, str] = {}+# `fails_at` says, for a fault a corpus entry can produce, which layer of checking rejects it FIRST.+# The two layers are the two ways a `.mthds` document gets read: a JSON-Schema pass over the raw+# document, which sees only shape, and the runtime, which interprets what the shape means. A schema+# check catches a section missing a required key or holding a value outside a closed set; everything+# else — an unresolved reference, an input the flow never supplies, an output whose concept does not+# fit — needs interpretation and belongs to the runtime.+#+# The set below is the `schema` half, and it is short by nature: only a fault visible in the shape+# alone can land in it. Every other non-excluded error type is `runtime`, so this set is the whole+# declaration and there is no second list to keep in step with it. An excluded type gets no+# `fails_at` at all — it has no entry, so there is nothing to have measured, and a value invented for+# one would be an argument dressed as a measurement.+#+# **This is measured, not argued, and CI is what keeps it measured.** The list was established by+# linting every invalid entry against the generated MTHDS JSON Schema: exactly the members below+# produce a diagnostic, every other invalid entry passes. That measurement is then held in place by+# `.pipelex/plxt.toml`, whose corpus exclusions are gated against this set — so the invalid entries+# NOT listed here are linted on every `make plxt-lint` run. Declare a fault `runtime` when the schema+# in fact rejects it and the lint goes red naming the entry; the loop closes without anyone+# remembering to re-measure.+#+# Keyed on the enums for the same reason the exclusion maps are: a code that leaves the registry+# breaks this module at import instead of leaving a stale value behind in the generated file.+_ERROR_TYPE_SCHEMA_FAULTS: frozenset[ValidationErrorType] = frozenset(+ {+ # A `[pipe.x]` section carrying implementation fields but no `type` key, which the schema+ # requires on any pipe that is not a bare signature.+ PipeValidationErrorType.MISSING_PIPE_TYPE,+ # A `type` naming a pipe kind that does not exist, which the schema's closed enum of pipe+ # kinds rejects on sight.+ PipeValidationErrorType.UNKNOWN_PIPE_TYPE,+ }+)++def normalize_registry_code(*, code: str) -> str:"""Turn a CamelCase registry code into a snake_case tag local name.@@ -192,12 +232,30 @@ def _render_feature_namespace() -> str:def _render_error_namespace() -> str:+ """The `error.*` namespace: every registry code, each either excluded or carrying a `fails_at`.++ The two are mutually exclusive by construction, and a type in both maps is a contradiction rather+ than a precedence question — an exclusion says no entry exists to measure, while a `fails_at`+ claims a measurement was taken on one. Raising routes it through the command's own failure path+ instead of quietly emitting whichever branch happens to come first.+ """+ contradictions = sorted(error_type.value for error_type in _ERROR_TYPE_SCHEMA_FAULTS if error_type in _ERROR_TYPE_EXCLUSIONS)+ if contradictions:+ msg = (+ f"Error types are declared both excluded and schema faults: {', '.join(contradictions)}. "+ "An excluded type has no corpus entry, so no schema measurement exists for it."+ )+ raise ValueError(msg)+blocks: list[str] = []for error_type in VALIDATION_ERROR_TYPES:lines = [f"[error.{normalize_registry_code(code=error_type.value)}]", f"code = {_toml_string(value=error_type.value)}"]exclusion_reason = _ERROR_TYPE_EXCLUSIONS.get(error_type)if exclusion_reason is not None:lines.append(f"excluded = {_toml_string(value=exclusion_reason)}")+ else:+ fails_at = ValidationLayer.SCHEMA if error_type in _ERROR_TYPE_SCHEMA_FAULTS else ValidationLayer.RUNTIME+ lines.append(f"fails_at = {_toml_string(value=fails_at)}")blocks.append("\n".join(lines))return "\n\n".join(blocks)diff --git a/pipelex/test_extras/mthds_corpus/vocabulary.py b/pipelex/test_extras/mthds_corpus/vocabulary.pyindex 2800ae191..e17e5a75a 100644--- a/pipelex/test_extras/mthds_corpus/vocabulary.py+++ b/pipelex/test_extras/mthds_corpus/vocabulary.py@@ -6,6 +6,7 @@ committed, so a consumer reading it needs neither the generator nor a registry wmodule is the reader; the generator lives in the dev CLI, which the wheel excludes."""+from enum import StrEnumfrom pathlib import Pathfrom pydantic import BaseModel, ConfigDict@@ -15,6 +16,38 @@ from pipelex.tools.misc.toml_utils import load_toml_from_pathVOCABULARY_FILE_NAME = "vocabulary.toml"+ERROR_NAMESPACE = "error"+++class ValidationLayer(StrEnum):+ """The layer of checking a fault is caught at — the value of a tag's ``fails_at``.++ The two layers are ordered, and ``fails_at`` names the **earliest** one that rejects a bundle+ carrying the fault. ``SCHEMA`` is what a JSON-Schema check over the raw document already+ catches, because the fault is structural: a section that is missing a required key, or one+ whose value is outside a closed set. ``RUNTIME`` is everything a document has to be+ *interpreted* to notice — an unresolved reference, an input the flow never provides, an+ output whose concept does not fit.++ ``SCHEMA`` faults are rejected by the runtime too. The layers name where a fault is caught+ first, not who is allowed to catch it, which is why a schema-fault entry still declares the+ ``expected_error`` the runtime reports for it. The spelling matches plxt's own+ ``error[schema]`` diagnostic category, so a consumer branching on this field and a human+ reading a linter's output are using one word for one thing.+ """++ SCHEMA = "schema"+ RUNTIME = "runtime"++ @property+ def is_schema(self) -> bool:+ """True for the layer a check of the document's shape alone already rejects."""+ match self:+ case ValidationLayer.SCHEMA:+ return True+ case ValidationLayer.RUNTIME:+ return False+class VocabularyTag(BaseModel):"""One tag in the vocabulary.@@ -26,11 +59,19 @@ class VocabularyTag(BaseModel):can live. ``excluded`` carries the reason a tag is not required to have a focused entry,and is present only on excluded tags.- All three fields are optional here, and no validator enforces the ``code``-or-``description``- split, deliberately. This model only *reads*: the generator is the single writer, and the- drift gate regenerates the committed file in memory and compares it byte for byte, so a tag- carrying neither cannot reach a reader through the channel this model serves. A validator- would guard a state the pipeline cannot produce.+ ``fails_at`` is the ``error.*`` namespace's own field, and it is the answer to a question+ every structural consumer was otherwise guessing at: does a check that only reads the+ document's *shape* reject this fault, or does it take the runtime to notice? A consumer+ sweeping the corpus with a schema validator expects a diagnostic on exactly the entries+ whose tag says :attr:`ValidationLayer.SCHEMA`, and expects silence on the rest — so the+ signal turns a sweep that had to hardcode a list of known-structural faults into one that+ reads the corpus it is sweeping.++ Every field is optional here, and no validator enforces which combinations are legal,+ deliberately. This model only *reads*: the generator is the single writer, and the drift+ gate regenerates the committed file in memory and compares it byte for byte, so a tag+ carrying a nonsensical combination cannot reach a reader through the channel this model+ serves. A validator would guard a state the pipeline cannot produce."""model_config = ConfigDict(extra="forbid", frozen=True)@@ -38,6 +79,7 @@ class VocabularyTag(BaseModel):code: str | None = Nonedescription: str | None = Noneexcluded: str | None = None+ fails_at: ValidationLayer | None = Noneclass CorpusVocabulary(BaseModel):@@ -55,6 +97,23 @@ class CorpusVocabulary(BaseModel):declared.update(f"{namespace}.{local_name}" for local_name in tags_in_namespace)return frozenset(declared)+ @property+ def schema_fault_tags(self) -> frozenset[str]:+ """The tags whose fault a check of the document's shape alone already rejects.++ This is the consumer rule in one call: a structural sweep — a JSON-Schema pass, an editor+ diagnostic, a linter run over the corpus — expects a diagnostic on an entry covering a tag+ in this set, and expects none on any other entry. Reading it off the shipped vocabulary is+ what keeps a consumer from re-deriving the runtime's own knowledge downstream, whether by+ hardcoding a list of structural faults or by pattern-matching tag names.+ """+ return frozenset(+ f"{namespace}.{local_name}"+ for namespace, tags_in_namespace in self.namespaces.items()+ for local_name, tag in tags_in_namespace.items()+ if tag.fails_at is not None and tag.fails_at.is_schema+ )+@propertydef required_tags(self) -> frozenset[str]:"""The tags the exhaustivity gate requires a focused entry for — every tag that is not excluded."""diff --git a/pipelex/test_extras/mthds_corpus/vocabulary.toml b/pipelex/test_extras/mthds_corpus/vocabulary.tomlindex 71bb04858..89ac417b6 100644--- a/pipelex/test_extras/mthds_corpus/vocabulary.toml+++ b/pipelex/test_extras/mthds_corpus/vocabulary.toml@@ -5,6 +5,11 @@# `feature.*` is hand-maintained in the generator because no registry describes a language feature,# and every exclusion reason is declared in the generator's source alongside them.#+# An `error.*` tag that is not excluded also carries `fails_at`, the earliest layer of checking that+# rejects a bundle carrying the fault: `schema` when a check of the document's shape alone already+# catches it, `runtime` when the document has to be interpreted to notice. A structural consumer+# expects a diagnostic on exactly the `schema` entries and none on the others.+## This file ships in the pipelex wheel, so it names a document its readers can reach:# https://docs.pipelex.com/contribute/mthds-test-corpus/ — which in turn points at the# cross-repo contract, `docs/specs/mthds-test-corpus.md` in the Pipelex workspace repo.@@ -90,18 +95,23 @@ code = "PipeSequence"[error.missing_input_variable]code = "missing_input_variable"+fails_at = "runtime"[error.extraneous_input_variable]code = "extraneous_input_variable"+fails_at = "runtime"[error.input_stuff_spec_mismatch]code = "input_stuff_spec_mismatch"+fails_at = "runtime"[error.inadequate_output_concept]code = "inadequate_output_concept"+fails_at = "runtime"[error.inadequate_output_multiplicity]code = "inadequate_output_multiplicity"+fails_at = "runtime"[error.circular_dependency_error]code = "circular_dependency_error"@@ -109,33 +119,43 @@ excluded = "Raised only from the pipe sorter, and the pipe sorter runs only on `[error.llm_output_cannot_be_image]code = "llm_output_cannot_be_image"+fails_at = "runtime"[error.invalid_pipe_code_syntax]code = "invalid_pipe_code_syntax"+fails_at = "runtime"[error.unknown_pipe_type]code = "unknown_pipe_type"+fails_at = "schema"[error.missing_pipe_type]code = "missing_pipe_type"+fails_at = "schema"[error.batch_item_name_collision]code = "batch_item_name_collision"+fails_at = "runtime"[error.optional_marker_invalid]code = "optional_marker_invalid"+fails_at = "runtime"[error.optional_not_handled]code = "optional_not_handled"+fails_at = "runtime"[error.optional_output_required]code = "optional_output_required"+fails_at = "runtime"[error.optional_input_unguarded]code = "optional_input_unguarded"+fails_at = "runtime"[error.optional_branch_required_field]code = "optional_branch_required_field"+fails_at = "runtime"[error.optional_force_redundant]code = "optional_force_redundant"@@ -143,12 +163,15 @@ excluded = "Advisory-only: it rides the validation report's `warnings` array and[error.native_concept_redeclaration]code = "native_concept_redeclaration"+fails_at = "runtime"[error.unresolved_concept]code = "unresolved_concept"+fails_at = "runtime"[error.unresolved_pipe_dependency]code = "unresolved_pipe_dependency"+fails_at = "runtime"[error.unknown_validation_error]code = "unknown_validation_error"@@ -164,6 +187,7 @@ excluded = "The generic fallback for a pipe-construction failure that matched no[error.dry_run_error]code = "DryRunError"+fails_at = "runtime"[feature.multi_file_library]description = "A method split across several `.mthds` files in one directory and merged additively behind a `bundle.mthds` entry point, so a pipe can be forward-declared as a signature in one file and satisfied by a concrete definition in another."diff --git a/tests/unit/pipelex/test_extras/test_mthds_corpus_exhaustivity.py b/tests/unit/pipelex/test_extras/test_mthds_corpus_exhaustivity.pyindex c8856ef68..17341093c 100644--- a/tests/unit/pipelex/test_extras/test_mthds_corpus_exhaustivity.py+++ b/tests/unit/pipelex/test_extras/test_mthds_corpus_exhaustivity.py@@ -13,11 +13,18 @@ fault twice in one file — once as ``expected_error``, the wire string the runtas an ``error.*`` tag in ``covers``, the normalized form the vocabulary declares. Nothing elsechecks that the two agree, so without it an entry could claim coverage of a tag it does notproduce and the exhaustivity arm above would count that claim.++The ``fails_at`` arm extends exhaustivity from tags to what a tag has to *say*. A consumer sweeping the+corpus with a structural checker branches on ``fails_at``, so a tag that is owed an entry and+declares no layer leaves that consumer with nothing to branch on — the same hole as a tag with no+entry, one level down. Keeping it here rather than with the generator's own drift gate is+deliberate: this reads the loaded vocabulary, which is what a consumer sees, so the requirement+survives a change in how the file comes to be written."""from pipelex.test_extras.mthds_corpus.loader import iter_entriesfrom pipelex.test_extras.mthds_corpus.manifest import EntryGranularity, EntryValidity-from pipelex.test_extras.mthds_corpus.vocabulary import load_vocabulary+from pipelex.test_extras.mthds_corpus.vocabulary import ERROR_NAMESPACE, load_vocabularyclass TestMthdsCorpusExhaustivity:@@ -53,7 +60,7 @@ class TestMthdsCorpusExhaustivity:tag_by_error_code = {tag.code: f"{namespace}.{local_name}"for namespace, tags_in_namespace in load_vocabulary().namespaces.items()- if namespace == "error"+ if namespace == ERROR_NAMESPACEfor local_name, tag in tags_in_namespace.items()if tag.code is not None}@@ -74,6 +81,28 @@ class TestMthdsCorpusExhaustivity:"claim for a working feature.")+ def test_every_required_error_tag_declares_the_layer_it_fails_at(self) -> None:+ """A fault a consumer is owed an entry for must also say which layer catches it first.++ Required rather than universal: an excluded ``error.*`` tag has no entry, so no measurement+ of it exists and inventing one would be an argument dressed as a measurement. The other+ namespaces name language features, not faults, and have no layer to declare at all.+ """+ vocabulary = load_vocabulary()+ error_prefix = f"{ERROR_NAMESPACE}."+ silent = sorted(+ f"{ERROR_NAMESPACE}.{local_name}"+ for local_name, tag in vocabulary.namespaces[ERROR_NAMESPACE].items()+ if tag.fails_at is None and f"{error_prefix}{local_name}" in vocabulary.required_tags+ )+ assert not silent, (+ "Required error.* tags declare no `fails_at`: "+ + ", ".join(silent)+ + ". A structural consumer expects a diagnostic on exactly the `schema` faults and none on the "+ "`runtime` ones, so a fault that names no layer is one it cannot sweep for. Declare it in the "+ "vocabulary generator's schema-fault set and regenerate with `make generate-corpus-vocabulary`."+ )+def test_no_valid_entry_claims_an_error_tag(self) -> None:"""The other half of the same hole: a bundle that validates cannot cover a fault.@@ -86,7 +115,7 @@ class TestMthdsCorpusExhaustivity:An ``error.*`` tag is a claim about a diagnostic, and only an entry that declares an``expected_error`` can honour one."""- error_namespace_prefix = "error."+ error_namespace_prefix = f"{ERROR_NAMESPACE}."offenders = [f"{entry.name}: {tag}"for entry in iter_entries(validity=EntryValidity.VALID)diff --git a/tests/unit/pipelex/test_extras/test_mthds_corpus_plxt_exclusions.py b/tests/unit/pipelex/test_extras/test_mthds_corpus_plxt_exclusions.pynew file mode 100644index 000000000..027cee3c1--- /dev/null+++ b/tests/unit/pipelex/test_extras/test_mthds_corpus_plxt_exclusions.py@@ -0,0 +1,57 @@+"""This repo's own linter config, gated against the corpus's ``fails_at`` signal.++Contract: ``docs/specs/mthds-test-corpus.md`` (workspace root), section "Tag vocabulary".++`pipelex` is the first consumer of its own signal, and `.pipelex/plxt.toml` is where it consumes+it. Every deliberately invalid corpus entry used to be excluded from `plxt lint` by one blanket+glob, because *some* of them carry a structural fault the schema rejects and the config had no way+to say which. `fails_at` is that way: the entries whose tag says `schema` stay excluded, and every+other invalid entry is linted like any ordinary `.mthds` file in the tree.++The config is gated rather than generated because `plxt` is a static binary reading a static file —+there is nothing to hook a generation step onto. What this test buys is the same guarantee in the+direction that matters: a fault mis-declared `runtime` leaves its entry linted, and `make plxt-lint`+goes red naming it, which is what keeps the signal measured instead of merely asserted.+"""++from pathlib import Path+from typing import Any++from pipelex.test_extras.mthds_corpus.loader import iter_entries+from pipelex.test_extras.mthds_corpus.manifest import EntryValidity+from pipelex.test_extras.mthds_corpus.vocabulary import load_vocabulary+from pipelex.tools.misc.toml_utils import load_toml_from_path++_REPO_ROOT = Path(__file__).resolve().parents[4]+_PLXT_CONFIG_PATH = _REPO_ROOT / ".pipelex" / "plxt.toml"++_CORPUS_GLOB_MARKER = "mthds_corpus"+++def _corpus_exclusion_globs() -> set[str]:+ """The corpus-related entries of the config's top-level `exclude` list.++ Narrowed by the marker rather than compared whole: the rest of that list is ordinary build-output+ noise (`.venv/**`, `target/**`) that has nothing to do with the corpus and should not have to be+ restated here to keep this gate green.+ """+ config: dict[str, Any] = load_toml_from_path(_PLXT_CONFIG_PATH)+ excluded: list[str] = config["exclude"]+ return {glob for glob in excluded if _CORPUS_GLOB_MARKER in glob}+++class TestMthdsCorpusPlxtExclusions:+ def test_the_excluded_entries_are_exactly_the_schema_fault_entries(self) -> None:+ schema_fault_tags = load_vocabulary().schema_fault_tags+ assert schema_fault_tags, "The vocabulary declares no schema fault, so this gate would pass vacuously"++ expected = {+ f"**/mthds_corpus/entries/{entry.name}/*.mthds"+ for entry in iter_entries(validity=EntryValidity.INVALID)+ if not schema_fault_tags.isdisjoint(entry.manifest.covers)+ }+ assert _corpus_exclusion_globs() == expected, (+ "`.pipelex/plxt.toml`'s corpus exclusions disagree with the vocabulary's `fails_at` signal. "+ "An entry whose tag says `schema` must be excluded — the schema rejects it as the very error "+ "it exists to trigger — and every other invalid entry must be linted like any other .mthds file."+ )