Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
The MTHDS Test Corpus — one canonical fixture set, with exhaustivity gated (#1132)
Louis Choquel339ccaacf60c
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 51a698107..afaac96be 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -1,5 +1,15 @@# Changelog+## [Unreleased]++### Added++- **The MTHDS Test Corpus: one canonical, tagged set of `.mthds` methods, with coverage of the language surface turned into a gate instead of a thing somebody has to remember**: Every repo in the workspace had grown its own pile of `.mthds` fixtures, each shaped by its own consumer, and coverage of the MTHDS language surface — a cross-repo invariant — was asserted locally in each of them against a private pile. The result was exactly what that arrangement predicts: the `Time` native concept had no pipelex fixture at all, while another repo had independently patched the same gap in its own corpus without either side knowing. The corpus lives at `pipelex/test_extras/mthds_corpus/` and ships in the wheel, so a consumer that already depends on `pipelex` reads it through `iter_entries()` / `get_entry()` with no copy and no drift — lockstep by construction via the pinned version. Each entry is a directory holding the method, an optional `inputs.json`, and an `entry.toml` declaring what it exercises: `covers` (the axes, drawn from a closed vocabulary), `tier` (the cheapest conformance tier at which it is meaningful), `granularity`, and `validity` — plus, for a deliberately invalid entry, the exact wire `error_type` it must produce. Metadata lives beside the bundle and never inside it, so a bundle stays a pure language artifact. The `native.*` tag vocabulary is **generated** from `NativeConceptCode` by `make generate-corpus-vocabulary` and committed, so a consumer reads the closed tag set without a registry walk; the generator itself stays unshipped dev tooling. Three gates hold it: regeneration must reproduce the committed vocabulary byte for byte, every non-excluded tag must have a `focused` entry covering it, and every entry must mean what its manifest claims — a valid one validating, an invalid one failing with exactly its declared error, both checked against the local runtime rather than a deployed API that lags it, and both run over every entry regardless of tier so an `inference`-tier entry that broke is caught without spending a token. The exhaustivity gate is the one that closes the hole: the moment a registry grows a code, regeneration adds its tag and the build stays red until an entry covers it. The corpus is seeded with a focused entry for every native concept, and no code needed excluding — each one turned out to support a real entry. Entry subject matter is ordinary-world by rule, never AI and never MTHDS's own vocabulary, because an entry exists so a red gate can be narrated in one sentence and *"the model output the model"* is not one. The cross-repo contract is the workspace spec `docs/specs/mthds-test-corpus.md`; the working guide is `docs/contribute/mthds-test-corpus.md`.++### Changed++- **The `Date` and `YesNo` dry-run e2e tests now source their bundles from the corpus** rather than from copies sitting beside them, and those copies are deleted. Nothing in this repo holds a second copy of a language-level `.mthds` method any more.+## [v0.47.0] - 2026-08-19### Addeddiff --git a/Makefile b/Makefileindex 98168531d..303008fab 100644--- a/Makefile+++ b/Makefile@@ -105,6 +105,7 @@ make generate-mthds-schema - Generate JSON Schema for .mthds filesmake gms - Shorthand -> generate-mthds-schemamake check-mthds-schema - Check MTHDS JSON Schema is up-to-datemake cms - Shorthand -> check-mthds-schema+make generate-corpus-vocabulary - Regenerate the MTHDS Test Corpus tag vocabularymake check-ledger - Check migration ledgers are legal and replay harmlesslymake cl - Shorthand -> check-ledgermake check-migration-schemas - Check configuration surfaces have accounted for their schema changes@@ -223,6 +224,7 @@ export HELPli check-unused-imports fix-unused-imports check-TODOs check-uv \docs docs-check docs-serve-versioned docs-list docs-deploy docs-deploy-stable docs-deploy-specific-version docs-delete \generate-mthds-schema generate-mthds-schema-quiet gms check-mthds-schema cms \+ generate-corpus-vocabulary generate-corpus-vocabulary-quiet \check-ledger cl check-migration-schemas cmig up-migration-schemas up-migration-schemas-force umig umigf \generate-error-pages generate-error-pages-quiet gep \generate-error-identity generate-error-identity-quiet gei \@@ -422,6 +424,15 @@ check-mthds-schema: envcms: check-mthds-schema@echo "> done: cms = check-mthds-schema"+# The corpus vocabulary is committed and ships in the wheel, so its freshness is gated by a unit+# test that regenerates it in memory and compares byte for byte — there is no `check-` twin here.+generate-corpus-vocabulary: env+ $(call PRINT_TITLE,"Generating MTHDS Test Corpus vocabulary")+ $(VENV_PIPELEX_DEV) generate-corpus-vocabulary++generate-corpus-vocabulary-quiet: env+ $(VENV_PIPELEX_DEV) generate-corpus-vocabulary --quiet+check-ledger: env$(call PRINT_TITLE,"Checking migration ledgers are legal and converge")$(VENV_PIPELEX_DEV) check-ledger --quiet@@ -1207,15 +1218,15 @@ vg: view-graphc: check-keyword-only format lint pyright mypy@echo "> done: c = check"-cc: cleanderived regenerate-test-models-quiet generate-mthds-schema-quiet update-gateway-models-quiet c- @echo "> done: cc = cleanderived regenerate-test-models generate-mthds-schema update-gateway-models format lint pyright mypy"+cc: cleanderived regenerate-test-models-quiet generate-mthds-schema-quiet generate-corpus-vocabulary-quiet update-gateway-models-quiet c+ @echo "> done: cc = cleanderived regenerate-test-models generate-mthds-schema generate-corpus-vocabulary update-gateway-models format lint pyright mypy"# `up-migration-schemas` is deliberately NOT part of this aggregate. Every other regenerator here# rewrites a derived artifact from a live source; the migration head golden is a *proof obligation*# the coverage gate reads, so folding it into a habitual bulk regeneration would let a removal be# erased by muscle memory. Run `make umig` on purpose, and read its diff.-up: generate-mthds-schema-quiet update-gateway-models-quiet up-kit-configs rules- @echo "> done: up = generate-mthds-schema update-gateway-models up-kit-configs rules"+up: generate-mthds-schema-quiet generate-corpus-vocabulary-quiet update-gateway-models-quiet up-kit-configs rules+ @echo "> done: up = generate-mthds-schema generate-corpus-vocabulary update-gateway-models up-kit-configs rules"check: cleanderived regenerate-test-models-quiet generate-mthds-schema-quiet update-gateway-models-quiet check-unused-imports check-config-sync check-rules check-urls check-gateway-models check-mthds-schema check-ledger check-migration-schemas check-keyword-only check-hub-layering drift-check format lint pyright mypy pylint@echo "> done: check"diff --git a/docs/contribute/mthds-test-corpus.md b/docs/contribute/mthds-test-corpus.mdnew file mode 100644index 000000000..a61696df9--- /dev/null+++ b/docs/contribute/mthds-test-corpus.md@@ -0,0 +1,112 @@+# The MTHDS Test Corpus++The MTHDS Test Corpus is one canonical, tagged set of `.mthds` methods that every repo in the workspace draws its language-level fixtures from. It lives at `pipelex/test_extras/mthds_corpus/`, ships in the wheel, and is gated on both sides: the corpus must cover every feature the runtime registers, and each consumer must exercise every entry in the slice it declares.++The cross-repo contract — the one other repos are written against — is the workspace-root spec `docs/specs/mthds-test-corpus.md`. This page is the pipelex-side working guide: where things are, how to add an entry, and what each gate is telling you when it goes red.++!!! note "Why it exists"++ Every repo grew its own pile of `.mthds` fixtures, and coverage of the MTHDS language surface is a cross-repo invariant asserted locally in each of them. The `Time` native concept had no pipelex fixture at all, and mthds-ui had independently patched the same gap in its own corpus. The exhaustivity gate makes that a red build instead of a thing someone has to remember.++## Layout++```+pipelex/test_extras/mthds_corpus/+ vocabulary.toml generated, committed, ships — the closed tag set+ manifest.py the strict entry.toml model+ vocabulary.py the vocabulary reader+ loader.py iter_entries() / get_entry()+ resources.py package-data path resolution+ exceptions.py+ entries/+ native_time_departure/+ entry.toml+ bundle.mthds+ inputs.json optional+```++An entry directory holds either exactly one `.mthds` file, or several with a `bundle.mthds` acting as the entry point — which is exactly what `pipelex validate bundle <dir>` already resolves, so an entry directory is a valid CLI argument as it stands.++## Adding an entry++1. **Pick an ordinary-world subject.** An entry's subject matter is never AI, machine learning, language models, agents, prompting or embeddings, and never MTHDS's own vocabulary either. The corpus *uses* `PipeLLM` constantly; it never *talks about* it. The reason is legibility under failure: an entry exists so a red gate can be narrated in one sentence, and *"the model output the model"* is not a sentence anyone can act on. Train timetables, invoices, recipes, weather observations, shipping manifests all work. The full rule, including the carve-out that lets an invalid entry be named for the defect it triggers, is in the spec.++2. **Create the directory** under `entries/`, named in `snake_case`, and write the bundle.++3. **Write `entry.toml`.** `name` must repeat the directory name; `covers` names the axes the entry exists for and must draw only on tags the vocabulary declares.++ ```toml+ name = "native_time_departure"+ description = "The departure time of day read off a printed timetable line"++ validity = "valid"+ tier = "dry"+ granularity = "focused"++ covers = [+ "native.time",+ ]+ ```++4. **Validate it locally** — against the local runtime, never the hosted API, which lags it:++ ```bash+ .venv/bin/pipelex validate bundle pipelex/test_extras/mthds_corpus/entries/native_time_departure/+ ```++5. **Run the gates**: `.venv/bin/pytest tests/unit/pipelex/test_extras tests/integration/pipelex/test_extras`.++An **invalid** entry additionally declares the exact wire `error_type` it must produce. Read that string off the runtime rather than inferring it from the source:++```bash+.venv/bin/pipelex-agent validate bundle <path> -L <entry_dir>/ --format json --error-format json+```++!!! warning "The `.mthds` editor hook fights invalid entries"++ The `pipelex` plugin's `PostToolUse` hook lints, formats and **blocks on an invalid verdict** for every `Write`/`Edit` of a `.mthds` file, and has no skip mechanism. Author valid entries with the editing tools to collect the free lint; write an invalid entry's bundle through a shell heredoc, which the hook's matcher does not intercept.++## The tag vocabulary++Tags are namespaced and the vocabulary is closed: an entry covering a tag the vocabulary does not declare fails the gate. `native.*` is generated from `NativeConceptCode`; `feature.*` is reserved for hand-maintained language features; `operator.*`, `controller.*` and `error.*` arrive with their registries.++`vocabulary.toml` is generated in full, so it is never edited by hand:++```bash+make generate-corpus-vocabulary+```++Everything the file says is decided in the generator, `pipelex/cli/dev_cli/commands/generate_corpus_vocabulary_cmd.py` — the registries it walks, and every exclusion reason. It is unshipped dev tooling (the wheel excludes `pipelex/cli/dev_cli`) while the file it writes ships, which is what lets a consumer read the closed tag set without a registry walk. A hand-maintained namespace will be declared there too when one first has content; `feature.*` is reserved in the contract but has no tags yet, and adding one before an entry covers it would simply red the exhaustivity gate.++An **exclusion** marks a registry code that no standalone focused entry could meaningfully exercise. It stays in the vocabulary and stays usable by an entry; it is simply not owed a focused entry. No native code is excluded today — every one of them turned out to support a real entry. The exclusion map is keyed on the registry enum rather than on strings, which is what keeps an exclusion from outliving the code it names: a removed code breaks the generator at import, naming itself.++## The gates, and what a red one means++| Gate | Where | Red means |+|---|---|---|+| Vocabulary drift | `tests/unit/pipelex/test_extras/test_mthds_corpus_vocabulary.py` | A registry changed and the committed vocabulary was not regenerated. Run `make generate-corpus-vocabulary`. |+| Exhaustivity | `tests/unit/pipelex/test_extras/test_mthds_corpus_exhaustivity.py` | A vocabulary tag has no `focused` entry covering it — write the entry, or record an exclusion with a reason. Also fires when an entry covers a tag that is not in the vocabulary. |+| Manifest | `tests/unit/pipelex/test_extras/test_mthds_corpus_manifest.py` | The strict `entry.toml` model rejected something. |+| Layout and filters | `tests/unit/pipelex/test_extras/test_mthds_corpus_loader.py` | An entry's name does not match its directory, its bundle does not resolve, or the loader's filter semantics changed. |+| Entry validation | `tests/integration/pipelex/test_extras/test_mthds_corpus_entries.py` | A valid entry stopped validating, or an invalid one stopped failing with exactly its declared error. Runs over every entry regardless of tier, so an `inference`-tier entry that broke is caught without spending a token. |++## Consuming the corpus from a test++```python+from pipelex.test_extras.mthds_corpus.loader import get_entry, iter_entries+from pipelex.test_extras.mthds_corpus.manifest import EntryTier++entry = get_entry(name="native_time_departure")+runner = PipelexMTHDSProtocol(library_dirs=[str(entry.directory)], pipe_run_mode=PipeRunMode.DRY)++for entry in iter_entries(tier=EntryTier.DRY):+ ... # every entry a dry-tier consumer can afford+```++`iter_entries()` filters compose conjunctively and each is optional: `tags` (the entry covers **all** of them), `tier` (a ceiling — the entry's tier is that one or cheaper), `validity`, `granularity`. Ordering is entry-name lexicographic and stable, so parametrized test ids do not churn.++The corpus is the single source for language-level `.mthds` methods in this repo: `tests/e2e/pipelex/pipes/date/` and `tests/e2e/pipelex/pipes/yes_no/` already reach for their bundles this way rather than keeping local copies.++!!! note "The entries are data, never auto-loaded"++ Nothing boots the corpus into the standard library — libraries load only from explicitly passed `library_dirs`. Be aware, though, that `pipelex/libraries/library_utils.py::get_pipelex_mthds_files_from_package()` recursively sweeps every `.mthds` file under the installed package. It has no callers today; wiring one up would pull the whole corpus into the standard library, so it would need to exclude the corpus tree first.diff --git a/mkdocs.yml b/mkdocs.ymlindex 0dc803125..c1cf11286 100644--- a/mkdocs.yml+++ b/mkdocs.yml@@ -342,6 +342,7 @@ plugins:- contribute/test-duration-map.md: "Test Duration Map"- contribute/registration-surface.md: "Registration Surface"- contribute/drift-contracts.md: "Drift Contracts"+ - contribute/mthds-test-corpus.md: "MTHDS Test Corpus"- migration-ledger.md: "Migration Ledger"- changelog.md: "Changelog"@@ -587,6 +588,7 @@ nav:- Test Duration Map: contribute/test-duration-map.md- Registration Surface: contribute/registration-surface.md- Drift Contracts: contribute/drift-contracts.md+ - MTHDS Test Corpus: contribute/mthds-test-corpus.md- Migration Ledger: migration-ledger.md- Code of Conduct: CODE_OF_CONDUCT.md- License: license.mddiff --git a/pipelex/cli/dev_cli/_dev_cli.py b/pipelex/cli/dev_cli/_dev_cli.pyindex d9cdeabd5..1ab7b3856 100644--- a/pipelex/cli/dev_cli/_dev_cli.py+++ b/pipelex/cli/dev_cli/_dev_cli.py@@ -20,6 +20,7 @@ from pipelex.cli.dev_cli.commands.check_mthds_schema_cmd import check_mthds_schefrom pipelex.cli.dev_cli.commands.check_rules_sync_cmd import check_rules_sync_cmdfrom pipelex.cli.dev_cli.commands.check_urls_cmd import DEFAULT_TIMEOUT, check_urls_cmdfrom pipelex.cli.dev_cli.commands.drift.drift_cmd import drift_app+from pipelex.cli.dev_cli.commands.generate_corpus_vocabulary_cmd import generate_corpus_vocabulary_cmdfrom pipelex.cli.dev_cli.commands.generate_error_identity_cmd import generate_error_identity_cmdfrom pipelex.cli.dev_cli.commands.generate_error_pages_cmd import generate_error_pages_cmdfrom pipelex.cli.dev_cli.commands.generate_mthds_schema_cmd import generate_mthds_schema_cmd@@ -53,6 +54,7 @@ class PipelexDevCLI(TyperGroup):"check-rules","check-urls","drift",+ "generate-corpus-vocabulary","generate-error-identity","generate-error-pages","generate-mthds-schema",@@ -179,6 +181,26 @@ def check_urls_command(sys.exit(1)[email protected](name="generate-corpus-vocabulary", help="Regenerate the MTHDS Test Corpus tag vocabulary from the runtime registries")+def generate_corpus_vocabulary_command(+ quiet: Annotated[bool, typer.Option("--quiet", "-q", help="Output only a single status line")] = False,+) -> None:+ """Regenerate the committed corpus tag vocabulary."""+ try:+ generate_corpus_vocabulary_cmd(quiet=quiet)+ except (typer.Exit, typer.Abort):+ # Typer control-flow exits carry an intended exit code — not a failure. Let them through.+ raise+ except Exception: # noqa: BLE001+ # Dev CLI command root: print a traceback for any unexpected failure and exit non-zero.+ console = get_console()+ console.print()+ console.print("[bold red]Unexpected error occurred[/bold red]")+ console.print()+ console.print(Traceback())+ sys.exit(1)++@app.command(name="generate-error-identity", help="Regenerate the committed (error_type, title, type_uri) snapshot of every PipelexError subclass")def generate_error_identity_command(output: Annotated[str | None, typer.Option("--output", "-o", help="Custom output path for the snapshot file")] = None,diff --git a/pipelex/cli/dev_cli/commands/generate_corpus_vocabulary_cmd.py b/pipelex/cli/dev_cli/commands/generate_corpus_vocabulary_cmd.pynew file mode 100644index 000000000..e7dd48838--- /dev/null+++ b/pipelex/cli/dev_cli/commands/generate_corpus_vocabulary_cmd.py@@ -0,0 +1,121 @@+"""Command to generate the MTHDS Test Corpus tag vocabulary.++Contract: ``docs/specs/mthds-test-corpus.md`` (workspace root), section "Tag vocabulary".++The generator is unshipped dev tooling — the wheel excludes ``pipelex/cli/dev_cli`` — while the+``vocabulary.toml`` it writes into the corpus tree is committed and ships. That split is the+point: a consumer reads the closed tag set without needing a registry walk, and the tag set+cannot drift from the registry because a unit test regenerates it in memory and compares byte+for byte.+"""++from __future__ import annotations++import re+import sys++from rich.markup import escape+from rich.panel import Panel++from pipelex.core.concepts.native.concept_native import NativeConceptCode+from pipelex.runtime_hub import get_console+from pipelex.test_extras.mthds_corpus.vocabulary import vocabulary_path++_ACRONYM_BOUNDARY_RE = re.compile(r"(?<=[A-Z0-9])(?=[A-Z][a-z])")+_WORD_BOUNDARY_RE = re.compile(r"(?<=[a-z0-9])(?=[A-Z])")++_HEADER = """# The MTHDS Test Corpus tag vocabulary — the closed set an entry's `covers` draws from.+#+# GENERATED by `pipelex-dev generate-corpus-vocabulary`. Do not edit this file: the generated+# namespaces are derived from the runtime's own registries, and every exclusion reason is+# declared in the generator's source.+#+# This file ships in the pipelex wheel, so it names a document its readers can reach:+# https://docs.pipelex.com/contribute/mthds-test-corpus/ — which in turn points at the+# cross-repo contract, `docs/specs/mthds-test-corpus.md` in the Pipelex workspace repo.+"""++# A code is excluded when no standalone focused entry could meaningfully exercise it. Exclusions+# are visible decisions, one reason each: an excluded tag stays in the vocabulary and stays usable+# by an entry, it is simply not required to have a focused entry of its own. Keying this map by+# the enum rather than by a string is what stops an exclusion from outliving the code it names —+# an entry reads `NativeConceptCode.PAGE: "…"`, so a code that leaves the registry breaks this+# module at import with an AttributeError naming it, before any generation runs. No code needs+# excluding today; every native concept turned out to support a real focused entry.+_NATIVE_EXCLUSIONS: dict[NativeConceptCode, str] = {}+++def normalize_registry_code(*, code: str) -> str:+ """Turn a CamelCase registry code into a snake_case tag local name.++ Insert a separator at every acronym boundary, then at every word boundary, then lowercase.+ The acronym rule is what keeps `JSON` from becoming `j_s_o_n` and splits `JSONSchema` into+ `json_schema`.+ """+ separated = _ACRONYM_BOUNDARY_RE.sub("_", code)+ separated = _WORD_BOUNDARY_RE.sub("_", separated)+ return separated.lower()+++def _toml_string(*, value: str) -> str:+ escaped = value.replace("\\", "\\\\").replace('"', '\\"')+ return f'"{escaped}"'+++def _render_native_namespace() -> str:+ blocks: list[str] = []+ for native_code in NativeConceptCode:+ lines = [f"[native.{normalize_registry_code(code=native_code.value)}]", f"code = {_toml_string(value=native_code.value)}"]+ exclusion_reason = _NATIVE_EXCLUSIONS.get(native_code)+ if exclusion_reason is not None:+ lines.append(f"excluded = {_toml_string(value=exclusion_reason)}")+ blocks.append("\n".join(lines))+ return "\n\n".join(blocks)+++def generate_corpus_vocabulary_content() -> str:+ """Render the whole vocabulary file, deterministically, in registry declaration order."""+ return f"{_HEADER}\n{_render_native_namespace()}\n"+++def generate_corpus_vocabulary_cmd(*, quiet: bool = False) -> None:+ """Write the corpus tag vocabulary into the corpus tree."""+ console = get_console()+ output_path = vocabulary_path()++ if not quiet:+ console.print()+ console.print("[bold]Generating MTHDS Test Corpus vocabulary...[/bold]")+ console.print()++ try:+ content = generate_corpus_vocabulary_content()+ except ValueError as exc:+ if quiet:+ console.print(f"[red]✗ Corpus vocabulary generation: FAILED[/red] - {escape(str(exc))}")+ else:+ console.print(+ Panel(+ f"[red]✗[/red] Failed to generate the corpus vocabulary\n\n[dim]{escape(str(exc))}[/dim]",+ title="[bold red]Corpus Vocabulary Generation: FAILED[/bold red]",+ border_style="red",+ padding=(1, 2),+ )+ )+ console.print()+ sys.exit(1)++ output_path.write_text(content, encoding="utf-8")++ if quiet:+ console.print("[green]✓ Corpus vocabulary generation: PASSED[/green]")+ else:+ console.print(+ Panel(+ f"[green]✓[/green] Vocabulary generated successfully!\n\n[dim]Output: {escape(str(output_path))}[/dim]",+ title="[bold green]Corpus Vocabulary Generation: PASSED[/bold green]",+ border_style="green",+ padding=(1, 2),+ )+ )+ console.print()diff --git a/pipelex/test_extras/mthds_corpus/__init__.py b/pipelex/test_extras/mthds_corpus/__init__.pynew file mode 100644index 000000000..e69de29bbdiff --git a/pipelex/test_extras/mthds_corpus/entries/native_anything_lost_item/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/native_anything_lost_item/bundle.mthdsnew file mode 100644index 000000000..69d59b0a4--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_anything_lost_item/bundle.mthds@@ -0,0 +1,14 @@+domain = "lost_property"+description = "Write a counter description for whatever was handed in to lost property"+main_pipe = "describe_handed_in_item"++[pipe.describe_handed_in_item]+type = "PipeLLM"+description = "Write a short counter description of a handed-in item, whatever it turns out to be"+inputs = { handed_in_item = "Anything" }+output = "Text"+prompt = """+Write two sentences describing this handed-in item for the lost property counter.++@handed_in_item+"""diff --git a/pipelex/test_extras/mthds_corpus/entries/native_anything_lost_item/entry.toml b/pipelex/test_extras/mthds_corpus/entries/native_anything_lost_item/entry.tomlnew file mode 100644index 000000000..409ac4d1f--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_anything_lost_item/entry.toml@@ -0,0 +1,10 @@+name = "native_anything_lost_item"+description = "A description written for a lost item of any kind"++validity = "valid"+tier = "dry"+granularity = "focused"++covers = [+ "native.anything",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/native_composite_weather_report/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/native_composite_weather_report/bundle.mthdsnew file mode 100644index 000000000..d9fb1297a--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_composite_weather_report/bundle.mthds@@ -0,0 +1,36 @@+domain = "weather_observations"+description = "Summarize a weather observation and read its temperature at the same time"+main_pipe = "read_observation"++[pipe.read_observation]+type = "PipeParallel"+description = "Summarize the observation and read its temperature side by side"+inputs = { observation = "Text" }+output = "Composite"+add_each_output = true+branches = [+ { pipe = "summarize_observation", result = "summary" },+ { pipe = "read_temperature", result = "temperature" },+]++[pipe.summarize_observation]+type = "PipeLLM"+description = "Summarize the weather observation in one sentence"+inputs = { observation = "Text" }+output = "Text"+prompt = """+Summarize this weather observation in one sentence.++@observation+"""++[pipe.read_temperature]+type = "PipeLLM"+description = "Read the temperature the observation records"+inputs = { observation = "Text" }+output = "Number"+prompt = """+What temperature in degrees Celsius does this observation record? Answer with the number alone.++@observation+"""diff --git a/pipelex/test_extras/mthds_corpus/entries/native_composite_weather_report/entry.toml b/pipelex/test_extras/mthds_corpus/entries/native_composite_weather_report/entry.tomlnew file mode 100644index 000000000..8f732cecf--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_composite_weather_report/entry.toml@@ -0,0 +1,10 @@+name = "native_composite_weather_report"+description = "A weather observation summarized and scored in one go"++validity = "valid"+tier = "dry"+granularity = "focused"++covers = [+ "native.composite",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/native_composite_weather_report/inputs.json b/pipelex/test_extras/mthds_corpus/entries/native_composite_weather_report/inputs.jsonnew file mode 100644index 000000000..fc56f7661--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_composite_weather_report/inputs.json@@ -0,0 +1,8 @@+{+ "observation": {+ "concept": "native.Text",+ "content": {+ "text": "Brest, 06:00 UTC: overcast, drizzle, 11 degrees Celsius, southwesterly wind 25 km/h, visibility 4 km."+ }+ }+}diff --git a/tests/e2e/pipelex/pipes/date/date_departure/date_departure.mthds b/pipelex/test_extras/mthds_corpus/entries/native_date_departure/bundle.mthdssimilarity index 82%rename from tests/e2e/pipelex/pipes/date/date_departure/date_departure.mthdsrename to pipelex/test_extras/mthds_corpus/entries/native_date_departure/bundle.mthdsindex b27f81e81..b424ebc26 100644--- a/tests/e2e/pipelex/pipes/date/date_departure/date_departure.mthds+++ b/pipelex/test_extras/mthds_corpus/entries/native_date_departure/bundle.mthds@@ -1,5 +1,5 @@-domain = "date_demo"-description = "Native Date as a pipeline input fed by TOML date/datetime literals"+domain = "ticket_desk"+description = "Read a departure date off a ticket, and write a note about it"main_pipe = "describe_departure"[pipe.describe_departure]diff --git a/pipelex/test_extras/mthds_corpus/entries/native_date_departure/entry.toml b/pipelex/test_extras/mthds_corpus/entries/native_date_departure/entry.tomlnew file mode 100644index 000000000..c6fc0c02d--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_date_departure/entry.toml@@ -0,0 +1,10 @@+name = "native_date_departure"+description = "A departure date read off a ticket, and a note written from it"++validity = "valid"+tier = "dry"+granularity = "focused"++covers = [+ "native.date",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/native_document_invoice_scan/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/native_document_invoice_scan/bundle.mthdsnew file mode 100644index 000000000..570adc4bc--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_document_invoice_scan/bundle.mthds@@ -0,0 +1,14 @@+domain = "supplier_records"+description = "Name the supplier that issued a scanned purchase invoice"+main_pipe = "name_supplier"++[pipe.name_supplier]+type = "PipeLLM"+description = "Name the supplier that issued the scanned invoice"+inputs = { invoice_scan = "Document" }+output = "Text"+prompt = """+Which supplier issued this invoice? Answer with the supplier name alone.++@invoice_scan+"""diff --git a/pipelex/test_extras/mthds_corpus/entries/native_document_invoice_scan/entry.toml b/pipelex/test_extras/mthds_corpus/entries/native_document_invoice_scan/entry.tomlnew file mode 100644index 000000000..db61c25e0--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_document_invoice_scan/entry.toml@@ -0,0 +1,10 @@+name = "native_document_invoice_scan"+description = "The supplier named on a scanned purchase invoice"++validity = "valid"+tier = "dry"+granularity = "focused"++covers = [+ "native.document",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/native_dynamic_tracking_field/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/native_dynamic_tracking_field/bundle.mthdsnew file mode 100644index 000000000..85b15c1ae--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_dynamic_tracking_field/bundle.mthds@@ -0,0 +1,20 @@+domain = "parcel_tracking"+description = "Read one requested field out of a parcel tracking record, leaving the field's concept for the caller to pick"+main_pipe = "read_requested_field"++[pipe.read_requested_field]+type = "PipeLLM"+description = "Read whichever field of the tracking record the caller asked for"+inputs = { tracking_record = "Text", wanted_field = "Text" }+output = "Dynamic"+prompt = """+Read the field named below out of this parcel tracking record.++## Field++@wanted_field++## Record++@tracking_record+"""diff --git a/pipelex/test_extras/mthds_corpus/entries/native_dynamic_tracking_field/entry.toml b/pipelex/test_extras/mthds_corpus/entries/native_dynamic_tracking_field/entry.tomlnew file mode 100644index 000000000..ef0c3e04f--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_dynamic_tracking_field/entry.toml@@ -0,0 +1,10 @@+name = "native_dynamic_tracking_field"+description = "A field of a parcel tracking record, whose concept the caller picks at run time"++validity = "valid"+tier = "dry"+granularity = "focused"++covers = [+ "native.dynamic",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/native_dynamic_tracking_field/inputs.json b/pipelex/test_extras/mthds_corpus/entries/native_dynamic_tracking_field/inputs.jsonnew file mode 100644index 000000000..85abb1cef--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_dynamic_tracking_field/inputs.json@@ -0,0 +1,14 @@+{+ "tracking_record": {+ "concept": "native.Text",+ "content": {+ "text": "Parcel RB884120: collected 3 March, sorted at Rennes 4 March, out for delivery 5 March, signed for by M. Fabre."+ }+ },+ "wanted_field": {+ "concept": "native.Text",+ "content": {+ "text": "who signed for the parcel"+ }+ }+}diff --git a/pipelex/test_extras/mthds_corpus/entries/native_html_menu_card/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/native_html_menu_card/bundle.mthdsnew file mode 100644index 000000000..9f0b67655--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_html_menu_card/bundle.mthds@@ -0,0 +1,14 @@+domain = "restaurant_menus"+description = "Render a handwritten daily menu as a small HTML card"+main_pipe = "render_menu_card"++[pipe.render_menu_card]+type = "PipeLLM"+description = "Render the daily menu as an HTML fragment"+inputs = { menu = "Text" }+output = "Html"+prompt = """+Render this daily menu as a small HTML card.++@menu+"""diff --git a/pipelex/test_extras/mthds_corpus/entries/native_html_menu_card/entry.toml b/pipelex/test_extras/mthds_corpus/entries/native_html_menu_card/entry.tomlnew file mode 100644index 000000000..a22169427--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_html_menu_card/entry.toml@@ -0,0 +1,10 @@+name = "native_html_menu_card"+description = "A restaurant menu rendered as an HTML card"++validity = "valid"+tier = "dry"+granularity = "focused"++covers = [+ "native.html",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/native_html_menu_card/inputs.json b/pipelex/test_extras/mthds_corpus/entries/native_html_menu_card/inputs.jsonnew file mode 100644index 000000000..e31a72df0--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_html_menu_card/inputs.json@@ -0,0 +1,8 @@+{+ "menu": {+ "concept": "native.Text",+ "content": {+ "text": "Today: leek and potato soup, roast chicken with thyme, apple crumble. Bread and butter included."+ }+ }+}diff --git a/pipelex/test_extras/mthds_corpus/entries/native_image_parcel_label/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/native_image_parcel_label/bundle.mthdsnew file mode 100644index 000000000..932e05299--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_image_parcel_label/bundle.mthds@@ -0,0 +1,14 @@+domain = "parcel_intake"+description = "Read the delivery address off a photograph of a parcel label"+main_pipe = "read_label_address"++[pipe.read_label_address]+type = "PipeLLM"+description = "Read the delivery address printed on a photographed parcel label"+inputs = { label_photo = "Image" }+output = "Text"+prompt = """+Read the delivery address printed on this parcel label.++@label_photo+"""diff --git a/pipelex/test_extras/mthds_corpus/entries/native_image_parcel_label/entry.toml b/pipelex/test_extras/mthds_corpus/entries/native_image_parcel_label/entry.tomlnew file mode 100644index 000000000..dd4dace35--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_image_parcel_label/entry.toml@@ -0,0 +1,10 @@+name = "native_image_parcel_label"+description = "A parcel's address read off a photograph of its label"++validity = "valid"+tier = "dry"+granularity = "focused"++covers = [+ "native.image",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/native_json_invoice_totals/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/native_json_invoice_totals/bundle.mthdsnew file mode 100644index 000000000..9a8ab9f31--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_json_invoice_totals/bundle.mthds@@ -0,0 +1,14 @@+domain = "invoice_intake"+description = "Pull the totals off a purchase invoice into a JSON object"+main_pipe = "read_totals"++[pipe.read_totals]+type = "PipeLLM"+description = "Read the invoice totals into a JSON object"+inputs = { invoice_text = "Text" }+output = "JSON"+prompt = """+Read the net total, the tax and the gross total from this invoice into a JSON object.++@invoice_text+"""diff --git a/pipelex/test_extras/mthds_corpus/entries/native_json_invoice_totals/entry.toml b/pipelex/test_extras/mthds_corpus/entries/native_json_invoice_totals/entry.tomlnew file mode 100644index 000000000..7a746a6c1--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_json_invoice_totals/entry.toml@@ -0,0 +1,10 @@+name = "native_json_invoice_totals"+description = "The totals of a purchase invoice, as a JSON object"++validity = "valid"+tier = "dry"+granularity = "focused"++covers = [+ "native.json",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/native_json_invoice_totals/inputs.json b/pipelex/test_extras/mthds_corpus/entries/native_json_invoice_totals/inputs.jsonnew file mode 100644index 000000000..8994b2ef9--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_json_invoice_totals/inputs.json@@ -0,0 +1,8 @@+{+ "invoice_text": {+ "concept": "native.Text",+ "content": {+ "text": "Invoice 4471. Net 240.00 EUR. VAT 20% 48.00 EUR. Gross 288.00 EUR. Due 30 days."+ }+ }+}diff --git a/pipelex/test_extras/mthds_corpus/entries/native_number_crate_count/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/native_number_crate_count/bundle.mthdsnew file mode 100644index 000000000..5ce576ba9--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_number_crate_count/bundle.mthds@@ -0,0 +1,14 @@+domain = "warehouse_intake"+description = "Count the crates listed on a delivery intake slip"+main_pipe = "count_crates"++[pipe.count_crates]+type = "PipeLLM"+description = "Count how many crates the intake slip lists"+inputs = { intake_slip = "Text" }+output = "Number"+prompt = """+How many crates does this intake slip list? Answer with the number alone.++@intake_slip+"""diff --git a/pipelex/test_extras/mthds_corpus/entries/native_number_crate_count/entry.toml b/pipelex/test_extras/mthds_corpus/entries/native_number_crate_count/entry.tomlnew file mode 100644index 000000000..9a8ea0e5c--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_number_crate_count/entry.toml@@ -0,0 +1,10 @@+name = "native_number_crate_count"+description = "The number of crates counted on a warehouse intake slip"++validity = "valid"+tier = "dry"+granularity = "focused"++covers = [+ "native.number",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/native_number_crate_count/inputs.json b/pipelex/test_extras/mthds_corpus/entries/native_number_crate_count/inputs.jsonnew file mode 100644index 000000000..f3585265c--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_number_crate_count/inputs.json@@ -0,0 +1,8 @@+{+ "intake_slip": {+ "concept": "native.Text",+ "content": {+ "text": "Intake 2026-03-04: 6 crates of tinned tomatoes, 3 crates of olive oil, 2 crates of dried pasta."+ }+ }+}diff --git a/pipelex/test_extras/mthds_corpus/entries/native_page_catalog_leaf/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/native_page_catalog_leaf/bundle.mthdsnew file mode 100644index 000000000..f9c9f7c15--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_page_catalog_leaf/bundle.mthds@@ -0,0 +1,14 @@+domain = "bookshop_catalog"+description = "List the titles printed on one page of a bookshop catalog"+main_pipe = "list_titles_on_page"++[pipe.list_titles_on_page]+type = "PipeLLM"+description = "List the titles printed on a catalog page"+inputs = { catalog_page = "Page" }+output = "Text"+prompt = """+List every title printed on this catalog page, one per line.++@catalog_page+"""diff --git a/pipelex/test_extras/mthds_corpus/entries/native_page_catalog_leaf/entry.toml b/pipelex/test_extras/mthds_corpus/entries/native_page_catalog_leaf/entry.tomlnew file mode 100644index 000000000..d908e8693--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_page_catalog_leaf/entry.toml@@ -0,0 +1,10 @@+name = "native_page_catalog_leaf"+description = "The titles listed on one page of a bookshop catalog"++validity = "valid"+tier = "dry"+granularity = "focused"++covers = [+ "native.page",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/native_search_result_supplier_lookup/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/native_search_result_supplier_lookup/bundle.mthdsnew file mode 100644index 000000000..ef614947a--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_search_result_supplier_lookup/bundle.mthds@@ -0,0 +1,14 @@+domain = "supplier_lookup"+description = "Write a buyer's shortlist from the results of a supplier lookup"+main_pipe = "write_shortlist"++[pipe.write_shortlist]+type = "PipeLLM"+description = "Write a shortlist of suppliers from the lookup results"+inputs = { lookup = "SearchResult" }+output = "Text"+prompt = """+Write a shortlist of three suppliers from these lookup results, one line each, naming the source for every one.++@lookup+"""diff --git a/pipelex/test_extras/mthds_corpus/entries/native_search_result_supplier_lookup/entry.toml b/pipelex/test_extras/mthds_corpus/entries/native_search_result_supplier_lookup/entry.tomlnew file mode 100644index 000000000..204228a6a--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_search_result_supplier_lookup/entry.toml@@ -0,0 +1,10 @@+name = "native_search_result_supplier_lookup"+description = "A shortlist written from the results of a supplier lookup"++validity = "valid"+tier = "dry"+granularity = "focused"++covers = [+ "native.search_result",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/native_text_and_images_recipe_card/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/native_text_and_images_recipe_card/bundle.mthdsnew file mode 100644index 000000000..cf78f7931--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_text_and_images_recipe_card/bundle.mthds@@ -0,0 +1,14 @@+domain = "recipes"+description = "Summarize a recipe and describe the photographs that would illustrate it"+main_pipe = "make_recipe_card"++[pipe.make_recipe_card]+type = "PipeLLM"+description = "Summarize the recipe and describe its illustrating photographs"+inputs = { recipe = "Text" }+output = "TextAndImages"+prompt = """+Summarize this recipe in two sentences and describe the photographs that would illustrate it.++@recipe+"""diff --git a/pipelex/test_extras/mthds_corpus/entries/native_text_and_images_recipe_card/entry.toml b/pipelex/test_extras/mthds_corpus/entries/native_text_and_images_recipe_card/entry.tomlnew file mode 100644index 000000000..2b764445b--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_text_and_images_recipe_card/entry.toml@@ -0,0 +1,10 @@+name = "native_text_and_images_recipe_card"+description = "A recipe summary paired with the photographs that would illustrate it"++validity = "valid"+tier = "dry"+granularity = "focused"++covers = [+ "native.text_and_images",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/native_text_and_images_recipe_card/inputs.json b/pipelex/test_extras/mthds_corpus/entries/native_text_and_images_recipe_card/inputs.jsonnew file mode 100644index 000000000..090d1a3e3--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_text_and_images_recipe_card/inputs.json@@ -0,0 +1,8 @@+{+ "recipe": {+ "concept": "native.Text",+ "content": {+ "text": "Soften onions in butter, add rice and stir until glossy, ladle in hot stock a little at a time, finish with parmesan."+ }+ }+}diff --git a/pipelex/test_extras/mthds_corpus/entries/native_text_delay_notice/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/native_text_delay_notice/bundle.mthdsnew file mode 100644index 000000000..681cc9581--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_text_delay_notice/bundle.mthds@@ -0,0 +1,14 @@+domain = "train_delays"+description = "Turn a delay report into the short notice shown on a platform board"+main_pipe = "write_notice"++[pipe.write_notice]+type = "PipeLLM"+description = "Write the one-line platform notice for a delay"+inputs = { report = "Text" }+output = "Text"+prompt = """+Write a one-line platform notice announcing this delay.++@report+"""diff --git a/pipelex/test_extras/mthds_corpus/entries/native_text_delay_notice/entry.toml b/pipelex/test_extras/mthds_corpus/entries/native_text_delay_notice/entry.tomlnew file mode 100644index 000000000..be5b9c7d5--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_text_delay_notice/entry.toml@@ -0,0 +1,10 @@+name = "native_text_delay_notice"+description = "A platform notice written from a train delay report"++validity = "valid"+tier = "dry"+granularity = "focused"++covers = [+ "native.text",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/native_text_delay_notice/inputs.json b/pipelex/test_extras/mthds_corpus/entries/native_text_delay_notice/inputs.jsonnew file mode 100644index 000000000..5de47404a--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_text_delay_notice/inputs.json@@ -0,0 +1,8 @@+{+ "report": {+ "concept": "native.Text",+ "content": {+ "text": "The 14:52 service to Bristol is running 20 minutes late because of a signal fault outside Reading."+ }+ }+}diff --git a/pipelex/test_extras/mthds_corpus/entries/native_time_departure/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/native_time_departure/bundle.mthdsnew file mode 100644index 000000000..ffc2e92da--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_time_departure/bundle.mthds@@ -0,0 +1,14 @@+domain = "timetables"+description = "Read the departure time of day printed on a timetable line"+main_pipe = "read_departure_time"++[pipe.read_departure_time]+type = "PipeLLM"+description = "Read the departure time of day from a timetable line"+inputs = { timetable_line = "Text" }+output = "Time"+prompt = """+Read the departure time of day printed on this timetable line.++@timetable_line+"""diff --git a/pipelex/test_extras/mthds_corpus/entries/native_time_departure/entry.toml b/pipelex/test_extras/mthds_corpus/entries/native_time_departure/entry.tomlnew file mode 100644index 000000000..86986162c--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_time_departure/entry.toml@@ -0,0 +1,10 @@+name = "native_time_departure"+description = "The departure time of day read off a printed timetable line"++validity = "valid"+tier = "dry"+granularity = "focused"++covers = [+ "native.time",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/native_time_departure/inputs.json b/pipelex/test_extras/mthds_corpus/entries/native_time_departure/inputs.jsonnew file mode 100644index 000000000..cd08e1c2b--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_time_departure/inputs.json@@ -0,0 +1,8 @@+{+ "timetable_line": {+ "concept": "native.Text",+ "content": {+ "text": "Platform 4 -- 14:52 -- Bristol Temple Meads -- calls at Reading, Swindon, Bath Spa"+ }+ }+}diff --git a/tests/e2e/pipelex/pipes/yes_no/yes_no_judgment/yes_no_judgment.mthds b/pipelex/test_extras/mthds_corpus/entries/native_yes_no_urgent_message/bundle.mthdssimilarity index 80%rename from tests/e2e/pipelex/pipes/yes_no/yes_no_judgment/yes_no_judgment.mthdsrename to pipelex/test_extras/mthds_corpus/entries/native_yes_no_urgent_message/bundle.mthdsindex 5085946ec..97fe2b6b9 100644--- a/tests/e2e/pipelex/pipes/yes_no/yes_no_judgment/yes_no_judgment.mthds+++ b/pipelex/test_extras/mthds_corpus/entries/native_yes_no_urgent_message/bundle.mthds@@ -1,5 +1,5 @@-domain = "yes_no_demo"-description = "Native YesNo as a PipeLLM output (a judgment) and as a pipeline input"+domain = "support_inbox"+description = "Judge whether an inbox message is urgent, and read a verdict back to explain it"main_pipe = "judge_is_urgent"[pipe.judge_is_urgent]diff --git a/pipelex/test_extras/mthds_corpus/entries/native_yes_no_urgent_message/entry.toml b/pipelex/test_extras/mthds_corpus/entries/native_yes_no_urgent_message/entry.tomlnew file mode 100644index 000000000..5dad451a1--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_yes_no_urgent_message/entry.toml@@ -0,0 +1,10 @@+name = "native_yes_no_urgent_message"+description = "A yes/no verdict on whether an inbox message is urgent"++validity = "valid"+tier = "dry"+granularity = "focused"++covers = [+ "native.yes_no",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/native_yes_no_urgent_message/inputs.json b/pipelex/test_extras/mthds_corpus/entries/native_yes_no_urgent_message/inputs.jsonnew file mode 100644index 000000000..d0d7cf24b--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/native_yes_no_urgent_message/inputs.json@@ -0,0 +1,8 @@+{+ "message": {+ "concept": "native.Text",+ "content": {+ "text": "The cold room is at 12 degrees and the delivery is due at seven tomorrow morning."+ }+ }+}diff --git a/pipelex/test_extras/mthds_corpus/exceptions.py b/pipelex/test_extras/mthds_corpus/exceptions.pynew file mode 100644index 000000000..6c7e19cf1--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/exceptions.py@@ -0,0 +1,8 @@+class CorpusEntryError(Exception):+ """An entry directory that does not satisfy the layout the corpus contract pins.++ Deliberately not a ``PipelexError``: error discovery excludes ``test_extras`` (see+ ``_FIXTURE_DIR_NAMES`` in the error-pages generator), so a ``PipelexError`` here would carry a+ ``type_uri`` pointing at a documentation page nothing generates, and a wire identity no+ snapshot gates. This failure is a malformed fixture directory, which is nobody's wire contract.+ """diff --git a/pipelex/test_extras/mthds_corpus/loader.py b/pipelex/test_extras/mthds_corpus/loader.pynew file mode 100644index 000000000..eabcf81a4--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/loader.py@@ -0,0 +1,125 @@+"""The MTHDS Test Corpus loader — how a consumer takes a view of the corpus.++Contract: ``docs/specs/mthds-test-corpus.md`` (workspace root), section "Loader API".+A consumer selects entries by the axes it declares rather than by path convention, so+reorganizing storage never breaks it.+"""++import re+from collections.abc import Iterator+from pathlib import Path++from pydantic import BaseModel, ConfigDict++from pipelex.builder.conventions import DEFAULT_INPUTS_FILE_NAME+from pipelex.cli.bundle_target_resolution import BundleTargetResolutionError, resolve_bundle_target_core+from pipelex.test_extras.mthds_corpus.exceptions import CorpusEntryError+from pipelex.test_extras.mthds_corpus.manifest import (+ ENTRY_NAME_PATTERN,+ MANIFEST_FILE_NAME,+ CorpusEntryManifest,+ EntryGranularity,+ EntryTier,+ EntryValidity,+)+from pipelex.test_extras.mthds_corpus.resources import entries_root+from pipelex.tools.misc.toml_utils import load_toml_from_path+++class CorpusEntry(BaseModel):+ """One corpus entry: its manifest, its directory, and the paths a consumer needs."""++ model_config = ConfigDict(frozen=True)++ manifest: CorpusEntryManifest+ directory: Path+ bundle_path: Path+ inputs_path: Path | None++ @property+ def name(self) -> str:+ return self.manifest.name+++def _resolve_bundle_path(*, directory: Path) -> Path:+ """Pick the entry's bundle file through the very resolution ``pipelex validate bundle <dir>`` runs.++ The contract says an entry directory is a valid CLI argument as it stands — either exactly one+ ``.mthds`` file, or several with a ``bundle.mthds`` entry point. That claim is made true by+ calling the CLI's own core rather than by keeping a corpus copy of its rules: a copy is how the+ two drift, and a hand-rolled glob had already lost the core's refusal to auto-detect a symlinked+ bundle. Only the rendering of a failure is ours.+ """+ resolved = resolve_bundle_target_core(str(directory), library_dir=None)+ if isinstance(resolved, BundleTargetResolutionError):+ candidates = ", ".join(candidate.name for candidate in resolved.candidate_files)+ detail = f" ({candidates})" if candidates else ""+ msg = f"Corpus entry '{directory.name}' does not resolve to a bundle: {resolved.kind}{detail}"+ raise CorpusEntryError(msg)+ return resolved.bundle_path+++def load_entry(*, directory: Path) -> CorpusEntry:+ """Load one entry from its directory, enforcing the layout rules the contract pins."""+ manifest_path = directory / MANIFEST_FILE_NAME+ if not manifest_path.is_file():+ msg = f"Corpus entry '{directory.name}' has no '{MANIFEST_FILE_NAME}'"+ raise CorpusEntryError(msg)+ manifest = CorpusEntryManifest.model_validate(load_toml_from_path(manifest_path))+ if manifest.name != directory.name:+ msg = f"Corpus entry directory '{directory.name}' declares the name '{manifest.name}'; the two must match"+ raise CorpusEntryError(msg)+ inputs_path = directory / DEFAULT_INPUTS_FILE_NAME+ return CorpusEntry(+ manifest=manifest,+ directory=directory,+ bundle_path=_resolve_bundle_path(directory=directory),+ inputs_path=inputs_path if inputs_path.is_file() else None,+ )+++def get_entry(*, name: str) -> CorpusEntry:+ """Load the one entry with this name — what a consumer that wants a specific method calls."""+ # A name is a name. Checking it against the pattern the contract already pins for entry names+ # is what keeps a path from being joined onto the entries root and resolving somewhere else.+ if not re.match(ENTRY_NAME_PATTERN, name):+ msg = f"'{name}' is not a corpus entry name; entry names are lowercase snake_case"+ raise CorpusEntryError(msg)+ directory = entries_root() / name+ if not directory.is_dir():+ msg = f"There is no corpus entry named '{name}'"+ raise CorpusEntryError(msg)+ return load_entry(directory=directory)+++def iter_entries(+ *,+ tags: frozenset[str] | None = None,+ tier: EntryTier | None = None,+ validity: EntryValidity | None = None,+ granularity: EntryGranularity | None = None,+) -> Iterator[CorpusEntry]:+ """Yield the corpus entries matching every filter given, in entry-name order.++ Filters compose conjunctively, and omitting one places no restriction on that axis:++ - ``tags`` matches an entry that covers **all** the requested tags.+ - ``tier`` matches an entry whose tier is the requested one or cheaper, which is what a+ consumer running at a given tier can actually execute.+ - ``validity`` and ``granularity`` match exactly.++ Ordering is entry-name lexicographic and stable, so parametrized test ids do not churn.+ """+ for directory in sorted(entries_root().iterdir(), key=lambda path: path.name):+ if not directory.is_dir():+ continue+ entry = load_entry(directory=directory)+ if tags is not None and not tags.issubset(entry.manifest.covers):+ continue+ if tier is not None and entry.manifest.tier.rank > tier.rank:+ continue+ if validity is not None and entry.manifest.validity != validity:+ continue+ if granularity is not None and entry.manifest.granularity != granularity:+ continue+ yield entrydiff --git a/pipelex/test_extras/mthds_corpus/manifest.py b/pipelex/test_extras/mthds_corpus/manifest.pynew file mode 100644index 000000000..e184f6f84--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/manifest.py@@ -0,0 +1,93 @@+"""The MTHDS Test Corpus entry manifest — the strict model behind every ``entry.toml``.++Contract: ``docs/specs/mthds-test-corpus.md`` (workspace root), section "The entry manifest".+The model is deliberately strict: an unknown key is an error rather than a+forward-compatibility affordance, so a new field is a change to the spec and to this+model in the same commit.+"""++from enum import StrEnum+from typing import Self++from pydantic import BaseModel, ConfigDict, Field, model_validator++MANIFEST_FILE_NAME = "entry.toml"++ENTRY_NAME_PATTERN = r"^[a-z][a-z0-9_]*$"+++class EntryValidity(StrEnum):+ """Whether an entry's bundle is expected to validate."""++ VALID = "valid"+ INVALID = "invalid"+++class EntryTier(StrEnum):+ """The cheapest conformance tier at which an entry is meaningful."""++ STATIC = "static"+ DRY = "dry"+ OFFLINE = "offline"+ INFERENCE = "inference"++ @property+ def rank(self) -> int:+ """Position in the cheapest-first tier order.++ A consumer running at tier ``T`` runs every entry whose rank is at most ``T``'s.+ """+ match self:+ case EntryTier.STATIC:+ return 0+ case EntryTier.DRY:+ return 1+ case EntryTier.OFFLINE:+ return 2+ case EntryTier.INFERENCE:+ return 3+++class EntryGranularity(StrEnum):+ """Whether an entry is minimal or realistically multi-feature."""++ FOCUSED = "focused"+ COMPOSITE = "composite"+++class CorpusEntryManifest(BaseModel):+ """One parsed ``entry.toml``."""++ model_config = ConfigDict(extra="forbid", frozen=True)++ name: str = Field(pattern=ENTRY_NAME_PATTERN)+ description: str = Field(min_length=1)+ validity: EntryValidity+ tier: EntryTier+ granularity: EntryGranularity+ covers: list[str] = Field(min_length=1)+ expected_error: str | None = None++ @model_validator(mode="after")+ def validate_covers_has_no_duplicates(self) -> Self:+ duplicates = sorted({tag for tag in self.covers if self.covers.count(tag) > 1})+ if duplicates:+ msg = f"Entry '{self.name}' repeats tags in `covers`: {', '.join(duplicates)}"+ raise ValueError(msg)+ return self++ @model_validator(mode="after")+ def validate_expected_error_matches_validity(self) -> Self:+ # `expected_error` is what makes an invalid entry surgical: it names the one error the+ # bundle must produce, so the gate is red both when it fails differently and when it+ # accidentally validates. A valid entry carrying one would be claiming both.+ match self.validity:+ case EntryValidity.INVALID:+ if not self.expected_error:+ msg = f"Entry '{self.name}' is invalid, so it must declare the `expected_error` it produces"+ raise ValueError(msg)+ case EntryValidity.VALID:+ if self.expected_error is not None:+ msg = f"Entry '{self.name}' is valid, so it must not declare an `expected_error`"+ raise ValueError(msg)+ return selfdiff --git a/pipelex/test_extras/mthds_corpus/resources.py b/pipelex/test_extras/mthds_corpus/resources.pynew file mode 100644index 000000000..3b572c16c--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/resources.py@@ -0,0 +1,23 @@+"""Filesystem access to the corpus data files, which ship as package data inside the wheel."""++from importlib.resources import files+from pathlib import Path++CORPUS_PACKAGE = "pipelex.test_extras.mthds_corpus"+ENTRIES_DIR_NAME = "entries"+++def corpus_root() -> Path:+ """The corpus tree's directory on disk.++ Resolved through ``importlib.resources`` so the corpus is reachable from an installed+ wheel and not only from a source checkout. Wheels install unzipped, so the traversable is+ a real filesystem path and no extraction step is needed — the same assumption the rest of+ pipelex's package-data access already makes.+ """+ return Path(str(files(CORPUS_PACKAGE)))+++def entries_root() -> Path:+ """The directory holding one subdirectory per corpus entry."""+ return corpus_root() / ENTRIES_DIR_NAMEdiff --git a/pipelex/test_extras/mthds_corpus/vocabulary.py b/pipelex/test_extras/mthds_corpus/vocabulary.pynew file mode 100644index 000000000..6632ff56e--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/vocabulary.py@@ -0,0 +1,65 @@+"""The MTHDS Test Corpus tag vocabulary — the closed set an entry's ``covers`` draws from.++Contract: ``docs/specs/mthds-test-corpus.md`` (workspace root), section "Tag vocabulary".+``vocabulary.toml`` is generated in full by ``pipelex-dev generate-corpus-vocabulary`` and+committed, so a consumer reading it needs neither the generator nor a registry walk. This+module is the reader; the generator lives in the dev CLI, which the wheel excludes.+"""++from pathlib import Path++from pydantic import BaseModel, ConfigDict++from pipelex.test_extras.mthds_corpus.resources import corpus_root+from pipelex.tools.misc.toml_utils import load_toml_from_path++VOCABULARY_FILE_NAME = "vocabulary.toml"+++class VocabularyTag(BaseModel):+ """One tag in the vocabulary.++ ``code`` is the registry code the tag was generated from. ``excluded`` carries the reason a+ tag is not required to have a focused entry, and is present only on excluded tags.+ """++ model_config = ConfigDict(extra="forbid", frozen=True)++ code: str+ excluded: str | None = None+++class CorpusVocabulary(BaseModel):+ """The whole vocabulary, namespace by namespace."""++ model_config = ConfigDict(extra="forbid", frozen=True)++ namespaces: dict[str, dict[str, VocabularyTag]]++ @property+ def tags(self) -> frozenset[str]:+ """Every declared tag, in full ``<namespace>.<local_name>`` form."""+ declared: set[str] = set()+ for namespace, tags_in_namespace in self.namespaces.items():+ declared.update(f"{namespace}.{local_name}" for local_name in tags_in_namespace)+ return frozenset(declared)++ @property+ def required_tags(self) -> frozenset[str]:+ """The tags the exhaustivity gate requires a focused entry for — every tag that is not excluded."""+ required: set[str] = set()+ for namespace, tags_in_namespace in self.namespaces.items():+ for local_name, tag in tags_in_namespace.items():+ if tag.excluded is None:+ required.add(f"{namespace}.{local_name}")+ return frozenset(required)+++def vocabulary_path() -> Path:+ """Where the committed vocabulary file sits."""+ return corpus_root() / VOCABULARY_FILE_NAME+++def load_vocabulary() -> CorpusVocabulary:+ """Read the committed ``vocabulary.toml`` that ships beside the corpus entries."""+ return CorpusVocabulary(namespaces=load_toml_from_path(vocabulary_path()))diff --git a/pipelex/test_extras/mthds_corpus/vocabulary.toml b/pipelex/test_extras/mthds_corpus/vocabulary.tomlnew file mode 100644index 000000000..82e658a9f--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/vocabulary.toml@@ -0,0 +1,54 @@+# The MTHDS Test Corpus tag vocabulary — the closed set an entry's `covers` draws from.+#+# GENERATED by `pipelex-dev generate-corpus-vocabulary`. Do not edit this file: the generated+# namespaces are derived from the runtime's own registries, and every exclusion reason is+# declared in the generator's source.+#+# This file ships in the pipelex wheel, so it names a document its readers can reach:+# https://docs.pipelex.com/contribute/mthds-test-corpus/ — which in turn points at the+# cross-repo contract, `docs/specs/mthds-test-corpus.md` in the Pipelex workspace repo.++[native.dynamic]+code = "Dynamic"++[native.text]+code = "Text"++[native.image]+code = "Image"++[native.document]+code = "Document"++[native.html]+code = "Html"++[native.text_and_images]+code = "TextAndImages"++[native.number]+code = "Number"++[native.yes_no]+code = "YesNo"++[native.date]+code = "Date"++[native.time]+code = "Time"++[native.page]+code = "Page"++[native.json]+code = "JSON"++[native.search_result]+code = "SearchResult"++[native.anything]+code = "Anything"++[native.composite]+code = "Composite"diff --git a/tests/e2e/pipelex/cli/test_date_inputs_run.py b/tests/e2e/pipelex/cli/test_date_inputs_run.pyindex 2ed734ae1..23e8f579e 100644--- a/tests/e2e/pipelex/cli/test_date_inputs_run.py+++ b/tests/e2e/pipelex/cli/test_date_inputs_run.py@@ -19,9 +19,14 @@ from pathlib import Pathimport pytest+from pipelex.test_extras.mthds_corpus.loader import get_entry+REPO_ROOT = Path(__file__).resolve().parents[4]PIPELEX_BIN = REPO_ROOT / ".venv" / "bin" / "pipelex"-BUNDLE_SRC = REPO_ROOT / "tests" / "e2e" / "pipelex" / "pipes" / "date" / "date_departure"+# The bundle is a corpus entry: the corpus is the single source for language-level `.mthds`+# methods. Only the bundle file is staged — the entry's manifest and canonical inputs are+# corpus metadata, and this test supplies its own `inputs.toml`.+BUNDLE_SRC = get_entry(name="native_date_departure").bundle_path# A bare top-level TOML offset-datetime literal — the loader converts it into a native Date input.INPUTS_TOML = "departure = 2026-07-07T15:40:00+02:00\n"@@ -31,7 +36,8 @@ class TestDateInputsRun:@pytest.mark.gha_disableddef test_date_toml_literal_input_dry_run(self, tmp_path: Path) -> None:staged = tmp_path / "date_departure"- shutil.copytree(BUNDLE_SRC, staged)+ staged.mkdir()+ shutil.copy2(BUNDLE_SRC, staged / BUNDLE_SRC.name)inputs_toml = staged / "inputs.toml"inputs_toml.write_text(INPUTS_TOML, encoding="utf-8")diff --git a/tests/e2e/pipelex/cli/test_yes_no_inputs_run.py b/tests/e2e/pipelex/cli/test_yes_no_inputs_run.pyindex ce6f89e3e..f0cc22c8a 100644--- a/tests/e2e/pipelex/cli/test_yes_no_inputs_run.py+++ b/tests/e2e/pipelex/cli/test_yes_no_inputs_run.py@@ -19,9 +19,14 @@ from pathlib import Pathimport pytest+from pipelex.test_extras.mthds_corpus.loader import get_entry+REPO_ROOT = Path(__file__).resolve().parents[4]PIPELEX_BIN = REPO_ROOT / ".venv" / "bin" / "pipelex"-BUNDLE_SRC = REPO_ROOT / "tests" / "e2e" / "pipelex" / "pipes" / "yes_no" / "yes_no_judgment"+# The bundle is a corpus entry: the corpus is the single source for language-level `.mthds`+# methods. Only the bundle file is staged — the entry's manifest and canonical inputs are+# corpus metadata, and this test supplies its own `inputs.toml`.+BUNDLE_SRC = get_entry(name="native_yes_no_urgent_message").bundle_pathINPUTS_TOML = """[verdict]concept = "YesNo"@@ -33,7 +38,8 @@ class TestYesNoInputsRun:@pytest.mark.gha_disableddef test_yes_no_envelope_input_dry_run(self, tmp_path: Path) -> None:staged = tmp_path / "yes_no_judgment"- shutil.copytree(BUNDLE_SRC, staged)+ staged.mkdir()+ shutil.copy2(BUNDLE_SRC, staged / BUNDLE_SRC.name)inputs_toml = staged / "inputs.toml"inputs_toml.write_text(INPUTS_TOML, encoding="utf-8")diff --git a/tests/e2e/pipelex/pipes/date/test_date_dry_run.py b/tests/e2e/pipelex/pipes/date/test_date_dry_run.pyindex 773fd074d..4dbf07191 100644--- a/tests/e2e/pipelex/pipes/date/test_date_dry_run.py+++ b/tests/e2e/pipelex/pipes/date/test_date_dry_run.py@@ -7,7 +7,6 @@ Two shapes, both keyless (dry mode mocks only the cogt leaf and rides the live r"""import datetime-from pathlib import Pathimport pytest@@ -15,8 +14,11 @@ from pipelex.core.stuffs.date_content import DateContentfrom pipelex.pipeline.pipeline_response import RunStatefrom pipelex.pipeline.runner import PipelexMTHDSProtocolfrom pipelex.system.pipe_run_mode import PipeRunMode+from pipelex.test_extras.mthds_corpus.loader import get_entry-_FIXTURE_DIR = Path(__file__).parent / "date_departure"+# The bundle is a corpus entry, not a fixture local to this test: the corpus is the single source+# for language-level `.mthds` methods, and covering `native.date` is exactly why the entry exists.+_FIXTURE_DIR = get_entry(name="native_date_departure").directory@pytest.mark.asyncio(loop_scope="class")diff --git a/tests/e2e/pipelex/pipes/yes_no/test_yes_no_dry_run.py b/tests/e2e/pipelex/pipes/yes_no/test_yes_no_dry_run.pyindex a06dcdb71..1f3e640ca 100644--- a/tests/e2e/pipelex/pipes/yes_no/test_yes_no_dry_run.py+++ b/tests/e2e/pipelex/pipes/yes_no/test_yes_no_dry_run.py@@ -6,15 +6,16 @@ Two shapes, both keyless (dry mode mocks only the cogt leaf and rides the live rbool flows through the factory's YesNo arm and renders inline in the downstream prompt."""-from pathlib import Path-import pytestfrom pipelex.pipeline.pipeline_response import RunStatefrom pipelex.pipeline.runner import PipelexMTHDSProtocolfrom pipelex.system.pipe_run_mode import PipeRunMode+from pipelex.test_extras.mthds_corpus.loader import get_entry-_FIXTURE_DIR = Path(__file__).parent / "yes_no_judgment"+# The bundle is a corpus entry, not a fixture local to this test: the corpus is the single source+# for language-level `.mthds` methods, and covering `native.yes_no` is exactly why the entry exists.+_FIXTURE_DIR = get_entry(name="native_yes_no_urgent_message").directory@pytest.mark.asyncio(loop_scope="class")diff --git a/tests/integration/pipelex/test_extras/test_mthds_corpus_entries.py b/tests/integration/pipelex/test_extras/test_mthds_corpus_entries.pynew file mode 100644index 000000000..d2d34394c--- /dev/null+++ b/tests/integration/pipelex/test_extras/test_mthds_corpus_entries.py@@ -0,0 +1,85 @@+"""The MTHDS Test Corpus entry-validation gate: every entry means what its manifest claims.++Contract: ``docs/specs/mthds-test-corpus.md`` (workspace root), section "Gates" →+"Corpus-side: entry validation".++It drives the in-process validation engine — the one behind ``pipelex validate bundle`` — and+never a hosted API: the corpus is defined against the local runtime, and a deployed API lags+it, so an API verdict would call a correct entry invalid on the day its feature lands.++The gate runs over **every** entry regardless of tier, so an ``inference``-tier entry that+stopped validating is caught in the default suite without spending a token.+"""++from pathlib import Path++import pytest++from pipelex.pipeline.exceptions import ValidateBundleError+from pipelex.pipeline.validate_bundle import validate_bundle+from pipelex.test_extras.mthds_corpus.loader import CorpusEntry, iter_entries+from pipelex.test_extras.mthds_corpus.manifest import EntryValidity++_VALID_ENTRIES = list(iter_entries(validity=EntryValidity.VALID))+_INVALID_ENTRIES = list(iter_entries(validity=EntryValidity.INVALID))+++def _entry_id(entry: CorpusEntry) -> str:+ return entry.name++[email protected](loop_scope="class")+class TestMthdsCorpusEntries:+ @pytest.mark.parametrize("entry", _VALID_ENTRIES, ids=_entry_id)+ async def test_valid_entry_validates(self, entry: CorpusEntry) -> None:+ result = await validate_bundle(mthds_file_path=entry.bundle_path, library_dirs=[entry.directory])++ assert result.blueprints, f"Corpus entry '{entry.name}' produced no blueprint"++ @pytest.mark.parametrize("entry", _INVALID_ENTRIES, ids=_entry_id)+ async def test_invalid_entry_fails_with_exactly_its_declared_error(self, entry: CorpusEntry) -> None:+ """Red both ways: when the entry fails differently, and when it accidentally validates.++ An invalid entry is surgically authored to trigger exactly one error, so a widened blast+ radius (two errors now, one of them the declared one) fails here too — that is the point+ of the discipline, not an accident of the assertion.+ """+ with pytest.raises(ValidateBundleError) as raised:+ await validate_bundle(mthds_file_path=entry.bundle_path, library_dirs=[entry.directory])++ observed = [item.error_type for item in raised.value.to_error_report().validation_errors or []]+ assert observed == [entry.manifest.expected_error], (+ f"Corpus entry '{entry.name}' declares expected_error={entry.manifest.expected_error!r} but produced {observed!r}"+ )++ async def test_the_invalid_arm_bites_on_a_deliberately_broken_bundle(self, tmp_path: Path) -> None:+ """No corpus entry is invalid yet, so the invalid arm is proven here instead.++ The `error.*` namespace an invalid entry's `covers` would draw on does not exist until+ the error-type registry lands, and an invalid entry tagged with a native it does not+ exist for would be a lie in the manifest. The machinery the arm above depends on — that+ a broken bundle raises, and that the raised error projects the exact wire `error_type`+ an `expected_error` is compared against — is exercised against a bundle built for the+ purpose, so the arm is not dead code waiting on Phase 3.+ """+ bundle_path = tmp_path / "bundle.mthds"+ bundle_path.write_text(+ """domain = "harbour_moorings"+description = "A berth note whose prompt reaches for a variable the pipe never declares"+main_pipe = "write_berth_note"++[pipe.write_berth_note]+type = "PipeLLM"+description = "Write a berth note"+inputs = { vessel = "Text" }+output = "Text"+prompt = "Write a berth note for $vessel arriving on $tide."+""",+ encoding="utf-8",+ )++ with pytest.raises(ValidateBundleError) as raised:+ await validate_bundle(mthds_file_path=bundle_path, library_dirs=[tmp_path])++ observed = [item.error_type for item in raised.value.to_error_report().validation_errors or []]+ assert observed == ["missing_input_variable"]diff --git a/tests/unit/pipelex/test_extras/test_mthds_corpus_exhaustivity.py b/tests/unit/pipelex/test_extras/test_mthds_corpus_exhaustivity.pynew file mode 100644index 000000000..46ebebc78--- /dev/null+++ b/tests/unit/pipelex/test_extras/test_mthds_corpus_exhaustivity.py@@ -0,0 +1,31 @@+"""The corpus-side exhaustivity gate — the gate the whole corpus exists for.++Contract: ``docs/specs/mthds-test-corpus.md`` (workspace root), section "Gates" →+"Corpus-side: exhaustivity".++The moment a runtime registry grows a code, regeneration adds its tag and this gate stays red+until a focused entry covers it. It is why "the `Time` native concept has no fixture anywhere"+cannot happen again silently.+"""++from pipelex.test_extras.mthds_corpus.loader import iter_entries+from pipelex.test_extras.mthds_corpus.manifest import EntryGranularity+from pipelex.test_extras.mthds_corpus.vocabulary import load_vocabulary+++class TestMthdsCorpusExhaustivity:+ def test_every_required_tag_has_a_focused_entry(self) -> None:+ covered_by_focused: set[str] = set()+ for entry in iter_entries(granularity=EntryGranularity.FOCUSED):+ covered_by_focused.update(entry.manifest.covers)+ uncovered = sorted(load_vocabulary().required_tags - covered_by_focused)+ assert not uncovered, f"Vocabulary tags with no focused corpus entry: {', '.join(uncovered)}"++ def test_every_tag_an_entry_covers_exists_in_the_vocabulary(self) -> None:+ declared = load_vocabulary().tags+ unknown: set[str] = set()+ for entry in iter_entries():+ for tag in entry.manifest.covers:+ if tag not in declared:+ unknown.add(f"{entry.name}: {tag}")+ assert not unknown, f"Corpus entries cover tags that are not in the vocabulary: {', '.join(sorted(unknown))}"diff --git a/tests/unit/pipelex/test_extras/test_mthds_corpus_loader.py b/tests/unit/pipelex/test_extras/test_mthds_corpus_loader.pynew file mode 100644index 000000000..f0718a102--- /dev/null+++ b/tests/unit/pipelex/test_extras/test_mthds_corpus_loader.py@@ -0,0 +1,87 @@+"""Loading the corpus and taking a view of it.++Contract: ``docs/specs/mthds-test-corpus.md`` (workspace root), sections "Entry layout" and+"Loader API".++The bundles themselves are validated by the integration-tier entry gate; this module pins the+layout rules and the filter semantics a consumer selects on.+"""++import pytest++from pipelex.test_extras.mthds_corpus.exceptions import CorpusEntryError+from pipelex.test_extras.mthds_corpus.loader import get_entry, iter_entries+from pipelex.test_extras.mthds_corpus.manifest import EntryGranularity, EntryTier, EntryValidity+++class TestMthdsCorpusLoader:+ def test_the_corpus_is_not_empty(self) -> None:+ """Anti-vacuity: every sweep below would pass trivially over an empty corpus."""+ assert list(iter_entries())++ def test_every_entry_loads_and_resolves_its_bundle(self) -> None:+ """Layout enforcement: the manifest name matches the directory, and the bundle resolves."""+ for entry in iter_entries():+ assert entry.manifest.name == entry.directory.name+ assert entry.bundle_path.is_file()++ def test_entries_are_yielded_in_name_order(self) -> None:+ names = [entry.name for entry in iter_entries()]+ assert names == sorted(names)++ def test_entry_names_are_unique(self) -> None:+ names = [entry.name for entry in iter_entries()]+ assert len(names) == len(set(names))++ def test_get_entry_returns_the_named_entry_and_refuses_an_unknown_one(self) -> None:+ first_name = next(iter(iter_entries())).name+ assert get_entry(name=first_name).name == first_name+ with pytest.raises(CorpusEntryError):+ get_entry(name="no_such_entry")++ def test_tag_filter_requires_every_requested_tag(self) -> None:+ for entry in iter_entries(tags=frozenset({"native.text"})):+ assert "native.text" in entry.manifest.covers+ assert not list(iter_entries(tags=frozenset({"native.no_such_tag"})))++ def test_tier_filter_is_a_ceiling_and_really_excludes(self) -> None:+ """A consumer running at a tier runs everything it can afford, not only what costs exactly that.++ Each arm is compared against the set the manifests independently say it should hold, so a+ filter that ignored its argument — returning the whole corpus at every tier — fails here+ instead of passing because the corpus happens to sit at one tier.+ """+ all_names = {entry.name for entry in iter_entries()}+ for tier in EntryTier:+ selected = {entry.name for entry in iter_entries(tier=tier)}+ affordable = {entry.name for entry in iter_entries() if entry.manifest.tier.rank <= tier.rank}+ assert selected == affordable+ assert {entry.name for entry in iter_entries(tier=EntryTier.INFERENCE)} == all_names++ def test_validity_arms_partition_the_corpus(self) -> None:+ """Disjoint and exhaustive, which a filter that ignored its argument could not be.++ Stated as a partition rather than as "every selected entry matches", because that weaker+ form passes for a no-op filter whenever the corpus happens to sit on one side of the axis —+ and today every entry is valid, so the weaker form would assert nothing at all.+ """+ all_names = {entry.name for entry in iter_entries()}+ valid_names = {entry.name for entry in iter_entries(validity=EntryValidity.VALID)}+ invalid_names = {entry.name for entry in iter_entries(validity=EntryValidity.INVALID)}+ assert valid_names | invalid_names == all_names+ assert not valid_names & invalid_names++ def test_granularity_arms_partition_the_corpus(self) -> None:+ all_names = {entry.name for entry in iter_entries()}+ focused_names = {entry.name for entry in iter_entries(granularity=EntryGranularity.FOCUSED)}+ composite_names = {entry.name for entry in iter_entries(granularity=EntryGranularity.COMPOSITE)}+ assert focused_names | composite_names == all_names+ assert not focused_names & composite_names++ def test_filters_compose_conjunctively(self) -> None:+ selected = list(iter_entries(validity=EntryValidity.VALID, granularity=EntryGranularity.FOCUSED, tier=EntryTier.DRY))+ assert selected+ for entry in selected:+ assert entry.manifest.validity is EntryValidity.VALID+ assert entry.manifest.granularity is EntryGranularity.FOCUSED+ assert entry.manifest.tier.rank <= EntryTier.DRY.rankdiff --git a/tests/unit/pipelex/test_extras/test_mthds_corpus_manifest.py b/tests/unit/pipelex/test_extras/test_mthds_corpus_manifest.pynew file mode 100644index 000000000..937cb250b--- /dev/null+++ b/tests/unit/pipelex/test_extras/test_mthds_corpus_manifest.py@@ -0,0 +1,61 @@+"""The strict `entry.toml` model.++Contract: ``docs/specs/mthds-test-corpus.md`` (workspace root), section "The entry manifest".++Strictness is the point: an unknown key is an error rather than a forward-compatibility+affordance, so a new field is a change to the spec and to the model in the same commit.+"""++import pytest+from pydantic import ValidationError++from pipelex.test_extras.mthds_corpus.manifest import CorpusEntryManifest, EntryTier, EntryValidity++_MINIMAL_MANIFEST_FIELDS: dict[str, object] = {+ "name": "shipping_manifest_totals",+ "description": "A description.",+ "validity": "valid",+ "tier": "static",+ "granularity": "focused",+ "covers": ["native.text"],+}+++def _manifest_fields(**overrides: object) -> dict[str, object]:+ return {**_MINIMAL_MANIFEST_FIELDS, **overrides}+++class TestMthdsCorpusManifest:+ def test_minimal_manifest_parses(self) -> None:+ manifest = CorpusEntryManifest.model_validate(_manifest_fields())+ assert manifest.tier is EntryTier.STATIC+ assert manifest.validity is EntryValidity.VALID++ def test_unknown_key_is_rejected(self) -> None:+ with pytest.raises(ValidationError):+ CorpusEntryManifest.model_validate(_manifest_fields(unexpected_field="x"))++ def test_empty_covers_is_rejected(self) -> None:+ """An entry that covers nothing has no reason to exist."""+ with pytest.raises(ValidationError):+ CorpusEntryManifest.model_validate(_manifest_fields(covers=[]))++ def test_repeated_tag_is_rejected(self) -> None:+ with pytest.raises(ValidationError):+ CorpusEntryManifest.model_validate(_manifest_fields(covers=["native.text", "native.text"]))++ def test_non_snake_case_name_is_rejected(self) -> None:+ with pytest.raises(ValidationError):+ CorpusEntryManifest.model_validate(_manifest_fields(name="ShippingManifestTotals"))++ def test_invalid_entry_must_declare_its_expected_error(self) -> None:+ with pytest.raises(ValidationError):+ CorpusEntryManifest.model_validate(_manifest_fields(validity="invalid"))++ def test_valid_entry_must_not_declare_an_expected_error(self) -> None:+ """A valid entry carrying an expected error would be claiming both verdicts at once."""+ with pytest.raises(ValidationError):+ CorpusEntryManifest.model_validate(_manifest_fields(expected_error="missing_input_variable"))++ def test_tiers_rank_cheapest_first(self) -> None:+ assert EntryTier.STATIC.rank < EntryTier.DRY.rank < EntryTier.OFFLINE.rank < EntryTier.INFERENCE.rankdiff --git a/tests/unit/pipelex/test_extras/test_mthds_corpus_vocabulary.py b/tests/unit/pipelex/test_extras/test_mthds_corpus_vocabulary.pynew file mode 100644index 000000000..2dfa0df2f--- /dev/null+++ b/tests/unit/pipelex/test_extras/test_mthds_corpus_vocabulary.py@@ -0,0 +1,54 @@+"""The corpus tag vocabulary: normalization, generation drift, and the exclusion mechanism.++Contract: ``docs/specs/mthds-test-corpus.md`` (workspace root), section "Tag vocabulary".++No `native.*` code is excluded today — every one of them turned out to support a meaningful+focused entry — so the exclusion mechanism's semantics are pinned here rather than left to be+discovered broken on the day `operator.*` or `error.*` first needs it. The other half of the+mechanism, that an exclusion cannot outlive the code it names, is held by the generator keying+its exclusion map on the enum: a removed code breaks that module at import, which no test can+provoke without smuggling in a key the annotation forbids.+"""++from pipelex.cli.dev_cli.commands.generate_corpus_vocabulary_cmd import (+ generate_corpus_vocabulary_content,+ normalize_registry_code,+)+from pipelex.test_extras.mthds_corpus.vocabulary import CorpusVocabulary, VocabularyTag, vocabulary_path+++class TestMthdsCorpusVocabulary:+ def test_normalization_splits_words_and_keeps_acronyms_whole(self) -> None:+ """The acronym rule is what keeps `JSON` from becoming `j_s_o_n`."""+ assert normalize_registry_code(code="Text") == "text"+ assert normalize_registry_code(code="Html") == "html"+ assert normalize_registry_code(code="YesNo") == "yes_no"+ assert normalize_registry_code(code="TextAndImages") == "text_and_images"+ assert normalize_registry_code(code="SearchResult") == "search_result"+ assert normalize_registry_code(code="JSON") == "json"+ assert normalize_registry_code(code="JSONSchema") == "json_schema"++ def test_committed_vocabulary_matches_a_fresh_generation(self) -> None:+ """The committed file is byte-identical to what the generator produces right now.++ This is the whole freshness story for the vocabulary. It is committed and ships in the+ wheel, so regenerating in memory here is a stronger gate than a `check-` CLI command+ would be, and needs no Make target of its own.+ """+ committed = vocabulary_path().read_text(encoding="utf-8")+ assert committed == generate_corpus_vocabulary_content(), (+ "The committed corpus vocabulary is out of date. Run `make generate-corpus-vocabulary`."+ )++ def test_an_excluded_tag_stays_usable_but_stops_being_required(self) -> None:+ """Exclusions are visible decisions: the tag stays in the vocabulary, it just stops being owed an entry."""+ vocabulary = CorpusVocabulary(+ namespaces={+ "native": {+ "text": VocabularyTag(code="Text"),+ "anything": VocabularyTag(code="Anything", excluded="No standalone focused entry could exercise it."),+ }+ }+ )+ assert vocabulary.tags == frozenset({"native.text", "native.anything"})+ assert vocabulary.required_tags == frozenset({"native.text"})