Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
feature/Plain-stored-names · L-261009-677091 (#1387)
Louis Choquel7c4adc2cb042
diff --git a/.pipelex/plxt.toml b/.pipelex/plxt.tomlindex afc14b3a8..17b6e5609 100644--- a/.pipelex/plxt.toml+++ b/.pipelex/plxt.toml@@ -59,8 +59,13 @@ exclude = ["**/mthds_corpus/entries/invalid_binding_step_pipe_and_from/*.mthds","**/mthds_corpus/entries/invalid_binding_step_result_not_plain/*.mthds","**/mthds_corpus/entries/invalid_binding_step_without_result/*.mthds",+ "**/mthds_corpus/entries/invalid_input_name_batch_item_name_not_plain/*.mthds",+ "**/mthds_corpus/entries/invalid_input_name_branch_batch_as_reserved/*.mthds",+ "**/mthds_corpus/entries/invalid_input_name_branch_result_not_plain/*.mthds","**/mthds_corpus/entries/invalid_input_name_lone_dotted/*.mthds","**/mthds_corpus/entries/invalid_input_name_redundant_dotted/*.mthds",+ "**/mthds_corpus/entries/invalid_input_name_step_batch_as_dotted/*.mthds",+ "**/mthds_corpus/entries/invalid_input_name_step_result_not_plain/*.mthds","**/mthds_corpus/entries/invalid_missing_pipe_type/*.mthds","**/mthds_corpus/entries/invalid_unknown_pipe_type/*.mthds",] # Glob patterns for files to ignore.diff --git a/CHANGELOG.md b/CHANGELOG.mdindex abf595e03..7fc730cb2 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -4,6 +4,7 @@### Changed+- **Stored names are plain input names (Breaking)**: every name under which a step stores a value in working memory, a `PipeSequence` step's or a `PipeParallel` branch's `result`, the `batch_as` of either, and a `PipeBatch`'s `input_item_name`, must now be a plain input name matching `[a-z][a-z0-9_]*`, as a binding step's `result` already had to, and a name in any other form, such as `Pages`, `doc.pages` or `_pages`, is refused as `invalid_input_name` wherever a bundle loads, and by the MTHDS JSON Schema, which now carries the input-name pattern on these fields. Rename such a name to a plain one, such as `pages`, along with every input, binding path and `batch_over` that reads it; a plain `batch_over` is still refused only when it starts with the reserved `_bound_`.- **Hosted-run guidance in messages and help**: the next step of a refused Pipelex API key names `pipelex login` and the working directory's `.env` rather than a shell export, which a `.env` file setting the key overrides, and `pipelex login` names `[run] execution = "hosted"` rather than `pipelex init` to make hosted runs the default. A CLI run missing a provider key, and the `pipelex doctor` report, offer the hosted Pipelex API as a way out; a failed `pipelex-agent` boot points at `pipelex-agent migrate` and `pipelex-agent init` rather than the interactive `pipelex init config`; and `FormerReleaseConfigError` carries its remedy, `pipelex migrate`, as its `user_action`. The `run` help of both CLIs names hosted runs, catalog ids, local method directories and TOML or inline inputs, and `pipelex-agent --help` describes the program rather than printing an internal note.## [v0.79.1] - 2026-10-09diff --git a/docs/building-methods/pipes/pipe-controllers/PipeBatch.md b/docs/building-methods/pipes/pipe-controllers/PipeBatch.mdindex d3c86a056..ae0f68e3e 100644--- a/docs/building-methods/pipes/pipe-controllers/PipeBatch.md+++ b/docs/building-methods/pipes/pipe-controllers/PipeBatch.md@@ -50,7 +50,7 @@ This is the batch arm of the "route or skip" pattern: batch over items with a co| `output` | string | The output concept produced by the batch operation. | Yes || `branch_pipe_code` | string | The name of the single pipe to execute for each item in the input list. | Yes || `input_list_name` | string | The name of the input list to iterate over. Must match one of the keys in `inputs`, so it is a plain input name, a `snake_case` identifier matching `[a-z][a-z0-9_]*`, and never a path into a field: a dotted name such as `catalog.pages` is refused as `invalid_input_name`. To map a pipe over a list held in a field, have the calling sequence bind it to the batch's list with a binding step, or run the branch pipe in a sequence step whose dotted `batch_over` binds the list and batches over it (see [Batching over a field](PipeSequence.md#batching-over-a-field)). Typically a plural noun (e.g. `articles`). | Yes |-| `input_item_name` | string | The name that an individual item from the list will have inside its execution branch — this is how the branch pipe finds its input. Must differ from `input_list_name` and from every key in `inputs`, and never starts with `_bound_`, a prefix reserved for the bound list of a dotted `batch_over` (see [Batching over a field](PipeSequence.md#batching-over-a-field)), which is refused as `invalid_input_name`. Typically the singular form of the list name (e.g. `article`). | Yes |+| `input_item_name` | string | The name that an individual item from the list will have inside its execution branch — this is how the branch pipe finds its input. It is a [stored name](PipeSequence.md#stored-names), so a plain input name like `input_list_name`, a `snake_case` identifier matching `[a-z][a-z0-9_]*`: a name in any other form, such as `Article`, `issue.article` or `_bound_article`, is refused as `invalid_input_name`. Must differ from `input_list_name` and from every key in `inputs`. Typically the singular form of the list name (e.g. `article`). | Yes |### Example: Summarizing a list of articlesdiff --git a/docs/building-methods/pipes/pipe-controllers/PipeParallel.md b/docs/building-methods/pipes/pipe-controllers/PipeParallel.mdindex daec6878e..1ea409563 100644--- a/docs/building-methods/pipes/pipe-controllers/PipeParallel.md+++ b/docs/building-methods/pipes/pipe-controllers/PipeParallel.md@@ -62,7 +62,9 @@ Each entry in the `branches` array is a table with the following keys:| Key | Type | Description | Required || -------- | ------ | ---------------------------------------------------------------------------------------- | -------- || `pipe` | string | The name of the pipe to execute for this branch. | Yes |-| `result` | string | The name for this branch's output. Must be unique within the `PipeParallel` definition. | Yes |+| `result` | string | The name for this branch's output, a plain `snake_case` input name such as `sentiment`. Must be unique within the `PipeParallel` definition. | Yes |++A branch's `result` and `batch_as` are [stored names](PipeSequence.md#stored-names), as on a sequence step: the branch stores a value under each for a pipe to read through an input, so each is a plain input name matching `[a-z][a-z0-9_]*`, and one in any other form, such as `Sentiment` or `text.sentiment`, is refused with `invalid_input_name`. Since the combined output holds each branch result under its `result` name, the fields of a structured output concept that the branches fill have plain names too.A branch is always a pipe step. A [binding step](PipeSequence.md#binding-steps), `{ from = "invoice.total", result = "total_amount" }`, is refused in `branches` with `binding_step_invalid`: the branches run at once, and a binding orders a value before the steps that read it, so bind the value in the calling sequence, in a step before the `PipeParallel`, and have the branch read the bound name. A branch may carry `batch_over` and `batch_as`, but its `batch_over` is a plain name: a dotted one, such as `catalog.pages`, binds before it batches (see [Batching over a field](PipeSequence.md#batching-over-a-field)), so it is refused with `binding_step_invalid` too, and the calling sequence binds the list before the `PipeParallel`.diff --git a/docs/building-methods/pipes/pipe-controllers/PipeSequence.md b/docs/building-methods/pipes/pipe-controllers/PipeSequence.mdindex db1d93e15..c37b2aeea 100644--- a/docs/building-methods/pipes/pipe-controllers/PipeSequence.md+++ b/docs/building-methods/pipes/pipe-controllers/PipeSequence.md@@ -39,11 +39,17 @@ A pipe step is a table with the following keys:| Key | Type | Description | Required || -------- | ------ | ------------------------------------------------------------------ | -------- || `pipe` | string | The name of the pipe to execute for this step. | Yes |-| `result` | string | The name to give to this step's output in the working memory. When omitted, the output is stored only in the unnamed `main_stuff` slot (the default output), so later steps can pick it up as their implicit input but cannot reference it by a dedicated name. | No |+| `result` | string | The name to give to this step's output in the working memory, a plain `snake_case` input name such as `pages` (see [Stored names](#stored-names)). When omitted, the output is stored only in the unnamed `main_stuff` slot (the default output), so later steps can pick it up as their implicit input but cannot reference it by a dedicated name. | No || `nb_output` | integer | Request a fixed number of outputs from this step's pipe. Cannot be combined with `multiple_output`. | No || `multiple_output` | boolean | Request a variable number of outputs from this step's pipe (the model decides how many). Cannot be combined with `nb_output`. | No || `batch_over` | string | The list in the working memory to batch this step over, running the pipe once per item: a name, or a dotted path to a list held in a field, such as `catalog.pages`, which is a binding followed by a batch (see [Batching over a field](#batching-over-a-field)). Must be provided together with `batch_as`. See [Understanding Multiplicity](../understanding-multiplicity.md). | No |-| `batch_as` | string | The name each item takes in the working memory during a `batch_over` run. Must differ from `batch_over` (e.g. `batch_over = "items"`, `batch_as = "item"`). Like `result` and `batch_over`, it never starts with `_bound_`, a prefix reserved for the bound list of a dotted `batch_over`. | No |+| `batch_as` | string | The name each item takes in the working memory during a `batch_over` run, under which the step's pipe reads it through an input: a plain `snake_case` input name, like `result` (see [Stored names](#stored-names)). Must differ from `batch_over` (e.g. `batch_over = "items"`, `batch_as = "item"`). | No |++#### Stored names++A step's `result` and `batch_as` are **stored names**: the step stores a value in the working memory under each, for a later pipe to read through an input. An input name is always a plain `snake_case` identifier matching `[a-z][a-z0-9_]*`, so a stored name takes the same form, and every value a step stores can be read. A `result` or a `batch_as` in any other form, such as `Pages`, a dotted `doc.pages` or an underscore-led `_pages`, is refused with `invalid_input_name`, which the schema catches too, and the message suggests a plain name. Rename it, such as to `pages`, along with every input, binding path and `batch_over` that reads it. The same rule holds a [`PipeParallel`](PipeParallel.md) branch's `result` and `batch_as`, a [`PipeBatch`](PipeBatch.md)'s `input_item_name` and a binding step's `result`, which a binding step that breaks it reports as `binding_step_invalid`.++A plain `batch_over` reads a name rather than storing one, so it is not held to this form, but it never starts with `_bound_`, a prefix reserved for the list a dotted `batch_over` binds (see [Batching over a field](#batching-over-a-field)), and one that does is refused with `invalid_input_name` too.#### Binding steps@@ -197,7 +203,7 @@ So a dotted `batch_over` shares everything a binding does:- A path outside the path grammar, such as `catalog..pages`, is refused with `binding_step_invalid`, which the schema catches too.- In the execution graph, the binding is a node of kind `binding`, fed by the root's producer, and it feeds the batch.-The private name starts with an underscore, `_bound_catalog_pages` for `catalog.pages`, so no template or input can read it: it shows only in the working memory the run leaves and in the graph. The `_bound_` prefix is reserved for these names. A nested `PipeSequence` binds in the working memory of the sequence calling it, so a name of the caller taking the prefix could be overwritten by the list a sequence it calls binds: a pipe step's `result`, `batch_as` or plain `batch_over`, on a sequence step or a `PipeParallel` branch, and a [`PipeBatch`](PipeBatch.md)'s `input_item_name` that starts with `_bound_` are refused with `invalid_input_name`, which the schema catches too, and the message asks for another name. Only a sequence's steps bind, so a [`PipeParallel`](PipeParallel.md) branch carries a plain `batch_over` only, and a dotted one there is refused with `binding_step_invalid`: bind the list in a step before the `PipeParallel`, and batch the branch over the bound name.+The private name starts with an underscore, `_bound_catalog_pages` for `catalog.pages`, so no template or input can read it: it shows only in the working memory the run leaves and in the graph. The `_bound_` prefix is reserved for these names. A nested `PipeSequence` binds in the working memory of the sequence calling it, so a name of the caller taking the prefix could be overwritten by the list a sequence it calls binds. No name a step stores a value under can take it, since a [stored name](#stored-names) is a plain input name, which never starts with an underscore, and a plain `batch_over` that starts with `_bound_`, on a sequence step or a `PipeParallel` branch, is refused with `invalid_input_name`, which the schema catches too, and the message asks for another name. Only a sequence's steps bind, so a [`PipeParallel`](PipeParallel.md) branch carries a plain `batch_over` only, and a dotted one there is refused with `binding_step_invalid`: bind the list in a step before the `PipeParallel`, and batch the branch over the bound name.### Examplediff --git a/docs/building-methods/pipes/working-memory.md b/docs/building-methods/pipes/working-memory.mdindex 414e40905..668309710 100644--- a/docs/building-methods/pipes/working-memory.md+++ b/docs/building-methods/pipes/working-memory.md@@ -35,6 +35,8 @@ How does data get from `generate_tagline` to `extract_keywords_from_tagline`? ThThis mechanism allows you to chain pipes together, creating a flow of information through your pipeline.+Because a later pipe reads a stored value through an input, every name a value is stored under is a plain input name, like `tagline`: a `result`, a `batch_as` or a `PipeBatch`'s `input_item_name` written in any other form, such as `Tagline` or `marketing.tagline`, is refused with `invalid_input_name` (see [Stored names](pipe-controllers/PipeSequence.md#stored-names)).+## Working Memory Lifecycle* **Creation**: The Working Memory is created at the start of a pipeline run.diff --git a/pipelex/core/memory/working_memory.py b/pipelex/core/memory/working_memory.pyindex 71f75d171..30c881379 100644--- a/pipelex/core/memory/working_memory.py+++ b/pipelex/core/memory/working_memory.py@@ -37,9 +37,10 @@ MAIN_STUFF_NAME = "main_stuff"BATCH_ITEM_STUFF_NAME = "BATCH_ITEM"# The prefix of the private name a PipeSequence binds a dotted `batch_over`'s list under: underscore-led, so no template or# input reads it, and never a param's, so working memory holds it like any stuff. The prefix is the runtime's alone: a nested-# sequence binds in its caller's working memory, so validation refuses any name an author writes taking it-# (`check_name_is_not_reserved`), and a caller's value is never overwritten by a list a sequence it calls binds. This one-# constant spells the prefix for the name `make_private_binding_name` generates, the params exemption below, and that refusal.+# sequence binds in its caller's working memory, so validation refuses any name an author writes taking it, a stored name by+# holding it to the plain input-name form (`check_stored_name`) and a plain `batch_over` by name (`check_name_is_not_reserved`),+# and a caller's value is never overwritten by a list a sequence it calls binds. This one constant spells the prefix for the+# name `make_private_binding_name` generates, the params exemption below, and those refusals.PRIVATE_BINDING_NAME_PREFIX = "_bound_"PRETTY_PRINT_MAX_LENGTH = 1000TEST_DUMMY_NAME = "dummy_result"diff --git a/pipelex/language/mthds_schema_generator.py b/pipelex/language/mthds_schema_generator.pyindex 4e6f4cb5c..5aba25154 100644--- a/pipelex/language/mthds_schema_generator.py+++ b/pipelex/language/mthds_schema_generator.py@@ -19,6 +19,7 @@ from pipelex.pipe_machinery.validation import (PARALLEL_BRANCH_BATCH_OVER_PATTERN,RESERVED_NAME_PATTERN,SEQUENCE_STEP_BATCH_OVER_PATTERN,+ STORED_NAME_PATTERN,)from pipelex.pipe_signature.pipe_signature_blueprint import PipeSignatureBlueprintfrom pipelex.tools.misc.package_utils import get_package_version@@ -48,9 +49,9 @@ _SIGNATURE_DEFINITION_NAME = PipeSignatureBlueprint.__name__# (`_constrain_batch_over`). No Python class carries the name, since steps and branches share one blueprint._PARALLEL_BRANCH_DEFINITION_NAME = "ParallelBranchBlueprint"-# The fields of a pipe step, and of a PipeParallel branch, naming a slot of working memory: the runtime refuses each when it-# takes the prefix reserved for the bound list of a dotted `batch_over` (`_reserve_private_binding_names`).-_PIPE_STEP_NAME_FIELDS: tuple[str, ...] = ("result", "batch_as", "batch_over")+# The fields of a pipe step, and of a PipeParallel branch, naming a slot of working memory the step stores a value under: each+# is a stored name, held to the input-name grammar (`_constrain_stored_names`).+_PIPE_STEP_STORED_NAME_FIELDS: tuple[str, ...] = ("result", "batch_as")def generate_mthds_schema() -> dict[str, Any]:@@ -74,8 +75,8 @@ def generate_mthds_schema() -> dict[str, Any]:schema = _convert_to_draft4(schema)schema = _patch_construct_schema(schema)schema = _constrain_input_names(schema)- # Before `_constrain_batch_over`, so the branch definition it copies from the pipe step's carries the reservation too.- schema = _reserve_private_binding_names(schema)+ # Before `_constrain_batch_over`, so the branch definition it copies from the pipe step's carries these constraints too.+ schema = _constrain_stored_names(schema)schema = _constrain_batch_over(schema)return _add_taplo_metadata(schema)@@ -309,31 +310,37 @@ def _constrain_input_names(schema: dict[str, Any]) -> dict[str, Any]:return schema-def _reserve_private_binding_names(schema: dict[str, Any]) -> dict[str, Any]:- """Refuse the names the runtime reserves for the bound list of a dotted `batch_over`, as the runtime does.+def _constrain_stored_names(schema: dict[str, Any]) -> dict[str, Any]:+ """Hold every stored name to the input-name grammar, and keep a plain `batch_over` off the reserved prefix, as the runtime does.- A pipe step's `result`, `batch_as` and plain `batch_over`, and a PipeBatch's `input_item_name`, never take the `_bound_`- prefix: the runtime refuses each as `invalid_input_name` when the bundle is parsed, which keeps that error type's- `fails_at = "schema"` true. Each is a Draft-4 `not` holding a `pattern` on the field's string arm: a negative lookahead in- the pattern itself would say the same, but not every validator's regex engine has one, while every Draft-4 validator- applies `not`, plxt's included. The other names an author writes into working memory, input names and a binding step's- `result`, already follow the plain-name grammar, which no underscore-led name matches.+ A pipe step's `result` and `batch_as`, the same fields on a PipeParallel branch, and a PipeBatch's `input_item_name` are+ stored names: a step stores a value under each for a pipe to read through an input, so each takes the plain input-name+ grammar (`STORED_NAME_PATTERN`), as a binding step's `result` does through its own field. The runtime refuses any other+ form as `invalid_input_name` when the bundle is parsed, which keeps that error type's `fails_at = "schema"` true, and no+ name of that grammar takes the `_bound_` prefix the runtime reserves for the bound list of a dotted `batch_over`, being+ never underscore-led. A plain `batch_over` reads a name rather than storing one, so no such grammar holds it, and it is+ kept off the prefix by a Draft-4 `not` holding a `pattern` on its string arm: a negative lookahead in the pattern itself+ would say the same, but not every validator's regex engine has one, while every Draft-4 validator applies `not`, plxt's+ included. `_constrain_batch_over` gives it its path grammar besides."""schema = copy.deepcopy(schema)definitions = schema.get("definitions", {})pipe_step_properties = definitions.get(SubPipeBlueprint.__name__, {}).get("properties", {})- for field_name in _PIPE_STEP_NAME_FIELDS:- _refuse_reserved_names(field_schema=pipe_step_properties.get(field_name))- _refuse_reserved_names(field_schema=definitions.get(PipeBatchBlueprint.__name__, {}).get("properties", {}).get("input_item_name"))+ for field_name in _PIPE_STEP_STORED_NAME_FIELDS:+ _set_string_arm_constraint(field_schema=pipe_step_properties.get(field_name), constraint={"pattern": STORED_NAME_PATTERN})+ _set_string_arm_constraint(field_schema=pipe_step_properties.get("batch_over"), constraint={"not": {"pattern": RESERVED_NAME_PATTERN}})+ batch_properties = definitions.get(PipeBatchBlueprint.__name__, {}).get("properties", {})+ _set_string_arm_constraint(field_schema=batch_properties.get("input_item_name"), constraint={"pattern": STORED_NAME_PATTERN})return schema-def _refuse_reserved_names(*, field_schema: dict[str, Any] | None) -> None:+def _set_string_arm_constraint(*, field_schema: dict[str, Any] | None, constraint: dict[str, Any]) -> None:+ """Add a constraint to every string arm of a field, the field itself standing for its one arm when it has no `anyOf`."""if field_schema is None:returnfor arm in field_schema.get("anyOf", [field_schema]):if arm.get("type") == "string":- arm["not"] = {"pattern": RESERVED_NAME_PATTERN}+ arm.update(copy.deepcopy(constraint))def _constrain_batch_over(schema: dict[str, Any]) -> dict[str, Any]:diff --git a/pipelex/pipe_controllers/batch/pipe_batch_blueprint.py b/pipelex/pipe_controllers/batch/pipe_batch_blueprint.pyindex 52e1bc491..f5e6c9dd4 100644--- a/pipelex/pipe_controllers/batch/pipe_batch_blueprint.py+++ b/pipelex/pipe_controllers/batch/pipe_batch_blueprint.py@@ -4,7 +4,7 @@ from typing_extensions import overridefrom pipelex.core.pipes.exceptions import PipeValidationErrorfrom pipelex.pipe_machinery.pipe_blueprint import PipeBlueprint-from pipelex.pipe_machinery.validation import check_input_list_name, check_name_is_not_reserved+from pipelex.pipe_machinery.validation import check_input_list_name, check_stored_namefrom pipelex.validation_error_types import PipeValidationErrorType@@ -35,9 +35,9 @@ class PipeBatchBlueprint(PipeBlueprint):if not self.input_item_name:msg = "Empty input item name is not allowed"raise ValueError(msg)- # The item is written into each branch's working memory under its name, so the name never takes the prefix the runtime- # reserves for the bound list of a dotted `batch_over`. The list's name needs no such check: it is a plain input name.- check_name_is_not_reserved(name=self.input_item_name, field_label="The PipeBatch's `input_item_name`")+ # The item is stored in each branch's working memory under its name, for the branch pipe to read through an input of+ # that name, so the name is a stored name and takes the plain input-name form.+ check_stored_name(name=self.input_item_name, field_label="The PipeBatch's `input_item_name`")# Specialization: catch the most common mistake (item name == list name) with a targeted messageif self.input_item_name == self.input_list_name:msg = (diff --git a/pipelex/pipe_controllers/binding/binding_step_blueprint.py b/pipelex/pipe_controllers/binding/binding_step_blueprint.pyindex f447b33d6..25b91ae9f 100644--- a/pipelex/pipe_controllers/binding/binding_step_blueprint.py+++ b/pipelex/pipe_controllers/binding/binding_step_blueprint.py@@ -4,7 +4,7 @@ from typing import Any, castfrom pydantic import BaseModel, ConfigDict, Field, SerializerFunctionWrapHandler, field_validator, model_serializerfrom pipelex.core.pipes.exceptions import PipeValidationError-from pipelex.pipe_machinery.validation import BINDING_PATH_PATTERN, BINDING_RESULT_PATTERN+from pipelex.pipe_machinery.validation import BINDING_PATH_PATTERN, STORED_NAME_PATTERNfrom pipelex.tools.misc.string_utils import FIELD_PATH_SEGMENT_REGEX, SNAKE_CASE_IDENTIFIER_REGEX, is_field_path, is_snake_casefrom pipelex.validation_error_types import PipeValidationErrorType@@ -81,7 +81,7 @@ class BindingStepBlueprint(BaseModel):)result: str = Field(description="The name under which the bound value is stored in working memory: a plain input name.",- json_schema_extra={"pattern": BINDING_RESULT_PATTERN},+ json_schema_extra={"pattern": STORED_NAME_PATTERN},)@field_validator("from_path", mode="after")diff --git a/pipelex/pipe_controllers/sequence/pipe_sequence.py b/pipelex/pipe_controllers/sequence/pipe_sequence.pyindex 4aa1a78d6..06a23bb81 100644--- a/pipelex/pipe_controllers/sequence/pipe_sequence.py+++ b/pipelex/pipe_controllers/sequence/pipe_sequence.py@@ -43,7 +43,7 @@ from pipelex.pipe_controllers.sequence.sequence_typed_flow import ()from pipelex.pipe_controllers.sub_pipe import SubPipefrom pipelex.pipe_machinery.memory_writes import MemoryWrite, SlotTaint, is_same_value_spec, taint_after_write-from pipelex.pipe_machinery.validation import is_valid_input_name+from pipelex.pipe_machinery.validation import is_valid_input_name, suggest_plain_namefrom pipelex.pipe_run.pipe_run_params import BatchParams, PipeRunParams, output_multiplicity_to_applyfrom pipelex.system.job_metadata import JobMetadatafrom pipelex.validation_error_types import PipeValidationErrorType@@ -275,7 +275,8 @@ class PipeSequence(PipeController):def refuse_undeclared_needed_input(self, *, variable_name: str) -> None:"""Refuse a binding's root that the sequence neither declares nor always stores, asking for the concept its path walks.- A root that is not a plain input name cannot be declared, so the refusal asks for a step storing it instead.+ A root that is not a plain input name can be neither declared nor stored by a step, since every stored name is a plain+ input name too, so the refusal asks to bind from a plain name instead."""_, binding_root_needs = self._walk_needed_inputs(visited_pipes=set())if variable_name not in binding_root_needs:@@ -287,9 +288,12 @@ class PipeSequence(PipeController):f"Declare '{variable_name}' in the sequence's `inputs`, with the concept whose structure holds the path '{step.from_path}'.")else:+ plain_name = suggest_plain_name(name=variable_name)+ suggestion = f", such as '{plain_name}'" if plain_name is not None else ""remedy = (- f"'{variable_name}' cannot be an input of the sequence, since an input name is a plain snake_case identifier: "- f"store a value under '{variable_name}' in an earlier step, whose `result` names it, or bind from a plain name."+ f"'{variable_name}' is not a plain snake_case identifier, so it can be neither an input of the sequence nor a name "+ f"a step stores a value under: bind from a plain name{suggestion}, declared in the sequence's `inputs` or stored by "+ "an earlier step.")msg = (f"In pipe '{self.code}', the {step.label} reads '{variable_name}', which is neither an input of the "diff --git a/pipelex/pipe_controllers/sub_pipe_blueprint.py b/pipelex/pipe_controllers/sub_pipe_blueprint.pyindex d55039ff5..f5b7e934d 100644--- a/pipelex/pipe_controllers/sub_pipe_blueprint.py+++ b/pipelex/pipe_controllers/sub_pipe_blueprint.py@@ -4,7 +4,7 @@ from pydantic import BaseModel, ConfigDict, model_validatorfrom pipelex.core.pipes.exceptions import PipeValidationErrorfrom pipelex.pipe_controllers.binding.binding_step_blueprint import PATH_GRAMMAR_DESCRIPTION-from pipelex.pipe_machinery.validation import check_name_is_not_reserved+from pipelex.pipe_machinery.validation import check_name_is_not_reserved, check_stored_namefrom pipelex.tools.misc.string_utils import is_field_pathfrom pipelex.tools.typing.validation_utils import has_more_than_one_among_attributes_from_listfrom pipelex.validation_error_types import PipeValidationErrorType@@ -68,16 +68,19 @@ class SubPipeBlueprint(BaseModel):raise PipeValidationError(message=msg, error_type=PipeValidationErrorType.BINDING_STEP_INVALID, variable_names=[self.batch_over])@model_validator(mode="after")- def validate_reserved_names(self) -> Self:- """Refuse a name taking the prefix the runtime reserves for the bound list of a dotted `batch_over`, on a sequence step and a- parallel branch alike.+ def validate_stored_names(self) -> Self:+ """Hold the names the step stores values under to the input-name form, and keep a plain `batch_over` off the reserved+ prefix, on a sequence step and a parallel branch alike.- A step stores its `result` and hands each item to its pipe under `batch_as`, both in working memory, and batches over a- plain `batch_over` by name. A nested sequence binds in its caller's working memory, so a name of the caller taking the- prefix could be overwritten by a list the nested sequence binds, and read back as another value.+ A step stores its `result` and hands each item to its pipe under `batch_as`, both in working memory, for a pipe to read+ through an input, so each is a stored name and takes the plain input-name form. A plain `batch_over` reads a name rather+ than storing one, and only stays off the prefix the runtime reserves for the bound list of a dotted `batch_over`: a nested+ sequence binds in its caller's working memory, so a step batching over a name taking the prefix could read a list a+ sequence it calls binds. A dotted `batch_over` follows the path grammar, whose segments are never underscore-led."""- named_fields = (("result", self.result), ("batch_as", self.batch_as), ("batch_over", self.batch_over))- for field_name, name in named_fields:+ for field_name, name in (("result", self.result), ("batch_as", self.batch_as)):if name is not None:- check_name_is_not_reserved(name=name, field_label=f"The `{field_name}` of the step running pipe '{self.pipe}'")+ check_stored_name(name=name, field_label=f"The `{field_name}` of the step running pipe '{self.pipe}'")+ if self.batch_over is not None:+ check_name_is_not_reserved(name=self.batch_over, field_label=f"The `batch_over` of the step running pipe '{self.pipe}'")return selfdiff --git a/pipelex/pipe_machinery/validation.py b/pipelex/pipe_machinery/validation.pyindex 7bdc5904d..8ff069793 100644--- a/pipelex/pipe_machinery/validation.py+++ b/pipelex/pipe_machinery/validation.py@@ -21,6 +21,7 @@ from pipelex.tools.misc.string_utils import (FIELD_PATH_SEGMENT_REGEX,SNAKE_CASE_IDENTIFIER_REGEX,SNAKE_CASE_PATTERN,+ camel_to_snake_case,get_root_from_dotted_path,is_snake_case,)@@ -33,9 +34,13 @@ from pipelex.validation_error_types import PipeValidationErrorType# snake_case grammar of `string_utils`, the one source every name grammar here is composed from.INPUT_NAME_PATTERN = SNAKE_CASE_PATTERN-# The grammar of a binding step's `result`: a plain input name, since it exists to be read by a later-# step's input or as a later binding's root, both of which are plain names.-BINDING_RESULT_PATTERN = INPUT_NAME_PATTERN+# The grammar of a stored name, a name under which a step stores a value in working memory: a pipe step's or+# a PipeParallel branch's `result`, the `batch_as` of either, a PipeBatch's `input_item_name`, and a binding+# step's `result`. A later pipe reads a stored value through an input, or a later binding through its root,+# both plain names, so a stored name is a plain input name and every value a step stores can be read. The+# MTHDS JSON Schema generator writes this pattern on each of these fields, so a structural check refuses+# exactly what `check_stored_name` and the binding step's own validator refuse.+STORED_NAME_PATTERN = INPUT_NAME_PATTERN# The grammar of a binding step's `from`: a root name followed by zero or more field names, separated by# single dots, each segment a letter followed by letters, digits and underscores. Subscripts, expressions,@@ -52,10 +57,11 @@ SEQUENCE_STEP_BATCH_OVER_PATTERN = rf"^(?:[^.]*|{FIELD_PATH_SEGMENT_REGEX}(?:\.{PARALLEL_BRANCH_BATCH_OVER_PATTERN = r"^[^.]*$"# The names the runtime reserves: those taking the prefix a PipeSequence binds a dotted `batch_over`'s list under. A nested-# sequence binds in its caller's working memory, so a caller's name taking the prefix could be overwritten by that list.-# `check_name_is_not_reserved` refuses it on every name an author writes into working memory or batches over by name, the-# ones no other grammar already keeps underscore-free, and the MTHDS JSON Schema generator writes this pattern under `not`-# on the same fields, so a structural check refuses exactly what the runtime refuses.+# sequence binds in its caller's working memory, so a caller's name taking the prefix could be overwritten by that list. No+# stored name or input name can take it, being plain and so never underscore-led, and `check_name_is_not_reserved` refuses+# it on the one name an author writes that no such grammar holds, a plain `batch_over`, which reads a name rather than+# storing one. The MTHDS JSON Schema generator writes this pattern under `not` on that field, so a structural check refuses+# exactly what the runtime refuses.RESERVED_NAME_PATTERN = f"^{re.escape(PRIVATE_BINDING_NAME_PREFIX)}"# The marker of an input declaration as `_input_marker` writes it after the concept: an optional multiplicity@@ -472,17 +478,70 @@ def check_input_list_name(*, input_list_name: str, branch_pipe_code: str, input_raise PipeValidationError(message=msg, error_type=PipeValidationErrorType.INVALID_INPUT_NAME, variable_names=[input_list_name])+def suggest_plain_name(*, name: str) -> str | None:+ """A plain input name close to a name that is not one, for a refusal to offer, or `None` when none comes out of it.++ The reserved prefix is dropped, dots and hyphens become underscores and PascalCase or camelCase becomes snake_case, so+ `Pages` gives `pages`, `catalog.pages` gives `catalog_pages` and `_bound_catalog_pages` gives `catalog_pages`.+ """+ unprefixed_name = name.removeprefix(PRIVATE_BINDING_NAME_PREFIX).replace(".", "_").replace("-", "_")+ candidate = re.sub(r"_+", "_", camel_to_snake_case(name=unprefixed_name)).strip("_")+ if candidate == name or not is_valid_input_name(candidate):+ return None+ return candidate+++def check_stored_name(*, name: str, field_label: str) -> None:+ """Refuse a stored name that is not a plain input name.++ A stored name is a name under which a step stores a value in working memory: a pipe step's or a PipeParallel branch's+ `result`, the `batch_as` of either, and a PipeBatch's `input_item_name`. A later pipe reads a stored value through an+ input, and an input name is a plain snake_case identifier, so a value stored under any other name could never be read.+ No plain name takes the prefix the runtime reserves for the bound list of a dotted `batch_over`, being never+ underscore-led, so this refuses such a name too, and the message says why the prefix is not the author's. A binding+ step's `result` is a stored name held to the same grammar, which its own blueprint refuses as `binding_step_invalid`.++ Args:+ name: The name as written.+ field_label: The field holding it, for the message, such as "The `result` of the step running pipe 'describe_page'".++ Raises:+ PipeValidationError: ``INVALID_INPUT_NAME`` naming the name.+ """+ if is_valid_input_name(name):+ return+ stored_name_rule = "a value is stored under it in working memory for a pipe to read through an input, so it must be a plain input name"+ reason: str+ if name.startswith(PRIVATE_BINDING_NAME_PREFIX):+ reason = (+ f"{stored_name_rule}, and the `{PRIVATE_BINDING_NAME_PREFIX}` prefix it starts with is reserved for the bound list of a "+ "dotted `batch_over`, which only the runtime writes or reads."+ )+ elif "." in name:+ reason = f"{stored_name_rule}, which names one whole value and cannot reach into a field with a dot."+ else:+ reason = f"{stored_name_rule}, and {_PLAIN_NAME_RULE}."+ plain_name = suggest_plain_name(name=name)+ suggestion = f", such as '{plain_name}'" if plain_name is not None else ""+ msg = (+ f"{field_label}, '{name}', is not a plain input name: {reason} Rename it to a plain name{suggestion}, and rename every input, "+ "binding path and `batch_over` reading it to match."+ )+ raise PipeValidationError(message=msg, error_type=PipeValidationErrorType.INVALID_INPUT_NAME, variable_names=[name])++def check_name_is_not_reserved(*, name: str, field_label: str) -> None:- """Refuse a name an author writes that takes the prefix the runtime reserves for the bound list of a dotted `batch_over`.+ """Refuse a plain `batch_over` taking the prefix the runtime reserves for the bound list of a dotted `batch_over`.A PipeSequence binds a dotted `batch_over`'s list under a private name taking the prefix, `_bound_catalog_pages` for- `catalog.pages`, and a nested sequence binds in the working memory of the sequence calling it. A caller's name taking- the prefix could then be overwritten by the list a sequence it calls binds, so the prefix is the runtime's alone: a- name an author stores a value under, hands an item to a pipe under, or batches over by name never takes it.+ `catalog.pages`, and a nested sequence binds in the working memory of the sequence calling it. A step batching over such+ a name by hand would read a list a sequence it calls binds, so the prefix is the runtime's alone. A stored name never+ takes it, being a plain input name (`check_stored_name`), but a plain `batch_over` reads a name rather than storing+ one, so no other grammar keeps it off the prefix.Args:name: The name as written.- field_label: The field holding it, for the message, such as "The `result` of the step running pipe 'describe_page'".+ field_label: The field holding it, for the message, such as "The `batch_over` of the step running pipe 'describe_page'".Raises:PipeValidationError: ``INVALID_INPUT_NAME`` naming the name.diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_batch_item_name_not_plain/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_batch_item_name_not_plain/bundle.mthdsnew file mode 100644index 000000000..ecd8cb211--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_batch_item_name_not_plain/bundle.mthds@@ -0,0 +1,28 @@+domain = "cheese_counter"+description = "Writing the price tag of every cheese on a shop counter"+main_pipe = "tag_cheeses"++[concept.Cheese]+description = "A cheese sold by weight at the counter"++[concept.Cheese.structure]+variety = { type = "text", description = "The kind of cheese", required = true }+price_per_kilo = { type = "number", description = "The price of one kilogram, in euros", required = true }++[pipe.tag_cheeses]+type = "PipeBatch"+description = "Writes the price tag of each cheese in turn"+inputs = { cheeses = "Cheese[]" }+output = "Text[]"+branch_pipe_code = "write_price_tag"+input_list_name = "cheeses"+# Wrong on purpose: each cheese is stored under `input_item_name` for the branch pipe to read through an input, so the name+# is a plain snake_case one. Renaming it to `cheese`, the input the branch pipe reads, is the whole fix.+input_item_name = "Cheese"++[pipe.write_price_tag]+type = "PipeCompose"+description = "Writes the price tag of one cheese"+inputs = { cheese = "Cheese" }+output = "Text"+template = "{{ cheese.variety }}: {{ cheese.price_per_kilo }} euros per kilo"diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_batch_item_name_not_plain/entry.toml b/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_batch_item_name_not_plain/entry.tomlnew file mode 100644index 000000000..10c31d5bb--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_batch_item_name_not_plain/entry.toml@@ -0,0 +1,12 @@+name = "invalid_input_name_batch_item_name_not_plain"+description = "The price tags of the cheeses on a shop counter written by a batch that hands each cheese to its branch pipe under a name that is not a plain input name"++validity = "invalid"+tier = "static"+granularity = "focused"++covers = [+ "error.invalid_input_name",+]++expected_error = "invalid_input_name"diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_branch_batch_as_reserved/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_branch_batch_as_reserved/bundle.mthdsnew file mode 100644index 000000000..0f4abaf08--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_branch_batch_as_reserved/bundle.mthds@@ -0,0 +1,38 @@+domain = "weather_station"+description = "Writing the daily log of a weather station"+main_pipe = "write_daily_log"++[concept.Reading]+description = "One reading taken by a weather station"++[concept.Reading.structure]+hour = { type = "text", description = "The hour the reading was taken", required = true }+temperature = { type = "number", description = "The temperature, in degrees Celsius", required = true }++[pipe.write_reading_line]+type = "PipeCompose"+description = "Writes the line of the daily log for one reading"+inputs = { reading = "Reading" }+output = "Text"+template = "{{ reading.hour }}: {{ reading.temperature }} degrees"++[pipe.write_log_heading]+type = "PipeCompose"+description = "Writes the heading of the daily log"+inputs = { station_name = "Text" }+output = "Text"+template = "Daily log of $station_name"++[pipe.write_daily_log]+type = "PipeParallel"+description = "Writes the heading and the line of every reading at the same time"+inputs = { readings = "Reading[]", station_name = "Text" }+output = "Composite"+add_each_output = true+# Wrong on purpose: each reading is stored under `batch_as` for the line pipe to read through an input, so the name is a plain+# snake_case one, which never takes the `_bound_` prefix the runtime keeps for itself. Renaming it to `reading`, the input the+# line pipe reads, is the whole fix.+branches = [+ { pipe = "write_reading_line", batch_over = "readings", batch_as = "_bound_reading", result = "reading_lines" },+ { pipe = "write_log_heading", result = "heading" },+]diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_branch_batch_as_reserved/entry.toml b/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_branch_batch_as_reserved/entry.tomlnew file mode 100644index 000000000..f2f7b73b5--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_branch_batch_as_reserved/entry.toml@@ -0,0 +1,12 @@+name = "invalid_input_name_branch_batch_as_reserved"+description = "The daily log of a weather station written in parallel, one branch handing each reading to its pipe under a name taking the reserved prefix"++validity = "invalid"+tier = "static"+granularity = "focused"++covers = [+ "error.invalid_input_name",+]++expected_error = "invalid_input_name"diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_branch_result_not_plain/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_branch_result_not_plain/bundle.mthdsnew file mode 100644index 000000000..c1db68d7d--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_branch_result_not_plain/bundle.mthds@@ -0,0 +1,38 @@+domain = "weeknight_recipes"+description = "Getting ready to cook a recipe"+main_pipe = "prepare_recipe"++[concept.Recipe]+description = "A recipe from a family cookbook"++[concept.Recipe.structure]+title = { type = "text", description = "The name of the dish", required = true }+ingredients = { type = "text", description = "The ingredients, as the cookbook lists them", required = true }+method = { type = "text", description = "How to cook the dish", required = true }++[pipe.write_shopping_list]+type = "PipeCompose"+description = "Writes the shopping list of a recipe"+inputs = { recipe = "Recipe" }+output = "Text"+template = "To buy for {{ recipe.title }}: {{ recipe.ingredients }}"++[pipe.write_cooking_card]+type = "PipeCompose"+description = "Writes the card pinned above the stove"+inputs = { recipe = "Recipe" }+output = "Text"+template = "{{ recipe.title }}: {{ recipe.method }}"++[pipe.prepare_recipe]+type = "PipeParallel"+description = "Writes the shopping list and the cooking card at the same time"+inputs = { recipe = "Recipe" }+output = "Composite"+add_each_output = true+# Wrong on purpose: a branch's result is stored for a later pipe to read through an input, so the name is a plain+# snake_case one. Renaming it to `shopping_list` is the whole fix.+branches = [+ { pipe = "write_shopping_list", result = "ShoppingList" },+ { pipe = "write_cooking_card", result = "cooking_card" },+]diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_branch_result_not_plain/entry.toml b/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_branch_result_not_plain/entry.tomlnew file mode 100644index 000000000..1b7f03728--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_branch_result_not_plain/entry.toml@@ -0,0 +1,12 @@+name = "invalid_input_name_branch_result_not_plain"+description = "The shopping list and the cooking card of a recipe written in parallel, one branch storing its list under a name that is not a plain input name"++validity = "invalid"+tier = "static"+granularity = "focused"++covers = [+ "error.invalid_input_name",+]++expected_error = "invalid_input_name"diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_step_batch_as_dotted/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_step_batch_as_dotted/bundle.mthdsnew file mode 100644index 000000000..0d162f338--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_step_batch_as_dotted/bundle.mthds@@ -0,0 +1,28 @@+domain = "parcel_dispatch"+description = "Writing the shipping label of every parcel in a delivery round"+main_pipe = "label_round"++[concept.Parcel]+description = "A parcel waiting to be delivered"++[concept.Parcel.structure]+recipient = { type = "text", description = "Who the parcel is for", required = true }+address = { type = "text", description = "Where the parcel is delivered", required = true }++[pipe.write_label]+type = "PipeCompose"+description = "Writes the shipping label of one parcel"+inputs = { parcel = "Parcel" }+output = "Text"+template = "{{ parcel.recipient }}, {{ parcel.address }}"++[pipe.label_round]+type = "PipeSequence"+description = "Writes the label of each parcel in the round"+inputs = { parcels = "Parcel[]" }+output = "Text[]"+# Wrong on purpose: each parcel is stored under `batch_as` for the label pipe to read through an input, so the name is never+# dotted. Renaming it to `parcel`, the input the label pipe reads, is the whole fix.+steps = [+ { pipe = "write_label", batch_over = "parcels", batch_as = "round.parcel", result = "labels" },+]diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_step_batch_as_dotted/entry.toml b/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_step_batch_as_dotted/entry.tomlnew file mode 100644index 000000000..a4583916d--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_step_batch_as_dotted/entry.toml@@ -0,0 +1,12 @@+name = "invalid_input_name_step_batch_as_dotted"+description = "The shipping labels of a delivery round written by a sequence step that hands each parcel to the label pipe under a dotted name"++validity = "invalid"+tier = "static"+granularity = "focused"++covers = [+ "error.invalid_input_name",+]++expected_error = "invalid_input_name"diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_step_result_not_plain/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_step_result_not_plain/bundle.mthdsnew file mode 100644index 000000000..82c5337e3--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_step_result_not_plain/bundle.mthds@@ -0,0 +1,36 @@+domain = "rail_timetables"+description = "Announcing the next departures from a station"+main_pipe = "announce_departures"++[concept.Departure]+description = "One train leaving the station"++[concept.Departure.structure]+destination = { type = "text", description = "Where the train goes", required = true }+platform = { type = "text", description = "The platform it leaves from", required = true }++[pipe.list_departures]+type = "PipeLLM"+description = "Lists the departures shown on a timetable board"+inputs = { board = "Text" }+output = "Departure[]"+prompt = "List the departures shown on this timetable board: $board"++[pipe.write_announcement]+type = "PipeCompose"+description = "Writes the announcement of one departure"+inputs = { departure = "Departure" }+output = "Text"+template = "The train to {{ departure.destination }} leaves from platform {{ departure.platform }}."++[pipe.announce_departures]+type = "PipeSequence"+description = "Lists the departures, then writes the announcement of each"+inputs = { board = "Text" }+output = "Text[]"+# Wrong on purpose: a step stores its result for a later pipe to read through an input, so the name is a plain+# snake_case one. Nothing reads it here, so renaming it to `announcements` is the whole fix.+steps = [+ { pipe = "list_departures", result = "departures" },+ { pipe = "write_announcement", batch_over = "departures", batch_as = "departure", result = "Announcements" },+]diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_step_result_not_plain/entry.toml b/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_step_result_not_plain/entry.tomlnew file mode 100644index 000000000..0eabc1964--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_input_name_step_result_not_plain/entry.toml@@ -0,0 +1,12 @@+name = "invalid_input_name_step_result_not_plain"+description = "The departures from a station announced by a sequence whose last step stores its announcements under a name that is not a plain input name"++validity = "invalid"+tier = "static"+granularity = "focused"++covers = [+ "error.invalid_input_name",+]++expected_error = "invalid_input_name"diff --git a/pipelex/validation_error_types.py b/pipelex/validation_error_types.pyindex 7a0c6cff3..320d7f0a8 100644--- a/pipelex/validation_error_types.py+++ b/pipelex/validation_error_types.py@@ -55,11 +55,13 @@ class PipeValidationErrorType(StrEnum):LLM_SETTING_REFUSED_BY_MODEL = "llm_setting_refused_by_model"INVALID_PIPE_CODE_SYNTAX = "invalid_pipe_code_syntax"# An input name that is not a plain snake_case identifier, a dotted name (`invoice.total`) included,- # on any pipe's `inputs` or as a PipeBatch's `input_list_name`; or a name taking the `_bound_` prefix- # the runtime reserves for the bound list of a dotted `batch_over`, as a pipe step's or a parallel- # branch's `result`, `batch_as` or plain `batch_over`, or a PipeBatch's `input_item_name`. Detected at- # blueprint parse time; a dotted name whose root the same `inputs` table declares carries the- # enrichment the fix planner turns into deleting the key.+ # on any pipe's `inputs` or as a PipeBatch's `input_list_name`; a stored name that is not one either,+ # in any form (`Pages`, `doc.pages`, `_bound_pages`), as a pipe step's or a parallel branch's `result`+ # or `batch_as`, or a PipeBatch's `input_item_name`; or a plain `batch_over` taking the `_bound_`+ # prefix the runtime reserves for the bound list of a dotted `batch_over`, the one check a plain+ # `batch_over` gets. A binding step's `result` is a stored name too, refused as BINDING_STEP_INVALID.+ # Detected at blueprint parse time; a dotted input name whose root the same `inputs` table declares+ # carries the enrichment the fix planner turns into deleting the key.INVALID_INPUT_NAME = "invalid_input_name"UNKNOWN_PIPE_TYPE = "unknown_pipe_type"# A `[pipe.x]` section declared no `type` yet carries fields beyond the signature contractdiff --git a/subject_grants.toml b/subject_grants.tomlindex da6ed3a72..0043e596d 100644--- a/subject_grants.toml+++ b/subject_grants.toml@@ -3556,6 +3556,10 @@ rationale = "One post-processing pass of the schema pipeline, called as a verb oparam = "schema"rationale = "Schema-pipeline stage transforming the schema dict; single operand."+["pipelex/language/mthds_schema_generator.py::_constrain_stored_names"]+param = "schema"+rationale = "One post-processing pass of the schema pipeline, called as a verb on the schema it rewrites, like its sibling passes."+["pipelex/language/mthds_schema_generator.py::_convert_to_draft4"]param = "schema"rationale = "Schema-pipeline stage transforming the schema dict; single operand."@@ -3584,10 +3588,6 @@ rationale = "Verb-object: mutates the schema object's required list; field_namesparam = "schema"rationale = "Schema-pipeline stage transforming the schema dict; single operand."-["pipelex/language/mthds_schema_generator.py::_reserve_private_binding_names"]-param = "schema"-rationale = "One post-processing pass of the schema pipeline, called as a verb on the schema it rewrites, like its sibling passes."-["pipelex/language/mthds_schema_generator.py::_walk_schema"]param = "node"rationale = "Verb-object: walks the schema node tree; the visitor stays keyword."diff --git a/tests/e2e/pipelex/pipes/pipe_controller/pipe_parallel/parallel_graph_add_each.mthds b/tests/e2e/pipelex/pipes/pipe_controller/pipe_parallel/parallel_graph_add_each.mthdsindex 60c2fe80d..a141073ff 100644--- a/tests/e2e/pipelex/pipes/pipe_controller/pipe_parallel/parallel_graph_add_each.mthds+++ b/tests/e2e/pipelex/pipes/pipe_controller/pipe_parallel/parallel_graph_add_each.mthds@@ -16,7 +16,7 @@ description = "Run parallel summaries then consume one downstream"inputs = { input_text = "Text" }output = "Text"steps = [- { pipe = "parallel_summarize", result = "..." },+ { pipe = "parallel_summarize", result = "summaries" },{ pipe = "combine_summaries" },]diff --git a/tests/integration/pipelex/pipes/controller/pipe_sequence/test_binding_refusal_remedy.py b/tests/integration/pipelex/pipes/controller/pipe_sequence/test_binding_refusal_remedy.pyindex e3d2c2caa..0d1695824 100644--- a/tests/integration/pipelex/pipes/controller/pipe_sequence/test_binding_refusal_remedy.py+++ b/tests/integration/pipelex/pipes/controller/pipe_sequence/test_binding_refusal_remedy.py@@ -48,8 +48,8 @@ output = "Text"template = "View: {{{{ page_view.caption }}}}""""-# A root that is not a plain name: no input can bear it, so only an earlier pipe step's `result`, which the standard leaves-# unrestricted, can store a value under it.+# A root that is not a plain name: no input can bear it, and no step can store a value under it either, since a pipe step's+# `result` is a stored name, held to the plain input-name form like an input name._UPPERCASE_ROOT_BUNDLE = """domain = "billing"description = "Reading the total of an invoice stored under a name that is not a plain name"main_pipe = "read_total"@@ -117,17 +117,21 @@ class TestBindingRefusalRemedy:assert result.pipe_output.main_stuff.as_text.text == "View: Garden chairs on a lawn"@pytest.mark.asyncio(loop_scope="class")- async def test_a_root_that_is_not_a_plain_name_reads_an_earlier_steps_result(self) -> None:- """The path grammar lets the root be any segment, so it can read a pipe step's `result` that is not a plain name."""- result = await PipelexMTHDSProtocol(pipe_run_mode=PipeRunMode.LIVE).execute(- mthds_contents=[_UPPERCASE_ROOT_BUNDLE], inputs={"amount": {"concept": "native.Number", "content": {"number": 120}}}- )+ async def test_a_pipe_step_storing_under_a_name_that_is_not_plain_is_refused(self) -> None:+ """A pipe step's `result` is a stored name, so it cannot store the value a root that is not a plain name would read."""+ with pytest.raises(ValidateBundleError) as exc_info:+ await validate_bundle(mthds_contents=[_UPPERCASE_ROOT_BUNDLE])- assert result.pipe_output.main_stuff_as_number.number == 120+ validation_errors = exc_info.value.to_error_report().validation_errors or []+ assert [error.error_type for error in validation_errors] == [PipeValidationErrorType.INVALID_INPUT_NAME]+ assert validation_errors[0].variable_names == ["Invoice"]+ message = validation_errors[0].message or ""+ assert "The `result` of the step running pipe 'make_invoice', 'Invoice', is not a plain input name" in message+ assert "Rename it to a plain name, such as 'invoice'," in message@pytest.mark.asyncio(loop_scope="class")async def test_a_missing_root_that_is_not_a_plain_name_is_never_asked_for_as_an_input(self) -> None:- """Regression: the refusal of a root nothing stores never tells the author to declare an input no input name can match."""+ """Regression: the refusal of a root nothing stores never asks to declare it, or to store a value under it, when it is not plain."""mthds_content = _UPPERCASE_ROOT_BUNDLE.replace(' { pipe = "make_invoice", result = "Invoice" },\n', "")with pytest.raises(ValidateBundleError) as exc_info:@@ -137,5 +141,6 @@ class TestBindingRefusalRemedy:assert [error.error_type for error in validation_errors] == [PipeValidationErrorType.MISSING_INPUT_VARIABLE]message = validation_errors[0].message or ""assert "Declare 'Invoice'" not in message- assert "'Invoice' cannot be an input of the sequence" in message- assert "store a value under 'Invoice' in an earlier step" in message+ assert "store a value under 'Invoice'" not in message+ assert "it can be neither an input of the sequence nor a name a step stores a value under" in message+ assert "bind from a plain name, such as 'invoice'," in messagediff --git a/tests/integration/pipelex/pipes/controller/pipe_sequence/test_dotted_batch_over.py b/tests/integration/pipelex/pipes/controller/pipe_sequence/test_dotted_batch_over.pyindex 5546694be..656cc8087 100644--- a/tests/integration/pipelex/pipes/controller/pipe_sequence/test_dotted_batch_over.py+++ b/tests/integration/pipelex/pipes/controller/pipe_sequence/test_dotted_batch_over.py@@ -415,29 +415,6 @@ class TestDottedBatchOver:assert response.pipe_output.main_stuff_as_items(item_type=TextContent) == []- async def test_a_batch_over_a_result_that_is_not_a_plain_name_runs(self) -> None:- """Regression: the batch a step runs was built from a blueprint, which refused any list name but a plain input name, so a- batch over a result named `IndexLines`, which a pipe step may store under, failed when it ran.- """- mthds_content = _catalog_bundle(- inputs='{ catalog = "Catalog" }',- steps=[- '{ pipe = "write_index_line", batch_over = "catalog.pages", batch_as = "page", result = "IndexLines" }',- '{ pipe = "copy_line", batch_over = "IndexLines", batch_as = "line", result = "copied_lines" }',- ],- ).replace(- "[pipe.make_catalog]",- '[pipe.copy_line]\ntype = "PipeCompose"\ndescription = "Copies a line"\ninputs = { line = "Text" }\noutput = "Text"\n'- 'template = "Copy of $line"\n\n[pipe.make_catalog]',- )-- response = await _index_lines(- mthds_content=mthds_content, inputs={"catalog": {"concept": "catalog_index.Catalog", "content": _SPRING_CATALOG}}- )-- lines = [item.text for item in response.pipe_output.main_stuff_as_items(item_type=TextContent)]- assert lines == ["Copy of Page: Garden chairs", "Copy of Page: Parasols"]-async def test_two_dotted_batch_overs_of_one_path_take_two_private_names(self, load_empty_library: Callable[[], str]) -> None:mthds_content = _catalog_bundle(inputs='{ catalog = "Catalog" }',@@ -470,11 +447,9 @@ class TestDottedBatchOver:result_refusals = [error for error in errors if "The `result` of the step running pipe 'write_index_line'" in error.message]assert len(result_refusals) == 1assert result_refusals[0].variable_names == [reserved_name]- assert (- f"'{reserved_name}', takes the `_bound_` prefix, which is reserved for the bound list of a dotted `batch_over`"- in result_refusals[0].message- )- assert "Choose another name, such as 'catalog_pages'." in result_refusals[0].message+ assert f"'{reserved_name}', is not a plain input name" in result_refusals[0].message+ assert "the `_bound_` prefix it starts with is reserved for the bound list of a dotted `batch_over`" in result_refusals[0].message+ assert "Rename it to a plain name, such as 'catalog_pages'," in result_refusals[0].messageasync def test_a_caller_name_outside_the_prefix_keeps_its_value_across_a_nested_binding(self) -> None:response = await _index_lines(diff --git a/tests/unit/pipelex/language/test_mthds_schema.py b/tests/unit/pipelex/language/test_mthds_schema.pyindex f06100366..bc2c85de4 100644--- a/tests/unit/pipelex/language/test_mthds_schema.py+++ b/tests/unit/pipelex/language/test_mthds_schema.py@@ -407,7 +407,7 @@ class TestMthdsSchemaGeneration:("batch_over", "is_valid_on_a_step", "is_valid_on_a_branch"),[pytest.param("pages", True, True, id="a-plain-name"),- pytest.param("PagesOfTheCatalog", True, True, id="a-name-a-pipe-step-may-store-under"),+ pytest.param("PagesOfTheCatalog", True, True, id="a-name-that-is-not-snake-case"),pytest.param("catalog.pages", True, False, id="a-dotted-path"),pytest.param("catalogs.pages.page_view", True, False, id="a-deep-dotted-path"),pytest.param("catalog..pages", False, False, id="an-empty-segment"),@@ -435,17 +435,23 @@ class TestMthdsSchemaGeneration:[pytest.param("catalog_pages", True, id="a-plain-name"),pytest.param("bound_pages", True, id="the-prefix-without-its-underscore"),- pytest.param("_draft", True, id="another-underscore-led-name"),+ pytest.param("page2", True, id="a-digit-after-the-first-letter"),+ pytest.param("Pages", False, id="pascal-case"),+ pytest.param("catalog.pages", False, id="dotted"),+ pytest.param("_draft", False, id="underscore-led"),+ pytest.param("2nd_pages", False, id="digit-led"),pytest.param("_bound_catalog_pages", False, id="the-reserved-prefix"),pytest.param("_bound_", False, id="the-reserved-prefix-alone"),],)- @pytest.mark.parametrize("field_name", ["result", "batch_as", "batch_over"])- def test_a_step_and_a_branch_refuse_the_reserved_prefix(self, schema: dict[str, Any], field_name: str, name: str, should_validate: bool) -> None:- """A pipe step's `result`, `batch_as` and plain `batch_over` never take `_bound_`, the runtime's prefix for a dotted `batch_over`'s list.+ @pytest.mark.parametrize("field_name", ["result", "batch_as"])+ def test_a_step_and_a_branch_hold_their_stored_names_to_the_plain_name_grammar(+ self, schema: dict[str, Any], field_name: str, name: str, should_validate: bool+ ) -> None:+ """A pipe step's `result` and `batch_as` are stored names, plain input names on a PipeSequence step and a PipeParallel branch alike.- The runtime refuses them as `invalid_input_name` when the bundle is parsed, and the schema refuses them first, through a- Draft-4 `not` on the string arm, on a PipeSequence step and on a PipeParallel branch alike.+ The runtime refuses any other form as `invalid_input_name` when the bundle is parsed, and the schema refuses it first,+ through the input-name `pattern` on the string arm, which no underscore-led name, the reserved prefix's included, matches."""validator = _pipe_union_oneof_validator(schema)pipe_step: dict[str, Any] = {"pipe": "describe_page", "batch_over": "pages", "batch_as": "page", "result": "descriptions", field_name: name}@@ -454,19 +460,68 @@ class TestMthdsSchemaGeneration:assert validator.is_valid(sequence_table) is should_validate, f"sequence step {field_name} {name!r}"assert validator.is_valid(parallel_table) is should_validate, f"parallel branch {field_name} {name!r}"+ @pytest.mark.parametrize(+ ("batch_over", "should_validate"),+ [+ pytest.param("catalog_pages", True, id="a-plain-name"),+ pytest.param("bound_pages", True, id="the-prefix-without-its-underscore"),+ pytest.param("_draft", True, id="another-underscore-led-name"),+ pytest.param("_bound_catalog_pages", False, id="the-reserved-prefix"),+ pytest.param("_bound_", False, id="the-reserved-prefix-alone"),+ ],+ )+ def test_a_step_and_a_branch_keep_a_plain_batch_over_off_the_reserved_prefix(+ self, schema: dict[str, Any], batch_over: str, should_validate: bool+ ) -> None:+ """A plain `batch_over` reads a name rather than storing one, and never `_bound_`, the runtime's prefix for a dotted `batch_over`'s list.++ The runtime refuses it as `invalid_input_name` when the bundle is parsed, and the schema refuses it first, through a+ Draft-4 `not` on the string arm, on a PipeSequence step and on a PipeParallel branch alike.+ """+ validator = _pipe_union_oneof_validator(schema)+ pipe_step: dict[str, Any] = {"pipe": "describe_page", "batch_over": batch_over, "batch_as": "page", "result": "descriptions"}+ sequence_table = {**_minimal_pipe_table("PipeSequence"), "steps": [pipe_step]}+ parallel_table = {**_minimal_pipe_table("PipeParallel"), "branches": [pipe_step]}+ assert validator.is_valid(sequence_table) is should_validate, f"sequence step batch_over {batch_over!r}"+ assert validator.is_valid(parallel_table) is should_validate, f"parallel branch batch_over {batch_over!r}"+@pytest.mark.parametrize(("input_item_name", "should_validate"),[pytest.param("item", True, id="a-plain-name"),- pytest.param("_draft_item", True, id="another-underscore-led-name"),+ pytest.param("item2", True, id="a-digit-after-the-first-letter"),+ pytest.param("Item", False, id="pascal-case"),+ pytest.param("catalog.item", False, id="dotted"),+ pytest.param("_draft_item", False, id="underscore-led"),pytest.param("_bound_item", False, id="the-reserved-prefix"),],)- def test_batch_input_item_name_refuses_the_reserved_prefix(self, schema: dict[str, Any], input_item_name: str, should_validate: bool) -> None:+ def test_batch_input_item_name_follows_the_plain_name_grammar(self, schema: dict[str, Any], input_item_name: str, should_validate: bool) -> None:+ """A PipeBatch's `input_item_name` is a stored name, the branch pipe reading each item through an input of that name."""validator = _pipe_union_oneof_validator(schema)table = {**_minimal_pipe_table("PipeBatch"), "input_item_name": input_item_name}assert validator.is_valid(table) is should_validate+ @pytest.mark.parametrize(+ "pattern_path",+ [+ pytest.param(("SubPipeBlueprint", "result"), id="step-result"),+ pytest.param(("SubPipeBlueprint", "batch_as"), id="step-batch-as"),+ pytest.param(("ParallelBranchBlueprint", "result"), id="branch-result"),+ pytest.param(("ParallelBranchBlueprint", "batch_as"), id="branch-batch-as"),+ pytest.param(("PipeBatchBlueprint", "input_item_name"), id="batch-input-item-name"),+ pytest.param(("BindingStepBlueprint", "result"), id="binding-step-result"),+ ],+ )+ def test_every_stored_name_carries_the_input_name_pattern(self, schema: dict[str, Any], pattern_path: tuple[str, str]) -> None:+ """Each stored name's string arm carries the same input-name pattern, and nothing else, as a structural consumer reads it."""+ definition_name, field_name = pattern_path+ field_schema = schema["definitions"][definition_name]["properties"][field_name]+ string_arms = [arm for arm in field_schema.get("anyOf", [field_schema]) if arm.get("type") == "string"]+ assert len(string_arms) == 1+ assert string_arms[0]["pattern"] == "^[a-z][a-z0-9_]*$"+ assert "not" not in string_arms[0]+def test_minimal_table_coverage_matches_schema_pipe_kinds(self, schema: dict[str, Any]) -> None:"""Guard: the test's per-kind table map covers exactly the *concrete* pipe kinds in the schema.diff --git a/tests/unit/pipelex/pipe_controllers/batch/test_pipe_batch_validation.py b/tests/unit/pipelex/pipe_controllers/batch/test_pipe_batch_validation.pyindex b1f0edd61..4e40aa440 100644--- a/tests/unit/pipelex/pipe_controllers/batch/test_pipe_batch_validation.py+++ b/tests/unit/pipelex/pipe_controllers/batch/test_pipe_batch_validation.py@@ -86,36 +86,69 @@ class TestPipeBatchValidation:)@pytest.mark.parametrize(- ("input_list_name", "input_item_name", "reserved_name", "message_fragment"),+ "input_item_name",[- pytest.param(- "items",- "_bound_item",- "_bound_item",- "The PipeBatch's `input_item_name`, '_bound_item', takes the `_bound_` prefix, which is reserved for the bound list of a "- "dotted `batch_over`",- id="input-item-name",- ),- pytest.param("_bound_items", "item", "_bound_items", "is not a valid input name", id="input-list-name"),+ pytest.param("Item", id="pascal-case"),+ pytest.param("catalog.item", id="dotted"),+ pytest.param("_bound_item", id="the-reserved-prefix"),+ pytest.param("_item", id="underscore-led"),+ pytest.param("2nd_item", id="digit-led"),],)- def test_rejects_a_name_taking_the_reserved_prefix(- self, input_list_name: str, input_item_name: str, reserved_name: str, message_fragment: str- ) -> None:- """The batch writes its item into the branch's memory under its own name, so the runtime's `_bound_` prefix is refused there too."""+ def test_rejects_an_input_item_name_that_is_not_a_plain_input_name(self, input_item_name: str) -> None:+ """The batch stores each item under `input_item_name` for the branch pipe to read through an input, so it is a plain name."""with pytest.raises(ValidationError) as exc_info:PipeBatchBlueprint(description="Process each item",inputs={"items": "Item[]"},output="Result[]",branch_pipe_code="process_item",- input_list_name=input_list_name,+ input_list_name="items",input_item_name=input_item_name,)refusals = [raw_error.get("ctx", {}).get("error") for raw_error in exc_info.value.errors()]- reserved_name_refusals = [- refusal for refusal in refusals if isinstance(refusal, PipeValidationError) and refusal.variable_names == [reserved_name]+ assert len(refusals) == 1+ refusal = refusals[0]+ assert isinstance(refusal, PipeValidationError)+ assert refusal.error_type == PipeValidationErrorType.INVALID_INPUT_NAME+ assert refusal.variable_names == [input_item_name]+ assert f"The PipeBatch's `input_item_name`, '{input_item_name}', is not a plain input name" in str(refusal)++ @pytest.mark.parametrize(+ "input_item_name",+ [+ pytest.param("item", id="a-plain-name"),+ pytest.param("bound_item", id="the-prefix-without-its-underscore"),+ pytest.param("item2", id="a-digit-after-the-first-letter"),+ ],+ )+ def test_accepts_a_plain_input_item_name(self, input_item_name: str) -> None:+ blueprint = PipeBatchBlueprint(+ description="Process each item",+ inputs={"items": "Item[]"},+ output="Result[]",+ branch_pipe_code="process_item",+ input_list_name="items",+ input_item_name=input_item_name,+ )++ assert blueprint.input_item_name == input_item_name++ def test_rejects_an_input_list_name_taking_the_reserved_prefix(self) -> None:+ """The list's name is a plain input name, which no underscore-led name is."""+ with pytest.raises(ValidationError) as exc_info:+ PipeBatchBlueprint(+ description="Process each item",+ inputs={"items": "Item[]"},+ output="Result[]",+ branch_pipe_code="process_item",+ input_list_name="_bound_items",+ input_item_name="item",+ )+ refusals = [raw_error.get("ctx", {}).get("error") for raw_error in exc_info.value.errors()]+ list_name_refusals = [+ refusal for refusal in refusals if isinstance(refusal, PipeValidationError) and refusal.variable_names == ["_bound_items"]]- assert len(reserved_name_refusals) == 1- assert reserved_name_refusals[0].error_type == PipeValidationErrorType.INVALID_INPUT_NAME- assert message_fragment in str(reserved_name_refusals[0])+ assert len(list_name_refusals) == 1+ assert list_name_refusals[0].error_type == PipeValidationErrorType.INVALID_INPUT_NAME+ assert "is not a valid input name" in str(list_name_refusals[0])diff --git a/tests/unit/pipelex/pipe_controllers/sequence/test_sequence_step_union.py b/tests/unit/pipelex/pipe_controllers/sequence/test_sequence_step_union.pyindex 233ff8322..06aeb5cc0 100644--- a/tests/unit/pipelex/pipe_controllers/sequence/test_sequence_step_union.py+++ b/tests/unit/pipelex/pipe_controllers/sequence/test_sequence_step_union.py@@ -239,28 +239,31 @@ branches = [assert refusal.error_type == PipeValidationErrorType.BINDING_STEP_INVALID@pytest.mark.parametrize(- ("step_toml", "field_name", "reserved_name"),+ ("field_name", "refused_name", "message_fragment"),[- pytest.param('{ pipe = "write_receipt", result = "_bound_invoice_lines" }', "result", "_bound_invoice_lines", id="result"),- pytest.param(- '{ pipe = "write_receipt", batch_over = "lines", batch_as = "_bound_line", result = "receipts" }',- "batch_as",- "_bound_line",- id="batch-as",- ),- pytest.param(- '{ pipe = "write_receipt", batch_over = "_bound_invoice_lines", batch_as = "total_amount", result = "receipts" }',- "batch_over",- "_bound_invoice_lines",- id="plain-batch-over",- ),+ pytest.param("result", "Receipts", "is not a plain input name", id="result-pascal-case"),+ pytest.param("result", "invoice.receipts", "is not a plain input name", id="result-dotted"),+ pytest.param("result", "_bound_invoice_lines", "is not a plain input name", id="result-reserved-prefix"),+ pytest.param("batch_as", "TotalAmount", "is not a plain input name", id="batch-as-pascal-case"),+ pytest.param("batch_as", "line.total", "is not a plain input name", id="batch-as-dotted"),+ pytest.param("batch_as", "_bound_line", "is not a plain input name", id="batch-as-reserved-prefix"),+ pytest.param("batch_over", "_bound_invoice_lines", "takes the `_bound_` prefix", id="plain-batch-over-reserved-prefix"),],)@pytest.mark.parametrize("controller", ["sequence", "parallel"])- def test_a_name_taking_the_reserved_prefix_is_refused_on_a_step_and_a_branch(- self, controller: str, step_toml: str, field_name: str, reserved_name: str+ def test_a_stored_name_that_is_not_plain_or_a_reserved_batch_over_is_refused_on_a_step_and_a_branch(+ self, controller: str, field_name: str, refused_name: str, message_fragment: str) -> None:- """A nested sequence binds a dotted `batch_over` in its caller's working memory, under a `_bound_` name no author may write."""+ """A step's `result` and `batch_as` are stored names, plain input names on a sequence step and a parallel branch alike.++ A plain `batch_over` reads a name rather than storing one, and only stays off the `_bound_` prefix, under which a nested+ sequence binds a dotted `batch_over` in its caller's working memory.+ """+ step_fields = {"result": "receipts", "batch_over": "lines", "batch_as": "total_amount", field_name: refused_name}+ step_toml = (+ f'{{ pipe = "write_receipt", batch_over = "{step_fields["batch_over"]}", batch_as = "{step_fields["batch_as"]}", '+ f'result = "{step_fields["result"]}" }}'+ )pipe_code: strmthds_content: strif controller == "sequence":@@ -286,9 +289,22 @@ branches = [errors = exc_info.value.validation_errorsassert [error.error_type for error in errors] == [PipeValidationErrorType.INVALID_INPUT_NAME]assert errors[0].pipe_code == pipe_code- assert errors[0].variable_names == [reserved_name]- assert f"The `{field_name}` of the step running pipe 'write_receipt', '{reserved_name}', takes the `_bound_` prefix" in errors[0].message- assert "which is reserved for the bound list of a dotted `batch_over`" in errors[0].message+ assert errors[0].variable_names == [refused_name]+ assert f"The `{field_name}` of the step running pipe 'write_receipt', '{refused_name}', {message_fragment}" in errors[0].message++ @pytest.mark.parametrize("controller", ["sequence", "parallel"])+ def test_plain_stored_names_parse_on_a_step_and_a_branch(self, controller: str) -> None:+ step = SubPipeBlueprint(pipe="write_receipt", batch_over="lines", batch_as="total_amount", result="receipts")+ if controller == "sequence":+ sequence = _parse_sequence(+ steps_toml=' { pipe = "write_receipt", batch_over = "lines", batch_as = "total_amount", result = "receipts" },'+ )+ assert sequence.steps == [step]+ else:+ parallel = PipeParallelBlueprint.model_validate(+ {"description": "Writes receipts in parallel", "output": "Composite", "branches": [step.model_dump(exclude_none=True)]}+ )+ assert parallel.branches == [step]def test_a_plain_batch_over_in_a_parallel_branch_parses(self) -> None:blueprint = PipeParallelBlueprint.model_validate(diff --git a/tests/unit/pipelex/pipe_controllers/test_sub_pipe_blueprint.py b/tests/unit/pipelex/pipe_controllers/test_sub_pipe_blueprint.pyindex 1a73e059a..520fd21ef 100644--- a/tests/unit/pipelex/pipe_controllers/test_sub_pipe_blueprint.py+++ b/tests/unit/pipelex/pipe_controllers/test_sub_pipe_blueprint.py@@ -69,44 +69,89 @@ class TestSubPipeBlueprint:)@pytest.mark.parametrize(- ("step_fields", "field_name", "reserved_name"),+ "stored_name",[- pytest.param({"result": "_bound_catalog_pages"}, "result", "_bound_catalog_pages", id="result"),- pytest.param({"batch_over": "catalog_pages", "batch_as": "_bound_page"}, "batch_as", "_bound_page", id="batch-as"),- pytest.param({"batch_over": "_bound_catalog_pages", "batch_as": "page"}, "batch_over", "_bound_catalog_pages", id="plain-batch-over"),+ pytest.param("Pages", id="pascal-case"),+ pytest.param("catalog.pages", id="dotted"),+ pytest.param("_bound_catalog_pages", id="the-reserved-prefix"),+ pytest.param("_draft", id="underscore-led"),+ pytest.param("2nd_pages", id="digit-led"),+ pytest.param("page-views", id="hyphenated"),+ pytest.param("...", id="dots-only"),],)- def test_a_name_taking_the_reserved_prefix_is_refused_as_invalid_input_name(- self, step_fields: dict[str, str], field_name: str, reserved_name: str- ) -> None:- """The `_bound_` prefix is the runtime's, for the list a dotted `batch_over` binds in a caller's working memory."""+ @pytest.mark.parametrize("field_name", ["result", "batch_as"])+ def test_a_stored_name_that_is_not_a_plain_input_name_is_refused_as_invalid_input_name(self, field_name: str, stored_name: str) -> None:+ """A step stores its `result`, and each item under `batch_as`, for a later pipe to read through an input, so both are plain names."""+ step_fields: dict[str, str] = {"result": "descriptions", "batch_over": "pages", "batch_as": "page", field_name: stored_name}with pytest.raises(ValidationError) as exc_info:SubPipeBlueprint.model_validate({"pipe": "describe_page", **step_fields})refusal = exc_info.value.errors()[0].get("ctx", {}).get("error")assert isinstance(refusal, PipeValidationError)assert refusal.error_type == PipeValidationErrorType.INVALID_INPUT_NAME- assert refusal.variable_names == [reserved_name]- message = str(refusal)- assert f"The `{field_name}` of the step running pipe 'describe_page', '{reserved_name}', takes the `_bound_` prefix" in message- assert "reserved for the bound list of a dotted `batch_over`" in message- assert "Choose another name" in message+ assert refusal.variable_names == [stored_name]+ assert f"The `{field_name}` of the step running pipe 'describe_page', '{stored_name}', is not a plain input name" in str(refusal)++ @pytest.mark.parametrize(+ ("stored_name", "message_fragment"),+ [+ pytest.param("Pages", "Rename it to a plain name, such as 'pages',", id="pascal-case"),+ pytest.param("PagesOfTheCatalog", "Rename it to a plain name, such as 'pages_of_the_catalog',", id="a-longer-pascal-case-name"),+ pytest.param("catalog.pages", "cannot reach into a field with a dot. Rename it to a plain name, such as 'catalog_pages',", id="dotted"),+ pytest.param(+ "_bound_catalog_pages",+ "the `_bound_` prefix it starts with is reserved for the bound list of a dotted `batch_over`",+ id="the-reserved-prefix",+ ),+ pytest.param("_bound_catalog_pages", "Rename it to a plain name, such as 'catalog_pages',", id="the-reserved-prefix-suggestion"),+ pytest.param("page-views", "Rename it to a plain name, such as 'page_views',", id="hyphenated"),+ pytest.param("...", "Rename it to a plain name, and", id="no-suggestion"),+ ],+ )+ def test_the_refusal_explains_the_form_and_suggests_a_plain_name(self, stored_name: str, message_fragment: str) -> None:+ with pytest.raises(ValidationError) as exc_info:+ SubPipeBlueprint(pipe="describe_page", result=stored_name)- def test_the_refusal_suggests_the_name_without_the_prefix(self) -> None:+ assert message_fragment in str(exc_info.value)++ def test_a_plain_batch_over_taking_the_reserved_prefix_is_refused_as_invalid_input_name(self) -> None:+ """A plain `batch_over` reads a name rather than storing one, but never a name the runtime binds under its reserved prefix."""with pytest.raises(ValidationError) as exc_info:- SubPipeBlueprint(pipe="describe_page", result="_bound_catalog_pages")+ SubPipeBlueprint(pipe="describe_page", batch_over="_bound_catalog_pages", batch_as="page")- assert "Choose another name, such as 'catalog_pages'." in str(exc_info.value)+ refusal = exc_info.value.errors()[0].get("ctx", {}).get("error")+ assert isinstance(refusal, PipeValidationError)+ assert refusal.error_type == PipeValidationErrorType.INVALID_INPUT_NAME+ assert refusal.variable_names == ["_bound_catalog_pages"]+ message = str(refusal)+ assert "The `batch_over` of the step running pipe 'describe_page', '_bound_catalog_pages', takes the `_bound_` prefix" in message+ assert "reserved for the bound list of a dotted `batch_over`" in message+ assert "Choose another name, such as 'catalog_pages'." in message@pytest.mark.parametrize(- "name",+ "stored_name",[+ pytest.param("pages", id="a-plain-name"),pytest.param("bound_pages", id="the-prefix-without-its-underscore"),- pytest.param("_draft", id="another-underscore-led-name"),pytest.param("catalog_bound_pages", id="the-prefix-inside-the-name"),+ pytest.param("page2", id="a-digit-after-the-first-letter"),+ ],+ )+ def test_a_plain_stored_name_is_accepted(self, stored_name: str) -> None:+ blueprint = SubPipeBlueprint(pipe="describe_page", result=stored_name, batch_over="catalog_pages", batch_as=stored_name)++ assert blueprint.result == blueprint.batch_as == stored_name++ @pytest.mark.parametrize(+ "batch_over",+ [+ pytest.param("PagesOfTheCatalog", id="pascal-case"),+ pytest.param("_draft_pages", id="underscore-led-outside-the-reserved-prefix"),],)- def test_a_name_outside_the_reserved_prefix_is_accepted(self, name: str) -> None:- blueprint = SubPipeBlueprint(pipe="describe_page", result=name, batch_over="pages", batch_as=name)+ def test_a_plain_batch_over_is_held_only_to_the_reserved_prefix(self, batch_over: str) -> None:+ """A plain `batch_over` reads a name rather than storing one, so the standard holds it only off the reserved prefix."""+ blueprint = SubPipeBlueprint(pipe="describe_page", batch_over=batch_over, batch_as="page")- assert blueprint.result == blueprint.batch_as == name+ assert blueprint.batch_over == batch_over