Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
feature/Binding-step · L-260911-65f829 (#1347)
Louis Choquel6f062b2938d4
diff --git a/.pipelex/plxt.toml b/.pipelex/plxt.tomlindex eda6fb249..d8c625617 100644--- a/.pipelex/plxt.toml+++ b/.pipelex/plxt.toml@@ -51,6 +51,12 @@ exclude = [# narrows which rule applies. What guards these entries instead is strictly stronger: the corpus's# entry-validation gate runs the real validation engine over every entry in CI and is red both when# an invalid entry fails differently and when it accidentally validates.+ "**/mthds_corpus/entries/invalid_binding_step_batch_fields/*.mthds",+ "**/mthds_corpus/entries/invalid_binding_step_malformed_path/*.mthds",+ "**/mthds_corpus/entries/invalid_binding_step_parallel_branch/*.mthds",+ "**/mthds_corpus/entries/invalid_binding_step_pipe_and_from/*.mthds",+ "**/mthds_corpus/entries/invalid_binding_step_result_not_plain/*.mthds",+ "**/mthds_corpus/entries/invalid_binding_step_without_result/*.mthds","**/mthds_corpus/entries/invalid_input_name_lone_dotted/*.mthds","**/mthds_corpus/entries/invalid_input_name_redundant_dotted/*.mthds","**/mthds_corpus/entries/invalid_missing_pipe_type/*.mthds",diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 2c6f19c1c..d56cfed1c 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -4,7 +4,8 @@### Added-- **`pipelex fix bundle` deletes a dotted input key beside its root**: the new rule `delete-redundant-dotted-input` removes a key such as `"page.page_view" = "Image"` from a pipe's `inputs` when the same table declares its root `page`, which already supplies every field a template reads through it. The fix is safe, and applied, only when the root keeps the presence marker and multiplicity the table gave it: a key that set them otherwise, such as `"data.text" = "Text!"` declared after `data = "Text?"`, gets an unsafe fix that `pipelex fix bundle` never applies, and both the fix and the refusal name only what differs, the multiplicity, the presence marker or both, with the edit that carries it onto the root (here, replace `?` with `!` on `data`). A lone dotted key gets no automatic fix, since nothing states its root's concept, and its refusal names the two remedies instead.+- **The binding step**: a PipeSequence step `{ from = "invoice.total", result = "total_amount" }` binds a deep copy of the value at a path in working memory to a new name, as a fresh stuff whose concept is derived from the declared structures the path walks (`Number` for a `number` field, `Image` for `page.page_view`, `X[]` for a path crossing a list, which is mapped and flattened), and a later step reads it under that plain name. A path reaching a field that holds nothing records a `DECLARED_ABSENT` absence and an absent root skips the binding, a step reading the result is checked against the derived concept, a step asking a sequence that ends with a binding for a count of outputs the binding does not bind is refused as `inadequate_output_multiplicity`, a binding derived as a single value that finds a list at run time fails rather than keep one item, and the execution graph draws the binding as a node of the new `NodeKind` `binding`. A malformed step is refused as the new `PipeValidationErrorType` value `binding_step_invalid`, which the MTHDS JSON Schema refuses too, through a `oneOf` of the two step shapes with the path grammar on `from` and the plain-name pattern on `result`, and a path the structures cannot walk, into a concept declared with a description alone included, as `binding_path_unresolved`. The typed flow types a batched step's result as `X[]` and a dependency package's concept by the package's own definitions, and a path may end on a field holding `native.Anything`, whose value is stored as an `Anything` input is. What a nested sequence, a condition's outcome or an `add_each_output` parallel stores in the caller's memory is read alike by the caller's typed flow, needed inputs and absence analysis, for a pipe step reading it as for a binding: a name it always stores types the root, carries its absence and is no input of the caller, so a sequence that declared such a name only to pass validation now has an extraneous input, a name only some outcomes store is still an input, and a nested sequence or condition skipped for an absent plain input records an absence for every name it always stores. A binding whose root may hold values of different concepts, as when the outcomes of a condition store it under different concepts, is refused before the run as `binding_path_unresolved`, the message naming each outcome and the concept it stores, while one whose root a pipe unresolved at validation stores is derived when it runs and checked then against the sequence's output and the input of the step reading it, a mismatch being a run error; the MTHDS Test Corpus covers both refusals and the `feature.binding_step` tag, and its offline-tier entries now run with their `inputs.json` in the test suite.+- **`pipelex fix bundle` deletes a dotted input key beside its root**: the new rule `delete-redundant-dotted-input` removes a key such as `"page.page_view" = "Image"` from a pipe's `inputs` when the same table declares its root `page`, which already supplies every field a template reads through it. The fix is safe, and applied, only when the root's declared concept holds the field the key names and the root keeps the presence marker and multiplicity the table gave it: a root declared with a concept that lacks the field, such as `data = "Number"` beside `"data.text" = "Text"`, or a key that set the markers otherwise, such as `"data.text" = "Text!"` declared after `data = "Text?"`, gets an unsafe fix that `pipelex fix bundle` never applies, and both the fix and the refusal name what differs, with the edit that settles it on the root (here, replace `?` with `!` on `data`). A lone dotted key gets no automatic fix, since nothing states its root's concept, and its refusal names the two remedies instead.### Changed@@ -15,9 +16,11 @@- **An unknown variable prefix is refused wherever it stands in a fallback chain (Breaking)**: `${env:KEY|secrte:KEY}` is refused for its `secrte` prefix whether or not `KEY` is set, where a boot read the candidates in turn and accepted the chain whenever an earlier one resolved. A boot without inference, which resolves nothing, refuses it too, so a file gets the same verdict on its syntax on every boot and every machine.- **`pipelex show backends` lists every backend with its credential variables**: a backend whose key is unset is listed instead of hidden, a new column names the variables each backend reads its credentials from, and an endpoint read from a variable is shown as not resolved.- **An input name is a plain name (Breaking)**: every key of a pipe's `inputs`, on every operator, controller and signature, and a PipeBatch's `input_list_name` must match `[a-z][a-z0-9_]*`, so a dotted name such as `"invoice.total" = "Number"` or a malformed one such as `InvoiceTotal` is refused when the bundle is parsed, as the new `PipeValidationErrorType` value `invalid_input_name`, which also replaces PipeJudge's own refusal that reached callers with no `error_type`. Its message names the remedies: declare the root with its whole concept (`invoice = "Invoice"`) and read the field in the template (`$invoice.total`), which every operator handles by the concept the path reaches, so `@page.page_view` with `page = "Page"` is still attached as an image, or have the calling sequence bind the field with a binding step (`{ from = "invoice.total", result = "total_amount" }`). The generated MTHDS JSON Schema refuses the same names, through `patternProperties` on every `inputs` table and a `pattern` on `input_list_name`, and the corpus vocabulary records `error.invalid_input_name` with `fails_at = "schema"`.+- **A sequence's steps include binding steps (Breaking)**: `PipeSequenceBlueprint.steps` holds `BindingStepBlueprint` beside `SubPipeBlueprint`, and `PipeSequence.sequential_sub_pipes` holds `BindingStep` beside `SubPipe`, so code reading `pipe` or `pipe_code` off every step reads the new `pipe_steps` property instead. A PipeParallel's `branches` keep the pipe-step shape.### Fixed+- **A PipeBatch branch gets its own copy of its item**: each branch's item is taken from the branch's own copy of the batched list, so a branch that rewrites its item in place, as a `PipeFunc` can, no longer changes the batch's input list.- **A dry-run condition's output is one stuff every outcome produces**: in the graph of a dry run, every outcome of a `PipeCondition` now carries the condition's output digest, so the step reading the condition's result is wired to all of them rather than to the outcome whose pipe code sorts last, and when the outcomes write or declare different outputs, the condition's declared output types that stuff on every node outside the outcomes: the condition, the controllers whose output it is, such as a sequence ending on the condition, and the steps reading it, while each outcome keeps the concept it writes. Each outcome but the last now runs on a copy of the memory the condition received, so an outcome reading the condition's slot reads the value from before the condition rather than a sibling's output, and the default outcome runs last, so the steps after the condition read its value.- **Validation and dry runs no longer depend on which keys the machine holds**: a boot without inference (`Pipelex.make(needs_inference=False)`, used by `pipelex validate`, `pipelex run --dry-run`, `pipelex show` and the agent CLI's validation) now loads every enabled backend with its models and constraints and resolves none of their credentials, where it dropped every backend whose key was missing. A method using a search preset or pinning a bare handle such as `model = "gpt-4o-mini"` now validates on a machine or CI runner with no key, and such a boot no longer reads the backends' keys or requests a Vertex AI token.- **Linkup uses the key its backend names**: the Linkup search and fetch workers take the key from the `[linkup]` table's `api_key`, like every other provider, where they read `LINKUP_API_KEY` themselves and ignored a configuration naming another variable. A Linkup backend with no `api_key` is refused with `LinkupError` when its worker is built.diff --git a/docs/api-server/openapi/pipelex-api.openapi.yaml b/docs/api-server/openapi/pipelex-api.openapi.yamlindex 0074bb2d4..8e8f37854 100644--- a/docs/api-server/openapi/pipelex-api.openapi.yaml+++ b/docs/api-server/openapi/pipelex-api.openapi.yaml@@ -1175,9 +1175,10 @@ components:- not_providedtitle: AbsenceKinddescription: "How a slot came to hold no value.\n\n- DECLARED_ABSENT: a producer with an optional (`?`) output declared\- \ it produced nothing\n (e.g. a PipeCondition `continue` outcome, or — phase 2 — an LLM maybe-wrapper).\n- SKIPPED:\- \ the producing pipe was lifted (skipped) because one of its plain inputs was absent.\n- NOT_PROVIDED: the caller\- \ omitted an optional method input from the pipeline inputs."+ \ it produced nothing\n (e.g. a PipeCondition `continue` outcome, or — phase 2 — an LLM maybe-wrapper), or a binding\n\+ \ step's path reached a field holding nothing.\n- SKIPPED: the producing pipe was lifted (skipped) because one of\+ \ its plain inputs was absent,\n or a binding step was, because its root was.\n- NOT_PROVIDED: the caller omitted\+ \ an optional method input from the pipeline inputs."AbsenceRecord:properties:variable_name:@@ -1236,6 +1237,32 @@ components:- opaque- autotitle: Background+ BindingStepBlueprint:+ properties:+ from:+ type: string+ pattern: ^[A-Za-z][A-Za-z0-9_]*(?:\.[A-Za-z][A-Za-z0-9_]*)*$+ title: From+ description: 'The path to bind: a name in working memory followed by zero or more field names, separated by dots.'+ result:+ type: string+ pattern: ^[a-z][a-z0-9_]*$+ title: Result+ description: 'The name under which the bound value is stored in working memory: a plain input name.'+ additionalProperties: false+ type: object+ required:+ - from+ - result+ title: BindingStepBlueprint+ description: 'A PipeSequence step that binds the value at a path in working memory to a new name.+++ `{ from = "invoice.total", result = "total_amount" }` stores a deep copy of the value at `invoice.total`++ under `total_amount`, as the concept the path derives through the declared structures. A binding step++ lives in a PipeSequence''s `steps` only, and carries exactly these two fields.'BooleanField:properties:title:@@ -3609,6 +3636,7 @@ components:- output- artifact- error+ - bindingtitle: NodeKinddescription: Types of nodes in the execution graph.NodeSpec:@@ -5384,7 +5412,9 @@ components:title: Outputsteps:items:- $ref: '#/components/schemas/SubPipeBlueprint'+ oneOf:+ - $ref: '#/components/schemas/SubPipeBlueprint'+ - $ref: '#/components/schemas/BindingStepBlueprint'type: arraytitle: StepsadditionalProperties: false@@ -5511,6 +5541,8 @@ components:- unknown_pipe_type- missing_pipe_type- batch_item_name_collision+ - binding_step_invalid+ - binding_path_unresolved- optional_marker_invalid- optional_not_handled- optional_output_requireddiff --git a/docs/building-methods/pipes/pipe-controllers/PipeBatch.md b/docs/building-methods/pipes/pipe-controllers/PipeBatch.mdindex b77216125..2c3efb199 100644--- a/docs/building-methods/pipes/pipe-controllers/PipeBatch.md+++ b/docs/building-methods/pipes/pipe-controllers/PipeBatch.md@@ -14,7 +14,7 @@ This is the ideal controller for processing collections of documents, images, or1. **Input List**: It identifies an input list from the working memory.2. **Branching**: For each item in the input list, it creates a new, isolated execution branch.-3. **Isolation & Injection**: Each branch gets a deep copy of the `WorkingMemory`. The specific item for that branch is injected into this memory with a defined name.+3. **Isolation & Injection**: Each branch gets a deep copy of the `WorkingMemory`. The specific item for that branch, taken from the branch's own copy of the list, is injected into this memory with a defined name, so a branch that rewrites its item in place leaves the batched list and the other branches as they were.4. **Concurrent Execution**: The specified `branch_pipe_code` runs across the branches concurrently — in bounded chunks, by default at most `max_concurrency` branches at a time. Each branch pipe operates only on its own item.5. **Aggregation**: After all branches have completed, `PipeBatch` collects the individual output from each one and aggregates them into a single new list. This list becomes the final output of the `PipeBatch` pipe.diff --git a/docs/building-methods/pipes/pipe-controllers/PipeParallel.md b/docs/building-methods/pipes/pipe-controllers/PipeParallel.mdindex 33d6d2059..ab8af5966 100644--- a/docs/building-methods/pipes/pipe-controllers/PipeParallel.md+++ b/docs/building-methods/pipes/pipe-controllers/PipeParallel.md@@ -64,6 +64,8 @@ Each entry in the `branches` array is a table with the following keys:| `pipe` | string | The name of the pipe to execute for this branch. | Yes || `result` | string | The name for this branch's output. Must be unique within the `PipeParallel` definition. | Yes |+A branch is always a pipe step. A [binding step](PipeSequence.md#binding-steps), `{ from = "invoice.total", result = "total_amount" }`, is refused in `branches` with `binding_step_invalid`: the branches run at once, and a binding orders a value before the steps that read it, so bind the value in the calling sequence, in a step before the `PipeParallel`, and have the branch read the bound name.+### Example: Extracting different details from a textImagine you have a product description and you want to extract the product features and the product sentiment at the same time.diff --git a/docs/building-methods/pipes/pipe-controllers/PipeSequence.md b/docs/building-methods/pipes/pipe-controllers/PipeSequence.mdindex 42c6e5744..ff28c2873 100644--- a/docs/building-methods/pipes/pipe-controllers/PipeSequence.md+++ b/docs/building-methods/pipes/pipe-controllers/PipeSequence.md@@ -8,11 +8,11 @@ The `PipeSequence` controller is used to execute a series of pipes one after ano## How it works-A `PipeSequence` defines a list of `steps`. Each step calls another pipe and gives a name to its output. The working memory is passed from one step to the next, accumulating results along the way.+A `PipeSequence` defines a list of `steps`. A step is either a **pipe step**, which calls another pipe and gives a name to its output, or a **binding step**, which binds the value at a path in working memory to a new name (see [Binding steps](#binding-steps)). The working memory is passed from one step to the next, accumulating results along the way.- The `input` of the `PipeSequence` is passed to the first pipe in the sequence.- The `output` of each intermediate step is named via the `result` key and becomes available in the working memory for all subsequent steps.-- The final `output` of the `PipeSequence` is the output produced by the very last step in the sequence.+- The final `output` of the `PipeSequence` is the output produced by the very last step in the sequence, a pipe step's output or the value a binding step binds.## Configuration@@ -26,11 +26,15 @@ A `PipeSequence` defines a list of `steps`. Each step calls another pipe and giv| `description` | string | A description of the sequence operation. | Yes || `inputs` | dictionary | The input concept(s) for the *first* pipe in the sequence, as a dictionary mapping input names to concept codes. | No || `output` | string | The output concept produced by the *last* pipe in the sequence. | Yes |-| `steps` | array of tables | An ordered list of the pipes to execute. Each table in the array defines a single step. | Yes |+| `steps` | array of tables | An ordered list of the steps to run. Each table in the array defines a single step, a pipe step or a binding step. | Yes |### Step Configuration-Each entry in the `steps` array is a table with the following keys:+A step carrying `pipe` is a pipe step, and a step carrying `from` is a binding step. A step carries one or the other, never both, and each kind is a closed table: a key the kind does not define is refused.++#### Pipe steps++A pipe step is a table with the following keys:| Key | Type | Description | Required || -------- | ------ | ------------------------------------------------------------------ | -------- |@@ -41,8 +45,100 @@ Each entry in the `steps` array is a table with the following keys:| `batch_over` | string | The name of a list in the working memory to batch this step over, running the pipe once per item. Must be provided together with `batch_as`. See [Understanding Multiplicity](../understanding-multiplicity.md). | No || `batch_as` | string | The name each item takes in the working memory during a `batch_over` run. Must differ from `batch_over` (e.g. `batch_over = "items"`, `batch_as = "item"`). | No |+#### Binding steps++A binding step is a table with exactly these two keys:++| Key | Type | Description | Required |+| -------- | ------ | ------------------------------------------------------------------ | -------- |+| `from` | string | The path to bind: a name in working memory followed by zero or more field names, separated by single dots, each a letter followed by letters, digits and underscores. No subscript, expression or whitespace. | Yes |+| `result` | string | The name the bound value is stored under, a plain snake_case input name. | Yes |+!!! important "Output Concept Matching"- The output concept of the `PipeSequence` has to match the output of the last pipe in the sequence.+ The output concept of the `PipeSequence` has to match the output of its last step: the output of the last pipe, or the concept a binding step derives when the sequence ends with one.++## Binding steps++A binding step hands one part of a bigger value to the steps after it, under a name of its own. An input name is always a plain name, so a pipe never declares `"invoice.total" = "Number"`: the calling sequence binds the field, and the pipe reads the bound name.++```toml+[pipe.acknowledge_invoice]+type = "PipeSequence"+description = "Acknowledges an invoice by its total"+inputs = { invoice = "Invoice" }+output = "Text"+steps = [+ { from = "invoice.total", result = "total_amount" },+ { pipe = "write_receipt", result = "receipt" },+]++[pipe.write_receipt]+type = "PipeCompose"+description = "Writes the receipt for an amount"+inputs = { total_amount = "Number" }+output = "Text"+template = "Received: $total_amount euros"+```++### What a binding binds++The concept of the result is derived before the method runs, from the declared structures the path walks. The first segment names a value in working memory, the root, and each later segment names a field of the concept reached so far.++| The path ends on | The result is |+| --- | --- |+| The root itself, `from = "departure_board"` | A renamed copy of the whole value, with its concept and multiplicity |+| A field of concept `X`, or a field whose concept is `X` | `X` |+| A `text` field, or a field declared by its `choices` | `Text` |+| A `number` or `integer` field | `Number` |+| A `boolean` field | `YesNo` |+| A `date` or `datetime` field | `Date` (a `datetime` keeps its time) |+| A `time` field | `Time` |+| A `dict` field | `JSON` |+| A `list` field of `X`, or of a plain type | `X[]`, or the native the plain type derives, as a list |+| A field holding `Anything` | `Anything`, its value stored as an `Anything` input is: a string as a `Text`, a number as a `Number`, an object as a `JSON`, and so on. A list, or a value of no such type, is refused when the step runs |++The root is typed by the latest step that stored a value under its name, or by the sequence's `inputs` when no step did. A step stores its `result`, and a batched step stores the list of its branches' results, `X[]`, whatever `nb_output` it carries. A nested `PipeSequence` runs its steps on the caller's working memory and a `PipeCondition` runs its chosen outcome there, so a name either of them stores counts as stored by the step that calls it, as do the branch results of a `PipeParallel` with `add_each_output`: a later step, a pipe step or a binding, reads it with no input of that name, and the name keeps its concept and whether it may be absent. A name only some outcomes of a condition store, when another outcome stores nothing under it or is `continue`, may still hold the value it held before, so the sequence needs it as an input, and it keeps its concept only when both values have the same one. When the values a name may hold have different concepts, its concept is not known before the run, so a binding reading it is refused with `binding_path_unresolved`, the message naming each outcome and the concept it stores. That happens when the outcomes of a condition store the name under different concepts, or when one outcome stores a value of another concept than the one an outcome storing nothing, such as `continue`, leaves. Store the name under one concept in every outcome, or bind inside each outcome, where its concept is known.++A root stored by a pipe that does not resolve at validation, such as a pipe of a dependency not loaded yet, is assumed to deliver, as the rest of the sequence's checks assume. Its binding is derived when it runs, from the value the root holds, and checked then against what reads it: the sequence's declared output when the binding ends the sequence, its concept, its multiplicity and whether it may be absent, and the input of a later step reading the result. A mismatch is a run error, never a value of another concept. A root that may be absent still makes the result maybe-absent.++A concept that refines another is walked through the structure it inherits, and a native concept through its pinned definition, so `page.page_view` binds an `Image`, every field of it kept, its `caption` included. A root holding a dependency package's concept, the output of one of the package's pipes, is walked through the package's own definitions, never through a concept of the method spelled the same.++A path is refused with `binding_path_unresolved` when the walk cannot follow it, and the message names the segment and the fields the concept does have:++- a segment naming no field of the concept reached;+- a segment after a plain field (`invoice.total.amount`), after a `dict` field, or after a list with no declared item type;+- a segment into a concept with no structure to walk: a concept declared with neither a `structure` nor `refines`, whether as a string (`Notice = "A notice"`) or as a table with a description alone, unless its code names a registered Python class, whose fields are walked; `Dynamic`, `Anything`, `Composite`; a native holding its value in a single field (`Text`, `Number`, `Time`, `JSON`, `Markdown`); or a concept refining one of these. Bind the value itself instead, `from = "note"`.++A path ending on a list with no declared item type is refused too, since nothing says what its items are.++### Lists map and flatten++When the path crosses a list, whether the root holds a list or a field does, the rest of the path is applied to every item, and every list crossed is flattened into one. The result is always one flat list, `X[]`: binding `pages.page_view` over `Page[]` gives `Image[]`, one image per page, and binding `shipments.parcels` over `Shipment[]` gives every parcel of every shipment. Items holding nothing are dropped, and an empty list is a valid result, so a list result is never absent.++A single result never chooses one item of a list. When the run finds a list under a root typed as a single value, as when a count asked of the whole run reaches the steps of the sequence, the binding step fails with a run error naming the path and both shapes, rather than keep the first item. A bare name typed as a list that finds a single value fails the same way.++### Absence++A binding that finds nothing records an absence, under the optionality model (see [Understanding Optionality](../understanding-optionality.md#binding-steps-under-absence)):++- when the path reaches a field holding nothing, the result is recorded as `DECLARED_ABSENT`, and the reason names the segment that held nothing;+- when the root itself is absent, the binding is skipped as a pipe with an absent plain input is, and the result is recorded as `SKIPPED`, chained to the root's own record.++A single result can be absent when its path walks a field that is not `required` and has no default, or, on a concept whose structure is a Python class, a field whose annotation admits `None`, required or not, so the static checks treat it as a maybe-absent value: a step reading it as a plain input may be lifted, and a sequence ending with it declares its output optional (`Text?`), or the validation refuses it with `optional_not_handled`.++### A copy, with an identity of its own++The bound value is a deep copy taken when the step runs: changing the root afterwards, or storing another value under the root's name, leaves the bound value as it was, and the reverse. The result is a new stuff with its own stuff code, and in the execution graph the binding is a node of its own, of kind `binding`, fed by the root's producer and producing the stuff it binds.++### What validation checks++- A step that reads a bound name is checked against the derived concept and multiplicity: reading `total_amount` as `Text` when the binding derives a `Number` is refused with `input_stuff_spec_mismatch`, and `batch_over` a bound name requires a list.+- A sequence ending with a binding step is checked as one ending with a pipe: its output concept with `inadequate_output_concept`, its multiplicity with `inadequate_output_multiplicity`, and a maybe-absent result with `optional_not_handled`.+- A step asking a sequence that ends with a binding step for a count of outputs, with `nb_output` or `multiple_output`, is refused with `inadequate_output_multiplicity` when the binding does not bind that count: a binding binds what its path derives, whatever count its caller asks for. A batched step asks its branches for nothing, so it is not checked.+- A root that is neither an input of the sequence nor stored by an earlier step on every run is refused with `missing_input_variable`.+- A malformed step is refused with `binding_step_invalid`: `pipe` beside `from`, a binding without `result`, a binding carrying `nb_output`, `multiple_output`, `batch_over` or `batch_as`, a `from` or a `result` outside its grammar. The schema refuses each of these too.++A binding step lives in a `PipeSequence`'s `steps` only. A [`PipeParallel`](PipeParallel.md) branch is always a pipe step, since its branches run at once and a binding orders a value before the steps that read it: bind in the sequence before the parallel.### Examplediff --git a/docs/building-methods/pipes/understanding-optionality.md b/docs/building-methods/pipes/understanding-optionality.mdindex f8ac1b1b6..c1e26e35c 100644--- a/docs/building-methods/pipes/understanding-optionality.md+++ b/docs/building-methods/pipes/understanding-optionality.md@@ -84,11 +84,21 @@ The guard-lint covers every pipe that renders authored templates over its inputsEach controller has a defined behavior when a value inside its flow resolves absent:-- **PipeSequence** — steps consuming an absent value plainly are lifted (skipped), and the absence propagates step to step. The sequence's own declared output must account for it: if the last producing step can be lifted, the sequence output must be declared `?` (otherwise `optional_not_handled`).+- **PipeSequence** — steps consuming an absent value plainly are lifted (skipped), and the absence propagates step to step. The sequence's own declared output must account for it: if the last producing step can be lifted, the sequence output must be declared `?` (otherwise `optional_not_handled`). A name a nested sequence, a condition's outcome or an `add_each_output` parallel stores in the sequence's memory carries its absence the same way, and a nested controller that is lifted records an absence for every name it always stores (an empty list for a list), so a later step reading one is lifted in turn.- **PipeCondition** — the `continue` outcome resolves the declared output as a **declared-absent** record (memory otherwise unchanged). A condition that can reach `continue` must declare its output `?` (`optional_output_required`). See [PipeCondition](pipe-controllers/PipeCondition.md).- **PipeParallel** — combining under absence: a `Composite` output omits the absent component (with a ledger note); a structured output absorbs an absent branch into a non-required field as that field's declared default; a **required** field fed by a maybe-absent branch is rejected statically (`optional_branch_required_field`). See [PipeParallel](pipe-controllers/PipeParallel.md).- **PipeBatch** — absent branch results are **compacted** out of the aggregated list (a list cannot hold a hole). See [PipeBatch](pipe-controllers/PipeBatch.md).+### Binding steps under absence++A [binding step](pipe-controllers/PipeSequence.md#binding-steps) of a PipeSequence, `{ from = "delivery.note", result = "note" }`, can find nothing, and records it the way a pipe records an absent output:++- **The path reaches a field holding nothing.** When `delivery.note` holds nothing, or `invoice.scan.url` is read from an invoice with no `scan`, the result is recorded as **declared-absent**, and the reason names the binding step, the sequence and the segment that held nothing (`'invoice.scan'`). The step does not fail.+- **The root is absent.** When `delivery` is itself absent, for instance an optional method input the caller did not provide, the binding is **skipped** like a pipe whose plain input is absent, and its record chains to the root's own record, so the provenance runs from the result back to where the absence entered the flow.+- **A list result is never absent.** A binding that crosses a list drops the items holding nothing and may bind an empty list, and over an absent root it binds an empty list, keeping the skip as a note.++Statically, a single result is **maybe-absent** when its path walks a field that is not `required` and has no default, or, on a concept whose structure is a Python class, a field whose annotation admits `None` (`str | None`), even a required one. It then flows like any maybe-absent value: a later step reading it as a plain input may be lifted, a step reading it as `X?` runs with its guard, and a sequence ending with it, or with a step it may lift, declares its output `?` or is refused with `optional_not_handled`, whose message names the binding step as the absence origin. A `required` field, or one with a default, binds a value that is never absent.+## The Static Safety NetYou never discover an unhandled absence at run time. Validation walks every controller flow and computes, per slot, whether it is guaranteed present or maybe-absent — and requires every possible absence to reach an explicit sink (a `?` boundary, a guard, an absorbing field). The optionality error types:diff --git a/docs/contribute/mthds-test-corpus.md b/docs/contribute/mthds-test-corpus.mdindex bf1ce7356..17a9af761 100644--- a/docs/contribute/mthds-test-corpus.md+++ b/docs/contribute/mthds-test-corpus.md@@ -56,7 +56,9 @@ An entry directory holds either exactly one `.mthds` file, or several with a `bu.venv/bin/pipelex validate bundle pipelex/test_extras/mthds_corpus/entries/native_time_departure/```-6. **Run the gates**: `.venv/bin/pytest tests/unit/pipelex/test_extras tests/integration/pipelex/test_extras`.+6. **Give an `offline` entry its expectations.** An `offline` entry ships an `inputs.json` and runs for real in the offline runner, which checks what the run leaves in working memory against the expectations in `tests/integration/pipelex/test_extras/test_data.py`, keyed by the entry's name: a value's concept and the fields its content must hold, or a recorded absence and its kind.++7. **Run the gates**: `.venv/bin/pytest tests/unit/pipelex/test_extras tests/integration/pipelex/test_extras`.An **invalid** entry additionally declares the exact wire `error_type` it must produce. Read that string off the runtime rather than inferring it from the source:@@ -129,6 +131,7 @@ The tier governs execution, never validation: the entry-validation gate below pa| Manifest | `tests/unit/pipelex/test_extras/test_mthds_corpus_manifest.py` | The strict `entry.toml` model rejected something. || Layout and filters | `tests/unit/pipelex/test_extras/test_mthds_corpus_loader.py` | An entry's name does not match its directory, its bundle does not resolve, or the loader's filter semantics changed. || Entry validation | `tests/integration/pipelex/test_extras/test_mthds_corpus_entries.py` | A valid entry stopped validating, or an invalid one stopped failing with exactly its declared error. Runs over every entry regardless of tier, so an `inference`-tier entry that broke is caught without spending a token. |+| Offline runs | `tests/integration/pipelex/test_extras/test_mthds_corpus_offline_runs.py` | An `offline`-tier entry stopped running live with its `inputs.json`, or left values in working memory other than the ones its expectations name. An entry holds no expected outputs, so each offline entry's expectations live in that suite's `test_data.py`, keyed by entry name, and a new offline entry with none there is red too. || Packaging | `tests/integration/pipelex/test_extras/test_mthds_corpus_packaging.py` | A corpus file stopped shipping in the wheel, or the generator started shipping. It builds a real wheel and derives what it expects from the corpus tree, so a new entry is covered with no wiring. |## Consuming the corpus from a testdiff --git a/docs/errors/authoring-and-language.md b/docs/errors/authoring-and-language.mdindex 374ea7117..a71452327 100644--- a/docs/errors/authoring-and-language.md+++ b/docs/errors/authoring-and-language.md@@ -95,6 +95,8 @@ own page. Classes are grouped by subsystem.## Pipe controllers+- [`BindingPathUnresolvedError`](binding-path-unresolved-error.md) — Binding path unresolved+- [`BindingStepRunError`](binding-step-run-error.md) — Binding step run- [`PipeBatchFactoryError`](pipe-batch-factory-error.md) — Pipe batch factory- [`PipeConditionFactoryError`](pipe-condition-factory-error.md) — Pipe condition factory- [`PipeControllerError`](pipe-controller-error.md) — Pipe controllerdiff --git a/docs/errors/binding-path-unresolved-error.md b/docs/errors/binding-path-unresolved-error.mdnew file mode 100644index 000000000..0d61364b5--- /dev/null+++ b/docs/errors/binding-path-unresolved-error.md@@ -0,0 +1,21 @@+---+title: "Binding path unresolved"+description: "Reference for the `BindingPathUnresolvedError` Pipelex error class."+---++<!-- pipelex:generated -->++# Binding path unresolved++A binding step's `from` path that cannot be walked before the run: the declared structures do not hold it, or the concept of its root is not known.++| Field | Value |+|---|---|+| `error_type` | `BindingPathUnresolvedError` |+| `title` | Binding path unresolved |+| `type_uri` | `https://docs.pipelex.com/latest/errors/binding-path-unresolved-error/` |+| `error_domain` | _(inherited from parent)_ |+| Defined in | `pipelex.pipe_controllers.binding.exceptions` |+| Parent class | [`PipelexError`](pipelex-error.md) |++[Back to Error Reference](index.md)diff --git a/docs/errors/binding-step-run-error.md b/docs/errors/binding-step-run-error.mdnew file mode 100644index 000000000..758c3e461--- /dev/null+++ b/docs/errors/binding-step-run-error.md@@ -0,0 +1,21 @@+---+title: "Binding step run"+description: "Reference for the `BindingStepRunError` Pipelex error class."+---++<!-- pipelex:generated -->++# Binding step run++A binding step that could not bind its value at run time, although its path was derived before the run.++| Field | Value |+|---|---|+| `error_type` | `BindingStepRunError` |+| `title` | Binding step run |+| `type_uri` | `https://docs.pipelex.com/latest/errors/binding-step-run-error/` |+| `error_domain` | _(inherited from parent)_ |+| Defined in | `pipelex.pipe_controllers.binding.exceptions` |+| Parent class | [`PipelexError`](pipelex-error.md) |++[Back to Error Reference](index.md)diff --git a/docs/under-the-hood/execution-graph-tracing.md b/docs/under-the-hood/execution-graph-tracing.mdindex 86b366e67..8ef637df3 100644--- a/docs/under-the-hood/execution-graph-tracing.md+++ b/docs/under-the-hood/execution-graph-tracing.md@@ -194,6 +194,7 @@ class GraphSpec(BaseModel):| `OUTPUT` | Pipeline output node || `ARTIFACT` | Generated artifact || `ERROR` | Error node |+| `BINDING` | A PipeSequence binding step, `{ from = "invoice.total", result = "total_amount" }`: its `pipe_code` is the `from` path and its `pipe_type` is `BindingStep`; its input is the root's stuff and its output the stuff it binds, so DATA edges run from the root's producer to the binding and from the binding to every step reading the result |### Edge Typesdiff --git a/pipelex/cli/dev_cli/commands/generate_corpus_vocabulary_cmd.py b/pipelex/cli/dev_cli/commands/generate_corpus_vocabulary_cmd.pyindex b2dbc0fd5..14c3ce0bc 100644--- a/pipelex/cli/dev_cli/commands/generate_corpus_vocabulary_cmd.py+++ b/pipelex/cli/dev_cli/commands/generate_corpus_vocabulary_cmd.py@@ -91,6 +91,12 @@ _PIPE_TYPE_PREFIX = "Pipe"# coverage the corpus does not have is worse than a short vocabulary. Rendered in alphabetical order,# there being no registry order to follow._FEATURE_TAGS: dict[str, str] = {+ "binding_step": (+ 'A PipeSequence step `{ from = "invoice.total", result = "total_amount" }` that binds the value at a '+ "path in working memory to a new name: its concept is derived from the declared structures the path walks, "+ "its value is a deep copy taken when the step runs, a path crossing a list binds a flat list, and a path "+ "reaching nothing records an absence."+ ),"optionals": ("An input declared optional with the `?` suffix, so a run that has no value for it proceeds ""instead of failing and the absence is recorded in the working memory. How a method copes with "@@ -224,6 +230,11 @@ _ERROR_TYPE_SCHEMA_FAULTS: frozenset[ValidationErrorType] = frozenset(# both of its entries: plxt reports `error[schema]` on the lone dotted key and on the one beside# its declared root.PipeValidationErrorType.INVALID_INPUT_NAME,+ # A malformed binding step: `pipe` beside `from`, a binding without `result` or carrying a pipe step's+ # batch fields, a `from` outside the path grammar or a `result` outside the plain-name grammar, and a+ # binding written as a parallel branch. The schema's closed step shapes and their patterns reject each.+ # Measured on all of its entries: plxt reports `error[schema]` on every one of them.+ PipeValidationErrorType.BINDING_STEP_INVALID,})diff --git a/pipelex/core/concepts/concept.py b/pipelex/core/concepts/concept.pyindex 0233b77d1..9aabf1dec 100644--- a/pipelex/core/concepts/concept.py+++ b/pipelex/core/concepts/concept.py@@ -1,12 +1,13 @@from typing import Any, Callablefrom mthds.protocol.concept import ConceptAbstract-from pydantic import field_validator+from pydantic import Field, field_validatorfrom pipelex.core.concepts.concept_representation_generator import (ConceptRepresentationFormat,ConceptRepresentationGenerator,)+from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprintfrom pipelex.core.concepts.exceptions import ConceptCodeError, ConceptValueErrorfrom pipelex.core.concepts.native.concept_native import NativeConceptCodefrom pipelex.core.concepts.validation import is_concept_ref_or_code_valid, validate_concept_code@@ -37,6 +38,24 @@ class Concept(ConceptAbstract):description: strstructure_class_name: strrefines: str | None = None+ declared_structure: dict[str, ConceptStructureBlueprint] | None = Field(default=None, exclude=True)+ """The structure the concept's blueprint declares, field by field, normalized; `None` when it declares none.++ A concept that refines another, one whose structure is a Python class, a native (whose definition is+ pinned, see `pinned_blueprints`) and one declared with a description alone carry `None`. The walk that+ derives a binding step's concept reads the declared fields here, because a structure class cannot say+ which concept a field holds when several concepts share one class. Excluded from serialization, like+ every definition field the concept's ref stands for on the wire.+ """+ is_described_only: bool = Field(default=False, exclude=True)+ """Whether the concept is declared with a description alone, neither a `structure` nor `refines`.++ That is `Note = "A note"`, or a `[concept.Note]` table holding only a description. The runtime gives such a+ concept a text placeholder for a structure class, and the table spelling refines `native.Text` besides, but the+ standard gives it no structure to walk, so the walk deriving a binding step's concept refuses to enter it+ whichever way it was written. A description-only concept whose code names a registered Python class takes that+ class as its structure, and is not described only. Excluded from serialization, like `declared_structure`.+ """@field_validator("code")@classmethoddiff --git a/pipelex/core/concepts/concept_factory.py b/pipelex/core/concepts/concept_factory.pyindex 06c567009..5afa89b83 100644--- a/pipelex/core/concepts/concept_factory.py+++ b/pipelex/core/concepts/concept_factory.py@@ -42,6 +42,8 @@ def _is_valid_structure_class(*, structure_class_name: str) -> bool:class StructureNameAndRefine(NamedTuple):structure_class_name: strrefine_string: str | None+ # Whether the class is the text placeholder a description-only concept gets, rather than a Python class named after it.+ is_text_placeholder: bool = Falseclass ConceptDeclarationType(StrEnum):@@ -413,7 +415,7 @@ class ConceptFactory:if _is_valid_structure_class(structure_class_name=concept_code):return StructureNameAndRefine(structure_class_name=concept_code, refine_string=None)if _is_valid_structure_class(structure_class_name=qualified_class_name):- return StructureNameAndRefine(structure_class_name=qualified_class_name, refine_string=None)+ return StructureNameAndRefine(structure_class_name=qualified_class_name, refine_string=None, is_text_placeholder=True)# Because native concepts have structure class names diffrent than other (with "Content")if concept_code in NativeConceptCode.values_list():@@ -435,7 +437,9 @@ class ConceptFactory:# Register the generated classget_class_registry().register_class(the_generated_class)- return StructureNameAndRefine(structure_class_name=qualified_class_name, refine_string=NativeConceptCode.TEXT.concept_ref)+ return StructureNameAndRefine(+ structure_class_name=qualified_class_name, refine_string=NativeConceptCode.TEXT.concept_ref, is_text_placeholder=True+ )@classmethoddef _handle_refines(@@ -546,7 +550,7 @@ class ConceptFactory:match declaration_type:case ConceptDeclarationType.STRING:assert isinstance(blueprint_or_string_description, str)- structure_class_name, _ = cls._handle_basic_blueprint(+ basic_structure = cls._handle_basic_blueprint(concept_code=concept_code,domain_code=domain_code,description=blueprint_or_string_description,@@ -555,12 +559,13 @@ class ConceptFactory:domain_code=domain_and_concept_code.domain_code,code=domain_and_concept_code.concept_code,description=blueprint_or_string_description,- structure_class_name=structure_class_name,+ structure_class_name=basic_structure.structure_class_name,+ is_described_only=basic_structure.is_text_placeholder,)case ConceptDeclarationType.BASIC_BLUEPRINT:assert isinstance(blueprint_or_string_description, ConceptBlueprint)- structure_class_name, refines = cls._handle_basic_blueprint(+ basic_structure = cls._handle_basic_blueprint(concept_code=concept_code,domain_code=domain_code,description=blueprint_or_string_description.description,@@ -569,8 +574,9 @@ class ConceptFactory:domain_code=domain_and_concept_code.domain_code,code=domain_and_concept_code.concept_code,description=blueprint_or_string_description.description,- structure_class_name=structure_class_name,- refines=refines,+ structure_class_name=basic_structure.structure_class_name,+ refines=basic_structure.refine_string,+ is_described_only=basic_structure.is_text_placeholder,)case ConceptDeclarationType.STRUCTURE_WITH_CLASSNAME:@@ -589,6 +595,9 @@ class ConceptFactory:case ConceptDeclarationType.BLUEPRINT_WITH_STRUCTURE:blueprint = cast("ConceptBlueprint", blueprint_or_string_description)+ if not isinstance(blueprint.structure, dict):+ msg = f"Expected structure to be a dict, got {type(blueprint.structure)}"+ raise ConceptFactoryError(msg)return Concept(domain_code=domain_and_concept_code.domain_code,code=domain_and_concept_code.concept_code,@@ -599,11 +608,12 @@ class ConceptFactory:domain_code=domain_code,),refines=None,+ declared_structure=normalize_structure_blueprint(blueprint.structure),)case ConceptDeclarationType.REFINES:blueprint = cast("ConceptBlueprint", blueprint_or_string_description)- structure_class_name, current_refine = cls._handle_refines(+ refined_structure = cls._handle_refines(blueprint=blueprint,concept_code=concept_code,domain_code=domain_code,@@ -612,6 +622,6 @@ class ConceptFactory:domain_code=domain_and_concept_code.domain_code,code=domain_and_concept_code.concept_code,description=blueprint.description,- structure_class_name=structure_class_name,- refines=current_refine,+ structure_class_name=refined_structure.structure_class_name,+ refines=refined_structure.refine_string,)diff --git a/pipelex/core/exceptions.py b/pipelex/core/exceptions.pyindex 1bdd49768..55d6733e6 100644--- a/pipelex/core/exceptions.py+++ b/pipelex/core/exceptions.py@@ -81,6 +81,11 @@ class PipelexBundleBlueprintValidationErrorData(BaseModel):# warning compares the two markers' multiplicity and presence apart, and names only the part that differs.root_input_marker: str | None = None+ # Set beside ``redundant_input_name`` only when the root's declared concept does not hold the field the dotted key named,+ # as the binding walk over the bundle's own concepts finds: the walk's refusal. Deleting the key would then leave the+ # root typed by a concept that lacks the field, so the planner offers the deletion as unsafe, whatever the markers say.+ redundant_input_unwalkable_reason: str | None = None+class PipesAndConceptValidationErrorData(BaseModel):"""Structured validation error data for Pipe/Concept validation errors.diff --git a/pipelex/core/memory/absence.py b/pipelex/core/memory/absence.pyindex 616f0e6c2..65848ee77 100644--- a/pipelex/core/memory/absence.py+++ b/pipelex/core/memory/absence.py@@ -9,8 +9,10 @@ class AbsenceKind(StrEnum):"""How a slot came to hold no value.- DECLARED_ABSENT: a producer with an optional (`?`) output declared it produced nothing- (e.g. a PipeCondition `continue` outcome, or — phase 2 — an LLM maybe-wrapper).- - SKIPPED: the producing pipe was lifted (skipped) because one of its plain inputs was absent.+ (e.g. a PipeCondition `continue` outcome, or — phase 2 — an LLM maybe-wrapper), or a binding+ step's path reached a field holding nothing.+ - SKIPPED: the producing pipe was lifted (skipped) because one of its plain inputs was absent,+ or a binding step was, because its root was.- NOT_PROVIDED: the caller omitted an optional method input from the pipeline inputs."""@@ -18,6 +20,14 @@ class AbsenceKind(StrEnum):SKIPPED = "skipped"NOT_PROVIDED = "not_provided"+ @property+ def is_skipped(self) -> bool:+ match self:+ case AbsenceKind.SKIPPED:+ return True+ case AbsenceKind.DECLARED_ABSENT | AbsenceKind.NOT_PROVIDED:+ return False+class AbsenceRecord(BaseModel):"""A recorded fact that a named slot holds no value, with provenance (D2).diff --git a/pipelex/graph/graphspec.py b/pipelex/graph/graphspec.pyindex 51710f3ea..8cd0da058 100644--- a/pipelex/graph/graphspec.py+++ b/pipelex/graph/graphspec.py@@ -48,6 +48,8 @@ class NodeKind(StrEnum):OUTPUT = "output"ARTIFACT = "artifact"ERROR = "error"+ # A PipeSequence binding step: it runs no pipe, and produces the stuff it binds from a path in working memory.+ BINDING = "binding"class NodeStatus(StrEnum):diff --git a/pipelex/graph/mermaidflow/mermaidflow_factory.py b/pipelex/graph/mermaidflow/mermaidflow_factory.pyindex e02ffe42b..76a893004 100644--- a/pipelex/graph/mermaidflow/mermaidflow_factory.py+++ b/pipelex/graph/mermaidflow/mermaidflow_factory.py@@ -378,6 +378,9 @@ class MermaidflowFactory:case NodeKind.ERROR:# Rectangle with failed classnode_str = f'{mermaid_id}["{label}"]:::failed'+ case NodeKind.BINDING:+ # Parallelogram for a binding step, which reshapes a value rather than running a pipe+ node_str = f'{mermaid_id}[/"{label}"/]'case NodeKind.CONTROLLER | NodeKind.PIPE_CALL | NodeKind.OPERATOR:# Rectangle for operators/pipesif node.status == NodeStatus.FAILED:diff --git a/pipelex/libraries/crate_qualification.py b/pipelex/libraries/crate_qualification.pyindex 63df8e757..7bd001e3b 100644--- a/pipelex/libraries/crate_qualification.py+++ b/pipelex/libraries/crate_qualification.py@@ -57,6 +57,7 @@ from pipelex.pipe_controllers.condition.pipe_condition_blueprint import PipeCondfrom pipelex.pipe_controllers.condition.special_outcome import SpecialOutcomefrom pipelex.pipe_controllers.parallel.pipe_parallel_blueprint import PipeParallelBlueprintfrom pipelex.pipe_controllers.sequence.pipe_sequence_blueprint import PipeSequenceBlueprint+from pipelex.pipe_controllers.sub_pipe_blueprint import SubPipeBlueprintif TYPE_CHECKING:from pipelex.pipe_machinery.pipe_blueprint import InputSlotBlueprint@@ -184,7 +185,11 @@ def _qualify_pipe_blueprint(blueprint: PipeBlueprintUnion, *, owner_domain: str,match blueprint:case PipeSequenceBlueprint():- updates["steps"] = [step.model_copy(update={"pipe": qualify(pipe_ref=step.pipe)}) for step in blueprint.steps]+ # A binding step names no pipe, only a path in working memory, so it is kept as written.+ updates["steps"] = [+ step.model_copy(update={"pipe": qualify(pipe_ref=step.pipe)}) if isinstance(step, SubPipeBlueprint) else step+ for step in blueprint.steps+ ]case PipeParallelBlueprint():updates["branches"] = [branch.model_copy(update={"pipe": qualify(pipe_ref=branch.pipe)}) for branch in blueprint.branches]case PipeConditionBlueprint():diff --git a/pipelex/mthds_parsing/pipelex_bundle_blueprint.py b/pipelex/mthds_parsing/pipelex_bundle_blueprint.pyindex df7a59e5b..0fbdb1b05 100644--- a/pipelex/mthds_parsing/pipelex_bundle_blueprint.py+++ b/pipelex/mthds_parsing/pipelex_bundle_blueprint.py@@ -16,6 +16,7 @@ from pipelex.pipe_controllers.batch.pipe_batch_blueprint import PipeBatchBlueprifrom pipelex.pipe_controllers.condition.pipe_condition_blueprint import PipeConditionBlueprintfrom pipelex.pipe_controllers.parallel.pipe_parallel_blueprint import PipeParallelBlueprintfrom pipelex.pipe_controllers.sequence.pipe_sequence_blueprint import PipeSequenceBlueprint+from pipelex.pipe_controllers.sub_pipe_blueprint import SubPipeBlueprintfrom pipelex.pipe_machinery.pipe_blueprint import normalize_typeless_signature_sectionfrom pipelex.pipe_machinery.validation import is_pipe_code_validfrom pipelex.pipe_operators.compose.pipe_compose_blueprint import PipeComposeBlueprint@@ -219,7 +220,9 @@ class PipelexBundleBlueprint(BaseModel):for pipe_code, pipe_blueprint in self.pipe.items():if isinstance(pipe_blueprint, PipeSequenceBlueprint):for step_index, step in enumerate(pipe_blueprint.steps):- pipe_refs.append((step.pipe, f"pipe.{pipe_code}.steps[{step_index}].pipe"))+ # A binding step names no pipe, only a path in working memory.+ if isinstance(step, SubPipeBlueprint):+ pipe_refs.append((step.pipe, f"pipe.{pipe_code}.steps[{step_index}].pipe"))elif isinstance(pipe_blueprint, PipeBatchBlueprint):pipe_refs.append((pipe_blueprint.branch_pipe_code, f"pipe.{pipe_code}.branch_pipe_code"))elif isinstance(pipe_blueprint, PipeConditionBlueprint):diff --git a/pipelex/mthds_parsing/validation_error_categorizer.py b/pipelex/mthds_parsing/validation_error_categorizer.pyindex d5a23c2fd..6de0c3379 100644--- a/pipelex/mthds_parsing/validation_error_categorizer.py+++ b/pipelex/mthds_parsing/validation_error_categorizer.py@@ -1,24 +1,33 @@import refrom typing import Any, cast, get_args+from pydantic import ValidationErrorfrom pydantic_core import ErrorDetailsfrom pydantic_core.core_schema import CoreSchemaTypefrom pipelex import log+from pipelex.core.concepts.concept_blueprint import ConceptBlueprintfrom pipelex.core.exceptions import PipelexBundleBlueprintValidationErrorData+from pipelex.core.pipes.exceptions import PipeVariableMultiplicityError+from pipelex.core.pipes.variable_multiplicity import parse_concept_with_multiplicityfrom pipelex.mthds_parsing.exceptions import (InvalidPipeCodeSyntaxError,NativeConceptRedeclarationError,)from pipelex.mthds_parsing.handle_pipe_errors import extract_wrapped_pipe_validation_errorfrom pipelex.mthds_parsing.helpers import ValidationErrorScope, get_error_scope+from pipelex.pipe_controllers.binding.binding_concept_resolvers import BlueprintConceptWalkResolver, qualify_concept_ref+from pipelex.pipe_controllers.binding.binding_derivation import BindingRoot, derive_binding+from pipelex.pipe_controllers.binding.exceptions import BindingPathUnresolvedErrorfrom pipelex.pipe_machinery.pipe_blueprint import PIPE_SIGNATURE_TYPE_TAG, PipeType+from pipelex.tools.misc.string_utils import get_root_from_dotted_pathfrom pipelex.validation_error_types import PipeValidationErrorTypePIPELEX_BUNDLE_BLUEPRINT_DOMAIN_FIELD = "domain"PIPELEX_BUNDLE_BLUEPRINT_SOURCE_FIELD = "source"PIPELEX_BUNDLE_BLUEPRINT_MAIN_PIPE_FIELD = "main_pipe"PIPELEX_BUNDLE_BLUEPRINT_PIPE_FIELD = "pipe"+PIPELEX_BUNDLE_BLUEPRINT_CONCEPT_FIELD = "concept"# Distinctive fragments of the two type-tag errors raised by the bundle blueprint's `pipe`# before-validator (via `normalize_typeless_signature_section` in `pipe_blueprint.py`). Kept in sync@@ -108,6 +117,60 @@ def _main_pipe_strip_is_safe(*, offending_code: str, stripped_code: str, blueprireturn (offending_code in pipe_keys) != (stripped_code in pipe_keys)+def _redundant_input_unwalkable_reason(+ *,+ dotted_input_name: str,+ pipe_code: str | None,+ domain: str | None,+ blueprint_dict: dict[str, Any],+) -> str | None:+ """Why a redundant dotted input's path cannot be read through its root's declared concept, or ``None`` when it can.++ Deleting the dotted key leaves the root typed by its own declaration, which the deletion is safe for only when+ that concept holds the field the key named. The raise site sees the `inputs` table alone, so the check is made+ here, on the raw bundle dict, by the binding walk over the bundle's own concepts and the natives: a concept the+ bundle does not declare cannot be walked, and the deletion is then not proven safe. A template reads a+ single-field native's pinned field (`$data.text` on a `Text`), which a binding refuses as a leaf, so the walk is+ told to read those natives through their field.+ """+ if pipe_code is None or domain is None:+ return "the pipe or its domain is unknown"+ raw_pipes = blueprint_dict.get(PIPELEX_BUNDLE_BLUEPRINT_PIPE_FIELD)+ raw_pipe = cast("dict[str, Any]", raw_pipes).get(pipe_code) if isinstance(raw_pipes, dict) else None+ raw_inputs = cast("dict[str, Any]", raw_pipe).get("inputs") if isinstance(raw_pipe, dict) else None+ root_name = get_root_from_dotted_path(dotted_input_name)+ raw_root_spec = cast("dict[str, Any]", raw_inputs).get(root_name) if isinstance(raw_inputs, dict) else None+ if not isinstance(raw_root_spec, str):+ return f"the root '{root_name}' is not declared with a concept"+ try:+ parsed_root_spec = parse_concept_with_multiplicity(raw_root_spec)+ except PipeVariableMultiplicityError:+ return f"the root '{root_name}' is declared as '{raw_root_spec}', which is not a concept"+ concept_blueprints: dict[str, ConceptBlueprint | str] = {}+ raw_concepts = blueprint_dict.get(PIPELEX_BUNDLE_BLUEPRINT_CONCEPT_FIELD)+ if isinstance(raw_concepts, dict):+ for concept_code, raw_concept in cast("dict[Any, Any]", raw_concepts).items():+ if not isinstance(concept_code, str):+ continue+ if isinstance(raw_concept, str):+ concept_blueprints[f"{domain}.{concept_code}"] = raw_concept+ continue+ try:+ concept_blueprints[f"{domain}.{concept_code}"] = ConceptBlueprint.model_validate(raw_concept)+ except ValidationError:+ continue+ root_concept_ref = qualify_concept_ref(concept_ref=parsed_root_spec.concept_ref_or_code, domain_code=domain)+ try:+ derive_binding(+ path=dotted_input_name,+ root=BindingRoot(concept_ref=root_concept_ref, multiplicity=parsed_root_spec.multiplicity),+ resolver=BlueprintConceptWalkResolver(concept_blueprints=concept_blueprints, walks_single_field_natives=True),+ )+ except BindingPathUnresolvedError as exc:+ return str(exc)+ return None++def _categorize_typeless_pipe_error(message: str,*,@@ -349,6 +412,16 @@ def categorize_blueprint_validation_error(# bundle dict, preferring any the error does carry.wrapped_pipe_error = extract_wrapped_pipe_validation_error(error)if wrapped_pipe_error is not None:+ # A redundant dotted input is safe to delete only when the root's declared concept holds the field the key named,+ # which only this document-level view, holding the bundle's concepts, can check.+ redundant_input_unwalkable_reason: str | None = None+ if wrapped_pipe_error.redundant_input_name is not None:+ redundant_input_unwalkable_reason = _redundant_input_unwalkable_reason(+ dotted_input_name=wrapped_pipe_error.redundant_input_name,+ pipe_code=wrapped_pipe_error.pipe_code or pipe_code,+ domain=domain,+ blueprint_dict=blueprint_dict,+ )return PipelexBundleBlueprintValidationErrorData(error_type=wrapped_pipe_error.error_type,domain_code=wrapped_pipe_error.domain_code or domain,@@ -359,6 +432,7 @@ def categorize_blueprint_validation_error(redundant_input_name=wrapped_pipe_error.redundant_input_name,dropped_input_marker=wrapped_pipe_error.dropped_input_marker,root_input_marker=wrapped_pipe_error.root_input_marker,+ redundant_input_unwalkable_reason=redundant_input_unwalkable_reason,)# A native-concept redeclaration: ``validate_concept_keys`` raised a typed ``ValueError``diff --git a/pipelex/pipe_controllers/absence_taint.py b/pipelex/pipe_controllers/absence_taint.pyindex 355dd299b..184e1a2b4 100644--- a/pipelex/pipe_controllers/absence_taint.py+++ b/pipelex/pipe_controllers/absence_taint.py@@ -14,34 +14,23 @@ batched step compacts absent branch results, so a list slot is always guaranteedThe walk logic itself lives on the controllers (`PipeSequence.analyze_taint`,`PipeParallel.analyze_branch_taint`); this module holds the shared value objects and thepresence-resolution helper so the pipeline layer (liftable-pipe inventory) can consume the-same shapes without importing controller internals.+same shapes without importing controller internals. `SlotTaint` itself lives in+`pipelex.pipe_machinery.memory_writes`, beside the model of what a nested controller stores in+its caller's memory, which carries it."""+from dataclasses import fieldfrom typing import NamedTuplefrom pydantic.dataclasses import dataclassfrom pipelex.core.pipes.inputs.input_stuff_specs import NamedStuffSpecfrom pipelex.core.pipes.variable_multiplicity import PresenceMarker, VariableMultiplicity+from pipelex.pipe_machinery.memory_writes import SlotTaintfrom pipelex.pipe_machinery.pipe_abstract import PipeAbstractfrom pipelex.pipe_run.pipe_run_params import output_multiplicity_to_apply-@dataclass(frozen=True)-class SlotTaint:- """A maybe-absent slot: where the absence originates and how it propagated here."""-- source: str- origin_slot_name: str- chain: tuple[str, ...] = ()-- def describe(self) -> str:- description = f"Absence origin: {self.source}."- if self.chain:- description += f" Propagation: {' → '.join(self.chain)}."- return description--@dataclass(frozen=True)class LiftableStepInfo:"""One pipe that may be lifted (skipped) within a controller's flow."""@@ -75,6 +64,8 @@ class SequenceTaintAnalysis:liftable_steps: tuple[LiftableStepInfo, ...]output_taint: SlotTaint | Noneforce_consumptions: tuple[ForceConsumptionInfo, ...] = ()+ # The maybe-absent slots once the last step ran, which give the absence of what the sequence stores in its caller's memory.+ final_slot_taints: dict[str, SlotTaint] = field(default_factory=dict[str, SlotTaint])@dataclass(frozen=True)@@ -99,18 +90,22 @@ class TaintTriggerScan(NamedTuple):redundant_force_names: tuple[str, ...] = ()-def scan_taint_triggers(pipe: PipeAbstract, *, slot_taints: dict[str, SlotTaint]) -> TaintTriggerScan:+def scan_taint_triggers(pipe: PipeAbstract, *, slot_taints: dict[str, SlotTaint], visited_pipes: set[str] | None = None) -> TaintTriggerScan:"""Apply the D3 trichotomy to a pipe's needed inputs against the given taint map.A tainted slot consumed PLAIN is a lift trigger; consumed `?` (absorb) or `!` (assert),the taint terminates at that consumption. `!` consumptions are also classified for theuseless-`!` lint: asserting when the slot was tainted, redundant when it was guaranteed.++ `visited_pipes` is handed to the pipe's `needed_inputs` when the scan runs inside a walk that+ is itself guarded against recursion, as `memory_writes` is, so a pipe that reaches itself again+ through its steps stops there."""trigger_names: list[str] = []trigger_taint: SlotTaint | None = Noneasserting_force_names: list[str] = []redundant_force_names: list[str] = []- for named_stuff_spec in pipe.needed_inputs().named_stuff_specs:+ for named_stuff_spec in pipe.needed_inputs(visited_pipes=visited_pipes).named_stuff_specs:incoming_taint = slot_taints.get(named_stuff_spec.variable_name)match effective_consumption_presence(pipe=pipe, named_stuff_spec=named_stuff_spec):case PresenceMarker.PLAIN:diff --git a/pipelex/pipe_controllers/batch/pipe_batch.py b/pipelex/pipe_controllers/batch/pipe_batch.pyindex 8f0ec0f14..353decc7e 100644--- a/pipelex/pipe_controllers/batch/pipe_batch.py+++ b/pipelex/pipe_controllers/batch/pipe_batch.py@@ -23,7 +23,6 @@ from pipelex.urls import URLsif TYPE_CHECKING:from collections.abc import Awaitable, Callable- from pipelex.core.stuffs.stuff import Stufffrom pipelex.core.stuffs.stuff_content import StuffContentfrom pipelex.libraries.library_crate import LibraryCrate@@ -141,8 +140,20 @@ class PipeBatch(PipeController):f"rate-limited, resumable runs: {URLs.durable_execution}")- async def _run_branch(item_input_stuff: "Stuff", *, branch_output_item_code: str) -> PipeOutput:+ item_concept = self.inputs.get_required_stuff_spec(input_list_stuff_name).concept++ async def _run_branch(*, branch_index: int, branch_input_item_code: str, branch_output_item_code: str) -> PipeOutput:branch_memory = working_memory.make_deep_copy()+ # The item is taken from the branch's own copy of the list, never from the parent's: a branch gets+ # its own copy of its item as it does of the rest of working memory, so a branch that rewrites its+ # item in place (a PipeFunc can) leaves the batched list and every sibling branch as they were.+ branch_list_content = cast("ListContent[StuffContent]", branch_memory.get_stuff(input_list_stuff_name).content)+ item_input_stuff = StuffFactory.make_stuff(+ code=branch_input_item_code,+ concept=item_concept,+ content=branch_list_content.items[branch_index],+ name=input_item_stuff_name,+ )branch_memory.set_new_main_stuff(stuff=item_input_stuff, name=input_item_stuff_name)# We create a deep copy of the run params to avoid modifying the original run params,@@ -174,15 +185,9 @@ class PipeBatch(PipeController):# Build one factory per branch. Each factory defers its working-memory deep copy until it# actually runs, so gather_bounded materializes at most `max_concurrency` of them at once.branch_factories: list[Callable[[], Awaitable[PipeOutput]]] = []- for branch_index, item in enumerate(input_content.items):+ for branch_index in range(item_count):branch_output_item_code = f"{batch_output_stuff_code}-branch-{branch_index}"branch_input_item_code = f"{input_stuff.stuff_code}-branch-{branch_index}"- item_input_stuff = StuffFactory.make_stuff(- code=branch_input_item_code,- concept=self.inputs.get_required_stuff_spec(input_list_stuff_name).concept,- content=item,- name=input_item_stuff_name,- )# Register batch item extraction with graph tracerif job_metadata.trace_context is not None:@@ -198,7 +203,14 @@ class PipeBatch(PipeController):batch_controller_node_id=batch_controller_node_id,)- branch_factories.append(functools.partial(_run_branch, item_input_stuff, branch_output_item_code=branch_output_item_code))+ branch_factories.append(+ functools.partial(+ _run_branch,+ branch_index=branch_index,+ branch_input_item_code=branch_input_item_code,+ branch_output_item_code=branch_output_item_code,+ )+ )pipe_outputs = await gather_bounded(branch_factories, max_concurrency=max_concurrency)diff --git a/pipelex/pipe_controllers/binding/__init__.py b/pipelex/pipe_controllers/binding/__init__.pynew file mode 100644index 000000000..e69de29bbdiff --git a/pipelex/pipe_controllers/binding/binding_concept_resolvers.py b/pipelex/pipe_controllers/binding/binding_concept_resolvers.pynew file mode 100644index 000000000..933e43221--- /dev/null+++ b/pipelex/pipe_controllers/binding/binding_concept_resolvers.py@@ -0,0 +1,456 @@+"""The two ways the binding walk reads a concept: from a loaded library, or from a bundle's own blueprints.++Both answer the one question the walk asks, `resolve_walkable_concept`, with the same `WalkableConcept`:++- `LibraryConceptWalkResolver` reads the concepts of a loaded library, which is what a sequence validates and+ runs against. A concept declaring a structure is walked through the fields its blueprint declares, kept on+ the runtime concept as `declared_structure`; a refinement through the structure it inherits; a native+ through its pinned definition; a concept declared with a description alone not at all, however it was+ written. A concept whose structure exists only as a Python class is walked through the class's fields, a+ field typed by a content class mapping to the concept registered for that class when exactly one concept+ is, and to nothing otherwise, which the walk refuses. A dependency package's concepts are held only under+ their aliased keys (`alias->domain.Code`), so the walk reads a dependency's concept, and every concept its+ fields and its `refines` name, under that package's alias: a host concept of the same spelling never+ supplies its structure.+- `BlueprintConceptWalkResolver` reads the concept blueprints of one bundle, before any library is loaded,+ which is what the fix planner needs to decide whether deleting a redundant dotted input is safe. What the+ bundle does not declare, it does not know, and the walk refuses it. The planner asks whether a template's+ read holds, so it walks a single-field native through its pinned field, where a binding stops.+"""++import datetime+import types+from collections.abc import Mapping+from typing import Any, Literal, Union, get_args, get_origin++from pydantic.fields import FieldInfo+from typing_extensions import override++from pipelex.core.concepts.concept import Concept+from pipelex.core.concepts.concept_blueprint import ConceptBlueprint, ConceptStructureBlueprintType+from pipelex.core.concepts.concept_provider_abstract import ConceptProviderAbstract+from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprint, ConceptStructureBlueprintFieldType+from pipelex.core.concepts.exceptions import ConceptStructureClassNotFoundError+from pipelex.core.concepts.helpers import normalize_structure_blueprint+from pipelex.core.concepts.native.concept_native import NativeConceptCode+from pipelex.core.concepts.native.pinned_blueprints import make_pinned_native_blueprint+from pipelex.core.domains.domain import SpecialDomain+from pipelex.core.qualified_ref import QualifiedRef+from pipelex.core.stuffs.stuff_content import StuffContent+from pipelex.libraries.concept.concept_library_abstract import ConceptLibraryAbstract+from pipelex.libraries.concept.exceptions import ConceptLibraryError+from pipelex.pipe_controllers.binding.binding_derivation import (+ BindingValueKind,+ ConceptShape,+ ConceptWalkResolver,+ WalkableConcept,+ WalkableField,+)++_SINGLE_FIELD_NATIVE_REASON = "holds its value in a single field"+_DESCRIBED_ONLY_REASON = "is declared with neither a structure nor refines"+++def qualify_concept_ref(*, concept_ref: str, domain_code: str, package_alias: str | None = None) -> str:+ """A concept ref as a structure field or a `refines` writes it, made domain-qualified in the declaring domain.++ When the declaring concept belongs to a dependency package, `package_alias` is that package's alias, and a ref to+ one of its concepts is keyed under it (`alias->domain.Code`), the only key the library holds a dependency's concept+ under. A native ref, and a ref that already names its package, are left as they are.+ """+ if QualifiedRef.has_cross_package_prefix(concept_ref):+ return concept_ref+ if NativeConceptCode.is_native_concept_ref_or_code(concept_ref_or_code=concept_ref):+ return f"{SpecialDomain.NATIVE}.{concept_ref.rsplit('.', maxsplit=1)[-1]}"+ domain_qualified_ref = concept_ref if "." in concept_ref else f"{domain_code}.{concept_ref}"+ if package_alias is None:+ return domain_qualified_ref+ return f"{package_alias}->{domain_qualified_ref}"+++def library_concept_key(*, concept_library: ConceptProviderAbstract, concept: Concept) -> str:+ """The key under which a library holds this very concept: its ref for a host or native concept, `alias->domain.Code` for a dependency's.++ A dependency package's concept reports the plain `domain.Code` its package declares, while the library holds it only+ under its aliased key, beside any host concept spelled the same. The walk must start from that key to read the+ package's own definitions. The concept is matched by identity, since pipes are built with the very concept objects+ the library holds, then by equality; failing both, a spelling held under one key alone is that key, and the+ concept's own ref is the answer otherwise.+ """+ candidate_keys = concept_library.list_concept_keys_for_ref(concept_ref=concept.concept_ref)+ candidate_concepts = {candidate_key: concept_library.get_required_concept(concept_ref=candidate_key) for candidate_key in candidate_keys}+ for candidate_key, candidate_concept in candidate_concepts.items():+ if candidate_concept is concept:+ return candidate_key+ equal_keys = [candidate_key for candidate_key, candidate_concept in candidate_concepts.items() if candidate_concept == concept]+ if len(equal_keys) == 1:+ return equal_keys[0]+ if len(candidate_keys) == 1:+ return candidate_keys[0]+ return concept.concept_ref+++def _package_alias_of(*, concept_key: str) -> str | None:+ """The alias of the dependency package a library key holds a concept under, `None` for a host or native concept."""+ if not QualifiedRef.has_cross_package_prefix(concept_key):+ return None+ package_alias, _ = QualifiedRef.split_cross_package_ref(concept_key)+ return package_alias+++def walkable_native_concept(*, native_code: NativeConceptCode, walks_single_field: bool = False) -> WalkableConcept:+ """A native concept as the walk sees it, read off its pinned definition, never off its content class.++ A native whose pinned definition is a single field holding the value itself (`Text`, `Number`, `Time`, `JSON`) is a+ leaf to a binding, which refuses to walk into it. A template reads that field all the same (`$note.text`), so a check+ asking whether a template's read holds passes `walks_single_field` to walk it through its pinned field.+ """+ pinned_blueprint = make_pinned_native_blueprint(native_code)+ if not isinstance(pinned_blueprint.structure, dict):+ return WalkableConcept(concept_ref=native_code.concept_ref, shape=ConceptShape.NO_STRUCTURE, shape_reason="is structureless by definition")+ if len(pinned_blueprint.structure) == 1 and not walks_single_field:+ return WalkableConcept(concept_ref=native_code.concept_ref, shape=ConceptShape.VALUE, shape_reason=_SINGLE_FIELD_NATIVE_REASON)+ return WalkableConcept(+ concept_ref=native_code.concept_ref,+ shape=ConceptShape.STRUCTURE,+ fields=walkable_fields_from_structure(structure=pinned_blueprint.structure, domain_code=SpecialDomain.NATIVE),+ )+++def _scalar_value_kind(*, field_type: ConceptStructureBlueprintFieldType) -> BindingValueKind | None:+ """The value kind of a plain field type, or `None` for the types that are not plain values (`concept`, `list`)."""+ match field_type:+ case ConceptStructureBlueprintFieldType.TEXT:+ return BindingValueKind.TEXT+ case ConceptStructureBlueprintFieldType.NUMBER | ConceptStructureBlueprintFieldType.INTEGER:+ return BindingValueKind.NUMBER+ case ConceptStructureBlueprintFieldType.BOOLEAN:+ return BindingValueKind.YES_NO+ case ConceptStructureBlueprintFieldType.DATE:+ return BindingValueKind.DATE+ case ConceptStructureBlueprintFieldType.DATETIME:+ return BindingValueKind.DATETIME+ case ConceptStructureBlueprintFieldType.TIME:+ return BindingValueKind.TIME+ case ConceptStructureBlueprintFieldType.DICT:+ return BindingValueKind.JSON+ case ConceptStructureBlueprintFieldType.CONCEPT | ConceptStructureBlueprintFieldType.LIST:+ return None+++def _walkable_field_from_blueprint(+ *, name: str, field_blueprint: ConceptStructureBlueprint, domain_code: str, package_alias: str | None+) -> WalkableField:+ # The standard's rule: a declared field may hold nothing when it is not `required` and has no `default_value`. A+ # `required` field is generated as a value that cannot be `None`.+ may_hold_nothing = not field_blueprint.required and field_blueprint.default_value is None+ if field_blueprint.type is None:+ # A field declared by its `choices` alone holds one of them, a text.+ return WalkableField(name=name, value_kind=BindingValueKind.TEXT, may_hold_nothing=may_hold_nothing)+ match field_blueprint.type:+ case ConceptStructureBlueprintFieldType.CONCEPT:+ concept_ref = (+ qualify_concept_ref(concept_ref=field_blueprint.concept_ref, domain_code=domain_code, package_alias=package_alias)+ if field_blueprint.concept_ref+ else None+ )+ return WalkableField(name=name, value_kind=BindingValueKind.CONCEPT, concept_ref=concept_ref, may_hold_nothing=may_hold_nothing)+ case ConceptStructureBlueprintFieldType.LIST:+ return _walkable_list_field(+ name=name, field_blueprint=field_blueprint, domain_code=domain_code, package_alias=package_alias, may_hold_nothing=may_hold_nothing+ )+ case (+ ConceptStructureBlueprintFieldType.TEXT+ | ConceptStructureBlueprintFieldType.NUMBER+ | ConceptStructureBlueprintFieldType.INTEGER+ | ConceptStructureBlueprintFieldType.BOOLEAN+ | ConceptStructureBlueprintFieldType.DATE+ | ConceptStructureBlueprintFieldType.DATETIME+ | ConceptStructureBlueprintFieldType.TIME+ | ConceptStructureBlueprintFieldType.DICT+ ):+ value_kind = _scalar_value_kind(field_type=field_blueprint.type) or BindingValueKind.UNDERIVABLE+ return WalkableField(name=name, value_kind=value_kind, may_hold_nothing=may_hold_nothing)+++def _walkable_list_field(+ *, name: str, field_blueprint: ConceptStructureBlueprint, domain_code: str, package_alias: str | None, may_hold_nothing: bool+) -> WalkableField:+ item_type = field_blueprint.item_type+ if item_type is None:+ return WalkableField(+ name=name,+ value_kind=BindingValueKind.UNDERIVABLE,+ is_list=True,+ may_hold_nothing=may_hold_nothing,+ underivable_reason="the list declares no item_type",+ )+ if item_type == ConceptStructureBlueprintFieldType.CONCEPT:+ item_concept_ref = (+ qualify_concept_ref(concept_ref=field_blueprint.item_concept_ref, domain_code=domain_code, package_alias=package_alias)+ if field_blueprint.item_concept_ref+ else None+ )+ return WalkableField(+ name=name, value_kind=BindingValueKind.CONCEPT, concept_ref=item_concept_ref, is_list=True, may_hold_nothing=may_hold_nothing+ )+ item_field_type: ConceptStructureBlueprintFieldType | None+ try:+ item_field_type = ConceptStructureBlueprintFieldType(item_type)+ except ValueError:+ item_field_type = None+ item_value_kind = _scalar_value_kind(field_type=item_field_type) if item_field_type is not None else None+ if item_value_kind is None:+ return WalkableField(+ name=name,+ value_kind=BindingValueKind.UNDERIVABLE,+ is_list=True,+ may_hold_nothing=may_hold_nothing,+ underivable_reason=f"the list's item_type '{item_type}' derives no concept",+ )+ return WalkableField(name=name, value_kind=item_value_kind, is_list=True, may_hold_nothing=may_hold_nothing)+++def walkable_fields_from_structure(+ *, structure: Mapping[str, ConceptStructureBlueprintType], domain_code: str, package_alias: str | None = None+) -> tuple[WalkableField, ...]:+ """The fields of a declared structure, in declaration order, their concept refs qualified in `domain_code`.++ A dependency package's structure passes the package's alias, under which its own concept refs are keyed.+ """+ normalized_structure = normalize_structure_blueprint(dict(structure))+ return tuple(+ _walkable_field_from_blueprint(name=field_name, field_blueprint=field_blueprint, domain_code=domain_code, package_alias=package_alias)+ for field_name, field_blueprint in normalized_structure.items()+ )+++def _inherited_walkable_concept(*, concept_ref: str, refined: WalkableConcept) -> WalkableConcept:+ """A refining concept as the walk sees it: the structure it inherits, or the leaf or the void it refines."""+ match refined.shape:+ case ConceptShape.STRUCTURE:+ return WalkableConcept(concept_ref=concept_ref, shape=ConceptShape.STRUCTURE, fields=refined.fields)+ case ConceptShape.VALUE:+ return WalkableConcept(+ concept_ref=concept_ref,+ shape=ConceptShape.VALUE,+ shape_reason=f"refines '{refined.concept_ref}', which {refined.shape_reason}",+ )+ case ConceptShape.NO_STRUCTURE:+ return WalkableConcept(+ concept_ref=concept_ref,+ shape=ConceptShape.NO_STRUCTURE,+ shape_reason=f"refines '{refined.concept_ref}', which {refined.shape_reason}",+ )+++class LibraryConceptWalkResolver(ConceptWalkResolver):+ """The walk's view of the concepts of a loaded library.++ A concept ref the walk hands it is a key of the library: a host or native concept's ref, or a dependency's+ `alias->domain.Code`. The concepts a dependency's concept names, through its fields or its `refines`, are keyed under+ the same alias, so a walk that starts in a package stays in it.+ """++ def __init__(self, *, concept_library: ConceptLibraryAbstract):+ self._concept_library = concept_library++ @override+ def resolve_walkable_concept(self, *, concept_ref: str) -> WalkableConcept:+ return self._resolve(concept_ref=concept_ref, visited=frozenset())++ def _resolve(self, *, concept_ref: str, visited: frozenset[str]) -> WalkableConcept:+ if concept_ref in visited:+ return WalkableConcept(concept_ref=concept_ref, shape=ConceptShape.NO_STRUCTURE, shape_reason="refines itself through a cycle")+ try:+ concept = self._concept_library.get_required_concept(concept_ref=concept_ref)+ except ConceptLibraryError:+ return WalkableConcept(concept_ref=concept_ref, shape=ConceptShape.NO_STRUCTURE, shape_reason="is not a concept of the library")+ if SpecialDomain.is_native(domain_code=concept.domain_code) and concept.code in NativeConceptCode.values_list():+ # Every native, a native refining another included (Markdown refines Text), is read off its own pinned definition.+ return walkable_native_concept(native_code=NativeConceptCode(concept.code))+ if concept.is_described_only:+ # Whether written `Note = "A note"` or as a table holding a description alone, which the runtime makes refine+ # `native.Text`: the standard gives such a concept no structure, so a path never enters it.+ return WalkableConcept(concept_ref=concept_ref, shape=ConceptShape.NO_STRUCTURE, shape_reason=_DESCRIBED_ONLY_REASON)+ package_alias = _package_alias_of(concept_key=concept_ref)+ if concept.declared_structure is not None:+ return WalkableConcept(+ concept_ref=concept_ref,+ shape=ConceptShape.STRUCTURE,+ fields=walkable_fields_from_structure(+ structure=concept.declared_structure, domain_code=concept.domain_code, package_alias=package_alias+ ),+ )+ if concept.refines is not None:+ refined_ref = qualify_concept_ref(concept_ref=concept.refines, domain_code=concept.domain_code, package_alias=package_alias)+ refined = self._resolve(concept_ref=refined_ref, visited=visited | {concept_ref})+ return _inherited_walkable_concept(concept_ref=concept_ref, refined=refined)+ return self._walkable_from_structure_class(concept=concept, concept_key=concept_ref, package_alias=package_alias)++ def _walkable_from_structure_class(self, *, concept: Concept, concept_key: str, package_alias: str | None) -> WalkableConcept:+ """A concept whose structure exists only as a Python class, walked through the class's own fields."""+ try:+ structure_class = self._concept_library.get_structure_class(concept=concept)+ except ConceptStructureClassNotFoundError:+ return WalkableConcept(+ concept_ref=concept_key, shape=ConceptShape.NO_STRUCTURE, shape_reason="has a structure class that cannot be resolved"+ )+ walkable_fields = tuple(+ self._walkable_field_from_class_field(name=field_name, field_info=field_info, package_alias=package_alias)+ for field_name, field_info in structure_class.model_fields.items()+ if not field_name.startswith("_")+ )+ return WalkableConcept(concept_ref=concept_key, shape=ConceptShape.STRUCTURE, fields=walkable_fields)++ def _walkable_field_from_class_field(self, *, name: str, field_info: FieldInfo, package_alias: str | None) -> WalkableField:+ # A class states what a field may hold through its annotation: one that admits `None` may hold nothing, whether it+ # is required or has a default, and so does one left unrequired with no default.+ is_nullable = _admits_none(annotation=field_info.annotation)+ may_hold_nothing = is_nullable or (not field_info.is_required() and field_info.default is None and field_info.default_factory is None)+ annotation, _ = _strip_optional(annotation=field_info.annotation)+ is_list = get_origin(annotation) is list+ if is_list:+ list_args = get_args(annotation)+ if not list_args:+ return WalkableField(+ name=name,+ value_kind=BindingValueKind.UNDERIVABLE,+ is_list=True,+ may_hold_nothing=may_hold_nothing,+ underivable_reason="the list declares no item type",+ )+ annotation, _ = _strip_optional(annotation=list_args[0])+ value_kind = _python_value_kind(annotation=annotation)+ if value_kind is not None:+ return WalkableField(name=name, value_kind=value_kind, is_list=is_list, may_hold_nothing=may_hold_nothing)+ if isinstance(annotation, type) and issubclass(annotation, StuffContent):+ concept_keys = self._concept_keys_for_class(class_name=annotation.__name__, package_alias=package_alias)+ if len(concept_keys) == 1:+ return WalkableField(+ name=name, value_kind=BindingValueKind.CONCEPT, concept_ref=concept_keys[0], is_list=is_list, may_hold_nothing=may_hold_nothing+ )+ reason = (+ f"its class '{annotation.__name__}' is the structure of no concept"+ if not concept_keys+ else f"its class '{annotation.__name__}' is the structure of several concepts ({', '.join(concept_keys)})"+ )+ return WalkableField(+ name=name, value_kind=BindingValueKind.UNDERIVABLE, is_list=is_list, may_hold_nothing=may_hold_nothing, underivable_reason=reason+ )+ return WalkableField(+ name=name,+ value_kind=BindingValueKind.UNDERIVABLE,+ is_list=is_list,+ may_hold_nothing=may_hold_nothing,+ underivable_reason=f"its Python type '{annotation}' maps to no concept",+ )++ def _concept_keys_for_class(self, *, class_name: str, package_alias: str | None) -> list[str]:+ """The keys of the concepts whose structure is the class named `class_name`, those of the walking package first.++ When several concepts share the class, the ones the walking concept's package can name, natives included, are+ kept, so a dependency's class-backed concept maps to the package's own concept rather than to a host one.+ """+ concept_keys = sorted(+ {+ library_concept_key(concept_library=self._concept_library, concept=listed_concept)+ for listed_concept in self._concept_library.list_concepts()+ if listed_concept.structure_class_name == class_name+ }+ )+ if len(concept_keys) <= 1:+ return concept_keys+ return [concept_key for concept_key in concept_keys if _package_alias_of(concept_key=concept_key) == package_alias] or concept_keys+++def _admits_none(*, annotation: Any) -> bool:+ """Whether a field annotation admits `None`: `None` itself, `Any`, or a union with a `None` arm."""+ if annotation is None or annotation is type(None) or annotation is Any:+ return True+ origin = get_origin(annotation)+ if origin is Union or origin is types.UnionType:+ return any(_admits_none(annotation=arm) for arm in get_args(annotation))+ return False+++def _strip_optional(*, annotation: Any) -> tuple[Any, bool]:+ """An annotation without its `None` arm, and whether it had one."""+ origin = get_origin(annotation)+ if origin is Union or origin is types.UnionType:+ arms = [arm for arm in get_args(annotation) if arm is not type(None)]+ if len(arms) == 1:+ return arms[0], True+ return annotation, False+++def _python_value_kind(*, annotation: Any) -> BindingValueKind | None:+ """The value kind of a plain Python type on a structure class, or `None` when it is not a plain value."""+ if get_origin(annotation) is Literal:+ return BindingValueKind.TEXT+ if get_origin(annotation) is dict or annotation is dict:+ return BindingValueKind.JSON+ if not isinstance(annotation, type):+ return None+ # `bool` before `int`, which it subclasses, and `datetime` before `date`, likewise.+ if issubclass(annotation, bool):+ return BindingValueKind.YES_NO+ if issubclass(annotation, (int, float)):+ return BindingValueKind.NUMBER+ if issubclass(annotation, str):+ return BindingValueKind.TEXT+ if issubclass(annotation, datetime.datetime):+ return BindingValueKind.DATETIME+ if issubclass(annotation, datetime.date):+ return BindingValueKind.DATE+ if issubclass(annotation, datetime.time):+ return BindingValueKind.TIME+ return None+++class BlueprintConceptWalkResolver(ConceptWalkResolver):+ """The walk's view of one bundle's concept blueprints, keyed by domain-qualified ref, and of the natives.++ With `walks_single_field_natives`, a single-field native is walked through its pinned field, as a template reads it,+ rather than refused as a binding's leaf: the fix planner asks whether a template's read of a dotted path holds.+ """++ def __init__(self, *, concept_blueprints: Mapping[str, ConceptBlueprint | str], walks_single_field_natives: bool = False):+ self._concept_blueprints = concept_blueprints+ self._walks_single_field_natives = walks_single_field_natives++ @override+ def resolve_walkable_concept(self, *, concept_ref: str) -> WalkableConcept:+ return self._resolve(concept_ref=concept_ref, visited=frozenset())++ def _resolve(self, *, concept_ref: str, visited: frozenset[str]) -> WalkableConcept:+ if NativeConceptCode.is_native_concept_ref_or_code(concept_ref_or_code=concept_ref):+ return walkable_native_concept(+ native_code=NativeConceptCode(concept_ref.rsplit(".", maxsplit=1)[-1]),+ walks_single_field=self._walks_single_field_natives,+ )+ if concept_ref in visited:+ return WalkableConcept(concept_ref=concept_ref, shape=ConceptShape.NO_STRUCTURE, shape_reason="refines itself through a cycle")+ concept_blueprint = self._concept_blueprints.get(concept_ref)+ if concept_blueprint is None:+ return WalkableConcept(concept_ref=concept_ref, shape=ConceptShape.NO_STRUCTURE, shape_reason="is not declared in this bundle")+ if isinstance(concept_blueprint, str):+ return WalkableConcept(concept_ref=concept_ref, shape=ConceptShape.NO_STRUCTURE, shape_reason=_DESCRIBED_ONLY_REASON)+ domain_code = concept_ref.rsplit(".", maxsplit=1)[0]+ if isinstance(concept_blueprint.structure, dict):+ return WalkableConcept(+ concept_ref=concept_ref,+ shape=ConceptShape.STRUCTURE,+ fields=walkable_fields_from_structure(structure=concept_blueprint.structure, domain_code=domain_code),+ )+ if isinstance(concept_blueprint.structure, str):+ return WalkableConcept(+ concept_ref=concept_ref,+ shape=ConceptShape.NO_STRUCTURE,+ shape_reason="has a Python class for a structure, which a bundle does not show",+ )+ if concept_blueprint.refines is not None:+ refined_ref = qualify_concept_ref(concept_ref=concept_blueprint.refines, domain_code=domain_code)+ refined = self._resolve(concept_ref=refined_ref, visited=visited | {concept_ref})+ return _inherited_walkable_concept(concept_ref=concept_ref, refined=refined)+ return WalkableConcept(concept_ref=concept_ref, shape=ConceptShape.NO_STRUCTURE, shape_reason=_DESCRIBED_ONLY_REASON)diff --git a/pipelex/pipe_controllers/binding/binding_derivation.py b/pipelex/pipe_controllers/binding/binding_derivation.pynew file mode 100644index 000000000..2878a5f5d--- /dev/null+++ b/pipelex/pipe_controllers/binding/binding_derivation.py@@ -0,0 +1,352 @@+"""The derivation walk of a binding step: the concept its `from` path reaches, read off declared structures alone.++A binding step `{ from = "invoice.total", result = "total_amount" }` stores the value at a path under a new+name, and the concept of that value is derived before any run, by walking the path through the structures+the concepts declare, one segment at a time:++| The segment names a field declared as | The walk continues into, or the result is |+|---|---|+| `type = "concept"`, `concept_ref = X` | `X`, whose structure the next segment walks |+| `type = "list"`, `item_type = "concept"`, `item_concept_ref = X` | `X`, crossing a list |+| `type = "text"`, or `choices` | `native.Text`, a leaf |+| `type = "number"` or `"integer"` | `native.Number`, a leaf |+| `type = "boolean"` | `native.YesNo`, a leaf |+| `type = "date"` or `"datetime"` | `native.Date`, a leaf |+| `type = "time"` | `native.Time`, a leaf |+| `type = "list"` with a scalar `item_type` | that scalar's native concept, a leaf, crossing a list |+| `type = "dict"` | `native.JSON`, a leaf |++The walk is pure: it reads concepts through a `ConceptWalkResolver`, which says, for a concept ref, what+the walk may see of it (`WalkableConcept`). The resolver over a loaded library and the one over a bundle's+own blueprints live in `binding_concept_resolvers`; this module never maps a Python class back to a concept.+"""++from enum import StrEnum+from typing import Protocol++from pydantic import BaseModel, ConfigDict++from pipelex.core.concepts.native.concept_native import NativeConceptCode+from pipelex.core.pipes.variable_multiplicity import VariableMultiplicity, is_multiple_multiplicity+from pipelex.pipe_controllers.binding.exceptions import BindingPathUnresolvedError+from pipelex.tools.misc.string_utils import get_root_from_dotted_path+++class BindingValueKind(StrEnum):+ """What a field holds, as the walk and the runtime see it: a concept's content, a plain value, or nothing derivable."""++ CONCEPT = "concept"+ TEXT = "text"+ NUMBER = "number"+ YES_NO = "yes_no"+ DATE = "date"+ DATETIME = "datetime"+ TIME = "time"+ JSON = "json"+ # A field holding `native.Anything`, whose generated class types it `Any`: any raw value, stored as the native its+ # type maps to, as an `Anything` input is shaped. A leaf like a plain value, since `Anything` has no structure.+ ANYTHING = "anything"+ # A list with no `item_type`, or a field whose concept cannot be derived: no concept exists for its value.+ UNDERIVABLE = "underivable"++ @property+ def leaf_native_concept_code(self) -> NativeConceptCode | None:+ """The native concept a plain value of this kind is stored as, or `None` for a concept's content and for the underivable."""+ match self:+ case BindingValueKind.TEXT:+ return NativeConceptCode.TEXT+ case BindingValueKind.NUMBER:+ return NativeConceptCode.NUMBER+ case BindingValueKind.YES_NO:+ return NativeConceptCode.YES_NO+ case BindingValueKind.DATE | BindingValueKind.DATETIME:+ return NativeConceptCode.DATE+ case BindingValueKind.TIME:+ return NativeConceptCode.TIME+ case BindingValueKind.JSON:+ return NativeConceptCode.JSON+ case BindingValueKind.ANYTHING:+ return NativeConceptCode.ANYTHING+ case BindingValueKind.CONCEPT | BindingValueKind.UNDERIVABLE:+ return None++ @property+ def leaf_description(self) -> str:+ """How a message names a field of this kind, which is a leaf unless it holds a concept."""+ match self:+ case BindingValueKind.TEXT:+ return "a text field"+ case BindingValueKind.NUMBER:+ return "a number field"+ case BindingValueKind.YES_NO:+ return "a boolean field"+ case BindingValueKind.DATE:+ return "a date field"+ case BindingValueKind.DATETIME:+ return "a datetime field"+ case BindingValueKind.TIME:+ return "a time field"+ case BindingValueKind.JSON:+ return "a dict field"+ case BindingValueKind.ANYTHING:+ return "a field holding any value"+ case BindingValueKind.CONCEPT:+ return "a concept field"+ case BindingValueKind.UNDERIVABLE:+ return "a field with no derivable concept"+++class WalkableField(BaseModel):+ """One field of a walkable structure, as the walk reads it."""++ model_config = ConfigDict(frozen=True, extra="forbid")++ name: str+ value_kind: BindingValueKind+ # The concept the field (or each item of the list) holds, domain-qualified, when `value_kind` is CONCEPT.+ concept_ref: str | None = None+ is_list: bool = False+ # Whether the field may hold nothing: it is neither `required` nor given a `default_value`.+ may_hold_nothing: bool = False+ # Why no concept can be derived for the field's value, when `value_kind` is UNDERIVABLE.+ underivable_reason: str | None = None+++class ConceptShape(StrEnum):+ """What a concept offers a path walking into it."""++ # Declared fields, which the next segment names.+ STRUCTURE = "structure"+ # A native holding its value in a single field (`Text`, `Number`, `Time`, `JSON`, `Markdown`), or a+ # concept refining one: a leaf wherever the walk reaches it, which a path may end on but never enter.+ VALUE = "value"+ # No structure to walk: a structureless native (`Dynamic`, `Anything`, `Composite`), a concept declared+ # with neither `structure` nor `refines`, a concept refining one of those, or a concept nothing resolves.+ NO_STRUCTURE = "no_structure"+++class WalkableConcept(BaseModel):+ """A concept as the walk sees it: its ref, its shape, and its fields when it has a structure."""++ model_config = ConfigDict(frozen=True, extra="forbid")++ concept_ref: str+ shape: ConceptShape+ fields: tuple[WalkableField, ...] = ()+ # Why the concept offers nothing to walk, for a VALUE or NO_STRUCTURE shape, written to follow "which".+ shape_reason: str | None = None++ def get_field(self, *, name: str) -> WalkableField | None:+ for walkable_field in self.fields:+ if walkable_field.name == name:+ return walkable_field+ return None++ @property+ def field_names(self) -> list[str]:+ return [walkable_field.name for walkable_field in self.fields]+++class ConceptWalkResolver(Protocol):+ """What the walk needs to know of a concept, given its domain-qualified ref."""++ def resolve_walkable_concept(self, *, concept_ref: str) -> WalkableConcept: ...+++class BindingSegment(BaseModel):+ """One field segment of a derived path, as the runtime walks it."""++ model_config = ConfigDict(frozen=True, extra="forbid")++ name: str+ # Whether the field holds a list, whose items the rest of the path is applied to.+ crosses_list: bool+++class BindingRoot(BaseModel):+ """The concept and multiplicity of a binding's root, as the sequence knows them at the binding step."""++ model_config = ConfigDict(frozen=True, extra="forbid")++ concept_ref: str+ multiplicity: VariableMultiplicity | None = None+++class BindingDerivation(BaseModel):+ """What a binding step binds, derived before any run: the concept, the multiplicity and the static absence."""++ model_config = ConfigDict(frozen=True, extra="forbid")++ path: str+ concept_ref: str+ # `None` for a single value, `True` for a list. A bare name keeps its root's multiplicity unchanged.+ multiplicity: VariableMultiplicity | None+ # Static, single results only: the path walks a field that may hold nothing. A list result never is.+ may_find_nothing: bool+ # The dotted path up to the first field along the way that may hold nothing, for the taint message.+ first_optional_path: str | None = None+ # How the runtime stores the value it reaches: a concept's content is copied whole, a plain value wrapped.+ leaf_kind: BindingValueKind+ segments: tuple[BindingSegment, ...] = ()++ @property+ def root_name(self) -> str:+ return get_root_from_dotted_path(self.path)++ @property+ def is_bare_name(self) -> bool:+ return not self.segments++ @property+ def is_plural(self) -> bool:+ return is_multiple_multiplicity(multiplicity=self.multiplicity)+++def _quoted(*, names: list[str]) -> str:+ return ", ".join(f"'{name}'" for name in names)+++def _fields_phrase(*, walkable_concept: WalkableConcept) -> str:+ if not walkable_concept.fields:+ return "It has no fields."+ return f"Its fields are: {_quoted(names=walkable_concept.field_names)}."+++def derive_binding(*, path: str, root: BindingRoot, resolver: ConceptWalkResolver) -> BindingDerivation:+ """Walk a binding step's `from` path through declared structures and derive what the step binds.++ Args:+ path: The binding step's `from`, already known to follow the path grammar.+ root: The root's concept and multiplicity, from the latest value stored under its name before the step.+ resolver: How the walk reads a concept.++ Returns:+ The derived concept, multiplicity and static absence, with the segments the runtime walks.++ Raises:+ BindingPathUnresolvedError: When the path cannot be walked, naming the segment that failed and the+ fields available there.+ """+ segment_names = path.split(".")[1:]+ if not segment_names:+ # A bare name binds a renamed copy of the whole value, with the root's concept and multiplicity unchanged.+ return BindingDerivation(+ path=path,+ concept_ref=root.concept_ref,+ multiplicity=root.multiplicity,+ may_find_nothing=False,+ leaf_kind=BindingValueKind.CONCEPT,+ )++ crosses_any_list = is_multiple_multiplicity(multiplicity=root.multiplicity)+ may_find_nothing = False+ first_optional_path: str | None = None+ segments: list[BindingSegment] = []+ reached_path = get_root_from_dotted_path(path)+ # What the next segment walks: a concept, or the plain field the previous segment ended on.+ current_concept: WalkableConcept | None = resolver.resolve_walkable_concept(concept_ref=root.concept_ref)+ current_leaf_field: WalkableField | None = None+ derived_concept_ref = root.concept_ref+ leaf_kind = BindingValueKind.CONCEPT++ for segment_index, segment_name in enumerate(segment_names):+ if current_leaf_field is not None:+ msg = (+ f"Cannot bind '{path}': '{reached_path}' is {current_leaf_field.value_kind.leaf_description}, a leaf holding a plain value "+ f"with no fields, so the segment '{segment_name}' cannot follow it. It has no fields."+ )+ raise BindingPathUnresolvedError(msg, path=path, failed_segment=segment_name, available_fields=[])+ if current_concept is None:+ msg = f"Cannot bind '{path}': nothing is known of '{reached_path}', so the segment '{segment_name}' cannot be walked."+ raise BindingPathUnresolvedError(msg, path=path, failed_segment=segment_name, available_fields=[])+ match current_concept.shape:+ case ConceptShape.VALUE:+ msg = (+ f"Cannot bind '{path}': '{reached_path}' holds a '{current_concept.concept_ref}', which {current_concept.shape_reason}, "+ f"so it is a leaf and the segment '{segment_name}' cannot follow it. It has no fields to walk."+ )+ raise BindingPathUnresolvedError(msg, path=path, failed_segment=segment_name, available_fields=[])+ case ConceptShape.NO_STRUCTURE:+ msg = (+ f"Cannot bind '{path}': '{reached_path}' holds a '{current_concept.concept_ref}', which {current_concept.shape_reason}, "+ f"so the segment '{segment_name}' has no structure to walk. It has no fields."+ )+ raise BindingPathUnresolvedError(msg, path=path, failed_segment=segment_name, available_fields=[])+ case ConceptShape.STRUCTURE:+ pass++ walkable_field = current_concept.get_field(name=segment_name)+ if walkable_field is None:+ msg = (+ f"Cannot bind '{path}': '{reached_path}' holds a '{current_concept.concept_ref}', which has no field '{segment_name}'. "+ f"{_fields_phrase(walkable_concept=current_concept)}"+ )+ raise BindingPathUnresolvedError(msg, path=path, failed_segment=segment_name, available_fields=current_concept.field_names)++ reached_path = f"{reached_path}.{segment_name}"+ is_last_segment = segment_index == len(segment_names) - 1+ if walkable_field.may_hold_nothing and not may_find_nothing:+ may_find_nothing = True+ first_optional_path = reached_path+ if walkable_field.is_list:+ crosses_any_list = True+ segments.append(BindingSegment(name=segment_name, crosses_list=walkable_field.is_list))++ match walkable_field.value_kind:+ case BindingValueKind.UNDERIVABLE:+ reason = walkable_field.underivable_reason or "no concept can be derived for its value"+ if is_last_segment:+ msg = (+ f"Cannot bind '{path}': the path ends on the segment '{segment_name}', at '{reached_path}', where {reason}, "+ "so no concept can be derived for the result."+ )+ raise BindingPathUnresolvedError(msg, path=path, failed_segment=segment_name, available_fields=[])+ next_segment = segment_names[segment_index + 1]+ msg = (+ f"Cannot bind '{path}': '{reached_path}' has no declared fields, since {reason}, "+ f"so the segment '{next_segment}' cannot follow it."+ )+ raise BindingPathUnresolvedError(msg, path=path, failed_segment=next_segment, available_fields=[])+ case BindingValueKind.CONCEPT:+ if walkable_field.concept_ref is None:+ msg = f"Cannot bind '{path}': the concept field '{reached_path}' names no concept."+ raise BindingPathUnresolvedError(msg, path=path, failed_segment=segment_name, available_fields=[])+ derived_concept_ref = walkable_field.concept_ref+ # A field holding `native.Anything` holds a raw value of any type, generated as `Any`, never a content.+ holds_anything = walkable_field.concept_ref == NativeConceptCode.ANYTHING.concept_ref+ leaf_kind = BindingValueKind.ANYTHING if holds_anything else BindingValueKind.CONCEPT+ current_leaf_field = None+ if is_last_segment:+ current_concept = None+ else:+ current_concept = resolver.resolve_walkable_concept(concept_ref=walkable_field.concept_ref)+ case (+ BindingValueKind.TEXT+ | BindingValueKind.NUMBER+ | BindingValueKind.YES_NO+ | BindingValueKind.DATE+ | BindingValueKind.DATETIME+ | BindingValueKind.TIME+ | BindingValueKind.JSON+ | BindingValueKind.ANYTHING+ ):+ native_code = walkable_field.value_kind.leaf_native_concept_code+ if native_code is None:+ msg = f"Cannot bind '{path}': '{reached_path}' holds a plain value with no native concept."+ raise BindingPathUnresolvedError(msg, path=path, failed_segment=segment_name, available_fields=[])+ derived_concept_ref = native_code.concept_ref+ leaf_kind = walkable_field.value_kind+ current_leaf_field = walkable_field+ current_concept = None++ multiplicity: VariableMultiplicity | None = True if crosses_any_list else None+ return BindingDerivation(+ path=path,+ concept_ref=derived_concept_ref,+ multiplicity=multiplicity,+ # A list result is never absent: wherever a segment holds nothing, that part of the path contributes no items.+ may_find_nothing=may_find_nothing and not crosses_any_list,+ first_optional_path=first_optional_path if not crosses_any_list else None,+ leaf_kind=leaf_kind,+ segments=tuple(segments),+ )diff --git a/pipelex/pipe_controllers/binding/binding_step.py b/pipelex/pipe_controllers/binding/binding_step.pynew file mode 100644index 000000000..6f07e1ff2--- /dev/null+++ b/pipelex/pipe_controllers/binding/binding_step.py@@ -0,0 +1,270 @@+"""The runtime binding step: binds the value at a path in working memory to a new name, inside a PipeSequence.++A binding step is not a pipe. The sequence holding it derives what it binds from its typed flow (the root's+concept walked through the declared structures, `binding_derivation`) and hands the derivation to `bind`,+which reads the root in working memory and stores the result under its own name:++- a root holding a recorded absence lifts the step, as an absent plain input lifts a pipe: a single result+ is recorded as a `SKIPPED` absence chained to the root's record, and a list result is an empty list;+- a single result whose path reaches nothing is recorded as a `DECLARED_ABSENT` absence naming the segment+ that held nothing;+- anything else is a deep copy of the value at the path, stored as a fresh stuff, of the derived concept,+ with a fresh stuff code, for which the binding is the producer in the execution graph.+"""++from datetime import UTC, datetime+from typing import Any++from pydantic import BaseModel, ConfigDict++from pipelex import log+from pipelex.core.concepts.concept import Concept+from pipelex.core.memory.absence import AbsenceKind, AbsenceRecord+from pipelex.core.memory.working_memory import WorkingMemory+from pipelex.core.pipes.inputs.exceptions import PipeRunInputsError+from pipelex.core.stuffs.list_content import ListContent+from pipelex.core.stuffs.stuff import Stuff+from pipelex.core.stuffs.stuff_content import StuffContent+from pipelex.core.stuffs.stuff_factory import StuffFactory+from pipelex.graph.graph_tracer_manager import GraphTracerManager+from pipelex.graph.graphspec import NodeKind+from pipelex.graph.stuff_io_spec import make_stuff_io_spec+from pipelex.pipe_controllers.binding.binding_derivation import BindingDerivation+from pipelex.pipe_controllers.binding.binding_value import FoundNothing, bind_content+from pipelex.system.job_metadata import JobMetadata+from pipelex.system.pipe_run_mode import PipeRunMode+from pipelex.tools.misc.string_utils import get_root_from_dotted_path++# The pipe type a binding node carries in the execution graph, beside its `binding` node kind.+BINDING_NODE_PIPE_TYPE = "BindingStep"+++class BindingOutcome(BaseModel):+ """What one binding step stored: the bound stuff, or the absence it recorded."""++ model_config = ConfigDict(frozen=True)++ stuff: Stuff | None = None+ absence: AbsenceRecord | None = None++ @property+ def is_skipped(self) -> bool:+ return self.absence is not None and self.absence.kind.is_skipped+++class BindingStep(BaseModel):+ """A PipeSequence step binding the value at `from_path` to `output_name`, as the runtime holds it."""++ model_config = ConfigDict(frozen=True, extra="forbid")++ from_path: str+ output_name: str++ @property+ def root_name(self) -> str:+ return get_root_from_dotted_path(self.from_path)++ @property+ def as_written(self) -> str:+ """The step as MTHDS writes it, for messages."""+ return f'{{ from = "{self.from_path}", result = "{self.output_name}" }}'++ def bind(+ self,+ *,+ working_memory: WorkingMemory,+ derivation: BindingDerivation,+ result_concept: Concept,+ calling_pipe_code: str,+ run_mode: PipeRunMode,+ stuff_code: str | None = None,+ ) -> BindingOutcome:+ """Bind the value at the path into `working_memory`, as the main stuff, or record why nothing was bound.++ Args:+ working_memory: The sequence's working memory, written in place.+ derivation: What the step binds, derived from the sequence's typed flow.+ result_concept: The derived concept, resolved in the library.+ calling_pipe_code: The sequence holding the step, for messages and provenance.+ run_mode: The run mode, for the error raised on a missing root.+ stuff_code: The stuff code to give the result, when the run fixes it; a fresh one otherwise.++ Raises:+ PipeRunInputsError: When the root holds neither a value nor a recorded absence.+ """+ root_stuff = working_memory.get_optional_stuff(self.root_name)+ if root_stuff is None:+ return self._lift(+ working_memory=working_memory,+ derivation=derivation,+ result_concept=result_concept,+ root_absence=self._required_root_absence(working_memory=working_memory, calling_pipe_code=calling_pipe_code, run_mode=run_mode),+ stuff_code=stuff_code,+ )++ bound_content = bind_content(root_content=root_stuff.content, derivation=derivation)+ if isinstance(bound_content, FoundNothing):+ record = AbsenceRecord(+ variable_name=self.output_name,+ kind=AbsenceKind.DECLARED_ABSENT,+ reason=(+ f"the binding step {self.as_written} of pipe '{calling_pipe_code}' bound nothing, "+ f"because '{bound_content.empty_path}' holds nothing"+ ),+ )+ working_memory.record_new_main_absence(record)+ log.verbose(f"Binding '{self.from_path}' found nothing at '{bound_content.empty_path}': '{self.output_name}' is recorded absent")+ return BindingOutcome(absence=record)++ # A bare name binds a renamed copy of the whole value, which keeps the concept the value has.+ concept = root_stuff.concept if derivation.is_bare_name else result_concept+ bound_stuff = StuffFactory.make_stuff(concept=concept, content=bound_content, name=self.output_name, code=stuff_code)+ working_memory.set_new_main_stuff(bound_stuff, name=self.output_name)+ log.verbose(f"Bound '{self.from_path}' to '{self.output_name}': {concept.concept_ref}")+ return BindingOutcome(stuff=bound_stuff)++ def skip_untyped_root(self, *, working_memory: WorkingMemory, calling_pipe_code: str, run_mode: PipeRunMode) -> BindingOutcome:+ """Skip the binding because its root is absent, when nothing tells what the root would have held.++ That is a root the sequence's flow cannot type, one a pipe that did not resolve at validation stored.+ Whether the path crosses a list is unknown without the root's concept, so the result is recorded as a single+ skipped absence, chained to the root's record, as `_lift` records a single result.++ Raises:+ PipeRunInputsError: When the root holds neither a value nor a recorded absence.+ """+ root_absence = self._required_root_absence(working_memory=working_memory, calling_pipe_code=calling_pipe_code, run_mode=run_mode)+ record = self._skipped_record(root_absence=root_absence)+ working_memory.record_new_main_absence(record)+ return BindingOutcome(absence=record)++ def _required_root_absence(self, *, working_memory: WorkingMemory, calling_pipe_code: str, run_mode: PipeRunMode) -> AbsenceRecord:+ """The absence recorded for a root that holds no value.++ Raises:+ PipeRunInputsError: When the root has no recorded absence either.+ """+ root_absence = working_memory.get_optional_absence(self.root_name)+ if root_absence is None:+ msg = (+ f"The binding step {self.as_written} of pipe '{calling_pipe_code}' reads '{self.root_name}', which is not in working "+ f"memory and has no recorded absence. Valid keys are: {working_memory.list_keys()}"+ )+ raise PipeRunInputsError(+ message=msg,+ run_mode=run_mode,+ pipe_code=calling_pipe_code,+ variable_name=self.root_name,+ concept_code=None,+ )+ return root_absence++ def _skipped_record(self, *, root_absence: AbsenceRecord) -> AbsenceRecord:+ return AbsenceRecord(+ variable_name=self.output_name,+ kind=AbsenceKind.SKIPPED,+ reason=f"skipped because input '{self.root_name}' is absent",+ upstream=root_absence,+ )++ def _lift(+ self,+ *,+ working_memory: WorkingMemory,+ derivation: BindingDerivation,+ result_concept: Concept,+ root_absence: AbsenceRecord,+ stuff_code: str | None,+ ) -> BindingOutcome:+ """Skip the binding because its root is absent, as an absent plain input lifts a pipe.++ A single result is recorded as a skipped absence chained to the root's record; a list result is an empty+ list, since a plural slot is never absent, with the skip kept as a note for observability. The empty list+ takes the stuff code the run fixes, when it fixes one, as a bound value would.+ """+ record = self._skipped_record(root_absence=root_absence)+ if derivation.is_plural:+ empty_list_stuff = StuffFactory.make_stuff(+ concept=result_concept,+ content=ListContent[StuffContent](items=[]),+ name=self.output_name,+ code=stuff_code,+ )+ working_memory.set_new_main_stuff(empty_list_stuff, name=self.output_name)+ working_memory.record_absence(record)+ return BindingOutcome(stuff=empty_list_stuff, absence=record)+ working_memory.record_new_main_absence(record)+ return BindingOutcome(absence=record)++ def trace_start(self, *, job_metadata: JobMetadata, working_memory: WorkingMemory, domain_code: str) -> str | None:+ """Open the binding's node in the execution graph, its input being the root's stuff, and return its id."""+ trace_context = job_metadata.trace_context+ if trace_context is None or not trace_context.emit_graph_events:+ return None+ tracer_manager = GraphTracerManager.get_instance()+ if tracer_manager is None:+ return None+ root_stuff = working_memory.get_optional_stuff(self.root_name)+ input_specs = (+ [make_stuff_io_spec(name=self.root_name, stuff=root_stuff, include_data=trace_context.data_inclusion.stuff_json_content)]+ if root_stuff is not None+ else None+ )+ node_id, _ = tracer_manager.on_pipe_start(+ trace_context=trace_context,+ pipe_code=self.from_path,+ pipe_type=BINDING_NODE_PIPE_TYPE,+ node_kind=NodeKind.BINDING,+ started_at=datetime.now(UTC),+ input_specs=input_specs,+ description=f"Binds '{self.from_path}' to '{self.output_name}'",+ domain_code=domain_code,+ )+ execution_data: dict[str, Any] = {"from": self.from_path, "result": self.output_name}+ tracer_manager.register_execution_data(lookup_key=trace_context.lookup_key, node_id=node_id, execution_data=execution_data)+ return node_id++ def trace_end(self, *, job_metadata: JobMetadata, node_id: str | None, outcome: BindingOutcome) -> None:+ """Close the binding's node, registering it as the producer of the stuff it bound."""+ trace_context = job_metadata.trace_context+ if trace_context is None or node_id is None:+ return+ tracer_manager = GraphTracerManager.get_instance()+ if tracer_manager is None:+ return+ output_spec = (+ make_stuff_io_spec(name=self.output_name, stuff=outcome.stuff, include_data=trace_context.data_inclusion.stuff_json_content)+ if outcome.stuff is not None+ else None+ )+ if outcome.is_skipped and outcome.absence is not None:+ tracer_manager.on_pipe_end_skipped(+ lookup_key=trace_context.lookup_key,+ node_id=node_id,+ ended_at=datetime.now(UTC),+ skip_reason=outcome.absence.reason,+ output_spec=output_spec,+ )+ return+ tracer_manager.on_pipe_end_success(+ lookup_key=trace_context.lookup_key,+ node_id=node_id,+ ended_at=datetime.now(UTC),+ output_spec=output_spec,+ )++ def trace_error(self, *, job_metadata: JobMetadata, node_id: str | None, error: BaseException | None) -> None:+ """Close the binding's node as failed, with the error that stopped it, whatever its class."""+ trace_context = job_metadata.trace_context+ if trace_context is None or node_id is None:+ return+ tracer_manager = GraphTracerManager.get_instance()+ if tracer_manager is None:+ return+ tracer_manager.on_pipe_end_error(+ lookup_key=trace_context.lookup_key,+ node_id=node_id,+ ended_at=datetime.now(UTC),+ error_type=type(error).__name__ if error is not None else "UnknownError",+ error_message=str(error) if error is not None else "the binding step stopped without binding a value",+ )diff --git a/pipelex/pipe_controllers/binding/binding_step_blueprint.py b/pipelex/pipe_controllers/binding/binding_step_blueprint.pynew file mode 100644index 000000000..328d6441a--- /dev/null+++ b/pipelex/pipe_controllers/binding/binding_step_blueprint.py@@ -0,0 +1,125 @@+from collections.abc import Mapping+from typing import Any, cast++from pydantic import BaseModel, ConfigDict, Field, SerializerFunctionWrapHandler, field_validator, model_serializer++from pipelex.core.pipes.exceptions import PipeValidationError+from pipelex.pipe_machinery.validation import BINDING_PATH_PATTERN, BINDING_RESULT_PATTERN+from pipelex.tools.misc.string_utils import FIELD_PATH_SEGMENT_REGEX, SNAKE_CASE_IDENTIFIER_REGEX, is_field_path, is_snake_case+from pipelex.validation_error_types import PipeValidationErrorType++# The key a binding step is recognized by, as MTHDS writes it. `from` is a Python keyword, so the blueprint field is+# `from_path`, as PipeCompose's construct spells it, but that name is Python's only: MTHDS spells the key `from`, and a+# step spelling it `from_path` is refused, as the schema refuses it, while every dump writes `from`.+BINDING_FROM_KEY = "from"++# The fields of a pipe step that a binding step never carries, `result` aside, which both shapes share.+PIPE_STEP_ONLY_FIELDS: tuple[str, ...] = ("pipe", "nb_output", "multiple_output", "batch_over", "batch_as")++_PATH_GRAMMAR = (+ "a name in working memory followed by zero or more field names, separated by single dots, each a letter followed by letters, "+ f"digits and underscores (`{FIELD_PATH_SEGMENT_REGEX}`), with no subscript, expression, whitespace or underscore-led segment"+)+_RESULT_GRAMMAR = f"a plain input name, a snake_case identifier matching `{SNAKE_CASE_IDENTIFIER_REGEX}`"+++def is_binding_step_dict(*, raw_step: Mapping[str, Any]) -> bool:+ """Whether a step, as written, is a binding step: it carries `from`."""+ return BINDING_FROM_KEY in raw_step+++def check_binding_step_shape(*, raw_step: Mapping[str, Any], step_label: str) -> None:+ """Refuse a step whose shape breaks the binding-step rules, before it is parsed into either shape.++ A step carries exactly one of `pipe` and `from`; a binding step carries `result` and none of a pipe step's+ other fields. A step carrying neither `pipe` nor `from` is left to the pipe-step shape, which refuses it+ with no error name of its own, as the standard says.++ Raises:+ PipeValidationError: ``BINDING_STEP_INVALID`` naming the fault.+ """+ if not is_binding_step_dict(raw_step=raw_step):+ return+ from_path = raw_step.get(BINDING_FROM_KEY)+ if "pipe" in raw_step:+ msg = (+ f"{step_label} carries both `pipe` and `from`: a step either runs a pipe or binds a value, never both. "+ f'Split it into a binding step `{{ from = "{from_path}", result = "<name>" }}` and a pipe step reading that name.'+ )+ raise PipeValidationError(message=msg, error_type=PipeValidationErrorType.BINDING_STEP_INVALID)+ pipe_step_fields = [field_name for field_name in PIPE_STEP_ONLY_FIELDS if field_name in raw_step]+ if pipe_step_fields:+ quoted_fields = ", ".join(f"`{field_name}`" for field_name in pipe_step_fields)+ msg = (+ f"{step_label} is a binding step (it carries `from`), which carries only `from` and `result`, so it cannot carry {quoted_fields}. "+ "To batch over a bound list, bind it first, then batch over the bound name in the next pipe step."+ )+ raise PipeValidationError(message=msg, error_type=PipeValidationErrorType.BINDING_STEP_INVALID)+ if "result" not in raw_step:+ msg = f"{step_label} is a binding step without `result`: a binding step names the value it binds, so `result` is required."+ raise PipeValidationError(message=msg, error_type=PipeValidationErrorType.BINDING_STEP_INVALID)+++class BindingStepBlueprint(BaseModel):+ """A PipeSequence step that binds the value at a path in working memory to a new name.++ `{ from = "invoice.total", result = "total_amount" }` stores a deep copy of the value at `invoice.total`+ under `total_amount`, as the concept the path derives through the declared structures. A binding step+ lives in a PipeSequence's `steps` only, and carries exactly these two fields.+ """++ # No `populate_by_name`: the path is read under `from` alone, so a `from_path` key is refused as an extra field.+ model_config = ConfigDict(extra="forbid")++ from_path: str = Field(+ validation_alias=BINDING_FROM_KEY,+ serialization_alias=BINDING_FROM_KEY,+ description="The path to bind: a name in working memory followed by zero or more field names, separated by dots.",+ json_schema_extra={"pattern": BINDING_PATH_PATTERN},+ )+ result: str = Field(+ description="The name under which the bound value is stored in working memory: a plain input name.",+ json_schema_extra={"pattern": BINDING_RESULT_PATTERN},+ )++ @field_validator("from_path", mode="after")+ @classmethod+ def validate_from_path(cls, from_path: str) -> str:+ if not is_field_path(path=from_path):+ msg = f"The binding step's `from` '{from_path}' is not a path: a path is {_PATH_GRAMMAR}."+ raise PipeValidationError(message=msg, error_type=PipeValidationErrorType.BINDING_STEP_INVALID, variable_names=[from_path])+ return from_path++ @field_validator("result", mode="after")+ @classmethod+ def validate_result(cls, result: str) -> str:+ if not is_snake_case(result):+ msg = (+ f"The binding step's `result` '{result}' is not {_RESULT_GRAMMAR}: a bound value is read by a later step's input, under a plain name."+ )+ raise PipeValidationError(message=msg, error_type=PipeValidationErrorType.BINDING_STEP_INVALID, variable_names=[result])+ return result++ @model_serializer(mode="wrap")+ def serialize_under_from(self, handler: SerializerFunctionWrapHandler):+ """Always write the path under `from`, the one key the parser reads back, whatever the caller asks.++ A dump without `by_alias` would otherwise write `from_path`, which a crate would carry and hash, and which+ a crate validated again refuses as an extra field. PipeCompose's construct writes `from` the same way.++ The return is deliberately unannotated — see `ConceptBlueprint.serialize_without_absent_hints`:+ an annotation here becomes the model's serialization JSON Schema and erases its shape.+ """+ dumped: dict[str, Any] = handler(self)+ return {(BINDING_FROM_KEY if key == "from_path" else key): value for key, value in dumped.items()}++ @property+ def root_name(self) -> str:+ return self.from_path.split(".", maxsplit=1)[0]+++def raw_step_mapping(*, raw_step: Any) -> Mapping[str, Any] | None:+ """A step as written, when it is a table; `None` for an already-parsed blueprint or anything else."""+ if isinstance(raw_step, Mapping):+ return cast("Mapping[str, Any]", raw_step)+ return Nonediff --git a/pipelex/pipe_controllers/binding/binding_value.py b/pipelex/pipe_controllers/binding/binding_value.pynew file mode 100644index 000000000..5951658a7--- /dev/null+++ b/pipelex/pipe_controllers/binding/binding_value.py@@ -0,0 +1,195 @@+"""The runtime half of a binding step: the value at a derived path, copied whole, or the segment that held nothing.++The walk follows the segments its derivation recorded, attribute by attribute, through the content of the+root's stuff. It is null-aware and list-aware, as the standard says:++- across a list, whether the root holds one or a field does, the rest of the path is applied to every item,+ and every list crossed is flattened into one list, items holding nothing being dropped;+- on a single value, a segment holding nothing ends the walk, and the binding records an absence naming it;+- a root whose shape contradicts the derived multiplicity is a run error: a single result never chooses one item of a list.++The value reached is a deep copy, never an alias of the root: a concept's content is copied whole, every field+included, and a plain value is stored as the native concept its field derives (`str` as `TextContent`, a number+as `NumberContent`, and so on), since the structure classes hold raw Python scalars. A field holding+`native.Anything` holds a raw value of any type, stored as the native its type maps to, as an `Anything` input is.+"""++import copy+import datetime+import math+from typing import Any, NamedTuple, cast++from pipelex.core.stuffs.date_content import DateContent+from pipelex.core.stuffs.json_content import JSONContent+from pipelex.core.stuffs.list_content import ListContent+from pipelex.core.stuffs.number_content import NumberContent+from pipelex.core.stuffs.stuff_content import StuffContent+from pipelex.core.stuffs.text_content import TextContent+from pipelex.core.stuffs.time_content import TimeContent+from pipelex.core.stuffs.yes_no_content import YesNoContent+from pipelex.pipe_controllers.binding.binding_derivation import BindingDerivation, BindingValueKind+from pipelex.pipe_controllers.binding.exceptions import BindingStepRunError+++class FoundNothing(NamedTuple):+ """A single-value binding whose path reached nothing, and the dotted path of the segment that held nothing."""++ empty_path: str+++def bind_content(*, root_content: StuffContent, derivation: BindingDerivation) -> StuffContent | FoundNothing:+ """The value a binding binds from its root's content: a fresh deep copy, or the segment that held nothing.++ Args:+ root_content: The content of the stuff stored under the binding's root name.+ derivation: What the binding binds, derived before the run.++ Returns:+ The bound content (a `ListContent` for a plural result, possibly empty), or `FoundNothing` for a single+ result whose path reached nothing.++ Raises:+ BindingStepRunError: When the value contradicts its declared structure, or its shape the derived multiplicity.+ """+ _refuse_shape_mismatch(root_content=root_content, derivation=derivation)+ if derivation.is_bare_name:+ return copy.deepcopy(root_content)++ values: list[Any]+ if isinstance(root_content, ListContent):+ values = list(cast("ListContent[StuffContent]", root_content).items)+ else:+ values = [root_content]+ reached_path = derivation.root_name+ empty_path: str | None = None+ for segment in derivation.segments:+ reached_path = f"{reached_path}.{segment.name}"+ next_values: list[Any] = []+ for value in values:+ if value is None:+ continue+ if not hasattr(value, segment.name):+ msg = (+ f"Binding '{derivation.path}' cannot read '{segment.name}' at '{reached_path}': the value there, a "+ f"'{type(value).__name__}', has no such field, although the declared structure has one."+ )+ raise BindingStepRunError(msg)+ field_value = getattr(value, segment.name)+ if field_value is None:+ if empty_path is None:+ empty_path = reached_path+ continue+ if segment.crosses_list:+ next_values.extend(_list_items(value=field_value, reached_path=reached_path, path=derivation.path))+ else:+ next_values.append(field_value)+ values = next_values++ if derivation.is_plural:+ stored_items = [_store_value(value=value, leaf_kind=derivation.leaf_kind, path=derivation.path) for value in values if value is not None]+ return ListContent[StuffContent](items=stored_items)+ # A single result walks one value at most: its root holds a single value (`_refuse_shape_mismatch`) and its path crosses no list.+ if not values or values[0] is None:+ return FoundNothing(empty_path=empty_path or reached_path)+ return _store_value(value=values[0], leaf_kind=derivation.leaf_kind, path=derivation.path)+++def _refuse_shape_mismatch(*, root_content: StuffContent, derivation: BindingDerivation) -> None:+ """Refuse a root whose shape contradicts the multiplicity derived for the binding, rather than bind a value of the wrong shape.++ A single result over a list would have to choose one item and drop the others, which a binding never does. A bare name+ binds its root's value as it is, so a root derived as a list must hold one. A path whose derivation is a list maps a+ single root as a list of one, so that shape binds as derived.+ """+ if isinstance(root_content, ListContent):+ if derivation.is_plural:+ return+ item_count = len(cast("ListContent[StuffContent]", root_content).items)+ msg = (+ f"Binding '{derivation.path}' was derived as a single value, but its root '{derivation.root_name}' holds a list of "+ f"{item_count} items, and a binding never chooses one item of a list. The root was typed before the run as a single value, "+ "so whatever stored it stored a list instead, such as a step whose caller asked for several outputs."+ )+ raise BindingStepRunError(msg)+ if derivation.is_bare_name and derivation.is_plural:+ msg = (+ f"Binding '{derivation.path}' was derived as a list, but its root '{derivation.root_name}' holds a single value, "+ "and a bare name binds its root's value as it is. The root was typed before the run as a list, so whatever stored it "+ "stored a single value instead."+ )+ raise BindingStepRunError(msg)+++def _list_items(*, value: Any, reached_path: str, path: str) -> list[Any]:+ if isinstance(value, ListContent):+ return list(cast("ListContent[StuffContent]", value).items)+ if isinstance(value, list):+ return list(cast("list[Any]", value))+ msg = f"Binding '{path}' expected a list at '{reached_path}', as its declared structure says, but found a '{type(value).__name__}'."+ raise BindingStepRunError(msg)+++def _store_value(*, value: Any, leaf_kind: BindingValueKind, path: str) -> StuffContent:+ """A deep copy of the value at the path, stored as content: whole when it is a concept's, wrapped when it is plain."""+ match leaf_kind:+ case BindingValueKind.CONCEPT:+ if isinstance(value, StuffContent):+ return copy.deepcopy(value)+ case BindingValueKind.TEXT:+ if isinstance(value, str):+ return TextContent(text=value)+ case BindingValueKind.NUMBER:+ if isinstance(value, (int, float)) and not isinstance(value, bool):+ return NumberContent(number=value)+ case BindingValueKind.YES_NO:+ if isinstance(value, bool):+ return YesNoContent(yes_no=value)+ case BindingValueKind.DATE:+ if isinstance(value, datetime.datetime):+ return DateContent(date=value.date(), time=value.timetz())+ if isinstance(value, datetime.date):+ return DateContent(date=value)+ case BindingValueKind.DATETIME:+ if isinstance(value, datetime.datetime):+ return DateContent(date=value.date(), time=value.timetz())+ if isinstance(value, datetime.date):+ return DateContent(date=value)+ case BindingValueKind.TIME:+ if isinstance(value, datetime.time):+ return TimeContent(time=value)+ case BindingValueKind.JSON:+ if isinstance(value, dict):+ return JSONContent(json_obj=copy.deepcopy(cast("dict[str, Any]", value)))+ case BindingValueKind.ANYTHING:+ if (natural_content := _natural_content(value=value)) is not None:+ return natural_content+ case BindingValueKind.UNDERIVABLE:+ pass+ msg = f"Binding '{path}' reached a '{type(value).__name__}', which cannot be stored as {leaf_kind.leaf_description}'s value."+ raise BindingStepRunError(msg)+++def _natural_content(*, value: Any) -> StuffContent | None:+ """The content a value of any type is stored as, keyed on its type as an `Anything` input is shaped, `None` when it has none.++ A content is copied whole; a boolean is a yes-no, a finite number a number, a time a time, a date or a datetime a date,+ a string a text and a dict a JSON object. A list, which a single value never is, and any other object have no content.+ """+ if isinstance(value, StuffContent):+ return copy.deepcopy(value)+ # `bool` before numbers, which it subclasses, and `datetime` before `date`, likewise.+ if isinstance(value, bool):+ return YesNoContent(yes_no=value)+ if isinstance(value, (int, float)):+ return NumberContent(number=value) if math.isfinite(value) else None+ if isinstance(value, datetime.time):+ return TimeContent(time=value)+ if isinstance(value, datetime.datetime):+ return DateContent(date=value.date(), time=value.timetz())+ if isinstance(value, datetime.date):+ return DateContent(date=value)+ if isinstance(value, str):+ return TextContent(text=value)+ if isinstance(value, dict):+ return JSONContent(json_obj=copy.deepcopy(cast("dict[str, Any]", value)))+ return Nonediff --git a/pipelex/pipe_controllers/binding/exceptions.py b/pipelex/pipe_controllers/binding/exceptions.pynew file mode 100644index 000000000..2d396b892--- /dev/null+++ b/pipelex/pipe_controllers/binding/exceptions.py@@ -0,0 +1,30 @@+from pipelex.base_exceptions import PipelexError+++class BindingPathUnresolvedError(PipelexError):+ """A binding step's `from` path that cannot be walked before the run: the declared structures do not hold it,+ or the concept of its root is not known.++ Raised by the derivation walk, which knows nothing of the sequence holding the step: the sequence turns+ it into a `binding_path_unresolved` validation error naming itself. The message names the segment that+ failed and the fields that were available there, so a typo can be repaired from the message alone. A+ root whose values have different concepts, as the outcomes of a condition storing it under different+ concepts, is refused the same way, the message naming each outcome and the concept it stores.+ """++ def __init__(self, message: str, *, path: str, failed_segment: str, available_fields: list[str]):+ self.path = path+ self.failed_segment = failed_segment+ self.available_fields = available_fields+ super().__init__(message)+++class BindingStepRunError(PipelexError):+ """A binding step that could not bind its value at run time, although its path was derived before the run.++ The derivation walk vouches for the path, so this signals a value whose shape contradicts its concept's+ declared structure, such as an attribute the declared field promised and the value lacks, or the derived+ multiplicity, such as a list held by a root derived as a single value. A binding whose root a pipe that+ did not resolve at validation stored is derived when it runs, and a result that contradicts the sequence's+ output, or the input of the step reading it, is reported this way too.+ """diff --git a/pipelex/pipe_controllers/condition/pipe_condition.py b/pipelex/pipe_controllers/condition/pipe_condition.pyindex 4e7c5096a..960012118 100644--- a/pipelex/pipe_controllers/condition/pipe_condition.py+++ b/pipelex/pipe_controllers/condition/pipe_condition.py@@ -20,6 +20,7 @@ from pipelex.interpreter_hub import get_optional_pipe, get_pipe_router, get_requfrom pipelex.pipe_controllers.condition.pipe_condition_blueprint import describe_expression_parse_failurefrom pipelex.pipe_controllers.condition.special_outcome import SpecialOutcomefrom pipelex.pipe_controllers.pipe_controller import PipeController+from pipelex.pipe_machinery.memory_writes import AlternativeWrites, MemoryWrite, merge_alternative_writesfrom pipelex.pipe_machinery.template_guard_lint import lint_authored_templatefrom pipelex.pipe_run.pipe_job_factory import PipeJobFactoryfrom pipelex.pipe_run.pipe_run_params import PipeRunParams, output_multiplicity_to_apply@@ -94,6 +95,37 @@ class PipeCondition(PipeController):# Exclude internal variables starting with `_`return {var for var in required_variables if not var.startswith("_")}+ @override+ def memory_writes(self, *, visited_pipes: set[str] | None = None) -> dict[str, MemoryWrite]:+ """What the outcome it runs stores besides its result: the chosen outcome runs on the caller's memory.++ The outcomes are merged as alternatives (`merge_alternative_writes`): a name is always written only if every outcome+ that can run stores it, may hold an absence if any outcome may leave one, and keeps a spec only if every outcome+ storing it stores the same one, recording otherwise which outcome stores which spec, so that a binding reading the name+ is refused before the run. A `continue` outcome stores nothing, a `fail` outcome stops the run, and an outcome+ pipe that does not resolve is left out, as a sequence assumes an unresolved pipe delivers. The alias the condition+ may add is left out too: the working memory refuses an alias over a name it already holds, so it never replaces a+ value the flow types.+ """+ if visited_pipes is None:+ visited_pipes = set()+ if self.visit_key in visited_pipes:+ return {}+ visited_pipes_with_current = visited_pipes | {self.visit_key}+ outcome_writes: list[AlternativeWrites] = []+ for outcome_pipe_code in sorted(self.pipe_dependencies()):+ outcome_pipe = get_optional_pipe(pipe_code=outcome_pipe_code)+ if outcome_pipe is not None:+ outcome_writes.append(+ AlternativeWrites(+ label=f"outcome '{outcome_pipe.code}' of pipe '{self.code}'",+ writes=outcome_pipe.memory_writes(visited_pipes=visited_pipes_with_current),+ )+ )+ if self._continue_reachable:+ outcome_writes.append(AlternativeWrites(label=f"the `continue` outcome of pipe '{self.code}'", writes={}))+ return merge_alternative_writes(alternatives=outcome_writes)+@overridedef needed_inputs(self, *, visited_pipes: set[str] | None = None) -> InputStuffSpecs:if visited_pipes is None:diff --git a/pipelex/pipe_controllers/parallel/pipe_parallel.py b/pipelex/pipe_controllers/parallel/pipe_parallel.pyindex 31d5deb0b..36795b967 100644--- a/pipelex/pipe_controllers/parallel/pipe_parallel.py+++ b/pipelex/pipe_controllers/parallel/pipe_parallel.py@@ -32,12 +32,12 @@ from pipelex.pipe_controllers.absence_taint import (ForceConsumptionInfo,LiftableStepInfo,ParallelTaintAnalysis,- SlotTaint,is_plural_step_result,scan_taint_triggers,)from pipelex.pipe_controllers.pipe_controller import PipeControllerfrom pipelex.pipe_controllers.sub_pipe import SubPipe+from pipelex.pipe_machinery.memory_writes import MemoryWrite, SlotTaintfrom pipelex.pipe_machinery.pipe_abstract import CompanionSlotfrom pipelex.pipe_run.pipe_run_params import PipeRunParams, output_multiplicity_to_applyfrom pipelex.system.job_metadata import JobMetadata@@ -78,6 +78,30 @@ class PipeParallel(PipeController):def required_variables(self) -> set[str]:return set()+ @override+ def memory_writes(self, *, visited_pipes: set[str] | None = None) -> dict[str, MemoryWrite]:+ """Each branch's result, when the parallel adds each output: its branches run on copies of the memory, and only+ the results it adds come back to it, every one of them on every run, as a value or as the absence its branch+ resolved (`analyze_branch_taint`).+ """+ if not self.add_each_output:+ return {}+ if visited_pipes is None:+ visited_pipes = set()+ if self.visit_key in visited_pipes:+ return {}+ branch_taints = self.analyze_branch_taint(visited_pipes=visited_pipes | {self.visit_key}).branch_taints+ branch_writes: dict[str, MemoryWrite] = {}+ for branch in self.parallel_sub_pipes:+ if not branch.output_name:+ continue+ branch_pipe = get_optional_pipe(pipe_code=branch.pipe_code)+ branch_writes[branch.output_name] = MemoryWrite(+ stuff_spec=branch.result_spec(step_pipe=branch_pipe) if branch_pipe is not None else None,+ absence=branch_taints.get(branch.output_name),+ )+ return branch_writes+@overridedef needed_inputs(self, *, visited_pipes: set[str] | None = None) -> InputStuffSpecs:if visited_pipes is None:@@ -256,7 +280,7 @@ class PipeParallel(PipeController):variable_names=[result_name],)- def analyze_branch_taint(self) -> ParallelTaintAnalysis:+ def analyze_branch_taint(self, *, visited_pipes: set[str] | None = None) -> ParallelTaintAnalysis:"""Static taint over the branches (D6): which branch results are maybe-absent.Within the parallel's frame, the maybe-absent slots are exactly its own `?`-declared@@ -265,6 +289,10 @@ class PipeParallel(PipeController):maybe-absent when it is liftable (a plain input fed by an optional input of theparallel) or when its pipe declares an optional output; a plural branch result isnever maybe-absent (D4: compaction / empty list).++ Args:+ visited_pipes: The recursion guard of the walk this one runs inside, `memory_writes`'s, handed to each branch's+ needed inputs; `None` for a walk of its own."""optional_input_taints: dict[str, SlotTaint] = {}for input_name, stuff_spec in self.inputs.root.items():@@ -281,7 +309,7 @@ class PipeParallel(PipeController):branch_pipe = get_optional_pipe(pipe_code=sub_pipe.pipe_code)if branch_pipe is None:continue- trigger_scan = scan_taint_triggers(branch_pipe, slot_taints=optional_input_taints)+ trigger_scan = scan_taint_triggers(branch_pipe, slot_taints=optional_input_taints, visited_pipes=visited_pipes)for asserting_name in trigger_scan.asserting_force_names:force_consumptions.append(ForceConsumptionInfo(diff --git a/pipelex/pipe_controllers/parallel/pipe_parallel_blueprint.py b/pipelex/pipe_controllers/parallel/pipe_parallel_blueprint.pyindex 095e51a93..e07ce1dde 100644--- a/pipelex/pipe_controllers/parallel/pipe_parallel_blueprint.py+++ b/pipelex/pipe_controllers/parallel/pipe_parallel_blueprint.py@@ -1,11 +1,15 @@-from typing import Literal+from typing import Any, Literal, cast+from pydantic import field_validatorfrom typing_extensions import overridefrom pipelex.core.concepts.native.concept_native import NativeConceptCode+from pipelex.core.pipes.exceptions import PipeValidationErrorfrom pipelex.core.pipes.variable_multiplicity import parse_concept_with_multiplicity+from pipelex.pipe_controllers.binding.binding_step_blueprint import BINDING_FROM_KEY, is_binding_step_dict, raw_step_mappingfrom pipelex.pipe_controllers.sub_pipe_blueprint import SubPipeBlueprintfrom pipelex.pipe_machinery.pipe_blueprint import PipeBlueprint+from pipelex.validation_error_types import PipeValidationErrorTypeclass PipeParallelBlueprint(PipeBlueprint):@@ -20,6 +24,30 @@ class PipeParallelBlueprint(PipeBlueprint):"""Return the set of pipe codes from the parallel branches."""return {branch.pipe for branch in self.branches}+ @field_validator("branches", mode="before")+ @classmethod+ def refuse_binding_branches(cls, branches: Any) -> Any:+ """Refuse a branch written as a binding step: a branch is always a pipe step.++ A binding orders a value before the steps that read it, and branches run concurrently, so a binding+ among them would only be a binding before the parallel, written in the wrong place.+ """+ if not isinstance(branches, list):+ # Not a list at all: pydantic's own `list_type` error names the field and what it holds.+ return branches+ branch_list = cast("list[Any]", branches)+ for branch_index, branch in enumerate(branch_list):+ raw_branch = raw_step_mapping(raw_step=branch)+ if raw_branch is not None and is_binding_step_dict(raw_step=raw_branch):+ from_path = raw_branch.get(BINDING_FROM_KEY)+ msg = (+ f"Branch {branch_index + 1} of the parallel is a binding step (it carries `from`), but a PipeParallel branch is always "+ f"a pipe step: bind the value in a step of the calling PipeSequence, before the PipeParallel step "+ f'(`{{ from = "{from_path}", result = "<name>" }}`), and have the branch read that name.'+ )+ raise PipeValidationError(message=msg, error_type=PipeValidationErrorType.BINDING_STEP_INVALID)+ return branch_list+@overridedef validate_output(self):"""A parallel always combines its branch outputs into its declared output.diff --git a/pipelex/pipe_controllers/sequence/pipe_sequence.py b/pipelex/pipe_controllers/sequence/pipe_sequence.pyindex 1a4911813..ca478691b 100644--- a/pipelex/pipe_controllers/sequence/pipe_sequence.py+++ b/pipelex/pipe_controllers/sequence/pipe_sequence.py@@ -1,8 +1,10 @@+import sysfrom typing import TYPE_CHECKING, Any, Literalfrom pydantic import field_validatorfrom typing_extensions import override+from pipelex.core.concepts.native.concept_native import NativeConceptCodefrom pipelex.core.memory.working_memory import WorkingMemoryfrom pipelex.core.pipes.exceptions import PipeValidationErrorfrom pipelex.core.pipes.inputs.exceptions import InputStuffSpecNotFoundError@@ -10,33 +12,50 @@ from pipelex.core.pipes.inputs.input_stuff_specs import InputStuffSpecsfrom pipelex.core.pipes.inputs.input_stuff_specs_factory import InputStuffSpecsFactoryfrom pipelex.core.pipes.pipe_output import PipeOutputfrom pipelex.core.pipes.stuff_spec.stuff_spec import StuffSpec-from pipelex.core.pipes.variable_multiplicity import PresenceMarker, VariableMultiplicity, is_multiplicity_compatible+from pipelex.core.pipes.variable_multiplicity import PresenceMarker, VariableMultiplicity, is_multiple_multiplicity, is_multiplicity_compatiblefrom pipelex.core.qualified_ref import QualifiedRef-from pipelex.interpreter_hub import get_concept_library, get_optional_pipe, get_required_pipe+from pipelex.core.stuffs.list_content import ListContent+from pipelex.core.stuffs.stuff import Stuff+from pipelex.interpreter_hub import get_concept_library, get_native_concept, get_optional_pipe, get_required_pipefrom pipelex.pipe_controllers.absence_taint import (ForceConsumptionInfo,LiftableStepInfo,SequenceTaintAnalysis,- SlotTaint,+ TaintTriggerScan,is_plural_step_result,scan_taint_triggers,)-from pipelex.pipe_controllers.parallel.pipe_parallel import PipeParallel+from pipelex.pipe_controllers.binding.binding_derivation import BindingDerivation+from pipelex.pipe_controllers.binding.binding_step import BindingOutcome, BindingStep+from pipelex.pipe_controllers.binding.exceptions import BindingStepRunErrorfrom pipelex.pipe_controllers.pipe_controller import PipeControllerfrom pipelex.pipe_controllers.sequence.exceptions import PipeSequenceValueError+from pipelex.pipe_controllers.sequence.sequence_typed_flow import (+ DerivedBinding,+ FlowSlot,+ SequenceStep,+ SequenceTypedFlow,+ build_sequence_typed_flow,+ derive_binding_spec,+ step_memory_writes,+)from pipelex.pipe_controllers.sub_pipe import SubPipe+from pipelex.pipe_machinery.memory_writes import MemoryWrite, SlotTaint, taint_after_write+from pipelex.pipe_machinery.validation import is_valid_input_namefrom pipelex.pipe_run.pipe_run_params import PipeRunParams, output_multiplicity_to_applyfrom pipelex.system.job_metadata import JobMetadatafrom pipelex.tools.misc.string_utils import get_root_from_dotted_pathfrom pipelex.validation_error_types import PipeValidationErrorTypeif TYPE_CHECKING:+ from pipelex.core.concepts.concept import Conceptfrom pipelex.libraries.library_crate import LibraryCrateclass PipeSequence(PipeController):type: Literal["PipeSequence"] = "PipeSequence"- sequential_sub_pipes: list[SubPipe]+ # The steps in order: pipe steps, which run a pipe, and binding steps, which bind a value already in working memory.+ sequential_sub_pipes: list[SequenceStep]@overridedef required_variables(self) -> set[str]:@@ -44,7 +63,7 @@ class PipeSequence(PipeController):@field_validator("sequential_sub_pipes", mode="after")@classmethod- def validate_sequential_sub_pipes(cls, value: list[SubPipe]) -> list[SubPipe]:+ def validate_sequential_sub_pipes(cls, value: list[SequenceStep]) -> list[SequenceStep]:if not value:msg = f"PipeSequence '{cls.code}' requires at least one sub-pipe"raise ValueError(msg)@@ -54,9 +73,323 @@ class PipeSequence(PipeController):def validate_inputs_static(self):pass+ @property+ def pipe_steps(self) -> list[SubPipe]:+ """The steps that run a pipe, in order; a binding step runs none."""+ return [step for step in self.sequential_sub_pipes if isinstance(step, SubPipe)]++ @property+ def has_binding_step(self) -> bool:+ """Whether a step binds a value: only binding steps read the typed flow, so a sequence without one never builds it."""+ return any(isinstance(step, BindingStep) for step in self.sequential_sub_pipes)++ def build_typed_flow(self) -> SequenceTypedFlow:+ """The typed flow of the steps, deriving what each binding step binds.++ Raises:+ PipeValidationError: ``BINDING_PATH_UNRESOLVED`` when a binding's path cannot be walked.+ """+ return self._build_typed_flow(visited_pipes=set())++ def _build_typed_flow(self, *, visited_pipes: set[str]) -> SequenceTypedFlow:+ return build_sequence_typed_flow(+ steps=self.sequential_sub_pipes,+ declared_inputs=self.inputs,+ sequence_code=self.code,+ domain_code=self.domain_code,+ visited_pipes=visited_pipes | {self.visit_key},+ )++ @override+ def memory_writes(self, *, visited_pipes: set[str] | None = None) -> dict[str, MemoryWrite]:+ """What the steps store, as the flow stands after the last one: a sequence runs its steps on its caller's memory.++ Each name the steps store carries the spec the typed flow gives it, the absence the taint walk leaves on it, and+ whether some step stores it on every run.++ Raises:+ PipeValidationError: ``BINDING_PATH_UNRESOLVED`` when a binding's path cannot be walked, as validating the+ sequence itself reports.+ """+ if visited_pipes is None:+ visited_pipes = set()+ if self.visit_key in visited_pipes:+ return {}+ typed_flow = self._build_typed_flow(visited_pipes=visited_pipes)+ taint_analysis = self._analyze_taint(visited_pipes=visited_pipes | {self.visit_key}, typed_flow=typed_flow)+ return {+ name: MemoryWrite(+ stuff_spec=slot.stuff_spec,+ disagreement=slot.disagreement,+ absence=taint_analysis.final_slot_taints.get(name),+ is_always_written=name in typed_flow.always_written_names,+ )+ for name, slot in typed_flow.written_slots.items()+ }++ def _walk_needed_inputs(self, *, visited_pipes: set[str]) -> tuple[InputStuffSpecs, set[str]]:+ """The inputs the steps need from the sequence's caller, and, among them, the roots of binding steps.++ A name is needed when a step reads it before any earlier step stores it on every run: a step stores its result, and+ what its pipe always stores besides (`step_memory_writes`). A name only some runs of an earlier step store, as a+ condition's outcomes that do not all store it, is still needed, since a run may leave it as the caller had it.+ """+ visited_pipes_with_current = visited_pipes | {self.visit_key}+ needed_inputs = InputStuffSpecsFactory.make_empty()+ binding_root_needs: set[str] = set()+ generated_outputs: set[str] = set()++ for sequential_sub_pipe in self.sequential_sub_pipes:+ if isinstance(sequential_sub_pipe, BindingStep):+ # A binding reads its root plainly. A root no earlier step always stores is a needed input of the sequence,+ # typed as the sequence declares it, which is the concept the binding's walk validates its path against. An+ # undeclared root is needed as `Anything`, a flexible need, and refused as a missing input.+ root_name = sequential_sub_pipe.root_name+ if root_name not in generated_outputs:+ binding_root_needs.add(root_name)+ if root_name not in needed_inputs.root:+ declared_root_spec = self.inputs.root.get(root_name)+ if declared_root_spec is not None:+ needed_inputs.add_stuff_spec(+ variable_name=root_name,+ concept=declared_root_spec.concept,+ multiplicity=declared_root_spec.multiplicity,+ presence=declared_root_spec.presence,+ )+ else:+ needed_inputs.add_stuff_spec(+ variable_name=root_name, concept=get_native_concept(native_concept=NativeConceptCode.ANYTHING)+ )+ generated_outputs.add(sequential_sub_pipe.output_name)+ continue+ # Skip cross-package pipe refs that aren't loaded yet (dependency not resolved)+ if QualifiedRef.has_cross_package_prefix(sequential_sub_pipe.pipe_code):+ sub_pipe = get_optional_pipe(pipe_code=sequential_sub_pipe.pipe_code)+ if sub_pipe is None:+ continue+ else:+ sub_pipe = get_required_pipe(pipe_code=sequential_sub_pipe.pipe_code)+ # Use the centralized recursion detection+ sub_pipe_needed_inputs = sub_pipe.needed_inputs(visited_pipes=visited_pipes_with_current)++ if sequential_sub_pipe.batch_params:+ input_list_root = get_root_from_dotted_path(sequential_sub_pipe.batch_params.input_list_stuff_name)+ if input_list_root not in generated_outputs:+ try:+ stuff_spec = sub_pipe_needed_inputs.get_required_stuff_spec(+ variable_name=sequential_sub_pipe.batch_params.input_item_stuff_name+ )+ except InputStuffSpecNotFoundError as exc:+ msg = (+ f"Batch input item named '{sequential_sub_pipe.batch_params.input_item_stuff_name}' is not "+ f"in this PipeSequence '{self.code}' input requirements: {sub_pipe_needed_inputs.format_for_display()}"+ )+ raise PipeSequenceValueError(msg) from exc+ is_dotted_path = "." in sequential_sub_pipe.batch_params.input_list_stuff_name+ needed_inputs.add_stuff_spec(+ variable_name=input_list_root,+ concept=stuff_spec.concept,+ multiplicity=True if not is_dotted_path else None,+ )+ for input_name, stuff_spec in sub_pipe_needed_inputs.items:+ if input_name != sequential_sub_pipe.batch_params.input_item_stuff_name and input_name not in generated_outputs:+ needed_inputs.add_stuff_spec(+ variable_name=input_name,+ concept=stuff_spec.concept,+ multiplicity=stuff_spec.multiplicity,+ presence=stuff_spec.presence,+ )+ else:+ for input_name, stuff_spec in sub_pipe_needed_inputs.items:+ if input_name not in generated_outputs:+ needed_inputs.add_stuff_spec(+ variable_name=input_name, concept=stuff_spec.concept, multiplicity=stuff_spec.multiplicity, presence=stuff_spec.presence+ )++ # What the step stores on every run, besides its result, and its result.+ step_writes = step_memory_writes(step=sequential_sub_pipe, step_pipe=sub_pipe, visited_pipes=visited_pipes_with_current)+ generated_outputs.update(written_name for written_name, memory_write in step_writes.items() if memory_write.is_always_written)+ if sequential_sub_pipe.output_name:+ generated_outputs.add(sequential_sub_pipe.output_name)++ return needed_inputs, binding_root_needs++ @override+ def refuse_undeclared_needed_input(self, *, variable_name: str) -> None:+ """Refuse a binding's root that the sequence neither declares nor always stores, asking for the concept its path walks.++ A root that is not a plain input name cannot be declared, so the refusal asks for a step storing it instead.+ """+ _, binding_root_needs = self._walk_needed_inputs(visited_pipes=set())+ if variable_name not in binding_root_needs:+ return+ for step in self.sequential_sub_pipes:+ if isinstance(step, BindingStep) and step.root_name == variable_name:+ if is_valid_input_name(variable_name):+ remedy = (+ f"Declare '{variable_name}' in the sequence's `inputs`, with the concept whose structure holds the path '{step.from_path}'."+ )+ else:+ remedy = (+ f"'{variable_name}' cannot be an input of the sequence, since an input name is a plain snake_case identifier: "+ f"store a value under '{variable_name}' in an earlier step, whose `result` names it, or bind from a plain name."+ )+ msg = (+ f"In pipe '{self.code}', the binding step {step.as_written} reads '{variable_name}', which is neither an input of the "+ f"sequence nor always stored by an earlier step. {remedy}"+ )+ raise PipeValidationError(+ message=msg,+ error_type=PipeValidationErrorType.MISSING_INPUT_VARIABLE,+ domain_code=self.domain_code,+ pipe_code=self.code,+ variable_names=[variable_name],+ )+@overridedef validate_inputs_with_library(self):- pass+ """Derive every binding step through the typed flow, then check the steps reading a binding's result against it.++ A step asking a sequence ending with a binding for a count of outputs is checked against what that binding binds first.+ """+ self._refuse_unhonoured_output_counts()+ if not self.has_binding_step:+ return+ typed_flow = self.build_typed_flow()+ for step_index, step in enumerate(self.sequential_sub_pipes):+ if not isinstance(step, SubPipe):+ continue+ binding_slots = typed_flow.binding_slots_by_pipe_step.get(step_index)+ if binding_slots:+ self._check_step_reading_bindings(step_index=step_index, step=step, binding_slots=binding_slots)++ def _check_step_reading_bindings(self, *, step_index: int, step: SubPipe, binding_slots: dict[str, FlowSlot]) -> None:+ """Refuse a pipe step reading, among the bindings' results it can see, one whose spec its pipe does not need.++ Raises:+ PipeValidationError: ``INPUT_STUFF_SPEC_MISMATCH`` naming the step and the binding.+ """+ step_pipe = get_optional_pipe(pipe_code=step.pipe_code)+ if step_pipe is None:+ return+ step_needs = step_pipe.needed_inputs()+ batch_item_name: str | None = None+ if step.batch_params is not None:+ batch_item_name = step.batch_params.input_item_stuff_name+ list_slot = binding_slots.get(step.batch_params.input_list_stuff_name)+ item_need = step_needs.root.get(batch_item_name)+ if list_slot is not None and list_slot.stuff_spec is not None and item_need is not None:+ self._check_binding_consumer(+ step_index=step_index,+ step_pipe_code=step_pipe.code,+ variable_name=step.batch_params.input_list_stuff_name,+ slot=list_slot,+ needed_spec=StuffSpec(concept=item_need.concept, multiplicity=True),+ )+ for input_name, needed_spec in step_needs.items:+ if input_name == batch_item_name:+ continue+ slot = binding_slots.get(input_name)+ if slot is None or slot.stuff_spec is None:+ continue+ self._check_binding_consumer(+ step_index=step_index, step_pipe_code=step_pipe.code, variable_name=input_name, slot=slot, needed_spec=needed_spec+ )++ def final_binding_spec(self) -> StuffSpec | None:+ """The spec the binding step ending the sequence binds, `None` when the last step runs a pipe or the flow cannot type the binding's root.++ Raises:+ PipeValidationError: ``BINDING_PATH_UNRESOLVED`` when a binding's path cannot be walked, as validating the sequence+ itself reports.+ """+ if not isinstance(self.sequential_sub_pipes[-1], BindingStep):+ return None+ return self.build_typed_flow().binding_specs.get(len(self.sequential_sub_pipes) - 1)++ def _refuse_unhonoured_output_counts(self) -> None:+ """Refuse a step asking, with `nb_output` or `multiple_output`, a sequence ending with a binding for a count its binding+ does not bind.++ A binding binds what its path derives, whatever count its caller asks for, so the request could only reach the called+ sequence's own steps, where a value typed single would hold a list. A batched step runs each branch with the pipe's own+ multiplicity, so its count is never asked of the sequence.++ Raises:+ PipeValidationError: ``INADEQUATE_OUTPUT_MULTIPLICITY`` naming the step and the binding.+ """+ for step_index, step in enumerate(self.sequential_sub_pipes):+ if not isinstance(step, SubPipe) or step.output_multiplicity is None or step.batch_params is not None:+ continue+ step_pipe = get_optional_pipe(pipe_code=step.pipe_code)+ if not isinstance(step_pipe, PipeSequence):+ continue+ bound_spec = step_pipe.final_binding_spec()+ if bound_spec is None:+ continue+ requested = output_multiplicity_to_apply(base_multiplicity=step_pipe.output.multiplicity, override_multiplicity=step.output_multiplicity)+ requested_multiplicity: VariableMultiplicity | None+ if not requested.is_multiple_outputs_enabled:+ requested_multiplicity = None+ requested_label = "a single output"+ elif requested.specific_output_count is not None:+ requested_multiplicity = requested.specific_output_count+ requested_label = f"{requested.specific_output_count} outputs"+ else:+ requested_multiplicity = True+ requested_label = "multiple outputs"+ if is_multiplicity_compatible(source_multiplicity=bound_spec.multiplicity, target_multiplicity=requested_multiplicity):+ continue+ final_binding = step_pipe.sequential_sub_pipes[-1]+ binding_label = final_binding.as_written if isinstance(final_binding, BindingStep) else "a binding step"+ bound_ref = bound_spec.to_bundle_representation(relative_to_domain=self.domain_code)+ msg = (+ f"In pipe '{self.code}', step {step_index + 1} asks pipe '{step_pipe.code}' for {requested_label}, but '{step_pipe.code}' "+ f"ends with the binding step {binding_label}, which binds '{bound_ref}': a binding binds what its path derives, whatever "+ f"count its caller asks for. Remove the step's `nb_output` or `multiple_output`, or ask for what the binding binds."+ )+ raise PipeValidationError(+ message=msg,+ error_type=PipeValidationErrorType.INADEQUATE_OUTPUT_MULTIPLICITY,+ domain_code=self.domain_code,+ pipe_code=self.code,+ provided_concept_code=bound_spec.concept.concept_ref,+ required_concept_codes=[step_pipe.output.concept.concept_ref],+ )++ def _check_binding_consumer(self, *, step_index: int, step_pipe_code: str, variable_name: str, slot: FlowSlot, needed_spec: StuffSpec) -> None:+ """Refuse a step reading a binding's result as a concept or multiplicity the binding does not derive."""+ bound_spec = slot.stuff_spec+ if bound_spec is None or slot.binding_step_index is None:+ return+ if needed_spec.concept.code in {NativeConceptCode.DYNAMIC, NativeConceptCode.ANYTHING}:+ return+ binding_step = self.sequential_sub_pipes[slot.binding_step_index]+ binding_label = binding_step.as_written if isinstance(binding_step, BindingStep) else f"step {slot.binding_step_index + 1}"+ is_concept_compatible = get_concept_library().is_compatible(tested_concept=bound_spec.concept, wanted_concept=needed_spec.concept)+ is_multiplicity_ok = is_multiplicity_compatible(source_multiplicity=bound_spec.multiplicity, target_multiplicity=needed_spec.multiplicity)+ if is_concept_compatible and is_multiplicity_ok:+ return+ bound_ref = StuffSpec(concept=bound_spec.concept, multiplicity=bound_spec.multiplicity).to_bundle_representation(+ relative_to_domain=self.domain_code+ )+ needed_ref = StuffSpec(concept=needed_spec.concept, multiplicity=needed_spec.multiplicity).to_bundle_representation(+ relative_to_domain=self.domain_code+ )+ msg = (+ f"In pipe '{self.code}', step {step_index + 1} (pipe '{step_pipe_code}') reads '{variable_name}' as '{needed_ref}', but the binding "+ f"step {binding_label} binds it as '{bound_ref}'. Declare the input as '{bound_ref}' in pipe '{step_pipe_code}', or bind a path "+ f"that reaches a '{needed_ref}'."+ )+ raise PipeValidationError(+ message=msg,+ error_type=PipeValidationErrorType.INPUT_STUFF_SPEC_MISMATCH,+ domain_code=self.domain_code,+ pipe_code=self.code,+ variable_names=[variable_name],+ provided_concept_code=bound_spec.concept.concept_ref,+ required_concept_codes=[needed_spec.concept.concept_ref],+ )@overridedef validate_output_static(self):@@ -66,35 +399,56 @@ class PipeSequence(PipeController):def validate_output_with_library(self):"""Validate the output for the pipe sequence.- The output of the pipe sequence should match the output of the last step,- both in terms of concept compatibility and multiplicity.+ The output of the pipe sequence should match the output of the last step, both in terms of concept+ compatibility and multiplicity. A last step that binds a value is checked by the spec it derives, exactly+ as a pipe step is checked by its pipe's output."""- last_sub_pipe = self.sequential_sub_pipes[-1]- last_step_pipe_code = last_sub_pipe.pipe_code- # Skip output validation if the last step is an unresolved cross-package ref- if QualifiedRef.has_cross_package_prefix(last_step_pipe_code) and get_optional_pipe(pipe_code=last_step_pipe_code) is None:- return- last_step_pipe = get_required_pipe(pipe_code=last_step_pipe_code)-- # The step's effective output, resolved exactly the way the run path resolves it (declared- # multiplicity + the step-level override), so this static promise cannot rule a count- # differently from the execution it describes. Reading the two apart — an override tested for- # truthiness, a declaration tested for `is None` — is what let a `nb_output = 1` step read as- # plural here while the runtime made it single.- multiplicity_resolution = output_multiplicity_to_apply(- base_multiplicity=last_step_pipe.output.multiplicity,- override_multiplicity=last_sub_pipe.output_multiplicity,- )- is_plural_last_step_output = multiplicity_resolution.is_multiple_outputs_enabled- # A declaration spells a singular slot with no suffix at all; `resolved_multiplicity` spells a- # forced single as `False`, which is override vocabulary and renders as nothing an author writes.+ last_step = self.sequential_sub_pipes[-1]+ last_step_concept: Concept+ last_step_label: str+ is_last_step_output_optional: booleffective_last_step_output_multiplicity: VariableMultiplicity | None- if not is_plural_last_step_output:- effective_last_step_output_multiplicity = None- elif multiplicity_resolution.specific_output_count is not None:- effective_last_step_output_multiplicity = multiplicity_resolution.specific_output_count+ if isinstance(last_step, BindingStep):+ typed_flow = self.build_typed_flow()+ binding_spec = typed_flow.binding_specs.get(len(self.sequential_sub_pipes) - 1)+ if binding_spec is None:+ # A root a pipe that does not resolve stored: nothing says what the binding derives, so only its absence is+ # checked here, and the run checks the binding it derives against the output (`_refuse_runtime_output_mismatch`).+ self._refuse_escaping_absence(taint_analysis=self._analyze_taint(visited_pipes=None, typed_flow=typed_flow))+ return+ last_step_concept = binding_spec.concept+ last_step_label = f"the binding step {last_step.as_written}"+ # A binding's own maybe-absence is the taint pass's to report, as its result's taint.+ is_last_step_output_optional = False+ effective_last_step_output_multiplicity = binding_spec.multiplicityelse:- effective_last_step_output_multiplicity = True+ last_step_pipe_code = last_step.pipe_code+ # Skip output validation if the last step is an unresolved cross-package ref+ if QualifiedRef.has_cross_package_prefix(last_step_pipe_code) and get_optional_pipe(pipe_code=last_step_pipe_code) is None:+ return+ last_step_pipe = get_required_pipe(pipe_code=last_step_pipe_code)+ last_step_concept = last_step_pipe.output.concept+ last_step_label = f"'{last_step_pipe.code}'"+ is_last_step_output_optional = last_step_pipe.output.presence.is_optional++ # The step's effective output, resolved exactly the way the run path resolves it (declared+ # multiplicity + the step-level override), so this static promise cannot rule a count+ # differently from the execution it describes. Reading the two apart — an override tested for+ # truthiness, a declaration tested for `is None` — is what let a `nb_output = 1` step read as+ # plural here while the runtime made it single.+ multiplicity_resolution = output_multiplicity_to_apply(+ base_multiplicity=last_step_pipe.output.multiplicity,+ override_multiplicity=last_step.output_multiplicity,+ )+ # A declaration spells a singular slot with no suffix at all; `resolved_multiplicity` spells a+ # forced single as `False`, which is override vocabulary and renders as nothing an author writes.+ if not multiplicity_resolution.is_multiple_outputs_enabled:+ effective_last_step_output_multiplicity = None+ elif multiplicity_resolution.specific_output_count is not None:+ effective_last_step_output_multiplicity = multiplicity_resolution.specific_output_count+ else:+ effective_last_step_output_multiplicity = True+ is_plural_last_step_output = is_multiple_multiplicity(multiplicity=effective_last_step_output_multiplicity)taint_analysis = self.analyze_taint()@@ -107,20 +461,20 @@ class PipeSequence(PipeController):# a presence marker and represents an absent plural result as an empty list.expected_output_presence = PresenceMarker.PLAINif not is_plural_last_step_output and (- self.output.presence.is_optional or last_step_pipe.output.presence.is_optional or taint_analysis.output_taint is not None+ self.output.presence.is_optional or is_last_step_output_optional or taint_analysis.output_taint is not None):expected_output_presence = PresenceMarker.OPTIONALexpected_output_ref = StuffSpec(- concept=last_step_pipe.output.concept,+ concept=last_step_concept,multiplicity=effective_last_step_output_multiplicity,presence=expected_output_presence,).to_bundle_representation(relative_to_domain=self.domain_code)# Check concept compatibility- if not get_concept_library().is_compatible(tested_concept=last_step_pipe.output.concept, wanted_concept=self.output.concept):+ if not get_concept_library().is_compatible(tested_concept=last_step_concept, wanted_concept=self.output.concept):msg = (- f"PipeSequence concept mismatch: the output concept '{last_step_pipe.output.concept.concept_ref}' "- f"of the last step '{last_step_pipe.code}' of sequence pipe '{self.code}' "+ f"PipeSequence concept mismatch: the output concept '{last_step_concept.concept_ref}' "+ f"of the last step {last_step_label} of sequence pipe '{self.code}' "f"is not compatible with the output concept '{self.output.concept.concept_ref}' of the sequence.")raise PipeValidationError(@@ -128,7 +482,7 @@ class PipeSequence(PipeController):error_type=PipeValidationErrorType.INADEQUATE_OUTPUT_CONCEPT,domain_code=self.domain_code,pipe_code=self.code,- provided_concept_code=last_step_pipe.output.concept.concept_ref,+ provided_concept_code=last_step_concept.concept_ref,required_concept_codes=[self.output.concept.concept_ref],expected_output_ref=expected_output_ref,)@@ -141,7 +495,7 @@ class PipeSequence(PipeController):declared_output_ref = self.output.to_bundle_representation(relative_to_domain=self.domain_code)msg = (f"PipeSequence output multiplicity mismatch: the sequence '{self.code}' declares its output as "- f"'{declared_output_ref}', but its last step '{last_step_pipe.code}' yields '{expected_output_ref}'. "+ f"'{declared_output_ref}', but its last step {last_step_label} yields '{expected_output_ref}'. "f"Update the sequence's output to '{expected_output_ref}' (or change the last step).")raise PipeValidationError(@@ -149,13 +503,17 @@ class PipeSequence(PipeController):error_type=PipeValidationErrorType.INADEQUATE_OUTPUT_MULTIPLICITY,domain_code=self.domain_code,pipe_code=self.code,- provided_concept_code=last_step_pipe.output.concept.concept_ref,+ provided_concept_code=last_step_concept.concept_ref,required_concept_codes=[self.output.concept.concept_ref],expected_output_ref=expected_output_ref,)- # The absence-taint boundary check (D6): a maybe-absent slot ending the sequence must be- # matched by an optional (`?`) declared output, or the taint silently escapes the boundary.+ self._refuse_escaping_absence(taint_analysis=taint_analysis)++ def _refuse_escaping_absence(self, *, taint_analysis: SequenceTaintAnalysis) -> None:+ """The absence-taint boundary check (D6): a maybe-absent slot ending the sequence must be matched by an optional+ (`?`) declared output, or the taint silently escapes the boundary.+ """if taint_analysis.output_taint is not None and not self.output.presence.is_optional:msg = (f"PipeSequence '{self.code}' output '{self.output.concept.concept_ref}' may resolve absent at run time, "@@ -180,7 +538,19 @@ class PipeSequence(PipeController):outputs; it propagates through lifted steps (plain input fed a tainted slot), terminatesat `?` (absorb) and `!` (assert) inputs, and never touches plural slots (D4). A step thatrewrites a tainted slot with a guaranteed value clears it — the static mirror of the- runtime value-supersedes-record invariant.+ runtime value-supersedes-record invariant. What a step stores besides its result, a nested+ controller's names (`step_memory_writes`), carries its own absence the same way.+ """+ return self._analyze_taint(visited_pipes=None, typed_flow=None)++ def _analyze_taint(self, *, visited_pipes: set[str] | None, typed_flow: SequenceTypedFlow | None) -> SequenceTaintAnalysis:+ """The taint walk, over a typed flow already built when `typed_flow` is given, whose per-step stores it then reads+ instead of computing them again.++ With `visited_pipes`, the walk runs inside the recursion guard of `memory_writes`, which reads only the absences it+ leaves on the slots: a step is then scanned for its consumptions only while some slot may be absent, since with none+ it lifts on nothing, and only the `!` lint reads what it consumes. That keeps a caller's needed inputs, which read+ `memory_writes` for every nested controller, from walking each nested controller's needs a second time."""slot_taints: dict[str, SlotTaint] = {}for input_name, stuff_spec in self.inputs.root.items():@@ -193,8 +563,23 @@ class PipeSequence(PipeController):liftable_steps: list[LiftableStepInfo] = []force_consumptions: list[ForceConsumptionInfo] = []last_step_taint: SlotTaint | None = None-- for sequential_sub_pipe in self.sequential_sub_pipes:+ if typed_flow is None and self.has_binding_step:+ typed_flow = self._build_typed_flow(visited_pipes=visited_pipes or set())+ writes_visited_pipes = visited_pipes or {self.visit_key}++ for step_index, sequential_sub_pipe in enumerate(self.sequential_sub_pipes):+ if isinstance(sequential_sub_pipe, BindingStep):+ binding_taint = self._binding_step_taint(+ binding_step=sequential_sub_pipe,+ derivation=typed_flow.binding_derivations.get(step_index) if typed_flow is not None else None,+ root_taint=slot_taints.get(sequential_sub_pipe.root_name),+ )+ if binding_taint is None:+ slot_taints.pop(sequential_sub_pipe.output_name, None)+ else:+ slot_taints[sequential_sub_pipe.output_name] = binding_taint+ last_step_taint = binding_taint+ continuesub_pipe = get_optional_pipe(pipe_code=sequential_sub_pipe.pipe_code)if sub_pipe is None:# Unresolved cross-package ref: assume the step delivers (conservative-permissive,@@ -205,7 +590,10 @@ class PipeSequence(PipeController):continue# How does this step consume the currently tainted slots?- trigger_scan = scan_taint_triggers(sub_pipe, slot_taints=slot_taints)+ if visited_pipes is not None and not slot_taints:+ trigger_scan = TaintTriggerScan(trigger_names=(), trigger_taint=None)+ else:+ trigger_scan = scan_taint_triggers(sub_pipe, slot_taints=slot_taints, visited_pipes=visited_pipes)for asserting_name in trigger_scan.asserting_force_names:force_consumptions.append(ForceConsumptionInfo(within_pipe_ref=self.pipe_ref, pipe_ref=sub_pipe.pipe_ref, variable_name=asserting_name, is_asserting=True)@@ -252,37 +640,37 @@ class PipeSequence(PipeController):origin_slot_name=output_slot_name or sub_pipe.code,)- # An add_each_output parallel also writes each branch's result slot into this flow.- if isinstance(sub_pipe, PipeParallel) and sub_pipe.add_each_output:+ # What the step stores besides its result: a nested sequence's steps, a condition's outcome, or an+ # add_each_output parallel's branches write into this flow, each name with its own absence.+ if typed_flow is not None and step_index in typed_flow.memory_writes_by_pipe_step:+ step_writes = typed_flow.memory_writes_by_pipe_step[step_index]+ else:+ step_writes = step_memory_writes(step=sequential_sub_pipe, step_pipe=sub_pipe, visited_pipes=writes_visited_pipes)+ for written_name, memory_write in step_writes.items():if step_lifted and trigger_taint is not None:- # The whole parallel lifts: companion (branch) slots resolve exactly like the- # runtime `_make_lifted_output` does — singular slots go absent, plural slots- # become guaranteed empty lists (D4).- for companion_slot in sub_pipe.lifted_companion_slots():- if companion_slot.is_plural:- slot_taints.pop(companion_slot.slot_name, None)- else:- slot_taints[companion_slot.slot_name] = SlotTaint(- source=trigger_taint.source,- origin_slot_name=trigger_taint.origin_slot_name,- chain=(- *trigger_taint.chain,- (- f"pipe '{sub_pipe.code}' may be skipped when '{trigger_scan.trigger_names[0]}' is absent"- f" → branch slot '{companion_slot.slot_name}'"- ),- ),- )+ # The whole step lifts: what it always stores resolves exactly like the runtime+ # `_make_lifted_output` resolves its companion slots — a singular slot goes absent,+ # a plural slot becomes a guaranteed empty list (D4) — and a name only some runs+ # store is left as it was.+ if not memory_write.is_always_written:+ continue+ if memory_write.stuff_spec is not None and memory_write.stuff_spec.is_multiple():+ slot_taints.pop(written_name, None)+ else:+ slot_taints[written_name] = SlotTaint(+ source=trigger_taint.source,+ origin_slot_name=trigger_taint.origin_slot_name,+ chain=(+ *trigger_taint.chain,+ f"pipe '{sub_pipe.code}' may be skipped when '{trigger_scan.trigger_names[0]}' is absent → slot '{written_name}'",+ ),+ )+ continue+ written_taint = taint_after_write(prior_taint=slot_taints.get(written_name), memory_write=memory_write)+ if written_taint is None:+ slot_taints.pop(written_name, None)else:- branch_taints = sub_pipe.analyze_branch_taint().branch_taints- for parallel_sub_pipe in sub_pipe.parallel_sub_pipes:- if not parallel_sub_pipe.output_name:- continue- branch_taint = branch_taints.get(parallel_sub_pipe.output_name)- if branch_taint is None:- slot_taints.pop(parallel_sub_pipe.output_name, None)- else:- slot_taints[parallel_sub_pipe.output_name] = branch_taint+ slot_taints[written_name] = written_taintif output_slot_name:if step_output_taint is None:@@ -295,8 +683,44 @@ class PipeSequence(PipeController):liftable_steps=tuple(liftable_steps),output_taint=last_step_taint,force_consumptions=tuple(force_consumptions),+ final_slot_taints=slot_taints,)+ @staticmethod+ def _binding_step_taint(+ *,+ binding_step: BindingStep,+ derivation: BindingDerivation | None,+ root_taint: SlotTaint | None,+ ) -> SlotTaint | None:+ """The presence of a binding's result: tainted by its root's taint, or by a path that may find nothing.++ A list result is never tainted, since a lifted binding over a list binds an empty list and a path reaching+ nothing on an item drops it. A root the flow cannot type has no derivation, so whether its path may find nothing,+ or crosses a list, is unknown: the run records a skipped absence when the root is absent (`skip_untyped_root`), so+ the root's own taint is kept, and its path is otherwise assumed to deliver, as an unresolved pipe is.+ """+ if derivation is not None and derivation.is_plural:+ return None+ if root_taint is not None:+ return SlotTaint(+ source=root_taint.source,+ origin_slot_name=root_taint.origin_slot_name,+ chain=(+ *root_taint.chain,+ (+ f"binding step {binding_step.as_written} may be skipped when '{binding_step.root_name}' is absent"+ f" → slot '{binding_step.output_name}'"+ ),+ ),+ )+ if derivation is not None and derivation.may_find_nothing:+ return SlotTaint(+ source=f"binding step {binding_step.as_written}, whose path may find nothing at '{derivation.first_optional_path}'",+ origin_slot_name=binding_step.output_name,+ )+ return None+@overridedef needed_inputs(self, *, visited_pipes: set[str] | None = None) -> InputStuffSpecs:if visited_pipes is None:@@ -306,71 +730,12 @@ class PipeSequence(PipeController):if self.visit_key in visited_pipes:return InputStuffSpecsFactory.make_empty()- # Add this pipe to visited set for recursive calls- visited_pipes_with_current = visited_pipes | {self.visit_key}-- needed_inputs = InputStuffSpecsFactory.make_empty()- generated_outputs: set[str] = set()-- for sequential_sub_pipe in self.sequential_sub_pipes:- # Skip cross-package pipe refs that aren't loaded yet (dependency not resolved)- if QualifiedRef.has_cross_package_prefix(sequential_sub_pipe.pipe_code):- sub_pipe = get_optional_pipe(pipe_code=sequential_sub_pipe.pipe_code)- if sub_pipe is None:- continue- else:- sub_pipe = get_required_pipe(pipe_code=sequential_sub_pipe.pipe_code)- # Use the centralized recursion detection- sub_pipe_needed_inputs = sub_pipe.needed_inputs(visited_pipes=visited_pipes_with_current)-- if isinstance(sub_pipe, PipeParallel) and sub_pipe.add_each_output:- for sub_parallel_pipe in sub_pipe.parallel_sub_pipes:- if (sub_pipe.add_each_output and sub_parallel_pipe.output_name) or sub_parallel_pipe.output_name:- generated_outputs.add(sub_parallel_pipe.output_name)-- if sequential_sub_pipe.batch_params:- input_list_root = get_root_from_dotted_path(sequential_sub_pipe.batch_params.input_list_stuff_name)- if input_list_root not in generated_outputs:- try:- stuff_spec = sub_pipe_needed_inputs.get_required_stuff_spec(- variable_name=sequential_sub_pipe.batch_params.input_item_stuff_name- )- except InputStuffSpecNotFoundError as exc:- msg = (- f"Batch input item named '{sequential_sub_pipe.batch_params.input_item_stuff_name}' is not "- f"in this PipeSequence '{self.code}' input requirements: {sub_pipe_needed_inputs.format_for_display()}"- )- raise PipeSequenceValueError(msg) from exc- is_dotted_path = "." in sequential_sub_pipe.batch_params.input_list_stuff_name- needed_inputs.add_stuff_spec(- variable_name=input_list_root,- concept=stuff_spec.concept,- multiplicity=True if not is_dotted_path else None,- )- for input_name, stuff_spec in sub_pipe_needed_inputs.items:- if input_name != sequential_sub_pipe.batch_params.input_item_stuff_name and input_name not in generated_outputs:- needed_inputs.add_stuff_spec(- variable_name=input_name,- concept=stuff_spec.concept,- multiplicity=stuff_spec.multiplicity,- presence=stuff_spec.presence,- )- else:- for input_name, stuff_spec in sub_pipe_needed_inputs.items:- if input_name not in generated_outputs:- needed_inputs.add_stuff_spec(- variable_name=input_name, concept=stuff_spec.concept, multiplicity=stuff_spec.multiplicity, presence=stuff_spec.presence- )-- # Add this step's output to generated outputs- if sequential_sub_pipe.output_name:- generated_outputs.add(sequential_sub_pipe.output_name)-+ needed_inputs, _ = self._walk_needed_inputs(visited_pipes=visited_pipes)return needed_inputs@overridedef pipe_dependencies(self) -> set[str]:- return {sub_pipe.pipe_code for sub_pipe in self.sequential_sub_pipes}+ return {sub_pipe.pipe_code for sub_pipe in self.pipe_steps}@overrideasync def _live_run_controller_pipe(@@ -383,10 +748,32 @@ class PipeSequence(PipeController):library_crate: "LibraryCrate | None" = None,) -> PipeOutput:evolving_memory = working_memory+ typed_flow: SequenceTypedFlow | None = None+ # By step index, each binding the run derived from the value its root held, the flow having no spec for that root.+ runtime_bindings: dict[int, DerivedBinding] = {}for sub_pipe_index, sub_pipe in enumerate(self.sequential_sub_pipes):+ is_last_step = sub_pipe_index == len(self.sequential_sub_pipes) - 1+ if isinstance(sub_pipe, BindingStep):+ if typed_flow is None:+ typed_flow = self.build_typed_flow()+ self._run_binding_step(+ binding_step=sub_pipe,+ step_index=sub_pipe_index,+ typed_flow=typed_flow,+ runtime_bindings=runtime_bindings,+ working_memory=evolving_memory,+ job_metadata=job_metadata,+ pipe_run_params=pipe_run_params,+ is_last_step=is_last_step,+ )+ continue+ if runtime_bindings and typed_flow is not None:+ self._refuse_runtime_consumer_mismatch(+ step_index=sub_pipe_index, step=sub_pipe, typed_flow=typed_flow, runtime_bindings=runtime_bindings+ )# Only the last step should apply the final_stuff_code- if sub_pipe_index == len(self.sequential_sub_pipes) - 1:+ if is_last_step:sub_pipe_run_params = pipe_run_params.model_copy()else:sub_pipe_run_params = pipe_run_params.model_copy(update=({"final_stuff_code": None}))@@ -409,6 +796,148 @@ class PipeSequence(PipeController):pipeline_run_id=job_metadata.run_metadata.pipeline_run_id,)+ def _run_binding_step(+ self,+ *,+ binding_step: BindingStep,+ step_index: int,+ typed_flow: SequenceTypedFlow,+ runtime_bindings: dict[int, DerivedBinding],+ working_memory: WorkingMemory,+ job_metadata: JobMetadata,+ pipe_run_params: PipeRunParams,+ is_last_step: bool,+ ) -> None:+ """Bind one value into the working memory, as a node of the execution graph producing the stuff it binds.++ A binding whose root the flow cannot type, a value a pipe that did not resolve at validation stored, is derived from+ the value the run holds under it, recorded in `runtime_bindings`, and checked against the sequence's output when it+ ends the sequence; it is skipped when the root holds a recorded absence.++ Raises:+ BindingStepRunError: When the path cannot be walked from the concept of the value the root holds, or when the+ binding derived from it ends the sequence and contradicts its declared output.+ """+ derived_binding = self._derived_binding(+ binding_step=binding_step, step_index=step_index, typed_flow=typed_flow, working_memory=working_memory+ )+ if derived_binding is not None and step_index not in typed_flow.binding_specs:+ runtime_bindings[step_index] = derived_binding+ if is_last_step:+ self._refuse_runtime_output_mismatch(binding_step=binding_step, derived_binding=derived_binding)+ node_id = binding_step.trace_start(job_metadata=job_metadata, working_memory=working_memory, domain_code=self.domain_code)+ outcome: BindingOutcome | None = None+ try:+ step_outcome = (+ binding_step.skip_untyped_root(working_memory=working_memory, calling_pipe_code=self.code, run_mode=pipe_run_params.run_mode)+ if derived_binding is None+ else binding_step.bind(+ working_memory=working_memory,+ derivation=derived_binding.derivation,+ result_concept=derived_binding.stuff_spec.concept,+ calling_pipe_code=self.code,+ run_mode=pipe_run_params.run_mode,+ stuff_code=pipe_run_params.final_stuff_code if is_last_step else None,+ )+ )+ outcome = step_outcome+ binding_step.trace_end(job_metadata=job_metadata, node_id=node_id, outcome=step_outcome)+ finally:+ # The node is closed on the way out whatever stopped the binding, a content failing its own validation or a+ # cancellation included, so no binding node is left running. The step returned nothing only if it raised, and+ # the error it raised is the one propagating.+ if outcome is None:+ binding_step.trace_error(job_metadata=job_metadata, node_id=node_id, error=sys.exc_info()[1])++ def _derived_binding(+ self, *, binding_step: BindingStep, step_index: int, typed_flow: SequenceTypedFlow, working_memory: WorkingMemory+ ) -> DerivedBinding | None:+ """What the binding binds: from the flow, or from the value the run holds when the flow cannot type the root, `None`+ when the root holds no value either.+ """+ derivation = typed_flow.binding_derivations.get(step_index)+ binding_spec = typed_flow.binding_specs.get(step_index)+ if derivation is not None and binding_spec is not None:+ return DerivedBinding(derivation=derivation, stuff_spec=binding_spec)+ root_stuff = working_memory.get_optional_stuff(binding_step.root_name)+ if root_stuff is None:+ return None+ return self._derive_from_held_value(binding_step=binding_step, root_stuff=root_stuff)++ def _derive_from_held_value(self, *, binding_step: BindingStep, root_stuff: Stuff) -> DerivedBinding:+ """Derive a binding whose root the flow cannot type from the concept and the shape of the value the root holds.++ Raises:+ BindingStepRunError: When the path cannot be walked from that concept.+ """+ root_multiplicity: VariableMultiplicity | None = True if isinstance(root_stuff.content, ListContent) else None+ try:+ return derive_binding_spec(+ binding_step=binding_step,+ root_spec=StuffSpec(concept=root_stuff.concept, multiplicity=root_multiplicity),+ sequence_code=self.code,+ domain_code=self.domain_code,+ )+ except PipeValidationError as exc:+ msg = (+ f"{exc} The sequence's flow could not type '{binding_step.root_name}', which a pipe that did not resolve at "+ f"validation stores, so the binding was derived from the '{root_stuff.concept.concept_ref}' it holds."+ )+ raise BindingStepRunError(msg) from exc++ def _refuse_runtime_output_mismatch(self, *, binding_step: BindingStep, derived_binding: DerivedBinding) -> None:+ """Refuse a binding ending the sequence, derived when it ran, that contradicts the sequence's declared output, as+ validation refuses one it derives: its concept, its multiplicity, and a result that may be absent behind a plain output.++ Raises:+ BindingStepRunError: Naming the binding, what it binds and the declared output.+ """+ bound_spec = derived_binding.stuff_spec+ is_concept_ok = get_concept_library().is_compatible(tested_concept=bound_spec.concept, wanted_concept=self.output.concept)+ is_multiplicity_ok = is_multiplicity_compatible(source_multiplicity=bound_spec.multiplicity, target_multiplicity=self.output.multiplicity)+ is_presence_ok = not derived_binding.derivation.may_find_nothing or self.output.presence.is_optional+ if is_concept_ok and is_multiplicity_ok and is_presence_ok:+ return+ bound_ref = StuffSpec(+ concept=bound_spec.concept,+ multiplicity=bound_spec.multiplicity,+ presence=PresenceMarker.OPTIONAL if derived_binding.derivation.may_find_nothing else PresenceMarker.PLAIN,+ ).to_bundle_representation(relative_to_domain=self.domain_code)+ declared_ref = self.output.to_bundle_representation(relative_to_domain=self.domain_code)+ msg = (+ f"In pipe '{self.code}', the binding step {binding_step.as_written} ends the sequence, whose output is declared "+ f"'{declared_ref}', but it binds '{bound_ref}'. A pipe that did not resolve at validation stored '{binding_step.root_name}', "+ "so the binding was derived when it ran, from the value it found. Declare the sequence's output as what the binding binds, "+ "or bind a path that reaches the declared output."+ )+ raise BindingStepRunError(msg)++ def _refuse_runtime_consumer_mismatch(+ self, *, step_index: int, step: SubPipe, typed_flow: SequenceTypedFlow, runtime_bindings: dict[int, DerivedBinding]+ ) -> None:+ """Refuse a pipe step reading the result of a binding derived when it ran as a spec its pipe does not need, as+ validation refuses one reading a binding it derives.++ Raises:+ BindingStepRunError: Naming the step, the binding and both specs.+ """+ runtime_slots: dict[str, FlowSlot] = {}+ for name, slot in typed_flow.binding_slots_by_pipe_step.get(step_index, {}).items():+ if slot.binding_step_index is None or slot.binding_step_index not in runtime_bindings:+ continue+ runtime_spec = runtime_bindings[slot.binding_step_index].stuff_spec+ runtime_slots[name] = FlowSlot(stuff_spec=runtime_spec, binding_step_index=slot.binding_step_index)+ if not runtime_slots:+ return+ try:+ self._check_step_reading_bindings(step_index=step_index, step=step, binding_slots=runtime_slots)+ except PipeValidationError as exc:+ msg = (+ f"{exc} The binding was derived when it ran, from the value its root held, since a pipe that did not resolve at "+ "validation stored the root."+ )+ raise BindingStepRunError(msg) from exc+@overrideasync def _dry_run_controller_pipe(self,diff --git a/pipelex/pipe_controllers/sequence/pipe_sequence_blueprint.py b/pipelex/pipe_controllers/sequence/pipe_sequence_blueprint.pyindex b058e4a70..04061de42 100644--- a/pipelex/pipe_controllers/sequence/pipe_sequence_blueprint.py+++ b/pipelex/pipe_controllers/sequence/pipe_sequence_blueprint.py@@ -1,39 +1,79 @@-from typing import Literal+from typing import Annotated, Any, Literal, cast-from pydantic import field_validator+from pydantic import Discriminator, Tag, field_validatorfrom typing_extensions import override+from pipelex.pipe_controllers.binding.binding_step_blueprint import (+ BindingStepBlueprint,+ check_binding_step_shape,+ is_binding_step_dict,+ raw_step_mapping,+)from pipelex.pipe_controllers.sub_pipe_blueprint import SubPipeBlueprintfrom pipelex.pipe_machinery.pipe_blueprint import PipeBlueprint+_PIPE_STEP_TAG = "pipe_step"+_BINDING_STEP_TAG = "binding_step"+++def _sequence_step_kind(step: Any) -> str: # kw-only: ignore+ """Which shape a sequence step takes: a binding step when it carries `from`, a pipe step otherwise."""+ if isinstance(step, BindingStepBlueprint):+ return _BINDING_STEP_TAG+ raw_step = raw_step_mapping(raw_step=step)+ if raw_step is not None and is_binding_step_dict(raw_step=raw_step):+ return _BINDING_STEP_TAG+ return _PIPE_STEP_TAG+++# A step of a PipeSequence: a pipe step, which runs a pipe, or a binding step, which binds the value at a path+# in working memory to a new name. Each shape is a closed table, and a step carrying `from` is a binding step.+SequenceStepBlueprint = Annotated[+ Annotated[SubPipeBlueprint, Tag(_PIPE_STEP_TAG)] | Annotated[BindingStepBlueprint, Tag(_BINDING_STEP_TAG)],+ Discriminator(_sequence_step_kind),+]+class PipeSequenceBlueprint(PipeBlueprint):type: Literal["PipeSequence"] = "PipeSequence"pipe_category: Literal["PipeController"] = "PipeController"- steps: list[SubPipeBlueprint]+ steps: list[SequenceStepBlueprint]++ @property+ def pipe_steps(self) -> list[SubPipeBlueprint]:+ """The steps that run a pipe, in order; a binding step runs none."""+ return [step for step in self.steps if isinstance(step, SubPipeBlueprint)]@property@overridedef pipe_dependencies(self) -> set[str]:- """Return the set of pipe codes from the sequence steps."""- return {step.pipe for step in self.steps}+ """Return the set of pipe codes from the sequence's pipe steps."""+ return {step.pipe for step in self.pipe_steps}@property@overridedef ordered_pipe_dependencies(self) -> list[str]:- """Return the ordered list of pipe codes from the sequence steps.+ """Return the ordered list of pipe codes from the sequence's pipe steps.For sequences, the order of steps matters, so we preserve it."""- return [step.pipe for step in self.steps]+ return [step.pipe for step in self.pipe_steps]@field_validator("steps", mode="before")@classmethod- def validate_steps(cls, steps: list[SubPipeBlueprint]) -> list[SubPipeBlueprint]:- if not steps:+ def validate_steps(cls, steps: Any) -> Any:+ if not isinstance(steps, list):+ # Not a list at all: pydantic's own `list_type` error names the field and what it holds.+ return steps+ step_list = cast("list[Any]", steps)+ if not step_list:msg = "PipeSequence must have at least 1 step"raise ValueError(msg)- return steps+ for step_index, step in enumerate(step_list):+ raw_step = raw_step_mapping(raw_step=step)+ if raw_step is not None:+ check_binding_step_shape(raw_step=raw_step, step_label=f"Step {step_index + 1} of the sequence")+ return step_list@overridedef validate_inputs(self):diff --git a/pipelex/pipe_controllers/sequence/pipe_sequence_factory.py b/pipelex/pipe_controllers/sequence/pipe_sequence_factory.pyindex 458e078b8..2d79aa2e4 100644--- a/pipelex/pipe_controllers/sequence/pipe_sequence_factory.py+++ b/pipelex/pipe_controllers/sequence/pipe_sequence_factory.py@@ -4,13 +4,15 @@ from typing_extensions import overridefrom pipelex.core.pipes.inputs.input_stuff_specs import InputStuffSpecsfrom pipelex.core.pipes.stuff_spec.stuff_spec import StuffSpec+from pipelex.pipe_controllers.binding.binding_step import BindingStep+from pipelex.pipe_controllers.binding.binding_step_blueprint import BindingStepBlueprintfrom pipelex.pipe_controllers.sequence.pipe_sequence import PipeSequencefrom pipelex.pipe_controllers.sequence.pipe_sequence_blueprint import PipeSequenceBlueprintfrom pipelex.pipe_controllers.sub_pipe_factory import SubPipeFactoryfrom pipelex.pipe_machinery.pipe_factory import PipeFactoryProtocolif TYPE_CHECKING:- from pipelex.pipe_controllers.sub_pipe import SubPipe+ from pipelex.pipe_controllers.sequence.sequence_typed_flow import SequenceStepclass PipeSequenceFactory(PipeFactoryProtocol[PipeSequenceBlueprint, PipeSequence]):@@ -28,11 +30,13 @@ class PipeSequenceFactory(PipeFactoryProtocol[PipeSequenceBlueprint, PipeSequencoutput: StuffSpec,blueprint: PipeSequenceBlueprint,) -> PipeSequence:- sequential_sub_pipes: list[SubPipe] = []+ sequential_sub_pipes: list[SequenceStep] = []for step in blueprint.steps:- sub_pipe = SubPipeFactory.make_from_blueprint(blueprint=step)- sequential_sub_pipes.append(sub_pipe)+ if isinstance(step, BindingStepBlueprint):+ sequential_sub_pipes.append(BindingStep(from_path=step.from_path, output_name=step.result))+ else:+ sequential_sub_pipes.append(SubPipeFactory.make_from_blueprint(blueprint=step))return PipeSequence(domain_code=domain_code,diff --git a/pipelex/pipe_controllers/sequence/sequence_typed_flow.py b/pipelex/pipe_controllers/sequence/sequence_typed_flow.pynew file mode 100644index 000000000..b62d5253e--- /dev/null+++ b/pipelex/pipe_controllers/sequence/sequence_typed_flow.py@@ -0,0 +1,282 @@+"""The typed flow of a PipeSequence: what each name in working memory holds, step by step, as a stuff spec.++The flow is built in step order. The sequence's declared inputs seed it; a pipe step's `result` takes its+pipe's output spec, made plural by `batch_over`, `nb_output` or `multiple_output`; a binding step contributes+the spec it derives. A pipe step also contributes what its pipe stores on the memory it runs on besides its+result (`step_memory_writes`): a nested sequence's steps and a condition's outcome run on the caller's+memory, and a PipeParallel with `add_each_output` stores each branch's result there. A name a later step+always stores replaces what was there, so a binding's root is typed by the latest value stored under it; a+name a step stores on some runs only, as a condition whose `continue` outcome stores nothing, keeps its spec+when the value stored has the same one, and is untyped otherwise.++A binding's root is untyped in two ways. The values it may hold have different specs, as the outcomes of a+condition storing it under different concepts: that is seen before the run (`SpecDisagreement`), so the+binding is refused as `binding_path_unresolved`. Or a pipe that does not resolve at validation, a dependency+not loaded yet, stored it: nothing can type it before the run, so the binding is assumed to deliver and the+run derives it from the value it holds, checking it there against what reads it.++`step_memory_writes` is the one place a step's stores beside its result are computed: the sequence's needed+inputs and its absence-taint walk read them from it too, so the three analyses agree on every name.++It is used in three places, and only these: a binding types its root from the flow, a step reading a+binding's result is checked against the spec the binding derives, and a binding ending the sequence is+checked against the sequence's output. Extending the check to every step's inputs is a separate change.+"""++from typing import NamedTuple++from pydantic import BaseModel, ConfigDict++from pipelex.core.concepts.concept import Concept+from pipelex.core.pipes.exceptions import PipeValidationError+from pipelex.core.pipes.inputs.input_stuff_specs import InputStuffSpecs+from pipelex.core.pipes.stuff_spec.stuff_spec import StuffSpec+from pipelex.interpreter_hub import get_concept_library, get_optional_pipe+from pipelex.libraries.concept.concept_library_abstract import ConceptLibraryAbstract+from pipelex.libraries.concept.exceptions import ConceptLibraryError+from pipelex.pipe_controllers.binding.binding_concept_resolvers import LibraryConceptWalkResolver, library_concept_key+from pipelex.pipe_controllers.binding.binding_derivation import BindingDerivation, BindingRoot, derive_binding+from pipelex.pipe_controllers.binding.binding_step import BindingStep+from pipelex.pipe_controllers.binding.exceptions import BindingPathUnresolvedError+from pipelex.pipe_controllers.sub_pipe import SubPipe+from pipelex.pipe_machinery.memory_writes import MemoryWrite, SpecDisagreement, StoredSpec, is_same_value_spec+from pipelex.pipe_machinery.pipe_abstract import PipeAbstract+from pipelex.validation_error_types import PipeValidationErrorType++# A step of a PipeSequence, as the runtime holds it: a pipe step or a binding step.+SequenceStep = SubPipe | BindingStep+++class FlowSlot(BaseModel):+ """What a name holds at a point of the flow, and which binding step stored it, if one did."""++ model_config = ConfigDict(frozen=True)++ # `None` when the flow cannot type the value: a pipe that does not resolve (an unloaded dependency) stored it, or the+ # values it may hold have different specs, which `disagreement` then says.+ stuff_spec: StuffSpec | None+ binding_step_index: int | None = None+ # Set only when `stuff_spec` is `None` because the values the name may hold have different specs, seen before the run.+ disagreement: SpecDisagreement | None = None+++class SequenceTypedFlow(BaseModel):+ """The typed flow of one sequence: the derivation of each binding step, and what each pipe step can read."""++ model_config = ConfigDict(frozen=True)++ # By step index, for each binding step whose root the flow types.+ binding_derivations: dict[int, BindingDerivation]+ # By step index, the derived spec of each binding step whose root the flow types.+ binding_specs: dict[int, StuffSpec]+ # By pipe-step index, the slots a binding step stored that are still visible when the step runs.+ binding_slots_by_pipe_step: dict[int, dict[str, FlowSlot]]+ # The flow after the last step.+ final_slots: dict[str, FlowSlot]+ # The slots the steps stored, as they stand after the last step: what the sequence leaves in the memory it runs on.+ written_slots: dict[str, FlowSlot]+ # The names some step stores on every run, among the written slots.+ always_written_names: frozenset[str]+ # By pipe-step index, what the step stores besides its result (`step_memory_writes`), for the walks that follow the flow.+ memory_writes_by_pipe_step: dict[int, dict[str, MemoryWrite]]+++class DerivedBinding(NamedTuple):+ """What a binding step binds: the derivation of its path, and the spec of the value it stores."""++ derivation: BindingDerivation+ stuff_spec: StuffSpec+++def resolve_library_concept(*, concept_library: ConceptLibraryAbstract, concept_ref: str) -> Concept | None:+ """The library's concept for a ref, looked up directly, then among a dependency's aliased keys when one alone matches."""+ try:+ return concept_library.get_required_concept(concept_ref=concept_ref)+ except ConceptLibraryError:+ candidate_keys = concept_library.list_concept_keys_for_ref(concept_ref=concept_ref)+ if len(candidate_keys) != 1:+ return None+ return concept_library.get_required_concept(concept_ref=candidate_keys[0])+++def binding_path_unresolved_error(+ *, sequence_code: str, domain_code: str, binding_step: BindingStep, exc: BindingPathUnresolvedError+) -> PipeValidationError:+ msg = f"In pipe '{sequence_code}', the binding step {binding_step.as_written} cannot be derived. {exc}"+ return PipeValidationError(+ message=msg,+ error_type=PipeValidationErrorType.BINDING_PATH_UNRESOLVED,+ domain_code=domain_code,+ pipe_code=sequence_code,+ variable_names=[binding_step.from_path],+ )+++def derive_binding_spec(*, binding_step: BindingStep, root_spec: StuffSpec, sequence_code: str, domain_code: str) -> DerivedBinding:+ """Derive what a binding step binds from the spec of its root.++ Raises:+ PipeValidationError: ``BINDING_PATH_UNRESOLVED`` when the path cannot be walked from the root's concept, or when the+ concept it derives is not in the library.+ """+ concept_library = get_concept_library()+ try:+ # The root is walked from the key the library holds its concept under, which for a dependency package's+ # concept names the package, so the package's own definitions are read and never a host's of the same spelling.+ root_concept_key = library_concept_key(concept_library=concept_library, concept=root_spec.concept)+ derivation = derive_binding(+ path=binding_step.from_path,+ root=BindingRoot(concept_ref=root_concept_key, multiplicity=root_spec.multiplicity),+ resolver=LibraryConceptWalkResolver(concept_library=concept_library),+ )+ except BindingPathUnresolvedError as exc:+ raise binding_path_unresolved_error(sequence_code=sequence_code, domain_code=domain_code, binding_step=binding_step, exc=exc) from exc+ result_concept: Concept | None+ if derivation.is_bare_name:+ result_concept = root_spec.concept+ else:+ result_concept = resolve_library_concept(concept_library=concept_library, concept_ref=derivation.concept_ref)+ if result_concept is None:+ missing_exc = BindingPathUnresolvedError(+ f"Cannot bind '{binding_step.from_path}': the concept it derives, '{derivation.concept_ref}', is not in the library.",+ path=binding_step.from_path,+ failed_segment=binding_step.from_path.rsplit(".", maxsplit=1)[-1],+ available_fields=[],+ )+ raise binding_path_unresolved_error(sequence_code=sequence_code, domain_code=domain_code, binding_step=binding_step, exc=missing_exc)+ return DerivedBinding(derivation=derivation, stuff_spec=StuffSpec(concept=result_concept, multiplicity=derivation.multiplicity))+++def step_memory_writes(*, step: SubPipe, step_pipe: PipeAbstract, visited_pipes: set[str]) -> dict[str, MemoryWrite]:+ """What a pipe step stores in the sequence's memory besides its result, as the typed flow, the needed inputs and the+ absence-taint walk of the sequence all read it.++ The step's pipe stores its `memory_writes`, unless the step batches: a batched step runs its pipe on a copy of the+ memory for each item, so only its result comes back.+ """+ if step.batch_params is not None:+ return {}+ return step_pipe.memory_writes(visited_pipes=visited_pipes)+++def slot_after_write(*, prior_slot: FlowSlot | None, memory_write: MemoryWrite, step_label: str) -> FlowSlot:+ """What a name holds once a step stored it: the stored spec, or, when the step may leave the name as it was, the spec+ the stored value and the value already there agree on, untyped when they do not.++ A name left untyped records why when the two specs are known and differ, or when either side carries a disagreement+ already; a side nothing could type, a pipe that does not resolve having stored it, leaves it untyped with no disagreement.+ `step_label` names the step for that record, e.g. "step 2 (pipe 'swap_record')".+ """+ if memory_write.is_always_written or prior_slot is None:+ return FlowSlot(stuff_spec=memory_write.stuff_spec, disagreement=memory_write.disagreement)+ if is_same_value_spec(first_spec=prior_slot.stuff_spec, second_spec=memory_write.stuff_spec):+ return prior_slot+ disagreement: SpecDisagreement | None+ if prior_slot.stuff_spec is not None and memory_write.stuff_spec is not None:+ disagreement = SpecDisagreement(+ stored_specs=(+ StoredSpec(+ stored_by=f"the value it held before {step_label}, which the step leaves when it stores nothing,",+ stuff_spec=prior_slot.stuff_spec,+ ),+ StoredSpec(stored_by=f"{step_label}, when it stores a value,", stuff_spec=memory_write.stuff_spec),+ )+ )+ else:+ disagreement = prior_slot.disagreement or memory_write.disagreement+ return FlowSlot(stuff_spec=None, disagreement=disagreement)+++def untyped_root_error(*, sequence_code: str, domain_code: str, binding_step: BindingStep, disagreement: SpecDisagreement) -> PipeValidationError:+ """The refusal of a binding whose root's values have different specs, so that its concept is not known before the run."""+ root_name = binding_step.root_name+ msg = (+ f"Cannot bind '{binding_step.from_path}': the concept of '{root_name}' is not known before the run, because the outcomes that may "+ f"store it store it under different concepts: {disagreement.describe(relative_to_domain=domain_code)}. Store '{root_name}' under "+ "one concept in every outcome, or bind inside each outcome, where its concept is known."+ )+ exc = BindingPathUnresolvedError(msg, path=binding_step.from_path, failed_segment=root_name, available_fields=[])+ return binding_path_unresolved_error(sequence_code=sequence_code, domain_code=domain_code, binding_step=binding_step, exc=exc)+++def build_sequence_typed_flow(+ *,+ steps: list[SequenceStep],+ declared_inputs: InputStuffSpecs,+ sequence_code: str,+ domain_code: str,+ visited_pipes: set[str],+) -> SequenceTypedFlow:+ """Build the typed flow of a sequence's steps, deriving every binding whose root it types.++ Args:+ steps: The sequence's steps, in order.+ declared_inputs: The sequence's declared inputs, which seed the flow.+ sequence_code: The sequence's code, for messages.+ domain_code: The sequence's domain, for messages.+ visited_pipes: The `visit_key` of each pipe whose flow is being built, the sequence's own included, so a pipe+ reached again through its own steps contributes nothing more.++ Raises:+ PipeValidationError: ``BINDING_PATH_UNRESOLVED`` when a binding's path cannot be walked from its root's concept,+ when the concept it derives is not in the library, or when the values its root may hold have different specs.+ """+ slots: dict[str, FlowSlot] = {name: FlowSlot(stuff_spec=stuff_spec) for name, stuff_spec in declared_inputs.root.items()}+ written_names: set[str] = set()+ always_written_names: set[str] = set()+ binding_derivations: dict[int, BindingDerivation] = {}+ binding_specs: dict[int, StuffSpec] = {}+ binding_slots_by_pipe_step: dict[int, dict[str, FlowSlot]] = {}+ memory_writes_by_pipe_step: dict[int, dict[str, MemoryWrite]] = {}++ for step_index, step in enumerate(steps):+ if isinstance(step, BindingStep):+ written_names.add(step.output_name)+ always_written_names.add(step.output_name)+ root_slot = slots.get(step.root_name)+ if root_slot is not None and root_slot.disagreement is not None:+ raise untyped_root_error(sequence_code=sequence_code, domain_code=domain_code, binding_step=step, disagreement=root_slot.disagreement)+ if root_slot is None or root_slot.stuff_spec is None:+ # A root nothing types: an undeclared one, refused as a missing input of the sequence; or a value a pipe that+ # does not resolve stored, assumed to deliver as the rest of the sequence's checks assume, which the run+ # derives from the value it holds and checks against what reads it.+ slots[step.output_name] = FlowSlot(stuff_spec=None, binding_step_index=step_index)+ continue+ derived_binding = derive_binding_spec(+ binding_step=step, root_spec=root_slot.stuff_spec, sequence_code=sequence_code, domain_code=domain_code+ )+ binding_derivations[step_index] = derived_binding.derivation+ binding_specs[step_index] = derived_binding.stuff_spec+ slots[step.output_name] = FlowSlot(stuff_spec=derived_binding.stuff_spec, binding_step_index=step_index)+ continue++ binding_slots_by_pipe_step[step_index] = {name: slot for name, slot in slots.items() if slot.binding_step_index is not None}+ step_pipe = get_optional_pipe(pipe_code=step.pipe_code)+ if step_pipe is None:+ if step.output_name:+ written_names.add(step.output_name)+ always_written_names.add(step.output_name)+ slots[step.output_name] = FlowSlot(stuff_spec=None)+ continue+ step_writes = step_memory_writes(step=step, step_pipe=step_pipe, visited_pipes=visited_pipes)+ memory_writes_by_pipe_step[step_index] = step_writes+ step_label = f"step {step_index + 1} (pipe '{step_pipe.code}')"+ for written_name, memory_write in step_writes.items():+ written_names.add(written_name)+ if memory_write.is_always_written:+ always_written_names.add(written_name)+ slots[written_name] = slot_after_write(prior_slot=slots.get(written_name), memory_write=memory_write, step_label=step_label)+ if step.output_name:+ written_names.add(step.output_name)+ always_written_names.add(step.output_name)+ slots[step.output_name] = FlowSlot(stuff_spec=step.result_spec(step_pipe=step_pipe))++ return SequenceTypedFlow(+ binding_derivations=binding_derivations,+ binding_specs=binding_specs,+ binding_slots_by_pipe_step=binding_slots_by_pipe_step,+ final_slots=slots,+ written_slots={name: slot for name, slot in slots.items() if name in written_names},+ always_written_names=frozenset(always_written_names),+ memory_writes_by_pipe_step=memory_writes_by_pipe_step,+ )diff --git a/pipelex/pipe_controllers/sub_pipe.py b/pipelex/pipe_controllers/sub_pipe.pyindex a3b36ff40..2aed0bb91 100644--- a/pipelex/pipe_controllers/sub_pipe.py+++ b/pipelex/pipe_controllers/sub_pipe.py@@ -7,7 +7,8 @@ from pipelex.core.memory.exceptions import WorkingMemoryStuffNotFoundErrorfrom pipelex.core.memory.working_memory import WorkingMemoryfrom pipelex.core.pipes.inputs.exceptions import InputStuffSpecNotFoundError, PipeRunInputsErrorfrom pipelex.core.pipes.pipe_output import PipeOutput-from pipelex.core.pipes.variable_multiplicity import VariableMultiplicity+from pipelex.core.pipes.stuff_spec.stuff_spec import StuffSpec+from pipelex.core.pipes.variable_multiplicity import PresenceMarker, VariableMultiplicityfrom pipelex.core.stuffs.list_content import ListContentfrom pipelex.core.stuffs.stuff_content import StuffContentfrom pipelex.core.stuffs.stuff_factory import StuffFactory@@ -15,9 +16,10 @@ from pipelex.interpreter_hub import get_pipe_router, get_required_pipefrom pipelex.pipe_controllers.batch.pipe_batch import PipeBatchfrom pipelex.pipe_controllers.batch.pipe_batch_blueprint import PipeBatchBlueprintfrom pipelex.pipe_controllers.condition.pipe_condition import PipeCondition+from pipelex.pipe_machinery.pipe_abstract import PipeAbstractfrom pipelex.pipe_machinery.pipe_factory import PipeFactoryfrom pipelex.pipe_run.pipe_job_factory import PipeJobFactory-from pipelex.pipe_run.pipe_run_params import BatchParams, PipeRunParams+from pipelex.pipe_run.pipe_run_params import BatchParams, PipeRunParams, output_multiplicity_to_applyfrom pipelex.system.job_metadata import JobMetadatafrom pipelex.tools.misc.string_utils import get_root_from_dotted_path@@ -31,6 +33,30 @@ class SubPipe(BaseModel):output_multiplicity: VariableMultiplicity | None = Nonebatch_params: BatchParams | None = None+ def result_spec(self, *, step_pipe: PipeAbstract) -> StuffSpec:+ """The spec of what the step stores under its result, `step_pipe` being the pipe it runs, resolved as the run path resolves it.++ A batched step stores the list of its branches' results, a `PipeBatch` wrapping the pipe: `X[]` whatever the pipe+ outputs and whatever count the step asks for, since an absent branch result is dropped from the list, which is never+ absent. Any other step stores the pipe's output, its multiplicity overridden by the step's `nb_output` or+ `multiple_output`, and a single result keeps the pipe's presence.+ """+ if self.batch_params is not None:+ return StuffSpec(concept=step_pipe.output.concept, multiplicity=True)+ multiplicity_resolution = output_multiplicity_to_apply(+ base_multiplicity=step_pipe.output.multiplicity,+ override_multiplicity=self.output_multiplicity,+ )+ multiplicity: VariableMultiplicity | None+ if not multiplicity_resolution.is_multiple_outputs_enabled:+ multiplicity = None+ elif multiplicity_resolution.specific_output_count is not None:+ multiplicity = multiplicity_resolution.specific_output_count+ else:+ multiplicity = True+ presence = step_pipe.output.presence if multiplicity is None else PresenceMarker.PLAIN+ return StuffSpec(concept=step_pipe.output.concept, multiplicity=multiplicity, presence=presence)+async def run_pipe(self,*,diff --git a/pipelex/pipe_machinery/memory_writes.py b/pipelex/pipe_machinery/memory_writes.pynew file mode 100644index 000000000..2d4713510--- /dev/null+++ b/pipelex/pipe_machinery/memory_writes.py@@ -0,0 +1,154 @@+"""What a pipe stores in the working memory it runs on besides its own result, as the static analyses of its caller read it.++A PipeSequence runs its steps on its caller's memory, a PipeCondition runs its chosen outcome there, and a PipeParallel with+`add_each_output` adds each branch's result to it, so each stores names there that its caller's step never named. A calling+PipeSequence has three static analyses, and all three read those names from this one model, `MemoryWrite`, computed by the+pipe's `memory_writes`: the typed flow takes the spec a name holds, the needed inputs count a name always written as stored,+and the absence-taint walk takes why the name may hold an absence. A batched step and a PipeBatch run their pipe on copies+of the memory, so they store nothing beyond their result.++`SlotTaint` lives here, beside the model that carries it, so that `PipeAbstract` can declare `memory_writes`; the rest of+the absence-taint pass is in `pipelex.pipe_controllers.absence_taint`.+"""++from typing import NamedTuple++from pydantic import BaseModel, ConfigDict+from pydantic.dataclasses import dataclass++from pipelex.core.pipes.stuff_spec.stuff_spec import StuffSpec+++@dataclass(frozen=True)+class SlotTaint:+ """A maybe-absent slot: where the absence originates and how it propagated here."""++ source: str+ origin_slot_name: str+ chain: tuple[str, ...] = ()++ def describe(self) -> str:+ description = f"Absence origin: {self.source}."+ if self.chain:+ description += f" Propagation: {' → '.join(self.chain)}."+ return description+++class StoredSpec(BaseModel):+ """One of the values a name may hold once a step returns, and what stores it, as a message names it."""++ model_config = ConfigDict(frozen=True)++ # What stores the value, written to precede "as 'X'", e.g. "outcome 'stash_parcel' of pipe 'stash_by_mode'".+ stored_by: str+ stuff_spec: StuffSpec+++class SpecDisagreement(BaseModel):+ """Why a name has no spec before the run although every value it may hold has one: the values have different specs.++ That is a condition whose outcomes store the name under different specs, or one whose outcomes store a value of another+ spec than the one an outcome storing nothing leaves. Unlike a value a pipe that does not resolve stores, which nothing+ can type before the run, the disagreement is seen at validation, so a binding reading the name is refused there.+ """++ model_config = ConfigDict(frozen=True)++ stored_specs: tuple[StoredSpec, ...]++ def describe(self, *, relative_to_domain: str) -> str:+ """Each value and what stores it, e.g. "outcome 'stash_crate' of pipe 'stash_by_mode' as 'Crate'; outcome ... as 'Parcel'"."""+ return "; ".join(+ f"{stored_spec.stored_by} as '{stored_spec.stuff_spec.to_bundle_representation(relative_to_domain=relative_to_domain)}'"+ for stored_spec in self.stored_specs+ )+++class MemoryWrite(BaseModel):+ """What a pipe stores under one name of the memory it runs on, besides its result."""++ model_config = ConfigDict(frozen=True)++ # The spec of the value stored, `None` when it cannot be typed: a pipe that does not resolve stored it, or the values the+ # name may hold have different specs, which `disagreement` then says.+ stuff_spec: StuffSpec | None+ # Set only when `stuff_spec` is `None` because the values the name may hold have different specs.+ disagreement: SpecDisagreement | None = None+ # Why the name may hold an absence once the pipe returns, `None` when it always holds a value; a list never holds one.+ absence: SlotTaint | None = None+ # Whether every run of the pipe that returns stores the name. A run that does not, a condition's outcome that stores+ # nothing under it, its `continue` outcome included, leaves the caller's value in place. A pipe skipped for an absent+ # plain input still stores what it always stores, as an absence or an empty list (`PipeAbstract.lifted_companion_slots`).+ is_always_written: bool = True+++def is_same_value_spec(*, first_spec: StuffSpec | None, second_spec: StuffSpec | None) -> bool:+ """Whether two specs type the same value: the same concept and multiplicity. Presence aside, which an absence carries."""+ if first_spec is None or second_spec is None:+ return False+ return first_spec.concept == second_spec.concept and first_spec.multiplicity == second_spec.multiplicity+++class AlternativeWrites(NamedTuple):+ """What one alternative stores, and how a message names it, e.g. "outcome 'stash_parcel' of pipe 'stash_by_mode'"."""++ label: str+ writes: dict[str, MemoryWrite]+++def merge_alternative_writes(*, alternatives: list[AlternativeWrites]) -> dict[str, MemoryWrite]:+ """What a pipe stores when exactly one of several alternatives runs, as a condition runs one of its outcomes.++ A name is always written only if every alternative always writes it. It may hold an absence if any alternative may leave+ one. It keeps a spec only if every alternative storing it stores the same value spec; an alternative that does not store+ it leaves the caller's value, which the caller merges with what the name held before (`is_always_written` is then false).+ A name left untyped records why when it is seen before the run (`SpecDisagreement`): the alternatives typing it store it+ under different specs, or one of them stores it with a disagreement of its own. Otherwise an alternative that could not+ type it, a pipe that does not resolve having stored it, leaves it untyped with no disagreement.+ """+ merged_writes: dict[str, MemoryWrite] = {}+ for alternative in alternatives:+ for written_name in alternative.writes:+ if written_name in merged_writes:+ continue+ name_writes = [other.writes.get(written_name) for other in alternatives]+ labeled_writes = [(other.label, other.writes[written_name]) for other in alternatives if written_name in other.writes]+ stored_writes = [name_write for _, name_write in labeled_writes]+ first_spec = stored_writes[0].stuff_spec+ is_typed = all(is_same_value_spec(first_spec=first_spec, second_spec=name_write.stuff_spec) for name_write in stored_writes)+ absences = [name_write.absence for name_write in stored_writes if name_write.absence is not None]+ merged_writes[written_name] = MemoryWrite(+ stuff_spec=first_spec if is_typed else None,+ disagreement=None if is_typed else _alternatives_disagreement(labeled_writes=labeled_writes),+ absence=absences[0] if absences else None,+ is_always_written=all(name_write is not None and name_write.is_always_written for name_write in name_writes),+ )+ return merged_writes+++def _alternatives_disagreement(*, labeled_writes: list[tuple[str, MemoryWrite]]) -> SpecDisagreement | None:+ """Why the alternatives storing a name leave it untyped, when it is seen before the run: the specs they store disagree, or+ one of them stores the name with a disagreement of its own. `None` when an alternative could not type the name at all.+ """+ typed_specs: list[StoredSpec] = []+ for label, name_write in labeled_writes:+ if name_write.stuff_spec is not None:+ typed_specs.append(StoredSpec(stored_by=label, stuff_spec=name_write.stuff_spec))+ first_typed_spec = typed_specs[0].stuff_spec if typed_specs else None+ if any(not is_same_value_spec(first_spec=first_typed_spec, second_spec=typed_spec.stuff_spec) for typed_spec in typed_specs):+ return SpecDisagreement(stored_specs=tuple(typed_specs))+ for _, name_write in labeled_writes:+ if name_write.disagreement is not None:+ return name_write.disagreement+ return None+++def taint_after_write(*, prior_taint: SlotTaint | None, memory_write: MemoryWrite) -> SlotTaint | None:+ """Why a name may hold an absence once a pipe stored it: the write's own reason, or, when the pipe may leave the name as+ it was, the reason it had before.+ """+ if memory_write.absence is not None:+ return memory_write.absence+ if memory_write.is_always_written:+ return None+ return prior_taintdiff --git a/pipelex/pipe_machinery/pipe_abstract.py b/pipelex/pipe_machinery/pipe_abstract.pyindex 321d8218c..85927181c 100644--- a/pipelex/pipe_machinery/pipe_abstract.py+++ b/pipelex/pipe_machinery/pipe_abstract.py@@ -26,6 +26,7 @@ from pipelex.core.stuffs.stuff_content import StuffContentfrom pipelex.graph.graph_tracer_manager import GraphTracerManager, IOSpec, NodeKindfrom pipelex.graph.stuff_io_spec import make_stuff_io_specfrom pipelex.libraries.library_crate import LibraryCrate+from pipelex.pipe_machinery.memory_writes import MemoryWritefrom pipelex.pipe_machinery.pipe_blueprint import PipeCategory, PipeType, valid_pipe_type_tagsfrom pipelex.pipe_run.pipe_run_params import PipeRunParams, output_multiplicity_to_applyfrom pipelex.pipe_signature.exceptions import PipeSignatureNotExecutableError@@ -70,7 +71,8 @@ class CompanionSlot(NamedTuple):"""slot_name: str- concept: Concept+ # The concept of the empty list a plural slot resolves to; `None` for a slot whose spec is unknown, which is never plural.+ concept: Concept | Noneis_plural: boolproducing_pipe_code: str@@ -328,6 +330,13 @@ class PipeAbstract(ABC, BaseModel):for var_name, declared_stuff_spec in self.inputs.root.items()}+ def refuse_undeclared_needed_input(self, *, variable_name: str) -> None:+ """Raise a pipe's own refusal of a needed input it does not declare, before the generic one; nothing by default.++ A controller whose need is better named than "required variable" overrides it: a PipeSequence asks, for a+ binding step's root, for the concept whose structure holds the path the binding walks.+ """+@finaldef generic_validate_inputs_with_library(self):# First validate required variables are in the inputs: a variable path reads the input its root names@@ -354,6 +363,7 @@ class PipeAbstract(ABC, BaseModel):var_name = named_stuff_spec.variable_nameif var_name not in self.inputs.variables:+ self.refuse_undeclared_needed_input(variable_name=var_name)msg = f"Required variable '{var_name}' is not in the inputs of pipe '{self.code}'. Current inputs: {self.inputs.format_for_display()}"raise PipeValidationError(message=msg,@@ -613,6 +623,19 @@ class PipeAbstract(ABC, BaseModel):"""+ def memory_writes(self, *, visited_pipes: set[str] | None = None) -> dict[str, MemoryWrite]: # ruff: ignore[unused-method-argument] (the overrides recurse)+ """What the pipe stores in the working memory it runs on, besides its own result, which its caller names.++ Each name maps to what it holds once the pipe returns: its spec, why it may hold an absence, and whether every run+ stores it (`MemoryWrite`). An operator stores only its result, so it stores nothing more. A controller that runs its+ steps or its outcome on its caller's memory, rather than on copies, overrides this, and its calling sequence's+ typed flow, needed inputs and absence-taint walk all read what it stores from here.++ Args:+ visited_pipes: The `visit_key` of each pipe currently being processed, to prevent infinite recursion.+ """+ return {}+def _format_pipe_run_info(self, pipe_run_params: PipeRunParams) -> str:indent_level = len(pipe_run_params.pipe_stack) - 1indent = " " * indent_level@@ -854,9 +877,26 @@ class PipeAbstract(ABC, BaseModel):def lifted_companion_slots(self) -> list[CompanionSlot]:"""Extra slots this pipe would have written besides its main output, to resolve when it- is lifted. Default: none; an `add_each_output` PipeParallel reports its branch slots.+ is lifted: every name its `memory_writes` always stores, so a name its caller's analyses+ count as stored is never left holding neither a value nor an absence. A name only some+ runs store is left as the caller had it, which those analyses already allow for. An+ operator stores nothing more; an `add_each_output` PipeParallel reports its branch slots+ with the branch pipe that produces each."""- return []+ companion_slots: list[CompanionSlot] = []+ for slot_name, memory_write in self.memory_writes().items():+ if not memory_write.is_always_written:+ continue+ stuff_spec = memory_write.stuff_spec+ companion_slots.append(+ CompanionSlot(+ slot_name=slot_name,+ concept=stuff_spec.concept if stuff_spec is not None else None,+ is_plural=stuff_spec is not None and stuff_spec.is_multiple(),+ producing_pipe_code=self.code,+ )+ )+ return companion_slots@classmethoddef _make_skip_reason(cls, *, liftable: list[AbsentInput]) -> str:@@ -919,7 +959,7 @@ class PipeAbstract(ABC, BaseModel):producing_pipe=companion_slot.producing_pipe_code,upstream=lifted_input.absence_record,)- if companion_slot.is_plural:+ if companion_slot.is_plural and companion_slot.concept is not None:empty_companion_stuff = Stuff(concept=companion_slot.concept,content=ListContent[StuffContent](items=[]),diff --git a/pipelex/pipe_machinery/rendering/output_renderer.py b/pipelex/pipe_machinery/rendering/output_renderer.pyindex 0bd2a525e..647378418 100644--- a/pipelex/pipe_machinery/rendering/output_renderer.py+++ b/pipelex/pipe_machinery/rendering/output_renderer.py@@ -5,6 +5,7 @@ from pipelex.base_exceptions import PipelexErrorfrom pipelex.core.concepts.concept_representation_generator import ConceptRepresentationFormatfrom pipelex.core.concepts.native.concept_native import NativeConceptCodefrom pipelex.interpreter_hub import get_concept_library, get_required_pipe+from pipelex.pipe_controllers.binding.binding_step import BindingStepfrom pipelex.pipe_machinery.pipe_abstract import PipeAbstractfrom pipelex.pipe_machinery.pipe_blueprint import PipeType@@ -81,6 +82,24 @@ def _collect_possible_outputs(return []last_sub_pipe = sequential_sub_pipes[-1]+ if isinstance(last_sub_pipe, BindingStep):+ # A binding ending the sequence outputs the spec it derives, rendered like a pipe's output. The library+ # lookup sits outside the try, as in the branches beside it; deriving the binding sits inside, since a+ # path that does not resolve is, like a structure that cannot be rendered, a pipe with no output to show.+ binding_concept_provider = get_concept_library()+ try:+ binding_spec = the_pipe.build_typed_flow().binding_specs.get(len(sequential_sub_pipes) - 1)+ if binding_spec is None:+ return []+ binding_output_dict = binding_spec.render_stuff_spec(concept_provider=binding_concept_provider, output_format=output_format)+ except (PipelexError, ValueError):+ return []+ return [+ {+ "concept_ref": binding_spec.concept.concept_ref,+ "content": binding_output_dict.get("content", binding_output_dict),+ }+ ]last_pipe_code: str = getattr(last_sub_pipe, "pipe_code", "")if not last_pipe_code:return []diff --git a/pipelex/pipe_machinery/validation.py b/pipelex/pipe_machinery/validation.pyindex 51e832a11..6fadef812 100644--- a/pipelex/pipe_machinery/validation.py+++ b/pipelex/pipe_machinery/validation.py@@ -15,14 +15,30 @@ from pipelex.core.pipes.variable_multiplicity import (presence_from_symbol,presence_symbol,)-from pipelex.tools.misc.string_utils import get_root_from_dotted_path, is_snake_case+from pipelex.tools.misc.string_utils import (+ FIELD_PATH_PATTERN,+ SNAKE_CASE_IDENTIFIER_REGEX,+ SNAKE_CASE_PATTERN,+ get_root_from_dotted_path,+ is_snake_case,+)from pipelex.validation_error_types import PipeValidationErrorType# The grammar of an input name, the standard's `[a-z][a-z0-9_]*`: a plain snake_case identifier, which# is never dotted. It applies to every key of a pipe's `inputs`, whatever the pipe, and to a PipeBatch's# `input_list_name`. The MTHDS JSON Schema generator writes this same pattern on both, so a structural-# check refuses exactly what `validate_input_names` and `check_input_list_name` refuse.-INPUT_NAME_PATTERN = r"^[a-z][a-z0-9_]*$"+# check refuses exactly what `validate_input_names` and `check_input_list_name` refuse. It is the+# snake_case grammar of `string_utils`, the one source every name grammar here is composed from.+INPUT_NAME_PATTERN = SNAKE_CASE_PATTERN++# The grammar of a binding step's `result`: a plain input name, since it exists to be read by a later+# step's input or as a later binding's root, both of which are plain names.+BINDING_RESULT_PATTERN = INPUT_NAME_PATTERN++# The grammar of a binding step's `from`: a root name followed by zero or more field names, separated by+# single dots, each segment a letter followed by letters, digits and underscores. Subscripts, expressions,+# whitespace and underscore-led segments are outside it.+BINDING_PATH_PATTERN = FIELD_PATH_PATTERN# The marker of an input declaration as `_input_marker` writes it after the concept: an optional multiplicity# suffix (`[]` or `[N]`), then an optional presence symbol (`?` or `!`). Group 1 is the bracket content, group 2@@ -123,11 +139,12 @@ def _quoted_names(*, names: list[str]) -> str:def is_valid_input_name(input_name: str) -> bool:"""Whether a name is a plain input name: a snake_case identifier matching ``INPUT_NAME_PATTERN``, never dotted."""- return re.fullmatch(INPUT_NAME_PATTERN, input_name) is not None+ return is_snake_case(input_name)_PLAIN_NAME_RULE = (- "an input name is a plain snake_case identifier, matching `[a-z][a-z0-9_]*`: a lowercase letter, then lowercase letters, digits and underscores"+ f"an input name is a plain snake_case identifier, matching `{SNAKE_CASE_IDENTIFIER_REGEX}`: a lowercase letter, then lowercase letters, "+ "digits and underscores")diff --git a/pipelex/pipeline/file_input_consumers.py b/pipelex/pipeline/file_input_consumers.pyindex 034bf0718..f348c96c8 100644--- a/pipelex/pipeline/file_input_consumers.py+++ b/pipelex/pipeline/file_input_consumers.py@@ -9,7 +9,8 @@ resolving sub-pipes through the hub:that step. A step overwrites its result's name, and also whatever a nested sequence or conditionoutcome writes, since those run on the caller's memory; after a step that may write a name thewalk cannot know, nothing more is followed. A step's batch parameters map the list slot to its- item slot.+ item slot. A binding step that binds a single value makes its result stand for the entry path it+ reads, a renamed copy for its root's own path; one gathering a list stops its result being followed.- **Parallel.** Every branch is visited.- **Batch.** The list slot maps to the item slot, and the branch pipe is visited.- **Condition.** Every outcome is visited, and whatever is found below it is conditional.@@ -42,6 +43,7 @@ from pipelex.kernel.extract_ops import resolve_extract_settingfrom pipelex.kernel.judgment_ops import judgment_setting_of_choicefrom pipelex.kernel.llm_ops import resolve_llm_setting_for_object, resolve_llm_setting_for_textfrom pipelex.pipe_controllers.batch.pipe_batch import PipeBatch+from pipelex.pipe_controllers.binding.binding_step import BindingStepfrom pipelex.pipe_controllers.condition.pipe_condition import PipeConditionfrom pipelex.pipe_controllers.parallel.pipe_parallel import PipeParallelfrom pipelex.pipe_controllers.sequence.pipe_sequence import PipeSequence@@ -323,8 +325,19 @@ def _visit_sequence(*, sequence: PipeSequence, frame: _Frame, is_conditional: bool, visiting: frozenset[str], consumers: list[FileInputConsumer]) -> None:liftable_refs = {liftable_step.pipe_ref for liftable_step in sequence.analyze_taint().liftable_steps}+ typed_flow = sequence.build_typed_flow() if any(isinstance(step, BindingStep) for step in sequence.sequential_sub_pipes) else Nonestep_frame = dict(frame)- for sub_pipe in sequence.sequential_sub_pipes:+ for step_index, sub_pipe in enumerate(sequence.sequential_sub_pipes):+ if isinstance(sub_pipe, BindingStep):+ # A binding stores a copy of the value at its path: a single value stands for the entry path it reads,+ # while a list gathered across items has no one entry path, so it stops being followed, as an unknown would.+ derivation = typed_flow.binding_derivations.get(step_index) if typed_flow is not None else None+ bound_path = _tracked_path(frame=step_frame, variable_path=sub_pipe.from_path) if derivation and not derivation.is_plural else None+ if bound_path is None:+ step_frame.pop(sub_pipe.output_name, None)+ else:+ step_frame[sub_pipe.output_name] = bound_path+ continuestep_pipe = _visit_sub_pipe(sub_pipe=sub_pipe, frame=step_frame, is_conditional=is_conditional, liftable_refs=liftable_refs, visiting=visiting, consumers=consumers)@@ -377,7 +390,9 @@ def _pipe_writes(*, pipe: PipeAbstract, visiting: frozenset[str]) -> _Writes:if isinstance(pipe, PipeSequence):return _merge_writes(writes=[- _sub_pipe_writes(sub_pipe=sub_pipe, step_pipe=get_optional_pipe(pipe_code=sub_pipe.pipe_code), visiting=visiting)+ _Writes(names=frozenset({sub_pipe.output_name}), may_write_any=False)+ if isinstance(sub_pipe, BindingStep)+ else _sub_pipe_writes(sub_pipe=sub_pipe, step_pipe=get_optional_pipe(pipe_code=sub_pipe.pipe_code), visiting=visiting)for sub_pipe in pipe.sequential_sub_pipes])diff --git a/pipelex/pipeline/fixes/planner.py b/pipelex/pipeline/fixes/planner.pyindex 802a34359..1aa026065 100644--- a/pipelex/pipeline/fixes/planner.py+++ b/pipelex/pipeline/fixes/planner.py@@ -8,6 +8,7 @@ of the validation report — CLI, API, MCP — sees fixes with zero extra plumbifrom pipelex.core.exceptions import PipelexBundleBlueprintValidationErrorData, PipesAndConceptValidationErrorDatafrom pipelex.pipe_machinery.validation import dropped_input_marker_warningfrom pipelex.suggested_fix import DeleteKeyOp, EnsureTableOp, FixOp, FixSafety, RemapValueOp, RenameTableKeyOp, SetKeyOp, SuggestedFix+from pipelex.tools.misc.string_utils import get_root_from_dotted_pathMATCH_SEQUENCE_OUTPUT_FIX_CODE = "match-sequence-output"SYNC_CONTROLLER_INPUTS_FIX_CODE = "sync-controller-inputs"@@ -203,9 +204,12 @@ def _plan_delete_redundant_dotted_input(*, error_data: PipelexBundleBlueprintVala run requires, so ``pipelex fix bundle`` never applies it, and its description names the part thatdiffers, the multiplicity, the presence or both, and the edit that carries it onto the root. The twomarkers are set together; a dropped marker without its root's is an incomplete enrichment and yields- no fix, never a ``SAFE`` one. A lone dotted name, whose root's concept nothing states, and a malformed- name carry no enrichment and are suppressed here structurally; their message names the remedies for- the author.+ no fix, never a ``SAFE`` one. The fix is also ``UNSAFE`` when ``redundant_input_unwalkable_reason`` is set:+ the root's declared concept does not hold the field the key named (``data = "Number"`` beside+ ``"data.text" = "Text"``), so deleting the key would leave the template reading a field the root lacks,+ and the description carries the walk's reason. A lone dotted name, whose root's concept nothing states,+ and a malformed name carry no enrichment and are suppressed here structurally; their message names the+ remedies for the author."""if error_data.redundant_input_name is None or error_data.pipe_code is None:return None@@ -224,6 +228,14 @@ def _plan_delete_redundant_dotted_input(*, error_data: PipelexBundleBlueprintVal)description = f"{description}. {marker_warning}"safety = FixSafety.UNSAFE+ if error_data.redundant_input_unwalkable_reason is not None:+ root_name = get_root_from_dotted_path(error_data.redundant_input_name)+ description = (+ f"{description}. Warning: '{error_data.redundant_input_name}' cannot be read through the concept declared for '{root_name}', "+ f"so a template reading it may fail once the key is deleted. {error_data.redundant_input_unwalkable_reason} "+ f"Declare '{root_name}' with the concept whose structure holds the field"+ )+ safety = FixSafety.UNSAFEreturn SuggestedFix(fix_code=DELETE_REDUNDANT_DOTTED_INPUT_FIX_CODE,description=description,diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_batch_views/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_batch_views/bundle.mthdsnew file mode 100644index 000000000..2b99dbb0d--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_batch_views/bundle.mthds@@ -0,0 +1,20 @@+domain = "catalog_index"+description = "Listing the caption of every photograph in a catalog"+main_pipe = "index_views"++[pipe.index_views]+type = "PipeSequence"+description = "Binds the views of every page, then writes one index line per view"+inputs = { pages = "Page[]" }+output = "Text[]"+steps = [+ { from = "pages.page_view", result = "page_views" },+ { pipe = "write_index_line", batch_over = "page_views", batch_as = "page_view", result = "index_lines" },+]++[pipe.write_index_line]+type = "PipeCompose"+description = "Writes the index line for one photograph"+inputs = { page_view = "Image" }+output = "Text"+template = "Photograph: {{ page_view.caption }}"diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_batch_views/entry.toml b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_batch_views/entry.tomlnew file mode 100644index 000000000..f0580dee4--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_batch_views/entry.toml@@ -0,0 +1,11 @@+name = "feature_binding_step_batch_views"+description = "The views of a catalog's pages bound into a list, then captioned one by one by a batch over the bound name"++validity = "valid"+tier = "offline"+granularity = "focused"++covers = [+ "feature.binding_step",+ "controller.batch",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_batch_views/inputs.json b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_batch_views/inputs.jsonnew file mode 100644index 000000000..50fedf3c3--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_batch_views/inputs.json@@ -0,0 +1,36 @@+{+ "pages": {+ "concept": "native.Page",+ "content": [+ {+ "text_and_images": {+ "text": {+ "text": "Garden chairs"+ }+ },+ "page_view": {+ "url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGOQOyEHAAIMAQUtuDZBAAAAAElFTkSuQmCC",+ "caption": "Garden chairs on a lawn"+ }+ },+ {+ "text_and_images": {+ "text": {+ "text": "Order form"+ }+ }+ },+ {+ "text_and_images": {+ "text": {+ "text": "Parasols"+ }+ },+ "page_view": {+ "url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGOQkzsBAAFiAQURG6MhAAAAAElFTkSuQmCC",+ "caption": "Two parasols"+ }+ }+ ]+ }+}diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_board_rename/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_board_rename/bundle.mthdsnew file mode 100644index 000000000..1eccec737--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_board_rename/bundle.mthds@@ -0,0 +1,27 @@+domain = "station_board"+description = "Reading out the departure board of a small station"+main_pipe = "read_out_board"++[concept.DepartureBoard]+description = "The board listing the next departures from a station"++[concept.DepartureBoard.structure]+station = { type = "text", description = "The station the board stands in", required = true }+next_train = { type = "text", description = "Where the next train goes", required = true }++[pipe.read_out_board]+type = "PipeSequence"+description = "Binds the departure board under the name the announcement reads, then writes the announcement"+inputs = { departure_board = "DepartureBoard" }+output = "Text"+steps = [+ { from = "departure_board", result = "board" },+ { pipe = "write_board_announcement", result = "announcement" },+]++[pipe.write_board_announcement]+type = "PipeCompose"+description = "Writes the announcement read out from a board"+inputs = { board = "DepartureBoard" }+output = "Text"+template = "At {{ board.station }}, the next train goes to {{ board.next_train }}."diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_board_rename/entry.toml b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_board_rename/entry.tomlnew file mode 100644index 000000000..faf4ed791--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_board_rename/entry.toml@@ -0,0 +1,10 @@+name = "feature_binding_step_board_rename"+description = "A departure board bound under the name the next step reads, a renamed copy of the whole value"++validity = "valid"+tier = "offline"+granularity = "focused"++covers = [+ "feature.binding_step",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_board_rename/inputs.json b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_board_rename/inputs.jsonnew file mode 100644index 000000000..29fe10e96--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_board_rename/inputs.json@@ -0,0 +1,9 @@+{+ "departure_board": {+ "concept": "station_board.DepartureBoard",+ "content": {+ "station": "Landerneau",+ "next_train": "Brest"+ }+ }+}diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_catalog_review/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_catalog_review/bundle.mthdsnew file mode 100644index 000000000..980006a8a--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_catalog_review/bundle.mthds@@ -0,0 +1,67 @@+domain = "catalog_review"+description = "Reviewing a supplier's catalog before it goes to print"+main_pipe = "review_catalog"++[concept.Catalog]+description = "A supplier's catalog waiting for review"++[concept.Catalog.structure]+title = { type = "text", description = "The title on the cover", required = true }+pages = { type = "list", item_type = "concept", item_concept_ref = "Page", description = "The pages of the catalog", required = true }+cheapest_price = { type = "number", description = "The lowest price printed in the catalog, in euros", required = true }+editor_note = { type = "text", description = "A note the editor left on the proof, when there is one" }++[concept.CatalogReview]+description = "The review sent back to the supplier"++[concept.CatalogReview.structure]+title = { type = "text", description = "The catalog reviewed", required = true }+captions = { type = "list", item_type = "text", description = "One caption per photograph", required = true }+price_is_low = { type = "boolean", description = "Whether the cheapest price looks too low to be true", required = true }++[pipe.review_catalog]+type = "PipeSequence"+description = "Binds the parts of a catalog, captions its photographs, judges its cheapest price and writes the review"+inputs = { catalog = "Catalog" }+output = "Text"+steps = [+ { from = "catalog.title", result = "title" },+ { from = "catalog.pages.page_view", result = "page_views" },+ { from = "catalog.cheapest_price", result = "cheapest_price" },+ { from = "catalog.editor_note", result = "editor_note" },+ { pipe = "caption_view", batch_over = "page_views", batch_as = "page_view", result = "captions" },+ { pipe = "judge_price_is_low", result = "price_is_low" },+ { pipe = "write_review", result = "review" },+]++[pipe.caption_view]+type = "PipeLLM"+description = "Writes a one-line caption for a catalog photograph"+inputs = { page_view = "Image" }+output = "Text"+prompt = """+Write a one-line caption for this catalog photograph:+@page_view+"""++[pipe.judge_price_is_low]+type = "PipeJudge"+description = "Judges whether a price looks too low to be true"+inputs = { cheapest_price = "Number" }+output = "YesNo"+model = "@default-judgment"+question = "Does this price, in euros, look too low to be true for furniture?"+criteria = { yes = "The price is below five euros", no = "The price is five euros or more" }+threshold = 0.7++[pipe.write_review]+type = "PipeCompose"+description = "Writes the review sent back to the supplier"+inputs = { title = "Text", captions = "Text[]", price_is_low = "YesNo", editor_note = "Text?" }+output = "Text"+template = """+Review of $title+{% for caption in captions %}- {{ caption.text }}+{% endfor %}Cheapest price suspicious: {{ price_is_low.yes_no }}+{% if editor_note is defined %}Editor's note: {{ editor_note.text }}{% endif %}+"""diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_catalog_review/entry.toml b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_catalog_review/entry.tomlnew file mode 100644index 000000000..79e314a46--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_catalog_review/entry.toml@@ -0,0 +1,13 @@+name = "feature_binding_step_catalog_review"+description = "A catalog reviewed through several bindings, a batch over bound views, a judge on a bound price and an optional note"++validity = "valid"+tier = "dry"+granularity = "composite"++covers = [+ "feature.binding_step",+ "feature.optionals",+ "controller.batch",+ "operator.judge",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_delivery_note/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_delivery_note/bundle.mthdsnew file mode 100644index 000000000..c9b631ed5--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_delivery_note/bundle.mthds@@ -0,0 +1,39 @@+domain = "round_notes"+description = "Passing on the notes left for the morning and evening delivery rounds"+main_pipe = "pass_on_notes"++[concept.DeliveryRound]+description = "A delivery round driven by one van"++[concept.DeliveryRound.structure]+driver = { type = "text", description = "Who drives the round", required = true }+note = { type = "text", description = "A note left for the driver, when there is one" }++[pipe.pass_on_notes]+type = "PipeSequence"+description = "Binds the note of each round, writes the board notice, then the evening reminder when there is a note"+inputs = { morning_round = "DeliveryRound", evening_round = "DeliveryRound" }+output = "Text?"+steps = [+ { from = "morning_round.note", result = "morning_note" },+ { from = "evening_round.note", result = "evening_note" },+ { pipe = "write_board_notice", result = "board_notice" },+ { pipe = "write_evening_reminder", result = "evening_reminder" },+]++[pipe.write_board_notice]+type = "PipeCompose"+description = "Writes the notice pinned on the depot board, mentioning each note that was left"+inputs = { morning_note = "Text?", evening_note = "Text?" }+output = "Text"+template = """+{% if morning_note is defined %}Morning: {{ morning_note.text }}{% else %}Morning: no note{% endif %}+{% if evening_note is defined %}Evening: {{ evening_note.text }}{% else %}Evening: no note{% endif %}+"""++[pipe.write_evening_reminder]+type = "PipeCompose"+description = "Writes the reminder handed to the evening driver"+inputs = { evening_note = "Text" }+output = "Text"+template = "Reminder for tonight: $evening_note"diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_delivery_note/entry.toml b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_delivery_note/entry.tomlnew file mode 100644index 000000000..ca0162a84--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_delivery_note/entry.toml@@ -0,0 +1,11 @@+name = "feature_binding_step_delivery_note"+description = "A delivery note bound from two rounds, one carrying a note and one not, read by a guarded step and by a plain one that lifts"++validity = "valid"+tier = "offline"+granularity = "focused"++covers = [+ "feature.binding_step",+ "feature.optionals",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_delivery_note/inputs.json b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_delivery_note/inputs.jsonnew file mode 100644index 000000000..d3f4919eb--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_delivery_note/inputs.json@@ -0,0 +1,15 @@+{+ "morning_round": {+ "concept": "round_notes.DeliveryRound",+ "content": {+ "driver": "Yann",+ "note": "Gate code changed to 4471"+ }+ },+ "evening_round": {+ "concept": "round_notes.DeliveryRound",+ "content": {+ "driver": "Soizic"+ }+ }+}diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_departure_times/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_departure_times/bundle.mthdsnew file mode 100644index 000000000..c27dd93de--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_departure_times/bundle.mthds@@ -0,0 +1,30 @@+domain = "ferry_timetable"+description = "Announcing a ferry departure from a row of the timetable"+main_pipe = "announce_departure"++[concept.Departure]+description = "One departure listed in the ferry timetable"++[concept.Departure.structure]+day = { type = "date", description = "The day the ferry leaves", required = true }+scheduled_at = { type = "datetime", description = "The moment the ferry is scheduled to cast off", required = true }+boarding = { type = "time", description = "The time boarding opens", required = true }++[pipe.announce_departure]+type = "PipeSequence"+description = "Binds the day, the scheduled moment and the boarding time of a departure, then announces it"+inputs = { departure = "Departure" }+output = "Text"+steps = [+ { from = "departure.day", result = "day" },+ { from = "departure.scheduled_at", result = "scheduled_at" },+ { from = "departure.boarding", result = "boarding_time" },+ { pipe = "write_announcement", result = "announcement" },+]++[pipe.write_announcement]+type = "PipeCompose"+description = "Writes the announcement read out on the quay"+inputs = { day = "Date", scheduled_at = "Date", boarding_time = "Time" }+output = "Text"+template = "The ferry of $day leaves at $scheduled_at. Boarding opens at $boarding_time."diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_departure_times/entry.toml b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_departure_times/entry.tomlnew file mode 100644index 000000000..4e6030a3c--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_departure_times/entry.toml@@ -0,0 +1,10 @@+name = "feature_binding_step_departure_times"+description = "A departure announced from its day, its scheduled moment and its boarding time, each bound from a timetable row"++validity = "valid"+tier = "offline"+granularity = "focused"++covers = [+ "feature.binding_step",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_departure_times/inputs.json b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_departure_times/inputs.jsonnew file mode 100644index 000000000..dd2c2db7e--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_departure_times/inputs.json@@ -0,0 +1,10 @@+{+ "departure": {+ "concept": "ferry_timetable.Departure",+ "content": {+ "day": "2026-04-02",+ "scheduled_at": "2026-04-02T08:15:00+02:00",+ "boarding": "07:45:00"+ }+ }+}diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_extracted_pages/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_extracted_pages/bundle.mthdsnew file mode 100644index 000000000..dc308693f--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_extracted_pages/bundle.mthds@@ -0,0 +1,19 @@+domain = "scan_archive"+description = "Keeping the page photographs of a scanned delivery note"+main_pipe = "keep_page_views"++[pipe.keep_page_views]+type = "PipeSequence"+description = "Reads the pages out of a scan, then binds their views"+inputs = { delivery_scan = "Document" }+output = "Image[]"+steps = [+ { pipe = "read_pages", result = "pages" },+ { from = "pages.page_view", result = "page_views" },+]++[pipe.read_pages]+type = "PipeExtract"+description = "Reads each page out of the scan"+inputs = { delivery_scan = "Document" }+output = "Page[]"diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_extracted_pages/entry.toml b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_extracted_pages/entry.tomlnew file mode 100644index 000000000..470f63f7f--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_extracted_pages/entry.toml@@ -0,0 +1,10 @@+name = "feature_binding_step_extracted_pages"+description = "The pages read out of a scan by an earlier step, their views then bound, the root typed by the step that made it"++validity = "valid"+tier = "dry"+granularity = "focused"++covers = [+ "feature.binding_step",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_invoice_fields/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_invoice_fields/bundle.mthdsnew file mode 100644index 000000000..95c5027b4--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_invoice_fields/bundle.mthds@@ -0,0 +1,55 @@+domain = "supplier_invoices"+description = "Acknowledging a supplier invoice from the fields printed on it"+main_pipe = "acknowledge_invoice"++[concept.Address]+description = "A postal address"++[concept.Address.structure]+city = { type = "text", description = "The town of the address", required = true }++[concept.Supplier]+description = "A company that sends invoices"++[concept.Supplier.structure]+name = { type = "text", description = "The supplier's trading name", required = true }+address = { type = "concept", concept_ref = "Address", description = "Where the supplier is based", required = true }++[concept.Invoice]+description = "An invoice received from a supplier"++[concept.Invoice.structure]+supplier_name = { type = "text", description = "The name printed at the top of the invoice", required = true }+total = { type = "number", description = "The amount due, in euros", required = true }+line_count = { type = "integer", description = "How many lines the invoice lists", required = true }+priority = { choices = [+ "routine",+ "urgent",+], description = "How soon the invoice must be paid", required = true }+order_details = { type = "dict", key_type = "text", value_type = "text", description = "The references the supplier printed for the order", required = true }+supplier = { type = "concept", concept_ref = "Supplier", description = "The supplier who sent the invoice", required = true }++[pipe.acknowledge_invoice]+type = "PipeSequence"+description = "Binds the fields of an invoice, then writes the acknowledgement from them"+inputs = { invoice = "Invoice" }+output = "Text"+steps = [+ { from = "invoice.supplier_name", result = "supplier_name" },+ { from = "invoice.total", result = "total_amount" },+ { from = "invoice.line_count", result = "line_count" },+ { from = "invoice.priority", result = "priority" },+ { from = "invoice.order_details", result = "order_details" },+ { from = "invoice.supplier.address.city", result = "supplier_city" },+ { pipe = "write_acknowledgement", result = "acknowledgement" },+]++[pipe.write_acknowledgement]+type = "PipeCompose"+description = "Writes the acknowledgement of an invoice from its total, its supplier and the supplier's town"+inputs = { supplier_name = "Text", total_amount = "Number", line_count = "Number", priority = "Text", order_details = "JSON", supplier_city = "Text" }+output = "Text"+template = """+Received from $supplier_name of $supplier_city: $line_count lines, $total_amount euros due, $priority.+Order details: $order_details+"""diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_invoice_fields/entry.toml b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_invoice_fields/entry.tomlnew file mode 100644index 000000000..422981830--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_invoice_fields/entry.toml@@ -0,0 +1,10 @@+name = "feature_binding_step_invoice_fields"+description = "An invoice acknowledged from the plain fields it carries, each bound to a name of its own"++validity = "valid"+tier = "offline"+granularity = "focused"++covers = [+ "feature.binding_step",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_invoice_fields/inputs.json b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_invoice_fields/inputs.jsonnew file mode 100644index 000000000..705ee48f6--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_invoice_fields/inputs.json@@ -0,0 +1,21 @@+{+ "invoice": {+ "concept": "supplier_invoices.Invoice",+ "content": {+ "supplier_name": "Atelier Morvan",+ "total": 1250.5,+ "line_count": 3,+ "priority": "urgent",+ "order_details": {+ "order_ref": "PO-118",+ "delivery_slot": "morning"+ },+ "supplier": {+ "name": "Atelier Morvan SARL",+ "address": {+ "city": "Quimper"+ }+ }+ }+ }+}diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_judge_amount/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_judge_amount/bundle.mthdsnew file mode 100644index 000000000..b66351bae--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_judge_amount/bundle.mthds@@ -0,0 +1,30 @@+domain = "invoice_approval"+description = "Deciding whether an invoice total needs a second signature"+main_pipe = "check_total"++[concept.Invoice]+description = "An invoice waiting for approval"++[concept.Invoice.structure]+supplier = { type = "text", description = "Who sent the invoice", required = true }+total = { type = "number", description = "The amount due, in euros", required = true }++[pipe.check_total]+type = "PipeSequence"+description = "Binds the total of an invoice, then judges whether it needs a second signature"+inputs = { invoice = "Invoice" }+output = "YesNo"+steps = [+ { from = "invoice.total", result = "total_amount" },+ { pipe = "judge_needs_second_signature", result = "needs_second_signature" },+]++[pipe.judge_needs_second_signature]+type = "PipeJudge"+description = "Judges whether an amount is high enough to need a second signature"+inputs = { total_amount = "Number" }+output = "YesNo"+model = "@default-judgment"+question = "Is this amount high enough to need a second signature?"+criteria = { yes = "The amount is above five thousand euros", no = "The amount is five thousand euros or less" }+threshold = 0.7diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_judge_amount/entry.toml b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_judge_amount/entry.tomlnew file mode 100644index 000000000..4035f75b2--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_judge_amount/entry.toml@@ -0,0 +1,10 @@+name = "feature_binding_step_judge_amount"+description = "The total of an invoice bound alone and handed to a judge, which sees nothing else of the invoice"++validity = "valid"+tier = "dry"+granularity = "focused"++covers = [+ "feature.binding_step",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_missing_values/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_missing_values/bundle.mthdsnew file mode 100644index 000000000..775429270--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_missing_values/bundle.mthds@@ -0,0 +1,34 @@+domain = "invoice_filing"+description = "Filing a supplier invoice, with its scan and courtesy copy when they exist"+main_pipe = "file_invoice"++[concept.Invoice]+description = "An invoice received from a supplier"++[concept.Invoice.structure]+number = { type = "text", description = "The invoice number", required = true }+total = { type = "number", description = "The amount due, in euros", required = true }+scan = { type = "concept", concept_ref = "Document", description = "The scan of the paper invoice, when one was made" }++[pipe.file_invoice]+type = "PipeSequence"+description = "Binds the scan's link and the courtesy copy's total, then writes the filing note"+inputs = { invoice = "Invoice", courtesy_copy = "Invoice?" }+output = "Text"+steps = [+ { from = "invoice.number", result = "invoice_number" },+ { from = "invoice.scan.url", result = "scan_link" },+ { from = "courtesy_copy.total", result = "copy_total" },+ { pipe = "write_filing_note", result = "filing_note" },+]++[pipe.write_filing_note]+type = "PipeCompose"+description = "Writes the note that goes into the filing cabinet with the invoice"+inputs = { invoice_number = "Text", scan_link = "Text?", copy_total = "Number?" }+output = "Text"+template = """+Invoice $invoice_number filed.+{% if scan_link is defined %}Scan: {{ scan_link.text }}{% else %}No scan on file.{% endif %}+{% if copy_total is defined %}Courtesy copy total: {{ copy_total.number }}{% else %}No courtesy copy.{% endif %}+"""diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_missing_values/entry.toml b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_missing_values/entry.tomlnew file mode 100644index 000000000..f944b6090--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_missing_values/entry.toml@@ -0,0 +1,11 @@+name = "feature_binding_step_missing_values"+description = "An invoice whose scan is missing binds nothing from it, and a courtesy copy that was never provided skips its binding"++validity = "valid"+tier = "offline"+granularity = "focused"++covers = [+ "feature.binding_step",+ "feature.optionals",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_missing_values/inputs.json b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_missing_values/inputs.jsonnew file mode 100644index 000000000..87ad6387c--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_missing_values/inputs.json@@ -0,0 +1,9 @@+{+ "invoice": {+ "concept": "invoice_filing.Invoice",+ "content": {+ "number": "F-2026-0412",+ "total": 318.0+ }+ }+}diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_nested_sequence/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_nested_sequence/bundle.mthdsnew file mode 100644index 000000000..6f0496fbd--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_nested_sequence/bundle.mthds@@ -0,0 +1,36 @@+domain = "bakery_orders"+description = "Confirming a bakery order through a confirmation sub-flow"+main_pipe = "confirm_order"++[concept.BakeryOrder]+description = "An order placed at a bakery for a given day"++[concept.BakeryOrder.structure]+customer = { type = "text", description = "Who placed the order", required = true }+loaves = { type = "integer", description = "How many loaves were ordered", required = true }++[pipe.confirm_order]+type = "PipeSequence"+description = "Calls the confirmation sub-flow for an order"+inputs = { order = "BakeryOrder" }+output = "Text"+steps = [+ { pipe = "write_confirmation_flow", result = "confirmation" },+]++[pipe.write_confirmation_flow]+type = "PipeSequence"+description = "Binds the loaf count of an order, then writes the confirmation"+inputs = { order = "BakeryOrder" }+output = "Text"+steps = [+ { from = "order.loaves", result = "loaf_count" },+ { pipe = "write_confirmation", result = "confirmation" },+]++[pipe.write_confirmation]+type = "PipeCompose"+description = "Writes the confirmation of a loaf count"+inputs = { loaf_count = "Number" }+output = "Text"+template = "Confirmed: $loaf_count loaves"diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_nested_sequence/entry.toml b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_nested_sequence/entry.tomlnew file mode 100644index 000000000..2fadc0895--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_nested_sequence/entry.toml@@ -0,0 +1,10 @@+name = "feature_binding_step_nested_sequence"+description = "A binding inside a sequence that another sequence calls, binding as it does at the top level"++validity = "valid"+tier = "offline"+granularity = "focused"++covers = [+ "feature.binding_step",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_nested_sequence/inputs.json b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_nested_sequence/inputs.jsonnew file mode 100644index 000000000..34b813467--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_nested_sequence/inputs.json@@ -0,0 +1,9 @@+{+ "order": {+ "concept": "bakery_orders.BakeryOrder",+ "content": {+ "customer": "Café de la Mairie",+ "loaves": 12+ }+ }+}diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_order_lines/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_order_lines/bundle.mthdsnew file mode 100644index 000000000..dfec4af9e--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_order_lines/bundle.mthds@@ -0,0 +1,39 @@+domain = "order_desk"+description = "Summing up an order and its backorder line by line"+main_pipe = "summarize_order"++[concept.OrderLine]+description = "One line of an order"++[concept.OrderLine.structure]+article = { type = "text", description = "The article ordered", required = true }+amount = { type = "number", description = "What the line costs, in euros" }++[concept.Order]+description = "An order placed by a shop"++[concept.Order.structure]+lines = { type = "list", item_type = "concept", item_concept_ref = "OrderLine", description = "The lines of the order", required = true }++[pipe.summarize_order]+type = "PipeSequence"+description = "Binds the lines of an order and the amounts of an order and of its backorder, then sums them up"+inputs = { order = "Order", backorder = "Order" }+output = "Text"+steps = [+ { from = "order.lines", result = "lines" },+ { from = "order.lines.amount", result = "amounts" },+ { from = "backorder.lines.amount", result = "backorder_amounts" },+ { pipe = "write_summary", result = "summary" },+]++[pipe.write_summary]+type = "PipeCompose"+description = "Writes one line per article, then the amounts of the order and of its backorder"+inputs = { lines = "OrderLine[]", amounts = "Number[]", backorder_amounts = "Number[]" }+output = "Text"+template = """+{% for line in lines %}- {{ line.article }}+{% endfor %}Amounts: {% for amount in amounts %}{{ amount.number }} {% endfor %}+Backordered amounts: {{ backorder_amounts | length }}+"""diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_order_lines/entry.toml b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_order_lines/entry.tomlnew file mode 100644index 000000000..a2aacf90e--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_order_lines/entry.toml@@ -0,0 +1,10 @@+name = "feature_binding_step_order_lines"+description = "The lines of an order bound as a list, and their amounts bound across the list, an order without lines giving an empty list"++validity = "valid"+tier = "offline"+granularity = "focused"++covers = [+ "feature.binding_step",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_order_lines/inputs.json b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_order_lines/inputs.jsonnew file mode 100644index 000000000..b5f849662--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_order_lines/inputs.json@@ -0,0 +1,26 @@+{+ "order": {+ "concept": "order_desk.Order",+ "content": {+ "lines": [+ {+ "article": "Linen tea towel",+ "amount": 12.5+ },+ {+ "article": "Sample swatch"+ },+ {+ "article": "Enamel mug",+ "amount": 9.0+ }+ ]+ }+ },+ "backorder": {+ "concept": "order_desk.Order",+ "content": {+ "lines": []+ }+ }+}diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_page_views/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_page_views/bundle.mthdsnew file mode 100644index 000000000..48c007f55--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_page_views/bundle.mthds@@ -0,0 +1,13 @@+domain = "catalog_pages"+description = "Gathering the photographs of a printed catalog"+main_pipe = "gather_page_views"++[pipe.gather_page_views]+type = "PipeSequence"+description = "Binds the view of the cover, then the views of every page, which the sequence returns"+inputs = { cover = "Page", pages = "Page[]" }+output = "Image[]"+steps = [+ { from = "cover.page_view", result = "cover_view" },+ { from = "pages.page_view", result = "page_views" },+]diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_page_views/entry.toml b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_page_views/entry.tomlnew file mode 100644index 000000000..86156bf3e--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_page_views/entry.toml@@ -0,0 +1,10 @@+name = "feature_binding_step_page_views"+description = "The views of a catalog's pages bound as the last step, a page without a view dropped, beside the cover's view"++validity = "valid"+tier = "offline"+granularity = "focused"++covers = [+ "feature.binding_step",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_page_views/inputs.json b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_page_views/inputs.jsonnew file mode 100644index 000000000..0d2308eef--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_page_views/inputs.json@@ -0,0 +1,51 @@+{+ "cover": {+ "concept": "native.Page",+ "content": {+ "text_and_images": {+ "text": {+ "text": "Spring catalog"+ }+ },+ "page_view": {+ "url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGM4IScHAAK2AQU0pnWqAAAAAElFTkSuQmCC",+ "caption": "The spring cover",+ "source_prompt": "A cover photographed flat"+ }+ }+ },+ "pages": {+ "concept": "native.Page",+ "content": [+ {+ "text_and_images": {+ "text": {+ "text": "Garden chairs"+ }+ },+ "page_view": {+ "url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGOQOyEHAAIMAQUtuDZBAAAAAElFTkSuQmCC",+ "caption": "Garden chairs on a lawn"+ }+ },+ {+ "text_and_images": {+ "text": {+ "text": "Order form"+ }+ }+ },+ {+ "text_and_images": {+ "text": {+ "text": "Parasols"+ }+ },+ "page_view": {+ "url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGOQkzsBAAFiAQURG6MhAAAAAElFTkSuQmCC",+ "caption": "Two parasols"+ }+ }+ ]+ }+}diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_parallel_root/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_parallel_root/bundle.mthdsnew file mode 100644index 000000000..dc9ac021a--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_parallel_root/bundle.mthds@@ -0,0 +1,56 @@+domain = "market_stall"+description = "Preparing a market stall's price tag and sign at once"+main_pipe = "prepare_stall"++[concept.PriceTag]+description = "The tag stuck on a crate of produce"++[concept.PriceTag.structure]+produce = { type = "text", description = "What is in the crate", required = true }+price = { type = "number", description = "The price per kilogram, in euros", required = true }++[pipe.prepare_stall]+type = "PipeSequence"+description = "Prepares the tag and the sign in parallel, then binds the price off the tag"+inputs = { produce = "Text" }+output = "Text"+steps = [+ { pipe = "prepare_tag_and_sign", result = "stall" },+ { from = "price_tag.price", result = "price" },+ { pipe = "write_price_line", result = "price_line" },+]++[pipe.prepare_tag_and_sign]+type = "PipeParallel"+description = "Writes the price tag and the stall sign at the same time"+inputs = { produce = "Text" }+output = "Composite"+add_each_output = true+branches = [+ { pipe = "write_price_tag", result = "price_tag" },+ { pipe = "write_stall_sign", result = "stall_sign" },+]++[pipe.write_price_tag]+type = "PipeCompose"+description = "Writes the price tag for a crate of produce"+inputs = { produce = "Text" }+output = "PriceTag"++[pipe.write_price_tag.construct]+produce = { from = "produce.text" }+price = 3.2++[pipe.write_stall_sign]+type = "PipeCompose"+description = "Writes the sign hung above the stall"+inputs = { produce = "Text" }+output = "Text"+template = "Fresh $produce today"++[pipe.write_price_line]+type = "PipeCompose"+description = "Writes the price line read out to customers"+inputs = { price = "Number" }+output = "Text"+template = "$price euros a kilo"diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_parallel_root/entry.toml b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_parallel_root/entry.tomlnew file mode 100644index 000000000..6a823b7b7--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_parallel_root/entry.toml@@ -0,0 +1,11 @@+name = "feature_binding_step_parallel_root"+description = "A field bound from the result of a parallel branch, the root typed by the branch that produced it"++validity = "valid"+tier = "offline"+granularity = "focused"++covers = [+ "feature.binding_step",+ "controller.parallel",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_parallel_root/inputs.json b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_parallel_root/inputs.jsonnew file mode 100644index 000000000..c2da95942--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_parallel_root/inputs.json@@ -0,0 +1,8 @@+{+ "produce": {+ "concept": "native.Text",+ "content": {+ "text": "Plougastel strawberries"+ }+ }+}diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_parcel_recipient/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_parcel_recipient/bundle.mthdsnew file mode 100644index 000000000..ab98a2804--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_parcel_recipient/bundle.mthds@@ -0,0 +1,50 @@+domain = "parcel_counter"+description = "Writing the label for a parcel handed in at the counter"+main_pipe = "label_parcel"++[concept.Recipient]+description = "The person a parcel is addressed to"++[concept.Recipient.structure]+name = { type = "text", description = "The recipient's full name", required = true }+street = { type = "text", description = "The street and number", required = true }+town = { type = "text", description = "The town", required = true }++[concept.Parcel]+description = "A parcel handed in at the counter"++[concept.Parcel.structure]+is_fragile = { type = "boolean", description = "Whether the parcel must be handled with care", required = true }+recipient = { type = "concept", concept_ref = "Recipient", description = "Who the parcel is for", required = true }++[concept.ParcelLabel]+description = "The label stuck on a parcel before it leaves the counter"++[concept.ParcelLabel.structure]+addressee = { type = "text", description = "Who the parcel is for", required = true }+street = { type = "text", description = "The street and number", required = true }+town = { type = "text", description = "The town", required = true }+handling = { type = "text", description = "How the parcel must be handled", required = true }++[pipe.label_parcel]+type = "PipeSequence"+description = "Binds the recipient and the fragile flag of a parcel, then writes its label"+inputs = { parcel = "Parcel" }+output = "ParcelLabel"+steps = [+ { from = "parcel.is_fragile", result = "fragile" },+ { from = "parcel.recipient", result = "recipient" },+ { pipe = "write_label", result = "label" },+]++[pipe.write_label]+type = "PipeCompose"+description = "Writes the label from the recipient and whether the parcel is fragile"+inputs = { recipient = "Recipient", fragile = "YesNo" }+output = "ParcelLabel"++[pipe.write_label.construct]+addressee = { from = "recipient.name" }+street = { from = "recipient.street" }+town = { from = "recipient.town" }+handling = { template = "{% if fragile.yes_no %}Fragile, handle with care{% else %}Standard handling{% endif %}" }diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_parcel_recipient/entry.toml b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_parcel_recipient/entry.tomlnew file mode 100644index 000000000..69f27c597--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_parcel_recipient/entry.toml@@ -0,0 +1,10 @@+name = "feature_binding_step_parcel_recipient"+description = "A parcel label written from the recipient bound whole and the fragile flag bound as a yes or no"++validity = "valid"+tier = "offline"+granularity = "focused"++covers = [+ "feature.binding_step",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_parcel_recipient/inputs.json b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_parcel_recipient/inputs.jsonnew file mode 100644index 000000000..d6d585503--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_parcel_recipient/inputs.json@@ -0,0 +1,13 @@+{+ "parcel": {+ "concept": "parcel_counter.Parcel",+ "content": {+ "is_fragile": true,+ "recipient": {+ "name": "Maëlle Le Gall",+ "street": "12 rue des Tanneurs",+ "town": "Morlaix"+ }+ }+ }+}diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_receipt_inherited/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_receipt_inherited/bundle.mthdsnew file mode 100644index 000000000..f291744f9--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_receipt_inherited/bundle.mthds@@ -0,0 +1,31 @@+domain = "shop_till"+description = "Reading the amount paid off a till receipt"+main_pipe = "read_amount_paid"++[concept.Invoice]+description = "A bill for goods sold"++[concept.Invoice.structure]+shop = { type = "text", description = "The shop that sold the goods", required = true }+total = { type = "number", description = "The amount of the bill, in euros", required = true }++[concept.Receipt]+description = "A bill once it has been paid at the till"+refines = "Invoice"++[pipe.read_amount_paid]+type = "PipeSequence"+description = "Binds the total a receipt inherits from the invoice, then writes it out"+inputs = { receipt = "Receipt" }+output = "Text"+steps = [+ { from = "receipt.total", result = "amount_paid" },+ { pipe = "write_amount_paid", result = "line" },+]++[pipe.write_amount_paid]+type = "PipeCompose"+description = "Writes the amount paid"+inputs = { amount_paid = "Number" }+output = "Text"+template = "Paid: $amount_paid euros"diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_receipt_inherited/entry.toml b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_receipt_inherited/entry.tomlnew file mode 100644index 000000000..56b588ff8--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_receipt_inherited/entry.toml@@ -0,0 +1,10 @@+name = "feature_binding_step_receipt_inherited"+description = "A receipt refining an invoice binds a field it inherits from the invoice's structure"++validity = "valid"+tier = "offline"+granularity = "focused"++covers = [+ "feature.binding_step",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_receipt_inherited/inputs.json b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_receipt_inherited/inputs.jsonnew file mode 100644index 000000000..d25f3b6fc--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_receipt_inherited/inputs.json@@ -0,0 +1,9 @@+{+ "receipt": {+ "concept": "shop_till.Receipt",+ "content": {+ "shop": "Épicerie du Port",+ "total": 42.8+ }+ }+}diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_shipment_parcels/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_shipment_parcels/bundle.mthdsnew file mode 100644index 000000000..2261eeb94--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_shipment_parcels/bundle.mthds@@ -0,0 +1,34 @@+domain = "depot_loading"+description = "Listing every parcel to load from the shipments of the day"+main_pipe = "list_parcels"++[concept.Parcel]+description = "A parcel waiting at the depot"++[concept.Parcel.structure]+tracking_number = { type = "text", description = "The number on the parcel's label", required = true }++[concept.Shipment]+description = "A shipment leaving the depot"++[concept.Shipment.structure]+parcels = { type = "list", item_type = "concept", item_concept_ref = "Parcel", description = "The parcels of the shipment", required = true }++[pipe.list_parcels]+type = "PipeSequence"+description = "Binds the parcels of every shipment into one list, then writes the loading list"+inputs = { shipments = "Shipment[]" }+output = "Text"+steps = [+ { from = "shipments.parcels", result = "parcels" },+ { pipe = "write_loading_list", result = "loading_list" },+]++[pipe.write_loading_list]+type = "PipeCompose"+description = "Writes one line per parcel to load"+inputs = { parcels = "Parcel[]" }+output = "Text"+template = """+{% for parcel in parcels %}- {{ parcel.tracking_number }}+{% endfor %}"""diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_shipment_parcels/entry.toml b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_shipment_parcels/entry.tomlnew file mode 100644index 000000000..1d5b5ebbe--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_shipment_parcels/entry.toml@@ -0,0 +1,10 @@+name = "feature_binding_step_shipment_parcels"+description = "The parcels of several shipments bound into one flat list, a shipment without parcels adding none"++validity = "valid"+tier = "offline"+granularity = "focused"++covers = [+ "feature.binding_step",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_shipment_parcels/inputs.json b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_shipment_parcels/inputs.jsonnew file mode 100644index 000000000..d061a5cb7--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_shipment_parcels/inputs.json@@ -0,0 +1,27 @@+{+ "shipments": {+ "concept": "depot_loading.Shipment",+ "content": [+ {+ "parcels": [+ {+ "tracking_number": "QP-1001"+ },+ {+ "tracking_number": "QP-1002"+ }+ ]+ },+ {+ "parcels": []+ },+ {+ "parcels": [+ {+ "tracking_number": "BR-2001"+ }+ ]+ }+ ]+ }+}diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_view_attached/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_view_attached/bundle.mthdsnew file mode 100644index 000000000..e6f990a82--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_view_attached/bundle.mthds@@ -0,0 +1,23 @@+domain = "catalog_captions"+description = "Captioning the photograph of a catalog page"+main_pipe = "caption_page"++[pipe.caption_page]+type = "PipeSequence"+description = "Binds the view of a page, then captions it, when the page has a view"+inputs = { page = "Page" }+output = "Text?"+steps = [+ { from = "page.page_view", result = "page_view" },+ { pipe = "write_caption", result = "caption" },+]++[pipe.write_caption]+type = "PipeLLM"+description = "Writes a one-line caption for a catalog photograph"+inputs = { page_view = "Image" }+output = "Text"+prompt = """+Write a one-line caption for this catalog photograph:+@page_view+"""diff --git a/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_view_attached/entry.toml b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_view_attached/entry.tomlnew file mode 100644index 000000000..db5b8e071--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/feature_binding_step_view_attached/entry.toml@@ -0,0 +1,10 @@+name = "feature_binding_step_view_attached"+description = "The view of a catalog page bound to a name of its own and attached to a prompt as an image"++validity = "valid"+tier = "dry"+granularity = "focused"++covers = [+ "feature.binding_step",+]diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_binding_path_unresolved_past_a_leaf/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_path_unresolved_past_a_leaf/bundle.mthdsnew file mode 100644index 000000000..fd4954474--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_path_unresolved_past_a_leaf/bundle.mthds@@ -0,0 +1,35 @@+domain = "supplier_invoices"+description = "Acknowledging a supplier invoice by the amount it asks for"+main_pipe = "acknowledge_invoice"++[concept.InvoiceLine]+description = "One line of an invoice"++[concept.InvoiceLine.structure]+amount = { type = "number", description = "What the line costs, in euros", required = true }++[concept.Invoice]+description = "An invoice received from a supplier"++[concept.Invoice.structure]+total = { type = "number", description = "The amount due, in euros", required = true }+lines = { type = "list", item_type = "concept", item_concept_ref = "InvoiceLine", description = "The lines of the invoice", required = true }+order_details = { type = "dict", key_type = "text", value_type = "text", description = "The references the supplier printed for the order", required = true }++[pipe.write_receipt]+type = "PipeCompose"+description = "Writes the receipt for an amount"+inputs = { total_amount = "Number" }+output = "Text"+template = "Received: $total_amount euros"++[pipe.acknowledge_invoice]+type = "PipeSequence"+description = "Binds the total of an invoice, then writes the receipt"+inputs = { invoice = "Invoice" }+output = "Text"+# Wrong on purpose: `total` is a number, a leaf, so no segment can follow it.+steps = [+ { from = "invoice.total.amount", result = "total_amount" },+ { pipe = "write_receipt", result = "receipt" },+]diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_binding_path_unresolved_past_a_leaf/entry.toml b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_path_unresolved_past_a_leaf/entry.tomlnew file mode 100644index 000000000..0eb0f6593--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_path_unresolved_past_a_leaf/entry.toml@@ -0,0 +1,12 @@+name = "invalid_binding_path_unresolved_past_a_leaf"+description = "An invoice total walked into as if the number had fields of its own"++validity = "invalid"+tier = "static"+granularity = "focused"++covers = [+ "error.binding_path_unresolved",+]++expected_error = "binding_path_unresolved"diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_binding_path_unresolved_root_without_structure/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_path_unresolved_root_without_structure/bundle.mthdsnew file mode 100644index 000000000..d312a10f7--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_path_unresolved_root_without_structure/bundle.mthds@@ -0,0 +1,21 @@+domain = "delivery_notes"+description = "Passing on the note left for a driver"+main_pipe = "pass_on_note"++[pipe.pass_on_note]+type = "PipeSequence"+description = "Binds the note's text, then writes the reminder"+inputs = { note = "Text" }+output = "Text"+# Wrong on purpose: a Text is a leaf to a binding, so `note.text` walks into nothing; bind `note` itself.+steps = [+ { from = "note.text", result = "note_text" },+ { pipe = "write_reminder", result = "reminder" },+]++[pipe.write_reminder]+type = "PipeCompose"+description = "Writes the reminder handed to the driver"+inputs = { note_text = "Text" }+output = "Text"+template = "Reminder: $note_text"diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_binding_path_unresolved_root_without_structure/entry.toml b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_path_unresolved_root_without_structure/entry.tomlnew file mode 100644index 000000000..c6113214e--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_path_unresolved_root_without_structure/entry.toml@@ -0,0 +1,12 @@+name = "invalid_binding_path_unresolved_root_without_structure"+description = "The text of a delivery note bound through a field of Text, which holds its value whole"++validity = "invalid"+tier = "static"+granularity = "focused"++covers = [+ "error.binding_path_unresolved",+]++expected_error = "binding_path_unresolved"diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_binding_path_unresolved_through_a_dict/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_path_unresolved_through_a_dict/bundle.mthdsnew file mode 100644index 000000000..37126916d--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_path_unresolved_through_a_dict/bundle.mthds@@ -0,0 +1,35 @@+domain = "supplier_invoices"+description = "Acknowledging a supplier invoice by the amount it asks for"+main_pipe = "acknowledge_invoice"++[concept.InvoiceLine]+description = "One line of an invoice"++[concept.InvoiceLine.structure]+amount = { type = "number", description = "What the line costs, in euros", required = true }++[concept.Invoice]+description = "An invoice received from a supplier"++[concept.Invoice.structure]+total = { type = "number", description = "The amount due, in euros", required = true }+lines = { type = "list", item_type = "concept", item_concept_ref = "InvoiceLine", description = "The lines of the invoice", required = true }+order_details = { type = "dict", key_type = "text", value_type = "text", description = "The references the supplier printed for the order", required = true }++[pipe.write_receipt]+type = "PipeCompose"+description = "Writes the receipt for an amount"+inputs = { total_amount = "Number" }+output = "Text"+template = "Received: $total_amount euros"++[pipe.acknowledge_invoice]+type = "PipeSequence"+description = "Binds the total of an invoice, then writes the receipt"+inputs = { invoice = "Invoice" }+output = "Text"+# Wrong on purpose: `order_details` is a dict, so no declared field follows it; bind the dict whole.+steps = [+ { from = "invoice.order_details.order_ref", result = "total_amount" },+ { pipe = "write_receipt", result = "receipt" },+]diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_binding_path_unresolved_through_a_dict/entry.toml b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_path_unresolved_through_a_dict/entry.tomlnew file mode 100644index 000000000..72fccc05e--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_path_unresolved_through_a_dict/entry.toml@@ -0,0 +1,12 @@+name = "invalid_binding_path_unresolved_through_a_dict"+description = "An order reference bound through a dict of references, whose keys no structure declares"++validity = "invalid"+tier = "static"+granularity = "focused"++covers = [+ "error.binding_path_unresolved",+]++expected_error = "binding_path_unresolved"diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_binding_path_unresolved_unknown_field/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_path_unresolved_unknown_field/bundle.mthdsnew file mode 100644index 000000000..e790623b7--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_path_unresolved_unknown_field/bundle.mthds@@ -0,0 +1,35 @@+domain = "supplier_invoices"+description = "Acknowledging a supplier invoice by the amount it asks for"+main_pipe = "acknowledge_invoice"++[concept.InvoiceLine]+description = "One line of an invoice"++[concept.InvoiceLine.structure]+amount = { type = "number", description = "What the line costs, in euros", required = true }++[concept.Invoice]+description = "An invoice received from a supplier"++[concept.Invoice.structure]+total = { type = "number", description = "The amount due, in euros", required = true }+lines = { type = "list", item_type = "concept", item_concept_ref = "InvoiceLine", description = "The lines of the invoice", required = true }+order_details = { type = "dict", key_type = "text", value_type = "text", description = "The references the supplier printed for the order", required = true }++[pipe.write_receipt]+type = "PipeCompose"+description = "Writes the receipt for an amount"+inputs = { total_amount = "Number" }+output = "Text"+template = "Received: $total_amount euros"++[pipe.acknowledge_invoice]+type = "PipeSequence"+description = "Binds the total of an invoice, then writes the receipt"+inputs = { invoice = "Invoice" }+output = "Text"+# Wrong on purpose: `totl` is not a field of Invoice; the message lists the fields it has.+steps = [+ { from = "invoice.totl", result = "total_amount" },+ { pipe = "write_receipt", result = "receipt" },+]diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_binding_path_unresolved_unknown_field/entry.toml b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_path_unresolved_unknown_field/entry.tomlnew file mode 100644index 000000000..bf6776fe1--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_path_unresolved_unknown_field/entry.toml@@ -0,0 +1,12 @@+name = "invalid_binding_path_unresolved_unknown_field"+description = "An invoice total bound through a misspelt field the invoice does not have"++validity = "invalid"+tier = "static"+granularity = "focused"++covers = [+ "error.binding_path_unresolved",+]++expected_error = "binding_path_unresolved"diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_batch_fields/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_batch_fields/bundle.mthdsnew file mode 100644index 000000000..d192034a4--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_batch_fields/bundle.mthds@@ -0,0 +1,36 @@+domain = "supplier_invoices"+description = "Acknowledging a supplier invoice by the amount it asks for"+main_pipe = "acknowledge_invoice"++[concept.InvoiceLine]+description = "One line of an invoice"++[concept.InvoiceLine.structure]+amount = { type = "number", description = "What the line costs, in euros", required = true }++[concept.Invoice]+description = "An invoice received from a supplier"++[concept.Invoice.structure]+total = { type = "number", description = "The amount due, in euros", required = true }+lines = { type = "list", item_type = "concept", item_concept_ref = "InvoiceLine", description = "The lines of the invoice", required = true }+order_details = { type = "dict", key_type = "text", value_type = "text", description = "The references the supplier printed for the order", required = true }++[pipe.write_receipt]+type = "PipeCompose"+description = "Writes the receipt for an amount"+inputs = { total_amount = "Number" }+output = "Text"+template = "Received: $total_amount euros"++[pipe.acknowledge_invoice]+type = "PipeSequence"+description = "Binds the total of an invoice, then writes the receipt"+inputs = { invoice = "Invoice" }+output = "Text"+# Wrong on purpose: a binding step carries only `from` and `result`; batching belongs on the pipe step that reads the bound list.+steps = [+ { from = "invoice.lines", result = "lines", batch_over = "lines", batch_as = "line" },+ { from = "invoice.total", result = "total_amount" },+ { pipe = "write_receipt", result = "receipt" },+]diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_batch_fields/entry.toml b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_batch_fields/entry.tomlnew file mode 100644index 000000000..b8f246c3e--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_batch_fields/entry.toml@@ -0,0 +1,12 @@+name = "invalid_binding_step_batch_fields"+description = "The lines of an invoice bound by a step that also tries to batch over them"++validity = "invalid"+tier = "static"+granularity = "focused"++covers = [+ "error.binding_step_invalid",+]++expected_error = "binding_step_invalid"diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_malformed_path/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_malformed_path/bundle.mthdsnew file mode 100644index 000000000..b1570b429--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_malformed_path/bundle.mthds@@ -0,0 +1,35 @@+domain = "supplier_invoices"+description = "Acknowledging a supplier invoice by the amount it asks for"+main_pipe = "acknowledge_invoice"++[concept.InvoiceLine]+description = "One line of an invoice"++[concept.InvoiceLine.structure]+amount = { type = "number", description = "What the line costs, in euros", required = true }++[concept.Invoice]+description = "An invoice received from a supplier"++[concept.Invoice.structure]+total = { type = "number", description = "The amount due, in euros", required = true }+lines = { type = "list", item_type = "concept", item_concept_ref = "InvoiceLine", description = "The lines of the invoice", required = true }+order_details = { type = "dict", key_type = "text", value_type = "text", description = "The references the supplier printed for the order", required = true }++[pipe.write_receipt]+type = "PipeCompose"+description = "Writes the receipt for an amount"+inputs = { total_amount = "Number" }+output = "Text"+template = "Received: $total_amount euros"++[pipe.acknowledge_invoice]+type = "PipeSequence"+description = "Binds the total of an invoice, then writes the receipt"+inputs = { invoice = "Invoice" }+output = "Text"+# Wrong on purpose: a path is field names separated by dots, so `[0]` is not part of one.+steps = [+ { from = "invoice.lines[0].amount", result = "total_amount" },+ { pipe = "write_receipt", result = "receipt" },+]diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_malformed_path/entry.toml b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_malformed_path/entry.tomlnew file mode 100644index 000000000..20eb87400--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_malformed_path/entry.toml@@ -0,0 +1,12 @@+name = "invalid_binding_step_malformed_path"+description = "The first line of an invoice bound through a subscript, which a binding path cannot hold"++validity = "invalid"+tier = "static"+granularity = "focused"++covers = [+ "error.binding_step_invalid",+]++expected_error = "binding_step_invalid"diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_parallel_branch/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_parallel_branch/bundle.mthdsnew file mode 100644index 000000000..eae6bcb02--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_parallel_branch/bundle.mthds@@ -0,0 +1,36 @@+domain = "supplier_invoices"+description = "Acknowledging a supplier invoice by the amount it asks for"+main_pipe = "acknowledge_invoice"++[concept.InvoiceLine]+description = "One line of an invoice"++[concept.InvoiceLine.structure]+amount = { type = "number", description = "What the line costs, in euros", required = true }++[concept.Invoice]+description = "An invoice received from a supplier"++[concept.Invoice.structure]+total = { type = "number", description = "The amount due, in euros", required = true }+lines = { type = "list", item_type = "concept", item_concept_ref = "InvoiceLine", description = "The lines of the invoice", required = true }+order_details = { type = "dict", key_type = "text", value_type = "text", description = "The references the supplier printed for the order", required = true }++[pipe.write_receipt]+type = "PipeCompose"+description = "Writes the receipt for an amount"+inputs = { total_amount = "Number" }+output = "Text"+template = "Received: $total_amount euros"++[pipe.acknowledge_invoice]+type = "PipeParallel"+description = "Writes the receipt and binds the total at the same time"+inputs = { invoice = "Invoice", total_amount = "Number" }+output = "Composite"+add_each_output = true+# Wrong on purpose: a parallel branch is always a pipe step; bind the total in a sequence before the parallel.+branches = [+ { from = "invoice.total", result = "bound_total" },+ { pipe = "write_receipt", result = "receipt" },+]diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_parallel_branch/entry.toml b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_parallel_branch/entry.tomlnew file mode 100644index 000000000..ded026e16--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_parallel_branch/entry.toml@@ -0,0 +1,12 @@+name = "invalid_binding_step_parallel_branch"+description = "An invoice total bound as a branch of a parallel, where only pipe steps run"++validity = "invalid"+tier = "static"+granularity = "focused"++covers = [+ "error.binding_step_invalid",+]++expected_error = "binding_step_invalid"diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_pipe_and_from/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_pipe_and_from/bundle.mthdsnew file mode 100644index 000000000..3130c7922--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_pipe_and_from/bundle.mthds@@ -0,0 +1,34 @@+domain = "supplier_invoices"+description = "Acknowledging a supplier invoice by the amount it asks for"+main_pipe = "acknowledge_invoice"++[concept.InvoiceLine]+description = "One line of an invoice"++[concept.InvoiceLine.structure]+amount = { type = "number", description = "What the line costs, in euros", required = true }++[concept.Invoice]+description = "An invoice received from a supplier"++[concept.Invoice.structure]+total = { type = "number", description = "The amount due, in euros", required = true }+lines = { type = "list", item_type = "concept", item_concept_ref = "InvoiceLine", description = "The lines of the invoice", required = true }+order_details = { type = "dict", key_type = "text", value_type = "text", description = "The references the supplier printed for the order", required = true }++[pipe.write_receipt]+type = "PipeCompose"+description = "Writes the receipt for an amount"+inputs = { total_amount = "Number" }+output = "Text"+template = "Received: $total_amount euros"++[pipe.acknowledge_invoice]+type = "PipeSequence"+description = "Binds the total of an invoice, then writes the receipt"+inputs = { invoice = "Invoice" }+output = "Text"+# Wrong on purpose: the step carries both `pipe` and `from`, so it neither runs a pipe nor binds a value.+steps = [+ { pipe = "write_receipt", from = "invoice.total", result = "total_amount" },+]diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_pipe_and_from/entry.toml b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_pipe_and_from/entry.tomlnew file mode 100644index 000000000..8e3a12136--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_pipe_and_from/entry.toml@@ -0,0 +1,12 @@+name = "invalid_binding_step_pipe_and_from"+description = "An invoice acknowledged by a step that both runs a pipe and binds a value"++validity = "invalid"+tier = "static"+granularity = "focused"++covers = [+ "error.binding_step_invalid",+]++expected_error = "binding_step_invalid"diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_result_not_plain/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_result_not_plain/bundle.mthdsnew file mode 100644index 000000000..9e5f1808b--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_result_not_plain/bundle.mthds@@ -0,0 +1,35 @@+domain = "supplier_invoices"+description = "Acknowledging a supplier invoice by the amount it asks for"+main_pipe = "acknowledge_invoice"++[concept.InvoiceLine]+description = "One line of an invoice"++[concept.InvoiceLine.structure]+amount = { type = "number", description = "What the line costs, in euros", required = true }++[concept.Invoice]+description = "An invoice received from a supplier"++[concept.Invoice.structure]+total = { type = "number", description = "The amount due, in euros", required = true }+lines = { type = "list", item_type = "concept", item_concept_ref = "InvoiceLine", description = "The lines of the invoice", required = true }+order_details = { type = "dict", key_type = "text", value_type = "text", description = "The references the supplier printed for the order", required = true }++[pipe.write_receipt]+type = "PipeCompose"+description = "Writes the receipt for an amount"+inputs = { total_amount = "Number" }+output = "Text"+template = "Received: $total_amount euros"++[pipe.acknowledge_invoice]+type = "PipeSequence"+description = "Binds the total of an invoice, then writes the receipt"+inputs = { invoice = "Invoice" }+output = "Text"+# Wrong on purpose: a bound value is read under a plain snake_case name, never a PascalCase one.+steps = [+ { from = "invoice.total", result = "TotalAmount" },+ { pipe = "write_receipt", result = "receipt" },+]diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_result_not_plain/entry.toml b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_result_not_plain/entry.tomlnew file mode 100644index 000000000..2eb85448a--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_result_not_plain/entry.toml@@ -0,0 +1,12 @@+name = "invalid_binding_step_result_not_plain"+description = "An invoice total bound under a name that is not a plain input name"++validity = "invalid"+tier = "static"+granularity = "focused"++covers = [+ "error.binding_step_invalid",+]++expected_error = "binding_step_invalid"diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_without_result/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_without_result/bundle.mthdsnew file mode 100644index 000000000..62f0697f0--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_without_result/bundle.mthds@@ -0,0 +1,35 @@+domain = "supplier_invoices"+description = "Acknowledging a supplier invoice by the amount it asks for"+main_pipe = "acknowledge_invoice"++[concept.InvoiceLine]+description = "One line of an invoice"++[concept.InvoiceLine.structure]+amount = { type = "number", description = "What the line costs, in euros", required = true }++[concept.Invoice]+description = "An invoice received from a supplier"++[concept.Invoice.structure]+total = { type = "number", description = "The amount due, in euros", required = true }+lines = { type = "list", item_type = "concept", item_concept_ref = "InvoiceLine", description = "The lines of the invoice", required = true }+order_details = { type = "dict", key_type = "text", value_type = "text", description = "The references the supplier printed for the order", required = true }++[pipe.write_receipt]+type = "PipeCompose"+description = "Writes the receipt for an amount"+inputs = { total_amount = "Number" }+output = "Text"+template = "Received: $total_amount euros"++[pipe.acknowledge_invoice]+type = "PipeSequence"+description = "Binds the total of an invoice, then writes the receipt"+inputs = { invoice = "Invoice" }+output = "Text"+# Wrong on purpose: the binding step has no `result`, so nothing can read what it binds.+steps = [+ { from = "invoice.total" },+ { pipe = "write_receipt", result = "receipt" },+]diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_without_result/entry.toml b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_without_result/entry.tomlnew file mode 100644index 000000000..15e711834--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_binding_step_without_result/entry.toml@@ -0,0 +1,12 @@+name = "invalid_binding_step_without_result"+description = "An invoice total bound by a step that never names the value it binds"++validity = "invalid"+tier = "static"+granularity = "focused"++covers = [+ "error.binding_step_invalid",+]++expected_error = "binding_step_invalid"diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_inadequate_output_multiplicity_binding/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/invalid_inadequate_output_multiplicity_binding/bundle.mthdsnew file mode 100644index 000000000..8701c4294--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_inadequate_output_multiplicity_binding/bundle.mthds@@ -0,0 +1,13 @@+domain = "catalog_pages"+description = "Picking the photograph of a printed catalog"+main_pipe = "pick_page_view"++[pipe.pick_page_view]+type = "PipeSequence"+description = "Binds the views of every page and returns them"+inputs = { pages = "Page[]" }+# Wrong on purpose: binding through a list of pages gives a list of images, and the output promises one.+output = "Image"+steps = [+ { from = "pages.page_view", result = "page_views" },+]diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_inadequate_output_multiplicity_binding/entry.toml b/pipelex/test_extras/mthds_corpus/entries/invalid_inadequate_output_multiplicity_binding/entry.tomlnew file mode 100644index 000000000..584f636d7--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_inadequate_output_multiplicity_binding/entry.toml@@ -0,0 +1,12 @@+name = "invalid_inadequate_output_multiplicity_binding"+description = "The views of a catalog's pages bound as a list and returned by a sequence that promises one image"++validity = "invalid"+tier = "static"+granularity = "focused"++covers = [+ "error.inadequate_output_multiplicity",+]++expected_error = "inadequate_output_multiplicity"diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_input_stuff_spec_mismatch_binding/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/invalid_input_stuff_spec_mismatch_binding/bundle.mthdsnew file mode 100644index 000000000..1c42c1a37--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_input_stuff_spec_mismatch_binding/bundle.mthds@@ -0,0 +1,42 @@+domain = "supplier_invoices"+description = "Acknowledging a supplier invoice by the amount it asks for"+main_pipe = "acknowledge_invoice"++[concept.InvoiceLine]+description = "One line of an invoice"++[concept.InvoiceLine.structure]+amount = { type = "number", description = "What the line costs, in euros", required = true }++[concept.Invoice]+description = "An invoice received from a supplier"++[concept.Invoice.structure]+total = { type = "number", description = "The amount due, in euros", required = true }+lines = { type = "list", item_type = "concept", item_concept_ref = "InvoiceLine", description = "The lines of the invoice", required = true }+order_details = { type = "dict", key_type = "text", value_type = "text", description = "The references the supplier printed for the order", required = true }++[pipe.write_receipt]+type = "PipeCompose"+description = "Writes the receipt for an amount"+inputs = { total_amount = "Number" }+output = "Text"+template = "Received: $total_amount euros"++[pipe.acknowledge_invoice]+type = "PipeSequence"+description = "Binds the total of an invoice, then writes the note"+inputs = { invoice = "Invoice" }+output = "Text"+# Wrong on purpose: `invoice.total` binds a Number, which `write_note` reads as Text.+steps = [+ { from = "invoice.total", result = "total_amount" },+ { pipe = "write_note", result = "note" },+]++[pipe.write_note]+type = "PipeCompose"+description = "Writes a note about an amount written out in words"+inputs = { total_amount = "Text" }+output = "Text"+template = "Amount in words: $total_amount"diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_input_stuff_spec_mismatch_binding/entry.toml b/pipelex/test_extras/mthds_corpus/entries/invalid_input_stuff_spec_mismatch_binding/entry.tomlnew file mode 100644index 000000000..84e832e08--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_input_stuff_spec_mismatch_binding/entry.toml@@ -0,0 +1,12 @@+name = "invalid_input_stuff_spec_mismatch_binding"+description = "An invoice total bound as a number and read by a step that expects text"++validity = "invalid"+tier = "static"+granularity = "focused"++covers = [+ "error.input_stuff_spec_mismatch",+]++expected_error = "input_stuff_spec_mismatch"diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_missing_input_variable_binding_root/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/invalid_missing_input_variable_binding_root/bundle.mthdsnew file mode 100644index 000000000..33493bc2a--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_missing_input_variable_binding_root/bundle.mthds@@ -0,0 +1,35 @@+domain = "supplier_invoices"+description = "Acknowledging a supplier invoice by the amount it asks for"+main_pipe = "acknowledge_invoice"++[concept.InvoiceLine]+description = "One line of an invoice"++[concept.InvoiceLine.structure]+amount = { type = "number", description = "What the line costs, in euros", required = true }++[concept.Invoice]+description = "An invoice received from a supplier"++[concept.Invoice.structure]+total = { type = "number", description = "The amount due, in euros", required = true }+lines = { type = "list", item_type = "concept", item_concept_ref = "InvoiceLine", description = "The lines of the invoice", required = true }+order_details = { type = "dict", key_type = "text", value_type = "text", description = "The references the supplier printed for the order", required = true }++[pipe.write_receipt]+type = "PipeCompose"+description = "Writes the receipt for an amount"+inputs = { total_amount = "Number" }+output = "Text"+template = "Received: $total_amount euros"++[pipe.acknowledge_invoice]+type = "PipeSequence"+description = "Binds the total of an invoice, then writes the receipt"+inputs = { supplier_name = "Text" }+output = "Text"+# Wrong on purpose: `invoice` is neither an input of the sequence nor the result of an earlier step.+steps = [+ { from = "invoice.total", result = "total_amount" },+ { pipe = "write_receipt", result = "receipt" },+]diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_missing_input_variable_binding_root/entry.toml b/pipelex/test_extras/mthds_corpus/entries/invalid_missing_input_variable_binding_root/entry.tomlnew file mode 100644index 000000000..9ff3a011c--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_missing_input_variable_binding_root/entry.toml@@ -0,0 +1,12 @@+name = "invalid_missing_input_variable_binding_root"+description = "An invoice total bound from an invoice the sequence neither declares nor produces"++validity = "invalid"+tier = "static"+granularity = "focused"++covers = [+ "error.missing_input_variable",+]++expected_error = "missing_input_variable"diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_optional_not_handled_binding/bundle.mthds b/pipelex/test_extras/mthds_corpus/entries/invalid_optional_not_handled_binding/bundle.mthdsnew file mode 100644index 000000000..e1f42c164--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_optional_not_handled_binding/bundle.mthds@@ -0,0 +1,20 @@+domain = "delivery_rounds"+description = "Passing on the note left for a delivery round"+main_pipe = "pass_on_note"++[concept.DeliveryRound]+description = "A delivery round driven by one van"++[concept.DeliveryRound.structure]+driver = { type = "text", description = "Who drives the round", required = true }+note = { type = "text", description = "A note left for the driver, when there is one" }++[pipe.pass_on_note]+type = "PipeSequence"+description = "Binds the note of a round and returns it"+inputs = { delivery_round = "DeliveryRound" }+# Wrong on purpose: `note` is not required, so the binding may find nothing, and the output is not declared `Text?`.+output = "Text"+steps = [+ { from = "delivery_round.note", result = "note" },+]diff --git a/pipelex/test_extras/mthds_corpus/entries/invalid_optional_not_handled_binding/entry.toml b/pipelex/test_extras/mthds_corpus/entries/invalid_optional_not_handled_binding/entry.tomlnew file mode 100644index 000000000..cb646cebb--- /dev/null+++ b/pipelex/test_extras/mthds_corpus/entries/invalid_optional_not_handled_binding/entry.toml@@ -0,0 +1,12 @@+name = "invalid_optional_not_handled_binding"+description = "A delivery note that may be missing, bound and returned by a sequence that promises one"++validity = "invalid"+tier = "static"+granularity = "focused"++covers = [+ "error.optional_not_handled",+]++expected_error = "optional_not_handled"diff --git a/pipelex/test_extras/mthds_corpus/vocabulary.toml b/pipelex/test_extras/mthds_corpus/vocabulary.tomlindex ce22a92d7..fd2df5ca0 100644--- a/pipelex/test_extras/mthds_corpus/vocabulary.toml+++ b/pipelex/test_extras/mthds_corpus/vocabulary.toml@@ -152,6 +152,14 @@ fails_at = "schema"code = "batch_item_name_collision"fails_at = "runtime"+[error.binding_step_invalid]+code = "binding_step_invalid"+fails_at = "schema"++[error.binding_path_unresolved]+code = "binding_path_unresolved"+fails_at = "runtime"+[error.optional_marker_invalid]code = "optional_marker_invalid"fails_at = "runtime"@@ -232,6 +240,9 @@ excluded = "Advisory-only: the intent-hints lint rides the validation report's `code = "hint_inapplicable_intent"excluded = "Advisory-only: the intent-hints lint rides the validation report's `warnings` array and never makes a verdict invalid (hints are non-normative; well-formed unknown content is preserved). Same shape problem as `optional_force_redundant`: an entry contract keyed on `expected_error` has nowhere to put a warning on a valid entry. Revisit if the manifest ever grows a warnings axis."+[feature.binding_step]+description = "A PipeSequence step `{ from = \"invoice.total\", result = \"total_amount\" }` that binds the value at a path in working memory to a new name: its concept is derived from the declared structures the path walks, its value is a deep copy taken when the step runs, a path crossing a list binds a flat list, and a path reaching nothing records an absence."+[feature.intent_hints]description = "An `intent` hint authored in a `hints` table — on a concept, a structure field, or an expanded input slot — carrying non-normative presentation intent (`prose`, `label`, `rating`, `quantity`) that renderers may honor and every other consumer safely ignores."diff --git a/pipelex/tools/misc/string_utils.py b/pipelex/tools/misc/string_utils.pyindex 79e448ea2..74666ef7d 100644--- a/pipelex/tools/misc/string_utils.py+++ b/pipelex/tools/misc/string_utils.py@@ -279,8 +279,27 @@ def snake_to_title_case(snake_str: str) -> str:return snake_str.replace("_", " ").title()+# The two identifier grammars of MTHDS, written once as unanchored regex fragments that every anchored+# pattern is composed from, so the grammars of pipe codes, input names, a binding step's `result` and the+# segments of a field path cannot drift apart:+# - a snake_case identifier, the grammar of a pipe code, of a domain segment and of an input name;+# - a field-path segment, the grammar of each segment of a path a binding step reads: a letter, then+# letters, digits and underscores, so a segment never starts with the underscore that marks a private name.+SNAKE_CASE_IDENTIFIER_REGEX = r"[a-z][a-z0-9_]*"+FIELD_PATH_SEGMENT_REGEX = r"[A-Za-z][A-Za-z0-9_]*"++# The anchored forms, as a JSON Schema `pattern` writes them.+SNAKE_CASE_PATTERN = rf"^{SNAKE_CASE_IDENTIFIER_REGEX}$"+FIELD_PATH_PATTERN = rf"^{FIELD_PATH_SEGMENT_REGEX}(?:\.{FIELD_PATH_SEGMENT_REGEX})*$"++def is_snake_case(word: str) -> bool:- return re.match(r"^[a-z][a-z0-9_]*$", word) is not None+ return re.fullmatch(SNAKE_CASE_IDENTIFIER_REGEX, word) is not None+++def is_field_path(*, path: str) -> bool:+ """Whether a string is a field path: segments matching ``FIELD_PATH_SEGMENT_REGEX``, separated by single dots."""+ return re.fullmatch(FIELD_PATH_PATTERN, path) is not Nonedef is_pascal_case(word: str) -> bool:diff --git a/pipelex/validation_error_types.py b/pipelex/validation_error_types.pyindex c8a567be0..915d7eaad 100644--- a/pipelex/validation_error_types.py+++ b/pipelex/validation_error_types.py@@ -60,6 +60,16 @@ class PipeValidationErrorType(StrEnum):MISSING_PIPE_TYPE = "missing_pipe_type"BATCH_ITEM_NAME_COLLISION = "batch_item_name_collision"+ # A PipeSequence step whose shape breaks the binding-step rules: a step carrying both `pipe` and `from`,+ # a binding step without `result` or carrying a pipe step's `nb_output`, `multiple_output`, `batch_over`+ # or `batch_as`, a binding `result` that is not a plain input name, a `from` outside the path grammar, or+ # a binding step in a PipeParallel's `branches`. Detected at blueprint parse time, as the schema does.+ BINDING_STEP_INVALID = "binding_step_invalid"+ # A binding step's `from` path that the declared structures cannot walk: a segment naming no field, a+ # segment after a leaf, a dict or a list with no `item_type`, a concept with no walkable structure, or a+ # path ending on a list with no `item_type`. Detected when the sequence is validated against the library.+ BINDING_PATH_UNRESOLVED = "binding_path_unresolved"+# Presence-marker grammar misuse, detected at blueprint parse time: a presence marker# (`?` or `!`) combined with a multiplicity suffix, or `!` on an output (D1, D4 of the# optionals design).@@ -129,6 +139,8 @@ class PipeValidationErrorType(StrEnum):| PipeValidationErrorType.UNKNOWN_PIPE_TYPE| PipeValidationErrorType.MISSING_PIPE_TYPE| PipeValidationErrorType.BATCH_ITEM_NAME_COLLISION+ | PipeValidationErrorType.BINDING_STEP_INVALID+ | PipeValidationErrorType.BINDING_PATH_UNRESOLVED| PipeValidationErrorType.OPTIONAL_MARKER_INVALID| PipeValidationErrorType.OPTIONAL_NOT_HANDLED| PipeValidationErrorType.OPTIONAL_OUTPUT_REQUIRED@@ -162,6 +174,8 @@ class PipeValidationErrorType(StrEnum):| PipeValidationErrorType.UNKNOWN_PIPE_TYPE| PipeValidationErrorType.MISSING_PIPE_TYPE| PipeValidationErrorType.BATCH_ITEM_NAME_COLLISION+ | PipeValidationErrorType.BINDING_STEP_INVALID+ | PipeValidationErrorType.BINDING_PATH_UNRESOLVED| PipeValidationErrorType.OPTIONAL_MARKER_INVALID| PipeValidationErrorType.OPTIONAL_NOT_HANDLED| PipeValidationErrorType.OPTIONAL_OUTPUT_REQUIRED@@ -202,6 +216,8 @@ class PipeValidationErrorType(StrEnum):| PipeValidationErrorType.UNKNOWN_PIPE_TYPE| PipeValidationErrorType.MISSING_PIPE_TYPE| PipeValidationErrorType.BATCH_ITEM_NAME_COLLISION+ | PipeValidationErrorType.BINDING_STEP_INVALID+ | PipeValidationErrorType.BINDING_PATH_UNRESOLVED| PipeValidationErrorType.OPTIONAL_MARKER_INVALID| PipeValidationErrorType.OPTIONAL_NOT_HANDLED| PipeValidationErrorType.OPTIONAL_OUTPUT_REQUIRED@@ -237,6 +253,8 @@ class PipeValidationErrorType(StrEnum):| PipeValidationErrorType.UNKNOWN_PIPE_TYPE| PipeValidationErrorType.MISSING_PIPE_TYPE| PipeValidationErrorType.BATCH_ITEM_NAME_COLLISION+ | PipeValidationErrorType.BINDING_STEP_INVALID+ | PipeValidationErrorType.BINDING_PATH_UNRESOLVED| PipeValidationErrorType.OPTIONAL_MARKER_INVALID| PipeValidationErrorType.OPTIONAL_NOT_HANDLED| PipeValidationErrorType.OPTIONAL_OUTPUT_REQUIRED@@ -275,6 +293,8 @@ class PipeValidationErrorType(StrEnum):| PipeValidationErrorType.UNKNOWN_PIPE_TYPE| PipeValidationErrorType.MISSING_PIPE_TYPE| PipeValidationErrorType.BATCH_ITEM_NAME_COLLISION+ | PipeValidationErrorType.BINDING_STEP_INVALID+ | PipeValidationErrorType.BINDING_PATH_UNRESOLVED| PipeValidationErrorType.OPTIONAL_MARKER_INVALID| PipeValidationErrorType.OPTIONAL_NOT_HANDLED| PipeValidationErrorType.OPTIONAL_OUTPUT_REQUIRED@@ -314,6 +334,8 @@ class PipeValidationErrorType(StrEnum):| PipeValidationErrorType.UNKNOWN_PIPE_TYPE| PipeValidationErrorType.MISSING_PIPE_TYPE| PipeValidationErrorType.BATCH_ITEM_NAME_COLLISION+ | PipeValidationErrorType.BINDING_STEP_INVALID+ | PipeValidationErrorType.BINDING_PATH_UNRESOLVED| PipeValidationErrorType.OPTIONAL_MARKER_INVALID| PipeValidationErrorType.OPTIONAL_NOT_HANDLED| PipeValidationErrorType.OPTIONAL_OUTPUT_REQUIRED@@ -349,6 +371,8 @@ class PipeValidationErrorType(StrEnum):| PipeValidationErrorType.UNKNOWN_PIPE_TYPE| PipeValidationErrorType.MISSING_PIPE_TYPE| PipeValidationErrorType.BATCH_ITEM_NAME_COLLISION+ | PipeValidationErrorType.BINDING_STEP_INVALID+ | PipeValidationErrorType.BINDING_PATH_UNRESOLVED| PipeValidationErrorType.OPTIONAL_MARKER_INVALID| PipeValidationErrorType.OPTIONAL_NOT_HANDLED| PipeValidationErrorType.OPTIONAL_OUTPUT_REQUIREDdiff --git a/subject_grants.toml b/subject_grants.tomlindex 5da239fea..56294bbee 100644--- a/subject_grants.toml+++ b/subject_grants.toml@@ -4012,10 +4012,6 @@ rationale = "Predicate on the pipe's step result; modifiers stay keyword."param = "pipe"rationale = "Verb-object: scans the pipe for taint triggers; slot_taints is context."-["pipelex/pipe_controllers/batch/pipe_batch.py::PipeBatch._live_run_controller_pipe._run_branch"]-param = "item_input_stuff"-rationale = "Verb-object: runs the branch for the item stuff; invoked positionally via functools.partial."-["pipelex/pipe_controllers/condition/special_outcome.py::SpecialOutcome.is_continue"]param = "outcome"rationale = "Predicate on the outcome string; single operand."diff --git a/tests/data/errors/error_identity.txt b/tests/data/errors/error_identity.txtindex 14c085131..44b6caca3 100644--- a/tests/data/errors/error_identity.txt+++ b/tests/data/errors/error_identity.txt@@ -20,6 +20,8 @@ AzureCredentialsError | Azure credentials | https://docs.pipelex.com/latest/erroBatchParamsError | Batch params | https://docs.pipelex.com/latest/errors/batch-params-error/BedrockFactoryError | Bedrock factory | https://docs.pipelex.com/latest/errors/bedrock-factory-error/BedrockWorkerConfigurationError | Bedrock worker configuration | https://docs.pipelex.com/latest/errors/bedrock-worker-configuration-error/+BindingPathUnresolvedError | Binding path unresolved | https://docs.pipelex.com/latest/errors/binding-path-unresolved-error/+BindingStepRunError | Binding step run | https://docs.pipelex.com/latest/errors/binding-step-run-error/BrokenPluginError | Broken plugin | https://docs.pipelex.com/latest/errors/broken-plugin-error/BundleElaboratorError | Bundle elaborator | https://docs.pipelex.com/latest/errors/bundle-elaborator-error/CodegenError | Codegen error | https://docs.pipelex.com/latest/errors/codegen-error/diff --git a/tests/integration/pipelex/codegen/test_resolve_flow.py b/tests/integration/pipelex/codegen/test_resolve_flow.pyindex 64ba36eb5..7a7e55081 100644--- a/tests/integration/pipelex/codegen/test_resolve_flow.py+++ b/tests/integration/pipelex/codegen/test_resolve_flow.py@@ -101,7 +101,7 @@ class TestResolveFlow:# Cross-file pipe ref inside the sequence controller qualified against the owner domain.run_pipeline = normalized.pipes["pipeline.run_pipeline"]assert isinstance(run_pipeline, PipeSequenceBlueprint)- assert run_pipeline.steps[0].pipe == "pipeline.compute_score"+ assert run_pipeline.pipe_steps[0].pipe == "pipeline.compute_score"# Both encodings agree on the fingerprint.json_doc = json.loads(encode_crate_json(normalized))diff --git a/tests/integration/pipelex/libraries/installed_packages.py b/tests/integration/pipelex/libraries/installed_packages.pyindex e93c733af..f24c6a353 100644--- a/tests/integration/pipelex/libraries/installed_packages.py+++ b/tests/integration/pipelex/libraries/installed_packages.py@@ -49,7 +49,7 @@ def sole_step_target(*, library: Library, pipe_key: str) -> str:pipe = library.pipe_library.get_required_pipe(pipe_code=pipe_key)assert isinstance(pipe, PipeSequence)assert len(pipe.sequential_sub_pipes) == 1- return pipe.sequential_sub_pipes[0].pipe_code+ return pipe.pipe_steps[0].pipe_codedef refusal_types(exc: LibraryLoadingError) -> list[PipeValidationErrorType | None]:diff --git a/tests/integration/pipelex/libraries/test_data.py b/tests/integration/pipelex/libraries/test_data.pyindex 800721dfe..3ca73fde9 100644--- a/tests/integration/pipelex/libraries/test_data.py+++ b/tests/integration/pipelex/libraries/test_data.py@@ -361,3 +361,144 @@ inputs = { data = "Text" }output = "Text"prompt = "private x: $data""""+++class LedgerPackageTestData:+ """An installed method package whose sequence binds fields of its own `Invoice`, and consumers binding the same; nothing here is real.++ A package's concept reaches a consumer's sequence as the output of one of the package's pipes, or as a field the consumer types+ with the package's alias. Its concepts are held in the consumer's library only under that alias, so a binding walking them must+ read them there, and never a consumer concept spelled the same.+ """++ METHOD_NAME: ClassVar[str] = "ledger"+ DEP_ALIAS: ClassVar[str] = "github.com/invented/ledger-lib/ledger"++ DEP_MANIFEST: ClassVar[str] = """[package]+name = "ledger"+address = "github.com/invented/ledger-lib"+version = "1.0.0"+description = "An invented ledger package"++[exports.invented_ledger]+pipes = ["make_invoice", "read_invoice", "write_line"]+"""++ DEP_BUNDLE: ClassVar[str] = """domain = "invented_ledger"+description = "An invented ledger package"++[concept.Supplier]+description = "Who sent an invoice"++[concept.Supplier.structure]+name = { type = "text", description = "The supplier's name", required = true }++[concept.Invoice]+description = "An invoice kept in the ledger"++[concept.Invoice.structure]+total = { type = "number", description = "The amount due, in euros", required = true }+supplier = { type = "concept", concept_ref = "Supplier", description = "Who sent it", required = true }++[pipe.make_invoice]+type = "PipeCompose"+description = "Writes out an invoice for an amount and a supplier"+inputs = { amount = "Number", sender = "Text" }+output = "Invoice"++[pipe.make_invoice.construct]+total = { from = "amount.number" }+supplier = { name = { from = "sender.text" } }++[pipe.write_line]+type = "PipeCompose"+description = "Writes the ledger line of a supplier and an amount"+inputs = { supplier_name = "Text", total_amount = "Number" }+output = "Text"+template = "$supplier_name: $total_amount euros"++[pipe.read_invoice]+type = "PipeSequence"+description = "Binds the supplier's name and the total of an invoice, then writes its ledger line"+inputs = { invoice = "Invoice" }+output = "Text"+steps = [+ { from = "invoice.supplier.name", result = "supplier_name" },+ { from = "invoice.total", result = "total_amount" },+ { pipe = "write_line", result = "line" },+]+"""++ #: The consumer binds the fields of an invoice the package makes, and has the package read one with its own bindings.+ CONSUMER_BUNDLE: ClassVar[str] = """domain = "invented_books"+description = "A consumer of the invented ledger package"++[pipe.check_invoice]+type = "PipeSequence"+description = "Has the package make an invoice, binds its fields, then writes the package's ledger line"+inputs = { amount = "Number", sender = "Text" }+output = "Text"+steps = [+ { pipe = "github.com/invented/ledger-lib/ledger->invented_ledger.make_invoice", result = "invoice" },+ { from = "invoice.supplier.name", result = "supplier_name" },+ { from = "invoice.total", result = "total_amount" },+ { pipe = "github.com/invented/ledger-lib/ledger->invented_ledger.write_line", result = "line" },+]++[pipe.relay_invoice]+type = "PipeSequence"+description = "Has the package make an invoice and read it with its own bindings"+inputs = { amount = "Number", sender = "Text" }+output = "Text"+steps = [+ { pipe = "github.com/invented/ledger-lib/ledger->invented_ledger.make_invoice", result = "invoice" },+ { pipe = "github.com/invented/ledger-lib/ledger->invented_ledger.read_invoice", result = "line" },+]+"""++ #: A consumer bundle declaring `invented_ledger.Invoice` and `invented_ledger.Supplier` of its own, with other fields, and a+ #: concept of its own whose field the consumer types with the package's `Invoice`.+ CONSUMER_NAMESAKE_BUNDLE: ClassVar[str] = """domain = "invented_ledger"+description = "The consumer's own concepts, in a domain named like the package's"++[concept.Supplier]+description = "The consumer's own idea of a supplier"++[concept.Supplier.structure]+city = { type = "text", description = "Where the supplier is", required = true }++[concept.Invoice]+description = "The consumer's own idea of an invoice"++[concept.Invoice.structure]+amount = { type = "number", description = "The amount, in the consumer's words", required = true }+supplier = { type = "concept", concept_ref = "Supplier", description = "Who sent it", required = true }++[concept.Filing]+description = "The consumer's filing of an invoice the package keeps"++[concept.Filing.structure.invoice]+type = "concept"+concept_ref = "github.com/invented/ledger-lib/ledger->invented_ledger.Invoice"+description = "The invoice filed"+required = true++[pipe.read_own_invoice]+type = "PipeSequence"+description = "Binds the fields of the consumer's own invoice"+inputs = { invoice = "Invoice" }+output = "Text"+steps = [+ { from = "invoice.amount", result = "amount" },+ { from = "invoice.supplier.city", result = "city" },+]++[pipe.read_filing]+type = "PipeSequence"+description = "Binds the supplier's name off the package's invoice a filing holds"+inputs = { filing = "Filing" }+output = "Text"+steps = [+ { from = "filing.invoice.supplier.name", result = "supplier_name" },+]+"""diff --git a/tests/integration/pipelex/libraries/test_dependency_binding_steps.py b/tests/integration/pipelex/libraries/test_dependency_binding_steps.pynew file mode 100644index 000000000..98cd9a364--- /dev/null+++ b/tests/integration/pipelex/libraries/test_dependency_binding_steps.py@@ -0,0 +1,134 @@+"""A binding step walks a dependency package's concept through the package's own definitions, from the consumer and from the package.++A package's concepts reach the consumer's library only under the package's alias (`alias->domain.Code`), while each one reports+the plain `domain.Code` its package declares. So a binding whose root holds a package's concept, the output of one of its pipes in+a consumer's sequence or an input of the package's own sequence, must start its walk from the aliased key, and follow the package's+own fields from there, never a consumer concept spelled the same. The packages are invented and installed under a temporary+`.mthds/methods/`.+"""++from pathlib import Path+from typing import TYPE_CHECKING++import pytest+from pytest_mock import MockerFixture++from pipelex.interpreter_hub import get_library_manager, scoped_current_library+from pipelex.libraries.library import Library+from pipelex.pipe_controllers.sequence.pipe_sequence import PipeSequence+from pipelex.pipeline.runner import PipelexMTHDSProtocol+from pipelex.system.pipe_run_mode import PipeRunMode+from tests.integration.pipelex.libraries.installed_packages import install_package, isolate_installed_methods, write_consumer+from tests.integration.pipelex.libraries.test_data import LedgerPackageTestData++if TYPE_CHECKING:+ from mthds.protocol.pipeline_inputs import PipelineInputs++LEDGER_ALIAS = LedgerPackageTestData.DEP_ALIAS+++def _derived_concept_refs(*, library: Library, pipe_key: str) -> list[str]:+ """The concepts the binding steps of the sequence held under `pipe_key` derive, in step order."""+ sequence = library.pipe_library.get_required_pipe(pipe_code=pipe_key)+ assert isinstance(sequence, PipeSequence)+ flow = sequence.build_typed_flow()+ return [flow.binding_specs[step_index].concept.concept_ref for step_index in sorted(flow.binding_specs)]+++def _install_ledger(*, root: Path) -> None:+ install_package(+ root=root,+ method_name=LedgerPackageTestData.METHOD_NAME,+ manifest=LedgerPackageTestData.DEP_MANIFEST,+ bundles={"invented_ledger.mthds": LedgerPackageTestData.DEP_BUNDLE},+ )+++class TestDependencyBindingSteps:+ @pytest.mark.parametrize(+ "consumer_bundles",+ [+ pytest.param({"books.mthds": LedgerPackageTestData.CONSUMER_BUNDLE}, id="the-package-s-concepts-alone"),+ pytest.param(+ {"books.mthds": LedgerPackageTestData.CONSUMER_BUNDLE, "own_ledger.mthds": LedgerPackageTestData.CONSUMER_NAMESAKE_BUNDLE},+ id="beside-consumer-concepts-of-the-same-spelling",+ ),+ ],+ )+ def test_bindings_over_a_package_concept_derive_from_the_package_s_structure(+ self, tmp_path: Path, mocker: MockerFixture, consumer_bundles: dict[str, str]+ ) -> None:+ isolate_installed_methods(mocker=mocker, root=tmp_path)+ _install_ledger(root=tmp_path)+ consumer_files = write_consumer(root=tmp_path, bundles=consumer_bundles)+ library_manager = get_library_manager()+ library_id, library = library_manager.open_library()+ try:+ library_manager.load_libraries(library_id=library_id, library_file_paths=consumer_files)++ with scoped_current_library(library_id=library_id):+ derived: dict[str, list[str]] = {}+ for pipe_key in ("invented_books.check_invoice", f"{LEDGER_ALIAS}->invented_ledger.read_invoice"):+ derived[pipe_key] = _derived_concept_refs(library=library, pipe_key=pipe_key)+ finally:+ library_manager.teardown(library_id=library_id)++ assert derived == {+ "invented_books.check_invoice": ["native.Text", "native.Number"],+ f"{LEDGER_ALIAS}->invented_ledger.read_invoice": ["native.Text", "native.Number"],+ }++ def test_consumer_concepts_beside_the_package_s_keep_their_own_structures(self, tmp_path: Path, mocker: MockerFixture) -> None:+ """The consumer's namesakes walk their own fields, and the consumer's field typed with the package's alias walks the package's."""+ isolate_installed_methods(mocker=mocker, root=tmp_path)+ _install_ledger(root=tmp_path)+ consumer_files = write_consumer(+ root=tmp_path,+ bundles={"books.mthds": LedgerPackageTestData.CONSUMER_BUNDLE, "own_ledger.mthds": LedgerPackageTestData.CONSUMER_NAMESAKE_BUNDLE},+ )+ library_manager = get_library_manager()+ library_id, library = library_manager.open_library()+ try:+ library_manager.load_libraries(library_id=library_id, library_file_paths=consumer_files)++ with scoped_current_library(library_id=library_id):+ own_invoice_refs = _derived_concept_refs(library=library, pipe_key="invented_ledger.read_own_invoice")+ filing_refs = _derived_concept_refs(library=library, pipe_key="invented_ledger.read_filing")+ finally:+ library_manager.teardown(library_id=library_id)++ assert own_invoice_refs == ["native.Number", "native.Text"]+ assert filing_refs == ["native.Text"]++ @pytest.mark.asyncio(loop_scope="class")+ @pytest.mark.parametrize("pipe_code", ["check_invoice", "relay_invoice"])+ async def test_a_binding_over_a_package_concept_runs(self, tmp_path: Path, mocker: MockerFixture, pipe_code: str) -> None:+ """Run live: the consumer's own bindings, and the package's sequence rebuilding its flow against the consumer's library."""+ isolate_installed_methods(mocker=mocker, root=tmp_path)+ _install_ledger(root=tmp_path)+ # The consumer's own directory, so the library directory holds the consumer's bundle alone and the package is found+ # installed under `.mthds/methods/` above it, never loaded as one of the consumer's bundles.+ consumer_dir = tmp_path / "books"+ consumer_dir.mkdir()+ write_consumer(root=consumer_dir, bundles={"books.mthds": LedgerPackageTestData.CONSUMER_BUNDLE})+ # What the run's library held when it was torn down: the packages it loaded, and whether it held the package's+ # `Invoice` under its plain spelling, which only a consumer bundle would put there.+ library_states: list[tuple[set[str], bool]] = []+ original_teardown = Library.teardown++ def recording_teardown(library: Library) -> None:+ library_states.append(+ (set(library.dependency_libraries), library.concept_library.is_concept_exists(concept_ref="invented_ledger.Invoice"))+ )+ original_teardown(library)++ mocker.patch.object(Library, "teardown", autospec=True, side_effect=recording_teardown)++ inputs: PipelineInputs = {"amount": {"concept": "native.Number", "content": {"number": 1250.5}}, "sender": "Atelier Morvan"}+ result = await PipelexMTHDSProtocol(library_dirs=[str(consumer_dir)], pipe_run_mode=PipeRunMode.LIVE).execute(+ pipe_code=f"invented_books.{pipe_code}",+ inputs=inputs,+ )++ assert ({LEDGER_ALIAS}, False) in library_states+ assert result.pipe_output.main_stuff.as_text.text == "Atelier Morvan: 1250.5 euros"diff --git a/tests/integration/pipelex/pipeline/test_fix_convergence_loop.py b/tests/integration/pipelex/pipeline/test_fix_convergence_loop.pyindex 930021a3b..ca2341679 100644--- a/tests/integration/pipelex/pipeline/test_fix_convergence_loop.py+++ b/tests/integration/pipelex/pipeline/test_fix_convergence_loop.py@@ -450,6 +450,22 @@ $draft.text\"\"\""""+_UNREADABLE_DOTTED_INPUT_MTHDS = """domain = "dottedfix_unreadable"+main_pipe = "summarize_draft"++# Wrong on purpose: a dotted input key beside a root declared with a concept that lacks the field. The key+# typed the root `Text` under the fold, so the template read worked; deleting it leaves `data` a Number.+[pipe.summarize_draft]+type = "PipeLLM"+description = "Summarize the text of a draft."+inputs = { data = "Number", "data.text" = "Text" }+output = "Text"+prompt = \"\"\"+Summarize this draft:+$data.text+\"\"\"+"""+_LONE_DOTTED_INPUT_MTHDS = """domain = "dottedfix_lone"main_pipe = "describe_page"@@ -1028,3 +1044,25 @@ class TestFixConvergenceLoop:assert "replace `?` with `!` on 'draft' if the root must be forced" in remaining.suggested_fix.descriptionassert remaining.message is not Noneassert "Deleting 'draft.text' drops its forced presence (`!`), where 'draft' is optional (`?`)" in remaining.message++ async def test_unreadable_dotted_input_is_offered_but_never_applied(+ self,+ tmp_path: Path,+ load_empty_library: Callable[[], str],+ ) -> None:+ """A dotted input whose path the root's declared concept cannot hold gets an UNSAFE fix naming the walk's refusal, never applied."""+ load_empty_library()+ bundle_path = tmp_path / "unreadable_dotted.mthds"+ bundle_path.write_text(_UNREADABLE_DOTTED_INPUT_MTHDS, encoding="utf-8")++ result = await fix_bundle_file(bundle_path)++ assert result.is_valid is False+ assert result.fixes_applied == []+ assert bundle_path.read_text(encoding="utf-8") == _UNREADABLE_DOTTED_INPUT_MTHDS+ assert [item.error_type for item in result.remaining_errors] == ["invalid_input_name"]+ remaining = result.remaining_errors[0]+ assert remaining.suggested_fix is not None+ assert remaining.suggested_fix.fix_code == "delete-redundant-dotted-input"+ assert not remaining.suggested_fix.safety.is_safe+ assert "'data' holds a 'native.Number', which has no field 'text'" in remaining.suggested_fix.descriptiondiff --git a/tests/integration/pipelex/pipes/controller/pipe_batch/test_pipe_batch_item_isolation.py b/tests/integration/pipelex/pipes/controller/pipe_batch/test_pipe_batch_item_isolation.pynew file mode 100644index 000000000..8d33ef9d1--- /dev/null+++ b/tests/integration/pipelex/pipes/controller/pipe_batch/test_pipe_batch_item_isolation.py@@ -0,0 +1,60 @@+import pytest++from pipelex.core.memory.working_memory import WorkingMemory+from pipelex.core.stuffs.text_content import TextContent+from pipelex.pipeline.runner import PipelexMTHDSProtocol+from pipelex.system.pipe_run_mode import PipeRunMode+from pipelex.system.registries.func_registry import func_registry++_BUNDLE = """+domain = "batch_item_isolation"+description = "A batch whose branch function rewrites the item it is given"+main_pipe = "shout_every_note"++[pipe.shout_every_note]+type = "PipeBatch"+description = "Shouts every note"+inputs = { notes = "Text[]" }+output = "Text[]"+branch_pipe_code = "shout_note"+input_list_name = "notes"+input_item_name = "note"++[pipe.shout_note]+type = "PipeFunc"+description = "Shouts one note, rewriting the item it was handed in place"+inputs = { note = "Text" }+output = "Text"+function_name = "batch_item_isolation_shout_note"+"""+++def batch_item_isolation_shout_note(working_memory: WorkingMemory) -> TextContent:+ note = working_memory.get_stuff_as_text(name="note")+ note.text = note.text.upper()+ return TextContent(text=f"{note.text}!")++[email protected](loop_scope="class")+class TestPipeBatchItemIsolation:+ @classmethod+ def setup_class(cls):+ func_registry.register_function(batch_item_isolation_shout_note)++ @classmethod+ def teardown_class(cls):+ if func_registry.has_function(batch_item_isolation_shout_note.__name__):+ func_registry.unregister_function_by_name(batch_item_isolation_shout_note.__name__)++ async def test_a_branch_rewriting_its_item_leaves_the_batched_list_unchanged(self) -> None:+ """Each branch gets its own copy of its item, as it gets its own copy of the rest of working memory."""+ result = await PipelexMTHDSProtocol(pipe_run_mode=PipeRunMode.LIVE).execute(+ mthds_contents=[_BUNDLE],+ inputs={"notes": {"concept": "native.Text", "content": [{"text": "bread"}, {"text": "milk"}]}},+ )++ working_memory = result.pipe_output.working_memory+ shouted = working_memory.main_stuff_as_list(item_type=TextContent)+ assert [item.text for item in shouted.items] == ["BREAD!", "MILK!"]+ notes = working_memory.get_stuff_as_list("notes", item_type=TextContent)+ assert [item.text for item in notes.items] == ["bread", "milk"]diff --git a/tests/integration/pipelex/pipes/controller/pipe_sequence/test_binding_graph.py b/tests/integration/pipelex/pipes/controller/pipe_sequence/test_binding_graph.pynew file mode 100644index 000000000..5b5143433--- /dev/null+++ b/tests/integration/pipelex/pipes/controller/pipe_sequence/test_binding_graph.py@@ -0,0 +1,125 @@+from typing import Any++import pytest+from pytest_mock import MockerFixture++from pipelex.base_exceptions import PipelexError+from pipelex.config import get_config+from pipelex.graph.graph_tracer import GraphTracer+from pipelex.graph.graphspec import EdgeKind, GraphSpec, NodeKind, NodeSpec, NodeStatus+from pipelex.pipe_controllers.binding.binding_step import BINDING_NODE_PIPE_TYPE+from pipelex.pipeline.runner import PipelexMTHDSProtocol+from pipelex.runtime_hub import scoped_event_log+from pipelex.system.pipe_run_mode import PipeRunMode+from pipelex.tracing.in_memory_event_log import InMemoryEventLog++_BUNDLE = """+domain = "binding_graph"+description = "Acknowledging an invoice by its total, to draw the binding in the graph"+main_pipe = "acknowledge_invoice"++[concept.Invoice]+description = "An invoice received from a supplier"++[concept.Invoice.structure]+total = { type = "number", description = "The amount due, in euros", required = true }+note = { type = "text", description = "A note from the supplier, when there is one" }++[pipe.make_invoice]+type = "PipeCompose"+description = "Writes out an invoice for an amount"+inputs = { amount = "Number" }+output = "Invoice"++[pipe.make_invoice.construct]+total = { from = "amount.number" }++[pipe.acknowledge_invoice]+type = "PipeSequence"+description = "Makes an invoice, binds its total and its note, then writes the receipt"+inputs = { amount = "Number" }+output = "Text"+steps = [+ { pipe = "make_invoice", result = "invoice" },+ { from = "invoice.total", result = "total_amount" },+ { from = "invoice.note", result = "supplier_note" },+ { pipe = "write_receipt", result = "receipt" },+]++[pipe.write_receipt]+type = "PipeCompose"+description = "Writes the receipt for an amount"+inputs = { total_amount = "Number" }+output = "Text"+template = "Received: $total_amount euros"+"""+++def _in_process_graph(teardown_spy: Any) -> GraphSpec:+ graphs = [graph for graph in teardown_spy.spy_return_list if isinstance(graph, GraphSpec)]+ assert len(graphs) == 1+ return graphs[0]+++def _node(graph: GraphSpec, *, pipe_code: str) -> NodeSpec:+ nodes = [node for node in graph.nodes if node.pipe_code == pipe_code]+ assert len(nodes) == 1, f"expected one node for '{pipe_code}', got {[(node.kind, node.pipe_code) for node in graph.nodes]}"+ return nodes[0]+++def _data_edges(graph: GraphSpec) -> set[tuple[str, str]]:+ return {(edge.source, edge.target) for edge in graph.edges if edge.kind == EdgeKind.DATA}++[email protected](loop_scope="class")+class TestBindingGraph:+ async def test_a_binding_is_a_node_producing_what_it_binds(self, mocker: MockerFixture) -> None:+ """The binding is a `binding` node fed by the root's producer, and the producer of the stuff it binds."""+ teardown_spy = mocker.spy(GraphTracer, "teardown")+ execution_config = get_config().interpreter.pipeline_execution.with_execution_overrides(generate_graph=True)+ runner = PipelexMTHDSProtocol(execution_config=execution_config, pipe_run_mode=PipeRunMode.LIVE)++ with scoped_event_log(InMemoryEventLog()):+ response = await runner.execute(mthds_contents=[_BUNDLE], inputs={"amount": {"concept": "native.Number", "content": {"number": 318}}})++ assert response.pipe_output.main_stuff.as_text.text == "Received: 318.0 euros"+ assembled_graph = response.pipe_output.graph_spec+ assert assembled_graph is not None+ in_process_graph = _in_process_graph(teardown_spy)++ for graph in (assembled_graph, in_process_graph):+ binding_node = _node(graph, pipe_code="invoice.total")+ assert binding_node.kind == NodeKind.BINDING+ assert binding_node.pipe_type == BINDING_NODE_PIPE_TYPE+ assert binding_node.status == NodeStatus.SUCCEEDED+ assert [io_spec.name for io_spec in binding_node.node_io.inputs] == ["invoice"]+ assert [io_spec.name for io_spec in binding_node.node_io.outputs] == ["total_amount"]+ assert [io_spec.concept for io_spec in binding_node.node_io.outputs] == ["Number"]++ producer_node = _node(graph, pipe_code="make_invoice")+ consumer_node = _node(graph, pipe_code="write_receipt")+ data_edges = _data_edges(graph)+ assert (producer_node.node_id, binding_node.node_id) in data_edges+ assert (binding_node.node_id, consumer_node.node_id) in data_edges+ assert (producer_node.node_id, consumer_node.node_id) not in data_edges++ note_node = _node(graph, pipe_code="invoice.note")+ assert note_node.kind == NodeKind.BINDING+ assert note_node.status == NodeStatus.SUCCEEDED+ assert note_node.node_io.outputs == []++ async def test_a_binding_failing_on_any_error_closes_its_node_as_failed(self, mocker: MockerFixture) -> None:+ """An error that is not a Pipelex one, such as a content failing its own validation, still closes the node."""+ mocker.patch("pipelex.pipe_controllers.binding.binding_step.bind_content", side_effect=ValueError("the content failed its validation"))+ teardown_spy = mocker.spy(GraphTracer, "teardown")+ execution_config = get_config().interpreter.pipeline_execution.with_execution_overrides(generate_graph=True)+ runner = PipelexMTHDSProtocol(execution_config=execution_config, pipe_run_mode=PipeRunMode.LIVE)++ with scoped_event_log(InMemoryEventLog()), pytest.raises(PipelexError):+ await runner.execute(mthds_contents=[_BUNDLE], inputs={"amount": {"concept": "native.Number", "content": {"number": 318}}})++ binding_node = _node(_in_process_graph(teardown_spy), pipe_code="invoice.total")+ assert binding_node.status == NodeStatus.FAILED+ assert binding_node.error is not None+ assert binding_node.error.error_type == "ValueError"+ assert binding_node.error.message == "the content failed its validation"diff --git a/tests/integration/pipelex/pipes/controller/pipe_sequence/test_binding_judge_material.py b/tests/integration/pipelex/pipes/controller/pipe_sequence/test_binding_judge_material.pynew file mode 100644index 000000000..8f15b78d0--- /dev/null+++ b/tests/integration/pipelex/pipes/controller/pipe_sequence/test_binding_judge_material.py@@ -0,0 +1,29 @@+from typing import Any++import pytest+from pytest_mock import MockerFixture++from pipelex.kernel import judgment_ops+from pipelex.pipeline.runner import PipelexMTHDSProtocol+from pipelex.system.pipe_run_mode import PipeRunMode+from pipelex.test_extras.mthds_corpus.loader import load_entry+from pipelex.test_extras.mthds_corpus.resources import entries_root++[email protected](loop_scope="class")+class TestBindingJudgeMaterial:+ async def test_v28_a_judge_reading_a_bound_total_sees_it_alone(self, mocker: MockerFixture) -> None:+ """The judgment's material is keyed by the judge's plain input names, so it holds the bound total and nothing else of the invoice."""+ entry = load_entry(directory=entries_root() / "feature_binding_step_judge_amount")+ material_spy = mocker.spy(judgment_ops, "build_judgment_material")+ inputs: dict[str, Any] = {+ "invoice": {"concept": "invoice_approval.Invoice", "content": {"supplier": "Chantier Naval Le Bihan", "total": 7420.0}},+ }++ await PipelexMTHDSProtocol(library_dirs=[str(entry.directory)], pipe_run_mode=PipeRunMode.DRY).execute(pipe_code="check_total", inputs=inputs)++ assert material_spy.call_count == 1+ state, images, documents = material_spy.spy_return+ assert state == {"total_amount": 7420.0}+ assert images == {}+ assert documents == {}diff --git a/tests/integration/pipelex/pipes/controller/pipe_sequence/test_binding_nested_writes.py b/tests/integration/pipelex/pipes/controller/pipe_sequence/test_binding_nested_writes.pynew file mode 100644index 000000000..3667f204f--- /dev/null+++ b/tests/integration/pipelex/pipes/controller/pipe_sequence/test_binding_nested_writes.py@@ -0,0 +1,539 @@+from collections.abc import Callable+from typing import TYPE_CHECKING, Any, NamedTuple++import pytest+from pytest_mock import MockerFixture++from pipelex.core.memory.absence import AbsenceKind, AbsenceRecord+from pipelex.core.pipes.exceptions import PipeValidationError+from pipelex.interpreter_hub import get_library_manager+from pipelex.mthds_parsing.parser import MthdsParser+from pipelex.pipe_controllers.sequence.pipe_sequence import PipeSequence+from pipelex.pipeline.runner import PipelexMTHDSProtocol+from pipelex.system.pipe_run_mode import PipeRunMode+from pipelex.validation_error_types import PipeValidationErrorType+from tests.integration.pipelex.pipes.controller.pipe_sequence.unresolved_pipes import hide_pipe_from_sequence_analyses++if TYPE_CHECKING:+ from mthds.protocol.pipeline_inputs import PipelineInputs++_DOMAIN = "depot_stashes"++_BUNDLE_HEADER = f"""domain = "{_DOMAIN}"+description = "Stashing a parcel inside a nested controller, then reading its weight"+main_pipe = "read_stashed_weight"++[concept.Invoice]+description = "An invoice received at the depot"++[concept.Invoice.structure]+total = {{ type = "number", description = "The amount due, in euros", required = true }}++[concept.Parcel]+description = "A parcel received at the depot"++[concept.Parcel.structure]+weight = {{ type = "number", description = "The weight, in kilograms", required = true }}++[concept.Crate]+description = "A crate received at the depot"++[concept.Crate.structure]+weight = {{ type = "number", description = "The weight, in kilograms", required = true }}++[pipe.weigh_parcel]+type = "PipeCompose"+description = "Writes out a parcel of a given weight"+inputs = {{ amount = "Number" }}+output = "Parcel"++[pipe.weigh_parcel.construct]+weight = {{ from = "amount.number" }}++[pipe.weigh_crate]+type = "PipeCompose"+description = "Writes out a crate of a given weight"+inputs = {{ amount = "Number" }}+output = "Crate"++[pipe.weigh_crate.construct]+weight = {{ from = "amount.number" }}++[pipe.write_note]+type = "PipeCompose"+description = "Writes a note on what was weighed"+inputs = {{ amount = "Number" }}+output = "Text"+template = "Weighed $amount kilograms"++[pipe.describe_parcel]+type = "PipeCompose"+description = "Describes a parcel by its weight"+inputs = {{ record = "Parcel" }}+output = "Text"+template = "A parcel of {{{{ record.weight }}}} kilograms"++[pipe.stash_parcel]+type = "PipeSequence"+description = "Stores a parcel under the record's name, then writes a note"+inputs = {{ amount = "Number" }}+output = "Text"+steps = [+ {{ pipe = "weigh_parcel", result = "record" }},+ {{ pipe = "write_note", result = "note" }},+]++[pipe.stash_crate]+type = "PipeSequence"+description = "Stores a crate under the record's name, then writes a note"+inputs = {{ amount = "Number" }}+output = "Text"+steps = [+ {{ pipe = "weigh_crate", result = "record" }},+ {{ pipe = "write_note", result = "note" }},+]++[pipe.stash_handed_parcel]+type = "PipeSequence"+description = "Stores the parcel handed over, when one is, under the record's name, then writes a note"+inputs = {{ parcel = "Parcel?", amount = "Number" }}+output = "Text"+steps = [+ {{ from = "parcel", result = "record" }},+ {{ pipe = "write_note", result = "note" }},+]++[pipe.stash_by_mode]+type = "PipeCondition"+description = "Stores a parcel or a crate under the record's name, as the mode says"+inputs = {{ amount = "Number", mode = "Text" }}+output = "Text"+expression = "mode"+default_outcome = "fail"++[pipe.stash_by_mode.outcomes]+parcel = "stash_parcel"+crate = "stash_crate"++[pipe.stash_in_parallel]+type = "PipeParallel"+description = "Weighs a parcel and writes a note at once, adding each result to the memory"+inputs = {{ amount = "Number" }}+output = "Composite"+add_each_output = true+branches = [+ {{ pipe = "weigh_parcel", result = "record" }},+ {{ pipe = "write_note", result = "note" }},+]++[pipe.pick_parcel]+type = "PipeCondition"+description = "Weighs a parcel, or nothing, as the mode says"+inputs = {{ amount = "Number", mode = "Text" }}+output = "Parcel?"+expression = "mode"+default_outcome = "continue"++[pipe.pick_parcel.outcomes]+parcel = "weigh_parcel"++[pipe.stash_picked_in_parallel]+type = "PipeParallel"+description = "Picks a parcel and writes a note at once, adding each result to the memory"+inputs = {{ amount = "Number", mode = "Text" }}+output = "Composite"+add_each_output = true+branches = [+ {{ pipe = "pick_parcel", result = "record" }},+ {{ pipe = "write_note", result = "note" }},+]+"""++_READ_WEIGHT_STEP = '{ from = "record.weight", result = "weight" }'++# The calling sequence of the bundles below: a document it may be given, swapped for a parcel or left as it is by a condition,+# then a binding of its total and a plain reader of that total. The outcome storing the parcel stores the `Parcel` its pipe+# writes, which disagrees with the `Invoice` the caller may give, unless the tests hide that pipe from the sequence's analyses,+# as a pipe that does not resolve at validation (`hide_pipe_from_sequence_analyses`): the flow then cannot type the document.+_UNTYPED_ROOT_PIPES = """+[pipe.stash_doc_parcel]+type = "PipeSequence"+description = "Stores a parcel under the document's name"+inputs = { amount = "Number" }+output = "Parcel"+steps = [+ { pipe = "weigh_parcel", result = "doc" },+]++[pipe.swap_doc]+type = "PipeCondition"+description = "Swaps the document for a parcel, or leaves it, as the mode says"+inputs = { amount = "Number", mode = "Text" }+output = "Parcel?"+expression = "mode"+default_outcome = "continue"++[pipe.swap_doc.outcomes]+parcel = "stash_doc_parcel"++[pipe.write_total]+type = "PipeCompose"+description = "Writes out a total"+inputs = { total = "Number" }+output = "Text"+template = "Total: $total euros"+"""+++def _bundle(*, stash_step: str, inputs: str, output: str, reading_step: str = _READ_WEIGHT_STEP) -> str:+ """The bundle with its calling sequence: a step running a nested shape that stores `record`, then a step reading it."""+ return (+ f"{_BUNDLE_HEADER}\n"+ "[pipe.read_stashed_weight]\n"+ 'type = "PipeSequence"\n'+ 'description = "Stashes a record inside a nested controller, then reads it"\n'+ f"inputs = {{ {inputs} }}\n"+ f'output = "{output}"\n'+ "steps = [\n"+ f" {stash_step},\n"+ f" {reading_step},\n"+ "]\n"+ )+++def _untyped_root_bundle(*, output: str, ends_with_the_binding: bool) -> str:+ reader_step = "" if ends_with_the_binding else ' { pipe = "write_total", result = "line" },\n'+ return (+ f"{_BUNDLE_HEADER}{_UNTYPED_ROOT_PIPES}\n"+ "[pipe.read_stashed_weight]\n"+ 'type = "PipeSequence"\n'+ 'description = "Swaps the document or leaves it, binds its total, then writes it out"\n'+ 'inputs = { doc = "Invoice?", amount = "Number", mode = "Text" }\n'+ f'output = "{output}"\n'+ "steps = [\n"+ ' { pipe = "swap_doc", result = "swapped" },\n'+ ' { from = "doc.total", result = "total" },\n'+ f"{reader_step}"+ "]\n"+ )+++def _load_sequence(*, mthds_content: str, library_id: str) -> PipeSequence:+ blueprint = MthdsParser.make_pipelex_bundle_blueprint(mthds_content=mthds_content, mthds_source="stashes.mthds")+ pipes = get_library_manager().load_from_blueprints(library_id=library_id, blueprints=[blueprint])+ sequences = [pipe for pipe in pipes if isinstance(pipe, PipeSequence) and pipe.code == "read_stashed_weight"]+ assert len(sequences) == 1+ return sequences[0]+++async def _run(*, mthds_content: str, inputs: "PipelineInputs") -> AbsenceRecord | float:+ """Run the bundle live, and return the weight it read, or the absence its output holds."""+ response = await PipelexMTHDSProtocol(pipe_run_mode=PipeRunMode.LIVE).execute(mthds_contents=[mthds_content], inputs=inputs)+ main_resolved = response.pipe_output.working_memory.resolve_main_stuff()+ if isinstance(main_resolved, AbsenceRecord):+ return main_resolved+ return response.pipe_output.main_stuff_as_number.number+++class NestedShape(NamedTuple):+ """A nested shape storing `record` in the calling sequence's memory, and what each analysis must make of it."""++ stash_step: str+ inputs: str+ needed_inputs: set[str]+ # The concept the flow types `record` as when the binding of its weight reads it.+ record_concept_ref: str+ # Whether `record`, and so the binding of its weight, may hold an absence.+ is_maybe_absent: bool+ # Each run: its inputs beside the amount, and the weight it reads, `None` for an absent output.+ runs: list[tuple["PipelineInputs", float | None]]+ # Edits of the shared pipes this shape makes, each an old text and the new text replacing it.+ replacements: tuple[tuple[str, str], ...] = ()+++# A parcel handed over as an input, weighing three kilograms.+_A_PARCEL: dict[str, Any] = {"concept": f"{_DOMAIN}.Parcel", "content": {"weight": 3}}++_NESTED_SHAPES = [+ pytest.param(+ NestedShape(+ stash_step='{ pipe = "stash_parcel", result = "stash_note" }',+ inputs='amount = "Number"',+ needed_inputs={"amount"},+ record_concept_ref=f"{_DOMAIN}.Parcel",+ is_maybe_absent=False,+ runs=[({}, 7.5)],+ ),+ id="a-nested-sequence",+ ),+ pytest.param(+ NestedShape(+ stash_step='{ pipe = "stash_handed_parcel", result = "stash_note" }',+ inputs='parcel = "Parcel?", amount = "Number"',+ needed_inputs={"parcel", "amount"},+ record_concept_ref=f"{_DOMAIN}.Parcel",+ is_maybe_absent=True,+ runs=[({"parcel": _A_PARCEL}, 3), ({}, None)],+ ),+ id="a-nested-sequence-storing-a-maybe-absent-value",+ ),+ pytest.param(+ NestedShape(+ stash_step='{ pipe = "stash_by_mode", result = "stash_note" }',+ inputs='amount = "Number", mode = "Text"',+ needed_inputs={"amount", "mode"},+ record_concept_ref=f"{_DOMAIN}.Parcel",+ is_maybe_absent=False,+ runs=[({"mode": "crate"}, 7.5)],+ replacements=(('crate = "stash_crate"', 'crate = "stash_parcel"'),),+ ),+ id="a-condition-whose-outcomes-agree",+ ),+ pytest.param(+ NestedShape(+ stash_step='{ pipe = "stash_by_mode", result = "stash_note" }',+ inputs='parcel = "Parcel?", amount = "Number", mode = "Text"',+ needed_inputs={"parcel", "amount", "mode"},+ record_concept_ref=f"{_DOMAIN}.Parcel",+ is_maybe_absent=True,+ runs=[({"mode": "parcel"}, 7.5), ({"mode": "crate"}, None)],+ replacements=(+ ('crate = "stash_crate"', 'crate = "stash_handed_parcel"'),+ (+ 'inputs = { amount = "Number", mode = "Text" }\noutput = "Text"',+ 'inputs = { parcel = "Parcel?", amount = "Number", mode = "Text" }\noutput = "Text"',+ ),+ ),+ ),+ id="a-condition-with-an-outcome-storing-a-maybe-absent-value",+ ),+ pytest.param(+ NestedShape(+ stash_step='{ pipe = "stash_by_mode", result = "stash_note" }',+ inputs='amount = "Number", mode = "Text", record = "Parcel"',+ needed_inputs={"amount", "mode", "record"},+ record_concept_ref=f"{_DOMAIN}.Parcel",+ is_maybe_absent=False,+ runs=[({"mode": "parcel", "record": _A_PARCEL}, 7.5), ({"mode": "crate", "record": _A_PARCEL}, 3)],+ replacements=(+ ('crate = "stash_crate"', 'crate = "continue"'),+ ('output = "Text"\nexpression = "mode"\ndefault_outcome = "fail"', 'output = "Text?"\nexpression = "mode"\ndefault_outcome = "fail"'),+ ),+ ),+ id="a-condition-with-a-continue-outcome-leaving-the-callers-value",+ ),+ pytest.param(+ NestedShape(+ stash_step='{ pipe = "stash_in_parallel", result = "stash" }',+ inputs='amount = "Number"',+ needed_inputs={"amount"},+ record_concept_ref=f"{_DOMAIN}.Parcel",+ is_maybe_absent=False,+ runs=[({}, 7.5)],+ ),+ id="a-parallel-adding-each-output",+ ),+ pytest.param(+ NestedShape(+ stash_step='{ pipe = "stash_picked_in_parallel", result = "stash" }',+ inputs='amount = "Number", mode = "Text"',+ needed_inputs={"amount", "mode"},+ record_concept_ref=f"{_DOMAIN}.Parcel",+ is_maybe_absent=True,+ runs=[({"mode": "parcel"}, 7.5), ({"mode": "none"}, None)],+ ),+ id="a-parallel-adding-a-maybe-absent-branch-result",+ ),+ pytest.param(+ NestedShape(+ stash_step='{ pipe = "stash_parcel", result = "stash_notes", batch_over = "amounts", batch_as = "amount" }',+ inputs='amounts = "Number[]", record = "Parcel"',+ needed_inputs={"amounts", "record"},+ record_concept_ref=f"{_DOMAIN}.Parcel",+ is_maybe_absent=False,+ runs=[({"record": _A_PARCEL}, 3)],+ ),+ id="a-batched-step-whose-branches-store-nothing-in-the-caller",+ ),+]+++class TestBindingNestedWrites:+ @pytest.mark.asyncio(loop_scope="class")+ @pytest.mark.parametrize("shape", _NESTED_SHAPES)+ async def test_the_three_analyses_agree_on_what_a_nested_shape_stores(self, load_empty_library: Callable[[], str], shape: NestedShape) -> None:+ """Needed inputs, derived type and absence agree on the name a nested shape stores, and the run does what they say."""+ mthds_content = _bundle(stash_step=shape.stash_step, inputs=shape.inputs, output="Number?" if shape.is_maybe_absent else "Number")+ for old_text, new_text in shape.replacements:+ assert old_text in mthds_content+ mthds_content = mthds_content.replace(old_text, new_text)+ sequence = _load_sequence(mthds_content=mthds_content, library_id=load_empty_library())++ assert set(sequence.needed_inputs().root) == shape.needed_inputs+ typed_flow = sequence.build_typed_flow()+ record_spec = typed_flow.final_slots["record"].stuff_spec+ assert record_spec is not None+ assert record_spec.concept.concept_ref == shape.record_concept_ref+ assert typed_flow.binding_specs[1].concept.concept_ref == "native.Number"+ assert (sequence.analyze_taint().output_taint is not None) is shape.is_maybe_absent++ for extra_inputs, expected_weight in shape.runs:+ amount_inputs: PipelineInputs = (+ {"amounts": {"concept": "native.Number", "content": [{"number": 7.5}]}}+ if "amounts" in shape.inputs+ else {"amount": {"concept": "native.Number", "content": {"number": 7.5}}}+ )+ outcome = await _run(mthds_content=mthds_content, inputs={**amount_inputs, **extra_inputs})+ if expected_weight is None:+ assert isinstance(outcome, AbsenceRecord)+ else:+ assert outcome == expected_weight++ @pytest.mark.parametrize(+ ("stash_step", "inputs"),+ [+ pytest.param('{ pipe = "stash_handed_parcel", result = "stash_note" }', 'parcel = "Parcel?", amount = "Number"', id="a-nested-sequence"),+ pytest.param('{ pipe = "stash_picked_in_parallel", result = "stash" }', 'amount = "Number", mode = "Text"', id="a-parallel"),+ ],+ )+ def test_a_nested_write_keeps_its_absence(self, load_empty_library: Callable[[], str], stash_step: str, inputs: str) -> None:+ """Regression: a nested value that may be absent makes the binding reading it maybe-absent, so a sequence ending with the+ binding declares its output `?` or is refused, rather than resolving absent behind a plain output.+ """+ mthds_content = _bundle(stash_step=stash_step, inputs=inputs, output="Number")++ with pytest.raises(PipeValidationError) as exc_info:+ _load_sequence(mthds_content=mthds_content, library_id=load_empty_library())++ assert exc_info.value.error_type == PipeValidationErrorType.OPTIONAL_NOT_HANDLED+ assert exc_info.value.pipe_code == "read_stashed_weight"+ assert "slot 'weight'" in str(exc_info.value)++ @pytest.mark.asyncio(loop_scope="class")+ @pytest.mark.parametrize(+ ("reading_step", "output", "expected_output"),+ [+ pytest.param(_READ_WEIGHT_STEP, "Number", 7.5, id="read-by-a-binding"),+ pytest.param('{ pipe = "describe_parcel", result = "description" }', "Text", "A parcel of 7.5 kilograms", id="read-by-a-pipe-step"),+ ],+ )+ async def test_a_name_a_nested_sequence_always_stores_is_no_input_of_the_caller(+ self, load_empty_library: Callable[[], str], reading_step: str, output: str, expected_output: float | str+ ) -> None:+ """Regression: a step reading a name a nested sequence always stores needs no input of that name."""+ mthds_content = _bundle(+ stash_step='{ pipe = "stash_parcel", result = "stash_note" }', inputs='amount = "Number"', output=output, reading_step=reading_step+ )++ sequence = _load_sequence(mthds_content=mthds_content, library_id=load_empty_library())++ assert set(sequence.needed_inputs().root) == {"amount"}+ response = await PipelexMTHDSProtocol(pipe_run_mode=PipeRunMode.LIVE).execute(+ mthds_contents=[mthds_content], inputs={"amount": {"concept": "native.Number", "content": {"number": 7.5}}}+ )+ if isinstance(expected_output, str):+ assert response.pipe_output.main_stuff_as_str == expected_output+ else:+ assert response.pipe_output.main_stuff_as_number.number == expected_output++ def test_a_name_only_some_outcomes_store_is_still_an_input_of_the_caller(self, load_empty_library: Callable[[], str]) -> None:+ """Regression: a condition with a `continue` outcome may leave the name as the caller had it, so the caller provides it."""+ mthds_content = _bundle(+ stash_step='{ pipe = "stash_by_mode", result = "stash_note" }', inputs='amount = "Number", mode = "Text"', output="Number"+ )+ mthds_content = mthds_content.replace('crate = "stash_crate"', 'crate = "continue"').replace(+ 'output = "Text"\nexpression = "mode"\ndefault_outcome = "fail"', 'output = "Text?"\nexpression = "mode"\ndefault_outcome = "fail"'+ )++ with pytest.raises(PipeValidationError) as exc_info:+ _load_sequence(mthds_content=mthds_content, library_id=load_empty_library())++ assert exc_info.value.error_type == PipeValidationErrorType.MISSING_INPUT_VARIABLE+ assert exc_info.value.variable_names == ["record"]+ assert "always stored by an earlier step" in str(exc_info.value)++ @pytest.mark.asyncio(loop_scope="class")+ async def test_a_lifted_nested_sequence_resolves_what_it_always_stores(self, load_empty_library: Callable[[], str]) -> None:+ """A nested sequence lifted for an absent plain input stores what it always stores as an absence, so a later step reading+ it is skipped in turn, as the absence analysis says, rather than meeting a name that holds neither a value nor an absence.+ """+ stash_step = '{ pipe = "stash_parcel", result = "stash_note" }'+ with pytest.raises(PipeValidationError) as exc_info:+ _load_sequence(+ mthds_content=_bundle(stash_step=stash_step, inputs='amount = "Number?"', output="Number"), library_id=load_empty_library()+ )+ assert exc_info.value.error_type == PipeValidationErrorType.OPTIONAL_NOT_HANDLED++ mthds_content = _bundle(stash_step=stash_step, inputs='amount = "Number?"', output="Number?")+ sequence = _load_sequence(mthds_content=mthds_content, library_id=load_empty_library())+ assert set(sequence.needed_inputs().root) == {"amount"}+ assert [liftable.pipe_ref for liftable in sequence.analyze_taint().liftable_steps] == [f"{_DOMAIN}.stash_parcel"]++ outcome = await _run(mthds_content=mthds_content, inputs={})+ assert isinstance(outcome, AbsenceRecord)+ assert outcome.kind == AbsenceKind.SKIPPED+ assert outcome.upstream is not None+ assert outcome.upstream.variable_name == "record"+ assert await _run(mthds_content=mthds_content, inputs={"amount": {"concept": "native.Number", "content": {"number": 7.5}}}) == 7.5++ def test_a_name_the_outcomes_store_under_different_concepts_is_refused(self, load_empty_library: Callable[[], str]) -> None:+ """Regression: the outcomes of a condition store `record` as a parcel or as a crate, so its concept is not known before+ the run, and the binding of its weight is refused there rather than derived when it runs.+ """+ mthds_content = _bundle(+ stash_step='{ pipe = "stash_by_mode", result = "stash_note" }', inputs='amount = "Number", mode = "Text"', output="Number"+ )++ with pytest.raises(PipeValidationError) as exc_info:+ _load_sequence(mthds_content=mthds_content, library_id=load_empty_library())++ assert exc_info.value.error_type == PipeValidationErrorType.BINDING_PATH_UNRESOLVED+ assert exc_info.value.pipe_code == "read_stashed_weight"+ assert "outcome 'stash_crate' of pipe 'stash_by_mode' as 'Crate'; outcome 'stash_parcel' of pipe 'stash_by_mode' as 'Parcel'" in str(+ exc_info.value+ )++ def test_a_document_swapped_for_another_concept_is_refused(self, load_empty_library: Callable[[], str]) -> None:+ """The condition may leave the `Invoice` the caller gives or store a `Parcel`, so the binding of the document's total+ is refused before the run, whatever its output says about absence.+ """+ with pytest.raises(PipeValidationError) as exc_info:+ _load_sequence(mthds_content=_untyped_root_bundle(output="Text?", ends_with_the_binding=False), library_id=load_empty_library())++ assert exc_info.value.error_type == PipeValidationErrorType.BINDING_PATH_UNRESOLVED+ assert "the concept of 'doc' is not known before the run" in str(exc_info.value)+ assert "which the step leaves when it stores nothing, as 'Invoice?'" in str(exc_info.value)++ @pytest.mark.asyncio(loop_scope="class")+ async def test_an_untyped_root_keeps_its_absence(self, load_empty_library: Callable[[], str], mocker: MockerFixture) -> None:+ """Regression: a root the flow cannot type, a pipe that does not resolve at validation having stored it, still carries+ its own absence to the binding's result, so a plain reader of the result is listed as liftable and the sequence's output+ is declared `?`, as the run skips both when the root is absent.+ """+ hide_pipe_from_sequence_analyses(mocker=mocker, pipe_code="weigh_parcel")+ with pytest.raises(PipeValidationError) as exc_info:+ _load_sequence(mthds_content=_untyped_root_bundle(output="Text", ends_with_the_binding=False), library_id=load_empty_library())+ assert exc_info.value.error_type == PipeValidationErrorType.OPTIONAL_NOT_HANDLED+ assert "optional input 'doc'" in str(exc_info.value)++ mthds_content = _untyped_root_bundle(output="Text?", ends_with_the_binding=False)+ sequence = _load_sequence(mthds_content=mthds_content, library_id=load_empty_library())+ assert 1 not in sequence.build_typed_flow().binding_derivations+ taint_analysis = sequence.analyze_taint()+ assert [liftable.pipe_ref for liftable in taint_analysis.liftable_steps] == [f"{_DOMAIN}.write_total"]+ assert taint_analysis.output_taint is not None++ response = await PipelexMTHDSProtocol(pipe_run_mode=PipeRunMode.LIVE).execute(+ mthds_contents=[mthds_content],+ inputs={"amount": {"concept": "native.Number", "content": {"number": 7.5}}, "mode": "none"},+ )+ assert isinstance(response.pipe_output.working_memory.resolve_main_stuff(), AbsenceRecord)++ def test_an_untyped_root_ending_the_sequence_keeps_its_absence(self, load_empty_library: Callable[[], str], mocker: MockerFixture) -> None:+ """Regression: a binding ending the sequence, whose root the flow cannot type, is still refused behind a plain output."""+ hide_pipe_from_sequence_analyses(mocker=mocker, pipe_code="weigh_parcel")+ with pytest.raises(PipeValidationError) as exc_info:+ _load_sequence(mthds_content=_untyped_root_bundle(output="Number", ends_with_the_binding=True), library_id=load_empty_library())++ assert exc_info.value.error_type == PipeValidationErrorType.OPTIONAL_NOT_HANDLED+ assert "optional input 'doc'" in str(exc_info.value)diff --git a/tests/integration/pipelex/pipes/controller/pipe_sequence/test_binding_refusal_remedy.py b/tests/integration/pipelex/pipes/controller/pipe_sequence/test_binding_refusal_remedy.pynew file mode 100644index 000000000..e3d2c2caa--- /dev/null+++ b/tests/integration/pipelex/pipes/controller/pipe_sequence/test_binding_refusal_remedy.py@@ -0,0 +1,141 @@+import re++import pytest++from pipelex.mthds_parsing.exceptions import MthdsParserError+from pipelex.mthds_parsing.parser import MthdsParser+from pipelex.pipe_controllers.binding.binding_step_blueprint import BindingStepBlueprint+from pipelex.pipe_controllers.sequence.pipe_sequence_blueprint import PipeSequenceBlueprint+from pipelex.pipeline.exceptions import ValidateBundleError+from pipelex.pipeline.runner import PipelexMTHDSProtocol+from pipelex.pipeline.validate_bundle import validate_bundle+from pipelex.system.pipe_run_mode import PipeRunMode+from pipelex.validation_error_types import PipeValidationErrorType++_IMAGE_URL = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGM4IScHAAK2AQU0pnWqAAAAAElFTkSuQmCC"++_REFUSED_BUNDLE = """domain = "catalog_views"+description = "Describing the photograph of a catalog page"+main_pipe = "describe_view"++[pipe.describe_view]+type = "PipeCompose"+description = "Describes the photograph of a catalog page"+inputs = { "page.page_view" = "Image" }+output = "Text"+template = "View: {{ page.page_view.caption }}"+"""++_REMEDIED_BUNDLE_TEMPLATE = """domain = "catalog_views"+description = "Describing the photograph of a catalog page"+main_pipe = "describe_page"++[pipe.describe_page]+type = "PipeSequence"+description = "Binds the view of a page, then describes it"+inputs = {{ page = "Page" }}+output = "Text?"+steps = [+ {binding_step},+ {{ pipe = "describe_view", result = "description" }},+]++[pipe.describe_view]+type = "PipeCompose"+description = "Describes the photograph of a catalog page"+inputs = {{ page_view = "Image" }}+output = "Text"+template = "View: {{{{ page_view.caption }}}}"+"""++# A root that is not a plain name: no input can bear it, so only an earlier pipe step's `result`, which the standard leaves+# unrestricted, can store a value under it.+_UPPERCASE_ROOT_BUNDLE = """domain = "billing"+description = "Reading the total of an invoice stored under a name that is not a plain name"+main_pipe = "read_total"++[concept.Invoice]+description = "An invoice sent by a supplier"++[concept.Invoice.structure]+total = { type = "number", description = "The amount due", required = true }++[pipe.make_invoice]+type = "PipeCompose"+description = "Writes out an invoice for an amount"+inputs = { amount = "Number" }+output = "Invoice"++[pipe.make_invoice.construct]+total = { from = "amount.number" }++[pipe.read_total]+type = "PipeSequence"+description = "Writes out an invoice, then binds its total"+inputs = { amount = "Number" }+output = "Number"+steps = [+ { pipe = "make_invoice", result = "Invoice" },+ { from = "Invoice.total", result = "total" },+]+"""+++class TestBindingRefusalRemedy:+ @pytest.mark.asyncio(loop_scope="class")+ async def test_the_binding_step_a_refusal_prints_parses_validates_and_runs(self) -> None:+ """The binding step the dotted-input refusal prints, pasted as it stands into a calling sequence, is a working step."""+ with pytest.raises(MthdsParserError) as exc_info:+ MthdsParser.make_pipelex_bundle_blueprint(mthds_content=_REFUSED_BUNDLE, mthds_source="refused.mthds")+ errors = exc_info.value.validation_errors+ assert [error.error_type for error in errors] == [PipeValidationErrorType.INVALID_INPUT_NAME]+ printed_steps = re.findall(r"`(\{ from = [^`]*\})`", errors[0].message)+ assert printed_steps == ['{ from = "page.page_view", result = "page_view" }']++ remedied_bundle = _REMEDIED_BUNDLE_TEMPLATE.format(binding_step=printed_steps[0])+ blueprint = MthdsParser.make_pipelex_bundle_blueprint(mthds_content=remedied_bundle, mthds_source="remedied.mthds")+ assert blueprint.pipe is not None+ sequence_blueprint = blueprint.pipe["describe_page"]+ assert isinstance(sequence_blueprint, PipeSequenceBlueprint)+ assert sequence_blueprint.steps[0] == BindingStepBlueprint.model_validate({"from": "page.page_view", "result": "page_view"})++ validation = await validate_bundle(mthds_contents=[remedied_bundle])+ assert validation.blueprints++ result = await PipelexMTHDSProtocol(pipe_run_mode=PipeRunMode.LIVE).execute(+ mthds_contents=[remedied_bundle],+ inputs={+ "page": {+ "concept": "native.Page",+ "content": {+ "text_and_images": {"text": {"text": "Garden chairs"}},+ "page_view": {"url": _IMAGE_URL, "caption": "Garden chairs on a lawn"},+ },+ }+ },+ )+ assert result.pipe_output.main_stuff.as_text.text == "View: Garden chairs on a lawn"++ @pytest.mark.asyncio(loop_scope="class")+ async def test_a_root_that_is_not_a_plain_name_reads_an_earlier_steps_result(self) -> None:+ """The path grammar lets the root be any segment, so it can read a pipe step's `result` that is not a plain name."""+ result = await PipelexMTHDSProtocol(pipe_run_mode=PipeRunMode.LIVE).execute(+ mthds_contents=[_UPPERCASE_ROOT_BUNDLE], inputs={"amount": {"concept": "native.Number", "content": {"number": 120}}}+ )++ assert result.pipe_output.main_stuff_as_number.number == 120++ @pytest.mark.asyncio(loop_scope="class")+ async def test_a_missing_root_that_is_not_a_plain_name_is_never_asked_for_as_an_input(self) -> None:+ """Regression: the refusal of a root nothing stores never tells the author to declare an input no input name can match."""+ mthds_content = _UPPERCASE_ROOT_BUNDLE.replace(' { pipe = "make_invoice", result = "Invoice" },\n', "")++ with pytest.raises(ValidateBundleError) as exc_info:+ await validate_bundle(mthds_contents=[mthds_content])++ validation_errors = exc_info.value.to_error_report().validation_errors or []+ assert [error.error_type for error in validation_errors] == [PipeValidationErrorType.MISSING_INPUT_VARIABLE]+ message = validation_errors[0].message or ""+ assert "Declare 'Invoice'" not in message+ assert "'Invoice' cannot be an input of the sequence" in message+ assert "store a value under 'Invoice' in an earlier step" in messagediff --git a/tests/integration/pipelex/pipes/controller/pipe_sequence/test_binding_typed_flow.py b/tests/integration/pipelex/pipes/controller/pipe_sequence/test_binding_typed_flow.pynew file mode 100644index 000000000..fb7a77b00--- /dev/null+++ b/tests/integration/pipelex/pipes/controller/pipe_sequence/test_binding_typed_flow.py@@ -0,0 +1,812 @@+from collections.abc import Callable+from typing import TYPE_CHECKING++import pytest+from pytest_mock import MockerFixture++from pipelex.base_exceptions import PipelexError+from pipelex.core.pipes.exceptions import PipeValidationError+from pipelex.core.stuffs.json_content import JSONContent+from pipelex.core.stuffs.number_content import NumberContent+from pipelex.core.stuffs.structured_content import StructuredContent+from pipelex.core.stuffs.stuff_content import StuffContent+from pipelex.core.stuffs.text_content import TextContent+from pipelex.interpreter_hub import get_library_manager+from pipelex.mthds_parsing.parser import MthdsParser+from pipelex.pipe_controllers.sequence import pipe_sequence as sequence_module+from pipelex.pipe_controllers.sequence.pipe_sequence import PipeSequence+from pipelex.pipeline.exceptions import ValidateBundleError+from pipelex.pipeline.runner import PipelexMTHDSProtocol+from pipelex.pipeline.validate_bundle import validate_bundle+from pipelex.system.pipe_run_mode import PipeRunMode+from pipelex.system.registries.class_registry_access import get_class_registry+from pipelex.test_extras.mthds_corpus.resources import entries_root+from pipelex.validation_error_types import PipeValidationErrorType+from tests.integration.pipelex.pipes.controller.pipe_sequence.unresolved_pipes import hide_pipe_from_sequence_analyses++if TYPE_CHECKING:+ from mthds.protocol.pipeline_inputs import PipelineInputs++_REBOUND_ROOT_BUNDLE = """domain = "depot_records"+description = "A record name holding an invoice, then a parcel"++[concept.Invoice]+description = "An invoice received at the depot"++[concept.Invoice.structure]+total = { type = "number", description = "The amount due, in euros", required = true }++[concept.Parcel]+description = "A parcel received at the depot"++[concept.Parcel.structure]+weight = { type = "number", description = "The weight, in kilograms", required = true }+labels = { type = "list", item_type = "text", description = "The labels stuck on the parcel", required = true }++[pipe.weigh_parcel]+type = "PipeCompose"+description = "Writes out a parcel of a given weight"+inputs = { amount = "Number" }+output = "Parcel"++[pipe.weigh_parcel.construct]+weight = { from = "amount.number" }+labels = ["fragile"]++[pipe.write_line]+type = "PipeCompose"+description = "Writes a line for an amount and a weight"+inputs = { amount = "Number", weight = "Number" }+output = "Text"+template = "$amount euros for $weight kilograms"++[pipe.read_records]+type = "PipeSequence"+description = "Binds the total of the invoice, replaces the record by a parcel, then binds the parcel's weight"+inputs = { record = "Invoice" }+output = "Text"+steps = [+ { from = "record.total", result = "amount" },+ { pipe = "weigh_parcel", result = "record" },+ { from = "record.weight", result = "weight" },+ { from = "record.labels", result = "labels" },+ { pipe = "write_line", result = "line" },+]+"""+++class DepotSlipRecord(StructuredContent):+ """A depot slip whose carrier note is required yet may be `None`, as a Python class can declare it."""++ reference: str+ carrier_note: str | None+++_NULLABLE_FIELD_BUNDLE = """domain = "depot_slips"+description = "Reading the carrier's note off a depot slip"++[concept.DepotSlip]+description = "A slip handed over at the depot"+structure = "DepotSlipRecord"++[pipe.read_note]+type = "PipeSequence"+description = "Binds the carrier's note of a slip"+inputs = { slip = "DepotSlip" }+output = "Text"+steps = [+ { from = "slip.carrier_note", result = "note" },+]+"""++_DESCRIPTION_ONLY_BUNDLE = """domain = "depot_notices"+description = "Notices and bulletins posted at the depot"++[concept]+Notice = "A notice posted at the depot"++[concept.Bulletin]+description = "A bulletin posted at the depot"++[pipe.read_notice]+type = "PipeSequence"+description = "Binds the text inside a notice"+inputs = { notice = "Notice" }+output = "Text"+steps = [+ { from = "notice.text", result = "notice_text" },+]+"""+++_BATCHED_STEP_BUNDLE = """domain = "depot_batches"+description = "Weighing a batch of parcels, then reading their weights"+main_pipe = "read_weights"++[concept.Parcel]+description = "A parcel received at the depot"++[concept.Parcel.structure]+weight = { type = "number", description = "The weight, in kilograms", required = true }++[pipe.weigh_parcel]+type = "PipeCompose"+description = "Writes out a parcel of a given weight"+inputs = { amount = "Number" }+output = "Parcel"++[pipe.weigh_parcel.construct]+weight = { from = "amount.number" }++[pipe.read_weights]+type = "PipeSequence"+description = "Weighs a parcel for each amount, then binds their weights"+inputs = { amounts = "Number[]" }+output = "Number[]"+steps = [+ { pipe = "weigh_parcel", result = "parcels", batch_over = "amounts", batch_as = "amount" },+ { from = "parcels.weight", result = "weights" },+]+"""+++_NESTED_WRITES_BUNDLE = """domain = "depot_swaps"+description = "Swapping a record for a parcel or a crate inside a nested controller, then reading its weight"+main_pipe = "read_swapped_weight"++[concept.Invoice]+description = "An invoice received at the depot"++[concept.Invoice.structure]+total = { type = "number", description = "The amount due, in euros", required = true }++[concept.Parcel]+description = "A parcel received at the depot"++[concept.Parcel.structure]+weight = { type = "number", description = "The weight, in kilograms", required = true }++[concept.Crate]+description = "A crate received at the depot"++[concept.Crate.structure]+weight = { type = "number", description = "The weight, in kilograms", required = true }++[pipe.weigh_parcel]+type = "PipeCompose"+description = "Writes out a parcel of a given weight"+inputs = { amount = "Number" }+output = "Parcel"++[pipe.weigh_parcel.construct]+weight = { from = "amount.number" }++[pipe.weigh_crate]+type = "PipeCompose"+description = "Writes out a crate of a given weight"+inputs = { amount = "Number" }+output = "Crate"++[pipe.weigh_crate.construct]+weight = { from = "amount.number" }++[pipe.write_note]+type = "PipeCompose"+description = "Writes a note on what was weighed"+inputs = { amount = "Number" }+output = "Text"+template = "Weighed $amount kilograms"++[pipe.swap_for_parcel]+type = "PipeSequence"+description = "Stores a parcel under the record's name, then writes a note"+inputs = { amount = "Number" }+output = "Text"+steps = [+ { pipe = "weigh_parcel", result = "record" },+ { pipe = "write_note", result = "note" },+]++[pipe.swap_for_crate]+type = "PipeSequence"+description = "Stores a crate under the record's name, then writes a note"+inputs = { amount = "Number" }+output = "Text"+steps = [+ { pipe = "weigh_crate", result = "record" },+ { pipe = "write_note", result = "note" },+]++[pipe.swap_record]+type = "PipeCondition"+description = "Swaps the record for a parcel or a crate, as the mode says"+inputs = { amount = "Number", mode = "Text" }+output = "Text"+expression = "mode"+default_outcome = "fail"++[pipe.swap_record.outcomes]+parcel = "swap_for_parcel"+crate = "swap_for_crate"++[pipe.read_swapped_weight]+type = "PipeSequence"+description = "Swaps the record, then binds the weight of what replaced it"+inputs = { amount = "Number", mode = "Text" }+output = "Number"+steps = [+ { pipe = "swap_record", result = "note" },+ { from = "record.weight", result = "weight" },+]+"""+++_ANYTHING_FIELD_BUNDLE = """domain = "depot_crates"+description = "Reading what a crate holds, whatever it is"+main_pipe = "read_contents"++[concept.Crate]+description = "A crate that may hold anything"++[concept.Crate.structure]+contents = { type = "concept", concept_ref = "native.Anything", description = "What the crate holds", required = true }++[pipe.read_contents]+type = "PipeSequence"+description = "Binds what a crate holds"+inputs = { crate = "Crate" }+output = "Anything"+steps = [+ { from = "crate.contents", result = "contents" },+]+"""+++_CALLED_BINDING_BUNDLE = """domain = "depot_totals"+description = "Reading totals through a sequence that ends with a binding"++[concept.Invoice]+description = "An invoice received at the depot"++[concept.Invoice.structure]+total = { type = "number", description = "The amount due, in euros", required = true }+amounts = { type = "list", item_type = "number", description = "The amounts of its lines, in euros", required = true }++[pipe.read_total]+type = "PipeSequence"+description = "Binds the total of an invoice"+inputs = { invoice = "Invoice" }+output = "Number"+steps = [+ { from = "invoice.total", result = "total" },+]++[pipe.read_amounts]+type = "PipeSequence"+description = "Binds the amounts of an invoice's lines"+inputs = { invoice = "Invoice" }+output = "Number[]"+steps = [+ { from = "invoice.amounts", result = "amounts" },+]++"""+++_WRITTEN_INVOICE_BUNDLE = """domain = "depot_reads"+description = "Reading the total of an invoice a step writes"+main_pipe = "read_written_total"++[concept.Invoice]+description = "An invoice received at the depot"++[concept.Invoice.structure]+total = { type = "number", description = "The amount due, in euros", required = true }++[pipe.write_invoice]+type = "PipeLLM"+description = "Writes an invoice from a note"+inputs = { note = "Text" }+output = "Invoice"+model = "$testing-text"+prompt = "Write the invoice described in $note"++[pipe.read_written_total]+type = "PipeSequence"+description = "Writes an invoice, then binds its total"+inputs = { note = "Text" }+output = "Number"+steps = [+ { pipe = "write_invoice", result = "invoice" },+ { from = "invoice.total", result = "total" },+]+"""+++# A parcel the depot weighs, whose producer the tests hide from the sequence's analyses (`hide_pipe_from_sequence_analyses`), as+# a pipe that does not resolve at validation: the flow cannot type `parcel`, so a binding reading it is derived when it runs.+_UNRESOLVED_PRODUCER_HEADER = """domain = "depot_carriers"+description = "Reading a parcel a pipe that does not resolve at validation weighs"+main_pipe = "read_parcel"++[concept.Parcel]+description = "A parcel received at the depot"++[concept.Parcel.structure]+weight = { type = "number", description = "The weight, in kilograms", required = true }+carrier = { type = "text", description = "The carrier who brought it", required = true }++[pipe.weigh_parcel]+type = "PipeCompose"+description = "Writes out a parcel of a given weight"+inputs = { amount = "Number" }+output = "Parcel"++[pipe.weigh_parcel.construct]+weight = { from = "amount.number" }+carrier = "Depot Express"++[pipe.write_note]+type = "PipeCompose"+description = "Writes a note on what was weighed"+inputs = { weight = "Number" }+output = "Text"+template = "Weighed $weight kilograms"+"""+++def _unresolved_producer_bundle(*, output: str, steps: list[str]) -> str:+ """The bundle with a sequence weighing a parcel, then running `steps` over it."""+ return (+ f"{_UNRESOLVED_PRODUCER_HEADER}\n"+ "[pipe.read_parcel]\n"+ 'type = "PipeSequence"\n'+ 'description = "Weighs a parcel, then reads it"\n'+ 'inputs = { amount = "Number" }\n'+ f'output = "{output}"\n'+ "steps = [\n"+ ' { pipe = "weigh_parcel", result = "parcel" },\n' + "".join(f" {step},\n" for step in steps) + "]\n"+ )+++def _called_binding_bundle(*, called_pipe: str, step_count: str, output: str) -> str:+ """The bundle with a sequence calling one of those ending with a binding, asking it for a count of outputs."""+ return (+ f"{_CALLED_BINDING_BUNDLE}\n"+ "[pipe.read_totals]\n"+ 'type = "PipeSequence"\n'+ 'description = "Reads an invoice through a sequence ending with a binding"\n'+ 'inputs = { invoice = "Invoice" }\n'+ f'output = "{output}"\n'+ "steps = [\n"+ f' {{ pipe = "{called_pipe}", result = "totals", {step_count} }},\n'+ "]\n"+ )+++def _load_sequence(*, mthds_content: str, library_id: str, pipe_code: str) -> PipeSequence:+ blueprint = MthdsParser.make_pipelex_bundle_blueprint(mthds_content=mthds_content, mthds_source="flow.mthds")+ pipes = get_library_manager().load_from_blueprints(library_id=library_id, blueprints=[blueprint])+ sequences = [pipe for pipe in pipes if isinstance(pipe, PipeSequence) and pipe.code == pipe_code]+ assert len(sequences) == 1+ return sequences[0]+++class TestBindingTypedFlow:+ def test_the_latest_producer_types_a_root(self, load_empty_library: Callable[[], str]) -> None:+ """A name a later step stores replaces what was there, so a binding after it walks the new value's concept."""+ sequence = _load_sequence(mthds_content=_REBOUND_ROOT_BUNDLE, library_id=load_empty_library(), pipe_code="read_records")++ flow = sequence.build_typed_flow()++ assert sorted(flow.binding_derivations) == [0, 2, 3]+ assert flow.binding_specs[0].concept.concept_ref == "native.Number"+ assert flow.binding_specs[0].multiplicity is None+ assert flow.binding_specs[2].concept.concept_ref == "native.Number"+ assert flow.binding_specs[3].concept.concept_ref == "native.Text"+ assert flow.binding_specs[3].multiplicity is True+ assert sorted(flow.binding_slots_by_pipe_step[1]) == ["amount"]+ assert sorted(flow.binding_slots_by_pipe_step[4]) == ["amount", "labels", "weight"]+ assert flow.final_slots["record"].stuff_spec is not None+ assert flow.final_slots["record"].stuff_spec.concept.concept_ref == "depot_records.Parcel"++ def test_a_sequence_without_a_binding_step_never_builds_its_flow(self, load_empty_library: Callable[[], str], mocker: MockerFixture) -> None:+ """Only binding steps read the typed flow, so validating a sequence of pipe steps alone never builds it."""+ flow_builder_spy = mocker.spy(sequence_module, "build_sequence_typed_flow")+ mthds_content = _REBOUND_ROOT_BUNDLE.replace(+ """ { from = "record.total", result = "amount" },+ { pipe = "weigh_parcel", result = "record" },+ { from = "record.weight", result = "weight" },+ { from = "record.labels", result = "labels" },+ { pipe = "write_line", result = "line" },""",+ """ { pipe = "weigh_parcel", result = "parcel" },+ { pipe = "write_line", result = "line" },""",+ ).replace('inputs = { record = "Invoice" }', 'inputs = { amount = "Number", weight = "Number" }')++ _load_sequence(mthds_content=mthds_content, library_id=load_empty_library(), pipe_code="read_records")++ assert flow_builder_spy.call_count == 0++ def test_a_binding_walking_the_replaced_concept_is_refused(self, load_empty_library: Callable[[], str]) -> None:+ """The library refuses the sequence when it loads: the root was replaced by a Parcel, which has no `total`."""+ mthds_content = _REBOUND_ROOT_BUNDLE.replace('{ from = "record.weight", result = "weight" }', '{ from = "record.total", result = "weight" }')++ with pytest.raises(PipeValidationError) as exc_info:+ _load_sequence(mthds_content=mthds_content, library_id=load_empty_library(), pipe_code="read_records")++ assert exc_info.value.error_type == PipeValidationErrorType.BINDING_PATH_UNRESOLVED+ assert exc_info.value.pipe_code == "read_records"+ assert "holds a 'depot_records.Parcel', which has no field 'total'. Its fields are: 'weight', 'labels'." in str(exc_info.value)++ @pytest.mark.parametrize(+ ("output", "is_refused"),+ [+ pytest.param("Text", True, id="a-plain-output-is-refused"),+ pytest.param("Text?", False, id="an-optional-output-validates"),+ ],+ )+ def test_a_required_field_that_admits_none_may_leave_the_binding_absent(+ self, load_empty_library: Callable[[], str], output: str, is_refused: bool+ ) -> None:+ """A Python class can require a field and still let it hold `None`, so a sequence ending on it must declare its output `?`."""+ library_id = load_empty_library()+ # Registered in the library just opened, whose class registry the concept's `structure` is resolved through.+ get_class_registry().register_class(DepotSlipRecord)+ mthds_content = _NULLABLE_FIELD_BUNDLE.replace('output = "Text"', f'output = "{output}"')+ if not is_refused:+ sequence = _load_sequence(mthds_content=mthds_content, library_id=library_id, pipe_code="read_note")+ assert sequence.build_typed_flow().binding_derivations[0].may_find_nothing is True+ return+ with pytest.raises(PipeValidationError) as exc_info:+ _load_sequence(mthds_content=mthds_content, library_id=library_id, pipe_code="read_note")++ assert exc_info.value.error_type == PipeValidationErrorType.OPTIONAL_NOT_HANDLED+ assert "'slip.carrier_note'" in str(exc_info.value)++ @pytest.mark.parametrize(+ ("root_name", "concept_code"),+ [+ pytest.param("notice", "Notice", id="declared-as-a-string"),+ pytest.param("bulletin", "Bulletin", id="declared-as-a-table-with-a-description"),+ ],+ )+ def test_a_description_only_concept_is_never_entered(self, load_empty_library: Callable[[], str], root_name: str, concept_code: str) -> None:+ """Both spellings of a concept declared with a description alone are refused the same way, as having no structure."""+ mthds_content = _DESCRIPTION_ONLY_BUNDLE.replace('inputs = { notice = "Notice" }', f'inputs = {{ {root_name} = "{concept_code}" }}').replace(+ '{ from = "notice.text", result = "notice_text" }', f'{{ from = "{root_name}.text", result = "notice_text" }}'+ )++ with pytest.raises(PipeValidationError) as exc_info:+ _load_sequence(mthds_content=mthds_content, library_id=load_empty_library(), pipe_code="read_notice")++ assert exc_info.value.error_type == PipeValidationErrorType.BINDING_PATH_UNRESOLVED+ assert f"'{root_name}' holds a 'depot_notices.{concept_code}', which is declared with neither a structure nor refines" in str(exc_info.value)++ @pytest.mark.asyncio(loop_scope="class")+ @pytest.mark.parametrize(+ "step_count",+ [+ pytest.param("", id="a-plain-batch"),+ pytest.param(", nb_output = 1", id="a-batch-asking-for-one-output"),+ pytest.param(", nb_output = 3", id="a-batch-asking-for-three-outputs"),+ pytest.param(", multiple_output = true", id="a-batch-asking-for-multiple-outputs"),+ ],+ )+ async def test_a_batched_step_stores_a_list_whatever_count_it_asks_for(self, load_empty_library: Callable[[], str], step_count: str) -> None:+ """A batched step stores the list of its branches' results, so a binding over it is plural, as the run binds it."""+ mthds_content = _BATCHED_STEP_BUNDLE.replace('batch_as = "amount" }', f'batch_as = "amount"{step_count} }}')+ sequence = _load_sequence(mthds_content=mthds_content, library_id=load_empty_library(), pipe_code="read_weights")++ flow = sequence.build_typed_flow()++ assert flow.binding_derivations[1].multiplicity is True+ assert flow.binding_specs[1].concept.concept_ref == "native.Number"+ assert flow.binding_specs[1].multiplicity is True+ assert flow.final_slots["parcels"].stuff_spec is not None+ assert flow.final_slots["parcels"].stuff_spec.multiplicity is True++ inputs: PipelineInputs = {"amounts": {"concept": "native.Number", "content": [{"number": 2.5}, {"number": 4}]}}+ response = await PipelexMTHDSProtocol(pipe_run_mode=PipeRunMode.LIVE).execute(mthds_contents=[mthds_content], inputs=inputs)++ assert [item.number for item in response.pipe_output.main_stuff_as_items(item_type=NumberContent)] == [2.5, 4]++ @pytest.mark.asyncio(loop_scope="class")+ @pytest.mark.parametrize(+ ("replacements", "mode", "typed_record_ref"),+ [+ pytest.param(+ {+ '{ pipe = "swap_record", result = "note" }': '{ pipe = "swap_for_parcel", result = "note" }',+ 'what replaced it"\ninputs = { amount = "Number", mode = "Text" }': 'what replaced it"\ninputs = { amount = "Number" }',+ },+ None,+ "depot_swaps.Parcel",+ id="a-nested-sequence",+ ),+ pytest.param(+ {'crate = "swap_for_crate"': 'crate = "swap_for_parcel"'}, "crate", "depot_swaps.Parcel", id="a-condition-whose-outcomes-agree"+ ),+ ],+ )+ async def test_a_name_a_nested_controller_stores_types_the_binding_reading_it(+ self, load_empty_library: Callable[[], str], replacements: dict[str, str], mode: str | None, typed_record_ref: str+ ) -> None:+ """A nested sequence, or a condition's outcome, runs on the caller's memory, so what it stores is stored by the step+ running it, and the caller needs no input of that name.+ """+ mthds_content = _NESTED_WRITES_BUNDLE+ for old_text, new_text in replacements.items():+ assert old_text in mthds_content+ mthds_content = mthds_content.replace(old_text, new_text)+ sequence = _load_sequence(mthds_content=mthds_content, library_id=load_empty_library(), pipe_code="read_swapped_weight")++ flow = sequence.build_typed_flow()++ record_spec = flow.final_slots["record"].stuff_spec+ assert record_spec is not None+ assert record_spec.concept.concept_ref == typed_record_ref+ assert flow.binding_specs[1].concept.concept_ref == "native.Number"++ inputs: PipelineInputs = {"amount": {"concept": "native.Number", "content": {"number": 7.5}}}+ if mode is not None:+ inputs["mode"] = mode+ response = await PipelexMTHDSProtocol(pipe_run_mode=PipeRunMode.LIVE).execute(mthds_contents=[mthds_content], inputs=inputs)++ assert response.pipe_output.main_stuff.concept.concept_ref == "native.Number"+ assert response.pipe_output.main_stuff_as_number.number == 7.5++ @pytest.mark.parametrize(+ ("replacements", "stored_specs"),+ [+ pytest.param(+ {},+ "outcome 'swap_for_crate' of pipe 'swap_record' as 'Crate'; outcome 'swap_for_parcel' of pipe 'swap_record' as 'Parcel'",+ id="outcomes-storing-different-concepts",+ ),+ pytest.param(+ {+ # A `continue` outcome stores nothing, so the condition may hold nothing and leaves the caller's record in place.+ 'crate = "swap_for_crate"': 'crate = "continue"',+ 'output = "Text"\nexpression': 'output = "Text?"\nexpression',+ 'what replaced it"\ninputs = { amount = "Number", mode = "Text" }': (+ 'what replaced it"\ninputs = { record = "Invoice", amount = "Number", mode = "Text" }'+ ),+ },+ (+ "the value it held before step 1 (pipe 'swap_record'), which the step leaves when it stores nothing, as 'Invoice'; "+ "step 1 (pipe 'swap_record'), when it stores a value, as 'Parcel'"+ ),+ id="an-outcome-storing-another-concept-than-the-one-continue-leaves",+ ),+ ],+ )+ def test_a_root_stored_under_different_concepts_is_refused_before_the_run(+ self, load_empty_library: Callable[[], str], replacements: dict[str, str], stored_specs: str+ ) -> None:+ """Regression: the flow could not type a root the outcomes of a condition store under different concepts, so its binding+ was derived when it ran and the sequence could return a concept other than the one it declares. The binding is refused+ before the run, naming each outcome and what it stores.+ """+ mthds_content = _NESTED_WRITES_BUNDLE+ for old_text, new_text in replacements.items():+ assert old_text in mthds_content+ mthds_content = mthds_content.replace(old_text, new_text)++ with pytest.raises(PipeValidationError) as exc_info:+ _load_sequence(mthds_content=mthds_content, library_id=load_empty_library(), pipe_code="read_swapped_weight")++ assert exc_info.value.error_type == PipeValidationErrorType.BINDING_PATH_UNRESOLVED+ assert exc_info.value.pipe_code == "read_swapped_weight"+ message = str(exc_info.value)+ assert 'the binding step { from = "record.weight", result = "weight" } cannot be derived' in message+ assert (+ "the concept of 'record' is not known before the run, because the outcomes that may store it store it under different concepts" in message+ )+ assert stored_specs in message+ assert "Store 'record' under one concept in every outcome, or bind inside each outcome" in message++ @pytest.mark.asyncio(loop_scope="class")+ @pytest.mark.parametrize(+ ("contents", "expected_content"),+ [+ pytest.param("a spare part", TextContent(text="a spare part"), id="a-string"),+ pytest.param(12, NumberContent(number=12), id="a-number"),+ pytest.param({"order_ref": "PO-118"}, JSONContent(json_obj={"order_ref": "PO-118"}), id="an-object"),+ ],+ )+ async def test_a_binding_ending_on_an_anything_field_runs(self, contents: object, expected_content: StuffContent) -> None:+ """The path may end on a field holding `native.Anything`, and the run stores its value as an `Anything` input is shaped."""+ inputs: PipelineInputs = {"crate": {"concept": "depot_crates.Crate", "content": {"contents": contents}}}++ response = await PipelexMTHDSProtocol(pipe_run_mode=PipeRunMode.LIVE).execute(mthds_contents=[_ANYTHING_FIELD_BUNDLE], inputs=inputs)++ assert response.pipe_output.main_stuff.concept.concept_ref == "native.Anything"+ assert response.pipe_output.main_stuff.content == expected_content++ @pytest.mark.asyncio(loop_scope="class")+ @pytest.mark.parametrize(+ ("entry_name", "message_fragments"),+ [+ pytest.param(+ "invalid_binding_path_unresolved_unknown_field",+ ["has no field 'totl'", "Its fields are: 'total', 'lines', 'order_details'"],+ id="I11-names-the-available-fields",+ ),+ pytest.param(+ "invalid_binding_path_unresolved_past_a_leaf",+ ["'invoice.total'", "'amount'"],+ id="I12-past-a-leaf",+ ),+ pytest.param(+ "invalid_binding_path_unresolved_root_without_structure",+ ["'note' holds a 'native.Text', which holds its value in a single field"],+ id="I13-root-without-structure",+ ),+ pytest.param(+ "invalid_binding_path_unresolved_through_a_dict",+ ["'order_ref'", "'invoice.order_details'"],+ id="I14-through-a-dict",+ ),+ pytest.param(+ "invalid_missing_input_variable_binding_root",+ ["Declare 'invoice' in the sequence's `inputs`"],+ id="I15-root-nowhere",+ ),+ pytest.param(+ "invalid_input_stuff_spec_mismatch_binding",+ ["total_amount", "Number"],+ id="I16-consumer-mismatch",+ ),+ pytest.param(+ "invalid_inadequate_output_multiplicity_binding",+ ["page_views"],+ id="I17-plural-into-singular",+ ),+ pytest.param(+ "invalid_optional_not_handled_binding",+ ['binding step { from = "delivery_round.note", result = "note" }', "'delivery_round.note'"],+ id="I18-absence-escapes",+ ),+ ],+ )+ async def test_a_binding_refusal_names_what_to_change(self, entry_name: str, message_fragments: list[str]) -> None:+ entry_directory = entries_root() / entry_name++ with pytest.raises(ValidateBundleError) as exc_info:+ await validate_bundle(mthds_file_path=entry_directory / "bundle.mthds", library_dirs=[entry_directory])++ validation_errors = exc_info.value.to_error_report().validation_errors or []+ assert len(validation_errors) == 1+ message = validation_errors[0].message or ""+ for fragment in message_fragments:+ assert fragment in message, f"'{fragment}' not in: {message}"++ @pytest.mark.parametrize(+ ("called_pipe", "step_count", "output", "requested"),+ [+ pytest.param("read_total", "nb_output = 2", "Number[]", "2 outputs", id="a-count-of-a-single-value"),+ pytest.param("read_total", "multiple_output = true", "Number[]", "multiple outputs", id="a-list-of-a-single-value"),+ pytest.param("read_amounts", "nb_output = 3", "Number[]", "3 outputs", id="a-count-of-a-list"),+ pytest.param("read_amounts", "nb_output = 1", "Number", "a single output", id="a-single-value-of-a-list"),+ ],+ )+ def test_a_count_a_final_binding_cannot_honour_is_refused(+ self, load_empty_library: Callable[[], str], called_pipe: str, step_count: str, output: str, requested: str+ ) -> None:+ """A binding binds what its path derives, whatever count its caller asks for, so a step asking a sequence ending with one for+ another count is refused, rather than run with the count reaching the sequence's own steps.+ """+ mthds_content = _called_binding_bundle(called_pipe=called_pipe, step_count=step_count, output=output)++ with pytest.raises(PipeValidationError) as exc_info:+ _load_sequence(mthds_content=mthds_content, library_id=load_empty_library(), pipe_code="read_totals")++ assert exc_info.value.error_type == PipeValidationErrorType.INADEQUATE_OUTPUT_MULTIPLICITY+ assert exc_info.value.pipe_code == "read_totals"+ assert f"step 1 asks pipe '{called_pipe}' for {requested}, but '{called_pipe}' ends with the binding step" in str(exc_info.value)++ @pytest.mark.parametrize(+ ("called_pipe", "step_count", "output"),+ [+ pytest.param("read_total", "nb_output = 1", "Number", id="one-output-of-a-single-value"),+ pytest.param("read_amounts", "multiple_output = true", "Number[]", id="multiple-outputs-of-a-list"),+ ],+ )+ def test_a_count_a_final_binding_honours_is_accepted(+ self, load_empty_library: Callable[[], str], called_pipe: str, step_count: str, output: str+ ) -> None:+ mthds_content = _called_binding_bundle(called_pipe=called_pipe, step_count=step_count, output=output)++ _load_sequence(mthds_content=mthds_content, library_id=load_empty_library(), pipe_code="read_totals")++ @pytest.mark.asyncio(loop_scope="class")+ async def test_a_single_binding_over_a_root_the_run_made_a_list_is_a_run_error(self) -> None:+ """Regression: a count asked of the whole run reaches the sequence's own steps, so the step typed as writing one invoice+ wrote two, and the binding of its total returned the first total alone. It now refuses to choose.+ """+ with pytest.raises(PipelexError) as exc_info:+ await PipelexMTHDSProtocol(pipe_run_mode=PipeRunMode.DRY).execute(+ mthds_contents=[_WRITTEN_INVOICE_BUNDLE], inputs={"note": "Two invoices from the depot"}, output_multiplicity=2+ )++ message = str(exc_info.value)+ assert "Pipe 'read_written_total' failed" in message+ assert "Binding 'invoice.total' was derived as a single value, but its root 'invoice' holds a list of 2 items" in message++ @pytest.mark.asyncio(loop_scope="class")+ @pytest.mark.parametrize(+ ("output", "steps", "expected_output"),+ [+ pytest.param("Number", ['{ from = "parcel.weight", result = "weight" }'], 7.5, id="a-final-binding"),+ pytest.param(+ "Text",+ ['{ from = "parcel.weight", result = "weight" }', '{ pipe = "write_note", result = "note" }'],+ "Weighed 7.5 kilograms",+ id="a-binding-read-by-a-later-step",+ ),+ ],+ )+ async def test_a_root_a_pipe_unresolved_at_validation_stores_is_bound_when_it_runs(+ self, load_empty_library: Callable[[], str], mocker: MockerFixture, output: str, steps: list[str], expected_output: float | str+ ) -> None:+ """A pipe that does not resolve at validation is assumed to deliver, so the binding reading what it stores is derived+ when it runs, from the value it finds, and runs when that value fits what reads it.+ """+ hide_pipe_from_sequence_analyses(mocker=mocker, pipe_code="weigh_parcel")+ mthds_content = _unresolved_producer_bundle(output=output, steps=steps)+ sequence = _load_sequence(mthds_content=mthds_content, library_id=load_empty_library(), pipe_code="read_parcel")+ assert 1 not in sequence.build_typed_flow().binding_specs++ response = await PipelexMTHDSProtocol(pipe_run_mode=PipeRunMode.LIVE).execute(+ mthds_contents=[mthds_content], inputs={"amount": {"concept": "native.Number", "content": {"number": 7.5}}}+ )++ if isinstance(expected_output, str):+ assert response.pipe_output.main_stuff_as_str == expected_output+ else:+ assert response.pipe_output.main_stuff_as_number.number == expected_output++ @pytest.mark.asyncio(loop_scope="class")+ @pytest.mark.parametrize(+ ("output", "steps", "message_fragments"),+ [+ pytest.param(+ "Number",+ ['{ from = "parcel.carrier", result = "weight" }'],+ ["ends the sequence, whose output is declared 'Number', but it binds 'Text'"],+ id="a-final-binding-of-another-concept",+ ),+ pytest.param(+ "Number",+ ['{ from = "parcel", result = "weight" }'],+ ["ends the sequence, whose output is declared 'Number', but it binds 'Parcel'"],+ id="a-final-bare-name-of-another-concept",+ ),+ pytest.param(+ "Text",+ ['{ from = "parcel.carrier", result = "weight" }', '{ pipe = "write_note", result = "note" }'],+ ["step 3 (pipe 'write_note') reads 'weight' as 'Number', but the binding step", "binds it as 'Text'"],+ id="a-binding-read-by-a-later-step-as-another-concept",+ ),+ ],+ )+ async def test_a_binding_derived_when_it_runs_that_contradicts_what_reads_it_is_a_run_error(+ self, load_empty_library: Callable[[], str], mocker: MockerFixture, output: str, steps: list[str], message_fragments: list[str]+ ) -> None:+ """Regression: a binding the flow could not type was derived when it ran and its result went unchecked, so the sequence+ returned a `Text` where it declares a `Number`. The run now checks it against the sequence's output, or against the input+ of the step reading it, as validation checks a binding it derives.+ """+ hide_pipe_from_sequence_analyses(mocker=mocker, pipe_code="weigh_parcel")+ mthds_content = _unresolved_producer_bundle(output=output, steps=steps)+ _load_sequence(mthds_content=mthds_content, library_id=load_empty_library(), pipe_code="read_parcel")++ with pytest.raises(PipelexError) as exc_info:+ await PipelexMTHDSProtocol(pipe_run_mode=PipeRunMode.LIVE).execute(+ mthds_contents=[mthds_content], inputs={"amount": {"concept": "native.Number", "content": {"number": 7.5}}}+ )++ message = str(exc_info.value)+ for fragment in message_fragments:+ assert fragment in message, f"'{fragment}' not in: {message}"+ assert "pipe that did not resolve at validation stored" in messagediff --git a/tests/integration/pipelex/pipes/controller/pipe_sequence/test_pipe_sequence_creation.py b/tests/integration/pipelex/pipes/controller/pipe_sequence/test_pipe_sequence_creation.pyindex 591bd3c53..e42c2106c 100644--- a/tests/integration/pipelex/pipes/controller/pipe_sequence/test_pipe_sequence_creation.py+++ b/tests/integration/pipelex/pipes/controller/pipe_sequence/test_pipe_sequence_creation.py@@ -45,6 +45,6 @@ class TestPipeSequenceCreation:assert pipe_sequence.code == "test_sequence"assert pipe_sequence.domain_code == domain_codeassert len(pipe_sequence.sequential_sub_pipes) == 1- assert pipe_sequence.sequential_sub_pipes[0].pipe_code == "test_pipe_1"+ assert pipe_sequence.pipe_steps[0].pipe_code == "test_pipe_1"assert pipe_sequence.sequential_sub_pipes[0].output_name == "intermediate_result"concept_library.teardown()diff --git a/tests/integration/pipelex/pipes/controller/pipe_sequence/test_pipe_sequence_simple.py b/tests/integration/pipelex/pipes/controller/pipe_sequence/test_pipe_sequence_simple.pyindex 0f359ff83..e1f6fa40a 100644--- a/tests/integration/pipelex/pipes/controller/pipe_sequence/test_pipe_sequence_simple.py+++ b/tests/integration/pipelex/pipes/controller/pipe_sequence/test_pipe_sequence_simple.py@@ -72,9 +72,9 @@ class TestPipeSequenceSimple:assert pipe_sequence.domain_code == "test_integration"assert pipe_sequence.code == "simple_sequence"assert len(pipe_sequence.sequential_sub_pipes) == 2- assert pipe_sequence.sequential_sub_pipes[0].pipe_code == "test_integration4.capitalize_text"+ assert pipe_sequence.pipe_steps[0].pipe_code == "test_integration4.capitalize_text"assert pipe_sequence.sequential_sub_pipes[0].output_name == "capitalized_text"- assert pipe_sequence.sequential_sub_pipes[1].pipe_code == "test_integration4.add_prefix"+ assert pipe_sequence.pipe_steps[1].pipe_code == "test_integration4.add_prefix"assert pipe_sequence.sequential_sub_pipes[1].output_name == "final_text"# Verify the working memory has the correct structurediff --git a/tests/integration/pipelex/pipes/controller/pipe_sequence/unresolved_pipes.py b/tests/integration/pipelex/pipes/controller/pipe_sequence/unresolved_pipes.pynew file mode 100644index 000000000..9584272be--- /dev/null+++ b/tests/integration/pipelex/pipes/controller/pipe_sequence/unresolved_pipes.py@@ -0,0 +1,27 @@+"""Make a pipe look unresolved to a sequence's static analyses while the run still finds it.++A pipe a sequence's step names may not resolve at validation, as a dependency's pipe the library has not loaded yet, and the+sequence then assumes it delivers: the typed flow cannot type what it stores, so a binding reading it is derived when it runs.+A bundle loaded whole always resolves its pipes, so the tests reach that state by hiding the pipe from the lookups of the+typed flow and of the sequence's own analyses, while the run resolves it as usual.+"""++from pytest_mock import MockerFixture++from pipelex.interpreter_hub import get_optional_pipe+from pipelex.pipe_controllers.sequence import pipe_sequence as sequence_module+from pipelex.pipe_controllers.sequence import sequence_typed_flow as typed_flow_module+from pipelex.pipe_machinery.pipe_abstract import PipeAbstract+++def hide_pipe_from_sequence_analyses(*, mocker: MockerFixture, pipe_code: str) -> None:+ """Have the typed flow and the sequence's static analyses find no pipe named `pipe_code`, whatever its domain."""+ hidden_pipe_code = pipe_code++ def get_optional_pipe_but_hidden(*, pipe_code: str) -> PipeAbstract | None:+ if pipe_code.rsplit(".", maxsplit=1)[-1] == hidden_pipe_code:+ return None+ return get_optional_pipe(pipe_code=pipe_code)++ mocker.patch.object(typed_flow_module, "get_optional_pipe", side_effect=get_optional_pipe_but_hidden)+ mocker.patch.object(sequence_module, "get_optional_pipe", side_effect=get_optional_pipe_but_hidden)diff --git a/tests/integration/pipelex/pipes/operator/pipe_structure/test_preliminary_text_e2e.py b/tests/integration/pipelex/pipes/operator/pipe_structure/test_preliminary_text_e2e.pyindex d85685cf9..adeea3b84 100644--- a/tests/integration/pipelex/pipes/operator/pipe_structure/test_preliminary_text_e2e.py+++ b/tests/integration/pipelex/pipes/operator/pipe_structure/test_preliminary_text_e2e.py@@ -96,8 +96,8 @@ class TestPreliminaryTextE2E:user_facing_pipe = get_required_entry_pipe(pipe_code=pipe_code)assert isinstance(user_facing_pipe, PipeSequence)assert len(user_facing_pipe.sequential_sub_pipes) == 2- assert user_facing_pipe.sequential_sub_pipes[0].pipe_code == f"test_preliminary_text_e2e.{pipe_code}__draft_text"- assert user_facing_pipe.sequential_sub_pipes[1].pipe_code == f"test_preliminary_text_e2e.{pipe_code}__structure"+ assert user_facing_pipe.pipe_steps[0].pipe_code == f"test_preliminary_text_e2e.{pipe_code}__draft_text"+ assert user_facing_pipe.pipe_steps[1].pipe_code == f"test_preliminary_text_e2e.{pipe_code}__structure"structure_step = get_required_entry_pipe(pipe_code=f"{pipe_code}__structure")assert isinstance(structure_step, PipeStructure)diff --git a/tests/integration/pipelex/pipes/operator/pipe_structure/test_preliminary_text_inline_e2e.py b/tests/integration/pipelex/pipes/operator/pipe_structure/test_preliminary_text_inline_e2e.pyindex 60cc071ac..7a694aab2 100644--- a/tests/integration/pipelex/pipes/operator/pipe_structure/test_preliminary_text_inline_e2e.py+++ b/tests/integration/pipelex/pipes/operator/pipe_structure/test_preliminary_text_inline_e2e.py@@ -94,8 +94,8 @@ class TestPreliminaryTextInlineE2E:user_facing_pipe = get_required_entry_pipe(pipe_code=pipe_code)assert isinstance(user_facing_pipe, PipeSequence)assert len(user_facing_pipe.sequential_sub_pipes) == 2- assert user_facing_pipe.sequential_sub_pipes[0].pipe_code == f"test_preliminary_text_inline_e2e.{pipe_code}__draft_text"- assert user_facing_pipe.sequential_sub_pipes[1].pipe_code == f"test_preliminary_text_inline_e2e.{pipe_code}__structure"+ assert user_facing_pipe.pipe_steps[0].pipe_code == f"test_preliminary_text_inline_e2e.{pipe_code}__draft_text"+ assert user_facing_pipe.pipe_steps[1].pipe_code == f"test_preliminary_text_inline_e2e.{pipe_code}__structure"structure_step = get_required_entry_pipe(pipe_code=f"{pipe_code}__structure")assert isinstance(structure_step, PipeStructure)diff --git a/tests/integration/pipelex/pipes/optionals/test_taint_sequence_validation.py b/tests/integration/pipelex/pipes/optionals/test_taint_sequence_validation.pyindex 45f9ec9bb..aba300a69 100644--- a/tests/integration/pipelex/pipes/optionals/test_taint_sequence_validation.py+++ b/tests/integration/pipelex/pipes/optionals/test_taint_sequence_validation.py@@ -91,7 +91,7 @@ def _build_sequence(*, output_ref: str, steps: list[SubPipeBlueprint], inputs: ddescription="Taint-pass test sequence",inputs=inputs,output=output_ref,- steps=steps,+ steps=list(steps),),)get_pipe_library().add_new_pipe(pipe=sequence)diff --git a/tests/integration/pipelex/test_extras/test_data.py b/tests/integration/pipelex/test_extras/test_data.pynew file mode 100644index 000000000..487dbc785--- /dev/null+++ b/tests/integration/pipelex/test_extras/test_data.py@@ -0,0 +1,144 @@+from typing import Any, ClassVar, NamedTuple++from pipelex.core.memory.absence import AbsenceKind+from pipelex.core.memory.working_memory import MAIN_STUFF_NAME+++class Bound(NamedTuple):+ """A value the run must leave in working memory: its concept, and the fields its content must hold.++ The content is matched as a subset: every key written here must be present with this value, and a list+ must have exactly these items, each matched the same way. Fields the run stamps (an image's mime type,+ a relocated url) are left out.+ """++ concept_ref: str+ content: Any+++class Absent(NamedTuple):+ """A name the run must leave without a value, with a recorded absence of this kind."""++ kind: AbsenceKind+++class OfflineRunExpectations:+ """What each offline-tier corpus entry leaves in working memory once run with its `inputs.json`, keyed by entry name.++ A corpus entry holds no expected outputs, so the expectations live here. Every offline entry has one, the+ entries that predate the binding step included, so the runner pins what each of them produces.+ """++ BY_ENTRY: ClassVar[dict[str, dict[str, Bound | Absent]]] = {+ "operator_compose_delivery_docket": {+ MAIN_STUFF_NAME: Bound(+ concept_ref="deliveries.DeliveryDocket",+ content={"weight_kg": 3.4, "destination": "Quimper", "carrier": "Morvan et Fils"},+ ),+ },+ "operator_doc_gen_door_notice": {+ MAIN_STUFF_NAME: Bound(+ concept_ref="shop_notices.PrintedNotice", content={"mime_type": "application/pdf", "filename": "notice-Boulangerie-Kerlann.pdf"}+ ),+ },+ "feature_binding_step_invoice_fields": {+ "supplier_name": Bound(concept_ref="native.Text", content={"text": "Atelier Morvan"}),+ "total_amount": Bound(concept_ref="native.Number", content={"number": 1250.5}),+ "line_count": Bound(concept_ref="native.Number", content={"number": 3}),+ "priority": Bound(concept_ref="native.Text", content={"text": "urgent"}),+ "order_details": Bound(concept_ref="native.JSON", content={"json_obj": {"order_ref": "PO-118", "delivery_slot": "morning"}}),+ "supplier_city": Bound(concept_ref="native.Text", content={"text": "Quimper"}),+ "acknowledgement": Bound(+ concept_ref="native.Text",+ content={+ "text": (+ "Received from Atelier Morvan of Quimper: 3 lines, 1250.5 euros due, urgent.\n"+ 'Order details: {\n "order_ref": "PO-118",\n "delivery_slot": "morning"\n}'+ )+ },+ ),+ },+ "feature_binding_step_parcel_recipient": {+ "fragile": Bound(concept_ref="native.YesNo", content={"yes_no": True}),+ "recipient": Bound(+ concept_ref="parcel_counter.Recipient",+ content={"name": "Maëlle Le Gall", "street": "12 rue des Tanneurs", "town": "Morlaix"},+ ),+ "label": Bound(+ concept_ref="parcel_counter.ParcelLabel",+ content={"addressee": "Maëlle Le Gall", "street": "12 rue des Tanneurs", "town": "Morlaix", "handling": "Fragile, handle with care"},+ ),+ },+ "feature_binding_step_departure_times": {+ "day": Bound(concept_ref="native.Date", content={"date": "2026-04-02"}),+ "scheduled_at": Bound(concept_ref="native.Date", content={"date": "2026-04-02", "time": "08:15:00+02:00"}),+ "boarding_time": Bound(concept_ref="native.Time", content={"time": "07:45:00"}),+ },+ "feature_binding_step_page_views": {+ "cover_view": Bound(+ concept_ref="native.Image",+ content={"caption": "The spring cover", "source_prompt": "A cover photographed flat"},+ ),+ "page_views": Bound(+ concept_ref="native.Image",+ content={"items": [{"caption": "Garden chairs on a lawn"}, {"caption": "Two parasols"}]},+ ),+ },+ "feature_binding_step_order_lines": {+ "lines": Bound(+ concept_ref="order_desk.OrderLine",+ content={+ "items": [+ {"article": "Linen tea towel", "amount": 12.5},+ {"article": "Sample swatch", "amount": None},+ {"article": "Enamel mug", "amount": 9.0},+ ]+ },+ ),+ "amounts": Bound(concept_ref="native.Number", content={"items": [{"number": 12.5}, {"number": 9.0}]}),+ "backorder_amounts": Bound(concept_ref="native.Number", content={"items": []}),+ },+ "feature_binding_step_delivery_note": {+ "morning_note": Bound(concept_ref="native.Text", content={"text": "Gate code changed to 4471"}),+ "evening_note": Absent(kind=AbsenceKind.DECLARED_ABSENT),+ "board_notice": Bound(concept_ref="native.Text", content={"text": "Morning: Gate code changed to 4471\nEvening: no note"}),+ "evening_reminder": Absent(kind=AbsenceKind.SKIPPED),+ },+ "feature_binding_step_missing_values": {+ "invoice_number": Bound(concept_ref="native.Text", content={"text": "F-2026-0412"}),+ "scan_link": Absent(kind=AbsenceKind.DECLARED_ABSENT),+ "copy_total": Absent(kind=AbsenceKind.SKIPPED),+ "filing_note": Bound(concept_ref="native.Text", content={"text": "Invoice F-2026-0412 filed.\nNo scan on file.\nNo courtesy copy."}),+ },+ "feature_binding_step_receipt_inherited": {+ "amount_paid": Bound(concept_ref="native.Number", content={"number": 42.8}),+ "line": Bound(concept_ref="native.Text", content={"text": "Paid: 42.8 euros"}),+ },+ "feature_binding_step_board_rename": {+ "board": Bound(concept_ref="station_board.DepartureBoard", content={"station": "Landerneau", "next_train": "Brest"}),+ "announcement": Bound(concept_ref="native.Text", content={"text": "At Landerneau, the next train goes to Brest."}),+ },+ "feature_binding_step_batch_views": {+ "page_views": Bound(concept_ref="native.Image", content={"items": [{"caption": "Garden chairs on a lawn"}, {"caption": "Two parasols"}]}),+ "index_lines": Bound(+ concept_ref="native.Text",+ content={"items": [{"text": "Photograph: Garden chairs on a lawn"}, {"text": "Photograph: Two parasols"}]},+ ),+ },+ "feature_binding_step_parallel_root": {+ "price_tag": Bound(concept_ref="market_stall.PriceTag", content={"produce": "Plougastel strawberries", "price": 3.2}),+ "price": Bound(concept_ref="native.Number", content={"number": 3.2}),+ "price_line": Bound(concept_ref="native.Text", content={"text": "3.2 euros a kilo"}),+ },+ "feature_binding_step_nested_sequence": {+ "loaf_count": Bound(concept_ref="native.Number", content={"number": 12}),+ "confirmation": Bound(concept_ref="native.Text", content={"text": "Confirmed: 12 loaves"}),+ },+ "feature_binding_step_shipment_parcels": {+ "parcels": Bound(+ concept_ref="depot_loading.Parcel",+ content={"items": [{"tracking_number": "QP-1001"}, {"tracking_number": "QP-1002"}, {"tracking_number": "BR-2001"}]},+ ),+ "loading_list": Bound(concept_ref="native.Text", content={"text": "- QP-1001\n- QP-1002\n- BR-2001\n"}),+ },+ }diff --git a/tests/integration/pipelex/test_extras/test_mthds_corpus_offline_runs.py b/tests/integration/pipelex/test_extras/test_mthds_corpus_offline_runs.pynew file mode 100644index 000000000..559932d3e--- /dev/null+++ b/tests/integration/pipelex/test_extras/test_mthds_corpus_offline_runs.py@@ -0,0 +1,90 @@+"""The MTHDS Test Corpus offline runner: every offline-tier entry runs live with its `inputs.json`.++An offline entry is one whose run needs no inference: a sequence of bindings, a PipeCompose, a PipeDocGen.+Running it is the only way to check the values a binding step binds, since the dry run fills every input+with a mock. A corpus entry holds no expected outputs, so what each entry must leave in working memory is+kept in this suite's test data, keyed by entry name, and every offline entry must have expectations there.+"""++import json+from typing import Any, cast++import pytest++from pipelex.core.stuffs.list_content import ListContent+from pipelex.core.stuffs.stuff_content import StuffContent+from pipelex.mthds_parsing.parser import MthdsParser+from pipelex.pipeline.runner import PipelexMTHDSProtocol+from pipelex.system.pipe_run_mode import PipeRunMode+from pipelex.test_extras.mthds_corpus.loader import CorpusEntry, iter_entries+from pipelex.test_extras.mthds_corpus.manifest import EntryTier, EntryValidity+from tests.integration.pipelex.test_extras.test_data import Absent, Bound, OfflineRunExpectations++_OFFLINE_ENTRIES = [entry for entry in iter_entries(validity=EntryValidity.VALID) if entry.manifest.tier is EntryTier.OFFLINE]+++def _entry_id(entry: CorpusEntry) -> str:+ return entry.name+++def _assert_matches(*, actual: Any, expected: Any, where: str) -> None:+ """Match `actual` against `expected` as a subset: every expected key, every list item, compared recursively."""+ if isinstance(expected, dict):+ assert isinstance(actual, dict), f"{where}: expected a table, got {actual!r}"+ actual_dict = cast("dict[str, Any]", actual)+ for key, expected_value in cast("dict[str, Any]", expected).items():+ assert key in actual_dict, f"{where}: missing '{key}' in {actual_dict!r}"+ _assert_matches(actual=actual_dict[key], expected=expected_value, where=f"{where}.{key}")+ return+ if isinstance(expected, list):+ assert isinstance(actual, list), f"{where}: expected a list, got {actual!r}"+ actual_list = cast("list[Any]", actual)+ expected_list = cast("list[Any]", expected)+ assert len(actual_list) == len(expected_list), f"{where}: expected {len(expected_list)} items, got {actual_list!r}"+ for index_item, (actual_item, expected_item) in enumerate(zip(actual_list, expected_list, strict=True)):+ _assert_matches(actual=actual_item, expected=expected_item, where=f"{where}[{index_item}]")+ return+ assert actual == expected, f"{where}: expected {expected!r}, got {actual!r}"+++def _content_dump(content: StuffContent) -> dict[str, Any]:+ if isinstance(content, ListContent):+ items = cast("ListContent[StuffContent]", content).items+ return {"items": [item.model_dump(mode="json") for item in items]}+ return content.model_dump(mode="json")+++class TestMthdsCorpusOfflineRuns:+ def test_every_offline_entry_has_expectations(self) -> None:+ offline_names = {entry.name for entry in _OFFLINE_ENTRIES}+ assert offline_names == set(OfflineRunExpectations.BY_ENTRY), (+ "Every offline-tier entry needs its expectations in OfflineRunExpectations.BY_ENTRY, and nothing else may be listed there"+ )++ @pytest.mark.asyncio(loop_scope="class")+ @pytest.mark.parametrize("entry", _OFFLINE_ENTRIES, ids=_entry_id)+ async def test_offline_entry_runs_and_leaves_the_expected_values(self, entry: CorpusEntry) -> None:+ assert entry.inputs_path is not None, f"Offline entry '{entry.name}' has no inputs.json to run with"+ inputs = json.loads(entry.inputs_path.read_text(encoding="utf-8"))+ main_pipe = MthdsParser.make_pipelex_bundle_blueprint(bundle_path=entry.bundle_path).main_pipe+ assert main_pipe is not None, f"Offline entry '{entry.name}' declares no main_pipe"++ result = await PipelexMTHDSProtocol(library_dirs=[str(entry.directory)], pipe_run_mode=PipeRunMode.LIVE).execute(+ pipe_code=main_pipe,+ inputs=inputs,+ )++ working_memory = result.pipe_output.working_memory+ for variable_name, expectation in OfflineRunExpectations.BY_ENTRY[entry.name].items():+ where = f"{entry.name}: '{variable_name}'"+ match expectation:+ case Bound(concept_ref=concept_ref, content=expected_content):+ stuff = working_memory.get_optional_stuff(variable_name)+ assert stuff is not None, f"{where} holds no value; absence: {working_memory.get_optional_absence(variable_name)}"+ assert stuff.concept.concept_ref == concept_ref, f"{where} is a '{stuff.concept.concept_ref}', expected '{concept_ref}'"+ _assert_matches(actual=_content_dump(stuff.content), expected=expected_content, where=where)+ case Absent(kind=kind):+ assert working_memory.get_optional_stuff(variable_name) is None, f"{where} holds a value, expected an absence"+ absence = working_memory.get_optional_absence(variable_name)+ assert absence is not None, f"{where} has no recorded absence"+ assert absence.kind == kind, f"{where} is recorded '{absence.kind}', expected '{kind}': {absence.reason}"diff --git a/tests/unit/pipelex/codegen/test_crate_encoding.py b/tests/unit/pipelex/codegen/test_crate_encoding.pyindex 4ac2a62c9..df73f2530 100644--- a/tests/unit/pipelex/codegen/test_crate_encoding.py+++ b/tests/unit/pipelex/codegen/test_crate_encoding.py@@ -8,6 +8,10 @@ from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlfrom pipelex.core.domains.domain_blueprint import DomainBlueprintfrom pipelex.libraries.crate_normalization import normalize_cratefrom pipelex.libraries.library_crate import LibraryCrate+from pipelex.pipe_controllers.binding.binding_step_blueprint import BindingStepBlueprint+from pipelex.pipe_controllers.sequence.pipe_sequence_blueprint import PipeSequenceBlueprint+from pipelex.pipe_controllers.sub_pipe_blueprint import SubPipeBlueprint+from pipelex.pipe_operators.compose.pipe_compose_blueprint import PipeComposeBlueprintfrom pipelex.pipe_operators.llm.pipe_llm_blueprint import PipeLLMBlueprintMTHDS_TEST_VERSION = "0.1.0-test"@@ -34,6 +38,39 @@ def _normalized_crate() -> LibraryCrate:return normalize_crate(crate, mthds_version=MTHDS_TEST_VERSION)+def _binding_crate() -> LibraryCrate:+ """A normalized crate whose sequence binds a field of its input before a pipe step reads it."""+ crate = LibraryCrate(+ concepts={+ "billing.Invoice": ConceptBlueprint(+ description="An invoice sent by a supplier",+ structure={+ "total": ConceptStructureBlueprint(description="The amount due", type=ConceptStructureBlueprintFieldType.NUMBER, required=True),+ },+ ),+ },+ pipes={+ "billing.write_receipt": PipeComposeBlueprint(+ description="Writes a receipt for an amount",+ inputs={"total_amount": "Number"},+ output="Text",+ template="Received $total_amount",+ ),+ "billing.acknowledge_invoice": PipeSequenceBlueprint(+ description="Acknowledges an invoice by its total",+ inputs={"invoice": "Invoice"},+ output="Text",+ steps=[+ BindingStepBlueprint.model_validate({"from": "invoice.total", "result": "total_amount"}),+ SubPipeBlueprint(pipe="write_receipt", result="receipt"),+ ],+ ),+ },+ domains={"billing": DomainBlueprint(code="billing", description="Billing domain")},+ )+ return normalize_crate(crate, mthds_version=MTHDS_TEST_VERSION)++class TestCrateEncoding:"""Unit tests for the JSON and TOML crate encodings."""@@ -83,3 +120,18 @@ class TestCrateEncoding:crate = _normalized_crate()assert encode_crate(crate, encoding=CrateEncoding.JSON) == encode_crate_json(crate)assert encode_crate(crate, encoding=CrateEncoding.TOML) == encode_crate_toml(crate)++ def test_a_binding_step_is_emitted_under_from_and_round_trips(self):+ """A binding step is written as MTHDS spells it, `from`, so a crate holding one validates again with the same fingerprint."""+ crate = _binding_crate()+ json_doc = json.loads(encode_crate_json(crate))+ toml_doc = tomli.loads(encode_crate_toml(crate))+ for doc in (json_doc, toml_doc):+ assert doc["pipes"]["billing.acknowledge_invoice"]["steps"][0] == {"from": "invoice.total", "result": "total_amount"}++ from_json = LibraryCrate.model_validate(json_doc)+ from_toml = LibraryCrate.model_validate(toml_doc)+ assert from_json.pipes == crate.pipes+ assert from_toml.pipes == crate.pipes+ assert from_json.compute_normalized() == crate.fingerprint+ assert from_toml.compute_normalized() == crate.fingerprintdiff --git a/tests/unit/pipelex/core/concepts/concept_factory/data.py b/tests/unit/pipelex/core/concepts/concept_factory/data.pyindex d248de96f..d22e96522 100644--- a/tests/unit/pipelex/core/concepts/concept_factory/data.py+++ b/tests/unit/pipelex/core/concepts/concept_factory/data.py@@ -10,6 +10,18 @@ from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlfrom pipelex.core.concepts.native.concept_native import NativeConceptCodefrom pipelex.core.domains.domain import SpecialDomain+# A concept's declared fields, which its runtime concept keeps as `declared_structure` for the binding walk.+PERSON_STRUCTURE: dict[str, ConceptStructureBlueprint] = {+ "name": ConceptStructureBlueprint(description="The person's name", type=ConceptStructureBlueprintFieldType.TEXT, required=True),+ "age": ConceptStructureBlueprint(description="The person's age", type=ConceptStructureBlueprintFieldType.NUMBER, required=True),+ "active": ConceptStructureBlueprint(+ description="Whether the person is active",+ type=ConceptStructureBlueprintFieldType.BOOLEAN,+ required=False,+ default_value=True,+ ),+}+class TestCases:# Test cases for make_refines method@@ -104,6 +116,7 @@ class TestCases:description="A concept from same domain",structure_class_name="my_domain__DomainConcept",refines="native.Text",+ is_described_only=True,),),# Test case 9: Blueprint with dict structure (same domain)@@ -113,16 +126,7 @@ class TestCases:"PersonConcept",ConceptBlueprint(description="A person with structured data",- structure={- "name": ConceptStructureBlueprint(description="The person's name", type=ConceptStructureBlueprintFieldType.TEXT, required=True),- "age": ConceptStructureBlueprint(description="The person's age", type=ConceptStructureBlueprintFieldType.NUMBER, required=True),- "active": ConceptStructureBlueprint(- description="Whether the person is active",- type=ConceptStructureBlueprintFieldType.BOOLEAN,- required=False,- default_value=True,- ),- },+ structure={**PERSON_STRUCTURE},),Concept(domain_code="my_domain",@@ -130,6 +134,7 @@ class TestCases:description="A person with structured data",structure_class_name="my_domain__PersonConcept",refines=None,+ declared_structure=PERSON_STRUCTURE,),),# Test case 11: Blueprint with refines (same domain)diff --git a/tests/unit/pipelex/language/test_mthds_schema.py b/tests/unit/pipelex/language/test_mthds_schema.pyindex bd591251d..bca7be9e2 100644--- a/tests/unit/pipelex/language/test_mthds_schema.py+++ b/tests/unit/pipelex/language/test_mthds_schema.py@@ -360,6 +360,43 @@ class TestMthdsSchemaGeneration:table = {**_minimal_pipe_table("PipeBatch"), "input_list_name": input_list_name}assert validator.is_valid(table) is should_validate+ @pytest.mark.parametrize(+ ("step", "should_validate"),+ [+ pytest.param({"pipe": "sub_pipe"}, True, id="pipe-step"),+ pytest.param({"from": "invoice.total", "result": "total_amount"}, True, id="binding-step"),+ pytest.param({"from": "invoice", "result": "invoice_copy"}, True, id="binding-step-bare-name"),+ pytest.param({"from": "page.page_view.caption", "result": "caption"}, True, id="binding-step-deep-path"),+ pytest.param({"from": "invoice.Total2", "result": "total"}, True, id="binding-step-mixed-case-segment"),+ pytest.param({"from": "invoice.total", "result": "TotalAmount"}, False, id="result-not-plain"),+ pytest.param({"from": "invoice.total", "result": "invoice.total"}, False, id="result-dotted"),+ pytest.param({"from": "pages[0].text", "result": "text"}, False, id="from-with-a-subscript"),+ pytest.param({"from": "invoice._total", "result": "total"}, False, id="from-with-an-underscore-led-segment"),+ pytest.param({"from": "invoice..total", "result": "total"}, False, id="from-with-an-empty-segment"),+ pytest.param({"from": "invoice.total ", "result": "total"}, False, id="from-with-whitespace"),+ pytest.param({"from": "invoice.total"}, False, id="binding-without-result"),+ pytest.param({"pipe": "sub_pipe", "from": "invoice.total", "result": "total"}, False, id="pipe-and-from"),+ pytest.param({"from": "invoice.lines", "result": "lines", "batch_over": "lines"}, False, id="binding-with-batch-over"),+ pytest.param({"from": "invoice.total", "result": "total", "note": "x"}, False, id="binding-with-a-stray-field"),+ pytest.param({"from_path": "invoice.total", "result": "total"}, False, id="binding-spelled-from-path"),+ pytest.param({"from": "invoice.total", "from_path": "invoice.total", "result": "total"}, False, id="binding-with-from-path-beside-from"),+ pytest.param({"result": "total"}, False, id="neither-pipe-nor-from"),+ ],+ )+ def test_sequence_steps_are_pipe_steps_or_binding_steps(self, schema: dict[str, Any], step: dict[str, Any], should_validate: bool) -> None:+ """A PipeSequence step is a closed pipe step or a closed binding step, `from` following the path grammar and `result` the plain name's."""+ validator = _pipe_union_oneof_validator(schema)+ table = {**_minimal_pipe_table("PipeSequence"), "steps": [step]}+ assert validator.is_valid(table) is should_validate, f"step {step!r}"++ def test_parallel_branches_keep_the_pipe_step_shape(self, schema: dict[str, Any]) -> None:+ """A PipeParallel branch is a pipe step only: a binding step there matches no shape, since branches run concurrently."""+ validator = _pipe_union_oneof_validator(schema)+ table = {**_minimal_pipe_table("PipeParallel"), "branches": [{"from": "invoice.total", "result": "total"}]}+ assert validator.is_valid(table) is False+ branch_items = schema["definitions"]["PipeParallelBlueprint"]["properties"]["branches"]["items"]+ assert branch_items == {"$ref": "#/definitions/SubPipeBlueprint"}+def test_minimal_table_coverage_matches_schema_pipe_kinds(self, schema: dict[str, Any]) -> None:"""Guard: the test's per-kind table map covers exactly the *concrete* pipe kinds in the schema.diff --git a/tests/unit/pipelex/libraries/test_crate_normalization.py b/tests/unit/pipelex/libraries/test_crate_normalization.pyindex e68e0fe86..d8528976d 100644--- a/tests/unit/pipelex/libraries/test_crate_normalization.py+++ b/tests/unit/pipelex/libraries/test_crate_normalization.py@@ -241,7 +241,7 @@ class TestCrateNormalization:result = normalize_crate(_authored_crate(), mthds_version=MTHDS_TEST_VERSION)pipeline = result.pipes["scoring.pipeline"]assert isinstance(pipeline, PipeSequenceBlueprint)- assert pipeline.steps[0].pipe == "scoring.compute_score"+ assert pipeline.pipe_steps[0].pipe == "scoring.compute_score"@pytest.mark.parametrize("crate_factory", [_authored_crate, _closed_all_kinds_crate])def test_normalized_crate_is_closed_over_its_pipe_refs(self, crate_factory: Callable[[], LibraryCrate]):@@ -276,7 +276,7 @@ class TestCrateNormalization:result = normalize_crate(_cross_domain_crate(), mthds_version=MTHDS_TEST_VERSION)sequence = result.pipes["orchestrator.run_all"]assert isinstance(sequence, PipeSequenceBlueprint)- assert sequence.steps[0].pipe == "orchestrator.present_as_markdown"+ assert sequence.pipe_steps[0].pipe == "orchestrator.present_as_markdown"parallel = result.pipes["orchestrator.fan_out"]assert isinstance(parallel, PipeParallelBlueprint)@@ -301,7 +301,7 @@ class TestCrateNormalization:result = normalize_crate(_authored_crate(), mthds_version=MTHDS_TEST_VERSION)pipeline = result.pipes["scoring.pipeline"]assert isinstance(pipeline, PipeSequenceBlueprint)- assert pipeline.steps[0].pipe == "scoring.compute_score"+ assert pipeline.pipe_steps[0].pipe == "scoring.compute_score"assert "scoring.compute_score" in result.pipesdef test_special_outcomes_are_left_alone(self):@@ -340,7 +340,7 @@ class TestCrateNormalization:# steps is a row that only half-holds.sequence = result.pipes["orchestrator.run_all"]assert isinstance(sequence, PipeSequenceBlueprint)- assert sequence.steps[0].pipe == "orchestrator.present_as_markdown"+ assert sequence.pipe_steps[0].pipe == "orchestrator.present_as_markdown"parallel = result.pipes["orchestrator.fan_out"]assert isinstance(parallel, PipeParallelBlueprint)@@ -378,7 +378,7 @@ class TestCrateNormalization:result = normalize_crate(crate, mthds_version=MTHDS_TEST_VERSION)sequence = result.pipes["orchestrator.run_all"]assert isinstance(sequence, PipeSequenceBlueprint)- assert sequence.steps[0].pipe == "orchestrator.nowhere_to_be_found"+ assert sequence.pipe_steps[0].pipe == "orchestrator.nowhere_to_be_found"def test_refinement_with_structured_base_is_flattened(self):"""Refining a concept with a structure adopts that (qualified) structure and drops `refines`."""diff --git a/tests/unit/pipelex/libraries/test_crate_qualification.py b/tests/unit/pipelex/libraries/test_crate_qualification.pyindex ff434a639..17d8d608f 100644--- a/tests/unit/pipelex/libraries/test_crate_qualification.py+++ b/tests/unit/pipelex/libraries/test_crate_qualification.py@@ -63,7 +63,7 @@ class TestCrateQualification:def test_sequence_step_refs(self):sequence = qualify_crate(CrateQualificationTestData.crate()).pipes["alpha.seq"]assert isinstance(sequence, PipeSequenceBlueprint)- assert [step.pipe for step in sequence.steps] == ["alpha.leaf"]+ assert [step.pipe for step in sequence.pipe_steps] == ["alpha.leaf"]def test_parallel_branch_refs(self):parallel = qualify_crate(CrateQualificationTestData.crate()).pipes["alpha.par"]@@ -105,7 +105,7 @@ class TestCrateQualification:sequence = qualify_crate(crate).pipes["alpha.calls_sibling"]assert isinstance(sequence, PipeSequenceBlueprint)- assert sequence.steps[0].pipe == "alpha.only_here"+ assert sequence.pipe_steps[0].pipe == "alpha.only_here"def test_a_second_domain_declaring_the_code_is_not_an_ambiguity(self):"""Ambiguity was a consequence of searching. Nothing is searched for, so nothing is ambiguous —@@ -117,7 +117,7 @@ class TestCrateQualification:sequence = qualify_crate(crate).pipes["alpha.seq"]assert isinstance(sequence, PipeSequenceBlueprint)- assert sequence.steps[0].pipe == "alpha.leaf"+ assert sequence.pipe_steps[0].pipe == "alpha.leaf"def test_cross_package_refs_pass_through_untouched(self):"""`alias->…` refs are the packaging project's to canonicalize; this pass must not touch them."""@@ -130,7 +130,7 @@ class TestCrateQualification:sequence = qualify_crate(crate).pipes["alpha.calls_dependency"]assert isinstance(sequence, PipeSequenceBlueprint)- assert sequence.steps[0].pipe == "dep->helper"+ assert sequence.pipe_steps[0].pipe == "dep->helper"def test_a_step_naming_a_pipe_called_continue_is_still_qualified(self):"""`continue` is a legal snake_case pipe code, so it is only special in a condition outcome.@@ -148,7 +148,7 @@ class TestCrateQualification:sequence = qualify_crate(crate).pipes["alpha.calls_continue"]assert isinstance(sequence, PipeSequenceBlueprint)- assert sequence.steps[0].pipe == "alpha.continue"+ assert sequence.pipe_steps[0].pipe == "alpha.continue"def test_special_outcomes_pass_through_untouched(self):"""`fail` / `continue` are outcomes, not pipe refs — in a condition, and only there."""@@ -176,7 +176,7 @@ class TestCrateQualification:sequence = crate.pipes["alpha.seq"]assert isinstance(sequence, PipeSequenceBlueprint)- assert sequence.steps[0].pipe == "leaf"+ assert sequence.pipe_steps[0].pipe == "leaf"assert crate.pipes["alpha.leaf"].output == "Report"detailed = crate.concepts["alpha.Detailed"]assert isinstance(detailed, ConceptBlueprint)@@ -194,7 +194,7 @@ class TestCrateQualification:# qualified one rather than whatever went in.second_pass_sequence = twice.pipes["alpha.seq"]assert isinstance(second_pass_sequence, PipeSequenceBlueprint)- assert second_pass_sequence.steps[0].pipe == "alpha.leaf"+ assert second_pass_sequence.pipe_steps[0].pipe == "alpha.leaf"# --- rejections, raised by the pass itself rather than through the normalizer ---diff --git a/tests/unit/pipelex/libraries/test_crate_qualification_under_package_alias.py b/tests/unit/pipelex/libraries/test_crate_qualification_under_package_alias.pyindex ee660d094..44fc1c46b 100644--- a/tests/unit/pipelex/libraries/test_crate_qualification_under_package_alias.py+++ b/tests/unit/pipelex/libraries/test_crate_qualification_under_package_alias.py@@ -42,7 +42,7 @@ class TestCrateQualificationUnderAPackageAlias:sequence = pipes["alpha.seq"]assert isinstance(sequence, PipeSequenceBlueprint)- assert sequence.steps[0].pipe == aliased_leaf+ assert sequence.pipe_steps[0].pipe == aliased_leafparallel = pipes["alpha.par"]assert isinstance(parallel, PipeParallelBlueprint)assert parallel.branches[0].pipe == aliased_leaf@@ -67,7 +67,7 @@ class TestCrateQualificationUnderAPackageAlias:assert condition.outcomes == {"stop": SpecialOutcome.FAIL, "hit": f"{self.ALIAS}->alpha.leaf"}other = pipes["alpha.calls_other_package"]assert isinstance(other, PipeSequenceBlueprint)- assert other.steps[0].pipe == "other->beta.helper"+ assert other.pipe_steps[0].pipe == "other->beta.helper"def test_is_idempotent(self):once = qualify_crate(CrateQualificationTestData.crate(), package_alias=self.ALIAS)@@ -77,4 +77,4 @@ class TestCrateQualificationUnderAPackageAlias:assert twice.pipes == once.pipessequence = twice.pipes["alpha.seq"]assert isinstance(sequence, PipeSequenceBlueprint)- assert sequence.steps[0].pipe == f"{self.ALIAS}->alpha.leaf"+ assert sequence.pipe_steps[0].pipe == f"{self.ALIAS}->alpha.leaf"diff --git a/tests/unit/pipelex/mthds_parsing/test_bundle_elaborator.py b/tests/unit/pipelex/mthds_parsing/test_bundle_elaborator.pyindex 3867a812b..ccd1bd21b 100644--- a/tests/unit/pipelex/mthds_parsing/test_bundle_elaborator.py+++ b/tests/unit/pipelex/mthds_parsing/test_bundle_elaborator.py@@ -67,9 +67,9 @@ class TestBundleElaborator:wrapping = elaborated.pipe["make_foo"]assert isinstance(wrapping, PipeSequenceBlueprint)assert len(wrapping.steps) == 2- assert wrapping.steps[0].pipe == "make_foo__draft_text"+ assert wrapping.pipe_steps[0].pipe == "make_foo__draft_text"assert wrapping.steps[0].result == "draft_text"- assert wrapping.steps[1].pipe == "make_foo__structure"+ assert wrapping.pipe_steps[1].pipe == "make_foo__structure"assert wrapping.output == "Foo"draft = elaborated.pipe["make_foo__draft_text"]diff --git a/tests/unit/pipelex/mthds_parsing/test_parser_preliminary_text.py b/tests/unit/pipelex/mthds_parsing/test_parser_preliminary_text.pyindex 5a01f1476..6d97dfe96 100644--- a/tests/unit/pipelex/mthds_parsing/test_parser_preliminary_text.py+++ b/tests/unit/pipelex/mthds_parsing/test_parser_preliminary_text.py@@ -29,9 +29,9 @@ structuring_method = "preliminary_text"wrapping = bundle.pipe["make_foo"]assert isinstance(wrapping, PipeSequenceBlueprint)assert len(wrapping.steps) == 2- assert wrapping.steps[0].pipe == "make_foo__draft_text"+ assert wrapping.pipe_steps[0].pipe == "make_foo__draft_text"assert wrapping.steps[0].result == "draft_text"- assert wrapping.steps[1].pipe == "make_foo__structure"+ assert wrapping.pipe_steps[1].pipe == "make_foo__structure"draft = bundle.pipe["make_foo__draft_text"]assert isinstance(draft, PipeLLMBlueprint)diff --git a/tests/unit/pipelex/pipe_controllers/binding/test_binding_derivation.py b/tests/unit/pipelex/pipe_controllers/binding/test_binding_derivation.pynew file mode 100644index 000000000..73807bf5e--- /dev/null+++ b/tests/unit/pipelex/pipe_controllers/binding/test_binding_derivation.py@@ -0,0 +1,99 @@+import pytest++from pipelex.core.pipes.variable_multiplicity import VariableMultiplicity+from pipelex.pipe_controllers.binding.binding_derivation import BindingRoot, BindingValueKind, derive_binding+from tests.unit.pipelex.pipe_controllers.binding.test_data import INVOICE, RESOLVER, make_root+++class TestBindingDerivationTable:+ @pytest.mark.parametrize(+ ("path", "root", "concept_ref", "multiplicity", "leaf_kind"),+ [+ # `type = "concept"`: the referenced concept, walked by the next segment+ ("invoice.supplier", INVOICE, "billing.Supplier", None, BindingValueKind.CONCEPT),+ ("invoice.supplier.address.city", INVOICE, "native.Text", None, BindingValueKind.TEXT),+ # `type = "list"` of concepts: the item concept, crossing a list+ ("invoice.lines", INVOICE, "billing.InvoiceLine", True, BindingValueKind.CONCEPT),+ ("invoice.lines.amount", INVOICE, "native.Number", True, BindingValueKind.NUMBER),+ # `type = "text"` and `choices`: a text leaf+ ("invoice.supplier_name", INVOICE, "native.Text", None, BindingValueKind.TEXT),+ ("invoice.priority", INVOICE, "native.Text", None, BindingValueKind.TEXT),+ # `type = "number"` and `"integer"`: a number leaf+ ("invoice.total", INVOICE, "native.Number", None, BindingValueKind.NUMBER),+ ("invoice.item_count", INVOICE, "native.Number", None, BindingValueKind.NUMBER),+ # `type = "boolean"`: a yes/no leaf+ ("invoice.paid", INVOICE, "native.YesNo", None, BindingValueKind.YES_NO),+ # `type = "date"` and `"datetime"`: a date leaf+ ("invoice.issued_on", INVOICE, "native.Date", None, BindingValueKind.DATE),+ ("invoice.issued_at", INVOICE, "native.Date", None, BindingValueKind.DATETIME),+ # `type = "time"`: a time leaf+ ("invoice.cutoff", INVOICE, "native.Time", None, BindingValueKind.TIME),+ # `type = "list"` of a scalar: that scalar's native concept, crossing a list+ ("invoice.tags", INVOICE, "native.Text", True, BindingValueKind.TEXT),+ # `type = "dict"`: a JSON leaf+ ("invoice.metadata", INVOICE, "native.JSON", None, BindingValueKind.JSON),+ # A native concept walks through its pinned definition+ ("page.page_view", make_root("native.Page"), "native.Image", None, BindingValueKind.CONCEPT),+ ("page.page_view.caption", make_root("native.Page"), "native.Text", None, BindingValueKind.TEXT),+ ("invoice.scan.url", INVOICE, "native.Text", None, BindingValueKind.TEXT),+ # A concept field may end on a native holding its value in a single field+ ("invoice.amount_due", INVOICE, "native.Number", None, BindingValueKind.CONCEPT),+ # A path may end on a concept declared with a description alone+ ("invoice.remark", INVOICE, "billing.Remark", None, BindingValueKind.CONCEPT),+ # A refined concept walks the structure it inherits+ ("receipt.supplier.name", make_root("billing.Receipt"), "native.Text", None, BindingValueKind.TEXT),+ # A list root maps the path over its items, and a fixed count becomes a variable-length list+ ("pages.page_view", make_root("native.Page", multiplicity=True), "native.Image", True, BindingValueKind.CONCEPT),+ ("pages.page_view", make_root("native.Page", multiplicity=3), "native.Image", True, BindingValueKind.CONCEPT),+ # Two lists crossed flatten into one+ ("shipments.parcels", make_root("billing.Shipment", multiplicity=True), "billing.Parcel", True, BindingValueKind.CONCEPT),+ ("invoice.pages.page_view", INVOICE, "native.Image", True, BindingValueKind.CONCEPT),+ # A bare name keeps its root's concept and multiplicity unchanged+ ("departure_board", make_root("rail.DepartureBoard"), "rail.DepartureBoard", None, BindingValueKind.CONCEPT),+ ("pages", make_root("native.Page", multiplicity=3), "native.Page", 3, BindingValueKind.CONCEPT),+ ],+ )+ def test_derives_the_concept_and_multiplicity(+ self,+ path: str,+ root: BindingRoot,+ concept_ref: str,+ multiplicity: VariableMultiplicity | None,+ leaf_kind: BindingValueKind,+ ):+ derivation = derive_binding(path=path, root=root, resolver=RESOLVER)++ assert derivation.concept_ref == concept_ref+ assert derivation.multiplicity == multiplicity+ assert derivation.leaf_kind == leaf_kind++ @pytest.mark.parametrize(+ ("path", "root", "may_find_nothing", "first_optional_path"),+ [+ # A required field is never absent, and neither is a field with a default value+ ("invoice.total", INVOICE, False, None),+ ("invoice.currency", INVOICE, False, None),+ # A field that is neither required nor defaulted may hold nothing, at the leaf or before it+ ("invoice.note", INVOICE, True, "invoice.note"),+ ("invoice.scan.url", INVOICE, True, "invoice.scan"),+ ("page.page_view", make_root("native.Page"), True, "page.page_view"),+ # A list result is never absent, even when a field before the first list may hold nothing+ ("invoice.lines.amount", INVOICE, False, None),+ ("invoice.pages.page_view", INVOICE, False, None),+ ("pages.page_view", make_root("native.Page", multiplicity=True), False, None),+ # A bare name adds no absence of its own+ ("invoice", INVOICE, False, None),+ ],+ )+ def test_derives_the_static_absence(self, path: str, root: BindingRoot, may_find_nothing: bool, first_optional_path: str | None):+ derivation = derive_binding(path=path, root=root, resolver=RESOLVER)++ assert derivation.may_find_nothing is may_find_nothing+ assert derivation.first_optional_path == first_optional_path++ def test_segments_record_the_lists_they_cross(self):+ derivation = derive_binding(path="invoice.lines.amount", root=INVOICE, resolver=RESOLVER)++ assert [(segment.name, segment.crosses_list) for segment in derivation.segments] == [("lines", True), ("amount", False)]+ assert derivation.root_name == "invoice"+ assert not derivation.is_bare_namediff --git a/tests/unit/pipelex/pipe_controllers/binding/test_binding_derivation_refusals.py b/tests/unit/pipelex/pipe_controllers/binding/test_binding_derivation_refusals.pynew file mode 100644index 000000000..5354aea19--- /dev/null+++ b/tests/unit/pipelex/pipe_controllers/binding/test_binding_derivation_refusals.py@@ -0,0 +1,67 @@+import pytest++from pipelex.pipe_controllers.binding.binding_derivation import BindingRoot, derive_binding+from pipelex.pipe_controllers.binding.exceptions import BindingPathUnresolvedError+from tests.unit.pipelex.pipe_controllers.binding.test_data import INVOICE, RESOLVER, make_root+++class TestBindingDerivationRefusals:+ @pytest.mark.parametrize(+ ("path", "root", "failed_segment", "available_fields", "message_fragment"),+ [+ # A segment naming no field lists the fields that were available+ ("invoice.totl", INVOICE, "totl", None, "has no field 'totl'"),+ ("invoice.supplier.adress", INVOICE, "adress", ["name", "address"], "Its fields are: 'name', 'address'."),+ # A segment after a plain field, whatever native it derives+ ("invoice.total.amount", INVOICE, "amount", [], "a number field, a leaf"),+ ("invoice.paid.probability", INVOICE, "probability", [], "a boolean field, a leaf"),+ ("invoice.issued_on.date", INVOICE, "date", [], "a date field, a leaf"),+ # A segment after a native holding its value in a single field, however the walk reached it+ ("invoice.amount_due.number", INVOICE, "number", [], "holds its value in a single field"),+ ("note.text", make_root("native.Text"), "text", [], "holds its value in a single field"),+ ("count.number", make_root("native.Number"), "number", [], "holds its value in a single field"),+ ("payload.json_obj", make_root("native.JSON"), "json_obj", [], "holds its value in a single field"),+ ("summary.text", make_root("native.Markdown"), "text", [], "holds its value in a single field"),+ ("departure.time", make_root("native.Time"), "time", [], "holds its value in a single field"),+ ("caption.text", make_root("billing.Caption"), "text", [], "refines 'native.Text'"),+ # A segment after a dict+ ("invoice.metadata.code", INVOICE, "code", [], "a dict field, a leaf"),+ # A segment after a list with no item_type, and a path ending on one+ ("invoice.attachments.name", INVOICE, "name", [], "the list declares no item_type"),+ ("invoice.attachments", INVOICE, "attachments", [], "the path ends on the segment 'attachments'"),+ # A concept with no structure to walk+ ("anything.name", make_root("native.Anything"), "name", [], "is structureless by definition"),+ ("dynamic.name", make_root("native.Dynamic"), "name", [], "is structureless by definition"),+ ("composite.name", make_root("native.Composite"), "name", [], "is structureless by definition"),+ ("remark.text", make_root("billing.Remark"), "text", [], "is declared with neither a structure nor refines"),+ ("memo.subject", make_root("billing.Memo"), "subject", [], "is declared with neither a structure nor refines"),+ ("memo_copy.subject", make_root("billing.MemoCopy"), "subject", [], "refines 'billing.Memo'"),+ ("invoice.remark.text", INVOICE, "text", [], "is declared with neither a structure nor refines"),+ ("stranger.name", make_root("billing.Stranger"), "name", [], "is not declared in this bundle"),+ ],+ )+ def test_refuses_the_path(+ self,+ path: str,+ root: BindingRoot,+ failed_segment: str,+ available_fields: list[str] | None,+ message_fragment: str,+ ):+ with pytest.raises(BindingPathUnresolvedError) as raised:+ derive_binding(path=path, root=root, resolver=RESOLVER)++ assert raised.value.failed_segment == failed_segment+ assert raised.value.path == path+ if available_fields is not None:+ assert raised.value.available_fields == available_fields+ assert message_fragment in str(raised.value)+ assert f"'{failed_segment}'" in str(raised.value)++ def test_an_unknown_field_lists_every_field_available_there(self):+ with pytest.raises(BindingPathUnresolvedError) as raised:+ derive_binding(path="invoice.totl", root=INVOICE, resolver=RESOLVER)++ assert "total" in raised.value.available_fields+ assert "'total'" in str(raised.value)+ assert raised.value.available_fields[0] == "supplier_name"diff --git a/tests/unit/pipelex/pipe_controllers/binding/test_binding_step.py b/tests/unit/pipelex/pipe_controllers/binding/test_binding_step.pynew file mode 100644index 000000000..6a4ee5e4a--- /dev/null+++ b/tests/unit/pipelex/pipe_controllers/binding/test_binding_step.py@@ -0,0 +1,246 @@+import pytest++from pipelex.core.concepts.concept import Concept+from pipelex.core.memory.absence import AbsenceKind, AbsenceRecord+from pipelex.core.memory.working_memory import MAIN_STUFF_NAME, WorkingMemory+from pipelex.core.pipes.inputs.exceptions import PipeRunInputsError+from pipelex.core.stuffs.list_content import ListContent+from pipelex.core.stuffs.number_content import NumberContent+from pipelex.core.stuffs.stuff_content import StuffContent+from pipelex.core.stuffs.stuff_factory import StuffFactory+from pipelex.pipe_controllers.binding.binding_derivation import BindingDerivation, BindingRoot, derive_binding+from pipelex.pipe_controllers.binding.binding_step import BindingStep+from pipelex.system.pipe_run_mode import PipeRunMode+from tests.unit.pipelex.pipe_controllers.binding.test_data import INVOICE, RESOLVER, SupplierRecord, make_invoice_record++_INVOICE_CONCEPT = Concept(domain_code="billing", code="Invoice", description="An invoice", structure_class_name="InvoiceRecord")+_SUPPLIER_CONCEPT = Concept(domain_code="billing", code="Supplier", description="A supplier", structure_class_name="SupplierRecord")+_NUMBER_CONCEPT = Concept(domain_code="native", code="Number", description="A number", structure_class_name="NumberContent")+_LINE_CONCEPT = Concept(domain_code="billing", code="InvoiceLine", description="A line", structure_class_name="InvoiceLineRecord")+_SEQUENCE_CODE = "acknowledge_invoice"+++def _derive(path: str, *, root: BindingRoot = INVOICE) -> BindingDerivation:+ return derive_binding(path=path, root=root, resolver=RESOLVER)+++def _memory_with_invoice(*, note: str | None = None) -> WorkingMemory:+ working_memory = WorkingMemory()+ invoice_stuff = StuffFactory.make_stuff(concept=_INVOICE_CONCEPT, content=make_invoice_record(note=note), name="invoice")+ working_memory.add_new_stuff(name="invoice", stuff=invoice_stuff)+ return working_memory+++def _memory_with_absent_invoice() -> tuple[WorkingMemory, AbsenceRecord]:+ working_memory = WorkingMemory()+ root_absence = AbsenceRecord(variable_name="invoice", kind=AbsenceKind.NOT_PROVIDED, reason="the caller did not provide 'invoice'")+ working_memory.record_absence(root_absence)+ return working_memory, root_absence+++class TestBindingStep:+ def test_c3_the_result_is_a_fresh_stuff_under_the_result_name(self) -> None:+ working_memory = _memory_with_invoice()+ root_stuff = working_memory.get_stuff("invoice")+ step = BindingStep(from_path="invoice.supplier", output_name="supplier")++ outcome = step.bind(+ working_memory=working_memory,+ derivation=_derive("invoice.supplier"),+ result_concept=_SUPPLIER_CONCEPT,+ calling_pipe_code=_SEQUENCE_CODE,+ run_mode=PipeRunMode.LIVE,+ )++ assert outcome.stuff is not None+ assert outcome.absence is None+ bound_stuff = working_memory.get_stuff("supplier")+ assert bound_stuff is outcome.stuff+ assert bound_stuff.stuff_name == "supplier"+ assert bound_stuff.concept == _SUPPLIER_CONCEPT+ assert bound_stuff.stuff_code != root_stuff.stuff_code+ assert working_memory.get_main_stuff() is bound_stuff+ assert isinstance(bound_stuff.content, SupplierRecord)+ assert bound_stuff.content.address.city == "Quimper"++ def test_the_run_can_fix_the_result_stuff_code(self) -> None:+ working_memory = _memory_with_invoice()+ step = BindingStep(from_path="invoice.total", output_name="total_amount")++ outcome = step.bind(+ working_memory=working_memory,+ derivation=_derive("invoice.total"),+ result_concept=_NUMBER_CONCEPT,+ calling_pipe_code=_SEQUENCE_CODE,+ run_mode=PipeRunMode.LIVE,+ stuff_code="final_code",+ )++ assert outcome.stuff is not None+ assert outcome.stuff.stuff_code == "final_code"+ assert outcome.stuff.content == NumberContent(number=1250.5)++ def test_v24_a_bare_name_keeps_the_root_concept(self) -> None:+ working_memory = _memory_with_invoice()+ step = BindingStep(from_path="invoice", output_name="invoice_copy")++ outcome = step.bind(+ working_memory=working_memory,+ derivation=_derive("invoice"),+ result_concept=_INVOICE_CONCEPT,+ calling_pipe_code=_SEQUENCE_CODE,+ run_mode=PipeRunMode.LIVE,+ )++ assert outcome.stuff is not None+ assert outcome.stuff.concept == _INVOICE_CONCEPT+ assert outcome.stuff.content == working_memory.get_stuff("invoice").content+ assert outcome.stuff.content is not working_memory.get_stuff("invoice").content++ def test_c2_rebinding_the_root_leaves_the_bound_value_as_it_was(self) -> None:+ working_memory = _memory_with_invoice()+ step = BindingStep(from_path="invoice.total", output_name="total_amount")+ step.bind(+ working_memory=working_memory,+ derivation=_derive("invoice.total"),+ result_concept=_NUMBER_CONCEPT,+ calling_pipe_code=_SEQUENCE_CODE,+ run_mode=PipeRunMode.LIVE,+ )+ other_invoice = make_invoice_record()+ other_invoice.total = 3.0+ working_memory.set_new_main_stuff(StuffFactory.make_stuff(concept=_INVOICE_CONCEPT, content=other_invoice, name="invoice"), name="invoice")++ assert working_memory.get_stuff("total_amount").content == NumberContent(number=1250.5)++ def test_v18_a_path_holding_nothing_records_a_declared_absence_naming_the_segment(self) -> None:+ working_memory = _memory_with_invoice(note=None)+ step = BindingStep(from_path="invoice.note", output_name="delivery_note")++ outcome = step.bind(+ working_memory=working_memory,+ derivation=_derive("invoice.note"),+ result_concept=_NUMBER_CONCEPT,+ calling_pipe_code=_SEQUENCE_CODE,+ run_mode=PipeRunMode.LIVE,+ )++ expected_record = AbsenceRecord(+ variable_name="delivery_note",+ kind=AbsenceKind.DECLARED_ABSENT,+ reason=(+ 'the binding step { from = "invoice.note", result = "delivery_note" } of pipe \'acknowledge_invoice\' bound nothing, '+ "because 'invoice.note' holds nothing"+ ),+ )+ assert outcome.stuff is None+ assert outcome.absence == expected_record+ assert outcome.is_skipped is False+ assert working_memory.get_optional_stuff("delivery_note") is None+ assert working_memory.get_optional_absence("delivery_note") == expected_record+ assert working_memory.get_optional_absence(MAIN_STUFF_NAME) == expected_record++ def test_v20_an_absent_root_skips_the_binding_with_upstream_provenance(self) -> None:+ working_memory, root_absence = _memory_with_absent_invoice()+ step = BindingStep(from_path="invoice.total", output_name="total_amount")++ outcome = step.bind(+ working_memory=working_memory,+ derivation=_derive("invoice.total"),+ result_concept=_NUMBER_CONCEPT,+ calling_pipe_code=_SEQUENCE_CODE,+ run_mode=PipeRunMode.LIVE,+ )++ assert outcome.stuff is None+ assert outcome.is_skipped is True+ assert outcome.absence == AbsenceRecord(+ variable_name="total_amount",+ kind=AbsenceKind.SKIPPED,+ reason="skipped because input 'invoice' is absent",+ upstream=root_absence,+ )+ recorded = working_memory.get_optional_absence("total_amount")+ assert recorded is not None+ assert recorded.provenance_chain() == [outcome.absence, root_absence]+ assert recorded.origin() == root_absence++ def test_a_list_result_over_an_absent_root_is_an_empty_list_with_a_note(self) -> None:+ """A plural slot is never absent: the skip is kept as a note beside an empty list."""+ working_memory, root_absence = _memory_with_absent_invoice()+ step = BindingStep(from_path="invoice.lines", output_name="lines")++ outcome = step.bind(+ working_memory=working_memory,+ derivation=_derive("invoice.lines"),+ result_concept=_LINE_CONCEPT,+ calling_pipe_code=_SEQUENCE_CODE,+ run_mode=PipeRunMode.LIVE,+ )++ bound_stuff = working_memory.get_stuff("lines")+ assert outcome.stuff is bound_stuff+ assert bound_stuff.content == ListContent[StuffContent](items=[])+ assert outcome.absence is not None+ assert outcome.absence.kind == AbsenceKind.SKIPPED+ assert outcome.absence.upstream == root_absence+ assert working_memory.get_optional_absence("lines") == outcome.absence++ def test_an_empty_list_over_an_absent_root_takes_the_stuff_code_the_run_fixes(self) -> None:+ """A binding ending a sequence takes the run's final stuff code, which the graph tracer links its producer by."""+ working_memory, _ = _memory_with_absent_invoice()+ step = BindingStep(from_path="invoice.lines", output_name="lines")++ outcome = step.bind(+ working_memory=working_memory,+ derivation=_derive("invoice.lines"),+ result_concept=_LINE_CONCEPT,+ calling_pipe_code=_SEQUENCE_CODE,+ run_mode=PipeRunMode.LIVE,+ stuff_code="final_code",+ )++ assert outcome.stuff is not None+ assert outcome.stuff.stuff_code == "final_code"+ assert working_memory.get_stuff("lines").stuff_code == "final_code"++ def test_a_root_neither_present_nor_recorded_absent_is_a_run_error(self) -> None:+ step = BindingStep(from_path="invoice.total", output_name="total_amount")++ with pytest.raises(PipeRunInputsError, match="reads 'invoice', which is not in working memory"):+ step.bind(+ working_memory=WorkingMemory(),+ derivation=_derive("invoice.total"),+ result_concept=_NUMBER_CONCEPT,+ calling_pipe_code=_SEQUENCE_CODE,+ run_mode=PipeRunMode.LIVE,+ )++ def test_an_absent_root_the_flow_cannot_type_skips_the_binding_as_a_single_result(self) -> None:+ """Without the root's concept, whether the path crosses a list is unknown, so the skip is a single absence."""+ working_memory, root_absence = _memory_with_absent_invoice()+ step = BindingStep(from_path="invoice.lines", output_name="lines")++ outcome = step.skip_untyped_root(working_memory=working_memory, calling_pipe_code=_SEQUENCE_CODE, run_mode=PipeRunMode.LIVE)++ assert outcome.stuff is None+ assert outcome.absence == AbsenceRecord(+ variable_name="lines",+ kind=AbsenceKind.SKIPPED,+ reason="skipped because input 'invoice' is absent",+ upstream=root_absence,+ )+ assert working_memory.get_optional_absence("lines") == outcome.absence+ assert working_memory.get_optional_stuff("lines") is None++ def test_a_root_the_flow_cannot_type_and_nothing_records_is_a_run_error(self) -> None:+ step = BindingStep(from_path="invoice.lines", output_name="lines")++ with pytest.raises(PipeRunInputsError, match="reads 'invoice', which is not in working memory"):+ step.skip_untyped_root(working_memory=WorkingMemory(), calling_pipe_code=_SEQUENCE_CODE, run_mode=PipeRunMode.LIVE)++ def test_the_step_is_written_back_as_mthds_writes_it(self) -> None:+ step = BindingStep(from_path="invoice.supplier.address.city", output_name="city")++ assert step.root_name == "invoice"+ assert step.as_written == '{ from = "invoice.supplier.address.city", result = "city" }'diff --git a/tests/unit/pipelex/pipe_controllers/binding/test_binding_value.py b/tests/unit/pipelex/pipe_controllers/binding/test_binding_value.pynew file mode 100644index 000000000..a0bab035a--- /dev/null+++ b/tests/unit/pipelex/pipe_controllers/binding/test_binding_value.py@@ -0,0 +1,292 @@+import datetime+from typing import cast++import pytest++from pipelex.core.stuffs.date_content import DateContent+from pipelex.core.stuffs.document_content import DocumentContent+from pipelex.core.stuffs.image_content import ImageContent+from pipelex.core.stuffs.json_content import JSONContent+from pipelex.core.stuffs.list_content import ListContent+from pipelex.core.stuffs.number_content import NumberContent+from pipelex.core.stuffs.stuff_content import StuffContent+from pipelex.core.stuffs.text_content import TextContent+from pipelex.core.stuffs.time_content import TimeContent+from pipelex.core.stuffs.yes_no_content import YesNoContent+from pipelex.pipe_controllers.binding.binding_derivation import BindingDerivation, BindingRoot, derive_binding+from pipelex.pipe_controllers.binding.binding_value import FoundNothing, bind_content+from pipelex.pipe_controllers.binding.exceptions import BindingStepRunError+from tests.unit.pipelex.pipe_controllers.binding.test_data import (+ INVOICE,+ INVOICE_ISSUED_AT,+ RESOLVER,+ CrateRecord,+ InvoiceLineRecord,+ ParcelRecord,+ ShipmentRecord,+ SupplierRecord,+ make_invoice_record,+ make_page,+ make_root,+ make_view,+)+++def _derive(path: str, *, root: BindingRoot = INVOICE) -> BindingDerivation:+ return derive_binding(path=path, root=root, resolver=RESOLVER)+++def _list_items(content: StuffContent | FoundNothing) -> list[StuffContent]:+ assert isinstance(content, ListContent)+ return list(cast("ListContent[StuffContent]", content).items)+++class TestBindContent:+ @pytest.mark.parametrize(+ ("path", "expected_content"),+ [+ pytest.param("invoice.supplier_name", TextContent(text="Atelier Morvan"), id="V1-text"),+ pytest.param("invoice.total", NumberContent(number=1250.5), id="V2-number"),+ pytest.param("invoice.item_count", NumberContent(number=3), id="V3-integer"),+ pytest.param("invoice.paid", YesNoContent(yes_no=True), id="V4-boolean"),+ pytest.param("invoice.issued_on", DateContent(date=datetime.date(2026, 3, 14)), id="V5-date"),+ pytest.param("invoice.issued_at", DateContent(date=INVOICE_ISSUED_AT.date(), time=INVOICE_ISSUED_AT.timetz()), id="V5-datetime"),+ pytest.param("invoice.cutoff", TimeContent(time=datetime.time(17, 0)), id="V5-time"),+ pytest.param("invoice.priority", TextContent(text="high"), id="V6-choices"),+ pytest.param("invoice.metadata", JSONContent(json_obj={"order_ref": "PO-118"}), id="V7-dict"),+ pytest.param("invoice.supplier.address.city", TextContent(text="Quimper"), id="V9-nested"),+ ],+ )+ def test_a_plain_value_is_stored_as_the_native_its_field_derives(self, path: str, expected_content: StuffContent) -> None:+ """The structure classes hold raw Python values; the binding stores each as the native concept the derivation names."""+ bound = bind_content(root_content=make_invoice_record(), derivation=_derive(path))++ assert bound == expected_content+ assert type(bound) is type(expected_content)++ def test_v8_a_concept_field_is_copied_whole(self) -> None:+ invoice = make_invoice_record()++ bound = bind_content(root_content=invoice, derivation=_derive("invoice.supplier"))++ assert isinstance(bound, SupplierRecord)+ assert bound == invoice.supplier+ assert bound is not invoice.supplier+ assert bound.address is not invoice.supplier.address++ def test_v11_a_list_of_concepts_binds_a_list_of_copies(self) -> None:+ invoice = make_invoice_record()++ items = _list_items(bind_content(root_content=invoice, derivation=_derive("invoice.lines")))++ assert items == invoice.lines+ assert all(item is not original for item, original in zip(items, invoice.lines, strict=True))++ def test_a_list_of_plain_values_binds_a_list_of_natives(self) -> None:+ items = _list_items(bind_content(root_content=make_invoice_record(), derivation=_derive("invoice.tags")))++ assert items == [TextContent(text="urgent"), TextContent(text="export")]++ def test_v13_v15_a_list_crossed_mid_path_maps_and_drops_items_holding_nothing(self) -> None:+ """The second line has no amount: it is dropped from the list, which is never an absence."""+ items = _list_items(bind_content(root_content=make_invoice_record(), derivation=_derive("invoice.lines.amount")))++ assert items == [NumberContent(number=1000.0), NumberContent(number=250.5)]++ def test_v16_an_empty_list_binds_an_empty_list(self) -> None:+ invoice = make_invoice_record()+ invoice.lines = []++ bound = bind_content(root_content=invoice, derivation=_derive("invoice.lines.amount"))++ assert _list_items(bound) == []++ def test_v12_v15_a_list_root_maps_its_items_and_drops_pages_without_a_view(self) -> None:+ pages = ListContent[StuffContent](+ items=[+ make_page(text="Cover", view=make_view(url="https://example.com/cover.png")),+ make_page(text="Blank", view=None),+ make_page(text="Index", view=make_view(url="https://example.com/index.png")),+ ]+ )++ items = _list_items(bind_content(root_content=pages, derivation=_derive("pages.page_view", root=make_root("native.Page", multiplicity=True))))++ assert items == [make_view(url="https://example.com/cover.png"), make_view(url="https://example.com/index.png")]++ def test_v14_two_lists_crossed_flatten_into_one(self) -> None:+ shipments = ListContent[StuffContent](+ items=[+ ShipmentRecord(parcels=[ParcelRecord(weight=1.5), ParcelRecord(weight=2.0)]),+ ShipmentRecord(parcels=[]),+ ShipmentRecord(parcels=[ParcelRecord(weight=4.25)]),+ ]+ )++ items = _list_items(+ bind_content(root_content=shipments, derivation=_derive("shipments.parcels", root=make_root("billing.Shipment", multiplicity=True)))+ )++ assert items == [ParcelRecord(weight=1.5), ParcelRecord(weight=2.0), ParcelRecord(weight=4.25)]++ def test_v17_an_optional_field_holding_a_value_binds_it(self) -> None:+ bound = bind_content(root_content=make_invoice_record(note="Deliver to the back door"), derivation=_derive("invoice.note"))++ assert bound == TextContent(text="Deliver to the back door")++ @pytest.mark.parametrize(+ ("path", "empty_path"),+ [+ pytest.param("invoice.note", "invoice.note", id="V18-optional-field-absent"),+ pytest.param("invoice.scan.url", "invoice.scan", id="V19-intermediate-none"),+ ],+ )+ def test_a_single_path_reaching_nothing_names_the_segment_that_held_nothing(self, path: str, empty_path: str) -> None:+ bound = bind_content(root_content=make_invoice_record(), derivation=_derive(path))++ assert bound == FoundNothing(empty_path=empty_path)++ def test_a_present_intermediate_value_is_walked_through(self) -> None:+ invoice = make_invoice_record(scan=DocumentContent(url="https://example.com/scan.pdf"))++ bound = bind_content(root_content=invoice, derivation=_derive("invoice.scan.url"))++ assert bound == TextContent(text="https://example.com/scan.pdf")++ def test_v24_a_bare_name_binds_a_deep_copy_of_the_whole_value(self) -> None:+ invoice = make_invoice_record()++ bound = bind_content(root_content=invoice, derivation=_derive("invoice"))++ assert bound == invoice+ assert bound is not invoice+ assert isinstance(bound, type(invoice))+ assert bound.supplier is not invoice.supplier++ def test_c1_the_bound_value_is_isolated_from_its_root_both_ways(self) -> None:+ invoice = make_invoice_record()+ bound = bind_content(root_content=invoice, derivation=_derive("invoice.supplier"))+ assert isinstance(bound, SupplierRecord)++ invoice.supplier.address.city = "Brest"+ assert bound.address.city == "Quimper"++ bound.name = "Someone else"+ assert invoice.supplier.name == "Atelier Morvan"++ def test_c1_a_bound_list_is_isolated_from_its_root(self) -> None:+ invoice = make_invoice_record()+ items = _list_items(bind_content(root_content=invoice, derivation=_derive("invoice.lines")))++ invoice.lines[0].amount = 1.0+ invoice.lines.append(InvoiceLineRecord(amount=9.0))++ assert items == [InvoiceLineRecord(amount=1000.0), InvoiceLineRecord(amount=None), InvoiceLineRecord(amount=250.5)]++ def test_c1_a_bound_dict_is_isolated_from_its_root(self) -> None:+ invoice = make_invoice_record()+ bound = bind_content(root_content=invoice, derivation=_derive("invoice.metadata"))+ assert isinstance(bound, JSONContent)++ invoice.metadata["order_ref"] = "PO-999"++ assert bound.json_obj == {"order_ref": "PO-118"}++ def test_c4_a_bound_image_keeps_every_field(self) -> None:+ view = make_view(url="https://example.com/cover.png")+ page = make_page(text="Cover", view=view)++ bound = bind_content(root_content=page, derivation=_derive("page.page_view", root=make_root("native.Page")))++ assert isinstance(bound, ImageContent)+ assert bound.model_dump() == view.model_dump()+ assert bound.caption == "The view at https://example.com/cover.png"+ assert bound.source_prompt == "A catalog page, photographed flat"+ assert bound.source_negative_prompt == "No glare"+ assert bound is not view++ @pytest.mark.parametrize(+ ("contents", "expected_content"),+ [+ pytest.param("a spare part", TextContent(text="a spare part"), id="a-string-is-text"),+ pytest.param(42, NumberContent(number=42), id="a-number-is-a-number"),+ pytest.param(True, YesNoContent(yes_no=True), id="a-boolean-is-yes-no"),+ pytest.param(datetime.date(2026, 3, 14), DateContent(date=datetime.date(2026, 3, 14)), id="a-date-is-a-date"),+ pytest.param(datetime.time(17, 0), TimeContent(time=datetime.time(17, 0)), id="a-time-is-a-time"),+ pytest.param({"order_ref": "PO-118"}, JSONContent(json_obj={"order_ref": "PO-118"}), id="an-object-is-json"),+ pytest.param(TextContent(text="already content"), TextContent(text="already content"), id="a-content-is-copied-whole"),+ ],+ )+ def test_a_value_an_anything_field_holds_is_stored_as_its_natural_content(self, contents: object, expected_content: StuffContent) -> None:+ """A field holding `native.Anything` holds a raw value, stored as the native its type maps to, as an `Anything` input is."""+ derivation = _derive("crate.contents", root=make_root("billing.Crate"))++ bound = bind_content(root_content=CrateRecord(contents=contents, extras=[]), derivation=derivation)++ assert derivation.concept_ref == "native.Anything"+ assert bound == expected_content+ assert bound is not contents++ def test_a_list_of_anything_binds_each_item_as_its_natural_content(self) -> None:+ derivation = _derive("crate.extras", root=make_root("billing.Crate"))++ bound = bind_content(root_content=CrateRecord(contents=None, extras=["a strap", 3]), derivation=derivation)++ assert _list_items(bound) == [TextContent(text="a strap"), NumberContent(number=3)]++ @pytest.mark.parametrize(+ "contents",+ [+ pytest.param(["a strap"], id="a-list"),+ pytest.param(float("nan"), id="a-number-that-is-not-finite"),+ pytest.param(object(), id="an-arbitrary-object"),+ ],+ )+ def test_a_value_an_anything_field_holds_with_no_natural_content_is_a_run_error(self, contents: object) -> None:+ derivation = _derive("crate.contents", root=make_root("billing.Crate"))++ with pytest.raises(BindingStepRunError, match="cannot be stored as a field holding any value's value"):+ bind_content(root_content=CrateRecord(contents=contents, extras=[]), derivation=derivation)++ def test_a_value_contradicting_its_declared_structure_is_a_run_error(self) -> None:+ """The structure says `supplier` holds a Supplier; content lacking the field is a broken contract, not an absence."""+ with pytest.raises(BindingStepRunError, match="has no such field"):+ bind_content(root_content=ParcelRecord(weight=1.0), derivation=_derive("invoice.supplier.name"))++ def test_a_plain_value_of_the_wrong_kind_is_a_run_error(self) -> None:+ invoice = make_invoice_record()+ invoice.supplier_name = cast("str", 12)++ with pytest.raises(BindingStepRunError, match="cannot be stored as"):+ bind_content(root_content=invoice, derivation=_derive("invoice.supplier_name"))++ @pytest.mark.parametrize(+ "path",+ [+ pytest.param("invoice.total", id="a-path"),+ pytest.param("invoice", id="a-bare-name"),+ ],+ )+ def test_a_single_binding_over_a_list_root_never_selects_an_item(self, path: str) -> None:+ """Regression: a derivation saying single met a list of invoices, and the binding kept the first total, losing the second."""+ second_invoice = make_invoice_record()+ second_invoice.total = 999.0+ invoices = ListContent[StuffContent](items=[make_invoice_record(), second_invoice])++ with pytest.raises(BindingStepRunError) as exc_info:+ bind_content(root_content=invoices, derivation=_derive(path))++ message = str(exc_info.value)+ assert f"Binding '{path}'" in message+ assert "a single value" in message+ assert "'invoice' holds a list of 2 items" in message++ def test_a_list_bare_name_over_a_single_root_is_a_run_error(self) -> None:+ """A bare name binds its root's value as it is, so a root derived as a list must hold one."""+ with pytest.raises(BindingStepRunError) as exc_info:+ bind_content(root_content=make_invoice_record(), derivation=_derive("invoices", root=make_root("billing.Invoice", multiplicity=True)))++ message = str(exc_info.value)+ assert "Binding 'invoices'" in message+ assert "a list" in message+ assert "'invoices' holds a single value" in messagediff --git a/tests/unit/pipelex/pipe_controllers/binding/test_data.py b/tests/unit/pipelex/pipe_controllers/binding/test_data.pynew file mode 100644index 000000000..80bcfc3ec--- /dev/null+++ b/tests/unit/pipelex/pipe_controllers/binding/test_data.py@@ -0,0 +1,172 @@+import datetime+from typing import Any++from pipelex.core.concepts.concept_blueprint import ConceptBlueprint+from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprint, ConceptStructureBlueprintFieldType+from pipelex.core.pipes.variable_multiplicity import VariableMultiplicity+from pipelex.core.stuffs.document_content import DocumentContent+from pipelex.core.stuffs.image_content import ImageContent+from pipelex.core.stuffs.page_content import PageContent+from pipelex.core.stuffs.structured_content import StructuredContent+from pipelex.core.stuffs.text_and_images_content import TextAndImagesContent+from pipelex.core.stuffs.text_content import TextContent+from pipelex.pipe_controllers.binding.binding_concept_resolvers import BlueprintConceptWalkResolver+from pipelex.pipe_controllers.binding.binding_derivation import BindingRoot++FieldType = ConceptStructureBlueprintFieldType+++def make_field(*, field_type: FieldType | None, required: bool = False, **kwargs: Any) -> ConceptStructureBlueprint:+ return ConceptStructureBlueprint(description="A field", type=field_type, required=required, **kwargs)+++CONCEPTS: dict[str, ConceptBlueprint | str] = {+ "billing.Invoice": ConceptBlueprint(+ description="An invoice received from a supplier",+ structure={+ "supplier_name": make_field(field_type=FieldType.TEXT, required=True),+ "total": make_field(field_type=FieldType.NUMBER, required=True),+ "item_count": make_field(field_type=FieldType.INTEGER, required=True),+ "paid": make_field(field_type=FieldType.BOOLEAN, required=True),+ "issued_on": make_field(field_type=FieldType.DATE, required=True),+ "issued_at": make_field(field_type=FieldType.DATETIME, required=True),+ "cutoff": make_field(field_type=FieldType.TIME, required=True),+ "priority": make_field(field_type=None, choices=["low", "high"], required=True),+ "metadata": make_field(field_type=FieldType.DICT, key_type="text", value_type="text", required=True),+ "supplier": make_field(field_type=FieldType.CONCEPT, concept_ref="Supplier", required=True),+ "lines": make_field(field_type=FieldType.LIST, item_type="concept", item_concept_ref="InvoiceLine", required=True),+ "tags": make_field(field_type=FieldType.LIST, item_type="text", required=True),+ "attachments": make_field(field_type=FieldType.LIST, required=True),+ "scan": make_field(field_type=FieldType.CONCEPT, concept_ref="native.Document"),+ "pages": make_field(field_type=FieldType.LIST, item_type="concept", item_concept_ref="native.Page"),+ "amount_due": make_field(field_type=FieldType.CONCEPT, concept_ref="native.Number", required=True),+ "note": make_field(field_type=FieldType.TEXT),+ "currency": make_field(field_type=FieldType.TEXT, default_value="EUR"),+ "remark": make_field(field_type=FieldType.CONCEPT, concept_ref="Remark", required=True),+ },+ ),+ "billing.Supplier": ConceptBlueprint(+ description="A supplier",+ structure={+ "name": make_field(field_type=FieldType.TEXT, required=True),+ "address": make_field(field_type=FieldType.CONCEPT, concept_ref="Address", required=True),+ },+ ),+ "billing.Address": ConceptBlueprint(description="A postal address", structure={"city": make_field(field_type=FieldType.TEXT, required=True)}),+ "billing.InvoiceLine": ConceptBlueprint(description="One line of an invoice", structure={"amount": make_field(field_type=FieldType.NUMBER)}),+ "billing.Receipt": ConceptBlueprint(description="A receipt, an invoice once paid", refines="Invoice"),+ "billing.Remark": "A remark written on an invoice",+ "billing.Memo": ConceptBlueprint(description="A memo with no structure"),+ "billing.MemoCopy": ConceptBlueprint(description="A copy of a memo", refines="Memo"),+ "billing.Caption": ConceptBlueprint(description="A caption", refines="Text"),+ "billing.Shipment": ConceptBlueprint(+ description="A shipment of parcels",+ structure={"parcels": make_field(field_type=FieldType.LIST, item_type="concept", item_concept_ref="Parcel", required=True)},+ ),+ "billing.Parcel": ConceptBlueprint(description="A parcel", structure={"weight": make_field(field_type=FieldType.NUMBER, required=True)}),+ "billing.Crate": ConceptBlueprint(+ description="A crate holding anything",+ structure={+ "contents": make_field(field_type=FieldType.CONCEPT, concept_ref="native.Anything", required=True),+ "extras": make_field(field_type=FieldType.LIST, item_type="concept", item_concept_ref="native.Anything", required=True),+ },+ ),+}+++def make_root(concept_ref: str, *, multiplicity: VariableMultiplicity | None = None) -> BindingRoot:+ return BindingRoot(concept_ref=concept_ref, multiplicity=multiplicity)+++RESOLVER = BlueprintConceptWalkResolver(concept_blueprints=CONCEPTS)++INVOICE = BindingRoot(concept_ref="billing.Invoice")+++# The content the runtime holds for the concepts above, as the structure generator would make it: raw Python+# values for plain fields, content classes for concept fields.+class AddressRecord(StructuredContent):+ city: str+++class SupplierRecord(StructuredContent):+ name: str+ address: AddressRecord+++class InvoiceLineRecord(StructuredContent):+ amount: float | None = None+++class InvoiceRecord(StructuredContent):+ supplier_name: str+ total: float+ item_count: int+ paid: bool+ issued_on: datetime.date+ issued_at: datetime.datetime+ cutoff: datetime.time+ priority: str+ metadata: dict[str, str]+ supplier: SupplierRecord+ lines: list[InvoiceLineRecord]+ tags: list[str]+ scan: DocumentContent | None = None+ pages: list[PageContent] | None = None+ note: str | None = None+ currency: str = "EUR"+++class ParcelRecord(StructuredContent):+ weight: float+++class ShipmentRecord(StructuredContent):+ parcels: list[ParcelRecord]+++class CrateRecord(StructuredContent):+ """A `native.Anything` field is generated as `Any`, so it holds a raw value of any type."""++ contents: Any+ extras: list[Any]+++INVOICE_ISSUED_AT = datetime.datetime(2026, 3, 14, 9, 30, tzinfo=datetime.UTC)+++def make_invoice_record(*, note: str | None = None, scan: DocumentContent | None = None) -> InvoiceRecord:+ """A fresh invoice each call, so a test mutating one never leaks into another."""+ return InvoiceRecord(+ supplier_name="Atelier Morvan",+ total=1250.5,+ item_count=3,+ paid=True,+ issued_on=datetime.date(2026, 3, 14),+ issued_at=INVOICE_ISSUED_AT,+ cutoff=datetime.time(17, 0),+ priority="high",+ metadata={"order_ref": "PO-118"},+ supplier=SupplierRecord(name="Atelier Morvan", address=AddressRecord(city="Quimper")),+ lines=[InvoiceLineRecord(amount=1000.0), InvoiceLineRecord(amount=None), InvoiceLineRecord(amount=250.5)],+ tags=["urgent", "export"],+ scan=scan,+ note=note,+ )+++def make_page(*, text: str, view: ImageContent | None) -> PageContent:+ return PageContent(text_and_images=TextAndImagesContent(text=TextContent(text=text)), page_view=view)+++def make_view(*, url: str) -> ImageContent:+ """An image carrying every field a whole-value copy must keep."""+ return ImageContent(+ url=url,+ caption=f"The view at {url}",+ source_prompt="A catalog page, photographed flat",+ source_negative_prompt="No glare",+ mime_type="image/png",+ width=800,+ height=1200,+ )diff --git a/tests/unit/pipelex/pipe_controllers/binding/test_library_concept_walk_resolver.py b/tests/unit/pipelex/pipe_controllers/binding/test_library_concept_walk_resolver.pynew file mode 100644index 000000000..30089e0e1--- /dev/null+++ b/tests/unit/pipelex/pipe_controllers/binding/test_library_concept_walk_resolver.py@@ -0,0 +1,356 @@+import datetime+from typing import Any, cast++import pytest+from pydantic import Field+from pytest_mock import MockerFixture++from pipelex.core.concepts.concept import Concept+from pipelex.core.concepts.concept_blueprint import ConceptBlueprint+from pipelex.core.concepts.concept_factory import ConceptFactory+from pipelex.core.concepts.concept_structure_blueprint import ConceptStructureBlueprint, ConceptStructureBlueprintFieldType+from pipelex.core.concepts.exceptions import ConceptStructureClassNotFoundError+from pipelex.core.stuffs.structured_content import StructuredContent+from pipelex.core.stuffs.stuff_content import StuffContent+from pipelex.libraries.concept.concept_library_abstract import ConceptLibraryAbstract+from pipelex.libraries.concept.exceptions import ConceptLibraryError+from pipelex.pipe_controllers.binding.binding_concept_resolvers import BlueprintConceptWalkResolver, LibraryConceptWalkResolver, library_concept_key+from pipelex.pipe_controllers.binding.binding_derivation import BindingRoot, BindingValueKind, ConceptShape, derive_binding+from pipelex.pipe_controllers.binding.exceptions import BindingPathUnresolvedError+++class _HarbourRecord(StructuredContent):+ name: str+++class _OrphanRecord(StructuredContent):+ label: str+++class _CaptainRecord(StructuredContent):+ name: str+++class _LogbookRecord(StructuredContent):+ captain: _CaptainRecord+++def _default_escort() -> str:+ return "the harbour tug"+++class _VesselRecord(StructuredContent):+ name: str+ tonnage: float+ crew: int+ is_moored: bool+ arrived_on: datetime.date+ arrived_at: datetime.datetime+ tide: datetime.time+ papers: dict[str, str]+ home_port: _HarbourRecord+ ports_of_call: list[_HarbourRecord]+ cargo: list[str]+ flag: str | None = None+ stowaway: _OrphanRecord | None = None+ holds: list[Any]+ berth_note: str | None+ pilot: str | None = "the harbour pilot"+ escort: str | None = Field(default_factory=_default_escort)+ draught: float = 4.5+++def _concept(*, code: str, structure_class_name: str, domain_code: str = "harbour", **kwargs: Any) -> Concept:+ return Concept(domain_code=domain_code, code=code, description=f"A {code}", structure_class_name=structure_class_name, **kwargs)+++_CONCEPTS: dict[str, Concept] = {+ "harbour.Vessel": _concept(code="Vessel", structure_class_name="_VesselRecord"),+ "harbour.Harbour": _concept(code="Harbour", structure_class_name="_HarbourRecord"),+ "harbour.Berth": _concept(+ code="Berth",+ structure_class_name="harbour__Berth",+ declared_structure={"number": ConceptStructureBlueprint(description="The berth number", type=ConceptStructureBlueprintFieldType.INTEGER)},+ ),+ "harbour.GuestBerth": _concept(code="GuestBerth", structure_class_name="harbour__Berth", refines="Berth"),+ "harbour.Lost": _concept(code="Lost", structure_class_name="NoSuchClass"),+}+_CLASSES: dict[str, type[StuffContent]] = {"_VesselRecord": _VesselRecord, "_HarbourRecord": _HarbourRecord}+++def _make_concept_library(+ mocker: MockerFixture, *, concepts_by_key: dict[str, Concept], classes: dict[str, type[StuffContent]] | None = None+) -> ConceptLibraryAbstract:+ """A library holding each concept under its key, `alias->domain.Code` for a dependency's, looked up as the real one is."""+ concept_library: Any = mocker.MagicMock(spec=ConceptLibraryAbstract)+ known_classes = classes if classes is not None else _CLASSES++ def get_required_concept(concept_ref: str) -> Concept:+ if concept_ref in concepts_by_key:+ return concepts_by_key[concept_ref]+ if concept_ref == "native.Page":+ return _concept(code="Page", structure_class_name="PageContent", domain_code="native")+ msg = f"Concept '{concept_ref}' not found"+ raise ConceptLibraryError(msg)++ def list_concept_keys_for_ref(*, concept_ref: str) -> list[str]:+ direct_keys = [concept_ref] if concept_ref in concepts_by_key else []+ return direct_keys + sorted(key for key in concepts_by_key if key.endswith(f"->{concept_ref}"))++ def get_structure_class(*, concept: Concept) -> type[StuffContent]:+ if concept.structure_class_name not in known_classes:+ msg = f"No class '{concept.structure_class_name}'"+ raise ConceptStructureClassNotFoundError(msg)+ return known_classes[concept.structure_class_name]++ concept_library.get_required_concept.side_effect = get_required_concept+ concept_library.list_concept_keys_for_ref.side_effect = list_concept_keys_for_ref+ concept_library.get_structure_class.side_effect = get_structure_class+ concept_library.list_concepts.return_value = list(concepts_by_key.values())+ return cast("ConceptLibraryAbstract", concept_library)+++def resolver(mocker: MockerFixture) -> LibraryConceptWalkResolver:+ return LibraryConceptWalkResolver(concept_library=_make_concept_library(mocker, concepts_by_key=_CONCEPTS))+++_HARBOUR_PACKAGE = "github.com/invented/harbour-lib/harbour"+++def _structure_field(*, field_type: ConceptStructureBlueprintFieldType, **kwargs: Any) -> ConceptStructureBlueprint:+ return ConceptStructureBlueprint(description="A field", type=field_type, required=True, **kwargs)+++# A host and a dependency package each declare `harbour.Manifest` and `harbour.Captain`, with different structures.+_HOST_MANIFEST = _concept(+ code="Manifest",+ structure_class_name="harbour__Manifest",+ declared_structure={+ "cargo_weight": _structure_field(field_type=ConceptStructureBlueprintFieldType.NUMBER),+ "captain": _structure_field(field_type=ConceptStructureBlueprintFieldType.CONCEPT, concept_ref="Captain"),+ },+)+_HOST_CAPTAIN = _concept(+ code="Captain",+ structure_class_name="_CaptainRecord",+ declared_structure={"licence": _structure_field(field_type=ConceptStructureBlueprintFieldType.TEXT)},+)+_PACKAGE_MANIFEST = _concept(+ code="Manifest",+ structure_class_name="harbour__Manifest",+ declared_structure={+ "tonnage": _structure_field(field_type=ConceptStructureBlueprintFieldType.NUMBER),+ "captain": _structure_field(field_type=ConceptStructureBlueprintFieldType.CONCEPT, concept_ref="harbour.Captain"),+ "crew": _structure_field(field_type=ConceptStructureBlueprintFieldType.LIST, item_type="concept", item_concept_ref="Captain"),+ },+)+_PACKAGE_CAPTAIN = _concept(+ code="Captain",+ structure_class_name="_CaptainRecord",+ declared_structure={"name": _structure_field(field_type=ConceptStructureBlueprintFieldType.TEXT)},+)+_PACKAGE_SEALED_MANIFEST = _concept(code="SealedManifest", structure_class_name="harbour__Manifest", refines="Manifest")+_PACKAGE_LOGBOOK = _concept(code="Logbook", structure_class_name="_LogbookRecord")+_HOST_LOGBOOK = _concept(code="Logbook", structure_class_name="_LogbookRecord")++_PACKAGE_CONCEPTS: dict[str, Concept] = {+ f"{_HARBOUR_PACKAGE}->harbour.Manifest": _PACKAGE_MANIFEST,+ f"{_HARBOUR_PACKAGE}->harbour.Captain": _PACKAGE_CAPTAIN,+ f"{_HARBOUR_PACKAGE}->harbour.SealedManifest": _PACKAGE_SEALED_MANIFEST,+ f"{_HARBOUR_PACKAGE}->harbour.Logbook": _PACKAGE_LOGBOOK,+}+_HOST_CONCEPTS: dict[str, Concept] = {+ "harbour.Manifest": _HOST_MANIFEST,+ "harbour.Captain": _HOST_CAPTAIN,+ "harbour.Logbook": _HOST_LOGBOOK,+}+_PACKAGE_CLASSES: dict[str, type[StuffContent]] = {"_CaptainRecord": _CaptainRecord, "_LogbookRecord": _LogbookRecord}+++class TestLibraryConceptWalkResolver:+ @pytest.mark.parametrize(+ ("field_name", "value_kind", "concept_ref", "is_list", "may_hold_nothing"),+ [+ pytest.param("name", BindingValueKind.TEXT, None, False, False, id="str"),+ pytest.param("tonnage", BindingValueKind.NUMBER, None, False, False, id="float"),+ pytest.param("crew", BindingValueKind.NUMBER, None, False, False, id="int"),+ pytest.param("is_moored", BindingValueKind.YES_NO, None, False, False, id="bool"),+ pytest.param("arrived_on", BindingValueKind.DATE, None, False, False, id="date"),+ pytest.param("arrived_at", BindingValueKind.DATETIME, None, False, False, id="datetime"),+ pytest.param("tide", BindingValueKind.TIME, None, False, False, id="time"),+ pytest.param("papers", BindingValueKind.JSON, None, False, False, id="dict"),+ pytest.param("home_port", BindingValueKind.CONCEPT, "harbour.Harbour", False, False, id="content-class-of-one-concept"),+ pytest.param("ports_of_call", BindingValueKind.CONCEPT, "harbour.Harbour", True, False, id="list-of-content-class"),+ pytest.param("cargo", BindingValueKind.TEXT, None, True, False, id="list-of-str"),+ pytest.param("flag", BindingValueKind.TEXT, None, False, True, id="optional-str"),+ pytest.param("stowaway", BindingValueKind.UNDERIVABLE, None, False, True, id="content-class-of-no-concept"),+ pytest.param("holds", BindingValueKind.UNDERIVABLE, None, True, False, id="list-of-anything"),+ pytest.param("berth_note", BindingValueKind.TEXT, None, False, True, id="required-but-nullable"),+ pytest.param("pilot", BindingValueKind.TEXT, None, False, True, id="nullable-with-a-default"),+ pytest.param("escort", BindingValueKind.TEXT, None, False, True, id="nullable-with-a-default-factory"),+ pytest.param("draught", BindingValueKind.NUMBER, None, False, False, id="not-nullable-with-a-default"),+ ],+ )+ def test_a_class_backed_concept_is_walked_through_its_class_fields(+ self,+ resolver: LibraryConceptWalkResolver,+ field_name: str,+ value_kind: BindingValueKind,+ concept_ref: str | None,+ is_list: bool,+ may_hold_nothing: bool,+ ) -> None:+ walkable = resolver.resolve_walkable_concept(concept_ref="harbour.Vessel")++ assert walkable.shape == ConceptShape.STRUCTURE+ walkable_field = walkable.get_field(name=field_name)+ assert walkable_field is not None+ assert walkable_field.value_kind == value_kind+ assert walkable_field.concept_ref == concept_ref+ assert walkable_field.is_list is is_list+ assert walkable_field.may_hold_nothing is may_hold_nothing++ def test_a_path_through_class_backed_concepts_derives(self, resolver: LibraryConceptWalkResolver) -> None:+ derivation = derive_binding(path="vessel.ports_of_call.name", root=BindingRoot(concept_ref="harbour.Vessel"), resolver=resolver)++ assert derivation.concept_ref == "native.Text"+ assert derivation.multiplicity is True++ def test_a_field_whose_class_maps_to_no_concept_is_refused(self, resolver: LibraryConceptWalkResolver) -> None:+ with pytest.raises(BindingPathUnresolvedError, match="_OrphanRecord"):+ derive_binding(path="vessel.stowaway", root=BindingRoot(concept_ref="harbour.Vessel"), resolver=resolver)++ def test_a_declared_structure_is_read_before_the_class(self, resolver: LibraryConceptWalkResolver) -> None:+ walkable = resolver.resolve_walkable_concept(concept_ref="harbour.Berth")++ assert walkable.shape == ConceptShape.STRUCTURE+ assert walkable.field_names == ["number"]++ def test_a_refinement_inherits_the_structure_it_refines(self, resolver: LibraryConceptWalkResolver) -> None:+ walkable = resolver.resolve_walkable_concept(concept_ref="harbour.GuestBerth")++ assert walkable.concept_ref == "harbour.GuestBerth"+ assert walkable.shape == ConceptShape.STRUCTURE+ assert walkable.field_names == ["number"]++ def test_a_native_is_read_off_its_pinned_definition(self, resolver: LibraryConceptWalkResolver) -> None:+ walkable = resolver.resolve_walkable_concept(concept_ref="native.Page")++ assert walkable.shape == ConceptShape.STRUCTURE+ page_view = walkable.get_field(name="page_view")+ assert page_view is not None+ assert page_view.concept_ref == "native.Image"++ @pytest.mark.parametrize(+ ("concept_ref", "shape_reason"),+ [+ pytest.param("harbour.Nowhere", "is not a concept of the library", id="unknown-concept"),+ pytest.param("harbour.Lost", "has a structure class that cannot be resolved", id="unresolvable-class"),+ ],+ )+ def test_a_concept_the_library_cannot_show_offers_nothing_to_walk(+ self, resolver: LibraryConceptWalkResolver, concept_ref: str, shape_reason: str+ ) -> None:+ walkable = resolver.resolve_walkable_concept(concept_ref=concept_ref)++ assert walkable.shape == ConceptShape.NO_STRUCTURE+ assert walkable.shape_reason == shape_reason++ def test_a_required_field_that_admits_none_may_find_nothing(self, resolver: LibraryConceptWalkResolver) -> None:+ """A class field required but annotated `str | None` can hold `None`, so a single result read through it may be absent."""+ derivation = derive_binding(path="vessel.berth_note", root=BindingRoot(concept_ref="harbour.Vessel"), resolver=resolver)++ assert derivation.may_find_nothing is True+ assert derivation.first_optional_path == "vessel.berth_note"++ def test_both_spellings_of_a_description_only_concept_offer_nothing_to_walk(self, mocker: MockerFixture) -> None:+ """`Notice = "..."` and a `[concept.Bulletin]` table holding a description alone get one verdict, from both resolvers."""+ notice = ConceptFactory.make_from_blueprint(domain_code="harbour", concept_code="Notice", blueprint_or_string_description="A notice")+ bulletin = ConceptFactory.make_from_blueprint(+ domain_code="harbour", concept_code="Bulletin", blueprint_or_string_description=ConceptBlueprint(description="A bulletin")+ )+ assert notice.is_described_only+ assert bulletin.is_described_only+ assert bulletin.refines == "native.Text"+ library_resolver = LibraryConceptWalkResolver(+ concept_library=_make_concept_library(mocker, concepts_by_key={**_CONCEPTS, "harbour.Notice": notice, "harbour.Bulletin": bulletin})+ )+ blueprint_resolver = BlueprintConceptWalkResolver(+ concept_blueprints={"harbour.Notice": "A notice", "harbour.Bulletin": ConceptBlueprint(description="A bulletin")}+ )++ for walk_resolver in (library_resolver, blueprint_resolver):+ for root_name, concept_ref in (("notice", "harbour.Notice"), ("bulletin", "harbour.Bulletin")):+ walkable = walk_resolver.resolve_walkable_concept(concept_ref=concept_ref)+ assert walkable.shape == ConceptShape.NO_STRUCTURE+ assert walkable.shape_reason == "is declared with neither a structure nor refines"+ with pytest.raises(BindingPathUnresolvedError, match="is declared with neither a structure nor refines") as raised:+ derive_binding(path=f"{root_name}.text", root=BindingRoot(concept_ref=concept_ref), resolver=walk_resolver)+ assert raised.value.failed_segment == "text"+ bare = derive_binding(path=root_name, root=BindingRoot(concept_ref=concept_ref), resolver=walk_resolver)+ assert bare.concept_ref == concept_ref++ @pytest.mark.parametrize(+ ("concept", "expected_key"),+ [+ pytest.param(_PACKAGE_MANIFEST, f"{_HARBOUR_PACKAGE}->harbour.Manifest", id="a-package-concept-by-identity"),+ pytest.param(_HOST_MANIFEST, "harbour.Manifest", id="a-host-concept-by-identity"),+ pytest.param(_PACKAGE_CAPTAIN.model_copy(), f"{_HARBOUR_PACKAGE}->harbour.Captain", id="a-copy-of-a-package-concept-by-equality"),+ pytest.param(_concept(code="Pier", structure_class_name="harbour__Pier"), "harbour.Pier", id="a-concept-the-library-lacks"),+ ],+ )+ def test_a_concept_is_keyed_where_the_library_holds_it(self, mocker: MockerFixture, concept: Concept, expected_key: str) -> None:+ concept_library = _make_concept_library(mocker, concepts_by_key={**_HOST_CONCEPTS, **_PACKAGE_CONCEPTS})++ assert library_concept_key(concept_library=concept_library, concept=concept) == expected_key++ @pytest.mark.parametrize(+ "host_concepts",+ [+ pytest.param({}, id="a-package-concept-alone"),+ pytest.param(_HOST_CONCEPTS, id="beside-host-concepts-of-the-same-spelling"),+ ],+ )+ def test_a_package_concept_is_walked_through_the_package_s_own_definitions(+ self, mocker: MockerFixture, host_concepts: dict[str, Concept]+ ) -> None:+ """Its fields, its list items, its refinements and its class-backed fields all name the package's concepts."""+ concept_library = _make_concept_library(mocker, concepts_by_key={**host_concepts, **_PACKAGE_CONCEPTS}, classes=_PACKAGE_CLASSES)+ package_resolver = LibraryConceptWalkResolver(concept_library=concept_library)+ root = BindingRoot(concept_ref=library_concept_key(concept_library=concept_library, concept=_PACKAGE_MANIFEST))++ assert root.concept_ref == f"{_HARBOUR_PACKAGE}->harbour.Manifest"+ assert derive_binding(path="manifest.tonnage", root=root, resolver=package_resolver).concept_ref == "native.Number"+ assert derive_binding(path="manifest.captain.name", root=root, resolver=package_resolver).concept_ref == "native.Text"+ assert derive_binding(path="manifest.captain", root=root, resolver=package_resolver).concept_ref == f"{_HARBOUR_PACKAGE}->harbour.Captain"+ crew = derive_binding(path="manifest.crew.name", root=root, resolver=package_resolver)+ assert crew.concept_ref == "native.Text"+ assert crew.multiplicity is True+ with pytest.raises(BindingPathUnresolvedError, match="has no field 'cargo_weight'"):+ derive_binding(path="manifest.cargo_weight", root=root, resolver=package_resolver)+ with pytest.raises(BindingPathUnresolvedError, match="has no field 'licence'"):+ derive_binding(path="manifest.captain.licence", root=root, resolver=package_resolver)++ sealed = BindingRoot(concept_ref=f"{_HARBOUR_PACKAGE}->harbour.SealedManifest")+ assert derive_binding(path="sealed.tonnage", root=sealed, resolver=package_resolver).concept_ref == "native.Number"++ logbook = package_resolver.resolve_walkable_concept(concept_ref=f"{_HARBOUR_PACKAGE}->harbour.Logbook")+ captain_field = logbook.get_field(name="captain")+ assert captain_field is not None+ assert captain_field.concept_ref == f"{_HARBOUR_PACKAGE}->harbour.Captain"++ def test_a_host_concept_is_walked_through_its_own_definitions_beside_a_package_namesake(self, mocker: MockerFixture) -> None:+ concept_library = _make_concept_library(mocker, concepts_by_key={**_HOST_CONCEPTS, **_PACKAGE_CONCEPTS}, classes=_PACKAGE_CLASSES)+ host_resolver = LibraryConceptWalkResolver(concept_library=concept_library)+ root = BindingRoot(concept_ref=library_concept_key(concept_library=concept_library, concept=_HOST_MANIFEST))++ assert root.concept_ref == "harbour.Manifest"+ assert derive_binding(path="manifest.captain.licence", root=root, resolver=host_resolver).concept_ref == "native.Text"+ assert derive_binding(path="manifest.captain", root=root, resolver=host_resolver).concept_ref == "harbour.Captain"+ with pytest.raises(BindingPathUnresolvedError, match="has no field 'tonnage'"):+ derive_binding(path="manifest.tonnage", root=root, resolver=host_resolver)+ logbook = host_resolver.resolve_walkable_concept(concept_ref="harbour.Logbook")+ captain_field = logbook.get_field(name="captain")+ assert captain_field is not None+ assert captain_field.concept_ref == "harbour.Captain"diff --git a/tests/unit/pipelex/pipe_controllers/sequence/test_sequence_step_union.py b/tests/unit/pipelex/pipe_controllers/sequence/test_sequence_step_union.pynew file mode 100644index 000000000..c087d89c5--- /dev/null+++ b/tests/unit/pipelex/pipe_controllers/sequence/test_sequence_step_union.py@@ -0,0 +1,200 @@+from typing import Any++import pytest+from pydantic import ValidationError++from pipelex.mthds_parsing.exceptions import MthdsParserError+from pipelex.mthds_parsing.parser import MthdsParser+from pipelex.pipe_controllers.binding.binding_step_blueprint import BindingStepBlueprint+from pipelex.pipe_controllers.parallel.pipe_parallel_blueprint import PipeParallelBlueprint+from pipelex.pipe_controllers.sequence.pipe_sequence_blueprint import PipeSequenceBlueprint+from pipelex.pipe_controllers.sub_pipe_blueprint import SubPipeBlueprint+from pipelex.validation_error_types import PipeValidationErrorType++_BUNDLE_HEADER = """domain = "billing"+description = "Reading the total of an invoice"++[concept.Invoice]+description = "An invoice sent by a supplier"++[concept.Invoice.structure]+total = { type = "number", description = "The amount due", required = true }++[pipe.write_receipt]+type = "PipeCompose"+description = "Writes a receipt for an amount"+inputs = { total_amount = "Number" }+output = "Text"+template = "Received $total_amount"++[pipe.acknowledge_invoice]+type = "PipeSequence"+description = "Acknowledges an invoice by its total"+inputs = { invoice = "Invoice" }+output = "Text"+"""+++def _sequence_bundle(*, steps_toml: str) -> str:+ return f"{_BUNDLE_HEADER}steps = [\n{steps_toml}\n]\n"+++def _parse_sequence(*, steps_toml: str) -> PipeSequenceBlueprint:+ bundle = MthdsParser.make_pipelex_bundle_blueprint(mthds_content=_sequence_bundle(steps_toml=steps_toml), mthds_source="main.mthds")+ assert bundle.pipe is not None+ sequence = bundle.pipe["acknowledge_invoice"]+ assert isinstance(sequence, PipeSequenceBlueprint)+ return sequence+++class TestSequenceStepUnion:+ def test_a_step_carrying_from_parses_as_a_binding_step(self) -> None:+ sequence = _parse_sequence(+ steps_toml=""" { from = "invoice.total", result = "total_amount" },+ { pipe = "write_receipt", result = "receipt" },""",+ )++ binding_step = sequence.steps[0]+ assert isinstance(binding_step, BindingStepBlueprint)+ assert binding_step.from_path == "invoice.total"+ assert binding_step.result == "total_amount"+ assert binding_step.root_name == "invoice"+ assert isinstance(sequence.steps[1], SubPipeBlueprint)+ assert sequence.pipe_steps == [SubPipeBlueprint(pipe="write_receipt", result="receipt")]++ def test_a_binding_step_runs_no_pipe(self) -> None:+ sequence = _parse_sequence(+ steps_toml=""" { from = "invoice.total", result = "total_amount" },+ { pipe = "write_receipt", result = "receipt" },""",+ )++ assert sequence.pipe_dependencies == {"write_receipt"}+ assert sequence.ordered_pipe_dependencies == ["write_receipt"]++ @pytest.mark.parametrize(+ "dump_kwargs",+ [+ pytest.param({}, id="default"),+ pytest.param({"mode": "json"}, id="json-mode"),+ pytest.param({"by_alias": True}, id="by-alias"),+ pytest.param({"by_alias": False}, id="by-field-name"),+ pytest.param({"mode": "json", "exclude_none": True}, id="crate-emission"),+ ],+ )+ def test_a_binding_step_serializes_under_from_whatever_the_caller_asks(self, dump_kwargs: dict[str, Any]) -> None:+ """`from_path` is Python's name only: every dump writes `from`, the one key the parser reads back."""+ binding_step = BindingStepBlueprint.model_validate({"from": "invoice.total", "result": "total_amount"})++ dumped = binding_step.model_dump(**dump_kwargs)++ assert dumped == {"from": "invoice.total", "result": "total_amount"}+ assert BindingStepBlueprint.model_validate(dumped) == binding_step++ def test_a_sequence_holding_a_binding_step_dumps_and_validates_again(self) -> None:+ sequence = _parse_sequence(+ steps_toml=""" { from = "invoice.total", result = "total_amount" },+ { pipe = "write_receipt", result = "receipt" },""",+ )++ dumped = sequence.model_dump(mode="json")++ assert dumped["steps"][0] == {"from": "invoice.total", "result": "total_amount"}+ assert PipeSequenceBlueprint.model_validate(dumped) == sequence++ @pytest.mark.parametrize(+ ("step_toml", "message_fragment"),+ [+ pytest.param(+ '{ pipe = "write_receipt", from = "invoice.total", result = "total_amount" }',+ "carries both `pipe` and `from`",+ id="I6-pipe-and-from",+ ),+ pytest.param('{ from = "invoice.total" }', "a binding step without `result`", id="I7-without-result"),+ pytest.param(+ '{ from = "invoice.total", result = "total_amount", batch_over = "totals", batch_as = "total" }',+ "cannot carry `batch_over`, `batch_as`",+ id="I8-batch-fields",+ ),+ pytest.param('{ from = "invoice.lines[0]", result = "first_line" }', "is not a path", id="I9-subscript"),+ pytest.param('{ from = "invoice._total", result = "total_amount" }', "is not a path", id="I9-underscore-led-segment"),+ pytest.param('{ from = "invoice.total", result = "TotalAmount" }', "is not a plain input name", id="result-not-plain"),+ pytest.param('{ from = "invoice.total", result = "total.amount" }', "is not a plain input name", id="result-dotted"),+ ],+ )+ def test_a_malformed_binding_step_is_refused_as_binding_step_invalid(self, step_toml: str, message_fragment: str) -> None:+ with pytest.raises(MthdsParserError) as exc_info:+ MthdsParser.make_pipelex_bundle_blueprint(mthds_content=_sequence_bundle(steps_toml=f" {step_toml},"), mthds_source="main.mthds")++ errors = exc_info.value.validation_errors+ assert [error.error_type for error in errors] == [PipeValidationErrorType.BINDING_STEP_INVALID]+ assert errors[0].pipe_code == "acknowledge_invoice"+ assert message_fragment in errors[0].message++ @pytest.mark.parametrize(+ "step_toml",+ [+ pytest.param('{ from_path = "invoice.total", result = "total_amount" }', id="from-path-alone"),+ pytest.param('{ from = "invoice.total", from_path = "invoice.total", result = "total_amount" }', id="from-path-beside-from"),+ ],+ )+ def test_a_step_spelling_its_path_from_path_is_refused(self, step_toml: str) -> None:+ """`from_path` is the Python name of the field, not MTHDS: the parser refuses it, as the schema does."""+ with pytest.raises(MthdsParserError) as exc_info:+ MthdsParser.make_pipelex_bundle_blueprint(mthds_content=_sequence_bundle(steps_toml=f" {step_toml},"), mthds_source="main.mthds")++ assert "from_path" in str(exc_info.value)+ with pytest.raises(ValidationError):+ BindingStepBlueprint.model_validate({"from_path": "invoice.total", "result": "total_amount"})++ def test_i10_a_binding_step_in_a_parallel_branch_is_refused(self) -> None:+ mthds_content = f"""{_BUNDLE_HEADER}steps = [{{ pipe = "acknowledge_in_parallel", result = "receipt" }}]++[pipe.acknowledge_in_parallel]+type = "PipeParallel"+description = "Writes receipts in parallel"+inputs = {{ invoice = "Invoice" }}+output = "Text"+add_each_output = true+branches = [+ {{ from = "invoice.total", result = "total_amount" }},+]+"""+ with pytest.raises(MthdsParserError) as exc_info:+ MthdsParser.make_pipelex_bundle_blueprint(mthds_content=mthds_content, mthds_source="main.mthds")++ errors = exc_info.value.validation_errors+ assert [error.error_type for error in errors] == [PipeValidationErrorType.BINDING_STEP_INVALID]+ assert errors[0].pipe_code == "acknowledge_in_parallel"+ assert "bind the value in a step of the calling PipeSequence, before the PipeParallel step" in errors[0].message++ def test_a_parallel_branch_keeps_the_pipe_step_shape(self) -> None:+ assert PipeParallelBlueprint.model_fields["branches"].annotation == list[SubPipeBlueprint]++ @pytest.mark.parametrize(+ "steps",+ [+ pytest.param(5, id="a-number"),+ pytest.param(None, id="null"),+ pytest.param("write_receipt", id="a-string"),+ ],+ )+ def test_steps_that_are_not_a_list_are_refused_as_a_list_type(self, steps: object) -> None:+ with pytest.raises(ValidationError) as exc_info:+ PipeSequenceBlueprint.model_validate({"description": "Acknowledges an invoice", "output": "Text", "steps": steps})++ assert [error["type"] for error in exc_info.value.errors()] == ["list_type"]+ assert exc_info.value.errors()[0]["loc"] == ("steps",)++ @pytest.mark.parametrize(+ "branches",+ [+ pytest.param(None, id="null"),+ pytest.param(5, id="a-number"),+ ],+ )+ def test_branches_that_are_not_a_list_are_refused_as_a_list_type(self, branches: object) -> None:+ with pytest.raises(ValidationError) as exc_info:+ PipeParallelBlueprint.model_validate({"description": "Writes receipts in parallel", "output": "Text", "branches": branches})++ assert [error["type"] for error in exc_info.value.errors()] == ["list_type"]+ assert exc_info.value.errors()[0]["loc"] == ("branches",)diff --git a/tests/unit/pipelex/pipe_machinery/rendering/test_output_renderer_collect_sequence.py b/tests/unit/pipelex/pipe_machinery/rendering/test_output_renderer_collect_sequence.pyindex db974aa0a..5511df741 100644--- a/tests/unit/pipelex/pipe_machinery/rendering/test_output_renderer_collect_sequence.py+++ b/tests/unit/pipelex/pipe_machinery/rendering/test_output_renderer_collect_sequence.py@@ -5,10 +5,13 @@ from pytest_mock import MockerFixturefrom pipelex.core.concepts.concept_representation_generator import ConceptRepresentationFormatfrom pipelex.core.concepts.exceptions import ConceptError+from pipelex.core.pipes.exceptions import PipeValidationErrorfrom pipelex.libraries.concept.concept_library import ConceptLibrary+from pipelex.pipe_controllers.binding.binding_step import BindingStepfrom pipelex.pipe_machinery.rendering.output_renderer import (_collect_possible_outputs, # pyright: ignore[reportPrivateUsage])+from pipelex.validation_error_types import PipeValidationErrorTypeGET_REQUIRED_PIPE_TARGET = "pipelex.pipe_machinery.rendering.output_renderer.get_required_pipe"@@ -119,3 +122,18 @@ class TestCollectPossibleOutputsSequence:leaf_pipe.output.render_stuff_spec.assert_called_once_with(concept_provider=renderer_concept_library, output_format=ConceptRepresentationFormat.JSON)++ def test_a_last_binding_the_flow_cannot_derive_yields_no_outputs(self, mocker: MockerFixture) -> None:+ """A binding ending the sequence whose path does not resolve is rendered as no outputs, as a render failure is."""+ sequence_pipe = _make_sequence_pipe(mocker, ["first_step"])+ sequence_pipe.sequential_sub_pipes.append(BindingStep(from_path="invoice.totl", output_name="total_amount"))+ sequence_pipe.build_typed_flow.side_effect = PipeValidationError(+ message="the binding step cannot be derived", error_type=PipeValidationErrorType.BINDING_PATH_UNRESOLVED+ )+ get_pipe_mock = mocker.patch(GET_REQUIRED_PIPE_TARGET)++ result = _collect_possible_outputs(sequence_pipe)++ assert result == []+ sequence_pipe.build_typed_flow.assert_called_once_with()+ get_pipe_mock.assert_not_called()diff --git a/tests/unit/pipelex/pipeline/fixes/test_blueprint_fix_planner.py b/tests/unit/pipelex/pipeline/fixes/test_blueprint_fix_planner.pyindex 46fc18ef2..35552d66b 100644--- a/tests/unit/pipelex/pipeline/fixes/test_blueprint_fix_planner.py+++ b/tests/unit/pipelex/pipeline/fixes/test_blueprint_fix_planner.py@@ -74,7 +74,13 @@ _REDUNDANT_INPUT_BUNDLE_HEADER = """domain = "dotted_safety"description = "Inputs declared once by their root and again by a dotted path into it"[concept]-Item = "An item of an order"+Remark = "A remark, declared without a structure"++[concept.Item]+description = "An item of an order"++[concept.Item.structure]+x = "The item's label"[pipe.read_input]type = "PipeLLM"@@ -343,6 +349,43 @@ class TestBlueprintFixPlanner:None,id="differing-concept-root-last",),+ pytest.param(+ [("data", "Number"), ("data.text", "Text")],+ FixSafety.UNSAFE,+ "data.text",+ "'data.text' cannot be read through the concept declared for 'data', so a template reading it may fail once the key is deleted. "+ "Cannot bind 'data.text': 'data' holds a 'native.Number', which has no field 'text'.",+ id="root-concept-without-the-field",+ ),+ pytest.param(+ [("items", "Item"), ("items.label", "Text")],+ FixSafety.UNSAFE,+ "items.label",+ "Cannot bind 'items.label': 'items' holds a 'dotted_safety.Item', which has no field 'label'. Its fields are: 'x'.",+ id="declared-structure-without-the-field",+ ),+ pytest.param(+ [("remark", "Remark"), ("remark.text", "Text")],+ FixSafety.UNSAFE,+ "remark.text",+ "'remark' holds a 'dotted_safety.Remark', which is declared with neither a structure nor refines",+ id="root-concept-without-a-structure",+ ),+ pytest.param(+ [("data", "Number?"), ("data.text", "Text")],+ FixSafety.UNSAFE,+ "data.text",+ "remove `?` from 'data' if the root must not be optional. "+ "Warning: 'data.text' cannot be read through the concept declared for 'data'",+ id="dropped-marker-and-unreadable-path",+ ),+ pytest.param(+ [("data", "Text"), ("data.text", "Text")],+ FixSafety.SAFE,+ "data.text",+ None,+ id="single-field-native-read-through-its-field",+ ),pytest.param([("data", "Text"), ("data.text", "Text[1]")],FixSafety.SAFE,