Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
The inference-backend surface: a fourth migration surface, and the two pre-history entries #1104 left behind (#1116)
Louis Choquel198d83da4e1c
diff --git a/.claude/skills/add-migration/SKILL.md b/.claude/skills/add-migration/SKILL.mdindex 8b4f648c9..ddaeff6b2 100644--- a/.claude/skills/add-migration/SKILL.md+++ b/.claude/skills/add-migration/SKILL.md@@ -15,7 +15,8 @@ description: ># Add a Migration EntryA configuration **surface** — `pipelex.toml` and its tiers, `telemetry.toml`,-`pipelex_service.toml` — ships a checked-in ledger at+`pipelex_service.toml`, the inference backend definitions in+`inference/backends/` — ships a checked-in ledger at`pipelex/migration/ledgers/<surface-id>.toml` recording, as data, every shapechange it has ever undergone. A user's existing file is repaired by replaying thatledger over it, so a schema change with no entry is a schema change that breaks@@ -221,9 +222,48 @@ pair to verify. This is rare and it is not a way past the gate — reach for itwhen there genuinely is no diff, and read`docs/migration-ledger.md` → "Pre-history entries" first.+### Inserting a pre-history entry *below* entries that already exist++The change you are accounting for may be older than an entry already in the+ledger — a key that moved before a later entry renamed the table it lived in. The+new entry then has to run **first**, because its operations address the historical+spelling, so it takes a version the ledger already uses and everything above it is+renumbered. Ids, versions and `current_schema_version` are forced to agree, so+this is one atomic edit, and the goldens move with it:++1. Insert the entry at `@N` and renumber every entry above it (`@N` → `@N+1`, and+ so on); bump `current_schema_version` to the new top.+2. `git mv goldens/<surface-id>/[email protected] fingerprint@N+1.json`, and the+ same for `[email protected]`. Repeat upwards, highest first.+3. **Copy** `[email protected] → [email protected]` and+ `[email protected] → [email protected]`. A pre-history entry's fingerprint pair+ must show no diff at all, and a copy makes that true by construction rather+ than by luck.+4. **Hand-edit one line per moved or copied fingerprint: the `"schema_version"` in+ its body.** A golden whose body disagrees with its filename is refused when it+ is read, and `make umig` reads the stored head *before* it writes anything — so+ a fingerprint still saying `N` under the name `@N+1` makes `umig` raise instead+ of regenerating. This is the one sanctioned exception to "never hand-edit a+ golden", and it is bounded to that single line.+5. Hand-author `goldens/<surface-id>/[email protected]` for the new entry.+6. Run `make umig`. **It must be a byte-pure no-op on everything the moves+ produced** — verify with `shasum -c` over the directory, or by diffing each+ moved golden against its pre-move self in `HEAD` and finding only the version+ line. That no-op *is* the proof the renumber was pure; any other diff is a+ finding, not something to regenerate around.+7. Sweep the tree for the old id (`git grep -n '<surface-id>@N'` across+ `pipelex/ tests/ docs/ CHANGELOG.md .claude/`). Every hit is either the new+ entry or a stale quote of the renumbered one; there is no third kind. Where a+ test or a docstring named a golden by version but meant "the shape before the+ change", make it say so rather than bumping the number — the construct is what+ goes stale, not the value.+8. Both ids need a changelog mention: a renumber otherwise reads as a new breaking+ entry that nobody wrote a bullet for.+## What this skill must never do-- **Never hand-edit a fingerprint or defaults golden.** They are generated; the+- **Never hand-edit a fingerprint or defaults golden**, with the single exception+ of the `"schema_version"` line during a renumber, above. They are generated; thegenerator is `make umig` and the diff is the review artifact.- **Never make a gate green by widening it.** If a check refuses the change,either the change owes an entry or the check found a real defect. Editing thediff --git a/.claude/skills/add-model/SKILL.md b/.claude/skills/add-model/SKILL.mdindex 633a11e5c..1046c602a 100644--- a/.claude/skills/add-model/SKILL.md+++ b/.claude/skills/add-model/SKILL.md@@ -179,6 +179,23 @@ make ccsThis checks that the two directories match. If it reports differences, somethingwent wrong with the sync.+### If you edited `portkey.toml`, one migration golden moves with it++The kit's `portkey.toml` is the `inference-backend` surface's **reference+document** — the migration gates byte-compare `pipelex/migration/goldens/inference-backend/[email protected]`+against it, so adding a model to that one file turns `make check-migration-schemas`+(alias `cmig`) red until the golden is regenerated:++```bash+make umig # rewrites the head goldens from the live source+make cmig # then green again+```++That is the designed workflow, not a workaround — the refusal names both the+golden and the file. No other backend TOML is coupled this way; the rest of the+directory is only read as convergence witnesses, which adding a model does not+disturb.+## Step 4: Add to test profile collectionsEdit `.pipelex-dev/test_profiles.toml` to include the new model in thediff --git a/.claude/skills/release/SKILL.md b/.claude/skills/release/SKILL.mdindex 0ce1b1652..f8f434b8f 100644--- a/.claude/skills/release/SKILL.md+++ b/.claude/skills/release/SKILL.md@@ -83,6 +83,14 @@ make check-migration-schemasthey are checked against each other — if the bullet is missing, write it now(house style: bold label, then two to four complete sentences).+ **A renumbered entry shows up in that diff as two ids, and both need a mention.**+ A pre-history entry inserted below existing ones takes a version already in use+ and pushes everything above it up, so the diff reads as one id modified and one+ added — which looks like two independent breaking changes and is one insertion.+ Do not treat the pushed-up id as an unbulleted new entry: the changelog must+ name the new entry *and* say that the existing one was renumbered, so a reader+ who quoted the old id somewhere can find it. Confirm `introduced_in` on both.+**A breaking ledger entry makes this a minor release**, per the houseconvention — if step 2 chose a patch bump and this step found one, go back andsettle the bump first, because the next check writes the new version into thediff --git a/.drift/acks/cli-docs.toml b/.drift/acks/cli-docs.tomlindex efafae729..ca638ab96 100644--- a/.drift/acks/cli-docs.toml+++ b/.drift/acks/cli-docs.toml@@ -1,8 +1,8 @@contract = "cli-docs"-digest = "sha256:1f527dbd72ff687de022ec7c8a7340ac386615487ce1405cec8f630f2a0ea6f2"-reviewed_by = "Claude (Opus 5)"-reviewed_at = "2026-08-17T06:56:06Z"-rationale = "Merging dev into the configuration-reshape branch; the trigger diff is PR 1115's would_write tri-state in agent_output.py and error_handlers.py plus the doctor's remaining Rich escaping. Both review targets arrived rewritten in the same change, so the review was to re-derive their claims from the code rather than to read them against each other. The field set agent-cli.md enumerates for the migration object - remedy, would_write, needs_attention, plans - is exactly what MigrationErrorBlock declares under extra=forbid, so nothing else can appear; its 'present when and only when the scan found something' holds because the builder in core/validation.py returns None on an empty or failed scan; and would_write is any(plan.did_change), which is also what makes the doctor's out_of_date row a promise the file gets rewritten. Re-derived both enumerated finding sets from the enums rather than from the table: TelemetryConfigFinding and PendingMigrationsFinding match the members agent-cli.md and the agent-CLI contract doc list, with nothing left over. Checked migrate.md's quantified claim that nothing will ever suggest deleting an out-of-date file and starting over: it survives and is now stronger, because the new would_write false branch points at the dry run and hand-correction while the start-over line remains only on the no-block path, where the file is wrong rather than old. docs/tools/cli/index.md still has no doctor page. The escaping half is presentation - no flag, stream, format, structured field or exit code moved. No doc change needed."+digest = "sha256:e0ae9abafd983c542d19e338a7b7f6cefa720d8c668872efa08d324ff417b13b"+reviewed_by = "Louis Choquel"+reviewed_at = "2026-08-17T13:45:26Z"+rationale = "Reviewed docs/tools/cli/ and the agent-CLI contract doc against the two CLI changes in this commit. One target was genuinely stale and is now false rather than merely incomplete: migrate.md said the command does not descend into subdirectories and that inference backends are never rewritten by it, which stopped being true when the walk learned to enter a directory a surface owns. Rewritten to say what it now does and, just as importantly, what it still does not - one level, only into a directory some configuration family owns, so inference/deck/ and the inference/backends.toml beside the directory stay out, and the model deck keeps its own update command. The other change is the wording of one rendered operation, which turns a literal wildcard segment into English: it renders ledger material only, so the agent-CLI doc's standing claim that paths, operation kinds and ledger-supplied values are all that appear is unaffected, and no flag, stream, format, structured field or exit code moved. docs/tools/cli/index.md still has no doctor page."[trigger_files]"pipelex/cli/__init__.py" = "blob:8b137891791fe96927ad78e64b0aad7bded08bdc"@@ -105,7 +105,7 @@ rationale = "Merging dev into the configuration-reshape branch; the trigger diff"pipelex/cli/commands/init/ui/types.py" = "blob:1361f29ca95bacc90bca90ea6c0e39f44b6734ab""pipelex/cli/commands/login/__init__.py" = "blob:e69de29bb2d1d6434b8b29ae775ad8c2e48c5391""pipelex/cli/commands/login/command.py" = "blob:ace1db9879626cc1722adaac12761aadaa6b24ed"-"pipelex/cli/commands/migrate_cmd.py" = "blob:be405b6310ad28be02f7a17c9abe2a37629845a3"+"pipelex/cli/commands/migrate_cmd.py" = "blob:fa352e722f238b4785e83e37e413d8728a820c0c""pipelex/cli/commands/plugins_cmd.py" = "blob:9657b713c3650ff4a929e92ae68af453d896393e""pipelex/cli/commands/resolve_cmd.py" = "blob:2928a0e26eab7983367a59a78533f6618817feba""pipelex/cli/commands/run/__init__.py" = "blob:e69de29bb2d1d6434b8b29ae775ad8c2e48c5391"diff --git a/.drift/acks/config-docs.toml b/.drift/acks/config-docs.tomlindex eb58b317c..7fa62aa1f 100644--- a/.drift/acks/config-docs.toml+++ b/.drift/acks/config-docs.toml@@ -1,8 +1,8 @@contract = "config-docs"-digest = "sha256:820c272d92e41c21d884f2a1daee775e689282ee51f176a99fa40ea3d47f86e3"+digest = "sha256:bb8a1f8414fe7bf387b68beadaeedf44cededf5f2213f9fbdf94f84968df7056"reviewed_by = "Louis Choquel"-reviewed_at = "2026-08-17T08:49:54Z"-rationale = "Re-ack after the review round on the first push. One trigger file moved, config_surface.py, and it carries a real fix rather than a wording change: _is_within resolved the whole path before taking its parent, so a pipelex.toml symlinked out to a dotfiles repository resolved to a parent outside the walk and was told pipelex migrate could not reach it, when the walk enumerates the link by iterdir and writes straight through. It now resolves the parent and not the file, which keeps the reason the resolve was there - a config_dir reaching the same directory by another spelling - and drops the question it was accidentally answering. Pinned by a new symlink test in test_boot_tolerance.py, mutation-tested by reverting the expression. Asked the contract's own question again and the answer is unchanged: no config model moved, configs.py and config_cogt.py are untouched, no ledger moved, so no unsafe entry carrying declared_narrowed_paths is owed, and make cl stays green. On the docs half, this is a correction to which files the warning names, not to what a migration may do, so the note added to docs/configuration/index.md at the previous ack is still accurate as written - it says the walk is the global and project directories and that a file loaded from elsewhere is yours to update, which is exactly what the fixed predicate now decides correctly."+reviewed_at = "2026-08-17T14:33:11Z"+rationale = "Phase E of S7b, docs and changelog for the fourth surface. The trigger is config_surface.py's module docstring, which had become self-contradictory: it listed backend definitions among the readers that should keep rejecting the reserved [meta] table, while the same paragraph now names them as a configuration surface. Corrected, and the asymmetry behind it recorded honestly - the backend loader still reads through the generic TOML reader rather than through config_surface, so a backend file carrying [meta] would be refused as an unknown model table even though the migration engine reads the key there. Nothing writes that key on any surface, so it is latent rather than live. Reviewed all four targets. docs/configuration/**: inference-backend-config.md gained the section the Phase B ack promised - what the boot says when a backend file is behind, which files pipelex migrate reaches and which it does not, that a key you added yourself is reported rather than dropped, and the backup beside each rewritten file; index.md line 62 already carried the one-clause pointer added in Phase B and its link now resolves to a real section; telemetry-config.md is about a different surface and is unaffected. pipelex/migration/ledgers/: no ledger was edited this session, all four are legal and convergent, and no configuration model moved. configs.py and config_cogt.py are untouched - no key removed, renamed or narrowed, so no validator-level narrowing is owed an unsafe entry."[trigger_files]"pipelex/cogt/config_cogt.py" = "blob:02f71cb4a6b01aebda59c2e31f262ef9c865e17b"@@ -13,7 +13,7 @@ rationale = "Re-ack after the review round on the first push. One trigger file m"pipelex/system/configuration/config_loader.py" = "blob:fadc4557422f0c93335446a854d9883b3cceaf8c""pipelex/system/configuration/config_model.py" = "blob:83b29dc6e93393c9b98c4353636caf0135c80339""pipelex/system/configuration/config_root.py" = "blob:c7d43851270912d17413cfc6a511247d3c7cc160"-"pipelex/system/configuration/config_surface.py" = "blob:0ab49a3bf69f55d5ab0930665ab98e14547c9b4a"+"pipelex/system/configuration/config_surface.py" = "blob:657ee0282ad07d831386f1cdadbad544a5b8ec40""pipelex/system/configuration/configs.py" = "blob:60f346ba1205b762d4b6cff63494b0b34171e6ad""pipelex/system/configuration/pipe_func_config.py" = "blob:dc8890e914fbcdb611147e42eaa0cd87b32288a5""pipelex/system/telemetry/telemetry_config.py" = "blob:7070951a6d3e4c10cb7d59822ce168c6590c53be"diff --git a/.gitignore b/.gitignoreindex eb14b7eb8..a95df5e3f 100644--- a/.gitignore+++ b/.gitignore@@ -53,6 +53,9 @@ telemetry_override.toml# local pipelex service configuration.pipelex/pipelex_service.toml+# the timestamped copy `pipelex migrate` leaves beside every file it rewrites+*.bak.[0-9]*Z+# temporary filestemp/diff --git a/CHANGELOG.md b/CHANGELOG.mdindex d62dded54..2822c2f3e 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -4,10 +4,12 @@### Added-- **A configuration that will not load now says whether it is wrong or merely old**: A validation error is raised against the merged configuration and carries no provenance — it names a key, not which file put it there, and certainly not whether that key was correct last month. When a configuration surface's model refuses, the error now also carries a dry-run scan of that surface over the same directories `pipelex migrate` walks: a structured `migration` block naming the remedy, whether that remedy would actually rewrite anything, whether anything there is a person's to resolve rather than the tool's, and the plan per file — the same shape `pipelex-agent migrate --dry-run --format json` emits, so an agent that parses one has already parsed the other. `error_domain` deliberately stays `config`: it is a closed cross-repo enum and the agent-hook specification routes anything else to BLOCK, so a stale configuration reported under a new domain would stop an agent instead of telling it what to run. Consumers branch on the block's presence — absent means the failure is not staleness — and then on `would_write`, because presence means the migration history has something to *say* about these files and not that a command repairs them: a block whose only finding is a path no entry explains, or an entry blocked before anything applied, has nothing to run, and naming the remedy there would send a reader to a run that writes nothing and leaves the same error standing. Every surface points that reader at `pipelex migrate --dry-run` instead, where the diagnosis is. The message gains a paragraph naming the files and the next move. No value read from a user's file appears in the block, which is the third of the three channels that rule covers. A caller validating something that is not a configuration surface — a `.mthds` bundle, a backend file, a model deck — names no surface and pays for no walk.+- **A configuration that will not load now says whether it is wrong or merely old**: A validation error is raised against the merged configuration and carries no provenance — it names a key, not which file put it there, and certainly not whether that key was correct last month. When a configuration surface's model refuses, the error now also carries a dry-run scan of that surface over the same directories `pipelex migrate` walks: a structured `migration` block naming the remedy, whether that remedy would actually rewrite anything, whether anything there is a person's to resolve rather than the tool's, and the plan per file — the same shape `pipelex-agent migrate --dry-run --format json` emits, so an agent that parses one has already parsed the other. `error_domain` deliberately stays `config`: it is a closed cross-repo enum and the agent-hook specification routes anything else to BLOCK, so a stale configuration reported under a new domain would stop an agent instead of telling it what to run. Consumers branch on the block's presence — absent means the failure is not staleness — and then on `would_write`, because presence means the migration history has something to *say* about these files and not that a command repairs them: a block whose only finding is a path no entry explains, or an entry blocked before anything applied, has nothing to run, and naming the remedy there would send a reader to a run that writes nothing and leaves the same error standing. Every surface points that reader at `pipelex migrate --dry-run` instead, where the diagnosis is. The message gains a paragraph naming the files and the next move. No value read from a user's file appears in the block, which is the third of the three channels that rule covers. A caller validating something that is not a configuration surface — a `.mthds` bundle, a model deck, a routing profile — names no surface and pays for no walk.++- **Inference backend definitions are a migration surface, and the key the templating change deleted is repaired on every machine that has one**: `#1104` removed `prompting_target` from the model-spec blueprint and from every backend file we ship — but `pipelex init` never overwrites, so the key survives in `inference/backends/*.toml` on every installation that predates it, where it is fatal twice over: in `[defaults]` it is copied wholesale into every model of the file, and on a single model it is rejected by name since unknown per-model keys must be header-shaped. Those files were reachable by no surface, no walk and no gate. They are now the `inference-backend` surface, which required the walk to learn about subdirectories — a file is claimed by the pair (directory, name), one level deep, and only into a directory some configuration family owns — and required the fingerprint to record a document whose *root* is the open node, since a backend file's root keys are model names rather than a field we could name. Boot tolerance joins at the backend loader too: a stale directory is replayed in memory, boots on the models the current files would produce, and warns once naming every file it carried. **Migration:** run `pipelex migrate` after upgrading — ledger entry `inference-backend@2` deletes `prompting_target` from every root table of every backend file, in `~/.pipelex/` and in each project `.pipelex/`, keeping one timestamped backup per file. Until you run it, each boot repeats the warning; nothing is ever written by a boot. A key *you* added that we have never heard of is still an error rather than something the entry silently drops, and the served Pipelex Gateway specs are untouched. The one key class the report stays silent about is a per-model request header, legal by shape rather than by name — and only on a model table: `[defaults]` is copied into every model of a file unsplit, so a header put there breaks all of them, and the report names it like any other key the schema cannot explain.- **A configuration file the ledger can explain no longer stops the boot**: A schema change used to leave every machine in the field with a boot that died on `extra="forbid"`, naming a key the user never chose to have. When a configuration surface fails to validate, its ledger is now replayed over the same files **in memory** and the result validated again; a boot that succeeds says so in a warning naming the files and, for each of them the command would actually reach, the `pipelex migrate` remedy. The retry replays whatever the loader merged, so a file loaded through `Pipelex.make(config_dir=…)` from outside the two directories `migrate` walks is named instead as the caller's to update where it lives, rather than pointed at a command that would then report nothing to do. Nothing is written either way — a tolerated boot leaves the file and its directory exactly as it found them, which is why the warning keeps coming back until the command is run. Tolerance widens what starts and never what is accepted: the re-validation is what decides, so material an `unsafe` entry is about still fails the boot because the model still refuses it. Anything that goes wrong inside the retry makes it decline rather than replace the error the user already has — theirs names the key to fix. A healthy configuration pays nothing: it reads no ledger, re-parses no document, and does not even import the migration engine, which is deferred for a second reason too, since the applier lives in the interpreter layer and the configuration loaders sit in the kernel layer that must not load it. This release is where it matters most: the configuration reshape gives `pipelex.toml` its first ledger entry, so the file every boot reads is the one tolerance carries forward on every machine that upgrades.-- **`pipelex migrate` and `pipelex-agent migrate`**: The migration engine now has commands, so a configuration that predates the installed pipelex can be repaired instead of re-initialized. Both walk the global `~/.pipelex/` and the project `.pipelex/` — those two directories only, and not recursively, because a subdirectory such as `inference/backends/` holds a different kind of file and one of them (`pipelex_gateway.toml`) matches the main configuration's tier glob exactly. Neither command boots: a broken configuration is the reason to reach for them, so they use the ledger, the applier and the filesystem and nothing else, and a test hands them a configuration that cannot load to keep it that way. The human command plans, shows the plan and asks; the machine one writes only with `--yes`, since it cannot ask, and passing `--dry-run` and `--yes` together is refused rather than resolved. Its answer is structured, and `needs_attention` is the verdict — *this run left something a person has to decide*, deliberately not *did anything get written* — with the exit code and the Markdown rendering as presentation. Every file rewritten is backed up beside itself first, and no value read from a user's file appears in either command's output or in the structured plan.+- **`pipelex migrate` and `pipelex-agent migrate`**: The migration engine now has commands, so a configuration that predates the installed pipelex can be repaired instead of re-initialized. Both walk the global `~/.pipelex/` and the project `.pipelex/` — those two directories only, one level deep, and into a subdirectory only when a configuration family owns it: `inference/backends/` is such a directory, `inference/deck/` is not and is never entered. A file is claimed by the pair (directory, name), which is what keeps `inference/backends/pipelex_gateway.toml` — a `pipelex_*.toml` match by name — from being rewritten under the main configuration's ledger. Neither command boots: a broken configuration is the reason to reach for them, so they use the ledger, the applier and the filesystem and nothing else, and a test hands them a configuration that cannot load to keep it that way. The human command plans, shows the plan and asks; the machine one writes only with `--yes`, since it cannot ask, and passing `--dry-run` and `--yes` together is refused rather than resolved. Its answer is structured, and `needs_attention` is the verdict — *this run left something a person has to decide*, deliberately not *did anything get written* — with the exit code and the Markdown rendering as presentation. Every file rewritten is backed up beside itself first, and no value read from a user's file appears in either command's output or in the structured plan.- **A migration now names what it cannot explain about a file**: A file *newer* than the running pipelex — an override migrated on one branch, then an older branch checked out — breaks the boot while a replay finds nothing to do, because every operation's source is absent. That silence beside a dead boot is the failure the migration project exists to remove, so every plan now carries an `unexplained[]` listing the paths of the migrated document that the current schema does not know and no ledger entry accounts for, each with the two readings the tool cannot tell apart: a typo, or a file written by a newer pipelex. This is the one part of a migration that needs the configuration model — the question goes to the surface's fingerprint, the same projection the coverage gate diffs, so the replay engine stays model-free. The document it reads is the one the run leaves behind rather than the one it found, which is what keeps every key the ledger is about to repair out of the list, and a path a *blocked* entry is about is left to that entry, which reports it by name with its own guidance. A key the user chose is never rendered: beneath an open mapping a typo is reported at `queues.*.retries`, naming only the segment the schema does not know.- **A configuration file may now be refused for declaring a schema version its ledger no longer reaches**: `min_supported_schema_version` had no reader, which left the floor a comment. A migration now reads the reserved `[meta] schema_version` — exactly as tolerantly as the configuration loader strips it, so a malformed declaration is no declaration — and refuses a file below the floor by name (`unsupported_schema_version`), writing nothing. Without it, a ledger whose oldest entries had been squashed away would run over a file older than them, change nothing, and report it clean, because the applier skips an operation whose target is absent.@@ -17,7 +19,7 @@- **A schema change that narrows what a value may *be* is now caught, not waved through as additive**: A change can keep every path and every enumerated spelling and still break a user's file — `int` becomes `str`, a free string becomes an enum, a bound moves from `ge=1` to `ge=5`. Read as a path diff those look like additions, so the coverage gate used to ask for a golden regeneration, demand no version bump and no entry, and let the next boot reject a valid file with a green gate behind it. It now splits its verdict by *direction*: a widened type or a relaxed bound still costs only a regeneration, while a narrowing demands a bump and an entry carrying a `remap_value` for each narrowed path — or marked `unsafe`, which is the only remedy a tightened numeric bound has, since no structural operation can repair a value. The same rule applies to a stored entry, compared by origin so a narrowing hidden behind a rename is still caught, and to a `pre_history` entry, so the flag is not a way past it. To see a bound at all the fingerprint now records one: a **closed whitelist** over `gt`, `ge`, `lt`, `le`, `min_length`, `max_length` and `multiple_of`, read from `annotated_types` (now a declared dependency) and from the field metadata pydantic folds a top-level `Field(ge=1)` into, with anything unrecognized dropped rather than serialized — `pattern` excluded permanently, because regex containment is not decidable and every pattern edit would read as a tightening. One correction rides along: an enumerated type relaxed into a free string no longer reads as the loss of every spelling it had, and that is read structurally, so `list[enum]` becoming `list[str]` is the same benign loosening one container down — while a `list[enum]` flattened to a bare `str`, which is no loosening at all, still loses every spelling it had. The contract states in the same change what the gate structurally cannot see — narrowing expressed in a validator, which no projection of the annotations can reach.- **The telemetry configuration can now be migrated instead of thrown away, and the escape hatch that would have let it ship unverified is closed**: `telemetry.toml` carries the package's first migration entry, `telemetry-config@2`, which lifts a flat first-generation file — `telemetry_mode`, `host`, `project_api_key` and the rest at the document root — into today's `[custom_posthog]` section, keeping every value the user chose; four settings the current shape has no home for are dropped, and the entry says which and why. Because that change predates the first fingerprint, no diff describes it, and such an entry is marked `pre_history` and exempted from the coverage gate's accounting — so the verification that replaces it lands in the same change and neither ships without the other. A pre-history entry must declare the paths it removes (an empty declaration is refused when the ledger is parsed), none of those paths may appear in any fingerprint at or below its own version, the fingerprint pair it sits between must show no removal at all, and every operation must act inside the declaration. It also ships a hand-authored `[email protected]` beside the golden chain, which the transform check migrates and holds against the current shape and the current model — the same three claims every other entry answers. One comparator refinement came with it: a created path is now checked against `defaults@N` **or** `fingerprint@N`, because an optional key defaulting to `None` is a legal destination that no TOML document can carry.-- **Configuration surfaces now have a migration ledger and a gate that forces one to be written**: Each of the three configuration surfaces — `pipelex.toml` and its tiers, `telemetry.toml`, `pipelex_service.toml` — carries a checked-in ledger at `pipelex/migration/ledgers/<surface-id>.toml` recording, as data, every shape change it has ever undergone, plus a golden chain (`fingerprint@N`, `defaults@N`) that pins what the schema looked like at each version. All three start at schema version 1 with no entries: this release cuts the baseline, it migrates nothing. The new `make check-migration-schemas` (alias `cmig`, in `make check`) recomputes each surface's fingerprint and refuses a change that would break a user's file without recording how to repair it — a removed path or a lost enumerated spelling demands a version bump and an operation accounting for it, a destination the new schema does not know is refused by name, and an entry that removes a path the schema still has is refused as over-deletion. Its counterpart `make up-migration-schemas` (alias `umig`, in `make up`) regenerates the goldens; the regeneration diff is the review artifact. One standing invariant is checked whether or not anything changed: every required path must have a value in its surface's defaults layer, because that is what makes an added key absorbable rather than breaking. The contract the whole thing implements is `docs/migration-ledger.md`.+- **Configuration surfaces now have a migration ledger and a gate that forces one to be written**: Every configuration surface — `pipelex.toml` and its tiers, `telemetry.toml`, `pipelex_service.toml`, and the inference backend definitions — carries a checked-in ledger at `pipelex/migration/ledgers/<surface-id>.toml` recording, as data, every shape change it has ever undergone, plus a golden chain (`fingerprint@N`, `defaults@N`) that pins what the schema looked like at each version. Each is cut at schema version 1 with no entries; the entries this release then ships on top of that baseline are the bullets below. The new `make check-migration-schemas` (alias `cmig`, in `make check`) recomputes each surface's fingerprint and refuses a change that would break a user's file without recording how to repair it — a removed path or a lost enumerated spelling demands a version bump and an operation accounting for it, a destination the new schema does not know is refused by name, and an entry that removes a path the schema still has is refused as over-deletion. Its counterpart `make up-migration-schemas` (alias `umig`, in `make up`) regenerates the goldens; the regeneration diff is the review artifact. One standing invariant is checked whether or not anything changed: every required path must have a value in its surface's defaults layer, because that is what makes an added key absorbable rather than breaking. The contract the whole thing implements is `docs/migration-ledger.md`.- **The migration operation vocabulary**: Fix operations became a discriminated union on `kind`, published as two subsets — every kind for the `.mthds` fix path, and the structural kinds only for ledgers, so an operation that would write a value into a user's file fails when the ledger is parsed rather than at review time. Two new kinds land with it: `move_key`, which relocates a key across tables (the whole subtree travels when the key is table-valued, and a missing destination parent is created), and `remap_value`, which rewrites a value against an explicit old-to-new mapping. A `*` segment addresses every entry of an open mapping, expanded over the keys the file actually holds. A new `CONFLICT` outcome separates "this cannot be done without choosing on your behalf" — a rename or move onto an occupied destination, or an `ensure_table` over a key holding something that is not a table — from the benign `SKIPPED` it used to hide inside; a conflicting operation writes nothing at all.- **`[meta] schema_version` is reserved in configuration files**: Every configuration-surface reader now tolerates and strips it. Nothing writes it — the key exists so a later release can stamp a version into a file without that being a breaking change today.- **`make check-ledger` (alias `cl`)**: The second migration gate, and the first one that joins `make agent-check` as well as `make check`. It asks what a ledger entry is allowed to *say* and whether replaying it is harmless, where `check-migration-schemas` asks whether a schema change was accounted for. An operation may only act on material some schema version removed — one addressing a path the schema still has would rewrite a perfectly valid file on every run — and it may never name a concrete key beneath an open mapping, whose keys are the user's and unbounded, while the `*` segment that addresses every entry of one is legal exactly there and nowhere else. A `safe` value remap whose old spelling is still legal, or whose target is a free string where staleness cannot be proven, is refused because it would rewrite a deliberate choice; a remap to a spelling the new schema rejects is refused because every migrated file would then fail to load with the tool reporting success. A retired path, or a remapped-away spelling, may never come back. Finally, replaying the whole ledger over each of the surface's two reference documents — the complete packaged configuration and the sparse kit starter template, deliberately different shapes — must apply nothing, report nothing, and hand back the very bytes it was given. It reads checked-in files only and fingerprints no live model, which is what makes it safe to run in the loop agents run constantly: every failure names a file the author wrote.@@ -28,7 +30,7 @@### Changed-- **The configuration root is reshaped along the layers of the runtime (Breaking)**: `pipelex.toml` had no scheme at the top — a grab-bag (`[pipelex]`), a codename (`[cogt]`), the plugin system's own table (`[plugins]`), a rename-hint table (`[migration]`) and a stray `session_id`. It now has one, and it is the split the code already had: `[runtime]` for process-scoped infrastructure (storage, secrets, logging, cloud credentials, reporting, tracing, observation, the plugin denylist), `[inference]` for the model-calling seam (the model deck, LLM, image generation, extraction, plus the default templating style and the shape of dry-run mocks), `[interpreter]` for library-scoped method machinery (MTHDS parsing, pipe runs, pipe functions, pipeline execution, scanning, the builder), and `[kit]` for the dev tooling. The redundant `_config` suffix is dropped from every key at every depth, so `[pipelex.log_config.rich_log_config]` is `[runtime.log.rich_log]` and `[cogt.llm_config.instructor_config]` is `[inference.llm.instructor]`. Class names keep their suffix — the redundancy was in the key, not the type. Every setting keeps its value; only its address changes, and the packaged defaults are now written in order so no root table is split across the file. **Migration:** run `pipelex migrate` after upgrading — the change ships as ledger entry `pipelex-config@2`, which rewrites `~/.pipelex/` and every project `.pipelex/` file in place. Three things are dropped rather than moved: `[migration.migration_maps.*]` (renames are ledger entries now, so the old-to-new name hints in validation errors go away, `.mthds` hints included), `session_id` (process state on the config object that nothing in `pipelex` read), and `plugins.boot_orchestrator`, which becomes a boot argument only — pass `--orchestrator` or `Pipelex.setup(boot_orchestrator=...)`, and read it back at run time from the `runtime_hub` accessor rather than off the config. Code reading the config tree moves with the keys: `get_config().pipelex.storage_config` is `get_config().runtime.storage`, and so on.+- **The configuration root is reshaped along the layers of the runtime (Breaking)**: `pipelex.toml` had no scheme at the top — a grab-bag (`[pipelex]`), a codename (`[cogt]`), the plugin system's own table (`[plugins]`), a rename-hint table (`[migration]`) and a stray `session_id`. It now has one, and it is the split the code already had: `[runtime]` for process-scoped infrastructure (storage, secrets, logging, cloud credentials, reporting, tracing, observation, the plugin denylist), `[inference]` for the model-calling seam (the model deck, LLM, image generation, extraction, plus the default templating style and the shape of dry-run mocks), `[interpreter]` for library-scoped method machinery (MTHDS parsing, pipe runs, pipe functions, pipeline execution, scanning, the builder), and `[kit]` for the dev tooling. The redundant `_config` suffix is dropped from every key at every depth, so `[pipelex.log_config.rich_log_config]` is `[runtime.log.rich_log]` and `[cogt.llm_config.instructor_config]` is `[inference.llm.instructor]`. Class names keep their suffix — the redundancy was in the key, not the type. Every setting keeps its value; only its address changes, and the packaged defaults are now written in order so no root table is split across the file. **Migration:** run `pipelex migrate` after upgrading — the change ships as ledger entry `pipelex-config@3`, which rewrites `~/.pipelex/` and every project `.pipelex/` file in place. Three things are dropped rather than moved: `[migration.migration_maps.*]` (renames are ledger entries now, so the old-to-new name hints in validation errors go away, `.mthds` hints included), `session_id` (process state on the config object that nothing in `pipelex` read), and `plugins.boot_orchestrator`, which becomes a boot argument only — pass `--orchestrator` or `Pipelex.setup(boot_orchestrator=...)`, and read it back at run time from the `runtime_hub` accessor rather than off the config. Code reading the config tree moves with the keys: `get_config().pipelex.storage_config` is `get_config().runtime.storage`, and so on.- **The migration ledger contract is published, and a config-model change now owes it a review**: `docs/migration-ledger.md` was written ahead of the engine it specifies and shipped built-but-unlisted; the commands it describes exist, so it joins the documentation navigation beside the other contributor contracts and loses its status banner. Alongside it, the `config-docs` drift contract's review list gains the migration ledgers and the two modules holding the four validators that narrow a configuration value domain — a narrowing expressed in a validator is invisible to the fingerprint projection the coverage gate reads, so no derived check can ever demand the ledger entry it owes, and a review obligation is the only place that can land. The migration goldens are deliberately not in that list: they are generator output, and their staleness is already `check-migration-schemas`' business.- **Writing a migration entry is a skill, and a release checks one was written**: `.claude/skills/add-migration/` is the only thing that writes a ledger — it derives the entry from what the coverage gate refused rather than from a reading of the diff, bumps the surface's schema version, regenerates the goldens, and adds the changelog bullet. `/release` gains step 3b before the version bump: `make check-migration-schemas` does not run in `make agent-check`, so an unaccounted schema change would otherwise reach a release unseen; the step blocks on it, sends a failure to `add-migration`, and confirms a `**Migration:**` changelog bullet and a correct `introduced_in` for every new breaking entry.- **A stale telemetry configuration is migrated instead of thrown away**: `telemetry.toml` failing to load used to get one answer everywhere — *the format has changed, run `pipelex init telemetry`* — written for the flat pre-`[custom_posthog]` file and then shown for every other way a file can fail. That command writes a fresh file, so on the very case it named it discarded the PostHog key, the Langfuse credentials and the OTLP exporters that the shipped `telemetry-config@2` entry carries forward. The telemetry loader now raises through the same reporting path as every other configuration surface, so its error carries the structured `migration` block and the human CLI, the agent envelope and `pipelex doctor` all say whether the file is *old*, *wrong*, or reported on by a migration that would not repair it — rather than each holding a hardcoded remedy. `TelemetryConfigError` is a `PipelexConfigError`, which is where its `error_domain = config` now comes from — its entry in the agent CLI's domain table is gone as redundant. The `doctor` telemetry row carries a structured `finding` (`healthy`, `not_found`, `unparseable`, `out_of_date`, `invalid`) that every caller branches on, and `out_of_date` is a promise the migration would rewrite that file, so a file it would only report on is `invalid` and keeps the dry run's pointer; writing a fresh file is offered for a missing file only, and regeneration is described as what it is, a way to start over that discards the file. The telemetry models stayed in `telemetry_config.py` and the loader moved into a new `telemetry_loader.py` beside it, because the loader's failure path reaches the migration registry and the registry imports every configuration model.@@ -36,7 +38,7 @@- **The narrowing relation stopped crying wolf on four shapes**: A container whose argument widened (`list[int]` → `list[int | str]`) is read structurally instead of as one opaque string; `int` is absorbed by `float`, which accepts every integer in strict validation as well as lax; `enum` and `literal` absorb each other, because they are two renderings of one thing and what moved between two member *sets* is reported by the enumerated-spelling half; and over an integral type `gt=n` and `ge=n+1` are folded to the same bound, so swapping one spelling for the other stops reading as a tightening. Each was a false breaking verdict on a schema no user's file can tell apart, and the remedy the gate named — bump the version, write an `unsafe` entry — was work with no beneficiary.- **`make up-migration-schemas` left `make up`, and refuses to erase a removal**: Every other regenerator in that aggregate rewrites a derived artifact from a live source, so running them out of habit is harmless; the head fingerprint golden is the proof obligation the coverage gate compares against, and rewriting it after deleting a field erases the very removal the gate exists to catch. It is now run on purpose, and it stops before overwriting a head golden recording a path, an enumerated spelling or a value domain the models no longer have. The refusal names both readings — a real removal wants a bump and an entry; a golden that merely predates a change to the fingerprint format wants the new `make umigf` — because the code cannot tell them apart.- **The fix applier moves and renames a table whole even when a descendant is written in several places**: A `[a.b]` … `[a.c]` … `[a.b.d]` layout stores `a.b` as several chunks under one key, and tomlkit's own header-cache invalidation walks the merged dict facade — one chunk per key — so renaming or moving `a` used to leave the later chunks rendering under the old name: the file came out split between the old name and the new, with an `applied` verdict. The applier now refreshes every chunk beneath a renamed or moved table, so the file re-parses to the same value under the new path, in the original order. This reaches every caller of the applier, the `.mthds` fix loop included.-- **Templating style is now an authoring decision (Breaking)**: How a pipe's inputs are tagged into a prompt is declared on the pipe — `templating_style` on `PipeLLM`, either a bare tag-style string (`"xml"`, `"ticks"`, `"square_brackets"`, `"no_tag"`) or a full `{ tag_style, text_format }` table — with a single runtime default for everything that declares nothing (`[inference.templating].default_templating_style`, shipped as `xml`). It is no longer derived from model metadata, so the same method renders the same prompt shape on every model. **Every prompt rendered for an OpenAI-family model changes shape, from back-tick fences to XML tags.** `PipelexKernel.llm_text` and `llm_object` take the same optional argument.+- **Templating style is now an authoring decision (Breaking)**: How a pipe's inputs are tagged into a prompt is declared on the pipe — `templating_style` on `PipeLLM`, either a bare tag-style string (`"xml"`, `"ticks"`, `"square_brackets"`, `"no_tag"`) or a full `{ tag_style, text_format }` table — with a single runtime default for everything that declares nothing (`[inference.templating].default_templating_style`, shipped as `xml`). It is no longer derived from model metadata, so the same method renders the same prompt shape on every model. **Every prompt rendered for an OpenAI-family model changes shape, from back-tick fences to XML tags.** `PipelexKernel.llm_text` and `llm_object` take the same optional argument. **Migration:** the change left two shapes behind on machines that already had configuration files, and both are now accounted for. Ledger entry `pipelex-config@2` carries `[pipelex.prompting_config]` forward — the default style travels to `default_templating_style` and lands on today's `[inference.templating]` once the reshape entry runs after it, while the per-target `prompting_styles` map is dropped, since a style is no longer chosen per model; the backup beside the file is where the old map remains readable. It is inserted *below* the configuration reshape, which addresses the post-rename spelling and is therefore renumbered from `pipelex-config@2` to `pipelex-config@3`. The per-model half of the same change is `inference-backend@2`, above. Running `pipelex migrate` once applies whichever of them your files need.- **Compose, image-generation, search and structuring prompts follow the same default (Breaking)**: These paths previously rendered style-less, which the Jinja2 filters silently absorbed as triple-backticks and plain text. They now resolve the same runtime default as the LLM path. Any template of yours that uses `| tag` — including `PipeCompose` templates and construct-mode template fields — changes shape accordingly, whichever model family you run on.- **Templating filters are strict**: The `tag`, `format` and `with_images` Jinja2 filters no longer fall back to a built-in style when the rendering context carries none; a missing tag style or text format raises `Jinja2ContextError`. Every prompt-rendering entry point now supplies a resolved style, so a missing one is a bug rather than a shape nobody chose. An explicit filter argument (`| format("markdown")`) is unaffected, and an unknown format name is now reported as a template error instead of a bare `ValueError`.- **Reasoning budgets are worker-owned**: The thinking-budget lookup is keyed by a family constant on the worker class rather than read from the model spec's `prompting_target`. Behavior is unchanged; the budget maps keep their `anthropic` and `gemini` keys.@@ -56,6 +58,9 @@- **Renaming a table written in dotted form no longer swallows the keys after it**: `k.x = 1` and a `[a.k]` header say the same thing to a TOML parser and are two different things to a TOML document, and which one a key is is a flag the parser sets rather than a property of its name. Rebuilding the key from its name alone — what tomlkit's own re-key primitive does — lost that flag, so a renamed dotted key came back out as a block header, and a block header absorbs every scalar that follows it in the same table: a neighbouring `m = 3` silently became a key of the renamed table, with an `applied` verdict and no diagnostic. Renamed at an inner segment the whole chain re-rendered at the document root, taking the subtree out of the table it lived in. The applier now renames the document's body entries where they sit, carrying the flag forward and covering every chunk of a key written on several dotted lines — which also stops a cosmetic blank line being injected after any table it renames. This reaches every caller of the applier, the `.mthds` fix loop included.- **A migration no longer replaces a symlinked configuration file with a regular one**: a file symlinked out of a dotfiles directory is now read, backed up and replaced at the end of the link, the way the `.mthds` fix loop already resolves its own targets — the link survives and the real file is the one that gets migrated. Replacing the link path instead put a regular file where the link was and left the real file unmigrated.- **A migration run no longer takes away a backup it did not make**: the backup stamp resolves to the second, so two runs of the same file can address the same name. A run now reserves the name atomically and leaves whatever is already there — an older copy of the file, which is to say the original — instead of overwriting it, and a run whose write is refused deletes only the copy it made itself. A copy kept because a write could not be confirmed is renamed out of the `.bak.` rotation into `.rescue.`, so the next successful run cannot prune away the very file the report told the user to go and get — and when that copy cannot be moved, because another run made it or the rescue name is taken, the report says it is still in the rotation and asks the user to take it now instead of promising it will keep; and the backup's directory entry is `fsync`-ed before the target is replaced, so "back up first, replace second" holds across a power loss and not only a process exit.+- **A backend file that is both out of date and wrong now says both, and stops offering to reset itself**: The boot builds its own message for the two libraries that refuse after the main configuration is already up — the inference backend library and the model deck — and it named no migration surface for either, which was right while neither had a ledger. Now one does, so a backend file the ledger only half explains never reached boot tolerance's warning and was reported on its unknown key alone, while the very same staleness in `pipelex.toml` produced the full migration block. The component now carries its own surface, and the classes that mean *the backend library's files are not loadable* are likewise named once — both stated beside each other rather than at each `except` clause, since a clause naming one class of a family is a hole rather than an error, and the scan is only reachable if the boot catches what the loader really raises. The refusal reaching that message need not carry a pydantic error at all, which is the second shape a backend file refuses in: a per-model key rejected by name for not being header-shaped arrives with no cause, and the surface is asked about the file either way. And the refusal's own sentence is now kept whole rather than reduced to the pydantic analysis behind it: the loader says *which model, which backend, which file* before it quotes the analysis, and a message cut down to `max_tokens: Input should be a valid integer` sent the reader to grep a directory of files that all have a `max_tokens` — the model deck's message, which names the deck's paths the same way, keeps them now too. In the same change, a message carrying a migration block stops appending *or run `pipelex init config` to regenerate them, which resets all config files to their default values*: the block has just promised that `pipelex migrate` keeps every value the file holds, and whichever remedy the reader followed, the other sentence made it the wrong one. The model deck, which has no ledger and gets no block, keeps that sentence — for it, it is the only answer there is.+- **A bad value in `backends.toml` is reported as a configuration error, naming the backend and the file**: A backend table whose own fields fail the blueprint — `endpoint = 42` — left the library loader as pydantic's bare `ValidationError`, which no clause in the boot named, so it reached the user as a traceback that named a field and neither the table it sat in nor the file. It is now raised as the library's own validation error, said with the backend and the file in front of the analysis, and handled by the boot like every other refusal of that library. The clause that used to raise that class wrapped a TOML read that never raises a pydantic error, and is gone.+- **`pipelex_gateway` with a non-boolean `enabled` no longer disappears from the deck**: The boot decides whether to fetch the Pipelex Gateway's model specs by reading `backends.toml` itself, and it read `enabled` as *is the literal `true`* while the library loader read the raw value's truthiness — so `enabled = 1` fetched no specs for a backend the loader then loaded, which a strict boot refused as *"remote model specs were not provided"* and a lenient one silently dropped from the deck. Both readers now read the value the same way.- **A blocked configuration file now says which state it is in**: `FileBlockedReason` gained `unreadable` and `state_uncertain`, so an unreadable file is no longer reported as *unwritable* and a write whose outcome the transaction could not describe is no longer reported as one that never happened. The wording of that last case described an unreachable path — for the single-file commit a migration performs, a failed replacement re-raises its own error, so the only transaction failure that reaches a plan is the one raised after the target was replaced.### Removeddiff --git a/TODOS.md b/TODOS.mdindex 1a703391a..346afe130 100644--- a/TODOS.md+++ b/TODOS.md@@ -1,286 +1,382 @@-# Pipe-refs build-time qualification — implementation plan+# S7b — the inference-backend surface: implementation plan-Source of truth for the design: [wip/pipe-refs/build-time-qualification.md](wip/pipe-refs/build-time-qualification.md) (decided 2026-08-11). Evidence and pre-change state: [wip/pipe-refs/README.md](wip/pipe-refs/README.md). Venue: the `_refs` worktree — treat this worktree as the repo root.+> **Charter:** [`wip/migrator-3/sequencing.md § S7b`](../wip/migrator-3/sequencing.md) (workspace root). **Substance:** [`wip/migrator-3/backend-config-surface.md`](../wip/migrator-3/backend-config-surface.md). **Ruling:** R10 (2026-08-17) — build the real fourth surface, not a doctor row and not an `update` bolt-on. Both documents were read in full when this plan was written (2026-08-17); this file orders the work, they hold the reasoning. When they disagree with what the code turns out to do, the code was measured later — record the disagreement in the Status block and in the substance document's *Decisions owed*, do not silently pick one.+>+> **Venue:** this worktree, `_backend/`, on `feature/Backend-surface`, cut from `dev` = `61c4786b6` (the reshape merge). Treat this worktree as the repo root; paths below are relative to it. `pipelex/` (the sibling checkout) stays on `dev` and is not touched. Nothing runs alongside S7b.+>+> **Why it blocks the release, in one sentence:** the reshape release is the first to carry `#1104`, which deleted `prompting_target` from the model-spec blueprint and from every backend TOML we ship — but `pipelex init` never overwrites, so the key survives in `inference/backends/*.toml` of every existing installation, where it is fatal twice (`extra_forbidden` from `[defaults]`; `NOT_HEADER_SHAPED`, fatal even leniently since `#1109`, from per-model entries), and no surface claims those files, no walk reaches them, and the coverage gate never asks because `#1104` predates the schema-1 cut. The remedy must ship *in* the release that breaks the file, since a user runs `pipelex migrate` after upgrading.-**What we are doing in one sentence:** bare in-body references (pipe steps, branches, outcomes, batch refs, concept I/O refs) get qualified to their owner domain once, at library build time, via one shared crate pass consumed by both the normalizer and `library_manager`; the live library lookup becomes strict; the crate-wide bare search survives only as an explicitly-named entry-point affordance for user-supplied codes.+**What we are doing in one sentence:** make `<config dir>/inference/backends/*.toml` a real migration surface (`inference-backend`, registry + ledger + goldens + boot tolerance), ship its first entry — a `pre_history` `delete_key` of `prompting_target` over every root table — and, in the same PR, account for the `pipelex.toml` half of `#1104` with a `pre_history` entry inserted as `pipelex-config@2`, renumbering the reshape to `@3`.-**Why it matters beyond tidiness:** no-fall-through is what makes `[exports]` enforceable. A visibility rule the resolver can reach around is not a visibility rule. Keep that argument at the front of every discussion about this change — it is the reason the blast radius is worth paying.--## The resolution rule, in one picture+## The shape of the change, in one picture```-in-body ref encountered- |- +-- contains "alias->" ? --YES--> cross-package: pass leaves it ALONE- | (resolved against the child library)- NO- |- +-- already qualified "d.c" ? --YES--> unchanged (pass is idempotent)- |- NO (bare code "c")- |- v- qualify to the OWNER domain of the declaring pipe --> "own_domain.c"- |- | existence is NOT checked here- v- ordinary dependency validation- |- +-- resolves ? --YES--> done- |- NO- v- error names the ref that was tried, and suggests a crate-wide- candidate when one exists (diagnostic scan, FAILURE PATH ONLY —- this is not a resolution fallback)+today after S7b+───── ─────────+registry: 3 surfaces, claim by file NAME registry: 4 surfaces, claim by (subdirectory, name)+walk: root dir only, iterdir, not recursive walk: each surface's own dir = root / surface.subdirectory+ ~/.pipelex/pipelex.toml pipelex-config ~/.pipelex/inference/backends/openai.toml inference-backend+ ~/.pipelex/telemetry.toml telemetry-config ~/.pipelex/inference/backends/pipelex_gateway.toml+ ~/.pipelex/pipelex_service.toml service-config ↑ matches pipelex_*.toml, must NOT go to pipelex-config+ (the landmine; test first)+ledgers ledgers+ pipelex-config@2 = reshape pipelex-config@2 = pre_history: prompting_config → templating_config+ telemetry-config@2 = pre_history pipelex-config@3 = reshape (renumbered)+ telemetry-config@2 = pre_history (unchanged)+ inference-backend@2 = pre_history: delete_key ["*"] prompting_target+boot: 3 loaders warn on a stale file boot: 4 loaders — InferenceBackendLibrary.load joins them```-Entry-shaped lookups (a human typing a pipe code at a CLI) do NOT follow that rule. They go through the entry affordance, which searches crate-wide, refuses ambiguity, and deliberately ignores `[exports]` — a hand-invoked pipe is not an in-body reference, so package visibility does not apply.-## Rules of engagement-- **Checkpoints are mandatory stops.** At each ⛔ CHECKPOINT below, the agent MUST stop working, complete the checkpoint checklist (status updates, gates, commit), and end the session. Do not roll into the next phase in the same session, even if context feels fresh. The next session cold-starts from this file's Status block plus the design doc.-- **Update this file as you go.** Tick checkboxes when items complete; record decisions in the Status block the moment they are taken, not at the end.-- **Open questions from the design** are folded into the phase tasks below as explicit decision items (marked OQ1/OQ2/OQ3). Record each ruling in the Status block and in the design doc. OQ1 is a go/no-go gate in Phase 0, not an implementation task.+- **Checkpoints are mandatory stops.** At each ⛔ CHECKPOINT below the agent MUST stop working, complete the checkpoint checklist (status update, gates, commit), and end the session — even if context feels fresh. The next session cold-starts from this file's Status block, the charter and the substance document. Two of the checkpoints are also **(Louis)** goes: the design ratification at CP1 and the merge at CP4.+- **Update this file as you go.** Tick checkboxes when items complete; record decisions in the Status block the moment they are taken, not at the end. The four *Decisions owed* (D1–D4 below) each get a one-line ruling in Status **and** in `wip/migrator-3/backend-config-surface.md § Decisions owed` (workspace repo — edit it, say so in the hand-off, do not commit it; that is Louis' call).+- **TDD.** Every behaviour below is written as a failing test first (red), then made green. Mutation-test the tests that guard a rule (break the rule, watch the test go red) — the collision test and the "no walk into `deck/`" test in particular.+- **Verify, don't assume.** The workspace has watched premises die on re-measurement. Phase 0 re-measures the breakage on this laptop before a line is written.+- **The tool migrates this laptop, not the agent.** `~/.pipelex/inference/backends/` is a genuine stale specimen and is the first-machine evidence for S7's release step. **Do not hand-edit it, do not `pipelex init --reset`, do not run `pipelex migrate` for real (non-`--dry-run`) on it during this session.** Read-only probes only. If a value from it is needed in a fixture, hand-author the fixture from the file's table headers — never copy a personal file into the tree.+- **A ledger says what *we* removed.** The backend entry deletes `prompting_target` and nothing else. It must not touch `pipelex_gateway.toml`'s *served* specs (they are not in the file) and must not try to repair a user's own unknown per-model key (that is `#1109`'s rejection, and its message already names file, key and fix).+- **Rules from the memory index that bite here:** `git checkout` restores from the INDEX and deletes unstaged work — never "restore" with git mid-loop; `make umig` on an unchanged head is a no-op, and a `umigf` is only for an unreleased format change; drift-check reads the index (stage first); a new error class needs `make gei` + `make gep` and only the **full** `agent-test` catches it; a type projection cannot decide schema membership; a generated artifact (the goldens) is never a drift review target; record a subject grant BEFORE the first `agent-check` on a new module.### Gates — the full checklist, run at every checkpoint-`make agent-check` and `make agent-test` do not cover everything this plan disturbs. Three of the gaps fail silently, which is why this is a checklist and not a sentence.+`make agent-check` and `make agent-test` do not cover everything this plan disturbs.1. `make agent-check` clean.-2. Full `make agent-test` green.-3. **`git add` the work, THEN `make drift-check`.** `agent-check` does not run `drift-check`, so a contract never opens "along the way" under the other gates. And `drift-check` reads the git *index*, so unstaged work reads as a false green. If a contract opens, resolve it via the drift-review flow — no bypassing.-4. **Reconcile `.test_durations`** after any test file, class, or function is renamed or repurposed. This plan repurposes named tests in both the normalizer and lookup suites. Orphan node ids are silent — pytest-split treats an unknown id as average duration. Collect with `.venv/bin/pytest --collect-only -q -m ""`; the marker filter in `addopts` hides a large slice of the suite (mostly `tests/e2e`) and makes every e2e entry look like an orphan.-5. **Record the subject grant BEFORE the first `agent-check` on the new module.** `agent-check` runs `fix-keyword-only`, which silently rewrites an ungranted positional subject to keyword-only. If the pass is meant to read as a verb–object sentence (`qualify_crate(crate)`), `make subject-grant` has to land first or the shape is gone before anyone decides.+2. `make cl` (`check-ledger`) and `make cmig` (`check-migration-schemas`) green — `cmig` is **not** in `agent-check`; it is in `make check`. Run both by name.+3. `make tb` (boot smoke) green — the registry imports the real config models, and a fourth surface that cannot build its defaults layer fails here first.+4. Full `make agent-test` green — this is what catches a new error class needing `make gei` / `make gep`, and a renamed test leaving `.test_durations` stale.+5. **`git add` the work, THEN `make drift-check`.** `drift-check` reads the git *index*; unstaged work reads as a false green. Touching `pipelex/system/configuration/config_surface.py` or `config_loader.py` opens `config-docs` (review: `docs/configuration/**`, `pipelex/migration/ledgers/`, `configs.py`, `config_cogt.py`); touching `pipelex/cli/commands/init/*.py` or `doctor_cmd.py` opens `cli-docs`. Resolve via the drift-review flow (`make drift-plan`, review for real, `make drift-ack CONTRACT=… RATIONALE="…"`); no bypass. Nothing in `drift.toml` triggers on `pipelex/migration/**` or `pipelex/cogt/model_backends/**` — so those edits open no contract, and their docs (`docs/migration-ledger.md`, `docs/configuration/config-technical/inference-backend-config.md`) must be updated by discipline, not by a gate.+6. `make docs-check` (strict mkdocs build) after any `docs/` edit.+7. Reconcile `.test_durations` after any test file/class/function rename (`.venv/bin/pytest --collect-only -q -m ""` to list ids).+8. `make check` once before the PR (it is what CI runs; it includes `cmig`, `openapi-check` and the drift gate).## Status-- **Current phase:** Phase 4a item 1 DONE (2026-08-13); 4a item 2 (transport-boundary spec edit) deliberately HELD to land together with its 4b verifying conformance test. **Phase 4b is blocked on user action:** merge PR #1099 (`fix/Pipe-refs-3` → dev), then cut a `pipelex` release carrying the `interpreter_hub` entry-pipe accessors — every 4b item needs that release installable by the sibling repos. The transport spec/test pair to touch in 4b: `docs/specs/pipelex-transport-boundary.md` §"The allowed import surface" (workspace root) + `conformance/tests/pipelex_transport/test_provider_surface.py` (the allowed-surface list lives in `test_data.py::TransportBoundaryContract.ALLOWED_SURFACE`; the workspace CLAUDE.md note claiming `conformance/tests/pipelex_transport/` does not exist is stale — the suite exists and runs).-- **Last completed item:** Phase 4a item 1 — the `mthds/` spec clarification (details on the 4a checklist line). Before that: C3 — the concept side landed as the `get_optional_entry_concept` / `get_required_entry_concept` pair with a package-aware `search_scope` (the entry pipe's domain, `alias->domain` for a dependency entry pipe) replacing the `search_domain_codes` list across the whole public chain; both README §4 latent defects deleted structurally; `fix_loop` rename collisions relaxed to per-domain. §4 marked HISTORICAL with a re-measured matrix (probe rewritten against the new affordance).-- **Phase 3 rulings.** (1) **Concept ambiguity raises, naming the candidates — but only after the scope preference has had its chance.** The old own-domain `insert(0)` was dead code; the preference now fires *before* the crate-wide scan. (2) **Both refusals (miss and ambiguity) raise `ConceptLibraryConceptNotFoundError`** — the class the `stuff_factory` handlers actually catch, which is what deletes the wrong-exception-class defect rather than patching around it; the four handler messages now say "could not be resolved: {exc}" so an ambiguity is not mislabeled "not found". (3) **The concept twin of OQ1 was paid in full here, unlike the pipe side** — the concept lookup is entry-only and its one call site holds the alias, so package scope is a string prefix, not an execution-path threading project. (4) The `search_domain_codes` parameter is deleted from `PipelexMTHDSProtocol` (public, breaking) rather than kept as an override — nothing supplied it in production and a caller-supplied list is exactly the contextual instability this branch removes.-- **Phase 3 surprises.** (1) Two integration tests pinned the crate-wide `fix_loop` collision scope and had to flip — the parametrized other-domain arm of `test_strip_namespace_bails_on_cross_file_collision` asserted a bail that is no longer correct behavior, and the rebuild-between-rounds pin only kept its purpose by moving to a same-domain fixture. (2) The stuff-factory param-id entries in `.test_durations` changed shape twice over: the rename changed the *argname* ids (`search_domain_codesN`), and the list→str change made the new ids embed the literal value (`test_domain`) instead of a positional placeholder — the reconciliation needed the full `-m ""` collection to map them. (3) `mocker.patch.object(module, "prepare_pipe_job", wraps=...)` on the run-setup module is enough to pin the scope wiring without touching the seam itself — the wiring mutation reddens both that pin and the end-to-end test.-- **Decisions taken:** phase structure reworked so no checkpoint rests on a self-contradicting tree; OQ1 promoted to a Phase 0 go/no-go gate; breaking-change posture held (no deprecation period) but preceded by a bounded corpus measurement. **OQ1 ruled GO** — owner-domain qualification is neutral-to-better for dependency packages; the dependency lookup-scope defect it exposed is pre-existing and deferred to the packaging project (`wip/pipe-refs/dependency-subpipe-scope-deferred.md`). **Crate envelope ruled: the pass does not deal in crates at all** — it returns a `QualifiedCrateContent` (just `concepts` and `pipes`), never a `LibraryCrate`, because it cannot pick the right digest scheme for a `fingerprint`. Each consumer makes the fingerprint correct at its own boundary (rationale in the design doc's Phase 1 outcome). *(This bullet previously read "the pass is envelope-preserving", describing the first cut that returned a crate; review changed the return type and the wording was stale in three places until a PR bot caught one of them.)*-- **Surprises / fallout:** (1) the dependency execution path has **no** child-library scoping at all — a dependency package using a controller internally either fails to load or silently binds a host pipe of the same bare code; the child-library special case in `Library.validate_pipe_library_with_libraries`'s first loop is neutralized by `pipe.validate_with_libraries()` in the second. Worse in the shape a package actually ships: with a manifest `[exports]`, the authored helper is dropped from the dependency's own child library and the load fails earlier still. (2) README §5's corpus scan used a hardcoded repo list that reached under a fifth of the bundles outside this repo; the widened scan is in `wip/pipe-refs/corpus-measurement.md`, and the second breaking reference lives in `cocode` — a shipped CLI, not an examples directory.-- **Open risks carried in:** four latent defects surfaced by the outside-voice pass are folded into Phases 1–3 below (batched sub-pipe code derivation, transported `PipeFunc` bare codes, the crate fingerprint contract — now ruled, aliased dependency keys polluting the crate-wide search). None of them is speculative; each names a file.-- **Phase 1 findings that change how Phase 2 must be tested:** (1) **this repo's entire bundle corpus is blind to the rule flip** — applying Phase 2's owner-domain rule to all 54 crates buildable here produces byte-identical output, so the two-domain fixture is not a nicety and the mutation check is what makes the resolution-row tests mean anything; (2) **a differential harness whose inputs each carry at most one defect cannot see an error-precedence change** — that is how Phase 1's own precedence shift got past it and had to be caught in review; Phase 2 moves far more error surface, so its equivalence inputs need multi-defect crates; (3) **a fixture full of bare refs proves nothing unless a test reads a ref back out of the result** — Phase 1's first test module passed intact against a no-op pass for exactly that reason; (4) nothing writes a normalized crate back over a `.mthds` file, so the flip does not rewrite user source — but the stamped `codegen.lock` `crate_fingerprint` moves on every committed projection; (5) `_qualify_io_ref`'s cross-package guard looks unreachable through a constructed blueprint (the io-ref validator rejects `alias->` refs outright) — preserved verbatim, flagged for whichever phase owns cross-package forms.-- **Phase 2 rulings.** **OQ2 — the entry affordance is a `get_optional_entry_pipe` / `get_required_entry_pipe` pair** on `PipeLibraryAbstract` + `PipeLibrary`, mirrored as hub accessors. A pair rather than one method because the migration found exactly one caller (`pipelex which`) that needs the optional form and eleven that want the required one; "entry" rather than the working name `find_pipe_by_bare_code` because it also takes fully-qualified refs, and because the name should say *why* the lookup differs, not *how*. **OQ3 — the cross-package bare-remainder search stays.** The qualification pass leaves `alias->…` refs alone (it cannot know a dependency's domain layout), so deleting the search would break every `alias->bare_code` ref with no canonical spelling to migrate to. It is alias-scoped and cannot reach a host pipe; the comment in `get_optional_pipe` says so and points at the packaging design. **DRY — the pipe and concept affordances stay separate**, written up with the reasoning in [wip/pipe-refs/entry-affordance-share-vs-duplicate.md](wip/pipe-refs/entry-affordance-share-vs-duplicate.md) so Phase 3 inherits a decision rather than making an accidental copy. The *rule* is shared (one `qualify_crate`); the affordance is the deliberate exception to it, and the two should not be conflated.-- **Phase 2 surprises.** (1) **The batched sub-pipe defect was real but not what the review said.** The plan (from Codex) predicted the invalid code `domain.foo_batch`; in fact `PipeAbstract` has a `code` validator that strips a namespace prefix, so the built ref was correct all along. What the wrong derivation actually produced was a `log.warning` on **every** batched run, about a code no user wrote. The fix stands; the first test written for it asserted on the ref and passed against the bug, and had to be rewritten to assert on the warning. (2) **Two call sites the plan did not enumerate** turned up by running the suite rather than grepping: `validate_bundle.py` resolved a bundle's own `[pipe.<code>]` keys as bare codes (now qualified with the bundle's domain — precise, not routed through the affordance), and `dry_run_in_process.best_effort_graph_spec` resolved a caller-supplied graph target (entry-shaped, now on the affordance). The second silently degrades to "no graph" on a miss, so it would never have failed loudly. (3) **`missing_pipe_code` changed value** — a structured, machine-read field, so a wire-visible change, recorded in the changelog. (4) The test sweep's own derivation over-reached twice and both were caught by verification rather than by the suite: a `.pipe_code ==` assertion resolved against the wrong source (that field is the *referring* pipe's local code and does not change), and a ref inside a `PipelexBundleBlueprint` fixture was hand-qualified when the product qualifies it — the new break message diagnosed that one itself.-- **Phase 2 PR review (cubic, 9 findings; greptile clean).** Two were real defects and both are fixed: the `SpecialOutcome` exemption applied to *every* controller kind, though `continue` / `fail` are legal snake_case pipe codes — so a sequence step naming a pipe called `continue` was left unqualified, which the strict lookup then fails to resolve (a Phase 2 regression: the fall-through used to rescue it); and `make-demos.sh`'s `mktemp -d -t` is BSD-only, so the README's own reproduction command aborted on Linux before writing a demo. **Four findings were claims of mine that had gone false**, which is the half worth remembering: the `python_sources` deferral credited `model_copy` for envelope preservation (true of the pass's *first* cut, which returned a crate — the same stale claim was in three places, and the bot found one), "no fix op writes a pipe ref" is wrong (`strip-namespace` plans a root `main_pipe` SET_KEY), and three line-number pointers had gone stale *inside this branch* between Phase 1 and Phase 2. Those were fixed by **removing** the line numbers, not correcting them — a pointer into code the same branch edits re-stales itself every phase.-- **The `_domain_of` / `CrateNormalizationError` debt, split and settled.** Phase 1 left two things wrong-sounding once `library_manager` became a second caller of the pass, and parked both on Phase 2. **Only one of them was Phase 2's to pay**, which the plan did not say clearly enough — a PR bot caught the bullet still assigning both to a phase declared complete. Settled: the **message** is fixed here (it now says "this pass expects…" rather than blaming the normalizer, which is no longer even the busier caller). The **error-class rename is deliberately re-deferred to Phase 4b**, where it belongs: `error_type` is a wire string, so a rename needs `make generate-error-identity`, `make generate-error-pages`, and the TypeScript-consumer sweep — all of which are already Phase 4b's "error-surface sweep" line items. Doing it here would open a cross-repo surface inside a runtime-only phase.-------## Phase 0 — answer the questions that can invalidate the design--Nothing here writes product code. Both items exist because their answers can change what gets built, and both are cheaper now than later.--- [x] **OQ1 — dependency-pipe lookup scoping (GO/NO-GO).** **Answered: GO.** Measured with `wip/pipe-refs/probes/dep-subpipe-scope.py` across three package shapes. Execution consults the **host** library unconditionally, and its bare-code fall-through skips `alias->` entries — so a dependency's own bare sub-pipe ref never reaches the dependency's own pipe *today*: a plain host fails to load, a host declaring the same bare code loads and silently binds the host's pipe, and a dependency with a real `[exports]` manifest fails earlier still because the export filter drops the authored helper from the child library. Owner-domain qualification does not regress that (the qualified ref resolves to `None` in the host exactly as the bare one does), converts the silent capture into a deterministic not-found, and makes the eventual package-scoped lookup a direct key hit in the child library. The pre-existing defect is written up in `wip/pipe-refs/dependency-subpipe-scope-deferred.md` and deferred to the packaging project.-- [x] **Measure the fall-through corpus.** **Two bare pipe refs break across every `.mthds` tree in the workspace**, each fixed by one qualified spelling. Recorded with the full table, the reproduction command and the denominator's caveats in `wip/pipe-refs/corpus-measurement.md`. README §5's scan used a hardcoded ten-repo list and missed most of the workspace; enumerating instead multiplies the corpus several times over and finds a second `sibling-only` case in `cocode` — a shipped CLI, not a sample. The `sibling-only` *concept* refs found are vscode-pipelex editor fixtures that already fail to load today, so they are not new breakage.--### ⛔ CHECKPOINT C0 — MANDATORY STOP--- [x] Record the OQ1 ruling and the corpus number in the Status block and the design doc.-- [x] If OQ1 came back negative, do not proceed — write up what broke the premise and end the session for a design conversation. *(Came back positive — proceeding.)*-- [x] Commit the investigation notes.-- [x] STOP. Next session starts at Phase 1.-------## Phase 1 — extract the pass (pure refactor, behavior-identical)--This phase moves code and changes nothing. That is the point: the checkpoint at the end is the safest handoff state in the whole plan, provable by an unchanged test suite. The rule flip does not happen here.--- [x] Read the design doc and README end-to-end; skim `pipelex/libraries/crate_normalization.py` (`_qualify_concept_ref`, `_qualify_pipe_ref`, `_qualify_io_ref`, `_normalize_pipe`) and `pipelex/libraries/library_crate_factory.py` to confirm the insertion point still matches the design.-- [x] **Record the subject grant first if the pass takes a positional subject** (see Gates item 5). Done — nine grants recorded before the first check. The `PostToolUse` hook blocked the write and named every ungranted subject, so the auto-fixer never got the chance to rewrite them; the gate's hazard is real but the hook catches it at edit time, not only at `agent-check` time.-- [x] Extract the in-body reference qualification step out of `crate_normalization.py` into a standalone pass over a `LibraryCrate`. Landed as `pipelex/libraries/crate_qualification.py::qualify_crate`. Covers pipe refs (steps, branches, outcomes, batch refs), concept I/O refs, `refines`, and structure-field refs. **Rule unchanged** — crate-wide search stays.-- [x] Make `normalize_crate` consume the extracted pass. No test flips. **Behavior-identical with one known, deliberate exception — error precedence between two independent defects.** HEAD's implementation was loaded beside the working tree's and both run over the same inputs, comparing full serialized output: identical across every crate buildable from this repo's `.mthds` files and across ten hand-built crates covering the branches the corpus cannot reach. The comparison was mutation-checked so its green means something: dropping structure-field qualification reddens one case, applying the Phase 2 rule reddens three.-- ⚠ **The exception, and how it was missed.** Qualification now runs as one complete phase ahead of refinement flattening, where HEAD interleaved them (concepts → flatten → pipes). A crate carrying *both* a refinement cycle and an ambiguous bare pipe ref used to report the cycle and now reports the ref. **The differential harness could not see this: every crate it compares carries at most one defect.** A single-defect corpus cannot detect a precedence change by construction — the finding came from review, not from the harness, and that is the lesson worth carrying into Phase 2, where far more error paths move. Accepted rather than fixed: restoring the old order would mean splitting the pass in two so the normalizer could interleave it, and Phase 2 runs the pass with no flattening at all. Both errors are `CrateNormalizationError`, the crate is invalid either way, and which of two independent defects is named first is not a contract. Recorded in `normalize_crate`'s docstring.-- [x] **Inventory every consumer of normalized crate output before the flip phase.** Recorded in [wip/pipe-refs/normalized-crate-consumers.md](wip/pipe-refs/normalized-crate-consumers.md). **The feared write-back does not exist** — no path writes a normalized crate back over a `.mthds` source file. `pipelex fix` rewrites user files but never through a crate, and no fix op writes a pipe ref (re-check in Phase 2 rather than assume). What does move is the stamped `codegen.lock` `crate_fingerprint` on every committed projection.-- [x] **Settle the crate envelope / fingerprint contract.** **Ruled: the pass does not deal in fingerprints at all** — it returns a `QualifiedCrateContent` (the rewritten `concepts` and `pipes`), not a `LibraryCrate`. Returning a crate would force it to supply a `fingerprint` it cannot get right, since a crate does not record which digest scheme produced its value; the pass would have to guess or carry over a digest that no longer describes its own content, contradicting `LibraryCrate.fingerprint`'s own docstring. Returning the two mappings makes that unrepresentable and costs the callers nothing — both already hold a crate to read the envelope from, and `normalize_crate` builds a fresh one anyway. Phase 2's `load_from_crate` keys idempotency on the incoming crate's fingerprint, which is what external callers pass to `is_crate_loaded`. *(First cut returned a crate with the envelope carried over and a paragraph excusing the stale digest; review pushed back and the smaller surface is plainly better — a paragraph of excuse is a design smell.)* The related `python_sources` drop in `normalize_crate` is real, untouched here, and deferred in [wip/pipe-refs/normalize-crate-drops-python-sources-deferred.md](wip/pipe-refs/normalize-crate-drops-python-sources-deferred.md) — the one-line fix moves customer Python source into a CLI stdout stream and an HTTP response body.-- [x] Gates checklist clean (all items).--### ⛔ CHECKPOINT C1 — MANDATORY STOP--A pure refactor has landed. The tree is fully self-consistent and the suite is unchanged — this is a genuinely safe place to walk away from.--- [x] Update the Status block (phase done, the consumer inventory, the fingerprint ruling).-- [x] Record status in the design doc's Phase 1 section.-- [x] Commit with a clear message. ⚠ **This item's own wording had to change:** it said "say plainly that this commit changes no behavior", and that turned out not to be true. The message states the extraction is behavior-preserving *with one documented exception* and names it. An instruction to assert a property is not a licence to assert it unverified.-- [x] Gates run at the checkpoint: `agent-check` clean, full `agent-test` green, `drift-check` passed against the staged index, `.test_durations` N/A (path-existence gate; nothing renamed or moved), subject grants recorded ahead of the first check.-- [x] Reviewed by a no-context `/code-review` fan-out; findings triaged and verified individually rather than deferred to. Three of the reviewer's claims held up, two against claims I had written down myself.-------## Phase 2 — flip the rule and make the lookup strict (these land together)--The normalizer and the live library must never disagree in a committed state, so the rule flip and the strict lookup are one phase. This is the phase that justifies the branch.--### The rule--- [x] Switch the pipe rule inside the pass from crate-wide search to **owner-domain qualification** — the exact twin of `_qualify_concept_ref`. Existence is NOT checked by the pass; the ordinary dependency validation catches missing refs afterwards (that is the standard's step 3 by construction).-- [x] Leave cross-package refs (`alias->…`) untouched by the pass, exactly as the normalizer already does.-- [x] **Fix the batched sub-pipe code derivation.** `SubPipe.pipe_code` becomes `domain.foo`, and the runtime derives `f"{self.pipe_code}_batch"` (`pipe_batch_adhoc_pipe_code` in `pipelex/pipe_controllers/sub_pipe.py`). Use the resolved sub-pipe's local code for that derivation. ⚠ **The predicted consequence turned out to be wrong** — see the Phase 2 surprises above: `PipeAbstract`'s `code` validator strips a namespace prefix, so the built ref was correct all along and what the bug really produced was a warning on every batched run.-- [x] Run the qualification pass in `pipelex/libraries/library_manager.py` on **all** crate→pipes paths — main load, secondary loads, dependency-package loads — before pipes are constructed via `PipeFactory.make_from_blueprint`.--### The strict lookup and the entry affordance--- [x] Strip the step-3 bare-code crate-wide fallback from `PipeLibrary.get_optional_pipe` (`pipelex/libraries/pipe/pipe_library.py`), including the `domain_hint` TODO. What remains: direct key lookup and cross-package (`alias->`) handling. `get_required_pipe`'s signature does not move (transport-pinned).-- [x] **OQ3 — decide** whether `get_optional_pipe` keeps the cross-package bare-remainder search (`alias->` + bare code). It is alias-scoped so it does not reopen the cross-domain hole; the design leans toward keeping it until the packaging design rules on cross-package reference forms. Record the ruling, and pin it with a test either way.-- [x] Close [wip/parity/d1-domain-hint-deferred.md](wip/parity/d1-domain-hint-deferred.md) as subsumed by this change (status note in that file).-- [x] **OQ2 — settle the entry affordance's final name and shape** (working name `find_pipe_by_bare_code`; one method or a required/optional pair; where the ambiguity error message lives).-- [x] Implement the entry affordance on the pipe library (+ its abstract): exact ref hits directly; a bare code matches crate-wide; ambiguity raises an error asking the user to qualify. Docstring states explicitly that it deliberately does NOT consult `[exports]` — a hand-invoked pipe is not an in-body reference, so package visibility does not apply.-- [x] **Exclude aliased dependency keys from the crate-wide bare search.** `PipeLibrary.root` also holds `alias->…` dependency entries (added by `add_dependency_pipe`), so "crate-wide" is not "every value in root". Without this exclusion, installing an unrelated dependency package can make a host pipe ambiguous — reintroducing, through the new affordance, exactly the contextual instability this branch exists to eliminate. Applies to the concept affordance in Phase 3 too.-- [x] **Decide and record the share-vs-duplicate question** between this pipe affordance and the concept affordance built in Phase 3. They are near-twins — the concept one adds a domain-preference step, and its spec currently stops at "unique match" without saying what happens when the match is not unique. Decide now whether they share a helper or stay deliberately separate, and settle whether the concept one raises on ambiguity. A written "separate, because X" is a valid answer; an accidental copy discovered in Phase 3 is not.-- [x] Expose the affordance through a new `pipelex/interpreter_hub.py` accessor (additive; the transport spec/conformance update for it happens in Phase 4b).-- [x] Migrate the in-repo entry-shaped call sites to the affordance: the CLI commands (`which`, `show`, `validate`, the `build`/`codegen` groups), `pipeline_run_setup`, and the builder operations. Controllers, `sub_pipe`, `signature_walk`, and `library` validation need NO edit — they look up refs stored on built objects, which now arrive qualified. ⚠ That is true *only because this phase lands as one change*: `Library.validate_pipe_library_with_libraries`'s first loop looks up a bare sub-pipe code against the child library, whose keys are qualified, so it is served today solely by the step-3 fall-through being deleted below. Deleting the fall-through without qualifying the dependency load path breaks it.-- [x] **Qualify the transported `PipeFunc` code.** `PipeFunc` used to send its bare code and the sandbox later calls the now-strict lookup with it (`pipelex/pipe_operators/func/pipe_func.py` → `pipelex/pipe_operators/func/direct_pipe_func_executor.py`). Under the strict lookup **every** transported `PipeFunc` fails, not merely ambiguous ones. The DTO must carry a qualified ref — it now transports `pipe_code=self.pipe_ref`.--### The break message--- [x] Spec the not-found message in **dependency validation** — that is where users actually meet it, not in the lookup. It names the qualified ref that was tried, and when a crate-wide candidate exists it suggests it and states the new rule. Without this, the user reads `advisory.present_as_markdown` while staring at a file that says `present_as_markdown`, and has no way to know the compiler added the prefix.-- [x] Mark the diagnostic crate-wide scan **failure-path only** in code and comment. It is the same scan being deleted from the resolution path, and an implementer in a hurry can reintroduce the fallback by accident.-- [x] Build the crate-wide bare-code index **once per validation run** and share it across every missing-ref diagnostic. Per-error scanning goes quadratic on the mass-breakage run — which is precisely the first-run-after-upgrade this feature exists to improve.+**Current phase:** Phase 0 **done**; **Phase A done and committed** (`feae4db1a`); **Phase B done and committed** (`4237a51c5` + `The kit's backend directory on the walk, and the ledger doc's open root`); **CP1 cleared** — Louis ratified D1–D4 as they stand on 2026-08-17; **Phase C done and committed** as two commits (`81554b3bd` C1, `c765430d4` C2); **CP2 cleared** (`c2a104a7a`); **Phase D done and committed** (`b7f093c31`); **Phase E done and committed**; **a CP3 review found two defects and both are fixed, and a second pass on its two judgement calls fixed those too, plus a third hole it found** (see the blocks below, uncommitted at the time of writing). ▶️ **CP3 is reached: every gate is green and the first-machine dry run is taken; the only thing left before Phase F is Louis' go to push.** Nothing is pushed.++**The CP3 review, and the two things it found. Both were measured through the real CLI rather than argued, and both are fixed with tests that go red under mutation.**++- **Phase E's own behaviour change was unreachable, and its test could not tell.** The `BootComponent.migration_surface_id` wiring is only entered from `except InferenceBackendLibraryValidationError` — and that class is raised at exactly one site (`backend_library.py`'s `except ValidationError` around `load_toml_from_path`, which raises `TomlError`/`FileNotFoundError` and never a pydantic error), so **it is raised nowhere at all**. What a stale-and-unexplainable backend file actually raises is `InferenceBackendLibraryError`, which no clause in `RuntimeBoot.setup` named — so it escaped `setup` as a bare traceback, with no `PipelexSetupError`, no migration block and no `pipelex migrate`. The test passed because it built the exception by hand and assigned `__cause__`. **Fixed** by naming the family once, `BACKEND_LIBRARY_REFUSED`, beside `BootComponent` and for the same reason (a clause naming one class of a family is a hole rather than an error), and by routing the message through a new `report_config_refusal` — because the second shape a backend file refuses in carries **no** pydantic error at all: a per-model key rejected by name arrives with no `__cause__`, and the old builder threw the refusal's own message away on the way to saying `Unexpexted error:None`. The test now takes every refusal from the real loader, so the `pytest.raises(BACKEND_LIBRARY_REFUSED)` inside the fixture *is* the assertion that the boot catches what the loader raises.+- **The diagnosis was silent about a header in `[defaults]`, which is fatal there.** `admits_unnamed_key` read the *schema* path, where `[defaults]` and a model table are the same node (`*` under an open root) — but the loader splits header-shaped keys off a model table and copies `[defaults]` in **unsplit**, so a header there is `extra_forbidden` on every model of the file. Measured: `x-portkey-provider` in `ollama.toml`'s `[defaults]` crashed the boot while `pipelex migrate --dry-run` said *"Every configuration file on this machine is at the current schema"* — on the very command the boot error sends that reader to for the diagnosis. `describe_model_spec_document_rejection` had it right all along; only the diagnosis was wrong. **Fixed** by passing `document_node_path` (which `_diagnose_table` already had beside the schema path) and excluding `MODEL_SPEC_DEFAULTS_TABLE`. Second benefit, now pinned by a test: a file that is wrong without being old earns a `would_write = false` block naming the path, which points at `migrate --dry-run` instead of promising a rewrite.+- **Verified after the fix, same isolated fake home:** the boot now names the key, the file, *"What `pipelex migrate` would carry forward: Drop prompting_target from every backend definition"*, the unexplained path as the user's part, and no `pipelex init config` tail; the dry run reports `? *.x-portkey-provider` beside the `v2` entry. Also verified on that home, which the earlier phases never did for real: a **non-dry-run** `pipelex migrate` repairs three planted files with one backup each, leaves `pipelex_gateway.toml` byte-identical (D1 holding on a write, not only on a dry run), and a second run writes nothing and adds no backup. `prune_backups_except` keeps one backup per file, which answers the *Open questions* worry about accumulation. **The protected `~/.pipelex/` specimen was never touched** — all thirteen files still carry the key and it holds no backup.+- **Three mutations, none applied with git** (the harness held each original in memory and wrote it back): narrowing `BACKEND_LIBRARY_REFUSED` back to the `…ValidationError` sibling reddens six cases; making `report_config_refusal` require a pydantic error reddens exactly the two no-cause parametrizations; dropping the `[defaults]` exclusion reddens the new diagnosis test and nothing else.+- **Prose that had gone false, now corrected:** the CHANGELOG bullet for the boot error (it claimed a behaviour that could not fire) and the same bullet's account of the surface's silence about headers. `docs/tools/cli/migrate.md`'s *"That error names this command too"* needed no edit — it is true now. `inference-backend-config.md` § *Sending extra request headers per model* gained the paragraph saying a header belongs on a model table and cannot go in `[defaults]`, with the reason.-### Tests--- [x] **Build a two-domain fixture.** In a single-domain crate, owner-domain qualification and crate-wide search give identical answers, so every resolution-row test below would pass just as happily under the rule being deleted. The rows only discriminate when a sibling domain declares the same pipe code. ⚠ **Measured in Phase 1, not merely suspected:** applying this phase's rule to all 54 crates buildable from this repo's `.mthds` files changes nothing — every normalized output is byte-identical under both rules. The repo's own corpus is blind to the flip, so the fixture must be hand-built, and the mutation check below is what proves it discriminates.-- [x] Flip the normalizer test that pins the old rule: `tests/unit/pipelex/libraries/test_crate_normalization.py::test_bare_cross_domain_pipe_refs_resolve_to_the_declaring_domain` becomes a test of owner-domain qualification.-- [x] Flip the lookup tests that pin the old rule: `tests/unit/pipelex/libraries/test_pipe_library_lookup.py::test_bare_code_ambiguous_raises` and `::test_bare_code_unambiguous` become tests of the strict lookup + entry affordance split.-- [x] Qualify the refs in tests that construct pipes directly with bare sub-refs (bypassing the crate pass) — they hit the strict lookup now. Find them by running the suite, not by grepping alone.-- [x] Conformance-shaped unit coverage for the resolution rows on the two-domain fixture: own-only resolves; sibling-only errors; both-declare resolves to own; nowhere errors. ⚠ **The export-bypass closure case is NOT covered by a test here** — only by re-running the demo probe by hand (recorded in README §3). Its *behaviour* is covered: a bare in-body ref cannot reach **any** sibling domain, exported or not, which is pinned at library level by `test_unresolved_pipe_ref_message.py::test_names_the_qualified_ref_and_explains_the_rule`, and export-bypass is a strict subcase of that. What has no test is the `[exports]`/`METHODS.toml` machinery *interacting* with the new rule, which needs a package fixture and is conformance-shaped — it is already scheduled in Phase 4b's spec-suite line item. Do not tick that 4b item without it.-- [x] **Mutation-check the rows.** Done for sibling-only (restoring the crate-wide search reddens the two rule tests and both message tests) and per-controller-kind for the closure walk. **Not done for export-bypass**, because there is no export-bypass test to mutate — see the item above.-- [x] Coverage for the entry affordance semantics: exact hit, crate-wide unique bare match, ambiguity error message, and no-match.-- [x] Pin the stated invariants that currently have no test: `alias->` refs pass through the pass untouched; all four ref kinds are covered (steps, branches, outcomes, batch refs — branches and batch refs are the ones a hand-written pass forgets); the pass is idempotent (`d.c` must not become `d.d.c`, and it now runs in two places); the pass runs on **all three** load paths (a miss on the secondary or dependency path is invisible for any single-domain library); the OQ3 ruling; and that the affordance reaches a **non-exported** pipe (the twin of the export-bypass closure — the docstring claim needs enforcement).-- [x] Execution test for `batch_over` covering the derived batch code.-- [x] Coverage for both branches of the new validation message: candidate exists, and no candidate exists.-- [x] Audit existing CLI coverage for the migrated surfaces, then add a bare-code end-to-end test wherever there is not one. A perfectly-tested affordance can still be perfectly mis-called, and `pipelex which <bare_code>` is the invocation the affordance exists to preserve.-- [x] Re-run the README's demo probes (`wip/pipe-refs/probes/make-demos.sh`) and move the closures accordingly: `fallthrough` and `export-bypass` must now FAIL resolve; `ambiguous` must resolve to the own domain. Update whatever the probes/README record as expected outcomes.-- [x] Gates checklist clean (all items).--### ⛔ CHECKPOINT C2 — MANDATORY STOP--The pipe rule is complete and self-consistent: canonical form and runtime agree, and hand-typed codes still work through a named door.--- [x] Update the Status block (OQ2/OQ3 rulings, the DRY ruling, surprises, corpus fallout).-- [x] Record status in the design doc's Phase 2 section (`build-time-qualification.md` → "Phase 2 outcome (C2)").-- [x] Update README §2's table — marked HISTORICAL with a banner rather than rewritten, and §3 gained a re-measured table.-- [x] Update the `docs/` pages that describe bare-ref resolution, pipe lookup, or the entry surfaces touched here. `docs/building-methods/libraries.md` gained a "How a pipe reference resolves" section; `docs/contribute/hub-layering.md` gained the two new hub symbols and the in-body-vs-entry guidance (that page is a drift contract target and was reviewed and acked).-- [x] Changelog entry under `[Unreleased]` noting the breaking language-behavior change. Also **removed** the superseded `[Unreleased]` entry describing the crate-wide rule this phase deletes — it never shipped, so leaving both would have published two contradictory claims.-- [x] Commit — landed on a stacked branch (PR #1092) rather than the base one, since PR #1091 carries Phase 0+1 and the two merge together.-- [x] STOP. Next session starts at Phase 3.--⚠ **These boxes sat unticked while the Status block above declared C2 passed** — caught by a PR bot, not by me. A checkpoint that reports itself complete from one place while its own checklist says otherwise is exactly the contradiction a cold-starting session cannot resolve. Tick the checklist in the same edit that updates the Status block.-------## Phase 3 — the concept side and the footprint cleanups--Same root cause as Phase 2 — crate-wide search where owner scope was meant — but an independent surface with its own caller sweep. Kept separate so the pipe fix has a finish line that does not depend on it.--- [x] Collapse `ConceptLibrary.get_required_concept_from_concept_ref_or_code`'s `search_domain_codes` machinery (`pipelex/libraries/concept/concept_library.py` + `concept_provider_abstract.py`) into the entry-affordance shape decided in Phase 2. Landed as the `get_optional_entry_concept` / `get_required_entry_concept` pair; the required form replaces the old method on `ConceptProviderAbstract` 1:1.-- [x] **Scope the concept lookup by package, not just domain.** `search_scope` is a single string derived from the resolved entry pipe — its domain, prefixed `alias->domain` when the caller invoked a dependency pipe. Dependency concepts are keyed `alias->domain.Concept`, so the scoped preference is a direct key construction; no child-library machinery needed. Unlike pipe-side OQ1 (deferred to packaging), the concept lookup is entry-only and its one call site holds the alias, so this was paid in full here.-- [x] Exclude aliased dependency keys from the concept crate-wide search too (same reasoning as Phase 2). Mutation-checked.-- [x] Delete the latent defects README §4 documents: both gone structurally — there is no per-domain walk left to die on a miss, and every escape is `ConceptLibraryConceptNotFoundError` (the class the `stuff_factory` handlers catch), ambiguity included. §4 marked HISTORICAL with the re-measured matrix appended (probe rewritten against the new affordance).-- [x] Decide whether `pipeline_run_setup`'s own-domain-first ordering becomes meaningful or gets deleted along with the list parameter — **it became meaningful**: the scope preference fires BEFORE the crate-wide scan, so the entry pipe's own domain wins outright instead of being collected into an ambiguity raise. The list parameter is deleted from the whole public chain (`runner.py` constructor included — breaking, in the changelog); callers were swept, none supplied it in production.-- [x] Relax `_pipe_codes_by_file`'s rename collision scope in `pipelex/pipeline/fixes/fix_loop.py` from crate-wide to per-domain. Files now carry their declared domain (`_FilePipeCodes`); a file with an unreadable domain conservatively collides with everything. ⚠ Two integration tests pinned the crate-wide scope and flipped: the rebuild-between-rounds pin moved to a same-domain fixture, and `test_strip_namespace_bails_on_cross_file_collision`'s other-domain arm became a convergence test.-- [x] Tests: the resolution rows for concepts on the two-domain fixture (unit, `test_concept_library_entry_lookup.py`); ambiguity raises naming candidates; the package-scoped dependency-concept case (bare + dotted + exclusion); fix_loop independent renames across domains; an end-to-end wiring test (`test_entry_concept_scope.py`) that reddens if the scope threading is dropped anywhere in the chain; a characterization pin that `pipeline_run_setup` derives the scope from the entry pipe.-- [x] Gates checklist clean (all items). Grants for the new entry-concept defs recorded before the first check; `.test_durations` reconciled against a full `-m ""` collection (renamed node ids and remapped param ids; the remaining orphans predate this branch).--**Carry these forward from Phase 2 — each cost real time to learn:**--- **A single-domain fixture cannot discriminate this rule.** Owner-scope and the crate-wide search agree on every input where only one domain exists, so a concept test without a competing sibling declaration passes just as happily under the rule being deleted. Mutation-check every resolution row: put the old rule back and confirm the row goes red.-- **Mutate the wiring, not just the rule.** Removing the qualification from the *dependency* load path reddened nothing in the entire suite — every other test loads a single-domain library. Assume the same blind spot exists on the concept side and go looking for it deliberately.-- **Run the whole suite, including `tests/e2e`.** Phase 2's fixture sweep collected failures from `tests/unit` + `tests/integration` only and shipped two e2e failures into a "green" checkpoint.-- **A review finding that names a file and a line is a claim to verify.** Phase 2's batched-derivation item predicted the wrong consequence, and the first test written for it passed against the bug.-- **Do not touch the working tree while `agent-test` runs**, and read the run's log rather than the harness's exit-code summary — one reported success over a log ending in `Error 1`.--### ⛔ CHECKPOINT C3 — MANDATORY STOP--The runtime is fully spec-compliant and self-consistent; only cross-repo work remains.--- [x] Update the Status block and the design doc's Phase 3 section (`build-time-qualification.md` → "Phase 3 outcome (C3)").-- [x] Re-run the README's probes and demos and confirm the post-change behavior matches the design's predictions. The concept-lookup-matrix probe was rewritten against the new affordance (the method it measured no longer exists) and its re-measured table appended to README §4, which is now marked HISTORICAL.-- [x] Changelog entry for the concept-side behavior change (breaking: the `search_domain_codes` public parameter and the `ConceptProviderAbstract` method rename) and the `fix_loop` per-domain relaxation.-- [x] Commit.-- [x] STOP. End the session. Next session starts at Phase 4a.--**Post-C3 review round (2026-08-13):** a medium code review of the C3 commit surfaced real gaps in the new affordance, fixed in a follow-up commit on the branch. A multi-domain dependency package's own concepts are now reachable by bare code from any of its entry pipes — the scoped preference probed exactly one key and the crate-wide scan excludes aliased keys, so the sibling-domain shape was unreachable; the pipe side's alias-scoped package search was ported over, with `alias->Concept` as its explicit spelling. Dotted refs now obey the same scope-wins precedence under a dependency scope instead of binding host-first (a host/dep domain-name collision would have bound the host concept silently, since compatibility short-circuits on equal `concept_ref`). The invalid-string escape joined miss and ambiguity on `ConceptLibraryConceptNotFoundError`, making the one-class claim actually true. The copy-paste `stuff_factory` handlers collapsed into one `_resolve_entry_concept` helper; `pipeline_run_setup` captures the caller's raw entry ref in its own local so the scope derivation no longer depends on `pipe_code`'s rebind order; the `fix_loop` bail message hedges the unreadable-domain case. Each behavioral fix mutation-checked.-------## Phase 4a — cross-repo, release-independent--These need no published `pipelex`. They can land as soon as Phase 3 is committed.--- [x] `mthds/` — additive clarification at § *Resolution Order for Bare Pipe References*: no-fall-through is what makes `[exports]` enforceable, so the next reader does not mistake the rule for a lookup convenience. No normative change. Landed 2026-08-13 in `mthds/` on branch `docs/bare-pipe-ref-no-fall-through` (off `dev`): an admonition note in `docs/spec/namespace-resolution.md` + a changelog entry; `make docs-check` (strict) passes. Not pushed — no PR opened without being asked. *(The design doc's "mthds/ last" ordering came from deferred-review-observations §4, whose constraint was D-1's domain_hint question — answered and subsumed in Phase 2, so the ordering dissolved and this plan's 4a placement governs.)*-- [ ] Add the new hub accessor to the transport boundary spec (workspace-level; exact path in the workspace private notes). The prose spec is a doc edit; its verifying test lands in 4b, and the two must land together per the spec/conformance sync rule — so hold this until 4b is ready to go with it.--## Phase 4b — cross-repo, gated on a published `pipelex`--**Gate:** every item below needs a `pipelex` release carrying the new `interpreter_hub` accessor, installable by the sibling repo. Cut that release first. Opening these PRs beforehand produces red CI for a reason unrelated to the change.--- [ ] our cross-repo spec suite — executable conformance cases for the resolution rows (own-only, sibling-only, both-declare, nowhere) and the export-bypass closure, on a two-domain fixture. Reuse the fixture built in Phase 2.-- [ ] our cross-repo spec suite — add the new hub accessor to the transport allowed-surface test, landing together with the spec doc edit from 4a (exact paths and the current state of that test in the workspace private notes).-- [ ] Run the spec suite's spec-link checker — must pass.-- [ ] the transport layer that is not open core — migrate its host-side bridge's payload-supplied lookup to the new accessor (that code is entry-shaped; `get_required_pipe`'s pinned signature is untouched).-- [ ] `pipelex-cookbook/` — qualify the first reference that leans on fall-through: `presentation.present_as_markdown` in `examples/wip/advisory_board/bundle.mthds`. Verify the example still validates and runs against the updated runtime.-- [ ] `cocode/` — qualify the second: `changelog.format_changelog_as_markdown` in `cocode/pipelines/swe_diff/changelog_enhanced.mthds` (a shipped CLI — see `wip/pipe-refs/corpus-measurement.md`). Verify the pipeline still validates against the updated runtime.-- [ ] **`search_domain_codes` consumer sweep beyond the transport bridge.** Found by enumerating the workspace (the transport item above covers only `pipelex-server/transport/`): `pipelex-api/api/routes/pipelex/pipeline.py` still passes the deleted `search_domain_codes=` kwarg to `pipeline_run_setup` — a `TypeError` on every execution request the moment the pin moves (and remember: its openapi-check runs under `make check` only, so an agent-gate-green bump still fails CI); `pipelex-js` mirrors the lookup semantics in `packages/runtime/src/library/live-library.ts` and drives it from `scripts/python-live-library-oracle.py` (semantic parity, not just a kwarg); `pipelex-mistralai-workflows` names it in integration tests; the demo repos (`pipelex-demo-mistral`, `pipelex-demo-vibe`) use it in example workflows. Each needs the entry-affordance/`search_scope` migration or the kwarg dropped.-- [ ] **Aliased-entry `search_scope` wiring test.** The package-fixture conformance work above must also pin the run-setup wiring: invoke a dependency pipe as the entry via `alias->domain.pipe`, supply a bare envelope concept declared only in the dependency, and assert it resolves. Today no test invokes an entry pipe through an alias end-to-end, so deleting the `has_cross_package_prefix` branch in `pipeline_run_setup` (or capturing `raw_entry_ref` after `pipe_code` rebinds) reddens nothing.-- [ ] **Error-surface sweep.** Run `make generate-error-identity` and `make generate-error-pages`, review the identity diff, and grep the TypeScript consumers for any changed or newly-introduced `error_type` literal — `mthds-starter-js/src/lib/errors.ts`, `pipelex-starter-js/src/lib/errors.ts`, `vscode-pipelex`'s `cliValidationBackend.test.ts`, and the internal playground app's graph route. The class name is a wire string; those repos switch on the literal and fall through to a generic branch without failing to compile. No Python tool will point at this.-- [ ] Changelog entries in each touched repo noting the breaking language-behavior change, per each repo's convention (skip `wip/` docs in changelogs).--### ⛔ CHECKPOINT C4 — MANDATORY STOP (final)--- [ ] Update the Status block; all phases complete.-- [ ] Close `wip/pipe-refs/` with a final status note in the design doc and README.-- [ ] Fold the outcome into the parity track's records (`wip/parity/`).-- [ ] Commit; prepare PRs per repo as instructed by the user — do not push or open PRs without being asked.-- [ ] STOP. The work is done; anything further (release sequencing before the packaging system) is a separate track.+**A second pass on the review's two judgement calls, at Louis' request — both fixed, and a third hole found on the way.** Measured on the isolated fake home, before and after:++- **The loader's own sentence was thrown away whenever a pydantic error was behind it.** `report_config_refusal` reached through to the `ValidationError` and translated *that*, so a bad value on `[gemma3-4b]` in `ollama.toml` reached the user as `max_prompt_images: int_type: Input should be a valid integer` — no model, no backend, no file, in a directory of a dozen files that all have that field — while the loader had said `Invalid inference model spec 'gemma3-4b' for backend 'ollama' from file '…/ollama.toml': …` one frame down. Pre-existing on the model deck path (its old builder did the same), new on the backend path only because that path had never been reachable. **Fixed**: the body is `str(refusal)`, and only a *bare* `ValidationError` is translated. The deck's message keeps its paths now too.+- **The gateway raise site the widened `except` routes to is reachable from a boot only through a disagreement between two readers of `backends.toml`.** `is_pipelex_gateway_enabled` read `enabled` as *is the literal `true`*, the loader reads the raw value's truthiness — so `enabled = 1` fetched no gateway specs for a backend the loader then loaded: refused as *"remote model specs were not provided"* on a strict boot, silently dropped from the deck on a lenient one (measured: `show backends` listed no `pipelex_gateway`). **Fixed** by making the flag read the value the way the loader does; the raise site's message now says it is the caller's omission (`pass gateway_config, or disable the backend`), because that is the only way left to reach it. The *framing* worry dissolves: no user file reaches that sentence any more.+- **Found on the way: a bad value in `backends.toml` itself escaped as pydantic's bare `ValidationError`.** `InferenceBackendBlueprint.model_validate` was unwrapped, and the boot named no clause for a bare pydantic error — `endpoint = 42` under `[ollama]` was a raw traceback naming a field and neither the table nor the file. **Fixed** by wrapping it as `InferenceBackendLibraryValidationError` — which gives that class its first real raise site, naming backend and file — and by removing the dead `except ValidationError` around `load_toml_from_path` that had made the class *look* raised. Verified through the CLI: `Invalid inference backend 'ollama' in '…/backends.toml': … endpoint: string_type …` plus the plain tail.+- **Three more mutations, restored from scratchpad copies**: reaching through to the pydantic error again reddens the two location-asserting cases; unwrapping the blueprint validation reddens the index-file case; `is True` back in the flag reddens the `1` and `"yes"` parametrizations. Every backend refusal in the boot test is still produced by the real loader.+- **Seen and left alone, deliberately:** a TOML *syntax* error in any configuration file — `pipelex.toml`, `backends.toml`, a per-model file — still reaches the user as a `TomlError` traceback. It names the file (not the position — that sits on the exception as `lineno`/`colno` and never reaches the message), it is uniform across every surface including the main configuration, and the agent CLI classifies it as `error_domain: "input"` because only the loader knows the file was configuration. Turning it into a proper configuration error is a cross-surface change of its own; so is the wider fact that the human CLI renders *every* boot configuration failure as a traceback with the message underneath. Both are written up as a follow-up brief in `wip/config-errors-toml-syntax-and-cli-panels.md`.++**Cold-start for the next session, in order:**++1. Read this Status block whole. Phases B, C, D and E are no longer designs — they are code and prose, and the *What Phase X actually built* sections below are the maps of them. Do not re-derive B1–B9 or the C/D/E items (they are kept only as the record of what was predicted and where the prediction was wrong).+2. Skim the charter (`wip/migrator-3/sequencing.md § S7b`) and the substance (`wip/migrator-3/backend-config-surface.md`) for *why*; this file holds the *what* and the *order*.+3. **There is no work left before the push.** The next action is Phase F: `git push -u origin feature/Backend-surface` and the PR against `dev`. Everything before that line is local and done.+4. Read *Open questions* below before the PR description is written — three of them are things a reviewer will raise, and two are deliberate deferrals with a named cost.++**What Phase E actually built, so the next session reads the diff rather than re-deriving it:**++- **The docs items landed as listed, and the four Phase-B spots were re-read against the finished code rather than trusted.** § *The fingerprint*'s open-root blockquote, § *The defaults layer*'s copied-not-layered paragraph, § *Surfaces*' fourth table row and the defaults-layer-kinds sentence all still describe what the code does; none needed an edit. What did change in `docs/migration-ledger.md`: § *What migrates* gained a paragraph putting the content/structure split **on one file** (which models a backend declares is content and no ledger will ever add one; which keys a model table may carry is structure and is a surface); "all three configuration surfaces" became "every configuration surface"; § *The helper owns the failure path* now names the fourth loader as the one that does the most between merge and validate, and gained a blockquote on why a surface of independent documents calls the helper **once per file** (the helper deep-merges, and merging two backend files would produce a backend nobody has); § *Boot tolerance* gained the fourth loader's parked-warning bullet, whose reason is **a caller and not boot order** — the doctor's per-backend probe.+- **`docs/tools/cli/migrate.md` § *What it touches* had one genuinely ambiguous sentence** ("Nothing there is ever rewritten" could be read as *nothing in `inference/backends/`*), now unambiguous, plus a new paragraph saying a migration only ever undoes a change *we* made. § *The warning that sends you here* now says the warning covers backend definitions too and arrives with the model setup.+- **`docs/configuration/config-technical/inference-backend-config.md`** gained the section the Phase B drift ack promised, in the page's own instructional voice: why `pipelex init` not overwriting is what leaves a file behind, the warning quoted **faithfully** (it is one paragraph of sentences, not the multi-line block the plan imagined — read from `config_surface.py:256-275`), the two commands, and a four-bullet what-it-touches list ending on the backup. `docs/configuration/index.md:62` already carried the pointer from Phase B and its link now resolves to a real section.+- **CHANGELOG.** The `:10` non-recursive sentence was rewritten to the (directory, name) rule; the surfaces bullet stopped saying "all three start at schema version 1 … it migrates nothing", which was **already false before S7b** (the reshape and the telemetry entry both land in this same Unreleased block) and is now "each is cut at schema version 1; the entries this release ships on top of that baseline are the bullets below"; the `.mthds`-bundle/backend/deck list in the validation-error bullet dropped the backend file. Two new bullets: an **Added** one for the fourth surface carrying the `inference-backend@2` `**Migration:**` clause, and a `**Migration:**` clause appended to the templating-style **Changed** bullet for `pipelex-config@2` and the renumber. A **Fixed** bullet covers the behaviour change below.+- **Skills.** `add-migration` gained the eight-step *Inserting a pre-history entry below existing entries* procedure this session performed in C2 — renumber, `git mv` highest-first, copy the frozen link, **hand-edit the one `"schema_version"` line** (the single sanctioned exception to "never hand-edit a golden", and the "never" bullet now says so), hand-author `before@N`, and `umig` must then be a byte-pure no-op *which is the proof*; plus the id sweep and the two-changelog-mentions rule. `release` step 3b gained the paragraph saying a renumber shows up as two ids and is one insertion, so the pushed-up id is not treated as an unbullet-ed new entry. `add-model` gained F12's coupling: editing the kit's `portkey.toml` reddens `cmig` until `make umig` runs, and no other backend TOML is coupled that way.+- **A behaviour change this phase took, deliberately, outside its declared docs-only scope — read this one.** Three places asserted that an inference backend file is *not* a configuration surface (`core/validation.py`'s docstring, a comment in `runtime_boot.py`, and `docs/migration-ledger.md` § *The scan is named, not guessed*), which S7b made false. Correcting only the prose would have meant writing down a hole, because the boot **acts** on that claim: `RuntimeBoot._get_validation_error_msg` named no surface, so a backend file that is *both* behind and carrying something we cannot explain — the one case that gets past boot tolerance — was reported on its unknown key alone, while identical staleness in `pipelex.toml` produced the full migration block. Fixed: a new **`BootComponent` StrEnum** pairs each late-refusing component with its surface (`INFERENCE_BACKEND_LIBRARY` → the surface id; `MODEL_DECK` and `ROUTING_PROFILE_LIBRARY` → `None`), stated once instead of at each `except` clause, which is what makes the wiring testable — the test parametrizes over the same constant the call site passes. It replaced the bare `component_name: str` on both message helpers, so the five call sites no longer repeat string literals. Second half: a message carrying a migration block **stops appending** *"or run `pipelex init config` to regenerate them … resets all config files to their default values"* — the contract's own "none of them says start over" rule, which that tail broke the moment the backend became a surface. The deck keeps the tail; for it, it is the only answer there is.+- **New `tests/unit/pipelex/test_runtime_boot_stale_backend_error.py`** (`TestAStaleBackendFileTheLedgerCannotFullyExplain`), following the house `test_runtime_boot_*.py` pattern. Four cases: the message names the stale file and the entry while still being about the user's own key; the regenerate tail is gone *and the block was found* (so the tail is not dropped for the wrong reason); the model deck is byte-for-byte unchanged in treatment; and a machine with nothing stale keeps the plain message, so naming a surface cannot invent a block. `_get_validation_error_msg` had **no** tests before. Two mutations, neither applied with git (the harness held the original in memory and wrote it back): making the backend return `None` for its surface reddens the first two; forcing the block branch off reddens the tail test alone.+- **`config_surface.py`'s module docstring was self-contradictory and is now honest.** It listed "backend definitions" among the readers that should keep *rejecting* the reserved `[meta]` table, in the same paragraph that names them a configuration surface. The real asymmetry, now written down: the backend loader reads through the generic TOML reader (`load_toml_from_path`) rather than through `config_surface`, so a backend file carrying `[meta]` would be refused as an unknown model table even though the migration engine reads the key there. **Nothing writes that key on any surface**, so it is latent, not live — see *Open questions*.+- **One local-environment cleanup, and a finding under it.** `make check` failed on `check-config-sync`, because Phase A's one-off `pipelex migrate --yes` left `.pipelex/pipelex_override.toml.bak.20260817T113323Z` — gitignored since Phase A, but the sync checker compares directories on disk and excludes by exact file **name**, which a timestamp cannot be. The backup was diffed against the live file first (pure pre-reshape shape; every setting present at its new address, `default_log_level` and `package_log_levels.pipelex` confirmed by eye) and then removed. The finding it exposes is in *Open questions* and is worth a reviewer's attention.++**The first-machine dry run, taken read-only at CP3 (`pipelex migrate --dry-run`, never a real run):**++- The real `~/.pipelex/inference/backends/` holds `.toml` files plus the two `.md` files. The report claims the backend `.toml`s that need it — `anthropic`, `azure_openai`, `bedrock`, `fal`, `google`, `minimax`, `mistral`, `ollama`, `openai`, `pipelex_inference`, `portkey`, `vertexai`, `xai` — each under `(inference-backend)`, each naming `→ v2 Drop prompting_target from every backend definition`, and each rendering the op as **`deleted 'prompting_target' from every table of the document`**. That is Phase D's wildcard wording, confirmed on real output rather than only in its unit test.+- **`pipelex_gateway.toml` is present in that directory, is clean, and is absent from the report** — absent because it needs nothing, not because it went unclaimed. That is D1 holding on a real machine: the collision specimen was claimed by `inference-backend` and not by `pipelex-config`, whose ledger would have rewritten it. The two `.md` files and `inference/backends.toml` are likewise absent.+- **No value read from any file appears anywhere in the report** — paths, entry titles and op kinds only. `pipelex_inference.toml`, which is this machine's own and not kit-shipped, is claimed like any other `*.toml`, which is the glob doing what D1 says.+- Neither configuration root reported a `pipelex-config` or `telemetry-config` file: both are already current. Ends on `Dry run — nothing was written.`, and the specimen still carries its `prompting_target` occurrences in `portkey.toml` afterwards. **It has still never been edited, reset, or migrated for real.**++**Gates at the Phase E commit — the full checklist, all eight:** `make agent-check` clean (pyright 0 errors, mypy clean over 2528 files, `cl` 4 ledgers legal and convergent, keyword-only PASSED, hub-layering PASSED); `make cmig` 4 surfaces accounted for; `make tb` 46 passed; **full `make agent-test` green** (the first full run since Phase C — no new error class, so no `gei`/`gep` was owed, and no test was renamed, so `.test_durations` did not go stale); `make docs-check` builds, with only the one pre-existing anchor INFO in `under-the-hood/orchestrator-plugins.md`, a file this session did not touch; `make drift-check` PASSED **after** staging and acking `config-docs` for real (it opened on `config_surface.py`; ack text in `.drift/acks/config-docs.toml`); `make check` green end to end, pylint 10.00/10.++**What Phase D actually built (`b7f093c31`), so the next session reads the diff rather than re-deriving it:**++- **The retry lives in `InferenceBackendLibrary.load`, per backend, once.** The per-model build was extracted to `_build_backend_model_specs` (a classmethod: pop `[defaults]` from a *copy*, split, merge, validate) so the retry can run the loader's own steps a second time without duplicating them. `_local_model_specs_the_ledger_can_explain` replays the ledger over **that one file**, re-substitutes `${VAR}`, rebuilds, and returns `RecoveredModelSpecs(model_specs, plans)` — or `None`, on which the caller's bare `raise` re-raises the error the file actually produced.+- **The warning is parked, not logged, and the reason is a caller rather than boot order.** `take_stale_configuration_warning()` on the library; `ModelManager.setup` takes it and emits it. **This is a deliberate deviation from the plan's recommendation of `runtime_boot.py`**, and the measurement behind it is `doctor_cmd.py:597`: the doctor probes backend files by loading the *whole* library once per backend, so a loader that logged for itself would repeat one directory's warning a dozen times. Emitting from `ModelManager.setup` gives a boot exactly one copy, gives `pipelex doctor`'s `check_models` one, and leaves the doctor's probe and `show_cmd`'s `--all` library silent by construction — with **no** widening of `ModelManagerAbstract`, which is a public injection point. One warning per `load()` covering every stale file, not one per backend.+- **The gateway is left out of the retry, and the plan's item for it rested on a false premise.** `GatewayConfigMerger.merge` **ignores a local `[defaults]` outright** and keeps only `sdk`/`structure_method` from a per-model override, so a stale key in `pipelex_gateway.toml` is filtered out before any spec is built and **can never be what refused a load**. There is nothing to carry forward at boot. `pipelex migrate` still repairs the file on disk — the surface claims every `*.toml` in the directory. A test pins the claim (`test_a_stale_gateway_override_never_reaches_the_loader_at_all`) and a two-part mutation of the merger reddens it, which is the day the gateway path would need a retry too.+- **A user-facing wording bug this PR itself introduced, found by the e2e and fixed here.** `inference-backend@2` is the **first** shipped entry with a wildcard `table_path`, and `describe_op` rendered it literally: *"deleted 'prompting_target' from '\*'"*. `_table_in_words` now reads a **trailing** wildcard as *"every table of the document"* (empty parent) or *"every entry of '<table>'"*; a wildcard anywhere else keeps the literal dotted spelling, because nothing addresses that shape and inventing a phrase would be guessing. New `tests/unit/pipelex/cli/test_migrate_op_wording.py` — `describe_op` had **no** tests at all before.+- **Tests.** New `tests/unit/pipelex/cogt/model_backends/test_backend_boot_tolerance.py` (`TestAStaleBackendDirectory`): the fixture copies the **kit's** backend directory into a faked home and plants `prompting_target` back — into `openai.toml`'s `[defaults]` and two of `portkey.toml`'s model tables — and both plantings **raise** if their anchor is gone. The strong assertion is `recovered.root == pristine.root`: what boots is byte-for-byte the library the untouched kit files produce, which is what makes "everything else intact" a real claim. New `TestAStaleBackendDirectory` in `tests/e2e/pipelex/cli/test_migrate_commands.py` — human repair with one backup per file, a second run that writes nothing and adds no backup, the `.md` files and `inference/backends.toml` byte-identical throughout, the agent loop naming both files with `entry_id` and `("delete_key", "prompting_target")`, the doctor row going `pending` → `up_to_date`, and the **warning through the human binary** (`pipelex show backends`), which is the half `pipelex-agent` cannot show because it silences logging process-wide.+- **Four mutations, all reddening the intended test, none applied with git** (the harness held the original text in memory and wrote it back): making the retry always decline reddens the "loads the same models the current files would" test; making the gateway merger permissive on both halves reddens the gateway test; and leaking the file's text into either the human report or the agent JSON payload reddens `test_no_channel_renders_the_value_the_files_carry`.+- **Two things measured and found already correct, so nothing changed.** `runtime_boot.py:488-500` is unchanged in kind — the retry sits inside `load` and either recovers or gets out of the way. And `"out of date"` was too weak an assertion for the warning: the same boot prints a *deck*-staleness notice containing that phrase, so the e2e matches the warning's own opening, `"Your configuration is out of date"`.+- **`docs/tools/cli/migrate.md:25` was false, not merely stale**, and the `cli-docs` drift contract is what surfaced it: it said the command does not descend into subdirectories and that inference backends are never rewritten by it. Rewritten to say what it now does *and* what it still does not — one level, only into a directory some configuration family owns, so `inference/deck/` and the `inference/backends.toml` beside the directory stay out. The ack text is in `.drift/acks/cli-docs.toml`.++**Gates at the Phase D commit:** `make agent-check` clean (pyright 0 errors, mypy clean over 2527 files, `cl` 4 ledgers legal and convergent, keyword-only PASSED, hub-layering PASSED); `make cmig` 4 surfaces accounted for; `make tb` 46 passed; `make docs-check` builds (the one anchor INFO is pre-existing, in a file this session did not touch); `make drift-check` PASSED after acking `cli-docs` on the staged index; targeted suite green — `tests/unit/pipelex/{cogt,cli,migration,system}/`, `tests/integration/pipelex/system/`, `tests/e2e/pipelex/cli/`, 3142 passed. **The full `agent-test` has not been run since Phase D; it is owed at CP3.**++**What Phase C actually built (two commits), so the next session reads the diff rather than re-deriving it:**++- **`inference-backend@2`** (`81554b3bd`) — `pre_history`, `breaking`, `safe`, `introduced_in = "0.46.0"`, `declared_removed_paths = ["*.prompting_target"]`, and **one** op: `delete_key`, `table_path = ["*"]`, `key = "prompting_target"`. `current_schema_version` went 1 → 2. `goldens/inference-backend/[email protected]` is hand-authored — the Portkey shape of the era, trimmed so the key is seen in `[defaults]`, on one model and absent from another, with two header-shaped keys as the neutrality witnesses. `make umig` wrote `[email protected]` + `[email protected]`; **the pair `@1`/`@2` differs by nothing but the `"schema_version"` line, and `defaults@1`/`defaults@2` are byte-identical** — measured with `diff`, which is what `PRE_HISTORY_HAS_A_DIFF` demands.+- **`pipelex-config@2` + the renumber** (`c765430d4`) — the pre-history entry is **inserted below** the reshape, which became `pipelex-config@3`; `current_schema_version` went 2 → 3. Ops in order: `move_key ["pipelex","prompting_config"] default_prompting_style → ["pipelex","templating_config"] default_templating_style`, `delete_table ["pipelex","prompting_config","prompting_styles"]`, `delete_table ["pipelex","prompting_config"]`. `declared_removed_paths` names the section and the two children the ops address; the parent alone would satisfy `_check_pre_history_source_is_declared_material` (it walks ancestors), and naming all three is the telemetry entry's precedent and the honest record.+- **The renumber is byte-pure, and here is the proof shape.** `git mv [email protected] [email protected]` + `git mv [email protected] [email protected]`, then `cp [email protected] [email protected]` + `cp [email protected] [email protected]`, then **one hand-edited line per fingerprint** (F10: `"schema_version"` in the body must agree with the filename or `read_fingerprint_golden` raises before `umig` writes). Then `make umig` rewrote the head from live and **every checksum in the directory was unchanged** (`shasum -c`, all seven OK). Against `HEAD`: old `@2` fingerprint vs new `@3` = one line, the version; old `@1` vs new `@2` = one line, the version; both `defaults` diffs empty. `git diff --cached --stat` presents it as `@2` *modified* and `@3` *added* rather than as renames, because the `git mv` was followed by a `cp` into the vacated name — the content is what the proof is about, not git's rename heuristic.+- **Nothing the checks refused had to be answered.** `make cl` and `make cmig` were green on the first run after each half, `_check_open_node_addressing` accepted `["*"]` at the open root without a further edit, and no golden needed `--force`.+- **The wildcard was verified by hand as well as by test.** `_expand_table_paths(table_path=["*"])` over the hand-authored document returns `[['defaults'], ['gpt-4o'], ['gemini-2.5-pro']]` — every root table, `[defaults]` included (F5 confirmed). Replaying the shipped ledger over **each** of the kit's backend `*.toml` files returns the *same string object* (`replay.text is original`), so convergence there is byte-neutrality and not merely equality.+- **Tests:** new `tests/unit/pipelex/migration/test_backend_back_entry.py` (`TestTheBackendBackEntry` — the migrated document is accepted by `describe_model_spec_document_rejection`, the whole document is compared key by key so the header keys are witnessed by value, every surviving *line* is compared as text so comments and quoting are witnessed too, and the replay is idempotent). New `TestAPrePromptingStyleMachine` in `tests/e2e/pipelex/cli/test_migrate_commands.py`: it rewinds `[email protected]` one release by swapping the templating section back to the prompting one (`TEMPLATING_SECTION_AT_SCHEMA_1` / `PROMPTING_SECTION_BEFORE_SCHEMA_1`, and the planting **raises** if the swap matched nothing), plants only the global tier, and proves the two entries compose — the human loop lands on exactly `_todays_pipelex_config_document()` with one backup, and the agent loop's plan names both titles in order.+- **Phase D is already half-proved, measured read-only at the end of this session.** A temp configuration root seeded with the kit's backend directory, `prompting_target` planted back into `openai.toml`'s `[defaults]` and into one of `portkey.toml`'s model tables, run through the **real** `migrate_config_directories(dry_run=True)`: each file names the entry (*"Drop prompting_target from every backend definition"*), `unexplained` is empty for both, and `needs_attention` is `False`. So the walk half of Phase D's end-to-end item is behaviour that already works — what is genuinely missing is the *loader* half: `InferenceBackendLibrary.load` still raises on such a directory instead of replaying and warning.+- **Stale quotes swept** (`git grep 'pipelex-config@2'`): `CHANGELOG.md` (the reshape's `**Migration:**` bullet → `@3`), the e2e module docstring, `TestAPreReshapeMachine`'s docstring, and `tests/unit/pipelex/pipeline/fixes/test_fix_applier_config_surface_shapes.py`. **No test read `[email protected]` as "the packaged file at schema 1"** — the two readers (`_pre_reshape_pipelex_config_document`, `_OUT_OF_ORDER_SPECIMEN`) already named `[email protected]`, so the plan's worry did not materialise; the helper's docstring now says *why* it names schema 1 rather than the version the reshape starts from. Two prose spots that named a version where the number was not the point were rewritten as constructs instead of bumped (`_todays_pipelex_config_document`'s "the head golden", and "the reference document the reshape starts from"). `test_boot_tolerance.py`'s module docstring lost "the other two surfaces have empty ledgers today" — it was already stale before this session, and the honest reason those tests plant a synthetic ledger is that the *wiring* is what they hold still.++**CP1 is closed: D1–D4 are ratified, verbatim as recorded under *Rulings taken* below.** Nothing about the shape is open any more — the `COPIED_DOCUMENT` name stands, `portkey.toml` stands as the reference document (with F12's cost accepted: adding a model to it reddens `cmig` until `make umig` runs), the open document root stands, and the `pipelex-config` pre-history entry ships in this session with the reshape renumbered to `@3`. The rulings block is kept below as the record of *why*, not as an open question. Phase C is the first work that builds on D4, and it is where the renumber happens.++**What Phase B actually built (two commits: `4237a51c5`, then `The kit's backend directory on the walk…`), so the next session reads the diff rather than re-deriving it:**++- **`pipelex/cogt/model_backends/model_spec_document.py`** (new) holds both halves of "a backend file is a document, not a model", so the loader and the gate cannot drift: `InferenceModelSpecFileNode` (the all-optional projection — a **subclass** of the blueprint, `sdk: str | None = None`) and `describe_model_spec_document_rejection` (the loader's own pop-`[defaults]` / split-headers / merge / validate, returning a reason string). `MODEL_SPEC_DEFAULTS_TABLE` is spelled here and `backend_library.py` now reads it instead of the literal `"defaults"`.+- **`DocumentShape`** (StrEnum, `WHOLE_DOCUMENT` / `MODEL_SPEC_TABLES`) on the `Surface`, with a `document_root_is_open` property. **This replaces B1's plain `Surface.document_root_is_open` bool** — the surface declares the *shape* once and the three behaviours are derived from it, while the **fingerprint** keeps the bool, because a golden should record a path-level fact and not our Python vocabulary.+- **`DefaultsLayerKind.COPIED_DOCUMENT`** plus the property `is_layered_beneath_the_users_file`. `read_defaults_document()` → `{}` (nothing faked), `render_reference_document()` → the kit's `portkey.toml`, and the reachability validator now pairs each kind with the one document path it reads (a table-driven loop, not a second hand-written `if`).+- **`Surface.validate_document`** and **`Surface.admits_unnamed_key`**, both `match`ing on `document_shape`. The first replaced `surface.config_model.model_validate` in `transform_check._check_the_migrated_document_is_accepted`, which also stopped deep-merging the reference document under a copied one. The second is the diagnosis seam.+- **`Surface.kit_template_dir`** beside `kit_template_path` (mutually exclusive, validated), enumerated **lazily** by `reference_documents()` — `is_file()`, matching the surface's own glob, excluding the reference document itself, labelled `kit template '<name>'`. So the whole kit backend directory witnesses convergence.+- **`compute_fingerprint(document_root_is_open=…)`** and `SurfaceFingerprint.document_root_is_open`. `coverage.check_surface` now goes through `surface.fingerprint_at(...)` so the flag has one route.+- **`_check_open_node_addressing`** reads `before.document_root_is_open` at index 0.+- **`diagnose_unexplained_paths`** takes the `surface`; `runner._diagnose` passes it.+- **The document generator** (`tests/.../document_strategies.py`) seeds from `starting_document(surface=…)` — the reference document for a copied surface — translates the open **root** in `fingerprint_at`, asks `surface.validate_document` rather than `config_model`, and **checks each drop** for a non-layered surface (`_drop_if_accepted`).+- **Tests:** new `tests/unit/pipelex/cogt/model_backends/test_model_spec_document.py`; open-root cases in `test_fingerprint.py`; three cases in `test_ledger_check.py` (`*` at an open root is legal, naming one root table is `CONCRETE_KEY_UNDER_OPEN_NODE`, `*` at a *closed* root is still refused — that last one is the mutation guard); two in `test_diagnosis.py` (a header key is admitted, a typo is still named); `test_real_surfaces.py` gained the fourth id, the per-kind defaults-layer rule and an open-root assertion; `test_migration_run.py::TestTheWalkIsNotRecursive` flipped to the new rule and gained "a directory no surface owns is never entered".+- **Ledger + goldens:** `ledgers/inference-backend.toml` at `current_schema_version = 1`, zero entries; `goldens/inference-backend/[email protected]` (20 paths) + `[email protected]`. The fingerprint was **read by eye**: `*` is a table with `required: true`, every blueprint field is `*.<field>` with `required: false`, there is no phantom parent, and there is no `prompting_target` — which is what makes the Phase C entry pre-history.+- **The walk, over the real kit directory** (second commit): new `tests/unit/pipelex/migration/test_backend_directory_walk.py` (`TestAFreshMachinesBackendDirectory`) copies the kit's backend directory into a temp configuration root the way `pipelex init` does, then asserts three things over the **real** registry and the **real** run — every `.toml` is claimed by `inference-backend` and the `.md` files by nobody; `migrate_config_directories(dry_run=True)` finds every one of them and reports each clean; and a planted misspelling (`maxx_tokens`, in a new root table) *is* reported, at `*.maxx_tokens`. A separate module rather than a fifth class in `test_migration_run.py`, per the one-TestClass-per-module rule.+- **Docs** (second commit): `docs/migration-ledger.md` § *The fingerprint* gained the open-root blockquote (why a root and not a field, what paths that produces, and why the flag is recorded on the fingerprint rather than read from the surface); § *The defaults layer* gained the copied-not-layered paragraph (no defaults layer, none faked, and what absorbs an additive change instead). Three coherence fixes came with them, because the two new paragraphs would otherwise contradict text a few lines away: the § Surfaces table gained its fourth row, the § Surfaces sentence enumerating the defaults-layer kinds became three, and § Transform goldens' "read the way a user's file is read — beneath the current defaults layer, not alone" now names both readings. The rest of the Phase E docs list is untouched.++**Did the registry-driven machinery absorb the fourth surface without edits? Yes where it counts, no where the document shape is genuinely new — and the split is worth reading, because it is what a *fifth* surface would cost.**++- **The four non-test callers of `build_config_surface_registry()` needed zero edits** — the three dev-CLI commands (`check_ledger`, `check_migration_schemas`, `update_migration_schemas`) and `migration/run.py`, which is what both `migrate` commands, the `doctor` row and the boot diagnosis all go through. That is the R10 assumption, and it held exactly: a fourth `Surface` plus a fourth ledger entered every consumer by data.+- **Every module that did change, changed because a document root can now be open or a defaults layer can now be unlayered — never because a surface was added.** `fingerprint.py` (the open-root seed and the recorded flag), `ledger_check.py` (`_check_open_node_addressing` reads the root), `transform_check.py` (acceptance goes through `Surface.validate_document`, and a non-layered document is read on its own), `diagnosis.py` + `runner.py` (the surface reaches the diagnosis — F7), `coverage.py` (one line, routing the flag through `fingerprint_at`). A fifth `WHOLE_DOCUMENT` surface with a packaged defaults layer would need none of them.+- **Two modules outside the migration package:** `config_surface.py` for the surface-id constant (the one place the ids are spelled), and `model_spec_document.py` (new) + one literal in `backend_library.py`, because *what a backend document is* had to be stated where both the loader and the gate read it. That is D3's deliberate cost.+- **The three existing head goldens gained one line each** (`"document_root_is_open": false`), written by `umig`, and no frozen link moved. The two test exceptions F8 predicted were the only real-surface tests that had to change.++**Findings from Phase B that its own design got wrong (the record, so nobody re-derives them):**++- **B1's seed spelling was wrong in one argument, and it matters.** `ancestry=(config_model,)` makes `_record_field`'s recursion guard fire immediately — `*` is recorded as a table with **nothing beneath it**. The seed must pass `ancestry=()`; `_record_field` then walks with `ancestry=(config_model,)` itself. There is a comment at the call site saying so.+- **B3's open question is answered, and the answer was better than the prediction.** `cmig` stayed **green over the three existing surfaces before any regeneration** — `_check_head_link` compares *parsed* fingerprints, so a stored golden lacking the new key parses with the default and equals the live one. `make umig` then added exactly one line (`"document_root_is_open": false`) to each of the three head goldens, needed no `--force`, and left every frozen link untouched. So the format move is an addition in the strictest sense.+- **The projection cannot be a subclass without one type-checker suppression, and both checkers object.** Widening `sdk` in a subclass is a real Liskov violation (`reportIncompatibleVariableOverride` + mypy `[assignment]`), so the line carries both ignores and a docstring paragraph saying why the alternatives are worse: a hand-written twin reopens *annotation* drift (a name-comparison test cannot see a field's type move, and the fingerprint would keep recording the old one — the exact narrowing the gate exists to catch, going quiet), `create_model` drops the per-field metadata the fingerprint reads, and a placeholder default (`sdk: str = ""`) buys silence by putting a value in the model that no file has.+- **The replay property was vacuous for the new surface, and fixing it was real work.** `test_the_generator_does_not_merely_re_emit_the_reference_document[inference-backend]` failed with `NoSuchExample`, because the generator seeds from `read_defaults_document()` — `{}` here. Three consequences the plan did not anticipate: the seed must be the *reference* document; `fingerprint_at` must translate the open **root** (`gpt-4o.max_tokens` → `*.max_tokens`); and **a drop must be checked** for a non-layered surface, because nothing restores it and dropping `[defaults] sdk` invalidates every model in the file. `test_every_generated_document_is_valid_at_the_current_schema` also had to stop validating the whole document against `config_model` (it would be `extra_forbidden` on every model table) and ask the surface instead.+- **Three guards were mutation-tested, all without git** (the harness held the original text in memory and wrote it back): widening `admits_unnamed_key` to "any key under a model table" reddens the typo test; *narrowing* it to `return False` reddens two of the three new walk tests, reporting every `x-portkey-*` header in the kit's `portkey.toml` as an unexplained `*.x-portkey-…` path — which is exactly the page a fresh machine would have been greeted with; and dropping the `sdk` override would put the blueprint's `required=True` back into `model_fields`, which is what the projection guard reads.+- **The convergence gate never asked the diagnosis anything, and that gap was real rather than theoretical.** `_check_convergence` compares a witness's text before and after a replay; nothing in it reports an unexplained path. So the header-admission seam (F7) was, until the walk test, proved only over the two synthetic cases in `test_diagnosis.py` — and what it protects is every request header in a real shipped file. The walk test is where the whole chain is exercised at once: real registry → real ledger → real fingerprint → real diagnosis.+- **The drift contract `config-docs` opened and was reviewed for real** (ack text in `.drift/acks/config-docs.toml`). One user-facing sentence was genuinely stale: `docs/configuration/index.md:62` said the command walks the two configuration directories, true only while every migratable file sat *directly* in one. It gained a clause about `inference/backends/` pointing at `docs/configuration/config-technical/inference-backend-config.md` — whose own section is Phase E.++**What is NOT done and must not be assumed:** **nothing is pushed, and no PR exists** — the push is Phase F's first line and Louis' go. The workspace plan docs (`wip/migrator-3/{sequencing,backend-config-surface,README,plan}.md`, in the workspace repo, not this one) are updated as CP3 requires but **deliberately not committed** — that is Louis' call. The protected `~/.pipelex/inference/backends/` specimen has still never been edited, reset, or migrated for real; only the CP3 `--dry-run` above was taken, and it wrote nothing. S7's release step, which is where that directory finally gets migrated by the tool, is a different session in a different venue.++**Gates at the CP1 commit:** `make agent-check` clean (pyright 0 errors, mypy clean over 2524 files, `cl` 4 ledgers legal and convergent, keyword-only PASSED, hub-layering PASSED, drift PASSED with no new obligation — nothing in `drift.toml` triggers on `pipelex/migration/**`, and this commit touches no configuration model); `make cmig` 4 surfaces accounted for; `make tb` 46 passed; `make docs-check` builds with no new anchor warning (which is how the three new in-page links were verified); **full `make agent-test` green**. `git status --short` empty, nothing pushed.++**Preconditions as re-measured on 2026-08-17 (Phase 0, read-only):**+- `git rev-list --left-right --count origin/dev...HEAD` = `0 2` — `dev` has not moved from `61c4786b6`; the branch carries `583f50de7 tidy` and `343bb3fa7 plan` (this file). `git status --short` empty.+- `grep -rn prompting_target ~/.pipelex/inference/backends/` still finds it: the `[defaults]` block of thirteen backend files there, plus five per-model occurrences in `portkey.toml`. **The premise is alive.**+- The loader probe from the substance § *Reproducing the evidence*, run from `.venv/bin/python` against the two real files, reproduces both halves exactly: `openai.toml` → `extra_forbidden ('prompting_target',)` on **all** of its model tables (every table fails, because the key sits in `[defaults]` and is merged into each); `portkey.toml` → the same on all of its tables, **plus** five `NOT_HEADER_SHAPED` rejections carrying the full remedy sentence.+- `~/.pipelex/inference/backends/` also holds `pipelex_inference.toml` (not kit-shipped) and two `.md` files — both specimens are real on this machine.+- **The registry-driven claim holds** (the assumption R10's cost estimate rests on). `build_config_surface_registry()` has exactly four non-test callers: the three dev-CLI commands (`pipelex/cli/dev_cli/commands/{check_ledger,check_migration_schemas,update_migration_schemas}_cmd.py`) and `migration/run.py`, which is what both `migrate` commands, the `doctor` row and the boot diagnosis go through. Each reaches the registry plus one top-level function. A fourth `Surface` plus a fourth ledger enters all of them without an edit — with the test-side exceptions in F8 below.++**Rulings taken — all four RATIFIED by Louis at CP1 on 2026-08-17, as built. Each carries the measurement that supported it; none is open.**+- **D1 — the surface claims every `*.toml` directly in `inference/backends/`, and nothing else.** Not `inference/backends.toml`: it sits one level up, it is a different shape (a table per backend, not per model), and `#1104` never touched it. The `.md` files fall out by extension. Reason is in the registry docstring, beside the surface.+ - *Measured:* `TestAFreshMachinesBackendDirectory` copies the real kit backend directory into a configuration root and the real registry claims every `.toml` in it for `inference-backend` and nothing else — the two `.md` files included, by extension and not by an exclusion list. `test_migration_run.py::test_a_directory_no_surface_owns_is_never_entered` pins the other side: `inference/backends.toml` and `inference/deck/x.toml` are claimed by nobody.+- **D2 — `inference/deck/` and `inference/routing_profiles.toml` do not join.** The deck already has a manifest-driven sync with the `x_custom_*` user-owned carve-out, which the migrator must not duplicate; routing profiles have neither mechanism and no unreleased break. Reason is in the registry docstring beside the surface, not only in the wip document.+ - *Measured:* the same "a directory no surface owns is never entered" test, over the real registry. The cost of the ruling is also known and named: a file in either place is outside every migration, so a future break there is a new surface and not a widened glob.+- **D3 — the shape, as designed against the code (see F4–F9):**+ - a third `DefaultsLayerKind`, **`COPIED_DOCUMENT`**: *a document shipped in the package and **copied** into the user's directory rather than merged beneath it; the values come from the model's own field defaults, applied to each entry as it is read.* It names the contrast with `PACKAGED_DOCUMENT` (merged beneath) in one word.+ - `reference_document_path` = the kit's **`portkey.toml`**. It is the one shipped backend file that exercises both halves of the document shape — a `[defaults]` block *and* header-shaped per-model keys — and it is the file whose per-model tables carried `prompting_target`, so it is the richest starting document a later non-pre-history entry could be replayed over. (`openai.toml` is the more typical file and would also work; rich was preferred, and ratified.)+ - `reference_documents()` = the reference document plus every other `*.toml` in the kit's backend directory, as convergence/neutrality witnesses.+ - `read_defaults_document()` → `{}`: there is no defaults *document* for this surface and one must not be faked. The defaults-layer rule holds because the projection has no required path, not because a document supplies one.+ - the document shape: `Surface.document_shape` (StrEnum, `WHOLE_DOCUMENT` for today's three, `MODEL_SPEC_TABLES` for this one) drives three things — the open-root fingerprint seed, `validate_document`, and `admits_unnamed_key`. `SurfaceFingerprint.document_root_is_open: bool = False` records the fact in the golden so a frozen link answers for itself.+ - the projection: `InferenceModelSpecFileNode`, in `model_backends/` beside the blueprint — the blueprint with `sdk` optional, nothing else changed, used **only** by the fingerprint. Its guard is a test that no field of it is required, which is what goes red the day a new required field lands on the blueprint.+ - `validate_document` for `MODEL_SPEC_TABLES` is **faithful to the loader**, not to the projection: pop `[defaults]`, split header-shaped keys off each model table, merge, validate against the real `InferenceModelSpecBlueprint`. One helper in `model_backends/` so the loader and the surface cannot drift.+ - *Measured, as built:* `_check_open_node_addressing` needed exactly the one extension F4 predicted and nothing else, and `*` at a *closed* root is still refused (the mutation guard in `test_ledger_check.py`). The defaults-layer rule is vacuous exactly as F6 said, and `test_real_surfaces.py` states it as `check_defaults_layer(...) == []` rather than through a truthiness proxy. The golden-format move was an addition in the strictest sense (B3: `cmig` green before regeneration; `umig` added one line to each head golden, no `--force`, no frozen link touched). `transform_check` now reads a copied document **on its own** rather than beneath a defaults layer it does not have. And the whole chain — registry, ledger, fingerprint, diagnosis — converges on the real kit backend directory, which the convergence gate alone never proved (it never asks the diagnosis anything).+- **D4 — the `pipelex-config` pre-history entry ships in this session**, with the reshape renumbered to `@3`.+ - *Ratified without a measurement behind it, because it is Phase C's first act — the one ruling nothing had been built against at CP1.* Its cost is known and stands: F10, one hand-edit of the `"schema_version"` line inside each moved fingerprint golden, because `umig` reads the stored head before it writes and would raise on a body disagreeing with its filename. C2 is where this ruling is executed.++**Findings from Phase 0 that the plan did not anticipate:**+- **F4 — `_check_open_node_addressing` refuses `*` at the root today**, by construction: `parent_record = before.paths.get(parent_path) if index else None`, so index 0 has no parent record and a leading `*` is reported `WILDCARD_NOT_AT_OPEN_NODE`. Its message already renders `parent_label` as *"the document root"*, so the extension is the one the code was written for: at index 0 the parent is the document root, and its openness is `before.document_root_is_open`. Bonus, and it is the behaviour we want: with the root open, an op naming a concrete model (`openai.sdk`) becomes `CONCRETE_KEY_UNDER_OPEN_NODE` — a backend ledger may not address one user's model by name.+- **F5 — `delete_key` with `table_path = ["*"]` does reach every root table.** `_expand_table_paths` seeds `[[]]`, and `_resolve_table` with an empty path returns the document itself, so the wildcard expands over every root key whose value is a table — `[defaults]` and every model entry alike. Proven by test in C1, not taken on trust.+- **F6 — the defaults-layer rule is vacuous for an open root, for free.** `check_defaults_layer` skips any path with a `*` segment, and every path of this surface has one.+- **F7 — the diagnosis needs one seam, and this is the item the plan missed.** An unknown *root* key resolves for free: `_resolve_against_schema` already falls back to the `*` child at the empty prefix, so `gpt-4o` lands on `*` with no change. But a **header-shaped per-model key** (`x-portkey-provider`, in the kit's `portkey.toml` and in every user's) resolves to neither `*.x-portkey-provider` nor `*.*`, so it would be reported `UnexplainedPath` — on **every** `pipelex migrate` run, since `migrate_file` diagnoses every file — and it would fail `test_no_surfaces_own_documents_carry_anything_the_diagnosis_cannot_explain` over the witnesses. Remedy: `diagnose_unexplained_paths` takes the surface (the runner already holds it at the call site) and asks `surface.admits_unnamed_key(node_path=…, key=…)` before reporting. For `MODEL_SPEC_TABLES` that is *the node is a model-spec table (`("*",)`) and the key is header-shaped with a legal header name* — deliberately narrower than "the model allows extras", so a genuine typo (`promting_target`) is still named. **Rejected alternative:** a per-record `accepts_unnamed_keys` on `PathFingerprint`. It is too coarse — it would silence exactly the key class this session exists for — and it would add a line to all ~200 records of every head golden.+- **F8 — two real-surface tests must change; they are the named exceptions to "absorbed without edits".** `test_the_registry_loads` pins the exact surface-id list, and `test_every_surface_has_a_reachable_defaults_layer` asserts `read_defaults_document()` is *truthy*, which is false for a surface that honestly has no defaults document (D3). The second is amended to state the rule per kind rather than by a truthiness proxy. `test_replay_properties.py` builds `REAL_SURFACES` at module level and the fourth surface joins its property runs automatically — watched, not pre-emptively changed.+- **F9 — there is no packaged (non-kit) backend directory** (`pipelex/inference/` does not exist), so both the reference document and the witnesses come from the kit. The kit's `pipelex_gateway.toml` is **comments only** — zero paths — which makes it a free witness and a poor reference document.+- **F10 — the renumber needs one hand-edit per moved golden, which the plan's `git mv`/copy recipe does not mention.** A fingerprint golden carries `"schema_version"` in its body, and `read_fingerprint_golden` refuses a body that disagrees with its filename. `umig` reads the stored head *first* (the destructive-overwrite guard), so a moved `[email protected] → @3.json` still saying `2` makes `umig` raise before it writes. So: move, hand-edit the one version line, then `umig` must be a byte-pure no-op — and that no-op is the proof. Same for the `@1 → @2` copy, which is frozen and never regenerated.+- **F11 — the loader reads the files `backends.toml` names, not a directory listing** (`f"{backends_dir_path}/{backend_name}.toml"`). So the boot-tolerance retry repairs what the loader *loaded*, while the walk repairs everything it *reaches* — the sentence Phase D owes a docstring. (`backends_dir_path` is a `str` in that API; the retry needs a `Path` at the edge.)+- **F12 — whichever kit backend file becomes the reference document is now coupled to the goldens.** `_check_head_defaults_document` byte-compares `[email protected]` against it, so adding a model to that file turns `cmig` red until `make umig` runs. That is the designed workflow and the message says so, but the `add-model` skill should say it too (Phase E).++**What Phase A actually built (`feae4db1a`), so the next session can read the diff rather than re-derive it:**++- `Surface` gained `subdirectory: Path = Path()` (`Path()`, `Path("")` and `Path(".")` are all `PosixPath('.')`, and `directory / Path()` is `directory` — verified, so one neutral spelling and no special-casing). `base_file` became `str | None`, with a `check_the_surface_claims_some_file` validator refusing a surface that has neither a base file nor a glob.+- `surface_for_file_name(file_name=…)` → **`surface_for_file(subdirectory=…, file_name=…)`**. Only surfaces owning the file's directory are candidates; exact names still claim before globs *within* that directory. `files_by_surface_in_directory(directory=…)` keeps its signature (the argument is still the configuration root) and now iterates `sorted({surface.subdirectory …})`, skipping a subdirectory that does not exist, one level, `is_file()` only, `sorted` within each.+- Duplicate rejection is now per directory: `_reject_duplicates_per_directory` on `(subdirectory, base_file)` and `(subdirectory, tier_glob)`; surface ids stay globally unique. So two surfaces may both glob `*.toml` when they own different directories.+- `SurfaceBlock` gained `subdirectory: str = ""` and `base_file: str | None`, with the same claims-something validator. `coverage._check_ledger_agrees_with_registry` cross-checks the directory alongside id/base_file/tier_glob, comparing `as_posix()` on both sides so `""` and `"."` cannot read as a disagreement.+- `config_surface._is_within` now takes the **plan**, not a path, and asks `_subdirectory_owned_by(surface_id=…)`, which reads `load_ledger_cached(...).surface.subdirectory` and falls back to `Path()` on `MigrationLedgerError`. The kernel-layer rule holds: `config_surface.py` still does not import `surfaces.py`. Reach is per surface, so a `telemetry.toml` under `inference/backends/` stays out of reach.+- Tests: new `tests/unit/pipelex/migration/test_surface_directories.py` (`TestADirectoryIsHalfTheClaim`) holds the collision rule; `test_surface_registry.py` gained the two glob-scoping cases and the claims-nothing refusal; `test_coverage_gate.py` gained `test_a_ledger_describing_another_directory_is_refused`; `test_boot_tolerance.py` gained the in-reach / out-of-reach pair against a synthetic ledger that declares a subdirectory, and `write_synthetic_ledger` gained a `subdirectory=` parameter. `conftest.py` exports `CONFIGURATION_ROOT = Path()` (a module constant, because pyright's `reportCallInDefaultInitializer` refuses `Path()` as a *parameter* default — it is fine as a pydantic field default).+- **Both guards were mutation-tested.** Making the claim rule name-only again (`candidates = list(self.surfaces)`) reddens three of the four tests in `test_surface_directories.py`; hardcoding `subdirectory = Path()` in `_is_within` reddens `test_a_file_in_the_directory_its_surface_owns_is_in_reach`. Neither mutation was applied with git — the harness held the original text in memory and wrote it back.+- Docs: `docs/migration-ledger.md` § Surfaces gained a **Directory** column and the (directory, name) claim rule replacing the exact-before-globs quote; § The commands' "the walk is not recursive" paragraph became "each surface's own directory, one level deep"; § Boot tolerance gained the reach-is-per-surface sentence; the registry-vs-ledger paragraph now says why the directory has a second reader. `run.py`'s module docstring rewritten the same way.+- The three shipped ledgers needed **zero edits** and `make cl` / `make cmig` stayed green — the empty-subdirectory default is what buys that.++**Gates at the Phase A commit:** `agent-check` clean, `cl` + `cmig` green, `tb` green, targeted suite green (`tests/unit/pipelex/migration/`, `tests/unit/pipelex/system/`, `tests/unit/pipelex/cli/`, `tests/e2e/pipelex/cli/`, `tests/integration/pipelex/system/` — 1674 passed), `docs-check` builds, `drift-check` green after acking `config-docs` (the ack text is in `.drift/acks/config-docs.toml`). The **full** `agent-test` has not been run yet; it is owed at CP1.++**One local-environment event, done on Louis' explicit go:** `make tb` and `tests/integration/pipelex/system/test_load_config.py` were failing **at HEAD, before any of this work**, because this worktree's own gitignored `_backend/.pipelex/pipelex_override.toml` was still in the pre-reshape `[pipelex.*]` shape (proved by swapping the five source files back to their `HEAD` text and re-running). `pipelex migrate --yes` was run once; it rewrote exactly that one file, left `pipelex_override.toml.bak.20260817T113323Z` beside it, and touched nothing in `~/.pipelex/` (already at the current schema) — so the protected `~/.pipelex/inference/backends/` specimen is **untouched and still stale**, as S7's release step needs. `.gitignore` gained `*.bak.[0-9]*Z` so the timestamped copy `pipelex migrate` leaves in a project's `.pipelex/` is never committed — by us or by a user.++**B1–B9 — Phase B designed against the code (the reading pass is done; these supersede the prose in the Phase B section below where they differ):**++- **B1 — the open-root seed.** `compute_fingerprint` seeds `_walk_model(prefix=())` from `config_model.model_fields`. For an open root, call `_record_field(path=(WILDCARD_SEGMENT,), annotation=config_model, required=True, defaults_value=None, ancestry=(config_model,), collected=collected)` instead: because `config_model` is a `BaseModel`, `_record_field` writes `collected["*"] = PathFingerprint(value_type=TABLE_TYPE, required=True)` and then walks its fields under the `("*",)` prefix, giving `*.sdk`, `*.enabled`, … with **no phantom parent**. That is exactly the shape D3 asks for, and it needs no new branch inside `_record_field`. Add `document_root_is_open: bool = False` as a parameter of `compute_fingerprint` and as a field of `SurfaceFingerprint`; there are only two non-test callers, `Surface.fingerprint_at` and `coverage.check_surface:456`, and the latter should be switched to `surface.fingerprint_at(schema_version=current_version)` so the flag has one route.+- **B2 — `_check_open_node_addressing` (F4) needs one edit.** Line ~395: `parent_record = before.paths.get(parent_path) if index else None`. Replace the two lines with `if index: parent_record = …; parent_is_open = parent_record is not None and parent_record.open_node` / `else: parent_is_open = before.document_root_is_open`. The message already renders `parent_label` as *"the document root"* at index 0, so nothing else changes. Red-first in `test_ledger_check.py`.+- **B3 — the golden format moves for every surface, and that is expected.** `render_fingerprint_golden` dumps with `exclude_none=True`, **not** `exclude_defaults`, so a new `bool` field lands in every head golden as `"document_root_is_open": false` the first time `umig` runs. That is an *addition*, and `_refuse_a_destructive_head_overwrite` only refuses on removals and narrowings — so no `--force` is needed. Confirm `_check_head_link` compares parsed fingerprints (a stored golden lacking the key parses with the default) rather than text, and expect the `umig` diff to touch the three existing head goldens by one line each. **Measure this before writing the ledger**, and if `cmig` goes red on the three untouched surfaces, that is the thing to fix, not to regenerate around.+- **B4 — `Surface.validate_document(*, document: dict[str, Any]) -> str | None`.** Returns the reason the current schema refuses the document, or `None`. **Returning a reason rather than raising is deliberate: it avoids a new error class, and therefore avoids `make gei` + `make gep` and the full-`agent-test`-only failure they cause.** `transform_check._check_the_migrated_document_is_accepted` (line ~226) calls it in place of `surface.config_model.model_validate(document)` and builds its `MIGRATED_DOCUMENT_REJECTED` issue from the returned string.+- **B5 — the same function must not deep-merge for a copied document.** `_check_the_migrated_document_is_accepted` loads `after_text` (the `defaults@N` reference) and `deep_update`s the migrated text over it, because a migrated file is read *beneath* its defaults layer. A copied-document surface has no layer beneath it, and merging two independent backend files would validate a hybrid nobody has. Give `DefaultsLayerKind` a property — `is_layered_beneath_the_users_file` — returning `True` for `PACKAGED_DOCUMENT` and `MODEL_DEFAULTS` and `False` for `COPIED_DOCUMENT` (exhaustive `match`, no `case _`), and skip the base load when it is `False`.+- **B6 — the loader-faithful validation, and where it lives.** `backend_library.load` reads one file, pops `[defaults]`, and for **each** remaining root table runs `split_model_spec_keys` (rejected keys are fatal for `LOCAL_FILE`) then `InferenceModelSpecBlueprint.model_validate({**defaults, **key_split.fields})`. Note the asymmetry that makes the `#1104` breakage double: **`[defaults]` is *not* passed through `split_model_spec_keys`** — it is copied wholesale into every model's blueprint dict, which is why one `prompting_target` there fails *every* table with `extra_forbidden`, while a per-model one is rejected by name as `NOT_HEADER_SHAPED`. Put both the projection and the validator in a new `pipelex/cogt/model_backends/model_spec_document.py`, so the loader and the surface cannot drift.+- **B7 — the projection is a one-line subclass.** `class InferenceModelSpecFileNode(InferenceModelSpecBlueprint): sdk: str | None = None`. Subclassing makes the field-set drift test true by construction (nothing to keep in sync), and the guard that matters is the one D3 names: **no field of it is required** — which goes red the day a new required field lands on the blueprint. `sdk` still renders as `str` in the fingerprint, since `_strip_annotated` drops the `| None`; only `required` moves.+- **B8 — the diagnosis seam (F7).** `diagnose_unexplained_paths` gains a `surface` parameter (its one non-test caller, `runner.py:124`, already holds one; `test_real_surfaces.py:48` and `test_diagnosis.py` call it directly and must be updated). In `_diagnose_table`, when `_resolve_against_schema` returns `None`, ask `surface.admits_unnamed_key(node_path=schema_prefix, key=key)` before reporting. For `MODEL_SPEC_TABLES` that is `node_path == (WILDCARD_SEGMENT,) and is_header_shaped(key=key) and is_legal_header_name(key=key)` — deliberately narrower than "the model allows extras", so `promting_target` is still named while `x-portkey-provider` is not. A hyphenated known field (`max-tokens`) is admitted here on purpose: the loader already rejects it by name with the right advice, and this channel has none to add. Confirmed by reading: a root **model name** (`gpt-4o`) already resolves for free, because `_resolve_against_schema` falls back to the `*` child at the empty prefix.+- **B9 — the kit witnesses.** `kit_template_path: Path | None` gains a sibling `kit_template_dir: Path | None` (mutually exclusive, enforced by a validator), and `reference_documents()` enumerates that directory lazily — `is_file()`, matching the surface's own `tier_glob`, **excluding `reference_document_path` itself** so the reference document is not witnessed twice — labelling each `kit template '<name>'`. Lazy rather than a `list[Path]` resolved at registry-build time, because building the registry must not touch the filesystem. `_check_convergence` (`ledger_check.py:595`) is the only reader.++**Commits on the branch:** `583f50de7 tidy`, `343bb3fa7 plan`, `34fefae4a S7b Phase 0`, `feae4db1a Migration walk: a surface owns a directory, and a file is claimed by (directory, name)`, `7a57f8dc5 S7b pause after Phase A`, `4237a51c5 The inference-backend surface: an open-root document, a copied-document defaults layer, schema 1 cut`, `81554b3bd inference-backend@2: delete prompting_target from every root table (pre-history)`, `c765430d4 pipelex-config@2: the prompting-style half of #1104 accounted for; the reshape becomes @3`, `c2a104a7a CP2 cleared: the two pre-history entries are built, and Phase D is next`, `b7f093c31 A stale backend directory boots with a warning and pipelex migrate repairs it in place`, `20d10c853 Phase D recorded: the fourth loader tolerates, and Phase E is next`, plus the Phase E commit below.++**Open questions found while building:**++- **B3's golden diff is the one unmeasured thing in the Phase B design.** *(Answered in Phase B: `cmig` stayed green over the three existing surfaces before any regeneration, and `umig` added one line to each head golden with no `--force`. Kept as the record of the prediction.)*+- **Whether `pipelex_override.toml.bak.…` should be pruned.** `pipelex migrate` left one in this worktree's `.pipelex/`; it is now gitignored rather than deleted, and the rotation/pruning behaviour of `backup.py` was not looked at this session. Not S7b's business unless a reviewer raises it.+- **A machine two entries behind on `pipelex-config` gets one backup, not two, and that is a claim about the runner rather than about the ledger.** The C2 e2e asserts exactly one backup after a run in which two entries applied to one file, which is right — a run writes a file once — but nothing in this session looked at what a *second* run a week later does to that backup, or whether backups accumulate per run. Same territory as the item above; a reviewer may join them.+- **A second backend directory is repaired by the command but never by the boot, and nobody is told which.** `backends_dir_path` resolves per directory — a project's `.pipelex/inference/backends/` wins the *whole* directory over the global one — so a machine with both loads one and walks two. The warning names the files the load merged, which is right (it is a warning about what booted), and `pipelex migrate` repairs both, which is also right. What is not decided is whether the warning should *say* that the command reaches more than it names. Phase D put the asymmetry in the `load` docstring and left the message alone; a reviewer may want it in the message.+- **The doctor emits the fourth surface's warning once more than a boot does.** `check_models` calls `ModelManager.setup`, so `pipelex doctor` on a stale machine prints the stale-configuration warning *and* shows the pending-migrations row. Arguably right — they say different things — but nobody chose it deliberately, and `pipelex doctor --fix` therefore warns before it offers the fix.+- **The `pipelex-config@2` guidance says a tuned `prompting_styles` map is dropped, and nothing tells the user what was in theirs.** The standing rule is that no value from a user's file is ever rendered, so the map cannot be echoed — but the backup beside the file has it, and the guidance does not say so. *(Phase E answered the half it could without touching a shipped message: the **changelog** bullet now says the backup is where the old map remains readable. The ledger entry's own `guidance` string is still silent, and changing it is still a wording call on a shipped message — left to a reviewer.)*++**Open questions Phase E found, none of which it resolved. These are the three a reviewer is most likely to raise, so the PR description should name them first.**++- **`check-config-sync` excludes by exact file name, and a migration backup's name contains a timestamp — so it can never be listed.** `pipelex migrate` writes `<file>.bak.<UTC stamp>` beside every file it rewrites, and `.gitignore` has covered that shape since Phase A (`*.bak.[0-9]*Z`), but the sync checker compares two directories **on disk** and does not read `.gitignore`. So any contributor who runs `pipelex migrate` inside a pipelex checkout fails `make check` afterwards — and **the failure's own recommended action makes it worse**: it says *"If .pipelex is correct, run make ukc"*, which would rsync their timestamped backups into `pipelex/kit/configs/`, from where they would be committed and shipped in the package. S7b makes this likelier rather than causing it (more files migrated, more backups, and now in `inference/backends/` too). The fix is pattern exclusion — `has_diff_dirs` takes only `exclude_files: AbstractSet[str]` today — and the reason it was **not** taken in Phase E is a real design question rather than timidity: the backup-name shape is already spelled in two places (`BACKUP_INFIX` + `_BACKUP_STAMP_PATTERN` in `pipelex/migration/backup.py`, and `.gitignore`), and `pipelex/kit/paths.py` cannot simply import the first — `backup.py` reaches `pipelex.pipeline.fixes.file_transaction`, the interpreter layer. So a third spelling, or a layering decision, is owed. Not S7b's, but S7b's neighbour.+- **A backend file carrying the reserved `[meta]` table would be refused by the loader, though the migration engine reads the key there.** Every other configuration surface's loader strips `[meta]` through `config_surface`; the backend loader reads through the generic `load_toml_from_path` and would see `[meta]` as one more root table — that is, as an unknown model spec. The contract says the key is reserved on every configuration surface, and the fourth surface is the one where that is not yet true. **Nothing writes the key on any surface today** (the changelog says so explicitly: it exists so a later release can stamp a version without that being breaking), so this is latent and not live — but it becomes live the day anything stamps one, and the wildcard `["*"]` in `inference-backend@2` would also expand over a `[meta]` table if one existed. The docstring in `config_surface.py` now records the asymmetry rather than denying it.+**Open questions the CP3 review added. None is S7b's to fix, and each was measured rather than suspected.**++- **A blocked `pipelex-config@2` has already deleted the table its move was supposed to empty.** `pipelex-config@2` is the first shipped entry whose ops are a `move_key` out of a table followed by a `delete_table` of that table, and the engine commits whichever ops applied even when one conflicts (`_apply_safe_entry` returns `application.text`, and the runner writes on `did_change_document`). Reproduced: a half-hand-migrated file — the new `[pipelex.templating_config].default_templating_style` already set, the old `[pipelex.prompting_config]` still present — comes out with the old section gone while the entry is reported blocked. **It is milder than it first looks, and the report is why:** `_print_blocked_entry` lists every op that *did* apply, so the two deletes are named on screen; `needs_attention` is set; and the conflict is by definition "the destination is already present", so the source value lost is always the one the user superseded themselves, with the backup beside the file holding it. What remains is that the entry's `guidance` asserts *"`default_prompting_style` travels"* directly under a conflict where it did not. Either roll an entry back on conflict, or word the guidance so it does not promise the move. Pre-existing engine behaviour; this entry is the first shaped to expose it.+- **A retired field carrying `${VAR}` is expanded before the ledger can delete it.** `_load_local_model_specs` substitutes immediately after reading, and the retry only runs after the spec build fails — so a retired key holding an unresolvable variable becomes an `InferenceBackendCredentialsError`, fatal in strict mode and *silently skipping the whole backend* in lenient mode, with no stale warning parked at all. Not reachable for `prompting_target` (every kit file shipped it as a literal), but it is the shape to check the next time an entry retires a field that could hold a variable.+- **`pipelex doctor` on a stale machine now says it twice, and nobody chose that.** *(Carried forward from Phase D and still open.)* `check_models` calls `ModelManager.setup`, which is the one caller that emits the parked warning — so the doctor prints the stale-configuration warning **and** shows the pending-migrations row, and `doctor --fix` therefore warns before it offers the fix. Arguably right, since the two say different things; but it was a consequence, not a decision.----## Phase dependency map+## Phase 0 — re-measure, then read the seams (no code)-```-Phase 0 (OQ1 gate + corpus measurement)- |- | a negative OQ1 stops everything- v-Phase 1 (extract the pass — pure refactor) <-- safe handoff point- |- v-Phase 2 (flip the rule + strict lookup, together) <-- the branch's reason to exist- |- +----------------------------+- | |- v v-Phase 3 (concept side, Phase 4a (mthds/ clarification —- fix_loop) release-independent)- | |- +----------------------------+- |- v requires a published pipelex carrying the accessor- Phase 4b (spec suite + transport + cookbook/cocode + error sweep)-```+Read-only. Nothing is written but this file's Status block.-Lane A: Phase 0 → 1 → 2 → 3 (sequential — all touch `pipelex/libraries/`). Lane B: Phase 4a (independent once Phase 2 lands; touches `mthds/` and the workspace-level boundary spec). Lanes A and B share no module directory. Phase 4b joins both and is gated on a release, so it cannot run in parallel with anything.+- [x] Re-verify the venue: `git fetch origin dev && git rev-list --left-right --count origin/dev...HEAD` (expect `0 1` or `0 N` — never `M N` with `M > 0`; if `dev` has moved, `git merge origin/dev` first, never rebase, and re-run the gates before going on). `git status --short` empty.+- [x] Re-measure the breakage on this laptop, read-only: `grep -rn prompting_target ~/.pipelex/inference/backends/` finds it; run the loader probe from the substance § *Reproducing the evidence* against `~/.pipelex/inference/backends/openai.toml` and `portkey.toml` from `.venv/bin/python` — expect `extra_forbidden ('prompting_target',)` from the `[defaults]` merge and *"'prompting_target' is not a known model-spec field, and not header-shaped"* per per-model occurrence. **If a `pipelex init --reset` has been run since and the key is gone, find another stale specimen (a teammate's machine, or a `git show 60aa77f5^:pipelex/kit/configs/inference/backends/portkey.toml` copy planted in a temp HOME) before building against a dead premise — and say so in Status.**+- [x] Read, in this order, so the design items in Phase A/B are built against the code and not the plan: `pipelex/migration/surfaces.py` (the `Surface` model, `surface_for_file_name`, `files_by_surface_in_directory`, `build_config_surface_registry`), `pipelex/migration/run.py` (the "walk is not recursive" docstring — it becomes false), `pipelex/migration/fingerprint.py:160-311` (`compute_fingerprint` seeds `prefix=()`; open nodes are `dict[str, X]` fields recorded under a literal `*`; `extra="allow"` keys are invisible; a `RootModel` projects a phantom `root.` prefix), `pipelex/migration/ledger.py:43-66` (`SurfaceBlock`) and `:175-196` (id ↔ version ↔ `current_schema_version` are forced to agree), `pipelex/migration/coverage.py:143-146, 223-266, 415-446, 496-519, 522-584` (pre-history claim, defaults-layer rule, ledger-vs-registry cross-check on exactly `id`/`base_file`/`tier_glob`, stored links), `pipelex/migration/ledger_check.py:157-163, 224-315, 595` (pre-history entries skip the walk entirely; `_check_open_node_addressing`; `_check_convergence` is the only reader of `reference_documents()`), `pipelex/migration/transform_check.py:130-149, 226-255, 279-315` (how `before@N` and `defaults@N` are consumed; the head-only "migrated document is accepted" check — **this validates the migrated document against `config_model`, and an open-root document cannot be validated that way; see Phase B**), `pipelex/migration/snapshot.py:47-104` (`umig` writes the head pair only, never `before@N`), `pipelex/migration/runner.py:296-325` and `backup.py:41-95` (backups land beside the file via `with_name`, so a subdirectory needs nothing there), `pipelex/system/configuration/config_surface.py` (`replay_surface_files_in_memory` **deep-merges** its paths — right for a tier stack, wrong for many independent backend files; `_is_within` says "directly, not underneath"; `stale_configuration_warning`), `pipelex/system/configuration/config_loader.py:129-151, 193-201, 430-487` (`existing_config_dirs` — the one derivation; `backends_dir_path` is resolved **per directory, project wins the whole directory** — different from the tiered merge; the copyable four-step tolerance pattern), `pipelex/system/pipelex_service/pipelex_service_config.py:59-75` (the smallest tolerance caller — the closest template), `pipelex/cogt/model_backends/backend_library.py:58-247, 291-326` (`load`, `_load_local_model_specs`, where a fourth caller hooks in), `pipelex/cogt/model_backends/model_spec_factory.py:18-74` (`InferenceModelSpecBlueprint`: `sdk` is the only required field; `ConfigModel` is `extra="forbid", strict=True`), `pipelex/cogt/model_backends/model_spec_keys.py:1-31, 138-174` (pure, silent, runs before log dispatch on some paths), `pipelex/cogt/models/model_manager.py:64-84` and `pipelex/runtime_boot.py:200-215, 480-505` (the boot path and where the parked warning is emitted), `tests/unit/pipelex/migration/conftest.py` (the synthetic-surface harness: `build_surface`, `build_entry(pre_history=…)`, `build_registry`), `tests/unit/pipelex/migration/test_migration_run.py:96-104` (the test that pins the *current* non-recursive rule with `inference/backends/pipelex_gateway.toml` as SPECIMEN — it flips meaning in Phase A), `tests/unit/pipelex/system/configuration/test_boot_tolerance.py`, `tests/e2e/pipelex/cli/test_migrate_commands.py:354-400` (`TestAPreReshapeMachine`), `docs/migration-ledger.md` § Surfaces (`:15-40`), § The commands (`:513`), § Boot tolerance (`:548`), § Reporting… (`:560`), the chain diagram (`:446`).+- [x] Verify early the assumption R10's cost estimate rests on: **`cmig`, `umig`, `cl`, `pipelex migrate`, the `doctor` pending-migrations row, `/release` step 3b and the `add-migration` skill are all registry-driven** — grep confirms the dev CLI reaches only `build_config_surface_registry()` plus one top-level function each (`check_ledger_cmd.py`, `check_migration_schemas_cmd.py`, `update_migration_schemas_cmd.py`, `migration/run.py`). Record in Status whether that held (it is a **Record** obligation of the S7b row).+- [x] Write D1–D4 into Status as *recommended*, with one sentence each, before any code. Phase A/B build to the recommendations; CP1 is where they get ratified or overturned.-## GSTACK REVIEW REPORT+## Phase A — the walk reaches a subdirectory, and the claim rule becomes (directory, name)-| Review | Trigger | Why | Runs | Status | Findings |-|--------|---------|-----|------|--------|----------|-| CEO Review | `/plan-ceo-review` | Scope & strategy | 0 | — | — |-| Codex Review | `/codex review` | Independent 2nd opinion | 0 | — | — |-| Eng Review | `/plan-eng-review` | Architecture & tests (required) | 1 | CLEAR (PLAN) | 16 issues, 0 critical gaps left open |-| Design Review | `/plan-design-review` | UI/UX gaps | 0 | — | — |-| DX Review | `/plan-devex-review` | Developer experience gaps | 0 | — | — |+Pure registry/walk work; independent of D3. **The landmine test comes first.**-**CODEX:** outside voice ran with repo access and surfaced findings the plan-text review could not reach — batched sub-pipe code derivation producing `domain.foo_batch`, transported `PipeFunc` carrying bare codes into the strict lookup, an undefined crate fingerprint contract, dependency concepts losing package identity, aliased dependency keys polluting the crate-wide search, and a stale `(done)` marker on the final checkpoint. All folded into the phases above.+- [x] **Red first — the collision test.** In `tests/unit/pipelex/migration/test_surface_registry.py` (or a new `test_surface_directories.py`): a registry with a `pipelex_*.toml`-globbed surface at the root and a `*.toml`-globbed surface at `inference/backends`; a temp directory holding `pipelex.toml`, `pipelex_override.toml`, `inference/backends/pipelex_gateway.toml`, `inference/backends/openai.toml`, `inference/backends/notes.md`, `inference/deck/x.toml`, `inference/backends.toml`. Assert: the two backend `.toml`s are claimed by the backend surface and **not** by the root surface (the `pipelex_gateway.toml` specimen, by name a `pipelex_*.toml` match); `notes.md`, `deck/x.toml` and `inference/backends.toml` are claimed by nobody; the root files are claimed by the root surface only. Mutation-test it: make the claim rule name-only again and watch it go red on `pipelex_gateway.toml`.+- [x] `Surface` gains `subdirectory: Path` (relative; `Path()` / empty for the three existing surfaces — decide the neutral spelling once and use `Path(".")`-free comparisons; keep `frozen=True`, `extra="forbid"`). `base_file` becomes `str | None` (a family with no distinguished member has none) with a validator: at least one of `base_file` / `tier_glob` is set, and a surface with `base_file=None` **must** carry a glob.+- [x] `SurfaceRegistry.check_no_surface_shadows_another`: duplicates are now rejected on `(subdirectory, base_file)` and `(subdirectory, tier_glob)` pairs, not bare strings — two surfaces in different directories may share a glob shape. Keep the surface-id uniqueness.+- [x] `surface_for_file_name(file_name)` → `surface_for_file(*, subdirectory: Path, file_name: str)`: only surfaces whose `subdirectory` equals the file's are candidates; exact base files claim before globs **within that directory**; two globs claiming one file in the same directory stays a `MigrationRegistryError` by name (the contract's rule, unchanged). Update its one caller.+- [x] `files_by_surface_in_directory(directory)`: for each surface, enumerate `directory / surface.subdirectory` with `iterdir()` (still one level, still `is_file()` only, still "a directory that does not exist is skipped, not refused" — a machine with no `inference/` is an ordinary machine); a symlinked file is still enumerated by `iterdir` and written through (keep the S7 asymmetry: resolve the parent, never the file). Return order stays deterministic (`sorted`).+- [x] The ledger's `[surface]` block (`SurfaceBlock`) gains `subdirectory: str = ""` (TOML has no Path; convert at the edge) and `base_file` becomes optional there too; `coverage._check_ledger_agrees_with_registry` cross-checks the new field alongside `id`/`base_file`/`tier_glob` (it compares exactly those three today — extend the tuple, and add a red-first test in `test_coverage_gate.py` for a `subdirectory` disagreement). The three existing ledgers need **no** edit if the default is the empty subdirectory — verify `make cl`/`cmig` are still green with zero ledger changes before adding the fourth.+- [x] `_is_within` in `config_surface.py`: "directly in one of these directories" must become "directly in the directory *this file's surface* owns under one of these roots". Recommendation: `MigrationPlan` already carries `surface_id`; read the surface's `subdirectory` from the cached ledger (`load_ledger_cached(...).surface.subdirectory` — `config_surface.py` already imports `pipelex.migration.ledger`, and may **not** import `surfaces.py`, which imports every config model into the kernel layer's closure) and compare `path.parent.resolve()` against `root.resolve() / subdirectory`. `walked_dirs` stays the roots (`config_manager.existing_config_dirs`) for every caller, so the three existing loaders do not change and the "one derivation" property S7 fought for survives: the roots come from the loader, the subdirectory from the ledger the registry cross-checks. Red-first test in `test_boot_tolerance.py`: a stale plan for `<root>/inference/backends/openai.toml` is *in reach* under `walked_dirs=[root]`, and one for `<root>/inference/deck/x.toml` is not.+- [x] Rewrite the three places that state the non-recursive rule as a guarantee: `pipelex/migration/run.py:19-23` docstring, `config_surface.py:186-206` docstring, `docs/migration-ledger.md:513` (§ The commands) — the new sentence: *the walk is each surface's own directory under each configuration root, one level, and a subdirectory no surface owns (`deck/`) is never entered*; keep `pipelex_gateway.toml` as the specimen, now for the (directory, name) rule. Flip `tests/unit/pipelex/migration/test_migration_run.py:96-104` from "the specimen is never claimed" to "the specimen is claimed by the backend surface and by nothing else" — once the fourth surface exists (Phase B); until then it must keep asserting the root surface does not claim it.+- [x] `docs/migration-ledger.md` § Surfaces: describe `subdirectory` and the (directory, name) claim rule; § Legality → *Reserved paths and names* if it names the claim rule; the CHANGELOG bullet at `CHANGELOG.md:10` that says "not recursively, because a subdirectory such as `inference/backends/`…" is rewritten in Phase E, not now.+- [x] Gates 1–5 (the drift contract `config-docs` will open on `config_surface.py` — review `docs/configuration/**` and `pipelex/migration/ledgers/` for real, then ack). Commit: `Migration walk: a surface owns a directory, and a file is claimed by (directory, name)`.-**CROSS-MODEL:** one tension. The eng review treated the repo's written no-backward-compatibility principle as settling the breaking-change question and invested in the error message as the migration path; Codex argued the decision was unevidenced because the fall-through corpus excludes customer bundles. Resolved by holding the no-deprecation posture while adding a bounded Phase 0 corpus measurement — the principle governs whether to break, the measurement governs how well the break is communicated.+## Phase B — the fourth surface: model, fingerprint shape, defaults-layer kind, schema-1 cut-**VERDICT:** ENG CLEARED — ready to implement, starting at Phase 0.+This is the central modelling phase. Item 4 of the substance is the question; the recommendation below is a starting point to be **measured** against `compute_fingerprint`, `_check_open_node_addressing`, `_check_the_migrated_document_is_accepted`, and `read_defaults_document`, and overturned if any of them says no.-NO UNRESOLVED DECISIONS+**The document shape.** A backend file is `[defaults]` + one root table per model name (+ header-shaped keys inside a model table). Neither `[defaults]` nor a model table is an `InferenceModelSpecBlueprint` on its own — only their merge is (`sdk` is required and normally lives in `[defaults]`). So the fingerprint must record the *file-node* shape: every root table is an all-optional partial spec, and the root itself is an open node.++- [x] **The partial projection.** Recommendation: a mechanically-derived all-optional projection of `InferenceModelSpecBlueprint` (every field `X | None = None`, `extra="forbid"` irrelevant to the fingerprint since extras are invisible to it), produced by one function in the migration package (or beside the blueprint) so there is **one** field list — with a test pinning `set(projection.model_fields) == set(InferenceModelSpecBlueprint.model_fields)` so the two can never drift. Alternative if dynamic model creation reads badly under the python standards: an explicit hand-written `InferenceModelSpecFileNode` model beside the blueprint with the same drift test. Either way it is a **projection for the fingerprint and the transform check only** — the loader keeps validating the merge against the real blueprint; nothing about how a backend file boots changes here.+- [x] **The open root.** `compute_fingerprint` seeds `prefix=()` from `config_model.model_fields`, so a `dict[str, Partial]` has to be a *field* to be recorded as an open node — which puts a phantom parent in every path (`models.*.sdk`) that no TOML document has, and a `RootModel` does the same with `root.`. Recommendation: give the surface a way to say *the document root is itself an open node whose value schema is `config_model`* — the smallest honest extension is a `Surface.document_root_is_open: bool` (name to taste) that `fingerprint_at` turns into a seed at path `("*",)` (one root open-node record with `open_node=True` and no parent, then the projection's fields beneath `*`), so the fingerprint records `*` and `*.sdk`, `*.enabled`, `*.model_type`, … and `[defaults]` is simply one of the `*` keys. Then `delete_key` with `table_path=["*"]`, `key="prompting_target"` addresses `*.prompting_target`, which is exactly the material the entry declares. **Measure `_check_open_node_addressing` against a root `*` before committing to this** — if it requires a recorded parent for the `*` segment, teach it that the root may be the open node (red-first test in `test_ledger_check.py`) rather than inventing a parent. Record the shape as built in Status and in `docs/migration-ledger.md` § The fingerprint.+- [x] **The third `DefaultsLayerKind`** (D3). Recommendation, name to be ruled at CP1: `OPTIONAL_DOCUMENT` (*"the file is copied, not layered; every key of the document is optional at the document level and the model supplies the value, so an added field is absorbed by construction"*). What it means for the four readers of the kind:+ - `read_defaults_document()` → `{}`: there is no defaults document, and the defaults-layer rule (`coverage.check_defaults_layer` — every *required* path has a default) holds vacuously because the projection has no required path. Say so in the docstring; do not fake a document.+ - `render_reference_document()` → the **representative kit file**, e.g. the kit's `openai.toml`, named on the surface as `reference_document_path` (a new field, not a reuse of `packaged_document_path`, whose name means "merged beneath the user's file", which this is not). This is what `umig` snapshots as `[email protected]`, so a *later, non-pre-history* entry has a real starting document to be replayed over (`transform_check._read_document_the_entry_migrates_from`), and `coverage._check_head_defaults_document` keeps its byte-compare. It is the substance's item 5 answered before a second entry arrives. Alternative considered and rejected: no `[email protected]` for this kind — it works for a pre-history first entry (the head has no `after` and `_check_link` returns `[]`) and breaks the second entry.+ - `reference_documents()` → the reference document **plus every `*.toml` in the kit's backend directory** as convergence/neutrality witnesses (`kit_template_path: Path | None` generalises to `kit_template_paths`, or a `kit_template_dir` + glob — pick the smaller change; the three existing surfaces keep one path). More witnesses than any other surface has, at no cost, and it is exactly the set `pipelex init` copies. `_check_convergence` is the only reader (`ledger_check.py:595`).+ - the `check_defaults_layer_is_reachable` validator: extend the "packaged path ⇔ PACKAGED_DOCUMENT" pairing to "reference document path ⇔ OPTIONAL_DOCUMENT" and keep both mutually exclusive.+ - **`transform_check._check_the_migrated_document_is_accepted`** (head only) validates the migrated document against `config_model` — for an open-root document that means validating *each root table* against the projection (and, since the loader merges `[defaults]` into each model table, the honest acceptance check for this surface is *the merge of `[defaults]` and each model table validates against the real blueprint* — decide which, write it as a `Surface.validate_document(...)` seam the check calls, and say in its docstring which of the two it proves). Red-first test.+- [x] **Register the surface** in `build_config_surface_registry()`: `surface_id = INFERENCE_BACKEND_CONFIG_SURFACE_ID` (spelled once in `config_surface.py` beside the other three — the constants comment there says "names the same three"; make it four), `title`, `subdirectory = Path("inference") / "backends"` (spell it from `config_loader`'s `INFERENCE_DIR_NAME` / `BACKENDS_DIR_NAME`, not a new literal), `base_file=None`, `tier_glob="*.toml"` (D1 — the `.md` files fall out by extension; `inference/backends.toml` is one level up and never enumerated), `config_model=<the projection>`, `document_root_is_open=True`, `defaults_layer_kind=<D3>`, `reference_document_path=<kit openai.toml>`, kit witnesses = the kit backend directory. Write the D1/D2 reasoning in the docstring right there (why not `backends.toml`, why not `deck/`, why not `routing_profiles.toml`).+- [x] **The ledger at schema 1**: `pipelex/migration/ledgers/inference-backend.toml` with the `[surface]` block (`subdirectory = "inference/backends"`, no `base_file`, `tier_glob = "*.toml"`, `current_schema_version = 1`, `min_supported_schema_version = 0`) and zero entries. `make umig` writes `goldens/inference-backend/[email protected]` + `[email protected]`. **Read the `[email protected]` by eye**: it must show `*` as an open node and `*.<field>` for every blueprint field, no phantom prefix, no `prompting_target` (the schema-1 cut is post-`#1104`, which is what makes the entry pre-history).+- [x] Extend `tests/unit/pipelex/migration/test_real_surfaces.py` (`TestTheRealRegistry`, `TestTheCheckedInGoldens`) — they iterate the registry, so they should pass unchanged; add the one assertion that is new: the backend surface's fingerprint has an open root. Update `test_migration_run.py:96-104` (the specimen is now claimed by `inference-backend`, and by nothing else). Add a real-directory test: a temp config root seeded with the **kit** backend directory (`get_kit_configs_dir()/inference/backends`) is walked and every `.toml` is claimed by the backend surface, the `.md`s by nobody, and `pipelex migrate --dry-run` over it reports every file clean (the kit is at the current schema — the convergence property, proved on the walk and not only in the gate).+- [x] Gates 1–5, plus **the full `agent-test`** (no new error class was added, so `gei`/`gep` were not owed). Committed as `The inference-backend surface: an open-root document, a copied-document defaults layer, schema 1 cut` (the kind was ruled `COPIED_DOCUMENT`, not `OPTIONAL_DOCUMENT`), plus a second commit for the walk test and the ledger doc.++### ⛔ CHECKPOINT CP1 — MANDATORY STOP **(Louis: ratify D1–D4)**++- [x] Status block updated: the shape as built (fingerprint seed, kind name, reference document, witnesses), what `_check_open_node_addressing` and the acceptance check needed, and whether the registry-driven machinery absorbed the fourth surface without edits (name every file outside `surfaces.py` + the new ledger that had to change, and why).+- [x] D1–D4 written as *recommended* with the measurement that supports each; the substance document's § Decisions owed updated (workspace repo, not committed by this session).+- [x] Gates 1–5 green; committed; **not pushed** (nothing is outward-facing before CP3).+- [x] STOP. Louis rules D1–D4 — **all four ratified as built, 2026-08-17, with no change requested.** Next session starts at Phase C.++## Phase C — the two pre-history entries++Both are legal under the rules as they stand; both are checked by the pre-history path (no walk: `coverage.py:143-146`, `ledger_check.py:157-163`), which is exactly what lets a `move_key` land on a destination the frozen previous fingerprint already has.++### C1 — `inference-backend@2`++- [x] **Red first**: a test in `tests/unit/pipelex/migration/` (beside `test_telemetry_back_entry.py`, e.g. `test_backend_back_entry.py`) that replays the packaged `inference-backend` ledger over a hand-authored pre-`#1104` backend document — `[defaults]` with `prompting_target`, one model table with it, one without, one header-shaped key (`x-portkey-config`) that must survive — and asserts: every `prompting_target` gone, every other key byte-untouched, the header key untouched, the replay is idempotent (second pass changes no bytes), the file then validates the way the loader validates it (merge `[defaults]` into each model, `InferenceModelSpecBlueprint.model_validate`, `split_model_spec_keys` reports nothing rejected).+- [x] Bump `current_schema_version = 2` and append the entry: `id = "inference-backend@2"`, `to_schema_version = 2`, `introduced_in = "0.46.0"` (the reshape release; `/release` step 3b confirms it against the version actually cut), `breaking = true`, `safety = "safe"`, `pre_history = true`, title/description/guidance in the ledger's voice (guidance says: the key is deleted, not moved — the templating style is authored on the pipe now — and that a user's own unknown per-model key is *not* something this entry repairs, the loader's message names it), `declared_removed_paths = ["*.prompting_target"]`, and one op: `kind = "delete_key"`, `table_path = ["*"]`, `key = "prompting_target"`.+- [x] Hand-author `goldens/inference-backend/[email protected]` from the pre-`#1104` kit shape (`git show 60aa77f5^:pipelex/kit/configs/inference/backends/portkey.toml` is the richest specimen — `[defaults]` *and* per-model occurrences); trim to a few tables; header comment in the style of `goldens/telemetry-config/[email protected]`; **no real value from any personal file**.+- [x] `make umig` (head is now 2: writes `[email protected]` + `[email protected]`; `fingerprint@1` stays behind frozen; the two must differ by nothing — no removal, no narrowing — or `PRE_HISTORY_HAS_A_DIFF` fires, which is the point). `make cl` and `make cmig` green. If `_check_open_node_addressing` or `_check_pre_history_source_is_declared_material` refuse `*.prompting_target` at the root, that is a Phase B shape defect — go back, do not widen the declaration.+- [x] Also verify by hand what the substance promises: `["*"]` at the root matches `[defaults]` **and** every model table (both are root tables), the wildcard is a legal `table_path` for `delete_key`, and the op skips silently on a file already at the new shape (every kit file: convergence).++### C2 — `pipelex-config@2` (pre-history) and the reshape renumbered to `@3` (D4)++- [x] **Red first**: extend `tests/e2e/pipelex/cli/test_migrate_commands.py::TestAPreReshapeMachine` (or a sibling class) with a v0.45.0-shaped `pipelex.toml` carrying `[pipelex.prompting_config]` (`default_prompting_style = { tag_style = "xml" }`) and `[pipelex.prompting_config.prompting_styles]` (a couple of targets) — the exact old shape is `git show v0.45.0:pipelex/pipelex.toml` lines 272–279 — and assert both loops carry it onto today's shape: `[inference.templating] default_templating_style = { tag_style = "xml" }`, no `prompting_config` anywhere, no `prompting_styles`, one backup, boot clean; and that the *packaged* `[email protected]` (which never carried `prompting_config`) is unaffected by the new entry (a no-op link).+- [x] The ledger edit, atomic (`MigrationLedger.check_entries_are_contiguous_and_named_for_their_version` forces id ↔ version ↔ `current_schema_version`): `current_schema_version = 3`; **insert** before the reshape an entry `id = "pipelex-config@2"`, `to_schema_version = 2`, `introduced_in = "0.46.0"`, `breaking = true`, `safety = "safe"`, `pre_history = true`, title *"Prompting styles become a templating default"* (or better), description (what `#1104` did to the file), guidance (per-target styles are gone as a concept — the style is authored on the pipe (`templating_style` on a `PipeLLM`); a tuned per-target map is dropped, and the default travels), `declared_removed_paths` = `pipelex.prompting_config`, `pipelex.prompting_config.default_prompting_style`, `pipelex.prompting_config.prompting_styles` (+ its `*` value paths if the checks want them spelled), ops in this order: `move_key` `["pipelex","prompting_config"] default_prompting_style → ["pipelex","templating_config"] default_templating_style` (creates the destination block table when absent — the Phase-1 placement rule), `delete_table ["pipelex","prompting_config","prompting_styles"]`, `delete_table ["pipelex","prompting_config"]`. **The operations address the true historical spelling `pipelex.…`, before the reshape renames `[pipelex]` — that is why it goes before, not after.** Then the reshape entry: `id = "pipelex-config@3"`, `to_schema_version = 3`, nothing else in it changes (its `move_key pipelex.templating_config → inference.templating` is what carries the new entry's destination onward).+- [x] The golden chain: `git mv goldens/pipelex-config/[email protected] [email protected]`, `git mv [email protected] [email protected]`; then **copy** `[email protected] → [email protected]` and `[email protected] → [email protected]` (the frozen no-diff link: nothing in the models changed between the two, so the "no removal" claim holds by construction and the reshape's derived accounting is unchanged — the `@2 → @3` diff *is* today's `@1 → @2` diff); hand-author `goldens/pipelex-config/[email protected]` (a small pre-`#1104` document: `[pipelex.prompting_config]` + `prompting_styles`, and one unrelated table so the neutrality of the other ops is witnessed). Then `make umig`: **the head (`@3`) is rewritten from live and must be byte-identical to the files `git mv` produced — `git diff --stat` on the goldens must show only the two copies and the `[email protected]`.** That diff is the proof the renumber was pure; paste its shape into Status.+- [x] Update every place that quotes the old id or files: `CHANGELOG.md:31` (`pipelex-config@2` → `@3` in the reshape's `**Migration:**` bullet), `tests/e2e/pipelex/cli/test_migrate_commands.py:17, 358`, `tests/unit/pipelex/pipeline/fixes/test_fix_applier_config_surface_shapes.py:55`, and grep the tree (`git grep -n 'pipelex-config@2'` across `pipelex/ tests/ docs/ CHANGELOG.md .claude/ wip/`) — every hit is either the *new* pre-history entry or a stale quote of the reshape; there is no third kind. The tests that read `goldens/pipelex-config/[email protected]` as "the packaged file at schema 1" (`test_migrate_commands.py:9, 96, 117, 379`, `test_boot_tolerance.py:17, 62` and the others listed in the Phase 0 reading) — check each: after the renumber `defaults@2` is *still* the schema-1 shape (a copy), so most keep working by accident; make them read `[email protected]` where they mean "the pre-reshape document", so the meaning is spelled and not inherited from a copy.+- [x] `test_boot_tolerance.py:15-20` module docstring ("the other two surfaces have empty ledgers today") goes stale — rewrite it.+- [x] `make cl`, `make cmig`, `make tb`, and the targeted e2e (`.venv/bin/pytest -x -q tests/e2e/pipelex/cli/test_migrate_commands.py`) green.+- [x] Commit C1 and C2 **separately**: `inference-backend@2: delete prompting_target from every root table (pre-history)` and `pipelex-config@2: the prompting-style half of #1104 accounted for; the reshape becomes @3`.++### ⛔ CHECKPOINT CP2 — MANDATORY STOP++- [x] Status: both entries recorded (ids, ops, declared paths), the `umig` byte-purity diff shape, anything the checks refused and how it was answered. **Nothing was refused** — `cl` and `cmig` were green on the first run after each half.+- [x] Gates 1–5 green; committed; not pushed. Gate 4 (the **full** `agent-test`) ran to completion after both commits and reported all tests passed; gate 6 (`docs-check`) was not owed — Phase C touched no page under `docs/` — and gate 7 was answered by `tests/unit/repo/test_test_durations_paths.py`, which is green, so the renamed-test/`.test_durations` hazard did not fire.+- [x] STOP. Next session starts at Phase D.++## Phase D — boot tolerance at the fourth loader, and the end-to-end proof++- [x] **Red first**: in `tests/unit/pipelex/system/configuration/test_boot_tolerance.py` (or a new `tests/unit/pipelex/cogt/model_backends/test_backend_boot_tolerance.py`): a temp `backends_dir_path` seeded from the kit's backend directory with `prompting_target` planted back into `openai.toml`'s `[defaults]` and into two of `portkey.toml`'s model tables (build the fixture by editing the kit copy programmatically — never from `~/.pipelex/`); `InferenceBackendLibrary.load(...)` over it **succeeds** in both `lenient` modes, the loaded model specs carry no `prompting_target` and everything else intact, and a stale-configuration warning is produced that names both files and `pipelex migrate` (in reach: the temp dir is passed as a walked root); a file with a *user's own* unknown key (`foo = 1` on a model) still fails exactly as before — tolerance is not leniency, and the ledger explains only what we removed; a stale `pipelex_gateway.toml` local-override file is repaired the same way (the served specs are untouched). Nothing is written to disk by the load (assert bytes unchanged).+- [x] The implementation, following the four-step pattern of `pipelex_service_config.py:59-75` / `config_loader.py:459-487`: in `InferenceBackendLibrary`, when a LOCAL_FILE model-spec table is rejected (`key_split.rejected`) **or** the merged blueprint fails validation, and only then, replay the ledger over **that one file** (`replay_surface_files_in_memory(surface_id=INFERENCE_BACKEND_CONFIG_SURFACE_ID, paths=[path])` — one path per call, since the helper deep-merges its paths and backend files are independent documents; if a single-file variant reads better, add it beside the helper rather than bending the loop) — then re-run the loader's own steps over what comes back (variable substitution, `[defaults]` pop, `split_model_spec_keys`, the merge, `model_validate`) and, if that succeeds, keep the result and **park** the warning; if it does not, the original error stands (raise exactly what was raised before). Do the retry per file, once, and structure it so the happy path never imports the migration engine (the contract's "the healthy path is untouched" — `replay_surface_files_in_memory` already defers the engine import).+- [x] **Where the warning goes.** `model_spec_keys.py` documents that this code runs before `runtime_hub.set_config()` on some paths, and `config_loader` parks its warning for the same reason. Recommendation: `InferenceBackendLibrary` gets `take_stale_configuration_warning()` on the same pattern as `ConfigLoader:62-70`, and `runtime_boot.py` emits it right after models-manager setup (find the spot beside `:210-211`; `doctor_cmd.py:1053-1054` likewise, and the `show_cmd.py:72-76` caller decides for itself — probably emit). `walked_dirs` = `config_manager.existing_config_dirs` (the roots — Phase A made `_is_within` resolve the surface's subdirectory from the ledger). Mind the layered resolution: `backends_dir_path` picks *one* directory (project wins the whole directory), so the files the loader replays are the ones under that directory; the walk still repairs both roots — the warning is about the files that were loaded, the command repairs everything it reaches. Say that in one docstring sentence.+- [x] `runtime_boot.py:488-500` (`InferenceBackendLibraryNotFoundError` / `…ValidationError` → `PipelexSetupError`): unchanged in kind — the retry sits *inside* `load` and either recovers or gets out of the way.+- [x] Update the `load` docstring (`backend_library.py:78-83`): leniency still covers credentials only; a stale file the ledger explains is now carried forward with a warning; a malformed file the ledger cannot explain stays fatal in both modes.+- [x] **End to end, both loops** (`tests/e2e/pipelex/cli/test_migrate_commands.py`, a new class beside `TestAPreReshapeMachine`, e.g. `TestAStaleBackendDirectory`): temp HOME with `~/.pipelex/inference/backends/` seeded from the kit + `prompting_target` planted (and a `pipelex_gateway.toml` in it, so the collision is exercised on the real walk); `pipelex doctor` shows the pending-migrations row naming the backend files; `pipelex migrate --dry-run` lists them with op kinds only (**no value from the file rendered** — the standing rule); `pipelex migrate` rewrites each in place with **one backup per file beside it** (`openai.toml.bak.<stamp>`), a second run is a no-op with no new backup, the `.md` files and `inference/backends.toml` are never touched, and a subsequent boot is clean with no warning; the agent loop (`pipelex-agent migrate`, JSON) reports the same files and op kinds. Mutation-test the "no value rendered" assertion.+- [x] Gates 1–5 (the `cli-docs` contract opens if `doctor_cmd.py` is touched — review `docs/tools/cli/` for real), full `agent-test`. Commit: `A stale backend directory boots with a warning and pipelex migrate repairs it in place`.++## Phase E — docs, changelog, skills, the stale sentence, and the plan records++- [x] `docs/migration-ledger.md` — **four of its items are already done** (Phase B did § Surfaces' fourth table row and its defaults-layer-kinds sentence, § The fingerprint's open root, § The defaults layer's copied-not-layered paragraph, and § Transform goldens' "beneath the defaults layer, not alone" clause; Phase A did `:513` and the claim rule). **What is left:** `:11` ("a new inference backend … content sync is out of scope") — sharpen: *the content* of a backend file (which models, which endpoints) is still content and out of scope; its *schema* (which keys a table may carry) is now a surface; `:374` "all three configuration surfaces" → write it so no count can go stale; `:548` "the three loaders" → the loaders (name the fourth); `:560` "an inference backend file … none of those has a ledger" → it has one now; the chain diagram `:446` unchanged in shape. Re-read the four already-done spots once against the finished code rather than trusting this note.+- [x] `docs/configuration/config-technical/inference-backend-config.md`: one short section — *when you upgrade and a backend file is out of date*: what the boot says, that `pipelex migrate` reaches `inference/backends/`, what it will and will not touch (your own unknown key is yours), backups beside the file. Keep it in the page's existing voice; `docs/configuration/index.md:35` gets the one-clause note that backend files are migratable. `make docs-check`.+- [x] `CHANGELOG.md` (Unreleased): `:10` — rewrite the "not recursively, because a subdirectory such as `inference/backends/`…" sentence to the new rule; `:20` — "each of the three configuration surfaces … all three start at schema version 1" → mention the fourth (`inference/backends/*.toml`) and its ledger; `:31` — the reshape bullet's entry id → `pipelex-config@3` (done in C2, re-check); add two `**Migration:**` bullets in the condensed house style (bold label + 2–4 sentences): the backend entry `inference-backend@2` — *run `pipelex migrate` after upgrading; a stale backend directory boots with a warning until you do* — and `pipelex-config@2` for the prompting-style half of the templating change (and one clause that the reshape entry is now `@3`). Do not mention `wip/` docs.+- [x] `.claude/skills/add-migration/SKILL.md:17-22` — the surface list gains the fourth (or stops enumerating); § *Special case: a change that predates the first fingerprint* — add the one procedure it lacks and this session performed: **inserting a pre-history entry below existing entries** (renumber, `git mv` the goldens, copy the frozen link, hand-author `before@N`, `umig` must then be a byte-pure no-op on the head — that no-op is the proof). `.claude/skills/release/SKILL.md` step 3b: nothing to change unless the entry-id diff logic assumes ids never move — read `:76-83` and decide; if a renumbered id would read as "a new breaking entry without a bullet", say in the skill that a renumber shows up as two ids and both need the changelog to mention them.+- [x] `pipelex/system/configuration/config_surface.py:4-5, 100-102` ("all three", "the same three") — the module docstring and the constants comment.+- [x] `tests/unit/pipelex/migration/test_transform_check.py:242` docstring ("all three surfaces are in today") — reword.+- [x] `wip/prompting-style/templating-style-implementation-plan.md:184` § *Out of scope here*: since `#1109` a leftover per-model `prompting_target` is a **hard error** (`NOT_HEADER_SHAPED`, fatal even leniently), not a silent outbound header — correct the sentence, and add that the sweep's scope is the repositories we ship, while a user's machine is what the `inference-backend@2` ledger entry now repairs.+- [x] **Added by Phase D — three spots it touched or created that Phase E must finish.** (a) `docs/tools/cli/migrate.md`'s § *What it touches* was rewritten in Phase D because it had become false; re-read it once against the finished walk, and check § *The warning that sends you here* now that a fourth loader emits one. (b) `describe_op`'s wildcard wording is new user-facing text — if `docs/migration-ledger.md` quotes a rendered operation anywhere, it must match. (c) the fourth loader parks its warning and `ModelManager.setup` emits it, which is **not** where `docs/migration-ledger.md` § *Boot tolerance* says warnings come from for the other three; say what this one does and why (the doctor's per-backend probe).+- [x] Gates 1–8, including `make check` and the **full** `agent-test`. Commit: `Docs and changelog for the fourth surface and the two pre-history entries`.++### ⛔ CHECKPOINT CP3 — MANDATORY STOP (pre-PR)++- [x] Status: every phase ticked; the S7b **Record** fields drafted here — decisions D1–D4 as taken; whether the registry-driven machinery absorbed the fourth surface without edits (name the exceptions); anything the tool got wrong on the *fixture* backend directory; the read-only `pipelex migrate --dry-run` report shape against **this laptop's real** `~/.pipelex/inference/backends/` (paths and op kinds only — never values; `--dry-run` only, the real run is S7's first-machine step).+- [x] The workspace plan docs updated and named in the hand-off (not committed by this session): `wip/migrator-3/sequencing.md` S7b row + a session record; `wip/migrator-3/backend-config-surface.md § Decisions owed`; `wip/migrator-3/README.md` — the `Scope` row of the key design decisions amended per R10 (backends are a surface; language and plugins still separate projects); `wip/migrator-3/plan.md § Checkpoint D` gains the S7b record placeholder.+- [x] `git rev-list --left-right --count origin/dev...HEAD` re-checked — `0 14` on 2026-08-17, `dev` has not moved from `61c4786b6`, so no merge is owed; if `dev` moved, `git merge origin/dev` (never rebase), gates again.+- [ ] STOP. Louis' go to push and open the PR — everything before this line is local; the push is the first outward-facing action.++## Phase F — the PR, the review rounds, the merge++- [ ] `git push -u origin feature/Backend-surface`; open the PR **against `dev`** titled *The inference-backend surface: a fourth migration surface, and the two pre-history entries #1104 left behind* — description opens with why it blocks the release (one paragraph from the top of this file), then the four decisions as taken, then the byte-purity proof of the renumber, then the first-machine dry-run shape.+- [ ] Bot review rounds (Codex, Greptile, Cubic — the pattern of #1113–#1115): fix real findings on the branch, answer and resolve every thread; **defer** design-tradeoff findings into `wip/migrator-3/` (workspace) or `wip/` here with an owner, per the standing feedback (a deferral is a recorded item, not an open thread nobody will read again). Re-run gates 1–8 after each round.+- [ ] `make check` and full `agent-test` green on the final head; PR `MERGEABLE`/`CLEAN`.++### ⛔ CHECKPOINT CP4 — MANDATORY STOP **(Louis go: merge)**++- [ ] Louis merges (squash) to `dev`. Record the PR number and the `dev` merge commit in Status and in `wip/migrator-3/sequencing.md`'s S7b row; verify `git diff --stat origin/dev feature/Backend-surface` is empty after the squash; delete the branch local and remote with `-d` after that check; **the worktree `_backend/` may be removed** (its two records live on `dev` once merged — this file included, if kept; otherwise archive this file into `wip/migrator-3/` per the house rule that the root `TODOS.md` belongs to the work in flight).+- [ ] STOP. **S7's release step resumes** in `pipelex/` on `dev` — its first line is still restoring this laptop's two `*.pre-reshape.bak` files, and it now has a second thing to prove: the global `inference/backends/` directory migrated **by the tool**, then a clean boot. That is a different session, in a different venue, on Louis' go.++## Known traps and open risks (read before each phase)++- **The pre-history checks skip the walk** (`coverage.py:143-146`, `ledger_check.py:157-163`) — that is why a `move_key` onto a destination the frozen previous fingerprint already has does not trip `DESTINATION_OCCUPIED`, and why every op is instead checked against `declared_removed_paths` at its **literal, pre-entry path** (`_check_pre_history_source_is_declared_material`, no rename chaining). Declare every path an op addresses.+- **`PRE_HISTORY_HAS_A_DIFF`** fires if the fingerprint pair around a pre-history entry shows any removal or narrowing. For `pipelex-config@2` the pair is (`@1`, a copy of `@1`) — trivially no diff. For `inference-backend@2` it is (`@1`, `@2`) both cut on today's projection — same. If either fires, the goldens were not what the plan says; do not `umigf` your way past it.+- **`umig` rewrites the head only and refuses a destructive overwrite**; after the renumber the head `@3` must come out byte-identical to the moved `@2` files, and after the backend cut `@2` must equal `@1` but for the version label. Any other diff is a finding.+- **`replay_surface_files_in_memory` deep-merges its paths** — one path per call for backend files.+- **`_is_within` resolves the parent, never the file** — keep the S7 asymmetry when it learns about subdirectories (a symlinked backend file out to a dotfiles repo is in reach).+- **`backends_dir_path` is resolved per directory, project wins the whole directory** — the walk repairs both roots; the loader loads one. Not a bug; a sentence in a docstring.+- **`extra="allow"` keys are invisible to the fingerprint**, a `RootModel` projects a phantom `root.`, and `compute_fingerprint` seeds `prefix=()` — the open-root seed is the smallest change that makes the paths honest; measure before building.+- **`InferenceModelSpecBlueprint.sdk` is required** — a `MODEL_DEFAULTS` surface on it raises `MigrationRegistryError` at registry build (`surfaces.py:149-163`); the projection is what makes the surface buildable.+- **Two `.md` files live in the backend directory** (`pipelex_gateway_models*.md`) and `pipelex_inference.toml` (not kit-shipped) is beside them on this laptop — the glob `*.toml` and the (directory, name) rule handle both; the tests must include both shapes.+- **`config_surface.py` may not import `surfaces.py`** (kernel-layer closure; `check-hub-layering` chases `interpreter_hub` only, so only the full `agent-test` would see it — see the memory note). Read the subdirectory from the ledger, not the registry.+- **`.claude/skills/add-migration` says never hand-edit a golden** — copying `fingerprint@1 → @2` and `git mv @2 → @3` is the one sanctioned exception, and its proof is that `umig` is a no-op afterwards; write that into the skill (Phase E) so the next person is not left to argue it.+- **No hardcoded counts** in the docs you touch — "the four surfaces" is the same trap "the three surfaces" was; write "every surface" where the number is not the point.diff --git a/docs/configuration/config-technical/inference-backend-config.md b/docs/configuration/config-technical/inference-backend-config.mdindex 997c2a95f..1dbb819f3 100644--- a/docs/configuration/config-technical/inference-backend-config.md+++ b/docs/configuration/config-technical/inference-backend-config.md@@ -327,6 +327,8 @@ model_id = "gpt-4o-mini"x-portkey-provider = "@openai" # forwarded as the `x-portkey-provider` request header```+**A header belongs on a model table and cannot go in `[defaults]`.** The two tables are read differently and it matters here: a model table has its header-shaped keys split off before validation, while `[defaults]` is copied into every model of the file as-is — so a header put there is not a shared header, it is an unknown model-spec field on every model at once, and the backend fails to load naming a model that never mentioned it. To apply the same header to several models, repeat it on each.+Because these strings go out over the network, an extra key is accepted as a header **only if it is shaped like one — it must contain a hyphen** (`x-portkey-provider`, `anthropic-beta`, `api-version`) **and its value must be a string** (quote it in TOML). Model-spec field names never contain a hyphen, so an unknown key without one is a misspelled setting or a field that no longer exists, and it is a configuration error rather than a header: the backend fails to load and the error names the key, the model and the file. A hyphenated spelling of a real field (`max-tokens` for `max_tokens`) is rejected the same way, with the field it resembles, and so is a header-shaped key whose value is not a string (`x-foo = 3`) — it is never stringified onto the wire. This holds in every boot mode, including the credential-free one used by `pipelex validate` and dry runs — a typo must never silently drop a backend and surface later as "model not found".```text@@ -659,6 +661,37 @@ Common error types:- `LLMHandleNotFoundError`: Referenced model or alias not found- `ModelChoiceNotFoundError`: Referenced preset/choice not found+## When you upgrade and a backend file is out of date++`pipelex init` writes the backend definitions once and never overwrites a file you already have — which is what keeps your edits, and which also means a file written by an older Pipelex stays on your machine after the model-spec format changes. When a release removes a key, your copy still carries it, and the strict model that reads it refuses.++You do not have to do anything about that at boot. Pipelex reads the migration history for the `inference/backends/` files, carries the out-of-date ones forward **in memory**, and starts with a warning naming each file:++```+Your configuration is out of date, and pipelex read it as if it had been migrated:+'~/.pipelex/inference/backends/openai.toml'. What the ledger carried forward: Drop+prompting_target from every backend definition. Nothing was written. Run+`pipelex migrate` to bring the files up to date.+```++Nothing has been written at that point, so the warning comes back at the next boot until you run the command:++```bash+pipelex migrate --dry-run # show what would change in each file, and stop+pipelex migrate # ask, then rewrite in place+```++What it touches and what it leaves alone:++- **Every `*.toml` directly in `inference/backends/`** — the files this page describes, in both the global `~/.pipelex/` and a project's `.pipelex/`.+- **Not** `inference/backends.toml`, which sits beside that directory rather than in it, and **not** the model deck under `inference/deck/`. The deck has its own `pipelex update`.+- **Not a key you added yourself.** The history only describes keys *we* removed or renamed. An unknown key of your own — a misspelled `maxx_tokens`, an extra header that is not header-shaped — is still an error, and it names the file, the key and what to do. That is deliberate: silently dropping a key you meant to set would change which model you get.+- **Every file it rewrites is copied first**, beside itself, as `<file>.bak.<UTC timestamp>`. Running the command twice is the same as running it once — a file already up to date comes back byte for byte identical.++`pipelex doctor` shows the same thing before anything fails: its **Configuration Migrations** row names every backend file a migration would rewrite, and `pipelex doctor --fix` offers to run it for you.++See [The Migration Ledger](../../migration-ledger.md) for what the history may contain and what it guarantees, and [`pipelex migrate`](../../tools/cli/migrate.md) for the command itself.+## Best Practices1. **Choosing Your Configuration Approach**:diff --git a/docs/configuration/index.md b/docs/configuration/index.mdindex ca414c789..295607369 100644--- a/docs/configuration/index.md+++ b/docs/configuration/index.md@@ -59,7 +59,7 @@ The configuration is organized into four main sections, which mirror the layersEach section contains multiple subsections for specific features and functionalities. A setting's address tells you which layer owns it: `[runtime.*]` applies to any process, whatever it loads; `[interpreter.*]` only means something once a method is loaded.!!! tip "If your file predates this layout"- A `pipelex.toml` written against an older layout still boots. Pipelex carries it forward in memory, warns you that it did, and changes nothing on disk; `pipelex doctor` reports the pending migration. `pipelex migrate` is what makes it permanent — it rewrites each file in place and keeps the original beside it as a timestamped `.bak`. Your settings move address, they do not change value. The command walks the global `~/.pipelex/` and your project's `.pipelex/`, so a file you load from somewhere else — through `Pipelex.make(config_dir=…)` — is yours to update where it lives, and the boot warning says so rather than naming a command that would not reach it. See [Migration Ledger](../migration-ledger.md) for what a migration may and may not do to your file.+ A `pipelex.toml` written against an older layout still boots. Pipelex carries it forward in memory, warns you that it did, and changes nothing on disk; `pipelex doctor` reports the pending migration. `pipelex migrate` is what makes it permanent — it rewrites each file in place and keeps the original beside it as a timestamped `.bak`. Your settings move address, they do not change value. The command walks the global `~/.pipelex/` and your project's `.pipelex/`, so a file you load from somewhere else — through `Pipelex.make(config_dir=…)` — is yours to update where it lives, and the boot warning says so rather than naming a command that would not reach it. Your inference backend definitions under `inference/backends/` are migratable the same way; the [inference backend page](config-technical/inference-backend-config.md) says what a migration will and will not touch in one. See [Migration Ledger](../migration-ledger.md) for what a migration may and may not do to your file.## Configuration Override Systemdiff --git a/docs/migration-ledger.md b/docs/migration-ledger.mdindex ad7a7c356..0c3cc14c4 100644--- a/docs/migration-ledger.md+++ b/docs/migration-ledger.md@@ -8,7 +8,9 @@ This page is the contract: what a ledger may contain, what the engine may do witA **structural** change is a key renamed, moved or removed, or an enumerated value renamed. It breaks a user's file: the model rejects it, and no amount of re-reading helps. That is what the ledger describes and what the engine repairs.-A **content** change is a value or an entry in a package-managed file that should track the package — a new inference backend, a changed default in the model deck. Those files are not broken, they are behind, and merging them without destroying the user's customizations is a different problem with different machinery. Content sync is out of scope here and is not described by this contract.+A **content** change is a value or an entry in a package-managed file that should track the package — a model the kit added to an inference backend, a changed default in the model deck. Those files are not broken, they are behind, and merging them without destroying the user's customizations is a different problem with different machinery. Content sync is out of scope here and is not described by this contract.++The two run along the same file, and an inference backend definition is where the split is easiest to see. *Which* models a backend declares, and what each of them costs, is content: no ledger will ever add a model to a user's file, and the kit's own copy moving ahead of theirs is not something this contract repairs. *Which keys a model table may carry* is structure, and that half is a surface with a ledger of its own. A key we deleted from the schema is repaired; a model we added to the kit is not.Everything else this document deliberately does not specify: the language surface (`.mthds` files and the MTHDS standard version), plugin-owned ledgers, and the promotion of this contract into a cross-repository specification. Each is a project of its own, and specifying it in advance would freeze decisions nobody has had to make yet.@@ -16,25 +18,30 @@ Everything else this document deliberately does not specify: the language surfacA **surface** is one artifact family with one schema version and one ledger. The in-scope surfaces are the configuration surfaces the `pipelex` package owns:-| Surface id | Base file | Tier files | Model | Defaults layer |-|---|---|---|---|---|-| `pipelex-config` | `pipelex.toml` | `pipelex_*.toml` | `PipelexConfig` | Packaged document (`pipelex/pipelex.toml`) |-| `telemetry-config` | `telemetry.toml` | `telemetry_*.toml` | `TelemetryConfig` | Model defaults |-| `pipelex-service-config` | `pipelex_service.toml` | — | `PipelexServiceConfig` | Model defaults |+| Surface id | Directory | Base file | Tier files | Model | Defaults layer |+|---|---|---|---|---|---|+| `pipelex-config` | — | `pipelex.toml` | `pipelex_*.toml` | `PipelexConfig` | Packaged document (`pipelex/pipelex.toml`) |+| `telemetry-config` | — | `telemetry.toml` | `telemetry_*.toml` | `TelemetryConfig` | Model defaults |+| `pipelex-service-config` | — | `pipelex_service.toml` | — | `PipelexServiceConfig` | Model defaults |+| `inference-backend` | `inference/backends/` | — | `*.toml` | `InferenceModelSpecFileNode`, one root table at a time | Copied document (the kit's `inference/backends/portkey.toml`) |++Paths are relative to a configuration directory, and every surface spans exactly two of them: the global `~/.pipelex/` and the project's `.pipelex/`. `pipelex migrate` walks both, and only those. Two paths that look like configuration are outside the walk by design: the unit-testing tier `./tests/pipelex_{run_mode}.toml`, and a directory an embedder hands `Pipelex.make(config_dir=…)` that is neither of the two. Boot tolerance and the error-path diagnosis still read such a directory (see [Boot tolerance](#boot-tolerance)), but its files are the embedder's to bring up to date: `pipelex migrate` has no way to be pointed at them, and does not pretend otherwise.-File names are relative to a configuration directory, and every surface spans exactly two of them: the global `~/.pipelex/` and the project's `.pipelex/`. `pipelex migrate` walks both, and only those. Two paths that look like configuration are outside the walk by design: the unit-testing tier `./tests/pipelex_{run_mode}.toml`, and a directory an embedder hands `Pipelex.make(config_dir=…)` that is neither of the two. Boot tolerance and the error-path diagnosis still read such a directory (see [Boot tolerance](#boot-tolerance)), but its files are the embedder's to bring up to date: `pipelex migrate` has no way to be pointed at them, and does not pretend otherwise.+A surface's **directory** is where its files sit *within* a configuration directory. Most surfaces have none and live directly in it; a surface may instead own a subdirectory, and then it owns that subdirectory one level deep and to the exclusion of every other surface. A subdirectory no surface owns is never walked at all.The tier set is **open**. Environment and run-mode names are dynamic, so the tier filenames of `pipelex-config` cannot be enumerated in advance — `pipelex_local.toml`, `pipelex_{environment}.toml`, `pipelex_{run_mode}.toml`, `pipelex_override.toml` and `pipelex_temporary_override.toml` are a description of today, not a closed list. That is why the registry matches tiers with a glob, and the glob is what makes the resolution rule necessary:-> **Exact filenames claim before globs, across all surfaces.** A file matched by any surface's exact pattern belongs to that surface and is excluded from every surface's glob. A file claimed by two globs, or by two exact names, is a registry error.+> **A file is claimed by the pair (directory, name).** Only the surfaces that own the directory a file sits in may claim it; among those, exact filenames claim before globs, and a file matched by any of their exact patterns is excluded from all of their globs. A file claimed by two globs, or by two exact names, in the same directory, is a registry error.-Without the rule, `pipelex_service.toml` is both the base file of one surface and a match for another's `pipelex_*.toml`, and which ledger runs over it becomes an accident of iteration order.+Both halves earn their place on a real file. Without the name half, `pipelex_service.toml` is both the base file of one surface and a match for another's `pipelex_*.toml`. Without the directory half, `inference/backends/pipelex_gateway.toml` is a `pipelex_*.toml` match as well — and it is an inference backend definition, so the main configuration's ledger would be replayed over it and rewrite it. Either way, which ledger runs over a file would become an accident of iteration order.-The registry error is raised at the two moments it is decidable, and no earlier. Two surfaces sharing an id, a base file or a *literal* glob string are refused when the registry loads. Two surfaces whose globs merely *could* overlap are not: whether two glob languages intersect is not cheaply decidable, and a registry has no files to look at. That one is refused the moment a real file proves it — resolution stops by name, on the file, rather than picking whichever surface came first.+The same scoping applies to what the registry refuses. A base file and a glob are claims over one directory's listing, so two surfaces collide on a spelling only when they own the same directory — `*.toml` is the natural glob for any surface that owns a directory of its own, and the second such surface must not have to narrow it for the registry's bookkeeping.-Each surface also declares its **defaults-layer kind**, because the checks need to know where a current-schema default value comes from: a packaged TOML document merged beneath the user's files, or model-level field defaults. The distinction matters twice — for synthesizing the complete reference document of a surface that has no packaged file, and for refusing an added path that has no default anywhere. A model-defaults surface whose model cannot be built with nothing set has no defaults layer at all, and says so when its reference document is asked for rather than surfacing later as a missing value.+The registry error is raised at the two moments it is decidable, and no earlier. Two surfaces sharing an id, or sharing a base file or a *literal* glob string within one directory, are refused when the registry loads. Two surfaces whose globs merely *could* overlap are not: whether two glob languages intersect is not cheaply decidable, and a registry has no files to look at. That one is refused the moment a real file proves it — resolution stops by name, on the file, rather than picking whichever surface came first.-The registry and each ledger's `[surface]` block both describe the same thing, and both are read as truth — the ledger's when a migration walks a directory, the registry's when a gate fingerprints a model. A disagreement between them is therefore checked, not trusted: it would otherwise wait silently for the day the two readers meet.+Each surface also declares its **defaults-layer kind**, because the checks need to know where a current-schema default value comes from: a packaged TOML document merged beneath the user's files, model-level field defaults, or — where the kit *copies* a document instead of layering one beneath it — nothing at all beneath the file, which is sound only for a surface no path of which is required (see [the defaults layer](#the-defaults-layer-and-why-additive-changes-never-migrate)). The distinction matters twice — for synthesizing the complete reference document of a surface that has no packaged file, and for refusing an added path that has no default anywhere. A model-defaults surface whose model cannot be built with nothing set has no defaults layer at all, and says so when its reference document is asked for rather than surfacing later as a missing value.++The registry and each ledger's `[surface]` block both describe the same thing, and both are read as truth — the ledger's when a migration walks a directory, the registry's when a gate fingerprints a model. A disagreement between them is therefore checked, not trusted: it would otherwise wait silently for the day the two readers meet. The directory is the field with a reader on the boot path as well: a stale-configuration warning names `pipelex migrate` only for a file the walk would reach, and it works out where the walk reaches from the **ledger**, because the module that composes that warning may not import the registry — the registry names every configuration model, and the warning is composed in the layer that loads none of them. One derivation, read from both ends.Every surface starts at `schema_version = 1` with an empty ledger. There is no retroactive numbering of changes that predate the ledger; the one case that genuinely needs to reach backwards has its own bounded mechanism, described under [pre-history entries](#pre-history-entries).@@ -48,6 +55,8 @@ The inverse is worth stating with equal force, because it is the tempting mistak**If a surface ever appears to need an additive migration, the defect is a missing defaults layer on that surface, and the fix is to give it one.** The rule is enforced rather than trusted: the coverage check refuses an added path that has no value in its surface's defaults layer, and names the only remedy the vocabulary allows.+**One surface's files are copied rather than layered, and it keeps the rule for a different reason.** `pipelex init` copies the kit's backend definitions into a configuration directory, and nothing merges beneath them afterwards: what a backend file does not say, no packaged document supplies. That surface therefore has no defaults layer under the user's file, and none is synthesized — a document nothing reads would be a fiction the gates then went on to depend on. What absorbs an additive change there is the shape of the file. A backend file is `[defaults]` plus one root table per model, and no single root table has to carry anything, because only the *merge* of `[defaults]` with a model's own table is a complete model specification. A key we add is therefore a key every existing file was already free to omit, and the required-path rule is satisfied by a surface that has no required path rather than by a value sitting beneath one — which is the same fact that makes [the fingerprint's document root an open node](#the-fingerprint). The kit document such a surface names is still read, for two purposes that are not this one: it is the starting document a transform golden replays an entry over, and it and its neighbours in that directory are [convergence witnesses](#convergence-replay-is-neutral-on-the-witnesses).+## Schema versions, and why every run replays everythingEach surface carries a monotonic integer, bumped only when its shape changes. Integers rather than semantic versions, because a schema either changed or it did not. Each entry also records the package version that introduced it, for orientation against the changelog only — nothing branches on it.@@ -338,6 +347,8 @@ An enumerated member set is collected from both sources that produce one: an enu> **Members beneath an open mapping are recorded on the wildcard record, not on the mapping.** `dict[str, LogLevel]` records its spellings at `levels.*`, whose own value is the enumerated one, and records none at `levels`, whose value is a table. Recording them twice made the coverage gate demand a `remap_value` at the mapping's own path as well, and a remap of a table value is a guarded skip on every run — a demand with no legal answer. Every *other* container is still descended into, because it has no such child record: a `list[enum]` keeps its members on the list's own path, where they are the only thing that makes a member lost inside one visible at all.+> **A document's own root can be the open node, and one surface's is.** An open node is ordinarily a `dict[str, X]` *field*, recorded under the name it has — but a backend file has no such field: its root keys are model names, and the model beneath each of them is the value schema. Making one up would put a segment in every path (`models.*.sdk`) that no TOML document has, and a `RootModel` puts `root.` there for the same reason. So the surface declares that its root itself is open, and the projection is seeded at `*` rather than at the root: the recorded paths are `*`, `*.sdk`, `*.max_tokens`, and so on, which are exactly the paths an operation can address — `delete_key` with `table_path = ["*"]` reaches one key of every model table, and no entry can name one machine's model. `[defaults]` is simply one of those keys, and the projection has nothing to say about which root keys a file happens to hold. Whether a root was open is recorded **on the fingerprint** rather than read from the surface, because a frozen link has to answer for its own version, and the live registry can only answer for today's.+Two shapes are recorded as terminal rather than descended into, because no operation can address what lies beneath them: an array of tables, which has no path segment syntax and no wildcard form, and a model reachable from itself, which has no finite path set at all. Terminal means blind: a change to the model beneath an array of tables leaves the fingerprint unchanged, and the telemetry surface's `otlp` exporters are that shape today (see [Limits you will meet](#limits-you-will-meet)).> **The array-of-tables blind spot stays, and the reason is that closing it would produce a demand nothing can answer.** Recording an item model's fields under a synthetic segment would make a renamed field inside an exporter entry visible to the coverage gate — and then demanded, with no operation in the vocabulary able to reach it and no remedy an author could write. A gate whose refusal has no legal answer is worse than a blind spot the contract names. Revisit on the day an addressing syntax for arrays of tables exists, and not before: the two are one change.@@ -371,7 +382,7 @@ Alongside the diff, one standing invariant is checked on the new fingerprint its> **Every path a document must carry has to have a value beneath it.** A path that is required, and whose ancestors are all required, must have a value in its surface's defaults layer — otherwise it is breaking on the day it lands, and the refusal names the only remedy the vocabulary allows: give it a default. Wildcard paths are exempt by construction, since the keys beneath an open node are the user's and there is no single value to supply.-Stating it over every required path rather than only over newly added ones costs nothing — it holds across all three configuration surfaces today — and it catches the same defect arriving by the other door, a path that becomes required without gaining a default.+Stating it over every required path rather than only over newly added ones costs nothing — it holds across every configuration surface today — and it catches the same defect arriving by the other door, a path that becomes required without gaining a default.**Every entry is re-checked on every run**, not only the one being authored: the walk above runs for each link of the chain, `fingerprint@N-1` against `fingerprint@N`. An entry verified once at bump time and never again would silently stop matching its own diff the first time somebody edited it.@@ -395,7 +406,7 @@ Freezing the head instead would rot. An additive change is absorbed by the defauThe check then applies entry N to `defaults@N−1` and asserts three things about what comes out, pairwise over the complete-defaults family:-> **Every path the migration creates, schema version N has** — in `defaults@N` or in `fingerprint@N`; **every path `defaults@N−1` and `defaults@N` share survives the migration**; and **the last link's migrated document is accepted by the current model**, read the way a user's file is read — beneath the current defaults layer, not alone.+> **Every path the migration creates, schema version N has** — in `defaults@N` or in `fingerprint@N`; **every path `defaults@N−1` and `defaults@N` share survives the migration**; and **the last link's migrated document is accepted by the current model**, read the way a user's file is read — beneath the current defaults layer where one is layered beneath it, and on its own where the kit copies the document instead of layering one under it.Each claim is one-directional on purpose, and together they are red on wrong destinations, wrong remap targets, wrong order and over-deletion. The first catches a destination that is misspelled or lands where the new shape carries nothing, which is the defect the whole check exists for: coverage accounts for the removed path and convergence skips the absent source, so without this the typo migrates every user file to a key `extra="forbid"` rejects, with the tool reporting success. The second catches an entry that dropped a parent table where it meant to drop one child. The third catches what moves no path at all — a value the schema does not accept, or a key it does not know.@@ -510,7 +521,7 @@ Two commands run a migration, and they are the same run with two audiences.**Both walk the global `~/.pipelex/` and the project `.pipelex/`, and nothing else.** An embedder's `config_dir=` outside those two and this repository's own `tests/pipelex_{run_mode}.toml` are outside the walk: the first is the embedder's to update, the second is not a user's configuration at all, and migrating a directory nobody asked about is how a tool earns a reputation for touching things. A directory that does not exist is skipped, and a project rooted at the home directory is walked once rather than twice.-**The walk is not recursive.** A surface's tier files sit beside its base file; a subdirectory holds a different kind of thing. The specimen is real: `.pipelex/inference/backends/pipelex_gateway.toml` matches the `pipelex-config` tier glob `pipelex_*.toml` exactly and is an inference backend definition, so a recursive walk would replay the main configuration's ledger over it.+**The walk is each surface's own directory, one level deep.** Most surfaces live directly in a configuration directory and their tier files sit beside their base file. A surface may instead own a subdirectory — `inference/backends/` — and then that subdirectory is walked the same way, one level, for that surface alone. A subdirectory no surface owns, such as `inference/deck/`, is never entered at all, and no walk goes deeper than the directory a surface names.> **`--dry-run` and `--yes` together are refused, not resolved.** One asks for no write and the other authorizes one; picking a winner would hide the bug that produced both.@@ -541,11 +552,14 @@ The rules this encodes:- **The retry honours the [schema-version floor](#schema-versions-and-why-every-run-replays-everything).** A file that declares a version below `min_supported_schema_version` is one `pipelex migrate` refuses; the retry declines it through the same predicate rather than carrying it forward under-migrated, and the user's own error — with the `migration` block naming the floor — is what they see.- **The retry never becomes the failure.** Anything that goes wrong inside it — a ledger that will not load, a file that will not parse — makes it decline, and the error the *configuration* produced is what the user sees. Their error names the key to fix; ours would name our packaging. A file that cannot be re-read abandons the whole retry rather than being skipped, because skipping it would drop a layer from the merge and a re-validation that then succeeded would boot on a configuration the user does not have.- **The main configuration's warning waits for the logger.** That configuration is what configures logging, so when its retry succeeds there is no logger yet — the loader parks the warning (`config_manager.take_stale_configuration_warning()`) and the boot emits it right after `log.configure`. The telemetry and service loaders run after that point and warn directly. The warning is the same either way; only *when* it is said moves.-- **The warning names `pipelex migrate` only for a file the command would reach.** The retry replays whatever the loader merged, and a loader pointed at a directory of its own — an embedder's `config_dir=`, or this repository's `tests/pipelex_{run_mode}.toml` — merges files [outside the walk](#the-commands). For those the warning says the file is the caller's to update where it lives, and does not name a command that would then report nothing to do; a load that spans both gets both sentences, each naming its own files. Same rule as [the remedy on a validation error](#every-surface-reports-it-the-same-way-and-none-of-them-says-start-over): *a remedy is named only where it would write*. The walk is `config_manager.existing_config_dirs`, which is also where `config_directories_to_migrate` reads it — one derivation, read from both ends, so a boot cannot promise what the command declines.+- **The inference backend loader parks its warning too, and for a different reason: a caller, not boot order.** By the time backends load, logging has been configured for a while. What that loader has instead is `pipelex doctor`, which probes the backend files by loading the whole library **once per backend** — so a loader that logged for itself would repeat one stale directory's warning a dozen times in one report. It hands the warning over instead (`take_stale_configuration_warning()`), and the caller that owes the user a copy takes it: the models-manager setup emits exactly one per boot, the doctor's per-backend probe simply never asks, and no abstract interface widens to carry it. One warning per load covering every stale file, not one per file and not one per backend.+- **The warning names `pipelex migrate` only for a file the command would reach.** The retry replays whatever the loader merged, and a loader pointed at a directory of its own — an embedder's `config_dir=`, or this repository's `tests/pipelex_{run_mode}.toml` — merges files [outside the walk](#the-commands). For those the warning says the file is the caller's to update where it lives, and does not name a command that would then report nothing to do; a load that spans both gets both sentences, each naming its own files. Same rule as [the remedy on a validation error](#every-surface-reports-it-the-same-way-and-none-of-them-says-start-over): *a remedy is named only where it would write*. The walk is `config_manager.existing_config_dirs`, which is also where `config_directories_to_migrate` reads it — one derivation, read from both ends, so a boot cannot promise what the command declines. Reach is per *surface*, not per path: the file must sit in the directory **its own surface** owns under one of those roots, so a `telemetry.toml` dropped into `inference/backends/` is out of reach exactly as it was before any surface owned a subdirectory.- **The healthy path is untouched.** The replay runs only on the failure path, so a current configuration never reads a ledger, never re-parses a document, and never even imports the migration engine. (Not "never loads tomlkit": the ordinary configuration read has always imported it. What a healthy boot avoids is the second, DOM-level read.) Replay neutrality is what makes the retry free when it does run.- **One shared helper**, called by each configuration-surface loader — and that helper is also where `[meta] schema_version` is stripped. The strip belongs there and **not** in the generic TOML reader, which also reads `.mthds` files, backend definitions and the kit index, none of which reserve that key.-**The helper owns the failure path, not the load.** The three loaders do different things between their merge and their validate — the main configuration deep-merges programmatic overrides, telemetry substitutes `${VAR}` placeholders, the service configuration does nothing — so a helper that owned the whole load would have to be told about all three. It is given the surface id and the same ordered path list the loader merged, and it hands back the migrated merge plus a `MigrationPlan` per file; each loader re-runs its own step over that and re-validates. The overrides in particular have to be re-applied by the caller: they are a layer of the *load*, and the replay only ever sees the *files*.+**The helper owns the failure path, not the load.** The loaders do different things between their merge and their validate — the main configuration deep-merges programmatic overrides, telemetry substitutes `${VAR}` placeholders, the service configuration does nothing, and the inference backend loader does the most of any of them: it pops `[defaults]` out of the document, splits the header-shaped keys off each remaining root table, and validates the *merge* of the two rather than the document. A helper that owned the whole load would have to be told about every one of them. It is given the surface id and the same ordered path list the loader merged, and it hands back the migrated merge plus a `MigrationPlan` per file; each loader re-runs its own step over that and re-validates. The overrides in particular have to be re-applied by the caller: they are a layer of the *load*, and the replay only ever sees the *files*.++> **A surface whose files are independent documents calls the helper once per file.** The helper deep-merges the paths it is given, which is exactly right for a tier stack: `pipelex.toml` beneath `pipelex_local.toml` beneath `pipelex_override.toml` are layers of one configuration, and the thing that has to validate is their merge. Backend definitions are not layers of anything — `openai.toml` and `portkey.toml` are separate documents that happen to share a directory, and merging them would produce a backend nobody has. So that loader passes one path per call and keeps the results apart, and composes one warning covering every file the round recovered.> **The migration engine is imported inside the retry, not at the top of the module.** Its applier lives under `pipelex.pipeline` — an interpreter package — while the configuration loaders sit in `runtime_hub`'s import closure, and the kernel layer's property is that importing it loads zero interpreter modules (see [`hub-layering.md`](contribute/hub-layering.md)). A module-level import would break that silently: the layering guard mechanically checks reachability to `interpreter_hub`, which `pipelex.pipeline.fixes.applier` does not have, so nothing would have gone red. The deferred import is also what makes "the healthy path is untouched" literal rather than approximate.@@ -557,7 +571,9 @@ When the boot's retry declines and the model's refusal stands, the error the useA validation error cannot answer that by itself. It is raised against the merged configuration and carries no provenance — it names a key, not which of the files that were merged put it there, and certainly not whether that key was correct last month. So `report_validation_error` asks the files instead: a **dry-run scan** of the surface that refused, over the same directories a `pipelex migrate` would walk — or, when the caller loaded one explicit directory (`doctor --global`, an embedder's `config_dir=`), over that directory alone, so the block never names a file the reader did not load.-**The scan is named, not guessed.** The caller passes the surface whose model refused; a caller validating something that is not a configuration surface — a `.mthds` bundle, an inference backend file, a model deck — passes nothing and gets the translation alone. None of those has a ledger, and offering them a `pipelex migrate` remedy would send a user to a command with nothing to do. That is also why the scan does not run at all when no surface is named: it is a directory walk and a ledger replay, and a bundle-validation failure has no business paying for one.+**The scan is named, not guessed.** The caller passes the surface whose model refused; a caller validating something that is not a configuration surface — a `.mthds` bundle, a model deck, a routing profile — passes nothing and gets the translation alone. None of those has a ledger, and offering them a `pipelex migrate` remedy would send a user to a command with nothing to do. That is also why the scan does not run at all when no surface is named: it is a directory walk and a ledger replay, and a bundle-validation failure has no business paying for one.++> **An inference backend file used to be on that list, and moving it off was a code change rather than a wording one.** The boot builds its own message for the two libraries that refuse after the main configuration is already up — the backend library and the model deck — and it named no surface for either, which was right while neither had a ledger. Now one of them does, and the two are told apart by the component itself rather than at the `except` clause: the pairing of a component with its surface is stated once, so a call site cannot quietly drop it. The case this reaches is narrow and real — a backend file that is *both* behind and carrying something we cannot explain never gets as far as boot tolerance's warning, and before this it was told only the second half. The scan is only reachable if the boot catches what the loader really raises, so the classes that mean *this library's files are not loadable* are named once, beside the component — a backend definition file refuses in two shapes, and only one of them carries a pydantic error; the library index file `backends.toml` refuses in a third, on a backend table's own fields, and each has to be a class the boot names.> **Scoping narrows the answer, never the registry.** Which surface owns a file is decided across *all* of them, because an exact base file claims before any glob. A registry built to hold only the surface being asked about removes the other claimants from that arbitration, and `pipelex_service.toml` — another surface's base file, and a match for `pipelex-config`'s `pipelex_*.toml` — is then replayed under the wrong ledger and diagnosed against the wrong model, so its ordinary settings come back reported as paths this build knows nothing about.@@ -565,7 +581,7 @@ The answer rides the error twice, and the two halves are not interchangeable:- **`error_domain` stays `"config"`.** It does not become a domain of its own. `ErrorDomain` is a closed cross-repo enum and the agent-hook specification routes any domain it does not know to BLOCK, so a stale configuration reported under a new domain would stop an agent rather than tell it what to run.- **The structured `migration` block is the contract, and consumers branch on its presence.** Absent means the failure is not staleness. Present, it carries `remedy` (the command), `would_write` (whether that command would rewrite any of these files), `needs_attention` (whether anything here is a person's rather than the tool's), and `plans` — the same `MigrationPlan` shape `pipelex-agent migrate --dry-run --format json` emits under its own `plans` key, so an agent that parses one has already parsed the other. Only the files the scan found something in are listed: unlike the commands' report, which answers *what did the walk visit*, the block answers *what is wrong with this machine*.-- **The message is presentation.** It carries the pydantic analysis followed by a paragraph naming the files, what the command would carry forward, and what it cannot do for anyone. Nothing branches on its wording.+- **The message is presentation.** It carries the refusal — the pydantic analysis where the surface's own model refused, or, for the two libraries the boot reports on, the loader's own sentence, kept whole because it names *which model, which backend, which file* before quoting that analysis, and a directory holds a dozen files with the same field names — followed by a paragraph naming the files, what the command would carry forward, and what it cannot do for anyone. Nothing branches on its wording.> **A remedy is named only where it would write, and `would_write` is what says so.** The block's presence means the migration history has something to report about these files — not that running the command repairs them. A file whose only finding is a path no entry explains, or an entry blocked before any of its operations landed, produces a block with nothing to apply; naming `remedy` there sends a reader to a run that visits the file, writes nothing, and leaves the same refusal in front of them. So every surface branches on the field rather than on the presence, and the ones with nothing to apply point at `pipelex migrate --dry-run`, where the diagnosis actually is. The two flags are independent and never both false: a file with nothing to write and nothing for a person is clean, and clean files are not in `plans`.@@ -581,12 +597,14 @@ A configuration error reaches a user through several surfaces at once — the huSo no surface holds its own answer any more:-- Every configuration surface's loader raises through `report_validation_error`, so the `migration` block reaches the error wherever it is caught. `TelemetryConfigError` is a `PipelexConfigError` for this reason — it carries `error_domain = CONFIG` from the class rather than from the agent CLI's lookup table, and it can carry a block.+- Every configuration surface's refusal goes through `report_validation_error`, so the `migration` block reaches the error wherever it is caught. Most surfaces do it in the loader; the inference backend definitions do it one frame out, in the boot that catches the library's own error, because that library refuses after the main configuration is already up and the boot has a message of its own to build. `TelemetryConfigError` is a `PipelexConfigError` for this reason — it carries `error_domain = CONFIG` from the class rather than from the agent CLI's lookup table, and it can carry a block.- The **human handler** prints the fields the model refused and then names `pipelex migrate` when the block says it would write, the dry run when it says it would not, and an edit when there is no block at all.- The **agent envelope** carries `migration` as a field and its hint says which of the three readings applies — no block, a block that would write, a block that would not; an agent branches on the fields, never on the sentence.- The **`doctor` row** carries a *finding* — `healthy`, `not_found`, `unparseable`, `out_of_date`, `invalid` — and every caller reads that instead of the row's wording. This one is a correctness fix and not only a tidy-up: `--fix` used to decide what it could repair by searching the message for `"format has changed"`, so rewording the row would have switched the whole repair path off in silence. The finding answers for the one file the row is about: the scan behind `out_of_date` runs over the directory that file lives in — which also holds the `telemetry_*.toml` tier — and the verdict is read off that file's own plan, so a stale tier file beside a wrong base file leaves the base file `invalid`, and the fields the model refused stay on the row. **`out_of_date` means the plan would rewrite that file**, not merely that it is unclean: a file the migration would leave untouched is `invalid`, which is the finding that routes to a person and which `--fix` does not offer to run a command over, and the row keeps the dry run's pointer so the diagnosis is not lost with the finding.> **Writing a fresh file repairs exactly one finding: a missing one.** There is nothing in a file that is not there to lose. Every other unhealthy state has the user's own settings in it, so an out-of-date file is offered `pipelex migrate` and a broken one is left to a person. Regeneration is still reachable, and it is described as what it is — a way to start over that discards the file.+>+> The two remedies are therefore never printed together. A message that has just carried a migration block has promised that `pipelex migrate` keeps every value the file holds; adding *or run `pipelex init config`, which resets them to their defaults* makes whichever one the reader follows the wrong choice. So the block's presence is what drops the regeneration sentence, and a component with no ledger — the model deck — keeps it, because for that one it is the only answer there is.## The health reportdiff --git a/docs/tools/cli/migrate.md b/docs/tools/cli/migrate.mdindex 8193f0743..206787c2d 100644--- a/docs/tools/cli/migrate.md+++ b/docs/tools/cli/migrate.md@@ -22,7 +22,11 @@ Two directories, and only those:- the global `~/.pipelex/`- the project `.pipelex/`, when the current directory is inside a project that has one-Within each, it looks at the configuration files themselves — `pipelex.toml` and its `pipelex_*.toml` tiers, `telemetry.toml` and its tiers, `pipelex_service.toml`. It does not descend into subdirectories, so your inference backends and model deck are never rewritten by it.+Within each, it looks at the configuration files themselves — `pipelex.toml` and its `pipelex_*.toml` tiers, `telemetry.toml` and its tiers, `pipelex_service.toml` — and at the inference backend definitions in `inference/backends/`.++It goes one level deep, and only into a directory some configuration family owns. `inference/backends/` is such a directory, so every `*.toml` in it is repaired the same way a `pipelex.toml` is. `inference/deck/` is not one, and neither is `inference/backends.toml`, which sits beside the backends directory rather than in it — neither is ever entered or rewritten by this command, and the model deck has its own `pipelex update`.++Within a file it repairs, it only ever undoes a change *we* made. A key you added yourself — a misspelling, or a setting from a plugin — is reported rather than removed, because the migration history describes our renames and removals and has nothing to say about your keys.## What it does to a file@@ -42,7 +46,7 @@ The command exits non-zero when it leaves something for you to look at.## The warning that sends you here-You will usually meet this command through a warning rather than a crash. When a configuration file is out of date in a way the migration history explains, Pipelex boots anyway: it carries the file forward **in memory**, tells you which files it did that for, and points at this command. Nothing is written, so the same warning appears at the next boot, and the one after — running `pipelex migrate` is what makes the change to the file and stops it.+You will usually meet this command through a warning rather than a crash. When a file is out of date in a way the migration history explains, Pipelex boots anyway: it carries the file forward **in memory**, tells you which files it did that for, and points at this command. That covers your inference backend definitions as well as the configuration files proper — a backend file left behind by a model-spec change boots on the models it would have loaded anyway, and the warning arrives with the rest of the model setup. Nothing is written, so the same warning appears at the next boot, and the one after — running `pipelex migrate` is what makes the change to the file and stops it.A file the history cannot explain is a different case, and it still fails the boot with the configuration error itself: tolerance widens what starts, never what is accepted. That error names this command too, though — beside the key it could not accept, it tells you which of your files a migration would touch, what it would carry forward, and what only you can decide.diff --git a/pipelex/cli/commands/migrate_cmd.py b/pipelex/cli/commands/migrate_cmd.pyindex be405b631..fa352e722 100644--- a/pipelex/cli/commands/migrate_cmd.py+++ b/pipelex/cli/commands/migrate_cmd.py@@ -206,8 +206,23 @@ def describe_op(*, op: MigrationOp) -> str:def _table_in_words(*, table_path: list[str]) -> str:- """A table path as a report names it, the empty one included."""- return f"'{'.'.join(table_path)}'" if table_path else "the document root"+ """A table path as a report names it — the empty one and a trailing wildcard included.++ A trailing `*` names no table of its own: it means *each entry of the table above it*, which is+ how an open node is addressed — a `dict[str, X]` field, or the whole document of a surface whose+ root is open, as every `inference/backends/*.toml` is. Spelling it verbatim would print `from+ '*'`, which tells a reader nothing about which part of their file changed.++ A wildcard anywhere but at the end falls back to the literal dotted spelling. Nothing addresses+ one today, and inventing a phrase for a shape no ledger has would be guessing at what it should+ say.+ """+ if not table_path:+ return "the document root"+ if table_path[-1] != WILDCARD_SEGMENT:+ return f"'{'.'.join(table_path)}'"+ parent = table_path[:-1]+ return f"every entry of '{'.'.join(parent)}'" if parent else "every table of the document"def _panel(*, message: str, style: str) -> Panel:diff --git a/pipelex/cogt/model_backends/backend_library.py b/pipelex/cogt/model_backends/backend_library.pyindex 8debcf65e..9a0ea3ee3 100644--- a/pipelex/cogt/model_backends/backend_library.py+++ b/pipelex/cogt/model_backends/backend_library.py@@ -1,7 +1,8 @@from functools import partial-from typing import TYPE_CHECKING, Any, Self, cast+from pathlib import Path+from typing import TYPE_CHECKING, Any, NamedTuple, Self, cast-from pydantic import Field, RootModel, ValidationError+from pydantic import Field, PrivateAttr, RootModel, ValidationErrorfrom pipelex import logfrom pipelex.cogt.exceptions import (@@ -19,12 +20,20 @@ from pipelex.cogt.model_backends.backend_factory import (InferenceBackendFactory,)from pipelex.cogt.model_backends.gateway_config import GatewayConfig, drop_unknown_gateway_defaults+from pipelex.cogt.model_backends.model_spec_document import MODEL_SPEC_DEFAULTS_TABLEfrom pipelex.cogt.model_backends.model_spec_factory import (BackendModelSpecs,InferenceModelSpecBlueprint,InferenceModelSpecFactory,)from pipelex.cogt.model_backends.model_spec_keys import ModelSpecSource, describe_rejected_keys, split_model_spec_keys+from pipelex.migration.plan import MigrationPlan+from pipelex.system.configuration.config_loader import config_manager+from pipelex.system.configuration.config_surface import (+ INFERENCE_BACKEND_CONFIG_SURFACE_ID,+ replay_surface_files_in_memory,+ stale_configuration_warning,+)from pipelex.system.environment import get_optional_envfrom pipelex.system.pipelex_service.gateway_config_merger import GatewayConfigMergerfrom pipelex.system.runtime import runtime_manager@@ -45,16 +54,51 @@ if TYPE_CHECKING:InferenceBackendLibraryRoot = dict[str, InferenceBackend]+class RecoveredModelSpecs(NamedTuple):+ """One backend's model specs rebuilt from a migrated file, and what the ledger did to get there."""++ model_specs: "dict[str, InferenceModelSpec]"+ plans: list[MigrationPlan]+++def backend_toml_path(*, backends_dir_path: str, backend_name: str) -> Path:+ """Where a backend's per-model file lives, spelled once.++ Boot tolerance replays a stale file in memory and rebuilds the specs from the result, so the+ retry must read *the file the load read*. Two independent constructions of that path is a+ divergence waiting to happen, and it already was one: a backend name is a raw top-level table+ key of the user's own `inference/backends.toml` and nothing validates it, TOML permits a quoted+ `["/abs/path"]` key, and `Path(directory) / "/abs/path.toml"` drops the directory — so the retry+ would leave the backends directory the load had stayed inside.+ """+ return Path(f"{backends_dir_path}/{backend_name}.toml")++class InferenceBackendLibrary(RootModel[InferenceBackendLibraryRoot]):root: InferenceBackendLibraryRoot = Field(default_factory=dict)+ _stale_warning: str | None = PrivateAttr(default=None)+def reset(self):self.root = {}+ self._stale_warning = None@classmethoddef make_empty(cls) -> Self:return cls(root={})+ def take_stale_configuration_warning(self) -> str | None:+ """The warning a tolerated load owes the user, once — or `None` when every file was current.++ Parked rather than logged, and the reason is a caller rather than boot order: `pipelex+ doctor` probes the backend files by loading the whole library once per backend, so a loader+ that logged for itself would repeat the same warning a dozen times over one stale directory.+ Handing it over instead lets each caller decide — `ModelManager.setup` emits it (one boot,+ one warning), and the doctor's per-backend probe simply never asks.+ """+ warning, self._stale_warning = self._stale_warning, None+ return warning+def load(self,*,@@ -69,6 +113,18 @@ class InferenceBackendLibrary(RootModel[InferenceBackendLibraryRoot]):For pipelex_gateway, uses the provided remote config and merges with local overrides.+ **A file left behind by a schema change is carried forward rather than fatal.** When a local+ per-backend TOML is refused, the `inference-backend` ledger is replayed over that one file+ **in memory** and the loader's own steps re-run over what comes back; a load that then+ succeeds parks a warning (`take_stale_configuration_warning`) naming the files and the+ `pipelex migrate` remedy. Nothing is written — only the explicit command writes, which is+ why the warning keeps coming back until it is run. A file the ledger cannot explain raises+ exactly what it raised before: the retry either recovers or gets out of the way.++ The warning names the files *this* load merged, which is not always every file the command+ would repair: `backends_dir_path` picks one directory (a project's `.pipelex/` wins the whole+ directory over the global one), while `pipelex migrate` walks every configuration root.+Args:secrets_provider: Provider for secrets/credentials.backends_library_path: Path to backends.toml.@@ -80,17 +136,16 @@ class InferenceBackendLibrary(RootModel[InferenceBackendLibraryRoot]):or invalid key, a model spec that is not a table, a missing per-backend TOML) staysfatal in both modes: a config typo must never silently delete a backend, because thecommands that boot leniently (validate, show, dry runs) would then report the far- more confusing "model not found" for every handle that backend served.+ more confusing "model not found" for every handle that backend served. A *stale* key+ — one the ledger explains — is not a typo and is not covered by this flag at all: it+ is carried forward in both modes, or fatal in both, according to the ledger alone."""+ stale_plans: list[MigrationPlan] = []try:backends_dict = load_toml_from_path(path=backends_library_path)except FileNotFoundError as file_not_found_exc:msg = f"Could not find inference backend library at '{backends_library_path}': {file_not_found_exc}"raise InferenceBackendLibraryNotFoundError(msg) from file_not_found_exc- except ValidationError as exc:- valiation_error_msg = format_pydantic_validation_error(exc)- msg = f"Invalid inference backend library configuration in '{backends_library_path}': {valiation_error_msg}"- raise InferenceBackendLibraryValidationError(msg) from exc# Create a partial function with the secrets provider boundsubstitute_vars_with_provider = partial(substitute_vars, secrets_provider=secrets_provider)@@ -154,7 +209,15 @@ class InferenceBackendLibrary(RootModel[InferenceBackendLibraryRoot]):for backend_blueprint_key in backend_dict:if backend_blueprint_key not in backend_blueprint_standard_fields:extra_config[backend_blueprint_key] = inference_backend_blueprint_dict.pop(backend_blueprint_key)- backend_blueprint = InferenceBackendBlueprint.model_validate(inference_backend_blueprint_dict)+ try:+ backend_blueprint = InferenceBackendBlueprint.model_validate(inference_backend_blueprint_dict)+ except ValidationError as validation_error:+ # The index file's own refusal, said with the backend and the file in front of the+ # analysis: pydantic's error alone names a field, and every table of this file has+ # that field.+ validation_error_msg = format_pydantic_validation_error(validation_error)+ msg = f"Invalid inference backend '{backend_name}' in '{backends_library_path}': {validation_error_msg}"+ raise InferenceBackendLibraryValidationError(msg) from validation_error# Handle pipelex_gateway specially - use remote configbackend_config_source: str@@ -164,7 +227,14 @@ class InferenceBackendLibrary(RootModel[InferenceBackendLibraryRoot]):if lenient:log.verbose(f"Skipping backend '{backend_name}': gateway model specs not available")continue- msg = "Pipelex Gateway backend is enabled but remote model specs were not provided"+ # A caller's omission rather than a user's: the boot fetches the gateway's+ # specs whenever `is_pipelex_gateway_enabled` reads this same file as enabled,+ # and that reader and this loop agree on what "enabled" means. Reachable only+ # by loading the library directly without a gateway config.+ msg = (+ f"Backend '{backend_name}' is enabled in '{backends_library_path}' but no Pipelex Gateway model specs "+ "were given to the loader: pass `gateway_config`, or disable the backend"+ )raise InferenceBackendLibraryError(msg)extra_config["aws_region"] = gateway_config.aws_regionmodel_spec_source = ModelSpecSource.REMOTE_GATEWAY@@ -182,56 +252,26 @@ class InferenceBackendLibrary(RootModel[InferenceBackendLibraryRoot]):substitute_vars_with_provider=substitute_vars_with_provider,)- defaults_dict: dict[str, Any] = model_specs_dict.pop("defaults", {})- backend_model_specs: dict[str, InferenceModelSpec] = {}- for model_spec_name, value in model_specs_dict.items():- if not isinstance(value, dict):- msg = f"Model spec '{model_spec_name}' for backend '{backend_name}' from {backend_config_source} is not a dictionary"- raise InferenceModelSpecError(msg)- model_spec_dict: dict[str, Any] = cast("dict[str, Any]", value)- try:- # A per-model key the blueprint does not know is a request header only if it is shaped- # like one; anything else is a typo or a dead field, and what happens to it depends on- # where the table came from.- key_split = split_model_spec_keys(model_spec_dict=model_spec_dict)- if key_split.rejected:- match model_spec_source:- case ModelSpecSource.LOCAL_FILE:- # Fatal in lenient mode too: leniency covers credentials only (see the docstring),- # and this is not a credentials error, so the lenient `except` below lets it through.- plural = "s" if len(key_split.rejected) > 1 else ""- msg = (- f"Unknown key{plural} on model '{model_spec_name}' for backend '{backend_name}' "- f"from {backend_config_source}: {describe_rejected_keys(rejected=key_split.rejected)}"- )- raise InferenceBackendLibraryError(msg)- case ModelSpecSource.REMOTE_GATEWAY:- # Version skew, the same judgement `drop_unknown_gateway_defaults` makes for the- # `defaults` block: pruned, and silently — this can run before the log hub is set.- pass- # Start from the defaults, then override with the model's own fields- model_spec_blueprint_dict = defaults_dict.copy()- model_spec_blueprint_dict.update(key_split.fields)- model_spec_blueprint = InferenceModelSpecBlueprint.model_validate(model_spec_blueprint_dict)- model_spec = InferenceModelSpecFactory.make_inference_model_spec(- backend_name=backend_name,- name=model_spec_name,- blueprint=model_spec_blueprint,- backend_listed_constraints=backend_blueprint.listed_constraints,- backend_valued_constraints=backend_blueprint.valued_constraints,- extra_headers=key_split.headers,- )- backend_model_specs[model_spec_name] = model_spec- except ValidationError as validation_error:- validation_error_msg = format_pydantic_validation_error(validation_error)- msg = (- f"Invalid inference model spec '{model_spec_name}' for backend '{backend_name}' "- f"from {backend_config_source}: {validation_error_msg}"- )- raise InferenceBackendLibraryError(msg) from validation_error- except InferenceModelSpecError as exc:- msg = f"Failed to load inference model spec '{model_spec_name}' for backend '{backend_name}' from {backend_config_source}"- raise InferenceBackendLibraryError(msg) from exc+ try:+ backend_model_specs = self._build_backend_model_specs(+ model_specs_dict=model_specs_dict,+ backend_name=backend_name,+ backend_config_source=backend_config_source,+ model_spec_source=model_spec_source,+ backend_blueprint=backend_blueprint,+ )+ except InferenceBackendLibraryError:+ recovered = self._local_model_specs_the_ledger_can_explain(+ backend_name=backend_name,+ backends_dir_path=backends_dir_path,+ model_spec_source=model_spec_source,+ backend_blueprint=backend_blueprint,+ substitute_vars_with_provider=substitute_vars_with_provider,+ )+ if recovered is None:+ raise+ backend_model_specs = recovered.model_specs+ stale_plans.extend(recovered.plans)backend = InferenceBackendFactory.make_inference_backend(name=backend_name,blueprint=backend_blueprint,@@ -245,6 +285,134 @@ class InferenceBackendLibrary(RootModel[InferenceBackendLibraryRoot]):continueraise+ # One warning for the whole load rather than one per backend: a schema change lands on every+ # file of the directory at once, and a user reading a dozen warnings would learn nothing the+ # first did not already say.+ self._stale_warning = stale_configuration_warning(plans=stale_plans, walked_dirs=config_manager.existing_config_dirs) if stale_plans else None++ @classmethod+ def _build_backend_model_specs(+ cls,+ *,+ model_specs_dict: BackendModelSpecs,+ backend_name: str,+ backend_config_source: str,+ model_spec_source: ModelSpecSource,+ backend_blueprint: InferenceBackendBlueprint,+ ) -> "dict[str, InferenceModelSpec]":+ """Turn one backend's raw tables into model specs: pop `[defaults]`, split, merge, validate.++ Its own method so the boot-tolerance retry can run it a second time over a migrated document+ without duplicating a line of it. The read is non-destructive — `[defaults]` is popped from a+ copy — because the caller may still need the original tables when this raises.+ """+ remaining_tables = dict(model_specs_dict)+ defaults_dict: dict[str, Any] = remaining_tables.pop(MODEL_SPEC_DEFAULTS_TABLE, {})+ backend_model_specs: dict[str, InferenceModelSpec] = {}+ for model_spec_name, value in remaining_tables.items():+ if not isinstance(value, dict):+ msg = f"Model spec '{model_spec_name}' for backend '{backend_name}' from {backend_config_source} is not a dictionary"+ raise InferenceModelSpecError(msg)+ model_spec_dict: dict[str, Any] = cast("dict[str, Any]", value)+ try:+ # A per-model key the blueprint does not know is a request header only if it is shaped+ # like one; anything else is a typo or a dead field, and what happens to it depends on+ # where the table came from.+ key_split = split_model_spec_keys(model_spec_dict=model_spec_dict)+ if key_split.rejected:+ match model_spec_source:+ case ModelSpecSource.LOCAL_FILE:+ # Fatal in lenient mode too: leniency covers credentials only (see the docstring),+ # and this is not a credentials error, so the lenient `except` in `load` lets it+ # through. What may still catch it is the ledger, one level up.+ plural = "s" if len(key_split.rejected) > 1 else ""+ msg = (+ f"Unknown key{plural} on model '{model_spec_name}' for backend '{backend_name}' "+ f"from {backend_config_source}: {describe_rejected_keys(rejected=key_split.rejected)}"+ )+ raise InferenceBackendLibraryError(msg)+ case ModelSpecSource.REMOTE_GATEWAY:+ # Version skew, the same judgement `drop_unknown_gateway_defaults` makes for the+ # `defaults` block: pruned, and silently — this can run before the log hub is set.+ pass+ # Start from the defaults, then override with the model's own fields+ model_spec_blueprint_dict = defaults_dict.copy()+ model_spec_blueprint_dict.update(key_split.fields)+ model_spec_blueprint = InferenceModelSpecBlueprint.model_validate(model_spec_blueprint_dict)+ model_spec = InferenceModelSpecFactory.make_inference_model_spec(+ backend_name=backend_name,+ name=model_spec_name,+ blueprint=model_spec_blueprint,+ backend_listed_constraints=backend_blueprint.listed_constraints,+ backend_valued_constraints=backend_blueprint.valued_constraints,+ extra_headers=key_split.headers,+ )+ backend_model_specs[model_spec_name] = model_spec+ except ValidationError as validation_error:+ validation_error_msg = format_pydantic_validation_error(validation_error)+ msg = (+ f"Invalid inference model spec '{model_spec_name}' for backend '{backend_name}' "+ f"from {backend_config_source}: {validation_error_msg}"+ )+ raise InferenceBackendLibraryError(msg) from validation_error+ except InferenceModelSpecError as exc:+ msg = f"Failed to load inference model spec '{model_spec_name}' for backend '{backend_name}' from {backend_config_source}"+ raise InferenceBackendLibraryError(msg) from exc+ return backend_model_specs++ def _local_model_specs_the_ledger_can_explain(+ self,+ *,+ backend_name: str,+ backends_dir_path: str,+ model_spec_source: ModelSpecSource,+ backend_blueprint: InferenceBackendBlueprint,+ substitute_vars_with_provider: Any,+ ) -> RecoveredModelSpecs | None:+ """This backend's specs rebuilt from its file as the ledger would leave it, or `None`.++ `None` covers both ways this declines — the ledger had nothing to say about the file, or it+ did and the result still does not load. Neither is this method's to report: the caller+ re-raises the error the file actually produced, which names the key the user can act on,+ where "migration did not help" would name nothing.++ **One file, and only a local one.** The helper deep-merges the paths it is given, which is+ right for a tier stack and wrong here — backend files are independent documents that share no+ keys, so they are replayed one at a time. And the gateway backend is left out on purpose:+ `GatewayConfigMerger` ignores a local `[defaults]` outright and keeps only `sdk` and+ `structure_method` from a per-model override, so a stale key in `pipelex_gateway.toml` is+ filtered out before any spec is built and can never be what refused the load. `pipelex+ migrate` still repairs that file on disk — the surface claims every `*.toml` in the directory+ — but at boot there is nothing there to carry forward.+ """+ match model_spec_source:+ case ModelSpecSource.REMOTE_GATEWAY:+ return None+ case ModelSpecSource.LOCAL_FILE:+ pass+ path_to_model_specs_toml = backend_toml_path(backends_dir_path=backends_dir_path, backend_name=backend_name)+ replayed = replay_surface_files_in_memory(surface_id=INFERENCE_BACKEND_CONFIG_SURFACE_ID, paths=[path_to_model_specs_toml])+ if replayed is None:+ return None+ backend_config_source = f"file '{path_to_model_specs_toml}'"+ try:+ migrated_specs_dict = self._substitute_model_spec_vars(+ model_specs_dict=replayed.config_dict,+ backend_name=backend_name,+ source=backend_config_source,+ substitute_vars_with_provider=substitute_vars_with_provider,+ )+ model_specs = self._build_backend_model_specs(+ model_specs_dict=migrated_specs_dict,+ backend_name=backend_name,+ backend_config_source=backend_config_source,+ model_spec_source=model_spec_source,+ backend_blueprint=backend_blueprint,+ )+ except (InferenceBackendLibraryError, InferenceModelSpecError, InferenceBackendCredentialsError):+ return None+ return RecoveredModelSpecs(model_specs=model_specs, plans=replayed.plans)+def _load_gateway_model_specs(self,gateway_config: GatewayConfig,@@ -309,7 +477,7 @@ class InferenceBackendLibrary(RootModel[InferenceBackendLibraryRoot]):InferenceBackendLibraryError: If the file is missing.InferenceBackendCredentialsError: If variable substitution fails."""- path_to_model_specs_toml = f"{backends_dir_path}/{backend_name}.toml"+ path_to_model_specs_toml = backend_toml_path(backends_dir_path=backends_dir_path, backend_name=backend_name)try:model_specs_dict_raw = load_toml_from_path(path=path_to_model_specs_toml)except FileNotFoundError as file_not_found_exc:diff --git a/pipelex/cogt/model_backends/model_spec_document.py b/pipelex/cogt/model_backends/model_spec_document.pynew file mode 100644index 000000000..b44bf1c11--- /dev/null+++ b/pipelex/cogt/model_backends/model_spec_document.py@@ -0,0 +1,95 @@+"""One backend definition file, read as a document — the shape both of its readers share.++A backend file is not one model. It is an optional `[defaults]` table plus one root table per model+name, and neither half is an `InferenceModelSpecBlueprint` on its own: `sdk` is required and+normally lives in `[defaults]`, so only the merge of the two validates. Two readers need that+stated, and they must never disagree:++- **the loader** (`InferenceBackendLibrary.load`) merges and validates for real, to build the specs;+- **the `inference-backend` migration surface**, which needs the same verdict on a whole document+ (is the migrated file still one the loader accepts?) and a *field-level* projection of one root+ table (what paths does a backend file have, for the fingerprint?).++So both live here rather than in the migration package: a surface that validated a backend document+its own way would let the gate go green over a file that does not boot, which is the one thing the+gate exists to prevent.++One asymmetry in the loader is load-bearing and reproduced faithfully below: **`[defaults]` is not+passed through `split_model_spec_keys`.** It is copied wholesale into every model's blueprint dict,+so a key the blueprint no longer knows fails *every* table with `extra_forbidden`, while the same key+on one model is rejected by name as `NOT_HEADER_SHAPED`. That is why one dead key in `[defaults]`+breaks a whole backend and why the two halves need two different remedies in the migration ledger.+"""++from typing import Any, cast++from pydantic import ValidationError++from pipelex.cogt.model_backends.model_spec_factory import InferenceModelSpecBlueprint+from pipelex.cogt.model_backends.model_spec_keys import describe_rejected_keys, split_model_spec_keys+from pipelex.tools.typing.pydantic_utils import format_pydantic_validation_error++MODEL_SPEC_DEFAULTS_TABLE = "defaults"+"""The one root table of a backend file that is not a model: what every model in it starts from."""+++class InferenceModelSpecFileNode(InferenceModelSpecBlueprint):+ """One root table of a backend file, on its own — every field optional.++ **A projection for the fingerprint, never a validator.** Nothing loads a backend file through+ this model: it exists so the migration fingerprint can record what paths a backend file *has*+ without claiming that any one table must carry them. Every root table is a partial spec —+ `[defaults]` holds what the models share, a model table holds what it overrides — so the only+ honest projection is one in which nothing is required.++ It is a subclass rather than a hand-written twin so that **nothing** can drift: not the field+ set, and not the annotations and bounds either. A twin kept honest by a name-comparison test+ would still let a field's *type* move on the blueprint alone, and the fingerprint would go on+ recording the old one — a narrowing the coverage gate is meant to catch, going quiet. The one+ thing to keep true is that nothing here is required, which `sdk` is the only field to override+ today; a new required field on the blueprint turns the guard in+ `tests/unit/pipelex/cogt/model_backends/test_model_spec_document.py` red, which is the moment to+ decide whether it is genuinely required of a *file* or only of a merged spec.++ The widening of `sdk` is what makes this a subclass a type checker objects to, and the objection+ is right in general: something holding a blueprint may read `.sdk` as a `str`. It is silenced+ rather than designed around because this model is never constructed and never validated against+ — only `model_fields` is ever read — and the alternatives all cost more than they buy: a+ hand-written twin reopens the type-drift hole above, a `create_model` derivation drops the+ per-field metadata the fingerprint reads, and a placeholder default (`sdk: str = ""`) buys the+ type checker's silence by putting a value in the model that no file ever has.+ """++ sdk: str | None = None # type: ignore[assignment] # pyright: ignore[reportIncompatibleVariableOverride]+++def describe_model_spec_document_rejection(*, document: dict[str, Any]) -> str | None:+ """Why the current schema refuses this backend document, or `None` when it accepts it.++ Faithful to `InferenceBackendLibrary.load` and to nothing else — it pops `[defaults]`, splits the+ header-shaped keys off each model table, merges, and validates the result against the real+ blueprint. A document this accepts is one the loader loads; a document this refuses is one that+ breaks a boot.++ A **reason string rather than an exception** on purpose. The caller is the migration gate, which+ reports the reason as one issue among many in its own vocabulary, and a new error class here+ would buy nothing and cost the error-identity snapshot and the generated reference page.+ """+ defaults = document.get(MODEL_SPEC_DEFAULTS_TABLE, {})+ if not isinstance(defaults, dict):+ return f"'{MODEL_SPEC_DEFAULTS_TABLE}' is not a table"+ typed_defaults = cast("dict[str, Any]", defaults)+ for model_name, value in document.items():+ if model_name == MODEL_SPEC_DEFAULTS_TABLE:+ continue+ if not isinstance(value, dict):+ return f"model '{model_name}' is not a table"+ typed_model_table = cast("dict[str, Any]", value)+ key_split = split_model_spec_keys(model_spec_dict=typed_model_table)+ if key_split.rejected:+ return f"model '{model_name}': {describe_rejected_keys(rejected=key_split.rejected)}"+ try:+ InferenceModelSpecBlueprint.model_validate({**typed_defaults, **key_split.fields})+ except ValidationError as exc:+ return f"model '{model_name}': {format_pydantic_validation_error(exc)}"+ return Nonediff --git a/pipelex/cogt/models/model_manager.py b/pipelex/cogt/models/model_manager.pyindex 95c764b40..05ea0d70e 100644--- a/pipelex/cogt/models/model_manager.py+++ b/pipelex/cogt/models/model_manager.py@@ -2,6 +2,7 @@ from pathlib import Pathfrom typing_extensions import override+from pipelex import logfrom pipelex.cogt.exceptions import GatewayUnknownModelError, ModelManagerErrorfrom pipelex.cogt.extract.extract_setting import ExtractSettingfrom pipelex.cogt.img_gen.img_gen_setting import ImgGenSetting@@ -82,6 +83,12 @@ class ModelManager(ModelManagerAbstract):gateway_config=gateway_config,lenient=not needs_inference,)+ # The loader parks its stale-configuration warning rather than logging it, so that the+ # doctor's per-backend probe — which loads the whole library once per backend — does not+ # repeat one directory's warning a dozen times. This is the boot that owes the user the+ # single copy, and by here logging is configured.+ if (stale_warning := self.inference_backend_library.take_stale_configuration_warning()) is not None:+ log.warning(stale_warning)enabled_backends = self.inference_backend_library.all_enabled_backends()self._routing_profile = load_active_routing_profile(routing_profile_library_path=routing_profile_library_path or str(config_manager.routing_profiles_file_path),diff --git a/pipelex/core/validation.py b/pipelex/core/validation.pyindex 6718fc3f5..b7f6a8ce0 100644--- a/pipelex/core/validation.py+++ b/pipelex/core/validation.py@@ -59,8 +59,9 @@ def report_validation_error(validation_error: The pydantic error to translate.surface_id: The configuration surface whose model refused, when one did. Naming it is whatturns on the migration scan; a caller validating something that is not a configuration- surface — a `.mthds` bundle, an inference backend file, a model deck — passes nothing- and gets the translation alone.+ surface — a `.mthds` bundle, a model deck, a routing profile — passes nothing and gets+ the translation alone. An inference backend file *is* one, and the boot that catches+ its library's refusal is what names it.config_dirs: The directories the refused configuration was loaded from, when the callerbypassed the global/project layering (`doctor --global`, an embedder's `config_dir=`).The scan then diagnoses those and only those; `None` is the ordinary load, and the@@ -70,6 +71,56 @@ def report_validation_error(The translated message and, when a scan found something, the structured migration block."""message = analyze_pydantic_validation_error(validation_error).error_msg+ return _report_after_the_scan(message=message, surface_id=surface_id, config_dirs=config_dirs)+++def report_config_refusal(+ *,+ refusal: Exception,+ surface_id: str | None = None,+ config_dirs: list[Path] | None = None,+) -> ValidationErrorReport:+ """The same report as `report_validation_error`, for a refusal a loader has already put into words.++ **The body is the refusal's own message, whole.** A loader that catches pydantic's error names+ *where* before it quotes the analysis — `Invalid inference model spec 'gpt-4o' for backend+ 'openai' from file '…/openai.toml': Validation error(s): …` — and that sentence is the only thing+ standing between a reader and a grep over a directory of files that all have a `max_tokens`. A+ report that reached through to the `ValidationError` and translated *that* threw the sentence+ away and kept the part the reader could least act on. So nothing is reached through here: the+ exception is trusted to have said its piece, and a bare `ValidationError` — one no loader+ wrapped — is the single case that gets translated, because pydantic's own rendering is not what+ a user should read.++ The migration scan is the constant half, and it runs whatever the body is. A backend definition+ file is the case that made this necessary: a per-model key that is not header-shaped is rejected+ by name rather than by the model, so the refusal reaching the boot carries no pydantic error at+ all — and a report keyed on "reach through to the `ValidationError`" would silently offer that+ file no migration block, which is the one thing it needs most.++ Not to be confused with `raise_config_setup_error`, which faces the same fork and takes the other+ branch on purpose: it *re-raises* a refusal that carries no pydantic error, because its callers+ cannot continue and the refusal's own message is already the whole account. A caller that has to+ compose a message rather than raise one — the boot, which names the component and chooses a tail+ — needs the report either way, and that is this function.++ Args:+ refusal: The refusal as raised. Its own message is the body — or, for a bare pydantic+ error, the field-level analysis of it.+ surface_id: The configuration surface whose files refused, when one did — see+ `report_validation_error`. `None` gets the body alone and pays for no walk.+ config_dirs: The directories the refused configuration was loaded from, when the caller+ named them — see `report_validation_error`.++ Returns:+ The body and, when a scan found something, the structured migration block.+ """+ message = analyze_pydantic_validation_error(refusal).error_msg if isinstance(refusal, ValidationError) else str(refusal)+ return _report_after_the_scan(message=message, surface_id=surface_id, config_dirs=config_dirs)+++def _report_after_the_scan(*, message: str, surface_id: str | None, config_dirs: list[Path] | None) -> ValidationErrorReport:+ """The body, plus the migration paragraph and block when a surface was named and the scan found something."""if surface_id is None:return ValidationErrorReport(message=message)block = _pending_migration(surface_id=surface_id, config_dirs=config_dirs)diff --git a/pipelex/migration/coverage.py b/pipelex/migration/coverage.pyindex 858bd7ef9..b6bf8b7f2 100644--- a/pipelex/migration/coverage.py+++ b/pipelex/migration/coverage.py@@ -43,7 +43,7 @@ from pathlib import Pathfrom pydantic import BaseModel, ConfigDict, Field-from pipelex.migration.fingerprint import PATH_SEPARATOR, TABLE_TYPE, SurfaceFingerprint, compute_fingerprint+from pipelex.migration.fingerprint import PATH_SEPARATOR, TABLE_TYPE, SurfaceFingerprintfrom pipelex.migration.goldens import defaults_golden_path, read_fingerprint_goldenfrom pipelex.migration.ledger import MigrationEntry, MigrationLedger, load_ledgerfrom pipelex.migration.narrowing import describe_narrowing, is_remappable, lost_enumerated_spellings@@ -453,12 +453,10 @@ def check_surface(*, surface: Surface, migration_dir: Path) -> list[CoverageIssuissues = _check_ledger_agrees_with_registry(surface=surface, ledger=ledger)current_version = ledger.surface.current_schema_version- live = compute_fingerprint(- surface_id=surface_id,- schema_version=current_version,- config_model=surface.config_model,- defaults_document=surface.read_defaults_document(),- )+ # Through the surface rather than straight to `compute_fingerprint`: the surface holds both+ # halves of the projection — the model tree and whether the document's root is open — and one+ # route is what keeps this gate and the regenerator fingerprinting the same shape.+ live = surface.fingerprint_at(schema_version=current_version)issues.extend(check_defaults_layer(surface_id=surface_id, fingerprint=live))issues.extend(_check_stored_links(surface_id=surface_id, ledger=ledger, live=live, migration_dir=migration_dir))issues.extend(_check_head_link(surface_id=surface_id, current_version=current_version, live=live, migration_dir=migration_dir))@@ -499,6 +497,12 @@ def _check_ledger_agrees_with_registry(*, surface: Surface, ledger: MigrationLedBoth halves are hand-written and both are consumed as truth — the ledger's when a migrationwalks a directory, the registry's when a gate fingerprints a model — so a disagreement betweenthem is a silent mis-migration waiting for the day the two readers meet.++ `subdirectory` is the one that has a second reader on the boot path: a stale-configuration+ warning names `pipelex migrate` only for a file the walk reaches, and it works out where the+ walk reaches from the **ledger**, because its own module may not import the registry. A ledger+ saying one directory while the registry walks another is a boot promising a remedy the command+ then declines, which is exactly the defect the one-derivation rule exists to prevent."""mismatches = [(label, declared, recorded)@@ -506,6 +510,7 @@ def _check_ledger_agrees_with_registry(*, surface: Surface, ledger: MigrationLed("id", surface.surface_id, ledger.surface.id),("base_file", surface.base_file, ledger.surface.base_file),("tier_glob", surface.tier_glob, ledger.surface.tier_glob),+ ("subdirectory", surface.subdirectory.as_posix(), Path(ledger.surface.subdirectory).as_posix()),)if declared != recorded]diff --git a/pipelex/migration/diagnosis.py b/pipelex/migration/diagnosis.pyindex c82516caf..0fae11cde 100644--- a/pipelex/migration/diagnosis.py+++ b/pipelex/migration/diagnosis.py@@ -14,7 +14,7 @@ put to the surface's *fingerprint*: the same projection the coverage gate diffs,model the surface names. The diagnosis therefore lives here, beside the runner that has a surface,rather than inside the engine that has only text.-Three rules make the answer trustworthy rather than merely computable.+Four rules make the answer trustworthy rather than merely computable.- **The document diagnosed is the one the run leaves behind**, not the one it found. Everything theledger explains has been carried forward by then, so what is left over is genuinely left over. On@@ -28,6 +28,10 @@ Three rules make the answer trustworthy rather than merely computable.says `levels.*` where the file says `levels.my_package`, and a typo *inside* such an entry isreported at `queues.*.retries`. The unknown segment is named, because naming it is the wholepoint; the user's own key beside it is not.+- **A key the surface admits by shape is not unexplained.** One surface has a key class that is legal+ without being named by any model: a backend file's per-model request headers, legal because they are+ shaped like headers. The surface is asked (`Surface.admits_unnamed_key`), and the answer is+ deliberately narrow — a key with no hyphen is still a typo, and still reported.See `docs/migration-ledger.md` → "The downgrade direction"."""@@ -41,6 +45,7 @@ from pipelex.migration.fingerprint import PATH_SEPARATOR, TABLE_TYPE, PathFingerfrom pipelex.migration.ledger import MigrationEntry, MigrationLedgerfrom pipelex.migration.material import spelling_after_replayfrom pipelex.migration.plan import BlockedEntry, UnexplainedPath+from pipelex.migration.surfaces import Surfacefrom pipelex.migration.walk import op_source_pathfrom pipelex.suggested_fix import WILDCARD_SEGMENT, RemapValueOpfrom pipelex.system.configuration.config_surface import strip_reserved_meta@@ -60,6 +65,7 @@ tell apart once they are asked.def diagnose_unexplained_paths(*,+ surface: Surface,fingerprint: SurfaceFingerprint,document: dict[str, Any],ledger: MigrationLedger,@@ -69,6 +75,13 @@ def diagnose_unexplained_paths(The document must be the one the run leaves behind. Passing the pre-migration document wouldreport every stale key the ledger is about to repair, which is the opposite of the diagnosis.++ The surface is here for the fourth rule, which the other three did not need: **a key the surface+ itself admits by shape rather than by name is not unexplained.** A backend file's per-model+ request headers are the case — see `Surface.admits_unnamed_key`. It is asked of the surface and+ not read off the fingerprint because it is a live rule about a key's spelling, and because the+ coarser answer a snapshot could carry ("this node takes extras") would silence exactly the key+ class the ledger is there to repair."""# A deep copy, because the strip below edits the nested `[meta]` table in place and this# function only ever reads what it was handed.@@ -83,6 +96,7 @@ def diagnose_unexplained_paths(table=diagnosed,document_prefix=(),schema_prefix=(),+ surface=surface,fingerprint=fingerprint,accounted=accounted,unexplained=unexplained,@@ -95,6 +109,7 @@ def _diagnose_table(table: dict[str, Any],document_prefix: tuple[str, ...],schema_prefix: tuple[str, ...],+ surface: Surface,fingerprint: SurfaceFingerprint,accounted: list[str],unexplained: list[UnexplainedPath],@@ -109,7 +124,9 @@ def _diagnose_table(document_path = (*document_prefix, str(key))resolved = _resolve_against_schema(schema_prefix=schema_prefix, key=str(key), fingerprint=fingerprint)if resolved is None:- if not _is_accounted_for(path=document_path, accounted=accounted):+ if not _is_accounted_for(path=document_path, accounted=accounted) and not surface.admits_unnamed_key(+ node_path=schema_prefix, document_node_path=document_prefix, key=str(key)+ ):unexplained.append(UnexplainedPath(path=PATH_SEPARATOR.join((*schema_prefix, str(key))), note=UNEXPLAINED_NOTE))# Never descended into: an unknown table's contents are unknown because it is, and# listing every key inside it buries the one name the user has to fix.@@ -126,6 +143,7 @@ def _diagnose_table(table=cast("dict[str, Any]", value),document_prefix=document_path,schema_prefix=schema_path,+ surface=surface,fingerprint=fingerprint,accounted=accounted,unexplained=unexplained,diff --git a/pipelex/migration/fingerprint.py b/pipelex/migration/fingerprint.pyindex d495af7b4..8e957a0f4 100644--- a/pipelex/migration/fingerprint.py+++ b/pipelex/migration/fingerprint.py@@ -138,6 +138,12 @@ class SurfaceFingerprint(BaseModel):surface_id: strschema_version: int+ document_root_is_open: bool = False+ """Whether the document's own root keys belong to the user, as they do in a backend definition+ file — one root table per model name. Recorded here rather than read from the live surface so+ that a **frozen** link answers for the version it describes: an operation is checked against the+ fingerprint before it, and whether *that* schema's root was open is a fact about that schema."""+paths: dict[str, PathFingerprint] = Field(default_factory=dict[str, PathFingerprint])def path_names(self) -> set[str]:@@ -163,19 +169,45 @@ def compute_fingerprint(schema_version: int,config_model: type[BaseModel],defaults_document: dict[str, Any],+ document_root_is_open: bool = False,) -> SurfaceFingerprint:- """Project a surface's model tree against its defaults document."""+ """Project a surface's model tree against its defaults document.++ `document_root_is_open` seeds the walk one segment down, at `*`, for a surface whose document is+ a table per user-chosen key rather than one model — a backend definition file, whose root keys+ are model names. It is a *seed* and not a modelling choice because the two ways of modelling it+ both record paths no such file has: a `dict[str, X]` field puts its own name in front of every+ path, and a `RootModel` puts `root.` there. Neither is addressable in a document, and the+ fingerprint's whole job is to be the vocabulary an operation is written in.++ The defaults document is not consulted for an open root: the keys beneath it are the user's, so+ no default can be attached to any of them, and there is nothing above them to attach one to.+ """collected: dict[str, PathFingerprint] = {}- _walk_model(- config_model=config_model,- prefix=(),- defaults=defaults_document,- ancestry=(config_model,),- collected=collected,- )+ if document_root_is_open:+ # `ancestry=()` and not `(config_model,)`: the recursion guard compares the annotation being+ # recorded against the ancestry, and the annotation *is* `config_model` here — seeded with+ # itself, the guard would fire immediately and record `*` as a table with nothing beneath it.+ _record_field(+ path=(WILDCARD_SEGMENT,),+ annotation=config_model,+ required=True,+ defaults_value=None,+ ancestry=(),+ collected=collected,+ )+ else:+ _walk_model(+ config_model=config_model,+ prefix=(),+ defaults=defaults_document,+ ancestry=(config_model,),+ collected=collected,+ )return SurfaceFingerprint(surface_id=surface_id,schema_version=schema_version,+ document_root_is_open=document_root_is_open,paths={path: collected[path] for path in sorted(collected)},)diff --git a/pipelex/migration/goldens/inference-backend/[email protected] b/pipelex/migration/goldens/inference-backend/[email protected]new file mode 100644index 000000000..e9982a3e3--- /dev/null+++ b/pipelex/migration/goldens/inference-backend/[email protected]@@ -0,0 +1,43 @@+# A backend definition file as it stood before `prompting_target` was removed — hand-authored, and+# the one document in this directory that is neither snapshotted nor regenerated. Entry+# `inference-backend@2` is marked `pre_history` because the change it describes predates the first+# fingerprint, so there is no `[email protected]` on its far side to start from; this file is what it+# starts from instead, and the transform check verifies the entry by migrating it and looking at+# what comes out.+#+# It is shaped on the Portkey backend of that era and trimmed to the tables that make the entry's+# single operation exercised rather than merely declared. Three things are deliberate:+#+# - `prompting_target` sits in `[defaults]`, where one occurrence used to break *every* model of the+# file, because the loader copies that table wholesale into each model's blueprint;+# - one model table carries it and one does not, so the operation is seen both applying and skipping+# within the same document;+# - the header-shaped keys belong to the request rather than to the model, and must come through+# byte-untouched — deleting a key the loader never validates is the failure this witnesses against.++[defaults]+model_type = "llm"+sdk = "portkey_completions"+structure_method = "instructor/openai_tools"+prompting_target = "anthropic"+thinking_mode = "none"++[gpt-4o]+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+costs = { input = 2.5, output = 10.0 }+sdk = "portkey_responses"+structure_method = "instructor/openai_responses_tools"+thinking_mode = "none"+x-portkey-provider = "@openai"++["gemini-2.5-pro"]+model_id = "gemini-2.5-pro"+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+max_prompt_images = 3000+costs = { input = 1.25, output = 10.0 }+thinking_mode = "manual"+prompting_target = "gemini"+x-portkey-provider = "@google"+x-portkey-config = "pc-example-config-id"diff --git a/pipelex/migration/goldens/inference-backend/[email protected] b/pipelex/migration/goldens/inference-backend/[email protected]new file mode 100644index 000000000..e17965a09--- /dev/null+++ b/pipelex/migration/goldens/inference-backend/[email protected]@@ -0,0 +1,301 @@+################################################################################+# Portkey Configuration+################################################################################+#+# This file defines the model specifications for the Portkey backend.+# It contains model definitions for various AI models.+#+# Configuration structure:+# - Each model is defined in its own section with the model name as the header+# - Headers with dots must be quoted (e.g., ["gpt-4.1"])+# - Model costs are in USD per million tokens (input/output)+#+# Documentation: https://docs.pipelex.com+# Support: https://go.pipelex.com/discord+#+################################################################################++################################################################################+# MODEL DEFAULTS+################################################################################++[defaults]+model_type = "llm"+sdk = "portkey_completions"+structure_method = "instructor/openai_tools"+thinking_mode = "none"++################################################################################+# LANGUAGE MODELS+################################################################################++# --- OpenAI LLMs --------------------------------------------------------------+[gpt-4o-mini]+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+costs = { input = 0.15, output = 0.6 }+sdk = "portkey_responses"+structure_method = "instructor/openai_responses_tools"+thinking_mode = "none"+x-portkey-provider = "@openai"++[gpt-4o]+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+costs = { input = 2.5, output = 10.0 }+sdk = "portkey_responses"+structure_method = "instructor/openai_responses_tools"+thinking_mode = "none"+x-portkey-provider = "@openai"++["gpt-4.1-nano"]+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+costs = { input = 0.1, output = 0.4 }+sdk = "portkey_responses"+structure_method = "instructor/openai_responses_tools"+thinking_mode = "none"+x-portkey-provider = "@openai"++["gpt-4.1-mini"]+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+costs = { input = 0.4, output = 1.6 }+sdk = "portkey_responses"+structure_method = "instructor/openai_responses_tools"+thinking_mode = "none"+x-portkey-provider = "@openai"++["gpt-4.1"]+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+costs = { input = 2, output = 8 }+sdk = "portkey_responses"+structure_method = "instructor/openai_responses_tools"+thinking_mode = "none"+x-portkey-provider = "@openai"++[o1]+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+costs = { input = 15.0, output = 60.0 }+valued_constraints = { fixed_temperature = 1 }+sdk = "portkey_responses"+structure_method = "instructor/openai_responses_tools"+thinking_mode = "manual"+x-portkey-provider = "@openai"++[o3-mini]+inputs = ["text"]+outputs = ["text", "structured"]+costs = { input = 1.1, output = 4.4 }+valued_constraints = { fixed_temperature = 1 }+sdk = "portkey_responses"+structure_method = "instructor/openai_responses_tools"+thinking_mode = "manual"+x-portkey-provider = "@openai"++[o3]+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+costs = { input = 2, output = 8 }+valued_constraints = { fixed_temperature = 1 }+sdk = "portkey_responses"+structure_method = "instructor/openai_responses_tools"+thinking_mode = "manual"+x-portkey-provider = "@openai"++[o4-mini]+inputs = ["text"]+outputs = ["text", "structured"]+costs = { input = 1.1, output = 4.4 }+valued_constraints = { fixed_temperature = 1 }+sdk = "portkey_responses"+structure_method = "instructor/openai_responses_tools"+thinking_mode = "manual"+x-portkey-provider = "@openai"++[gpt-5-nano]+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+costs = { input = 0.05, output = 0.4 }+valued_constraints = { fixed_temperature = 1 }+sdk = "portkey_responses"+structure_method = "instructor/openai_responses_tools"+thinking_mode = "manual"+x-portkey-provider = "@openai"++[gpt-5-mini]+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+costs = { input = 0.25, output = 2.0 }+valued_constraints = { fixed_temperature = 1 }+sdk = "portkey_responses"+structure_method = "instructor/openai_responses_tools"+thinking_mode = "manual"+x-portkey-provider = "@openai"++[gpt-5]+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+costs = { input = 1.25, output = 10.0 }+valued_constraints = { fixed_temperature = 1 }+sdk = "portkey_responses"+structure_method = "instructor/openai_responses_tools"+thinking_mode = "manual"+x-portkey-provider = "@openai"++["gpt-5.1"]+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+costs = { input = 1.25, output = 10.0 }+valued_constraints = { fixed_temperature = 1 }+sdk = "portkey_responses"+structure_method = "instructor/openai_responses_tools"+thinking_mode = "manual"+x-portkey-provider = "@openai"++["gpt-5.1-codex"]+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+costs = { input = 1.25, output = 10.0 }+valued_constraints = { fixed_temperature = 1 }+sdk = "portkey_responses"+structure_method = "instructor/openai_responses_tools"+thinking_mode = "manual"+x-portkey-provider = "@openai"++# --- Claude LLMs --------------------------------------------------------------+[claude-3-haiku]+model_id = "claude-3-haiku-20240307"+max_tokens = 4096+inputs = ["text", "images"]+outputs = ["text", "structured"]+max_prompt_images = 100+costs = { input = 0.25, output = 1.25 }+thinking_mode = "none"+x-portkey-provider = "@anthropic"++["claude-3.7-sonnet"]+model_id = "claude-3-7-sonnet-20250219"+max_tokens = 8192+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+max_prompt_images = 100+costs = { input = 3.0, output = 15.0 }+thinking_mode = "manual"+x-portkey-provider = "@anthropic"++[claude-4-sonnet]+model_id = "claude-sonnet-4-20250514"+max_tokens = 64000+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+max_prompt_images = 100+costs = { input = 3.0, output = 15.0 }+thinking_mode = "manual"+x-portkey-provider = "@anthropic"++[claude-4-opus]+model_id = "claude-opus-4-20250514"+max_tokens = 32000+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+max_prompt_images = 100+costs = { input = 3.0, output = 15.0 }+thinking_mode = "manual"+x-portkey-provider = "@anthropic"++["claude-4.1-opus"]+model_id = "claude-opus-4-1-20250805"+max_tokens = 32000+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+max_prompt_images = 100+costs = { input = 3.0, output = 15.0 }+thinking_mode = "manual"+x-portkey-provider = "@anthropic"++["claude-4.5-sonnet"]+model_id = "claude-sonnet-4-5-20250929"+max_tokens = 64000+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+max_prompt_images = 100+costs = { input = 3.0, output = 15.0 }+thinking_mode = "manual"+x-portkey-provider = "@anthropic"++["claude-4.5-haiku"]+model_id = "claude-haiku-4-5-20251001"+max_tokens = 64000+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+max_prompt_images = 100+costs = { input = 1.0, output = 5.0 }+thinking_mode = "manual"+x-portkey-provider = "@anthropic"++["claude-4.5-opus"]+model_id = "claude-opus-4-5-20251101"+max_tokens = 64000+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+max_prompt_images = 100+costs = { input = 5.0, output = 25.0 }+thinking_mode = "manual"+x-portkey-provider = "@anthropic"++["claude-4.6-opus"]+model_id = "claude-opus-4-6"+max_tokens = 64000+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+max_prompt_images = 100+costs = { input = 5.0, output = 25.0 }+thinking_mode = "adaptive"+x-portkey-provider = "@anthropic"++# --- Gemini LLMs --------------------------------------------------------------+["gemini-2.5-pro"]+model_id = "gemini-2.5-pro"+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+max_prompt_images = 3000+costs = { input = 1.25, output = 10.0 }+thinking_mode = "manual"+x-portkey-provider = "@google"++["gemini-2.5-flash"]+model_id = "gemini-2.5-flash"+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+costs = { input = 0.30, output = 2.50 }+thinking_mode = "manual"+x-portkey-provider = "@google"++["gemini-2.5-flash-lite"]+model_id = "gemini-2.5-flash-lite"+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+costs = { input = 0.10, output = 0.40 }+thinking_mode = "manual"+x-portkey-provider = "@google"++["gemini-3.1-pro"]+model_id = "gemini-3.1-pro-preview"+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+max_prompt_images = 3000+costs = { input = 2, output = 12.0 }+thinking_mode = "adaptive"+x-portkey-provider = "@google"++["gemini-3.0-flash"]+model_id = "gemini-3-flash-preview"+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+max_prompt_images = 3000+costs = { input = 0.5, output = 3.0 }+thinking_mode = "adaptive"+x-portkey-provider = "@google"diff --git a/pipelex/migration/goldens/inference-backend/[email protected] b/pipelex/migration/goldens/inference-backend/[email protected]new file mode 100644index 000000000..e17965a09--- /dev/null+++ b/pipelex/migration/goldens/inference-backend/[email protected]@@ -0,0 +1,301 @@+################################################################################+# Portkey Configuration+################################################################################+#+# This file defines the model specifications for the Portkey backend.+# It contains model definitions for various AI models.+#+# Configuration structure:+# - Each model is defined in its own section with the model name as the header+# - Headers with dots must be quoted (e.g., ["gpt-4.1"])+# - Model costs are in USD per million tokens (input/output)+#+# Documentation: https://docs.pipelex.com+# Support: https://go.pipelex.com/discord+#+################################################################################++################################################################################+# MODEL DEFAULTS+################################################################################++[defaults]+model_type = "llm"+sdk = "portkey_completions"+structure_method = "instructor/openai_tools"+thinking_mode = "none"++################################################################################+# LANGUAGE MODELS+################################################################################++# --- OpenAI LLMs --------------------------------------------------------------+[gpt-4o-mini]+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+costs = { input = 0.15, output = 0.6 }+sdk = "portkey_responses"+structure_method = "instructor/openai_responses_tools"+thinking_mode = "none"+x-portkey-provider = "@openai"++[gpt-4o]+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+costs = { input = 2.5, output = 10.0 }+sdk = "portkey_responses"+structure_method = "instructor/openai_responses_tools"+thinking_mode = "none"+x-portkey-provider = "@openai"++["gpt-4.1-nano"]+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+costs = { input = 0.1, output = 0.4 }+sdk = "portkey_responses"+structure_method = "instructor/openai_responses_tools"+thinking_mode = "none"+x-portkey-provider = "@openai"++["gpt-4.1-mini"]+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+costs = { input = 0.4, output = 1.6 }+sdk = "portkey_responses"+structure_method = "instructor/openai_responses_tools"+thinking_mode = "none"+x-portkey-provider = "@openai"++["gpt-4.1"]+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+costs = { input = 2, output = 8 }+sdk = "portkey_responses"+structure_method = "instructor/openai_responses_tools"+thinking_mode = "none"+x-portkey-provider = "@openai"++[o1]+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+costs = { input = 15.0, output = 60.0 }+valued_constraints = { fixed_temperature = 1 }+sdk = "portkey_responses"+structure_method = "instructor/openai_responses_tools"+thinking_mode = "manual"+x-portkey-provider = "@openai"++[o3-mini]+inputs = ["text"]+outputs = ["text", "structured"]+costs = { input = 1.1, output = 4.4 }+valued_constraints = { fixed_temperature = 1 }+sdk = "portkey_responses"+structure_method = "instructor/openai_responses_tools"+thinking_mode = "manual"+x-portkey-provider = "@openai"++[o3]+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+costs = { input = 2, output = 8 }+valued_constraints = { fixed_temperature = 1 }+sdk = "portkey_responses"+structure_method = "instructor/openai_responses_tools"+thinking_mode = "manual"+x-portkey-provider = "@openai"++[o4-mini]+inputs = ["text"]+outputs = ["text", "structured"]+costs = { input = 1.1, output = 4.4 }+valued_constraints = { fixed_temperature = 1 }+sdk = "portkey_responses"+structure_method = "instructor/openai_responses_tools"+thinking_mode = "manual"+x-portkey-provider = "@openai"++[gpt-5-nano]+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+costs = { input = 0.05, output = 0.4 }+valued_constraints = { fixed_temperature = 1 }+sdk = "portkey_responses"+structure_method = "instructor/openai_responses_tools"+thinking_mode = "manual"+x-portkey-provider = "@openai"++[gpt-5-mini]+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+costs = { input = 0.25, output = 2.0 }+valued_constraints = { fixed_temperature = 1 }+sdk = "portkey_responses"+structure_method = "instructor/openai_responses_tools"+thinking_mode = "manual"+x-portkey-provider = "@openai"++[gpt-5]+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+costs = { input = 1.25, output = 10.0 }+valued_constraints = { fixed_temperature = 1 }+sdk = "portkey_responses"+structure_method = "instructor/openai_responses_tools"+thinking_mode = "manual"+x-portkey-provider = "@openai"++["gpt-5.1"]+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+costs = { input = 1.25, output = 10.0 }+valued_constraints = { fixed_temperature = 1 }+sdk = "portkey_responses"+structure_method = "instructor/openai_responses_tools"+thinking_mode = "manual"+x-portkey-provider = "@openai"++["gpt-5.1-codex"]+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+costs = { input = 1.25, output = 10.0 }+valued_constraints = { fixed_temperature = 1 }+sdk = "portkey_responses"+structure_method = "instructor/openai_responses_tools"+thinking_mode = "manual"+x-portkey-provider = "@openai"++# --- Claude LLMs --------------------------------------------------------------+[claude-3-haiku]+model_id = "claude-3-haiku-20240307"+max_tokens = 4096+inputs = ["text", "images"]+outputs = ["text", "structured"]+max_prompt_images = 100+costs = { input = 0.25, output = 1.25 }+thinking_mode = "none"+x-portkey-provider = "@anthropic"++["claude-3.7-sonnet"]+model_id = "claude-3-7-sonnet-20250219"+max_tokens = 8192+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+max_prompt_images = 100+costs = { input = 3.0, output = 15.0 }+thinking_mode = "manual"+x-portkey-provider = "@anthropic"++[claude-4-sonnet]+model_id = "claude-sonnet-4-20250514"+max_tokens = 64000+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+max_prompt_images = 100+costs = { input = 3.0, output = 15.0 }+thinking_mode = "manual"+x-portkey-provider = "@anthropic"++[claude-4-opus]+model_id = "claude-opus-4-20250514"+max_tokens = 32000+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+max_prompt_images = 100+costs = { input = 3.0, output = 15.0 }+thinking_mode = "manual"+x-portkey-provider = "@anthropic"++["claude-4.1-opus"]+model_id = "claude-opus-4-1-20250805"+max_tokens = 32000+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+max_prompt_images = 100+costs = { input = 3.0, output = 15.0 }+thinking_mode = "manual"+x-portkey-provider = "@anthropic"++["claude-4.5-sonnet"]+model_id = "claude-sonnet-4-5-20250929"+max_tokens = 64000+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+max_prompt_images = 100+costs = { input = 3.0, output = 15.0 }+thinking_mode = "manual"+x-portkey-provider = "@anthropic"++["claude-4.5-haiku"]+model_id = "claude-haiku-4-5-20251001"+max_tokens = 64000+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+max_prompt_images = 100+costs = { input = 1.0, output = 5.0 }+thinking_mode = "manual"+x-portkey-provider = "@anthropic"++["claude-4.5-opus"]+model_id = "claude-opus-4-5-20251101"+max_tokens = 64000+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+max_prompt_images = 100+costs = { input = 5.0, output = 25.0 }+thinking_mode = "manual"+x-portkey-provider = "@anthropic"++["claude-4.6-opus"]+model_id = "claude-opus-4-6"+max_tokens = 64000+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+max_prompt_images = 100+costs = { input = 5.0, output = 25.0 }+thinking_mode = "adaptive"+x-portkey-provider = "@anthropic"++# --- Gemini LLMs --------------------------------------------------------------+["gemini-2.5-pro"]+model_id = "gemini-2.5-pro"+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+max_prompt_images = 3000+costs = { input = 1.25, output = 10.0 }+thinking_mode = "manual"+x-portkey-provider = "@google"++["gemini-2.5-flash"]+model_id = "gemini-2.5-flash"+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+costs = { input = 0.30, output = 2.50 }+thinking_mode = "manual"+x-portkey-provider = "@google"++["gemini-2.5-flash-lite"]+model_id = "gemini-2.5-flash-lite"+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+costs = { input = 0.10, output = 0.40 }+thinking_mode = "manual"+x-portkey-provider = "@google"++["gemini-3.1-pro"]+model_id = "gemini-3.1-pro-preview"+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+max_prompt_images = 3000+costs = { input = 2, output = 12.0 }+thinking_mode = "adaptive"+x-portkey-provider = "@google"++["gemini-3.0-flash"]+model_id = "gemini-3-flash-preview"+inputs = ["text", "images", "pdf"]+outputs = ["text", "structured"]+max_prompt_images = 3000+costs = { input = 0.5, output = 3.0 }+thinking_mode = "adaptive"+x-portkey-provider = "@google"diff --git a/pipelex/migration/goldens/inference-backend/[email protected] b/pipelex/migration/goldens/inference-backend/[email protected]new file mode 100644index 000000000..8d078725d--- /dev/null+++ b/pipelex/migration/goldens/inference-backend/[email protected]@@ -0,0 +1,155 @@+{+ "surface_id": "inference-backend",+ "schema_version": 1,+ "document_root_is_open": true,+ "paths": {+ "*": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "*.costs": {+ "value_type": "dict[enum, float]",+ "required": false,+ "open_node": true+ },+ "*.costs.*": {+ "value_type": "float",+ "required": true,+ "open_node": false+ },+ "*.enabled": {+ "value_type": "bool",+ "required": false,+ "open_node": false+ },+ "*.endpoint_path": {+ "value_type": "str",+ "required": false,+ "open_node": false+ },+ "*.inputs": {+ "value_type": "list[str]",+ "required": false,+ "open_node": false+ },+ "*.listed_constraints": {+ "value_type": "list[enum]",+ "required": false,+ "enum_members": [+ "max_tokens_must_be_high_enough",+ "temperature_must_be_multiplied_by_2",+ "temperature_unsupported"+ ],+ "open_node": false+ },+ "*.max_prompt_images": {+ "value_type": "int",+ "required": false,+ "open_node": false+ },+ "*.max_tokens": {+ "value_type": "int",+ "required": false,+ "open_node": false+ },+ "*.model_id": {+ "value_type": "str",+ "required": false,+ "open_node": false+ },+ "*.model_type": {+ "value_type": "enum",+ "required": false,+ "enum_members": [+ "img_gen",+ "llm",+ "search",+ "text_extractor"+ ],+ "open_node": false+ },+ "*.outputs": {+ "value_type": "list[str]",+ "required": false,+ "open_node": false+ },+ "*.rules": {+ "value_type": "dict[enum, str]",+ "required": false,+ "open_node": true+ },+ "*.rules.*": {+ "value_type": "str",+ "required": true,+ "open_node": false+ },+ "*.sdk": {+ "value_type": "str",+ "required": false,+ "open_node": false+ },+ "*.structure_method": {+ "value_type": "enum",+ "required": false,+ "enum_members": [+ "instructor/anthropic_json",+ "instructor/anthropic_reasoning_tools",+ "instructor/anthropic_tools",+ "instructor/bedrock_json",+ "instructor/bedrock_tools",+ "instructor/cerebras_json",+ "instructor/cerebras_tools",+ "instructor/cohere_json_schema",+ "instructor/cohere_tools",+ "instructor/fireworks_json",+ "instructor/fireworks_tools",+ "instructor/genai_structured_outputs",+ "instructor/genai_tools",+ "instructor/json",+ "instructor/json_schema",+ "instructor/md_json",+ "instructor/mistral_structured_outputs",+ "instructor/mistral_tools",+ "instructor/openai_json_o1",+ "instructor/openai_parallel_tools",+ "instructor/openai_responses_tools",+ "instructor/openai_responses_tools_with_inbuilt_tools",+ "instructor/openai_structured_outputs",+ "instructor/openai_tools",+ "instructor/openrouter_structured_outputs",+ "instructor/perplexity_json",+ "instructor/vertexai_json",+ "instructor/vertexai_parallel_tools",+ "instructor/vertexai_tools",+ "instructor/writer_tools"+ ],+ "open_node": false+ },+ "*.thinking_mode": {+ "value_type": "enum",+ "required": false,+ "enum_members": [+ "adaptive",+ "manual",+ "none"+ ],+ "open_node": false+ },+ "*.valued_constraints": {+ "value_type": "dict[enum, any]",+ "required": false,+ "open_node": true+ },+ "*.valued_constraints.*": {+ "value_type": "any",+ "required": true,+ "open_node": false+ },+ "*.variant": {+ "value_type": "str",+ "required": false,+ "open_node": false+ }+ }+}diff --git a/pipelex/migration/goldens/inference-backend/[email protected] b/pipelex/migration/goldens/inference-backend/[email protected]new file mode 100644index 000000000..1e07b766c--- /dev/null+++ b/pipelex/migration/goldens/inference-backend/[email protected]@@ -0,0 +1,155 @@+{+ "surface_id": "inference-backend",+ "schema_version": 2,+ "document_root_is_open": true,+ "paths": {+ "*": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "*.costs": {+ "value_type": "dict[enum, float]",+ "required": false,+ "open_node": true+ },+ "*.costs.*": {+ "value_type": "float",+ "required": true,+ "open_node": false+ },+ "*.enabled": {+ "value_type": "bool",+ "required": false,+ "open_node": false+ },+ "*.endpoint_path": {+ "value_type": "str",+ "required": false,+ "open_node": false+ },+ "*.inputs": {+ "value_type": "list[str]",+ "required": false,+ "open_node": false+ },+ "*.listed_constraints": {+ "value_type": "list[enum]",+ "required": false,+ "enum_members": [+ "max_tokens_must_be_high_enough",+ "temperature_must_be_multiplied_by_2",+ "temperature_unsupported"+ ],+ "open_node": false+ },+ "*.max_prompt_images": {+ "value_type": "int",+ "required": false,+ "open_node": false+ },+ "*.max_tokens": {+ "value_type": "int",+ "required": false,+ "open_node": false+ },+ "*.model_id": {+ "value_type": "str",+ "required": false,+ "open_node": false+ },+ "*.model_type": {+ "value_type": "enum",+ "required": false,+ "enum_members": [+ "img_gen",+ "llm",+ "search",+ "text_extractor"+ ],+ "open_node": false+ },+ "*.outputs": {+ "value_type": "list[str]",+ "required": false,+ "open_node": false+ },+ "*.rules": {+ "value_type": "dict[enum, str]",+ "required": false,+ "open_node": true+ },+ "*.rules.*": {+ "value_type": "str",+ "required": true,+ "open_node": false+ },+ "*.sdk": {+ "value_type": "str",+ "required": false,+ "open_node": false+ },+ "*.structure_method": {+ "value_type": "enum",+ "required": false,+ "enum_members": [+ "instructor/anthropic_json",+ "instructor/anthropic_reasoning_tools",+ "instructor/anthropic_tools",+ "instructor/bedrock_json",+ "instructor/bedrock_tools",+ "instructor/cerebras_json",+ "instructor/cerebras_tools",+ "instructor/cohere_json_schema",+ "instructor/cohere_tools",+ "instructor/fireworks_json",+ "instructor/fireworks_tools",+ "instructor/genai_structured_outputs",+ "instructor/genai_tools",+ "instructor/json",+ "instructor/json_schema",+ "instructor/md_json",+ "instructor/mistral_structured_outputs",+ "instructor/mistral_tools",+ "instructor/openai_json_o1",+ "instructor/openai_parallel_tools",+ "instructor/openai_responses_tools",+ "instructor/openai_responses_tools_with_inbuilt_tools",+ "instructor/openai_structured_outputs",+ "instructor/openai_tools",+ "instructor/openrouter_structured_outputs",+ "instructor/perplexity_json",+ "instructor/vertexai_json",+ "instructor/vertexai_parallel_tools",+ "instructor/vertexai_tools",+ "instructor/writer_tools"+ ],+ "open_node": false+ },+ "*.thinking_mode": {+ "value_type": "enum",+ "required": false,+ "enum_members": [+ "adaptive",+ "manual",+ "none"+ ],+ "open_node": false+ },+ "*.valued_constraints": {+ "value_type": "dict[enum, any]",+ "required": false,+ "open_node": true+ },+ "*.valued_constraints.*": {+ "value_type": "any",+ "required": true,+ "open_node": false+ },+ "*.variant": {+ "value_type": "str",+ "required": false,+ "open_node": false+ }+ }+}diff --git a/pipelex/migration/goldens/pipelex-config/[email protected] b/pipelex/migration/goldens/pipelex-config/[email protected]new file mode 100644index 000000000..4cba48750--- /dev/null+++ b/pipelex/migration/goldens/pipelex-config/[email protected]@@ -0,0 +1,24 @@+# The main configuration as it stood before prompting styles became a templating default —+# hand-authored, and the one document in this directory that is neither snapshotted nor+# regenerated. Entry `pipelex-config@2` is marked `pre_history` because the change it describes+# predates the first fingerprint, so there is no `[email protected]` on its far side to start from;+# this file is what it starts from instead, and the transform check verifies the entry by migrating+# it and looking at what comes out.+#+# It is the prompting section as the release before schema 1 shipped it — the per-target map and+# the default that sat beside it — plus one table that has nothing to do with the change. That+# table is the neutrality witness: it is in this document and in the reference document of the+# version this entry lands on alike, so an operation that reached a parent where it meant to reach+# one child would take it with them, and the transform check would say so.++[pipelex.prompting_config]+default_prompting_style = { tag_style = "xml" }++[pipelex.prompting_config.prompting_styles]+openai = { tag_style = "ticks" }+anthropic = { tag_style = "xml" }+mistral = { tag_style = "square_brackets" }+gemini = { tag_style = "xml" }++[pipelex.pipe_run_config]+pipe_stack_limit = 20diff --git a/pipelex/migration/goldens/pipelex-config/[email protected] b/pipelex/migration/goldens/pipelex-config/[email protected]index ce9fd8469..abaf4f65f 100644--- a/pipelex/migration/goldens/pipelex-config/[email protected]+++ b/pipelex/migration/goldens/pipelex-config/[email protected]@@ -1,41 +1,73 @@-####################################################################################################-# Runtime — kernel-layer, process-scoped infrastructure (what runtime_hub brokers)-####################################################################################################-[runtime]-[runtime.storage]+[pipelex]++[pipelex.storage_config]is_fetch_remote_content_enabled = trueis_upload_local_content_enabled = truemethod = "local" # local, in_memory, s3, gcp-[runtime.storage.local]+[pipelex.storage_config.local]uri_format = "{primary_id}/{secondary_id}/{hash}.{extension}"local_storage_path = ".pipelex/storage"-[runtime.storage.in_memory]+[pipelex.storage_config.in_memory]uri_format = "{primary_id}/{secondary_id}/{hash}.{extension}"-[runtime.storage.s3]+[pipelex.storage_config.s3]uri_format = "{primary_id}/{secondary_id}/{hash}.{extension}"bucket_name = ""region = ""signed_urls_lifespan_seconds = 3600-[runtime.storage.gcp]+[pipelex.storage_config.gcp]uri_format = "{primary_id}/{secondary_id}/{hash}.{extension}"bucket_name = ""project_id = ""signed_urls_lifespan_seconds = 3600-[runtime.secrets]+[pipelex.secrets_config]method = "env" # env (built-in); an external plugin may add e.g. vault+[pipelex.observer_config]+observer_dir = "results/observer"++[pipelex.scan_config]+excluded_dirs = [+ ".venv",+ "venv",+ "env",+ ".env",+ "virtualenv",+ ".virtualenv",+ ".git",+ "__pycache__",+ ".pytest_cache",+ ".mypy_cache",+ ".ruff_cache",+ "node_modules",+ "results",+]++[pipelex.builder_config]+fix_loop_max_attempts = 10+default_output_dir = "."+default_bundle_file_name = "bundle"+default_directory_base_name = "pipeline"++[pipelex.reporting_config]+is_log_costs_to_console = true+is_generate_cost_report_file_enabled = false+cost_report_dir_path = "reports"+cost_report_base_name = "cost_report"+cost_report_extension = "csv"+cost_report_unit_scale = 1.0+##################################################################################################### Log config####################################################################################################-[runtime.log]+[pipelex.log_config]default_log_level = "INFO"# console_log_target controls the RichHandler used by the logging system (log.debug/info/...).# Logs are diagnostics → stderr.@@ -62,7 +94,7 @@ log_mode = "rich"poor_loggers = []generic_poor_logger = "#poor-log"-[runtime.log.package_log_levels]+[pipelex.log_config.package_log_levels]# for logger names, the dots "." have been replaced by "-" to avoid toml parsing issues# TODO: use quotes to re-enable names with dotsanthropic = "INFO"@@ -80,7 +112,7 @@ urllib3-util-retry = "INFO"pipelex = "INFO"-[runtime.log.rich_log]+[pipelex.log_config.rich_log_config]is_show_time = falseis_show_level = trueis_link_path_enabled = true@@ -92,65 +124,34 @@ is_tracebacks_show_locals = falsetracebacks_suppress = []keywords_to_hilight = []-[runtime.aws]+[pipelex.aws_config]api_key_method = "env"bedrock_access_variant = "aws_access"-[runtime.reporting]-is_log_costs_to_console = true-is_generate_cost_report_file_enabled = false-cost_report_dir_path = "reports"-cost_report_base_name = "cost_report"-cost_report_extension = "csv"-cost_report_unit_scale = 1.0-####################################################################################################-# Tracing config (event log for distributed tracing across workers)+# Cogt inference config####################################################################################################-[runtime.tracing]-is_enabled = true-backend = "ndjson"--[runtime.tracing.ndjson]-traces_dir = ".pipelex/traces"--[runtime.tracing.dynamodb]-table_name = ""-region = ""--[runtime.observer]-observer_dir = "results/observer"--[runtime.plugins]-# Names of discovered plugins to skip at startup (a denylist). Discovery is the-# source of truth for which plugins are present; this only turns one off.-disabled = []--####################################################################################################-# Inference — the model-calling seam (was [cogt])-####################################################################################################--[inference]+[cogt]transport_max_retries = 2-[inference.model_deck]+[cogt.model_deck_config]is_model_fallback_enabled = truemissing_presets_reaction = "log"-[inference.llm]+[cogt.llm_config]default_max_images = 100is_structure_prompt_enabled = trueis_dump_text_prompts_enabled = falseis_dump_response_text_enabled = falseschema_reask_max_attempts = 3-[inference.llm.instructor]+[cogt.llm_config.instructor_config]is_dump_kwargs_enabled = falseis_dump_response_enabled = falseis_dump_error_enabled = false-[inference.llm.openai.effort_to_level_map]+[cogt.llm_config.openai_config.effort_to_level_map]none = "none"minimal = "minimal"low = "low"@@ -159,10 +160,10 @@ high = "high"xhigh = "xhigh"max = "xhigh"-[inference.llm.anthropic]+[cogt.llm_config.anthropic_config]structured_output_timeout_seconds = 1200-[inference.llm.anthropic.effort_to_level_map]+[cogt.llm_config.anthropic_config.effort_to_level_map]none = "disabled"minimal = "low"low = "low"@@ -171,7 +172,7 @@ high = "high"xhigh = "xhigh"max = "max"-[inference.llm.google.effort_to_level_map]+[cogt.llm_config.google_config.effort_to_level_map]none = "disabled"minimal = "minimal"low = "low"@@ -180,7 +181,7 @@ high = "high"xhigh = "high"max = "high"-[inference.llm.mistral.effort_to_level_map]+[cogt.llm_config.mistral_config.effort_to_level_map]none = "disabled"minimal = "reasoning"low = "reasoning"@@ -189,7 +190,7 @@ high = "reasoning"xhigh = "reasoning"max = "reasoning"-[inference.llm.effort_to_budget_maps.anthropic]+[cogt.llm_config.effort_to_budget_maps.anthropic]none = 0 # Required by validator; unreachable at runtime (level map gates NONE as disabled before budget lookup)minimal = 512low = 1024@@ -198,7 +199,7 @@ high = 16384xhigh = 32768max = 65536-[inference.llm.effort_to_budget_maps.gemini]+[cogt.llm_config.effort_to_budget_maps.gemini]none = 0 # Required by validator; unreachable at runtime (level map gates NONE as disabled before budget lookup)minimal = 512low = 1024@@ -207,7 +208,7 @@ high = 16384xhigh = 32768max = 65536-[inference.llm.generic_templates]+[cogt.llm_config.generic_templates]output_structure_prompt = """@@ -230,10 +231,10 @@ Read the following text carefully and produce the requested structured output fr# Image generation config####################################################################################################-[inference.img_gen.img_gen_job]+[cogt.img_gen_config.img_gen_job_config]is_sync_mode = false-[inference.img_gen.img_gen_param_defaults]+[cogt.img_gen_config.img_gen_param_defaults]nb_steps = 28 # 28 is a good default for Flux, possible values are [1,2,4,8] for SDXL Lightningquality = "low"aspect_ratio = "square" # "square", "landscape_4_3", "landscape_3_2", "landscape_16_9", "landscape_21_9", "landscape_4_1", "landscape_8_1", "portrait_3_4", "portrait_2_3", "portrait_9_16", "portrait_9_21", "portrait_1_4", "portrait_1_8"@@ -244,7 +245,7 @@ safety_tolerance = 5is_raw = falseseed = "auto"-[inference.img_gen.quality_to_steps_maps]+[cogt.img_gen_config.quality_to_steps_maps]flux = { "low" = 14, "medium" = 28, "high" = 56 }sdxl_lightning = { "low" = 2, "medium" = 4, "high" = 8 }qwen_image = { "low" = 10, "medium" = 20, "high" = 28 }@@ -253,14 +254,14 @@ qwen_image = { "low" = 10, "medium" = 20, "high" = 28 }# Extract config####################################################################################################-[inference.extract]+[cogt.extract_config]default_page_views_dpi = 72##################################################################################################### Gateway test config####################################################################################################-[inference.gateway_test.config_id_substitutions]+[cogt.gateway_test_config.config_id_substitutions]# this is an empty dict, it's only used by Pipelex staff to override the ¨Pipelex Gateway¨ config# in order to test new model integrations or debug issues with the Gateway@@ -268,71 +269,21 @@ default_page_views_dpi = 72# Templating config####################################################################################################-[inference.templating]+[pipelex.templating_config]default_templating_style = { tag_style = "xml" }-####################################################################################################-# Dry run config-####################################################################################################--[inference.dry_run]-text_gen_truncate_length = 256-nb_list_items = 3-nb_extract_pages = 4-allowed_to_fail_pipes = [- "failing_pipelines.infinite_loop_1", # Loop but only for testing purposes — matched on the namespaced pipe_ref-]-image_urls = [- "https://storage.googleapis.com/public_test_files_7fa6_4277_9ab/fashion/fashion_photo_1.jpg",- "https://storage.googleapis.com/public_test_files_7fa6_4277_9ab/fashion/fashion_photo_2.png",-]--####################################################################################################-# Interpreter — library-scoped method machinery (what interpreter_hub brokers)-####################################################################################################--[interpreter]--####################################################################################################-# MTHDS config-####################################################################################################--[interpreter.mthds.inline_tables]-spaces_inside_curly_braces = true--[interpreter.mthds.strings]-prefer_literal = false-force_multiline = false-length_limit_to_multiline = 100-ensure_trailing_newline = true-ensure_leading_blank_line = true--[interpreter.mthds.concepts]-structure_field_ordering = [- "type",- "concept_ref",- "item_type",- "item_concept_ref",- "description",- "choices",- "required",-]--[interpreter.mthds.pipes]-field_ordering = ["type", "description", "inputs", "output"]-##################################################################################################### Pipe run config####################################################################################################-[interpreter.pipe_run]+[pipelex.pipe_run_config]pipe_stack_limit = 20##################################################################################################### PipeFunc config####################################################################################################-[interpreter.pipe_func]+[pipelex.pipe_func_config]# PipeFunc execution mode, selected from the PipeFuncExecutorRegistry the plugins populate. "direct"# (core) imports and runs customer functions in this process; "daytona" (our Daytona sandbox plugin)# runs them in a remote box. Any non-"direct" mode carries customer PipeFunc code as source text and@@ -343,7 +294,7 @@ execution_mode = "direct"# Pipeline execution config####################################################################################################-[interpreter.pipeline_execution]+[pipelex.pipeline_execution_config]is_normalize_data_urls_to_storage = trueis_mock_inputs = falseis_generate_graph = true@@ -354,32 +305,32 @@ is_generate_usage = true# asyncio, memory, and provider rate limits. Set to "unbounded" to disable the bound.max_concurrency = 8-[interpreter.pipeline_execution.graph]+[pipelex.pipeline_execution_config.graph_config]-[interpreter.pipeline_execution.graph.data_inclusion]+[pipelex.pipeline_execution_config.graph_config.data_inclusion]stuff_json_content = truestuff_text_content = truestuff_html_content = trueerror_stack_traces = truepipe_and_concept_registry = true-[interpreter.pipeline_execution.graph.graphs_inclusion]+[pipelex.pipeline_execution_config.graph_config.graphs_inclusion]graphspec_json = truemermaidflow_mmd = truemermaidflow_html = truereactflow_html = true-[interpreter.pipeline_execution.graph.mermaid]+[pipelex.pipeline_execution_config.graph_config.mermaid_config]direction = "left_to_right"is_include_data_edges = trueis_include_contains_edges = falseis_include_selected_outcome_edges = trueis_show_stuff_codes = false-[interpreter.pipeline_execution.graph.mermaid.style]+[pipelex.pipeline_execution_config.graph_config.mermaid_config.style]theme = "dark"-[interpreter.pipeline_execution.graph.reactflow]+[pipelex.pipeline_execution_config.graph_config.reactflow_config]is_use_cdn = truelayout_direction = "left_to_right"nodesep = 50@@ -393,40 +344,109 @@ default_title = "Pipelex Graph"# when false, they collapse out of the graph and edges connect stuff nodes directly.show_batch_controller = true-[interpreter.pipeline_execution.graph.reactflow.style]+[pipelex.pipeline_execution_config.graph_config.reactflow_config.style]theme = "dark"palette = "dracula"-[interpreter.scan]-excluded_dirs = [- ".venv",- "venv",- "env",- ".env",- "virtualenv",- ".virtualenv",- ".git",- "__pycache__",- ".pytest_cache",- ".mypy_cache",- ".ruff_cache",- "node_modules",- "results",+####################################################################################################+# Tracing config (event log for distributed tracing across workers)+####################################################################################################++[pipelex.tracing_config]+is_enabled = true+backend = "ndjson"++[pipelex.tracing_config.ndjson]+traces_dir = ".pipelex/traces"++[pipelex.tracing_config.dynamodb]+table_name = ""+region = ""++####################################################################################################+# Dry run config+####################################################################################################++[pipelex.dry_run_config]+text_gen_truncate_length = 256+nb_list_items = 3+nb_extract_pages = 4+allowed_to_fail_pipes = [+ "failing_pipelines.infinite_loop_1", # Loop but only for testing purposes — matched on the namespaced pipe_ref+]+image_urls = [+ "https://storage.googleapis.com/public_test_files_7fa6_4277_9ab/fashion/fashion_photo_1.jpg",+ "https://storage.googleapis.com/public_test_files_7fa6_4277_9ab/fashion/fashion_photo_2.png",]-[interpreter.builder]-fix_loop_max_attempts = 10-default_output_dir = "."-default_bundle_file_name = "bundle"-default_directory_base_name = "pipeline"+####################################################################################################+# MTHDS config+####################################################################################################++[pipelex.mthds_config.inline_tables]+spaces_inside_curly_braces = true++[pipelex.mthds_config.strings]+prefer_literal = false+force_multiline = false+length_limit_to_multiline = 100+ensure_trailing_newline = true+ensure_leading_blank_line = true++[pipelex.mthds_config.concepts]+structure_field_ordering = [+ "type",+ "concept_ref",+ "item_type",+ "item_concept_ref",+ "description",+ "choices",+ "required",+]++[pipelex.mthds_config.pipes]+field_ordering = ["type", "description", "inputs", "output"]++####################################################################################################+# Migration config+####################################################################################################++[migration.migration_maps.config]+definition = "description"+ocr_config = "extract_config"+ocr = "extract"+img_gen = "model"+llm_handle = "model"+llm = "model"+llm_to_structure = "model_to_structure"+task_queue = "default_task_queue"++[migration.migration_maps.mthds]+img_gen = "model"+ocr = "model"+llm_handle = "model"+llm = "model"+llm_to_structure = "model_to_structure"+llm_skill = "llm_talent"+img_gen_skill = "img_gen_talent"+extract_skill = "extract_talent"+parallels = "branches"+##################################################################################################### Kit Config####################################################################################################-[kit]+[pipelex.kit_config]# Agent rule files to install (used by `pipelex-dev kit rules`)# Each entry is one of: "cursor", "agents", "claude"# - "cursor" installs .cursor/rules/*.mdc files (cannot be combined with others)# - "agents" / "claude" install AGENTS.md / CLAUDE.md respectively — both can be listed togetherpreferred_agent_targets = ["claude", "agents"]++[migration.migration_maps.telemetry]++[plugins]+# Names of discovered plugins to skip at startup (a denylist). Discovery is the+# source of truth for which plugins are present; this only turns one off.+disabled = []diff --git a/pipelex/migration/goldens/pipelex-config/[email protected] b/pipelex/migration/goldens/pipelex-config/[email protected]new file mode 100644index 000000000..ce9fd8469--- /dev/null+++ b/pipelex/migration/goldens/pipelex-config/[email protected]@@ -0,0 +1,432 @@+####################################################################################################+# Runtime — kernel-layer, process-scoped infrastructure (what runtime_hub brokers)+####################################################################################################++[runtime]++[runtime.storage]+is_fetch_remote_content_enabled = true+is_upload_local_content_enabled = true+method = "local" # local, in_memory, s3, gcp++[runtime.storage.local]+uri_format = "{primary_id}/{secondary_id}/{hash}.{extension}"+local_storage_path = ".pipelex/storage"++[runtime.storage.in_memory]+uri_format = "{primary_id}/{secondary_id}/{hash}.{extension}"++[runtime.storage.s3]+uri_format = "{primary_id}/{secondary_id}/{hash}.{extension}"+bucket_name = ""+region = ""+signed_urls_lifespan_seconds = 3600++[runtime.storage.gcp]+uri_format = "{primary_id}/{secondary_id}/{hash}.{extension}"+bucket_name = ""+project_id = ""+signed_urls_lifespan_seconds = 3600++[runtime.secrets]+method = "env" # env (built-in); an external plugin may add e.g. vault++####################################################################################################+# Log config+####################################################################################################++[runtime.log]+default_log_level = "INFO"+# console_log_target controls the RichHandler used by the logging system (log.debug/info/...).+# Logs are diagnostics → stderr.+console_log_target = "stderr"+# console_print_target controls the Console returned by get_console(), used for banners,+# deck notices, and the main `pipelex` CLI's data tables (`show backends`, `show models`,+# `which`, `doctor`). Tables are data the user pipes to a file → stdout.+# Note: the agent CLI (`pipelex-agent`) emits its actual JSON/markdown results via bare+# builtin `print(...)` to sys.stdout, bypassing Rich entirely — so this knob does NOT+# affect the agent CLI data channel. The agent CLI factory additionally forces both+# targets to stderr internally to keep its stdout strictly reserved for the success+# envelope (see `_AGENT_CLI_STDERR_CONSOLE_OVERRIDES` in `agent_cli_factory.py`).+console_print_target = "stdout"+is_console_logging_enabled = true++json_logs_indent = 4+presentation_line_width = 120+silenced_problem_ids = ["azure_openai_no_stream_options"]+caller_info_template = "file_line"+is_caller_info_enabled = false++log_mode = "rich"++poor_loggers = []+generic_poor_logger = "#poor-log"++[runtime.log.package_log_levels]+# for logger names, the dots "." have been replaced by "-" to avoid toml parsing issues+# TODO: use quotes to re-enable names with dots+anthropic = "INFO"+asyncio = "INFO"+botocore = "INFO"+botocore-credentials = "WARNING"+google = "INFO"+httpx = "WARNING"+httpcore = "INFO"+openai = "INFO"+instructor = "INFO"+urllib3-connectionpool = "INFO"+urllib3-util-retry = "INFO"++pipelex = "INFO"+++[runtime.log.rich_log]+is_show_time = false+is_show_level = true+is_link_path_enabled = true+is_markup_enabled = true+highlighter_name = "json"+is_rich_tracebacks = true+is_tracebacks_word_wrap = true+is_tracebacks_show_locals = false+tracebacks_suppress = []+keywords_to_hilight = []++[runtime.aws]+api_key_method = "env"+bedrock_access_variant = "aws_access"++[runtime.reporting]+is_log_costs_to_console = true+is_generate_cost_report_file_enabled = false+cost_report_dir_path = "reports"+cost_report_base_name = "cost_report"+cost_report_extension = "csv"+cost_report_unit_scale = 1.0++####################################################################################################+# Tracing config (event log for distributed tracing across workers)+####################################################################################################++[runtime.tracing]+is_enabled = true+backend = "ndjson"++[runtime.tracing.ndjson]+traces_dir = ".pipelex/traces"++[runtime.tracing.dynamodb]+table_name = ""+region = ""++[runtime.observer]+observer_dir = "results/observer"++[runtime.plugins]+# Names of discovered plugins to skip at startup (a denylist). Discovery is the+# source of truth for which plugins are present; this only turns one off.+disabled = []++####################################################################################################+# Inference — the model-calling seam (was [cogt])+####################################################################################################++[inference]+transport_max_retries = 2++[inference.model_deck]+is_model_fallback_enabled = true+missing_presets_reaction = "log"++[inference.llm]+default_max_images = 100+is_structure_prompt_enabled = true+is_dump_text_prompts_enabled = false+is_dump_response_text_enabled = false+schema_reask_max_attempts = 3++[inference.llm.instructor]+is_dump_kwargs_enabled = false+is_dump_response_enabled = false+is_dump_error_enabled = false++[inference.llm.openai.effort_to_level_map]+none = "none"+minimal = "minimal"+low = "low"+medium = "medium"+high = "high"+xhigh = "xhigh"+max = "xhigh"++[inference.llm.anthropic]+structured_output_timeout_seconds = 1200++[inference.llm.anthropic.effort_to_level_map]+none = "disabled"+minimal = "low"+low = "low"+medium = "medium"+high = "high"+xhigh = "xhigh"+max = "max"++[inference.llm.google.effort_to_level_map]+none = "disabled"+minimal = "minimal"+low = "low"+medium = "medium"+high = "high"+xhigh = "high"+max = "high"++[inference.llm.mistral.effort_to_level_map]+none = "disabled"+minimal = "reasoning"+low = "reasoning"+medium = "reasoning"+high = "reasoning"+xhigh = "reasoning"+max = "reasoning"++[inference.llm.effort_to_budget_maps.anthropic]+none = 0 # Required by validator; unreachable at runtime (level map gates NONE as disabled before budget lookup)+minimal = 512+low = 1024+medium = 5000+high = 16384+xhigh = 32768+max = 65536++[inference.llm.effort_to_budget_maps.gemini]+none = 0 # Required by validator; unreachable at runtime (level map gates NONE as disabled before budget lookup)+minimal = 512+low = 1024+medium = 5000+high = 16384+xhigh = 32768+max = 65536++[inference.llm.generic_templates]+output_structure_prompt = """+++---+The instance we want to generate will be for the following class:+{{ class_structure_str }}++DO NOT create information.+If some information is not present for an attribute, output the default value or None according to the attribute definition.+"""++structuring_prompt = """+Read the following text carefully and produce the requested structured output from it.++---+{{ text }}+"""++####################################################################################################+# Image generation config+####################################################################################################++[inference.img_gen.img_gen_job]+is_sync_mode = false++[inference.img_gen.img_gen_param_defaults]+nb_steps = 28 # 28 is a good default for Flux, possible values are [1,2,4,8] for SDXL Lightning+quality = "low"+aspect_ratio = "square" # "square", "landscape_4_3", "landscape_3_2", "landscape_16_9", "landscape_21_9", "landscape_4_1", "landscape_8_1", "portrait_3_4", "portrait_2_3", "portrait_9_16", "portrait_9_21", "portrait_1_4", "portrait_1_8"+guidance_scale = 2.5+background = "auto"+is_moderated = true+safety_tolerance = 5+is_raw = false+seed = "auto"++[inference.img_gen.quality_to_steps_maps]+flux = { "low" = 14, "medium" = 28, "high" = 56 }+sdxl_lightning = { "low" = 2, "medium" = 4, "high" = 8 }+qwen_image = { "low" = 10, "medium" = 20, "high" = 28 }++####################################################################################################+# Extract config+####################################################################################################++[inference.extract]+default_page_views_dpi = 72++####################################################################################################+# Gateway test config+####################################################################################################++[inference.gateway_test.config_id_substitutions]+# this is an empty dict, it's only used by Pipelex staff to override the ¨Pipelex Gateway¨ config+# in order to test new model integrations or debug issues with the Gateway++####################################################################################################+# Templating config+####################################################################################################++[inference.templating]+default_templating_style = { tag_style = "xml" }++####################################################################################################+# Dry run config+####################################################################################################++[inference.dry_run]+text_gen_truncate_length = 256+nb_list_items = 3+nb_extract_pages = 4+allowed_to_fail_pipes = [+ "failing_pipelines.infinite_loop_1", # Loop but only for testing purposes — matched on the namespaced pipe_ref+]+image_urls = [+ "https://storage.googleapis.com/public_test_files_7fa6_4277_9ab/fashion/fashion_photo_1.jpg",+ "https://storage.googleapis.com/public_test_files_7fa6_4277_9ab/fashion/fashion_photo_2.png",+]++####################################################################################################+# Interpreter — library-scoped method machinery (what interpreter_hub brokers)+####################################################################################################++[interpreter]++####################################################################################################+# MTHDS config+####################################################################################################++[interpreter.mthds.inline_tables]+spaces_inside_curly_braces = true++[interpreter.mthds.strings]+prefer_literal = false+force_multiline = false+length_limit_to_multiline = 100+ensure_trailing_newline = true+ensure_leading_blank_line = true++[interpreter.mthds.concepts]+structure_field_ordering = [+ "type",+ "concept_ref",+ "item_type",+ "item_concept_ref",+ "description",+ "choices",+ "required",+]++[interpreter.mthds.pipes]+field_ordering = ["type", "description", "inputs", "output"]++####################################################################################################+# Pipe run config+####################################################################################################++[interpreter.pipe_run]+pipe_stack_limit = 20++####################################################################################################+# PipeFunc config+####################################################################################################++[interpreter.pipe_func]+# PipeFunc execution mode, selected from the PipeFuncExecutorRegistry the plugins populate. "direct"+# (core) imports and runs customer functions in this process; "daytona" (our Daytona sandbox plugin)+# runs them in a remote box. Any non-"direct" mode carries customer PipeFunc code as source text and+# executes it only inside an isolated sandbox, never imported here.+execution_mode = "direct"++####################################################################################################+# Pipeline execution config+####################################################################################################++[interpreter.pipeline_execution]+is_normalize_data_urls_to_storage = true+is_mock_inputs = false+is_generate_graph = true+is_generate_usage = true++# Bounded fan-out concurrency for PipeBatch: at most this many branches run at once.+# A modest default keeps a large workload (one pipe over many items) from overwhelming+# asyncio, memory, and provider rate limits. Set to "unbounded" to disable the bound.+max_concurrency = 8++[interpreter.pipeline_execution.graph]++[interpreter.pipeline_execution.graph.data_inclusion]+stuff_json_content = true+stuff_text_content = true+stuff_html_content = true+error_stack_traces = true+pipe_and_concept_registry = true++[interpreter.pipeline_execution.graph.graphs_inclusion]+graphspec_json = true+mermaidflow_mmd = true+mermaidflow_html = true+reactflow_html = true++[interpreter.pipeline_execution.graph.mermaid]+direction = "left_to_right"+is_include_data_edges = true+is_include_contains_edges = false+is_include_selected_outcome_edges = true+is_show_stuff_codes = false++[interpreter.pipeline_execution.graph.mermaid.style]+theme = "dark"++[interpreter.pipeline_execution.graph.reactflow]+is_use_cdn = true+layout_direction = "left_to_right"+nodesep = 50+ranksep = 30+edge_type = "bezier"+initial_zoom = 1.0+pan_to_top = true+default_title = "Pipelex Graph"+# Forwarded to mthds-ui as `showControllers`: when true, controller pipes (PipeSequence,+# PipeParallel, PipeCondition, PipeBatch) render as grouping containers around their children;+# when false, they collapse out of the graph and edges connect stuff nodes directly.+show_batch_controller = true++[interpreter.pipeline_execution.graph.reactflow.style]+theme = "dark"+palette = "dracula"++[interpreter.scan]+excluded_dirs = [+ ".venv",+ "venv",+ "env",+ ".env",+ "virtualenv",+ ".virtualenv",+ ".git",+ "__pycache__",+ ".pytest_cache",+ ".mypy_cache",+ ".ruff_cache",+ "node_modules",+ "results",+]++[interpreter.builder]+fix_loop_max_attempts = 10+default_output_dir = "."+default_bundle_file_name = "bundle"+default_directory_base_name = "pipeline"++####################################################################################################+# Kit Config+####################################################################################################++[kit]+# Agent rule files to install (used by `pipelex-dev kit rules`)+# Each entry is one of: "cursor", "agents", "claude"+# - "cursor" installs .cursor/rules/*.mdc files (cannot be combined with others)+# - "agents" / "claude" install AGENTS.md / CLAUDE.md respectively — both can be listed together+preferred_agent_targets = ["claude", "agents"]diff --git a/pipelex/migration/goldens/pipelex-config/[email protected] b/pipelex/migration/goldens/pipelex-config/[email protected]index 79eb465d6..ae899dc62 100644--- a/pipelex/migration/goldens/pipelex-config/[email protected]+++ b/pipelex/migration/goldens/pipelex-config/[email protected]@@ -2,100 +2,60 @@"surface_id": "pipelex-config","schema_version": 2,"paths": {- "inference": {+ "cogt": {"value_type": "table","required": true,"open_node": false},- "inference.dry_run": {+ "cogt.extract_config": {"value_type": "table","required": true,"open_node": false},- "inference.dry_run.allowed_to_fail_pipes": {- "value_type": "list[str]",- "required": false,- "default": [- "failing_pipelines.infinite_loop_1"- ],- "open_node": false- },- "inference.dry_run.image_urls": {- "value_type": "list[str]",- "required": true,- "default": [- "https://storage.googleapis.com/public_test_files_7fa6_4277_9ab/fashion/fashion_photo_1.jpg",- "https://storage.googleapis.com/public_test_files_7fa6_4277_9ab/fashion/fashion_photo_2.png"- ],- "open_node": false- },- "inference.dry_run.nb_extract_pages": {- "value_type": "int",- "required": true,- "default": 4,- "open_node": false- },- "inference.dry_run.nb_list_items": {- "value_type": "int",- "required": true,- "default": 3,- "open_node": false- },- "inference.dry_run.text_gen_truncate_length": {- "value_type": "int",- "required": true,- "default": 256,- "open_node": false- },- "inference.extract": {- "value_type": "table",- "required": true,- "open_node": false- },- "inference.extract.default_page_views_dpi": {+ "cogt.extract_config.default_page_views_dpi": {"value_type": "int","required": true,"default": 72,"open_node": false},- "inference.gateway_test": {+ "cogt.gateway_test_config": {"value_type": "table","required": true,"open_node": false},- "inference.gateway_test.config_id_substitutions": {+ "cogt.gateway_test_config.config_id_substitutions": {"value_type": "dict[str, str]","required": true,"default": {},"open_node": true},- "inference.gateway_test.config_id_substitutions.*": {+ "cogt.gateway_test_config.config_id_substitutions.*": {"value_type": "str","required": true,"open_node": false},- "inference.img_gen": {+ "cogt.img_gen_config": {"value_type": "table","required": true,"open_node": false},- "inference.img_gen.img_gen_job": {+ "cogt.img_gen_config.img_gen_job_config": {"value_type": "table","required": true,"open_node": false},- "inference.img_gen.img_gen_job.is_sync_mode": {+ "cogt.img_gen_config.img_gen_job_config.is_sync_mode": {"value_type": "bool","required": true,"default": false,"open_node": false},- "inference.img_gen.img_gen_param_defaults": {+ "cogt.img_gen_config.img_gen_param_defaults": {"value_type": "table","required": true,"open_node": false},- "inference.img_gen.img_gen_param_defaults.aspect_ratio": {+ "cogt.img_gen_config.img_gen_param_defaults.aspect_ratio": {"value_type": "enum","required": true,"enum_members": [@@ -116,7 +76,7 @@"default": "square","open_node": false},- "inference.img_gen.img_gen_param_defaults.background": {+ "cogt.img_gen_config.img_gen_param_defaults.background": {"value_type": "enum","required": true,"enum_members": [@@ -127,7 +87,7 @@"default": "auto","open_node": false},- "inference.img_gen.img_gen_param_defaults.guidance_scale": {+ "cogt.img_gen_config.img_gen_param_defaults.guidance_scale": {"value_type": "float","required": true,"default": 2.5,@@ -136,19 +96,19 @@"gt": 0}},- "inference.img_gen.img_gen_param_defaults.is_moderated": {+ "cogt.img_gen_config.img_gen_param_defaults.is_moderated": {"value_type": "bool","required": false,"default": true,"open_node": false},- "inference.img_gen.img_gen_param_defaults.is_raw": {+ "cogt.img_gen_config.img_gen_param_defaults.is_raw": {"value_type": "bool","required": false,"default": false,"open_node": false},- "inference.img_gen.img_gen_param_defaults.nb_steps": {+ "cogt.img_gen_config.img_gen_param_defaults.nb_steps": {"value_type": "int","required": false,"default": 28,@@ -157,7 +117,7 @@"gt": 0}},- "inference.img_gen.img_gen_param_defaults.quality": {+ "cogt.img_gen_config.img_gen_param_defaults.quality": {"value_type": "enum","required": false,"enum_members": [@@ -168,7 +128,7 @@"default": "low","open_node": false},- "inference.img_gen.img_gen_param_defaults.safety_tolerance": {+ "cogt.img_gen_config.img_gen_param_defaults.safety_tolerance": {"value_type": "int","required": true,"default": 5,@@ -178,7 +138,7 @@"le": 6}},- "inference.img_gen.img_gen_param_defaults.seed": {+ "cogt.img_gen_config.img_gen_param_defaults.seed": {"value_type": "int | literal","required": true,"enum_members": [@@ -187,7 +147,7 @@"default": "auto","open_node": false},- "inference.img_gen.img_gen_param_defaults.size": {+ "cogt.img_gen_config.img_gen_param_defaults.size": {"value_type": "enum | table","required": false,"enum_members": [@@ -198,7 +158,7 @@],"open_node": false},- "inference.img_gen.quality_to_steps_maps": {+ "cogt.img_gen_config.quality_to_steps_maps": {"value_type": "dict[str, dict[str, int]]","required": true,"default": {@@ -220,27 +180,27 @@},"open_node": true},- "inference.img_gen.quality_to_steps_maps.*": {+ "cogt.img_gen_config.quality_to_steps_maps.*": {"value_type": "dict[str, int]","required": true,"open_node": true},- "inference.img_gen.quality_to_steps_maps.*.*": {+ "cogt.img_gen_config.quality_to_steps_maps.*.*": {"value_type": "int","required": true,"open_node": false},- "inference.llm": {+ "cogt.llm_config": {"value_type": "table","required": true,"open_node": false},- "inference.llm.anthropic": {+ "cogt.llm_config.anthropic_config": {"value_type": "table","required": true,"open_node": false},- "inference.llm.anthropic.effort_to_level_map": {+ "cogt.llm_config.anthropic_config.effort_to_level_map": {"value_type": "dict[str, str]","required": true,"default": {@@ -254,24 +214,24 @@},"open_node": true},- "inference.llm.anthropic.effort_to_level_map.*": {+ "cogt.llm_config.anthropic_config.effort_to_level_map.*": {"value_type": "str","required": true,"open_node": false},- "inference.llm.anthropic.structured_output_timeout_seconds": {+ "cogt.llm_config.anthropic_config.structured_output_timeout_seconds": {"value_type": "int","required": true,"default": 1200,"open_node": false},- "inference.llm.default_max_images": {+ "cogt.llm_config.default_max_images": {"value_type": "int","required": true,"default": 100,"open_node": false},- "inference.llm.effort_to_budget_maps": {+ "cogt.llm_config.effort_to_budget_maps": {"value_type": "dict[str, dict[str, int]]","required": true,"default": {@@ -296,17 +256,17 @@},"open_node": true},- "inference.llm.effort_to_budget_maps.*": {+ "cogt.llm_config.effort_to_budget_maps.*": {"value_type": "dict[str, int]","required": true,"open_node": true},- "inference.llm.effort_to_budget_maps.*.*": {+ "cogt.llm_config.effort_to_budget_maps.*.*": {"value_type": "int","required": true,"open_node": false},- "inference.llm.generic_templates": {+ "cogt.llm_config.generic_templates": {"value_type": "dict[str, str]","required": true,"default": {@@ -315,17 +275,17 @@},"open_node": true},- "inference.llm.generic_templates.*": {+ "cogt.llm_config.generic_templates.*": {"value_type": "str","required": true,"open_node": false},- "inference.llm.google": {+ "cogt.llm_config.google_config": {"value_type": "table","required": true,"open_node": false},- "inference.llm.google.effort_to_level_map": {+ "cogt.llm_config.google_config.effort_to_level_map": {"value_type": "dict[str, str]","required": true,"default": {@@ -339,58 +299,58 @@},"open_node": true},- "inference.llm.google.effort_to_level_map.*": {+ "cogt.llm_config.google_config.effort_to_level_map.*": {"value_type": "str","required": true,"open_node": false},- "inference.llm.instructor": {+ "cogt.llm_config.instructor_config": {"value_type": "table","required": true,"open_node": false},- "inference.llm.instructor.is_dump_error_enabled": {+ "cogt.llm_config.instructor_config.is_dump_error_enabled": {"value_type": "bool","required": true,"default": false,"open_node": false},- "inference.llm.instructor.is_dump_kwargs_enabled": {+ "cogt.llm_config.instructor_config.is_dump_kwargs_enabled": {"value_type": "bool","required": true,"default": false,"open_node": false},- "inference.llm.instructor.is_dump_response_enabled": {+ "cogt.llm_config.instructor_config.is_dump_response_enabled": {"value_type": "bool","required": true,"default": false,"open_node": false},- "inference.llm.is_dump_response_text_enabled": {+ "cogt.llm_config.is_dump_response_text_enabled": {"value_type": "bool","required": true,"default": false,"open_node": false},- "inference.llm.is_dump_text_prompts_enabled": {+ "cogt.llm_config.is_dump_text_prompts_enabled": {"value_type": "bool","required": true,"default": false,"open_node": false},- "inference.llm.is_structure_prompt_enabled": {+ "cogt.llm_config.is_structure_prompt_enabled": {"value_type": "bool","required": true,"default": true,"open_node": false},- "inference.llm.mistral": {+ "cogt.llm_config.mistral_config": {"value_type": "table","required": true,"open_node": false},- "inference.llm.mistral.effort_to_level_map": {+ "cogt.llm_config.mistral_config.effort_to_level_map": {"value_type": "dict[str, str]","required": true,"default": {@@ -404,17 +364,17 @@},"open_node": true},- "inference.llm.mistral.effort_to_level_map.*": {+ "cogt.llm_config.mistral_config.effort_to_level_map.*": {"value_type": "str","required": true,"open_node": false},- "inference.llm.openai": {+ "cogt.llm_config.openai_config": {"value_type": "table","required": true,"open_node": false},- "inference.llm.openai.effort_to_level_map": {+ "cogt.llm_config.openai_config.effort_to_level_map": {"value_type": "dict[str, str]","required": true,"default": {@@ -428,12 +388,12 @@},"open_node": true},- "inference.llm.openai.effort_to_level_map.*": {+ "cogt.llm_config.openai_config.effort_to_level_map.*": {"value_type": "str","required": true,"open_node": false},- "inference.llm.schema_reask_max_attempts": {+ "cogt.llm_config.schema_reask_max_attempts": {"value_type": "int","required": true,"default": 3,@@ -443,18 +403,18 @@"le": 10}},- "inference.model_deck": {+ "cogt.model_deck_config": {"value_type": "table","required": true,"open_node": false},- "inference.model_deck.is_model_fallback_enabled": {+ "cogt.model_deck_config.is_model_fallback_enabled": {"value_type": "bool","required": true,"default": true,"open_node": false},- "inference.model_deck.missing_presets_reaction": {+ "cogt.model_deck_config.missing_presets_reaction": {"value_type": "enum","required": true,"enum_members": [@@ -465,788 +425,798 @@"default": "log","open_node": false},- "inference.templating": {+ "cogt.transport_max_retries": {+ "value_type": "int",+ "required": true,+ "default": 2,+ "open_node": false,+ "constraints": {+ "ge": 0,+ "le": 10+ }+ },+ "migration": {"value_type": "table","required": true,"open_node": false},- "inference.templating.default_templating_style": {+ "migration.migration_maps": {+ "value_type": "dict[str, dict[str, str]]",+ "required": true,+ "default": {+ "config": {+ "definition": "description",+ "ocr_config": "extract_config",+ "ocr": "extract",+ "img_gen": "model",+ "llm_handle": "model",+ "llm": "model",+ "llm_to_structure": "model_to_structure",+ "task_queue": "default_task_queue"+ },+ "mthds": {+ "img_gen": "model",+ "ocr": "model",+ "llm_handle": "model",+ "llm": "model",+ "llm_to_structure": "model_to_structure",+ "llm_skill": "llm_talent",+ "img_gen_skill": "img_gen_talent",+ "extract_skill": "extract_talent",+ "parallels": "branches"+ },+ "telemetry": {}+ },+ "open_node": true+ },+ "migration.migration_maps.*": {+ "value_type": "dict[str, str]",+ "required": true,+ "open_node": true+ },+ "migration.migration_maps.*.*": {+ "value_type": "str",+ "required": true,+ "open_node": false+ },+ "pipelex": {"value_type": "table","required": true,"open_node": false},- "inference.templating.default_templating_style.tag_style": {- "value_type": "enum",+ "pipelex.aws_config": {+ "value_type": "table","required": true,- "enum_members": [- "no_tag",- "square_brackets",- "ticks",- "xml"- ],- "default": "xml","open_node": false},- "inference.templating.default_templating_style.text_format": {+ "pipelex.aws_config.api_key_method": {"value_type": "enum",- "required": false,+ "required": true,"enum_members": [- "html",- "json",- "markdown",- "plain"+ "env",+ "secret_provider"],+ "default": "env","open_node": false},- "inference.transport_max_retries": {- "value_type": "int",- "required": true,- "default": 2,- "open_node": false,- "constraints": {- "ge": 0,- "le": 10- }- },- "interpreter": {- "value_type": "table",+ "pipelex.aws_config.bedrock_access_variant": {+ "value_type": "enum","required": true,+ "enum_members": [+ "aws_access",+ "bedrock_token"+ ],+ "default": "aws_access","open_node": false},- "interpreter.builder": {+ "pipelex.builder_config": {"value_type": "table","required": true,"open_node": false},- "interpreter.builder.default_bundle_file_name": {+ "pipelex.builder_config.default_bundle_file_name": {"value_type": "str","required": true,"default": "bundle","open_node": false},- "interpreter.builder.default_directory_base_name": {+ "pipelex.builder_config.default_directory_base_name": {"value_type": "str","required": true,"default": "pipeline","open_node": false},- "interpreter.builder.default_output_dir": {+ "pipelex.builder_config.default_output_dir": {"value_type": "str","required": true,"default": ".","open_node": false},- "interpreter.builder.fix_loop_max_attempts": {+ "pipelex.builder_config.fix_loop_max_attempts": {"value_type": "int","required": true,"default": 10,"open_node": false},- "interpreter.mthds": {+ "pipelex.dry_run_config": {"value_type": "table","required": true,"open_node": false},- "interpreter.mthds.concepts": {- "value_type": "table",- "required": true,+ "pipelex.dry_run_config.allowed_to_fail_pipes": {+ "value_type": "list[str]",+ "required": false,+ "default": [+ "failing_pipelines.infinite_loop_1"+ ],"open_node": false},- "interpreter.mthds.concepts.structure_field_ordering": {+ "pipelex.dry_run_config.image_urls": {"value_type": "list[str]","required": true,"default": [- "type",- "concept_ref",- "item_type",- "item_concept_ref",- "description",- "choices",- "required"+ "https://storage.googleapis.com/public_test_files_7fa6_4277_9ab/fashion/fashion_photo_1.jpg",+ "https://storage.googleapis.com/public_test_files_7fa6_4277_9ab/fashion/fashion_photo_2.png"],"open_node": false},- "interpreter.mthds.inline_tables": {- "value_type": "table",+ "pipelex.dry_run_config.nb_extract_pages": {+ "value_type": "int","required": true,+ "default": 4,"open_node": false},- "interpreter.mthds.inline_tables.spaces_inside_curly_braces": {- "value_type": "bool",+ "pipelex.dry_run_config.nb_list_items": {+ "value_type": "int","required": true,- "default": true,+ "default": 3,+ "open_node": false+ },+ "pipelex.dry_run_config.text_gen_truncate_length": {+ "value_type": "int",+ "required": true,+ "default": 256,"open_node": false},- "interpreter.mthds.pipes": {+ "pipelex.kit_config": {"value_type": "table","required": true,"open_node": false},- "interpreter.mthds.pipes.field_ordering": {- "value_type": "list[str]",+ "pipelex.kit_config.preferred_agent_targets": {+ "value_type": "list[enum]","required": true,+ "enum_members": [+ "agents",+ "claude",+ "cursor"+ ],"default": [- "type",- "description",- "inputs",- "output"+ "claude",+ "agents"],"open_node": false},- "interpreter.mthds.strings": {+ "pipelex.log_config": {"value_type": "table","required": true,"open_node": false},- "interpreter.mthds.strings.ensure_leading_blank_line": {- "value_type": "bool",+ "pipelex.log_config.caller_info_template": {+ "value_type": "enum","required": true,- "default": true,+ "enum_members": [+ "file_func",+ "file_line",+ "file_line_func",+ "func",+ "func_line",+ "func_module",+ "func_module_line"+ ],+ "default": "file_line","open_node": false},- "interpreter.mthds.strings.ensure_trailing_newline": {- "value_type": "bool",+ "pipelex.log_config.console_log_target": {+ "value_type": "enum","required": true,- "default": true,+ "enum_members": [+ "file",+ "stderr",+ "stdout"+ ],+ "default": "stderr","open_node": false},- "interpreter.mthds.strings.force_multiline": {- "value_type": "bool",+ "pipelex.log_config.console_print_target": {+ "value_type": "enum","required": true,- "default": false,+ "enum_members": [+ "file",+ "stderr",+ "stdout"+ ],+ "default": "stdout","open_node": false},- "interpreter.mthds.strings.length_limit_to_multiline": {- "value_type": "int",+ "pipelex.log_config.default_log_level": {+ "value_type": "enum","required": true,- "default": 100,+ "enum_members": [+ "CRITICAL",+ "DEBUG",+ "DEV",+ "ERROR",+ "INFO",+ "OFF",+ "VERBOSE",+ "WARNING"+ ],+ "default": "INFO",+ "open_node": false+ },+ "pipelex.log_config.generic_poor_logger": {+ "value_type": "str",+ "required": true,+ "default": "#poor-log","open_node": false},- "interpreter.mthds.strings.prefer_literal": {+ "pipelex.log_config.is_caller_info_enabled": {"value_type": "bool","required": true,"default": false,"open_node": false},- "interpreter.pipe_func": {- "value_type": "table",+ "pipelex.log_config.is_console_logging_enabled": {+ "value_type": "bool","required": true,+ "default": true,"open_node": false},- "interpreter.pipe_func.execution_mode": {- "value_type": "str",+ "pipelex.log_config.json_logs_indent": {+ "value_type": "int","required": true,- "default": "direct",+ "default": 4,"open_node": false},- "interpreter.pipe_run": {- "value_type": "table",+ "pipelex.log_config.log_mode": {+ "value_type": "enum","required": true,+ "enum_members": [+ "poor",+ "rich"+ ],+ "default": "rich","open_node": false},- "interpreter.pipe_run.pipe_stack_limit": {- "value_type": "int",+ "pipelex.log_config.package_log_levels": {+ "value_type": "dict[str, enum]","required": true,- "default": 20,+ "default": {+ "anthropic": "INFO",+ "asyncio": "INFO",+ "botocore": "INFO",+ "botocore-credentials": "WARNING",+ "google": "INFO",+ "httpx": "WARNING",+ "httpcore": "INFO",+ "openai": "INFO",+ "instructor": "INFO",+ "urllib3-connectionpool": "INFO",+ "urllib3-util-retry": "INFO",+ "pipelex": "INFO"+ },+ "open_node": true+ },+ "pipelex.log_config.package_log_levels.*": {+ "value_type": "enum",+ "required": true,+ "enum_members": [+ "CRITICAL",+ "DEBUG",+ "DEV",+ "ERROR",+ "INFO",+ "OFF",+ "VERBOSE",+ "WARNING"+ ],"open_node": false},- "interpreter.pipeline_execution": {- "value_type": "table",+ "pipelex.log_config.poor_loggers": {+ "value_type": "list[str]","required": true,+ "default": [],"open_node": false},- "interpreter.pipeline_execution.graph": {- "value_type": "table",+ "pipelex.log_config.presentation_line_width": {+ "value_type": "int","required": true,+ "default": 120,"open_node": false},- "interpreter.pipeline_execution.graph.data_inclusion": {+ "pipelex.log_config.rich_log_config": {"value_type": "table","required": true,"open_node": false},- "interpreter.pipeline_execution.graph.data_inclusion.error_stack_traces": {- "value_type": "bool",+ "pipelex.log_config.rich_log_config.highlighter_name": {+ "value_type": "enum","required": true,- "default": true,+ "enum_members": [+ "json",+ "repr"+ ],+ "default": "json","open_node": false},- "interpreter.pipeline_execution.graph.data_inclusion.pipe_and_concept_registry": {+ "pipelex.log_config.rich_log_config.is_link_path_enabled": {"value_type": "bool","required": true,"default": true,"open_node": false},- "interpreter.pipeline_execution.graph.data_inclusion.stuff_html_content": {+ "pipelex.log_config.rich_log_config.is_markup_enabled": {"value_type": "bool","required": true,"default": true,"open_node": false},- "interpreter.pipeline_execution.graph.data_inclusion.stuff_json_content": {+ "pipelex.log_config.rich_log_config.is_rich_tracebacks": {"value_type": "bool","required": true,"default": true,"open_node": false},- "interpreter.pipeline_execution.graph.data_inclusion.stuff_text_content": {+ "pipelex.log_config.rich_log_config.is_show_level": {"value_type": "bool","required": true,"default": true,"open_node": false},- "interpreter.pipeline_execution.graph.graphs_inclusion": {- "value_type": "table",+ "pipelex.log_config.rich_log_config.is_show_time": {+ "value_type": "bool","required": true,+ "default": false,"open_node": false},- "interpreter.pipeline_execution.graph.graphs_inclusion.graphspec_json": {+ "pipelex.log_config.rich_log_config.is_tracebacks_show_locals": {"value_type": "bool","required": true,- "default": true,+ "default": false,"open_node": false},- "interpreter.pipeline_execution.graph.graphs_inclusion.mermaidflow_html": {+ "pipelex.log_config.rich_log_config.is_tracebacks_word_wrap": {"value_type": "bool","required": true,"default": true,"open_node": false},- "interpreter.pipeline_execution.graph.graphs_inclusion.mermaidflow_mmd": {- "value_type": "bool",+ "pipelex.log_config.rich_log_config.keywords_to_hilight": {+ "value_type": "list[str]","required": true,- "default": true,+ "default": [],"open_node": false},- "interpreter.pipeline_execution.graph.graphs_inclusion.reactflow_html": {- "value_type": "bool",+ "pipelex.log_config.rich_log_config.tracebacks_suppress": {+ "value_type": "list[str]","required": true,- "default": true,+ "default": [],"open_node": false},- "interpreter.pipeline_execution.graph.mermaid": {- "value_type": "table",+ "pipelex.log_config.silenced_problem_ids": {+ "value_type": "list[str]","required": true,+ "default": [+ "azure_openai_no_stream_options"+ ],"open_node": false},- "interpreter.pipeline_execution.graph.mermaid.direction": {- "value_type": "enum",+ "pipelex.mthds_config": {+ "value_type": "table","required": true,- "enum_members": [- "left_to_right",- "top_down"- ],- "default": "left_to_right","open_node": false},- "interpreter.pipeline_execution.graph.mermaid.is_include_contains_edges": {- "value_type": "bool",+ "pipelex.mthds_config.concepts": {+ "value_type": "table","required": true,- "default": false,"open_node": false},- "interpreter.pipeline_execution.graph.mermaid.is_include_data_edges": {- "value_type": "bool",+ "pipelex.mthds_config.concepts.structure_field_ordering": {+ "value_type": "list[str]","required": true,- "default": true,+ "default": [+ "type",+ "concept_ref",+ "item_type",+ "item_concept_ref",+ "description",+ "choices",+ "required"+ ],"open_node": false},- "interpreter.pipeline_execution.graph.mermaid.is_include_selected_outcome_edges": {- "value_type": "bool",+ "pipelex.mthds_config.inline_tables": {+ "value_type": "table","required": true,- "default": true,"open_node": false},- "interpreter.pipeline_execution.graph.mermaid.is_show_stuff_codes": {+ "pipelex.mthds_config.inline_tables.spaces_inside_curly_braces": {"value_type": "bool","required": true,- "default": false,+ "default": true,"open_node": false},- "interpreter.pipeline_execution.graph.mermaid.style": {+ "pipelex.mthds_config.pipes": {"value_type": "table","required": true,"open_node": false},- "interpreter.pipeline_execution.graph.mermaid.style.theme": {- "value_type": "enum",+ "pipelex.mthds_config.pipes.field_ordering": {+ "value_type": "list[str]","required": true,- "enum_members": [- "base",- "dark",- "default",- "forest",- "neutral"+ "default": [+ "type",+ "description",+ "inputs",+ "output"],- "default": "dark","open_node": false},- "interpreter.pipeline_execution.graph.reactflow": {+ "pipelex.mthds_config.strings": {"value_type": "table","required": true,"open_node": false},- "interpreter.pipeline_execution.graph.reactflow.default_title": {- "value_type": "str",+ "pipelex.mthds_config.strings.ensure_leading_blank_line": {+ "value_type": "bool","required": true,- "default": "Pipelex Graph",+ "default": true,"open_node": false},- "interpreter.pipeline_execution.graph.reactflow.edge_type": {- "value_type": "enum",+ "pipelex.mthds_config.strings.ensure_trailing_newline": {+ "value_type": "bool","required": true,- "enum_members": [- "bezier",- "smoothstep",- "step",- "straight"- ],- "default": "bezier",+ "default": true,"open_node": false},- "interpreter.pipeline_execution.graph.reactflow.initial_zoom": {- "value_type": "float",+ "pipelex.mthds_config.strings.force_multiline": {+ "value_type": "bool","required": true,- "default": 1.0,+ "default": false,+ "open_node": false+ },+ "pipelex.mthds_config.strings.length_limit_to_multiline": {+ "value_type": "int",+ "required": true,+ "default": 100,"open_node": false},- "interpreter.pipeline_execution.graph.reactflow.is_use_cdn": {+ "pipelex.mthds_config.strings.prefer_literal": {"value_type": "bool","required": true,- "default": true,+ "default": false,"open_node": false},- "interpreter.pipeline_execution.graph.reactflow.layout_direction": {- "value_type": "enum",+ "pipelex.observer_config": {+ "value_type": "table","required": true,- "enum_members": [- "left_to_right",- "top_down"- ],- "default": "left_to_right","open_node": false},- "interpreter.pipeline_execution.graph.reactflow.nodesep": {- "value_type": "int",+ "pipelex.observer_config.observer_dir": {+ "value_type": "str","required": true,- "default": 50,+ "default": "results/observer","open_node": false},- "interpreter.pipeline_execution.graph.reactflow.pan_to_top": {- "value_type": "bool",+ "pipelex.pipe_func_config": {+ "value_type": "table","required": true,- "default": true,"open_node": false},- "interpreter.pipeline_execution.graph.reactflow.ranksep": {- "value_type": "int",+ "pipelex.pipe_func_config.execution_mode": {+ "value_type": "str","required": true,- "default": 30,+ "default": "direct","open_node": false},- "interpreter.pipeline_execution.graph.reactflow.show_batch_controller": {- "value_type": "bool",+ "pipelex.pipe_run_config": {+ "value_type": "table","required": true,- "default": true,"open_node": false},- "interpreter.pipeline_execution.graph.reactflow.style": {+ "pipelex.pipe_run_config.pipe_stack_limit": {+ "value_type": "int",+ "required": true,+ "default": 20,+ "open_node": false+ },+ "pipelex.pipeline_execution_config": {"value_type": "table","required": true,"open_node": false},- "interpreter.pipeline_execution.graph.reactflow.style.palette": {- "value_type": "enum",+ "pipelex.pipeline_execution_config.graph_config": {+ "value_type": "table","required": true,- "enum_members": [- "dracula",- "yellow_blue"- ],- "default": "dracula","open_node": false},- "interpreter.pipeline_execution.graph.reactflow.style.theme": {- "value_type": "enum",+ "pipelex.pipeline_execution_config.graph_config.data_inclusion": {+ "value_type": "table","required": true,- "enum_members": [- "dark",- "light",- "system"- ],- "default": "dark","open_node": false},- "interpreter.pipeline_execution.is_generate_graph": {+ "pipelex.pipeline_execution_config.graph_config.data_inclusion.error_stack_traces": {"value_type": "bool","required": true,"default": true,"open_node": false},- "interpreter.pipeline_execution.is_generate_usage": {+ "pipelex.pipeline_execution_config.graph_config.data_inclusion.pipe_and_concept_registry": {"value_type": "bool","required": true,"default": true,"open_node": false},- "interpreter.pipeline_execution.is_mock_inputs": {+ "pipelex.pipeline_execution_config.graph_config.data_inclusion.stuff_html_content": {"value_type": "bool","required": true,- "default": false,+ "default": true,"open_node": false},- "interpreter.pipeline_execution.is_normalize_data_urls_to_storage": {+ "pipelex.pipeline_execution_config.graph_config.data_inclusion.stuff_json_content": {"value_type": "bool","required": true,"default": true,"open_node": false},- "interpreter.pipeline_execution.max_concurrency": {- "value_type": "int | literal",+ "pipelex.pipeline_execution_config.graph_config.data_inclusion.stuff_text_content": {+ "value_type": "bool","required": true,- "enum_members": [- "unbounded"- ],- "default": 8,- "open_node": false,- "constraints": {- "ge": 1- }+ "default": true,+ "open_node": false},- "interpreter.scan": {+ "pipelex.pipeline_execution_config.graph_config.graphs_inclusion": {"value_type": "table","required": true,"open_node": false},- "interpreter.scan.excluded_dirs": {- "value_type": "frozenset[str]",+ "pipelex.pipeline_execution_config.graph_config.graphs_inclusion.graphspec_json": {+ "value_type": "bool","required": true,- "default": [- ".venv",- "venv",- "env",- ".env",- "virtualenv",- ".virtualenv",- ".git",- "__pycache__",- ".pytest_cache",- ".mypy_cache",- ".ruff_cache",- "node_modules",- "results"- ],+ "default": true,"open_node": false},- "kit": {- "value_type": "table",+ "pipelex.pipeline_execution_config.graph_config.graphs_inclusion.mermaidflow_html": {+ "value_type": "bool","required": true,+ "default": true,"open_node": false},- "kit.preferred_agent_targets": {- "value_type": "list[enum]",+ "pipelex.pipeline_execution_config.graph_config.graphs_inclusion.mermaidflow_mmd": {+ "value_type": "bool","required": true,- "enum_members": [- "agents",- "claude",- "cursor"- ],- "default": [- "claude",- "agents"- ],+ "default": true,"open_node": false},- "runtime": {- "value_type": "table",+ "pipelex.pipeline_execution_config.graph_config.graphs_inclusion.reactflow_html": {+ "value_type": "bool","required": true,+ "default": true,"open_node": false},- "runtime.aws": {+ "pipelex.pipeline_execution_config.graph_config.mermaid_config": {"value_type": "table","required": true,"open_node": false},- "runtime.aws.api_key_method": {+ "pipelex.pipeline_execution_config.graph_config.mermaid_config.direction": {"value_type": "enum","required": true,"enum_members": [- "env",- "secret_provider"+ "left_to_right",+ "top_down"],- "default": "env",+ "default": "left_to_right","open_node": false},- "runtime.aws.bedrock_access_variant": {- "value_type": "enum",+ "pipelex.pipeline_execution_config.graph_config.mermaid_config.is_include_contains_edges": {+ "value_type": "bool","required": true,- "enum_members": [- "aws_access",- "bedrock_token"- ],- "default": "aws_access",+ "default": false,"open_node": false},- "runtime.log": {- "value_type": "table",+ "pipelex.pipeline_execution_config.graph_config.mermaid_config.is_include_data_edges": {+ "value_type": "bool","required": true,+ "default": true,"open_node": false},- "runtime.log.caller_info_template": {- "value_type": "enum",+ "pipelex.pipeline_execution_config.graph_config.mermaid_config.is_include_selected_outcome_edges": {+ "value_type": "bool","required": true,- "enum_members": [- "file_func",- "file_line",- "file_line_func",- "func",- "func_line",- "func_module",- "func_module_line"- ],- "default": "file_line",+ "default": true,"open_node": false},- "runtime.log.console_log_target": {- "value_type": "enum",+ "pipelex.pipeline_execution_config.graph_config.mermaid_config.is_show_stuff_codes": {+ "value_type": "bool","required": true,- "enum_members": [- "file",- "stderr",- "stdout"- ],- "default": "stderr",+ "default": false,"open_node": false},- "runtime.log.console_print_target": {- "value_type": "enum",+ "pipelex.pipeline_execution_config.graph_config.mermaid_config.style": {+ "value_type": "table","required": true,- "enum_members": [- "file",- "stderr",- "stdout"- ],- "default": "stdout","open_node": false},- "runtime.log.default_log_level": {+ "pipelex.pipeline_execution_config.graph_config.mermaid_config.style.theme": {"value_type": "enum","required": true,"enum_members": [- "CRITICAL",- "DEBUG",- "DEV",- "ERROR",- "INFO",- "OFF",- "VERBOSE",- "WARNING"+ "base",+ "dark",+ "default",+ "forest",+ "neutral"],- "default": "INFO",+ "default": "dark","open_node": false},- "runtime.log.generic_poor_logger": {+ "pipelex.pipeline_execution_config.graph_config.reactflow_config": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "pipelex.pipeline_execution_config.graph_config.reactflow_config.default_title": {"value_type": "str","required": true,- "default": "#poor-log",+ "default": "Pipelex Graph","open_node": false},- "runtime.log.is_caller_info_enabled": {- "value_type": "bool",+ "pipelex.pipeline_execution_config.graph_config.reactflow_config.edge_type": {+ "value_type": "enum","required": true,- "default": false,+ "enum_members": [+ "bezier",+ "smoothstep",+ "step",+ "straight"+ ],+ "default": "bezier","open_node": false},- "runtime.log.is_console_logging_enabled": {- "value_type": "bool",+ "pipelex.pipeline_execution_config.graph_config.reactflow_config.initial_zoom": {+ "value_type": "float","required": true,- "default": true,+ "default": 1.0,"open_node": false},- "runtime.log.json_logs_indent": {- "value_type": "int",+ "pipelex.pipeline_execution_config.graph_config.reactflow_config.is_use_cdn": {+ "value_type": "bool","required": true,- "default": 4,+ "default": true,"open_node": false},- "runtime.log.log_mode": {+ "pipelex.pipeline_execution_config.graph_config.reactflow_config.layout_direction": {"value_type": "enum","required": true,"enum_members": [- "poor",- "rich"+ "left_to_right",+ "top_down"],- "default": "rich",+ "default": "left_to_right","open_node": false},- "runtime.log.package_log_levels": {- "value_type": "dict[str, enum]",+ "pipelex.pipeline_execution_config.graph_config.reactflow_config.nodesep": {+ "value_type": "int","required": true,- "default": {- "anthropic": "INFO",- "asyncio": "INFO",- "botocore": "INFO",- "botocore-credentials": "WARNING",- "google": "INFO",- "httpx": "WARNING",- "httpcore": "INFO",- "openai": "INFO",- "instructor": "INFO",- "urllib3-connectionpool": "INFO",- "urllib3-util-retry": "INFO",- "pipelex": "INFO"- },- "open_node": true+ "default": 50,+ "open_node": false},- "runtime.log.package_log_levels.*": {- "value_type": "enum",+ "pipelex.pipeline_execution_config.graph_config.reactflow_config.pan_to_top": {+ "value_type": "bool","required": true,- "enum_members": [- "CRITICAL",- "DEBUG",- "DEV",- "ERROR",- "INFO",- "OFF",- "VERBOSE",- "WARNING"- ],+ "default": true,"open_node": false},- "runtime.log.poor_loggers": {- "value_type": "list[str]",+ "pipelex.pipeline_execution_config.graph_config.reactflow_config.ranksep": {+ "value_type": "int","required": true,- "default": [],+ "default": 30,"open_node": false},- "runtime.log.presentation_line_width": {- "value_type": "int",+ "pipelex.pipeline_execution_config.graph_config.reactflow_config.show_batch_controller": {+ "value_type": "bool","required": true,- "default": 120,+ "default": true,"open_node": false},- "runtime.log.rich_log": {+ "pipelex.pipeline_execution_config.graph_config.reactflow_config.style": {"value_type": "table","required": true,"open_node": false},- "runtime.log.rich_log.highlighter_name": {+ "pipelex.pipeline_execution_config.graph_config.reactflow_config.style.palette": {"value_type": "enum","required": true,"enum_members": [- "json",- "repr"+ "dracula",+ "yellow_blue"],- "default": "json",- "open_node": false- },- "runtime.log.rich_log.is_link_path_enabled": {- "value_type": "bool",- "required": true,- "default": true,+ "default": "dracula","open_node": false},- "runtime.log.rich_log.is_markup_enabled": {- "value_type": "bool",+ "pipelex.pipeline_execution_config.graph_config.reactflow_config.style.theme": {+ "value_type": "enum","required": true,- "default": true,+ "enum_members": [+ "dark",+ "light",+ "system"+ ],+ "default": "dark","open_node": false},- "runtime.log.rich_log.is_rich_tracebacks": {+ "pipelex.pipeline_execution_config.is_generate_graph": {"value_type": "bool","required": true,"default": true,"open_node": false},- "runtime.log.rich_log.is_show_level": {+ "pipelex.pipeline_execution_config.is_generate_usage": {"value_type": "bool","required": true,"default": true,"open_node": false},- "runtime.log.rich_log.is_show_time": {+ "pipelex.pipeline_execution_config.is_mock_inputs": {"value_type": "bool","required": true,"default": false,"open_node": false},- "runtime.log.rich_log.is_tracebacks_show_locals": {- "value_type": "bool",- "required": true,- "default": false,- "open_node": false- },- "runtime.log.rich_log.is_tracebacks_word_wrap": {+ "pipelex.pipeline_execution_config.is_normalize_data_urls_to_storage": {"value_type": "bool","required": true,"default": true,"open_node": false},- "runtime.log.rich_log.keywords_to_hilight": {- "value_type": "list[str]",- "required": true,- "default": [],- "open_node": false- },- "runtime.log.rich_log.tracebacks_suppress": {- "value_type": "list[str]",- "required": true,- "default": [],- "open_node": false- },- "runtime.log.silenced_problem_ids": {- "value_type": "list[str]",+ "pipelex.pipeline_execution_config.max_concurrency": {+ "value_type": "int | literal","required": true,- "default": [- "azure_openai_no_stream_options"+ "enum_members": [+ "unbounded"],- "open_node": false- },- "runtime.observer": {- "value_type": "table",- "required": true,- "open_node": false- },- "runtime.observer.observer_dir": {- "value_type": "str",- "required": true,- "default": "results/observer",- "open_node": false- },- "runtime.plugins": {- "value_type": "table",- "required": true,- "open_node": false- },- "runtime.plugins.disabled": {- "value_type": "list[str]",- "required": true,- "default": [],- "open_node": false+ "default": 8,+ "open_node": false,+ "constraints": {+ "ge": 1+ }},- "runtime.reporting": {+ "pipelex.reporting_config": {"value_type": "table","required": true,"open_node": false},- "runtime.reporting.cost_report_base_name": {+ "pipelex.reporting_config.cost_report_base_name": {"value_type": "str","required": true,"default": "cost_report","open_node": false},- "runtime.reporting.cost_report_dir_path": {+ "pipelex.reporting_config.cost_report_dir_path": {"value_type": "str","required": true,"default": "reports","open_node": false},- "runtime.reporting.cost_report_extension": {+ "pipelex.reporting_config.cost_report_extension": {"value_type": "str","required": true,"default": "csv","open_node": false},- "runtime.reporting.cost_report_unit_scale": {+ "pipelex.reporting_config.cost_report_unit_scale": {"value_type": "float","required": true,"default": 1.0,@@ -1255,52 +1225,77 @@"gt": 0}},- "runtime.reporting.is_generate_cost_report_file_enabled": {+ "pipelex.reporting_config.is_generate_cost_report_file_enabled": {"value_type": "bool","required": true,"default": false,"open_node": false},- "runtime.reporting.is_log_costs_to_console": {+ "pipelex.reporting_config.is_log_costs_to_console": {"value_type": "bool","required": true,"default": true,"open_node": false},- "runtime.secrets": {+ "pipelex.scan_config": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "pipelex.scan_config.excluded_dirs": {+ "value_type": "frozenset[str]",+ "required": true,+ "default": [+ ".venv",+ "venv",+ "env",+ ".env",+ "virtualenv",+ ".virtualenv",+ ".git",+ "__pycache__",+ ".pytest_cache",+ ".mypy_cache",+ ".ruff_cache",+ "node_modules",+ "results"+ ],+ "open_node": false+ },+ "pipelex.secrets_config": {"value_type": "table","required": true,"open_node": false},- "runtime.secrets.method": {+ "pipelex.secrets_config.method": {"value_type": "str","required": true,"default": "env","open_node": false},- "runtime.storage": {+ "pipelex.storage_config": {"value_type": "table","required": true,"open_node": false},- "runtime.storage.gcp": {+ "pipelex.storage_config.gcp": {"value_type": "table","required": false,"open_node": false},- "runtime.storage.gcp.bucket_name": {+ "pipelex.storage_config.gcp.bucket_name": {"value_type": "str","required": true,"default": "","open_node": false},- "runtime.storage.gcp.project_id": {+ "pipelex.storage_config.gcp.project_id": {"value_type": "str","required": true,"default": "","open_node": false},- "runtime.storage.gcp.signed_urls_lifespan_seconds": {+ "pipelex.storage_config.gcp.signed_urls_lifespan_seconds": {"value_type": "int | literal","required": true,"enum_members": [@@ -1309,76 +1304,76 @@"default": 3600,"open_node": false},- "runtime.storage.gcp.uri_format": {+ "pipelex.storage_config.gcp.uri_format": {"value_type": "str","required": true,"default": "{primary_id}/{secondary_id}/{hash}.{extension}","open_node": false},- "runtime.storage.in_memory": {+ "pipelex.storage_config.in_memory": {"value_type": "table","required": false,"open_node": false},- "runtime.storage.in_memory.uri_format": {+ "pipelex.storage_config.in_memory.uri_format": {"value_type": "str","required": true,"default": "{primary_id}/{secondary_id}/{hash}.{extension}","open_node": false},- "runtime.storage.is_fetch_remote_content_enabled": {+ "pipelex.storage_config.is_fetch_remote_content_enabled": {"value_type": "bool","required": true,"default": true,"open_node": false},- "runtime.storage.is_upload_local_content_enabled": {+ "pipelex.storage_config.is_upload_local_content_enabled": {"value_type": "bool","required": true,"default": true,"open_node": false},- "runtime.storage.local": {+ "pipelex.storage_config.local": {"value_type": "table","required": false,"open_node": false},- "runtime.storage.local.local_storage_path": {+ "pipelex.storage_config.local.local_storage_path": {"value_type": "str","required": true,"default": ".pipelex/storage","open_node": false},- "runtime.storage.local.uri_format": {+ "pipelex.storage_config.local.uri_format": {"value_type": "str","required": true,"default": "{primary_id}/{secondary_id}/{hash}.{extension}","open_node": false},- "runtime.storage.method": {+ "pipelex.storage_config.method": {"value_type": "str","required": true,"default": "local","open_node": false},- "runtime.storage.s3": {+ "pipelex.storage_config.s3": {"value_type": "table","required": false,"open_node": false},- "runtime.storage.s3.bucket_name": {+ "pipelex.storage_config.s3.bucket_name": {"value_type": "str","required": true,"default": "","open_node": false},- "runtime.storage.s3.region": {+ "pipelex.storage_config.s3.region": {"value_type": "str","required": true,"default": "","open_node": false},- "runtime.storage.s3.signed_urls_lifespan_seconds": {+ "pipelex.storage_config.s3.signed_urls_lifespan_seconds": {"value_type": "int | literal","required": true,"enum_members": [@@ -1387,18 +1382,51 @@"default": 3600,"open_node": false},- "runtime.storage.s3.uri_format": {+ "pipelex.storage_config.s3.uri_format": {"value_type": "str","required": true,"default": "{primary_id}/{secondary_id}/{hash}.{extension}","open_node": false},- "runtime.tracing": {+ "pipelex.templating_config": {"value_type": "table","required": true,"open_node": false},- "runtime.tracing.backend": {+ "pipelex.templating_config.default_templating_style": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "pipelex.templating_config.default_templating_style.tag_style": {+ "value_type": "enum",+ "required": true,+ "enum_members": [+ "no_tag",+ "square_brackets",+ "ticks",+ "xml"+ ],+ "default": "xml",+ "open_node": false+ },+ "pipelex.templating_config.default_templating_style.text_format": {+ "value_type": "enum",+ "required": false,+ "enum_members": [+ "html",+ "json",+ "markdown",+ "plain"+ ],+ "open_node": false+ },+ "pipelex.tracing_config": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "pipelex.tracing_config.backend": {"value_type": "enum","required": true,"enum_members": [@@ -1408,39 +1436,60 @@"default": "ndjson","open_node": false},- "runtime.tracing.dynamodb": {+ "pipelex.tracing_config.dynamodb": {"value_type": "table","required": false,"open_node": false},- "runtime.tracing.dynamodb.region": {+ "pipelex.tracing_config.dynamodb.region": {"value_type": "str","required": true,"default": "","open_node": false},- "runtime.tracing.dynamodb.table_name": {+ "pipelex.tracing_config.dynamodb.table_name": {"value_type": "str","required": true,"default": "","open_node": false},- "runtime.tracing.is_enabled": {+ "pipelex.tracing_config.is_enabled": {"value_type": "bool","required": true,"default": true,"open_node": false},- "runtime.tracing.ndjson": {+ "pipelex.tracing_config.ndjson": {"value_type": "table","required": false,"open_node": false},- "runtime.tracing.ndjson.traces_dir": {+ "pipelex.tracing_config.ndjson.traces_dir": {"value_type": "str","required": true,"default": ".pipelex/traces","open_node": false+ },+ "plugins": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "plugins.boot_orchestrator": {+ "value_type": "str",+ "required": false,+ "open_node": false+ },+ "plugins.disabled": {+ "value_type": "list[str]",+ "required": true,+ "default": [],+ "open_node": false+ },+ "session_id": {+ "value_type": "str",+ "required": false,+ "open_node": false}}}diff --git a/pipelex/migration/goldens/pipelex-config/[email protected] b/pipelex/migration/goldens/pipelex-config/[email protected]new file mode 100644index 000000000..dd4485384--- /dev/null+++ b/pipelex/migration/goldens/pipelex-config/[email protected]@@ -0,0 +1,1447 @@+{+ "surface_id": "pipelex-config",+ "schema_version": 3,+ "document_root_is_open": false,+ "paths": {+ "inference": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "inference.dry_run": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "inference.dry_run.allowed_to_fail_pipes": {+ "value_type": "list[str]",+ "required": false,+ "default": [+ "failing_pipelines.infinite_loop_1"+ ],+ "open_node": false+ },+ "inference.dry_run.image_urls": {+ "value_type": "list[str]",+ "required": true,+ "default": [+ "https://storage.googleapis.com/public_test_files_7fa6_4277_9ab/fashion/fashion_photo_1.jpg",+ "https://storage.googleapis.com/public_test_files_7fa6_4277_9ab/fashion/fashion_photo_2.png"+ ],+ "open_node": false+ },+ "inference.dry_run.nb_extract_pages": {+ "value_type": "int",+ "required": true,+ "default": 4,+ "open_node": false+ },+ "inference.dry_run.nb_list_items": {+ "value_type": "int",+ "required": true,+ "default": 3,+ "open_node": false+ },+ "inference.dry_run.text_gen_truncate_length": {+ "value_type": "int",+ "required": true,+ "default": 256,+ "open_node": false+ },+ "inference.extract": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "inference.extract.default_page_views_dpi": {+ "value_type": "int",+ "required": true,+ "default": 72,+ "open_node": false+ },+ "inference.gateway_test": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "inference.gateway_test.config_id_substitutions": {+ "value_type": "dict[str, str]",+ "required": true,+ "default": {},+ "open_node": true+ },+ "inference.gateway_test.config_id_substitutions.*": {+ "value_type": "str",+ "required": true,+ "open_node": false+ },+ "inference.img_gen": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "inference.img_gen.img_gen_job": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "inference.img_gen.img_gen_job.is_sync_mode": {+ "value_type": "bool",+ "required": true,+ "default": false,+ "open_node": false+ },+ "inference.img_gen.img_gen_param_defaults": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "inference.img_gen.img_gen_param_defaults.aspect_ratio": {+ "value_type": "enum",+ "required": true,+ "enum_members": [+ "landscape_16_9",+ "landscape_21_9",+ "landscape_3_2",+ "landscape_4_1",+ "landscape_4_3",+ "landscape_8_1",+ "portrait_1_4",+ "portrait_1_8",+ "portrait_2_3",+ "portrait_3_4",+ "portrait_9_16",+ "portrait_9_21",+ "square"+ ],+ "default": "square",+ "open_node": false+ },+ "inference.img_gen.img_gen_param_defaults.background": {+ "value_type": "enum",+ "required": true,+ "enum_members": [+ "auto",+ "opaque",+ "transparent"+ ],+ "default": "auto",+ "open_node": false+ },+ "inference.img_gen.img_gen_param_defaults.guidance_scale": {+ "value_type": "float",+ "required": true,+ "default": 2.5,+ "open_node": false,+ "constraints": {+ "gt": 0+ }+ },+ "inference.img_gen.img_gen_param_defaults.is_moderated": {+ "value_type": "bool",+ "required": false,+ "default": true,+ "open_node": false+ },+ "inference.img_gen.img_gen_param_defaults.is_raw": {+ "value_type": "bool",+ "required": false,+ "default": false,+ "open_node": false+ },+ "inference.img_gen.img_gen_param_defaults.nb_steps": {+ "value_type": "int",+ "required": false,+ "default": 28,+ "open_node": false,+ "constraints": {+ "gt": 0+ }+ },+ "inference.img_gen.img_gen_param_defaults.quality": {+ "value_type": "enum",+ "required": false,+ "enum_members": [+ "high",+ "low",+ "medium"+ ],+ "default": "low",+ "open_node": false+ },+ "inference.img_gen.img_gen_param_defaults.safety_tolerance": {+ "value_type": "int",+ "required": true,+ "default": 5,+ "open_node": false,+ "constraints": {+ "ge": 1,+ "le": 6+ }+ },+ "inference.img_gen.img_gen_param_defaults.seed": {+ "value_type": "int | literal",+ "required": true,+ "enum_members": [+ "auto"+ ],+ "default": "auto",+ "open_node": false+ },+ "inference.img_gen.img_gen_param_defaults.size": {+ "value_type": "enum | table",+ "required": false,+ "enum_members": [+ "0.5k",+ "1k",+ "2k",+ "4k"+ ],+ "open_node": false+ },+ "inference.img_gen.quality_to_steps_maps": {+ "value_type": "dict[str, dict[str, int]]",+ "required": true,+ "default": {+ "flux": {+ "low": 14,+ "medium": 28,+ "high": 56+ },+ "sdxl_lightning": {+ "low": 2,+ "medium": 4,+ "high": 8+ },+ "qwen_image": {+ "low": 10,+ "medium": 20,+ "high": 28+ }+ },+ "open_node": true+ },+ "inference.img_gen.quality_to_steps_maps.*": {+ "value_type": "dict[str, int]",+ "required": true,+ "open_node": true+ },+ "inference.img_gen.quality_to_steps_maps.*.*": {+ "value_type": "int",+ "required": true,+ "open_node": false+ },+ "inference.llm": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "inference.llm.anthropic": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "inference.llm.anthropic.effort_to_level_map": {+ "value_type": "dict[str, str]",+ "required": true,+ "default": {+ "none": "disabled",+ "minimal": "low",+ "low": "low",+ "medium": "medium",+ "high": "high",+ "xhigh": "xhigh",+ "max": "max"+ },+ "open_node": true+ },+ "inference.llm.anthropic.effort_to_level_map.*": {+ "value_type": "str",+ "required": true,+ "open_node": false+ },+ "inference.llm.anthropic.structured_output_timeout_seconds": {+ "value_type": "int",+ "required": true,+ "default": 1200,+ "open_node": false+ },+ "inference.llm.default_max_images": {+ "value_type": "int",+ "required": true,+ "default": 100,+ "open_node": false+ },+ "inference.llm.effort_to_budget_maps": {+ "value_type": "dict[str, dict[str, int]]",+ "required": true,+ "default": {+ "anthropic": {+ "none": 0,+ "minimal": 512,+ "low": 1024,+ "medium": 5000,+ "high": 16384,+ "xhigh": 32768,+ "max": 65536+ },+ "gemini": {+ "none": 0,+ "minimal": 512,+ "low": 1024,+ "medium": 5000,+ "high": 16384,+ "xhigh": 32768,+ "max": 65536+ }+ },+ "open_node": true+ },+ "inference.llm.effort_to_budget_maps.*": {+ "value_type": "dict[str, int]",+ "required": true,+ "open_node": true+ },+ "inference.llm.effort_to_budget_maps.*.*": {+ "value_type": "int",+ "required": true,+ "open_node": false+ },+ "inference.llm.generic_templates": {+ "value_type": "dict[str, str]",+ "required": true,+ "default": {+ "output_structure_prompt": "\n\n---\nThe instance we want to generate will be for the following class:\n{{ class_structure_str }}\n\nDO NOT create information.\nIf some information is not present for an attribute, output the default value or None according to the attribute definition.\n",+ "structuring_prompt": "Read the following text carefully and produce the requested structured output from it.\n\n---\n{{ text }}\n"+ },+ "open_node": true+ },+ "inference.llm.generic_templates.*": {+ "value_type": "str",+ "required": true,+ "open_node": false+ },+ "inference.llm.google": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "inference.llm.google.effort_to_level_map": {+ "value_type": "dict[str, str]",+ "required": true,+ "default": {+ "none": "disabled",+ "minimal": "minimal",+ "low": "low",+ "medium": "medium",+ "high": "high",+ "xhigh": "high",+ "max": "high"+ },+ "open_node": true+ },+ "inference.llm.google.effort_to_level_map.*": {+ "value_type": "str",+ "required": true,+ "open_node": false+ },+ "inference.llm.instructor": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "inference.llm.instructor.is_dump_error_enabled": {+ "value_type": "bool",+ "required": true,+ "default": false,+ "open_node": false+ },+ "inference.llm.instructor.is_dump_kwargs_enabled": {+ "value_type": "bool",+ "required": true,+ "default": false,+ "open_node": false+ },+ "inference.llm.instructor.is_dump_response_enabled": {+ "value_type": "bool",+ "required": true,+ "default": false,+ "open_node": false+ },+ "inference.llm.is_dump_response_text_enabled": {+ "value_type": "bool",+ "required": true,+ "default": false,+ "open_node": false+ },+ "inference.llm.is_dump_text_prompts_enabled": {+ "value_type": "bool",+ "required": true,+ "default": false,+ "open_node": false+ },+ "inference.llm.is_structure_prompt_enabled": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "inference.llm.mistral": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "inference.llm.mistral.effort_to_level_map": {+ "value_type": "dict[str, str]",+ "required": true,+ "default": {+ "none": "disabled",+ "minimal": "reasoning",+ "low": "reasoning",+ "medium": "reasoning",+ "high": "reasoning",+ "xhigh": "reasoning",+ "max": "reasoning"+ },+ "open_node": true+ },+ "inference.llm.mistral.effort_to_level_map.*": {+ "value_type": "str",+ "required": true,+ "open_node": false+ },+ "inference.llm.openai": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "inference.llm.openai.effort_to_level_map": {+ "value_type": "dict[str, str]",+ "required": true,+ "default": {+ "none": "none",+ "minimal": "minimal",+ "low": "low",+ "medium": "medium",+ "high": "high",+ "xhigh": "xhigh",+ "max": "xhigh"+ },+ "open_node": true+ },+ "inference.llm.openai.effort_to_level_map.*": {+ "value_type": "str",+ "required": true,+ "open_node": false+ },+ "inference.llm.schema_reask_max_attempts": {+ "value_type": "int",+ "required": true,+ "default": 3,+ "open_node": false,+ "constraints": {+ "ge": 1,+ "le": 10+ }+ },+ "inference.model_deck": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "inference.model_deck.is_model_fallback_enabled": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "inference.model_deck.missing_presets_reaction": {+ "value_type": "enum",+ "required": true,+ "enum_members": [+ "log",+ "none",+ "raise"+ ],+ "default": "log",+ "open_node": false+ },+ "inference.templating": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "inference.templating.default_templating_style": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "inference.templating.default_templating_style.tag_style": {+ "value_type": "enum",+ "required": true,+ "enum_members": [+ "no_tag",+ "square_brackets",+ "ticks",+ "xml"+ ],+ "default": "xml",+ "open_node": false+ },+ "inference.templating.default_templating_style.text_format": {+ "value_type": "enum",+ "required": false,+ "enum_members": [+ "html",+ "json",+ "markdown",+ "plain"+ ],+ "open_node": false+ },+ "inference.transport_max_retries": {+ "value_type": "int",+ "required": true,+ "default": 2,+ "open_node": false,+ "constraints": {+ "ge": 0,+ "le": 10+ }+ },+ "interpreter": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "interpreter.builder": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "interpreter.builder.default_bundle_file_name": {+ "value_type": "str",+ "required": true,+ "default": "bundle",+ "open_node": false+ },+ "interpreter.builder.default_directory_base_name": {+ "value_type": "str",+ "required": true,+ "default": "pipeline",+ "open_node": false+ },+ "interpreter.builder.default_output_dir": {+ "value_type": "str",+ "required": true,+ "default": ".",+ "open_node": false+ },+ "interpreter.builder.fix_loop_max_attempts": {+ "value_type": "int",+ "required": true,+ "default": 10,+ "open_node": false+ },+ "interpreter.mthds": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "interpreter.mthds.concepts": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "interpreter.mthds.concepts.structure_field_ordering": {+ "value_type": "list[str]",+ "required": true,+ "default": [+ "type",+ "concept_ref",+ "item_type",+ "item_concept_ref",+ "description",+ "choices",+ "required"+ ],+ "open_node": false+ },+ "interpreter.mthds.inline_tables": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "interpreter.mthds.inline_tables.spaces_inside_curly_braces": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "interpreter.mthds.pipes": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "interpreter.mthds.pipes.field_ordering": {+ "value_type": "list[str]",+ "required": true,+ "default": [+ "type",+ "description",+ "inputs",+ "output"+ ],+ "open_node": false+ },+ "interpreter.mthds.strings": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "interpreter.mthds.strings.ensure_leading_blank_line": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "interpreter.mthds.strings.ensure_trailing_newline": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "interpreter.mthds.strings.force_multiline": {+ "value_type": "bool",+ "required": true,+ "default": false,+ "open_node": false+ },+ "interpreter.mthds.strings.length_limit_to_multiline": {+ "value_type": "int",+ "required": true,+ "default": 100,+ "open_node": false+ },+ "interpreter.mthds.strings.prefer_literal": {+ "value_type": "bool",+ "required": true,+ "default": false,+ "open_node": false+ },+ "interpreter.pipe_func": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "interpreter.pipe_func.execution_mode": {+ "value_type": "str",+ "required": true,+ "default": "direct",+ "open_node": false+ },+ "interpreter.pipe_run": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "interpreter.pipe_run.pipe_stack_limit": {+ "value_type": "int",+ "required": true,+ "default": 20,+ "open_node": false+ },+ "interpreter.pipeline_execution": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "interpreter.pipeline_execution.graph": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.data_inclusion": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.data_inclusion.error_stack_traces": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.data_inclusion.pipe_and_concept_registry": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.data_inclusion.stuff_html_content": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.data_inclusion.stuff_json_content": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.data_inclusion.stuff_text_content": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.graphs_inclusion": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.graphs_inclusion.graphspec_json": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.graphs_inclusion.mermaidflow_html": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.graphs_inclusion.mermaidflow_mmd": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.graphs_inclusion.reactflow_html": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.mermaid": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.mermaid.direction": {+ "value_type": "enum",+ "required": true,+ "enum_members": [+ "left_to_right",+ "top_down"+ ],+ "default": "left_to_right",+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.mermaid.is_include_contains_edges": {+ "value_type": "bool",+ "required": true,+ "default": false,+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.mermaid.is_include_data_edges": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.mermaid.is_include_selected_outcome_edges": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.mermaid.is_show_stuff_codes": {+ "value_type": "bool",+ "required": true,+ "default": false,+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.mermaid.style": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.mermaid.style.theme": {+ "value_type": "enum",+ "required": true,+ "enum_members": [+ "base",+ "dark",+ "default",+ "forest",+ "neutral"+ ],+ "default": "dark",+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.reactflow": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.reactflow.default_title": {+ "value_type": "str",+ "required": true,+ "default": "Pipelex Graph",+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.reactflow.edge_type": {+ "value_type": "enum",+ "required": true,+ "enum_members": [+ "bezier",+ "smoothstep",+ "step",+ "straight"+ ],+ "default": "bezier",+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.reactflow.initial_zoom": {+ "value_type": "float",+ "required": true,+ "default": 1.0,+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.reactflow.is_use_cdn": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.reactflow.layout_direction": {+ "value_type": "enum",+ "required": true,+ "enum_members": [+ "left_to_right",+ "top_down"+ ],+ "default": "left_to_right",+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.reactflow.nodesep": {+ "value_type": "int",+ "required": true,+ "default": 50,+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.reactflow.pan_to_top": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.reactflow.ranksep": {+ "value_type": "int",+ "required": true,+ "default": 30,+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.reactflow.show_batch_controller": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.reactflow.style": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.reactflow.style.palette": {+ "value_type": "enum",+ "required": true,+ "enum_members": [+ "dracula",+ "yellow_blue"+ ],+ "default": "dracula",+ "open_node": false+ },+ "interpreter.pipeline_execution.graph.reactflow.style.theme": {+ "value_type": "enum",+ "required": true,+ "enum_members": [+ "dark",+ "light",+ "system"+ ],+ "default": "dark",+ "open_node": false+ },+ "interpreter.pipeline_execution.is_generate_graph": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "interpreter.pipeline_execution.is_generate_usage": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "interpreter.pipeline_execution.is_mock_inputs": {+ "value_type": "bool",+ "required": true,+ "default": false,+ "open_node": false+ },+ "interpreter.pipeline_execution.is_normalize_data_urls_to_storage": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "interpreter.pipeline_execution.max_concurrency": {+ "value_type": "int | literal",+ "required": true,+ "enum_members": [+ "unbounded"+ ],+ "default": 8,+ "open_node": false,+ "constraints": {+ "ge": 1+ }+ },+ "interpreter.scan": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "interpreter.scan.excluded_dirs": {+ "value_type": "frozenset[str]",+ "required": true,+ "default": [+ ".venv",+ "venv",+ "env",+ ".env",+ "virtualenv",+ ".virtualenv",+ ".git",+ "__pycache__",+ ".pytest_cache",+ ".mypy_cache",+ ".ruff_cache",+ "node_modules",+ "results"+ ],+ "open_node": false+ },+ "kit": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "kit.preferred_agent_targets": {+ "value_type": "list[enum]",+ "required": true,+ "enum_members": [+ "agents",+ "claude",+ "cursor"+ ],+ "default": [+ "claude",+ "agents"+ ],+ "open_node": false+ },+ "runtime": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "runtime.aws": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "runtime.aws.api_key_method": {+ "value_type": "enum",+ "required": true,+ "enum_members": [+ "env",+ "secret_provider"+ ],+ "default": "env",+ "open_node": false+ },+ "runtime.aws.bedrock_access_variant": {+ "value_type": "enum",+ "required": true,+ "enum_members": [+ "aws_access",+ "bedrock_token"+ ],+ "default": "aws_access",+ "open_node": false+ },+ "runtime.log": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "runtime.log.caller_info_template": {+ "value_type": "enum",+ "required": true,+ "enum_members": [+ "file_func",+ "file_line",+ "file_line_func",+ "func",+ "func_line",+ "func_module",+ "func_module_line"+ ],+ "default": "file_line",+ "open_node": false+ },+ "runtime.log.console_log_target": {+ "value_type": "enum",+ "required": true,+ "enum_members": [+ "file",+ "stderr",+ "stdout"+ ],+ "default": "stderr",+ "open_node": false+ },+ "runtime.log.console_print_target": {+ "value_type": "enum",+ "required": true,+ "enum_members": [+ "file",+ "stderr",+ "stdout"+ ],+ "default": "stdout",+ "open_node": false+ },+ "runtime.log.default_log_level": {+ "value_type": "enum",+ "required": true,+ "enum_members": [+ "CRITICAL",+ "DEBUG",+ "DEV",+ "ERROR",+ "INFO",+ "OFF",+ "VERBOSE",+ "WARNING"+ ],+ "default": "INFO",+ "open_node": false+ },+ "runtime.log.generic_poor_logger": {+ "value_type": "str",+ "required": true,+ "default": "#poor-log",+ "open_node": false+ },+ "runtime.log.is_caller_info_enabled": {+ "value_type": "bool",+ "required": true,+ "default": false,+ "open_node": false+ },+ "runtime.log.is_console_logging_enabled": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "runtime.log.json_logs_indent": {+ "value_type": "int",+ "required": true,+ "default": 4,+ "open_node": false+ },+ "runtime.log.log_mode": {+ "value_type": "enum",+ "required": true,+ "enum_members": [+ "poor",+ "rich"+ ],+ "default": "rich",+ "open_node": false+ },+ "runtime.log.package_log_levels": {+ "value_type": "dict[str, enum]",+ "required": true,+ "default": {+ "anthropic": "INFO",+ "asyncio": "INFO",+ "botocore": "INFO",+ "botocore-credentials": "WARNING",+ "google": "INFO",+ "httpx": "WARNING",+ "httpcore": "INFO",+ "openai": "INFO",+ "instructor": "INFO",+ "urllib3-connectionpool": "INFO",+ "urllib3-util-retry": "INFO",+ "pipelex": "INFO"+ },+ "open_node": true+ },+ "runtime.log.package_log_levels.*": {+ "value_type": "enum",+ "required": true,+ "enum_members": [+ "CRITICAL",+ "DEBUG",+ "DEV",+ "ERROR",+ "INFO",+ "OFF",+ "VERBOSE",+ "WARNING"+ ],+ "open_node": false+ },+ "runtime.log.poor_loggers": {+ "value_type": "list[str]",+ "required": true,+ "default": [],+ "open_node": false+ },+ "runtime.log.presentation_line_width": {+ "value_type": "int",+ "required": true,+ "default": 120,+ "open_node": false+ },+ "runtime.log.rich_log": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "runtime.log.rich_log.highlighter_name": {+ "value_type": "enum",+ "required": true,+ "enum_members": [+ "json",+ "repr"+ ],+ "default": "json",+ "open_node": false+ },+ "runtime.log.rich_log.is_link_path_enabled": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "runtime.log.rich_log.is_markup_enabled": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "runtime.log.rich_log.is_rich_tracebacks": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "runtime.log.rich_log.is_show_level": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "runtime.log.rich_log.is_show_time": {+ "value_type": "bool",+ "required": true,+ "default": false,+ "open_node": false+ },+ "runtime.log.rich_log.is_tracebacks_show_locals": {+ "value_type": "bool",+ "required": true,+ "default": false,+ "open_node": false+ },+ "runtime.log.rich_log.is_tracebacks_word_wrap": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "runtime.log.rich_log.keywords_to_hilight": {+ "value_type": "list[str]",+ "required": true,+ "default": [],+ "open_node": false+ },+ "runtime.log.rich_log.tracebacks_suppress": {+ "value_type": "list[str]",+ "required": true,+ "default": [],+ "open_node": false+ },+ "runtime.log.silenced_problem_ids": {+ "value_type": "list[str]",+ "required": true,+ "default": [+ "azure_openai_no_stream_options"+ ],+ "open_node": false+ },+ "runtime.observer": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "runtime.observer.observer_dir": {+ "value_type": "str",+ "required": true,+ "default": "results/observer",+ "open_node": false+ },+ "runtime.plugins": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "runtime.plugins.disabled": {+ "value_type": "list[str]",+ "required": true,+ "default": [],+ "open_node": false+ },+ "runtime.reporting": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "runtime.reporting.cost_report_base_name": {+ "value_type": "str",+ "required": true,+ "default": "cost_report",+ "open_node": false+ },+ "runtime.reporting.cost_report_dir_path": {+ "value_type": "str",+ "required": true,+ "default": "reports",+ "open_node": false+ },+ "runtime.reporting.cost_report_extension": {+ "value_type": "str",+ "required": true,+ "default": "csv",+ "open_node": false+ },+ "runtime.reporting.cost_report_unit_scale": {+ "value_type": "float",+ "required": true,+ "default": 1.0,+ "open_node": false,+ "constraints": {+ "gt": 0+ }+ },+ "runtime.reporting.is_generate_cost_report_file_enabled": {+ "value_type": "bool",+ "required": true,+ "default": false,+ "open_node": false+ },+ "runtime.reporting.is_log_costs_to_console": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "runtime.secrets": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "runtime.secrets.method": {+ "value_type": "str",+ "required": true,+ "default": "env",+ "open_node": false+ },+ "runtime.storage": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "runtime.storage.gcp": {+ "value_type": "table",+ "required": false,+ "open_node": false+ },+ "runtime.storage.gcp.bucket_name": {+ "value_type": "str",+ "required": true,+ "default": "",+ "open_node": false+ },+ "runtime.storage.gcp.project_id": {+ "value_type": "str",+ "required": true,+ "default": "",+ "open_node": false+ },+ "runtime.storage.gcp.signed_urls_lifespan_seconds": {+ "value_type": "int | literal",+ "required": true,+ "enum_members": [+ "disabled"+ ],+ "default": 3600,+ "open_node": false+ },+ "runtime.storage.gcp.uri_format": {+ "value_type": "str",+ "required": true,+ "default": "{primary_id}/{secondary_id}/{hash}.{extension}",+ "open_node": false+ },+ "runtime.storage.in_memory": {+ "value_type": "table",+ "required": false,+ "open_node": false+ },+ "runtime.storage.in_memory.uri_format": {+ "value_type": "str",+ "required": true,+ "default": "{primary_id}/{secondary_id}/{hash}.{extension}",+ "open_node": false+ },+ "runtime.storage.is_fetch_remote_content_enabled": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "runtime.storage.is_upload_local_content_enabled": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "runtime.storage.local": {+ "value_type": "table",+ "required": false,+ "open_node": false+ },+ "runtime.storage.local.local_storage_path": {+ "value_type": "str",+ "required": true,+ "default": ".pipelex/storage",+ "open_node": false+ },+ "runtime.storage.local.uri_format": {+ "value_type": "str",+ "required": true,+ "default": "{primary_id}/{secondary_id}/{hash}.{extension}",+ "open_node": false+ },+ "runtime.storage.method": {+ "value_type": "str",+ "required": true,+ "default": "local",+ "open_node": false+ },+ "runtime.storage.s3": {+ "value_type": "table",+ "required": false,+ "open_node": false+ },+ "runtime.storage.s3.bucket_name": {+ "value_type": "str",+ "required": true,+ "default": "",+ "open_node": false+ },+ "runtime.storage.s3.region": {+ "value_type": "str",+ "required": true,+ "default": "",+ "open_node": false+ },+ "runtime.storage.s3.signed_urls_lifespan_seconds": {+ "value_type": "int | literal",+ "required": true,+ "enum_members": [+ "disabled"+ ],+ "default": 3600,+ "open_node": false+ },+ "runtime.storage.s3.uri_format": {+ "value_type": "str",+ "required": true,+ "default": "{primary_id}/{secondary_id}/{hash}.{extension}",+ "open_node": false+ },+ "runtime.tracing": {+ "value_type": "table",+ "required": true,+ "open_node": false+ },+ "runtime.tracing.backend": {+ "value_type": "enum",+ "required": true,+ "enum_members": [+ "dynamodb",+ "ndjson"+ ],+ "default": "ndjson",+ "open_node": false+ },+ "runtime.tracing.dynamodb": {+ "value_type": "table",+ "required": false,+ "open_node": false+ },+ "runtime.tracing.dynamodb.region": {+ "value_type": "str",+ "required": true,+ "default": "",+ "open_node": false+ },+ "runtime.tracing.dynamodb.table_name": {+ "value_type": "str",+ "required": true,+ "default": "",+ "open_node": false+ },+ "runtime.tracing.is_enabled": {+ "value_type": "bool",+ "required": true,+ "default": true,+ "open_node": false+ },+ "runtime.tracing.ndjson": {+ "value_type": "table",+ "required": false,+ "open_node": false+ },+ "runtime.tracing.ndjson.traces_dir": {+ "value_type": "str",+ "required": true,+ "default": ".pipelex/traces",+ "open_node": false+ }+ }+}diff --git a/pipelex/migration/goldens/pipelex-service-config/[email protected] b/pipelex/migration/goldens/pipelex-service-config/[email protected]index de28b24f5..d5305e923 100644--- a/pipelex/migration/goldens/pipelex-service-config/[email protected]+++ b/pipelex/migration/goldens/pipelex-service-config/[email protected]@@ -1,6 +1,7 @@{"surface_id": "pipelex-service-config","schema_version": 1,+ "document_root_is_open": false,"paths": {"agreement": {"value_type": "table",diff --git a/pipelex/migration/goldens/telemetry-config/[email protected] b/pipelex/migration/goldens/telemetry-config/[email protected]index 1a2ebb466..58b62196e 100644--- a/pipelex/migration/goldens/telemetry-config/[email protected]+++ b/pipelex/migration/goldens/telemetry-config/[email protected]@@ -1,6 +1,7 @@{"surface_id": "telemetry-config","schema_version": 2,+ "document_root_is_open": false,"paths": {"custom_portkey": {"value_type": "table",diff --git a/pipelex/migration/ledger.py b/pipelex/migration/ledger.pyindex 29a62d636..2102f8772 100644--- a/pipelex/migration/ledger.py+++ b/pipelex/migration/ledger.py@@ -47,14 +47,31 @@ class SurfaceBlock(BaseModel):id: strtitle: str- base_file: str+ base_file: str | None = Nonetier_glob: str | None = None++ subdirectory: str = ""+ """The directory this surface's files sit in, relative to a configuration directory.++ Empty for a surface that lives directly in `~/.pipelex/` or `.pipelex/`. Spelled with forward+ slashes, because TOML has no `Path` and the registry converts at the edge. Read here rather+ than from the registry by `stale_configuration_warning`, whose module may not import the+ registry: the registry pulls in every configuration model, and it sits in the kernel layer's+ import closure. The coverage gate cross-checks the two spellings against each other."""+current_schema_version: int = Field(ge=INITIAL_SCHEMA_VERSION)min_supported_schema_version: int = Field(ge=0)"""Held at zero until a ledger squash ever moves it. Without the floor, a squash silentlyunder-migrates the oldest files in the field, because the applier skips absent targets andreports success. With it, the loader fails loudly instead."""+ @model_validator(mode="after")+ def check_the_surface_claims_some_file(self) -> Self:+ if not self.base_file and not self.tier_glob:+ msg = f"surface '{self.id}': claims no file — a surface with no base file must claim its files by a tier glob"+ raise ValueError(msg)+ return self+@model_validator(mode="after")def check_floor_is_below_the_current_version(self) -> Self:if self.min_supported_schema_version > self.current_schema_version:diff --git a/pipelex/migration/ledger_check.py b/pipelex/migration/ledger_check.pyindex 867fe86db..1ca0cc0fb 100644--- a/pipelex/migration/ledger_check.py+++ b/pipelex/migration/ledger_check.py@@ -376,6 +376,12 @@ def _check_open_node_addressing() -> list[LedgerIssue]:"""`*` exactly at an open node, and never a concrete key beneath one.+ **The document root is one of the nodes this rule reads.** For most surfaces it is closed: the+ root keys of `pipelex.toml` are ours and enumerable, so a `*` there addresses nothing. For a+ backend definition file the root keys are model names the user chose, and then `*` at the root is+ the only legal address for the whole file — while naming one of those tables is the same defect+ as reaching inside an open mapping, and is reported as one.+The check runs on the source traced back to the previous fingerprint's spelling, so that anoperation acting inside a table an earlier operation in the same entry renamed is judgedagainst the node it really addresses. A pre-history source has no such spelling to trace back@@ -392,8 +398,16 @@ def _check_open_node_addressing(segments = origin_path.split(PATH_SEPARATOR)for index, segment in enumerate(segments):parent_path = PATH_SEPARATOR.join(segments[:index])- parent_record = before.paths.get(parent_path) if index else None- parent_is_open = parent_record is not None and parent_record.open_node+ if index:+ parent_record = before.paths.get(parent_path)+ parent_is_open = parent_record is not None and parent_record.open_node+ else:+ # The first segment's parent is the document itself, which has no record of its own — its+ # openness is a property of the whole fingerprint. Both halves of the rule then hold at+ # the root for free: `*` is legal there for a backend definition file, and naming one+ # root table of one is a concrete key under an open node, which is exactly right — those+ # keys are the model names that machine chose.+ parent_is_open = before.document_root_is_openif segment == WILDCARD_SEGMENT and not parent_is_open:parent_label = f"'{parent_path}'" if parent_path else "the document root"issues.append(diff --git a/pipelex/migration/ledgers/inference-backend.toml b/pipelex/migration/ledgers/inference-backend.tomlnew file mode 100644index 000000000..b735e18a0--- /dev/null+++ b/pipelex/migration/ledgers/inference-backend.toml@@ -0,0 +1,55 @@+# The migration ledger for the inference backend definitions.+#+# One file per backend under `inference/backends/`, each a `[defaults]` table plus one root table per+# model name. The root keys of such a file are the model names *that machine* chose, so every+# operation here addresses `["*"]` — every root table, `[defaults]` included — and none may name one.+#+# See docs/migration-ledger.md for the contract this file obeys.++[surface]+id = "inference-backend"+title = "Inference backend model definitions"+subdirectory = "inference/backends"+tier_glob = "*.toml"+current_schema_version = 2+min_supported_schema_version = 0++[[migration]]+id = "inference-backend@2"+to_schema_version = 2+introduced_in = "0.46.0"+breaking = true+safety = "safe"+pre_history = true+title = "Drop prompting_target from every backend definition"+description = """+`prompting_target` used to name the prompt dialect a model wanted — `anthropic`, `gemini` — and a \+backend file set it in `[defaults]` for the whole file and overrode it on the models that differed. \+Prompt templating is now authored on the pipe (`templating_style` on a `PipeLLM`) rather than \+inferred from the model, so the field was removed from the model-spec blueprint and from every \+backend file we ship. A file written before that removal is not merely out of date: the blueprint \+is strict, and `pipelex init` never overwrites a file you already have, so the key stays behind and \+breaks the boot. This entry deletes it wherever it sits.+"""+guidance = """+The key is deleted rather than moved, because nothing today holds a per-model prompt dialect: choose+the style on the pipe that needs it, with `templating_style` on the `PipeLLM`.++One occurrence in `[defaults]` is enough to break the whole file — that table is copied into every+model of it before validation — which is why the operation below addresses every root table and not+only the models that spelled the key out.++This entry accounts for what *we* removed, and for nothing else. A per-model key of your own that+the loader refuses is still yours to fix: it is named, with the reason, in the message that refuses+it.+"""++# Declared rather than derived: this change predates the first fingerprint, so no snapshot shows the+# path going away and there is no diff to account the entry against. The hand-authored document the+# operation below is verified against is goldens/inference-backend/[email protected].+declared_removed_paths = ["*.prompting_target"]++[[migration.ops]] # `["*"]` is every root table of the file: `[defaults]` and each model alike+kind = "delete_key"+table_path = ["*"]+key = "prompting_target"diff --git a/pipelex/migration/ledgers/pipelex-config.toml b/pipelex/migration/ledgers/pipelex-config.tomlindex 5e4bf4fe2..dfe51c301 100644--- a/pipelex/migration/ledgers/pipelex-config.toml+++ b/pipelex/migration/ledgers/pipelex-config.toml@@ -12,15 +12,68 @@ id = "pipelex-config"title = "The main Pipelex configuration"base_file = "pipelex.toml"tier_glob = "pipelex_*.toml"-current_schema_version = 2+current_schema_version = 3min_supported_schema_version = 0+# This entry is inserted *below* the reshape rather than appended above it, and the order is what+# makes it work: it addresses `pipelex.prompting_config`, the spelling that exists only while the+# root table is still called `[pipelex]`. After the reshape it would find nothing, and the file+# would come out of the run carrying `[interpreter.prompting_config]` — a table nothing reads.[[migration]]id = "pipelex-config@2"to_schema_version = 2introduced_in = "0.46.0"breaking = truesafety = "safe"+pre_history = true+title = "Prompting styles become a templating default"+description = """+A prompting style used to be chosen for you, per *target*: the configuration held a map from a \+model family — `openai`, `anthropic`, `mistral`, `gemini` — to the tag style prompts for it should \+use, and a `default_prompting_style` for everything else. Styling is now a property of the prompt \+rather than of the model that receives it, so the map went away and what remains is one default, \+under the name the templating section gives it. This entry carries a file written before that \+change onto the shape the reshape then moves.+"""+guidance = """+`default_prompting_style` travels: it becomes `default_templating_style`, and the reshape entry \+after this one carries it on to `[inference.templating]` with the rest of that section.++A tuned `[pipelex.prompting_config.prompting_styles]` map is **dropped**, and nothing restores it \+from somewhere else — per-target styling is not a concept any more. Where a particular pipe needs \+a particular style, say so on the pipe: `templating_style` on a `PipeLLM`.+"""++# Declared rather than derived: this change predates the first fingerprint, so no snapshot shows+# these paths going away and there is no diff to account the entry against. The hand-authored+# document the operations below are verified against is goldens/pipelex-config/[email protected].+declared_removed_paths = [+ "pipelex.prompting_config",+ "pipelex.prompting_config.default_prompting_style",+ "pipelex.prompting_config.prompting_styles",+]++[[migration.ops]] # the destination table does not exist before this: the move creates it+kind = "move_key"+table_path = ["pipelex", "prompting_config"]+key = "default_prompting_style"+new_table_path = ["pipelex", "templating_config"]+new_key = "default_templating_style"++[[migration.ops]] # the map goes first, so what is left of the section is empty when it goes+kind = "delete_table"+table_path = ["pipelex", "prompting_config", "prompting_styles"]++[[migration.ops]]+kind = "delete_table"+table_path = ["pipelex", "prompting_config"]++[[migration]]+id = "pipelex-config@3"+to_schema_version = 3+introduced_in = "0.46.0"+breaking = true+safety = "safe"title = "The configuration reshape: one scheme for the root"description = """The root gets the layers the code already has, instead of a grab-bag (`[pipelex]`), a codename \diff --git a/pipelex/migration/run.py b/pipelex/migration/run.pyindex 0f27e360e..fc4ea9ac7 100644--- a/pipelex/migration/run.py+++ b/pipelex/migration/run.py@@ -16,11 +16,16 @@ user would be told one thing by their boot and another by their tool.> this repository's own test fixture. Neither is a user's configuration, and migrating a> directory nobody asked about is how a tool earns a reputation for touching things.-> **The walk is not recursive.** A surface's tier files sit beside its base file; a subdirectory-> under a configuration directory holds a different kind of thing — `inference/backends/`,-> `deck/` — whose files are not configuration surfaces even when their names would match a tier-> glob. `.pipelex/inference/backends/pipelex_gateway.toml` is the specimen: it matches the-> `pipelex-config` tier glob `pipelex_*.toml` and must never be claimed by it.+> **The walk is each surface's own directory, one level, and nothing else.** Most surfaces live+> directly in a configuration directory; a surface may instead own a subdirectory of it, and then+> that subdirectory is walked one level in the same way. A subdirectory no surface owns —+> `inference/deck/` — is never entered at all.+>+> **A file is claimed by the pair (directory, name), never by its name alone.**+> `.pipelex/inference/backends/pipelex_gateway.toml` is the specimen: its name matches the+> `pipelex-config` tier glob `pipelex_*.toml` exactly, and the directory it sits in is what says+> it belongs to `inference-backend` instead. Depth used to be what protected it, back when no+> surface owned a subdirectory; now the claim rule is.See `docs/migration-ledger.md` → "Surfaces" and "Applying"."""diff --git a/pipelex/migration/runner.py b/pipelex/migration/runner.pyindex 1f308a49d..f4071a96e 100644--- a/pipelex/migration/runner.py+++ b/pipelex/migration/runner.py@@ -121,7 +121,7 @@ def _diagnose(*, surface: Surface, ledger: MigrationLedger, document: TOMLDocume"""fingerprint = surface.fingerprint_at(schema_version=ledger.surface.current_schema_version)migrated = document if replay.text == text else tomlkit.loads(replay.text)- return diagnose_unexplained_paths(fingerprint=fingerprint, document=migrated.unwrap(), ledger=ledger, blocked=replay.blocked)+ return diagnose_unexplained_paths(surface=surface, fingerprint=fingerprint, document=migrated.unwrap(), ledger=ledger, blocked=replay.blocked)def _refuse_a_file_below_the_floor(*, surface: Surface, ledger: MigrationLedger, file_path: Path, document: TOMLDocument) -> MigrationPlan | None:diff --git a/pipelex/migration/surfaces.py b/pipelex/migration/surfaces.pyindex 8a15fed85..706d60fdc 100644--- a/pipelex/migration/surfaces.py+++ b/pipelex/migration/surfaces.py@@ -20,9 +20,18 @@ import tomlkitfrom pydantic import BaseModel, ConfigDict, Field, ValidationError, model_validatorfrom typing_extensions import Self+from pipelex.cogt.model_backends.model_spec_document import (+ MODEL_SPEC_DEFAULTS_TABLE,+ InferenceModelSpecFileNode,+ describe_model_spec_document_rejection,+)+from pipelex.cogt.model_backends.model_spec_keys import is_header_shaped, is_legal_header_namefrom pipelex.migration.exceptions import MigrationRegistryErrorfrom pipelex.migration.fingerprint import SurfaceFingerprint, compute_fingerprint+from pipelex.suggested_fix import WILDCARD_SEGMENT+from pipelex.system.configuration.config_loader import BACKENDS_DIR_NAME, INFERENCE_DIR_NAMEfrom pipelex.system.configuration.config_surface import (+ INFERENCE_BACKEND_CONFIG_SURFACE_ID,PIPELEX_CONFIG_SURFACE_ID,PIPELEX_SERVICE_CONFIG_SURFACE_ID,TELEMETRY_CONFIG_SURFACE_ID,@@ -53,6 +62,54 @@ class DefaultsLayerKind(StrEnum):MODEL_DEFAULTS = "model_defaults""""Field-level defaults on the model; the reference document is synthesized from them."""+ COPIED_DOCUMENT = "copied_document"+ """A document shipped in the package and **copied** into the user's directory rather than merged+ beneath it — one inference backend definition per file. The values come from the model's own field+ defaults, applied to each entry as it is read, so an added key is absorbed exactly as it is for a+ packaged document; what is different is that the user's file stands alone, with nothing under it."""++ @property+ def is_layered_beneath_the_users_file(self) -> bool:+ """Whether a user's file is read *on top of* our document, or on its own.++ The one place it decides something: a migrated document is checked against the current schema+ the way it is really read, and for a layered surface that means beneath the reference+ document. A copied document has nothing beneath it, and merging two of them — one user's+ backend file over the reference copy of another — would validate a hybrid no machine has,+ turning both a false pass and a false failure into possibilities.+ """+ match self:+ case DefaultsLayerKind.PACKAGED_DOCUMENT | DefaultsLayerKind.MODEL_DEFAULTS:+ return True+ case DefaultsLayerKind.COPIED_DOCUMENT:+ return False+++class DocumentShape(StrEnum):+ """What one of a surface's files *is*, as a document.++ Three things read it, and they have to agree or the gate goes green over a file that does not+ boot: what the fingerprint records, what "the current schema accepts this document" means, and+ which unnamed keys the diagnosis is willing to leave unexplained.+ """++ WHOLE_DOCUMENT = "whole_document"+ """The file is one instance of `config_model`: its root keys are ours and enumerable."""++ MODEL_SPEC_TABLES = "model_spec_tables"+ """The file is an inference backend definition — a `[defaults]` table plus one root table per+ model name, where `config_model` describes one such table rather than the whole document. Neither+ half validates alone; only their merge does, which is why this shape has a validator of its own."""++ @property+ def document_root_is_open(self) -> bool:+ """Whether the document's own root keys belong to the user."""+ match self:+ case DocumentShape.WHOLE_DOCUMENT:+ return False+ case DocumentShape.MODEL_SPEC_TABLES:+ return True+class Surface(BaseModel):"""One artifact family with one schema version and one ledger."""@@ -61,12 +118,42 @@ class Surface(BaseModel):surface_id: strtitle: str- base_file: str++ base_file: str | None = None+ """The one file of the family that is distinguished by name, when there is one.++ A family whose members are all alike — the inference backend definitions, one file per backend+ — has none, and claims its files by `tier_glob` alone."""+tier_glob: str | None = None++ subdirectory: Path = Path()+ """The directory, relative to a configuration directory, whose files this surface claims.++ Empty for a surface that lives directly in `~/.pipelex/` or `.pipelex/`. A surface that owns a+ subdirectory owns it *one level deep and to the exclusion of every other surface*, which is+ what makes `inference/backends/pipelex_gateway.toml` safe: its name matches the main+ configuration's tier glob exactly, and only the directory it sits in says it is not a+ `pipelex.toml` tier file."""+config_model: type[BaseModel]++ document_shape: DocumentShape = DocumentShape.WHOLE_DOCUMENT+ """What one of this surface's files is, as a document — one instance of `config_model`, or a table+ per user-chosen key. See `DocumentShape`; everything the fingerprint, the ledger check and the+ diagnosis do differently for a backend definition file follows from this one declaration."""+defaults_layer_kind: DefaultsLayerKindpackaged_document_path: Path | None = None+ reference_document_path: Path | None = None+ """The shipped document a **copied-document** surface snapshots as its reference.++ Deliberately not a reuse of `packaged_document_path`, whose name means *merged beneath the user's+ file* — which this is not. What it is for: a later, non-pre-history entry is replayed over+ `[email protected]`, so the surface owes the chain one real starting document even though nothing is+ layered beneath a user's file at boot."""+kit_template_path: Path | None = None"""The sparse starter file `pipelex init` copies into a configuration directory.@@ -76,15 +163,52 @@ class Surface(BaseModel):set, the template has almost none — and an operation that misbehaves on an absent key wouldpass over one and fail over the other."""+ kit_template_dir: Path | None = None+ """The kit directory whose files `pipelex init` copies, when a surface's starter is a whole+ directory rather than one file.++ Every file in it that this surface claims becomes a witness, which is more of them than any other+ surface has and costs nothing: they are read live, and they are exactly the documents a fresh+ machine starts from. It is a *directory* and not a resolved list because building the registry+ must touch no filesystem — a registry that read a directory at construction would make every+ consumer, including the ones that never fingerprint anything, depend on the package layout."""++ @model_validator(mode="after")+ def check_the_surface_claims_some_file(self) -> Self:+ if not self.base_file and not self.tier_glob:+ msg = f"surface '{self.surface_id}': claims no file — a surface with no base file must claim its files by a tier glob"+ raise ValueError(msg)+ return self+@model_validator(mode="after")def check_defaults_layer_is_reachable(self) -> Self:- has_packaged_path = self.packaged_document_path is not None- wants_packaged_path = self.defaults_layer_kind is DefaultsLayerKind.PACKAGED_DOCUMENT- if has_packaged_path != wants_packaged_path:+ """Each defaults-layer kind names exactly the source it reads, and no other.++ The pairing is what makes `render_reference_document` and `read_defaults_document` total: each+ branch asserts the path its kind requires, and a surface that named the wrong one — or none —+ would fail at the first regeneration instead of at registry build, which is where a+ declaration error belongs.+ """+ declared_paths = {+ DefaultsLayerKind.PACKAGED_DOCUMENT: self.packaged_document_path,+ DefaultsLayerKind.COPIED_DOCUMENT: self.reference_document_path,+ }+ for kind, path in declared_paths.items():+ if (path is not None) != (self.defaults_layer_kind is kind):+ msg = (+ f"surface '{self.surface_id}': defaults_layer_kind '{self.defaults_layer_kind}' and the "+ f"document path declared for '{kind}' ({path!r}) disagree — each kind names the one "+ f"document it reads, and a model-defaults surface names none"+ )+ raise ValueError(msg)+ return self++ @model_validator(mode="after")+ def check_the_kit_starter_is_one_thing(self) -> Self:+ if self.kit_template_path is not None and self.kit_template_dir is not None:msg = (- f"surface '{self.surface_id}': defaults_layer_kind '{self.defaults_layer_kind}' and "- f"packaged_document_path {self.packaged_document_path!r} disagree — a packaged-document "- f"surface names its document and a model-defaults surface does not"+ f"surface '{self.surface_id}': names both a kit template file and a kit template directory — "+ f"the kit ships a surface one starter, either a file or a directory of them")raise ValueError(msg)return self@@ -102,8 +226,80 @@ class Surface(BaseModel):schema_version=schema_version,config_model=self.config_model,defaults_document=self.read_defaults_document(),+ document_root_is_open=self.document_shape.document_root_is_open,)+ def validate_document(self, *, document: dict[str, Any]) -> str | None:+ """Why the current schema refuses this document, or `None` when it accepts it.++ The seam the transform check asks *is a migrated file still loadable?* through, and the only+ place a surface's document shape decides what "loadable" means. For a whole document that is+ `config_model` validating it. For a backend definition file it is what the loader itself does+ — pop `[defaults]`, split the header-shaped keys off each model table, merge, validate the+ merge — because neither half validates alone and a check that projected the shape its own way+ could pass a document that does not boot.++ A **reason string rather than an exception**, on purpose: the caller reports it as one issue+ among many in its own vocabulary, and a new error class here would earn nothing while costing+ the error-identity snapshot and a generated reference page.+ """+ match self.document_shape:+ case DocumentShape.WHOLE_DOCUMENT:+ try:+ self.config_model.model_validate(document)+ except ValidationError as exc:+ return f"{self.config_model.__name__} rejects it — {exc}"+ return None+ case DocumentShape.MODEL_SPEC_TABLES:+ return describe_model_spec_document_rejection(document=document)++ def admits_unnamed_key(self, *, node_path: tuple[str, ...], document_node_path: tuple[str, ...], key: str) -> bool:+ """Whether a key the fingerprint cannot resolve is nevertheless one this surface expects.++ The diagnosis reports every path a migrated file carries that the current schema cannot+ explain, on **every** run, so a key class we ship and endorse must not be reported. A backend+ file has exactly one: a per-model request header (`x-portkey-provider`), which is a legal key+ by *shape* rather than by name and therefore resolves against no recorded path.++ Deliberately narrower than "the model allows extras": a typo (`promting_target`) is still+ named, because it is not header-shaped. A hyphenated spelling of a known field (`max-tokens`)+ *is* admitted here on purpose — the loader already rejects it by name with the right advice,+ and this channel has nothing to add to that.++ **And narrower than the schema path can express, which is why the document path is here+ too.** `[defaults]` and a model table are the same node to the fingerprint — both are the `*`+ beneath an open root — but they are not the same to the loader: a model table goes through+ `split_model_spec_keys`, and `[defaults]` is copied into every model of the file *unsplit*.+ So a header in `[defaults]` is `extra_forbidden` on every model, and admitting it here would+ report the one file that cannot boot as the one file with nothing to explain — on the command+ the boot error sends that reader to.++ **Only the key, never its value, and that is deliberate.** A header whose value the loader+ refuses (`x-foo = 3`, or one carrying a line break) is admitted here, because this channel+ speaks one sentence — *the schema has no setting there and no entry retires it, so either the+ name is a typo or this file came from a newer pipelex* — and that sentence is simply false+ about a well-named header with a bad value. No surface's diagnosis reads values, and the+ loader already says the right thing in full, naming the model, the backend and the file; a+ scan that finds nothing attaches no migration block, so nothing competes with it.++ Args:+ node_path: The containing table's path in the *fingerprint's* vocabulary, where an open+ node's children collapse to `*`.+ document_node_path: The containing table's path as the document spells it, which is what+ tells `[defaults]` from a model the user named.+ key: The unresolved key itself.+ """+ match self.document_shape:+ case DocumentShape.WHOLE_DOCUMENT:+ return False+ case DocumentShape.MODEL_SPEC_TABLES:+ return (+ node_path == (WILDCARD_SEGMENT,)+ and document_node_path != (MODEL_SPEC_DEFAULTS_TABLE,)+ and is_header_shaped(key=key)+ and is_legal_header_name(key=key)+ )+def render_reference_document(self) -> str:"""The surface's complete reference document as TOML text, ready to check in.@@ -117,6 +313,10 @@ class Surface(BaseModel):# Guaranteed present by the validator above.assert self.packaged_document_path is not Nonereturn self.packaged_document_path.read_text(encoding="utf-8")+ case DefaultsLayerKind.COPIED_DOCUMENT:+ # Likewise guaranteed by the validator above.+ assert self.reference_document_path is not None+ return self.reference_document_path.read_text(encoding="utf-8")case DefaultsLayerKind.MODEL_DEFAULTS:document: str = tomlkit.dumps(self.read_defaults_document()) # pyright: ignore[reportUnknownMemberType]return document@@ -124,15 +324,39 @@ class Surface(BaseModel):def reference_documents(self) -> list[tuple[str, str]]:"""Every document a replay must be neutral over, as `(label, text)` pairs.- Both are at the current schema by construction — one is the packaged defaults layer or the- document the model's own defaults synthesize, the other is the starter template the kit- ships — so neither is a fixture anyone has to keep up to date.+ All of them are at the current schema by construction — the packaged defaults layer or the+ document the model's own defaults synthesize, plus whatever starter the kit ships — so none is+ a fixture anyone has to keep up to date.++ A surface whose kit starter is a whole directory is witnessed by every file in it that the+ surface claims, minus the one that is already the reference document: witnessing a document+ twice proves nothing, and reading it under two labels would report one defect as two."""documents = [("reference document", self.render_reference_document())]if self.kit_template_path is not None:documents.append(("kit template", self.kit_template_path.read_text(encoding="utf-8")))+ for path in self._kit_template_directory_files():+ documents.append((f"kit template '{path.name}'", path.read_text(encoding="utf-8")))return documents+ def _kit_template_directory_files(self) -> list[Path]:+ """The files of `kit_template_dir` this surface claims, sorted, minus its reference document.++ Read here rather than at registry build, so that constructing the registry touches no+ filesystem. A directory the package does not ship is empty rather than an error: the caller is+ collecting witnesses, and a missing one is the gate's own packaging problem, which the+ reference document's absence reports first and by name.+ """+ if self.kit_template_dir is None or not self.kit_template_dir.is_dir():+ return []+ return [+ path+ for path in sorted(self.kit_template_dir.iterdir())+ if path.is_file()+ and path != self.reference_document_path+ and (self.base_file == path.name or (self.tier_glob is not None and fnmatch(path.name, self.tier_glob)))+ ]+def read_defaults_document(self) -> dict[str, Any]:"""The surface's complete reference document, as a plain mapping.@@ -146,6 +370,18 @@ class Surface(BaseModel):# Guaranteed present by the validator above; the assert keeps the type checker honest.assert self.packaged_document_path is not Nonereturn load_toml_from_path(path=self.packaged_document_path)+ case DefaultsLayerKind.COPIED_DOCUMENT:+ # **No defaults document, and none must be faked.** The reference document is one+ # backend definition among several, not a layer beneath the others, so reading it here+ # would attribute one file's values to every file of the surface — and the fingerprint+ # would record them as the defaults a migrated file may rely on.+ #+ # The rule the defaults layer exists to enforce holds anyway, for a better reason:+ # `check_defaults_layer` demands a value for every *required* path, and this surface+ # has none — every path of it sits under a `*` segment, and the projection beneath that+ # segment has no required field. So an added key is absorbed because nothing is+ # required of a file, not because a document supplies it.+ return {}case DefaultsLayerKind.MODEL_DEFAULTS:try:instance = self.config_model()@@ -201,11 +437,19 @@ class SurfaceRegistry(BaseModel):filenames claim before globs across all surfaces, so a base file matching anothersurface's glob is fine and resolved by that rule — but two surfaces sharing an id, abase file or a tier glob have no rule to separate them.++ **A name collides only within one directory.** A base file and a glob are both claims over+ a directory's own listing, so the same spelling in two directories is two different claims+ — and `*.toml` in particular is the whole language of a surface that owns a directory,+ which the second such surface must not have to narrow for the registry's sake."""_reject_duplicates(values=[surface.surface_id for surface in self.surfaces], label="surface id")- _reject_duplicates(values=[surface.base_file for surface in self.surfaces], label="base file")- _reject_duplicates(- values=[surface.tier_glob for surface in self.surfaces if surface.tier_glob is not None],+ _reject_duplicates_per_directory(+ values=[(surface.subdirectory, surface.base_file) for surface in self.surfaces if surface.base_file is not None],+ label="base file",+ )+ _reject_duplicates_per_directory(+ values=[(surface.subdirectory, surface.tier_glob) for surface in self.surfaces if surface.tier_glob is not None],label="tier glob",)return self@@ -217,16 +461,23 @@ class SurfaceRegistry(BaseModel):msg = f"no surface '{surface_id}' in this registry"raise MigrationRegistryError(msg)- def surface_for_file_name(self, *, file_name: str) -> Surface | None:- """Which surface owns a file, by its name alone, or `None` when no surface claims it.+ def surface_for_file(self, *, subdirectory: Path, file_name: str) -> Surface | None:+ """Which surface owns a file, by the directory it sits in and its name, or `None`.++ > **A file is claimed by the pair (directory, name).** Only the surfaces that own the+ > directory the file sits in are candidates; among those, exact filenames claim before+ > globs.- > **Exact filenames claim before globs, across all surfaces.** A file matched by any- > surface's exact pattern belongs to that surface and is excluded from every surface's- > glob.+ Both halves are load-bearing and each answers a real collision. Without the *name* rule,+ `pipelex_service.toml` is both the base file of one surface and a match for another's+ `pipelex_*.toml`. Without the *directory* rule, `inference/backends/pipelex_gateway.toml`+ is a `pipelex_*.toml` match too — and it is an inference backend definition, so the main+ configuration's ledger would be replayed over it and rewrite it.- Without the rule, `pipelex_service.toml` is both the base file of one surface and a match- for another's `pipelex_*.toml`, and which ledger runs over it becomes an accident of- iteration order.+ Args:+ subdirectory: The file's directory, relative to a configuration directory. Empty for a+ file sitting directly in one.+ file_name: The file's own name, with no directory part.Raises:MigrationRegistryError: two surfaces' globs both claim this file. That the registry@@ -235,10 +486,11 @@ class SurfaceRegistry(BaseModel):contract's *a file claimed by two globs is a registry error* becomes enforceable,by name, on the file that proves it."""- for surface in self.surfaces:+ candidates = [surface for surface in self.surfaces if surface.subdirectory == subdirectory]+ for surface in candidates:if surface.base_file == file_name:return surface- claimants = [surface for surface in self.surfaces if surface.tier_glob is not None and fnmatch(file_name, surface.tier_glob)]+ claimants = [surface for surface in candidates if surface.tier_glob is not None and fnmatch(file_name, surface.tier_glob)]if len(claimants) > 1:named = ", ".join(f"'{surface.surface_id}' ({surface.tier_glob})" for surface in claimants)msg = f"registry error: '{file_name}' is claimed by the tier globs of {named} — a file belongs to exactly one surface"@@ -246,21 +498,29 @@ class SurfaceRegistry(BaseModel):return claimants[0] if claimants else Nonedef files_by_surface_in_directory(self, *, directory: Path) -> list[tuple[Surface, Path]]:- """Every file in one configuration directory that a surface claims, with its surface.+ """Every file one configuration directory holds that a surface claims, with its surface.++ The walk is **each surface's own directory under this one, one level deep**: the+ configuration directory itself for the surfaces that live in it, and `directory /+ subdirectory` for a surface that owns one. A directory no surface owns — `inference/deck/`+ — is never entered at all, which is the property that keeps the walk from wandering into+ the parts of a configuration directory that are not configuration surfaces.- A directory that does not exist is skipped rather than refused: the global `~/.pipelex/`- and a project's `.pipelex/` are both optional, and a machine that has only one of them is- an ordinary machine, not a broken one.+ A directory that does not exist is skipped rather than refused, and that goes for both+ levels: the global `~/.pipelex/` and a project's `.pipelex/` are both optional, and a+ machine that has never touched an inference backend has no `inference/` under either."""- if not directory.is_dir():- return []claimed: list[tuple[Surface, Path]] = []- for path in sorted(directory.iterdir()):- if not path.is_file():+ for subdirectory in sorted({surface.subdirectory for surface in self.surfaces}):+ owned_directory = directory / subdirectory+ if not owned_directory.is_dir():continue- surface = self.surface_for_file_name(file_name=path.name)- if surface is not None:- claimed.append((surface, path))+ for path in sorted(owned_directory.iterdir()):+ if not path.is_file():+ continue+ surface = self.surface_for_file(subdirectory=subdirectory, file_name=path.name)+ if surface is not None:+ claimed.append((surface, path))return claimed@@ -273,6 +533,20 @@ def _reject_duplicates(*, values: list[str], label: str) -> None:seen.add(value)+def _reject_duplicates_per_directory(*, values: list[tuple[Path, str]], label: str) -> None:+ """The same rule as `_reject_duplicates`, scoped to one directory.++ A base file and a tier glob are claims over a directory's own listing, so two surfaces collide+ on one only when they own the same directory.+ """+ seen: set[tuple[Path, str]] = set()+ for subdirectory, value in values:+ if (subdirectory, value) in seen:+ msg = f"registry error: two surfaces share the {label} '{value}' in the same directory '{subdirectory.as_posix()}'"+ raise ValueError(msg)+ seen.add((subdirectory, value))++def build_config_surface_registry() -> SurfaceRegistry:"""The real registry: the configuration surfaces the `pipelex` package owns.@@ -309,5 +583,36 @@ def build_config_surface_registry() -> SurfaceRegistry:defaults_layer_kind=DefaultsLayerKind.MODEL_DEFAULTS,kit_template_path=_KIT_CONFIGS_DIR / PIPELEX_SERVICE_CONFIG_FILE_NAME,),+ # **Every `*.toml` directly in `inference/backends/`, and nothing else.**+ #+ # Not `inference/backends.toml`: it sits one level up, it is a different document+ # altogether — a table per backend, not per model — and no schema change has broken it.+ # The two `.md` files that live in the backend directory fall out by extension.+ #+ # Not `inference/deck/`: the model decks already have a manifest-driven sync with an+ # `x_custom_*` carve-out for what the user owns, and a second mechanism replaying a+ # ledger over the same files would be two tools writing one directory. Not+ # `inference/routing_profiles.toml` either: it has neither mechanism, but it also has no+ # break to repair, and a surface is claimed when there is something to carry forward.+ #+ # What *is* in scope is the schema of a backend file — which keys a model table may carry+ # — and not its content: which models a machine defines is the machine's business.+ Surface(+ surface_id=INFERENCE_BACKEND_CONFIG_SURFACE_ID,+ title="Inference backend model definitions",+ base_file=None,+ tier_glob="*.toml",+ subdirectory=Path(INFERENCE_DIR_NAME) / BACKENDS_DIR_NAME,+ config_model=InferenceModelSpecFileNode,+ document_shape=DocumentShape.MODEL_SPEC_TABLES,+ defaults_layer_kind=DefaultsLayerKind.COPIED_DOCUMENT,+ # `portkey.toml` of the kit, because it is the one shipped backend file that+ # exercises both halves of the document shape — a `[defaults]` block *and*+ # header-shaped per-model keys — and it is where the per-model occurrences of the key+ # the first entry deletes actually were. A richer starting document is the one a+ # later, non-pre-history entry is replayed over.+ reference_document_path=_KIT_CONFIGS_DIR / INFERENCE_DIR_NAME / BACKENDS_DIR_NAME / "portkey.toml",+ kit_template_dir=_KIT_CONFIGS_DIR / INFERENCE_DIR_NAME / BACKENDS_DIR_NAME,+ ),])diff --git a/pipelex/migration/transform_check.py b/pipelex/migration/transform_check.pyindex 9067b265a..d0cb91c63 100644--- a/pipelex/migration/transform_check.py+++ b/pipelex/migration/transform_check.py@@ -25,9 +25,11 @@ keys, edits comments and flips unrelated defaults:an entry that dropped a parent table where it meant to drop one child. A path the new shape nolonger has is not this check's business: the schema removed it, and coverage is what demands theoperation that accounts for it.-- **The last link's migrated document is accepted by the current model**, read the way a user's- file is actually read: beneath the current defaults layer. This is where a wrong value lands —- a remap rewriting to a spelling the schema rejects, a destination the model does not know.+- **The last link's migrated document is accepted at the current schema**, read the way a user's+ file is actually read: beneath the current defaults layer for a surface that has one, on its own for+ a copied document, and validated the way that surface's loader validates. This is where a wrong+ value lands — a remap rewriting to a spelling the schema rejects, a destination the model does not+ know.This is also where a **pre-history** entry is verified, and it is verified by exactly these threeclaims: an entry whose change predates the first fingerprint has no `defaults@N-1` to start from,@@ -68,7 +70,7 @@ from enum import StrEnumfrom pathlib import Pathfrom typing import Any-from pydantic import BaseModel, ConfigDict, Field, ValidationError+from pydantic import BaseModel, ConfigDict, Fieldfrom pipelex.migration.documents import document_paths, path_matches_patternfrom pipelex.migration.engine import apply_ops_over_text@@ -224,30 +226,41 @@ def _check_paths(def _check_the_migrated_document_is_accepted(*, surface: Surface, entry: MigrationEntry, after_text: str, migrated_text: str) -> list[TransformIssue]:- """The last link's output must be something the current model accepts.+ """The last link's output must be something the current schema accepts.- Read the way a user's file is really read: beneath the surface's current defaults layer, not- alone. A migrated file is *expected* to lack whatever the new version added — that is what- makes an additive change absorbable — so validating it on its own would report the defaults- layer doing its job as a failure.+ Read the way a user's file is really read, which is where the surface's own two declarations come+ in. A **layered** surface's file is read beneath the current defaults layer, and a migrated file is+ *expected* to lack whatever the new version added — that is what makes an additive change+ absorbable — so validating it alone would report the defaults layer doing its job as a failure. A+ **copied** document has nothing beneath it, and merging the reference copy of one backend+ definition under another user's would validate a hybrid no machine has.++ What "accepted" means is likewise the surface's to say (`Surface.validate_document`): for a backend+ definition file it is the loader's own merge-then-validate, because neither `[defaults]` nor a model+ table validates alone.Only the last link can be checked this way, and that is not a shortcut: it is the only linkwhose model we still have. Earlier ones are covered by induction, having been the last linkwhen they were authored."""- document: dict[str, Any] = load_toml_from_content(after_text)+ document: dict[str, Any]+ if surface.defaults_layer_kind.is_layered_beneath_the_users_file:+ document = load_toml_from_content(after_text)+ read_as = "read beneath the current defaults"+ else:+ document = {}+ read_as = "read on its own, as a copied document is"deep_update(document, updates=load_toml_from_content(migrated_text))strip_reserved_meta(config_dict=document)- try:- surface.config_model.model_validate(document)- except ValidationError as exc:+ rejection = surface.validate_document(document=document)+ if rejection is not None:return [_issue(surface=surface,kind=TransformIssueKind.MIGRATED_DOCUMENT_REJECTED,message=(f"entry '{entry.id}': {_starting_document_label(entry=entry)}, migrated by this "- f"entry and read beneath the current defaults, is not accepted by {surface.config_model.__name__} — {exc}. "+ f"entry and {read_as}, is not accepted at the current schema — {rejection}. "f"An operation writes a value or a key the schema does not take, so every file it migrates fails to load"),)diff --git a/pipelex/runtime_boot.py b/pipelex/runtime_boot.pyindex e6d305448..abf813c5a 100644--- a/pipelex/runtime_boot.py+++ b/pipelex/runtime_boot.py@@ -37,13 +37,13 @@ orchestration-venue sense and keeps the word for good.import typesimport warningsfrom collections.abc import Callable+from enum import StrEnumfrom pathlib import Pathfrom typing import TYPE_CHECKING, Any, Self, TypeVar, castfrom kajson.class_registry import ClassRegistryfrom kajson.class_registry_abstract import ClassRegistryAbstractfrom kajson.kajson_manager import KajsonManager-from pydantic import ValidationErrorfrom pipelex import logfrom pipelex.base_exceptions import PipelexSetupError@@ -54,6 +54,7 @@ from pipelex.cogt.content_generation.content_generator_protocol import (from pipelex.cogt.content_generation.generated_content_factory import GeneratedContentFactoryfrom pipelex.cogt.exceptions import (InferenceBackendCredentialsError,+ InferenceBackendLibraryError,InferenceBackendLibraryNotFoundError,InferenceBackendLibraryValidationError,ModelDeckNotFoundError,@@ -71,7 +72,7 @@ from pipelex.cogt.models.model_manager_abstract import ModelManagerAbstractfrom pipelex.config import get_configfrom pipelex.core.registry_models import CoreRegistryModelsfrom pipelex.core.stuffs.stuff_template_set import STUFF_TEMPLATE_SET-from pipelex.core.validation import raise_config_setup_error, report_validation_error+from pipelex.core.validation import raise_config_setup_error, report_config_refusalfrom pipelex.graph.mermaidflow.template_set import MERMAID_TEMPLATE_SETfrom pipelex.graph.reactflow.template_set import REACTFLOW_TEMPLATE_SETfrom pipelex.observer.multi_observer import MultiObserver@@ -92,7 +93,7 @@ from pipelex.reporting.reporting_protocol import ReportingProtocolfrom pipelex.runtime_hub import RuntimeHub, set_runtime_hubfrom pipelex.system.configuration.config_loader import CONFIG_REFUSED, config_managerfrom pipelex.system.configuration.config_root import ConfigRoot-from pipelex.system.configuration.config_surface import PIPELEX_CONFIG_SURFACE_ID+from pipelex.system.configuration.config_surface import INFERENCE_BACKEND_CONFIG_SURFACE_ID, PIPELEX_CONFIG_SURFACE_IDfrom pipelex.system.configuration.configs import PipelexConfigfrom pipelex.system.pipelex_service.exceptions import (GatewayTermsNotAcceptedError,@@ -137,6 +138,58 @@ PACKAGE_NAME, PACKAGE_VERSION = get_package_info()_HubSlotImplT = TypeVar("_HubSlotImplT")+BACKEND_LIBRARY_REFUSED: tuple[type[Exception], ...] = (+ InferenceBackendLibraryValidationError,+ InferenceBackendLibraryError,+)+"""Every way the inference backend library can report that its files are not loadable.++Spelled once, and beside `BootComponent` for the same reason that enum exists: what the boot does+with a refusal — name the surface, scan the ledger, drop the start-over tail — is only reachable if+the `except` clause names the class the loader really raises, and a clause that names one class of a+family is a silent hole rather than an error. `InferenceBackendLibraryError` is the one that matters+in practice: it is what the model-spec build raises for an `extra_forbidden` merge and for a+per-model key rejected by name, and it is what boot tolerance re-raises when the ledger cannot+explain the file. It escaped `setup` uncaught until this constant, so the scan could never run for+the one late component that has a ledger. `…ValidationError` is its sibling for the library index+file, `backends.toml`: a backend table whose own fields fail the blueprint — and until it was raised+there, that refusal left the loader as pydantic's bare error, which no clause here named either.+Neither has subclasses, and no other class the chain handles descends from either, so this clause+shadows nothing — `tests/unit/pipelex/test_runtime_boot_stale_backend_error.py` pins both halves+against a refusal the real loader produced.+"""+++class BootComponent(StrEnum):+ """A library whose files can refuse to load after the main configuration is already up.++ The value is how the boot names the component to a user, and the member is what decides whether+ its files belong to a migration surface. Pairing the two here rather than at each `except`+ clause is deliberate: the surface is the difference between telling a user their files are+ *old* and telling them only that they are *wrong*, and a call site passing a bare name is a+ call site that can silently drop it.+ """++ ROUTING_PROFILE_LIBRARY = "routing profile library"+ INFERENCE_BACKEND_LIBRARY = "inference backend library"+ MODEL_DECK = "model deck"++ @property+ def migration_surface_id(self) -> str | None:+ """The migration surface that claims this component's files, or `None` when none does.++ Only the inference backend definitions are one. The model deck is the case worth naming:+ it is package-managed content rather than a schema, so it has a content sync and no+ ledger, and offering it `pipelex migrate` would send a user to a command with nothing to+ do. Routing profiles have neither mechanism.+ """+ match self:+ case BootComponent.INFERENCE_BACKEND_LIBRARY:+ return INFERENCE_BACKEND_CONFIG_SURFACE_ID+ case BootComponent.ROUTING_PROFILE_LIBRARY | BootComponent.MODEL_DECK:+ return None++class RuntimeBoot(metaclass=MetaSingleton):"""Boot the kernel layer: inference, storage, models, telemetry — no method interpreter.@@ -226,30 +279,58 @@ class RuntimeBoot(metaclass=MetaSingleton):log.verbose(f"{PACKAGE_NAME} version {PACKAGE_VERSION} runtime init done")@staticmethod- def _get_config_file_not_found_error_msg(*, component_name: str) -> str:+ def _get_config_file_not_found_error_msg(*, component: BootComponent) -> str:"""Generate error message for missing config files."""- return f"Config files are missing for the {component_name}. Run `pipelex init config` to generate the missing files."+ return f"Config files are missing for the {component}. Run `pipelex init config` to generate the missing files."@staticmethod- def _get_validation_error_msg(*, component_name: str, validation_exc: Exception) -> str:- """Generate error message for invalid config files."""- msg = ""- cause_exc = validation_exc.__cause__- if cause_exc is None:- msg += f"\nUnexpexted error:{cause_exc}"- raise PipelexSetupError(msg) from cause_exc- if not isinstance(cause_exc, ValidationError):- msg += f"\nUnexpexted cause:{cause_exc}"- raise PipelexSetupError(msg) from cause_exc- # No surface is named on purpose: the inference backend library and the model deck are not- # configuration surfaces — they have no ledger and the `migrate` walk does not claim their- # files — so offering a migration remedy for them would send the user to a command with- # nothing to do.- report = report_validation_error(validation_error=cause_exc).message- return f"""{msg}-{report}--Config files are invalid for the {component_name}.+ def _get_validation_error_msg(*, component: BootComponent, validation_exc: Exception) -> str:+ """Generate error message for invalid config files.++ A component whose files belong to a migration surface gets the same treatment the loaders+ reporting through `raise_config_setup_error` get: the refusal is scanned against that+ surface's ledger, so a user whose files are merely behind is told so instead of being left+ with the model's refusal alone.++ **The refusal's own message is the body, and that is the whole reason this goes through+ `report_config_refusal`.** Every loader behind these components says *where* before it says+ what — model, backend and file, or the deck's paths — and then quotes the pydantic analysis;+ a builder that reached through to the pydantic error and translated only that kept the half+ a reader can least act on. It also could not handle the second shape a backend file refuses+ in: a per-model key rejected by name for not being header-shaped carries no pydantic error at+ all, and the old builder offered it no block on its way to saying `Unexpexted error:None`.++ > **The block names every root `pipelex migrate` walks, which can be more than the one the+ > loader read.** `backends_dir_path` resolves to a single directory (a project's `.pipelex/`+ > wins the whole directory over the global one) while the remedy walks both, and the block+ > describes what the remedy would do. Narrowing it to the directory the loader used would+ > need the path off the concrete `ModelManager`, which `ModelManagerAbstract` — a public+ > injection point — deliberately does not expose; see the note at the `models_manager.setup`+ > call below.++ Args:+ component: What refused — the message names it, and it carries the surface.+ validation_exc: The refusal, whose own message is the body of what the user reads.++ Returns:+ The message, carrying the migration paragraph when the scan found one.+ """+ report = report_config_refusal(refusal=validation_exc, surface_id=component.migration_surface_id)+ if report.migration is not None:+ # Regeneration is never offered beside a migration: the block has just promised that+ # `pipelex migrate` keeps every value the file holds, and `pipelex init config` is+ # described in the next breath as resetting them. Whichever a user followed, the other+ # sentence made it the wrong choice.+ return f"""+{report.message}++Config files are invalid for the {component}.+If you need help, drop by our Discord: we're happy to assist: {URLs.discord}.+"""+ return f"""+{report.message}++Config files are invalid for the {component}.You can fix them manually, or run `pipelex init config` to regenerate them.Note that this command resets all config files to their default values.If you need help, drop by our Discord: we're happy to assist: {URLs.discord}.@@ -483,23 +564,23 @@ If you need help, drop by our Discord: we're happy to assist: {URLs.discord}.needs_inference=needs_inference,)except RoutingProfileLibraryNotFoundError as routing_not_found_exc:- msg = self._get_config_file_not_found_error_msg(component_name="routing profile library")+ msg = self._get_config_file_not_found_error_msg(component=BootComponent.ROUTING_PROFILE_LIBRARY)raise PipelexSetupError(msg) from routing_not_found_excexcept InferenceBackendLibraryNotFoundError as backend_not_found_exc:- msg = self._get_config_file_not_found_error_msg(component_name="inference backend library")+ msg = self._get_config_file_not_found_error_msg(component=BootComponent.INFERENCE_BACKEND_LIBRARY)raise PipelexSetupError(msg) from backend_not_found_excexcept ModelDeckNotFoundError as deck_not_found_exc:- msg = self._get_config_file_not_found_error_msg(component_name="model deck")+ msg = self._get_config_file_not_found_error_msg(component=BootComponent.MODEL_DECK)raise PipelexSetupError(msg) from deck_not_found_excexcept RoutingProfileDisabledBackendError as routing_profile_exc:msg = f"Some backend(s) required for a routing profile is not enabled: {routing_profile_exc}"raise PipelexSetupError(msg) from routing_profile_exc- except InferenceBackendLibraryValidationError as backend_validation_exc:- msg = self._get_validation_error_msg(component_name="inference backend library", validation_exc=backend_validation_exc)+ except BACKEND_LIBRARY_REFUSED as backend_validation_exc:+ msg = self._get_validation_error_msg(component=BootComponent.INFERENCE_BACKEND_LIBRARY, validation_exc=backend_validation_exc)raise PipelexSetupError(msg) from backend_validation_excexcept ModelDeckValidationError as deck_validation_exc:- msg = self._get_validation_error_msg(component_name="model deck", validation_exc=deck_validation_exc)+ msg = self._get_validation_error_msg(component=BootComponent.MODEL_DECK, validation_exc=deck_validation_exc)msg += "\n\nIf you added your own config files to the model deck then you'll have to change them manually."raise PipelexSetupError(msg) from deck_validation_excdiff --git a/pipelex/system/configuration/config_surface.py b/pipelex/system/configuration/config_surface.pyindex 0ab49a3bf..657ee0282 100644--- a/pipelex/system/configuration/config_surface.py+++ b/pipelex/system/configuration/config_surface.py@@ -1,21 +1,27 @@"""Read-side helpers shared by every configuration surface.A **configuration surface** is one family of user-owned TOML files with one schema and one-migration ledger — today `pipelex.toml` and its tiers, `telemetry.toml`, and-`pipelex_service.toml`. Two things are the same for all three on the read path, and this module-is the one place that knows about either.+migration ledger — `pipelex.toml` and its tiers, `telemetry.toml`, `pipelex_service.toml`, and the+inference backend definitions under `inference/backends/`. Two things are the same for all of them on+the read path, and this module is the one place that knows about either.**The reserved `[meta]` table.** The strip lives here and deliberately **not** in the generic TOML-reader (`pipelex.tools.misc.toml_utils`), which also reads `.mthds` files, backend definitions and-the kit index — none of which reserve that key, and all of which should keep rejecting it.+reader (`pipelex.tools.misc.toml_utils`), which also reads `.mthds` files and the kit index —+neither of which reserves that key, and both of which should keep rejecting it. Backend definitions+are the one surface whose *loader* still reads through that generic reader rather than through this+module, so a backend file carrying `[meta]` would be refused as an unknown model table even though+the migration engine reads the key there. Nothing writes it, on any surface, so this is latent+rather than live — but it is the asymmetry to close if anything ever starts stamping a version into+a file.**Boot tolerance.** A stale configuration should warn rather than stop the world, but only whenthe ledger can explain it. Each surface's loader validates as it always did and, only when thatfails, asks `replay_surface_files_in_memory` for the same files carried forward — writing nothing,because nothing writes but the explicit `migrate` command. What the loaders do *not* share is thestep between their merge and their validate: one deep-merges programmatic overrides, one-substitutes `${VAR}` placeholders, one does nothing at all. So the shared part is the failure path-alone, and each loader re-runs its own steps over what comes back.+substitutes `${VAR}` placeholders, one splits header-shaped keys off each model table, one does+nothing at all. So the shared part is the failure path alone, and each loader re-runs its own steps+over what comes back.See `docs/migration-ledger.md` → "Schema versions, and why every run replays everything" and"Boot tolerance".@@ -97,13 +103,14 @@ def version_declared_below_the_floor(*, ledger: MigrationLedger, config_dict: direturn declared-# The surface ids, spelled once. The migration registry names the same three, and every ledger+# The surface ids, spelled once. The migration registry names the same ones, and every ledger# file's `[surface] id` must agree with them — a boot-tolerance retry that asked for a surface id# nothing ships would raise rather than tolerate, so the constant is the seam that keeps the# loader and the registry from drifting apart on a string literal.PIPELEX_CONFIG_SURFACE_ID = "pipelex-config"TELEMETRY_CONFIG_SURFACE_ID = "telemetry-config"PIPELEX_SERVICE_CONFIG_SURFACE_ID = "pipelex-service-config"+INFERENCE_BACKEND_CONFIG_SURFACE_ID = "inference-backend"class ReplayedSurface(NamedTuple):@@ -183,12 +190,36 @@ def replay_surface_files_in_memory(*, surface_id: str, paths: Sequence[Path]) ->return ReplayedSurface(config_dict=merged, plans=plans)-def _is_within(*, path: Path, directories: Sequence[Path]) -> bool:- """Whether a file sits directly in one of these directories.+def _subdirectory_owned_by(*, surface_id: str) -> Path:+ """The directory this surface's files sit in, relative to a configuration directory.- Directly, not underneath: the walk is not recursive, so a file in a subdirectory of a- configuration directory is as far out of `pipelex migrate`'s reach as one on the other side of- the disk.+ Read from the **ledger**, not from the surface registry, and the reason is layering: the+ registry imports every configuration model, and this module sits in `runtime_hub`'s import+ closure — the kernel layer, which loads none of them. A ledger is the one half of the pair a+ loader may reach, and `make check-migration-schemas` cross-checks it against the registry, so+ reading it here is not a second derivation of the walk but the same one read from its other+ end.++ A ledger that will not load answers "directly in a configuration directory", which is what+ every surface but one says anyway. Nothing here is allowed to become the failure the user+ sees: they have a configuration error in front of them, and a packaging problem of ours is+ loud in `make check-ledger` and in `pipelex migrate` instead.+ """+ try:+ ledger = load_ledger_cached(migration_dir=packaged_migration_dir(), surface_id=surface_id)+ except MigrationLedgerError:+ return Path()+ return Path(ledger.surface.subdirectory)+++def _is_within(*, plan: MigrationPlan, directories: Sequence[Path]) -> bool:+ """Whether `pipelex migrate` would reach this plan's file, given the directories it walks.++ The walk is one level deep, but not always at the top: it is **the directory this file's own+ surface owns** under each configuration directory. So a `telemetry.toml` under+ `inference/backends/` is out of reach — that directory belongs to another surface — while an+ `openai.toml` in it is in reach, and a file under `inference/deck/` is out of reach whatever+ it is called, because no surface owns that directory and the walk never enters it.**The parent is resolved; the file is not**, and that asymmetry is the whole of it. Resolvingthe *directory* is what the walk itself effectively does — a caller's `config_dir=` may be the@@ -199,11 +230,12 @@ def _is_within(*, path: Path, directories: Sequence[Path]) -> bool:when the walk enumerates the link by `iterdir` and writes straight through it. Where the file*points* is not where the command looks for it."""+ subdirectory = _subdirectory_owned_by(surface_id=plan.surface_id)try:- parent = path.parent.resolve()+ parent = plan.file_path.parent.resolve()except OSError:return False- return any(parent == directory.resolve() for directory in directories)+ return any(parent == (directory / subdirectory).resolve() for directory in directories)def _quoted_files(*, plans: Sequence[MigrationPlan]) -> str:@@ -227,8 +259,8 @@ def stale_configuration_warning(*, plans: Sequence[MigrationPlan], walked_dirs:directory is theirs to update, and `--config-dir` is deliberately not the answer."""stale = [plan for plan in plans if not plan.is_clean]- in_reach = [plan for plan in stale if _is_within(path=plan.file_path, directories=walked_dirs)]- out_of_reach = [plan for plan in stale if not _is_within(path=plan.file_path, directories=walked_dirs)]+ in_reach = [plan for plan in stale if _is_within(plan=plan, directories=walked_dirs)]+ out_of_reach = [plan for plan in stale if not _is_within(plan=plan, directories=walked_dirs)]carried = sorted({step.title for plan in stale for step in plan.steps})sentences = [f"Your configuration is out of date, and pipelex read it as if it had been migrated: {_quoted_files(plans=stale)}."]if carried:diff --git a/pipelex/system/pipelex_service/pipelex_service_config.py b/pipelex/system/pipelex_service/pipelex_service_config.pyindex a62d66ac1..8ede1a6c3 100644--- a/pipelex/system/pipelex_service/pipelex_service_config.py+++ b/pipelex/system/pipelex_service/pipelex_service_config.py@@ -80,6 +80,14 @@ def is_pipelex_gateway_enabled(backends_file_path: Path | None = None) -> bool:This reads the backends.toml file directly without loading the full backend library.+ **It must read `enabled` exactly as the library loader does** — the raw value's truthiness, and+ enabled when the key is absent — because the two are read over the same file and the boot acts+ on the answer here: it fetches the gateway's model specs only when this says enabled, then hands+ them to a loader that decides for itself which backends are enabled. Reading the literal `true`+ here while the loader read truthiness left `enabled = 1` in a state neither could name: no specs+ fetched, backend loaded — refused as *"model specs were not provided"* on a strict boot, silently+ dropped from the deck on a lenient one.+Args:backends_file_path: Explicit path to the ``backends.toml`` file to inspect. When``None`` (default), uses the layered/project-preferred path from@@ -100,5 +108,4 @@ def is_pipelex_gateway_enabled(backends_file_path: Path | None = None) -> bool:return Falsegateway_config_dict = cast("dict[str, Any]", gateway_config)- enabled_value = gateway_config_dict.get("enabled", True)- return enabled_value is True+ return bool(gateway_config_dict.get("enabled", True))diff --git a/tests/e2e/pipelex/cli/test_migrate_commands.py b/tests/e2e/pipelex/cli/test_migrate_commands.pyindex f0fe0de32..f12f4caea 100644--- a/tests/e2e/pipelex/cli/test_migrate_commands.py+++ b/tests/e2e/pipelex/cli/test_migrate_commands.py@@ -14,7 +14,7 @@ fails ``TelemetryConfig``'s ``extra="forbid"`` today, in the field.**Two surfaces are planted here, and the second one is the whole main configuration.**``telemetry-config`` is the small specimen — one flat document, nested — and it is what most of-this module runs on because it is cheap and it shipped first. ``pipelex-config@2``, the+this module runs on because it is cheap and it shipped first. ``pipelex-config@3``, theconfiguration reshape, is the other size of thing: it renames the root tables of the file everyboot reads, so the machine it migrates is every existing installation. Its fixture is``goldens/pipelex-config/[email protected]`` — the packaged ``pipelex.toml`` as it stood at schema 1@@ -45,7 +45,12 @@ from pipelex.migration.backup import existing_backups_offrom pipelex.migration.goldens import defaults_golden_path, pre_history_document_pathfrom pipelex.migration.ledger import packaged_migration_dirfrom pipelex.migration.surfaces import build_config_surface_registry-from pipelex.system.configuration.config_surface import PIPELEX_CONFIG_SURFACE_ID, TELEMETRY_CONFIG_SURFACE_ID+from pipelex.system.configuration.config_loader import BACKENDS_DIR_NAME, BACKENDS_FILE_NAME, INFERENCE_DIR_NAME+from pipelex.system.configuration.config_surface import (+ INFERENCE_BACKEND_CONFIG_SURFACE_ID,+ PIPELEX_CONFIG_SURFACE_ID,+ TELEMETRY_CONFIG_SURFACE_ID,+)from pipelex.tools.misc.toml_utils import load_toml_from_pathfrom tests.e2e.agent_cli.conftest import REPO_ROOT@@ -91,6 +96,40 @@ default_output_dir = "build""""+# The templating section as schema 1 spells it, and as the release before it did. `#1104` renamed+# the section and its one key and dropped the per-target map, and the pre-history entry+# `pipelex-config@2` is what carries a file written before that. The pair is used to rewind+# `[email protected]` one release further back — a text swap rather than a transcribed document, so+# that everything the reshape entry is about goes on being read live and only the half this is+# about is stated here. A swap that stopped matching would leave a fixture that is not old at all,+# which is why the planting asserts on the result.+TEMPLATING_SECTION_AT_SCHEMA_1 = """\+[pipelex.templating_config]+default_templating_style = { tag_style = "xml" }+"""++PROMPTING_SECTION_BEFORE_SCHEMA_1 = """\+[pipelex.prompting_config]+default_prompting_style = { tag_style = "xml" }++[pipelex.prompting_config.prompting_styles]+openai = { tag_style = "ticks" }+anthropic = { tag_style = "xml" }+mistral = { tag_style = "square_brackets" }+gemini = { tag_style = "xml" }+"""++# The key `#1104` deleted from the model-spec blueprint and from every backend file we ship, and a+# value no ledger sentence contains — so the reports can be searched for it, and finding it would be+# a value read from a user's file having reached a channel that must never carry one.+RETIRED_BACKEND_KEY = "prompting_target"+PLANTED_BACKEND_VALUE = "a_prompting_target_no_report_may_ever_render"++# How `stale_configuration_warning` opens. Matched in full rather than on a fragment: the same boot+# also prints a deck-staleness notice, and "out of date" appears in both.+STALE_CONFIGURATION_OPENING = "Your configuration is out of date"++def _old_shape_telemetry_document() -> str:"""The flat pre-history document the shipped entry is about, read from the package."""path = pre_history_document_path(migration_dir=packaged_migration_dir(), surface_id=TELEMETRY_CONFIG_SURFACE_ID, schema_version=2)@@ -105,6 +144,11 @@ def _pre_reshape_pipelex_config_document() -> str:does not. Read live rather than transcribed here, for the same reason the telemetry fixture is:a transcribed one would eventually describe a shape the shipped ledger no longer migrates, andwould go on passing.++ It is read at schema 1 by name and not at the version the reshape now starts from, which is one+ higher: the pre-history entry inserted below the reshape changed nothing in the models, so its+ reference document is a byte copy of this one. Naming the version where the document's shape was+ actually cut is the spelling that says what this is, rather than one inherited from a copy."""path = defaults_golden_path(migration_dir=packaged_migration_dir(), surface_id=PIPELEX_CONFIG_SURFACE_ID, schema_version=1)return path.read_text(encoding="utf-8")@@ -114,7 +158,7 @@ def _todays_pipelex_config_document() -> dict[str, Any]:"""What the package's own `pipelex.toml` says now — the shape a migration has to land on.Read through the surface rather than by path, so this tracks wherever the packaged document- lives, and read as the live file rather than as `[email protected]`: the golden is a snapshot of+ lives, and read as the live file rather than as the head golden: that golden is a snapshot ofthis, and comparing against the snapshot would leave a gap exactly the width of a stale one."""registry = build_config_surface_registry()@@ -168,6 +212,76 @@ def _plant_a_pre_reshape_machine(*, hermetic_home: Path) -> tuple[Path, Path, Pareturn hermetic_home / PROJECT_DIR_NAME, global_file, project_file+def _plant_a_pre_prompting_style_machine(*, hermetic_home: Path) -> tuple[Path, Path]:+ """One release further back than the machine above: a file that still names prompting styles.++ Two entries have to run on it, in order, and the order is the whole point — the first addresses+ `pipelex.prompting_config` at the spelling it had *before* the reshape renames `[pipelex]`, so a+ file that met only the reshape would arrive at `[interpreter.prompting_config]`, which nothing+ reads and the model refuses.++ Only the global tier is planted. The project tier beside it is the reshape's specimen and it has+ nothing to say about prompting styles; a second file carrying the same two tables would prove the+ same thing twice.++ Returns the project directory and the global file.+ """+ at_schema_1 = _pre_reshape_pipelex_config_document()+ older = at_schema_1.replace(TEMPLATING_SECTION_AT_SCHEMA_1, PROMPTING_SECTION_BEFORE_SCHEMA_1)+ if older == at_schema_1:+ msg = f"the schema-1 templating section is no longer spelled the way this fixture rewinds:\n{TEMPLATING_SECTION_AT_SCHEMA_1}"+ raise AssertionError(msg)++ global_file = hermetic_home / ".pipelex" / "pipelex.toml"+ global_file.write_text(older, encoding="utf-8")++ _project_config_dir(hermetic_home=hermetic_home)+ return hermetic_home / PROJECT_DIR_NAME, global_file+++def _backends_dir(*, hermetic_home: Path) -> Path:+ return hermetic_home / ".pipelex" / INFERENCE_DIR_NAME / BACKENDS_DIR_NAME+++def _plant_a_stale_backend_directory(*, hermetic_home: Path) -> tuple[Path, Path]:+ """The third surface's machine: backend definitions written before `#1104` deleted a key.++ `hermetic_home` seeds `inference/backends/` from the kit, which is exactly what `pipelex init`+ puts on a machine — so ageing it is a matter of putting `prompting_target` back where an+ installation from before this release still has it: in one file's `[defaults]`, where it breaks+ every model of that file at once, and on another file's model tables, where each is rejected by+ name. Two files rather than one because the surface claims a directory, and a run that reached+ only the first would come back visibly short.++ The value planted is a marker no ledger sentence contains, which is what lets the reports be+ checked for it. Both plantings raise if their anchor is gone, so a kit file that stops carrying+ it turns this red rather than quietly ageing nothing.++ Returns the two stale files.+ """+ backends_dir = _backends_dir(hermetic_home=hermetic_home)+ openai_file = backends_dir / "openai.toml"+ portkey_file = backends_dir / "portkey.toml"++ in_defaults = openai_file.read_text(encoding="utf-8")+ aged_defaults = in_defaults.replace("[defaults]\n", f'[defaults]\n{RETIRED_BACKEND_KEY} = "{PLANTED_BACKEND_VALUE}"\n', 1)+ if aged_defaults == in_defaults:+ msg = f"the kit's openai.toml no longer has a [defaults] block to age: {openai_file}"+ raise AssertionError(msg)+ openai_file.write_text(aged_defaults, encoding="utf-8")++ per_model = portkey_file.read_text(encoding="utf-8")+ aged_models = per_model+ for table_header in ("[gpt-4o]", '["gemini-2.5-pro"]'):+ aged_models = aged_models.replace(f"{table_header}\n", f'{table_header}\n{RETIRED_BACKEND_KEY} = "{PLANTED_BACKEND_VALUE}"\n', 1)+ if aged_models == per_model:+ msg = f"the kit's portkey.toml no longer has the model tables this fixture ages: {portkey_file}"+ raise AssertionError(msg)+ portkey_file.write_text(aged_models, encoding="utf-8")++ return openai_file, portkey_file++def _run(*, args: list[str], env: dict[str, str], cwd: Path, answers: str | None = None) -> subprocess.CompletedProcess[str]:# A wide terminal, because the human CLI renders through Rich and Rich wraps at the console# width. A temp-directory path is long enough to be broken across two lines at the default 80,@@ -355,7 +469,7 @@ class TestAPreReshapeMachine:"""The machine this release actually meets: a main configuration written before the reshape.Everything above is planted on `telemetry-config`, whose entry nests one flat document — a- small change, and one that shipped before the reshape did. `pipelex-config@2` is the other+ small change, and one that shipped before the reshape did. `pipelex-config@3` is the othersize of thing. It renames the root tables of the file every boot reads, so the machine itmigrates is not a corner case but every installation that predates this release, and the twotiers planted here are the two a real one has: the global `~/.pipelex/pipelex.toml` an install@@ -375,8 +489,9 @@ class TestAPreReshapeMachine:"""The whole loop through `pipelex migrate`, with the strong assertion on the global file.Migrated, it says exactly what the package's own `pipelex.toml` says today.- `make check-migration-schemas` already proves the *engine* turns `defaults@1` into- `defaults@2`; what is proved here is that the *command* does — walking two directories,+ `make check-migration-schemas` already proves the *engine* turns the reference document the+ reshape starts from into the one it lands on; what is proved here is that the *command*+ does — walking two directories,reading a user's file off a disk, writing it back and leaving the original beside it —which no golden can show.@@ -565,6 +680,82 @@ class TestAPreReshapeMachine:assert planted in global_file.read_text(encoding="utf-8"), "nothing was written, so the value is still there"+class TestAPrePromptingStyleMachine:+ """Two entries on one file, in the order that makes the first one's paths exist.++ `pipelex-config@2` is the other half of the templating change: `#1104` deleted+ `prompting_config` from the model and from the packaged file, and — because `pipelex init` never+ overwrites — left it behind on every machine that had one. It is a *pre-history* entry inserted+ below the reshape rather than appended above it, because it addresses `pipelex.prompting_config`,+ the spelling that only exists while `[pipelex]` is still called `[pipelex]`. Appended after the+ reshape it would find nothing, and the file would come out of the run carrying+ `[interpreter.prompting_config]` — a table the model refuses, on a machine the tool has just+ reported as migrated.++ So the property here is not "the entry works" but "the two entries compose": one machine, one+ run, and a file that arrives at exactly what the package ships today.+ """++ def test_the_human_command_carries_a_pre_prompting_style_machine_onto_todays_shape(+ self,+ hermetic_home: Path,+ offline_subprocess_env: dict[str, str],+ ) -> None:+ project_dir, global_file = _plant_a_pre_prompting_style_machine(hermetic_home=hermetic_home)+ original = global_file.read_text(encoding="utf-8")+ planted: dict[str, Any] = load_toml_from_path(path=global_file)+ assert "prompting_config" in planted["pipelex"], "the fixture has to start out old, or nothing below is measured"++ _assert_boots(env=offline_subprocess_env, cwd=project_dir)++ migrated = _run(args=[str(PIPELEX_BIN), "--no-logo", "migrate", "--yes"], env=offline_subprocess_env, cwd=project_dir)++ assert migrated.returncode == 0, migrated.stdout + migrated.stderr+ assert str(global_file) in migrated.stdout + migrated.stderr++ # The whole document, against the live packaged one: the tuned per-target map is gone, the+ # default it sat beside travels to where the reshape puts templating, and nothing else about+ # a file that crossed two entries in one run came out different.+ assert load_toml_from_path(path=global_file) == _todays_pipelex_config_document()++ backups = existing_backups_of(path=global_file)+ assert len(backups) == 1, f"expected exactly one backup of {global_file}, found {backups}"+ assert backups[0].read_text(encoding="utf-8") == original++ _assert_boots(env=offline_subprocess_env, cwd=project_dir)++ def test_the_agent_loop_names_both_entries_for_the_one_file(+ self,+ hermetic_home: Path,+ offline_subprocess_env: dict[str, str],+ ) -> None:+ """A plan that named only the reshape would be a plan a consumer could not act on.++ The order of the steps is the order they ran in, and it is the load-bearing half: a reader+ of this plan has to be able to see that the prompting section was dealt with while it was+ still under `[pipelex]`.+ """+ project_dir, global_file = _plant_a_pre_prompting_style_machine(hermetic_home=hermetic_home)++ planned = _run(+ args=[str(PIPELEX_AGENT_BIN), "migrate", "--dry-run", "--format", "json"],+ env=offline_subprocess_env,+ cwd=project_dir,+ )+ assert planned.returncode == 0, planned.stderr+ plan: dict[str, Any] = json.loads(planned.stdout)++ assert plan["is_clean"] is False+ assert plan["needs_attention"] is False, "every path of a pre-prompting-style file is one the two entries explain"+ stepped = {plan_dict["file_path"]: [step["title"] for step in plan_dict["steps"]] for plan_dict in plan["plans"] if plan_dict["steps"]}+ assert stepped == {+ str(global_file): [+ "Prompting styles become a templating default",+ "The configuration reshape: one scheme for the root",+ ]+ }++class TestTheBootstrapPath:"""The property that makes this command reachable at all: it runs when nothing else does.@@ -888,6 +1079,207 @@ class TestAStaleTelemetryFileIsMigratedNotReset:assert "pipelex init telemetry" not in printed+class TestAStaleBackendDirectory:+ """The fourth surface, end to end: `inference/backends/*.toml` on a machine set up before `#1104`.++ Everything else in this module is planted on a file that sits *directly* in a configuration+ directory. These files do not: they are the reason the walk learned to enter a subdirectory at+ all, and the reason a file is claimed by `(directory, name)` rather than by name — the kit ships+ a `pipelex_gateway.toml` in there, which the main configuration's `pipelex_*.toml` glob would+ otherwise have claimed. It is present here for that reason, untouched, and the assertions say so.++ What a boot *says* is not asserted here and cannot be: the probe is `pipelex-agent`, which cuts+ logging off process-wide as its first act. The warning is asserted through the human binary+ below, and per surface in `tests/unit/pipelex/cogt/model_backends/test_backend_boot_tolerance.py`.+ """++ def _untouched_neighbours(self, *, hermetic_home: Path, migrated: tuple[Path, ...]) -> dict[Path, bytes]:+ """Everything in and beside the directory that no migration may rewrite.++ Excluded by name rather than by extension, and the difference is the whole point of this+ class: the kit ships `pipelex_gateway.toml` in this directory, which the main configuration's+ `pipelex_*.toml` glob would claim by name alone, so it is the one file a walk that forgot+ about directories would rewrite. An extension filter dropped it — and every other `.toml` in+ here — out of the snapshot, leaving the class's own claim unasserted. Only the files the+ fixture aged are excluded now. `inference/backends.toml` is added back from one level up,+ which is a different claim entirely: it is a table per backend rather than a table per model,+ and `#1104` never touched it.++ The snapshot is taken before the migration runs, so the `.bak` siblings it writes never enter+ it and cannot break the comparison.+ """+ backends_dir = _backends_dir(hermetic_home=hermetic_home)+ neighbours = [path for path in sorted(backends_dir.iterdir()) if path not in migrated]+ neighbours.append(hermetic_home / ".pipelex" / INFERENCE_DIR_NAME / BACKENDS_FILE_NAME)+ return {path: path.read_bytes() for path in neighbours}++ def test_the_human_command_repairs_the_directory_in_place_and_leaves_one_backup_per_file(+ self,+ hermetic_home: Path,+ offline_subprocess_env: dict[str, str],+ ) -> None:+ openai_file, portkey_file = _plant_a_stale_backend_directory(hermetic_home=hermetic_home)+ originals = {path: path.read_text(encoding="utf-8") for path in (openai_file, portkey_file)}+ neighbours = self._untouched_neighbours(hermetic_home=hermetic_home, migrated=(openai_file, portkey_file))++ _assert_boots(env=offline_subprocess_env, cwd=hermetic_home)++ migrated = _run(args=[str(PIPELEX_BIN), "--no-logo", "migrate", "--yes"], env=offline_subprocess_env, cwd=hermetic_home)++ assert migrated.returncode == 0, migrated.stdout + migrated.stderr+ report = migrated.stdout + migrated.stderr+ assert str(openai_file) in report+ assert str(portkey_file) in report++ for path, original in originals.items():+ assert RETIRED_BACKEND_KEY not in path.read_text(encoding="utf-8"), f"{path} still carries the retired key"+ backups = existing_backups_of(path=path)+ assert len(backups) == 1, f"expected exactly one backup of {path}, found {backups}"+ assert backups[0].read_text(encoding="utf-8") == original+ assert {path: path.read_bytes() for path in neighbours} == neighbours++ _assert_boots(env=offline_subprocess_env, cwd=hermetic_home)++ def test_a_second_run_writes_nothing_and_leaves_no_second_backup(+ self,+ hermetic_home: Path,+ offline_subprocess_env: dict[str, str],+ ) -> None:+ """Replay neutrality over a directory, which is where it is easiest to get wrong.++ Every file of the surface is replayed on every run — twenty of them here — so a run that+ was not neutral would show up as twenty backups of files nothing was ever wrong with.+ """+ openai_file, portkey_file = _plant_a_stale_backend_directory(hermetic_home=hermetic_home)++ first = _run(args=[str(PIPELEX_AGENT_BIN), "migrate", "--yes", "--format", "json"], env=offline_subprocess_env, cwd=hermetic_home)+ assert first.returncode == 0, first.stderr+ assert json.loads(first.stdout)["summary"]["files_written"] == 2++ after_first = {path: path.read_bytes() for path in sorted(_backends_dir(hermetic_home=hermetic_home).iterdir())}++ second = _run(args=[str(PIPELEX_AGENT_BIN), "migrate", "--yes", "--format", "json"], env=offline_subprocess_env, cwd=hermetic_home)+ assert second.returncode == 0, second.stderr+ outcome: dict[str, Any] = json.loads(second.stdout)+ assert outcome["is_clean"] is True+ assert outcome["summary"]["files_written"] == 0+ assert {path: path.read_bytes() for path in sorted(_backends_dir(hermetic_home=hermetic_home).iterdir())} == after_first+ for path in (openai_file, portkey_file):+ assert len(existing_backups_of(path=path)) == 1++ def test_the_agent_loop_names_both_files_the_entry_and_its_operation(+ self,+ hermetic_home: Path,+ offline_subprocess_env: dict[str, str],+ ) -> None:+ openai_file, portkey_file = _plant_a_stale_backend_directory(hermetic_home=hermetic_home)+ original = openai_file.read_text(encoding="utf-8")++ planned = _run(+ args=[str(PIPELEX_AGENT_BIN), "migrate", "--dry-run", "--format", "json"],+ env=offline_subprocess_env,+ cwd=hermetic_home,+ )++ assert planned.returncode == 0, planned.stderr+ plan: dict[str, Any] = json.loads(planned.stdout)+ assert plan["applied"] is False+ assert plan["needs_attention"] is False, "the ledger explains every key these files carry"+ assert plan["summary"]["files_changed"] == 2++ changed = {plan_dict["file_path"]: plan_dict for plan_dict in plan["plans"] if plan_dict["steps"]}+ assert set(changed) == {str(openai_file), str(portkey_file)}+ for plan_dict in changed.values():+ assert [step["entry_id"] for step in plan_dict["steps"]] == [f"{INFERENCE_BACKEND_CONFIG_SURFACE_ID}@2"]+ ops = [op for step in plan_dict["steps"] for op in step["applied_ops"]]+ assert [(op["kind"], op["key"]) for op in ops] == [("delete_key", RETIRED_BACKEND_KEY)]++ assert openai_file.read_text(encoding="utf-8") == original, "a plan is not a write"+ assert existing_backups_of(path=openai_file) == []++ def test_no_channel_renders_the_value_the_files_carry(+ self,+ hermetic_home: Path,+ offline_subprocess_env: dict[str, str],+ ) -> None:+ """The standing rule, on the surface whose files hold a value per model rather than per key.++ The operation is a delete, so what a report *could* leak is the value it is deleting — which+ is why the fixture plants a marker rather than a plausible-looking one: an assertion that+ passed because the value happened to look like ledger text would prove nothing.+ """+ _plant_a_stale_backend_directory(hermetic_home=hermetic_home)++ channels = [+ _run(args=[str(PIPELEX_BIN), "--no-logo", "migrate", "--dry-run"], env=offline_subprocess_env, cwd=hermetic_home),+ _run(+ args=[str(PIPELEX_AGENT_BIN), "migrate", "--dry-run", "--format", "json"],+ env=offline_subprocess_env,+ cwd=hermetic_home,+ ),+ _run(+ args=[str(PIPELEX_AGENT_BIN), "migrate", "--dry-run", "--format", "markdown"],+ env=offline_subprocess_env,+ cwd=hermetic_home,+ ),+ _run(args=[str(PIPELEX_BIN), "--no-logo", "migrate", "--yes"], env=offline_subprocess_env, cwd=hermetic_home),+ ]+ for channel in channels:+ assert PLANTED_BACKEND_VALUE not in channel.stdout+ assert PLANTED_BACKEND_VALUE not in channel.stderr++ def test_the_doctor_row_names_the_backend_files_and_goes_quiet_once_they_are_migrated(+ self,+ hermetic_home: Path,+ offline_subprocess_env: dict[str, str],+ ) -> None:+ """The one channel a machine has for a pending migration, now that a stale directory boots."""+ openai_file, portkey_file = _plant_a_stale_backend_directory(hermetic_home=hermetic_home)++ checked = _run(args=[str(PIPELEX_AGENT_BIN), "doctor", "--format", "json"], env=offline_subprocess_env, cwd=hermetic_home)+ row: dict[str, Any] = json.loads(checked.stdout)["checks"]["pending_migrations"]++ assert row["finding"] == "pending"+ assert sorted(row["migratable_files"]) == sorted([str(openai_file), str(portkey_file)])++ _run(args=[str(PIPELEX_AGENT_BIN), "migrate", "--yes", "--format", "json"], env=offline_subprocess_env, cwd=hermetic_home)++ rechecked = _run(args=[str(PIPELEX_AGENT_BIN), "doctor", "--format", "json"], env=offline_subprocess_env, cwd=hermetic_home)+ assert json.loads(rechecked.stdout)["checks"]["pending_migrations"]["finding"] == "up_to_date"++ def test_a_person_booting_the_stale_machine_is_told_and_stops_being_told_once_it_is_migrated(+ self,+ hermetic_home: Path,+ offline_subprocess_env: dict[str, str],+ ) -> None:+ """The warning itself, through the human binary — the half `pipelex-agent` cannot show.++ `show backends` is the cheapest human command that performs the inference half of a boot,+ which is where the backend library is loaded and where a directory the ledger carried+ forward is reported. The second run is what makes it a warning about staleness rather than+ a line the command always prints.+ """+ openai_file, portkey_file = _plant_a_stale_backend_directory(hermetic_home=hermetic_home)++ booted = _run(args=[str(PIPELEX_BIN), "--no-logo", "show", "backends"], env=offline_subprocess_env, cwd=hermetic_home)++ assert booted.returncode == 0, booted.stdout + booted.stderr+ printed = booted.stdout + booted.stderr+ # The warning's own opening words, not a phrase it shares with the deck-staleness notice+ # this same boot prints — "out of date" alone would match either of them.+ assert STALE_CONFIGURATION_OPENING in printed+ assert "pipelex migrate" in printed+ assert PLANTED_BACKEND_VALUE not in printed+ for path in (openai_file, portkey_file):+ assert existing_backups_of(path=path) == [], "a boot writes nothing"++ _run(args=[str(PIPELEX_AGENT_BIN), "migrate", "--yes", "--format", "json"], env=offline_subprocess_env, cwd=hermetic_home)++ rebooted = _run(args=[str(PIPELEX_BIN), "--no-logo", "show", "backends"], env=offline_subprocess_env, cwd=hermetic_home)+ assert rebooted.returncode == 0, rebooted.stdout + rebooted.stderr+ assert STALE_CONFIGURATION_OPENING not in rebooted.stdout + rebooted.stderr++class TestTheDoctorReportsAndRepairsAStaleMachine:"""The row that tells a machine a migration is pending, and the fix mode that runs it.diff --git a/tests/unit/pipelex/cli/test_migrate_op_wording.py b/tests/unit/pipelex/cli/test_migrate_op_wording.pynew file mode 100644index 000000000..2d75ee021--- /dev/null+++ b/tests/unit/pipelex/cli/test_migrate_op_wording.py@@ -0,0 +1,88 @@+"""One migration operation, in words — the sentence both `migrate` commands render.++`describe_op` is shared: the human CLI prints it under each step and the agent CLI puts it in its+Markdown. So it is the one place where the standing rule is at its most exposed —++> No value read from a user's file is ever rendered.++— and the one place it is cheap to hold, because the function is given an operation and nothing+else. Every part of every sentence below comes from the ledger: a kind, a path, a key, and the+spellings a remap names.++The wildcard cases are the reason this module exists. `inference-backend@2` is the first shipped+entry to address one — `table_path = ["*"]`, meaning every root table of a backend file — and the+literal rendering of that is `from '*'`, which names nothing a reader can find in their file.+"""++from pipelex.cli.commands.migrate_cmd import describe_op+from pipelex.suggested_fix import WILDCARD_SEGMENT, DeleteKeyOp, DeleteTableOp, MoveKeyOp, RemapValueOp, RenameTableKeyOp+++class TestAnOperationInWords:+ def test_a_key_deleted_from_a_named_table(self) -> None:+ described = describe_op(op=DeleteKeyOp(table_path=["runtime", "log"], key="old_setting"))++ assert described == "deleted 'old_setting' from 'runtime.log'"++ def test_a_key_deleted_from_the_document_root(self) -> None:+ """A surface whose entries live at the top level addresses the root with an empty path."""+ described = describe_op(op=DeleteKeyOp(table_path=[], key="telemetry_mode"))++ assert described == "deleted 'telemetry_mode' from the document root"++ def test_a_key_deleted_from_every_table_of_an_open_document(self) -> None:+ """The shape `inference-backend@2` ships: one operation over every root table of the file.++ A backend file is `[defaults]` plus one table per model, and the entry removes the key from+ all of them — so the sentence has to say *every*, or a reader looks for a table called `*`.+ """+ described = describe_op(op=DeleteKeyOp(table_path=[WILDCARD_SEGMENT], key="prompting_target"))++ assert described == "deleted 'prompting_target' from every table of the document"++ def test_a_key_deleted_from_every_entry_of_a_named_open_node(self) -> None:+ """The other place a wildcard comes from: a `dict[str, X]` field inside a document."""+ described = describe_op(op=DeleteKeyOp(table_path=["backends", WILDCARD_SEGMENT], key="prompting_target"))++ assert described == "deleted 'prompting_target' from every entry of 'backends'"++ def test_a_wildcard_that_is_not_the_last_segment_keeps_its_literal_spelling(self) -> None:+ """No ledger addresses this shape, and inventing a phrase for it would be guessing."""+ described = describe_op(op=DeleteKeyOp(table_path=["backends", WILDCARD_SEGMENT, "limits"], key="prompting_target"))++ assert described == "deleted 'prompting_target' from 'backends.*.limits'"++ def test_a_deleted_table(self) -> None:+ described = describe_op(op=DeleteTableOp(table_path=["pipelex", "prompting_config"]))++ assert described == "deleted the table 'pipelex.prompting_config'"++ def test_a_renamed_key(self) -> None:+ described = describe_op(op=RenameTableKeyOp(table_path=["runtime", "log"], key="old_name", new_key="new_name"))++ assert described == "renamed 'old_name' to 'new_name' in 'runtime.log'"++ def test_a_moved_key(self) -> None:+ described = describe_op(+ op=MoveKeyOp(+ table_path=["pipelex", "prompting_config"],+ key="default_prompting_style",+ new_table_path=["pipelex", "templating_config"],+ new_key="default_templating_style",+ )+ )++ assert described == (+ "moved 'default_prompting_style' from 'pipelex.prompting_config' to 'default_templating_style' in 'pipelex.templating_config'"+ )++ def test_a_remapped_value_names_both_spellings_because_both_come_from_the_ledger(self) -> None:+ described = describe_op(op=RemapValueOp(table_path=["runtime", "log"], key="default_log_level", mapping={"VERBOSE": "DEBUG"}))++ assert described == "rewrote the value of 'default_log_level' in 'runtime.log': 'VERBOSE' -> 'DEBUG'"++ def test_a_remap_over_every_key_of_a_table_says_so_rather_than_naming_a_key(self) -> None:+ """The user's own keys are what a `*` remap reaches, and naming them would be quoting the file."""+ described = describe_op(op=RemapValueOp(table_path=["runtime", "log", "package_log_levels"], key=WILDCARD_SEGMENT, mapping={"a": "b"}))++ assert described == "rewrote every value in 'runtime.log.package_log_levels': 'a' -> 'b'"diff --git a/tests/unit/pipelex/cogt/model_backends/test_backend_boot_tolerance.py b/tests/unit/pipelex/cogt/model_backends/test_backend_boot_tolerance.pynew file mode 100644index 000000000..60988e5c8--- /dev/null+++ b/tests/unit/pipelex/cogt/model_backends/test_backend_boot_tolerance.py@@ -0,0 +1,278 @@+"""A stale backend directory boots with a warning instead of stopping the world.++This is the fourth loader to join boot tolerance, and it is the one the tolerance was built for:+`#1104` deleted `prompting_target` from the model-spec blueprint, `pipelex init` never overwrites an+existing file, so the key survives in `inference/backends/*.toml` on every machine that was set up+before the change — where it is fatal twice over. In `[defaults]` it is copied wholesale into every+model of the file and fails all of them with `extra_forbidden`; on one model it is rejected by name+as `NOT_HEADER_SHAPED`, which has been fatal in lenient mode too since the rogue-headers guard.++The two standing properties are the same as for the other three surfaces, and most of what follows+holds one of them:++- **Only what the ledger explains is tolerated.** A key the user chose to have still stops the boot,+ in both lenient modes, with exactly the error it produced before. Tolerance widens what starts, it+ never widens what is accepted.+- **Boot never writes.** Only `pipelex migrate` does, which is why the warning keeps coming back+ until it is run.++The stale documents here are made by planting the key back into a copy of **the kit's own backend+directory** — the files `pipelex init` puts on a machine — rather than by copying anything from this+laptop, and the planting raises if it matches nothing, so a kit file that stops carrying the anchor+turns these tests red instead of quietly testing an unplanted document.+"""++from __future__ import annotations++import shutil+from pathlib import Path+from typing import TYPE_CHECKING, Any++import pytest++from pipelex.cogt.exceptions import InferenceBackendLibraryError+from pipelex.cogt.model_backends.backend import PipelexBackend+from pipelex.cogt.model_backends.backend_library import InferenceBackendLibrary+from pipelex.cogt.model_backends.gateway_config import GatewayConfig+from pipelex.kit.paths import get_kit_configs_dir+from pipelex.system.configuration.config_loader import BACKENDS_DIR_NAME, CONFIG_DIR_NAME, INFERENCE_DIR_NAME+from pipelex.tools.secrets.env_secrets_provider import EnvSecretsProvider++if TYPE_CHECKING:+ from pytest_mock import MockerFixture++# Two backends, both with a literal key: what is under test is the *shape* of the per-backend files,+# and a `${VAR}` here would make every case below depend on the machine's environment instead.+BACKENDS_TOML = """+[openai]+enabled = true+api_key = "sk-not-a-real-key"++[portkey]+enabled = true+api_key = "sk-not-a-real-key-either"+"""++GATEWAY_BACKENDS_TOML = """+[pipelex_gateway]+enabled = true+api_key = "sk-not-a-real-key"+"""++GATEWAY_SERVED_SPECS: dict[str, Any] = {+ "defaults": {"model_type": "llm", "sdk": "openai_responses", "thinking_mode": "none"},+ "gpt-4o": {"model_id": "gpt-4o"},+}++# The key `#1104` deleted, and the two shapes it survives in. The value is immaterial to every+# assertion below — what matters is that the blueprint no longer has anywhere to put it.+RETIRED_KEY = "prompting_target"+++def kit_backends_dir() -> Path:+ return Path(str(get_kit_configs_dir())) / INFERENCE_DIR_NAME / BACKENDS_DIR_NAME+++def plant_in_defaults(*, path: Path) -> None:+ """Put the retired key back into a file's `[defaults]` block, where it breaks every model in it."""+ text = path.read_text(encoding="utf-8")+ anchor = "[defaults]\n"+ if anchor not in text:+ msg = f"'{path.name}' no longer has a [defaults] block to plant into — the fixture is testing nothing."+ raise AssertionError(msg)+ path.write_text(text.replace(anchor, f'{anchor}{RETIRED_KEY} = "openai"\n', 1), encoding="utf-8")+++def plant_on_model(*, path: Path, table_header: str, key: str = RETIRED_KEY, value: str = '"gemini"') -> None:+ """Put a key back on one model table, exactly as a pre-`#1104` file carried it."""+ text = path.read_text(encoding="utf-8")+ anchor = f"{table_header}\n"+ if anchor not in text:+ msg = f"'{path.name}' no longer has a {table_header} table to plant into — the fixture is testing nothing."+ raise AssertionError(msg)+ path.write_text(text.replace(anchor, f"{anchor}{key} = {value}\n", 1), encoding="utf-8")+++def machine(tmp_path: Path, mocker: MockerFixture) -> Path:+ """A machine whose global configuration directory holds a fresh copy of the kit's backend files.++ The home directory is faked so that `config_manager.existing_config_dirs` — the one derivation of+ the walk, and what decides whether the warning may name `pipelex migrate` — answers with this+ directory rather than the developer's own.+ """+ fake_home = tmp_path / "home"+ global_dir = fake_home / CONFIG_DIR_NAME+ inference_dir = global_dir / INFERENCE_DIR_NAME+ inference_dir.mkdir(parents=True)+ shutil.copytree(kit_backends_dir(), inference_dir / BACKENDS_DIR_NAME)++ project_root = tmp_path / "project"+ (project_root / ".git").mkdir(parents=True)++ mocker.patch.object(Path, "home", return_value=fake_home)+ mocker.patch.object(Path, "cwd", return_value=project_root)++ return global_dir+++class TestAStaleBackendDirectory:+ def _backends_dir(self, machine: Path) -> Path:+ return machine / INFERENCE_DIR_NAME / BACKENDS_DIR_NAME++ def _write_library(self, machine: Path, *, body: str = BACKENDS_TOML) -> Path:+ library_path = machine / INFERENCE_DIR_NAME / "backends.toml"+ library_path.write_text(body, encoding="utf-8")+ return library_path++ def _make_stale(self, machine: Path) -> tuple[Path, Path]:+ """`prompting_target` back where an upgrade leaves it: in one file's defaults, on another's models."""+ backends_dir = self._backends_dir(machine)+ openai_file = backends_dir / "openai.toml"+ portkey_file = backends_dir / "portkey.toml"+ plant_in_defaults(path=openai_file)+ plant_on_model(path=portkey_file, table_header="[gpt-4o]")+ plant_on_model(path=portkey_file, table_header='["gemini-2.5-pro"]')+ return openai_file, portkey_file++ def _load(+ self,+ machine: Path,+ *,+ lenient: bool,+ library_body: str = BACKENDS_TOML,+ gateway_config: GatewayConfig | None = None,+ ) -> InferenceBackendLibrary:+ library_path = self._write_library(machine, body=library_body)+ library = InferenceBackendLibrary.make_empty()+ library.load(+ secrets_provider=EnvSecretsProvider(),+ backends_library_path=str(library_path),+ backends_dir_path=str(self._backends_dir(machine)),+ gateway_config=gateway_config,+ lenient=lenient,+ )+ return library++ @pytest.mark.parametrize("lenient", [True, False])+ def test_a_healthy_directory_loads_and_says_nothing(self, machine: Path, lenient: bool) -> None:+ """The control. Without it every case below would pass on a library that loaded nothing."""+ library = self._load(machine, lenient=lenient)++ assert set(library.root) == {"openai", "portkey"}+ assert library.take_stale_configuration_warning() is None++ @pytest.mark.parametrize("lenient", [True, False])+ def test_a_stale_directory_loads_the_same_models_the_current_files_would(self, machine: Path, lenient: bool) -> None:+ """The whole point: what boots is the migrated configuration, not a degraded one.++ Comparing against the library the untouched kit files produce is what makes "everything else+ intact" a real claim — every model, every cost, every request header, from both files.+ """+ pristine = self._load(machine, lenient=lenient)+ self._make_stale(machine)++ recovered = self._load(machine, lenient=lenient)++ assert recovered.root == pristine.root++ @pytest.mark.parametrize("lenient", [True, False])+ def test_it_parks_one_warning_naming_every_stale_file_and_the_remedy(self, machine: Path, lenient: bool) -> None:+ openai_file, portkey_file = self._make_stale(machine)++ library = self._load(machine, lenient=lenient)++ warning = library.take_stale_configuration_warning()+ assert warning is not None+ assert str(openai_file) in warning+ assert str(portkey_file) in warning+ assert "Drop prompting_target from every backend definition" in warning+ assert "Run `pipelex migrate`" in warning+ assert "Nothing was written" in warning+ assert library.take_stale_configuration_warning() is None, "a warning is handed over once"++ def test_the_warning_carries_no_value_read_from_the_users_file(self, machine: Path) -> None:+ """Ledger text only, the same rule the migration report obeys."""+ self._make_stale(machine)++ library = self._load(machine, lenient=False)++ warning = library.take_stale_configuration_warning()+ assert warning is not None+ assert "gemini" not in warning+ assert "gpt-4o" not in warning++ def test_nothing_is_written_to_disk(self, machine: Path) -> None:+ """Boot never writes — no rewrite, and no backup beside the file either."""+ self._make_stale(machine)+ backends_dir = self._backends_dir(machine)+ before = {path.name: path.read_bytes() for path in sorted(backends_dir.iterdir())}++ self._load(machine, lenient=False)++ assert {path.name: path.read_bytes() for path in sorted(backends_dir.iterdir())} == before++ @pytest.mark.parametrize("lenient", [True, False])+ def test_a_key_the_user_chose_to_have_is_still_fatal(self, machine: Path, lenient: bool) -> None:+ """Tolerance is not leniency. The ledger explains what *we* removed and nothing else."""+ plant_on_model(path=self._backends_dir(machine) / "openai.toml", table_header="[gpt-4o]", key="foo", value="1")++ with pytest.raises(InferenceBackendLibraryError) as exc_info:+ self._load(machine, lenient=lenient)++ assert "'foo'" in str(exc_info.value)++ @pytest.mark.parametrize("lenient", [True, False])+ def test_a_file_the_ledger_only_half_explains_raises_the_users_own_error(self, machine: Path, lenient: bool, mocker: MockerFixture) -> None:+ """The branch where the replay does real work and still does not get there.++ This file carries both the key the ledger removes and one it has never heard of, and the+ user's own key sits on the *earlier* model — so the error the load produces is about `foo`,+ the replay then fires for real (there is a `prompting_target` further down to remove), and+ the re-validation fails anyway. What the user sees is still their own error, naming the key+ they can act on; "migration did not help" would name nothing.+ """+ openai_file = self._backends_dir(machine) / "openai.toml"+ plant_on_model(path=openai_file, table_header='["gpt-3.5-turbo"]', key="foo", value="1")+ plant_on_model(path=openai_file, table_header="[gpt-4o]")+ retry = mocker.spy(InferenceBackendLibrary, "_local_model_specs_the_ledger_can_explain")++ with pytest.raises(InferenceBackendLibraryError) as exc_info:+ self._load(machine, lenient=lenient)++ assert "'foo'" in str(exc_info.value)+ assert retry.call_count == 1, "the retry was attempted"+ assert retry.spy_return is None, "and it declined, rather than the error never reaching it"++ def test_a_healthy_directory_never_reaches_the_migration_engine(self, machine: Path, mocker: MockerFixture) -> None:+ """The retry runs on the failure path only, so a current machine pays nothing for it."""+ retry = mocker.spy(InferenceBackendLibrary, "_local_model_specs_the_ledger_can_explain")++ self._load(machine, lenient=False)++ assert retry.call_count == 0++ def test_a_stale_gateway_override_never_reaches_the_loader_at_all(self, machine: Path) -> None:+ """The gateway's local file is the one backend file a stale key cannot break, and here is why.++ `GatewayConfigMerger` ignores a local `[defaults]` outright and keeps only `sdk` and+ `structure_method` from a per-model override, so the retired key is filtered out before any+ spec is built. That is why the retry below is wired to local backend files only — and this+ test is what would go red if the merger ever stopped filtering, which is the day the gateway+ path would need one too. `pipelex migrate` still repairs the file on disk: it is a `*.toml`+ in the directory the surface owns, and the walk claims it like any other.+ """+ gateway_file = self._backends_dir(machine) / f"{PipelexBackend.GATEWAY}.toml"+ gateway_file.write_text(f'[defaults]\n{RETIRED_KEY} = "openai"\n\n[gpt-4o]\n{RETIRED_KEY} = "openai"\n', encoding="utf-8")++ library = self._load(+ machine,+ lenient=False,+ library_body=GATEWAY_BACKENDS_TOML,+ gateway_config=GatewayConfig(model_specs=GATEWAY_SERVED_SPECS, aws_region="eu-west-1"),+ )++ backend = library.get_inference_backend(backend_name=PipelexBackend.GATEWAY)+ assert backend is not None+ assert backend.model_specs["gpt-4o"].model_id == "gpt-4o", "the served spec is what loads, untouched"+ assert library.take_stale_configuration_warning() is Nonediff --git a/tests/unit/pipelex/cogt/model_backends/test_backend_toml_path.py b/tests/unit/pipelex/cogt/model_backends/test_backend_toml_path.pynew file mode 100644index 000000000..8c39de37f--- /dev/null+++ b/tests/unit/pipelex/cogt/model_backends/test_backend_toml_path.py@@ -0,0 +1,28 @@+"""A backend's file path is spelled once, and a path-like backend name cannot escape the directory.++Boot tolerance replays a stale backend file in memory and rebuilds its specs from the result. That+retry must read *the file the loader read* — and until this test existed, that held by coincidence+rather than by construction: the load spelled the path with an f-string and the retry spelled it+with `Path.__truediv__`, which are the same for every ordinary name and differ for exactly one+class. A backend name is a raw top-level table key of the user's own `inference/backends.toml`, TOML+permits a quoted `["/abs/path"]` key, and `Path(directory) / "/abs/path.toml"` discards the+directory. The retry would then read a file the load never saw, and name it in the warning.+"""++from pathlib import Path++from pipelex.cogt.model_backends.backend_library import backend_toml_path+++class TestBackendTomlPath:+ def test_an_ordinary_backend_name_lands_beside_its_siblings(self) -> None:+ path = backend_toml_path(backends_dir_path="/home/someone/.pipelex/inference/backends", backend_name="openai")++ assert path == Path("/home/someone/.pipelex/inference/backends/openai.toml")++ def test_an_absolute_backend_name_stays_under_the_backends_directory(self) -> None:+ backends_dir = Path("/home/someone/.pipelex/inference/backends")++ path = backend_toml_path(backends_dir_path=str(backends_dir), backend_name="/etc/passwd")++ assert path.is_relative_to(backends_dir), f"a path-like backend name escaped the backends directory: {path}"diff --git a/tests/unit/pipelex/cogt/model_backends/test_model_spec_document.py b/tests/unit/pipelex/cogt/model_backends/test_model_spec_document.pynew file mode 100644index 000000000..fa3e01723--- /dev/null+++ b/tests/unit/pipelex/cogt/model_backends/test_model_spec_document.py@@ -0,0 +1,99 @@+"""A backend definition file is a document, not a model — and both of its readers say so here.++`[defaults]` and a per-model table are each half of an `InferenceModelSpecBlueprint`: `sdk` is+required and normally lives in `[defaults]`, so only their merge validates. The loader has always+known that; the migration surface needs the same two answers — a field-level projection of one root+table to fingerprint, and the loader's own verdict on a whole document — and `model_spec_document`+is where the two are stated once so they cannot drift.+"""++from typing import Any++import pytest++from pipelex.cogt.model_backends.model_spec_document import (+ InferenceModelSpecFileNode,+ describe_model_spec_document_rejection,+)+from pipelex.cogt.model_backends.model_spec_factory import InferenceModelSpecBlueprint+++class TestABackendFileIsADocumentNotAModel:+ def test_no_field_of_the_file_node_projection_is_required(self) -> None:+ """The one guard the fingerprint's honesty rests on.++ Every root table of a backend file is a *partial* spec — `[defaults]` carries what the models+ share, a model table carries what it overrides — so a projection with a required field would+ claim of every table a key that most of them legitimately lack. This goes red the day a new+ required field lands on the blueprint, which is exactly when the projection needs a decision.+ """+ required = [name for name, field in InferenceModelSpecFileNode.model_fields.items() if field.is_required()]++ assert required == []++ def test_the_projection_holds_exactly_the_blueprint_fields(self) -> None:+ """It is a projection of the blueprint, not a model of its own: same keys, different requiredness."""+ assert set(InferenceModelSpecFileNode.model_fields) == set(InferenceModelSpecBlueprint.model_fields)++ def test_a_document_the_loader_accepts_is_accepted(self) -> None:+ document: dict[str, Any] = {+ "defaults": {"sdk": "openai", "model_type": "llm"},+ "gpt-4o": {"model_id": "gpt-4o-2024-11-20", "max_tokens": 4096},+ "o3": {"thinking_mode": "adaptive", "x-portkey-provider": "openai"},+ }++ assert describe_model_spec_document_rejection(document=document) is None++ def test_a_stale_key_in_the_defaults_block_is_refused_and_names_the_key(self) -> None:+ """The `[defaults]` half of what `#1104` left behind: one dead key there fails every model.++ `[defaults]` is copied wholesale into each model's blueprint dict — it is deliberately *not*+ passed through the header split — so a key the blueprint no longer knows is `extra_forbidden`+ on every table in the file rather than on one.+ """+ document: dict[str, Any] = {+ "defaults": {"sdk": "openai", "prompting_target": "openai"},+ "gpt-4o": {"model_id": "gpt-4o-2024-11-20"},+ }++ rejection = describe_model_spec_document_rejection(document=document)++ assert rejection is not None+ assert "prompting_target" in rejection+ assert "gpt-4o" in rejection++ def test_a_stale_key_on_one_model_is_refused_by_name(self) -> None:+ """The per-model half: not header-shaped, so it is a typo or a dead field, and fatal since #1109."""+ document: dict[str, Any] = {+ "defaults": {"sdk": "openai"},+ "gpt-4o": {"prompting_target": "openai"},+ }++ rejection = describe_model_spec_document_rejection(document=document)++ assert rejection is not None+ assert "prompting_target" in rejection++ def test_a_header_shaped_key_is_not_a_rejection(self) -> None:+ """The keys this surface must never touch: they are the author's headers, and the loader sends them."""+ document: dict[str, Any] = {+ "defaults": {"sdk": "openai"},+ "gpt-4o": {"x-portkey-config": "cfg-123", "anthropic-beta": "output-128k"},+ }++ assert describe_model_spec_document_rejection(document=document) is None++ def test_a_document_with_no_model_at_all_is_accepted(self) -> None:+ """`pipelex_gateway.toml` in the kit is comments only, and the loader loads it without complaint."""+ assert describe_model_spec_document_rejection(document={}) is None++ @pytest.mark.parametrize(+ "document",+ [+ {"defaults": "not a table"},+ {"defaults": {"sdk": "openai"}, "gpt-4o": "not a table"},+ ],+ ids=["defaults", "model"],+ )+ def test_a_root_key_whose_value_is_not_a_table_is_refused(self, document: dict[str, Any]) -> None:+ assert describe_model_spec_document_rejection(document=document) is not Nonediff --git a/tests/unit/pipelex/migration/conftest.py b/tests/unit/pipelex/migration/conftest.pyindex df2d3212c..6c63d7cfc 100644--- a/tests/unit/pipelex/migration/conftest.py+++ b/tests/unit/pipelex/migration/conftest.py@@ -18,6 +18,8 @@ from pipelex.suggested_fix import MigrationOpEXAMPLE_SURFACE_ID = "example-config"EXAMPLE_BASE_FILE = "example.toml"EXAMPLE_TIER_GLOB = "example_*.toml"+CONFIGURATION_ROOT = Path()+"""The subdirectory a surface that lives directly in a configuration directory owns — none."""EntryBuilder = Callable[..., MigrationEntry]LedgerBuilder = Callable[..., MigrationLedger]@@ -110,14 +112,16 @@ def build_surface() -> SurfaceBuilder:def _build_surface(*,surface_id: str = EXAMPLE_SURFACE_ID,- base_file: str = EXAMPLE_BASE_FILE,+ base_file: str | None = EXAMPLE_BASE_FILE,tier_glob: str | None = EXAMPLE_TIER_GLOB,+ subdirectory: Path = CONFIGURATION_ROOT,) -> Surface:return Surface(surface_id=surface_id,title=surface_id,base_file=base_file,tier_glob=tier_glob,+ subdirectory=subdirectory,config_model=ExampleConfig,defaults_layer_kind=DefaultsLayerKind.MODEL_DEFAULTS,)diff --git a/tests/unit/pipelex/migration/document_strategies.py b/tests/unit/pipelex/migration/document_strategies.pyindex 6776f804c..b891a927f 100644--- a/tests/unit/pipelex/migration/document_strategies.py+++ b/tests/unit/pipelex/migration/document_strategies.py@@ -55,10 +55,10 @@ from typing import Any, castimport tomlkitfrom hypothesis import strategies as st-from pydantic import BaseModel, ConfigDict, Field, ValidationError+from pydantic import BaseModel, ConfigDict, Fieldfrom pipelex.migration.documents import flatten_document-from pipelex.migration.fingerprint import ENUM_TYPE, PATH_SEPARATOR, PathFingerprint, SurfaceFingerprint, compute_fingerprint+from pipelex.migration.fingerprint import ENUM_TYPE, PATH_SEPARATOR, PathFingerprint, SurfaceFingerprintfrom pipelex.migration.surfaces import Surfacefrom pipelex.suggested_fix import WILDCARD_SEGMENTfrom pipelex.system.configuration.config_surface import strip_reserved_meta@@ -93,19 +93,29 @@ class GeneratedDocument(BaseModel):def surface_fingerprint(*, surface: Surface) -> SurfaceFingerprint:"""The surface's model tree projected against its own defaults, at its current shape."""- return compute_fingerprint(- surface_id=surface.surface_id,- schema_version=1,- config_model=surface.config_model,- defaults_document=surface.read_defaults_document(),- )+ return surface.fingerprint_at(schema_version=1)+++def starting_document(*, surface: Surface) -> dict[str, Any]:+ """The document every draw mutates: the richest one the surface has at its current schema.++ For a layered surface that is the defaults layer, which is also what a user's file is read+ beneath. A **copied**-document surface has no defaults layer to seed from — its files stand alone+ — so the seed is its reference document, one real backend definition. Seeding from `{}` there+ would leave the sampler with nothing to mutate and every property over it vacuously green.+ """+ if surface.defaults_layer_kind.is_layered_beneath_the_users_file:+ return surface.read_defaults_document()+ return tomlkit.loads(surface.render_reference_document()).unwrap()def merge_beneath_defaults(*, surface: Surface, document: dict[str, Any]) -> dict[str, Any]:"""A file's content as the models will really see it: merged beneath the defaults layer.- Validating a user's file on its own would report the defaults layer doing its job as a- failure — an absent optional key is the normal shape of a configuration file, not an error.+ Validating a layered surface's file on its own would report the defaults layer doing its job as a+ failure — an absent optional key is the normal shape of a configuration file, not an error. A+ copied document is read exactly as it is, so there is nothing to merge and merging the reference+ copy of one backend definition beneath another would validate a hybrid no machine has."""merged: dict[str, Any] = surface.read_defaults_document()deep_update(merged, updates=document)@@ -119,20 +129,24 @@ def merge_text_beneath_defaults(*, surface: Surface, text: str) -> dict[str, Anydef is_accepted_by_the_surface(*, surface: Surface, document: dict[str, Any]) -> bool:- """Whether the models accept this document — the only authority on schema membership."""+ """Whether the surface accepts this document — the only authority on schema membership.++ Asked of the surface rather than of `config_model` directly, because for a backend definition file+ the model describes one root table and the document is a table per model: what "accepted" means is+ the loader's own merge-then-validate, and `Surface.validate_document` is where that is stated once.+ """try:- surface.config_model.model_validate(merge_beneath_defaults(surface=surface, document=document))- except (ValidationError, ConfigValidationError):- # Both spellings of the same answer: a `ConfigRoot` re-raises pydantic's rejection as its- # own error, and the surfaces are a mix of the two.+ return surface.validate_document(document=merge_beneath_defaults(surface=surface, document=document)) is None+ except ConfigValidationError:+ # A `ConfigRoot` re-raises pydantic's rejection as its own error rather than returning it, and+ # the surfaces are a mix of the two.return False- return Truedef within_schema_documents(*, surface: Surface) -> st.SearchStrategy[GeneratedDocument]:"""Documents a user could legitimately have, at the surface's current schema."""fingerprint = surface_fingerprint(surface=surface)- flattened = flatten_document(document=surface.read_defaults_document())+ flattened = flatten_document(document=starting_document(surface=surface))return _sampled_documents(surface=surface,@@ -157,7 +171,7 @@ def _sampled_documents( # kw-only: ignore — Hypothesis passes `draw` positionswapped = draw(_path_subsets(paths=swappable, max_size=MAX_SWAPPED_ENUM_MEMBERS))flipped = draw(_path_subsets(paths=flippable, max_size=MAX_FLIPPED_BOOLS))- document = surface.read_defaults_document()+ document = starting_document(surface=surface)mutations: list[DocumentMutation] = []for path in sorted(swapped):@@ -171,10 +185,17 @@ def _sampled_documents( # kw-only: ignore — Hypothesis passes `draw` positionif _assign_if_accepted(surface=surface, document=document, path=path, value=not _value_at(mapping=document, path=path)):_record(mutations=mutations, mutation=DocumentMutation.FLIPPED_BOOL)- # Dropping runs last and is never checked, because the defaults layer restores whatever it- # removes: the document the models see is the same one they just accepted.+ # Dropping runs last. For a layered surface it needs no check, because the defaults layer restores+ # whatever it removes: the document the models see is the same one they just accepted. A copied+ # document has nothing to restore it, so there a drop is checked like any other mutation —+ # `[defaults] sdk` is the case that proves it, since dropping it invalidates every model in the+ # file, and an unchecked drop would emit a document no machine could boot on.+ layer_restores_a_dropped_key = surface.defaults_layer_kind.is_layered_beneath_the_users_filefor path in sorted(dropped):- if _drop(mapping=document, path=path):+ applied = (+ _drop(mapping=document, path=path) if layer_restores_a_dropped_key else _drop_if_accepted(surface=surface, document=document, path=path)+ )+ if applied:_record(mutations=mutations, mutation=DocumentMutation.DROPPED_PATH)text: str = tomlkit.dumps(document) # pyright: ignore[reportUnknownMemberType]@@ -211,11 +232,18 @@ def fingerprint_at(*, fingerprint: SurfaceFingerprint, path: str) -> PathFingerpdocument carries the user's own key where the fingerprint carries a `*`. Walking segment bysegment, and substituting `*` the moment the parent is an open node, translates one into theother — including through nesting, where `dict[str, dict[str, int]]` addresses `a.*.*`.++ The **document root** is one of the nodes that can be open, and then the very first segment is the+ user's: `gpt-4o.max_tokens` in a backend file is `*.max_tokens` to the fingerprint."""translated: list[str] = []for segment in path.split(PATH_SEPARATOR):- parent = fingerprint.paths.get(PATH_SEPARATOR.join(translated)) if translated else None- translated.append(WILDCARD_SEGMENT if parent is not None and parent.open_node else segment)+ if translated:+ parent = fingerprint.paths.get(PATH_SEPARATOR.join(translated))+ parent_is_open = parent is not None and parent.open_node+ else:+ parent_is_open = fingerprint.document_root_is_open+ translated.append(WILDCARD_SEGMENT if parent_is_open else segment)if PATH_SEPARATOR.join(translated) not in fingerprint.paths:return Nonereturn fingerprint.paths[PATH_SEPARATOR.join(translated)]@@ -240,6 +268,25 @@ def _assign_if_accepted(*, surface: Surface, document: dict[str, Any], path: strreturn False+def _drop_if_accepted(*, surface: Surface, document: dict[str, Any], path: str) -> bool:+ """Remove a path, keep it removed only if the surface still accepts the document, and say which.++ The drop counterpart of `_assign_if_accepted`, for a surface with no defaults layer beneath its+ files. Restoring puts the value back where it was rather than deep-merging it, so a drawn set of+ paths cannot leave the document holding something no draw asked for.+ """+ located = _parent_of(mapping=document, path=path)+ if located is None:+ return False+ parent, key = located+ previous = parent[key]+ del parent[key]+ if is_accepted_by_the_surface(surface=surface, document=document):+ return True+ parent[key] = previous+ return False++def _parent_of(*, mapping: dict[str, Any], path: str) -> tuple[dict[str, Any], str] | None:segments = path.split(PATH_SEPARATOR)node = mappingdiff --git a/tests/unit/pipelex/migration/test_backend_back_entry.py b/tests/unit/pipelex/migration/test_backend_back_entry.pynew file mode 100644index 000000000..b2fcc8c39--- /dev/null+++ b/tests/unit/pipelex/migration/test_backend_back_entry.py@@ -0,0 +1,100 @@+"""The backend entry, exercised on the shape it is about.++`inference-backend@2` deletes `prompting_target` from every root table of a backend definition file.+Like `telemetry-config@2` beside it, it is about real files on real machines rather than about the+migration machinery, and what it must do to such a file cannot be asked of a synthetic surface: the+key sits both in `[defaults]` — where the loader's wholesale copy makes one occurrence break *every*+model of the file — and on individual models, where it is rejected by name. Both halves are fatal+today, so the assertion that matters is not "the key is gone" but "the loader now accepts what comes+out", which is what `describe_model_spec_document_rejection` answers here.++The other half of the entry's job is the one a delete is most likely to get wrong: the header-shaped+keys of a model table belong to the request, the loader never validates them, and a wildcard that+reached them would silently strip a user's routing. They are asserted present, by value.+"""++from typing import Any++from pipelex.cogt.model_backends.model_spec_document import describe_model_spec_document_rejection+from pipelex.migration.engine import replay_ledger_over_text+from pipelex.migration.goldens import pre_history_document_path+from pipelex.migration.ledger import load_ledger, packaged_migration_dir+from pipelex.tools.misc.toml_utils import load_toml_from_content++SURFACE_ID = "inference-backend"+REMOVED_KEY = "prompting_target"+++class TestTheBackendBackEntry:+ def _old_shape_document(self) -> str:+ """The hand-authored `[email protected]` — a pre-#1104 backend file, as the entry's own check reads it."""+ path = pre_history_document_path(migration_dir=packaged_migration_dir(), surface_id=SURFACE_ID, schema_version=2)+ return path.read_text(encoding="utf-8")++ def test_replaying_the_ledger_over_a_stale_backend_file_produces_one_the_loader_accepts(self) -> None:+ ledger = load_ledger(migration_dir=packaged_migration_dir(), surface_id=SURFACE_ID)+ replay = replay_ledger_over_text(ledger=ledger, text=self._old_shape_document())++ assert [step.entry_id for step in replay.steps] == ["inference-backend@2"]+ assert replay.blocked == []++ migrated: dict[str, Any] = load_toml_from_content(replay.text)+ # The verdict the boot gives, not a proxy for it: `[defaults]` is merged into every model+ # table and the merge is what has to validate.+ assert describe_model_spec_document_rejection(document=migrated) is None+ # The key is gone from both the shared table and the model that overrode it, which are the+ # two occurrences the wildcard has to reach.+ assert REMOVED_KEY not in migrated["defaults"]+ assert REMOVED_KEY not in migrated["gemini-2.5-pro"]+ assert not any(REMOVED_KEY in table for table in migrated.values())++ # Nothing else moved. Stated as the whole document rather than key by key, because what this+ # entry must not do is broader than any list of keys worth naming — the header-shaped keys of+ # a model table in particular, which the loader never validates and a stray wildcard would+ # strip without anything noticing.+ assert migrated == {+ "defaults": {+ "model_type": "llm",+ "sdk": "portkey_completions",+ "structure_method": "instructor/openai_tools",+ "thinking_mode": "none",+ },+ "gpt-4o": {+ "inputs": ["text", "images", "pdf"],+ "outputs": ["text", "structured"],+ "costs": {"input": 2.5, "output": 10.0},+ "sdk": "portkey_responses",+ "structure_method": "instructor/openai_responses_tools",+ "thinking_mode": "none",+ "x-portkey-provider": "@openai",+ },+ "gemini-2.5-pro": {+ "model_id": "gemini-2.5-pro",+ "inputs": ["text", "images", "pdf"],+ "outputs": ["text", "structured"],+ "max_prompt_images": 3000,+ "costs": {"input": 1.25, "output": 10.0},+ "thinking_mode": "manual",+ "x-portkey-provider": "@google",+ "x-portkey-config": "pc-example-config-id",+ },+ }++ def test_every_line_the_entry_does_not_delete_survives_byte_for_byte(self) -> None:+ """Parsing proves the values; only the text proves the comments, the spacing and the quoting."""+ ledger = load_ledger(migration_dir=packaged_migration_dir(), surface_id=SURFACE_ID)+ before_text = self._old_shape_document()+ after_text = replay_ledger_over_text(ledger=ledger, text=before_text).text++ deleted_lines = [line for line in before_text.splitlines() if line.startswith(f"{REMOVED_KEY} =")]+ assert len(deleted_lines) == 2+ surviving_lines = [line for line in before_text.splitlines() if line not in deleted_lines]+ assert after_text.splitlines() == surviving_lines++ def test_replaying_it_again_changes_nothing(self) -> None:+ """A migrated file is a current file, and every kit backend file is already one — replay must not touch either."""+ ledger = load_ledger(migration_dir=packaged_migration_dir(), surface_id=SURFACE_ID)+ once = replay_ledger_over_text(ledger=ledger, text=self._old_shape_document())+ twice = replay_ledger_over_text(ledger=ledger, text=once.text)+ assert twice.steps == []+ assert twice.text is once.textdiff --git a/tests/unit/pipelex/migration/test_backend_directory_walk.py b/tests/unit/pipelex/migration/test_backend_directory_walk.pynew file mode 100644index 000000000..226701286--- /dev/null+++ b/tests/unit/pipelex/migration/test_backend_directory_walk.py@@ -0,0 +1,93 @@+"""A fresh machine's backend directory, put on the real walk.++Every other test of this surface holds one half of it still. `test_surface_directories.py` proves the+claim rule over a synthetic registry and hand-written file names; `test_real_surfaces.py` proves the+shipped ledger is neutral over the kit's backend documents by replaying it over their *text*, in+memory, inside the gate. Neither one is what a user meets, and the gap between them is where a defect+of this surface would live: a real directory of real backend files, resolved by the real registry,+replayed by the real run, and diagnosed against the real fingerprint.++The directory is the one `pipelex init` copies, so it is at the current schema by construction and is+nobody's fixture to keep up to date — the same reason the gate uses it as a witness. What this module+adds is that the walk *reaches* it and the run leaves it alone.+"""++import shutil+from pathlib import Path++from pipelex.kit.paths import get_kit_configs_dir+from pipelex.migration.run import migrate_config_directories+from pipelex.migration.surfaces import build_config_surface_registry+from pipelex.system.configuration.config_loader import BACKENDS_DIR_NAME, INFERENCE_DIR_NAME+from pipelex.system.configuration.config_surface import INFERENCE_BACKEND_CONFIG_SURFACE_ID+++class TestAFreshMachinesBackendDirectory:+ @staticmethod+ def _seed(*, root: Path) -> list[Path]:+ """Copy the kit's backend directory into a configuration root, and say what landed there.++ Copied rather than pointed at: the walk claims files inside a *configuration* directory, and+ the kit's own copy of them is a package resource. `pipelex init` copies every `.toml` and+ `.md` of it, so every file is copied here — a file the kit ships and the walk then has an+ opinion about is exactly what this module is for.+ """+ kit_backends = Path(str(get_kit_configs_dir())) / INFERENCE_DIR_NAME / BACKENDS_DIR_NAME+ target = root / INFERENCE_DIR_NAME / BACKENDS_DIR_NAME+ target.mkdir(parents=True)+ for kit_file in sorted(kit_backends.iterdir()):+ shutil.copy2(kit_file, target / kit_file.name)+ return sorted(target.iterdir())++ def test_every_backend_definition_is_claimed_and_the_documentation_beside_them_is_not(self, tmp_path: Path) -> None:+ """The claim rule over the real registry and a real directory listing, not a synthetic one.++ The `.md` files are the half that only a real directory can prove: they are there because the+ kit ships them next to the backends it documents, and a surface claiming `*.toml` leaves them+ alone by extension rather than by an exclusion anybody has to maintain.+ """+ seeded = self._seed(root=tmp_path)+ assert [path.name for path in seeded if path.suffix == ".md"], "worthless unless the kit still ships documentation in there"++ claimed = build_config_surface_registry().files_by_surface_in_directory(directory=tmp_path)++ assert {surface.surface_id for surface, _ in claimed} == {INFERENCE_BACKEND_CONFIG_SURFACE_ID}+ assert [path for _, path in claimed] == [path for path in seeded if path.suffix == ".toml"]++ def test_a_dry_run_over_it_finds_every_file_and_reports_them_all_clean(self, tmp_path: Path) -> None:+ """Convergence proved on the walk: the shipped ledger has nothing to do to a current file.++ Both halves are load-bearing. That the run *finds* every backend definition is the walk; that+ it reports each one clean is the property — nothing applied, nothing blocked, and nothing the+ current schema cannot explain. The last one is new here rather than a restatement of the+ gate's convergence check: that check compares text before and after a replay, and never asks+ the diagnosis anything. A Portkey backend is full of `x-portkey-*` request headers, so a+ surface that did not admit them by shape would pass every gate and then greet a fresh machine+ with a page of unexplained paths.+ """+ seeded = self._seed(root=tmp_path)++ report = migrate_config_directories(config_dirs=[tmp_path], dry_run=True)++ assert [plan.file_path for plan in report.plans] == [path for path in seeded if path.suffix == ".toml"]+ assert [(plan.file_path.name, found.path) for plan in report.plans for found in plan.unexplained] == []+ assert report.is_clean++ def test_a_misspelled_setting_in_one_of_them_is_named_by_the_walk(self, tmp_path: Path) -> None:+ """Without this, the test above would pass just as well over a run that diagnoses nothing.++ A misspelling rather than one of the genuinely dead keys this surface exists to retire: a+ dead key earns a ledger entry that explains it, and a test asserting it is *unexplained*+ would have to be rewritten the day that entry lands. A typo no ledger will ever carry keeps+ saying the same thing.++ It is reported as the schema spells it — `*.<key>`, not `<model>.<key>` — because the model+ name is the user's own and every backend file has different ones.+ """+ self._seed(root=tmp_path)+ target = tmp_path / INFERENCE_DIR_NAME / BACKENDS_DIR_NAME / "openai.toml"+ target.write_text(f"{target.read_text(encoding='utf-8')}\n[witness-model]\nmaxx_tokens = 8\n", encoding="utf-8")++ report = migrate_config_directories(config_dirs=[tmp_path], dry_run=True)++ assert [(plan.file_path.name, found.path) for plan in report.plans for found in plan.unexplained] == [("openai.toml", "*.maxx_tokens")]diff --git a/tests/unit/pipelex/migration/test_coverage_gate.py b/tests/unit/pipelex/migration/test_coverage_gate.pyindex 797168eec..43025ddf8 100644--- a/tests/unit/pipelex/migration/test_coverage_gate.py+++ b/tests/unit/pipelex/migration/test_coverage_gate.py@@ -1030,6 +1030,34 @@ min_supported_schema_version = 0assert _kinds(issues) == [CoverageIssueKind.LEDGER_DISAGREES_WITH_REGISTRY]assert "elsewhere.toml" in issues[0].message+ def test_a_ledger_describing_another_directory_is_refused(self, tmp_path: Path) -> None:+ """The directory is the half with a second reader, and it is on the boot path.++ A stale-configuration warning names `pipelex migrate` only for a file the walk would+ reach, and it works out where the walk reaches from the ledger — its module may not import+ the registry. So a ledger naming one directory while the registry walks another is a boot+ offering a remedy the command then declines, and this is where the two are made to agree.+ """+ directory = ledgers_dir(migration_dir=tmp_path)+ directory.mkdir(parents=True, exist_ok=True)+ (directory / f"{SURFACE_ID}.toml").write_text(+ f"""+[surface]+id = "{SURFACE_ID}"+title = "A synthetic surface"+base_file = "synthetic.toml"+tier_glob = "synthetic_*.toml"+subdirectory = "somewhere/else"+current_schema_version = 1+min_supported_schema_version = 0+""",+ encoding="utf-8",+ )+ _snapshot(migration_dir=tmp_path, config_model=_SchemaOne, schema_version=1)+ issues = check_surface(surface=_surface(config_model=_SchemaOne), migration_dir=tmp_path)+ assert _kinds(issues) == [CoverageIssueKind.LEDGER_DISAGREES_WITH_REGISTRY]+ assert "somewhere/else" in issues[0].message+class TestTheDiff:def test_a_renamed_key_reads_as_one_removal_and_one_addition(self) -> None:diff --git a/tests/unit/pipelex/migration/test_diagnosis.py b/tests/unit/pipelex/migration/test_diagnosis.pyindex 01a609f61..ad24a5b0b 100644--- a/tests/unit/pipelex/migration/test_diagnosis.py+++ b/tests/unit/pipelex/migration/test_diagnosis.py@@ -16,6 +16,7 @@ from pipelex.migration.fingerprint import SurfaceFingerprint, compute_fingerprinfrom pipelex.migration.ledger import MigrationLedgerfrom pipelex.migration.plan import BlockedEntry, BlockedEntryReasonfrom pipelex.migration.safety import MigrationSafety+from pipelex.migration.surfaces import DefaultsLayerKind, DocumentShape, Surfacefrom pipelex.suggested_fix import DeleteKeyOp, MigrationOp, RemapValueOp, RenameTableKeyOpfrom tests.unit.pipelex.migration.conftest import EXAMPLE_SURFACE_ID, EntryBuilder, LedgerBuilder@@ -40,12 +41,48 @@ class Schema(BaseModel):queues: dict[str, Queue] = Field(default_factory=dict[str, Queue])+class BackendFileNode(BaseModel):+ """One root table of a document whose root keys are the user's — every field optional."""++ sdk: str | None = None+ max_tokens: int | None = None+++def _surface(*, config_model: type[BaseModel] = Schema, document_shape: DocumentShape = DocumentShape.WHOLE_DOCUMENT) -> Surface:+ return Surface(+ surface_id=EXAMPLE_SURFACE_ID,+ title="An example configuration surface",+ base_file="example.toml",+ config_model=config_model,+ document_shape=document_shape,+ defaults_layer_kind=DefaultsLayerKind.MODEL_DEFAULTS,+ )++def _fingerprint() -> SurfaceFingerprint:return compute_fingerprint(surface_id=EXAMPLE_SURFACE_ID, schema_version=3, config_model=Schema, defaults_document={})def _diagnose(*, document: dict[str, Any], ledger: MigrationLedger, blocked: list[BlockedEntry] | None = None) -> list[str]:- found = diagnose_unexplained_paths(fingerprint=_fingerprint(), document=document, ledger=ledger, blocked=blocked or [])+ found = diagnose_unexplained_paths(+ surface=_surface(),+ fingerprint=_fingerprint(),+ document=document,+ ledger=ledger,+ blocked=blocked or [],+ )+ return [unexplained.path for unexplained in found]+++def _diagnose_backend_document(*, document: dict[str, Any], ledger: MigrationLedger) -> list[str]:+ surface = _surface(config_model=BackendFileNode, document_shape=DocumentShape.MODEL_SPEC_TABLES)+ found = diagnose_unexplained_paths(+ surface=surface,+ fingerprint=surface.fingerprint_at(schema_version=1),+ document=document,+ ledger=ledger,+ blocked=[],+ )return [unexplained.path for unexplained in found]@@ -66,6 +103,7 @@ def empty_ledger(build_ledger: LedgerBuilder) -> MigrationLedger:class TestWhatTheSchemaDoesNotKnow:def test_a_root_key_no_model_knows_is_reported_with_both_readings(self, empty_ledger: MigrationLedger) -> None:found = diagnose_unexplained_paths(+ surface=_surface(),fingerprint=_fingerprint(),document={"label": "hi", "not_a_real_setting": True},ledger=empty_ledger,@@ -77,6 +115,41 @@ class TestWhatTheSchemaDoesNotKnow:assert "typo" in noteassert "newer pipelex" in note, "the downgrade reading is the one nothing else in the report offers"+ def test_a_per_model_request_header_is_not_something_the_schema_cannot_explain(self, empty_ledger: MigrationLedger) -> None:+ """A backend file's headers are legal by *shape*, so they resolve against no recorded path.++ Without this the diagnosis would report `x-portkey-provider` on **every** `pipelex migrate`+ run over a machine that has a portkey backend — the kit's own file carries such keys — and the+ report would be telling the user to fix a key we ship and endorse.+ """+ document = {"defaults": {"sdk": "openai"}, "gpt-4o": {"max_tokens": 1, "x-portkey-provider": "openai"}}++ assert _diagnose_backend_document(document=document, ledger=empty_ledger) == []++ def test_the_same_header_inside_defaults_is_reported_because_the_loader_refuses_it_there(self, empty_ledger: MigrationLedger) -> None:+ """`[defaults]` and a model table are one node to the fingerprint and two to the loader.++ A model table goes through `split_model_spec_keys`, so a header on it is legal. `[defaults]`+ is copied into every model of the file **unsplit**, so the same key there is+ `extra_forbidden` on all of them and the file does not boot. Admitting it would report the+ one file that cannot start as the one file with nothing to explain — and on the command the+ boot error sends that reader to for the diagnosis.+ """+ document = {"defaults": {"sdk": "openai", "x-portkey-provider": "openai"}, "gpt-4o": {"max_tokens": 1}}++ assert _diagnose_backend_document(document=document, ledger=empty_ledger) == ["*.x-portkey-provider"]++ def test_a_typo_inside_a_model_table_is_still_reported_at_the_wildcard(self, empty_ledger: MigrationLedger) -> None:+ """The other half: the admission is by shape, not by "the model allows extras".++ `promting_target` carries no hyphen, so it is a typo or a dead field rather than a header —+ exactly the key class this surface exists to explain — and it is reported at the spelling the+ schema uses, with the model name the user chose left out of it.+ """+ document = {"defaults": {"sdk": "openai"}, "gpt-4o": {"promting_target": "openai"}}++ assert _diagnose_backend_document(document=document, ledger=empty_ledger) == ["*.promting_target"]+def test_a_document_the_schema_knows_entirely_is_reported_clean(self, empty_ledger: MigrationLedger) -> None:document = {"label": "hi", "reporting": {"output": {"directory": "elsewhere"}, "retries": 3}}diff --git a/tests/unit/pipelex/migration/test_fingerprint.py b/tests/unit/pipelex/migration/test_fingerprint.pyindex ed6654b9f..eeb7b4c57 100644--- a/tests/unit/pipelex/migration/test_fingerprint.py+++ b/tests/unit/pipelex/migration/test_fingerprint.py@@ -150,6 +150,32 @@ class TestFingerprintPaths:assert fingerprint.paths["optional_leaf"].required is Falseassert fingerprint.paths["optional_leaf.name"].required is True+ def test_an_open_document_root_records_a_wildcard_with_the_model_beneath_it(self) -> None:+ """A document whose *root* keys belong to the user — one table per inference-backend model.++ The shape has to be seeded rather than modelled, and that is the whole reason the flag+ exists. A `dict[str, _Leaf]` **field** would record every path under a parent segment+ (`models.*.name`) that no such file has, and a `RootModel` puts a `root.` prefix there+ instead. Seeding the walk at `*` is what makes the recorded paths the ones a ledger+ operation can actually address in the file.+ """+ fingerprint = compute_fingerprint(+ surface_id="synthetic",+ schema_version=1,+ config_model=_Leaf,+ defaults_document={},+ document_root_is_open=True,+ )++ assert fingerprint.document_root_is_open is True+ assert fingerprint.path_names() == {"*", "*.name", "*.count"}+ assert fingerprint.paths["*"].value_type == TABLE_TYPE+ assert fingerprint.paths["*.name"].value_type == "str"++ def test_a_document_root_is_closed_unless_the_surface_says_otherwise(self) -> None:+ """The flag is recorded, not inferred, so a frozen golden answers for the version it describes."""+ assert _fingerprint().document_root_is_open is False+def test_a_self_referential_model_terminates(self) -> None:"""Nothing in the configuration tree is recursive today; this is what stops the next one hanging the gate.diff --git a/tests/unit/pipelex/migration/test_ledger_check.py b/tests/unit/pipelex/migration/test_ledger_check.pyindex b029c14fb..78bd2d17b 100644--- a/tests/unit/pipelex/migration/test_ledger_check.py+++ b/tests/unit/pipelex/migration/test_ledger_check.py@@ -17,11 +17,10 @@ import pytestfrom pydantic import BaseModel, Fieldfrom pipelex.migration.exceptions import MigrationLedgerError-from pipelex.migration.fingerprint import compute_fingerprintfrom pipelex.migration.goldens import write_fingerprint_goldenfrom pipelex.migration.ledger import ledgers_dirfrom pipelex.migration.ledger_check import LedgerIssue, LedgerIssueKind, check_ledger-from pipelex.migration.surfaces import DefaultsLayerKind, Surface+from pipelex.migration.surfaces import DefaultsLayerKind, DocumentShape, SurfaceSURFACE_ID = "synthetic-config"@@ -137,19 +136,39 @@ class _SchemaThreeLabelBack(BaseModel):deck: dict[str, _DeckEntry] = Field(default_factory=dict[str, _DeckEntry])+class _RootTableWithLegacyKey(BaseModel):+ """One root table of a document whose *root* keys are the user's — a backend definition file.++ Every field is optional, as every field of such a table has to be: the document's own root+ carries one table per user-chosen name, and each is a partial description.+ """++ sdk: str | None = None+ legacy_style: str | None = None+++class _RootTableLegacyKeyGone(BaseModel):+ """The same table with `legacy_style` retired."""++ sdk: str | None = None++def _surface(*,config_model: type[BaseModel],packaged_document_path: Path | None = None,kit_template_path: Path | None = None,+ document_root_is_open: bool = False,) -> Surface:defaults_layer_kind = DefaultsLayerKind.PACKAGED_DOCUMENT if packaged_document_path is not None else DefaultsLayerKind.MODEL_DEFAULTS+ document_shape = DocumentShape.MODEL_SPEC_TABLES if document_root_is_open else DocumentShape.WHOLE_DOCUMENTreturn Surface(surface_id=SURFACE_ID,title="A synthetic surface",base_file="synthetic.toml",tier_glob="synthetic_*.toml",config_model=config_model,+ document_shape=document_shape,defaults_layer_kind=defaults_layer_kind,packaged_document_path=packaged_document_path,kit_template_path=kit_template_path,@@ -172,17 +191,9 @@ min_supported_schema_version = 0(directory / f"{SURFACE_ID}.toml").write_text(body, encoding="utf-8")-def _snapshot(*, migration_dir: Path, config_model: type[BaseModel], schema_version: int) -> None:- surface = _surface(config_model=config_model)- write_fingerprint_golden(- migration_dir=migration_dir,- fingerprint=compute_fingerprint(- surface_id=SURFACE_ID,- schema_version=schema_version,- config_model=config_model,- defaults_document=surface.read_defaults_document(),- ),- )+def _snapshot(*, migration_dir: Path, config_model: type[BaseModel], schema_version: int, document_root_is_open: bool = False) -> None:+ surface = _surface(config_model=config_model, document_root_is_open=document_root_is_open)+ write_fingerprint_golden(migration_dir=migration_dir, fingerprint=surface.fingerprint_at(schema_version=schema_version))def _entry(*, ops: str, to_schema_version: int = 2, safety: str = "safe", pre_history: str = "", declared_narrowed: str = "") -> str:@@ -283,6 +294,71 @@ key = "some_model_the_user_chose"assert _kinds(issues) == [LedgerIssueKind.CONCRETE_KEY_UNDER_OPEN_NODE]assert "some_model_the_user_chose" in issues[0].message+ def test_a_wildcard_at_an_open_document_root_stands_at_an_open_node(self, tmp_path: Path) -> None:+ """The one address a backend-file ledger can use: every root table, none of them by name.++ The keys of such a document are model names the user chose, so `["*"]` is not a convenience+ — it is the only legal way to reach the key we retired, in `[defaults]` and in every model+ table alike.+ """+ ops = """+[[migration.ops]]+kind = "delete_key"+table_path = ["*"]+key = "legacy_style"+"""+ _write_ledger(migration_dir=tmp_path, current_schema_version=2, entries=_entry(ops=ops))+ _snapshot(migration_dir=tmp_path, config_model=_RootTableWithLegacyKey, schema_version=1, document_root_is_open=True)+ _snapshot(migration_dir=tmp_path, config_model=_RootTableLegacyKeyGone, schema_version=2, document_root_is_open=True)++ surface = _surface(config_model=_RootTableLegacyKeyGone, document_root_is_open=True)+ assert check_ledger(surface=surface, migration_dir=tmp_path) == []++ def test_an_operation_naming_one_root_table_of_an_open_document_is_refused(self, tmp_path: Path) -> None:+ """A backend ledger may not address one user's model by name, and the root is where that bites.++ `openai.toml` on one machine holds the models that machine chose. An entry naming one of them+ would fire on that machine and on no other, which is the same defect as reaching inside an+ open mapping — reported here at the document root.+ """+ ops = """+[[migration.ops]]+kind = "delete_key"+table_path = ["gpt-4o"]+key = "legacy_style"+"""+ _write_ledger(migration_dir=tmp_path, current_schema_version=2, entries=_entry(ops=ops))+ _snapshot(migration_dir=tmp_path, config_model=_RootTableWithLegacyKey, schema_version=1, document_root_is_open=True)+ _snapshot(migration_dir=tmp_path, config_model=_RootTableLegacyKeyGone, schema_version=2, document_root_is_open=True)++ surface = _surface(config_model=_RootTableLegacyKeyGone, document_root_is_open=True)+ issues = check_ledger(surface=surface, migration_dir=tmp_path)++ assert LedgerIssueKind.CONCRETE_KEY_UNDER_OPEN_NODE in _kinds(issues)+ assert "gpt-4o" in issues[0].message++ def test_a_wildcard_at_a_closed_document_root_is_still_refused(self, tmp_path: Path) -> None:+ """The other half of the same rule, and the mutation guard on it: openness is read, not assumed.++ Teach the check that the document root is always open and this goes green — an entry over+ `pipelex.toml` could then claim to address "every root key" of a document whose root keys are+ ours and enumerable.+ """+ ops = """+[[migration.ops]]+kind = "delete_key"+table_path = ["*"]+key = "label"+"""+ _write_ledger(migration_dir=tmp_path, current_schema_version=2, entries=_entry(ops=ops))+ _snapshot(migration_dir=tmp_path, config_model=_SchemaOne, schema_version=1)+ _snapshot(migration_dir=tmp_path, config_model=_SchemaTwoLabelGone, schema_version=2)++ issues = check_ledger(surface=_surface(config_model=_SchemaTwoLabelGone), migration_dir=tmp_path)++ assert LedgerIssueKind.WILDCARD_NOT_AT_OPEN_NODE in _kinds(issues)+ assert "the document root" in issues[0].message+def test_a_wildcard_anywhere_but_an_open_node_is_refused(self, tmp_path: Path) -> None:ops = """[[migration.ops]]diff --git a/tests/unit/pipelex/migration/test_ledger_models.py b/tests/unit/pipelex/migration/test_ledger_models.pyindex 0b0751e40..44b4a47b9 100644--- a/tests/unit/pipelex/migration/test_ledger_models.py+++ b/tests/unit/pipelex/migration/test_ledger_models.py@@ -60,6 +60,27 @@ class TestTheSurfaceBlock:with pytest.raises(ValidationError):MigrationLedger.model_validate({"surface": surface})+ @pytest.mark.parametrize(+ ("base_file", "tier_glob"),+ [+ (None, None),+ ("", ""),+ ("", None),+ (None, ""),+ ],+ )+ def test_a_surface_block_claiming_nothing_is_refused(self, base_file: str | None, tier_glob: str | None) -> None:+ """Absent and empty are the same claim — none — and the guard has to say so for both.++ An empty string is not a narrower claim than `None`, it is the same claim spelled in a way+ that reads as a value: `fnmatch(name, "")` never matches, and a `base_file` of `""` equals no+ filename. Left admitted, a ledger like that parses, resolves nothing, and migrates nothing,+ which is the one failure mode a migration must never have — silence.+ """+ surface = {**_SURFACE_BLOCK, "base_file": base_file, "tier_glob": tier_glob}+ with pytest.raises(ValidationError, match="claims no file"):+ MigrationLedger.model_validate({"surface": surface})+class TestEntryInvariants:def test_a_well_formed_entry_parses_into_typed_operations(self) -> None:diff --git a/tests/unit/pipelex/migration/test_migration_run.py b/tests/unit/pipelex/migration/test_migration_run.pyindex 75a225e56..d51fc0f16 100644--- a/tests/unit/pipelex/migration/test_migration_run.py+++ b/tests/unit/pipelex/migration/test_migration_run.py@@ -94,14 +94,15 @@ class TestTheWalkIsNotRecursive:The specimen is real and it is the reason this is pinned rather than assumed:`.pipelex/inference/backends/pipelex_gateway.toml` matches the `pipelex-config` tier glob- `pipelex_*.toml` exactly, and is an inference backend definition rather than a configuration- surface file. A recursive walk would claim it, replay the main configuration's ledger over it,- and rewrite it.+ `pipelex_*.toml` exactly, and is an inference backend definition rather than a main-configuration+ tier file. The walk *does* reach it now, because `inference/backends/` has a surface of its own —+ which makes the directory half of the claim rule load-bearing rather than academic: get it wrong+ and the main configuration's ledger is replayed over a backend definition and rewrites it."""SPECIMEN = Path("inference") / "backends" / "pipelex_gateway.toml"- def test_the_real_gateway_backend_file_is_not_claimed_where_it_actually_lives(self, tmp_path: Path) -> None:+ def test_the_real_gateway_backend_file_is_claimed_by_its_own_directorys_surface(self, tmp_path: Path) -> None:specimen = tmp_path / self.SPECIMENspecimen.parent.mkdir(parents=True)specimen.write_text("", encoding="utf-8")@@ -109,13 +110,30 @@ class TestTheWalkIsNotRecursive:claimed = build_config_surface_registry().files_by_surface_in_directory(directory=tmp_path)- assert [path.name for _, path in claimed] == ["pipelex.toml"]+ assert [(surface.surface_id, path.name) for surface, path in claimed] == [+ ("pipelex-config", "pipelex.toml"),+ ("inference-backend", self.SPECIMEN.name),+ ]- def test_the_same_file_name_at_the_top_level_is_claimed(self, tmp_path: Path) -> None:- """The other half of the claim: depth is what saves the specimen, not its name.+ def test_a_directory_no_surface_owns_is_never_entered(self, tmp_path: Path) -> None:+ """`inference/deck/` is a different mechanism's directory, and one level up is a different document.++ Both are deliberate exclusions rather than accidents of the glob — a model deck has its own+ manifest-driven sync, and `inference/backends.toml` is a table per backend, not per model.+ """+ (tmp_path / "inference" / "deck").mkdir(parents=True)+ (tmp_path / "inference" / "deck" / "x.toml").write_text("", encoding="utf-8")+ (tmp_path / "inference" / "backends.toml").write_text("", encoding="utf-8")++ claimed = build_config_surface_registry().files_by_surface_in_directory(directory=tmp_path)++ assert claimed == []++ def test_the_same_file_name_at_the_top_level_is_claimed_by_the_main_configuration(self, tmp_path: Path) -> None:+ """The other half of the claim: which directory it sits in is what decides, not its name.Without this, the test above would still pass if the tier glob stopped matching, and it- would be proving nothing about recursion at all.+ would be proving nothing about the directory rule at all."""(tmp_path / self.SPECIMEN.name).write_text("", encoding="utf-8")diff --git a/tests/unit/pipelex/migration/test_real_surfaces.py b/tests/unit/pipelex/migration/test_real_surfaces.pyindex ff6a78d89..7b4bf7631 100644--- a/tests/unit/pipelex/migration/test_real_surfaces.py+++ b/tests/unit/pipelex/migration/test_real_surfaces.py@@ -11,7 +11,7 @@ that is not about configuration, which teaches the reflex the whole design existimport tomlkit-from pipelex.migration.coverage import check_registry+from pipelex.migration.coverage import check_defaults_layer, check_registryfrom pipelex.migration.diagnosis import diagnose_unexplained_pathsfrom pipelex.migration.ledger import INITIAL_SCHEMA_VERSION, load_ledger, packaged_migration_dirfrom pipelex.migration.ledger_check import check_ledgers@@ -26,12 +26,41 @@ class TestTheRealRegistry:"pipelex-config","telemetry-config","pipelex-service-config",+ "inference-backend",]def test_every_surface_has_a_reachable_defaults_layer(self) -> None:- """The premise the whole vocabulary rests on: without it, an added key breaks every file."""+ """The premise the whole vocabulary rests on: without it, an added key breaks every file.++ Stated per kind rather than as "the defaults document is non-empty", which was a proxy that+ held only while every surface had one. A **copied**-document surface honestly has none — its+ files stand alone, nothing is merged beneath them — and its half of the premise is that the+ rule holds anyway, because no path of such a file is one a defaults layer would have to supply.+ That is what `check_defaults_layer` decides, and it is asked here rather than assumed: a+ document faked to satisfy a truthiness check would attribute one backend file's values to+ every other one.+ """+ for surface in build_config_surface_registry().surfaces:+ if surface.defaults_layer_kind.is_layered_beneath_the_users_file:+ assert surface.read_defaults_document(), f"surface '{surface.surface_id}' supplies no defaults"+ continue+ assert surface.read_defaults_document() == {}, f"surface '{surface.surface_id}' fakes a defaults document"+ issues = check_defaults_layer(surface_id=surface.surface_id, fingerprint=surface.fingerprint_at(schema_version=1))+ assert issues == [], f"surface '{surface.surface_id}': {[issue.message for issue in issues]}"++ def test_a_surface_whose_documents_root_keys_are_the_users_records_an_open_root(self) -> None:+ """The shape the whole fourth surface rests on, asserted once against the real registry.++ `*` and every path beneath it, and no path above it: a fingerprint that recorded `sdk` at the+ top would be describing a document nobody has, and every operation written against it would+ be dead on every file in the field.+ """for surface in build_config_surface_registry().surfaces:- assert surface.read_defaults_document(), f"surface '{surface.surface_id}' supplies no defaults"+ if not surface.document_shape.document_root_is_open:+ continue+ fingerprint = surface.fingerprint_at(schema_version=1)+ assert fingerprint.document_root_is_open is True+ assert all(path == "*" or path.startswith("*.") for path in fingerprint.paths)def test_no_surfaces_own_documents_carry_anything_the_diagnosis_cannot_explain(self) -> None:"""The downgrade diagnosis, over the two documents it must always be silent about.@@ -46,6 +75,7 @@ class TestTheRealRegistry:fingerprint = surface.fingerprint_at(schema_version=ledger.surface.current_schema_version)for label, text in surface.reference_documents():unexplained = diagnose_unexplained_paths(+ surface=surface,fingerprint=fingerprint,document=tomlkit.loads(text).unwrap(),ledger=ledger,diff --git a/tests/unit/pipelex/migration/test_replay_properties.py b/tests/unit/pipelex/migration/test_replay_properties.pyindex 8a4b315a1..6f8d4e788 100644--- a/tests/unit/pipelex/migration/test_replay_properties.py+++ b/tests/unit/pipelex/migration/test_replay_properties.py@@ -199,13 +199,17 @@ class TestTheReplayProperties:individual value. It is asserted here, on the emitted text, because that is the artifactthe property is quantified over.- The document is validated the way a user's file is really read, merged beneath the- surface's defaults layer: an absent optional key is the ordinary shape of a configuration- file, not an error.+ The document is validated the way a user's file is really read — for a layered surface, merged+ beneath its defaults layer, because an absent optional key is the ordinary shape of a+ configuration file rather than an error; and through the surface rather than through+ `config_model`, because for a backend definition file the model describes one root table while+ the document is a table per model."""generated = data.draw(_documents_for(surface=surface))- surface.config_model.model_validate(merge_text_beneath_defaults(surface=surface, text=generated.text))+ rejection = surface.validate_document(document=merge_text_beneath_defaults(surface=surface, text=generated.text))++ assert rejection is None, rejection@pytest.mark.parametrize("surface", REAL_SURFACES, ids=REAL_SURFACE_IDS)def test_the_generator_does_not_merely_re_emit_the_reference_document(self, surface: Surface) -> None:diff --git a/tests/unit/pipelex/migration/test_surface_directories.py b/tests/unit/pipelex/migration/test_surface_directories.pynew file mode 100644index 000000000..0854d01be--- /dev/null+++ b/tests/unit/pipelex/migration/test_surface_directories.py@@ -0,0 +1,95 @@+"""A surface owns a directory, and a file is claimed by the pair (directory, name).++`test_surface_resolution.py` covers the name half of the claim — exact filenames before globs, and+two globs over one file refused by name. This module covers the half a subdirectory adds: the same+name in two directories is two different files, and only the surface that owns the directory it+sits in may claim it.++The specimen is real and it is the reason this exists at all:+`inference/backends/pipelex_gateway.toml` matches the `pipelex-config` tier glob `pipelex_*.toml`+exactly. Before the walk reached subdirectories, depth alone kept it safe. Now that the walk goes+there on purpose, the *directory* is what has to keep it safe — and if it ever stops doing so, the+main configuration's ledger gets replayed over an inference backend definition and rewrites it.+"""++from pathlib import Path++from pipelex.migration.surfaces import SurfaceRegistry+from tests.unit.pipelex.migration.conftest import SurfaceBuilder++BACKENDS = Path("inference") / "backends"+++class TestADirectoryIsHalfTheClaim:+ @staticmethod+ def _registry(build_surface: SurfaceBuilder) -> SurfaceRegistry:+ """The shape the real registry has after S7b: a root family and a subdirectory family."""+ return SurfaceRegistry(+ surfaces=[+ build_surface(surface_id="pipelex-config", base_file="pipelex.toml", tier_glob="pipelex_*.toml"),+ build_surface(surface_id="inference-backend", base_file=None, tier_glob="*.toml", subdirectory=BACKENDS),+ ]+ )++ @staticmethod+ def _seed(root: Path) -> None:+ """A configuration directory holding one of everything the claim rule has to separate."""+ (root / "pipelex.toml").write_text("", encoding="utf-8")+ (root / "pipelex_override.toml").write_text("", encoding="utf-8")+ (root / BACKENDS).mkdir(parents=True)+ (root / BACKENDS / "pipelex_gateway.toml").write_text("", encoding="utf-8")+ (root / BACKENDS / "openai.toml").write_text("", encoding="utf-8")+ (root / BACKENDS / "notes.md").write_text("", encoding="utf-8")+ (root / "inference" / "deck").mkdir(parents=True)+ (root / "inference" / "deck" / "x.toml").write_text("", encoding="utf-8")+ (root / "inference" / "backends.toml").write_text("", encoding="utf-8")++ def test_each_file_goes_to_the_surface_that_owns_the_directory_it_sits_in(self, tmp_path: Path, build_surface: SurfaceBuilder) -> None:+ """One assertion for the whole rule, because every line of it is a separate way to get this wrong.++ `pipelex_gateway.toml` is claimed by the backend surface and *not* by the root surface whose+ glob its name matches; `notes.md` falls out by extension; `deck/` is a directory no surface+ owns and is never entered; `inference/backends.toml` sits one level above the directory the+ backend surface owns and is nobody's.+ """+ self._seed(tmp_path)++ claimed = self._registry(build_surface).files_by_surface_in_directory(directory=tmp_path)++ assert [(surface.surface_id, path.relative_to(tmp_path).as_posix()) for surface, path in claimed] == [+ ("pipelex-config", "pipelex.toml"),+ ("pipelex-config", "pipelex_override.toml"),+ ("inference-backend", "inference/backends/openai.toml"),+ ("inference-backend", "inference/backends/pipelex_gateway.toml"),+ ]++ def test_the_same_name_resolves_to_a_different_surface_in_each_directory(self, build_surface: SurfaceBuilder) -> None:+ """The claim is the pair, so the specimen's name alone answers nothing.++ Without the directory in the question, `pipelex_gateway.toml` is a `pipelex_*.toml` match+ and the root surface takes it wherever it lives.+ """+ registry = self._registry(build_surface)++ at_the_root = registry.surface_for_file(subdirectory=Path(), file_name="pipelex_gateway.toml")+ in_the_backends_directory = registry.surface_for_file(subdirectory=BACKENDS, file_name="pipelex_gateway.toml")++ assert at_the_root is not None+ assert at_the_root.surface_id == "pipelex-config"+ assert in_the_backends_directory is not None+ assert in_the_backends_directory.surface_id == "inference-backend"++ def test_a_root_surfaces_glob_does_not_reach_into_a_directory_it_does_not_own(self, build_surface: SurfaceBuilder) -> None:+ """The other direction of the same rule: the backend directory is not the root surface's to claim in."""+ registry = SurfaceRegistry(surfaces=[build_surface(surface_id="pipelex-config", base_file="pipelex.toml", tier_glob="pipelex_*.toml")])++ assert registry.surface_for_file(subdirectory=BACKENDS, file_name="pipelex_gateway.toml") is None+ assert registry.surface_for_file(subdirectory=BACKENDS, file_name="pipelex.toml") is None++ def test_a_subdirectory_that_does_not_exist_is_skipped_rather_than_refused(self, tmp_path: Path, build_surface: SurfaceBuilder) -> None:+ """A machine that has never run an inference backend has no `inference/` at all."""+ (tmp_path / "pipelex.toml").write_text("", encoding="utf-8")++ claimed = self._registry(build_surface).files_by_surface_in_directory(directory=tmp_path)++ assert [path.name for _, path in claimed] == ["pipelex.toml"]diff --git a/tests/unit/pipelex/migration/test_surface_registry.py b/tests/unit/pipelex/migration/test_surface_registry.pyindex 85b2530f6..c9e17ca74 100644--- a/tests/unit/pipelex/migration/test_surface_registry.py+++ b/tests/unit/pipelex/migration/test_surface_registry.py@@ -23,12 +23,17 @@ class _Required(BaseModel):label: str-def _surface(*, surface_id: str, base_file: str, tier_glob: str | None = None) -> Surface:+CONFIGURATION_ROOT = Path()+BACKENDS = Path("inference") / "backends"+++def _surface(*, surface_id: str, base_file: str | None, tier_glob: str | None = None, subdirectory: Path = CONFIGURATION_ROOT) -> Surface:return Surface(surface_id=surface_id,title=surface_id,base_file=base_file,tier_glob=tier_glob,+ subdirectory=subdirectory,config_model=_Defaulted,defaults_layer_kind=DefaultsLayerKind.MODEL_DEFAULTS,)@@ -139,6 +144,50 @@ class TestRegistryConsistency:])+ def test_two_surfaces_sharing_a_tier_glob_in_different_directories_are_fine(self) -> None:+ """A glob is a claim within one directory, so the same glob shape twice is not a collision.++ `*.toml` is the obvious one: it would be the whole language of any surface that owns a+ directory of its own, and forbidding it globally would mean the second such surface has to+ invent a narrower glob for no reason but the registry's bookkeeping.+ """+ registry = SurfaceRegistry(+ surfaces=[+ _surface(surface_id="a", base_file="a.toml", tier_glob="*.toml"),+ _surface(surface_id="b", base_file=None, tier_glob="*.toml", subdirectory=BACKENDS),+ ]+ )+ assert len(registry.surfaces) == 2++ def test_two_surfaces_sharing_a_tier_glob_in_the_same_directory_are_still_refused(self) -> None:+ with pytest.raises(ValidationError, match="share the tier glob"):+ SurfaceRegistry(+ surfaces=[+ _surface(surface_id="a", base_file="a.toml", tier_glob="shared_*.toml", subdirectory=BACKENDS),+ _surface(surface_id="b", base_file="b.toml", tier_glob="shared_*.toml", subdirectory=BACKENDS),+ ]+ )++ @pytest.mark.parametrize(+ ("base_file", "tier_glob"),+ [+ (None, None),+ ("", ""),+ ("", None),+ (None, ""),+ ],+ )+ def test_a_surface_that_claims_nothing_is_refused(self, base_file: str | None, tier_glob: str | None) -> None:+ """A surface with no base file must say which files in its directory are its own.++ Empty counts as none: `fnmatch(name, "")` matches nothing and no file is named `""`, so an+ empty spelling claims exactly as little as an absent one while looking like a declaration.+ The ledger block carries the same guard and is parametrized the same way — one rule, and it+ has to be one rule in both places or the coverage gate is comparing two different questions.+ """+ with pytest.raises(ValidationError, match="claims no file"):+ _surface(surface_id="a", base_file=base_file, tier_glob=tier_glob, subdirectory=BACKENDS)+def test_a_base_file_matching_another_surfaces_glob_is_fine(self) -> None:"""This is the real configuration: `pipelex_service.toml` matches `pipelex_*.toml`.diff --git a/tests/unit/pipelex/migration/test_surface_resolution.py b/tests/unit/pipelex/migration/test_surface_resolution.pyindex ee1bb17f1..64cd381f3 100644--- a/tests/unit/pipelex/migration/test_surface_resolution.py+++ b/tests/unit/pipelex/migration/test_surface_resolution.py@@ -3,6 +3,9 @@Which ledger runs over a file must never be an accident of iteration order. The registry refuseswhat it can decide by name alone when it loads; what it cannot decide — whether two glob languagesoverlap — is decided here, where there is a real file to point at.++A claim is the pair (directory, name); this module is the *name* half, with every file at the root+of a configuration directory. `test_surface_directories.py` is the directory half."""import re@@ -12,7 +15,7 @@ import pytestfrom pipelex.migration.exceptions import MigrationRegistryErrorfrom pipelex.migration.surfaces import SurfaceRegistry-from tests.unit.pipelex.migration.conftest import SurfaceBuilder+from tests.unit.pipelex.migration.conftest import CONFIGURATION_ROOT, SurfaceBuilderclass TestSurfaceResolution:@@ -25,7 +28,7 @@ class TestSurfaceResolution:])- resolved = registry.surface_for_file_name(file_name="pipelex_service.toml")+ resolved = registry.surface_for_file(subdirectory=CONFIGURATION_ROOT, file_name="pipelex_service.toml")assert resolved is not Noneassert resolved.surface_id == "pipelex-service-config"@@ -33,7 +36,7 @@ class TestSurfaceResolution:def test_a_tier_file_resolves_to_the_surface_whose_glob_matches(self, build_surface: SurfaceBuilder) -> None:registry = SurfaceRegistry(surfaces=[build_surface(surface_id="pipelex-config", base_file="pipelex.toml", tier_glob="pipelex_*.toml")])- resolved = registry.surface_for_file_name(file_name="pipelex_staging.toml")+ resolved = registry.surface_for_file(subdirectory=CONFIGURATION_ROOT, file_name="pipelex_staging.toml")assert resolved is not Noneassert resolved.surface_id == "pipelex-config"@@ -41,7 +44,7 @@ class TestSurfaceResolution:def test_a_file_no_surface_claims_resolves_to_nothing(self, build_surface: SurfaceBuilder) -> None:registry = SurfaceRegistry(surfaces=[build_surface(surface_id="pipelex-config", base_file="pipelex.toml", tier_glob="pipelex_*.toml")])- assert registry.surface_for_file_name(file_name="plxt.toml") is None+ assert registry.surface_for_file(subdirectory=CONFIGURATION_ROOT, file_name="plxt.toml") is Nonedef test_a_file_two_globs_both_claim_stops_by_name(self, build_surface: SurfaceBuilder) -> None:"""The registry accepts these globs — deciding overlap between glob *languages* is not its@@ -58,7 +61,7 @@ class TestSurfaceResolution:)with pytest.raises(MigrationRegistryError, match=re.escape("pipelex_local.toml")):- registry.surface_for_file_name(file_name="pipelex_local.toml")+ registry.surface_for_file(subdirectory=CONFIGURATION_ROOT, file_name="pipelex_local.toml")def test_a_directory_walk_returns_the_claimed_files_with_their_surfaces(self,diff --git a/tests/unit/pipelex/migration/test_transform_check.py b/tests/unit/pipelex/migration/test_transform_check.pyindex c85993f01..4bdc72188 100644--- a/tests/unit/pipelex/migration/test_transform_check.py+++ b/tests/unit/pipelex/migration/test_transform_check.py@@ -239,7 +239,7 @@ class TestTheTransformGoldens:assert check_transform_chain(surface=_surface(config_model=_WithOptional), migration_dir=tmp_path) == []def test_a_surface_that_has_never_changed_shape_has_no_link_to_check(self, tmp_path: Path) -> None:- """The overwhelmingly common state, and the one all three surfaces are in today."""+ """The overwhelmingly common state, and the one every shipped surface is in today."""_write_ledger(migration_dir=tmp_path, entries="", current_schema_version=1)_defaults(migration_dir=tmp_path, schema_version=1, document='label = "hello"\n')assert check_transform_chain(surface=_surface(config_model=_Renamed), migration_dir=tmp_path) == []diff --git a/tests/unit/pipelex/pipeline/fixes/test_fix_applier_config_surface_shapes.py b/tests/unit/pipelex/pipeline/fixes/test_fix_applier_config_surface_shapes.pyindex 05b5c85b8..a2a3412eb 100644--- a/tests/unit/pipelex/pipeline/fixes/test_fix_applier_config_surface_shapes.py+++ b/tests/unit/pipelex/pipeline/fixes/test_fix_applier_config_surface_shapes.py@@ -52,7 +52,7 @@ _ARRAY_OF_TABLES_FILE = Path("pipelex/kit/configs/plxt.toml")# The out-of-order specimen. The packaged defaults used to be written out of order — `[pipelex]` and# `[migration]` each split into several chunks — and the proxy tests below were written against them.-# The configuration reshape (`pipelex-config@2`) rewrote that file in order, so the live document+# The configuration reshape (`pipelex-config@3`) rewrote that file in order, so the live document# produces no proxy at all and can no longer witness the behaviour. Its schema-1 golden can: it is a# byte copy of the pre-reshape file, and every golden below the chain's head is frozen, so this# specimen keeps the shape permanently. Reaching for it here is deliberate — the alternative, adiff --git a/tests/unit/pipelex/system/configuration/test_boot_tolerance.py b/tests/unit/pipelex/system/configuration/test_boot_tolerance.pyindex 9b9b1a373..3adeb585d 100644--- a/tests/unit/pipelex/system/configuration/test_boot_tolerance.py+++ b/tests/unit/pipelex/system/configuration/test_boot_tolerance.py@@ -15,9 +15,10 @@ Two properties carry the whole design, and most of the tests below exist to holdThe stale documents here are **real**: `telemetry-config@2` is the entry the package ships and`goldens/telemetry-config/[email protected]` is the flat document it exists to carry forward, read-live rather than copied. The other two surfaces have empty ledgers today, so their tests plant a-synthetic one in a temporary migration directory — the wiring is what is under test there, and a-synthetic ledger tests it without inventing a schema change nobody made.+live rather than copied. The tests on the other surfaces plant a synthetic ledger in a temporary+migration directory instead: what is under test there is the wiring — which loader retries, over+which files, and what it warns — and a synthetic ledger holds that still while the shipped ledgers+go on gaining entries underneath it."""from __future__ import annotations@@ -63,7 +64,15 @@ def old_shape_telemetry_document() -> str:return path.read_text(encoding="utf-8")-def write_synthetic_ledger(*, migration_dir: Path, surface_id: str, base_file: str, ops_body: str, min_supported_schema_version: int = 0) -> None:+def write_synthetic_ledger(+ *,+ migration_dir: Path,+ surface_id: str,+ base_file: str,+ ops_body: str,+ min_supported_schema_version: int = 0,+ subdirectory: str = "",+) -> None:"""A one-entry ledger for a surface whose real ledger has nothing in it yet."""ledgers_dir = migration_dir / "ledgers"ledgers_dir.mkdir(parents=True, exist_ok=True)@@ -73,6 +82,7 @@ def write_synthetic_ledger(*, migration_dir: Path, surface_id: str, base_file: sid = "{surface_id}"title = "A surface under test"base_file = "{base_file}"+subdirectory = "{subdirectory}"current_schema_version = 2min_supported_schema_version = {min_supported_schema_version}@@ -266,6 +276,12 @@ class TestTheRetryHonoursTheSchemaVersionFloor:class TestWhatTheWarningSays:+ RENAME_OPS = (+ '[[migration.ops]]\nkind = "rename_table_key"\ntable_path = ["runtime", "log"]\n'+ 'key = "old_default_log_level"\nnew_key = "default_log_level"\n'+ )+ STALE_BODY = '[runtime.log]\nold_default_log_level = "DEBUG"\n'+@staticmethoddef _stale_telemetry_plans(*, directory: Path, body: str | None = None) -> tuple[Path, list[MigrationPlan]]:stale = directory / "telemetry.toml"@@ -333,8 +349,13 @@ class TestWhatTheWarningSays:assert f"Run `pipelex migrate` to bring '{inside_file}' up to date." in warningassert f"`pipelex migrate` does not reach '{outside_file}' — that file is yours to update where it lives." in warning- def test_the_walk_is_not_recursive_so_a_subdirectory_is_out_of_reach(self, tmp_path: Path) -> None:- """`.pipelex/inference/backends/` is the specimen the walk deliberately skips."""+ def test_a_file_in_a_directory_its_own_surface_does_not_own_is_out_of_reach(self, tmp_path: Path) -> None:+ """The walk reaches `inference/backends/` now — but for the surface that owns it, not for any file in it.++ A `telemetry.toml` sitting there is out of reach exactly as it was when the walk stopped at+ the top level, because the telemetry surface's own directory is the configuration directory+ itself. Reach is per surface, not per path.+ """nested = tmp_path / "inference" / "backends"nested.mkdir(parents=True)_, plans = self._stale_telemetry_plans(directory=nested)@@ -344,6 +365,59 @@ class TestWhatTheWarningSays:assert "does not reach" in warningassert "Run `pipelex migrate`" not in warning+ def test_a_file_in_the_directory_its_surface_owns_is_in_reach(self, tmp_path: Path, synthetic_migration_dir: Path) -> None:+ """The other half, and the one the fourth surface needs: reach follows the surface's own directory.++ The ledger is synthetic because the reach is read from the ledger rather than from the+ surface registry — this module's own layering rule — so the ledger *is* what is under test+ here, and a synthetic one exercises it without waiting on a real surface to own a+ subdirectory.+ """+ write_synthetic_ledger(+ migration_dir=synthetic_migration_dir,+ surface_id=PIPELEX_CONFIG_SURFACE_ID,+ base_file="pipelex.toml",+ subdirectory="inference/backends",+ ops_body=self.RENAME_OPS,+ )+ owned = tmp_path / "inference" / "backends"+ owned.mkdir(parents=True)+ stale = owned / "pipelex.toml"+ stale.write_text(self.STALE_BODY, encoding="utf-8")+ replayed = replay_surface_files_in_memory(surface_id=PIPELEX_CONFIG_SURFACE_ID, paths=[stale])+ assert replayed is not None++ warning = stale_configuration_warning(plans=replayed.plans, walked_dirs=[tmp_path])++ assert "Run `pipelex migrate` to bring the files up to date." in warning+ assert "does not reach" not in warning++ def test_a_directory_no_surface_owns_stays_out_of_reach(self, tmp_path: Path, synthetic_migration_dir: Path) -> None:+ """`inference/deck/` is beside a directory the walk now enters and is still never entered itself.++ The pair with the test above is the point: the same surface, the same configuration+ directory, one directory deeper on either side of the claim — and only the one the ledger+ names is offered the command.+ """+ write_synthetic_ledger(+ migration_dir=synthetic_migration_dir,+ surface_id=PIPELEX_CONFIG_SURFACE_ID,+ base_file="pipelex.toml",+ subdirectory="inference/backends",+ ops_body=self.RENAME_OPS,+ )+ unowned = tmp_path / "inference" / "deck"+ unowned.mkdir(parents=True)+ stale = unowned / "pipelex.toml"+ stale.write_text(self.STALE_BODY, encoding="utf-8")+ replayed = replay_surface_files_in_memory(surface_id=PIPELEX_CONFIG_SURFACE_ID, paths=[stale])+ assert replayed is not None++ warning = stale_configuration_warning(plans=replayed.plans, walked_dirs=[tmp_path])++ assert "does not reach" in warning+ assert "Run `pipelex migrate`" not in warning+def test_a_file_symlinked_out_of_a_walked_directory_is_still_in_reach(self, tmp_path: Path) -> None:"""Where the file points is not where the command looks for it.diff --git a/tests/unit/pipelex/system/pipelex_service/test_pipelex_service_config.py b/tests/unit/pipelex/system/pipelex_service/test_pipelex_service_config.pyindex 51257a4f6..219777e86 100644--- a/tests/unit/pipelex/system/pipelex_service/test_pipelex_service_config.py+++ b/tests/unit/pipelex/system/pipelex_service/test_pipelex_service_config.py@@ -3,11 +3,15 @@import osimport pathlibimport tempfile+from pathlib import Path++import pytestfrom pipelex.system.pipelex_service.pipelex_service_config import (PIPELEX_SERVICE_CONFIG_FILE_NAME,PipelexServiceAgreement,PipelexServiceConfig,+ is_pipelex_gateway_enabled,load_pipelex_service_config_if_exists,)@@ -15,6 +19,46 @@ from pipelex.system.pipelex_service.pipelex_service_config import (class TestPipelexServiceConfig:"""Tests for PipelexServiceConfig and related functions."""+ @pytest.mark.parametrize(+ ("enabled_value", "expected"),+ [+ ("true", True),+ ("false", False),+ # Not booleans, and exactly what the loader also treats as truthy — the two readers of+ # `backends.toml` must agree, or the boot fetches no gateway specs for a backend the+ # loader then insists is enabled.+ ("1", True),+ ("0", False),+ ('"yes"', True),+ ],+ )+ def test_is_pipelex_gateway_enabled_reads_enabled_the_way_the_loader_does(self, tmp_path: Path, enabled_value: str, expected: bool) -> None:+ """A `pipelex_gateway` table with a non-boolean `enabled` is enabled iff the loader would load it.++ `InferenceBackendLibrary.load` gates on the raw value's truthiness before it validates+ anything, and `RuntimeBoot` decides whether to fetch the gateway's model specs from *this*+ function over the same file. Reading `enabled` as "is the literal `true`" here left `enabled = 1`+ in a state the loader could not name: enabled with no specs, so refused as *"remote model specs+ were not provided"* on a strict boot, and silently dropped from the deck on a lenient one.+ """+ backends_file = tmp_path / "backends.toml"+ backends_file.write_text(f'[pipelex_gateway]\nenabled = {enabled_value}\napi_key = "x"\n', encoding="utf-8")++ assert is_pipelex_gateway_enabled(backends_file_path=backends_file) is expected++ def test_is_pipelex_gateway_enabled_defaults_to_enabled_when_the_key_is_absent(self, tmp_path: Path) -> None:+ """No `enabled` key means enabled — the loader's default too."""+ backends_file = tmp_path / "backends.toml"+ backends_file.write_text('[pipelex_gateway]\napi_key = "x"\n', encoding="utf-8")++ assert is_pipelex_gateway_enabled(backends_file_path=backends_file) is True++ def test_is_pipelex_gateway_enabled_is_false_without_a_gateway_table(self, tmp_path: Path) -> None:+ backends_file = tmp_path / "backends.toml"+ backends_file.write_text('[openai]\napi_key = "x"\n', encoding="utf-8")++ assert is_pipelex_gateway_enabled(backends_file_path=backends_file) is False+def test_gateway_config_default_terms_not_accepted(self) -> None:"""Test that gateway terms_accepted defaults to False."""config = PipelexServiceAgreement()diff --git a/tests/unit/pipelex/test_runtime_boot_stale_backend_error.py b/tests/unit/pipelex/test_runtime_boot_stale_backend_error.pynew file mode 100644index 000000000..e63c07f88--- /dev/null+++ b/tests/unit/pipelex/test_runtime_boot_stale_backend_error.py@@ -0,0 +1,280 @@+"""When the backend loader's retry declines, the error still says *old* rather than only *wrong*.++The fourth surface reaches a user through two paths, and until this test only one of them knew it+was a surface. `InferenceBackendLibrary.load` replays the ledger in memory and boots with a warning+when that works — but when it does not, the refusal travels out of `ModelManager.setup` and+`RuntimeBoot` turns it into a message of its own instead of going through the shared+`raise_config_setup_error`. That message named no surface, so the scan never ran, and a machine whose+backend file was *both* stale and carrying something we cannot explain was told only about the second+half — while the very same staleness in `pipelex.toml` produced the migration block.++Four things under test, and the first one is the one a hand-built exception cannot check:++- **The boot catches the class the loader actually raises.** Every refusal below is produced by+ running the real `InferenceBackendLibrary.load` over a planted directory, and the first assertion+ on each is that `BACKEND_LIBRARY_REFUSED` covers it. A fixture that constructed the exception+ itself passed while the clause named a sibling class that is raised nowhere, and the whole feature+ was unreachable underneath it. The library index file is covered too: a bad value in+ `backends.toml` used to leave the loader as pydantic's own error, which no clause named.+- **The refusal's own account survives.** The loader says *which model, which backend, which file*+ before it quotes the pydantic analysis, and a directory holds a dozen files with the same field+ names — so a message reduced to `max_tokens: Input should be a valid integer` sends the reader to+ grep. Whatever the message builder adds, it adds *around* the loader's sentence.+- **A surface's failure carries its migration block.** The contract's rule is that every configuration+ surface's loader reports through the migration scan; this is the fourth surface honouring it. Both+ refusal shapes are covered, because only one of them carries a pydantic error: an unknown key in+ `[defaults]` is `extra_forbidden` on every model of the file, while an unknown key on one model+ table is rejected by name and arrives with no `__cause__` at all.+- **And then nothing offers to start over.** The tail that sends a user to `pipelex init config` — a+ command described in the same breath as resetting files to their defaults — must not be printed+ beside a block that just promised `pipelex migrate` keeps their values. The model deck, which has no+ ledger and gets no block, keeps that tail exactly as it was.+"""++from __future__ import annotations++import shutil+from pathlib import Path+from typing import TYPE_CHECKING++import pytest++from pipelex.cogt.model_backends.backend_library import InferenceBackendLibrary+from pipelex.kit.paths import get_kit_configs_dir+from pipelex.runtime_boot import BACKEND_LIBRARY_REFUSED, BootComponent, RuntimeBoot+from pipelex.system.configuration.config_loader import BACKENDS_DIR_NAME, CONFIG_DIR_NAME, INFERENCE_DIR_NAME+from pipelex.tools.secrets.env_secrets_provider import EnvSecretsProvider++if TYPE_CHECKING:+ from pytest_mock import MockerFixture++RETIRED_KEY = "prompting_target"+"""What `#1104` removed, and what the ledger entry explains."""++UNEXPLAINED_KEY = "not_a_setting_we_ever_had"+"""What nothing explains, and what keeps the retry from succeeding. Carries no hyphen on purpose: a+header-shaped key would be a legal per-model key rather than a refusal."""++REGENERATE_TAIL = "pipelex init config"++MIGRATION_PROSE = "migrat"+"""The stem shared by every sentence the migration block adds — `migration history`, `pipelex migrate`,+`--dry-run` — and by nothing the loader says on its own."""++BACKENDS_TOML = """+[openai]+enabled = true+api_key = "sk-not-a-real-key"+"""++BACKENDS_TOML_WITH_A_BAD_VALUE = """+[openai]+enabled = true+api_key = "sk-not-a-real-key"+endpoint = 42+"""+"""The library index file refusing on a *known* field: `endpoint` is a string. This is not a+per-model file and belongs to no surface, so the message it earns is the plain one."""+++class TestAStaleBackendFileTheLedgerCannotFullyExplain:+ @pytest.fixture+ def machine(self, tmp_path: Path, mocker: MockerFixture) -> Path:+ """A global configuration directory holding a fresh copy of the kit's backend files.++ The home is faked so that the scan walks this directory rather than the developer's own — it+ is `config_manager.existing_config_dirs` on both ends, the one derivation.+ """+ fake_home = tmp_path / "home"+ global_dir = fake_home / CONFIG_DIR_NAME+ inference_dir = global_dir / INFERENCE_DIR_NAME+ inference_dir.mkdir(parents=True)+ shutil.copytree(Path(str(get_kit_configs_dir())) / INFERENCE_DIR_NAME / BACKENDS_DIR_NAME, inference_dir / BACKENDS_DIR_NAME)+ (inference_dir / "backends.toml").write_text(BACKENDS_TOML, encoding="utf-8")++ project_root = tmp_path / "project"+ (project_root / ".git").mkdir(parents=True)+ mocker.patch.object(Path, "home", return_value=fake_home)+ mocker.patch.object(Path, "cwd", return_value=project_root)++ return global_dir++ def _plant(self, *, path: Path, anchor: str, keys: str) -> None:+ text = path.read_text(encoding="utf-8")+ if anchor not in text:+ msg = f"'{path.name}' no longer has a {anchor.strip()} table to plant into — the fixture is testing nothing."+ raise AssertionError(msg)+ path.write_text(text.replace(anchor, f"{anchor}{keys}", 1), encoding="utf-8")++ def _refusal_from_the_real_loader(self, *, machine: Path) -> Exception:+ """Whatever `InferenceBackendLibrary.load` raises over this directory, unwrapped by nobody.++ The point of going through the loader rather than building the exception: the class it raises+ is exactly what the boot's `except` clause has to name, and that is the link that was broken.+ """+ library = InferenceBackendLibrary.make_empty()+ with pytest.raises(BACKEND_LIBRARY_REFUSED) as refused:+ library.load(+ secrets_provider=EnvSecretsProvider(),+ backends_library_path=str(machine / INFERENCE_DIR_NAME / "backends.toml"),+ backends_dir_path=str(machine / INFERENCE_DIR_NAME / BACKENDS_DIR_NAME),+ gateway_config=None,+ lenient=False,+ )+ return refused.value++ @pytest.fixture+ def stale_and_unexplained_in_defaults(self, machine: Path) -> tuple[Path, Exception]:+ """`[defaults]` carries both keys: every model of the file fails with a pydantic error."""+ stale_file = machine / INFERENCE_DIR_NAME / BACKENDS_DIR_NAME / "openai.toml"+ self._plant(path=stale_file, anchor="[defaults]\n", keys=f'{RETIRED_KEY} = "openai"\n{UNEXPLAINED_KEY} = 1\n')+ return stale_file, self._refusal_from_the_real_loader(machine=machine)++ @pytest.fixture+ def both_keys_on_one_model_table(self, machine: Path) -> tuple[Path, Exception]:+ """The shape with **no** pydantic error at all: keys rejected by name rather than by the model.++ Both on the *same* table, and both halves of that matter. On a model table rather than in+ `[defaults]`, so nothing is merged into another model and no `extra_forbidden` fires — the+ refusal is `split_model_spec_keys`' own, and it arrives with no `__cause__`. On one table+ rather than two, so the refusal names the key the user has to fix: split across two models,+ the first table to fail is the *stale* one, the retry then repairs it and the second table+ refuses, and what is re-raised is the first error — which names only the key a migration+ would have removed.+ """+ stale_file = machine / INFERENCE_DIR_NAME / BACKENDS_DIR_NAME / "openai.toml"+ self._plant(path=stale_file, anchor="[gpt-4o]\n", keys=f'{RETIRED_KEY} = "openai"\n{UNEXPLAINED_KEY} = 1\n')+ return stale_file, self._refusal_from_the_real_loader(machine=machine)++ def test_the_refusal_the_loader_raises_is_one_the_boot_handles(self, stale_and_unexplained_in_defaults: tuple[Path, Exception]) -> None:+ """The link that was missing: `pytest.raises(BACKEND_LIBRARY_REFUSED)` above is the assertion.++ Kept as a test of its own rather than left implicit in the fixture, because it is the claim+ the other cases all rest on — reach the message builder with an exception the boot never+ catches and every assertion below is about code no user runs.+ """+ _, refusal = stale_and_unexplained_in_defaults++ assert isinstance(refusal, BACKEND_LIBRARY_REFUSED)++ @pytest.mark.parametrize(+ "fixture_name",+ ["stale_and_unexplained_in_defaults", "both_keys_on_one_model_table"],+ )+ def test_the_message_names_the_stale_file_and_the_migration_that_explains_it(self, request: pytest.FixtureRequest, fixture_name: str) -> None:+ """Both refusal shapes, because only one of them carries a pydantic error to translate."""+ stale_file, refusal = request.getfixturevalue(fixture_name)++ message = RuntimeBoot._get_validation_error_msg( # noqa: SLF001 # pyright: ignore[reportPrivateUsage]+ component=BootComponent.INFERENCE_BACKEND_LIBRARY,+ validation_exc=refusal,+ )++ assert UNEXPLAINED_KEY in message, "the user's own key is still what the error is about"+ assert str(stale_file) in message+ assert "Drop prompting_target from every backend definition" in message+ assert "pipelex migrate" in message++ @pytest.mark.parametrize(+ "fixture_name",+ ["stale_and_unexplained_in_defaults", "both_keys_on_one_model_table"],+ )+ def test_it_stops_offering_to_start_over_once_it_has_offered_to_migrate(self, request: pytest.FixtureRequest, fixture_name: str) -> None:+ """Two remedies, one of which discards the file, is worse advice than either alone."""+ _, refusal = request.getfixturevalue(fixture_name)++ message = RuntimeBoot._get_validation_error_msg( # noqa: SLF001 # pyright: ignore[reportPrivateUsage]+ component=BootComponent.INFERENCE_BACKEND_LIBRARY,+ validation_exc=refusal,+ )++ assert "pipelex migrate" in message, "the tail is dropped because a block was found, not because none was looked for"+ assert REGENERATE_TAIL not in message++ def test_a_component_with_no_ledger_is_unchanged(self, stale_and_unexplained_in_defaults: tuple[Path, Exception]) -> None:+ """The model deck shares this message and is not a surface: no scan, no block, same tail as before."""+ _, refusal = stale_and_unexplained_in_defaults++ message = RuntimeBoot._get_validation_error_msg( # noqa: SLF001 # pyright: ignore[reportPrivateUsage]+ component=BootComponent.MODEL_DECK,+ validation_exc=refusal,+ )++ assert REGENERATE_TAIL in message+ assert MIGRATION_PROSE not in message++ def test_a_bad_value_on_a_known_field_keeps_the_plain_message_and_the_loaders_own_account(self, machine: Path) -> None:+ """No scan result, no block — naming the surface must not invent one. And nothing is lost on the way.++ A **wrong value on a field the schema still has** is the refusal that leaves the scan with+ nothing to report: every path of the file resolves, so the diagnosis finds none unexplained+ and no entry applies. An unknown *key* would not do here, and that is worth knowing rather+ than working around — it is an unexplained path, so it earns a block of its own with+ `would_write` false, pointing at `migrate --dry-run` instead of at a rewrite. This case is the+ one where the message must be exactly what it was before the surface existed.++ It is also the case that showed the loader's sentence being thrown away: this refusal carries+ a pydantic error, and a builder that translated *that* and dropped the rest reduced the whole+ message to `max_tokens: Input should be a valid integer` — no model, no backend, no file, in a+ directory of a dozen files that all have a `max_tokens`. The pydantic analysis stays, and the+ loader's *where* stays around it.+ """+ stale_file = machine / INFERENCE_DIR_NAME / BACKENDS_DIR_NAME / "openai.toml"+ self._plant(path=stale_file, anchor="[gpt-4o]\n", keys='max_tokens = "lots"\n')+ refusal = self._refusal_from_the_real_loader(machine=machine)++ message = RuntimeBoot._get_validation_error_msg( # noqa: SLF001 # pyright: ignore[reportPrivateUsage]+ component=BootComponent.INFERENCE_BACKEND_LIBRARY,+ validation_exc=refusal,+ )++ assert "max_tokens" in message+ assert "'gpt-4o'" in message, "the model whose table holds the bad value"+ assert "'openai'" in message, "the backend"+ assert str(stale_file) in message, "the file, which the pydantic analysis alone never names"+ assert MIGRATION_PROSE not in message+ assert REGENERATE_TAIL in message++ def test_a_bad_value_in_the_library_index_is_a_refusal_the_boot_handles(self, machine: Path) -> None:+ """`backends.toml` is the other file the loader reads, and it refused in a class nobody named.++ A wrong value on a backend's own table — `endpoint = 42` — failed `InferenceBackendBlueprint`+ and left `load` as pydantic's bare `ValidationError`, so it reached the user as a traceback+ naming neither the backend nor the file. `pytest.raises(BACKEND_LIBRARY_REFUSED)` in the+ helper is again the assertion that matters; the rest is that the message says where.+ """+ (machine / INFERENCE_DIR_NAME / "backends.toml").write_text(BACKENDS_TOML_WITH_A_BAD_VALUE, encoding="utf-8")+ refusal = self._refusal_from_the_real_loader(machine=machine)++ message = RuntimeBoot._get_validation_error_msg( # noqa: SLF001 # pyright: ignore[reportPrivateUsage]+ component=BootComponent.INFERENCE_BACKEND_LIBRARY,+ validation_exc=refusal,+ )++ assert "endpoint" in message+ assert "'openai'" in message, "the backend whose table holds the bad value"+ assert str(machine / INFERENCE_DIR_NAME / "backends.toml") in message+ assert MIGRATION_PROSE not in message, "the index file belongs to no surface, and the per-model files are pristine"+ assert REGENERATE_TAIL in message++ def test_an_unexplained_key_alone_earns_a_block_that_points_at_the_dry_run(self, machine: Path) -> None:+ """The other side of the case above, and the second thing F2's fix buys.++ A file that is wrong without being old still has something the migration channel can say: the+ path nothing explains. The block is there, `would_write` is false, and the prose sends the+ reader to `migrate --dry-run` rather than promising a rewrite — so the start-over tail goes,+ because the reader has been given a diagnosis to read first.+ """+ stale_file = machine / INFERENCE_DIR_NAME / BACKENDS_DIR_NAME / "openai.toml"+ self._plant(path=stale_file, anchor="[defaults]\n", keys=f"{UNEXPLAINED_KEY} = 1\n")+ refusal = self._refusal_from_the_real_loader(machine=machine)++ message = RuntimeBoot._get_validation_error_msg( # noqa: SLF001 # pyright: ignore[reportPrivateUsage]+ component=BootComponent.INFERENCE_BACKEND_LIBRARY,+ validation_exc=refusal,+ )++ assert UNEXPLAINED_KEY in message+ assert str(stale_file) in message+ assert "pipelex migrate --dry-run" in message+ assert "Drop prompting_target" not in message, "no entry applies, so nothing may claim one carried anything forward"diff --git a/wip/config-errors-toml-syntax-and-cli-panels.md b/wip/config-errors-toml-syntax-and-cli-panels.mdnew file mode 100644index 000000000..b962699b4--- /dev/null+++ b/wip/config-errors-toml-syntax-and-cli-panels.md@@ -0,0 +1,89 @@+# Follow-up brief — a broken configuration file should fail like a configuration error, on both CLIs++Two items found while polishing the boot's configuration error messages on `feature/Backend-surface` (S7b, PR #1116) and **deliberately left out of that branch**. They are independent of the fourth migration surface and of each other. This brief is written from what was measured during that session; the agent picking it up should do the actual exploration — grep every site, read every handler, run every scenario — rather than trust the site lists below as complete.++**Where this stands relative to the branch that found it.** S7b fixed four error-message defects on the boot's *validation* path: the unreachable `except`, the `[defaults]` header silence, the loader's own sentence being reduced to the pydantic analysis, and a bad value in `backends.toml` escaping as a bare pydantic error. All of those are about a file that **parses** and then **fails validation**. Neither item here touches that path; both are about what happens *around* it. Do them after S7b lands, so nothing rebases across the boot handler while it is under review.++## Item 1 — a TOML *syntax* error in a configuration file is classified as the user's *input* being wrong++### What was measured++Append `[broken\nfoo = 1` to `~/.pipelex/pipelex.toml` in an isolated fake home (an empty project dir as cwd, so the repo's own `.pipelex/` does not shadow it), then:++- `pipelex show backends` → a Rich traceback panel of a few dozen frames, then the one useful line: `TomlError: TOML parsing error in file '…/.pipelex/pipelex.toml': Expected ']' at the end of a table declaration`.+- `pipelex-agent models --format json` → a proper JSON envelope, **but** with `"error_type": "TomlError"`, **`"error_domain": "input"`**, and the hint *"Initialization failed. Run 'pipelex-agent doctor' to diagnose, or 'pipelex init config' to reset configuration"*.++The same happens for `backends.toml` and for a per-backend file under `inference/backends/` (measured on `ollama.toml`); it is uniform across surfaces because nothing between `load_toml_from_path` and the CLI root catches `TomlError` for a configuration file.++### Why it is wrong, not merely unpolished++- **`error_domain` is the contract field.** The agent-hook specification branches on it (see the workspace-root `CLAUDE.md` § *Surface output conventions*, and `docs/specs/` for the hook pipeline). `input` says *the input you gave this run was wrong*; a broken `pipelex.toml` is `config`. An agent following the spec will treat its own request as the mistake.+- **The hint is the "start over" advice S7b spent two fixes removing** from the migration path: `pipelex init config` resets every file, and the honest hint is *fix line N of this file*.+- **The position is dropped from the message.** `pipelex/tools/misc/toml_utils.py::load_toml_from_path` composes the message from `exc.msg` — the bare tomli message — and stores `lineno`/`colno` on the `TomlError` as attributes only. The user reads "Expected ']' at the end of a table declaration" with no line. (Memory note `reference_tomli_vs_tomllib_error_attrs` is about this attribute surface: stdlib `tomllib` exposes them only from 3.14; we are on `tomli`.)++### Where the classification comes from++`pipelex/cli/agent_cli/commands/agent_output.py` maps `"TomlError"` → `"input"` in its error-domain table (line ~186 at the time of writing) and carries a `TomlError` hint (line ~155). That mapping is **right** for the other producers of `TomlError` — an inputs file (`cli/commands/run/_run_core.py`, `agent_cli/commands/run/stdin_resolver.py`) or a `.mthds` bundle (`mthds_parsing/parser.py` catches it) — and wrong only for a configuration file. **The CLI cannot tell those apart by class; only the loader that opened the file knows which kind it was.** So the fix is not in `agent_output.py`.++### What to do++Convert `TomlError` into that surface's own configuration error **at each configuration loader**, naming the file *and the line and column*, so it arrives at both CLIs as a `PipelexConfigError`-family error with `error_domain = config` and the same shape as a validation refusal — including being catchable by the boot's existing arms (`CONFIG_REFUSED` for the main config in `runtime_boot.py`, `BACKEND_LIBRARY_REFUSED` for the backend library) so the human CLI message builder and `raise_config_setup_error` see it like any other refusal.++Sites known from the session (**verify by grepping callers of `load_toml_from_path` / `load_toml_from_path_if_exists` under `pipelex/system/configuration/`, `pipelex/system/telemetry/`, `pipelex/system/pipelex_service/`, `pipelex/cogt/model_backends/`, `pipelex/cogt/models/`, `pipelex/cogt/routing/` — the list below is what was seen, not a census**):++- the main configuration tiers (`ConfigLoader` in `pipelex/system/configuration/config_loader.py` — this is the one whose refusal is `ConfigValidationError`, and it should keep raising in that family so `CONFIG_REFUSED` catches it);+- `telemetry.toml` and `pipelex_service.toml` loaders;+- `backends.toml` (`InferenceBackendLibrary.load`, the `load_toml_from_path` at the top — today it catches only `FileNotFoundError` there; the right class is `InferenceBackendLibraryValidationError`, which S7b just gave its first real raise site, or a sibling — decide whether "does not parse" and "does not validate" want distinct classes for the doctor's per-backend probe, which greps messages);+- per-backend model-spec files (`_load_local_model_specs`, same file) and the gateway local override (`load_toml_from_path_if_exists` in `_load_gateway_model_specs`);+- routing profiles and model decks (their loaders were not opened during the session — check whether they go through the same helper).++Decide once whether the position goes into the message at `load_toml_from_path` itself (one site, every consumer benefits — but check the `.mthds` parser and the inputs-file paths, which may already append it) or per loader.++### What must not change++- A configuration that will not *parse* has no ledger story: the migration engine reads the document with tomlkit and a file that will not parse is a `blocked_reason` in its plan already (`FileBlockedReason` — check the enum for the parse-failure member). Do not try to route a syntax error through `report_validation_error`'s scan; there is nothing to scan. It is a plain configuration error with a file and a line.+- Boot tolerance (`replay_surface_files_in_memory`) must not attempt to replay an unparseable file. Confirm the retry declines cleanly rather than raising something new; the rule from S7b is that nothing that goes wrong inside the retry may replace the error the user already has.+- The other `TomlError` producers keep `error_domain = input`. Add a test on the agent CLI that a broken inputs file still reports `input` and a broken `pipelex.toml` reports `config` — that pair is the whole point.++### Tests to write++- Per surface, a unit test that plants a syntax error and asserts the loader raises the surface's config error naming the file and the line — from the real loader, never a hand-built exception (memory: `reference_a_hand_built_exception_cannot_prove_a_handler_catches_it`).+- The agent-CLI domain pair above.+- Mutation-check the boot: narrow the arm back and watch the syntax case go red.++## Item 2 — the human CLI renders every boot configuration failure as a traceback++### What was measured++Every configuration failure that leaves the boot as `PipelexSetupError` or `PipelexConfigError` — the stale-file message with its migration block, the model/backend/file sentence S7b just restored, a bad value in `backends.toml` — prints in the human CLI as **a Rich traceback panel with the message underneath**. The messages are good now; the reader finds them at the bottom of forty frames of `runtime_boot.py` / `cli_factory.py`.++`pipelex/cli/cli_factory.py::make_pipelex_for_cli` catches a hand-picked list — `InferenceSetupRequiredError`, `TelemetryConfigValidationError`, `GatewayTermsNotAcceptedError`, `GatewayApiKeyMissingError`, `GatewayDoNotTrackConflictError`, `RemoteConfigUnavailableError`, `RemoteConfigValidationError`, `GatewayUnknownModelError` — and routes each to a handler in `pipelex/cli/error_handlers.py` (`display_error_panel`, and per-class handlers). `PipelexSetupError` and `PipelexConfigError` are **not on the list**, so they fall through to typer's pretty-exception rendering (`pretty_exceptions_show_locals=False` in `pipelex/cli/_cli.py`). `error_handlers.py` already has the machinery the fix wants: `display_error_panel`, `set_traceback_requested` / `is_traceback_requested`, `print_traceback_if_requested` — a `--traceback` flag exists precisely so that the default is a panel and the traceback is opt-in.++This is **not** TOML-specific and not S7b-specific; item 1 merely lands one more class in the same gap.++### What to do++Route boot configuration failures through the panel: catch the `PipelexConfigError` / `PipelexSetupError` family in `make_pipelex_for_cli` (or wherever the exploration shows the single right boundary is — check the agent CLI's equivalent factory too, which already produces an envelope, to keep the two roots symmetric), display the message in the panel, print the traceback only when `--traceback` was requested, and exit non-zero the way the other handlers do.++Things to settle during exploration, not assumed:++- **Which classes.** `PipelexConfigError` carries the structured `migration` block; the panel should render the message (which already contains the migration paragraph) and need not render the block separately — but confirm nothing else in the human CLI was reading the block. `PipelexSetupError` is what the boot raises for the late components (backend library, model deck, routing profiles); it may carry no block.+- **Whether the doctor's boot paths are affected.** `doctor_cmd.py` catches `CONFIG_REFUSED` itself in two places and `PipelexConfigError` in a third; those are its own presentation and should stay.+- **The exit code and the `--traceback` interplay** — follow what the existing handlers do; do not invent a second convention.+- **The tests that pin today's rendering.** Some CLI tests may assert on a traceback or on `SystemExit` shape; find them before changing the boundary.++### Tests to write++- A CLI-level test per family (a stale/wrong config → panel with the message, no traceback; same with `--traceback` → traceback present).+- Nothing about message *content* — that is pinned one layer down already (`tests/unit/pipelex/test_runtime_boot_stale_backend_error.py`, `tests/unit/pipelex/core/test_validation_report.py`).++## Suggested order and scope++1. **Item 1 first**, one PR, `fix/` prefix against `dev`. It is a correctness fix (a wrong `error_domain`) with a small mechanical footprint; it also carries the position into the message. Changelog under *Fixed*; docs: `docs/migration-ledger.md` § *Reporting a stale configuration on a validation error* has a sentence about which loaders report through the scan — say in one line that a file that does not parse is a plain configuration error and is not scanned. Update the agent-CLI output docs if the `TomlError` hint text changes.+2. **Item 2 second**, its own PR, `fix/` or `chore/`. Presentation only; changelog under *Fixed* or *Changed* depending on what the exploration finds about exit codes.++Neither blocks a release, and neither should be folded into S7b.++## Reproduction harness++The session's fake home is a scratchpad copy that will not survive; rebuild one: a temp `HOME` with `.pipelex/` populated by `printf 'y\n' | env HOME=<tmp> pipelex init config`, an **empty** project directory as cwd (the repo root has its own `.pipelex/inference/backends/` and wins the whole directory), and `pipelex show backends` as the command that boots through every library. Plant faults with an editor, never in `~/.pipelex/` — the real one is a stale specimen kept for the release step (see `TODOS.md` on `feature/Backend-surface`, *Rules of engagement*).diff --git a/wip/migrator-pr-1116-review-followups.md b/wip/migrator-pr-1116-review-followups.mdnew file mode 100644index 000000000..6247d8e44--- /dev/null+++ b/wip/migrator-pr-1116-review-followups.md@@ -0,0 +1,33 @@+# The fourth surface — items deferred from the PR #1116 review++What the review-bot pass on [PR #1116](https://github.com/Pipelex/pipelex/pull/1116) surfaced and this branch did **not** fix, with why. Four of the confirmed items were fixed on the branch: the boot-tolerance retry rebuilt a backend's file path with `Path(dir) / name` while the load that preceded it used an f-string, which agree for every ordinary name and diverge for an absolute one — the two are now one `backend_toml_path` helper, so the retry reads the file the load read by construction rather than by coincidence; the "claims no file" guard on both the ledger's `[surface]` block and the registry's `Surface` tested `is None`, so an empty `base_file` or `tier_glob` parsed, claimed nothing and migrated nothing in silence, and both now reject empty alongside absent; the stale-backend e2e class promised in its own docstring that the kit's `pipelex_gateway.toml` is asserted untouched while its neighbours snapshot filtered out every `.toml` in the directory, so the one file the whole `(directory, name)` claiming rule exists to protect was the one file nothing checked — it excludes the two aged files by name now, and a simulated rewrite of the gateway file reddens it; and the `/release` skill's renumbering paragraph said "unbullet-ed". Two of the highest-severity findings — the boot catching only `InferenceBackendLibraryValidationError` when the loader raises its sibling, and `[defaults]` being admitted as if it were a model table — were already fixed by the branch's head commit before the bots' verdicts were read, and are answered on the threads with the code and the pinning tests. Three were refuted: `fingerprint@2` is not inverted against `@3` (the chain runs the other way — `fingerprint@N` is the shape *at* version N, `pipelex-config@3` *is* the reshape, and `check-ledger` and `check-migration-schemas` are both green); the `cli-docs` drift ack does not mismatch (`make drift-check` passes on a clean tree, and the recorded trigger blob OIDs equal `git rev-parse :<path>` — the bot most likely recomputed against GitHub's PR merge commit); and the private-repo-names finding is about text this PR only relocates, byte-identical to what has been on public `main` since `#1099`.++## 1. A backend name is never checked for being a path++**Reporter:** Cubic (`pipelex/cogt/model_backends/backend_library.py`) — [thread](https://github.com/Pipelex/pipelex/pull/1116#discussion_r3797076387).++The fix that landed makes an absolute backend name *harmless* — both the load and the retry now resolve it under the backends directory — but it leaves the name itself unexamined. A backend name is a raw top-level table key of the user's own `inference/backends.toml`, TOML permits a quoted `["/abs/path"]` or `["../elsewhere"]` key, and nothing between the file and `InferenceBackend.name` (a plain `str`) rejects one. The same shape is built a third time for the health report at `pipelex/cli/commands/doctor_cmd.py`, which still uses the `/` operator.++**Why deferred:** rejecting a path-like key is a *new refusal* on a user-facing surface, not a review fix — it decides that a configuration which loads today stops loading, and it needs its own message, its own error class placement (`InferenceBackendLibraryValidationError`, so it lands in `BACKEND_LIBRARY_REFUSED` and inherits the surface-aware migration report for free) and its own migration question. Nothing is broken while it is absent.++**Recommendation:** add the guard in `load`, right after the enabled/CI skips, refusing a name that is absolute, contains a separator, or is `.`/`..`, and naming the backend and `backends.toml` the way the index-file refusal already does. Do it together with a sweep of the doctor's own path construction onto `backend_toml_path`, so all three sites are one spelling rather than two.++## 2. The diagnosis reports a header's name, never its value++**Reporter:** Cubic, second half of the admission finding (`pipelex/migration/surfaces.py`) — [thread](https://github.com/Pipelex/pipelex/pull/1116#discussion_r3797076393).++The first half — `[defaults]` treated as a model table — was already fixed on the branch. The half that remains is real: a header-shaped key on a *model* table whose value the loader refuses (`x-foo = 3`, or a value with a line break or a trailing space) is admitted by `admits_unnamed_key` and therefore never appears in a downgrade diagnosis, because only the key's spelling is examined. The illegal-header-*name* case the same comment raises is already closed by `is_legal_header_name`.++**Why deferred — and this one is a "no", not a "later".** The diagnosis has exactly one sentence to say about a key it reports: *the current schema has no setting there, and no ledger entry retires it — either the name is a typo, or this file was written by a newer pipelex*. For `x-foo = 3` both halves of that are false, so reporting it would replace silence with wrong advice. It would also make the backend the only surface whose diagnosis reads values (`[runtime.log] level = 42` is accepted by the main configuration's diagnosis the same way), and it would fix nothing the user is missing: a scan that finds nothing attaches no migration block, so the loader's own sentence reaches them whole, naming the model, the backend and the file. The reasoning is now recorded in the `admits_unnamed_key` docstring, where the next reader will find it.++**Recommendation:** leave it. Revisit only if the diagnosis ever grows a second note — a *this key is known but its value is not usable* verdict — at which point admitting the value half becomes cheap and honest at the same time.++## 3. Private repo names and deep paths in committed docs of a public repo++**Reporter:** Cubic (`wip/pipe-refs/implementation-plan.md`) — [thread](https://github.com/Pipelex/pipelex/pull/1116#discussion_r3797076399).++The observation about the doc is accurate — it names internal sibling repos and their exact file paths, and it is internally inconsistent about it, calling the transport layer "the layer that is not open core" on one line and spelling its exact path two lines down. The claim *about this PR*, though, does not hold: those paragraphs are byte-identical to `TODOS.md` on public `main`, where they landed with `#1099`, and this PR only moves them into `wip/pipe-refs/`. Net new public exposure is zero. The bot's own private/public list is also partly wrong: one repo it calls private is public in a different org, and two of the names it flags do not exist under those names at all.++**Why deferred:** the real question is repo-wide and predates this branch. It is not answerable inside a PR that relocates already-published text, and answering it there would leave every other occurrence standing while looking like the matter was handled.++**Recommendation:** decide once whether internal repo names belong in this repo's committed docs at all, then sweep. The sweep must start with shipped code, not docs — `pipelex/pipe_run/delivery_executor.py` names an internal repo and a path inside it in a comment that goes out in the PyPI wheel, which is a wider audience than a `wip/` doc has. If the answer is "role language only", the `#1099` genericization pass simply needs finishing; if it is "these names are fine", the doc's inconsistency should be resolved the other way, by naming the transport layer plainly on both lines.diff --git a/wip/pipe-refs/README.md b/wip/pipe-refs/README.mdindex ee6322c2c..953ffd689 100644--- a/wip/pipe-refs/README.md+++ b/wip/pipe-refs/README.md@@ -1,6 +1,6 @@# Bare references resolve across domains; the standard says they must not-**Status (2026-08-11).** Decided: tighten the runtime to the standard, implemented via build-time qualification of in-body references. The design and phased plan are in [build-time-qualification.md](build-time-qualification.md); §7 below was its starting sketch and is superseded by that document. The evidence in §1–§6 remains the record of why.+**Status (2026-08-11).** Decided: tighten the runtime to the standard, implemented via build-time qualification of in-body references. The design is in [build-time-qualification.md](build-time-qualification.md) and the phased plan in [implementation-plan.md](implementation-plan.md) (archived here from the repo-root `TODOS.md` on 2026-08-17, with the runtime phases done and the release-gated cross-repo tail still open); §7 below was the starting sketch and is superseded by those documents. The evidence in §1–§6 remains the record of why.**What this is.** A decision brief for the divergence recorded in [`../parity/bare-pipe-ref-spec-divergence.md`](../parity/bare-pipe-ref-spec-divergence.md), which closed with *"the direction of the fix is genuinely open"* and deferred it. This directory re-opens it with measurements instead of priors, and adds two things that note does not contain: the rule turns out to be **load-bearing for `[exports]`**, and the same divergence is **still live for concepts**, one file over from the pipes that were fixed.diff --git a/wip/pipe-refs/build-time-qualification.md b/wip/pipe-refs/build-time-qualification.mdindex 82f04f9ad..841075ae6 100644--- a/wip/pipe-refs/build-time-qualification.md+++ b/wip/pipe-refs/build-time-qualification.md@@ -83,7 +83,7 @@ The upcoming remote-methods system (GitHub addressing, cloning, version manageme## Implementation phases-The executable plan — phases, checkpoints, and per-phase task lists — lives in [../../TODOS.md](../../TODOS.md) and is the plan of record. It was restructured after the engineering review into C0–C4, splitting what this document originally described as one "Phase 1" into a pure-refactor extraction (Phase 1) and the rule flip (Phase 2), so that no checkpoint rests on a tree where the normalizer and the live library disagree. Do not maintain a second phase list here; record **outcomes** below as each checkpoint passes.+The executable plan — phases, checkpoints, and per-phase task lists — lives in [implementation-plan.md](implementation-plan.md) and is the plan of record. It was restructured after the engineering review into C0–C4, splitting what this document originally described as one "Phase 1" into a pure-refactor extraction (Phase 1) and the rule flip (Phase 2), so that no checkpoint rests on a tree where the normalizer and the live library disagree. Do not maintain a second phase list here; record **outcomes** below as each checkpoint passes.### Phase 1 outcome (C1)diff --git a/wip/pipe-refs/corpus-measurement.md b/wip/pipe-refs/corpus-measurement.mdindex 22ff8c821..276c9d131 100644--- a/wip/pipe-refs/corpus-measurement.md+++ b/wip/pipe-refs/corpus-measurement.md@@ -1,6 +1,6 @@# The fall-through corpus, measured across the whole workspace-**Status.** Measured 2026-08-11 for Phase 0 of [../../TODOS.md](../../TODOS.md). This widens [README.md](README.md) §5, which scoped its scan to a hand-written list of ten sibling repos. It does **not** gate shipping — the repo's no-backward-compatibility principle stands and there is no deprecation period. It calibrates the changelog wording and how much to invest in the error message.+**Status.** Measured 2026-08-11 for Phase 0 of [implementation-plan.md](implementation-plan.md). This widens [README.md](README.md) §5, which scoped its scan to a hand-written list of ten sibling repos. It does **not** gate shipping — the repo's no-backward-compatibility principle stands and there is no deprecation period. It calibrates the changelog wording and how much to invest in the error message.This is a dated snapshot with a reproduction command, which is why it carries raw counts. Other documents should point here rather than restate them.diff --git a/wip/pipe-refs/implementation-plan.md b/wip/pipe-refs/implementation-plan.mdnew file mode 100644index 000000000..c2efd2bb6--- /dev/null+++ b/wip/pipe-refs/implementation-plan.md@@ -0,0 +1,290 @@+# Pipe-refs build-time qualification — implementation plan++> **ARCHIVED from the repo-root `TODOS.md` on 2026-08-17 — the runtime work is complete, the cross-repo tail is not.** Phases 0 through 3 and Phase 4a item 1 shipped (PRs #1091, #1092, #1099, merged to `dev`); what remains open is the transport-boundary spec edit held for its verifying test, and all of Phase 4b, which is gated on a `pipelex` release carrying the `interpreter_hub` entry-pipe accessors. This file stays the plan of record for that tail: pick it up here rather than re-deriving it. It was moved off the repo root because the root tracker belongs to whatever work is currently in flight, and this work is waiting on a release rather than on a keyboard.+>+> **Paths below are relative to the repository root, not to this file's own directory** — they were written while this document sat at the root and are deliberately left as they were.++Source of truth for the design: [wip/pipe-refs/build-time-qualification.md](wip/pipe-refs/build-time-qualification.md) (decided 2026-08-11). Evidence and pre-change state: [wip/pipe-refs/README.md](wip/pipe-refs/README.md). Venue: the `_refs` worktree — treat this worktree as the repo root.++**What we are doing in one sentence:** bare in-body references (pipe steps, branches, outcomes, batch refs, concept I/O refs) get qualified to their owner domain once, at library build time, via one shared crate pass consumed by both the normalizer and `library_manager`; the live library lookup becomes strict; the crate-wide bare search survives only as an explicitly-named entry-point affordance for user-supplied codes.++**Why it matters beyond tidiness:** no-fall-through is what makes `[exports]` enforceable. A visibility rule the resolver can reach around is not a visibility rule. Keep that argument at the front of every discussion about this change — it is the reason the blast radius is worth paying.++## The resolution rule, in one picture++```+in-body ref encountered+ |+ +-- contains "alias->" ? --YES--> cross-package: pass leaves it ALONE+ | (resolved against the child library)+ NO+ |+ +-- already qualified "d.c" ? --YES--> unchanged (pass is idempotent)+ |+ NO (bare code "c")+ |+ v+ qualify to the OWNER domain of the declaring pipe --> "own_domain.c"+ |+ | existence is NOT checked here+ v+ ordinary dependency validation+ |+ +-- resolves ? --YES--> done+ |+ NO+ v+ error names the ref that was tried, and suggests a crate-wide+ candidate when one exists (diagnostic scan, FAILURE PATH ONLY —+ this is not a resolution fallback)+```++Entry-shaped lookups (a human typing a pipe code at a CLI) do NOT follow that rule. They go through the entry affordance, which searches crate-wide, refuses ambiguity, and deliberately ignores `[exports]` — a hand-invoked pipe is not an in-body reference, so package visibility does not apply.++## Rules of engagement++- **Checkpoints are mandatory stops.** At each ⛔ CHECKPOINT below, the agent MUST stop working, complete the checkpoint checklist (status updates, gates, commit), and end the session. Do not roll into the next phase in the same session, even if context feels fresh. The next session cold-starts from this file's Status block plus the design doc.+- **Update this file as you go.** Tick checkboxes when items complete; record decisions in the Status block the moment they are taken, not at the end.+- **Open questions from the design** are folded into the phase tasks below as explicit decision items (marked OQ1/OQ2/OQ3). Record each ruling in the Status block and in the design doc. OQ1 is a go/no-go gate in Phase 0, not an implementation task.++### Gates — the full checklist, run at every checkpoint++`make agent-check` and `make agent-test` do not cover everything this plan disturbs. Three of the gaps fail silently, which is why this is a checklist and not a sentence.++1. `make agent-check` clean.+2. Full `make agent-test` green.+3. **`git add` the work, THEN `make drift-check`.** `agent-check` does not run `drift-check`, so a contract never opens "along the way" under the other gates. And `drift-check` reads the git *index*, so unstaged work reads as a false green. If a contract opens, resolve it via the drift-review flow — no bypassing.+4. **Reconcile `.test_durations`** after any test file, class, or function is renamed or repurposed. This plan repurposes named tests in both the normalizer and lookup suites. Orphan node ids are silent — pytest-split treats an unknown id as average duration. Collect with `.venv/bin/pytest --collect-only -q -m ""`; the marker filter in `addopts` hides a large slice of the suite (mostly `tests/e2e`) and makes every e2e entry look like an orphan.+5. **Record the subject grant BEFORE the first `agent-check` on the new module.** `agent-check` runs `fix-keyword-only`, which silently rewrites an ungranted positional subject to keyword-only. If the pass is meant to read as a verb–object sentence (`qualify_crate(crate)`), `make subject-grant` has to land first or the shape is gone before anyone decides.++## Status++- **Current phase:** Phase 4a item 1 DONE (2026-08-13); 4a item 2 (transport-boundary spec edit) deliberately HELD to land together with its 4b verifying conformance test. **Phase 4b is blocked on user action:** merge PR #1099 (`fix/Pipe-refs-3` → dev), then cut a `pipelex` release carrying the `interpreter_hub` entry-pipe accessors — every 4b item needs that release installable by the sibling repos. The transport spec/test pair to touch in 4b: `docs/specs/pipelex-transport-boundary.md` §"The allowed import surface" (workspace root) + `conformance/tests/pipelex_transport/test_provider_surface.py` (the allowed-surface list lives in `test_data.py::TransportBoundaryContract.ALLOWED_SURFACE`; the workspace CLAUDE.md note claiming `conformance/tests/pipelex_transport/` does not exist is stale — the suite exists and runs).+- **Last completed item:** Phase 4a item 1 — the `mthds/` spec clarification (details on the 4a checklist line). Before that: C3 — the concept side landed as the `get_optional_entry_concept` / `get_required_entry_concept` pair with a package-aware `search_scope` (the entry pipe's domain, `alias->domain` for a dependency entry pipe) replacing the `search_domain_codes` list across the whole public chain; both README §4 latent defects deleted structurally; `fix_loop` rename collisions relaxed to per-domain. §4 marked HISTORICAL with a re-measured matrix (probe rewritten against the new affordance).+- **Phase 3 rulings.** (1) **Concept ambiguity raises, naming the candidates — but only after the scope preference has had its chance.** The old own-domain `insert(0)` was dead code; the preference now fires *before* the crate-wide scan. (2) **Both refusals (miss and ambiguity) raise `ConceptLibraryConceptNotFoundError`** — the class the `stuff_factory` handlers actually catch, which is what deletes the wrong-exception-class defect rather than patching around it; the four handler messages now say "could not be resolved: {exc}" so an ambiguity is not mislabeled "not found". (3) **The concept twin of OQ1 was paid in full here, unlike the pipe side** — the concept lookup is entry-only and its one call site holds the alias, so package scope is a string prefix, not an execution-path threading project. (4) The `search_domain_codes` parameter is deleted from `PipelexMTHDSProtocol` (public, breaking) rather than kept as an override — nothing supplied it in production and a caller-supplied list is exactly the contextual instability this branch removes.+- **Phase 3 surprises.** (1) Two integration tests pinned the crate-wide `fix_loop` collision scope and had to flip — the parametrized other-domain arm of `test_strip_namespace_bails_on_cross_file_collision` asserted a bail that is no longer correct behavior, and the rebuild-between-rounds pin only kept its purpose by moving to a same-domain fixture. (2) The stuff-factory param-id entries in `.test_durations` changed shape twice over: the rename changed the *argname* ids (`search_domain_codesN`), and the list→str change made the new ids embed the literal value (`test_domain`) instead of a positional placeholder — the reconciliation needed the full `-m ""` collection to map them. (3) `mocker.patch.object(module, "prepare_pipe_job", wraps=...)` on the run-setup module is enough to pin the scope wiring without touching the seam itself — the wiring mutation reddens both that pin and the end-to-end test.+- **Decisions taken:** phase structure reworked so no checkpoint rests on a self-contradicting tree; OQ1 promoted to a Phase 0 go/no-go gate; breaking-change posture held (no deprecation period) but preceded by a bounded corpus measurement. **OQ1 ruled GO** — owner-domain qualification is neutral-to-better for dependency packages; the dependency lookup-scope defect it exposed is pre-existing and deferred to the packaging project (`wip/pipe-refs/dependency-subpipe-scope-deferred.md`). **Crate envelope ruled: the pass does not deal in crates at all** — it returns a `QualifiedCrateContent` (just `concepts` and `pipes`), never a `LibraryCrate`, because it cannot pick the right digest scheme for a `fingerprint`. Each consumer makes the fingerprint correct at its own boundary (rationale in the design doc's Phase 1 outcome). *(This bullet previously read "the pass is envelope-preserving", describing the first cut that returned a crate; review changed the return type and the wording was stale in three places until a PR bot caught one of them.)*+- **Surprises / fallout:** (1) the dependency execution path has **no** child-library scoping at all — a dependency package using a controller internally either fails to load or silently binds a host pipe of the same bare code; the child-library special case in `Library.validate_pipe_library_with_libraries`'s first loop is neutralized by `pipe.validate_with_libraries()` in the second. Worse in the shape a package actually ships: with a manifest `[exports]`, the authored helper is dropped from the dependency's own child library and the load fails earlier still. (2) README §5's corpus scan used a hardcoded repo list that reached under a fifth of the bundles outside this repo; the widened scan is in `wip/pipe-refs/corpus-measurement.md`, and the second breaking reference lives in `cocode` — a shipped CLI, not an examples directory.+- **Open risks carried in:** four latent defects surfaced by the outside-voice pass are folded into Phases 1–3 below (batched sub-pipe code derivation, transported `PipeFunc` bare codes, the crate fingerprint contract — now ruled, aliased dependency keys polluting the crate-wide search). None of them is speculative; each names a file.+- **Phase 1 findings that change how Phase 2 must be tested:** (1) **this repo's entire bundle corpus is blind to the rule flip** — applying Phase 2's owner-domain rule to all 54 crates buildable here produces byte-identical output, so the two-domain fixture is not a nicety and the mutation check is what makes the resolution-row tests mean anything; (2) **a differential harness whose inputs each carry at most one defect cannot see an error-precedence change** — that is how Phase 1's own precedence shift got past it and had to be caught in review; Phase 2 moves far more error surface, so its equivalence inputs need multi-defect crates; (3) **a fixture full of bare refs proves nothing unless a test reads a ref back out of the result** — Phase 1's first test module passed intact against a no-op pass for exactly that reason; (4) nothing writes a normalized crate back over a `.mthds` file, so the flip does not rewrite user source — but the stamped `codegen.lock` `crate_fingerprint` moves on every committed projection; (5) `_qualify_io_ref`'s cross-package guard looks unreachable through a constructed blueprint (the io-ref validator rejects `alias->` refs outright) — preserved verbatim, flagged for whichever phase owns cross-package forms.+- **Phase 2 rulings.** **OQ2 — the entry affordance is a `get_optional_entry_pipe` / `get_required_entry_pipe` pair** on `PipeLibraryAbstract` + `PipeLibrary`, mirrored as hub accessors. A pair rather than one method because the migration found exactly one caller (`pipelex which`) that needs the optional form and eleven that want the required one; "entry" rather than the working name `find_pipe_by_bare_code` because it also takes fully-qualified refs, and because the name should say *why* the lookup differs, not *how*. **OQ3 — the cross-package bare-remainder search stays.** The qualification pass leaves `alias->…` refs alone (it cannot know a dependency's domain layout), so deleting the search would break every `alias->bare_code` ref with no canonical spelling to migrate to. It is alias-scoped and cannot reach a host pipe; the comment in `get_optional_pipe` says so and points at the packaging design. **DRY — the pipe and concept affordances stay separate**, written up with the reasoning in [wip/pipe-refs/entry-affordance-share-vs-duplicate.md](wip/pipe-refs/entry-affordance-share-vs-duplicate.md) so Phase 3 inherits a decision rather than making an accidental copy. The *rule* is shared (one `qualify_crate`); the affordance is the deliberate exception to it, and the two should not be conflated.+- **Phase 2 surprises.** (1) **The batched sub-pipe defect was real but not what the review said.** The plan (from Codex) predicted the invalid code `domain.foo_batch`; in fact `PipeAbstract` has a `code` validator that strips a namespace prefix, so the built ref was correct all along. What the wrong derivation actually produced was a `log.warning` on **every** batched run, about a code no user wrote. The fix stands; the first test written for it asserted on the ref and passed against the bug, and had to be rewritten to assert on the warning. (2) **Two call sites the plan did not enumerate** turned up by running the suite rather than grepping: `validate_bundle.py` resolved a bundle's own `[pipe.<code>]` keys as bare codes (now qualified with the bundle's domain — precise, not routed through the affordance), and `dry_run_in_process.best_effort_graph_spec` resolved a caller-supplied graph target (entry-shaped, now on the affordance). The second silently degrades to "no graph" on a miss, so it would never have failed loudly. (3) **`missing_pipe_code` changed value** — a structured, machine-read field, so a wire-visible change, recorded in the changelog. (4) The test sweep's own derivation over-reached twice and both were caught by verification rather than by the suite: a `.pipe_code ==` assertion resolved against the wrong source (that field is the *referring* pipe's local code and does not change), and a ref inside a `PipelexBundleBlueprint` fixture was hand-qualified when the product qualifies it — the new break message diagnosed that one itself.+- **Phase 2 PR review (cubic, 9 findings; greptile clean).** Two were real defects and both are fixed: the `SpecialOutcome` exemption applied to *every* controller kind, though `continue` / `fail` are legal snake_case pipe codes — so a sequence step naming a pipe called `continue` was left unqualified, which the strict lookup then fails to resolve (a Phase 2 regression: the fall-through used to rescue it); and `make-demos.sh`'s `mktemp -d -t` is BSD-only, so the README's own reproduction command aborted on Linux before writing a demo. **Four findings were claims of mine that had gone false**, which is the half worth remembering: the `python_sources` deferral credited `model_copy` for envelope preservation (true of the pass's *first* cut, which returned a crate — the same stale claim was in three places, and the bot found one), "no fix op writes a pipe ref" is wrong (`strip-namespace` plans a root `main_pipe` SET_KEY), and three line-number pointers had gone stale *inside this branch* between Phase 1 and Phase 2. Those were fixed by **removing** the line numbers, not correcting them — a pointer into code the same branch edits re-stales itself every phase.+- **The `_domain_of` / `CrateNormalizationError` debt, split and settled.** Phase 1 left two things wrong-sounding once `library_manager` became a second caller of the pass, and parked both on Phase 2. **Only one of them was Phase 2's to pay**, which the plan did not say clearly enough — a PR bot caught the bullet still assigning both to a phase declared complete. Settled: the **message** is fixed here (it now says "this pass expects…" rather than blaming the normalizer, which is no longer even the busier caller). The **error-class rename is deliberately re-deferred to Phase 4b**, where it belongs: `error_type` is a wire string, so a rename needs `make generate-error-identity`, `make generate-error-pages`, and the TypeScript-consumer sweep — all of which are already Phase 4b's "error-surface sweep" line items. Doing it here would open a cross-repo surface inside a runtime-only phase.++---++## Phase 0 — answer the questions that can invalidate the design++Nothing here writes product code. Both items exist because their answers can change what gets built, and both are cheaper now than later.++- [x] **OQ1 — dependency-pipe lookup scoping (GO/NO-GO).** **Answered: GO.** Measured with `wip/pipe-refs/probes/dep-subpipe-scope.py` across three package shapes. Execution consults the **host** library unconditionally, and its bare-code fall-through skips `alias->` entries — so a dependency's own bare sub-pipe ref never reaches the dependency's own pipe *today*: a plain host fails to load, a host declaring the same bare code loads and silently binds the host's pipe, and a dependency with a real `[exports]` manifest fails earlier still because the export filter drops the authored helper from the child library. Owner-domain qualification does not regress that (the qualified ref resolves to `None` in the host exactly as the bare one does), converts the silent capture into a deterministic not-found, and makes the eventual package-scoped lookup a direct key hit in the child library. The pre-existing defect is written up in `wip/pipe-refs/dependency-subpipe-scope-deferred.md` and deferred to the packaging project.+- [x] **Measure the fall-through corpus.** **Two bare pipe refs break across every `.mthds` tree in the workspace**, each fixed by one qualified spelling. Recorded with the full table, the reproduction command and the denominator's caveats in `wip/pipe-refs/corpus-measurement.md`. README §5's scan used a hardcoded ten-repo list and missed most of the workspace; enumerating instead multiplies the corpus several times over and finds a second `sibling-only` case in `cocode` — a shipped CLI, not a sample. The `sibling-only` *concept* refs found are vscode-pipelex editor fixtures that already fail to load today, so they are not new breakage.++### ⛔ CHECKPOINT C0 — MANDATORY STOP++- [x] Record the OQ1 ruling and the corpus number in the Status block and the design doc.+- [x] If OQ1 came back negative, do not proceed — write up what broke the premise and end the session for a design conversation. *(Came back positive — proceeding.)*+- [x] Commit the investigation notes.+- [x] STOP. Next session starts at Phase 1.++---++## Phase 1 — extract the pass (pure refactor, behavior-identical)++This phase moves code and changes nothing. That is the point: the checkpoint at the end is the safest handoff state in the whole plan, provable by an unchanged test suite. The rule flip does not happen here.++- [x] Read the design doc and README end-to-end; skim `pipelex/libraries/crate_normalization.py` (`_qualify_concept_ref`, `_qualify_pipe_ref`, `_qualify_io_ref`, `_normalize_pipe`) and `pipelex/libraries/library_crate_factory.py` to confirm the insertion point still matches the design.+- [x] **Record the subject grant first if the pass takes a positional subject** (see Gates item 5). Done — nine grants recorded before the first check. The `PostToolUse` hook blocked the write and named every ungranted subject, so the auto-fixer never got the chance to rewrite them; the gate's hazard is real but the hook catches it at edit time, not only at `agent-check` time.+- [x] Extract the in-body reference qualification step out of `crate_normalization.py` into a standalone pass over a `LibraryCrate`. Landed as `pipelex/libraries/crate_qualification.py::qualify_crate`. Covers pipe refs (steps, branches, outcomes, batch refs), concept I/O refs, `refines`, and structure-field refs. **Rule unchanged** — crate-wide search stays.+- [x] Make `normalize_crate` consume the extracted pass. No test flips. **Behavior-identical with one known, deliberate exception — error precedence between two independent defects.** HEAD's implementation was loaded beside the working tree's and both run over the same inputs, comparing full serialized output: identical across every crate buildable from this repo's `.mthds` files and across ten hand-built crates covering the branches the corpus cannot reach. The comparison was mutation-checked so its green means something: dropping structure-field qualification reddens one case, applying the Phase 2 rule reddens three.++ ⚠ **The exception, and how it was missed.** Qualification now runs as one complete phase ahead of refinement flattening, where HEAD interleaved them (concepts → flatten → pipes). A crate carrying *both* a refinement cycle and an ambiguous bare pipe ref used to report the cycle and now reports the ref. **The differential harness could not see this: every crate it compares carries at most one defect.** A single-defect corpus cannot detect a precedence change by construction — the finding came from review, not from the harness, and that is the lesson worth carrying into Phase 2, where far more error paths move. Accepted rather than fixed: restoring the old order would mean splitting the pass in two so the normalizer could interleave it, and Phase 2 runs the pass with no flattening at all. Both errors are `CrateNormalizationError`, the crate is invalid either way, and which of two independent defects is named first is not a contract. Recorded in `normalize_crate`'s docstring.+- [x] **Inventory every consumer of normalized crate output before the flip phase.** Recorded in [wip/pipe-refs/normalized-crate-consumers.md](wip/pipe-refs/normalized-crate-consumers.md). **The feared write-back does not exist** — no path writes a normalized crate back over a `.mthds` source file. `pipelex fix` rewrites user files but never through a crate, and no fix op writes a pipe ref (re-check in Phase 2 rather than assume). What does move is the stamped `codegen.lock` `crate_fingerprint` on every committed projection.+- [x] **Settle the crate envelope / fingerprint contract.** **Ruled: the pass does not deal in fingerprints at all** — it returns a `QualifiedCrateContent` (the rewritten `concepts` and `pipes`), not a `LibraryCrate`. Returning a crate would force it to supply a `fingerprint` it cannot get right, since a crate does not record which digest scheme produced its value; the pass would have to guess or carry over a digest that no longer describes its own content, contradicting `LibraryCrate.fingerprint`'s own docstring. Returning the two mappings makes that unrepresentable and costs the callers nothing — both already hold a crate to read the envelope from, and `normalize_crate` builds a fresh one anyway. Phase 2's `load_from_crate` keys idempotency on the incoming crate's fingerprint, which is what external callers pass to `is_crate_loaded`. *(First cut returned a crate with the envelope carried over and a paragraph excusing the stale digest; review pushed back and the smaller surface is plainly better — a paragraph of excuse is a design smell.)* The related `python_sources` drop in `normalize_crate` is real, untouched here, and deferred in [wip/pipe-refs/normalize-crate-drops-python-sources-deferred.md](wip/pipe-refs/normalize-crate-drops-python-sources-deferred.md) — the one-line fix moves customer Python source into a CLI stdout stream and an HTTP response body.+- [x] Gates checklist clean (all items).++### ⛔ CHECKPOINT C1 — MANDATORY STOP++A pure refactor has landed. The tree is fully self-consistent and the suite is unchanged — this is a genuinely safe place to walk away from.++- [x] Update the Status block (phase done, the consumer inventory, the fingerprint ruling).+- [x] Record status in the design doc's Phase 1 section.+- [x] Commit with a clear message. ⚠ **This item's own wording had to change:** it said "say plainly that this commit changes no behavior", and that turned out not to be true. The message states the extraction is behavior-preserving *with one documented exception* and names it. An instruction to assert a property is not a licence to assert it unverified.+- [x] Gates run at the checkpoint: `agent-check` clean, full `agent-test` green, `drift-check` passed against the staged index, `.test_durations` N/A (path-existence gate; nothing renamed or moved), subject grants recorded ahead of the first check.+- [x] Reviewed by a no-context `/code-review` fan-out; findings triaged and verified individually rather than deferred to. Three of the reviewer's claims held up, two against claims I had written down myself.++---++## Phase 2 — flip the rule and make the lookup strict (these land together)++The normalizer and the live library must never disagree in a committed state, so the rule flip and the strict lookup are one phase. This is the phase that justifies the branch.++### The rule++- [x] Switch the pipe rule inside the pass from crate-wide search to **owner-domain qualification** — the exact twin of `_qualify_concept_ref`. Existence is NOT checked by the pass; the ordinary dependency validation catches missing refs afterwards (that is the standard's step 3 by construction).+- [x] Leave cross-package refs (`alias->…`) untouched by the pass, exactly as the normalizer already does.+- [x] **Fix the batched sub-pipe code derivation.** `SubPipe.pipe_code` becomes `domain.foo`, and the runtime derives `f"{self.pipe_code}_batch"` (`pipe_batch_adhoc_pipe_code` in `pipelex/pipe_controllers/sub_pipe.py`). Use the resolved sub-pipe's local code for that derivation. ⚠ **The predicted consequence turned out to be wrong** — see the Phase 2 surprises above: `PipeAbstract`'s `code` validator strips a namespace prefix, so the built ref was correct all along and what the bug really produced was a warning on every batched run.+- [x] Run the qualification pass in `pipelex/libraries/library_manager.py` on **all** crate→pipes paths — main load, secondary loads, dependency-package loads — before pipes are constructed via `PipeFactory.make_from_blueprint`.++### The strict lookup and the entry affordance++- [x] Strip the step-3 bare-code crate-wide fallback from `PipeLibrary.get_optional_pipe` (`pipelex/libraries/pipe/pipe_library.py`), including the `domain_hint` TODO. What remains: direct key lookup and cross-package (`alias->`) handling. `get_required_pipe`'s signature does not move (transport-pinned).+- [x] **OQ3 — decide** whether `get_optional_pipe` keeps the cross-package bare-remainder search (`alias->` + bare code). It is alias-scoped so it does not reopen the cross-domain hole; the design leans toward keeping it until the packaging design rules on cross-package reference forms. Record the ruling, and pin it with a test either way.+- [x] Close [wip/parity/d1-domain-hint-deferred.md](wip/parity/d1-domain-hint-deferred.md) as subsumed by this change (status note in that file).+- [x] **OQ2 — settle the entry affordance's final name and shape** (working name `find_pipe_by_bare_code`; one method or a required/optional pair; where the ambiguity error message lives).+- [x] Implement the entry affordance on the pipe library (+ its abstract): exact ref hits directly; a bare code matches crate-wide; ambiguity raises an error asking the user to qualify. Docstring states explicitly that it deliberately does NOT consult `[exports]` — a hand-invoked pipe is not an in-body reference, so package visibility does not apply.+- [x] **Exclude aliased dependency keys from the crate-wide bare search.** `PipeLibrary.root` also holds `alias->…` dependency entries (added by `add_dependency_pipe`), so "crate-wide" is not "every value in root". Without this exclusion, installing an unrelated dependency package can make a host pipe ambiguous — reintroducing, through the new affordance, exactly the contextual instability this branch exists to eliminate. Applies to the concept affordance in Phase 3 too.+- [x] **Decide and record the share-vs-duplicate question** between this pipe affordance and the concept affordance built in Phase 3. They are near-twins — the concept one adds a domain-preference step, and its spec currently stops at "unique match" without saying what happens when the match is not unique. Decide now whether they share a helper or stay deliberately separate, and settle whether the concept one raises on ambiguity. A written "separate, because X" is a valid answer; an accidental copy discovered in Phase 3 is not.+- [x] Expose the affordance through a new `pipelex/interpreter_hub.py` accessor (additive; the transport spec/conformance update for it happens in Phase 4b).+- [x] Migrate the in-repo entry-shaped call sites to the affordance: the CLI commands (`which`, `show`, `validate`, the `build`/`codegen` groups), `pipeline_run_setup`, and the builder operations. Controllers, `sub_pipe`, `signature_walk`, and `library` validation need NO edit — they look up refs stored on built objects, which now arrive qualified. ⚠ That is true *only because this phase lands as one change*: `Library.validate_pipe_library_with_libraries`'s first loop looks up a bare sub-pipe code against the child library, whose keys are qualified, so it is served today solely by the step-3 fall-through being deleted below. Deleting the fall-through without qualifying the dependency load path breaks it.+- [x] **Qualify the transported `PipeFunc` code.** `PipeFunc` used to send its bare code and the sandbox later calls the now-strict lookup with it (`pipelex/pipe_operators/func/pipe_func.py` → `pipelex/pipe_operators/func/direct_pipe_func_executor.py`). Under the strict lookup **every** transported `PipeFunc` fails, not merely ambiguous ones. The DTO must carry a qualified ref — it now transports `pipe_code=self.pipe_ref`.++### The break message++- [x] Spec the not-found message in **dependency validation** — that is where users actually meet it, not in the lookup. It names the qualified ref that was tried, and when a crate-wide candidate exists it suggests it and states the new rule. Without this, the user reads `advisory.present_as_markdown` while staring at a file that says `present_as_markdown`, and has no way to know the compiler added the prefix.+- [x] Mark the diagnostic crate-wide scan **failure-path only** in code and comment. It is the same scan being deleted from the resolution path, and an implementer in a hurry can reintroduce the fallback by accident.+- [x] Build the crate-wide bare-code index **once per validation run** and share it across every missing-ref diagnostic. Per-error scanning goes quadratic on the mass-breakage run — which is precisely the first-run-after-upgrade this feature exists to improve.++### Tests++- [x] **Build a two-domain fixture.** In a single-domain crate, owner-domain qualification and crate-wide search give identical answers, so every resolution-row test below would pass just as happily under the rule being deleted. The rows only discriminate when a sibling domain declares the same pipe code. ⚠ **Measured in Phase 1, not merely suspected:** applying this phase's rule to all 54 crates buildable from this repo's `.mthds` files changes nothing — every normalized output is byte-identical under both rules. The repo's own corpus is blind to the flip, so the fixture must be hand-built, and the mutation check below is what proves it discriminates.+- [x] Flip the normalizer test that pins the old rule: `tests/unit/pipelex/libraries/test_crate_normalization.py::test_bare_cross_domain_pipe_refs_resolve_to_the_declaring_domain` becomes a test of owner-domain qualification.+- [x] Flip the lookup tests that pin the old rule: `tests/unit/pipelex/libraries/test_pipe_library_lookup.py::test_bare_code_ambiguous_raises` and `::test_bare_code_unambiguous` become tests of the strict lookup + entry affordance split.+- [x] Qualify the refs in tests that construct pipes directly with bare sub-refs (bypassing the crate pass) — they hit the strict lookup now. Find them by running the suite, not by grepping alone.+- [x] Conformance-shaped unit coverage for the resolution rows on the two-domain fixture: own-only resolves; sibling-only errors; both-declare resolves to own; nowhere errors. ⚠ **The export-bypass closure case is NOT covered by a test here** — only by re-running the demo probe by hand (recorded in README §3). Its *behaviour* is covered: a bare in-body ref cannot reach **any** sibling domain, exported or not, which is pinned at library level by `test_unresolved_pipe_ref_message.py::test_names_the_qualified_ref_and_explains_the_rule`, and export-bypass is a strict subcase of that. What has no test is the `[exports]`/`METHODS.toml` machinery *interacting* with the new rule, which needs a package fixture and is conformance-shaped — it is already scheduled in Phase 4b's spec-suite line item. Do not tick that 4b item without it.+- [x] **Mutation-check the rows.** Done for sibling-only (restoring the crate-wide search reddens the two rule tests and both message tests) and per-controller-kind for the closure walk. **Not done for export-bypass**, because there is no export-bypass test to mutate — see the item above.+- [x] Coverage for the entry affordance semantics: exact hit, crate-wide unique bare match, ambiguity error message, and no-match.+- [x] Pin the stated invariants that currently have no test: `alias->` refs pass through the pass untouched; all four ref kinds are covered (steps, branches, outcomes, batch refs — branches and batch refs are the ones a hand-written pass forgets); the pass is idempotent (`d.c` must not become `d.d.c`, and it now runs in two places); the pass runs on **all three** load paths (a miss on the secondary or dependency path is invisible for any single-domain library); the OQ3 ruling; and that the affordance reaches a **non-exported** pipe (the twin of the export-bypass closure — the docstring claim needs enforcement).+- [x] Execution test for `batch_over` covering the derived batch code.+- [x] Coverage for both branches of the new validation message: candidate exists, and no candidate exists.+- [x] Audit existing CLI coverage for the migrated surfaces, then add a bare-code end-to-end test wherever there is not one. A perfectly-tested affordance can still be perfectly mis-called, and `pipelex which <bare_code>` is the invocation the affordance exists to preserve.+- [x] Re-run the README's demo probes (`wip/pipe-refs/probes/make-demos.sh`) and move the closures accordingly: `fallthrough` and `export-bypass` must now FAIL resolve; `ambiguous` must resolve to the own domain. Update whatever the probes/README record as expected outcomes.+- [x] Gates checklist clean (all items).++### ⛔ CHECKPOINT C2 — MANDATORY STOP++The pipe rule is complete and self-consistent: canonical form and runtime agree, and hand-typed codes still work through a named door.++- [x] Update the Status block (OQ2/OQ3 rulings, the DRY ruling, surprises, corpus fallout).+- [x] Record status in the design doc's Phase 2 section (`build-time-qualification.md` → "Phase 2 outcome (C2)").+- [x] Update README §2's table — marked HISTORICAL with a banner rather than rewritten, and §3 gained a re-measured table.+- [x] Update the `docs/` pages that describe bare-ref resolution, pipe lookup, or the entry surfaces touched here. `docs/building-methods/libraries.md` gained a "How a pipe reference resolves" section; `docs/contribute/hub-layering.md` gained the two new hub symbols and the in-body-vs-entry guidance (that page is a drift contract target and was reviewed and acked).+- [x] Changelog entry under `[Unreleased]` noting the breaking language-behavior change. Also **removed** the superseded `[Unreleased]` entry describing the crate-wide rule this phase deletes — it never shipped, so leaving both would have published two contradictory claims.+- [x] Commit — landed on a stacked branch (PR #1092) rather than the base one, since PR #1091 carries Phase 0+1 and the two merge together.+- [x] STOP. Next session starts at Phase 3.++⚠ **These boxes sat unticked while the Status block above declared C2 passed** — caught by a PR bot, not by me. A checkpoint that reports itself complete from one place while its own checklist says otherwise is exactly the contradiction a cold-starting session cannot resolve. Tick the checklist in the same edit that updates the Status block.++---++## Phase 3 — the concept side and the footprint cleanups++Same root cause as Phase 2 — crate-wide search where owner scope was meant — but an independent surface with its own caller sweep. Kept separate so the pipe fix has a finish line that does not depend on it.++- [x] Collapse `ConceptLibrary.get_required_concept_from_concept_ref_or_code`'s `search_domain_codes` machinery (`pipelex/libraries/concept/concept_library.py` + `concept_provider_abstract.py`) into the entry-affordance shape decided in Phase 2. Landed as the `get_optional_entry_concept` / `get_required_entry_concept` pair; the required form replaces the old method on `ConceptProviderAbstract` 1:1.+- [x] **Scope the concept lookup by package, not just domain.** `search_scope` is a single string derived from the resolved entry pipe — its domain, prefixed `alias->domain` when the caller invoked a dependency pipe. Dependency concepts are keyed `alias->domain.Concept`, so the scoped preference is a direct key construction; no child-library machinery needed. Unlike pipe-side OQ1 (deferred to packaging), the concept lookup is entry-only and its one call site holds the alias, so this was paid in full here.+- [x] Exclude aliased dependency keys from the concept crate-wide search too (same reasoning as Phase 2). Mutation-checked.+- [x] Delete the latent defects README §4 documents: both gone structurally — there is no per-domain walk left to die on a miss, and every escape is `ConceptLibraryConceptNotFoundError` (the class the `stuff_factory` handlers catch), ambiguity included. §4 marked HISTORICAL with the re-measured matrix appended (probe rewritten against the new affordance).+- [x] Decide whether `pipeline_run_setup`'s own-domain-first ordering becomes meaningful or gets deleted along with the list parameter — **it became meaningful**: the scope preference fires BEFORE the crate-wide scan, so the entry pipe's own domain wins outright instead of being collected into an ambiguity raise. The list parameter is deleted from the whole public chain (`runner.py` constructor included — breaking, in the changelog); callers were swept, none supplied it in production.+- [x] Relax `_pipe_codes_by_file`'s rename collision scope in `pipelex/pipeline/fixes/fix_loop.py` from crate-wide to per-domain. Files now carry their declared domain (`_FilePipeCodes`); a file with an unreadable domain conservatively collides with everything. ⚠ Two integration tests pinned the crate-wide scope and flipped: the rebuild-between-rounds pin moved to a same-domain fixture, and `test_strip_namespace_bails_on_cross_file_collision`'s other-domain arm became a convergence test.+- [x] Tests: the resolution rows for concepts on the two-domain fixture (unit, `test_concept_library_entry_lookup.py`); ambiguity raises naming candidates; the package-scoped dependency-concept case (bare + dotted + exclusion); fix_loop independent renames across domains; an end-to-end wiring test (`test_entry_concept_scope.py`) that reddens if the scope threading is dropped anywhere in the chain; a characterization pin that `pipeline_run_setup` derives the scope from the entry pipe.+- [x] Gates checklist clean (all items). Grants for the new entry-concept defs recorded before the first check; `.test_durations` reconciled against a full `-m ""` collection (renamed node ids and remapped param ids; the remaining orphans predate this branch).++**Carry these forward from Phase 2 — each cost real time to learn:**++- **A single-domain fixture cannot discriminate this rule.** Owner-scope and the crate-wide search agree on every input where only one domain exists, so a concept test without a competing sibling declaration passes just as happily under the rule being deleted. Mutation-check every resolution row: put the old rule back and confirm the row goes red.+- **Mutate the wiring, not just the rule.** Removing the qualification from the *dependency* load path reddened nothing in the entire suite — every other test loads a single-domain library. Assume the same blind spot exists on the concept side and go looking for it deliberately.+- **Run the whole suite, including `tests/e2e`.** Phase 2's fixture sweep collected failures from `tests/unit` + `tests/integration` only and shipped two e2e failures into a "green" checkpoint.+- **A review finding that names a file and a line is a claim to verify.** Phase 2's batched-derivation item predicted the wrong consequence, and the first test written for it passed against the bug.+- **Do not touch the working tree while `agent-test` runs**, and read the run's log rather than the harness's exit-code summary — one reported success over a log ending in `Error 1`.++### ⛔ CHECKPOINT C3 — MANDATORY STOP++The runtime is fully spec-compliant and self-consistent; only cross-repo work remains.++- [x] Update the Status block and the design doc's Phase 3 section (`build-time-qualification.md` → "Phase 3 outcome (C3)").+- [x] Re-run the README's probes and demos and confirm the post-change behavior matches the design's predictions. The concept-lookup-matrix probe was rewritten against the new affordance (the method it measured no longer exists) and its re-measured table appended to README §4, which is now marked HISTORICAL.+- [x] Changelog entry for the concept-side behavior change (breaking: the `search_domain_codes` public parameter and the `ConceptProviderAbstract` method rename) and the `fix_loop` per-domain relaxation.+- [x] Commit.+- [x] STOP. End the session. Next session starts at Phase 4a.++**Post-C3 review round (2026-08-13):** a medium code review of the C3 commit surfaced real gaps in the new affordance, fixed in a follow-up commit on the branch. A multi-domain dependency package's own concepts are now reachable by bare code from any of its entry pipes — the scoped preference probed exactly one key and the crate-wide scan excludes aliased keys, so the sibling-domain shape was unreachable; the pipe side's alias-scoped package search was ported over, with `alias->Concept` as its explicit spelling. Dotted refs now obey the same scope-wins precedence under a dependency scope instead of binding host-first (a host/dep domain-name collision would have bound the host concept silently, since compatibility short-circuits on equal `concept_ref`). The invalid-string escape joined miss and ambiguity on `ConceptLibraryConceptNotFoundError`, making the one-class claim actually true. The copy-paste `stuff_factory` handlers collapsed into one `_resolve_entry_concept` helper; `pipeline_run_setup` captures the caller's raw entry ref in its own local so the scope derivation no longer depends on `pipe_code`'s rebind order; the `fix_loop` bail message hedges the unreadable-domain case. Each behavioral fix mutation-checked.++---++## Phase 4a — cross-repo, release-independent++These need no published `pipelex`. They can land as soon as Phase 3 is committed.++- [x] `mthds/` — additive clarification at § *Resolution Order for Bare Pipe References*: no-fall-through is what makes `[exports]` enforceable, so the next reader does not mistake the rule for a lookup convenience. No normative change. Landed 2026-08-13 in `mthds/` on branch `docs/bare-pipe-ref-no-fall-through` (off `dev`): an admonition note in `docs/spec/namespace-resolution.md` + a changelog entry; `make docs-check` (strict) passes. Not pushed — no PR opened without being asked. *(The design doc's "mthds/ last" ordering came from deferred-review-observations §4, whose constraint was D-1's domain_hint question — answered and subsumed in Phase 2, so the ordering dissolved and this plan's 4a placement governs.)*+- [ ] Add the new hub accessor to the transport boundary spec (workspace-level; exact path in the workspace private notes). The prose spec is a doc edit; its verifying test lands in 4b, and the two must land together per the spec/conformance sync rule — so hold this until 4b is ready to go with it.++## Phase 4b — cross-repo, gated on a published `pipelex`++**Gate:** every item below needs a `pipelex` release carrying the new `interpreter_hub` accessor, installable by the sibling repo. Cut that release first. Opening these PRs beforehand produces red CI for a reason unrelated to the change.++- [ ] our cross-repo spec suite — executable conformance cases for the resolution rows (own-only, sibling-only, both-declare, nowhere) and the export-bypass closure, on a two-domain fixture. Reuse the fixture built in Phase 2.+- [ ] our cross-repo spec suite — add the new hub accessor to the transport allowed-surface test, landing together with the spec doc edit from 4a (exact paths and the current state of that test in the workspace private notes).+- [ ] Run the spec suite's spec-link checker — must pass.+- [ ] the transport layer that is not open core — migrate its host-side bridge's payload-supplied lookup to the new accessor (that code is entry-shaped; `get_required_pipe`'s pinned signature is untouched).+- [ ] `pipelex-cookbook/` — qualify the first reference that leans on fall-through: `presentation.present_as_markdown` in `examples/wip/advisory_board/bundle.mthds`. Verify the example still validates and runs against the updated runtime.+- [ ] `cocode/` — qualify the second: `changelog.format_changelog_as_markdown` in `cocode/pipelines/swe_diff/changelog_enhanced.mthds` (a shipped CLI — see `wip/pipe-refs/corpus-measurement.md`). Verify the pipeline still validates against the updated runtime.+- [ ] **`search_domain_codes` consumer sweep beyond the transport bridge.** Found by enumerating the workspace (the transport item above covers only `pipelex-server/transport/`): `pipelex-api/api/routes/pipelex/pipeline.py` still passes the deleted `search_domain_codes=` kwarg to `pipeline_run_setup` — a `TypeError` on every execution request the moment the pin moves (and remember: its openapi-check runs under `make check` only, so an agent-gate-green bump still fails CI); `pipelex-js` mirrors the lookup semantics in `packages/runtime/src/library/live-library.ts` and drives it from `scripts/python-live-library-oracle.py` (semantic parity, not just a kwarg); `pipelex-mistralai-workflows` names it in integration tests; the demo repos (`pipelex-demo-mistral`, `pipelex-demo-vibe`) use it in example workflows. Each needs the entry-affordance/`search_scope` migration or the kwarg dropped.+- [ ] **Aliased-entry `search_scope` wiring test.** The package-fixture conformance work above must also pin the run-setup wiring: invoke a dependency pipe as the entry via `alias->domain.pipe`, supply a bare envelope concept declared only in the dependency, and assert it resolves. Today no test invokes an entry pipe through an alias end-to-end, so deleting the `has_cross_package_prefix` branch in `pipeline_run_setup` (or capturing `raw_entry_ref` after `pipe_code` rebinds) reddens nothing.+- [ ] **Error-surface sweep.** Run `make generate-error-identity` and `make generate-error-pages`, review the identity diff, and grep the TypeScript consumers for any changed or newly-introduced `error_type` literal — `mthds-starter-js/src/lib/errors.ts`, `pipelex-starter-js/src/lib/errors.ts`, `vscode-pipelex`'s `cliValidationBackend.test.ts`, and the internal playground app's graph route. The class name is a wire string; those repos switch on the literal and fall through to a generic branch without failing to compile. No Python tool will point at this.+- [ ] Changelog entries in each touched repo noting the breaking language-behavior change, per each repo's convention (skip `wip/` docs in changelogs).++### ⛔ CHECKPOINT C4 — MANDATORY STOP (final)++- [ ] Update the Status block; all phases complete.+- [ ] Close `wip/pipe-refs/` with a final status note in the design doc and README.+- [ ] Fold the outcome into the parity track's records (`wip/parity/`).+- [ ] Commit; prepare PRs per repo as instructed by the user — do not push or open PRs without being asked.+- [ ] STOP. The work is done; anything further (release sequencing before the packaging system) is a separate track.++---++## Phase dependency map++```+Phase 0 (OQ1 gate + corpus measurement)+ |+ | a negative OQ1 stops everything+ v+Phase 1 (extract the pass — pure refactor) <-- safe handoff point+ |+ v+Phase 2 (flip the rule + strict lookup, together) <-- the branch's reason to exist+ |+ +----------------------------++ | |+ v v+Phase 3 (concept side, Phase 4a (mthds/ clarification —+ fix_loop) release-independent)+ | |+ +----------------------------++ |+ v requires a published pipelex carrying the accessor+ Phase 4b (spec suite + transport + cookbook/cocode + error sweep)+```++Lane A: Phase 0 → 1 → 2 → 3 (sequential — all touch `pipelex/libraries/`). Lane B: Phase 4a (independent once Phase 2 lands; touches `mthds/` and the workspace-level boundary spec). Lanes A and B share no module directory. Phase 4b joins both and is gated on a release, so it cannot run in parallel with anything.++## GSTACK REVIEW REPORT++| Review | Trigger | Why | Runs | Status | Findings |+|--------|---------|-----|------|--------|----------|+| CEO Review | `/plan-ceo-review` | Scope & strategy | 0 | — | — |+| Codex Review | `/codex review` | Independent 2nd opinion | 0 | — | — |+| Eng Review | `/plan-eng-review` | Architecture & tests (required) | 1 | CLEAR (PLAN) | 16 issues, 0 critical gaps left open |+| Design Review | `/plan-design-review` | UI/UX gaps | 0 | — | — |+| DX Review | `/plan-devex-review` | Developer experience gaps | 0 | — | — |++**CODEX:** outside voice ran with repo access and surfaced findings the plan-text review could not reach — batched sub-pipe code derivation producing `domain.foo_batch`, transported `PipeFunc` carrying bare codes into the strict lookup, an undefined crate fingerprint contract, dependency concepts losing package identity, aliased dependency keys polluting the crate-wide search, and a stale `(done)` marker on the final checkpoint. All folded into the phases above.++**CROSS-MODEL:** one tension. The eng review treated the repo's written no-backward-compatibility principle as settling the breaking-change question and invested in the error message as the migration path; Codex argued the decision was unevidenced because the fall-through corpus excludes customer bundles. Resolved by holding the no-deprecation posture while adding a bounded Phase 0 corpus measurement — the principle governs whether to break, the measurement governs how well the break is communicated.++**VERDICT:** ENG CLEARED — ready to implement, starting at Phase 0.++NO UNRESOLVED DECISIONSdiff --git a/wip/pipe-refs/normalize-crate-drops-python-sources-deferred.md b/wip/pipe-refs/normalize-crate-drops-python-sources-deferred.mdindex 84034f922..70c938585 100644--- a/wip/pipe-refs/normalize-crate-drops-python-sources-deferred.md+++ b/wip/pipe-refs/normalize-crate-drops-python-sources-deferred.md@@ -1,6 +1,6 @@# Deferred: `normalize_crate` drops `python_sources`-**Status.** Noticed 2026-08-11 during Phase 1 of [../../TODOS.md](../../TODOS.md), while auditing the crate copy path for dropped fields (the plan asks for that audit precisely so a new pass does not reproduce the bug). Pre-existing, unrelated to the pipe-refs change, and deliberately **not** fixed on `fix/Pipe-refs` — the fix is a one-line addition whose *consequences* need a decision this branch is not the place to take.+**Status.** Noticed 2026-08-11 during Phase 1 of [implementation-plan.md](implementation-plan.md), while auditing the crate copy path for dropped fields (the plan asks for that audit precisely so a new pass does not reproduce the bug). Pre-existing, unrelated to the pipe-refs change, and deliberately **not** fixed on `fix/Pipe-refs` — the fix is a one-line addition whose *consequences* need a decision this branch is not the place to take.## What happensdiff --git a/wip/pipe-refs/normalized-crate-consumers.md b/wip/pipe-refs/normalized-crate-consumers.mdindex acd17e3d1..3ef11d911 100644--- a/wip/pipe-refs/normalized-crate-consumers.md+++ b/wip/pipe-refs/normalized-crate-consumers.md@@ -1,6 +1,6 @@# Who consumes normalized-crate output-**Status.** Inventoried 2026-08-11 for Phase 1 of [../../TODOS.md](../../TODOS.md). The plan asks for this before the Phase 2 rule flip, because the flip changes what a normalized in-body ref *says*: a bare `present_as_markdown` that today normalizes to the sibling domain's `presentation.present_as_markdown` will normalize to the owner's `orchestrator.present_as_markdown`. Anything that persists, stamps, or compares that output moves with it.+**Status.** Inventoried 2026-08-11 for Phase 1 of [implementation-plan.md](implementation-plan.md). The plan asks for this before the Phase 2 rule flip, because the flip changes what a normalized in-body ref *says*: a bare `present_as_markdown` that today normalizes to the sibling domain's `presentation.present_as_markdown` will normalize to the owner's `orchestrator.present_as_markdown`. Anything that persists, stamps, or compares that output moves with it.## The headline: nothing writes a normalized crate back to `.mthds`diff --git a/wip/prompting-style/templating-style-implementation-plan.md b/wip/prompting-style/templating-style-implementation-plan.mdindex 6fcece2ca..ec3463900 100644--- a/wip/prompting-style/templating-style-implementation-plan.md+++ b/wip/prompting-style/templating-style-implementation-plan.md@@ -181,7 +181,9 @@ Everything is green and documented; nothing is committed beyond what earlier che- **MTHDS spec prose** — a pipe-level `templating_style` is a language-surface change owned by the `mthds/` repo; the schema regen is only the mechanical half. A deliverable of its own after release.- **`pipelex-api` CI** — its openapi check runs only under its own `make check` and has been bitten by pipelex bumps before; verify on the release bump whether the schema change surfaces there.- **Cookbook / starter sweeps** — no shipped `.mthds` uses any styling surface today, so no migration is expected; a post-release sanity pass confirms.-- **Downstream backend-TOML sweep — a boot-breaker, and the one item on this list that is not optional.** Every repo that ships its own `.pipelex/inference/backends/*.toml` still declares `prompting_target`, and local backend files are read strictly: on the `pipelex` version bump each one dies at boot with `Extra forbidden fields: 'prompting_target'`. Tracked-file hits confirmed in `pipelex-api/`, `cocode/`, `pipelex-cookbook/`, `mthds-ui/`, plus our hosted worker, our demo repos, and two fixtures in our JS runtime (all private). **`portkey.toml` is the trap in every one of them**: the key appears once in `[defaults]` *and* again on several per-model entries, so a "fix" that deletes only the `[defaults]` line unblocks the boot while leaving the per-model ones — which the loader reclassifies as **outbound HTTP headers** sent to Portkey. Delete every occurrence, `[defaults]` and per-model alike, and grep to prove it.+- **Downstream backend-TOML sweep — a boot-breaker, and the one item on this list that is not optional.** Every repo that ships its own `.pipelex/inference/backends/*.toml` still declares `prompting_target`, and local backend files are read strictly: on the `pipelex` version bump each one dies at boot with `Extra forbidden fields: 'prompting_target'`. Tracked-file hits confirmed in `pipelex-api/`, `cocode/`, `pipelex-cookbook/`, `mthds-ui/`, plus our hosted worker, our demo repos, and two fixtures in our JS runtime (all private). **`portkey.toml` is the trap in every one of them**: the key appears once in `[defaults]` *and* again on several per-model entries, so a "fix" that deletes only the `[defaults]` line leaves the per-model ones behind. Since the rogue-headers guard those are a **hard error** — `NOT_HEADER_SHAPED`, fatal in lenient boots too, because a key that is not a model-spec field and has no hyphen in it is neither a setting nor a header — rather than the silent outbound header to Portkey they used to be. So the file still does not boot, and it now says why, naming the key, the model and the file. Delete every occurrence, `[defaults]` and per-model alike, and grep to prove it.++ **The scope of this sweep is the repositories *we* ship, and only those.** A user's own machine is not on this list and never could be — `pipelex init` does not overwrite, so every installation predating the change still carries the key in `~/.pipelex/inference/backends/`. That half is repaired by the tool rather than by a sweep: the `inference-backend` migration surface claims those files, ledger entry `inference-backend@2` deletes the key from every root table, and a stale directory boots with a warning until `pipelex migrate` is run. The two halves ship in the same release deliberately, since a user runs the command *after* upgrading.- **The served gateway config — done (2026-08-14), and it changed the branch.** The config is owned by our back-office repo, not the hosted server (both private): its gateway-models TOML declared `prompting_target = "anthropic"` in its `[defaults]`, and its build service publishes that block as the remote config's `backend_model_specs` — that file is the reason `drop_unknown_gateway_defaults` exists. The key is deleted there and **published**.Publishing it was only safe because **the served config is versioned in its URL, and this branch bumps it**: `pipelex/system/pipelex_service/pipelex_details.py` now points at `pipelex_remote_config_12.json` (was `_11`), committed on this branch with its own changelog entry. `_11` is frozen and still carries the key, so earlier releases keep resolving `anthropic` → `xml`; had they read a key-less config they would have fallen through to `TagStyle.TICKS`, flipping every gateway prompt in the field to backtick fences. Verified live against both URLs. Retires the *occasion* for the tolerance, not the tolerance itself, which is general. See [`gateway-config-outlived-prompting-target.md`](gateway-config-outlived-prompting-target.md).