Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
Answer the PR #1113 review: the backup a refused write kept deleting, and four gate corrections
Louis Choquel1a0a65b6d895
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex a00828ab1..a4b24ae91 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -8,7 +8,7 @@- **A migration operation can now remap every value beneath an open mapping**: `remap_value` accepts `key = "*"`, meaning "each key of the addressed table" — the same "each of these" reading the wildcard already had in a `table_path`, applied to keys. A mapping from the user's own keys to an enumerated value (`dict[str, LogLevel]`) keeps its spellings under keys only the document can enumerate, so no fixed key reaches them and this is the only shape in which a member renamed beneath one can be repaired at all. On every other kind a `*` key is now refused when the operation is parsed: deleting every key of a table is `delete_table`, and renaming or moving every key onto one fixed name is the many-to-one that already rules a wildcard out of a `move_key`. Refused rather than tolerated, because an unrefused one is a *dead* operation that skips forever while looking like an accounting.-- **A schema change that narrows what a value may *be* is now caught, not waved through as additive**: A change can keep every path and every enumerated spelling and still break a user's file — `int` becomes `str`, a free string becomes an enum, a bound moves from `ge=1` to `ge=5`. Read as a path diff those look like additions, so the coverage gate used to ask for a golden regeneration, demand no version bump and no entry, and let the next boot reject a valid file with a green gate behind it. It now splits its verdict by *direction*: a widened type or a relaxed bound still costs only a regeneration, while a narrowing demands a bump and an entry carrying a `remap_value` for each narrowed path — or marked `unsafe`, which is the only remedy a tightened numeric bound has, since no structural operation can repair a value. The same rule applies to a stored entry, compared by origin so a narrowing hidden behind a rename is still caught, and to a `pre_history` entry, so the flag is not a way past it. To see a bound at all the fingerprint now records one: a **closed whitelist** over `gt`, `ge`, `lt`, `le`, `min_length`, `max_length` and `multiple_of`, read from `annotated_types` (now a declared dependency) and from the field metadata pydantic folds a top-level `Field(ge=1)` into, with anything unrecognized dropped rather than serialized — `pattern` excluded permanently, because regex containment is not decidable and every pattern edit would read as a tightening. One correction rides along: an enumerated type relaxed into a free string no longer reads as the loss of every spelling it had. The contract states in the same change what the gate structurally cannot see — narrowing expressed in a validator, which no projection of the annotations can reach.+- **A schema change that narrows what a value may *be* is now caught, not waved through as additive**: A change can keep every path and every enumerated spelling and still break a user's file — `int` becomes `str`, a free string becomes an enum, a bound moves from `ge=1` to `ge=5`. Read as a path diff those look like additions, so the coverage gate used to ask for a golden regeneration, demand no version bump and no entry, and let the next boot reject a valid file with a green gate behind it. It now splits its verdict by *direction*: a widened type or a relaxed bound still costs only a regeneration, while a narrowing demands a bump and an entry carrying a `remap_value` for each narrowed path — or marked `unsafe`, which is the only remedy a tightened numeric bound has, since no structural operation can repair a value. The same rule applies to a stored entry, compared by origin so a narrowing hidden behind a rename is still caught, and to a `pre_history` entry, so the flag is not a way past it. To see a bound at all the fingerprint now records one: a **closed whitelist** over `gt`, `ge`, `lt`, `le`, `min_length`, `max_length` and `multiple_of`, read from `annotated_types` (now a declared dependency) and from the field metadata pydantic folds a top-level `Field(ge=1)` into, with anything unrecognized dropped rather than serialized — `pattern` excluded permanently, because regex containment is not decidable and every pattern edit would read as a tightening. One correction rides along: an enumerated type relaxed into a free string no longer reads as the loss of every spelling it had, and that is read structurally, so `list[enum]` becoming `list[str]` is the same benign loosening one container down — while a `list[enum]` flattened to a bare `str`, which is no loosening at all, still loses every spelling it had. The contract states in the same change what the gate structurally cannot see — narrowing expressed in a validator, which no projection of the annotations can reach.- **The telemetry configuration can now be migrated instead of thrown away, and the escape hatch that would have let it ship unverified is closed**: `telemetry.toml` carries the package's first migration entry, `telemetry-config@2`, which lifts a flat first-generation file — `telemetry_mode`, `host`, `project_api_key` and the rest at the document root — into today's `[custom_posthog]` section, keeping every value the user chose; four settings the current shape has no home for are dropped, and the entry says which and why. Because that change predates the first fingerprint, no diff describes it, and such an entry is marked `pre_history` and exempted from the coverage gate's accounting — so the verification that replaces it lands in the same change and neither ships without the other. A pre-history entry must declare the paths it removes (an empty declaration is refused when the ledger is parsed), none of those paths may appear in any fingerprint at or below its own version, the fingerprint pair it sits between must show no removal at all, and every operation must act inside the declaration. It also ships a hand-authored `[email protected]` beside the golden chain, which the transform check migrates and holds against the current shape and the current model — the same three claims every other entry answers. One comparator refinement came with it: a created path is now checked against `defaults@N` **or** `fingerprint@N`, because an optional key defaulting to `None` is a legal destination that no TOML document can carry.- **Configuration surfaces now have a migration ledger and a gate that forces one to be written**: Each of the three configuration surfaces — `pipelex.toml` and its tiers, `telemetry.toml`, `pipelex_service.toml` — carries a checked-in ledger at `pipelex/migration/ledgers/<surface-id>.toml` recording, as data, every shape change it has ever undergone, plus a golden chain (`fingerprint@N`, `defaults@N`) that pins what the schema looked like at each version. All three start at schema version 1 with no entries: this release cuts the baseline, it migrates nothing. The new `make check-migration-schemas` (alias `cmig`, in `make check`) recomputes each surface's fingerprint and refuses a change that would break a user's file without recording how to repair it — a removed path or a lost enumerated spelling demands a version bump and an operation accounting for it, a destination the new schema does not know is refused by name, and an entry that removes a path the schema still has is refused as over-deletion. Its counterpart `make up-migration-schemas` (alias `umig`, in `make up`) regenerates the goldens; the regeneration diff is the review artifact. One standing invariant is checked whether or not anything changed: every required path must have a value in its surface's defaults layer, because that is what makes an added key absorbable rather than breaking. The contract the whole thing implements is `docs/migration-ledger.md`.- **The migration operation vocabulary**: Fix operations became a discriminated union on `kind`, published as two subsets — every kind for the `.mthds` fix path, and the structural kinds only for ledgers, so an operation that would write a value into a user's file fails when the ledger is parsed rather than at review time. Two new kinds land with it: `move_key`, which relocates a key across tables (the whole subtree travels when the key is table-valued, and a missing destination parent is created), and `remap_value`, which rewrites a value against an explicit old-to-new mapping. A `*` segment addresses every entry of an open mapping, expanded over the keys the file actually holds. A new `CONFLICT` outcome separates "this cannot be done without choosing on your behalf" — a rename or move onto an occupied destination, or an `ensure_table` over a key holding something that is not a table — from the benign `SKIPPED` it used to hide inside; a conflicting operation writes nothing at all.diff --git a/docs/migration-ledger.md b/docs/migration-ledger.mdindex e268b51d9..2af4c83c4 100644--- a/docs/migration-ledger.md+++ b/docs/migration-ledger.md@@ -191,7 +191,7 @@ Both halves are load-bearing, and they pull against each other. Report too littl- **Operations are questioned by rehearsing them** against the document and throwing the result away. This is value-sensitive by construction, which is the point: a `remap_value` is about a stale *spelling*, not about a path, so a user whose value was never stale hears nothing.- **Declared narrowed paths are questioned by looking them up.** No operation can express a value the new schema refuses, so there is nothing to rehearse. Presence is the whole predicate — the engine is model-free by design, and cannot tell a value the narrowed domain rejects from one it accepts. What it reports is therefore *check this key*, not *this value is wrong*.-> **An entry with no operations must declare at least one narrowed path, and the ledger refuses one that does not.** That shape is the contract's own form for "a change only a human can make", and the [coverage table](#coverage-every-schema-change-is-accounted-for)'s only remedy for a tightened numeric bound — but with nothing to rehearse and nothing to look up, such an entry answers "nothing to say" for every file there will ever be. The accounting accepts it; the engine then guarantees it reaches nobody.+> **An entry with no operations must declare at least one narrowed path, and the ledger refuses one that does not.** No operations *and* one declaration is the contract's own form for "a change only a human can make", and the [coverage table](#coverage-every-schema-change-is-accounted-for)'s only remedy for a tightened numeric bound. What the refusal rules out is the shape one step below it — no operations *and* no declaration — which has nothing to rehearse and nothing to look up, and so answers "nothing to say" for every file there will ever be. The accounting would take that entry at its word; the engine would then guarantee it reaches nobody.> **`unsafe` on its own accounts for nothing.** The coverage gate lets an `unsafe` entry leave a narrowing unremapped and an enumerated spelling unmapped — that is what `unsafe` is for — but it demands the path be named in `declared_narrowed_paths`. Otherwise the word satisfies a reader while the user whose value the new schema refuses is left with a failing boot and no message.diff --git a/pipelex/cli/dev_cli/commands/update_migration_schemas_cmd.py b/pipelex/cli/dev_cli/commands/update_migration_schemas_cmd.pyindex a1386cfe9..afa5c4855 100644--- a/pipelex/cli/dev_cli/commands/update_migration_schemas_cmd.py+++ b/pipelex/cli/dev_cli/commands/update_migration_schemas_cmd.py@@ -3,7 +3,8 @@Writes each surface's fingerprint and complete reference document for the schema version itsledger currently declares. Older versions are never rewritten — a bump leaves the previousversion's snapshot behind as the frozen history the chain is made of — and the head version is-only rewritten when doing so erases nothing (see `pipelex/migration/snapshot.py`).+rewritten only when doing so erases nothing (see `pipelex/migration/snapshot.py`). `--force` is the+one exception, for a golden that predates a change to the fingerprint format itself.The regeneration diff is the point. A fingerprint golden is checked in so that a reviewer sees,line by line, which paths a change added, removed or renamed; running this command without readingdiff --git a/pipelex/migration/backup.py b/pipelex/migration/backup.pyindex d5b3bd6f0..df418398f 100644--- a/pipelex/migration/backup.py+++ b/pipelex/migration/backup.py@@ -152,10 +152,16 @@ def keep_backup_for_rescue(*, path: Path, backup_path: Path, moment: datetime) -successful run of the same file must not prune it away, which is exactly what it would do toanything still named `.bak.<stamp>`.- Returns the path the copy is actually at, so the report always names a file that exists: a- rename that will not go, or a rescue name another run has already taken, leaves the copy where- it is rather than losing it to a tidier name. It leaves it inside the rotation too, and- `was_rescued` is how the report knows to tell the user to take it now rather than later.+ Returns the path the copy is actually at, rather than the one it was headed for: a rename that+ will not go, or a rescue name another run has already taken, leaves the copy where it is rather+ than losing it to a tidier name. It leaves it inside the rotation too, and `was_rescued` is how+ the report knows to tell the user to take it now rather than later.++ "Where it is" is answered as of this call. The stamp resolves to the second and nothing here+ locks, so a third run of the same file that commits in between can prune the copy out from+ under an unrescued name — the one case in which the returned path names a file that has since+ gone. Closing it means coordinating pruning across processes, which is more machinery than a+ module with no command driving it yet has earned."""rescue_path = rescue_path_for(path=path, moment=moment)try:diff --git a/pipelex/migration/coverage.py b/pipelex/migration/coverage.pyindex 9072adc27..ff3c23ff5 100644--- a/pipelex/migration/coverage.py+++ b/pipelex/migration/coverage.py@@ -388,7 +388,16 @@ def _check_narrowing_accounting(f"or nobody hears about this")case MigrationSafety.SAFE:- remedy = "no structural operation can repair a value, so the entry needs a remap_value for that path, or must be marked unsafe"+ # Same gate `_check_enum_accounting` applies, for the same reason: a remap rewrites+ # a *string* value, so on a path whose own value is a list it is a guarded skip on+ # every run — and `make check-ledger` refuses one on a path that is not enumerated+ # as illegal outright. Naming it here would send the author to write an operation+ # one gate demands and the other rejects.+ remedy = (+ "no structural operation can repair a value, so the entry needs a remap_value for that path, or must be marked unsafe"+ if is_remappable(record=before_record)+ else "and no operation can rewrite a value inside a container, so the entry must be marked unsafe"+ )issues.append(CoverageIssue(surface_id=surface_id,diff --git a/pipelex/migration/fingerprint.py b/pipelex/migration/fingerprint.pyindex 2786ed4ee..d495af7b4 100644--- a/pipelex/migration/fingerprint.py+++ b/pipelex/migration/fingerprint.py@@ -238,7 +238,10 @@ def _record_field(collected[dotted] = PathFingerprint(value_type=_render_type(annotation=resolved),required=required,- enum_members=_collect_enum_members(annotation=resolved),+ # The members of an open node's value schema belong on the `*` child recorded below, whose+ # own value is the enumerated one — the only place an operation can address them. Every+ # other container keeps its members here, because it gets no child record at all.+ enum_members=None if open_value_type is not None else _collect_enum_members(annotation=resolved),default=_json_safe(value=defaults_value),open_node=open_value_type is not None,constraints=_collect_constraints(annotation=annotation, field_metadata=field_metadata),@@ -435,7 +438,16 @@ def _fold_bound(*, collected: dict[ConstraintKind, int | float], other: dict[Conoverclaim — a union member with no bound of that kind is unbounded in it, so the honest unionwould drop the kind entirely — and the overclaim is what keeps a tightening visible on thecommon `Annotated[int, Field(ge=1)] | Literal["auto"]` shape, where the literal member can- carry no numeric bound at all. Symmetric on both fingerprints, so it cannot invent a narrowing.+ carry no numeric bound at all.++ It is symmetric, so it costs nothing while a union's member set holds still. Where that set+ *moves* between two versions, the per-kind aggregate loses which member each bound came from+ and can travel in a direction the accepted values did not — a union of two separately bounded+ members can report a tightening that widened, and hide one that really narrowed. Reading it+ correctly means recording bounds per member in the golden; no surface has a union with two+ bound-carrying members today, and the alternative that needs no format change — dropping a kind+ absent from some pool — cannot tell `gt` from `ge` here and would trade this over-report for a+ silent under-report, which is the worse of the two. See `wip/pr-1113-review-notes.md`."""for kind, value in other.items():existing = collected.get(kind)@@ -460,19 +472,16 @@ def _collect_enum_members(*, annotation: Any) -> list[str] | None:def _gather_enum_members(*, annotation: Any, members: set[str]) -> None:if isinstance(annotation, type) and issubclass(annotation, Enum):- members.update(str(member.value) for member in annotation)+ # Only the string-valued members, exactly as a `Literal` is read below and for the same+ # reason: a `remap_value` rewrites a *string*, so recording `1` as a spelling would let the+ # accounting credit a remap the applier skips on every run, leaving a green gate over a file+ # the new schema rejects. An enum over non-strings records nothing, which is the honest+ # answer — and the same blind spot `Literal[1, 2]` already declares.+ members.update(member.value for member in annotation if isinstance(member.value, str))returnif get_origin(annotation) is Literal:members.update(str(arg) for arg in get_args(annotation) if isinstance(arg, str))return- if _as_open_mapping_value(annotation=annotation) is not None:- # The value schema of an open mapping gets its own record beneath the `*` segment, and its- # members belong there. Recording them on the container as well made the coverage gate- # demand a `remap_value` at a path whose value is a table — an operation that can never- # fire, so the demand had no answer an author could give. Descending into every *other*- # container stays right: a `list[enum]` gets no child record, so a member lost inside one- # would otherwise be invisible.- returnfor arg in get_args(annotation):_gather_enum_members(annotation=arg, members=members)diff --git a/pipelex/migration/material.py b/pipelex/migration/material.pyindex 78c1a4a2b..1457fb9b6 100644--- a/pipelex/migration/material.py+++ b/pipelex/migration/material.py@@ -33,7 +33,7 @@ from collections.abc import Sequencefrom pipelex.migration.fingerprint import PATH_SEPARATORfrom pipelex.migration.ledger import MigrationEntry, MigrationLedger, MigrationSafety-from pipelex.migration.walk import op_source_path+from pipelex.migration.walk import joined, op_source_pathfrom pipelex.suggested_fix import DeleteKeyOp, DeleteTableOp, MigrationOp, MoveKeyOp, RemapValueOp, RenameTableKeyOp@@ -167,9 +167,9 @@ def _path_through_op(*, path: str, op: MigrationOp) -> str | None:case DeleteKeyOp() | DeleteTableOp():return None if _is_at_or_under(path=path, ancestor=op_source_path(op=op)) else pathcase RenameTableKeyOp():- return _respelled(path=path, source=op_source_path(op=op), destination=_joined(segments=[*op.table_path, op.new_key]))+ return _respelled(path=path, source=op_source_path(op=op), destination=joined(segments=[*op.table_path, op.new_key]))case MoveKeyOp():- return _respelled(path=path, source=op_source_path(op=op), destination=_joined(segments=[*op.new_table_path, op.new_key]))+ return _respelled(path=path, source=op_source_path(op=op), destination=joined(segments=[*op.new_table_path, op.new_key]))def _path_before_op(*, path: str, op: MigrationOp) -> str:@@ -182,9 +182,9 @@ def _path_before_op(*, path: str, op: MigrationOp) -> str:case RemapValueOp() | DeleteKeyOp() | DeleteTableOp():return pathcase RenameTableKeyOp():- return _respelled(path=path, source=_joined(segments=[*op.table_path, op.new_key]), destination=op_source_path(op=op))+ return _respelled(path=path, source=joined(segments=[*op.table_path, op.new_key]), destination=op_source_path(op=op))case MoveKeyOp():- return _respelled(path=path, source=_joined(segments=[*op.new_table_path, op.new_key]), destination=op_source_path(op=op))+ return _respelled(path=path, source=joined(segments=[*op.new_table_path, op.new_key]), destination=op_source_path(op=op))def _with_source_path(*, op: MigrationOp, source_path: str) -> MigrationOp:@@ -213,7 +213,3 @@ def _respelled(*, path: str, source: str, destination: str) -> str:def _is_at_or_under(*, path: str, ancestor: str) -> bool:return path == ancestor or path.startswith(ancestor + PATH_SEPARATOR)---def _joined(*, segments: Sequence[str]) -> str:- return PATH_SEPARATOR.join(segments)diff --git a/pipelex/migration/narrowing.py b/pipelex/migration/narrowing.pyindex 77a6558d6..f54bbd45d 100644--- a/pipelex/migration/narrowing.py+++ b/pipelex/migration/narrowing.py@@ -80,17 +80,61 @@ def lost_enumerated_spellings(*, before: PathFingerprint, after: PathFingerprintof them still validates. That change is a widening and must ask for nothing — a bump demandedthere is a gate crying wolf on a change no user's file notices.- The exemption is exactly that case — the new type admits `str` — and not "the type did not- narrow": `Literal['a']` to `Literal[1]` renders the same type on both sides and records no- members after, and every file carrying `'a'` stops validating.+ The exemption is exactly that case — every enumerated part of the old type absorbed by a new+ one that enumerates nothing — and not "the type did not narrow": `Literal['a']` to `Literal[1]`+ renders the same type on both sides and records no members after, and every file carrying `'a'`+ stops validating.++ It is read structurally, at whatever depth the spellings sit, because that is where the type+ half of this relation reads it: `list[enum]` to `list[str]` is the same benign loosening one+ container down, and answering it differently here would have the two halves contradict each+ other on one change. The depth cuts both ways — a `list[enum]` flattened to a bare `str` is no+ widening at all, and every spelling it had is lost with it."""if not before.enum_members:return []- if not after.enum_members and STRING_TYPE in _union_members(rendered=after.value_type):+ if not after.enum_members and _is_relaxed_into_free_strings(before=before.value_type, after=after.value_type):return []return sorted(set(before.enum_members) - set(after.enum_members or []))+def _is_relaxed_into_free_strings(*, before: str, after: str) -> bool:+ """Whether the new type still accepts every enumerated spelling the old one carried.++ Two conditions, and the first is what keeps this from reading as "the type did not narrow": the+ new type must enumerate nothing anywhere, so that a record with no members means "free" rather+ than "enumerated over values the fingerprint cannot spell". Then every old member that carried+ spellings must be absorbed — by `str`, or by the same member one container down.++ Members that carry no spellings are not asked to be absorbed. `enum | int` becoming `str` loses+ the numbers, and that is a narrowing of the *type*, reported by `describe_narrowing`; this half+ answers only for the spellings.+ """+ if _records_enumerated_members(rendered=after):+ return False+ after_members = _union_members(rendered=after)+ return all(+ _is_member_absorbed(member=member, after_members=after_members)+ for member in _union_members(rendered=before)+ if _records_enumerated_members(rendered=member)+ )+++def _records_enumerated_members(*, rendered: str) -> bool:+ """Whether a rendered type spells out a closed set of values anywhere inside it.++ `enum` and `literal` do, at any depth: the members of a `list[enum]` are recorded on the list's+ own path, since a container argument gets no record of its own.+ """+ for member in _union_members(rendered=rendered):+ if member in _ENUMERATED_MEMBERS:+ return True+ _, arguments = _split_container(rendered=member)+ if any(_records_enumerated_members(rendered=argument) for argument in arguments):+ return True+ return False++def is_remappable(*, record: PathFingerprint) -> bool:"""Whether a `remap_value` on this path can ever rewrite anything.@@ -248,22 +292,39 @@ def _strictest_bound(makes `max` mean "strictest" in both. Exclusivity breaks the tie, since `gt=0` admits one fewervalue than `ge=0` at the same threshold.- Over the integers exclusivity is not a tie-break but a step: `gt=0` *is* `ge=1`, so the- exclusive form is folded into the inclusive one before comparing, and the two spellings of one- bound stop reading as a tightening of each other.+ Over the integers a bound is met at the first whole number past its threshold, so every+ spelling of one bound folds onto that number before anything is compared: `gt=t` binds at+ `floor(t) + 1` and `ge=t` at `ceil(t)`. For a whole threshold — which is what `gt=0` *is*+ `ge=1` says, and the only kind pydantic will build on an integer field — both are identities+ beyond turning the exclusive form inclusive. For a fractional one they are what stops `gt=0.5`+ and `gt=0.9`, which admit exactly the same integers, from reading as a tightening."""keys: list[tuple[float, bool]] = []for kind, exclusive in kinds:if kind not in constraints:continuethreshold = sign * constraints[kind]- if integral and exclusive:- keys.append((threshold + 1.0, False))+ if integral:+ keys.append((_first_integer_meeting(threshold=threshold, exclusive=exclusive), False))continuekeys.append((threshold, exclusive))return max(keys) if keys else None+def _first_integer_meeting(*, threshold: float, exclusive: bool) -> float:+ """The smallest whole number that satisfies this bound, in the signed comparison domain.++ Read exactly rather than in binary floating point — `Fraction` for the same reason+ `_is_multiple` uses it, so a threshold a user wrote as a decimal is not floored one short of+ itself by a representation error.+ """+ exact = Fraction(str(threshold))+ floored = exact.numerator // exact.denominator+ if exclusive:+ return float(floored + 1)+ return float(floored if exact.denominator == 1 else floored + 1)++def _render_bounds(*, constraints: dict[ConstraintKind, int | float], kinds: tuple[tuple[ConstraintKind, bool], ...]) -> str:present = [f"{kind}={constraints[kind]}" for kind, _ in kinds if kind in constraints]return ", ".join(present) if present else "unbounded"diff --git a/pipelex/migration/runner.py b/pipelex/migration/runner.pyindex da24674df..108c89011 100644--- a/pipelex/migration/runner.py+++ b/pipelex/migration/runner.py@@ -112,12 +112,13 @@ def _write_migrated_file(*, plan: MigrationPlan, snapshot: FileSnapshot, new_contry:commit_file_updates([PendingFileUpdate(snapshot=snapshot, new_content=new_content)])except FixWriteConflictError as exc:- # The primitive refused before touching the target, so the directory is as it was — minus- # the copy this run just made, which has nothing to back up.- _discard_backup(backup=backup)+ # The primitive refused before touching the target, so everything this run did is undone by+ # taking back the copy it just made, which has nothing to back up. What the primitive+ # refused *over* is somebody else's write, and `_discard_backup` is what asks whose.+ _discard_backup(backup=backup, snapshot=snapshot, new_content=new_content)return plan.model_copy(update={"blocked_reason": FileBlockedReason.CHANGED_DURING_RUN, "blocked_detail": str(exc)})except OSError as exc:- _discard_backup(backup=backup)+ _discard_backup(backup=backup, snapshot=snapshot, new_content=new_content)return plan.model_copy(update={"blocked_reason": FileBlockedReason.UNWRITABLE, "blocked_detail": f"the file could not be written: {exc.strerror or exc}"})@@ -180,7 +181,7 @@ def _whereabouts_of(*, kept: RescuedBackup) -> str:)-def _discard_backup(*, backup: WrittenBackup) -> None:+def _discard_backup(*, backup: WrittenBackup, snapshot: FileSnapshot, new_content: str) -> None:"""Remove the copy this run made of a file it then did not rewrite.Only the copy this run made. A backup already sitting at that name belongs to a concurrent run@@ -188,10 +189,21 @@ def _discard_backup(*, backup: WrittenBackup) -> None:it here would destroy the one thing backups exist for on the way to reporting a write thatnever happened.+ And only while nobody else has done this run's work. The stamp resolves to the second, so a+ concurrent run of the same file that found this name taken adopted it as *its* restore point+ without writing anything there. If that run has since committed, the file already holds the+ text this run was going to write — and the copy under this name is the last of the original,+ named by a report that is already out. So the file is asked what it holds rather than assumed+ to be as it was: only this run's own text acquits the other run of having got there first.++ A user's edit is the other reason a write is refused, and it is not this: the file then holds+ the user's text, the copy goes as it always did, and nothing is left beside a file they did not+ ask to have backed up.+A copy that will not go is a stray file beside an untouched original — worth a warning, neverworth an exception crossing the per-file boundary and aborting the siblings."""- if not backup.was_created:+ if not backup.was_created or _carries(path=snapshot.path, content=new_content):returntry:backup.path.unlink(missing_ok=True)diff --git a/pipelex/migration/snapshot.py b/pipelex/migration/snapshot.pyindex 891fb4a62..10c78d8f1 100644--- a/pipelex/migration/snapshot.py+++ b/pipelex/migration/snapshot.py@@ -103,9 +103,11 @@ def _refuse_a_destructive_head_overwrite(msg = (f"surface '{surface.surface_id}': refusing to overwrite the golden for schema version {schema_version}, because "f"{' and '.join(lost)}. Rewriting the head link here would erase exactly what the coverage gate exists to catch, "- f"leaving it green over a change that breaks every file carrying that material. If this is a real removal, bump "+ f"leaving it green over a change that breaks every file carrying that material. If this is a real change, bump "f"current_schema_version to {schema_version + 1} and add the entry '{surface.surface_id}@{schema_version + 1}' "- f"accounting for it. If schema version {schema_version} has not been released and the golden merely predates a "+ f"accounting for it — a removal by the operation that removes it, and a narrowing by a remap_value on the path, "+ f"or by marking the entry unsafe and naming the path in declared_narrowed_paths, since no structural operation "+ f"can repair a value. If schema version {schema_version} has not been released and the golden merely predates a "f"change to the fingerprint format itself, re-record it with `--force`")raise MigrationSnapshotRefusedError(msg)diff --git a/pipelex/pipeline/fixes/applier.py b/pipelex/pipeline/fixes/applier.pyindex 4614b2377..bb0b0c9eb 100644--- a/pipelex/pipeline/fixes/applier.py+++ b/pipelex/pipeline/fixes/applier.py@@ -505,12 +505,19 @@ def _remap_every_value(*, target_table: dict[str, Any], mapping: dict[str, str],no conflict rule: any rewrite makes the operation applied, none makes it skipped."""remapped = 0+ # Only the string values are ones a mapping could have named, so they are what the report+ # counts against: a table also holding sub-tables or numbers would otherwise read as though the+ # operation had left values behind that it was never able to address.+ reachable = 0for key in list(target_table):+ if not isinstance(target_table[key], str):+ continue+ reachable += 1if _remap_one_value(target_table=target_table, key=key, mapping=mapping, table_path_str=table_path_str).outcome.did_apply:remapped += 1if not remapped:return _OpResult(FixOpOutcome.SKIPPED, f"no value in '{table_path_str}' is one this operation's mapping names")- return _OpResult(FixOpOutcome.APPLIED, f"remapped {remapped} of {len(target_table)} values in '{table_path_str}'")+ return _OpResult(FixOpOutcome.APPLIED, f"remapped {remapped} of {reachable} values in '{table_path_str}'")def _refresh_table_headers(*, item: Any) -> None:diff --git a/tests/unit/pipelex/migration/test_coverage_gate.py b/tests/unit/pipelex/migration/test_coverage_gate.pyindex 1bbb0f574..797168eec 100644--- a/tests/unit/pipelex/migration/test_coverage_gate.py+++ b/tests/unit/pipelex/migration/test_coverage_gate.py@@ -23,6 +23,7 @@ from pipelex.migration.fingerprint import compute_fingerprintfrom pipelex.migration.goldens import write_defaults_golden, write_fingerprint_goldenfrom pipelex.migration.ledger import ledgers_dirfrom pipelex.migration.surfaces import DefaultsLayerKind, Surface+from pipelex.tools.typing.pydantic_utils import empty_list_factory_ofSURFACE_ID = "synthetic-config"@@ -125,6 +126,19 @@ class _BoundedTightenedAndRenamed(BaseModel):retries: int = Field(default=3, ge=2)+class _MixedItemList(BaseModel):+ """A list whose items may be numbers or strings, beside a key the entry has an operation for."""++ gone: str = "x"+ items: list[int | str] = Field(default_factory=list[int | str])+++class _IntegerItemList(BaseModel):+ """The items narrowed to numbers. No operation can reach inside the list to repair one."""++ items: list[int] = Field(default_factory=empty_list_factory_of(int))++class _BoundedOrAuto(BaseModel):"""A bounded number that may also be spelled `auto` — the `int | literal` shape real surfaces carry."""@@ -158,6 +172,18 @@ class _EnumeratedTier(BaseModel):tier: _Tier = _Tier.BASIC+class _EnumeratedTierList(BaseModel):+ """`tiers` accepts a list of the two spellings — the shape a real surface already has."""++ tiers: list[_Tier] = Field(default_factory=empty_list_factory_of(_Tier))+++class _FreeStringTierList(BaseModel):+ """The same list, relaxed to accept any string."""++ tiers: list[str] = Field(default_factory=list)++class _SchemaOneWithBothNames(BaseModel):"""A starting shape that already has `title`, so a rename of `label` onto it collides."""@@ -769,6 +795,20 @@ description = "The domain shrank."issues = check_surface(surface=_surface(config_model=_FreeStringTier), migration_dir=tmp_path)assert _kinds(issues) == [CoverageIssueKind.FINGERPRINT_DRIFTED]+ def test_an_enumerated_list_relaxed_into_free_strings_asks_only_for_a_regeneration(self, tmp_path: Path) -> None:+ """The same widening one container down, which a real surface already carries as `list[AgentTarget]`.++ The type half reads a container structurally; the member half read its exemption only at the+ top level, so this change reported no narrowing and every spelling lost at once. The remedy+ the gate then named — bump the version and mark the entry `unsafe`, because no operation can+ rewrite a value inside a list — would be reported to every user, at every boot, forever, for+ a change that invalidates nothing.+ """+ _write_ledger(migration_dir=tmp_path, current_schema_version=1)+ _snapshot(migration_dir=tmp_path, config_model=_EnumeratedTierList, schema_version=1)+ issues = check_surface(surface=_surface(config_model=_FreeStringTierList), migration_dir=tmp_path)+ assert _kinds(issues) == [CoverageIssueKind.FINGERPRINT_DRIFTED]+def test_a_bumped_entry_that_does_not_account_for_the_narrowing_is_refused(self, tmp_path: Path) -> None:"""The entry explains the rename it made and says nothing about the bound it moved."""ops = """@@ -785,6 +825,28 @@ new_key = "title"assert _kinds(issues) == [CoverageIssueKind.VALUE_DOMAIN_NARROWED]assert "'retries'" in issues[0].message+ def test_a_narrowed_container_is_sent_to_unsafe_rather_than_to_a_remap_it_cannot_write(self, tmp_path: Path) -> None:+ """The remedy a gate names has to be one the other gates accept.++ A `remap_value` rewrites a string value, so it never reaches an item inside a list — and+ `make check-ledger` refuses one on a path that is not enumerated, as illegal. Offering it+ here sent the author to write an operation one gate demanded and the other rejected.+ """+ ops = """+[[migration.ops]]+kind = "delete_key"+table_path = []+key = "gone"+"""+ _write_ledger(migration_dir=tmp_path, current_schema_version=2, entries=self._entry_with_ops(safety="safe", ops=ops))+ _snapshot(migration_dir=tmp_path, config_model=_MixedItemList, schema_version=1)+ _snapshot(migration_dir=tmp_path, config_model=_IntegerItemList, schema_version=2)+ issues = check_surface(surface=_surface(config_model=_IntegerItemList), migration_dir=tmp_path)+ narrowings = [issue for issue in issues if issue.kind is CoverageIssueKind.VALUE_DOMAIN_NARROWED]+ assert len(narrowings) == 1+ assert "must be marked unsafe" in narrowings[0].message+ assert "remap_value" not in narrowings[0].message+def test_a_remap_on_the_narrowed_path_accounts_for_it(self, tmp_path: Path) -> None:"""The remedy that repairs the file rather than only warning about it, where it applies."""ops = """diff --git a/tests/unit/pipelex/migration/test_fingerprint.py b/tests/unit/pipelex/migration/test_fingerprint.pyindex c3fcf6680..ed6654b9f 100644--- a/tests/unit/pipelex/migration/test_fingerprint.py+++ b/tests/unit/pipelex/migration/test_fingerprint.py@@ -9,7 +9,7 @@ red on every legitimate configuration change and teach everyone to regenerate rafrom collections.abc import Mappingfrom decimal import Decimal-from enum import StrEnum+from enum import Enum, StrEnumfrom typing import Annotated, Any, Literalfrom pydantic import BaseModel, Field@@ -22,6 +22,13 @@ class _Mode(StrEnum):SLOW = "slow"+class _Level(int, Enum):+ """Deliberately not a `StrEnum`: the projection has to say so rather than stringify it."""++ LOW = 1+ HIGH = 2++class _Leaf(BaseModel):name: strcount: int = 3@@ -35,6 +42,8 @@ class _Synthetic(BaseModel):labels: dict[str, str]modes: dict[str, _Mode]mode_list: list[_Mode]+ mode_maps: list[dict[str, _Mode]]+ level: _Level = _Level.LOWtags: list[str]bounded: Annotated[int, Field(ge=1)] | Literal["unbounded"] = 1# A field-level bound binds the whole field, whatever a union member declares for itself —@@ -89,6 +98,8 @@ class TestFingerprintPaths:"modes","modes.*","mode_list",+ "mode_maps",+ "level","tags","bounded","capped",@@ -181,6 +192,28 @@ class TestFingerprintRecords:"""assert _fingerprint().paths["mode_list"].enum_members == ["fast", "slow"]+ def test_the_members_of_a_mapping_nested_in_a_list_are_recorded_on_the_list_itself(self) -> None:+ """The suppression belongs to the field's own open node, not to every mapping anywhere inside it.++ A `list[dict[str, enum]]` has no `*` child — the field's open node is the list, and a list+ gets no child record — so suppressing the mapping's members here dropped them entirely, and+ retiring one moved nothing in the fingerprint at all. The gate would have passed a change+ that breaks every file carrying the spelling.+ """+ fingerprint = _fingerprint()+ assert fingerprint.paths["mode_maps"].enum_members == ["fast", "slow"]+ assert "mode_maps.*" not in fingerprint.path_names()++ def test_an_enum_over_non_string_values_records_no_spelling(self) -> None:+ """The same rule a `Literal` over non-strings already follows, for the same reason.++ A remap rewrites a *string* value, so recording `1` and `2` as spellings would let the+ accounting credit a `remap_value` the applier skips on every run — a green gate over a file+ the new schema rejects. Recording nothing makes the projection's blind spot visible as a+ blind spot, which is what the `Literal` rule already does.+ """+ assert _fingerprint().paths["level"].enum_members is None+def test_a_string_literal_counts_as_an_enumerated_spelling(self) -> None:"""A `Literal` and an `Enum` are the same thing to a TOML file: a closed set of spellings."""assert _fingerprint().paths["bounded"].enum_members == ["unbounded"]diff --git a/tests/unit/pipelex/migration/test_narrowing.py b/tests/unit/pipelex/migration/test_narrowing.pyindex 7216175f8..3e9d07971 100644--- a/tests/unit/pipelex/migration/test_narrowing.py+++ b/tests/unit/pipelex/migration/test_narrowing.py@@ -116,6 +116,16 @@ class TestTheNarrowingRelation:("int", {ConstraintKind.GE: 1}, {ConstraintKind.GT: 0}),("int", {ConstraintKind.LT: 10}, {ConstraintKind.LE: 9}),("int | literal", {ConstraintKind.GT: 0}, {ConstraintKind.GE: 1}),+ # A fractional threshold names no integer of its own, so it is the same bound as the+ # whole number it rounds to — in both directions, and inclusive as well as exclusive.+ ("int", {ConstraintKind.GT: 1.5}, {ConstraintKind.GE: 2}),+ ("int", {ConstraintKind.GE: 2}, {ConstraintKind.GT: 1.5}),+ ("int", {ConstraintKind.GT: 0.5}, {ConstraintKind.GT: 0.9}),+ ("int", {ConstraintKind.GE: 0.5}, {ConstraintKind.GE: 1}),+ ("int", {ConstraintKind.LE: 10}, {ConstraintKind.LT: 10.5}),+ ("int", {ConstraintKind.LT: 10.5}, {ConstraintKind.LT: 10.1}),+ ("int", {ConstraintKind.LE: 9.5}, {ConstraintKind.LE: 9}),+ ("int", {ConstraintKind.GE: 0}, {ConstraintKind.GT: -0.5}),],)def test_two_spellings_of_one_integer_bound_are_the_same_bound(@@ -218,6 +228,36 @@ class TestTheNarrowingRelation:)assert lost == []+ def test_an_enumerated_container_relaxed_into_free_strings_loses_no_spelling(self) -> None:+ """The same relaxation one container down, which the type half already reads structurally.++ `list[enum]` to `list[str]` is the widening `test_a_container_or_scalar_that_only_gained_room_is_not_a_narrowing`+ asserts it is. Reading the exemption only at the top level made this half disagree with that+ one: no narrowing, and every spelling lost — a bump and an `unsafe` entry demanded, then+ reported to every user forever, for a change no file notices.+ """+ lost = lost_enumerated_spellings(+ before=_record(value_type="list[enum]", enum_members=["basic", "premium"]),+ after=_record(value_type="list[str]"),+ )+ assert lost == []++ def test_an_enumerated_container_relaxed_into_something_else_still_loses_its_spellings(self) -> None:+ """The exemption travels into the container, and so does its limit: `list[int]` accepts no spelling."""+ lost = lost_enumerated_spellings(+ before=_record(value_type="list[enum]", enum_members=["basic", "premium"]),+ after=_record(value_type="list[int]"),+ )+ assert lost == ["basic", "premium"]++ def test_an_enumerated_container_flattened_to_a_bare_string_still_loses_its_spellings(self) -> None:+ """A `list[enum]` is not absorbed by `str`: a file carrying a list stops validating outright."""+ lost = lost_enumerated_spellings(+ before=_record(value_type="list[enum]", enum_members=["basic", "premium"]),+ after=_record(value_type="str"),+ )+ assert lost == ["basic", "premium"]+def test_an_enum_that_became_something_else_entirely_still_loses_its_spellings(self) -> None:"""The exemption is for widening only — `int` accepts none of the spellings `str` did."""lost = lost_enumerated_spellings(diff --git a/tests/unit/pipelex/migration/test_runner.py b/tests/unit/pipelex/migration/test_runner.pyindex 1999b4b4e..a5474c58c 100644--- a/tests/unit/pipelex/migration/test_runner.py+++ b/tests/unit/pipelex/migration/test_runner.py@@ -694,6 +694,42 @@ class TestMigrationRunner:assert occupied.exists()assert occupied.read_text(encoding="utf-8") == OLD_SHAPE+ def test_a_refused_write_keeps_its_copy_when_the_original_is_no_longer_on_disk(+ self,+ tmp_path: Path,+ mocker: MockerFixture,+ build_entry: EntryBuilder,+ build_ledger: LedgerBuilder,+ build_surface: SurfaceBuilder,+ ) -> None:+ """The other half of the same-second collision: the copy this run made is the one another run named.++ A run whose write is refused deletes the copy it made, on the reasoning that the directory+ is as it was and a write that did not happen has nothing to back up. That reasoning is about+ this run's own actions, and the write is refused precisely because somebody else's landed —+ the concurrent run that found this name taken, adopted it as its restore point, and has+ already replaced the file. Deleting it here leaves the migrated file with no copy of the+ original anywhere, and the other run's report naming a path that is gone.+ """+ target = tmp_path / "example.toml"+ target.write_text(OLD_SHAPE, encoding="utf-8")+ ledger = build_ledger(+ entries=[build_entry(to_schema_version=2, ops=[RenameTableKeyOp(table_path=["reporting"], key="output_config", new_key="output")])]+ )++ def _the_other_run_got_there_first(_updates: list[PendingFileUpdate]) -> None:+ target.write_text('[reporting]\noutput = { directory = "out" }\n', encoding="utf-8")+ msg = "refusing to overwrite: the file changed while changes were being prepared"+ raise FixWriteConflictError(msg)++ mocker.patch("pipelex.migration.runner.commit_file_updates", side_effect=_the_other_run_got_there_first)++ plan = migrate_file(surface=build_surface(), ledger=ledger, file_path=target, dry_run=False, moment=MOMENT)++ assert plan.blocked_reason is FileBlockedReason.CHANGED_DURING_RUN+ backups = existing_backups_of(path=target)+ assert [backup.read_text(encoding="utf-8") for backup in backups] == [OLD_SHAPE]+def test_a_rescue_copy_survives_the_next_successful_run_of_the_same_file(self,tmp_path: Path,diff --git a/tests/unit/pipelex/migration/test_snapshot_guard.py b/tests/unit/pipelex/migration/test_snapshot_guard.pyindex b73095f0c..68c872b6d 100644--- a/tests/unit/pipelex/migration/test_snapshot_guard.py+++ b/tests/unit/pipelex/migration/test_snapshot_guard.py@@ -139,6 +139,25 @@ class TestTheSnapshotGuard:assert "--force" in str(caught.value)+ def test_the_refusal_names_the_remedy_a_narrowing_actually_has(self, tmp_path: Path) -> None:+ """A tightened bound is not repaired by a bump alone, and the message must not imply it is.++ The guard refuses on a narrowing as well as on a removal, but named only the removal's+ remedy — "add the entry accounting for it" — which for a bound an author has just tightened+ reads as though bumping the version were the whole of it. No structural operation can+ repair a value, so the entry has to carry a remap or say `unsafe` and name the path.+ """+ _write_ledger(migration_dir=tmp_path)+ snapshot_surface(surface=_surface(config_model=_SchemaOne), migration_dir=tmp_path)++ with pytest.raises(MigrationSnapshotRefusedError) as caught:+ snapshot_surface(surface=_surface(config_model=_BoundTightened), migration_dir=tmp_path)++ message = str(caught.value)+ assert "accepts fewer values at" in message+ assert "declared_narrowed_paths" in message+ assert "remap_value" in message+def test_force_re_records_the_head_over_lost_material(self, tmp_path: Path) -> None:"""The escape a fingerprint *format* change needs, over a schema version nobody has released."""_write_ledger(migration_dir=tmp_path)diff --git a/tests/unit/pipelex/migration/test_transform_check.py b/tests/unit/pipelex/migration/test_transform_check.pyindex 44b4b99a4..c85993f01 100644--- a/tests/unit/pipelex/migration/test_transform_check.py+++ b/tests/unit/pipelex/migration/test_transform_check.py@@ -390,7 +390,11 @@ mapping = { basic = "standrad" }legal precisely when it is unsafe — so demanding its operations reach the new shape wouldcontradict the one thing `unsafe` means."""- _write_ledger(migration_dir=tmp_path, entries=_entry(ops="", safety="unsafe", declared='declared_narrowed_paths = ["label"]'))+ # `title`, not `label`: an entry declares a path its *own* version records, and `label` is+ # the one this bump removes. Declaring a removed path is the shape `make check-ledger`+ # refuses as DECLARED_NARROWED_PATH_IS_ABSENT — a fixture that only escapes it by calling a+ # different gate would put an illegal ledger in front of the next reader as an example.+ _write_ledger(migration_dir=tmp_path, entries=_entry(ops="", safety="unsafe", declared='declared_narrowed_paths = ["title"]'))_defaults(migration_dir=tmp_path, schema_version=1, document='label = "hello"\n')_defaults(migration_dir=tmp_path, schema_version=2, document='title = "hello"\n')assert check_transform_chain(surface=_surface(config_model=_Renamed), migration_dir=tmp_path) == []diff --git a/wip/migrator-phase3-progress.md b/wip/migrator-phase3-progress.mdindex 37dc72374..c53e79126 100644--- a/wip/migrator-phase3-progress.md+++ b/wip/migrator-phase3-progress.md@@ -192,7 +192,7 @@ Every new runner test was mutation-tested: dropping the resolve, clobbering the## Already true before this session started (do not rebuild)-- **The directory walk's overlap refusal** — the S6 "Done when" item carried from the #1110 review — **is already built and tested**: `SurfaceRegistry.surface_for_file_name` raises `MigrationRegistryError` naming the file, and `tests/unit/pipelex/migration/test_surface_resolution.py::test_a_file_two_globs_both_claim_stops_by_name` proves it with the `pipelex_*.toml` / `*_local.toml` pair. It landed in Phase 2 ahead of schedule. **What is still owed** is the #1111 note-5 half: use the *real* specimen — `.pipelex/inference/backends/pipelex_gateway.toml`, which matches the `pipelex-config` tier glob and is claimed by nothing — rather than a synthetic name, and pin that the walk is **non-recursive** (it is: `files_by_surface_in_directory` skips anything that is not a file, so the `inference/backends/` subdirectory is never descended). That is a test, not a behaviour change.+- **The directory walk's overlap refusal** — the S6 "Done when" item carried from the #1110 review — **is already built and tested**: `SurfaceRegistry.surface_for_file_name` raises `MigrationRegistryError` naming the file, and `tests/unit/pipelex/migration/test_surface_resolution.py::test_a_file_two_globs_both_claim_stops_by_name` proves it with the `pipelex_*.toml` / `*_local.toml` pair. It landed in Phase 2 ahead of schedule, and the synthetic pair stays — a two-claimant refusal needs two globs that both match, which no real file in the tree produces. **What is still owed** is the #1111 note-5 half, which is a *different* claim about a *different* specimen: that the walk is **non-recursive**, pinned with the real `.pipelex/inference/backends/pipelex_gateway.toml`. That file matches the `pipelex-config` tier glob but sits one directory down, and `files_by_surface_in_directory` skips anything that is not a file, so the walk never descends to it — which is exactly what the test would assert. That is a test, not a behaviour change, and it neither replaces nor re-aims the overlap test.## Still open, in the charter's own wordsdiff --git a/wip/migrator-write-scope-and-rename-fidelity.md b/wip/migrator-write-scope-and-rename-fidelity.mdindex 7b5eee619..6bb3d238d 100644--- a/wip/migrator-write-scope-and-rename-fidelity.md+++ b/wip/migrator-write-scope-and-rename-fidelity.md@@ -42,6 +42,8 @@ The scenarios where it is *not* correct both require a file to be sitting at `<fWhat *was* wrong on this path — the `state_uncertain` report promising to keep a copy it had not moved out of the pruning rotation — is fixed: `keep_backup_for_rescue` now returns `RescuedBackup(path, was_rescued)` and the report tells the user to take the copy now when it could not be secured.+A second thing was wrong on it and is now fixed too, found by the PR #1113 bot review: the *creator* of a shared name could delete a copy the adopting run was already relying on. Run A writes `bak.T`; run B finds the name taken, adopts it as its restore point, commits and prunes; A's commit is then refused *because B's landed*, and `_discard_backup` unlinked `bak.T` on the reasoning that a write that did not happen has nothing to back up. That reasoning is about A's own actions, and it left the migrated file with no copy of the original anywhere. `_discard_backup` now asks the file what it holds: a refused write discards its copy unless the file already carries the text this run was going to write, which only the other run can have put there. A user's own edit is unaffected — the file then holds the user's text, and the copy goes as it always did.+**Residual worth knowing.** If this ever needs revisiting, the honest lever is not a content check but the plan's vocabulary: `MigrationPlan.backup_path` currently means "where a copy of the pre-migration file is", not "where this run wrote one", and nothing in the report distinguishes the two.## Residual, unrelated to the three abovediff --git a/wip/pr-1113-review-notes.md b/wip/pr-1113-review-notes.mdnew file mode 100644index 000000000..ecba823b0--- /dev/null+++ b/wip/pr-1113-review-notes.md@@ -0,0 +1,38 @@+# Deferred from the PR #1113 bot review++Three findings from the review bots on PR #1113 were verified as real and deliberately not fixed. Each is a decision rather than a defect, and each is written down here so the decision is taken on purpose rather than by omission. The rest of that review's findings were fixed in the same change.++Two further findings — an occupied `.bak.<stamp>` reported as this run's copy, and an `unsafe` entry reported at a spelling a later `safe` entry deletes — were already recorded, before this review, in [`migrator-write-scope-and-rename-fidelity.md`](migrator-write-scope-and-rename-fidelity.md) §3 and § "Residual". They are unchanged and are not repeated here.++## 1. `is_remappable` credits a union whose container branch it cannot reach++`is_remappable` in `pipelex/migration/narrowing.py` answers "can a `remap_value` on this path ever rewrite anything", and it answers yes when **any** top-level union member is string-typed. For `enum | list[enum]` that is true of the scalar branch, so the coverage gate credits a remap for the whole path — while the applier skips the list value because it is not a string. A file carrying the list keeps a retired spelling behind a green gate.++The reasoning is sound and the shape does not exist. The only scalar-plus-composite union on any registered surface is `cogt.img_gen_config.img_gen_param_defaults.size` → `SizeTier | ImageSize`, rendered `enum | table`. Its non-string branch is a `BaseModel`, and `_gather_enum_members` never descends into one, so every member recorded on that path genuinely does come from the string branch and the credited remap reaches all of them. Reaching the hole needs a branch that is *both* a container *and* enumerated — `enum | list[enum]`, `literal | list[literal]`, or `str | dict[str, enum]` — and none is written anywhere in `pipelex/`.++**Why it is not fixed here.** The fix is not local: `is_remappable` would have to ask "does any *non*-string branch carry enumerated spellings", which changes what the predicate means for every union path and which two accountings then read differently. The only test that can exercise it is one built on a model shape the repo's own standards discourage, so the guard would be pinned by a fixture and by nothing real.++**The decision to take.** Either take the change — `_records_enumerated_members` (added in this same PR for the container-widening fix) makes it about four lines — or leave the predicate as it is and say in its docstring that a union is answered by its string branches alone. Worth doing either way: **`is_remappable` has no direct unit test at all** today, only the indirect coverage-gate one, and it now gates two separate accountings.++## 2. Bounds are aggregated per kind across union members, losing which member each came from++`_fold_bound` / `_merge_carriers` in `pipelex/migration/fingerprint.py` merge the members of a union into one bound map, one constraint kind at a time. Which member a bound belonged to is gone by the time `narrowing.py` compares two versions, and the aggregate can then move in a direction the accepted values did not. Both directions were reproduced against the real projection:++- **A tightening that widened.** `int` → `Annotated[int, Field(ge=1)] | Annotated[int, Field(le=0)]` reports both a tightened lower bound and a tightened upper bound, into an aggregate box (`ge=1 ∧ le=0`) that is empty and that neither member has. Every value valid before is still valid; the domain strictly grew.+- **A narrowing that went unreported, which is worse.** `Annotated[int, Field(ge=1, le=10)] | Annotated[int, Field(ge=100, le=200)]` tightened to `ge=150` on the second member aggregates to `{ge: 1, le: 200}` on both sides. The gate says nothing while every file carrying `120` stops validating.++Reaching either needs a union with **two or more bound-carrying members**. There is none: the only union carrying any bound on any surface is `pipeline_execution_config.max_concurrency` → `Annotated[int, Field(ge=1)] | Literal["unbounded"]`, one non-empty pool, so the across-members fold never runs across anything.++**Why it is not fixed here.** The cheap-looking fix — drop a kind that is absent from some member's pool, the "honest union" — is family-blind: the fingerprint stores `gt` and `ge` as separate kinds and only `narrowing.py` knows they are one lower-bound family, so it would drop a real `ge=1` because a sibling member spells its lower bound `gt=0`. That converts the module's *preferred* error, an over-report the author can read and act on, into its *dispreferred* one, a silent under-report. Reading a union correctly means recording bounds per member in the golden and comparing member-wise, which is a fingerprint-format change and a comparator rewrite.++**The decision to take.** Leave it until a surface actually grows a union with two bounded members, then do it properly (per-member constraint sets in the golden). The docstring of `_fold_bound` was corrected in this pass to state the limitation instead of claiming, as it did, that symmetry makes inventing a narrowing impossible — it does not, once the member set moves.++## 3. A rescue copy can be pruned out from under the rename that was rescuing it++`keep_backup_for_rescue` in `pipelex/migration/backup.py` reserves the `.rescue.<stamp>` name and then renames `.bak.<stamp>` onto it. Between those two steps a *third* run of the same file can commit and call `prune_backups_except`, which deletes by name shape and so takes the `.bak.` copy the rename was about to move. The rename then fails, the reservation is cleaned up, and the function returns `RescuedBackup(path=backup_path, was_rescued=False)` — a path that no longer exists, which the report hands to the user with "copy it aside before the next successful run of this file".++Nothing is lost that was not already going to be pruned, and the run in question is one whose write it could not vouch for either way. What is wrong is the promise: the docstring said the returned path always names a file that exists.++**Why it is not fixed here.** Closing it means coordinating pruning with rescue across processes — a lock file — and pruning is deliberately name-shaped rather than ownership-aware. That is real machinery for a module that no command drives yet (`pipelex migrate` does not exist). The docstring was corrected in this pass to state the window rather than deny it.++**The decision to take.** Revisit when `pipelex migrate` lands and concurrent runs become something a user can actually produce. The honest lever then is probably not a lock but pruning that spares backups younger than some age, which fixes this and nothing else has to change.