Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.
Answer the #1119 review threads: stale status line and orphaned wip pointers
Louis Choquel39059feb743d
diff --git a/docs/migration-ledger.md b/docs/migration-ledger.mdindex cc417aeb6..eed3a198f 100644--- a/docs/migration-ledger.md+++ b/docs/migration-ledger.md@@ -685,7 +685,7 @@ Three further rules make that safety net worth the name.What a backup carries across is the file's **permission bits**, and that is the one deliberate exception to "no value read from a user's file is ever rendered" — a backup contains the user's values by definition, so a `0600` configuration must not acquire a world-readable copy beside it. Ownership, ACLs and extended attributes are **not** carried across the replace, on either the backup or the migrated file: an atomic same-directory replace cannot preserve what the running process has no right to set, and re-attaching an attribute blindly (a quarantine flag, a security label) is a worse guess than leaving it off. The security-relevant bit of a configuration file is its mode.-A configuration file that is a **symlink** is followed: the file the user means is the one at the end of the link, so the run reads, backs up and replaces *that* file, and the link survives. Replacing the link path instead would put a regular file where the link was and leave the real file unmigrated. The plan keeps naming the path the directory walk found; the backup path shows where the bytes actually went. This is what the `.mthds` fix loop already does with its own targets. The two callers differ on one half, and the difference is deliberate: **a migration's write scope is the resolved target of any file the walk claims**, not the walked directories themselves. The `.mthds` fix loop pairs its resolution with a write-scope check because it is handed a bundle directory and must not write outside it; a configuration directory is a place a user keeps links to files they own, and a dotfiles repository is the ordinary reason one is there. Refusing it would mean the tool declines to migrate exactly the machines whose owner was most deliberate about their configuration. The plan names the walked path and the backup names the resolved one, so a run that followed a link out of the directory says so. The reasoning behind both readings is in `wip/migrator-write-scope-and-rename-fidelity.md`.+A configuration file that is a **symlink** is followed: the file the user means is the one at the end of the link, so the run reads, backs up and replaces *that* file, and the link survives. Replacing the link path instead would put a regular file where the link was and leave the real file unmigrated. The plan keeps naming the path the directory walk found; the backup path shows where the bytes actually went. This is what the `.mthds` fix loop already does with its own targets. The two callers differ on one half, and the difference is deliberate: **a migration's write scope is the resolved target of any file the walk claims**, not the walked directories themselves. The `.mthds` fix loop pairs its resolution with a write-scope check because it is handed a bundle directory and must not write outside it; a configuration directory is a place a user keeps links to files they own, and a dotfiles repository is the ordinary reason one is there. Refusing it would mean the tool declines to migrate exactly the machines whose owner was most deliberate about their configuration. The plan names the walked path and the backup names the resolved one, so a run that followed a link out of the directory says so. The reasoning behind both readings is in `wip/migrator/migrator-write-scope-and-rename-fidelity.md`.### Per-file transactionsdiff --git a/wip/migrator/migrator-comment-fidelity.md b/wip/migrator/migrator-comment-fidelity.mdindex 6d3d7df43..05ec7af0d 100644--- a/wip/migrator/migrator-comment-fidelity.md+++ b/wip/migrator/migrator-comment-fidelity.md@@ -1,6 +1,6 @@# Comment fidelity across a move — the bug report after v0.46.0, and what closed it-**Status: fixed on `fix/Migrator`, unreleased.** The applier now keeps a comment on the item it introduces through `move_key`, `delete_key` and `delete_table`, and settles what `set_key` and `ensure_table` insert before a table's trailing banner rather than after it. This note records why the phase-0 spike wrote the behaviour off as unfixable, why that was wrong, the rule that was chosen and where it is a reading rather than a fact, and what it does not do for files already migrated by v0.46.0.+**Status: fixed on `fix/Migrator` (PR #1118, merged to `dev`), shipped in v0.46.1.** The applier now keeps a comment on the item it introduces through `move_key`, `delete_key` and `delete_table`, and settles what `set_key` and `ensure_table` insert before a table's trailing banner rather than after it. This note records why the phase-0 spike wrote the behaviour off as unfixable, why that was wrong, the rule that was chosen and where it is a reading rather than a fact, and what it does not do for files already migrated by v0.46.0.## The reportdiff --git a/wip/migrator/migrator-phase3-progress.md b/wip/migrator/migrator-phase3-progress.mdindex 278a78b73..4bce65135 100644--- a/wip/migrator/migrator-phase3-progress.md+++ b/wip/migrator/migrator-phase3-progress.md@@ -1,6 +1,6 @@# S6 — Migrator Phase 3: working tracker-The charter is [`wip/migrator-3/sequencing.md` § S6](../../wip/migrator-3/sequencing.md) at the **workspace root** (from this repo: `../wip/migrator-3/sequencing.md`), with the phase content in `plan.md § Phase 3`. This file is the session-crossing record of what S6 has built, what it decided, and what is still open. Open the charter first, then this.+The charter is [`wip/migrator-3/sequencing.md` § S6](../../../wip/migrator-3/sequencing.md) at the **workspace root** (from this repo: `../wip/migrator-3/sequencing.md`), with the phase content in `plan.md § Phase 3`. This file is the session-crossing record of what S6 has built, what it decided, and what is still open. Open the charter first, then this.**Venue.** S6 lands as **two** PRs, not the one the charter assumes.@@ -208,7 +208,7 @@ Every new runner test was mutation-tested: dropping the resolve, clobbering the## Still open, in the charter's own words-Everything in [§ S6](../../wip/migrator-3/sequencing.md#s6--migrator-phase-3) not listed under Milestone 1 above. The ones that must be settled *here* rather than at S7:+Everything in [§ S6](../../../wip/migrator-3/sequencing.md#s6--migrator-phase-3) not listed under Milestone 1 above. The ones that must be settled *here* rather than at S7:- ~~**The dotted-key rename policy**~~ — done, see Milestone 3(a).- ~~**The backup and replace semantics pass**~~ — done, see Milestone 3(b).@@ -607,7 +607,7 @@ Three contracts were acked at milestone 7: `hub-layering-convention`, `cli-docs`## What is actually left-Measured against the charter's own **Done when** list ([§ S6](../../wip/migrator-3/sequencing.md#s6--migrator-phase-3)), verified against the tree rather than against this file:+Measured against the charter's own **Done when** list ([§ S6](../../../wip/migrator-3/sequencing.md#s6--migrator-phase-3)), verified against the tree rather than against this file:| Done when | State ||---|---|@@ -663,7 +663,7 @@ The last parked #1114 item. `display_health_report` printed nearly every dynamic### Deferred from the #1114 review-The items the review-bot pass on PR #1114 raised and this branch parked rather than fixed are in [`migrator-pr-1114-review-followups.md`](./migrator-pr-1114-review-followups.md), each verified against the code and each with its trigger, and their owners are recorded in the charter's [S6 Session 4](../../wip/migrator-3/sequencing.md#s6-session-4--milestones-6-to-10-pr-1114-and-the-two-review-rounds):+The items the review-bot pass on PR #1114 raised and this branch parked rather than fixed are in [`migrator-pr-1114-review-followups.md`](./migrator-pr-1114-review-followups.md), each verified against the code and each with its trigger, and their owners are recorded in the charter's [S6 Session 4](../../../wip/migrator-3/sequencing.md#s6-session-4--milestones-6-to-10-pr-1114-and-the-two-review-rounds):- **§1, the out-of-walk warning text** → S7's final join, because the reshape's `pipelex-config@2` is the first `pipelex-config` entry and is what makes it live (no consumer in the workspace passes `config_dir=` outside tests, so it is an embedder wording fix, not a blocker).- **§2, what a `migration` block's *presence* means for an unexplained-only plan** (agent hint, `_migration_prose`, the doctor's telemetry row, and the contract's own sentence — one ruling over four surfaces) → **part 3**, as a wording ruling the session may take on its own authority; not blocking S7, but S7's first machine is where a real user first meets it.diff --git a/wip/migrator/migrator-pr-1114-review-followups.md b/wip/migrator/migrator-pr-1114-review-followups.mdindex cf4e299d5..1449decb1 100644--- a/wip/migrator/migrator-pr-1114-review-followups.md+++ b/wip/migrator/migrator-pr-1114-review-followups.md@@ -22,7 +22,7 @@ An embedder's `Pipelex.make(config_dir=…)` outside `~/.pipelex/` and the projeThe doctor's telemetry row is a further surface of the same ruling ([thread](https://github.com/Pipelex/pipelex/pull/1114#discussion_r3792785607), which reported the sibling-file bug that was fixed): `_telemetry_is_out_of_date_or_wrong` read the plan of the file the row is about, but branched on `not plan.is_clean`, so an unexplained-only file — a typo under `extra="forbid"` — read `out_of_date`, named `pipelex migrate` (which writes nothing), and suppressed the manual-fixes section.-**The ruling, taken and built in part 3:** *a remedy is named only where it would write.* `MigrationErrorBlock` carries a `would_write: bool` (`any(plan.did_change for plan in plans)`), presence keeps meaning only *the migration history has something to say about these files*, and every surface branches on the field: the agent hint, the human handler in `error_handlers.py` (a fifth surface, which this section did not name and which had the identical shape), `_migration_prose`'s opening and closing sentences, and the doctor row, whose `out_of_date` now requires `own_plan.did_change` so an unexplained-only file falls to `invalid` and reaches Manual Fixes with the dry run's pointer. The three docs that stated the old contract moved with it. See `wip/migrator-phase3-progress.md` → Milestone 15.+**The ruling, taken and built in part 3:** *a remedy is named only where it would write.* `MigrationErrorBlock` carries a `would_write: bool` (`any(plan.did_change for plan in plans)`), presence keeps meaning only *the migration history has something to say about these files*, and every surface branches on the field: the agent hint, the human handler in `error_handlers.py` (a fifth surface, which this section did not name and which had the identical shape), `_migration_prose`'s opening and closing sentences, and the doctor row, whose `out_of_date` now requires `own_plan.did_change` so an unexplained-only file falls to `invalid` and reaches Manual Fixes with the dry run's pointer. The three docs that stated the old contract moved with it. See `wip/migrator/migrator-phase3-progress.md` → Milestone 15.## 3. The service-configuration loader still warns directly, and two commands load it before logging existsdiff --git a/wip/migrator/pr-1119-review-notes.md b/wip/migrator/pr-1119-review-notes.mdnew file mode 100644index 000000000..0bd56ffde--- /dev/null+++ b/wip/migrator/pr-1119-review-notes.md@@ -0,0 +1,43 @@+# PR #1119 review follow-ups — the one finding that was deferred rather than fixed++The v0.46.1 release PR drew three review-bot threads. Two were false positives answered on the spot, and one is deferred here: it names a real imprecision in the applier, but the state it describes cannot be produced by any migration operation today, and the patch the bot suggested would introduce a different wrong answer in its place. Deciding whether to harden the predicate anyway is a judgment call, not something a review pass should settle on its own, so the thread was left open and the reasoning written down here.++## The claim++`cubic` on `pipelex/pipeline/fixes/applier.py`, at `_at_document_start`:++> When an insertion has no trivia immediately before it but a banner after it, this returns `True` solely because the run is empty. `_settle_inserted` can then move the following section's banner above the newly inserted item instead of leaving it with that section; only a non-empty run can establish document-start status.++with the suggested one-liner `return bool(run.positions) and _is_first_entry(container=run.container, index=run.positions[0])`.++## The predicate really is imprecise++`_at_document_start` answers a question about the *run*, but every caller is asking about the *insertion point*. An empty run carries no position, so the function falls back to `not run.positions` — which is right when the insertion sits at index 0 and wrong when it sits anywhere else. The docstring ("whether the run is the first thing in the file") papers over the gap rather than closing it.++The imprecision can only ever surface in one place. At the two `_introduction_of_slot` call sites, and in `_settle_inserted`'s `else` arm, an empty run reaches `_introduction`, which finds no `Comment`, returns at `last_comment is None`, and never looks at the flag. Only `_settle_inserted`'s `if after.positions` arm can consume it, because that arm computes the flag from `before` and then applies it to a *combined* run whose contents come entirely from `after`.++## Why it is unreachable++Instrumented `_settle_inserted` over roughly 1,800 synthetic document/operation combinations (root and nested `set_key`, `ensure_table`, `move_key` in both directions, `delete_key`, `delete_table`, over documents varying the preamble, the banner shape, the blank lines and the tail) and over all 184 real ledger replays — the four shipped ledgers against every golden, kit config and packaged template.++- The flagged state (empty `before` run, root container, non-empty `after` run) is reached 102 times.+- In **every** one of them the `after` run is pure `Whitespace`. Not one contains a `Comment`, so `_introduction` short-circuits and the flag is never read.+- Output is byte-identical with and without the suggested patch across the whole corpus — zero differing cases.++The structural reason: tomlkit places a root-level insertion *after* any comment run that precedes the first table. Case A of the probe — `key_a = 1`, a banner, a blank line, `[b]` — puts the new key at body index 2, between the `Comment` and the `Whitespace`, so `before` comes back holding the comment rather than empty. For a comment to sit in the `after` run with nothing in the `before` run, an insertion would have to land *between* a value and the comment below it, which no placement path does.++## Why the suggested patch is not the fix++Forcing the state by hand (parsing a document with the key already in place and calling `_settle_inserted` on it) reproduces exactly what `cubic` described — and also shows the trap. The same patch flips the *genuine* document-start case:++| input | current | with the patch |+|---|---|---|+| `new_key = 2` / `# My pipelex config` / blank / `[b]` | preamble stays on top, above `new_key` | `new_key` stays on top, preamble pushed below it |++That first column is the documented preamble rule — "a hand-written config that starts `# My pipelex config` and then a section keeps its header line at the top". The `not run.positions` disjunct is what implements it when the insertion lands at index 0. Replacing it with `bool(run.positions)` trades one unreachable wrong answer for a different unreachable wrong answer, and picks the one that contradicts the stated rule.++## What a real fix would look like, and the open question++The honest repair is not to the run, it is to the signature: decide document-start from the insertion index (`run.positions[0]` when the run has one, `slot.first` when it does not) rather than from the run alone. That means changing `_at_document_start`'s parameters and both other call sites, and pinning it with a test for a state no operation can currently reach.++**Open for a human:** is that worth doing now, purely as hardening? The argument for is that the placement this relies on belongs to tomlkit, not to us, so a library upgrade could make the region live without anything in this repo changing. The argument against is that it is churn plus a test asserting an impossible state, for a defect no user can hit. Left open deliberately; the thread on PR #1119 is still unresolved so the question does not disappear.diff --git a/wip/reshape/s7-progress.md b/wip/reshape/s7-progress.mdindex 5ea33773d..cc4f95ff7 100644--- a/wip/reshape/s7-progress.md+++ b/wip/reshape/s7-progress.md@@ -83,4 +83,4 @@ The branch is pushed as two commits on top of the fourth join — the work, then- The runbook: [`wip/migrator-3/sequencing.md`](../../../wip/migrator-3/sequencing.md) § S7, and ⏸ C above it for the four corrections it folded in.- The #1111 review deferrals: [`pr-1111-review-notes.md`](./pr-1111-review-notes.md) — items 1 and 2 are the ones S7 owns and both are now done; 3, 4, 5, 6 and 7 have their own homes.-- The #1114 review deferrals: `wip/migrator-pr-1114-review-followups.md` on `dev` — §1 is the one S7 owns, and it is done.+- The #1114 review deferrals: `wip/migrator/migrator-pr-1114-review-followups.md` on `dev` — §1 is the one S7 owns, and it is done.