Git designed for AI agents.
PLAN11 docs: the land identity as built, and the documents it changes
Federico Elia420fd953eda5
diff --git a/doc/graft/ARCHITECTURE.md b/doc/graft/ARCHITECTURE.mdindex b33a6dc..692489b 100644--- a/doc/graft/ARCHITECTURE.md+++ b/doc/graft/ARCHITECTURE.md@@ -94,12 +94,15 @@ client graft-cache (optional) gitaly (standalone)|------------------------------->|----------------------------->| 1. read the tip T of main| | | 2. T is an ancestor of C: fast-forward| | | C is reachable from T: already_landed+ | | | C landed before as C', C' reachable+ | | | from T: already_landed (PLAN11)| | | otherwise rebase C onto T| | | 3. update hooks and the standalone policy| | | 4. compare-and-swap main from T to C'| | | lost the race: retry from 1 (jittered)| | | 5. changeset: operation=land, actor,- | | | rewritten=true when history changed+ | | | rewritten=true when history changed,+ | | | lands=[C -> C']| UserLandResponse | ||<-------------------------------|<-----------------------------|| | |diff --git a/doc/graft/CONFIGURATION.md b/doc/graft/CONFIGURATION.mdindex 8e3e96b..87f0397 100644--- a/doc/graft/CONFIGURATION.md+++ b/doc/graft/CONFIGURATION.md@@ -169,7 +169,7 @@ See [LAND-BATCHING.md](LAND-BATCHING.md).|---|---|---|---|| `disabled` | bool | `false` | Turn batching off. Every request then runs its own compare-and-swap loop, as in phase 2. || `window` | string (Go duration) | `"10ms"` | How long a lane waits after its first queued request before it runs the batch. `"0s"` means no wait: a batch is whatever queued while the previous batch ran. Must not be negative. |-| `max_size` | int | `50` | Largest number of requests in one batch. Zero means the default; negative values are rejected. |+| `max_size` | int | `50` | Largest number of requests in one batch, at most `100`: the update of a batch records the identity of every member ([PLAN11](PLAN11.md)) in the environment of its git processes. Zero means the default; negative values and values above `100` are rejected. || `target_latency` | string (Go duration) | `"500ms"` | Batch duration the lane aims for. A lane halves its effective maximum batch size after a batch slower than this, and grows it back (up to `max_size`) after batches faster than half of it. `"0s"` disables the adaptation, and the lane always uses `max_size`. Must be a valid, non-negative Go duration. See [LAND-BATCHING.md](LAND-BATCHING.md#adaptive-sizing). |### `[graft.land.resolver]`diff --git a/doc/graft/LAND-BATCHING.md b/doc/graft/LAND-BATCHING.mdindex e4d3aee..826d001 100644--- a/doc/graft/LAND-BATCHING.md+++ b/doc/graft/LAND-BATCHING.md@@ -69,6 +69,10 @@ One batch attempt:reports `already_landed` and changes nothing.- If the running tip is an ancestor of the commit, the commit becomes the newrunning tip (fast-forward).+ - If an earlier land of the commit on the branch put a commit on it that is+ still reachable from the running tip, the request reports+ `already_landed` with that commit and changes nothing (the land identity+ of [PLAN11](PLAN11.md)).- Otherwise the commit is rebased onto the running tip, or merged for`STRATEGY_MERGE`. `STRATEGY_FF_ONLY` fails that request with a`not_ancestor` error.@@ -374,8 +378,9 @@ graph_ancestry = false # true: ancestry from the stored graph indexThe keys are documented in [CONFIGURATION.md](CONFIGURATION.md#graftlandbatch)and [CONFIGURATION.md](CONFIGURATION.md#graftland). `window` and-`target_latency` must be non-negative Go durations. `max_size` must not be-negative; zero means the default.+`target_latency` must be non-negative Go durations. `max_size` must be between+0 and 100; zero means the default. The update of a batch carries the identity+of every member in the environment of its git processes, which bounds it.## Metrics@@ -483,16 +488,17 @@ member) and removed it; see [PLAN8.md](PLAN8.md).an exponential backoff from 100 ms to 2 s with full jitter, stopped by thecontext, reported in `LandResult.UnavailableRetries`. A client that does notuse the SDK must do the same: retry `Unavailable`, and only `Unavailable`, with- a bounded backoff, and only for the lands below.- - Merge and fast-forward lands are retried by default. The response of a call- can be lost after the change landed; the change itself is then on the- branch, and the retry reports it as already landed.- - Rebase lands, with or without conflict resolution, are not retried by- default (`RebaseLands` turns it on). The branch holds a rebased copy of the- change, and the server cannot yet tell that this change landed: a retry- replays it, and a change that a later commit reverted in between (for- example graft-verify reverting the culprit) would land again. The land- identity index of [PLAN11](PLAN11.md) removes this limit.+ a bounded backoff. Every strategy is retried by default+ (`UnavailableRetry.SkipRebaseLands` turns it off for rebase lands). The+ response of a call can be lost after the change landed, and the retry then+ lands nothing and reports `already_landed`: with a merge or a fast-forward,+ the change itself is on the branch; with a rebase, with or without conflict+ resolution, the branch holds a rebased copy, and the server answers from its+ record of the first land (the land identity of [PLAN11](PLAN11.md)), with+ `landed_commit_id` the copy and `landed_sequence` its changelog sequence.+ This holds also when a later commit reverted the change in between (for+ example graft-verify reverting the culprit): the change is not landed again,+ and `reverted_by_commit_id` names the revert.- A land that arrives alone waits up to `window` before its batch runs. Set`window = "0s"` to remove the wait, or `disabled = true` to remove batching.- Requests that do not qualify (see *When a land is batched*) run unbatched anddiff --git a/doc/graft/LIMITATIONS.md b/doc/graft/LIMITATIONS.mdindex b0f155b..49d08e4 100644--- a/doc/graft/LIMITATIONS.md+++ b/doc/graft/LIMITATIONS.md@@ -182,6 +182,20 @@ Git hooks use private.concurrent writer is only detected when the snapshot is published. The landthen fails after one attempt with `reference_update`, and the client has toland again. The changeset log only lists the lands that were published.+- A change that landed is not landed again from the same `commit_id` on the+ same branch while the commit it landed as is on the branch, also when a+ later commit reverted it: the land answers `already_landed` and names the+ revert when its message says "This reverts commit <id>" (the land identity+ of [PLAN11](PLAN11.md)). Landing it again needs a new commit: a revert of+ the revert, or the change committed again. A branch reset past the land+ lands the change again.+- The land identity is recorded in the changeset log. With+ `[graft.changelog] disabled = true` nothing is recorded, and a retried+ rebase land replays the change: a change reverted in between lands again.+- In the file changelog, the land identity index (`graft/changelog.lands`)+ holds every land of the repository and is kept in memory for the 64 most+ recently used repositories of a process; a repository beyond them reads its+ index file again on its next land that diverges.- Hooks run on every attempt, so a slow custom hook adds latency undercontention.- `graft land --strategy merge` and `ff_only` are available in the CLI. Thediff --git a/doc/graft/PLAN11.md b/doc/graft/PLAN11.mdindex 728dfd7..7e4416c 100644--- a/doc/graft/PLAN11.md+++ b/doc/graft/PLAN11.md@@ -17,8 +17,8 @@ twice and must report what happened.in between (graft-verify reverting the culprit), the replay is not empty: the retrylands the change again.-Until this plan is in, the SDK does not retry rebase lands by default-(`UnavailableRetry.RebaseLands`, commit f13ad58).+Until this plan was in, the SDK did not retry rebase lands by default+(commit f13ad58).Federico's decision (2026-10-19): the server keeps an index of what landed, by theidentity of the request, with no trailer in the commits.@@ -44,12 +44,15 @@ lands and for the unbatched lands of a lane (`UpdateSolo`), not for the classic`landWithRetries` path, and `Members` are capped at 32 KiB in the hook payload, whichcan drop members of a large batch.-* A new field of the changelog metadata, `Lands []LandRecord{Original, Landed}`,- separate from `Members`, carries the identity pairs. It is never cut: a pair is- under 200 bytes, a batch has at most `max_size` (50) members, so the field is at most- about 10 KiB. A request whose pairs would not fit the payload budget fails before- its update rather than landing without its identity (it cannot happen with the- configured limits; the check makes the bound an invariant).+* A new field of the changelog metadata, `Lands []Land{Original, Landed}` with the+ branch they landed on (`LandReference`), separate from `Members`, carries the+ identity pairs; only the entry of that branch's update carries them. It is never+ cut. `changelog.WithLands` validates them (a branch, full object IDs, at most 100+ pairs, at most 40 KiB encoded) and a land whose identities do not pass fails+ before its update rather than landing without its identity. A SHA-256 pair+ encodes in 156 bytes and `max_size` is now at most 100, so 100 pairs and a+ 4096-byte branch name, escaped by JSON at worst into 6 bytes per byte, fit: the+ check makes the bound an invariant.* Every land path sets it: the batch update, `UpdateSolo` and `landWithRetries`(through `updateLandedReference`, which knows `commit_id` and the new tip).* The MVCC path records after its commit (`AfterCommit`), with the same metadata.@@ -74,9 +77,13 @@ graft_land_identities(A land of an identity that has a row and whose landed commit is still on the branchdoes not land (section 5), so it writes no row. One whose landed commit is no longer onthe branch (the branch was reset past it) lands as a new change and replaces the row-with its newer sequence (`ON CONFLICT ... DO UPDATE` when the new sequence is greater).+with its newer sequence (`ON CONFLICT ... DO UPDATE`: every writer appends after the+last entry, so the last write is the latest land). The rows are written by every+insert of changelog entries: a changeset, the initial history of a registered+repository and the history that shadow mode imports, so a repository moved to the+authoritative mode keeps the identities of its file changelog.-Migration `2026102xxxxxxx_graft_land_identities`, in both dialects, with the rule of+Migration `20261020000000_graft_land_identities`, in both dialects, with the rule ofmilestone 3: a node whose database has a migration it does not know refuses to start(`unknown migrations`), so an older binary never writes changesets without the rows.A repository removed by `RemoveRepository` loses its rows by `ON DELETE CASCADE`.@@ -88,28 +95,44 @@ The record is part of the log entry itself: the entry of the branch update getswrite and fsync. A secondary index file next to the log,`graft/changelog.lands`, makes the lookup cheap:-* JSON lines `{"seq":N,"branch":"...","original":"...","landed":"..."}`, appended- after the log append, with its own fsync, and a header line recording the log- sequence it is complete up to.-* It is derived data. A crash between the log append and the index append leaves the- index behind the log; the first lookup after it reads the log after the index's- sequence (`Scan` from that cursor) and appends what is missing. A torn last line is- cut off, as in the log. A missing or corrupt index is rebuilt from the log.-* The lookup loads the index once per process and keeps it in memory, updated by the- appends of the process and caught up from the log when the log's durable end is past- the index (another process appended).+* The file is a sequence of blocks of JSON lines: the records+ `{"seq":N,"ref":"<base64>","original":"...","landed":"..."}` of the lands of some+ log entries, then a mark `{"mark":N,"ref":"<base64>","new_oid":"...","crc":C}`+ naming the last log entry the block covers and the CRC-32 of its records. The index+ is complete up to its last mark.+* It is derived data, written by the lookups, not by the appends: a lookup reads the+ durable log entries after the last mark (`Scan` from it), adds their lands to the+ index and appends them as one block. An append therefore never waits for the index,+ and a crash between the log append and the index leaves nothing to repair: the+ next lookup reads those entries.+* Reading the file stops at the first line that is not a record or a mark, at a mark+ whose CRC does not match its records, and at a mark that does not follow the one+ before; the file is cut there and caught up from the log. A torn tail, a lost or+ corrupt file are handled so. The mark proves that the index describes this log:+ the lookup reads the log entry at the mark first, and an index whose mark names+ another entry, or one past the end of the log (the repository was created again at+ the same path), is rebuilt from the whole log. The file is written without fsync,+ since whatever a crash leaves of it is either a good prefix or cut off.+* The lookup keeps the index in memory, for the 64 most recently used repositories of+ the process, and reads the file again when another process changed it. A lookup+ whose index is already at the log's last durable sequence reads nothing.## 5. The land path-Before a rebase replays a diverged change (`LandDiverged` with the rebase strategy),-the land looks up `(branch, commit_id)`:+Before a land computes a diverged change (`LandDiverged`, whatever the strategy: a+rebase replays it, a merge or a resolution merges it, `STRATEGY_FF_ONLY` fails), the+land looks up `(branch, commit_id)`. A merge commit, of a merge or of a resolution,+has `commit_id` as a parent, so its retry already finds `commit_id` reachable; the+lookup answers the rebased lands, and makes the answer the same for every strategy:* A hit whose landed commit is reachable from the current tip (one`merge-base --is-ancestor`): the change is on the branch. The answer is`AlreadyLanded`, `landed_commit_id` the recorded commit, nothing replayed, noresolver run, whatever happened on the branch since (a revert included). The answer- explains itself: its message says "landed as <oid> at changelog <seq>", and adds "it- was reverted later by <oid>" when a revert of it is on the branch (section 5.1).+ explains itself: `UserLandResponse` gains `landed_sequence`, `reverted_by_commit_id`+ and `already_landed_detail`, which says "landed as <oid> at changelog <seq>", and+ adds "; it was reverted later by <oid>" when a revert of it is on the branch+ (section 5.1). `branch_update.commit_id` stays the tip.* A hit whose landed commit is not reachable (the branch was reset past it): the changeis no longer on the branch; the land proceeds as a new one.* No hit: the land proceeds.@@ -133,12 +156,15 @@ landed again from the same commit. The land answers `AlreadyLanded`, because thedid land and is still in the branch's history. Landing it again needs a new commit (arevert of the revert, or the change recommitted), which is a new identity. A merge or afast-forward land behaves so already today (the reverted `commit_id` is still reachable,-so it answers `AlreadyLanded`); this plan gives a rebase land the same rule. Today a-rebase re-land lands the change again; the documentation of `UserLand` and of-graft-verify's revert says so with this plan.+so it answers `AlreadyLanded`); this plan gives a rebase land the same rule. Before+it, a rebase re-land landed the change again; the documentation of `UserLand`+(LIMITATIONS, LAND-BATCHING) and of graft-verify's revert (VERIFY) says so.-With this in, the SDK retries rebase lands by default again (`RebaseLands` stays, to-turn it off).+With this in, the SDK retries rebase lands by default again;+`UnavailableRetry.SkipRebaseLands` turns it off (it replaces `RebaseLands`, which was+never released: a field whose zero value is the default cannot turn a default off).+`LandResult` reports `LandedSequence`, `RevertedByCommitID` and+`AlreadyLandedDetail`.## 6. Option: `Unavailable` that is always safe to retry (m2)@@ -153,7 +179,8 @@ after this plan.## 7. Tests* Lost response, then retry: `AlreadyLanded` with the rebased copy, for rebase,- rebase with resolution, merge and fast-forward.+ rebase with resolution, merge and fast-forward. (Built: a resolution lands a merge+ commit, so its retry answers by reachability, as a merge does.)* Lost response, a revert of the change on the branch, retry: `AlreadyLanded`, therevert stays, nothing replayed (the case of 1).* A batch: every member recorded, a retried member answers `AlreadyLanded`.@@ -164,8 +191,8 @@ after this plan.catches up and answers `AlreadyLanded`. In the database the row and the changeset areone transaction (a test that fails the row insert fails the changeset).* The payload bound: a batch of `max_size` members with the longest branch name fits.-* The SDK: `RebaseLands` on by default again; an end-to-end retry after a forced lost- response.+* The SDK: rebase lands retried by default again; an end-to-end retry after a forced+ lost response.## 8. Ownership neededdiff --git a/doc/graft/VERIFY.md b/doc/graft/VERIFY.mdindex 0175359..448e49e 100644--- a/doc/graft/VERIFY.md+++ b/doc/graft/VERIFY.md@@ -118,9 +118,18 @@ retry rules as `UserLand`. The commit message is:```Revert "<culprit subject>"+This reverts commit <culprit oid>, which failed the verification test command.+Graft-Auto-Revert: <culprit oid>```+A reverted culprit is not landed again from the same commit: a `UserLand` of+its `commit_id` on the branch, for example the retry of a land whose response+was lost, answers `already_landed` with the culprit as `landed_commit_id` and+this revert as `reverted_by_commit_id` (the land identity of+[PLAN11](PLAN11.md)). To land the change again, the author makes a new commit:+the fixed change, or a revert of the revert.+Safeguards:- `dry_run = true` reports the culprit and the planned revert, and changes