Skip to content

Git designed for AI agents.

doc: record the G1 decision and the PLAN8 amendments

Federico Elia22f7e82622d9
diff --git a/doc/graft/LAND-BATCHING.md b/doc/graft/LAND-BATCHING.md
index d9e4219..aa47f51 100644
--- a/doc/graft/LAND-BATCHING.md
+++ b/doc/graft/LAND-BATCHING.md
@@ -113,8 +113,9 @@ With `[graft.land] graph_ancestry = true` the listing walks the stored `graph`
index in process instead of running `git rev-list`; both return the same
commits. The graph checker refuses repositories with shallow commits, grafts or
replace refs, because git walks their history differently; for those it logs a
-warning and uses `git`. The single-land path answers its ancestry questions with
-the same switch, and still rebases with the repository's `Rebase`.
+warning and uses `git`. A land that is not batched is decided, rebased and
+published the same way, by the same code (`landingAttempt`): it only differs in
+its update, which runs with the request's own identity and hooks.
### Durability of the objects of a batch
diff --git a/doc/graft/LIMITATIONS.md b/doc/graft/LIMITATIONS.md
index 137bca4..6e13b31 100644
--- a/doc/graft/LIMITATIONS.md
+++ b/doc/graft/LIMITATIONS.md
@@ -268,12 +268,16 @@ commitment.
(seed 1, conflict rate 0.2; see [BENCH](BENCH.md#measured-phase-4b-results)).
The runs vary a lot, the benchmark saturates the machine, and the stretch goal
passed by only 3.7% in one of the two runs.
-2. Batches still run one at a time per lane. Each batch now spends most of its time
+2. Batches still run one at a time per lane, and each batch spends most of its time
publishing (packing its objects and the update with its hooks and the
- changeset log). Why the update takes much longer under load than at low load is
- not yet known; overlapping the publish of one batch with the work of the next
- (pipelined batches) is not implemented and is only planned if that cost turns
- out to be inherent to a batch.
+ changeset log). Pipelined batches, which would overlap the publish of one batch
+ with the work of the next, are not built: a profile showed that the inherent part
+ of the publish (about 65-70 ms) is smaller than the member work of a batch, and
+ that its growth under load comes from two causes outside the lane, which are
+ planned next: the quarantine migration fsyncs again every file that is already
+ durable, and every reference update of a repository waits for the changeset-log
+ order of the updates before it (shared with `UserCommitFiles`). See PLAN8,
+ section 9.
3. Each published batch adds one small pack to the repository, as a push does.
Housekeeping repacks them like the packs of pushes, so busy branches make
repacks more frequent.
@@ -281,13 +285,13 @@ commitment.
written before the update. An operator who sets `core.fsync` in the `[git]`
configuration overrides this for the helper processes too (the configuration
of the administrator is applied last), which keeps the old per-object fsyncs.
-5. Until phase 4b's single-land work (WP86 of PLAN8) lands, an unbatched land still
- decides with several history walks and rebases with `localrepo.Rebase`, without
- the pack of a batch.
+5. An unbatched land decides, rebases and publishes like a batch attempt (one range
+ listing, the rebase engine, one pack), but it is still bounded by its own
+ compare-and-swap retries under contention (`max_attempts`).
## Phase 4a limitations
-(Item 1 is resolved by phase 4b, see above. The text describes phase 4a.)
+(Items 1 and 2 are resolved by phase 4b, see above. The text describes phase 4a.)
1. Batches still run one at a time per lane. Pipelined batches (phase 4b) are not
implemented, so a lane's throughput is bounded by its batch duration. The
diff --git a/doc/graft/PLAN8.md b/doc/graft/PLAN8.md
index 0186c32..b8054a3 100644
--- a/doc/graft/PLAN8.md
+++ b/doc/graft/PLAN8.md
@@ -1,7 +1,8 @@
# Graft phase 4b: land batch latency, root causes first (binding plan)
-Status: proposal of the p4b workstream, waiting for the PM's approval. Nothing in
-this plan is implemented. Owner: p4b (Opus). Branch `p4b-pipelined-batches`.
+Status: approved by the PM; stage 1 implemented and accepted (window 1), stage 2
+closed by measurement (section 9); amendments in section 13a. Owner: p4b (Opus).
+Branch `p4b-pipelined-batches`.
Phase 4a is in [PLAN4.md](PLAN4.md); its acceptance and the measured misses are in
[BENCH.md](BENCH.md#measured-phase-4a-results) and
@@ -441,6 +442,29 @@ phase table shows the publish as the largest share of the lane time. Otherwise i
is not built: it would add two batches in flight per lane for a gain the
acceptance does not need.
+**G1 decision (after window 1): closed by measurement, stage 2 is not built.**
+Window 1 (two acceptance runs on `1899a0d`, see BENCH.md) passed A1, A2, A3, S and
+T in both runs. By the letter G1 opened: the stretch margin was 3.7% in run 1, and
+the publish (pack + update) was about 70% of a batch at 100 agents. The rule asks
+whether that publish is inherent per-batch work, so the update phase was profiled
+(probes, batching on, 50 and 100 agents, load 5-8 at the start):
+
+| part of the publish, per batch | 50 agents | 100 agents | nature |
+|---|---:|---:|---|
+| pack (pack-objects, 3 fsyncs) | 53 ms | 46 ms | inherent |
+| quarantine migrate (3 pack files and 2 directories, fsynced again) | 38 ms | 42 ms | redundant: the pack files are already fsynced |
+| committed hook: changeset-log ticket wait | ~40 ms | ~22 ms | contention with the transactions of `UserCommitFiles` |
+| committed hook: changeset-log append | 11 ms | 11 ms | inherent |
+| update-ref (lock, prepare, commit) | 11 ms | 11 ms | inherent |
+
+The inherent publish is about 65-70 ms against about 170 ms of member work per batch
+(14 members of 12 ms), so overlapping it would hide less than it costs in
+complexity, and the growth of the update under load (135-212 ms in window 1) is
+the redundant fsyncs and the ticket wait. Both have their cause outside the lane:
+the durability contract of `quarantine.Migrate` (it re-fsyncs every file it links,
+for every writer) and the per-repository changeset-log order. They move to the
+next workstream (p5w, PLAN9), with WP87.
+
## 10. Test plan
Unit and package tests:
@@ -519,6 +543,35 @@ output. Haiku (one at a time, never two in this worktree) drafts the `TestSpec*`
files from an exact spec, the `bench-batch.sh` table rows and the first drafts of
the doc sections, all checked by Opus.
+## 13a. Amendments after approval (binding, recorded by p4b with the PM's decisions)
+
+* **WP86 (single-land path, in p4b):** after p4b there is one land decision and one
+ replay. `landingAttempt` (`land_attempt.go`) is the attempt of every land: quarantine,
+ peel through the object info process, `ancestry.Checker.Range` +
+ `rebase.PlanLanding`, `Engine.Replay`, the merge and fast-forward-only strategies,
+ and `publish` (the pack). The batch stepper embeds it and adds the in-process
+ admission and the combined update; `landAttempt` (unbatched) uses it with the
+ request's own update and hooks. Deleted: `computeLandedTip`,
+ `rejectMergeCommitsInRange`, `hasMergeBase`, the `localrepo.Rebase` call of the
+ single path, and `IsAncestor` / `MergeBase` of `ancestry.Checker` (no caller was
+ left; their tests now check `Range`). The unbatched land observes no batch phase.
+* **WP88 (command factory, in p4b):** a git command outside the attr.tree whitelist
+ no longer starts `git rev-parse --show-object-format` first; the object format is
+ read from the configuration file of the common git directory (resolving `.git`
+ files and `commondir` like git), and a failure to read it fails the command.
+ `cmdCfg.worktreePath` is never set in this tree (dead upstream code that predates
+ gitai); the resolution is implemented and tested for it anyway.
+* **Engine.Replay** takes `committer *git.Signature` instead of rebase options (only
+ the committer applied); `localrepo/rebase_options_export.go` was deleted as dead.
+* **Peel errors:** only `catfile.NotFoundError` is `InvalidArgument` ("cannot resolve
+ commit ID"); a failure of the object info process is `Internal`.
+* **G1:** closed by measurement (section 9).
+* **Not in p4b (moved to p5w, PLAN9):** writing the pack straight into
+ `objects/pack` was rejected (it would make a rejected attempt's objects
+ addressable before the hooks accept the update); the root cause, the re-fsync in
+ `quarantine.Migrate`, the Migrate-versus-housekeeping race (WP87) and the
+ changeset-log ordering go to p5w.
+
## 14. Risks and open questions
Risks: