Git designed for AI agents.
doc: phase 4b acceptance results of window 1
Federico Elia17ecf6fe172d
diff --git a/doc/graft/BENCH.md b/doc/graft/BENCH.mdindex 55ff5ff..cddfcdf 100644--- a/doc/graft/BENCH.md+++ b/doc/graft/BENCH.md@@ -377,6 +377,83 @@ Reading it:(9868 to 2789 ms) and 50 agents (15317 to 12658 ms), and rose slightly at 100agents (21808 to 23253 ms), on a noisy machine.+#### Measured phase 4b results++Phase 4b ([PLAN8.md](PLAN8.md)) fixed the two causes of the latency that the+profile of the phase 4a tree found: about 12 fsyncs per batched request (each new+object fsynced when written and again by the quarantine migration) and nine short+git processes per request where one listing of the change's commits answers every+question. It also removed a second git process that the command factory started+for most git commands (`gitcmd` attr.tree detection), which speeds up every+command of the server, so the "off" column below is measured on that corrected+base too.++Runs of record: two runs of `acceptance-phase4.sh` in one benchmark window, on+the tree of commit `1899a0d`, with the defaults (`AGENT_COUNTS="20 50 100"`,+`CHANGES=5`, `RATE=0.2`, `SEED=1`, `window = "10ms"`, `max_size = 50`,+`target_latency = "500ms"`, `graph_ancestry = false`). Same machine as above+(Intel Core Ultra 7 255H, 16 logical CPUs, WSL2, ext4 on a virtual disk, an+fsync of a small file takes about 6 ms), with no other workload: the load average+was 4.2 when the first run started and rose to 38 and 48 during the runs, which is+the benchmark itself (both servers and up to 100 agents).++Both runs pass every criterion, the stretch goal included:++| criterion | 20 agents | 50 agents | 100 agents |+|---|---|---|---|+| attempts/landed (on), run 1 / run 2 | 1.00 / 1.00 | 1.00 / 1.00 | 1.00 / 1.00 |+| failed = planned conflicts (on) | 22 / 22 | 50 / 50 | 108 / 108 |+| p99 on vs off (ms), run 1 | 1347 vs 7437 | 2525 vs 12030 | 4854 vs 10077 |+| p99 on vs off (ms), run 2 | 1193 vs 6741 | 4839 vs 12746 | 2897 vs 11905 |+| stretch: p99 on-100 <= 0.5 x off-100 | | | 4854 <= 5038 (run 1), 2897 <= 5952 (run 2) |++The stretch goal passed by 3.7% in run 1 and by 51% in run 2.++| metric (batching on), run 1 / run 2 | 20 agents | 50 agents | 100 agents |+|---|---|---|---|+| lands/min | 1020 / 1070 | 1230 / 944 | 1616 / 2325 |+| lands/min, phase 4a run of record | 412 | 251 | 258 |+| p50 landed (ms) | 911 / 739 | 1700 / 1980 | 2653 / 1824 |+| batch size mean | 8.2 / 8.3 | 13.4 / 8.9 | 13.9 / 18.5 |+| duration | 4.6 s / 4.4 s | 9.8 s / 12.7 s | 14.6 s / 10.1 s |++With batching off the runs took 17 to 41 s and landed 61 to 116 changes, as before:+the off column still loses most lands to the 20-attempt limit at 50 and 100 agents.++Time per phase, mean per observation, run 1 / run 2:++| phase | 20 agents | 50 agents | 100 agents |+|---|---|---|---|+| land_member (per request) | 8.5 / 7.6 ms | 7.4 / 8.5 ms | 7.5 / 8.1 ms |+| plan (per request) | 3.3 / 3.1 ms | 3.1 / 3.3 ms | 3.1 / 3.6 ms |+| replay (per rebased request) | 4.8 / 4.2 ms | 3.9 / 4.8 ms | 4.1 / 4.2 ms |+| pack (per batch) | 55 / 47 ms | 75 / 69 ms | 53 / 45 ms |+| update (per batch) | 126 / 136 ms | 212 / 210 ms | 175 / 135 ms |+| total (per batch) | 238 / 233 ms | 351 / 335 ms | 321 / 324 ms |++Reading it:++- A batched request costs about 8 ms instead of 80 to 137 ms in phase 4a: one+ `git rev-list` for the decision and a replay through the long-lived merge-tree+ and hash-object processes, without fsync per object.+- The lane is still busy for most of a run, but each batch now spends most of its+ time publishing: packing the objects of the batch and the update (the+ compare-and-swap, the hooks and the changeset log). At 100 agents the publish+ is about 70% of a batch. The update took 135 to 212 ms here, against 20 to 30 ms+ (without the quarantine migration) in a profile at lower load; why it grew under+ load is being profiled.+- The variance between runs is large: p99 at 50 agents was 2525 ms in run 1 and+ 4839 ms in run 2, p99 at 100 agents 4854 and 2897 ms. The benchmark saturates+ the machine (load average 38 to 48 during the runs), so single runs are not+ precise; compare runs of one window, and rerun before comparing across trees.+- The stretch goal passed with a margin of only 3.7% in run 1.+- The off column moved too. At 100 agents its p99 (10077 and 11905 ms) is lower+ than in the phase 4a run of record (13368 ms); at 50 agents it is higher (12030+ and 12746 ms against 9939 ms). One known change applies to it: every git command+ of the server now starts one process instead of two for most commands. Why the+ off p99 at 50 agents rose is not explained; its p99 covers only the 97 changes+ that landed, out of 250.+## Reading the numbers- Compare `land` and `land-resolve` at the same `--conflict-rate`. The firstdiff --git a/doc/graft/LAND-BATCHING.md b/doc/graft/LAND-BATCHING.mdindex 154bc63..d9e4219 100644--- a/doc/graft/LAND-BATCHING.md+++ b/doc/graft/LAND-BATCHING.md@@ -392,6 +392,28 @@ the running tip, so it cannot delay or block the other requests of its batch.## Measured results+### Phase 4b++Two runs of `examples/graft/demo/acceptance-phase4.sh` on the phase 4b tree (one+benchmark window; full tables, phases and the machine in+[BENCH.md](BENCH.md#measured-phase-4b-results)) **meet the phase 4a acceptance**,+the stretch goal included: one attempt per landed change, only planned conflicts+fail, and p99 with batching on is below off at every size.++| p99 landed (ms), run 1 / run 2 | 20 agents | 50 agents | 100 agents |+|---|---|---|---|+| batching on | 1347 / 1193 | 2525 / 4839 | 4854 / 2897 |+| batching off | 7437 / 6741 | 12030 / 12746 | 10077 / 11905 |+| lands/min, batching on | 1020 / 1070 | 1230 / 944 | 1616 / 2325 |++The stretch goal (p99 on at 100 agents at most half of off) passed by 3.7% in run 1+and by 51% in run 2; the runs vary a lot, and the benchmark saturates the machine.+A batched request now costs about 8 ms (one range listing and a replay without+fsync per object, see *How a request is decided and rebased*), and most of a batch+is its publish: the pack of its objects and the update.++### Phase 4a+The phase 4a acceptance run of record (`examples/graft/demo/acceptance-phase4.sh`,20, 50 and 100 agents of 5 changes, conflict rate 0.2, seed 1; full tables, theother runs and the machine in [BENCH.md](BENCH.md#measured-phase-4a-results))@@ -417,9 +439,9 @@ The phase breakdown shows where a batch spends its time: landing the membersanother) takes 56 to 72% of it, and the reference update with its hooks(`update`) 27 to 42%. The lane is busy for almost the whole run, so with manyagents most of the latency is time spent queued behind earlier batches. Turning-adaptive sizing off does not change that. Phase 4b (pipelined batches) must let-a batch land its members while the batch before it publishes, and must cut or-parallelize the per-member cost.+adaptive sizing off does not change that. The profile of phase 4b found where the+per-member time went (fsyncs of loose objects and nine short git processes per+member) and removed it; see [PLAN8.md](PLAN8.md).## Limitsdiff --git a/doc/graft/LIMITATIONS.md b/doc/graft/LIMITATIONS.mdindex d7a9e15..137bca4 100644--- a/doc/graft/LIMITATIONS.md+++ b/doc/graft/LIMITATIONS.md@@ -262,8 +262,33 @@ commitment.6. Multi-node cache tiers with explicit invalidation for mutable keys.7. Signed changesets and a verifiable export of the log.+## Phase 4b limitations++1. The phase 4a acceptance is met by two runs on one machine and one workload+ (seed 1, conflict rate 0.2; see [BENCH](BENCH.md#measured-phase-4b-results)).+ The runs vary a lot, the benchmark saturates the machine, and the stretch goal+ passed by only 3.7% in one of the two runs.+2. Batches still run one at a time per lane. Each batch now spends most of its time+ publishing (packing its objects and the update with its hooks and the+ changeset log). Why the update takes much longer under load than at low load is+ not yet known; overlapping the publish of one batch with the work of the next+ (pipelined batches) is not implemented and is only planned if that cost turns+ out to be inherent to a batch.+3. Each published batch adds one small pack to the repository, as a push does.+ Housekeeping repacks them like the packs of pushes, so busy branches make+ repacks more frequent.+4. The objects of a batch are written without fsync and made durable by the pack+ written before the update. An operator who sets `core.fsync` in the `[git]`+ configuration overrides this for the helper processes too (the configuration+ of the administrator is applied last), which keeps the old per-object fsyncs.+5. Until phase 4b's single-land work (WP86 of PLAN8) lands, an unbatched land still+ decides with several history walks and rebases with `localrepo.Rebase`, without+ the pack of a batch.+## Phase 4a limitations+(Item 1 is resolved by phase 4b, see above. The text describes phase 4a.)+1. Batches still run one at a time per lane. Pipelined batches (phase 4b) are notimplemented, so a lane's throughput is bounded by its batch duration. Thelatency gain is meant to come from the streaming rebase engine and early