Git designed for AI agents.
docs(plan7): propose review roots as references (amendment)
Federico Elia786e350d9a8e
diff --git a/doc/graft/PLAN7.md b/doc/graft/PLAN7.mdindex c7945cb..575a7b5 100644--- a/doc/graft/PLAN7.md+++ b/doc/graft/PLAN7.md@@ -416,6 +416,123 @@ between a review write and a checkpoint commit.change, with tests), `internal/graft/corestore/dbreview/**` (carries `S` to thestore), `sqlstore` (the conditioned review writes and the R2 check).+### REVIEW ROOTS amendment: review roots are references (proposed after WP74, PM decision pending)++**Finding (WP74).** Gitaly housekeeping on a node keeps only what local+references reach. After its expiry it can drop the commit of an open review+whose branch was deleted. Checkpoint builds on that node then refuse with+`FailedPrecondition` until the review closes. Re-hydrating would loop:+housekeeping would prune the object again. The cause is the same as for R1/R2:+an open review is a root that neither the local repository nor the reference+graph of the storage core knows about. Every mechanism that decides liveness+(local housekeeping, the voter's `--not --all`, the checkpoint closure, GC)+respects references only.++**Facts in the code (evidence).**+1. Every reference write by a service goes through the voter:+ `updateref.Updater` always runs with `gitcmd.WithRefTxHook`+ (`internal/git/updateref/updateref.go:320`). The hook manager calls+ Preparing/Prepared for every reference transaction (`voter.go` header). So a+ ref the review service writes becomes a changeset with a database+ compare-and-swap. Its pack holds every object not reachable from the local+ references (`gitops.BuildPack`, `--not --all`), so a commit that only the+ cache has is uploaded with it.+2. The database records every reference name, while the Graft changelog records+ only branches and tags (`sqlstore/changeset.go:202`, `IsChangelogRef`;+ `changelog.ParseUpdates` names `refs/keep-around/*` as not recorded). A+ keep-around write therefore adds a changeset but no changelog entry and no+ `graft watch` event.+3. `refs/keep-around/` is an internal hidden prefix (`git/reference.go:62`,+ `InternalReferenceTypeHidden`). Gitaly adds it to `uploadpack.hideRefs` and+ `receive.hideRefs` (`gitcmd/command_description.go:453-484`), so clients+ never see it advertised and can neither push nor delete it.+ `ListRefs` shows it only to a caller whose patterns include it, as in+ upstream Gitaly. The graft CLI and the SDK list no references.+4. Hydration, catch-up and verify treat every database reference alike, so+ keep-around references reach every node through the normal path.++**Option (a), recommended: review roots are references.** A review's head and+base are protected by `refs/keep-around/<oid>` (target `<oid>`), written+through the ordinary write path (voter, database CAS, changeset).+* **Writes.** `CreateReview` writes the keep-around references of head and base+ (one reference transaction, which creates the absent ones), then inserts the+ review. `UpdateReview` writes the new head's reference, then updates the+ review. Comments, verdicts and closes write no reference. The request path+ never deletes one.+* **Invariant, in the database (authoritative mode).** "Every open review's head+ and base has its keep-around reference in `graft_refs`." Two checks, both+ under the repository row lock (LOCK ORDER), keep it:+ * `InsertReview` and `UpdateReview` with `requireRoots` (replacing `rootsAt`)+ require the references to exist with that value, else+ `ErrReviewRootsMissing`. The service then writes them again and retries, at+ most 3 attempts, then Aborted. This happens only if a sweep deleted the+ reference between the two steps.+ * `CommitChangeset` refuses to delete a `refs/keep-around/<oid>` that an open+ review uses (a reference conflict). This also protects them from+ `DeleteRefs`/`WriteRef` callers.+* **Deletion.** One path: a maintenance sweep (WP77) deletes keep-around+ references that no open review uses. The changeset constraint makes it safe+ against a concurrent creation. A leaked reference (crash between the steps,+ closed review) only keeps objects alive until the next sweep.+* **What this removes.**+ * R2: `CheckpointInput.Roots`, `checkReviewRoots`,+ `ErrCheckpointRootsChanged`.+ * The checkpoint build's review step: `ListReviews`, the head `H`, and the+ missing-review-commit path with its metric. The closure is "refs at N" and+ EnsureFresh needs only `>= N`.+ * R1's reachability check, its checkpoint-sequence condition (`rootsAt`,+ `ErrReviewRootsStale`), and `gitops.IsReachable`'s only caller.+ The roots rule becomes the general reference rule. A checkpoint can no+ longer be blocked by a review: a review whose commits are already lost+ simply has no reference.+* **What it adds.**+ * One reference transaction per review creation and per head move. Each+ produces one changeset, plus a pack only when the commit was not yet in+ the storage core.+ * The two database checks.+ * The sweep.+ * A backfill in maintenance for reviews written before this amendment or+ imported from files, which writes their references when the objects are+ present (a review whose objects are missing is logged and counted, and+ blocks nothing).+* **UX.** `CreateReview` on a commit that only the serving node has works again:+ the reference write uploads it. The STORAGE.md sentence added in WP74 ("push+ it first") is replaced.+* **Modes.** Off and shadow mode use the file review store. The service writes+ the same references there too: the same local protection against+ housekeeping, and in shadow mode the voter records them. The database+ invariant exists only in authoritative mode, where the database decides.+ This is one code path for all modes; in off mode it fixes the existing+ "verdict commit pruned after a force push" limitation for review heads.+* **Costs to measure (WP79).** The latency of a review creation (one extra+ reference transaction, comparable to a one-reference land). The number of+ changesets per review (1 per creation or head move, plus 1 per sweep batch).++**Option (b), rejected: local-only hidden keep-alive references.** References+present on each serving node but excluded from the database, from verify and+from the voter.+* It does not address liveness in the storage core. R1 and R2 stay, and so does+ everything this amendment removes.+* Every database-side reader of local references would need an exclusion: the+ voter's hook input, `ImportRepository`'s `ReadRefs` (it records every local+ reference), verify, the syncer's checks of local values, and the stale-cache+ checks.+* `--not --all` in the voter would count them as reachable. A later changeset's+ pack could then omit objects that only such a local reference protects:+ a new class of "present locally, not live in the storage core" objects.+* A node that hydrates later, or that has not served the repository since a+ review was created, has no such reference and does not learn about the+ review until a request arrives. Its housekeeping can prune in between,+ which is the same stall. A per-node rebuild of the references from the+ database on every hydration and catch-up would be needed, and it stays+ racy.++**Recommendation: (a).** It removes R2 and the checkpoint's review step+entirely, and replaces R1 by a reference-existence check. It uses only existing+mechanisms (voter, CAS, hidden refs, sweep) and fixes the housekeeping finding+on every node. Implementation would be a WP74b before WP76, replacing the+WP74 R1/R2 code (doc and tests included).+### Data model (new migration, both dialects)Migration `20261018000000_graft_storage_gc`, appended to `sqliteMigrations` and