Skip to content

Git designed for AI agents.

sdk: rebase lands are not retried by default

Federico Eliaf13ad58953be
diff --git a/doc/graft/BENCH.md b/doc/graft/BENCH.md
index 6b2fd34..db080a9 100644
--- a/doc/graft/BENCH.md
+++ b/doc/graft/BENCH.md
@@ -245,7 +245,7 @@ Markdown report show the same values.
| `resolutions` | Conflicting paths the server resolved. |
| `attempts` | Compare-and-swap attempts the server used, summed over every change, landed or failed. A land that gave up because the branch kept moving counts the attempts it made (the number the server reports in its "giving up after N attempts" error). Other failed lands count none, because the server does not report their attempts. Above `landed` means the branch moved during lands. |
| `attempts_per_land` | `attempts` divided by `landed`. The attempts wasted on failed changes raise it, so it is the server work per landed change. |
-| `unavailable_retries` | Calls that the server answered with `Unavailable` and the SDK retried (`LandOptions.RetryUnavailable`), summed over the landed changes. Each is a whole land request again, the conflict resolver included. A failed land reports none. |
+| `unavailable_retries` | Calls that the server answered with `Unavailable` and the SDK retried (`LandOptions.RetryUnavailable`), summed over the landed changes. Each is a whole land request again, the conflict resolver included. A failed land reports none. The benchmark lands with rebase, which the SDK does not retry by default, so it is 0 until rebase lands are retried. |
| `max_attempts` | The largest number of attempts of any change, landed or failed. |
| `attempts_histogram` | Landed changes by their number of attempts (`{"1": 95, "2": 5}`: 95 changes landed at the first attempt). Landed changes that report no attempts are left out. |
| `failed_attempts_histogram` | Failed changes that report attempts, by their number of attempts. A land that gave up after `max_attempts` rounds shows up here. |
diff --git a/doc/graft/LAND-BATCHING.md b/doc/graft/LAND-BATCHING.md
index 5e32f6a..66fadcc 100644
--- a/doc/graft/LAND-BATCHING.md
+++ b/doc/graft/LAND-BATCHING.md
@@ -478,16 +478,20 @@ member) and removed it; see [PLAN8.md](PLAN8.md).
branch still race on the reference, as before.
- The lane lives in memory. Requests queued when a Gitaly process stops are not
landed. When the lane registry is closed, queued requests fail with
- `Unavailable`, and callers retry. The SDK does this for `Land` by default
- (`LandOptions.RetryUnavailable`: up to 5 retries of `Unavailable` only, with
+ `Unavailable`, and callers retry. The SDK does this for `Land` with
+ `LandOptions.RetryUnavailable`: up to 5 retries of `Unavailable` only, with
an exponential backoff from 100 ms to 2 s with full jitter, stopped by the
- context), and reports the retries in `LandResult.UnavailableRetries`. A client
- that does not use the SDK must do the same: retry `Unavailable`, and only
- `Unavailable`, with a bounded backoff. A land is safe to retry after a lost
- response: with a merge, a fast-forward or a resolved land the change is
- reported as already landed; with a rebase the change is replayed onto the tip,
- comes out empty, and the branch is left unchanged (a change that a later
- commit reverted in between is replayed again, as a cherry-pick would be).
+ context, reported in `LandResult.UnavailableRetries`. A client that does not
+ use the SDK must do the same: retry `Unavailable`, and only `Unavailable`, with
+ a bounded backoff, and only for the lands below.
+ - Merge and fast-forward lands are retried by default. The response of a call
+ can be lost after the change landed; the change itself is then on the
+ branch, and the retry reports it as already landed.
+ - Rebase lands, with or without conflict resolution, are not retried by
+ default (`RebaseLands` turns it on). The branch holds a rebased copy of the
+ change, and the server cannot yet tell that this change landed: a retry
+ replays it, and a change that a later commit reverted in between (for
+ example graft-verify reverting the culprit) would land again.
- A land that arrives alone waits up to `window` before its batch runs. Set
`window = "0s"` to remove the wait, or `disabled = true` to remove batching.
- Requests that do not qualify (see *When a land is batched*) run unbatched and
diff --git a/sdk/land.go b/sdk/land.go
index 97b55f4..e265bd9 100644
--- a/sdk/land.go
+++ b/sdk/land.go
@@ -58,7 +58,8 @@ type LandOptions struct {
// merge conflict as without this option.
ResolveConflicts bool
// RetryUnavailable is how Land retries a call that the server answers
- // with Unavailable (see UnavailableRetry). It is on by default.
+ // with Unavailable (see UnavailableRetry). It is on by default for
+ // LandMerge and LandFastForwardOnly, and off for LandRebase.
RetryUnavailable UnavailableRetry
}
@@ -70,22 +71,25 @@ type LandOptions struct {
// backoff with full jitter: before retry n it waits a random time between zero
// and min(Cap, Base x 2^(n-1)). A context that ends stops the waiting.
//
-// A land is safe to retry when the response of an earlier call was lost after
-// the server landed the change: the server lands by compare-and-swap, and the
-// retry finds the change on the branch. With LandMerge, LandFastForwardOnly or
-// ResolveConflicts, Commit is then reachable from the tip, and the retry is
-// reported as AlreadyLanded. With LandRebase the branch holds a rebased copy,
-// not Commit itself: the retry replays the change onto the tip, every commit
-// of it comes out empty and is dropped, and the branch is left unchanged; the
-// result then has LandedCommitID equal to the tip, Rewritten false and
-// AlreadyLanded false. A change that a later commit reverted in between is
-// replayed again, as git's cherry-pick would. A retried land with
-// ResolveConflicts may run the conflict resolver again, a model included, for
-// the same conflict; LandResult reports the retries (UnavailableRetries), so
-// that this cost is visible.
+// The response of a call can be lost after the server landed the change. With
+// LandMerge and LandFastForwardOnly, Commit is then reachable from the tip, and
+// the retry is reported as AlreadyLanded: these lands are retried by default.
+// With LandRebase, ResolveConflicts included (a land without conflicts is a
+// plain rebase), the branch holds a rebased copy, not Commit itself, and the
+// server cannot tell yet that the change landed: the retry replays the change
+// onto the tip. Every commit of it comes out empty and is dropped, unless a
+// later commit reverted the change in between, for example graft-verify
+// reverting the culprit: the retry would then land the change again. Rebase
+// lands are therefore not retried unless RebaseLands is set by a caller that
+// accepts this. A retried land with ResolveConflicts may run the conflict
+// resolver again, a model included, for the same conflict; LandResult reports
+// the retries (UnavailableRetries), so that this cost is visible.
type UnavailableRetry struct {
// Disabled turns the retries off: Land returns Unavailable at once.
Disabled bool
+ // RebaseLands also retries LandRebase lands, which a retry after a lost
+ // response may land again when the change was reverted in between.
+ RebaseLands bool
// MaxRetries bounds the retries after the first call. Zero means 5.
MaxRetries int
// Base is the backoff of the first retry. Zero means 100ms.
@@ -118,9 +122,10 @@ func (r UnavailableRetry) backoff(n int) time.Duration {
return time.Duration(rand.Int64N(int64(limit) + 1))
}
-func (r UnavailableRetry) maxRetries() int {
+// maxRetries returns the retries of a land with the given strategy.
+func (r UnavailableRetry) maxRetries(strategy LandStrategy) int {
switch {
- case r.Disabled:
+ case r.Disabled, strategy == LandRebase && !r.RebaseLands:
return 0
case r.MaxRetries > 0:
return r.MaxRetries
@@ -215,7 +220,7 @@ func (r *Repo) Land(ctx context.Context, opts LandOptions) (*LandResult, error)
)
for {
resp, err = r.client.operations.UserLand(ctx, req)
- if status.Code(err) != codes.Unavailable || retries >= opts.RetryUnavailable.maxRetries() {
+ if status.Code(err) != codes.Unavailable || retries >= opts.RetryUnavailable.maxRetries(opts.Strategy) {
break
}
retries++
diff --git a/sdk/land_retry_test.go b/sdk/land_retry_test.go
index d164e08..857164c 100644
--- a/sdk/land_retry_test.go
+++ b/sdk/land_retry_test.go
@@ -16,6 +16,44 @@ import (
// fastRetry is the retry of Unavailable with a backoff short enough for tests.
var fastRetry = sdk.UnavailableRetry{Base: time.Millisecond, Cap: 2 * time.Millisecond}
+// mergeLand is a land with LandMerge, which RetryUnavailable retries by default.
+func mergeLand() sdk.LandOptions {
+ opts := validLand()
+ opts.Strategy = sdk.LandMerge
+ return opts
+}
+
+func TestRepo_Land_RebaseLandsAreNotRetriedByDefault(t *testing.T) {
+ t.Parallel()
+
+ for _, tc := range []struct {
+ desc string
+ resolve bool
+ retry sdk.UnavailableRetry
+ expectedRequests int
+ }{
+ {desc: "rebase", retry: fastRetry, expectedRequests: 1},
+ {desc: "rebase with conflict resolution", resolve: true, retry: fastRetry, expectedRequests: 1},
+ {desc: "rebase, opted in", retry: sdk.UnavailableRetry{RebaseLands: true, Base: time.Millisecond, Cap: 2 * time.Millisecond}, expectedRequests: 6},
+ } {
+ t.Run(tc.desc, func(t *testing.T) {
+ t.Parallel()
+
+ fakes := newFakeServices()
+ fakes.operations.landErr = status.Error(codes.Unavailable, "still syncing")
+ repo := dialFake(t, fakes.register).Repo("repo")
+
+ opts := validLand()
+ opts.Strategy = sdk.LandRebase
+ opts.ResolveConflicts = tc.resolve
+ opts.RetryUnavailable = tc.retry
+ _, err := repo.Land(t.Context(), opts)
+ require.Equal(t, codes.Unavailable, sdk.Code(err))
+ require.Len(t, fakes.operations.landRequests, tc.expectedRequests)
+ })
+ }
+}
+
func TestRepo_Land_RetriesUnavailable(t *testing.T) {
t.Parallel()
@@ -27,7 +65,7 @@ func TestRepo_Land_RetriesUnavailable(t *testing.T) {
fakes.operations.landResponse = &gitalypb.UserLandResponse{LandedCommitId: "landed-sha", Attempts: 1}
repo := dialFake(t, fakes.register).Repo("repo")
- opts := validLand()
+ opts := mergeLand()
opts.RetryUnavailable = fastRetry
result, err := repo.Land(t.Context(), opts)
require.NoError(t, err)
@@ -56,7 +94,7 @@ func TestRepo_Land_UnavailableRetriesAreBounded(t *testing.T) {
fakes.operations.landErr = status.Error(codes.Unavailable, "still syncing")
repo := dialFake(t, fakes.register).Repo("repo")
- opts := validLand()
+ opts := mergeLand()
opts.RetryUnavailable = tc.retry
_, err := repo.Land(t.Context(), opts)
require.Equal(t, codes.Unavailable, sdk.Code(err), "the last Unavailable is returned")
@@ -76,7 +114,7 @@ func TestRepo_Land_RetriesOnlyUnavailable(t *testing.T) {
fakes.operations.landErr = status.Error(code, "not to be retried")
repo := dialFake(t, fakes.register).Repo("repo")
- opts := validLand()
+ opts := mergeLand()
opts.RetryUnavailable = fastRetry
_, err := repo.Land(t.Context(), opts)
require.Equal(t, code, sdk.Code(err))
@@ -95,7 +133,7 @@ func TestRepo_Land_ContextStopsTheRetries(t *testing.T) {
// The backoff is longer than the context, which ends during the first wait.
ctx, cancel := context.WithTimeout(t.Context(), 50*time.Millisecond)
defer cancel()
- opts := validLand()
+ opts := mergeLand()
opts.RetryUnavailable = sdk.UnavailableRetry{Base: time.Hour, Cap: time.Hour}
// Each wait is drawn from [0, 1h]: the context ends during one of them unless five draws in
// a row fall below 50ms.
@@ -120,7 +158,7 @@ func TestRepo_Land_RetriedResolveLandReportsItsOutcome(t *testing.T) {
}
repo := dialFake(t, fakes.register).Repo("repo")
- opts := validLand()
+ opts := mergeLand()
opts.ResolveConflicts = true
opts.RetryUnavailable = fastRetry
result, err := repo.Land(t.Context(), opts)