Git designed for AI agents.
Review m2: bound the open reviews of a repository
Federico Elia5ab6ab37c09b
diff --git a/doc/graft/PLAN7.md b/doc/graft/PLAN7.mdindex 1daaefd..37e976b 100644--- a/doc/graft/PLAN7.md+++ b/doc/graft/PLAN7.md@@ -432,6 +432,22 @@ guarded nor swept).missing from the repository is logged; the checkpoint cannot hold it.Test: `TestStorageCheckpointBackfillsReviewRoots` (e2e; a mutant withoutstep 0 fails it on both dialects).+ * **Bound (integrated review m2).** The references are written in every+ mode, so an unbounded number of open reviews would be an unbounded number+ of references and kept objects, with the storage core off too.+ `[graft.review] max_open_reviews` (default 1000) bounds the open reviews of+ a repository. `CreateReview` checks it before it writes any reference+ (`ResourceExhausted`, nothing written, no reference transaction); the store+ checks it again atomically with the record (`store.Store` under its file+ lock, `InsertReview` under the repository row lock with+ `ReviewInsert.MaxOpen`), so concurrent creates never pass it. A create that+ passed the first check and lost the second has written its references:+ the service releases them as a close does (sweep of those commits unless+ another open review uses them; the backstop otherwise). Imports are not+ bounded. Tests: `TestCreateIsBoundedByTheOpenReviews` (both stores, twelve+ concurrent creates from two handles), `TestCreateReviewIsBoundedByMaxOpenReviews`,+ `TestCreateReviewPastTheLimitReleasesItsRoots`; mutants without either+ check fail them.* A sweep may skip references younger than `gc_grace` to make thecreation/sweep race rare. That is an efficiency measure only; correctnesscomes from the invariant and the bounded retry.diff --git a/doc/graft/REVIEW.md b/doc/graft/REVIEW.mdindex 510965e..0bda6ad 100644--- a/doc/graft/REVIEW.md+++ b/doc/graft/REVIEW.md@@ -255,6 +255,28 @@ approvals = 1That is, code and configuration changes need one approval by someone otherthan the author; comment-only, docs, tests and generated changes need none.+## Configuration: `[graft.review] max_open_reviews`++```toml+[graft.review]+max_open_reviews = 1000 # default when unset or 0; negative is rejected+```++Every open review keeps its head and base commits by hidden references+(`refs/keep-around/review/<commit>`, see [STORAGE.md](STORAGE.md)), in every+mode, storage core on or off. `max_open_reviews` bounds the open reviews of one+repository, and so those references and the objects they keep. `CreateReview`+refuses a review beyond it with `ResourceExhausted` ("the repository has N+open reviews, the limit is N; close or land an open review first, or raise+[graft.review] max_open_reviews") and writes nothing. The bound holds against+concurrent creates, on one node and, in authoritative mode, across nodes: the+store counts the open reviews and records the review under one lock (the file+lock, or the repository row in the database). Landed and closed reviews do not+count; closing one makes room. Reviews brought in by `graft storage import`+are not bounded: they exist already. Counting reads every review of the+repository in the files store (it has no index of the open ones), as listing+does.+## StorageReviews live in the repository's git directory, under `graft/reviews/`:diff --git a/doc/graft/STORAGE.md b/doc/graft/STORAGE.mdindex 558a8a7..86f6b5c 100644--- a/doc/graft/STORAGE.md+++ b/doc/graft/STORAGE.md@@ -100,7 +100,9 @@ maintenance). Where they differ, PLAN7.md is the later decision.references reach), in checkpoints and through garbage collection. A review ofa commit that only the serving node has works: the reference write uploads it.Closing a review deletes the references that no other open review uses; a- closed or landed review keeps nothing. Clients never see these references over+ closed or landed review keeps nothing. The open reviews of a repository are+ bounded by `[graft.review] max_open_reviews` (default 1000, see+ [REVIEW.md](REVIEW.md)). Clients never see these references overthe git protocols and cannot push or delete them. In authoritative mode thedatabase enforces the rule: a review write requires the references, and nowrite (`DeleteRefs`, `WriteRef`, `UpdateReferences`, a push) can delete or@@ -133,7 +135,7 @@ repository's object hash, and every pack against the SHA-256 in its key.| Mode | Source of truth | What the storage core does ||---|---|---|-| `off` (default, also empty) | local git | Nothing. No database or bucket is opened. |+| `off` (default, also empty) | local git | No database or bucket is opened. What milestone 3 adds outside the storage core still applies: open reviews keep their commits by `refs/keep-around/review/<commit>` references (bounded by `[graft.review] max_open_reviews`, see [REVIEW.md](REVIEW.md)), and the daily housekeeping backfills and sweeps them. || `shadow` | local git | Every reference transaction of an imported repository is also committed to the database, with its pack. The database never rejects a write: a compare-and-swap mismatch is recorded as a forced changeset and counted as a divergence. Any storage core error, timeout or panic is logged and counted, and the git write proceeds. Reads, the file changelog and the file review store are unchanged. Works with or without Praefect. || `authoritative` | database | A reference transaction commits only if its database compare-and-swap succeeds. Every request first catches the local repository up. The changelog and the reviews are served from the database. Requires `[graft] standalone = true`. |diff --git a/internal/gitaly/config/config_graft.go b/internal/gitaly/config/config_graft.goindex 5dc93e3..db91fc0 100644--- a/internal/gitaly/config/config_graft.go+++ b/internal/gitaly/config/config_graft.go@@ -107,6 +107,24 @@ type GraftReview struct {// that applies to at least one file has enough current approvals. When// Rules is empty, DefaultGraftReviewRules apply.Rules []GraftReviewRule `json:"rules" toml:"rule,omitempty"`+ // MaxOpenReviews bounds the open reviews of one repository: CreateReview+ // refuses a review beyond it with ResourceExhausted. Every open review+ // keeps its head and base commits by references, in every mode, so the+ // bound also bounds those references. Zero means+ // DefaultGraftMaxOpenReviews.+ MaxOpenReviews int `json:"max_open_reviews" toml:"max_open_reviews,omitempty"`+}++// DefaultGraftMaxOpenReviews is the bound on the open reviews of a repository+// when max_open_reviews is not set.+const DefaultGraftMaxOpenReviews = 1000++// MaxOpenReviewsOrDefault returns MaxOpenReviews or DefaultGraftMaxOpenReviews.+func (r GraftReview) MaxOpenReviewsOrDefault() int {+ if r.MaxOpenReviews > 0 {+ return r.MaxOpenReviews+ }+ return DefaultGraftMaxOpenReviews}// GraftReviewRule is one approval rule.@@ -148,6 +166,9 @@ func (r GraftReview) RulesOrDefault() []GraftReviewRule {}func (r GraftReview) validate() error {+ if r.MaxOpenReviews < 0 {+ return fmt.Errorf("graft review: max_open_reviews must not be negative (0 means the default, %d)", DefaultGraftMaxOpenReviews)+ }for i, rule := range r.Rules {if rule.Name == "" {return fmt.Errorf("graft review rule %d: name is required", i+1)diff --git a/internal/gitaly/config/config_graft_test.go b/internal/gitaly/config/config_graft_test.goindex 18dd9d3..df14003 100644--- a/internal/gitaly/config/config_graft_test.go+++ b/internal/gitaly/config/config_graft_test.go@@ -237,3 +237,17 @@ func TestGraft_ObjectServiceDefaults(t *testing.T) {require.Equal(t, 5, custom.MaxObjectsOrDefault())require.Equal(t, 64, custom.ChunkSizeOrDefault())}++func TestGraft_MaxOpenReviews(t *testing.T) {+ t.Parallel()++ require.Equal(t, DefaultGraftMaxOpenReviews, GraftReview{}.MaxOpenReviewsOrDefault())+ require.Equal(t, 7, GraftReview{MaxOpenReviews: 7}.MaxOpenReviewsOrDefault())++ cfg := graftTestCfg(t)+ cfg.Graft.Standalone = true+ cfg.Graft.Review.MaxOpenReviews = -1+ for _, err := range []error{cfg.Graft.Validate(), cfg.Validate(), cfg.ValidateV2()} {+ require.ErrorContains(t, err, "max_open_reviews must not be negative")+ }+}diff --git a/internal/gitaly/service/review/review.go b/internal/gitaly/service/review/review.goindex 65e99ec..88381ca 100644--- a/internal/gitaly/service/review/review.go+++ b/internal/gitaly/service/review/review.go@@ -80,6 +80,18 @@ func (s *server) CreateReview(ctx context.Context, req *gitalypb.CreateReviewReqreturn nil, structerr.NewInternal("%w", err)}+ // A repository at its limit refuses here, before the keep-around+ // references are written. The store refuses again, atomically, a create+ // that a concurrent one put past the limit.+ maxOpen := s.deps.GetCfg().Graft.Review.MaxOpenReviewsOrDefault()+ open, err := st.List("", false, 0)+ if err != nil {+ return nil, toStatus(err)+ }+ if len(open) >= maxOpen {+ return nil, toStatus(fmt.Errorf("%w: the repository has %d open reviews, the limit is %d", store.ErrTooManyOpenReviews, len(open), maxOpen))+ }+var stored *gitalypb.Reviewerr = s.writeRoots(ctx, repo, st, func(st store.ReviewStore, protect rootsProtect) error {if err := protect("", head, base); err != nil {@@ -99,6 +111,10 @@ func (s *server) CreateReview(ctx context.Context, req *gitalypb.CreateReviewReq})return err})+ if errors.Is(err, store.ErrTooManyOpenReviews) {+ // The references were written for a review that was not recorded.+ s.releaseRoots(ctx, repo, st, reviewCommits(head, base))+ }if err != nil {return nil, toStatus(err)}@@ -524,6 +540,9 @@ func toStatus(err error) error {if errors.Is(err, store.ErrNotFound) {return structerr.NewNotFound("review not found")}+ if errors.Is(err, store.ErrTooManyOpenReviews) {+ return structerr.NewResourceExhausted("%w; close or land an open review first, or raise [graft.review] max_open_reviews", err)+ }if _, ok := status.FromError(err); ok {return err}diff --git a/internal/gitaly/service/review/review_authoritative_test.go b/internal/gitaly/service/review/review_authoritative_test.goindex c345525..e7adc8b 100644--- a/internal/gitaly/service/review/review_authoritative_test.go+++ b/internal/gitaly/service/review/review_authoritative_test.go@@ -30,10 +30,18 @@ import (// server before it serves.func setupWithStorageCore(t *testing.T, mode corestore.Mode, configure ...func(*server)) (context.Context, config.Cfg, *corestore.Core, gitalypb.ReviewServiceClient) {t.Helper()+ return setupWithConfig(t, mode, func(*config.Cfg) {}, configure...)+}++// setupWithConfig is setupWithStorageCore with adjust applied to the+// configuration before the server starts.+func setupWithConfig(t *testing.T, mode corestore.Mode, adjust func(*config.Cfg), configure ...func(*server)) (context.Context, config.Cfg, *corestore.Core, gitalypb.ReviewServiceClient) {+ t.Helper()ctx := testhelper.Context(t)cfg := testcfg.Build(t)cfg.Graft.Standalone = true+ adjust(&cfg)// The service writes keep-around references through the// reference-transaction hook.testcfg.BuildGitalyHooks(t, cfg)diff --git a/internal/gitaly/service/review/roots.go b/internal/gitaly/service/review/roots.goindex a6edf5f..f58bd21 100644--- a/internal/gitaly/service/review/roots.go+++ b/internal/gitaly/service/review/roots.go@@ -75,14 +75,15 @@ func (s *server) writeRoots(ctx context.Context, repo *localrepo.Repo, st store.}// releaseRoots deletes the keep-around references of the commits of a review-// that was just closed, unless another open review uses them. The close is-// already recorded, so a failure is not the request's: it is logged and counted+// that was just closed, or that was refused after they were written, unless+// another open review uses them. The close (or the refusal) is the request's+// outcome already, so a failure is not the request's: it is logged and counted// (graft_review_roots_sweep_failures_total{trigger="close"}), and the reference// only keeps its objects alive until the housekeeping backstop deletes it// (reviewroots.Sweeper.Housekeep, which runs in every mode).func (s *server) releaseRoots(ctx context.Context, repo *localrepo.Repo, st store.ReviewStore, commits []git.ObjectID) {if _, err := reviewroots.DefaultSweeper.Sweep(ctx, repo, st, commits, reviewroots.TriggerClose); err != nil {s.deps.GetLogger().WithError(err).WithField("commits", commits).WarnContext(ctx,- "review: the keep-around references of a closed review were not deleted; the housekeeping backstop deletes them")+ "review: the keep-around references of a closed or refused review were not deleted; the housekeeping backstop deletes them")}}diff --git a/internal/gitaly/service/review/roots_test.go b/internal/gitaly/service/review/roots_test.goindex 95fbf04..b09a754 100644--- a/internal/gitaly/service/review/roots_test.go+++ b/internal/gitaly/service/review/roots_test.go@@ -306,3 +306,95 @@ func TestReviewRootsOutsideAuthoritativeMode(t *testing.T) {require.True(t, hasObject(t, cfg, repoPath, head))})}++// localKeepArounds returns the keep-around references of the local repository.+func localKeepArounds(t *testing.T, cfg config.Cfg, repoPath string) []string {+ t.Helper()+ out := strings.TrimSpace(string(gittest.Exec(t, cfg, "-C", repoPath, "for-each-ref", "--format=%(refname)", corestore.KeepAroundPrefix)))+ if out == "" {+ return nil+ }+ return strings.Split(out, "\n")+}++// TestCreateReviewIsBoundedByMaxOpenReviews: a repository with+// max_open_reviews open reviews refuses another with ResourceExhausted before+// it writes any reference, in every store; closing one makes room+// (integrated review m2).+func TestCreateReviewIsBoundedByMaxOpenReviews(t *testing.T) {+ for _, mode := range []corestore.Mode{corestore.ModeShadow, corestore.ModeAuthoritative} {+ t.Run(string(mode), func(t *testing.T) {+ ctx, cfg, core, client := setupWithConfig(t, mode, func(cfg *config.Cfg) { cfg.Graft.Review.MaxOpenReviews = 2 })+ repo, repoPath := createLocalRepository(t, ctx, cfg)+ base := gittest.WriteCommit(t, cfg, repoPath, gittest.WithMessage("base"), gittest.WithBranch("main"))+ heads := make([]git.ObjectID, 3)+ for i := range heads {+ heads[i] = gittest.WriteCommit(t, cfg, repoPath, gittest.WithMessage(fmt.Sprint("head ", i)), gittest.WithParents(base), gittest.WithBranch(fmt.Sprint("feature-", i)))+ }+ tracked := importRepository(t, cfg, core, repo, repoPath, coretest.ImportOptions{})++ create := func(head git.ObjectID) (*gitalypb.CreateReviewResponse, error) {+ return client.CreateReview(ctx, &gitalypb.CreateReviewRequest{Repository: repo, TargetBranch: []byte("main"), HeadCommitId: head.String()})+ }+ first, err := create(heads[0])+ require.NoError(t, err)+ _, err = create(heads[1])+ require.NoError(t, err)++ headSeq := func() uint64 {+ record, err := core.Meta.GetRepository(ctx, corestore.KeyOf(repo))+ require.NoError(t, err)+ return record.HeadSeq+ }+ before := headSeq()+ _, err = create(heads[2])+ require.Equal(t, codes.ResourceExhausted, status.Code(err), "%v", err)+ require.Equal(t, before, headSeq(), "the refusal writes no reference transaction")+ require.ErrorContains(t, err, "the repository has 2 open reviews, the limit is 2")+ require.ErrorContains(t, err, "max_open_reviews")+ require.Len(t, listStoredReviews(t, ctx, client, repo, true), 2, "nothing is recorded")+ require.Equal(t, keepAroundOf(heads[0], heads[1], base), keepArounds(t, ctx, core, tracked.ID), "no reference is written")+ require.NotContains(t, localKeepArounds(t, cfg, repoPath), corestore.KeepAroundRef(heads[2].String()))++ _, err = client.CloseReview(ctx, &gitalypb.CloseReviewRequest{Repository: repo, Id: first.GetReview().GetId()})+ require.NoError(t, err)+ _, err = create(heads[2])+ require.NoError(t, err, "closing a review makes room")+ require.Equal(t, keepAroundOf(heads[1], heads[2], base), keepArounds(t, ctx, core, tracked.ID))+ })+ }+}++// TestCreateReviewPastTheLimitReleasesItsRoots: a create that passes the+// service's check, and that a concurrent create then puts past the limit, is+// refused by the store, atomically, and its keep-around references, written+// already, are released.+func TestCreateReviewPastTheLimitReleasesItsRoots(t *testing.T) {+ var (+ core *corestore.Core+ tracked corestore.Repository+ other git.ObjectID+ )+ ctx, cfg, core, client := setupWithConfig(t, corestore.ModeAuthoritative,+ func(cfg *config.Cfg) { cfg.Graft.Review.MaxOpenReviews = 1 },+ func(s *server) {+ s.rootsProtected = func(ctx context.Context, _ *localrepo.Repo, _ store.ReviewStore) {+ // The concurrent create: recorded between the check and the write.+ _, err := core.Meta.InsertReview(ctx, tracked.ID, func(id string) (*gitalypb.Review, error) {+ return &gitalypb.Review{Id: id, TargetBranch: []byte("main"), HeadCommitId: other.String(), BaseCommitId: other.String()}, nil+ }, corestore.ReviewInsert{})+ require.NoError(t, err)+ }+ })+ repo, repoPath := createLocalRepository(t, ctx, cfg)+ base := gittest.WriteCommit(t, cfg, repoPath, gittest.WithMessage("base"), gittest.WithBranch("main"))+ head := gittest.WriteCommit(t, cfg, repoPath, gittest.WithMessage("head"), gittest.WithParents(base), gittest.WithBranch("feature"))+ other = gittest.WriteCommit(t, cfg, repoPath, gittest.WithMessage("other"), gittest.WithParents(base), gittest.WithBranch("other"))+ tracked = importRepository(t, cfg, core, repo, repoPath, coretest.ImportOptions{})++ _, err := client.CreateReview(ctx, &gitalypb.CreateReviewRequest{Repository: repo, TargetBranch: []byte("main"), HeadCommitId: head.String()})+ require.Equal(t, codes.ResourceExhausted, status.Code(err), "%v", err)+ require.Len(t, listStoredReviews(t, ctx, client, repo, true), 1, "only the concurrent review is recorded")+ require.Empty(t, keepArounds(t, ctx, core, tracked.ID), "the references of the refused review are released")+ require.Empty(t, localKeepArounds(t, cfg, repoPath))+}diff --git a/internal/gitaly/service/review/server.go b/internal/gitaly/service/review/server.goindex 13cae1a..6024e7b 100644--- a/internal/gitaly/service/review/server.go+++ b/internal/gitaly/service/review/server.go@@ -52,7 +52,7 @@ func newServer(deps *service.Dependencies) *server {// openRepo returns the repository and its review store. In authoritative mode// the reviews live in the storage core's database; otherwise they are the files-// of the repository.+// of the repository. The store's Create is bounded by max_open_reviews.func (s *server) openRepo(ctx context.Context, repoProto *gitalypb.Repository) (*localrepo.Repo, store.ReviewStore, error) {repo := s.deps.GetRepositoryFactory().Build(repoProto)@@ -61,9 +61,10 @@ func (s *server) openRepo(ctx context.Context, repoProto *gitalypb.Repository) (return nil, nil, structerr.NewInternal("repository path: %w", err)}+ maxOpen := s.deps.GetCfg().Graft.Review.MaxOpenReviewsOrDefault()core := s.deps.GetGraftStorage()if !core.Authoritative() {- return repo, store.Open(path), nil+ return repo, store.Open(path).WithMaxOpen(maxOpen), nil}tracked, err := core.Meta.GetRepository(ctx, corestore.KeyOf(repoProto))@@ -74,7 +75,7 @@ func (s *server) openRepo(ctx context.Context, repoProto *gitalypb.Repository) (return nil, nil, structerr.NewUnavailable("storage core: %w", err)}- return repo, dbreview.New(ctx, core.Meta, tracked.ID), nil+ return repo, dbreview.New(ctx, core.Meta, tracked.ID).(*dbreview.Store).WithMaxOpen(maxOpen), nil}// withRequirements sets the requirements and approval of r from the configureddiff --git a/internal/graft/corestore/dbreview/dbreview.go b/internal/graft/corestore/dbreview/dbreview.goindex bd3786c..8acf29c 100644--- a/internal/graft/corestore/dbreview/dbreview.go+++ b/internal/graft/corestore/dbreview/dbreview.go@@ -40,6 +40,9 @@ type Store struct {// requireRoots makes Create and Update require the keep-around references of// the review's commits (doc/graft/PLAN7.md, REVIEW ROOTS RULE).requireRoots bool+ // maxOpen, when positive, bounds the open reviews of the repository that+ // Create may reach (WithMaxOpen).+ maxOpen int}var _ store.ReviewStore = (*Store)(nil)@@ -60,6 +63,17 @@ func (s *Store) WithRequiredRoots() store.ReviewStore {return &required}+// WithMaxOpen returns a store whose Create refuses an open review with+// store.ErrTooManyOpenReviews, and writes nothing, when the repository has+// limit open reviews already. The count and the insert are one transaction+// under the repository row lock, so concurrent creates on any node cannot+// pass the limit. A limit of zero or less means no limit.+func (s *Store) WithMaxOpen(limit int) *Store {+ bounded := *s+ bounded.maxOpen = limit+ return &bounded+}+// Create stores a new review. It assigns the id, created_at and updated_at.// requirements and approved are not stored. The returned review is a copy.func (s *Store) Create(r *gitalypb.Review) (*gitalypb.Review, error) {@@ -72,7 +86,7 @@ func (s *Store) Create(r *gitalypb.Review) (*gitalypb.Review, error) {review.Requirements = nilreview.Approved = falsereturn review, nil- }, s.requireRoots)+ }, corestore.ReviewInsert{RequireRoots: s.requireRoots, MaxOpen: s.maxOpen})if err != nil {return nil, mapErr(err)}@@ -167,8 +181,11 @@ func (s *Store) backoff(attempt int) error {// mapErr translates the storage core errors into the errors of the review// store.func mapErr(err error) error {- if errors.Is(err, corestore.ErrReviewNotFound) {+ switch {+ case errors.Is(err, corestore.ErrReviewNotFound):return store.ErrNotFound+ case errors.Is(err, corestore.ErrTooManyOpenReviews):+ return fmt.Errorf("%w: %w", store.ErrTooManyOpenReviews, err)}return err}diff --git a/internal/graft/corestore/dbreview/dbreview_test.go b/internal/graft/corestore/dbreview/dbreview_test.goindex c42bb9e..7c577d6 100644--- a/internal/graft/corestore/dbreview/dbreview_test.go+++ b/internal/graft/corestore/dbreview/dbreview_test.go@@ -3,6 +3,7 @@ package dbreview_testimport ("context""errors"+ "fmt""sync""testing""time"@@ -155,6 +156,70 @@ func TestUpdate(t *testing.T) {})}+// withMaxOpen returns rs bounded to limit open reviews.+func withMaxOpen(t *testing.T, rs store.ReviewStore, limit int) store.ReviewStore {+ switch s := rs.(type) {+ case *store.Store:+ return s.WithMaxOpen(limit)+ case *dbreview.Store:+ return s.WithMaxOpen(limit)+ }+ require.FailNow(t, "unknown review store", "%T", rs)+ return nil+}++// TestCreateIsBoundedByTheOpenReviews: concurrent creates on two handles (two+// nodes) never pass the limit of open reviews; reviews that are not open do+// not count and are not refused, and closing one makes room for one.+func TestCreateIsBoundedByTheOpenReviews(t *testing.T) {+ eachBackend(t, func(t *testing.T, rs store.ReviewStore, other store.ReviewStore) {+ const limit, creates = 3, 12+ handles := []store.ReviewStore{withMaxOpen(t, rs, limit), withMaxOpen(t, other, limit)}++ landed, err := handles[0].Create(&gitalypb.Review{TargetBranch: []byte("main"), State: gitalypb.Review_STATE_LANDED})+ require.NoError(t, err, "a review that is not open is not bounded")++ start := make(chan struct{})+ errs := make(chan error, creates)+ for i := 0; i < creates; i++ {+ go func() {+ <-start+ _, err := handles[i%2].Create(&gitalypb.Review{TargetBranch: []byte("main")})+ errs <- err+ }()+ }+ close(start)+ created := 0+ for i := 0; i < creates; i++ {+ err := <-errs+ if err == nil {+ created+++ continue+ }+ require.ErrorIs(t, err, store.ErrTooManyOpenReviews)+ }+ require.Equal(t, limit, created, "exactly the limit is created")++ open, err := rs.List("", false, 0)+ require.NoError(t, err)+ require.Len(t, open, limit)+ all, err := rs.List("", true, 0)+ require.NoError(t, err)+ require.Len(t, all, limit+1, "a refused create writes nothing")+ require.Contains(t, ids(all), landed.GetId())++ _, err = handles[1].Create(&gitalypb.Review{TargetBranch: []byte("main")})+ require.ErrorIs(t, err, store.ErrTooManyOpenReviews)+ _, err = rs.Update(open[0].GetId(), func(r *gitalypb.Review) error {+ r.State = gitalypb.Review_STATE_CLOSED+ return nil+ })+ require.NoError(t, err)+ _, err = handles[1].Create(&gitalypb.Review{TargetBranch: []byte("main")})+ require.NoError(t, err, "closing a review makes room")+ })+}+func TestCreateTimestampsAreNow(t *testing.T) {eachBackend(t, func(t *testing.T, rs store.ReviewStore, _ store.ReviewStore) {before := time.Now().Add(-time.Second)@@ -209,6 +274,8 @@ type conflictingMeta struct {updateErr error// requireRoots are the requireRoots arguments of the writes, in order.requireRoots []bool+ // maxOpen are the open-review limits of the inserts, in order.+ maxOpen []int}func (m *conflictingMeta) GetReview(context.Context, int64, string) (*gitalypb.Review, int64, error) {@@ -222,8 +289,9 @@ func (m *conflictingMeta) UpdateReview(_ context.Context, _ int64, _ string, _ ireturn m.updateErr}-func (m *conflictingMeta) InsertReview(_ context.Context, _ int64, build func(string) (*gitalypb.Review, error), requireRoots bool) (*gitalypb.Review, error) {- m.requireRoots = append(m.requireRoots, requireRoots)+func (m *conflictingMeta) InsertReview(_ context.Context, _ int64, build func(string) (*gitalypb.Review, error), opts corestore.ReviewInsert) (*gitalypb.Review, error) {+ m.requireRoots = append(m.requireRoots, opts.RequireRoots)+ m.maxOpen = append(m.maxOpen, opts.MaxOpen)return build("1")}@@ -255,6 +323,38 @@ func TestRequiredRootsReachTheDatabase(t *testing.T) {require.Equal(t, 1, meta.updates)}+// TestMaxOpenReachesTheDatabase: the open-review limit of WithMaxOpen goes to+// the insert, also through WithRequiredRoots, and the database's refusal is+// the store's ErrTooManyOpenReviews.+func TestMaxOpenReachesTheDatabase(t *testing.T) {+ ctx := testhelper.Context(t)+ meta := &conflictingMeta{}+ plain := dbreview.New(ctx, meta, 1).(*dbreview.Store)++ _, err := plain.Create(&gitalypb.Review{})+ require.NoError(t, err)+ _, err = plain.WithMaxOpen(5).Create(&gitalypb.Review{})+ require.NoError(t, err)+ _, err = plain.WithMaxOpen(7).WithRequiredRoots().Create(&gitalypb.Review{})+ require.NoError(t, err)+ require.Equal(t, []int{0, 5, 7}, meta.maxOpen)+ require.Equal(t, []bool{false, false, true}, meta.requireRoots)++ refusing := &refusingMeta{}+ _, err = dbreview.New(ctx, refusing, 1).(*dbreview.Store).WithMaxOpen(1).Create(&gitalypb.Review{})+ require.ErrorIs(t, err, store.ErrTooManyOpenReviews)+ require.ErrorContains(t, err, "the limit is 1")+}++// refusingMeta refuses every insert as beyond the open-review limit.+type refusingMeta struct {+ corestore.MetaStore+}++func (refusingMeta) InsertReview(_ context.Context, _ int64, _ func(string) (*gitalypb.Review, error), opts corestore.ReviewInsert) (*gitalypb.Review, error) {+ return nil, fmt.Errorf("%w: the repository has %d open reviews, the limit is %d", corestore.ErrTooManyOpenReviews, opts.MaxOpen, opts.MaxOpen)+}+func TestUpdateGivesUpAfterTenConflicts(t *testing.T) {ctx := testhelper.Context(t)meta := &conflictingMeta{updateErr: corestore.ErrReviewVersionConflict}diff --git a/internal/graft/corestore/metastore.go b/internal/graft/corestore/metastore.goindex 691a4a7..4f7dbdf 100644--- a/internal/graft/corestore/metastore.go+++ b/internal/graft/corestore/metastore.go@@ -89,7 +89,25 @@ type CheckpointInput struct {PackDeadline time.Time}+// ReviewInsert are the conditions under which MetaStore.InsertReview writes a+// review; the zero value has none.+type ReviewInsert struct {+ // RequireRoots requires the keep-around reference (KeepAroundRef) of the+ // review's head and base commits to exist with that value, else+ // ErrReviewRootsMissing and nothing is written (doc/graft/PLAN7.md, REVIEW+ // ROOTS RULE). Together with CommitChangeset's refusal to delete such a+ // reference, every open review written so has its commits referenced.+ RequireRoots bool+ // MaxOpen, when positive, refuses an open review with+ // ErrTooManyOpenReviews when the repository has MaxOpen open reviews+ // already, and nothing is written.+ MaxOpen int+}+var (+ // ErrTooManyOpenReviews is returned by InsertReview for an open review+ // beyond ReviewInsert.MaxOpen.+ ErrTooManyOpenReviews = errors.New("storage core: too many open reviews")// ErrReviewNotFound is returned for an unknown review ID.ErrReviewNotFound = errors.New("storage core: review not found")// ErrReviewVersionConflict is returned by UpdateReview when the review@@ -156,13 +174,9 @@ type MetaStore interface {// with it and stores the returned review (version 1), atomically. build runs// inside the write transaction and must not call the MetaStore.//- // With requireRoots the transaction locks the repository row first and- // requires the keep-around reference (KeepAroundRef) of the review's head and- // base commits to exist with that value, else ErrReviewRootsMissing and- // nothing is written (doc/graft/PLAN7.md, REVIEW ROOTS RULE). Together with- // CommitChangeset's refusal to delete such a reference, every open review- // written so has its commits referenced.- InsertReview(ctx context.Context, repoID int64, build func(id string) (*gitalypb.Review, error), requireRoots bool) (*gitalypb.Review, error)+ // The transaction locks the repository row first, so the conditions of+ // opts hold against every concurrent insert (ReviewInsert).+ InsertReview(ctx context.Context, repoID int64, build func(id string) (*gitalypb.Review, error), opts ReviewInsert) (*gitalypb.Review, error)// GetReview returns the review and its version, ErrReviewNotFound if unknown.GetReview(ctx context.Context, repoID int64, id string) (*gitalypb.Review, int64, error)// ListReviews returns the reviews of the repository newest firstdiff --git a/internal/graft/corestore/sqlstore/gc_test.go b/internal/graft/corestore/sqlstore/gc_test.goindex b5dafec..bcd1563 100644--- a/internal/graft/corestore/sqlstore/gc_test.go+++ b/internal/graft/corestore/sqlstore/gc_test.go@@ -514,7 +514,7 @@ func TestLockOrderConcurrentWriters(t *testing.T) {_, err := a.InsertReview(ctx, repo.ID, func(string) (*gitalypb.Review, error) {return &gitalypb.Review{TargetBranch: []byte("main"), HeadCommitId: string(oid(500))}, nil- }, false)+ }, corestore.ReviewInsert{})require.NoError(t, err)const (@@ -599,7 +599,7 @@ func TestLockOrderConcurrentWriters(t *testing.T) {if err := writeRoots(s, head, func() error {_, err := s.InsertReview(ctx, repo.ID, func(string) (*gitalypb.Review, error) {return &gitalypb.Review{TargetBranch: []byte("main"), HeadCommitId: head}, nil- }, true)+ }, corestore.ReviewInsert{RequireRoots: true})return err}); err != nil {recordUnexpected(fmt.Errorf("insert review: %w", err))diff --git a/internal/graft/corestore/sqlstore/history_test.go b/internal/graft/corestore/sqlstore/history_test.goindex 2fb57f5..b6799dd 100644--- a/internal/graft/corestore/sqlstore/history_test.go+++ b/internal/graft/corestore/sqlstore/history_test.go@@ -85,7 +85,7 @@ func TestImportHistory(t *testing.T) {inserted, err := s.InsertReview(ctx, repo.ID, func(id string) (*gitalypb.Review, error) {return newReview("next", "main", gitalypb.Review_STATE_OPEN_UNSPECIFIED), nil- }, false)+ }, corestore.ReviewInsert{})require.NoError(t, err)require.Equal(t, "8", inserted.GetId(), "the next review ID follows the largest imported one")diff --git a/internal/graft/corestore/sqlstore/review.go b/internal/graft/corestore/sqlstore/review.goindex d31842c..421242a 100644--- a/internal/graft/corestore/sqlstore/review.go+++ b/internal/graft/corestore/sqlstore/review.go@@ -41,7 +41,7 @@ func unmarshalReview(body string) (*gitalypb.Review, error) {}// InsertReview implements corestore.MetaStore.-func (s *Store) InsertReview(ctx context.Context, repoID int64, build func(id string) (*gitalypb.Review, error), requireRoots bool) (*gitalypb.Review, error) {+func (s *Store) InsertReview(ctx context.Context, repoID int64, build func(id string) (*gitalypb.Review, error), opts corestore.ReviewInsert) (*gitalypb.Review, error) {var stored *gitalypb.Reviewerr := s.withTx(ctx, nil, func(tx *sql.Tx) error {@@ -63,7 +63,19 @@ func (s *Store) InsertReview(ctx context.Context, repoID int64, build func(id st}r := prepareReview(built, fmt.Sprintf("%d", id))- if requireRoots {+ if opts.MaxOpen > 0 && r.GetState() == gitalypb.Review_STATE_OPEN_UNSPECIFIED {+ // Under the repository row lock, which every insert takes: the count+ // cannot change before the commit.+ var open int+ if err := tx.QueryRowContext(ctx, s.bind("SELECT COUNT(*) FROM graft_reviews WHERE repo_id = ? AND closed = ?"),+ repoID, false).Scan(&open); err != nil {+ return fmt.Errorf("count open reviews: %w", err)+ }+ if open >= opts.MaxOpen {+ return fmt.Errorf("%w: the repository has %d open reviews, the limit is %d", corestore.ErrTooManyOpenReviews, open, opts.MaxOpen)+ }+ }+ if opts.RequireRoots {if err := s.requireReviewRoots(ctx, tx, repoID, r); err != nil {return err}diff --git a/internal/graft/corestore/sqlstore/review_test.go b/internal/graft/corestore/sqlstore/review_test.goindex 07c1744..5013db3 100644--- a/internal/graft/corestore/sqlstore/review_test.go+++ b/internal/graft/corestore/sqlstore/review_test.go@@ -19,7 +19,7 @@ func insertReview(t *testing.T, ctx context.Context, s *sqlstore.Store, repoID iinserted, err := s.InsertReview(ctx, repoID, func(id string) (*gitalypb.Review, error) {return r, nil- }, false)+ }, corestore.ReviewInsert{})require.NoError(t, err)return inserted}diff --git a/internal/graft/corestore/sqlstore/reviewroots_test.go b/internal/graft/corestore/sqlstore/reviewroots_test.goindex 9d44bc3..e47a64b 100644--- a/internal/graft/corestore/sqlstore/reviewroots_test.go+++ b/internal/graft/corestore/sqlstore/reviewroots_test.go@@ -38,22 +38,22 @@ func TestReviewWritesRequireRootReferences(t *testing.T) {require.Len(t, reviews, n)}- _, err := s.InsertReview(ctx, repo.ID, build(head, base), true)+ _, err := s.InsertReview(ctx, repo.ID, build(head, base), corestore.ReviewInsert{RequireRoots: true})require.ErrorIs(t, err, corestore.ErrReviewRootsMissing, "no reference at all")keepAround(t, ctx, s, repo.ID, head)- _, err = s.InsertReview(ctx, repo.ID, build(head, base), true)+ _, err = s.InsertReview(ctx, repo.ID, build(head, base), corestore.ReviewInsert{RequireRoots: true})require.ErrorIs(t, err, corestore.ErrReviewRootsMissing, "the base has none")requireReviews(0)// A keep-around reference with another value does not protect the commit._, err = commit(ctx, s, repo.ID, create(corestore.KeepAroundRef(string(base)), oid(12)))require.NoError(t, err)- _, err = s.InsertReview(ctx, repo.ID, build(head, base), true)+ _, err = s.InsertReview(ctx, repo.ID, build(head, base), corestore.ReviewInsert{RequireRoots: true})require.ErrorIs(t, err, corestore.ErrReviewRootsMissing)_, err = commit(ctx, s, repo.ID, update(corestore.KeepAroundRef(string(base)), oid(12), base))require.NoError(t, err)- created, err := s.InsertReview(ctx, repo.ID, build(head, base), true)+ created, err := s.InsertReview(ctx, repo.ID, build(head, base), corestore.ReviewInsert{RequireRoots: true})require.NoError(t, err)requireReviews(1)@@ -72,14 +72,14 @@ func TestReviewWritesRequireRootReferences(t *testing.T) {// Writes that do not require the roots are not conditioned: imports and// comments.- _, err = s.InsertReview(ctx, repo.ID, build(oid(14), ""), false)+ _, err = s.InsertReview(ctx, repo.ID, build(oid(14), ""), corestore.ReviewInsert{})require.NoError(t, err)r.Title = []byte("a comment")require.NoError(t, s.UpdateReview(ctx, repo.ID, created.GetId(), version+1, r, false))// A required write of a removed repository finds no repository row.gone, _ := headRepo(t, ctx, s, "root-refs-gone", 1)- _, err = s.InsertReview(ctx, gone.ID, build(oid(15), ""), false)+ _, err = s.InsertReview(ctx, gone.ID, build(oid(15), ""), corestore.ReviewInsert{})require.NoError(t, err)removed, err := s.DeleteRepository(ctx, gone.Key)require.NoError(t, err)@@ -101,11 +101,11 @@ func TestOpenReviewRootsCannotBeReleased(t *testing.T) {}open, err := s.InsertReview(ctx, repo.ID, func(string) (*gitalypb.Review, error) {return &gitalypb.Review{TargetBranch: []byte("main"), HeadCommitId: string(head), BaseCommitId: string(base)}, nil- }, true)+ }, corestore.ReviewInsert{RequireRoots: true})require.NoError(t, err)_, err = s.InsertReview(ctx, repo.ID, func(string) (*gitalypb.Review, error) {return &gitalypb.Review{TargetBranch: []byte("main"), HeadCommitId: string(closed), State: gitalypb.Review_STATE_CLOSED}, nil- }, true)+ }, corestore.ReviewInsert{RequireRoots: true})require.NoError(t, err)before, seq, err := s.Refs(ctx, repo.ID)diff --git a/internal/graft/corestore/sqlstore/robustness_test.go b/internal/graft/corestore/sqlstore/robustness_test.goindex 9a110c5..6d16a2a 100644--- a/internal/graft/corestore/sqlstore/robustness_test.go+++ b/internal/graft/corestore/sqlstore/robustness_test.go@@ -79,7 +79,7 @@ func TestWritesHonourDeadlineWhileLocked(t *testing.T) {call: func(ctx context.Context) error {_, err := s.InsertReview(ctx, repo.ID, func(string) (*gitalypb.Review, error) {return newReview("t", "refs/heads/main", gitalypb.Review_STATE_OPEN_UNSPECIFIED), nil- }, false)+ }, corestore.ReviewInsert{})return err},},diff --git a/internal/graft/corestore/sqlstore/store_test.go b/internal/graft/corestore/sqlstore/store_test.goindex 8531aab..3ce4869 100644--- a/internal/graft/corestore/sqlstore/store_test.go+++ b/internal/graft/corestore/sqlstore/store_test.go@@ -454,7 +454,7 @@ func TestRegisterWithHistory(t *testing.T) {inserted, err := s.InsertReview(ctx, repo.ID, func(id string) (*gitalypb.Review, error) {require.Equal(t, "4", id)return newReview("new", "main", gitalypb.Review_STATE_OPEN_UNSPECIFIED), nil- }, false)+ }, corestore.ReviewInsert{})require.NoError(t, err)require.Equal(t, "4", inserted.GetId())@@ -491,7 +491,7 @@ func TestDeleteCascadesAndReRegisterGetsNewID(t *testing.T) {require.NoError(t, err)_, err = s.InsertReview(ctx, first.ID, func(id string) (*gitalypb.Review, error) {return newReview("doomed", "main", gitalypb.Review_STATE_OPEN_UNSPECIFIED), nil- }, false)+ }, corestore.ReviewInsert{})require.NoError(t, err)deleted, err := s.DeleteRepository(ctx, key)diff --git a/internal/graft/e2e/storage_gc_test.go b/internal/graft/e2e/storage_gc_test.goindex 641bf85..9f9ad89 100644--- a/internal/graft/e2e/storage_gc_test.go+++ b/internal/graft/e2e/storage_gc_test.go@@ -162,7 +162,7 @@ func TestStorageCheckpointBackfillsReviewRoots(t *testing.T) {repoID := a.record(t, rel).ID_, err = a.core.Meta.InsertReview(a.ctx, repoID, func(id string) (*gitalypb.Review, error) {return &gitalypb.Review{Id: id, TargetBranch: []byte("main"), HeadCommitId: head, BaseCommitId: base, Title: []byte("before the rule")}, nil- }, false)+ }, corestore.ReviewInsert{})require.NoError(t, err)cp, err := a.operator(t, rel).StorageCheckpoint(a.ctx)diff --git a/internal/graft/e2e/storage_localrefs_test.go b/internal/graft/e2e/storage_localrefs_test.goindex 30399f0..f512b61 100644--- a/internal/graft/e2e/storage_localrefs_test.go+++ b/internal/graft/e2e/storage_localrefs_test.go@@ -538,7 +538,7 @@ func TestStorageHousekeepingBackfillsReviewRoots(t *testing.T) {t.Cleanup(func() { _ = meta.Close() })_, err = meta.InsertReview(n.ctx, state.RepositoryID, func(id string) (*gitalypb.Review, error) {return &gitalypb.Review{Id: id, TargetBranch: []byte("main"), HeadCommitId: head, BaseCommitId: base, Title: []byte("before the rule")}, nil- }, false)+ }, corestore.ReviewInsert{})require.NoError(t, err)roots := []string{corestore.KeepAroundRef(head), corestore.KeepAroundRef(base)}diff --git a/internal/graft/review/store/store.go b/internal/graft/review/store/store.goindex d469e85..3d0b0a0 100644--- a/internal/graft/review/store/store.go+++ b/internal/graft/review/store/store.go@@ -25,8 +25,13 @@ import ("google.golang.org/protobuf/types/known/timestamppb")-// ErrNotFound is returned when a review does not exist.-var ErrNotFound = errors.New("review not found")+var (+ // ErrNotFound is returned when a review does not exist.+ ErrNotFound = errors.New("review not found")+ // ErrTooManyOpenReviews is returned by Create, and nothing is written, for+ // an open review beyond the limit of the store (WithMaxOpen).+ ErrTooManyOpenReviews = errors.New("too many open reviews")+)const (reviewsDirRel = "graft/reviews"@@ -39,6 +44,8 @@ const (// Open does not touch the disk.type Store struct {repoPath string+ // maxOpen, when positive, bounds the open reviews that Create may reach.+ maxOpen int}// Open returns a Store for the repository at repoPath (its git directory).@@ -46,6 +53,17 @@ func Open(repoPath string) *Store {return &Store{repoPath: repoPath}}+// WithMaxOpen returns a store whose Create refuses an open review with+// ErrTooManyOpenReviews, and writes nothing, when the repository has limit+// open reviews already. Create counts them under the lock that serialises+// every mutation, so concurrent creates cannot pass the limit. A limit of zero+// or less means no limit.+func (s *Store) WithMaxOpen(limit int) *Store {+ bounded := *s+ bounded.maxOpen = limit+ return &bounded+}+// dir returns the reviews directory.func (s *Store) dir() string {return filepath.Join(s.repoPath, filepath.FromSlash(reviewsDirRel))@@ -60,6 +78,18 @@ func (s *Store) Create(r *gitalypb.Review) (*gitalypb.Review, error) {}defer unlock()+ if s.maxOpen > 0 && r.GetState() == gitalypb.Review_STATE_OPEN_UNSPECIFIED {+ // List reads every review file, closed ones included: the files keep+ // no index of the open ones.+ open, err := s.List("", false, 0)+ if err != nil {+ return nil, fmt.Errorf("count open reviews: %w", err)+ }+ if len(open) >= s.maxOpen {+ return nil, fmt.Errorf("%w: the repository has %d open reviews, the limit is %d", ErrTooManyOpenReviews, len(open), s.maxOpen)+ }+ }+dir := s.dir()nextID, err := readNextID(dir)if err != nil {