Git designed for AI agents.
graft: review of m6 (9586fd2) and of the stdin writers (59a9800)
Federico Elia9cb7ea21f981
diff --git a/doc/graft/AGENT-SECURITY.md b/doc/graft/AGENT-SECURITY.mdindex 39151f0..7a14b24 100644--- a/doc/graft/AGENT-SECURITY.md+++ b/doc/graft/AGENT-SECURITY.md@@ -104,12 +104,18 @@ the hiding:2. **The sinks.** Every writer of revisions into a newline-delimited git stdin(`git rev-list --stdin`) refuses a revision with a line feed, a carriagereturn or a NUL, whatever its caller validated, through- `git.ValidateStdinRevision`. Today there are two: `gitpipe.Revlist` (ListCommits,- ListBlobs, ListLFSPointers, GetTagSignatures and the other revision walks) and- `ObjectsSize`. A test (`internal/git/stdin_writers_test.go`) lists every- `--stdin` writer by file and the way it is safe (a request's revisions- through the helper; NUL-delimited `-z`; input the server builds), and fails- on a writer it does not know, so a new one is classified when it is added.+ `git.ValidateStdinRevision`. Today there are three: `gitpipe.Revlist` (ListCommits,+ ListBlobs, ListLFSPointers, GetTagSignatures and the other revision walks),+ `ObjectsSize`, and `CreateBundleFromRefList`, whose patterns go to+ `git bundle create --stdin`. A test (`internal/git/stdin_writers_test.go`)+ parses the code and lists every file that gives a git command a flag that+ makes it read its stdin (`--stdin`, `--revs`, `--batch` and the flags that+ begin with them) by how it is safe: a request's revisions through the helper+ (the test finds the call in the syntax tree, not a word in a comment);+ NUL-delimited (`-z`, `-Z`); or input the server builds from object IDs (a+ request's revisions are resolved to object IDs first), packs, content or+ paths of its own. It fails on a writer it does not know, so a new one is+ classified when it is added.3. **The filter reads lines as git does.** The `--exclude-hidden` flag is putbefore a line that git reads as `--all` or `--glob=...`: without its linefeed and the carriage return before it, and cut at the first NUL.diff --git a/doc/graft/CONFIGURATION.md b/doc/graft/CONFIGURATION.mdindex db97313..5369d98 100644--- a/doc/graft/CONFIGURATION.md+++ b/doc/graft/CONFIGURATION.md@@ -193,7 +193,7 @@ cause.| `command` | array of strings | empty | Program and arguments. It reads one request on stdin and writes the answer on stdout. || `url` | string | empty | `http://` or `https://` endpoint that receives the request as a JSON POST. Exactly one of `command` and `url` must be set when `enabled` is true. || `token_file` | string | empty | For `url`: file with the bearer token, read at startup. |-| `timeout` | string | `"120s"` | Bound on one resolver call (Go duration). |+| `timeout` | string | `"120s"` | Bound on one resolver call (Go duration), and on the whole land that the server resolves for `resolve_conflicts`: its rounds (the resolutions, the waits for a resolve slot, and the passes through the branch's lane) end at this deadline. A round that the lane hands back after it fails the land with the merge conflict, "files kept changing", naming the files. || `max_files` | int | `20` | Conflicted files per land. A land with more is not sent to the resolver. || `max_file_bytes` | int | `1048576` (1 MiB) | Bound on every side of every file sent and on every resolved file accepted. At most 12566528 (about 12 MiB): one file, its three sides and its resolution, must fit one check request. || `proposal_ttl` | string | `"168h"` | How long the proposals of a failed land are kept under `refs/graft/proposals/` (Go duration, by the server's clock); at most 20 are kept per branch. Lands with a resolution and the housekeeping pass prune them. |diff --git a/doc/graft/PLAN10.md b/doc/graft/PLAN10.mdindex 5671d46..4036aa3 100644--- a/doc/graft/PLAN10.md+++ b/doc/graft/PLAN10.md@@ -1098,6 +1098,15 @@ The resolution is bound to the files instead (the PM's correction):runs the classification again on every file it reuses: local git work, bounded like thepackage's, never the resolver. A file whose classification at the lane's tip differs from itsverdict's counts as changed.+ * A request carries the files that still need a verdict, and which files travel together depends+ on the round: after a tip moved, only the files whose sides changed are sent, alone. A resolver+ can therefore not rely on seeing the other conflicting files of the land, and a verdict must+ depend on its file only (its path, its sides and modes, the change's message, the+ configuration). A third-party resolver (the command or the HTTP service) must+ respect that contract; one that reads the tip, another file or the history of the repository+ through its own means would have its answer reused where it no longer holds. The risk is+ bounded: an answer of a model never lands unverified (the final ruling, 14 and 15.9), and the+ operator's rules are the resolver's own.* The request no longer carries the tip, its merge base or its commit message: the fields`tip`, `merge_base` and `messages.tip` are always empty, kept in the protocol forcompatibility (a test asserts it). A verdict reused at another tip is then correct bydiff --git a/doc/graft/RESOLVE.md b/doc/graft/RESOLVE.mdindex 6e08fe4..027af61 100644--- a/doc/graft/RESOLVE.md+++ b/doc/graft/RESOLVE.md@@ -252,6 +252,12 @@ Response (resolver to server):- `action` is `write` (with `mode` and `content`) or `delete`.- `resolver` names the resolver. It goes into the `Graft-Resolved-By`trailer, so it must be 1 to 128 bytes of printable, single-line UTF-8.+ The server answers with one `resolver` for a land it resolved in rounds (the+ files of a round that were resolved earlier keep their resolver): the+ distinct names of the rounds, joined by commas in the order they were made+ (`graft-resolve/trivial,graft-resolve/llm`), and, when the list would exceed+ 128 bytes, the names that fit followed by `,+N` for the N that do not. It is+ always one line, and one `Graft-Resolved-By` trailer carries it.- `explanation` is free text. It is logged with the resolution (cut to4 KiB) and never interpreted.diff --git a/internal/git/stdin_writers_test.go b/internal/git/stdin_writers_test.goindex 91b2317..b74a2d0 100755--- a/internal/git/stdin_writers_test.go+++ b/internal/git/stdin_writers_test.go@@ -1,10 +1,14 @@package gitimport (+ "go/ast"+ "go/parser"+ "go/token""io/fs""os""path/filepath""runtime"+ "strconv""strings""testing"@@ -12,84 +16,184 @@ import ()// Where a git command reads revisions from its stdin one per line, a line break in a revision is-// another revision. This test lists every non-test file that gives a git command the flag --stdin,-// and how each is safe, so that a new writer is classified when it is added.+// another revision. This test lists every non-test Go file that gives a git command a flag that makes+// it read its stdin (--stdin and the flags that begin with it, --revs, --batch and the flags that begin+// with it), and how each is safe, so that a new writer is classified when it is added. The files are+// parsed, so a flag in a comment is not one, and a guard is a call of ValidateStdinRevision, not a word.++// stdinSafety is how the writer of a git stdin is safe.+type stdinSafety string+const (- // stdinRevisionsOfRequests: revisions that callers send, written one per line; the file calls- // ValidateStdinRevision for each.- stdinRevisionsOfRequests = "revisions of requests, through ValidateStdinRevision"- // stdinNULDelimited: the input is NUL-delimited (-z), so a line break is data.- stdinNULDelimited = "NUL-delimited (-z)"- // stdinServerInput: the input is built by the server from object IDs, packs or content, never- // from a caller's revision or pattern.- stdinServerInput = "built by the server from object IDs, packs or content"+ // stdinRevisionsOfRequests: revisions or patterns that callers send, written one per line; the+ // guard file calls ValidateStdinRevision for each.+ stdinRevisionsOfRequests stdinSafety = "revisions of requests, through ValidateStdinRevision"+ // stdinNULDelimited: git reads the input NUL-delimited (-z, -Z), so a line break is data.+ stdinNULDelimited stdinSafety = "NUL-delimited (-z, -Z)"+ // stdinServerInput: the server builds the input, whatever line it writes: object IDs (those of a+ // request are resolved to object IDs first), packs, content, or paths of its own.+ stdinServerInput stdinSafety = "built by the server from object IDs, packs, content or paths of its own"// stdinNamesTheFlag: not a writer; the file only compares the flag.- stdinNamesTheFlag = "not a writer"+ stdinNamesTheFlag stdinSafety = "not a writer")-var stdinWriters = map[string]string{- "internal/git/gitpipe/revision.go": stdinRevisionsOfRequests,- "internal/gitaly/service/repository/objects_size.go": stdinRevisionsOfRequests,- "internal/gitaly/service/diff/find_changed_paths.go": stdinNULDelimited,- "internal/gitaly/service/operations/land_package.go": stdinNULDelimited,- "internal/git/localrepo/merge_tree_stream.go": stdinNULDelimited,- "internal/graft/corestore/syncer/syncer.go": stdinNULDelimited,- "internal/git/gitattributes/check_attr.go": stdinNULDelimited,- "internal/git/updateref/updateref.go": stdinNULDelimited,- "internal/git/localrepo/bundle.go": stdinServerInput,- "internal/git/localrepo/objects.go": stdinServerInput,- "internal/git/localrepo/tree.go": stdinServerInput,- "internal/git/localrepo/rebase.go": stdinServerInput,- "internal/git/gitcmd/blob.go": stdinServerInput,- "internal/graft/corestore/gitops/gitops.go": stdinServerInput,- "internal/gitaly/service/storagecore/verify_objects.go": stdinServerInput,- "internal/gitaly/storage/storagemgr/partition/transaction_manager.go": stdinServerInput,- "internal/git/gitcmd/agent_walk.go": stdinNamesTheFlag,+type stdinWriter struct {+ safety stdinSafety+ // guard is the file that calls ValidateStdinRevision, when it is not the writer's own: the file+ // that joins the patterns of a request is not the one that names the flag.+ guard string}-func TestStdinWriters_areAllClassified(t *testing.T) {- t.Parallel()+var stdinWriters = map[string]stdinWriter{+ "internal/git/gitpipe/revision.go": {safety: stdinRevisionsOfRequests},+ "internal/gitaly/service/repository/objects_size.go": {safety: stdinRevisionsOfRequests},+ "internal/git/localrepo/bundle.go": {safety: stdinRevisionsOfRequests, guard: "internal/gitaly/service/repository/create_bundle_from_ref_list.go"},+ "internal/git/updateref/updateref.go": {safety: stdinNULDelimited},+ "internal/graft/corestore/syncer/syncer.go": {safety: stdinNULDelimited},+ "internal/git/gitattributes/check_attr.go": {safety: stdinNULDelimited},+ "internal/gitaly/service/operations/land_package.go": {safety: stdinNULDelimited},+ "internal/git/catfile/object_reader.go": {safety: stdinNULDelimited},+ "internal/gitaly/service/diff/find_changed_paths.go": {safety: stdinServerInput},+ "internal/git/localrepo/merge_tree_stream.go": {safety: stdinServerInput},+ "internal/git/localrepo/objects.go": {safety: stdinServerInput},+ "internal/git/localrepo/tree.go": {safety: stdinServerInput},+ "internal/git/localrepo/rebase.go": {safety: stdinServerInput},+ "internal/git/localrepo/commit_writer.go": {safety: stdinServerInput},+ "internal/git/gitcmd/blob.go": {safety: stdinServerInput},+ "internal/git/gitpipe/catfile_info.go": {safety: stdinServerInput},+ "internal/graft/corestore/gitops/gitops.go": {safety: stdinServerInput},+ "internal/graft/corestore/voter/shadowpack.go": {safety: stdinServerInput},+ "internal/gitaly/service/storagecore/verify_objects.go": {safety: stdinServerInput},+ "internal/gitaly/service/storagecore/import_pack.go": {safety: stdinServerInput},+ "internal/gitaly/storage/storagemgr/partition/check_objects.go": {safety: stdinServerInput},+ "internal/gitaly/storage/storagemgr/partition/transaction_manager.go": {safety: stdinServerInput},+ "internal/git/gitcmd/agent_walk.go": {safety: stdinNamesTheFlag},+}++// stdinFlagPrefixes are the beginnings of the flags that make a git command read its stdin.+var stdinFlagPrefixes = []string{"--stdin", "--revs", "--batch"}++func moduleRoot(t *testing.T) string {+ t.Helper()_, self, _, ok := runtime.Caller(0)require.True(t, ok)- moduleRoot := filepath.Join(filepath.Dir(self), "..", "..")+ return filepath.Join(filepath.Dir(self), "..", "..")+}++// stdinFlagFiles returns the non-test Go files of the module that hold a string literal beginning+// like a flag that makes git read its stdin.+func stdinFlagFiles(t *testing.T, root string) map[string]bool {+ t.Helper()found := map[string]bool{}- require.NoError(t, filepath.WalkDir(moduleRoot, func(path string, d fs.DirEntry, err error) error {- if err != nil {- return err- }+ require.NoError(t, filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {+ require.NoError(t, err)+ rel, err := filepath.Rel(root, path)+ require.NoError(t, err)+ rel = filepath.ToSlash(rel)if d.IsDir() {- switch d.Name() {- case "_build", ".git", "node_modules", "testdata":+ switch {+ case d.Name() == "_build", d.Name() == ".git", d.Name() == "node_modules", d.Name() == "testdata":return filepath.SkipDir+ case rel == "internal/git/gittest", rel == "internal/testhelper":+ return filepath.SkipDir // helpers of tests+ case rel != ".":+ if _, err := os.Stat(filepath.Join(path, "go.mod")); err == nil {+ return filepath.SkipDir // another module+ }}return nil}if !strings.HasSuffix(path, ".go") || strings.HasSuffix(path, "_test.go") {return nil}- raw, err := os.ReadFile(path)- require.NoError(t, err)- if strings.Contains(string(raw), `"--stdin"`) {- rel, err := filepath.Rel(moduleRoot, path)- require.NoError(t, err)- found[filepath.ToSlash(rel)] = true-- if stdinWriters[filepath.ToSlash(rel)] == stdinRevisionsOfRequests {- require.Contains(t, string(raw), "ValidateStdinRevision", "%s writes revisions of requests to a git stdin without the guard", rel)+ file, err := parser.ParseFile(token.NewFileSet(), path, nil, 0)+ require.NoError(t, err, rel)+ ast.Inspect(file, func(n ast.Node) bool {+ lit, ok := n.(*ast.BasicLit)+ if !ok || lit.Kind != token.STRING {+ return true}- }+ value, err := strconv.Unquote(lit.Value)+ if err != nil {+ return true+ }+ for _, prefix := range stdinFlagPrefixes {+ if strings.HasPrefix(value, prefix) {+ found[rel] = true+ }+ }+ return true+ })return nil}))+ return found+}++// callsValidateStdinRevision reports whether the file calls ValidateStdinRevision: a call, as the+// syntax tree has it, and not a comment or a string that says so.+func callsValidateStdinRevision(t *testing.T, path string) bool {+ t.Helper()++ file, err := parser.ParseFile(token.NewFileSet(), path, nil, 0)+ require.NoError(t, err, path)+ called := false+ ast.Inspect(file, func(n ast.Node) bool {+ call, ok := n.(*ast.CallExpr)+ if !ok {+ return true+ }+ switch fun := call.Fun.(type) {+ case *ast.SelectorExpr:+ called = called || fun.Sel.Name == "ValidateStdinRevision"+ case *ast.Ident:+ called = called || fun.Name == "ValidateStdinRevision"+ }+ return true+ })+ return called+}++func TestStdinWriters_areAllClassified(t *testing.T) {+ t.Parallel()++ root := moduleRoot(t)+ found := stdinFlagFiles(t, root)for file := range found {require.Contains(t, stdinWriters, file,- "%s gives a git command --stdin: classify it in stdinWriters (are its revisions written one per line? then call ValidateStdinRevision for each)", file)+ "%s gives a git command a flag that makes it read its stdin: classify it in stdinWriters (are its revisions written one per line? then call ValidateStdinRevision for each)", file)+ }+ for file, writer := range stdinWriters {+ require.Contains(t, found, file, "%s is classified but no longer has such a flag: remove it from stdinWriters", file)+ if writer.safety != stdinRevisionsOfRequests {+ continue+ }+ guard := writer.guard+ if guard == "" {+ guard = file+ }+ require.True(t, callsValidateStdinRevision(t, filepath.Join(root, filepath.FromSlash(guard))),+ "%s writes revisions of requests to a git stdin, and %s does not call ValidateStdinRevision", file, guard)}- for file := range stdinWriters {- require.Contains(t, found, file, "%s is classified but no longer uses --stdin: remove it from stdinWriters", file)+}++// TestStdinWriters_theGuardIsACallNotAWord: a comment or a string that names the helper is no guard.+func TestStdinWriters_theGuardIsACallNotAWord(t *testing.T) {+ t.Parallel()++ dir := t.TempDir()+ write := func(name, body string) string {+ path := filepath.Join(dir, name)+ require.NoError(t, os.WriteFile(path, []byte(body), 0o600))+ return path}+ require.True(t, callsValidateStdinRevision(t, write("call.go", "package x\nfunc f() { _ = git.ValidateStdinRevision(\"a\") }\n")))+ require.True(t, callsValidateStdinRevision(t, write("bare.go", "package x\nfunc f() { _ = ValidateStdinRevision(\"a\") }\n")))+ require.False(t, callsValidateStdinRevision(t, write("comment.go", "package x\n// ValidateStdinRevision is called here (it is not)\nfunc f() {}\n")))+ require.False(t, callsValidateStdinRevision(t, write("string.go", "package x\nvar s = \"git.ValidateStdinRevision(x)\"\n")))+ require.False(t, callsValidateStdinRevision(t, write("reference.go", "package x\nvar f = git.ValidateStdinRevision\n")), "a reference is not a call")}func TestValidateStdinRevision(t *testing.T) {diff --git a/internal/gitaly/service/operations/land_batch_metrics.go b/internal/gitaly/service/operations/land_batch_metrics.goindex 03c30af..b67cf62 100644--- a/internal/gitaly/service/operations/land_batch_metrics.go+++ b/internal/gitaly/service/operations/land_batch_metrics.go@@ -1,6 +1,7 @@package operationsimport (+ "errors""time""github.com/feder-cr/git-ai/proto/go/gitalypb"@@ -43,6 +44,10 @@ const (landBatchOutcomeConflict = "conflict"// landBatchOutcomeDenied is a request that the access policy or the hooks denied.landBatchOutcomeDenied = "denied"+ // landBatchOutcomeNeedsResolution is a request the lane handed back because the files it found+ // conflicting have no verdict at its tip (PLAN10 15.8): not a failure, the land resolves them+ // out of the lane and enters the lane again, so a land that loses k rounds counts k of these.+ landBatchOutcomeNeedsResolution = "needs_resolution"// landBatchOutcomeError is a request that failed for any other reason.landBatchOutcomeError = "error")@@ -63,7 +68,7 @@ var (landBatchMembersTotal = promauto.NewCounterVec(prometheus.CounterOpts{Name: "graft_land_batch_members_total",- Help: "Land requests executed by batches, by outcome: landed, already_landed, conflict, denied or error.",+ Help: "Land requests executed by batches, by outcome: landed, already_landed, conflict, denied, needs_resolution or error.",}, []string{"outcome"}))@@ -105,6 +110,11 @@ func landBatchOutcomeLabel(outcome landBatchOutcome) string {// landBatchErrorLabel returns the outcome label of a request that failed with err.func landBatchErrorLabel(err error) string {+ var needs *landNeedsResolution+ if errors.As(err, &needs) {+ return landBatchOutcomeNeedsResolution+ }+st := status.Convert(err)if st.Code() == codes.PermissionDenied {return landBatchOutcomeDenieddiff --git a/internal/gitaly/service/operations/land_preresolve.go b/internal/gitaly/service/operations/land_preresolve.goindex aa4c752..bef5e67 100644--- a/internal/gitaly/service/operations/land_preresolve.go+++ b/internal/gitaly/service/operations/land_preresolve.go@@ -8,6 +8,7 @@ import ("github.com/feder-cr/git-ai/internal/git""github.com/feder-cr/git-ai/internal/graft/resolver"+ "github.com/feder-cr/git-ai/internal/structerr""github.com/feder-cr/git-ai/proto/go/gitalypb")@@ -116,6 +117,21 @@ func (s *Server) landPreResolve(ctx context.Context, in *gitalypb.UserLandRequescase err == nil, errors.Is(err, errLandNoConflicts):return nil}+ // Only a conflict that the resolution left unresolved is a merge conflict, and only the policy's+ // refusal is a refusal. Any other error (a git command that failed, a caller that gave up) means+ // the conflicts could not be listed or resolved, and goes out with its cause.+ var unresolved *landUnresolvedError+ var denied *landPolicyDeniedError+ if !errors.As(err, &unresolved) && !errors.As(err, &denied) {+ switch {+ case errors.Is(err, context.Canceled):+ return structerr.NewCanceled("resolving the conflicts of the land: %w", err)+ case errors.Is(err, context.DeadlineExceeded):+ return structerr.NewDeadlineExceeded("resolving the conflicts of the land: %w", err)+ }+ s.logger.WithError(err).WithField("branch", string(in.GetBranch())).Error("graft: the conflicts of the land could not be resolved")+ return structerr.NewInternal("resolving the conflicts of the land: %w", err)+ }files := l.files[:min(len(l.files), landMaxPackageFiles)]paths := make([][]byte, 0, len(files))for _, f := range files {diff --git a/internal/gitaly/service/operations/land_preresolve_test.go b/internal/gitaly/service/operations/land_preresolve_test.goindex 52928b8..d873f31 100755--- a/internal/gitaly/service/operations/land_preresolve_test.go+++ b/internal/gitaly/service/operations/land_preresolve_test.go@@ -302,9 +302,10 @@ func TestUserLand_resolvesOutOfTheLane(t *testing.T) {require.Eventually(t, func() bool { return countIn(t, filepath.Join(h.dir, "calls")) == 2 }, 30*time.Second, 10*time.Millisecond)third := landAsync(h.fx, h.c, "Land c")require.Eventually(t, func() bool { return countIn(t, filepath.Join(h.dir, "calls")) == 3 }, 30*time.Second, 10*time.Millisecond)- // The two requests submit to the lane right after their answers; the lane is held for as- // long as the test wants, so this wait only lets them queue.- time.Sleep(500 * time.Millisecond)+ // The lane is held: the two requests wait in it together. Wait until it says so, not for a+ // while.+ require.Eventually(t, func() bool { return h.fx.laneQueueDepth(t) == 2 }, 30*time.Second, 10*time.Millisecond,+ "b and c never queued in the lane")requireRunning(t, h.first, "the land of a")h.release()diff --git a/internal/gitaly/service/operations/land_resolve.go b/internal/gitaly/service/operations/land_resolve.goindex a9deac4..77b09aa 100644--- a/internal/gitaly/service/operations/land_resolve.go+++ b/internal/gitaly/service/operations/land_resolve.go@@ -174,14 +174,7 @@ func (s *Server) landResolveAt(// The client applies the timeout, the request policy and the acceptance rules of the// answer; the request waits for a resolve slot within the deadline of ctx.- var resp *resolver.Response- var err error- if slot, slotErr := s.conflictResolver.AcquireResolveSlot(ctx); slotErr != nil {- err = slotErr- } else {- resp, err = slot.Resolve(ctx, req)- slot.Release()- }+ resp, err := s.landResolveSlotted(ctx, req)resolveErr = errswitch {case err != nil:@@ -248,6 +241,20 @@ func (s *Server) landResolveAt(return l, combined, nil}+// landResolveSlotted sends the resolve request with a resolve slot. The slot is given back when the+// call returns, even on a panic: a lost slot would refuse every later resolution of the process.+func (s *Server) landResolveSlotted(ctx context.Context, req resolver.Request) (*resolver.Response, error) {+ slot, err := s.conflictResolver.AcquireResolveSlot(ctx)+ if err != nil {+ return nil, err+ }+ defer slot.Release()+ if s.landResolveHook != nil {+ s.landResolveHook()+ }+ return slot.Resolve(ctx, req)+}+// landCombinedAnswer is the answer of a land whose files are resolved in rounds: the files the// resolver answered now and the verdicts reused, under the names of the resolvers that made them.// It is nil when the resolver was asked and did not answer, or when there is nothing at all.@@ -335,7 +342,16 @@ func (s *Server) landApplyVerdicts(}}}- if len(missing) > 0 || len(l.files) == 0 {+ if len(l.files) == 0 {+ // A merge that is unclean without a conflicting file (a directory rename split) at this tip.+ // The pre-resolution refuses it with the same reason when it sees it at its own tip; here the+ // tip moved to it. No file can be resolved, so asking for files again would only repeat the+ // round: the land is refused, with the package that says why.+ s.logger.WithField("branch", string(in.GetBranch())).WithField("tip", tip.String()).+ Warn("graft: the merge at the lane's tip has conflicts without a conflicting file")+ return landResolution{}, l.notAttempted(landNoConflictingFiles, nil)+ }+ if len(missing) > 0 {return landResolution{}, &landNeedsResolution{tip: tip, commit: commitID, paths: missing}}diff --git a/internal/gitaly/service/operations/land_resolve_test.go b/internal/gitaly/service/operations/land_resolve_test.goindex 77cb60b..45ed684 100644--- a/internal/gitaly/service/operations/land_resolve_test.go+++ b/internal/gitaly/service/operations/land_resolve_test.go@@ -26,10 +26,12 @@ import ("github.com/feder-cr/git-ai/internal/gitaly/config""github.com/feder-cr/git-ai/internal/gitaly/service""github.com/feder-cr/git-ai/internal/graft/identity"+ "github.com/feder-cr/git-ai/internal/graft/landlane""github.com/feder-cr/git-ai/internal/graft/resolver""github.com/feder-cr/git-ai/internal/testhelper""github.com/feder-cr/git-ai/internal/testhelper/testcfg""github.com/feder-cr/git-ai/proto/go/gitalypb"+ "github.com/prometheus/client_golang/prometheus")// resolverScript writes a resolver command into a new temporary directory and returns the@@ -131,6 +133,27 @@ type resolveFixture struct {base git.ObjectIDtip git.ObjectIDcommit git.ObjectID+ // lanes are the land lanes of the service when the fixture runs a standalone server (the lands+ // then batch through them); nil otherwise.+ lanes *landlane.Shared+}++// laneQueueDepth returns the number of land requests queued in the lanes of the fixture.+func (fx resolveFixture) laneQueueDepth(t *testing.T) float64 {+ t.Helper()++ require.NotNil(t, fx.lanes, "the fixture has no lanes of its own")+ reg := prometheus.NewPedanticRegistry()+ require.NoError(t, reg.Register(fx.lanes))+ families, err := reg.Gather()+ require.NoError(t, err)+ for _, family := range families {+ if family.GetName() == "graft_land_lane_queue_depth" {+ return family.GetMetric()[0].GetGauge().GetValue()+ }+ }+ require.FailNow(t, "the lanes export no graft_land_lane_queue_depth")+ return 0}// newResolveFixture starts an operation service that resolves with rcfg and creates the@@ -173,7 +196,19 @@ func newResolveFixtureIn(t *testing.T, graft config.Graft, baseEntries, tipEntricfg := testcfg.Build(t)cfg.Graft = graft- ctx, cfg, client := setupOperationsServiceWithCfg(t, testhelper.Context(t), cfg)+ var lanes *landlane.Shared+ if graft.Standalone {+ laneCfg, err := landlane.ConfigFromGraft(graft.Land.Batch)+ require.NoError(t, err)+ lanes = landlane.NewShared(laneCfg)+ t.Cleanup(lanes.Close)+ }+ testcfg.BuildGitalySSH(t, cfg)+ testcfg.BuildGitalyHooks(t, cfg)+ cfg.SocketPath = runOperationServiceServerWithLanes(t, cfg, lanes)+ client, conn := newOperationClient(t, cfg.SocketPath)+ t.Cleanup(func() { conn.Close() })+ ctx := testhelper.MergeOutgoingMetadata(testhelper.Context(t), testcfg.GitalyServersMetadataFromCfg(t, cfg))// The resolution writes commits directly, outside an RPC, so the feature flags that// commit writing checks are set explicitly.ctx = featureflag.ContextWithFeatureFlag(ctx, featureflag.GPGSigning, true)@@ -207,6 +242,7 @@ func newResolveFixtureIn(t *testing.T, graft config.Graft, baseEntries, tipEntribase: base,tip: tip,commit: commit,+ lanes: lanes,}}diff --git a/internal/gitaly/service/operations/land_review_test.go b/internal/gitaly/service/operations/land_review_test.gonew file mode 100644index 0000000..2cff1d5--- /dev/null+++ b/internal/gitaly/service/operations/land_review_test.go@@ -0,0 +1,142 @@+package operations++import (+ "context"+ "os"+ "path/filepath"+ "testing"+ "time"++ "github.com/stretchr/testify/require"+ "google.golang.org/grpc/codes"+ "google.golang.org/grpc/status"++ "github.com/feder-cr/git-ai/internal/git/gittest"+)++// Tests of the review of m6 (9586fd2).++// TestResolveSlot_aPanickingResolverCallGivesItsSlotBack: a panic in the resolver call (recovered by the+// gRPC server) must not lose the resolve slot: a lost slot would refuse every later resolution.+func TestResolveSlot_aPanickingResolverCallGivesItsSlotBack(t *testing.T) {+ t.Parallel()++ dir, argv := resolverScript(t, "")+ writeAnswer(t, dir, validAnswer())+ rcfg := enabledResolver(argv)+ rcfg.ResolveConcurrency = 1+ fx := newResolveFixture(t, rcfg)+ s := fx.server(t, rcfg)+ s.landResolveHook = func() { panic("a resolution failed") }++ require.PanicsWithValue(t, "a resolution failed", func() {+ _, _, _ = s.landResolveAt(fx.ctx, fx.request("Land"), fx.repo, fx.tip, fx.commit, fx.committer(), landVerdicts{})+ })+ require.Equal(t, 0, resolverCalls(t, dir), "the hook ran before the call")++ ctx, cancel := context.WithTimeout(fx.ctx, time.Second)+ defer cancel()+ slot, err := s.conflictResolver.AcquireResolveSlot(ctx)+ require.NoError(t, err, "the slot of the panicking call was given back")+ slot.Release()+}++// TestLandBatchMetrics_aLostRoundIsNotAnError is not parallel: the metrics are process-wide, so the+// difference of their values is only this land when no other batch runs meanwhile. A land whose files+// change under it is handed back by the lane for each round it loses (PLAN10 15.8): that is a+// needs_resolution, not an error. Losing k rounds gives k of them and one landed.+func TestLandBatchMetrics_aLostRoundIsNotAnError(t *testing.T) {+ dir, argv := resolverScript(t, answerByCall)+ writeAnswer(t, dir, validAnswer())+ writeAnswerN(t, dir, 2, answerOf("a"))+ writeAnswerN(t, dir, 3, answerOf("a"))+ graft := batchingGraft()+ graft.Land.Resolver = enabledResolver(argv)+ fx := newResolveFixtureIn(t, graft,+ entries("a", "base\n", "b", "base\n", "c", "c\n"),+ entries("a", "ours\n", "b", "ours\n", "c", "c\n"),+ entries("a", "theirs\n", "b", "theirs\n", "c", "c\n", "d", "d\n"),+ )+ // While the first and the second resolutions run, a writer changes a: the land loses two rounds.+ w1 := gittest.WriteCommit(t, fx.cfg, fx.repoPath, gittest.WithParents(fx.tip),+ gittest.WithTreeEntries(entries("a", "ours-1\n", "b", "ours\n", "c", "c\n")...))+ w2 := gittest.WriteCommit(t, fx.cfg, fx.repoPath, gittest.WithParents(w1),+ gittest.WithTreeEntries(entries("a", "ours-2\n", "b", "ours\n", "c", "c\n")...))+ writeMoves(t, dir, fx, w1, w2)++ _, before := landBatchMetricCounts(t)+ resp, err := fx.client.UserLand(fx.ctx, fx.request("Land"))+ require.NoError(t, err)+ _, after := landBatchMetricCounts(t)++ require.Equal(t, 3, resolverCalls(t, dir), "two rounds were lost, the third landed")+ require.Equal(t, w2.String(), resp.GetOldTip())+ delta := func(outcome string) float64 { return after[outcome] - before[outcome] }+ require.Equal(t, 2.0, delta(landBatchOutcomeNeedsResolution), "one per lost round")+ require.Equal(t, 1.0, delta(landBatchOutcomeLanded))+ require.Equal(t, 0.0, delta(landBatchOutcomeError), "a lost round is not an error")+ require.Equal(t, 0.0, delta(landBatchOutcomeConflict))+}++// TestUserLand_aFailureThatIsNoConflictIsNotAMergeConflict: only a conflict that the resolution left+// unresolved is a merge conflict. When the conflicts cannot even be listed (here, the change's tree is+// missing from the repository), the land fails with the cause, as an internal error.+func TestUserLand_aFailureThatIsNoConflictIsNotAMergeConflict(t *testing.T) {+ t.Parallel()++ dir, argv := resolverScript(t, "")+ writeAnswer(t, dir, validAnswer())+ fx := newResolveFixture(t, enabledResolver(argv))+ tree := gittest.ResolveRevision(t, fx.cfg, fx.repoPath, fx.commit.String()+"^{tree}").String()+ require.NoError(t, os.Remove(filepath.Join(fx.repoPath, "objects", tree[:2], tree[2:])))++ _, err := fx.client.UserLand(fx.ctx, fx.request("Land"))+ require.Error(t, err)+ require.Equal(t, codes.Internal, status.Code(err), "%v", err)+ require.ErrorContains(t, err, "resolving the conflicts of the land")+ require.False(t, isLandMergeConflict(err), "no merge conflict was found: the error is the cause")+ require.Equal(t, 0, resolverCalls(t, dir))+ requireBranchAt(t, fx.cfg, fx.repoPath, "main", fx.tip)+}++// TestUserLand_aConflictWithoutAConflictingFileAtTheLanesTip: while the land is resolved, the branch moves+// to a tip where the merge is unclean without a conflicting file (a directory rename split). No file can+// be resolved there, so asking again for files would only repeat the round until the deadline and end with+// "files kept changing:" and no name. The land is refused at once, with the package that says why.+func TestUserLand_aConflictWithoutAConflictingFileAtTheLanesTip(t *testing.T) {+ t.Parallel()++ dir, argv := resolverScript(t, answerByCall)+ writeAnswer(t, dir, answerOf("m"))+ dirsOf := func(x, y string) []gittest.TreeEntry {+ return []gittest.TreeEntry{+ {Mode: "100644", Path: x, Content: "one\none\none\none\n"},+ {Mode: "100644", Path: y, Content: "two\ntwo\ntwo\ntwo\n"},+ }+ }+ with := func(m string, entries ...gittest.TreeEntry) []gittest.TreeEntry {+ return append(entries, gittest.TreeEntry{Mode: "100644", Path: "m", Content: m})+ }+ // One round: the lane's tip is the last thing the land sees, as when the rounds or the deadline+ // run out on it. Another round would pre-resolve at the new tip and find the same state itself.+ rcfg := enabledResolver(argv)+ rcfg.ResolveRounds = 1+ fx := newResolveFixtureWith(t, rcfg,+ with("base\n", dirsOf("A/x", "A/y")...),+ with("ours\n", dirsOf("A/x", "A/y")...),+ with("theirs\n", append(dirsOf("A/x", "A/y"), gittest.TreeEntry{Mode: "100644", Path: "A/z", Content: "new\n"})...),+ )+ // The writer splits the directory A over B and C and takes the change's m: no file conflicts.+ split := gittest.WriteCommit(t, fx.cfg, fx.repoPath, gittest.WithParents(fx.tip),+ gittest.WithTreeEntries(nestTreeEntries(t, fx.cfg, fx.repoPath, with("theirs\n", dirsOf("B/x", "C/y")...))...))+ writeMoves(t, dir, fx, split)++ _, err := fx.client.UserLand(fx.ctx, fx.request("Land"))+ requireMergeConflict(t, err)+ require.NotContains(t, err.Error(), "files kept changing")+ pkg := landConflictDetail(err)+ require.NotNil(t, pkg)+ require.Contains(t, pkg.GetReason(), "conflicts without a conflicting file")+ require.Equal(t, 1, resolverCalls(t, dir), "no round was repeated")+ requireBranchAt(t, fx.cfg, fx.repoPath, "main", split)+}diff --git a/internal/gitaly/service/operations/server.go b/internal/gitaly/service/operations/server.goindex 973ab7d..9c15cbf 100644--- a/internal/gitaly/service/operations/server.go+++ b/internal/gitaly/service/operations/server.go@@ -41,6 +41,9 @@ type Server struct {// landCheckBatchHook, when set, runs in every check batch while it holds its slot; only tests// set it.landCheckBatchHook func()+ // landResolveHook, when set, runs in every resolver call while it holds its resolve slot; only+ // tests set it.+ landResolveHook func()// conflictResolver answers the resolve requests of UserLand. Nil when resolution is disabled,// or when the configured resolver cannot be built: then resolverErr says why, and every land// that needs the resolver fails closed with it (resolveLandConflicts, landWithResolution).diff --git a/internal/gitaly/service/operations/testhelper_test.go b/internal/gitaly/service/operations/testhelper_test.goindex c1aec86..9902239 100644--- a/internal/gitaly/service/operations/testhelper_test.go+++ b/internal/gitaly/service/operations/testhelper_test.go@@ -14,6 +14,7 @@ import ("github.com/feder-cr/git-ai/internal/gitaly/service/repository""github.com/feder-cr/git-ai/internal/gitaly/service/ssh""github.com/feder-cr/git-ai/internal/gitlab"+ "github.com/feder-cr/git-ai/internal/graft/landlane""github.com/feder-cr/git-ai/internal/grpc/client""github.com/feder-cr/git-ai/internal/structerr""github.com/feder-cr/git-ai/internal/testhelper"@@ -78,7 +79,19 @@ func setupOperationsService(func runOperationServiceServer(tb testing.TB, cfg config.Cfg, options ...testserver.GitalyServerOpt) string {tb.Helper()+ return runOperationServiceServerWithLanes(tb, cfg, nil, options...)+}++// runOperationServiceServerWithLanes is runOperationServiceServer with the land lanes of the test, which+// the test can then observe (their queue depth is a metric of the lanes). Nil means lanes of the server's+// own, as in every other test.+func runOperationServiceServerWithLanes(tb testing.TB, cfg config.Cfg, lanes *landlane.Shared, options ...testserver.GitalyServerOpt) string {+ tb.Helper()+return testserver.RunGitalyServer(tb, cfg, func(srv *grpc.Server, deps *service.Dependencies) {+ if lanes != nil {+ deps.GraftLandLanes = lanes+ }gitalypb.RegisterOperationServiceServer(srv, NewServer(deps))gitalypb.RegisterHookServiceServer(srv, hook.NewServer(deps))gitalypb.RegisterRepositoryServiceServer(srv, repository.NewServer(deps))diff --git a/internal/gitaly/service/repository/create_bundle_from_ref_list.go b/internal/gitaly/service/repository/create_bundle_from_ref_list.goindex eac194f..ed2002f 100644--- a/internal/gitaly/service/repository/create_bundle_from_ref_list.go+++ b/internal/gitaly/service/repository/create_bundle_from_ref_list.go@@ -4,6 +4,7 @@ import ("bytes""errors"+ "github.com/feder-cr/git-ai/internal/git""github.com/feder-cr/git-ai/internal/git/housekeeping""github.com/feder-cr/git-ai/internal/git/localrepo""github.com/feder-cr/git-ai/internal/structerr"@@ -30,6 +31,22 @@ func (s *server) CreateBundleFromRefList(stream gitalypb.RepositoryService_Creatreturn structerr.NewInternal("cleaning up worktrees: %w", err)}+ // The patterns go to git-bundle(1) one per line on its stdin: a pattern with a line break or a NUL+ // would be another pattern, or cut short (git.ValidateStdinRevision). The first request is checked+ // before anything starts; a later one is checked as it is read, and its refusal is the answer+ // whatever the command made of the stream.+ var invalidPattern error+ checkPatterns := func(request *gitalypb.CreateBundleFromRefListRequest) error {+ for _, pattern := range request.GetPatterns() {+ if err := git.ValidateStdinRevision(string(pattern)); err != nil {+ return structerr.NewInvalidArgument("invalid pattern: %w", err)+ }+ }+ return nil+ }+ if err := checkPatterns(firstRequest); err != nil {+ return err+ }firstRead := truepatterns := streamio.NewReader(func() ([]byte, error) {var request *gitalypb.CreateBundleFromRefListRequest@@ -42,6 +59,10 @@ func (s *server) CreateBundleFromRefList(stream gitalypb.RepositoryService_Creatif err != nil {return nil, err}+ if err := checkPatterns(request); err != nil {+ invalidPattern = err+ return nil, err+ }}return append(bytes.Join(request.GetPatterns(), []byte("\n")), '\n'), nil})@@ -51,6 +72,8 @@ func (s *server) CreateBundleFromRefList(stream gitalypb.RepositoryService_Createrr = repo.CreateBundle(ctx, writer, &localrepo.CreateBundleOpts{Patterns: patterns})switch {+ case invalidPattern != nil:+ return invalidPatterncase errors.Is(err, localrepo.ErrEmptyBundle):return structerr.NewFailedPrecondition("%w", err)case err != nil:diff --git a/internal/gitaly/service/repository/create_bundle_from_ref_list_test.go b/internal/gitaly/service/repository/create_bundle_from_ref_list_test.goindex 52ee7af..158ef61 100644--- a/internal/gitaly/service/repository/create_bundle_from_ref_list_test.go+++ b/internal/gitaly/service/repository/create_bundle_from_ref_list_test.go@@ -152,3 +152,70 @@ func TestCreateBundleFromRefList_validations(t *testing.T) {})}}++// TestCreateBundleFromRefList_refusesPatternsThatAreMoreThanOneLine: the patterns reach git-bundle(1) one+// per line on its stdin, so a pattern with a line feed, a carriage return or a NUL is another pattern, or+// cut short: it names references the request never showed. Every pattern is checked, those of the first+// request before anything starts, and those of a later one as it is read.+func TestCreateBundleFromRefList_refusesPatternsThatAreMoreThanOneLine(t *testing.T) {+ t.Parallel()++ ctx := testhelper.Context(t)+ cfg, client := setupRepositoryService(t)+ repo, repoPath := gittest.CreateRepository(t, ctx, cfg)+ gittest.WriteCommit(t, cfg, repoPath, gittest.WithBranch("master"))+ gittest.WriteCommit(t, cfg, repoPath, gittest.WithReference("refs/graft/green/main"), gittest.WithMessage("hidden"))++ for desc, pattern := range map[string]string{+ "a line feed and a second pattern": "master\nrefs/graft/green/main",+ "a carriage return": "master\r",+ "a NUL": "master\x00refs/graft/green/main",+ } {+ t.Run("first request, "+desc, func(t *testing.T) {+ t.Parallel()++ stream, err := client.CreateBundleFromRefList(ctx)+ require.NoError(t, err)+ require.NoError(t, stream.Send(&gitalypb.CreateBundleFromRefListRequest{Repository: repo, Patterns: [][]byte{[]byte(pattern)}}))+ require.NoError(t, stream.CloseSend())+ var data int+ for resp, err := stream.Recv(); ; resp, err = stream.Recv() {+ data += len(resp.GetData())+ if err != nil {+ require.Equal(t, codes.InvalidArgument, status.Code(err), "%v", err)+ require.ErrorContains(t, err, "a revision can't contain a line break or NUL")+ break+ }+ }+ require.Zero(t, data, "no bundle was made")+ })++ t.Run("a later request, "+desc, func(t *testing.T) {+ t.Parallel()++ stream, err := client.CreateBundleFromRefList(ctx)+ require.NoError(t, err)+ require.NoError(t, stream.Send(&gitalypb.CreateBundleFromRefListRequest{Repository: repo, Patterns: [][]byte{[]byte("master")}}))+ require.NoError(t, stream.Send(&gitalypb.CreateBundleFromRefListRequest{Patterns: [][]byte{[]byte(pattern)}}))+ require.NoError(t, stream.CloseSend())+ for _, err = stream.Recv(); err == nil; _, err = stream.Recv() {+ }+ require.Equal(t, codes.InvalidArgument, status.Code(err), "%v", err)+ require.ErrorContains(t, err, "a revision can't contain a line break or NUL")+ })+ }++ t.Run("an ordinary pattern still makes a bundle", func(t *testing.T) {+ t.Parallel()++ stream, err := client.CreateBundleFromRefList(ctx)+ require.NoError(t, err)+ require.NoError(t, stream.Send(&gitalypb.CreateBundleFromRefListRequest{Repository: repo, Patterns: [][]byte{[]byte("master")}}))+ require.NoError(t, stream.CloseSend())+ var data int+ for resp, err := stream.Recv(); err == nil; resp, err = stream.Recv() {+ data += len(resp.GetData())+ }+ require.Positive(t, data)+ })+}