Git designed for AI agents.
graft: a model is what is not marked as a rule, and the verification sees the tree byte for byte
Federico Elia0b6693a00feb
diff --git a/doc/graft/PLAN10.md b/doc/graft/PLAN10.mdindex 68d637b..04f48f5 100644--- a/doc/graft/PLAN10.md+++ b/doc/graft/PLAN10.md@@ -760,8 +760,10 @@ flag.* **Protocol (additive).** A ResponseFile gains `"proposal": true`; graft-resolve sets it on everyfile whose result came from the llm strategy, and only there. The server treats an answer with any- proposal file as not landable; it also refuses to land any answer whose resolver names the llm- strategy, should the flag be missing.+ proposal file as not landable. *The name-based fallback this item had (an answer whose resolver+ names the llm strategy) is removed by the review of f404a71: it read only the first "x/" of a name,+ so a list joined over rounds, a list cut to ",+N" or another vendor's name missed it. Replaced by+ the fail-safe origin of 15.9.** **Server.** With resolve_conflicts, a resolution by rules alone lands as today. When a file is aproposal, the land fails with the LandConflict package: proposal files get OUTCOME_PROPOSED,files that rules resolved keep OUTCOME_RESOLVED, and nothing lands partially. Every file with@@ -1267,6 +1269,21 @@ and the tree it verifies depends on the tip.* **Rules are unchanged.** A resolution made by rules alone lands after verify.command, which runs inthe lane, as before; so does an agent's resolution. Moving them onto the same out-of-lane mechanismis the next step, measured on its own, so that in the end there is one verification mechanism.+* **What is a model's resolution: the origin of each file, fail-safe.** Each resolved file carries+ `origin`, `rule` or `model` (the older `proposal: true` means `model` and is still read). A file+ is a rule's only when it says so; a file without an origin, or with an unknown one, is a model's.+ It lands without verification only if it is marked `rule`; otherwise decideLand's one rule+ applies. The name of the resolver decides nothing (a joined list, a list cut to ",+N", another+ vendor's name). The server normalizes the origin when the answer arrives, so the answers kept by+ their keys carry it and the rounds that reuse them never depend on a name. Third-party resolvers+ that do not set `origin` no longer land without the operator's verification: that is the change.+* **What the verification sees: the tree, byte for byte.** git-archive(1) honours the attributes of+ the tree itself: export-ignore hides a file from the command, and a .gitattributes written by an+ agent or by a model could therefore decide what the operator's tests see. The checkout is written+ from git-ls-tree(1) -r and git-cat-file(1) --batch, which apply no attribute (no export-*, eol,+ filter or smudge), with modes, symbolic links as links and gitlinks as empty directories, and is+ read back and compared with the listing; a difference fails the checkout closed. The same function+ serves the verification of rules and of agents' resolutions.* **Tests.** A model's resolution that passes lands, signed, with the command run once (so not inthe lane); one that fails gives a proposal and the output; without a verify.command it is aproposal; a tip that moved gets its new tree verified and lands, with no resolver call; a tip thatdiff --git a/doc/graft/RESOLVE.md b/doc/graft/RESOLVE.mdindex da29a8c..9d54b57 100644--- a/doc/graft/RESOLVE.md+++ b/doc/graft/RESOLVE.md@@ -171,10 +171,13 @@ A model's resolution lands only through the operator's verification(PLAN10 14 and 15.9): when `verify.command` is configured and passes, out ofthe lane, on the exact tree that would land (see "Optional verification of aresolution"). Without a `verify.command` the server never lands what a model-resolved: the agent decides. graft-resolve marks every file its `llm` strategy-resolved as a proposal (`"proposal": true`), and the server treats an answer as-proposing when a file is marked, or when the resolver names the `llm` strategy-without marking any. A land of such an answer that does not land (no+resolved: the agent decides. Every resolved file says what made it, in `origin`+(see "Response"): `rule` or `model`. The check is fail-safe: a file is a rule's+only when it says so, and anything else is a model's, whatever the name of the+resolver. graft-resolve marks the files of trivial, union, rerere and the+command strategies as `rule` and those of its `llm` strategy as `model` (with+the older `"proposal": true`). A land of an answer that holds a model's file and+does not land (no`verify.command`, a verification that failed, or none finished in time) failswith the package: the model's files are`PROPOSED`, the files the rules resolved keep `RESOLVED`, and every file the@@ -253,11 +256,22 @@ Response (resolver to server):```json{"version": 1, "resolver": "graft-resolve/trivial",- "files": [{"path": "a/b.go", "action": "write", "mode": "100644", "content": "<base64>"}],+ "files": [{"path": "a/b.go", "action": "write", "mode": "100644", "content": "<base64>", "origin": "rule"}],"explanation": "free text"}```- `action` is `write` (with `mode` and `content`) or `delete`.+- `origin` says what made the file: `rule` (an operator's rule: trivial, union,+ rerere, command) or `model` (a model). **It is fail-safe, and a change for+ resolvers that do not set it: a file without `origin`, or with any other+ value, is a model's.** Only a file marked `rule` lands without the+ operator's verification of the tree (see "Optional verification of a+ resolution"); a model's file lands only if `verify.command` passed on the+ tree that lands, and is otherwise proposed. `"proposal": true` is the older+ mark of a model's file, still read: it means `model` whatever `origin` says.+ The name in `resolver` decides nothing: it may be a list joined over+ rounds, or cut to `,+N`, and a name that says `llm` makes no rule's file a+ model's.- `resolver` names the resolver. It goes into the `Graft-Resolved-By`trailer, so it must be 1 to 128 bytes of printable, single-line UTF-8.The server answers with one `resolver` for a land it resolved in rounds (the@@ -384,8 +398,14 @@ command = ["/usr/local/bin/graft-check-build"]timeout = "5m"```-The server archives the merged tree (the resolution included) into a-temporary directory and runs `command` there, with a minimal environment+The server writes the merged tree (the resolution included) into a+temporary directory byte for byte, from the entries of `git ls-tree -r` and the+contents of `git cat-file --batch`, with their modes, the symbolic links as+links and the submodules as empty directories. No attribute is applied: not+`export-ignore`, which would hide a file from the command (`git archive`+honours it), nor `export-subst`, `eol`, a filter or a smudge. The directory is+read back and compared with the listing, and a difference fails the checkout+closed. It then runs `command` there, with a minimal environment(`PATH`, and `HOME` and `TMPDIR` in a directory of their own). A non-zero exit or atimeout rejects the resolution: the land fails, every file of the package is`REJECTED`, and the reason is `verification failed:` with the exit status anddiff --git a/internal/gitaly/service/operations/land_package_test.go b/internal/gitaly/service/operations/land_package_test.goindex 57f52bb..57aa993 100644--- a/internal/gitaly/service/operations/land_package_test.go+++ b/internal/gitaly/service/operations/land_package_test.go@@ -208,7 +208,7 @@ func TestUserLand_conflictPackage(t *testing.T) {writeAnswer(t, dir, resolver.Response{Version: resolver.ProtocolVersion,Resolver: "graft-resolve/fake",- Files: []resolver.ResponseFile{{Path: "a", Action: resolver.ActionWrite, Mode: resolver.ModeRegular, Content: []byte("merged-a\n")}},+ Files: []resolver.ResponseFile{{Origin: resolver.OriginRule, Path: "a", Action: resolver.ActionWrite, Mode: resolver.ModeRegular, Content: []byte("merged-a\n")}},})fx := newResolveFixtureWith(t, enabledResolver(argv),[]gittest.TreeEntry{{Mode: "100644", Path: "a", Content: "base\n"}, {Mode: "100644", Path: "go.sum", Content: "base\n"}},diff --git a/internal/gitaly/service/operations/land_preresolve_test.go b/internal/gitaly/service/operations/land_preresolve_test.goold mode 100755new mode 100644index fa0590b..545e403--- a/internal/gitaly/service/operations/land_preresolve_test.go+++ b/internal/gitaly/service/operations/land_preresolve_test.go@@ -59,7 +59,7 @@ func writeMoves(t *testing.T, dir string, fx resolveFixture, moves ...git.Objectfunc answerOf(paths ...string) resolver.Response {resp := resolver.Response{Version: resolver.ProtocolVersion, Resolver: "graft-resolve/fake"}for _, p := range paths {- resp.Files = append(resp.Files, resolver.ResponseFile{+ resp.Files = append(resp.Files, resolver.ResponseFile{Origin: resolver.OriginRule,Path: p, Action: resolver.ActionWrite, Mode: resolver.ModeRegular, Content: []byte("merged-" + p + "\n"),})}@@ -266,6 +266,7 @@ func TestUserLand_resolvesOutOfTheLane(t *testing.T) {// runs out of the lane too, while the lane is held.proposal := answerOf("b")proposal.Resolver = "graft-resolve/llm"+ proposal.Files[0].Origin = resolver.OriginModelwriteAnswerN(t, h.dir, 2, proposal)require.NoError(t, os.WriteFile(filepath.Join(h.vdir, "fail"), nil, 0o600))diff --git a/internal/gitaly/service/operations/land_proposals.go b/internal/gitaly/service/operations/land_proposals.goindex 13b9dbd..0c5fe67 100644--- a/internal/gitaly/service/operations/land_proposals.go+++ b/internal/gitaly/service/operations/land_proposals.go@@ -7,7 +7,6 @@ import ("fmt""path""strconv"- "strings""syscall""time"@@ -50,28 +49,27 @@ const landProposedReason = "a model resolved the file: it is proposed (see propo// landNoVerifyReason follows landProposedReason when no verify.command is configured.const landNoVerifyReason = "no verify.command is configured, so nothing verifies a model's resolution"-// landAnswerFlagsProposals reports whether a file of the answer is marked as a proposal.-func landAnswerFlagsProposals(answer *resolver.Response) bool {- for _, f := range answer.Files {- if f.Proposal {- return true+// landNormalizeOrigins gives each file of an answer its origin, as the server reads it: OriginRule+// only for a file that says it is a rule's and is not marked as a proposal, OriginModel for any+// other. It is done when the answer arrives, so that the answers kept by their keys and the rounds+// that reuse them never depend on how a resolver spelled the mark, nor on its name.+func landNormalizeOrigins(answer *resolver.Response) {+ for i := range answer.Files {+ if answer.Files[i].FromModel() {+ answer.Files[i].Origin = resolver.OriginModel+ } else {+ answer.Files[i].Origin = resolver.OriginRule}}- return false}-// landAnswerProposes reports whether a resolver's answer holds a model's resolution: a file marked-// as a proposal, or, should a resolver omit the mark, a resolver that names the llm strategy.+// landAnswerProposes reports whether a resolver's answer holds a model's resolution: a file that is+// not marked as a rule's. The name of the resolver decides nothing: a list of names joined over+// rounds, cut to ",+N", or spelled in a way nobody foresaw, can neither hide a model's file nor+// make a rule's one a model's. This is fail-safe: a file that says nothing is a model's.func landAnswerProposes(answer *resolver.Response) bool {- if landAnswerFlagsProposals(answer) {- return true- }- name := answer.Resolver- if i := strings.IndexByte(name, '/'); i >= 0 {- name = name[i+1:]- }- for _, strategy := range strings.Split(name, ",") {- if strings.TrimSpace(strategy) == "llm" {+ for _, f := range answer.Files {+ if f.FromModel() {return true}}diff --git a/internal/gitaly/service/operations/land_proposals_test.go b/internal/gitaly/service/operations/land_proposals_test.goindex cde0b1d..6bd0754 100755--- a/internal/gitaly/service/operations/land_proposals_test.go+++ b/internal/gitaly/service/operations/land_proposals_test.go@@ -35,6 +35,7 @@ func modelAnswer() resolver.Response {answer.Resolver = "graft-resolve/llm"answer.Explanation = "patched by the model"for i := range answer.Files {+ answer.Files[i].Origin = resolver.OriginModelanswer.Files[i].Proposal = true}return answer@@ -92,23 +93,56 @@ func TestUserLand_proposals(t *testing.T) {}})- t.Run("an answer of the llm strategy without the mark is proposed too", func(t *testing.T) {- t.Parallel()-- dir, argv := resolverScript(t, "")- answer := validAnswer()- answer.Resolver = "graft-resolve/trivial,llm"- writeAnswer(t, dir, answer)- fx := newResolveFixture(t, enabledResolver(argv))-- _, err := fx.client.UserLand(fx.ctx, fx.request("Land"))- requireMergeConflict(t, err)- requireBranchAt(t, fx.cfg, fx.repoPath, git.DefaultBranch, fx.tip)- pkg := landConflictDetail(err)- for _, p := range []string{"a", "b"} {- require.Equal(t, gitalypb.LandConflictFile_OUTCOME_PROPOSED, packageFile(t, pkg, p).GetOutcome(), p)- }- })+ // The origin of a file decides, never the name of the resolver (PLAN10 15.9): a file lands without+ // the operator's verification only if it says it is a rule's. Anything else is a model's, and+ // with no verify.command it is proposed. The names are the joined and the cut ones a resolver+ // gives after rounds, which a test of the names could not tell from a rule's.+ for desc, tc := range map[string]struct {+ resolverName string+ origin string+ proposal bool+ lands bool+ }{+ "nothing says what made the files, a plain name": {"graft-resolve/trivial", "", false, false},+ "nothing says what made the files, joined names": {"graft-resolve/trivial,graft-resolve/llm", "", false, false},+ "nothing says what made the files, a list cut to +N": {"graft-resolve/trivial,+1", "", false, false},+ "nothing says what made the files, other names": {"acme/union,acme/llm", "", false, false},+ "an origin nobody knows": {"graft-resolve/trivial", "wizard", false, false},+ "the files are marked as a model's": {"graft-resolve/trivial", resolver.OriginModel, false, false},+ "the files are marked as rules and as proposals": {"graft-resolve/trivial", resolver.OriginRule, true, false},+ "rules' files under joined names that say llm": {"graft-resolve/trivial,graft-resolve/llm", resolver.OriginRule, false, true},+ "rules' files under a list cut to +N": {"graft-resolve/trivial,+1", resolver.OriginRule, false, true},+ "rules' files under a plain name": {"graft-resolve/trivial", resolver.OriginRule, false, true},+ } {+ t.Run("the origin decides, not the name: "+desc, func(t *testing.T) {+ t.Parallel()++ dir, argv := resolverScript(t, "")+ answer := validAnswer()+ answer.Resolver = tc.resolverName+ for i := range answer.Files {+ answer.Files[i].Origin = tc.origin+ answer.Files[i].Proposal = tc.proposal+ }+ writeAnswer(t, dir, answer)+ fx := newResolveFixture(t, enabledResolver(argv))++ resp, err := fx.client.UserLand(fx.ctx, fx.request("Land"))+ if tc.lands {+ require.NoError(t, err)+ requireBranchAt(t, fx.cfg, fx.repoPath, git.DefaultBranch, git.ObjectID(resp.GetLandedCommitId()))+ return+ }+ requireMergeConflict(t, err)+ requireBranchAt(t, fx.cfg, fx.repoPath, git.DefaultBranch, fx.tip)+ pkg := landConflictDetail(err)+ for _, p := range []string{"a", "b"} {+ f := packageFile(t, pkg, p)+ require.Equal(t, gitalypb.LandConflictFile_OUTCOME_PROPOSED, f.GetOutcome(), p)+ require.Contains(t, f.GetReason(), "no verify.command is configured", p)+ }+ })+ }t.Run("a file the rules resolved keeps its outcome and gets its proposal", func(t *testing.T) {t.Parallel()@@ -116,6 +150,7 @@ func TestUserLand_proposals(t *testing.T) {dir, argv := resolverScript(t, "")answer := modelAnswer()answer.Resolver = "graft-resolve/trivial,llm"+ answer.Files[1].Origin = resolver.OriginRuleanswer.Files[1].Proposal = falsewriteAnswer(t, dir, answer)fx := newResolveFixture(t, enabledResolver(argv))@@ -277,20 +312,32 @@ func TestUserLand_proposals(t *testing.T) {func TestLandAnswerProposes(t *testing.T) {t.Parallel()- for _, tc := range []struct {- resolver string- flagged bool- want bool+ rule := resolver.ResponseFile{Path: "a", Origin: resolver.OriginRule}+ model := resolver.ResponseFile{Path: "b", Origin: resolver.OriginModel}+ for desc, tc := range map[string]struct {+ files []resolver.ResponseFile+ want bool}{- {"graft-resolve/trivial", false, false},- {"graft-resolve/trivial,union", false, false},- {"graft-resolve/llm", false, true},- {"graft-resolve/trivial, llm", false, true},- {"graft-resolve/llm-ish", false, false},- {"graft-resolve/trivial", true, true},+ "a rule's file": {[]resolver.ResponseFile{rule}, false},+ "two rules' files": {[]resolver.ResponseFile{rule, {Path: "c", Origin: resolver.OriginRule}}, false},+ "a model's file": {[]resolver.ResponseFile{model}, true},+ "a rule's and a model's": {[]resolver.ResponseFile{rule, model}, true},+ "a file that says nothing": {[]resolver.ResponseFile{{Path: "a"}}, true},+ "a rule's and one that says nothing": {[]resolver.ResponseFile{rule, {Path: "b"}}, true},+ "an origin nobody knows": {[]resolver.ResponseFile{{Path: "a", Origin: "wizard"}}, true},+ "the origin in another case": {[]resolver.ResponseFile{{Path: "a", Origin: "Rule"}}, true},+ "a rule's mark with the older proposal": {[]resolver.ResponseFile{{Path: "a", Origin: resolver.OriginRule, Proposal: true}}, true},+ "the older proposal mark alone": {[]resolver.ResponseFile{{Path: "a", Proposal: true}}, true},+ "no file": {nil, false},} {- answer := &resolver.Response{Resolver: tc.resolver, Files: []resolver.ResponseFile{{Path: "a", Proposal: tc.flagged}}}- require.Equal(t, tc.want, landAnswerProposes(answer), "%s, flagged %v", tc.resolver, tc.flagged)+ // The name of the resolver decides nothing, whatever it says.+ for _, name := range []string{+ "graft-resolve/trivial", "graft-resolve/llm", "graft-resolve/trivial,graft-resolve/llm",+ "acme/union,acme/llm", "graft-resolve/trivial,+1", "graft-resolve/trivial, llm", "",+ } {+ answer := &resolver.Response{Resolver: name, Files: tc.files}+ require.Equal(t, tc.want, landAnswerProposes(answer), "%s, resolver %q", desc, name)+ }}}@@ -302,7 +349,9 @@ func TestDecideLand(t *testing.T) {resolved := func() []*landFileState {return []*landFileState{{outcome: gitalypb.LandConflictFile_OUTCOME_RESOLVED}, {outcome: gitalypb.LandConflictFile_OUTCOME_RESOLVED}}}- rules := &resolver.Response{Resolver: "graft-resolve/trivial,union", Files: []resolver.ResponseFile{{Path: "a"}, {Path: "b"}}}+ rules := &resolver.Response{Resolver: "graft-resolve/trivial,union", Files: []resolver.ResponseFile{+ {Path: "a", Origin: resolver.OriginRule}, {Path: "b", Origin: resolver.OriginRule},+ }}require.True(t, decideLand(rules, resolved(), "", ""), "rules resolved every file")require.True(t, decideLand(rules, resolved(), "t1", "t2"), "rules land whatever tree was verified: their verification stays in the lane")require.False(t, decideLand(nil, resolved(), "t1", "t1"), "no answer")@@ -319,12 +368,14 @@ func TestDecideLand(t *testing.T) {// THE rule (PLAN10 15.9): a file that proposes may land only if verify.command passed on the exact// tree that lands, compared by object ID.- flagged := &resolver.Response{Resolver: "graft-resolve/trivial", Files: []resolver.ResponseFile{{Path: "a"}, {Path: "b", Proposal: true}}}- named := &resolver.Response{Resolver: "graft-resolve/trivial,llm", Files: []resolver.ResponseFile{{Path: "a"}, {Path: "b"}}}+ flagged := &resolver.Response{Resolver: "graft-resolve/trivial", Files: []resolver.ResponseFile{+ {Path: "a", Origin: resolver.OriginRule}, {Path: "b", Origin: resolver.OriginRule, Proposal: true},+ }}+ unmarked := &resolver.Response{Resolver: "graft-resolve/trivial", Files: []resolver.ResponseFile{{Path: "a", Origin: resolver.OriginRule}, {Path: "b"}}}proposing := func() []*landFileState {return []*landFileState{{outcome: gitalypb.LandConflictFile_OUTCOME_RESOLVED}, {outcome: gitalypb.LandConflictFile_OUTCOME_PROPOSED}}}- for desc, answer := range map[string]*resolver.Response{"a flagged file": flagged, "the llm strategy by name": named} {+ for desc, answer := range map[string]*resolver.Response{"a file marked as a proposal": flagged, "a file that says nothing": unmarked} {require.True(t, decideLand(answer, proposing(), "tree-1", "tree-1"), "%s: verified on this very tree", desc)require.False(t, decideLand(answer, proposing(), "tree-1", ""), "%s: nothing verified", desc)require.False(t, decideLand(answer, proposing(), "tree-1", "tree-2"), "%s: verified on another tree (the tip moved)", desc)diff --git a/internal/gitaly/service/operations/land_resolve.go b/internal/gitaly/service/operations/land_resolve.goindex 72d0f6c..daca754 100644--- a/internal/gitaly/service/operations/land_resolve.go+++ b/internal/gitaly/service/operations/land_resolve.go@@ -198,6 +198,7 @@ func (s *Server) landResolveAt(landFailSent(files, gitalypb.LandConflictFile_OUTCOME_FAILED, fmt.Sprintf("the resolver's answer names an invalid resolver %q", resp.Resolver))default:answer = resp+ landNormalizeOrigins(answer)}}@@ -270,24 +271,24 @@ func (s *Server) landResolveAt(}// landSetOutcomes sets the outcome of each of files from the answer that covers them: RESOLVED, or-// REJECTED when the resolution writes a mode no side has, or PROPOSED when a model made the file.-// A resolver that names the llm strategy but marks no file is taken to have proposed every file:-// the mark may be missing, never the other way round. The lane sets them the same way from the-// verdicts it applies, so that both ask decideLand the same question.+// REJECTED when the resolution writes a mode no side has, or PROPOSED when a model made the file,+// that is when the file is not marked as a rule's (the mark may be missing, never the other way+// round). The lane sets them the same way from the verdicts it applies, so that both ask decideLand+// the same question.func landSetOutcomes(answer *resolver.Response, files []*landFileState) {- proposes := landAnswerProposes(answer)- flagged := landAnswerFlagsProposals(answer)written := make(map[string]resolver.ResponseFile, len(answer.Files))for _, rf := range answer.Files {written[rf.Path] = rf}for _, f := range files {- rf := written[f.conflict.path]+ rf, ok := written[f.conflict.path]f.outcome, f.reason = gitalypb.LandConflictFile_OUTCOME_RESOLVED, ""switch err := landModeOfSides(f.conflict, rf); {+ case !ok:+ f.outcome, f.reason = gitalypb.LandConflictFile_OUTCOME_FAILED, "the resolver gave no answer for the file"case err != nil:f.outcome, f.reason = gitalypb.LandConflictFile_OUTCOME_REJECTED, err.Error()- case rf.Proposal || (proposes && !flagged):+ case rf.FromModel():f.outcome, f.reason = gitalypb.LandConflictFile_OUTCOME_PROPOSED, landProposedReason}}diff --git a/internal/gitaly/service/operations/land_resolve_bench_test.go b/internal/gitaly/service/operations/land_resolve_bench_test.goindex caa2e9f..0544f69 100755--- a/internal/gitaly/service/operations/land_resolve_bench_test.go+++ b/internal/gitaly/service/operations/land_resolve_bench_test.go@@ -140,7 +140,7 @@ cat "$DIR/answer.json"func answerOfPaths(paths ...string) resolver.Response {resp := resolver.Response{Version: resolver.ProtocolVersion, Resolver: "graft-resolve/bench"}for _, p := range paths {- resp.Files = append(resp.Files, resolver.ResponseFile{+ resp.Files = append(resp.Files, resolver.ResponseFile{Origin: resolver.OriginRule,Path: p, Action: resolver.ActionWrite, Mode: resolver.ModeRegular, Content: []byte("merged-" + p + "\n"),})}diff --git a/internal/gitaly/service/operations/land_resolve_paths_test.go b/internal/gitaly/service/operations/land_resolve_paths_test.goindex ee2e1f2..14c04e2 100644--- a/internal/gitaly/service/operations/land_resolve_paths_test.go+++ b/internal/gitaly/service/operations/land_resolve_paths_test.go@@ -134,7 +134,7 @@ func TestUserLand_pathStartingWithDashIsResolved(t *testing.T) {Version: resolver.ProtocolVersion,Resolver: "graft-resolve/fake",Files: []resolver.ResponseFile{- {Path: "-a", Action: resolver.ActionWrite, Mode: resolver.ModeRegular, Content: []byte("merged-a\n")},+ {Origin: resolver.OriginRule, Path: "-a", Action: resolver.ActionWrite, Mode: resolver.ModeRegular, Content: []byte("merged-a\n")},},})fx := newResolveFixtureWith(t, enabledResolver(argv),diff --git a/internal/gitaly/service/operations/land_resolve_rename_test.go b/internal/gitaly/service/operations/land_resolve_rename_test.goindex 645babf..b687169 100644--- a/internal/gitaly/service/operations/land_resolve_rename_test.go+++ b/internal/gitaly/service/operations/land_resolve_rename_test.go@@ -43,7 +43,7 @@ func TestResolveLandConflicts_renameOnCommitSide(t *testing.T) {Version: resolver.ProtocolVersion,Resolver: "graft-resolve/fake",Files: []resolver.ResponseFile{- {Path: "b", Action: resolver.ActionWrite, Mode: resolver.ModeRegular, Content: []byte("merged\n")},+ {Origin: resolver.OriginRule, Path: "b", Action: resolver.ActionWrite, Mode: resolver.ModeRegular, Content: []byte("merged\n")},},})@@ -90,7 +90,7 @@ func TestResolveLandConflicts_renameOnTipSide(t *testing.T) {Version: resolver.ProtocolVersion,Resolver: "graft-resolve/fake",Files: []resolver.ResponseFile{- {Path: "b", Action: resolver.ActionWrite, Mode: resolver.ModeRegular, Content: []byte("merged\n")},+ {Origin: resolver.OriginRule, Path: "b", Action: resolver.ActionWrite, Mode: resolver.ModeRegular, Content: []byte("merged\n")},},})@@ -171,7 +171,7 @@ func TestUserLand_renameFromDeniedPathIsNotResolved(t *testing.T) {Version: resolver.ProtocolVersion,Resolver: "graft-resolve/fake",Files: []resolver.ResponseFile{- {Path: "pub/k", Action: resolver.ActionWrite, Mode: resolver.ModeRegular, Content: []byte("merged\n")},+ {Origin: resolver.OriginRule, Path: "pub/k", Action: resolver.ActionWrite, Mode: resolver.ModeRegular, Content: []byte("merged\n")},},})rcfg := enabledResolver(argv)@@ -214,7 +214,7 @@ func TestUserLand_renameBetweenAllowedPathsIsResolved(t *testing.T) {Version: resolver.ProtocolVersion,Resolver: "graft-resolve/fake",Files: []resolver.ResponseFile{- {Path: "pub/k", Action: resolver.ActionWrite, Mode: resolver.ModeRegular, Content: []byte("merged\n")},+ {Origin: resolver.OriginRule, Path: "pub/k", Action: resolver.ActionWrite, Mode: resolver.ModeRegular, Content: []byte("merged\n")},},})rcfg := enabledResolver(argv)diff --git a/internal/gitaly/service/operations/land_resolve_test.go b/internal/gitaly/service/operations/land_resolve_test.goindex 45ed684..94e7914 100644--- a/internal/gitaly/service/operations/land_resolve_test.go+++ b/internal/gitaly/service/operations/land_resolve_test.go@@ -100,8 +100,8 @@ func validAnswer() resolver.Response {Version: resolver.ProtocolVersion,Resolver: "graft-resolve/fake",Files: []resolver.ResponseFile{- {Path: "a", Action: resolver.ActionWrite, Mode: resolver.ModeRegular, Content: []byte("merged-a\n")},- {Path: "b", Action: resolver.ActionWrite, Mode: resolver.ModeRegular, Content: []byte("merged-b\n")},+ {Origin: resolver.OriginRule, Path: "a", Action: resolver.ActionWrite, Mode: resolver.ModeRegular, Content: []byte("merged-a\n")},+ {Origin: resolver.OriginRule, Path: "b", Action: resolver.ActionWrite, Mode: resolver.ModeRegular, Content: []byte("merged-b\n")},},}}@@ -412,7 +412,7 @@ func TestResolveLandConflicts_pathEncoding(t *testing.T) {Version: resolver.ProtocolVersion,Resolver: "graft-resolve/fake",Files: []resolver.ResponseFile{- {Path: strings.ToValidUTF8(p, "�"), Action: resolver.ActionWrite, Mode: resolver.ModeRegular, Content: []byte("merged\n")},+ {Origin: resolver.OriginRule, Path: strings.ToValidUTF8(p, "�"), Action: resolver.ActionWrite, Mode: resolver.ModeRegular, Content: []byte("merged\n")},},}writeAnswer(t, dir, answer)@@ -433,7 +433,7 @@ func TestResolveLandConflicts_pathEncoding(t *testing.T) {Version: resolver.ProtocolVersion,Resolver: "graft-resolve/fake",Files: []resolver.ResponseFile{- {Path: p, Action: resolver.ActionWrite, Mode: resolver.ModeRegular, Content: []byte("merged\n")},+ {Origin: resolver.OriginRule, Path: p, Action: resolver.ActionWrite, Mode: resolver.ModeRegular, Content: []byte("merged\n")},},})fx := newResolveFixtureWith(t, config.GraftResolver{}, entries(p, "base\n"), entries(p, "ours\n"), entries(p, "theirs\n"))@@ -477,7 +477,7 @@ func TestResolveLandConflicts_rejectedAnswers(t *testing.T) {}{{desc: "unsupported version", mutate: func(r *resolver.Response) { r.Version = 2 }},{desc: "path that did not conflict", mutate: func(r *resolver.Response) {- r.Files = append(r.Files, resolver.ResponseFile{Path: "c", Action: resolver.ActionWrite, Mode: resolver.ModeRegular, Content: []byte("x\n")})+ r.Files = append(r.Files, resolver.ResponseFile{Origin: resolver.OriginRule, Path: "c", Action: resolver.ActionWrite, Mode: resolver.ModeRegular, Content: []byte("x\n")})}},{desc: "path answered twice", mutate: func(r *resolver.Response) {r.Files = append(r.Files, r.Files[0])@@ -558,7 +558,7 @@ func TestResolveLandConflicts_policy(t *testing.T) {writeAnswer(t, dir, resolver.Response{Version: resolver.ProtocolVersion,Resolver: "graft-resolve/fake",- Files: []resolver.ResponseFile{{Path: "b", Action: resolver.ActionWrite, Mode: resolver.ModeRegular, Content: []byte("merged-b\n")}},+ Files: []resolver.ResponseFile{{Origin: resolver.OriginRule, Path: "b", Action: resolver.ActionWrite, Mode: resolver.ModeRegular, Content: []byte("merged-b\n")}},})s := fx.server(t, tc.rcfg(enabledResolver(argv)))diff --git a/internal/gitaly/service/operations/land_verified_test.go b/internal/gitaly/service/operations/land_verified_test.goindex f2a2c2a..8dbf184 100644--- a/internal/gitaly/service/operations/land_verified_test.go+++ b/internal/gitaly/service/operations/land_verified_test.go@@ -120,6 +120,42 @@ func TestUserLand_verifiedModelResolution(t *testing.T) {require.Empty(t, proposalRefTarget(t, fx), "nothing was proposed")})+ // A file that says nothing about what made it is a model's (fail-safe): whatever the name of the+ // resolver, it lands only through the verification.+ for desc, name := range map[string]string{+ "a plain name": "graft-resolve/trivial",+ "joined names": "graft-resolve/trivial,graft-resolve/llm",+ "a list cut to +N": "graft-resolve/trivial,+1",+ } {+ t.Run("an answer that says nothing about its files lands only if the verification passes, "+desc, func(t *testing.T) {+ t.Parallel()++ unmarked := validAnswer()+ unmarked.Resolver = name+ for i := range unmarked.Files {+ unmarked.Files[i].Origin = ""+ }++ passing, _, passingDir := verifiedFixture(t, unmarked, "exit 0", nil)+ resp, err := passing.client.UserLand(passing.ctx, passing.request("Land"))+ require.NoError(t, err)+ commit, _ := landedCommit(t, passing, resp)+ require.Equal(t, 1, verifyRuns(t, passingDir))+ require.Contains(t, string(commit.GetBody()), "Graft-Checked-By: verify:verify.sh\n")++ failing, _, failingDir := verifiedFixture(t, unmarked, `echo "tests failed"; exit 3`, nil)+ _, err = failing.client.UserLand(failing.ctx, failing.request("Land"))+ requireMergeConflict(t, err)+ for _, p := range []string{"a", "b"} {+ f := packageFile(t, landConflictDetail(err), p)+ require.Equal(t, gitalypb.LandConflictFile_OUTCOME_PROPOSED, f.GetOutcome(), p)+ require.Contains(t, f.GetReason(), "verification failed: exit status 3: tests failed", p)+ }+ require.Equal(t, 1, verifyRuns(t, failingDir))+ requireBranchAt(t, failing.cfg, failing.repoPath, git.DefaultBranch, failing.tip)+ })+ }+t.Run("a model's file next to a rule's file lands when the tree passes", func(t *testing.T) {t.Parallel()diff --git a/internal/gitaly/service/operations/land_verify.go b/internal/gitaly/service/operations/land_verify.goindex 0f1f9ac..1ea46e7 100644--- a/internal/gitaly/service/operations/land_verify.go+++ b/internal/gitaly/service/operations/land_verify.go@@ -1,14 +1,17 @@package operationsimport (- "archive/tar"+ "bufio"+ "bytes""context""errors""fmt""io"+ "io/fs""os""path""path/filepath"+ "strconv""strings""github.com/feder-cr/git-ai/internal/command"@@ -21,6 +24,11 @@ import (// 3.6): before a resolution lands, the configured command runs in a temporary checkout of the// merged tree, the resolution included. A failure or a timeout rejects the resolution, and the// conflict package says why. It applies to resolver answers and agents' resolutions alike.+//+// The checkout is the tree byte for byte (landCheckoutTree): what the operator's command sees is+// what lands. git-archive(1) is not used, since it honours the attributes of the tree itself+// (export-ignore hides a file, export-subst changes its content), and a .gitattributes written by+// an agent or by a model would decide what the verification sees.// landVerifyOutputBytes bounds the output of the command kept as the reason.const landVerifyOutputBytes = 4 << 10@@ -55,24 +63,7 @@ func (s *Server) landVerify(ctx context.Context, repo *localrepo.Repo, tree git.}}- archive, err := os.Create(filepath.Join(dir, "tree.tar"))- if err != nil {- return fmt.Errorf("verification archive: %w", err)- }- defer archive.Close()- var stderr landBoundedBuffer- stderr.limit = landVerifyOutputBytes- if err := repo.ExecAndWait(ctx,- gitcmd.Command{Name: "archive", Flags: []gitcmd.Option{gitcmd.ValueFlag{Name: "--format", Value: "tar"}}, Args: []string{tree.String()}},- gitcmd.WithStdout(archive),- gitcmd.WithStderr(&stderr),- ); err != nil {- return fmt.Errorf("archiving the merged tree: %w, stderr: %q", err, stderr.String())- }- if _, err := archive.Seek(0, io.SeekStart); err != nil {- return fmt.Errorf("verification archive: %w", err)- }- if err := landExtractTar(archive, checkout); err != nil {+ if err := landCheckoutTree(ctx, repo, tree, checkout); err != nil {return fmt.Errorf("checking out the merged tree: %w", err)}@@ -155,69 +146,284 @@ func (s *Server) landVerifyTrailer() string {return name}-// landExtractTar writes the files of the archive of a tree under dir. It refuses an entry that-// would leave dir, or that lies under a symbolic link it wrote: a link is written as a link and-// never followed.-func landExtractTar(r io.Reader, dir string) error {- links := map[string]bool{}- tr := tar.NewReader(r)- for {- h, err := tr.Next()- if errors.Is(err, io.EOF) {- return nil+// landTreeEntry is one entry of a tree as git-ls-tree(1) -r lists it: a file, a symbolic link or a+// gitlink (a submodule), by path. size and target are filled when it is written: the length of the+// file, the target of the link.+type landTreeEntry struct {+ mode uint32+ oid git.ObjectID+ path string+ size int64+ target string+}++const (+ landTreeModeTypeMask = 0o170000+ landTreeModeRegular = 0o100000+ landTreeModeSymlink = 0o120000+ landTreeModeGitlink = 0o160000+ // landMaxLinkTarget bounds the target of a symbolic link, which is read into memory.+ landMaxLinkTarget = 64 << 10+)++func (e landTreeEntry) isGitlink() bool { return e.mode&landTreeModeTypeMask == landTreeModeGitlink }+func (e landTreeEntry) isSymlink() bool { return e.mode&landTreeModeTypeMask == landTreeModeSymlink }+func (e landTreeEntry) isRegular() bool { return e.mode&landTreeModeTypeMask == landTreeModeRegular }++// landCheckoutTree writes the tree under dir byte for byte: the entries of git-ls-tree(1) -r and the+// contents of git-cat-file(1) --batch, which apply no attribute (no export-ignore, export-subst,+// eol, filter or smudge), with the modes (a file is executable or not), the symbolic links as links,+// and the gitlinks as empty directories. It then reads the directory back and compares it with the+// listing: a difference fails the checkout, closed, since the operator's command would otherwise+// verify a tree that is not the one that lands.+func landCheckoutTree(ctx context.Context, repo *localrepo.Repo, tree git.ObjectID, dir string) error {+ return landCheckoutTreeThen(ctx, repo, tree, dir, nil)+}++// landCheckoutTreeThen is landCheckoutTree with a function that runs between the writing and the+// check; only tests give one, to damage the checkout and see that the check notices.+func landCheckoutTreeThen(ctx context.Context, repo *localrepo.Repo, tree git.ObjectID, dir string, afterWrite func()) error {+ var listing bytes.Buffer+ var stderr landBoundedBuffer+ stderr.limit = landVerifyOutputBytes+ if err := repo.ExecAndWait(ctx,+ gitcmd.Command{Name: "ls-tree", Flags: []gitcmd.Option{gitcmd.Flag{Name: "-r"}, gitcmd.Flag{Name: "-z"}, gitcmd.Flag{Name: "--full-tree"}}, Args: []string{tree.String()}},+ gitcmd.WithStdout(&listing),+ gitcmd.WithStderr(&stderr),+ ); err != nil {+ return fmt.Errorf("listing the tree: %w, stderr: %q", err, stderr.String())+ }+ entries, err := landParseTreeListing(listing.Bytes())+ if err != nil {+ return err+ }++ // The contents come in the order of the listing, one blob after the other: the files and the links.+ var input bytes.Buffer+ for _, e := range entries {+ if !e.isGitlink() {+ input.WriteString(e.oid.String() + "\n")+ }+ }+ stderr = landBoundedBuffer{limit: landVerifyOutputBytes}+ cmd, err := repo.Exec(ctx,+ gitcmd.Command{Name: "cat-file", Flags: []gitcmd.Option{gitcmd.Flag{Name: "--batch"}}},+ gitcmd.WithStdin(&input),+ gitcmd.WithSetupStdout(),+ gitcmd.WithStderr(&stderr),+ )+ if err != nil {+ return fmt.Errorf("reading the blobs of the tree: %w", err)+ }+ batch := bufio.NewReaderSize(cmd, 1<<20)+ writeErr := landMaterialize(entries, batch, dir)+ if writeErr == nil {+ // Nothing else may follow the last blob.+ if _, err := batch.ReadByte(); !errors.Is(err, io.EOF) {+ writeErr = fmt.Errorf("git-cat-file(1) answered more than the blobs asked for: %v", err)}+ }+ if waitErr := cmd.Wait(); waitErr != nil && writeErr == nil {+ return fmt.Errorf("reading the blobs of the tree: %w, stderr: %q", waitErr, stderr.String())+ }+ if writeErr != nil {+ return writeErr+ }+ if afterWrite != nil {+ afterWrite()+ }+ return landCheckMaterialized(entries, dir)+}++// landParseTreeListing parses the output of git-ls-tree(1) -r -z: "<mode> <type> <object>\t<path>"+// entries ended by NUL. A tree holds files, links and gitlinks once listed recursively.+func landParseTreeListing(raw []byte) ([]landTreeEntry, error) {+ var entries []landTreeEntry+ for _, record := range strings.Split(string(raw), "\x00") {+ if record == "" {+ continue+ }+ meta, name, ok := strings.Cut(record, "\t")+ fields := strings.Fields(meta)+ if !ok || len(fields) != 3 {+ return nil, fmt.Errorf("unexpected entry in the listing of the tree: %q", record)+ }+ mode, err := strconv.ParseUint(fields[0], 8, 32)if err != nil {- return err+ return nil, fmt.Errorf("unexpected mode in the listing of the tree: %q", record)}- if h.Typeflag == tar.TypeXGlobalHeader {- continue // git archive's commit header+ e := landTreeEntry{mode: uint32(mode), oid: git.ObjectID(fields[2]), path: name}+ switch {+ case e.isRegular() && fields[1] == "blob", e.isSymlink() && fields[1] == "blob", e.isGitlink() && fields[1] == "commit":+ default:+ return nil, fmt.Errorf("entry %q has the mode %s and the type %s, which a checkout does not write", name, fields[0], fields[1])}- name := strings.TrimSuffix(h.Name, "/")- if name == "" || path.IsAbs(name) || path.Clean(name) != name || name == ".." || strings.HasPrefix(name, "../") {- return fmt.Errorf("unsafe archive entry %q", h.Name)+ entries = append(entries, e)+ }+ return entries, nil+}++// landTreePathValid reports whether the path of a tree entry may be written under a directory: a+// clean relative path that does not leave it.+func landTreePathValid(name string) bool {+ return name != "" && !path.IsAbs(name) && path.Clean(name) == name && name != ".." && !strings.HasPrefix(name, "../") &&+ !strings.ContainsRune(name, 0)+}++// landMaterialize writes the entries under dir, reading the contents of the files and links from batch+// in their order (the answer of git-cat-file(1) --batch to the blobs of the listing, gitlinks left+// out). It refuses a path that would leave dir or lie under a symbolic link it wrote: a link is+// written as a link and never followed. It sets size and target of the entries.+func landMaterialize(entries []landTreeEntry, batch *bufio.Reader, dir string) error {+ links := map[string]bool{}+ for i := range entries {+ e := &entries[i]+ if !landTreePathValid(e.path) {+ return fmt.Errorf("unsafe path %q in the tree", e.path)}- for p := path.Dir(name); p != "."; p = path.Dir(p) {+ for p := path.Dir(e.path); p != "."; p = path.Dir(p) {if links[p] {- return fmt.Errorf("archive entry %q lies under the symbolic link %q", h.Name, p)+ return fmt.Errorf("entry %q lies under the symbolic link %q", e.path, p)}}- target := filepath.Join(dir, filepath.FromSlash(name))- switch h.Typeflag {- case tar.TypeDir:+ target := filepath.Join(dir, filepath.FromSlash(e.path))+ if e.isGitlink() {if err := os.MkdirAll(target, 0o755); err != nil {return err}- case tar.TypeReg:- if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil {+ continue+ }++ size, err := landReadBatchHeader(batch, e.oid)+ if err != nil {+ return fmt.Errorf("%q: %w", e.path, err)+ }+ if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil {+ return err+ }+ if e.isSymlink() {+ if size > landMaxLinkTarget {+ return fmt.Errorf("the symbolic link %q has a target of %d bytes", e.path, size)+ }+ content := make([]byte, size)+ if _, err := io.ReadFull(batch, content); err != nil {+ return fmt.Errorf("%q: %w", e.path, err)+ }+ if err := os.Symlink(string(content), target); err != nil {return err}- mode := os.FileMode(0o644)- if h.Mode&0o111 != 0 {- mode = 0o755+ e.target, e.size = string(content), size+ links[e.path] = true+ } else {+ perm := os.FileMode(0o644)+ if e.mode&0o111 != 0 {+ perm = 0o755}- f, err := os.OpenFile(target, os.O_WRONLY|os.O_CREATE|os.O_EXCL, mode)+ f, err := os.OpenFile(target, os.O_WRONLY|os.O_CREATE|os.O_EXCL, perm)if err != nil {return err}- if _, err := io.Copy(f, tr); err != nil {+ if _, err := io.CopyN(f, batch, size); err != nil {_ = f.Close()- return err+ return fmt.Errorf("%q: %w", e.path, err)}if err := f.Close(); err != nil {return err}- case tar.TypeSymlink:- if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil {+ // The mode is the tree's, whatever the umask of the process.+ if err := os.Chmod(target, perm); err != nil {return err}- if err := os.Symlink(h.Linkname, target); err != nil {- return err+ e.size = size+ }+ if b, err := batch.ReadByte(); err != nil || b != '\n' {+ return fmt.Errorf("%q: the blob is not followed by a line feed", e.path)+ }+ }+ return nil+}++// landReadBatchHeader reads the header "<object> <type> <size>" of an answer of git-cat-file(1) --batch+// and returns the size of the blob; the object must be the one asked for.+func landReadBatchHeader(batch *bufio.Reader, want git.ObjectID) (int64, error) {+ line, err := batch.ReadString('\n')+ if err != nil {+ return 0, fmt.Errorf("reading the header of the blob: %w", err)+ }+ fields := strings.Fields(line)+ if len(fields) != 3 || fields[0] != want.String() || fields[1] != "blob" {+ return 0, fmt.Errorf("unexpected answer of git-cat-file(1) for %s: %q", want, strings.TrimSpace(line))+ }+ size, err := strconv.ParseInt(fields[2], 10, 64)+ if err != nil || size < 0 {+ return 0, fmt.Errorf("unexpected size in the answer of git-cat-file(1): %q", strings.TrimSpace(line))+ }+ return size, nil+}++// landCheckMaterialized reads dir back and compares it with the entries that were written: every entry+// is there with its type, its executable bit, the length of its file or the target of its link; a+// gitlink is an empty directory; and nothing else is there but the directories that hold the+// entries. A difference is an error.+func landCheckMaterialized(entries []landTreeEntry, dir string) error {+ want := make(map[string]*landTreeEntry, len(entries))+ parents := map[string]bool{}+ for i := range entries {+ e := &entries[i]+ want[e.path] = e+ for p := path.Dir(e.path); p != "."; p = path.Dir(p) {+ parents[p] = true+ }+ }+ seen := 0+ err := filepath.WalkDir(dir, func(full string, d fs.DirEntry, err error) error {+ if err != nil {+ return err+ }+ rel, err := filepath.Rel(dir, full)+ if err != nil {+ return err+ }+ if rel == "." {+ return nil+ }+ name := filepath.ToSlash(rel)+ info, err := d.Info()+ if err != nil {+ return err+ }+ e, isEntry := want[name]+ switch {+ case isEntry && e.isGitlink():+ if !info.IsDir() {+ return fmt.Errorf("the gitlink %q is not a directory", name)+ }+ case isEntry && e.isSymlink():+ got, err := os.Readlink(full)+ if err != nil || info.Mode()&os.ModeSymlink == 0 || got != e.target {+ return fmt.Errorf("the symbolic link %q does not point to %q", name, e.target)}- links[name] = true+ case isEntry && e.isRegular():+ if !info.Mode().IsRegular() || info.Size() != e.size || (info.Mode().Perm()&0o111 != 0) != (e.mode&0o111 != 0) {+ return fmt.Errorf("the file %q is not the file of the tree (size %d, executable %v)", name, e.size, e.mode&0o111 != 0)+ }+ case isEntry:+ return fmt.Errorf("unexpected entry %q", name)+ case d.IsDir() && parents[name]:+ return nildefault:- return fmt.Errorf("archive entry %q has the unexpected type %q", h.Name, h.Typeflag)+ return fmt.Errorf("%q is in the checkout and not in the tree", name)}+ seen+++ return nil+ })+ if err != nil {+ return fmt.Errorf("the checkout differs from the tree: %w", err)}+ if seen != len(entries) {+ return fmt.Errorf("the checkout differs from the tree: %d of %d entries are there", seen, len(entries))+ }+ return nil}// landBoundedBuffer keeps the first limit bytes written to it and drops the rest, so that adiff --git a/internal/gitaly/service/operations/land_verify_test.go b/internal/gitaly/service/operations/land_verify_test.goindex 360558c..9fb7d09 100644--- a/internal/gitaly/service/operations/land_verify_test.go+++ b/internal/gitaly/service/operations/land_verify_test.go@@ -2,15 +2,24 @@ package operationsimport ("archive/tar"+ "bufio""bytes"+ "fmt"+ "io""os""path/filepath"+ "strings""testing""github.com/stretchr/testify/require""github.com/feder-cr/git-ai/internal/git"+ "github.com/feder-cr/git-ai/internal/git/gittest"+ "github.com/feder-cr/git-ai/internal/git/localrepo""github.com/feder-cr/git-ai/internal/gitaly/config"+ "github.com/feder-cr/git-ai/internal/graft/resolver"+ "github.com/feder-cr/git-ai/internal/testhelper"+ "github.com/feder-cr/git-ai/internal/testhelper/testcfg""github.com/feder-cr/git-ai/proto/go/gitalypb")@@ -92,45 +101,315 @@ func TestUserLand_verify(t *testing.T) {})}-func TestLandExtractTar(t *testing.T) {+// treeFixture is a repository and the tree in it that the tests of the checkout write out: files that+// the attributes of git-archive(1) would drop or change, modes, links, a gitlink, odd names, binary+// content.+type treeFixture struct {+ cfg config.Cfg+ repoPath string+ repo *localrepo.Repo+ tree git.ObjectID+}++func newTreeFixture(t *testing.T) treeFixture {+ t.Helper()++ ctx := testhelper.Context(t)+ cfg := testcfg.Build(t)+ repoProto, repoPath := gittest.CreateRepository(t, ctx, cfg, gittest.CreateRepositoryConfig{SkipCreationViaService: true})+ commit := gittest.WriteCommit(t, cfg, repoPath, gittest.WithBranch("main"))+ entries := []gittest.TreeEntry{+ {Mode: "100644", Path: ".gitattributes", Content: "export-ignore.log export-ignore\nsubst.txt export-subst\ncrlf.txt text eol=crlf\n*.bin -text\n"},+ {Mode: "100644", Path: "export-ignore.log", Content: "hidden from git archive\n"},+ {Mode: "100644", Path: "subst.txt", Content: "$Format:%H$\n"},+ {Mode: "100644", Path: "crlf.txt", Content: "one\ntwo\n"},+ {Mode: "100644", Path: "bin.bin", Content: "\x00\x01\xff\xfe\r\n\x00"},+ {Mode: "100644", Path: "empty", Content: ""},+ {Mode: "100755", Path: "run.sh", Content: "#!/bin/sh\nexit 0\n"},+ {Mode: "120000", Path: "link", Content: "../../etc/passwd"},+ {Mode: "100644", Path: "dir/sub dir/file with space.txt", Content: "space\n"},+ {Mode: "100644", Path: "tab\tname.txt", Content: "tab\n"},+ {Mode: "100644", Path: "line\nbreak.txt", Content: "line\n"},+ {Mode: "100644", Path: "é/日本語.txt", Content: "unicode\n"},+ {Mode: "160000", Path: "sub", OID: commit},+ }+ tree := gittest.WriteTree(t, cfg, repoPath, nestTreeEntries(t, cfg, repoPath, entries))+ return treeFixture{cfg: cfg, repoPath: repoPath, repo: localrepo.NewTestRepo(t, cfg, repoProto), tree: tree}+}++// TestLandCheckoutTree: the checkout is the tree byte for byte, whatever the attributes of the tree say+// (PLAN10 15.9). git-archive(1), which it replaces, drops the file marked export-ignore, and would expand+// export-subst for a commit: a .gitattributes of an agent, or of a model, would have decided what the operator's+// command verifies.+func TestLandCheckoutTree(t *testing.T) {t.Parallel()- archive := func(entries ...*tar.Header) *bytes.Buffer {- var b bytes.Buffer- w := tar.NewWriter(&b)- for _, h := range entries {- require.NoError(t, w.WriteHeader(h))- if h.Typeflag == tar.TypeReg {- _, err := w.Write(make([]byte, h.Size))- require.NoError(t, err)- }+ fx := newTreeFixture(t)+ ctx := testhelper.Context(t)++ // The repro: what git archive makes of this very tree.+ archived := map[string]string{}+ tr := tar.NewReader(bytes.NewReader(gittest.Exec(t, fx.cfg, "-C", fx.repoPath, "archive", "--format=tar", fx.tree.String())))+ for {+ h, err := tr.Next()+ if err != nil {+ break+ }+ if h.Typeflag == tar.TypeReg {+ content, err := io.ReadAll(tr)+ require.NoError(t, err)+ archived[h.Name] = string(content)}- require.NoError(t, w.Close())- return &b- }- file := func(name string) *tar.Header {- return &tar.Header{Name: name, Typeflag: tar.TypeReg, Mode: 0o644, Size: 1}}+ require.NotContains(t, archived, "export-ignore.log", "git archive drops the file marked export-ignore")+ // export-subst expands only when a commit is archived, and a tree is: so it changed nothing of what+ // the verification saw, and the checkout still applies neither.+ require.Equal(t, "$Format:%H$\n", archived["subst.txt"])dir := t.TempDir()- require.NoError(t, landExtractTar(archive(- &tar.Header{Name: "d/", Typeflag: tar.TypeDir, Mode: 0o755},- file("d/f"),- &tar.Header{Name: "x", Typeflag: tar.TypeReg, Mode: 0o755, Size: 1},- &tar.Header{Name: "l", Typeflag: tar.TypeSymlink, Linkname: "/etc"},- ), dir))- info, err := os.Stat(filepath.Join(dir, "x"))+ require.NoError(t, landCheckoutTree(ctx, fx.repo, fx.tree, dir))++ for path, want := range map[string]string{+ ".gitattributes": "export-ignore.log export-ignore\nsubst.txt export-subst\ncrlf.txt text eol=crlf\n*.bin -text\n",+ "export-ignore.log": "hidden from git archive\n",+ "subst.txt": "$Format:%H$\n",+ "crlf.txt": "one\ntwo\n",+ "bin.bin": "\x00\x01\xff\xfe\r\n\x00",+ "empty": "",+ "run.sh": "#!/bin/sh\nexit 0\n",+ "dir/sub dir/file with space.txt": "space\n",+ "tab\tname.txt": "tab\n",+ "line\nbreak.txt": "line\n",+ "é/日本語.txt": "unicode\n",+ } {+ got, err := os.ReadFile(filepath.Join(dir, filepath.FromSlash(path)))+ require.NoError(t, err, path)+ require.Equal(t, want, string(got), path)+ }+ info, err := os.Stat(filepath.Join(dir, "run.sh"))require.NoError(t, err)require.Equal(t, os.FileMode(0o755), info.Mode().Perm())- target, err := os.Readlink(filepath.Join(dir, "l"))+ info, err = os.Stat(filepath.Join(dir, "crlf.txt"))+ require.NoError(t, err)+ require.Equal(t, os.FileMode(0o644), info.Mode().Perm())++ target, err := os.Readlink(filepath.Join(dir, "link"))+ require.NoError(t, err)+ require.Equal(t, "../../etc/passwd", target, "a link is written as a link, and not followed")++ gitlink, err := os.Stat(filepath.Join(dir, "sub"))+ require.NoError(t, err)+ require.True(t, gitlink.IsDir())+ children, err := os.ReadDir(filepath.Join(dir, "sub"))require.NoError(t, err)- require.Equal(t, "/etc", target)+ require.Empty(t, children, "a gitlink is an empty directory")+}++// TestLandCheckoutTree_theCheckIsPartOfIt: a checkout that does not equal the tree fails, closed, through+// landCheckoutTree itself, whatever the damage and wherever it came from.+func TestLandCheckoutTree_theCheckIsPartOfIt(t *testing.T) {+ t.Parallel()- for name, a := range map[string]*bytes.Buffer{- "under a link": archive(&tar.Header{Name: "l", Typeflag: tar.TypeSymlink, Linkname: "/tmp"}, file("l/x")),- "leaves the dir": archive(file("../x")),- "absolute": archive(file("/x")),+ fx := newTreeFixture(t)+ ctx := testhelper.Context(t)++ require.NoError(t, landCheckoutTreeThen(ctx, fx.repo, fx.tree, t.TempDir(), func() {}), "without damage it passes")+ for name, damage := range map[string]func(dir string){+ "a file written in": func(dir string) { _ = os.WriteFile(filepath.Join(dir, "planted"), []byte("x"), 0o644) },+ "a file of the tree gone": func(dir string) {+ _ = os.Remove(filepath.Join(dir, "export-ignore.log"))+ },+ "a file rewritten": func(dir string) {+ _ = os.WriteFile(filepath.Join(dir, "subst.txt"), []byte("expanded content, longer than the tree has\n"), 0o644)+ },} {- require.Error(t, landExtractTar(a, t.TempDir()), name)+ dir := t.TempDir()+ err := landCheckoutTreeThen(ctx, fx.repo, fx.tree, dir, func() { damage(dir) })+ require.ErrorContains(t, err, "the checkout differs from the tree", name)+ }+}++// batchOf is the answer of git-cat-file(1) --batch to the blobs of entries, gitlinks left out.+func batchOf(contents map[string]string, entries []landTreeEntry) *bufio.Reader {+ var b bytes.Buffer+ for _, e := range entries {+ if e.isGitlink() {+ continue+ }+ c := contents[e.path]+ fmt.Fprintf(&b, "%s blob %d\n%s\n", e.oid, len(c), c)+ }+ return bufio.NewReader(&b)+}++func fakeEntries() ([]landTreeEntry, map[string]string) {+ oid := func(c string) git.ObjectID { return git.ObjectID(strings.Repeat(c, 40)) }+ return []landTreeEntry{+ {mode: 0o100644, oid: oid("1"), path: "a.txt"},+ {mode: 0o100755, oid: oid("2"), path: "bin/run"},+ {mode: 0o120000, oid: oid("3"), path: "link"},+ {mode: 0o160000, oid: oid("4"), path: "mod"},+ }, map[string]string{+ "a.txt": "content\n",+ "bin/run": "#!/bin/sh\n",+ "link": "a.txt",+ }+}++// TestLandMaterialize_refusesWhatLeavesTheDirectory: a path that is not clean, that is absolute or that+// goes up, and an entry under a symbolic link that was written, are refused.+func TestLandMaterialize_refusesWhatLeavesTheDirectory(t *testing.T) {+ t.Parallel()++ oid := git.ObjectID(strings.Repeat("a", 40))+ for name, entries := range map[string][]landTreeEntry{+ "up": {{mode: 0o100644, oid: oid, path: "../x"}},+ "absolute": {{mode: 0o100644, oid: oid, path: "/x"}},+ "not clean": {{mode: 0o100644, oid: oid, path: "a/../x"}},+ "a dot": {{mode: 0o100644, oid: oid, path: "./x"}},+ "empty": {{mode: 0o100644, oid: oid, path: ""}},+ "under a link": {{mode: 0o120000, oid: oid, path: "l"}, {mode: 0o100644, oid: oid, path: "l/x"}},+ } {+ contents := map[string]string{"../x": "x", "/x": "x", "a/../x": "x", "./x": "x", "": "x", "l": "/tmp", "l/x": "x"}+ require.Error(t, landMaterialize(entries, batchOf(contents, entries), t.TempDir()), name)+ }+}++// TestLandMaterialize_refusesAnswersItDidNotAskFor: the header of each blob must be the object asked for.+func TestLandMaterialize_refusesAnswersItDidNotAskFor(t *testing.T) {+ t.Parallel()++ entries, contents := fakeEntries()+ good := batchOf(contents, entries)+ require.NoError(t, landMaterialize(entries, good, t.TempDir()))++ for name, answer := range map[string]string{+ "another object": strings.Repeat("9", 40) + " blob 3\nabc\n",+ "a tree": entries[0].oid.String() + " tree 3\nabc\n",+ "missing": entries[0].oid.String() + " missing\n",+ "a negative size": entries[0].oid.String() + " blob -1\n",+ "a short blob": entries[0].oid.String() + " blob 30\nabc\n",+ "no line feed behind": entries[0].oid.String() + " blob 3\nabcX",+ } {+ fresh, _ := fakeEntries()+ require.Error(t, landMaterialize(fresh, bufio.NewReader(strings.NewReader(answer)), t.TempDir()), name)+ }+}++// TestLandCheckMaterialized_failsClosed: the checkout is read back and compared with the listing; any+// difference is an error.+func TestLandCheckMaterialized_failsClosed(t *testing.T) {+ t.Parallel()++ materialized := func(t *testing.T) ([]landTreeEntry, string) {+ t.Helper()+ entries, contents := fakeEntries()+ dir := t.TempDir()+ require.NoError(t, landMaterialize(entries, batchOf(contents, entries), dir))+ require.NoError(t, landCheckMaterialized(entries, dir), "an exact checkout passes")+ return entries, dir+ }++ for name, damage := range map[string]func(t *testing.T, dir string){+ "an extra file": func(t *testing.T, dir string) {+ require.NoError(t, os.WriteFile(filepath.Join(dir, "extra"), nil, 0o644))+ },+ "an extra empty directory": func(t *testing.T, dir string) {+ require.NoError(t, os.Mkdir(filepath.Join(dir, "extra"), 0o755))+ },+ "a file in a gitlink": func(t *testing.T, dir string) {+ require.NoError(t, os.WriteFile(filepath.Join(dir, "mod", "x"), nil, 0o644))+ },+ "a missing file": func(t *testing.T, dir string) {+ require.NoError(t, os.Remove(filepath.Join(dir, "a.txt")))+ },+ "a changed length": func(t *testing.T, dir string) {+ require.NoError(t, os.WriteFile(filepath.Join(dir, "a.txt"), []byte("longer than before\n"), 0o644))+ },+ "an executable file that is not": func(t *testing.T, dir string) {+ require.NoError(t, os.Chmod(filepath.Join(dir, "a.txt"), 0o755))+ },+ "a file that lost its executable bit": func(t *testing.T, dir string) {+ require.NoError(t, os.Chmod(filepath.Join(dir, "bin", "run"), 0o644))+ },+ "another target of a link": func(t *testing.T, dir string) {+ require.NoError(t, os.Remove(filepath.Join(dir, "link")))+ require.NoError(t, os.Symlink("elsewhere", filepath.Join(dir, "link")))+ },+ "a link that became a file": func(t *testing.T, dir string) {+ require.NoError(t, os.Remove(filepath.Join(dir, "link")))+ require.NoError(t, os.WriteFile(filepath.Join(dir, "link"), []byte("a.txt"), 0o644))+ },+ "a gitlink that became a file": func(t *testing.T, dir string) {+ require.NoError(t, os.Remove(filepath.Join(dir, "mod")))+ require.NoError(t, os.WriteFile(filepath.Join(dir, "mod"), nil, 0o644))+ },+ } {+ entries, dir := materialized(t)+ damage(t, dir)+ require.ErrorContains(t, landCheckMaterialized(entries, dir), "the checkout differs from the tree", name)+ }+}++func TestLandParseTreeListing(t *testing.T) {+ t.Parallel()++ oid := strings.Repeat("a", 40)+ entries, err := landParseTreeListing([]byte(+ "100644 blob " + oid + "\tplain\x00" ++ "100755 blob " + oid + "\twith space/and\ttab\nand line break\x00" ++ "120000 blob " + oid + "\tlink\x00" ++ "160000 commit " + oid + "\tmod\x00"))+ require.NoError(t, err)+ require.Len(t, entries, 4)+ require.Equal(t, "with space/and\ttab\nand line break", entries[1].path)+ require.True(t, entries[1].isRegular())+ require.True(t, entries[2].isSymlink())+ require.True(t, entries[3].isGitlink())++ for name, raw := range map[string]string{+ "no tab": "100644 blob " + oid + " plain\x00",+ "a tree": "040000 tree " + oid + "\tdir\x00",+ "a blob that is a gitlink": "160000 blob " + oid + "\tmod\x00",+ "a commit that is a file": "100644 commit " + oid + "\tx\x00",+ "a mode nobody has": "777777 blob " + oid + "\tx\x00",+ "a mode that is no number": "10064z blob " + oid + "\tx\x00",+ "too few fields": "100644 blob\tx\x00",+ } {+ _, err := landParseTreeListing([]byte(raw))+ require.Error(t, err, name)+ }+}++// TestUserLand_verifySeesTheWholeTree: the operator's command is run on the tree that lands, and a+// .gitattributes in it, written by an agent or by a model, hides nothing from it: the file marked+// export-ignore is there, and the file marked export-subst has its own content. A verification that+// did not see them would pass a tree it had not seen.+func TestUserLand_verifySeesTheWholeTree(t *testing.T) {+ t.Parallel()++ for desc, answer := range map[string]resolver.Response{"a model's resolution": modelAnswer(), "a rules' resolution": validAnswer()} {+ t.Run(desc, func(t *testing.T) {+ t.Parallel()++ dir, argv := resolverScript(t, "")+ writeAnswer(t, dir, answer)+ attributes := []gittest.TreeEntry{+ {Mode: "100644", Path: ".gitattributes", Content: "hidden.txt export-ignore\nsubst.txt export-subst\n"},+ {Mode: "100644", Path: "hidden.txt", Content: "must be seen\n"},+ {Mode: "100644", Path: "subst.txt", Content: "$Format:%H$\n"},+ }+ verify := verifyScript(t, `test -f .gitattributes && grep -qx 'must be seen' hidden.txt && grep -qxF '$Format:%H$' subst.txt && grep -qx merged-a a`)+ fx := newResolveFixtureWith(t, withVerify(enabledResolver(argv), verify, ""),+ []gittest.TreeEntry{{Mode: "100644", Path: "a", Content: "base\n"}, {Mode: "100644", Path: "b", Content: "base\n"}},+ []gittest.TreeEntry{{Mode: "100644", Path: "a", Content: "ours\n"}, {Mode: "100644", Path: "b", Content: "ours\n"}},+ append([]gittest.TreeEntry{{Mode: "100644", Path: "a", Content: "theirs\n"}, {Mode: "100644", Path: "b", Content: "theirs\n"}}, attributes...),+ )++ resp, err := fx.client.UserLand(fx.ctx, fx.request("Land"))+ require.NoError(t, err, "the verification saw the hidden file and the unexpanded content")+ landed := git.ObjectID(resp.GetLandedCommitId())+ require.Equal(t, "must be seen\n", fx.blobAt(t, landed, "hidden.txt"))+ })}}diff --git a/internal/graft/resolver/protocol.go b/internal/graft/resolver/protocol.goindex 1fd5dc2..6ff6f82 100644--- a/internal/graft/resolver/protocol.go+++ b/internal/graft/resolver/protocol.go@@ -204,14 +204,35 @@ type Response struct {Explanation string `json:"explanation"`}+// The origins of a resolved file: what made the resolution.+const (+ // OriginRule: an operator's rule (trivial, union, rerere, command) made it. Only a file marked+ // as a rule lands without the operator's verification of the tree.+ OriginRule = "rule"+ // OriginModel: a model made it. It lands only if verify.command passed on the tree that lands+ // (PLAN10 14, 15.9), and is otherwise proposed to the agent.+ OriginModel = "model"+)+// ResponseFile is the resolution of one conflicting path. Content is base64-// on the wire and is only meaningful for ActionWrite. Proposal marks a-// resolution made by a model: the server lands it only if verify.command passed-// on the tree that lands, and otherwise proposes it to the agent (PLAN10 14, 15.9).+// on the wire and is only meaningful for ActionWrite.+//+// Origin says what made the resolution: OriginRule or OriginModel. It is fail-safe: a file is a+// rule's only when it says so, and a file without an origin, or with one that is none of the two,+// is a model's, which lands only through the operator's verification. Proposal is the older mark of+// a model's resolution, still read: it means OriginModel, whatever Origin says. The name of the+// resolver decides nothing.type ResponseFile struct {Path string `json:"path"`Action string `json:"action"`Mode string `json:"mode,omitempty"`Content []byte `json:"content,omitempty"`+ Origin string `json:"origin,omitempty"`Proposal bool `json:"proposal,omitempty"`}++// FromModel reports whether the file is a model's resolution: anything that is not marked as a+// rule's, and anything marked as a proposal.+func (f ResponseFile) FromModel() bool {+ return f.Proposal || f.Origin != OriginRule+}diff --git a/internal/graft/resolver/testdata/protocol-fields.json b/internal/graft/resolver/testdata/protocol-fields.jsonindex eeb08ff..039833e 100644--- a/internal/graft/resolver/testdata/protocol-fields.json+++ b/internal/graft/resolver/testdata/protocol-fields.json@@ -7,7 +7,7 @@"side": ["content", "mode", "oid"],"limits": ["max_file_bytes"],"response": ["explanation", "files", "resolver", "version"],- "response_file": ["action", "content", "mode", "path", "proposal"],+ "response_file": ["action", "content", "mode", "origin", "path", "proposal"],"error_response": ["error", "files"],"file_outcome": ["outcome", "path", "reason"],"check_response": ["files", "operation", "resolver", "version"],diff --git a/tools/graft-agents/protocol/protocol.go b/tools/graft-agents/protocol/protocol.goindex 169bd95..4bfae7b 100644--- a/tools/graft-agents/protocol/protocol.go+++ b/tools/graft-agents/protocol/protocol.go@@ -151,12 +151,21 @@ type ResolvedFile struct {Action string `json:"action"`Mode string `json:"mode,omitempty"`Content []byte `json:"content"`- // Proposal marks a resolution made by a model (the llm strategy): the server lands it only if- // verify.command passed on the tree that lands, and otherwise proposes it to the agent- // (PLAN10 14, 15.9).+ // Origin says what made the resolution: OriginRule for an operator's rule (trivial, union,+ // rerere, command), OriginModel for a model (the llm strategy). The server lands a resolution+ // without the operator's verification only if it is marked as a rule's: a file without an origin+ // is a model's (PLAN10 14, 15.9).+ Origin string `json:"origin,omitempty"`+ // Proposal is the older mark of a model's resolution, still sent with OriginModel.Proposal bool `json:"proposal,omitempty"`}+// The origins of a resolved file (see ResolvedFile.Origin).+const (+ OriginRule = "rule"+ OriginModel = "model"+)+// ErrorResponse is the body of an HTTP 422 (unresolvable) or other error.type ErrorResponse struct {Error string `json:"error"`diff --git a/tools/graft-agents/resolve/resolve.go b/tools/graft-agents/resolve/resolve.goindex e6dbcdf..6f5db6c 100644--- a/tools/graft-agents/resolve/resolve.go+++ b/tools/graft-agents/resolve/resolve.go@@ -458,11 +458,16 @@ func (r *Resolver) resolveFile(ctx context.Context, req protocol.Request, f Fileif !proposal {r.record(ctx, f, res)}+ origin := protocol.OriginRule+ if proposal {+ origin = protocol.OriginModel+ }return protocol.ResolvedFile{Path: f.Path,Action: res.Action,Mode: modeOf(res),Content: contentOf(res),+ Origin: origin,Proposal: proposal,}, b.Name, nil, nil}diff --git a/tools/graft-agents/resolve/resolve_test.go b/tools/graft-agents/resolve/resolve_test.goindex b48a916..a018cfb 100644--- a/tools/graft-agents/resolve/resolve_test.go+++ b/tools/graft-agents/resolve/resolve_test.go@@ -597,8 +597,8 @@ func (*proposingStrategy) proposes() {}var _ proposer = (*llmStrategy)(nil)// TestProposalsAreMarkedAndNotRecorded: a model's answer is marked as a proposal, which the server-// never lands, and no Recorder learns it, so that it can never come back as a rule's answer; a-// rule's answer is not marked, and is recorded.+// lands only after verification, and no Recorder learns it, so that it can never come back as a rule's answer; a+// rule's answer is marked as a rule's, and is recorded.func TestProposalsAreMarkedAndNotRecorded(t *testing.T) {model := &proposingStrategy{fakeStrategy{fn: answer(protocol.ModeFile, "proposed\n")}}rec := &recordingStrategy{fakeStrategy: fakeStrategy{fn: decline}}@@ -607,8 +607,8 @@ func TestProposalsAreMarkedAndNotRecorded(t *testing.T) {if err != nil {t.Fatal(err)}- if len(resp.Files) != 1 || !resp.Files[0].Proposal {- t.Fatalf("files %+v, want one proposal", resp.Files)+ if len(resp.Files) != 1 || !resp.Files[0].Proposal || resp.Files[0].Origin != protocol.OriginModel {+ t.Fatalf("files %+v, want one proposal of origin %q", resp.Files, protocol.OriginModel)}if len(rec.recorded) != 0 {t.Fatalf("a proposal was recorded: %+v", rec.recorded)@@ -623,4 +623,8 @@ func TestProposalsAreMarkedAndNotRecorded(t *testing.T) {if resp.Files[0].Proposal || len(rule.recorded) != 1 {t.Fatalf("rule answer: proposal %v, recorded %d", resp.Files[0].Proposal, len(rule.recorded))}+ // The server lands a file without the operator's verification only if it says it is a rule's.+ if resp.Files[0].Origin != protocol.OriginRule {+ t.Fatalf("rule answer: origin %q, want %q", resp.Files[0].Origin, protocol.OriginRule)+ }}