Git designed for AI agents.
graft: a model's resolution lands only through the operator's verification (PLAN10 14, 15.9)
Federico Eliaf404a713fda0
diff --git a/cmd/graft/conflicts.go b/cmd/graft/conflicts.goindex c44c8e0..84121d2 100644--- a/cmd/graft/conflicts.go+++ b/cmd/graft/conflicts.go@@ -30,7 +30,8 @@ import (// DIR/sides/ours/<path>// DIR/sides/theirs/<path>// DIR/proposals/<path> the resolver's proposal for the path, when it made one (a model's-// resolution, which the server never lands, or the rules' of a land+// resolution, which did not land because no verification passed on+// it, or the rules' of a land// that failed for another file); copy it over DIR/files/<path> to// land it as yours// DIR/conflict.json the conflict packagediff --git a/doc/graft/CONFIGURATION.md b/doc/graft/CONFIGURATION.mdindex 5369d98..2328196 100644--- a/doc/graft/CONFIGURATION.md+++ b/doc/graft/CONFIGURATION.md@@ -193,11 +193,12 @@ cause.| `command` | array of strings | empty | Program and arguments. It reads one request on stdin and writes the answer on stdout. || `url` | string | empty | `http://` or `https://` endpoint that receives the request as a JSON POST. Exactly one of `command` and `url` must be set when `enabled` is true. || `token_file` | string | empty | For `url`: file with the bearer token, read at startup. |-| `timeout` | string | `"120s"` | Bound on one resolver call (Go duration), and on the whole land that the server resolves for `resolve_conflicts`: its rounds (the resolutions, the waits for a resolve slot, and the passes through the branch's lane) end at this deadline. A round that the lane hands back after it fails the land with the merge conflict, "files kept changing", naming the files. |+| `timeout` | string | `"120s"` | Bound on one resolver call (Go duration), and on the whole land that the server resolves for `resolve_conflicts`: its rounds (the resolutions, the verifications of a model's resolution, the waits for slots, and the passes through the branch's lane) end at this deadline, which is `timeout` plus `verify.timeout` when `verify.enabled`. A round that the lane hands back after it fails the land with the merge conflict, "files kept changing", naming the files. || `max_files` | int | `20` | Conflicted files per land. A land with more is not sent to the resolver. || `max_file_bytes` | int | `1048576` (1 MiB) | Bound on every side of every file sent and on every resolved file accepted. At most 12566528 (about 12 MiB): one file, its three sides and its resolution, must fit one check request. || `proposal_ttl` | string | `"168h"` | How long the proposals of a failed land are kept under `refs/graft/proposals/` (Go duration, by the server's clock); at most 20 are kept per branch. Lands with a resolution and the housekeeping pass prune them. || `check_concurrency` | int | `2` | Checks of agents' resolutions that run at once in the Gitaly process, across all its listeners (one resolver client is shared by them); each holds a request of up to 64 MiB. A check waits for a free slot within its deadline (`timeout`), and fails closed when it expires. The wait is in `graft_resolver_check_slot_wait_seconds`. |+| `verify_concurrency` | int | `2` | Verifications of a model's resolution that run at once in the Gitaly process, out of the lane, in a pool of their own, as `verify.command` takes minutes. A verification waits for a free slot within the land's deadline. The wait is in `graft_resolver_verify_slot_wait_seconds`. || `resolve_concurrency` | int | `2` | Resolutions that run at once in the Gitaly process, in a pool apart from the checks', so that long resolutions never delay a check. A resolution waits for a free slot within the land's deadline (`timeout`), and fails closed when it expires. The wait is in `graft_resolver_resolve_slot_wait_seconds`. || `resolve_rounds` | int | `32` | Safety cap on the rounds of a land whose conflicting files keep changing while it is resolved out of the lane. The rounds are bounded by `timeout`; the cap is set well above realistic contention and is no contract. || `paths_allow` | array of strings | empty (every path) | Globs of paths that may be sent to the resolver, with the syntax of policy refs. |@@ -208,9 +209,9 @@ cause.| `minified.globs` | array of strings | empty | Base-name globs of minified files, besides the built-in `*.min.js`, `*.min.mjs`, `*.min.css` and `*.bundle.js`. Such files are never sent. || `minified.average_line_bytes` | int | `300` | A side larger than 2 KiB whose average line is longer is minified. || `minified.longest_line_bytes` | int | `5000` | A side larger than 2 KiB with a longer line is minified. |-| `verify.enabled` | bool | `false` | Run `verify.command` on the merged tree of every resolution (a resolver's or an agent's) before it lands; it applies even when `enabled` is false. |+| `verify.enabled` | bool | `false` | Run `verify.command` on the merged tree of every resolution (a resolver's or an agent's) before it lands; it applies even when `enabled` is false. It is also what lets a model's resolution land: only a tree it passed on lands, and without it a model's resolution is only ever a proposal. || `verify.command` | array of strings | empty | Program and arguments, run in a temporary checkout of the merged tree with `PATH`, `HOME` and `TMPDIR` only. A non-zero exit rejects the resolution. |-| `verify.timeout` | string | `"5m"` | Bound on one run (Go duration); a run that does not finish rejects the resolution. |+| `verify.timeout` | string | `"5m"` | Bound on one run (Go duration); a run that does not finish rejects the resolution. When `verify.enabled`, it is added to `timeout` for the deadline of a resolved land. |A file that is never sent goes to the agent with its reason in the conflictpackage: large (a side over `max_file_bytes`), binary (a NUL byte), lockfile,diff --git a/doc/graft/PLAN10.md b/doc/graft/PLAN10.mdindex 4036aa3..68d637b 100644--- a/doc/graft/PLAN10.md+++ b/doc/graft/PLAN10.md@@ -730,9 +730,30 @@ whole series under $5 real, and a run over $1.00 real stops the series until the## 14. The agent always resolves; the server's AI only proposes (Federico, PM-approved design)-This supersedes "AI lands with trace only". The server never lands a resolution of the llm strategy-by itself: it is a proposal, which the agent reviews, edits or replaces, and lands with its own-resolution (UserLandRequest.resolution, Graft-Resolved-By: agent/<actor>). The deterministic+**Final ruling (Federico, 2026-10-19, relayed by the PM; it supersedes the amendment at the end of+this section and every statement below that the server never lands a model's resolution).** An AI+resolution lands only through the operator's verification (15.9).++* Server AI configured and verify.command configured: the server resolves. If every deterministic+ check passes and verify.command passes on the merged tree, it LANDS, signed Graft-Resolved-By:+ <the resolver> and Graft-Checked-By: verify:<command>. Otherwise it goes to the agent as a+ PROPOSAL, with the verify output when verify failed.+* Server AI configured, no verify.command: proposal only, and the agent decides. An unverified AI+ resolution never lands.+* No AI: the agent resolves.+* Rules (trivial, union, rerere, command) are unchanged: they land after verify.command when one is+ configured, in the lane, as today.+* The composition report (15.3) stays a report, not a gate.+* One rule, in decideLand: a file that proposes may land only if verify.command ran and passed on+ the exact tree that lands, compared by tree object ID. No other code path lands a model's+ resolution.+* The trailer names what ran: Graft-Checked-By: verify:<command>. The syntax and operator checks+ run inside the resolver and are not separate trailers; the composition report is not built, so+ the trailer does not claim it.++This supersedes "AI lands with trace only". Before the ruling above, the server never landed a+resolution of the llm strategy by itself: it was a proposal, which the agent reviews, edits or+replaces, and lands with its own resolution (UserLandRequest.resolution, Graft-Resolved-By: agent/<actor>). The deterministicstrategies that the operator configures (trivial, union, rerere, command) are rules, not AI: aresolution made by rules alone still lands. The llm auto-land path is removed, not kept behind aflag.@@ -835,7 +856,8 @@ flag.resolve_conflicts, takes seconds. Moving both out of the lane, like the precheck of an agent'sresolution, is a step of its own.-* **Amendment: Federico's product decision (2026-10-19, relayed by the PM).** "When the server AI+* **Amendment: Federico's product decision (2026-10-19, relayed by the PM). Superseded by the final+ ruling at the top of this section: the gate is verify.command, not the composition check.** "When the server AIis configured, the server always tries to resolve, and it LANDS its resolution by itself ONLY ifthe proposal passes every deterministic check: syntax, anchors, no markers, and the compositioncheck from §15 (keeps both changes, no duplicated both-added lines). There is still no@@ -880,10 +902,12 @@ not exist yet; (not enabled) marks what is designed and measured but switched of| agent may not update the branch (policy) -> PermissionDenied| the conflicting files not yet resolved -> one VERDICT per file| (15.2), each kept by its key (path, base, ours, theirs, modes),- | resolve slots; decideLand (15.4):+ | resolve slots; a model's answer: verify.command runs on the exact+ | tree that would land, verify slots (15.9); decideLand (15.4):| not landable -> proposals kept (14) and PACKAGE now: the lane- | is not entered- | landable ----> the verdicts go to the lane+ | is not entered; a model's files are PROPOSED,+ | with the verify output when verify failed+ | landable ----> the verdicts, and the verified tree, go to the lanevLANE (one per branch)|-- a resolution -----> landWithResolution: tip moved -> refuse, fresh PACKAGE@@ -894,16 +918,28 @@ not exist yet; (not enabled) marks what is designed and measured but switched of3-way merge (rebase or merge) of the tip and the change|-- clean -------------------------------------------------------------> LAND|-- conflicts, no resolve_conflicts ------------------------------------> merge_conflict- |-- conflicts, every conflicting file has a landable verdict with its key, and the lane's+ |-- conflicts, every conflicting file has a verdict with its key, and the lane's| classification of the file agrees with the verdict's- | -> operator verify.command; passes -> merge commit signed by the rules -> LAND- | fails -> PACKAGE (every file REJECTED, reason)+ | rules only -> operator verify.command (in the lane, for now); passes -> merge commit+ | signed by the rules -> LAND; fails -> PACKAGE (every file REJECTED, reason)+ | a model's file -> decideLand (15.4) on the tree the lane would write: it LANDS only if+ | that tree is the tree verify.command passed on (tree object IDs). The lane+ | runs no verification of a model's resolution. Merge commit signed+ | Graft-Resolved-By: <resolver> + Graft-Checked-By: verify:<command>.+ | Another tree (the tip moved) -> NEEDS VERIFICATION, below|-- conflicts, a file without such a verdict (new, or its sides changed)- -> NEEDS RESOLUTION, out of the lane: PRE-RESOLVE only those files against the new tip,- keeping the other verdicts, and enter the lane again; at most resolve_rounds rounds- (default 3) within the resolver's timeout, then merge_conflict, "files kept- changing: a, b"- The resolver is never called inside the lane.+ | -> NEEDS RESOLUTION, out of the lane: PRE-RESOLVE only those files against the new tip,+ | keeping the other verdicts, and enter the lane again+ |-- a model's answer whose tree is not the verified one (the tip moved)+ -> NEEDS VERIFICATION, out of the lane: rebuild the tree from the same verdicts (the+ resolver is called only for files that changed), VERIFY it, enter the lane again+ The rounds are bounded by the deadline of the land (the resolver's timeout, plus verify.timeout when+ a verify.command is configured) and by resolve_rounds (default 32, a safety cap, no contract). At+ the deadline: files that kept changing -> merge_conflict, "files kept changing: a, b"; a model's+ resolution that was not verified in time -> PACKAGE, the model's files PROPOSED, "could not verify+ before the branch moved again" (a verified AI land is opportunistic: losing the race is harmless,+ the agent gets the proposal).+ The resolver is never called inside the lane, and neither is the verification of a model's resolution.PACKAGE to the agent (merge_conflict detail): sides by OID, hunks, per-file outcome + reason,per-file proposal (+ its report, planned), the proposals kept under refs/graft/proposals/ (14),@@ -941,11 +977,16 @@ not exist yet; (not enabled) marks what is designed and measured but switched ofassemble the file; file checks: markers, tree-sitter syntax, operator checks|-- a check fails ----------------------------------------> AGENT (REJECTED, reason)v- composition report (15.3) (planned)+ composition report (15.3) (planned; a report, not a gate)vVERDICT PROPOSAL (PROPOSED, the proposal + its report)- [AI landing when every deterministic check passes: designed by Federico's amendment (14),- NOT ENABLED: the measurement in 15.6 shows no deterministic check that reaches the bar]+ v+ verify.command configured? --no--> stays a PROPOSAL: the agent decides ("no verify.command")+ | yes: the exact tree that would land is verified, out of the lane (15.9)+ |-- fails ------------------------------------------------> stays a PROPOSAL (+ the verify output)+ |-- not verified before the branch moved again -----------> stays a PROPOSAL (+ the reason)+ v+ passes --> LANDS, if it is still the tree to land (decideLand, 15.4), signed Graft-Checked-By: verify:...```### 15.3 Composition report (planned; a report, not a gate)@@ -973,10 +1014,13 @@ not exist yet; (not enabled) marks what is designed and measured but switched of### 15.4 The one decision point```- decideLand, before the lane (the pre-resolution calls it; landAnswerProposes is its guard)- every file resolved by an operator rule ...... LAND (then verify.command)- anything else, including any model answer .... PACKAGE (the model's files are proposals)- No other code path lands a model's resolution. No check disables itself: a configuration that+ decideLand(answer, files, tree, verified): the pre-resolution calls it with the tree it verified,+ the lane with the tree it would write+ every file resolved by an operator rule ...... LAND (then verify.command, in the lane)+ a file a model resolved ...................... LAND only if verify.command ran and passed on THIS+ tree: tree object ID equality, not the tip+ anything else ................................ PACKAGE (the model's files are proposals)+ One rule about a model's resolution. No other code path lands it. No check disables itself: a configuration thatcannot be used stops the server at startup, and a failure at run time refuses the land. Noconfidence that the model declares about itself decides anything.```@@ -991,8 +1035,11 @@ not exist yet; (not enabled) marks what is designed and measured but switched of| LANE, re-land | `landAttempt`, `landingAttempt.decide`, `landWithResolution` | operations/land.go, land_attempt.go, land_reland.go || 3-way merge, listing | `landingAttempt.decide`, `landListConflicts` | operations/land_attempt.go, land_resolve.go || out of the lane: rounds, deadline | `landResolving`, `landPreResolve` | operations/land_preresolve.go |-| per-file verdicts, decide | `landResolveAt`, `decideLand`, `landAnswerProposes`, `landKeyOf` | operations/land_resolve.go, land_proposals.go, land_preresolve.go |+| per-file verdicts, decide | `landResolveAt`, `landSetOutcomes`, `decideLand`, `landAnswerProposes`, `landKeyOf` | operations/land_resolve.go, land_proposals.go, land_preresolve.go |+| the tree that would land | `landResolvedTree` | operations/land_resolve.go |+| verify a model's resolution, out of the lane | `landVerifyOutOfLane`, `landVerify`, `Client.AcquireVerifySlot` | operations/land_verify.go; graft/resolver/check.go || verdicts applied in the lane | `landApplyVerdicts`, `landNeedsResolution` | operations/land_resolve.go, land_preresolve.go |+| unverified in time: proposals | `landUnverified`, `landNotVerifiedReason` | operations/land_preresolve.go || resolver call, bounded slots | `Client.AcquireResolveSlot`, `ResolveSlot.Resolve` | graft/resolver/check.go || policy before resolving | `hook.CheckAgentPolicy`, `landPolicyDeniedError` | operations/land_resolve.go, land_proposals.go || classify | `landClassify`, `resolver.SendRules` | operations/land_resolve.go; graft/resolver |@@ -1041,10 +1088,10 @@ the same base lines supersedes it. Measured with zero model calls:measures overlap with the sides.**Decisions.**-1. The AI stays proposal-only.-2. Federico's amendment (AI landing when every deterministic check passes) stays recorded and is- not enabled: no deterministic check reaches the bar. Pending the Sol-100 result, which Federico- uses to decide.+1. The AI stays proposal-only unless verify.command passes on the tree (the final ruling, 14 and 15.9).+2. Federico's amendment (AI landing when every deterministic check passes) is superseded by the+ final ruling: the gate is the operator's verify.command, because no deterministic check reaches+ the bar.3. The composition report goes with every proposal, honest about what it can say: "keeps everyuncontested change of both sides" or "drops THEIRS' change at base 40-44", "duplicates a lineboth sides added", "changes lines outside the conflicts", "N contested regions resolved freely".@@ -1149,8 +1196,9 @@ The resolution is bound to the files instead (the PM's correction):over two plain semaphores; the memory bound of the process is the sum of the two pools' requests.Each request waits for its slot within the resolver's timeout and fails closed when it expires.* **Still in the lane, stated:** verify.command, the operator's tests on the merged tree, runs in- the lane for the rules' resolutions and for agents' resolutions alike. It is the next candidate- to move out, as its own step.+ the lane for the rules' resolutions and for agents' resolutions alike. For a model's resolution+ it already runs out of the lane (15.9). Moving the other two onto the same mechanism is the next+ step, measured on its own, so that in the end there is one verification mechanism.* **Measured (TestLandResolveBench, GRAFT_LAND_BENCH=1).** Six lands that the server resolves,all on the same file, and six plain lands that conflict with nothing, at once through the lanesof a standalone server, with a resolver that takes 500 ms; the same bench on the tree before@@ -1182,3 +1230,48 @@ The resolution is bound to the files instead (the PM's correction):* A proposal kept while a batch holds the lane has the published tip as parent (review of6090d5b, item 4): proposals are written before the lane only.* The two pools: a held resolve slot does not delay a check.++### 15.9 Design delta: a model's resolution lands through the verification (PM ruling)++The final ruling of 14: an AI resolution lands only through the operator's verification. The+verification runs the operator's tests, which take minutes, so it cannot run in the lane (15.8),+and the tree it verifies depends on the tip.++* **Verification out of the lane, on the exact tree.** For an answer that holds a model's+ resolution, and when nothing else keeps the land from landing (every file resolved or proposed),+ the pre-resolution builds the tree that would land (`landResolvedTree`: the merged tree of the+ tip and the change, the answer's files written in) and runs verify.command on it, in the+ quarantine, with a slot of the verify pool (`verify_concurrency`, default 2, a pool of its own:+ `graft_resolver_verify_slot_wait_seconds`). The tree object ID that passed is kept.+* **In the lane: tree equality.** The lane merges its own tip, applies the verdicts, builds the+ tree it would write, and asks decideLand about it: a model's resolution lands only if that tree is+ a tree that passed. Tree object IDs are compared, not tips: a tip that moved to the same tree+ keeps the verification; a tip that moved to another tree does not. The lane runs no verification+ of a model's resolution, and writes the merge commit with Graft-Checked-By: verify:<command>.+* **Tip moved: out again, no resolver call.** When the tree is not a verified one, the lane does not+ land. The land goes back out, rebuilds the tree from the same verdicts (the resolver is called only+ for files whose sides changed, as in 15.8), verifies the new tree, and enters the lane again. Each+ round costs one verification; the progress argument is the one of 15.8 (a round is lost only when+ another land landed on the branch), and the rounds are bounded by the deadline of the land.+* **Deadline: a proposal, not a failure.** The deadline of the land is the resolver's timeout plus+ verify.timeout when a verify.command is configured. A verified AI land is opportunistic: when the+ deadline ends before a tree could be verified and landed, the land does not fail for the agent. The+ model's files become PROPOSED with the reason "could not verify before the branch moved again" (or+ "before the deadline of <t>" when the verification itself was cut), the proposals are kept (14), and+ the package goes to the agent. This final pass verifies nothing and calls no resolver. A caller that+ gives up is not a deadline: it gets its cancellation.+* **A verification that fails** (non-zero exit, or its own timeout) is a verdict, not a deadline: the+ files stay PROPOSED and carry "verification failed: exit status N: <the first 4 KiB of the+ output>" (the package cuts a reason to 512 bytes). Nothing is verified for a land that cannot land+ anyway (a file refused, a file left to the agent).+* **Rules are unchanged.** A resolution made by rules alone lands after verify.command, which runs in+ the lane, as before; so does an agent's resolution. Moving them onto the same out-of-lane mechanism+ is the next step, measured on its own, so that in the end there is one verification mechanism.+* **Tests.** A model's resolution that passes lands, signed, with the command run once (so not in+ the lane); one that fails gives a proposal and the output; without a verify.command it is a+ proposal; a tip that moved gets its new tree verified and lands, with no resolver call; a tip that+ moved to the same tree keeps the verification; churn past the deadline gives a proposal, and so does+ the cap on the rounds; the lane is free while a verification runs (a plain land passes through it);+ the lane never verifies a model's resolution; the verify pool bounds the verifications at once;+ rules keep their verification in the lane; and the table test of decideLand, whose one rule is+ compared by tree object ID.diff --git a/doc/graft/RESOLVE.md b/doc/graft/RESOLVE.mdindex 027af61..da29a8c 100644--- a/doc/graft/RESOLVE.md+++ b/doc/graft/RESOLVE.md@@ -46,6 +46,9 @@ actors = ["builder", "planner"]never delay the short checks. `resolve_rounds` (default 32) is a safety capon the rounds of a land whose files keep changing (see "Out of the lane"below); the rounds are bounded by `timeout`, and the cap is no contract.+- `verify_concurrency` (default 2): the verifications of a model's resolution+ that run at once in the Gitaly process, in a pool of their own (see "Optional+ verification of a resolution").- A configuration the server could not use is a startup error (see[CONFIGURATION](CONFIGURATION.md)). A configured resolver is never treatedas no resolver: if it cannot be built later, every land that needs it fails@@ -162,13 +165,18 @@ and the error carries the conflict package as a second detail (see[The conflict package and the re-land](#the-conflict-package-and-the-re-land)).Nothing lands partially: the branch is never left in a partial state.-### Proposals: a model's resolution never lands--The server never lands what a model resolved: the agent always resolves-(PLAN10 14). graft-resolve marks every file its `llm` strategy resolved as a-proposal (`"proposal": true`), and the server treats an answer as proposing-when a file is marked, or when the resolver names the `llm` strategy without-marking any. Such a land fails with the package: the model's files are+### Proposals, and when a model's resolution lands++A model's resolution lands only through the operator's verification+(PLAN10 14 and 15.9): when `verify.command` is configured and passes, out of+the lane, on the exact tree that would land (see "Optional verification of a+resolution"). Without a `verify.command` the server never lands what a model+resolved: the agent decides. graft-resolve marks every file its `llm` strategy+resolved as a proposal (`"proposal": true`), and the server treats an answer as+proposing when a file is marked, or when the resolver names the `llm` strategy+without marking any. A land of such an answer that does not land (no+`verify.command`, a verification that failed, or none finished in time) fails+with the package: the model's files are`PROPOSED`, the files the rules resolved keep `RESOLVED`, and every file theanswer resolved carries its `proposal` (action, blob OID, mode, resolver, andthe explanation cut to 512 bytes). An answer that does not land for another@@ -381,7 +389,33 @@ temporary directory and runs `command` there, with a minimal environment(`PATH`, and `HOME` and `TMPDIR` in a directory of their own). A non-zero exit or atimeout rejects the resolution: the land fails, every file of the package is`REJECTED`, and the reason is `verification failed:` with the exit status and-the first 4 KiB of the output.+the first 4 KiB of the output. That is the verification of a resolution made by+rules and of an agent's, which runs in the branch's lane, on the merged tree of+the lane's tip.++**For a model's resolution the verification is the gate that lets it land**+(PLAN10 15.9), and it runs out of the lane:++- The server builds the tree that would land (the merged tree of the tip and+ the change, the model's files written in) and runs `command` on it with a slot+ of the verify pool (`verify_concurrency`, default 2;+ `graft_resolver_verify_slot_wait_seconds`). The lane never runs it.+- In the lane, the model's resolution lands only if the tree it would write is+ a tree that passed (compared by tree object ID, not by tip). It is signed+ `Graft-Resolved-By: <resolver>` and `Graft-Checked-By: verify:<command>`.+ The trailer names what ran: the syntax and operator checks run inside the+ resolver and are not named, and the composition report is not a check.+- If the branch moved to another tree, the land goes back out: it rebuilds the+ tree from the same answers (the resolver is called again only for files whose+ sides changed), verifies the new tree, and tries the lane again.+- If the verification fails, the model's files stay `PROPOSED`, with the reason+ `verification failed:` and the output. If the deadline ends before a tree could+ be verified and landed, the land does not fail for the agent: the files stay+ `PROPOSED` with the reason `could not verify before the branch moved again`+ (or `before the deadline of <t>`), and the package carries the proposals. The+ deadline is `timeout` plus `verify.timeout`.+- Nothing is verified for a land that cannot land anyway (a file refused, a+ file left to the agent).**A verify command such as a build or a test suite executes code from themerged tree: code that agents and the AI wrote, run as the gitaly user on the@@ -494,7 +528,7 @@ sides read and 64 Mi cells of diff work for its hunks, and 4 MiB encoded| `REJECTED` | the result failed a rule or a check, e.g. a syntax error at line:column || `FAILED` | the resolver failed on it: a model error, `AI budget exhausted`, no strategy applied || `RESOLVED` | resolved by rules, but the land failed because another file was not; see `proposal` |-| `PROPOSED` | a model resolved it: proposed, never landed; see `proposal` |+| `PROPOSED` | a model resolved it and it did not land (nothing verified it, the verification failed, or none finished in time): proposed; see `proposal` and the reason || `NOT_ATTEMPTED` | the request was not attempted; see `reason` |The agent reads the sides by OID with `GetBlobs`, resolves every file, and@@ -952,6 +986,7 @@ in blank lines only, and the accepted results that got no syntax verdict.| check answer | 64 KiB + 8 KiB per file | server || checks of agents' resolutions at once | 2 | `check_concurrency` || resolutions at once | 2 | `resolve_concurrency` |+| verifications of a model's resolution at once | 2 | `verify_concurrency` || rounds of a land whose files keep changing | 32 (a safety cap; `timeout` bounds the land) | `resolve_rounds` || verification run | 5 min, output kept 4 KiB | `verify.timeout` || request body read by `graft-resolve` | 128 MiB (`protocol.DefaultMaxRequestBytes`) | `--max-request-bytes` |diff --git a/internal/gitaly/config/config_graft.go b/internal/gitaly/config/config_graft.goindex 0afb605..fce28dc 100644--- a/internal/gitaly/config/config_graft.go+++ b/internal/gitaly/config/config_graft.go@@ -327,6 +327,11 @@ type GraftResolver struct {// pool of their own so that long resolutions never delay the short checks. Zero means// DefaultGraftResolverResolveConcurrency.ResolveConcurrency int `json:"resolve_concurrency" toml:"resolve_concurrency,omitempty"`+ // VerifyConcurrency bounds the verifications that run at once on this server out of the lane+ // (PLAN10 15.9): the operator's verify.command on the exact tree of a model's resolution, which+ // lands only if it passes. It is a pool of its own, as the command takes minutes. Zero means+ // DefaultGraftResolverVerifyConcurrency.+ VerifyConcurrency int `json:"verify_concurrency" toml:"verify_concurrency,omitempty"`// ResolveRounds is a safety cap on the rounds of a land whose conflicting files keep changing// while it is resolved (PLAN10 15.8). The rounds are bounded by the resolver's timeout; the cap// is set well above realistic contention and is no contract. Zero means@@ -518,8 +523,8 @@ func (r GraftResolver) validate() error {if _, err := r.ProposalTTLOrDefault(); err != nil {return err}- if r.MaxFiles < 0 || r.MaxFileBytes < 0 || r.CheckConcurrency < 0 || r.ResolveConcurrency < 0 || r.ResolveRounds < 0 {- return fmt.Errorf("graft land resolver: max_files, max_file_bytes, check_concurrency, resolve_concurrency and resolve_rounds must not be negative")+ if r.MaxFiles < 0 || r.MaxFileBytes < 0 || r.CheckConcurrency < 0 || r.ResolveConcurrency < 0 || r.VerifyConcurrency < 0 || r.ResolveRounds < 0 {+ return fmt.Errorf("graft land resolver: max_files, max_file_bytes, check_concurrency, resolve_concurrency, verify_concurrency and resolve_rounds must not be negative")}// The check of an agent's resolution sends a file's three sides and its resolution in one// request, which must fit its bound.@@ -587,6 +592,17 @@ func (r GraftResolver) CheckConcurrencyOrDefault() int {return DefaultGraftResolverCheckConcurrency}+// DefaultGraftResolverVerifyConcurrency is the default of GraftResolver.VerifyConcurrency.+const DefaultGraftResolverVerifyConcurrency = 2++// VerifyConcurrencyOrDefault returns VerifyConcurrency or its default.+func (r GraftResolver) VerifyConcurrencyOrDefault() int {+ if r.VerifyConcurrency > 0 {+ return r.VerifyConcurrency+ }+ return DefaultGraftResolverVerifyConcurrency+}+// DefaultGraftResolverResolveConcurrency is the default of GraftResolver.ResolveConcurrency.const DefaultGraftResolverResolveConcurrency = 2diff --git a/internal/gitaly/config/config_graft_resolver_test.go b/internal/gitaly/config/config_graft_resolver_test.goindex 4d248c6..f9fc16a 100644--- a/internal/gitaly/config/config_graft_resolver_test.go+++ b/internal/gitaly/config/config_graft_resolver_test.go@@ -113,6 +113,9 @@ func TestGraftResolverValidateBuildable(t *testing.T) {"negative resolve_concurrency": {GraftResolver{Command: []string{"graft-resolve"}, ResolveConcurrency: -1}, "must not be negative",},+ "negative verify_concurrency": {+ GraftResolver{Command: []string{"graft-resolve"}, VerifyConcurrency: -1}, "must not be negative",+ },"negative resolve_rounds": {GraftResolver{Command: []string{"graft-resolve"}, ResolveRounds: -1}, "must not be negative",},@@ -135,6 +138,8 @@ func TestGraftResolverValidateBuildable(t *testing.T) {require.Equal(t, 5, GraftResolver{CheckConcurrency: 5}.CheckConcurrencyOrDefault())require.Equal(t, DefaultGraftResolverResolveConcurrency, GraftResolver{}.ResolveConcurrencyOrDefault())require.Equal(t, 4, GraftResolver{ResolveConcurrency: 4}.ResolveConcurrencyOrDefault())+ require.Equal(t, DefaultGraftResolverVerifyConcurrency, GraftResolver{}.VerifyConcurrencyOrDefault())+ require.Equal(t, 3, GraftResolver{VerifyConcurrency: 3}.VerifyConcurrencyOrDefault())require.Equal(t, DefaultGraftResolverResolveRounds, GraftResolver{}.ResolveRoundsOrDefault())require.Equal(t, 7, GraftResolver{ResolveRounds: 7}.ResolveRoundsOrDefault())}diff --git a/internal/gitaly/service/operations/land_preresolve.go b/internal/gitaly/service/operations/land_preresolve.goindex bef5e67..18085ea 100644--- a/internal/gitaly/service/operations/land_preresolve.go+++ b/internal/gitaly/service/operations/land_preresolve.go@@ -53,17 +53,46 @@ type landVerdicts map[landFileKey]landVerdict// landPreResolution is what the pre-resolution of a land hands to its lane.type landPreResolution struct {verdicts landVerdicts+ // verified are the trees that verify.command passed on, out of the lane (15.9): a tree, by its+ // object ID, so that a tip that moved to the same tree keeps the verification.+ verified map[git.ObjectID]bool+ // unverifiable, when set, is why the land has run out of time to verify a model's resolution:+ // the pre-resolution then verifies nothing and calls no resolver, and the files of the model+ // become proposals with this reason.+ unverifiable string}-// landNeedsResolution is the answer of a lane that met conflicting files without a verdict: new-// files, or files whose sides changed since they were resolved. UserLand resolves them out of the-// lane and lands again.+func newLandPreResolution() *landPreResolution {+ return &landPreResolution{verdicts: landVerdicts{}, verified: map[git.ObjectID]bool{}}+}++// markVerified records that verify.command passed on tree.+func (p *landPreResolution) markVerified(tree git.ObjectID) {+ if p.verified == nil {+ p.verified = map[git.ObjectID]bool{}+ }+ p.verified[tree] = true+}++// isVerified reports whether verify.command passed on tree, out of the lane.+func (p *landPreResolution) isVerified(tree git.ObjectID) bool {+ return p != nil && tree != "" && p.verified[tree]+}++// landNeedsResolution is the answer of a lane that cannot land the land as it stands: it met+// conflicting files without a verdict (new files, or files whose sides changed since they were+// resolved), or the tree it would land is not one that verify.command passed on (unverified: the+// tip moved). UserLand resolves and verifies again out of the lane, and lands again.type landNeedsResolution struct {tip, commit git.ObjectIDpaths []string+ unverified git.ObjectID}func (e *landNeedsResolution) Error() string {+ if e.unverified != "" {+ return fmt.Sprintf("the tree %s of the land at tip %s was not verified", e.unverified, e.tip)+ }return fmt.Sprintf("conflicting files to resolve at tip %s: %s", e.tip, strings.Join(e.paths, ", "))}@@ -112,10 +141,16 @@ func (s *Server) landPreResolve(ctx context.Context, in *gitalypb.UserLandRequesreturn nil}- l, _, err := s.landResolveAt(ctx, in, repo, tip, commitID, committer, pre.verdicts)+ l, _, err := s.landResolveAt(ctx, in, repo, tip, commitID, committer, pre)+ var unfinished *landVerifyUnfinishedError+ var needs *landNeedsResolutionswitch {case err == nil, errors.Is(err, errLandNoConflicts):return nil+ case errors.As(err, &unfinished), errors.As(err, &needs):+ // Not the land's failure: UserLand decides what a verification that ran out of time, or a+ // land that still needs files resolved, comes to.+ return err}// Only a conflict that the resolution left unresolved is a merge conflict, and only the policy's// refusal is a refusal. Any other error (a git command that failed, a caller that gave up) means@@ -141,35 +176,56 @@ func (s *Server) landPreResolve(ctx context.Context, in *gitalypb.UserLandRequesreturn landWithPackage(ctx, landMergeConflictError(tip, commitID, paths, err), err)}-// landResolving lands a request with resolve_conflicts (PLAN10 15.8): it pre-resolves out of the-// lane and lands; when the lane meets files without a verdict, it resolves only those, out of the-// lane, and lands again. The rounds are bounded by the resolver's timeout for the whole land, and-// by resolve_rounds, a safety cap. A round is lost only when another land changed one of the-// files, so the lands that contend for a file all land, one per round; a land that keeps losing-// to new arrivals fails at its deadline with the merge conflict, naming the files that kept-// changing.+// landResolving lands a request with resolve_conflicts (PLAN10 15.8, 15.9): it pre-resolves out of+// the lane and lands; when the lane meets files without a verdict, or a tree that verify.command+// did not pass on, it resolves and verifies only what changed, out of the lane, and lands again.+// The rounds are bounded by the deadline of the land, the resolver's timeout plus the verification's+// when one is configured, and by resolve_rounds, a safety cap. A round is lost only when another+// land changed the files or the tip, so the lands that contend for a file all land, one per round.+// A land that runs out of its deadline fails with the merge conflict naming the files that kept+// changing; but a model's resolution that could not be verified in time does not fail the agent: its+// files become proposals, with the reason, in the package (landUnverified).func (s *Server) landResolving(ctx context.Context, in *gitalypb.UserLandRequest, committer git.Signature) (*gitalypb.UserLandResponse, error) {timeout, err := s.resolveCfg.TimeoutOrDefault()if err != nil {return nil, fmt.Errorf("resolver timeout: %w", err)}+ if s.resolveCfg.Verify.Enabled {+ verifyTimeout, err := s.resolveCfg.Verify.TimeoutOrDefault()+ if err != nil {+ return nil, fmt.Errorf("verify timeout: %w", err)+ }+ timeout += verifyTimeout+ }resolveCtx, cancel := context.WithTimeout(ctx, timeout)defer cancel()- pre := &landPreResolution{verdicts: landVerdicts{}}+ pre := newLandPreResolution()var need *landNeedsResolution- // deadline reports whether the land ran out of the resolver's timeout, and not its caller.+ // deadline reports whether the land ran out of its own timeout, and not its caller.deadline := func() bool {return ctx.Err() == nil && errors.Is(resolveCtx.Err(), context.DeadlineExceeded)}until := fmt.Sprintf("until the deadline of %s", timeout)+ // giveUp ends a land that cannot go on: a model's resolution that was not verified in time goes+ // to the agent as proposals; anything else is the merge conflict of files that kept changing.+ giveUp := func(when, unverifiedReason string) (*gitalypb.UserLandResponse, error) {+ if unverifiedReason != "" {+ return s.landUnverified(ctx, in, committer, pre, unverifiedReason, need, when)+ }+ return nil, need.keptChanging(when)+ }rounds := s.resolveCfg.ResolveRoundsOrDefault()for round := 1; round <= rounds; round++ {if err := s.landPreResolve(resolveCtx, in, committer, pre); err != nil {- // A land that lost a round to its files changing, and then ran out of its deadline- // while it resolved them again, failed because they kept changing.- if need != nil && deadline() {- return nil, need.keptChanging(until)+ var unfinished *landVerifyUnfinishedError+ switch {+ case errors.As(err, &unfinished) && deadline():+ return s.landUnverified(ctx, in, committer, pre, fmt.Sprintf("could not verify before the deadline of %s", timeout), need, until)+ case need != nil && deadline():+ // A land that lost a round to its files or its tip changing, and then ran out of its+ // deadline while it resolved or verified again, failed because they kept changing.+ return giveUp(until, landNotVerifiedReason(need))}return nil, err}@@ -178,8 +234,54 @@ func (s *Server) landResolving(ctx context.Context, in *gitalypb.UserLandRequestreturn resp, err}if deadline() {- return nil, need.keptChanging(until)+ return giveUp(until, landNotVerifiedReason(need))}}- return nil, need.keptChanging(fmt.Sprintf("after %d rounds", rounds))+ return giveUp(fmt.Sprintf("after %d rounds", rounds), landNotVerifiedReason(need))+}++// landNotVerifiedReason is why a model's files become proposals when the lane handed the land back+// for a tree that was not verified: the branch moved on after the verification. It is empty when the+// lane asked for files to be resolved.+func landNotVerifiedReason(need *landNeedsResolution) string {+ if need != nil && need.unverified != "" {+ return "could not verify before the branch moved again"+ }+ return ""+}++// landUnverified ends a land whose model resolution could not be verified in time. A verified AI+// land is opportunistic, and losing the race is harmless: the agent gets the proposals. It+// pre-resolves once more, with the request's context, in a new quarantine, verifying nothing and+// calling no resolver (the verdicts are kept): the files of the model are PROPOSED with the reason,+// the proposals are kept, and the land fails with the package. need, when set, is the state the+// lane handed back, for the merge conflict if the land cannot be reported as proposals after all.+func (s *Server) landUnverified(+ ctx context.Context,+ in *gitalypb.UserLandRequest,+ committer git.Signature,+ pre *landPreResolution,+ reason string,+ need *landNeedsResolution,+ when string,+) (*gitalypb.UserLandResponse, error) {+ pre.unverifiable = reason+ err := s.landPreResolve(ctx, in, committer, pre)+ var needs *landNeedsResolution+ switch {+ case errors.As(err, &needs):+ if need == nil {+ need = needs+ }+ return nil, need.keptChanging(when)+ case err != nil:+ return nil, err+ }+ // The land is no longer a conflict to resolve (the change landed, or the merge is clean): the+ // lane answers it.+ resp, err := s.landOnce(ctx, in, committer, nil, pre)+ if errors.As(err, &needs) {+ return nil, needs.keptChanging(when)+ }+ return resp, err}diff --git a/internal/gitaly/service/operations/land_preresolve_test.go b/internal/gitaly/service/operations/land_preresolve_test.goindex d873f31..fa0590b 100755--- a/internal/gitaly/service/operations/land_preresolve_test.go+++ b/internal/gitaly/service/operations/land_preresolve_test.go@@ -190,6 +190,10 @@ if [ "$N" = 1 ]; theni=0while [ ! -f "$V/release" ] && [ "$i" -lt 600 ]; do sleep 0.1; i=$((i+1)); donefi+if [ "$N" != 1 ] && [ -f "$V/fail" ]; then+ echo "tests failed"+ exit 3+fiexit 0`, vdir))graft := batchingGraft()graft.Land.Resolver = withVerify(enabledResolver(argv), verify, "")@@ -258,9 +262,12 @@ func TestUserLand_resolvesOutOfTheLane(t *testing.T) {t.Parallel()h := newHeldLane(t)+ // A model's answer for b, whose verification fails: b cannot land (15.9). The verification+ // runs out of the lane too, while the lane is held.proposal := answerOf("b")proposal.Resolver = "graft-resolve/llm"writeAnswerN(t, h.dir, 2, proposal)+ require.NoError(t, os.WriteFile(filepath.Join(h.vdir, "fail"), nil, 0o600))// The land of b returns while the lane is still held: it never queued behind a.req := h.fx.request("Land b")@@ -269,7 +276,8 @@ func TestUserLand_resolvesOutOfTheLane(t *testing.T) {requireMergeConflict(t, err)require.Equal(t, [][]byte{[]byte("b")}, landErrorDetail(t, err).GetMergeConflict().GetConflictingFiles())requireRunning(t, h.first, "the land of a")- require.Equal(t, 1, h.verifications(t))+ require.Equal(t, 2, h.verifications(t), "a's, held in the lane, and b's, out of it")+ require.Contains(t, packageFile(t, landConflictDetail(err), "b").GetReason(), "verification failed: exit status 3: tests failed")// The proposal was kept although the lane was busy with a batch: it was written before the// lane, on the published tip, never on a tip that exists only in a batch's quarantine@@ -532,12 +540,11 @@ func TestLandApplyVerdicts_neverCallsTheResolver(t *testing.T) {s := fx.server(t, rcfg)in := fx.request("Land agent work")- verdicts := landVerdicts{}- _, _, err := s.landResolveAt(fx.ctx, in, fx.repo, fx.tip, fx.commit, fx.committer(), verdicts)+ pre := newLandPreResolution()+ _, _, err := s.landResolveAt(fx.ctx, in, fx.repo, fx.tip, fx.commit, fx.committer(), pre)require.NoError(t, err)require.Equal(t, 1, resolverCalls(t, dir))- require.Len(t, verdicts, 2)- pre := &landPreResolution{verdicts: verdicts}+ require.Len(t, pre.verdicts, 2)// At the tip they were made at, the verdicts are written.res, err := s.landApplyVerdicts(fx.ctx, in, fx.repo, fx.tip, fx.commit, fx.committer(), pre)diff --git a/internal/gitaly/service/operations/land_proposals.go b/internal/gitaly/service/operations/land_proposals.goindex 8b1e22e..13b9dbd 100644--- a/internal/gitaly/service/operations/land_proposals.go+++ b/internal/gitaly/service/operations/land_proposals.go@@ -24,8 +24,9 @@ import ("github.com/feder-cr/git-ai/proto/go/gitalypb")-// A resolution that a model made is never landed by the server: it is a proposal, which the agent-// reviews, edits or replaces, and lands as its own (PLAN10 14). The server keeps the proposals of a+// A resolution that a model made lands only through the operator's verification (PLAN10 14, 15.9):+// when verify.command ran and passed on the exact tree that lands. Otherwise it is a proposal, which+// the agent reviews, edits or replaces, and lands as its own. The server keeps the proposals of a// failed land under a hidden reference, which the agent reads by object ID (GetObjects).// landProposalPruneErrors counts the prunings of proposal references that failed after a land.@@ -41,8 +42,13 @@ var landProposalWriteErrors = promauto.NewCounter(prometheus.CounterOpts{Help: "Resolver answers of a failed land that could not be kept as proposals.",})-// landProposedReason is the reason of a file that a model resolved.-const landProposedReason = "a model resolved the file: the server never lands a model's resolution, it proposes it (see proposal); land it, edited or not, as your own resolution"+// landProposedReason is the reason of a file that a model resolved and that did not land: a model's+// resolution lands only after verify.command passed on the merged tree, and the reason that it did+// not follows it (landNoVerifyReason, the output of the verification, or why it could not run).+const landProposedReason = "a model resolved the file: it is proposed (see proposal), and lands by itself only after verify.command passed on the merged tree; land it, edited or not, as your own resolution"++// landNoVerifyReason follows landProposedReason when no verify.command is configured.+const landNoVerifyReason = "no verify.command is configured, so nothing verifies a model's resolution"// landAnswerFlagsProposals reports whether a file of the answer is marked as a proposal.func landAnswerFlagsProposals(answer *resolver.Response) bool {@@ -198,18 +204,29 @@ func landTrailerValueOrQuoted(v string) string {}// decideLand is the one decision point of a land that asked the server to resolve its conflicts-// (PLAN10 15.4): it lands only when the resolver answered, the answer holds no model's resolution,-// and every conflicting file is resolved. A file that proposes is never landed. This is where a-// landing gate for model resolutions would live; today there is none.-func decideLand(answer *resolver.Response, files []*landFileState) bool {+// (PLAN10 15.4): it lands only when the resolver answered and every conflicting file is resolved,+// by a rule (RESOLVED) or by a model (PROPOSED). The one rule about a model: a file that proposes+// may land only if verify.command ran and passed on the exact tree that lands. tree is the tree+// the land would write and verified the tree the verification passed on, compared by object ID,+// not by the tip they were made at; a verified tree is never the empty ID. No other code path+// lands a model's resolution.+func decideLand(answer *resolver.Response, files []*landFileState, tree, verified git.ObjectID) bool {if answer == nil {return false}- if landAnswerProposes(answer) {+ proposes := landAnswerProposes(answer)+ if proposes && (verified == "" || verified != tree) {return false}for _, f := range files {- if f.outcome != gitalypb.LandConflictFile_OUTCOME_RESOLVED {+ switch f.outcome {+ case gitalypb.LandConflictFile_OUTCOME_RESOLVED:+ case gitalypb.LandConflictFile_OUTCOME_PROPOSED:+ // Only an answer that proposes has a PROPOSED file, and it passed the rule above.+ if !proposes {+ return false+ }+ default:return false}}diff --git a/internal/gitaly/service/operations/land_proposals_test.go b/internal/gitaly/service/operations/land_proposals_test.goindex 5301a24..cde0b1d 100755--- a/internal/gitaly/service/operations/land_proposals_test.go+++ b/internal/gitaly/service/operations/land_proposals_test.go@@ -54,7 +54,7 @@ func proposalRefTarget(t *testing.T, fx resolveFixture) string {func TestUserLand_proposals(t *testing.T) {t.Parallel()- t.Run("a model's answer is proposed, never landed", func(t *testing.T) {+ t.Run("a model's answer without a verify.command is proposed, not landed", func(t *testing.T) {t.Parallel()dir, argv := resolverScript(t, "")@@ -79,7 +79,7 @@ func TestUserLand_proposals(t *testing.T) {for _, p := range []string{"a", "b"} {f := packageFile(t, pkg, p)require.Equal(t, gitalypb.LandConflictFile_OUTCOME_PROPOSED, f.GetOutcome(), p)- require.Contains(t, f.GetReason(), "never lands a model's resolution", p)+ require.Contains(t, f.GetReason(), "no verify.command is configured", p)prop := f.GetProposal()require.NotNil(t, prop, p)require.Equal(t, resolver.ActionWrite, prop.GetAction())@@ -303,17 +303,41 @@ func TestDecideLand(t *testing.T) {return []*landFileState{{outcome: gitalypb.LandConflictFile_OUTCOME_RESOLVED}, {outcome: gitalypb.LandConflictFile_OUTCOME_RESOLVED}}}rules := &resolver.Response{Resolver: "graft-resolve/trivial,union", Files: []resolver.ResponseFile{{Path: "a"}, {Path: "b"}}}- require.True(t, decideLand(rules, resolved()), "rules resolved every file")- require.False(t, decideLand(nil, resolved()), "no answer")+ require.True(t, decideLand(rules, resolved(), "", ""), "rules resolved every file")+ require.True(t, decideLand(rules, resolved(), "t1", "t2"), "rules land whatever tree was verified: their verification stays in the lane")+ require.False(t, decideLand(nil, resolved(), "t1", "t1"), "no answer")left := resolved()left[1].outcome = gitalypb.LandConflictFile_OUTCOME_NOT_SENT- require.False(t, decideLand(rules, left), "a file left to the agent")-+ require.False(t, decideLand(rules, left, "", ""), "a file left to the agent")+ rejected := resolved()+ rejected[0].outcome = gitalypb.LandConflictFile_OUTCOME_REJECTED+ require.False(t, decideLand(rules, rejected, "", ""), "a file the answer is refused for")+ proposedByRules := resolved()+ proposedByRules[0].outcome = gitalypb.LandConflictFile_OUTCOME_PROPOSED+ require.False(t, decideLand(rules, proposedByRules, "", ""), "a PROPOSED file in an answer that does not propose")++ // THE rule (PLAN10 15.9): a file that proposes may land only if verify.command passed on the exact+ // tree that lands, compared by object ID.flagged := &resolver.Response{Resolver: "graft-resolve/trivial", Files: []resolver.ResponseFile{{Path: "a"}, {Path: "b", Proposal: true}}}- require.False(t, decideLand(flagged, resolved()), "a file that proposes, even with every outcome RESOLVED")named := &resolver.Response{Resolver: "graft-resolve/trivial,llm", Files: []resolver.ResponseFile{{Path: "a"}, {Path: "b"}}}- require.False(t, decideLand(named, resolved()), "a resolver that names the llm strategy")+ proposing := func() []*landFileState {+ return []*landFileState{{outcome: gitalypb.LandConflictFile_OUTCOME_RESOLVED}, {outcome: gitalypb.LandConflictFile_OUTCOME_PROPOSED}}+ }+ for desc, answer := range map[string]*resolver.Response{"a flagged file": flagged, "the llm strategy by name": named} {+ require.True(t, decideLand(answer, proposing(), "tree-1", "tree-1"), "%s: verified on this very tree", desc)+ require.False(t, decideLand(answer, proposing(), "tree-1", ""), "%s: nothing verified", desc)+ require.False(t, decideLand(answer, proposing(), "tree-1", "tree-2"), "%s: verified on another tree (the tip moved)", desc)+ require.False(t, decideLand(answer, proposing(), "", ""), "%s: no tree at all: empty is not equal", desc)+ require.False(t, decideLand(answer, resolved(), "tree-1", ""), "%s: even with every outcome RESOLVED", desc)++ leftOver := proposing()+ leftOver[0].outcome = gitalypb.LandConflictFile_OUTCOME_FAILED+ require.False(t, decideLand(answer, leftOver, "tree-1", "tree-1"), "%s: verified, but a file is left to the agent", desc)+ refused := proposing()+ refused[1].outcome = gitalypb.LandConflictFile_OUTCOME_REJECTED+ require.False(t, decideLand(answer, refused, "tree-1", "tree-1"), "%s: verified, but a file is refused", desc)+ }}// TestResolveLandConflicts_policyFirst: an agent the policy does not let update the branch getsdiff --git a/internal/gitaly/service/operations/land_reland.go b/internal/gitaly/service/operations/land_reland.goindex 080ecb7..b739bf7 100644--- a/internal/gitaly/service/operations/land_reland.go+++ b/internal/gitaly/service/operations/land_reland.go@@ -306,7 +306,7 @@ func (s *Server) landWithResolution(for _, f := range l.files {resp.Files = append(resp.Files, review.resolved[f.conflict.path])}- res, err := s.landWriteResolution(ctx, in, repo, tip, commitID, l.mergedTree, committer, review.actor, resp, checkedBy)+ res, err := s.landWriteResolution(ctx, in, repo, tip, commitID, l.mergedTree, committer, review.actor, resp, checkedBy, "", false)res.syntaxUnchecked = syntaxUncheckedreturn res, l.verifyRejected(err)}diff --git a/internal/gitaly/service/operations/land_resolve.go b/internal/gitaly/service/operations/land_resolve.goindex 77b09aa..72d0f6c 100644--- a/internal/gitaly/service/operations/land_resolve.go+++ b/internal/gitaly/service/operations/land_resolve.go@@ -77,24 +77,27 @@ type landFileConflict struct {}// landResolveAt resolves, against tip, the conflicting files of the land that have no verdict in-// verdicts yet (PLAN10 15.8). It runs out of the lane: the policy first (an agent that may not+// pre yet (PLAN10 15.8). It runs out of the lane: the policy first (an agent that may not// update the branch gets nothing resolved), the listing, the classification of every file, one-// resolver request for the files to resolve, taken with a resolve slot, and the validation; then-// decideLand over every file, the verdicts reused included. A request carries only what a file's-// verdict may depend on: the sides, the path and modes, the change's message, the configuration;-// not the tip, its merge base or its message, so that a verdict can be reused at another tip-// (15.8). When the land is landable, the new verdicts are added to verdicts, and the listing and-// the answer to write are returned. Otherwise the answer is kept as proposals (14) and the land-// fails with a *landUnresolvedError, whose package says what happened to each file; a request the-// server does not attempt at all also gets a package, with the reason. Any other error means the-// conflicts could not be listed, and the caller reports the merge conflict alone.+// resolver request for the files to resolve, taken with a resolve slot, and the validation; then,+// for an answer that holds a model's resolution, the verification of the exact tree that would+// land (15.9), with a verify slot; then decideLand over every file, the verdicts reused included.+// A request carries only what a file's verdict may depend on: the sides, the path and modes, the+// change's message, the configuration; not the tip, its merge base or its message, so that a+// verdict can be reused at another tip (15.8). When the land is landable, the new verdicts are+// added to pre, the verified tree is recorded there, and the listing and the answer to write are+// returned. Otherwise the answer is kept as proposals (14) and the land fails with a+// *landUnresolvedError, whose package says what happened to each file, and why a model's+// resolution did not land; a request the server does not attempt at all also gets a package, with+// the reason. A verification that the context ended is a *landVerifyUnfinishedError. Any other+// error means the conflicts could not be listed, and the caller reports the merge conflict alone.func (s *Server) landResolveAt(ctx context.Context,in *gitalypb.UserLandRequest,repo *localrepo.Repo,tip, commitID git.ObjectID,committer git.Signature,- verdicts landVerdicts,+ pre *landPreResolution,) (landListing, *resolver.Response, error) {// An agent that may not update the branch gets nothing resolved, and no proposal is kept for// it: the policy decides on the actor and the reference alone, so it is checked first.@@ -136,14 +139,17 @@ func (s *Server) landResolveAt(}// A file the classification sends and that has a verdict with its key is not resolved again.+ // mine are the files with an answer, reused or fresh, whose outcome the answer decides.reused := map[string]resolver.ResponseFile{}var reusedNames []stringvar sent []resolver.File+ var mine []*landFileStatefor _, f := range files {if !f.sent() {continue}- if v, ok := verdicts[landKeyOf(f.conflict)]; ok {+ mine = append(mine, f)+ if v, ok := pre.verdicts[landKeyOf(f.conflict)]; ok {reused[f.conflict.path] = v.filereusedNames = append(reusedNames, v.resolver)f.outcome = gitalypb.LandConflictFile_OUTCOME_RESOLVED@@ -151,6 +157,14 @@ func (s *Server) landResolveAt(}sent = append(sent, landRequestFile(f))}+ // Past its deadline a land only reports what it has: the resolver is not called.+ if pre.unverifiable != "" && len(sent) > 0 {+ paths := make([]string, 0, len(sent))+ for _, f := range sent {+ paths = append(paths, f.Path)+ }+ return l, nil, &landNeedsResolution{tip: tip, commit: commitID, paths: paths}+ }// answer is the resolver's accepted answer, the only thing ever written. A refusal is never// written, whatever its outcomes say: answer stays nil, and the land fails with the package.@@ -184,32 +198,55 @@ func (s *Server) landResolveAt(landFailSent(files, gitalypb.LandConflictFile_OUTCOME_FAILED, fmt.Sprintf("the resolver's answer names an invalid resolver %q", resp.Resolver))default:answer = resp- proposes := landAnswerProposes(resp)- // A resolver that names the llm strategy but marks no file is taken to have proposed- // every file: the mark may be missing, never the other way round.- flagged := landAnswerFlagsProposals(resp)- written := make(map[string]resolver.ResponseFile, len(resp.Files))- for _, rf := range resp.Files {- written[rf.Path] = rf- }- for _, f := range files {- if !f.sent() {- continue- }- rf := written[f.conflict.path]- f.outcome = gitalypb.LandConflictFile_OUTCOME_RESOLVED- switch err := landModeOfSides(f.conflict, rf); {- case err != nil:- f.outcome, f.reason = gitalypb.LandConflictFile_OUTCOME_REJECTED, err.Error()- case rf.Proposal || (proposes && !flagged):- f.outcome, f.reason = gitalypb.LandConflictFile_OUTCOME_PROPOSED, landProposedReason+ }+ }++ combined := landCombinedAnswer(answer, reused, reusedNames, len(sent) > 0)+ if combined != nil {+ landSetOutcomes(combined, mine)+ }+ // The verdicts of an answer that nothing refuses are kept by their keys before it is verified or+ // decided: the lane and the next round reuse them, whatever becomes of this verification.+ if answer != nil && landAllAnswered(files) {+ for _, f := range files {+ for _, rf := range answer.Files {+ if rf.Path == f.conflict.path {+ pre.verdicts[landKeyOf(f.conflict)] = landVerdict{file: rf, resolver: answer.Resolver}}}}}- combined := landCombinedAnswer(answer, reused, reusedNames, len(sent) > 0)- landed := decideLand(combined, files)+ // A model's resolution lands only through the operator's verification (15.9): verify.command+ // runs here, out of the lane, on the exact tree that would land, once nothing else stops the+ // land. Whatever keeps a model's file from landing is said in the reason of its file.+ var tree, verified git.ObjectID+ if combined != nil && landAnswerProposes(combined) {+ switch {+ case pre.unverifiable != "":+ landAnnotateProposed(files, pre.unverifiable)+ case !s.resolveCfg.Verify.Enabled:+ landAnnotateProposed(files, landNoVerifyReason)+ case landAllAnswered(files):+ var err error+ if tree, err = s.landResolvedTree(ctx, repo, l.mergedTree, combined); err != nil {+ return l, nil, err+ }+ var verifyErr *landVerifyError+ switch err := s.landVerifyOutOfLane(ctx, repo, tree); {+ case err == nil:+ verified = tree+ pre.markVerified(tree)+ s.logger.WithFields(log.Fields{"branch": string(in.GetBranch()), "tip": tip.String(), "commit": commitID.String(), "tree": tree.String()}).+ InfoContext(ctx, "graft: a model's resolution passed the verification out of the lane")+ case errors.As(err, &verifyErr):+ landAnnotateProposed(files, verifyErr.Error())+ default:+ return l, nil, err+ }+ }+ }+ landed := decideLand(combined, files, tree, verified)// An answer that does not land, because a model made part of it or another file is left, is// kept for the agent: the files that rules resolved can be landed as they are. The proposals// are an aid and the package is the contract: when they cannot be kept, the package still goes@@ -229,16 +266,56 @@ func (s *Server) landResolveAt(return l, nil, l.unresolved("", landUnresolvedCause(files), resolveErr)}- if answer != nil {- for _, f := range files {- for _, rf := range answer.Files {- if rf.Path == f.conflict.path {- verdicts[landKeyOf(f.conflict)] = landVerdict{file: rf, resolver: answer.Resolver}- }- }+ return l, combined, nil+}++// landSetOutcomes sets the outcome of each of files from the answer that covers them: RESOLVED, or+// REJECTED when the resolution writes a mode no side has, or PROPOSED when a model made the file.+// A resolver that names the llm strategy but marks no file is taken to have proposed every file:+// the mark may be missing, never the other way round. The lane sets them the same way from the+// verdicts it applies, so that both ask decideLand the same question.+func landSetOutcomes(answer *resolver.Response, files []*landFileState) {+ proposes := landAnswerProposes(answer)+ flagged := landAnswerFlagsProposals(answer)+ written := make(map[string]resolver.ResponseFile, len(answer.Files))+ for _, rf := range answer.Files {+ written[rf.Path] = rf+ }+ for _, f := range files {+ rf := written[f.conflict.path]+ f.outcome, f.reason = gitalypb.LandConflictFile_OUTCOME_RESOLVED, ""+ switch err := landModeOfSides(f.conflict, rf); {+ case err != nil:+ f.outcome, f.reason = gitalypb.LandConflictFile_OUTCOME_REJECTED, err.Error()+ case rf.Proposal || (proposes && !flagged):+ f.outcome, f.reason = gitalypb.LandConflictFile_OUTCOME_PROPOSED, landProposedReason+ }+ }+}++// landAllAnswered reports whether every file is resolved or proposed: nothing else keeps the land+// from landing, so that a model's resolution is worth verifying.+func landAllAnswered(files []*landFileState) bool {+ for _, f := range files {+ if f.outcome != gitalypb.LandConflictFile_OUTCOME_RESOLVED && f.outcome != gitalypb.LandConflictFile_OUTCOME_PROPOSED {+ return false+ }+ }+ return true+}++// landAnnotateProposed adds why a model's files did not land to the reason of each.+func landAnnotateProposed(files []*landFileState, why string) {+ for _, f := range files {+ if f.outcome != gitalypb.LandConflictFile_OUTCOME_PROPOSED {+ continue+ }+ if f.reason == "" {+ f.reason = why+ } else {+ f.reason += "; " + why}}- return l, combined, nil}// landResolveSlotted sends the resolve request with a resolve slot. The slot is given back when the@@ -363,7 +440,36 @@ func (s *Server) landApplyVerdicts(names = append(names, v.resolver)}answer := landCombinedAnswer(nil, reused, names, false)- res, err := s.landWriteResolution(ctx, in, repo, tip, commitID, l.mergedTree, committer, identity.Actor(ctx, ""), answer, "")+ landSetOutcomes(answer, l.files)++ // The one rule about a model's resolution, asked of the tree this tip would land: it lands only+ // if verify.command passed on this very tree (tree OIDs, not tips). When the tip moved, the tree+ // is another one, and the land goes back out of the lane to be verified again. The lane never+ // runs the verification of a model's resolution.+ var tree, verified git.ObjectID+ if landAnswerProposes(answer) {+ if tree, err = s.landResolvedTree(ctx, repo, l.mergedTree, answer); err != nil {+ return landResolution{}, err+ }+ if pre.isVerified(tree) {+ verified = tree+ }+ }+ if !decideLand(answer, l.files, tree, verified) {+ need := &landNeedsResolution{tip: tip, commit: commitID, unverified: tree}+ if tree == "" {+ // Nothing to verify, yet not landable: a file the answer rejects. Resolving again says why.+ for _, f := range l.files {+ need.paths = append(need.paths, f.conflict.path)+ }+ }+ return landResolution{}, need+ }+ checkedBy := ""+ if verified != "" {+ checkedBy = s.landVerifyTrailer()+ }+ res, err := s.landWriteResolution(ctx, in, repo, tip, commitID, l.mergedTree, committer, identity.Actor(ctx, ""), answer, checkedBy, tree, verified != "")return res, l.verifyRejected(err)}@@ -555,8 +661,12 @@ func landUnresolvedCause(files []*landFileState) string {}// landWriteResolution writes a resolution (the resolver's answer, or the agent's) into the merged-// tree and commits it with parents [tip, commitID]; checkedBy names the resolver that checked an-// agent's resolution, if one did. The commit message carries the trailers.+// tree and commits it with parents [tip, commitID]; checkedBy names what checked it, if anything+// did. The commit message carries the trailers. built is the resolved tree when the caller already+// built it (landResolvedTree). verified says that verify.command passed on that very tree, out of+// the lane (15.9), so that it is not run again; otherwise the operator's verification runs here,+// in the lane, on the merged tree, before it lands: that is a resolution made by rules or by an+// agent, which keeps its verification in the lane for now.func (s *Server) landWriteResolution(ctx context.Context,in *gitalypb.UserLandRequest,@@ -566,11 +676,9 @@ func (s *Server) landWriteResolution(actor string,resp *resolver.Response,checkedBy string,+ built git.ObjectID,+ verified bool,) (landResolution, error) {- tree, err := repo.ReadTree(ctx, git.Revision(mergedTree.String()), localrepo.WithRecursive())- if err != nil {- return landResolution{}, fmt.Errorf("reading merged tree: %w", err)- }paths := make([]string, 0, len(resp.Files))for _, f := range resp.Files {paths = append(paths, f.Path)@@ -581,35 +689,23 @@ func (s *Server) landWriteResolution(if err != nil {return landResolution{}, err}- for _, f := range resp.Files {- if f.Action == resolver.ActionDelete {- if err := tree.Delete(f.Path); err != nil && !errors.Is(err, localrepo.ErrEntryNotFound) {- return landResolution{}, fmt.Errorf("deleting %q: %w", f.Path, err)- }- continue- }-- blob, err := repo.WriteBlob(ctx, bytes.NewReader(f.Content), localrepo.WriteBlobConfig{Path: f.Path})- if err != nil {- return landResolution{}, fmt.Errorf("writing resolved %q: %w", f.Path, err)- }- entry := localrepo.TreeEntry{OID: blob, Mode: f.Mode, Path: path.Base(f.Path), Type: localrepo.Blob}- if err := tree.Add(f.Path, entry, localrepo.WithOverwriteFile()); err != nil {- return landResolution{}, fmt.Errorf("adding resolved %q: %w", f.Path, err)+ treeID := built+ if treeID == "" {+ var err error+ if treeID, err = s.landResolvedTree(ctx, repo, mergedTree, resp); err != nil {+ return landResolution{}, err}}- if err := tree.Write(ctx, repo); err != nil {- return landResolution{}, fmt.Errorf("writing merged tree: %w", err)- }- // The optional verification runs on the merged tree, the resolution included, before it lands.- if s.resolveCfg.Verify.Enabled {- if err := s.landVerify(ctx, repo, tree.OID); err != nil {+ // The optional verification runs on the merged tree, the resolution included, before it lands,+ // unless it already passed on this tree out of the lane.+ if s.resolveCfg.Verify.Enabled && !verified {+ if err := s.landVerify(ctx, repo, treeID); err != nil {return landResolution{}, err}}newTip, err := repo.WriteCommit(ctx, localrepo.WriteCommitConfig{- TreeID: tree.OID,+ TreeID: treeID,Message: message,Parents: []git.ObjectID{tip, commitID},AuthorName: committer.Name,@@ -647,6 +743,36 @@ func (s *Server) landWriteResolution(return landResolution{commit: newTip, resolvedPaths: resolvedPaths, resolver: resp.Resolver}, nil}+// landResolvedTree writes a resolution into the merged tree and returns the tree: the exact tree+// that a land of this resolution would commit. Its object ID is what the verification is tied to.+func (s *Server) landResolvedTree(ctx context.Context, repo *localrepo.Repo, mergedTree git.ObjectID, resp *resolver.Response) (git.ObjectID, error) {+ tree, err := repo.ReadTree(ctx, git.Revision(mergedTree.String()), localrepo.WithRecursive())+ if err != nil {+ return "", fmt.Errorf("reading merged tree: %w", err)+ }+ for _, f := range resp.Files {+ if f.Action == resolver.ActionDelete {+ if err := tree.Delete(f.Path); err != nil && !errors.Is(err, localrepo.ErrEntryNotFound) {+ return "", fmt.Errorf("deleting %q: %w", f.Path, err)+ }+ continue+ }++ blob, err := repo.WriteBlob(ctx, bytes.NewReader(f.Content), localrepo.WriteBlobConfig{Path: f.Path})+ if err != nil {+ return "", fmt.Errorf("writing resolved %q: %w", f.Path, err)+ }+ entry := localrepo.TreeEntry{OID: blob, Mode: f.Mode, Path: path.Base(f.Path), Type: localrepo.Blob}+ if err := tree.Add(f.Path, entry, localrepo.WithOverwriteFile()); err != nil {+ return "", fmt.Errorf("adding resolved %q: %w", f.Path, err)+ }+ }+ if err := tree.Write(ctx, repo); err != nil {+ return "", fmt.Errorf("writing merged tree: %w", err)+ }+ return tree.OID, nil+}+// landClassify decides, for every file, whether it may be sent to the resolver, and reads the// sides of those that may (and of those whose content decided against it, for the hunks of the// package). The order: the path (a commit trailer must carry it, the protocol must carry it, thediff --git a/internal/gitaly/service/operations/land_resolve_helper_test.go b/internal/gitaly/service/operations/land_resolve_helper_test.goindex 4f6dc9a..5c318c8 100644--- a/internal/gitaly/service/operations/land_resolve_helper_test.go+++ b/internal/gitaly/service/operations/land_resolve_helper_test.go@@ -13,10 +13,19 @@ import (// the land is landable: the pre-resolution (landResolveAt) and the lane's write in one call, as the// tests of the resolution itself need it. UserLand runs the two apart, the first out of the lane.func (s *Server) resolveLandConflicts(ctx context.Context, in *gitalypb.UserLandRequest, repo *localrepo.Repo, tip, commitID git.ObjectID, committer git.Signature) (landResolution, error) {- l, answer, err := s.landResolveAt(ctx, in, repo, tip, commitID, committer, landVerdicts{})+ pre := newLandPreResolution()+ l, answer, err := s.landResolveAt(ctx, in, repo, tip, commitID, committer, pre)if err != nil {return landResolution{}, err}- res, err := s.landWriteResolution(ctx, in, repo, tip, commitID, l.mergedTree, committer, identity.Actor(ctx, ""), answer, "")+ // As the lane does for a model's resolution: the tree is the one verify.command passed on, out+ // of the lane, or the answer would not have been landable.+ var tree git.ObjectID+ if landAnswerProposes(answer) {+ if tree, err = s.landResolvedTree(ctx, repo, l.mergedTree, answer); err != nil {+ return landResolution{}, err+ }+ }+ res, err := s.landWriteResolution(ctx, in, repo, tip, commitID, l.mergedTree, committer, identity.Actor(ctx, ""), answer, "", tree, pre.isVerified(tree))return res, l.verifyRejected(err)}diff --git a/internal/gitaly/service/operations/land_review_test.go b/internal/gitaly/service/operations/land_review_test.goindex 2cff1d5..4f9e576 100644--- a/internal/gitaly/service/operations/land_review_test.go+++ b/internal/gitaly/service/operations/land_review_test.go@@ -30,7 +30,7 @@ func TestResolveSlot_aPanickingResolverCallGivesItsSlotBack(t *testing.T) {s.landResolveHook = func() { panic("a resolution failed") }require.PanicsWithValue(t, "a resolution failed", func() {- _, _, _ = s.landResolveAt(fx.ctx, fx.request("Land"), fx.repo, fx.tip, fx.commit, fx.committer(), landVerdicts{})+ _, _, _ = s.landResolveAt(fx.ctx, fx.request("Land"), fx.repo, fx.tip, fx.commit, fx.committer(), newLandPreResolution())})require.Equal(t, 0, resolverCalls(t, dir), "the hook ran before the call")diff --git a/internal/gitaly/service/operations/land_verified_test.go b/internal/gitaly/service/operations/land_verified_test.gonew file mode 100644index 0000000..f2a2c2a--- /dev/null+++ b/internal/gitaly/service/operations/land_verified_test.go@@ -0,0 +1,524 @@+package operations++import (+ "context"+ "errors"+ "fmt"+ "io/fs"+ "os"+ "path/filepath"+ "strings"+ "testing"+ "time"++ "github.com/stretchr/testify/require"+ "google.golang.org/grpc/codes"+ "google.golang.org/grpc/status"++ "github.com/feder-cr/git-ai/internal/git"+ "github.com/feder-cr/git-ai/internal/git/gittest"+ "github.com/feder-cr/git-ai/internal/gitaly/config"+ "github.com/feder-cr/git-ai/internal/graft/proposals"+ "github.com/feder-cr/git-ai/internal/graft/resolver"+ "github.com/feder-cr/git-ai/internal/helper/text"+ "github.com/feder-cr/git-ai/proto/go/gitalypb"+)++// The tests of a model's resolution that lands through the operator's verification (PLAN10 15.9): it+// lands only if verify.command passed, out of the lane, on the exact tree that lands.++// countingVerify writes a verification command into a new temporary directory and returns the+// directory and the argv of the command. Every run appends a line to dir/vlines before body runs,+// and N is the number of runs so far. body may block, move the branch or fail.+func countingVerify(t *testing.T, body string) (string, []string) {+ t.Helper()++ dir := t.TempDir()+ script := fmt.Sprintf(`#!/bin/sh+DIR='%s'+echo x >> "$DIR/vlines"+N=$(wc -l < "$DIR/vlines" | tr -d ' ')+%s+`, dir, body)+ path := filepath.Join(dir, "verify.sh")+ require.NoError(t, os.WriteFile(path, []byte(script), 0o755))+ return dir, []string{path}+}++// verifyRuns returns the number of times the verification of dir ran.+func verifyRuns(t *testing.T, dir string) int {+ t.Helper()++ raw, err := os.ReadFile(filepath.Join(dir, "vlines"))+ if errors.Is(err, fs.ErrNotExist) {+ return 0+ }+ require.NoError(t, err)+ return len(raw) / 2+}++// moveBranchBody makes the n-th run of a countingVerify command move the branch to the commit in+// dir/move-n, when the test wrote one (writeVerifyMoves), before it goes on.+var moveBranchBody = fmt.Sprintf(`if [ -f "$DIR/move-$N" ]; then+ "$(cat "$DIR/git")" -C "$(cat "$DIR/repo")" update-ref refs/heads/%s "$(cat "$DIR/move-$N")"+fi`, git.DefaultBranch)++// writeVerifyMoves lets the n-th run of the verification of dir move the branch of fx to moves[n-1].+func writeVerifyMoves(t *testing.T, dir string, fx resolveFixture, moves ...git.ObjectID) {+ t.Helper()++ writeMoves(t, dir, fx, moves...)+}++// verifiedFixture is a land whose resolver answers with a model's resolution, and whose+// verify.command is the countingVerify command with the given body.+func verifiedFixture(t *testing.T, answer resolver.Response, verifyBody string, tune func(*config.GraftResolver)) (resolveFixture, string, string) {+ t.Helper()++ rdir, rargv := resolverScript(t, "")+ writeAnswer(t, rdir, answer)+ vdir, vargv := countingVerify(t, verifyBody)+ rcfg := withVerify(enabledResolver(rargv), vargv, "")+ if tune != nil {+ tune(&rcfg)+ }+ return newResolveFixture(t, rcfg), rdir, vdir+}++// landCommit returns the commit with the given ID.+func landedCommit(t *testing.T, fx resolveFixture, resp *gitalypb.UserLandResponse) (*gitalypb.GitCommit, git.ObjectID) {+ t.Helper()++ id := git.ObjectID(resp.GetLandedCommitId())+ commit, err := fx.repo.ReadCommit(fx.ctx, id.Revision())+ require.NoError(t, err)+ return commit, id+}++func TestUserLand_verifiedModelResolution(t *testing.T) {+ t.Parallel()++ t.Run("a model's resolution that passes the verification lands, signed", func(t *testing.T) {+ t.Parallel()++ fx, rdir, vdir := verifiedFixture(t, modelAnswer(), "grep -qx merged-a a && grep -qx merged-b b && grep -qx d d", nil)++ resp, err := fx.client.UserLand(fx.ctx, fx.request("Land agent work"))+ require.NoError(t, err)+ commit, landed := landedCommit(t, fx, resp)+ requireBranchAt(t, fx.cfg, fx.repoPath, git.DefaultBranch, landed)+ require.Equal(t, []string{fx.tip.String(), fx.commit.String()}, commit.GetParentIds())+ require.Equal(t, "graft-resolve/llm", resp.GetResolver())+ require.Equal(t, "merged-a\n", fx.blobAt(t, landed, "a"))+ require.Equal(t, "merged-b\n", fx.blobAt(t, landed, "b"))+ body := string(commit.GetBody())+ require.Contains(t, body, "Graft-Resolved-By: graft-resolve/llm\n")+ require.Contains(t, body, "Graft-Checked-By: verify:verify.sh\n")+ require.Equal(t, 1, resolverCalls(t, rdir))+ // Once: out of the lane. The lane lands the verified tree without verifying it again.+ require.Equal(t, 1, verifyRuns(t, vdir))+ require.Empty(t, proposalRefTarget(t, fx), "nothing was proposed")+ })++ t.Run("a model's file next to a rule's file lands when the tree passes", func(t *testing.T) {+ t.Parallel()++ answer := modelAnswer()+ answer.Resolver = "graft-resolve/trivial,llm"+ answer.Files[1].Proposal = false+ fx, _, vdir := verifiedFixture(t, answer, "exit 0", nil)++ resp, err := fx.client.UserLand(fx.ctx, fx.request("Land"))+ require.NoError(t, err)+ _, landed := landedCommit(t, fx, resp)+ require.Equal(t, "merged-a\n", fx.blobAt(t, landed, "a"))+ require.Equal(t, "merged-b\n", fx.blobAt(t, landed, "b"))+ require.Equal(t, 1, verifyRuns(t, vdir))+ })++ t.Run("a verification that fails gives a proposal and the output", func(t *testing.T) {+ t.Parallel()++ fx, rdir, vdir := verifiedFixture(t, modelAnswer(), `echo "2 tests failed: TestFoo"; exit 3`, nil)++ _, err := fx.client.UserLand(fx.ctx, fx.request("Land"))+ requireMergeConflict(t, err)+ requireBranchAt(t, fx.cfg, fx.repoPath, git.DefaultBranch, fx.tip)+ pkg := landConflictDetail(err)+ require.NotNil(t, pkg)+ require.Equal(t, proposals.Ref(git.DefaultBranch, fx.commit).String(), pkg.GetProposalRef())+ require.NotEmpty(t, proposalRefTarget(t, fx))+ for _, p := range []string{"a", "b"} {+ f := packageFile(t, pkg, p)+ require.Equal(t, gitalypb.LandConflictFile_OUTCOME_PROPOSED, f.GetOutcome(), p)+ require.Contains(t, f.GetReason(), "verification failed: exit status 3: 2 tests failed: TestFoo", p)+ require.NotNil(t, f.GetProposal(), p)+ require.Equal(t, "merged-"+p+"\n", string(gittest.Exec(t, fx.cfg, "-C", fx.repoPath, "cat-file", "blob", f.GetProposal().GetOid())))+ }+ require.Equal(t, 1, resolverCalls(t, rdir))+ require.Equal(t, 1, verifyRuns(t, vdir), "once, out of the lane")+ })++ t.Run("a verification that times out is a failure with its reason", func(t *testing.T) {+ t.Parallel()++ rdir, rargv := resolverScript(t, "")+ writeAnswer(t, rdir, modelAnswer())+ vdir, vargv := countingVerify(t, "sleep 30")+ fx := newResolveFixture(t, withVerify(enabledResolver(rargv), vargv, "300ms"))++ _, err := fx.client.UserLand(fx.ctx, fx.request("Land"))+ requireMergeConflict(t, err)+ f := packageFile(t, landConflictDetail(err), "a")+ require.Equal(t, gitalypb.LandConflictFile_OUTCOME_PROPOSED, f.GetOutcome())+ require.Contains(t, f.GetReason(), "the command did not finish within 300ms")+ require.Equal(t, 1, verifyRuns(t, vdir))+ })++ t.Run("without a verify.command a model's resolution is a proposal", func(t *testing.T) {+ t.Parallel()++ rdir, rargv := resolverScript(t, "")+ writeAnswer(t, rdir, modelAnswer())+ fx := newResolveFixture(t, enabledResolver(rargv))++ _, err := fx.client.UserLand(fx.ctx, fx.request("Land"))+ requireMergeConflict(t, err)+ requireBranchAt(t, fx.cfg, fx.repoPath, git.DefaultBranch, fx.tip)+ for _, p := range []string{"a", "b"} {+ f := packageFile(t, landConflictDetail(err), p)+ require.Equal(t, gitalypb.LandConflictFile_OUTCOME_PROPOSED, f.GetOutcome(), p)+ require.Contains(t, f.GetReason(), "no verify.command is configured", p)+ require.NotNil(t, f.GetProposal(), p)+ }+ })++ t.Run("nothing is verified when another file keeps the land from landing", func(t *testing.T) {+ t.Parallel()++ // b is written with a mode no side has: refused. Verifying the tree would cost the operator's+ // tests for a land that cannot land.+ answer := modelAnswer()+ answer.Files[1].Mode = "100755"+ fx, _, vdir := verifiedFixture(t, answer, "exit 0", nil)++ _, err := fx.client.UserLand(fx.ctx, fx.request("Land"))+ requireMergeConflict(t, err)+ pkg := landConflictDetail(err)+ require.Equal(t, gitalypb.LandConflictFile_OUTCOME_PROPOSED, packageFile(t, pkg, "a").GetOutcome())+ require.Equal(t, gitalypb.LandConflictFile_OUTCOME_REJECTED, packageFile(t, pkg, "b").GetOutcome())+ require.Equal(t, 0, verifyRuns(t, vdir))+ })++ t.Run("a tip that moved gets the new tree verified, then lands", func(t *testing.T) {+ t.Parallel()++ // While the first verification runs, a writer adds e to the branch: the tree to land is no+ // longer the verified one, so the land goes back out, rebuilds the tree with the same verdicts+ // (the resolver is not called again), verifies it, and lands that.+ fx, rdir, vdir := verifiedFixture(t, modelAnswer(), moveBranchBody, nil)+ writer := gittest.WriteCommit(t, fx.cfg, fx.repoPath, gittest.WithParents(fx.tip),+ gittest.WithTreeEntries(entries("a", "ours\n", "b", "ours\n", "c", "c\n", "e", "e\n")...))+ writeVerifyMoves(t, vdir, fx, writer)++ resp, err := fx.client.UserLand(fx.ctx, fx.request("Land"))+ require.NoError(t, err)+ commit, landed := landedCommit(t, fx, resp)+ require.Equal(t, []string{writer.String(), fx.commit.String()}, commit.GetParentIds())+ require.Equal(t, "e\n", fx.blobAt(t, landed, "e"))+ require.Equal(t, "merged-a\n", fx.blobAt(t, landed, "a"))+ require.Equal(t, 1, resolverCalls(t, rdir), "no resolver call: the files did not change")+ require.Equal(t, 2, verifyRuns(t, vdir), "the tree of the new tip is verified, out of the lane")+ require.Contains(t, string(commit.GetBody()), "Graft-Checked-By: verify:verify.sh\n")+ })++ t.Run("a tip that moved to the same tree keeps the verification", func(t *testing.T) {+ t.Parallel()++ // The tree is what is verified, not the tip: a writer that moves the branch to another commit+ // with the very tree the land started from leaves the verified tree the tree to land.+ fx, _, vdir := verifiedFixture(t, modelAnswer(), moveBranchBody, nil)+ same := gittest.WriteCommit(t, fx.cfg, fx.repoPath, gittest.WithParents(fx.tip), gittest.WithMessage("same tree, another commit"),+ gittest.WithTreeEntries(entries("a", "ours\n", "b", "ours\n", "c", "c\n")...))+ writeVerifyMoves(t, vdir, fx, same)++ resp, err := fx.client.UserLand(fx.ctx, fx.request("Land"))+ require.NoError(t, err)+ commit, _ := landedCommit(t, fx, resp)+ require.Equal(t, []string{same.String(), fx.commit.String()}, commit.GetParentIds())+ require.Equal(t, 1, verifyRuns(t, vdir), "the same tree: verified once")+ })++ t.Run("a land the lane hands back for the last round gives a proposal, not a failure", func(t *testing.T) {+ t.Parallel()++ // resolve_rounds is a cap: past it, a tree the branch moved away from is not verified in time+ // any more, whatever the clock says. The model's files are proposals, with the reason that the+ // branch moved again, and the resolver is not called again.+ fx, rdir, vdir := verifiedFixture(t, modelAnswer(), moveBranchBody, func(r *config.GraftResolver) {+ r.ResolveRounds = 1+ })+ writer := gittest.WriteCommit(t, fx.cfg, fx.repoPath, gittest.WithParents(fx.tip),+ gittest.WithTreeEntries(entries("a", "ours\n", "b", "ours\n", "c", "c\n", "e", "e\n")...))+ writeVerifyMoves(t, vdir, fx, writer)++ _, err := fx.client.UserLand(fx.ctx, fx.request("Land"))+ requireMergeConflict(t, err)+ require.NotContains(t, err.Error(), "files kept changing")+ pkg := landConflictDetail(err)+ require.NotNil(t, pkg)+ require.Equal(t, writer.String(), pkg.GetTip(), "the package is about the tip the branch moved to")+ for _, p := range []string{"a", "b"} {+ f := packageFile(t, pkg, p)+ require.Equal(t, gitalypb.LandConflictFile_OUTCOME_PROPOSED, f.GetOutcome(), p)+ require.Contains(t, f.GetReason(), "could not verify before the branch moved again", p)+ require.NotNil(t, f.GetProposal(), p)+ }+ require.Equal(t, 1, resolverCalls(t, rdir))+ require.Equal(t, 1, verifyRuns(t, vdir), "the give-up verifies nothing")+ requireBranchAt(t, fx.cfg, fx.repoPath, git.DefaultBranch, writer)+ })++ t.Run("churn past the deadline gives a proposal, not a failure", func(t *testing.T) {+ t.Parallel()++ // The branch moves under every verification, faster than the land can verify. At the deadline+ // the model's files become proposals, with the reason, and the agent gets the package.+ fx, rdir, vdir := verifiedFixture(t, modelAnswer(), "sleep 0.4\n"+moveBranchBody, func(r *config.GraftResolver) {+ r.Timeout = "1s"+ r.Verify.Timeout = "2s"+ r.ResolveRounds = 1000+ })+ var moves []git.ObjectID+ parent := fx.tip+ for i := 0; i < 40; i++ {+ parent = gittest.WriteCommit(t, fx.cfg, fx.repoPath, gittest.WithParents(parent),+ gittest.WithTreeEntries(entries("a", "ours\n", "b", "ours\n", "c", "c\n", "e", fmt.Sprintf("e-%d\n", i))...))+ moves = append(moves, parent)+ }+ writeVerifyMoves(t, vdir, fx, moves...)++ // The bound is a check that the land does not hang, not a measure: the deadline of the land is+ // 3 s, and a loaded machine takes a multiple of that for the pass that follows it.+ start := time.Now()+ _, err := fx.client.UserLand(fx.ctx, fx.request("Land"))+ took := time.Since(start)+ t.Logf("the land ended after %s (deadline 3s)", took)+ require.Less(t, took, 3*time.Minute)+ requireMergeConflict(t, err)+ require.NotContains(t, err.Error(), "files kept changing")+ pkg := landConflictDetail(err)+ require.NotNil(t, pkg)+ require.NotEmpty(t, pkg.GetProposalRef())+ for _, p := range []string{"a", "b"} {+ f := packageFile(t, pkg, p)+ require.Equal(t, gitalypb.LandConflictFile_OUTCOME_PROPOSED, f.GetOutcome(), p)+ require.Contains(t, f.GetReason(), "could not verify before", p)+ require.NotNil(t, f.GetProposal(), p)+ require.Equal(t, "merged-"+p+"\n", string(gittest.Exec(t, fx.cfg, "-C", fx.repoPath, "cat-file", "blob", f.GetProposal().GetOid())))+ }+ require.Equal(t, 1, resolverCalls(t, rdir), "the verdicts are reused: the resolver is called once")+ // One round or several, by the speed of the machine: a loaded one spends the whole deadline on+ // the first. What does not depend on speed is that the verification ran, that the writers did+ // not run out (the deadline ended the land), and that the resolver was called once. The round+ // that goes back out after a tip moved is the test above, deterministic.+ require.GreaterOrEqual(t, verifyRuns(t, vdir), 1)+ require.Less(t, verifyRuns(t, vdir), len(moves), "the deadline ended it, not the writers")+ // The branch was only moved by the writers: the model's resolution did not land.+ tip := gittest.ResolveRevision(t, fx.cfg, fx.repoPath, git.DefaultBranch)+ parents := strings.Fields(text.ChompBytes(gittest.Exec(t, fx.cfg, "-C", fx.repoPath, "rev-list", "--parents", "-n1", tip.String())))+ require.NotContains(t, parents, fx.commit.String())+ })+}++func TestUserLand_verifiedModelResolution_laneIsFree(t *testing.T) {+ t.Parallel()++ // The first verification blocks until the test releases it. While it does, the branch's lane is+ // free: a plain land of another change passes through it. When the verification is released, the+ // tree of the model's land is not the tree that was verified (the plain land moved the branch),+ // and the land verifies the new tree and lands on top of the plain land.+ rdir, rargv := resolverScript(t, "")+ writeAnswer(t, rdir, modelAnswer())+ vdir, vargv := countingVerify(t, `if [ "$N" = 1 ]; then+ i=0+ while [ ! -f "$DIR/release" ] && [ "$i" -lt 1200 ]; do sleep 0.1; i=$((i+1)); done+fi`)+ graft := batchingGraft()+ graft.Land.Resolver = withVerify(enabledResolver(rargv), vargv, "")+ fx := newResolveFixtureIn(t, graft,+ entries("a", "base\n", "b", "base\n", "c", "c\n"),+ entries("a", "ours\n", "b", "ours\n", "c", "c\n"),+ entries("a", "theirs\n", "b", "theirs\n", "c", "c\n", "d", "d\n"),+ )+ plain := gittest.WriteCommit(t, fx.cfg, fx.repoPath, gittest.WithParents(fx.base),+ gittest.WithTreeEntries(entries("a", "base\n", "b", "base\n", "c", "c\n", "z", "z\n")...))++ model := landAsync(fx, fx.commit, "Land the model's")+ require.Eventually(t, func() bool { return verifyRuns(t, vdir) == 1 }, 30*time.Second, 10*time.Millisecond, "the verification never started")++ // The verification holds until the test releases it (or two minutes pass): a plain land that ends+ // before that went through a free lane, whatever the speed of the machine.+ plainReq := fx.request("Land the plain one")+ plainReq.CommitId = plain.String()+ plainReq.ResolveConflicts = false+ plainDone := make(chan landResult, 1)+ go func() {+ resp, err := fx.client.UserLand(fx.ctx, plainReq)+ plainDone <- landResult{resp: resp, err: err}+ }()+ var plainResult landResult+ select {+ case plainResult = <-plainDone:+ case <-time.After(60 * time.Second):+ t.Cleanup(func() { _ = os.WriteFile(filepath.Join(vdir, "release"), nil, 0o600) })+ require.FailNow(t, "the plain land did not end while the verification ran: the lane is held")+ }+ require.NoError(t, plainResult.err, "the lane is free while the verification runs")+ plainResp := plainResult.resp+ _, notReleased := os.Stat(filepath.Join(vdir, "release"))+ require.ErrorIs(t, notReleased, fs.ErrNotExist, "the plain land ended before the release")+ select {+ case r := <-model:+ require.FailNow(t, "the model's land ended before the verification was released", "%v %v", r.resp, r.err)+ default:+ }+ require.Equal(t, 1, verifyRuns(t, vdir))++ require.NoError(t, os.WriteFile(filepath.Join(vdir, "release"), nil, 0o600))+ resp := requireLanded(t, model)+ commit, landed := landedCommit(t, fx, resp)+ require.Equal(t, []string{plainResp.GetLandedCommitId(), fx.commit.String()}, commit.GetParentIds())+ require.Equal(t, "z\n", fx.blobAt(t, landed, "z"))+ require.Equal(t, "merged-a\n", fx.blobAt(t, landed, "a"))+ require.Equal(t, 2, verifyRuns(t, vdir))+ require.Equal(t, 1, resolverCalls(t, rdir))+}++// TestUserLand_verifiedModelResolution_poolBoundsVerifications: with verify_concurrency 1, two lands+// at once never run the operator's command at once.+func TestUserLand_verifiedModelResolution_poolBoundsVerifications(t *testing.T) {+ t.Parallel()++ rdir, rargv := resolverScript(t, "")+ writeAnswer(t, rdir, modelAnswer())+ vdir, vargv := countingVerify(t, `if ! mkdir "$DIR/lock" 2>/dev/null; then+ echo x >> "$DIR/overlap"+ sleep 0.3+ exit 0+fi+sleep 0.3+rmdir "$DIR/lock"`)+ rcfg := withVerify(enabledResolver(rargv), vargv, "")+ rcfg.VerifyConcurrency = 1+ graft := batchingGraft()+ graft.Land.Resolver = rcfg+ fx := newResolveFixtureIn(t, graft,+ entries("a", "base\n", "b", "base\n"),+ entries("a", "ours\n", "b", "ours\n"),+ entries("a", "theirs-0\n", "b", "theirs-0\n", "d0", "0\n"),+ )+ second := gittest.WriteCommit(t, fx.cfg, fx.repoPath, gittest.WithParents(fx.base),+ gittest.WithTreeEntries(entries("a", "theirs-1\n", "b", "theirs-1\n", "d1", "1\n")...))++ first := landAsync(fx, fx.commit, "Land 0")+ other := landAsync(fx, second, "Land 1")+ requireLanded(t, first)+ requireLanded(t, other)++ require.GreaterOrEqual(t, verifyRuns(t, vdir), 2)+ _, err := os.Stat(filepath.Join(vdir, "overlap"))+ require.ErrorIs(t, err, fs.ErrNotExist, "two verifications ran at once")+}++// TestLandApplyVerdicts_neverVerifiesAModelsResolution: the lane writes a model's resolution only+// for a tree that verify.command passed on out of it, and never runs the command itself.+func TestLandApplyVerdicts_neverVerifiesAModelsResolution(t *testing.T) {+ t.Parallel()++ rdir, rargv := resolverScript(t, "")+ writeAnswer(t, rdir, modelAnswer())+ vdir, vargv := countingVerify(t, "exit 0")+ rcfg := withVerify(enabledResolver(rargv), vargv, "")+ fx := newResolveFixture(t, rcfg)+ s := fx.server(t, rcfg)+ in := fx.request("Land")++ // Out of the lane: the answer is resolved and its tree verified, once.+ pre := newLandPreResolution()+ _, _, err := s.landResolveAt(fx.ctx, in, fx.repo, fx.tip, fx.commit, fx.committer(), pre)+ require.NoError(t, err)+ require.Equal(t, 1, verifyRuns(t, vdir))+ require.Len(t, pre.verified, 1)++ // In the lane, with the verified tree: it lands, and does not verify.+ res, err := s.landApplyVerdicts(fx.ctx, in, fx.repo, fx.tip, fx.commit, fx.committer(), pre)+ require.NoError(t, err)+ require.Equal(t, "merged-a\n", fx.blobAt(t, res.commit, "a"))+ require.Equal(t, 1, verifyRuns(t, vdir), "the lane did not run the command")+ message := string(gittest.Exec(t, fx.cfg, "-C", fx.repoPath, "log", "-1", "--format=%B", res.commit.String()))+ require.Contains(t, message, "Graft-Checked-By: verify:verify.sh\n")++ // In the lane, with no tree verified (the tip moved to a tree nobody verified): it writes nothing+ // and hands the land back to be verified; it does not verify either.+ unverified := &landPreResolution{verdicts: pre.verdicts, verified: map[git.ObjectID]bool{}}+ _, err = s.landApplyVerdicts(fx.ctx, in, fx.repo, fx.tip, fx.commit, fx.committer(), unverified)+ var need *landNeedsResolution+ require.ErrorAs(t, err, &need)+ require.NotEmpty(t, need.unverified, "the tree to verify")+ require.Equal(t, 1, verifyRuns(t, vdir), "the lane did not run the command")++ // Another verified tree is not this one: tree OIDs are compared.+ other := &landPreResolution{verdicts: pre.verdicts, verified: map[git.ObjectID]bool{"0000000000000000000000000000000000000000": true}}+ _, err = s.landApplyVerdicts(fx.ctx, in, fx.repo, fx.tip, fx.commit, fx.committer(), other)+ require.ErrorAs(t, err, &need)+ require.Equal(t, 1, verifyRuns(t, vdir))+}++// TestUserLand_verifiedRulesKeepTheirVerificationInTheLane: a resolution made by rules alone keeps its+// verify.command in the lane, as before (15.9): the next step moves it.+func TestUserLand_verifiedRulesKeepTheirVerificationInTheLane(t *testing.T) {+ t.Parallel()++ fx, _, vdir := verifiedFixture(t, validAnswer(), "grep -qx merged-a a", nil)++ resp, err := fx.client.UserLand(fx.ctx, fx.request("Land"))+ require.NoError(t, err)+ commit, _ := landedCommit(t, fx, resp)+ require.Equal(t, 1, verifyRuns(t, vdir))+ require.Contains(t, string(commit.GetBody()), "Graft-Resolved-By: graft-resolve/fake\n")+ require.NotContains(t, string(commit.GetBody()), "Graft-Checked-By", "rules are unchanged")++ failing, _, failingDir := verifiedFixture(t, validAnswer(), "exit 1", nil)+ _, err = failing.client.UserLand(failing.ctx, failing.request("Land"))+ requireMergeConflict(t, err)+ for _, f := range landConflictDetail(err).GetFiles() {+ require.Equal(t, gitalypb.LandConflictFile_OUTCOME_REJECTED, f.GetOutcome())+ }+ require.Equal(t, 1, verifyRuns(t, failingDir))+}++// TestUserLand_verifiedModelResolution_policyAndCancellation: a land whose caller gives up while it+// verifies fails with the caller's cancellation, and not as proposals.+func TestUserLand_verifiedModelResolution_callerGivesUp(t *testing.T) {+ t.Parallel()++ fx, _, vdir := verifiedFixture(t, modelAnswer(), "sleep 30", func(r *config.GraftResolver) { r.Verify.Timeout = "1m" })+ ctx, cancel := context.WithCancel(fx.ctx)+ req := fx.request("Land")+ done := make(chan error, 1)+ go func() {+ _, err := fx.client.UserLand(ctx, req)+ done <- err+ }()+ require.Eventually(t, func() bool { return verifyRuns(t, vdir) == 1 }, 30*time.Second, 10*time.Millisecond)+ cancel()+ select {+ case err := <-done:+ require.Error(t, err)+ require.Equal(t, codes.Canceled, status.Code(err), "%v", err)+ case <-time.After(30 * time.Second):+ require.FailNow(t, "the land did not end when its caller gave up")+ }+ requireBranchAt(t, fx.cfg, fx.repoPath, git.DefaultBranch, fx.tip)+}diff --git a/internal/gitaly/service/operations/land_verify.go b/internal/gitaly/service/operations/land_verify.goindex 00c4845..0f1f9ac 100644--- a/internal/gitaly/service/operations/land_verify.go+++ b/internal/gitaly/service/operations/land_verify.go@@ -91,10 +91,11 @@ func (s *Server) landVerify(ctx context.Context, repo *localrepo.Repo, tree git.}runErr := cmd.Wait()switch {- case errors.Is(runCtx.Err(), context.DeadlineExceeded):- return &landVerifyError{reason: fmt.Sprintf("the command did not finish within %s", timeout)}case ctx.Err() != nil:+ // The caller's deadline or cancellation, not a verdict of the command.return ctx.Err()+ case errors.Is(runCtx.Err(), context.DeadlineExceeded):+ return &landVerifyError{reason: fmt.Sprintf("the command did not finish within %s", timeout)}case runErr != nil:reason := strings.TrimSpace(output.String())if code, ok := command.ExitStatus(runErr); ok {@@ -105,6 +106,55 @@ func (s *Server) landVerify(ctx context.Context, repo *localrepo.Repo, tree git.return nil}+// landVerifyUnfinishedError is a verification that could not be completed, because its context ended+// (the deadline of the land, or the caller) while it waited for a slot or ran: not a verdict.+type landVerifyUnfinishedError struct {+ err error+}++func (e *landVerifyUnfinishedError) Error() string {+ return "verification not finished: " + e.err.Error()+}++func (e *landVerifyUnfinishedError) Unwrap() error { return e.err }++// landVerifyOutOfLane runs the verification of a model's resolution out of the lane (PLAN10 15.9),+// with a slot of the verify pool. It returns a *landVerifyError when the command fails or times+// out, a *landVerifyUnfinishedError when ctx ended first, and another error when the verification+// could not be run.+func (s *Server) landVerifyOutOfLane(ctx context.Context, repo *localrepo.Repo, tree git.ObjectID) error {+ slot, err := s.conflictResolver.AcquireVerifySlot(ctx)+ if err != nil {+ if ctx.Err() != nil {+ return &landVerifyUnfinishedError{err: err}+ }+ return err+ }+ defer slot.Release()++ err = s.landVerify(ctx, repo, tree)+ var verifyErr *landVerifyError+ switch {+ case err == nil, errors.As(err, &verifyErr):+ return err+ case ctx.Err() != nil:+ return &landVerifyUnfinishedError{err: err}+ }+ return err+}++// landVerifyTrailer is what Graft-Checked-By says of a verification that passed: verify:<command>,+// the base name of the command, or verify when that is not a valid trailer value.+func (s *Server) landVerifyTrailer() string {+ name := "verify"+ if cmd := s.resolveCfg.Verify.Command; len(cmd) > 0 {+ if v := "verify:" + filepath.Base(cmd[0]); landTrailerValueValid(v) {+ name = v+ }+ }+ return name+}+// landExtractTar writes the files of the archive of a tree under dir. It refuses an entry that// would leave dir, or that lies under a symbolic link it wrote: a link is written as a link and// never followed.diff --git a/internal/graft/resolver/check.go b/internal/graft/resolver/check.goindex 1359c1a..3744c44 100644--- a/internal/graft/resolver/check.go+++ b/internal/graft/resolver/check.go@@ -128,6 +128,47 @@ func (s *ResolveSlot) Resolve(ctx context.Context, req Request) (*Response, erroreturn s.c.resolve(ctx, req)}+// verifySlotWait is the time the verifications waited for a verify slot.+var verifySlotWait = promauto.NewHistogram(prometheus.HistogramOpts{+ Name: "graft_resolver_verify_slot_wait_seconds",+ Help: "Time a verification of a model's resolution waited for one of the resolver's verify slots.",+ Buckets: prometheus.ExponentialBuckets(0.01, 4, 8),+})++// VerifySlot is the right to run one verification at a time: the client runs at most+// verify_concurrency of them at once (AcquireVerifySlot), in a pool apart from the checks' and the+// resolutions'. Release it when the verification is done.+type VerifySlot struct {+ c *Client+ released bool+}++// AcquireVerifySlot waits for a verify slot. The wait counts against the deadline of ctx: when ctx+// ends first, it fails, and the land cannot be verified in time.+func (c *Client) AcquireVerifySlot(ctx context.Context) (*VerifySlot, error) {+ if c == nil {+ return nil, ErrDisabled+ }+ start := time.Now()+ select {+ case c.verifySlots <- struct{}{}:+ verifySlotWait.Observe(time.Since(start).Seconds())+ return &VerifySlot{c: c}, nil+ case <-ctx.Done():+ verifySlotWait.Observe(time.Since(start).Seconds())+ return nil, fmt.Errorf("waiting for a verify slot of the resolver: %w", ctx.Err())+ }+}++// Release gives the slot back; a second call does nothing.+func (s *VerifySlot) Release() {+ if s == nil || s.released {+ return+ }+ s.released = true+ <-s.c.verifySlots+}+// Check sends one check request with the slot (Client.check).func (s *CheckSlot) Check(ctx context.Context, req Request) (*CheckResponse, error) {if s == nil || s.released {diff --git a/internal/graft/resolver/client.go b/internal/graft/resolver/client.goindex 70ef97b..59c44fd 100644--- a/internal/graft/resolver/client.go+++ b/internal/graft/resolver/client.go@@ -102,6 +102,9 @@ type Client struct {// the short checks (PLAN10 15.8).checkSlots chan struct{}resolveSlots chan struct{}+ // verifySlots bounds the verifications of a model's resolution that run at once (VerifySlot),+ // a third pool, as the operator's command takes minutes (PLAN10 15.9).+ verifySlots chan struct{}// sendRules decide the files that are never sent (SendRules).sendRules *SendRules}@@ -135,8 +138,8 @@ func New(cfg config.GraftResolver) (*Client, error) {return nil, err}- if cfg.MaxFiles < 0 || cfg.MaxFileBytes < 0 || cfg.CheckConcurrency < 0 || cfg.ResolveConcurrency < 0 || cfg.ResolveRounds < 0 {- return nil, errors.New("graft land resolver: max_files, max_file_bytes, check_concurrency, resolve_concurrency and resolve_rounds must not be negative")+ if cfg.MaxFiles < 0 || cfg.MaxFileBytes < 0 || cfg.CheckConcurrency < 0 || cfg.ResolveConcurrency < 0 || cfg.VerifyConcurrency < 0 || cfg.ResolveRounds < 0 {+ return nil, errors.New("graft land resolver: max_files, max_file_bytes, check_concurrency, resolve_concurrency, verify_concurrency and resolve_rounds must not be negative")}// The check of an agent's resolution sends a file's three sides and its resolution in one// request, which must fit its bound.@@ -159,6 +162,7 @@ func New(cfg config.GraftResolver) (*Client, error) {sendRules: rules,checkSlots: make(chan struct{}, cfg.CheckConcurrencyOrDefault()),resolveSlots: make(chan struct{}, cfg.ResolveConcurrencyOrDefault()),+ verifySlots: make(chan struct{}, cfg.VerifyConcurrencyOrDefault()),}if cfg.URL != "" {diff --git a/internal/graft/resolver/client_test.go b/internal/graft/resolver/client_test.goindex 6997d2f..b3570d5 100644--- a/internal/graft/resolver/client_test.go+++ b/internal/graft/resolver/client_test.go@@ -768,6 +768,55 @@ func TestAcquireResolveSlot_boundsTheResolutionsAtOnce(t *testing.T) {require.ErrorIs(t, err, ErrDisabled)}+// TestAcquireVerifySlot_boundsTheVerificationsAtOnce: at most verify_concurrency slots are held at+// once; a wait counts against the caller's deadline; a released slot frees its place once; the pool+// is apart from the checks' and the resolutions'.+func TestAcquireVerifySlot_boundsTheVerificationsAtOnce(t *testing.T) {+ cfg := commandConfig([]string{"/bin/true"})+ cfg.VerifyConcurrency = 1+ cfg.CheckConcurrency = 1+ cfg.ResolveConcurrency = 1+ client := newClient(t, cfg)+ ctx := testhelper.Context(t)++ first, err := client.AcquireVerifySlot(ctx)+ require.NoError(t, err)++ waitCtx, cancel := context.WithTimeout(ctx, 50*time.Millisecond)+ defer cancel()+ _, err = client.AcquireVerifySlot(waitCtx)+ require.ErrorIs(t, err, context.DeadlineExceeded)+ require.ErrorContains(t, err, "waiting for a verify slot of the resolver")++ // A held verify slot delays neither a check nor a resolution.+ quick, cancelQuick := context.WithTimeout(ctx, time.Second)+ defer cancelQuick()+ checkSlot, err := client.AcquireCheckSlot(quick)+ require.NoError(t, err)+ resolveSlot, err := client.AcquireResolveSlot(quick)+ require.NoError(t, err)+ checkSlot.Release()+ resolveSlot.Release()++ first.Release()+ first.Release()+ second, err := client.AcquireVerifySlot(ctx)+ require.NoError(t, err)+ waitCtx2, cancel2 := context.WithTimeout(ctx, 50*time.Millisecond)+ defer cancel2()+ _, err = client.AcquireVerifySlot(waitCtx2)+ require.ErrorIs(t, err, context.DeadlineExceeded, "a second release must not free a second place")+ second.Release()++ var none *VerifySlot+ none.Release()+ var disabled *Client+ _, err = disabled.AcquireVerifySlot(ctx)+ require.ErrorIs(t, err, ErrDisabled)++ require.Equal(t, config.DefaultGraftResolverVerifyConcurrency, cap(newClient(t, commandConfig([]string{"/bin/true"})).verifySlots))+}+// TestResolveSlots_apartFromTheCheckSlots: the resolutions and the checks have pools of their own// (PLAN10 15.8): every resolve slot held does not delay a check, and every check slot held does not// delay a resolution.@@ -806,6 +855,7 @@ func TestAcquireResolveSlot_defaultConcurrency(t *testing.T) {func TestNew_refusesNegativePools(t *testing.T) {for name, set := range map[string]func(*config.GraftResolver){"resolve_concurrency": func(c *config.GraftResolver) { c.ResolveConcurrency = -1 },+ "verify_concurrency": func(c *config.GraftResolver) { c.VerifyConcurrency = -1 },"resolve_rounds": func(c *config.GraftResolver) { c.ResolveRounds = -1 },} {cfg := commandConfig([]string{"/bin/true"})diff --git a/internal/graft/resolver/protocol.go b/internal/graft/resolver/protocol.goindex a87ab21..1fd5dc2 100644--- a/internal/graft/resolver/protocol.go+++ b/internal/graft/resolver/protocol.go@@ -206,8 +206,8 @@ type Response struct {// ResponseFile is the resolution of one conflicting path. Content is base64// on the wire and is only meaningful for ActionWrite. Proposal marks a-// resolution made by a model: the server never lands it, it proposes it to the-// agent (PLAN10 14).+// resolution made by a model: the server lands it only if verify.command passed+// on the tree that lands, and otherwise proposes it to the agent (PLAN10 14, 15.9).type ResponseFile struct {Path string `json:"path"`Action string `json:"action"`diff --git a/proto/go/gitalypb/operations.pb.go b/proto/go/gitalypb/operations.pb.goindex a598ac2..3dff2b1 100644--- a/proto/go/gitalypb/operations.pb.go+++ b/proto/go/gitalypb/operations.pb.go@@ -278,8 +278,9 @@ const (// OUTCOME_NOT_ATTEMPTED: the request was not resolved at all (see// LandConflict.reason).LandConflictFile_OUTCOME_NOT_ATTEMPTED LandConflictFile_Outcome = 5- // OUTCOME_PROPOSED: a model resolved the file; the server never lands a- // model's resolution, it proposes it (see proposal).+ // OUTCOME_PROPOSED: a model resolved the file; the server lands a+ // model's resolution only after verify.command passed on the merged tree,+ // otherwise it proposes it (see proposal).LandConflictFile_OUTCOME_PROPOSED LandConflictFile_Outcome = 6)diff --git a/proto/operations.proto b/proto/operations.protoindex ed5d3b0..541e3c1 100644--- a/proto/operations.proto+++ b/proto/operations.proto@@ -1349,8 +1349,9 @@ message LandConflictFile {// OUTCOME_NOT_ATTEMPTED: the request was not resolved at all (see// LandConflict.reason).OUTCOME_NOT_ATTEMPTED = 5;- // OUTCOME_PROPOSED: a model resolved the file; the server never lands a- // model's resolution, it proposes it (see proposal).+ // OUTCOME_PROPOSED: a model resolved the file; the server lands a+ // model's resolution only after verify.command passed on the merged tree,+ // otherwise it proposes it (see proposal).OUTCOME_PROPOSED = 6;}// path is the path of the conflict in the merged tree.diff --git a/sdk/conflict.go b/sdk/conflict.goindex 4c0af30..512ca1f 100644--- a/sdk/conflict.go+++ b/sdk/conflict.go@@ -28,8 +28,9 @@ func LandConflict(err error) (*gitalypb.LandConflict, bool) {type ConflictSides struct {Base, Ours, Theirs []byteUnreadable map[string]string- // Proposal is the resolver's resolution of the file, when it made one: a model's, which the- // server never lands (OUTCOME_PROPOSED), or the rules' in a land that failed for another file.+ // Proposal is the resolver's resolution of the file, when it made one: a model's, which did not+ // land because no verification passed on it (OUTCOME_PROPOSED), or the rules' in a land that+ // failed for another file.// Land it, edited or not, as your own (ResolutionFromProposals).Proposal *Proposal}diff --git a/tools/graft-agents/protocol/protocol.go b/tools/graft-agents/protocol/protocol.goindex 4709fe9..169bd95 100644--- a/tools/graft-agents/protocol/protocol.go+++ b/tools/graft-agents/protocol/protocol.go@@ -151,8 +151,9 @@ type ResolvedFile struct {Action string `json:"action"`Mode string `json:"mode,omitempty"`Content []byte `json:"content"`- // Proposal marks a resolution made by a model (the llm strategy): the server never lands it,- // it proposes it to the agent (PLAN10 14).+ // Proposal marks a resolution made by a model (the llm strategy): the server lands it only if+ // verify.command passed on the tree that lands, and otherwise proposes it to the agent+ // (PLAN10 14, 15.9).Proposal bool `json:"proposal,omitempty"`}diff --git a/tools/graft-agents/resolve/llm_strategy.go b/tools/graft-agents/resolve/llm_strategy.goindex 711714a..15c6ba7 100644--- a/tools/graft-agents/resolve/llm_strategy.go+++ b/tools/graft-agents/resolve/llm_strategy.go@@ -181,7 +181,8 @@ func (s *llmStrategy) Resolve(ctx context.Context, f File, req protocol.Request)func (s *llmStrategy) syntaxChecked() {}// proposer marks a strategy whose results are proposals: those of a model, which the server-// proposes to the agent and never lands (PLAN10 14).+// proposes to the agent, and lands only when verify.command passed on the tree that lands+// (PLAN10 14, 15.9).type proposer interface{ proposes() }// proposes marks the llm answers as proposals (resolveFile).diff --git a/tools/graft-agents/resolve/resolve.go b/tools/graft-agents/resolve/resolve.goindex b2d8dbd..e6dbcdf 100644--- a/tools/graft-agents/resolve/resolve.go+++ b/tools/graft-agents/resolve/resolve.go@@ -450,7 +450,8 @@ func (r *Resolver) resolveFile(ctx context.Context, req protocol.Request, f Fileattempts = append(attempts, Attempt{Strategy: b.Name, Kind: AttemptCheckFailed, Err: err})continue}- // A model resolution is a proposal: the server never lands it (PLAN10 14). It is not+ // A model resolution is a proposal: the server lands it only after verify.command passed on+ // the tree that lands (PLAN10 14, 15.9). It is not// recorded either: a strategy that learns accepted answers (Recorder) would otherwise// replay a model's resolution later as a rule's._, proposal := b.Strategy.(proposer)