Git designed for AI agents.
docs: the gate counts findings in full, version skew, costs, wording
Federico Elia538b9e3b0c7c
diff --git a/README.md b/README.mdindex e86d145..c870952 100644--- a/README.md+++ b/README.md@@ -47,7 +47,7 @@ ideas, with normal Git repositories underneath.| **Always-online clients** that fetch exactly what they need | `ObjectService` serves single objects and sparse trees; `graft clone` and `graft sparse` materialize only the paths you ask for. || **Source control as an API**: history runs on the server | Blame, path history, ancestry and code search run server-side and are memoized as derived data, computed once per commit. || **Landing without a merge queue** | `UserLand` rebases a change onto the current tip and lands it with a compare-and-swap; concurrent lands on one branch are batched into a single reference update. |-| **Automatic conflict resolution** | A conflicting land is resolved by a resolver you choose (any LLM provider, a deterministic tool, or none) through a versioned, vendor-neutral protocol. A model's resolution is only a proposal: it lands by itself only if the `verify.command` you configure passes on the exact tree that would land, and otherwise the agent gets the conflict package with the proposal. The deterministic rules you configure need no such condition of their own, and are verified on the exact tree as well whenever you configure a `verify.command`. |+| **Automatic conflict resolution** | A conflicting land is resolved by a resolver you choose (any LLM provider, a deterministic tool, or none) through a versioned, vendor-neutral protocol. A model's resolution is only a proposal: it lands by itself only if the `verify.command` you configure passes on the exact tree that would land, and otherwise the agent gets the conflict package with the proposal. The deterministic rules you configure need no such condition of their own, and are verified on the exact tree as well when you turn the verification on (`verify.enabled`). || **Changeset log** | Every accepted reference update is recorded with its operation and actor in a dense, ordered log that clients list or watch live. || **Storage core: a database and object storage, stateless servers** | Each change is a changeset in a write-ahead log in SQLite or PostgreSQL, with a compare-and-swap per reference; packs live in object storage (file, S3, GCS, Azure). Server nodes are caches that any node can rebuild, so you add nodes to scale. || **Test after landing, revert the culprit** | `graft-verify` tests the tip, bisects a failure and reverts the change that broke it. |diff --git a/doc/graft/CONFIGURATION.md b/doc/graft/CONFIGURATION.mdindex 796d92f..2bfac8c 100644--- a/doc/graft/CONFIGURATION.md+++ b/doc/graft/CONFIGURATION.md@@ -240,7 +240,7 @@ are in [STORAGE.md](STORAGE.md).| `database` | string | required when on | `sqlite:///absolute/path.db`, or `postgres://user@host:port/db?sslmode=...` (`postgresql://` too). All nodes use the same database. Credentials are not written here: Postgres reads the `PG*` environment variables and `.pgpass`. The schema is created at startup. || `bucket` | string | required when on | gocloud URL of the object storage: `file:///absolute/dir`, `s3://bucket?region=...`, `gs://bucket` or `azblob://container`. All nodes use the same bucket. Credentials come from the cloud SDK's environment (`AWS_*`, `GOOGLE_APPLICATION_CREDENTIALS`, `AZURE_*`). || `node_id` | string | the host name | Name of this node in the database (changesets record it, and `graft storage state` shows it). Must not contain spaces or slashes. Give every node its own. |-| `sync_timeout` | string (Go duration) | `"10s"` | Budget of one sync round before a request in authoritative mode. A round that cannot reach the head in time keeps what it did: the local repository stays at the changeset it reached, the request fails with `Unavailable`, and the next request's round goes on from there. A round that cannot finish at all (for example because the database is unreachable or a local writer holds a reference lock) fails the request with `Unavailable` too. Concurrent requests share a round; a request that arrives during a round that read the head before it then runs one of its own, so it waits at most two rounds (and stops waiting with `Unavailable` after twice `sync_timeout` plus a second per round, should a round not end within its budget). Clients retry `Unavailable`: the SDK does it for every land (`LandOptions.RetryUnavailable`, see [LAND-BATCHING](LAND-BATCHING.md#limits)): the retry of a land whose response was lost answers `already_landed` from the server's record of the first land, rebase lands included (`SkipRebaseLands` turns the retry of rebase lands off for servers without that record). A client without the SDK must follow the same rules. Must be positive. |+| `sync_timeout` | string (Go duration) | `"10s"` | Budget of one sync round before a request in authoritative mode. A round that cannot reach the head in time keeps what it did: the local repository stays at the changeset it reached, the request fails with `Unavailable`, and the next request's round goes on from there. A round that cannot finish at all (for example because the database is unreachable or a local writer holds a reference lock) fails the request with `Unavailable` too. Concurrent requests share a round; a request that arrives during a round that read the head before it then runs one of its own, so it waits at most two rounds (and stops waiting with `Unavailable` after twice `sync_timeout` plus a second per round, should a round not end within its budget). Clients retry `Unavailable`: the SDK does it for every land (`LandOptions.RetryUnavailable`, see [LAND-BATCHING](LAND-BATCHING.md#limits)): the retry of a land whose response was lost answers `already_landed` from the server's record of the first land, rebase lands included (that record needs a server released after v0.2.0 with its changeset log on, `[graft.changelog]` not disabled; against another server the retry of a lost rebase land replays the change, and `SkipRebaseLands` turns the retry of rebase lands off). A client without the SDK must follow the same rules. Must be positive. || `shadow_timeout` | string (Go duration) | `"5s"` | Budget of the shadow-mode work of one reference transaction, for its Preparing phase (pack build and upload, before git locks the references) and its Prepared phase together. When it runs out, the work is abandoned, logged and counted, and the git write proceeds. Must be positive. || `gc_grace` | string (Go duration) | `"1h"` | How long garbage collection waits before it deletes a pack it observed unreferenced, and how old a checkpoint must be before the packs it covers are deleted. Must be greater than `sync_timeout + gc_margin`. || `gc_margin` | string (Go duration) | `"5m"` | Safety margin of the writers' deadline: a write uploads its pack and commits its changeset within `gc_grace - gc_margin`, else it fails. It also covers the difference between a node's clock and the database's. Must be positive. |diff --git a/doc/graft/PLAN10.md b/doc/graft/PLAN10.mdindex b7e8354..e614ac8 100644--- a/doc/graft/PLAN10.md+++ b/doc/graft/PLAN10.md@@ -941,7 +941,7 @@ end of this section.| writer | proposes a resolution per region | a model, configurable (15.13) | 15.10: the elements kept and removed, 15.12 || checks and repair | refuse a broken answer, repair it once | deterministic checks | 15.10 || report | tells the agent what the answer keeps or drops | deterministic | 15.6: it separates (43% against 60% right), it does not gate |-| agree_with (optional) | holds a proposal back unless a second model from another vendor gives the same file and both reports are clean | code (`llmStrategy`), configured by the operator | 15.12: G1, 59.8% of the files landed with 15.5% wrong under two judges and at most 15.5% under a third, before the operator's tests |+| agree_with (optional) | holds a proposal back unless a second model from another vendor gives the same file and neither composition check found a drop or a duplicate | code (`llmStrategy`), configured by the operator | 15.12: G1, 59.8% of the files landed with 15.5% wrong under two judges and at most 15.5% under a third, before the operator's tests || verify | runs the operator's build and tests on the exact tree | the operator's command | 15.9, 14 || decideLand | the one landing rule | code | 15.4 |@@ -1142,7 +1142,7 @@ whole file (`agree_with`, 15.13): the one filter that chose better proposals tha| regions (the fallback of git's, and the assembly of the file) | `llmRegions`, `llmJoin` | tools/.../resolve/llm_core.go, llm_gitmerge.go || one call per region, its checks | `llmPatchSystem`, `llmPatchUser`, `llmPatchAnswer`, `llmCheckPatched`, `llmCheckAnswer` | tools/.../resolve/llm_patch.go, llm_core.go || proposal mark, no recording | `proposer`, `Resolver.resolveFile` | tools/.../resolve/llm_strategy.go, resolve.go |-| agreement gate: second model, comparison, hold | `llmStrategy.Resolve`, `llmSide.start`, `llmStrategy.agreement`, `LLMNormalize`, `LLMReportClean`, `Resolution.Hold`, `ResponseFile.Hold` (a held file is a model's), `landAnswerHolds`, `decideLand`, `landSetOutcomes` | tools/.../resolve/llm_strategy.go, llm_core.go, resolve.go; graft/resolver/protocol.go; operations/land_proposals.go, land_resolve.go |+| agreement gate: second model, comparison, hold | `llmStrategy.Resolve`, `llmSide.start`, `llmStrategy.agreement`, `LLMNormalize`, `LLMResult.Findings`, `Resolution.Hold`, `ResponseFile.Hold` (a held file is a model's), `landAnswerHolds`, `decideLand`, `landSetOutcomes` | tools/.../resolve/llm_strategy.go, llm_core.go, resolve.go; graft/resolver/protocol.go; operations/land_proposals.go, land_resolve.go || verify.command's checkout | `landCheckoutTree`, `landMaterializer.add`, `landCheckMaterialized`, `landRemoveCheckout`, `resolver.IsGitDirName` | operations/land_verify.go; graft/resolver/validate.go || merge commit, trailers | `landWriteResolution` | operations/land_resolve.go || PACKAGE | `landWithPackage`, `landListing.conflictPackage` | operations/land_package.go |@@ -1699,7 +1699,7 @@ five times a normal call (an empty answer is billed, and a retry is four attemptand 29 agent-PR answers unjudged and 3 agent-PR calls still empty. A third pass on the OSS set only (approved) judged 11 of the24 and stopped at its cap again: 13 OSS answers are unjudged and 2 calls are empty, and the agent-PR set stays as it was afterthe second pass. All of those are counted as wrong here, so its figures are a ceiling. **Judge 3 alone is the number to-quote**, with the majority of judges 1, 2 and 3 beside it. It cost $0.74 in all.+quote**, with the majority of judges 1, 2 and 3 beside it. It cost $0.76 booked ($0.74 by the key).**What the writer's proposals are worth** (the prompt of record, Luna, effort low; "right" = the strict judge; theinterval is Wilson 95%; accepted = the answer passed the rules and the syntax check):@@ -1821,7 +1821,9 @@ the tuning data (the stratified 100 and the 300) and run once on data they were(family-stratified, spring-boot capped at 60; manifest-heldout-487) and the 284 replayed agent-PR files (manifest-agent-284).* **G1** = the prompt of record with Luna (`openai/gpt-6-luna`, effort low) and with glm-5.3-flash (`z-ai/glm-5.3-flash`pinned to Z.AI, effort low) both accept the byte-identical answer after the judge's normalization, both composition- reports are clean, and the verify simulator does not fail it.+ reports are clean, and the verify simulator does not fail it. ("Clean" was measured on the lines of the report, which is cut+ at 16 lines; the gate in the code counts the drops and duplicates of every region (`LLMResult.Findings`), so it can hold+ slightly more than the measured G1, on the few files whose report is longer. That difference was not measured.)* **G2** = G1 and the file is a java file. **G0** = Luna alone, accepted, report clean, simulator not failed.The goal counts as met by a gate only if coverage >= 20% and the Wilson upper bound of the wrong rate <= 5% under judge A on aheld-out set. (The first registration said 307 agent-PR files; the replay counted 23 files twice, and the set is 284.)@@ -1912,7 +1914,7 @@ read from the key before and after each run and agrees, except where stated.| 20 | held-out runs of the prompt of record: Luna and glm-5.3-flash on 487 OSS files and on 284 agent-PR files | manifest-heldout-487, manifest-agent-284 | 7fd0ab2's tools tree (8812cee) | $0.300 + $0.302 + $0.227 + $0.237 || 20 | judge 1 (glm-5.3-flash) on the held-out answers | the answers of Luna | scripts outside the repository | $0.025 + $0.021 || 20 | judge 2 (openai/gpt-6.1-sol, effort low) on the answers landed by G0 | OSS held-out 359, agent-PR 210 | scripts outside the repository | $0.239 + $0.268 |-| 20 | judge 3 (mistralai/mistral-large-4-0, unrelated vendor) on the answers landed by G1: a first pass stopped at its cap of $0.25 per set, a second pass (retry of the empty answers with a backoff, and the unjudged) at +$0.03 and +$0.09, a third on the OSS set only at +$0.10; 13 OSS and 29 agent-PR answers unjudged and 2 OSS and 3 agent-PR calls empty at the end, counted as wrong | OSS held-out 291, agent-PR 130 | scripts outside the repository | $0.404 + $0.354 (the key: $0.74) |+| 20 | judge 3 (mistralai/mistral-large-4-0, unrelated vendor) on the answers landed by G1: a first pass stopped at its cap of $0.25 per set, a second pass (retry of the empty answers with a backoff, and the unjudged) at +$0.03 and +$0.09, a third on the OSS set only (cap +$0.10, +$0.106 booked); 13 OSS and 29 agent-PR answers unjudged and 2 OSS and 3 agent-PR calls empty at the end, counted as wrong | OSS held-out 291, agent-PR 130 | scripts outside the repository | $0.404 + $0.354 = $0.758 booked (the key: $0.739) || 20 | the agree_with gate through fake providers and the real graft-resolve with two fake command models | tests only | 3cc898c | $0 |**Why a screened model is skipped.** The screening is a smoke check of five cases (a go, a java, a python, adiff --git a/doc/graft/RESOLVE.md b/doc/graft/RESOLVE.mdindex ff0cd2c..bd488a6 100644--- a/doc/graft/RESOLVE.md+++ b/doc/graft/RESOLVE.md@@ -727,7 +727,9 @@ one that is proposed. It lands only if all three hold:lines elsewhere, the order of lines and every other character must be equal. It is the normalization that themeasurements and the judges used, and it decides only whether the models agree: the bytes that land, and that theverification runs on, are the first model's, exactly as it wrote them;-2. **the composition report of both is clean** (no line saying that the answer drops or duplicates a change of a side);+2. **the composition check of both found nothing**: not one drop of a change of a side and not one duplicate, counted in+ full from the check of every region (`LLMResult.Findings`), not read from the text of the report, which is cut at 16+ lines;3. **`verify.command` passes** on the exact tree, as for every model resolution: `verify.enabled = true` is mandatory(the gate does not replace it, it chooses the proposals worth verifying).@@ -740,7 +742,8 @@ vendors: two models of one family make the same mistakes. A `[strategy.llm.budge`[strategy.llm.agree_with.budget]` beside it (a cap that covers one of two models is not a cap), and the spend of bothis counted for the file. The bench (`graft-resolve bench run`) runs the same code, prices the two models fromOpenRouter and splits `--max-usd` between them; its report counts the held answers apart. Upgrade `graft-resolve` and the-server together: an older server does not know `hold` and would verify a held proposal like any other.+server together (see "Version skew" above: a `graft-resolve` with `agree_with` gives no model's resolution to a server that+does not announce `hold`).What it was worth, measured on 487 held-out files of 13 open-source repositories (Luna and glm-5.3-flash, with theverify simulation of the bench in place of the operator's tests): 59.8% of the files landed, and 15.5% of those differed