Skip to content

Git designed for AI agents.

graft: agents walk no reference under refs/graft/, whatever asks

Federico Eliab985c820e0f9
diff --git a/doc/graft/AGENT-SECURITY.md b/doc/graft/AGENT-SECURITY.md
index 09cdb35..2f311e4 100644
--- a/doc/graft/AGENT-SECURITY.md
+++ b/doc/graft/AGENT-SECURITY.md
@@ -65,11 +65,33 @@ reference under it (including the short forms `graft/proposals/...` and
`graft/green/...`, which git completes to the server's namespaces; other
`graft/...` short names, such as the workspace's temporary branches, are
branches), and removes such references from every response; git upload-pack and
-receive-pack hide the prefix from agents. A test classifies every string or
-bytes field of every RPC agents may call by default, so a new field cannot
-escape the filter; RPCs added with `graft.agent_rpcs` are checked by field
-name. Other callers still see graft-verify's green references, which are
-documented for consumers.
+receive-pack hide the prefix from agents. Git's pseudo references reach every
+reference without naming one, so every git command of an agent that walks them
+(`rev-list`, `log`, `show`, `rev-parse`, `bundle`, `fast-export`,
+`format-patch`) gets `--exclude-hidden=uploadpack` before each `--all` and
+`--glob`, in its arguments and on its stdin, with `uploadpack.hideRefs` set to
+`refs/graft/`: `--all` and `--glob=refs/*` in `ListCommits` or `ListBlobs`,
+`FindCommits` with `all`, or any other RPC or field that asks for such a walk,
+leave the namespace out (`internal/git/gitcmd/agent_walk.go`). `--alternate-refs`
+walks the references of the object pool, which mirrors the member's references,
+`refs/graft/` included: for an agent it walks only the pool's branches and tags
+(`core.alternateRefsPrefixes`). A test classifies,
+by message and field, every string, bytes, bool and enum field of every RPC
+agents may call by default, so a new field cannot escape the filter; RPCs added
+with `graft.agent_rpcs` are checked by field name. Other callers still see
+graft-verify's green references, which are documented for consumers.
+
+What the hiding covers, and what it does not: it covers the reference **names**
+under `refs/graft/` and the reachability that goes through them. An agent can
+neither address those references nor walk to their objects through them, so it
+can neither depend on a proposal reference nor learn of one from a listing. It
+does **not** make the objects secret. A proposal's commits and blobs are
+ordinary objects of the repository: `ListAllBlobs` and `ListAllLFSPointers`
+enumerate every blob, `GetObjects` reads any object by its ID, and an agent with
+access to the repository can read their content, a proposal's message included. The goal is
+that agents can neither address nor depend on `refs/graft/`, not object
+confidentiality; anything that must stay secret from an agent must not be
+written into a repository the agent can read.
Whatever the RPC and whatever the policy says, an agent never creates,
updates or deletes a reference under `refs/graft/`: the reference-transaction
@@ -137,6 +159,17 @@ A denial by policy is a `PermissionDenied` error. Standalone denial messages
do not start with `GitLab:`, so they are not mistaken for a GitLab hook
rejection.
+Two known limits, accepted:
+
+- A land with `resolve_conflicts` is checked against the standalone policy
+ before the resolver is called, so an actor the policy denies gets nothing
+ resolved. A shared-token caller whose GitLab user the GitLab pre-receive
+ check would deny is not stopped there: that check runs when the references
+ are updated, so the resolver is called, and the land is then refused.
+- A branch that is a symbolic reference is judged by its own name, not by the
+ name of the branch it points to. Creating a symbolic branch needs the rights
+ of an operator.
+
## Resolution and verification are opt-in
- Automatic conflict resolution is off by default and can be limited to
diff --git a/doc/graft/CONFIGURATION.md b/doc/graft/CONFIGURATION.md
index 277c368..84664ed 100644
--- a/doc/graft/CONFIGURATION.md
+++ b/doc/graft/CONFIGURATION.md
@@ -112,7 +112,7 @@ The policy is read at startup. A change needs a restart.
| Key | Type | Default | Description |
|---|---|---|---|
-| `name` | string | required | Identity of the agent. It is the identity the token maps to, the actor the policy decides on for the token's requests, the actor that the changeset log records for the token's updates and, in standalone mode, the user ID (`gl_id`, `gl_username`) of every request made with the token, whatever the client sent. The name `shared` is reserved for the shared token. |
+| `name` | string | required | Identity of the agent. It is the identity the token maps to, the actor the policy decides on for the token's requests, the actor that the changeset log records for the token's updates and, in standalone mode, the user ID (`gl_id`, `gl_username`) of every request made with the token, whatever the client sent. The name `shared` is reserved for the shared token, and `proposals` and `green` for the server's references under `refs/graft/`: the workspace's temporary branches of an agent are `graft/<name>/...`, which for those names an agent may not use. |
| `token` | string | — | The secret. Set either `token` or `token_file`. |
| `token_file` | string | — | Path to a file that holds the secret, surrounding whitespace removed. Read at startup. |
diff --git a/internal/command/command.go b/internal/command/command.go
index 0fa4520..97d0a40 100644
--- a/internal/command/command.go
+++ b/internal/command/command.go
@@ -388,14 +388,23 @@ func New(ctx context.Context, logger log.Logger, nameAndArgs []string, opts ...O
// If requested, we will set up the command such that `Write()` can be called on it directly. Otherwise,
// we simply pass as stdin whatever the user has asked us to set up. If no `stdin` was set up, the command
// will implicitly read from `/dev/null`.
- if _, ok := cfg.stdin.(stdinSentinel); ok {
+ switch _, ok := cfg.stdin.(stdinSentinel); {
+ case ok && cfg.stdinFilter != nil:
+ pipe, err := cmd.StdinPipe()
+ if err != nil {
+ return nil, fmt.Errorf("creating stdin pipe: %w", err)
+ }
+ command.writer = filteredStdin(pipe, cfg.stdinFilter)
+ case ok:
pipe, err := cmd.StdinPipe()
if err != nil {
return nil, fmt.Errorf("creating stdin pipe: %w", err)
}
command.writer = pipe
- } else {
+ case cfg.stdin != nil && cfg.stdinFilter != nil:
+ cmd.Stdin = cfg.stdinFilter(cfg.stdin)
+ default:
cmd.Stdin = cfg.stdin
}
@@ -824,6 +833,26 @@ func (c *Command) exitStatus() (int, syscall.Signal) {
return state.ExitCode(), 0
}
+// filteredStdin returns the writer of a command's stdin that reaches pipe, the stdin of the
+// process, through filter. A goroutine copies the filtered stream into pipe and closes pipe when
+// the writer is closed. When the process stops reading, the copy fails, and the writes after that
+// fail with its error, as writes to pipe itself would: a writer never blocks on a process that is
+// gone.
+func filteredStdin(pipe io.WriteCloser, filter func(io.Reader) io.Reader) io.WriteCloser {
+ r, w := io.Pipe()
+ go func() {
+ _, err := io.Copy(pipe, filter(r))
+ if closeErr := pipe.Close(); err == nil {
+ err = closeErr
+ }
+ if err == nil {
+ err = io.ErrClosedPipe
+ }
+ r.CloseWithError(err)
+ }()
+ return w
+}
+
type stdinSentinel struct{}
func (stdinSentinel) Read([]byte) (int, error) {
diff --git a/internal/command/command_test.go b/internal/command/command_test.go
index 4300dc9..19063c0 100644
--- a/internal/command/command_test.go
+++ b/internal/command/command_test.go
@@ -15,6 +15,7 @@ import (
"sync"
"syscall"
"testing"
+ "testing/iotest"
"time"
"github.com/feder-cr/git-ai/internal/cgroups"
@@ -219,6 +220,56 @@ func TestNew_setupStdin(t *testing.T) {
require.Equal(t, stdin, buf.String())
}
+// TestNew_stdinFilter: the command reads its stdin through the filter, both a reader given to it
+// and what is written to it; a writer whose process stopped reading gets an error and never blocks.
+func TestNew_stdinFilter(t *testing.T) {
+ t.Parallel()
+
+ ctx := testhelper.Context(t)
+ upper := func(r io.Reader) io.Reader {
+ raw, err := io.ReadAll(r)
+ if err != nil {
+ return iotest.ErrReader(err)
+ }
+ return strings.NewReader(strings.ToUpper(string(raw)))
+ }
+
+ var buf bytes.Buffer
+ cmd, err := New(ctx, testhelper.SharedLogger(t), []string{"cat"}, WithStdin(strings.NewReader("given")), WithStdinFilter(upper), WithStdout(&buf))
+ require.NoError(t, err)
+ require.NoError(t, cmd.Wait())
+ require.Equal(t, "GIVEN", buf.String())
+
+ buf.Reset()
+ cmd, err = New(ctx, testhelper.SharedLogger(t), []string{"cat"}, WithSetupStdin(), WithStdinFilter(upper), WithStdout(&buf))
+ require.NoError(t, err)
+ _, err = fmt.Fprintf(cmd, "written")
+ require.NoError(t, err)
+ require.NoError(t, cmd.Wait())
+ require.Equal(t, "WRITTEN", buf.String())
+
+ // A process that exits without reading: the writes fail once the copy into its stdin failed.
+ passThrough := func(r io.Reader) io.Reader { return r }
+ cmd, err = New(ctx, testhelper.SharedLogger(t), []string{"true"}, WithSetupStdin(), WithStdinFilter(passThrough))
+ require.NoError(t, err)
+ done := make(chan error, 1)
+ go func() {
+ chunk := bytes.Repeat([]byte("x"), 1<<16)
+ var err error
+ for i := 0; i < 1<<10 && err == nil; i++ {
+ _, err = cmd.Write(chunk)
+ }
+ done <- err
+ }()
+ select {
+ case err := <-done:
+ require.Error(t, err)
+ case <-time.After(30 * time.Second):
+ require.FailNow(t, "the writes blocked on a process that exited")
+ }
+ require.NoError(t, cmd.Wait())
+}
+
func TestCommand_closeStdin(t *testing.T) {
t.Parallel()
diff --git a/internal/command/option.go b/internal/command/option.go
index 9cf2abd..09d236a 100644
--- a/internal/command/option.go
+++ b/internal/command/option.go
@@ -11,6 +11,7 @@ import (
type config struct {
stdin io.Reader
+ stdinFilter func(io.Reader) io.Reader
stdout io.Writer
stderr io.Writer
dir string
@@ -51,6 +52,14 @@ func WithSetupStdin() Option {
}
}
+// WithStdinFilter makes the command read its stdin through filter: the reader given with
+// WithStdin, or what is written to the command when it was set up with WithSetupStdin.
+func WithStdinFilter(filter func(io.Reader) io.Reader) Option {
+ return func(cfg *config) {
+ cfg.stdinFilter = filter
+ }
+}
+
// WithStdout sets up the command to write standard output to the given writer.
func WithStdout(stdout io.Writer) Option {
return func(cfg *config) {
diff --git a/internal/git/gitcmd/agent_walk.go b/internal/git/gitcmd/agent_walk.go
new file mode 100644
index 0000000..bd025ea
--- /dev/null
+++ b/internal/git/gitcmd/agent_walk.go
@@ -0,0 +1,131 @@
+package gitcmd
+
+import (
+ "bufio"
+ "context"
+ "io"
+ "strings"
+
+ "github.com/feder-cr/git-ai/internal/git"
+ "github.com/feder-cr/git-ai/internal/graft/agentrefs"
+ "github.com/feder-cr/git-ai/internal/graft/identity"
+)
+
+// An agent does not see the references under refs/graft/ (agentrefs). The requests of agents are
+// checked for revisions that name them, but git's pseudo references reach every reference without
+// naming one: --all, and --glob with a pattern such as refs/*. So every command of an agent that
+// walks them (scWalksRefs), whatever RPC or request field asked for the walk, gets
+// --exclude-hidden=uploadpack before each --all and --glob, in its arguments and on its stdin, with
+// uploadpack.hideRefs set to refs/graft/: git then leaves the hidden references out of each walk.
+// --exclude-hidden applies to the next pseudo reference only, hence one before each. --branches,
+// --tags and --remotes reach no reference under refs/graft/, and git refuses --exclude-hidden
+// before them. --alternate-refs walks the references of the object pool, which mirrors the
+// member's references under git.ObjectPoolRefNamespace, refs/graft/ included: an agent's walk uses
+// only the pool's branches and tags (core.alternateRefsPrefixes).
+
+// excludeHiddenFlag makes git leave the references hidden from upload-pack out of the next --all
+// or --glob.
+const excludeHiddenFlag = "--exclude-hidden=uploadpack"
+
+// agentWalkConfig is the configuration of an agent's command that walks git's pseudo references:
+// the references --exclude-hidden=uploadpack leaves out, and the references of the object pool
+// that --alternate-refs walks.
+func agentWalkConfig() []GlobalOption {
+ return []GlobalOption{
+ ConfigPair{Key: "uploadpack.hideRefs", Value: agentrefs.HiddenPrefix},
+ ConfigPair{Key: "core.alternateRefsPrefixes", Value: git.ObjectPoolRefNamespace + "/heads/ " + git.ObjectPoolRefNamespace + "/tags/"},
+ }
+}
+
+// agentRefWalk reports whether sc is a command of an agent that walks git's pseudo references.
+func agentRefWalk(ctx context.Context, sc Command) bool {
+ id, ok := identity.FromContext(ctx)
+ if !ok || id.Source != identity.SourceAgent {
+ return false
+ }
+ description, ok := commandDescriptions[sc.Name]
+ return ok && description.flags&scWalksRefs != 0
+}
+
+// walksAllRefs reports whether a pseudo reference argument reaches every reference, or every one
+// a pattern matches: --all, --glob=<pattern>, or --glob before its pattern.
+func walksAllRefs(arg string) bool {
+ return arg == "--all" || arg == "--glob" || strings.HasPrefix(arg, "--glob=")
+}
+
+// excludeHiddenRefs returns sc with excludeHiddenFlag before each pseudo reference that reaches
+// every reference, among its options and its arguments. An option is matched as a whole, so that
+// the value of an option, which git does not take for a pseudo reference, is left alone.
+func excludeHiddenRefs(sc Command) Command {
+ flags := make([]Option, 0, len(sc.Flags)+1)
+ for _, o := range sc.Flags {
+ switch f := o.(type) {
+ case Flag:
+ if walksAllRefs(f.Name) {
+ flags = append(flags, Flag{Name: excludeHiddenFlag})
+ }
+ case ValueFlag:
+ if f.Name == "--glob" {
+ flags = append(flags, Flag{Name: excludeHiddenFlag})
+ }
+ }
+ flags = append(flags, o)
+ }
+ sc.Flags = flags
+
+ args := make([]string, 0, len(sc.Args))
+ for _, a := range sc.Args {
+ if walksAllRefs(a) {
+ args = append(args, excludeHiddenFlag)
+ }
+ args = append(args, a)
+ }
+ sc.Args = args
+ return sc
+}
+
+// readsRevisionsFromStdin reports whether sc reads revisions, and so pseudo references, from its
+// stdin.
+func readsRevisionsFromStdin(sc Command) bool {
+ for _, o := range sc.Flags {
+ if f, ok := o.(Flag); ok && f.Name == "--stdin" {
+ return true
+ }
+ }
+ return false
+}
+
+// excludeHiddenRefsOnStdin is the stdin of a command that reads revisions there, one per line,
+// with excludeHiddenFlag on a line of its own before each --all and --glob=<pattern>. A line "--"
+// ends the revisions: paths follow, which are left alone. It reads r as it is read, a line at a
+// time, and runs nothing of its own.
+func excludeHiddenRefsOnStdin(r io.Reader) io.Reader {
+ return &excludeHiddenReader{in: bufio.NewReader(r)}
+}
+
+// excludeHiddenReader is the reader of excludeHiddenRefsOnStdin.
+type excludeHiddenReader struct {
+ in *bufio.Reader
+ pending []byte
+ paths bool
+ err error
+}
+
+func (r *excludeHiddenReader) Read(p []byte) (int, error) {
+ for len(r.pending) == 0 {
+ if r.err != nil {
+ return 0, r.err
+ }
+ var line string
+ line, r.err = r.in.ReadString('\n')
+ revision := strings.TrimSuffix(line, "\n")
+ if !r.paths && (revision == "--all" || strings.HasPrefix(revision, "--glob=")) {
+ r.pending = append(r.pending, excludeHiddenFlag+"\n"...)
+ }
+ r.paths = r.paths || revision == "--"
+ r.pending = append(r.pending, line...)
+ }
+ n := copy(p, r.pending)
+ r.pending = r.pending[n:]
+ return n, nil
+}
diff --git a/internal/git/gitcmd/agent_walk_git_test.go b/internal/git/gitcmd/agent_walk_git_test.go
new file mode 100755
index 0000000..6e8169b
--- /dev/null
+++ b/internal/git/gitcmd/agent_walk_git_test.go
@@ -0,0 +1,152 @@
+package gitcmd_test
+
+import (
+ "bytes"
+ "context"
+ "os"
+ "path/filepath"
+ "strings"
+ "testing"
+ "time"
+
+ "github.com/stretchr/testify/require"
+
+ "github.com/feder-cr/git-ai/internal/git"
+ "github.com/feder-cr/git-ai/internal/git/gitcmd"
+ "github.com/feder-cr/git-ai/internal/git/gittest"
+ "github.com/feder-cr/git-ai/internal/graft/identity"
+ "github.com/feder-cr/git-ai/internal/testhelper"
+ "github.com/feder-cr/git-ai/internal/testhelper/testcfg"
+)
+
+// TestAgentWalk_leavesHiddenRefsOut: whatever asks git to walk every reference, as an option, an
+// argument or a line on stdin, the walk of an agent's command leaves out the references under
+// refs/graft/, and another caller's walk still reaches them. The commit only a proposal reference
+// reaches stands for a proposal.
+func TestAgentWalk_leavesHiddenRefsOut(t *testing.T) {
+ t.Parallel()
+
+ ctx := testhelper.Context(t)
+ cfg := testcfg.Build(t)
+ repo, repoPath := gittest.CreateRepository(t, ctx, cfg, gittest.CreateRepositoryConfig{SkipCreationViaService: true})
+ main := gittest.WriteCommit(t, cfg, repoPath, gittest.WithBranch("main"),
+ gittest.WithTreeEntries(gittest.TreeEntry{Mode: "100644", Path: "a", Content: "a\n"}))
+ secret := gittest.WriteCommit(t, cfg, repoPath, gittest.WithParents(main),
+ gittest.WithReference("refs/graft/proposals/0123456789abcdef/1"), gittest.WithMessage("SECRET proposal"),
+ gittest.WithTreeEntries(gittest.TreeEntry{Mode: "100644", Path: "a", Content: "proposed\n"}))
+ factory := gittest.NewCommandFactory(t, cfg)
+ agent := identity.NewContext(ctx, identity.Identity{Name: "agent-a", Source: identity.SourceAgent})
+
+ run := func(t *testing.T, ctx context.Context, sc gitcmd.Command, stdin string) string {
+ t.Helper()
+ var stdout, stderr bytes.Buffer
+ opts := []gitcmd.CmdOpt{gitcmd.WithStdout(&stdout), gitcmd.WithStderr(&stderr)}
+ if stdin != "" {
+ opts = append(opts, gitcmd.WithStdin(strings.NewReader(stdin)))
+ }
+ cmd, err := factory.New(ctx, repo, sc, opts...)
+ require.NoError(t, err)
+ require.NoError(t, cmd.Wait(), "stderr: %s", stderr.String())
+ return stdout.String()
+ }
+
+ for _, tc := range []struct {
+ desc string
+ sc gitcmd.Command
+ stdin string
+ seen string
+ }{
+ {desc: "rev-list --all", sc: gitcmd.Command{Name: "rev-list", Flags: []gitcmd.Option{gitcmd.Flag{Name: "--all"}}}, seen: secret.String()},
+ {desc: "rev-list --all as an argument", sc: gitcmd.Command{Name: "rev-list", Args: []string{"--all"}}, seen: secret.String()},
+ {desc: "rev-list --glob=refs/*", sc: gitcmd.Command{Name: "rev-list", Args: []string{"--glob=refs/*"}}, seen: secret.String()},
+ {desc: "rev-list --not --all", sc: gitcmd.Command{Name: "rev-list", Args: []string{secret.String(), "--not", "--all"}}, seen: ""},
+ {desc: "rev-list --all on stdin", sc: gitcmd.Command{Name: "rev-list", Flags: []gitcmd.Option{gitcmd.Flag{Name: "--stdin"}}}, stdin: "--all\n", seen: secret.String()},
+ {desc: "rev-list --glob= on stdin", sc: gitcmd.Command{Name: "rev-list", Flags: []gitcmd.Option{gitcmd.Flag{Name: "--stdin"}}}, stdin: "--glob=refs/graft/*\n", seen: secret.String()},
+ {desc: "log --all", sc: gitcmd.Command{Name: "log", Flags: []gitcmd.Option{gitcmd.Flag{Name: "--format=%H"}, gitcmd.Flag{Name: "--all"}}}, seen: secret.String()},
+ {desc: "show --all", sc: gitcmd.Command{Name: "show", Flags: []gitcmd.Option{gitcmd.Flag{Name: "--no-patch"}, gitcmd.Flag{Name: "--format=%H"}, gitcmd.Flag{Name: "--all"}}}, seen: secret.String()},
+ {desc: "rev-parse --all", sc: gitcmd.Command{Name: "rev-parse", Flags: []gitcmd.Option{gitcmd.Flag{Name: "--all"}}}, seen: secret.String()},
+ {desc: "fast-export --all", sc: gitcmd.Command{Name: "fast-export", Flags: []gitcmd.Option{gitcmd.Flag{Name: "--all"}}}, seen: "SECRET proposal"},
+ {desc: "format-patch --all", sc: gitcmd.Command{Name: "format-patch", Flags: []gitcmd.Option{gitcmd.Flag{Name: "--stdout"}, gitcmd.Flag{Name: "--all"}}}, seen: "SECRET proposal"},
+ {desc: "bundle create --all", sc: gitcmd.Command{Name: "bundle", Action: "create", Flags: []gitcmd.Option{gitcmd.OutputToStdout, gitcmd.Flag{Name: "--all"}}}, seen: "refs/graft/"},
+ } {
+ t.Run(tc.desc, func(t *testing.T) {
+ agentOut := run(t, agent, tc.sc, tc.stdin)
+ otherOut := run(t, ctx, tc.sc, tc.stdin)
+ if tc.seen == "" {
+ // The proposal is no reference to an agent: it is new to it.
+ require.Contains(t, agentOut, secret.String())
+ require.Empty(t, otherOut)
+ return
+ }
+ require.NotContains(t, agentOut, tc.seen)
+ require.Contains(t, otherOut, tc.seen)
+ })
+ }
+
+ t.Run("--branches is left alone", func(t *testing.T) {
+ out := run(t, agent, gitcmd.Command{Name: "rev-list", Args: []string{"--branches"}}, "")
+ require.Contains(t, out, main.String())
+ })
+
+ t.Run("--alternate-refs walks the pool's branches and tags only", func(t *testing.T) {
+ member, memberPath := gittest.CreateRepository(t, ctx, cfg, gittest.CreateRepositoryConfig{SkipCreationViaService: true})
+ _, poolPath := gittest.CreateRepository(t, ctx, cfg, gittest.CreateRepositoryConfig{SkipCreationViaService: true})
+ base := gittest.WriteCommit(t, cfg, poolPath, gittest.WithReference(git.ObjectPoolRefNamespace+"/heads/main"))
+ mirrored := gittest.WriteCommit(t, cfg, poolPath, gittest.WithParents(base), gittest.WithMessage("mirrored proposal"),
+ gittest.WithReference(git.ObjectPoolRefNamespace+"/graft/proposals/0123456789abcdef/1"))
+ require.NoError(t, os.WriteFile(filepath.Join(memberPath, "objects", "info", "alternates"), []byte(filepath.Join(poolPath, "objects")+"\n"), 0o644))
+
+ walk := func(ctx context.Context) string {
+ var stdout, stderr bytes.Buffer
+ cmd, err := factory.New(ctx, member, gitcmd.Command{Name: "rev-list", Args: []string{"--alternate-refs"}},
+ gitcmd.WithStdout(&stdout), gitcmd.WithStderr(&stderr))
+ require.NoError(t, err)
+ require.NoError(t, cmd.Wait(), "stderr: %s", stderr.String())
+ return stdout.String()
+ }
+ agentOut := walk(agent)
+ require.Contains(t, agentOut, base.String())
+ require.NotContains(t, agentOut, mirrored.String())
+ require.Contains(t, walk(ctx), mirrored.String())
+ })
+
+ t.Run("revisions written to stdin", func(t *testing.T) {
+ var stdout bytes.Buffer
+ cmd, err := factory.New(agent, repo, gitcmd.Command{Name: "rev-list", Flags: []gitcmd.Option{gitcmd.Flag{Name: "--stdin"}}},
+ gitcmd.WithSetupStdin(), gitcmd.WithStdout(&stdout))
+ require.NoError(t, err)
+ _, err = cmd.Write([]byte("--glob=refs/*\n"))
+ require.NoError(t, err)
+ require.NoError(t, cmd.Wait())
+ require.Contains(t, stdout.String(), main.String())
+ require.NotContains(t, stdout.String(), secret.String())
+ })
+
+ t.Run("a writer to a git that stopped reading does not block", func(t *testing.T) {
+ var stderr bytes.Buffer
+ cmd, err := factory.New(agent, repo, gitcmd.Command{Name: "rev-list", Flags: []gitcmd.Option{gitcmd.Flag{Name: "--stdin"}}},
+ gitcmd.WithSetupStdin(), gitcmd.WithStderr(&stderr))
+ require.NoError(t, err)
+
+ done := make(chan error, 1)
+ go func() {
+ line := []byte(strings.Repeat("0", 39) + "1\n")
+ var err error
+ for i := 0; i < 1<<16 && err == nil; i++ {
+ if i == 0 {
+ _, err = cmd.Write([]byte("no-such-revision\n"))
+ continue
+ }
+ _, err = cmd.Write(line)
+ }
+ done <- err
+ }()
+ select {
+ case <-done:
+ case <-time.After(30 * time.Second):
+ require.FailNow(t, "the writes blocked on a git that exited")
+ }
+ require.Error(t, cmd.Wait())
+ require.Contains(t, stderr.String(), "no-such-revision")
+ })
+}
diff --git a/internal/git/gitcmd/agent_walk_test.go b/internal/git/gitcmd/agent_walk_test.go
new file mode 100755
index 0000000..fb82e18
--- /dev/null
+++ b/internal/git/gitcmd/agent_walk_test.go
@@ -0,0 +1,95 @@
+package gitcmd
+
+import (
+ "context"
+ "errors"
+ "io"
+ "strings"
+ "testing"
+ "testing/iotest"
+
+ "github.com/stretchr/testify/require"
+
+ "github.com/feder-cr/git-ai/internal/graft/identity"
+)
+
+func agentWalkContext() context.Context {
+ return identity.NewContext(context.Background(), identity.Identity{Name: "agent-a", Source: identity.SourceAgent})
+}
+
+// TestAgentRefWalk: only an agent's commands that walk git's pseudo references are rewritten; a
+// command whose --all means something else (merge-base) is not.
+func TestAgentRefWalk(t *testing.T) {
+ t.Parallel()
+
+ agent := agentWalkContext()
+ for _, name := range []string{"bundle", "fast-export", "format-patch", "log", "rev-list", "rev-parse", "show"} {
+ require.True(t, agentRefWalk(agent, Command{Name: name}), name)
+ require.False(t, agentRefWalk(context.Background(), Command{Name: name}), name)
+ }
+ for _, name := range []string{"merge-base", "cat-file", "for-each-ref", "unknown"} {
+ require.False(t, agentRefWalk(agent, Command{Name: name}), name)
+ }
+ shared := identity.NewContext(context.Background(), identity.Identity{Name: "ci", Source: identity.SourceSharedToken})
+ require.False(t, agentRefWalk(shared, Command{Name: "rev-list"}))
+}
+
+// TestExcludeHiddenRefs: the flag goes before each --all and --glob among the options and the
+// arguments, and nowhere else: not before an option's value that reads --all, nor before
+// --branches, which git refuses after it.
+func TestExcludeHiddenRefs(t *testing.T) {
+ t.Parallel()
+
+ got := excludeHiddenRefs(Command{
+ Name: "rev-list",
+ Flags: []Option{
+ Flag{Name: "--all"},
+ ValueFlag{Name: "--grep", Value: "--all"},
+ ValueFlag{Name: "--glob", Value: "refs/*"},
+ Flag{Name: "--glob=refs/*"},
+ Flag{Name: "--branches"},
+ },
+ Args: []string{"main", "--not", "--all", "--glob=refs/heads/*", "--tags"},
+ PostSepArgs: []string{"--all"},
+ })
+ require.Equal(t, []Option{
+ Flag{Name: excludeHiddenFlag}, Flag{Name: "--all"},
+ ValueFlag{Name: "--grep", Value: "--all"},
+ Flag{Name: excludeHiddenFlag}, ValueFlag{Name: "--glob", Value: "refs/*"},
+ Flag{Name: excludeHiddenFlag}, Flag{Name: "--glob=refs/*"},
+ Flag{Name: "--branches"},
+ }, got.Flags)
+ require.Equal(t, []string{"main", "--not", excludeHiddenFlag, "--all", excludeHiddenFlag, "--glob=refs/heads/*", "--tags"}, got.Args)
+ require.Equal(t, []string{"--all"}, got.PostSepArgs, "paths are left alone")
+
+ args, err := got.CommandArgs()
+ require.NoError(t, err, "the rewritten command passes gitcmd's own validation")
+ require.Contains(t, args, excludeHiddenFlag)
+}
+
+// TestExcludeHiddenRefsOnStdin: on stdin, the flag goes on a line before each --all and --glob=,
+// until the line "--" after which paths follow; the stream is otherwise unchanged, read in any
+// chunks, and an error of the input reaches the reader.
+func TestExcludeHiddenRefsOnStdin(t *testing.T) {
+ t.Parallel()
+
+ in := "main\n--all\n--not\n--glob=refs/*\n--branches\n--\n--all\nlast"
+ want := "main\n" + excludeHiddenFlag + "\n--all\n--not\n" + excludeHiddenFlag + "\n--glob=refs/*\n--branches\n--\n--all\nlast"
+
+ out, err := io.ReadAll(excludeHiddenRefsOnStdin(strings.NewReader(in)))
+ require.NoError(t, err)
+ require.Equal(t, want, string(out))
+
+ out, err = io.ReadAll(iotest.OneByteReader(excludeHiddenRefsOnStdin(iotest.HalfReader(strings.NewReader(in)))))
+ require.NoError(t, err)
+ require.Equal(t, want, string(out))
+
+ out, err = io.ReadAll(excludeHiddenRefsOnStdin(strings.NewReader("")))
+ require.NoError(t, err)
+ require.Empty(t, out)
+
+ broken := errors.New("broken input")
+ out, err = io.ReadAll(excludeHiddenRefsOnStdin(io.MultiReader(strings.NewReader("--all\n"), iotest.ErrReader(broken))))
+ require.ErrorIs(t, err, broken)
+ require.Equal(t, excludeHiddenFlag+"\n--all\n", string(out))
+}
diff --git a/internal/git/gitcmd/command_description.go b/internal/git/gitcmd/command_description.go
index bae5fb0..16ffa42 100644
--- a/internal/git/gitcmd/command_description.go
+++ b/internal/git/gitcmd/command_description.go
@@ -18,6 +18,10 @@ const (
scNoRefUpdates = 1 << iota
// scNoEndOfOptions denotes a command which doesn't know --end-of-options
scNoEndOfOptions
+ // scWalksRefs denotes a command whose revisions take git's pseudo references --all and
+ // --glob, which an agent's command never resolves to references agents do not see
+ // (agentRefWalk).
+ scWalksRefs
)
type commandDescription struct {
@@ -41,7 +45,7 @@ var commandDescriptions = map[string]commandDescription{
flags: scNoRefUpdates | scNoEndOfOptions,
},
"bundle": {
- flags: scNoRefUpdates,
+ flags: scNoRefUpdates | scWalksRefs,
opts: packConfiguration,
},
"cat-file": {
@@ -96,7 +100,7 @@ var commandDescriptions = map[string]commandDescription{
flags: scNoRefUpdates,
},
"fast-export": {
- flags: scNoRefUpdates,
+ flags: scNoRefUpdates | scWalksRefs,
},
"fetch": {
flags: 0,
@@ -146,7 +150,7 @@ var commandDescriptions = map[string]commandDescription{
flags: scNoRefUpdates,
},
"format-patch": {
- flags: scNoRefUpdates,
+ flags: scNoRefUpdates | scWalksRefs,
},
"fsck": {
flags: scNoRefUpdates,
@@ -197,7 +201,7 @@ var commandDescriptions = map[string]commandDescription{
},
},
"log": {
- flags: scNoRefUpdates,
+ flags: scNoRefUpdates | scWalksRefs,
},
"last-modified": {
flags: scNoRefUpdates,
@@ -319,7 +323,7 @@ var commandDescriptions = map[string]commandDescription{
"rev-list": {
// We cannot use --end-of-options here because pseudo revisions like `--all`
// and `--not` count as options.
- flags: scNoRefUpdates | scNoEndOfOptions,
+ flags: scNoRefUpdates | scNoEndOfOptions | scWalksRefs,
validatePositionalArgs: func(args []string) error {
for _, arg := range args {
// git-rev-list(1) supports pseudo-revision arguments which can be
@@ -328,6 +332,10 @@ var commandDescriptions = map[string]commandDescription{
// refuse them as positional arguments. We thus override validation
// for two of these which we are using in our codebase.
if strings.HasPrefix(arg, "-") {
+ // The flag an agent's walk gets before each --all and --glob (agent_walk.go).
+ if arg == excludeHiddenFlag {
+ continue
+ }
if err := git.ValidateRevision([]byte(arg), git.AllowPseudoRevision()); err != nil {
return structerr.NewInvalidArgument(
"validating positional argument: %w", err,
@@ -344,10 +352,10 @@ var commandDescriptions = map[string]commandDescription{
"rev-parse": {
// --end-of-options is echoed by git-rev-parse(1) if used without
// `--verify`.
- flags: scNoRefUpdates | scNoEndOfOptions,
+ flags: scNoRefUpdates | scNoEndOfOptions | scWalksRefs,
},
"show": {
- flags: scNoRefUpdates,
+ flags: scNoRefUpdates | scWalksRefs,
},
"show-index": {
flags: scNoRefUpdates,
diff --git a/internal/git/gitcmd/command_factory.go b/internal/git/gitcmd/command_factory.go
index 946c4f8..1624f1c 100644
--- a/internal/git/gitcmd/command_factory.go
+++ b/internal/git/gitcmd/command_factory.go
@@ -477,10 +477,22 @@ func (cf *ExecCommandFactory) newCommand(ctx context.Context, repo storage.Repos
sc.Flags = append(sc.Flags, Flag{Name: fmt.Sprintf("--ref-format=%s", backend.Name)})
}
+ // An agent's walk of git's pseudo references leaves out the references agents do not see.
+ walk := agentRefWalk(ctx, sc)
+ if walk {
+ sc = excludeHiddenRefs(sc)
+ }
+
config, err := cf.combineOpts(ctx, sc, opts)
if err != nil {
return nil, err
}
+ if walk {
+ config.globals = append(config.globals, agentWalkConfig()...)
+ if readsRevisionsFromStdin(sc) {
+ config.commandOpts = append(config.commandOpts, command.WithStdinFilter(excludeHiddenRefsOnStdin))
+ }
+ }
cmdGitVersion, err := cf.GitVersion(ctx)
if err != nil {
diff --git a/internal/gitaly/server/auth/agent.go b/internal/gitaly/server/auth/agent.go
index efa058f..b5aebaa 100644
--- a/internal/gitaly/server/auth/agent.go
+++ b/internal/gitaly/server/auth/agent.go
@@ -5,6 +5,7 @@ import (
"crypto/subtle"
"fmt"
"os"
+ "slices"
"strings"
"time"
"unicode"
@@ -13,6 +14,7 @@ import (
"github.com/feder-cr/git-ai/internal/gitaly/config"
gitalycfgauth "github.com/feder-cr/git-ai/internal/gitaly/config/auth"
"github.com/feder-cr/git-ai/internal/gitaly/storage"
+ "github.com/feder-cr/git-ai/internal/graft/agentrefs"
"github.com/feder-cr/git-ai/internal/graft/identity"
grpcmwauth "github.com/grpc-ecosystem/go-grpc-middleware/v2/interceptors/auth"
"google.golang.org/grpc"
@@ -223,6 +225,12 @@ func loadAgentCredentials(sharedToken string, agents []config.GraftAgent) ([]age
if agent.Name == sharedTokenIdentityName {
return nil, fmt.Errorf("graft agent %d: name %q is reserved", i, agent.Name)
}
+ // The workspace's temporary branches of an agent are graft/<name>/..., and an agent may not
+ // name graft/<namespace>/... for a namespace of the server (agentrefs): an agent named after
+ // one could not use its own branches.
+ if slices.Contains(agentrefs.ServerNamespaces, agent.Name) {
+ return nil, fmt.Errorf("graft agent %d: name %q is reserved for the server's references under %s", i, agent.Name, agentrefs.HiddenPrefix)
+ }
if _, ok := names[agent.Name]; ok {
return nil, fmt.Errorf("graft agent %q: duplicate name", agent.Name)
}
diff --git a/internal/gitaly/server/auth/agent_refs.go b/internal/gitaly/server/auth/agent_refs.go
index 1218f58..e4948c6 100644
--- a/internal/gitaly/server/auth/agent_refs.go
+++ b/internal/gitaly/server/auth/agent_refs.go
@@ -18,12 +18,14 @@ import (
// names refs/graft/ (agentrefs.RevisionHidden);
// - a response loses every reference under refs/graft/ it lists: a list element that is such a
// name, or a message that names one in a reference field.
-// Which fields name references is decided by agentRefField: an exact table for the messages of
-// the RPCs agents may call by default (a test fails on any string or bytes field that the table
-// does not classify), and a conservative rule by field name for the RPCs an operator adds with
-// [graft] agent_rpcs.
+// Which fields name references is decided by agentRefField: an exact table, by message and field,
+// for the messages of the RPCs agents may call by default (a test fails on any string, bytes, bool
+// or enum field of theirs that the table does not classify), and a conservative rule by field name
+// for the RPCs an operator adds with [graft] agent_rpcs. A flag that makes git walk every reference
+// (walksAllRefs) is not checked here: gitcmd leaves the hidden references out of every walk of an
+// agent's command (gitcmd/agent_walk.go), whatever asked for it.
-// refFieldKind is what a string or bytes field of a request or a response holds.
+// refFieldKind is what a field of a request or a response holds, or does.
type refFieldKind int
const (
@@ -33,62 +35,445 @@ const (
// isBranchName: a branch name the server completes under refs/heads/ when it is short; only a
// full name under refs/graft/ is refused (no revision rule completes it to refs/graft/).
isBranchName
+ // walksAllRefs: a flag that makes git walk every reference (git's --all); the walk of an agent
+ // leaves out the hidden references (gitcmd).
+ walksAllRefs
)
-// agentRefFieldsByName classifies a field by its name alone.
-var agentRefFieldsByName = map[protoreflect.Name]refFieldKind{
- // Revisions, references and patterns of requests, and references of responses.
- "commit": isRef, "commit_id": isRef, "commit_ids": isRef, "commit_revision": isRef, "commits": isRef,
- "expected_head_commit_id": isRef, "expected_old_oid": isRef, "head_commit_id": isRef, "base_commit_id": isRef,
- "include_referenced_by": isRef, "left_commit_id": isRef, "right_commit_id": isRef,
- "left_tree_revision": isRef, "right_tree_revision": isRef, "new_revision": isRef, "old_revision": isRef,
- "oid": isRef, "oids": isRef, "parent_commit_revisions": isRef, "patterns": isRef, "pointing_at_oids": isRef,
- "ref_names": isRef, "ref_patterns": isRef, "resume_oid": isRef, "revision": isRef, "revisions": isRef,
- "start_point": isRef, "start_sha": isRef, "tag_ids": isRef, "tag_revisions": isRef, "tip": isRef,
- "tree_oid": isRef, "blob_ids": isRef, "refs": isRef, "ref_name": isRef, "reference": isRef,
- "referenced_by": isRef, "branch_names": isRef, "tag_names": isRef, "target": isRef,
- // Branch and tag names: the server completes them under refs/heads/ and refs/tags/, so they
- // cannot name refs/graft/; and fields that hold no reference.
- "branch": notRef, "branch_name": notRef, "start_branch_name": notRef, "target_branch": notRef,
- "merged_branches": notRef, "tag_name": notRef, "remote_name": notRef, "exclude_patterns": notRef,
- "author": notRef, "body": notRef, "commit_author_email": notRef, "commit_author_name": notRef,
- "commit_message": notRef, "commit_message_patterns": notRef, "content": notRef, "description": notRef,
- "email": notRef, "file_path": notRef, "git_alternate_object_directories": notRef,
- "git_object_directory": notRef, "gl_id": notRef, "gl_project_path": notRef, "gl_repository": notRef,
- "gl_username": notRef, "id": notRef, "merge_message": notRef, "message_regex": notRef, "mode": notRef,
- "page_token": notRef, "path": notRef, "paths": notRef, "previous_path": notRef, "query": notRef,
- "relative_path": notRef, "sort_field": notRef, "storage_name": notRef, "timezone": notRef, "title": notRef,
- "actor": notRef, "commit_oid": notRef, "data": notRef, "encoding": notRef, "file_oid": notRef,
- "filesystem_id": notRef, "flat_path": notRef, "following_lines": notRef, "fs_type": notRef,
- "git_version": notRef, "index_error": notRef, "key": notRef, "landed_commit_id": notRef, "line": notRef,
- "message": notRef, "new_blob_id": notRef, "new_oid": notRef, "next_cursor": notRef, "old_blob_id": notRef,
- "old_oid": notRef, "old_path": notRef, "old_tip": notRef, "operation": notRef, "original_commit_id": notRef,
- "parent_ids": notRef, "patch_id": notRef, "path_bytes": notRef, "peeled_target": notRef,
- "pre_receive_error": notRef, "preceding_lines": notRef, "reason": notRef, "resolved_paths": notRef,
- "resolver": notRef, "rule": notRef, "server_version": notRef, "signature": notRef, "signed_text": notRef,
- "subject": notRef, "tag_id": notRef, "tree_id": notRef, "value": notRef,
-}
+// agentRefFields classifies, by message and field, the string, bytes, bool and enum fields of the
+// requests and responses of the RPCs agents may call by default (builtinAgentRPCs), at any depth.
+// Numeric fields name no reference and select no walk, and are not listed.
+var agentRefFields = map[protoreflect.FullName]refFieldKind{
+ "gitaly.AddReviewCommentRequest.body": notRef,
+ "gitaly.AddReviewCommentRequest.id": notRef,
+ "gitaly.AddReviewCommentRequest.path": notRef,
+
+ "gitaly.BlameRequest.path": notRef,
+ "gitaly.BlameRequest.revision": isRef,
+
+ "gitaly.BlameResponse.Line.commit_id": isRef,
+ "gitaly.BlameResponse.Line.content": notRef,
+
+ "gitaly.BlameResponse.commit_id": isRef,
+
+ "gitaly.Branch.name": isRef,
+
+ "gitaly.ChangedPaths.commit_id": isRef,
+ "gitaly.ChangedPaths.new_blob_id": notRef,
+ "gitaly.ChangedPaths.old_blob_id": notRef,
+ "gitaly.ChangedPaths.old_path": notRef,
+ "gitaly.ChangedPaths.path": notRef,
+ "gitaly.ChangedPaths.status": notRef,
+
+ "gitaly.Changeset.actor": notRef,
+ "gitaly.Changeset.new_oid": notRef,
+ "gitaly.Changeset.old_oid": notRef,
+ "gitaly.Changeset.operation": notRef,
+ "gitaly.Changeset.reference": isRef,
+ "gitaly.Changeset.rewritten": notRef,
+
+ "gitaly.ChangesetMember.actor": notRef,
+ "gitaly.ChangesetMember.commit_id": isRef,
+ "gitaly.ChangesetMember.original_commit_id": notRef,
+ "gitaly.ChangesetMember.resolved_paths": notRef,
+ "gitaly.ChangesetMember.rewritten": notRef,
+
+ "gitaly.CloseReviewRequest.id": notRef,
+ "gitaly.CloseReviewRequest.landed": notRef,
+
+ "gitaly.CommitAuthor.email": notRef,
+ "gitaly.CommitAuthor.name": notRef,
+ "gitaly.CommitAuthor.timezone": notRef,
+
+ "gitaly.CommitTrailer.key": notRef,
+ "gitaly.CommitTrailer.value": notRef,
+
+ "gitaly.CreateReviewRequest.base_commit_id": isRef,
+ "gitaly.CreateReviewRequest.description": notRef,
+ "gitaly.CreateReviewRequest.head_commit_id": isRef,
+ "gitaly.CreateReviewRequest.target_branch": notRef,
+ "gitaly.CreateReviewRequest.title": notRef,
+
+ "gitaly.DiffStats.old_path": notRef,
+ "gitaly.DiffStats.path": notRef,
+
+ "gitaly.DiffStatsRequest.left_commit_id": isRef,
+ "gitaly.DiffStatsRequest.right_commit_id": isRef,
+ "gitaly.DiffStatsRequest.whitespace_changes": notRef,
+
+ "gitaly.FindAllBranchesRequest.merged_branches": notRef,
+ "gitaly.FindAllBranchesRequest.merged_only": notRef,
+
+ "gitaly.FindAllBranchesResponse.Branch.name": isRef,
+
+ "gitaly.FindAllRemoteBranchesRequest.remote_name": notRef,
+
+ "gitaly.FindAllTagsRequest.SortBy.direction": notRef,
+ "gitaly.FindAllTagsRequest.SortBy.key": notRef,
+
+ "gitaly.FindBranchRequest.name": isBranchName,
+
+ "gitaly.FindChangedPathsRequest.Request.CommitRequest.commit_revision": isRef,
+ "gitaly.FindChangedPathsRequest.Request.CommitRequest.parent_commit_revisions": isRef,
+
+ "gitaly.FindChangedPathsRequest.Request.TreeRequest.left_tree_revision": isRef,
+ "gitaly.FindChangedPathsRequest.Request.TreeRequest.right_tree_revision": isRef,
+
+ "gitaly.FindChangedPathsRequest.commits": isRef,
+ "gitaly.FindChangedPathsRequest.diff_filters": notRef,
+ "gitaly.FindChangedPathsRequest.find_renames": notRef,
+ "gitaly.FindChangedPathsRequest.merge_commit_diff_mode": notRef,
+
+ "gitaly.FindCommitRequest.revision": isRef,
+ "gitaly.FindCommitRequest.trailers": notRef,
+
+ "gitaly.FindCommitsRequest.all": walksAllRefs,
+ "gitaly.FindCommitsRequest.author": notRef,
+ "gitaly.FindCommitsRequest.disable_walk": notRef,
+ "gitaly.FindCommitsRequest.first_parent": notRef,
+ "gitaly.FindCommitsRequest.follow": notRef,
+ "gitaly.FindCommitsRequest.include_referenced_by": isRef,
+ "gitaly.FindCommitsRequest.include_shortstat": notRef,
+ "gitaly.FindCommitsRequest.message_regex": notRef,
+ "gitaly.FindCommitsRequest.order": notRef,
+ "gitaly.FindCommitsRequest.paths": notRef,
+ "gitaly.FindCommitsRequest.revision": isRef,
+ "gitaly.FindCommitsRequest.skip_merges": notRef,
+ "gitaly.FindCommitsRequest.trailers": notRef,
+
+ "gitaly.FindDefaultBranchNameRequest.head_only": notRef,
+
+ "gitaly.FindDefaultBranchNameResponse.name": isRef,
+
+ "gitaly.FindLocalBranchesRequest.exclude_patterns": notRef,
+ "gitaly.FindLocalBranchesRequest.sort_by": notRef,
+
+ "gitaly.FindRefsByOIDRequest.oid": isRef,
+ "gitaly.FindRefsByOIDRequest.ref_patterns": isRef,
+ "gitaly.FindRefsByOIDRequest.sort_field": notRef,
+
+ "gitaly.FindRefsByOIDResponse.refs": isRef,
+
+ "gitaly.FindTagRequest.tag_name": notRef,
+
+ "gitaly.GetBlobRequest.oid": isRef,
+
+ "gitaly.GetBlobResponse.data": notRef,
+ "gitaly.GetBlobResponse.oid": isRef,
+
+ "gitaly.GetBlobsRequest.RevisionPath.path": notRef,
+ "gitaly.GetBlobsRequest.RevisionPath.revision": isRef,
+
+ "gitaly.GetBlobsResponse.data": notRef,
+ "gitaly.GetBlobsResponse.is_submodule": notRef,
+ "gitaly.GetBlobsResponse.oid": isRef,
+ "gitaly.GetBlobsResponse.path": notRef,
+ "gitaly.GetBlobsResponse.revision": isRef,
+ "gitaly.GetBlobsResponse.type": notRef,
+
+ "gitaly.GetCommitMessagesRequest.commit_ids": isRef,
+
+ "gitaly.GetCommitMessagesResponse.commit_id": isRef,
+ "gitaly.GetCommitMessagesResponse.message": notRef,
+
+ "gitaly.GetCommitSignaturesRequest.commit_ids": isRef,
+
+ "gitaly.GetCommitSignaturesResponse.commit_id": isRef,
+ "gitaly.GetCommitSignaturesResponse.signature": notRef,
+ "gitaly.GetCommitSignaturesResponse.signed_text": notRef,
+ "gitaly.GetCommitSignaturesResponse.signer": notRef,
+
+ "gitaly.GetLFSPointersRequest.blob_ids": isRef,
+
+ "gitaly.GetObjectsRequest.oids": isRef,
+ "gitaly.GetObjectsRequest.resume_oid": isRef,
+
+ "gitaly.GetObjectsResponse.data": notRef,
+ "gitaly.GetObjectsResponse.missing": notRef,
+ "gitaly.GetObjectsResponse.oid": isRef,
+ "gitaly.GetObjectsResponse.type": notRef,
+
+ "gitaly.GetPatchIDRequest.context_free": notRef,
+ "gitaly.GetPatchIDRequest.new_revision": isRef,
+ "gitaly.GetPatchIDRequest.old_revision": isRef,
+ "gitaly.GetPatchIDRequest.whole_diff_hash": notRef,
+
+ "gitaly.GetPatchIDResponse.patch_id": notRef,
+
+ "gitaly.GetReviewRequest.id": notRef,
+
+ "gitaly.GetTagMessagesRequest.tag_ids": isRef,
+
+ "gitaly.GetTagMessagesResponse.message": notRef,
+ "gitaly.GetTagMessagesResponse.tag_id": notRef,
+
+ "gitaly.GetTagSignaturesRequest.tag_revisions": isRef,
+
+ "gitaly.GetTagSignaturesResponse.TagSignature.content": notRef,
+ "gitaly.GetTagSignaturesResponse.TagSignature.signature": notRef,
+ "gitaly.GetTagSignaturesResponse.TagSignature.tag_id": notRef,
+
+ "gitaly.GetTreeEntriesRequest.path": notRef,
+ "gitaly.GetTreeEntriesRequest.recursive": notRef,
+ "gitaly.GetTreeEntriesRequest.revision": isRef,
+ "gitaly.GetTreeEntriesRequest.skip_flat_paths": notRef,
+ "gitaly.GetTreeEntriesRequest.sort": notRef,
+ "gitaly.GetTreeEntriesRequest.with_last_commit": notRef,
+
+ "gitaly.GetTreeRequest.include_sizes": notRef,
+ "gitaly.GetTreeRequest.path": notRef,
+ "gitaly.GetTreeRequest.recursive": notRef,
+ "gitaly.GetTreeRequest.revision": isRef,
+ "gitaly.GetTreeRequest.tree_oid": isRef,
+
+ "gitaly.GetTreeResponse.Entry.oid": isRef,
+ "gitaly.GetTreeResponse.Entry.path": notRef,
+ "gitaly.GetTreeResponse.Entry.type": notRef,
+
+ "gitaly.GetTreeResponse.tree_oid": isRef,
+
+ "gitaly.GitCommit.body": notRef,
+ "gitaly.GitCommit.encoding": notRef,
+ "gitaly.GitCommit.id": notRef,
+ "gitaly.GitCommit.parent_ids": notRef,
+ "gitaly.GitCommit.referenced_by": isRef,
+ "gitaly.GitCommit.signature_type": notRef,
+ "gitaly.GitCommit.subject": notRef,
+ "gitaly.GitCommit.tree_id": notRef,
+
+ "gitaly.GlobalOptions.literal_pathspecs": notRef,
-// agentRefNameFields classifies the field "name", whose meaning depends on its message.
-var agentRefNameFields = map[protoreflect.FullName]refFieldKind{
- "gitaly.FindBranchRequest": isBranchName,
- "gitaly.Branch": isRef,
- "gitaly.FindAllBranchesResponse.Branch": isRef,
- "gitaly.FindDefaultBranchNameResponse": isRef,
- "gitaly.ListRefsResponse.Reference": isRef,
- "gitaly.Tag": isRef,
- "gitaly.User": notRef,
- "gitaly.CommitAuthor": notRef,
+ "gitaly.LFSPointer.data": notRef,
+ "gitaly.LFSPointer.file_oid": notRef,
+ "gitaly.LFSPointer.oid": isRef,
+
+ "gitaly.LandResolution.commit": isRef,
+ "gitaly.LandResolution.tip": isRef,
+
+ "gitaly.LandResolvedFile.action": notRef,
+ "gitaly.LandResolvedFile.content": notRef,
+ "gitaly.LandResolvedFile.mode": notRef,
+ "gitaly.LandResolvedFile.path": notRef,
+
+ "gitaly.ListAllBlobsResponse.Blob.data": notRef,
+ "gitaly.ListAllBlobsResponse.Blob.oid": isRef,
+
+ "gitaly.ListBlobsRequest.revisions": isRef,
+ "gitaly.ListBlobsRequest.with_paths": notRef,
+
+ "gitaly.ListBlobsResponse.Blob.data": notRef,
+ "gitaly.ListBlobsResponse.Blob.oid": isRef,
+ "gitaly.ListBlobsResponse.Blob.path": notRef,
+
+ "gitaly.ListBranchNamesContainingCommitRequest.commit_id": isRef,
+
+ "gitaly.ListBranchNamesContainingCommitResponse.branch_names": isRef,
+
+ "gitaly.ListCommitsByOidRequest.oid": isRef,
+
+ "gitaly.ListCommitsByRefNameRequest.ref_names": isRef,
+
+ "gitaly.ListCommitsByRefNameResponse.CommitForRef.ref_name": isRef,
+
+ "gitaly.ListCommitsRequest.author": notRef,
+ "gitaly.ListCommitsRequest.commit_message_patterns": notRef,
+ "gitaly.ListCommitsRequest.disable_walk": notRef,
+ "gitaly.ListCommitsRequest.first_parent": notRef,
+ "gitaly.ListCommitsRequest.follow": notRef,
+ "gitaly.ListCommitsRequest.ignore_case": notRef,
+ "gitaly.ListCommitsRequest.order": notRef,
+ "gitaly.ListCommitsRequest.paths": notRef,
+ "gitaly.ListCommitsRequest.reverse": notRef,
+ "gitaly.ListCommitsRequest.revisions": isRef,
+ "gitaly.ListCommitsRequest.trailers": notRef,
+
+ "gitaly.ListFilesRequest.revision": isRef,
+
+ "gitaly.ListFilesResponse.paths": notRef,
+
+ "gitaly.ListLFSPointersRequest.revisions": isRef,
+
+ "gitaly.ListLastCommitsForTreeRequest.literal_pathspec": notRef,
+ "gitaly.ListLastCommitsForTreeRequest.path": notRef,
+ "gitaly.ListLastCommitsForTreeRequest.revision": isRef,
+
+ "gitaly.ListLastCommitsForTreeResponse.CommitForTree.path_bytes": notRef,
+
+ "gitaly.ListRefsRequest.SortBy.direction": notRef,
+ "gitaly.ListRefsRequest.SortBy.key": notRef,
+
+ "gitaly.ListRefsRequest.exclude_patterns": notRef,
+ "gitaly.ListRefsRequest.head": notRef,
+ "gitaly.ListRefsRequest.ignore_case": notRef,
+ "gitaly.ListRefsRequest.include_tagger_date": notRef,
+ "gitaly.ListRefsRequest.patterns": isRef,
+ "gitaly.ListRefsRequest.peel_tags": notRef,
+ "gitaly.ListRefsRequest.pointing_at_oids": isRef,
+
+ "gitaly.ListRefsResponse.Reference.name": isRef,
+ "gitaly.ListRefsResponse.Reference.peeled_target": notRef,
+ "gitaly.ListRefsResponse.Reference.target": isRef,
+
+ "gitaly.ListReviewsRequest.include_closed": notRef,
+ "gitaly.ListReviewsRequest.target_branch": notRef,
+
+ "gitaly.ListTagNamesContainingCommitRequest.commit_id": isRef,
+
+ "gitaly.ListTagNamesContainingCommitResponse.tag_names": isRef,
+
+ "gitaly.OperationBranchUpdate.branch_created": notRef,
+ "gitaly.OperationBranchUpdate.commit_id": isRef,
+ "gitaly.OperationBranchUpdate.repo_created": notRef,
+
+ "gitaly.PaginationCursor.next_cursor": notRef,
+
+ "gitaly.PaginationParameter.page_token": notRef,
+
+ "gitaly.PathHistoryRequest.path": notRef,
+ "gitaly.PathHistoryRequest.revision": isRef,
+
+ "gitaly.Repository.git_alternate_object_directories": notRef,
+ "gitaly.Repository.git_object_directory": notRef,
+ "gitaly.Repository.gl_project_path": notRef,
+ "gitaly.Repository.gl_repository": notRef,
+ "gitaly.Repository.relative_path": notRef,
+ "gitaly.Repository.storage_name": notRef,
+
+ "gitaly.RepositoryExistsResponse.exists": notRef,
+
+ "gitaly.Review.approved": notRef,
+ "gitaly.Review.author": notRef,
+ "gitaly.Review.base_commit_id": isRef,
+ "gitaly.Review.description": notRef,
+ "gitaly.Review.head_commit_id": isRef,
+ "gitaly.Review.id": notRef,
+ "gitaly.Review.state": notRef,
+ "gitaly.Review.target_branch": notRef,
+ "gitaly.Review.title": notRef,
+
+ "gitaly.ReviewComment.actor": notRef,
+ "gitaly.ReviewComment.body": notRef,
+ "gitaly.ReviewComment.commit_id": isRef,
+ "gitaly.ReviewComment.id": notRef,
+ "gitaly.ReviewComment.path": notRef,
+
+ "gitaly.ReviewFile.class": notRef,
+ "gitaly.ReviewFile.path": notRef,
+ "gitaly.ReviewFile.reason": notRef,
+
+ "gitaly.ReviewRequirement.description": notRef,
+ "gitaly.ReviewRequirement.paths": notRef,
+ "gitaly.ReviewRequirement.rule": notRef,
+ "gitaly.ReviewRequirement.satisfied": notRef,
+
+ "gitaly.ReviewVerdict.actor": notRef,
+ "gitaly.ReviewVerdict.body": notRef,
+ "gitaly.ReviewVerdict.commit_id": isRef,
+ "gitaly.ReviewVerdict.current": notRef,
+ "gitaly.ReviewVerdict.kind": notRef,
+
+ "gitaly.SearchCodeRequest.case_sensitive": notRef,
+ "gitaly.SearchCodeRequest.paths": notRef,
+ "gitaly.SearchCodeRequest.query": notRef,
+ "gitaly.SearchCodeRequest.regex": notRef,
+ "gitaly.SearchCodeRequest.revision": isRef,
+
+ "gitaly.SearchCodeResponse.Match.following_lines": notRef,
+ "gitaly.SearchCodeResponse.Match.line": notRef,
+ "gitaly.SearchCodeResponse.Match.path": notRef,
+ "gitaly.SearchCodeResponse.Match.preceding_lines": notRef,
+
+ "gitaly.SearchCodeResponse.commit_id": isRef,
+ "gitaly.SearchCodeResponse.truncated": notRef,
+
+ "gitaly.ServerInfoResponse.StorageStatus.filesystem_id": notRef,
+ "gitaly.ServerInfoResponse.StorageStatus.fs_type": notRef,
+ "gitaly.ServerInfoResponse.StorageStatus.readable": notRef,
+ "gitaly.ServerInfoResponse.StorageStatus.storage_name": notRef,
+ "gitaly.ServerInfoResponse.StorageStatus.writeable": notRef,
+
+ "gitaly.ServerInfoResponse.git_version": notRef,
+ "gitaly.ServerInfoResponse.server_version": notRef,
+
+ "gitaly.SubmitReviewVerdictRequest.body": notRef,
+ "gitaly.SubmitReviewVerdictRequest.head_commit_id": isRef,
+ "gitaly.SubmitReviewVerdictRequest.id": notRef,
+ "gitaly.SubmitReviewVerdictRequest.kind": notRef,
+
+ "gitaly.Tag.id": notRef,
+ "gitaly.Tag.message": notRef,
+ "gitaly.Tag.name": isRef,
+ "gitaly.Tag.signature_type": notRef,
+
+ "gitaly.TreeEntry.commit_oid": notRef,
+ "gitaly.TreeEntry.flat_path": notRef,
+ "gitaly.TreeEntry.oid": isRef,
+ "gitaly.TreeEntry.path": notRef,
+ "gitaly.TreeEntry.type": notRef,
+
+ "gitaly.TreeEntryRequest.path": notRef,
+ "gitaly.TreeEntryRequest.revision": isRef,
+
+ "gitaly.TreeEntryResponse.data": notRef,
+ "gitaly.TreeEntryResponse.oid": isRef,
+ "gitaly.TreeEntryResponse.type": notRef,
+
+ "gitaly.UpdateReviewRequest.expected_head_commit_id": isRef,
+ "gitaly.UpdateReviewRequest.head_commit_id": isRef,
+ "gitaly.UpdateReviewRequest.id": notRef,
+
+ "gitaly.User.email": notRef,
+ "gitaly.User.gl_id": notRef,
+ "gitaly.User.gl_username": notRef,
+ "gitaly.User.name": notRef,
+ "gitaly.User.timezone": notRef,
+
+ "gitaly.UserCommitFilesAction.content": notRef,
+
+ "gitaly.UserCommitFilesActionHeader.action": notRef,
+ "gitaly.UserCommitFilesActionHeader.base64_content": notRef,
+ "gitaly.UserCommitFilesActionHeader.execute_filemode": notRef,
+ "gitaly.UserCommitFilesActionHeader.file_path": notRef,
+ "gitaly.UserCommitFilesActionHeader.infer_content": notRef,
+ "gitaly.UserCommitFilesActionHeader.previous_path": notRef,
+
+ "gitaly.UserCommitFilesRequestHeader.branch_name": notRef,
+ "gitaly.UserCommitFilesRequestHeader.commit_author_email": notRef,
+ "gitaly.UserCommitFilesRequestHeader.commit_author_name": notRef,
+ "gitaly.UserCommitFilesRequestHeader.commit_message": notRef,
+ "gitaly.UserCommitFilesRequestHeader.expected_old_oid": isRef,
+ "gitaly.UserCommitFilesRequestHeader.force": notRef,
+ "gitaly.UserCommitFilesRequestHeader.sign": notRef,
+ "gitaly.UserCommitFilesRequestHeader.start_branch_name": notRef,
+ "gitaly.UserCommitFilesRequestHeader.start_sha": isRef,
+
+ "gitaly.UserCommitFilesResponse.index_error": notRef,
+ "gitaly.UserCommitFilesResponse.pre_receive_error": notRef,
+
+ "gitaly.UserCreateBranchRequest.branch_name": notRef,
+ "gitaly.UserCreateBranchRequest.start_point": isRef,
+
+ "gitaly.UserDeleteBranchRequest.branch_name": notRef,
+ "gitaly.UserDeleteBranchRequest.expected_old_oid": isRef,
+
+ "gitaly.UserLandRequest.branch": notRef,
+ "gitaly.UserLandRequest.commit_id": isRef,
+ "gitaly.UserLandRequest.create_branch": notRef,
+ "gitaly.UserLandRequest.merge_message": notRef,
+ "gitaly.UserLandRequest.resolve_conflicts": notRef,
+ "gitaly.UserLandRequest.strategy": notRef,
+
+ "gitaly.UserLandResponse.already_landed": notRef,
+ "gitaly.UserLandResponse.landed_commit_id": notRef,
+ "gitaly.UserLandResponse.old_tip": notRef,
+ "gitaly.UserLandResponse.resolved_paths": notRef,
+ "gitaly.UserLandResponse.resolver": notRef,
+ "gitaly.UserLandResponse.rewritten": notRef,
}
-// agentRefField returns what a string or bytes field holds, and whether the exact table knows it.
-func agentRefField(md protoreflect.MessageDescriptor, fd protoreflect.FieldDescriptor) (refFieldKind, bool) {
- if fd.Name() == "name" {
- kind, ok := agentRefNameFields[md.FullName()]
- if ok {
- return kind, true
- }
- } else if kind, ok := agentRefFieldsByName[fd.Name()]; ok {
+// agentRefField returns what a field holds, and whether the exact table knows it.
+func agentRefField(fd protoreflect.FieldDescriptor) (refFieldKind, bool) {
+ if kind, ok := agentRefFields[fd.FullName()]; ok {
return kind, true
}
// A field of an RPC an operator added: a name that may hold a revision, a reference or a
@@ -133,8 +518,8 @@ func findHiddenRef(m protoreflect.Message) (protoreflect.FullName, string, bool)
m.Range(func(fd protoreflect.FieldDescriptor, v protoreflect.Value) bool {
switch fd.Kind() {
case protoreflect.StringKind, protoreflect.BytesKind:
- kind, _ := agentRefField(m.Descriptor(), fd)
- if kind == notRef {
+ kind, _ := agentRefField(fd)
+ if kind != isRef && kind != isBranchName {
return true
}
for _, s := range scalarStrings(fd, v) {
@@ -183,7 +568,7 @@ func filterMessage(m protoreflect.Message, top bool) bool {
m.Range(func(fd protoreflect.FieldDescriptor, v protoreflect.Value) bool {
switch fd.Kind() {
case protoreflect.StringKind, protoreflect.BytesKind:
- if kind, _ := agentRefField(m.Descriptor(), fd); kind == notRef {
+ if kind, _ := agentRefField(fd); kind != isRef && kind != isBranchName {
return true
}
if fd.IsList() {
diff --git a/internal/gitaly/server/auth/agent_refs_test.go b/internal/gitaly/server/auth/agent_refs_test.go
index 5ba9f0e..d309d2c 100755
--- a/internal/gitaly/server/auth/agent_refs_test.go
+++ b/internal/gitaly/server/auth/agent_refs_test.go
@@ -18,10 +18,11 @@ import (
"google.golang.org/protobuf/reflect/protoregistry"
)
-// TestAgentRefFields_everyAllowedRPCIsClassified: every string or bytes field of the requests and
-// responses of every RPC agents may call by default, at any depth, is classified by the exact table
-// of agentRefField, as a reference or not. A new RPC or field fails here until it is classified, so
-// the filter cannot miss it.
+// TestAgentRefFields_everyAllowedRPCIsClassified: every string, bytes, bool or enum field of the
+// requests and responses of every RPC agents may call by default, at any depth, is classified by
+// the exact table of agentRefField, by message and field: as a reference or not, or as a flag that
+// walks every reference. A new RPC or field fails here until it is classified, so the filter cannot
+// miss it, nor a new field that a generic name would have passed.
func TestAgentRefFields_everyAllowedRPCIsClassified(t *testing.T) {
t.Parallel()
@@ -44,8 +45,8 @@ func TestAgentRefFields_everyAllowedRPCIsClassified(t *testing.T) {
for i := 0; i < m.Fields().Len(); i++ {
fd := m.Fields().Get(i)
switch fd.Kind() {
- case protoreflect.StringKind, protoreflect.BytesKind:
- if _, exact := agentRefField(m, fd); !exact {
+ case protoreflect.StringKind, protoreflect.BytesKind, protoreflect.BoolKind, protoreflect.EnumKind:
+ if _, exact := agentRefField(fd); !exact {
unclassified = append(unclassified, string(m.FullName())+"."+string(fd.Name())+" ("+method+")")
}
case protoreflect.MessageKind, protoreflect.GroupKind:
@@ -59,7 +60,7 @@ func TestAgentRefFields_everyAllowedRPCIsClassified(t *testing.T) {
}
}
sort.Strings(unclassified)
- require.Empty(t, unclassified, "string or bytes fields of agent RPCs that agentRefField does not classify")
+ require.Empty(t, unclassified, "fields of agent RPCs that agentRefField does not classify")
}
func agentContext() context.Context {
@@ -190,16 +191,54 @@ func TestAgentRefFilter_throughTheInterceptors(t *testing.T) {
require.Equal(t, "refs/heads/main", strings.Join(names, ","))
}
-// TestAgentRefNameFields_messagesExist: every message the classification of "name" names exists and
-// has a field "name", so that a typo cannot leave a field unclassified.
-func TestAgentRefNameFields_messagesExist(t *testing.T) {
+// TestAgentRefFields_entriesExist: every field the exact table classifies exists, so that a typo
+// cannot leave a field unclassified; a reference is a string or bytes field, and a walk of every
+// reference a bool.
+func TestAgentRefFields_entriesExist(t *testing.T) {
t.Parallel()
- for name := range agentRefNameFields {
+ for name, kind := range agentRefFields {
d, err := protoregistry.GlobalFiles.FindDescriptorByName(name)
require.NoError(t, err, name)
- md, ok := d.(protoreflect.MessageDescriptor)
+ fd, ok := d.(protoreflect.FieldDescriptor)
require.True(t, ok, name)
- require.NotNil(t, md.Fields().ByName("name"), name)
+ switch kind {
+ case isRef, isBranchName:
+ require.Contains(t, []protoreflect.Kind{protoreflect.StringKind, protoreflect.BytesKind}, fd.Kind(), name)
+ case walksAllRefs:
+ require.Equal(t, protoreflect.BoolKind, fd.Kind(), name)
+ }
}
}
+
+// TestAgentRefField_byMessage: a field is classified by its message, not its name: the same name
+// in two messages may differ, and a field an operator's RPC adds falls back to the conservative
+// rule by name.
+func TestAgentRefField_byMessage(t *testing.T) {
+ t.Parallel()
+
+ field := func(message, name string) protoreflect.FieldDescriptor {
+ d, err := protoregistry.GlobalFiles.FindDescriptorByName(protoreflect.FullName(message))
+ require.NoError(t, err, message)
+ fd := d.(protoreflect.MessageDescriptor).Fields().ByName(protoreflect.Name(name))
+ require.NotNil(t, fd, message+"."+name)
+ return fd
+ }
+
+ kind, exact := agentRefField(field("gitaly.FindBranchRequest", "name"))
+ require.True(t, exact)
+ require.Equal(t, isBranchName, kind)
+ kind, exact = agentRefField(field("gitaly.User", "name"))
+ require.True(t, exact)
+ require.Equal(t, notRef, kind)
+ kind, exact = agentRefField(field("gitaly.FindCommitsRequest", "all"))
+ require.True(t, exact)
+ require.Equal(t, walksAllRefs, kind)
+
+ // CountCommitsRequest is no RPC agents call by default: its fields are judged by name.
+ kind, exact = agentRefField(field("gitaly.CountCommitsRequest", "revision"))
+ require.False(t, exact)
+ require.Equal(t, isRef, kind)
+ _, exact = agentRefField(field("gitaly.CountCommitsRequest", "all"))
+ require.False(t, exact)
+}
diff --git a/internal/gitaly/server/auth/agent_test.go b/internal/gitaly/server/auth/agent_test.go
index c075ac6..2f0a9bb 100644
--- a/internal/gitaly/server/auth/agent_test.go
+++ b/internal/gitaly/server/auth/agent_test.go
@@ -400,6 +400,16 @@ func TestLoadAgentCredentials(t *testing.T) {
agents: []config.GraftAgent{{Name: sharedTokenIdentityName, Token: "tok"}},
wantErr: "is reserved",
},
+ {
+ desc: "reserved name of the proposals",
+ agents: []config.GraftAgent{{Name: "proposals", Token: "tok"}},
+ wantErr: "is reserved for the server's references under refs/graft/",
+ },
+ {
+ desc: "reserved name of the green references",
+ agents: []config.GraftAgent{{Name: "green", Token: "tok"}},
+ wantErr: "is reserved for the server's references under refs/graft/",
+ },
{
desc: "duplicate name",
agents: []config.GraftAgent{{Name: "a", Token: "one"}, {Name: "a", Token: "two"}},
diff --git a/internal/gitaly/service/commit/agent_walk_service_test.go b/internal/gitaly/service/commit/agent_walk_service_test.go
new file mode 100755
index 0000000..2f62a24
--- /dev/null
+++ b/internal/gitaly/service/commit/agent_walk_service_test.go
@@ -0,0 +1,139 @@
+package commit
+
+import (
+ "context"
+ "errors"
+ "io"
+ "testing"
+
+ "github.com/stretchr/testify/require"
+ "google.golang.org/grpc"
+ "google.golang.org/grpc/metadata"
+
+ "github.com/feder-cr/git-ai/internal/git/gittest"
+ "github.com/feder-cr/git-ai/internal/git/localrepo"
+ "github.com/feder-cr/git-ai/internal/gitaly/service"
+ "github.com/feder-cr/git-ai/internal/gitaly/service/blob"
+ "github.com/feder-cr/git-ai/internal/gitaly/service/repository"
+ "github.com/feder-cr/git-ai/internal/graft/identity"
+ "github.com/feder-cr/git-ai/internal/log"
+ "github.com/feder-cr/git-ai/internal/testhelper"
+ "github.com/feder-cr/git-ai/internal/testhelper/testcfg"
+ "github.com/feder-cr/git-ai/internal/testhelper/testserver"
+ "github.com/feder-cr/git-ai/proto/go/gitalypb"
+)
+
+// testAgentHeader is the metadata that names the agent a test request is made as.
+const testAgentHeader = "x-test-agent"
+
+// agentStream is a server stream whose context carries the identity of an agent.
+type agentStream struct {
+ grpc.ServerStream
+ ctx context.Context
+}
+
+func (s agentStream) Context() context.Context { return s.ctx }
+
+// agentInterceptors authenticates each streaming request as the agent named in its test header, the
+// way the per-agent token authentication does.
+func agentInterceptors(log.Logger, localrepo.Factory) ([]grpc.UnaryServerInterceptor, []grpc.StreamServerInterceptor) {
+ return nil, []grpc.StreamServerInterceptor{
+ func(srv any, ss grpc.ServerStream, _ *grpc.StreamServerInfo, handler grpc.StreamHandler) error {
+ if md, ok := metadata.FromIncomingContext(ss.Context()); ok {
+ if names := md.Get(testAgentHeader); len(names) > 0 {
+ ctx := identity.NewContext(ss.Context(), identity.Identity{Name: names[0], Source: identity.SourceAgent})
+ return handler(srv, agentStream{ServerStream: ss, ctx: ctx})
+ }
+ }
+ return handler(srv, ss)
+ },
+ }
+}
+
+// TestAgentWalk_rpcsReachNoProposal: an agent that asks for every reference, by --all or --glob in
+// the revisions of ListCommits and ListBlobs, or by FindCommits' all, gets neither the commits nor
+// the blobs that only a reference under refs/graft/ reaches (a proposal, whose message names other
+// agents); another caller still gets them.
+func TestAgentWalk_rpcsReachNoProposal(t *testing.T) {
+ t.Parallel()
+
+ ctx := testhelper.Context(t)
+ cfg := testcfg.Build(t)
+ cfg.SocketPath = testserver.RunGitalyServer(t, cfg, func(srv *grpc.Server, deps *service.Dependencies) {
+ gitalypb.RegisterCommitServiceServer(srv, NewServer(deps))
+ gitalypb.RegisterBlobServiceServer(srv, blob.NewServer(deps))
+ gitalypb.RegisterRepositoryServiceServer(srv, repository.NewServer(deps))
+ }, testserver.WithInterceptors(agentInterceptors))
+ conn := dial(t, cfg.SocketPath)
+ commits := gitalypb.NewCommitServiceClient(conn)
+ blobs := gitalypb.NewBlobServiceClient(conn)
+
+ repo, repoPath := gittest.CreateRepository(t, ctx, cfg)
+ main := gittest.WriteCommit(t, cfg, repoPath, gittest.WithBranch("main"),
+ gittest.WithTreeEntries(gittest.TreeEntry{Mode: "100644", Path: "a", Content: "a\n"}))
+ proposal := gittest.WriteCommit(t, cfg, repoPath, gittest.WithParents(main),
+ gittest.WithReference("refs/graft/proposals/0123456789abcdef/1"),
+ gittest.WithMessage("graft: proposal\n\nRequested-By: agent-b\n"),
+ gittest.WithTreeEntries(gittest.TreeEntry{Mode: "100644", Path: "a", Content: "proposed by a model\n"}))
+ proposedBlob := gittest.ResolveRevision(t, cfg, repoPath, proposal.String()+":a")
+
+ agent := metadata.AppendToOutgoingContext(ctx, testAgentHeader, "agent-a")
+
+ listCommits := func(ctx context.Context, revisions ...string) []string {
+ stream, err := commits.ListCommits(ctx, &gitalypb.ListCommitsRequest{Repository: repo, Revisions: revisions})
+ require.NoError(t, err)
+ var ids []string
+ for {
+ resp, err := stream.Recv()
+ if errors.Is(err, io.EOF) {
+ return ids
+ }
+ require.NoError(t, err)
+ for _, c := range resp.GetCommits() {
+ ids = append(ids, c.GetId())
+ }
+ }
+ }
+ listBlobs := func(ctx context.Context, revisions ...string) []string {
+ stream, err := blobs.ListBlobs(ctx, &gitalypb.ListBlobsRequest{Repository: repo, Revisions: revisions, Limit: 100})
+ require.NoError(t, err)
+ var ids []string
+ for {
+ resp, err := stream.Recv()
+ if errors.Is(err, io.EOF) {
+ return ids
+ }
+ require.NoError(t, err)
+ for _, b := range resp.GetBlobs() {
+ ids = append(ids, b.GetOid())
+ }
+ }
+ }
+ findAll := func(ctx context.Context) []string {
+ stream, err := commits.FindCommits(ctx, &gitalypb.FindCommitsRequest{Repository: repo, All: true, Limit: 100})
+ require.NoError(t, err)
+ var ids []string
+ for {
+ resp, err := stream.Recv()
+ if errors.Is(err, io.EOF) {
+ return ids
+ }
+ require.NoError(t, err)
+ for _, c := range resp.GetCommits() {
+ ids = append(ids, c.GetId())
+ }
+ }
+ }
+
+ for _, revisions := range [][]string{{"--all"}, {"--glob=refs/*"}, {"main", "--glob=refs/*"}} {
+ require.Contains(t, listCommits(agent, revisions...), main.String(), revisions)
+ require.NotContains(t, listCommits(agent, revisions...), proposal.String(), revisions)
+ require.Contains(t, listCommits(ctx, revisions...), proposal.String(), revisions)
+
+ require.NotContains(t, listBlobs(agent, revisions...), proposedBlob.String(), revisions)
+ require.Contains(t, listBlobs(ctx, revisions...), proposedBlob.String(), revisions)
+ }
+
+ require.Equal(t, []string{main.String()}, findAll(agent))
+ require.Contains(t, findAll(ctx), proposal.String())
+}
diff --git a/internal/gitaly/service/operations/land_proposals.go b/internal/gitaly/service/operations/land_proposals.go
index 34dc72b..8b1e22e 100644
--- a/internal/gitaly/service/operations/land_proposals.go
+++ b/internal/gitaly/service/operations/land_proposals.go
@@ -8,6 +8,7 @@ import (
"path"
"strconv"
"strings"
+ "syscall"
"time"
"github.com/prometheus/client_golang/prometheus"
@@ -15,6 +16,8 @@ import (
"github.com/feder-cr/git-ai/internal/git"
"github.com/feder-cr/git-ai/internal/git/localrepo"
+ "github.com/feder-cr/git-ai/internal/git/updateref"
+ "github.com/feder-cr/git-ai/internal/gitaly/storage"
"github.com/feder-cr/git-ai/internal/graft/proposals"
"github.com/feder-cr/git-ai/internal/graft/resolver"
"github.com/feder-cr/git-ai/internal/log"
@@ -224,7 +227,7 @@ func landProposalsNotStored(files []*landFileState, answer *resolver.Response, c
continue
}
f.proposal = nil
- note := "proposal not stored: " + cause.Error()
+ note := "proposal not stored: " + landProposalWriteCause(cause)
if f.reason == "" {
f.reason = note
} else {
@@ -233,6 +236,30 @@ func landProposalsNotStored(files []*landFileState, answer *resolver.Response, c
}
}
+// landProposalWriteCause is what the package says of a failed write of the proposals: a class of
+// the failure, never the error itself, whose git output may name references or the paths of the
+// storage. The error is logged where the write failed.
+func landProposalWriteCause(err error) string {
+ var (
+ locked updateref.AlreadyLockedError
+ mismatch updateref.MismatchingStateError
+ exists updateref.ReferenceAlreadyExistsError
+ directory updateref.FileDirectoryConflictError
+ inTx updateref.InTransactionConflictError
+ multiple updateref.MultipleUpdatesError
+ )
+ switch {
+ case errors.As(err, &locked), errors.As(err, &mismatch), errors.As(err, &exists), errors.As(err, &directory),
+ errors.As(err, &inTx), errors.As(err, &multiple), errors.Is(err, updateref.ErrPackedRefsLocked):
+ return "reference update conflict"
+ case errors.Is(err, storage.ErrRepositoryNotFound), errors.Is(err, storage.ErrStorageNotFound),
+ errors.Is(err, syscall.ENOSPC), errors.Is(err, syscall.EDQUOT), errors.Is(err, syscall.EROFS), errors.Is(err, syscall.EIO):
+ return "storage unavailable"
+ default:
+ return "write failed"
+ }
+}
+
// landPolicyDeniedError is the policy's refusal of a land that asked the server to resolve: the
// agent may not update the branch, so nothing is resolved and no proposal is kept. err is the
// refusal (PermissionDenied).
diff --git a/internal/gitaly/service/operations/land_proposals_test.go b/internal/gitaly/service/operations/land_proposals_test.go
index 7232000..5301a24 100755
--- a/internal/gitaly/service/operations/land_proposals_test.go
+++ b/internal/gitaly/service/operations/land_proposals_test.go
@@ -1,9 +1,12 @@
package operations
import (
+ "errors"
"fmt"
+ "os"
"strconv"
"strings"
+ "syscall"
"testing"
"time"
@@ -15,7 +18,9 @@ import (
"github.com/feder-cr/git-ai/internal/git"
"github.com/feder-cr/git-ai/internal/git/gittest"
+ "github.com/feder-cr/git-ai/internal/git/updateref"
gitalycfg "github.com/feder-cr/git-ai/internal/gitaly/config"
+ "github.com/feder-cr/git-ai/internal/gitaly/storage"
"github.com/feder-cr/git-ai/internal/graft/identity"
"github.com/feder-cr/git-ai/internal/graft/proposals"
"github.com/feder-cr/git-ai/internal/graft/resolver"
@@ -226,7 +231,10 @@ func TestUserLand_proposals(t *testing.T) {
f := packageFile(t, pkg, p)
require.Equal(t, gitalypb.LandConflictFile_OUTCOME_PROPOSED, f.GetOutcome(), p)
require.Nil(t, f.GetProposal(), p)
- require.Contains(t, f.GetReason(), "proposal not stored: ", p)
+ // The class of the failure, not git's words, which name the references.
+ require.Contains(t, f.GetReason(), "proposal not stored: reference update conflict", p)
+ require.NotContains(t, f.GetReason(), "refs/", p)
+ require.NotContains(t, f.GetReason(), fx.repoPath, p)
}
})
@@ -336,3 +344,18 @@ func TestResolveLandConflicts_policyFirst(t *testing.T) {
require.ErrorAs(t, err, &unresolved)
require.NotEmpty(t, proposalRefTarget(t, fx))
}
+
+// TestLandProposalWriteCause: a failed write of proposals is reported by its class.
+func TestLandProposalWriteCause(t *testing.T) {
+ t.Parallel()
+
+ for want, err := range map[string]error{
+ "reference update conflict": fmt.Errorf("updating x: %w", updateref.MismatchingStateError{ReferenceName: "refs/graft/proposals/x"}),
+ "storage unavailable": fmt.Errorf("writing: %w", &os.PathError{Op: "write", Path: "/storage/@hashed/ab", Err: syscall.ENOSPC}),
+ "write failed": errors.New("exit status 128: fatal: unable to write /storage/@hashed/ab"),
+ } {
+ require.Equal(t, want, landProposalWriteCause(err))
+ }
+ require.Equal(t, "storage unavailable", landProposalWriteCause(storage.NewRepositoryNotFoundError("default", "x.git")))
+ require.Equal(t, "reference update conflict", landProposalWriteCause(updateref.AlreadyLockedError{ReferenceName: "refs/graft/proposals/x"}))
+}
diff --git a/internal/graft/agentrefs/agentrefs.go b/internal/graft/agentrefs/agentrefs.go
index 62da05a..2e0747a 100644
--- a/internal/graft/agentrefs/agentrefs.go
+++ b/internal/graft/agentrefs/agentrefs.go
@@ -23,8 +23,8 @@ func Hidden(ref string) bool {
}
// RevisionHidden reports whether a revision, reference or pattern an agent sends names a reference
-// under HiddenPrefix: a part of its revision (before a ":path", between "..", "...", "^", "~" or
-// "@{") that starts with refs/graft/, or with graft/<namespace>/ for a namespace of the server
+// under HiddenPrefix: a part of its revision (before a ":path", between "..", "...", "^", "~", "@{"
+// or "=", as in --glob=<pattern>) that starts with refs/graft/, or with graft/<namespace>/ for a namespace of the server
// (ServerNamespaces), which git completes to refs/graft/<namespace>/. Other short names under
// graft/ are branches (the workspace's temporary branches are graft/<agent>/...): git completes
// them under refs/heads/, since nothing else exists under refs/graft/. The path after the first
@@ -32,7 +32,7 @@ func Hidden(ref string) bool {
func RevisionHidden(rev string) bool {
rev, _, _ = strings.Cut(rev, ":")
for _, part := range strings.FieldsFunc(rev, func(r rune) bool {
- return r == '.' || r == '^' || r == '~' || r == '@' || r == '{' || r == '}' || r == ' '
+ return r == '.' || r == '^' || r == '~' || r == '@' || r == '{' || r == '}' || r == ' ' || r == '='
}) {
part = strings.TrimPrefix(part, "!")
if strings.HasPrefix(part, HiddenPrefix) {
diff --git a/internal/graft/agentrefs/agentrefs_test.go b/internal/graft/agentrefs/agentrefs_test.go
index 50da016..f551e13 100644
--- a/internal/graft/agentrefs/agentrefs_test.go
+++ b/internal/graft/agentrefs/agentrefs_test.go
@@ -22,6 +22,10 @@ func TestRevisionHidden(t *testing.T) {
"refs/heads/graft/x": false,
"main": false,
"refs/*": false,
+ "--glob=refs/graft/*": true, // a pattern after =
+ "--glob=graft/proposals/*": true,
+ "--glob=refs/*": false, // gitcmd leaves refs/graft/ out of the walk
+ "--all": false, // likewise
"0123456789abcdef0123456789abcdef01234": false,
"main:refs/notes/graft": false,
"main:graft/x.go": false, // a path, not a reference
diff --git a/internal/graft/proposals/proposals.go b/internal/graft/proposals/proposals.go
index e16739b..04d01b1 100644
--- a/internal/graft/proposals/proposals.go
+++ b/internal/graft/proposals/proposals.go
@@ -8,15 +8,20 @@ import (
"context"
"crypto/sha256"
"encoding/hex"
+ "errors"
"fmt"
"sort"
"strings"
"time"
+ "github.com/prometheus/client_golang/prometheus"
+ "github.com/prometheus/client_golang/prometheus/promauto"
+
"github.com/feder-cr/git-ai/internal/git"
"github.com/feder-cr/git-ai/internal/git/localrepo"
"github.com/feder-cr/git-ai/internal/git/updateref"
"github.com/feder-cr/git-ai/internal/graft/identity"
+ "github.com/feder-cr/git-ai/internal/log"
)
const (
@@ -97,11 +102,18 @@ func Update(ctx context.Context, repo *localrepo.Repo, ops []Op) error {
return nil
}
+// pruneSkipped counts the proposal references that Prune left because they point to no commit.
+var pruneSkipped = promauto.NewCounter(prometheus.CounterOpts{
+ Name: "graft_proposal_prune_skipped_total",
+ Help: "Proposal references left by the pruning because they point to no commit.",
+})
+
// Prune deletes, among the proposal references under dir (a branch's BranchDir, or RefPrefix for
// every branch), those older than ttl, those beyond the newest MaxPerBranch of each branch, and
// drop when it is set. The age is the committer date of the proposals' commit, which the server
// writes with its own clock (never the request's). Each deletion expects the target it read. It
-// returns the number of references deleted.
+// returns the number of references deleted. A reference to no commit is left, logged and counted
+// (graft_proposal_prune_skipped_total), and does not stop the pruning of the others.
func Prune(ctx context.Context, repo *localrepo.Repo, dir string, ttl time.Duration, drop git.ReferenceName, now time.Time) (int, error) {
if !strings.HasPrefix(dir, RefPrefix) {
return 0, fmt.Errorf("pruning proposals: %q is not under %s", dir, RefPrefix)
@@ -130,6 +142,14 @@ func Prune(ctx context.Context, repo *localrepo.Repo, dir string, ttl time.Durat
continue
}
c, err := repo.ReadCommit(ctx, git.Revision(r.Target))
+ if errors.Is(err, localrepo.ErrObjectNotFound) {
+ // A reference to no commit was not written by Update: it is left as it is, and the
+ // others are pruned.
+ pruneSkipped.Inc()
+ log.FromContext(ctx).WithField("reference", r.Name.String()).WarnContext(ctx,
+ "graft: a proposal reference points to no commit; it is not pruned")
+ continue
+ }
if err != nil {
return 0, fmt.Errorf("reading %s: %w", r.Name, err)
}
diff --git a/internal/graft/proposals/proposals_test.go b/internal/graft/proposals/proposals_test.go
index f359d34..59f4f51 100755
--- a/internal/graft/proposals/proposals_test.go
+++ b/internal/graft/proposals/proposals_test.go
@@ -6,6 +6,7 @@ import (
"testing"
"time"
+ "github.com/prometheus/client_golang/prometheus/testutil"
"github.com/stretchr/testify/require"
"github.com/feder-cr/git-ai/internal/git"
@@ -109,6 +110,32 @@ func TestPrune(t *testing.T) {
require.ErrorContains(t, err, "is not under")
}
+// TestPrune_skipsAReferenceToNoCommit: a proposal reference that points to a blob, which Update
+// never writes, is left and counted, and the pruning of the others goes on.
+func TestPrune_skipsAReferenceToNoCommit(t *testing.T) {
+ t.Parallel()
+
+ repo, _, commit, list := setup(t)
+ ctx := testhelper.Context(t)
+ now := time.Now()
+ zero := gittest.DefaultObjectHash.ZeroOID
+ blob, err := repo.WriteBlob(ctx, strings.NewReader("not a commit"), localrepo.WriteBlobConfig{})
+ require.NoError(t, err)
+ stray := git.ReferenceName(fmt.Sprintf("%s%040x", BranchDir("main"), 1))
+ old := git.ReferenceName(fmt.Sprintf("%s%040x", BranchDir("main"), 2))
+ require.NoError(t, Update(ctx, repo, []Op{
+ {Name: stray, NewOID: blob, OldOID: zero},
+ {Name: old, NewOID: commit(now.Add(-8 * 24 * time.Hour)), OldOID: zero},
+ }))
+
+ before := testutil.ToFloat64(pruneSkipped)
+ n, err := Prune(ctx, repo, RefPrefix, 7*24*time.Hour, "", now)
+ require.NoError(t, err)
+ require.Equal(t, 1, n, "the old proposal is pruned")
+ require.Equal(t, []string{stray.String()}, list(BranchDir("main")))
+ require.GreaterOrEqual(t, testutil.ToFloat64(pruneSkipped)-before, 1.0)
+}
+
// TestUpdate_compareAndSwap: a deletion that expects a target the reference no longer has fails,
// and the reference stays.
func TestUpdate_compareAndSwap(t *testing.T) {