Skip to content

Git designed for AI agents.

Last review of dac3ea4, items 1-5: startup cleanup, build lock, lockless age, typed invariant U, deviation path

Federico Eliab72bf630092a
diff --git a/doc/graft/PLAN7.md b/doc/graft/PLAN7.md
index 54f32bb..652b337 100644
--- a/doc/graft/PLAN7.md
+++ b/doc/graft/PLAN7.md
@@ -1753,13 +1753,26 @@ leaves it:
* **Lifecycle (final minor 3).** `StopSeeds` (deferred by `gitaly serve` after
its servers, before the last durable step) cancels the seed jobs, waits for
them, and refuses new ones ("the server is stopping"). Every build directory
- (`hydrate-*`) holds a flock on its `build.lock` until it is removed;
- `CleanBuilds`, at startup before the listeners, removes those whose lock is
- free (their process died) or that have none and are older than a minute,
- and keeps those of a process that runs (the old one of a graceful upgrade).
+ (`hydrate-*`) holds a flock on its `build.lock` until it is removed; after
+ the flock the build checks that the file it locked is still the one at the
+ path, else it fails (a cleanup found the lock free in between and removed
+ the directory). `CleanBuilds`, at startup before the listeners, removes the
+ directories whose lock is free (their process died), and those without a
+ lock file older than the larger of W and `sync_timeout` (no hydration or
+ seed lives longer; a v0.2 process, whose builds have no lock, may still run
+ one during a graceful upgrade); it keeps those of a process that runs. A
+ directory gone meanwhile (its process removed it) is not an error; one that
+ cannot be checked or removed is kept, logged and counted
+ (`graft_storage_build_cleanup_failures_total`): the cleanup never stops the
+ server.
* Tests: `TestSeedSupersededByANewerCheckpoint`, `TestSeedPastItsDeadline`,
`TestSeedOfARepositoryRemovedAndCreatedAgain`, `TestStopSeedsEndsTheSeedJobs`,
- `TestCleanBuildsRemovesOnlyDeadBuilds`.
+ `TestCleanBuildsRemovesOnlyDeadBuilds`, `TestCleanBuildsNeverStopsTheServer`,
+ `TestLockBuildFailsWhenItsDirectoryWasCleanedUp`, and on Postgres
+ `TestSeedInstalledPastItsDeadlineThenSuperseded` (the path above: a seed that
+ indexed its pack is installed past its deadline on a database clock that
+ runs ahead of the host's, then C_sync moves and the next pass seeds again
+ from the newest).
**Tests.** `TestSyncerResumesAHydrationAcrossRounds`: rounds of 4 changesets
on a 10-changeset repository give Unavailable at 4 and at 8, with the
@@ -2313,11 +2326,12 @@ path: the voter is m3's; the land paths call it unchanged.
repository row lock.
26. **One upload path**: every production upload goes through `Core.UploadPack`, and
every row that references an upload is committed under its `CommitContext`.
- Enforced by `TestUploadPackIsTheOnlyUploader` (`internal/graft/corestore`): in
- the non-test code that can reach a `*corestore.Core`, `Blobs.Put` is called in
- `core.go` only and `Blobs` is never taken as a value outside the core's wiring
- (`core.go`, `open/open.go`). It is invariant U of the reconciliation
- ("Removed repositories").
+ Enforced by `TestUploadPackIsTheOnlyUploader` (`internal/graft/corestore`):
+ the module's non-test packages that depend on corestore are type-checked
+ (`go list -export` and `go/types`, no new dependency), and the calls of
+ `Put` on a value whose static type implements `corestore.BlobStore`,
+ whatever its name, must be exactly one, in `Core.UploadPack`. It is
+ invariant U of the reconciliation ("Removed repositories").
27. **Local references are a cache of the database**: in authoritative mode a local
reference write without fsync happens only after its repository has a durable
dirty marker; a marker is removed only after the stack is durable, under git's
diff --git a/doc/graft/STORAGE.md b/doc/graft/STORAGE.md
index a8ce3c6..76dfea9 100644
--- a/doc/graft/STORAGE.md
+++ b/doc/graft/STORAGE.md
@@ -721,6 +721,7 @@ without a single divergence.
| `graft_storage_sync_seconds` | Duration of the catch-up before a request. |
| `graft_storage_sync_changesets_total`, `graft_storage_sync_bytes_total` | Changesets applied and pack bytes downloaded by catch-up and hydration. |
| `graft_storage_sync_checkpoint_seeds_total` | Hydrations that started from a checkpoint instead of changeset 1. |
+| `graft_storage_build_cleanup_failures_total` | Build directories of hydrations (`+gitaly/tmp/hydrate-*`) that the startup cleanup could not check or remove; they are kept and logged. |
| `graft_storage_maintenance_runs_total{result}` | Runs of the background maintenance; `ok`, or `error` when at least one step failed. |
| `graft_storage_maintenance_checkpoints_total` | Checkpoints created by the background maintenance. |
| `graft_storage_maintenance_packs_deleted_total`, `graft_storage_maintenance_bytes_reclaimed_total` | Bucket objects, and their bytes, deleted by the background garbage collection. |
diff --git a/internal/cli/gitaly/serve.go b/internal/cli/gitaly/serve.go
index 2047bae..be0cf21 100644
--- a/internal/cli/gitaly/serve.go
+++ b/internal/cli/gitaly/serve.go
@@ -643,10 +643,9 @@ func run(appCtx *cli.Command, cfg config.Cfg, logger log.Logger) error {
}
// The build directories of hydrations of a process that died are
// removed; those of a process that runs (the old one of a graceful
- // upgrade) hold their lock and are kept.
- if err := syncer.CleanBuilds(ctx, storageNames); err != nil {
- return err
- }
+ // upgrade) hold their lock and are kept. A failure is logged and
+ // counted: it never stops the server.
+ syncer.CleanBuilds(ctx, storageNames)
stopLocalReferences, err := syncer.StartLocalReferences(ctx, storageNames)
if err != nil {
return err
diff --git a/internal/graft/corestore/syncer/builds.go b/internal/graft/corestore/syncer/builds.go
index 3c40337..9c24578 100644
--- a/internal/graft/corestore/syncer/builds.go
+++ b/internal/graft/corestore/syncer/builds.go
@@ -19,79 +19,122 @@ import (
// of a graceful upgrade).
const buildLockName = "build.lock"
-// unlockedBuildAge is how old a build directory without its lock file must be
-// before CleanBuilds removes it: one whose process died between creating the
-// directory and its lock file, which takes microseconds.
-const unlockedBuildAge = time.Minute
-
// lockBuild creates the lock file of the build directory dir and takes its
-// flock.
-func lockBuild(dir string) (*os.File, error) {
- f, err := os.OpenFile(filepath.Join(dir, buildLockName), os.O_CREATE|os.O_RDWR, 0o600)
+// flock. The file locked must still be the one at the path: CleanBuilds may
+// have found the lock file free before the flock, as for a process that died,
+// and removed the directory; the build then fails. afterOpen, when not nil (a
+// test), is called between the open and the flock.
+func lockBuild(dir string, afterOpen func(path string)) (*os.File, error) {
+ path := filepath.Join(dir, buildLockName)
+ f, err := os.OpenFile(path, os.O_CREATE|os.O_RDWR, 0o600)
if err != nil {
return nil, err
}
+ if afterOpen != nil {
+ afterOpen(path)
+ }
if err := syscall.Flock(int(f.Fd()), syscall.LOCK_EX|syscall.LOCK_NB); err != nil {
_ = f.Close()
return nil, fmt.Errorf("lock: %w", err)
}
+ locked, err := f.Stat()
+ if err != nil {
+ _ = f.Close()
+ return nil, fmt.Errorf("lock: %w", err)
+ }
+ current, err := os.Stat(path)
+ if err != nil || !os.SameFile(locked, current) {
+ _ = f.Close()
+ return nil, fmt.Errorf("lock: the build directory was removed before its lock was taken (%v)", err)
+ }
return f, nil
}
+// unlockedBuildAge is how old a build directory without a lock file must be
+// before CleanBuilds removes it: one whose process died between creating the
+// directory and its lock file, or one of a version before the lock files
+// (v0.2), which may still run during a graceful upgrade. No hydration or seed
+// lives longer than the larger of a seed's deadline W and sync_timeout, so an
+// older directory has no process.
+func (s *Syncer) unlockedBuildAge() time.Duration {
+ return max(s.core.WriteDeadline(), s.syncTimeout())
+}
+
// CleanBuilds removes the build directories (hydrate-*) of hydrations and
// seeds that no process runs any more: their process died (a crash, a kill)
// before it removed them, and nothing else does. A directory whose lock file
-// another process holds is in use and kept. gitaly serve calls it at startup,
-// before the listeners open (final minor 3).
-func (s *Syncer) CleanBuilds(ctx context.Context, storages []string) error {
+// another process holds is in use and kept. A directory gone meanwhile was
+// removed by its process. A directory that cannot be checked or removed is
+// kept, logged and counted (graft_storage_build_cleanup_failures_total): the
+// cleanup never stops the server. gitaly serve calls it at startup, before
+// the listeners open (final minor 3).
+func (s *Syncer) CleanBuilds(ctx context.Context, storages []string) {
removed := 0
+ fail := func(dir string, err error) {
+ s.buildCleanupFailures.Inc()
+ s.logger.WithError(err).WithField("directory", dir).WarnContext(ctx, "storage core: a build directory could not be cleaned up; it is kept")
+ }
for _, storage := range storages {
tmpDir, err := s.locator.TempDir(storage)
if err != nil {
- return err
+ fail(storage, err)
+ continue
}
entries, err := os.ReadDir(tmpDir)
if errors.Is(err, fs.ErrNotExist) {
continue
}
if err != nil {
- return fmt.Errorf("storage core: list the temporary directory of %s: %w", storage, err)
+ fail(tmpDir, err)
+ continue
}
for _, entry := range entries {
if !entry.IsDir() || !strings.HasPrefix(entry.Name(), strings.TrimSuffix(tmpHydratePatt, "*")) {
continue
}
dir := filepath.Join(tmpDir, entry.Name())
- stale, err := staleBuild(dir)
+ ok, err := s.cleanBuild(dir)
if err != nil {
- return fmt.Errorf("storage core: check the build directory %s: %w", dir, err)
- }
- if !stale {
+ fail(dir, err)
continue
}
- if err := os.RemoveAll(dir); err != nil {
- return fmt.Errorf("storage core: remove the build directory %s: %w", dir, err)
+ if ok {
+ removed++
}
- removed++
}
}
if removed > 0 {
s.logger.WithField("directories", removed).InfoContext(ctx, "storage core: removed the build directories of hydrations that no process runs")
}
- return nil
+}
+
+// cleanBuild removes the build directory dir if no process runs its build,
+// and reports whether it did. A directory that is gone is not an error.
+func (s *Syncer) cleanBuild(dir string) (bool, error) {
+ stale, err := s.staleBuild(dir)
+ if errors.Is(err, fs.ErrNotExist) {
+ return false, nil
+ }
+ if err != nil || !stale {
+ return false, err
+ }
+ if err := os.RemoveAll(dir); err != nil {
+ return false, err
+ }
+ return true, nil
}
// staleBuild reports whether no process runs the build in dir: its lock file
// can be locked, or it has none and dir is older than unlockedBuildAge. A
// lock taken here is released when the directory is gone or kept.
-func staleBuild(dir string) (bool, error) {
+func (s *Syncer) staleBuild(dir string) (bool, error) {
f, err := os.OpenFile(filepath.Join(dir, buildLockName), os.O_RDWR, 0)
if errors.Is(err, fs.ErrNotExist) {
info, err := os.Stat(dir)
if err != nil {
return false, err
}
- return time.Since(info.ModTime()) >= unlockedBuildAge, nil
+ return time.Since(info.ModTime()) >= s.unlockedBuildAge(), nil
}
if err != nil {
return false, err
diff --git a/internal/graft/corestore/syncer/export_test.go b/internal/graft/corestore/syncer/export_test.go
index 7203704..9ab2d4a 100644
--- a/internal/graft/corestore/syncer/export_test.go
+++ b/internal/graft/corestore/syncer/export_test.go
@@ -1,9 +1,26 @@
package syncer
-import "context"
+import (
+ "context"
+ "os"
+)
+
+// CleanBuild runs the cleanup of one build directory, as CleanBuilds does for
+// each.
+func CleanBuild(s *Syncer, dir string) (bool, error) {
+ return s.cleanBuild(dir)
+}
+
+// LockBuild takes the lock of the build directory dir as a build does, calling
+// afterOpen between the open of the lock file and its flock.
+func LockBuild(dir string, afterOpen func(path string)) (*os.File, error) {
+ return lockBuild(dir, afterOpen)
+}
// SetTestHook installs a function called at the named points of a catch-up
-// ("fetched", "checked"). Only for tests; set it before the first EnsureFresh.
+// ("fetched", "checked") and of a seed ("seed" before the pack is indexed,
+// "seeded" after, "installed" after the install). Only for tests; set it
+// before the first EnsureFresh.
func (s *Syncer) SetTestHook(hook func(point string)) {
s.testHook = hook
}
diff --git a/internal/graft/corestore/syncer/syncer.go b/internal/graft/corestore/syncer/syncer.go
index e1bad94..2711f9a 100644
--- a/internal/graft/corestore/syncer/syncer.go
+++ b/internal/graft/corestore/syncer/syncer.go
@@ -159,6 +159,9 @@ type Syncer struct {
syncChangesets prometheus.Counter
syncBytes prometheus.Counter
syncSeeds prometheus.Counter
+ // buildCleanupFailures counts the build directories CleanBuilds could not
+ // check or remove.
+ buildCleanupFailures prometheus.Counter
// keptStorages are the storages whose kept untrusted copies (KeptDirName)
// the metrics report, set by StartLocalReferences.
@@ -217,6 +220,10 @@ func New(core *corestore.Core, logger log.Logger, locator storage.Locator, repoF
Name: "graft_storage_sync_checkpoint_seeds_total",
Help: "Number of hydrations that started from a checkpoint instead of changeset 1.",
}),
+ buildCleanupFailures: prometheus.NewCounter(prometheus.CounterOpts{
+ Name: "graft_storage_build_cleanup_failures_total",
+ Help: "Number of build directories of hydrations that the startup cleanup could not check or remove; they are kept.",
+ }),
}
for _, result := range []string{resultFresh, resultCaughtUp, resultHydrated, resultPartial, resultSeeding, resultError} {
s.syncTotal.WithLabelValues(result)
@@ -342,6 +349,7 @@ func (s *Syncer) Describe(descs chan<- *prometheus.Desc) {
s.syncChangesets.Describe(descs)
s.syncBytes.Describe(descs)
s.syncSeeds.Describe(descs)
+ s.buildCleanupFailures.Describe(descs)
descs <- keptRepositoriesDesc
descs <- keptBytesDesc
}
@@ -356,6 +364,7 @@ func (s *Syncer) Collect(metrics chan<- prometheus.Metric) {
s.syncChangesets.Collect(metrics)
s.syncBytes.Collect(metrics)
s.syncSeeds.Collect(metrics)
+ s.buildCleanupFailures.Collect(metrics)
s.collectKept(metrics)
}
@@ -1569,7 +1578,7 @@ func (s *Syncer) newBuild(ctx context.Context, repo *gitalypb.Repository, r core
return nil, structerr.NewInternal("storage core: create hydration directory: %w", err)
}
b := &build{dir: dir, path: filepath.Join(dir, "repo.git"), tmpDir: tmpDir}
- if b.lock, err = lockBuild(dir); err != nil {
+ if b.lock, err = lockBuild(dir, nil); err != nil {
b.remove()
return nil, structerr.NewInternal("storage core: lock the hydration directory: %w", err)
}
@@ -1778,6 +1787,7 @@ func (s *Syncer) seedAndInstall(ctx context.Context, repo *gitalypb.Repository,
if err != nil {
return err
}
+ s.hook("seeded")
// A seed that indexed the whole pack holds every object of the
// checkpoint, verified against its key, so it may be installed after its
// deadline; what the deadline protects is the changeset packs after the
@@ -1796,6 +1806,7 @@ func (s *Syncer) seedAndInstall(ctx context.Context, repo *gitalypb.Repository,
case err != nil:
return err
}
+ s.hook("installed")
s.syncBytes.Add(float64(n))
s.report(ctx, r, cp.Seq)
diff --git a/internal/graft/corestore/syncer/syncer_seed_test.go b/internal/graft/corestore/syncer/syncer_seed_test.go
index d22c4e5..e0bce69 100644
--- a/internal/graft/corestore/syncer/syncer_seed_test.go
+++ b/internal/graft/corestore/syncer/syncer_seed_test.go
@@ -1,9 +1,12 @@
package syncer_test
import (
+ "context"
"os"
"path/filepath"
+ "slices"
"sync"
+ "sync/atomic"
"syscall"
"testing"
"time"
@@ -13,7 +16,9 @@ import (
"google.golang.org/grpc/status"
"github.com/feder-cr/git-ai/internal/graft/corestore"
+ "github.com/feder-cr/git-ai/internal/graft/corestore/coretest"
"github.com/feder-cr/git-ai/internal/graft/corestore/sqlstore"
+ "github.com/feder-cr/git-ai/internal/graft/corestore/syncer"
)
// seedFixture is a repository written on node a with a checkpoint followed by
@@ -138,6 +143,78 @@ func TestSeedOfARepositoryRemovedAndCreatedAgain(t *testing.T) {
f.b.verifySynced(t, again)
}
+// skewedMeta is a MetaStore whose database clock, as GetRepository reports
+// it, runs skew ahead of the host's: the host was suspended, its monotonic
+// clock stopped, the database's did not.
+type skewedMeta struct {
+ corestore.MetaStore
+ skew atomic.Int64
+}
+
+func (m *skewedMeta) GetRepository(ctx context.Context, key corestore.RepoKey) (corestore.Repository, error) {
+ r, err := m.MetaStore.GetRepository(ctx, key)
+ r.ReadAt = r.ReadAt.Add(time.Duration(m.skew.Load()))
+ return r, err
+}
+
+// TestSeedInstalledPastItsDeadlineThenSuperseded is the path of a seed that
+// indexed its whole pack and passed its deadline on the database clock (the
+// host's clock lagged): it is installed, its objects being complete; then
+// C_sync moves past it, and the next pass of the round finds the copy below
+// C_sync and seeds again from the newest checkpoint, never catching up over
+// packs that may be collected (final review item 5). On Postgres, whose clock
+// is the database's own.
+func TestSeedInstalledPastItsDeadlineThenSuperseded(t *testing.T) {
+ t.Parallel()
+ if !slices.Contains(coretest.Dialects(), sqlstore.Postgres) {
+ t.Skip("Postgres is not configured")
+ }
+
+ a := newEnv(t, envOptions{dialect: sqlstore.Postgres})
+ src := a.newSource(t)
+ c := a.writeCommit(t, src)
+ repo := a.register(t, a.key(localRelPath), src, c)
+ c = a.advanceMain(t, repo.ID, src, c)
+ first := a.checkpoint(t, repo.ID, src)
+ a.advanceMain(t, repo.ID, src, c)
+
+ b := a.secondNode(t, sqlstore.Postgres)
+ b.core.SyncTimeout = time.Minute
+ skewed := &skewedMeta{MetaStore: b.core.Meta}
+ b.core.Meta = skewed
+
+ var (
+ installs atomic.Int32
+ pastDeadline atomic.Bool
+ newer corestore.Checkpoint
+ )
+ b.sy.SetTestHook(func(point string) {
+ switch {
+ case point == "seeded" && installs.Load() == 0:
+ // The whole pack is indexed; on the database clock the seed's
+ // deadline (its record's read + W) has passed.
+ skewed.skew.Store(int64(2 * b.core.WriteDeadline()))
+ r, err := skewed.GetRepository(b.ctx, repo.Key)
+ require.NoError(t, err)
+ pastDeadline.Store(r.ReadAt.After(time.Now().Add(b.core.WriteDeadline())))
+ case point == "installed" && installs.Add(1) == 1:
+ // C_sync moves past the installed copy.
+ newer = a.checkpoint(t, repo.ID, src)
+ a.ageCheckpoint(t, repo.ID, newer.Seq, b.core.GCGrace+time.Minute)
+ }
+ })
+
+ require.NoError(t, b.sy.EnsureFresh(b.ctx, b.pbRepo(localRelPath)))
+ require.True(t, pastDeadline.Load(), "the first seed was past its deadline on the database clock")
+ require.Equal(t, int32(2), installs.Load(), "installed past the deadline, then seeded again from the newest")
+ require.Greater(t, newer.Seq, first.Seq)
+ require.Equal(t, float64(2), counter(t, b.sy, "graft_storage_sync_checkpoint_seeds_total", ""))
+
+ current, err := b.core.Meta.GetRepository(b.ctx, repo.Key)
+ require.NoError(t, err)
+ b.verifySynced(t, current)
+}
+
// TestStopSeedsEndsTheSeedJobs: StopSeeds cancels a running seed, waits for it
// and its build directory to be gone, and no seed starts after it (final
// minor 3).
@@ -199,13 +276,19 @@ func TestCleanBuildsRemovesOnlyDeadBuilds(t *testing.T) {
require.NoError(t, syscall.Flock(int(lockFile(running).Fd()), syscall.LOCK_EX|syscall.LOCK_NB))
dead := mkdir("hydrate-dead")
require.NoError(t, lockFile(dead).Close(), "a lock file nobody holds: its process died")
+ // A directory without a lock file (a v0.2 process, or one that died before
+ // its lock) is removed only once it is older than any hydration or seed
+ // lives: the larger of W and sync_timeout.
+ limit := max(e.core.WriteDeadline(), e.core.SyncTimeout)
oldUnlocked := mkdir("hydrate-old-unlocked")
- require.NoError(t, os.Chtimes(oldUnlocked, time.Now().Add(-2*time.Minute), time.Now().Add(-2*time.Minute)))
- newUnlocked := mkdir("hydrate-new-unlocked")
+ require.NoError(t, os.Chtimes(oldUnlocked, time.Now().Add(-limit-time.Minute), time.Now().Add(-limit-time.Minute)))
+ youngUnlocked := mkdir("hydrate-young-unlocked")
+ require.NoError(t, os.Chtimes(youngUnlocked, time.Now().Add(-2*time.Minute), time.Now().Add(-2*time.Minute)))
other := mkdir("stale-other")
- require.NoError(t, e.sy.CleanBuilds(e.ctx, []string{e.storageName}))
- for dir, kept := range map[string]bool{running: true, dead: false, oldUnlocked: false, newUnlocked: true, other: true} {
+ e.sy.CleanBuilds(e.ctx, []string{e.storageName})
+ require.Zero(t, counter(t, e.sy, "graft_storage_build_cleanup_failures_total", ""))
+ for dir, kept := range map[string]bool{running: true, dead: false, oldUnlocked: false, youngUnlocked: true, other: true} {
_, err := os.Stat(dir)
if kept {
require.NoError(t, err, "%s is kept", dir)
@@ -214,3 +297,63 @@ func TestCleanBuildsRemovesOnlyDeadBuilds(t *testing.T) {
}
}
}
+
+// TestCleanBuildsNeverStopsTheServer: a build directory that its process
+// removes while the cleanup looks at it is not an error, and one that cannot
+// be checked is kept, logged and counted; the cleanup goes on (final review
+// item 1).
+func TestCleanBuildsNeverStopsTheServer(t *testing.T) {
+ t.Parallel()
+ e := newEnv(t, envOptions{})
+ tmpDir, err := e.locator.TempDir(e.storageName)
+ require.NoError(t, err)
+ require.NoError(t, os.MkdirAll(tmpDir, 0o700))
+
+ // Gone between the listing and the check: the old process of a graceful
+ // upgrade removed its finished build.
+ removed, err := syncer.CleanBuild(e.sy, filepath.Join(tmpDir, "hydrate-gone"))
+ require.NoError(t, err)
+ require.False(t, removed)
+
+ // A lock file that cannot be opened: kept, counted, and the dead build
+ // next to it is still removed.
+ unreadable := filepath.Join(tmpDir, "hydrate-unreadable")
+ require.NoError(t, os.MkdirAll(unreadable, 0o700))
+ require.NoError(t, os.WriteFile(filepath.Join(unreadable, "build.lock"), nil, 0o000))
+ dead := filepath.Join(tmpDir, "hydrate-dead")
+ require.NoError(t, os.MkdirAll(dead, 0o700))
+ require.NoError(t, os.WriteFile(filepath.Join(dead, "build.lock"), nil, 0o600))
+
+ e.sy.CleanBuilds(e.ctx, []string{e.storageName})
+ require.Equal(t, 1.0, counter(t, e.sy, "graft_storage_build_cleanup_failures_total", ""))
+ _, err = os.Stat(unreadable)
+ require.NoError(t, err, "kept")
+ _, err = os.Stat(dead)
+ require.ErrorIs(t, err, os.ErrNotExist, "the cleanup went on")
+}
+
+// TestLockBuildFailsWhenItsDirectoryWasCleanedUp: CleanBuilds may find the
+// lock file of a build free between its creation and its flock, and remove
+// the directory; the build then fails instead of running without its lock
+// (final review item 2).
+func TestLockBuildFailsWhenItsDirectoryWasCleanedUp(t *testing.T) {
+ t.Parallel()
+ dir := t.TempDir()
+
+ f, err := syncer.LockBuild(dir, nil)
+ require.NoError(t, err)
+ require.NoError(t, f.Close())
+
+ other := t.TempDir()
+ _, err = syncer.LockBuild(other, func(path string) {
+ // The cleanup's work, between the open and the flock: the file opened
+ // is unlinked; a new build or a cleanup may then create another.
+ require.NoError(t, os.Remove(path))
+ require.NoError(t, os.WriteFile(path, nil, 0o600))
+ })
+ require.ErrorContains(t, err, "removed before its lock was taken")
+
+ gone := t.TempDir()
+ _, err = syncer.LockBuild(gone, func(path string) { require.NoError(t, os.RemoveAll(filepath.Dir(path))) })
+ require.ErrorContains(t, err, "removed before its lock was taken")
+}
diff --git a/internal/graft/corestore/uploadpack_only_test.go b/internal/graft/corestore/uploadpack_only_test.go
index 1405f67..b9ef394 100644
--- a/internal/graft/corestore/uploadpack_only_test.go
+++ b/internal/graft/corestore/uploadpack_only_test.go
@@ -1,116 +1,157 @@
package corestore_test
import (
+ "bytes"
+ "encoding/json"
+ "errors"
+ "fmt"
"go/ast"
+ "go/importer"
"go/parser"
"go/token"
- "io/fs"
+ "go/types"
+ "io"
"os"
+ "os/exec"
"path/filepath"
- "strings"
+ "slices"
"testing"
"github.com/stretchr/testify/require"
)
+const corestorePath = "github.com/feder-cr/git-ai/internal/graft/corestore"
+
+// listedPackage is the part of `go list -json` the test reads.
+type listedPackage struct {
+ ImportPath string
+ Dir string
+ GoFiles []string
+ Export string
+ Deps []string
+ Module *struct{ Path string }
+}
+
+// goList runs go list -json with args in the module root and decodes the
+// packages it prints.
+func goList(t *testing.T, root string, args ...string) []listedPackage {
+ t.Helper()
+ cmd := exec.Command("go", append([]string{"list", "-e", "-json=ImportPath,Dir,GoFiles,Export,Deps,Module"}, args...)...)
+ cmd.Dir = root
+ var stderr bytes.Buffer
+ cmd.Stderr = &stderr
+ out, err := cmd.Output()
+ require.NoError(t, err, "go list: %s", stderr.String())
+
+ var pkgs []listedPackage
+ dec := json.NewDecoder(bytes.NewReader(out))
+ for {
+ var p listedPackage
+ if err := dec.Decode(&p); errors.Is(err, io.EOF) {
+ break
+ } else {
+ require.NoError(t, err)
+ }
+ pkgs = append(pkgs, p)
+ }
+ return pkgs
+}
+
// TestUploadPackIsTheOnlyUploader enforces invariant U (doc/graft/PLAN7.md,
// RULE 26 and "Removed repositories"): every object of the bucket is written
-// by Core.UploadPack, which needs a repository record that a read found. The
-// bucket is reached through the field Core.Blobs only, so in the module's
-// non-test code:
-// - Blobs.Put is called in core.go only (UploadPack);
-// - Blobs is not taken as a value (assigned, passed, returned), which would
-// let a Put happen under another name, except where the core is built.
-//
-// Tests may write the bucket directly to set up states. The files checked are
-// those that can reach a *corestore.Core: they import corestore or one of its
-// packages, call the dependencies' GetGraftStorage, or are in internal/graft
-// or the storagecore service (other fields named Blobs, in protobuf messages
-// and git packfile code, are unrelated).
+// by Core.UploadPack, which needs a repository record that a read found. In
+// the module's non-test code, type-checked, the calls of Put on a value whose
+// static type implements corestore.BlobStore, whatever its name (the field
+// Core.Blobs, a variable, a parameter, a wrapper), are exactly one: in
+// Core.UploadPack. Only packages that depend on corestore can hold such a
+// value. Tests may write the bucket directly to set up states.
func TestUploadPackIsTheOnlyUploader(t *testing.T) {
root := moduleRoot(t)
- // Where Blobs may be used as a value: the core is built and closed there.
- wiring := map[string]bool{
- "internal/graft/corestore/core.go": true,
- "internal/graft/corestore/open/open.go": true,
+
+ // The module's packages that depend on corestore, and corestore.
+ var targets []string
+ for _, p := range goList(t, root, "./...") {
+ if p.ImportPath == corestorePath || slices.Contains(p.Deps, corestorePath) {
+ targets = append(targets, p.ImportPath)
+ }
}
+ require.Contains(t, targets, corestorePath)
- var violations []string
+ // Their export data, and that of everything they import.
+ exports := map[string]string{}
+ listed := map[string]listedPackage{}
+ for _, p := range goList(t, root, append([]string{"-export", "-deps"}, targets...)...) {
+ exports[p.ImportPath] = p.Export
+ listed[p.ImportPath] = p
+ }
fset := token.NewFileSet()
- err := filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
- if err != nil {
- return err
- }
- rel, err := filepath.Rel(root, path)
- if err != nil {
- return err
- }
- if d.IsDir() {
- if rel != "." && (strings.HasPrefix(d.Name(), ".") || d.Name() == "_build" || d.Name() == "_support" || d.Name() == "testdata") {
- return filepath.SkipDir
- }
- return nil
- }
- if !strings.HasSuffix(path, ".go") || strings.HasSuffix(path, "_test.go") {
- return nil
- }
- src, err := os.ReadFile(path)
- if err != nil {
- return err
+ imp := importer.ForCompiler(fset, "gc", func(path string) (io.ReadCloser, error) {
+ file, ok := exports[path]
+ if !ok || file == "" {
+ return nil, fmt.Errorf("no export data for %s", path)
}
- file, err := parser.ParseFile(fset, path, src, parser.SkipObjectResolution)
- if err != nil {
- return err
+ return os.Open(file)
+ })
+
+ corestore, err := imp.Import(corestorePath)
+ require.NoError(t, err)
+ blobStore, ok := corestore.Scope().Lookup("BlobStore").Type().Underlying().(*types.Interface)
+ require.True(t, ok, "corestore.BlobStore is an interface")
+
+ type put struct{ file, function string }
+ var puts []put
+ for _, path := range targets {
+ p := listed[path]
+ var files []*ast.File
+ for _, name := range p.GoFiles {
+ f, err := parser.ParseFile(fset, filepath.Join(p.Dir, name), nil, parser.SkipObjectResolution)
+ require.NoError(t, err)
+ files = append(files, f)
}
- rel = filepath.ToSlash(rel)
- if !reachesCore(rel, file, src) {
- return nil
+ info := &types.Info{Selections: map[*ast.SelectorExpr]*types.Selection{}}
+ conf := types.Config{Importer: imp}
+ pkg, err := conf.Check(path, fset, files, info)
+ require.NoError(t, err, "type-check %s", path)
+ // Checked from source, corestore has its own BlobStore type (the
+ // importer's is that of the export data).
+ iface := blobStore
+ if path == corestorePath {
+ iface = pkg.Scope().Lookup("BlobStore").Type().Underlying().(*types.Interface)
}
- var stack []ast.Node
- ast.Inspect(file, func(n ast.Node) bool {
- if n == nil {
- stack = stack[:len(stack)-1]
- return true
- }
- stack = append(stack, n)
- sel, ok := n.(*ast.SelectorExpr)
- if !ok || sel.Sel.Name != "Blobs" {
- return true
- }
- pos := fset.Position(sel.Pos())
- // The parent of x.Blobs: a method selection (x.Blobs.M) or a use as
- // a value.
- if len(stack) >= 2 {
- if method, ok := stack[len(stack)-2].(*ast.SelectorExpr); ok && method.X == sel {
- if method.Sel.Name == "Put" && rel != "internal/graft/corestore/core.go" {
- violations = append(violations, pos.String()+": Blobs.Put outside Core.UploadPack")
+ for _, f := range files {
+ for _, decl := range f.Decls {
+ function := ""
+ if fn, ok := decl.(*ast.FuncDecl); ok {
+ function = fn.Name.Name
+ }
+ ast.Inspect(decl, func(n ast.Node) bool {
+ call, ok := n.(*ast.CallExpr)
+ if !ok {
+ return true
+ }
+ sel, ok := call.Fun.(*ast.SelectorExpr)
+ if !ok || sel.Sel.Name != "Put" {
+ return true
+ }
+ selection, ok := info.Selections[sel]
+ if !ok || selection.Kind() != types.MethodVal {
+ return true
+ }
+ recv := selection.Recv()
+ if types.Implements(recv, iface) || types.Implements(types.NewPointer(recv), iface) {
+ rel, err := filepath.Rel(root, fset.Position(call.Pos()).Filename)
+ require.NoError(t, err)
+ puts = append(puts, put{file: filepath.ToSlash(rel), function: function})
}
return true
- }
- }
- if !wiring[rel] {
- violations = append(violations, pos.String()+": Blobs used as a value outside the core's wiring")
+ })
}
- return true
- })
- return nil
- })
- require.NoError(t, err)
- require.Empty(t, violations, "every bucket write goes through Core.UploadPack (RULE 26, invariant U)")
-}
-
-// reachesCore reports whether the file at rel can reach a *corestore.Core.
-func reachesCore(rel string, file *ast.File, src []byte) bool {
- if strings.HasPrefix(rel, "internal/graft/") || strings.HasPrefix(rel, "internal/gitaly/service/storagecore/") {
- return true
- }
- for _, imp := range file.Imports {
- if strings.HasPrefix(strings.Trim(imp.Path.Value, `"`), "github.com/feder-cr/git-ai/internal/graft/corestore") {
- return true
}
}
- return strings.Contains(string(src), "GetGraftStorage")
+
+ require.Equal(t, []put{{file: "internal/graft/corestore/core.go", function: "UploadPack"}}, puts,
+ "every bucket write goes through Core.UploadPack (RULE 26, invariant U)")
}
// moduleRoot returns the directory of the module's go.mod.