Skip to content

Worktrunk is a CLI for Git worktree management, designed for parallel AI agent workflows

refactor(bench): canonicalize prune cold to probe-cold via bench_wt (#3437)

Maximilian Roosc3107672168a
diff --git a/benches/CLAUDE.md b/benches/CLAUDE.md
index a58d864b1..1caa32bd5 100644
--- a/benches/CLAUDE.md
+++ b/benches/CLAUDE.md
@@ -90,10 +90,14 @@ so bench iterations read from prior iterations unless invalidated.
**Rule:** if a benchmark runs a `wt` subcommand that populates these caches, every
iteration must start cold — otherwise iter 1 measures the real cost and iter 2+ measure
a cache hit. Run the iteration through `wt_perf::bench_wt`, the one home of the
-warm/cold strategy (plain `iter` warm; invalidate-per-iteration cold — its doc
-comment carries the `BatchSize::PerIteration` rationale). A cold benchmark whose
-setup is not plain invalidation (e.g. `remove_e2e` recreating the removed
-worktree) uses `iter_batched` with `BatchSize::PerIteration` directly.
+warm/cold strategy (plain `iter` warm; invalidate-per-iteration full- or
+probe-cold, picked by `CacheState` — its doc comment carries the
+`BatchSize::PerIteration` rationale). A benchmark whose setup is not plain
+invalidation (e.g. `remove_e2e` recreating the removed worktree, prune's
+`live` re-creating its candidates) uses `iter_batched` with
+`BatchSize::PerIteration` directly. Timed iterations assert only the child's
+exit status; anything stronger (candidate counts, fixture shape) runs once
+outside the timed loop (see `verify_candidates` in `benches/prune.rs`).
`invalidate_caches_auto` clears:
@@ -117,15 +121,18 @@ setup).
as empty, so `git status` reports every tracked file as a staged deletion — a
*different repo state*, which flips any clean-worktree gate a benchmarked
command exercises (e.g. `wt step prune`'s removability check would silently
-drop every worktree candidate). A benchmark exercising such a gate must pair
-`invalidate_caches_auto` with `wt_perf::restore_worktree_indexes`, which
-`git reset -q`s every worktree back to a clean `git status` while leaving the
-integration probes cold. It's a separate call, not folded into
-`invalidate_caches_auto`, because `git reset --mixed` discards
-staged-but-uncommitted index state that some fixtures plant on purpose (and
-that a real repo targeted by `wt-perf invalidate` / `timeline --cold` may hold
-as genuine work in progress) — pair it only with fixtures whose dirt is
-untracked files.
+drop every worktree candidate). A benchmark exercising such a gate must
+invalidate with `wt_perf::invalidate_probe_caches` instead, which clears only
+`.git/wt/cache/`: the integration probes re-run while git's own state stays
+warm — indexes with stat data, commit graph, `worktrunk.default-branch` —
+exactly the state a real repo is in right after a fetch. The prune benches use
+it (via `bench_wt`'s `ProbeCold` state) for their `dry_run_probe_cold`
+variants; a one-time post-setup dry-run (`verify_candidates`) pins the
+candidate count, so a gate-flipping invalidation can't silently shrink the
+scanned workload. (`ensure_prune_real_repo` heals a
+fixture whose indexes were deleted by a stray `wt-perf invalidate` /
+`timeline --cold`, rebuilding them via `git reset -q` — safe there because the
+fixture's only dirt is untracked files.)
**Which commands populate `.git/wt/cache/`:**
@@ -133,14 +140,16 @@ untracked files.
|---------|------------|-------|
| `wt list` | Yes | Post-skeleton tasks. Exits early under `WORKTRUNK_SKELETON_ONLY=1` / `WORKTRUNK_FIRST_OUTPUT=1` — those skip the writing phase. |
| `wt remove` | Yes | `prepare_worktree_removal` → `compute_integration_lazy` writes `is-ancestor` / `has-added-changes` / `merge-add-probe` whenever `BranchDeletionMode` is not `ForceDelete` (CLI `--force` is `force_worktree`, not `--force-delete`). |
-| `wt step prune` | Yes | Every scanned worktree/branch runs `integration_reason` → the same probe writes as `wt remove`. First scan after new commits is cold; re-runs are warm (`prune_e2e/dry_run_cold` vs `dry_run_warm`). |
+| `wt step prune` | Yes | Every scanned worktree/branch runs `integration_reason` → the same probe writes as `wt remove`. First scan after new commits is cold; re-runs are warm (`prune_e2e/dry_run_probe_cold` vs `dry_run_warm`). |
| `wt switch <branch>` | No | No sha_cache writers on the direct-switch path. |
| `wt switch` (picker) | Yes | Preview pre-compute writes `picker-preview/{log,branch-diff,upstream-diff}-…` entries. Exercised under `WORKTRUNK_PREVIEW_BENCH=1` / `WORKTRUNK_PICKER_DRY_RUN=1`. |
| `wt` (completion via `COMPLETE=$SHELL`) | No | Only `for-each-ref` + worktree list. |
Default-branch cache contribution is ~17ms per iteration on a typical-8 synthetic repo
(measured: 166ms with default-branch cached → 183ms fully cold). Small enough that
-always clearing it is simpler than introducing a "warm default-branch" bench mode.
+clearing it as part of the full invalidation is simpler than introducing a "warm
+default-branch" bench mode. (`invalidate_probe_caches` leaves it warm, like
+everything else outside `.git/wt/cache/`.)
**Bench fixtures don't exercise the wire path.** `setup_fake_remote` writes
`refs/remotes/origin/HEAD` directly into every repo, so a cold-cache iteration
@@ -171,7 +180,7 @@ Record them in two layers:
numbers on an M-series Mac (`prune-4-8` fixture: 4 squash-merged worktrees +
4 squash-merged branches as candidates, 8 two-sided-diverged worktrees + 8
diverged branches as backdrop, 200 commits, 100 files; `prune_real_repo` runs
-a warm dry-run on the `prune-real` fixture — a rust-lang/rust clone with 12
+warm and probe-cold dry-runs on the `prune-real` fixture — a rust-lang/rust clone with 12
squash-merged candidate pairs + 24 diverged worktrees and 24 diverged
branches, i.e. 36 linked worktrees, cached and self-repairing in
`target/bench-repos/rust-prune-12-24/`. That group is opt-in —
@@ -180,21 +189,23 @@ because its ~15 GiB fixture must never build on a hosted CI runner):
| Variant | Expected | What it measures |
|---------|----------|------------------|
-| `prune_e2e/dry_run_cold` | ~160 ms | full parallel scan, integration probes uncached |
+| `prune_e2e/dry_run_probe_cold` | ~160 ms | full parallel scan, probes re-run (`.git/wt/cache/` cleared; git's own caches stay warm — the "first prune after fetching main" shape) |
| `prune_e2e/dry_run_warm` | ~90 ms | steady-state re-scan, probes hit sha_cache |
-| `prune_e2e/live` | ~620 ms | cold scan + serial removal of the 8 candidates (~60 ms each, under the scan write lock) |
-| `prune_real_repo/dry_run_warm` | ~0.3–0.8 s | steady-state scan of 72 items (36 worktrees + 36 branches) at 331k-commit scale |
+| `prune_e2e/live` | ~620 ms | probe-cold scan + serial removal of the 8 candidates (~60 ms each, under the scan write lock) |
+| `prune_real_repo/dry_run_warm` | ~0.25–0.8 s | steady-state scan of 72 items (36 worktrees + 36 branches) at 331k-commit scale |
+| `prune_real_repo/dry_run_probe_cold` | ~0.6–1 s | the same 72-item scan with probes re-running at real cost (statuses stay stat-warm) |
| `first_output/remove` | ~86 ms | single-target validation up to first output (`benches/time_to_first_output.rs`) |
-Cold and live at rust scale are **one-shot timelines, not criterion groups**
-(a cold criterion iteration costs ~1 min in re-hashing statuses alone; a live
-one consumes the candidates). Expected one-shots on the `prune-real` fixture:
+Full-cold and live at rust scale are **one-shot timelines, not criterion
+groups** (a full-cold criterion iteration costs ~1 min in re-hashing statuses
+alone; a live one consumes the candidates). Expected one-shots on the
+`prune-real` fixture:
-- **cold dry-run ~5.5 s wall** (~46 s CPU over 472 subprocesses absorbed by
- the rayon pool) — dominated by stat-cold `git status` at ~4.5 s per fresh
- worktree; the probes are `merge-base --is-ancestor` ~40 ms and `merge-tree
- --write-tree` ~130 ms (vs 4–25 ms synthetic, where shallow history walks
- bottom out at subprocess-spawn cost)
+- **full-cold dry-run ~5.5 s wall** (~46 s CPU over 472 subprocesses absorbed
+ by the rayon pool) — the fresh-fixture shape, dominated by stat-cold
+ `git status` at ~4.5 s per fresh worktree; the probes are `merge-base
+ --is-ancestor` ~40 ms and `merge-tree --write-tree` ~130 ms (vs 4–25 ms
+ synthetic, where shallow history walks bottom out at subprocess-spawn cost)
- **live ~12 s wall** — all 24 removals serialize under the scan write lock
inside the `prune-scan` window: each of the 12 worktree candidates takes
~0.5–1.7 s (pre-remove re-checks plus drain waits), branch-only candidates
@@ -210,10 +221,13 @@ ambient machine load (sibling builds, Spotlight): treat them as shape, not
thresholds, and read criterion "regressed" verdicts on this bench against
`uptime` before believing them.
-Cold prune benches pair `invalidate_caches_auto` with
-`restore_worktree_indexes` (see "Deleting a worktree's index isn't a cold
-cache" under Cache Handling); the dry-run variants assert the listed
-candidate count so that degradation can't come back unnoticed.
+The probe-cold prune benches run through `bench_wt` with
+`CacheState::ProbeCold`, never full invalidation (see "Deleting a worktree's
+index isn't a cold cache" under Cache Handling). Fixture correctness is
+checked once, outside the timed loops: a post-setup dry-run must list the
+expected candidate count, and a live run that removed nothing trips the next
+iteration's candidate re-creation (branch-name collision) instead of being
+silently timed.
**Phase attribution** — `wt-perf timeline` plus the removal spans. Prune emits
`prune-gather` (worktree+branch enumeration), `prune-scan` (the whole parallel
diff --git a/benches/alias.rs b/benches/alias.rs
index 993e4e971..a88fa8dda 100644
--- a/benches/alias.rs
+++ b/benches/alias.rs
@@ -33,7 +33,7 @@ use criterion::{BenchmarkId, Criterion, criterion_group, criterion_main};
use std::path::Path;
use std::process::Command;
use worktrunk::testing::isolate_subprocess_env;
-use wt_perf::{RepoConfig, bench_wt, create_repo, run_and_check};
+use wt_perf::{CacheState, RepoConfig, bench_wt, create_repo, run_and_check};
/// Alias body is a shell builtin so the wall-clock is dominated by the
/// parent's dispatch — not by running a real subcommand.
@@ -115,7 +115,12 @@ fn bench_dispatch(c: &mut Criterion) {
// Cold matters here: `build_hook_context` resolves the
// default branch, which writes `worktrunk.default-branch`
// and would otherwise be a cache hit on iters 2-N.
- bench_wt(b, &repo_path, cold, || {
+ let cache = if cold {
+ CacheState::Cold
+ } else {
+ CacheState::Warm
+ };
+ bench_wt(b, &repo_path, cache, || {
wt_cmd(binary, &repo_path, user_config, &[alias_name])
});
});
diff --git a/benches/list.rs b/benches/list.rs
index 2b446fbae..b786e7b95 100644
--- a/benches/list.rs
+++ b/benches/list.rs
@@ -32,7 +32,7 @@ use std::path::{Path, PathBuf};
use std::process::Command;
use worktrunk::testing::isolate_subprocess_env;
use wt_perf::{
- RepoConfig, add_history_spread_branches, add_worktrees, bench_wt, clone_rust_repo,
+ CacheState, RepoConfig, add_history_spread_branches, add_worktrees, bench_wt, clone_rust_repo,
create_mixed_repo, create_repo, run_git, setup_fake_remote,
};
@@ -75,7 +75,12 @@ fn run_benchmark(
args: &[&str],
env: Option<(&str, &str)>,
) {
- bench_wt(b, repo_path, cold_cache, || {
+ let cache = if cold_cache {
+ CacheState::Cold
+ } else {
+ CacheState::Warm
+ };
+ bench_wt(b, repo_path, cache, || {
let mut cmd = Command::new(binary);
cmd.args(args).current_dir(repo_path);
isolate_subprocess_env(&mut cmd, None);
@@ -173,7 +178,12 @@ fn bench_real_repo(c: &mut Criterion) {
add_worktrees(&config, &workspace_main);
run_git(&workspace_main, &["status"]);
- bench_wt(b, &workspace_main, cold, || {
+ let cache = if cold {
+ CacheState::Cold
+ } else {
+ CacheState::Warm
+ };
+ bench_wt(b, &workspace_main, cache, || {
let mut cmd = Command::new(binary);
cmd.arg("list").current_dir(&workspace_main);
isolate_subprocess_env(&mut cmd, None);
@@ -274,7 +284,7 @@ fn bench_real_repo_many_branches(c: &mut Criterion) {
// Baseline: all branches
group.bench_function("warm", |b| {
let (_temp, workspace_main) = setup_workspace();
- bench_wt(b, &workspace_main, false, || {
+ bench_wt(b, &workspace_main, CacheState::Warm, || {
let mut cmd = Command::new(binary);
cmd.args(["list", "--branches"])
.current_dir(&workspace_main);
@@ -286,7 +296,7 @@ fn bench_real_repo_many_branches(c: &mut Criterion) {
// Worktrees only: no branch enumeration, skips expensive %(ahead-behind) batch
group.bench_function("warm_worktrees_only", |b| {
let (_temp, workspace_main) = setup_workspace();
- bench_wt(b, &workspace_main, false, || {
+ bench_wt(b, &workspace_main, CacheState::Warm, || {
let mut cmd = Command::new(binary);
cmd.arg("list").current_dir(&workspace_main); // no --branches
isolate_subprocess_env(&mut cmd, None);
diff --git a/benches/picker_preview.rs b/benches/picker_preview.rs
index f0620334f..1e2370121 100644
--- a/benches/picker_preview.rs
+++ b/benches/picker_preview.rs
@@ -51,7 +51,7 @@ use std::process::Command;
#[cfg(unix)]
use worktrunk::testing::isolate_subprocess_env;
#[cfg(unix)]
-use wt_perf::{RepoConfig, bench_wt, create_repo, setup_fake_remote};
+use wt_perf::{CacheState, RepoConfig, bench_wt, create_repo, setup_fake_remote};
#[cfg(unix)]
fn bench_picker_preview(c: &mut Criterion) {
@@ -92,7 +92,12 @@ fn bench_picker_preview(c: &mut Criterion) {
// `.git/wt/cache/picker-preview/` (Log / BranchDiff /
// UpstreamDiff entries), so without invalidation iter 1
// measures real cost and iter 2+ measure cache hits.
- bench_wt(b, &repo_path, *cold, make_cmd);
+ let cache = if *cold {
+ CacheState::Cold
+ } else {
+ CacheState::Warm
+ };
+ bench_wt(b, &repo_path, cache, make_cmd);
},
);
}
diff --git a/benches/prune.rs b/benches/prune.rs
index 9c217cf8d..c91c4eb94 100644
--- a/benches/prune.rs
+++ b/benches/prune.rs
@@ -17,18 +17,21 @@
// are scanned every run but never removed.
//
// Benchmark variants:
-// - prune_e2e/dry_run_cold — full scan, sha_cache invalidated per iteration
+// - prune_e2e/dry_run_probe_cold — full scan, `.git/wt/cache/` cleared per
+// iteration (probe-cold: the "first prune after
+// new commits on main" shape; git's own caches
+// stay warm, as after a real fetch)
// - prune_e2e/dry_run_warm — full scan, caches warm (steady-state re-run)
// - prune_e2e/live — scan + removal of the squash-merged candidates,
// which are re-created before every iteration
-// - prune_real_repo/ — warm dry-run on a rust-lang/rust clone with
-// dry_run_warm both populations at "dozens of worktrees" scale
-// (cached ~15 GiB fixture; first-ever run clones,
-// minutes). Opt-in via `--features
-// real-repo-benches` so the daily CI benchmarks
-// job never builds it; cold and live at this
-// scale are one-shot timelines, not criterion
-// groups (see below)
+// - prune_real_repo/ — the same probe-cold and warm dry-runs on a
+// dry_run_probe_cold, rust-lang/rust clone with both populations at
+// dry_run_warm "dozens of worktrees" scale (cached ~15 GiB
+// fixture; first-ever run clones, minutes).
+// Opt-in via `--features real-repo-benches` so
+// the daily CI benchmarks job never builds it;
+// full-cold and live at this scale are one-shot
+// timelines, not criterion groups (see below)
//
// Run examples:
// cargo bench --bench prune # Synthetic variants
@@ -47,8 +50,8 @@ use std::process::Command;
use criterion::{Criterion, criterion_group, criterion_main};
use worktrunk::testing::isolate_subprocess_env;
use wt_perf::{
- PRUNE_REAL_MERGED, PRUNE_REAL_UNMERGED, add_squash_merged, create_prune_repo_at,
- ensure_prune_real_repo, invalidate_caches_auto, restore_worktree_indexes,
+ CacheState, PRUNE_REAL_MERGED, PRUNE_REAL_UNMERGED, add_squash_merged, bench_wt,
+ create_prune_repo_at, ensure_prune_real_repo, run_and_check,
};
/// Squash-merged candidates per population (worktrees and orphan branches) —
@@ -57,8 +60,8 @@ const MERGED: usize = 4;
/// Unmerged worktrees and orphan branches — scanned every run, never removed.
const UNMERGED: usize = 8;
-/// One invocation shape for both dry-run variants, so cold and warm can never
-/// silently benchmark different commands.
+/// One invocation shape for every dry-run variant and the fixture check, so
+/// the verified command and the timed commands can never silently diverge.
const DRY_RUN_ARGS: &[&str] = &[
"step",
"prune",
@@ -70,33 +73,35 @@ const DRY_RUN_ARGS: &[&str] = &[
];
const LIVE_ARGS: &[&str] = &["step", "prune", "--min-age", "0s", "--format", "json"];
-/// Run `wt <args>` in `repo`, panicking with stderr on failure; returns stdout.
-fn run(repo: &Path, args: &[&str], label: &str) -> Vec<u8> {
+/// Build the `wt <args>` command for `repo`.
+fn wt_cmd(repo: &Path, args: &[&str]) -> Command {
let mut cmd = Command::new(Path::new(env!("CARGO_BIN_EXE_wt")));
cmd.args(args).current_dir(repo);
isolate_subprocess_env(&mut cmd, None);
- let output = cmd.output().unwrap();
+ cmd
+}
+
+/// One-time fixture check, run once after setup and never inside a timed
+/// loop: a dry-run scan must list exactly `expected` candidates. Catches a
+/// fixture whose candidates prune doesn't detect, and detection false
+/// positives against the unmerged backdrop (this once caught an invalidation
+/// deleting worktree indexes and silently flipping the removability gate).
+/// The timed iterations themselves assert only exit status (`bench_wt`); a
+/// live run that removes nothing surfaces on the next iteration's
+/// re-creation instead — branch names don't carry the round, so
+/// `add_squash_merged` fails loudly on the collision.
+fn verify_candidates(repo: &Path, expected: usize) {
+ let output = wt_cmd(repo, DRY_RUN_ARGS).output().unwrap();
assert!(
output.status.success(),
- "{label} failed:\nstderr: {}",
+ "fixture-check dry-run failed:\nstderr: {}",
String::from_utf8_lossy(&output.stderr)
);
- output.stdout
-}
-
-/// Guard against silently measuring a degraded scan: every run must list
-/// exactly `expected` candidates as planned (dry run) or removed (live). This
-/// is what caught the invalidated-index state flipping the removability gate,
-/// and on the live variant it fails the offending iteration directly instead
-/// of the next iteration's re-creation tripping over a leftover branch. An
-/// exact count also catches integration-detection false positives against the
-/// unmerged backdrop.
-fn assert_candidates(stdout: &[u8], expected: usize, label: &str) {
- let items: serde_json::Value = serde_json::from_slice(stdout).unwrap();
+ let items: serde_json::Value = serde_json::from_slice(&output.stdout).unwrap();
let found = items.as_array().unwrap().len();
assert_eq!(
found, expected,
- "{label}: expected {expected} candidates, run listed {found}"
+ "fixture check: expected {expected} candidates, dry-run listed {found}"
);
}
@@ -108,33 +113,21 @@ fn bench_prune_e2e(c: &mut Criterion) {
let temp_dry = tempfile::tempdir().unwrap();
let repo_dry = temp_dry.path().join("repo");
create_prune_repo_at(MERGED, UNMERGED, &repo_dry);
-
- // Cold: every iteration re-pays the integration probes that sha_cache
- // would otherwise absorb — the "first prune after fetching main" cost.
- // `BatchSize::PerIteration` so the invalidation runs before every
- // measured iter (under `SmallInput`, only iter 1 per batch is cold).
- // The index restore is load-bearing: without it the invalidated indexes
- // make every worktree read dirty and prune's removability gate silently
- // drops the worktree candidates (see `restore_worktree_indexes`).
- group.bench_function("dry_run_cold", |b| {
- b.iter_batched(
- || {
- invalidate_caches_auto(&repo_dry);
- restore_worktree_indexes(&repo_dry);
- },
- |_| {
- let stdout = run(&repo_dry, DRY_RUN_ARGS, "dry_run_cold");
- assert_candidates(&stdout, MERGED * 2, "dry_run_cold");
- },
- criterion::BatchSize::PerIteration,
- );
+ verify_candidates(&repo_dry, MERGED * 2);
+
+ // Probe-cold: every iteration re-pays the integration probes that
+ // sha_cache would otherwise absorb — the "first prune after fetching
+ // main" cost, with git's own caches staying warm as after a real fetch.
+ group.bench_function("dry_run_probe_cold", |b| {
+ bench_wt(b, &repo_dry, CacheState::ProbeCold, || {
+ wt_cmd(&repo_dry, DRY_RUN_ARGS)
+ });
});
// Warm: the steady-state re-scan where every probe hits sha_cache.
group.bench_function("dry_run_warm", |b| {
- b.iter(|| {
- let stdout = run(&repo_dry, DRY_RUN_ARGS, "dry_run_warm");
- assert_candidates(&stdout, MERGED * 2, "dry_run_warm");
+ bench_wt(b, &repo_dry, CacheState::Warm, || {
+ wt_cmd(&repo_dry, DRY_RUN_ARGS)
});
});
@@ -155,18 +148,18 @@ fn bench_prune_e2e(c: &mut Criterion) {
let temp_live = tempfile::tempdir().unwrap();
let repo_live = temp_live.path().join("repo");
create_prune_repo_at(0, UNMERGED, &repo_live);
+ verify_candidates(&repo_live, 0);
let round = Cell::new(0usize);
+ // Setup here is candidate re-creation, not invalidation, so this arm
+ // can't go through `bench_wt` — the same carve-out as `remove_e2e`.
group.bench_function("live", |b| {
b.iter_batched(
|| {
add_squash_merged(&repo_live, MERGED, round.get());
round.set(round.get() + 1);
},
- |_| {
- let stdout = run(&repo_live, LIVE_ARGS, "live");
- assert_candidates(&stdout, MERGED * 2, "live");
- },
+ |_| run_and_check(&mut wt_cmd(&repo_live, LIVE_ARGS)),
criterion::BatchSize::PerIteration,
);
});
@@ -184,16 +177,21 @@ fn bench_prune_e2e(c: &mut Criterion) {
/// rust-lang/rust from the network (minutes), then builds ~36 worktrees at
/// ~3 s each; both are cached in `target/bench-repos/` across runs.
///
-/// Warm dry-run only, for two reasons. A live iteration would consume the
-/// candidates and pay a minutes-long re-creation per sample. And a cold
-/// iteration costs ~1 min: `invalidate_caches_auto` deletes every worktree
-/// index and the `git reset` restore rebuilds them without stat data, so all
-/// 36 statuses re-hash their trees — criterion's 10-sample minimum turns that
-/// into a ~10-minute benchmark whose probe-cost signal the synthetic
-/// `dry_run_cold` already tracks. Measure cold and live at this scale as
-/// one-shots instead (`wt-perf setup prune-real`, then `wt-perf timeline --
-/// -C <repo> step prune --min-age 0s`); the next `ensure_prune_real_repo`
-/// call repairs the consumed candidates.
+/// Dry-runs only, in two flavors: warm (steady-state re-scan) and probe-cold
+/// (`.git/wt/cache/` cleared per iteration — the "first prune after fetching
+/// main" shape, where probes re-run at real cost but statuses stay
+/// stat-warm). Probe-cold is criterion-feasible precisely because the
+/// invalidation leaves the indexes alone; a *full*-cold iteration deletes
+/// indexes and must rebuild them via `git reset` (a missing index reads as
+/// all-tracked-files-deleted), and the rebuilt indexes carry no stat data,
+/// so all 36 statuses re-hash ~60k files — ~1 min per iteration, turning
+/// criterion's 10-sample minimum into a ~10-minute benchmark of a
+/// mostly-artificial state. A live iteration would consume
+/// the candidates and pay a minutes-long re-creation per sample. Measure
+/// full-cold and live at this scale as one-shots instead (`wt-perf setup
+/// prune-real`, then `wt-perf timeline -- -C <repo> step prune --min-age
+/// 0s`); the next `ensure_prune_real_repo` call repairs the consumed
+/// candidates.
fn bench_prune_real_repo(c: &mut Criterion) {
// Opt-in only (`--features real-repo-benches`): the fixture is ~15 GiB —
// bigger than a hosted CI runner's disk and the actions cache cap — so
@@ -214,10 +212,17 @@ fn bench_prune_real_repo(c: &mut Criterion) {
// prune-real` and wiping the fixture it is mid-build. Repeat
// invocations re-validate the cached fixture in a few git commands.
let repo = ensure_prune_real_repo(PRUNE_REAL_MERGED, PRUNE_REAL_UNMERGED);
- let expected = PRUNE_REAL_MERGED * 2;
- b.iter(|| {
- let stdout = run(&repo, DRY_RUN_ARGS, "real dry_run_warm");
- assert_candidates(&stdout, expected, "real dry_run_warm");
+ verify_candidates(&repo, PRUNE_REAL_MERGED * 2);
+ bench_wt(b, &repo, CacheState::Warm, || wt_cmd(&repo, DRY_RUN_ARGS));
+ });
+
+ group.bench_function("dry_run_probe_cold", |b| {
+ // Built inside the closure for the same filter-matching reason as
+ // dry_run_warm above; a second call re-validates cheaply.
+ let repo = ensure_prune_real_repo(PRUNE_REAL_MERGED, PRUNE_REAL_UNMERGED);
+ verify_candidates(&repo, PRUNE_REAL_MERGED * 2);
+ bench_wt(b, &repo, CacheState::ProbeCold, || {
+ wt_cmd(&repo, DRY_RUN_ARGS)
});
});
diff --git a/benches/time_to_first_output.rs b/benches/time_to_first_output.rs
index e0e1036cc..4c55464aa 100644
--- a/benches/time_to_first_output.rs
+++ b/benches/time_to_first_output.rs
@@ -17,7 +17,7 @@ use criterion::{Criterion, criterion_group, criterion_main};
use std::path::Path;
use std::process::Command;
use worktrunk::testing::isolate_subprocess_env;
-use wt_perf::{RepoConfig, bench_wt, create_repo, setup_fake_remote};
+use wt_perf::{CacheState, RepoConfig, bench_wt, create_repo, setup_fake_remote};
fn bench_first_output(c: &mut Criterion) {
let mut group = c.benchmark_group("first_output");
@@ -44,14 +44,14 @@ fn bench_first_output(c: &mut Criterion) {
// first invocation. Without per-iteration invalidation the reported
// timing would be warm cache, not the first-invocation TTFO a user sees.
group.bench_function("remove", |b| {
- bench_wt(b, &repo_path, true, || {
+ bench_wt(b, &repo_path, CacheState::Cold, || {
make_cmd(&["remove", "--yes", "--no-hooks", "--force", "feature-wt-1"])
});
});
// switch: exits after execute_switch, before mismatch computation and output
group.bench_function("switch", |b| {
- bench_wt(b, &repo_path, false, || {
+ bench_wt(b, &repo_path, CacheState::Warm, || {
make_cmd(&["switch", "--yes", "--no-hooks", "feature-wt-1"])
});
});
diff --git a/tests/helpers/wt-perf/src/lib.rs b/tests/helpers/wt-perf/src/lib.rs
index 1a484ef63..215810fe1 100644
--- a/tests/helpers/wt-perf/src/lib.rs
+++ b/tests/helpers/wt-perf/src/lib.rs
@@ -163,11 +163,25 @@ fn git_command() -> Command {
cmd
}
+/// Cache state a [`bench_wt`] iteration starts from.
+#[derive(Clone, Copy)]
+pub enum CacheState {
+ /// Persistent caches stay hot across iterations (steady-state re-run).
+ Warm,
+ /// [`invalidate_caches_auto`] per iteration: git's own caches (indexes,
+ /// commit graph) plus worktrunk's — a first run against fresh state.
+ Cold,
+ /// [`invalidate_probe_caches`] per iteration: only `.git/wt/cache/` —
+ /// the first scan after new commits land, git's state staying warm.
+ ProbeCold,
+}
+
/// Run a `wt` benchmark iteration function under criterion, warm or cold.
///
/// The one place the warm/cold iteration strategy lives: warm uses plain
-/// `Bencher::iter` (persistent caches stay hot across iterations); cold
-/// invalidates the repo's caches immediately before every measured iteration.
+/// `Bencher::iter` (persistent caches stay hot across iterations); the cold
+/// states invalidate the repo's caches immediately before every measured
+/// iteration — see [`CacheState`] for what each clears.
///
/// Cold uses `iter_batched` with `BatchSize::PerIteration`, not `SmallInput`:
/// under `SmallInput`, criterion calls the setup once for an entire batch up
@@ -181,23 +195,30 @@ fn git_command() -> Command {
/// 2.4ms → 0.65ms) and medians rose to their true cold cost.
///
/// `make_cmd` builds a fresh command per iteration; the child's exit status is
-/// asserted so a benchmark can't silently time a failing command.
+/// asserted so a benchmark can't silently time a failing command. That status
+/// check is the only per-iteration validation — anything stronger (fixture
+/// shape, candidate counts) belongs in a one-time check outside the timed
+/// loop, not on the measured path.
pub fn bench_wt(
b: &mut criterion::Bencher,
repo_path: &Path,
- cold: bool,
+ cache: CacheState,
mut make_cmd: impl FnMut() -> Command,
) {
let mut run = move || run_and_check(&mut make_cmd());
- if cold {
- b.iter_batched(
- || invalidate_caches_auto(repo_path),
- |_| run(),
- criterion::BatchSize::PerIteration,
- );
- } else {
- b.iter(run);
- }
+ let invalidate: fn(&Path) = match cache {
+ CacheState::Warm => {
+ b.iter(run);
+ return;
+ }
+ CacheState::Cold => invalidate_caches_auto,
+ CacheState::ProbeCold => invalidate_probe_caches,
+ };
+ b.iter_batched(
+ || invalidate(repo_path),
+ |_| run(),
+ criterion::BatchSize::PerIteration,
+ );
}
/// Spawn the command, wait, and panic with its stderr if it failed.
@@ -496,7 +517,7 @@ pub fn invalidate_caches_auto(repo_path: &Path) {
// simulate by deleting it.
// All worktrunk persistent caches: every kind dir under wt/cache/.
- let _ = std::fs::remove_dir_all(git_dir.join("wt/cache"));
+ invalidate_probe_caches(repo_path);
// Worktrunk's default-branch cache lives in git config; we have no
// safe way to edit that file ourselves (escaping rules), so shell
@@ -519,22 +540,34 @@ pub fn invalidate_caches_auto(repo_path: &Path) {
}
}
-/// Rebuild every worktree's index after [`invalidate_caches_auto`].
+/// Invalidate only worktrunk's persistent probe caches (`.git/wt/cache/`).
///
-/// Deleting an index doesn't model a cold cache — git treats a missing index
-/// as empty, so `git status` reports every tracked file as a staged deletion.
-/// That is a *different repo state*, and it flips any clean-worktree gate:
-/// `wt step prune`'s removability check drops all worktree candidates against
-/// such a worktree. Benches that exercise those gates call this after
-/// invalidation; `git reset -q` rewrites the index from HEAD, leaving the
-/// integration probes cold but the working trees reading clean again.
+/// Models the recurring cold scan: after new commits land on the default
+/// branch, every sha_cache entry keyed on the old tips misses, while git's
+/// own state stays warm — indexes keep their stat data, and the commit graph
+/// and `worktrunk.default-branch` config entry survive a fetch. This is the
+/// "first `wt step prune` after fetching main" shape, and it is safe against
+/// clean-worktree gates: deleting an *index* (as [`invalidate_caches_auto`]
+/// does) doesn't model a cold cache — git treats a missing index as empty, so
+/// `git status` reports every tracked file as a staged deletion, a different
+/// repo state that flips removability checks and silently drops worktree
+/// candidates from a prune scan.
+pub fn invalidate_probe_caches(repo_path: &Path) {
+ let Some(git_dir) = resolve_git_common_dir(repo_path) else {
+ return;
+ };
+ let _ = std::fs::remove_dir_all(git_dir.join("wt/cache"));
+}
+
+/// Rebuild every worktree's index via `git reset -q`.
///
-/// This is deliberately NOT folded into `invalidate_caches_auto`: `git reset
-/// --mixed` discards staged-but-uncommitted index state, which the mixed
-/// fixture plants on purpose (worktree state 2) and which a real repo — the
-/// `wt-perf invalidate` / `timeline --cold` targets — may hold as the user's
-/// work in progress. Only pair it with fixtures whose dirt is untracked files.
-pub fn restore_worktree_indexes(repo_path: &Path) {
+/// Used by [`ensure_prune_real_repo`] to heal a fixture whose indexes were
+/// deleted by a stray `wt-perf invalidate` / `timeline --cold` (a missing
+/// index reads as all-tracked-files-deleted and flips prune's clean-worktree
+/// gate — see [`invalidate_probe_caches`]). `git reset --mixed` discards
+/// staged-but-uncommitted index state, so this is safe only on fixtures
+/// whose dirt is untracked files.
+fn restore_worktree_indexes(repo_path: &Path) {
let output = git_command()
.args(["worktree", "list", "--porcelain"])
.current_dir(repo_path)
@@ -544,8 +577,7 @@ pub fn restore_worktree_indexes(repo_path: &Path) {
let stdout = String::from_utf8_lossy(&output.stdout);
// Parallel: each reset rebuilds one worktree's index from its own HEAD,
// fully independent of the others. On the rust-scale fixture a rebuild
- // is ~2.5 s per worktree — serially that would dominate every cold-bench
- // iteration's setup.
+ // is ~2.5 s per worktree — serially that would dominate the repair.
std::thread::scope(|s| {
for line in stdout.lines() {
if let Some(path) = line.strip_prefix("worktree ") {
@@ -1238,7 +1270,7 @@ fn next_squash_round(repo: &Path) -> usize {
/// - `Broken` (interrupted prune or build, corruption) → wiped and rebuilt.
///
/// Deleted worktree indexes (a stray `wt-perf invalidate` / `timeline --cold`
-/// against the fixture) are healed first with [`restore_worktree_indexes`] —
+/// against the fixture) are healed first with `restore_worktree_indexes` —
/// without an index, `git status` reports every tracked file as a staged
/// deletion and prune's clean-worktree gate silently drops the worktree
/// candidates. Safe here because the fixture's only dirt is untracked files.