Skip to content

Worktrunk is a CLI for Git worktree management, designed for parallel AI agent workflows

Refactor benchmark naming for clarity and consistency

Maximilian Roosdb5cc8895fea
diff --git a/benches/CLAUDE.md b/benches/CLAUDE.md
index e171cca4d..764096d7c 100644
--- a/benches/CLAUDE.md
+++ b/benches/CLAUDE.md
@@ -1,123 +1,38 @@
# Benchmark Guidelines
+See `list.rs` header for the authoritative list of benchmark groups and run examples.
+
## Quick Start
```console
-# Run fast synthetic benchmarks (skip slow ones)
+# Fast iteration (skip slow benchmarks)
cargo bench --bench list -- --skip cold --skip real
-# Run all benchmarks (includes real repo, takes ~1 hour)
-cargo bench --bench list
-
-# Run specific benchmark
-cargo bench --bench list bench_list_by_worktree_count
-cargo bench --bench list bench_list_cold_cache
-cargo bench --bench list bench_list_real_repo
+# Run specific group
+cargo bench --bench list many_branches
-# Run filtered benchmarks (expensive setup only runs for matching benchmarks)
-cargo bench --bench list many_branches # Only many_branches, no rust repo clone
-cargo bench --bench list real_repo # Only real_repo benchmarks
+# All benchmarks (~1 hour)
+cargo bench --bench list
```
-## Benchmark Types
-
-### 1. Synthetic Benchmarks (Fast, ~5-10 minutes)
-- `bench_time_to_skeleton` - Time until skeleton appears (progressive mode, warm caches)
-- `bench_time_to_skeleton_cold` - Same as above but with packed-refs invalidated
-- `bench_time_to_complete` - Full `wt list` execution (all data filled in), warm caches
-- `bench_time_to_complete_cold` - Same as above but with packed-refs invalidated
-- `bench_list_by_worktree_count` - Scaling with worktree count (1-8), warm caches
-- `bench_list_by_repo_profile` - Scaling with repo size (minimal/typical/large), warm caches
-- `bench_list_sequential_vs_parallel` - Sequential vs parallel comparison
-- `bench_list_cold_cache` - Cold cache performance (all git caches invalidated)
-
-### 2. Many Branches Benchmarks (Fast, ~3 minutes)
-- `bench_list_many_branches` - Tests 25/50/100 branches with unique commits
-- `bench_list_many_branches_cold` - Same with packed-refs invalidated
-
-### 3. Real Repository Benchmarks (Slow, ~30-60 minutes)
-- `bench_list_real_repo` - rust-lang/rust repo, warm caches
-- `bench_list_real_repo_cold_cache` - rust-lang/rust repo, cold caches
-
## Rust Repo Caching
-The rust-lang/rust repository is automatically cached:
-- **First run**: Clones to `target/bench-repos/rust` (~2-5 minutes)
-- **Subsequent runs**: Reuses cached clone (instant)
-- **Clean cache**: `rm -rf target/bench-repos/` or `cargo clean`
-- **Auto-recovery**: Corrupted caches are automatically removed and re-cloned
-
-**No manual intervention needed** - the cache works automatically.
+Real repo benchmarks clone rust-lang/rust on first run (~2-5 minutes). The clone is cached in `target/bench-repos/` and reused. Corrupted caches are auto-recovered.
-## Faster Iteration During Development
+## Faster Iteration
-### Option 1: Skip Slow Benchmarks
+**Skip slow benchmarks:**
```console
-# Skip cold cache and real repo benchmarks
cargo bench --bench list -- --skip cold --skip real
-
-# Run only specific benchmark
-cargo bench --bench list bench_list_by_worktree_count
```
-### Option 2: Use Pattern Matching
+**Pattern matching:**
```console
-# Run all benchmarks with "list_by" in the name
-cargo bench --bench list "list_by"
-
-# Run all "warm cache" benchmarks (excludes cold cache variants)
-cargo bench --bench list -- --skip cold
-```
-
-### Option 3: Reduce Sample Size (Future)
-Currently sample size is fixed at 30 samples per benchmark. To add a quick profile:
-
-```rust
-// In benches/list.rs, replace the criterion_group! with:
-#[cfg(feature = "quick-bench")]
-criterion_group! {
- name = benches;
- config = Criterion::default()
- .sample_size(10) // 3x faster
- .measurement_time(std::time::Duration::from_secs(10))
- .warm_up_time(std::time::Duration::from_secs(2));
- targets = /* ... */
-}
-
-#[cfg(not(feature = "quick-bench"))]
-criterion_group! {
- name = benches;
- config = Criterion::default()
- .sample_size(30)
- .measurement_time(std::time::Duration::from_secs(15))
- .warm_up_time(std::time::Duration::from_secs(3));
- targets = /* ... */
-}
+cargo bench --bench list scaling # All scaling benchmarks
+cargo bench --bench list -- --skip cold # Warm cache only
```
-Then run: `cargo bench --bench list --features quick-bench`
-
-## Understanding Benchmark Results
-
-### Warm vs Cold Cache
-
-**Warm cache** (default):
-- Git caches populated from previous operations
-- Represents typical interactive usage
-- Results: Faster, but may hit bottlenecks with many worktrees
-
-**Cold cache**:
-- All git caches invalidated before measurement
-- Simulates first-run performance
-- Caches invalidated:
- - Index (`.git/index`) - speeds up `git status`
- - Commit graph (`.git/objects/info/commit-graph`) - speeds up `git rev-list`
- - Packed refs (`.git/packed-refs`) - speeds up ref resolution
-- NOT invalidated:
- - Filesystem cache (OS-level, can't control)
- - Pack files (object storage, not a cache)
-
-### Expected Performance
+## Expected Performance
**Modest repos** (500 commits, 100 files):
- Cold cache penalty: ~5-16% slower
@@ -125,25 +40,10 @@ Then run: `cargo bench --bench list --features quick-bench`
**Large repos** (rust-lang/rust):
- Cold cache penalty: ~4x slower for single worktree
-- Scaling: Warm cache shows superlinear degradation, cold cache scales much better
-- Surprising result: Cold cache is faster than warm at 8 worktrees!
+- Scaling: Warm cache shows superlinear degradation, cold cache scales better
-## Benchmark Output Locations
+## Output Locations
- Results: `target/criterion/`
- Cached rust repo: `target/bench-repos/rust/`
-- Benchmark reports: `target/criterion/*/report/index.html`
-
-## Common Issues
-
-### "Git rev-parse failed in cached repo"
-**Solution**: Already handled automatically - corrupted cache is removed and re-cloned.
-
-### Benchmarks taking too long
-**Solutions**:
-1. Skip slow benchmarks: `cargo bench --bench list -- --skip cold --skip real`
-2. Run specific benchmark: `cargo bench --bench list bench_list_by_worktree_count`
-3. Use pattern matching to run subset
-
-### Out of disk space
-**Solution**: Clean the rust repo cache: `rm -rf target/bench-repos/`
+- HTML reports: `target/criterion/*/report/index.html`
diff --git a/benches/list.rs b/benches/list.rs
index 266f1a876..19d82dfee 100644
--- a/benches/list.rs
+++ b/benches/list.rs
@@ -1,62 +1,45 @@
// Benchmarks for `wt list` command
//
-// This suite measures raw performance and scaling characteristics:
+// Benchmark groups (Criterion group names in parentheses):
//
-// 1. Synthetic benchmarks (fast, deterministic):
-// - bench_time_to_skeleton: Measures time until skeleton appears (progressive mode)
-// - bench_time_to_skeleton_cold: Same as above but with packed-refs invalidated
-// - bench_time_to_complete: Measures full `wt list` execution (when all data appears), warm caches
-// - bench_time_to_complete_cold: Same as above but with packed-refs invalidated
-// - bench_list_by_worktree_count: Measures scaling with worktree count (1-8), warm caches
-// - bench_list_by_repo_profile: Measures scaling with repo size (minimal/typical/large), warm caches
-// - bench_sequential_vs_parallel: Light comparison of sequential vs parallel (3 data points), warm caches
-// - bench_list_cold_cache: Measures performance with all git caches invalidated (1, 4, 8 worktrees)
-// Invalidates: index, commit-graph, packed-refs
+// 1. Progressive rendering (skeleton vs complete):
+// - skeleton: Time until skeleton appears (progressive mode, warm cache)
+// - skeleton_cold: Same with packed-refs invalidated
+// - complete: Full execution time (warm cache)
+// - complete_cold: Same with packed-refs invalidated
//
-// 2. Real repository benchmarks (slower, more realistic):
-// - bench_list_real_repo: Uses rust-lang/rust repo (cloned to target/bench-repos/), warm caches
-// - bench_list_real_repo_cold_cache: Same as above but with all caches invalidated (1, 4, 8 worktrees)
+// 2. Scaling benchmarks:
+// - worktree_scaling: Worktree count scaling (1-8 worktrees, warm cache)
+// - worktree_scaling_cold: Same with all caches invalidated (index, commit-graph, packed-refs)
+// - profile_scaling: Repo size scaling (minimal/typical/large profiles)
+// - sequential_vs_parallel: Compare execution strategies
//
-// 3. Many branches benchmarks:
-// - bench_list_many_branches: 25/50/100 branches with unique commits, no worktrees, warm caches
-// - bench_list_many_branches_cold: Same as above but with packed-refs invalidated
+// 3. Real repository (rust-lang/rust):
+// - real_repo: Warm cache (1-8 worktrees)
+// - real_repo_cold: All caches invalidated (1, 4, 8 worktrees)
//
-// Run all benchmarks:
-// cargo bench --bench list
+// 4. Many branches (`wt list --branches`):
+// - many_branches: 25/50/100 branches (warm cache)
+// - many_branches_cold: Same with packed-refs invalidated
//
-// Run specific benchmark:
-// cargo bench --bench list bench_time_to_skeleton
-// cargo bench --bench list bench_list_by_worktree_count
-// cargo bench --bench list bench_list_cold_cache
-// cargo bench --bench list bench_list_real_repo
-// cargo bench --bench list bench_list_real_repo_cold_cache
-// cargo bench --bench list bench_list_many_branches
-// cargo bench --bench list bench_list_many_branches_cold
+// Run examples:
+// cargo bench --bench list # All benchmarks
+// cargo bench --bench list skeleton # Progressive rendering benchmarks
+// cargo bench --bench list worktree_scaling # Worktree scaling (warm + cold)
+// cargo bench --bench list many_branches # Many branches (no rust repo clone)
+// cargo bench --bench list real_repo # Real repo benchmarks only
+// cargo bench --bench list -- --skip cold # Skip all cold cache variants
//
-// Run only specific benchmarks (expensive setup is skipped via Criterion's filter):
-// cargo bench --bench list many_branches
-// cargo bench --bench list real_repo
-//
-// Compare warm vs cold on real repo:
-// cargo bench --bench list -- real_repo
-//
-// Note: Real repo benchmarks will clone rust-lang/rust on first run (~2-5 minutes).
+// Note: Real repo benchmarks clone rust-lang/rust on first run (~2-5 minutes).
// The clone is cached in target/bench-repos/ and reused across runs.
//
-// Cold cache benchmarks remove all git caches before each iteration to measure performance
-// without any git caching. This simulates first-run performance. Caches invalidated:
-// - Index (.git/index) - speeds up `git status` by ~10x
-// - Commit graph (.git/objects/info/commit-graph) - speeds up `git rev-list --count`
-// - Packed refs (.git/packed-refs) - speeds up ref resolution
+// Cold cache variants:
+// - "_cold" suffix means caches invalidated before each iteration
+// - skeleton_cold, complete_cold, many_branches_cold: Only packed-refs invalidated
+// - worktree_scaling_cold, real_repo_cold: All caches invalidated (index, commit-graph, packed-refs)
//
// Note: Filesystem cache (OS-level) and pack files are not invalidated. Pack files are
-// part of git's object storage, not a cache, so they remain for all benchmarks.
-//
-// CI benchmarks are not included because:
-// - The expensive operations (GitHub/GitLab API calls) are network-dependent
-// - CI detection (env var checking) is trivial (<1μs overhead)
-// - Output formatting differences are minimal
-// - Mocking APIs for reproducible benchmarks would be complex and not representative
+// part of git's object storage, not a cache.
use criterion::{BenchmarkId, Criterion, criterion_group, criterion_main};
use std::path::{Path, PathBuf};
@@ -270,8 +253,8 @@ fn add_worktree_with_divergence(
///
/// Uses WT_SKELETON_ONLY=1 to exit after skeleton is rendered.
/// This measures the actual worktrunk code path: git I/O, parsing, layout, render skeleton.
-fn bench_time_to_skeleton(c: &mut Criterion) {
- let mut group = c.benchmark_group("time_to_skeleton");
+fn bench_skeleton(c: &mut Criterion) {
+ let mut group = c.benchmark_group("skeleton");
let binary = get_release_binary();
let profile = &PROFILES[1];
@@ -321,10 +304,10 @@ fn bench_time_to_skeleton(c: &mut Criterion) {
group.finish();
}
-/// Cold cache variant of time_to_skeleton benchmark.
+/// Cold cache variant of skeleton benchmark.
/// Invalidates packed-refs before each measurement.
-fn bench_time_to_skeleton_cold(c: &mut Criterion) {
- let mut group = c.benchmark_group("time_to_skeleton_cold");
+fn bench_skeleton_cold(c: &mut Criterion) {
+ let mut group = c.benchmark_group("skeleton_cold");
let binary = get_release_binary();
let profile = &PROFILES[1];
@@ -389,8 +372,8 @@ fn bench_time_to_skeleton_cold(c: &mut Criterion) {
///
/// This measures the actual worktrunk code path including git operations, parsing,
/// layout calculation, and rendering - i.e., when all data is filled in.
-fn bench_time_to_complete(c: &mut Criterion) {
- let mut group = c.benchmark_group("time_to_complete");
+fn bench_complete(c: &mut Criterion) {
+ let mut group = c.benchmark_group("complete");
let binary = get_release_binary();
let profile = &PROFILES[1];
@@ -439,10 +422,10 @@ fn bench_time_to_complete(c: &mut Criterion) {
group.finish();
}
-/// Cold cache variant of time_to_complete benchmark.
+/// Cold cache variant of complete benchmark.
/// Invalidates packed-refs before each measurement.
-fn bench_time_to_complete_cold(c: &mut Criterion) {
- let mut group = c.benchmark_group("time_to_complete_cold");
+fn bench_complete_cold(c: &mut Criterion) {
+ let mut group = c.benchmark_group("complete_cold");
let binary = get_release_binary();
let profile = &PROFILES[1];
@@ -502,8 +485,8 @@ fn bench_time_to_complete_cold(c: &mut Criterion) {
group.finish();
}
-fn bench_list_by_worktree_count(c: &mut Criterion) {
- let mut group = c.benchmark_group("list_by_worktree_count");
+fn bench_worktree_scaling(c: &mut Criterion) {
+ let mut group = c.benchmark_group("worktree_scaling");
let binary = get_release_binary();
@@ -549,8 +532,8 @@ fn bench_list_by_worktree_count(c: &mut Criterion) {
group.finish();
}
-fn bench_list_cold_cache(c: &mut Criterion) {
- let mut group = c.benchmark_group("list_cold_cache");
+fn bench_worktree_scaling_cold(c: &mut Criterion) {
+ let mut group = c.benchmark_group("worktree_scaling_cold");
let binary = get_release_binary();
@@ -651,8 +634,8 @@ fn bench_list_cold_cache(c: &mut Criterion) {
group.finish();
}
-fn bench_list_by_repo_profile(c: &mut Criterion) {
- let mut group = c.benchmark_group("list_by_profile");
+fn bench_profile_scaling(c: &mut Criterion) {
+ let mut group = c.benchmark_group("profile_scaling");
let binary = get_release_binary();
@@ -822,8 +805,8 @@ fn get_or_clone_rust_repo() -> PathBuf {
.clone()
}
-fn bench_list_real_repo(c: &mut Criterion) {
- let mut group = c.benchmark_group("list_real_repo");
+fn bench_real_repo(c: &mut Criterion) {
+ let mut group = c.benchmark_group("real_repo");
let binary = get_release_binary();
@@ -891,8 +874,8 @@ fn bench_list_real_repo(c: &mut Criterion) {
group.finish();
}
-fn bench_list_real_repo_cold_cache(c: &mut Criterion) {
- let mut group = c.benchmark_group("list_real_repo_cold_cache");
+fn bench_real_repo_cold(c: &mut Criterion) {
+ let mut group = c.benchmark_group("real_repo_cold");
let binary = get_release_binary();
@@ -1069,8 +1052,8 @@ fn create_repo_with_many_branches(num_branches: usize) -> TempDir {
/// Benchmark `wt list --branches` with many branches.
/// Tests performance scaling with 25, 50, and 100 branches (no worktrees).
-fn bench_list_many_branches(c: &mut Criterion) {
- let mut group = c.benchmark_group("list_many_branches");
+fn bench_many_branches(c: &mut Criterion) {
+ let mut group = c.benchmark_group("many_branches");
let binary = get_release_binary();
@@ -1102,8 +1085,8 @@ fn bench_list_many_branches(c: &mut Criterion) {
/// Cold cache variant of many branches benchmark.
/// Invalidates packed-refs before each measurement to simulate first-run performance.
-fn bench_list_many_branches_cold(c: &mut Criterion) {
- let mut group = c.benchmark_group("list_many_branches_cold");
+fn bench_many_branches_cold(c: &mut Criterion) {
+ let mut group = c.benchmark_group("many_branches_cold");
let binary = get_release_binary();
@@ -1147,6 +1130,6 @@ criterion_group! {
.sample_size(30)
.measurement_time(std::time::Duration::from_secs(15))
.warm_up_time(std::time::Duration::from_secs(3));
- targets = bench_time_to_skeleton, bench_time_to_skeleton_cold, bench_time_to_complete, bench_time_to_complete_cold, bench_list_by_worktree_count, bench_list_by_repo_profile, bench_sequential_vs_parallel, bench_list_cold_cache, bench_list_real_repo, bench_list_real_repo_cold_cache, bench_list_many_branches, bench_list_many_branches_cold
+ targets = bench_skeleton, bench_skeleton_cold, bench_complete, bench_complete_cold, bench_worktree_scaling, bench_profile_scaling, bench_sequential_vs_parallel, bench_worktree_scaling_cold, bench_real_repo, bench_real_repo_cold, bench_many_branches, bench_many_branches_cold
}
criterion_main!(benches);