Worktrunk is a CLI for Git worktree management, designed for parallel AI agent workflows
Refactor benchmark naming for clarity and consistency
Maximilian Roosdb5cc8895fea
diff --git a/benches/CLAUDE.md b/benches/CLAUDE.mdindex e171cca4d..764096d7c 100644--- a/benches/CLAUDE.md+++ b/benches/CLAUDE.md@@ -1,123 +1,38 @@# Benchmark Guidelines+See `list.rs` header for the authoritative list of benchmark groups and run examples.+## Quick Start```console-# Run fast synthetic benchmarks (skip slow ones)+# Fast iteration (skip slow benchmarks)cargo bench --bench list -- --skip cold --skip real-# Run all benchmarks (includes real repo, takes ~1 hour)-cargo bench --bench list--# Run specific benchmark-cargo bench --bench list bench_list_by_worktree_count-cargo bench --bench list bench_list_cold_cache-cargo bench --bench list bench_list_real_repo+# Run specific group+cargo bench --bench list many_branches-# Run filtered benchmarks (expensive setup only runs for matching benchmarks)-cargo bench --bench list many_branches # Only many_branches, no rust repo clone-cargo bench --bench list real_repo # Only real_repo benchmarks+# All benchmarks (~1 hour)+cargo bench --bench list```-## Benchmark Types--### 1. Synthetic Benchmarks (Fast, ~5-10 minutes)-- `bench_time_to_skeleton` - Time until skeleton appears (progressive mode, warm caches)-- `bench_time_to_skeleton_cold` - Same as above but with packed-refs invalidated-- `bench_time_to_complete` - Full `wt list` execution (all data filled in), warm caches-- `bench_time_to_complete_cold` - Same as above but with packed-refs invalidated-- `bench_list_by_worktree_count` - Scaling with worktree count (1-8), warm caches-- `bench_list_by_repo_profile` - Scaling with repo size (minimal/typical/large), warm caches-- `bench_list_sequential_vs_parallel` - Sequential vs parallel comparison-- `bench_list_cold_cache` - Cold cache performance (all git caches invalidated)--### 2. Many Branches Benchmarks (Fast, ~3 minutes)-- `bench_list_many_branches` - Tests 25/50/100 branches with unique commits-- `bench_list_many_branches_cold` - Same with packed-refs invalidated--### 3. Real Repository Benchmarks (Slow, ~30-60 minutes)-- `bench_list_real_repo` - rust-lang/rust repo, warm caches-- `bench_list_real_repo_cold_cache` - rust-lang/rust repo, cold caches-## Rust Repo Caching-The rust-lang/rust repository is automatically cached:-- **First run**: Clones to `target/bench-repos/rust` (~2-5 minutes)-- **Subsequent runs**: Reuses cached clone (instant)-- **Clean cache**: `rm -rf target/bench-repos/` or `cargo clean`-- **Auto-recovery**: Corrupted caches are automatically removed and re-cloned--**No manual intervention needed** - the cache works automatically.+Real repo benchmarks clone rust-lang/rust on first run (~2-5 minutes). The clone is cached in `target/bench-repos/` and reused. Corrupted caches are auto-recovered.-## Faster Iteration During Development+## Faster Iteration-### Option 1: Skip Slow Benchmarks+**Skip slow benchmarks:**```console-# Skip cold cache and real repo benchmarkscargo bench --bench list -- --skip cold --skip real--# Run only specific benchmark-cargo bench --bench list bench_list_by_worktree_count```-### Option 2: Use Pattern Matching+**Pattern matching:**```console-# Run all benchmarks with "list_by" in the name-cargo bench --bench list "list_by"--# Run all "warm cache" benchmarks (excludes cold cache variants)-cargo bench --bench list -- --skip cold-```--### Option 3: Reduce Sample Size (Future)-Currently sample size is fixed at 30 samples per benchmark. To add a quick profile:--```rust-// In benches/list.rs, replace the criterion_group! with:-#[cfg(feature = "quick-bench")]-criterion_group! {- name = benches;- config = Criterion::default()- .sample_size(10) // 3x faster- .measurement_time(std::time::Duration::from_secs(10))- .warm_up_time(std::time::Duration::from_secs(2));- targets = /* ... */-}--#[cfg(not(feature = "quick-bench"))]-criterion_group! {- name = benches;- config = Criterion::default()- .sample_size(30)- .measurement_time(std::time::Duration::from_secs(15))- .warm_up_time(std::time::Duration::from_secs(3));- targets = /* ... */-}+cargo bench --bench list scaling # All scaling benchmarks+cargo bench --bench list -- --skip cold # Warm cache only```-Then run: `cargo bench --bench list --features quick-bench`--## Understanding Benchmark Results--### Warm vs Cold Cache--**Warm cache** (default):-- Git caches populated from previous operations-- Represents typical interactive usage-- Results: Faster, but may hit bottlenecks with many worktrees--**Cold cache**:-- All git caches invalidated before measurement-- Simulates first-run performance-- Caches invalidated:- - Index (`.git/index`) - speeds up `git status`- - Commit graph (`.git/objects/info/commit-graph`) - speeds up `git rev-list`- - Packed refs (`.git/packed-refs`) - speeds up ref resolution-- NOT invalidated:- - Filesystem cache (OS-level, can't control)- - Pack files (object storage, not a cache)--### Expected Performance+## Expected Performance**Modest repos** (500 commits, 100 files):- Cold cache penalty: ~5-16% slower@@ -125,25 +40,10 @@ Then run: `cargo bench --bench list --features quick-bench`**Large repos** (rust-lang/rust):- Cold cache penalty: ~4x slower for single worktree-- Scaling: Warm cache shows superlinear degradation, cold cache scales much better-- Surprising result: Cold cache is faster than warm at 8 worktrees!+- Scaling: Warm cache shows superlinear degradation, cold cache scales better-## Benchmark Output Locations+## Output Locations- Results: `target/criterion/`- Cached rust repo: `target/bench-repos/rust/`-- Benchmark reports: `target/criterion/*/report/index.html`--## Common Issues--### "Git rev-parse failed in cached repo"-**Solution**: Already handled automatically - corrupted cache is removed and re-cloned.--### Benchmarks taking too long-**Solutions**:-1. Skip slow benchmarks: `cargo bench --bench list -- --skip cold --skip real`-2. Run specific benchmark: `cargo bench --bench list bench_list_by_worktree_count`-3. Use pattern matching to run subset--### Out of disk space-**Solution**: Clean the rust repo cache: `rm -rf target/bench-repos/`+- HTML reports: `target/criterion/*/report/index.html`diff --git a/benches/list.rs b/benches/list.rsindex 266f1a876..19d82dfee 100644--- a/benches/list.rs+++ b/benches/list.rs@@ -1,62 +1,45 @@// Benchmarks for `wt list` command//-// This suite measures raw performance and scaling characteristics:+// Benchmark groups (Criterion group names in parentheses)://-// 1. Synthetic benchmarks (fast, deterministic):-// - bench_time_to_skeleton: Measures time until skeleton appears (progressive mode)-// - bench_time_to_skeleton_cold: Same as above but with packed-refs invalidated-// - bench_time_to_complete: Measures full `wt list` execution (when all data appears), warm caches-// - bench_time_to_complete_cold: Same as above but with packed-refs invalidated-// - bench_list_by_worktree_count: Measures scaling with worktree count (1-8), warm caches-// - bench_list_by_repo_profile: Measures scaling with repo size (minimal/typical/large), warm caches-// - bench_sequential_vs_parallel: Light comparison of sequential vs parallel (3 data points), warm caches-// - bench_list_cold_cache: Measures performance with all git caches invalidated (1, 4, 8 worktrees)-// Invalidates: index, commit-graph, packed-refs+// 1. Progressive rendering (skeleton vs complete):+// - skeleton: Time until skeleton appears (progressive mode, warm cache)+// - skeleton_cold: Same with packed-refs invalidated+// - complete: Full execution time (warm cache)+// - complete_cold: Same with packed-refs invalidated//-// 2. Real repository benchmarks (slower, more realistic):-// - bench_list_real_repo: Uses rust-lang/rust repo (cloned to target/bench-repos/), warm caches-// - bench_list_real_repo_cold_cache: Same as above but with all caches invalidated (1, 4, 8 worktrees)+// 2. Scaling benchmarks:+// - worktree_scaling: Worktree count scaling (1-8 worktrees, warm cache)+// - worktree_scaling_cold: Same with all caches invalidated (index, commit-graph, packed-refs)+// - profile_scaling: Repo size scaling (minimal/typical/large profiles)+// - sequential_vs_parallel: Compare execution strategies//-// 3. Many branches benchmarks:-// - bench_list_many_branches: 25/50/100 branches with unique commits, no worktrees, warm caches-// - bench_list_many_branches_cold: Same as above but with packed-refs invalidated+// 3. Real repository (rust-lang/rust):+// - real_repo: Warm cache (1-8 worktrees)+// - real_repo_cold: All caches invalidated (1, 4, 8 worktrees)//-// Run all benchmarks:-// cargo bench --bench list+// 4. Many branches (`wt list --branches`):+// - many_branches: 25/50/100 branches (warm cache)+// - many_branches_cold: Same with packed-refs invalidated//-// Run specific benchmark:-// cargo bench --bench list bench_time_to_skeleton-// cargo bench --bench list bench_list_by_worktree_count-// cargo bench --bench list bench_list_cold_cache-// cargo bench --bench list bench_list_real_repo-// cargo bench --bench list bench_list_real_repo_cold_cache-// cargo bench --bench list bench_list_many_branches-// cargo bench --bench list bench_list_many_branches_cold+// Run examples:+// cargo bench --bench list # All benchmarks+// cargo bench --bench list skeleton # Progressive rendering benchmarks+// cargo bench --bench list worktree_scaling # Worktree scaling (warm + cold)+// cargo bench --bench list many_branches # Many branches (no rust repo clone)+// cargo bench --bench list real_repo # Real repo benchmarks only+// cargo bench --bench list -- --skip cold # Skip all cold cache variants//-// Run only specific benchmarks (expensive setup is skipped via Criterion's filter):-// cargo bench --bench list many_branches-// cargo bench --bench list real_repo-//-// Compare warm vs cold on real repo:-// cargo bench --bench list -- real_repo-//-// Note: Real repo benchmarks will clone rust-lang/rust on first run (~2-5 minutes).+// Note: Real repo benchmarks clone rust-lang/rust on first run (~2-5 minutes).// The clone is cached in target/bench-repos/ and reused across runs.//-// Cold cache benchmarks remove all git caches before each iteration to measure performance-// without any git caching. This simulates first-run performance. Caches invalidated:-// - Index (.git/index) - speeds up `git status` by ~10x-// - Commit graph (.git/objects/info/commit-graph) - speeds up `git rev-list --count`-// - Packed refs (.git/packed-refs) - speeds up ref resolution+// Cold cache variants:+// - "_cold" suffix means caches invalidated before each iteration+// - skeleton_cold, complete_cold, many_branches_cold: Only packed-refs invalidated+// - worktree_scaling_cold, real_repo_cold: All caches invalidated (index, commit-graph, packed-refs)//// Note: Filesystem cache (OS-level) and pack files are not invalidated. Pack files are-// part of git's object storage, not a cache, so they remain for all benchmarks.-//-// CI benchmarks are not included because:-// - The expensive operations (GitHub/GitLab API calls) are network-dependent-// - CI detection (env var checking) is trivial (<1μs overhead)-// - Output formatting differences are minimal-// - Mocking APIs for reproducible benchmarks would be complex and not representative+// part of git's object storage, not a cache.use criterion::{BenchmarkId, Criterion, criterion_group, criterion_main};use std::path::{Path, PathBuf};@@ -270,8 +253,8 @@ fn add_worktree_with_divergence(////// Uses WT_SKELETON_ONLY=1 to exit after skeleton is rendered./// This measures the actual worktrunk code path: git I/O, parsing, layout, render skeleton.-fn bench_time_to_skeleton(c: &mut Criterion) {- let mut group = c.benchmark_group("time_to_skeleton");+fn bench_skeleton(c: &mut Criterion) {+ let mut group = c.benchmark_group("skeleton");let binary = get_release_binary();let profile = &PROFILES[1];@@ -321,10 +304,10 @@ fn bench_time_to_skeleton(c: &mut Criterion) {group.finish();}-/// Cold cache variant of time_to_skeleton benchmark.+/// Cold cache variant of skeleton benchmark./// Invalidates packed-refs before each measurement.-fn bench_time_to_skeleton_cold(c: &mut Criterion) {- let mut group = c.benchmark_group("time_to_skeleton_cold");+fn bench_skeleton_cold(c: &mut Criterion) {+ let mut group = c.benchmark_group("skeleton_cold");let binary = get_release_binary();let profile = &PROFILES[1];@@ -389,8 +372,8 @@ fn bench_time_to_skeleton_cold(c: &mut Criterion) {////// This measures the actual worktrunk code path including git operations, parsing,/// layout calculation, and rendering - i.e., when all data is filled in.-fn bench_time_to_complete(c: &mut Criterion) {- let mut group = c.benchmark_group("time_to_complete");+fn bench_complete(c: &mut Criterion) {+ let mut group = c.benchmark_group("complete");let binary = get_release_binary();let profile = &PROFILES[1];@@ -439,10 +422,10 @@ fn bench_time_to_complete(c: &mut Criterion) {group.finish();}-/// Cold cache variant of time_to_complete benchmark.+/// Cold cache variant of complete benchmark./// Invalidates packed-refs before each measurement.-fn bench_time_to_complete_cold(c: &mut Criterion) {- let mut group = c.benchmark_group("time_to_complete_cold");+fn bench_complete_cold(c: &mut Criterion) {+ let mut group = c.benchmark_group("complete_cold");let binary = get_release_binary();let profile = &PROFILES[1];@@ -502,8 +485,8 @@ fn bench_time_to_complete_cold(c: &mut Criterion) {group.finish();}-fn bench_list_by_worktree_count(c: &mut Criterion) {- let mut group = c.benchmark_group("list_by_worktree_count");+fn bench_worktree_scaling(c: &mut Criterion) {+ let mut group = c.benchmark_group("worktree_scaling");let binary = get_release_binary();@@ -549,8 +532,8 @@ fn bench_list_by_worktree_count(c: &mut Criterion) {group.finish();}-fn bench_list_cold_cache(c: &mut Criterion) {- let mut group = c.benchmark_group("list_cold_cache");+fn bench_worktree_scaling_cold(c: &mut Criterion) {+ let mut group = c.benchmark_group("worktree_scaling_cold");let binary = get_release_binary();@@ -651,8 +634,8 @@ fn bench_list_cold_cache(c: &mut Criterion) {group.finish();}-fn bench_list_by_repo_profile(c: &mut Criterion) {- let mut group = c.benchmark_group("list_by_profile");+fn bench_profile_scaling(c: &mut Criterion) {+ let mut group = c.benchmark_group("profile_scaling");let binary = get_release_binary();@@ -822,8 +805,8 @@ fn get_or_clone_rust_repo() -> PathBuf {.clone()}-fn bench_list_real_repo(c: &mut Criterion) {- let mut group = c.benchmark_group("list_real_repo");+fn bench_real_repo(c: &mut Criterion) {+ let mut group = c.benchmark_group("real_repo");let binary = get_release_binary();@@ -891,8 +874,8 @@ fn bench_list_real_repo(c: &mut Criterion) {group.finish();}-fn bench_list_real_repo_cold_cache(c: &mut Criterion) {- let mut group = c.benchmark_group("list_real_repo_cold_cache");+fn bench_real_repo_cold(c: &mut Criterion) {+ let mut group = c.benchmark_group("real_repo_cold");let binary = get_release_binary();@@ -1069,8 +1052,8 @@ fn create_repo_with_many_branches(num_branches: usize) -> TempDir {/// Benchmark `wt list --branches` with many branches./// Tests performance scaling with 25, 50, and 100 branches (no worktrees).-fn bench_list_many_branches(c: &mut Criterion) {- let mut group = c.benchmark_group("list_many_branches");+fn bench_many_branches(c: &mut Criterion) {+ let mut group = c.benchmark_group("many_branches");let binary = get_release_binary();@@ -1102,8 +1085,8 @@ fn bench_list_many_branches(c: &mut Criterion) {/// Cold cache variant of many branches benchmark./// Invalidates packed-refs before each measurement to simulate first-run performance.-fn bench_list_many_branches_cold(c: &mut Criterion) {- let mut group = c.benchmark_group("list_many_branches_cold");+fn bench_many_branches_cold(c: &mut Criterion) {+ let mut group = c.benchmark_group("many_branches_cold");let binary = get_release_binary();@@ -1147,6 +1130,6 @@ criterion_group! {.sample_size(30).measurement_time(std::time::Duration::from_secs(15)).warm_up_time(std::time::Duration::from_secs(3));- targets = bench_time_to_skeleton, bench_time_to_skeleton_cold, bench_time_to_complete, bench_time_to_complete_cold, bench_list_by_worktree_count, bench_list_by_repo_profile, bench_sequential_vs_parallel, bench_list_cold_cache, bench_list_real_repo, bench_list_real_repo_cold_cache, bench_list_many_branches, bench_list_many_branches_cold+ targets = bench_skeleton, bench_skeleton_cold, bench_complete, bench_complete_cold, bench_worktree_scaling, bench_profile_scaling, bench_sequential_vs_parallel, bench_worktree_scaling_cold, bench_real_repo, bench_real_repo_cold, bench_many_branches, bench_many_branches_cold}criterion_main!(benches);