Worktrunk is a CLI for Git worktree management, designed for parallel AI agent workflows
ci: run benchmarks on their own daily schedule, split from nightly (#3337)
Maximilian Roos820666741f4f
diff --git a/.claude/skills/running-tend/SKILL.md b/.claude/skills/running-tend/SKILL.mdindex 8d13b0ee5..81b7484d7 100644--- a/.claude/skills/running-tend/SKILL.md+++ b/.claude/skills/running-tend/SKILL.md@@ -278,7 +278,7 @@ Pinned third-party versions in CI are invisible to Dependabot — it follows `CaFor each weekly run, check upstream and bump:- **`baptiste0928/cargo-install@v3` blocks** in `.github/workflows/ci.yaml`, `.github/workflows/nightly.yaml`, and `.github/actions/{test,claude}-setup/action.yaml` — every `version: "=X.Y.Z"` against `cargo info <crate>`. Today: `cargo-insta`, `cargo-nextest`, `cargo-llvm-cov`, `cargo-msrv`, `cargo-udeps`, `lychee`, `worktrunk`. The `cargo-affected` install has no version pin (follows default branch) — leave it alone. Verify each crate's `rust-version` against the pinned toolchain and note compatibility in the PR body (see PR #1657 for the format).-- **`hustcer/setup-nu@v3`** `version:` input — latest from `gh api repos/nushell/nushell/releases/latest --jq '.tag_name'`. Three call sites: `ci.yaml` (`code-coverage`), `nightly.yaml` (`benchmarks`), and `actions/test-setup/action.yaml`.+- **`hustcer/setup-nu@v3`** `version:` input — latest from `gh api repos/nushell/nushell/releases/latest --jq '.tag_name'`. Four call sites: `ci.yaml` (`code-coverage`), `nightly.yaml` (`feature-powerset`), `benchmarks.yaml` (`benchmarks`), and `actions/test-setup/action.yaml`.- **`taiki-e/[email protected]`** `tool: zola@<ver>` in the `check-docs` job — latest from `gh api repos/getzola/zola/releases/latest --jq '.tag_name'`.- **Runner images** — `ubuntu-24.04`, `macos-15`, `windows-2022`. Keep `windows-2022` pinned (actions/runner-images#12677 — windows-2025 lacks the D: drive).diff --git a/.github/benchmark-failure.md b/.github/benchmark-failure.mdnew file mode 100644index 000000000..621939fa7--- /dev/null+++ b/.github/benchmark-failure.md@@ -0,0 +1,6 @@+---+title: Benchmarks failed+labels: ci+---++Benchmarks [failed on {{ date | date('YYYY-MM-DD') }}]({{ env.LINK }})diff --git a/.github/workflows/benchmarks.yaml b/.github/workflows/benchmarks.yamlnew file mode 100644index 000000000..c5cf17597--- /dev/null+++ b/.github/workflows/benchmarks.yaml@@ -0,0 +1,117 @@+name: benchmarks+# Full criterion suite plus the time-series gist append. Runs ~80 min and+# checks performance, not correctness, so it stands apart from the PR/merge+# flow and from nightly's correctness checks: a slow or failing bench never+# gates a merge. The daily cron is the perf-history feed (it appends to the+# gist); workflow_dispatch covers on-demand runs against a chosen branch.+#+# Not run on PRs or pushes, so the daily run is the only signal that the bench+# harness still builds — create-issue-on-benchmark-failure surfaces a break.+#+# Runner version pinned; see ci.yaml header comment for rationale.++on:+ schedule:+ # 3:47 UTC daily. Off-peak, offset from nightly's 5:37 so the two long+ # runs don't contend, and off :00 to be a good citizen w.r.t. GitHub's+ # cron scheduler.+ - cron: '47 3 * * *'+ workflow_dispatch:++concurrency:+ group: ${{ github.workflow }}-${{ github.ref }}+ cancel-in-progress: false++env:+ # Match ci.yaml/nightly.yaml: Swatinem/rust-cache hashes CARGO*/RUST* into+ # the cache key, so the shared `test` cache only restores when these agree+ # (see .github/CLAUDE.md).+ CARGO_TERM_COLOR: always+ CARGO_INCREMENTAL: 0+ RUSTFLAGS: -C debuginfo=0++jobs:+ benchmarks:+ runs-on: ubuntu-24.04+ steps:+ - name: 📂 Checkout code+ uses: actions/checkout@v7++ - name: 💰 Cache+ uses: Swatinem/rust-cache@v2+ with:+ # Restore the shared `test` cache (registry + deps), never save —+ # `cargo bench` is release-profile so it rebuilds its own artifacts,+ # but the dependency download/extract is still worth restoring.+ prefix-key: v1-rust+ shared-key: shared+ cache-bin: "false"+ save-if: false++ - name: Install shells (zsh, fish)+ run: sudo apt-get update && sudo apt-get install -y zsh fish++ - name: Install nushell+ uses: hustcer/setup-nu@v3+ with:+ version: '0.113.1'++ - name: 💰 Rust repo cache+ uses: actions/cache@v6+ with:+ path: target/bench-repos+ key: bench-repos-rust-${{ runner.os }}++ - name: 📊 Run benchmarks+ run: cargo bench++ - name: 📦 Upload benchmark results+ uses: actions/upload-artifact@v7+ with:+ name: benchmark-results-${{ github.run_id }}+ path: target/criterion++ # Time-series benchmark store, owned by worktrunk-bot:+ # https://gist.github.com/worktrunk-bot/19bb23cb9658722abfe69479d0a4f9bf+ #+ # Cron-only: workflow_dispatch runs aren't appended (would pollute the+ # time series). Skipped on forks: `TEND_BOT_TOKEN` is not exposed there.+ - name: 💾 Append results to gist+ if: github.repository_owner == 'max-sixty' && github.event_name == 'schedule'+ env:+ GITHUB_TOKEN: ${{ secrets.TEND_BOT_TOKEN }}+ GIST_ID: 19bb23cb9658722abfe69479d0a4f9bf+ run: |+ set -euo pipefail+ python3 .github/scripts/criterion-to-jsonl.py --sha "$GITHUB_SHA" > new-rows.jsonl+ git clone "https://x-access-token:${GITHUB_TOKEN}@gist.github.com/${GIST_ID}.git" /tmp/gist+ cat new-rows.jsonl >> /tmp/gist/results.jsonl+ git -C /tmp/gist \+ -c user.name=worktrunk-bot \+ -c [email protected] \+ commit -am "benchmarks: ${GITHUB_SHA::7}"+ git -C /tmp/gist push++ create-issue-on-benchmark-failure:+ needs: benchmarks+ if: always() && contains(needs.*.result, 'failure') && github.repository_owner == 'max-sixty' && github.event_name == 'schedule'+ runs-on: ubuntu-24.04+ permissions:+ contents: read+ issues: write+ steps:+ - name: 📂 Checkout code+ uses: actions/checkout@v7++ - uses: JasonEtco/create-an-issue@v2+ env:+ # Use TEND_BOT_TOKEN for a consistent bot identity (per+ # .github/CLAUDE.md) and so any future issue-triage automation can+ # cascade off issue creation — events from the default GITHUB_TOKEN+ # don't trigger other workflows.+ GITHUB_TOKEN: ${{ secrets.TEND_BOT_TOKEN }}+ LINK: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}+ with:+ filename: .github/benchmark-failure.md+ update_existing: true+ search_existing: opendiff --git a/.github/workflows/nightly.yaml b/.github/workflows/nightly.yamlindex 175e440c6..a456a3813 100644--- a/.github/workflows/nightly.yaml+++ b/.github/workflows/nightly.yaml@@ -5,7 +5,6 @@ name: nightly# - full-tests: full nextest suite on the linux/macos/windows matrix — the# cargo-affected safety net (see the job comment and ci.yaml's affected block)# - release-target: PTY+shell suite on the release triples ci.yaml doesn't cover-# - benchmarks: full criterion suite + time-series gist append (cron only)# - check-unused-dependencies: cargo-udeps on nightly toolchain# - minimal-versions: cargo check against minimum-version dep resolution# - nix-flake: nix flake check (packaging-environment bugs, motivated by #2624)@@ -252,73 +251,6 @@ jobs:NEXTEST_NO_INPUT_HANDLER: 1run: cargo nextest run --target ${{ matrix.target }}- benchmarks:- # Full criterion suite. Moved here from ci.yaml — runs ~80 min, was- # already advisory non-required (CLAUDE.md: "Don't wait for CI- # benchmarks before merging"). The cron run also appends results to the- # time-series gist so we have daily perf history on main.- needs: gate- if: needs.gate.outputs.run == 'true'- runs-on: ubuntu-24.04- steps:- - name: 📂 Checkout code- uses: actions/checkout@v7-- - name: 💰 Cache- uses: Swatinem/rust-cache@v2- with:- # Restore the shared `test` cache (registry + deps), never save —- # `cargo bench` is release-profile so it rebuilds its own artifacts,- # but the dependency download/extract is still worth restoring.- prefix-key: v1-rust- shared-key: shared- cache-bin: "false"- save-if: false-- - name: Install shells (zsh, fish)- run: sudo apt-get update && sudo apt-get install -y zsh fish-- - name: Install nushell- uses: hustcer/setup-nu@v3- with:- version: '0.113.1'-- - name: 💰 Rust repo cache- uses: actions/cache@v6- with:- path: target/bench-repos- key: bench-repos-rust-${{ runner.os }}-- - name: 📊 Run benchmarks- run: cargo bench-- - name: 📦 Upload benchmark results- uses: actions/upload-artifact@v7- with:- name: benchmark-results-${{ github.run_id }}- path: target/criterion-- # Time-series benchmark store, owned by worktrunk-bot:- # https://gist.github.com/worktrunk-bot/19bb23cb9658722abfe69479d0a4f9bf- #- # Cron-only: PR/push runs aren't appended (would pollute the time- # series). Skipped on forks: `TEND_BOT_TOKEN` is not exposed there.- - name: 💾 Append results to gist- if: github.repository_owner == 'max-sixty' && github.event_name == 'schedule'- env:- GITHUB_TOKEN: ${{ secrets.TEND_BOT_TOKEN }}- GIST_ID: 19bb23cb9658722abfe69479d0a4f9bf- run: |- set -euo pipefail- python3 .github/scripts/criterion-to-jsonl.py --sha "$GITHUB_SHA" > new-rows.jsonl- git clone "https://x-access-token:${GITHUB_TOKEN}@gist.github.com/${GIST_ID}.git" /tmp/gist- cat new-rows.jsonl >> /tmp/gist/results.jsonl- git -C /tmp/gist \- -c user.name=worktrunk-bot \- -c [email protected] \- commit -am "benchmarks: ${GITHUB_SHA::7}"- git -C /tmp/gist push-check-unused-dependencies:# Moved from ci.yaml — `cargo udeps` requires nightly toolchain anyway,# so it's already in the "nightly concern" bucket; rarely flips between@@ -453,7 +385,6 @@ jobs:needs:- feature-powerset- release-target- - benchmarks- check-unused-dependencies- minimal-versions- nix-flakediff --git a/CLAUDE.md b/CLAUDE.mdindex 447e228ff..092539b76 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -143,7 +143,7 @@ Why: wt installs a `signal_hook` SIGINT/SIGTERM handler so it can forward signal## Benchmarks & Traces-`cargo bench --bench list <filter>` (Criterion takes a positional substring filter; there's no `--skip`). `cargo run -p wt-perf -- timeline -- <args>` traces one `wt` invocation. Real-repo benchmarks clone rust-lang/rust on first run. The `benchmarks` CI job is non-required — only `test (linux|macos|windows)` block merge; `mergeStateStatus: UNSTABLE` from a still-pending bench run is mergeable. Filter map, expected numbers, and trace queries: `benches/CLAUDE.md`.+`cargo bench --bench list <filter>` (Criterion takes a positional substring filter; there's no `--skip`). `cargo run -p wt-perf -- timeline -- <args>` traces one `wt` invocation. Real-repo benchmarks clone rust-lang/rust on first run. Benchmarks run as a standalone scheduled workflow (`.github/workflows/benchmarks.yaml`, daily cron plus `workflow_dispatch`), not on PRs, so they never gate a merge; only `test (linux|macos|windows)` block it. Filter map, expected numbers, and trace queries: `benches/CLAUDE.md`.## Code Quality