Skip to content

Worktrunk is a CLI for Git worktree management, designed for parallel AI agent workflows

fix(ci): split ci by cadence so coverage uploads on every main commit (#3608)

Maximilian Roos4202cffe6184
diff --git a/.claude/skills/running-tend/SKILL.md b/.claude/skills/running-tend/SKILL.md
index 0c150dbae..4e3029b74 100644
--- a/.claude/skills/running-tend/SKILL.md
+++ b/.claude/skills/running-tend/SKILL.md
@@ -47,26 +47,16 @@ source-compiling installs cascaded into bash-tool interrupts that blocked
even `pwd` and `echo`. (Pre-built single-script installers like Determinate
Nix's are fine — see **Weekly Maintenance: MSRV & Toolchain** for the one we
use. The block is specifically about long-running cargo compiles.) Instead,
-query Codecov directly:
-
-```bash
-REPO=$(gh repo view --json nameWithOwner --jq '.nameWithOwner')
-curl -sL "https://api.codecov.io/api/v2/gh/${REPO%/*}/repos/${REPO#*/}/compare/?pullid=<N>" > /tmp/codecov.json
-
-# Patch-level summary per file:
-jq '.files[] | {name: .name.head, patch: .totals.patch}' /tmp/codecov.json
-
-# Uncovered added lines in a specific changed file
-# (coverage.head is a LineType enum: 0=hit, 1=miss, 2=partial — filter on 1=miss):
-jq '.files[] | select(.name.head == "<path>") | .lines[] | select(.is_diff and .added and .coverage.head == 1) | {line: .number.head, code: .value}' /tmp/codecov.json
-```
+query Codecov directly, following `tests/CLAUDE.md` → **Coverage
+Investigation** for the endpoints and their traps.
If the Codecov API markers aren't enough, download the `code-coverage-report`
-artifact from the PR head's `ci` workflow run — it contains a `cobertura.xml`
-with per-line hit counts:
+artifact from the PR head's `coverage` workflow run — it contains a
+`cobertura.xml` with per-line hit counts:
```bash
-# Find the ci run on the PR head SHA:
+REPO=$(gh repo view --json nameWithOwner --jq '.nameWithOwner')
+# Find the coverage run on the PR head SHA:
CI_RUN=$(gh api "repos/$REPO/commits/<sha>/check-runs" --jq '.check_runs[] | select(.name == "code-coverage") | .details_url | capture("runs/(?<id>[0-9]+)") | .id')
# List artifacts, then download the coverage one:
gh api "repos/$REPO/actions/runs/$CI_RUN/artifacts" --jq '.artifacts[] | {name, id}'
@@ -297,8 +287,8 @@ Pinned third-party versions in CI are invisible to Dependabot — it follows `Ca
For each weekly run, check upstream and bump:
-- **`baptiste0928/cargo-install@v3` blocks** in `.github/workflows/ci.yaml`, `.github/workflows/nightly.yaml`, and `.github/actions/{test,claude}-setup/action.yaml` — every `version: "=X.Y.Z"` against `cargo info <crate>`. Today: `cargo-insta`, `cargo-nextest`, `cargo-llvm-cov`, `cargo-msrv`, `cargo-udeps`, `lychee`, `worktrunk`. The `cargo-affected` install has no version pin (follows default branch) — leave it alone. Verify each crate's `rust-version` against the pinned toolchain and note compatibility in the PR body (see PR #1657 for the format).
-- **`hustcer/setup-nu@v3`** `version:` input — latest from `gh api repos/nushell/nushell/releases/latest --jq '.tag_name'`. Four call sites: `ci.yaml` (`code-coverage`), `nightly.yaml` (`feature-powerset`), `benchmarks.yaml` (`benchmarks`), and `actions/test-setup/action.yaml`.
+- **`baptiste0928/cargo-install@v3` blocks** in `.github/workflows/{ci,coverage,nightly}.yaml` and `.github/actions/{test,claude}-setup/action.yaml` — every `version: "=X.Y.Z"` against `cargo info <crate>`. Today: `cargo-insta`, `cargo-nextest`, `cargo-llvm-cov`, `cargo-msrv`, `cargo-udeps`, `lychee`, `worktrunk`. The `cargo-affected` install has no version pin (follows default branch) — leave it alone. Verify each crate's `rust-version` against the pinned toolchain and note compatibility in the PR body (see PR #1657 for the format).
+- **`hustcer/setup-nu@v3`** `version:` input — latest from `gh api repos/nushell/nushell/releases/latest --jq '.tag_name'`. Four call sites: `coverage.yaml` (`code-coverage`), `nightly.yaml` (`feature-powerset`), `benchmarks.yaml` (`benchmarks`), and `actions/test-setup/action.yaml`.
- **`taiki-e/[email protected]`** `tool: zola@<ver>` in the `check-docs` job — latest from `gh api repos/getzola/zola/releases/latest --jq '.tag_name'`.
- **Runner images** — `ubuntu-24.04`, `macos-15`, `windows-2022`. Keep `windows-2022` pinned (actions/runner-images#12677 — windows-2025 lacks the D: drive).
diff --git a/.claude/skills/running-tend/references/nightly-cleaner.md b/.claude/skills/running-tend/references/nightly-cleaner.md
index 307b6a049..d14dffe8f 100644
--- a/.claude/skills/running-tend/references/nightly-cleaner.md
+++ b/.claude/skills/running-tend/references/nightly-cleaner.md
@@ -12,4 +12,4 @@ For each `.rs` file in the survey, also check:
## Repo-Wide CI Breakage
-A failure that reproduces identically on `main` and every PR (a broken system-package install in `code-coverage`, say) belongs to `tend-ci-fix`, which fires on any `ci`-workflow `failure` on `main`; a non-required job failing is enough to trigger it. It does not fire on runs that end `cancelled`, so record the breakage in the summary rather than assuming the handoff lands.
+A failure that reproduces identically on `main` and every PR (a broken system-package install in `code-coverage`, say) belongs to `tend-ci-fix`, which fires on any watched-workflow `failure` on `main` — the list lives in `.config/tend.yaml` under `ci-fix.watched_workflows`; a non-required job failing is enough to trigger it. It does not fire on runs that end `cancelled`, so record the breakage in the summary rather than assuming the handoff lands.
diff --git a/.config/tend.yaml b/.config/tend.yaml
index a440a76d2..970e3c58b 100644
--- a/.config/tend.yaml
+++ b/.config/tend.yaml
@@ -8,6 +8,11 @@ workflows:
ci-fix:
watched_workflows:
- ci
+ - coverage
+ # `affected-tests` is continue-on-error, so watching this workflow only
+ # signs Tend up for `collect-affected` failures — a stale DB degrades
+ # selection on every later PR, so it is worth fixing.
+ - affected
- publish-docs
# nightly hosts `full-tests` (the cargo-affected safety net), so a red
# nightly is actionable: Tend fixes it instead of leaving main's gaps
diff --git a/.github/.codecov.yaml b/.github/.codecov.yaml
index 456a22e89..534c3a179 100644
--- a/.github/.codecov.yaml
+++ b/.github/.codecov.yaml
@@ -1,7 +1,10 @@
comment: false
+# `cargo llvm-cov` reports only `src/`, so this is a guard rather than a live
+# filter. It has to be `tests/**`: codecov compiles `**/tests/**` to
+# `.*/tests/.*`, which needs a leading directory and so misses `tests/` itself.
ignore:
- - "**/tests/**"
+ - "tests/**"
coverage:
status:
diff --git a/.github/workflows/affected.yaml b/.github/workflows/affected.yaml
new file mode 100644
index 000000000..da10c8463
--- /dev/null
+++ b/.github/workflows/affected.yaml
@@ -0,0 +1,272 @@
+name: affected
+
+on:
+ push:
+ branches: [ main ]
+ pull_request:
+ branches: [ main ]
+ workflow_dispatch:
+
+# Grouped per-ref on main, which samples: a merge train cancels the queued run
+# and only the newest one collects. That's the intended cadence — a DB a few
+# commits old still anchors a correct superset (see the cache strategy below),
+# and collecting per-commit would mean three ~2h matrices per merge. Sampling
+# is also why these jobs sit in their own workflow: `collect` holds a group for
+# ~2h, so anything sharing it gets sampled too, whether or not that suits it.
+# PR runs supersede their own.
+concurrency:
+ group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+ cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
+env:
+ # Both jobs restore the rust-cache that `test` saves, and Swatinem/rust-cache
+ # hashes CARGO*/RUST* into the key, so these must match ci.yaml exactly
+ # (.github/CLAUDE.md → Build environment).
+ CARGO_TERM_COLOR: always
+ CARGO_INCREMENTAL: 0
+ RUSTFLAGS: -C debuginfo=0
+
+jobs:
+ # cargo-affected: runs only the tests whose recorded coverage overlaps the
+ # diff. Advisory at this stage — `test (linux/macos/windows)` still runs the
+ # full suite. The signal is the runtime delta on the exact-match path and
+ # whether selection misses any failures the full suite catches.
+ #
+ # Both jobs run as the same linux/macos/windows matrix as `test`. The
+ # fingerprint embeds `rustc -vV` (host triple), so a DB collected on one OS
+ # cache-misses on the others — each OS's `affected-tests` leg pairs with a
+ # `collect-affected` DB from the same OS via the `runner.os` cache key.
+ #
+ # TODO(affected-only PRs): the fast pre-merge gates are already split into the
+ # `fast-checks` job, so `test (linux/macos/windows)` is now purely the full
+ # nextest suite — the long-pole, and a drop-in shape for the affected legs.
+ # Once `affected tests (*)` has a track record of catching every failure the
+ # full suite catches on all three OSes, flip: make the affected legs required
+ # and delete `ci`'s PR `test` job — its full-matrix coverage already runs as
+ # `full-tests` in the `nightly` workflow. `fast-checks` stays required and
+ # unchanged. Confidence bar: a sustained stretch of PRs where the advisory
+ # legs and the required full matrix agree — no failure the full suite caught
+ # that affected selection missed.
+ #
+ # Until then the full `test` matrix stays required and runs on every PR,
+ # unchanged. Do NOT reach affected-only by making the required `test (*)`
+ # checks skip=pass on PRs — relaxing the merge gate is a deliberate
+ # branch-protection change owned by the repo admin (see `.github/CLAUDE.md`),
+ # not a silent workflow skip.
+ #
+ # The full suite must keep running after the flip, because cargo-affected
+ # misses non-Rust inputs (`include_str!`, templates, SQL), build-time inputs
+ # not in its fingerprint (build.rs, rust-toolchain.toml, .cargo/config.toml),
+ # and proc-macro source edits. Its home is the `full-tests` matrix in the
+ # `nightly` workflow (nightly cron, the `nightly` label, and before a
+ # release) — NOT push-to-main: a failure there means affected under-selected,
+ # and that must not redden main. Nightly failures are non-blocking and
+ # Tend-fixable. The Linux `--unreferenced reject` orphan check, intrinsically
+ # full-suite, rides `full-tests` too.
+ #
+ # Cache strategy:
+ # - `collect-affected` (push to main) saves `target/affected/coverage.db`
+ # to actions/cache keyed on the main commit sha. We cache only the DB,
+ # not the parent dir — cargo-affected drops its profraw staging dir
+ # at the end of every successful collect, but caching the path
+ # explicitly keeps the contract obvious and ~10 GB of profile bundles
+ # from leaking into the cache if that cleanup ever regresses.
+ # - `collect-affected` also restores the most recent prior main DB before
+ # collecting, so the new DB accumulates rows for up to FINGERPRINT_KEEP
+ # (=10) recent main-tip env_fingerprints (LRU-evicted in `Db::gc`).
+ # PRs whose manifests match any of those fingerprints get exact-match
+ # selection instead of the all-or-nothing single-fingerprint cache.
+ # Don't "simplify" by removing the restore — it's load-bearing.
+ # - `affected-tests` (PRs) restores the cache. Primary key is the
+ # PR/main merge-base — when collect ran on that exact commit, we get a
+ # tight diff (`PR changes only`) and the smallest possible selection.
+ # - Restore-keys fall back to the most recent main DB. Its `collect_sha`
+ # is typically a sibling of the PR's HEAD, not a strict ancestor.
+ # cargo-affected uses any sha still in the repo as a diff anchor, so
+ # the fallback drives normal selection — over-includes tests touched
+ # by main commits between the merge-base and collect_sha (correct
+ # superset), but never widens to "run everything" unless the cache is
+ # missing entirely.
+ # - Cache keys include `runner.os` (fingerprint embeds `rustc -vV`) and a
+ # manual `db-v{N}` marker. Bump the marker if cargo-affected ships an
+ # on-disk schema change; the cache is otherwise version-agnostic.
+ collect-affected:
+ name: collect affected coverage (${{ matrix.name }})
+ if: ${{ (github.event_name == 'push' && github.ref == 'refs/heads/main') || github.event_name == 'workflow_dispatch' }}
+ strategy:
+ fail-fast: false
+ matrix:
+ include:
+ - os: ubuntu-24.04
+ name: linux
+ - os: macos-15
+ name: macos
+ - os: windows-2022
+ name: windows
+ runs-on: ${{ matrix.os }}
+ steps:
+ - name: 📂 Checkout code
+ uses: actions/checkout@v7
+ with:
+ # `cargo affected run` later diffs PR HEAD against the sha that was
+ # HEAD when collect ran. That sha must be reachable.
+ fetch-depth: 0
+
+ - uses: ./.github/actions/test-setup
+ with:
+ # Restore-only: collect builds coverage-instrumented artifacts (it adds
+ # llvm-tools below), which must not overwrite the shared cache that
+ # `test` saves for everyone else.
+ save-cache: "false"
+
+ - name: "Use fast D: drive for temp files (Windows)"
+ if: runner.os == 'Windows'
+ shell: pwsh
+ run: |
+ New-Item -ItemType Directory -Force -Path "D:\tmp" | Out-Null
+ echo "TEMP=D:\tmp" >> $env:GITHUB_ENV
+ echo "TMP=D:\tmp" >> $env:GITHUB_ENV
+
+ - name: Install cargo-affected
+ uses: baptiste0928/cargo-install@v3
+ with:
+ crate: cargo-affected
+ git: https://github.com/max-sixty/cargo-affected
+ # No rev/branch/tag → action resolves to latest default-branch sha.
+
+ - name: Install llvm-tools
+ run: rustup component add llvm-tools
+
+ - name: 💾 Restore prior coverage DB
+ uses: actions/cache/restore@v6
+ with:
+ path: target/affected/coverage.db
+ # Primary key matches the current sha (no-op on first push of this
+ # commit; on re-runs of the same sha, lets us skip rebuilding from
+ # scratch).
+ key: cargo-affected-db-v1-${{ runner.os }}-${{ github.sha }}
+ # Fall back to any prior main DB. cargo-affected preserves rows for
+ # up to FINGERPRINT_KEEP (=10) distinct fingerprints in one DB,
+ # evicting LRU on each collect. By feeding a prior DB into the new
+ # collect, we accumulate fingerprint snapshots across main commits —
+ # PRs whose manifests match any of the last ~10 main fingerprints
+ # get exact-match selection, instead of the all-or-nothing
+ # single-fingerprint cache.
+ restore-keys: |
+ cargo-affected-db-v1-${{ runner.os }}-
+
+ - name: 📊 Collect coverage data
+ run: cargo affected collect -- --features shell-integration-tests
+
+ - name: 💾 Save coverage DB
+ uses: actions/cache/save@v6
+ with:
+ # Only the SQLite DB. `target/affected/profraw-<PID>/` holds raw
+ # profile bundles (~5 GB on this repo) that aren't needed past the
+ # current collect — caching them blows past the 10 GB repo cache
+ # cap and forces eviction of every prior cache.
+ path: target/affected/coverage.db
+ key: cargo-affected-db-v1-${{ runner.os }}-${{ github.sha }}
+
+ affected-tests:
+ name: affected tests (${{ matrix.name }}, advisory)
+ if: github.event_name == 'pull_request'
+ strategy:
+ fail-fast: false
+ matrix:
+ include:
+ - os: ubuntu-24.04
+ name: linux
+ - os: macos-15
+ name: macos
+ - os: windows-2022
+ name: windows
+ runs-on: ${{ matrix.os }}
+ # Advisory while we calibrate selection accuracy against the full suite.
+ # The full matrix (`test (linux/macos/windows)`) is still required for
+ # merge.
+ continue-on-error: true
+ steps:
+ - name: 📂 Checkout code
+ uses: actions/checkout@v7
+ with:
+ # Need the cached `collect_sha` reachable from HEAD for the diff;
+ # see collect-affected above.
+ fetch-depth: 0
+
+ - name: Compute merge-base for cache key
+ id: mb
+ shell: bash
+ env:
+ PR_HEAD: ${{ github.event.pull_request.head.sha }}
+ run: |
+ git fetch --no-tags --depth=200 origin main
+ sha=$(git merge-base origin/main "$PR_HEAD")
+ echo "sha=$sha" >> "$GITHUB_OUTPUT"
+ echo "merge-base with origin/main: $sha"
+
+ - uses: ./.github/actions/test-setup
+ with:
+ # PR-only job: restore `test`'s shared main cache, never save. This is
+ # the job that most needs the shared key — a per-job cache would have
+ # no main baseline (it never runs on main) and cold-build every PR.
+ save-cache: "false"
+
+ - name: "Use fast D: drive for temp files (Windows)"
+ if: runner.os == 'Windows'
+ shell: pwsh
+ run: |
+ New-Item -ItemType Directory -Force -Path "D:\tmp" | Out-Null
+ echo "TEMP=D:\tmp" >> $env:GITHUB_ENV
+ echo "TMP=D:\tmp" >> $env:GITHUB_ENV
+
+ # Restore AFTER test-setup so we land on top of rust-cache's tar
+ # extraction. rust-cache's restore step calls `cleanTargetDir` on a
+ # partial cache hit (`full match: false`), which deletes file children
+ # of `target/affected/` — including a `coverage.db` we'd just dropped
+ # there. Order: rust-cache populates `target/` first; we then drop the
+ # DB on top, where nothing else touches it.
+ - name: 💾 Restore coverage DB
+ uses: actions/cache/restore@v6
+ with:
+ path: target/affected/coverage.db
+ # Exact match: collect ran on the PR's merge-base → tight diff.
+ key: cargo-affected-db-v1-${{ runner.os }}-${{ steps.mb.outputs.sha }}
+ # Fallback: most recent main DB. cargo-affected runs all tests when
+ # the cached `collect_sha` is missing from the repo (rebased and
+ # pruned, beyond a shallow clone boundary). Sibling shas — including
+ # PR-vs-main-tip — drive normal selection.
+ restore-keys: |
+ cargo-affected-db-v1-${{ runner.os }}-
+
+ - name: Install cargo-affected
+ uses: baptiste0928/cargo-install@v3
+ with:
+ crate: cargo-affected
+ git: https://github.com/max-sixty/cargo-affected
+ # No rev/branch/tag → action resolves to latest default-branch sha.
+
+ - name: 🎯 Run affected tests
+ env:
+ # An empty affected selection is a valid outcome — a diff can touch
+ # nothing with recorded coverage. nextest's default exits 4 ("no
+ # tests to run"), which fails this advisory job spuriously. Windows
+ # hits this every run: the `#[cfg_attr(windows, ignore)]` tests never
+ # execute during `collect` (nextest skips ignored tests), so they get
+ # no coverage rows and cargo-affected keeps re-flagging them as `new`
+ # and selecting them — then `nextest run` skips them as ignored too,
+ # so zero run. `warn` exits 0 and leaves the breadcrumb in the log.
+ NEXTEST_NO_TESTS: warn
+ run: cargo affected run --report-json target/affected/report.json -- --features shell-integration-tests
+
+ # Report writes BEFORE nextest runs, so it survives test failures —
+ # uploading on `!cancelled()` makes it the most useful diagnostic
+ # when tests fail (cache state, fingerprint divergence, selection
+ # reasons). Schema documented at:
+ # https://github.com/max-sixty/cargo-affected/blob/main/docs/report-json.md
+ - name: 📊 Upload cargo-affected report
+ if: ${{ !cancelled() }}
+ uses: actions/upload-artifact@v7
+ with:
+ name: cargo-affected-report-${{ matrix.name }}
+ path: target/affected/report.json
diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml
index 59d838151..034ffb2a7 100644
--- a/.github/workflows/ci.yaml
+++ b/.github/workflows/ci.yaml
@@ -233,295 +233,3 @@ jobs:
- name: 🕷️ Build docs
run: zola build
working-directory: docs
-
- # cargo-affected: runs only the tests whose recorded coverage overlaps the
- # diff. Advisory at this stage — `test (linux/macos/windows)` still runs the
- # full suite. The signal is the runtime delta on the exact-match path and
- # whether selection misses any failures the full suite catches.
- #
- # Both jobs run as the same linux/macos/windows matrix as `test`. The
- # fingerprint embeds `rustc -vV` (host triple), so a DB collected on one OS
- # cache-misses on the others — each OS's `affected-tests` leg pairs with a
- # `collect-affected` DB from the same OS via the `runner.os` cache key.
- #
- # TODO(affected-only PRs): the fast pre-merge gates are already split into the
- # `fast-checks` job, so `test (linux/macos/windows)` is now purely the full
- # nextest suite — the long-pole, and a drop-in shape for the affected legs.
- # Once `affected tests (*)` has a track record of catching every failure the
- # full suite catches on all three OSes, flip: make the affected legs required
- # and delete this PR `test` job — its full-matrix coverage already runs as
- # `full-tests` in the `nightly` workflow. `fast-checks` stays required and
- # unchanged. Confidence bar: a sustained stretch of PRs where the advisory
- # legs and the required full matrix agree — no failure the full suite caught
- # that affected selection missed.
- #
- # Until then the full `test` matrix stays required and runs on every PR,
- # unchanged. Do NOT reach affected-only by making the required `test (*)`
- # checks skip=pass on PRs — relaxing the merge gate is a deliberate
- # branch-protection change owned by the repo admin (see `.github/CLAUDE.md`),
- # not a silent workflow skip.
- #
- # The full suite must keep running after the flip, because cargo-affected
- # misses non-Rust inputs (`include_str!`, templates, SQL), build-time inputs
- # not in its fingerprint (build.rs, rust-toolchain.toml, .cargo/config.toml),
- # and proc-macro source edits. Its home is the `full-tests` matrix in the
- # `nightly` workflow (nightly cron, the `nightly` label, and before a
- # release) — NOT push-to-main: a failure there means affected under-selected,
- # and that must not redden main. Nightly failures are non-blocking and
- # Tend-fixable. The Linux `--unreferenced reject` orphan check, intrinsically
- # full-suite, rides `full-tests` too.
- #
- # Cache strategy:
- # - `collect-affected` (push to main) saves `target/affected/coverage.db`
- # to actions/cache keyed on the main commit sha. We cache only the DB,
- # not the parent dir — cargo-affected drops its profraw staging dir
- # at the end of every successful collect, but caching the path
- # explicitly keeps the contract obvious and ~10 GB of profile bundles
- # from leaking into the cache if that cleanup ever regresses.
- # - `collect-affected` also restores the most recent prior main DB before
- # collecting, so the new DB accumulates rows for up to FINGERPRINT_KEEP
- # (=10) recent main-tip env_fingerprints (LRU-evicted in `Db::gc`).
- # PRs whose manifests match any of those fingerprints get exact-match
- # selection instead of the all-or-nothing single-fingerprint cache.
- # Don't "simplify" by removing the restore — it's load-bearing.
- # - `affected-tests` (PRs) restores the cache. Primary key is the
- # PR/main merge-base — when collect ran on that exact commit, we get a
- # tight diff (`PR changes only`) and the smallest possible selection.
- # - Restore-keys fall back to the most recent main DB. Its `collect_sha`
- # is typically a sibling of the PR's HEAD, not a strict ancestor.
- # cargo-affected uses any sha still in the repo as a diff anchor, so
- # the fallback drives normal selection — over-includes tests touched
- # by main commits between the merge-base and collect_sha (correct
- # superset), but never widens to "run everything" unless the cache is
- # missing entirely.
- # - Cache keys include `runner.os` (fingerprint embeds `rustc -vV`) and a
- # manual `db-v{N}` marker. Bump the marker if cargo-affected ships an
- # on-disk schema change; the cache is otherwise version-agnostic.
- collect-affected:
- name: collect affected coverage (${{ matrix.name }})
- if: ${{ (github.event_name == 'push' && github.ref == 'refs/heads/main') || github.event_name == 'workflow_dispatch' }}
- strategy:
- fail-fast: false
- matrix:
- include:
- - os: ubuntu-24.04
- name: linux
- - os: macos-15
- name: macos
- - os: windows-2022
- name: windows
- runs-on: ${{ matrix.os }}
- steps:
- - name: 📂 Checkout code
- uses: actions/checkout@v7
- with:
- # `cargo affected run` later diffs PR HEAD against the sha that was
- # HEAD when collect ran. That sha must be reachable.
- fetch-depth: 0
-
- - uses: ./.github/actions/test-setup
- with:
- # Restore-only: collect builds coverage-instrumented artifacts (it adds
- # llvm-tools below), which must not overwrite the shared cache that
- # `test` saves for everyone else.
- save-cache: "false"
-
- - name: "Use fast D: drive for temp files (Windows)"
- if: runner.os == 'Windows'
- shell: pwsh
- run: |
- New-Item -ItemType Directory -Force -Path "D:\tmp" | Out-Null
- echo "TEMP=D:\tmp" >> $env:GITHUB_ENV
- echo "TMP=D:\tmp" >> $env:GITHUB_ENV
-
- - name: Install cargo-affected
- uses: baptiste0928/cargo-install@v3
- with:
- crate: cargo-affected
- git: https://github.com/max-sixty/cargo-affected
- # No rev/branch/tag → action resolves to latest default-branch sha.
-
- - name: Install llvm-tools
- run: rustup component add llvm-tools
-
- - name: 💾 Restore prior coverage DB
- uses: actions/cache/restore@v6
- with:
- path: target/affected/coverage.db
- # Primary key matches the current sha (no-op on first push of this
- # commit; on re-runs of the same sha, lets us skip rebuilding from
- # scratch).
- key: cargo-affected-db-v1-${{ runner.os }}-${{ github.sha }}
- # Fall back to any prior main DB. cargo-affected preserves rows for
- # up to FINGERPRINT_KEEP (=10) distinct fingerprints in one DB,
- # evicting LRU on each collect. By feeding a prior DB into the new
- # collect, we accumulate fingerprint snapshots across main commits —
- # PRs whose manifests match any of the last ~10 main fingerprints
- # get exact-match selection, instead of the all-or-nothing
- # single-fingerprint cache.
- restore-keys: |
- cargo-affected-db-v1-${{ runner.os }}-
-
- - name: 📊 Collect coverage data
- run: cargo affected collect -- --features shell-integration-tests
-
- - name: 💾 Save coverage DB
- uses: actions/cache/save@v6
- with:
- # Only the SQLite DB. `target/affected/profraw-<PID>/` holds raw
- # profile bundles (~5 GB on this repo) that aren't needed past the
- # current collect — caching them blows past the 10 GB repo cache
- # cap and forces eviction of every prior cache.
- path: target/affected/coverage.db
- key: cargo-affected-db-v1-${{ runner.os }}-${{ github.sha }}
-
- affected-tests:
- name: affected tests (${{ matrix.name }}, advisory)
- if: github.event_name == 'pull_request'
- strategy:
- fail-fast: false
- matrix:
- include:
- - os: ubuntu-24.04
- name: linux
- - os: macos-15
- name: macos
- - os: windows-2022
- name: windows
- runs-on: ${{ matrix.os }}
- # Advisory while we calibrate selection accuracy against the full suite.
- # The full matrix (`test (linux/macos/windows)`) is still required for
- # merge.
- continue-on-error: true
- steps:
- - name: 📂 Checkout code
- uses: actions/checkout@v7
- with:
- # Need the cached `collect_sha` reachable from HEAD for the diff;
- # see collect-affected above.
- fetch-depth: 0
-
- - name: Compute merge-base for cache key
- id: mb
- shell: bash
- env:
- PR_HEAD: ${{ github.event.pull_request.head.sha }}
- run: |
- git fetch --no-tags --depth=200 origin main
- sha=$(git merge-base origin/main "$PR_HEAD")
- echo "sha=$sha" >> "$GITHUB_OUTPUT"
- echo "merge-base with origin/main: $sha"
-
- - uses: ./.github/actions/test-setup
- with:
- # PR-only job: restore `test`'s shared main cache, never save. This is
- # the job that most needs the shared key — a per-job cache would have
- # no main baseline (it never runs on main) and cold-build every PR.
- save-cache: "false"
-
- - name: "Use fast D: drive for temp files (Windows)"
- if: runner.os == 'Windows'
- shell: pwsh
- run: |
- New-Item -ItemType Directory -Force -Path "D:\tmp" | Out-Null
- echo "TEMP=D:\tmp" >> $env:GITHUB_ENV
- echo "TMP=D:\tmp" >> $env:GITHUB_ENV
-
- # Restore AFTER test-setup so we land on top of rust-cache's tar
- # extraction. rust-cache's restore step calls `cleanTargetDir` on a
- # partial cache hit (`full match: false`), which deletes file children
- # of `target/affected/` — including a `coverage.db` we'd just dropped
- # there. Order: rust-cache populates `target/` first; we then drop the
- # DB on top, where nothing else touches it.
- - name: 💾 Restore coverage DB
- uses: actions/cache/restore@v6
- with:
- path: target/affected/coverage.db
- # Exact match: collect ran on the PR's merge-base → tight diff.
- key: cargo-affected-db-v1-${{ runner.os }}-${{ steps.mb.outputs.sha }}
- # Fallback: most recent main DB. cargo-affected runs all tests when
- # the cached `collect_sha` is missing from the repo (rebased and
- # pruned, beyond a shallow clone boundary). Sibling shas — including
- # PR-vs-main-tip — drive normal selection.
- restore-keys: |
- cargo-affected-db-v1-${{ runner.os }}-
-
- - name: Install cargo-affected
- uses: baptiste0928/cargo-install@v3
- with:
- crate: cargo-affected
- git: https://github.com/max-sixty/cargo-affected
- # No rev/branch/tag → action resolves to latest default-branch sha.
-
- - name: 🎯 Run affected tests
- env:
- # An empty affected selection is a valid outcome — a diff can touch
- # nothing with recorded coverage. nextest's default exits 4 ("no
- # tests to run"), which fails this advisory job spuriously. Windows
- # hits this every run: the `#[cfg_attr(windows, ignore)]` tests never
- # execute during `collect` (nextest skips ignored tests), so they get
- # no coverage rows and cargo-affected keeps re-flagging them as `new`
- # and selecting them — then `nextest run` skips them as ignored too,
- # so zero run. `warn` exits 0 and leaves the breadcrumb in the log.
- NEXTEST_NO_TESTS: warn
- run: cargo affected run --report-json target/affected/report.json -- --features shell-integration-tests
-
- # Report writes BEFORE nextest runs, so it survives test failures —
- # uploading on `!cancelled()` makes it the most useful diagnostic
- # when tests fail (cache state, fingerprint divergence, selection
- # reasons). Schema documented at:
- # https://github.com/max-sixty/cargo-affected/blob/main/docs/report-json.md
- - name: 📊 Upload cargo-affected report
- if: ${{ !cancelled() }}
- uses: actions/upload-artifact@v7
- with:
- name: cargo-affected-report-${{ matrix.name }}
- path: target/affected/report.json
-
- code-coverage:
- runs-on: ubuntu-24.04
- steps:
- - name: 📂 Checkout code
- uses: actions/checkout@v7
-
- - uses: baptiste0928/cargo-install@v3
- with:
- crate: cargo-llvm-cov
- version: "=0.8.7"
-
- - name: 💰 Cache
- uses: Swatinem/rust-cache@v2
- with:
- # Own cache — coverage-instrumented builds differ from the shared test
- # cache. Already main-gated; cache-bin:false matches everywhere else.
- cache-bin: "false"
- save-if: ${{ github.ref == 'refs/heads/main' }}
-
- - name: Install shells (zsh, fish)
- run: sudo apt-get update && sudo apt-get install -y zsh fish
-
- - name: Install nushell
- uses: hustcer/setup-nu@v3
- with:
- version: '0.114.1'
-
- # Ensure nothing remains from caching
- - run: cargo llvm-cov clean --workspace
-
- - name: 📊 Generate coverage report
- run: cargo llvm-cov --features shell-integration-tests --cobertura --output-path=cobertura.xml
-
- - name: Upload code coverage results
- uses: actions/upload-artifact@v7
- with:
- name: code-coverage-report
- path: cobertura.xml
-
- - name: Upload to codecov.io
- uses: codecov/[email protected]
- with:
- files: cobertura.xml
- # Soft-fail on fork PRs: secrets aren't available there, so we rely on
- # tokenless upload, which can hiccup. Keep hard-fail on the main repo.
- fail_ci_if_error: ${{ github.repository_owner == 'max-sixty' }}
- token: ${{ secrets.CODECOV_TOKEN }}
diff --git a/.github/workflows/coverage.yaml b/.github/workflows/coverage.yaml
new file mode 100644
index 000000000..5b8022ac4
--- /dev/null
+++ b/.github/workflows/coverage.yaml
@@ -0,0 +1,77 @@
+name: coverage
+
+on:
+ push:
+ branches: [ main ]
+ pull_request:
+ branches: [ main ]
+ workflow_dispatch:
+
+# Keyed per-commit on main, so a merge train can't cancel a queued upload:
+# GitHub cancels the pending run whenever a newer one joins the group, and
+# codecov picks a PR's base by walking back to the newest ancestor that has a
+# report. A main commit that skips its upload therefore leaves later PRs
+# comparing against a stale base. That's why coverage sits outside `ci`, whose
+# group is shared with jobs that hold a run open for hours. PR runs still
+# supersede their own.
+concurrency:
+ group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}
+ cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
+env:
+ # Swatinem/rust-cache hashes CARGO*/RUST* into the cache key, so these track
+ # ci.yaml (.github/CLAUDE.md → Build environment).
+ CARGO_TERM_COLOR: always
+ CARGO_INCREMENTAL: 0
+ RUSTFLAGS: -C debuginfo=0
+
+jobs:
+ code-coverage:
+ runs-on: ubuntu-24.04
+ steps:
+ - name: 📂 Checkout code
+ uses: actions/checkout@v7
+
+ - uses: baptiste0928/cargo-install@v3
+ with:
+ crate: cargo-llvm-cov
+ version: "=0.8.7"
+
+ - name: 💰 Cache
+ uses: Swatinem/rust-cache@v2
+ with:
+ # Own cache — coverage-instrumented builds differ from the shared test
+ # cache. Already main-gated; cache-bin:false matches everywhere else.
+ cache-bin: "false"
+ save-if: ${{ github.ref == 'refs/heads/main' }}
+
+ - name: Install shells (zsh, fish)
+ run: sudo apt-get update && sudo apt-get install -y zsh fish
+
+ - name: Install nushell
+ uses: hustcer/setup-nu@v3
+ with:
+ version: '0.114.1'
+
+ # Ensure nothing remains from caching
+ - run: cargo llvm-cov clean --workspace
+
+ - name: 📊 Generate coverage report
+ run: cargo llvm-cov --features shell-integration-tests --cobertura --output-path=cobertura.xml
+
+ - name: Upload code coverage results
+ uses: actions/upload-artifact@v7
+ with:
+ name: code-coverage-report
+ path: cobertura.xml
+
+ - name: Upload to codecov.io
+ uses: codecov/[email protected]
+ with:
+ files: cobertura.xml
+ # Soft-fail on fork PRs: secrets aren't available there, so we rely on
+ # tokenless upload, which can hiccup. Keep hard-fail on the main repo.
+ # The head repo is what tells the two apart — `github.repository_owner`
+ # is the base repo's owner on a fork PR too.
+ fail_ci_if_error: ${{ github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository }}
+ token: ${{ secrets.CODECOV_TOKEN }}
diff --git a/.github/workflows/tend-ci-fix.yaml b/.github/workflows/tend-ci-fix.yaml
index e9fa3a0ae..498066864 100644
--- a/.github/workflows/tend-ci-fix.yaml
+++ b/.github/workflows/tend-ci-fix.yaml
@@ -9,7 +9,7 @@
name: tend-ci-fix
on:
workflow_run:
- workflows: ["ci", "publish-docs", "nightly"]
+ workflows: ["ci", "coverage", "affected", "publish-docs", "nightly"]
types: [completed]
branches: ["main"]
diff --git a/tests/CLAUDE.md b/tests/CLAUDE.md
index b4f4997c0..9f91f35a7 100644
--- a/tests/CLAUDE.md
+++ b/tests/CLAUDE.md
@@ -20,7 +20,7 @@ A target-filtered run (`--lib`, `--test integration`, …) on a fresh `target/`
## Coverage Investigation
-`task coverage` runs the suite and writes an HTML report to `target/llvm-cov/html/index.html`. Both CI (`code-coverage` job) and local `task coverage` pass `--features shell-integration-tests`, so code behind that flag is compiled and measured.
+`task coverage` runs the suite and writes an HTML report to `target/llvm-cov/html/index.html`. Both CI (the `coverage` workflow) and local `task coverage` pass `--features shell-integration-tests`, so code behind that flag is compiled and measured.
When `codecov/patch` fails, investigate before declaring ready (the merge gate itself is in the root `CLAUDE.md` → Coverage):
@@ -29,7 +29,34 @@ task coverage
cargo llvm-cov report --show-missing-lines | grep <file> # authoritative miss list; matches codecov line-for-line
```
-For each uncovered function, either write a test (integration tests via `assert_cmd_snapshot!` do capture subprocess coverage) or document why it's intentionally untested. If codecov's compare API must be queried directly, `coverage.head` is a `LineType` enum: `0=hit`, `1=miss`, `2=partial`, and per-file `.totals.head.diff` (`[files, lines, hits, misses, partials, coverage, …]`) is what reproduces the posted patch percentage — the top-level `totals.base.diff` reports different numbers. Prefer measuring: the API is for disputing a posted check, not a substitute for `task coverage`.
+For each uncovered function, either write a test (integration tests via `assert_cmd_snapshot!` do capture subprocess coverage) or document why it's intentionally untested.
+
+**Querying codecov directly** serves two cases the local report can't: disputing a posted check, and running in CI, where `task coverage` isn't installed. Prefer measuring everywhere else.
+
+```bash
+API=https://api.codecov.io/api/v2/github/max-sixty/repos/worktrunk
+# Full SHAs throughout; an abbreviation 404s. `?pullid=N` compares the PR's
+# *current* head, so name both SHAs to ask about an earlier commit.
+curl -sL "$API/compare/?base=<base-sha>&head=<head-sha>" > /tmp/codecov.json
+
+# Patch coverage per file. `.name` is an object, and the files carrying patch
+# lines are the ones with `has_diff`:
+jq '.files[] | select(.has_diff) | {name: .name.head, patch: .totals.patch}' /tmp/codecov.json
+
+# The missed patch lines in one file. `.coverage.head` is a LineType enum
+# (0=hit, 1=miss, 2=partial), and `.added` keeps context lines inside a hunk
+# from reading as patch misses:
+jq '.files[] | select(.name.head == "<path>") | .lines[]
+ | select(.is_diff and .added and .coverage.head == 1) | {line: .number.head, code: .value}' /tmp/codecov.json
+
+# Whole-file line coverage at one commit. No trailing slash after the path —
+# the route swallows it and answers 404 "coverage info not found":
+curl -sL "$API/file_report/<path>?sha=<sha>"
+```
+
+Per-file `.totals.patch` (equivalently `.totals.head.diff`, `[files, lines, hits, misses, …]`) holds that file's patch numbers, and the posted percentage aggregates them over the files in the PR's own diff. The top-level `totals.base.diff` is a different quantity: the base's coverage of those lines. Commit messages arrive with raw newlines in them, so the JSON is strictly invalid — `jq` copes, Python needs `json.loads(…, strict=False)`.
+
+**A compare listing files the PR never touched** means the merge-base has no report, so codecov walked back to the newest ancestor that does and diffed from there. The posted patch check still scopes to the PR's own diff; it's the API object that widens. Every main commit uploads from the `coverage` workflow, so this points at a failed or missing run on the base commit.
**`skim` fails with E0554 (`#![feature]` on stable):** the local `cargo-llvm-cov` predates 0.7.0, which stopped putting the coverage flags in global `RUSTFLAGS` and started instrumenting only workspace crates. Older versions leak `--cfg=coverage` into every dependency, and `skim` gates a nightly feature on it. Install the version the `code-coverage` job pins rather than working around it (`--no-cfg-coverage` also avoids it; `--no-rustc-wrapper` reinstates it).