SolidCraft: an open-source parametric 3D CAD application built in pure Rust. Desktop, web, and automation through MCP.
CI: GPU path tests in three shards of their own; Windows in the queue only when a group touches it (#7066)
bherbrucka6354dfec56d
diff --git a/.config/nextest.toml b/.config/nextest.tomlindex 1a0a7ddcf2..853777e723 100644--- a/.config/nextest.toml+++ b/.config/nextest.toml@@ -1,5 +1,6 @@# cargo-nextest settings for CI (`cargo xtask test` does not use nextest).-# CI builds the tests once and runs them in one job on a 32 vCPU runner (.github/workflows/ci.yml).+# CI builds the tests once and runs them in one job on a 32 vCPU runner (.github/workflows/ci.yml),+# except the GPU path tracer's tests (the gpu-path group below), which run in three shards of their own.[profile.ci]# Heavy tests (one to twelve minutes each on a CI runner) that pull requests skip. They run on@@ -72,6 +73,7 @@ priority = 100# Each nextest test has its own process/device. Concurrent cold lavapipe path# compiles contend for CPU and can exhaust the 600 s pipeline setup budget.# Serialize this module per CI shard; other tests retain their normal parallelism.+# CI runs these in their own job (GPU path tests in ci.yml), split into hash partitions.[test-groups.gpu-path]max-threads = 1diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.ymlindex 097cece0c2..3f4bcc583d 100644--- a/.github/workflows/ci.yml+++ b/.github/workflows/ci.yml@@ -9,12 +9,14 @@# Documentation, packaging, the other workflows and the GitHub bots' scripts select nothing:# no native job runs for them, on pull requests or on main.# Tests are built and run in one job on a big runner, slowest first, starting beside the repository-# checks (no archive to upload and download). A few heavy tests (see .config/nextest.toml) are-# skipped on pull requests and run on pushes to main and nightly. The Windows jobs (D3D12 shaders-# and app startup), the reference tests and the doc tests run on main and nightly only, not on-# pull requests (the fast lane, in Select checks); a break there is fixed forward on main. Only-# main saves the build caches; pull requests restore them. sccache is on a Blacksmith sticky disk that-# only main commits.+# checks (no archive to upload and download). The GPU path tracer's tests run one at a time+# (.config/nextest.toml), so they have a job of their own in three shards (GPU path tests). A few+# heavy tests (see .config/nextest.toml) are skipped on pull requests and run on pushes to main and+# nightly. The reference tests and the doc tests run in the merge queue, on main and nightly, not+# on pull requests (the fast lane, in Select checks); a break there is fixed forward on main. The+# Windows jobs (D3D12 shaders and app startup) run nightly, and in the merge queue only for a group+# that changes what they exercise. Only main saves the build caches; pull requests restore them.+# sccache is on a Blacksmith sticky disk that only main commits.# A push to main runs only the repository checks when a passing merge-queue run tested that very# commit, or when nothing a build reads changed since the last push to main whose run passed.# Otherwise it runs everything.@@ -30,7 +32,7 @@# reads `ci` from the newest run of a commit, so a run without it would block a PR that passed.# Cross-compilation is opt-in through Run workflow; release builds remain in release.yml.# Every Linux job runs on Blacksmith's Ubuntu 24.04 runners (blacksmith-<N>vcpu-ubuntu-2404), sized-# per job: 32 vCPU for the tests, 16 for Clippy, the archive build (nightly) and the merge queue's, 8 for compiles, doc and reference tests, 4 for the light checks, 2 for Select checks and ci.+# per job: 32 vCPU for the tests, 16 for Clippy, the archive build (nightly) and the merge queue's, 8 for compiles, the GPU path tests, doc and reference tests, 4 for the light checks, 2 for Select checks and ci.# The Windows shader job runs on Blacksmith's Windows Server 2025 runners (blacksmith-8vcpu-windows-2025);# the Windows app startup smoke stays on windows-latest, which has a desktop session that presents frames.# Pull requests, forks included, the merge queue, pushes to main and the nightly run all use them:@@ -103,6 +105,9 @@ jobs:kernel: ${{ steps.paths.outputs.kernel }}goldens: ${{ steps.paths.outputs.goldens }}windows: ${{ steps.paths.outputs.windows }}+ # The GPU path tests run: native, and the whole workspace or solidcraft-gpu selected. The+ # job's `if` and the `ci` job's check both read this, so they cannot disagree.+ gpu: ${{ steps.paths.outputs.gpu }}full: ${{ steps.paths.outputs.full }}# Code changed on a pull request that has no approved-for-ci label yet: only Quick compile.quick: ${{ steps.paths.outputs.quick }}@@ -122,6 +127,8 @@ jobs:BASE_SHA: ${{ github.event.pull_request.base.sha }}HEAD_SHA: ${{ github.event.pull_request.head.sha }}PUSH_SHA: ${{ github.sha }}+ GROUP_BASE_SHA: ${{ github.event.merge_group.base_sha }}+ GROUP_HEAD_SHA: ${{ github.event.merge_group.head_sha }}PR_NUMBER: ${{ github.event.pull_request.number }}HEAD_REPO: ${{ github.event.pull_request.head.repo.full_name }}HEAD_REF: ${{ github.event.pull_request.head.ref }}@@ -144,6 +151,23 @@ jobs:full=$(sed -n 's/^full=//p' "$RUNNER_TEMP/ci-affected")packages=$(sed -n 's/^packages=//p' "$RUNNER_TEMP/ci-affected")}+ # Whether the NUL-separated paths in $1 (the diff $2) reach what the Windows D3D12 jobs+ # exercise: the GPU crate and its shaders, the app and its startup, the viewport's wgpu+ # glue, the vendored egui-wgpu, their scripts, or a lockfile change that moves the+ # graphics stack (wgpu, naga, egui, winit, windows). Pull requests and merge groups share it.+ windows_paths() {+ while IFS= read -r -d '' path; do+ case "$path" in+ crates/gpu/*|apps/solidcraft/*|crates/ui-egui/src/gpu.rs|vendor/egui-wgpu/*) windows=true ;;+ tools/windows-*.ps1|.github/workflows/ci.yml|rust-toolchain*|.cargo/*) windows=true ;;+ esac+ done < "$1"+ if [ "$windows" = false ] && grep -qxz 'Cargo.lock' "$1" \+ && git diff "$2" -- Cargo.lock \+ | grep -Eq '^[ +-]name = "(wgpu[a-z_-]*|naga|egui[a-z_-]*|eframe|winit|windows[a-z_-]*|d3d12)"'; then+ windows=true+ fi+ }if [ "$EVENT_NAME" = push ]; then# The queue lands the very commit its merge_group run tested. If that run passed,# testing the push again proves nothing new; otherwise (a merge that skipped the@@ -164,13 +188,25 @@ jobs:# push before may never have run: a newer push cancels a waiting one).echo "$tested..$PUSH_SHA changes no code: repository checks only."else- # Code changed (or the push can't be compared): main runs everything.+ # Code changed (or the push can't be compared): main runs everything but Windows.+ # The merge queue ran Windows for the changes that reach it, and the nightly runs it+ # for everything else (it passed 129 of 129 non-PR runs before this).full=truepackages=native=true+ fi+ elif [ "$EVENT_NAME" = merge_group ]; then+ native=true+ # Windows (about 16 runner-minutes) only for a group that changes what it exercises;+ # the nightly runs it whatever changed. A group that can't be compared runs it.+ if git diff --no-renames --name-only -z "$GROUP_BASE_SHA...$GROUP_HEAD_SHA" > "$RUNNER_TEMP/ci-paths"; then+ windows_paths "$RUNNER_TEMP/ci-paths" "$GROUP_BASE_SHA...$GROUP_HEAD_SHA"+ elsewindows=truefi+ echo "windows=$windows for $GROUP_BASE_SHA...$GROUP_HEAD_SHA"elif [ "$EVENT_NAME" != pull_request ]; then+ # The nightly run and Run workflow: everything, Windows included.native=truewindows=trueelse@@ -185,19 +221,9 @@ jobs:case "$path" incrates/kernel/*|vendor/truck-*|tests/robustness/*) goldens=true ;;esac- # What the Windows D3D12 jobs exercise: the GPU crate and its shaders, the app and- # its startup, the viewport's wgpu glue, the vendored egui-wgpu, their scripts.- case "$path" in- crates/gpu/*|apps/solidcraft/*|crates/ui-egui/src/gpu.rs|vendor/egui-wgpu/*) windows=true ;;- tools/windows-*.ps1|.github/workflows/ci.yml|rust-toolchain*|.cargo/*) windows=true ;;- esacdone < "$RUNNER_TEMP/ci-paths"- # A lockfile change that moves the graphics stack (wgpu, naga, egui, winit, windows).- if [ "$windows" = false ] && grep -qxz 'Cargo.lock' "$RUNNER_TEMP/ci-paths" \- && git diff "$BASE_SHA...$HEAD_SHA" -- Cargo.lock \- | grep -Eq '^[ +-]name = "(wgpu[a-z_-]*|naga|egui[a-z_-]*|eframe|winit|windows[a-z_-]*|d3d12)"'; then- windows=true- fi+ # Computed as for a merge group, then turned off below (the fast lane).+ windows_paths "$RUNNER_TEMP/ci-paths" "$BASE_SHA...$HEAD_SHA"# Pull requests build and test the touched packages and their dependents; nothing# selected means no code changed.select_packages "$RUNNER_TEMP/ci-paths"@@ -242,8 +268,16 @@ jobs:goldens=falsefifi+ # The GPU path tests (their own job) run with the other tests, when solidcraft-gpu is+ # among the packages they test.+ gpu=false+ if [ "$native" = true ]; then+ case " $packages " in *" solidcraft-gpu "*) gpu=true ;; esac+ if [ "$full" = true ]; then gpu=true; fi+ fi{echo "native=$native"+ echo "gpu=$gpu"echo "kernel=$kernel"echo "goldens=$goldens"echo "windows=$windows"@@ -487,8 +521,10 @@ jobs:scope=(--workspace)if [ "$FULL" != true ]; then scope=(); for p in $PACKAGES; do scope+=(-p "$p"); done; fistatus=0+ # The GPU path tracer's tests run in their own job (GPU path tests): one at a time, they+ # took about 36 of this job's 45 minutes.cargo nextest run "${scope[@]}" --cargo-profile ci --locked --profile ci --no-tests=pass \- --ignore-default-filter -E "$base" || status=1+ --ignore-default-filter -E "($base) & not (package(=solidcraft-gpu) & test(/^path::/))" || status=1if [ "$status" != 0 ]; then# A failed golden gets the hint to re-record it (junit.xml lists the failures).if [ -f "$junit" ] && tr '\n' ' ' < "$junit" | grep -q '<testcase name="tests::robustness_goldens::[^"]*"[^>]*>[[:space:]]*<failure'; then@@ -508,6 +544,80 @@ jobs:path: ${{ env.CARGO_TARGET_DIR || 'target' }}/nextest/ci/junit.xmlretention-days: 7+ gpu-path-tests:+ name: GPU path tests (${{ matrix.shard }}/3)+ # The GPU path tracer's tests (about 108) compile their pipelines on lavapipe, one test at a+ # time (the gpu-path test group in .config/nextest.toml keeps them within the 600 s pipeline+ # setup budget). Serial, they took about 36 minutes, the longest chain in Native Linux tests;+ # here three shards run beside it. Their filter is the one Native Linux tests leaves out, so+ # every test runs exactly once.+ needs: changes+ if: ${{ !cancelled() && needs.changes.result == 'success' && needs.changes.outputs.gpu == 'true' }}+ runs-on: blacksmith-8vcpu-ubuntu-2404+ timeout-minutes: 45+ strategy:+ fail-fast: false+ matrix:+ shard: [1, 2, 3]+ env:+ # As in Native Linux tests: lavapipe draws, and a test without an adapter fails.+ SOLIDCRAFT_GPU: software+ SOLIDCRAFT_REQUIRE_GPU: "1"+ CARGO_INCREMENTAL: "0"+ RUSTC_WRAPPER: sccache+ SCCACHE_DIR: /home/runner/.cache/sccache+ SCCACHE_CACHE_SIZE: 20G+ steps:+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1+ - name: Install a software Vulkan driver (lavapipe)+ run: |+ sudo apt-get update+ sudo apt-get install -y --no-install-recommends mesa-vulkan-drivers libvulkan1+ - name: Size parallel compiles to the job's memory+ run: scripts/ci/build-jobs.sh+ - uses: dtolnay/rust-toolchain@stable+ # sccache lives on a Blacksmith sticky disk (an ext4 volume cloned from the last snapshot,+ # no 10 GB limit and no download). Only main commits it; pull requests, the merge queue and+ # forks mount a clone and never write back, so nothing they build can reach main's cache.+ # The shards build the same thing, so only the first one commits it (and saves the cache).+ - name: Mount the compiler cache+ uses: useblacksmith/stickydisk@3f6be1451e3cad893b778bc93495c80452a5da26 # v1.7.2+ continue-on-error: true+ with:+ key: ${{ github.repository }}-sccache-gpu+ path: /home/runner/.cache/sccache+ commit: ${{ github.ref == 'refs/heads/main' && matrix.shard == 1 && 'on-change' || 'false' }}+ # Main saves the cache; every other run restores it, read-only.+ - uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2+ with:+ key: native-gpu+ save-if: ${{ github.ref == 'refs/heads/main' && matrix.shard == 1 }}+ - name: Set up compiler cache+ uses: mozilla-actions/sccache-action@fc920bf0ec8de6ee65d409111f7ec508035751ba # v0.0.11+ with:+ version: v0.16.0+ - uses: taiki-e/install-action@1168f6810a3cb30a3514bf5756a3b7f0422c6df8 # v2.87.28+ with:+ tool: cargo-nextest+ - name: Run one shard of the GPU path tests+ env:+ SHARD: ${{ matrix.shard }}+ run: |+ # None of these is in the default filter (the heavy tests), so pull requests, the merge+ # queue, main and nightly all run every one of them, as before.+ cargo nextest run -p solidcraft-gpu --cargo-profile ci --locked --profile ci --no-tests=pass \+ --ignore-default-filter -E 'test(/^path::/)' --partition "hash:$SHARD/3"+ - name: Compiler cache statistics+ if: always()+ run: sccache --show-stats+ - name: Upload test timings+ if: always()+ uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2+ with:+ name: junit-gpu-${{ matrix.shard }}+ path: ${{ env.CARGO_TARGET_DIR || 'target' }}/nextest/ci/junit.xml+ retention-days: 7+action-matrix:# Every ordinary action on every face, edge, vertex and body of the design corpus, against# tests/action_matrix/baseline.json (docs/action-matrix.md). Every test run checks a sample;@@ -769,7 +879,7 @@ jobs:# commit's real run instead: the merge queue reads `ci` from the newest run of a commit, so a# run without a passing `ci` would block a pull request that passed (#1850, #3182, #3252).name: ci- needs: [changes, checks, quick-compile, clippy, build-tests, tests, doc-tests, oracle, windows-fxc, windows-startup]+ needs: [changes, checks, quick-compile, clippy, build-tests, tests, gpu-path-tests, doc-tests, oracle, windows-fxc, windows-startup]if: always() && (github.event_name != 'pull_request' || !github.event.pull_request.draft)runs-on: blacksmith-2vcpu-ubuntu-2404permissions:@@ -817,6 +927,8 @@ jobs:BUILD: ${{ needs.build-tests.result }}EVENT_NAME: ${{ github.event_name }}TESTS: ${{ needs.tests.result }}+ GPU_RUN: ${{ needs.changes.outputs.gpu }}+ GPU: ${{ needs.gpu-path-tests.result }}DOCS: ${{ needs.doc-tests.result }}ORACLE: ${{ needs.oracle.result }}WINDOWS_FXC: ${{ needs.windows-fxc.result }}@@ -836,6 +948,10 @@ jobs:fitest "$TESTS" = successfi+ # The same output decides whether the GPU path tests run (their job's `if`).+ if [ "$GPU_RUN" = true ]; then+ test "$GPU" = success+ fiif [ "$DOCTESTS" = true ]; thentest "$DOCS" = successfi