Skip to content

Dodaios is an open-source agent OS for your desktop: it runs your coding agents, browser and apps by voice or chat, on your own machine. Formerly Personal Jarvis.

perf(ci): classify lanes on pushes to main, keep the full test suite there

rubenluetke10-beep765e96029eef
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index fd26ee3dc0..4e6a1cd7b9 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -14,8 +14,9 @@
# * Every PR ran every job (~20 runners, three 45-min macOS jobs on most),
# and pushes to main cancelled each other, so main was never proven.
# -> `detect` classifies the diff into lanes (scripts/ci/classify_changes.py,
-# fail-open); only affected lanes run on a PR. Pushes to main, the nightly
-# run and manual dispatches run everything and are never cancelled.
+# fail-open); only affected lanes run. Every push to main runs the whole
+# test suite and is never cancelled; the nightly run and manual
+# dispatches run every lane.
# * Branch protection listed five job names; renaming a job meant a settings
# change. -> `CI gate` aggregates every job (scripts/ci/required_results.py)
# and is the ONLY required check, for humans and for the merge train.
@@ -91,6 +92,7 @@ jobs:
pr_mode: ${{ steps.mode.outputs.pr_mode }}
linux_shards: ${{ steps.shards.outputs.linux_shards }}
windows_shards: ${{ steps.shards.outputs.windows_shards }}
+ tests_full: ${{ steps.shards.outputs.tests_full }}
os_matrix: ${{ steps.mode.outputs.os_matrix }}
steps:
- name: Checkout (full history for the diff base)
@@ -108,12 +110,16 @@ jobs:
IN_MACOS: ${{ inputs.include_macos }}
PR_BASE: ${{ github.event.pull_request.base.sha }}
run: |
- full=false; macos=false; pr_mode=false
+ # A push to main classifies its lanes like a pull request (the
+ # concurrent-job limit is shared by every agent's PR), but always
+ # runs the WHOLE test suite on Linux and Windows: post-merge proof
+ # is never narrowed. Nightly and manual full runs force everything.
+ full=false; macos=false; pr_mode=false; tests_full=false
case "$EVENT" in
- push) full=true ;;
- schedule) full=true; macos=true ;;
+ push) tests_full=true ;;
+ schedule) full=true; macos=true; tests_full=true ;;
workflow_dispatch)
- [ "$IN_FULL" = "true" ] && full=true
+ [ "$IN_FULL" = "true" ] && full=true && tests_full=true
[ "$IN_MACOS" = "true" ] && macos=true ;;
pull_request) pr_mode=true ;;
esac
@@ -130,7 +136,8 @@ jobs:
fi
if [ "$macos" = "true" ]; then os='["ubuntu-latest","windows-latest","macos-latest"]'; else os='["ubuntu-latest","windows-latest"]'; fi
{
- echo "full_mode=$full"; echo "macos=$macos"; echo "pr_mode=$pr_mode"; echo "base=$base"
+ echo "full_mode=$full"; echo "tests_full=$tests_full"; echo "macos=$macos"
+ echo "pr_mode=$pr_mode"; echo "base=$base"
echo "os_matrix=$os"
} | tee -a "$GITHUB_OUTPUT"
@@ -147,18 +154,21 @@ jobs:
git diff --name-only "$BASE" HEAD | tee changed.txt | python3 scripts/ci/classify_changes.py
fi
- # Linux always runs the whole suite in six shards. Windows runs four
- # shards only when every lane is on; otherwise one runner takes the
- # tests the diff can reach.
+ # Linux always runs the whole suite in six shards. Windows runs the
+ # whole suite in four shards on main, nightly, full manual runs and
+ # pipeline changes; on a pull request one runner takes the tests the
+ # diff can reach. A push always runs the python lane's test jobs.
- name: Size the test matrices
id: shards
shell: bash
run: |
echo 'linux_shards=[1,2,3,4,5,6]' >> "$GITHUB_OUTPUT"
- if [ "${{ steps.lanes.outputs.full }}" = "true" ]; then
+ if [ "${{ steps.mode.outputs.tests_full }}" = "true" ] || [ "${{ steps.lanes.outputs.full }}" = "true" ]; then
echo 'windows_shards=[1,2,3,4]' >> "$GITHUB_OUTPUT"
+ echo 'tests_full=true' >> "$GITHUB_OUTPUT"
else
echo 'windows_shards=[1]' >> "$GITHUB_OUTPUT"
+ echo 'tests_full=false' >> "$GITHUB_OUTPUT"
fi
gates:
@@ -313,7 +323,7 @@ jobs:
- name: Pick the tests (full run, or what this diff can reach)
id: pick
env:
- FULL: ${{ needs.detect.outputs.full }}
+ FULL: ${{ needs.detect.outputs.tests_full }}
BASE: ${{ needs.detect.outputs.base }}
SHARDS: ${{ needs.detect.outputs.windows_shards }}
run: |
diff --git a/docs/ci-pipeline.md b/docs/ci-pipeline.md
index 6e2f8301b4..d32716b38b 100644
--- a/docs/ci-pipeline.md
+++ b/docs/ci-pipeline.md
@@ -18,7 +18,7 @@ agent branch ──► pull request ──► CI (lanes) ──► CI gate ─
| Stage | What it does | Script |
| --- | --- | --- |
-| `detect` | Classifies the diff into lanes. Fails open: an empty diff, a pipeline change, a push to main, the nightly run and a manual run turn every lane on. | `scripts/ci/classify_changes.py` |
+| `detect` | Classifies the diff into lanes. Fails open: an empty diff, a pipeline change, the nightly run and a manual run turn every lane on. A push to main classifies its lanes too (the concurrent-job limit is shared), but always runs the whole test suite on Linux and Windows. | `scripts/ci/classify_changes.py` |
| `static gates` | ~20 repository gates (keys, bundle, mirrors, privacy, docs, CLI coverage, ratchets, bash 3.2, no new German) in one job. Every gate reports. | `scripts/ci/run_gates.py` |
| `python contracts (fast)` | Import cleanliness on the bare install, the named contract guards, skill-routing precision and recall, plugin auth. Blocking, no baseline. | — |
| `tests linux 1..6` | The whole suite in six shards. Batches of files run in fresh processes with a wall-clock budget; a failed batch is re-run file by file, a failed file once more (a pass there is reported as flaky). | `scripts/ci/run_tests_parallel.py` |