ServerKit is a lightweight, modern server control panel for managing web applications, databases, and services on your VPS or dedicated server. Built with Python/Flask backend and React frontend.
Python astracloudflarecontrol-panelcyberpaneldnsfablehosting-control-panelhosting-panelkimi-k3server-managementserverkitubuntu-web-panelwebserver
perf(ci): shard the backend suite across 4 runners
Juan Denisac48dc5b4e4a
diff --git a/.github/workflows/backend-ci.yml b/.github/workflows/backend-ci.ymlindex 34b1142f..83b79e60 100644--- a/.github/workflows/backend-ci.yml+++ b/.github/workflows/backend-ci.yml@@ -5,22 +5,28 @@ name: Backend CI# registry, pairing, and the panel<->agent command loop exercised end-to-end# by tests/test_agent_poll_e2e.py — instead of finding out at install time.+# main is deliberately absent from `push`: release.yml gates itself on this+# workflow via `ci-gate`, so a push to main already runs the suite once. Listing+# main here ran it a second time, standalone, on every merge.on:push:- branches: [dev, main]+ branches: [dev]paths:- 'backend/**'- '.github/workflows/backend-ci.yml'pull_request:- branches: [dev, main]+ branches: [main]paths:- 'backend/**'- '.github/workflows/backend-ci.yml'workflow_call:jobs:- pytest:- name: pytest+ # The ratchet needs to see the WHOLE suite, so it cannot live inside a shard —+ # each shard only collects its own quarter. Its own job costs no wall-clock+ # time: it runs beside the shards and finishes in seconds.+ ratchet:+ name: test-count ratchetruns-on: ubuntu-lateststeps:- uses: actions/checkout@v4@@ -42,6 +48,49 @@ jobs:# Lowering the floor requires editing BASELINE_COUNT in the same commit.working-directory: backendrun: python tests/check_test_count.py- - name: Run tests++ # Sharded because this job WAS the entire wait: 21m50s of a ~22m pipeline,+ # while every other workflow finished inside a minute. Splitting the 3173+ # tests over 4 runners cuts the critical path to roughly a quarter.+ #+ # Sharding rather than pytest-xdist is deliberate: each shard is its own VM,+ # so the process-shared state that makes in-process parallelism unsafe here+ # (templates.json, APPS_DIR — see the per-PID DB dance in tests/conftest.py)+ # simply isn't shared. No test code has to change.+ #+ # NOTE: with no .test_durations file, pytest-split balances by test COUNT,+ # not by time — and these tests range from 0.1s to 10s+. If one shard lands+ # most of the slow ones it becomes the new critical path. If that shows up,+ # run once with `--store-durations` and commit backend/.test_durations to+ # switch it to duration-based balancing.+ pytest:+ name: pytest (${{ matrix.group }}/4)+ runs-on: ubuntu-latest+ strategy:+ # Report every failing shard in one pass instead of hiding shards 2-4+ # behind the first failure — one round of fixes instead of four.+ fail-fast: false+ matrix:+ group: [1, 2, 3, 4]+ steps:+ - uses: actions/checkout@v4+ - uses: actions/setup-python@v5+ with:+ python-version: '3.11'+ cache: pip+ cache-dependency-path: backend/requirements.txt+ - name: Install dependencies+ working-directory: backend+ run: |+ python -m pip install --upgrade pip+ pip install -r requirements.txt+ pip install pytest pytest-split+ - name: Run tests (shard ${{ matrix.group }} of 4)+ # Scoped to `tests` rather than a bare `pytest`. Identical here (3173+ # either way, since backend/dev-data/ is gitignored and absent from a+ # CI checkout), but it makes the command reproducible on a dev box: a+ # bare pytest there tries to collect the locally deployed apps under+ # backend/dev-data/ and dies during collection. Copy this line verbatim+ # to debug a red shard locally.working-directory: backend- run: python -m pytest -v+ run: python -m pytest tests -v --splits 4 --group ${{ matrix.group }}