:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
perf(ci): size jest shards to a six-minute wall, not ten (#126907)
James Greenhill6bc1516c0b78
diff --git a/.github/scripts/jest-select.js b/.github/scripts/jest-select.jsindex 0b4e399b05d6..c6ab98202572 100644--- a/.github/scripts/jest-select.js+++ b/.github/scripts/jest-select.js@@ -6,7 +6,12 @@ const path = require('node:path')const FRONTEND_DIR = path.resolve(__dirname, '../../frontend')// File counts approximate runtime; tune this from observed shard durations.-const TEST_FILES_PER_SHARD = 342+// A shard costs a fixed ~102s (checkout, pnpm install, quarantine gate) plus about+// 31s + 1.38s per test file, measured across selective and full PR runs:+// 2 files -> 34s, 318 files -> 503-520s. 171 files therefore lands near a 6-minute+// shard instead of the 10 minutes 342 files produced. Re-fit this from the+// `posthog-ci-jest-selection` shard durations when the suite's weight changes.+const TEST_FILES_PER_SHARD = 171const MAX_SHARDS = 8const MAX_CHANGED_FILES = 200diff --git a/.github/scripts/jest-select.test.js b/.github/scripts/jest-select.test.jsindex c27b096f61ce..2e99319c8617 100644--- a/.github/scripts/jest-select.test.js+++ b/.github/scripts/jest-select.test.js@@ -67,10 +67,10 @@ test('the changed-file ceiling includes its boundary', () => {for (const [testCount, shardCount] of [[0, 0],- [342, 1],- [343, 2],- [1200, 4],- [2000, 6],+ [171, 1],+ [172, 2],+ [684, 4],+ [1368, 8],[100000, 8],]) {test(`${testCount} related tests use ${shardCount} shards`, () => {diff --git a/docs/internal/ci-things-already-tried.md b/docs/internal/ci-things-already-tried.mdindex 5f33135cc456..8a33d7f989df 100644--- a/docs/internal/ci-things-already-tried.md+++ b/docs/internal/ci-things-already-tried.md@@ -226,7 +226,7 @@ _Also asked as:_ snob, is test selection on, why does CI run all the tests, do wJest selection on PRs ran three shards whatever it selected, and the selection was all or nothing: 60% of PR runs executed 2,171 of 2,720 test files and 39% executed under 50.The import graph that `--findRelatedTests` walks held one cycle of 2,851 files, and 92% of its edges were imports the transformer removes. Reading imports from the transformer output shrinks it to 218 files and 997 tests.-The 997 tests that load the cycle cost 3.65 seconds each against 1.07 for the rest, so a changed-file count sizes nothing. `.github/scripts/jest-select.js` counts the related tests and assigns one shard per 342 files, capped at eight. This initial threshold comes from the probe runs and needs tuning against observed shard durations; file counts do not guarantee a ten-minute wall. The selector classifies known fallback cases before installing dependencies. Pricing each shard from the hourly run's JUnit was built and removed: two probe runs sized by count landed within 1.4x of the estimate, and the per-file map cost a recorder job and a cache.+The 997 tests that load the cycle cost 3.65 seconds each against 1.07 for the rest, so a changed-file count sizes nothing. `.github/scripts/jest-select.js` counts the related tests and assigns one shard per 171 files, capped at eight. Shard durations measured after that first threshold put a shard at about 102s of fixed setup plus 31s and 1.38s per test file, so the original 342 files produced a ten-minute shard where 171 gives a six-minute one. The selector classifies known fallback cases before installing dependencies. Pricing each shard from the hourly run's JUnit was built and removed: two probe runs sized by count landed within 1.4x of the estimate, and the per-file map cost a recorder job and a cache.Measured on the way and not worth repeating: `--maxWorkers=100%` on the selective path gained nothing on the 4 vCPU runners; persisting jest's cache made a second run on the same runner 5% (12 seconds) faster; the V8 leak fix (`--retain-maps-for-n-gc=0`, [#114618](https://github.com/PostHog/posthog/pull/114618)) was worth 15 to 20% of a shard and about 5% on the merge queue; a different fixed shard count gained nothing.