Skip to content

:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.

perf(ci): size jest shards to a six-minute wall, not ten (#126907)

James Greenhill6bc1516c0b78
diff --git a/.github/scripts/jest-select.js b/.github/scripts/jest-select.js
index 0b4e399b05d6..c6ab98202572 100644
--- a/.github/scripts/jest-select.js
+++ b/.github/scripts/jest-select.js
@@ -6,7 +6,12 @@ const path = require('node:path')
const FRONTEND_DIR = path.resolve(__dirname, '../../frontend')
// File counts approximate runtime; tune this from observed shard durations.
-const TEST_FILES_PER_SHARD = 342
+// A shard costs a fixed ~102s (checkout, pnpm install, quarantine gate) plus about
+// 31s + 1.38s per test file, measured across selective and full PR runs:
+// 2 files -> 34s, 318 files -> 503-520s. 171 files therefore lands near a 6-minute
+// shard instead of the 10 minutes 342 files produced. Re-fit this from the
+// `posthog-ci-jest-selection` shard durations when the suite's weight changes.
+const TEST_FILES_PER_SHARD = 171
const MAX_SHARDS = 8
const MAX_CHANGED_FILES = 200
diff --git a/.github/scripts/jest-select.test.js b/.github/scripts/jest-select.test.js
index c27b096f61ce..2e99319c8617 100644
--- a/.github/scripts/jest-select.test.js
+++ b/.github/scripts/jest-select.test.js
@@ -67,10 +67,10 @@ test('the changed-file ceiling includes its boundary', () => {
for (const [testCount, shardCount] of [
[0, 0],
- [342, 1],
- [343, 2],
- [1200, 4],
- [2000, 6],
+ [171, 1],
+ [172, 2],
+ [684, 4],
+ [1368, 8],
[100000, 8],
]) {
test(`${testCount} related tests use ${shardCount} shards`, () => {
diff --git a/docs/internal/ci-things-already-tried.md b/docs/internal/ci-things-already-tried.md
index 5f33135cc456..8a33d7f989df 100644
--- a/docs/internal/ci-things-already-tried.md
+++ b/docs/internal/ci-things-already-tried.md
@@ -226,7 +226,7 @@ _Also asked as:_ snob, is test selection on, why does CI run all the tests, do w
Jest selection on PRs ran three shards whatever it selected, and the selection was all or nothing: 60% of PR runs executed 2,171 of 2,720 test files and 39% executed under 50.
The import graph that `--findRelatedTests` walks held one cycle of 2,851 files, and 92% of its edges were imports the transformer removes. Reading imports from the transformer output shrinks it to 218 files and 997 tests.
-The 997 tests that load the cycle cost 3.65 seconds each against 1.07 for the rest, so a changed-file count sizes nothing. `.github/scripts/jest-select.js` counts the related tests and assigns one shard per 342 files, capped at eight. This initial threshold comes from the probe runs and needs tuning against observed shard durations; file counts do not guarantee a ten-minute wall. The selector classifies known fallback cases before installing dependencies. Pricing each shard from the hourly run's JUnit was built and removed: two probe runs sized by count landed within 1.4x of the estimate, and the per-file map cost a recorder job and a cache.
+The 997 tests that load the cycle cost 3.65 seconds each against 1.07 for the rest, so a changed-file count sizes nothing. `.github/scripts/jest-select.js` counts the related tests and assigns one shard per 171 files, capped at eight. Shard durations measured after that first threshold put a shard at about 102s of fixed setup plus 31s and 1.38s per test file, so the original 342 files produced a ten-minute shard where 171 gives a six-minute one. The selector classifies known fallback cases before installing dependencies. Pricing each shard from the hourly run's JUnit was built and removed: two probe runs sized by count landed within 1.4x of the estimate, and the per-file map cost a recorder job and a cache.
Measured on the way and not worth repeating: `--maxWorkers=100%` on the selective path gained nothing on the 4 vCPU runners; persisting jest's cache made a second run on the same runner 5% (12 seconds) faster; the V8 leak fix (`--retain-maps-for-n-gc=0`, [#114618](https://github.com/PostHog/posthog/pull/114618)) was worth 15 to 20% of a shard and about 5% on the merge queue; a different fixed shard count gained nothing.