An open source design system that's fully customizable and agent ready
TypeScript
ci: pull-request runs out of time start one at a time, at most one in ten minutes (#9640)
Cindy Zhang550ff216d586
diff --git a/.github/actions/ci-admission/action.yml b/.github/actions/ci-admission/action.ymlindex d9284997ae..7a3d041665 100644--- a/.github/actions/ci-admission/action.yml+++ b/.github/actions/ci-admission/action.yml@@ -15,7 +15,9 @@# "Wait for room under the CI ceiling": a waiting run is found by that name.# - A pull-request run that finds no room and ten other pull-request runs# already waiting is turned away (`full`): CI is full, and the run stops with-# that message. `main` runs are never turned away.+# that message. So is one still without room at its longest wait, unless it+# is the only pull-request run to start there in ten minutes and no run that+# waited longer still waits. `main` runs are never turned away.# - A run starts once the other active runs' estimated jobs plus `peak` fit# under the ceiling. Small workflows that are not admitted (one or two slim# jobs a run) always count at least eighteen jobs, which covers all but their@@ -26,7 +28,7 @@# - It waits at most `max-wait-minutes` (fifteen for `main`, twelve for a pull# request, inside its fifteen-minute job budget), looking at even intervals# plus a random 0-20 s so runs that look together spread out, and then-# starts.+# starts (a pull request: see above).# - Each CI or Main run counts its live jobs, read once per look. A# pull-request run still waiting here counts the jobs it runs, its waiting# job and a Storybook build that found room, so waiting runs never hold each@@ -151,6 +153,25 @@ runs:return {spends: true, interval: LOOK_MS};};const STEP = 'Wait for room under the CI ceiling';+ // A pull-request run that reaches its longest wait without room+ // starts only if no other pull-request run started at its longest+ // wait in the last ten minutes, about the time one run takes, and+ // no run that has waited longer still waits; otherwise CI is full+ // for it. So runs that waited out the same burst start one at a+ // time, and CI passes ninety by about one run's jobs at most+ // (spec:AST-030 DEC-16).+ const AT_LIMIT_SPACING_MS = 10 * 60_000;+ // A wait this close to the longest wait counts as one that ran out.+ const AT_LIMIT_SLACK_MS = 30_000;+ const isPullRun = run => run.event === 'pull_request' && run.path === '.github/workflows/ci.yml';+ // When a run's wait began and, once admitted, ended: its first wait+ // step's start and its last wait step's end.+ const waitSpan = waits => {+ const times = key => waits.map(step => Date.parse(step[key] ?? '')).filter(Number.isFinite);+ const began = times('started_at');+ const ended = times('completed_at');+ return {began: began.length ? Math.min(...began) : NaN, ended: ended.length ? Math.max(...ended) : NaN};+ };// At most this many pull-request runs wait at once; one more finding// no room is turned away (spec:AST-030 DEC-16).const MAX_WAITING = 10;@@ -219,11 +240,13 @@ runs:// A look planned past the longest wait comes at its end instead.const left = MAX_WAIT_MS - (Date.now() - started);const next = Math.min((planned > 0 ? planned : started + Math.max(1, looked) * LOOK_MS) - Date.now(), left - JITTER_MS);- if (looked >= MAX_LOOKS || left <= 0) {+ // At its longest wait, a pull request decides on one more look+ // whether it may start (AT_LIMIT_SPACING_MS); other runs start.+ if ((looked >= MAX_LOOKS || left <= 0) && pr == null) {core.warning(`Still no room after ${MAX_WAIT_MS / 60_000} minutes; starting.`);return;}- if (next > 0) await new Promise(resolve => setTimeout(resolve, next + Math.floor(Math.random() * JITTER_MS)));+ if (next > 0 && looked < MAX_LOOKS) await new Promise(resolve => setTimeout(resolve, next + Math.floor(Math.random() * JITTER_MS)));}// Looks that found no room; a look that only yielded to `main` is// not one of them.@@ -238,6 +261,12 @@ runs:let small = 0;let mainWaiting = false;let waitingPulls = 0;+ // Pull-request runs that started at their longest wait lately,+ // and those still waiting that began waiting before this one.+ const startedAtLimit = [];+ // Pull-request runs still waiting, or out of time and not yet+ // decided, by when their wait began.+ const pullWaiters = [];// What each run counted, logged on every look.const counted = [];const smallBy = {};@@ -272,7 +301,9 @@ runs:const waits = jobs.flatMap(job => job.steps ?? []).filter(step => step.name === STEP);+ const span = waitSpan(waits);if (waits.some(step => step.status === 'in_progress')) {+ if (isPullRun(run)) pullWaiters.push({id: run.id, began: span.began});// Still waiting for room. A `main` run holds its whole// peak, ahead of this one. A pull-request run holds what// it runs: the job that waits and its Storybook build,@@ -284,6 +315,16 @@ runs:counted.push(`${short(run)} ${run.id} ${run.event}: ${holds} waiting`);continue;}+ if (isPullRun(run) && span.ended - span.began >= MAX_WAIT_MS - AT_LIMIT_SLACK_MS) {+ // Out of time: it started only once its admission job+ // succeeded. Until admission decides, it ranks as a waiter;+ // a run CI was full for never started.+ const decided = jobs.find(job => job.name === 'admission' && job.status === 'completed');+ if (decided == null) pullWaiters.push({id: run.id, began: span.began});+ else if (decided.conclusion === 'success' && Date.now() - span.ended < AT_LIMIT_SPACING_MS) {+ startedAtLimit.push({id: run.id, ago: Date.now() - span.ended});+ }+ }const live = jobs.filter(job => LIVE.includes(job.status)).length;const ended = waits.map(step => Date.parse(step.completed_at ?? '')).filter(Number.isFinite);const admittedAt = ended.length > 0 ? Math.max(...ended) : NaN;@@ -327,7 +368,39 @@ runs:const left = MAX_WAIT_MS - (Date.now() - started);const wait = Math.min(next.interval + Math.floor(Math.random() * JITTER_MS), left);if (spent >= MAX_LOOKS || wait <= 0) {- core.warning(`Still no room after ${MAX_WAIT_MS / 60_000} minutes (about ${others} other jobs${mainWaiting ? ', a main check waiting' : ''}); starting.`);+ const why = `Still no room after ${MAX_WAIT_MS / 60_000} minutes (about ${others} other jobs${mainWaiting ? ', a main check waiting' : ''})`;+ if (pr == null) {+ core.warning(`${why}; starting.`);+ return;+ }+ // Wait starts compared on one clock: this run's own first wait+ // step as GitHub recorded it (whole seconds), else its clock+ // floored to the second; a tie goes to the lower run id.+ let mine = Math.floor(started / 1000) * 1000;+ try {+ calls += 1;+ const {data: {jobs: own}} = await github.rest.actions.listJobsForWorkflowRun({+ ...context.repo, run_id: context.runId, filter: 'latest', per_page: 100,+ });+ const began = waitSpan(own.flatMap(job => job.steps ?? []).filter(step => step.name === STEP)).began;+ if (Number.isFinite(began)) mine = began;+ } catch {+ // The floored clock stands.+ }+ const waitedLonger = pullWaiters.filter(+ other => other.began < mine || (other.began === mine && other.id < context.runId),+ ).length;+ if (startedAtLimit.length === 0 && waitedLonger === 0) {+ core.warning(`${why}; starting, the one pull-request run to start at its longest wait for ${AT_LIMIT_SPACING_MS / 60_000} minutes.`);+ // The whole run starts now, not only its Storybook build.+ core.setOutput('fits', 'true');+ return;+ }+ const other = startedAtLimit.length > 0+ ? `run ${startedAtLimit[0].id} started at its longest wait ${Math.round(startedAtLimit[0].ago / 60_000)} minutes ago`+ : `${waitedLonger} pull-request run${waitedLonger === 1 ? '' : 's'} that waited longer still wait${waitedLonger === 1 ? 's' : ''}`;+ core.setOutput('full', 'true');+ core.warning(`CI is full: ${why.charAt(0).toLowerCase()}${why.slice(1)}, and ${other}.`);return;}core.info(`Waiting for room: about ${others} other jobs + ${peak}${mainWaiting ? ', a main check waiting' : ''}.`);diff --git a/.github/scripts/ci-admission.test.mjs b/.github/scripts/ci-admission.test.mjsindex 729bf81b24..c47f7babd8 100644--- a/.github/scripts/ci-admission.test.mjs+++ b/.github/scripts/ci-admission.test.mjs@@ -17,7 +17,7 @@ import path from 'node:path';import {afterEach, describe, expect, it, vi} from 'vitest';import yaml from 'yaml';import {matrixFor} from './ci-matrix.mjs';-import {PR_CI_GATE} from './ci-pr-gate.mjs';+import {PR_CI_GATE, ungated} from './ci-pr-gate.mjs';const root = path.resolve(import.meta.dirname, '../..');const read = file => yaml.parse(fs.readFileSync(path.join(root, file), 'utf8'));@@ -87,10 +87,13 @@ describe('where runs wait', () => {it('stops a pull-request run that CI is full for, with the reason, and builds nothing', () => {const job = ci.jobs['ci-full'];- expect(needsOf(job)).toEqual(['room']);- expect(job.if).toBe(- "github.event_name == 'pull_request' && needs.room.outputs.full == 'true'",+ // Either wait can find CI full: room, with ten runs waiting, or the rest+ // of the run at its longest wait.+ expect(needsOf(job)).toEqual(['room', 'admission']);+ expect(ungated(job.if)).toBe(+ "github.event_name == 'pull_request' && !cancelled() && (needs.room.outputs.full == 'true' || needs.admission.outputs.full == 'true')",);+ expect(admission.outputs.full).toBe('${{ steps.rest.outputs.full }}');expect(job.permissions).toEqual({actions: 'write'});const [stop] = job.steps;expect(stop['continue-on-error']).toBeUndefined();@@ -105,35 +108,86 @@ describe('where runs wait', () => {});it("fails a fork's run that CI is full for, which cannot cancel itself, in every required context", () => {- const isStop = step =>+ const ROOM_FULL = "needs.room.outputs.full == 'true'";+ const EITHER_FULL = `${ROOM_FULL} || needs.admission.outputs.full == 'true'`;+ const isStop = (step, condition = ROOM_FULL) =>step.name === 'Stop when CI is full' &&- step.if === "needs.room.outputs.full == 'true'" &&+ step.if === condition &&step.run.includes('CI is full: re-run all jobs or push later.') &&step.run.trim().endsWith('exit 1') &&step['continue-on-error'] === undefined;// Admission fails first, so every job behind it without its own status- // rule skips.+ // rule skips; at its longest wait it fails after the wait.expect(isStop(admission.steps[0])).toBe(true);- // The required contexts run, and fail with the reason before any work.+ const last = admission.steps.at(-1);+ expect(last.name).toBe('Stop when CI is full at the longest wait');+ expect(last.if).toBe("steps.rest.outputs.full == 'true'");+ expect(last.run).toContain('CI is full: re-run all jobs or push later.');+ expect(last.run.trim().endsWith('exit 1')).toBe(true);+ // The required contexts run, and fail with the reason before any work,+ // whichever wait found CI full.for (const id of ['build', 'test', 'docsite-test']) {const job = ci.jobs[id];- expect(needsOf(job), id).toContain('room');+ expect(needsOf(job), id).toEqual(+ expect.arrayContaining(['room', 'admission']),+ );+ expect(isStop(job.steps[1], EITHER_FULL), id).toBe(true);expect(job.steps[0].name, id).toBe('Require a branch in this repository or the ci:run label',);- expect(isStop(job.steps[1]), id).toBe(true);}// build runs although the Storybook build it reads skipped.expect(String(ci.jobs.build.if)).toContain("needs.room.outputs.full == 'true'",);- // The jobs that would still run behind a failed admission build nothing.- for (const id of ['typecheck', 'pr-interaction', 'pr-interaction-webkit']) {- expect(needsOf(ci.jobs[id]), id).toContain('room');- expect(String(ci.jobs[id].if), id).toContain(- "needs.room.outputs.full != 'true'",+ // Every job that would still run behind a failed admission builds+ // nothing. It is every job with a status function that reaches admission+ // or the Storybook build, unless it stops with the reason (the required+ // contexts, ci-full), runs only after admission or the label lookup+ // succeeded, never runs on a pull request, or does no work without a job+ // behind admission.+ const reaches = (id, target, seen = new Set()) =>+ needsOf(ci.jobs[id]).some(+ need =>+ need === target ||+ (!seen.has(need) && (seen.add(need), reaches(need, target, seen))),);+ const FULL_GUARD = "needs.admission.outputs.full != 'true'";+ const ADMITTED =+ /needs\.(admission|run-labels)\.result == 'success'|needs\.run-labels\.outputs\./;+ const DISPATCH_ONLY =+ /^(always\(\) && )?\(?github\.event_name == 'workflow_dispatch' && inputs\.operation == 'release-check'/;+ // Its every step after the runner pick reads native-swift-runner's+ // `applies`, empty when that job skipped behind admission.+ const NO_WORK_ALONE = ['test-native-swift'];+ expect(ADMITTED.test(String(ci.jobs['run-labels'].if))).toBe(true);+ expect(String(ci.jobs['run-labels'].if)).toContain(+ "needs.admission.result == 'success'",+ );+ const guarded = [];+ for (const [id, job] of Object.entries(ci.jobs)) {+ const condition = String(job.if ?? '');+ if (!/\b(always|cancelled)\(\)/.test(condition)) continue;+ if (!reaches(id, 'admission') && !reaches(id, 'build-storybook'))+ continue;+ if (['build', 'test', 'docsite-test', 'ci-full'].includes(id)) continue;+ if (+ ADMITTED.test(condition) ||+ DISPATCH_ONLY.test(ungated(condition).trim())+ )+ continue;+ if (NO_WORK_ALONE.includes(id)) continue;+ expect(needsOf(job), id).toContain('admission');+ expect(condition, id).toContain(FULL_GUARD);+ guarded.push(id);}+ expect(guarded.sort()).toEqual([+ 'pr-interaction',+ 'pr-interaction-webkit',+ 'pr-touched-components',+ 'pr-visual',+ 'typecheck',+ ]);});it('skips only with every other job, and never fails, so every job and required check decides as before', () => {@@ -183,6 +237,9 @@ describe('where runs wait', () => {expect(admission.steps.filter(step => step !== wait)).toEqual([expect.objectContaining({name: 'Stop when CI is full'}),expect.objectContaining({uses: 'actions/checkout@v7'}),+ expect.objectContaining({+ name: 'Stop when CI is full at the longest wait',+ }),]);expect(ci.jobs.room.steps.filter(step => step.name !== STEP)).toEqual([expect.objectContaining({uses: 'actions/checkout@v7'}),@@ -205,7 +262,11 @@ describe('where runs wait', () => {]) {const condition = String(ci.jobs[id].if);expect(condition, id).toMatch(/\b(always|cancelled)\(\)/);- expect(condition, id).not.toContain('needs.admission');+ // Only whether CI was full for the run, never admission's result.+ expect(+ condition.replaceAll("needs.admission.outputs.full != 'true'", ''),+ id,+ ).not.toContain('needs.admission');}// lint and review-required are written by workflows admission does not touch.for (const file of ['lint.yml', 'review-signal.yml', 'review-clear.yml']) {@@ -461,7 +522,7 @@ describe('ci admission', () => {expect(per_page).toBe(100);calls += 1;return {- data: {jobs: runs().find(run => run.id === run_id).jobs ?? []},+ data: {jobs: runs().find(run => run.id === run_id)?.jobs ?? []},};},},@@ -731,18 +792,33 @@ describe('ci admission', () => {});expect(near.lookTimes[0]).toBeGreaterThanOrEqual(4 * 60_000);expect(near.lookTimes[0]).toBeLessThanOrEqual(4 * 60_000 + 20_000);- // Room already spent both looks: the rest starts at once.+ // Room already spent both looks: a pull request decides on one more+ // look, at once, whether it may start at its longest wait; it may.const spent = await admit({active: prs(7),peak: 16,phase: 'rest',fits: 'false',- started: String(NOW),+ started: String(NOW - 12 * 60_000),looked: '2',maxWait: 12,maxLooks: 2,});- expect(spent).toMatchObject({looks: 0, calls: 0});+ expect(spent.looks).toBe(1);+ expect(spent.lookTimes[0]).toBe(0);+ expect(spent.warning[0]).toContain('starting, the one pull-request run');+ expect(spent.outputs.full).toBeUndefined();+ // A main run there starts at once, without a look.+ const mainSpent = await admit({+ active: prs(7),+ peak: 16,+ isMain: true,+ phase: 'rest',+ fits: 'false',+ started: String(NOW - 15 * 60_000),+ looked: '8',+ });+ expect(mainSpent).toMatchObject({looks: 0, calls: 0});});it('leaves fits unset when a later look stops early, so the rest starts', async () => {@@ -923,6 +999,238 @@ describe('ci admission', () => {});});+ describe('one start at the longest wait at a time', () => {+ const at = minutes => new Date(NOW + minutes * 60_000).toISOString();+ /** A pull-request run admitted at `endedAt` minutes after its wait began `waited` minutes before. */+ const startedRun = (+ id,+ endedAt,+ waited,+ {jobs = 12, admission = 'success'} = {},+ ) => ({+ id,+ status: 'in_progress',+ event: 'pull_request',+ path: '.github/workflows/ci.yml',+ jobs: [+ {+ name: 'room',+ status: 'completed',+ steps: [+ {+ name: STEP,+ status: 'completed',+ started_at: at(endedAt - waited),+ completed_at: at(endedAt),+ },+ ],+ },+ admission == null+ ? {name: 'admission', status: 'queued', steps: []}+ : {+ name: 'admission',+ status: 'completed',+ conclusion: admission,+ steps: [],+ },+ ...busy(jobs),+ ],+ });+ /** A pull-request run waiting since `beganAt` minutes. */+ const waiter = (id, beganAt) => ({+ id,+ status: 'in_progress',+ event: 'pull_request',+ path: '.github/workflows/ci.yml',+ jobs: [+ {+ name: 'room',+ status: 'in_progress',+ steps: [{name: STEP, status: 'in_progress', started_at: at(beganAt)}],+ },+ ],+ });+ // Far from room all along, even for one job: 7 x 12 + 18 + 1 > 90 + 10.+ const full = prs(7);+ const atLimit = (active, more = {}) =>+ admit({+ active,+ peak: 16,+ phase: 'room',+ maxWait: 12,+ maxLooks: 2,+ ...more,+ });++ it('starts the whole run, once, when no other run started at its longest wait', async () => {+ const result = await atLimit(full);+ expect(result.looks).toBe(2);+ expect(result.warning).toEqual([+ expect.stringContaining(+ 'starting, the one pull-request run to start at its longest wait for 10 minutes',+ ),+ ]);+ // The rest of the run starts without another wait.+ expect(result.outputs).toMatchObject({fits: 'true'});+ expect(result.outputs.full).toBeUndefined();+ });++ it('turns the run away while another started at its longest wait in the last ten minutes', async () => {+ // Run 80 waited twelve minutes and started three minutes before this+ // run's longest wait ends (at 12).+ const result = await atLimit([...full, startedRun(80, 9, 12)]);+ expect(result.outputs.full).toBe('true');+ expect(result.outputs.fits).toBe('false');+ expect(result.warning[0]).toContain(+ 'CI is full: still no room after 12 minutes',+ );+ expect(result.warning[0]).toContain(+ 'run 80 started at its longest wait 3 minutes ago',+ );+ });++ it('starts again once ten minutes have passed since the last start at the longest wait', async () => {+ const result = await atLimit([...full, startedRun(80, 1, 12)]);+ expect(result.outputs.full).toBeUndefined();+ expect(result.outputs.fits).toBe('true');+ });++ it('lets a run that has waited longer start first', async () => {+ const older = await atLimit([...full, waiter(81, -1)]);+ expect(older.outputs.full).toBe('true');+ expect(older.warning[0]).toContain(+ '1 pull-request run that waited longer still waits',+ );+ // A run that began waiting after this one does not hold it back.+ const younger = await atLimit([...full, waiter(82, 0.5)]);+ expect(younger.outputs.full).toBeUndefined();+ expect(younger.outputs.fits).toBe('true');+ });++ it('counts only pull-request runs that waited their longest, and not a fork CI was full for', async () => {+ const result = await atLimit([+ ...full,+ // Found room after six minutes.+ startedRun(83, 9, 6),+ // A run CI was full for: its admission failed, so it never started.+ startedRun(84, 9, 12, {admission: 'failure'}),+ // A main check that waited its longest.+ {...startedRun(85, 9, 15), event: 'workflow_dispatch'},+ ]);+ expect(result.outputs.full).toBeUndefined();+ expect(result.outputs.fits).toBe('true');+ });++ it('ranks a run out of time whose admission has not decided as a waiter, never as a start', async () => {+ // Run 86 began waiting first and ran out of time just before this one;+ // its admission has not decided yet. It goes first; this run is stopped,+ // and the reason names no run that did not start.+ const older = await atLimit([+ ...full,+ startedRun(86, 11.9, 12.1, {admission: null}),+ ]);+ expect(older.outputs.full).toBe('true');+ expect(older.warning[0]).toContain(+ '1 pull-request run that waited longer still waits',+ );+ expect(older.warning[0]).not.toContain('run 86 started');+ // One that began after this run does not hold it back.+ const younger = await atLimit([+ ...full,+ startedRun(87, 12, 11.5, {admission: null}),+ ]);+ expect(younger.outputs.fits).toBe('true');+ });++ it("compares wait starts on GitHub's clock, ties going to the lower run id", async () => {+ // This run's own wait step began 3 s before its script read the clock.+ // A run that began 1 s before the script, after this run's step, is+ // younger, not older.+ const own = {+ id: OWN,+ status: 'completed',+ event: 'pull_request',+ path: '.github/workflows/ci.yml',+ jobs: [+ {+ name: 'room',+ status: 'in_progress',+ steps: [{name: STEP, status: 'in_progress', started_at: at(-0.05)}],+ },+ ],+ };+ const younger = await atLimit([...full, own, waiter(88, -1 / 60)]);+ expect(younger.outputs.fits).toBe('true');+ // The same second: the lower run id waited longer.+ const tieLower = await atLimit([...full, own, {...waiter(0, -0.05)}]);+ expect(tieLower.outputs.full).toBe('true');+ const tieHigher = await atLimit([...full, own, waiter(89, -0.05)]);+ expect(tieHigher.outputs.fits).toBe('true');+ });++ it('never turns away a main run or a merge-queue entry at its longest wait', async () => {+ const recent = [...full, startedRun(80, 12, 12), waiter(81, -5)];+ const mainRun = await admit({+ active: recent,+ peak: 39,+ isMain: true,+ phase: 'room',+ });+ expect(mainRun.outputs.full).toBeUndefined();+ expect(mainRun.warning[0]).toContain('starting.');+ const entry = await admit({+ active: recent,+ peak: 16,+ merge: true,+ phase: 'room',+ maxWait: 12,+ maxLooks: 2,+ });+ expect(entry.outputs.full).toBeUndefined();+ });++ it('decides in the rest of the run too, when the Storybook build found room', async () => {+ const result = await admit({+ active: [...full, startedRun(80, 9, 12)],+ peak: 16,+ phase: 'rest',+ fits: 'false',+ started: String(NOW),+ looked: '1',+ next: String(NOW + 11.67 * 60_000),+ maxWait: 12,+ maxLooks: 2,+ });+ expect(result.outputs.full).toBe('true');+ });++ it('replays 11/02 19:06-19:20Z: ten runs out of time together start one at a time', async () => {+ // Ten pull-request runs began waiting a few seconds apart while a main+ // check waited; all ran out of time within two minutes. Each decides at+ // its own limit, seeing the runs that decided before it.+ const began = Array.from({length: 10}, (_, i) => i * 0.2);+ const decided = [];+ for (const [i, b] of began.entries()) {+ const before = decided+ .map(({id, start, at: endedAt}) =>+ start ? startedRun(id, endedAt - b, 12) : null,+ )+ .filter(Boolean);+ const later = began+ .slice(i + 1)+ .map((c, k) => waiter(200 + i + 1 + k, c - b));+ const result = await atLimit([...full, ...before, ...later]);+ decided.push({+ id: 200 + i,+ start: result.outputs.full !== 'true',+ at: b + 12,+ });+ }+ expect(decided.filter(d => d.start)).toHaveLength(1);+ expect(decided[0].start).toBe(true);+ });+ });+it("keeps room for small workflows' measured most", async () => {expect((await admit({active: [...prs(5), ...slims(18)]})).looks).toBe(1);expect(diff --git a/.github/scripts/ci-heavy-lanes.test.mjs b/.github/scripts/ci-heavy-lanes.test.mjsindex e0b4a7a100..d4bd126f02 100644--- a/.github/scripts/ci-heavy-lanes.test.mjs+++ b/.github/scripts/ci-heavy-lanes.test.mjs@@ -67,7 +67,7 @@ describe('visual, accessibility and RTL owners', () => {const visual = ci.jobs['pr-visual'];const condition = ungated(visual.if);expect(condition).toContain(- "(github.event_name == 'pull_request' &&\n needs.build-storybook.result == 'success')",+ "(github.event_name == 'pull_request' &&\n needs.build-storybook.result == 'success' &&\n needs.admission.outputs.full != 'true')",);expect(visual.needs).toContain('run-labels');const step = name =>diff --git a/.github/scripts/ci-interaction-webkit.test.mjs b/.github/scripts/ci-interaction-webkit.test.mjsindex 06e4e38c8d..168c129da4 100644--- a/.github/scripts/ci-interaction-webkit.test.mjs+++ b/.github/scripts/ci-interaction-webkit.test.mjs@@ -56,7 +56,7 @@ describe('WebKit interaction proof routing', () => {it('always reports, behind the same pull-request gate as the Chromium proofs', () => {expect(webkit.if).toBe(chromium.if);expect(ungated(webkit.if)).toBe(- "github.event_name != 'workflow_dispatch' && always() && !cancelled() && needs.room.outputs.full != 'true'",+ "github.event_name != 'workflow_dispatch' && always() && !cancelled() && needs.room.outputs.full != 'true' && needs.admission.outputs.full != 'true'",);// Admission too: the Storybook build starts beside it (spec:AST-030 FR16).expect(webkit.needs).toEqual([diff --git a/.github/scripts/ci-interaction.test.mjs b/.github/scripts/ci-interaction.test.mjsindex 296967dd64..bc8b5df6ca 100644--- a/.github/scripts/ci-interaction.test.mjs+++ b/.github/scripts/ci-interaction.test.mjs@@ -47,7 +47,7 @@ describe('interaction proof routing', () => {// Behind the pull-request gate (spec:AST-030 FR17).expect(proofs.if).toMatch(/^\$\{\{ \(github\.event_name != 'pull_request'/);expect(ungated(proofs.if)).toBe(- "github.event_name != 'workflow_dispatch' && always() && !cancelled() && needs.room.outputs.full != 'true'",+ "github.event_name != 'workflow_dispatch' && always() && !cancelled() && needs.room.outputs.full != 'true' && needs.admission.outputs.full != 'true'",);// Admission too: the Storybook build starts beside it (spec:AST-030 FR16).expect(proofs.needs).toEqual([diff --git a/.github/scripts/ci-tooling-routing.test.mjs b/.github/scripts/ci-tooling-routing.test.mjsindex ffaac1875a..2fc3450f12 100644--- a/.github/scripts/ci-tooling-routing.test.mjs+++ b/.github/scripts/ci-tooling-routing.test.mjs@@ -443,9 +443,10 @@ describe('Node-tooling CI routing', () => {});it('preserves the historical joins and fails them on owned-lane failure', () => {- // room only tells test when CI is full (ci-admission.test.mjs).+ // room and admission only tell test when CI is full (ci-admission.test.mjs).expect(ci.jobs.test.needs).toEqual(['room',+ 'admission','heavy-ci','release-bump','test-ui',@@ -457,6 +458,7 @@ describe('Node-tooling CI routing', () => {]);expect(ci.jobs.build.needs).toEqual(['room',+ 'admission','heavy-ci','build-storybook','typecheck',diff --git a/.github/scripts/touched-components.test.mjs b/.github/scripts/touched-components.test.mjsindex dde0f7c0dc..31a2cd1e96 100644--- a/.github/scripts/touched-components.test.mjs+++ b/.github/scripts/touched-components.test.mjs@@ -488,9 +488,11 @@ describe('per-component results', () => {describe('pr-touched-components routing', () => {it('runs on every pull request after the Storybook build and pr-visual, on one standard arm runner', () => {- expect(job.needs).toEqual(['build-storybook', 'pr-visual']);+ // Admission too: a run CI is full for at its longest wait has its+ // Storybook build, and builds nothing more (ci-admission.test.mjs).+ expect(job.needs).toEqual(['admission', 'build-storybook', 'pr-visual']);expect(ungated(job.if)).toBe(- "!cancelled() && github.event_name == 'pull_request' && needs.build-storybook.result == 'success' && needs.build-storybook.outputs.lab_only != 'true'",+ "!cancelled() && github.event_name == 'pull_request' && needs.build-storybook.result == 'success' && needs.build-storybook.outputs.lab_only != 'true' && needs.admission.outputs.full != 'true'",);expect(job['runs-on']).toBe('ubuntu-24.04-arm');expect(job['timeout-minutes']).toBeLessThanOrEqual(15);diff --git a/.github/scripts/visual-gate/workflow-concurrency.test.mjs b/.github/scripts/visual-gate/workflow-concurrency.test.mjsindex a83fb51cd4..a633b4371b 100644--- a/.github/scripts/visual-gate/workflow-concurrency.test.mjs+++ b/.github/scripts/visual-gate/workflow-concurrency.test.mjs@@ -95,9 +95,13 @@ describe('PR report and deployment workflow contracts', () => {value.indexOf(' pr-visual:'),value.indexOf(' pr-rtl-shard:'),);- expect(visual).toContain(- '[build-storybook, maintenance-request, visual-scout, run-labels]',- );+ expect(yaml.parse(value).jobs['pr-visual'].needs).toEqual([+ 'admission',+ 'build-storybook',+ 'maintenance-request',+ 'visual-scout',+ 'run-labels',+ ]);expect(visual).toContain('name: storybook-${{ needs.build-storybook.outputs.short_hash }}',);diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.ymlindex 3d6a8ec059..ea8d2eae0f 100644--- a/.github/workflows/ci.yml+++ b/.github/workflows/ci.yml@@ -152,11 +152,13 @@ jobs:permissions:actions: readcontents: read+ outputs:+ full: ${{ steps.rest.outputs.full }}steps:- name: Stop when CI is fullif: needs.room.outputs.full == 'true'run: |- echo "::error title=CI is full::CI is full: re-run all jobs or push later. Ten pull-request runs were already waiting for room."+ echo "::error title=CI is full::CI is full: re-run all jobs or push later. The run found no room under the CI ceiling."exit 1- name: Check out the admission action@@ -167,6 +169,7 @@ jobs:persist-credentials: false- name: Wait for room under the CI ceiling+ id: restuses: ./.ci-admission/.github/actions/ci-admissioncontinue-on-error: truetimeout-minutes: ${{ github.event_name == 'workflow_dispatch' && 16 || 13 }}@@ -182,14 +185,24 @@ jobs:max-looks: ${{ github.event_name == 'workflow_dispatch' && 8 || 2 }}label-legs: '{"ci:run-visual": 6, "ci:run-rtl": 9, "ci:run-a11y": 3}'- # Stops a pull-request run that CI is full for (room found no room and ten- # other pull-request runs waiting), with the reason, so it builds nothing.- # It cancels the run when its token can. A fork's read-only token cannot;- # then admission fails, the jobs behind it skip, and build, test and+ # A pull-request run that waited its longest while another started at+ # its longest wait is turned away (spec:AST-030 DEC-16): ci-full stops+ # the run, and the jobs behind admission skip.+ - name: Stop when CI is full at the longest wait+ if: steps.rest.outputs.full == 'true'+ run: |+ echo "::error title=CI is full::CI is full: re-run all jobs or push later. The run found no room under the CI ceiling."+ exit 1++ # Stops a pull-request run that CI is full for, with the reason, so it builds+ # nothing: room found no room and ten other pull-request runs waiting, or the+ # run waited its longest while another started at its longest wait. It+ # cancels the run when its token can. A fork's read-only token cannot; then+ # admission fails, the jobs behind it skip or stop, and build, test and# docsite-test fail with the same reason, never skip or pass.ci-full:- needs: [room]- if: github.event_name == 'pull_request' && needs.room.outputs.full == 'true'+ needs: [room, admission]+ if: ${{ (github.event_name != 'pull_request' || !github.event.pull_request.draft && (github.event.pull_request.head.repo.full_name == github.repository || contains(github.event.pull_request.labels.*.name, 'ci:run'))) && (github.event_name == 'pull_request' && !cancelled() && (needs.room.outputs.full == 'true' || needs.admission.outputs.full == 'true')) }}runs-on: ubuntu-slimtimeout-minutes: 5permissions:@@ -200,7 +213,7 @@ jobs:env:GH_TOKEN: ${{ github.token }}run: |- echo "::error title=CI is full::CI is full: re-run all jobs or push later. Ten pull-request runs were already waiting for room."+ echo "::error title=CI is full::CI is full: re-run all jobs or push later. The run found no room under the CI ceiling."gh run cancel "$GITHUB_RUN_ID" --repo "$GITHUB_REPOSITORY" || trueexit 1@@ -562,9 +575,9 @@ jobs:# reads the PR's file list and the base's trusted classifier, and# everything is in scope when it cannot decide.- name: Stop when CI is full- if: needs.room.outputs.full == 'true'+ if: needs.room.outputs.full == 'true' || needs.admission.outputs.full == 'true'run: |- echo "::error title=CI is full::CI is full: re-run all jobs or push later. Ten pull-request runs were already waiting for room."+ echo "::error title=CI is full::CI is full: re-run all jobs or push later. The run found no room under the CI ceiling."exit 1# An agent lane's push defers the heavy suites (spec:AST-030 FR20): the@@ -1118,6 +1131,7 @@ jobs:needs:[room,+ admission,heavy-ci,release-bump,test-ui,@@ -1142,9 +1156,9 @@ jobs:# reads the PR's file list and the base's trusted classifier, and# everything is in scope when it cannot decide.- name: Stop when CI is full- if: needs.room.outputs.full == 'true'+ if: needs.room.outputs.full == 'true' || needs.admission.outputs.full == 'true'run: |- echo "::error title=CI is full::CI is full: re-run all jobs or push later. Ten pull-request runs were already waiting for room."+ echo "::error title=CI is full::CI is full: re-run all jobs or push later. The run found no room under the CI ceiling."exit 1# The PR report says why the heavy checks did not run. Recorded here,@@ -1682,7 +1696,7 @@ jobs:# so a type error still fails both.typecheck:needs: [room, admission]- if: ${{ (github.event_name != 'pull_request' || !github.event.pull_request.draft && (github.event.pull_request.head.repo.full_name == github.repository || contains(github.event.pull_request.labels.*.name, 'ci:run'))) && ((github.event_name != 'workflow_dispatch' || inputs.operation == 'release-check') && always() && !cancelled() && needs.room.outputs.full != 'true') }}+ if: ${{ (github.event_name != 'pull_request' || !github.event.pull_request.draft && (github.event.pull_request.head.repo.full_name == github.repository || contains(github.event.pull_request.labels.*.name, 'ci:run'))) && ((github.event_name != 'workflow_dispatch' || inputs.operation == 'release-check') && always() && !cancelled() && needs.room.outputs.full != 'true' && needs.admission.outputs.full != 'true') }}runs-on: ${{ (github.event_name == 'workflow_dispatch' && inputs.operation == 'release-check' && github.triggering_actor != 'github-actions[bot]') && '2-core-ubuntu-arm' || 'ubuntu-24.04-arm' }}timeout-minutes: ${{ github.event_name == 'workflow_dispatch' && 360 || 15 }}permissions:@@ -1754,7 +1768,7 @@ jobs:# gate passes. A skipped build-storybook job (a draft) skips this gate too;# a fork pull request without `ci:run` still reports here.build:- needs: [room, heavy-ci, build-storybook, typecheck]+ needs: [room, admission, heavy-ci, build-storybook, typecheck]if: ${{ (github.event_name != 'pull_request' || !github.event.pull_request.draft) && (github.event_name != 'workflow_dispatch' && !cancelled() && (needs.build-storybook.result != 'skipped' || needs.room.outputs.full == 'true' || needs.heavy-ci.outputs.deferred == 'true' || github.event_name == 'pull_request' && !(github.event.pull_request.head.repo.full_name == github.repository || contains(github.event.pull_request.labels.*.name, 'ci:run')))) }}runs-on: ubuntu-slimpermissions: {}@@ -1765,9 +1779,9 @@ jobs:echo "::error::Waiting for a maintainer's ci:run label: an outside contributor's pull request runs CI once a maintainer adds it (spec:AST-030 FR17)."exit 1- name: Stop when CI is full- if: needs.room.outputs.full == 'true'+ if: needs.room.outputs.full == 'true' || needs.admission.outputs.full == 'true'run: |- echo "::error title=CI is full::CI is full: re-run all jobs or push later. Ten pull-request runs were already waiting for room."+ echo "::error title=CI is full::CI is full: re-run all jobs or push later. The run found no room under the CI ceiling."exit 1# An agent lane's push defers the heavy suites (spec:AST-030 FR20): the@@ -2274,7 +2288,7 @@ jobs:pr-interaction:needs: [room, admission, heavy-ci, build-storybook]- if: ${{ (github.event_name != 'pull_request' || !github.event.pull_request.draft && (github.event.pull_request.head.repo.full_name == github.repository || contains(github.event.pull_request.labels.*.name, 'ci:run'))) && (!cancelled() && needs.heavy-ci.outputs.deferred != 'true' && (github.event_name != 'workflow_dispatch' && always() && !cancelled() && needs.room.outputs.full != 'true')) }}+ if: ${{ (github.event_name != 'pull_request' || !github.event.pull_request.draft && (github.event.pull_request.head.repo.full_name == github.repository || contains(github.event.pull_request.labels.*.name, 'ci:run'))) && (!cancelled() && needs.heavy-ci.outputs.deferred != 'true' && (github.event_name != 'workflow_dispatch' && always() && !cancelled() && needs.room.outputs.full != 'true' && needs.admission.outputs.full != 'true')) }}# Frame evidence measures timing: four cores, one Playwright worker.runs-on: ubuntu-latesttimeout-minutes: 15@@ -2392,7 +2406,7 @@ jobs:# in WebKit.pr-interaction-webkit:needs: [room, admission, heavy-ci, build-storybook]- if: ${{ (github.event_name != 'pull_request' || !github.event.pull_request.draft && (github.event.pull_request.head.repo.full_name == github.repository || contains(github.event.pull_request.labels.*.name, 'ci:run'))) && (!cancelled() && needs.heavy-ci.outputs.deferred != 'true' && (github.event_name != 'workflow_dispatch' && always() && !cancelled() && needs.room.outputs.full != 'true')) }}+ if: ${{ (github.event_name != 'pull_request' || !github.event.pull_request.draft && (github.event.pull_request.head.repo.full_name == github.repository || contains(github.event.pull_request.labels.*.name, 'ci:run'))) && (!cancelled() && needs.heavy-ci.outputs.deferred != 'true' && (github.event_name != 'workflow_dispatch' && always() && !cancelled() && needs.room.outputs.full != 'true' && needs.admission.outputs.full != 'true')) }}# Frame evidence measures timing: four cores, one Playwright worker.runs-on: ubuntu-latesttimeout-minutes: 15@@ -2716,11 +2730,19 @@ jobs:# Audit and interaction fixtures stay with their existing browser checks.pr-visual:name: Stable visual regression- needs: [build-storybook, maintenance-request, visual-scout, run-labels]+ needs:+ [+ admission,+ build-storybook,+ maintenance-request,+ visual-scout,+ run-labels,+ ]if: >-(github.event_name != 'pull_request' || !github.event.pull_request.draft && (github.event.pull_request.head.repo.full_name == github.repository || contains(github.event.pull_request.labels.*.name, 'ci:run'))) && (!cancelled() &&((github.event_name == 'pull_request' &&- needs.build-storybook.result == 'success') ||+ needs.build-storybook.result == 'success' &&+ needs.admission.outputs.full != 'true') ||(github.event_name == 'workflow_dispatch' &&((inputs.operation == 'release-check' &&needs.build-storybook.result == 'success') ||@@ -3475,8 +3497,8 @@ jobs:# pull-request comment. No required context depends on it.pr-touched-components:name: Touched components (visual, accessibility, RTL)- needs: [build-storybook, pr-visual]- if: ${{ (github.event_name != 'pull_request' || !github.event.pull_request.draft && (github.event.pull_request.head.repo.full_name == github.repository || contains(github.event.pull_request.labels.*.name, 'ci:run'))) && (!cancelled() && github.event_name == 'pull_request' && needs.build-storybook.result == 'success' && needs.build-storybook.outputs.lab_only != 'true') }}+ needs: [admission, build-storybook, pr-visual]+ if: ${{ (github.event_name != 'pull_request' || !github.event.pull_request.draft && (github.event.pull_request.head.repo.full_name == github.repository || contains(github.event.pull_request.labels.*.name, 'ci:run'))) && (!cancelled() && github.event_name == 'pull_request' && needs.build-storybook.result == 'success' && needs.build-storybook.outputs.lab_only != 'true' && needs.admission.outputs.full != 'true') }}runs-on: ubuntu-24.04-armtimeout-minutes: 15permissions:diff --git a/docs/specs/AST-030/spec.md b/docs/specs/AST-030/spec.mdindex 7d68eff217..2f0903ead8 100644--- a/docs/specs/AST-030/spec.md+++ b/docs/specs/AST-030/spec.md@@ -362,7 +362,10 @@ they are separately named.jobs of room every four minutes, inside its longest wait. At most tenpull-request runs wait at once: one more that finds no room stops at once with"CI is full: re-run all jobs or push later", and its required checks report- the stopped run. A pull request that may not run CI admits nothing, and+ the stopped run. A pull-request run still without room at its longest wait+ looks once more and starts only if no other pull-request run started at its+ longest wait in the last ten minutes and no run that waited longer still+ waits; otherwise it stops the same way. A pull request that may not run CI admits nothing, andwaiting never fails a job, so every required context reports as before.- **FR17 — A pull request runs CI when it is ready and a maintainer can vouchfor it.** A draft pull request runs no CI, and marking it ready for review@@ -559,6 +562,7 @@ The nightly perf lane is the one scheduled measurement outside them (FR18).| FR14 | `workflow-paths.test.mjs` over every workflow | path allowlist; required-context workflow | a `paths-ignore`, a match-all or exclusion glob, or a path filter on a workflow that reports a required context fails || FR15 | workflow contract and reconciler tests | ENGOWNER spec PR; owner command; other comment; fork review; non-spec PR; run over the request budget; maintainer dispatch; fork PR, code-only: an owner's exact-head approval plus the owner command clears the code half | `review-required` succeeds while spec approval is missing, another workflow publishes a review context, a fork review starts the spec checks, a non-command comment starts a runner, an over-budget run decides, or a fork's approved code-only PR stays pending after the owner command || FR16 | `heavy-caps-action.test.mjs` cap cases and heavy-job contract; `ci-admission.test.mjs` | sixteenth-oldest PR; 101st heavy run of the day, beside cancelled, light and other-branch runs; agent lane and its five-PR and forty-run backstops; an engineering owner, including the shared login past those backstops, a commented-out or trailing-comment handle and an unreadable roster; bot; the deciding run; re-run; failed lookup; a run with room, without room, past its longest wait, beside a waiting `main` check, as a draft or unlabelled fork, with opt-in labels, beside small workflows past their measured most, and with the API unreadable; each workflow's most jobs; every required context's rule | a heavy suite that runs ahead of the author's fifteen oldest PRs, or past the daily cap, a capped job that passes, an engineering owner or a re-run that is held, or a roster read from the pull request's head, a run that starts without room before its longest wait, a pull-request run that starts while `main` waits, a job admission could skip, or a peak below the jobs its run holds fails |+| FR16 | `ci-admission.test.mjs` cases for one start at the longest wait at a time and the workflow contract | a pull-request run at its longest wait alone, within ten minutes of another pull-request run's start there and after them, behind a run that waited longer and ahead of a younger one; another run that found room earlier, a fork's stopped run and a `main` run that waited its longest; a `main` run and a merge-queue entry at their longest wait; the rest of a run whose Storybook build found room; ten runs out of time together; the required contexts and ci-full for either wait | two pull-request runs start at their longest wait within ten minutes, a younger run starts ahead of a longer waiter, a `main` run or merge-queue entry stops, a run CI is full for builds, or a required context passes or skips || FR17 | `ci-pr-gate.test.mjs` over every `pull_request` workflow and the shared conditions | branch here, fork with and without `ci:run`, another label, draft with the label, ready, push, merge group, dispatch | a job that can start on a pull request without the condition, a label trigger, a required job that skips for a pull request that may not run CI, or a draft or unlabelled fork that runs fails || FR18 | `perf-nightly-workflow.test.mjs`, `perf-trend.test.mjs` and `run-labels.test.mjs` | nightly and dispatch; labelled, unlabelled, draft and unvouched-fork pull requests; write, read and no access; newest label event; failed lookup; 10% boundary; metric floors; seven-day window edges; fewer than three runs; a metric over once, twice in a row, or a different metric before; recorded and pull request runs | a run on an unlabelled pull request, a label honoured from read access, a regression called at exactly 10%, under its floor or on one run alone, a run outside the window in the baseline, a pull request run that records, or a second regression issue fails || FR19 | `ci-impact.test.mjs`, `pr-preview.test.mjs` and the maintenance template | CI change and non-CI change; standard, non-standard and expression runners; added, removed and changed jobs; trigger change | a CI change reports no impact, a move onto a non-standard runner or a pool or time-to-green rise past its threshold goes unflagged, or the section names a price |@@ -889,14 +893,23 @@ budget, so `main` fits within fifteen minutes. Nothing is reserved for `main`outside those minutes. At most ten pull-request runs wait at once, countingpull-request runs only; one more that finds no room stops at once, "CI is full:re-run all jobs or push later", rather than add a waiting job to a full CI. A-`main` run is never stopped.+pull-request run still without room at its longest wait looks once more, at a+cost of one more look plus one call for its own jobs, and starts only if no+other pull-request run started at its longest wait in the last ten minutes,+about the time one run takes, and no run that waited longer still waits;+otherwise it stops the same way. A run counts as started there once its+admission job succeeded, and any run that waited out about its whole longest+wait counts, room on its last look or not; until then it ranks as a waiter. Runs that wait out the same burst then start+one at a time instead of together. A `main` run is never stopped.CI can still pass ninety in five bounded ways, the first three rare:- A run that has waited its longest (fifteen minutes for `main`, twelve for a- pull request) starts whatever the load, even while a `main` check waits. That- happens when a job outlives its budget, such as the thirty-minute macOS Swift- job, and the run is over by at most its own jobs until that job ends.+ pull request) starts whatever the load, even while a `main` check waits, at+ most one pull-request run in ten minutes. That happens when a job outlives+ its budget, such as the thirty-minute macOS Swift job, or when a `main` check+ held the waiting runs until their longest waits; CI is over by at most one+ pull-request run's jobs, and each `main` run's, until they end.- Runs that look within the same few seconds can each see room. A random0–20 s on every later look spreads them, and the excess is at most one run'sjobs until its next look.