Skip to content

An open source design system that's fully customizable and agent ready

TypeScript

ci: a pull-request run that finds no room waits instead of stopping at once (#10064)

Cindy Zhang20febb38712f
diff --git a/.github/actions/ci-admission/action.yml b/.github/actions/ci-admission/action.yml
index e0cd5d6d5d..c74706c122 100644
--- a/.github/actions/ci-admission/action.yml
+++ b/.github/actions/ci-admission/action.yml
@@ -13,9 +13,9 @@
# (`phase: whole`). A waiting run so holds only its one waiting job, plus a
# Storybook build that found room. A step that uses this action MUST be named
# "Wait for room under the CI ceiling": a waiting run is found by that name.
-# - A pull-request run that finds no room and ten other pull-request runs
-# already waiting is turned away (`full`): CI is full, and the run stops with
-# that message. So is one still without room at its longest wait, unless it
+# - A pull-request run that finds no room waits like the others, on its slim
+# runner, until its longest wait. One still without room there is turned
+# away (`full`): CI is full, and the run stops with that message, unless it
# is the only pull-request run to start there in ten minutes and no run that
# waited longer still waits. `main` runs are never turned away.
# - A run starts once the other active runs' estimated jobs plus `peak` fit
@@ -174,9 +174,6 @@ runs:
const ended = times('completed_at');
return {began: began.length ? Math.min(...began) : NaN, ended: ended.length ? Math.max(...ended) : NaN};
};
- // At most this many pull-request runs wait at once; one more finding
- // no room is turned away (spec:AST-030 DEC-16).
- const MAX_WAITING = 10;
// A pull-request run yields to a waiting `main` run for the first
// fifteen minutes of that wait. After that the `main` run still counts
// its whole peak, so a pull-request run starts only when it fits
@@ -267,7 +264,6 @@ runs:
let large = 0;
let small = 0;
let mainWaiting = false;
- let waitingPulls = 0;
// Pull-request runs that started at their longest wait lately,
// and those still waiting that began waiting before this one.
const startedAtLimit = [];
@@ -317,7 +313,6 @@ runs:
// which starts beside it.
const holds = isMain(run) ? estimate(run) : Math.max(1, jobs.filter(job => LIVE.includes(job.status)).length);
if (isMain(run) && !main && !(Date.now() - span.began >= MAIN_YIELD_MS)) mainWaiting = true;
- if (run.event === 'pull_request') waitingPulls += 1;
large += holds;
counted.push(`${short(run)} ${run.id} ${run.event}: ${holds} waiting`);
continue;
@@ -351,12 +346,6 @@ runs:
return;
}
if (PHASE === 'room') {
- // Pull requests only: merge-queue entries and `main` runs are never turned away.
- if (pr != null && look === 1 && waitingPulls >= MAX_WAITING) {
- core.setOutput('full', 'true');
- core.warning(`CI is full: ${waitingPulls} pull-request runs are already waiting and there is no room (about ${others} other jobs + ${peak}).`);
- return;
- }
if (others + 1 <= CI_CEILING && !mainWaiting) {
// The rest of the run carries on this wait from this look.
const next = plan(others + peak - CI_CEILING, false);
diff --git a/.github/scripts/ci-admission.test.mjs b/.github/scripts/ci-admission.test.mjs
index 2c9c716970..884a6b8390 100644
--- a/.github/scripts/ci-admission.test.mjs
+++ b/.github/scripts/ci-admission.test.mjs
@@ -1023,8 +1023,9 @@ describe('ci admission', () => {
const waiting = count =>
Array.from({length: count}, (_, i) => waitingRun(300 + i));
- it('turns away a pull-request run that finds no room and ten others waiting', async () => {
- // 5 x 12 + 10 waiting + 18 + 16 = 104: no room, and ten already wait.
+ it('lets an eleventh pull-request run wait instead of turning it away at once', async () => {
+ // 5 x 12 + 10 waiting + 18 + 16 = 104: no room for the run, and ten
+ // already wait, but room for its Storybook build (89 <= 90).
const result = await admit({
active: [...prs(5), ...waiting(10)],
peak: 16,
@@ -1032,25 +1033,13 @@ describe('ci admission', () => {
maxWait: 12,
maxLooks: 2,
});
- expect(result.looks).toBe(1);
- expect(result.outputs.full).toBe('true');
- expect(result.warning[0]).toContain(
- 'CI is full: 10 pull-request runs are already waiting',
- );
- });
-
- it('counts only waiting pull-request runs toward the ten', async () => {
- const queued = Array.from({length: 12}, (_, i) =>
- waitingRun(400 + i, {event: 'merge_group'}),
- );
- const result = await admit({
- active: [...prs(5), ...waiting(9), ...queued],
- peak: 16,
- phase: 'room',
- maxWait: 12,
- maxLooks: 2,
- });
expect(result.outputs.full).toBeUndefined();
+ expect(result.info).toEqual(
+ expect.arrayContaining([
+ expect.stringContaining('Room for the Storybook build'),
+ ]),
+ );
+ expect(result.warning.join('\n')).not.toContain('already waiting');
});
it('never turns away a merge-queue entry', async () => {
@@ -1068,7 +1057,7 @@ describe('ci admission', () => {
expect(entry.looks).toBeGreaterThan(1);
});
- it('lets the tenth waiter wait, and never turns away a run that fits or a main run', async () => {
+ it('lets a waiter wait, and never turns away a run that fits or a main run', async () => {
const ninth = await admit({
active: [...prs(5), ...waiting(9)],
peak: 16,
@@ -1157,6 +1146,18 @@ describe('ci admission', () => {
...more,
});
+ it('still stops a run at its longest wait when more than ten that waited longer still wait', async () => {
+ // The at-once stop at ten waiting runs is gone; the longest-wait stop stays.
+ const longer = Array.from({length: 11}, (_, i) => waiter(90 + i, -1));
+ const result = await atLimit([...full, ...longer]);
+ expect(result.looks).toBe(2);
+ expect(result.outputs.full).toBe('true');
+ expect(result.warning.join('\n')).toContain(
+ 'CI is full: still no room after 12 minutes',
+ );
+ expect(result.warning.join('\n')).not.toContain('already waiting');
+ });
+
it('starts the whole run, once, when no other run started at its longest wait', async () => {
const result = await atLimit(full);
expect(result.looks).toBe(2);
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 8bdd06030a..6134820ee4 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -110,8 +110,8 @@ jobs:
# inside the fifteen-minute job budget.
room:
# Finds room for the run's first job, its Storybook build, and records
- # whether the whole run fit on that look. A pull-request run finding ten
- # others already waiting, and no room, is turned away: CI is full.
+ # whether the whole run fit on that look. A pull-request run that finds no
+ # room waits, up to its longest wait.
if: (github.event_name != 'pull_request' || !github.event.pull_request.draft && (github.event.pull_request.head.repo.full_name == github.repository || contains(github.event.pull_request.labels.*.name, 'ci:run')))
# A dispatch (a main or release check) may wait longer than a slim job's
# fifteen-minute cap, so it waits on a standard runner (spec:AST-030 DEC-16).
@@ -200,9 +200,9 @@ jobs:
exit 1
# Stops a pull-request run that CI is full for, with the reason, so it builds
- # nothing: room found no room and ten other pull-request runs waiting, or the
- # run waited its longest while another started at its longest wait. It
- # cancels the run when its token can. A fork's read-only token cannot; then
+ # nothing: the run waited its longest while another started at its longest
+ # wait, or a run that waited longer still waits. It cancels the run when its
+ # token can. A fork's read-only token cannot; then
# admission fails, the jobs behind it skip or stop, and build, test and
# docsite-test fail with the same reason, never skip or pass.
ci-full:
diff --git a/docs/specs/AST-030/spec.md b/docs/specs/AST-030/spec.md
index 2ed811a8ab..d09a44d4cc 100644
--- a/docs/specs/AST-030/spec.md
+++ b/docs/specs/AST-030/spec.md
@@ -397,13 +397,14 @@ they are separately named.
request, at once and after twelve minutes, and about every four minutes for
up to two hours for `main`, whose wait runs on a standard runner (DEC-16); a pull-request run that would fit but waits for a
`main` check looks again every two minutes while it does, and a run within ten
- jobs of room every four minutes, inside its longest wait. At most ten
- pull-request runs wait at once: one more that finds no room stops at once with
- "CI is full: re-run all jobs or push later", and its required checks report
- the stopped run. A pull-request run still without room at its longest wait
- looks once more and starts only if no other pull-request run started at its
- longest wait in the last ten minutes and no run that waited longer still
- waits; otherwise it stops the same way. A pull request that may not run CI admits nothing, and
+ jobs of room every four minutes, inside its longest wait. A pull-request run
+ that finds no room waits like the others, on its slim runner, until its
+ longest wait (12 minutes); waiting runs count only what they run: their
+ waiting job and a Storybook build that found room. A pull-request run still without room at its
+ longest wait looks once more and starts only if no other pull-request run
+ started at its longest wait in the last ten minutes and no run that waited
+ longer still waits; otherwise it stops with "CI is full: re-run all jobs or
+ push later", and its required checks report the stopped run. A pull request that may not run CI admits nothing, and
waiting never fails a job, so every required context reports as before.
- **FR17 — A pull request runs CI when it is ready and a maintainer can vouch
for it.** A draft pull request runs no CI, and marking it ready for review
@@ -629,7 +630,7 @@ The nightly perf lane is the one scheduled measurement outside them (FR18).
| FR14 | `workflow-paths.test.mjs` over every workflow | path allowlist; required-context workflow | a `paths-ignore`, a match-all or exclusion glob, or a path filter on a workflow that reports a required context fails |
| FR15 | workflow contract and reconciler tests | ENGOWNER spec PR; owner command; other comment; fork review; non-spec PR; run over the request budget; maintainer dispatch; fork PR, code-only: an owner's exact-head approval plus the owner command clears the code half | `review-required` succeeds while spec approval is missing, another workflow publishes a review context, a fork review starts the spec checks, a non-command comment starts a runner, an over-budget run decides, or a fork's approved code-only PR stays pending after the owner command |
| FR16 | `heavy-caps-action.test.mjs` cap cases and heavy-job contract; `ci-admission.test.mjs` | sixteenth-oldest PR; 101st heavy run of the day, beside cancelled, light and other-branch runs; agent lane and its five-PR and forty-run backstops; an engineering owner, including the shared login past those backstops, a commented-out or trailing-comment handle and an unreadable roster; bot; the deciding run; re-run; failed lookup; a run with room, without room, past its longest wait, beside a waiting `main` check, as a draft or unlabelled fork, with opt-in labels, beside small workflows past their measured most, and with the API unreadable; each workflow's most jobs; every required context's rule | a heavy suite that runs ahead of the author's fifteen oldest PRs, or past the daily cap, a capped job that passes, an engineering owner or a re-run that is held, or a roster read from the pull request's head, a run that starts without room before its longest wait, a pull-request run that starts while `main` has waited under fifteen minutes, a job admission could skip, a peak below the jobs its run holds, or a job on a slim runner planned past its fifteen-minute cap fails |
-| FR16 | `ci-admission.test.mjs` cases for one start at the longest wait at a time and the workflow contract | a pull-request run at its longest wait alone, within ten minutes of another pull-request run's start there and after them, behind a run that waited longer and ahead of a younger one; another run that found room earlier, a fork's stopped run and a `main` run that waited its longest; a `main` run and a merge-queue entry at their longest wait; the rest of a run whose Storybook build found room; ten runs out of time together; the required contexts and ci-full for either wait | two pull-request runs start at their longest wait within ten minutes, a younger run starts ahead of a longer waiter, a `main` run or merge-queue entry stops, a run CI is full for builds, or a required context passes or skips |
+| FR16 | `ci-admission.test.mjs` cases for one start at the longest wait at a time and the workflow contract | a pull-request run at its longest wait alone, within ten minutes of another pull-request run's start there and after them, behind a run that waited longer and ahead of a younger one; another run that found room earlier, a fork's stopped run and a `main` run that waited its longest; a `main` run and a merge-queue entry at their longest wait; the rest of a run whose Storybook build found room; ten runs out of time together; the required contexts and ci-full for either wait; a pull-request run that finds ten others waiting and room only for its Storybook build, and one at its longest wait behind more than ten that waited longer | two pull-request runs start at their longest wait within ten minutes, a younger run starts ahead of a longer waiter, a `main` run or merge-queue entry stops, a run CI is full for builds, or a required context passes or skips |
| FR17 | `ci-pr-gate.test.mjs` over every `pull_request` workflow and the shared conditions | branch here, fork with and without `ci:run`, another label, draft with the label, ready, push, merge group, dispatch | a job that can start on a pull request without the condition, a label trigger, a required job that skips for a pull request that may not run CI, or a draft or unlabelled fork that runs fails |
| FR18 | `perf-nightly-workflow.test.mjs`, `perf-trend.test.mjs` and `run-labels.test.mjs` | nightly and dispatch; labelled, unlabelled, draft and unvouched-fork pull requests; write, read and no access; newest label event; failed lookup; 10% boundary; metric floors; seven-day window edges; fewer than three runs; a metric over once, twice in a row, or a different metric before; recorded and pull request runs | a run on an unlabelled pull request, a label honoured from read access, a regression called at exactly 10%, under its floor or on one run alone, a run outside the window in the baseline, a pull request run that records, or a second regression issue fails |
| FR19 | `ci-impact.test.mjs`, `pr-preview.test.mjs` and the maintenance template | CI change and non-CI change; standard, non-standard and expression runners; added, removed and changed jobs; trigger change | a CI change reports no impact, a move onto a non-standard runner or a pool or time-to-green rise past its threshold goes unflagged, or the section names a price |
@@ -966,9 +967,9 @@ CI is near full or `main` waits. Every pull-request job ends within its
fifteen-minute budget, so `main` usually fits within fifteen minutes. A `main`
run still waits until there is room, up to two hours, and its waiting job runs
on a standard runner: a slim runner stops a job at fifteen minutes and so cancels the whole run. Nothing is
-reserved for `main` outside its wait. At most ten pull-request runs wait at once, counting
-pull-request runs only; one more that finds no room stops at once, "CI is full:
-re-run all jobs or push later", rather than add a waiting job to a full CI. A
+reserved for `main` outside its wait. A pull-request run that finds no room
+waits like the others, on its slim runner, until its longest wait: a waiting slim
+job costs less than a stopped run that someone has to start again. A
pull-request run still without room at its longest wait looks once more, at a
cost of one more look plus one call for its own jobs, and starts only if no
other pull-request run started at its longest wait in the last ten minutes,
@@ -993,9 +994,9 @@ CI can still pass ninety in five bounded ways, the first three rare:
over by at most its own jobs.
- A waiting run's own waiting job starts before it finds room; its Storybook
build starts only once there is room for it. CI is over by at most one
- waiting job per waiting run: at most ten for pull requests, plus one for each
- waiting merge-queue entry and `main` run, which are never turned away, for at
- most a wait (twelve minutes for a pull request, two hours for `main`).
+ waiting job per waiting run, pull requests, merge-queue entries and `main`
+ runs alike, with no limit on how many wait, for at most a wait (twelve
+ minutes for a pull request, two hours for `main`).
- Small workflows can run more jobs at once than the eighteen counted for them:
they peaked at twenty in a day's runs, two over, while their jobs ran.