Skip to content

An open source design system that's fully customizable and agent ready

TypeScript

ci: one shared count of CI's jobs for admission and the tools that grant capacity (#10422)

Cindy Zhangb3f8d8ff8195
diff --git a/.github/actions/ci-admission/action.yml b/.github/actions/ci-admission/action.yml
index 093a16565c..2106fff459 100644
--- a/.github/actions/ci-admission/action.yml
+++ b/.github/actions/ci-admission/action.yml
@@ -128,13 +128,16 @@ runs:
LABEL_LEGS: ${{ inputs.label-legs }}
MAX_WAIT_MINUTES: ${{ inputs.max-wait-minutes }}
MAX_LOOKS: ${{ inputs.max-looks }}
+ COUNT_MODULE: ${{ github.action_path }}/count.mjs
with:
retries: 3
script: |
- const CI_CEILING = 90;
- // Below this many API calls left this hour, a run starts rather than
- // spend the budget every job shares.
- const BUDGET_FLOOR = 1500;
+ // The count itself is shared with the tools that grant CI capacity,
+ // so both read CI the same way (count.mjs).
+ const {pathToFileURL} = await import('node:url');
+ const {CI_CEILING, STEP, AT_LIMIT_SPACING_MS, waitSpan, countLook} = await import(
+ pathToFileURL(process.env.COUNT_MODULE).href
+ );
const JITTER_MS = 20_000;
const MAX_WAIT_MS = Number(process.env.MAX_WAIT_MINUTES || '15') * 60_000;
const MAX_LOOKS = Math.max(1, Number(process.env.MAX_LOOKS || '8'));
@@ -156,7 +159,6 @@ runs:
if (over <= NEAR_JOBS) return {spends: false, interval: Math.min(NEAR_LOOK_MS, LOOK_MS)};
return {spends: true, interval: LOOK_MS};
};
- const STEP = 'Wait for room under the CI ceiling';
// A pull-request run that reaches its longest wait without room
// starts only if no other pull-request run started at its longest
// wait in the last ten minutes, about the time one run takes, and
@@ -164,53 +166,7 @@ runs:
// for it. So runs that waited out the same burst start one at a
// time, and CI passes ninety by about one run's jobs at most
// (spec:AST-030 DEC-16).
- const AT_LIMIT_SPACING_MS = 10 * 60_000;
- // A wait this close to the longest wait counts as one that ran out.
- const AT_LIMIT_SLACK_MS = 30_000;
- const isPullRun = run => run.event === 'pull_request' && run.path === '.github/workflows/ci.yml';
- // When a run's wait began and, once admitted, ended: its first wait
- // step's start and its last wait step's end.
- const waitSpan = waits => {
- const times = key => waits.map(step => Date.parse(step[key] ?? '')).filter(Number.isFinite);
- const began = times('started_at');
- const ended = times('completed_at');
- return {began: began.length ? Math.min(...began) : NaN, ended: ended.length ? Math.max(...ended) : NaN};
- };
- // A pull-request run yields to a waiting `main` run for the first
- // fifteen minutes of that wait. After that the `main` run still counts
- // its whole peak, so a pull-request run starts only when it fits
- // beside it (spec:AST-030 DEC-16).
- const MAIN_YIELD_MS = 15 * 60_000;
const PHASE = process.env.PHASE || 'whole';
- // The most jobs a run holds at once, by workflow: ci.yml's
- // release-check and Main's runs (a push's fast checks or a unit run)
- // from their `peak`
- // (ci-admission.test.mjs), a pull-request ci.yml run from its
- // measured most, every other workflow's run one or two slim jobs.
- const ESTIMATE = {
- '.github/workflows/ci.yml': run => (run.event === 'workflow_dispatch' ? 39 : 16),
- '.github/workflows/deploy.yml': run => (run.event === 'push' ? 4 : 14),
- '.github/workflows/review-signal.yml': () => 2,
- };
- const estimate = run => (ESTIMATE[run.path] ?? (() => 1))(run);
- // Jobs always counted for small, unadmitted workflows: they peaked at
- // twenty at once in a day's runs and rose by fourteen within two
- // minutes in ninety-nine looks of a hundred (spec:AST-030 DEC-16).
- const SMALL_MOST = 18;
- const ADMITTED = ['.github/workflows/ci.yml', '.github/workflows/deploy.yml'];
- const admitted = run => ADMITTED.includes(run.path);
- const LIVE = ['in_progress', 'queued', 'waiting', 'pending'];
- // A run's jobs appear within seconds of its admission; until then it
- // counts at its planned most.
- const JUST_ADMITTED_MS = 45_000;
- // GitHub keeps listing some long-finished runs as queued; a run not
- // updated in this long holds no runner.
- const STALE_MS = 2 * 60 * 60_000;
- const short = run => (run.path ?? '').replace(/^.*\//, '').replace(/\.ya?ml$/, '');
- const isMain = run =>
- ['push', 'schedule', 'workflow_dispatch'].includes(run.event) &&
- ['.github/workflows/ci.yml', '.github/workflows/deploy.yml'].includes(run.path);
-
let peak = Number(process.env.PEAK);
if (!Number.isInteger(peak) || peak < 1) {
throw new Error(`peak must be a positive integer, got "${process.env.PEAK}".`);
@@ -282,91 +238,34 @@ runs:
// `fits` unset, so the rest starts rather than wait on an older look.
core.setOutput('fits', '');
core.setOutput('looks', String(spent + 1));
- const seen = new Set();
- let large = 0;
- let small = 0;
- let mainWaiting = false;
- // Pull-request runs that started at their longest wait lately,
- // and those still waiting that began waiting before this one.
- const startedAtLimit = [];
- // Pull-request runs still waiting, or out of time and not yet
- // decided, by when their wait began.
- const pullWaiters = [];
- // What each run counted, logged on every look.
- const counted = [];
- const smallBy = {};
- let stale = 0;
- for (const status of ['in_progress', 'queued']) {
- calls += 1;
- const {data, headers} = await github.rest.actions.listWorkflowRunsForRepo({
- ...context.repo, status, per_page: 100,
- });
- const left = Number(headers?.['x-ratelimit-remaining']);
- if (Number.isFinite(left) && left < BUDGET_FLOOR) {
- core.warning(`Only ${left} API calls left this hour; starting without spending them.`);
- return;
- }
- for (const run of data.workflow_runs) {
- if (seen.has(run.id) || run.id === context.runId) continue;
- seen.add(run.id);
- if (Date.now() - Date.parse(run.updated_at ?? '') > STALE_MS) {
- stale += 1;
- continue;
- }
- if (!admitted(run)) {
- small += estimate(run);
- smallBy[short(run)] = (smallBy[short(run)] ?? 0) + estimate(run);
- continue;
- }
- calls += 1;
+ const lookAt = await countLook({
+ listRuns: async status => {
+ const {data, headers} = await github.rest.actions.listWorkflowRunsForRepo({
+ ...context.repo, status, per_page: 100,
+ });
+ return {runs: data.workflow_runs, remaining: Number(headers?.['x-ratelimit-remaining'])};
+ },
+ listJobs: async run_id => {
const {data: {jobs}} = await github.rest.actions.listJobsForWorkflowRun({
- ...context.repo, run_id: run.id, filter: 'latest', per_page: 100,
+ ...context.repo, run_id, filter: 'latest', per_page: 100,
});
- // A CI run waits in `room`, then in `admission`.
- const waits = jobs
- .flatMap(job => job.steps ?? [])
- .filter(step => step.name === STEP);
- const span = waitSpan(waits);
- if (waits.some(step => step.status === 'in_progress')) {
- if (isPullRun(run)) pullWaiters.push({id: run.id, began: span.began});
- // Still waiting for room. A `main` run holds its whole
- // peak, ahead of this one. A pull-request run holds what
- // it runs: the job that waits and its Storybook build,
- // which starts beside it. `main` runs queue by when they
- // began waiting, both read on GitHub's clock, a tie going
- // to the lower run id: a `main` run that began after this
- // one holds only what it runs, so of two waiting `main`
- // runs exactly one counts the other's peak.
- const mine = main && isMain(run) ? await ownWaitStart() : NaN;
- const ahead = !main || !(span.began > mine || (span.began === mine && run.id > context.runId));
- const holds = isMain(run) && ahead ? estimate(run) : Math.max(1, jobs.filter(job => LIVE.includes(job.status)).length);
- if (isMain(run) && !main && !(Date.now() - span.began >= MAIN_YIELD_MS)) mainWaiting = true;
- large += holds;
- counted.push(`${short(run)} ${run.id} ${run.event}: ${holds} waiting`);
- continue;
- }
- if (isPullRun(run) && span.ended - span.began >= MAX_WAIT_MS - AT_LIMIT_SLACK_MS) {
- // Out of time: it started only once its admission job
- // succeeded. Until admission decides, it ranks as a waiter;
- // a run CI was full for never started.
- const decided = jobs.find(job => job.name === 'admission' && job.status === 'completed');
- if (decided == null) pullWaiters.push({id: run.id, began: span.began});
- else if (decided.conclusion === 'success' && Date.now() - span.ended < AT_LIMIT_SPACING_MS) {
- startedAtLimit.push({id: run.id, ago: Date.now() - span.ended});
- }
- }
- const live = jobs.filter(job => LIVE.includes(job.status)).length;
- const ended = waits.map(step => Date.parse(step.completed_at ?? '')).filter(Number.isFinite);
- const admittedAt = ended.length > 0 ? Math.max(...ended) : NaN;
- const fresh = Date.now() - admittedAt < JUST_ADMITTED_MS;
- const holds = fresh ? Math.max(live, estimate(run)) : live;
- large += holds;
- counted.push(`${short(run)} ${run.id} ${run.event}: ${holds}${fresh && holds > live ? ` (admitted just now, ${live} live)` : ' live'}`);
- }
+ return jobs;
+ },
+ selfId: context.runId,
+ main,
+ maxWaitMs: MAX_WAIT_MS,
+ ownWaitStart,
+ });
+ calls += lookAt.calls;
+ if (lookAt.lowBudget != null) {
+ core.warning(`Only ${lookAt.lowBudget} API calls left this hour; starting without spending them.`);
+ return;
}
- const others = large + Math.max(small, SMALL_MOST);
+ const {others, small, smallBy, stale, counted, mainWaiting, pullWaiters, startedAtLimit} = lookAt;
+ // Small workflows count at least their usual most (count.mjs).
+ const smallCounted = others - lookAt.large;
const smallList = Object.entries(smallBy).map(([name, count]) => `${name} ${count}`).join(', ');
- core.info(`Look ${look}: ${counted.join('; ') || 'no CI or Main runs'}. Small workflows ${small}${smallList ? ` (${smallList})` : ''}, counted as ${Math.max(small, SMALL_MOST)}.${stale ? ` Ignored ${stale} stale run${stale === 1 ? '' : 's'}.` : ''}`);
+ core.info(`Look ${look}: ${counted.join('; ') || 'no CI or Main runs'}. Small workflows ${small}${smallList ? ` (${smallList})` : ''}, counted as ${smallCounted}.${stale ? ` Ignored ${stale} stale run${stale === 1 ? '' : 's'}.` : ''}`);
const fits = others + peak <= CI_CEILING && !mainWaiting;
core.setOutput('fits', String(fits));
if (fits) {
diff --git a/.github/actions/ci-admission/count.mjs b/.github/actions/ci-admission/count.mjs
new file mode 100644
index 0000000000..2419c20000
--- /dev/null
+++ b/.github/actions/ci-admission/count.mjs
@@ -0,0 +1,330 @@
+// Copyright (c) Meta Platforms, Inc. and affiliates.
+
+/**
+ * @file How many jobs CI holds at one look, read the way the admission action
+ * reads it (spec:AST-030 FR16, DEC-16).
+ * @input Two run listings (in progress, queued) and each active CI or Main
+ * run's jobs, through functions the caller passes in.
+ * @output The other runs' jobs, whether a `main` check waits, the waiting and
+ * recently started pull-request runs, and a log of what each run counted.
+ * @position The admission action (action.yml) imports it on every look, and
+ * a tool that grants CI capacity can run it as a command (`node count.mjs
+ * --repo <owner/name> --pr <number> [--reserved <jobs>]`), so both count CI
+ * the same way. Node built-ins only: the action's checkout holds this
+ * folder alone.
+ */
+
+import {execFileSync} from 'node:child_process';
+import fs from 'node:fs';
+import {fileURLToPath} from 'node:url';
+
+export const CI_CEILING = 90;
+// Below this many API calls left this hour, a run starts rather than spend
+// the budget every job shares.
+export const BUDGET_FLOOR = 1500;
+// A step that uses the admission action is named this; a waiting run is found
+// by it.
+export const STEP = 'Wait for room under the CI ceiling';
+// A pull-request run yields to a waiting `main` run for the first fifteen
+// minutes of that wait. After that the `main` run still counts its whole
+// peak, so a pull-request run starts only when it fits beside it
+// (spec:AST-030 DEC-16).
+export const MAIN_YIELD_MS = 15 * 60_000;
+// A pull-request run that reaches its longest wait without room starts only
+// if no other pull-request run started at its longest wait in the last ten
+// minutes, about the time one run takes, and no run that has waited longer
+// still waits.
+export const AT_LIMIT_SPACING_MS = 10 * 60_000;
+// A wait this close to the longest wait counts as one that ran out.
+export const AT_LIMIT_SLACK_MS = 30_000;
+// The most jobs a run holds at once, by workflow: ci.yml's release-check and
+// Main's runs (a push's fast checks or a unit run) from their `peak`
+// (ci-admission.test.mjs), a pull-request ci.yml run from its measured most,
+// every other workflow's run one or two slim jobs.
+// One line per workflow: ci-admission.test.mjs reads each estimate from it.
+// prettier-ignore
+const ESTIMATE = {
+ '.github/workflows/ci.yml': run => (run.event === 'workflow_dispatch' ? 39 : 16),
+ '.github/workflows/deploy.yml': run => (run.event === 'push' ? 4 : 14),
+ '.github/workflows/review-signal.yml': () => 2,
+};
+export const estimate = run => (ESTIMATE[run.path] ?? (() => 1))(run);
+// Jobs always counted for small, unadmitted workflows: they peaked at twenty
+// at once in a day's runs and rose by fourteen within two minutes in
+// ninety-nine looks of a hundred (spec:AST-030 DEC-16).
+export const SMALL_MOST = 18;
+const ADMITTED = ['.github/workflows/ci.yml', '.github/workflows/deploy.yml'];
+const admitted = run => ADMITTED.includes(run.path);
+const LIVE = ['in_progress', 'queued', 'waiting', 'pending'];
+// A run's jobs appear within seconds of its admission; until then it counts at
+// its planned most.
+const JUST_ADMITTED_MS = 45_000;
+// GitHub keeps listing some long-finished runs as queued; a run not updated in
+// this long holds no runner.
+const STALE_MS = 2 * 60 * 60_000;
+export const short = run =>
+ (run.path ?? '').replace(/^.*\//, '').replace(/\.ya?ml$/, '');
+export const isPullRun = run =>
+ run.event === 'pull_request' && run.path === '.github/workflows/ci.yml';
+export const isMain = run =>
+ ['push', 'schedule', 'workflow_dispatch'].includes(run.event) &&
+ ['.github/workflows/ci.yml', '.github/workflows/deploy.yml'].includes(
+ run.path,
+ );
+
+/**
+ * When a run's wait began and, once admitted, ended: its first wait step's
+ * start and its last wait step's end.
+ */
+export const waitSpan = waits => {
+ const times = key =>
+ waits.map(step => Date.parse(step[key] ?? '')).filter(Number.isFinite);
+ const began = times('started_at');
+ const ended = times('completed_at');
+ return {
+ began: began.length ? Math.min(...began) : NaN,
+ ended: ended.length ? Math.max(...ended) : NaN,
+ };
+};
+
+/**
+ * One look at CI.
+ * @param {object} options
+ * @param {(status: string) => Promise<{runs: object[], remaining?: number}>} options.listRuns
+ * @param {(runId: number) => Promise<object[]>} options.listJobs
+ * @param {number | null} options.selfId this run's id, left out of the count
+ * @param {boolean} options.main whether the looking run is a `main` check
+ * @param {number} options.maxWaitMs a pull-request run's longest wait
+ * @param {() => Promise<number>} [options.ownWaitStart] when the looking `main`
+ * run began waiting, on GitHub's clock
+ * @returns {Promise<{lowBudget?: number, others: number, large: number, small: number, smallBy: Record<string, number>, mainWaiting: boolean, pullWaiters: {id: number, began: number}[], startedAtLimit: {id: number, ago: number}[], counted: string[], stale: number, calls: number}>}
+ */
+export async function countLook({
+ listRuns,
+ listJobs,
+ selfId,
+ main,
+ maxWaitMs,
+ ownWaitStart = async () => NaN,
+}) {
+ const seen = new Set();
+ let large = 0;
+ let small = 0;
+ let mainWaiting = false;
+ let calls = 0;
+ // Pull-request runs that started at their longest wait lately, and those
+ // still waiting, by when their wait began.
+ const startedAtLimit = [];
+ const pullWaiters = [];
+ // What each run counted, logged on every look.
+ const counted = [];
+ const smallBy = {};
+ let stale = 0;
+ for (const status of ['in_progress', 'queued']) {
+ calls += 1;
+ const {runs, remaining} = await listRuns(status);
+ if (Number.isFinite(remaining) && remaining < BUDGET_FLOOR) {
+ return {lowBudget: remaining, calls};
+ }
+ for (const run of runs) {
+ if (seen.has(run.id) || run.id === selfId) continue;
+ seen.add(run.id);
+ if (Date.now() - Date.parse(run.updated_at ?? '') > STALE_MS) {
+ stale += 1;
+ continue;
+ }
+ if (!admitted(run)) {
+ small += estimate(run);
+ smallBy[short(run)] = (smallBy[short(run)] ?? 0) + estimate(run);
+ continue;
+ }
+ calls += 1;
+ const jobs = await listJobs(run.id);
+ // A CI run waits in `room`, then in `admission`.
+ const waits = jobs
+ .flatMap(job => job.steps ?? [])
+ .filter(step => step.name === STEP);
+ const span = waitSpan(waits);
+ if (waits.some(step => step.status === 'in_progress')) {
+ if (isPullRun(run)) pullWaiters.push({id: run.id, began: span.began});
+ // Still waiting for room. A `main` run holds its whole peak, ahead of
+ // this one. A pull-request run holds what it runs: the job that waits
+ // and its Storybook build, which starts beside it. `main` runs queue
+ // by when they began waiting, both read on GitHub's clock, a tie going
+ // to the lower run id: a `main` run that began after this one holds
+ // only what it runs, so of two waiting `main` runs exactly one counts
+ // the other's peak.
+ const mine = main && isMain(run) ? await ownWaitStart() : NaN;
+ const ahead =
+ !main ||
+ !(span.began > mine || (span.began === mine && run.id > selfId));
+ const holds =
+ isMain(run) && ahead
+ ? estimate(run)
+ : Math.max(1, jobs.filter(job => LIVE.includes(job.status)).length);
+ if (isMain(run) && !main && !(Date.now() - span.began >= MAIN_YIELD_MS))
+ mainWaiting = true;
+ large += holds;
+ counted.push(`${short(run)} ${run.id} ${run.event}: ${holds} waiting`);
+ continue;
+ }
+ if (
+ isPullRun(run) &&
+ span.ended - span.began >= maxWaitMs - AT_LIMIT_SLACK_MS
+ ) {
+ // Out of time: it started only once its admission job succeeded.
+ // Until admission decides, it ranks as a waiter; a run CI was full
+ // for never started.
+ const decided = jobs.find(
+ job => job.name === 'admission' && job.status === 'completed',
+ );
+ if (decided == null) pullWaiters.push({id: run.id, began: span.began});
+ else if (
+ decided.conclusion === 'success' &&
+ Date.now() - span.ended < AT_LIMIT_SPACING_MS
+ ) {
+ startedAtLimit.push({id: run.id, ago: Date.now() - span.ended});
+ }
+ }
+ const live = jobs.filter(job => LIVE.includes(job.status)).length;
+ const ended = waits
+ .map(step => Date.parse(step.completed_at ?? ''))
+ .filter(Number.isFinite);
+ const admittedAt = ended.length > 0 ? Math.max(...ended) : NaN;
+ const fresh = Date.now() - admittedAt < JUST_ADMITTED_MS;
+ const holds = fresh ? Math.max(live, estimate(run)) : live;
+ large += holds;
+ counted.push(
+ `${short(run)} ${run.id} ${run.event}: ${holds}${fresh && holds > live ? ` (admitted just now, ${live} live)` : ' live'}`,
+ );
+ }
+ }
+ return {
+ others: large + Math.max(small, SMALL_MOST),
+ large,
+ small,
+ smallBy,
+ mainWaiting,
+ pullWaiters,
+ startedAtLimit,
+ counted,
+ stale,
+ calls,
+ };
+}
+
+function ghJson(path) {
+ return JSON.parse(
+ execFileSync('gh', ['api', path], {
+ encoding: 'utf8',
+ maxBuffer: 64 * 1024 * 1024,
+ }),
+ );
+}
+
+/**
+ * A pull-request run's peak as ci.yml's admission step declares it: its
+ * pull-request `peak` plus the jobs each of the pull request's labels adds
+ * (`label-legs`). Read from the workflow itself, so the sizes live in one
+ * place.
+ * @param {string} workflow ci.yml's text
+ * @param {string[]} labels
+ */
+export function pullRequestPeak(workflow, labels) {
+ const peak =
+ /peak: \$\{\{ github\.event_name == 'workflow_dispatch' && \d+ \|\| (\d+) \}\}/.exec(
+ workflow,
+ );
+ const legs = /label-legs: '([^']*)'/.exec(workflow);
+ if (!peak || !legs)
+ throw new Error("ci.yml's admission step has no peak or label-legs.");
+ const extra = JSON.parse(legs[1]);
+ return labels.reduce(
+ (total, label) => total + (extra[label] ?? 0),
+ Number(peak[1]),
+ );
+}
+
+/**
+ * The command a capacity tool runs: one look, read through `gh api`, as a
+ * pull-request run sees CI. `--pr` takes the run's peak from that pull
+ * request's labels and ci.yml; `--peak` sets it instead; `--main` looks as a
+ * `main` check. `--reserved` adds jobs the tool has granted that do not show
+ * yet, and `--run` leaves out the pull request's own run if it already waits. It prints one JSON line and exits 0, or 2 when a lookup fails, which
+ * the tool reads as no room.
+ */
+async function main() {
+ const argv = process.argv.slice(2);
+ const value = name => {
+ const at = argv.indexOf(name);
+ return at === -1 ? undefined : argv[at + 1];
+ };
+ const repo = value('--repo');
+ const pr = value('--pr');
+ const reserved = Number(value('--reserved') ?? 0);
+ // The pull request's own CI run, if it already waits: left out, as
+ // admission leaves itself out, so its jobs are not counted twice.
+ const runId = value('--run') == null ? null : Number(value('--run'));
+ const usage = () => {
+ console.error(
+ 'usage: count.mjs --repo <owner/name> (--pr <number> | --peak <jobs>) [--reserved <jobs>] [--run <id>] [--main]',
+ );
+ process.exit(2);
+ };
+ if (!repo || !Number.isInteger(reserved) || reserved < 0) usage();
+ const isMainCheck = argv.includes('--main');
+ try {
+ let peak = Number(value('--peak'));
+ // Every API call this command makes, the labels read included.
+ let labelCalls = 0;
+ if (value('--peak') == null && pr != null) {
+ labelCalls = 1;
+ const workflow = fs.readFileSync(
+ fileURLToPath(new URL('../../workflows/ci.yml', import.meta.url)),
+ 'utf8',
+ );
+ const labels = ghJson(`repos/${repo}/pulls/${pr}`).labels.map(
+ label => label.name,
+ );
+ peak = pullRequestPeak(workflow, labels);
+ }
+ if (!Number.isInteger(peak) || peak < 1) usage();
+ const look = await countLook({
+ listRuns: async status => ({
+ runs: ghJson(`repos/${repo}/actions/runs?status=${status}&per_page=100`)
+ .workflow_runs,
+ }),
+ listJobs: async id =>
+ ghJson(
+ `repos/${repo}/actions/runs/${id}/jobs?filter=latest&per_page=100`,
+ ).jobs,
+ selfId: runId,
+ main: isMainCheck,
+ maxWaitMs: (isMainCheck ? 120 : 12) * 60_000,
+ });
+ const others = look.others + reserved;
+ console.log(
+ JSON.stringify({
+ others,
+ reserved,
+ peak,
+ ceiling: CI_CEILING,
+ fits: others + peak <= CI_CEILING && (isMainCheck || !look.mainWaiting),
+ mainWaiting: look.mainWaiting,
+ calls: look.calls + labelCalls,
+ counted: look.counted,
+ }),
+ );
+ } catch (error) {
+ console.error(`CI could not be counted: ${error.message}`);
+ process.exit(2);
+ }
+}
+
+// Real paths on both sides: Node keeps a symlinked path as typed in argv but
+// resolves the module's own URL.
+if (
+ process.argv[1] &&
+ fs.realpathSync(process.argv[1]) === fileURLToPath(import.meta.url)
+)
+ await main();
diff --git a/.github/scripts/ci-admission-count.test.mjs b/.github/scripts/ci-admission-count.test.mjs
new file mode 100644
index 0000000000..b78b9111f4
--- /dev/null
+++ b/.github/scripts/ci-admission-count.test.mjs
@@ -0,0 +1,179 @@
+// Copyright (c) Meta Platforms, Inc. and affiliates.
+
+/**
+ * @file The shared CI count as a command: a tool that grants CI capacity
+ * reads CI the way the admission action does (count.mjs, spec:AST-030 FR16).
+ * @input count.mjs, ci.yml, and a stub `gh` that answers the API reads.
+ * @output A pull request's peak from ci.yml and its labels, and the command's
+ * count, fit and failure exit.
+ * @position Node contract beside ci-admission.test.mjs, which runs the same
+ * count inside the action.
+ */
+
+import {spawnSync} from 'node:child_process';
+import fs from 'node:fs';
+import os from 'node:os';
+import path from 'node:path';
+
+import {afterAll, describe, expect, it} from 'vitest';
+
+import {
+ CI_CEILING,
+ SMALL_MOST,
+ pullRequestPeak,
+} from '../actions/ci-admission/count.mjs';
+
+const root = path.resolve(import.meta.dirname, '../..');
+const COUNT = path.join(root, '.github/actions/ci-admission/count.mjs');
+const workflow = fs.readFileSync(
+ path.join(root, '.github/workflows/ci.yml'),
+ 'utf8',
+);
+const scratch = [];
+afterAll(() => {
+ for (const dir of scratch) fs.rmSync(dir, {recursive: true, force: true});
+});
+
+describe("a pull request's peak", () => {
+ it("is ci.yml's pull-request peak plus the jobs its labels add", () => {
+ const base = pullRequestPeak(workflow, []);
+ expect(workflow).toContain(
+ `peak: \${{ github.event_name == 'workflow_dispatch' && 39 || ${base} }}`,
+ );
+ const legs = JSON.parse(/label-legs: '([^']*)'/.exec(workflow)[1]);
+ expect(pullRequestPeak(workflow, ['ci:run-visual'])).toBe(
+ base + legs['ci:run-visual'],
+ );
+ expect(pullRequestPeak(workflow, Object.keys(legs))).toBe(
+ base + Object.values(legs).reduce((total, jobs) => total + jobs, 0),
+ );
+ // A label that adds no suite adds no jobs.
+ expect(pullRequestPeak(workflow, ['CLA Signed'])).toBe(base);
+ });
+});
+
+describe('the count command', () => {
+ // A stub `gh` that answers `gh api <path>` from a fixture, or fails.
+ function stubGh(fixture) {
+ const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'ci-count-'));
+ scratch.push(dir);
+ fs.writeFileSync(path.join(dir, 'fixture.json'), JSON.stringify(fixture));
+ fs.writeFileSync(
+ path.join(dir, 'gh'),
+ [
+ `#!${process.execPath}`,
+ "const fixture = require(require('path').join(__dirname, 'fixture.json'));",
+ 'const [, , command, route] = process.argv;',
+ "if (command !== 'api' || fixture.fail) process.exit(1);",
+ 'const key = Object.keys(fixture).find(prefix => route.startsWith(prefix));',
+ 'if (key == null) process.exit(1);',
+ 'process.stdout.write(JSON.stringify(fixture[key]));',
+ '',
+ ].join('\n'),
+ {mode: 0o755},
+ );
+ return dir;
+ }
+ const now = new Date().toISOString();
+ const longAgo = new Date(Date.now() - 30 * 60_000).toISOString();
+ // One pull-request CI run admitted half an hour ago with 20 live jobs, and
+ // one small workflow's run.
+ const fixture = {
+ 'repos/o/r/actions/runs?status=in_progress': {
+ workflow_runs: [
+ {
+ id: 11,
+ event: 'pull_request',
+ path: '.github/workflows/ci.yml',
+ updated_at: now,
+ },
+ {
+ id: 12,
+ event: 'pull_request',
+ path: '.github/workflows/lint.yml',
+ updated_at: now,
+ },
+ ],
+ },
+ 'repos/o/r/actions/runs?status=queued': {workflow_runs: []},
+ 'repos/o/r/actions/runs/11/jobs': {
+ jobs: [
+ {
+ name: 'admission',
+ status: 'completed',
+ conclusion: 'success',
+ steps: [
+ {
+ name: 'Wait for room under the CI ceiling',
+ status: 'completed',
+ started_at: longAgo,
+ completed_at: longAgo,
+ },
+ ],
+ },
+ ...Array.from({length: 20}, () => ({status: 'in_progress', steps: []})),
+ ],
+ },
+ 'repos/o/r/pulls/7': {labels: [{name: 'CLA Signed'}]},
+ };
+ const run = (args, data = fixture) =>
+ spawnSync(process.execPath, [COUNT, ...args], {
+ encoding: 'utf8',
+ env: {...process.env, PATH: `${stubGh(data)}:${process.env.PATH}`},
+ });
+
+ it("counts CI as the admission action does, with the pull request's peak and the tool's reservations", () => {
+ const result = run(['--repo', 'o/r', '--pr', '7', '--reserved', '5']);
+ expect(result.status, result.stderr).toBe(0);
+ const look = JSON.parse(result.stdout);
+ const peak = pullRequestPeak(workflow, ['CLA Signed']);
+ // 20 live jobs, small workflows at their floor, and 5 reserved.
+ expect(look).toMatchObject({
+ others: 20 + SMALL_MOST + 5,
+ reserved: 5,
+ peak,
+ ceiling: CI_CEILING,
+ fits: 20 + SMALL_MOST + 5 + peak <= CI_CEILING,
+ mainWaiting: false,
+ // Two run listings, one CI run's jobs and the pull request's labels.
+ calls: 4,
+ });
+ // Reservations that fill CI leave no room.
+ const full = JSON.parse(
+ run(['--repo', 'o/r', '--pr', '7', '--reserved', '60']).stdout,
+ );
+ expect(full.fits).toBe(false);
+ // `--peak` sets the peak instead of the pull request's labels.
+ expect(JSON.parse(run(['--repo', 'o/r', '--peak', '30']).stdout).peak).toBe(
+ 30,
+ );
+ });
+
+ it("leaves out the pull request's own run with --run, and runs through a symlinked path", () => {
+ // Its own waiting run is not counted twice.
+ const own = JSON.parse(
+ run(['--repo', 'o/r', '--peak', '16', '--run', '11']).stdout,
+ );
+ expect(own.others).toBe(SMALL_MOST);
+ const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'ci-count-link-'));
+ scratch.push(dir);
+ const link = path.join(dir, 'count.mjs');
+ fs.symlinkSync(COUNT, link);
+ const linked = spawnSync(
+ process.execPath,
+ [link, '--repo', 'o/r', '--peak', '16'],
+ {
+ encoding: 'utf8',
+ env: {...process.env, PATH: `${stubGh(fixture)}:${process.env.PATH}`},
+ },
+ );
+ expect(linked.status, linked.stderr).toBe(0);
+ expect(JSON.parse(linked.stdout).others).toBe(20 + SMALL_MOST);
+ });
+
+ it('exits 2 when a lookup fails or the arguments are incomplete, which the tool reads as no room', () => {
+ expect(run(['--repo', 'o/r', '--pr', '7'], {fail: true}).status).toBe(2);
+ expect(run(['--repo', 'o/r']).status).toBe(2);
+ expect(run(['--pr', '7']).status).toBe(2);
+ });
+});
diff --git a/.github/scripts/ci-admission.test.mjs b/.github/scripts/ci-admission.test.mjs
index b5a020339e..1f1fb50fc4 100644
--- a/.github/scripts/ci-admission.test.mjs
+++ b/.github/scripts/ci-admission.test.mjs
@@ -11,8 +11,10 @@
* keeps the per-author limits
*/
+import {spawnSync} from 'node:child_process';
import fs from 'node:fs';
import {createRequire} from 'node:module';
+import os from 'node:os';
import path from 'node:path';
import {afterEach, describe, expect, it, vi} from 'vitest';
@@ -27,13 +29,22 @@ const action = read('.github/actions/ci-admission/action.yml');
const ci = read('.github/workflows/ci.yml');
const main = read('.github/workflows/deploy.yml');
const script = action.runs.steps[0].with.script;
+// The count each look runs, shared with the tools that grant CI capacity.
+const COUNT_MODULE = path.join(root, '.github/actions/ci-admission/count.mjs');
+const counter = fs.readFileSync(COUNT_MODULE, 'utf8');
+// Loaded once: under fake timers a fresh import inside the script would
+// resolve after the timers have run.
+const preloaded = {
+ 'node:url': await import('node:url'),
+ count: await import(COUNT_MODULE),
+};
const AsyncFunction = Object.getPrototypeOf(async () => {}).constructor;
const STEP = 'Wait for room under the CI ceiling';
-const CEILING = Number(/const CI_CEILING = (\d+);/.exec(script)[1]);
+const CEILING = Number(/const CI_CEILING = (\d+);/.exec(counter)[1]);
const needsOf = job => [job.needs ?? []].flat();
const stepOf = job => job.steps.find(step => step.name === STEP);
const estimateOf = workflow =>
- new RegExp(`'\\.github/workflows/${workflow}': [^\\n]*`).exec(script)[0];
+ new RegExp(`'\\.github/workflows/${workflow}': [^\\n]*`).exec(counter)[0];
/** Legs a job runs for an event: its matrix size, or one. */
function legs(job, event = 'workflow_dispatch') {
@@ -503,6 +514,26 @@ describe('peaks', () => {
});
});
+describe('the shared count', () => {
+ it("is loaded from the action's own folder, which its checkout holds", () => {
+ // A wrong path would start every run without waiting: each admission
+ // step continues on error.
+ expect(action.runs.steps[0].env.COUNT_MODULE).toBe(
+ '${{ github.action_path }}/count.mjs',
+ );
+ expect(script).toContain('pathToFileURL(process.env.COUNT_MODULE).href');
+ expect(fs.existsSync(COUNT_MODULE)).toBe(true);
+ for (const job of [ci.jobs.room, ci.jobs.admission]) {
+ const checkout = job.steps.find(step =>
+ String(step.uses ?? '').startsWith('actions/checkout'),
+ );
+ expect(checkout.with['sparse-checkout']).toBe(
+ '.github/actions/ci-admission',
+ );
+ }
+ });
+});
+
describe('ci admission', () => {
afterEach(() => vi.useRealTimers());
afterEach(() => vi.restoreAllMocks());
@@ -644,6 +675,7 @@ describe('ci admission', () => {
},
};
Object.assign(process.env, {
+ COUNT_MODULE,
PEAK: String(peak),
MAIN: String(isMain),
LABEL_LEGS: '{"ci:run-visual": 6}',
@@ -655,10 +687,16 @@ describe('ci admission', () => {
LOOKS: looked,
NEXT: next,
});
- const done = new AsyncFunction('github', 'context', 'core', script)(
- github,
- context,
- core,
+ // The script loads the shared count with `import()`, which a function
+ // built here cannot call; the test passes it in.
+ const done = new AsyncFunction(
+ 'github',
+ 'context',
+ 'core',
+ 'importModule',
+ script.replaceAll('await import(', 'await importModule('),
+ )(github, context, core, async specifier =>
+ specifier === 'node:url' ? preloaded['node:url'] : preloaded.count,
);
// Observed here; `await done` below still rethrows a failure.
done.catch(() => {});
@@ -667,6 +705,65 @@ describe('ci admission', () => {
return {...logs, looks, calls, lookTimes, outputs};
}
+ it('agrees with a capacity tool on a full queue: the same count, and no room for either', async () => {
+ // Six pull-request runs at 12 live jobs, one waiting, and small
+ // workflows: 72 + 1 + 18 = 91 other jobs, no room for a run of 16.
+ const active = [...prs(6, 12), waitingRun(40), ...slims(3)];
+ // The tool's reading: the shared count as a command, through a stub `gh`
+ // that answers from the same runs.
+ const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'ci-agree-'));
+ try {
+ const fixture = {
+ 'repos/o/r/actions/runs?status=in_progress': {
+ workflow_runs: active.map(({jobs, ...run}) => run),
+ },
+ 'repos/o/r/actions/runs?status=queued': {workflow_runs: []},
+ ...Object.fromEntries(
+ active.map(run => [
+ `repos/o/r/actions/runs/${run.id}/jobs`,
+ {jobs: run.jobs ?? []},
+ ]),
+ ),
+ };
+ fs.writeFileSync(path.join(dir, 'fixture.json'), JSON.stringify(fixture));
+ fs.writeFileSync(
+ path.join(dir, 'gh'),
+ [
+ `#!${process.execPath}`,
+ "const fixture = require(require('path').join(__dirname, 'fixture.json'));",
+ 'const route = process.argv[3];',
+ 'const key = Object.keys(fixture).sort((a, b) => b.length - a.length).find(prefix => route.startsWith(prefix));',
+ 'if (key == null) process.exit(1);',
+ 'process.stdout.write(JSON.stringify(fixture[key]));',
+ '',
+ ].join('\n'),
+ {mode: 0o755},
+ );
+ const tool = spawnSync(
+ process.execPath,
+ [COUNT_MODULE, '--repo', 'o/r', '--peak', '16'],
+ {
+ encoding: 'utf8',
+ env: {...process.env, PATH: `${dir}:${process.env.PATH}`},
+ },
+ );
+ expect(tool.status, tool.stderr).toBe(0);
+ const reading = JSON.parse(tool.stdout);
+ // Admission's first look on the same runs.
+ const result = await admit({active, peak: 16, maxWait: 12, maxLooks: 2});
+ const look = result.info.find(line =>
+ line.startsWith('Waiting for room: about '),
+ );
+ expect(look).toBeDefined();
+ const admissionOthers = Number(/about (\d+) other jobs/.exec(look)[1]);
+ expect(reading.others).toBe(admissionOthers);
+ expect(reading.fits).toBe(false);
+ expect(admissionOthers + 16).toBeGreaterThan(CEILING);
+ } finally {
+ fs.rmSync(dir, {recursive: true, force: true});
+ }
+ });
+
it("admits at once at today's load: 56 live jobs and many pull-request runs waiting", async () => {
// 2026-11-01 ~01:50Z: 56 jobs ran, a dozen of them the waiting jobs of
// pull-request runs. The first estimate counted each waiting run at 12 and
diff --git a/docs/specs/AST-030/spec.md b/docs/specs/AST-030/spec.md
index 86f1a1e715..e8a4f2511c 100644
--- a/docs/specs/AST-030/spec.md
+++ b/docs/specs/AST-030/spec.md
@@ -410,7 +410,9 @@ they are separately named.
started at its longest wait in the last ten minutes and no run that waited
longer still waits; otherwise it stops with "CI is full: re-run all jobs or
push later", and its required checks report the stopped run. A pull request that may not run CI admits nothing, and
- waiting never fails a job, so every required context reports as before.
+ waiting never fails a job, so every required context reports as before. A
+ tool that grants CI capacity reads it through the same count admission uses
+ (`.github/actions/ci-admission/count.mjs`), with no count of its own.
- **FR17 — A pull request runs CI when it is ready and a maintainer can vouch
for it.** A draft pull request runs no CI, and marking it ready for review
starts its pull-request run, which runs what the change reaches (FR13,
@@ -645,7 +647,7 @@ The nightly perf lane is the one scheduled measurement outside them (FR18).
| FR13 | classifier, workflow contract, component-analysis and RTL-scope tests | component change; shared module; Changeset beside code; infrastructure path; incomplete list; no build | an infrastructure path is scoped, a file-system guard test is skipped, a reverse dependent leaves visual/RTL or enters PR accessibility, or a full-suite event is narrowed |
| FR14 | `workflow-paths.test.mjs` over every workflow | path allowlist; required-context workflow | a `paths-ignore`, a match-all or exclusion glob, or a path filter on a workflow that reports a required context fails |
| FR15 | workflow contract and reconciler tests | ENGOWNER spec PR; owner command; other comment; fork review; non-spec PR; run over the request budget; maintainer dispatch; fork PR, code-only: an owner's exact-head approval plus the owner command clears the code half | `review-required` succeeds while spec approval is missing, another workflow publishes a review context, a fork review starts the spec checks, a non-command comment starts a runner, an over-budget run decides, or a fork's approved code-only PR stays pending after the owner command |
-| FR16 | `heavy-caps-action.test.mjs` cap cases and heavy-job contract; `ci-admission.test.mjs` | sixteenth-oldest PR; 101st heavy run of the day, beside cancelled, light and other-branch runs; agent lane and its five-PR and forty-run backstops; an engineering owner, including the shared login past those backstops, a commented-out or trailing-comment handle and an unreadable roster; bot; the deciding run; re-run; failed lookup; a run with room, without room, past its longest wait, beside a waiting `main` check, as a draft or unlabelled fork, with opt-in labels, beside small workflows past their measured most, and with the API unreadable; each workflow's most jobs; every required context's rule | a heavy suite that runs ahead of the author's fifteen oldest PRs, or past the daily cap, a capped job that passes, an engineering owner or a re-run that is held, or a roster read from the pull request's head, a run that starts without room before its longest wait, a pull-request run that starts while `main` has waited under fifteen minutes, a job admission could skip, a peak below the jobs its run holds, or a job on a slim runner planned past its fifteen-minute cap fails |
+| FR16 | `heavy-caps-action.test.mjs` cap cases and heavy-job contract; `ci-admission.test.mjs` | sixteenth-oldest PR; 101st heavy run of the day, beside cancelled, light and other-branch runs; agent lane and its five-PR and forty-run backstops; an engineering owner, including the shared login past those backstops, a commented-out or trailing-comment handle and an unreadable roster; bot; the deciding run; re-run; failed lookup; a run with room, without room, past its longest wait, beside a waiting `main` check, as a draft or unlabelled fork, with opt-in labels, beside small workflows past their measured most, and with the API unreadable; each workflow's most jobs; every required context's rule; a full queue: the capacity tool and admission agree there is no room | a heavy suite that runs ahead of the author's fifteen oldest PRs, or past the daily cap, a capped job that passes, an engineering owner or a re-run that is held, or a roster read from the pull request's head, a run that starts without room before its longest wait, a pull-request run that starts while `main` has waited under fifteen minutes, a job admission could skip, a peak below the jobs its run holds, or a job on a slim runner planned past its fifteen-minute cap fails |
| FR16 | `ci-admission.test.mjs` cases for one start at the longest wait at a time and the workflow contract | a pull-request run at its longest wait alone, within ten minutes of another pull-request run's start there and after them, behind a run that waited longer and ahead of a younger one; another run that found room earlier, a fork's stopped run and a `main` run that waited its longest; a `main` run and a merge-queue entry at their longest wait; the rest of a run whose Storybook build found room; ten runs out of time together; the required contexts and ci-full for either wait; a pull-request run that finds ten others waiting and room only for its Storybook build, and one at its longest wait behind more than ten that waited longer; two waiting `main` dispatches, the older and the younger, and two that began in the same second (each counting the other's peak fails) | two pull-request runs start at their longest wait within ten minutes, a younger run starts ahead of a longer waiter, a `main` run or merge-queue entry stops, a run CI is full for builds, or a required context passes or skips |
| FR17 | `ci-pr-gate.test.mjs` over every `pull_request` workflow and the shared conditions | branch here, fork with and without `ci:run`, another label, draft with the label, ready, push, merge group, dispatch | a job that can start on a pull request without the condition, a label trigger, a required job that skips for a pull request that may not run CI, or a draft or unlabelled fork that runs fails |
| FR18 | `perf-nightly-workflow.test.mjs`, `perf-trend.test.mjs` and `run-labels.test.mjs` | nightly and dispatch; labelled, unlabelled, draft and unvouched-fork pull requests; write, read and no access; newest label event; failed lookup; 10% boundary; metric floors; seven-day window edges; fewer than three runs; a metric over once, twice in a row, or a different metric before; recorded and pull request runs | a run on an unlabelled pull request, a label honoured from read access, a regression called at exactly 10%, under its floor or on one run alone, a run outside the window in the baseline, a pull request run that records, or a second regression issue fails |