Skip to content

An open source design system that's fully customizable and agent ready

TypeScript

ci: a component's source change runs its own accessibility audits (#10629)

Cindy Zhangc9d3383fbe3f
diff --git a/.github/scripts/a11y-dependent-audits.mjs b/.github/scripts/a11y-dependent-audits.mjs
index f573371c25..32d05c4daf 100644
--- a/.github/scripts/a11y-dependent-audits.mjs
+++ b/.github/scripts/a11y-dependent-audits.mjs
@@ -2,17 +2,17 @@
// Copyright (c) Meta Platforms, Inc. and affiliates.
/**
- * @file The dependent audits a shared primitive's change runs (spec:AST-030
- * FR12).
+ * @file The audits a pull request's change owes (spec:AST-030 FR12).
* @input The changed files, the shared primitives that
* `.github/a11y-shared-primitives.json` lists, the repository's source
* import graph, the `audit` specs, and their recorded times.
- * @output The `audit` specs of the components that use a changed primitive,
- * split into at most three legs of about ten minutes each by recorded time.
- * @position The plan job writes the legs; each leg runs its share. An `audit`
- * spec otherwise runs only in its own component-audit pull request and in the
- * release check, so a change to a shared component such as Button could break
- * a dependent's audit with no pull-request signal.
+ * @output The `audit` specs of each changed component, then of each changed
+ * primitive and the components that use it; this run's legs, at most three
+ * of about ten minutes on a first run; and the record merge-guard reads:
+ * owed, ran, deferred, complete.
+ * @position The plan job writes the legs and the record; each leg runs its
+ * share. An `audit` spec otherwise runs only in its own component-audit pull
+ * request and in the release check.
*/
import {execFileSync} from 'node:child_process';
@@ -25,14 +25,18 @@ import {
fileSeconds,
importerGraph,
isRenderedSource,
+ LEG_BUDGET_MS,
parseRouting,
planLegs,
renderingFolders,
} from './a11y-pr-contracts.mjs';
+import {costOf, splitByDuration} from '../../scripts/lib/duration-split.mjs';
import {isA11ySpecFile} from './lib/a11y-spec-files.mjs';
export const PRIMITIVES_FILE = '.github/a11y-shared-primitives.json';
export const MAX_LEGS = 3;
+/** A later run at the same head runs every owed audit, in up to six legs. */
+export const LATER_MAX_LEGS = 6;
const DURATIONS_FILE = '.github/a11y-contract-durations.json';
/**
@@ -100,15 +104,73 @@ export function listAuditSpecs(root) {
}
/**
- * The legs for a change: the audits of the changed primitives and of the
- * component folders that render them.
+ * Legs for the files, each about ten minutes or less when `maxLegs` allows:
+ * like `planLegs`, with one more leg at a time while the longest is over.
+ * @param {string[]} files
+ * @param {Record<string, number>} seconds
+ * @param {number} maxLegs
+ */
+export function fitLegs(files, seconds, maxLegs) {
+ const cost = costOf(files, seconds);
+ const split = (/** @type {number} */ count) => {
+ const shares = splitByDuration(files, count, cost);
+ return {
+ count,
+ shares,
+ longestMs: Math.max(
+ ...shares.map(
+ share => share.reduce((total, file) => total + cost(file), 0) * 1000,
+ ),
+ ),
+ };
+ };
+ const most = Math.max(1, Math.min(maxLegs, files.length));
+ let plan = split(Math.min(planLegs(files, seconds, maxLegs).count, most));
+ while (plan.longestMs > budgetFor(files, seconds) && plan.count < most) {
+ plan = split(plan.count + 1);
+ }
+ return plan;
+}
+
+/**
+ * A leg's budget for these files: ten minutes, or the longest single file's
+ * recorded time when one file alone is longer.
+ * @param {string[]} files
+ * @param {Record<string, number>} seconds
+ */
+export function budgetFor(files, seconds) {
+ const cost = costOf(files, seconds);
+ return Math.max(LEG_BUDGET_MS, ...files.map(file => cost(file) * 1000));
+}
+
+/**
+ * The component folders whose own rendered source a change touches.
+ * @param {string[]} changed
+ */
+export function changedComponents(changed) {
+ return [
+ ...new Set(
+ changed.filter(isRenderedSource).map(componentFolderOf).filter(Boolean),
+ ),
+ ].sort();
+}
+
+/**
+ * The audits a change owes and this run's legs. A changed component's own
+ * audits come first, then each changed primitive's and those of the folders
+ * that render it, nearest first. A first run fits what it can into
+ * {@link MAX_LEGS} legs of about ten minutes and names the rest as deferred; a
+ * later run at the same head runs every owed audit, in up to
+ * {@link LATER_MAX_LEGS} legs.
* @param {object} options
* @param {string[]} options.changed
* @param {string[]} options.primitives
* @param {{file: string, mode: string}[]} options.specs
* @param {Map<string, Set<string>>} options.importers
* @param {Record<string, number>} options.seconds
- * @param {number} [options.maxLegs]
+ * @param {number} [options.attempt] the run's attempt; 1 for its first run
+ * @param {boolean} [options.everything] the changed files are unknown and
+ * everything is in scope: every component's audits are owed
*/
export function planDependentAudits({
changed,
@@ -116,22 +178,60 @@ export function planDependentAudits({
specs,
importers,
seconds,
- maxLegs = MAX_LEGS,
+ attempt = 1,
+ everything = false,
}) {
+ const components = everything
+ ? [
+ ...new Set(
+ specs
+ .filter(({mode}) => mode === 'audit')
+ .map(({file}) => componentFolderOf(file))
+ .filter(Boolean),
+ ),
+ ].sort()
+ : changedComponents(changed);
const touched = changedPrimitives(primitives, changed);
- if (touched.length === 0) {
- return {primitives: [], files: [], count: 0, shares: [], longestMs: 0};
- }
- const folders = new Set([
+ const order = [
+ ...components,
...touched,
...renderingFolders(importers, touched).map(({folder}) => folder),
- ]);
- const files = dependentAudits(specs, folders);
- if (files.length === 0) {
- return {primitives: touched, files, count: 0, shares: [], longestMs: 0};
+ ];
+ /** @type {string[]} */
+ const owed = [];
+ for (const folder of order) {
+ for (const file of dependentAudits(specs, new Set([folder]))) {
+ if (!owed.includes(file)) owed.push(file);
+ }
+ }
+ const base = {components, primitives: touched, owed};
+ if (owed.length === 0) {
+ return {
+ ...base,
+ files: [],
+ deferred: [],
+ complete: true,
+ count: 0,
+ shares: [],
+ longestMs: 0,
+ };
}
- const legs = planLegs(files, seconds, maxLegs);
- return {primitives: touched, files, ...legs};
+ if (attempt > 1) {
+ const legs = fitLegs(owed, seconds, LATER_MAX_LEGS);
+ return {...base, files: owed, deferred: [], complete: true, ...legs};
+ }
+ /** @type {string[]} */
+ const files = [];
+ /** @type {string[]} */
+ const deferred = [];
+ for (const file of owed) {
+ const fits =
+ fitLegs([...files, file], seconds, MAX_LEGS).longestMs <=
+ budgetFor([...files, file], seconds);
+ (fits || files.length === 0 ? files : deferred).push(file);
+ }
+ const legs = fitLegs(files, seconds, MAX_LEGS);
+ return {...base, files, deferred, complete: deferred.length === 0, ...legs};
}
function main() {
@@ -142,23 +242,29 @@ function main() {
};
const root = process.cwd();
const changedFile = value('--changed');
+ const everything = argv.includes('--all');
const out = value('--out');
- if (!changedFile || !out) {
+ const attempt = Number(value('--attempt') ?? '1');
+ if (
+ (!changedFile && !everything) ||
+ !out ||
+ !Number.isInteger(attempt) ||
+ attempt < 1
+ ) {
console.error(
- 'usage: a11y-dependent-audits.mjs --changed <file> --out <file>',
+ 'usage: a11y-dependent-audits.mjs (--changed <file> | --all) --out <file> [--attempt <run attempt>]',
);
process.exit(2);
}
- // Without the changed files the primitives a change touches are unknown,
- // and a missed audit would read as passed.
- if (!fs.existsSync(changedFile)) {
+ // Without the changed files the audits a change owes are unknown, and a
+ // missed audit would read as passed.
+ if (!everything && !fs.existsSync(changedFile)) {
console.error(`::error::The changed-file list ${changedFile} is missing.`);
process.exit(1);
}
- const changed = fs
- .readFileSync(changedFile, 'utf8')
- .split('\n')
- .filter(Boolean);
+ const changed = everything
+ ? []
+ : fs.readFileSync(changedFile, 'utf8').split('\n').filter(Boolean);
const plan = planDependentAudits({
changed,
primitives: readPrimitives(root),
@@ -167,28 +273,61 @@ function main() {
seconds: fileSeconds(
JSON.parse(fs.readFileSync(path.join(root, DURATIONS_FILE), 'utf8')),
),
+ attempt,
+ everything,
});
- fs.writeFileSync(out, `${JSON.stringify(plan)}\n`);
- if (plan.primitives.length === 0) {
- console.log(
- 'This change touches no shared primitive; no dependent audit runs.',
- );
+ // The record merge-guard reads: what this change owes, what this run ran,
+ // and whether that is every owed audit.
+ const record = {
+ head: process.env.ASTRYX_HEAD_SHA ?? '',
+ attempt,
+ owed: plan.owed,
+ ran: plan.files,
+ deferred: plan.deferred,
+ complete: plan.complete,
+ };
+ fs.writeFileSync(out, `${JSON.stringify(record, null, 2)}\n`);
+ if (plan.owed.length === 0) {
+ console.log('This change owes no audit; no dependent audit runs.');
} else {
const minutes = (plan.longestMs / 60000).toFixed(1);
console.log(
- `Shared primitives changed: ${plan.primitives.join(', ')}. ${plan.files.length} dependent audit spec files in ${plan.count} leg(s), the longest leg's recorded time ${minutes} min.`,
+ `Changed components: ${plan.components.join(', ') || 'none'}. Shared primitives changed: ${plan.primitives.join(', ') || 'none'}.`,
+ );
+ console.log(
+ `This run (attempt ${attempt}) runs ${plan.files.length} of ${plan.owed.length} owed audit spec files in ${plan.count} leg(s), the longest leg's recorded time ${minutes} min:`,
);
for (const file of plan.files) console.log(` ${file}`);
- if (plan.longestMs > 10 * 60 * 1000) {
+ if (plan.deferred.length > 0) {
console.log(
- `::warning title=Dependent audits over ten minutes::${plan.count} legs still leave ${minutes} min of recorded time in one leg.`,
+ `::warning title=Audits not yet run::${plan.deferred.length} owed audit spec files do not fit three legs of ten minutes. Re-run all jobs at this head to run every owed audit; merge-guard merges only on a run that ran them all.`,
);
+ console.log('Not yet run:');
+ for (const file of plan.deferred) console.log(` ${file}`);
}
}
if (process.env.GITHUB_OUTPUT) {
fs.appendFileSync(
process.env.GITHUB_OUTPUT,
- `count=${plan.count}\nshares=${JSON.stringify(plan.shares)}\n`,
+ [
+ `count=${plan.count}`,
+ `shares=${JSON.stringify(plan.shares)}`,
+ `owed=${JSON.stringify(plan.owed)}`,
+ `ran=${JSON.stringify(plan.files)}`,
+ `deferred=${JSON.stringify(plan.deferred)}`,
+ `complete=${plan.complete}`,
+ '',
+ ].join('\n'),
+ );
+ }
+ if (process.env.GITHUB_STEP_SUMMARY && plan.owed.length > 0) {
+ const list = files => files.map(file => `- \`${file}\``).join('\n');
+ fs.appendFileSync(
+ process.env.GITHUB_STEP_SUMMARY,
+ `### Dependent audits\n\nRan (${plan.files.length}):\n\n${list(plan.files)}\n\n` +
+ (plan.deferred.length
+ ? `Not yet run (${plan.deferred.length}); re-run all jobs at this head:\n\n${list(plan.deferred)}\n`
+ : 'Every owed audit ran.\n'),
);
}
}
diff --git a/.github/scripts/a11y-dependent-audits.test.mjs b/.github/scripts/a11y-dependent-audits.test.mjs
index 55797cfa4e..88c885ac83 100644
--- a/.github/scripts/a11y-dependent-audits.test.mjs
+++ b/.github/scripts/a11y-dependent-audits.test.mjs
@@ -18,8 +18,10 @@ import {describe, expect, it} from 'vitest';
import yaml from 'yaml';
import {
+ budgetFor,
changedPrimitives,
dependentAudits,
+ LATER_MAX_LEGS,
listAuditSpecs,
MAX_LEGS,
planDependentAudits,
@@ -114,13 +116,19 @@ describe("a Button change runs its dependents' audits", () => {
expect(shares.flat().sort()).toEqual([...files].sort());
});
- it('plans nothing for a change that touches no primitive', () => {
+ it('plans nothing for a change to no component source', () => {
for (const changed of [
- ['packages/core/src/Chat/ChatSendButton.tsx'],
['packages/core/src/Button/Button.test.tsx'],
+ ['packages/core/src/Button/Button.doc.mjs'],
[],
]) {
- expect(plan(changed)).toMatchObject({count: 0, files: [], shares: []});
+ expect(plan(changed)).toMatchObject({
+ count: 0,
+ files: [],
+ shares: [],
+ owed: [],
+ complete: true,
+ });
}
});
@@ -145,6 +153,110 @@ describe("a Button change runs its dependents' audits", () => {
});
});
+describe("a component's source change runs its own audits", () => {
+ it("runs DropdownMenu's own audits for a DropdownMenu change", () => {
+ const {components, primitives, files, deferred, complete} = plan([
+ 'packages/core/src/DropdownMenu/DropdownMenu.tsx',
+ ]);
+ expect(components).toEqual(['core/DropdownMenu']);
+ expect(primitives).toEqual([]);
+ expect(files).toEqual(
+ expect.arrayContaining([
+ 'packages/core/src/DropdownMenu/__tests__/DropdownMenuRadioItemAudit.a11y.chromium.spec.ts',
+ 'packages/core/src/DropdownMenu/__tests__/DropdownMenuSubMenuAudit.a11y.chromium.spec.ts',
+ ]),
+ );
+ expect(deferred).toEqual([]);
+ expect(complete).toBe(true);
+ });
+
+ // Five components, each with one audit of about seven minutes: three fit
+ // three legs of ten minutes on a first run.
+ const wide = {
+ changed: ['A', 'B', 'C', 'D', 'E'].map(
+ name => `packages/core/src/${name}/${name}.tsx`,
+ ),
+ primitives: ['core/Button'],
+ specs: ['A', 'B', 'C', 'D', 'E'].map(name => ({
+ file: `packages/core/src/${name}/__tests__/${name}Audit.a11y.chromium.spec.ts`,
+ mode: 'audit',
+ })),
+ importers: new Map(),
+ seconds: Object.fromEntries(
+ ['A', 'B', 'C', 'D', 'E'].map(name => [
+ `packages/core/src/${name}/__tests__/${name}Audit.a11y.chromium.spec.ts`,
+ 420,
+ ]),
+ ),
+ };
+
+ it('gives an audit over ten minutes its own leg and still fits the rest', () => {
+ const file = name =>
+ `packages/core/src/${name}/__tests__/${name}Audit.a11y.chromium.spec.ts`;
+ const long = {
+ changed: ['A', 'B', 'C'].map(
+ name => `packages/core/src/${name}/${name}.tsx`,
+ ),
+ primitives: ['core/Button'],
+ specs: ['A', 'B', 'C'].map(name => ({file: file(name), mode: 'audit'})),
+ importers: new Map(),
+ seconds: {[file('A')]: 700, [file('B')]: 300, [file('C')]: 300},
+ };
+ const both = planDependentAudits(long);
+ expect(both.files).toEqual([file('A'), file('B'), file('C')]);
+ expect(both.deferred).toEqual([]);
+ expect(both.shares).toContainEqual([file('A')]);
+ // A lone long audit plans one leg, never an empty one.
+ const alone = planDependentAudits({...long, changed: [long.changed[0]]});
+ expect(alone.count).toBe(1);
+ expect(alone.shares).toEqual([[file('A')]]);
+ });
+
+ it('names the audits a first run cannot fit, and runs them all later at the same head', () => {
+ const first = planDependentAudits(wide);
+ expect(first.owed).toHaveLength(5);
+ expect(first.files).toHaveLength(3);
+ expect(first.deferred).toEqual([
+ 'packages/core/src/D/__tests__/DAudit.a11y.chromium.spec.ts',
+ 'packages/core/src/E/__tests__/EAudit.a11y.chromium.spec.ts',
+ ]);
+ expect(first.complete).toBe(false);
+ expect(first.count).toBeLessThanOrEqual(MAX_LEGS);
+ expect(first.longestMs).toBeLessThanOrEqual(LEG_BUDGET_MS);
+ const later = planDependentAudits({...wide, attempt: 2});
+ expect(later.files).toEqual(later.owed);
+ expect(later.deferred).toEqual([]);
+ expect(later.complete).toBe(true);
+ expect(later.count).toBeLessThanOrEqual(LATER_MAX_LEGS);
+ expect(later.longestMs).toBeLessThanOrEqual(LEG_BUDGET_MS);
+ });
+});
+
+describe('an undecided scope', () => {
+ it('owes every audit, and a first run lists what does not fit', () => {
+ const all = plan([]);
+ expect(all.owed).toEqual([]);
+ const everything = planDependentAudits({
+ changed: [],
+ primitives: readPrimitives(root),
+ specs,
+ importers,
+ seconds,
+ everything: true,
+ });
+ const audits = specs
+ .filter(({mode}) => mode === 'audit')
+ .map(({file}) => file);
+ expect([...everything.owed].sort()).toEqual([...audits].sort());
+ expect([...everything.files, ...everything.deferred].sort()).toEqual(
+ [...audits].sort(),
+ );
+ expect(everything.longestMs).toBeLessThanOrEqual(
+ budgetFor(everything.files, seconds),
+ );
+ });
+});
+
describe('the dependent-audits jobs', () => {
const planJob = ci.jobs['a11y-dependent-audits-plan'];
const legs = ci.jobs['pr-a11y-dependent-audits'];
@@ -154,16 +266,40 @@ describe('the dependent-audits jobs', () => {
expect(planJob['runs-on']).toBe('ubuntu-slim');
expect(planJob.needs).toContain('build-storybook');
const run = step(planJob, 'Plan the dependent audits');
- expect(run.run).toBe(
- 'node .github/scripts/a11y-dependent-audits.mjs --changed "$CHANGED_FILES" --out "$RUNNER_TEMP/dependent-audits.json"',
+ expect(run.run).toContain(
+ 'node .github/scripts/a11y-dependent-audits.mjs "${changes[@]}" --out "$RUNNER_TEMP/a11y-dependent-audits.json" --attempt "$RUN_ATTEMPT"',
+ );
+ // An undecided scope (a truncated diff) owes every audit; a decided
+ // scope without its list stops.
+ expect(run.run).toContain('changes=(--all)');
+ expect(step(planJob, 'Require the changed-file list').if).toContain(
+ "steps.scope.outputs.decided != 'false'",
);
+ expect(run.env.RUN_ATTEMPT).toBe('${{ github.run_attempt }}');
expect(step(planJob, 'Require the changed-file list').run).toContain(
'exit 1',
);
});
- it('run up to three legs, on a full runner only when a leg has a share', () => {
- expect(legs.strategy.matrix.leg).toEqual([1, 2, 3]);
+ it('give merge-guard the record of owed, ran, deferred and complete', () => {
+ for (const key of ['owed', 'ran', 'deferred'])
+ expect(planJob.outputs[key]).toBe(
+ `\${{ steps.plan.outputs.${key} || '[]' }}`,
+ );
+ expect(planJob.outputs.complete).toBe(
+ "${{ steps.plan.outputs.complete || 'true' }}",
+ );
+ const upload = step(planJob, 'Upload the dependent-audit record');
+ expect(upload.with.name).toBe(
+ 'a11y-dependent-audits-record-${{ github.run_attempt }}',
+ );
+ expect(upload.with.path).toBe(
+ '${{ runner.temp }}/a11y-dependent-audits.json',
+ );
+ });
+
+ it('run up to six legs, on a full runner only when a leg has a share', () => {
+ expect(legs.strategy.matrix.leg).toEqual([1, 2, 3, 4, 5, 6]);
expect(legs.strategy['fail-fast']).toBe(false);
expect(legs['runs-on']).toBe(
"${{ needs.a11y-dependent-audits-plan.outputs.count >= matrix.leg && 'ubuntu-latest' || 'ubuntu-slim' }}",
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index efeb5ea329..fbb8b26335 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -4436,9 +4436,10 @@ jobs:
if-no-files-found: ignore
retention-days: 7
- # Dependent audits (spec:AST-030 FR12): a change to a shared primitive that
- # .github/a11y-shared-primitives.json lists runs the `audit` specs of the
- # components that use it, which otherwise run only in their own
+ # Dependent audits (spec:AST-030 FR12): a change to a component's source runs
+ # that component's own `audit` specs, and a change to a shared primitive that
+ # .github/a11y-shared-primitives.json lists also runs the `audit` specs of
+ # the components that use it. These otherwise run only in their own
# component-audit pull request and in the release check. The plan job reads
# the changed files and the import graph at the merge commit and splits the
# audits into at most three legs of about ten minutes each by recorded time.
@@ -4451,9 +4452,16 @@ jobs:
permissions:
contents: read
pull-requests: read
+ # owed, ran, deferred and complete are the record merge-guard reads: a
+ # head merges only on a run whose complete is true. The same record is
+ # the a11y-dependent-audits-record-<attempt> artifact.
outputs:
count: ${{ steps.plan.outputs.count || '0' }}
shares: ${{ steps.plan.outputs.shares || '[]' }}
+ owed: ${{ steps.plan.outputs.owed || '[]' }}
+ ran: ${{ steps.plan.outputs.ran || '[]' }}
+ deferred: ${{ steps.plan.outputs.deferred || '[]' }}
+ complete: ${{ steps.plan.outputs.complete || 'true' }}
steps:
- name: Check out the scope action
uses: actions/checkout@v7
@@ -4484,18 +4492,41 @@ jobs:
# Without the list a changed primitive would go unseen and its
# dependents' audits would not run.
+ # When the scope could not be decided (a diff GitHub truncates, say),
+ # everything is in scope and every audit is owed. A decided scope
+ # without its list is an error.
- name: Require the changed-file list
- if: steps.scope.outputs.applies == 'true' && steps.scope.outputs.changed_files_path == ''
+ if: steps.scope.outputs.applies == 'true' && steps.scope.outputs.changed_files_path == '' && steps.scope.outputs.decided != 'false'
run: |
echo "::error::The scope step could not list the changed files, so the dependent audits to run are unknown. Re-run the job."
exit 1
+ # A first run fits what it can into three legs of about ten minutes and
+ # names the rest as not yet run; a later run at the same head (a re-run
+ # of all jobs) runs every owed audit.
- name: Plan the dependent audits
id: plan
if: steps.scope.outputs.applies == 'true'
env:
CHANGED_FILES: ${{ steps.scope.outputs.changed_files_path }}
- run: node .github/scripts/a11y-dependent-audits.mjs --changed "$CHANGED_FILES" --out "$RUNNER_TEMP/dependent-audits.json"
+ ASTRYX_HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
+ RUN_ATTEMPT: ${{ github.run_attempt }}
+ run: |
+ if [ -n "$CHANGED_FILES" ]; then
+ changes=(--changed "$CHANGED_FILES")
+ else
+ changes=(--all)
+ fi
+ node .github/scripts/a11y-dependent-audits.mjs "${changes[@]}" --out "$RUNNER_TEMP/a11y-dependent-audits.json" --attempt "$RUN_ATTEMPT"
+
+ - name: Upload the dependent-audit record
+ if: steps.plan.outcome == 'success'
+ uses: actions/upload-artifact@v7
+ with:
+ # One record per attempt: merge-guard reads the latest attempt's.
+ name: a11y-dependent-audits-record-${{ github.run_attempt }}
+ path: ${{ runner.temp }}/a11y-dependent-audits.json
+ retention-days: 30
# Each leg runs its share of the plan at the head that built the Storybook.
# A leg without a share runs on a slim runner and only says so. The `test`
@@ -4508,10 +4539,12 @@ jobs:
# An empty count (the plan skipped) compares as 0.
runs-on: ${{ needs.a11y-dependent-audits-plan.outputs.count >= matrix.leg && 'ubuntu-latest' || 'ubuntu-slim' }}
timeout-minutes: 15
+ # A first run plans at most three legs; a later run at the same head may
+ # plan up to six to run every owed audit.
strategy:
fail-fast: false
matrix:
- leg: [1, 2, 3]
+ leg: [1, 2, 3, 4, 5, 6]
permissions:
contents: read
steps:
diff --git a/docs/specs/AST-030/spec.md b/docs/specs/AST-030/spec.md
index 8d1dec2bb8..1863d43cf5 100644
--- a/docs/specs/AST-030/spec.md
+++ b/docs/specs/AST-030/spec.md
@@ -249,7 +249,7 @@ they are separately named.
scope and spec-location mapping the labelled run uses. Then, in the time
left and for at most one minute of it, it runs the `scoped` and
`release-only` spec files of the component folders that render a touched
- component through their source value imports, nearest first. It runs no unchanged `audit` or `always` spec. For a shared primitive named in `.github/a11y-shared-primitives.json`, FR12's touched-component rule also runs a dependent-audits leg on every pull-request CI run that includes a change to it: at most three legs, each at most 10 minutes, covering the audits of components that use it. The list starts with Button and IconButton. Layer joins when a Layer change breaks release-check or a dependent audit at release; other additions are a CI & Release and chief-of-staff call. The contract job runs in parallel with the unit-test jobs and
+ component through their source value imports, nearest first. It runs no unchanged `audit` or `always` spec. For a shared primitive named in `.github/a11y-shared-primitives.json`, FR12's touched-component rule also runs a dependent-audits leg on every pull-request CI run that includes a change to it: at most three legs, each at most 10 minutes, covering the audits of components that use it. The list starts with Button and IconButton. Layer joins when a Layer change breaks release-check or a dependent audit at release; other additions are a CI & Release and chief-of-staff call. A change to a component's source also runs that component's own audit specs in the same leg, on every pull-request CI run that includes the change, within the same three legs of ten minutes. Audits beyond that budget are listed by name as not yet run, and they run on a later run at the same head, which may use more legs; merge-guard merges only on a run at the exact head that ran every owed audit. The contract job runs in parallel with the unit-test jobs and
MUST NOT become the pull request's critical path: it starts after the
Storybook build and MUST finish before the slowest unit-test job, so the
time until the required `test` join reports does not grow. It stops after
@@ -661,6 +661,7 @@ The nightly perf lane is the one scheduled measurement outside them (FR18).
| FR12 | `touched-components.test.mjs` changed-spec routing; `a11y-pr-contracts.test.mjs` `changedOnly` cases; `a11y-changed-stories.test.mjs` and the touched job's story-audit contract | a pull request that adds a browser accessibility spec; one that changes an `always` spec; one that changes a `release-only` or an `audit` spec; one that changes more spec files than the limit; a component change with no spec change; a pull request that adds an `a11y/` story no component owns; one that adds a story to an untouched component's story file beside a touched component | a pull request that adds a browser accessibility spec that fails is not caught on that pull request, a spec of a component that is neither changed, touched nor rendering a touched component runs in the contract job, or an unchanged `audit` spec does, a not-run file reads as passed or fails the join, a failing contract leaves the `test` join green, or a new violation in a changed story no component owns passes the touched check |
| FR12 | `a11y-pr-contracts.test.mjs` ownership cases; the CI workflow contract; contract-job and unit-test-job timings on 17 component pull requests | a pull request that touches one component with an owned `release-only` contract, passing and failing; a contract file that cannot finish within the budget; a wide change; a reverse dependent of the touched component; measured: setup up to about 1.5 minutes plus the 8-minute stop, against the slowest unit-test job finishing 9.9 to 25 minutes after the Storybook build; a wide change touching Spinner (its own specs run); the folders that render a touched component, nearest first, type-only imports left out, within a one-minute share, and one that does not fit (reported as not run) | a touched component's owned `release-only` contract does not run on its pull request, a failing contract leaves the `test` join green, a skipped file is split, does not show as "not run" in the check's summary or fails the join, the contract job finishes after the slowest unit-test job, or a touched component's own spec does not run on a wide change, or a rendering folder's spec runs before an owned one |
| FR12 | `a11y-dependent-audits.test.mjs` planning cases and the CI workflow contract | a Button change; an IconButton change; a test, story or doc change in a primitive's folder; a change to no primitive; the plan's legs and their recorded times | a Button change that leaves a dependent component's audit out, plans more than three legs or a leg over ten minutes, or a dependent audit failure the `test` join passes |
+| FR12 | `a11y-dependent-audits.test.mjs` own-audit and deferral cases; the dependent-audit record | a component source change; an over-budget set on a first run; a later run at the same head | a component source change that leaves its own audits out; an over-budget set whose deferred audits are not listed by name; a head that merges on a run that left an owed audit unrun |
| FR13 | classifier, workflow contract, component-analysis and RTL-scope tests | component change; shared module; Changeset beside code; infrastructure path; incomplete list; no build; an interaction-harness or lint-plugin path beside code, alone and with another shared path, and a script or other shared path beside one | an infrastructure path is scoped, a file-system guard test is skipped, a reverse dependent leaves visual/RTL or enters PR accessibility, or a full-suite event is narrowed |
| FR14 | `workflow-paths.test.mjs` over every workflow | path allowlist; required-context workflow | a `paths-ignore`, a match-all or exclusion glob, or a path filter on a workflow that reports a required context fails |
| FR15 | workflow contract and reconciler tests | ENGOWNER spec PR; owner command; other comment; fork review; non-spec PR; run over the request budget; maintainer dispatch; fork PR, code-only: an owner's exact-head approval plus the owner command clears the code half | `review-required` succeeds while spec approval is missing, another workflow publishes a review context, a fork review starts the spec checks, a non-command comment starts a runner, an over-budget run decides, or a fork's approved code-only PR stays pending after the owner command |
@@ -1163,8 +1164,9 @@ fix-forward lands).
**Reference:** `spec:AST-030/DEC-23`
**Decider:** chief of staff with the release owner, `2026-11-05`
-**Superseded in part by:** `spec:AST-030/DEC-25` (a shared primitive's change
-runs the `audit` specs of the components that use it)
+**Superseded in part by:** `spec:AST-030/DEC-25` (a changed component runs its
+own `audit` specs, and a shared primitive's change runs the `audit` specs of
+the components that use it)
A wide refactor that changed Spinner's behavior (its role and its shared
announcement, across eight components) merged without Spinner's own motion
@@ -1182,7 +1184,8 @@ the label that runs it. The one-minute share holds the added runner time to abou
rendering folders' share widens only if it catches failures the owned
contracts miss. The default touched-components check keeps its wide-change skip, and
unchanged `audit` and `always` specs still do not run on a pull request, except
-the dependent audits of a shared primitive (DEC-25).
+a changed component's own audits and the dependent audits of a shared primitive
+(DEC-25).
Rejected: dropping the five-component limit for every audit (the visual,
accessibility and RTL audits of a wide change do not fit the default check's
@@ -1221,6 +1224,11 @@ a merge rests on the pull request's own run at its exact head. Layer,
which nearly every audited component uses, joins only when a Layer change
breaks the release check or a dependent audit at release.
+Extended 2026-11-07 (19:40) by the chief of staff with CI & Release: a
+component's own audits join the leg. A first run fits what it can into the same
+three legs and lists the rest by name; a later run at the same head runs every
+owed audit, and merge-guard merges only on a run that ran them all.
+
Rejected: running the audits only when the pull request is marked ready and on
a re-run (a push after ready merges without them); every audit spec on every
Layer change (about 37 minutes of recorded time on a change that lands about