Skip to content

An open source design system that's fully customizable and agent ready

TypeScript

ci: an engineering owner's owner command re-evaluates both halves of review-required, so a fork's approval takes effect (#9587)

Cindy Zhangfda92ace189e
diff --git a/.github/REVIEW_GATE.md b/.github/REVIEW_GATE.md
index 6c2b863ebc..1944d4d91f 100644
--- a/.github/REVIEW_GATE.md
+++ b/.github/REVIEW_GATE.md
@@ -19,6 +19,12 @@ neutral yellow "waiting" signal, not a red failure).
- **Design review is advisory** — it labels the PR and requests design owners,
but does not block the merge.
- An entitled owner's **approval clears the gate automatically** (no manual step).
+- **On a fork pull request, also post the owner command.** GitHub can hold a
+ fork's review event until a maintainer approves the run (a first-time
+ contributor, for example), so the approval alone may not reach the gate. An
+ engineering owner who approves a fork pull request also posts
+ `/approve-spec <full-sha>`: the command re-evaluates both halves of
+ `review-required` from the current head, the code review included.
- **Write-access authors can also clear it with the review app.** When the PR
author has write or admin permission on the repository (internal
contributors, including designers and partners who are not owners), the
diff --git a/.github/scripts/review-signal-workflow.test.mjs b/.github/scripts/review-signal-workflow.test.mjs
index a7dbd32697..ee771c198d 100644
--- a/.github/scripts/review-signal-workflow.test.mjs
+++ b/.github/scripts/review-signal-workflow.test.mjs
@@ -44,6 +44,9 @@ async function runSignal({
rateRemaining = 5000,
batch = '',
draft = false,
+ commenter = 'engineer',
+ engOwners = '@engineer',
+ commentBody = `/approve-spec ${HEAD}`,
// The spec-owner job's decision for a PR that changes no spec record.
specStatuses = [
{
@@ -159,13 +162,18 @@ async function runSignal({
payload:
eventName === 'workflow_dispatch'
? {inputs: {pr: backfill ? '' : '42', batch}}
- : {pull_request: pr},
+ : eventName === 'issue_comment'
+ ? {
+ issue: {number: 42, pull_request: {}},
+ comment: {user: {login: commenter}, body: commentBody},
+ }
+ : {pull_request: pr},
},
core,
{
env: {
...parsed.env,
- ENG_OWNERS: '@engineer',
+ ENG_OWNERS: engOwners,
DESIGN_OWNERS: '@designer',
REVIEW_APP: reviewApp,
},
@@ -192,8 +200,8 @@ function expectTrustedHelperRead(getContent) {
describe('review-signal trusted helper ref', () => {
it('only runs the helper-loading job in trusted execution contexts', () => {
- expect(parsed.jobs.flag.if.trim()).toBe(
- "github.event_name == 'pull_request_target' || github.event_name == 'workflow_dispatch'",
+ expect(parsed.jobs.flag.if.trim().replace(/\s+/g, ' ')).toBe(
+ "github.event_name == 'pull_request_target' || github.event_name == 'workflow_dispatch' || (github.event_name == 'issue_comment' && github.event.issue.pull_request != null && (startsWith(github.event.comment.body, '/approve-spec') || startsWith(github.event.comment.body, '/revoke-spec')))",
);
expect(
parsed.jobs.flag.steps.some(step =>
@@ -329,6 +337,87 @@ describe('review-signal trusted helper ref', () => {
);
});
+describe("an engineering owner's owner command re-evaluates the code half (spec:AST-030 FR15)", () => {
+ const approved = [
+ {user: {login: 'engineer'}, state: 'APPROVED', commit_id: HEAD},
+ ];
+
+ it("clears a fork's code-only gate when the owner approved the exact head and then comments", async () => {
+ const h = await runSignal({eventName: 'issue_comment', reviews: approved});
+ expectTrustedHelperRead(h.getContent);
+ expect(h.core.setFailed).not.toHaveBeenCalled();
+ expect(h.github.rest.pulls.get).toHaveBeenCalledExactlyOnceWith({
+ owner: 'meta',
+ repo: 'astryx',
+ pull_number: 42,
+ });
+ expect(h.mutations.createCommitStatus).toHaveBeenCalledWith(
+ expect.objectContaining({
+ sha: HEAD,
+ context: 'review-required',
+ state: 'success',
+ }),
+ );
+ expect(h.mutations.removeLabel).toHaveBeenCalledWith(
+ expect.objectContaining({name: 'needs:code-review'}),
+ );
+ });
+
+ it('keeps the gate pending when the approval is on an older head', async () => {
+ const h = await runSignal({
+ eventName: 'issue_comment',
+ reviews: [{...approved[0], commit_id: OLD_BASE}],
+ });
+ expect(h.mutations.createCommitStatus).toHaveBeenCalledWith(
+ expect.objectContaining({sha: HEAD, state: 'pending'}),
+ );
+ });
+
+ it("changes nothing for a command from someone who isn't an engineering owner", async () => {
+ const h = await runSignal({
+ eventName: 'issue_comment',
+ commenter: 'contributor',
+ reviews: approved,
+ });
+ expect(h.github.rest.pulls.get).not.toHaveBeenCalled();
+ for (const mutation of Object.values(h.mutations)) {
+ expect(mutation).not.toHaveBeenCalled();
+ }
+ });
+
+ it('fails closed when the engineering owner list is unset, never a silent no-op', async () => {
+ await expect(
+ runSignal({eventName: 'issue_comment', engOwners: '', reviews: approved}),
+ ).rejects.toThrow(/Owner variable ENG_OWNERS is empty or unset/);
+ });
+
+ it("never cancels a pull request event's run: a command run has its own group", () => {
+ const group = parsed.jobs.flag.concurrency.group;
+ expect(group).toContain(
+ "github.event_name == 'issue_comment' && format('command-{0}', github.run_id)",
+ );
+ expect(group).not.toContain('github.event.issue.number');
+ expect(parsed.jobs.flag.concurrency['cancel-in-progress']).toBe(true);
+ });
+
+ it('starts no runner for a comment that is not an owner command, and runs no pull-request code', () => {
+ const condition = parsed.jobs.flag.if;
+ expect(condition).toContain('github.event.issue.pull_request != null');
+ expect(condition).toContain(
+ "startsWith(github.event.comment.body, '/approve-spec')",
+ );
+ expect(condition).toContain(
+ "startsWith(github.event.comment.body, '/revoke-spec')",
+ );
+ // The job token can write; nothing from the pull request runs with it.
+ expect(
+ parsed.jobs.flag.steps.some(
+ step => step.uses?.startsWith('actions/checkout@') || step.run,
+ ),
+ ).toBe(false);
+ });
+});
+
describe('review-signal appearance-only contract', () => {
it('keeps the embedded privileged script syntactically valid', () => {
const parsed = YAML.parse(workflow);
diff --git a/.github/workflows/review-signal.yml b/.github/workflows/review-signal.yml
index 72f999924a..c6a41fcfff 100644
--- a/.github/workflows/review-signal.yml
+++ b/.github/workflows/review-signal.yml
@@ -51,7 +51,10 @@
# appearance-only Core runtime changes.
#
# Uses pull_request_target so it can label / request reviewers / disable
-# auto-merge on fork PRs. It reads the PR file list, diff text, and changed Core
+# auto-merge on fork PRs. An engineering owner's owner command
+# (`/approve-spec`, `/revoke-spec`, an issue_comment in the base context) also
+# runs it, so the command re-evaluates the code half on a fork PR whose review
+# event GitHub held for approval. It reads the PR file list, diff text, and changed Core
# base/head source bytes as inert API data. The approval helper comes from the
# trusted workflow SHA; the visual classifier comes from the PR base ref. It reads
# eng/design owner lists from repository VARIABLES (vars.ENGOWNERS /
@@ -69,8 +72,10 @@ on:
types: [opened, synchronize, reopened, ready_for_review, converted_to_draft]
pull_request_review:
types: [submitted, dismissed]
- # Owner commands for the spec-owner job. The job-level filter skips every
- # other comment before a runner starts.
+ # Owner commands for the spec-owner job, which also run the flag job so the
+ # command re-evaluates both halves of the gate (a fork's review event can be
+ # held for approval). The job-level filters skip every other comment before a
+ # runner starts.
issue_comment:
types: [created]
# Manual re-trigger. The App only auto-runs on the PR events above, so if it
@@ -106,12 +111,18 @@ jobs:
if: >-
github.event_name == 'pull_request_target'
|| github.event_name == 'workflow_dispatch'
+ || (github.event_name == 'issue_comment'
+ && github.event.issue.pull_request != null
+ && (startsWith(github.event.comment.body, '/approve-spec')
+ || startsWith(github.event.comment.body, '/revoke-spec')))
name: Flag review signals
runs-on: ubuntu-slim
# Per PR (run-scoped for backfill-all). Only this job cancels: a cancelled
- # spec-owner reconcile could leave its head without a terminal status.
+ # spec-owner reconcile could leave its head without a terminal status. An
+ # owner-command run is run-scoped too, so a comment never cancels a pull
+ # request event's run (which may be the only writer of a new head's gate).
concurrency:
- group: review-signal-${{ github.event.pull_request.number || github.event.inputs.pr || github.run_id }}
+ group: review-signal-${{ github.event_name == 'issue_comment' && format('command-{0}', github.run_id) || github.event.pull_request.number || github.event.inputs.pr || github.run_id }}
cancel-in-progress: true
permissions:
pull-requests: write
@@ -157,6 +168,21 @@ jobs:
const DESIGN = process.env.DESIGN_LABEL;
const COMMUNITY = process.env.COMMUNITY_LABEL;
+ // Owner handles from a repository variable. An empty or unset
+ // list throws: it is a configuration error, never "no owners"
+ // (see flagOne).
+ function parseHandles(text) {
+ return ((text || '').match(/@?[A-Za-z0-9-]+/g) || [])
+ .map((h) => h.replace(/^@/, '').toLowerCase());
+ }
+ function ownersFromVar(name) {
+ const handles = parseHandles(process.env[name]);
+ if (!handles.length) {
+ throw new Error(`Owner variable ${name} is empty or unset — refusing to bucket authors from an empty list.`);
+ }
+ return handles;
+ }
+
// Resolve which PR(s) to flag. On a PR event we get the payload PR.
// On manual dispatch we take the `pr` input (one PR), or re-flag
// every open PR when it's blank (backfill / mass recovery).
@@ -193,6 +219,21 @@ jobs:
`${open.length} open PR(s); ${open.length - ready.length} draft(s) skipped.`,
);
}
+ } else if (context.eventName === 'issue_comment') {
+ // An engineering owner's owner command re-evaluates the code
+ // half from the PR's reviews: GitHub can hold a fork's
+ // pull_request_review run for approval, so the approval alone
+ // may never reach the gate. Anyone else's command changes
+ // nothing here.
+ const commenter = (context.payload.comment?.user?.login || '').toLowerCase();
+ if (!ownersFromVar('ENG_OWNERS').includes(commenter)) {
+ core.info(`@${commenter} is not an engineering owner; the code half is unchanged.`);
+ return;
+ }
+ const { data } = await github.rest.pulls.get({
+ owner, repo, pull_number: context.payload.issue.number,
+ });
+ prs = [data];
} else {
prs = [context.payload.pull_request];
}
@@ -254,17 +295,6 @@ jobs:
// per-PR try/catch then skips this PR, LEAVES ITS GATE UNTOUCHED
// (never a false success), and fails the run visibly. An unreadable
// owner list is never "no owners".
- function parseHandles(text) {
- return ((text || '').match(/@?[A-Za-z0-9-]+/g) || [])
- .map((h) => h.replace(/^@/, '').toLowerCase());
- }
- function ownersFromVar(name) {
- const handles = parseHandles(process.env[name]);
- if (!handles.length) {
- throw new Error(`Owner variable ${name} is empty or unset — refusing to bucket authors from an empty list.`);
- }
- return handles;
- }
const engOwners = ownersFromVar('ENG_OWNERS');
const designOwners = ownersFromVar('DESIGN_OWNERS');
const authorIsEng = engOwners.includes(author);
diff --git a/docs/specs/AST-030/spec.md b/docs/specs/AST-030/spec.md
index 8881f00f36..7d68eff217 100644
--- a/docs/specs/AST-030/spec.md
+++ b/docs/specs/AST-030/spec.md
@@ -322,10 +322,13 @@ they are separately named.
pending, saying the pull request is too large to reconcile automatically; a
maintainer-dispatched run is exempt from the budget and decides normally. It
skips fork review events, whose tokens are read-only; fork approvals
- use the owner command. Only the review-signal job keeps a per-pull-request
- cancel group. Each writer of `review-required` composes both halves and links
- a spec wait to the spec decision's details. Comments that are not owner
- commands start no runner, and no scheduled job reads approvals.
+ use the owner command. An engineering owner's owner command re-evaluates
+ both halves of `review-required` from the head it reads, including the code
+ review, so a fork's approval takes effect through it. Only the review-signal
+ job keeps a per-pull-request cancel group. Each writer of `review-required`
+ composes both halves and links a spec wait to the spec decision's details.
+ Comments that are not owner commands start no runner, and no scheduled job
+ reads approvals.
- **FR16 — One author's pull requests share the heavy suites.** On a pull
request, the heavy browser suites (the interaction proofs and the Build
package's browser and fixture-contrast contracts) run only when the pull
@@ -554,7 +557,7 @@ The nightly perf lane is the one scheduled measurement outside them (FR18).
| FR12 | `touched-components.test.mjs` changed-spec routing; `a11y-pr-contracts.test.mjs` `changedOnly` cases | a pull request that adds a browser accessibility spec; one that changes an `always` spec; one that changes a `release-only` or an `audit` spec; one that changes more spec files than the limit; a component change with no spec change | a pull request that adds a browser accessibility spec that fails is not caught on that pull request, an unchanged, `release-only` or `audit` spec runs in the default check, a file over the limit is reported as passed, or the default check widens past the changed spec files |
| FR13 | classifier, workflow contract, component-analysis and RTL-scope tests | component change; shared module; Changeset beside code; infrastructure path; incomplete list; no build | an infrastructure path is scoped, a file-system guard test is skipped, a reverse dependent leaves visual/RTL or enters PR accessibility, or a full-suite event is narrowed |
| FR14 | `workflow-paths.test.mjs` over every workflow | path allowlist; required-context workflow | a `paths-ignore`, a match-all or exclusion glob, or a path filter on a workflow that reports a required context fails |
-| FR15 | workflow contract and reconciler tests | ENGOWNER spec PR; owner command; other comment; fork review; non-spec PR; run over the request budget; maintainer dispatch | `review-required` succeeds while spec approval is missing, another workflow publishes a review context, a fork review starts the spec checks, a non-command comment starts a runner, or an over-budget run decides |
+| FR15 | workflow contract and reconciler tests | ENGOWNER spec PR; owner command; other comment; fork review; non-spec PR; run over the request budget; maintainer dispatch; fork PR, code-only: an owner's exact-head approval plus the owner command clears the code half | `review-required` succeeds while spec approval is missing, another workflow publishes a review context, a fork review starts the spec checks, a non-command comment starts a runner, an over-budget run decides, or a fork's approved code-only PR stays pending after the owner command |
| FR16 | `heavy-caps-action.test.mjs` cap cases and heavy-job contract; `ci-admission.test.mjs` | sixteenth-oldest PR; 101st heavy run of the day, beside cancelled, light and other-branch runs; agent lane and its five-PR and forty-run backstops; an engineering owner, including the shared login past those backstops, a commented-out or trailing-comment handle and an unreadable roster; bot; the deciding run; re-run; failed lookup; a run with room, without room, past its longest wait, beside a waiting `main` check, as a draft or unlabelled fork, with opt-in labels, beside small workflows past their measured most, and with the API unreadable; each workflow's most jobs; every required context's rule | a heavy suite that runs ahead of the author's fifteen oldest PRs, or past the daily cap, a capped job that passes, an engineering owner or a re-run that is held, or a roster read from the pull request's head, a run that starts without room before its longest wait, a pull-request run that starts while `main` waits, a job admission could skip, or a peak below the jobs its run holds fails |
| FR17 | `ci-pr-gate.test.mjs` over every `pull_request` workflow and the shared conditions | branch here, fork with and without `ci:run`, another label, draft with the label, ready, push, merge group, dispatch | a job that can start on a pull request without the condition, a label trigger, a required job that skips for a pull request that may not run CI, or a draft or unlabelled fork that runs fails |
| FR18 | `perf-nightly-workflow.test.mjs`, `perf-trend.test.mjs` and `run-labels.test.mjs` | nightly and dispatch; labelled, unlabelled, draft and unvouched-fork pull requests; write, read and no access; newest label event; failed lookup; 10% boundary; metric floors; seven-day window edges; fewer than three runs; a metric over once, twice in a row, or a different metric before; recorded and pull request runs | a run on an unlabelled pull request, a label honoured from read access, a regression called at exactly 10%, under its floor or on one run alone, a run outside the window in the baseline, a pull request run that records, or a second regression issue fails |