Skip to content

An open source design system that's fully customizable and agent ready

TypeScript

feat(review-signal): a backfill skips drafts and splits into batches (#9104)

Cindy Zhang786c16e2e897
diff --git a/.github/REVIEW_GATE.md b/.github/REVIEW_GATE.md
index 2d5ce948d8..a1cb9b8f72 100644
--- a/.github/REVIEW_GATE.md
+++ b/.github/REVIEW_GATE.md
@@ -163,8 +163,8 @@ bypass.)
### The mirror
`spec-owner-approval` now only mirrors the spec half of `review-required`.
-First re-flag every open PR (`gh workflow run review-signal.yml` with a blank
-`pr`) so each `review-required` carries its spec half; then remove the mirror
+First re-flag every open, non-draft PR (the blank-`pr` backfill, in batches;
+see below) so each `review-required` carries its spec half; then remove the mirror
from the required status checks on `main`. To re-publish a head's statuses, run
`gh workflow run review-signal.yml -f pr=<number>`.
@@ -186,7 +186,13 @@ to failure).
`review-signal.yml` has a `workflow_dispatch`:
-- blank `pr` input → re-flag **all** open PRs (backfill / mass recovery)
+- blank `pr` input → re-flag every open, **non-draft** PR (backfill / mass
+ recovery). A draft is re-flagged when it becomes ready for review.
+- blank `pr` plus `batch` `k/n` → re-flag only the open, non-draft PRs whose
+ number mod n is k−1. One job has 15 minutes, about 100 PRs, so split a large
+ backfill: `gh workflow run review-signal.yml -f batch=1/4`, then `2/4`, `3/4`
+ and `4/4`. A backfill also stops, failing with where it stopped, before the
+ job token's hourly API budget runs low; re-run after the reset it names.
- a PR number → re-flag just that one
Use it if a PR's `review-required` ever gets stuck (e.g. the workflow was added
diff --git a/.github/scripts/review-signal-workflow.test.mjs b/.github/scripts/review-signal-workflow.test.mjs
index 05d8015d04..915082538e 100644
--- a/.github/scripts/review-signal-workflow.test.mjs
+++ b/.github/scripts/review-signal-workflow.test.mjs
@@ -36,6 +36,8 @@ async function runSignal({
reviewApp = '',
permission = 'write',
rateRemaining = 5000,
+ batch = '',
+ draft = false,
// The spec-owner job's decision for a PR that changes no spec record.
specStatuses = [
{
@@ -52,6 +54,7 @@ async function runSignal({
base: {sha: baseSha, ref: 'main'},
head: {sha: HEAD, ref: 'feature'},
changed_files: 1,
+ draft,
labels: [{name: 'needs:code-review'}, {name: 'community'}],
};
const mutations = {
@@ -145,7 +148,7 @@ async function runSignal({
eventName,
payload:
eventName === 'workflow_dispatch'
- ? {inputs: {pr: backfill ? '' : '42'}}
+ ? {inputs: {pr: backfill ? '' : '42', batch}}
: {pull_request: pr},
},
core,
@@ -228,6 +231,37 @@ describe('review-signal trusted helper ref', () => {
},
);
+ it.each([
+ ['its batch', {batch: '1/2'}, true],
+ ['another batch', {batch: '2/2'}, false],
+ ['a draft', {draft: true}, false],
+ ])(
+ 'a backfill flags a PR in %s only when it should',
+ async (_name, options, flagged) => {
+ const h = await runSignal({
+ eventName: 'workflow_dispatch',
+ backfill: true,
+ ...options,
+ });
+ expect(h.core.setFailed).not.toHaveBeenCalled();
+ expect(h.mutations.createCommitStatus).toHaveBeenCalledTimes(
+ flagged ? 1 : 0,
+ );
+ },
+ );
+
+ it('refuses a malformed batch before reading any PR', async () => {
+ const h = await runSignal({
+ eventName: 'workflow_dispatch',
+ backfill: true,
+ batch: '3/2',
+ });
+ expect(h.core.setFailed).toHaveBeenCalledExactlyOnceWith(
+ 'batch must be k/n with 1 <= k <= n; got "3/2".',
+ );
+ expect(h.github.rest.pulls.get).not.toHaveBeenCalled();
+ });
+
it('stops a backfill before the token runs low, and says so', async () => {
const h = await runSignal({
eventName: 'workflow_dispatch',
diff --git a/.github/workflows/review-signal.yml b/.github/workflows/review-signal.yml
index 647a22a8d0..85fa65a4ed 100644
--- a/.github/workflows/review-signal.yml
+++ b/.github/workflows/review-signal.yml
@@ -76,12 +76,17 @@ on:
# Manual re-trigger. The App only auto-runs on the PR events above, so if it
# ever misses one (e.g. the workflow was added after a PR was opened, leaving
# the required "review-required" check stuck as Expected/pending), this is the
- # recovery lever. Leave `pr` blank to re-flag EVERY open PR (backfill); pass a
- # number to re-flag just that one.
+ # recovery lever. Leave `pr` blank to re-flag EVERY open, non-draft PR
+ # (backfill); pass a number to re-flag just that one. A backfill too large for
+ # one job splits with `batch` k/n (PRs whose number mod n is k-1).
workflow_dispatch:
inputs:
pr:
- description: 'PR number to re-flag (blank = all open PRs)'
+ description: 'PR number to re-flag (blank = every open, non-draft PR)'
+ required: false
+ type: string
+ batch:
+ description: 'With a blank pr: re-flag only PRs whose number mod n is k-1 (k/n, e.g. 2/4)'
required: false
type: string
@@ -143,10 +148,29 @@ jobs:
});
prs = [data];
} else {
- prs = await github.paginate(github.rest.pulls.list, {
+ const open = await github.paginate(github.rest.pulls.list, {
owner, repo, state: 'open', per_page: 100,
});
- core.info(`Backfill: re-flagging ${prs.length} open PR(s).`);
+ // A draft is re-flagged when it becomes ready
+ // (ready_for_review), so a backfill skips it.
+ const ready = open.filter(listed => !listed.draft);
+ // A shard keys on the PR number, so it stays the same set
+ // while PRs open and close between dispatches.
+ const batch = (context.payload.inputs.batch || '').trim();
+ prs = ready;
+ if (batch) {
+ const match = /^(\d+)\/(\d+)$/.exec(batch);
+ const [k, n] = match ? [Number(match[1]), Number(match[2])] : [0, 0];
+ if (!(n >= 1 && k >= 1 && k <= n)) {
+ core.setFailed(`batch must be k/n with 1 <= k <= n; got "${batch}".`);
+ return;
+ }
+ prs = ready.filter(listed => listed.number % n === k - 1);
+ }
+ core.info(
+ `Backfill${batch ? ` ${batch}` : ''}: re-flagging ${prs.length} of ` +
+ `${open.length} open PR(s); ${open.length - ready.length} draft(s) skipped.`,
+ );
}
} else {
prs = [context.payload.pull_request];