Skip to content

armgit / macro

Public Imported from GitHub
0

Macro is a unified workspace for teams: email, chat, docs, tasks, agents, calls, and CRM — @-linked together with shared AI memory.

ci: run PR checks on request with the run-ci label (#9887)

teoafc1e4e1a47a
diff --git a/.agents/skills/git/SKILL.md b/.agents/skills/git/SKILL.md
index 7b5241d8a8..d880424bb2 100644
--- a/.agents/skills/git/SKILL.md
+++ b/.agents/skills/git/SKILL.md
@@ -140,8 +140,10 @@ when available). Do not commit verification captures unless explicitly requested
## Handing off a PR
Open PRs ready for review, not as drafts; if one was created as a draft, mark it
-ready. A PR whose checks are failing or still running is not done: watch CI and fix
-failures before handing off.
+ready. The `CI gate` passes only after a run requested with the `run-ci` label;
+the [`run-ci` skill](../run-ci/SKILL.md) says when to request one. A PR whose
+`CI gate` is failing, still running, or not run on the latest commit is not done:
+wait on it and fix failures before handing off.
## Updating a PR
diff --git a/.agents/skills/run-ci/SKILL.md b/.agents/skills/run-ci/SKILL.md
new file mode 100644
index 0000000000..31852631b1
--- /dev/null
+++ b/.agents/skills/run-ci/SKILL.md
@@ -0,0 +1,89 @@
+---
+name: run-ci
+description: Run CI on a pull request and wait for its result. Use at a checkpoint (a coherent, locally tested chunk), when you think a PR is done, before asking for review or merging, after fixing a CI failure, and whenever the `CI gate` status says CI has not run.
+---
+
+# Run CI on a pull request
+
+## When to run CI
+
+Run CI deliberately, after the local checks for what you changed pass
+(`just check`, the affected tests):
+
+- at a checkpoint: a coherent, locally tested chunk you want real CI on before
+ building further;
+- when you think the PR is done, and before asking for review or merging;
+- after pushing the fix for a CI failure.
+
+Not after every small push, not while you are still iterating, and never
+instead of local checks: every run costs CI compute.
+
+## How it works
+
+CI runs when someone asks for it, not on every push. Adding the `run-ci` label
+to a PR runs every check on its head commit (the `PR CI` workflow), and the
+label disappears as the run starts. The result is the `CI gate` commit status on
+that commit. The merge gate is `CI gate` passing on the PR's latest commit, so
+every push needs a new run before the PR can merge (see the end of this page
+for the switch-over).
+
+## Run it
+
+Push first, then ask for the run:
+
+```sh
+gh pr create --title "..." --body-file body.md --label run-ci # a new PR
+gh api repos/macro-inc/macro/issues/<pr>/labels -f "labels[]=run-ci" # after a push
+```
+
+`gh pr edit <pr> --add-label run-ci` does the same on a recent gh; older
+releases fail on a Projects (classic) error before adding the label.
+
+- Adding the label again starts a fresh run. A run still going for the same PR
+ is cancelled.
+- Draft PRs run too.
+- Other labels never start or cancel CI.
+- A label added within seconds of a push can reach GitHub before the push
+ does. `Start` then fails within seconds and `CI gate` says to add the label
+ again. To avoid it, add the label once `gh pr view <pr> --json headRefOid`
+ shows the commit you pushed.
+
+## Wait for it
+
+Use the `babysit-pr` skill's watcher: `pr-wait <pr> --until green`. It exits 1
+with the command to run when CI has not run on the head, and exits 1 when
+`CI gate` fails. Never poll with your own loop.
+
+To read the state once, look at the `CI gate` row of `gh pr checks <pr>`:
+
+| `CI gate` | Meaning | Do |
+| --- | --- | --- |
+| pending, `Not run on this commit. Run: ...` | No run on this head | Add the label |
+| pending, `Running on <sha>` | A run is going | Wait |
+| pass, `Passed on <sha>` | Every check passed or was path-skipped | Done |
+| fail, `Failed: <jobs>` | Those called workflows failed | Open the linked run, fix, push, run again |
+| fail, `The PR moved on ...` or `The merge commit did not include ...` | Labelled too soon after a push | Add the label again |
+| missing | A fork PR, before its first run | A maintainer adds the label |
+
+## Fix a failure
+
+- A flaky job: re-run the failed jobs of the `PR CI` run
+ (`gh run rerun <run-id> --failed`). `CI gate` re-runs with them and updates
+ the status; no new label is needed.
+- A real failure: fix it, push, and add the label again.
+- The checks inside the run are named `<workflow> / <job>`, such as
+ `Web App / Typecheck` or `Cloud Storage / test`.
+
+## Fork PRs
+
+A maintainer adds the label after reading the diff, workflow files included:
+the run uses the fork's copies. The fork's token can't set statuses, so
+`CI Gate for Forks` removes the label and sets `CI gate` to `Running on <sha>`
+on the head the label went on, then copies the run's verdict onto that head.
+Re-running a fork's failed jobs doesn't update `CI gate`; add the label again.
+
+## While the switch is in progress
+
+The checks still run on every push as well, and `main` still requires their
+old status checks (`Web App Status Check`, `Cloud Storage Status Check`,
+`Infra Status Check`) until its ruleset requires `CI gate`.
diff --git a/.github/workflows/android-play-release.yml b/.github/workflows/android-play-release.yml
index 4db7b475b2..0a143096ca 100644
--- a/.github/workflows/android-play-release.yml
+++ b/.github/workflows/android-play-release.yml
@@ -204,8 +204,10 @@ jobs:
# (`gh pr checks` names are job names, not workflow names, plus the contexts
# of commit statuses, such as the android-play/* ones Android Play Publish
- # and Android Release Watch set.)
- ours='^(Android Native Check|Android Play Release|Re-run the native check|Web App Status Check|Android Release Drift|iOS Native Check|android-play/)'
+ # and Android Release Watch set. PR CI prefixes the jobs it calls with
+ # `Web App / `. The CI gate status aggregates every check, and stays pending
+ # on a head no one has run PR CI on.)
+ ours='^((Web App / )?(Android Native Check|Web App Status Check|iOS Native Check)|Android Play Release|Re-run the native check|Android Release Drift|android-play/|CI gate$)'
deadline=$(( $(date +%s) + WAIT_MINUTES * 60 ))
while true; do
checks=$(gh pr checks "$PR_NUMBER" --repo "$GH_REPO" --json name,bucket \
diff --git a/.github/workflows/cargo_deny.yml b/.github/workflows/cargo_deny.yml
index 9e8ef3fbd4..557468073c 100644
--- a/.github/workflows/cargo_deny.yml
+++ b/.github/workflows/cargo_deny.yml
@@ -18,6 +18,7 @@ name: cargo deny check
- Cargo.lock
- deny.toml
- '.github/workflows/cargo_deny.yml'
+ workflow_call: {}
jobs:
cargo-deny:
runs-on: namespace-profile-linux-small
@@ -29,5 +30,5 @@ jobs:
with:
manifest-path: Cargo.toml
concurrency:
- group: cargo-deny-${{ github.ref }}
+ group: cargo-deny-${{ github.event.action == 'labeled' && github.run_id || github.ref }}
cancel-in-progress: true
diff --git a/.github/workflows/cargo_workspace_dependency_check.yml b/.github/workflows/cargo_workspace_dependency_check.yml
index 4da7b7f985..c5cb771eda 100644
--- a/.github/workflows/cargo_workspace_dependency_check.yml
+++ b/.github/workflows/cargo_workspace_dependency_check.yml
@@ -22,6 +22,7 @@ name: cargo workspace dependency check
- tooling/xtask/crates/xtask_deps/**
- tooling/xtask/crates/xtask_graph/**
- '.github/workspace-dep-closures.json'
+ workflow_call: {}
jobs:
dependency-check:
runs-on: namespace-profile-linux-rust-ci;overrides.cache-tag=sccache-ci
@@ -147,5 +148,5 @@ jobs:
if: always()
uses: './.github/actions/teardown-nix'
concurrency:
- group: cargo-workspace-dependency-check-${{ github.ref }}
+ group: cargo-workspace-dependency-check-${{ github.event.action == 'labeled' && github.run_id || github.ref }}
cancel-in-progress: true
diff --git a/.github/workflows/check_generated.yml b/.github/workflows/check_generated.yml
index a459d2ac3e..d28ebbec19 100644
--- a/.github/workflows/check_generated.yml
+++ b/.github/workflows/check_generated.yml
@@ -10,6 +10,7 @@ name: check generated workflows
- crates/macro_event_topics/**
- '.github/workflows/**'
- '.github/kafka-cluster-topics.json'
+ workflow_call: {}
jobs:
check-workflows:
runs-on: namespace-profile-linux-small
diff --git a/.github/workflows/check_node_modules_nix.yml b/.github/workflows/check_node_modules_nix.yml
index c7e60a7a9e..a4ab812365 100644
--- a/.github/workflows/check_node_modules_nix.yml
+++ b/.github/workflows/check_node_modules_nix.yml
@@ -20,6 +20,7 @@ name: check node_modules nix
- flake.nix
- flake.lock
- '.github/workflows/check_node_modules_nix.yml'
+ workflow_call: {}
permissions:
contents: read
jobs:
diff --git a/.github/workflows/check_tauri_lockfiles.yml b/.github/workflows/check_tauri_lockfiles.yml
index b25dd233bc..44bc835b95 100644
--- a/.github/workflows/check_tauri_lockfiles.yml
+++ b/.github/workflows/check_tauri_lockfiles.yml
@@ -13,6 +13,7 @@ name: check tauri lockfiles
- flake.nix
- flake.lock
- '.github/workflows/check_tauri_lockfiles.yml'
+ workflow_call: {}
permissions:
contents: read
jobs:
diff --git a/.github/workflows/ci-gate-fork.yml b/.github/workflows/ci-gate-fork.yml
new file mode 100644
index 0000000000..9ba9c9c3ea
--- /dev/null
+++ b/.github/workflows/ci-gate-fork.yml
@@ -0,0 +1,605 @@
+# DO NOT EDIT — regenerate with `cargo x workflows` (from the repository root).
+# Source: tooling/xtask/crates/xtask_workflows/src/workflows/ci_gate_fork.rs
+name: CI Gate for Forks
+'on':
+ pull_request_target:
+ types:
+ - labeled
+ branches:
+ - main
+ workflow_run:
+ types:
+ - completed
+ workflows:
+ - PR CI
+jobs:
+ request:
+ if: github.event_name == 'pull_request_target' && github.event.label.name == 'run-ci' && github.event.pull_request.head.repo.full_name != github.repository
+ name: Take a fork's CI request
+ runs-on: ubuntu-latest
+ permissions:
+ issues: write
+ pull-requests: write
+ statuses: write
+ timeout-minutes: 5
+ steps:
+ - name: Open the CI gate request
+ run: |
+ node --disable-warning=ExperimentalWarning --input-type=module-typescript <<'NODE_SCRIPT'
+ // Sets the commit status `main` requires before a merge: it is green only on a
+ // commit that a requested `PR CI` run passed. `CI_GATE_MODE` picks the step:
+ //
+ // remind a new commit: pending "not run" with the command to run it, unless
+ // the commit already has a gate status
+ // start a requested run begins: remove the label, check the merge commit
+ // holds the PR head, set pending
+ // finish the run ends: success when every job passed or was path-skipped
+ // fork-request the label goes on a fork PR: what start does with a write
+ // token, from a trusted run of `main`'s workflow
+ // fork after a fork PR's run, whose read-only token could not set the status
+ //
+ // Statuses go on the PR head commit, so a later push starts without one.
+
+ import { readFileSync } from 'node:fs';
+
+ const API = 'https://api.github.com';
+ const DESCRIPTION_LIMIT = 140;
+ // An open request. Only trusted runs set statuses: a fork's token can't.
+ const RUNNING = 'Running on';
+ // Job names in `PR CI`; the fork step reads them back from the finished run.
+ export const START_JOB = 'Start';
+ export const GATE_JOB = 'CI gate';
+
+ type State = 'pending' | 'success' | 'failure' | 'error';
+ export type Needs = Record<string, { result: string }>;
+ export type RunJob = { name: string; conclusion: string | null };
+ type Outcome = { state: State; description: string } | null;
+
+ function env(name: string): string {
+ const value = process.env[name];
+ if (!value) {
+ throw new Error(`${name} is not set`);
+ }
+ return value;
+ }
+
+ async function github(method: string, path: string, body?: unknown): Promise<any> {
+ const response = await fetch(`${API}${path}`, {
+ method,
+ headers: {
+ accept: 'application/vnd.github+json',
+ authorization: `Bearer ${env('GH_TOKEN')}`,
+ 'x-github-api-version': '2022-11-28',
+ },
+ body: body === undefined ? undefined : JSON.stringify(body),
+ });
+ // Removing a label that is already gone is the outcome we wanted.
+ if (method === 'DELETE' && response.status === 404) {
+ return null;
+ }
+ if (!response.ok) {
+ throw new Error(`${method} ${path}: HTTP ${response.status} ${await response.text()}`);
+ }
+ return response.status === 204 ? null : response.json();
+ }
+
+ const repo = () => env('GITHUB_REPOSITORY');
+ const context = () => env('CI_GATE_CONTEXT');
+ const label = () => env('CI_GATE_LABEL');
+
+ /// REST, because `gh pr edit` fails on older gh releases.
+ export function runCommand(repository: string, pr: string, labelName: string): string {
+ return `gh api repos/${repository}/issues/${pr}/labels -f "labels[]=${labelName}"`;
+ }
+
+ /// The finished run's verdict from `toJSON(needs)`. Start must have succeeded;
+ /// a called workflow may be skipped, because its path filter did not match.
+ /// Null when start failed: it already set a status that says why.
+ export function gateOutcome(needs: Needs, head: string): Outcome {
+ if (needs.start?.result !== 'success') {
+ return null;
+ }
+
+ const failed = Object.entries(needs)
+ .filter(([, need]) => need.result !== 'success' && need.result !== 'skipped')
+ .map(([job]) => job);
+
+ if (failed.length > 0) {
+ return { state: 'failure', description: `Failed: ${failed.join(', ')}` };
+ }
+
+ return { state: 'success', description: `Passed on ${head.slice(0, 7)}` };
+ }
+
+ /// A fork run's verdict from its jobs. Null when the run was not a request
+ /// (unrelated label) or did not finish (cancelled).
+ export function forkOutcome(jobs: RunJob[], head: string, labelName: string): Outcome {
+ const start = jobs.find((job) => job.name === START_JOB);
+ const gate = jobs.find((job) => job.name === GATE_JOB);
+
+ if (!start || start.conclusion === 'skipped' || start.conclusion === 'cancelled') {
+ return null;
+ }
+
+ if (start.conclusion !== 'success') {
+ return {
+ state: 'failure',
+ description: `Could not start on ${head.slice(0, 7)}. Add the ${labelName} label again.`,
+ };
+ }
+
+ if (gate?.conclusion === 'success') {
+ return { state: 'success', description: `Passed on ${head.slice(0, 7)}` };
+ }
+
+ if (gate?.conclusion === 'failure') {
+ return { state: 'failure', description: 'Failed: see the PR CI run' };
+ }
+
+ return null;
+ }
+
+ async function setStatus(sha: string, state: State, description: string, targetUrl: string) {
+ await github('POST', `/repos/${repo()}/statuses/${sha}`, {
+ state,
+ context: context(),
+ description: description.slice(0, DESCRIPTION_LIMIT),
+ target_url: targetUrl,
+ });
+ console.log(`${context()} on ${sha.slice(0, 10)}: ${state}: ${description}`);
+ }
+
+ /// The newest gate status on a commit (GitHub lists statuses newest first).
+ async function gateStatus(sha: string): Promise<{ state: string; description: string } | undefined> {
+ const statuses: { context: string; state: string; description: string | null }[] = await github(
+ 'GET',
+ `/repos/${repo()}/commits/${sha}/statuses?per_page=100`,
+ );
+ const status = statuses.find((each) => each.context === context());
+ return status && { state: status.state, description: status.description ?? '' };
+ }
+
+ /// Whether a trusted step opened a request on this commit that no result has
+ /// closed yet.
+ export function isOpenRequest(status: { state: string; description: string } | undefined): boolean {
+ return status?.state === 'pending' && status.description.startsWith(RUNNING);
+ }
+
+ async function removeLabel(pr: string) {
+ await github('DELETE', `/repos/${repo()}/issues/${pr}/labels/${encodeURIComponent(label())}`);
+ }
+
+ async function remind() {
+ const head = env('HEAD_SHA');
+ const pr = env('PR_NUMBER');
+
+ // A re-open or leaving draft keeps the head, so it may have a result already.
+ if (await gateStatus(head)) {
+ console.log(`${head.slice(0, 10)} already has a ${context()} status`);
+ return;
+ }
+
+ await setStatus(
+ head,
+ 'pending',
+ `Not run on this commit. Run: ${runCommand(repo(), pr, label())}`,
+ env('CI_GATE_DOCS_URL'),
+ );
+ }
+
+ async function start() {
+ const head = env('HEAD_SHA');
+ const pr = env('PR_NUMBER');
+ const runUrl = env('RUN_URL');
+ const sameRepo = env('SAME_REPO') === 'true';
+
+ // Fork tokens are read-only; the fork request step did both already.
+ if (sameRepo) {
+ await removeLabel(pr);
+ }
+
+ // A label added right after a push can carry the previous head, or a merge
+ // commit GitHub has not rebuilt yet. Either would test an old commit.
+ const pull = await github('GET', `/repos/${repo()}/pulls/${pr}`);
+ if (pull.head.sha !== head) {
+ const description = `The PR moved on from ${head.slice(0, 7)} to ${pull.head.sha.slice(0, 7)}. Run again: ${runCommand(repo(), pr, label())}`;
+ if (sameRepo) {
+ await setStatus(head, 'failure', description, runUrl);
+ }
+ throw new Error(description);
+ }
+
+ const merge = await github('GET', `/repos/${repo()}/commits/${env('MERGE_SHA')}`);
+ const parents: string[] = merge.parents.map((parent: { sha: string }) => parent.sha);
+
+ if (!parents.includes(head)) {
+ const description = `The merge commit did not include ${head.slice(0, 7)} yet. Run again: ${runCommand(repo(), pr, label())}`;
+ if (sameRepo) {
+ await setStatus(head, 'failure', description, runUrl);
+ }
+ throw new Error(description);
+ }
+
+ if (sameRepo) {
+ await setStatus(head, 'pending', `${RUNNING} ${head.slice(0, 7)}`, runUrl);
+ }
+ }
+
+ async function finish() {
+ const head = env('HEAD_SHA');
+ const outcome = gateOutcome(JSON.parse(env('NEEDS')), head);
+
+ if (!outcome) {
+ console.log('Start did not succeed; its status stands.');
+ process.exitCode = 1;
+ return;
+ }
+
+ if (env('SAME_REPO') === 'true') {
+ await setStatus(head, outcome.state, outcome.description, env('RUN_URL'));
+ }
+
+ if (outcome.state !== 'success') {
+ console.log(outcome.description);
+ process.exitCode = 1;
+ }
+ }
+
+ async function forkRequest() {
+ const head = env('HEAD_SHA');
+ const pr = env('PR_NUMBER');
+
+ await removeLabel(pr);
+ await setStatus(head, 'pending', `${RUNNING} ${head.slice(0, 7)}`, env('PR_CHECKS_URL'));
+ }
+
+ async function fork() {
+ const event = JSON.parse(readFileSync(env('GITHUB_EVENT_PATH'), 'utf8'));
+ const run = event.workflow_run;
+ const head: string = run.head_sha;
+
+ // The request step opened this on the head the label was added at. A run on
+ // any other commit, such as one the fork pushed with a rewritten workflow,
+ // or a run after the result is in, finds no open request.
+ if (!isOpenRequest(await gateStatus(head))) {
+ console.log(`${head.slice(0, 10)} has no open ${context()} request; nothing to report.`);
+ return;
+ }
+
+ const jobs: RunJob[] = [];
+ for (let page = 1; ; page++) {
+ const { jobs: batch } = await github(
+ 'GET',
+ `/repos/${repo()}/actions/runs/${run.id}/attempts/${run.run_attempt}/jobs?per_page=100&page=${page}`,
+ );
+ jobs.push(...batch);
+ if (batch.length < 100) {
+ break;
+ }
+ }
+
+ const outcome = forkOutcome(jobs, head, label());
+ if (outcome) {
+ await setStatus(head, outcome.state, outcome.description, run.html_url);
+ }
+ }
+
+ export async function main() {
+ const modes: Record<string, () => Promise<void>> = {
+ remind,
+ start,
+ finish,
+ 'fork-request': forkRequest,
+ fork,
+ };
+ const mode = modes[env('CI_GATE_MODE')];
+ if (!mode) {
+ throw new Error(`unknown CI_GATE_MODE ${process.env.CI_GATE_MODE}`);
+ }
+ await mode();
+ }
+
+ await main();
+ NODE_SCRIPT
+ shell: bash
+ env:
+ CI_GATE_MODE: fork-request
+ CI_GATE_CONTEXT: CI gate
+ CI_GATE_LABEL: run-ci
+ GH_TOKEN: ${{ github.token }}
+ PR_NUMBER: ${{ github.event.pull_request.number }}
+ HEAD_SHA: ${{ github.event.pull_request.head.sha }}
+ PR_CHECKS_URL: ${{ github.event.pull_request.html_url }}/checks
+ report:
+ if: github.event_name == 'workflow_run' && github.event.workflow_run.event == 'pull_request' && github.event.workflow_run.head_repository.full_name != github.repository
+ name: Report a fork's CI result
+ runs-on: ubuntu-latest
+ permissions:
+ actions: read
+ statuses: write
+ timeout-minutes: 5
+ steps:
+ - name: Set the CI gate status
+ run: |
+ node --disable-warning=ExperimentalWarning --input-type=module-typescript <<'NODE_SCRIPT'
+ // Sets the commit status `main` requires before a merge: it is green only on a
+ // commit that a requested `PR CI` run passed. `CI_GATE_MODE` picks the step:
+ //
+ // remind a new commit: pending "not run" with the command to run it, unless
+ // the commit already has a gate status
+ // start a requested run begins: remove the label, check the merge commit
+ // holds the PR head, set pending
+ // finish the run ends: success when every job passed or was path-skipped
+ // fork-request the label goes on a fork PR: what start does with a write
+ // token, from a trusted run of `main`'s workflow
+ // fork after a fork PR's run, whose read-only token could not set the status
+ //
+ // Statuses go on the PR head commit, so a later push starts without one.
+
+ import { readFileSync } from 'node:fs';
+
+ const API = 'https://api.github.com';
+ const DESCRIPTION_LIMIT = 140;
+ // An open request. Only trusted runs set statuses: a fork's token can't.
+ const RUNNING = 'Running on';
+ // Job names in `PR CI`; the fork step reads them back from the finished run.
+ export const START_JOB = 'Start';
+ export const GATE_JOB = 'CI gate';
+
+ type State = 'pending' | 'success' | 'failure' | 'error';
+ export type Needs = Record<string, { result: string }>;
+ export type RunJob = { name: string; conclusion: string | null };
+ type Outcome = { state: State; description: string } | null;
+
+ function env(name: string): string {
+ const value = process.env[name];
+ if (!value) {
+ throw new Error(`${name} is not set`);
+ }
+ return value;
+ }
+
+ async function github(method: string, path: string, body?: unknown): Promise<any> {
+ const response = await fetch(`${API}${path}`, {
+ method,
+ headers: {
+ accept: 'application/vnd.github+json',
+ authorization: `Bearer ${env('GH_TOKEN')}`,
+ 'x-github-api-version': '2022-11-28',
+ },
+ body: body === undefined ? undefined : JSON.stringify(body),
+ });
+ // Removing a label that is already gone is the outcome we wanted.
+ if (method === 'DELETE' && response.status === 404) {
+ return null;
+ }
+ if (!response.ok) {
+ throw new Error(`${method} ${path}: HTTP ${response.status} ${await response.text()}`);
+ }
+ return response.status === 204 ? null : response.json();
+ }
+
+ const repo = () => env('GITHUB_REPOSITORY');
+ const context = () => env('CI_GATE_CONTEXT');
+ const label = () => env('CI_GATE_LABEL');
+
+ /// REST, because `gh pr edit` fails on older gh releases.
+ export function runCommand(repository: string, pr: string, labelName: string): string {
+ return `gh api repos/${repository}/issues/${pr}/labels -f "labels[]=${labelName}"`;
+ }
+
+ /// The finished run's verdict from `toJSON(needs)`. Start must have succeeded;
+ /// a called workflow may be skipped, because its path filter did not match.
+ /// Null when start failed: it already set a status that says why.
+ export function gateOutcome(needs: Needs, head: string): Outcome {
+ if (needs.start?.result !== 'success') {
+ return null;
+ }
+
+ const failed = Object.entries(needs)
+ .filter(([, need]) => need.result !== 'success' && need.result !== 'skipped')
+ .map(([job]) => job);
+
+ if (failed.length > 0) {
+ return { state: 'failure', description: `Failed: ${failed.join(', ')}` };
+ }
+
+ return { state: 'success', description: `Passed on ${head.slice(0, 7)}` };
+ }
+
+ /// A fork run's verdict from its jobs. Null when the run was not a request
+ /// (unrelated label) or did not finish (cancelled).
+ export function forkOutcome(jobs: RunJob[], head: string, labelName: string): Outcome {
+ const start = jobs.find((job) => job.name === START_JOB);
+ const gate = jobs.find((job) => job.name === GATE_JOB);
+
+ if (!start || start.conclusion === 'skipped' || start.conclusion === 'cancelled') {
+ return null;
+ }
+
+ if (start.conclusion !== 'success') {
+ return {
+ state: 'failure',
+ description: `Could not start on ${head.slice(0, 7)}. Add the ${labelName} label again.`,
+ };
+ }
+
+ if (gate?.conclusion === 'success') {
+ return { state: 'success', description: `Passed on ${head.slice(0, 7)}` };
+ }
+
+ if (gate?.conclusion === 'failure') {
+ return { state: 'failure', description: 'Failed: see the PR CI run' };
+ }
+
+ return null;
+ }
+
+ async function setStatus(sha: string, state: State, description: string, targetUrl: string) {
+ await github('POST', `/repos/${repo()}/statuses/${sha}`, {
+ state,
+ context: context(),
+ description: description.slice(0, DESCRIPTION_LIMIT),
+ target_url: targetUrl,
+ });
+ console.log(`${context()} on ${sha.slice(0, 10)}: ${state}: ${description}`);
+ }
+
+ /// The newest gate status on a commit (GitHub lists statuses newest first).
+ async function gateStatus(sha: string): Promise<{ state: string; description: string } | undefined> {
+ const statuses: { context: string; state: string; description: string | null }[] = await github(
+ 'GET',
+ `/repos/${repo()}/commits/${sha}/statuses?per_page=100`,
+ );
+ const status = statuses.find((each) => each.context === context());
+ return status && { state: status.state, description: status.description ?? '' };
+ }
+
+ /// Whether a trusted step opened a request on this commit that no result has
+ /// closed yet.
+ export function isOpenRequest(status: { state: string; description: string } | undefined): boolean {
+ return status?.state === 'pending' && status.description.startsWith(RUNNING);
+ }
+
+ async function removeLabel(pr: string) {
+ await github('DELETE', `/repos/${repo()}/issues/${pr}/labels/${encodeURIComponent(label())}`);
+ }
+
+ async function remind() {
+ const head = env('HEAD_SHA');
+ const pr = env('PR_NUMBER');
+
+ // A re-open or leaving draft keeps the head, so it may have a result already.
+ if (await gateStatus(head)) {
+ console.log(`${head.slice(0, 10)} already has a ${context()} status`);
+ return;
+ }
+
+ await setStatus(
+ head,
+ 'pending',
+ `Not run on this commit. Run: ${runCommand(repo(), pr, label())}`,
+ env('CI_GATE_DOCS_URL'),
+ );
+ }
+
+ async function start() {
+ const head = env('HEAD_SHA');
+ const pr = env('PR_NUMBER');
+ const runUrl = env('RUN_URL');
+ const sameRepo = env('SAME_REPO') === 'true';
+
+ // Fork tokens are read-only; the fork request step did both already.
+ if (sameRepo) {
+ await removeLabel(pr);
+ }
+
+ // A label added right after a push can carry the previous head, or a merge
+ // commit GitHub has not rebuilt yet. Either would test an old commit.
+ const pull = await github('GET', `/repos/${repo()}/pulls/${pr}`);
+ if (pull.head.sha !== head) {
+ const description = `The PR moved on from ${head.slice(0, 7)} to ${pull.head.sha.slice(0, 7)}. Run again: ${runCommand(repo(), pr, label())}`;
+ if (sameRepo) {
+ await setStatus(head, 'failure', description, runUrl);
+ }
+ throw new Error(description);
+ }
+
+ const merge = await github('GET', `/repos/${repo()}/commits/${env('MERGE_SHA')}`);
+ const parents: string[] = merge.parents.map((parent: { sha: string }) => parent.sha);
+
+ if (!parents.includes(head)) {
+ const description = `The merge commit did not include ${head.slice(0, 7)} yet. Run again: ${runCommand(repo(), pr, label())}`;
+ if (sameRepo) {
+ await setStatus(head, 'failure', description, runUrl);
+ }
+ throw new Error(description);
+ }
+
+ if (sameRepo) {
+ await setStatus(head, 'pending', `${RUNNING} ${head.slice(0, 7)}`, runUrl);
+ }
+ }
+
+ async function finish() {
+ const head = env('HEAD_SHA');
+ const outcome = gateOutcome(JSON.parse(env('NEEDS')), head);
+
+ if (!outcome) {
+ console.log('Start did not succeed; its status stands.');
+ process.exitCode = 1;
+ return;
+ }
+
+ if (env('SAME_REPO') === 'true') {
+ await setStatus(head, outcome.state, outcome.description, env('RUN_URL'));
+ }
+
+ if (outcome.state !== 'success') {
+ console.log(outcome.description);
+ process.exitCode = 1;
+ }
+ }
+
+ async function forkRequest() {
+ const head = env('HEAD_SHA');
+ const pr = env('PR_NUMBER');
+
+ await removeLabel(pr);
+ await setStatus(head, 'pending', `${RUNNING} ${head.slice(0, 7)}`, env('PR_CHECKS_URL'));
+ }
+
+ async function fork() {
+ const event = JSON.parse(readFileSync(env('GITHUB_EVENT_PATH'), 'utf8'));
+ const run = event.workflow_run;
+ const head: string = run.head_sha;
+
+ // The request step opened this on the head the label was added at. A run on
+ // any other commit, such as one the fork pushed with a rewritten workflow,
+ // or a run after the result is in, finds no open request.
+ if (!isOpenRequest(await gateStatus(head))) {
+ console.log(`${head.slice(0, 10)} has no open ${context()} request; nothing to report.`);
+ return;
+ }
+
+ const jobs: RunJob[] = [];
+ for (let page = 1; ; page++) {
+ const { jobs: batch } = await github(
+ 'GET',
+ `/repos/${repo()}/actions/runs/${run.id}/attempts/${run.run_attempt}/jobs?per_page=100&page=${page}`,
+ );
+ jobs.push(...batch);
+ if (batch.length < 100) {
+ break;
+ }
+ }
+
+ const outcome = forkOutcome(jobs, head, label());
+ if (outcome) {
+ await setStatus(head, outcome.state, outcome.description, run.html_url);
+ }
+ }
+
+ export async function main() {
+ const modes: Record<string, () => Promise<void>> = {
+ remind,
+ start,
+ finish,
+ 'fork-request': forkRequest,
+ fork,
+ };
+ const mode = modes[env('CI_GATE_MODE')];
+ if (!mode) {
+ throw new Error(`unknown CI_GATE_MODE ${process.env.CI_GATE_MODE}`);
+ }
+ await mode();
+ }
+
+ await main();
+ NODE_SCRIPT
+ shell: bash
+ env:
+ CI_GATE_MODE: fork
+ CI_GATE_CONTEXT: CI gate
+ CI_GATE_LABEL: run-ci
+ GH_TOKEN: ${{ github.token }}
diff --git a/.github/workflows/ci-gate-reminder.yml b/.github/workflows/ci-gate-reminder.yml
new file mode 100644
index 0000000000..913569bb22
--- /dev/null
+++ b/.github/workflows/ci-gate-reminder.yml
@@ -0,0 +1,307 @@
+# DO NOT EDIT — regenerate with `cargo x workflows` (from the repository root).
+# Source: tooling/xtask/crates/xtask_workflows/src/workflows/ci_gate_reminder.rs
+name: CI Gate Reminder
+'on':
+ pull_request:
+ types:
+ - opened
+ - synchronize
+ - reopened
+ branches:
+ - main
+jobs:
+ remind:
+ if: github.event.pull_request.head.repo.full_name == github.repository
+ name: CI not run yet
+ runs-on: ubuntu-latest
+ permissions:
+ statuses: write
+ timeout-minutes: 5
+ steps:
+ - name: Mark the commit as not run
+ run: |
+ node --disable-warning=ExperimentalWarning --input-type=module-typescript <<'NODE_SCRIPT'
+ // Sets the commit status `main` requires before a merge: it is green only on a
+ // commit that a requested `PR CI` run passed. `CI_GATE_MODE` picks the step:
+ //
+ // remind a new commit: pending "not run" with the command to run it, unless
+ // the commit already has a gate status
+ // start a requested run begins: remove the label, check the merge commit
+ // holds the PR head, set pending
+ // finish the run ends: success when every job passed or was path-skipped
+ // fork-request the label goes on a fork PR: what start does with a write
+ // token, from a trusted run of `main`'s workflow
+ // fork after a fork PR's run, whose read-only token could not set the status
+ //
+ // Statuses go on the PR head commit, so a later push starts without one.
+
+ import { readFileSync } from 'node:fs';
+
+ const API = 'https://api.github.com';
+ const DESCRIPTION_LIMIT = 140;
+ // An open request. Only trusted runs set statuses: a fork's token can't.
+ const RUNNING = 'Running on';
+ // Job names in `PR CI`; the fork step reads them back from the finished run.
+ export const START_JOB = 'Start';
+ export const GATE_JOB = 'CI gate';
+
+ type State = 'pending' | 'success' | 'failure' | 'error';
+ export type Needs = Record<string, { result: string }>;
+ export type RunJob = { name: string; conclusion: string | null };
+ type Outcome = { state: State; description: string } | null;
+
+ function env(name: string): string {
+ const value = process.env[name];
+ if (!value) {
+ throw new Error(`${name} is not set`);
+ }
+ return value;
+ }
+
+ async function github(method: string, path: string, body?: unknown): Promise<any> {
+ const response = await fetch(`${API}${path}`, {
+ method,
+ headers: {
+ accept: 'application/vnd.github+json',
+ authorization: `Bearer ${env('GH_TOKEN')}`,
+ 'x-github-api-version': '2022-11-28',
+ },
+ body: body === undefined ? undefined : JSON.stringify(body),
+ });
+ // Removing a label that is already gone is the outcome we wanted.
+ if (method === 'DELETE' && response.status === 404) {
+ return null;
+ }
+ if (!response.ok) {
+ throw new Error(`${method} ${path}: HTTP ${response.status} ${await response.text()}`);
+ }
+ return response.status === 204 ? null : response.json();
+ }
+
+ const repo = () => env('GITHUB_REPOSITORY');
+ const context = () => env('CI_GATE_CONTEXT');
+ const label = () => env('CI_GATE_LABEL');
+
+ /// REST, because `gh pr edit` fails on older gh releases.
+ export function runCommand(repository: string, pr: string, labelName: string): string {
+ return `gh api repos/${repository}/issues/${pr}/labels -f "labels[]=${labelName}"`;
+ }
+
+ /// The finished run's verdict from `toJSON(needs)`. Start must have succeeded;
+ /// a called workflow may be skipped, because its path filter did not match.
+ /// Null when start failed: it already set a status that says why.
+ export function gateOutcome(needs: Needs, head: string): Outcome {
+ if (needs.start?.result !== 'success') {
+ return null;
+ }
+
+ const failed = Object.entries(needs)
+ .filter(([, need]) => need.result !== 'success' && need.result !== 'skipped')
+ .map(([job]) => job);
+
+ if (failed.length > 0) {
+ return { state: 'failure', description: `Failed: ${failed.join(', ')}` };
+ }
+
+ return { state: 'success', description: `Passed on ${head.slice(0, 7)}` };
+ }
+
+ /// A fork run's verdict from its jobs. Null when the run was not a request
+ /// (unrelated label) or did not finish (cancelled).
+ export function forkOutcome(jobs: RunJob[], head: string, labelName: string): Outcome {
+ const start = jobs.find((job) => job.name === START_JOB);
+ const gate = jobs.find((job) => job.name === GATE_JOB);
+
+ if (!start || start.conclusion === 'skipped' || start.conclusion === 'cancelled') {
+ return null;
+ }
+
+ if (start.conclusion !== 'success') {
+ return {
+ state: 'failure',
+ description: `Could not start on ${head.slice(0, 7)}. Add the ${labelName} label again.`,
+ };
+ }
+
+ if (gate?.conclusion === 'success') {
+ return { state: 'success', description: `Passed on ${head.slice(0, 7)}` };
+ }
+
+ if (gate?.conclusion === 'failure') {
+ return { state: 'failure', description: 'Failed: see the PR CI run' };
+ }
+
+ return null;
+ }
+
+ async function setStatus(sha: string, state: State, description: string, targetUrl: string) {
+ await github('POST', `/repos/${repo()}/statuses/${sha}`, {
+ state,
+ context: context(),
+ description: description.slice(0, DESCRIPTION_LIMIT),
+ target_url: targetUrl,
+ });
+ console.log(`${context()} on ${sha.slice(0, 10)}: ${state}: ${description}`);
+ }
+
+ /// The newest gate status on a commit (GitHub lists statuses newest first).
+ async function gateStatus(sha: string): Promise<{ state: string; description: string } | undefined> {
+ const statuses: { context: string; state: string; description: string | null }[] = await github(
+ 'GET',
+ `/repos/${repo()}/commits/${sha}/statuses?per_page=100`,
+ );
+ const status = statuses.find((each) => each.context === context());
+ return status && { state: status.state, description: status.description ?? '' };
+ }
+
+ /// Whether a trusted step opened a request on this commit that no result has
+ /// closed yet.
+ export function isOpenRequest(status: { state: string; description: string } | undefined): boolean {
+ return status?.state === 'pending' && status.description.startsWith(RUNNING);
+ }
+
+ async function removeLabel(pr: string) {
+ await github('DELETE', `/repos/${repo()}/issues/${pr}/labels/${encodeURIComponent(label())}`);
+ }
+
+ async function remind() {
+ const head = env('HEAD_SHA');
+ const pr = env('PR_NUMBER');
+
+ // A re-open or leaving draft keeps the head, so it may have a result already.
+ if (await gateStatus(head)) {
+ console.log(`${head.slice(0, 10)} already has a ${context()} status`);
+ return;
+ }
+
+ await setStatus(
+ head,
+ 'pending',
+ `Not run on this commit. Run: ${runCommand(repo(), pr, label())}`,
+ env('CI_GATE_DOCS_URL'),
+ );
+ }
+
+ async function start() {
+ const head = env('HEAD_SHA');
+ const pr = env('PR_NUMBER');
+ const runUrl = env('RUN_URL');
+ const sameRepo = env('SAME_REPO') === 'true';
+
+ // Fork tokens are read-only; the fork request step did both already.
+ if (sameRepo) {
+ await removeLabel(pr);
+ }
+
+ // A label added right after a push can carry the previous head, or a merge
+ // commit GitHub has not rebuilt yet. Either would test an old commit.
+ const pull = await github('GET', `/repos/${repo()}/pulls/${pr}`);
+ if (pull.head.sha !== head) {
+ const description = `The PR moved on from ${head.slice(0, 7)} to ${pull.head.sha.slice(0, 7)}. Run again: ${runCommand(repo(), pr, label())}`;
+ if (sameRepo) {
+ await setStatus(head, 'failure', description, runUrl);
+ }
+ throw new Error(description);
+ }
+
+ const merge = await github('GET', `/repos/${repo()}/commits/${env('MERGE_SHA')}`);
+ const parents: string[] = merge.parents.map((parent: { sha: string }) => parent.sha);
+
+ if (!parents.includes(head)) {
+ const description = `The merge commit did not include ${head.slice(0, 7)} yet. Run again: ${runCommand(repo(), pr, label())}`;
+ if (sameRepo) {
+ await setStatus(head, 'failure', description, runUrl);
+ }
+ throw new Error(description);
+ }
+
+ if (sameRepo) {
+ await setStatus(head, 'pending', `${RUNNING} ${head.slice(0, 7)}`, runUrl);
+ }
+ }
+
+ async function finish() {
+ const head = env('HEAD_SHA');
+ const outcome = gateOutcome(JSON.parse(env('NEEDS')), head);
+
+ if (!outcome) {
+ console.log('Start did not succeed; its status stands.');
+ process.exitCode = 1;
+ return;
+ }
+
+ if (env('SAME_REPO') === 'true') {
+ await setStatus(head, outcome.state, outcome.description, env('RUN_URL'));
+ }
+
+ if (outcome.state !== 'success') {
+ console.log(outcome.description);
+ process.exitCode = 1;
+ }
+ }
+
+ async function forkRequest() {
+ const head = env('HEAD_SHA');
+ const pr = env('PR_NUMBER');
+
+ await removeLabel(pr);
+ await setStatus(head, 'pending', `${RUNNING} ${head.slice(0, 7)}`, env('PR_CHECKS_URL'));
+ }
+
+ async function fork() {
+ const event = JSON.parse(readFileSync(env('GITHUB_EVENT_PATH'), 'utf8'));
+ const run = event.workflow_run;
+ const head: string = run.head_sha;
+
+ // The request step opened this on the head the label was added at. A run on
+ // any other commit, such as one the fork pushed with a rewritten workflow,
+ // or a run after the result is in, finds no open request.
+ if (!isOpenRequest(await gateStatus(head))) {
+ console.log(`${head.slice(0, 10)} has no open ${context()} request; nothing to report.`);
+ return;
+ }
+
+ const jobs: RunJob[] = [];
+ for (let page = 1; ; page++) {
+ const { jobs: batch } = await github(
+ 'GET',
+ `/repos/${repo()}/actions/runs/${run.id}/attempts/${run.run_attempt}/jobs?per_page=100&page=${page}`,
+ );
+ jobs.push(...batch);
+ if (batch.length < 100) {
+ break;
+ }
+ }
+
+ const outcome = forkOutcome(jobs, head, label());
+ if (outcome) {
+ await setStatus(head, outcome.state, outcome.description, run.html_url);
+ }
+ }
+
+ export async function main() {
+ const modes: Record<string, () => Promise<void>> = {
+ remind,
+ start,
+ finish,
+ 'fork-request': forkRequest,
+ fork,
+ };
+ const mode = modes[env('CI_GATE_MODE')];
+ if (!mode) {
+ throw new Error(`unknown CI_GATE_MODE ${process.env.CI_GATE_MODE}`);
+ }
+ await mode();
+ }
+
+ await main();
+ NODE_SCRIPT
+ shell: bash
+ env:
+ CI_GATE_MODE: remind
+ CI_GATE_CONTEXT: CI gate
+ CI_GATE_LABEL: run-ci
+ GH_TOKEN: ${{ github.token }}
+ PR_NUMBER: ${{ github.event.pull_request.number }}
+ HEAD_SHA: ${{ github.event.pull_request.head.sha }}
+ CI_GATE_DOCS_URL: https://github.com/macro-inc/macro/blob/main/.agents/skills/run-ci/SKILL.md
diff --git a/.github/workflows/code_check_cloud_storage.yml b/.github/workflows/code_check_cloud_storage.yml
index 314cdaa02d..e79fab801a 100644
--- a/.github/workflows/code_check_cloud_storage.yml
+++ b/.github/workflows/code_check_cloud_storage.yml
@@ -17,6 +17,7 @@ env:
- ready_for_review
branches:
- main
+ workflow_call: {}
jobs:
path-check:
runs-on: namespace-profile-linux-small;overrides.cache-tag=sccache-ci
@@ -322,7 +323,7 @@ jobs:
check:
needs:
- path-check
- if: needs.path-check.outputs.should_run == 'true' && github.event.pull_request.draft == false
+ if: needs.path-check.outputs.should_run == 'true' && (github.event.pull_request.draft == false || github.event.action == 'labeled')
runs-on: namespace-profile-linux-large-build;overrides.cache-tag=sccache-ci
env:
RUSTFLAGS: '-Dwarnings -Dclippy::disallowed_methods -C link-arg=-fuse-ld=mold'
@@ -522,7 +523,7 @@ jobs:
doppler-config:
needs:
- path-check
- if: needs.path-check.outputs.should_run == 'true' && github.event.pull_request.draft == false && needs.path-check.outputs.doppler_candidates == 'true'
+ if: needs.path-check.outputs.should_run == 'true' && (github.event.pull_request.draft == false || github.event.action == 'labeled') && needs.path-check.outputs.doppler_candidates == 'true'
runs-on: namespace-profile-linux-rust-ci;overrides.cache-tag=sccache-ci
env:
RUSTFLAGS: '-Dwarnings -Dclippy::disallowed_methods -C link-arg=-fuse-ld=mold'
@@ -753,7 +754,7 @@ jobs:
test:
needs:
- path-check
- if: needs.path-check.outputs.should_run == 'true' && github.event.pull_request.draft == false && needs.path-check.outputs.rust_packages != 'none' && needs.path-check.outputs.skip_tests != 'true'
+ if: needs.path-check.outputs.should_run == 'true' && (github.event.pull_request.draft == false || github.event.action == 'labeled') && needs.path-check.outputs.rust_packages != 'none' && needs.path-check.outputs.skip_tests != 'true'
runs-on: namespace-profile-linux-large-build;overrides.cache-tag=sccache-ci
env:
RUST_PACKAGES: ${{ needs.path-check.outputs.rust_packages }}
@@ -1052,7 +1053,7 @@ jobs:
build-service-bins:
needs:
- path-check
- if: needs.path-check.outputs.should_run == 'true' && github.event.pull_request.draft == false && needs.path-check.outputs.skip_tests != 'true' && (needs.path-check.outputs.rust_packages == 'all' || contains(needs.path-check.outputs.rust_packages, 'document_storage_service') || contains(needs.path-check.outputs.rust_packages, 'email_service') || contains(needs.path-check.outputs.rust_packages, 'authentication_service'))
+ if: needs.path-check.outputs.should_run == 'true' && (github.event.pull_request.draft == false || github.event.action == 'labeled') && needs.path-check.outputs.skip_tests != 'true' && (needs.path-check.outputs.rust_packages == 'all' || contains(needs.path-check.outputs.rust_packages, 'document_storage_service') || contains(needs.path-check.outputs.rust_packages, 'email_service') || contains(needs.path-check.outputs.rust_packages, 'authentication_service'))
runs-on: namespace-profile-linux-large-build;overrides.cache-tag=sccache-ci
env:
RUST_PACKAGES: ${{ needs.path-check.outputs.rust_packages }}
@@ -1226,7 +1227,7 @@ jobs:
graphql-schema-compat:
needs:
- path-check
- if: needs.path-check.outputs.should_run == 'true' && github.event.pull_request.draft == false && needs.path-check.outputs.graphql_schema == 'true'
+ if: needs.path-check.outputs.should_run == 'true' && (github.event.pull_request.draft == false || github.event.action == 'labeled') && needs.path-check.outputs.graphql_schema == 'true'
runs-on: namespace-profile-linux-small
timeout-minutes: 15
steps:
@@ -1282,5 +1283,5 @@ jobs:
echo "✅ All jobs passed or were skipped"
concurrency:
- group: code-check-cloud-storage-${{ github.ref }}
+ group: code-check-cloud-storage-${{ github.event.action == 'labeled' && github.run_id || github.ref }}
cancel-in-progress: true
diff --git a/.github/workflows/code_check_conventions.yml b/.github/workflows/code_check_conventions.yml
index e86007b67a..d8290548de 100644
--- a/.github/workflows/code_check_conventions.yml
+++ b/.github/workflows/code_check_conventions.yml
@@ -10,6 +10,7 @@ name: conventions code check
- ready_for_review
branches:
- main
+ workflow_call: {}
permissions:
contents: read
jobs:
@@ -41,7 +42,7 @@ jobs:
ast-grep:
needs:
- path-check
- if: needs.path-check.outputs.should_run == 'true' && github.event.pull_request.draft == false
+ if: needs.path-check.outputs.should_run == 'true' && (github.event.pull_request.draft == false || github.event.action == 'labeled')
name: ast-grep Conventions
runs-on: namespace-profile-linux-small;overrides.cache-tag=web-ci
steps:
@@ -89,5 +90,5 @@ jobs:
echo "✅ All jobs passed or were skipped"
concurrency:
- group: code-check-conventions-${{ github.ref }}
+ group: code-check-conventions-${{ github.event.action == 'labeled' && github.run_id || github.ref }}
cancel-in-progress: true
diff --git a/.github/workflows/code_check_infra.yml b/.github/workflows/code_check_infra.yml
index 653ffda64c..a9e64f4373 100644
--- a/.github/workflows/code_check_infra.yml
+++ b/.github/workflows/code_check_infra.yml
@@ -10,6 +10,7 @@ name: infra code check
- ready_for_review
branches:
- main
+ workflow_call: {}
jobs:
path-check:
runs-on: namespace-profile-linux-small
@@ -33,7 +34,7 @@ jobs:
biome-check:
needs:
- path-check
- if: needs.path-check.outputs.should_run == 'true' && github.event.pull_request.draft == false
+ if: needs.path-check.outputs.should_run == 'true' && (github.event.pull_request.draft == false || github.event.action == 'labeled')
name: Biome Check
runs-on: namespace-profile-linux-small;overrides.cache-tag=web-ci
steps:
@@ -59,7 +60,7 @@ jobs:
check:
needs:
- path-check
- if: needs.path-check.outputs.should_run == 'true' && github.event.pull_request.draft == false
+ if: needs.path-check.outputs.should_run == 'true' && (github.event.pull_request.draft == false || github.event.action == 'labeled')
runs-on: namespace-profile-linux-small;overrides.cache-tag=web-ci
steps:
- name: Checkout Repo
@@ -108,5 +109,5 @@ jobs:
echo "✅ All jobs passed or were skipped"
concurrency:
- group: code-check-infra-${{ github.ref }}
+ group: code-check-infra-${{ github.event.action == 'labeled' && github.run_id || github.ref }}
cancel-in-progress: true
diff --git a/.github/workflows/export-ci-telemetry.yml b/.github/workflows/export-ci-telemetry.yml
index 2bfc508a06..7de9291028 100644
--- a/.github/workflows/export-ci-telemetry.yml
+++ b/.github/workflows/export-ci-telemetry.yml
@@ -16,6 +16,8 @@ name: Export CI telemetry
- Build Rust CI Image
- Build Web App
- Build macOS DMG
+ - CI Gate Reminder
+ - CI Gate for Forks
- CLA
- Cancel Stuck Cloud Storage Deploys
- Cleanup Preview
@@ -35,6 +37,7 @@ name: Export CI telemetry
- Export Datadog observability definitions
- Migrate Macro Database
- Native Check Label
+ - PR CI
- PR Privacy Check
- Publish Android to Google Play
- Publish SDK
@@ -115,7 +118,7 @@ jobs:
- name: Collect push, job and step events
if: always()
run: |
- node --disable-warning=ExperimentalWarning --input-type=module-typescript <<'TELEMETRY_SCRIPT'
+ node --disable-warning=ExperimentalWarning --input-type=module-typescript <<'NODE_SCRIPT'
// Turn one completed workflow run into CI telemetry events, as an OTLP logs file.
//
// Runs in the privileged exporter without the ingestion token. Everything here is
@@ -709,7 +712,7 @@ jobs:
}
await main();
- TELEMETRY_SCRIPT
+ NODE_SCRIPT
shell: bash
env:
GITHUB_TOKEN: ${{ github.token }}
@@ -742,7 +745,7 @@ jobs:
- name: Send push, job and step events
if: always()
run: |
- node --disable-warning=ExperimentalWarning --input-type=module-typescript <<'TELEMETRY_SCRIPT'
+ node --disable-warning=ExperimentalWarning --input-type=module-typescript <<'NODE_SCRIPT'
// Send the collected CI telemetry to the OTLP logs route. Best effort, never fails the job.
import { existsSync, readFileSync } from 'node:fs';
@@ -817,7 +820,7 @@ jobs:
}
await main();
- TELEMETRY_SCRIPT
+ NODE_SCRIPT
shell: bash
env:
CI_TELEMETRY_EVENTS: ${{ runner.temp }}/ci-telemetry-events.json
diff --git a/.github/workflows/ios-app-store-release.yml b/.github/workflows/ios-app-store-release.yml
index d89dc60105..c435e0c0f9 100644
--- a/.github/workflows/ios-app-store-release.yml
+++ b/.github/workflows/ios-app-store-release.yml
@@ -253,8 +253,10 @@ jobs:
# Web App Status Check aggregates the gate, so it is red whenever the gate is;
# its other inputs (Typecheck, Test, Build, ...) are separate checks and stay.
- # (`gh pr checks` names are job names, not workflow names.)
- ours='^(iOS Native Check|Re-run the native check|Web App Status Check|iOS App Store Release|iOS Release Drift|Android Native Check)'
+ # The CI gate status aggregates all of them, and stays pending on a head no one
+ # has run PR CI on. (`gh pr checks` names are job names, not workflow names;
+ # PR CI prefixes the ones it calls with `Web App / `.)
+ ours='^((Web App / )?(iOS Native Check|Web App Status Check|Android Native Check)|Re-run the native check|iOS App Store Release|iOS Release Drift|CI gate$)'
deadline=$(( $(date +%s) + WAIT_MINUTES * 60 ))
while true; do
checks=$(gh pr checks "$PR_NUMBER" --repo "$GH_REPO" --json name,bucket \
diff --git a/.github/workflows/pr-ci.yml b/.github/workflows/pr-ci.yml
new file mode 100644
index 0000000000..3005e39842
--- /dev/null
+++ b/.github/workflows/pr-ci.yml
@@ -0,0 +1,763 @@
+# DO NOT EDIT — regenerate with `cargo x workflows` (from the repository root).
+# Source: tooling/xtask/crates/xtask_workflows/src/workflows/pr_ci.rs
+name: PR CI
+on:
+ pull_request:
+ types:
+ - labeled
+ branches:
+ - main
+jobs:
+ start:
+ if: github.event.label.name == 'run-ci'
+ name: Start
+ runs-on: ubuntu-latest
+ permissions:
+ contents: read
+ issues: write
+ pull-requests: write
+ statuses: write
+ outputs:
+ cargo_deny: ${{ steps.filter.outputs.cargo_deny }}
+ workspace_dependencies: ${{ steps.filter.outputs.workspace_dependencies }}
+ generated_workflows: ${{ steps.filter.outputs.generated_workflows }}
+ node_modules_nix: ${{ steps.filter.outputs.node_modules_nix }}
+ tauri_lockfiles: ${{ steps.filter.outputs.tauri_lockfiles }}
+ sdk: ${{ steps.filter.outputs.sdk }}
+ macrod_update: ${{ steps.filter.outputs.macrod_update }}
+ timeout-minutes: 5
+ steps:
+ - name: Take the request
+ run: |
+ node --disable-warning=ExperimentalWarning --input-type=module-typescript <<'NODE_SCRIPT'
+ // Sets the commit status `main` requires before a merge: it is green only on a
+ // commit that a requested `PR CI` run passed. `CI_GATE_MODE` picks the step:
+ //
+ // remind a new commit: pending "not run" with the command to run it, unless
+ // the commit already has a gate status
+ // start a requested run begins: remove the label, check the merge commit
+ // holds the PR head, set pending
+ // finish the run ends: success when every job passed or was path-skipped
+ // fork-request the label goes on a fork PR: what start does with a write
+ // token, from a trusted run of `main`'s workflow
+ // fork after a fork PR's run, whose read-only token could not set the status
+ //
+ // Statuses go on the PR head commit, so a later push starts without one.
+
+ import { readFileSync } from 'node:fs';
+
+ const API = 'https://api.github.com';
+ const DESCRIPTION_LIMIT = 140;
+ // An open request. Only trusted runs set statuses: a fork's token can't.
+ const RUNNING = 'Running on';
+ // Job names in `PR CI`; the fork step reads them back from the finished run.
+ export const START_JOB = 'Start';
+ export const GATE_JOB = 'CI gate';
+
+ type State = 'pending' | 'success' | 'failure' | 'error';
+ export type Needs = Record<string, { result: string }>;
+ export type RunJob = { name: string; conclusion: string | null };
+ type Outcome = { state: State; description: string } | null;
+
+ function env(name: string): string {
+ const value = process.env[name];
+ if (!value) {
+ throw new Error(`${name} is not set`);
+ }
+ return value;
+ }
+
+ async function github(method: string, path: string, body?: unknown): Promise<any> {
+ const response = await fetch(`${API}${path}`, {
+ method,
+ headers: {
+ accept: 'application/vnd.github+json',
+ authorization: `Bearer ${env('GH_TOKEN')}`,
+ 'x-github-api-version': '2022-11-28',
+ },
+ body: body === undefined ? undefined : JSON.stringify(body),
+ });
+ // Removing a label that is already gone is the outcome we wanted.
+ if (method === 'DELETE' && response.status === 404) {
+ return null;
+ }
+ if (!response.ok) {
+ throw new Error(`${method} ${path}: HTTP ${response.status} ${await response.text()}`);
+ }
+ return response.status === 204 ? null : response.json();
+ }
+
+ const repo = () => env('GITHUB_REPOSITORY');
+ const context = () => env('CI_GATE_CONTEXT');
+ const label = () => env('CI_GATE_LABEL');
+
+ /// REST, because `gh pr edit` fails on older gh releases.
+ export function runCommand(repository: string, pr: string, labelName: string): string {
+ return `gh api repos/${repository}/issues/${pr}/labels -f "labels[]=${labelName}"`;
+ }
+
+ /// The finished run's verdict from `toJSON(needs)`. Start must have succeeded;
+ /// a called workflow may be skipped, because its path filter did not match.
+ /// Null when start failed: it already set a status that says why.
+ export function gateOutcome(needs: Needs, head: string): Outcome {
+ if (needs.start?.result !== 'success') {
+ return null;
+ }
+
+ const failed = Object.entries(needs)
+ .filter(([, need]) => need.result !== 'success' && need.result !== 'skipped')
+ .map(([job]) => job);
+
+ if (failed.length > 0) {
+ return { state: 'failure', description: `Failed: ${failed.join(', ')}` };
+ }
+
+ return { state: 'success', description: `Passed on ${head.slice(0, 7)}` };
+ }
+
+ /// A fork run's verdict from its jobs. Null when the run was not a request
+ /// (unrelated label) or did not finish (cancelled).
+ export function forkOutcome(jobs: RunJob[], head: string, labelName: string): Outcome {
+ const start = jobs.find((job) => job.name === START_JOB);
+ const gate = jobs.find((job) => job.name === GATE_JOB);
+
+ if (!start || start.conclusion === 'skipped' || start.conclusion === 'cancelled') {
+ return null;
+ }
+
+ if (start.conclusion !== 'success') {
+ return {
+ state: 'failure',
+ description: `Could not start on ${head.slice(0, 7)}. Add the ${labelName} label again.`,
+ };
+ }
+
+ if (gate?.conclusion === 'success') {
+ return { state: 'success', description: `Passed on ${head.slice(0, 7)}` };
+ }
+
+ if (gate?.conclusion === 'failure') {
+ return { state: 'failure', description: 'Failed: see the PR CI run' };
+ }
+
+ return null;
+ }
+
+ async function setStatus(sha: string, state: State, description: string, targetUrl: string) {
+ await github('POST', `/repos/${repo()}/statuses/${sha}`, {
+ state,
+ context: context(),
+ description: description.slice(0, DESCRIPTION_LIMIT),
+ target_url: targetUrl,
+ });
+ console.log(`${context()} on ${sha.slice(0, 10)}: ${state}: ${description}`);
+ }
+
+ /// The newest gate status on a commit (GitHub lists statuses newest first).
+ async function gateStatus(sha: string): Promise<{ state: string; description: string } | undefined> {
+ const statuses: { context: string; state: string; description: string | null }[] = await github(
+ 'GET',
+ `/repos/${repo()}/commits/${sha}/statuses?per_page=100`,
+ );
+ const status = statuses.find((each) => each.context === context());
+ return status && { state: status.state, description: status.description ?? '' };
+ }
+
+ /// Whether a trusted step opened a request on this commit that no result has
+ /// closed yet.
+ export function isOpenRequest(status: { state: string; description: string } | undefined): boolean {
+ return status?.state === 'pending' && status.description.startsWith(RUNNING);
+ }
+
+ async function removeLabel(pr: string) {
+ await github('DELETE', `/repos/${repo()}/issues/${pr}/labels/${encodeURIComponent(label())}`);
+ }
+
+ async function remind() {
+ const head = env('HEAD_SHA');
+ const pr = env('PR_NUMBER');
+
+ // A re-open or leaving draft keeps the head, so it may have a result already.
+ if (await gateStatus(head)) {
+ console.log(`${head.slice(0, 10)} already has a ${context()} status`);
+ return;
+ }
+
+ await setStatus(
+ head,
+ 'pending',
+ `Not run on this commit. Run: ${runCommand(repo(), pr, label())}`,
+ env('CI_GATE_DOCS_URL'),
+ );
+ }
+
+ async function start() {
+ const head = env('HEAD_SHA');
+ const pr = env('PR_NUMBER');
+ const runUrl = env('RUN_URL');
+ const sameRepo = env('SAME_REPO') === 'true';
+
+ // Fork tokens are read-only; the fork request step did both already.
+ if (sameRepo) {
+ await removeLabel(pr);
+ }
+
+ // A label added right after a push can carry the previous head, or a merge
+ // commit GitHub has not rebuilt yet. Either would test an old commit.
+ const pull = await github('GET', `/repos/${repo()}/pulls/${pr}`);
+ if (pull.head.sha !== head) {
+ const description = `The PR moved on from ${head.slice(0, 7)} to ${pull.head.sha.slice(0, 7)}. Run again: ${runCommand(repo(), pr, label())}`;
+ if (sameRepo) {
+ await setStatus(head, 'failure', description, runUrl);
+ }
+ throw new Error(description);
+ }
+
+ const merge = await github('GET', `/repos/${repo()}/commits/${env('MERGE_SHA')}`);
+ const parents: string[] = merge.parents.map((parent: { sha: string }) => parent.sha);
+
+ if (!parents.includes(head)) {
+ const description = `The merge commit did not include ${head.slice(0, 7)} yet. Run again: ${runCommand(repo(), pr, label())}`;
+ if (sameRepo) {
+ await setStatus(head, 'failure', description, runUrl);
+ }
+ throw new Error(description);
+ }
+
+ if (sameRepo) {
+ await setStatus(head, 'pending', `${RUNNING} ${head.slice(0, 7)}`, runUrl);
+ }
+ }
+
+ async function finish() {
+ const head = env('HEAD_SHA');
+ const outcome = gateOutcome(JSON.parse(env('NEEDS')), head);
+
+ if (!outcome) {
+ console.log('Start did not succeed; its status stands.');
+ process.exitCode = 1;
+ return;
+ }
+
+ if (env('SAME_REPO') === 'true') {
+ await setStatus(head, outcome.state, outcome.description, env('RUN_URL'));
+ }
+
+ if (outcome.state !== 'success') {
+ console.log(outcome.description);
+ process.exitCode = 1;
+ }
+ }
+
+ async function forkRequest() {
+ const head = env('HEAD_SHA');
+ const pr = env('PR_NUMBER');
+
+ await removeLabel(pr);
+ await setStatus(head, 'pending', `${RUNNING} ${head.slice(0, 7)}`, env('PR_CHECKS_URL'));
+ }
+
+ async function fork() {
+ const event = JSON.parse(readFileSync(env('GITHUB_EVENT_PATH'), 'utf8'));
+ const run = event.workflow_run;
+ const head: string = run.head_sha;
+
+ // The request step opened this on the head the label was added at. A run on
+ // any other commit, such as one the fork pushed with a rewritten workflow,
+ // or a run after the result is in, finds no open request.
+ if (!isOpenRequest(await gateStatus(head))) {
+ console.log(`${head.slice(0, 10)} has no open ${context()} request; nothing to report.`);
+ return;
+ }
+
+ const jobs: RunJob[] = [];
+ for (let page = 1; ; page++) {
+ const { jobs: batch } = await github(
+ 'GET',
+ `/repos/${repo()}/actions/runs/${run.id}/attempts/${run.run_attempt}/jobs?per_page=100&page=${page}`,
+ );
+ jobs.push(...batch);
+ if (batch.length < 100) {
+ break;
+ }
+ }
+
+ const outcome = forkOutcome(jobs, head, label());
+ if (outcome) {
+ await setStatus(head, outcome.state, outcome.description, run.html_url);
+ }
+ }
+
+ export async function main() {
+ const modes: Record<string, () => Promise<void>> = {
+ remind,
+ start,
+ finish,
+ 'fork-request': forkRequest,
+ fork,
+ };
+ const mode = modes[env('CI_GATE_MODE')];
+ if (!mode) {
+ throw new Error(`unknown CI_GATE_MODE ${process.env.CI_GATE_MODE}`);
+ }
+ await mode();
+ }
+
+ await main();
+ NODE_SCRIPT
+ shell: bash
+ env:
+ CI_GATE_MODE: start
+ CI_GATE_CONTEXT: CI gate
+ CI_GATE_LABEL: run-ci
+ GH_TOKEN: ${{ github.token }}
+ PR_NUMBER: ${{ github.event.pull_request.number }}
+ HEAD_SHA: ${{ github.event.pull_request.head.sha }}
+ SAME_REPO: ${{ github.event.pull_request.head.repo.full_name == github.repository }}
+ RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
+ MERGE_SHA: ${{ github.sha }}
+ - id: filter
+ name: Filter changed paths
+ uses: dorny/paths-filter@d1c1ffe0248fe513906c8e24db8ea791d46f8590
+ with:
+ filters: |
+ cargo_deny:
+ - 'Cargo.toml'
+ - 'crates/**/Cargo.toml'
+ - 'services/**/Cargo.toml'
+ - 'tooling/xtask/**/Cargo.toml'
+ - 'Cargo.lock'
+ - 'deny.toml'
+ - '.github/workflows/cargo_deny.yml'
+ workspace_dependencies:
+ - 'Cargo.toml'
+ - 'crates/**/Cargo.toml'
+ - 'services/**/Cargo.toml'
+ - 'tooling/xtask/**/Cargo.toml'
+ - 'Cargo.lock'
+ - '.config/hakari.toml'
+ - 'crates/workspace-hack/**'
+ - 'tooling/xtask/src/**'
+ - 'tooling/xtask/crates/xtask_deps/**'
+ - 'tooling/xtask/crates/xtask_graph/**'
+ - '.github/workspace-dep-closures.json'
+ generated_workflows:
+ - 'tooling/xtask/**'
+ - 'crates/macro_event_topics/**'
+ - '.github/workflows/**'
+ - '.github/kafka-cluster-topics.json'
+ node_modules_nix:
+ - 'bun.lock'
+ - 'packages/sdk/bun.lock'
+ - 'package.json'
+ - 'apps/**/package.json'
+ - 'packages/**/package.json'
+ - 'services/**/package.json'
+ - 'apps/web/patches/**'
+ - 'nix-support/node_modules.nix'
+ - 'nix-support/node_modules-hashes.json'
+ - 'nix/tauri-desktop.nix'
+ - 'tooling/scripts/update-node-modules-hash.sh'
+ - 'flake.nix'
+ - 'flake.lock'
+ - '.github/workflows/check_node_modules_nix.yml'
+ tauri_lockfiles:
+ - 'apps/web/tauri/**'
+ - 'crates/**/Cargo.toml'
+ - 'Cargo.toml'
+ - 'nix/tauri-desktop.nix'
+ - 'flake.nix'
+ - 'flake.lock'
+ - '.github/workflows/check_tauri_lockfiles.yml'
+ sdk:
+ - 'packages/sdk/**'
+ - '.github/workflows/sdk-check.yml'
+ macrod_update:
+ - 'crates/coding_agent_worker/**'
+ - '.github/workflows/test_macrod_update.yml'
+ cloud-storage:
+ needs:
+ - start
+ name: Cloud Storage
+ uses: ./.github/workflows/code_check_cloud_storage.yml
+ secrets: inherit
+ web-app:
+ needs:
+ - start
+ name: Web App
+ uses: ./.github/workflows/web-app-check-main.yml
+ secrets: inherit
+ infra:
+ needs:
+ - start
+ name: Infra
+ uses: ./.github/workflows/code_check_infra.yml
+ secrets: inherit
+ conventions:
+ needs:
+ - start
+ name: Conventions
+ uses: ./.github/workflows/code_check_conventions.yml
+ secrets: inherit
+ cargo-deny:
+ needs:
+ - start
+ if: needs.start.outputs.cargo_deny == 'true'
+ name: cargo deny
+ uses: ./.github/workflows/cargo_deny.yml
+ secrets: inherit
+ workspace-dependencies:
+ needs:
+ - start
+ if: needs.start.outputs.workspace_dependencies == 'true'
+ name: Workspace dependencies
+ uses: ./.github/workflows/cargo_workspace_dependency_check.yml
+ secrets: inherit
+ generated-workflows:
+ needs:
+ - start
+ if: needs.start.outputs.generated_workflows == 'true'
+ name: Generated workflows
+ uses: ./.github/workflows/check_generated.yml
+ secrets: inherit
+ node-modules-nix:
+ needs:
+ - start
+ if: needs.start.outputs.node_modules_nix == 'true'
+ name: node_modules nix
+ uses: ./.github/workflows/check_node_modules_nix.yml
+ secrets: inherit
+ tauri-lockfiles:
+ needs:
+ - start
+ if: needs.start.outputs.tauri_lockfiles == 'true'
+ name: Tauri lockfiles
+ uses: ./.github/workflows/check_tauri_lockfiles.yml
+ secrets: inherit
+ sdk:
+ needs:
+ - start
+ if: needs.start.outputs.sdk == 'true'
+ name: SDK
+ uses: ./.github/workflows/sdk-check.yml
+ secrets: inherit
+ macrod-update:
+ needs:
+ - start
+ if: needs.start.outputs.macrod_update == 'true'
+ name: macrod update
+ uses: ./.github/workflows/test_macrod_update.yml
+ secrets: inherit
+ gate:
+ needs:
+ - start
+ - cloud-storage
+ - web-app
+ - infra
+ - conventions
+ - cargo-deny
+ - workspace-dependencies
+ - generated-workflows
+ - node-modules-nix
+ - tauri-lockfiles
+ - sdk
+ - macrod-update
+ if: '!cancelled() && needs.start.result != ''skipped'''
+ name: CI gate
+ runs-on: ubuntu-latest
+ permissions:
+ statuses: write
+ timeout-minutes: 5
+ steps:
+ - name: Set the CI gate status
+ run: |
+ node --disable-warning=ExperimentalWarning --input-type=module-typescript <<'NODE_SCRIPT'
+ // Sets the commit status `main` requires before a merge: it is green only on a
+ // commit that a requested `PR CI` run passed. `CI_GATE_MODE` picks the step:
+ //
+ // remind a new commit: pending "not run" with the command to run it, unless
+ // the commit already has a gate status
+ // start a requested run begins: remove the label, check the merge commit
+ // holds the PR head, set pending
+ // finish the run ends: success when every job passed or was path-skipped
+ // fork-request the label goes on a fork PR: what start does with a write
+ // token, from a trusted run of `main`'s workflow
+ // fork after a fork PR's run, whose read-only token could not set the status
+ //
+ // Statuses go on the PR head commit, so a later push starts without one.
+
+ import { readFileSync } from 'node:fs';
+
+ const API = 'https://api.github.com';
+ const DESCRIPTION_LIMIT = 140;
+ // An open request. Only trusted runs set statuses: a fork's token can't.
+ const RUNNING = 'Running on';
+ // Job names in `PR CI`; the fork step reads them back from the finished run.
+ export const START_JOB = 'Start';
+ export const GATE_JOB = 'CI gate';
+
+ type State = 'pending' | 'success' | 'failure' | 'error';
+ export type Needs = Record<string, { result: string }>;
+ export type RunJob = { name: string; conclusion: string | null };
+ type Outcome = { state: State; description: string } | null;
+
+ function env(name: string): string {
+ const value = process.env[name];
+ if (!value) {
+ throw new Error(`${name} is not set`);
+ }
+ return value;
+ }
+
+ async function github(method: string, path: string, body?: unknown): Promise<any> {
+ const response = await fetch(`${API}${path}`, {
+ method,
+ headers: {
+ accept: 'application/vnd.github+json',
+ authorization: `Bearer ${env('GH_TOKEN')}`,
+ 'x-github-api-version': '2022-11-28',
+ },
+ body: body === undefined ? undefined : JSON.stringify(body),
+ });
+ // Removing a label that is already gone is the outcome we wanted.
+ if (method === 'DELETE' && response.status === 404) {
+ return null;
+ }
+ if (!response.ok) {
+ throw new Error(`${method} ${path}: HTTP ${response.status} ${await response.text()}`);
+ }
+ return response.status === 204 ? null : response.json();
+ }
+
+ const repo = () => env('GITHUB_REPOSITORY');
+ const context = () => env('CI_GATE_CONTEXT');
+ const label = () => env('CI_GATE_LABEL');
+
+ /// REST, because `gh pr edit` fails on older gh releases.
+ export function runCommand(repository: string, pr: string, labelName: string): string {
+ return `gh api repos/${repository}/issues/${pr}/labels -f "labels[]=${labelName}"`;
+ }
+
+ /// The finished run's verdict from `toJSON(needs)`. Start must have succeeded;
+ /// a called workflow may be skipped, because its path filter did not match.
+ /// Null when start failed: it already set a status that says why.
+ export function gateOutcome(needs: Needs, head: string): Outcome {
+ if (needs.start?.result !== 'success') {
+ return null;
+ }
+
+ const failed = Object.entries(needs)
+ .filter(([, need]) => need.result !== 'success' && need.result !== 'skipped')
+ .map(([job]) => job);
+
+ if (failed.length > 0) {
+ return { state: 'failure', description: `Failed: ${failed.join(', ')}` };
+ }
+
+ return { state: 'success', description: `Passed on ${head.slice(0, 7)}` };
+ }
+
+ /// A fork run's verdict from its jobs. Null when the run was not a request
+ /// (unrelated label) or did not finish (cancelled).
+ export function forkOutcome(jobs: RunJob[], head: string, labelName: string): Outcome {
+ const start = jobs.find((job) => job.name === START_JOB);
+ const gate = jobs.find((job) => job.name === GATE_JOB);
+
+ if (!start || start.conclusion === 'skipped' || start.conclusion === 'cancelled') {
+ return null;
+ }
+
+ if (start.conclusion !== 'success') {
+ return {
+ state: 'failure',
+ description: `Could not start on ${head.slice(0, 7)}. Add the ${labelName} label again.`,
+ };
+ }
+
+ if (gate?.conclusion === 'success') {
+ return { state: 'success', description: `Passed on ${head.slice(0, 7)}` };
+ }
+
+ if (gate?.conclusion === 'failure') {
+ return { state: 'failure', description: 'Failed: see the PR CI run' };
+ }
+
+ return null;
+ }
+
+ async function setStatus(sha: string, state: State, description: string, targetUrl: string) {
+ await github('POST', `/repos/${repo()}/statuses/${sha}`, {
+ state,
+ context: context(),
+ description: description.slice(0, DESCRIPTION_LIMIT),
+ target_url: targetUrl,
+ });
+ console.log(`${context()} on ${sha.slice(0, 10)}: ${state}: ${description}`);
+ }
+
+ /// The newest gate status on a commit (GitHub lists statuses newest first).
+ async function gateStatus(sha: string): Promise<{ state: string; description: string } | undefined> {
+ const statuses: { context: string; state: string; description: string | null }[] = await github(
+ 'GET',
+ `/repos/${repo()}/commits/${sha}/statuses?per_page=100`,
+ );
+ const status = statuses.find((each) => each.context === context());
+ return status && { state: status.state, description: status.description ?? '' };
+ }
+
+ /// Whether a trusted step opened a request on this commit that no result has
+ /// closed yet.
+ export function isOpenRequest(status: { state: string; description: string } | undefined): boolean {
+ return status?.state === 'pending' && status.description.startsWith(RUNNING);
+ }
+
+ async function removeLabel(pr: string) {
+ await github('DELETE', `/repos/${repo()}/issues/${pr}/labels/${encodeURIComponent(label())}`);
+ }
+
+ async function remind() {
+ const head = env('HEAD_SHA');
+ const pr = env('PR_NUMBER');
+
+ // A re-open or leaving draft keeps the head, so it may have a result already.
+ if (await gateStatus(head)) {
+ console.log(`${head.slice(0, 10)} already has a ${context()} status`);
+ return;
+ }
+
+ await setStatus(
+ head,
+ 'pending',
+ `Not run on this commit. Run: ${runCommand(repo(), pr, label())}`,
+ env('CI_GATE_DOCS_URL'),
+ );
+ }
+
+ async function start() {
+ const head = env('HEAD_SHA');
+ const pr = env('PR_NUMBER');
+ const runUrl = env('RUN_URL');
+ const sameRepo = env('SAME_REPO') === 'true';
+
+ // Fork tokens are read-only; the fork request step did both already.
+ if (sameRepo) {
+ await removeLabel(pr);
+ }
+
+ // A label added right after a push can carry the previous head, or a merge
+ // commit GitHub has not rebuilt yet. Either would test an old commit.
+ const pull = await github('GET', `/repos/${repo()}/pulls/${pr}`);
+ if (pull.head.sha !== head) {
+ const description = `The PR moved on from ${head.slice(0, 7)} to ${pull.head.sha.slice(0, 7)}. Run again: ${runCommand(repo(), pr, label())}`;
+ if (sameRepo) {
+ await setStatus(head, 'failure', description, runUrl);
+ }
+ throw new Error(description);
+ }
+
+ const merge = await github('GET', `/repos/${repo()}/commits/${env('MERGE_SHA')}`);
+ const parents: string[] = merge.parents.map((parent: { sha: string }) => parent.sha);
+
+ if (!parents.includes(head)) {
+ const description = `The merge commit did not include ${head.slice(0, 7)} yet. Run again: ${runCommand(repo(), pr, label())}`;
+ if (sameRepo) {
+ await setStatus(head, 'failure', description, runUrl);
+ }
+ throw new Error(description);
+ }
+
+ if (sameRepo) {
+ await setStatus(head, 'pending', `${RUNNING} ${head.slice(0, 7)}`, runUrl);
+ }
+ }
+
+ async function finish() {
+ const head = env('HEAD_SHA');
+ const outcome = gateOutcome(JSON.parse(env('NEEDS')), head);
+
+ if (!outcome) {
+ console.log('Start did not succeed; its status stands.');
+ process.exitCode = 1;
+ return;
+ }
+
+ if (env('SAME_REPO') === 'true') {
+ await setStatus(head, outcome.state, outcome.description, env('RUN_URL'));
+ }
+
+ if (outcome.state !== 'success') {
+ console.log(outcome.description);
+ process.exitCode = 1;
+ }
+ }
+
+ async function forkRequest() {
+ const head = env('HEAD_SHA');
+ const pr = env('PR_NUMBER');
+
+ await removeLabel(pr);
+ await setStatus(head, 'pending', `${RUNNING} ${head.slice(0, 7)}`, env('PR_CHECKS_URL'));
+ }
+
+ async function fork() {
+ const event = JSON.parse(readFileSync(env('GITHUB_EVENT_PATH'), 'utf8'));
+ const run = event.workflow_run;
+ const head: string = run.head_sha;
+
+ // The request step opened this on the head the label was added at. A run on
+ // any other commit, such as one the fork pushed with a rewritten workflow,
+ // or a run after the result is in, finds no open request.
+ if (!isOpenRequest(await gateStatus(head))) {
+ console.log(`${head.slice(0, 10)} has no open ${context()} request; nothing to report.`);
+ return;
+ }
+
+ const jobs: RunJob[] = [];
+ for (let page = 1; ; page++) {
+ const { jobs: batch } = await github(
+ 'GET',
+ `/repos/${repo()}/actions/runs/${run.id}/attempts/${run.run_attempt}/jobs?per_page=100&page=${page}`,
+ );
+ jobs.push(...batch);
+ if (batch.length < 100) {
+ break;
+ }
+ }
+
+ const outcome = forkOutcome(jobs, head, label());
+ if (outcome) {
+ await setStatus(head, outcome.state, outcome.description, run.html_url);
+ }
+ }
+
+ export async function main() {
+ const modes: Record<string, () => Promise<void>> = {
+ remind,
+ start,
+ finish,
+ 'fork-request': forkRequest,
+ fork,
+ };
+ const mode = modes[env('CI_GATE_MODE')];
+ if (!mode) {
+ throw new Error(`unknown CI_GATE_MODE ${process.env.CI_GATE_MODE}`);
+ }
+ await mode();
+ }
+
+ await main();
+ NODE_SCRIPT
+ shell: bash
+ env:
+ CI_GATE_MODE: finish
+ CI_GATE_CONTEXT: CI gate
+ CI_GATE_LABEL: run-ci
+ GH_TOKEN: ${{ github.token }}
+ PR_NUMBER: ${{ github.event.pull_request.number }}
+ HEAD_SHA: ${{ github.event.pull_request.head.sha }}
+ SAME_REPO: ${{ github.event.pull_request.head.repo.full_name == github.repository }}
+ RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
+ NEEDS: ${{ toJSON(needs) }}
+concurrency:
+ group: pr-ci-${{ github.event.pull_request.number }}-${{ github.event.label.name == 'run-ci' && 'requested' || github.run_id }}
+ cancel-in-progress: true
diff --git a/.github/workflows/sdk-check.yml b/.github/workflows/sdk-check.yml
index cb857e8c57..d8a02dd678 100644
--- a/.github/workflows/sdk-check.yml
+++ b/.github/workflows/sdk-check.yml
@@ -14,6 +14,7 @@ name: SDK Check
paths:
- packages/sdk/**
- '.github/workflows/sdk-check.yml'
+ workflow_call: {}
jobs:
check-package:
name: SDK Package Check
@@ -41,5 +42,5 @@ jobs:
run: bun run coverage
working-directory: packages/sdk
concurrency:
- group: ${{ github.workflow }}-${{ github.ref }}-check
+ group: sdk-check-${{ github.event.action == 'labeled' && github.run_id || github.ref }}
cancel-in-progress: true
diff --git a/.github/workflows/test_macrod_update.yml b/.github/workflows/test_macrod_update.yml
index 1623c0a385..666e8e0b94 100644
--- a/.github/workflows/test_macrod_update.yml
+++ b/.github/workflows/test_macrod_update.yml
@@ -8,6 +8,7 @@ name: Test macrod update
paths:
- crates/coding_agent_worker/**
- '.github/workflows/test_macrod_update.yml'
+ workflow_call: {}
workflow_dispatch: {}
permissions:
contents: read
diff --git a/.github/workflows/web-app-check-main.yml b/.github/workflows/web-app-check-main.yml
index 4d4a499054..e4ed2bbf90 100644
--- a/.github/workflows/web-app-check-main.yml
+++ b/.github/workflows/web-app-check-main.yml
@@ -10,6 +10,7 @@ name: Web App Pr Checks
- ready_for_review
branches:
- main
+ workflow_call: {}
jobs:
path-check:
runs-on: namespace-profile-linux-tiny-no-cache
@@ -1418,5 +1419,5 @@ jobs:
echo "All jobs passed or were skipped"
concurrency:
- group: ${{ github.workflow }}-${{ github.ref }}-check
+ group: web-app-pr-checks-${{ github.event.action == 'labeled' && github.run_id || github.ref }}
cancel-in-progress: true
diff --git a/CLAUDE.md b/CLAUDE.md
index daf6506daa..c9c6dc689a 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -58,6 +58,7 @@ Real files live in `.agents/`; tool-specific directories only symlink to it.
| Web frontend | [Web agent guide](apps/web/AGENTS.md) |
| iOS App Store releases, breaking Tauri changes, native build floors | [iOS releases](apps/web/docs/ios-releases.md) |
| Android Play releases from pull requests | [Android development](docs/development/android.md) |
+| Running CI on a pull request | [CI on pull requests](docs/development/ci.md), `run-ci` skill |
| Email body rendering or snapshots | [Standalone renderer](packages/email-renderer/README.md) |
| Running the frontend or backend on a local machine | [Running locally](docs/development/running-locally.md) |
| Giving the user a local preview link | `local-preview` skill |
@@ -122,8 +123,11 @@ Real files live in `.agents/`; tool-specific directories only symlink to it.
reads it too.
- Open pull requests ready for review. If one is created as a draft, mark it
ready for review before handing off.
-- Get CI passing before handing off. A pull request whose checks are failing
- or still running is not done.
+- Run CI deliberately with the `run-ci` label (`run-ci` skill): at a
+ checkpoint, a coherent locally tested chunk you want real CI on before
+ building further, or when you think the PR is done. Never after every push,
+ and never instead of local checks. A PR is done only when `CI gate` passes on
+ its latest commit.
- Test the affected packages/services individually before committing. Use the
relevant guide for setup; there is no root `just test` recipe.
- For code changes, run `just check` (format/lint/code rules scoped to changes).
diff --git a/docs/development/ci.md b/docs/development/ci.md
new file mode 100644
index 0000000000..42b56406ca
--- /dev/null
+++ b/docs/development/ci.md
@@ -0,0 +1,19 @@
+# CI on pull requests
+
+CI runs on a pull request when someone adds the `run-ci` label. The `PR CI`
+workflow then runs every check on the PR's head commit, removes the label, and
+reports the result as the `CI gate` commit status on that commit. Each push
+leaves the new commit at `Not run on this commit`, so a PR can merge only after
+a passing run on its latest commit.
+
+Run CI deliberately: at a checkpoint, a coherent locally tested chunk you want
+real CI on before building further, or when you think the PR is done. Never
+after every push, and never instead of local checks.
+
+The [`run-ci` skill](../../.agents/skills/run-ci/SKILL.md) has the commands,
+what each `CI gate` state means, re-runs, and fork PRs. The workflows are
+generated from Rust; see the
+[workflows rule](../../.agents/rules/infra/github-workflows.md).
+
+Until `main`'s ruleset requires `CI gate`, the checks also still run on every
+push, and `main` still requires their old status checks.
diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows.rs b/tooling/xtask/crates/xtask_workflows/src/workflows.rs
index 0053fc0d5f..89e31b8f5e 100644
--- a/tooling/xtask/crates/xtask_workflows/src/workflows.rs
+++ b/tooling/xtask/crates/xtask_workflows/src/workflows.rs
@@ -27,6 +27,8 @@ mod cargo_workspace_dependency_check;
mod check_generated;
mod check_node_modules_nix;
mod check_tauri_lockfiles;
+mod ci_gate_fork;
+mod ci_gate_reminder;
mod cla;
mod cleanup_preview;
mod code_check_cloud_storage;
@@ -50,6 +52,7 @@ mod ios_release_drift;
mod ios_release_watch;
mod native_check_label;
mod path_validation;
+mod pr_ci;
mod pr_privacy_check;
mod publish_sdk;
mod pulumi_preview_pr;
@@ -295,6 +298,21 @@ const WORKFLOWS: &[WorkflowFile] = &[
file_name: "ios_preview.yml",
render_yaml: || render_gh_workflow(ios_preview::ios_preview)(),
},
+ WorkflowFile {
+ slug: "pr_ci",
+ file_name: "pr-ci.yml",
+ render_yaml: || render_patched(pr_ci::pr_ci, pr_ci::patch),
+ },
+ WorkflowFile {
+ slug: "ci_gate_reminder",
+ file_name: "ci-gate-reminder.yml",
+ render_yaml: || render_gh_workflow(ci_gate_reminder::ci_gate_reminder)(),
+ },
+ WorkflowFile {
+ slug: "ci_gate_fork",
+ file_name: "ci-gate-fork.yml",
+ render_yaml: || render_gh_workflow(ci_gate_fork::ci_gate_fork)(),
+ },
WorkflowFile {
slug: "pr_privacy_check",
file_name: "pr-privacy-check.yml",
diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/cargo_deny.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/cargo_deny.rs
index 2bfbc63fcb..eed5a2e13b 100644
--- a/tooling/xtask/crates/xtask_workflows/src/workflows/cargo_deny.rs
+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/cargo_deny.rs
@@ -2,34 +2,40 @@
//! workspace on PRs that touch manifests. Generated into `cargo_deny.yml`
//! (replaces the hand-written `cargo-deny.yml`).
-use gh_workflow::{
- Concurrency, Event, Expression, Job, PullRequest, PullRequestType, Step, Use, Workflow,
-};
+use gh_workflow::{Event, Job, PullRequest, PullRequestType, Step, Use, Workflow, WorkflowCall};
+use xtask_paths::RepoGlob;
-use crate::workflows::runners;
+use crate::workflows::{runners, vars};
+
+/// Files whose change runs the check, on a push and in `PR CI`.
+pub const PATHS: &[RepoGlob<'static>] = &[
+ xtask_paths::repo_glob!("Cargo.toml"),
+ xtask_paths::repo_glob!("crates/**/Cargo.toml"),
+ xtask_paths::repo_glob!("services/**/Cargo.toml"),
+ xtask_paths::repo_glob!("tooling/xtask/**/Cargo.toml"),
+ xtask_paths::repo_glob!("Cargo.lock"),
+ xtask_paths::repo_glob!("deny.toml"),
+ xtask_paths::repo_glob!(".github/workflows/cargo_deny.yml"),
+];
/// Build the workflow.
pub fn cargo_deny() -> Workflow {
+ let pull_request = PullRequest::default()
+ .add_branch("main")
+ .add_type(PullRequestType::Opened)
+ .add_type(PullRequestType::Synchronize)
+ .add_type(PullRequestType::Reopened)
+ .add_type(PullRequestType::ReadyForReview);
+
Workflow::new("cargo deny check")
- .on(Event::default().pull_request(
- PullRequest::default()
- .add_branch("main")
- .add_type(PullRequestType::Opened)
- .add_type(PullRequestType::Synchronize)
- .add_type(PullRequestType::Reopened)
- .add_type(PullRequestType::ReadyForReview)
- .add_path(xtask_paths::repo_glob!("Cargo.toml"))
- .add_path(xtask_paths::repo_glob!("crates/**/Cargo.toml"))
- .add_path(xtask_paths::repo_glob!("services/**/Cargo.toml"))
- .add_path(xtask_paths::repo_glob!("tooling/xtask/**/Cargo.toml"))
- .add_path(xtask_paths::repo_glob!("Cargo.lock"))
- .add_path(xtask_paths::repo_glob!("deny.toml"))
- .add_path(xtask_paths::repo_glob!(".github/workflows/cargo_deny.yml")),
- ))
- .concurrency(
- Concurrency::new(Expression::new("cargo-deny-${{ github.ref }}"))
- .cancel_in_progress(true),
- )
+ .on(Event::default()
+ .pull_request(
+ PATHS
+ .iter()
+ .fold(pull_request, |trigger, path| trigger.add_path(*path)),
+ )
+ .workflow_call(WorkflowCall::default()))
+ .concurrency(vars::concurrency("cargo-deny"))
.add_job("cargo-deny", cargo_deny_job())
}
diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/cargo_workspace_dependency_check.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/cargo_workspace_dependency_check.rs
index 674d11438b..1372be6659 100644
--- a/tooling/xtask/crates/xtask_workflows/src/workflows/cargo_workspace_dependency_check.rs
+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/cargo_workspace_dependency_check.rs
@@ -10,48 +10,47 @@
//! and xtask builds warm caches instead of compiling cold on every run.
use gh_workflow::{
- Concurrency, Event, Expression, Job, PullRequest, PullRequestType, Run, Step, Use, Workflow,
+ Event, Job, PullRequest, PullRequestType, Run, Step, Use, Workflow, WorkflowCall,
};
+use xtask_paths::RepoGlob;
use crate::workflows::{runners, steps, vars};
+/// Files whose change runs the workflow, on a push and in `PR CI`.
+pub const PATHS: &[RepoGlob<'static>] = &[
+ xtask_paths::repo_glob!("Cargo.toml"),
+ xtask_paths::repo_glob!("crates/**/Cargo.toml"),
+ xtask_paths::repo_glob!("services/**/Cargo.toml"),
+ xtask_paths::repo_glob!("tooling/xtask/**/Cargo.toml"),
+ xtask_paths::repo_glob!("Cargo.lock"),
+ xtask_paths::repo_glob!(".config/hakari.toml"),
+ xtask_paths::repo_glob!("crates/workspace-hack/**"),
+ // Source of `cargo x deps` only — not every xtask crate. A nextest-filter
+ // or workflow-generator edit cannot drift the hack crate or closure map.
+ xtask_paths::repo_glob!("tooling/xtask/src/**"),
+ xtask_paths::repo_glob!("tooling/xtask/crates/xtask_deps/**"),
+ xtask_paths::repo_glob!("tooling/xtask/crates/xtask_graph/**"),
+ xtask_paths::repo_glob!(".github/workspace-dep-closures.json"),
+];
+
/// Build the workflow.
pub fn cargo_workspace_dependency_check() -> Workflow {
+ let pull_request = PullRequest::default()
+ .add_branch("main")
+ .add_type(PullRequestType::Opened)
+ .add_type(PullRequestType::Synchronize)
+ .add_type(PullRequestType::Reopened)
+ .add_type(PullRequestType::ReadyForReview);
+
Workflow::new("cargo workspace dependency check")
- .on(Event::default().pull_request(
- PullRequest::default()
- .add_branch("main")
- .add_type(PullRequestType::Opened)
- .add_type(PullRequestType::Synchronize)
- .add_type(PullRequestType::Reopened)
- .add_type(PullRequestType::ReadyForReview)
- .add_path(xtask_paths::repo_glob!("Cargo.toml"))
- .add_path(xtask_paths::repo_glob!("crates/**/Cargo.toml"))
- .add_path(xtask_paths::repo_glob!("services/**/Cargo.toml"))
- .add_path(xtask_paths::repo_glob!("tooling/xtask/**/Cargo.toml"))
- .add_path(xtask_paths::repo_glob!("Cargo.lock"))
- .add_path(xtask_paths::repo_glob!(".config/hakari.toml"))
- .add_path(xtask_paths::repo_glob!("crates/workspace-hack/**"))
- // Source of `cargo x deps` only — not every xtask crate. A
- // nextest-filter or workflow-generator edit cannot drift the
- // hack crate or closure map.
- .add_path(xtask_paths::repo_glob!("tooling/xtask/src/**"))
- .add_path(xtask_paths::repo_glob!(
- "tooling/xtask/crates/xtask_deps/**"
- ))
- .add_path(xtask_paths::repo_glob!(
- "tooling/xtask/crates/xtask_graph/**"
- ))
- .add_path(xtask_paths::repo_glob!(
- ".github/workspace-dep-closures.json"
- )),
- ))
- .concurrency(
- Concurrency::new(Expression::new(
- "cargo-workspace-dependency-check-${{ github.ref }}",
- ))
- .cancel_in_progress(true),
- )
+ .on(Event::default()
+ .pull_request(
+ PATHS
+ .iter()
+ .fold(pull_request, |trigger, path| trigger.add_path(*path)),
+ )
+ .workflow_call(WorkflowCall::default()))
+ .concurrency(vars::concurrency("cargo-workspace-dependency-check"))
.add_job("dependency-check", dependency_check())
.add_job("hakari-and-dep-closures", hakari_and_dep_closures())
}
diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/check_generated.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/check_generated.rs
index 5c1ca6b647..af486acc3d 100644
--- a/tooling/xtask/crates/xtask_workflows/src/workflows/check_generated.rs
+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/check_generated.rs
@@ -4,21 +4,31 @@
//! (and the kafka topics file) are defined in Rust" enforceable rather than
//! aspirational.
-use gh_workflow::{Event, Job, PullRequest, Step, Workflow};
+use gh_workflow::{Event, Job, PullRequest, Step, Workflow, WorkflowCall};
+use xtask_paths::RepoGlob;
use crate::workflows::{runners, steps};
+/// Files whose change runs the workflow, on a push and in `PR CI`.
+pub const PATHS: &[RepoGlob<'static>] = &[
+ xtask_paths::repo_glob!("tooling/xtask/**"),
+ xtask_paths::repo_glob!("crates/macro_event_topics/**"),
+ xtask_paths::repo_glob!(".github/workflows/**"),
+ xtask_paths::repo_glob!(".github/kafka-cluster-topics.json"),
+];
+
/// Build the workflow.
pub fn check_generated_workflows() -> Workflow {
+ let pull_request = PullRequest::default().add_branch("main");
+
Workflow::new("check generated workflows")
- .on(Event::default().pull_request(
- PullRequest::default()
- .add_branch("main")
- .add_path(xtask_paths::repo_glob!("tooling/xtask/**"))
- .add_path(xtask_paths::repo_glob!("crates/macro_event_topics/**"))
- .add_path(xtask_paths::repo_glob!(".github/workflows/**"))
- .add_path(xtask_paths::repo_glob!(".github/kafka-cluster-topics.json")),
- ))
+ .on(Event::default()
+ .pull_request(
+ PATHS
+ .iter()
+ .fold(pull_request, |trigger, path| trigger.add_path(*path)),
+ )
+ .workflow_call(WorkflowCall::default()))
.add_job("check-workflows", check_workflows())
}
diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/check_node_modules_nix.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/check_node_modules_nix.rs
index 826bcb24ff..fbb6dce34b 100644
--- a/tooling/xtask/crates/xtask_workflows/src/workflows/check_node_modules_nix.rs
+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/check_node_modules_nix.rs
@@ -2,41 +2,45 @@
//! derivation still matches the committed lockfile/package manifests. Generated
//! into `check_node_modules_nix.yml`.
-use gh_workflow::{Event, Job, Level, Permissions, PullRequest, Step, Workflow};
+use gh_workflow::{Event, Job, Level, Permissions, PullRequest, Step, Workflow, WorkflowCall};
+use xtask_paths::RepoGlob;
use crate::workflows::{runners, steps};
+/// Files whose change runs the workflow, on a push and in `PR CI`.
+pub const PATHS: &[RepoGlob<'static>] = &[
+ xtask_paths::repo_glob!("bun.lock"),
+ xtask_paths::repo_glob!("packages/sdk/bun.lock"),
+ xtask_paths::repo_glob!("package.json"),
+ xtask_paths::repo_glob!("apps/**/package.json"),
+ xtask_paths::repo_glob!("packages/**/package.json"),
+ xtask_paths::repo_glob!("services/**/package.json"),
+ xtask_paths::repo_glob!("apps/web/patches/**"),
+ xtask_paths::repo_glob!("nix-support/node_modules.nix"),
+ xtask_paths::repo_glob!("nix-support/node_modules-hashes.json"),
+ xtask_paths::repo_glob!("nix/tauri-desktop.nix"),
+ xtask_paths::repo_glob!("tooling/scripts/update-node-modules-hash.sh"),
+ xtask_paths::repo_glob!("flake.nix"),
+ xtask_paths::repo_glob!("flake.lock"),
+ xtask_paths::repo_glob!(".github/workflows/check_node_modules_nix.yml"),
+];
+
/// Build the workflow.
pub fn check_node_modules_nix() -> Workflow {
+ let pull_request = PullRequest::default().add_branch("main");
+
Workflow::new("check node_modules nix")
.permissions(Permissions {
contents: Some(Level::Read),
..Default::default()
})
- .on(Event::default().pull_request(
- PullRequest::default()
- .add_branch("main")
- .add_path(xtask_paths::repo_glob!("bun.lock"))
- .add_path(xtask_paths::repo_glob!("packages/sdk/bun.lock"))
- .add_path(xtask_paths::repo_glob!("package.json"))
- .add_path(xtask_paths::repo_glob!("apps/**/package.json"))
- .add_path(xtask_paths::repo_glob!("packages/**/package.json"))
- .add_path(xtask_paths::repo_glob!("services/**/package.json"))
- .add_path(xtask_paths::repo_glob!("apps/web/patches/**"))
- .add_path(xtask_paths::repo_glob!("nix-support/node_modules.nix"))
- .add_path(xtask_paths::repo_glob!(
- "nix-support/node_modules-hashes.json"
- ))
- .add_path(xtask_paths::repo_glob!("nix/tauri-desktop.nix"))
- .add_path(xtask_paths::repo_glob!(
- "tooling/scripts/update-node-modules-hash.sh"
- ))
- .add_path(xtask_paths::repo_glob!("flake.nix"))
- .add_path(xtask_paths::repo_glob!("flake.lock"))
- .add_path(xtask_paths::repo_glob!(
- ".github/workflows/check_node_modules_nix.yml"
- )),
- ))
+ .on(Event::default()
+ .pull_request(
+ PATHS
+ .iter()
+ .fold(pull_request, |trigger, path| trigger.add_path(*path)),
+ )
+ .workflow_call(WorkflowCall::default()))
.add_job(
"check-linux-node-modules-nix",
check_linux_node_modules_nix(),
diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/check_tauri_lockfiles.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/check_tauri_lockfiles.rs
index bd96d3268a..67caabfc2a 100644
--- a/tooling/xtask/crates/xtask_workflows/src/workflows/check_tauri_lockfiles.rs
+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/check_tauri_lockfiles.rs
@@ -9,32 +9,40 @@
//! runs only for those.
//! Generated into `check_tauri_lockfiles.yml`.
-use gh_workflow::{Event, Expression, Job, Level, Permissions, PullRequest, Step, Use, Workflow};
+use gh_workflow::{
+ Event, Expression, Job, Level, Permissions, PullRequest, Step, Use, Workflow, WorkflowCall,
+};
+use xtask_paths::RepoGlob;
use crate::workflows::{runners, steps, vars};
+/// Files whose change runs the workflow, on a push and in `PR CI`.
+pub const PATHS: &[RepoGlob<'static>] = &[
+ xtask_paths::repo_glob!("apps/web/tauri/**"),
+ xtask_paths::repo_glob!("crates/**/Cargo.toml"),
+ xtask_paths::repo_glob!("Cargo.toml"),
+ xtask_paths::repo_glob!("nix/tauri-desktop.nix"),
+ xtask_paths::repo_glob!("flake.nix"),
+ xtask_paths::repo_glob!("flake.lock"),
+ xtask_paths::repo_glob!(".github/workflows/check_tauri_lockfiles.yml"),
+];
+
/// Build the workflow.
pub fn check_tauri_lockfiles() -> Workflow {
+ let pull_request = PullRequest::default().add_branch("main");
+
Workflow::new("check tauri lockfiles")
.permissions(Permissions {
contents: Some(Level::Read),
..Default::default()
})
- .on(Event::default().pull_request(
- PullRequest::default()
- .add_branch("main")
- .add_path(xtask_paths::repo_glob!("apps/web/tauri/**"))
- // Repository crates the Tauri workspaces reach by path, and the
- // root workspace they inherit dependencies from.
- .add_path(xtask_paths::repo_glob!("crates/**/Cargo.toml"))
- .add_path(xtask_paths::repo_glob!("Cargo.toml"))
- .add_path(xtask_paths::repo_glob!("nix/tauri-desktop.nix"))
- .add_path(xtask_paths::repo_glob!("flake.nix"))
- .add_path(xtask_paths::repo_glob!("flake.lock"))
- .add_path(xtask_paths::repo_glob!(
- ".github/workflows/check_tauri_lockfiles.yml"
- )),
- ))
+ .on(Event::default()
+ .pull_request(
+ PATHS
+ .iter()
+ .fold(pull_request, |trigger, path| trigger.add_path(*path)),
+ )
+ .workflow_call(WorkflowCall::default()))
.add_job("check-tauri-lockfiles", check_tauri_lockfiles_job())
.add_job("check-tauri-vendoring", check_tauri_vendoring_job())
}
diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/ci_gate_fork.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/ci_gate_fork.rs
new file mode 100644
index 0000000000..4e73d91a21
--- /dev/null
+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/ci_gate_fork.rs
@@ -0,0 +1,79 @@
+//! `CI Gate for Forks` — what `PR CI`'s start and gate steps can't do for a
+//! fork PR, whose token is read-only. Generated into `ci-gate-fork.yml`.
+//!
+//! Both triggers run this file from `main` with a write token and never check
+//! out the fork's code. When the label goes on, `request` removes it and opens
+//! a pending `CI gate` request on the head it was added at. When the fork's run
+//! ends, `report` copies its verdict, but only onto a commit with an open
+//! request: a fork token can't set statuses, so a run on any other commit, such
+//! as one pushed with a rewritten workflow, reports nothing.
+
+use gh_workflow::{
+ Event, Expression, Job, Level, Permissions, PullRequestTarget, PullRequestType, Workflow,
+ WorkflowRun, WorkflowRunType,
+};
+
+use crate::workflows::pr_ci;
+
+const FORK_PULL_REQUEST: &str =
+ "github.event.pull_request.head.repo.full_name != github.repository";
+
+/// Build the workflow.
+pub fn ci_gate_fork() -> Workflow {
+ Workflow::new("CI Gate for Forks")
+ .on(Event::default()
+ .pull_request_target(
+ PullRequestTarget::default()
+ .add_branch("main")
+ .add_type(PullRequestType::Labeled),
+ )
+ .workflow_run(
+ WorkflowRun::default()
+ .add_type(WorkflowRunType::Completed)
+ .add_workflow("PR CI"),
+ ))
+ .add_job("request", request())
+ .add_job("report", report())
+}
+
+fn request() -> Job {
+ Job::default()
+ .name("Take a fork's CI request")
+ .cond(Expression::new(format!(
+ "github.event_name == 'pull_request_target' && github.event.label.name == '{}' && {FORK_PULL_REQUEST}",
+ pr_ci::RUN_LABEL
+ )))
+ .runs_on("ubuntu-latest")
+ .timeout_minutes(5u32)
+ .permissions(Permissions {
+ issues: Some(Level::Write),
+ pull_requests: Some(Level::Write),
+ statuses: Some(Level::Write),
+ ..Default::default()
+ })
+ .add_step(
+ pr_ci::gate_step("Open the CI gate request", "fork-request")
+ .add_env(("PR_NUMBER", "${{ github.event.pull_request.number }}"))
+ .add_env(("HEAD_SHA", "${{ github.event.pull_request.head.sha }}"))
+ .add_env((
+ "PR_CHECKS_URL",
+ "${{ github.event.pull_request.html_url }}/checks",
+ )),
+ )
+}
+
+fn report() -> Job {
+ Job::default()
+ .name("Report a fork's CI result")
+ .cond(Expression::new(
+ "github.event_name == 'workflow_run' && github.event.workflow_run.event == 'pull_request' && github.event.workflow_run.head_repository.full_name != github.repository",
+ ))
+ .runs_on("ubuntu-latest")
+ .timeout_minutes(5u32)
+ .permissions(Permissions {
+ actions: Some(Level::Read),
+ statuses: Some(Level::Write),
+ ..Default::default()
+ })
+ .add_step(pr_ci::gate_step("Set the CI gate status", "fork"))
+}
diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/ci_gate_reminder.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/ci_gate_reminder.rs
new file mode 100644
index 0000000000..57b1944103
--- /dev/null
+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/ci_gate_reminder.rs
@@ -0,0 +1,46 @@
+//! `CI Gate Reminder` — on every new PR commit, sets the `CI gate` status to
+//! pending with the command that runs CI, so the commit reads as not passed
+//! until someone runs [`pr_ci`](crate::workflows::pr_ci). Generated into
+//! `ci-gate-reminder.yml`.
+
+use gh_workflow::{
+ Event, Expression, Job, Level, Permissions, PullRequest, PullRequestType, Workflow,
+};
+
+use crate::workflows::pr_ci;
+
+/// Build the workflow.
+pub fn ci_gate_reminder() -> Workflow {
+ Workflow::new("CI Gate Reminder")
+ .on(Event::default().pull_request(
+ PullRequest::default()
+ .add_branch("main")
+ .add_type(PullRequestType::Opened)
+ .add_type(PullRequestType::Synchronize)
+ .add_type(PullRequestType::Reopened),
+ ))
+ .add_job("remind", remind())
+}
+
+/// GitHub's hosted runner: a status call needs no Namespace machine. A fork's
+/// read-only token can't set statuses, so its commits show the required status
+/// as expected instead.
+fn remind() -> Job {
+ Job::default()
+ .name("CI not run yet")
+ .cond(Expression::new(
+ "github.event.pull_request.head.repo.full_name == github.repository",
+ ))
+ .runs_on("ubuntu-latest")
+ .timeout_minutes(5u32)
+ .permissions(Permissions {
+ statuses: Some(Level::Write),
+ ..Default::default()
+ })
+ .add_step(
+ pr_ci::gate_step("Mark the commit as not run", "remind")
+ .add_env(("PR_NUMBER", "${{ github.event.pull_request.number }}"))
+ .add_env(("HEAD_SHA", "${{ github.event.pull_request.head.sha }}"))
+ .add_env(("CI_GATE_DOCS_URL", pr_ci::GATE_DOCS_URL)),
+ )
+}
diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/code_check_cloud_storage.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/code_check_cloud_storage.rs
index 99c7a6bd4d..261dac6a2a 100644
--- a/tooling/xtask/crates/xtask_workflows/src/workflows/code_check_cloud_storage.rs
+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/code_check_cloud_storage.rs
@@ -11,7 +11,8 @@
//! `graphql-schema-compat` run alongside `test`.
use gh_workflow::{
- Container, Env, Event, Expression, Job, Port, PullRequest, PullRequestType, Run, Step, Workflow,
+ Container, Env, Event, Expression, Job, Port, PullRequest, PullRequestType, Run, Step,
+ Workflow, WorkflowCall,
};
use xtask_paths::RuntimePath;
@@ -27,14 +28,16 @@ mod test;
/// Build the workflow.
pub fn code_check_cloud_storage() -> Workflow {
Workflow::new("cloud storage code check")
- .on(Event::default().pull_request(
- PullRequest::default()
- .add_branch("main")
- .add_type(PullRequestType::Opened)
- .add_type(PullRequestType::Synchronize)
- .add_type(PullRequestType::Reopened)
- .add_type(PullRequestType::ReadyForReview),
- ))
+ .on(Event::default()
+ .pull_request(
+ PullRequest::default()
+ .add_branch("main")
+ .add_type(PullRequestType::Opened)
+ .add_type(PullRequestType::Synchronize)
+ .add_type(PullRequestType::Reopened)
+ .add_type(PullRequestType::ReadyForReview),
+ )
+ .workflow_call(WorkflowCall::default()))
.map(vars::with_global_env)
.concurrency(vars::concurrency("code-check-cloud-storage"))
.add_job("path-check", path_check())
@@ -134,7 +137,7 @@ fn doppler_config() -> Job {
steps::gated_job()
.timeout_minutes(45u32)
.cond(Expression::new(
- "needs.path-check.outputs.should_run == 'true' && github.event.pull_request.draft == false && needs.path-check.outputs.doppler_candidates == 'true'",
+ "needs.path-check.outputs.should_run == 'true' && (github.event.pull_request.draft == false || github.event.action == 'labeled') && needs.path-check.outputs.doppler_candidates == 'true'",
))
.runs_on(runners::Runner::RustCi.with_cache_tag(vars::CI_CACHE_TAG))
.map(with_check_env)
@@ -160,7 +163,7 @@ fn graphql_schema_compat() -> Job {
steps::gated_job()
.timeout_minutes(15u32)
.cond(Expression::new(
- "needs.path-check.outputs.should_run == 'true' && github.event.pull_request.draft == false && needs.path-check.outputs.graphql_schema == 'true'",
+ "needs.path-check.outputs.should_run == 'true' && (github.event.pull_request.draft == false || github.event.action == 'labeled') && needs.path-check.outputs.graphql_schema == 'true'",
))
.runs_on(runners::Runner::Small.to_string())
.add_step(steps::checkout(false, false).add_with(("fetch-depth", 2)))
@@ -179,7 +182,7 @@ fn test() -> Job {
steps::gated_job()
.timeout_minutes(90u32)
.cond(Expression::new(
- "needs.path-check.outputs.should_run == 'true' && github.event.pull_request.draft == false && needs.path-check.outputs.rust_packages != 'none' && needs.path-check.outputs.skip_tests != 'true'",
+ "needs.path-check.outputs.should_run == 'true' && (github.event.pull_request.draft == false || github.event.action == 'labeled') && needs.path-check.outputs.rust_packages != 'none' && needs.path-check.outputs.skip_tests != 'true'",
))
.runs_on(runners::Runner::Large.with_cache_tag(vars::CI_CACHE_TAG))
.add_env((
@@ -247,7 +250,7 @@ fn build_service_bins() -> Job {
}
const BUILD_SERVICE_BINS_CONDITION: &str = "needs.path-check.outputs.should_run == 'true' \
- && github.event.pull_request.draft == false \
+ && (github.event.pull_request.draft == false || github.event.action == 'labeled') \
&& needs.path-check.outputs.skip_tests != 'true' \
&& (needs.path-check.outputs.rust_packages == 'all' \
|| contains(needs.path-check.outputs.rust_packages, 'document_storage_service') \
diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/code_check_conventions.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/code_check_conventions.rs
index 465069ec2a..0a4b87008a 100644
--- a/tooling/xtask/crates/xtask_workflows/src/workflows/code_check_conventions.rs
+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/code_check_conventions.rs
@@ -8,8 +8,8 @@
//! conventions from `docs/development/style-guide.md` at the repo root.
use gh_workflow::{
- Concurrency, Event, Expression, Job, Level, Permissions, PullRequest, PullRequestType, Run,
- Step, Use, Workflow,
+ Event, Expression, Job, Level, Permissions, PullRequest, PullRequestType, Run, Step, Use,
+ Workflow, WorkflowCall,
};
use crate::workflows::{runners, steps, vars};
@@ -28,18 +28,17 @@ pub fn code_check_conventions() -> Workflow {
contents: Some(Level::Read),
..Default::default()
})
- .on(Event::default().pull_request(
- PullRequest::default()
- .add_branch("main")
- .add_type(PullRequestType::Opened)
- .add_type(PullRequestType::Synchronize)
- .add_type(PullRequestType::Reopened)
- .add_type(PullRequestType::ReadyForReview),
- ))
- .concurrency(
- Concurrency::new(Expression::new("code-check-conventions-${{ github.ref }}"))
- .cancel_in_progress(true),
- )
+ .on(Event::default()
+ .pull_request(
+ PullRequest::default()
+ .add_branch("main")
+ .add_type(PullRequestType::Opened)
+ .add_type(PullRequestType::Synchronize)
+ .add_type(PullRequestType::Reopened)
+ .add_type(PullRequestType::ReadyForReview),
+ )
+ .workflow_call(WorkflowCall::default()))
+ .concurrency(vars::concurrency("code-check-conventions"))
.add_job("path-check", path_check())
.add_job("ast-grep", ast_grep())
.add_job("status-check", status_check())
diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/code_check_infra.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/code_check_infra.rs
index 4d67dc8b76..02c4f22a0c 100644
--- a/tooling/xtask/crates/xtask_workflows/src/workflows/code_check_infra.rs
+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/code_check_infra.rs
@@ -5,7 +5,7 @@
//! unaffected by the file rename).
use gh_workflow::{
- Concurrency, Event, Expression, Job, PullRequest, PullRequestType, Run, Step, Use, Workflow,
+ Event, Expression, Job, PullRequest, PullRequestType, Run, Step, Use, Workflow, WorkflowCall,
};
use crate::workflows::{
@@ -26,18 +26,17 @@ fn infra_runner() -> String {
/// Build the workflow.
pub fn code_check_infra() -> Workflow {
Workflow::new("infra code check")
- .on(Event::default().pull_request(
- PullRequest::default()
- .add_branch("main")
- .add_type(PullRequestType::Opened)
- .add_type(PullRequestType::Synchronize)
- .add_type(PullRequestType::Reopened)
- .add_type(PullRequestType::ReadyForReview),
- ))
- .concurrency(
- Concurrency::new(Expression::new("code-check-infra-${{ github.ref }}"))
- .cancel_in_progress(true),
- )
+ .on(Event::default()
+ .pull_request(
+ PullRequest::default()
+ .add_branch("main")
+ .add_type(PullRequestType::Opened)
+ .add_type(PullRequestType::Synchronize)
+ .add_type(PullRequestType::Reopened)
+ .add_type(PullRequestType::ReadyForReview),
+ )
+ .workflow_call(WorkflowCall::default()))
+ .concurrency(vars::concurrency("code-check-infra"))
.add_job("path-check", path_check())
.add_job("biome-check", biome_check())
.add_job("check", check())
diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/export_ci_telemetry.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/export_ci_telemetry.rs
index a292ef165e..3184a6c8d3 100644
--- a/tooling/xtask/crates/xtask_workflows/src/workflows/export_ci_telemetry.rs
+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/export_ci_telemetry.rs
@@ -5,6 +5,8 @@ use gh_workflow::{
Event, Expression, Job, Level, Permissions, Step, Workflow, WorkflowRun, WorkflowRunType,
};
+use crate::workflows::steps;
+
#[cfg(test)]
mod test;
@@ -21,6 +23,8 @@ const OBSERVED_WORKFLOWS: &[&str] = &[
"Build Rust CI Image",
"Build Web App",
"Build macOS DMG",
+ "CI Gate Reminder",
+ "CI Gate for Forks",
"CLA",
"Cancel Stuck Cloud Storage Deploys",
"Cleanup Preview",
@@ -40,6 +44,7 @@ const OBSERVED_WORKFLOWS: &[&str] = &[
"Export Datadog observability definitions",
"Migrate Macro Database",
"Native Check Label",
+ "PR CI",
"PR Privacy Check",
"Publish Android to Google Play",
"Publish SDK",
@@ -93,14 +98,6 @@ pub fn export_ci_telemetry() -> Workflow {
.add_job("export", export())
}
-/// Run embedded TypeScript on the hosted image's Node, which strips types itself, so
-/// the exporter needs no checkout, build or dependency.
-fn node_script(source: &str) -> String {
- format!(
- "node --disable-warning=ExperimentalWarning --input-type=module-typescript <<'TELEMETRY_SCRIPT'\n{source}\nawait main();\nTELEMETRY_SCRIPT\n"
- )
-}
-
fn export() -> Job {
Job::default()
// GitHub's hosted image supplies AWS CLI, Python and Node without a checkout/setup step.
@@ -119,7 +116,9 @@ fn export() -> Job {
// Before any credential exists, so nothing the run produced shares a step with it.
.add_step(
Step::new("Collect push, job and step events")
- .run(node_script(include_str!("scripts/ci_telemetry_collect.ts")))
+ .run(steps::node_script(include_str!(
+ "scripts/ci_telemetry_collect.ts"
+ )))
.shell("bash")
.if_condition(Expression::new("always()"))
.continue_on_error(true)
@@ -153,7 +152,9 @@ fn export() -> Job {
// the only record of a run the trace exporter refuses.
.add_step(
Step::new("Send push, job and step events")
- .run(node_script(include_str!("scripts/ci_telemetry_send.ts")))
+ .run(steps::node_script(include_str!(
+ "scripts/ci_telemetry_send.ts"
+ )))
.shell("bash")
.if_condition(Expression::new("always()"))
.continue_on_error(true)
diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/pr_ci.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/pr_ci.rs
new file mode 100644
index 0000000000..eff105c5e7
--- /dev/null
+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/pr_ci.rs
@@ -0,0 +1,286 @@
+//! `PR CI` — the pull request checks, run when someone adds the [`RUN_LABEL`]
+//! label instead of on every push. Generated into `pr-ci.yml`.
+//!
+//! `start` removes the label, so adding it again runs CI again, and checks that
+//! the merge commit GitHub built holds the PR head. The check workflows run as
+//! reusable workflows; the path-filtered ones only when their files changed.
+//! `CI gate` then sets the [`GATE_CONTEXT`] commit status on the PR head, which
+//! `main` requires: a new push has no status until CI runs on it.
+//!
+//! Every label fires `labeled`, so runs for any other label skip every job and
+//! take a concurrency group of their own: they can't cancel a requested run or
+//! post a result. Fork PRs get a read-only token, so `ci_gate_fork` removes
+//! their label and sets their status.
+
+use anyhow::Result;
+use gh_workflow::{
+ Concurrency, Event, Expression, Job, Level, Permissions, PullRequest, PullRequestType, Run,
+ Step, Use, Workflow,
+};
+use xtask_paths::RepoGlob;
+
+use crate::workflows::{
+ cargo_deny, cargo_workspace_dependency_check, check_generated, check_node_modules_nix,
+ check_tauri_lockfiles, sdk_check, steps, test_macrod_update,
+};
+
+#[cfg(test)]
+mod test;
+
+/// Adding this label to a PR runs CI on its head.
+pub const RUN_LABEL: &str = "run-ci";
+
+/// The commit status `main`'s ruleset requires.
+pub const GATE_CONTEXT: &str = "CI gate";
+
+/// Where the status links before CI has run: how to run it and wait on it.
+pub const GATE_DOCS_URL: &str =
+ "https://github.com/macro-inc/macro/blob/main/.agents/skills/run-ci/SKILL.md";
+
+const REQUESTED: &str = "github.event.label.name == 'run-ci'";
+
+/// Job names the fork step reads back from a finished run (`ci_gate.ts`).
+const START_JOB: &str = "Start";
+const GATE_JOB: &str = "CI gate";
+
+/// A check workflow `PR CI` calls.
+struct Call {
+ id: &'static str,
+ /// Prefixes the called jobs' check names: `Web App / Typecheck`.
+ name: &'static str,
+ file: &'static str,
+ /// Files that make it run; `None` runs it on every request.
+ paths: Option<&'static [RepoGlob<'static>]>,
+}
+
+const CALLS: &[Call] = &[
+ Call {
+ id: "cloud-storage",
+ name: "Cloud Storage",
+ file: "code_check_cloud_storage.yml",
+ paths: None,
+ },
+ Call {
+ id: "web-app",
+ name: "Web App",
+ file: "web-app-check-main.yml",
+ paths: None,
+ },
+ Call {
+ id: "infra",
+ name: "Infra",
+ file: "code_check_infra.yml",
+ paths: None,
+ },
+ Call {
+ id: "conventions",
+ name: "Conventions",
+ file: "code_check_conventions.yml",
+ paths: None,
+ },
+ Call {
+ id: "cargo-deny",
+ name: "cargo deny",
+ file: "cargo_deny.yml",
+ paths: Some(cargo_deny::PATHS),
+ },
+ Call {
+ id: "workspace-dependencies",
+ name: "Workspace dependencies",
+ file: "cargo_workspace_dependency_check.yml",
+ paths: Some(cargo_workspace_dependency_check::PATHS),
+ },
+ Call {
+ id: "generated-workflows",
+ name: "Generated workflows",
+ file: "check_generated.yml",
+ paths: Some(check_generated::PATHS),
+ },
+ Call {
+ id: "node-modules-nix",
+ name: "node_modules nix",
+ file: "check_node_modules_nix.yml",
+ paths: Some(check_node_modules_nix::PATHS),
+ },
+ Call {
+ id: "tauri-lockfiles",
+ name: "Tauri lockfiles",
+ file: "check_tauri_lockfiles.yml",
+ paths: Some(check_tauri_lockfiles::PATHS),
+ },
+ Call {
+ id: "sdk",
+ name: "SDK",
+ file: "sdk-check.yml",
+ paths: Some(sdk_check::PATHS),
+ },
+ Call {
+ id: "macrod-update",
+ name: "macrod update",
+ file: "test_macrod_update.yml",
+ paths: Some(test_macrod_update::PATHS),
+ },
+];
+
+/// Build the workflow.
+pub fn pr_ci() -> Workflow {
+ let workflow = Workflow::new("PR CI")
+ .on(Event::default().pull_request(
+ PullRequest::default()
+ .add_branch("main")
+ .add_type(PullRequestType::Labeled),
+ ))
+ .concurrency(
+ // One group per PR for requests, so a newer request cancels the
+ // older run; a group per run for every other label.
+ Concurrency::new(Expression::new(format!(
+ "pr-ci-${{{{ github.event.pull_request.number }}}}-${{{{ {REQUESTED} && 'requested' || github.run_id }}}}"
+ )))
+ .cancel_in_progress(true),
+ )
+ .add_job("start", start());
+
+ CALLS
+ .iter()
+ .fold(workflow, |workflow, call| {
+ workflow.add_job(call.id, call_job(call))
+ })
+ .add_job("gate", gate())
+}
+
+/// Fill in what gh-workflow can't express on a reusable-workflow call.
+pub fn patch(root: &mut serde_yaml::Value) -> Result<()> {
+ for call in CALLS {
+ let job = crate::workflows::job_mut(root, call.id)?;
+ job.remove("runs-on");
+ job.insert("secrets".into(), "inherit".into());
+ }
+ Ok(())
+}
+
+/// The output that says whether a path-filtered call runs.
+fn output_name(call: &Call) -> String {
+ call.id.replace('-', "_")
+}
+
+/// A step running `scripts/ci_gate.ts` in one of its modes.
+pub(crate) fn gate_step(name: &str, mode: &str) -> Step<Run> {
+ Step::new(name)
+ .run(steps::node_script(include_str!("scripts/ci_gate.ts")))
+ .shell("bash")
+ .add_env(("CI_GATE_MODE", mode))
+ .add_env(("CI_GATE_CONTEXT", GATE_CONTEXT))
+ .add_env(("CI_GATE_LABEL", RUN_LABEL))
+ .add_env(("GH_TOKEN", "${{ github.token }}"))
+}
+
+/// The PR facts the start and gate steps act on.
+fn with_pull_request(step: Step<Run>) -> Step<Run> {
+ step.add_env(("PR_NUMBER", "${{ github.event.pull_request.number }}"))
+ .add_env(("HEAD_SHA", "${{ github.event.pull_request.head.sha }}"))
+ .add_env((
+ "SAME_REPO",
+ "${{ github.event.pull_request.head.repo.full_name == github.repository }}",
+ ))
+ .add_env((
+ "RUN_URL",
+ "${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}",
+ ))
+}
+
+fn start() -> Job {
+ let job = Job::default()
+ .name(START_JOB)
+ .cond(Expression::new(REQUESTED))
+ .runs_on("ubuntu-latest")
+ .timeout_minutes(5u32)
+ .permissions(Permissions {
+ contents: Some(Level::Read),
+ issues: Some(Level::Write),
+ pull_requests: Some(Level::Write),
+ statuses: Some(Level::Write),
+ ..Default::default()
+ })
+ .add_step(
+ with_pull_request(gate_step("Take the request", "start"))
+ .add_env(("MERGE_SHA", "${{ github.sha }}")),
+ )
+ .add_step(paths_filter());
+
+ CALLS
+ .iter()
+ .filter(|call| call.paths.is_some())
+ .fold(job, |job, call| {
+ let output = output_name(call);
+ job.add_output(
+ output.clone(),
+ format!("${{{{ steps.filter.outputs.{output} }}}}"),
+ )
+ })
+}
+
+/// One filter per path-filtered call, from the same list its push trigger uses.
+/// The PR's files come from the API, so no checkout.
+fn paths_filter() -> Step<Use> {
+ let filters: String = CALLS
+ .iter()
+ .filter_map(|call| call.paths.map(|paths| (output_name(call), paths)))
+ .map(|(output, paths)| {
+ let globs: String = paths
+ .iter()
+ .map(|path| format!(" - '{}'\n", path.as_str()))
+ .collect();
+ format!("{output}:\n{globs}")
+ })
+ .collect();
+
+ Step::new("Filter changed paths")
+ .uses(
+ "dorny",
+ "paths-filter",
+ "d1c1ffe0248fe513906c8e24db8ea791d46f8590",
+ ) // v3.0.3
+ .id("filter")
+ .add_with(("filters", filters))
+}
+
+fn call_job(call: &Call) -> Job {
+ let job = Job::default()
+ .name(call.name)
+ .needs(vec!["start".to_string()])
+ .uses(format!("./.github/workflows/{}", call.file));
+
+ match call.paths {
+ Some(_) => job.cond(Expression::new(format!(
+ "needs.start.outputs.{} == 'true'",
+ output_name(call)
+ ))),
+ None => job,
+ }
+}
+
+/// Runs unless the run was cancelled (a newer request superseded it) or was
+/// never a request. A skipped call is a path filter that didn't match.
+fn gate() -> Job {
+ let needs = std::iter::once("start")
+ .chain(CALLS.iter().map(|call| call.id))
+ .map(str::to_string)
+ .collect::<Vec<_>>();
+
+ Job::default()
+ .name(GATE_JOB)
+ .needs(needs)
+ .cond(Expression::new(
+ "!cancelled() && needs.start.result != 'skipped'",
+ ))
+ .runs_on("ubuntu-latest")
+ .timeout_minutes(5u32)
+ .permissions(Permissions {
+ statuses: Some(Level::Write),
+ ..Default::default()
+ })
+ .add_step(
+ with_pull_request(gate_step("Set the CI gate status", "finish"))
+ .add_env(("NEEDS", "${{ toJSON(needs) }}")),
+ )
+}
diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/pr_ci/test.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/pr_ci/test.rs
new file mode 100644
index 0000000000..9597b23eff
--- /dev/null
+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/pr_ci/test.rs
@@ -0,0 +1,90 @@
+use super::*;
+
+fn rendered() -> serde_yaml::Value {
+ let yaml = crate::workflows::render_patched(pr_ci, patch).expect("PR CI should render");
+ serde_yaml::from_str(&yaml).expect("PR CI should parse")
+}
+
+/// A called workflow the gate doesn't wait for could fail after the gate passed.
+#[test]
+fn the_gate_waits_for_start_and_every_call() {
+ let workflow = rendered();
+ let needs: Vec<&str> = workflow["jobs"]["gate"]["needs"]
+ .as_sequence()
+ .expect("gate needs")
+ .iter()
+ .filter_map(serde_yaml::Value::as_str)
+ .collect();
+
+ assert_eq!(needs[0], "start");
+ assert_eq!(
+ &needs[1..],
+ CALLS.iter().map(|call| call.id).collect::<Vec<_>>()
+ );
+}
+
+/// GitHub refuses to call a workflow without a `workflow_call` trigger.
+#[test]
+fn every_called_workflow_can_be_called() {
+ let workflows = concat!(env!("CARGO_MANIFEST_DIR"), "/../../../../.github/workflows");
+ for call in CALLS {
+ let path = format!("{workflows}/{}", call.file);
+ let workflow: serde_yaml::Value =
+ serde_yaml::from_str(&std::fs::read_to_string(&path).expect("called workflow exists"))
+ .expect("called workflow parses");
+ let triggers = &workflow["on"];
+
+ assert!(
+ triggers.get("workflow_call").is_some(),
+ "{} has no workflow_call trigger",
+ call.file
+ );
+ }
+}
+
+/// Runs for other labels must neither share a group with a request (and
+/// cancel it) nor reach a called workflow.
+#[test]
+fn only_the_run_label_starts_anything() {
+ let workflow = rendered();
+
+ assert_eq!(
+ workflow["jobs"]["start"]["if"].as_str(),
+ Some("github.event.label.name == 'run-ci'")
+ );
+ for call in CALLS {
+ let needs = &workflow["jobs"][call.id]["needs"];
+ assert_eq!(
+ needs[0].as_str(),
+ Some("start"),
+ "{} must wait for start",
+ call.id
+ );
+ }
+ let group = workflow["concurrency"]["group"].as_str().expect("group");
+ assert!(group.contains("github.event.label.name == 'run-ci' && 'requested' || github.run_id"));
+}
+
+/// Actions would expand this in the embedded source before the shell ever saw it.
+#[test]
+fn the_gate_script_contains_no_actions_expressions() {
+ assert!(!include_str!("../scripts/ci_gate.ts").contains("${{"));
+}
+
+#[test]
+fn gate_script_scenarios() {
+ let output = std::process::Command::new("node")
+ .args(["--test", "ci_gate.test.ts"])
+ .current_dir(concat!(
+ env!("CARGO_MANIFEST_DIR"),
+ "/src/workflows/scripts"
+ ))
+ .output()
+ .expect("run CI gate scenarios with Node");
+ assert!(
+ output.status.success(),
+ "CI gate scenarios failed:\n{}\n{}",
+ String::from_utf8_lossy(&output.stdout),
+ String::from_utf8_lossy(&output.stderr)
+ );
+}
diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/scripts/android_release_preconditions.sh b/tooling/xtask/crates/xtask_workflows/src/workflows/scripts/android_release_preconditions.sh
index 43ef184c24..e7b9c4996a 100644
--- a/tooling/xtask/crates/xtask_workflows/src/workflows/scripts/android_release_preconditions.sh
+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/scripts/android_release_preconditions.sh
@@ -49,8 +49,10 @@ echo "Releasing Android version code ${version_code}."
# (`gh pr checks` names are job names, not workflow names, plus the contexts
# of commit statuses, such as the android-play/* ones Android Play Publish
-# and Android Release Watch set.)
-ours='^(Android Native Check|Android Play Release|Re-run the native check|Web App Status Check|Android Release Drift|iOS Native Check|android-play/)'
+# and Android Release Watch set. PR CI prefixes the jobs it calls with
+# `Web App / `. The CI gate status aggregates every check, and stays pending
+# on a head no one has run PR CI on.)
+ours='^((Web App / )?(Android Native Check|Web App Status Check|iOS Native Check)|Android Play Release|Re-run the native check|Android Release Drift|android-play/|CI gate$)'
deadline=$(( $(date +%s) + WAIT_MINUTES * 60 ))
while true; do
checks=$(gh pr checks "$PR_NUMBER" --repo "$GH_REPO" --json name,bucket \
diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/scripts/ci_gate.test.ts b/tooling/xtask/crates/xtask_workflows/src/workflows/scripts/ci_gate.test.ts
new file mode 100644
index 0000000000..09d0493412
--- /dev/null
+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/scripts/ci_gate.test.ts
@@ -0,0 +1,133 @@
+// Verdicts of the CI gate. Run by xtask_workflows' tests.
+
+import assert from 'node:assert/strict';
+import { describe, test } from 'node:test';
+import { forkOutcome, GATE_JOB, gateOutcome, isOpenRequest, START_JOB } from './ci_gate.ts';
+
+const HEAD = '0123456789abcdef0123456789abcdef01234567';
+
+describe('gateOutcome', () => {
+ test('passes when every called workflow passed or was path-skipped', () => {
+ const outcome = gateOutcome(
+ {
+ start: { result: 'success' },
+ 'web-app': { result: 'success' },
+ 'cargo-deny': { result: 'skipped' },
+ },
+ HEAD,
+ );
+
+ assert.deepEqual(outcome, { state: 'success', description: 'Passed on 0123456' });
+ });
+
+ test('fails naming every workflow that did not pass', () => {
+ const outcome = gateOutcome(
+ {
+ start: { result: 'success' },
+ 'web-app': { result: 'failure' },
+ 'cloud-storage': { result: 'cancelled' },
+ infra: { result: 'success' },
+ },
+ HEAD,
+ );
+
+ assert.deepEqual(outcome, { state: 'failure', description: 'Failed: web-app, cloud-storage' });
+ });
+
+ test('leaves the status alone when start failed, because start already explained why', () => {
+ const outcome = gateOutcome(
+ { start: { result: 'failure' }, 'web-app': { result: 'skipped' } },
+ HEAD,
+ );
+
+ assert.equal(outcome, null);
+ });
+
+ test('a skipped start never passes', () => {
+ const outcome = gateOutcome(
+ { start: { result: 'skipped' }, 'web-app': { result: 'skipped' } },
+ HEAD,
+ );
+
+ assert.equal(outcome, null);
+ });
+});
+
+describe('forkOutcome', () => {
+ test('an unrelated label run reports nothing', () => {
+ const outcome = forkOutcome(
+ [
+ { name: START_JOB, conclusion: 'skipped' },
+ { name: GATE_JOB, conclusion: 'skipped' },
+ ],
+ HEAD,
+ 'run-ci',
+ );
+
+ assert.equal(outcome, null);
+ });
+
+ test('a cancelled run reports nothing', () => {
+ const outcome = forkOutcome(
+ [
+ { name: START_JOB, conclusion: 'success' },
+ { name: GATE_JOB, conclusion: 'cancelled' },
+ ],
+ HEAD,
+ 'run-ci',
+ );
+
+ assert.equal(outcome, null);
+ });
+
+ test('copies the gate job verdict', () => {
+ const passed = forkOutcome(
+ [
+ { name: START_JOB, conclusion: 'success' },
+ { name: 'Web App / Typecheck', conclusion: 'success' },
+ { name: GATE_JOB, conclusion: 'success' },
+ ],
+ HEAD,
+ 'run-ci',
+ );
+ const failed = forkOutcome(
+ [
+ { name: START_JOB, conclusion: 'success' },
+ { name: GATE_JOB, conclusion: 'failure' },
+ ],
+ HEAD,
+ 'run-ci',
+ );
+
+ assert.equal(passed?.state, 'success');
+ assert.equal(failed?.state, 'failure');
+ });
+
+ test('a failed start fails with the command to run again', () => {
+ const outcome = forkOutcome(
+ [
+ { name: START_JOB, conclusion: 'failure' },
+ { name: GATE_JOB, conclusion: 'failure' },
+ ],
+ HEAD,
+ 'run-ci',
+ );
+
+ assert.deepEqual(outcome, {
+ state: 'failure',
+ description: 'Could not start on 0123456. Add the run-ci label again.',
+ });
+ });
+});
+
+describe('isOpenRequest', () => {
+ test('only a pending request that no result has closed', () => {
+ assert.equal(isOpenRequest({ state: 'pending', description: 'Running on 0123456' }), true);
+ assert.equal(isOpenRequest({ state: 'success', description: 'Passed on 0123456' }), false);
+ assert.equal(
+ isOpenRequest({ state: 'pending', description: 'Not run on this commit. Run: ...' }),
+ false,
+ );
+ assert.equal(isOpenRequest(undefined), false);
+ });
+});
diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/scripts/ci_gate.ts b/tooling/xtask/crates/xtask_workflows/src/workflows/scripts/ci_gate.ts
new file mode 100644
index 0000000000..7e50caf2e2
--- /dev/null
+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/scripts/ci_gate.ts
@@ -0,0 +1,272 @@
+// Sets the commit status `main` requires before a merge: it is green only on a
+// commit that a requested `PR CI` run passed. `CI_GATE_MODE` picks the step:
+//
+// remind a new commit: pending "not run" with the command to run it, unless
+// the commit already has a gate status
+// start a requested run begins: remove the label, check the merge commit
+// holds the PR head, set pending
+// finish the run ends: success when every job passed or was path-skipped
+// fork-request the label goes on a fork PR: what start does with a write
+// token, from a trusted run of `main`'s workflow
+// fork after a fork PR's run, whose read-only token could not set the status
+//
+// Statuses go on the PR head commit, so a later push starts without one.
+
+import { readFileSync } from 'node:fs';
+
+const API = 'https://api.github.com';
+const DESCRIPTION_LIMIT = 140;
+// An open request. Only trusted runs set statuses: a fork's token can't.
+const RUNNING = 'Running on';
+// Job names in `PR CI`; the fork step reads them back from the finished run.
+export const START_JOB = 'Start';
+export const GATE_JOB = 'CI gate';
+
+type State = 'pending' | 'success' | 'failure' | 'error';
+export type Needs = Record<string, { result: string }>;
+export type RunJob = { name: string; conclusion: string | null };
+type Outcome = { state: State; description: string } | null;
+
+function env(name: string): string {
+ const value = process.env[name];
+ if (!value) {
+ throw new Error(`${name} is not set`);
+ }
+ return value;
+}
+
+async function github(method: string, path: string, body?: unknown): Promise<any> {
+ const response = await fetch(`${API}${path}`, {
+ method,
+ headers: {
+ accept: 'application/vnd.github+json',
+ authorization: `Bearer ${env('GH_TOKEN')}`,
+ 'x-github-api-version': '2022-11-28',
+ },
+ body: body === undefined ? undefined : JSON.stringify(body),
+ });
+ // Removing a label that is already gone is the outcome we wanted.
+ if (method === 'DELETE' && response.status === 404) {
+ return null;
+ }
+ if (!response.ok) {
+ throw new Error(`${method} ${path}: HTTP ${response.status} ${await response.text()}`);
+ }
+ return response.status === 204 ? null : response.json();
+}
+
+const repo = () => env('GITHUB_REPOSITORY');
+const context = () => env('CI_GATE_CONTEXT');
+const label = () => env('CI_GATE_LABEL');
+
+/// REST, because `gh pr edit` fails on older gh releases.
+export function runCommand(repository: string, pr: string, labelName: string): string {
+ return `gh api repos/${repository}/issues/${pr}/labels -f "labels[]=${labelName}"`;
+}
+
+/// The finished run's verdict from `toJSON(needs)`. Start must have succeeded;
+/// a called workflow may be skipped, because its path filter did not match.
+/// Null when start failed: it already set a status that says why.
+export function gateOutcome(needs: Needs, head: string): Outcome {
+ if (needs.start?.result !== 'success') {
+ return null;
+ }
+
+ const failed = Object.entries(needs)
+ .filter(([, need]) => need.result !== 'success' && need.result !== 'skipped')
+ .map(([job]) => job);
+
+ if (failed.length > 0) {
+ return { state: 'failure', description: `Failed: ${failed.join(', ')}` };
+ }
+
+ return { state: 'success', description: `Passed on ${head.slice(0, 7)}` };
+}
+
+/// A fork run's verdict from its jobs. Null when the run was not a request
+/// (unrelated label) or did not finish (cancelled).
+export function forkOutcome(jobs: RunJob[], head: string, labelName: string): Outcome {
+ const start = jobs.find((job) => job.name === START_JOB);
+ const gate = jobs.find((job) => job.name === GATE_JOB);
+
+ if (!start || start.conclusion === 'skipped' || start.conclusion === 'cancelled') {
+ return null;
+ }
+
+ if (start.conclusion !== 'success') {
+ return {
+ state: 'failure',
+ description: `Could not start on ${head.slice(0, 7)}. Add the ${labelName} label again.`,
+ };
+ }
+
+ if (gate?.conclusion === 'success') {
+ return { state: 'success', description: `Passed on ${head.slice(0, 7)}` };
+ }
+
+ if (gate?.conclusion === 'failure') {
+ return { state: 'failure', description: 'Failed: see the PR CI run' };
+ }
+
+ return null;
+}
+
+async function setStatus(sha: string, state: State, description: string, targetUrl: string) {
+ await github('POST', `/repos/${repo()}/statuses/${sha}`, {
+ state,
+ context: context(),
+ description: description.slice(0, DESCRIPTION_LIMIT),
+ target_url: targetUrl,
+ });
+ console.log(`${context()} on ${sha.slice(0, 10)}: ${state}: ${description}`);
+}
+
+/// The newest gate status on a commit (GitHub lists statuses newest first).
+async function gateStatus(sha: string): Promise<{ state: string; description: string } | undefined> {
+ const statuses: { context: string; state: string; description: string | null }[] = await github(
+ 'GET',
+ `/repos/${repo()}/commits/${sha}/statuses?per_page=100`,
+ );
+ const status = statuses.find((each) => each.context === context());
+ return status && { state: status.state, description: status.description ?? '' };
+}
+
+/// Whether a trusted step opened a request on this commit that no result has
+/// closed yet.
+export function isOpenRequest(status: { state: string; description: string } | undefined): boolean {
+ return status?.state === 'pending' && status.description.startsWith(RUNNING);
+}
+
+async function removeLabel(pr: string) {
+ await github('DELETE', `/repos/${repo()}/issues/${pr}/labels/${encodeURIComponent(label())}`);
+}
+
+async function remind() {
+ const head = env('HEAD_SHA');
+ const pr = env('PR_NUMBER');
+
+ // A re-open or leaving draft keeps the head, so it may have a result already.
+ if (await gateStatus(head)) {
+ console.log(`${head.slice(0, 10)} already has a ${context()} status`);
+ return;
+ }
+
+ await setStatus(
+ head,
+ 'pending',
+ `Not run on this commit. Run: ${runCommand(repo(), pr, label())}`,
+ env('CI_GATE_DOCS_URL'),
+ );
+}
+
+async function start() {
+ const head = env('HEAD_SHA');
+ const pr = env('PR_NUMBER');
+ const runUrl = env('RUN_URL');
+ const sameRepo = env('SAME_REPO') === 'true';
+
+ // Fork tokens are read-only; the fork request step did both already.
+ if (sameRepo) {
+ await removeLabel(pr);
+ }
+
+ // A label added right after a push can carry the previous head, or a merge
+ // commit GitHub has not rebuilt yet. Either would test an old commit.
+ const pull = await github('GET', `/repos/${repo()}/pulls/${pr}`);
+ if (pull.head.sha !== head) {
+ const description = `The PR moved on from ${head.slice(0, 7)} to ${pull.head.sha.slice(0, 7)}. Run again: ${runCommand(repo(), pr, label())}`;
+ if (sameRepo) {
+ await setStatus(head, 'failure', description, runUrl);
+ }
+ throw new Error(description);
+ }
+
+ const merge = await github('GET', `/repos/${repo()}/commits/${env('MERGE_SHA')}`);
+ const parents: string[] = merge.parents.map((parent: { sha: string }) => parent.sha);
+
+ if (!parents.includes(head)) {
+ const description = `The merge commit did not include ${head.slice(0, 7)} yet. Run again: ${runCommand(repo(), pr, label())}`;
+ if (sameRepo) {
+ await setStatus(head, 'failure', description, runUrl);
+ }
+ throw new Error(description);
+ }
+
+ if (sameRepo) {
+ await setStatus(head, 'pending', `${RUNNING} ${head.slice(0, 7)}`, runUrl);
+ }
+}
+
+async function finish() {
+ const head = env('HEAD_SHA');
+ const outcome = gateOutcome(JSON.parse(env('NEEDS')), head);
+
+ if (!outcome) {
+ console.log('Start did not succeed; its status stands.');
+ process.exitCode = 1;
+ return;
+ }
+
+ if (env('SAME_REPO') === 'true') {
+ await setStatus(head, outcome.state, outcome.description, env('RUN_URL'));
+ }
+
+ if (outcome.state !== 'success') {
+ console.log(outcome.description);
+ process.exitCode = 1;
+ }
+}
+
+async function forkRequest() {
+ const head = env('HEAD_SHA');
+ const pr = env('PR_NUMBER');
+
+ await removeLabel(pr);
+ await setStatus(head, 'pending', `${RUNNING} ${head.slice(0, 7)}`, env('PR_CHECKS_URL'));
+}
+
+async function fork() {
+ const event = JSON.parse(readFileSync(env('GITHUB_EVENT_PATH'), 'utf8'));
+ const run = event.workflow_run;
+ const head: string = run.head_sha;
+
+ // The request step opened this on the head the label was added at. A run on
+ // any other commit, such as one the fork pushed with a rewritten workflow,
+ // or a run after the result is in, finds no open request.
+ if (!isOpenRequest(await gateStatus(head))) {
+ console.log(`${head.slice(0, 10)} has no open ${context()} request; nothing to report.`);
+ return;
+ }
+
+ const jobs: RunJob[] = [];
+ for (let page = 1; ; page++) {
+ const { jobs: batch } = await github(
+ 'GET',
+ `/repos/${repo()}/actions/runs/${run.id}/attempts/${run.run_attempt}/jobs?per_page=100&page=${page}`,
+ );
+ jobs.push(...batch);
+ if (batch.length < 100) {
+ break;
+ }
+ }
+
+ const outcome = forkOutcome(jobs, head, label());
+ if (outcome) {
+ await setStatus(head, outcome.state, outcome.description, run.html_url);
+ }
+}
+
+export async function main() {
+ const modes: Record<string, () => Promise<void>> = {
+ remind,
+ start,
+ finish,
+ 'fork-request': forkRequest,
+ fork,
+ };
+ const mode = modes[env('CI_GATE_MODE')];
+ if (!mode) {
+ throw new Error(`unknown CI_GATE_MODE ${process.env.CI_GATE_MODE}`);
+ }
+ await mode();
+}
diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/scripts/ios_release_preconditions.sh b/tooling/xtask/crates/xtask_workflows/src/workflows/scripts/ios_release_preconditions.sh
index 9729e1a953..44c6523bde 100644
--- a/tooling/xtask/crates/xtask_workflows/src/workflows/scripts/ios_release_preconditions.sh
+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/scripts/ios_release_preconditions.sh
@@ -34,8 +34,10 @@ fi
# Web App Status Check aggregates the gate, so it is red whenever the gate is;
# its other inputs (Typecheck, Test, Build, ...) are separate checks and stay.
-# (`gh pr checks` names are job names, not workflow names.)
-ours='^(iOS Native Check|Re-run the native check|Web App Status Check|iOS App Store Release|iOS Release Drift|Android Native Check)'
+# The CI gate status aggregates all of them, and stays pending on a head no one
+# has run PR CI on. (`gh pr checks` names are job names, not workflow names;
+# PR CI prefixes the ones it calls with `Web App / `.)
+ours='^((Web App / )?(iOS Native Check|Web App Status Check|Android Native Check)|Re-run the native check|iOS App Store Release|iOS Release Drift|CI gate$)'
deadline=$(( $(date +%s) + WAIT_MINUTES * 60 ))
while true; do
checks=$(gh pr checks "$PR_NUMBER" --repo "$GH_REPO" --json name,bucket \
diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/sdk_check.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/sdk_check.rs
index 6bf09fc914..69211fcd30 100644
--- a/tooling/xtask/crates/xtask_workflows/src/workflows/sdk_check.rs
+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/sdk_check.rs
@@ -1,36 +1,41 @@
-use gh_workflow::{Concurrency, Event, Expression, Job, PullRequest, Push, Run, Step, Workflow};
+use gh_workflow::{Event, Job, PullRequest, Push, Run, Step, Workflow, WorkflowCall};
+use xtask_paths::RepoGlob;
-use crate::workflows::steps;
+use crate::workflows::{steps, vars};
#[cfg(test)]
mod test;
+/// Files whose change runs the workflow, on a push and in `PR CI`.
+pub const PATHS: &[RepoGlob<'static>] = &[
+ xtask_paths::repo_glob!("packages/sdk/**"),
+ xtask_paths::repo_glob!(".github/workflows/sdk-check.yml"),
+];
+
/// Build the workflow. Whether `packages/sdk`'s generated layer is fresh is
/// checked by the web app workflow's `Generated Code Check`, which builds the
/// spec binaries once for every Rust-generated artifact.
pub fn sdk_check() -> Workflow {
+ let pull_request = PullRequest::default().add_branch("main");
+
Workflow::new("SDK Check")
.on(Event::default()
.pull_request(
- PullRequest::default()
- .add_branch("main")
- .add_path("packages/sdk/**")
- .add_path(".github/workflows/sdk-check.yml"),
+ PATHS
+ .iter()
+ .fold(pull_request, |trigger, path| trigger.add_path(*path)),
)
// `Release SDK` publishes from `main` without re-checking the
// package, so the same suite has to guard what lands there.
.push(
- Push::default()
- .add_branch("main")
- .add_path("packages/sdk/**")
- .add_path(".github/workflows/sdk-check.yml"),
- ))
- .concurrency(
- Concurrency::new(Expression::new(
- "${{ github.workflow }}-${{ github.ref }}-check",
- ))
- .cancel_in_progress(true),
- )
+ PATHS
+ .iter()
+ .fold(Push::default().add_branch("main"), |trigger, path| {
+ trigger.add_path(*path)
+ }),
+ )
+ .workflow_call(WorkflowCall::default()))
+ .concurrency(vars::concurrency("sdk-check"))
.add_job("check-package", check_package())
}
diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/steps.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/steps.rs
index 39e379a231..bb7b95ce21 100644
--- a/tooling/xtask/crates/xtask_workflows/src/workflows/steps.rs
+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/steps.rs
@@ -458,6 +458,15 @@ pub fn start_sccache_server() -> Step<Run> {
Step::new("Start sccache server").run("sccache --start-server")
}
+/// Run embedded TypeScript on the hosted image's Node, which strips types
+/// itself, so the step needs no checkout, build or dependency. The script
+/// exports `main`.
+pub fn node_script(source: &str) -> String {
+ format!(
+ "node --disable-warning=ExperimentalWarning --input-type=module-typescript <<'NODE_SCRIPT'\n{source}\nawait main();\nNODE_SCRIPT\n"
+ )
+}
+
/// `sccache --show-stats` at the end of a job (never fails the job).
pub fn show_sccache_stats() -> Step<Run> {
Step::new("show sccache stats")
@@ -465,13 +474,14 @@ pub fn show_sccache_stats() -> Step<Run> {
.if_condition(Expression::new("always()"))
}
-/// Base for jobs gated behind `path-check`: depends on it and runs only on
-/// non-draft PRs where the path filter matched. Shared by `check` and `test`.
+/// Base for jobs gated behind `path-check`: depends on it and runs only where
+/// the path filter matched, on non-draft PRs unless the run was requested with
+/// the `PR CI` label. Shared by `check` and `test`.
pub fn gated_job() -> Job {
Job::default()
.needs(vec!["path-check".to_string()])
.cond(Expression::new(
- "needs.path-check.outputs.should_run == 'true' && github.event.pull_request.draft == false",
+ "needs.path-check.outputs.should_run == 'true' && (github.event.pull_request.draft == false || github.event.action == 'labeled')",
))
}
diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/test_macrod_update.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/test_macrod_update.rs
index 600d8e480a..6b379d83df 100644
--- a/tooling/xtask/crates/xtask_workflows/src/workflows/test_macrod_update.rs
+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/test_macrod_update.rs
@@ -7,7 +7,8 @@
//! and macOS, and the launcher layout plus `install.ps1` on Windows. Nothing
//! is published. Runs on pull requests that touch macrod, and on demand.
-use gh_workflow::{Event, Job, Level, Permissions, PullRequest, Run, Step, Workflow};
+use gh_workflow::{Event, Job, Level, Permissions, PullRequest, Run, Step, Workflow, WorkflowCall};
+use xtask_paths::RepoGlob;
use crate::workflows::{runners, steps, vars};
@@ -18,8 +19,16 @@ const E2E: &str = "crates/coding_agent_worker/e2e";
const V1: &str = "v2000.1.1.1";
const V2: &str = "v2000.1.1.2";
+/// Files whose change runs the workflow, on a push and in `PR CI`.
+pub const PATHS: &[RepoGlob<'static>] = &[
+ xtask_paths::repo_glob!("crates/coding_agent_worker/**"),
+ xtask_paths::repo_glob!(".github/workflows/test_macrod_update.yml"),
+];
+
/// Build the workflow.
pub fn test_macrod_update() -> Workflow {
+ let pull_request = PullRequest::default().add_branch("main");
+
Workflow::new("Test macrod update")
.permissions(Permissions {
contents: Some(Level::Read),
@@ -27,14 +36,12 @@ pub fn test_macrod_update() -> Workflow {
})
.on(Event::default()
.pull_request(
- PullRequest::default()
- .add_branch("main")
- .add_path(xtask_paths::repo_glob!("crates/coding_agent_worker/**"))
- .add_path(xtask_paths::repo_glob!(
- ".github/workflows/test_macrod_update.yml"
- )),
+ PATHS
+ .iter()
+ .fold(pull_request, |trigger, path| trigger.add_path(*path)),
)
- .workflow_dispatch(gh_workflow::WorkflowDispatch::default()))
+ .workflow_dispatch(gh_workflow::WorkflowDispatch::default())
+ .workflow_call(WorkflowCall::default()))
.add_job("linux", linux_job())
.add_job("macos", macos_job())
.add_job("windows", windows_job())
diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/vars.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/vars.rs
index 7861be5867..caf4f80b2a 100644
--- a/tooling/xtask/crates/xtask_workflows/src/workflows/vars.rs
+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/vars.rs
@@ -117,8 +117,13 @@ pub fn with_global_env(workflow: Workflow) -> Workflow {
.add_env(("RUSTFLAGS", "-C link-arg=-fuse-ld=lld"))
}
-/// Cancel superseded runs of this workflow on the same git ref.
+/// Cancel superseded runs of this workflow on the same git ref. When `PR CI`
+/// calls the workflow (a `labeled` event), the group is that run's own:
+/// `PR CI` cancels its superseded runs itself, and a requested run must not
+/// cancel, or be cancelled by, the same checks running for a push.
pub fn concurrency(prefix: &str) -> Concurrency {
- Concurrency::new(Expression::new(format!("{prefix}-${{{{ github.ref }}}}")))
- .cancel_in_progress(true)
+ Concurrency::new(Expression::new(format!(
+ "{prefix}-${{{{ github.event.action == 'labeled' && github.run_id || github.ref }}}}"
+ )))
+ .cancel_in_progress(true)
}
diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/web_app_check_main.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/web_app_check_main.rs
index 12b630c25f..43cda84ad2 100644
--- a/tooling/xtask/crates/xtask_workflows/src/workflows/web_app_check_main.rs
+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/web_app_check_main.rs
@@ -2,8 +2,8 @@
//! Generated into `web-app-check-main.yml`.
use gh_workflow::{
- Concurrency, Event, Expression, Job, Level, Permissions, PullRequest, PullRequestType, Run,
- Step, Strategy, Use, Workflow,
+ Event, Expression, Job, Level, Permissions, PullRequest, PullRequestType, Run, Step, Strategy,
+ Use, Workflow, WorkflowCall,
};
use crate::workflows::{
@@ -36,20 +36,17 @@ fn mid_web_runner() -> String {
/// Build the workflow.
pub fn web_app_check_main() -> Workflow {
Workflow::new("Web App Pr Checks")
- .on(Event::default().pull_request(
- PullRequest::default()
- .add_branch("main")
- .add_type(PullRequestType::Opened)
- .add_type(PullRequestType::Synchronize)
- .add_type(PullRequestType::Reopened)
- .add_type(PullRequestType::ReadyForReview),
- ))
- .concurrency(
- Concurrency::new(Expression::new(
- "${{ github.workflow }}-${{ github.ref }}-check",
- ))
- .cancel_in_progress(true),
- )
+ .on(Event::default()
+ .pull_request(
+ PullRequest::default()
+ .add_branch("main")
+ .add_type(PullRequestType::Opened)
+ .add_type(PullRequestType::Synchronize)
+ .add_type(PullRequestType::Reopened)
+ .add_type(PullRequestType::ReadyForReview),
+ )
+ .workflow_call(WorkflowCall::default()))
+ .concurrency(vars::concurrency("web-app-pr-checks"))
.add_job("path-check", path_check())
.add_job(android_native_check::JOB_ID, android_native_check())
.add_job("typescript", typescript())