Macro is a unified workspace for teams: email, chat, docs, tasks, agents, calls, and CRM — @-linked together with shared AI memory.
TypeScript agentaiai-agentsall-in-onecrmcrm-systememaillinearmcpmessagingnotesnotionofficerustslackslack-alternativestartupstartup-toolstypescriptworkspace
ci: run PR checks on request with the run-ci label (#9887)
teoafc1e4e1a47a
diff --git a/.agents/skills/git/SKILL.md b/.agents/skills/git/SKILL.mdindex 7b5241d8a8..d880424bb2 100644--- a/.agents/skills/git/SKILL.md+++ b/.agents/skills/git/SKILL.md@@ -140,8 +140,10 @@ when available). Do not commit verification captures unless explicitly requested## Handing off a PROpen PRs ready for review, not as drafts; if one was created as a draft, mark it-ready. A PR whose checks are failing or still running is not done: watch CI and fix-failures before handing off.+ready. The `CI gate` passes only after a run requested with the `run-ci` label;+the [`run-ci` skill](../run-ci/SKILL.md) says when to request one. A PR whose+`CI gate` is failing, still running, or not run on the latest commit is not done:+wait on it and fix failures before handing off.## Updating a PRdiff --git a/.agents/skills/run-ci/SKILL.md b/.agents/skills/run-ci/SKILL.mdnew file mode 100644index 0000000000..31852631b1--- /dev/null+++ b/.agents/skills/run-ci/SKILL.md@@ -0,0 +1,89 @@+---+name: run-ci+description: Run CI on a pull request and wait for its result. Use at a checkpoint (a coherent, locally tested chunk), when you think a PR is done, before asking for review or merging, after fixing a CI failure, and whenever the `CI gate` status says CI has not run.+---++# Run CI on a pull request++## When to run CI++Run CI deliberately, after the local checks for what you changed pass+(`just check`, the affected tests):++- at a checkpoint: a coherent, locally tested chunk you want real CI on before+ building further;+- when you think the PR is done, and before asking for review or merging;+- after pushing the fix for a CI failure.++Not after every small push, not while you are still iterating, and never+instead of local checks: every run costs CI compute.++## How it works++CI runs when someone asks for it, not on every push. Adding the `run-ci` label+to a PR runs every check on its head commit (the `PR CI` workflow), and the+label disappears as the run starts. The result is the `CI gate` commit status on+that commit. The merge gate is `CI gate` passing on the PR's latest commit, so+every push needs a new run before the PR can merge (see the end of this page+for the switch-over).++## Run it++Push first, then ask for the run:++```sh+gh pr create --title "..." --body-file body.md --label run-ci # a new PR+gh api repos/macro-inc/macro/issues/<pr>/labels -f "labels[]=run-ci" # after a push+```++`gh pr edit <pr> --add-label run-ci` does the same on a recent gh; older+releases fail on a Projects (classic) error before adding the label.++- Adding the label again starts a fresh run. A run still going for the same PR+ is cancelled.+- Draft PRs run too.+- Other labels never start or cancel CI.+- A label added within seconds of a push can reach GitHub before the push+ does. `Start` then fails within seconds and `CI gate` says to add the label+ again. To avoid it, add the label once `gh pr view <pr> --json headRefOid`+ shows the commit you pushed.++## Wait for it++Use the `babysit-pr` skill's watcher: `pr-wait <pr> --until green`. It exits 1+with the command to run when CI has not run on the head, and exits 1 when+`CI gate` fails. Never poll with your own loop.++To read the state once, look at the `CI gate` row of `gh pr checks <pr>`:++| `CI gate` | Meaning | Do |+| --- | --- | --- |+| pending, `Not run on this commit. Run: ...` | No run on this head | Add the label |+| pending, `Running on <sha>` | A run is going | Wait |+| pass, `Passed on <sha>` | Every check passed or was path-skipped | Done |+| fail, `Failed: <jobs>` | Those called workflows failed | Open the linked run, fix, push, run again |+| fail, `The PR moved on ...` or `The merge commit did not include ...` | Labelled too soon after a push | Add the label again |+| missing | A fork PR, before its first run | A maintainer adds the label |++## Fix a failure++- A flaky job: re-run the failed jobs of the `PR CI` run+ (`gh run rerun <run-id> --failed`). `CI gate` re-runs with them and updates+ the status; no new label is needed.+- A real failure: fix it, push, and add the label again.+- The checks inside the run are named `<workflow> / <job>`, such as+ `Web App / Typecheck` or `Cloud Storage / test`.++## Fork PRs++A maintainer adds the label after reading the diff, workflow files included:+the run uses the fork's copies. The fork's token can't set statuses, so+`CI Gate for Forks` removes the label and sets `CI gate` to `Running on <sha>`+on the head the label went on, then copies the run's verdict onto that head.+Re-running a fork's failed jobs doesn't update `CI gate`; add the label again.++## While the switch is in progress++The checks still run on every push as well, and `main` still requires their+old status checks (`Web App Status Check`, `Cloud Storage Status Check`,+`Infra Status Check`) until its ruleset requires `CI gate`.diff --git a/.github/workflows/android-play-release.yml b/.github/workflows/android-play-release.ymlindex 4db7b475b2..0a143096ca 100644--- a/.github/workflows/android-play-release.yml+++ b/.github/workflows/android-play-release.yml@@ -204,8 +204,10 @@ jobs:# (`gh pr checks` names are job names, not workflow names, plus the contexts# of commit statuses, such as the android-play/* ones Android Play Publish- # and Android Release Watch set.)- ours='^(Android Native Check|Android Play Release|Re-run the native check|Web App Status Check|Android Release Drift|iOS Native Check|android-play/)'+ # and Android Release Watch set. PR CI prefixes the jobs it calls with+ # `Web App / `. The CI gate status aggregates every check, and stays pending+ # on a head no one has run PR CI on.)+ ours='^((Web App / )?(Android Native Check|Web App Status Check|iOS Native Check)|Android Play Release|Re-run the native check|Android Release Drift|android-play/|CI gate$)'deadline=$(( $(date +%s) + WAIT_MINUTES * 60 ))while true; dochecks=$(gh pr checks "$PR_NUMBER" --repo "$GH_REPO" --json name,bucket \diff --git a/.github/workflows/cargo_deny.yml b/.github/workflows/cargo_deny.ymlindex 9e8ef3fbd4..557468073c 100644--- a/.github/workflows/cargo_deny.yml+++ b/.github/workflows/cargo_deny.yml@@ -18,6 +18,7 @@ name: cargo deny check- Cargo.lock- deny.toml- '.github/workflows/cargo_deny.yml'+ workflow_call: {}jobs:cargo-deny:runs-on: namespace-profile-linux-small@@ -29,5 +30,5 @@ jobs:with:manifest-path: Cargo.tomlconcurrency:- group: cargo-deny-${{ github.ref }}+ group: cargo-deny-${{ github.event.action == 'labeled' && github.run_id || github.ref }}cancel-in-progress: truediff --git a/.github/workflows/cargo_workspace_dependency_check.yml b/.github/workflows/cargo_workspace_dependency_check.ymlindex 4da7b7f985..c5cb771eda 100644--- a/.github/workflows/cargo_workspace_dependency_check.yml+++ b/.github/workflows/cargo_workspace_dependency_check.yml@@ -22,6 +22,7 @@ name: cargo workspace dependency check- tooling/xtask/crates/xtask_deps/**- tooling/xtask/crates/xtask_graph/**- '.github/workspace-dep-closures.json'+ workflow_call: {}jobs:dependency-check:runs-on: namespace-profile-linux-rust-ci;overrides.cache-tag=sccache-ci@@ -147,5 +148,5 @@ jobs:if: always()uses: './.github/actions/teardown-nix'concurrency:- group: cargo-workspace-dependency-check-${{ github.ref }}+ group: cargo-workspace-dependency-check-${{ github.event.action == 'labeled' && github.run_id || github.ref }}cancel-in-progress: truediff --git a/.github/workflows/check_generated.yml b/.github/workflows/check_generated.ymlindex a459d2ac3e..d28ebbec19 100644--- a/.github/workflows/check_generated.yml+++ b/.github/workflows/check_generated.yml@@ -10,6 +10,7 @@ name: check generated workflows- crates/macro_event_topics/**- '.github/workflows/**'- '.github/kafka-cluster-topics.json'+ workflow_call: {}jobs:check-workflows:runs-on: namespace-profile-linux-smalldiff --git a/.github/workflows/check_node_modules_nix.yml b/.github/workflows/check_node_modules_nix.ymlindex c7e60a7a9e..a4ab812365 100644--- a/.github/workflows/check_node_modules_nix.yml+++ b/.github/workflows/check_node_modules_nix.yml@@ -20,6 +20,7 @@ name: check node_modules nix- flake.nix- flake.lock- '.github/workflows/check_node_modules_nix.yml'+ workflow_call: {}permissions:contents: readjobs:diff --git a/.github/workflows/check_tauri_lockfiles.yml b/.github/workflows/check_tauri_lockfiles.ymlindex b25dd233bc..44bc835b95 100644--- a/.github/workflows/check_tauri_lockfiles.yml+++ b/.github/workflows/check_tauri_lockfiles.yml@@ -13,6 +13,7 @@ name: check tauri lockfiles- flake.nix- flake.lock- '.github/workflows/check_tauri_lockfiles.yml'+ workflow_call: {}permissions:contents: readjobs:diff --git a/.github/workflows/ci-gate-fork.yml b/.github/workflows/ci-gate-fork.ymlnew file mode 100644index 0000000000..9ba9c9c3ea--- /dev/null+++ b/.github/workflows/ci-gate-fork.yml@@ -0,0 +1,605 @@+# DO NOT EDIT — regenerate with `cargo x workflows` (from the repository root).+# Source: tooling/xtask/crates/xtask_workflows/src/workflows/ci_gate_fork.rs+name: CI Gate for Forks+'on':+ pull_request_target:+ types:+ - labeled+ branches:+ - main+ workflow_run:+ types:+ - completed+ workflows:+ - PR CI+jobs:+ request:+ if: github.event_name == 'pull_request_target' && github.event.label.name == 'run-ci' && github.event.pull_request.head.repo.full_name != github.repository+ name: Take a fork's CI request+ runs-on: ubuntu-latest+ permissions:+ issues: write+ pull-requests: write+ statuses: write+ timeout-minutes: 5+ steps:+ - name: Open the CI gate request+ run: |+ node --disable-warning=ExperimentalWarning --input-type=module-typescript <<'NODE_SCRIPT'+ // Sets the commit status `main` requires before a merge: it is green only on a+ // commit that a requested `PR CI` run passed. `CI_GATE_MODE` picks the step:+ //+ // remind a new commit: pending "not run" with the command to run it, unless+ // the commit already has a gate status+ // start a requested run begins: remove the label, check the merge commit+ // holds the PR head, set pending+ // finish the run ends: success when every job passed or was path-skipped+ // fork-request the label goes on a fork PR: what start does with a write+ // token, from a trusted run of `main`'s workflow+ // fork after a fork PR's run, whose read-only token could not set the status+ //+ // Statuses go on the PR head commit, so a later push starts without one.++ import { readFileSync } from 'node:fs';++ const API = 'https://api.github.com';+ const DESCRIPTION_LIMIT = 140;+ // An open request. Only trusted runs set statuses: a fork's token can't.+ const RUNNING = 'Running on';+ // Job names in `PR CI`; the fork step reads them back from the finished run.+ export const START_JOB = 'Start';+ export const GATE_JOB = 'CI gate';++ type State = 'pending' | 'success' | 'failure' | 'error';+ export type Needs = Record<string, { result: string }>;+ export type RunJob = { name: string; conclusion: string | null };+ type Outcome = { state: State; description: string } | null;++ function env(name: string): string {+ const value = process.env[name];+ if (!value) {+ throw new Error(`${name} is not set`);+ }+ return value;+ }++ async function github(method: string, path: string, body?: unknown): Promise<any> {+ const response = await fetch(`${API}${path}`, {+ method,+ headers: {+ accept: 'application/vnd.github+json',+ authorization: `Bearer ${env('GH_TOKEN')}`,+ 'x-github-api-version': '2022-11-28',+ },+ body: body === undefined ? undefined : JSON.stringify(body),+ });+ // Removing a label that is already gone is the outcome we wanted.+ if (method === 'DELETE' && response.status === 404) {+ return null;+ }+ if (!response.ok) {+ throw new Error(`${method} ${path}: HTTP ${response.status} ${await response.text()}`);+ }+ return response.status === 204 ? null : response.json();+ }++ const repo = () => env('GITHUB_REPOSITORY');+ const context = () => env('CI_GATE_CONTEXT');+ const label = () => env('CI_GATE_LABEL');++ /// REST, because `gh pr edit` fails on older gh releases.+ export function runCommand(repository: string, pr: string, labelName: string): string {+ return `gh api repos/${repository}/issues/${pr}/labels -f "labels[]=${labelName}"`;+ }++ /// The finished run's verdict from `toJSON(needs)`. Start must have succeeded;+ /// a called workflow may be skipped, because its path filter did not match.+ /// Null when start failed: it already set a status that says why.+ export function gateOutcome(needs: Needs, head: string): Outcome {+ if (needs.start?.result !== 'success') {+ return null;+ }++ const failed = Object.entries(needs)+ .filter(([, need]) => need.result !== 'success' && need.result !== 'skipped')+ .map(([job]) => job);++ if (failed.length > 0) {+ return { state: 'failure', description: `Failed: ${failed.join(', ')}` };+ }++ return { state: 'success', description: `Passed on ${head.slice(0, 7)}` };+ }++ /// A fork run's verdict from its jobs. Null when the run was not a request+ /// (unrelated label) or did not finish (cancelled).+ export function forkOutcome(jobs: RunJob[], head: string, labelName: string): Outcome {+ const start = jobs.find((job) => job.name === START_JOB);+ const gate = jobs.find((job) => job.name === GATE_JOB);++ if (!start || start.conclusion === 'skipped' || start.conclusion === 'cancelled') {+ return null;+ }++ if (start.conclusion !== 'success') {+ return {+ state: 'failure',+ description: `Could not start on ${head.slice(0, 7)}. Add the ${labelName} label again.`,+ };+ }++ if (gate?.conclusion === 'success') {+ return { state: 'success', description: `Passed on ${head.slice(0, 7)}` };+ }++ if (gate?.conclusion === 'failure') {+ return { state: 'failure', description: 'Failed: see the PR CI run' };+ }++ return null;+ }++ async function setStatus(sha: string, state: State, description: string, targetUrl: string) {+ await github('POST', `/repos/${repo()}/statuses/${sha}`, {+ state,+ context: context(),+ description: description.slice(0, DESCRIPTION_LIMIT),+ target_url: targetUrl,+ });+ console.log(`${context()} on ${sha.slice(0, 10)}: ${state}: ${description}`);+ }++ /// The newest gate status on a commit (GitHub lists statuses newest first).+ async function gateStatus(sha: string): Promise<{ state: string; description: string } | undefined> {+ const statuses: { context: string; state: string; description: string | null }[] = await github(+ 'GET',+ `/repos/${repo()}/commits/${sha}/statuses?per_page=100`,+ );+ const status = statuses.find((each) => each.context === context());+ return status && { state: status.state, description: status.description ?? '' };+ }++ /// Whether a trusted step opened a request on this commit that no result has+ /// closed yet.+ export function isOpenRequest(status: { state: string; description: string } | undefined): boolean {+ return status?.state === 'pending' && status.description.startsWith(RUNNING);+ }++ async function removeLabel(pr: string) {+ await github('DELETE', `/repos/${repo()}/issues/${pr}/labels/${encodeURIComponent(label())}`);+ }++ async function remind() {+ const head = env('HEAD_SHA');+ const pr = env('PR_NUMBER');++ // A re-open or leaving draft keeps the head, so it may have a result already.+ if (await gateStatus(head)) {+ console.log(`${head.slice(0, 10)} already has a ${context()} status`);+ return;+ }++ await setStatus(+ head,+ 'pending',+ `Not run on this commit. Run: ${runCommand(repo(), pr, label())}`,+ env('CI_GATE_DOCS_URL'),+ );+ }++ async function start() {+ const head = env('HEAD_SHA');+ const pr = env('PR_NUMBER');+ const runUrl = env('RUN_URL');+ const sameRepo = env('SAME_REPO') === 'true';++ // Fork tokens are read-only; the fork request step did both already.+ if (sameRepo) {+ await removeLabel(pr);+ }++ // A label added right after a push can carry the previous head, or a merge+ // commit GitHub has not rebuilt yet. Either would test an old commit.+ const pull = await github('GET', `/repos/${repo()}/pulls/${pr}`);+ if (pull.head.sha !== head) {+ const description = `The PR moved on from ${head.slice(0, 7)} to ${pull.head.sha.slice(0, 7)}. Run again: ${runCommand(repo(), pr, label())}`;+ if (sameRepo) {+ await setStatus(head, 'failure', description, runUrl);+ }+ throw new Error(description);+ }++ const merge = await github('GET', `/repos/${repo()}/commits/${env('MERGE_SHA')}`);+ const parents: string[] = merge.parents.map((parent: { sha: string }) => parent.sha);++ if (!parents.includes(head)) {+ const description = `The merge commit did not include ${head.slice(0, 7)} yet. Run again: ${runCommand(repo(), pr, label())}`;+ if (sameRepo) {+ await setStatus(head, 'failure', description, runUrl);+ }+ throw new Error(description);+ }++ if (sameRepo) {+ await setStatus(head, 'pending', `${RUNNING} ${head.slice(0, 7)}`, runUrl);+ }+ }++ async function finish() {+ const head = env('HEAD_SHA');+ const outcome = gateOutcome(JSON.parse(env('NEEDS')), head);++ if (!outcome) {+ console.log('Start did not succeed; its status stands.');+ process.exitCode = 1;+ return;+ }++ if (env('SAME_REPO') === 'true') {+ await setStatus(head, outcome.state, outcome.description, env('RUN_URL'));+ }++ if (outcome.state !== 'success') {+ console.log(outcome.description);+ process.exitCode = 1;+ }+ }++ async function forkRequest() {+ const head = env('HEAD_SHA');+ const pr = env('PR_NUMBER');++ await removeLabel(pr);+ await setStatus(head, 'pending', `${RUNNING} ${head.slice(0, 7)}`, env('PR_CHECKS_URL'));+ }++ async function fork() {+ const event = JSON.parse(readFileSync(env('GITHUB_EVENT_PATH'), 'utf8'));+ const run = event.workflow_run;+ const head: string = run.head_sha;++ // The request step opened this on the head the label was added at. A run on+ // any other commit, such as one the fork pushed with a rewritten workflow,+ // or a run after the result is in, finds no open request.+ if (!isOpenRequest(await gateStatus(head))) {+ console.log(`${head.slice(0, 10)} has no open ${context()} request; nothing to report.`);+ return;+ }++ const jobs: RunJob[] = [];+ for (let page = 1; ; page++) {+ const { jobs: batch } = await github(+ 'GET',+ `/repos/${repo()}/actions/runs/${run.id}/attempts/${run.run_attempt}/jobs?per_page=100&page=${page}`,+ );+ jobs.push(...batch);+ if (batch.length < 100) {+ break;+ }+ }++ const outcome = forkOutcome(jobs, head, label());+ if (outcome) {+ await setStatus(head, outcome.state, outcome.description, run.html_url);+ }+ }++ export async function main() {+ const modes: Record<string, () => Promise<void>> = {+ remind,+ start,+ finish,+ 'fork-request': forkRequest,+ fork,+ };+ const mode = modes[env('CI_GATE_MODE')];+ if (!mode) {+ throw new Error(`unknown CI_GATE_MODE ${process.env.CI_GATE_MODE}`);+ }+ await mode();+ }++ await main();+ NODE_SCRIPT+ shell: bash+ env:+ CI_GATE_MODE: fork-request+ CI_GATE_CONTEXT: CI gate+ CI_GATE_LABEL: run-ci+ GH_TOKEN: ${{ github.token }}+ PR_NUMBER: ${{ github.event.pull_request.number }}+ HEAD_SHA: ${{ github.event.pull_request.head.sha }}+ PR_CHECKS_URL: ${{ github.event.pull_request.html_url }}/checks+ report:+ if: github.event_name == 'workflow_run' && github.event.workflow_run.event == 'pull_request' && github.event.workflow_run.head_repository.full_name != github.repository+ name: Report a fork's CI result+ runs-on: ubuntu-latest+ permissions:+ actions: read+ statuses: write+ timeout-minutes: 5+ steps:+ - name: Set the CI gate status+ run: |+ node --disable-warning=ExperimentalWarning --input-type=module-typescript <<'NODE_SCRIPT'+ // Sets the commit status `main` requires before a merge: it is green only on a+ // commit that a requested `PR CI` run passed. `CI_GATE_MODE` picks the step:+ //+ // remind a new commit: pending "not run" with the command to run it, unless+ // the commit already has a gate status+ // start a requested run begins: remove the label, check the merge commit+ // holds the PR head, set pending+ // finish the run ends: success when every job passed or was path-skipped+ // fork-request the label goes on a fork PR: what start does with a write+ // token, from a trusted run of `main`'s workflow+ // fork after a fork PR's run, whose read-only token could not set the status+ //+ // Statuses go on the PR head commit, so a later push starts without one.++ import { readFileSync } from 'node:fs';++ const API = 'https://api.github.com';+ const DESCRIPTION_LIMIT = 140;+ // An open request. Only trusted runs set statuses: a fork's token can't.+ const RUNNING = 'Running on';+ // Job names in `PR CI`; the fork step reads them back from the finished run.+ export const START_JOB = 'Start';+ export const GATE_JOB = 'CI gate';++ type State = 'pending' | 'success' | 'failure' | 'error';+ export type Needs = Record<string, { result: string }>;+ export type RunJob = { name: string; conclusion: string | null };+ type Outcome = { state: State; description: string } | null;++ function env(name: string): string {+ const value = process.env[name];+ if (!value) {+ throw new Error(`${name} is not set`);+ }+ return value;+ }++ async function github(method: string, path: string, body?: unknown): Promise<any> {+ const response = await fetch(`${API}${path}`, {+ method,+ headers: {+ accept: 'application/vnd.github+json',+ authorization: `Bearer ${env('GH_TOKEN')}`,+ 'x-github-api-version': '2022-11-28',+ },+ body: body === undefined ? undefined : JSON.stringify(body),+ });+ // Removing a label that is already gone is the outcome we wanted.+ if (method === 'DELETE' && response.status === 404) {+ return null;+ }+ if (!response.ok) {+ throw new Error(`${method} ${path}: HTTP ${response.status} ${await response.text()}`);+ }+ return response.status === 204 ? null : response.json();+ }++ const repo = () => env('GITHUB_REPOSITORY');+ const context = () => env('CI_GATE_CONTEXT');+ const label = () => env('CI_GATE_LABEL');++ /// REST, because `gh pr edit` fails on older gh releases.+ export function runCommand(repository: string, pr: string, labelName: string): string {+ return `gh api repos/${repository}/issues/${pr}/labels -f "labels[]=${labelName}"`;+ }++ /// The finished run's verdict from `toJSON(needs)`. Start must have succeeded;+ /// a called workflow may be skipped, because its path filter did not match.+ /// Null when start failed: it already set a status that says why.+ export function gateOutcome(needs: Needs, head: string): Outcome {+ if (needs.start?.result !== 'success') {+ return null;+ }++ const failed = Object.entries(needs)+ .filter(([, need]) => need.result !== 'success' && need.result !== 'skipped')+ .map(([job]) => job);++ if (failed.length > 0) {+ return { state: 'failure', description: `Failed: ${failed.join(', ')}` };+ }++ return { state: 'success', description: `Passed on ${head.slice(0, 7)}` };+ }++ /// A fork run's verdict from its jobs. Null when the run was not a request+ /// (unrelated label) or did not finish (cancelled).+ export function forkOutcome(jobs: RunJob[], head: string, labelName: string): Outcome {+ const start = jobs.find((job) => job.name === START_JOB);+ const gate = jobs.find((job) => job.name === GATE_JOB);++ if (!start || start.conclusion === 'skipped' || start.conclusion === 'cancelled') {+ return null;+ }++ if (start.conclusion !== 'success') {+ return {+ state: 'failure',+ description: `Could not start on ${head.slice(0, 7)}. Add the ${labelName} label again.`,+ };+ }++ if (gate?.conclusion === 'success') {+ return { state: 'success', description: `Passed on ${head.slice(0, 7)}` };+ }++ if (gate?.conclusion === 'failure') {+ return { state: 'failure', description: 'Failed: see the PR CI run' };+ }++ return null;+ }++ async function setStatus(sha: string, state: State, description: string, targetUrl: string) {+ await github('POST', `/repos/${repo()}/statuses/${sha}`, {+ state,+ context: context(),+ description: description.slice(0, DESCRIPTION_LIMIT),+ target_url: targetUrl,+ });+ console.log(`${context()} on ${sha.slice(0, 10)}: ${state}: ${description}`);+ }++ /// The newest gate status on a commit (GitHub lists statuses newest first).+ async function gateStatus(sha: string): Promise<{ state: string; description: string } | undefined> {+ const statuses: { context: string; state: string; description: string | null }[] = await github(+ 'GET',+ `/repos/${repo()}/commits/${sha}/statuses?per_page=100`,+ );+ const status = statuses.find((each) => each.context === context());+ return status && { state: status.state, description: status.description ?? '' };+ }++ /// Whether a trusted step opened a request on this commit that no result has+ /// closed yet.+ export function isOpenRequest(status: { state: string; description: string } | undefined): boolean {+ return status?.state === 'pending' && status.description.startsWith(RUNNING);+ }++ async function removeLabel(pr: string) {+ await github('DELETE', `/repos/${repo()}/issues/${pr}/labels/${encodeURIComponent(label())}`);+ }++ async function remind() {+ const head = env('HEAD_SHA');+ const pr = env('PR_NUMBER');++ // A re-open or leaving draft keeps the head, so it may have a result already.+ if (await gateStatus(head)) {+ console.log(`${head.slice(0, 10)} already has a ${context()} status`);+ return;+ }++ await setStatus(+ head,+ 'pending',+ `Not run on this commit. Run: ${runCommand(repo(), pr, label())}`,+ env('CI_GATE_DOCS_URL'),+ );+ }++ async function start() {+ const head = env('HEAD_SHA');+ const pr = env('PR_NUMBER');+ const runUrl = env('RUN_URL');+ const sameRepo = env('SAME_REPO') === 'true';++ // Fork tokens are read-only; the fork request step did both already.+ if (sameRepo) {+ await removeLabel(pr);+ }++ // A label added right after a push can carry the previous head, or a merge+ // commit GitHub has not rebuilt yet. Either would test an old commit.+ const pull = await github('GET', `/repos/${repo()}/pulls/${pr}`);+ if (pull.head.sha !== head) {+ const description = `The PR moved on from ${head.slice(0, 7)} to ${pull.head.sha.slice(0, 7)}. Run again: ${runCommand(repo(), pr, label())}`;+ if (sameRepo) {+ await setStatus(head, 'failure', description, runUrl);+ }+ throw new Error(description);+ }++ const merge = await github('GET', `/repos/${repo()}/commits/${env('MERGE_SHA')}`);+ const parents: string[] = merge.parents.map((parent: { sha: string }) => parent.sha);++ if (!parents.includes(head)) {+ const description = `The merge commit did not include ${head.slice(0, 7)} yet. Run again: ${runCommand(repo(), pr, label())}`;+ if (sameRepo) {+ await setStatus(head, 'failure', description, runUrl);+ }+ throw new Error(description);+ }++ if (sameRepo) {+ await setStatus(head, 'pending', `${RUNNING} ${head.slice(0, 7)}`, runUrl);+ }+ }++ async function finish() {+ const head = env('HEAD_SHA');+ const outcome = gateOutcome(JSON.parse(env('NEEDS')), head);++ if (!outcome) {+ console.log('Start did not succeed; its status stands.');+ process.exitCode = 1;+ return;+ }++ if (env('SAME_REPO') === 'true') {+ await setStatus(head, outcome.state, outcome.description, env('RUN_URL'));+ }++ if (outcome.state !== 'success') {+ console.log(outcome.description);+ process.exitCode = 1;+ }+ }++ async function forkRequest() {+ const head = env('HEAD_SHA');+ const pr = env('PR_NUMBER');++ await removeLabel(pr);+ await setStatus(head, 'pending', `${RUNNING} ${head.slice(0, 7)}`, env('PR_CHECKS_URL'));+ }++ async function fork() {+ const event = JSON.parse(readFileSync(env('GITHUB_EVENT_PATH'), 'utf8'));+ const run = event.workflow_run;+ const head: string = run.head_sha;++ // The request step opened this on the head the label was added at. A run on+ // any other commit, such as one the fork pushed with a rewritten workflow,+ // or a run after the result is in, finds no open request.+ if (!isOpenRequest(await gateStatus(head))) {+ console.log(`${head.slice(0, 10)} has no open ${context()} request; nothing to report.`);+ return;+ }++ const jobs: RunJob[] = [];+ for (let page = 1; ; page++) {+ const { jobs: batch } = await github(+ 'GET',+ `/repos/${repo()}/actions/runs/${run.id}/attempts/${run.run_attempt}/jobs?per_page=100&page=${page}`,+ );+ jobs.push(...batch);+ if (batch.length < 100) {+ break;+ }+ }++ const outcome = forkOutcome(jobs, head, label());+ if (outcome) {+ await setStatus(head, outcome.state, outcome.description, run.html_url);+ }+ }++ export async function main() {+ const modes: Record<string, () => Promise<void>> = {+ remind,+ start,+ finish,+ 'fork-request': forkRequest,+ fork,+ };+ const mode = modes[env('CI_GATE_MODE')];+ if (!mode) {+ throw new Error(`unknown CI_GATE_MODE ${process.env.CI_GATE_MODE}`);+ }+ await mode();+ }++ await main();+ NODE_SCRIPT+ shell: bash+ env:+ CI_GATE_MODE: fork+ CI_GATE_CONTEXT: CI gate+ CI_GATE_LABEL: run-ci+ GH_TOKEN: ${{ github.token }}diff --git a/.github/workflows/ci-gate-reminder.yml b/.github/workflows/ci-gate-reminder.ymlnew file mode 100644index 0000000000..913569bb22--- /dev/null+++ b/.github/workflows/ci-gate-reminder.yml@@ -0,0 +1,307 @@+# DO NOT EDIT — regenerate with `cargo x workflows` (from the repository root).+# Source: tooling/xtask/crates/xtask_workflows/src/workflows/ci_gate_reminder.rs+name: CI Gate Reminder+'on':+ pull_request:+ types:+ - opened+ - synchronize+ - reopened+ branches:+ - main+jobs:+ remind:+ if: github.event.pull_request.head.repo.full_name == github.repository+ name: CI not run yet+ runs-on: ubuntu-latest+ permissions:+ statuses: write+ timeout-minutes: 5+ steps:+ - name: Mark the commit as not run+ run: |+ node --disable-warning=ExperimentalWarning --input-type=module-typescript <<'NODE_SCRIPT'+ // Sets the commit status `main` requires before a merge: it is green only on a+ // commit that a requested `PR CI` run passed. `CI_GATE_MODE` picks the step:+ //+ // remind a new commit: pending "not run" with the command to run it, unless+ // the commit already has a gate status+ // start a requested run begins: remove the label, check the merge commit+ // holds the PR head, set pending+ // finish the run ends: success when every job passed or was path-skipped+ // fork-request the label goes on a fork PR: what start does with a write+ // token, from a trusted run of `main`'s workflow+ // fork after a fork PR's run, whose read-only token could not set the status+ //+ // Statuses go on the PR head commit, so a later push starts without one.++ import { readFileSync } from 'node:fs';++ const API = 'https://api.github.com';+ const DESCRIPTION_LIMIT = 140;+ // An open request. Only trusted runs set statuses: a fork's token can't.+ const RUNNING = 'Running on';+ // Job names in `PR CI`; the fork step reads them back from the finished run.+ export const START_JOB = 'Start';+ export const GATE_JOB = 'CI gate';++ type State = 'pending' | 'success' | 'failure' | 'error';+ export type Needs = Record<string, { result: string }>;+ export type RunJob = { name: string; conclusion: string | null };+ type Outcome = { state: State; description: string } | null;++ function env(name: string): string {+ const value = process.env[name];+ if (!value) {+ throw new Error(`${name} is not set`);+ }+ return value;+ }++ async function github(method: string, path: string, body?: unknown): Promise<any> {+ const response = await fetch(`${API}${path}`, {+ method,+ headers: {+ accept: 'application/vnd.github+json',+ authorization: `Bearer ${env('GH_TOKEN')}`,+ 'x-github-api-version': '2022-11-28',+ },+ body: body === undefined ? undefined : JSON.stringify(body),+ });+ // Removing a label that is already gone is the outcome we wanted.+ if (method === 'DELETE' && response.status === 404) {+ return null;+ }+ if (!response.ok) {+ throw new Error(`${method} ${path}: HTTP ${response.status} ${await response.text()}`);+ }+ return response.status === 204 ? null : response.json();+ }++ const repo = () => env('GITHUB_REPOSITORY');+ const context = () => env('CI_GATE_CONTEXT');+ const label = () => env('CI_GATE_LABEL');++ /// REST, because `gh pr edit` fails on older gh releases.+ export function runCommand(repository: string, pr: string, labelName: string): string {+ return `gh api repos/${repository}/issues/${pr}/labels -f "labels[]=${labelName}"`;+ }++ /// The finished run's verdict from `toJSON(needs)`. Start must have succeeded;+ /// a called workflow may be skipped, because its path filter did not match.+ /// Null when start failed: it already set a status that says why.+ export function gateOutcome(needs: Needs, head: string): Outcome {+ if (needs.start?.result !== 'success') {+ return null;+ }++ const failed = Object.entries(needs)+ .filter(([, need]) => need.result !== 'success' && need.result !== 'skipped')+ .map(([job]) => job);++ if (failed.length > 0) {+ return { state: 'failure', description: `Failed: ${failed.join(', ')}` };+ }++ return { state: 'success', description: `Passed on ${head.slice(0, 7)}` };+ }++ /// A fork run's verdict from its jobs. Null when the run was not a request+ /// (unrelated label) or did not finish (cancelled).+ export function forkOutcome(jobs: RunJob[], head: string, labelName: string): Outcome {+ const start = jobs.find((job) => job.name === START_JOB);+ const gate = jobs.find((job) => job.name === GATE_JOB);++ if (!start || start.conclusion === 'skipped' || start.conclusion === 'cancelled') {+ return null;+ }++ if (start.conclusion !== 'success') {+ return {+ state: 'failure',+ description: `Could not start on ${head.slice(0, 7)}. Add the ${labelName} label again.`,+ };+ }++ if (gate?.conclusion === 'success') {+ return { state: 'success', description: `Passed on ${head.slice(0, 7)}` };+ }++ if (gate?.conclusion === 'failure') {+ return { state: 'failure', description: 'Failed: see the PR CI run' };+ }++ return null;+ }++ async function setStatus(sha: string, state: State, description: string, targetUrl: string) {+ await github('POST', `/repos/${repo()}/statuses/${sha}`, {+ state,+ context: context(),+ description: description.slice(0, DESCRIPTION_LIMIT),+ target_url: targetUrl,+ });+ console.log(`${context()} on ${sha.slice(0, 10)}: ${state}: ${description}`);+ }++ /// The newest gate status on a commit (GitHub lists statuses newest first).+ async function gateStatus(sha: string): Promise<{ state: string; description: string } | undefined> {+ const statuses: { context: string; state: string; description: string | null }[] = await github(+ 'GET',+ `/repos/${repo()}/commits/${sha}/statuses?per_page=100`,+ );+ const status = statuses.find((each) => each.context === context());+ return status && { state: status.state, description: status.description ?? '' };+ }++ /// Whether a trusted step opened a request on this commit that no result has+ /// closed yet.+ export function isOpenRequest(status: { state: string; description: string } | undefined): boolean {+ return status?.state === 'pending' && status.description.startsWith(RUNNING);+ }++ async function removeLabel(pr: string) {+ await github('DELETE', `/repos/${repo()}/issues/${pr}/labels/${encodeURIComponent(label())}`);+ }++ async function remind() {+ const head = env('HEAD_SHA');+ const pr = env('PR_NUMBER');++ // A re-open or leaving draft keeps the head, so it may have a result already.+ if (await gateStatus(head)) {+ console.log(`${head.slice(0, 10)} already has a ${context()} status`);+ return;+ }++ await setStatus(+ head,+ 'pending',+ `Not run on this commit. Run: ${runCommand(repo(), pr, label())}`,+ env('CI_GATE_DOCS_URL'),+ );+ }++ async function start() {+ const head = env('HEAD_SHA');+ const pr = env('PR_NUMBER');+ const runUrl = env('RUN_URL');+ const sameRepo = env('SAME_REPO') === 'true';++ // Fork tokens are read-only; the fork request step did both already.+ if (sameRepo) {+ await removeLabel(pr);+ }++ // A label added right after a push can carry the previous head, or a merge+ // commit GitHub has not rebuilt yet. Either would test an old commit.+ const pull = await github('GET', `/repos/${repo()}/pulls/${pr}`);+ if (pull.head.sha !== head) {+ const description = `The PR moved on from ${head.slice(0, 7)} to ${pull.head.sha.slice(0, 7)}. Run again: ${runCommand(repo(), pr, label())}`;+ if (sameRepo) {+ await setStatus(head, 'failure', description, runUrl);+ }+ throw new Error(description);+ }++ const merge = await github('GET', `/repos/${repo()}/commits/${env('MERGE_SHA')}`);+ const parents: string[] = merge.parents.map((parent: { sha: string }) => parent.sha);++ if (!parents.includes(head)) {+ const description = `The merge commit did not include ${head.slice(0, 7)} yet. Run again: ${runCommand(repo(), pr, label())}`;+ if (sameRepo) {+ await setStatus(head, 'failure', description, runUrl);+ }+ throw new Error(description);+ }++ if (sameRepo) {+ await setStatus(head, 'pending', `${RUNNING} ${head.slice(0, 7)}`, runUrl);+ }+ }++ async function finish() {+ const head = env('HEAD_SHA');+ const outcome = gateOutcome(JSON.parse(env('NEEDS')), head);++ if (!outcome) {+ console.log('Start did not succeed; its status stands.');+ process.exitCode = 1;+ return;+ }++ if (env('SAME_REPO') === 'true') {+ await setStatus(head, outcome.state, outcome.description, env('RUN_URL'));+ }++ if (outcome.state !== 'success') {+ console.log(outcome.description);+ process.exitCode = 1;+ }+ }++ async function forkRequest() {+ const head = env('HEAD_SHA');+ const pr = env('PR_NUMBER');++ await removeLabel(pr);+ await setStatus(head, 'pending', `${RUNNING} ${head.slice(0, 7)}`, env('PR_CHECKS_URL'));+ }++ async function fork() {+ const event = JSON.parse(readFileSync(env('GITHUB_EVENT_PATH'), 'utf8'));+ const run = event.workflow_run;+ const head: string = run.head_sha;++ // The request step opened this on the head the label was added at. A run on+ // any other commit, such as one the fork pushed with a rewritten workflow,+ // or a run after the result is in, finds no open request.+ if (!isOpenRequest(await gateStatus(head))) {+ console.log(`${head.slice(0, 10)} has no open ${context()} request; nothing to report.`);+ return;+ }++ const jobs: RunJob[] = [];+ for (let page = 1; ; page++) {+ const { jobs: batch } = await github(+ 'GET',+ `/repos/${repo()}/actions/runs/${run.id}/attempts/${run.run_attempt}/jobs?per_page=100&page=${page}`,+ );+ jobs.push(...batch);+ if (batch.length < 100) {+ break;+ }+ }++ const outcome = forkOutcome(jobs, head, label());+ if (outcome) {+ await setStatus(head, outcome.state, outcome.description, run.html_url);+ }+ }++ export async function main() {+ const modes: Record<string, () => Promise<void>> = {+ remind,+ start,+ finish,+ 'fork-request': forkRequest,+ fork,+ };+ const mode = modes[env('CI_GATE_MODE')];+ if (!mode) {+ throw new Error(`unknown CI_GATE_MODE ${process.env.CI_GATE_MODE}`);+ }+ await mode();+ }++ await main();+ NODE_SCRIPT+ shell: bash+ env:+ CI_GATE_MODE: remind+ CI_GATE_CONTEXT: CI gate+ CI_GATE_LABEL: run-ci+ GH_TOKEN: ${{ github.token }}+ PR_NUMBER: ${{ github.event.pull_request.number }}+ HEAD_SHA: ${{ github.event.pull_request.head.sha }}+ CI_GATE_DOCS_URL: https://github.com/macro-inc/macro/blob/main/.agents/skills/run-ci/SKILL.mddiff --git a/.github/workflows/code_check_cloud_storage.yml b/.github/workflows/code_check_cloud_storage.ymlindex 314cdaa02d..e79fab801a 100644--- a/.github/workflows/code_check_cloud_storage.yml+++ b/.github/workflows/code_check_cloud_storage.yml@@ -17,6 +17,7 @@ env:- ready_for_reviewbranches:- main+ workflow_call: {}jobs:path-check:runs-on: namespace-profile-linux-small;overrides.cache-tag=sccache-ci@@ -322,7 +323,7 @@ jobs:check:needs:- path-check- if: needs.path-check.outputs.should_run == 'true' && github.event.pull_request.draft == false+ if: needs.path-check.outputs.should_run == 'true' && (github.event.pull_request.draft == false || github.event.action == 'labeled')runs-on: namespace-profile-linux-large-build;overrides.cache-tag=sccache-cienv:RUSTFLAGS: '-Dwarnings -Dclippy::disallowed_methods -C link-arg=-fuse-ld=mold'@@ -522,7 +523,7 @@ jobs:doppler-config:needs:- path-check- if: needs.path-check.outputs.should_run == 'true' && github.event.pull_request.draft == false && needs.path-check.outputs.doppler_candidates == 'true'+ if: needs.path-check.outputs.should_run == 'true' && (github.event.pull_request.draft == false || github.event.action == 'labeled') && needs.path-check.outputs.doppler_candidates == 'true'runs-on: namespace-profile-linux-rust-ci;overrides.cache-tag=sccache-cienv:RUSTFLAGS: '-Dwarnings -Dclippy::disallowed_methods -C link-arg=-fuse-ld=mold'@@ -753,7 +754,7 @@ jobs:test:needs:- path-check- if: needs.path-check.outputs.should_run == 'true' && github.event.pull_request.draft == false && needs.path-check.outputs.rust_packages != 'none' && needs.path-check.outputs.skip_tests != 'true'+ if: needs.path-check.outputs.should_run == 'true' && (github.event.pull_request.draft == false || github.event.action == 'labeled') && needs.path-check.outputs.rust_packages != 'none' && needs.path-check.outputs.skip_tests != 'true'runs-on: namespace-profile-linux-large-build;overrides.cache-tag=sccache-cienv:RUST_PACKAGES: ${{ needs.path-check.outputs.rust_packages }}@@ -1052,7 +1053,7 @@ jobs:build-service-bins:needs:- path-check- if: needs.path-check.outputs.should_run == 'true' && github.event.pull_request.draft == false && needs.path-check.outputs.skip_tests != 'true' && (needs.path-check.outputs.rust_packages == 'all' || contains(needs.path-check.outputs.rust_packages, 'document_storage_service') || contains(needs.path-check.outputs.rust_packages, 'email_service') || contains(needs.path-check.outputs.rust_packages, 'authentication_service'))+ if: needs.path-check.outputs.should_run == 'true' && (github.event.pull_request.draft == false || github.event.action == 'labeled') && needs.path-check.outputs.skip_tests != 'true' && (needs.path-check.outputs.rust_packages == 'all' || contains(needs.path-check.outputs.rust_packages, 'document_storage_service') || contains(needs.path-check.outputs.rust_packages, 'email_service') || contains(needs.path-check.outputs.rust_packages, 'authentication_service'))runs-on: namespace-profile-linux-large-build;overrides.cache-tag=sccache-cienv:RUST_PACKAGES: ${{ needs.path-check.outputs.rust_packages }}@@ -1226,7 +1227,7 @@ jobs:graphql-schema-compat:needs:- path-check- if: needs.path-check.outputs.should_run == 'true' && github.event.pull_request.draft == false && needs.path-check.outputs.graphql_schema == 'true'+ if: needs.path-check.outputs.should_run == 'true' && (github.event.pull_request.draft == false || github.event.action == 'labeled') && needs.path-check.outputs.graphql_schema == 'true'runs-on: namespace-profile-linux-smalltimeout-minutes: 15steps:@@ -1282,5 +1283,5 @@ jobs:echo "✅ All jobs passed or were skipped"concurrency:- group: code-check-cloud-storage-${{ github.ref }}+ group: code-check-cloud-storage-${{ github.event.action == 'labeled' && github.run_id || github.ref }}cancel-in-progress: truediff --git a/.github/workflows/code_check_conventions.yml b/.github/workflows/code_check_conventions.ymlindex e86007b67a..d8290548de 100644--- a/.github/workflows/code_check_conventions.yml+++ b/.github/workflows/code_check_conventions.yml@@ -10,6 +10,7 @@ name: conventions code check- ready_for_reviewbranches:- main+ workflow_call: {}permissions:contents: readjobs:@@ -41,7 +42,7 @@ jobs:ast-grep:needs:- path-check- if: needs.path-check.outputs.should_run == 'true' && github.event.pull_request.draft == false+ if: needs.path-check.outputs.should_run == 'true' && (github.event.pull_request.draft == false || github.event.action == 'labeled')name: ast-grep Conventionsruns-on: namespace-profile-linux-small;overrides.cache-tag=web-cisteps:@@ -89,5 +90,5 @@ jobs:echo "✅ All jobs passed or were skipped"concurrency:- group: code-check-conventions-${{ github.ref }}+ group: code-check-conventions-${{ github.event.action == 'labeled' && github.run_id || github.ref }}cancel-in-progress: truediff --git a/.github/workflows/code_check_infra.yml b/.github/workflows/code_check_infra.ymlindex 653ffda64c..a9e64f4373 100644--- a/.github/workflows/code_check_infra.yml+++ b/.github/workflows/code_check_infra.yml@@ -10,6 +10,7 @@ name: infra code check- ready_for_reviewbranches:- main+ workflow_call: {}jobs:path-check:runs-on: namespace-profile-linux-small@@ -33,7 +34,7 @@ jobs:biome-check:needs:- path-check- if: needs.path-check.outputs.should_run == 'true' && github.event.pull_request.draft == false+ if: needs.path-check.outputs.should_run == 'true' && (github.event.pull_request.draft == false || github.event.action == 'labeled')name: Biome Checkruns-on: namespace-profile-linux-small;overrides.cache-tag=web-cisteps:@@ -59,7 +60,7 @@ jobs:check:needs:- path-check- if: needs.path-check.outputs.should_run == 'true' && github.event.pull_request.draft == false+ if: needs.path-check.outputs.should_run == 'true' && (github.event.pull_request.draft == false || github.event.action == 'labeled')runs-on: namespace-profile-linux-small;overrides.cache-tag=web-cisteps:- name: Checkout Repo@@ -108,5 +109,5 @@ jobs:echo "✅ All jobs passed or were skipped"concurrency:- group: code-check-infra-${{ github.ref }}+ group: code-check-infra-${{ github.event.action == 'labeled' && github.run_id || github.ref }}cancel-in-progress: truediff --git a/.github/workflows/export-ci-telemetry.yml b/.github/workflows/export-ci-telemetry.ymlindex 2bfc508a06..7de9291028 100644--- a/.github/workflows/export-ci-telemetry.yml+++ b/.github/workflows/export-ci-telemetry.yml@@ -16,6 +16,8 @@ name: Export CI telemetry- Build Rust CI Image- Build Web App- Build macOS DMG+ - CI Gate Reminder+ - CI Gate for Forks- CLA- Cancel Stuck Cloud Storage Deploys- Cleanup Preview@@ -35,6 +37,7 @@ name: Export CI telemetry- Export Datadog observability definitions- Migrate Macro Database- Native Check Label+ - PR CI- PR Privacy Check- Publish Android to Google Play- Publish SDK@@ -115,7 +118,7 @@ jobs:- name: Collect push, job and step eventsif: always()run: |- node --disable-warning=ExperimentalWarning --input-type=module-typescript <<'TELEMETRY_SCRIPT'+ node --disable-warning=ExperimentalWarning --input-type=module-typescript <<'NODE_SCRIPT'// Turn one completed workflow run into CI telemetry events, as an OTLP logs file.//// Runs in the privileged exporter without the ingestion token. Everything here is@@ -709,7 +712,7 @@ jobs:}await main();- TELEMETRY_SCRIPT+ NODE_SCRIPTshell: bashenv:GITHUB_TOKEN: ${{ github.token }}@@ -742,7 +745,7 @@ jobs:- name: Send push, job and step eventsif: always()run: |- node --disable-warning=ExperimentalWarning --input-type=module-typescript <<'TELEMETRY_SCRIPT'+ node --disable-warning=ExperimentalWarning --input-type=module-typescript <<'NODE_SCRIPT'// Send the collected CI telemetry to the OTLP logs route. Best effort, never fails the job.import { existsSync, readFileSync } from 'node:fs';@@ -817,7 +820,7 @@ jobs:}await main();- TELEMETRY_SCRIPT+ NODE_SCRIPTshell: bashenv:CI_TELEMETRY_EVENTS: ${{ runner.temp }}/ci-telemetry-events.jsondiff --git a/.github/workflows/ios-app-store-release.yml b/.github/workflows/ios-app-store-release.ymlindex d89dc60105..c435e0c0f9 100644--- a/.github/workflows/ios-app-store-release.yml+++ b/.github/workflows/ios-app-store-release.yml@@ -253,8 +253,10 @@ jobs:# Web App Status Check aggregates the gate, so it is red whenever the gate is;# its other inputs (Typecheck, Test, Build, ...) are separate checks and stay.- # (`gh pr checks` names are job names, not workflow names.)- ours='^(iOS Native Check|Re-run the native check|Web App Status Check|iOS App Store Release|iOS Release Drift|Android Native Check)'+ # The CI gate status aggregates all of them, and stays pending on a head no one+ # has run PR CI on. (`gh pr checks` names are job names, not workflow names;+ # PR CI prefixes the ones it calls with `Web App / `.)+ ours='^((Web App / )?(iOS Native Check|Web App Status Check|Android Native Check)|Re-run the native check|iOS App Store Release|iOS Release Drift|CI gate$)'deadline=$(( $(date +%s) + WAIT_MINUTES * 60 ))while true; dochecks=$(gh pr checks "$PR_NUMBER" --repo "$GH_REPO" --json name,bucket \diff --git a/.github/workflows/pr-ci.yml b/.github/workflows/pr-ci.ymlnew file mode 100644index 0000000000..3005e39842--- /dev/null+++ b/.github/workflows/pr-ci.yml@@ -0,0 +1,763 @@+# DO NOT EDIT — regenerate with `cargo x workflows` (from the repository root).+# Source: tooling/xtask/crates/xtask_workflows/src/workflows/pr_ci.rs+name: PR CI+on:+ pull_request:+ types:+ - labeled+ branches:+ - main+jobs:+ start:+ if: github.event.label.name == 'run-ci'+ name: Start+ runs-on: ubuntu-latest+ permissions:+ contents: read+ issues: write+ pull-requests: write+ statuses: write+ outputs:+ cargo_deny: ${{ steps.filter.outputs.cargo_deny }}+ workspace_dependencies: ${{ steps.filter.outputs.workspace_dependencies }}+ generated_workflows: ${{ steps.filter.outputs.generated_workflows }}+ node_modules_nix: ${{ steps.filter.outputs.node_modules_nix }}+ tauri_lockfiles: ${{ steps.filter.outputs.tauri_lockfiles }}+ sdk: ${{ steps.filter.outputs.sdk }}+ macrod_update: ${{ steps.filter.outputs.macrod_update }}+ timeout-minutes: 5+ steps:+ - name: Take the request+ run: |+ node --disable-warning=ExperimentalWarning --input-type=module-typescript <<'NODE_SCRIPT'+ // Sets the commit status `main` requires before a merge: it is green only on a+ // commit that a requested `PR CI` run passed. `CI_GATE_MODE` picks the step:+ //+ // remind a new commit: pending "not run" with the command to run it, unless+ // the commit already has a gate status+ // start a requested run begins: remove the label, check the merge commit+ // holds the PR head, set pending+ // finish the run ends: success when every job passed or was path-skipped+ // fork-request the label goes on a fork PR: what start does with a write+ // token, from a trusted run of `main`'s workflow+ // fork after a fork PR's run, whose read-only token could not set the status+ //+ // Statuses go on the PR head commit, so a later push starts without one.++ import { readFileSync } from 'node:fs';++ const API = 'https://api.github.com';+ const DESCRIPTION_LIMIT = 140;+ // An open request. Only trusted runs set statuses: a fork's token can't.+ const RUNNING = 'Running on';+ // Job names in `PR CI`; the fork step reads them back from the finished run.+ export const START_JOB = 'Start';+ export const GATE_JOB = 'CI gate';++ type State = 'pending' | 'success' | 'failure' | 'error';+ export type Needs = Record<string, { result: string }>;+ export type RunJob = { name: string; conclusion: string | null };+ type Outcome = { state: State; description: string } | null;++ function env(name: string): string {+ const value = process.env[name];+ if (!value) {+ throw new Error(`${name} is not set`);+ }+ return value;+ }++ async function github(method: string, path: string, body?: unknown): Promise<any> {+ const response = await fetch(`${API}${path}`, {+ method,+ headers: {+ accept: 'application/vnd.github+json',+ authorization: `Bearer ${env('GH_TOKEN')}`,+ 'x-github-api-version': '2022-11-28',+ },+ body: body === undefined ? undefined : JSON.stringify(body),+ });+ // Removing a label that is already gone is the outcome we wanted.+ if (method === 'DELETE' && response.status === 404) {+ return null;+ }+ if (!response.ok) {+ throw new Error(`${method} ${path}: HTTP ${response.status} ${await response.text()}`);+ }+ return response.status === 204 ? null : response.json();+ }++ const repo = () => env('GITHUB_REPOSITORY');+ const context = () => env('CI_GATE_CONTEXT');+ const label = () => env('CI_GATE_LABEL');++ /// REST, because `gh pr edit` fails on older gh releases.+ export function runCommand(repository: string, pr: string, labelName: string): string {+ return `gh api repos/${repository}/issues/${pr}/labels -f "labels[]=${labelName}"`;+ }++ /// The finished run's verdict from `toJSON(needs)`. Start must have succeeded;+ /// a called workflow may be skipped, because its path filter did not match.+ /// Null when start failed: it already set a status that says why.+ export function gateOutcome(needs: Needs, head: string): Outcome {+ if (needs.start?.result !== 'success') {+ return null;+ }++ const failed = Object.entries(needs)+ .filter(([, need]) => need.result !== 'success' && need.result !== 'skipped')+ .map(([job]) => job);++ if (failed.length > 0) {+ return { state: 'failure', description: `Failed: ${failed.join(', ')}` };+ }++ return { state: 'success', description: `Passed on ${head.slice(0, 7)}` };+ }++ /// A fork run's verdict from its jobs. Null when the run was not a request+ /// (unrelated label) or did not finish (cancelled).+ export function forkOutcome(jobs: RunJob[], head: string, labelName: string): Outcome {+ const start = jobs.find((job) => job.name === START_JOB);+ const gate = jobs.find((job) => job.name === GATE_JOB);++ if (!start || start.conclusion === 'skipped' || start.conclusion === 'cancelled') {+ return null;+ }++ if (start.conclusion !== 'success') {+ return {+ state: 'failure',+ description: `Could not start on ${head.slice(0, 7)}. Add the ${labelName} label again.`,+ };+ }++ if (gate?.conclusion === 'success') {+ return { state: 'success', description: `Passed on ${head.slice(0, 7)}` };+ }++ if (gate?.conclusion === 'failure') {+ return { state: 'failure', description: 'Failed: see the PR CI run' };+ }++ return null;+ }++ async function setStatus(sha: string, state: State, description: string, targetUrl: string) {+ await github('POST', `/repos/${repo()}/statuses/${sha}`, {+ state,+ context: context(),+ description: description.slice(0, DESCRIPTION_LIMIT),+ target_url: targetUrl,+ });+ console.log(`${context()} on ${sha.slice(0, 10)}: ${state}: ${description}`);+ }++ /// The newest gate status on a commit (GitHub lists statuses newest first).+ async function gateStatus(sha: string): Promise<{ state: string; description: string } | undefined> {+ const statuses: { context: string; state: string; description: string | null }[] = await github(+ 'GET',+ `/repos/${repo()}/commits/${sha}/statuses?per_page=100`,+ );+ const status = statuses.find((each) => each.context === context());+ return status && { state: status.state, description: status.description ?? '' };+ }++ /// Whether a trusted step opened a request on this commit that no result has+ /// closed yet.+ export function isOpenRequest(status: { state: string; description: string } | undefined): boolean {+ return status?.state === 'pending' && status.description.startsWith(RUNNING);+ }++ async function removeLabel(pr: string) {+ await github('DELETE', `/repos/${repo()}/issues/${pr}/labels/${encodeURIComponent(label())}`);+ }++ async function remind() {+ const head = env('HEAD_SHA');+ const pr = env('PR_NUMBER');++ // A re-open or leaving draft keeps the head, so it may have a result already.+ if (await gateStatus(head)) {+ console.log(`${head.slice(0, 10)} already has a ${context()} status`);+ return;+ }++ await setStatus(+ head,+ 'pending',+ `Not run on this commit. Run: ${runCommand(repo(), pr, label())}`,+ env('CI_GATE_DOCS_URL'),+ );+ }++ async function start() {+ const head = env('HEAD_SHA');+ const pr = env('PR_NUMBER');+ const runUrl = env('RUN_URL');+ const sameRepo = env('SAME_REPO') === 'true';++ // Fork tokens are read-only; the fork request step did both already.+ if (sameRepo) {+ await removeLabel(pr);+ }++ // A label added right after a push can carry the previous head, or a merge+ // commit GitHub has not rebuilt yet. Either would test an old commit.+ const pull = await github('GET', `/repos/${repo()}/pulls/${pr}`);+ if (pull.head.sha !== head) {+ const description = `The PR moved on from ${head.slice(0, 7)} to ${pull.head.sha.slice(0, 7)}. Run again: ${runCommand(repo(), pr, label())}`;+ if (sameRepo) {+ await setStatus(head, 'failure', description, runUrl);+ }+ throw new Error(description);+ }++ const merge = await github('GET', `/repos/${repo()}/commits/${env('MERGE_SHA')}`);+ const parents: string[] = merge.parents.map((parent: { sha: string }) => parent.sha);++ if (!parents.includes(head)) {+ const description = `The merge commit did not include ${head.slice(0, 7)} yet. Run again: ${runCommand(repo(), pr, label())}`;+ if (sameRepo) {+ await setStatus(head, 'failure', description, runUrl);+ }+ throw new Error(description);+ }++ if (sameRepo) {+ await setStatus(head, 'pending', `${RUNNING} ${head.slice(0, 7)}`, runUrl);+ }+ }++ async function finish() {+ const head = env('HEAD_SHA');+ const outcome = gateOutcome(JSON.parse(env('NEEDS')), head);++ if (!outcome) {+ console.log('Start did not succeed; its status stands.');+ process.exitCode = 1;+ return;+ }++ if (env('SAME_REPO') === 'true') {+ await setStatus(head, outcome.state, outcome.description, env('RUN_URL'));+ }++ if (outcome.state !== 'success') {+ console.log(outcome.description);+ process.exitCode = 1;+ }+ }++ async function forkRequest() {+ const head = env('HEAD_SHA');+ const pr = env('PR_NUMBER');++ await removeLabel(pr);+ await setStatus(head, 'pending', `${RUNNING} ${head.slice(0, 7)}`, env('PR_CHECKS_URL'));+ }++ async function fork() {+ const event = JSON.parse(readFileSync(env('GITHUB_EVENT_PATH'), 'utf8'));+ const run = event.workflow_run;+ const head: string = run.head_sha;++ // The request step opened this on the head the label was added at. A run on+ // any other commit, such as one the fork pushed with a rewritten workflow,+ // or a run after the result is in, finds no open request.+ if (!isOpenRequest(await gateStatus(head))) {+ console.log(`${head.slice(0, 10)} has no open ${context()} request; nothing to report.`);+ return;+ }++ const jobs: RunJob[] = [];+ for (let page = 1; ; page++) {+ const { jobs: batch } = await github(+ 'GET',+ `/repos/${repo()}/actions/runs/${run.id}/attempts/${run.run_attempt}/jobs?per_page=100&page=${page}`,+ );+ jobs.push(...batch);+ if (batch.length < 100) {+ break;+ }+ }++ const outcome = forkOutcome(jobs, head, label());+ if (outcome) {+ await setStatus(head, outcome.state, outcome.description, run.html_url);+ }+ }++ export async function main() {+ const modes: Record<string, () => Promise<void>> = {+ remind,+ start,+ finish,+ 'fork-request': forkRequest,+ fork,+ };+ const mode = modes[env('CI_GATE_MODE')];+ if (!mode) {+ throw new Error(`unknown CI_GATE_MODE ${process.env.CI_GATE_MODE}`);+ }+ await mode();+ }++ await main();+ NODE_SCRIPT+ shell: bash+ env:+ CI_GATE_MODE: start+ CI_GATE_CONTEXT: CI gate+ CI_GATE_LABEL: run-ci+ GH_TOKEN: ${{ github.token }}+ PR_NUMBER: ${{ github.event.pull_request.number }}+ HEAD_SHA: ${{ github.event.pull_request.head.sha }}+ SAME_REPO: ${{ github.event.pull_request.head.repo.full_name == github.repository }}+ RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}+ MERGE_SHA: ${{ github.sha }}+ - id: filter+ name: Filter changed paths+ uses: dorny/paths-filter@d1c1ffe0248fe513906c8e24db8ea791d46f8590+ with:+ filters: |+ cargo_deny:+ - 'Cargo.toml'+ - 'crates/**/Cargo.toml'+ - 'services/**/Cargo.toml'+ - 'tooling/xtask/**/Cargo.toml'+ - 'Cargo.lock'+ - 'deny.toml'+ - '.github/workflows/cargo_deny.yml'+ workspace_dependencies:+ - 'Cargo.toml'+ - 'crates/**/Cargo.toml'+ - 'services/**/Cargo.toml'+ - 'tooling/xtask/**/Cargo.toml'+ - 'Cargo.lock'+ - '.config/hakari.toml'+ - 'crates/workspace-hack/**'+ - 'tooling/xtask/src/**'+ - 'tooling/xtask/crates/xtask_deps/**'+ - 'tooling/xtask/crates/xtask_graph/**'+ - '.github/workspace-dep-closures.json'+ generated_workflows:+ - 'tooling/xtask/**'+ - 'crates/macro_event_topics/**'+ - '.github/workflows/**'+ - '.github/kafka-cluster-topics.json'+ node_modules_nix:+ - 'bun.lock'+ - 'packages/sdk/bun.lock'+ - 'package.json'+ - 'apps/**/package.json'+ - 'packages/**/package.json'+ - 'services/**/package.json'+ - 'apps/web/patches/**'+ - 'nix-support/node_modules.nix'+ - 'nix-support/node_modules-hashes.json'+ - 'nix/tauri-desktop.nix'+ - 'tooling/scripts/update-node-modules-hash.sh'+ - 'flake.nix'+ - 'flake.lock'+ - '.github/workflows/check_node_modules_nix.yml'+ tauri_lockfiles:+ - 'apps/web/tauri/**'+ - 'crates/**/Cargo.toml'+ - 'Cargo.toml'+ - 'nix/tauri-desktop.nix'+ - 'flake.nix'+ - 'flake.lock'+ - '.github/workflows/check_tauri_lockfiles.yml'+ sdk:+ - 'packages/sdk/**'+ - '.github/workflows/sdk-check.yml'+ macrod_update:+ - 'crates/coding_agent_worker/**'+ - '.github/workflows/test_macrod_update.yml'+ cloud-storage:+ needs:+ - start+ name: Cloud Storage+ uses: ./.github/workflows/code_check_cloud_storage.yml+ secrets: inherit+ web-app:+ needs:+ - start+ name: Web App+ uses: ./.github/workflows/web-app-check-main.yml+ secrets: inherit+ infra:+ needs:+ - start+ name: Infra+ uses: ./.github/workflows/code_check_infra.yml+ secrets: inherit+ conventions:+ needs:+ - start+ name: Conventions+ uses: ./.github/workflows/code_check_conventions.yml+ secrets: inherit+ cargo-deny:+ needs:+ - start+ if: needs.start.outputs.cargo_deny == 'true'+ name: cargo deny+ uses: ./.github/workflows/cargo_deny.yml+ secrets: inherit+ workspace-dependencies:+ needs:+ - start+ if: needs.start.outputs.workspace_dependencies == 'true'+ name: Workspace dependencies+ uses: ./.github/workflows/cargo_workspace_dependency_check.yml+ secrets: inherit+ generated-workflows:+ needs:+ - start+ if: needs.start.outputs.generated_workflows == 'true'+ name: Generated workflows+ uses: ./.github/workflows/check_generated.yml+ secrets: inherit+ node-modules-nix:+ needs:+ - start+ if: needs.start.outputs.node_modules_nix == 'true'+ name: node_modules nix+ uses: ./.github/workflows/check_node_modules_nix.yml+ secrets: inherit+ tauri-lockfiles:+ needs:+ - start+ if: needs.start.outputs.tauri_lockfiles == 'true'+ name: Tauri lockfiles+ uses: ./.github/workflows/check_tauri_lockfiles.yml+ secrets: inherit+ sdk:+ needs:+ - start+ if: needs.start.outputs.sdk == 'true'+ name: SDK+ uses: ./.github/workflows/sdk-check.yml+ secrets: inherit+ macrod-update:+ needs:+ - start+ if: needs.start.outputs.macrod_update == 'true'+ name: macrod update+ uses: ./.github/workflows/test_macrod_update.yml+ secrets: inherit+ gate:+ needs:+ - start+ - cloud-storage+ - web-app+ - infra+ - conventions+ - cargo-deny+ - workspace-dependencies+ - generated-workflows+ - node-modules-nix+ - tauri-lockfiles+ - sdk+ - macrod-update+ if: '!cancelled() && needs.start.result != ''skipped'''+ name: CI gate+ runs-on: ubuntu-latest+ permissions:+ statuses: write+ timeout-minutes: 5+ steps:+ - name: Set the CI gate status+ run: |+ node --disable-warning=ExperimentalWarning --input-type=module-typescript <<'NODE_SCRIPT'+ // Sets the commit status `main` requires before a merge: it is green only on a+ // commit that a requested `PR CI` run passed. `CI_GATE_MODE` picks the step:+ //+ // remind a new commit: pending "not run" with the command to run it, unless+ // the commit already has a gate status+ // start a requested run begins: remove the label, check the merge commit+ // holds the PR head, set pending+ // finish the run ends: success when every job passed or was path-skipped+ // fork-request the label goes on a fork PR: what start does with a write+ // token, from a trusted run of `main`'s workflow+ // fork after a fork PR's run, whose read-only token could not set the status+ //+ // Statuses go on the PR head commit, so a later push starts without one.++ import { readFileSync } from 'node:fs';++ const API = 'https://api.github.com';+ const DESCRIPTION_LIMIT = 140;+ // An open request. Only trusted runs set statuses: a fork's token can't.+ const RUNNING = 'Running on';+ // Job names in `PR CI`; the fork step reads them back from the finished run.+ export const START_JOB = 'Start';+ export const GATE_JOB = 'CI gate';++ type State = 'pending' | 'success' | 'failure' | 'error';+ export type Needs = Record<string, { result: string }>;+ export type RunJob = { name: string; conclusion: string | null };+ type Outcome = { state: State; description: string } | null;++ function env(name: string): string {+ const value = process.env[name];+ if (!value) {+ throw new Error(`${name} is not set`);+ }+ return value;+ }++ async function github(method: string, path: string, body?: unknown): Promise<any> {+ const response = await fetch(`${API}${path}`, {+ method,+ headers: {+ accept: 'application/vnd.github+json',+ authorization: `Bearer ${env('GH_TOKEN')}`,+ 'x-github-api-version': '2022-11-28',+ },+ body: body === undefined ? undefined : JSON.stringify(body),+ });+ // Removing a label that is already gone is the outcome we wanted.+ if (method === 'DELETE' && response.status === 404) {+ return null;+ }+ if (!response.ok) {+ throw new Error(`${method} ${path}: HTTP ${response.status} ${await response.text()}`);+ }+ return response.status === 204 ? null : response.json();+ }++ const repo = () => env('GITHUB_REPOSITORY');+ const context = () => env('CI_GATE_CONTEXT');+ const label = () => env('CI_GATE_LABEL');++ /// REST, because `gh pr edit` fails on older gh releases.+ export function runCommand(repository: string, pr: string, labelName: string): string {+ return `gh api repos/${repository}/issues/${pr}/labels -f "labels[]=${labelName}"`;+ }++ /// The finished run's verdict from `toJSON(needs)`. Start must have succeeded;+ /// a called workflow may be skipped, because its path filter did not match.+ /// Null when start failed: it already set a status that says why.+ export function gateOutcome(needs: Needs, head: string): Outcome {+ if (needs.start?.result !== 'success') {+ return null;+ }++ const failed = Object.entries(needs)+ .filter(([, need]) => need.result !== 'success' && need.result !== 'skipped')+ .map(([job]) => job);++ if (failed.length > 0) {+ return { state: 'failure', description: `Failed: ${failed.join(', ')}` };+ }++ return { state: 'success', description: `Passed on ${head.slice(0, 7)}` };+ }++ /// A fork run's verdict from its jobs. Null when the run was not a request+ /// (unrelated label) or did not finish (cancelled).+ export function forkOutcome(jobs: RunJob[], head: string, labelName: string): Outcome {+ const start = jobs.find((job) => job.name === START_JOB);+ const gate = jobs.find((job) => job.name === GATE_JOB);++ if (!start || start.conclusion === 'skipped' || start.conclusion === 'cancelled') {+ return null;+ }++ if (start.conclusion !== 'success') {+ return {+ state: 'failure',+ description: `Could not start on ${head.slice(0, 7)}. Add the ${labelName} label again.`,+ };+ }++ if (gate?.conclusion === 'success') {+ return { state: 'success', description: `Passed on ${head.slice(0, 7)}` };+ }++ if (gate?.conclusion === 'failure') {+ return { state: 'failure', description: 'Failed: see the PR CI run' };+ }++ return null;+ }++ async function setStatus(sha: string, state: State, description: string, targetUrl: string) {+ await github('POST', `/repos/${repo()}/statuses/${sha}`, {+ state,+ context: context(),+ description: description.slice(0, DESCRIPTION_LIMIT),+ target_url: targetUrl,+ });+ console.log(`${context()} on ${sha.slice(0, 10)}: ${state}: ${description}`);+ }++ /// The newest gate status on a commit (GitHub lists statuses newest first).+ async function gateStatus(sha: string): Promise<{ state: string; description: string } | undefined> {+ const statuses: { context: string; state: string; description: string | null }[] = await github(+ 'GET',+ `/repos/${repo()}/commits/${sha}/statuses?per_page=100`,+ );+ const status = statuses.find((each) => each.context === context());+ return status && { state: status.state, description: status.description ?? '' };+ }++ /// Whether a trusted step opened a request on this commit that no result has+ /// closed yet.+ export function isOpenRequest(status: { state: string; description: string } | undefined): boolean {+ return status?.state === 'pending' && status.description.startsWith(RUNNING);+ }++ async function removeLabel(pr: string) {+ await github('DELETE', `/repos/${repo()}/issues/${pr}/labels/${encodeURIComponent(label())}`);+ }++ async function remind() {+ const head = env('HEAD_SHA');+ const pr = env('PR_NUMBER');++ // A re-open or leaving draft keeps the head, so it may have a result already.+ if (await gateStatus(head)) {+ console.log(`${head.slice(0, 10)} already has a ${context()} status`);+ return;+ }++ await setStatus(+ head,+ 'pending',+ `Not run on this commit. Run: ${runCommand(repo(), pr, label())}`,+ env('CI_GATE_DOCS_URL'),+ );+ }++ async function start() {+ const head = env('HEAD_SHA');+ const pr = env('PR_NUMBER');+ const runUrl = env('RUN_URL');+ const sameRepo = env('SAME_REPO') === 'true';++ // Fork tokens are read-only; the fork request step did both already.+ if (sameRepo) {+ await removeLabel(pr);+ }++ // A label added right after a push can carry the previous head, or a merge+ // commit GitHub has not rebuilt yet. Either would test an old commit.+ const pull = await github('GET', `/repos/${repo()}/pulls/${pr}`);+ if (pull.head.sha !== head) {+ const description = `The PR moved on from ${head.slice(0, 7)} to ${pull.head.sha.slice(0, 7)}. Run again: ${runCommand(repo(), pr, label())}`;+ if (sameRepo) {+ await setStatus(head, 'failure', description, runUrl);+ }+ throw new Error(description);+ }++ const merge = await github('GET', `/repos/${repo()}/commits/${env('MERGE_SHA')}`);+ const parents: string[] = merge.parents.map((parent: { sha: string }) => parent.sha);++ if (!parents.includes(head)) {+ const description = `The merge commit did not include ${head.slice(0, 7)} yet. Run again: ${runCommand(repo(), pr, label())}`;+ if (sameRepo) {+ await setStatus(head, 'failure', description, runUrl);+ }+ throw new Error(description);+ }++ if (sameRepo) {+ await setStatus(head, 'pending', `${RUNNING} ${head.slice(0, 7)}`, runUrl);+ }+ }++ async function finish() {+ const head = env('HEAD_SHA');+ const outcome = gateOutcome(JSON.parse(env('NEEDS')), head);++ if (!outcome) {+ console.log('Start did not succeed; its status stands.');+ process.exitCode = 1;+ return;+ }++ if (env('SAME_REPO') === 'true') {+ await setStatus(head, outcome.state, outcome.description, env('RUN_URL'));+ }++ if (outcome.state !== 'success') {+ console.log(outcome.description);+ process.exitCode = 1;+ }+ }++ async function forkRequest() {+ const head = env('HEAD_SHA');+ const pr = env('PR_NUMBER');++ await removeLabel(pr);+ await setStatus(head, 'pending', `${RUNNING} ${head.slice(0, 7)}`, env('PR_CHECKS_URL'));+ }++ async function fork() {+ const event = JSON.parse(readFileSync(env('GITHUB_EVENT_PATH'), 'utf8'));+ const run = event.workflow_run;+ const head: string = run.head_sha;++ // The request step opened this on the head the label was added at. A run on+ // any other commit, such as one the fork pushed with a rewritten workflow,+ // or a run after the result is in, finds no open request.+ if (!isOpenRequest(await gateStatus(head))) {+ console.log(`${head.slice(0, 10)} has no open ${context()} request; nothing to report.`);+ return;+ }++ const jobs: RunJob[] = [];+ for (let page = 1; ; page++) {+ const { jobs: batch } = await github(+ 'GET',+ `/repos/${repo()}/actions/runs/${run.id}/attempts/${run.run_attempt}/jobs?per_page=100&page=${page}`,+ );+ jobs.push(...batch);+ if (batch.length < 100) {+ break;+ }+ }++ const outcome = forkOutcome(jobs, head, label());+ if (outcome) {+ await setStatus(head, outcome.state, outcome.description, run.html_url);+ }+ }++ export async function main() {+ const modes: Record<string, () => Promise<void>> = {+ remind,+ start,+ finish,+ 'fork-request': forkRequest,+ fork,+ };+ const mode = modes[env('CI_GATE_MODE')];+ if (!mode) {+ throw new Error(`unknown CI_GATE_MODE ${process.env.CI_GATE_MODE}`);+ }+ await mode();+ }++ await main();+ NODE_SCRIPT+ shell: bash+ env:+ CI_GATE_MODE: finish+ CI_GATE_CONTEXT: CI gate+ CI_GATE_LABEL: run-ci+ GH_TOKEN: ${{ github.token }}+ PR_NUMBER: ${{ github.event.pull_request.number }}+ HEAD_SHA: ${{ github.event.pull_request.head.sha }}+ SAME_REPO: ${{ github.event.pull_request.head.repo.full_name == github.repository }}+ RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}+ NEEDS: ${{ toJSON(needs) }}+concurrency:+ group: pr-ci-${{ github.event.pull_request.number }}-${{ github.event.label.name == 'run-ci' && 'requested' || github.run_id }}+ cancel-in-progress: truediff --git a/.github/workflows/sdk-check.yml b/.github/workflows/sdk-check.ymlindex cb857e8c57..d8a02dd678 100644--- a/.github/workflows/sdk-check.yml+++ b/.github/workflows/sdk-check.yml@@ -14,6 +14,7 @@ name: SDK Checkpaths:- packages/sdk/**- '.github/workflows/sdk-check.yml'+ workflow_call: {}jobs:check-package:name: SDK Package Check@@ -41,5 +42,5 @@ jobs:run: bun run coverageworking-directory: packages/sdkconcurrency:- group: ${{ github.workflow }}-${{ github.ref }}-check+ group: sdk-check-${{ github.event.action == 'labeled' && github.run_id || github.ref }}cancel-in-progress: truediff --git a/.github/workflows/test_macrod_update.yml b/.github/workflows/test_macrod_update.ymlindex 1623c0a385..666e8e0b94 100644--- a/.github/workflows/test_macrod_update.yml+++ b/.github/workflows/test_macrod_update.yml@@ -8,6 +8,7 @@ name: Test macrod updatepaths:- crates/coding_agent_worker/**- '.github/workflows/test_macrod_update.yml'+ workflow_call: {}workflow_dispatch: {}permissions:contents: readdiff --git a/.github/workflows/web-app-check-main.yml b/.github/workflows/web-app-check-main.ymlindex 4d4a499054..e4ed2bbf90 100644--- a/.github/workflows/web-app-check-main.yml+++ b/.github/workflows/web-app-check-main.yml@@ -10,6 +10,7 @@ name: Web App Pr Checks- ready_for_reviewbranches:- main+ workflow_call: {}jobs:path-check:runs-on: namespace-profile-linux-tiny-no-cache@@ -1418,5 +1419,5 @@ jobs:echo "All jobs passed or were skipped"concurrency:- group: ${{ github.workflow }}-${{ github.ref }}-check+ group: web-app-pr-checks-${{ github.event.action == 'labeled' && github.run_id || github.ref }}cancel-in-progress: truediff --git a/CLAUDE.md b/CLAUDE.mdindex daf6506daa..c9c6dc689a 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -58,6 +58,7 @@ Real files live in `.agents/`; tool-specific directories only symlink to it.| Web frontend | [Web agent guide](apps/web/AGENTS.md) || iOS App Store releases, breaking Tauri changes, native build floors | [iOS releases](apps/web/docs/ios-releases.md) || Android Play releases from pull requests | [Android development](docs/development/android.md) |+| Running CI on a pull request | [CI on pull requests](docs/development/ci.md), `run-ci` skill || Email body rendering or snapshots | [Standalone renderer](packages/email-renderer/README.md) || Running the frontend or backend on a local machine | [Running locally](docs/development/running-locally.md) || Giving the user a local preview link | `local-preview` skill |@@ -122,8 +123,11 @@ Real files live in `.agents/`; tool-specific directories only symlink to it.reads it too.- Open pull requests ready for review. If one is created as a draft, mark itready for review before handing off.-- Get CI passing before handing off. A pull request whose checks are failing- or still running is not done.+- Run CI deliberately with the `run-ci` label (`run-ci` skill): at a+ checkpoint, a coherent locally tested chunk you want real CI on before+ building further, or when you think the PR is done. Never after every push,+ and never instead of local checks. A PR is done only when `CI gate` passes on+ its latest commit.- Test the affected packages/services individually before committing. Use therelevant guide for setup; there is no root `just test` recipe.- For code changes, run `just check` (format/lint/code rules scoped to changes).diff --git a/docs/development/ci.md b/docs/development/ci.mdnew file mode 100644index 0000000000..42b56406ca--- /dev/null+++ b/docs/development/ci.md@@ -0,0 +1,19 @@+# CI on pull requests++CI runs on a pull request when someone adds the `run-ci` label. The `PR CI`+workflow then runs every check on the PR's head commit, removes the label, and+reports the result as the `CI gate` commit status on that commit. Each push+leaves the new commit at `Not run on this commit`, so a PR can merge only after+a passing run on its latest commit.++Run CI deliberately: at a checkpoint, a coherent locally tested chunk you want+real CI on before building further, or when you think the PR is done. Never+after every push, and never instead of local checks.++The [`run-ci` skill](../../.agents/skills/run-ci/SKILL.md) has the commands,+what each `CI gate` state means, re-runs, and fork PRs. The workflows are+generated from Rust; see the+[workflows rule](../../.agents/rules/infra/github-workflows.md).++Until `main`'s ruleset requires `CI gate`, the checks also still run on every+push, and `main` still requires their old status checks.diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows.rs b/tooling/xtask/crates/xtask_workflows/src/workflows.rsindex 0053fc0d5f..89e31b8f5e 100644--- a/tooling/xtask/crates/xtask_workflows/src/workflows.rs+++ b/tooling/xtask/crates/xtask_workflows/src/workflows.rs@@ -27,6 +27,8 @@ mod cargo_workspace_dependency_check;mod check_generated;mod check_node_modules_nix;mod check_tauri_lockfiles;+mod ci_gate_fork;+mod ci_gate_reminder;mod cla;mod cleanup_preview;mod code_check_cloud_storage;@@ -50,6 +52,7 @@ mod ios_release_drift;mod ios_release_watch;mod native_check_label;mod path_validation;+mod pr_ci;mod pr_privacy_check;mod publish_sdk;mod pulumi_preview_pr;@@ -295,6 +298,21 @@ const WORKFLOWS: &[WorkflowFile] = &[file_name: "ios_preview.yml",render_yaml: || render_gh_workflow(ios_preview::ios_preview)(),},+ WorkflowFile {+ slug: "pr_ci",+ file_name: "pr-ci.yml",+ render_yaml: || render_patched(pr_ci::pr_ci, pr_ci::patch),+ },+ WorkflowFile {+ slug: "ci_gate_reminder",+ file_name: "ci-gate-reminder.yml",+ render_yaml: || render_gh_workflow(ci_gate_reminder::ci_gate_reminder)(),+ },+ WorkflowFile {+ slug: "ci_gate_fork",+ file_name: "ci-gate-fork.yml",+ render_yaml: || render_gh_workflow(ci_gate_fork::ci_gate_fork)(),+ },WorkflowFile {slug: "pr_privacy_check",file_name: "pr-privacy-check.yml",diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/cargo_deny.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/cargo_deny.rsindex 2bfbc63fcb..eed5a2e13b 100644--- a/tooling/xtask/crates/xtask_workflows/src/workflows/cargo_deny.rs+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/cargo_deny.rs@@ -2,34 +2,40 @@//! workspace on PRs that touch manifests. Generated into `cargo_deny.yml`//! (replaces the hand-written `cargo-deny.yml`).-use gh_workflow::{- Concurrency, Event, Expression, Job, PullRequest, PullRequestType, Step, Use, Workflow,-};+use gh_workflow::{Event, Job, PullRequest, PullRequestType, Step, Use, Workflow, WorkflowCall};+use xtask_paths::RepoGlob;-use crate::workflows::runners;+use crate::workflows::{runners, vars};++/// Files whose change runs the check, on a push and in `PR CI`.+pub const PATHS: &[RepoGlob<'static>] = &[+ xtask_paths::repo_glob!("Cargo.toml"),+ xtask_paths::repo_glob!("crates/**/Cargo.toml"),+ xtask_paths::repo_glob!("services/**/Cargo.toml"),+ xtask_paths::repo_glob!("tooling/xtask/**/Cargo.toml"),+ xtask_paths::repo_glob!("Cargo.lock"),+ xtask_paths::repo_glob!("deny.toml"),+ xtask_paths::repo_glob!(".github/workflows/cargo_deny.yml"),+];/// Build the workflow.pub fn cargo_deny() -> Workflow {+ let pull_request = PullRequest::default()+ .add_branch("main")+ .add_type(PullRequestType::Opened)+ .add_type(PullRequestType::Synchronize)+ .add_type(PullRequestType::Reopened)+ .add_type(PullRequestType::ReadyForReview);+Workflow::new("cargo deny check")- .on(Event::default().pull_request(- PullRequest::default()- .add_branch("main")- .add_type(PullRequestType::Opened)- .add_type(PullRequestType::Synchronize)- .add_type(PullRequestType::Reopened)- .add_type(PullRequestType::ReadyForReview)- .add_path(xtask_paths::repo_glob!("Cargo.toml"))- .add_path(xtask_paths::repo_glob!("crates/**/Cargo.toml"))- .add_path(xtask_paths::repo_glob!("services/**/Cargo.toml"))- .add_path(xtask_paths::repo_glob!("tooling/xtask/**/Cargo.toml"))- .add_path(xtask_paths::repo_glob!("Cargo.lock"))- .add_path(xtask_paths::repo_glob!("deny.toml"))- .add_path(xtask_paths::repo_glob!(".github/workflows/cargo_deny.yml")),- ))- .concurrency(- Concurrency::new(Expression::new("cargo-deny-${{ github.ref }}"))- .cancel_in_progress(true),- )+ .on(Event::default()+ .pull_request(+ PATHS+ .iter()+ .fold(pull_request, |trigger, path| trigger.add_path(*path)),+ )+ .workflow_call(WorkflowCall::default()))+ .concurrency(vars::concurrency("cargo-deny")).add_job("cargo-deny", cargo_deny_job())}diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/cargo_workspace_dependency_check.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/cargo_workspace_dependency_check.rsindex 674d11438b..1372be6659 100644--- a/tooling/xtask/crates/xtask_workflows/src/workflows/cargo_workspace_dependency_check.rs+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/cargo_workspace_dependency_check.rs@@ -10,48 +10,47 @@//! and xtask builds warm caches instead of compiling cold on every run.use gh_workflow::{- Concurrency, Event, Expression, Job, PullRequest, PullRequestType, Run, Step, Use, Workflow,+ Event, Job, PullRequest, PullRequestType, Run, Step, Use, Workflow, WorkflowCall,};+use xtask_paths::RepoGlob;use crate::workflows::{runners, steps, vars};+/// Files whose change runs the workflow, on a push and in `PR CI`.+pub const PATHS: &[RepoGlob<'static>] = &[+ xtask_paths::repo_glob!("Cargo.toml"),+ xtask_paths::repo_glob!("crates/**/Cargo.toml"),+ xtask_paths::repo_glob!("services/**/Cargo.toml"),+ xtask_paths::repo_glob!("tooling/xtask/**/Cargo.toml"),+ xtask_paths::repo_glob!("Cargo.lock"),+ xtask_paths::repo_glob!(".config/hakari.toml"),+ xtask_paths::repo_glob!("crates/workspace-hack/**"),+ // Source of `cargo x deps` only — not every xtask crate. A nextest-filter+ // or workflow-generator edit cannot drift the hack crate or closure map.+ xtask_paths::repo_glob!("tooling/xtask/src/**"),+ xtask_paths::repo_glob!("tooling/xtask/crates/xtask_deps/**"),+ xtask_paths::repo_glob!("tooling/xtask/crates/xtask_graph/**"),+ xtask_paths::repo_glob!(".github/workspace-dep-closures.json"),+];+/// Build the workflow.pub fn cargo_workspace_dependency_check() -> Workflow {+ let pull_request = PullRequest::default()+ .add_branch("main")+ .add_type(PullRequestType::Opened)+ .add_type(PullRequestType::Synchronize)+ .add_type(PullRequestType::Reopened)+ .add_type(PullRequestType::ReadyForReview);+Workflow::new("cargo workspace dependency check")- .on(Event::default().pull_request(- PullRequest::default()- .add_branch("main")- .add_type(PullRequestType::Opened)- .add_type(PullRequestType::Synchronize)- .add_type(PullRequestType::Reopened)- .add_type(PullRequestType::ReadyForReview)- .add_path(xtask_paths::repo_glob!("Cargo.toml"))- .add_path(xtask_paths::repo_glob!("crates/**/Cargo.toml"))- .add_path(xtask_paths::repo_glob!("services/**/Cargo.toml"))- .add_path(xtask_paths::repo_glob!("tooling/xtask/**/Cargo.toml"))- .add_path(xtask_paths::repo_glob!("Cargo.lock"))- .add_path(xtask_paths::repo_glob!(".config/hakari.toml"))- .add_path(xtask_paths::repo_glob!("crates/workspace-hack/**"))- // Source of `cargo x deps` only — not every xtask crate. A- // nextest-filter or workflow-generator edit cannot drift the- // hack crate or closure map.- .add_path(xtask_paths::repo_glob!("tooling/xtask/src/**"))- .add_path(xtask_paths::repo_glob!(- "tooling/xtask/crates/xtask_deps/**"- ))- .add_path(xtask_paths::repo_glob!(- "tooling/xtask/crates/xtask_graph/**"- ))- .add_path(xtask_paths::repo_glob!(- ".github/workspace-dep-closures.json"- )),- ))- .concurrency(- Concurrency::new(Expression::new(- "cargo-workspace-dependency-check-${{ github.ref }}",- ))- .cancel_in_progress(true),- )+ .on(Event::default()+ .pull_request(+ PATHS+ .iter()+ .fold(pull_request, |trigger, path| trigger.add_path(*path)),+ )+ .workflow_call(WorkflowCall::default()))+ .concurrency(vars::concurrency("cargo-workspace-dependency-check")).add_job("dependency-check", dependency_check()).add_job("hakari-and-dep-closures", hakari_and_dep_closures())}diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/check_generated.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/check_generated.rsindex 5c1ca6b647..af486acc3d 100644--- a/tooling/xtask/crates/xtask_workflows/src/workflows/check_generated.rs+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/check_generated.rs@@ -4,21 +4,31 @@//! (and the kafka topics file) are defined in Rust" enforceable rather than//! aspirational.-use gh_workflow::{Event, Job, PullRequest, Step, Workflow};+use gh_workflow::{Event, Job, PullRequest, Step, Workflow, WorkflowCall};+use xtask_paths::RepoGlob;use crate::workflows::{runners, steps};+/// Files whose change runs the workflow, on a push and in `PR CI`.+pub const PATHS: &[RepoGlob<'static>] = &[+ xtask_paths::repo_glob!("tooling/xtask/**"),+ xtask_paths::repo_glob!("crates/macro_event_topics/**"),+ xtask_paths::repo_glob!(".github/workflows/**"),+ xtask_paths::repo_glob!(".github/kafka-cluster-topics.json"),+];+/// Build the workflow.pub fn check_generated_workflows() -> Workflow {+ let pull_request = PullRequest::default().add_branch("main");+Workflow::new("check generated workflows")- .on(Event::default().pull_request(- PullRequest::default()- .add_branch("main")- .add_path(xtask_paths::repo_glob!("tooling/xtask/**"))- .add_path(xtask_paths::repo_glob!("crates/macro_event_topics/**"))- .add_path(xtask_paths::repo_glob!(".github/workflows/**"))- .add_path(xtask_paths::repo_glob!(".github/kafka-cluster-topics.json")),- ))+ .on(Event::default()+ .pull_request(+ PATHS+ .iter()+ .fold(pull_request, |trigger, path| trigger.add_path(*path)),+ )+ .workflow_call(WorkflowCall::default())).add_job("check-workflows", check_workflows())}diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/check_node_modules_nix.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/check_node_modules_nix.rsindex 826bcb24ff..fbb6dce34b 100644--- a/tooling/xtask/crates/xtask_workflows/src/workflows/check_node_modules_nix.rs+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/check_node_modules_nix.rs@@ -2,41 +2,45 @@//! derivation still matches the committed lockfile/package manifests. Generated//! into `check_node_modules_nix.yml`.-use gh_workflow::{Event, Job, Level, Permissions, PullRequest, Step, Workflow};+use gh_workflow::{Event, Job, Level, Permissions, PullRequest, Step, Workflow, WorkflowCall};+use xtask_paths::RepoGlob;use crate::workflows::{runners, steps};+/// Files whose change runs the workflow, on a push and in `PR CI`.+pub const PATHS: &[RepoGlob<'static>] = &[+ xtask_paths::repo_glob!("bun.lock"),+ xtask_paths::repo_glob!("packages/sdk/bun.lock"),+ xtask_paths::repo_glob!("package.json"),+ xtask_paths::repo_glob!("apps/**/package.json"),+ xtask_paths::repo_glob!("packages/**/package.json"),+ xtask_paths::repo_glob!("services/**/package.json"),+ xtask_paths::repo_glob!("apps/web/patches/**"),+ xtask_paths::repo_glob!("nix-support/node_modules.nix"),+ xtask_paths::repo_glob!("nix-support/node_modules-hashes.json"),+ xtask_paths::repo_glob!("nix/tauri-desktop.nix"),+ xtask_paths::repo_glob!("tooling/scripts/update-node-modules-hash.sh"),+ xtask_paths::repo_glob!("flake.nix"),+ xtask_paths::repo_glob!("flake.lock"),+ xtask_paths::repo_glob!(".github/workflows/check_node_modules_nix.yml"),+];+/// Build the workflow.pub fn check_node_modules_nix() -> Workflow {+ let pull_request = PullRequest::default().add_branch("main");+Workflow::new("check node_modules nix").permissions(Permissions {contents: Some(Level::Read),..Default::default()})- .on(Event::default().pull_request(- PullRequest::default()- .add_branch("main")- .add_path(xtask_paths::repo_glob!("bun.lock"))- .add_path(xtask_paths::repo_glob!("packages/sdk/bun.lock"))- .add_path(xtask_paths::repo_glob!("package.json"))- .add_path(xtask_paths::repo_glob!("apps/**/package.json"))- .add_path(xtask_paths::repo_glob!("packages/**/package.json"))- .add_path(xtask_paths::repo_glob!("services/**/package.json"))- .add_path(xtask_paths::repo_glob!("apps/web/patches/**"))- .add_path(xtask_paths::repo_glob!("nix-support/node_modules.nix"))- .add_path(xtask_paths::repo_glob!(- "nix-support/node_modules-hashes.json"- ))- .add_path(xtask_paths::repo_glob!("nix/tauri-desktop.nix"))- .add_path(xtask_paths::repo_glob!(- "tooling/scripts/update-node-modules-hash.sh"- ))- .add_path(xtask_paths::repo_glob!("flake.nix"))- .add_path(xtask_paths::repo_glob!("flake.lock"))- .add_path(xtask_paths::repo_glob!(- ".github/workflows/check_node_modules_nix.yml"- )),- ))+ .on(Event::default()+ .pull_request(+ PATHS+ .iter()+ .fold(pull_request, |trigger, path| trigger.add_path(*path)),+ )+ .workflow_call(WorkflowCall::default())).add_job("check-linux-node-modules-nix",check_linux_node_modules_nix(),diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/check_tauri_lockfiles.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/check_tauri_lockfiles.rsindex bd96d3268a..67caabfc2a 100644--- a/tooling/xtask/crates/xtask_workflows/src/workflows/check_tauri_lockfiles.rs+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/check_tauri_lockfiles.rs@@ -9,32 +9,40 @@//! runs only for those.//! Generated into `check_tauri_lockfiles.yml`.-use gh_workflow::{Event, Expression, Job, Level, Permissions, PullRequest, Step, Use, Workflow};+use gh_workflow::{+ Event, Expression, Job, Level, Permissions, PullRequest, Step, Use, Workflow, WorkflowCall,+};+use xtask_paths::RepoGlob;use crate::workflows::{runners, steps, vars};+/// Files whose change runs the workflow, on a push and in `PR CI`.+pub const PATHS: &[RepoGlob<'static>] = &[+ xtask_paths::repo_glob!("apps/web/tauri/**"),+ xtask_paths::repo_glob!("crates/**/Cargo.toml"),+ xtask_paths::repo_glob!("Cargo.toml"),+ xtask_paths::repo_glob!("nix/tauri-desktop.nix"),+ xtask_paths::repo_glob!("flake.nix"),+ xtask_paths::repo_glob!("flake.lock"),+ xtask_paths::repo_glob!(".github/workflows/check_tauri_lockfiles.yml"),+];+/// Build the workflow.pub fn check_tauri_lockfiles() -> Workflow {+ let pull_request = PullRequest::default().add_branch("main");+Workflow::new("check tauri lockfiles").permissions(Permissions {contents: Some(Level::Read),..Default::default()})- .on(Event::default().pull_request(- PullRequest::default()- .add_branch("main")- .add_path(xtask_paths::repo_glob!("apps/web/tauri/**"))- // Repository crates the Tauri workspaces reach by path, and the- // root workspace they inherit dependencies from.- .add_path(xtask_paths::repo_glob!("crates/**/Cargo.toml"))- .add_path(xtask_paths::repo_glob!("Cargo.toml"))- .add_path(xtask_paths::repo_glob!("nix/tauri-desktop.nix"))- .add_path(xtask_paths::repo_glob!("flake.nix"))- .add_path(xtask_paths::repo_glob!("flake.lock"))- .add_path(xtask_paths::repo_glob!(- ".github/workflows/check_tauri_lockfiles.yml"- )),- ))+ .on(Event::default()+ .pull_request(+ PATHS+ .iter()+ .fold(pull_request, |trigger, path| trigger.add_path(*path)),+ )+ .workflow_call(WorkflowCall::default())).add_job("check-tauri-lockfiles", check_tauri_lockfiles_job()).add_job("check-tauri-vendoring", check_tauri_vendoring_job())}diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/ci_gate_fork.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/ci_gate_fork.rsnew file mode 100644index 0000000000..4e73d91a21--- /dev/null+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/ci_gate_fork.rs@@ -0,0 +1,79 @@+//! `CI Gate for Forks` — what `PR CI`'s start and gate steps can't do for a+//! fork PR, whose token is read-only. Generated into `ci-gate-fork.yml`.+//!+//! Both triggers run this file from `main` with a write token and never check+//! out the fork's code. When the label goes on, `request` removes it and opens+//! a pending `CI gate` request on the head it was added at. When the fork's run+//! ends, `report` copies its verdict, but only onto a commit with an open+//! request: a fork token can't set statuses, so a run on any other commit, such+//! as one pushed with a rewritten workflow, reports nothing.++use gh_workflow::{+ Event, Expression, Job, Level, Permissions, PullRequestTarget, PullRequestType, Workflow,+ WorkflowRun, WorkflowRunType,+};++use crate::workflows::pr_ci;++const FORK_PULL_REQUEST: &str =+ "github.event.pull_request.head.repo.full_name != github.repository";++/// Build the workflow.+pub fn ci_gate_fork() -> Workflow {+ Workflow::new("CI Gate for Forks")+ .on(Event::default()+ .pull_request_target(+ PullRequestTarget::default()+ .add_branch("main")+ .add_type(PullRequestType::Labeled),+ )+ .workflow_run(+ WorkflowRun::default()+ .add_type(WorkflowRunType::Completed)+ .add_workflow("PR CI"),+ ))+ .add_job("request", request())+ .add_job("report", report())+}++fn request() -> Job {+ Job::default()+ .name("Take a fork's CI request")+ .cond(Expression::new(format!(+ "github.event_name == 'pull_request_target' && github.event.label.name == '{}' && {FORK_PULL_REQUEST}",+ pr_ci::RUN_LABEL+ )))+ .runs_on("ubuntu-latest")+ .timeout_minutes(5u32)+ .permissions(Permissions {+ issues: Some(Level::Write),+ pull_requests: Some(Level::Write),+ statuses: Some(Level::Write),+ ..Default::default()+ })+ .add_step(+ pr_ci::gate_step("Open the CI gate request", "fork-request")+ .add_env(("PR_NUMBER", "${{ github.event.pull_request.number }}"))+ .add_env(("HEAD_SHA", "${{ github.event.pull_request.head.sha }}"))+ .add_env((+ "PR_CHECKS_URL",+ "${{ github.event.pull_request.html_url }}/checks",+ )),+ )+}++fn report() -> Job {+ Job::default()+ .name("Report a fork's CI result")+ .cond(Expression::new(+ "github.event_name == 'workflow_run' && github.event.workflow_run.event == 'pull_request' && github.event.workflow_run.head_repository.full_name != github.repository",+ ))+ .runs_on("ubuntu-latest")+ .timeout_minutes(5u32)+ .permissions(Permissions {+ actions: Some(Level::Read),+ statuses: Some(Level::Write),+ ..Default::default()+ })+ .add_step(pr_ci::gate_step("Set the CI gate status", "fork"))+}diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/ci_gate_reminder.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/ci_gate_reminder.rsnew file mode 100644index 0000000000..57b1944103--- /dev/null+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/ci_gate_reminder.rs@@ -0,0 +1,46 @@+//! `CI Gate Reminder` — on every new PR commit, sets the `CI gate` status to+//! pending with the command that runs CI, so the commit reads as not passed+//! until someone runs [`pr_ci`](crate::workflows::pr_ci). Generated into+//! `ci-gate-reminder.yml`.++use gh_workflow::{+ Event, Expression, Job, Level, Permissions, PullRequest, PullRequestType, Workflow,+};++use crate::workflows::pr_ci;++/// Build the workflow.+pub fn ci_gate_reminder() -> Workflow {+ Workflow::new("CI Gate Reminder")+ .on(Event::default().pull_request(+ PullRequest::default()+ .add_branch("main")+ .add_type(PullRequestType::Opened)+ .add_type(PullRequestType::Synchronize)+ .add_type(PullRequestType::Reopened),+ ))+ .add_job("remind", remind())+}++/// GitHub's hosted runner: a status call needs no Namespace machine. A fork's+/// read-only token can't set statuses, so its commits show the required status+/// as expected instead.+fn remind() -> Job {+ Job::default()+ .name("CI not run yet")+ .cond(Expression::new(+ "github.event.pull_request.head.repo.full_name == github.repository",+ ))+ .runs_on("ubuntu-latest")+ .timeout_minutes(5u32)+ .permissions(Permissions {+ statuses: Some(Level::Write),+ ..Default::default()+ })+ .add_step(+ pr_ci::gate_step("Mark the commit as not run", "remind")+ .add_env(("PR_NUMBER", "${{ github.event.pull_request.number }}"))+ .add_env(("HEAD_SHA", "${{ github.event.pull_request.head.sha }}"))+ .add_env(("CI_GATE_DOCS_URL", pr_ci::GATE_DOCS_URL)),+ )+}diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/code_check_cloud_storage.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/code_check_cloud_storage.rsindex 99c7a6bd4d..261dac6a2a 100644--- a/tooling/xtask/crates/xtask_workflows/src/workflows/code_check_cloud_storage.rs+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/code_check_cloud_storage.rs@@ -11,7 +11,8 @@//! `graphql-schema-compat` run alongside `test`.use gh_workflow::{- Container, Env, Event, Expression, Job, Port, PullRequest, PullRequestType, Run, Step, Workflow,+ Container, Env, Event, Expression, Job, Port, PullRequest, PullRequestType, Run, Step,+ Workflow, WorkflowCall,};use xtask_paths::RuntimePath;@@ -27,14 +28,16 @@ mod test;/// Build the workflow.pub fn code_check_cloud_storage() -> Workflow {Workflow::new("cloud storage code check")- .on(Event::default().pull_request(- PullRequest::default()- .add_branch("main")- .add_type(PullRequestType::Opened)- .add_type(PullRequestType::Synchronize)- .add_type(PullRequestType::Reopened)- .add_type(PullRequestType::ReadyForReview),- ))+ .on(Event::default()+ .pull_request(+ PullRequest::default()+ .add_branch("main")+ .add_type(PullRequestType::Opened)+ .add_type(PullRequestType::Synchronize)+ .add_type(PullRequestType::Reopened)+ .add_type(PullRequestType::ReadyForReview),+ )+ .workflow_call(WorkflowCall::default())).map(vars::with_global_env).concurrency(vars::concurrency("code-check-cloud-storage")).add_job("path-check", path_check())@@ -134,7 +137,7 @@ fn doppler_config() -> Job {steps::gated_job().timeout_minutes(45u32).cond(Expression::new(- "needs.path-check.outputs.should_run == 'true' && github.event.pull_request.draft == false && needs.path-check.outputs.doppler_candidates == 'true'",+ "needs.path-check.outputs.should_run == 'true' && (github.event.pull_request.draft == false || github.event.action == 'labeled') && needs.path-check.outputs.doppler_candidates == 'true'",)).runs_on(runners::Runner::RustCi.with_cache_tag(vars::CI_CACHE_TAG)).map(with_check_env)@@ -160,7 +163,7 @@ fn graphql_schema_compat() -> Job {steps::gated_job().timeout_minutes(15u32).cond(Expression::new(- "needs.path-check.outputs.should_run == 'true' && github.event.pull_request.draft == false && needs.path-check.outputs.graphql_schema == 'true'",+ "needs.path-check.outputs.should_run == 'true' && (github.event.pull_request.draft == false || github.event.action == 'labeled') && needs.path-check.outputs.graphql_schema == 'true'",)).runs_on(runners::Runner::Small.to_string()).add_step(steps::checkout(false, false).add_with(("fetch-depth", 2)))@@ -179,7 +182,7 @@ fn test() -> Job {steps::gated_job().timeout_minutes(90u32).cond(Expression::new(- "needs.path-check.outputs.should_run == 'true' && github.event.pull_request.draft == false && needs.path-check.outputs.rust_packages != 'none' && needs.path-check.outputs.skip_tests != 'true'",+ "needs.path-check.outputs.should_run == 'true' && (github.event.pull_request.draft == false || github.event.action == 'labeled') && needs.path-check.outputs.rust_packages != 'none' && needs.path-check.outputs.skip_tests != 'true'",)).runs_on(runners::Runner::Large.with_cache_tag(vars::CI_CACHE_TAG)).add_env((@@ -247,7 +250,7 @@ fn build_service_bins() -> Job {}const BUILD_SERVICE_BINS_CONDITION: &str = "needs.path-check.outputs.should_run == 'true' \- && github.event.pull_request.draft == false \+ && (github.event.pull_request.draft == false || github.event.action == 'labeled') \&& needs.path-check.outputs.skip_tests != 'true' \&& (needs.path-check.outputs.rust_packages == 'all' \|| contains(needs.path-check.outputs.rust_packages, 'document_storage_service') \diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/code_check_conventions.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/code_check_conventions.rsindex 465069ec2a..0a4b87008a 100644--- a/tooling/xtask/crates/xtask_workflows/src/workflows/code_check_conventions.rs+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/code_check_conventions.rs@@ -8,8 +8,8 @@//! conventions from `docs/development/style-guide.md` at the repo root.use gh_workflow::{- Concurrency, Event, Expression, Job, Level, Permissions, PullRequest, PullRequestType, Run,- Step, Use, Workflow,+ Event, Expression, Job, Level, Permissions, PullRequest, PullRequestType, Run, Step, Use,+ Workflow, WorkflowCall,};use crate::workflows::{runners, steps, vars};@@ -28,18 +28,17 @@ pub fn code_check_conventions() -> Workflow {contents: Some(Level::Read),..Default::default()})- .on(Event::default().pull_request(- PullRequest::default()- .add_branch("main")- .add_type(PullRequestType::Opened)- .add_type(PullRequestType::Synchronize)- .add_type(PullRequestType::Reopened)- .add_type(PullRequestType::ReadyForReview),- ))- .concurrency(- Concurrency::new(Expression::new("code-check-conventions-${{ github.ref }}"))- .cancel_in_progress(true),- )+ .on(Event::default()+ .pull_request(+ PullRequest::default()+ .add_branch("main")+ .add_type(PullRequestType::Opened)+ .add_type(PullRequestType::Synchronize)+ .add_type(PullRequestType::Reopened)+ .add_type(PullRequestType::ReadyForReview),+ )+ .workflow_call(WorkflowCall::default()))+ .concurrency(vars::concurrency("code-check-conventions")).add_job("path-check", path_check()).add_job("ast-grep", ast_grep()).add_job("status-check", status_check())diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/code_check_infra.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/code_check_infra.rsindex 4d67dc8b76..02c4f22a0c 100644--- a/tooling/xtask/crates/xtask_workflows/src/workflows/code_check_infra.rs+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/code_check_infra.rs@@ -5,7 +5,7 @@//! unaffected by the file rename).use gh_workflow::{- Concurrency, Event, Expression, Job, PullRequest, PullRequestType, Run, Step, Use, Workflow,+ Event, Expression, Job, PullRequest, PullRequestType, Run, Step, Use, Workflow, WorkflowCall,};use crate::workflows::{@@ -26,18 +26,17 @@ fn infra_runner() -> String {/// Build the workflow.pub fn code_check_infra() -> Workflow {Workflow::new("infra code check")- .on(Event::default().pull_request(- PullRequest::default()- .add_branch("main")- .add_type(PullRequestType::Opened)- .add_type(PullRequestType::Synchronize)- .add_type(PullRequestType::Reopened)- .add_type(PullRequestType::ReadyForReview),- ))- .concurrency(- Concurrency::new(Expression::new("code-check-infra-${{ github.ref }}"))- .cancel_in_progress(true),- )+ .on(Event::default()+ .pull_request(+ PullRequest::default()+ .add_branch("main")+ .add_type(PullRequestType::Opened)+ .add_type(PullRequestType::Synchronize)+ .add_type(PullRequestType::Reopened)+ .add_type(PullRequestType::ReadyForReview),+ )+ .workflow_call(WorkflowCall::default()))+ .concurrency(vars::concurrency("code-check-infra")).add_job("path-check", path_check()).add_job("biome-check", biome_check()).add_job("check", check())diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/export_ci_telemetry.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/export_ci_telemetry.rsindex a292ef165e..3184a6c8d3 100644--- a/tooling/xtask/crates/xtask_workflows/src/workflows/export_ci_telemetry.rs+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/export_ci_telemetry.rs@@ -5,6 +5,8 @@ use gh_workflow::{Event, Expression, Job, Level, Permissions, Step, Workflow, WorkflowRun, WorkflowRunType,};+use crate::workflows::steps;+#[cfg(test)]mod test;@@ -21,6 +23,8 @@ const OBSERVED_WORKFLOWS: &[&str] = &["Build Rust CI Image","Build Web App","Build macOS DMG",+ "CI Gate Reminder",+ "CI Gate for Forks","CLA","Cancel Stuck Cloud Storage Deploys","Cleanup Preview",@@ -40,6 +44,7 @@ const OBSERVED_WORKFLOWS: &[&str] = &["Export Datadog observability definitions","Migrate Macro Database","Native Check Label",+ "PR CI","PR Privacy Check","Publish Android to Google Play","Publish SDK",@@ -93,14 +98,6 @@ pub fn export_ci_telemetry() -> Workflow {.add_job("export", export())}-/// Run embedded TypeScript on the hosted image's Node, which strips types itself, so-/// the exporter needs no checkout, build or dependency.-fn node_script(source: &str) -> String {- format!(- "node --disable-warning=ExperimentalWarning --input-type=module-typescript <<'TELEMETRY_SCRIPT'\n{source}\nawait main();\nTELEMETRY_SCRIPT\n"- )-}-fn export() -> Job {Job::default()// GitHub's hosted image supplies AWS CLI, Python and Node without a checkout/setup step.@@ -119,7 +116,9 @@ fn export() -> Job {// Before any credential exists, so nothing the run produced shares a step with it..add_step(Step::new("Collect push, job and step events")- .run(node_script(include_str!("scripts/ci_telemetry_collect.ts")))+ .run(steps::node_script(include_str!(+ "scripts/ci_telemetry_collect.ts"+ ))).shell("bash").if_condition(Expression::new("always()")).continue_on_error(true)@@ -153,7 +152,9 @@ fn export() -> Job {// the only record of a run the trace exporter refuses..add_step(Step::new("Send push, job and step events")- .run(node_script(include_str!("scripts/ci_telemetry_send.ts")))+ .run(steps::node_script(include_str!(+ "scripts/ci_telemetry_send.ts"+ ))).shell("bash").if_condition(Expression::new("always()")).continue_on_error(true)diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/pr_ci.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/pr_ci.rsnew file mode 100644index 0000000000..eff105c5e7--- /dev/null+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/pr_ci.rs@@ -0,0 +1,286 @@+//! `PR CI` — the pull request checks, run when someone adds the [`RUN_LABEL`]+//! label instead of on every push. Generated into `pr-ci.yml`.+//!+//! `start` removes the label, so adding it again runs CI again, and checks that+//! the merge commit GitHub built holds the PR head. The check workflows run as+//! reusable workflows; the path-filtered ones only when their files changed.+//! `CI gate` then sets the [`GATE_CONTEXT`] commit status on the PR head, which+//! `main` requires: a new push has no status until CI runs on it.+//!+//! Every label fires `labeled`, so runs for any other label skip every job and+//! take a concurrency group of their own: they can't cancel a requested run or+//! post a result. Fork PRs get a read-only token, so `ci_gate_fork` removes+//! their label and sets their status.++use anyhow::Result;+use gh_workflow::{+ Concurrency, Event, Expression, Job, Level, Permissions, PullRequest, PullRequestType, Run,+ Step, Use, Workflow,+};+use xtask_paths::RepoGlob;++use crate::workflows::{+ cargo_deny, cargo_workspace_dependency_check, check_generated, check_node_modules_nix,+ check_tauri_lockfiles, sdk_check, steps, test_macrod_update,+};++#[cfg(test)]+mod test;++/// Adding this label to a PR runs CI on its head.+pub const RUN_LABEL: &str = "run-ci";++/// The commit status `main`'s ruleset requires.+pub const GATE_CONTEXT: &str = "CI gate";++/// Where the status links before CI has run: how to run it and wait on it.+pub const GATE_DOCS_URL: &str =+ "https://github.com/macro-inc/macro/blob/main/.agents/skills/run-ci/SKILL.md";++const REQUESTED: &str = "github.event.label.name == 'run-ci'";++/// Job names the fork step reads back from a finished run (`ci_gate.ts`).+const START_JOB: &str = "Start";+const GATE_JOB: &str = "CI gate";++/// A check workflow `PR CI` calls.+struct Call {+ id: &'static str,+ /// Prefixes the called jobs' check names: `Web App / Typecheck`.+ name: &'static str,+ file: &'static str,+ /// Files that make it run; `None` runs it on every request.+ paths: Option<&'static [RepoGlob<'static>]>,+}++const CALLS: &[Call] = &[+ Call {+ id: "cloud-storage",+ name: "Cloud Storage",+ file: "code_check_cloud_storage.yml",+ paths: None,+ },+ Call {+ id: "web-app",+ name: "Web App",+ file: "web-app-check-main.yml",+ paths: None,+ },+ Call {+ id: "infra",+ name: "Infra",+ file: "code_check_infra.yml",+ paths: None,+ },+ Call {+ id: "conventions",+ name: "Conventions",+ file: "code_check_conventions.yml",+ paths: None,+ },+ Call {+ id: "cargo-deny",+ name: "cargo deny",+ file: "cargo_deny.yml",+ paths: Some(cargo_deny::PATHS),+ },+ Call {+ id: "workspace-dependencies",+ name: "Workspace dependencies",+ file: "cargo_workspace_dependency_check.yml",+ paths: Some(cargo_workspace_dependency_check::PATHS),+ },+ Call {+ id: "generated-workflows",+ name: "Generated workflows",+ file: "check_generated.yml",+ paths: Some(check_generated::PATHS),+ },+ Call {+ id: "node-modules-nix",+ name: "node_modules nix",+ file: "check_node_modules_nix.yml",+ paths: Some(check_node_modules_nix::PATHS),+ },+ Call {+ id: "tauri-lockfiles",+ name: "Tauri lockfiles",+ file: "check_tauri_lockfiles.yml",+ paths: Some(check_tauri_lockfiles::PATHS),+ },+ Call {+ id: "sdk",+ name: "SDK",+ file: "sdk-check.yml",+ paths: Some(sdk_check::PATHS),+ },+ Call {+ id: "macrod-update",+ name: "macrod update",+ file: "test_macrod_update.yml",+ paths: Some(test_macrod_update::PATHS),+ },+];++/// Build the workflow.+pub fn pr_ci() -> Workflow {+ let workflow = Workflow::new("PR CI")+ .on(Event::default().pull_request(+ PullRequest::default()+ .add_branch("main")+ .add_type(PullRequestType::Labeled),+ ))+ .concurrency(+ // One group per PR for requests, so a newer request cancels the+ // older run; a group per run for every other label.+ Concurrency::new(Expression::new(format!(+ "pr-ci-${{{{ github.event.pull_request.number }}}}-${{{{ {REQUESTED} && 'requested' || github.run_id }}}}"+ )))+ .cancel_in_progress(true),+ )+ .add_job("start", start());++ CALLS+ .iter()+ .fold(workflow, |workflow, call| {+ workflow.add_job(call.id, call_job(call))+ })+ .add_job("gate", gate())+}++/// Fill in what gh-workflow can't express on a reusable-workflow call.+pub fn patch(root: &mut serde_yaml::Value) -> Result<()> {+ for call in CALLS {+ let job = crate::workflows::job_mut(root, call.id)?;+ job.remove("runs-on");+ job.insert("secrets".into(), "inherit".into());+ }+ Ok(())+}++/// The output that says whether a path-filtered call runs.+fn output_name(call: &Call) -> String {+ call.id.replace('-', "_")+}++/// A step running `scripts/ci_gate.ts` in one of its modes.+pub(crate) fn gate_step(name: &str, mode: &str) -> Step<Run> {+ Step::new(name)+ .run(steps::node_script(include_str!("scripts/ci_gate.ts")))+ .shell("bash")+ .add_env(("CI_GATE_MODE", mode))+ .add_env(("CI_GATE_CONTEXT", GATE_CONTEXT))+ .add_env(("CI_GATE_LABEL", RUN_LABEL))+ .add_env(("GH_TOKEN", "${{ github.token }}"))+}++/// The PR facts the start and gate steps act on.+fn with_pull_request(step: Step<Run>) -> Step<Run> {+ step.add_env(("PR_NUMBER", "${{ github.event.pull_request.number }}"))+ .add_env(("HEAD_SHA", "${{ github.event.pull_request.head.sha }}"))+ .add_env((+ "SAME_REPO",+ "${{ github.event.pull_request.head.repo.full_name == github.repository }}",+ ))+ .add_env((+ "RUN_URL",+ "${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}",+ ))+}++fn start() -> Job {+ let job = Job::default()+ .name(START_JOB)+ .cond(Expression::new(REQUESTED))+ .runs_on("ubuntu-latest")+ .timeout_minutes(5u32)+ .permissions(Permissions {+ contents: Some(Level::Read),+ issues: Some(Level::Write),+ pull_requests: Some(Level::Write),+ statuses: Some(Level::Write),+ ..Default::default()+ })+ .add_step(+ with_pull_request(gate_step("Take the request", "start"))+ .add_env(("MERGE_SHA", "${{ github.sha }}")),+ )+ .add_step(paths_filter());++ CALLS+ .iter()+ .filter(|call| call.paths.is_some())+ .fold(job, |job, call| {+ let output = output_name(call);+ job.add_output(+ output.clone(),+ format!("${{{{ steps.filter.outputs.{output} }}}}"),+ )+ })+}++/// One filter per path-filtered call, from the same list its push trigger uses.+/// The PR's files come from the API, so no checkout.+fn paths_filter() -> Step<Use> {+ let filters: String = CALLS+ .iter()+ .filter_map(|call| call.paths.map(|paths| (output_name(call), paths)))+ .map(|(output, paths)| {+ let globs: String = paths+ .iter()+ .map(|path| format!(" - '{}'\n", path.as_str()))+ .collect();+ format!("{output}:\n{globs}")+ })+ .collect();++ Step::new("Filter changed paths")+ .uses(+ "dorny",+ "paths-filter",+ "d1c1ffe0248fe513906c8e24db8ea791d46f8590",+ ) // v3.0.3+ .id("filter")+ .add_with(("filters", filters))+}++fn call_job(call: &Call) -> Job {+ let job = Job::default()+ .name(call.name)+ .needs(vec!["start".to_string()])+ .uses(format!("./.github/workflows/{}", call.file));++ match call.paths {+ Some(_) => job.cond(Expression::new(format!(+ "needs.start.outputs.{} == 'true'",+ output_name(call)+ ))),+ None => job,+ }+}++/// Runs unless the run was cancelled (a newer request superseded it) or was+/// never a request. A skipped call is a path filter that didn't match.+fn gate() -> Job {+ let needs = std::iter::once("start")+ .chain(CALLS.iter().map(|call| call.id))+ .map(str::to_string)+ .collect::<Vec<_>>();++ Job::default()+ .name(GATE_JOB)+ .needs(needs)+ .cond(Expression::new(+ "!cancelled() && needs.start.result != 'skipped'",+ ))+ .runs_on("ubuntu-latest")+ .timeout_minutes(5u32)+ .permissions(Permissions {+ statuses: Some(Level::Write),+ ..Default::default()+ })+ .add_step(+ with_pull_request(gate_step("Set the CI gate status", "finish"))+ .add_env(("NEEDS", "${{ toJSON(needs) }}")),+ )+}diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/pr_ci/test.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/pr_ci/test.rsnew file mode 100644index 0000000000..9597b23eff--- /dev/null+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/pr_ci/test.rs@@ -0,0 +1,90 @@+use super::*;++fn rendered() -> serde_yaml::Value {+ let yaml = crate::workflows::render_patched(pr_ci, patch).expect("PR CI should render");+ serde_yaml::from_str(&yaml).expect("PR CI should parse")+}++/// A called workflow the gate doesn't wait for could fail after the gate passed.+#[test]+fn the_gate_waits_for_start_and_every_call() {+ let workflow = rendered();+ let needs: Vec<&str> = workflow["jobs"]["gate"]["needs"]+ .as_sequence()+ .expect("gate needs")+ .iter()+ .filter_map(serde_yaml::Value::as_str)+ .collect();++ assert_eq!(needs[0], "start");+ assert_eq!(+ &needs[1..],+ CALLS.iter().map(|call| call.id).collect::<Vec<_>>()+ );+}++/// GitHub refuses to call a workflow without a `workflow_call` trigger.+#[test]+fn every_called_workflow_can_be_called() {+ let workflows = concat!(env!("CARGO_MANIFEST_DIR"), "/../../../../.github/workflows");+ for call in CALLS {+ let path = format!("{workflows}/{}", call.file);+ let workflow: serde_yaml::Value =+ serde_yaml::from_str(&std::fs::read_to_string(&path).expect("called workflow exists"))+ .expect("called workflow parses");+ let triggers = &workflow["on"];++ assert!(+ triggers.get("workflow_call").is_some(),+ "{} has no workflow_call trigger",+ call.file+ );+ }+}++/// Runs for other labels must neither share a group with a request (and+/// cancel it) nor reach a called workflow.+#[test]+fn only_the_run_label_starts_anything() {+ let workflow = rendered();++ assert_eq!(+ workflow["jobs"]["start"]["if"].as_str(),+ Some("github.event.label.name == 'run-ci'")+ );+ for call in CALLS {+ let needs = &workflow["jobs"][call.id]["needs"];+ assert_eq!(+ needs[0].as_str(),+ Some("start"),+ "{} must wait for start",+ call.id+ );+ }+ let group = workflow["concurrency"]["group"].as_str().expect("group");+ assert!(group.contains("github.event.label.name == 'run-ci' && 'requested' || github.run_id"));+}++/// Actions would expand this in the embedded source before the shell ever saw it.+#[test]+fn the_gate_script_contains_no_actions_expressions() {+ assert!(!include_str!("../scripts/ci_gate.ts").contains("${{"));+}++#[test]+fn gate_script_scenarios() {+ let output = std::process::Command::new("node")+ .args(["--test", "ci_gate.test.ts"])+ .current_dir(concat!(+ env!("CARGO_MANIFEST_DIR"),+ "/src/workflows/scripts"+ ))+ .output()+ .expect("run CI gate scenarios with Node");+ assert!(+ output.status.success(),+ "CI gate scenarios failed:\n{}\n{}",+ String::from_utf8_lossy(&output.stdout),+ String::from_utf8_lossy(&output.stderr)+ );+}diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/scripts/android_release_preconditions.sh b/tooling/xtask/crates/xtask_workflows/src/workflows/scripts/android_release_preconditions.shindex 43ef184c24..e7b9c4996a 100644--- a/tooling/xtask/crates/xtask_workflows/src/workflows/scripts/android_release_preconditions.sh+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/scripts/android_release_preconditions.sh@@ -49,8 +49,10 @@ echo "Releasing Android version code ${version_code}."# (`gh pr checks` names are job names, not workflow names, plus the contexts# of commit statuses, such as the android-play/* ones Android Play Publish-# and Android Release Watch set.)-ours='^(Android Native Check|Android Play Release|Re-run the native check|Web App Status Check|Android Release Drift|iOS Native Check|android-play/)'+# and Android Release Watch set. PR CI prefixes the jobs it calls with+# `Web App / `. The CI gate status aggregates every check, and stays pending+# on a head no one has run PR CI on.)+ours='^((Web App / )?(Android Native Check|Web App Status Check|iOS Native Check)|Android Play Release|Re-run the native check|Android Release Drift|android-play/|CI gate$)'deadline=$(( $(date +%s) + WAIT_MINUTES * 60 ))while true; dochecks=$(gh pr checks "$PR_NUMBER" --repo "$GH_REPO" --json name,bucket \diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/scripts/ci_gate.test.ts b/tooling/xtask/crates/xtask_workflows/src/workflows/scripts/ci_gate.test.tsnew file mode 100644index 0000000000..09d0493412--- /dev/null+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/scripts/ci_gate.test.ts@@ -0,0 +1,133 @@+// Verdicts of the CI gate. Run by xtask_workflows' tests.++import assert from 'node:assert/strict';+import { describe, test } from 'node:test';+import { forkOutcome, GATE_JOB, gateOutcome, isOpenRequest, START_JOB } from './ci_gate.ts';++const HEAD = '0123456789abcdef0123456789abcdef01234567';++describe('gateOutcome', () => {+ test('passes when every called workflow passed or was path-skipped', () => {+ const outcome = gateOutcome(+ {+ start: { result: 'success' },+ 'web-app': { result: 'success' },+ 'cargo-deny': { result: 'skipped' },+ },+ HEAD,+ );++ assert.deepEqual(outcome, { state: 'success', description: 'Passed on 0123456' });+ });++ test('fails naming every workflow that did not pass', () => {+ const outcome = gateOutcome(+ {+ start: { result: 'success' },+ 'web-app': { result: 'failure' },+ 'cloud-storage': { result: 'cancelled' },+ infra: { result: 'success' },+ },+ HEAD,+ );++ assert.deepEqual(outcome, { state: 'failure', description: 'Failed: web-app, cloud-storage' });+ });++ test('leaves the status alone when start failed, because start already explained why', () => {+ const outcome = gateOutcome(+ { start: { result: 'failure' }, 'web-app': { result: 'skipped' } },+ HEAD,+ );++ assert.equal(outcome, null);+ });++ test('a skipped start never passes', () => {+ const outcome = gateOutcome(+ { start: { result: 'skipped' }, 'web-app': { result: 'skipped' } },+ HEAD,+ );++ assert.equal(outcome, null);+ });+});++describe('forkOutcome', () => {+ test('an unrelated label run reports nothing', () => {+ const outcome = forkOutcome(+ [+ { name: START_JOB, conclusion: 'skipped' },+ { name: GATE_JOB, conclusion: 'skipped' },+ ],+ HEAD,+ 'run-ci',+ );++ assert.equal(outcome, null);+ });++ test('a cancelled run reports nothing', () => {+ const outcome = forkOutcome(+ [+ { name: START_JOB, conclusion: 'success' },+ { name: GATE_JOB, conclusion: 'cancelled' },+ ],+ HEAD,+ 'run-ci',+ );++ assert.equal(outcome, null);+ });++ test('copies the gate job verdict', () => {+ const passed = forkOutcome(+ [+ { name: START_JOB, conclusion: 'success' },+ { name: 'Web App / Typecheck', conclusion: 'success' },+ { name: GATE_JOB, conclusion: 'success' },+ ],+ HEAD,+ 'run-ci',+ );+ const failed = forkOutcome(+ [+ { name: START_JOB, conclusion: 'success' },+ { name: GATE_JOB, conclusion: 'failure' },+ ],+ HEAD,+ 'run-ci',+ );++ assert.equal(passed?.state, 'success');+ assert.equal(failed?.state, 'failure');+ });++ test('a failed start fails with the command to run again', () => {+ const outcome = forkOutcome(+ [+ { name: START_JOB, conclusion: 'failure' },+ { name: GATE_JOB, conclusion: 'failure' },+ ],+ HEAD,+ 'run-ci',+ );++ assert.deepEqual(outcome, {+ state: 'failure',+ description: 'Could not start on 0123456. Add the run-ci label again.',+ });+ });+});++describe('isOpenRequest', () => {+ test('only a pending request that no result has closed', () => {+ assert.equal(isOpenRequest({ state: 'pending', description: 'Running on 0123456' }), true);+ assert.equal(isOpenRequest({ state: 'success', description: 'Passed on 0123456' }), false);+ assert.equal(+ isOpenRequest({ state: 'pending', description: 'Not run on this commit. Run: ...' }),+ false,+ );+ assert.equal(isOpenRequest(undefined), false);+ });+});diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/scripts/ci_gate.ts b/tooling/xtask/crates/xtask_workflows/src/workflows/scripts/ci_gate.tsnew file mode 100644index 0000000000..7e50caf2e2--- /dev/null+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/scripts/ci_gate.ts@@ -0,0 +1,272 @@+// Sets the commit status `main` requires before a merge: it is green only on a+// commit that a requested `PR CI` run passed. `CI_GATE_MODE` picks the step:+//+// remind a new commit: pending "not run" with the command to run it, unless+// the commit already has a gate status+// start a requested run begins: remove the label, check the merge commit+// holds the PR head, set pending+// finish the run ends: success when every job passed or was path-skipped+// fork-request the label goes on a fork PR: what start does with a write+// token, from a trusted run of `main`'s workflow+// fork after a fork PR's run, whose read-only token could not set the status+//+// Statuses go on the PR head commit, so a later push starts without one.++import { readFileSync } from 'node:fs';++const API = 'https://api.github.com';+const DESCRIPTION_LIMIT = 140;+// An open request. Only trusted runs set statuses: a fork's token can't.+const RUNNING = 'Running on';+// Job names in `PR CI`; the fork step reads them back from the finished run.+export const START_JOB = 'Start';+export const GATE_JOB = 'CI gate';++type State = 'pending' | 'success' | 'failure' | 'error';+export type Needs = Record<string, { result: string }>;+export type RunJob = { name: string; conclusion: string | null };+type Outcome = { state: State; description: string } | null;++function env(name: string): string {+ const value = process.env[name];+ if (!value) {+ throw new Error(`${name} is not set`);+ }+ return value;+}++async function github(method: string, path: string, body?: unknown): Promise<any> {+ const response = await fetch(`${API}${path}`, {+ method,+ headers: {+ accept: 'application/vnd.github+json',+ authorization: `Bearer ${env('GH_TOKEN')}`,+ 'x-github-api-version': '2022-11-28',+ },+ body: body === undefined ? undefined : JSON.stringify(body),+ });+ // Removing a label that is already gone is the outcome we wanted.+ if (method === 'DELETE' && response.status === 404) {+ return null;+ }+ if (!response.ok) {+ throw new Error(`${method} ${path}: HTTP ${response.status} ${await response.text()}`);+ }+ return response.status === 204 ? null : response.json();+}++const repo = () => env('GITHUB_REPOSITORY');+const context = () => env('CI_GATE_CONTEXT');+const label = () => env('CI_GATE_LABEL');++/// REST, because `gh pr edit` fails on older gh releases.+export function runCommand(repository: string, pr: string, labelName: string): string {+ return `gh api repos/${repository}/issues/${pr}/labels -f "labels[]=${labelName}"`;+}++/// The finished run's verdict from `toJSON(needs)`. Start must have succeeded;+/// a called workflow may be skipped, because its path filter did not match.+/// Null when start failed: it already set a status that says why.+export function gateOutcome(needs: Needs, head: string): Outcome {+ if (needs.start?.result !== 'success') {+ return null;+ }++ const failed = Object.entries(needs)+ .filter(([, need]) => need.result !== 'success' && need.result !== 'skipped')+ .map(([job]) => job);++ if (failed.length > 0) {+ return { state: 'failure', description: `Failed: ${failed.join(', ')}` };+ }++ return { state: 'success', description: `Passed on ${head.slice(0, 7)}` };+}++/// A fork run's verdict from its jobs. Null when the run was not a request+/// (unrelated label) or did not finish (cancelled).+export function forkOutcome(jobs: RunJob[], head: string, labelName: string): Outcome {+ const start = jobs.find((job) => job.name === START_JOB);+ const gate = jobs.find((job) => job.name === GATE_JOB);++ if (!start || start.conclusion === 'skipped' || start.conclusion === 'cancelled') {+ return null;+ }++ if (start.conclusion !== 'success') {+ return {+ state: 'failure',+ description: `Could not start on ${head.slice(0, 7)}. Add the ${labelName} label again.`,+ };+ }++ if (gate?.conclusion === 'success') {+ return { state: 'success', description: `Passed on ${head.slice(0, 7)}` };+ }++ if (gate?.conclusion === 'failure') {+ return { state: 'failure', description: 'Failed: see the PR CI run' };+ }++ return null;+}++async function setStatus(sha: string, state: State, description: string, targetUrl: string) {+ await github('POST', `/repos/${repo()}/statuses/${sha}`, {+ state,+ context: context(),+ description: description.slice(0, DESCRIPTION_LIMIT),+ target_url: targetUrl,+ });+ console.log(`${context()} on ${sha.slice(0, 10)}: ${state}: ${description}`);+}++/// The newest gate status on a commit (GitHub lists statuses newest first).+async function gateStatus(sha: string): Promise<{ state: string; description: string } | undefined> {+ const statuses: { context: string; state: string; description: string | null }[] = await github(+ 'GET',+ `/repos/${repo()}/commits/${sha}/statuses?per_page=100`,+ );+ const status = statuses.find((each) => each.context === context());+ return status && { state: status.state, description: status.description ?? '' };+}++/// Whether a trusted step opened a request on this commit that no result has+/// closed yet.+export function isOpenRequest(status: { state: string; description: string } | undefined): boolean {+ return status?.state === 'pending' && status.description.startsWith(RUNNING);+}++async function removeLabel(pr: string) {+ await github('DELETE', `/repos/${repo()}/issues/${pr}/labels/${encodeURIComponent(label())}`);+}++async function remind() {+ const head = env('HEAD_SHA');+ const pr = env('PR_NUMBER');++ // A re-open or leaving draft keeps the head, so it may have a result already.+ if (await gateStatus(head)) {+ console.log(`${head.slice(0, 10)} already has a ${context()} status`);+ return;+ }++ await setStatus(+ head,+ 'pending',+ `Not run on this commit. Run: ${runCommand(repo(), pr, label())}`,+ env('CI_GATE_DOCS_URL'),+ );+}++async function start() {+ const head = env('HEAD_SHA');+ const pr = env('PR_NUMBER');+ const runUrl = env('RUN_URL');+ const sameRepo = env('SAME_REPO') === 'true';++ // Fork tokens are read-only; the fork request step did both already.+ if (sameRepo) {+ await removeLabel(pr);+ }++ // A label added right after a push can carry the previous head, or a merge+ // commit GitHub has not rebuilt yet. Either would test an old commit.+ const pull = await github('GET', `/repos/${repo()}/pulls/${pr}`);+ if (pull.head.sha !== head) {+ const description = `The PR moved on from ${head.slice(0, 7)} to ${pull.head.sha.slice(0, 7)}. Run again: ${runCommand(repo(), pr, label())}`;+ if (sameRepo) {+ await setStatus(head, 'failure', description, runUrl);+ }+ throw new Error(description);+ }++ const merge = await github('GET', `/repos/${repo()}/commits/${env('MERGE_SHA')}`);+ const parents: string[] = merge.parents.map((parent: { sha: string }) => parent.sha);++ if (!parents.includes(head)) {+ const description = `The merge commit did not include ${head.slice(0, 7)} yet. Run again: ${runCommand(repo(), pr, label())}`;+ if (sameRepo) {+ await setStatus(head, 'failure', description, runUrl);+ }+ throw new Error(description);+ }++ if (sameRepo) {+ await setStatus(head, 'pending', `${RUNNING} ${head.slice(0, 7)}`, runUrl);+ }+}++async function finish() {+ const head = env('HEAD_SHA');+ const outcome = gateOutcome(JSON.parse(env('NEEDS')), head);++ if (!outcome) {+ console.log('Start did not succeed; its status stands.');+ process.exitCode = 1;+ return;+ }++ if (env('SAME_REPO') === 'true') {+ await setStatus(head, outcome.state, outcome.description, env('RUN_URL'));+ }++ if (outcome.state !== 'success') {+ console.log(outcome.description);+ process.exitCode = 1;+ }+}++async function forkRequest() {+ const head = env('HEAD_SHA');+ const pr = env('PR_NUMBER');++ await removeLabel(pr);+ await setStatus(head, 'pending', `${RUNNING} ${head.slice(0, 7)}`, env('PR_CHECKS_URL'));+}++async function fork() {+ const event = JSON.parse(readFileSync(env('GITHUB_EVENT_PATH'), 'utf8'));+ const run = event.workflow_run;+ const head: string = run.head_sha;++ // The request step opened this on the head the label was added at. A run on+ // any other commit, such as one the fork pushed with a rewritten workflow,+ // or a run after the result is in, finds no open request.+ if (!isOpenRequest(await gateStatus(head))) {+ console.log(`${head.slice(0, 10)} has no open ${context()} request; nothing to report.`);+ return;+ }++ const jobs: RunJob[] = [];+ for (let page = 1; ; page++) {+ const { jobs: batch } = await github(+ 'GET',+ `/repos/${repo()}/actions/runs/${run.id}/attempts/${run.run_attempt}/jobs?per_page=100&page=${page}`,+ );+ jobs.push(...batch);+ if (batch.length < 100) {+ break;+ }+ }++ const outcome = forkOutcome(jobs, head, label());+ if (outcome) {+ await setStatus(head, outcome.state, outcome.description, run.html_url);+ }+}++export async function main() {+ const modes: Record<string, () => Promise<void>> = {+ remind,+ start,+ finish,+ 'fork-request': forkRequest,+ fork,+ };+ const mode = modes[env('CI_GATE_MODE')];+ if (!mode) {+ throw new Error(`unknown CI_GATE_MODE ${process.env.CI_GATE_MODE}`);+ }+ await mode();+}diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/scripts/ios_release_preconditions.sh b/tooling/xtask/crates/xtask_workflows/src/workflows/scripts/ios_release_preconditions.shindex 9729e1a953..44c6523bde 100644--- a/tooling/xtask/crates/xtask_workflows/src/workflows/scripts/ios_release_preconditions.sh+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/scripts/ios_release_preconditions.sh@@ -34,8 +34,10 @@ fi# Web App Status Check aggregates the gate, so it is red whenever the gate is;# its other inputs (Typecheck, Test, Build, ...) are separate checks and stay.-# (`gh pr checks` names are job names, not workflow names.)-ours='^(iOS Native Check|Re-run the native check|Web App Status Check|iOS App Store Release|iOS Release Drift|Android Native Check)'+# The CI gate status aggregates all of them, and stays pending on a head no one+# has run PR CI on. (`gh pr checks` names are job names, not workflow names;+# PR CI prefixes the ones it calls with `Web App / `.)+ours='^((Web App / )?(iOS Native Check|Web App Status Check|Android Native Check)|Re-run the native check|iOS App Store Release|iOS Release Drift|CI gate$)'deadline=$(( $(date +%s) + WAIT_MINUTES * 60 ))while true; dochecks=$(gh pr checks "$PR_NUMBER" --repo "$GH_REPO" --json name,bucket \diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/sdk_check.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/sdk_check.rsindex 6bf09fc914..69211fcd30 100644--- a/tooling/xtask/crates/xtask_workflows/src/workflows/sdk_check.rs+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/sdk_check.rs@@ -1,36 +1,41 @@-use gh_workflow::{Concurrency, Event, Expression, Job, PullRequest, Push, Run, Step, Workflow};+use gh_workflow::{Event, Job, PullRequest, Push, Run, Step, Workflow, WorkflowCall};+use xtask_paths::RepoGlob;-use crate::workflows::steps;+use crate::workflows::{steps, vars};#[cfg(test)]mod test;+/// Files whose change runs the workflow, on a push and in `PR CI`.+pub const PATHS: &[RepoGlob<'static>] = &[+ xtask_paths::repo_glob!("packages/sdk/**"),+ xtask_paths::repo_glob!(".github/workflows/sdk-check.yml"),+];+/// Build the workflow. Whether `packages/sdk`'s generated layer is fresh is/// checked by the web app workflow's `Generated Code Check`, which builds the/// spec binaries once for every Rust-generated artifact.pub fn sdk_check() -> Workflow {+ let pull_request = PullRequest::default().add_branch("main");+Workflow::new("SDK Check").on(Event::default().pull_request(- PullRequest::default()- .add_branch("main")- .add_path("packages/sdk/**")- .add_path(".github/workflows/sdk-check.yml"),+ PATHS+ .iter()+ .fold(pull_request, |trigger, path| trigger.add_path(*path)),)// `Release SDK` publishes from `main` without re-checking the// package, so the same suite has to guard what lands there..push(- Push::default()- .add_branch("main")- .add_path("packages/sdk/**")- .add_path(".github/workflows/sdk-check.yml"),- ))- .concurrency(- Concurrency::new(Expression::new(- "${{ github.workflow }}-${{ github.ref }}-check",- ))- .cancel_in_progress(true),- )+ PATHS+ .iter()+ .fold(Push::default().add_branch("main"), |trigger, path| {+ trigger.add_path(*path)+ }),+ )+ .workflow_call(WorkflowCall::default()))+ .concurrency(vars::concurrency("sdk-check")).add_job("check-package", check_package())}diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/steps.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/steps.rsindex 39e379a231..bb7b95ce21 100644--- a/tooling/xtask/crates/xtask_workflows/src/workflows/steps.rs+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/steps.rs@@ -458,6 +458,15 @@ pub fn start_sccache_server() -> Step<Run> {Step::new("Start sccache server").run("sccache --start-server")}+/// Run embedded TypeScript on the hosted image's Node, which strips types+/// itself, so the step needs no checkout, build or dependency. The script+/// exports `main`.+pub fn node_script(source: &str) -> String {+ format!(+ "node --disable-warning=ExperimentalWarning --input-type=module-typescript <<'NODE_SCRIPT'\n{source}\nawait main();\nNODE_SCRIPT\n"+ )+}+/// `sccache --show-stats` at the end of a job (never fails the job).pub fn show_sccache_stats() -> Step<Run> {Step::new("show sccache stats")@@ -465,13 +474,14 @@ pub fn show_sccache_stats() -> Step<Run> {.if_condition(Expression::new("always()"))}-/// Base for jobs gated behind `path-check`: depends on it and runs only on-/// non-draft PRs where the path filter matched. Shared by `check` and `test`.+/// Base for jobs gated behind `path-check`: depends on it and runs only where+/// the path filter matched, on non-draft PRs unless the run was requested with+/// the `PR CI` label. Shared by `check` and `test`.pub fn gated_job() -> Job {Job::default().needs(vec!["path-check".to_string()]).cond(Expression::new(- "needs.path-check.outputs.should_run == 'true' && github.event.pull_request.draft == false",+ "needs.path-check.outputs.should_run == 'true' && (github.event.pull_request.draft == false || github.event.action == 'labeled')",))}diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/test_macrod_update.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/test_macrod_update.rsindex 600d8e480a..6b379d83df 100644--- a/tooling/xtask/crates/xtask_workflows/src/workflows/test_macrod_update.rs+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/test_macrod_update.rs@@ -7,7 +7,8 @@//! and macOS, and the launcher layout plus `install.ps1` on Windows. Nothing//! is published. Runs on pull requests that touch macrod, and on demand.-use gh_workflow::{Event, Job, Level, Permissions, PullRequest, Run, Step, Workflow};+use gh_workflow::{Event, Job, Level, Permissions, PullRequest, Run, Step, Workflow, WorkflowCall};+use xtask_paths::RepoGlob;use crate::workflows::{runners, steps, vars};@@ -18,8 +19,16 @@ const E2E: &str = "crates/coding_agent_worker/e2e";const V1: &str = "v2000.1.1.1";const V2: &str = "v2000.1.1.2";+/// Files whose change runs the workflow, on a push and in `PR CI`.+pub const PATHS: &[RepoGlob<'static>] = &[+ xtask_paths::repo_glob!("crates/coding_agent_worker/**"),+ xtask_paths::repo_glob!(".github/workflows/test_macrod_update.yml"),+];+/// Build the workflow.pub fn test_macrod_update() -> Workflow {+ let pull_request = PullRequest::default().add_branch("main");+Workflow::new("Test macrod update").permissions(Permissions {contents: Some(Level::Read),@@ -27,14 +36,12 @@ pub fn test_macrod_update() -> Workflow {}).on(Event::default().pull_request(- PullRequest::default()- .add_branch("main")- .add_path(xtask_paths::repo_glob!("crates/coding_agent_worker/**"))- .add_path(xtask_paths::repo_glob!(- ".github/workflows/test_macrod_update.yml"- )),+ PATHS+ .iter()+ .fold(pull_request, |trigger, path| trigger.add_path(*path)),)- .workflow_dispatch(gh_workflow::WorkflowDispatch::default()))+ .workflow_dispatch(gh_workflow::WorkflowDispatch::default())+ .workflow_call(WorkflowCall::default())).add_job("linux", linux_job()).add_job("macos", macos_job()).add_job("windows", windows_job())diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/vars.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/vars.rsindex 7861be5867..caf4f80b2a 100644--- a/tooling/xtask/crates/xtask_workflows/src/workflows/vars.rs+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/vars.rs@@ -117,8 +117,13 @@ pub fn with_global_env(workflow: Workflow) -> Workflow {.add_env(("RUSTFLAGS", "-C link-arg=-fuse-ld=lld"))}-/// Cancel superseded runs of this workflow on the same git ref.+/// Cancel superseded runs of this workflow on the same git ref. When `PR CI`+/// calls the workflow (a `labeled` event), the group is that run's own:+/// `PR CI` cancels its superseded runs itself, and a requested run must not+/// cancel, or be cancelled by, the same checks running for a push.pub fn concurrency(prefix: &str) -> Concurrency {- Concurrency::new(Expression::new(format!("{prefix}-${{{{ github.ref }}}}")))- .cancel_in_progress(true)+ Concurrency::new(Expression::new(format!(+ "{prefix}-${{{{ github.event.action == 'labeled' && github.run_id || github.ref }}}}"+ )))+ .cancel_in_progress(true)}diff --git a/tooling/xtask/crates/xtask_workflows/src/workflows/web_app_check_main.rs b/tooling/xtask/crates/xtask_workflows/src/workflows/web_app_check_main.rsindex 12b630c25f..43cda84ad2 100644--- a/tooling/xtask/crates/xtask_workflows/src/workflows/web_app_check_main.rs+++ b/tooling/xtask/crates/xtask_workflows/src/workflows/web_app_check_main.rs@@ -2,8 +2,8 @@//! Generated into `web-app-check-main.yml`.use gh_workflow::{- Concurrency, Event, Expression, Job, Level, Permissions, PullRequest, PullRequestType, Run,- Step, Strategy, Use, Workflow,+ Event, Expression, Job, Level, Permissions, PullRequest, PullRequestType, Run, Step, Strategy,+ Use, Workflow, WorkflowCall,};use crate::workflows::{@@ -36,20 +36,17 @@ fn mid_web_runner() -> String {/// Build the workflow.pub fn web_app_check_main() -> Workflow {Workflow::new("Web App Pr Checks")- .on(Event::default().pull_request(- PullRequest::default()- .add_branch("main")- .add_type(PullRequestType::Opened)- .add_type(PullRequestType::Synchronize)- .add_type(PullRequestType::Reopened)- .add_type(PullRequestType::ReadyForReview),- ))- .concurrency(- Concurrency::new(Expression::new(- "${{ github.workflow }}-${{ github.ref }}-check",- ))- .cancel_in_progress(true),- )+ .on(Event::default()+ .pull_request(+ PullRequest::default()+ .add_branch("main")+ .add_type(PullRequestType::Opened)+ .add_type(PullRequestType::Synchronize)+ .add_type(PullRequestType::Reopened)+ .add_type(PullRequestType::ReadyForReview),+ )+ .workflow_call(WorkflowCall::default()))+ .concurrency(vars::concurrency("web-app-pr-checks")).add_job("path-check", path_check()).add_job(android_native_check::JOB_ID, android_native_check()).add_job("typescript", typescript())