Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

The full-ci label comes off once the full run is dispatched, so later pushes don't rerun it (GST-3002) (#9682)

Garry Tan48ee767ea1cb
diff --git a/.github/workflows/full-ci.yml b/.github/workflows/full-ci.yml
index 14da88a9ba..3200cbaa28 100644
--- a/.github/workflows/full-ci.yml
+++ b/.github/workflows/full-ci.yml
@@ -4,7 +4,9 @@ name: Full CI dispatch
# pull-request run of Test is the cheap tier, and E2E does not run on pull
# requests. The full-ci label starts the full profile: this workflow dispatches
# Test and E2E on the pull request's head branch, the same shape as a push to
-# master, and dispatches again on every later push while the label stays.
+# master, then removes the label once both dispatches were accepted, so a later
+# push does not start a full run nobody asked for. A refused dispatch leaves the
+# label on, visibly. Add the label again at each head you want gated.
# Applying a label needs triage access or higher, so contributors cannot start
# it, and a fork's head branch is never dispatched. A thread can do the same by
# hand before its slot:
@@ -30,6 +32,7 @@ jobs:
permissions:
actions: write
contents: read
+ pull-requests: write
steps:
- name: Dispatch Test and E2E on the head
env:
@@ -37,6 +40,7 @@ jobs:
REPO: ${{ github.repository }}
BRANCH: ${{ github.event.pull_request.head.ref }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
+ PR: ${{ github.event.pull_request.number }}
run: |
current=$(gh api "repos/$REPO/commits/heads/$BRANCH" --jq .sha)
if [ "$current" != "$HEAD_SHA" ]; then
@@ -46,3 +50,10 @@ jobs:
gh workflow run test.yml -R "$REPO" --ref "$BRANCH"
gh workflow run e2e.yml -R "$REPO" --ref "$BRANCH"
echo "Dispatched Test and E2E (full profile) on $BRANCH at $HEAD_SHA." >> "$GITHUB_STEP_SUMMARY"
+ if ! removed=$(gh api -X DELETE "repos/$REPO/issues/$PR/labels/full-ci" --silent 2>&1); then
+ case "$removed" in
+ *"(HTTP 404)"*) echo "::notice::The full-ci label was already off this pull request." ;;
+ *) echo "$removed" >&2; exit 1 ;;
+ esac
+ fi
+ echo "Removed the full-ci label: add it again at the next head you want gated." >> "$GITHUB_STEP_SUMMARY"
diff --git a/changelog.d/ci-trial-full-ci-label.md b/changelog.d/ci-trial-full-ci-label.md
new file mode 100644
index 0000000000..02148bae32
--- /dev/null
+++ b/changelog.d/ci-trial-full-ci-label.md
@@ -0,0 +1 @@
+**The `full-ci` label now removes itself once Test and E2E are dispatched (add it again at each head you want gated; a refused dispatch leaves it on), and the stress gate's shard-limit error gives its own reason and says to split the PR.**
diff --git a/docs/ci-event-parity.md b/docs/ci-event-parity.md
index 61287e5838..ab35dd2d3e 100644
--- a/docs/ci-event-parity.md
+++ b/docs/ci-event-parity.md
@@ -53,7 +53,7 @@ checks ([below](#full-ci-on-pull-requests)), never per-cell matrix names.
| `nightly-watch.yml` | `workflow_run` event guard | Not a test scope: it files issues from qualifying scheduled and push-to-master runs | — |
| `fix-wave-gate.yml`, `fix-wave-closeout.yml` | Pull-request action and merge state | Not a test scope: merge-process automation | — |
| `harness-check.yml` | `pull_request` only; a missing `HARNESS_EVALS_READ_TOKEN` skips the job (succeeds with a notice); a fork head is refused | Named exception: advisory, non-required check ([below](#harness-check)) | None: pushes to master and schedules do not run it; the pin bump runs its steps by hand on master |
-| `full-ci.yml` | `full-ci` label, later pushes to a labeled pull request, same-repository head | Not a test scope: dispatches the full profile on the head | — |
+| `full-ci.yml` | `full-ci` label, same-repository head; the label comes off once both dispatches are accepted | Not a test scope: dispatches the full profile on the head | — |
| `test.yml`, `scale-tier.yml` | `dynamic-ci` filter (Trunk Dynamic CI) on pull requests; its gated jobs run unless the verdict is exactly `'false'` | Named exception ([below](#trunk-dynamic-ci)): pull requests only, Learning mode skips nothing | The full-ci dispatch, every push to master, the merge queue and the nightly schedules, which never run the filter |
## Full CI on pull requests
@@ -73,8 +73,12 @@ The full suite runs on the exact head before merge, with the same shape as a
push to master: every unit shard, serial, slow, both PostgreSQL arms and the
whole E2E corpus. A maintainer adds the `full-ci` label (a label needs triage
access, so contributors cannot add one). `full-ci.yml` then dispatches Test
-and E2E on the head branch, and again on every later push while the label
-stays. A fork's branch is never dispatched. A thread that wants the full suite
+and E2E on the head branch and removes the label once both dispatches were
+accepted, so a later push does not start a full run nobody asked for. A refused
+or failed dispatch leaves the label on, where it shows. To gate a later head, add
+the label again at that head. A head that moved after the label was added keeps
+the label, and the push that moved it dispatches and removes it. A fork's branch
+is never dispatched. A thread that wants the full suite
earlier, for a risky change, dispatches it itself:
`gh workflow run test.yml --ref <branch> && gh workflow run e2e.yml --ref <branch>`.
diff --git a/scripts/stress/gate.ts b/scripts/stress/gate.ts
index d04db87336..48d05101e9 100644
--- a/scripts/stress/gate.ts
+++ b/scripts/stress/gate.ts
@@ -85,7 +85,7 @@ function classify(root: string, files: string[], weights: Weights, unmeasured =
export interface GatePlanInput { event: string; baseSha?: string; headSha?: string; stressFiles?: string; stressBase?: string; stressHead?: string }
/** The PR gate: every changed test file plus helper importers, ten iterations, shards under the budget. */
-export function planGate(root: string, input: GatePlanInput, weights = loadWeights(root)): Plan | { error: string } {
+export function planGate(root: string, input: GatePlanInput, weights = loadWeights(root)): Plan | { error: string; why?: string } {
let files: string[];
let helpers: HelperExpansion[] = [];
let base = input.baseSha ?? '';
@@ -122,7 +122,8 @@ export function planGate(root: string, input: GatePlanInput, weights = loadWeigh
const { stress, notStressed } = classify(root, files, weights, GATE_UNMEASURED_QUANTILE);
let shards = planShards(stress, DEFAULT_ITERATIONS, GATE_SHARD_BUDGET_MS);
if (shards.length > GATE_MAX_SHARDS) shards = planShards(stress, DEFAULT_ITERATIONS, GATE_FALLBACK_SHARD_BUDGET_MS);
- if (shards.length > GATE_MAX_SHARDS) return { error: `the plan needs ${shards.length} shards of up to ${GATE_FALLBACK_SHARD_BUDGET_MS / 60_000} minutes, past GitHub's ${GATE_MAX_SHARDS}-job matrix limit. Next: split the PR, or dispatch the stress workflow with stress_files naming a subset` };
+ if (shards.length > GATE_MAX_SHARDS) return { error: `the plan needs ${shards.length} shards of up to ${GATE_FALLBACK_SHARD_BUDGET_MS / 60_000} minutes, past GitHub's ${GATE_MAX_SHARDS}-job matrix limit. Next: split the PR so each part plans ${GATE_MAX_SHARDS} shards or fewer; this head's stress check stays red until then`,
+ why: `GitHub refuses a matrix over ${GATE_MAX_SHARDS} jobs, and stressing only part of the changed files would let the rest land untested` };
return {
...emptyPlan('gate', stress.length ? `${stress.length} file(s) × ${DEFAULT_ITERATIONS} iterations` : 'no changed test files to stress'),
run: stress.length > 0, diagnostic, base, head, helpers, notStressed, shards,
@@ -441,7 +442,7 @@ async function main(argv: string[]): Promise<number> {
if (mode === 'gate') {
const result = planGate(ROOT, { event: env.EVENT ?? '', baseSha: env.BASE_SHA, headSha: env.HEAD_SHA, stressFiles: env.STRESS_FILES, stressBase: env.STRESS_BASE, stressHead: env.STRESS_HEAD });
if ('error' in result) {
- console.error(`::error::stress gate: the plan could not be computed: ${result.error}. Why: the gate stresses every test file the diff touches, so a guessed list could skip one. Docs: ${STRESS_DOCS}`);
+ console.error(`::error::stress gate: the plan could not be computed: ${result.error}. Why: ${result.why ?? 'the gate stresses every test file the diff touches, so a guessed list could skip one'}. Docs: ${STRESS_DOCS}`);
return 1;
}
plan = result;
diff --git a/scripts/structural-suites.tsv b/scripts/structural-suites.tsv
index 31192325b2..438568480c 100644
--- a/scripts/structural-suites.tsv
+++ b/scripts/structural-suites.tsv
@@ -349,6 +349,7 @@ test/scripts/e2e-wiring.test.ts (file-level) 14 readFileSync
test/scripts/engine-raw-sql-ratchet.test.ts check-engine-raw-sql-ratchet.ts: shrink-only baseline 11 readFileSync
test/scripts/fix-wave-gate.test.ts fix-wave gate workflow 3 readFileSync
test/scripts/full-ci-workflow.test.ts full CI on pull requests 7 readFileSync
+test/scripts/full-ci-workflow.test.ts full-ci label lifetime 4 readFileSync
test/scripts/full-ci-workflow.test.ts master-push reuse of a passed full-suite run 2 readFileSync
test/scripts/gh-issue-agent-filed.test.ts agent-filed label 2 readFileSync
test/scripts/heavy-door-workflow.test.ts heavy door supply chain and coverage gates 8 readFileSync
diff --git a/test/scripts/full-ci-workflow.test.ts b/test/scripts/full-ci-workflow.test.ts
index 599bee36a6..ddd4da7b4e 100644
--- a/test/scripts/full-ci-workflow.test.ts
+++ b/test/scripts/full-ci-workflow.test.ts
@@ -211,7 +211,7 @@ describe('full CI on pull requests', () => {
expect(dispatcher.on.pull_request).toEqual({ branches: ['master'], types: ['labeled', 'synchronize', 'reopened'] });
expect(dispatcher.permissions).toEqual({});
const job = dispatcher.jobs.dispatch!;
- expect(job.permissions).toEqual({ actions: 'write', contents: 'read' });
+ expect(job.permissions).toEqual({ actions: 'write', contents: 'read', 'pull-requests': 'write' });
const own = { head: { repo: { full_name: 'acme-example/gbrain' } } };
const fork = { head: { repo: { full_name: 'fork-a/gbrain' } } };
const labels = (...names: string[]) => names.map(name => ({ name }));
@@ -231,6 +231,66 @@ describe('full CI on pull requests', () => {
expect(run.indexOf('"$current" != "$HEAD_SHA"')).toBeLessThan(run.indexOf('gh workflow run'));
});
+ // The label is a per-head request: it comes off once both dispatches were accepted, stays on (visibly) when either was
+ // refused, and a maintainer adding it again at a later head fires again.
+ describe('full-ci label lifetime', () => {
+ const job = dispatcher.jobs.dispatch!;
+ const own = { head: { repo: { full_name: 'acme-example/gbrain' } } };
+ const dispatch = (opts: { refuse?: string; moved?: boolean; removal?: number } = {}) => {
+ const dir = mkdtempSync(join(tmpdir(), 'gbrain-full-ci-'));
+ try {
+ const log = join(dir, 'gh.log');
+ writeFileSync(join(dir, 'gh'), [
+ '#!/usr/bin/env bash',
+ `echo "$*" >> ${log}`,
+ `if [ "$1 $2" = "api repos/acme-example/gbrain/commits/heads/feature" ]; then echo ${opts.moved ? 'b'.repeat(40) : 'a'.repeat(40)}; exit 0; fi`,
+ `if [ "$1 $2 $3" = "workflow run ${opts.refuse ?? 'none'}" ]; then echo "HTTP 422: refused" >&2; exit 1; fi`,
+ ...(opts.removal ? [`if [ "$1 $2 $3" = "api -X DELETE" ]; then echo "gh: refused (HTTP ${opts.removal})" >&2; exit 1; fi`] : []),
+ 'exit 0',
+ ].join('\n'), { mode: 0o755 });
+ const result = spawnSync('bash', ['--noprofile', '--norc', '-e', '-o', 'pipefail', '-c', job.steps![0]!.run!], { encoding: 'utf8', env: {
+ PATH: `${dir}:${process.env.PATH}`, GH_TOKEN: 'x', REPO: 'acme-example/gbrain', BRANCH: 'feature', HEAD_SHA: 'a'.repeat(40), PR: '42',
+ GITHUB_STEP_SUMMARY: join(dir, 'summary.md') } });
+ return { status: result.status, calls: readFileSync(log, 'utf8').trim().split('\n') };
+ } finally {
+ rmSync(dir, { recursive: true, force: true });
+ }
+ };
+ const removal = 'api -X DELETE repos/acme-example/gbrain/issues/42/labels/full-ci --silent';
+
+ test('the label comes off only after both Test and E2E dispatches were accepted', () => {
+ const ok = dispatch();
+ expect(ok.status).toBe(0);
+ expect(ok.calls.slice(1)).toEqual(['workflow run test.yml -R acme-example/gbrain --ref feature', 'workflow run e2e.yml -R acme-example/gbrain --ref feature', removal]);
+ });
+
+ test.each(['test.yml', 'e2e.yml'])('a refused %s dispatch fails the job and leaves the label on', workflow => {
+ const refused = dispatch({ refuse: workflow });
+ expect(refused.status).not.toBe(0);
+ expect(refused.calls).not.toContain(removal);
+ });
+
+ test('a label already removed (404) leaves the job green; any other removal error fails it', () => {
+ const gone = dispatch({ removal: 404 });
+ expect(gone.status).toBe(0);
+ expect(gone.calls).toContain(removal);
+ expect(dispatch({ removal: 403 }).status).not.toBe(0);
+ });
+
+ test('a head that moved after the event dispatches nothing and keeps the label for the push that moved it', () => {
+ const moved = dispatch({ moved: true });
+ expect(moved.status).toBe(0);
+ expect(moved.calls).toHaveLength(1);
+ });
+
+ test('once removed, a later push does not fire; the label added again at a later head does', () => {
+ const labels = (...names: string[]) => names.map(name => ({ name }));
+ expect(Boolean(evaluate(job.if!, { event: 'pull_request', action: 'synchronize', pr: { ...own, labels: labels() } }))).toBe(false);
+ expect(Boolean(evaluate(job.if!, { event: 'pull_request', action: 'labeled', label: 'full-ci', pr: { ...own, labels: labels('full-ci') } }))).toBe(true);
+ expect(dispatch().calls).toContain(removal);
+ });
+ });
+
describe('cheap-tier aggregator', () => {
const green = { CHANGES: 'success', GITLEAKS: 'success', SECURITY: 'success', AUDIT: 'success', VERIFY: 'success', TYPECHECK: 'success', STRESS: 'success',
NATIVE: 'skipped', NATIVE_SCOPE: 'smoke', PERSISTENCE: 'skipped', CONVERGENCE: 'skip' };
diff --git a/test/scripts/stress-gate.test.ts b/test/scripts/stress-gate.test.ts
index 7ae60bf838..8d3d80e03e 100644
--- a/test/scripts/stress-gate.test.ts
+++ b/test/scripts/stress-gate.test.ts
@@ -86,7 +86,11 @@ describe('gate plan', () => {
const head = commit(root, Object.fromEntries(files.map(f => [f, "test('n', () => {});\n"])));
const unit = new Map(files.map(f => [f, 100_000] as [string, number]));
const result = planGate(root, { event: 'pull_request', baseSha: base, headSha: head }, { ...weights(), unit });
- expect(result).toMatchObject({ error: expect.stringContaining("GitHub's 256-job matrix limit") });
+ const { error, why } = result as { error: string; why?: string };
+ expect(error).toContain("GitHub's 256-job matrix limit");
+ expect(error).toContain('Next: split the PR so each part plans 256 shards or fewer');
+ expect(error).not.toContain('stress_files');
+ expect(why).toContain('would let the rest land untested');
});
test('a missing merge base fails the plan with the fetch next step', () => {