Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

Rerun an aborted typecheck once from cold instead of failing on a stale cache

garrytan6c39c35f0ee2
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
index d2ca47b9bf..936f5cdb8b 100644
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -523,10 +523,15 @@ jobs:
# 121 s in verify's pool and timed out at the cap. A timed-out tsc also
# writes no tsbuildinfo, so every rerun restored the same stale state.
# Step timeout from measurement: 3 x the worst cold CI time (121 s) is
- # about 6 minutes. tsc is single-threaded, so 4 vCPUs (16 GB; peak heap
- # is about 5.3 GB) do the work of verify's 8.
+ # about 6 minutes per attempt. tsc is single-threaded, so 4 vCPUs (16 GB)
+ # do the work of verify's 8. A cold tsc of the tree peaks at about 6.5 GB
+ # RSS (6.51 GB on gbrain#9184's head, 2026-11-11), close to the heap cap
+ # in package.json's typecheck script; an incremental run seeded from a
+ # distant head's tsbuildinfo goes past it and aborts (exit 134).
+ # scripts/ci-typecheck.sh then runs it once more from cold, so the step
+ # and job allow two attempts.
runs-on: ubicloud-standard-4-ubuntu-2404
- timeout-minutes: 15
+ timeout-minutes: 21
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
@@ -552,14 +557,18 @@ jobs:
key: tsbuildinfo-${{ runner.os }}-${{ hashFiles('bun.lock', 'tsconfig.json') }}-${{ github.sha }}
restore-keys: |
tsbuildinfo-${{ runner.os }}-${{ hashFiles('bun.lock', 'tsconfig.json') }}-
- - name: Typecheck (tsc, incremental)
- timeout-minutes: 6
- run: bun run typecheck
+ - name: Typecheck (tsc, incremental; cold once more on an abort)
+ id: tsc
+ timeout-minutes: 12
+ run: bash scripts/ci-typecheck.sh bun run typecheck
# Saved even when this check finds type errors, and whatever the other
# jobs do: tsc writes its state on an error too, so the next master run
- # starts from this commit instead of an older, colder state.
+ # starts from this commit instead of an older, colder state. Not saved
+ # when tsc's last attempt aborted or timed out, or the step never
+ # reported (cancelled): tsc wrote no state, and saving the restored file
+ # under this head's key would make every rerun restore it again.
- name: Save the typecheck state
- if: always()
+ if: always() && steps.tsc.outputs.state_written == 'true'
uses: actions/cache/save@5a3ec84eff668545956fd18022155c47e93e2684 # v4.2.3
with:
path: node_modules/.cache/gbrain-typecheck.tsbuildinfo
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 38b0d9b8cc..cfdf02008a 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -21,6 +21,7 @@ identifiers and attribution are available in the pre-removal Git revision
- `scripts/bench/sync-freshness-gate.ts`: the Postgres managed serve-watch cell is now measured 3 times, and the gate uses the median of the three p50s, as the PGLite managed cell already did. The baseline and the 25% limit are unchanged. The cell's nightly p50 moves in steps of the 50 ms probe. Single runs at 656.5, 659.2 and 706.8 ms failed the 627 ms limit on 3 of 19 nightlies, and each passed when rerun on the same commit. A regression that slows every run still fails.
- `scripts/ci-census-tests.sh`: the pull-request census job now also runs `test/scripts/run-unit-shard.test.ts` (1.5 s, no database). It reads every test file's per-test timeouts, so a bad timeout now fails the pull request instead of a later full run.
- `scripts/sharding.ts`, `.github/workflows/test.yml`, `.github/workflows/e2e.yml`: the unit suite now splits into 30 shards instead of 28, run two per job (15 jobs), and the shard weights are refreshed from the median of five Test nightlies and the latest full E2E nightly. On the new weights 28 shards came to about 601 s each, over the 600 s per-shard budget; 30 shards come to about 561 s. Both coverage merges now expect shards 29 and 30.
+- `scripts/ci-typecheck.sh`, `.github/workflows/test.yml`: when the typecheck job's tsc aborts without a type error (exit 134 or a signal, as when V8 hits the heap limit), the job deletes the restored `tsbuildinfo` and runs tsc once more from cold. A run seeded with a distant head's state could go past the heap cap where a cold run of the same tree fits, and because the aborted run saved that same state under its own head's key, every rerun failed the same way. A type error (exit 1 or 2) still fails at once with no retry, and a run whose tsc aborted or timed out no longer saves its state. `test/scripts/ci-typecheck.test.ts` pins the retry to aborts only.
## [0.60.675.0] - 2026-11-11
diff --git a/scripts/ci-typecheck.sh b/scripts/ci-typecheck.sh
new file mode 100755
index 0000000000..b149eb1a89
--- /dev/null
+++ b/scripts/ci-typecheck.sh
@@ -0,0 +1,55 @@
+#!/usr/bin/env bash
+# test.yml's typecheck step: run the incremental tsc, and if it aborts, run it
+# once more from cold.
+#
+# Usage: scripts/ci-typecheck.sh <command>... (test.yml: bun run typecheck)
+#
+# The job restores node_modules/.cache/gbrain-typecheck.tsbuildinfo from this
+# head's cache entry or, failing that, from the newest entry of any head with
+# the same bun.lock and tsconfig.json. Seeded from a distant head, an
+# incremental tsc holds the old and the new program at once and dies at the
+# heap cap (V8 "Ineffective mark-compacts near heap limit", exit 134) while a
+# cold run of the same tree fits. Rerunning the job restores the same stale
+# file, so it fails the same way every time.
+#
+# So: an attempt that ends by abort or signal (exit status above 128) deletes
+# the buildinfo and runs once more, cold. Any other status ends the step at
+# once with that status, so a type error (tsc exits 1 or 2) is never retried,
+# and neither is an attempt cut off by its own time limit (timeout exits 124).
+#
+# Writes state_written=true|false to $GITHUB_OUTPUT: false when the last
+# attempt aborted or hit its time limit, because tsc then wrote no state. The
+# cache-save step saves only on true; saving the restored file under this
+# head's key would make every rerun restore it again.
+#
+# GBRAIN_TYPECHECK_ATTEMPT_SECONDS bounds each attempt (default 360, the
+# measured bound the step had before the retry).
+set -uo pipefail
+
+[ "$#" -gt 0 ] || { echo "usage: scripts/ci-typecheck.sh <command>..." >&2; exit 64; }
+BUILDINFO=node_modules/.cache/gbrain-typecheck.tsbuildinfo
+SECONDS_PER_ATTEMPT="${GBRAIN_TYPECHECK_ATTEMPT_SECONDS:-360}"
+
+record() {
+ local written=true
+ { [ "$1" -gt 128 ] || [ "$1" -eq 124 ]; } && written=false
+ [ -n "${GITHUB_OUTPUT:-}" ] && echo "state_written=$written" >> "$GITHUB_OUTPUT"
+ return 0
+}
+
+timeout "$SECONDS_PER_ATTEMPT" "$@"
+status=$?
+if [ "$status" -le 128 ]; then
+ record "$status"
+ exit "$status"
+fi
+
+echo "::warning title=typecheck aborted::tsc ended by abort or signal (exit $status) without a type error; deleting $BUILDINFO and running it once more from cold"
+rm -f "$BUILDINFO"
+timeout "$SECONDS_PER_ATTEMPT" "$@"
+status=$?
+record "$status"
+if [ "$status" -gt 128 ]; then
+ echo "::error title=typecheck aborted twice::tsc ended by abort or signal (exit $status) on the cold run too; this is not a type error"
+fi
+exit "$status"
diff --git a/test/scripts/ci-gates.test.ts b/test/scripts/ci-gates.test.ts
index 8c802175d7..7918d0f519 100644
--- a/test/scripts/ci-gates.test.ts
+++ b/test/scripts/ci-gates.test.ts
@@ -57,13 +57,14 @@ describe('CI execution evidence', () => {
expect(pkg.scripts.verify).toContain('scripts/run-verify-parallel.sh');
});
- test('typecheck runs as its own job beside verify, under a measured step timeout, saves its state on every outcome and gates test-status', () => {
+ test('typecheck runs as its own job beside verify, under a measured step timeout, saves its state whenever tsc finished and gates test-status', () => {
const job = unit.jobs.typecheck;
- const tsc = job.steps.find(step => step.run === 'bun run typecheck')!;
- expect(tsc['timeout-minutes']).toBe(6);
+ const tsc = job.steps.find(step => step.run === 'bash scripts/ci-typecheck.sh bun run typecheck')! as Job['steps'][number] & { id?: string };
+ expect(tsc.id).toBe('tsc');
+ expect(tsc['timeout-minutes']).toBe(12);
const restore = job.steps.find(step => step.uses?.startsWith('actions/cache/restore@') && step.with?.path === 'node_modules/.cache/gbrain-typecheck.tsbuildinfo')!;
const save = job.steps.find(step => step.uses?.startsWith('actions/cache/save@'))!;
- expect(save.if).toBe('always()');
+ expect(save.if).toBe("always() && steps.tsc.outputs.state_written == 'true'");
expect(save.with).toEqual({ path: restore.with!.path, key: restore.with!.key });
expect(job.steps.indexOf(save)).toBeGreaterThan(job.steps.indexOf(tsc));
const verify = unit.jobs.verify.steps.find(step => step.run === 'bun run verify')!;
diff --git a/test/scripts/ci-typecheck.test.ts b/test/scripts/ci-typecheck.test.ts
new file mode 100644
index 0000000000..7b4b1ddd06
--- /dev/null
+++ b/test/scripts/ci-typecheck.test.ts
@@ -0,0 +1,112 @@
+/**
+ * scripts/ci-typecheck.sh runs test.yml's tsc step. When tsc aborts (exit
+ * above 128: V8's heap-limit abort is 134), it deletes the restored
+ * tsbuildinfo and runs once more from cold. Every other status, a type error
+ * (1 or 2) above all, ends the step at once with no retry. These cases pin
+ * the retry to aborts only, and keep an aborted or timed-out run from saving
+ * its stale state.
+ */
+import { afterEach, describe, expect, test } from 'bun:test';
+import { chmodSync, existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+
+const SCRIPT = join(import.meta.dir, '..', '..', 'scripts', 'ci-typecheck.sh');
+const BUILDINFO = 'node_modules/.cache/gbrain-typecheck.tsbuildinfo';
+const roots: string[] = [];
+afterEach(() => {
+ for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true });
+});
+
+// A stand-in for `bun run typecheck`: logs whether the buildinfo was present,
+// then ends the way the next line of $FIXTURE_PLAN says: an exit code, a
+// signal name (the process kills itself with it), or "hang".
+const FAKE_TSC = String.raw`#!/usr/bin/env bash
+n=$(( $(wc -l < "$FIXTURE_LOG") + 1 ))
+if [ -f node_modules/.cache/gbrain-typecheck.tsbuildinfo ]; then echo warm >> "$FIXTURE_LOG"; else echo cold >> "$FIXTURE_LOG"; fi
+step=$(sed -n "$n"p "$FIXTURE_PLAN")
+case "$step" in
+ ABRT|KILL) kill -s "$step" $$ ;;
+ hang) sleep 30 ;;
+ *) exit "$step" ;;
+esac
+`;
+
+function run(plan: string[], seconds = '20') {
+ const root = mkdtempSync(join(tmpdir(), 'gbrain-ci-typecheck-'));
+ roots.push(root);
+ mkdirSync(join(root, 'node_modules/.cache'), { recursive: true });
+ writeFileSync(join(root, BUILDINFO), 'restored from another head');
+ writeFileSync(join(root, 'tsc'), FAKE_TSC);
+ chmodSync(join(root, 'tsc'), 0o755);
+ writeFileSync(join(root, 'plan'), plan.join('\n') + '\n');
+ writeFileSync(join(root, 'log'), '');
+ writeFileSync(join(root, 'output'), '');
+ const r = Bun.spawnSync(['bash', SCRIPT, join(root, 'tsc')], {
+ cwd: root,
+ env: {
+ PATH: process.env.PATH ?? '/usr/bin:/bin',
+ FIXTURE_PLAN: join(root, 'plan'),
+ FIXTURE_LOG: join(root, 'log'),
+ GITHUB_OUTPUT: join(root, 'output'),
+ GBRAIN_TYPECHECK_ATTEMPT_SECONDS: seconds,
+ },
+ });
+ return {
+ status: r.exitCode,
+ stdout: r.stdout.toString(),
+ attempts: readFileSync(join(root, 'log'), 'utf8').split('\n').filter(Boolean),
+ output: readFileSync(join(root, 'output'), 'utf8').trim(),
+ buildinfoLeft: existsSync(join(root, BUILDINFO)),
+ };
+}
+
+describe('scripts/ci-typecheck.sh', () => {
+ test('a clean incremental run is one attempt and saves its state', () => {
+ const r = run(['0']);
+ expect(r).toMatchObject({ status: 0, attempts: ['warm'], output: 'state_written=true', buildinfoLeft: true });
+ });
+
+ test('a type error (tsc exit 1 or 2) fails at once, with no retry, and saves its state', () => {
+ for (const code of ['1', '2']) {
+ const r = run([code, '0']);
+ expect(r).toMatchObject({ status: Number(code), attempts: ['warm'], output: 'state_written=true', buildinfoLeft: true });
+ expect(r.stdout).not.toContain('typecheck aborted');
+ }
+ });
+
+ test('no exit status from 1 to 128 is retried', () => {
+ for (const code of ['3', '64', '127', '128']) {
+ expect(run([code, '0']).attempts).toEqual(['warm']);
+ }
+ });
+
+ test('an abort (exit 134) deletes the restored buildinfo and runs once more from cold', () => {
+ const r = run(['134', '0']);
+ expect(r).toMatchObject({ status: 0, attempts: ['warm', 'cold'], output: 'state_written=true' });
+ expect(r.stdout).toContain('::warning title=typecheck aborted::');
+ });
+
+ test('a death by signal is an abort too', () => {
+ for (const signal of ['ABRT', 'KILL']) {
+ const r = run([signal, '0']);
+ expect(r).toMatchObject({ status: 0, attempts: ['warm', 'cold'] });
+ }
+ });
+
+ test('a type error on the cold run fails the step, and nothing runs a third time', () => {
+ const r = run(['134', '2', '0']);
+ expect(r).toMatchObject({ status: 2, attempts: ['warm', 'cold'], output: 'state_written=true' });
+ });
+
+ test('a second abort fails the step after two attempts and saves nothing', () => {
+ const r = run(['134', '134', '0']);
+ expect(r).toMatchObject({ status: 134, attempts: ['warm', 'cold'], output: 'state_written=false' });
+ expect(r.stdout).toContain('::error title=typecheck aborted twice::');
+ });
+
+ test('an attempt past its time limit is not retried and saves nothing', () => {
+ const r = run(['hang', '0'], '1');
+ expect(r).toMatchObject({ status: 124, attempts: ['warm'], output: 'state_written=false', buildinfoLeft: true });
+ });
+});