Garry's Opinionated OpenClaw/Hermes Agent Brain
CI event parity
The classification table docs/TESTING.md links to.
A red that only a push to master can produce is a red no PR could have
caught. So every event-conditional behavior in the workflows is either
parity (pull requests and merge-queue runs do the same work) or a named
exception whose comment, beside the condition, names the scheduled run that
covers it. A condition with no covering scheduled run defaults to parity;
anything else needs a new scheduled run or a decision recorded beside the
condition. test/scripts/ci-pr-scope.test.ts fails on an event-conditional
job or block in a test workflow without that comment, and on a Bun-version
cell excluded on any event. Merge reads the full-suite and e2e-full-suite
checks (below), never per-cell matrix names.
Classification
| Workflow | Condition | Class | Covered by |
|---|---|---|---|
test.yml |
security-regressions Bun matrix |
Parity: every supported Bun version on every OS, every event, except the macOS cell (below) | — |
test.yml |
security-regressions macos-26 cell: left out of pull requests and of a master push that reuses a run |
Named exception (below) | The full-ci dispatch, every master push that runs in full, the merge queue and the nightly Test schedule |
test.yml |
changes native scope (smoke/primary/full) |
Named exception | Every push to master and the nightly Test schedule (full) |
test.yml |
gitleaks commit range by event |
Parity: each event scans the range it introduces | — |
test.yml |
dependency-audit blocking scope |
Named exception with a rule (below) | Every push to master and the nightly Test schedule (always blocking) |
test.yml |
export-scale pages, 10,001 vs 100,001 |
Named exception | Every push to master and the nightly Test schedule |
test.yml, stress.yml |
stress-changed-tests file set |
Named exception: PR-only gate; succeeds with "not a PR event" elsewhere | Push and nightly runs run every file; the nightly race hunt repeats the Postgres arms |
test.yml |
race-hunt |
Named exception: schedule and dispatch only, outside test-status |
Itself (nightly Test schedule) |
e2e.yml |
order-hunt |
Named exception: schedule and order_hunt dispatch only, outside e2e-status |
Itself (nightly E2E schedule) |
test.yml |
native_only, race_hunt, stress_* dispatch inputs; concurrency suffixes |
Parity: manual modes and run grouping, never applied to PR, push or schedule events | — |
persistence-validation.yml |
Bun matrix of read latency, deployment matrix, unit-lane PostgreSQL arms, soak, crash robot, reconciliation, convergence | Parity: every supported Bun version on every event that runs the lane | — |
test.yml, persistence-validation.yml |
Cheap tier on pull requests: no unit shards, serial, slow, BrainBench, admin-browser, shared-skills, coverage or test-status; of the persistence lanes only the crash robot and convergence, and those and the native cells only when the diff selects them |
Named exception (below) | The full-ci dispatch on the head before merge (merge-train requires full-suite), every push to master, the merge queue and the nightly Test schedule |
test.yml |
enumeration-census-tests on pull requests that aren't doc-only |
Named exception (below): pull requests only, an early copy of files the unit shards already run | The full-ci dispatch, every push to master, the merge queue and the nightly Test schedule run every one of these files in the unit shards |
e2e.yml |
No pull_request trigger |
Named exception (below) | The full-ci dispatch on the head before merge (merge-train requires e2e-full-suite), every push to master, the merge queue and the nightly E2E schedule |
test.yml, e2e.yml |
A push to master skips its full lanes when it reuses a passed full-suite run of its exact tree | Named exception (below) | The reused run (same tree, full profile); schedules never reuse |
persistence-validation.yml |
convergence selected on pull requests by scripts/ci-convergence-scope.sh (test.yml changes; never by package.json alone) |
Named exception | Every merge-queue run and every push to master; the nightly cell runs in convergence-nightly.yml |
convergence-nightly.yml |
The convergence gate's nightly cell (2,000 pages, 50 ms, 20 kills, run-id seed): schedule and dispatch only, outside test-status; the PR cell (200 pages, 10 ms, 6 kills, seed 6650) runs in persistence-validation.yml |
Named exception | Itself (nightly Convergence schedule) |
persistence-validation.yml |
SOAK_OPERATIONS 2,500 vs 10,000 |
Named exception | Every push to master and the nightly Test schedule |
persistence-validation.yml |
Crash robot --robot-scope=pr (Postgres lease-bound seams skipped, 820 s PGLite and 620 s Postgres ceilings) vs full (1200 s PGLite, 880 s Postgres; ceilings); the nightly adds that day's exploration seeds |
Named exception | The full-ci dispatch, every push to master (same fixed list, so it may reuse that run) and the nightly Test schedule |
native-locks.yml |
inputs.scope (OpenClaw, musl, Windows probes, cross-target cells) |
Named exception | Every push to master and the nightly Test schedule (full) |
native-locks.yml |
darwin-arm64 and darwin-x64 cells and the darwin-x64 heartbeat finalize: never on a pull request | Named exception (below) | The full-ci dispatch, every push to master, the merge queue and the nightly Test schedule |
e2e.yml |
Doc-only merge-queue entry selects no E2E file | Named exception | Every push to master (whole corpus) and the nightly E2E schedule |
e2e.yml |
Full-corpus coverage lanes and their e2e-status checks |
Named exception | Nightly E2E schedule (or a full_corpus dispatch) |
e2e.yml |
Tier 2 (LLM Skills) runs keyed on the nightly only; other events skip it, and e2e-status prints "not run: keys withheld" instead of counting it green |
Named exception | Nightly E2E schedule (or a full_corpus dispatch), keyed; scripts/require-executed-tests.ts fails the job when any test skipped or none ran, and each test fails unless the agent called a gbrain MCP tool |
heavy-tests.yml |
Label-gated jobs on pull requests | Named exception | Nightly Heavy Tests schedule |
heavy-tests.yml |
grok-door (label or input only) |
Recorded decision: no scheduled run until the XAI_API_KEY secret exists |
None yet, by decision |
heavy-tests.yml |
opencode-door-canary |
Named exception: schedule-only, non-gating canary | Itself |
macos-validation.yml |
Label-gated on pull requests | Named exception | Nightly macOS validation schedule |
apple-helper.yml |
Pull requests and master pushes that touch apple/** or the workflow; not a required check |
Named exception: the Swift helper is inert files to every other workflow | Every push to master that touches those paths |
scale-tier.yml |
Path- or label-gated 10k on pull requests | Named exception | Nightly Scale tier schedule (10k-50k) |
upgrade-matrix.yml |
Pull requests run N-1 and N-50, keyless repo-less cells only, against the head; the full matrix (N-1, N-10, N-50, 30 days, stable × both engines × keys × brain shapes) is schedule and dispatch only, and those refuse any ref but master (docs) | Named exception | Nightly Upgrade matrix schedule |
semgrep.yml |
PR diff scan vs full-tree scan | Named exception | Weekly Semgrep schedule |
osv-scanner.yml |
Pull requests scan only on a manifest change | Named exception | Weekly OSV schedule; dependency-audit on every PR |
nightly-watch.yml |
workflow_run event guard |
Not a test scope: it files issues from qualifying scheduled and push-to-master runs | — |
fix-wave-gate.yml, fix-wave-closeout.yml |
Pull-request action and merge state | Not a test scope: merge-process automation | — |
harness-check.yml |
pull_request only; a missing HARNESS_EVALS_READ_TOKEN skips the job (succeeds with a notice); a fork head is refused |
Named exception: advisory, non-required check (below) | None: pushes to master and schedules do not run it; the pin bump runs its steps by hand on master |
full-ci.yml |
full-ci label, same-repository head; the label comes off once both dispatches are accepted |
Not a test scope: dispatches the full profile on the head | — |
test.yml, scale-tier.yml |
dynamic-ci filter (Trunk Dynamic CI) on pull requests; its gated jobs run unless the verdict is exactly 'false' |
Named exception (below): pull requests only, Learning mode skips nothing | The full-ci dispatch, every push to master, the merge queue and the nightly schedules, which never run the filter |
Full CI on pull requests
Every pull-request push runs the cheap tier in Test: verify, typecheck,
gitleaks, the security regressions, the dependency audit, the stress gate
on changed tests, and the GitHub-only lanes the diff selects. The crash robot
and convergence run when scripts/ci-convergence-scope.sh selects
persistence or sync paths. The native cells run when
scripts/ci-native-scope.sh selects more than smoke. No macOS job runs
(below). The
cheap-tier (full suite not run) check aggregates them
(scripts/ci-cheap-tier.sh): a selected lane that skipped fails it, and its
summary says the full suite did not run. E2E does not run on pull requests.
The full suite runs on the exact head before merge, with the same shape as a
push to master: every unit shard, serial, slow, both PostgreSQL arms and the
whole E2E corpus. A maintainer adds the full-ci label (a label needs triage
access, so contributors cannot add one). full-ci.yml then dispatches Test
and E2E on the head branch and removes the label once both dispatches were
accepted, so a later push does not start a full run nobody asked for. A refused
or failed dispatch leaves the label on, where it shows. To gate a later head, add
the label again at that head. A head that moved after the label was added keeps
the label, and the push that moved it dispatches and removes it. A fork's branch
is never dispatched. A thread that wants the full suite
earlier, for a risky change, dispatches it itself:
gh workflow run test.yml --ref <branch> && gh workflow run e2e.yml --ref <branch>.
merge-train requires the full-suite (Test) and e2e-full-suite (E2E)
checks to have succeeded on the exact head. Each one succeeds only when its
own run's aggregate did (test-status, e2e-status). A pull-request run
skips both and names them full-suite-not-run, so a cheap green can never
produce either one. Each uploads full-suite-tree / e2e-full-suite-tree,
a tree.json recording the tested tree (scripts/ci-full-suite-tree.sh).
Pushes to master, the merge queue and schedules always run the full profile.
macOS runners
GitHub's macOS runners are scarce. In busy hours of 2026-11-16 to 11-17 a
macOS job waited a median of 4 to 9 minutes for one (90th percentile 14 to 35
minutes), while the tests themselves took 1.5 to 4.3 minutes. So a
pull request's cheap tier starts no macOS job: security-regressions runs its
Linux and Windows cells, and native-locks runs every target except
darwin-arm64 and darwin-x64. The one exception is the path-gated Apple helper
workflow (apple-helper.yml), which runs a macos-26 job only when files under
apple/ change and is not a required check. A master push that reuses a passed full run of
its exact tree (below) also leaves out the macos-26
security-regressions cell, because the reused run already ran it on the
same tree. The full-ci dispatch, a master push that runs in full, the merge
queue and the nightly Test schedule run every macOS cell, so a macOS-only
break shows up at full-ci before landing instead of on the first push.
test/scripts/macos-runner-events.test.ts evaluates both workflows'
conditions for each event.
Enumeration and census tests
Some unit tests fail whenever a change adds a new operation, command, --flag,
error code, doctor check or repair kind without updating the list that tracks
it, adds a caller or route that a source census counts, or changes output that
a golden file pins. The cheap tier used to skip them, so
the first full-ci run on a pull request was often where they failed, and each
miss cost another push and another full run.
enumeration-census-tests runs them on every pull-request push whose diff
isn't doc-only (the changes job classifies the diff with
scripts/select-e2e.ts --classify-only; an unreadable diff runs them). It needs
no Postgres. scripts/ci-census-tests.sh prints the file list, and the job and
a person checking a branch by hand use the same command:
bash scripts/ci-census-tests.sh # print the files
bash scripts/ci-census-tests.sh --run # run them all (about 10 minutes on 2 cores)
bash scripts/ci-census-tests.sh --run 1 2 # run shard 1 of 2, as CI does
The list holds the source censuses (files marked
test-reads-source-ok[structural]), every file that imports
test/helpers/golden, the files whose names match the enumeration pattern in
the script, the consumer idle-pass statement bounds
(test/persistence-consumer-* files that mention idle), and the tests that read
every file under src/ or test/, named in the script's WALKERS list (such
as test/test-reads-source-smell.test.ts). It leaves out
test/e2e/, serial and slow files, and test/setup.test.ts. To add a file,
change the script, not the workflow.
The job is two shards on 2-vCPU runners. It is advisory: cheap-tier,
full-suite, merge-train, the remote pregate and master-push reuse never read
it, and contributor-gate stays the only required check. It is gated by Trunk
Dynamic CI like the other cheap-tier jobs (below). Every
full run already runs these files in the unit shards, so nothing is lost when
it doesn't run.
Master-push reuse
A push to master runs nothing again when its tree equals the tree of a passed,
complete full-suite run of the same workflow (scripts/ci-reuse.ts, in Test
changes and E2E prepare-e2e). The candidate must meet all of these:
- It is a
workflow_dispatchorpushrun from this repository. - Its latest attempt completed with success.
- Its single
full-suite(ore2e-full-suite) job succeeded in that attempt. - Its
tree.jsonmatches the run (id, attempt, commit, event, workflow, profilefull). - The recorded tree equals both the push tree and the tree of the run's head.
The push must be a merge commit or a one-parent commit. A merge commit tries the
runs on its merged head first, newest first, then other heads with the same
tree. A one-parent commit (a squashed landing, a hand revert, a direct push) has
no merged head, so its tree alone decides. The heads with that tree are the
heads GitHub names for the commit, each with its first parents that still have
the tree (a merge of master that changed no file leaves the full run on the
commit before it), and the heads of recent dispatch and push runs. Every run on
those heads is read, and the newest one decides, as in the
remote gate: if that run failed or is still running, the
push runs in full; a cancelled run, or one that skipped its full job, is passed
over for the next older one. Anything else, an API error or a
partial, cancelled or diagnostic run included, leaves reuse unset, so the push
runs in full. A reusing push skips every full lane, the macOS security-regressions cell and both aggregates, records the
reused run in its summary, and runs the reused-full-suite job with the
reuse.json artifact. The release gate (scripts/release-gate.ts) accepts a
reusing push only with a readable reuse.json naming the same workflow, the
push head and a valid source run. It prints the reused run per workflow and
writes test_reused_run_id / e2e_reused_run_id. Schedules never reuse.
Remote-gate reuse
GitHub's full profile is the source of truth for the Ubicloud gate too.
scripts/pregate-remote-gate.sh, the run that clears gstack's remote_gate
obligation, first asks bun scripts/ci-reuse.ts --gate whether the clean
working tree's commit, or another head with the same tree, already passed the
complete full runs of both Test and E2E under the master-push rules above. If
both did, the receipt comes from them: the same PREGATE_REMOTE: tree= tests= failed= line, with the runs' executed test identities as the counts, and no VM
starts. The counts take each job at its newest attempt, so a run whose failed
jobs were re-run counts its whole suite; a ledger that cannot show that is
labelled attempt N, partial count on the [ci-reuse] gate: line. A ledger an
earlier attempt wrote counts only while its executed-receipts job was not re-run
since. A full run
on the tree still in progress is waited for (90 minutes at most, a progress
line every minute), also while the other workflow has no full run on the tree:
full-ci's second dispatch can still appear, and a failure of the run in
progress refuses instead of starting VMs. A passed run on the head whose
executed-test receipt is refused (an unreadable, stale or empty ledger) ends the
wait at once, since no later dispatch of the head is coming; the same refusal on
another head with the same tree keeps waiting for the head's own dispatch, and a
receipt download that failed transiently (no response, a 5xx, a 429, a rate-limit
403, or a refusal from the artifact storage the download redirects to) is read
again on the next poll. Any other 4xx, such as a 403 or an expired artifact's
410, refuses at once. The newest full run on the tree decides,
across the head and every recent head with the same tree, so neither an empty
commit nor an older pass elsewhere skips a newer failure; a failed one refuses
with no receipt. A run that did not run the full job (a diagnostic dispatch, a
push that reused) is skipped. A tree that changes .github/workflows/test.yml,
.github/workflows/e2e.yml, scripts/ci-reuse.ts, scripts/lib/gh-issue.ts,
scripts/ci-full-suite-tree.sh, scripts/ci-executed-counts.ts or scripts/ci-manifest.ts (which writes the run's ci-manifest.json) relative to origin/master never reuses, since its runs would attest to themselves, and the
check runs without the model and database keys, like the gate. Every other case runs the gate: a dirty tree, a commit GitHub
doesn't have, a missing workflow, an API error, or GBRAIN_PREGATE_REUSE=0.
Only the identical tree reuses. A head that /ship changed after full-ci has a
new tree, so it needs its own passed full run or gate. A stamp-only difference
reaches the head through gstack's stamp rule (gstack-pregate clear --receipt
with the stamp files in .gstack/ship-policy.json), never through this check.
Merge still requires full-suite and e2e-full-suite on the exact head.
Run evidence for gstack (gstack-pregate clear --run)
gstack can take the same GitHub runs as evidence itself, so a slot runs one full
suite per tree instead of a full-ci pass and a gate
(gstack docs/pregate.md#run-evidence).
gstack-pregate clear --remote pregate.json/remote_gate --run <Test run> --run <E2E run> --allow-repo-commands
reads each run's full-suite-tree / e2e-full-suite-tree artifact and accepts the pair
when both passed on pregate.json's tree and their executed files cover the gate's
inventory:
tree.jsoncarriesfiles(every test file the run executed, repo-relative) andtests(its executed tests).scripts/ci-full-suite-tree.shtakes them from the run'sci-manifest.json, and only when that manifest is complete and names the same run, attempt and commit. Otherwise both are left out with a warning: the run still counts for merge, but gstack refuses it (inventory_uncovered). A push that reused a run has nofull-suitejob, so it is never evidence.bash scripts/pregate-remote-gate.sh --listprints the inventory: the gate command with--list(scripts/ci-ubicloud.ts --list), the same lanes and discovery the gate runs, one file per line. It exits non-zero when the inventory fails or is empty (a doc-only tree under:diffruns no test file, so it clears through the gate as before). It also prints no inventory for a tree that changes one of the seven files above, or the inventory's own buildersscripts/pregate-remote-gate.shandscripts/ci-ubicloud.ts, relative toorigin/master(or when there is noorigin/master, orGBRAIN_PREGATE_REUSE=0), so gstack refuses that tree's own runs (inventory_failed) and the gate runs, as remote-gate reuse already does. The gate leaves out the keyed-only Tier 2 file (test/e2e/skills.test.ts): a VM has neither the keys nor openclaw, so every test in it would skip.- gstack accepts only
pushandworkflow_dispatchruns whose newest attempt succeeded. A flaky job means re-running the whole workflow. "Re-run failed jobs" carries every job that passed, the ledger job that writes the manifest included, over from the older attempt, so the new attempt has no manifest of its own and itstree.jsonhas nofiles(inventory_uncovered); atree.jsonleft from an older attempt is refused too (artifact_mismatch). - The profiles live in
.gstack/pregate.json:run_profilestest(test.yml, jobfull-suite, artifactfull-suite-tree) ande2e(e2e.yml, jobe2e-full-suite, artifacte2e-full-suite-tree), each acceptingworkflow_dispatchandpushruns from the last 24 hours, andremote_gate_runsnaming both with this inventory command. gstack reads them only fromorigin/<base>, so a branch cannot loosen them. gstack accepts these keys from 1.91.377.0 on.
/ship lanes and CI carry
CI's full run is the one full test run. gstack 1.94.0.0 and newer carry a /ship
test lane from a passed full-ci run at the exact, clean tree instead of running
it again (LANE: <label> carried run=<id>/<attempt> tests_ran=<n>), through
ship_lanes_from in .gstack/pregate.json. gstack matches the lane's command
byte for byte, so pass these commands to /ship as --lane <label>=<command>
exactly as written:
| Lane | Exact command | Run profiles | Inventory lanes |
|---|---|---|---|
unit |
bun run test |
test |
unit,serial |
serial |
bun run test:serial |
test |
serial |
slow |
bun run test:slow |
test |
slow |
e2e |
bun run test:e2e |
test, e2e |
e2e |
A lane carries only when every one of these holds:
- The newest Test (
full-suite) run at HEAD passed, and fore2ethe newest E2E (e2e-full-suite) run too (Test is named because the reconciliation job runs twotest/e2e/files). gstack refuses a newer red or running run on HEAD, a run older than 24 hours, and a working tree that is not HEAD's committed tree. - Every file the lane runs is in the runs'
tree.json. The inventory isGBRAIN_PREGATE_GATE_CMD='bun run ci:ubicloud' bash scripts/pregate-remote-gate.sh --list --lanes <lanes>, the same lists the full Ubicloud gate runs. Like the remote gate's own inventory, it refuses (exit 1, no list) on a tree that changes the CI reuse files or the inventory builders, so such a tree's runs never carry its lanes (remote-gate reuse). - The
e2einventory leaves outtest/e2e/skills.test.ts, which needs provider keys and openclaw. full-ci runs neither, so the file never runs there. Run it by name when your change touches it.
bun run verify and bun run typecheck don't carry: tree.json records test
files only, so no inventory can show that CI ran them. They take about a minute
locally. pregate's guards, secrets and sealed checks always run.
To check a lane before /ship does, at the pushed head:
~/.claude/skills/gstack/bin/gstack-ci-carry find --base master --command 'bun run test' --allow-repo-commands.
Dependency-audit rule
dependency-audit blocks on pushes, schedules, manual runs and pull requests
or merge-queue entries that change a dependency manifest or carry the
dependency-audit label; on other pull requests a new upstream advisory is a
warning, so one published advisory cannot turn every open PR red at once.
The red it produces lands on master instead, and is handled like any other:
a push-to-master red opens the Test master-red issue, a scheduled red the
nightly-red issue. The repair PR bumps, overrides, patches or removes the
dependency. When no fix exists, an ignore entry needs Garry's approval and an
expiry date.
Ignore entries live in admin/osv-scanner.toml and apply to admin/bun.lock
only; the root lockfile has none. OSV-Scanner reads the file for that lockfile
and stops ignoring an entry after its ignoreUntil date, and
scripts/ci-dependency-audit.sh passes each id to the admin bun audit until
that date, then reports audit_ignore_expired, which blocks like an advisory.
Current entry: GHSA-fc5q-39xc-vjpf (picomatch, a build-time dependency of the
admin UI's bundler, with no fixed version), until 2026-12-17.
Crash robot ceilings
Every crash-robot run executes one fixed schedule list (scripts/persistence/robot-driver.ts
ROBOT_RANDOM_SEEDS), so a master push can reuse a passed run of its tree. --robot-seconds
is a ceiling, not a budget: a run past it fails with over_budget instead of covering less.
Each cell writes its elapsed time as a share of its ceiling to the job summary, so drift
shows long before a run fails.
The Bun 1.4.0 rows in the tables below were measured before the floor moved to Bun 1.4.4 (#7644); that cell no longer runs.
Measured on ubicloud-standard-4, full scope, run 39996381819:
| Cell | Wall | Cross-boundary crashes | Process faults | 8 random seeds |
|---|---|---|---|---|
| pglite / Bun 1.4.0 | 724 s | 89 runs, 494 s | 67 s | 128 s |
| pglite / Bun 1.4.4 | 880 s | 89 runs, 618 s | 72 s | 157 s |
| postgres / Bun 1.4.0 | 1,290 s | 27 runs, 884 s | 188 s | 190 s |
| postgres / Bun 1.4.4 | 967 s | 27 runs, 767 s | 68 s | 100 s |
PGLite, 1200 s. The slowest cell ran 880 s. The nightly's two exploration seeds add about 45 s, and the slower cells in the previous week's manifests project to about 985 s. 1200 s is that projection plus about 20%.
Postgres, 880 s. Before #7454 the slowest cell ran 1,290 s, including one 128 s wait for a dead owner's effect-claim lease, under an 1800 s ceiling. Since #7454 releases that claim early, measured on master with a full-scope dispatch (run 40062601017):
Cell Wall Cross-boundary crashes Process faults 8 random seeds postgres / Bun 1.4.0 391 s 27 runs 4 runs no lease wait postgres / Bun 1.4.4 679 s 27 runs 4 runs, one 127 s no lease wait The 679 s cell includes one 127 s
pooler_disconnectprocess fault onoverlapping_slugs_two_sources(2.4 s in the other cell). 880 s is the slowest cell plus about 30%; the nightly's two exploration seeds (about 15–20 s each on Postgres) fit inside that margin.Two runs at a time (#8036). A full Postgres run went 899 s against 880 s on a pull request that doesn't touch persistence (run 41087538943). Across the 13 full-scope Postgres cells on ubicloud-standard-4 from 2026-10-30, wall time ranged from 379 s to 899 s with the same 54 runs. Two things varied: runner speed (the runs other than the two waiting faults took 342 s to 769 s), and whether
pooler_disconnectwaited out a 2-minute effect-claim lease (125–132 s in 8 of the 13 cells, 2–8 s in the rest). One run at a time adds both waits to the wall clock in full. Since #8036 the driver runsROBOT_CONCURRENCY(2) runs at a time. Each run still owns its database, scratch root and home. Every fault is drawn before any crash runs, so the runs, the faults and the manifest order are the same as one at a time (unit-tested, and identical in all four local manifests below). The process faults start first, so their waits overlap the crash runs.Measurement One at a time Two at a time Local postgres, 4 vCPU, full scope 423 s 258 s (each run about 15% slower) Local pglite, 4 vCPU, full scope 699 s 418 s (each run about 20% slower) Slowest CI cell (run 41087538943) 899 s measured 528 s predicted The prediction replays that cell's 54 measured run times, slowed by 15%, through two lanes. The same model predicts 250 s for the local cell, which measured 258 s. The ceilings are unchanged, so the slowest cell observed now uses about 60% of 880 s.
Pull requests, 820 s PGLite and 620 s Postgres. Scope
prcrashes each seam once and skips the Postgres lease-bound seams. PGLite measured 510 s and 427 s (run 40003479271), then 529 s and 474 s (run 40018836178); 820 s is the slowest plus about 30%. A Postgres crash run takes about 10 s. Before #7454 it took about 130 s when the killed process held an effect claim, because recovery waited out that claim's 2-minute lease; since #7454 a dead owner's effect claim is released early, so no cell waits on a lease. Measured on master after #7454 (run 40059909892, two attempts):Attempt Cell Time 1 postgres / Bun 1.4.0 357 s 1 postgres / Bun 1.4.4 367 s 2 postgres / Bun 1.4.0 475 s 2 postgres / Bun 1.4.4 362 s 620 s is the slowest plus about 30%. The cells before #7454 ran 567 to 785 s with one to three lease waits, under a 1020 s ceiling.
Cost of parity
The Bun 1.4.0 cells below were dropped when the floor moved to Bun 1.4.4 (#7644); the figures are the measurements from before that change.
Measured from the 15 latest green pull-request Test runs before parity (job
start to finish, PR parameters): the Bun 1.4.0 twins add 14 cells per
pull_request run, 55.9 runner-minutes in total: 233 Ubicloud vCPU-minutes
(the two unit-lane PostgreSQL arm shards, 19.8 minutes, are the largest share)
and 2.4 GitHub-hosted minutes (macOS 0.9, Windows 1.5). At the week's 713
completed PR runs that is about 40,000 runner-minutes a week. Wall time does
not grow on an idle pool: on the 10 latest push runs, which already run both
versions, the 1.4.0 cells finished between 2.6 minutes before and 2.1 minutes
after every other job (median 0.1 minutes before), and the PR critical path
stays the unit test shards (median 13.9 minutes). A merge_group run uses
the same profile and costs the same per run; the merge queue is inert, so it
adds nothing today. If the added median PR wall time exceeds 15 minutes over
a week, the agent on release duty records it on the CI health issue.
Harness check
.github/workflows/harness-check.yml runs on every pull request to master from a branch of this repository, on a 4-vCPU Ubicloud runner with a 45-minute limit. It is not a required check. It clones harness-evals at the commit HARNESS_EVALS_PIN names (7e8d872edb, harness-evals v0.0.136, main on 2026-11-10), builds the gbrain sidecar image for the pull request's head commit with harness-evals' scripts/build-gbrain-image.sh (the keyword target: gbrain at that commit run from source, four keyword PGLite brains over the messy-company-v1 corpus, the install bundle from gbrain skillpack scaffold, compile-context and gbrain setup; no key anywhere), and runs the keyless rungs of scripts/gbrain-check.ts against that image with --no-build: the fake harness through harness-evals' real runner and its world's gbrain tap, one trial on the starter surface, one on the verbs surface, one with the gbrain-docs bundle mounted, one on the company brain under scope sources with two members, plus a direct probe of the company brain that calls search, query, recall and list_pages for each member under that member's source_ids (15 members, 479 calls, 3,140 rows over 28 sources in the sample run at this pin). It gates on one number: the count of pages or rows returned to a member whose grant does not hold their source (cross-scope). The job fails when that count is not zero, when the scoped probe did not run, or when no check.json was written (the image did not build or the check did not finish); every other failed assertion of gbrain-check (a tool count off harness-evals' pin, a missing search tool or verb, a retrieval that misses the known dev page, an install bundle or view digest, the Hermes context, the installer's brain-first block or per-turn hook files) is a ::warning:: in the job log and a line in the job summary, so a pull request that changes the tool list or the install output is told and not blocked. The only secret it reads is HARNESS_EVALS_READ_TOKEN, a fine-grained token with Contents: read on the private harness-evals repository; it reads no model key and runs no paid trial (the metering proxy runs with placeholder keys the fake harness never uses). When the secret is not set, the job's first step writes a ::notice:: and a "skipped" job summary naming the missing secret, every later step is skipped, and the job succeeds. A pull request from a fork is refused before that check: the image build fetches the commit from this repository, where a fork's commit does not exist. The artifact harness-check-keyless holds summary.md, keyless/check.json (every assertion and the evidence behind it, including evidence.scoped the gate reads) and trials.jsonl; never a trial's state or a token file.
Measured by hand on 2026-10-26 against master at b06ac32b (0.60.465.0) on a 2-core, 7 GB Linux machine (Docker 29.9.2, bun 1.4.4): image build 359 s, check 59 s (30 assertions passed, 0 failed, 0 pin diffs, cross-scope pages 0), 457 s end to end. A CI runner is not measured yet. The pin moved to v0.0.59 on 2026-10-31: v0.0.9 failed to build its image against master once gbrain setup hermes --instructions-only began writing the shell-hooks allowlist lock; v0.0.59 against the train R head (32f1748a) built and passed (37 assertions, 0 failed, 0 pin diffs, cross-scope pages 0, check 62 s).
Bumping the harness-evals pin. The pin is one line in the workflow, HARNESS_EVALS_PIN, and it must be a commit on harness-evals main. To move it, read the new head with git ls-remote https://github.com/garrytan/harness-evals refs/heads/main, run the workflow's steps once by hand against master with the new pin (clone at the pin, bun install --frozen-lockfile, scripts/build-gbrain-image.sh, scripts/gbrain-check.ts --no-build, then the gate step's shell), and read its last lines (gate passed and the warning count). Then change the line and open the bump as its own pull request, so the first run under the new pin is on a commit whose gbrain did not change. A pin diff reported as a warning after a deliberate gbrain change is fixed on the harness-evals side (its gbrain/pin.json), and the bump that follows picks it up.
Trunk Dynamic CI
Trunk Dynamic CI (private beta; docs) decides per pull-request push which jobs still need to run, from this repository's own CI history. A dynamic-ci job in test.yml and in scale-tier.yml runs trunk-io/dynamic-ci (pinned to v1) with the TRUNK_API_TOKEN repository secret and writes one output per gated job. The step has continue-on-error: true and a 3-minute step timeout inside the job's 5, so a failed or hung Trunk call leaves the outputs empty and the check green instead of red on the head (a lost runner still fails the job).
- Gated jobs.
test.yml:security-regressions(including the Windows and macOS cells),native-locks,persistence-validation,stress-changed-testsand the pull-request-onlyenumeration-census-tests(above).scale-tier.yml:scale. The first five were chosen from the seven days before 2026-11-04 as the pull-request jobs that take minutes and nearly always pass.changes,gitleaks,dependency-audit,verify,typecheck,semgrep, the scaleplanand the aggregates are not gated. - Form. A gated job lists
dynamic-ciinneedsand itsifstarts!cancelled() && needs.dynamic-ci.outputs.<job key> != 'false' &&, followed byneeds.<other>.result == 'success'for every other job it needs and then its original condition.!cancelled()keeps a skipped or failed filter from skipping the job,!= 'false'makes an empty output (no token, a Trunk outage, no verdict yet) run it, and theresult == 'success'checks keep what the implicitsuccess()did before. - Events. The filter's
ifisgithub.event_name == 'pull_request'. On a push, merge-queue entry, schedule or dispatch (the full-ci run) it is skipped, every output is empty, and every gated job runs exactly as before (enumeration-census-testsruns only on pull requests anyway), sofull-suite,e2e-full-suiteand the ci-reuse evidence are unchanged.e2e.ymlhas no pull-request trigger and no filter. - Modes. A repository starts in Learning mode: Trunk records a verdict for every gated job and skips nothing. Enforcement is the Mode dropdown in the repository's row on the Dynamic CI settings page in the Trunk web app (app.trunk.io), not this file; Learning mode can also enforce only for listed engineers or labels (the Allow list column). Under enforcement
cheap-tieraccepts a gated lane that skipped only when its verdict was exactly'false'andchangessucceeded (scripts/ci-cheap-tier.sh). Before enforcing, weigh three gaps.stress-changed-testsrepeats changed tests only on pull requests (no full run repeats them), so a skip there removes that check for the push. A'false'verdict also skipsscaleon a pull request that carries thescale-tierlabel, since Trunk does not read labels. And Trunk learns false skips from its own merge queue; gbrain's backstop is the full-ci dispatch, which skips the filter, so a gated lane that a pull request skipped and full-ci then failed never reaches Trunk as a false skip. Pin a job to run on the Configuration tab when a gap matters. - Guard.
scripts/check-dynamic-ci.ts(inbun run verify) fails when a filter job can run on another event, sits in a workflow without a pull-request trigger, is not SHA-pinned, lackscontinue-on-error: trueor a step timeout below the job's, or exports outputs that differ from itsjob-keys; when a gated job'siflacks the form above or has a top-level||after it; and when a merge or release aggregate is gated.