Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript
38.1 KB
Raw Blame

CI event parity

The classification table docs/TESTING.md links to.

A red that only a push to master can produce is a red no PR could have caught. So every event-conditional behavior in the workflows is either parity (pull requests and merge-queue runs do the same work) or a named exception whose comment, beside the condition, names the scheduled run that covers it. A condition with no covering scheduled run defaults to parity; anything else needs a new scheduled run or a decision recorded beside the condition. test/scripts/ci-pr-scope.test.ts fails on an event-conditional job or block in a test workflow without that comment, and on a Bun-version cell excluded on any event. Merge reads the full-suite and e2e-full-suite checks (below), never per-cell matrix names.

Classification

Workflow Condition Class Covered by
test.yml security-regressions Bun matrix Parity: every supported Bun version on every OS, every event, except the macOS cell (below) —
test.yml security-regressions macos-26 cell: left out of pull requests and of a master push that reuses a run Named exception (below) The full-ci dispatch, every master push that runs in full, the merge queue and the nightly Test schedule
test.yml changes native scope (smoke/primary/full) Named exception Every push to master and the nightly Test schedule (full)
test.yml gitleaks commit range by event Parity: each event scans the range it introduces —
test.yml dependency-audit blocking scope Named exception with a rule (below) Every push to master and the nightly Test schedule (always blocking)
test.yml export-scale pages, 10,001 vs 100,001 Named exception Every push to master and the nightly Test schedule
test.yml, stress.yml stress-changed-tests file set Named exception: PR-only gate; succeeds with "not a PR event" elsewhere Push and nightly runs run every file; the nightly race hunt repeats the Postgres arms
test.yml race-hunt Named exception: schedule and dispatch only, outside test-status Itself (nightly Test schedule)
e2e.yml order-hunt Named exception: schedule and order_hunt dispatch only, outside e2e-status Itself (nightly E2E schedule)
test.yml native_only, race_hunt, stress_* dispatch inputs; concurrency suffixes Parity: manual modes and run grouping, never applied to PR, push or schedule events —
persistence-validation.yml Bun matrix of read latency, deployment matrix, unit-lane PostgreSQL arms, soak, crash robot, reconciliation, convergence Parity: every supported Bun version on every event that runs the lane —
test.yml, persistence-validation.yml Cheap tier on pull requests: no unit shards, serial, slow, BrainBench, admin-browser, shared-skills, coverage or test-status; of the persistence lanes only the crash robot and convergence, and those and the native cells only when the diff selects them Named exception (below) The full-ci dispatch on the head before merge (merge-train requires full-suite), every push to master, the merge queue and the nightly Test schedule
test.yml enumeration-census-tests on pull requests that aren't doc-only Named exception (below): pull requests only, an early copy of files the unit shards already run The full-ci dispatch, every push to master, the merge queue and the nightly Test schedule run every one of these files in the unit shards
e2e.yml No pull_request trigger Named exception (below) The full-ci dispatch on the head before merge (merge-train requires e2e-full-suite), every push to master, the merge queue and the nightly E2E schedule
test.yml, e2e.yml A push to master skips its full lanes when it reuses a passed full-suite run of its exact tree Named exception (below) The reused run (same tree, full profile); schedules never reuse
persistence-validation.yml convergence selected on pull requests by scripts/ci-convergence-scope.sh (test.yml changes; never by package.json alone) Named exception Every merge-queue run and every push to master; the nightly cell runs in convergence-nightly.yml
convergence-nightly.yml The convergence gate's nightly cell (2,000 pages, 50 ms, 20 kills, run-id seed): schedule and dispatch only, outside test-status; the PR cell (200 pages, 10 ms, 6 kills, seed 6650) runs in persistence-validation.yml Named exception Itself (nightly Convergence schedule)
persistence-validation.yml SOAK_OPERATIONS 2,500 vs 10,000 Named exception Every push to master and the nightly Test schedule
persistence-validation.yml Crash robot --robot-scope=pr (Postgres lease-bound seams skipped, 820 s PGLite and 620 s Postgres ceilings) vs full (1200 s PGLite, 880 s Postgres; ceilings); the nightly adds that day's exploration seeds Named exception The full-ci dispatch, every push to master (same fixed list, so it may reuse that run) and the nightly Test schedule
native-locks.yml inputs.scope (OpenClaw, musl, Windows probes, cross-target cells) Named exception Every push to master and the nightly Test schedule (full)
native-locks.yml darwin-arm64 and darwin-x64 cells and the darwin-x64 heartbeat finalize: never on a pull request Named exception (below) The full-ci dispatch, every push to master, the merge queue and the nightly Test schedule
e2e.yml Doc-only merge-queue entry selects no E2E file Named exception Every push to master (whole corpus) and the nightly E2E schedule
e2e.yml Full-corpus coverage lanes and their e2e-status checks Named exception Nightly E2E schedule (or a full_corpus dispatch)
e2e.yml Tier 2 (LLM Skills) runs keyed on the nightly only; other events skip it, and e2e-status prints "not run: keys withheld" instead of counting it green Named exception Nightly E2E schedule (or a full_corpus dispatch), keyed; scripts/require-executed-tests.ts fails the job when any test skipped or none ran, and each test fails unless the agent called a gbrain MCP tool
heavy-tests.yml Label-gated jobs on pull requests Named exception Nightly Heavy Tests schedule
heavy-tests.yml grok-door (label or input only) Recorded decision: no scheduled run until the XAI_API_KEY secret exists None yet, by decision
heavy-tests.yml opencode-door-canary Named exception: schedule-only, non-gating canary Itself
macos-validation.yml Label-gated on pull requests Named exception Nightly macOS validation schedule
apple-helper.yml Pull requests and master pushes that touch apple/** or the workflow; not a required check Named exception: the Swift helper is inert files to every other workflow Every push to master that touches those paths
scale-tier.yml Path- or label-gated 10k on pull requests Named exception Nightly Scale tier schedule (10k-50k)
upgrade-matrix.yml Pull requests run N-1 and N-50, keyless repo-less cells only, against the head; the full matrix (N-1, N-10, N-50, 30 days, stable × both engines × keys × brain shapes) is schedule and dispatch only, and those refuse any ref but master (docs) Named exception Nightly Upgrade matrix schedule
semgrep.yml PR diff scan vs full-tree scan Named exception Weekly Semgrep schedule
osv-scanner.yml Pull requests scan only on a manifest change Named exception Weekly OSV schedule; dependency-audit on every PR
nightly-watch.yml workflow_run event guard Not a test scope: it files issues from qualifying scheduled and push-to-master runs —
fix-wave-gate.yml, fix-wave-closeout.yml Pull-request action and merge state Not a test scope: merge-process automation —
harness-check.yml pull_request only; a missing HARNESS_EVALS_READ_TOKEN skips the job (succeeds with a notice); a fork head is refused Named exception: advisory, non-required check (below) None: pushes to master and schedules do not run it; the pin bump runs its steps by hand on master
full-ci.yml full-ci label, same-repository head; the label comes off once both dispatches are accepted Not a test scope: dispatches the full profile on the head —
test.yml, scale-tier.yml dynamic-ci filter (Trunk Dynamic CI) on pull requests; its gated jobs run unless the verdict is exactly 'false' Named exception (below): pull requests only, Learning mode skips nothing The full-ci dispatch, every push to master, the merge queue and the nightly schedules, which never run the filter

Full CI on pull requests

Every pull-request push runs the cheap tier in Test: verify, typecheck, gitleaks, the security regressions, the dependency audit, the stress gate on changed tests, and the GitHub-only lanes the diff selects. The crash robot and convergence run when scripts/ci-convergence-scope.sh selects persistence or sync paths. The native cells run when scripts/ci-native-scope.sh selects more than smoke. No macOS job runs (below). The cheap-tier (full suite not run) check aggregates them (scripts/ci-cheap-tier.sh): a selected lane that skipped fails it, and its summary says the full suite did not run. E2E does not run on pull requests.

The full suite runs on the exact head before merge, with the same shape as a push to master: every unit shard, serial, slow, both PostgreSQL arms and the whole E2E corpus. A maintainer adds the full-ci label (a label needs triage access, so contributors cannot add one). full-ci.yml then dispatches Test and E2E on the head branch and removes the label once both dispatches were accepted, so a later push does not start a full run nobody asked for. A refused or failed dispatch leaves the label on, where it shows. To gate a later head, add the label again at that head. A head that moved after the label was added keeps the label, and the push that moved it dispatches and removes it. A fork's branch is never dispatched. A thread that wants the full suite earlier, for a risky change, dispatches it itself: gh workflow run test.yml --ref <branch> && gh workflow run e2e.yml --ref <branch>.

merge-train requires the full-suite (Test) and e2e-full-suite (E2E) checks to have succeeded on the exact head. Each one succeeds only when its own run's aggregate did (test-status, e2e-status). A pull-request run skips both and names them full-suite-not-run, so a cheap green can never produce either one. Each uploads full-suite-tree / e2e-full-suite-tree, a tree.json recording the tested tree (scripts/ci-full-suite-tree.sh). Pushes to master, the merge queue and schedules always run the full profile.

macOS runners

GitHub's macOS runners are scarce. In busy hours of 2026-11-16 to 11-17 a macOS job waited a median of 4 to 9 minutes for one (90th percentile 14 to 35 minutes), while the tests themselves took 1.5 to 4.3 minutes. So a pull request's cheap tier starts no macOS job: security-regressions runs its Linux and Windows cells, and native-locks runs every target except darwin-arm64 and darwin-x64. The one exception is the path-gated Apple helper workflow (apple-helper.yml), which runs a macos-26 job only when files under apple/ change and is not a required check. A master push that reuses a passed full run of its exact tree (below) also leaves out the macos-26 security-regressions cell, because the reused run already ran it on the same tree. The full-ci dispatch, a master push that runs in full, the merge queue and the nightly Test schedule run every macOS cell, so a macOS-only break shows up at full-ci before landing instead of on the first push. test/scripts/macos-runner-events.test.ts evaluates both workflows' conditions for each event.

Enumeration and census tests

Some unit tests fail whenever a change adds a new operation, command, --flag, error code, doctor check or repair kind without updating the list that tracks it, adds a caller or route that a source census counts, or changes output that a golden file pins. The cheap tier used to skip them, so the first full-ci run on a pull request was often where they failed, and each miss cost another push and another full run.

enumeration-census-tests runs them on every pull-request push whose diff isn't doc-only (the changes job classifies the diff with scripts/select-e2e.ts --classify-only; an unreadable diff runs them). It needs no Postgres. scripts/ci-census-tests.sh prints the file list, and the job and a person checking a branch by hand use the same command:

bash scripts/ci-census-tests.sh                 # print the files
bash scripts/ci-census-tests.sh --run           # run them all (about 10 minutes on 2 cores)
bash scripts/ci-census-tests.sh --run 1 2       # run shard 1 of 2, as CI does

The list holds the source censuses (files marked test-reads-source-ok[structural]), every file that imports test/helpers/golden, the files whose names match the enumeration pattern in the script, the consumer idle-pass statement bounds (test/persistence-consumer-* files that mention idle), and the tests that read every file under src/ or test/, named in the script's WALKERS list (such as test/test-reads-source-smell.test.ts). It leaves out test/e2e/, serial and slow files, and test/setup.test.ts. To add a file, change the script, not the workflow.

The job is two shards on 2-vCPU runners. It is advisory: cheap-tier, full-suite, merge-train, the remote pregate and master-push reuse never read it, and contributor-gate stays the only required check. It is gated by Trunk Dynamic CI like the other cheap-tier jobs (below). Every full run already runs these files in the unit shards, so nothing is lost when it doesn't run.

Master-push reuse

A push to master runs nothing again when its tree equals the tree of a passed, complete full-suite run of the same workflow (scripts/ci-reuse.ts, in Test changes and E2E prepare-e2e). The candidate must meet all of these:

  • It is a workflow_dispatch or push run from this repository.
  • Its latest attempt completed with success.
  • Its single full-suite (or e2e-full-suite) job succeeded in that attempt.
  • Its tree.json matches the run (id, attempt, commit, event, workflow, profile full).
  • The recorded tree equals both the push tree and the tree of the run's head.

The push must be a merge commit or a one-parent commit. A merge commit tries the runs on its merged head first, newest first, then other heads with the same tree. A one-parent commit (a squashed landing, a hand revert, a direct push) has no merged head, so its tree alone decides. The heads with that tree are the heads GitHub names for the commit, each with its first parents that still have the tree (a merge of master that changed no file leaves the full run on the commit before it), and the heads of recent dispatch and push runs. Every run on those heads is read, and the newest one decides, as in the remote gate: if that run failed or is still running, the push runs in full; a cancelled run, or one that skipped its full job, is passed over for the next older one. Anything else, an API error or a partial, cancelled or diagnostic run included, leaves reuse unset, so the push runs in full. A reusing push skips every full lane, the macOS security-regressions cell and both aggregates, records the reused run in its summary, and runs the reused-full-suite job with the reuse.json artifact. The release gate (scripts/release-gate.ts) accepts a reusing push only with a readable reuse.json naming the same workflow, the push head and a valid source run. It prints the reused run per workflow and writes test_reused_run_id / e2e_reused_run_id. Schedules never reuse.

Remote-gate reuse

GitHub's full profile is the source of truth for the Ubicloud gate too. scripts/pregate-remote-gate.sh, the run that clears gstack's remote_gate obligation, first asks bun scripts/ci-reuse.ts --gate whether the clean working tree's commit, or another head with the same tree, already passed the complete full runs of both Test and E2E under the master-push rules above. If both did, the receipt comes from them: the same PREGATE_REMOTE: tree= tests= failed= line, with the runs' executed test identities as the counts, and no VM starts. The counts take each job at its newest attempt, so a run whose failed jobs were re-run counts its whole suite; a ledger that cannot show that is labelled attempt N, partial count on the [ci-reuse] gate: line. A ledger an earlier attempt wrote counts only while its executed-receipts job was not re-run since. A full run on the tree still in progress is waited for (90 minutes at most, a progress line every minute), also while the other workflow has no full run on the tree: full-ci's second dispatch can still appear, and a failure of the run in progress refuses instead of starting VMs. A passed run on the head whose executed-test receipt is refused (an unreadable, stale or empty ledger) ends the wait at once, since no later dispatch of the head is coming; the same refusal on another head with the same tree keeps waiting for the head's own dispatch, and a receipt download that failed transiently (no response, a 5xx, a 429, a rate-limit 403, or a refusal from the artifact storage the download redirects to) is read again on the next poll. Any other 4xx, such as a 403 or an expired artifact's 410, refuses at once. The newest full run on the tree decides, across the head and every recent head with the same tree, so neither an empty commit nor an older pass elsewhere skips a newer failure; a failed one refuses with no receipt. A run that did not run the full job (a diagnostic dispatch, a push that reused) is skipped. A tree that changes .github/workflows/test.yml, .github/workflows/e2e.yml, scripts/ci-reuse.ts, scripts/lib/gh-issue.ts, scripts/ci-full-suite-tree.sh, scripts/ci-executed-counts.ts or scripts/ci-manifest.ts (which writes the run's ci-manifest.json) relative to origin/master never reuses, since its runs would attest to themselves, and the check runs without the model and database keys, like the gate. Every other case runs the gate: a dirty tree, a commit GitHub doesn't have, a missing workflow, an API error, or GBRAIN_PREGATE_REUSE=0.

Only the identical tree reuses. A head that /ship changed after full-ci has a new tree, so it needs its own passed full run or gate. A stamp-only difference reaches the head through gstack's stamp rule (gstack-pregate clear --receipt with the stamp files in .gstack/ship-policy.json), never through this check. Merge still requires full-suite and e2e-full-suite on the exact head.

Run evidence for gstack (gstack-pregate clear --run)

gstack can take the same GitHub runs as evidence itself, so a slot runs one full suite per tree instead of a full-ci pass and a gate (gstack docs/pregate.md#run-evidence). gstack-pregate clear --remote pregate.json/remote_gate --run <Test run> --run <E2E run> --allow-repo-commands reads each run's full-suite-tree / e2e-full-suite-tree artifact and accepts the pair when both passed on pregate.json's tree and their executed files cover the gate's inventory:

  • tree.json carries files (every test file the run executed, repo-relative) and tests (its executed tests). scripts/ci-full-suite-tree.sh takes them from the run's ci-manifest.json, and only when that manifest is complete and names the same run, attempt and commit. Otherwise both are left out with a warning: the run still counts for merge, but gstack refuses it (inventory_uncovered). A push that reused a run has no full-suite job, so it is never evidence.
  • bash scripts/pregate-remote-gate.sh --list prints the inventory: the gate command with --list (scripts/ci-ubicloud.ts --list), the same lanes and discovery the gate runs, one file per line. It exits non-zero when the inventory fails or is empty (a doc-only tree under :diff runs no test file, so it clears through the gate as before). It also prints no inventory for a tree that changes one of the seven files above, or the inventory's own builders scripts/pregate-remote-gate.sh and scripts/ci-ubicloud.ts, relative to origin/master (or when there is no origin/master, or GBRAIN_PREGATE_REUSE=0), so gstack refuses that tree's own runs (inventory_failed) and the gate runs, as remote-gate reuse already does. The gate leaves out the keyed-only Tier 2 file (test/e2e/skills.test.ts): a VM has neither the keys nor openclaw, so every test in it would skip.
  • gstack accepts only push and workflow_dispatch runs whose newest attempt succeeded. A flaky job means re-running the whole workflow. "Re-run failed jobs" carries every job that passed, the ledger job that writes the manifest included, over from the older attempt, so the new attempt has no manifest of its own and its tree.json has no files (inventory_uncovered); a tree.json left from an older attempt is refused too (artifact_mismatch).
  • The profiles live in .gstack/pregate.json: run_profiles test (test.yml, job full-suite, artifact full-suite-tree) and e2e (e2e.yml, job e2e-full-suite, artifact e2e-full-suite-tree), each accepting workflow_dispatch and push runs from the last 24 hours, and remote_gate_runs naming both with this inventory command. gstack reads them only from origin/<base>, so a branch cannot loosen them. gstack accepts these keys from 1.91.377.0 on.

/ship lanes and CI carry

CI's full run is the one full test run. gstack 1.94.0.0 and newer carry a /ship test lane from a passed full-ci run at the exact, clean tree instead of running it again (LANE: <label> carried run=<id>/<attempt> tests_ran=<n>), through ship_lanes_from in .gstack/pregate.json. gstack matches the lane's command byte for byte, so pass these commands to /ship as --lane <label>=<command> exactly as written:

Lane Exact command Run profiles Inventory lanes
unit bun run test test unit,serial
serial bun run test:serial test serial
slow bun run test:slow test slow
e2e bun run test:e2e test, e2e e2e

A lane carries only when every one of these holds:

  • The newest Test (full-suite) run at HEAD passed, and for e2e the newest E2E (e2e-full-suite) run too (Test is named because the reconciliation job runs two test/e2e/ files). gstack refuses a newer red or running run on HEAD, a run older than 24 hours, and a working tree that is not HEAD's committed tree.
  • Every file the lane runs is in the runs' tree.json. The inventory is GBRAIN_PREGATE_GATE_CMD='bun run ci:ubicloud' bash scripts/pregate-remote-gate.sh --list --lanes <lanes>, the same lists the full Ubicloud gate runs. Like the remote gate's own inventory, it refuses (exit 1, no list) on a tree that changes the CI reuse files or the inventory builders, so such a tree's runs never carry its lanes (remote-gate reuse).
  • The e2e inventory leaves out test/e2e/skills.test.ts, which needs provider keys and openclaw. full-ci runs neither, so the file never runs there. Run it by name when your change touches it.

bun run verify and bun run typecheck don't carry: tree.json records test files only, so no inventory can show that CI ran them. They take about a minute locally. pregate's guards, secrets and sealed checks always run.

To check a lane before /ship does, at the pushed head: ~/.claude/skills/gstack/bin/gstack-ci-carry find --base master --command 'bun run test' --allow-repo-commands.

Dependency-audit rule

dependency-audit blocks on pushes, schedules, manual runs and pull requests or merge-queue entries that change a dependency manifest or carry the dependency-audit label; on other pull requests a new upstream advisory is a warning, so one published advisory cannot turn every open PR red at once. The red it produces lands on master instead, and is handled like any other: a push-to-master red opens the Test master-red issue, a scheduled red the nightly-red issue. The repair PR bumps, overrides, patches or removes the dependency. When no fix exists, an ignore entry needs Garry's approval and an expiry date.

Ignore entries live in admin/osv-scanner.toml and apply to admin/bun.lock only; the root lockfile has none. OSV-Scanner reads the file for that lockfile and stops ignoring an entry after its ignoreUntil date, and scripts/ci-dependency-audit.sh passes each id to the admin bun audit until that date, then reports audit_ignore_expired, which blocks like an advisory. Current entry: GHSA-fc5q-39xc-vjpf (picomatch, a build-time dependency of the admin UI's bundler, with no fixed version), until 2026-12-17.

Crash robot ceilings

Every crash-robot run executes one fixed schedule list (scripts/persistence/robot-driver.ts ROBOT_RANDOM_SEEDS), so a master push can reuse a passed run of its tree. --robot-seconds is a ceiling, not a budget: a run past it fails with over_budget instead of covering less. Each cell writes its elapsed time as a share of its ceiling to the job summary, so drift shows long before a run fails.

The Bun 1.4.0 rows in the tables below were measured before the floor moved to Bun 1.4.4 (#7644); that cell no longer runs.

Measured on ubicloud-standard-4, full scope, run 39996381819:

Cell Wall Cross-boundary crashes Process faults 8 random seeds
pglite / Bun 1.4.0 724 s 89 runs, 494 s 67 s 128 s
pglite / Bun 1.4.4 880 s 89 runs, 618 s 72 s 157 s
postgres / Bun 1.4.0 1,290 s 27 runs, 884 s 188 s 190 s
postgres / Bun 1.4.4 967 s 27 runs, 767 s 68 s 100 s
  • PGLite, 1200 s. The slowest cell ran 880 s. The nightly's two exploration seeds add about 45 s, and the slower cells in the previous week's manifests project to about 985 s. 1200 s is that projection plus about 20%.

  • Postgres, 880 s. Before #7454 the slowest cell ran 1,290 s, including one 128 s wait for a dead owner's effect-claim lease, under an 1800 s ceiling. Since #7454 releases that claim early, measured on master with a full-scope dispatch (run 40062601017):

    Cell Wall Cross-boundary crashes Process faults 8 random seeds
    postgres / Bun 1.4.0 391 s 27 runs 4 runs no lease wait
    postgres / Bun 1.4.4 679 s 27 runs 4 runs, one 127 s no lease wait

    The 679 s cell includes one 127 s pooler_disconnect process fault on overlapping_slugs_two_sources (2.4 s in the other cell). 880 s is the slowest cell plus about 30%; the nightly's two exploration seeds (about 15–20 s each on Postgres) fit inside that margin.

  • Two runs at a time (#8036). A full Postgres run went 899 s against 880 s on a pull request that doesn't touch persistence (run 41087538943). Across the 13 full-scope Postgres cells on ubicloud-standard-4 from 2026-10-30, wall time ranged from 379 s to 899 s with the same 54 runs. Two things varied: runner speed (the runs other than the two waiting faults took 342 s to 769 s), and whether pooler_disconnect waited out a 2-minute effect-claim lease (125–132 s in 8 of the 13 cells, 2–8 s in the rest). One run at a time adds both waits to the wall clock in full. Since #8036 the driver runs ROBOT_CONCURRENCY (2) runs at a time. Each run still owns its database, scratch root and home. Every fault is drawn before any crash runs, so the runs, the faults and the manifest order are the same as one at a time (unit-tested, and identical in all four local manifests below). The process faults start first, so their waits overlap the crash runs.

    Measurement One at a time Two at a time
    Local postgres, 4 vCPU, full scope 423 s 258 s (each run about 15% slower)
    Local pglite, 4 vCPU, full scope 699 s 418 s (each run about 20% slower)
    Slowest CI cell (run 41087538943) 899 s measured 528 s predicted

    The prediction replays that cell's 54 measured run times, slowed by 15%, through two lanes. The same model predicts 250 s for the local cell, which measured 258 s. The ceilings are unchanged, so the slowest cell observed now uses about 60% of 880 s.

  • Pull requests, 820 s PGLite and 620 s Postgres. Scope pr crashes each seam once and skips the Postgres lease-bound seams. PGLite measured 510 s and 427 s (run 40003479271), then 529 s and 474 s (run 40018836178); 820 s is the slowest plus about 30%. A Postgres crash run takes about 10 s. Before #7454 it took about 130 s when the killed process held an effect claim, because recovery waited out that claim's 2-minute lease; since #7454 a dead owner's effect claim is released early, so no cell waits on a lease. Measured on master after #7454 (run 40059909892, two attempts):

    Attempt Cell Time
    1 postgres / Bun 1.4.0 357 s
    1 postgres / Bun 1.4.4 367 s
    2 postgres / Bun 1.4.0 475 s
    2 postgres / Bun 1.4.4 362 s

    620 s is the slowest plus about 30%. The cells before #7454 ran 567 to 785 s with one to three lease waits, under a 1020 s ceiling.

Cost of parity

The Bun 1.4.0 cells below were dropped when the floor moved to Bun 1.4.4 (#7644); the figures are the measurements from before that change.

Measured from the 15 latest green pull-request Test runs before parity (job start to finish, PR parameters): the Bun 1.4.0 twins add 14 cells per pull_request run, 55.9 runner-minutes in total: 233 Ubicloud vCPU-minutes (the two unit-lane PostgreSQL arm shards, 19.8 minutes, are the largest share) and 2.4 GitHub-hosted minutes (macOS 0.9, Windows 1.5). At the week's 713 completed PR runs that is about 40,000 runner-minutes a week. Wall time does not grow on an idle pool: on the 10 latest push runs, which already run both versions, the 1.4.0 cells finished between 2.6 minutes before and 2.1 minutes after every other job (median 0.1 minutes before), and the PR critical path stays the unit test shards (median 13.9 minutes). A merge_group run uses the same profile and costs the same per run; the merge queue is inert, so it adds nothing today. If the added median PR wall time exceeds 15 minutes over a week, the agent on release duty records it on the CI health issue.

Harness check

.github/workflows/harness-check.yml runs on every pull request to master from a branch of this repository, on a 4-vCPU Ubicloud runner with a 45-minute limit. It is not a required check. It clones harness-evals at the commit HARNESS_EVALS_PIN names (7e8d872edb, harness-evals v0.0.136, main on 2026-11-10), builds the gbrain sidecar image for the pull request's head commit with harness-evals' scripts/build-gbrain-image.sh (the keyword target: gbrain at that commit run from source, four keyword PGLite brains over the messy-company-v1 corpus, the install bundle from gbrain skillpack scaffold, compile-context and gbrain setup; no key anywhere), and runs the keyless rungs of scripts/gbrain-check.ts against that image with --no-build: the fake harness through harness-evals' real runner and its world's gbrain tap, one trial on the starter surface, one on the verbs surface, one with the gbrain-docs bundle mounted, one on the company brain under scope sources with two members, plus a direct probe of the company brain that calls search, query, recall and list_pages for each member under that member's source_ids (15 members, 479 calls, 3,140 rows over 28 sources in the sample run at this pin). It gates on one number: the count of pages or rows returned to a member whose grant does not hold their source (cross-scope). The job fails when that count is not zero, when the scoped probe did not run, or when no check.json was written (the image did not build or the check did not finish); every other failed assertion of gbrain-check (a tool count off harness-evals' pin, a missing search tool or verb, a retrieval that misses the known dev page, an install bundle or view digest, the Hermes context, the installer's brain-first block or per-turn hook files) is a ::warning:: in the job log and a line in the job summary, so a pull request that changes the tool list or the install output is told and not blocked. The only secret it reads is HARNESS_EVALS_READ_TOKEN, a fine-grained token with Contents: read on the private harness-evals repository; it reads no model key and runs no paid trial (the metering proxy runs with placeholder keys the fake harness never uses). When the secret is not set, the job's first step writes a ::notice:: and a "skipped" job summary naming the missing secret, every later step is skipped, and the job succeeds. A pull request from a fork is refused before that check: the image build fetches the commit from this repository, where a fork's commit does not exist. The artifact harness-check-keyless holds summary.md, keyless/check.json (every assertion and the evidence behind it, including evidence.scoped the gate reads) and trials.jsonl; never a trial's state or a token file.

Measured by hand on 2026-10-26 against master at b06ac32b (0.60.465.0) on a 2-core, 7 GB Linux machine (Docker 29.9.2, bun 1.4.4): image build 359 s, check 59 s (30 assertions passed, 0 failed, 0 pin diffs, cross-scope pages 0), 457 s end to end. A CI runner is not measured yet. The pin moved to v0.0.59 on 2026-10-31: v0.0.9 failed to build its image against master once gbrain setup hermes --instructions-only began writing the shell-hooks allowlist lock; v0.0.59 against the train R head (32f1748a) built and passed (37 assertions, 0 failed, 0 pin diffs, cross-scope pages 0, check 62 s).

Bumping the harness-evals pin. The pin is one line in the workflow, HARNESS_EVALS_PIN, and it must be a commit on harness-evals main. To move it, read the new head with git ls-remote https://github.com/garrytan/harness-evals refs/heads/main, run the workflow's steps once by hand against master with the new pin (clone at the pin, bun install --frozen-lockfile, scripts/build-gbrain-image.sh, scripts/gbrain-check.ts --no-build, then the gate step's shell), and read its last lines (gate passed and the warning count). Then change the line and open the bump as its own pull request, so the first run under the new pin is on a commit whose gbrain did not change. A pin diff reported as a warning after a deliberate gbrain change is fixed on the harness-evals side (its gbrain/pin.json), and the bump that follows picks it up.

Trunk Dynamic CI

Trunk Dynamic CI (private beta; docs) decides per pull-request push which jobs still need to run, from this repository's own CI history. A dynamic-ci job in test.yml and in scale-tier.yml runs trunk-io/dynamic-ci (pinned to v1) with the TRUNK_API_TOKEN repository secret and writes one output per gated job. The step has continue-on-error: true and a 3-minute step timeout inside the job's 5, so a failed or hung Trunk call leaves the outputs empty and the check green instead of red on the head (a lost runner still fails the job).

  • Gated jobs. test.yml: security-regressions (including the Windows and macOS cells), native-locks, persistence-validation, stress-changed-tests and the pull-request-only enumeration-census-tests (above). scale-tier.yml: scale. The first five were chosen from the seven days before 2026-11-04 as the pull-request jobs that take minutes and nearly always pass. changes, gitleaks, dependency-audit, verify, typecheck, semgrep, the scale plan and the aggregates are not gated.
  • Form. A gated job lists dynamic-ci in needs and its if starts !cancelled() && needs.dynamic-ci.outputs.<job key> != 'false' &&, followed by needs.<other>.result == 'success' for every other job it needs and then its original condition. !cancelled() keeps a skipped or failed filter from skipping the job, != 'false' makes an empty output (no token, a Trunk outage, no verdict yet) run it, and the result == 'success' checks keep what the implicit success() did before.
  • Events. The filter's if is github.event_name == 'pull_request'. On a push, merge-queue entry, schedule or dispatch (the full-ci run) it is skipped, every output is empty, and every gated job runs exactly as before (enumeration-census-tests runs only on pull requests anyway), so full-suite, e2e-full-suite and the ci-reuse evidence are unchanged. e2e.yml has no pull-request trigger and no filter.
  • Modes. A repository starts in Learning mode: Trunk records a verdict for every gated job and skips nothing. Enforcement is the Mode dropdown in the repository's row on the Dynamic CI settings page in the Trunk web app (app.trunk.io), not this file; Learning mode can also enforce only for listed engineers or labels (the Allow list column). Under enforcement cheap-tier accepts a gated lane that skipped only when its verdict was exactly 'false' and changes succeeded (scripts/ci-cheap-tier.sh). Before enforcing, weigh three gaps. stress-changed-tests repeats changed tests only on pull requests (no full run repeats them), so a skip there removes that check for the push. A 'false' verdict also skips scale on a pull request that carries the scale-tier label, since Trunk does not read labels. And Trunk learns false skips from its own merge queue; gbrain's backstop is the full-ci dispatch, which skips the filter, so a gated lane that a pull request skipped and full-ci then failed never reaches Trunk as a false skip. Pin a job to run on the Configuration tab when a gap matters.
  • Guard. scripts/check-dynamic-ci.ts (in bun run verify) fails when a filter job can run on another event, sits in a workflow without a pull-request trigger, is not SHA-pinned, lacks continue-on-error: true or a step timeout below the job's, or exports outputs that differ from its job-keys; when a gated job's if lacks the form above or has a top-level || after it; and when a merge or release aggregate is gated.