Skip to content

The ultimate harness for coding agents and beyond. All your agents. All your machines. One command center. Start with code, then follow your curiosity and build across disciplines: CAD, circuits, robots, games and music.

ci: transition workflow to on-demand execution and fix runner test flakiness

0xken15fa58fc220733
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 98e2a9509..0fbb6611a 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -1,32 +1,19 @@
name: CI
-# Modelled on autonomous-harness-desktop's ci.yml, with ONE deliberate difference: that one has no
-# `pull_request` trigger and this one does.
+# Modelled on autonomous-harness-desktop's ci.yml. Like that one, this runs ON DEMAND only.
#
-# The desktop repo is private, and everything that reaches its main branch was pushed by a
-# maintainer who could run the suite locally; a per-commit run there is the same verdict paid for
-# twice. THIS repo is public, takes fork PRs (CONTRIBUTING.md walks through it), and until this file
-# existed said so out loud: "there is no CI on pull requests yet. Nothing runs automatically when
-# you push, so what you report is what a reviewer starts from." A contributor's own report is not a
-# verdict — it is a claim about a machine nobody else can see — and a maintainer pulling every
-# branch to find out is the manual step CI exists to delete.
+# It used to run on every pull request as well (2026-09-10 → 2026-09-16). That was switched off
+# deliberately: the suite is timing-sensitive on the shared ubuntu runner (real tmux server, shell
+# scripts, sqlite3 polls), so a red check on a PR said more about the runner than about the change,
+# and every contributor had to read the log to find out which. The verdict on a branch is still one
+# click away — Actions -> CI -> Run workflow, pick the branch — which is what a reviewer does when
+# they want it. If per-PR CI comes back, use `pull_request`, never `pull_request_target`: a fork's PR
+# must run with a read-only token and no secrets, since this job executes the contributor's code.
#
-# `pull_request`, never `pull_request_target`: a fork's PR runs with a read-only token and no access
-# to secrets, and this job wants neither. It checks out and runs the contributor's code, which is
-# exactly why it must not be handed a writable token.
-#
-# There is still no `push` trigger. Nobody pushes to main directly (CONTRIBUTING.md step 2) and
-# merges are squashed, so a push run would re-test the commit the PR run just cleared.
+# There is no `push` trigger either. Nobody pushes to main directly (CONTRIBUTING.md step 2) and
+# merges are squashed, so a push run would re-test a commit somebody already ran by hand.
on:
- pull_request:
- branches: [main]
- # Deliberately unfiltered by `paths`. A docs-only PR running the CLI suite is 90 seconds of noise;
- # a `paths` filter that makes the check never REPORT is a PR nobody can merge the day somebody
- # turns on branch protection. The cheap failure is the better one, and public repos do not pay for
- # standard runners.
- #
- # It is also the escape hatch for a verdict on any branch without opening a PR:
- # Actions -> CI -> Run workflow.
+ # The only way this runs on its own: Actions -> CI -> Run workflow, on any branch.
workflow_dispatch:
# Callable as a reusable workflow. release.yml used to gate on this and take the Node pin from
# its output; it no longer does (see the note above `publish` there) and pins the same version
@@ -42,10 +29,9 @@ permissions:
concurrency:
group: cli-ci-${{ github.workflow }}-${{ github.ref }}
- # A second push to an open PR makes the first run's verdict worthless, so that one is cancelled.
- # Everything else — a release's gate, a run somebody started by hand — is the record of whether a
- # commit was green, and is never killed to make room.
- cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+ # Every run here was started by a person or by a release, and is the record of whether that commit
+ # was green — never killed to make room for a newer one.
+ cancel-in-progress: false
jobs:
# `make cli-test`, exactly: typecheck then the vitest suite. Nothing else — the repo has no lint
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 2bf945862..7553b4558 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -44,13 +44,15 @@ not code at all.
you ran them against** — "`npm test` plus `test:tmux-real` on tmux 3.5a, macOS" is worth more than
a green checkbox.
- That last part is not politeness. CI now runs `npm run typecheck` and `npm test` on every pull
- request (Ubuntu, on the same pinned Node the released bundle is built with), so a green check
- means those two passed — and **nothing more**. The suites that need real software are gated
- behind `RUN_*` env vars precisely because a hosted runner has no tmux server, no Herdr and no
- Cursor, so `test:tmux-real`, `test:herdr-real` and `test:cursor-e2e` are still only ever run by
- you and by the maintainer reviewing you. For an engine or a multiplexer, that is the half of the
- verdict that matters, and it starts from what you report.
+ That last part is not politeness. Nothing runs automatically on a pull request: the CI workflow
+ (`npm run typecheck` and `npm test` on Ubuntu, on the same pinned Node the released bundle is
+ built with) is started by hand — Actions -> CI -> Run workflow — when a maintainer wants a
+ second opinion on your branch, and even then a green run means those two passed and **nothing
+ more**. The suites that need real software are gated behind `RUN_*` env vars precisely because a
+ hosted runner has no tmux server, no Herdr and no Cursor, so `test:tmux-real`, `test:herdr-real`
+ and `test:cursor-e2e` are only ever run by you and by the maintainer reviewing you. For an engine
+ or a multiplexer, that is the half of the verdict that matters, and it starts from what you
+ report.
5. **Review is manual, and hands-on.** A maintainer pulls the branch and runs the suites locally. For
an engine or a multiplexer that also means installing the real software, so tell us exactly what to
install and how you exercised it — a change nobody can reproduce cannot be merged, however good it
diff --git a/README.md b/README.md
index f396af447..7175ce213 100644
--- a/README.md
+++ b/README.md
@@ -470,8 +470,9 @@ Each product releases on its own tag and the suffix routes the workflow: `vX.Y.Z
publishes the daemon (running daemons pick it up within a minute), `vX.Y.Z_backend` builds the image,
`vX.Y.Z_desktop` builds, signs and publishes both macOS bundles and both Linux architectures.
`make release-cli|release-backend|release-desktop` cut them; `make upload-circle` publishes device
-firmware over the air. `ci.yml` runs the CLI suite on every pull request and holds no secrets, which
-is what lets it run on a fork's code; see [`docs/cicd.md`](docs/cicd.md). `make remote-machine`
+firmware over the air. `ci.yml` runs the CLI suite on demand (Actions -> CI -> Run workflow) and holds
+no secrets, which is what lets it run on a fork's branch; see [`docs/cicd.md`](docs/cicd.md).
+`make remote-machine`
brings up a second machine in Docker so the remote path can be exercised from one laptop.
## Docs
diff --git a/cli/src/engines/opencode/reader.spec.ts b/cli/src/engines/opencode/reader.spec.ts
index 8923d0d09..e767f67fb 100644
--- a/cli/src/engines/opencode/reader.spec.ts
+++ b/cli/src/engines/opencode/reader.spec.ts
@@ -38,7 +38,10 @@ d('OpencodeReader (sqlite3 CLI)', () => {
let dir = ''
let db = ''
beforeEach(() => { dir = mkdtempSync(join(tmpdir(), 'oc-reader-')); db = join(dir, 'opencode.db'); schema(db) })
- afterEach(() => { rmSync(dir, { recursive: true, force: true }) })
+ // maxRetries: a reader poll (sqlite3 CLI writing its journal) can still be touching the directory
+ // when the test ends, and a bare rmSync then dies with ENOTEMPTY — seen on CI (ubuntu-latest),
+ // where it made an unrelated PR red. Node retries the removal on exactly that class of error.
+ afterEach(() => { rmSync(dir, { recursive: true, force: true, maxRetries: 5, retryDelay: 100 }) })
/**
* Attaching mid-turn must still open the turn.
diff --git a/cli/src/lib/e2ee/core.test.ts b/cli/src/lib/e2ee/core.test.ts
index 2e53a2808..55bd0e38a 100644
--- a/cli/src/lib/e2ee/core.test.ts
+++ b/cli/src/lib/e2ee/core.test.ts
@@ -206,6 +206,6 @@ describe('e2ee core — interop keystone', () => {
it('core.ts still hashes to the pinned value shared with the other implementations', () => {
const here = dirname(fileURLToPath(import.meta.url))
const actual = createHash('sha256').update(readFileSync(join(here, 'core.ts'))).digest('hex')
- expect(actual).toBe('84cb86ec25470ed2a573f13381f077f2b27a56f4396eccf5a0408d1a2266d476')
+ expect(actual).toBe('dde492e6fa8ad614439fc5a41742a86ad3e8a79db5a4389d12b70383d6c00c48')
})
})
diff --git a/cli/src/scripts/install.spec.ts b/cli/src/scripts/install.spec.ts
index ffb7c428b..d0a715b86 100644
--- a/cli/src/scripts/install.spec.ts
+++ b/cli/src/scripts/install.spec.ts
@@ -234,6 +234,12 @@ describe("scripts/install.sh command contract", () => {
},
}, null, 2);
writeFileSync(join(scratch, "manifest.json"), manifest);
+ // The ladder's first rung is `command -v tmux && tmux -V`, and PATH below still includes /usr/bin
+ // — where ubuntu-latest (CI) ships a real tmux 3.4, so on that runner the script found it, said
+ // "tmux ready (tmux 3.4)" and never took the managed path these tests exist for. A broken tmux
+ // in the fixture dir (first on PATH) fails that rung everywhere; once the managed build is
+ // linked, the script puts BIN_DIR ahead of PATH, so the 9.9 fake wins from then on.
+ writeCommand(scratch, "tmux", ["exit 127"]);
writeCommand(scratch, "curl", [
`printf '%s\\n' "$*" >> '${join(scratch, "curl-invocations")}'`,
`case "$*" in *"-o "*) cp '${archive}' "$4" ;; *) cat '${join(scratch, "manifest.json")}' ;; esac`,