A desktop coding agent with free access to frontier models.
Testing without blocking unrelated work
Run pnpm test (alias: pnpm test:core) before a PR. Install both workspaces
with pnpm install --frozen-lockfile and
pnpm --dir evals install --frozen-lockfile first. Use Node 24, Bun 1.3.14,
pnpm 11.4.0, and the OpenCode version in constants.json on PATH, matching CI.
The core suite uses local fixtures and scripted providers; no cloud account,
provider key, Docker daemon, or running Den database is required.
What blocks a merge
The omnirush-tests-required check keeps its existing name and fails closed.
For ordinary code changes it requires two independent Linux jobs:
- Core regressions: session admission, streaming and reconnects, permission state, attachments, provider credentials, and Connect reconciliation in the client; real server routes for threads, groups, proxying, folder permissions, upload approval, artifact I/O, cloud configuration, engine eviction and reloads; token scope and export safety; Den authentication; desktop workspace persistence, archives, links, credential keys, automation execution, process resilience, and TLS. Three existing real-engine journeys additionally check remembered thread approvals, effective permission attribution, and PDF model routing.
- Packaging: outbound-access declarations, the server's actual Node-target plugin build, and Electron's IPC contract typecheck. A test failure cannot prevent this independent job from reporting a packaging regression.
Model-snapshot-only and docs-only PRs keep their existing dedicated validation. New commits cancel obsolete runs on the same PR. Dev pushes still run the core and packaging checks. PRs no longer pay for two OS copies of the broad suites.
The package test:core scripts are the selection. There is no second inventory,
selection generator, coverage ratchet, or test asserting that inventory matches
itself. To expand coverage, extend an existing test for a core failure mode.
Add a test to the gate when it catches an observable regression in a critical
journey, runs deterministically with declared prerequisites, and earns its cost.
Do not add source-text/layout assertions, test-runner wrappers, or bookkeeping
checks to this gate. A failing core test must be diagnosed and fixed, not retried
until green or silently ignored.
Broader coverage
The same OmniRush.ai Tests workflow runs the broad app, server, Den, desktop,
release, test-framework, PR-spec, and engine-smoke suites on Linux and macOS at
07:37 UTC daily, or through Run workflow on a selected branch. Each suite
reports even if an earlier suite fails; failures still make that run red.
pnpm test:extended runs those test suites locally. Packaging can be reproduced
with pnpm --filter omnirush-server build and
pnpm --filter @omnirush/desktop typecheck:electron.
Use the full package test command when changing its internals, and
pnpm test:eval-runner when changing the test framework. These tests remain in
the repository. Moving them out of the universal gate means regressions outside
the selected core may first be detected by targeted validation or the nightly
run. Check the macOS nightly before releases; it is no longer a PR prerequisite.
The existing Daytona E2E and nightly flake-report workflows are unchanged.
A skipped journey is incomplete coverage, even if a runner exits successfully. Do not describe a run containing skips as full proof.
Why this changed
An audit of OmniRush.ai Tests runs created August 28–September 3, 2026 (UTC) found 159 failures among 632 runs, including 17 awaiting approval. Among the 615 success/failure results, 25.9% failed. These are run counts, including repeated branch updates, not a measured flake rate.
Sampled failure logs show different problems that need different fixes:
- September 3:
spec-impactandspec-quarantineinventory assertions failed on both OSes while 115/116 other spec files passed. Those specific specs have since been removed; keeping test-framework bookkeeping out of the default gate prevents rebuilding the same barrier elsewhere. - August 28: a compatibility spec spawned another test runner, obscuring the underlying failure behind a wrapper assertion.
- PR #4442: the shared suite failed on the same engine-retirement timing assertion seen in a dev run. PR #4439 independently repairs that race. Core coverage still exercises real engine eviction and reload behavior; the broad test is retained.
- The separate SDK check on #4442
failed when schema generation connected to MySQL at
127.0.0.1:3306without a database. That is a setup dependency to fix in the SDK change, not a reason to suppress schema-drift validation. This CI cleanup does not fix that branch.
This change adds no tests of tests and no new test files. Runtime savings must be measured after rollout; reducing four broad PR jobs to two focused jobs is not itself evidence of a particular wall-clock improvement.