Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.
Rust agentai-agentapplecomputer-usecomputer-use-agentcontainerizationcuadesktop-automationhacktoberfestlumemacosmanusoperatorswiftvirtualizationvirtualization-frameworkwindowswindows-sandbox
docs(cua-driver): run the macOS Lume lane from the verify skill (#6476)
injaneity406b9b2757ec
diff --git a/.cursor/skills/verify-cua-driver/SKILL.md b/.cursor/skills/verify-cua-driver/SKILL.mdindex ab31b28509..ca02568f60 100644--- a/.cursor/skills/verify-cua-driver/SKILL.md+++ b/.cursor/skills/verify-cua-driver/SKILL.md@@ -1,12 +1,22 @@---name: verify-cua-driver-description: Verify the Cua Driver daemon and CLI/MCP desktop tools against real fixture apps after driver changes, using Linux diagnostics, AX-first actions, side-effect checks, and retained evidence.+description: Verify the Cua Driver daemon and CLI/MCP desktop tools against real fixture apps after driver changes - Linux X11/Wayland on a disposable desktop, macOS/AppKit in a disposable Lume guest - using AX-first actions, side-effect checks, and retained evidence.---# Verify Cua DriverRead [the feature map](features/README.md) first. This verifies the daemon's desktop-tool surface, not the bench, and does not certify every mapped entry point by exercising one convenient path. Use only a disposable desktop: the foreground cells intentionally move focus and input.+## Pick the lane++| Change touches | Lane | Entry point |+| --- | --- | --- |+| Linux platform code, shared core/contract/SDK, this skill | Linux X11, plus Wayland for capture/input routes | `scripts/verify-linux.sh x11\|wayland` (below) |+| `platform-macos`, AppKit/SwiftUI/TextEdit harnesses, `scripts/ci/macos/**`, `tests/runners/macos-lume/**` | macOS Lume guest | `scripts/verify-macos.sh` ([macOS lane](macos-lume.md)) |+| Shared code with macOS behavior | both | both |++Run every lane the change touches that the host can run, and name the lanes you skip in the report. Launch through Cleanup below cover Linux. The macOS lane keeps the same proof standards; its wrapper owns the worker VM, preflight and evidence collection.+## LaunchRun from the repository root. Use the CI toolchain (Rust 1.97.1, Node 22) and install the Linux dependencies from `.github/workflows/e2e-rust-linux.yml`:@@ -87,19 +97,26 @@ After session teardown, the helper invokes `scripts/check-evidence.sh` on the pr## Helpers-The shipped executable is:+The shipped executables are:```bash.cursor/skills/verify-cua-driver/scripts/verify-linux.sh x11.cursor/skills/verify-cua-driver/scripts/verify-linux.sh x11 --no-build.cursor/skills/verify-cua-driver/scripts/verify-linux.sh wayland --no-build+.cursor/skills/verify-cua-driver/scripts/verify-macos.sh doctor+.cursor/skills/verify-cua-driver/scripts/verify-macos.sh --source /abs/clean/checkout focus <cell>+.cursor/skills/verify-cua-driver/scripts/verify-macos.sh --source /abs/clean/checkout full```-It reuses `scripts/ci/linux/run-rust-e2e.sh` through `CUA_E2E_INTERNAL_LANE=verify` and `CUA_E2E_ARTIFACT_DIR`; the runner is responsible for strict fixtures, preflight, recordings and report validation. Each desktop run is bounded to 1200 seconds internally. No bench code or second harness is involved.+It reuses `scripts/ci/linux/run-rust-e2e.sh` through `CUA_E2E_INTERNAL_LANE=verify` and `CUA_E2E_ARTIFACT_DIR`; the runner is responsible for strict fixtures, preflight, recordings and report validation. Each desktop run is bounded to 1200 seconds internally. `verify-macos.sh` calls the Lume host's installed `lume-e2e` wrapper, which runs `run-all.sh` unchanged in the guest. No bench code or second harness is involved.++## macOS++Follow [the macOS lane](macos-lume.md). It runs the canonical `libs/cua-driver/tests/runners/macos-lume/run-all.sh` in a disposable clone of a granted Lume seed, through `scripts/verify-macos.sh`. Don't run AppKit E2E on a host desktop.-## macOS and Windows: unverified+## Windows: unverified-macOS needs the installed TCC-authorized daemon (`libs/cua-driver/scripts/install-local.sh`, `cua-driver permissions grant`) and Accessibility/Screen Recording consent; its tests use the existing daemon proxy rather than Linux Xvfb. Windows needs an interactive desktop, native UIA and named-pipe transport; testkit uses Job Objects for ownership. Use their existing CI recipes and recording paths. Neither platform was run here; Linux's unrestricted testkit mode does not prove their production permission paths.+Windows needs an interactive desktop, native UIA and named-pipe transport; testkit uses Job Objects for ownership. Use its existing CI recipe and recording paths. Linux's unrestricted testkit mode does not prove its production permission paths.## Execution status and maintenancediff --git a/.cursor/skills/verify-cua-driver/execution.md b/.cursor/skills/verify-cua-driver/execution.mdindex f6ee108b16..3b727436aa 100644--- a/.cursor/skills/verify-cua-driver/execution.md+++ b/.cursor/skills/verify-cua-driver/execution.md@@ -36,4 +36,4 @@ Setup logs survived cleanup. Doctor, preflight and both click cells did not run:- Disposable driver HOME/XDG state, disabled overlay/permissions gate and testkit unrestricted GUI authorization/approval bypass.- Fixture control only establishes sentinel posture; it does not set the target action result.-These do not certify production admission, interactive consent, physical GPU capture or arbitrary app compatibility. Wayland, macOS and Windows remain unverified.+These do not certify production admission, interactive consent, physical GPU capture or arbitrary app compatibility. Wayland and Windows remain unverified here; macOS runs in its own [Lume lane](macos-lume.md).diff --git a/.cursor/skills/verify-cua-driver/macos-lume.md b/.cursor/skills/verify-cua-driver/macos-lume.mdnew file mode 100644index 0000000000..3eee3db721--- /dev/null+++ b/.cursor/skills/verify-cua-driver/macos-lume.md@@ -0,0 +1,50 @@+# macOS lane: Lume guest++Use this lane for any change to macOS or AppKit code: `platform-macos`, the AppKit/SwiftUI/TextEdit harnesses, `scripts/ci/macos/**`, or `libs/cua-driver/tests/runners/macos-lume/**`. It runs the canonical guest entry point, `libs/cua-driver/tests/runners/macos-lume/run-all.sh`, inside a disposable clone of a stopped, already-granted Lume seed. The seed contract and the by-hand flow are in [the runner README](../../../libs/cua-driver/tests/runners/macos-lume/README.md); this page covers only how the verify skill drives it.++Do not run the AppKit E2E tests directly on the maintainer host's own desktop. That desktop isn't disposable, its TCC grants belong to whoever installed it, and on the relay Mac mini the fixture hangs (see [Host notes](#host-notes)).++## Run++The Mac with Lume has an installed wrapper, `~/.local/bin/lume-e2e`, configured by `~/.local/share/cua-lume-e2e/config.json`. It clones the configured seed and boots the worker. It then copies a `git archive` of the exact SHA, starts `run-all.sh` from the guest's GUI Terminal (not over SSH), pulls back the evidence and stops the worker. It never deletes a VM. The skill helper wraps it:++```bash+H=.cursor/skills/verify-cua-driver/scripts/verify-macos.sh+$H doctor # host + seed readiness, no VM started+$H --source /abs/clean/checkout focus <cell> # one registered cell, one attempt+$H --source /abs/clean/checkout full # full matrix (about an hour)+$H --source /abs/clean/checkout full --standalone-browser+```++`--source` defaults to this checkout. The source must be a clean git root, and its `HEAD` is the SHA under test. Use a detached worktree at the PR head, not a working branch with edits. The helper runs `lume-e2e --doctor` first. It refuses a focused AppKit cell when the SHA has no `scripts/ci/macos/appkit-cells.tsv`; use `full` there. The wrapper prints `evidence=<run dir>` before it clones and `result.json` at the end. Exit 0 means the runner passed and the wrapper verified the evidence and the standard-daemon restore. Exit 1 means a test or infrastructure failure. Exit 2 means it was blocked before execution.++Focused cells are the names in `scripts/ci/macos/appkit-cells.tsv`, plus the shared scroll and typed-SDK click cells that the wrapper accepts. A focused pass is `mode: focused`, `focused_passing_cells: 1`. It is evidence for that cell only, never full-matrix certification.++A Mac without the wrapper has to follow the runner README by hand: clone a worker, run `sync-vm-worktree.sh push`, run `run-all.sh` in the guest Terminal, run `pull-artifacts`, then stop the worker.++## Evidence++Read `<run dir>/result.json` first: `sha`, `mode`, `status`, `runner_exit`, `clone_source`, `clone_source_note`, `evidence_collected`, `worker_stopped`. The guest log is `evidence/runner.log`. The canonical reports are under `evidence/source/artifacts/cua-driver/macos/`: `summary.md`, `results.jsonl`, `failures.json`, `retry-record.json` for focused runs, and per-cell `recordings/` with MP4s. Check that `sha` is still the PR head before you claim a result. A moved head is untested.++## Seed++The wrapper clones `seed` from `config.json` and records `seed_note` in every result. On the relay Mac mini the seed is `cua-driver-browser-baseline-20261104-tk86-ax`: Tk 9.1, Chrome, and the Terminal Accessibility grant below. `cua-driver-browser-baseline-20261104-tk86` is its parent and has no Terminal grant. The seed must be stopped; never boot or edit it for a run. To change it, clone it to a new name, make the one change in the clone, stop the clone, and point `config.json` at it with an updated `seed_note`.++The runner stops at exit 2 when the seed lacks one of these. The seed steps are in the [runner README](../../../libs/cua-driver/tests/runners/macos-lume/README.md#create-the-private-seed):++- **Tk 8.6 or later** (`require_modern_tk`). Apple's `/usr/bin/python3` has Tk 8.5.9. `brew install python-tk` puts Homebrew's `python3` first; the current seed has Tk 9.1.+- **Terminal → System Events Automation** ("Terminal cannot control System Events").+- **Terminal Accessibility** ("grant Terminal Accessibility in the seed"). The switch in System Settings looks on before it is saved. It persists only after the admin-password dialog accepts the guest password; then the runner's `AXRole` probe returns `AXApplication`. With no human at the display, the guest's own `cua-driver-local` can do the clicks and typing over SSH.+- **CuaDriverLocal** Accessibility, Screen Recording, direct capture and a stable signing identity.++## Host notes++On the relay Mac mini:++- **AppKit fixture hang.** On the host desktop, `~/Library/Application Support/com.apple.sharedfilelist` returns `Interrupted system call` (EINTR). The AppKit fixture's main thread then blocks in the Apple menu's recent-items lookup, and AX calls return `-25204`. `harness_appkit_smoke` fails on main this way too. Run AppKit cells in the Lume guest only. Host-side `cargo test` for `platform-macos` unit tests is fine.+- **Root-owned `~/.npm`.** Host-side `npm ci` and `npm install` fail with `EACCES`. Pass a private cache, for example `npm ci --cache "$(mktemp -d)"`, and delete it afterwards. The guest is unaffected.+- `lume ls` lists every past `cua-e2e-*` worker. The wrapper only stops them, so don't treat a stopped worker as the seed, and don't prune VMs from a verify run.++## Report++The `verify:` line names the lane, the mode and the cell, for example `macOS Lume focused macos-appkit-native-geometry-mismatch-px-background`. Name what was not run: the full matrix, standalone browsers, SIP-on permission flow, Linux or Windows.diff --git a/.cursor/skills/verify-cua-driver/scripts/verify-macos.sh b/.cursor/skills/verify-cua-driver/scripts/verify-macos.shnew file mode 100755index 0000000000..45bd240e87--- /dev/null+++ b/.cursor/skills/verify-cua-driver/scripts/verify-macos.sh@@ -0,0 +1,46 @@+#!/usr/bin/env bash+# macOS lane of verify-cua-driver: runs the canonical Lume guest matrix+# (libs/cua-driver/tests/runners/macos-lume/run-all.sh) through the maintainer+# host's installed lume-e2e wrapper. See ../macos-lume.md.+set -euo pipefail+ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../../.." && pwd)"+LUME_E2E="${LUME_E2E:-$HOME/.local/bin/lume-e2e}"+usage() {+ echo 'usage: verify-macos.sh [--source CLEAN_CHECKOUT] focus <registered-cell> | full [--standalone-browser] | doctor' >&2+ exit 2+}+SOURCE="$ROOT"+if [[ "${1:-}" == --source ]]; then+ [[ $# -ge 2 ]] || usage+ SOURCE="$(cd "$2" && pwd)"+ shift 2+fi+MODE="${1:-}"+shift "$(( $# > 0 ? 1 : 0 ))"+args=()+case "$MODE" in+ focus) [[ $# == 1 ]] || usage; args=(--focus-cell "$1") ;;+ full) [[ $# == 0 || "$*" == --standalone-browser ]] || usage; args=("$@") ;;+ doctor) [[ $# == 0 ]] || usage ;;+ *) usage ;;+esac+[[ "$(uname -s)" == Darwin ]] || { echo 'macOS lane runs on the Lume host only' >&2; exit 2; }+[[ -x "$LUME_E2E" ]] || {+ echo "no lume-e2e wrapper at $LUME_E2E; on another Mac follow libs/cua-driver/tests/runners/macos-lume/README.md by hand" >&2+ exit 2+}+"$LUME_E2E" --doctor+[[ "$MODE" == doctor ]] && exit 0+SHA="$(git -C "$SOURCE" rev-parse HEAD)"+if [[ -n "$(git -C "$SOURCE" status --porcelain --untracked-files=normal)" ]]; then+ echo "source $SOURCE is dirty; commit or use a clean worktree at the SHA under test" >&2+ exit 2+fi+if [[ "$MODE" == focus ]] && ! git -C "$SOURCE" cat-file -e "$SHA:scripts/ci/macos/appkit-cells.tsv" 2>/dev/null \+ && [[ "${args[1]}" == macos-appkit-* ]]; then+ echo "$SHA has no scripts/ci/macos/appkit-cells.tsv, so AppKit cells cannot run focused; use: full" >&2+ exit 2+fi+echo "Source: $SOURCE at $SHA"+# The wrapper prints evidence=<run dir> before cloning and result.json at the end.+exec "$LUME_E2E" --source "$SOURCE" --sha "$SHA" ${args[@]+"${args[@]}"}diff --git a/libs/cua-driver/tests/runners/macos-lume/README.md b/libs/cua-driver/tests/runners/macos-lume/README.mdindex ecc64a3657..19e86c1872 100644--- a/libs/cua-driver/tests/runners/macos-lume/README.md+++ b/libs/cua-driver/tests/runners/macos-lume/README.md@@ -248,6 +248,8 @@ Choose Allow for `Terminal` -> `System Events` and on the CuaDriverLocaldirect-capture prompt. Then add Terminal under System Settings > Privacy &Security > Accessibility: the `bring_to_front` oracles raise fixture windows andread `AXFocusedWindow` through System Events UI scripting, which needs it. The+switch is saved only after the admin-password dialog accepts the account+password; until then the runner's `AXRole` probe still fails. Therunner checks both Terminal grants before it builds. CuaDriverLocal app enumeration must not ask for SystemEvents. Target-specific Automation prompts may still appear later when a userexplicitly requests an Apple Events-backed browser or app operation; do not