Skip to content

armgit / cua

Public Imported from GitHub
0

Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.

blog(cua-driver-bench): apply Scale review notes on naming and sourcing (#6709)

Francesco Bonacci5863940f8180
diff --git a/blog/cua-driver-bench.md b/blog/cua-driver-bench.md
index 2589c55989..81ebd5a0da 100644
--- a/blog/cua-driver-bench.md
+++ b/blog/cua-driver-bench.md
@@ -48,9 +48,9 @@ In the **foreground** condition the agent may take focus. It answers a simpler q
Cua Driver Bench has ten public tasks. Three are multi-app workflows (CDB-G02 to CDB-G04: mail, chat, a calendar, LibreOffice Calc, GnuCash, a case tracker and Chrome). Three are precision tasks (MB-09 drag to reorder a list, MB-10 a hover-only toolbar, MB-11 a tooltip). Four are interruptions (IR-01 a permission sheet, IR-02 a banner, IR-03 an unsaved-changes prompt, IR-04 a cookie-consent overlay). Eight of the ten are scored in the background. MB-10 and MB-11 need hover, which no OS can deliver to a background window without moving the user's pointer, so we agreed with Scale to score them in the foreground only.
-We compare four drivers: cua-driver 0.38.0 (Cua), agent-desktop 1.2.0 (Vercel Labs), browser-use (macos-harness 0.2.1, windows-harness 0.2.2) and Codex Computer Use, OpenAI's computer-use plugin from the ChatGPT app. In our setup Claude chooses the actions for every driver, so the Codex row is OpenAI's tool layer driven by Claude, not OpenAI's own agent. Codex Computer Use isn't available on Linux yet ([OpenAI's Linux app docs](https://learn.chatgpt.com/docs/linux/linux-app#compatibility-and-limitations)), and on Windows it runs in the foreground only ([OpenAI's computer use docs](https://learn.chatgpt.com/docs/computer-use)).
+We compare four drivers: cua-driver 0.38.0 (Cua), Vercel Labs' agent-desktop 1.2.0, browser-use (macos-harness 0.2.1, windows-harness 0.2.2) and Codex Computer Use, OpenAI's computer-use plugin from the ChatGPT app. In our setup Claude chooses the actions for every driver, so the Codex row is OpenAI's tool layer driven by Claude, not OpenAI's own agent. Codex Computer Use isn't available on Linux yet ([OpenAI's Linux app docs](https://learn.chatgpt.com/docs/linux/linux-app#compatibility-and-limitations)), and on Windows it runs in the foreground only ([OpenAI's computer use docs](https://learn.chatgpt.com/docs/computer-use)).
-**Where the numbers come from.** Scale ran cua-driver, agent-desktop and browser-use on macOS, and cua-driver on Linux (Ubuntu 24.04 with XFCE), on AgentEnv, and published every trial. Those are the numbers we report for them. Where Scale has no result, the number is Cua's, labelled "run by Cua": Codex Computer Use on macOS (plain macOS VMs), all four drivers on Windows (one Windows 11 VM), and agent-desktop on Linux (the same Ubuntu 24.04 and XFCE desktop). Every rate below is k out of n trials, and charts show a Wilson 95% interval. With 30 to 50 trials per driver, some gaps are within noise, and we say so where they are.
+**Where the numbers come from.** Scale ran cua-driver, agent-desktop and browser-use on macOS, and cua-driver on Linux (Ubuntu 24.04 with XFCE), on AgentEnv, and published every trial. Those are the numbers we report for them. Where Scale has no result, the number is Cua's, labelled "run by Cua": Codex Computer Use on macOS (plain macOS VMs), all four drivers on Windows (one Windows 11 VM), and agent-desktop on Linux (the same Ubuntu 24.04 and XFCE desktop). Next week, AgentEnv will add the Windows runs, possibly Codex Computer Use as well, and 30+ more tasks. Until then, those numbers here stay run by Cua. Every rate below is k out of n trials, and charts show a Wilson 95% interval. With 30 to 50 trials per driver, some gaps are within noise, and we say so where they are.
## 2. Results
@@ -162,6 +162,7 @@ For the runs we did ourselves, our invocation differs from the public runner in
- cua-driver fixes for the two macOS gaps: choosing from a pop-up menu without showing it over the user's window, and drag and drop in the background. Then the Windows activation cases on IR-01, IR-03 and IR-04, and on Linux the two limits behind cua-driver's refusals in Scale's run: background input to Chromium and Electron windows on X11, and background keystrokes without a virtual keyboard device.
- OpenAI's models through OpenAI's own harness are already in Scale's dataset: GPT-6 Luna, GPT-6.1 Terra and GPT-6.1 Sol through Codex CLI at medium effort, on Linux with cua-driver, 90 trials each. One caveat if you build on Codex: it hands the model only a tool result's structuredContent, so the model never sees cua-driver's screenshots, only its accessibility tree. Scale ran Codex as shipped.
+- Next week, AgentEnv will add the Windows runs, possibly Codex Computer Use as well, and 30+ more tasks.
- The sealed held-out task set, run under the same rules.
If you build a driver, an agent or a desktop and want your setup in the next run, open an issue on the repository. We're especially interested in hearing from anyone working on background hover at the OS or compositor level.