Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.
Rust agentai-agentapplecomputer-usecomputer-use-agentcontainerizationcuadesktop-automationhacktoberfestlumemacosmanusoperatorswiftvirtualizationvirtualization-frameworkwindowswindows-sandbox
feat(cua-driver-bench): run all ten Bench v2 tasks on Linux XFCE (X11 launcher, desktop setup, agent invocation) (#6688)
cua-agents[bot]486e31316569
diff --git a/libs/cua-driver-bench/README.md b/libs/cua-driver-bench/README.mdindex 3bdf8985cf..98779fdfff 100644--- a/libs/cua-driver-bench/README.md+++ b/libs/cua-driver-bench/README.md@@ -350,12 +350,70 @@ launch/linux_session.sh --evidence "$CDB_BENCH_WORK/evidence" -- <runner commandSeeds, the state file, the event log, the checkers and the sentinel summary are the same as on macOS.Of the comparison setups, agent-desktop runs on Linux (`npm install [email protected]`, then-`run_bench.py --arms cc-agent-desktop` inside the session; the adapter is platform-neutral, but the release-tables do not include a Linux agent-desktop run). The Browser Use harnesses exist for macOS and Windows only, and+`run_bench.py --arms cc-agent-desktop` inside the session; or all ten tasks through the XFCE launcher below). The Browser Use harnesses exist for macOS and Windows only, andCodex computer use is not available on Linux yet([compatibility and limitations](https://learn.chatgpt.com/docs/linux/linux-app#compatibility-and-limitations)).`tests/test_linux_session.py` and `tests/test_linux_tools.py` check the session and the tools without a model.+### Linux XFCE (all ten tasks)++[`launch/xfce_session.py`](automated-eval/macos_bench/launch/xfce_session.py) runs one trial of any of the ten Bench+v2 tasks on an X11 desktop with an EWMH window manager (XFCE 4.18 with xfwm4 here), as the desktop user. The BenchLab+tasks use `linux/benchlab.py` and `linux/sentinel.py`; the three multi-app tasks (CDB-G02 to CDB-G04) reset their+workspace and start their apps from the task's own `platform/launch.linux.json`. It parks the pointer, arms the+sentinel, runs your agent command under the GUI-only guard, stops it at the limit, runs the checker and writes+`row.json` next to the transcript, the sentinel log, the app logs and the final workspace. The Linux XFCE rows of+the Bench v2 report (agent-desktop 1.2.0 and a cua-driver 0.38.0 cross-check, 5 trials per task, 6 Nov 2026) were run+with exactly this.++**1. The desktop.** Ubuntu 24.04 with XFCE 4.18 on Xvfb at 1920 x 1080, as an image or a VM:++```sh+# Cua's Linux image (XFCE on Xvfb); CUA_RESOLUTION sets the 1920x1080 the shared window frames need+docker run -d --name bench-xfce -e CUA_RESOLUTION=1920x1080 ghcr.io/trycua/linux:24.04+# or plain Ubuntu 24.04: apt-get install xfce4 xvfb dbus-x11 at-spi2-core python3-gi gir1.2-gtk-3.0 gir1.2-atspi-2.0,+# then start Xvfb :0 -screen 0 1920x1080x24 and an XFCE session with a private D-Bus and AT-SPI+```++You need Node 24 (we used 24.21) and Python 3.11 or later with PyGObject on the desktop, and the checkout of this+repository (or just `libs/cua-driver-bench`) as the desktop user. Then, inside the desktop, as root and then as the+desktop user:++```sh+sudo launch/xfce/provision-root.sh # xdotool, wmctrl, LibreOffice 26.2.7, Google Chrome, GnuCash 5.17 (Flathub)+launch/xfce/provision-user.sh # cua-driver 0.38.0, agent-desktop 1.2.0, npm ci of the four Electron apps+```++In a container neither Chrome, the Electron apps nor flatpak can create user namespaces, so the scripts run Chrome and+Electron with `--no-sandbox` through shims (Chrome then shows its unsupported-flag infobar) and run GnuCash outside+bubblewrap on its own GNOME runtime. On a full VM set `BENCH_NO_SANDBOX_SHIMS=0` for both scripts and skip+`shim-gnucash.sh`. LibreOffice shows its Welcome dialog and GnuCash its Tip of the Day on start, as on Windows.++**2. One trial.** Run it as the desktop user with the desktop's environment (`DISPLAY`, `DBUS_SESSION_BUS_ADDRESS`):++```sh+cd libs/cua-driver-bench/automated-eval/macos_bench+python3 launch/xfce_session.py --task IR-02 --run-index 0 --condition background \+ --out /tmp/trial --label cua-driver --timeout 360 -- launch/xfce/agent.sh cua-driver+```++`--condition` is `foreground` or `background`; `--run-index` 0 to 4 picks the fairness seed (the same seeds as the other+platforms). `agent.sh ARM` (`cua-driver` or `agent-desktop`) starts the agent: Claude Code 2.1.316 `claude -p` with+`--model claude-sonnet-5-5 --effort medium`, `MAX_THINKING_TOKENS=16000`, `--tools ""` (no tools beside the driver's MCP+server(s)), `--strict-mcp-config`, `--permission-mode bypassPermissions`, `--max-turns 45`, stream-json output, and the+task brief as the prompt with no system prompt. agent-desktop gets two servers, strict headless and `--headed`, and the+agent picks per call. The limit is 360 s and 45 turns per trial. `row.json` holds the checker result, the GUI-only+verdict and the sentinel's criteria; a trial counts as a background full pass only when the checker passed, there was+no GUI-only violation and all eight undisturbed criteria held (an unmeasured one counts as disturbed).++**3. The run.** The report's plan is, per arm, 8 background tasks (CDB-G02 to CDB-G04, MB-09, IR-01 to IR-04) and 10+foreground tasks, each with run indexes 0 to 4, one trial at a time per desktop (kill the task apps between trials; the+session resets the workspace itself). We ran six desktops in parallel, one per 8 vCPU / 32 GB cloud VM with the Linux+image as a 6 vCPU / 24 GB container, and ran the agent on a separate host with its MCP server over ssh; with the+agent on the desktop itself, as `agent.sh` does, the tool calls skip that network round trip. Pins: cua-driver+`cua-driver-rs-v0.38.0` linux-x86_64 release, agent-desktop 1.2.0 linux-x64 from npm. Not run on Linux: Browser Use (no+Linux desktop harness) and Codex computer use (not available on Linux).+### Linux on Hyprland (Wayland; the BenchLab tasks)The Hyprland port runs the same GTK3 BenchLab (`linux/benchlab.py`) as a native Wayland client, with adiff --git a/libs/cua-driver-bench/automated-eval/macos_bench/launch/xfce/agent.sh b/libs/cua-driver-bench/automated-eval/macos_bench/launch/xfce/agent.shnew file mode 100755index 0000000000..760fbeee43--- /dev/null+++ b/libs/cua-driver-bench/automated-eval/macos_bench/launch/xfce/agent.sh@@ -0,0 +1,47 @@+#!/bin/bash+# SPDX-License-Identifier: FSL-1.1-MIT+# Copyright (c) 2026 Cua AI, Inc.+#+# The agent of one Linux XFCE trial: Claude Code `claude -p` with the driver's MCP server(s) and no other tools,+# exactly the invocation of the macOS, Windows and Hyprland release runs. Run it as the COMMAND of xfce_session.py+# (the session sets CDB_BRIEF_FILE and runs it from the trial's output directory):+#+# python3 launch/xfce_session.py --task IR-02 --run-index 0 --condition background --out /tmp/trial \+# --label cua-driver --timeout 360 -- launch/xfce/agent.sh cua-driver+#+# ARM is cua-driver or agent-desktop. The agent needs the claude CLI (Claude Code 2.1.316), ANTHROPIC_API_KEY or a+# logged-in session, and the desktop's environment (DISPLAY, DBUS_SESSION_BUS_ADDRESS; AT-SPI running).+set -euo pipefail+ARM=${1:?usage: agent.sh cua-driver|agent-desktop}+CDB_HOME=${CDB_HOME:-$HOME/cdb}+BRIEF=${CDB_BRIEF_FILE:?CDB_BRIEF_FILE is set by xfce_session.py}+CLAUDE=${CLAUDE:-claude}+MODEL=${MODEL:-claude-sonnet-5-5}+TURNS=${TURNS:-45}++case "$ARM" in+ cua-driver)+ export CUA_DRIVER_RS_TELEMETRY_ENABLED=false DO_NOT_TRACK=1+ cat > mcp.json <<JSON+{"mcpServers": {"cua": {"command": "$CDB_HOME/cua-driver-rs-0.38.0-linux-x86_64/cua-driver", "args": ["mcp"]}}}+JSON+ ALLOWED=mcp__cua ;;+ agent-desktop)+ export AGENT_DESKTOP_TELEMETRY=0 DO_NOT_TRACK=1+ AD=$CDB_HOME/agent-desktop/node_modules/agent-desktop/bin/agent-desktop-linux-x64+ # Two servers, as on Windows and Hyprland: strict headless (default) and --headed; the agent picks per call.+ cat > mcp.json <<JSON+{"mcpServers": {"agentdesktop": {"command": "$AD", "args": ["mcp"]},+ "agentdesktop_headed": {"command": "$AD", "args": ["mcp", "--headed"]}}}+JSON+ ALLOWED=mcp__agentdesktop,mcp__agentdesktop_headed ;;+ *) echo "unknown arm $ARM" >&2; exit 2 ;;+esac++export MAX_THINKING_TOKENS=16000 DISABLE_AUTOUPDATER=1 CLAUDE_CODE_DISABLE_AUTO_MEMORY=1 \+ CLAUDE_CODE_DISABLE_CLAUDE_MDS=1 DISABLE_TELEMETRY=1+mkdir -p cwd && cd cwd+exec "$CLAUDE" -p "$(cat "$BRIEF")" --model "$MODEL" --effort medium \+ --mcp-config ../mcp.json --strict-mcp-config --tools "" --allowed-tools "$ALLOWED" \+ --permission-mode bypassPermissions --setting-sources project --max-turns "$TURNS" \+ --output-format stream-json --verbose > ../agent-transcript.jsonl 2> ../agent-stderr.logdiff --git a/libs/cua-driver-bench/automated-eval/macos_bench/launch/xfce/provision-root.sh b/libs/cua-driver-bench/automated-eval/macos_bench/launch/xfce/provision-root.shnew file mode 100755index 0000000000..ec83f8d106--- /dev/null+++ b/libs/cua-driver-bench/automated-eval/macos_bench/launch/xfce/provision-root.sh@@ -0,0 +1,46 @@+#!/bin/bash+# SPDX-License-Identifier: FSL-1.1-MIT+# Copyright (c) 2026 Cua AI, Inc.+#+# Provision a Linux XFCE desktop for the Bench v2 Linux rows (run as root, once). Ubuntu 24.04 with XFCE 4.18+# (xfwm4) on Xvfb at 1920x1080, e.g. Cua's `ghcr.io/trycua/linux:24.04` image, or plain Ubuntu with+# `apt-get install xfce4 xvfb dbus-x11 at-spi2-core` and a session that starts them.+#+# Installs the X11 tools the launcher uses, the apps of the multi-app tasks (LibreOffice Calc 26.2, GnuCash 5.17,+# Google Chrome), and the environment shims below. Usage: sudo ./provision-root.sh+set -euo pipefail+export DEBIAN_FRONTEND=noninteractive++apt-get update -qq+apt-get install -y -qq xdotool wmctrl psmisc x11-utils flatpak curl ca-certificates++# LibreOffice 26.2 from The Document Foundation's debs (the distro's version differs).+LO=26.2.7+if [ ! -x /opt/libreoffice26.2/program/soffice ]; then+ tmp=$(mktemp -d)+ curl -fsSL -o "$tmp/lo.tgz" "https://download.documentfoundation.org/libreoffice/stable/$LO/deb/x86_64/LibreOffice_${LO}_Linux_x86-64_deb.tar.gz"+ tar -C "$tmp" -xzf "$tmp/lo.tgz"+ apt-get purge -y -qq 'libreoffice*' >/dev/null 2>&1 || true+ apt-get install -y -qq "$tmp"/LibreOffice_${LO}*_deb/DEBS/*.deb+ rm -rf "$tmp"+fi+ln -sf /opt/libreoffice26.2/program/soffice /usr/local/bin/libreoffice++# Google Chrome (stable).+tmp=$(mktemp -d)+curl -fsSL -o "$tmp/chrome.deb" https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb+apt-get install -y -qq "$tmp/chrome.deb"+rm -rf "$tmp"++# GnuCash 5.17: the Flathub build (Ubuntu 24.04 ships an older GnuCash).+flatpak remote-add --if-not-exists --system flathub https://dl.flathub.org/repo/flathub.flatpakrepo+flatpak install -y --system --noninteractive flathub org.gnucash.GnuCash+"$(dirname "$0")/shim-gnucash.sh"++# Chrome and Electron cannot use their sandbox inside a container (no user namespaces): run them with --no-sandbox+# through PATH shims. Chrome then shows its unsupported-flag infobar. Not needed on a full VM; skip it there.+if [ "${BENCH_NO_SANDBOX_SHIMS:-1}" = 1 ]; then+ printf '#!/bin/sh\nexec /usr/bin/google-chrome --no-sandbox "$@"\n' > /usr/local/bin/google-chrome+ chmod 755 /usr/local/bin/google-chrome+fi+echo PROVISION_ROOT_OKdiff --git a/libs/cua-driver-bench/automated-eval/macos_bench/launch/xfce/provision-user.sh b/libs/cua-driver-bench/automated-eval/macos_bench/launch/xfce/provision-user.shnew file mode 100755index 0000000000..49a7aa9066--- /dev/null+++ b/libs/cua-driver-bench/automated-eval/macos_bench/launch/xfce/provision-user.sh@@ -0,0 +1,33 @@+#!/bin/bash+# SPDX-License-Identifier: FSL-1.1-MIT+# Copyright (c) 2026 Cua AI, Inc.+#+# Per-user setup for the Linux XFCE rows (run as the desktop user, after provision-root.sh): the two drivers under+# test and the Electron apps of the task pack.+#+# Usage: ./provision-user.sh [TASKPACK_DIR] (default: <repo>/libs/cua-driver-bench/taskpack)+# Drivers land in $CDB_HOME (default ~/cdb): cua-driver 0.38.0 and agent-desktop 1.2.0.+set -euo pipefail+CDB_HOME=${CDB_HOME:-$HOME/cdb}+HERE=$(cd "$(dirname "$0")" && pwd)+PACK=${1:-$(cd "$HERE/../../../../taskpack" && pwd)}+mkdir -p "$CDB_HOME" && cd "$CDB_HOME"++if [ ! -x cua-driver-rs-0.38.0-linux-x86_64/cua-driver ]; then+ curl -fsSL -o cd.tgz https://github.com/trycua/cua/releases/download/cua-driver-rs-v0.38.0/cua-driver-rs-0.38.0-linux-x86_64.tar.gz+ tar -xzf cd.tgz && rm cd.tgz+fi+mkdir -p agent-desktop && (cd agent-desktop && [ -d node_modules/agent-desktop ] || npm install --no-audit --no-fund --silent [email protected])+sha256sum cua-driver-rs-0.38.0-linux-x86_64/cua-driver agent-desktop/node_modules/agent-desktop/bin/agent-desktop-linux-x64++# Electron apps of CDB-G02..G04 (`npx electron .` in each app directory).+for a in cdb-s02/apps/lighthouse-desk cdb-s03/apps/ledger-lantern-mail cdb-s04/apps/rillwatch-cases cdb-s04/apps/margin-lantern-manual; do+ (cd "$PACK/tasks/shared/$a" && npm ci --no-audit --no-fund --silent && npx electron --version >/dev/null)+ if [ "${BENCH_NO_SANDBOX_SHIMS:-1}" = 1 ]; then+ d=$PACK/tasks/shared/$a/node_modules/electron/dist+ [ -f "$d/electron.bin" ] || mv "$d/electron" "$d/electron.bin"+ printf '#!/bin/sh\nexec "$(dirname "$0")/electron.bin" --no-sandbox "$@"\n' > "$d/electron"+ chmod 755 "$d/electron"+ fi+done+echo PROVISION_USER_OKdiff --git a/libs/cua-driver-bench/automated-eval/macos_bench/launch/xfce/shim-gnucash.sh b/libs/cua-driver-bench/automated-eval/macos_bench/launch/xfce/shim-gnucash.shnew file mode 100755index 0000000000..6d8355f138--- /dev/null+++ b/libs/cua-driver-bench/automated-eval/macos_bench/launch/xfce/shim-gnucash.sh@@ -0,0 +1,36 @@+#!/bin/bash+# SPDX-License-Identifier: FSL-1.1-MIT+# Copyright (c) 2026 Cua AI, Inc.+#+# `gnucash` on PATH for the Flathub build, run outside bubblewrap on its own GNOME runtime. Needed only where+# flatpak cannot create user namespaces (a container); on a full VM `flatpak run org.gnucash.GnuCash` works and the+# pack can use a plain wrapper for it. Run as root after `flatpak install org.gnucash.GnuCash`.+set -euo pipefail+APPF=/var/lib/flatpak/app/org.gnucash.GnuCash/current/active/files+RTF=/var/lib/flatpak/runtime/org.gnome.Platform/x86_64/50/active/files+ln -sfn "$APPF" /app+cp -f "$RTF/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2" "$(readlink -f "$APPF")/bin/gnucash-ld"+cat > /usr/local/bin/gnucash <<'SH'+#!/bin/bash+# GnuCash 5.17 (Flathub build, /app) on its GNOME 50 runtime's loader and libraries, outside bubblewrap.+RTF=/var/lib/flatpak/runtime/org.gnome.Platform/x86_64/50/active/files+export XDG_DATA_DIRS=/app/share:$RTF/share:/usr/share+export GIO_MODULE_DIR=$RTF/lib/x86_64-linux-gnu/gio/modules+export GDK_PIXBUF_MODULE_FILE=$RTF/lib/x86_64-linux-gnu/gdk-pixbuf-2.0/2.10.0/loaders.cache+export GTK_PATH=$RTF/lib/x86_64-linux-gnu/gtk-3.0+export GTK_EXE_PREFIX=$RTF GTK_DATA_PREFIX=$RTF+unset GTK_MODULES+export GNC_HOME=/app+exec -a gnucash /app/bin/gnucash-ld --library-path /app/lib:/app/lib/gnucash:$RTF/lib/x86_64-linux-gnu:$RTF/lib /app/bin/gnucash "$@"+SH+chmod 755 /usr/local/bin/gnucash+# glycin image loaders: the runtime's config names /usr/libexec/glycin-loaders/...; wrap each on the runtime loader.+for d in "$RTF"/libexec/glycin-loaders/*; do+ v=$(basename "$d"); mkdir -p "/usr/libexec/glycin-loaders/$v"+ for b in "$d"/*; do+ printf '#!/bin/sh\nexec %s/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 --library-path %s/lib/x86_64-linux-gnu:%s/lib %s "$@"\n' \+ "$RTF" "$RTF" "$RTF" "$b" > "/usr/libexec/glycin-loaders/$v/$(basename "$b")"+ chmod 755 "/usr/libexec/glycin-loaders/$v/$(basename "$b")"+ done+done+echo GNUCASH_SHIM_OKdiff --git a/libs/cua-driver-bench/automated-eval/macos_bench/launch/xfce_session.py b/libs/cua-driver-bench/automated-eval/macos_bench/launch/xfce_session.pynew file mode 100644index 0000000000..dd9ff5254b--- /dev/null+++ b/libs/cua-driver-bench/automated-eval/macos_bench/launch/xfce_session.py@@ -0,0 +1,728 @@+#!/usr/bin/env python3+# SPDX-License-Identifier: FSL-1.1-MIT+# Copyright (c) 2026 Cua AI, Inc.+#+# See "Linux XFCE" in libs/cua-driver-bench/README.md for provisioning the desktop and the exact agent invocation.+#+# Usage: xfce_session.py --task ID --run-index N --condition foreground|background --out DIR+# [--timeout S] -- COMMAND [ARG...]+#+# One Bench v2 trial on an existing X11 desktop with an EWMH window manager (XFCE/xfwm4 here), run as the desktop+# user with the session's environment. The X11 counterpart of windows_session.py for the ten public tasks:+# BenchLab tasks (MB-09..MB-11, IR-01..IR-04): linux/benchlab.py + linux/sentinel.py;+# pack tasks (CDB-G02..CDB-G04): reset the workspace and start the apps of the task's own+# platform/launch.linux.json (LinuxCdbTask below, the Linux port of windows/cdb.py), place their windows.+# Then: park the pointer (Amendment 24), arm the sentinel, run COMMAND with the GUI-only guard (hyprland/guard.py,+# compositor-independent) watching, stop it at the limit or on a violation, disarm, run the checker, summarise the+# sentinel, write DIR/row.json.+from __future__ import annotations++import argparse+import json+import os+import re+import shutil+import signal+import subprocess+import sys+import time+import urllib.request+import zipfile+from pathlib import Path+from typing import Any++HERE = Path(__file__).resolve().parent+sys.path.insert(0, str(HERE))+import hyprland_session as hs # noqa: E402++v2, cdb_fairness, summarize_sentinel = hs.v2, hs.cdb_fairness, hs.summarize_sentinel+BENCH = hs.BENCH+LAB_TASKS = ("MB-09", "MB-10", "MB-11", "IR-01", "IR-02", "IR-03", "IR-04")+CDB_TASKS = ("CDB-G02", "CDB-G03", "CDB-G04")+TASKS = CDB_TASKS + LAB_TASKS+LAB_FRAME = hs.LAB_FRAME+WITNESS_FRAME = hs.WITNESS_FRAME+SENTINEL_ID = "ai.cua.benchsentinel"+TASKPACK = Path(os.environ.get("CDB_TASKPACK") or BENCH.parents[1] / "taskpack")+SKIP_KINDS = {"terminal", "editor"}+++def _out(argv: list[str], timeout: float = 5.0) -> str:+ try:+ return subprocess.run(argv, capture_output=True, text=True, timeout=timeout).stdout+ except (OSError, subprocess.SubprocessError):+ return ""+++class X:+ @staticmethod+ def screen() -> tuple[int, int]:+ parts = _out(["xdotool", "getdisplaygeometry"]).split()+ return (int(parts[0]), int(parts[1])) if len(parts) == 2 else (0, 0)++ @staticmethod+ def windows_of(pid: int) -> list[int]:+ return [+ int(w) for w in _out(["xdotool", "search", "--onlyvisible", "--pid", str(pid)]).split()+ ]++ @staticmethod+ def visible() -> list[int]:+ return [int(w) for w in _out(["xdotool", "search", "--onlyvisible", "--name", "."]).split()]++ @staticmethod+ def wm_class(window: int) -> str:+ names = re.findall(r'"([^"]+)"', _out(["xprop", "-id", str(window), "WM_CLASS"]))+ return names[-1] if names else ""++ @staticmethod+ def active() -> int:+ text = _out(["xdotool", "getactivewindow"]).strip()+ return int(text) if text.isdigit() else 0++ @classmethod+ def active_class(cls) -> str:+ w = cls.active()+ return cls.wm_class(w) if w else ""++ @staticmethod+ def stacking() -> list[int]:+ return [+ int(v, 16)+ for v in re.findall(+ r"0x[0-9a-fA-F]+", _out(["xprop", "-root", "_NET_CLIENT_LIST_STACKING"])+ )+ ]++ @classmethod+ def normal_stack(cls) -> list[int]:+ """Stacking order without dock and desktop windows (the XFCE panels stay above every client)."""+ out = []+ for w in cls.stacking():+ kind = _out(["xprop", "-id", str(w), "_NET_WM_WINDOW_TYPE"])+ if "_NET_WM_WINDOW_TYPE_DOCK" in kind or "_NET_WM_WINDOW_TYPE_DESKTOP" in kind:+ continue+ out.append(w)+ return out++ @staticmethod+ def geometry(window: int) -> list[int]:+ shell = dict(+ ln.split("=", 1)+ for ln in _out(["xdotool", "getwindowgeometry", "--shell", str(window)]).split()+ if "=" in ln+ )+ try:+ return [int(shell[k]) for k in ("X", "Y", "WIDTH", "HEIGHT")]+ except (KeyError, ValueError):+ return []++ @staticmethod+ def title(window: int) -> str:+ return _out(["xdotool", "getwindowname", str(window)]).strip()++ @staticmethod+ def activate(window: int) -> None:+ _out(["xdotool", "windowactivate", "--sync", str(window)])++ @staticmethod+ def place(window: int, b: dict) -> None:+ _out(["wmctrl", "-i", "-r", hex(window), "-b", "remove,maximized_vert,maximized_horz"])+ _out(+ [+ "xdotool",+ "windowmove",+ str(window),+ str(b["x"]),+ str(b["y"]),+ "windowsize",+ str(window),+ str(b["width"]),+ str(b["height"]),+ ],+ 10,+ )++ @classmethod+ def frames(cls) -> dict[str, list[int]]:+ return {+ (cls.title(w) or cls.wm_class(w) or hex(w)): cls.geometry(w) for w in cls.stacking()+ }++ @staticmethod+ def cursor() -> tuple[float, float] | None:+ m = re.search(r"x:(\d+) y:(\d+)", _out(["xdotool", "getmouselocation"]))+ return (float(m.group(1)), float(m.group(2))) if m else None+++def park_pointer() -> dict:+ """Amendment 24 (#6660): move the real pointer to the park point (XTest) and read it back."""+ x, y = v2.PARK_POINT+ _out(["xdotool", "mousemove", "--sync", str(x), str(y)])+ return v2.park_record(X.cursor(), "xdotool mousemove")+++# ---------------------------------------------------------------- pack tasks on Linux (port of windows/cdb.py)+def substitute(text: str, values: dict[str, str]) -> str:+ return re.sub(r"\$\{(\w+)\}", lambda m: values.get(m.group(1), m.group(0)), text)+++def title_matches(win: dict[str, Any], title: str) -> bool:+ if win.get("title"):+ return title == win["title"] or title.startswith(win["title"] + " ")+ if win.get("title_contains"):+ return win["title_contains"].lower() in title.lower()+ return False+++class LinuxCdbTask:+ """One pack task, driven by its own platform/launch.linux.json. Electron apps run as the descriptor says+ (`npx electron .`), as run_bench.py does on Linux."""++ KILL_NAMES = (+ "electron",+ "soffice.bin",+ "soffice",+ "oosplash",+ "gnucash",+ "chrome",+ "google-chrome",+ )++ def __init__(self, spec: dict[str, Any], artifacts: Path, seed: int | None = None) -> None:+ self.spec = spec+ self.seed = seed+ self.bundle = TASKPACK / "tasks" / spec["pack_task"]+ self.descriptor = json.loads(+ (self.bundle / "platform" / "launch.linux.json").read_text("utf-8")+ )+ self.artifacts = artifacts+ self.workspace = Path(+ self.descriptor["workspace_root"].replace("${HOME}", str(Path.home()))+ )+ self.procs: list[tuple[dict[str, Any], subprocess.Popen]] = []+ self.windows: dict[str, int] = {}+ self.log: list[str] = []++ def values(self, **extra: str) -> dict[str, str]:+ return {+ "python": sys.executable,+ "seed": str(self.seed),+ "workspace": str(self.workspace),+ "workspace_uri": self.workspace.as_uri(),+ "bundle": str(self.bundle),+ "HOME": str(Path.home()),+ "artifacts": str(self.artifacts),+ **extra,+ }++ def sub(self, text: str, **extra: str) -> str:+ return substitute(text, self.values(**extra))++ def _run(+ self, command: list[str], timeout: int = 300, **extra: str+ ) -> subprocess.CompletedProcess:+ return subprocess.run(+ [self.sub(p, **extra) for p in command],+ cwd=str(self.bundle),+ capture_output=True,+ text=True,+ errors="replace",+ timeout=timeout,+ )++ def ports(self) -> list[int]:+ text = json.dumps(self.descriptor)+ found = {int(p) for p in re.findall(r"127\.0\.0\.1:(\d+)", text)}+ for app in self.descriptor["apps"]:+ port = (app.get("env") or {}).get("CDB_S04_PORT")+ if port:+ found.add(int(port))+ return sorted(found)++ def clear_leftovers(self) -> None:+ """Disposable desktop: the pack apps by name, and whatever listens on the task's loopback ports."""+ for name in self.KILL_NAMES:+ subprocess.run(["pkill", "-9", "-x", name], capture_output=True)+ subprocess.run(+ ["pkill", "-9", "-f", "node_modules/electron/dist/electron"], capture_output=True+ )+ subprocess.run(["pkill", "-9", "-f", "/opt/google/chrome/chrome"], capture_output=True)+ for port in self.ports():+ subprocess.run(["fuser", "-k", "-9", f"{port}/tcp"], capture_output=True)+ time.sleep(0.5)++ def reset(self) -> None:+ self.clear_leftovers()+ shutil.rmtree(self.workspace, ignore_errors=True)+ sem = self.descriptor["semantics"]["reset"]+ for step in ("setup", "verify"):+ done = self._run(sem[step])+ self.log.append(f"reset.{step} rc={done.returncode}")+ if done.returncode != 0:+ raise RuntimeError(f"reset {step} failed: {(done.stderr or done.stdout)[-400:]}")++ def health_check(self) -> list[dict[str, Any]]:+ checks = []+ for name in ("python", "node"):+ path = sys.executable if name == "python" else shutil.which(name)+ checks.append({"name": f"tool:{name}", "ok": bool(path), "path": path})+ bad = [c["name"] for c in checks if not c["ok"]]+ if bad:+ raise RuntimeError("harness health: missing " + ", ".join(bad))+ return checks++ def brief(self) -> str:+ text = (self.bundle / self.descriptor["semantics"].get("brief", "brief.md")).read_text(+ "utf-8"+ )+ text = text.replace("configured `cua` MCP server", "configured computer-use MCP server")+ return text.strip() + f"\n\nYour workspace directory is `{self.workspace}`.\n"++ def apps(self) -> list[dict[str, Any]]:+ return [a for a in self.descriptor["apps"] if a["kind"] not in SKIP_KINDS]++ def start_apps(self) -> None:+ for app in self.apps():+ env = dict(os.environ)+ env.update({k: self.sub(v) for k, v in (app.get("env") or {}).items()})+ for key in app.get("unset_env", []):+ env.pop(key, None)+ cwd = self.sub(app["cwd"]) if app.get("cwd") else str(self.bundle)+ Path(cwd).mkdir(parents=True, exist_ok=True)+ argv = [self.sub(p) for p in app["command"]]+ for p in argv: # runtime dirs named in the arguments (GnuCash --logto)+ if p.startswith("--logto="):+ Path(p.split("=", 1)[1]).parent.mkdir(parents=True, exist_ok=True)+ out = (self.artifacts / f"app-{app['id']}.log").open("ab")+ proc = subprocess.Popen(+ argv,+ cwd=cwd,+ env=env,+ stdin=subprocess.DEVNULL,+ stdout=out,+ stderr=out,+ start_new_session=True,+ )+ self.procs.append((app, proc))+ ready = (app.get("ready") or {}).get("url")+ if ready:+ self._wait_http(self.sub(ready), float(app["ready"].get("timeout_seconds", 20)))+ self.log.append(f"started {app['id']} pid={proc.pid}")+ time.sleep(1.0)++ @staticmethod+ def _wait_http(url: str, timeout: float) -> None:+ deadline = time.monotonic() + timeout+ while time.monotonic() < deadline:+ try:+ with urllib.request.urlopen(url, timeout=2) as r:+ if r.status == 200:+ return+ except OSError:+ pass+ time.sleep(0.4)+ raise RuntimeError(f"app not ready: {url}")++ def place_windows(self, limit: float = 120.0) -> None:+ pending = [a for a, _ in self.procs if (a.get("window") or {}).get("bounds")]+ deadline = time.monotonic() + limit+ while pending and time.monotonic() < deadline:+ for window in X.visible():+ if window in self.windows.values():+ continue+ title = X.title(window)+ for app in list(pending):+ if title_matches(app["window"], title):+ X.place(window, app["window"]["bounds"])+ self.windows[app["id"]] = window+ self.log.append(f"window {app['id']} id={window} title={title!r}")+ pending.remove(app)+ break+ time.sleep(0.5)+ if pending:+ raise RuntimeError(+ "windows did not appear: "+ + ", ".join(a["id"] for a in pending)+ + f" (visible: {[X.title(w) for w in X.visible()][:20]})"+ )+ time.sleep(1.5)+ for app_id, window in self.windows.items(): # re-apply: some apps resize after mapping+ app = next(a for a in self.apps() if a["id"] == app_id)+ X.place(window, app["window"]["bounds"])+ time.sleep(0.5)++ def evaluate(self, agent_exit_code: int) -> dict[str, Any]:+ result = self.artifacts / "cdb-result.json"+ result.unlink(missing_ok=True)+ try:+ done = self._run(+ self.descriptor["semantics"]["evaluate"],+ timeout=300,+ result=str(result),+ agent_exit_code=str(agent_exit_code),+ )+ (self.artifacts / "cdb-evaluator.stderr").write_text(+ (done.stderr or "")[-4000:], "utf-8"+ )+ except subprocess.TimeoutExpired:+ pass+ if not result.is_file():+ return {+ "passed": False,+ "score": None,+ "error": "evaluator wrote no result",+ "harness_error": "evaluator_no_result",+ }+ data = json.loads(result.read_text("utf-8"))+ checks_raw = (data.get("detail") or {}).get("checks") or data.get("checks") or {}+ checks = {+ n: bool(c.get("passed")) if isinstance(c, dict) else bool(c)+ for n, c in checks_raw.items()+ }+ return {+ "seed": data.get("seed"),+ "passed": bool(data.get("passed")),+ "score": data.get("score"),+ "checks": checks,+ "diagnostics": {+ "raw_keys": sorted(data)[:12],+ **(data.get("diagnostics") if isinstance(data.get("diagnostics"), dict) else {}),+ },+ }++ def snapshot_workspace(self, target: Path) -> None:+ with zipfile.ZipFile(target, "w", zipfile.ZIP_DEFLATED) as z:+ for path in sorted(self.workspace.rglob("*")):+ rel = path.relative_to(self.workspace)+ if path.is_file() and not ({".runtime", "node_modules"} & set(rel.parts)):+ try:+ z.write(path, str(rel))+ except OSError:+ pass++ def stop_apps(self) -> None:+ for _, proc in reversed(self.procs):+ if proc.poll() is None:+ hs.kill_group(proc)+ self.procs.clear()+ self.clear_leftovers()++ def clean_workspace(self) -> None:+ shutil.rmtree(self.workspace, ignore_errors=True)+++# ---------------------------------------------------------------- trial+def start_sentinel(env: dict, out: Path, log: Path, frame, extra: list[str]):+ proc = subprocess.Popen(+ [+ sys.executable,+ str(BENCH / "linux" / "sentinel.py"),+ "--log",+ str(log),+ "--frame",+ ",".join(map(str, frame)),+ *extra,+ ],+ env=env,+ start_new_session=True,+ stdout=(out / "sentinel.log").open("w"),+ stderr=subprocess.STDOUT,+ )+ if not hs.wait_for(lambda: any(ln.get("ev") == "ready" for ln in hs.sentinel_lines(log)), 30):+ return proc, None+ if not hs.wait_for(lambda: X.windows_of(proc.pid), 10):+ return proc, None+ return proc, X.windows_of(proc.pid)[0]+++def bring_sentinel_front(proc: subprocess.Popen, own: int) -> bool:+ def in_front() -> bool:+ stack = X.normal_stack()+ return X.active() == own and bool(stack) and stack[-1] == own++ os.kill(proc.pid, signal.SIGUSR1)+ if hs.wait_for(in_front, 5):+ return True+ X.activate(own)+ os.kill(proc.pid, signal.SIGUSR1)+ return hs.wait_for(in_front, 5)+++def toggle(proc: subprocess.Popen, log: Path, event: str) -> bool:+ os.kill(proc.pid, signal.SIGUSR2)+ return hs.wait_for(lambda: any(ln.get("ev") == event for ln in hs.sentinel_lines(log)), 5)+++def guard_loop(+ G, agent: subprocess.Popen, t0: float, timeout: float, enforce: bool, baseline: set[int]+):+ own_tree: set[int] = set()+ front = G.FrontWatch()+ violations: list[dict] = []+ timed_out = False+ while agent.poll() is None:+ rows = G.process_rows()+ own_tree |= G.descendants(rows, os.getpid())+ for pid, label in G.new_side_doors(rows, baseline, os.getpid(), own_tree):+ violations.append(+ {+ "t_s": round(time.monotonic() - t0, 1),+ "kind": "process",+ "detail": label,+ "pid": pid,+ }+ )+ hs.kill_pid(pid)+ hit = front.sample(X.active_class())+ if hit:+ violations.append(+ {"t_s": round(time.monotonic() - t0, 1), "kind": "front", "detail": hit}+ )+ if violations and enforce:+ hs.kill_group(agent)+ break+ if time.monotonic() - t0 > timeout:+ timed_out = True+ hs.kill_group(agent)+ break+ time.sleep(0.5)+ try:+ agent.wait(timeout=30)+ except subprocess.TimeoutExpired:+ hs.kill_group(agent)+ return violations, front, timed_out+++def read_stream(path: Path) -> list[dict]:+ events = []+ if path.exists():+ for raw in path.read_text(encoding="utf-8", errors="replace").splitlines():+ try:+ events.append(json.loads(raw))+ except json.JSONDecodeError:+ pass+ return events+++def run(args: argparse.Namespace, command: list[str]) -> dict:+ G = hs.hyprland_module("guard")+ spec = hs.task_spec(args.task)+ out: Path = args.out.resolve()+ out.mkdir(parents=True, exist_ok=True)+ seed = cdb_fairness.trial_seed(args.task, args.run_index)+ background = args.condition == "background"+ cdb = LinuxCdbTask(spec, out, seed=seed) if spec.get("kind") == "cdb" else None+ if cdb is not None:+ mode = spec["pack_task"]+ brief = v2.brief_for(cdb.brief().strip(), args.condition)+ else:+ mode = spec["app_args"][spec["app_args"].index("--mode") + 1]+ brief = hs.render_brief(args.task, seed, args.condition)+ (out / "brief.md").write_text(brief, encoding="utf-8")+ row: dict = {+ "task": args.task,+ "run_index": args.run_index,+ "seed": seed,+ "condition": args.condition,+ "mode": mode,+ "platform": "linux-xfce",+ "label": args.label,+ "started": time.time(),+ }+ state, events, log = out / "state.json", out / "events.jsonl", out / "sentinel.jsonl"+ for stale in (state, events, log):+ stale.unlink(missing_ok=True)+ procs: list[subprocess.Popen] = []+ env = {**os.environ, "GDK_BACKEND": "x11", "NO_AT_BRIDGE": "0"}+ try:+ w, h = X.screen()+ row["screen"] = [w, h]+ if w < 1320 or h < 860:+ row["harness_error"] = f"screen too small: {w}x{h}"+ return row+ row["pointer_parked"] = park_pointer()+ if not row["pointer_parked"]["ok"]:+ row["harness_error"] = f"pointer park failed: {row['pointer_parked']}"+ return row+ first_window = None+ if cdb is not None:+ row["workspace"] = str(cdb.workspace)+ try:+ cdb.reset()+ cdb.start_apps()+ cdb.place_windows()+ row["health"] = cdb.health_check()+ except Exception as error: # noqa: BLE001 - setup failure = harness error, never scored+ row["harness_error"] = f"setup: {type(error).__name__}: {str(error)[:600]}"+ return row+ finally:+ row["setup_log"] = list(cdb.log)+ first_window = next(iter(cdb.windows.values()), None)+ else:+ lab = subprocess.Popen(+ [+ sys.executable,+ str(BENCH / "linux" / "benchlab.py"),+ "--mode",+ mode,+ "--seed",+ str(seed),+ "--state",+ str(state),+ "--events",+ str(events),+ ],+ env=env,+ start_new_session=True,+ stdout=(out / "benchlab.log").open("w"),+ stderr=subprocess.STDOUT,+ )+ procs.append(lab)+ if not hs.wait_for(lambda: state.exists() and X.windows_of(lab.pid), 30):+ row["harness_error"] = "BenchLab did not start"+ return row+ first_window = X.windows_of(lab.pid)[0]+ X.place(first_window, dict(zip(("x", "y", "width", "height"), LAB_FRAME)))+ if not background and first_window:+ X.activate(first_window)+ frame = v2.USER_WINDOW_FRAME if background else WITNESS_FRAME+ sentinel, own = start_sentinel(+ env, out, log, frame, ["--user-activity"] if background else []+ )+ procs.append(sentinel)+ if own is None:+ row["harness_error"] = "sentinel did not start"+ return row+ row["sentinel_front_at_start"] = bring_sentinel_front(sentinel, own) if background else None+ if background and not row["sentinel_front_at_start"]:+ st = X.normal_stack()+ row["harness_error"] = (+ f"sentinel could not be brought to the front: own={own} active={X.active()} "+ f"({X.title(X.active())!r}) top={[X.title(w) for w in st[-3:]]} own_in_stack={own in st}"+ )+ return row+ park_pointer() # the pointer may have moved while windows were placed+ row["frames"] = X.frames()+ time.sleep(1.0)+ toggle(sentinel, log, "armed")+ baseline = {pid for pid, _, _ in G.process_rows()}+ t0 = time.monotonic()+ argv = [+ a.replace("{brief}", str(out / "brief.md"))+ .replace("{out}", str(out))+ .replace("{seed}", str(seed))+ .replace("{mode}", mode)+ for a in command+ ]+ agent = subprocess.Popen(+ argv,+ cwd=str(out),+ env={**os.environ, "CDB_BRIEF_FILE": str(out / "brief.md")},+ start_new_session=True,+ )+ procs.append(agent)+ violations, front, timed_out = guard_loop(+ G, agent, t0, args.timeout, args.enforce, baseline+ )+ row["agent_exit"] = agent.returncode+ row["agent_wall_s"] = round(time.monotonic() - t0, 1)+ row["timed_out"] = timed_out+ toggle(sentinel, log, "disarmed")+ time.sleep(0.3)+ result = out / "result.json"+ if cdb is not None:+ evaluation = cdb.evaluate(124 if timed_out else int(agent.returncode or 0))+ result.write_text(json.dumps(evaluation, indent=1), encoding="utf-8")+ try:+ cdb.snapshot_workspace(out / "workspace-final.zip")+ except OSError:+ pass+ if evaluation.get("harness_error"):+ row["harness_error"] = f"evaluate: {evaluation['harness_error']}"+ else:+ subprocess.run(+ [+ sys.executable,+ str(BENCH / "probes" / args.task / "evaluate.py"),+ "--seed",+ str(seed),+ "--state",+ str(state),+ "--events",+ str(events),+ "--result",+ str(result),+ ],+ capture_output=True,+ timeout=60,+ )+ evaluation = json.loads(result.read_text(encoding="utf-8")) if result.exists() else {}+ soft: list = []+ hard, mentions = G.scan_events(read_stream(out / "agent-transcript.jsonl"))+ soft += mentions+ violations += [{"t_s": None, "kind": "tool_input", "detail": x} for x in hard]+ disturbance = summarize_sentinel.summarize(str(log))+ armed = next((ln for ln in hs.sentinel_lines(log) if ln.get("ev") == "armed"), {})+ frontmost = (armed.get("front") or {}).get("bid") == SENTINEL_ID+ passed = bool(evaluation.get("passed")) and not violations+ bg = v2.background_score(disturbance, frontmost)+ row.update(+ evaluation={k: evaluation.get(k) for k in ("passed", "score", "checks", "diagnostics")},+ checker_passed=bool(evaluation.get("passed")),+ gui_only={+ "violation": bool(violations),+ "violations": violations[:20],+ "mentions": sorted(set(map(str, soft)))[:20],+ "frontmost_seen": sorted(front.seen),+ },+ passed=passed,+ disturbance=disturbance,+ background=bg,+ a15=v2.background_condition_score(bg, disturbance, passed, args.condition),+ a22=v2.multiplayer_score(disturbance, bg["measured"], passed, args.condition),+ )+ return row+ finally:+ for proc in procs:+ if proc.poll() is None:+ hs.kill_group(proc)+ if cdb is not None:+ cdb.stop_apps()+ cdb.clean_workspace()+ row["finished"] = time.time()+ (out / "row.json").write_text(json.dumps(row, indent=1), encoding="utf-8")+++def main(argv: list[str]) -> int:+ if "--" not in argv:+ print(+ "usage: xfce_session.py --task ID --run-index N --condition C --out DIR -- COMMAND...",+ file=sys.stderr,+ )+ return 2+ split = argv.index("--")+ parser = argparse.ArgumentParser()+ parser.add_argument("--task", choices=TASKS, required=True)+ parser.add_argument("--run-index", type=int, required=True)+ parser.add_argument("--condition", choices=v2.CONDITIONS, required=True)+ parser.add_argument("--out", type=Path, required=True)+ parser.add_argument("--label", default="")+ parser.add_argument("--timeout", type=float, default=360.0)+ parser.add_argument("--no-enforce", dest="enforce", action="store_false")+ args = parser.parse_args(argv[:split])+ command = argv[split + 1 :]+ if not os.environ.get("DISPLAY") or not command:+ print("xfce_session.py needs an X11 session (DISPLAY) and a command", file=sys.stderr)+ return 2+ row = run(args, command)+ print(+ json.dumps(+ {k: row.get(k) for k in ("task", "seed", "condition", "passed", "harness_error")}+ )+ )+ return 0 if not row.get("harness_error") else 3+++if __name__ == "__main__":+ sys.exit(main(sys.argv[1:]))diff --git a/libs/cua-driver-bench/automated-eval/macos_bench/tests/test_xfce_session.py b/libs/cua-driver-bench/automated-eval/macos_bench/tests/test_xfce_session.pynew file mode 100644index 0000000000..f78203e0c9--- /dev/null+++ b/libs/cua-driver-bench/automated-eval/macos_bench/tests/test_xfce_session.py@@ -0,0 +1,49 @@+# SPDX-License-Identifier: FSL-1.1-MIT+# Copyright (c) 2026 Cua AI, Inc.++"""Linux XFCE launcher contracts that hold on every OS (no desktop, no model): the ten tasks, the pack tasks'+Linux launch descriptors, placeholder substitution and window-title matching."""++from __future__ import annotations++import json+import sys+import unittest+from pathlib import Path++HERE = Path(__file__).resolve().parent+BENCH = HERE.parent+sys.path.insert(0, str(BENCH))+sys.path.insert(0, str(BENCH / "launch"))+import xfce_session as XS # noqa: E402+++class XfceSessionTest(unittest.TestCase):+ def test_ten_tasks(self) -> None:+ self.assertEqual(len(XS.TASKS), 10)+ self.assertEqual(set(XS.TASKS), set(XS.CDB_TASKS) | set(XS.LAB_TASKS))++ def test_pack_tasks_have_linux_descriptors(self) -> None:+ for task in XS.CDB_TASKS:+ spec = XS.hs.task_spec(task)+ self.assertEqual(spec["kind"], "cdb")+ path = XS.TASKPACK / "tasks" / spec["pack_task"] / "platform" / "launch.linux.json"+ descriptor = json.loads(path.read_text("utf-8"))+ self.assertTrue(descriptor["apps"])+ for step in ("setup", "verify"):+ self.assertTrue(descriptor["semantics"]["reset"][step])+ self.assertTrue(descriptor["semantics"]["evaluate"])++ def test_substitute_leaves_unknown_names(self) -> None:+ self.assertEqual(XS.substitute("${a}/${b}", {"a": "x"}), "x/${b}")++ def test_title_matches(self) -> None:+ self.assertTrue(XS.title_matches({"title": "Calc"}, "Calc"))+ self.assertTrue(XS.title_matches({"title": "Calc"}, "Calc - file.ods"))+ self.assertFalse(XS.title_matches({"title": "Calc"}, "Calculator"))+ self.assertTrue(XS.title_matches({"title_contains": "mail"}, "Copperfield Mail"))+ self.assertFalse(XS.title_matches({}, "anything"))+++if __name__ == "__main__":+ unittest.main()