Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.
Rust agentai-agentapplecomputer-usecomputer-use-agentcontainerizationcuadesktop-automationhacktoberfestlumemacosmanusoperatorswiftvirtualizationvirtualization-frameworkwindowswindows-sandbox
feat(cua-driver-bench): run the BenchLab tasks on Hyprland (Wayland), e.g. Omarchy (#6630)
Francesco Bonaccic643cb396865
diff --git a/libs/cua-driver-bench/README.md b/libs/cua-driver-bench/README.mdindex b956816545..0ab2575e9f 100644--- a/libs/cua-driver-bench/README.md+++ b/libs/cua-driver-bench/README.md@@ -43,8 +43,8 @@ Ten tasks in three categories. Each task has a `PROVENANCE.md` next to it.The CDB-G tasks are the GUI-only variants of pack tasks `cdb-s02` to `cdb-s04`:the same files, with no coding tools offered. The MB and IR tasks run inBenchLab, a small AppKit test app built from source in this directory (with a-Linux port in `automated-eval/macos_bench/linux` and a Windows port in-`automated-eval/macos_bench/windows`). A fourth category, Web apps &+Linux port in `automated-eval/macos_bench/linux`, which `automated-eval/macos_bench/hyprland`+runs on Hyprland (Wayland), and a Windows port in `automated-eval/macos_bench/windows`). A fourth category, Web apps &forms, has no Bench v2 task yet.### Platforms and conditions@@ -356,6 +356,49 @@ Codex computer use is not available on Linux yet([compatibility and limitations](https://learn.chatgpt.com/docs/linux/linux-app#compatibility-and-limitations)).`tests/test_linux_session.py` and `tests/test_linux_tools.py` check the session and the tools without a model.+### Linux on Hyprland (Wayland; the BenchLab tasks)++The Hyprland port runs the same GTK3 BenchLab (`linux/benchlab.py`) as a native Wayland client, with a+Wayland sentinel and GUI-only guard in [`automated-eval/macos_bench/hyprland`](automated-eval/macos_bench/hyprland).+It is built for Omarchy (Arch Linux with Hyprland 0.55 or later and a Lua config), for example Cua's+`ghcr.io/trycua/omarchy:edge` image. Seeds, the state file, the event log, the checkers and the sentinel+summary are the same as on macOS, Linux/X11 and Windows.++You need Python 3.11 or later with PyGObject, GTK 3 and AT-SPI, and `wl-clipboard`. Load+`hyprland/bench.lua` last from `~/.config/hypr/hyprland.lua` (it sets 1920x1080 so the shared window frames+fit), then run each trial as the desktop user with the session's environment (`WAYLAND_DISPLAY`,+`HYPRLAND_INSTANCE_SIGNATURE`, `XDG_RUNTIME_DIR`, `DBUS_SESSION_BUS_ADDRESS`). One trial:++```sh+python3 launch/hyprland_session.py --task IR-02 --run-index 0 --condition background --out ~/bench/IR-02-r0 -- <agent command>+```++Where Hyprland differs:++* **Placement.** A Wayland client cannot place its own windows, so the launcher floats BenchLab and the+ sentinel and gives them their frames through Hyprland's IPC socket. Window rules (applied on every trial)+ keep BenchLab's windows from taking focus when they open and open its dialogs over BenchLab's top-left+ corner, clear of the "User activity" window, as a sheet hangs from BenchLab on macOS.+* **Sentinel.** The compositor state comes from Hyprland's IPC: the active window, the stacking order of+ `clients` (a window counts as raised when it comes after the user window, draws over it and overlaps it;+ top and overlay layer-shell surfaces that overlap it count too, named after their owning process), the+ real pointer (`cursorpos`), the active and special workspaces and full-screen windows (Space), and a+ second window of the sentinel's own process (dialog). The clipboard is watched with `wl-paste --watch`+ (data-control), so a change counts whether or not the user window has focus. Key focus, typed keys,+ clicks and scrolls are GTK events on the user window, as on X11.+* **GUI-only guard.** A terminal emulator (Alacritty, Ghostty, kitty, foot, ...) started outside the+ runner's own processes, or a terminal or Omarchy TUI window active for three samples, fails and stops the+ trial. Shells alone are not counted (tool servers reached over SSH run under one). Tool inputs are also+ scanned for `hyprctl`, compositor `exec` dispatches, uwsm and Omarchy launchers, and terminal names.+* **Brief.** "on your Mac" reads "on this Linux computer".++`hyprland/oracle.py` solves each task through the GUI with real input (a uinput pointer and keyboard; the+desktop user needs access to `/dev/uinput` while it runs). It validates the port: the oracle passes every+checker, doing nothing fails every task and leaves the user undisturbed, and a deliberate focus change,+raise, pointer move, clipboard change, keystroke or workspace switch is each seen.+`tests/test_hyprland_tools.py` checks the guard, the brief, the descriptors and the window rules without a+desktop or a model.+### WindowsThe Windows port of BenchLab and the sentinel is in@@ -476,6 +519,7 @@ Where the multi-app tasks differ on Windows:cd automated-eval/macos_benchpython3 -m pytest tests probes/tests swift/tests # harness, checkers, sentinel (no GUI, no model)python -m pytest tests/test_windows_desktop.py # on a Windows desktop: BenchLab, sentinel, guard, oracle+python3 -m pytest tests/test_hyprland_tools.py # Hyprland port contracts (no desktop needed)cd ../../taskpack/tasks/shared/cdb-s02 && python3 -m unittest discover -s testscd ../../../../fairness && python3 -m unittest discover -s tests # fairness and health checkspython3 ../automated-eval/macos_bench/tools/taskset.py check # manifest, selectors, pins and docs agreediff --git a/libs/cua-driver-bench/automated-eval/macos_bench/hyprland/bench.lua b/libs/cua-driver-bench/automated-eval/macos_bench/hyprland/bench.luanew file mode 100644index 0000000000..15b74afd54--- /dev/null+++ b/libs/cua-driver-bench/automated-eval/macos_bench/hyprland/bench.lua@@ -0,0 +1,22 @@+-- Cua Driver Bench v2, Hyprland port: the desktop settings the BenchLab trials need.+-- Load it last from ~/.config/hypr/hyprland.lua: dofile(os.getenv("HOME") .. "/.config/hypr/bench.lua")+-- (Hyprland 0.55+ with a Lua config, as on Omarchy edge). launch/hyprland_session.py applies the same window+-- rules on every trial, so only the screen size needs this file.++-- 1920x1080 so the shared frames fit: BenchLab at 40,40 760x540, the User activity window at 420,340 900x520.+hl.monitor({ output = "", mode = "1920x1080@60", position = "0x0", scale = 1 })++-- BenchLab and the sentinel float at the frames the session gives them (Hyprland would tile them).+-- BenchLab's windows never take focus when they open, and its dialogs open over BenchLab's top-left corner,+-- clear of the User activity window: an app working in the background does not cover the user's window with+-- its own dialogs (on macOS a sheet hangs from BenchLab's title bar; on Windows the dialog opens just above+-- BenchLab).+hl.window_rule({ name = "cdb-benchlab", match = { class = "^BenchLab$" }, float = true, no_initial_focus = true })+hl.window_rule({+ name = "cdb-benchlab-dialogs",+ match = { class = "^BenchLab$", title = "^(Allow BenchLab.*|Save changes.*|Cookie preferences)$" },+ float = true,+ no_initial_focus = true,+ move = "80 120",+})+hl.window_rule({ name = "cdb-sentinel", match = { class = "^BenchSentinel$" }, float = true })diff --git a/libs/cua-driver-bench/automated-eval/macos_bench/hyprland/guard.py b/libs/cua-driver-bench/automated-eval/macos_bench/hyprland/guard.pynew file mode 100644index 0000000000..d4d4ba1f64--- /dev/null+++ b/libs/cua-driver-bench/automated-eval/macos_bench/hyprland/guard.py@@ -0,0 +1,178 @@+# SPDX-License-Identifier: FSL-1.1-MIT+# Copyright (c) 2026 Cua AI, Inc.++"""The GUI-only guard on Hyprland (the counterpart of windows/guard.py and the X11 rules in bench_v2.py).++A trial fails, and is stopped, when the agent reaches a shell through the GUI:+* a terminal emulator starts during the trial outside the runner's own process tree (processes already running+ when the trial started are ignored), or+* a terminal window is the active window for FRONT_GRACE_SAMPLES samples in a row, or+* a computer-use tool input carries code that runs outside the GUI (bench_v2.EXEC_PATTERNS, plus the Hyprland+ patterns below: dispatching `exec` to the compositor, Omarchy's and uwsm's launchers, terminal names).++Shells alone are not counted as processes: the tool servers reach the desktop through ssh, which runs them under a+shell. A shell the agent could type into needs a terminal window, which is caught.++The pure functions take process rows and window classes as data, so they are tested on every OS.+"""++from __future__ import annotations++import os+import re+import sys+from pathlib import Path+from typing import Any, Iterable++sys.path.insert(0, str(Path(__file__).resolve().parents[1]))+import bench_v2 as v2 # noqa: E402++# Process names (comm, lower case) of terminal emulators.+TERMINALS = {+ "alacritty": "Alacritty",+ "kitty": "kitty",+ "foot": "foot",+ "footclient": "foot",+ "ghostty": "Ghostty",+ "wezterm": "WezTerm",+ "wezterm-gui": "WezTerm",+ "xterm": "xterm",+ "uxterm": "xterm",+ "urxvt": "urxvt",+ "st": "st",+ "konsole": "Konsole",+ "gnome-terminal": "GNOME Terminal",+ "gnome-terminal-": "GNOME Terminal",+ "kgx": "GNOME Console",+ "ptyxis": "Ptyxis",+ "ptyxis-agent": "Ptyxis",+ "xfce4-terminal": "Xfce Terminal",+ "terminator": "Terminator",+ "tilix": "Tilix",+ "rio": "Rio",+ "warp": "Warp",+}+# Window classes (lower case) of terminal windows, Omarchy's floating and tiled TUI classes included.+TERMINAL_CLASSES = {+ "alacritty": "Alacritty",+ "kitty": "kitty",+ "foot": "foot",+ "com.mitchellh.ghostty": "Ghostty",+ "ghostty": "Ghostty",+ "org.wezfurlong.wezterm": "WezTerm",+ "xterm": "xterm",+ "urxvt": "urxvt",+ "st-256color": "st",+ "org.kde.konsole": "Konsole",+ "org.gnome.terminal": "GNOME Terminal",+ "org.gnome.console": "GNOME Console",+ "org.gnome.ptyxis": "Ptyxis",+ "xfce4-terminal": "Xfce Terminal",+ "tui.float": "Omarchy TUI",+ "tui.tile": "Omarchy TUI",+}+# Text in a tool input that runs code outside the GUI on Hyprland/Omarchy.+HYPRLAND_EXEC_PATTERNS = (+ "hyprctl",+ "dispatch exec",+ "uwsm app",+ "uwsm-app",+ "omarchy-launch",+ "xdg-terminal-exec",+ "systemd-run",+ "sh -c",+)+# Terminal names in a tool input (launching one opens a shell): whole words only.+TERMINAL_NAME_RE = re.compile(+ r"\b(alacritty|ghostty|kitty|foot|footclient|wezterm|xterm|konsole|gnome-terminal|kgx|ptyxis)\b"+)+FRONT_GRACE_SAMPLES = v2.FRONT_GRACE_SAMPLES+++def terminal_of(name: str) -> str | None:+ return TERMINALS.get(name.lower())+++def process_rows() -> list[tuple[int, int, str]]:+ """(pid, ppid, comm) for every process."""+ rows = []+ for entry in os.listdir("/proc"):+ if not entry.isdigit():+ continue+ try:+ stat = Path("/proc", entry, "stat").read_text()+ except OSError:+ continue+ name = stat[stat.index("(") + 1 : stat.rindex(")")]+ ppid = int(stat[stat.rindex(")") + 2 :].split()[1])+ rows.append((int(entry), ppid, name))+ return rows+++def descendants(rows: Iterable[tuple[int, int, str]], root: int) -> set[int]:+ children: dict[int, list[int]] = {}+ for pid, ppid, _ in rows:+ if pid != ppid:+ children.setdefault(ppid, []).append(pid)+ out, todo = set(), [root]+ while todo:+ pid = todo.pop()+ if pid in out:+ continue+ out.add(pid)+ todo.extend(children.get(pid, []))+ return out+++def new_side_doors(+ rows: list[tuple[int, int, str]], baseline: set[int], runner: int, own: set[int] | None = None+) -> list[tuple[int, str]]:+ own = descendants(rows, runner) | (own if own is not None else set())+ return [+ (pid, terminal_of(name))+ for pid, _, name in rows+ if pid not in baseline and pid not in own and terminal_of(name)+ ]+++def scan_tool_use(name: str, tool_input: Any) -> tuple[list[str], list[str]]:+ hard, soft = v2.scan_tool_use(name, tool_input)+ if name.startswith("mcp__"):+ text = "\n".join(v2._strings(tool_input)).lower()+ hard += [f"{name}:{p}" for p in HYPRLAND_EXEC_PATTERNS if p in text]+ hard += [f"{name}:{m}" for m in TERMINAL_NAME_RE.findall(text)]+ return sorted(set(hard)), soft+++def scan_events(events: list[dict[str, Any]]) -> tuple[list[str], list[str]]:+ hard: set[str] = set()+ soft: set[str] = set()+ for event in events:+ for name, tool_input in v2.tool_uses(event):+ h, s = scan_tool_use(name, tool_input)+ hard.update(h)+ soft.update(s)+ return sorted(hard)[:20], sorted(soft)[:20]+++class FrontWatch:+ """A terminal window active for FRONT_GRACE_SAMPLES samples in a row is a violation."""++ def __init__(self) -> None:+ self.run = 0+ self.seen: set[str] = set()++ def sample(self, window_class: str) -> str | None:+ name = (window_class or "").lower()+ if name:+ self.seen.add(name)+ label = TERMINAL_CLASSES.get(name) or (+ "Omarchy TUI" if name.startswith("org.omarchy.") else None+ )+ if label:+ self.run += 1+ if self.run >= FRONT_GRACE_SAMPLES:+ return label+ else:+ self.run = 0+ return Nonediff --git a/libs/cua-driver-bench/automated-eval/macos_bench/hyprland/hypr.py b/libs/cua-driver-bench/automated-eval/macos_bench/hyprland/hypr.pynew file mode 100644index 0000000000..83e6e6a781--- /dev/null+++ b/libs/cua-driver-bench/automated-eval/macos_bench/hyprland/hypr.py@@ -0,0 +1,110 @@+# SPDX-License-Identifier: FSL-1.1-MIT+# Copyright (c) 2026 Cua AI, Inc.++"""A small Hyprland IPC client (the request socket `hyprctl` uses), for the Hyprland port of BenchLab.++The session, the sentinel and the guard read the compositor through it: clients (in stacking order), the active+window and workspace, the cursor position, and dispatchers for window placement.+"""++from __future__ import annotations++import json+import os+import socket+from pathlib import Path+from typing import Any+++def socket_path() -> Path:+ sig = os.environ.get("HYPRLAND_INSTANCE_SIGNATURE", "")+ runtime = os.environ.get("XDG_RUNTIME_DIR") or f"/run/user/{os.getuid()}"+ if not sig:+ found = sorted(+ Path(runtime, "hypr").glob("*/.socket.sock"), key=lambda p: p.stat().st_mtime+ )+ if not found:+ raise RuntimeError("no Hyprland instance (HYPRLAND_INSTANCE_SIGNATURE is unset)")+ return found[-1]+ return Path(runtime, "hypr", sig, ".socket.sock")+++def request(command: str, timeout: float = 8.0) -> str:+ with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as conn:+ conn.settimeout(timeout)+ conn.connect(str(socket_path()))+ conn.sendall(command.encode())+ chunks = []+ while True:+ chunk = conn.recv(65536)+ if not chunk:+ break+ chunks.append(chunk)+ return b"".join(chunks).decode(errors="replace")+++def query(what: str) -> Any:+ return json.loads(request(f"j/{what}") or "null")+++def clients() -> list[dict[str, Any]]:+ return [c for c in query("clients") or [] if c.get("mapped", True)]+++def active_window() -> dict[str, Any]:+ return query("activewindow") or {}+++def active_workspace() -> dict[str, Any]:+ return query("activeworkspace") or {}+++def cursor() -> tuple[float, float]:+ pos = query("cursorpos") or {}+ return float(pos.get("x", 0)), float(pos.get("y", 0))+++def dispatch(expression: str) -> str:+ """Run one dispatcher. Hyprland 0.55+ with a Lua config takes Lua (`hl.dsp.window.close()`)."""+ return request("dispatch " + expression)+++def window_ref(address: str) -> str:+ return f'window = "address:{address}"'+++def by_pid(pid: int) -> list[dict[str, Any]]:+ return [c for c in clients() if c.get("pid") == pid]+++def place(address: str, frame: tuple[int, int, int, int]) -> None:+ """Float the window and give it an exact frame (x, y, w, h) in layout coordinates."""+ x, y, w, h = frame+ ref = window_ref(address)+ dispatch(f'hl.dsp.window.float({{ action = "on", {ref} }})')+ dispatch(f"hl.dsp.window.resize({{ x = {w}, y = {h}, {ref} }})")+ dispatch(f"hl.dsp.window.move({{ x = {x}, y = {y}, {ref} }})")+++def focus(address: str) -> None:+ dispatch(f"hl.dsp.focus({{ {window_ref(address)} }})")+++def raise_to_top(address: str) -> None:+ dispatch(f'hl.dsp.window.alter_zorder({{ mode = "top", {window_ref(address)} }})')+++def overlaps(a: dict[str, Any], b: dict[str, Any]) -> bool:+ (ax, ay), (aw, ah) = a.get("at", (0, 0)), a.get("size", (0, 0))+ (bx, by), (bw, bh) = b.get("at", (0, 0)), b.get("size", (0, 0))+ return ax < bx + bw and bx < ax + aw and ay < by + bh and by < ay + ah+++def identity(client: dict[str, Any]) -> str:+ """The name a window is reported under: its class (the app id), else its title, else its address."""+ return (+ client.get("class")+ or client.get("initialClass")+ or client.get("title")+ or client.get("address", "")+ )diff --git a/libs/cua-driver-bench/automated-eval/macos_bench/hyprland/oracle.py b/libs/cua-driver-bench/automated-eval/macos_bench/hyprland/oracle.pynew file mode 100644index 0000000000..0b5c5dc3c4--- /dev/null+++ b/libs/cua-driver-bench/automated-eval/macos_bench/hyprland/oracle.py@@ -0,0 +1,373 @@+# SPDX-License-Identifier: FSL-1.1-MIT+# Copyright (c) 2026 Cua AI, Inc.++"""Scripted oracle for the BenchLab modes on Hyprland: solves a task through the GUI with real input.++Used only to validate the port (each task solvable, its checker passes); never a bench setup. It finds controls+through AT-SPI (window-relative extents plus the window's frame from Hyprland) and acts in the foreground with a+uinput device: an absolute pointer with buttons and a keyboard, the same kernel path as a USB mouse and keyboard.+The desktop user needs read-write access to /dev/uinput while it runs.++Usage: oracle.py --mode MODE --seed N [--disturb KIND]+`--disturb KIND` instead performs one deliberate disturbance of the user window (validates the sentinel):+focus, raise, pointer, clipboard, typing, space or all.+"""++from __future__ import annotations++import argparse+import ctypes+import fcntl+import os+import struct+import subprocess+import sys+import time+from pathlib import Path++import gi++gi.require_version("Atspi", "2.0")+from gi.repository import Atspi # noqa: E402++sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "probes" / "_common"))+sys.path.insert(0, str(Path(__file__).resolve().parent))+import benchlab_common as C # noqa: E402+import hypr # noqa: E402++EV_SYN, EV_KEY, EV_ABS = 0, 1, 3+ABS_X, ABS_Y = 0, 1+BTN_LEFT = 0x110+UI_SET_EVBIT, UI_SET_KEYBIT, UI_SET_ABSBIT = 0x40045564, 0x40045565, 0x40045567+UI_DEV_CREATE, UI_DEV_DESTROY = 0x5501, 0x5502+KEYS = {+ "ctrl": 29,+ "a": 30,+ "backspace": 14,+ "down": 108,+ "up": 103,+ "enter": 28,+ "tab": 15,+ "space": 57,+}+++class Input:+ """A uinput tablet-style pointer (absolute, mapped to the whole layout) and keyboard."""++ def __init__(self) -> None:+ width, height = self.layout()+ self.size = (width, height)+ self.fd = os.open("/dev/uinput", os.O_WRONLY | os.O_NONBLOCK)+ fcntl.ioctl(self.fd, UI_SET_EVBIT, EV_KEY)+ fcntl.ioctl(self.fd, UI_SET_EVBIT, EV_ABS)+ for code in (BTN_LEFT, BTN_LEFT + 1, *KEYS.values()):+ fcntl.ioctl(self.fd, UI_SET_KEYBIT, code)+ fcntl.ioctl(self.fd, UI_SET_ABSBIT, ABS_X)+ fcntl.ioctl(self.fd, UI_SET_ABSBIT, ABS_Y)+ absmax = [0] * 64+ absmax[ABS_X], absmax[ABS_Y] = width - 1, height - 1+ dev = struct.pack("80sHHHHi", b"cdb-oracle", 0x03, 0x1234, 0x5678, 1, 0)+ dev += struct.pack("64i", *absmax) + struct.pack("64i", *([0] * 64)) * 3+ os.write(self.fd, dev)+ fcntl.ioctl(self.fd, UI_DEV_CREATE)+ time.sleep(1.0) # the compositor picks the device up++ @staticmethod+ def layout() -> tuple[int, int]:+ monitors = hypr.query("monitors") or []+ return (+ max(m["x"] + m["width"] for m in monitors),+ max(m["y"] + m["height"] for m in monitors),+ )++ def emit(self, kind: int, code: int, value: int) -> None:+ os.write(self.fd, struct.pack("llHHi", 0, 0, kind, code, value))++ def sync(self) -> None:+ self.emit(EV_SYN, 0, 0)++ def move(self, x: float, y: float) -> None:+ self.emit(EV_ABS, ABS_X, int(x))+ self.emit(EV_ABS, ABS_Y, int(y))+ self.sync()+ time.sleep(0.03)++ def button(self, down: bool) -> None:+ self.emit(EV_KEY, BTN_LEFT, 1 if down else 0)+ self.sync()+ time.sleep(0.05)++ def click(self, x: float, y: float) -> None:+ self.move(x, y)+ time.sleep(0.15)+ self.button(True)+ self.button(False)+ time.sleep(0.3)++ def key(self, name: str, *mods: str) -> None:+ for m in mods:+ self.emit(EV_KEY, KEYS[m], 1)+ self.emit(EV_KEY, KEYS[name], 1)+ self.sync()+ self.emit(EV_KEY, KEYS[name], 0)+ for m in mods:+ self.emit(EV_KEY, KEYS[m], 0)+ self.sync()+ time.sleep(0.05)++ def close(self) -> None:+ fcntl.ioctl(self.fd, UI_DEV_DESTROY)+ os.close(self.fd)+++IN: Input | None = None+++def type_text(text: str) -> None:+ # wtype: Wayland virtual-keyboard input to the focused surface (any character, no keymap table here).+ subprocess.run(["wtype", "-d", "20", text], check=True, timeout=60)+ time.sleep(0.2)+++def app() -> Atspi.Accessible:+ desktop = Atspi.get_desktop(0)+ for _ in range(50):+ for i in range(desktop.get_child_count()):+ child = desktop.get_child_at_index(i)+ if child is not None and child.get_name() in ("BenchLab", "benchlab.py"):+ return child+ time.sleep(0.2)+ raise RuntimeError("BenchLab is not on the accessibility bus")+++def walk(node: Atspi.Accessible):+ yield node+ for i in range(node.get_child_count()):+ child = node.get_child_at_index(i)+ if child is not None:+ yield from walk(child)+++def toplevel(title_prefix: str) -> tuple[Atspi.Accessible, dict]:+ for _ in range(80):+ root = app()+ for i in range(root.get_child_count()):+ win = root.get_child_at_index(i)+ if win is not None and win.get_name().startswith(title_prefix):+ frames = [c for c in hypr.clients() if c.get("title", "").startswith(title_prefix)]+ if frames:+ return win, frames[-1]+ time.sleep(0.25)+ raise RuntimeError(f"no window {title_prefix!r}")+++ROLE_ALIASES = {"push button": ("push button", "button"), "check box": ("check box", "checkbox")}+++def find(win: Atspi.Accessible, role: str, name: str, nth: int = 0) -> Atspi.Accessible:+ roles = ROLE_ALIASES.get(role, (role,))+ hits = [+ n for n in walk(win)+ if n.get_role_name() in roles and n.get_name() == name and n.get_state_set().contains(Atspi.StateType.SHOWING)+ ] # fmt: skip+ if len(hits) <= nth:+ raise RuntimeError(f"no {role} {name!r}")+ return hits[nth]+++def center(node: Atspi.Accessible, frame: dict) -> tuple[float, float]:+ box = node.get_extents(Atspi.CoordType.WINDOW)+ x0, y0 = frame["at"]+ return x0 + box.x + box.width / 2, y0 + box.y + box.height / 2+++def lab(mode: str) -> tuple[Atspi.Accessible, dict]:+ win, frame = toplevel(f"BenchLab — {mode}")+ hypr.focus(frame["address"])+ hypr.raise_to_top(frame["address"])+ time.sleep(0.4)+ return win, frame+++def click(node: Atspi.Accessible, frame: dict) -> None:+ IN.click(*center(node, frame))+++def button(mode: str, title: str) -> None:+ win, frame = lab(mode)+ click(find(win, "push button", title), frame)+++def type_into(mode: str, name: str, value: str) -> None:+ win, frame = lab(mode)+ click(find(win, "text", name), frame)+ IN.key("a", "ctrl")+ IN.key("backspace")+ type_text(str(value))+++def choose(+ mode: str, name: str, choices: list[str], value: str, current: str | None = None+) -> None:+ """Open the combo box (its popup opens on the current item) and step to VALUE with the arrow keys."""+ win, frame = lab(mode)+ click(find(win, "combo box", name), frame)+ time.sleep(0.5)+ steps = choices.index(value) - choices.index(current or choices[0])+ for _ in range(abs(steps)):+ IN.key("down" if steps > 0 else "up")+ time.sleep(0.1)+ IN.key("enter")+ time.sleep(0.4)+++def dialog_button(title_prefix: str, label: str) -> None:+ win, frame = toplevel(title_prefix)+ hypr.focus(frame["address"])+ time.sleep(0.3)+ click(find(win, "push button", label), frame)+++def drag(src: tuple[float, float], dst: tuple[float, float]) -> None:+ IN.move(*src)+ time.sleep(0.2)+ IN.button(True)+ for i in range(1, 21):+ IN.move(src[0] + (dst[0] - src[0]) * i / 20, src[1] + (dst[1] - src[1]) * i / 20)+ time.sleep(0.03)+ time.sleep(0.3)+ IN.button(False)+ time.sleep(0.6)+++def listdrag(seed: int) -> None:+ p = C.derive_listdrag(seed)+ order = list(p["initial"])+ for index, item in enumerate(p["target"]):+ source = order.index(item)+ if source == index:+ continue+ win, frame = lab("listdrag")+ rows = [find(win, "label", name) for name in order]+ drag(center(rows[source], frame), center(rows[index], frame))+ order.pop(source)+ order.insert(index, item)+ button("listdrag", "Done")+++def hover(seed: int) -> None:+ p = C.derive_hover(seed)+ win, frame = lab("hover")+ x, y = center(find(win, "label", "Hover here to reveal actions"), frame)+ IN.move(x, y)+ time.sleep(0.6)+ IN.move(x + 2, y + 20)+ time.sleep(0.6)+ win, frame = lab("hover")+ click(find(win, "push button", p["target"]), frame)+++def tooltip(seed: int) -> None:+ p = C.derive_tooltip(seed)+ win, frame = lab("tooltip")+ x, y = center(find(win, "label", p["target"]), frame)+ IN.move(x - 3, y)+ time.sleep(0.2)+ IN.move(x, y)+ time.sleep(2.0)+ type_into("tooltip", "Code", p["target_code"])+ button("tooltip", "Submit")+++def irmodal(seed: int) -> None:+ p = C.derive_irmodal(seed)+ type_into("irmodal", "Name", p["name"])+ type_into("irmodal", "Email", p["email"])+ button("irmodal", "Next")+ dialog_button("Allow BenchLab", "Don’t Allow")+ choose("irmodal", "Team", C.IR_TEAMS, p["team"])+ type_into("irmodal", "Seats", str(p["seats"]))+ button("irmodal", "Submit")+++def irbanner(seed: int) -> None:+ p = C.derive_irbanner(seed)+ type_into("irbanner", "Quantity", str(p["quantity"]))+ time.sleep(0.6)+ button("irbanner", "Dismiss")+ choose("irbanner", "Priority", C.PRIORITIES, p["priority"])+ button("irbanner", "Apply")+++def irunsaved(seed: int) -> None:+ p = C.derive_irunsaved(seed)+ statuses = [C.IR_INITIAL_STATUS, *C.IR_STATUS_WORDS]+ button("irunsaved", p["titles"][p["status_index"]])+ choose("irunsaved", "Status", statuses, p["status"])+ button("irunsaved", p["titles"][p["rename_index"]])+ dialog_button("Save changes", "Save")+ type_into("irunsaved", "Title", p["new_title"])+ button("irunsaved", "Done")+ dialog_button("Save changes", "Save")+++def irconsent(seed: int) -> None:+ p = C.derive_irconsent(seed)+ type_into("irconsent", "Email", p["email"])+ dialog_button("Cookie preferences", "Reject optional")+ type_into(+ "irconsent", "Email", p["email"]+ ) # keys typed while the modal prompt was open went nowhere+ choose("irconsent", "Plan", C.IR_PLANS, p["plan"])+ win, frame = lab("irconsent")+ box = find(win, "check box", "Email digest")+ if box.get_state_set().contains(Atspi.StateType.CHECKED) != p["digest"]:+ click(box, frame)+ button("irconsent", "Sign up")+++def disturb(kind: str) -> None:+ """One deliberate disturbance of the user window, of the given kind."""+ time.sleep(2)+ lab_frame = next(c for c in hypr.clients() if c.get("class") == "BenchLab")+ if kind in ("focus", "all"):+ hypr.focus(lab_frame["address"]) # key focus and activation move to BenchLab+ if kind in ("raise", "all"):+ hypr.raise_to_top(lab_frame["address"]) # BenchLab drawn over the user window+ if kind in ("pointer", "all"):+ IN.move(300, 300)+ IN.move(320, 310)+ if kind in ("clipboard", "all"):+ subprocess.run(["wl-copy", "cdb-disturbance"], check=True, timeout=10)+ if kind == "typing":+ type_text("x") # a keystroke lands in the user window+ if kind == "space":+ hypr.dispatch('hl.dsp.focus({ workspace = "2" })')+ time.sleep(1.5)+ hypr.dispatch('hl.dsp.focus({ workspace = "1" })')+ time.sleep(2)+++def main() -> None:+ global IN+ parser = argparse.ArgumentParser()+ parser.add_argument("--mode", required=True)+ parser.add_argument("--seed", type=int, required=True)+ parser.add_argument("--disturb")+ args = parser.parse_args()+ ctypes.CDLL(None).prctl(15, b"cdb-oracle", 0, 0, 0)+ Atspi.init()+ IN = Input()+ try:+ if args.disturb:+ disturb(args.disturb)+ else:+ globals()[args.mode](args.seed)+ time.sleep(1.0)+ finally:+ IN.close()+++if __name__ == "__main__":+ main()diff --git a/libs/cua-driver-bench/automated-eval/macos_bench/hyprland/sentinel.py b/libs/cua-driver-bench/automated-eval/macos_bench/hyprland/sentinel.pynew file mode 100644index 0000000000..7f05d51912--- /dev/null+++ b/libs/cua-driver-bench/automated-eval/macos_bench/hyprland/sentinel.py@@ -0,0 +1,317 @@+# SPDX-License-Identifier: FSL-1.1-MIT+# Copyright (c) 2026 Cua AI, Inc.++"""BenchSentinel for Hyprland (Wayland): the witness and simulated user of the Linux/Omarchy BenchLab port.++A GTK3 window ("User activity" in the background condition) that writes the same JSONL schema as the macOS+BenchSentinel, read by swift/summarize_sentinel.py. A Wayland client cannot see other windows, so the compositor+state comes from Hyprland's IPC socket (hypr.py):++* front: the active window (`activewindow`); the sentinel's own window is reported as ai.cua.benchsentinel.+* above: windows after the sentinel's in Hyprland's stacking order (`clients`) on its workspace (or pinned) that+ overlap it and draw over it (floating, pinned or full screen), plus top and overlay layer-shell surfaces that+ overlap it. A layer surface owned by a cua-driver process is named after it, so the overlay rule applies.+* mouse: the real pointer (`cursorpos`); the agent seats of the cua Hyprland plugin have their own pointers.+* onspace: the sentinel's workspace is the active one, no special workspace is open over it, and no other window+ is full screen on it.+* sheet: another window of the sentinel's own process (a dialog on the user's app).+* clip: clipboard changes, from `wl-paste --watch` (data-control; GTK's owner-change without wl-clipboard).+* key focus, typed keys, clicks and scrolls: GTK events on the sentinel's own window.++Control: one command per line appended to --control (activate, arm, disarm, quit); SIGUSR1 activates and SIGUSR2+toggles the armed state, as on X11.+"""++from __future__ import annotations++import argparse+import ctypes+import json+import os+import shutil+import signal+import subprocess+import sys+import threading+import time+from pathlib import Path++import gi++gi.require_version("Gtk", "3.0")+gi.require_version("Gdk", "3.0")+from gi.repository import Gdk, GLib, Gtk # noqa: E402++sys.path.insert(0, str(Path(__file__).resolve().parent))+import hypr # noqa: E402++OWN_BID = "ai.cua.benchsentinel"+DRAWS_OVER = ("floating", "pinned")+++def comm(pid: int) -> str:+ try:+ return Path(f"/proc/{pid}/comm").read_text().strip()+ except OSError:+ return ""+++class Sentinel:+ def __init__(self, args: argparse.Namespace) -> None:+ ctypes.CDLL(None).prctl(15, b"BenchSentinel", 0, 0, 0)+ GLib.set_prgname("BenchSentinel")+ GLib.set_application_name("BenchSentinel")+ self.args = args+ args.log.parent.mkdir(parents=True, exist_ok=True)+ self.log = args.log.open("a", buffering=1)+ self.armed = False+ self.clip = 0+ self.expected = ""+ self.previous = None+ self.address = ""+ self.control_at = 0+ self.lock = threading.Lock()+ self.latest_front: dict = {}+ self.title = "User activity" if args.user_activity else "Bench Sentinel"+ self.window = Gtk.Window(title=self.title)+ self.window.set_wmclass("BenchSentinel", "BenchSentinel")+ self.window.set_default_size(600, 320)+ if args.frame:+ _, _, w, h = map(int, args.frame.split(","))+ self.window.set_default_size(w, h)+ self.entry = Gtk.Entry()+ self.entry.get_accessible().set_name("User activity text")+ self.entry.connect("changed", self.changed)+ self.window.add(self.entry)+ self.window.connect("destroy", Gtk.main_quit)+ self.window.connect("focus-out-event", lambda *_: self.emit("didResignKey"))+ self.window.connect("focus-in-event", lambda *_: self.emit("didBecomeActive"))+ self.entry.connect("key-press-event", lambda *_: self.emit("keyDown"))+ self.entry.connect("button-press-event", lambda *_: self.emit("mouseDown"))+ self.entry.connect("scroll-event", lambda *_: self.emit("scrollWheel"))+ self.watcher = None+ if not shutil.which("wl-paste"):+ # GTK only hears of selection changes while the window has keyboard focus.+ self.clipboard = Gtk.Clipboard.get(Gdk.SELECTION_CLIPBOARD)+ self.clipboard.connect("owner-change", self.clip_changed)+ else:+ # data-control sees every selection change, focused or not (and the current one when it starts,+ # long before the sentinel is armed; changes are counted from the armed sample).+ self.watcher = subprocess.Popen(+ ["wl-paste", "--watch", "echo", "change"], stdout=subprocess.PIPE,+ stderr=subprocess.DEVNULL, text=True,+ ) # fmt: skip+ threading.Thread(target=self.watch_clipboard, daemon=True).start()+ self.window.show_all()+ GLib.unix_signal_add(GLib.PRIORITY_DEFAULT, signal.SIGUSR1, self.activate)+ GLib.unix_signal_add(GLib.PRIORITY_DEFAULT, signal.SIGUSR2, self.toggle)+ GLib.timeout_add(100, self.find_window)++ def find_window(self) -> bool:+ mine = [c for c in hypr.by_pid(os.getpid()) if c.get("title") == self.title]+ if not mine:+ return True+ self.address = mine[0]["address"]+ if self.args.frame:+ hypr.place(self.address, tuple(map(int, self.args.frame.split(","))))+ if not self.args.no_activate:+ self.activate()+ # Compositor reads can stall while the compositor is busy; they run off the GTK thread so the user's+ # typing, focus events and control commands never wait on them.+ threading.Thread(target=self.sampler, daemon=True).start()+ GLib.timeout_add(100, self.poll_control)+ if self.args.user_activity:+ GLib.timeout_add(500, self.activity)+ self.emit("ready", address=self.address, pid=os.getpid())+ return False++ def watch_clipboard(self) -> None:+ for _ in self.watcher.stdout:+ GLib.idle_add(self.clip_changed)++ def clip_changed(self, *_: object) -> bool:+ self.clip += 1+ return False++ def changed(self, *_: object) -> None:+ if self.entry.get_text() != self.expected:+ self.emit("user_text_disrupted")++ def own(self, clients: list[dict]) -> dict:+ return next((c for c in clients if c.get("address") == self.address), {})++ def identity(self, client: dict) -> dict:+ if not client:+ return {"bid": "none", "pid": -1}+ if client.get("address") == self.address:+ return {"bid": OWN_BID, "pid": os.getpid()}+ return {"bid": hypr.identity(client), "pid": int(client.get("pid", -1))}++ def snapshot(self) -> dict:+ clients = hypr.query("clients") or []+ own = self.own(clients)+ workspace = (own.get("workspace") or {}).get("id")+ front = self.identity(hypr.active_window())+ x, y = hypr.cursor()+ above: list[str] = []+ if own in clients:+ for c in clients[clients.index(own) + 1 :]:+ if not c.get("mapped", True) or c.get("hidden"):+ continue+ here = c.get("pinned") or (c.get("workspace") or {}).get("id") == workspace+ over = c.get("floating") or c.get("pinned") or c.get("fullscreen")+ if here and over and hypr.overlaps(c, own):+ above.append(self.identity(c)["bid"])+ for c in clients:+ if (+ c is not own+ and c.get("fullscreen")+ and (c.get("workspace") or {}).get("id") == workspace+ ):+ name = self.identity(c)["bid"]+ if name not in above:+ above.append(name)+ layers = hypr.query("layers") or {}+ for monitor in layers.values():+ for level in ("2", "3"):+ for layer in (monitor.get("levels") or {}).get(level, []):+ box = {+ "at": (layer.get("x", 0), layer.get("y", 0)),+ "size": (layer.get("w", 0), layer.get("h", 0)),+ }+ if own and hypr.overlaps(box, own):+ pid = int(layer.get("pid", -1))+ owner = comm(pid) or f"pid{pid}"+ above.append(f"{owner}:layer:{layer.get('namespace', '')}")+ monitors = hypr.query("monitors") or []+ monitor = next(+ (m for m in monitors if m.get("id") == own.get("monitor")),+ monitors[0] if monitors else {},+ )+ special = bool((monitor.get("specialWorkspace") or {}).get("id"))+ fullscreen_other = any(+ c is not own+ and c.get("fullscreen")+ and (c.get("workspace") or {}).get("id") == workspace+ for c in clients+ )+ onspace = (+ (monitor.get("activeWorkspace") or {}).get("id") == workspace+ and not special+ and not fullscreen_other+ )+ sheet = any(+ c.get("pid") == os.getpid()+ and c.get("address") != self.address+ and c.get("mapped", True)+ for c in clients+ )+ return {+ "front": front,+ "mouse": [x, y],+ "clip": self.clip,+ "onspace": bool(onspace),+ "sheet": sheet,+ "above": above,+ "above_n": len(above),+ }++ def emit(self, event: str, **details: object) -> bool:+ if self.armed or event in ("armed", "disarmed", "ready"):+ with self.lock:+ self.log.write(json.dumps({"t": time.time() * 1000, "ev": event, **details}) + "\n")+ return False++ def activate(self, *_: object) -> bool:+ if self.address:+ hypr.focus(self.address)+ hypr.raise_to_top(self.address)+ self.window.present()+ self.entry.grab_focus()+ return True++ def toggle(self, *_: object) -> bool:+ if self.armed:+ if self.args.user_activity:+ self.emit("user_text", intact=self.entry.get_text() == self.expected)+ self.emit("disarmed", **self.snapshot())+ self.armed = False+ else:+ self.armed = True+ self.emit("armed", **self.snapshot())+ return True++ def poll_control(self) -> bool:+ if not self.args.control or not self.args.control.exists():+ return True+ with self.args.control.open(encoding="utf-8") as handle:+ handle.seek(self.control_at)+ lines = handle.read().splitlines()+ self.control_at = handle.tell()+ for line in lines:+ command = line.strip()+ if command == "activate":+ self.activate()+ elif command == "arm" and not self.armed:+ self.toggle()+ elif command == "disarm" and self.armed:+ self.toggle()+ elif command == "quit":+ Gtk.main_quit()+ return False+ return True++ def sampler(self) -> None:+ while True:+ started = time.monotonic()+ self.sample()+ time.sleep(max(0.0, 0.2 - (time.monotonic() - started)))++ def sample(self) -> None:+ try:+ sample = self.snapshot()+ except (OSError, ValueError, RuntimeError) as error:+ self.emit("sample_error", error=str(error))+ return+ self.latest_front = sample["front"]+ if not self.armed:+ return+ if sample["front"] != self.previous:+ self.emit("front", **sample)+ if self.previous and self.previous.get("bid") == OWN_BID:+ self.emit("didResignActive")+ self.previous = sample["front"]+ self.emit("s", **sample)++ def activity(self) -> bool:+ if self.armed:+ if self.latest_front.get("bid") == OWN_BID and self.window.has_toplevel_focus():+ if self.entry.get_text() != self.expected:+ self.emit("user_text_disrupted")+ self.expected += "a"+ self.entry.set_text(self.expected)+ self.entry.set_position(-1)+ self.emit("user_type")+ else:+ self.emit("user_blocked")+ return True+++def main() -> None:+ parser = argparse.ArgumentParser()+ parser.add_argument("--log", type=Path, required=True)+ parser.add_argument("--control", type=Path)+ parser.add_argument("--no-activate", action="store_true")+ parser.add_argument("--user-activity", action="store_true")+ parser.add_argument("--frame")+ witness = Sentinel(parser.parse_args())+ try:+ Gtk.main()+ finally:+ if witness.watcher:+ witness.watcher.terminate()+ witness.log.close()+++if __name__ == "__main__":+ main()diff --git a/libs/cua-driver-bench/automated-eval/macos_bench/launch/hyprland_session.py b/libs/cua-driver-bench/automated-eval/macos_bench/launch/hyprland_session.pynew file mode 100644index 0000000000..937343f51e--- /dev/null+++ b/libs/cua-driver-bench/automated-eval/macos_bench/launch/hyprland_session.py@@ -0,0 +1,345 @@+#!/usr/bin/env python3+# SPDX-License-Identifier: FSL-1.1-MIT+# Copyright (c) 2026 Cua AI, Inc.+#+# Usage: hyprland_session.py --task ID --run-index N --condition foreground|background --out DIR+# [--timeout S] -- COMMAND [ARG...]+#+# One BenchLab trial on a Hyprland (Wayland) desktop such as Omarchy, run as the desktop user with the session's+# environment (WAYLAND_DISPLAY, HYPRLAND_INSTANCE_SIGNATURE, XDG_RUNTIME_DIR, DBUS_SESSION_BUS_ADDRESS):+# 1. derive the seed (fairness rule), render the brief (+ the background paragraph), write DIR/brief.md;+# 2. start BenchLab (linux/benchlab.py, GTK3) and the sentinel (hyprland/sentinel.py) and place both through+# Hyprland's IPC: in the background condition the "User activity" window partly covers BenchLab, is brought+# to the front and types;+# 3. arm the sentinel, run COMMAND (the agent; `{brief}`, `{out}`, `{seed}`, `{mode}` in its arguments are filled+# in) with the GUI-only guard watching, stop it at the time limit or on a violation;+# 4. disarm, run the task's checker, summarise the sentinel, write DIR/row.json.+# The desktop needs BenchLab and BenchSentinel to float, and BenchLab's windows not to take focus when they open+# (hyprland/bench.lua).+from __future__ import annotations++import argparse+import importlib.util+import json+import os+import signal+import subprocess+import sys+import time+from pathlib import Path++BENCH = Path(__file__).resolve().parent.parent+sys.path.insert(0, str(BENCH))+sys.path.insert(0, str(BENCH / "probes" / "_common"))+sys.path.insert(0, str(BENCH / "swift"))+sys.path.insert(0, str(BENCH.parents[1] / "fairness"))+import bench_v2 as v2 # noqa: E402+import cdb_fairness # noqa: E402+import summarize_sentinel # noqa: E402++TASKS = ("MB-09", "MB-10", "MB-11", "IR-01", "IR-02", "IR-03", "IR-04")+LAB_FRAME = (40, 40, 760, 540)+WITNESS_FRAME = (1000, 640, 600, 320)+PLATFORM_WORDS = ("on your Mac", "on this Linux computer")+RULES = BENCH / "hyprland" / "bench.lua"+++def apply_rules(H) -> None:+ """The window rules of hyprland/bench.lua (named, so applying them again replaces them)."""+ text = RULES.read_text(encoding="utf-8")+ for chunk in text.split("hl.window_rule(")[1:]:+ depth, end = 1, 0+ for i, ch in enumerate(chunk):+ depth += ch == "("+ depth -= ch == ")"+ if depth == 0:+ end = i+ break+ H.request("eval hl.window_rule(" + chunk[: end + 1])+++def hyprland_module(name: str):+ """hyprland/NAME.py under its own module name (windows/ has a guard.py too)."""+ spec = importlib.util.spec_from_file_location(+ f"hyprland_{name}", BENCH / "hyprland" / f"{name}.py"+ )+ module = importlib.util.module_from_spec(spec)+ spec.loader.exec_module(module)+ return module+++def task_spec(task: str) -> dict:+ return json.loads((BENCH / "probes" / task / "task.json").read_text(encoding="utf-8"))+++def render_brief(task: str, seed: int, condition: str) -> str:+ spec = importlib.util.spec_from_file_location(+ f"render_{task}", BENCH / "probes" / task / "render_brief.py"+ )+ module = importlib.util.module_from_spec(spec)+ spec.loader.exec_module(module)+ brief = module.render(seed).replace(*PLATFORM_WORDS)+ return v2.brief_for(brief, condition)+++def kill_group(proc: subprocess.Popen) -> None:+ for sig in (signal.SIGTERM, signal.SIGKILL):+ try:+ os.killpg(proc.pid, sig)+ except OSError:+ return+ try:+ proc.wait(timeout=3)+ return+ except subprocess.TimeoutExpired:+ continue+++def kill_pid(pid: int) -> None:+ try:+ os.kill(pid, signal.SIGKILL)+ except OSError:+ pass+++def wait_for(predicate, limit: float, step: float = 0.1) -> bool:+ end = time.monotonic() + limit+ while time.monotonic() < end:+ if predicate():+ return True+ time.sleep(step)+ return False+++def sentinel_lines(log: Path) -> list[dict]:+ if not log.exists():+ return []+ out = []+ for raw in log.read_text(encoding="utf-8").splitlines():+ try:+ out.append(json.loads(raw))+ except json.JSONDecodeError:+ pass+ return out+++def run(args: argparse.Namespace, command: list[str]) -> dict:+ G = hyprland_module("guard")+ H = hyprland_module("hypr")++ out: Path = args.out.resolve()+ out.mkdir(parents=True, exist_ok=True)+ spec = task_spec(args.task)+ mode = spec["app_args"][spec["app_args"].index("--mode") + 1]+ seed = cdb_fairness.trial_seed(args.task, args.run_index)+ brief = render_brief(args.task, seed, args.condition)+ (out / "brief.md").write_text(brief, encoding="utf-8")+ background = args.condition == "background"+ row: dict = {+ "task": args.task, "run_index": args.run_index, "seed": seed, "condition": args.condition,+ "mode": mode, "platform": "linux-hyprland", "label": args.label, "started": time.time(),+ } # fmt: skip++ state, events, log, control = (+ out / "state.json", out / "events.jsonl", out / "sentinel.jsonl", out / "sentinel.control",+ ) # fmt: skip+ for stale in (state, events, log, control):+ stale.unlink(missing_ok=True)+ control.write_text("", encoding="utf-8")+ apply_rules(H)+ monitors = H.query("monitors") or []+ if (+ not monitors+ or max(m["width"] for m in monitors) < 1320+ or max(m["height"] for m in monitors) < 860+ ):+ row["harness_error"] = "screen smaller than 1320x860 (load hyprland/bench.lua)"+ (out / "row.json").write_text(json.dumps(row, indent=1), encoding="utf-8")+ return row+ env = {**os.environ, "GDK_BACKEND": "wayland", "NO_AT_BRIDGE": "0"}+ lab = subprocess.Popen(+ [sys.executable, str(BENCH / "linux" / "benchlab.py"), "--mode", mode, "--seed", str(seed),+ "--state", str(state), "--events", str(events)],+ env=env, start_new_session=True, stdout=(out / "benchlab.log").open("w"), stderr=subprocess.STDOUT,+ ) # fmt: skip+ procs = [lab]+ try:+ if not wait_for(lambda: state.exists() and H.by_pid(lab.pid), 30):+ row["harness_error"] = "BenchLab did not start"+ return row+ lab_address = H.by_pid(lab.pid)[0]["address"]+ H.place(lab_address, LAB_FRAME)+ if not background:+ H.focus(lab_address)+ frame = v2.USER_WINDOW_FRAME if background else WITNESS_FRAME+ sentinel = subprocess.Popen(+ [sys.executable, str(BENCH / "hyprland" / "sentinel.py"), "--log", str(log),+ "--control", str(control), "--frame", ",".join(map(str, frame)),+ *(["--user-activity"] if background else [])],+ env=env, start_new_session=True, stdout=(out / "sentinel.log").open("w"), stderr=subprocess.STDOUT,+ ) # fmt: skip+ procs.append(sentinel)+ if not wait_for(lambda: any(ln.get("ev") == "ready" for ln in sentinel_lines(log)), 30):+ row["harness_error"] = "sentinel did not start"+ return row+ own = next(ln["address"] for ln in sentinel_lines(log) if ln.get("ev") == "ready")++ def in_front() -> bool:+ return H.active_window().get("address") == own and H.clients()[-1].get("address") == own++ with control.open("a", encoding="utf-8") as c:+ c.write("activate\n")+ front_ok = wait_for(in_front, 10)+ if not front_ok:+ with control.open("a", encoding="utf-8") as c:+ c.write("activate\n")+ front_ok = wait_for(in_front, 5)+ row["sentinel_front_at_start"] = front_ok+ row["frames"] = {+ c.get("class"): [*c.get("at", []), *c.get("size", [])] for c in H.clients()+ }+ time.sleep(1.0)+ with control.open("a", encoding="utf-8") as c:+ c.write("arm\n")+ wait_for(lambda: any(ln.get("ev") == "armed" for ln in sentinel_lines(log)), 5)++ subs = {+ "{brief}": str(out / "brief.md"),+ "{out}": str(out),+ "{seed}": str(seed),+ "{mode}": mode,+ }+ argv = []+ for a in command:+ for key, value in subs.items():+ a = a.replace(key, value)+ argv.append(a)+ baseline = {pid for pid, _, _ in G.process_rows()}+ t0 = time.monotonic()+ agent = subprocess.Popen(+ argv, cwd=str(out), env={**os.environ, "CDB_BRIEF_FILE": str(out / "brief.md")},+ start_new_session=True,+ ) # fmt: skip+ procs.append(agent)+ own_tree: set[int] = set()+ front = G.FrontWatch()+ violations: list[dict] = []+ timed_out = False+ while agent.poll() is None:+ rows = G.process_rows()+ own_tree |= G.descendants(rows, os.getpid())+ for pid, label in G.new_side_doors(rows, baseline, os.getpid(), own_tree):+ violations.append(+ {+ "t_s": round(time.monotonic() - t0, 1),+ "kind": "process",+ "detail": label,+ "pid": pid,+ }+ )+ kill_pid(pid)+ try:+ hit = front.sample(H.active_window().get("class", ""))+ except (OSError, ValueError, RuntimeError):+ hit = None+ if hit:+ violations.append(+ {"t_s": round(time.monotonic() - t0, 1), "kind": "front", "detail": hit}+ )+ if violations and args.enforce:+ kill_group(agent)+ break+ if time.monotonic() - t0 > args.timeout:+ timed_out = True+ kill_group(agent)+ break+ time.sleep(0.5)+ try:+ agent.wait(timeout=30)+ except subprocess.TimeoutExpired:+ kill_group(agent)+ row["agent_exit"] = agent.returncode+ row["agent_wall_s"] = round(time.monotonic() - t0, 1)+ row["timed_out"] = timed_out++ with control.open("a", encoding="utf-8") as c:+ c.write("disarm\n")+ wait_for(lambda: any(ln.get("ev") == "disarmed" for ln in sentinel_lines(log)), 5)+ time.sleep(0.3)+ result = out / "result.json"+ subprocess.run(+ [sys.executable, str(BENCH / "probes" / args.task / "evaluate.py"), "--seed", str(seed),+ "--state", str(state), "--events", str(events), "--result", str(result)],+ capture_output=True, timeout=60,+ ) # fmt: skip+ evaluation = json.loads(result.read_text(encoding="utf-8")) if result.exists() else {}+ transcript = out / "agent-transcript.jsonl"+ stream = []+ if transcript.exists():+ for raw in transcript.read_text(encoding="utf-8", errors="replace").splitlines():+ try:+ stream.append(json.loads(raw))+ except json.JSONDecodeError:+ pass+ hard, soft = G.scan_events(stream)+ violations += [{"t_s": None, "kind": "tool_input", "detail": h} for h in hard]+ disturbance = summarize_sentinel.summarize(str(log))+ armed = next((ln for ln in sentinel_lines(log) if ln.get("ev") == "armed"), {})+ frontmost = (armed.get("front") or {}).get("bid") == "ai.cua.benchsentinel"+ passed = bool(evaluation.get("passed")) and not violations+ bg = v2.background_score(disturbance, frontmost)+ row.update(+ evaluation={k: evaluation.get(k) for k in ("passed", "score", "checks", "diagnostics")},+ checker_passed=bool(evaluation.get("passed")),+ gui_only={"violation": bool(violations), "violations": violations[:20],+ "mentions": soft, "frontmost_seen": sorted(front.seen)},+ passed=passed,+ disturbance=disturbance,+ background=bg,+ a15=v2.background_condition_score(bg, disturbance, passed, args.condition),+ a22=v2.multiplayer_score(disturbance, bg["measured"], passed, args.condition),+ ) # fmt: skip+ return row+ finally:+ with control.open("a", encoding="utf-8") as c:+ c.write("quit\n")+ time.sleep(0.5)+ for proc in procs:+ if proc.poll() is None:+ kill_group(proc)+ row["finished"] = time.time()+ (out / "row.json").write_text(json.dumps(row, indent=1), encoding="utf-8")+++def main(argv: list[str]) -> int:+ if "--" not in argv:+ print(+ "usage: hyprland_session.py --task ID --run-index N --condition C --out DIR -- COMMAND...",+ file=sys.stderr,+ )+ return 2+ split = argv.index("--")+ parser = argparse.ArgumentParser()+ parser.add_argument("--task", choices=TASKS, required=True)+ parser.add_argument("--run-index", type=int, required=True)+ parser.add_argument("--condition", choices=v2.CONDITIONS, required=True)+ parser.add_argument("--out", type=Path, required=True)+ parser.add_argument("--label", default="")+ parser.add_argument("--timeout", type=float, default=360.0)+ parser.add_argument("--no-enforce", dest="enforce", action="store_false")+ args = parser.parse_args(argv[:split])+ command = argv[split + 1 :]+ if not os.environ.get("HYPRLAND_INSTANCE_SIGNATURE") or not command:+ print("hyprland_session.py needs a Hyprland session and a command", file=sys.stderr)+ return 2+ row = run(args, command)+ print(+ json.dumps(+ {k: row.get(k) for k in ("task", "seed", "condition", "passed", "harness_error")}+ )+ )+ return 0 if not row.get("harness_error") else 3+++if __name__ == "__main__":+ sys.exit(main(sys.argv[1:]))diff --git a/libs/cua-driver-bench/automated-eval/macos_bench/probes/IR-01/launch.hyprland.json b/libs/cua-driver-bench/automated-eval/macos_bench/probes/IR-01/launch.hyprland.jsonnew file mode 100644index 0000000000..fbeae24d56--- /dev/null+++ b/libs/cua-driver-bench/automated-eval/macos_bench/probes/IR-01/launch.hyprland.json@@ -0,0 +1,48 @@+{+ "platform": "linux-hyprland",+ "descriptor_version": "1",+ "task_id": "IR-01",+ "app": "linux/benchlab.py",+ "app_args": [+ "--mode",+ "irmodal"+ ],+ "app_executable": "python3 linux/benchlab.py (GTK3, native Wayland; process name and app id BenchLab)",+ "sentinel": "hyprland/sentinel.py",+ "launcher": [+ "${python}",+ "launch/hyprland_session.py",+ "--task",+ "IR-01",+ "--run-index",+ "${run_index}",+ "--condition",+ "${condition}",+ "--out",+ "${artifacts}",+ "--",+ "${agent_command}"+ ],+ "conditions": [+ "foreground",+ "background"+ ],+ "evaluate": [+ "${python}",+ "probes/IR-01/evaluate.py",+ "--seed",+ "${seed}",+ "--state",+ "${artifacts}/state.json",+ "--events",+ "${artifacts}/events.jsonl",+ "--result",+ "${result}"+ ],+ "requires": [+ "Python 3.11 or later with PyGObject, GTK 3 and AT-SPI",+ "Hyprland 0.55 or later with a Lua config, hyprland/bench.lua loaded (1920x1080)",+ "wl-clipboard",+ "the desktop user's Hyprland session environment"+ ]+}diff --git a/libs/cua-driver-bench/automated-eval/macos_bench/probes/IR-02/launch.hyprland.json b/libs/cua-driver-bench/automated-eval/macos_bench/probes/IR-02/launch.hyprland.jsonnew file mode 100644index 0000000000..74d934a222--- /dev/null+++ b/libs/cua-driver-bench/automated-eval/macos_bench/probes/IR-02/launch.hyprland.json@@ -0,0 +1,48 @@+{+ "platform": "linux-hyprland",+ "descriptor_version": "1",+ "task_id": "IR-02",+ "app": "linux/benchlab.py",+ "app_args": [+ "--mode",+ "irbanner"+ ],+ "app_executable": "python3 linux/benchlab.py (GTK3, native Wayland; process name and app id BenchLab)",+ "sentinel": "hyprland/sentinel.py",+ "launcher": [+ "${python}",+ "launch/hyprland_session.py",+ "--task",+ "IR-02",+ "--run-index",+ "${run_index}",+ "--condition",+ "${condition}",+ "--out",+ "${artifacts}",+ "--",+ "${agent_command}"+ ],+ "conditions": [+ "foreground",+ "background"+ ],+ "evaluate": [+ "${python}",+ "probes/IR-02/evaluate.py",+ "--seed",+ "${seed}",+ "--state",+ "${artifacts}/state.json",+ "--events",+ "${artifacts}/events.jsonl",+ "--result",+ "${result}"+ ],+ "requires": [+ "Python 3.11 or later with PyGObject, GTK 3 and AT-SPI",+ "Hyprland 0.55 or later with a Lua config, hyprland/bench.lua loaded (1920x1080)",+ "wl-clipboard",+ "the desktop user's Hyprland session environment"+ ]+}diff --git a/libs/cua-driver-bench/automated-eval/macos_bench/probes/IR-03/launch.hyprland.json b/libs/cua-driver-bench/automated-eval/macos_bench/probes/IR-03/launch.hyprland.jsonnew file mode 100644index 0000000000..d00b3ad43c--- /dev/null+++ b/libs/cua-driver-bench/automated-eval/macos_bench/probes/IR-03/launch.hyprland.json@@ -0,0 +1,48 @@+{+ "platform": "linux-hyprland",+ "descriptor_version": "1",+ "task_id": "IR-03",+ "app": "linux/benchlab.py",+ "app_args": [+ "--mode",+ "irunsaved"+ ],+ "app_executable": "python3 linux/benchlab.py (GTK3, native Wayland; process name and app id BenchLab)",+ "sentinel": "hyprland/sentinel.py",+ "launcher": [+ "${python}",+ "launch/hyprland_session.py",+ "--task",+ "IR-03",+ "--run-index",+ "${run_index}",+ "--condition",+ "${condition}",+ "--out",+ "${artifacts}",+ "--",+ "${agent_command}"+ ],+ "conditions": [+ "foreground",+ "background"+ ],+ "evaluate": [+ "${python}",+ "probes/IR-03/evaluate.py",+ "--seed",+ "${seed}",+ "--state",+ "${artifacts}/state.json",+ "--events",+ "${artifacts}/events.jsonl",+ "--result",+ "${result}"+ ],+ "requires": [+ "Python 3.11 or later with PyGObject, GTK 3 and AT-SPI",+ "Hyprland 0.55 or later with a Lua config, hyprland/bench.lua loaded (1920x1080)",+ "wl-clipboard",+ "the desktop user's Hyprland session environment"+ ]+}diff --git a/libs/cua-driver-bench/automated-eval/macos_bench/probes/IR-04/launch.hyprland.json b/libs/cua-driver-bench/automated-eval/macos_bench/probes/IR-04/launch.hyprland.jsonnew file mode 100644index 0000000000..ff398b7d01--- /dev/null+++ b/libs/cua-driver-bench/automated-eval/macos_bench/probes/IR-04/launch.hyprland.json@@ -0,0 +1,48 @@+{+ "platform": "linux-hyprland",+ "descriptor_version": "1",+ "task_id": "IR-04",+ "app": "linux/benchlab.py",+ "app_args": [+ "--mode",+ "irconsent"+ ],+ "app_executable": "python3 linux/benchlab.py (GTK3, native Wayland; process name and app id BenchLab)",+ "sentinel": "hyprland/sentinel.py",+ "launcher": [+ "${python}",+ "launch/hyprland_session.py",+ "--task",+ "IR-04",+ "--run-index",+ "${run_index}",+ "--condition",+ "${condition}",+ "--out",+ "${artifacts}",+ "--",+ "${agent_command}"+ ],+ "conditions": [+ "foreground",+ "background"+ ],+ "evaluate": [+ "${python}",+ "probes/IR-04/evaluate.py",+ "--seed",+ "${seed}",+ "--state",+ "${artifacts}/state.json",+ "--events",+ "${artifacts}/events.jsonl",+ "--result",+ "${result}"+ ],+ "requires": [+ "Python 3.11 or later with PyGObject, GTK 3 and AT-SPI",+ "Hyprland 0.55 or later with a Lua config, hyprland/bench.lua loaded (1920x1080)",+ "wl-clipboard",+ "the desktop user's Hyprland session environment"+ ]+}diff --git a/libs/cua-driver-bench/automated-eval/macos_bench/probes/MB-09/launch.hyprland.json b/libs/cua-driver-bench/automated-eval/macos_bench/probes/MB-09/launch.hyprland.jsonnew file mode 100644index 0000000000..e3d9b59738--- /dev/null+++ b/libs/cua-driver-bench/automated-eval/macos_bench/probes/MB-09/launch.hyprland.json@@ -0,0 +1,48 @@+{+ "platform": "linux-hyprland",+ "descriptor_version": "1",+ "task_id": "MB-09",+ "app": "linux/benchlab.py",+ "app_args": [+ "--mode",+ "listdrag"+ ],+ "app_executable": "python3 linux/benchlab.py (GTK3, native Wayland; process name and app id BenchLab)",+ "sentinel": "hyprland/sentinel.py",+ "launcher": [+ "${python}",+ "launch/hyprland_session.py",+ "--task",+ "MB-09",+ "--run-index",+ "${run_index}",+ "--condition",+ "${condition}",+ "--out",+ "${artifacts}",+ "--",+ "${agent_command}"+ ],+ "conditions": [+ "foreground",+ "background"+ ],+ "evaluate": [+ "${python}",+ "probes/MB-09/evaluate.py",+ "--seed",+ "${seed}",+ "--state",+ "${artifacts}/state.json",+ "--events",+ "${artifacts}/events.jsonl",+ "--result",+ "${result}"+ ],+ "requires": [+ "Python 3.11 or later with PyGObject, GTK 3 and AT-SPI",+ "Hyprland 0.55 or later with a Lua config, hyprland/bench.lua loaded (1920x1080)",+ "wl-clipboard",+ "the desktop user's Hyprland session environment"+ ]+}diff --git a/libs/cua-driver-bench/automated-eval/macos_bench/probes/MB-10/launch.hyprland.json b/libs/cua-driver-bench/automated-eval/macos_bench/probes/MB-10/launch.hyprland.jsonnew file mode 100644index 0000000000..dff53fac10--- /dev/null+++ b/libs/cua-driver-bench/automated-eval/macos_bench/probes/MB-10/launch.hyprland.json@@ -0,0 +1,47 @@+{+ "platform": "linux-hyprland",+ "descriptor_version": "1",+ "task_id": "MB-10",+ "app": "linux/benchlab.py",+ "app_args": [+ "--mode",+ "hover"+ ],+ "app_executable": "python3 linux/benchlab.py (GTK3, native Wayland; process name and app id BenchLab)",+ "sentinel": "hyprland/sentinel.py",+ "launcher": [+ "${python}",+ "launch/hyprland_session.py",+ "--task",+ "MB-10",+ "--run-index",+ "${run_index}",+ "--condition",+ "${condition}",+ "--out",+ "${artifacts}",+ "--",+ "${agent_command}"+ ],+ "conditions": [+ "foreground"+ ],+ "evaluate": [+ "${python}",+ "probes/MB-10/evaluate.py",+ "--seed",+ "${seed}",+ "--state",+ "${artifacts}/state.json",+ "--events",+ "${artifacts}/events.jsonl",+ "--result",+ "${result}"+ ],+ "requires": [+ "Python 3.11 or later with PyGObject, GTK 3 and AT-SPI",+ "Hyprland 0.55 or later with a Lua config, hyprland/bench.lua loaded (1920x1080)",+ "wl-clipboard",+ "the desktop user's Hyprland session environment"+ ]+}diff --git a/libs/cua-driver-bench/automated-eval/macos_bench/probes/MB-11/launch.hyprland.json b/libs/cua-driver-bench/automated-eval/macos_bench/probes/MB-11/launch.hyprland.jsonnew file mode 100644index 0000000000..02ab4f0603--- /dev/null+++ b/libs/cua-driver-bench/automated-eval/macos_bench/probes/MB-11/launch.hyprland.json@@ -0,0 +1,47 @@+{+ "platform": "linux-hyprland",+ "descriptor_version": "1",+ "task_id": "MB-11",+ "app": "linux/benchlab.py",+ "app_args": [+ "--mode",+ "tooltip"+ ],+ "app_executable": "python3 linux/benchlab.py (GTK3, native Wayland; process name and app id BenchLab)",+ "sentinel": "hyprland/sentinel.py",+ "launcher": [+ "${python}",+ "launch/hyprland_session.py",+ "--task",+ "MB-11",+ "--run-index",+ "${run_index}",+ "--condition",+ "${condition}",+ "--out",+ "${artifacts}",+ "--",+ "${agent_command}"+ ],+ "conditions": [+ "foreground"+ ],+ "evaluate": [+ "${python}",+ "probes/MB-11/evaluate.py",+ "--seed",+ "${seed}",+ "--state",+ "${artifacts}/state.json",+ "--events",+ "${artifacts}/events.jsonl",+ "--result",+ "${result}"+ ],+ "requires": [+ "Python 3.11 or later with PyGObject, GTK 3 and AT-SPI",+ "Hyprland 0.55 or later with a Lua config, hyprland/bench.lua loaded (1920x1080)",+ "wl-clipboard",+ "the desktop user's Hyprland session environment"+ ]+}diff --git a/libs/cua-driver-bench/automated-eval/macos_bench/swift/summarize_sentinel.py b/libs/cua-driver-bench/automated-eval/macos_bench/swift/summarize_sentinel.pyindex 7cdb32ba0b..c69bed17bc 100644--- a/libs/cua-driver-bench/automated-eval/macos_bench/swift/summarize_sentinel.py+++ b/libs/cua-driver-bench/automated-eval/macos_bench/swift/summarize_sentinel.py@@ -21,9 +21,10 @@ SENTINEL_BID = "ai.cua.benchsentinel"DEVIATION_PX = 10.0HID_KEYS = ("move", "down", "key", "scroll")# Amendment 15: overlay and indicator windows of the computer-use tools themselves (the agent cursor, the-# "ChatGPT Computer Use" banner, the `cua-driver` process on Windows) are not raises. They are logged as metadata (`overlay_owners`) only.+# "ChatGPT Computer Use" banner, the `cua-driver` process on Windows, the cua-driver agent cursor's Wayland layer+# `cua-agent-cursor`) are not raises. They are logged as metadata (`overlay_owners`) only.OVERLAY_OWNER_RE = re.compile(- r"(?i)cua[ _-]?driver|chatgpt computer use|computer use|codex|app-cu-helper|claude"+ r"(?i)cua[ _-]?driver|cua-agent-cursor|chatgpt computer use|computer use|codex|app-cu-helper|claude")FRONT_EVENTS = ("s", "front", "armed", "disarmed")# Idle values only ever grow between samples unless a real event resets them.diff --git a/libs/cua-driver-bench/automated-eval/macos_bench/tests/test_hyprland_tools.py b/libs/cua-driver-bench/automated-eval/macos_bench/tests/test_hyprland_tools.pynew file mode 100644index 0000000000..516baa25df--- /dev/null+++ b/libs/cua-driver-bench/automated-eval/macos_bench/tests/test_hyprland_tools.py@@ -0,0 +1,153 @@+# SPDX-License-Identifier: FSL-1.1-MIT+# Copyright (c) 2026 Cua AI, Inc.++"""Hyprland (Wayland) port contracts that hold on every OS (no GUI, no model): the GUI-only guard, the brief,+the launch descriptors, the window rules, window geometry, and that the Hyprland sentinel writes every event+the X11 sentinel writes."""++from __future__ import annotations++import importlib.util+import json+import re+import sys+import unittest+from pathlib import Path++HERE = Path(__file__).resolve().parent+BENCH = HERE.parent+sys.path.insert(0, str(BENCH))+sys.path.insert(0, str(BENCH / "launch"))+import bench_v2 as v2 # noqa: E402+import hyprland_session as HS # noqa: E402++G = HS.hyprland_module("guard")+H = HS.hyprland_module("hypr")+BENCHLAB_TASKS = ("MB-09", "MB-10", "MB-11", "IR-01", "IR-02", "IR-03", "IR-04")+++def _emitted(path: Path) -> set[str]:+ return set(re.findall(r'self\.emit\(\s*"([a-z_A-Z]+)"', path.read_text("utf-8")))+++class PortTest(unittest.TestCase):+ def test_sentinel_events_cover_linux(self) -> None:+ linux = _emitted(BENCH / "linux/sentinel.py") - {+ "ax_focus"+ } # AT-SPI focus trace: diagnostic only+ self.assertLessEqual(linux, _emitted(BENCH / "hyprland/sentinel.py"))++ def test_every_benchlab_task_has_a_hyprland_descriptor(self) -> None:+ manifest = {t["id"]: t for t in v2.load_taskset()["tasks"]}+ for tid in BENCHLAB_TASKS:+ spec = json.loads((BENCH / "probes" / tid / "task.json").read_text("utf-8"))+ launch = json.loads(+ (BENCH / "probes" / tid / "launch.hyprland.json").read_text("utf-8")+ )+ self.assertEqual(launch["platform"], "linux-hyprland")+ self.assertEqual(launch["app_args"], spec["app_args"], tid)+ self.assertEqual(launch["conditions"], manifest[tid]["conditions"], tid)+ self.assertIn("linux", manifest[tid]["os"])+ self.assertIn("launch/hyprland_session.py", launch["launcher"])+ self.assertEqual(set(HS.TASKS), set(BENCHLAB_TASKS))++ def test_window_rules(self) -> None:+ sent = []++ class Fake:+ @staticmethod+ def request(command: str) -> str:+ sent.append(command)+ return "ok"++ HS.apply_rules(Fake)+ self.assertEqual(len(sent), 3)+ self.assertTrue(+ all(c.startswith("eval hl.window_rule({") and c.endswith("})") for c in sent)+ )+ dialogs = next(c for c in sent if "cdb-benchlab-dialogs" in c)+ x, y = map(int, re.search(r'move = "(\d+) (\d+)"', dialogs).groups())+ ux, uy, uw, uh = v2.USER_WINDOW_FRAME+ lx, ly, lw, lh = HS.LAB_FRAME+ # Dialogs open inside BenchLab's frame, above the top edge of the user window.+ self.assertTrue(lx <= x < lx + lw and ly <= y < uy, (x, y))+++class OverlayTest(unittest.TestCase):+ def test_agent_cursor_layer_is_the_tools_own(self) -> None:+ sys.path.insert(0, str(BENCH / "swift"))+ import summarize_sentinel++ for name in ("cua-driver:layer:cua-agent-cursor", "pid123:layer:cua-agent-cursor"):+ self.assertTrue(summarize_sentinel.OVERLAY_OWNER_RE.search(name), name)+ self.assertFalse(summarize_sentinel.OVERLAY_OWNER_RE.search("quickshell:layer:omarchy-bar"))+++class BriefTest(unittest.TestCase):+ def test_platform_words_and_background_paragraph(self) -> None:+ for tid in BENCHLAB_TASKS:+ fg = HS.render_brief(tid, 123456, "foreground")+ bg = HS.render_brief(tid, 123456, "background")+ self.assertNotIn("your Mac", fg, tid)+ self.assertIn("on this Linux computer", fg, tid)+ self.assertNotIn(v2.BACKGROUND_BRIEF, fg)+ self.assertTrue(bg.rstrip().endswith(v2.BACKGROUND_BRIEF), tid)+ self.assertNotIn("{{", bg, tid)+++class GeometryTest(unittest.TestCase):+ def test_overlaps(self) -> None:+ user = {"at": [420, 340], "size": [900, 520]}+ self.assertTrue(H.overlaps({"at": [40, 40], "size": [760, 540]}, user))+ self.assertFalse(H.overlaps({"at": [80, 120], "size": [296, 105]}, user))+ self.assertFalse(H.overlaps({"at": [1521, 0], "size": [399, 1080]}, user))++ def test_identity(self) -> None:+ self.assertEqual(H.identity({"class": "BenchLab", "title": "x"}), "BenchLab")+ self.assertEqual(H.identity({"class": "", "initialClass": "", "title": "t"}), "t")+++class GuardTest(unittest.TestCase):+ ROWS = [+ (1, 0, "systemd"),+ (100, 1, "Hyprland"),+ (200, 1, "python3"), # the runner+ (210, 200, "python3"),+ (220, 210, "kitty"), # under the runner: not a violation+ (230, 100, "alacritty"), # started from the desktop: a violation+ (240, 100, "bash"), # a shell alone is not counted+ (250, 999, "foot"), # parent gone, but seen under the runner before: the runner's+ (260, 100, "ghostty"),+ ]++ def test_new_side_doors(self) -> None:+ hits = G.new_side_doors(self.ROWS, baseline={1, 100}, runner=200, own={250})+ self.assertEqual(sorted(hits), [(230, "Alacritty"), (260, "Ghostty")])+ self.assertEqual(+ G.new_side_doors(self.ROWS, baseline={p for p, _, _ in self.ROWS}, runner=200), []+ )++ def test_tool_input_patterns(self) -> None:+ hard, _ = G.scan_tool_use("mcp__x__launch_app", {"name": "kitty"})+ self.assertEqual(hard, ["mcp__x__launch_app:kitty"])+ hard, _ = G.scan_tool_use("mcp__x__type_text", {"text": "hyprctl dispatch exec foot"})+ self.assertIn("mcp__x__type_text:hyprctl", hard)+ self.assertIn("mcp__x__type_text:foot", hard)+ self.assertEqual(+ G.scan_tool_use("mcp__x__type_text", {"text": "Football club footer"}), ([], [])+ )+ self.assertEqual(G.scan_tool_use("Bash", {"command": "ls"})[0], ["builtin:Bash"])++ def test_front_grace(self) -> None:+ watch = G.FrontWatch()+ self.assertIsNone(watch.sample("com.mitchellh.ghostty"))+ self.assertIsNone(watch.sample("com.mitchellh.ghostty"))+ self.assertEqual(watch.sample("com.mitchellh.ghostty"), "Ghostty")+ self.assertIsNone(watch.sample("BenchLab"))+ for _ in range(2):+ self.assertIsNone(watch.sample("org.omarchy.btop"))+ self.assertEqual(watch.sample("org.omarchy.btop"), "Omarchy TUI")+++if __name__ == "__main__":+ unittest.main()