Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.
Rust agentai-agentapplecomputer-usecomputer-use-agentcontainerizationcuadesktop-automationhacktoberfestlumemacosmanusoperatorswiftvirtualizationvirtualization-frameworkwindowswindows-sandbox
docs(cua-driver): document GNOME WinRects setup (#4258)
PandaHUN777a86335beec14
diff --git a/docs/content/docs/how-to-guides/driver/install.mdx b/docs/content/docs/how-to-guides/driver/install.mdxindex 5f2ebd36c4..72f067dda8 100644--- a/docs/content/docs/how-to-guides/driver/install.mdx+++ b/docs/content/docs/how-to-guides/driver/install.mdx@@ -86,6 +86,25 @@ If these names are unfamiliar, start withThe experimental Hyprland plugin is separate from this installer; installingthe driver does not enable isolated raw background input on Omarchy.+### Install the GNOME WinRects helper++The helper is included with Cua Driver. For an installed release, run its+installer from the current package:++```bash+~/.cua-driver/packages/current/wayland-helper/install.sh+```++From a source checkout, run `./libs/cua-driver/wayland-helper/install.sh` from+the repository root instead. The script installs and enables the extension for+your user; it does not need `sudo`. Log out of GNOME and back in once so Shell+loads the helper, then verify it is active:++```bash+gnome-extensions info winrects@cua+# State: ACTIVE+```+</Tab></Tabs>diff --git a/docs/content/docs/reference/cua-driver/browser-profile-attachment.mdx b/docs/content/docs/reference/cua-driver/browser-profile-attachment.mdxindex cef7b77dd2..8c5ea9bf2f 100644--- a/docs/content/docs/reference/cua-driver/browser-profile-attachment.mdx+++ b/docs/content/docs/reference/cua-driver/browser-profile-attachment.mdx@@ -203,8 +203,8 @@ the Chromium-family process with `--force-renderer-accessibility`, or run ascreen reader that enables full renderer accessibility. On Sway and validatedGNOME Shell sessions, Cua Driver briefly focuses the exactcompositor-attested window for fixed internal-page navigation and restores the-previous window before continuing. GNOME requires the maintained WinRects-helper API v4 and an immutable D-Bus owner that resolves to the current user's+previous window before continuing. GNOME requires the maintained [WinRects helper](/how-to-guides/driver/install#install-the-gnome-winrects-helper)+API v4 and an immutable D-Bus owner that resolves to the current user'ssystem-installed GNOME Shell process. Cua Driver refuses generic Waylandsessions that cannot prove exact process, window, and geometry identity.diff --git a/docs/content/docs/reference/cua-driver/platform-support.mdx b/docs/content/docs/reference/cua-driver/platform-support.mdxindex 5a679fbe0c..efa0c561bc 100644--- a/docs/content/docs/reference/cua-driver/platform-support.mdx+++ b/docs/content/docs/reference/cua-driver/platform-support.mdx@@ -80,7 +80,7 @@ For the relationship between distributions, desktops, and compositors, read| X11/Xorg | Supported | Window discovery and capture, AT-SPI trees and actions, foreground pointer and keyboard input, semantic background delivery, desktop scope, and structured refusals | Raw background delivery depends on the target toolkit. X11 accepting an event does not prove the application handled it. || Sway (wlroots reference lane) | Supported with limits | The complete typed Electron, Tauri, GTK, capture, and desktop-scope catalog, including foreground/background and AX/PX outcomes | Focus-bound and raw background input shapes without a target-addressed protocol return structured refusals. Other wlroots compositors are expected to share protocol support but are not yet proven. || Hyprland / Omarchy | Experimental | [Qualified isolated input released in 0.24.0](#hyprland-and-omarchy), with separate native-harness and bounded app evidence; not general Hyprland support | Do not inherit Sway coverage. The default plugin build is discovery-only. The separate opt-in input v3 plugin is limited to the qualified applications and exact compositor ABI/toolchain. |-| GNOME/Mutter | Supported with limits | AT-SPI actions, GTK controls, compositor-backed window geometry and capture, verified foreground activation, and portal/libei foreground input | The bundled WinRects Shell helper and one Shell-session restart are prerequisites for authoritative geometry and activation. Portal video recording remains incomplete. |+| GNOME/Mutter | Supported with limits | AT-SPI actions, GTK controls, compositor-backed window geometry and capture, verified foreground activation, and portal/libei foreground input | The [bundled WinRects Shell helper](/how-to-guides/driver/install#install-the-gnome-winrects-helper) and one Shell-session restart are prerequisites for authoritative geometry and activation. Portal video recording remains incomplete. || KDE/KWin | Experimental | Plasma 6 session startup, GTK AT-SPI discovery, generic discovery where exposed, and portal interface availability | The optional KWin helper exposes read-only window identity and state, not activation or input. Raw target-addressed input refuses until a target-bound KWin input path exists; no complete behavioral matrix is accepted. || `cua-compositor` nested session | Experimental | Native GTK behavior, capture and scope, private route metadata, independent observation, and per-cell video | The complete shared renderer matrix is not accepted. Unicode text and a canonical parallel-drag row remain unproven; this route does not establish a stock-Wayland capability. || XWayland | Supported with limits | X11 routes are used when the application exposes a real X11 window; native Wayland and AT-SPI fallbacks cover mixed sessions | Capabilities depend on whether the application is actually using X11 or native Wayland. |