Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.
Rust agentai-agentapplecomputer-usecomputer-use-agentcontainerizationcuadesktop-automationhacktoberfestlumemacosmanusoperatorswiftvirtualizationvirtualization-frameworkwindowswindows-sandbox
feat(cua-driver): add a niri compositor adapter for Linux Wayland (#6209)
injaneity33a57bb0a82d
diff --git a/.github/workflows/e2e-rust-linux-wayland.yml b/.github/workflows/e2e-rust-linux-wayland.ymlindex b4560b08c2..054983fdcb 100644--- a/.github/workflows/e2e-rust-linux-wayland.yml+++ b/.github/workflows/e2e-rust-linux-wayland.yml@@ -16,6 +16,7 @@ on:- sway- sway-xwayland- cua-compositor+ - nirilane:description: "Diagnostic lane; canonical dispatches use all"required: false@@ -109,8 +110,8 @@ jobs:libxext-dev libdrm-dev libgbm-dev libwebkit2gtk-4.1-dev \libssl-dev libxdo-dev libayatana-appindicator3-dev librsvg2-dev \ffmpeg jq gnome-keyring- - name: Install Nix for the custom compositor environment- if: inputs.environment == 'cua-compositor'+ - name: Install Nix for the nested compositor environments+ if: inputs.environment == 'cua-compositor' || inputs.environment == 'niri'uses: cachix/install-nix-action@08dcb3a5e62fa31e2da3d490afc4176ef55ecd72 # v30with:extra_nix_config: |@@ -123,6 +124,9 @@ jobs:if [[ "${{ inputs.environment }}" == "cua-compositor" ]]; thennix develop .#cua-driver-inject-e2e \-c scripts/ci/linux/run-rust-e2e-inject.sh+ elif [[ "${{ inputs.environment }}" == "niri" ]]; then+ nix develop .#cua-driver-niri-e2e \+ -c scripts/ci/linux/run-rust-e2e-wayland.shelsescripts/ci/linux/run-rust-e2e-wayland.shfidiff --git a/docs/content/docs/cua-driver/concepts/platform-support.mdx b/docs/content/docs/cua-driver/concepts/platform-support.mdxindex 0397ab8ce5..b6075efab7 100644--- a/docs/content/docs/cua-driver/concepts/platform-support.mdx+++ b/docs/content/docs/cua-driver/concepts/platform-support.mdx@@ -36,6 +36,7 @@ routes where the app has a real X11 window.| --- | --- | --- | --- || X11/Xorg | Supported | Discovery, capture, AT-SPI, foreground input, semantic background actions, desktop scope | Raw background delivery depends on the toolkit. || Sway (wlroots) | Supported with limits | The full Electron, Tauri, GTK, and capture catalog | Focus-bound raw background input refuses. Other wlroots compositors unproven. |+| niri | Supported with limits | The full Electron, Tauri, GTK, and capture catalog on floating windows in a nested niri session | Needs niri 25.08 or later. No background input. niri has no per-toplevel capture protocol, so floating windows are cropped from the output. Tiled windows get no window capture or window-coordinate input; use `get_desktop_state` and desktop-scope actions. Overlapping floating windows refuse. Keep animations under 1 s. Client-side-decorated apps need `prefer-no-csd` for element frames. || GNOME/Mutter | Supported with limits | AT-SPI, GTK controls, compositor geometry and capture, foreground activation, portal/libei input | Needs the bundled [WinRects Shell helper](/cua-driver/quickstart) and one Shell restart. Portal video recording incomplete. || KDE/KWin | Experimental | Plasma 6 startup, GTK AT-SPI discovery, portal availability | The KWin helper is read-only; raw target-addressed input refuses. || Hyprland / Omarchy | Experimental | [Qualified isolated input](#hyprland-and-omarchy) since 0.24.0 | Does not inherit Sway coverage. |diff --git a/docs/content/docs/cua-driver/reference/mcp-tools/click-and-type.mdx b/docs/content/docs/cua-driver/reference/mcp-tools/click-and-type.mdxindex 127ff9b2b8..dff564def6 100644--- a/docs/content/docs/cua-driver/reference/mcp-tools/click-and-type.mdx+++ b/docs/content/docs/cua-driver/reference/mcp-tools/click-and-type.mdx@@ -84,7 +84,7 @@ button: "left" (default), "right", or "middle". Defaults to left so the field is| <span id="click--linux--button"></span>`button` | `"left" \| "right" \| "middle"` | | Mouse button. Default: "left" (legacy back-compat). X11: routed via ButtonPress/Release with the matching evdev code. Native Wayland: only left-button is supported via the virtual-pointer protocol; right/middle return an error. || <span id="click--linux--capture_id"></span>`capture_id` | `string` | | Optional one-shot binding to the exact PNG returned by get_window_state or get_desktop_state. When present, x/y are interpreted in that capture and admitted before native dispatch. || <span id="click--linux--coordinate_frame"></span>`coordinate_frame` | `"window" \| "desktop"` | | Frame of x/y (and from_x/from_y/to_x/to_y). Default "window": window-local screenshot pixels as returned by get_window_state. "desktop": full-screen pixels as returned by get_desktop_state, translated to the target window. Passing scope:"desktop" together with pid/window_id means the same thing. |-| <span id="click--linux--count"></span>`count` | `integer` | | Number of clicks in one press train: 1 (default), 2 = double-click, 3 = triple-click (line/paragraph selection). There is no separate triple_click tool: pass count: 3 here. Minimum: 1. |+| <span id="click--linux--count"></span>`count` | `integer` | | Number of clicks in one press train: 1 (default), 2 = double-click, 3 = triple-click (line/paragraph selection). There is no separate triple_click tool: pass count: 3 here. Range: 1 to 3. || <span id="click--linux--cursor_id"></span>`cursor_id` | `string` | | Optional multi-cursor instance id. Default: 'default'. || <span id="click--linux--delivery_mode"></span>`delivery_mode` | `"background" \| "foreground"` | `"background"` | Input delivery mode. 'background' (default) never activates or raises the target window. On X11 it injects via XTEST / the XInput2 master pointer (no focus steal). On Wayland it goes through libei + xdg-desktop-portal, which injects to the compositor's input focus: Wayland's security model has no per-window background targeting, so a specific non-focused window cannot be aimed at; when no libei backend is available the tool returns a structured background_unavailable error. 'foreground' is the explicit escalation: activate the target (X11 _NET_ACTIVE_WINDOW; Wayland compositor activate), inject, then restore the prior active window; a brief focus swap unless the target was already active. Matches the macOS / Windows delivery_mode surface. || <span id="click--linux--from_zoom"></span>`from_zoom` | `boolean` | | Set true after a zoom call to auto-translate zoom-image pixel coordinates back to full-window space. |diff --git a/docs/contributing/TESTING.md b/docs/contributing/TESTING.mdindex dbfb50c824..ee1d00fac0 100644--- a/docs/contributing/TESTING.md+++ b/docs/contributing/TESTING.md@@ -140,6 +140,7 @@ Canonical entry points:```textLinux X11/session: scripts/ci/linux/run-rust-e2e.shLinux Sway: scripts/ci/linux/run-rust-e2e-wayland.sh+Linux niri: CUA_E2E_WAYLAND_SESSION=niri scripts/ci/linux/run-rust-e2e-wayland.shLinux nested: scripts/ci/linux/run-rust-e2e-inject.shLinux GNOME/KDE: scripts/ci/linux/run-rust-e2e-desktop.sh <gnome|kde>Linux real Xorg: scripts/ci/linux/run-rust-e2e-desktop.sh xorg@@ -147,7 +148,15 @@ Windows: .\scripts\ci\windows\run-rust-e2e.ps1 -RequireGuimacOS: scripts/ci/macos/run-rust-e2e.sh```-The hosted Sway and nested-compositor runners create controlled sessions.+The hosted Sway, nested niri, and nested-compositor runners create controlled+sessions. The niri lane covers floating-window and desktop-scope input only;+tiled windows, overlapping floating windows, an open overview and more than one+output are refused (see `libs/cua-driver/rust/Skills/cua-driver/LINUX.md`).+Unit tests for bounded input workers and late-send refusal live in+`platform-linux/src/input/action.rs` and `input/mod.rs`; niri pointer admission+goes through `wayland::niri::prepare_target`. Window-coordinate niri input+carries the window rectangle its point was translated against+(`wayland::AimedPoint`) from the tool body to that gate.GNOME, KDE, real Xorg, Windows, and macOS use an existing graphical login. Thesuites are often maintainer-triggered and retain typed case/results,screenshots, accessibility state, trajectories, logs, and video where the lanediff --git a/flake.nix b/flake.nixindex 636043f338..e92b61f9f3 100644--- a/flake.nix+++ b/flake.nix@@ -108,12 +108,28 @@webkitgtk_4_1];+ # wf-recorder 0.6.0 always binds linux-dmabuf version 4. niri without+ # a DRM render node (any GPU-less runner) advertises version 3, so the+ # bind is a protocol error and recording never starts. Bind the+ # advertised version and request dmabuf feedback only when it exists.+ niriWfRecorder = pkgs.wf-recorder.overrideAttrs (old: {+ postPatch = (old.postPatch or "") + ''+ substituteInPlace src/main.cpp \+ --replace-fail "const char *interface, uint32_t) {" \+ "const char *interface, uint32_t version) {" \+ --replace-fail "&zwp_linux_dmabuf_v1_interface, 4);" \+ "&zwp_linux_dmabuf_v1_interface, version < 4 ? version : 4);" \+ --replace-fail "if (dmabuf) {" "if (dmabuf && version >= 4) {"+ '';+ });+waylandE2eShell = extraPackages: pkgs.mkShell {# hostAtSpi is referenced by absolute launcher path below, but is# deliberately not a shell package: adding its rebuilt library and# typelib hooks alongside GTK's stock AT-SPI closure loads two ATK# copies and crashes PyGObject during Gtk import.- packages = (with pkgs; [+ # Extra packages come first so a lane can shadow a base tool.+ packages = extraPackages ++ (with pkgs; [cargoclangchromium@@ -135,7 +151,7 @@wtype# Keep the GTK3 fixture on the mature Python/PyGObject combination.(python312.withPackages (pythonPackages: [ pythonPackages.pygobject3 ]))- ]) ++ extraPackages;+ ]);buildInputs = waylandE2eLibraries;LD_LIBRARY_PATH = pkgs.lib.makeLibraryPath waylandE2eLibraries;shellHook = ''@@ -172,6 +188,32 @@devShells.cua-driver-wayland-e2e = waylandE2eShell [ ];devShells.cua-driver-inject-e2e = waylandE2eShell [ cuaCompositorPackage ];+ # niri has no headless backend: it runs on its winit backend inside a+ # headless cage, and wlr-randr sizes cage's output. niri renders only+ # through EGL, so name the store's Mesa vendor (llvmpipe without a GPU)+ # instead of whatever the host's glvnd directories provide.+ # GTK on Wayland ignores gtk-xft-dpi in settings.ini and takes it from+ # GSettings' text-scaling-factor; without that schema, WebKitGTK+ # reads a screen DPI of -1 and lays pages out at devicePixelRatio+ # -1/96. Name the store's schemas and keep their defaults in memory,+ # so neither the host's schemas nor its dconf play a part.+ devShells.cua-driver-niri-e2e =+ (waylandE2eShell (+ with pkgs;+ [+ cage+ niri+ niriWfRecorder+ wlr-randr+ ]+ )).overrideAttrs+ (old: {+ shellHook = old.shellHook + ''+ export __EGL_VENDOR_LIBRARY_FILENAMES="${pkgs.mesa}/share/glvnd/egl_vendor.d/50_mesa.json"+ export GSETTINGS_SCHEMA_DIR="${pkgs.glib.getSchemaPath pkgs.gsettings-desktop-schemas}"+ export GSETTINGS_BACKEND=memory+ '';+ });})// {diff --git a/libs/cua-driver/README.md b/libs/cua-driver/README.mdindex 9d267f50d7..af9865102c 100644--- a/libs/cua-driver/README.md+++ b/libs/cua-driver/README.md@@ -118,6 +118,7 @@ The canonical GUI runners execute the complete Rust harness catalog:```textLinux X11/session: scripts/ci/linux/run-rust-e2e.shLinux Sway: scripts/ci/linux/run-rust-e2e-wayland.sh+Linux niri: CUA_E2E_WAYLAND_SESSION=niri scripts/ci/linux/run-rust-e2e-wayland.shLinux nested: scripts/ci/linux/run-rust-e2e-inject.shLinux GNOME/KDE: scripts/ci/linux/run-rust-e2e-desktop.sh <gnome|kde>Linux real Xorg: scripts/ci/linux/run-rust-e2e-desktop.sh xorgdiff --git a/libs/cua-driver/docs/action-support.md b/libs/cua-driver/docs/action-support.mdindex a94369a600..ab30295aeb 100644--- a/libs/cua-driver/docs/action-support.md+++ b/libs/cua-driver/docs/action-support.md@@ -22,6 +22,7 @@ when that is the background-safe route.| macOS/Quartz | `0b9b5d94` and `64e82449` | Logged-in local matrix: 138 deliveries and 6 exact refusals passed; one cursor-interrupted row passed alone on the final head, for effective 145/145 coverage || Linux/X11 | `268a6ab1` | Run `29254936043`: 116/116 rows, 75 delivered and 41 exact refusals || Linux/Sway | `268a6ab1` and `64e82449` | Run `29255208927` passed native and capture 36/36; run `29257961614` passed shared 80/80, for effective 116/116 coverage |+| Linux/niri | `48ad4918fc43d9e3dcc0b0365d4973a35d48e3e9` | Run [`39243725489`](https://github.com/ayagmar/cua/actions/runs/39243725489) passed native 39/39, capture 8/8, and shared 85/85 on nested niri 26.04, for 132/132 rows: 84 delivered and 48 exact refusals. Sway run [`39243729097`](https://github.com/ayagmar/cua/actions/runs/39243729097) passed 132/132, 85 delivered and 47 exact refusals; macOS unit run [`39243741639`](https://github.com/ayagmar/cua/actions/runs/39243741639) passed. A local capture smoke at scale 1.5 passed 8/8 |## Shared web harnesses@@ -76,6 +77,7 @@ are capabilities, not one uniform API.| --- | --- | --- || X11/Openbox | Exact run `29254936043` passed all 116 declared outcomes: 75 deliveries and 41 exact refusals. GTK3 AT-SPI actions and values, foreground XTest pointer/keyboard routes, capture, desktop scope, and the complete Electron/Tauri catalog all reached fixture-owned oracles. | Xvfb does not prove the real-Xorg MPX/uinput background pointer route. Toolkits that reject XSendEvent retain exact refusals. || Sway/wlroots | Runs `29255208927` and `29257961614` passed the complete 116 outcomes: native and capture 36/36, then shared 80/80. The Electron, Tauri, GTK3, capture, and desktop-scope catalogs use live Sway identity/geometry and external fixture oracles. | Stock Wayland cannot target raw focus-bound input at an occluded surface. Those rows retain exact refusals; another wlroots compositor is not assumed equivalent until a material divergence is reported and tested. Exact cursor preservation remains unproven and is tracked in [issue #2194](https://github.com/trycua/cua/issues/2194). |+| niri | Run [`39243725489`](https://github.com/ayagmar/cua/actions/runs/39243725489) at `48ad4918fc43d9e3dcc0b0365d4973a35d48e3e9` passed all 132 declared outcomes on nested niri 26.04: 84 deliveries and 48 exact refusals. The lane runs niri's winit backend inside a headless cage with one 1920x1080 output, animations off, `prefer-no-csd`, and floating harness windows. A capture smoke at scale 1.5 passed 8/8. Window identity is niri's window id from its IPC socket, used only when the socket's peer is the compositor serving `WAYLAND_DISPLAY`. Activation is niri `FocusWindow`, read back from IPC, and foreground actions give focus back to the previously focused window, except `right_click` and `hotkey`: as on Sway they leave the target focused, because giving focus back closes the menu they opened. Before niri is recognised, a session probe that times out refuses the call rather than taking the generic Wayland routes. Floating windows get window capture and window-coordinate pointer input from niri's floating geometry. Display capture applies the output transform. Text, keys, and chords go out on niri's own keymap through a virtual keyboard (not wtype); characters the layout lacks go on spare keys of a temporary keymap, and niri's keymap is restored afterwards. Focus is checked before every key. Tiled windows, niri's default layout, get no window capture and no window-coordinate pointer input, because niri does not report where they are; those calls refuse and name `get_desktop_state`. Desktop-scope pointer input on niri takes pixels of the single output as `get_desktop_state` shows it; the driver does not prove that screenshot is still current. Unit tests and a nested smoke cover this path, not this matrix. `coordinate_frame:"desktop"` with a window target refuses. `list_windows` gives tiled windows no bounds and reports one on screen only while it is the active window of a shown workspace. | Needs niri 25.08 or later; older releases are recognized and refused. niri has no per-toplevel capture protocol and no RemoteDesktop/libei, so there is no background input, and the libei/portal recovery is refused. Overlapping floating windows refuse the affected point or crop, because niri IPC has no stacking order. Floating windows are treated as hidden behind a fullscreen tile when the active tile is the output's size, because niri 26.04 IPC has no fullscreen flag. niri IPC reports where a window will be, not where an animation currently draws it, and reports no animation state. Window-coordinate input and window capture first re-read niri's layout, at most five reads 30 ms apart, until two consecutive reads agree on the window, refusing a window that keeps moving; they then check it again before every event. A window still animating can still be checked at its destination, and a move away and back between two reads is not detected. niri also scrolls its view without an event for `center-column`, `center-visible-columns`, `center-window`, and a view gesture that ends on the same column; nothing detects that. Layer-shell surfaces (bars, notifications, launchers) are not in niri IPC, so one drawn over a window can take a click or appear in its capture. With several outputs, screenshots and screen size work; window capture, `get_desktop_state`, window-coordinate pointer input, and desktop-scope input refuse. Element frames are published only when the app's accessible frame is niri's window size. With niri's default client-side decorations, GTK 3 and Chromium frames include shadow margins niri does not report, so those windows have no element frames; `prefer-no-csd` fixes GTK 3. Multi-window apps whose accessible roots do not uniquely match their niri titles publish no element tokens. A click train (at most 3 clicks) is checked before every press and a scroll before every tick; once one has landed, a failed check stops the rest with `click stopped after i of n presses` or `scroll stopped after i of n ticks`. A `drag` rechecks the target window's place and the pointer space before every motion; if either changed, it releases the button where the pointer is and stops with `drag stopped after i of n motions`. A held button (`mouse_button_down`, then `mouse_drag`) rereads the pointer space before every motion; if an output was added or rescaled, the drag stops and the button is released where the pointer was. Every Wayland wait of a niri capture or pointer action has a deadline, so a niri that stops answering ends the call instead of hanging it; once a call has timed out or its client cancelled it, its worker sends no further input (including a held `mouse_button_down`/`mouse_drag` pointer), a button it pressed is released, and focus it took goes back to the window that had it; the next desktop action waits until that worker has stopped, for at most 12 s counted from when the timed-out call ended, then refuses with `input_busy` saying a previous call's input is still finishing; retry once it has stopped; a worker that never stops keeps later desktop actions refusing until the driver restarts. niri can move focus between the check and a key, so one key can reach a newly focused window; the error then says so. The driver does not check whether the session is locked: a locked niri reports no focused window, so keyboard and window-addressed pointer input refuse, but desktop-scope pointer input and capture are not refused. Chromium cuts text outside the Basic Multilingual Plane to 16 bits (😀 arrived as U+F600). Browser identity routes are not accepted on niri (no browser matrix ran); the embedded Electron browser row is the exact refusal `browser_wrong_target_refused`, and existing-profile browser setup refuses before it touches the clipboard. No XWayland variant ran. || GNOME/Mutter | A real GNOME 46 Wayland run passed the full GTK3 matrix, 31/31. The WinRects helper supplies stable window ids, frame and buffer geometry, stacking, verified activation, stage capture, and the compositor cursor. AT-SPI handles semantic actions; persistent portal/libei sessions deliver foreground PX click, right/double click, drag, scroll, type, key, and hotkey. Background rows either deliver through AT-SPI with focus and leak guards or return the declared exact refusal. | The helper requires installation plus one Shell-session restart. Without it, target-bound foreground input refuses. Portal video and a shared Electron/Tauri GNOME run remain open. || KDE/KWin | A Plasma 6 Wayland session has a live-validated Cua-owned native KWin identity adapter: trusted in-process helper, opaque KWin-lifetime tokens, exact PID-and-token window lookup, active/minimized metadata, and live-window token pruning. | Raw target-addressed keyboard and pointer actions are intentionally refused. KWin portal/libei delivery is focus-bound, so activation plus read-back cannot prevent focus changing before compositor EIS processing. Delivery remains disabled until a KWin input path binds the mutation itself to the target token/window. The ABI-specific helper remains optional and is currently obtained by a host build. || Nested `cua-compositor` | The optional backend owns its nested session and is covered by typed shared, native, and capture catalogs. Run `29197643541` passed native GTK3 31/31; run `29197887387` passed capture/scope 5/5. Full shared run `29199596935` passed 26/36 Electron rows before the occluded-subtree and wheel-frame repairs. | The lane remains experimental. The accepted full shared run still has 10 failures; focused replacement runs reduce that set but do not promote the environment. Unicode text and a canonical parallel-drag row remain unproven. This private route is not evidence of a stock-Wayland capability. |diff --git a/libs/cua-driver/docs/test-harnesses-guide.md b/libs/cua-driver/docs/test-harnesses-guide.mdindex d0efb0fcca..86d91e8f84 100644--- a/libs/cua-driver/docs/test-harnesses-guide.md+++ b/libs/cua-driver/docs/test-harnesses-guide.md@@ -377,7 +377,16 @@ Wayland session. Linux does not need GIF output; MP4, screenshots, accessibilitytrees, trajectories, and logs are the useful evidence.Wayland results are compositor-specific. The hosted lane uses Sway to prove-wlroots protocols. GNOME requires the optional WinRects Shell helper for+wlroots protocols. The niri lane (`CUA_E2E_WAYLAND_SESSION=niri` in+`nix develop .#cua-driver-niri-e2e`) runs niri's winit backend inside a+headless cage; its observer reads niri IPC, which reports no stacking order and+no tiled-window positions, and cannot tell a fullscreen tile from a maximized+one. It reports a floating harness window as occluded only behind the focused+sentinel while the sentinel itself reports being fullscreen, and one that+overlaps no other floating window as visible. It refuses tiled targets,+overlapping floating windows, an open overview, and any other output-sized+tile, including a maximized sentinel.+GNOME requires the optional WinRects Shell helper forauthoritative frame and buffer geometry, observation, capture, and verifiedtarget activation. A portal/libei grant persists until the user revokes it, sosubsequent driver processes do not reopen the consent dialog.diff --git a/libs/cua-driver/docs/test-matrix.md b/libs/cua-driver/docs/test-matrix.mdindex a6cf57db16..2d2cf7a24c 100644--- a/libs/cua-driver/docs/test-matrix.md+++ b/libs/cua-driver/docs/test-matrix.md@@ -216,8 +216,8 @@ transient-panel AX discovery gap.| Harness | Source test | Window system | Coverage || --- | --- | --- | --- |-| Electron | `cross_platform_behavior_test.rs` | X11 and hosted Sway | Shared web action matrix |-| Tauri | `cross_platform_behavior_test.rs` | X11 and hosted Sway | Shared web action matrix |+| Electron | `cross_platform_behavior_test.rs` | X11, hosted Sway, and nested niri | Shared web action matrix |+| Tauri | `cross_platform_behavior_test.rs` | X11, hosted Sway, and nested niri | Shared web action matrix || GTK3 | `harness_gtk3_test.rs` | X11/AT-SPI and Wayland/AT-SPI where configured | Native GTK controls and input || Desktop target | `desktop_scope_linux_test.rs` | X11/Wayland | Per-call window/desktop modality, screen-absolute action delivery, and strict rejection |@@ -263,6 +263,7 @@ allowed, and the desktop-side-effect oracles pass.| Windows Harness E2E | Active Windows user session | `scripts/ci/windows/run-rust-e2e.ps1 -RequireGui` || Linux X11 Harness E2E | Nix X11 session | `scripts/ci/linux/run-rust-e2e.sh` || Linux Sway Harness E2E | Controlled wlroots session | `scripts/ci/linux/run-rust-e2e-wayland.sh` |+| Linux niri Harness E2E | Nested niri in a headless cage | `CUA_E2E_WAYLAND_SESSION=niri scripts/ci/linux/run-rust-e2e-wayland.sh` || Linux nested-compositor E2E | Controlled experimental session | `scripts/ci/linux/run-rust-e2e-inject.sh` || Linux representative desktop E2E | Existing GNOME, KDE, or Xorg login | `scripts/ci/linux/run-rust-e2e-desktop.sh <desktop>` || macOS Harness E2E | Maintainer Lume SIP-off worker with a logged-in session and inherited grants, or a disposable worker prepared with the checked-in macOS consent seed helper | `libs/cua-driver/tests/runners/macos-lume/run-all.sh` |diff --git a/libs/cua-driver/docs/wayland-compositor-identity-plan.md b/libs/cua-driver/docs/wayland-compositor-identity-plan.mdindex 6bd666c08a..a788c5b71a 100644--- a/libs/cua-driver/docs/wayland-compositor-identity-plan.md+++ b/libs/cua-driver/docs/wayland-compositor-identity-plan.md@@ -72,6 +72,14 @@ wlroots compositors may expose additional management or compositor IPC. Eachroute remains compositor-specific until it proves all required fields. Noscreen-coordinate fallback is allowed.+niri (not a wlroots compositor) reports window identity on its IPC socket:+a window id, PID, title, application ID, workspace, and, for floating windows,+geometry. Cua Driver uses that socket only when its peer is the compositor+serving `WAYLAND_DISPLAY`. Once niri is recognized, a lost socket, a changed+peer, or a new compositor process refuses instead of falling back to generic+discovery. Browser attachment through this route is not accepted: no niri+browser matrix has run against the threat cases below.+## Threat CasesEach adapter and harness must cover:diff --git a/libs/cua-driver/rust/Skills/cua-driver/LINUX.md b/libs/cua-driver/rust/Skills/cua-driver/LINUX.mdindex 5364943c21..b053e9c89f 100644--- a/libs/cua-driver/rust/Skills/cua-driver/LINUX.md+++ b/libs/cua-driver/rust/Skills/cua-driver/LINUX.md@@ -365,6 +365,50 @@ packaging and lifecycle validation require separate evidence. Physical Omarchyparity requires separate acceptance; it is not a gate for publishing a validatedimage.+### niri++niri 25.08 or later is recognised when `NIRI_SOCKET` and `WAYLAND_DISPLAY`+belong to the same niri process; window ids are niri's own. niri has no+RemoteDesktop or libei, so input is foreground only and the libei/portal+recovery refuses. Refusals carry `foreground_unavailable` and a `suggestion`+that names the remedy.++- Floating windows get window capture, cropped from the output, and+ window-coordinate pointer input from niri's floating geometry. A point under+ another floating window refuses, because niri reports no stacking order.+- Tiled windows, niri's default layout, refuse window capture and+ window-coordinate pointer input: niri does not report where they are.+ `get_window_state` still returns their accessibility tree, and element+ actions work. For pixels, call `get_desktop_state` and act at desktop scope+ (no `pid` or `window_id`), or float the window.+- Desktop-scope pointer input takes pixels of the single output, as+ `get_desktop_state` shows it, and refuses points past the output edge.+ niri does not report where tiled windows are or every scroll of its view,+ so the driver does not prove a screenshot is still current: take a new one+ after anything that may have moved the layout.+- Window-coordinate pointer input and window capture first wait, at most+ five reads 30 ms apart, for two consecutive reads that agree on the+ window's place, and refuse a window that keeps moving. They then check it+ again before every press, wheel tick or motion, along with exact keyboard+ focus for pointer input. A move away and back between two reads, or an+ animation still drawing the window elsewhere, is not detected.+- The driver does not check whether the session is locked. At a lock screen+ niri reports no focused window, so keyboard and window-coordinate pointer+ input refuse; desktop-scope pointer input and capture are not refused.+- `coordinate_frame:"desktop"` with a window target refuses.+- Foreground actions activate the target, act, and give focus back to the+ window that had it. `right_click` and `hotkey` leave the target focused, as+ on Sway: giving focus back would close the menu they opened. Text and keys+ use niri's own keymap, and focus is checked before every key.+- `list_windows` gives tiled windows 0x0 bounds, and a tiled window reads+ `is_on_screen:true` only while it is the active window of a shown workspace.+ With a floating window active, visible tiled windows read `false`, so do not+ filter niri windows on `is_on_screen`.+- With more than one output, window capture, `get_desktop_state`,+ window-coordinate pointer input, and desktop-scope input refuse.+- A floating window that moves between the coordinate translation and the+ input refuses the input; take a new screenshot and retry.+## Quick triageIf a tool call surprises you on Linux:@@ -384,6 +428,20 @@ If a tool call surprises you on Linux:(`gsettings set org.gnome.desktop.interface toolkit-accessibility true`);(c) GTK4 / Qt6 / Chromium populate lazily: re-snapshot after aninteraction or an AX-enable settle.+4. **`input_busy` saying a previous call's input is still finishing**: an+ earlier call timed out or was cancelled, and its worker still holds the+ desktop. Once a call is over, its worker sends no further XTest or+ virtual-device input; what can still run is cleanup (releasing a held+ button or key, giving focus back) and a `wtype` child already started,+ which keeps typing until it exits. The 12 s bound counts from when that+ call ended, not from when the next action arrives: a desktop action that+ finds the worker stale for 12 s or more refuses at once, before sending+ anything, and one that arrives sooner waits only for the rest of it.+ Retry once the worker has stopped. A worker that never stops, such as+ one blocked on a compositor or X server that stopped answering, keeps+ every later desktop action refusing until the driver restarts. An+ `input_busy` that names a pid is different: text input to that process is+ already running.## Forbidden vectorsdiff --git a/libs/cua-driver/rust/crates/cua-driver-core/Cargo.toml b/libs/cua-driver/rust/crates/cua-driver-core/Cargo.tomlindex 5000ae69e4..67f4be712a 100644--- a/libs/cua-driver/rust/crates/cua-driver-core/Cargo.toml+++ b/libs/cua-driver/rust/crates/cua-driver-core/Cargo.toml@@ -79,3 +79,6 @@ windows = { version = "0.58", features = ["Win32_System_Threading","Win32_UI_WindowsAndMessaging",] }++[dev-dependencies]+tokio = { workspace = true, features = ["test-util"] }diff --git a/libs/cua-driver/rust/crates/cua-driver-core/src/recovery_hint.rs b/libs/cua-driver/rust/crates/cua-driver-core/src/recovery_hint.rsindex 9b7bc262f2..bb749cc11c 100644--- a/libs/cua-driver/rust/crates/cua-driver-core/src/recovery_hint.rs+++ b/libs/cua-driver/rust/crates/cua-driver-core/src/recovery_hint.rs@@ -310,7 +310,7 @@ static RULES: &[Rule] = &[},},Rule {- codes: &["target_changed", "stale_target", "stale_geometry"],+ codes: &["target_changed", "stale_target", "stale_observation"],build: |c| {RecoveryHint::new("The window changed since your last read, so nothing was sent. Read it again and aim from \diff --git a/libs/cua-driver/rust/crates/cua-driver-core/src/recovery_hint/tests.rs b/libs/cua-driver/rust/crates/cua-driver-core/src/recovery_hint/tests.rsindex af032110de..2bbdda2f3a 100644--- a/libs/cua-driver/rust/crates/cua-driver-core/src/recovery_hint/tests.rs+++ b/libs/cua-driver/rust/crates/cua-driver-core/src/recovery_hint/tests.rs@@ -144,7 +144,7 @@ fn the_table_covers_every_code_agents_hit_most() {"element_bounds_unavailable","target_changed","stale_target",- "stale_geometry",+ "stale_observation","desktop_changed","point_outside_window","px_frame_mismatch",@@ -301,7 +301,7 @@ fn a_window_target_comes_from_the_typed_target_when_the_call_has_no_top_level_id"click",&seed(json!({"target": {"kind": "window", "pid": 9, "window_id": 4}, "x": 1, "y": 2})),true,- json!({"code": "stale_geometry"}),+ json!({"code": "stale_observation"}),);assert_eq!(call_value(&hint),diff --git a/libs/cua-driver/rust/crates/cua-driver-core/src/tool.rs b/libs/cua-driver/rust/crates/cua-driver-core/src/tool.rsindex 845e3d6312..961e35dbc4 100644--- a/libs/cua-driver/rust/crates/cua-driver-core/src/tool.rs+++ b/libs/cua-driver/rust/crates/cua-driver-core/src/tool.rs@@ -36,6 +36,8 @@ tokio::task_local! {/// Opaque generation for runtime-owned mutable resources. Nested/// dispatches inherit this key, while public arguments can never select it.static DISPATCH_RUNTIME_SCOPE: String;+ /// The physical desktop action this task dispatches.+ static DESKTOP_LEASE: DesktopLease;}/// Return the immutable authorization context bound to the current dispatch.@@ -71,9 +73,154 @@ pub fn with_runtime_scope<T>(scope: String, action: impl FnOnce() -> T) -> T {action()}-fn desktop_action_coordinator() -> &'static tokio::sync::Mutex<()> {- static COORDINATOR: OnceLock<tokio::sync::Mutex<()>> = OnceLock::new();- COORDINATOR.get_or_init(|| tokio::sync::Mutex::new(()))+/// How long a call waits for the desktop lane once its holder is a worker+/// whose call already ended. It matches the base budget of one foreground+/// input call, so a waiting call never outlasts its own budget behind it.+const STALE_LANE_WAIT: Duration = Duration::from_secs(12);++/// The process-wide physical input lane: one desktop action at a time.+struct DesktopLane {+ lock: Arc<tokio::sync::Mutex<()>>,+ holder: Arc<Mutex<LaneHolder>>,+ stale_wait: Duration,+}++/// Which hold of the lane is current, and when its call ended while a+/// worker still held the lane. A mark belongs to its holder's generation,+/// so it never applies to the holder after it.+#[derive(Default)]+struct LaneHolder {+ generation: u64,+ stale_since: Option<tokio::time::Instant>,+}++fn desktop_lane() -> &'static DesktopLane {+ static LANE: OnceLock<DesktopLane> = OnceLock::new();+ LANE.get_or_init(|| DesktopLane::new(STALE_LANE_WAIT))+}++fn lock_holder(holder: &Mutex<LaneHolder>) -> std::sync::MutexGuard<'_, LaneHolder> {+ holder+ .lock()+ .unwrap_or_else(|poisoned| poisoned.into_inner())+}++impl DesktopLane {+ fn new(stale_wait: Duration) -> Self {+ Self {+ lock: Arc::new(tokio::sync::Mutex::new(())),+ holder: Arc::new(Mutex::new(LaneHolder::default())),+ stale_wait,+ }+ }++ /// Wait for the lane. Waiting behind a call still running is ordinary+ /// serialization; waiting behind a worker that outlived its call gives+ /// up after `stale_wait` with an `input_busy` refusal.+ async fn acquire(&self) -> Result<DesktopCall, ToolResult> {+ // Avoid yielding the dispatch task when the process-wide input+ // lane is uncontended. On Windows, that yield creates a window in+ // which the foreground target can lose keyboard eligibility+ // between the fixture's focus proof and SendInput. Contended+ // runtimes still wait and serialize through the same mutex.+ let guard = match self.lock.clone().try_lock_owned() {+ Ok(guard) => guard,+ Err(_) => {+ let wait = self.lock.clone().lock_owned();+ tokio::pin!(wait);+ loop {+ // Read the holder on every pass, the first one before any+ // wait included: a stale worker seen earlier may have+ // stopped, and its successor be a call still running.+ let stale_since = lock_holder(&self.holder).stale_since;+ if stale_since+ .is_some_and(|since| tokio::time::Instant::now() >= since + self.stale_wait)+ {+ return Err(protected_refusal(+ "input_busy",+ "a previous call's input is still finishing: its call ended, but its worker still holds the desktop; retry once it has stopped",+ ));+ }+ tokio::select! {+ guard = &mut wait => break guard,+ _ = tokio::time::sleep(Duration::from_millis(100)) => {}+ }+ }+ }+ };+ let generation = {+ let mut holder = lock_holder(&self.holder);+ holder.generation += 1;+ holder.stale_since = None;+ holder.generation+ };+ Ok(DesktopCall(DesktopLease(Arc::new(HeldLane {+ holder: self.holder.clone(),+ generation,+ _guard: guard,+ }))))+ }+}++struct HeldLane {+ holder: Arc<Mutex<LaneHolder>>,+ generation: u64,+ _guard: tokio::sync::OwnedMutexGuard<()>,+}++impl Drop for HeldLane {+ fn drop(&mut self) {+ // Runs before `_guard` unlocks the lane for the next holder.+ let mut holder = lock_holder(&self.holder);+ if holder.generation == self.generation {+ holder.stale_since = None;+ }+ }+}++/// One physical desktop action's hold on the process-wide input lane, freed+/// once every clone is dropped. A platform worker that outlives its call (it+/// timed out, or its caller went away) keeps a clone, as does any cleanup it+/// hands off, so the next desktop action waits until both have stopped+/// instead of overlapping their late input, for at most [`STALE_LANE_WAIT`].+#[derive(Clone)]+pub struct DesktopLease(Arc<HeldLane>);++impl DesktopLease {+ /// The lease of the desktop action this task runs, if it holds one.+ pub fn current() -> Option<Self> {+ DESKTOP_LEASE.try_with(Clone::clone).ok()+ }+}++/// The dispatching call's own hold on the desktop lane. Dropping it while a+/// worker still holds a [`DesktopLease`] clone marks that worker stale.+pub struct DesktopCall(DesktopLease);++impl DesktopCall {+ /// Wait for the process-wide input lane, or refuse with `input_busy`+ /// once a worker that outlived its call has held it too long.+ pub async fn acquire() -> Result<Self, ToolResult> {+ desktop_lane().acquire().await+ }++ /// Run `action` as the desktop action holding this lane, so that+ /// [`DesktopLease::current`] returns it.+ pub async fn scope<F: std::future::Future>(&self, action: F) -> F::Output {+ DESKTOP_LEASE.scope(self.0.clone(), action).await+ }+}++impl Drop for DesktopCall {+ fn drop(&mut self) {+ let held = &self.0 .0;+ // Checked under the holder lock that the last clone's drop takes,+ // so a worker stopping now cannot leave its mark behind.+ let mut holder = lock_holder(&held.holder);+ if holder.generation == held.generation && Arc::strong_count(held) > 1 {+ holder.stale_since = Some(tokio::time::Instant::now());+ }+ }}fn active_text_input_pids() -> &'static Mutex<HashSet<i64>> {@@ -2280,23 +2427,17 @@ impl ToolRegistry {| "end_session");let private_consent_turn = is_existing_profile_prepare(resolved_name, &args);- let _desktop_action = if requires_desktop_coordination(+ let (desktop_call, lane_refusal) = if requires_desktop_coordination(resolved_name,&args,tool.has_independent_input_lane(&args),) {- let coordinator = desktop_action_coordinator();- // Avoid yielding the dispatch task when the process-wide input- // lane is uncontended. On Windows, that yield creates a window in- // which the foreground target can lose keyboard eligibility- // between the fixture's focus proof and SendInput. Contended- // runtimes still wait and serialize through the same mutex.- Some(match coordinator.try_lock() {- Ok(guard) => guard,- Err(_) => coordinator.lock().await,- })+ match DesktopCall::acquire().await {+ Ok(call) => (Some(call), None),+ Err(refusal) => (None, Some(refusal)),+ }} else {- None+ (None, None)};let pending_turn = should_record.then(|| {@@ -2336,11 +2477,16 @@ impl ToolRegistry {// back to the uncapped capture before any platform interprets them.crate::desktop_capture_scale::map_desktop_args(&mut args);crate::tool_activity::note_tool_activity();- let mut result = crate::recording::scope_dispatch_click_capture(- pending_turn.as_ref(),- tool.invoke(args.clone()),- )- .await;+ let invoke = tool.invoke(args.clone());+ let mut result =+ crate::recording::scope_dispatch_click_capture(pending_turn.as_ref(), async {+ match (&desktop_call, lane_refusal) {+ (_, Some(refusal)) => refusal,+ (Some(call), None) => call.scope(invoke).await,+ (None, None) => invoke.await,+ }+ })+ .await;match resolved_name {"get_desktop_state" if result.is_error != Some(true) => {crate::desktop_capture_scale::record_desktop_state(@@ -2399,8 +2545,9 @@ impl ToolRegistry {// Coordinate the physical action itself, not post-action evidence// capture or result shaping. Keeping the global desktop lock through// recording/PiP screenshots would unnecessarily block an unrelated- // runtime after the input side effect has already completed.- drop(_desktop_action);+ // runtime after the input side effect has already completed. A+ // platform worker still running keeps its own clone of the lease.+ drop(desktop_call);restore_public_runtime_result(&mut result, &runtime_prefix);// Preserve the producer's private summary for recording/replay before// the public ActionResult projection deliberately replaces legacy@@ -3611,9 +3758,9 @@ fn restore_public_runtime_value(value: &mut Value, runtime_prefix: &str) -> bool#[cfg(test)]mod runtime_isolation_tests {use super::{- canonical_proposed_path, desktop_action_coordinator, namespace_runtime_args,- publish_action_result, restore_public_runtime_result, try_admit_text_input,- TrustedInvocationEvidence, DISPATCH_RUNTIME_SCOPE,+ canonical_proposed_path, namespace_runtime_args, publish_action_result,+ restore_public_runtime_result, try_admit_text_input, DesktopCall, DesktopLane,+ DesktopLease, TrustedInvocationEvidence, DISPATCH_RUNTIME_SCOPE,};use crate::{authorization::PermissionMode,@@ -6128,7 +6275,7 @@ resources:let active = active.clone();let max_active = max_active.clone();tasks.push(tokio::spawn(async move {- let _admission = desktop_action_coordinator().lock().await;+ let _admission = DesktopCall::acquire().await.unwrap();let now = active.fetch_add(1, Ordering::SeqCst) + 1;max_active.fetch_max(now, Ordering::SeqCst);tokio::task::yield_now().await;@@ -6141,6 +6288,85 @@ resources:assert_eq!(max_active.load(Ordering::SeqCst), 1);}+ #[tokio::test(flavor = "multi_thread", worker_threads = 2)]+ async fn a_call_behind_a_worker_that_never_returns_is_refused_within_its_budget() {+ let lane = DesktopLane::new(Duration::from_millis(300));+ let call = lane.acquire().await.unwrap();+ // The call timed out; its platform worker never returns, so its lease+ // clone is never dropped.+ call.scope(async { std::mem::forget(DesktopLease::current().unwrap()) })+ .await;+ drop(call);+ let next = tokio::time::timeout(Duration::from_secs(10), lane.acquire())+ .await+ .expect("the next call waited forever behind a stale worker");+ let Err(refusal) = next else {+ panic!("the lane was handed out while the stale worker still holds it");+ };+ assert_eq!(+ refusal.structured_content.as_ref().unwrap()["refusal"]["code"],+ "input_busy"+ );+ }++ #[tokio::test(start_paused = true)]+ async fn calls_after_the_stale_bound_are_refused_without_waiting_again() {+ let lane = DesktopLane::new(Duration::from_secs(12));+ let call = lane.acquire().await.unwrap();+ let _worker = call.scope(async { DesktopLease::current().unwrap() }).await;+ drop(call);+ assert!(lane.acquire().await.is_err());+ let started = tokio::time::Instant::now();+ assert!(lane.acquire().await.is_err());+ assert_eq!(started.elapsed(), Duration::ZERO);+ }++ #[tokio::test(start_paused = true)]+ async fn a_call_behind_the_healthy_successor_of_a_stale_worker_waits_its_turn() {+ let lane = Arc::new(DesktopLane::new(Duration::from_secs(12)));+ let call = lane.acquire().await.unwrap();+ let worker = call.scope(async { DesktopLease::current().unwrap() }).await;+ drop(call);+ // Queued first: runs for 400 ms once the stale worker stops.+ let first = tokio::spawn({+ let lane = lane.clone();+ async move {+ let call = lane.acquire().await.ok()?;+ tokio::time::sleep(Duration::from_millis(400)).await;+ drop(call);+ Some(())+ }+ });+ tokio::task::yield_now().await;+ let second = tokio::spawn({+ let lane = lane.clone();+ async move { lane.acquire().await.map(drop) }+ });+ // The stale worker stops just before the bound.+ tokio::time::sleep(Duration::from_millis(11_983)).await;+ drop(worker);+ assert!(first.await.unwrap().is_some());+ let second = second.await.unwrap();+ assert!(+ second.is_ok(),+ "refused behind a call still running: {:?}",+ second.err().map(|refusal| refusal.structured_content)+ );+ }++ #[tokio::test(flavor = "multi_thread", worker_threads = 2)]+ async fn a_call_still_running_is_waited_for_past_the_stale_bound() {+ let lane = Arc::new(DesktopLane::new(Duration::from_millis(100)));+ let call = lane.acquire().await.unwrap();+ let next = tokio::spawn({+ let lane = lane.clone();+ async move { lane.acquire().await.is_ok() }+ });+ tokio::time::sleep(Duration::from_millis(500)).await;+ drop(call);+ assert!(next.await.unwrap());+ }+#[test]fn independent_input_requires_adapter_attestation_and_an_exact_background_window() {use super::requires_desktop_coordination;diff --git a/libs/cua-driver/rust/crates/cua-driver-e2e/tests/agent_cursor_showcase_test.rs b/libs/cua-driver/rust/crates/cua-driver-e2e/tests/agent_cursor_showcase_test.rsindex c4b5e97921..a9e4dec0cd 100644--- a/libs/cua-driver/rust/crates/cua-driver-e2e/tests/agent_cursor_showcase_test.rs+++ b/libs/cua-driver/rust/crates/cua-driver-e2e/tests/agent_cursor_showcase_test.rs@@ -151,8 +151,7 @@ fn semantic_cursor_showcase_records_session_and_action_states() {);settle(900);- call_ok(- &mut driver,+ let typed = driver.call("type_text",serde_json::json!({"session": SESSION,@@ -161,6 +160,21 @@ fn semantic_cursor_showcase_records_session_and_action_states() {"delivery_mode": "foreground"}),);+ if niri_session() {+ // The click landed on the empty desktop, so no window has+ // keyboard focus. niri reports that and types nothing, where+ // other compositors accept text that reaches no one.+ assert!(+ typed.is_error()+ && typed+ .text()+ .starts_with("foreground_unavailable: no niri window has keyboard focus"),+ "type_text on the empty niri desktop did not refuse exactly: {}",+ typed.text()+ );+ } else {+ assert!(!typed.is_error(), "type_text failed: {}", typed.text());+ }settle(900);call_ok(@@ -887,6 +901,18 @@ fn call_ok(driver: &mut McpDriver, tool: &str, arguments: serde_json::Value) {assert!(!response.is_error(), "{tool} failed: {}", response.text());}+fn niri_session() -> bool {+ #[cfg(target_os = "linux")]+ {+ platform_linux::wayland::wayland_input_enabled()+ && platform_linux::wayland::niri::is_session()+ }+ #[cfg(not(target_os = "linux"))]+ {+ false+ }+}+fn settle(milliseconds: u64) {std::thread::sleep(Duration::from_millis(milliseconds));}diff --git a/libs/cua-driver/rust/crates/cua-driver-e2e/tests/cross_platform_behavior_test.rs b/libs/cua-driver/rust/crates/cua-driver-e2e/tests/cross_platform_behavior_test.rsindex 0d0ed3a324..2c295a5131 100644--- a/libs/cua-driver/rust/crates/cua-driver-e2e/tests/cross_platform_behavior_test.rs+++ b/libs/cua-driver/rust/crates/cua-driver-e2e/tests/cross_platform_behavior_test.rs@@ -284,6 +284,35 @@ struct Fixture {window_y: f64,name: &'static str,journal: FixtureJournal,+ /// The last lines a Tauri host and its WebKit processes wrote to stderr.+ stderr_tail: Option<StderrTail>,+}++type StderrTail = std::sync::Arc<std::sync::Mutex<std::collections::VecDeque<String>>>;++/// Copy a host's stderr to the test's own and keep its last lines.+fn tee_stderr(stderr: std::process::ChildStderr) -> StderrTail {+ use std::io::{BufRead as _, Write as _};+ let tail = StderrTail::default();+ let kept = tail.clone();+ thread::spawn(move || {+ for line in std::io::BufReader::new(stderr)+ .lines()+ .map_while(Result::ok)+ {+ // Write to fd 2 itself, as the inherited stream did, not+ // through the test harness's output capture.+ let _ = writeln!(std::io::stderr(), "{line}");+ let mut kept = kept+ .lock()+ .unwrap_or_else(std::sync::PoisonError::into_inner);+ if kept.len() == 40 {+ kept.pop_front();+ }+ kept.push_back(line);+ }+ });+ tail}impl Drop for Fixture {@@ -340,7 +369,11 @@ fn launch_host_with_evidence(spec: &HostSpec, scenario: &str, evidence: &mut Evi.args(&spec.args).env("CUA_E2E_FIXTURE_JOURNAL_URL", journal.url()).stdout(Stdio::inherit())- .stderr(Stdio::inherit());+ .stderr(if spec.name == "tauri" {+ Stdio::piped()+ } else {+ Stdio::inherit()+ });match spec.name {"electron" => {command.env(@@ -367,8 +400,9 @@ fn launch_host_with_evidence(spec: &HostSpec, scenario: &str, evidence: &mut Evi.collect::<std::collections::HashSet<_>>()}).unwrap_or_default();- let child = spawn_in_job(&mut command)+ let mut child = spawn_in_job(&mut command).unwrap_or_else(|error| panic!("failed to launch {} fixture: {error}", spec.name));+ let stderr_tail = child.stderr.take().map(tee_stderr);let pid = child.id();driver.reaper().push(child);@@ -395,6 +429,7 @@ fn launch_host_with_evidence(spec: &HostSpec, scenario: &str, evidence: &mut Eviwindow_y: bounds["y"].as_f64().unwrap_or(0.0),name: spec.name,journal,+ stderr_tail,};// Electron and Tauri can need more than the generic fixture// budget to start their renderer and expose the remote@@ -403,12 +438,13 @@ fn launch_host_with_evidence(spec: &HostSpec, scenario: &str, evidence: &mut Evilet ax_deadline = Instant::now() + ax_timeout;let mut last_tree = String::new();while Instant::now() < ax_deadline {- let state = snapshot(&mut fixture);+ let state = window_state(&mut fixture);last_tree.clear();last_tree.push_str(state.tree_text());if last_tree.contains("WEB_HARNESS_MARKER_v1")&& fixture.journal.contains("WEB_HARNESS_MARKER_v1"){+ assert_laid_out(&fixture);eprintln!("READY {scenario}: daemon usable, fixture pid={actual_pid} window_id={wid}, AX and journal marker present");return fixture;}@@ -426,19 +462,70 @@ fn launch_host_with_evidence(spec: &HostSpec, scenario: &str, evidence: &mut Evipanic!("{} fixture window did not appear", spec.name);}-fn snapshot(fixture: &mut Fixture) -> ToolResponse {+fn window_state(fixture: &mut Fixture) -> ToolResponse {let state = fixture.driver.call("get_window_state",serde_json::json!({"pid": fixture.pid as i64,"window_id": fixture.wid,- "capture_mode": "ax"+ "capture_mode": "ax",+ "timeout_ms": 10_000}),);refresh_wayland_window_geometry(fixture);state}+/// A ready fixture's accessibility tree. A walk cut short at its deadline+/// lacks elements a row then reads as missing, so it gets room on a slow+/// runner and fails as truncated if that is not enough.+fn snapshot(fixture: &mut Fixture) -> ToolResponse {+ let state = window_state(fixture);+ let structured = state.structured();+ assert!(+ structured["truncated"].as_bool() != Some(true),+ "the accessibility walk of window {} was truncated ({}), so its tree is incomplete",+ fixture.wid,+ structured["truncation_reason"]+ .as_str()+ .unwrap_or("no reason given")+ );+ state+}++/// A web fixture page laid out collapsed (seen with WebKitGTK on a runner+/// without a DRM render node) stacks its controls, so pointer rows would+/// fail as misdelivered. Report the host's renderer instead, with what the+/// page and the host process reported about it.+fn assert_laid_out(fixture: &Fixture) {+ let scroll = &fixture.journal.snapshot()["scroll-tall"];+ if scroll["clientHeight"].as_u64() != Some(0) {+ return;+ }+ let renderer_env: Vec<String> = std::env::vars()+ .filter(|(name, _)| name.starts_with("WEBKIT_") || name.starts_with("GDK_"))+ .map(|(name, value)| format!("{name}={value}"))+ .collect();+ let stderr_tail = fixture.stderr_tail.as_ref().map_or_else(+ || "not captured".to_owned(),+ |tail| {+ let tail = tail+ .lock()+ .unwrap_or_else(std::sync::PoisonError::into_inner);+ tail.iter().cloned().collect::<Vec<_>>().join("\n")+ },+ );+ panic!(+ "environment: the {} fixture page laid out collapsed (#scroll-tall clientHeight 0), \+ so its controls are not where its accessibility tree places them; this host's web \+ renderer is broken, and no row on this fixture can be judged\n\+ page layout: {}\nhost env: {}\nhost stderr tail:\n{stderr_tail}",+ fixture.name,+ scroll["layout"],+ renderer_env.join(" "),+ );+}+#[cfg(target_os = "linux")]fn refresh_wayland_window_geometry(fixture: &mut Fixture) {if std::env::var_os("WAYLAND_DISPLAY").is_none() {@@ -616,6 +703,11 @@ fn require_element(snapshot: &ToolResponse, id: &str) -> u64 {}fn element_center(snapshot: &ToolResponse, element_index: u64) -> (f64, f64) {+ element_frame_center(snapshot, element_index)+ .unwrap_or_else(|| panic!("element [{element_index}] has no frame"))+}++fn element_frame_center(snapshot: &ToolResponse, element_index: u64) -> Option<(f64, f64)> {let frame = snapshot.structured()["elements"].as_array().and_then(|elements| {@@ -623,12 +715,22 @@ fn element_center(snapshot: &ToolResponse, element_index: u64) -> (f64, f64) {.iter().find(|element| element["element_index"].as_u64() == Some(element_index))})- .and_then(|element| element["frame"].as_object())- .unwrap_or_else(|| panic!("element [{element_index}] has no frame"));- (+ .and_then(|element| element["frame"].as_object())?;+ Some((frame["x"].as_f64().unwrap_or(0.0) + frame["w"].as_f64().unwrap_or(0.0) / 2.0,frame["y"].as_f64().unwrap_or(0.0) + frame["h"].as_f64().unwrap_or(0.0) / 2.0,- )+ ))+}++/// Native Wayland background pixel rows are typed refusals. A compositor may+/// withhold the per-window crop size or the element frames for a window it+/// hides (niri hides a floating window behind a fullscreen one and reports no+/// geometry for it), so these rows aim at inert coordinates and let the+/// driver publish its refusal without an unverifiable screenshot oracle.+fn wayland_background_px_row(delivery: &str) -> bool {+ delivery == "background"+ && cua_driver_testkit::e2e::DisplayServer::current()+ == cua_driver_testkit::e2e::DisplayServer::Wayland}fn assert_fixture_contains(fixture: &Fixture, marker: &str) {@@ -789,27 +891,21 @@ fn action_target_args(} else {let origin = window_origin(fixture, state);let scale = screenshot_scale(state);- let screen_point = element_center(state, index);- let (local_x, local_y) = screenshot_local_point(screen_point, origin, scale);let width = state.structured()["screenshot_width"].as_f64();let height = state.structured()["screenshot_height"].as_f64();- if width.is_none() || height.is_none() {- // Native Wayland may intentionally omit a per-window crop size.- // Background pixel rows are typed refusals in that environment;- // use inert coordinates so the driver can publish that refusal- // without requiring an unverifiable screenshot oracle.- if delivery == "background"- && cua_driver_testkit::e2e::DisplayServer::current()- == cua_driver_testkit::e2e::DisplayServer::Wayland- {+ let frame_center = element_frame_center(state, index);+ let (Some(width), Some(height), Some(screen_point)) = (width, height, frame_center) else {+ if wayland_background_px_row(delivery) {object.insert("x".to_owned(), serde_json::json!(0.0));object.insert("y".to_owned(), serde_json::json!(0.0));return args;}+ if frame_center.is_none() {+ panic!("element [{index}] has no frame");+ }panic!("PX action requires screenshot dimensions for {delivery} delivery");- }- let width = width.unwrap();- let height = height.unwrap();+ };+ let (local_x, local_y) = screenshot_local_point(screen_point, origin, scale);eprintln!("[shared-px] {} target={id} screen=({:.1},{:.1}) origin=({:.1},{:.1}) scale=({:.3},{:.3}) local=({local_x:.1},{local_y:.1}) capture=({width:.1}x{height:.1})",fixture.name, screen_point.0, screen_point.1, origin.0, origin.1, scale.0, scale.1@@ -1436,9 +1532,15 @@ fn run_drag_action(fixture: &mut Fixture, delivery: &str) -> Observation {let target = require_element(&pre, "drop-target");let origin = window_origin(fixture, &pre);let scale = screenshot_scale(&pre);- let point = |index: u64| screenshot_local_point(element_center(&pre, index), origin, scale);- let (from_x, from_y) = point(source);- let (to_x, to_y) = point(target);+ let frames = element_frame_center(&pre, source).zip(element_frame_center(&pre, target));+ let ((from_x, from_y), (to_x, to_y)) = match frames {+ Some((from, to)) => (+ screenshot_local_point(from, origin, scale),+ screenshot_local_point(to, origin, scale),+ ),+ None if wayland_background_px_row(delivery) => ((0.0, 0.0), (0.0, 0.0)),+ None => (element_center(&pre, source), element_center(&pre, target)),+ };let response = fixture.driver.call("drag",serde_json::json!({@@ -2067,7 +2169,7 @@ fn run_browser_tool_roundtrip(fixture: &mut Fixture) -> Observation {delivered_observation()}-fn run_browser_tool_refusal(fixture: &mut Fixture) -> Observation {+fn run_browser_tool_refusal(fixture: &mut Fixture, expected: RefusalCode) -> Observation {let session = format!("browser-embedded-refusal-{}", fixture.pid);let started = fixture.driver@@ -2096,8 +2198,7 @@ fn run_browser_tool_refusal(fixture: &mut Fixture) -> Observation {)});assert_eq!(- code,- RefusalCode::BrowserRouteUnavailable,+ code, expected,"embedded browser returned the wrong refusal: {}",response.raw);@@ -2153,10 +2254,34 @@ fn embedded_browser_case(spec: &HostSpec) -> CaseSpec {cua_driver_testkit::e2e::DriverRoute::Cdp,oracles,);- if spec.name == "electron" {- case+ match embedded_browser_refusal(spec) {+ None => case,+ Some(code) => case.expecting_refusal(vec![code]),+ }+}++/// The refusal an embedded browser row declares, or `None` when it must+/// bind. Browser identity is not accepted on niri: its Linux browser route+/// has no exact window geometry there, so Electron binding fails closed.+fn embedded_browser_refusal(spec: &HostSpec) -> Option<RefusalCode> {+ if spec.name != "electron" {+ Some(RefusalCode::BrowserRouteUnavailable)+ } else if niri_session() {+ Some(RefusalCode::BrowserWrongTargetRefused)} else {- case.expecting_refusal(vec![RefusalCode::BrowserRouteUnavailable])+ None+ }+}++fn niri_session() -> bool {+ #[cfg(target_os = "linux")]+ {+ platform_linux::wayland::wayland_input_enabled()+ && platform_linux::wayland::niri::is_session()+ }+ #[cfg(not(target_os = "linux"))]+ {+ false}}@@ -2171,10 +2296,11 @@ fn embedded_browser_routes_are_exact_or_refused() {continue;}selected += 1;- let result = if spec.name == "electron" {- run_host_case_with_outcome(case, &spec, run_browser_tool_roundtrip)- } else {- run_host_case_with_outcome(case, &spec, run_browser_tool_refusal)+ let result = match embedded_browser_refusal(&spec) {+ None => run_host_case_with_outcome(case, &spec, run_browser_tool_roundtrip),+ Some(code) => run_host_case_with_outcome(case, &spec, |fixture| {+ run_browser_tool_refusal(fixture, code)+ }),};if failure.is_none() {failure = result;diff --git a/libs/cua-driver/rust/crates/cua-driver-e2e/tests/desktop_scope_linux_test.rs b/libs/cua-driver/rust/crates/cua-driver-e2e/tests/desktop_scope_linux_test.rsindex 7ad7b3c25d..52fc9a2d14 100644--- a/libs/cua-driver/rust/crates/cua-driver-e2e/tests/desktop_scope_linux_test.rs+++ b/libs/cua-driver/rust/crates/cua-driver-e2e/tests/desktop_scope_linux_test.rs@@ -171,6 +171,21 @@ fn desktop_input_route() -> DriverRoute {}}+/// niri does not report where tiled windows are, so it takes desktop-scope+/// points only as pixels of the latest `get_desktop_state` screenshot.+fn desktop_screenshot_on_niri(driver: &mut McpDriver, session: &str) {+ if !(platform_linux::wayland::wayland_input_enabled()+ && platform_linux::wayland::niri::is_session())+ {+ return;+ }+ let r = driver.call(+ "get_desktop_state",+ serde_json::json!({ "session": session }),+ );+ assert!(!r.is_error(), "get_desktop_state failed: {}", r.text());+}+fn start_scope(driver: &mut McpDriver, session: &str, scope: &str) {let r = driver.call("start_session",@@ -235,6 +250,7 @@ fn desktop_scope_windowless_click_lands_on_control() {posture.text());std::thread::sleep(Duration::from_millis(300));+ desktop_screenshot_on_niri(&mut driver, &desktop_session);driver.start_behavior_recording();// Retry the window-less desktop click until the counter advances. A@@ -331,6 +347,45 @@ fn desktop_scope_windowless_scroll_lands_on_control() {std::thread::sleep(Duration::from_millis(300));snap = ax_snapshot(&mut driver, &window_session, pid, wid);};+ // The fixture's outer scroller keeps every control reachable on a short+ // window. At niri's 1.5 scale smoke the window is 520 logical px tall+ // and the target lies below it, so scroll the outer scroller first.+ let mut target = (x, y);+ desktop_screenshot_on_niri(&mut driver, &desktop_session);+ for _ in 0..6 {+ let Some(bounds) = snap.get("window_bounds").filter(|b| b.is_object()) else {+ break;+ };+ let (bx, by, bw, bh) = (+ bounds["x"].as_i64().unwrap_or(0),+ bounds["y"].as_i64().unwrap_or(0),+ bounds["width"].as_i64().unwrap_or(0),+ bounds["height"].as_i64().unwrap_or(0),+ );+ if target.1 < by + bh - 16 {+ break;+ }+ // The root box's 12 px border at the right edge holds no control+ // that takes the wheel, so the outer scroller gets it.+ let outer = driver.call(+ "scroll",+ serde_json::json!({+ "session": desktop_session, "scope": "desktop",+ "x": bx + bw - 6, "y": by + bh / 2,+ "direction": "down", "by": "line", "amount": 5+ }),+ );+ assert!(+ !outer.is_error(),+ "could not scroll the GTK3 scroll target into view: {}",+ outer.text()+ );+ std::thread::sleep(Duration::from_millis(300));+ snap = ax_snapshot(&mut driver, &window_session, pid, wid);+ target = element_center_containing(&snap, "scroll-tall-viewport")+ .expect("GTK3 scroll target left the AT-SPI tree");+ }+ let (x, y) = target;let before = marker_value(&snap, "scroll_offset=").unwrap_or(0);driver.start_behavior_recording();let response = driver.call(diff --git a/libs/cua-driver/rust/crates/cua-driver-testkit/src/e2e.rs b/libs/cua-driver/rust/crates/cua-driver-testkit/src/e2e.rsindex 388fc60b19..2a7d93e651 100644--- a/libs/cua-driver/rust/crates/cua-driver-testkit/src/e2e.rs+++ b/libs/cua-driver/rust/crates/cua-driver-testkit/src/e2e.rs@@ -449,6 +449,8 @@ fn compositor_from_env() -> Option<String> {.to_ascii_lowercase();if desktop.contains("sway") {Some("sway".to_owned())+ } else if desktop.contains("niri") {+ Some("niri".to_owned())} else if desktop.contains("gnome") {Some("gnome-mutter".to_owned())} else if desktop.contains("kde") || desktop.contains("plasma") {diff --git a/libs/cua-driver/rust/crates/cua-driver-testkit/src/observer.rs b/libs/cua-driver/rust/crates/cua-driver-testkit/src/observer.rsindex a80c7dc19d..39a7950e07 100644--- a/libs/cua-driver/rust/crates/cua-driver-testkit/src/observer.rs+++ b/libs/cua-driver/rust/crates/cua-driver-testkit/src/observer.rs@@ -919,6 +919,11 @@ pub mod macos {#[path = "observer_hyprland.rs"]mod hyprland;+// niri's transport uses Linux-only peer credentials.+#[cfg(target_os = "linux")]+#[path = "observer_niri.rs"]+mod niri;+#[cfg(target_os = "linux")]pub mod linux {use std::process::{Command, Stdio};@@ -944,6 +949,7 @@ pub mod linux {X11,Sway,Hyprland,+ Niri,Gnome,CuaCompositor,Missing,@@ -955,6 +961,7 @@ pub mod linux {events: Arc<Mutex<Vec<FocusEvent>>>,sampler: Option<JoinHandle<()>>,hyprland_journal: Option<super::hyprland::Journal>,+ niri_journal: Option<super::niri::Journal>,probe_error: Option<ObserverError>,}@@ -970,6 +977,14 @@ pub mod linux {SessionKind::CuaCompositor} else if sway_available() {SessionKind::Sway+ } else if let Some(niri) = niri_session() {+ match niri {+ Ok(()) => SessionKind::Niri,+ Err(error) => {+ probe_error = Some(error);+ SessionKind::Missing+ }+ }} else if std::env::var_os("HYPRLAND_INSTANCE_SIGNATURE").is_some() {match super::hyprland::probe() {Ok(()) => SessionKind::Hyprland,@@ -998,6 +1013,7 @@ pub mod linux {events: Arc::new(Mutex::new(Vec::new())),sampler: None,hyprland_journal: None,+ niri_journal: None,probe_error,}}@@ -1018,14 +1034,15 @@ pub mod linux {cursor: true,leaked_input: false,},- SessionKind::Sway | SessionKind::Gnome | SessionKind::CuaCompositor => {- ObserverCapabilities {- focus: true,- z_order: true,- cursor: false,- leaked_input: false,- }- }+ SessionKind::Sway+ | SessionKind::Niri+ | SessionKind::Gnome+ | SessionKind::CuaCompositor => ObserverCapabilities {+ focus: true,+ z_order: true,+ cursor: false,+ leaked_input: false,+ },SessionKind::Missing => ObserverCapabilities::default(),}}@@ -1038,6 +1055,7 @@ pub mod linux {SessionKind::X11 => x11_snapshot(target),SessionKind::Sway => sway_snapshot(target),SessionKind::Hyprland => super::hyprland::snapshot(target),+ SessionKind::Niri => super::niri::snapshot(target),SessionKind::Gnome => gnome_snapshot(target),SessionKind::CuaCompositor => cua_compositor_snapshot(target),SessionKind::Missing => Ok(DesktopSnapshot {@@ -1057,6 +1075,13 @@ pub mod linux {self.hyprland_journal = Some(super::hyprland::Journal::start()?);return Ok(());}+ if self.session == SessionKind::Niri {+ if self.niri_journal.is_some() {+ return Err(ObserverError::new("niri focus journal already active"));+ }+ self.niri_journal = Some(super::niri::Journal::start()?);+ return Ok(());+ }if self.sampler.is_some() {return Err(ObserverError::new("Linux focus journal already active"));}@@ -1073,6 +1098,7 @@ pub mod linux {SessionKind::X11 => x11_focus_identity(),SessionKind::Sway => sway_focus_identity(),SessionKind::Hyprland => super::hyprland::focus_identity(),+ SessionKind::Niri => super::niri::focus_identity(),SessionKind::Gnome => gnome_focus_identity(),SessionKind::CuaCompositor => cua_compositor_focus_identity(),SessionKind::Missing => Ok(None),@@ -1098,6 +1124,9 @@ pub mod linux {if let Some(journal) = self.hyprland_journal.take() {return journal.drain();}+ if let Some(journal) = self.niri_journal.take() {+ return journal.drain();+ }self.stop.store(true, Ordering::Release);if let Some(sampler) = self.sampler.take() {sampler@@ -1123,6 +1152,34 @@ pub mod linux {super::hyprland::cursor_position()}+ /// `None` unless `NIRI_SOCKET` belongs to this session's compositor, so an+ /// unrelated inherited socket leaves the session to the other observers.+ /// Once niri was recognised, losing it is an error, not another session.+ fn niri_session() -> Option<Result<(), ObserverError>> {+ match super::niri::attest() {+ Ok(false) => None,+ Ok(true) => Some(super::niri::probe()),+ Err(error) => Some(Err(error)),+ }+ }++ /// Lets the niri observer certify occlusion behind this sentinel window+ /// while its state file reports fullscreen; `None` clears it.+ pub(crate) fn niri_register_sentinel(sentinel: Option<(TargetWindow, std::path::PathBuf)>) {+ super::niri::register_sentinel(sentinel);+ }++ pub(crate) fn niri_focus(target: TargetWindow) -> Result<(), ObserverError> {+ super::niri::focus(target)+ }++ pub(crate) fn niri_wait_for_focus_transfer(+ from: TargetWindow,+ to: TargetWindow,+ ) -> Result<(), ObserverError> {+ super::niri::wait_for_focus_transfer(from, to, Duration::from_secs(3))+ }+pub(crate) fn hyprland_monitor_regions() -> Result<Vec<(f64, f64, f64, f64)>, ObserverError> {super::hyprland::monitor_regions()}diff --git a/libs/cua-driver/rust/crates/cua-driver-testkit/src/observer_niri.rs b/libs/cua-driver/rust/crates/cua-driver-testkit/src/observer_niri.rsnew file mode 100644index 0000000000..627a29fe5e--- /dev/null+++ b/libs/cua-driver/rust/crates/cua-driver-testkit/src/observer_niri.rs@@ -0,0 +1,1282 @@+//! Independent niri observations over niri's IPC socket. This module never+//! calls the driver's niri adapter, capture, geometry, or input code.+//!+//! niri's IPC reports no stacking order and no position for tiled windows, so+//! occlusion is classified only where niri's own rendering rule decides it:+//! floating windows are not drawn while a tiled window in a fullscreen column+//! holds focus on the same workspace (`Workspace::is_floating_visible`). That+//! rule needs a fullscreen column, and IPC cannot tell one from a maximized+//! column of the same size: a client's `xdg_toplevel.set_maximized` or the+//! maximize-to-edges action fills the output while floating windows stay+//! drawn. So the rule certifies occlusion only behind the foreground sentinel,+//! by exact id, while the sentinel itself reports fullscreen from niri's last+//! configure (`ForegroundSentinel` registers it). The rule also needs a static+//! view and focus that does not follow the mouse, which the canonical niri+//! lane's config provides (animations and focus-follows-mouse off). Every+//! other overlap is reported as ambiguous instead of guessed.++use std::collections::HashMap;+use std::io::{BufRead, BufReader, Read, Write};+use std::os::fd::{AsRawFd, FromRawFd};+use std::os::unix::net::UnixStream;+use std::path::{Path, PathBuf};+use std::sync::atomic::{AtomicBool, Ordering};+use std::sync::{Arc, Mutex, OnceLock};+use std::thread::JoinHandle;+use std::time::{Duration, Instant, SystemTime};++use serde::de::DeserializeOwned;+use serde::Deserialize;++use super::{DesktopJournal, DesktopSnapshot, FocusEvent, ObserverError, TargetWindow, TargetZ};++const QUERY_TIMEOUT: Duration = Duration::from_secs(1);+const REPLY_LIMIT: u64 = 1024 * 1024;+/// Logical pixels of slack when comparing a fullscreen tile with its output.+const FULLSCREEN_TOLERANCE: f64 = 1.0;+/// How long a snapshot waits for the sentinel's next fullscreen report. The+/// sentinel writes one on each tick of its 100 ms timer, the same timer as+/// its heartbeat, and a snapshot needs just one tick after the window list+/// read. That timer can stall for hundreds of milliseconds on a loaded+/// runner, so the wait is the harness's liveness bound for it; a punctual+/// tick still returns within about 100 ms.+const SENTINEL_REPORT_TIMEOUT: Duration = crate::sentinel::SENTINEL_TICK_LIVENESS;++/// Connects without blocking past `timeout`: a wedged compositor stops+/// accepting, and a full backlog would block a plain connect forever.+fn connect(path: &Path, timeout: Duration) -> std::io::Result<UnixStream> {+ use std::os::unix::ffi::OsStrExt as _;++ let bytes = path.as_os_str().as_bytes();+ // SAFETY: an all-zero sockaddr_un is a valid empty address.+ let mut address: libc::sockaddr_un = unsafe { std::mem::zeroed() };+ if bytes.len() >= address.sun_path.len() {+ return Err(std::io::Error::new(+ std::io::ErrorKind::InvalidInput,+ "socket path is too long",+ ));+ }+ address.sun_family = libc::AF_UNIX as libc::sa_family_t;+ for (slot, byte) in address.sun_path.iter_mut().zip(bytes) {+ *slot = *byte as libc::c_char;+ }+ let deadline = Instant::now() + timeout;+ loop {+ // SAFETY: plain socket creation; ownership moves into UnixStream below.+ let fd = unsafe {+ libc::socket(+ libc::AF_UNIX,+ libc::SOCK_STREAM | libc::SOCK_NONBLOCK | libc::SOCK_CLOEXEC,+ 0,+ )+ };+ if fd < 0 {+ return Err(std::io::Error::last_os_error());+ }+ // SAFETY: fd is a fresh socket owned only here.+ let stream = unsafe { UnixStream::from_raw_fd(fd) };+ // SAFETY: address is a valid sockaddr_un of the given length.+ let connected = unsafe {+ libc::connect(+ fd,+ (&address as *const libc::sockaddr_un).cast(),+ std::mem::size_of::<libc::sockaddr_un>() as libc::socklen_t,+ )+ };+ if connected == 0 {+ stream.set_nonblocking(false)?;+ return Ok(stream);+ }+ let error = std::io::Error::last_os_error();+ // A Unix stream connect either completes or reports a full backlog.+ if error.raw_os_error() != Some(libc::EAGAIN) || Instant::now() >= deadline {+ return Err(error);+ }+ std::thread::sleep(Duration::from_millis(10));+ }+}++fn peer_pid(stream: &UnixStream) -> std::io::Result<i32> {+ // SAFETY: an all-zero ucred is valid output storage.+ let mut credentials: libc::ucred = unsafe { std::mem::zeroed() };+ let mut length = std::mem::size_of::<libc::ucred>() as libc::socklen_t;+ // SAFETY: the pointer and length describe `credentials`.+ let result = unsafe {+ libc::getsockopt(+ stream.as_raw_fd(),+ libc::SOL_SOCKET,+ libc::SO_PEERCRED,+ (&mut credentials as *mut libc::ucred).cast(),+ &mut length,+ )+ };+ if result != 0 {+ return Err(std::io::Error::last_os_error());+ }+ Ok(credentials.pid)+}++fn socket_peer(path: &Path) -> std::io::Result<i32> {+ peer_pid(&connect(path, QUERY_TIMEOUT)?)+}++fn niri_socket() -> Option<PathBuf> {+ std::env::var_os("NIRI_SOCKET")+ .filter(|socket| !socket.is_empty())+ .map(PathBuf::from)+}++fn wayland_socket() -> Option<PathBuf> {+ let display = PathBuf::from(std::env::var_os("WAYLAND_DISPLAY")?);+ if display.is_absolute() {+ return Some(display);+ }+ Some(PathBuf::from(std::env::var_os("XDG_RUNTIME_DIR")?).join(display))+}++/// The niri pid this process recognised first. A later niri, for example a+/// restarted compositor, is a different session and is refused.+static ATTESTED: OnceLock<i32> = OnceLock::new();++/// The pids serving `NIRI_SOCKET` and `WAYLAND_DISPLAY`.+fn peers() -> Result<(i32, i32), String> {+ let niri = niri_socket().ok_or("NIRI_SOCKET is unset")?;+ let wayland = wayland_socket().ok_or("WAYLAND_DISPLAY is unset")?;+ let peer = |path: &Path| {+ socket_peer(path).map_err(|error| format!("{} is unreachable: {error}", path.display()))+ };+ Ok((peer(&niri)?, peer(&wayland)?))+}++/// Before niri is recognised, `Ok(None)` leaves the session to the other+/// observers: `NIRI_SOCKET` is missing, stale or served by an unrelated+/// compositor, as when a nested session inherits it from a niri desktop.+/// `Ok(Some(pid))` recognises the niri serving both sockets. Once niri was+/// recognised, losing either socket or a different peer is an error.+fn attestation(+ recognised: Option<i32>,+ peers: Result<(i32, i32), String>,+) -> Result<Option<i32>, ObserverError> {+ match (recognised, peers) {+ (None, Ok((niri, wayland))) if niri == wayland => Ok(Some(niri)),+ (None, _) => Ok(None),+ (Some(pid), Ok((niri, wayland))) if niri == pid && wayland == pid => Ok(Some(pid)),+ (Some(pid), Ok((niri, wayland))) => Err(ObserverError::new(format!(+ "niri {pid} was recognised, but NIRI_SOCKET is now served by {niri} and \+ WAYLAND_DISPLAY by {wayland}"+ ))),+ (Some(pid), Err(error)) => Err(ObserverError::new(format!(+ "niri {pid} was recognised, but {error}"+ ))),+ }+}++/// Whether `NIRI_SOCKET` belongs to the compositor serving `WAYLAND_DISPLAY`;+/// see [`attestation`].+pub(super) fn attest() -> Result<bool, ObserverError> {+ let Some(pid) = attestation(ATTESTED.get().copied(), peers())? else {+ return Ok(false);+ };+ let attested = *ATTESTED.get_or_init(|| pid);+ if attested != pid {+ return Err(ObserverError::new(format!(+ "niri pid changed from {attested} to {pid}; refusing a new compositor"+ )));+ }+ Ok(true)+}++/// Attests both sockets again. Each snapshot, journal and focus operation+/// does this once at its start; its connections then check only niri's peer,+/// so polling does not open a Wayland client per request.+fn ensure_attested() -> Result<(), ObserverError> {+ if attest()? {+ Ok(())+ } else {+ Err(ObserverError::new(+ "NIRI_SOCKET is not served by the WAYLAND_DISPLAY compositor",+ ))+ }+}++/// A connection to the attested niri, with read and write deadlines. A peer+/// other than the attested niri is refused.+fn attested_connection(purpose: &str) -> Result<UnixStream, ObserverError> {+ let socket =+ niri_socket().ok_or_else(|| ObserverError::new("niri observer requires NIRI_SOCKET"))?;+ let error = |operation: &str, error: std::io::Error| {+ ObserverError::new(format!("niri IPC {operation} {purpose}: {error}"))+ };+ let stream = connect(&socket, QUERY_TIMEOUT).map_err(|e| error("connect", e))?;+ let peer = peer_pid(&stream).map_err(|e| error("read peer of", e))?;+ if Some(&peer) != ATTESTED.get() {+ return Err(ObserverError::new(format!(+ "niri IPC peer {peer} is not the attested niri {:?}",+ ATTESTED.get()+ )));+ }+ stream+ .set_read_timeout(Some(QUERY_TIMEOUT))+ .and_then(|()| stream.set_write_timeout(Some(QUERY_TIMEOUT)))+ .map_err(|e| error("configure", e))?;+ Ok(stream)+}++/// Sends one request on a fresh connection and returns the `Ok` payload.+fn request(request: &serde_json::Value) -> Result<serde_json::Value, ObserverError> {+ let error = |operation: &str, error: std::io::Error| {+ ObserverError::new(format!("niri IPC {operation} {request}: {error}"))+ };+ let mut stream = attested_connection(&request.to_string())?;+ writeln!(stream, "{request}").map_err(|e| error("write", e))?;+ let mut line = String::new();+ BufReader::new(stream.take(REPLY_LIMIT))+ .read_line(&mut line)+ .map_err(|e| error("read", e))?;+ parse_reply(&line)+}++fn parse_reply(line: &str) -> Result<serde_json::Value, ObserverError> {+ let reply: serde_json::Value = serde_json::from_str(line)+ .map_err(|error| ObserverError::new(format!("invalid niri IPC reply: {error}")))?;+ match reply {+ serde_json::Value::Object(mut reply) if reply.len() == 1 => {+ if let Some(payload) = reply.remove("Ok") {+ Ok(payload)+ } else if let Some(message) = reply.remove("Err") {+ Err(ObserverError::new(format!("niri IPC refused: {message}")))+ } else {+ Err(ObserverError::new("niri IPC reply is neither Ok nor Err"))+ }+ }+ _ => Err(ObserverError::new("niri IPC reply is neither Ok nor Err")),+ }+}++/// Reads one response kind, e.g. `{"Ok":{"Windows":[...]}}` for `"Windows"`.+fn query<T: DeserializeOwned>(name: &str) -> Result<T, ObserverError> {+ let mut payload = request(&serde_json::Value::String(name.to_owned()))?;+ let value = payload+ .get_mut(name)+ .map(serde_json::Value::take)+ .ok_or_else(|| ObserverError::new(format!("niri answered {name} with another response")))?;+ serde_json::from_value(value)+ .map_err(|error| ObserverError::new(format!("invalid niri {name} JSON: {error}")))+}++#[derive(Clone, Debug, Deserialize, PartialEq)]+struct Layout {+ tile_size: (f64, f64),+ tile_pos_in_workspace_view: Option<(f64, f64)>,+}++#[derive(Clone, Debug, Deserialize, PartialEq)]+struct Window {+ id: u64,+ pid: Option<i64>,+ workspace_id: Option<u64>,+ is_focused: bool,+ is_floating: bool,+ layout: Layout,+}++#[derive(Clone, Debug, Deserialize)]+struct Workspace {+ id: u64,+ output: Option<String>,+ is_active: bool,+}++#[derive(Clone, Debug, Deserialize)]+struct Logical {+ width: u32,+ height: u32,+}++#[derive(Clone, Debug, Deserialize)]+struct Output {+ logical: Option<Logical>,+}++#[derive(Clone, Debug, Deserialize)]+struct Overview {+ is_open: bool,+}++struct Desktop {+ windows: Vec<Window>,+ workspaces: Vec<Workspace>,+ outputs: HashMap<String, Output>,+ overview_open: bool,+ sentinel: Option<Sentinel>,+ /// When the window list was first read; the sentinel's fullscreen report+ /// must be at least this recent.+ started: Duration,+}++/// The foreground sentinel's window and its fullscreen state file, which the+/// sentinel replaces on every tick and setup never resets.+#[derive(Clone, Debug)]+struct Sentinel {+ window: TargetWindow,+ state: PathBuf,+}++static SENTINEL: Mutex<Option<Sentinel>> = Mutex::new(None);++/// Registers the foreground sentinel whose fullscreen reports may certify+/// occlusion, or clears it with `None`.+pub(super) fn register_sentinel(sentinel: Option<(TargetWindow, PathBuf)>) {+ *SENTINEL+ .lock()+ .unwrap_or_else(|poisoned| poisoned.into_inner()) =+ sentinel.map(|(window, state)| Sentinel { window, state });+}++/// Milliseconds since the epoch, the sentinel's report clock.+fn report_clock() -> Duration {+ let now = SystemTime::now()+ .duration_since(SystemTime::UNIX_EPOCH)+ .unwrap_or_default();+ Duration::from_millis(now.as_millis() as u64)+}++fn registered_sentinel() -> Option<Sentinel> {+ SENTINEL+ .lock()+ .unwrap_or_else(|poisoned| poisoned.into_inner())+ .clone()+}++/// The fullscreen state the sentinel wrote to `state` at or after `since`,+/// waiting up to `timeout` for that report. An older report says nothing+/// about the window list read since.+fn fullscreen_since(state: &Path, since: Duration, timeout: Duration) -> Option<bool> {+ let deadline = Instant::now() + timeout;+ loop {+ let report = std::fs::read_to_string(state)+ .ok()+ .and_then(|state| fullscreen_report(&state));+ if let Some((_, fullscreen)) = report.filter(|(at, _)| *at >= since) {+ return Some(fullscreen);+ }+ if Instant::now() >= deadline {+ return None;+ }+ std::thread::sleep(Duration::from_millis(10));+ }+}++/// `{"at_ms":…,"fullscreen":…}` as the sentinel writes it.+fn fullscreen_report(state: &str) -> Option<(Duration, bool)> {+ let state: serde_json::Value = serde_json::from_str(state).ok()?;+ Some((+ Duration::from_millis(state["at_ms"].as_u64()?),+ state["fullscreen"].as_bool()?,+ ))+}++/// One desktop state from four requests. Each request is its own connection,+/// so the windows are read again last and the read is retried when they+/// changed in between, for example across a workspace switch.+fn desktop() -> Result<Desktop, ObserverError> {+ ensure_attested()?;+ for _ in 0..3 {+ let started = report_clock();+ let windows: Vec<Window> = query("Windows")?;+ let workspaces = query("Workspaces")?;+ let outputs = query("Outputs")?;+ let overview_open = query::<Overview>("OverviewState")?.is_open;+ if query::<Vec<Window>>("Windows")? == windows {+ return Ok(Desktop {+ windows,+ workspaces,+ outputs,+ overview_open,+ sentinel: registered_sentinel(),+ started,+ });+ }+ }+ Err(ObserverError::new(+ "niri windows kept changing while the desktop was read",+ ))+}++fn focused(windows: &[Window]) -> Result<Option<&Window>, ObserverError> {+ let mut focused = windows.iter().filter(|window| window.is_focused);+ let first = focused.next();+ if focused.next().is_some() {+ return Err(ObserverError::new(+ "niri reports more than one focused window",+ ));+ }+ Ok(first)+}++/// The target by exact niri id, or by its pid's only window when no id is known.+fn select(windows: &[Window], target: TargetWindow) -> Result<Option<&Window>, ObserverError> {+ let pid = Some(i64::from(target.pid));+ if target.native_id != 0 {+ let window = windows.iter().find(|window| window.id == target.native_id);+ if window.is_some_and(|window| window.pid != pid) {+ return Err(ObserverError::new(+ "niri target window id belongs to a different pid",+ ));+ }+ return Ok(window);+ }+ let mut matches = windows.iter().filter(|window| window.pid == pid);+ let first = matches.next();+ if matches.next().is_some() {+ return Err(ObserverError::new(+ "niri target pid has multiple windows; native_id required",+ ));+ }+ Ok(first)+}++fn covers_output(window: &Window, output: &Logical) -> bool {+ let (width, height) = window.layout.tile_size;+ width + FULLSCREEN_TOLERANCE >= f64::from(output.width)+ && height + FULLSCREEN_TOLERANCE >= f64::from(output.height)+}++fn floating_overlap(a: &Window, b: &Window) -> Option<bool> {+ let (ax, ay) = a.layout.tile_pos_in_workspace_view?;+ let (bx, by) = b.layout.tile_pos_in_workspace_view?;+ let (aw, ah) = a.layout.tile_size;+ let (bw, bh) = b.layout.tile_size;+ Some(ax < bx + bw && bx < ax + aw && ay < by + bh && by < ay + ah)+}++/// Whether the focused output-sized `tile` hides floating windows: only when+/// it is the sentinel and the sentinel reports fullscreen after the window+/// list was read. Only this case waits for the report.+fn sentinel_occludes(desktop: &Desktop, tile: &Window) -> Result<TargetZ, ObserverError> {+ let Some(sentinel) = desktop.sentinel.as_ref().filter(|sentinel| {+ sentinel.window.native_id == tile.id && tile.pid == Some(i64::from(sentinel.window.pid))+ }) else {+ return Err(ObserverError::new(+ "niri focused output-sized tile is not the foreground sentinel; \+ fullscreen and occlusion are unproven",+ ));+ };+ match fullscreen_since(&sentinel.state, desktop.started, SENTINEL_REPORT_TIMEOUT) {+ Some(true) => Ok(TargetZ::BackgroundOccluded),+ Some(false) => Err(ObserverError::new(+ "niri sentinel reports it is not fullscreen; a maximized tile does not hide \+ floating windows",+ )),+ None => Err(ObserverError::new(format!(+ "niri sentinel wrote no fullscreen report since the window list was read, \+ after waiting {SENTINEL_REPORT_TIMEOUT:?}: its timer is stalled"+ ))),+ }+}++fn classify(desktop: &Desktop, target: TargetWindow) -> Result<TargetZ, ObserverError> {+ let Some(target) = select(&desktop.windows, target)? else {+ return Ok(TargetZ::NotFound);+ };+ if target.is_focused {+ return Ok(TargetZ::Foreground);+ }+ let workspace = target.workspace_id.and_then(|id| {+ desktop+ .workspaces+ .iter()+ .find(|workspace| workspace.id == id)+ });+ let Some(workspace) = workspace.filter(|workspace| workspace.is_active) else {+ // niri has no minimize; a window off the active workspace is not shown.+ return Ok(TargetZ::Minimized);+ };+ if desktop.overview_open {+ return Err(ObserverError::new(+ "niri overview is open; z-order is ambiguous",+ ));+ }+ let output = workspace+ .output+ .as_ref()+ .and_then(|name| desktop.outputs.get(name))+ .and_then(|output| output.logical.as_ref())+ .ok_or_else(|| ObserverError::new("niri target workspace has no logical output"))?;+ if !target.is_floating {+ return Err(ObserverError::new(+ "niri IPC reports no position for tiled windows; z-order is ambiguous",+ ));+ }+ let focused = focused(&desktop.windows)?;+ if let Some(tile) = focused.filter(|focused| {+ !focused.is_floating+ && focused.workspace_id == Some(workspace.id)+ && covers_output(focused, output)+ }) {+ return sentinel_occludes(desktop, tile);+ }+ for other in &desktop.windows {+ if other.id == target.id || !other.is_floating || other.workspace_id != target.workspace_id+ {+ continue;+ }+ // IPC has no floating stacking order, so any overlap is ambiguous.+ if floating_overlap(target, other) != Some(false) {+ return Err(ObserverError::new(+ "niri overlapping floating windows have ambiguous z-order",+ ));+ }+ }+ Ok(TargetZ::BackgroundVisible)+}++pub(super) fn probe() -> Result<(), ObserverError> {+ let version: serde_json::Value = query("Version")?;+ if !version.is_string() {+ return Err(ObserverError::new(+ "niri answered Version without a version",+ ));+ }+ desktop().map(|_| ())+}++pub(super) fn snapshot(target: TargetWindow) -> Result<DesktopSnapshot, ObserverError> {+ let desktop = desktop()?;+ let focus = focused(&desktop.windows)?.map(|window| window.id);+ Ok(DesktopSnapshot {+ foreground: focus,+ input_focus: focus,+ target_z: classify(&desktop, target)?,+ cursor_pos: None,+ })+}++pub(super) fn focus_identity() -> Result<Option<u64>, ObserverError> {+ ensure_attested()?;+ Ok(focused(&query::<Vec<Window>>("Windows")?)?.map(|window| window.id))+}++#[derive(Deserialize)]+struct WindowFocus {+ id: u64,+ is_focused: bool,+}++/// Which windows niri reports focused, updated from its event stream the way+/// niri-ipc's `EventStreamState` applies window events.+#[derive(Default)]+struct FocusState {+ windows: Option<HashMap<u64, bool>>,+}++impl FocusState {+ /// Applies one event line. Events before the first `WindowsChanged` and+ /// events about other state are ignored.+ fn apply(&mut self, line: &[u8]) -> Result<(), ObserverError> {+ let event: serde_json::Value = serde_json::from_slice(line)+ .map_err(|error| ObserverError::new(format!("invalid niri event: {error}")))?;+ let invalid =+ |error: serde_json::Error| ObserverError::new(format!("invalid niri event: {error}"));+ if let Some(change) = event.get("WindowsChanged") {+ let windows: Vec<WindowFocus> =+ serde_json::from_value(change["windows"].clone()).map_err(invalid)?;+ self.windows = Some(+ windows+ .into_iter()+ .map(|window| (window.id, window.is_focused))+ .collect(),+ );+ return Ok(());+ }+ let Some(windows) = self.windows.as_mut() else {+ return Ok(());+ };+ if let Some(change) = event.get("WindowOpenedOrChanged") {+ let window: WindowFocus =+ serde_json::from_value(change["window"].clone()).map_err(invalid)?;+ if window.is_focused {+ windows.values_mut().for_each(|focused| *focused = false);+ }+ windows.insert(window.id, window.is_focused);+ } else if let Some(change) = event.get("WindowClosed") {+ let id: u64 = serde_json::from_value(change["id"].clone()).map_err(invalid)?;+ windows.remove(&id);+ } else if let Some(change) = event.get("WindowFocusChanged") {+ let id: Option<u64> = serde_json::from_value(change["id"].clone()).map_err(invalid)?;+ for (window, focused) in windows.iter_mut() {+ *focused = Some(*window) == id;+ }+ }+ Ok(())+ }++ /// The focused window, once the window list is known.+ fn focus(&self) -> Result<Option<Option<u64>>, ObserverError> {+ let Some(windows) = &self.windows else {+ return Ok(None);+ };+ let mut focused = windows.iter().filter(|(_, focused)| **focused);+ let first = focused.next().map(|(id, _)| *id);+ if focused.next().is_some() {+ return Err(ObserverError::new(+ "niri reports more than one focused window",+ ));+ }+ Ok(Some(first))+ }+}++const JOURNAL_EVENT_LIMIT: usize = 10_000;++/// Records every focus change niri reports through its event stream, so a+/// focus that leaves and returns between two polls is still seen.+pub(super) struct Journal {+ stop: Arc<AtomicBool>,+ sampler: Option<JoinHandle<Result<DesktopJournal, ObserverError>>>,+}++impl Journal {+ pub(super) fn start() -> Result<Self, ObserverError> {+ ensure_attested()?;+ let mut stream = attested_connection("EventStream")?;+ writeln!(stream, "\"EventStream\"")+ .map_err(|error| ObserverError::new(format!("niri EventStream request: {error}")))?;+ Self::with_stream(stream)+ }++ /// Ready barrier: the subscription is acknowledged and niri has sent its+ /// current window list before the observed action may start.+ fn with_stream(stream: UnixStream) -> Result<Self, ObserverError> {+ let mut reader = BufReader::new(stream);+ let mut line = Vec::new();+ let deadline = Instant::now() + QUERY_TIMEOUT;+ let read_line = |reader: &mut BufReader<UnixStream>, line: &mut Vec<u8>| {+ line.clear();+ match reader.read_until(b'\n', line) {+ Ok(0) => Err(ObserverError::new("niri event stream closed")),+ Ok(_) => Ok(()),+ Err(error) => Err(ObserverError::new(format!("niri event stream: {error}"))),+ }+ };+ read_line(&mut reader, &mut line)?;+ let reply = parse_reply(&String::from_utf8_lossy(&line))?;+ if reply != serde_json::json!("Handled") {+ return Err(ObserverError::new(format!(+ "niri answered EventStream with {reply}"+ )));+ }+ let mut state = FocusState::default();+ let mut focus = loop {+ if Instant::now() >= deadline {+ return Err(ObserverError::new("niri event stream sent no window list"));+ }+ read_line(&mut reader, &mut line)?;+ state.apply(&line)?;+ if let Some(focus) = state.focus()? {+ break focus;+ }+ };+ reader+ .get_ref()+ .set_read_timeout(Some(Duration::from_millis(25)))+ .map_err(|error| ObserverError::new(format!("niri event stream: {error}")))?;+ let stop = Arc::new(AtomicBool::new(false));+ let thread_stop = Arc::clone(&stop);+ let sampler = std::thread::spawn(move || {+ let mut journal = DesktopJournal::default();+ let mut line = Vec::new();+ // After a stop, keep reading until the socket is quiet: niri may+ // still be writing events it queued during the action.+ let mut quiet_deadline = None;+ loop {+ if quiet_deadline.is_none() && thread_stop.load(Ordering::Acquire) {+ quiet_deadline = Some(Instant::now() + QUERY_TIMEOUT);+ }+ // A timed-out read keeps its partial line for the next read.+ match reader.read_until(b'\n', &mut line) {+ Ok(0) => return Err(ObserverError::new("niri event stream closed")),+ Ok(_) if line.ends_with(b"\n") => {+ state.apply(&line)?;+ if let Some(current) = state.focus()? {+ if current != focus {+ if journal.focus_events.len() >= JOURNAL_EVENT_LIMIT {+ return Err(ObserverError::new(+ "niri focus journal exceeded event limit",+ ));+ }+ journal.focus_events.push(FocusEvent {+ from: focus,+ to: current,+ });+ focus = current;+ }+ }+ line.clear();+ }+ Ok(_) => {}+ Err(error)+ if matches!(+ error.kind(),+ std::io::ErrorKind::WouldBlock | std::io::ErrorKind::TimedOut+ ) =>+ {+ if quiet_deadline.is_some() && line.is_empty() {+ return Ok(journal);+ }+ }+ Err(error) => {+ return Err(ObserverError::new(format!("niri event stream: {error}")))+ }+ }+ if quiet_deadline.is_some_and(|deadline| Instant::now() >= deadline) {+ return Err(ObserverError::new(format!(+ "niri event stream did not go quiet within {QUERY_TIMEOUT:?}"+ )));+ }+ }+ });+ Ok(Self {+ stop,+ sampler: Some(sampler),+ })+ }++ pub(super) fn drain(mut self) -> Result<DesktopJournal, ObserverError> {+ self.stop.store(true, Ordering::Release);+ self.sampler+ .take()+ .expect("active journal")+ .join()+ .map_err(|_| ObserverError::new("niri focus journal panicked"))?+ }+}++impl Drop for Journal {+ fn drop(&mut self) {+ self.stop.store(true, Ordering::Release);+ if let Some(sampler) = self.sampler.take() {+ let _ = sampler.join();+ }+ }+}++/// Whether focus left `from` for `to`, both resolved by exact identity. This+/// does not use z-order: the tiled sentinel has no IPC position, so its own+/// z-order is ambiguous once a floating window holds focus.+fn focus_transferred(+ windows: &[Window],+ from: TargetWindow,+ to: TargetWindow,+) -> Result<bool, ObserverError> {+ let resolve = |target| {+ select(windows, target)?+ .map(|window| window.id)+ .ok_or_else(|| ObserverError::new("niri focus-transfer window not found"))+ };+ let (from, to) = (resolve(from)?, resolve(to)?);+ if from == to {+ return Err(ObserverError::new(+ "niri focus transfer requires two distinct windows",+ ));+ }+ Ok(focused(windows)?.map(|window| window.id) == Some(to))+}++pub(super) fn wait_for_focus_transfer(+ from: TargetWindow,+ to: TargetWindow,+ timeout: Duration,+) -> Result<(), ObserverError> {+ ensure_attested()?;+ let deadline = std::time::Instant::now() + timeout;+ loop {+ let windows: Vec<Window> = query("Windows")?;+ if focus_transferred(&windows, from, to)? {+ return Ok(());+ }+ if std::time::Instant::now() >= deadline {+ return Err(ObserverError::new(format!(+ "niri focus did not move from window {} to window {} within {timeout:?}; focused: {:?}",+ from.native_id,+ to.native_id,+ focused(&windows)?.map(|window| window.id)+ )));+ }+ std::thread::sleep(Duration::from_millis(25));+ }+}++/// Focuses the target through niri's own action, for test setup only.+pub(super) fn focus(target: TargetWindow) -> Result<(), ObserverError> {+ ensure_attested()?;+ let id = select(&query::<Vec<Window>>("Windows")?, target)?+ .ok_or_else(|| ObserverError::new("niri target window not found"))?+ .id;+ let reply = request(&serde_json::json!({ "Action": { "FocusWindow": { "id": id } } }))?;+ if reply == serde_json::json!("Handled") {+ Ok(())+ } else {+ Err(ObserverError::new(format!(+ "niri answered FocusWindow with {reply}"+ )))+ }+}++#[cfg(test)]+mod tests {+ use super::*;+ use serde_json::json;++ fn window(id: u64, pid: i64, floating: bool, focused: bool) -> serde_json::Value {+ json!({+ "id": id, "title": format!("window {id}"), "app_id": "fixture", "pid": pid,+ "workspace_id": 1, "is_focused": focused, "is_floating": floating,+ "is_urgent": false, "focus_timestamp": null,+ "layout": {+ "pos_in_scrolling_layout": if floating { json!(null) } else { json!([1, 1]) },+ "tile_size": [940.0, 780.0], "window_size": [940, 780],+ "tile_pos_in_workspace_view": if floating { json!([0.0, 0.0]) } else { json!(null) },+ "window_offset_in_tile": [0.0, 0.0],+ },+ })+ }++ fn fullscreen(mut window: serde_json::Value) -> serde_json::Value {+ window["layout"]["tile_size"] = json!([1920.0, 1080.0]);+ window["layout"]["window_size"] = json!([1920, 1080]);+ window+ }++ fn desktop(windows: Vec<serde_json::Value>) -> Desktop {+ Desktop {+ windows: serde_json::from_value(json!(windows)).unwrap(),+ workspaces: serde_json::from_value(json!([+ {"id": 1, "idx": 1, "name": null, "output": "winit", "is_urgent": false,+ "is_active": true, "is_focused": true, "active_window_id": 10},+ {"id": 2, "idx": 2, "name": null, "output": "winit", "is_urgent": false,+ "is_active": false, "is_focused": false, "active_window_id": null},+ ]))+ .unwrap(),+ outputs: serde_json::from_value(json!({+ "winit": {"name": "winit", "make": "Smithay", "model": "Winit",+ "serial": null, "physical_size": null, "modes": [], "current_mode": null,+ "is_custom_mode": true, "vrr_supported": false, "vrr_enabled": false,+ "logical": {"x": 0, "y": 0, "width": 1920, "height": 1080, "scale": 1.0,+ "transform": "Flipped180"}}+ }))+ .unwrap(),+ overview_open: false,+ sentinel: Some(Sentinel {+ window: target(10, 100),+ state: PathBuf::from("/nonexistent/sentinel-state.json"),+ }),+ started: Duration::ZERO,+ }+ }++ /// Points the desktop's sentinel at a state file holding `report`.+ fn report(desktop: &mut Desktop, report: &str) -> tempfile::TempDir {+ let directory = tempfile::tempdir().unwrap();+ let state = directory.path().join("sentinel-state.json");+ std::fs::write(&state, report).unwrap();+ desktop.sentinel.as_mut().unwrap().state = state;+ directory+ }++ fn target(native_id: u64, pid: u32) -> TargetWindow {+ TargetWindow { pid, native_id }+ }++ #[test]+ fn floating_target_behind_focused_fullscreen_sentinel_is_occluded() {+ let mut desktop = desktop(vec![+ fullscreen(window(10, 100, false, true)),+ window(20, 200, true, false),+ ]);+ let _state = report(&mut desktop, r#"{"at_ms":1,"fullscreen":true}"#);+ assert_eq!(+ classify(&desktop, target(20, 200)).unwrap(),+ TargetZ::BackgroundOccluded+ );+ assert_eq!(+ classify(&desktop, target(10, 100)).unwrap(),+ TargetZ::Foreground+ );+ }++ #[test]+ fn client_maximized_sentinel_does_not_certify_occlusion() {+ // A client maximize fills the output like fullscreen, but floating+ // windows stay drawn above it. Only the sentinel's report differs.+ let mut desktop = desktop(vec![+ fullscreen(window(10, 100, false, true)),+ window(20, 200, true, false),+ ]);+ let _state = report(&mut desktop, r#"{"at_ms":1,"fullscreen":false}"#);+ assert!(classify(&desktop, target(20, 200))+ .unwrap_err()+ .to_string()+ .contains("not fullscreen"));+ let _state = report(&mut desktop, r#"{"at_ms":1}"#);+ assert!(classify(&desktop, target(20, 200))+ .unwrap_err()+ .to_string()+ .contains("no fullscreen report"));+ }++ #[test]+ fn output_sized_tile_other_than_the_sentinel_is_ambiguous() {+ let mut desktop = desktop(vec![+ fullscreen(window(10, 100, false, true)),+ window(20, 200, true, false),+ ]);+ for sentinel in [None, Some(target(11, 100)), Some(target(10, 101))] {+ desktop.sentinel = sentinel.map(|window| Sentinel {+ window,+ state: PathBuf::from("/nonexistent/sentinel-state.json"),+ });+ assert!(classify(&desktop, target(20, 200))+ .unwrap_err()+ .to_string()+ .contains("not the foreground sentinel"));+ }+ }++ #[test]+ fn sentinel_fullscreen_report_survives_a_journal_reset_before_the_posture_snapshot() {+ // ForegroundSentinel waits for a heartbeat, resets its event journal+ // and immediately checks the background posture.+ let directory = tempfile::tempdir().unwrap();+ let journal = directory.path().join("sentinel-events.jsonl");+ let state = directory.path().join("sentinel-state.json");+ let since = report_clock();+ let now = since.as_millis();+ std::fs::write(+ &journal,+ format!("{{\"kind\":\"heartbeat\",\"at_ms\":{now}}}\n"),+ )+ .unwrap();+ std::fs::write(&state, format!(r#"{{"at_ms":{now},"fullscreen":true}}"#)).unwrap();+ register_sentinel(Some((target(10, 100), state)));+ std::fs::write(&journal, "").unwrap();+ let mut posture = desktop(vec![+ fullscreen(window(10, 100, false, true)),+ window(20, 200, true, false),+ ]);+ posture.sentinel = registered_sentinel();+ posture.started = since;+ register_sentinel(None);+ assert_eq!(+ classify(&posture, target(20, 200)).unwrap(),+ TargetZ::BackgroundOccluded+ );+ }++ #[test]+ fn focused_sentinel_snapshot_does_not_wait_for_its_fullscreen_report() {+ // Most snapshots never consult the report, for example while waiting+ // for the sentinel itself to hold focus. Here no report ever arrives.+ let desktop = desktop(vec![+ fullscreen(window(10, 100, false, true)),+ window(20, 200, true, false),+ ]);+ let started = Instant::now();+ assert_eq!(+ classify(&desktop, target(10, 100)).unwrap(),+ TargetZ::Foreground+ );+ assert!(started.elapsed() < SENTINEL_REPORT_TIMEOUT);+ }++ #[test]+ fn sentinel_fullscreen_report_must_follow_the_window_list_read() {+ let directory = tempfile::tempdir().unwrap();+ let state = directory.path().join("sentinel-state.json");+ let since = Duration::from_millis(10_000);+ let timeout = Duration::from_millis(30);+ // A report from before the read may predate an unfullscreen.+ std::fs::write(&state, r#"{"at_ms":9999,"fullscreen":true}"#).unwrap();+ assert_eq!(fullscreen_since(&state, since, timeout), None);+ std::fs::write(&state, r#"{"at_ms":10000,"fullscreen":false}"#).unwrap();+ assert_eq!(fullscreen_since(&state, since, timeout), Some(false));+ std::fs::write(&state, r#"{"at_ms":10000}"#).unwrap();+ assert_eq!(fullscreen_since(&state, since, timeout), None);++ // The snapshot waits for the next report.+ std::fs::write(&state, r#"{"at_ms":9999,"fullscreen":false}"#).unwrap();+ let writer = {+ let state = state.clone();+ std::thread::spawn(move || {+ std::thread::sleep(Duration::from_millis(20));+ std::fs::write(state, r#"{"at_ms":10100,"fullscreen":true}"#).unwrap();+ })+ };+ assert_eq!(+ fullscreen_since(&state, since, SENTINEL_REPORT_TIMEOUT),+ Some(true)+ );+ writer.join().unwrap();+ }++ #[test]+ fn a_sentinel_tick_that_runs_late_still_reports() {+ // On a loaded runner the sentinel's timer skipped three ticks, which+ // its heartbeat liveness bound allows.+ let directory = tempfile::tempdir().unwrap();+ let state = directory.path().join("sentinel-state.json");+ let since = Duration::from_millis(10_000);+ std::fs::write(&state, r#"{"at_ms":9999,"fullscreen":true}"#).unwrap();+ let writer = {+ let state = state.clone();+ std::thread::spawn(move || {+ std::thread::sleep(Duration::from_millis(400));+ std::fs::write(state, r#"{"at_ms":10400,"fullscreen":true}"#).unwrap();+ })+ };+ assert_eq!(+ fullscreen_since(&state, since, SENTINEL_REPORT_TIMEOUT),+ Some(true)+ );+ writer.join().unwrap();+ }++ #[test]+ fn floating_target_above_a_non_fullscreen_tile_is_visible() {+ let desktop = desktop(vec![+ window(10, 100, false, true),+ window(20, 200, true, false),+ ]);+ assert_eq!(+ classify(&desktop, target(20, 200)).unwrap(),+ TargetZ::BackgroundVisible+ );+ }++ #[test]+ fn floating_overlap_tiled_target_and_open_overview_are_ambiguous() {+ let overlapping = desktop(vec![+ window(10, 100, true, true),+ window(20, 200, true, false),+ ]);+ assert!(classify(&overlapping, target(20, 200))+ .unwrap_err()+ .to_string()+ .contains("overlapping floating windows"));++ let tiled = desktop(vec![+ fullscreen(window(10, 100, false, true)),+ window(20, 200, false, false),+ ]);+ assert!(classify(&tiled, target(20, 200))+ .unwrap_err()+ .to_string()+ .contains("no position for tiled windows"));++ let mut overview = desktop(vec![+ fullscreen(window(10, 100, false, true)),+ window(20, 200, true, false),+ ]);+ overview.overview_open = true;+ assert!(classify(&overview, target(20, 200))+ .unwrap_err()+ .to_string()+ .contains("overview is open"));+ }++ #[test]+ fn window_on_an_inactive_workspace_is_not_shown() {+ let mut hidden = window(20, 200, true, false);+ hidden["workspace_id"] = json!(2);+ let desktop = desktop(vec![fullscreen(window(10, 100, false, true)), hidden]);+ assert_eq!(+ classify(&desktop, target(20, 200)).unwrap(),+ TargetZ::Minimized+ );+ }++ #[test]+ fn identity_is_the_exact_niri_id_and_never_a_pid_sibling() {+ let desktop = desktop(vec![+ fullscreen(window(10, 100, false, true)),+ window(20, 200, true, false),+ window(21, 200, true, false),+ ]);+ assert_eq!(+ classify(&desktop, target(30, 200)).unwrap(),+ TargetZ::NotFound+ );+ assert!(classify(&desktop, target(20, 300))+ .unwrap_err()+ .to_string()+ .contains("different pid"));+ assert!(classify(&desktop, target(0, 200))+ .unwrap_err()+ .to_string()+ .contains("native_id required"));+ }++ #[test]+ fn focus_canary_sees_the_floating_target_take_focus_from_the_sentinel() {+ let sentinel = target(10, 100);+ let harness = target(20, 200);+ let before = desktop(vec![+ fullscreen(window(10, 100, false, true)),+ window(20, 200, true, false),+ ]);+ assert!(!focus_transferred(&before.windows, sentinel, harness).unwrap());++ let after = desktop(vec![+ fullscreen(window(10, 100, false, false)),+ window(20, 200, true, true),+ ]);+ assert!(focus_transferred(&after.windows, sentinel, harness).unwrap());+ // The unfocused tiled sentinel has no IPC position, so z-order cannot+ // decide this canary.+ assert!(classify(&after, sentinel).is_err());+ assert!(focus_transferred(&after.windows, sentinel, sentinel).is_err());+ }++ fn windows_changed(focused_id: u64) -> String {+ json!({"WindowsChanged": {"windows": [+ fullscreen(window(10, 100, false, focused_id == 10)),+ window(20, 200, true, focused_id == 20),+ ]}})+ .to_string()+ }++ #[test]+ fn journal_records_a_focus_that_leaves_and_returns() {+ let (stream, mut niri) = UnixStream::pair().unwrap();+ writeln!(niri, r#"{{"Ok":"Handled"}}"#).unwrap();+ writeln!(niri, r#"{{"WorkspacesChanged":{{"workspaces":[]}}}}"#).unwrap();+ writeln!(niri, "{}", windows_changed(10)).unwrap();+ let journal = Journal::with_stream(stream).unwrap();+ // The second focus event arrives in two writes to exercise a read+ // that times out mid-line.+ writeln!(niri, r#"{{"WindowFocusChanged":{{"id":20}}}}"#).unwrap();+ write!(niri, r#"{{"WindowFocusChanged":"#).unwrap();+ std::thread::sleep(Duration::from_millis(60));+ writeln!(niri, r#"{{"id":10}}}}"#).unwrap();+ let journal = journal.drain().unwrap();+ assert_eq!(+ journal.focus_events,+ vec![+ FocusEvent {+ from: Some(10),+ to: Some(20)+ },+ FocusEvent {+ from: Some(20),+ to: Some(10)+ },+ ]+ );+ }++ #[test]+ fn journal_records_focus_taken_by_an_opened_or_changed_window() {+ // niri reports focus on a window it opens, or whose title or+ // workspace changes in the same refresh, only inside+ // WindowOpenedOrChanged; no WindowFocusChanged follows.+ let (stream, mut niri) = UnixStream::pair().unwrap();+ writeln!(niri, r#"{{"Ok":"Handled"}}"#).unwrap();+ writeln!(niri, "{}", windows_changed(10)).unwrap();+ let journal = Journal::with_stream(stream).unwrap();+ let mut dialog = window(30, 200, true, true);+ dialog["title"] = json!("dialog");+ writeln!(+ niri,+ "{}",+ json!({"WindowOpenedOrChanged": {"window": dialog}})+ )+ .unwrap();+ writeln!(niri, r#"{{"WindowClosed":{{"id":30}}}}"#).unwrap();+ writeln!(niri, r#"{{"WindowFocusChanged":{{"id":10}}}}"#).unwrap();+ let mut renamed = window(20, 200, true, true);+ renamed["title"] = json!("renamed");+ writeln!(+ niri,+ "{}",+ json!({"WindowOpenedOrChanged": {"window": renamed}})+ )+ .unwrap();+ writeln!(niri, r#"{{"WindowFocusChanged":{{"id":10}}}}"#).unwrap();+ let events = journal.drain().unwrap().focus_events;+ let transitions = events+ .iter()+ .map(|event| (event.from, event.to))+ .collect::<Vec<_>>();+ assert_eq!(+ transitions,+ vec![+ (Some(10), Some(30)),+ (Some(30), None),+ (None, Some(10)),+ (Some(10), Some(20)),+ (Some(20), Some(10)),+ ]+ );+ }++ #[test]+ fn journal_waits_for_the_window_list_and_reports_a_closed_stream() {+ let (stream, mut niri) = UnixStream::pair().unwrap();+ writeln!(niri, r#"{{"Ok":"Handled"}}"#).unwrap();+ drop(niri);+ assert!(Journal::with_stream(stream)+ .err()+ .unwrap()+ .to_string()+ .contains("closed"));++ let (stream, mut niri) = UnixStream::pair().unwrap();+ writeln!(niri, r#"{{"Ok":"Handled"}}"#).unwrap();+ writeln!(niri, "{}", windows_changed(10)).unwrap();+ let journal = Journal::with_stream(stream).unwrap();+ drop(niri);+ std::thread::sleep(Duration::from_millis(60));+ assert!(journal.drain().unwrap_err().to_string().contains("closed"));+ }++ #[test]+ fn socket_peer_is_the_listening_process() {+ let directory = tempfile::tempdir().unwrap();+ let path = directory.path().join("niri.sock");+ let _listener = std::os::unix::net::UnixListener::bind(&path).unwrap();+ assert_eq!(socket_peer(&path).unwrap() as u32, std::process::id());+ assert!(socket_peer(&directory.path().join("missing.sock")).is_err());+ }++ #[test]+ fn attestation_falls_through_until_niri_is_recognised_then_fails_closed() {+ let lost = || Err("NIRI_SOCKET is unreachable".to_owned());+ assert_eq!(attestation(None, Ok((7, 7))).unwrap(), Some(7));+ assert_eq!(attestation(None, Ok((7, 8))).unwrap(), None);+ assert_eq!(attestation(None, lost()).unwrap(), None);+ assert_eq!(attestation(Some(7), Ok((7, 7))).unwrap(), Some(7));+ assert!(attestation(Some(7), lost())+ .unwrap_err()+ .to_string()+ .contains("unreachable"));+ for peers in [(8, 8), (7, 8), (8, 7)] {+ assert!(attestation(Some(7), Ok(peers))+ .unwrap_err()+ .to_string()+ .contains("now served by"));+ }+ }++ #[test]+ fn replies_unwrap_ok_and_preserve_err() {+ assert_eq!(+ parse_reply(r#"{"Ok":{"Version":"26.04"}}"#).unwrap(),+ json!({"Version": "26.04"})+ );+ assert!(parse_reply(r#"{"Err":"no such window"}"#)+ .unwrap_err()+ .to_string()+ .contains("no such window"));+ assert!(parse_reply("").is_err());+ }+}diff --git a/libs/cua-driver/rust/crates/cua-driver-testkit/src/sentinel.rs b/libs/cua-driver/rust/crates/cua-driver-testkit/src/sentinel.rsindex 18ab8744da..4e58771733 100644--- a/libs/cua-driver/rust/crates/cua-driver-testkit/src/sentinel.rs+++ b/libs/cua-driver/rust/crates/cua-driver-testkit/src/sentinel.rs@@ -9,6 +9,12 @@ use crate::e2e::OracleKind;use crate::observer::{DesktopObserver, NativeObserver, TargetWindow};use crate::{harness_app, spawn_in_job, BehaviorRecording, ChildReaper, Driver};+/// How late the sentinel's 100 ms main-process timer may tick before the+/// harness treats the sentinel as stalled. Its heartbeat and, on Linux, its+/// fullscreen state file both come from that timer, which runs late on a+/// loaded GPU-less runner.+pub(crate) const SENTINEL_TICK_LIVENESS: Duration = Duration::from_secs(2);+#[cfg(any(target_os = "linux", test))]#[path = "sentinel_hyprland.rs"]mod hyprland;@@ -24,6 +30,13 @@ pub struct ForegroundSentinel {_user_data: tempfile::TempDir,}+#[cfg(target_os = "linux")]+impl Drop for ForegroundSentinel {+ fn drop(&mut self) {+ crate::observer::linux::niri_register_sentinel(None);+ }+}+impl ForegroundSentinel {pub fn launch(driver: &mut impl Driver) -> Self {let mut last_error = None;@@ -79,6 +92,13 @@ impl ForegroundSentinel {.env("CUA_ELECTRON_CDP_PORT", cdp_port.to_string()).stdout(Stdio::null()).stderr(Stdio::null());+ // Only the niri observer reads the fullscreen state file, so other+ // platforms never pay for the sentinel rewriting it every tick.+ #[cfg(target_os = "linux")]+ command.env(+ "CUA_E2E_SENTINEL_STATE",+ user_data.path().join("sentinel-state.json"),+ );let child = spawn_in_job(&mut command).map_err(|error| format!("launch foreground sentinel: {error}"))?;let launched_pid = child.id();@@ -157,6 +177,11 @@ impl ForegroundSentinel {};fs::write(&journal_path, "").map_err(|error| format!("reset focused sentinel journal: {error}"))?;+ #[cfg(target_os = "linux")]+ crate::observer::linux::niri_register_sentinel(Some((+ target,+ user_data.path().join("sentinel-state.json"),+ )));Ok(Self {evidence_dir: None,@@ -234,7 +259,7 @@ impl ForegroundSentinel {driver: &mut impl Driver,background_target: TargetWindow,) -> Result<(), String> {- wait_for_event(&self.journal_path, "heartbeat", Duration::from_secs(2))?;+ wait_for_event(&self.journal_path, "heartbeat", SENTINEL_TICK_LIVENESS)?;reset_journal(&self.journal_path)?;let canary_key = if std::env::var("CUA_E2E_WAYLAND_SESSION")@@ -279,6 +304,8 @@ impl ForegroundSentinel {raise_for_focus_canary(driver, background_target)?;#[cfg(target_os = "linux")]focus_sway_target(driver, background_target)?;+ #[cfg(target_os = "linux")]+ focus_niri_target(background_target)?;if is_wayland_session() {wait_for_native_focus_lost(self.target, background_target)?;} else {@@ -298,9 +325,14 @@ impl ForegroundSentinel {#[cfg(target_os = "linux")]set_sway_fullscreen(driver, self.target, true)?;wait_for_native_focus_stable(self.target);+ #[cfg(target_os = "linux")]+ if is_niri_session() {+ assert_niri_transient_focus_detected(self.target, background_target)?;+ wait_for_native_focus_stable(self.target);+ }std::thread::sleep(Duration::from_millis(150));reset_journal(&self.journal_path)?;- wait_for_event(&self.journal_path, "heartbeat", Duration::from_secs(2))?;+ wait_for_event(&self.journal_path, "heartbeat", SENTINEL_TICK_LIVENESS)?;reset_journal(&self.journal_path)?;self.assert_background_posture(background_target)}@@ -350,7 +382,7 @@ impl ForegroundSentinel {reset_journal(&self.journal_path)?;// This heartbeat checks liveness only. Windows setup input has already// crossed its explicit renderer/main journal barrier before the reset.- wait_for_event(&self.journal_path, "heartbeat", Duration::from_secs(2))?;+ wait_for_event(&self.journal_path, "heartbeat", SENTINEL_TICK_LIVENESS)?;reset_journal(&self.journal_path)?;self.assert_background_posture(target)}@@ -599,6 +631,10 @@ fn try_activate_native_foreground(focus_sway_target(driver, target).map_err(|error| {format!("could not focus foreground sentinel through Sway IPC: {error}")})?;+ #[cfg(target_os = "linux")]+ focus_niri_target(target).map_err(|error| {+ format!("could not focus foreground sentinel through niri IPC: {error}")+ })?;#[cfg(target_os = "windows")]physically_focus_windows_sentinel(target);#[cfg(target_os = "macos")]@@ -646,6 +682,56 @@ fn focus_sway_target(driver: &mut impl Driver, target: TargetWindow) -> Result<(run_sway_container_command(con_id, &["focus"], "focus canary")}+/// niri's focus oracle must see a focus that leaves the sentinel and returns+/// inside one observation, where both endpoint snapshots agree. Only the+/// action-time journal can report it.+#[cfg(target_os = "linux")]+fn assert_niri_transient_focus_detected(+ sentinel: TargetWindow,+ background: TargetWindow,+) -> Result<(), String> {+ use crate::observer::linux::{niri_focus, niri_wait_for_focus_transfer};++ let mut observer = DesktopObserver::new(NativeObserver::new(), background);+ let (moved, delta) = observer+ .observe(&[OracleKind::Focus], || {+ niri_focus(background)?;+ niri_wait_for_focus_transfer(sentinel, background)?;+ niri_focus(sentinel)?;+ niri_wait_for_focus_transfer(background, sentinel)+ })+ .map_err(|error| error.to_string())?;+ moved.map_err(|error| format!("niri transient focus canary could not move focus: {error}"))?;+ if delta.before.foreground != delta.after.foreground+ || !delta+ .violations()+ .iter()+ .any(|violation| violation.contains("changed transiently"))+ {+ return Err(format!(+ "niri transient focus canary was not detected: {:?}",+ delta.violations()+ ));+ }+ Ok(())+}++#[cfg(target_os = "linux")]+fn is_niri_session() -> bool {+ is_wayland_session()+ && std::env::var("CUA_E2E_WAYLAND_SESSION").is_ok_and(|session| session == "niri")+}++/// niri shows a focused floating window above the fullscreen sentinel, so the+/// niri canary needs no fullscreen toggle, only the compositor's own focus.+#[cfg(target_os = "linux")]+fn focus_niri_target(target: TargetWindow) -> Result<(), String> {+ if !is_niri_session() {+ return Ok(());+ }+ crate::observer::linux::niri_focus(target).map_err(|error| error.to_string())+}+#[cfg(target_os = "linux")]fn set_sway_fullscreen(driver: &mut impl Driver,@@ -1176,6 +1262,10 @@ fn wait_for_native_focus_lost(if hyprland::is_session() {return hyprland::wait_for_focus_transfer(target, background_target);}+ if is_niri_session() {+ return crate::observer::linux::niri_wait_for_focus_transfer(target, background_target)+ .map_err(|error| error.to_string());+ }let backend = NativeObserver::new();let deadline = Instant::now() + Duration::from_secs(3);diff --git a/libs/cua-driver/rust/crates/platform-linux/Cargo.toml b/libs/cua-driver/rust/crates/platform-linux/Cargo.tomlindex 24dea65e43..4c124c8f83 100644--- a/libs/cua-driver/rust/crates/platform-linux/Cargo.toml+++ b/libs/cua-driver/rust/crates/platform-linux/Cargo.toml@@ -112,16 +112,16 @@ calloop = { version = "0.14", optional = true }# re-exports this same major version; pinning explicitly prevents# duplicate-crate hazards.enumflags2 = { version = "0.7", optional = true }-# Keymap fallback path for libei keyboard input when the EIS server does-# NOT advertise `ei_text` (libei < 1.6, i.e. older GNOME / KDE Plasma 6.0).-# Mirrors the reis `type-text.rs` example's keycode reverse-mapping.-xkbcommon = { version = "0.9", optional = true }# Bounded MPSC command/reply channel for the persistent virtual-pointer# owner thread. EventQueue<State> is !Send, so the persistent device must# live on its own thread; crossbeam gives us a Send sender and synchronous# reply channels that preserve the "tool call blocks until compositor# roundtripped" contract the X11 path provides.crossbeam-channel = "0.5"+# Resolves text against the compositor's XKB keymap: niri text input and the+# libei keymap fallback when the EIS server does NOT advertise `ei_text`+# (libei < 1.6, i.e. older GNOME / KDE Plasma 6.0).+xkbcommon = "0.9"# Resolve `$XDG_STATE_HOME` (with platform fallback) for persisting the# ScreenCast and RemoteDesktop `restore_token`s so users see the consent# dialog only once per install, not once per process.@@ -140,7 +140,6 @@ portal-input = ["dep:reis","dep:calloop","dep:enumflags2",- "dep:xkbcommon",]# PipeWire-backed per-window ScreenCast capture keeps its newer system-header# requirement and remains enabled by Nix/modern desktop builds.diff --git a/libs/cua-driver/rust/crates/platform-linux/src/atspi/native.rs b/libs/cua-driver/rust/crates/platform-linux/src/atspi/native.rsindex 1ec531a0a5..92d4e8ddfe 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/atspi/native.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/atspi/native.rs@@ -1079,6 +1079,38 @@ fn correlate_hyprland_frame(Some(frame.ordinal)}+/// A niri window as a correlation client: its title and size. Tiled windows+/// have a size but no placement; an unknown size (0x0) never matches.+fn niri_client(window: &crate::wayland::niri::Window) -> (String, u32, u32) {+ let (width, height) = window.size.unwrap_or((0, 0));+ (window.title.clone(), width, height)+}++/// [`correlate_hyprland_frame`] for niri, which also keeps the single-root+/// shortcut, but only while niri shows the process exactly one window: the+/// requested one.+fn correlate_niri_frame(+ frames: &[HyprlandFrame],+ clients: &[(&str, u32, u32)],+ target: usize,+ roots: usize,+) -> Option<usize> {+ if roots == 1 && clients.len() == 1 && target == 0 {+ return Some(0);+ }+ correlate_hyprland_frame(frames, clients, target)+}++/// Whether a toolkit frame's Window extents are niri's window size, give or+/// take a pixel of rounding. An unknown frame or window never fits.+fn niri_frame_fits(frame: Option<(i32, i32, i32, i32)>, window: Option<(u32, u32)>) -> bool {+ let (Some((_, _, width, height)), Some((window_width, window_height))) = (frame, window) else {+ return false;+ };+ (i64::from(width) - i64::from(window_width)).abs() <= 1+ && (i64::from(height) - i64::from(window_height)).abs() <= 1+}+#[cfg(test)]mod hyprland_frame_tests {use super::*;@@ -1150,6 +1182,73 @@ mod hyprland_frame_tests {);}+ #[test]+ fn niri_binds_only_the_window_a_lone_root_belongs_to() {+ // niri lists A and B for the process; only B's root is exported yet.+ let mut b = frame("B");+ b.browser_root = false;+ b.size = None;+ let clients = [("A", 800, 600), ("B", 800, 600)];+ assert_eq!(correlate_niri_frame(&[b], &clients, 0, 1), None);+ let mut b = frame("B");+ b.browser_root = false;+ assert_eq!(correlate_niri_frame(&[b], &clients, 1, 1), Some(7));+ // With one window for the process, its lone root is that window's,+ // whatever the toolkit names it.+ assert_eq!(+ correlate_niri_frame(&[frame("untitled")], &[("A", 800, 600)], 0, 1),+ Some(0)+ );+ }++ #[test]+ fn niri_frames_publish_only_when_the_toolkit_frame_is_the_window() {+ // Measured in the nested niri lane: GTK 3's frame with client-side+ // decorations includes a 26 px shadow on every side of a 940x780+ // window, which put every element 26 px right and below its pixels.+ assert!(!niri_frame_fits(Some((0, 0, 992, 832)), Some((940, 780))));+ // With niri's prefer-no-csd the frame is the window.+ assert!(niri_frame_fits(Some((0, 0, 940, 780)), Some((940, 780))));+ // A pixel of rounding (fractional scale) and nothing more.+ assert!(niri_frame_fits(Some((0, 0, 627, 519)), Some((626, 520))));+ assert!(!niri_frame_fits(Some((0, 0, 942, 780)), Some((940, 780))));+ // An unknown frame or an unplaced window proves nothing.+ assert!(!niri_frame_fits(None, Some((940, 780))));+ assert!(!niri_frame_fits(Some((0, 0, 940, 780)), None));+ }++ #[test]+ fn niri_binds_a_tiled_chromium_root_by_size() {+ // niri reports no position for tiled windows, but their size still+ // identifies the BrowserRootView whose title carries the profile.+ let tiled = |id, title: &str, size| crate::wayland::niri::Window {+ id,+ pid: Some(42),+ title: title.into(),+ app_id: "chromium".into(),+ workspace: Some(1),+ focused: false,+ focus_rank: 0,+ on_screen: true,+ size: Some(size),+ placement: Err(crate::wayland::niri::Unplaced::Tiled),+ };+ let owned = [+ niri_client(&tiled(1, "Page - Google Chrome", (800, 600))),+ niri_client(&tiled(2, "Other - Google Chrome", (640, 480))),+ ];+ let clients = owned+ .iter()+ .map(|(title, width, height)| (title.as_str(), *width, *height))+ .collect::<Vec<_>>();+ let mut other = frame("Other - Google Chrome - Cua Test");+ other.ordinal = 8;+ other.size = Some((640, 480));+ let frames = [frame("Page - Google Chrome - Cua Test"), other];+ assert_eq!(correlate_niri_frame(&frames, &clients, 0, 2), Some(7));+ assert_eq!(correlate_niri_frame(&frames, &clients, 1, 2), Some(8));+ }+#[test]fn preserves_exact_non_chromium_matching() {let mut candidate = frame("Editor");@@ -1176,10 +1275,32 @@ async fn resolve_window_frame(xid: u64,seeds: &[RawObjectRef],) -> Option<usize> {- if crate::wayland::is_wayland() && crate::wayland::hyprland::is_session() {- let windows = crate::wayland::hyprland::list_windows().ok()?;- let owned: Vec<_> = windows.iter().filter(|window| window.pid == pid).collect();- let target = owned.iter().position(|window| window.address == xid)?;+ let hyprland = crate::wayland::is_wayland() && crate::wayland::hyprland::is_session();+ // Before the single-root shortcut: niri may list a window whose root is+ // not exported yet, and the one root then belongs to another window.+ let niri = !hyprland && crate::wayland::is_wayland() && crate::wayland::niri::is_session();+ if hyprland || niri {+ let (owned, target): (Vec<(String, u32, u32)>, usize) = if hyprland {+ let windows = crate::wayland::hyprland::list_windows().ok()?;+ let owned: Vec<_> = windows.iter().filter(|window| window.pid == pid).collect();+ let target = owned.iter().position(|window| window.address == xid)?;+ let owned = owned+ .iter()+ .map(|window| (window.title.clone(), window.width, window.height))+ .collect();+ (owned, target)+ } else {+ // An X11 client's niri pid is xwayland-satellite's, so it never+ // matches and its walk stays unscoped.+ let windows = crate::wayland::niri::list_windows().ok()?;+ let owned: Vec<_> = windows+ .iter()+ .filter(|window| window.pid == Some(pid))+ .collect();+ let target = owned.iter().position(|window| window.id == xid)?;+ let owned = owned.iter().map(|window| niri_client(window)).collect();+ (owned, target)+ };let chromium = std::fs::read_link(format!("/proc/{pid}/exe")).ok().and_then(|path| {@@ -1225,9 +1346,13 @@ async fn resolve_window_frame(}let clients = owned.iter()- .map(|window| (window.title.as_str(), window.width, window.height))+ .map(|(title, width, height)| (title.as_str(), *width, *height)).collect::<Vec<_>>();- return correlate_hyprland_frame(&frames, &clients, target);+ return if niri {+ correlate_niri_frame(&frames, &clients, target, seeds.len())+ } else {+ correlate_hyprland_frame(&frames, &clients, target)+ };}if seeds.len() == 1 {// One top-level: on X11, and on compositors without a GNOME Shell,@@ -3698,6 +3823,11 @@ pub fn element_bounds_ref("web-content bounds need the document origin; re-walk"));}+ if crate::wayland::is_wayland() && crate::wayland::niri::is_session() {+ return Err(anyhow!(+ "niri element bounds need the window's frame size check; re-walk"+ ));+ }let offset = window_to_screen_offset(pid, xid, None);require_exact_origin(offset, xid, exact_wayland_origin_required(xid))?;let coord = if offset.is_some() {@@ -5888,16 +6018,24 @@ fn screen_answer_overrides_window(!native_origin_attested && screen_extents_trusted(raw, display)}-/// True only when the Hyprland compositor identifies this exact client-/// (nonzero address `xid` AND `pid`) as native Wayland, not XWayland. Missing,-/// mismatched, or unreported clients fail closed, so `Screen` extents keep-/// winning, which is correct for XWayland GTK and LibreOffice. Performs-/// blocking compositor IPC: async callers must use `bounded_blocking`.+/// True only when the compositor identifies this exact client (nonzero+/// window id `xid` AND `pid`) as native Wayland, not XWayland, with an origin.+/// Hyprland reports XWayland per client; under niri an X11 client belongs to+/// xwayland-satellite's pid, so a matching pid on a placed niri window is a+/// native client. Missing, mismatched, or unreported clients fail closed, so+/// `Screen` extents keep winning, which is correct for XWayland GTK and+/// LibreOffice. Performs blocking compositor IPC: async callers must use+/// `bounded_blocking`.fn native_wayland_origin_attested(pid: u32, xid: u64) -> bool {xid != 0&& crate::wayland::is_wayland()- && crate::wayland::hyprland::is_session()- && crate::wayland::hyprland::native_client_attested(xid, pid)+ && if crate::wayland::hyprland::is_session() {+ crate::wayland::hyprland::native_client_attested(xid, pid)+ } else {+ crate::wayland::niri::is_session()+ && crate::wayland::niri::window_for_id(xid)+ .is_ok_and(|window| window.pid == Some(pid) && window.placement.is_ok())+ }}pub(crate) fn x11_window_origin(xid: u64) -> Option<(i32, i32)> {@@ -6015,6 +6153,17 @@ fn window_to_screen_offset(pid: u32, xid: u64, title: Option<&str>) -> Option<(i}?;return (window.pid == pid).then_some((window.x, window.y));}+ if crate::wayland::niri::is_session() {+ // Only a placed floating niri window has an origin. A tiled one+ // has none, and no observed or default origin may stand in.+ let window = if xid != 0 {+ crate::wayland::niri::window_for_id(xid).ok()+ } else {+ crate::wayland::niri::window_for_pid(pid)+ }?;+ let placed = window.placement.ok()?;+ return (window.pid == Some(pid)).then_some((placed.rect.x, placed.rect.y));+ }// Native Wayland: clients can't query a window's screen origin, and// AT-SPI CoordType::Screen collapses to (0,0) on Mutter. The bundled// `org.cua.WinRects` GNOME Shell extension supplies the window's screen@@ -6482,17 +6631,19 @@ async fn gnome_window_bound_element_async(}/// Whether a snapshot of window `xid` may bind its elements to that window as-/// element tokens. Hyprland needs a correlated top-level. Elsewhere a native-/// walk's correlated top-level counts, or else the window-scope rule for an-/// uncorrelated walk: always on X11 and on compositors without a GNOME Shell,-/// and on GNOME only when the attested Shell lists exactly `xid` for the PID.+/// element tokens. Compositors addressed by exact window ids (Hyprland, niri)+/// need a correlated top-level, since an application-wide walk can include+/// another window of the same process. Elsewhere a native walk's correlated+/// top-level counts, or else the window-scope rule for an uncorrelated walk:+/// always on X11 and on compositors without a GNOME Shell, and on GNOME only+/// when the attested Shell lists exactly `xid` for the PID.pub(crate) fn snapshot_binds_window(- hyprland_session: bool,+ exact_window_ids: bool,window_scoped: bool,trusted: bool,unscoped_walk_proven: impl FnOnce() -> bool,) -> bool {- if hyprland_session {+ if exact_window_ids {return window_scoped;}(trusted && window_scoped) || unscoped_walk_proven()@@ -6500,7 +6651,8 @@ pub(crate) fn snapshot_binds_window(pub fn snapshot_binds_window_for(pid: u32, xid: u64, window_scoped: bool, trusted: bool) -> bool {snapshot_binds_window(- crate::wayland::is_wayland() && crate::wayland::hyprland::is_session(),+ crate::wayland::is_wayland()+ && (crate::wayland::hyprland::is_session() || crate::wayland::niri::is_session()),window_scoped,trusted,|| window_scope_proven(pid, xid, None),@@ -6631,7 +6783,7 @@ async fn element_bounds_for_visited(.await.flatten();if crate::wayland::is_wayland()- && crate::wayland::hyprland::is_session()+ && (crate::wayland::hyprland::is_session() || crate::wayland::niri::is_session())&& (offset.is_none() || scoped_frame.is_none()){return (Vec::new(), true);@@ -6704,7 +6856,7 @@ async fn element_bounds_for_visited(// title-bar offset). Normalize that frame to the compositor window origin// before adding the screen offset. Native GTK reports (0,0), so this is a// no-op there.- let window_frame_origin = if offset.is_some() {+ let window_frame_extents = if offset.is_some() {let frame = visited.iter().find(|node| {scoped_frame.is_none_or(|scope| node.frame_ordinal == scope)&& node.has_component@@ -6718,7 +6870,7 @@ async fn element_bounds_for_visited(Some(Ok(proxies)) => match call(proxies.component()).await {Some(Ok(component)) => {match call(component.get_extents(CoordType::Window)).await {- Some(Ok((x, y, _, _))) => Some((x, y)),+ Some(Ok(extents)) => Some(extents),_ => None,}}@@ -6732,6 +6884,33 @@ async fn element_bounds_for_visited(} else {None};+ let window_frame_origin = window_frame_extents.map(|(x, y, _, _)| (x, y));+ // niri's origin is its window geometry. A toolkit's Window extents start+ // there only when its frame is that size: GTK 3 measures from its+ // client-side shadow, Chromium from its resize margins, and where the+ // window starts inside such a frame is not on the bus. Publish no frames+ // rather than shifted ones.+ if crate::wayland::is_wayland() && crate::wayland::niri::is_session() {+ let window = bounded_blocking(move || {+ if xid != 0 {+ crate::wayland::niri::window_for_id(xid).ok()+ } else {+ crate::wayland::niri::window_for_pid(pid)+ }+ })+ .await+ .flatten();+ let window_size = window+ .and_then(|window| window.placement.ok())+ .map(|placed| (placed.rect.width, placed.rect.height));+ if !niri_frame_fits(window_frame_extents, window_size) {+ dlog!(+ "element bounds: niri window {xid} is {window_size:?} but its accessible frame \+ is {window_frame_extents:?}; publishing no frames"+ );+ return (Vec::new(), true);+ }+ }// Add compositor decorations and the embedded document origin for web// descendants only. Electron commonly contributes zero for both; WebKitGTK// under Sway needs the sum.@@ -8461,10 +8640,10 @@ mod token_scope_tests {assert!(snapshot_binds_window(false, window_scoped, trusted, || {gnome_window_scope_proven(REQUESTED_B, None, false, || false, || None)}));- // Hyprland keeps its correlated-top-level rule.+ // Hyprland and niri keep their correlated-top-level rule.assert_eq!(snapshot_binds_window(true, window_scoped, trusted, || {- panic!("Hyprland asks no Shell")+ panic!("exact-id compositors ask no Shell")}),window_scoped);diff --git a/libs/cua-driver/rust/crates/platform-linux/src/browser_consent_ui.rs b/libs/cua-driver/rust/crates/platform-linux/src/browser_consent_ui.rsindex 23d3c7525a..7b8fb7c794 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/browser_consent_ui.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/browser_consent_ui.rs@@ -8,6 +8,7 @@ use cua_driver_core::browser::{};use crate::atspi::AtspiNode;+use crate::interaction::{with_target_foreground, RestoreMode};fn normalized_text(node: &AtspiNode) -> String {[@@ -171,18 +172,6 @@ fn prove_window_owner(pid: u32, window_id: u64) -> Result<(), BrowserRefusal> {Ok(())}-fn with_target_foreground<T>(- pid: u32,- window_id: u64,- body: impl FnOnce() -> anyhow::Result<T>,-) -> anyhow::Result<T> {- if std::env::var_os("WAYLAND_DISPLAY").is_some() {- crate::wayland::with_target_foreground(pid, window_id, body)- } else {- crate::input::with_x11_foreground(window_id, 80, body)- }-}-fn exact_button_center(bounds: &[(usize, i32, i32, u32, u32)],element_index: usize,@@ -201,7 +190,7 @@ fn exact_button_center(}fn trusted_allow_click(pid: u32, window_id: u64) -> anyhow::Result<()> {- with_target_foreground(pid, window_id, || {+ with_target_foreground(pid, window_id, RestoreMode::RestorePrevious, || {let tree = crate::atspi::walk_tree(pid, window_id, None);let index = exact_allow_button(&tree.nodes, &tree.bounds).map_err(|error| anyhow::anyhow!(error.message))?@@ -210,7 +199,7 @@ fn trusted_allow_click(pid: u32, window_id: u64) -> anyhow::Result<()> {})?;let (center_x, center_y) = exact_button_center(&tree.bounds, index)?;if std::env::var_os("WAYLAND_DISPLAY").is_some() {- crate::wayland::click_focused(center_x, center_y, 1, 1)+ crate::wayland::click_focused((center_x, center_y).into(), 1, 1)} else {crate::input::send_click_xtest_desktop(center_x, center_y, 1, 1)}@@ -285,7 +274,14 @@ async fn consent_action<T: Send + 'static>(check_deadline(deadline)?;// Commit before scheduling: dropping a running blocking task cannot cancel it.let worker = action.perform(|| {+ // The worker waits on the one foreground transaction and runs under+ // no action of its own, so it binds this one: every wait it makes,+ // that one included, ends at the consent deadline instead of+ // outliving the attempt it serves.+ let bound =+ crate::input::action::Action::new(deadline.saturating_duration_since(Instant::now()));tokio::task::spawn_blocking(move || {+ let _bound = crate::input::action::bind(bound);check_deadline(deadline)?;operation().map_err(|error| {BrowserRefusal::new(diff --git a/libs/cua-driver/rust/crates/platform-linux/src/browser_setup_ui.rs b/libs/cua-driver/rust/crates/platform-linux/src/browser_setup_ui.rsindex 126cae1ff5..7dafb53c0f 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/browser_setup_ui.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/browser_setup_ui.rs@@ -12,6 +12,7 @@ use cua_driver_core::browser::{};use crate::atspi::AtspiNode;+use crate::interaction::{with_target_foreground, RestoreMode};fn field_equals(node: &AtspiNode, expected: &str) -> bool {[@@ -100,26 +101,8 @@ fn setup_not_ready_message(descriptor: &BrowserSetupDescriptor) -> String {)}-fn with_target_foreground<T>(- pid: u32,- window_id: u64,- body: impl FnOnce() -> anyhow::Result<T>,-) -> anyhow::Result<T> {- if std::env::var_os("WAYLAND_DISPLAY").is_some() {- if let Some(window) =- crate::wayland::sway_ipc::window_for_id(window_id).filter(|window| window.pid == pid)- {- crate::wayland::sway_ipc::with_focused_container(window.id, body)- } else {- crate::wayland::shell_helper::with_focused_window(pid, window_id, body)- }- } else {- crate::input::with_x11_foreground(window_id, 80, body)- }-}-fn close_tab(pid: u32, window_id: u64) -> anyhow::Result<()> {- with_target_foreground(pid, window_id, || {+ with_target_foreground(pid, window_id, RestoreMode::RestorePrevious, || {if std::env::var_os("WAYLAND_DISPLAY").is_some() {crate::wayland::hotkey(window_id, &["ctrl".to_owned(), "w".to_owned()])} else {@@ -187,6 +170,7 @@ fn trusted_setup_navigation(pid: u32,window_id: u64,descriptor: &BrowserSetupDescriptor,+ clipboard: &dyn cua_driver_core::clipboard::ClipboardBackend,) -> anyhow::Result<()> {let wayland = std::env::var_os("WAYLAND_DISPLAY").is_some();@@ -194,7 +178,7 @@ fn trusted_setup_navigation(// `ctrl+l` then focuses the address field. Both are single letters, so// neither depends on the keyboard layout the way punctuation does.let focus_omnibox = || -> anyhow::Result<()> {- with_target_foreground(pid, window_id, || {+ with_target_foreground(pid, window_id, RestoreMode::RestorePrevious, || {if wayland {crate::wayland::hotkey_focused(&["ctrl".to_owned(), "l".to_owned()])} else {@@ -203,7 +187,7 @@ fn trusted_setup_navigation(})};- with_target_foreground(pid, window_id, || {+ with_target_foreground(pid, window_id, RestoreMode::RestorePrevious, || {if wayland {crate::wayland::hotkey_focused(&["ctrl".to_owned(), "t".to_owned()])} else {@@ -242,13 +226,11 @@ fn trusted_setup_navigation(// arrives in one operation, with no per-keysym synthesis to be mistranslated// by the active layout or dropped by a virtual-keyboard seat. `ctrl+a` and// `ctrl+v` are plain letters, so they carry no layout dependency of their own.- use cua_driver_core::clipboard::ClipboardBackend;- let clipboard = crate::clipboard::LinuxClipboard::new();let restore = clipboard.read_text().ok().flatten();clipboard.write_text(descriptor.setup_url.to_owned()).map_err(|error| anyhow::anyhow!("could not stage the fixed setup URL: {error}"))?;- let paste = with_target_foreground(pid, window_id, || {+ let paste = with_target_foreground(pid, window_id, RestoreMode::RestorePrevious, || {if wayland {crate::wayland::hotkey_focused(&["ctrl".to_owned(), "a".to_owned()])?;std::thread::sleep(Duration::from_millis(60));@@ -268,7 +250,7 @@ fn trusted_setup_navigation(// Commit. Enter is the one synthesized keystroke left, and it carries no// layout dependency.- with_target_foreground(pid, window_id, || {+ with_target_foreground(pid, window_id, RestoreMode::RestorePrevious, || {if wayland {crate::wayland::hotkey_focused(&["enter".to_owned()])} else {@@ -489,6 +471,53 @@ pub fn abort_pending(pid: u32, window_id: u64, error: BrowserRefusal) -> Browser}}+/// Navigate the exact window to its fixed setup page. Under niri this route+/// has no foreground path: it focuses the window through the GNOME Shell+/// helper and writes the address field through the user's clipboard, so it+/// refuses before any input and before the clipboard is read or written.+fn open_setup_page(+ pid: u32,+ window_id: u64,+ descriptor: &'static BrowserSetupDescriptor,+ on_niri: bool,+ clipboard: &dyn cua_driver_core::clipboard::ClipboardBackend,+) -> Result<SetupUiHandle, BrowserRefusal> {+ if on_niri {+ return Err(BrowserRefusal::new(+ BrowserRefusalCode::BrowserRouteUnavailable,+ format!(+ "existing-profile {} setup has no niri foreground route: it would focus the \+ window through the GNOME Shell helper and paste the setup address through \+ the clipboard; nothing was sent and the clipboard was not touched",+ descriptor.product_name+ ),+ ));+ }+ let handle = SetupUiHandle {+ pid,+ window_id,+ descriptor,+ trusted_setup_navigation: true,+ enable_attempted: false,+ trusted_checkbox_fallback_attempted: false,+ opened_setup_page: true,+ enabled_remote_debugging: false,+ focused_setup_address_field: true,+ foregrounded_window: true,+ injected_global_input: true,+ };+ if let Err(error) = trusted_setup_navigation(pid, window_id, descriptor, clipboard) {+ return Err(handle.abort(BrowserRefusal::new(+ BrowserRefusalCode::BrowserWrongTargetRefused,+ format!(+ "could not navigate the exact {} window to its fixed setup page: {error}",+ descriptor.product_name+ ),+ )));+ }+ Ok(handle)+}+fn set_remote_debugging(pid: u32,window_id: u64,@@ -512,29 +541,13 @@ fn set_remote_debugging(injected_global_input: false,}} else {- let handle = SetupUiHandle {+ open_setup_page(pid,window_id,descriptor,- trusted_setup_navigation: true,- enable_attempted: false,- trusted_checkbox_fallback_attempted: false,- opened_setup_page: true,- enabled_remote_debugging: false,- focused_setup_address_field: true,- foregrounded_window: true,- injected_global_input: true,- };- if let Err(error) = trusted_setup_navigation(pid, window_id, descriptor) {- return Err(handle.abort(BrowserRefusal::new(- BrowserRefusalCode::BrowserWrongTargetRefused,- format!(- "could not navigate the exact {} window to its fixed setup page: {error}",- descriptor.product_name- ),- )));- }- handle+ crate::wayland::is_wayland() && crate::wayland::niri::is_session(),+ &crate::clipboard::LinuxClipboard::new(),+ )?};let deadline = Instant::now() + EXISTING_PROFILE_SETUP_READY_TIMEOUT;@@ -569,10 +582,14 @@ fn set_remote_debugging(handle.trusted_checkbox_fallback_attempted = true;handle.foregrounded_window = true;let trusted_navigation = handle.trusted_setup_navigation;- let clicked = with_target_foreground(pid, window_id, || {- std::thread::sleep(Duration::from_millis(60));- let tree = crate::atspi::walk_tree(pid, window_id, None);- let checkbox = exact_setup_checkbox(+ let clicked = with_target_foreground(+ pid,+ window_id,+ RestoreMode::RestorePrevious,+ || {+ std::thread::sleep(Duration::from_millis(60));+ let tree = crate::atspi::walk_tree(pid, window_id, None);+ let checkbox = exact_setup_checkbox(&tree.nodes,descriptor,trusted_navigation,@@ -583,49 +600,50 @@ fn set_remote_debugging("the exact Microsoft Edge remote-debugging checkbox became stale before the trusted click")})?;- if checkbox.checked == Some(desired_enabled) {- return Ok(false);- }- if checkbox.checked != Some(!desired_enabled) {- anyhow::bail!(+ if checkbox.checked == Some(desired_enabled) {+ return Ok(false);+ }+ if checkbox.checked != Some(!desired_enabled) {+ anyhow::bail!("the exact Microsoft Edge remote-debugging checkbox had an unknown state before the trusted click");- }- let index = checkbox.element_index.expect("actionable checkbox index");- // The frame this walk recorded for the checkbox, not a- // lookup of its index in another snapshot.- let (x, y, width, height) = tree- .bounds- .iter()- .find(|(bounded, ..)| *bounded == index)- .map(|&(_, x, y, width, height)| (x, y, width, height))- .ok_or_else(|| {- anyhow::anyhow!(- "the exact Microsoft Edge remote-debugging checkbox had no screen bounds"- )- })?;- if width <= 1 || height <= 1 {- anyhow::bail!(+ }+ let index = checkbox.element_index.expect("actionable checkbox index");+ // The frame this walk recorded for the checkbox, not a+ // lookup of its index in another snapshot.+ let (x, y, width, height) = tree+ .bounds+ .iter()+ .find(|(bounded, ..)| *bounded == index)+ .map(|&(_, x, y, width, height)| (x, y, width, height))+ .ok_or_else(|| {+ anyhow::anyhow!(+ "the exact Microsoft Edge remote-debugging checkbox had no screen bounds"+ )+ })?;+ if width <= 1 || height <= 1 {+ anyhow::bail!("the exact Microsoft Edge remote-debugging checkbox had empty screen bounds");- }- let center_x = x- .checked_add(i32::try_from(width / 2)?)- .ok_or_else(|| anyhow::anyhow!("checkbox center x overflowed"))?;- let center_y = y- .checked_add(i32::try_from(height / 2)?)- .ok_or_else(|| anyhow::anyhow!("checkbox center y overflowed"))?;- if std::env::var_os("WAYLAND_DISPLAY").is_some() {- // AT-SPI screen bounds are layout coordinates, as in- // browser_consent_ui's click_focused path, not- // desktop-frame points, so no frame conversion applies.- let space = crate::wayland::DesktopInputSpace::default();- crate::wayland::click_desktop(&space, center_x, center_y, 1, 1)?;- } else {- crate::input::send_click_xtest_desktop(center_x, center_y, 1, 1)?;- }- Ok(true)- });+ }+ let center_x = x+ .checked_add(i32::try_from(width / 2)?)+ .ok_or_else(|| anyhow::anyhow!("checkbox center x overflowed"))?;+ let center_y = y+ .checked_add(i32::try_from(height / 2)?)+ .ok_or_else(|| anyhow::anyhow!("checkbox center y overflowed"))?;+ if std::env::var_os("WAYLAND_DISPLAY").is_some() {+ // AT-SPI screen bounds are layout coordinates, as in+ // browser_consent_ui's click_focused path, not+ // desktop-frame points, so no frame conversion applies.+ let space = crate::wayland::DesktopInputSpace::default();+ crate::wayland::click_desktop(&space, center_x, center_y, 1, 1)?;+ } else {+ crate::input::send_click_xtest_desktop(center_x, center_y, 1, 1)?;+ }+ Ok(true)+ },+ );match clicked {Ok(injected) => handle.injected_global_input |= injected,Err(error) => {@@ -769,6 +787,50 @@ mod tests {);}+ /// Counts every clipboard call; the niri refusal must make none.+ #[derive(Default)]+ struct WatchedClipboard {+ calls: std::sync::atomic::AtomicUsize,+ }++ impl cua_driver_core::clipboard::ClipboardBackend for WatchedClipboard {+ fn available_formats(&self) -> Result<Vec<String>, String> {+ self.calls.fetch_add(1, std::sync::atomic::Ordering::SeqCst);+ Ok(Vec::new())+ }+ fn read_text(&self) -> Result<Option<String>, String> {+ self.calls.fetch_add(1, std::sync::atomic::Ordering::SeqCst);+ Ok(None)+ }+ fn write_text(&self, _: String) -> Result<(), String> {+ self.calls.fetch_add(1, std::sync::atomic::Ordering::SeqCst);+ Ok(())+ }+ fn write_image(&self, _: &str) -> Result<(), String> {+ self.calls.fetch_add(1, std::sync::atomic::Ordering::SeqCst);+ Ok(())+ }+ fn write_file_url(&self, _: &str) -> Result<(), String> {+ self.calls.fetch_add(1, std::sync::atomic::Ordering::SeqCst);+ Ok(())+ }+ }++ #[test]+ fn niri_refuses_setup_navigation_without_touching_the_clipboard() {+ let clipboard = WatchedClipboard::default();+ let Err(refused) = open_setup_page(1, 2, descriptor(), true, &clipboard) else {+ panic!("setup navigation ran under niri");+ };+ assert_eq!(refused.code, BrowserRefusalCode::BrowserRouteUnavailable);+ assert!(+ refused.message.contains("no niri foreground route"),+ "{}",+ refused.message+ );+ assert_eq!(clipboard.calls.load(std::sync::atomic::Ordering::SeqCst), 0);+ }+#[test]fn setup_timeout_explains_linux_renderer_accessibility_precondition() {let message = setup_not_ready_message(descriptor());diff --git a/libs/cua-driver/rust/crates/platform-linux/src/capture_action_frame.rs b/libs/cua-driver/rust/crates/platform-linux/src/capture_action_frame.rsindex 4ca394df82..4a740a2599 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/capture_action_frame.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/capture_action_frame.rs@@ -17,6 +17,11 @@ fn publish(native_action_dimensions: (u32, u32),screenshot_to_action: ScreenshotToActionTransform,) -> anyhow::Result<String> {+ #[cfg(target_os = "linux")]+ crate::interaction::admit_lock(+ crate::interaction::policy::Scope::Capture,+ crate::interaction::facts::Fact::Unknown("capture needs no focus".into()),+ )?;let binding = service.binding_from_args(args)?;let capture_id = service.publish(CapturePublication {png_bytes: png_bytes.to_vec(),diff --git a/libs/cua-driver/rust/crates/platform-linux/src/health_report.rs b/libs/cua-driver/rust/crates/platform-linux/src/health_report.rsindex fbdf341328..0a9676e7ac 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/health_report.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/health_report.rs@@ -170,8 +170,8 @@ async fn check_ax_capability() -> CheckEntry {async fn check_screen_capture_capability() -> CheckEntry {// Native Wayland: capture flows through a cascade — wlroots screencopy- // (sway / labwc / kwin 5.27+ / hyprland) → ext-image-copy-capture-v1- // (sway 1.10+ / labwc 0.8+ / niri / KDE 6.2+ / GNOME 47+) →+ // (sway / labwc / kwin 5.27+ / hyprland / niri) →+ // ext-image-copy-capture-v1 (sway 1.10+ / labwc 0.8+ / KDE 6.2+ / GNOME 47+) →// xdg-desktop-portal Screenshot (GNOME / KDE / COSMIC + portal backend).// Report which tier is reachable so users on mutter / kwin don't see a// misleading "screen capture will fail" just because wlroots isn't@@ -299,12 +299,20 @@ async fn check_wayland_backend() -> CheckEntry {} else {false};- classify_wayland_backend(+ let mut entry = classify_wayland_backend(&snap,portal_input_enabled(),remote_desktop_portal_reachable,target_activation_available(),- )+ );+ if let Some(note) = tokio::task::spawn_blocking(compositor_note)+ .await+ .ok()+ .flatten()+ {+ entry.message = format!("{} {note}", entry.message);+ }+ entry}fn classify_wayland_backend(@@ -521,6 +529,16 @@ fn target_activation_available() -> bool {false}+#[cfg(target_os = "linux")]+fn compositor_note() -> Option<String> {+ crate::wayland::niri::health_note()+}++#[cfg(not(target_os = "linux"))]+fn compositor_note() -> Option<String> {+ None+}+#[cfg(target_os = "linux")]type WaylandManagers = crate::wayland::WaylandManagers;diff --git a/libs/cua-driver/rust/crates/platform-linux/src/input/action.rs b/libs/cua-driver/rust/crates/platform-linux/src/input/action.rsnew file mode 100644index 0000000000..0e6d2ee5f0--- /dev/null+++ b/libs/cua-driver/rust/crates/platform-linux/src/input/action.rs@@ -0,0 +1,413 @@+//! The tool call a blocking input worker serves.+//!+//! A tool call stops waiting for its blocking worker at its deadline, but+//! the worker cannot be cancelled: it keeps running on the blocking pool.+//! Two mechanisms limit what it does afterwards.+//!+//! Send admission: an input route that reads the [`Action`] bound to its+//! thread hands no further event to the compositor once the call is over.+//! Releasing a held button or key is cleanup and still goes out. A route+//! that does not read the action is not stopped: a `wtype` child already+//! running keeps sending until it exits.+//!+//! Successor isolation: the action keeps the call's desktop lease until its+//! last holder drops, so whatever still goes out, cleanup or such a child,+//! ends before the next desktop action starts.++use std::cell::RefCell;+use std::os::fd::{AsFd, AsRawFd, OwnedFd};+use std::sync::{Arc, Mutex};+use std::time::{Duration, Instant};++use anyhow::{bail, Result};+use cua_driver_core::tool::DesktopLease;++/// One tool call: its deadline, whether its caller stopped waiting, and+/// the desktop lease of the task that started it.+pub struct Action {+ deadline: Instant,+ abandoned: Mutex<bool>,+ fenced: Mutex<(bool, Vec<OwnedFd>)>,+ _lease: Option<DesktopLease>,+}++impl Action {+ pub fn new(budget: Duration) -> Arc<Self> {+ Arc::new(Self {+ deadline: Instant::now() + budget,+ abandoned: Mutex::new(false),+ fenced: Mutex::new((false, Vec::new())),+ _lease: DesktopLease::current(),+ })+ }++ pub fn deadline(&self) -> Instant {+ self.deadline+ }++ /// Run `send`, which hands input events to the compositor, unless the+ /// call is past its deadline or abandoned. [`Action::abandon`] waits for+ /// a `send` under way, so nothing goes out once it has returned. `send`+ /// must not block: queue the events and flush them, then wait for the+ /// compositor outside.+ pub fn admit<T>(&self, send: impl FnOnce() -> T) -> Result<T> {+ let abandoned = self+ .abandoned+ .lock()+ .unwrap_or_else(std::sync::PoisonError::into_inner);+ if *abandoned || Instant::now() >= self.deadline {+ bail!(+ "{}: the call ran out of time, so no further input was sent",+ super::foreground::CODE_TIMEOUT+ );+ }+ Ok(send())+ }++ /// The caller stopped waiting: refuse every later [`Action::admit`].+ pub fn abandon(&self) {+ let mut fenced = self+ .fenced+ .lock()+ .unwrap_or_else(std::sync::PoisonError::into_inner);+ fenced.0 = true;+ fenced.1.iter().for_each(shut_down);+ drop(fenced);+ *self+ .abandoned+ .lock()+ .unwrap_or_else(std::sync::PoisonError::into_inner) = true;+ }++ /// Whether the caller stopped waiting, so fenced sockets are shut.+ pub fn is_abandoned(&self) -> bool {+ self.fenced+ .lock()+ .unwrap_or_else(std::sync::PoisonError::into_inner)+ .0+ }++ /// Shut `socket` down once this action is abandoned, before waiting for a+ /// send under way, so nothing queued on it is written later.+ pub fn fence(&self, socket: impl AsFd) -> std::io::Result<()> {+ let socket = socket.as_fd().try_clone_to_owned()?;+ let mut fenced = self+ .fenced+ .lock()+ .unwrap_or_else(std::sync::PoisonError::into_inner);+ if fenced.0 {+ shut_down(&socket);+ }+ fenced.1.push(socket);+ Ok(())+ }++ /// Abandon this action when the returned guard drops. Hold it where the+ /// caller waits, so a caller that is itself dropped (a cancelled future)+ /// abandons the action as one that timed out does.+ pub fn abandon_on_drop(self: &Arc<Self>) -> AbandonOnDrop {+ AbandonOnDrop(self.clone())+ }+}++fn shut_down(socket: &OwnedFd) {+ unsafe { libc::shutdown(socket.as_raw_fd(), libc::SHUT_RDWR) };+}++/// Abandons its [`Action`] when dropped; see [`Action::abandon_on_drop`].+pub struct AbandonOnDrop(Arc<Action>);++impl Drop for AbandonOnDrop {+ fn drop(&mut self) {+ self.0.abandon();+ }+}++thread_local! {+ static CURRENT: RefCell<Option<Arc<Action>>> = const { RefCell::new(None) };+}++/// The action bound to this thread, if its worker was started for one.+pub fn current() -> Option<Arc<Action>> {+ CURRENT.with(|current| current.borrow().clone())+}++/// Restores the previous binding when dropped.+pub struct Bound(Option<Arc<Action>>);++impl Drop for Bound {+ fn drop(&mut self) {+ let previous = self.0.take();+ CURRENT.with(|current| *current.borrow_mut() = previous);+ }+}++/// Bind `action` to this thread for the input routes it runs.+pub fn bind(action: Arc<Action>) -> Bound {+ Bound(CURRENT.with(|current| current.replace(Some(action))))+}++/// Admit `send` under the action bound to this thread; with none, run it.+pub fn admit<T>(send: impl FnOnce() -> T) -> Result<T> {+ match current() {+ Some(action) => action.admit(send),+ None => Ok(send()),+ }+}++/// Fence `socket` under the action bound to this thread; with none, do nothing.+pub fn fence(socket: impl AsFd) -> std::io::Result<()> {+ current().map_or(Ok(()), |action| action.fence(socket))+}++/// `spawn_blocking` with a hard deadline. A wedged X server (or a client+/// holding a server grab) must surface as a structured `foreground_timeout`+/// error instead of a tool call that never returns. The blocking task is not+/// cancellable; it is left to finish (or leak) on the blocking pool, with its+/// [`Action`] abandoned so its input routes send nothing more. The action+/// keeps the call's desktop lease, so the next desktop action waits until+/// the worker has stopped. Started from a thread already bound to an action, the worker+/// runs under that action, within what is left of its deadline, and its+/// timeout abandons that action too.+pub async fn spawn_blocking_bounded<T: Send + 'static>(+ label: &'static str,+ budget: Duration,+ f: impl FnOnce() -> Result<T> + Send + 'static,+) -> std::result::Result<Result<T>, tokio::task::JoinError> {+ let outer = current();+ let action = outer.clone().unwrap_or_else(|| Action::new(budget));+ let budget = budget.min(action.deadline.saturating_duration_since(Instant::now()));+ let _abandon = outer.is_none().then(|| action.abandon_on_drop());+ let worker = {+ let action = action.clone();+ move || {+ let _bound = bind(action);+ f()+ }+ };+ match tokio::time::timeout(budget, tokio::task::spawn_blocking(worker)).await {+ Ok(joined) => joined,+ Err(_elapsed) => {+ action.abandon();+ Ok(Err(anyhow::anyhow!(+ "{}: {label} did not complete within {budget:?}; the X server or the target \+ application may be wedged. Take a screenshot before retrying",+ super::foreground::CODE_TIMEOUT+ )))+ }+ }+}++#[cfg(test)]+mod tests {+ use super::*;++ #[test]+ fn an_abandoned_or_expired_action_admits_nothing() {+ let action = Action::new(Duration::from_secs(60));+ assert_eq!(action.admit(|| 1).unwrap(), 1);+ action.abandon();+ let refused = action.admit(|| panic!("sent after abandon")).unwrap_err();+ assert_eq!(+ crate::input::foreground::error_code(&refused),+ Some(crate::input::foreground::CODE_TIMEOUT)+ );++ let expired = Action::new(Duration::ZERO);+ expired+ .admit(|| panic!("sent after the deadline"))+ .unwrap_err();+ }++ #[test]+ fn abandon_waits_for_a_send_under_way() {+ let action = Action::new(Duration::from_secs(60));+ let (started, sending) = std::sync::mpsc::channel();+ let worker = {+ let action = action.clone();+ std::thread::spawn(move || {+ action+ .admit(|| {+ started.send(()).unwrap();+ std::thread::sleep(Duration::from_millis(200));+ Instant::now()+ })+ .unwrap()+ })+ };+ sending.recv().unwrap();+ action.abandon();+ let abandoned_at = Instant::now();+ assert!(worker.join().unwrap() <= abandoned_at);+ }++ #[test]+ fn abandon_shuts_a_fenced_socket_without_waiting_for_a_wedged_send() {+ use std::io::Write;+ let (mut socket, peer) = std::os::unix::net::UnixStream::pair().unwrap();+ let action = Action::new(Duration::from_secs(60));+ action.fence(&socket).unwrap();+ let (started, sending) = std::sync::mpsc::channel();+ let worker = {+ let action = action.clone();+ std::thread::spawn(move || {+ action.admit(|| {+ started.send(()).unwrap();+ loop {+ if let Err(error) = socket.write_all(&[0; 65536]) {+ return error;+ }+ }+ })+ })+ };+ sending.recv().unwrap();+ let abandoning = Instant::now();+ action.abandon();+ assert!(abandoning.elapsed() < Duration::from_secs(5));+ assert!(worker.join().unwrap().is_ok());+ drop(peer);+ }++ #[test]+ fn a_socket_fenced_after_abandon_is_shut_at_once() {+ use std::io::{Read, Write};+ let (mut socket, mut peer) = std::os::unix::net::UnixStream::pair().unwrap();+ let action = Action::new(Duration::from_secs(60));+ action.abandon();+ action.fence(&socket).unwrap();+ assert!(socket.write_all(b"late").is_err());+ assert_eq!(peer.read(&mut [0; 4]).unwrap(), 0);+ }++ fn sending_worker(sent: Arc<std::sync::atomic::AtomicUsize>) -> impl FnOnce() -> Result<()> {+ move || {+ for _ in 0..50 {+ admit(|| sent.fetch_add(1, std::sync::atomic::Ordering::SeqCst))?;+ std::thread::sleep(Duration::from_millis(10));+ }+ Ok(())+ }+ }++ #[tokio::test]+ async fn a_timed_out_input_worker_sends_nothing_more() {+ use std::sync::atomic::{AtomicUsize, Ordering};+ let sent = Arc::new(AtomicUsize::new(0));+ let result = spawn_blocking_bounded(+ "test input",+ Duration::from_millis(100),+ sending_worker(sent.clone()),+ )+ .await+ .unwrap();+ assert!(result.is_err());+ let at_timeout = sent.load(Ordering::SeqCst);+ tokio::time::sleep(Duration::from_millis(300)).await;+ assert_eq!(sent.load(Ordering::SeqCst), at_timeout);+ }++ #[tokio::test]+ async fn an_input_worker_whose_caller_was_dropped_sends_nothing_more() {+ use std::sync::atomic::{AtomicUsize, Ordering};+ let sent = Arc::new(AtomicUsize::new(0));+ let call = spawn_blocking_bounded(+ "test input",+ Duration::from_secs(60),+ sending_worker(sent.clone()),+ );+ assert!(tokio::time::timeout(Duration::from_millis(100), call)+ .await+ .is_err());+ let at_drop = sent.load(Ordering::SeqCst);+ tokio::time::sleep(Duration::from_millis(300)).await;+ assert!(sent.load(Ordering::SeqCst) <= at_drop + 1);+ }++ #[test]+ fn a_nested_worker_keeps_the_outer_deadline_and_its_timeout_ends_the_outer_call() {+ let outer = Action::new(Duration::from_millis(100));+ let _bound = bind(outer.clone());+ let runtime = tokio::runtime::Builder::new_current_thread()+ .enable_time()+ .build()+ .unwrap();+ let started = Instant::now();+ let inner = runtime+ .block_on(spawn_blocking_bounded(+ "inner",+ Duration::from_secs(60),+ || {+ std::thread::sleep(Duration::from_millis(400));+ Ok(())+ },+ ))+ .unwrap();+ assert!(inner.is_err());+ assert!(started.elapsed() < Duration::from_secs(5));+ admit(|| panic!("the outer call sent after its inner worker timed out")).unwrap_err();+ }++ #[tokio::test(flavor = "multi_thread", worker_threads = 2)]+ async fn cleanup_that_outlives_a_timed_out_worker_ends_before_the_next_call() {+ use cua_driver_core::tool::DesktopCall;+ let events = Arc::new(Mutex::new(Vec::new()));+ let cleanup = Arc::new(Mutex::new(None));+ let call_1 = {+ let events = events.clone();+ let cleanup = cleanup.clone();+ DesktopCall::acquire()+ .await+ .unwrap()+ .scope(spawn_blocking_bounded(+ "call 1",+ Duration::from_millis(50),+ move || {+ std::thread::sleep(Duration::from_millis(100));+ let action = current().unwrap();+ *cleanup.lock().unwrap() = Some(std::thread::spawn(move || {+ let _desktop = action;+ std::thread::sleep(Duration::from_millis(300));+ events.lock().unwrap().push("call 1 released its press");+ }));+ Ok(())+ },+ ))+ .await+ .unwrap()+ };+ assert!(call_1.is_err());+ DesktopCall::acquire()+ .await+ .unwrap()+ .scope(async { events.lock().unwrap().push("call 2 pressed") })+ .await;+ if let Some(cleanup) = cleanup.lock().unwrap().take() {+ cleanup.join().unwrap();+ }+ assert_eq!(+ *events.lock().unwrap(),+ ["call 1 released its press", "call 2 pressed"]+ );+ }++ #[test]+ fn a_nested_worker_shares_the_outer_action() {+ let outer = Action::new(Duration::from_secs(60));+ let _bound = bind(outer.clone());+ let runtime = tokio::runtime::Builder::new_current_thread()+ .enable_time()+ .build()+ .unwrap();+ let inner = runtime+ .block_on(spawn_blocking_bounded(+ "inner",+ Duration::from_secs(60),+ || Ok(current().unwrap()),+ ))+ .unwrap()+ .unwrap();+ assert!(Arc::ptr_eq(&inner, &outer));+ assert_eq!(admit(|| 1).unwrap(), 1);+ }+}diff --git a/libs/cua-driver/rust/crates/platform-linux/src/input/focus_guard.rs b/libs/cua-driver/rust/crates/platform-linux/src/input/focus_guard.rsindex 4e0581d9e5..de8f081c92 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/input/focus_guard.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/input/focus_guard.rs@@ -23,6 +23,7 @@//! hidden: a toolkit keyboard grab held by an open menu cannot be undone//! without closing the menu, so it is surfaced as `grab_held_by`.+use crate::interaction::policy::{guard_restore, GuardRestore};use anyhow::{anyhow, Result};use std::collections::HashSet;use std::time::{Duration, Instant};@@ -45,11 +46,6 @@ const SETTLE_WATCH: Duration = Duration::from_millis(220);/// its own and there is nothing to wait for.const SETTLE_WATCH_NEW_WINDOW: Duration = Duration::from_millis(900);const SETTLE_POLL: Duration = Duration::from_millis(30);-/// Bound on the restore loop: re-activation, verification, one re-send.-// Together with the extended settle watch this bounds the guard at-// ~1.5 s worst case (menubar clicks were spending 4-9 s here while a popup's-// keyboard grab kept every re-activation from sticking).-const RESTORE_BUDGET: Duration = Duration::from_millis(600);const RESTORE_POLL: Duration = Duration::from_millis(50);/// Consecutive stable polls before the restore is called done.const STABLE_POLLS: u32 = 3;@@ -289,34 +285,6 @@ impl X {}}-/// Where a focus move went, relative to the action's target application.-#[derive(Clone, Copy, Debug, PartialEq, Eq)]-pub enum FocusMove {- /// The target app owned the focus before and still does: it opened one- /// of its own windows (a dialog). Not an intrusion; leave it.- SameAppDialog,- /// The focus left the user's application (or arrived in the target from- /// another app): restore it.- OtherApp,-}--/// The same-app rule: a move is the app's own dialog only when the target-/// pid owned the focus before the action *and* owns it now. A move from a-/// decoy (another pid) into the target's dialog is a steal, as is a move-/// from the target to anything else.-pub fn classify_focus_move(- target_pid: Option<u32>,- previous_owner: Option<u32>,- new_owner: Option<u32>,-) -> FocusMove {- match (target_pid, previous_owner, new_owner) {- (Some(target), Some(previous), Some(new)) if previous == target && new == target => {- FocusMove::SameAppDialog- }- _ => FocusMove::OtherApp,- }-}-/// What a background action must not change.#[derive(Clone, Debug, PartialEq, Eq)]pub struct FocusSnapshot {@@ -585,16 +553,15 @@ impl FocusSnapshot {return FocusGuardReport::default();};let mut report = FocusGuardReport::default();- // The target app owns the focus: whatever it maps next is its own.- let target_owns_focus = target_pid.is_some() && self.previous_owner == target_pid;// Settle watch: stop at the first observed change. A new top-level// (dialog being mapped) extends the watch when the focus belongs to// another application, since the WM focuses it only once mapped and// that steal must be undone; when the target app already owns the// focus the new window is its own and the watch ends at once.+ let action_deadline = crate::interaction::operation_deadline(SETTLE_WATCH_NEW_WINDOW);let mut watch_until = if settle_watch {- started + SETTLE_WATCH+ (started + SETTLE_WATCH).min(action_deadline)} else {started};@@ -604,16 +571,18 @@ impl FocusSnapshot {let mut own_window = self.own_new_window(&x, target_pid, &new_clients);if !settle_watch && !new_clients.is_empty() && own_window.is_none() {extended = true;- watch_until = started + SETTLE_WATCH_NEW_WINDOW;+ watch_until = (started + SETTLE_WATCH_NEW_WINDOW).min(action_deadline);}while changes.is_empty() && own_window.is_none() && Instant::now() < watch_until {- std::thread::sleep(SETTLE_POLL);+ std::thread::sleep(+ SETTLE_POLL.min(watch_until.saturating_duration_since(Instant::now())),+ );changes = self.diff(&x);new_clients = self.new_clients(&x);own_window = self.own_new_window(&x, target_pid, &new_clients);if !extended && !new_clients.is_empty() && own_window.is_none() {extended = true;- watch_until = started + SETTLE_WATCH_NEW_WINDOW;+ watch_until = (started + SETTLE_WATCH_NEW_WINDOW).min(action_deadline);}}@@ -648,43 +617,42 @@ impl FocusSnapshot {// The action closed the window that held the focus (Escape / OK on// a dialog): nothing to restore to, the WM already moved on.let focus_window = self.current_focus_window(&x);- if let Some(previous) = self+ let closed = self.previous_focus_window()- .filter(|w| !x.window_viewable(*w))- {- report.closed_window = Some(u64::from(previous));- report.same_app_window = focus_window.map(|w| SameAppWindow {- window: u64::from(w),- title: x.window_title(w),- focused: true,- });- report.elapsed_ms = started.elapsed().as_millis() as u64;- return report;- }-- // Same-app rule: the focus stayed inside the target application- // (it opened a dialog). Leave it and say so.+ .filter(|w| !x.window_viewable(*w));let new_owner = focus_window.and_then(|w| x.owner_pid(w));- if target_owns_focus- && classify_focus_move(target_pid, self.previous_owner, new_owner)- == FocusMove::SameAppDialog- {- let window = focus_window.unwrap_or(0);- report.same_app_window = Some(SameAppWindow {- window: u64::from(window),- title: x.window_title(window),- focused: true,- });- report.elapsed_ms = started.elapsed().as_millis() as u64;- return report;- }-- // A popup (menu, combo list) that the action opened holds a keyboard- // grab: re-activating the previous window cannot take effect until it- // closes, and closing it would undo the action. Report and return.- if report.grab_held_by.is_some() {- report.elapsed_ms = started.elapsed().as_millis() as u64;- return report;+ match guard_restore(+ closed.is_some(),+ target_pid,+ self.previous_owner,+ new_owner,+ report.grab_held_by.is_some(),+ ) {+ GuardRestore::PriorClosed => {+ report.closed_window = closed.map(u64::from);+ report.same_app_window = focus_window.map(|w| SameAppWindow {+ window: u64::from(w),+ title: x.window_title(w),+ focused: true,+ });+ report.elapsed_ms = started.elapsed().as_millis() as u64;+ return report;+ }+ GuardRestore::SameAppDialog => {+ let window = focus_window.unwrap_or(0);+ report.same_app_window = Some(SameAppWindow {+ window: u64::from(window),+ title: x.window_title(window),+ focused: true,+ });+ report.elapsed_ms = started.elapsed().as_millis() as u64;+ return report;+ }+ GuardRestore::GrabHeld => {+ report.elapsed_ms = started.elapsed().as_millis() as u64;+ return report;+ }+ GuardRestore::Restore => {}}// Restore. EWMH re-activation makes the WM restore stacking and its@@ -692,10 +660,12 @@ impl FocusSnapshot {// treat _NET_ACTIVE_WINDOW as raise-only and the no-WM case.let mut stable = 0;let mut resent = 0;- let deadline = started + RESTORE_BUDGET;+ let deadline = Instant::now() + crate::interaction::CLEANUP_CAP;self.reassert(&x);while Instant::now() < deadline {- std::thread::sleep(RESTORE_POLL);+ std::thread::sleep(+ RESTORE_POLL.min(deadline.saturating_duration_since(Instant::now())),+ );let now = self.diff(&x);if now.is_empty() {stable += 1;@@ -908,39 +878,6 @@ mod tests {assert!(!report.summary().contains("not_restored"));}- #[test]- fn same_app_rule_only_when_the_target_owned_the_focus_before_and_after() {- // The target app was active and opened its own dialog: leave it.- assert_eq!(- classify_focus_move(Some(7), Some(7), Some(7)),- FocusMove::SameAppDialog- );- // Decoy case: another app owned the focus and the target's dialog- // stole it -> restore.- assert_eq!(- classify_focus_move(Some(7), Some(3), Some(7)),- FocusMove::OtherApp- );- // The target was active and something else took the focus -> restore.- assert_eq!(- classify_focus_move(Some(7), Some(7), Some(3)),- FocusMove::OtherApp- );- // Unknown owners never qualify.- assert_eq!(- classify_focus_move(Some(7), None, Some(7)),- FocusMove::OtherApp- );- assert_eq!(- classify_focus_move(Some(7), Some(7), None),- FocusMove::OtherApp- );- assert_eq!(- classify_focus_move(None, Some(7), Some(7)),- FocusMove::OtherApp- );- }-#[test]fn capture_without_display_degrades_to_none() {let display = crate::test_env::unreachable_x11_display();@@ -977,6 +914,9 @@ mod grab_tests {#[test]fn restore_and_settle_budgets_bound_the_guard_under_1500ms() {- assert!(SETTLE_WATCH_NEW_WINDOW + RESTORE_BUDGET <= Duration::from_millis(1500));+ assert!(+ SETTLE_WATCH_NEW_WINDOW + crate::interaction::CLEANUP_CAP+ <= Duration::from_millis(1500)+ );}}diff --git a/libs/cua-driver/rust/crates/platform-linux/src/input/foreground.rs b/libs/cua-driver/rust/crates/platform-linux/src/input/foreground.rsindex 2e45a4b7d4..ad1e49659a 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/input/foreground.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/input/foreground.rs@@ -1,4 +1,5 @@-//! X11 foreground transaction (`delivery_mode:"foreground"`).+//! X11 foreground steps (`delivery_mode:"foreground"`) that+//! [`crate::interaction::with_x11_foreground`] runs.//!//! A foreground action behaves like a user at the keyboard: activate + raise the//! target toplevel, wait until the window manager confirms that it is the@@ -16,6 +17,7 @@//! is used: protocol errors come back as `Result`s rather than going through//! Xlib's process-wide error handler.+use crate::interaction::operation_deadline;use anyhow::{anyhow, Result};use cua_driver_core::window_observation::WindowObservationBounds;use std::sync::mpsc;@@ -69,13 +71,6 @@ impl ForegroundOptions {}}- /// Legacy `settle_ms` hint: folded into the pointer budget (never shorter).- pub fn from_settle_hint(settle_ms: u64) -> Self {- let mut opts = Self::pointer();- opts.settle = opts.settle.max(Duration::from_millis(settle_ms));- opts- }-/// Name the process the target window belongs to (see `target_pid`).pub fn for_pid(mut self, pid: u32) -> Self {self.target_pid = Some(pid);@@ -285,15 +280,15 @@ fn wait_for_window_change(pid: Option<u32>,before: &[WindowEntry],bounds: WindowObservationBounds,+ deadline: Instant,) -> (Vec<WindowEntry>, Option<WindowChange>) {if bounds.skips_observation() {return (pid_window_set(pid), None);}- let deadline = Instant::now() + bounds.timeout;loop {let after = pid_window_set(pid);if let Some(change) = describe_window_change(before, &after) {- std::thread::sleep(WINDOW_CHANGE_TITLE_GRACE);+ std::thread::sleep(remaining(deadline).min(WINDOW_CHANGE_TITLE_GRACE));let settled = pid_window_set(pid);return match describe_window_change(before, &settled) {Some(change) => (settled, Some(change)),@@ -303,7 +298,7 @@ fn wait_for_window_change(if Instant::now() >= deadline {return (after, None);}- std::thread::sleep(bounds.poll);+ std::thread::sleep(remaining(deadline).min(bounds.poll));}}@@ -337,7 +332,7 @@ pub fn error_code(error: &anyhow::Error) -> Option<&'static str> {/// late result from a leaked thread is silently dropped instead of crashing/// the caller that already moved on.fn run_with_deadline<T: Send + 'static>(- deadline: Duration,+ deadline: Instant,f: impl FnOnce() -> T + Send + 'static,) -> Option<T> {let (tx, rx) = mpsc::channel();@@ -347,7 +342,11 @@ fn run_with_deadline<T: Send + 'static>(let _ = tx.send(f());}).ok()?;- rx.recv_timeout(deadline).ok()+ rx.recv_timeout(remaining(deadline)).ok()+}++fn remaining(deadline: Instant) -> Duration {+ deadline.saturating_duration_since(Instant::now())}struct X11 {@@ -373,6 +372,7 @@ impl X11 {let (conn, screen) = RustConnection::connect(None).map_err(|e| {anyhow!("{CODE_UNAVAILABLE}: cannot open DISPLAY to verify X11 input focus: {e}")})?;+ super::action::fence(conn.stream())?;let root = conn.setup().roots[screen].root;let resource_id_mask = conn.setup().resource_id_mask;let net_active_window = conn@@ -439,8 +439,9 @@ impl X11 {/// Current server time via the PropertyNotify round-trip. `_NET_ACTIVE_WINDOW`/// stamped `CurrentTime` (0) loses to focus-stealing prevention whenever any- /// newer user input exists. Bounded: falls back to 0 after 300 ms.- fn server_time(&self) -> u32 {+ /// newer user input exists. Bounded: falls back to 0 after 300 ms or at+ /// `deadline`.+ fn server_time(&self, deadline: Instant) -> u32 {let probe = match self.conn.generate_id() {Ok(id) => id,Err(_) => return 0,@@ -476,7 +477,7 @@ impl X11 {&[0u8],);let _ = self.conn.flush();- let deadline = Instant::now() + Duration::from_millis(300);+ let deadline = deadline.min(Instant::now() + Duration::from_millis(300));let mut time = 0;while Instant::now() < deadline {match self.conn.poll_for_event() {@@ -485,7 +486,7 @@ impl X11 {break;}Ok(Some(_)) => continue,- Ok(None) => std::thread::sleep(Duration::from_millis(5)),+ Ok(None) => std::thread::sleep(remaining(deadline).min(Duration::from_millis(5))),Err(_) => break,}}@@ -494,17 +495,19 @@ impl X11 {time}- fn activate(&self, target: Window, prior: Window) -> Result<()> {- let time = self.server_time();+ fn activate(&self, target: Window, prior: Window, deadline: Instant) -> Result<()> {+ let time = self.server_time(deadline);let event =ClientMessageEvent::new(32, target, self.net_active_window, [2, time, prior, 0, 0]);debug_assert_eq!(event.response_type, CLIENT_MESSAGE_EVENT);- self.conn.send_event(- false,- self.root,- EventMask::SUBSTRUCTURE_REDIRECT | EventMask::SUBSTRUCTURE_NOTIFY,- event,- )?;+ super::action::admit(|| {+ self.conn.send_event(+ false,+ self.root,+ EventMask::SUBSTRUCTURE_REDIRECT | EventMask::SUBSTRUCTURE_NOTIFY,+ event,+ )+ })??;// Set the core focus explicitly as well (what `xdotool windowactivate`// does): WMs with focus-stealing prevention honour `_NET_ACTIVE_WINDOW`// as raise-only. BadMatch on a not-yet-viewable window is expected and@@ -617,14 +620,21 @@ struct ConfirmOutcome {focus_holder: Option<String>,}-fn confirm_phase(target: Window, settle: Duration) -> Result<ConfirmOutcome> {+fn confirmed(target: Window, active: Option<Window>, focus_within: bool) -> bool {+ crate::interaction::policy::focused(+ &target,+ &crate::interaction::facts::Fact::Known(active.filter(|_| focus_within)),+ )+}++fn confirm_phase(target: Window, deadline: Instant) -> Result<ConfirmOutcome> {let x = X11::open()?;let start = Instant::now();let prior = x.active_window().unwrap_or(0);let mut retried = false;// Already active and focused: leave the window (and any open menu / grab)// untouched — re-activating is what popped GTK menus back down.- if prior == target && x.focus_is_within(target) {+ if confirmed(target, Some(prior), x.focus_is_within(target)) {return Ok(ConfirmOutcome {confirmed: true,already_active: true,@@ -635,14 +645,12 @@ fn confirm_phase(target: Window, settle: Duration) -> Result<ConfirmOutcome> {focus_holder: None,});}- x.activate(target, prior)?;- let deadline = start + settle;- let retry_at = start + settle.mul_f32(0.4);+ x.activate(target, prior, deadline)?;+ let retry_at = start + deadline.saturating_duration_since(start).mul_f32(0.4);let mut retry_pending = true;loop {- let active = x.active_window() == Some(target);let focus_within = x.focus_is_within(target);- if active && focus_within {+ if confirmed(target, x.active_window(), focus_within) {return Ok(ConfirmOutcome {confirmed: true,already_active: false,@@ -668,9 +676,9 @@ fn confirm_phase(target: Window, settle: Duration) -> Result<ConfirmOutcome> {if retry_pending && now >= retry_at {retry_pending = false;retried = true;- x.activate(target, x.active_window().unwrap_or(0))?;+ x.activate(target, x.active_window().unwrap_or(0), deadline)?;}- std::thread::sleep(Duration::from_millis(15));+ std::thread::sleep(remaining(deadline).min(Duration::from_millis(15)));}}@@ -689,7 +697,7 @@ fn await_focus_destination(if Instant::now() >= deadline {return None;}- std::thread::sleep(Duration::from_millis(25));+ std::thread::sleep(remaining(deadline).min(Duration::from_millis(25)));}}@@ -700,54 +708,56 @@ fn await_focus_destination(/// click, the popup that the click opened). A focus that is unset or parked/// on the window manager's own window for the whole `FOCUS_RETRY` is/// `Unknown`: it names no client the input could have reached.-fn post_check(target: Window, target_pid: Option<u32>, pid_windows: &[WindowEntry]) -> FocusAfter {+fn post_check(+ target: Window,+ target_pid: Option<u32>,+ pid_windows: &[WindowEntry],+ deadline: Instant,+) -> FocusAfter {let Ok(x) = X11::open() else {return FocusAfter::Unknown;};let Some(focused) = await_focus_destination(- Instant::now() + FOCUS_RETRY,+ deadline,|| x.focused(),|focused| x.held_by_window_manager(focused),) else {return FocusAfter::Unknown;};- if x.is_within(focused, target) {- return FocusAfter::Target;- }- if pid_windows- .iter()- .any(|entry| x.is_within(focused, entry.window as Window))- {- return FocusAfter::SamePid;- }- // The body may have closed the target itself (Escape on a dialog, alt+F4,- // a popup item click): its pid was read (or named by the caller) before- // the body, so the comparison still works once the window is gone.- match (+ crate::interaction::policy::focus_after(+ x.is_within(focused, target),+ pid_windows+ .iter()+ .any(|entry| x.is_within(focused, entry.window as Window)),target_pid.or_else(|| x.owning_pid(target)),x.owning_pid(focused),- ) {- (Some(a), Some(b)) if a == b => FocusAfter::SamePid,- _ => FocusAfter::Elsewhere,- }+ )}-/// Activate `xid`, confirm it holds the active window + input focus within-/// `opts.settle`, run `body`, then (for keyboard actions) report where focus-/// ended up. Returns `foreground_unavailable` (no input sent) when the WM never-/// confirms, and `foreground_timeout` when the X server stops answering.-pub fn with_x11_foreground_opts<T>(- xid: u64,- opts: ForegroundOptions,- body: impl FnOnce() -> Result<T>,-) -> Result<(T, ForegroundReport)> {+/// The target window confirmed active and focused, with the target+/// process's window set read before the body for [`Activated::report`].+pub struct Activated {+ target: Window,+ target_pid: Option<u32>,+ outcome: ConfirmOutcome,+ windows_before: Vec<WindowEntry>,+}++/// Activate `xid` and read back that it holds the active window + input+/// focus within `opts.settle`, capped by the action's deadline. Returns `foreground_unavailable` (no input+/// sent) when the WM never confirms, and `foreground_timeout` when the X+/// server stops answering.+pub fn activate(xid: u64, opts: ForegroundOptions) -> Result<Activated> {let target = xid as Window;- let phase_budget = opts.settle + Duration::from_millis(1500);- let outcome = run_with_deadline(phase_budget, move || confirm_phase(target, opts.settle))- .ok_or_else(|| {+ let started = Instant::now();+ let settle = operation_deadline(opts.settle);+ let phase = operation_deadline(opts.settle + Duration::from_millis(1500));+ let outcome =+ run_with_deadline(phase, move || confirm_phase(target, settle)).ok_or_else(|| {anyhow!("{CODE_TIMEOUT}: the X server did not answer the activation of window \- 0x{xid:x} within {phase_budget:?}; no input was sent"+ 0x{xid:x} within {:?}; no input was sent",+ started.elapsed())})??;if !outcome.confirmed {@@ -764,31 +774,53 @@ pub fn with_x11_foreground_opts<T>("{CODE_UNAVAILABLE}: window 0x{xid:x} did not become the active, focused \window within {:?} (active=0x{:x}, focus_within_target={}, \input focus: {holder}); no input was sent. {cause}",- opts.settle,+ outcome.elapsed,outcome.active_after,outcome.focus_within));}let target_pid = crate::x11::window_pid(xid).or(opts.target_pid);- let windows_before = pid_window_set(target_pid);- let value = body()?;- // The post-check is what the tool reports as evidence for both keyboard- // and pointer bodies: a real click that opened a menu or a dialog moves- // the window set / focus within the target's process, and one that- // landed elsewhere moves focus out of it. The window set is polled- // rather than read after a fixed settle: it returns on the first change.- let bounds = window_change_bounds();- let (windows_after, window_change) = run_with_deadline(- bounds.timeout + bounds.poll + WINDOW_CHANGE_TITLE_GRACE + Duration::from_millis(1500),- move || wait_for_window_change(target_pid, &windows_before, bounds),- )- .unwrap_or((Vec::new(), None));- let focus_after = run_with_deadline(Duration::from_millis(1500), move || {- post_check(target, target_pid, &windows_after)+ Ok(Activated {+ target,+ target_pid,+ outcome,+ windows_before: pid_window_set(target_pid),})- .unwrap_or(FocusAfter::Unknown);- Ok((- value,+}++impl Activated {+ /// Where focus and the target process's window set ended up once the+ /// body ran.+ pub fn report(self) -> ForegroundReport {+ let Activated {+ target,+ target_pid,+ outcome,+ windows_before,+ } = self;+ // The post-check is what the tool reports as evidence for both keyboard+ // and pointer bodies: a real click that opened a menu or a dialog moves+ // the window set / focus within the target's process, and one that+ // landed elsewhere moves focus out of it. The window set is polled+ // rather than read after a fixed settle: it returns on the first change.+ let bounds = window_change_bounds();+ let observed = operation_deadline(bounds.timeout);+ let (windows_after, window_change) = run_with_deadline(+ operation_deadline(+ bounds.timeout+ + bounds.poll+ + WINDOW_CHANGE_TITLE_GRACE+ + Duration::from_millis(1500),+ ),+ move || wait_for_window_change(target_pid, &windows_before, bounds, observed),+ )+ .unwrap_or((Vec::new(), None));+ let focus_retry = operation_deadline(FOCUS_RETRY);+ let focus_after =+ run_with_deadline(operation_deadline(Duration::from_millis(1500)), move || {+ post_check(target, target_pid, &windows_after, focus_retry)+ })+ .unwrap_or(FocusAfter::Unknown);ForegroundReport {already_active: outcome.already_active,retried_activation: outcome.retried,@@ -796,8 +828,8 @@ pub fn with_x11_foreground_opts<T>(focus_after,window_change_owned: window_change.as_ref().is_some_and(|c| c.owned),window_change: window_change.map(|c| c.summary),- },- ))+ }+ }}/// `0x…` plus what owns it: the class and process of a client window, or a@@ -864,7 +896,7 @@ mod tests {#[test]fn deadline_returns_none_when_work_hangs() {let started = Instant::now();- let result = run_with_deadline(Duration::from_millis(50), || {+ let result = run_with_deadline(Instant::now() + Duration::from_millis(50), || {std::thread::sleep(Duration::from_secs(5));1});@@ -905,7 +937,12 @@ mod tests {);let before = vec![entry(u64::MAX, "window that never existed", true)];let started = Instant::now();- let (_after, change) = wait_for_window_change(Some(u32::MAX), &before, bounds);+ let (_after, change) = wait_for_window_change(+ Some(u32::MAX),+ &before,+ bounds,+ Instant::now() + bounds.timeout,+ );assert_eq!(change, None);assert!(started.elapsed() < WINDOW_CHANGE_DEADLINE);}@@ -967,17 +1004,15 @@ mod tests {#[test]fn deadline_returns_value_when_work_finishes() {- assert_eq!(run_with_deadline(Duration::from_secs(2), || 7), Some(7));+ assert_eq!(+ run_with_deadline(Instant::now() + Duration::from_secs(2), || 7),+ Some(7)+ );}#[test]fn keyboard_budget_exceeds_pointer_budget() {assert!(ForegroundOptions::keyboard().settle > ForegroundOptions::pointer().settle);- assert!(ForegroundOptions::from_settle_hint(80).settle >= Duration::from_millis(400));- assert_eq!(- ForegroundOptions::from_settle_hint(5000).settle,- Duration::from_millis(5000)- );}#[test]@@ -987,7 +1022,7 @@ mod tests {ForegroundOptions::keyboard().for_pid(42).target_pid,Some(42));- assert!(!ForegroundOptions::from_settle_hint(80).for_pid(1).keyboard);+ assert!(!ForegroundOptions::pointer().for_pid(1).keyboard);}#[test]@@ -1055,9 +1090,11 @@ mod tests {#[test]fn unavailable_display_is_structured() {let display = crate::test_env::unreachable_x11_display();- let result = with_x11_foreground_opts(0x1234, ForegroundOptions::pointer(), || Ok(()));+ let result = activate(0x1234, ForegroundOptions::pointer());drop(display);- let error = result.expect_err("connect must fail");+ let Err(error) = result else {+ panic!("connect must fail")+ };assert!(error_code(&error).is_some(), "{error}");}}diff --git a/libs/cua-driver/rust/crates/platform-linux/src/input/mod.rs b/libs/cua-driver/rust/crates/platform-linux/src/input/mod.rsindex bdf0a5b185..cd1f936cb6 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/input/mod.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/input/mod.rs@@ -11,6 +11,7 @@//! them, because XTest delivers to the *focused* window and would break the//! no-focus-steal contract.+pub mod action;/// Shared `delivery_mode` contract (background|foreground) — mirrors macOS/// `tools::DeliveryMode` and Windows `input::delivery`.pub mod delivery;@@ -25,7 +26,7 @@ mod mpx_owner;pub mod targeted;pub use focus_guard::{FocusGuardReport, FocusSnapshot, SameAppWindow};-pub use foreground::{with_x11_foreground_opts, FocusAfter, ForegroundOptions, ForegroundReport};+pub use foreground::{FocusAfter, ForegroundOptions, ForegroundReport};pub use mpx_keyboard::{real_keyboard_input_available, send_popup_keyboard_key, send_popup_keyboard_text,send_virtual_keyboard_key, send_virtual_keyboard_text, KeyboardDeliveryReport, MPX_UINPUT_PATH,@@ -991,7 +992,7 @@ fn wait_for_slave_id(use_: i32,timeout: Duration,) -> Result<i32> {- let deadline = std::time::Instant::now() + timeout;+ let deadline = crate::interaction::operation_deadline(timeout);loop {for (device_id, seen_use, seen_name) in xi2_query_devices(display)? {if seen_use == use_ && seen_name == device_name {@@ -1001,7 +1002,7 @@ fn wait_for_slave_id(if std::time::Instant::now() >= deadline {bail!("timed out waiting for X input slave device '{device_name}'");}- sleep(Duration::from_millis(50));+ crate::interaction::pause(Duration::from_millis(50));}}@@ -1438,21 +1439,6 @@ fn ewmh_activate_window(}}-/// Foreground rung for X11 (`delivery_mode:"foreground"`): activate `xid`-/// (EWMH `_NET_ACTIVE_WINDOW` + core input focus), confirm the transition-/// against both the WM's active window and the X input-focus tree, then run-/// `body` (which injects the input while the window holds focus). The target-/// is left active afterwards; see [`foreground`] for the rationale and the-/// deadline/watchdog behaviour. `settle_ms` is a minimum confirmation budget.-pub fn with_x11_foreground<T>(- xid: u64,- settle_ms: u64,- body: impl FnOnce() -> Result<T>,-) -> Result<T> {- with_x11_foreground_opts(xid, ForegroundOptions::from_settle_hint(settle_ms), body)- .map(|(value, _report)| value)-}-/// Activate `xid` and LEAVE it active (no restore) — the persistent foreground/// swap behind `bring_to_front`. Returns the window that was active before, so/// the caller can report/inspect it. Best-effort; returns `None` prior on a@@ -1636,7 +1622,7 @@ pub fn send_parallel_virtual_pointer_drags(drags: &[(String, VirtualPointerDrag)let now = std::time::Instant::now();if start_at > now {- std::thread::sleep(start_at - now);+ crate::interaction::pause(start_at - now);}// Press each drag straight from its virtual master (see@@ -1660,7 +1646,7 @@ pub fn send_parallel_virtual_pointer_drags(drags: &[(String, VirtualPointerDrag)emit_button(&mut device, item.drag.button, true)?;}}- sleep(Duration::from_millis(40));+ crate::interaction::pause(Duration::from_millis(40));while active.iter().any(|item| item.current_step < item.steps) {let now = std::time::Instant::now();@@ -1708,7 +1694,7 @@ pub fn send_parallel_virtual_pointer_drags(drags: &[(String, VirtualPointerDrag)if let Some(deadline) = next_deadline {let now = std::time::Instant::now();if deadline > now {- std::thread::sleep(deadline - now);+ crate::interaction::pause(deadline - now);}}}@@ -2667,7 +2653,7 @@ fn pointer_effect(y: i32,) -> PointerEffect {unsafe { x11::xlib::XSync(display, 0) };- sleep(EFFECT_SETTLE);+ crate::interaction::pause(EFFECT_SETTLE);let after = root_region_pixels(x, y);let popups = new_popups(popups_before, mapped_popups(display));let foreign_window = foreign_window_effect(foreign_before);@@ -2735,14 +2721,14 @@ pub fn send_virtual_pointer_click(emit_button(&mut device, click.button, true)?;// A real press and release are separate evdev frames; give// the toolkit a press it can see before the release lands.- sleep(Duration::from_millis(press_ms));+ crate::interaction::pause(Duration::from_millis(press_ms));emit_button(&mut device, click.button, false)?;}// Multi-click cadence: keep press→press well under the toolkit// double-click threshold (GTK default 250 ms) so count=2 lands// as a real double-click, not two singles.if count > 1 && i + 1 < count {- sleep(Duration::from_millis(gap_ms));+ crate::interaction::pause(Duration::from_millis(gap_ms));}}Ok(())@@ -2834,7 +2820,7 @@ pub fn send_virtual_pointer_drag(}// Let the toolkit register the press (and arm its drag threshold)// before the first motion.- sleep(Duration::from_millis(40));+ crate::interaction::pause(Duration::from_millis(40));let (mut last_x, mut last_y) = start;for step in 1..=steps {let t = step as f64 / steps as f64;@@ -2854,12 +2840,12 @@ pub fn send_virtual_pointer_drag(}last_x = ix;last_y = iy;- sleep(step_delay);+ crate::interaction::pause(step_delay);}// Relative motion accumulates libinput rounding; pin the release// to the exact end point before letting go.warp_master_pointer(display, ids, end.0, end.1)?;- sleep(Duration::from_millis(20));+ crate::interaction::pause(Duration::from_millis(20));let mut device = device.lock().unwrap();emit_button(&mut device, drag.button, false)?;Ok(())@@ -2933,7 +2919,7 @@ pub fn send_virtual_pointer_scroll(cursor_id: &str, scroll: &VirtualPointerScrollet mut device = device.lock().unwrap();emit_scroll(&mut device, scroll.horizontal, unit)?;}- sleep(Duration::from_millis(CLICK_DELAY_MS));+ crate::interaction::pause(Duration::from_millis(CLICK_DELAY_MS));}Ok(())})();@@ -2975,6 +2961,10 @@ unsafe extern "C" fn ignore_x_error(}fn restore_focus_state(display: *mut x11::xlib::Display, saved: &SavedFocus) {+ let cleanup = std::time::Instant::now() + crate::interaction::CLEANUP_CAP;+ let nap = |wait: Duration| {+ sleep(wait.min(cleanup.saturating_duration_since(std::time::Instant::now())))+ };// Let the release/focus events from the drag settle before reading the// post-drag state, so we don't race the WM's own focus update.unsafe { x11::xlib::XSync(display, 0) };@@ -2986,7 +2976,7 @@ fn restore_focus_state(display: *mut x11::xlib::Display, saved: &SavedFocus) {if ewmh_active_window(display) == Some(prev) {let mut stable = 0;for _ in 0..4 {- sleep(Duration::from_millis(25));+ nap(Duration::from_millis(25));if ewmh_active_window(display) == Some(prev) {stable += 1;} else {@@ -3022,9 +3012,11 @@ fn restore_focus_state(display: *mut x11::xlib::Display, saved: &SavedFocus) {if own_dialog(ewmh_active_window(display)) {return;}- sleep(Duration::from_millis(300));+ nap(Duration::from_millis(150));let mut stable = 0;- for attempt in 0..15 {+ let mut attempt = 0;+ while std::time::Instant::now() < cleanup {+ attempt += 1;let now = ewmh_active_window(display);if now == Some(prev) {stable += 1;@@ -3042,7 +3034,7 @@ fn restore_focus_state(display: *mut x11::xlib::Display, saved: &SavedFocus) {// updates. Bounce the core focus onto the window the WM// believes active so the activation produces a real focus// transition the WM can observe.- if attempt >= 2 {+ if attempt > 2 {if let Some(now_win) = now {unsafe {let prev_handler = x11::xlib::XSetErrorHandler(Some(ignore_x_error));@@ -3055,12 +3047,12 @@ fn restore_focus_state(display: *mut x11::xlib::Display, saved: &SavedFocus) {x11::xlib::XSync(display, 0);x11::xlib::XSetErrorHandler(prev_handler);}- sleep(Duration::from_millis(100));+ nap(Duration::from_millis(50));}}ewmh_activate_window(display, prev, now.unwrap_or(0));}- sleep(Duration::from_millis(200));+ nap(Duration::from_millis(50));}if stable == 0 {tracing::warn!("focus restore: WM did not re-activate 0x{prev:x}");@@ -3265,12 +3257,115 @@ fn connect_x11_for_input() -> Result<(RustConnection, usize)> {if let Some(reason) = crate::wayland::wayland_input_unavailable_reason() {bail!("{reason}");}- RustConnection::connect(None).map_err(|e| {+ let (conn, screen) = RustConnection::connect(None).map_err(|e| {anyhow!("cannot inject input: X11 connection failed (DISPLAY={:?}): {e}",std::env::var("DISPLAY").ok())- })+ })?;+ action::fence(conn.stream())?;+ Ok((conn, screen))+}++fn admitted<T, E: Into<anyhow::Error>>(send: impl FnOnce() -> Result<T, E>) -> Result<T> {+ action::admit(send)?.map_err(Into::into)+}++/// How long dropping an abandoned call's [`Delivered`] waits for its+/// releases. It bounds the wait only: the releasing thread is not stopped,+/// and keeps the call's desktop lease until its X11 requests return.+const ABANDONED_RELEASE_WAIT: Duration = Duration::from_secs(2);++/// XTest presses this call queued, and whether each is still held. A route+/// releases what is held on its own connection, even after a failure.+/// Abandoning the call shuts that connection, so the releases it still owes+/// cannot go out there: dropping this then sends them on a fresh connection+/// from a thread that holds the call's [`action::Action`], and with it the+/// desktop, until those releases are done. The drop waits for them at most+/// [`ABANDONED_RELEASE_WAIT`]; past it the thread still holds the desktop.+/// The server ignores a release for a key or button that is already up.+#[derive(Default)]+struct Delivered(Vec<(u8, u8, bool)>);++impl Delivered {+ fn press<C>(+ &mut self,+ kind: u8,+ detail: u8,+ send: impl FnOnce() -> Result<C, x11rb::errors::ConnectionError>,+ ) -> Result<C> {+ let sent = admitted(send)?;+ self.0.push((kind + 1, detail, true));+ Ok(sent)+ }++ /// The release of the latest held `detail` pressed by `kind + 1` went out.+ fn released(&mut self, release: u8, detail: u8) {+ if let Some(press) = self+ .0+ .iter_mut()+ .rev()+ .find(|press| **press == (release, detail, true))+ {+ press.2 = false;+ }+ }++ /// Release on `conn` whatever is still held, latest first.+ fn release_held(&mut self, conn: &RustConnection) -> Result<()> {+ use x11rb::protocol::xtest::ConnectionExt as _;+ let mut result = Ok(());+ for (release, detail, held) in self.0.iter_mut().rev().filter(|press| press.2) {+ *held = false;+ if let Err(error) = conn.xtest_fake_input(*release, *detail, 0, x11rb::NONE, 0, 0, 0) {+ result = result.and(Err(error.into()));+ }+ }+ result+ }++ /// The server took every release: nothing is owed after abandon.+ fn confirmed(&mut self) {+ self.0.clear();+ }+}++impl Drop for Delivered {+ fn drop(&mut self) {+ let Some(action) = action::current().filter(|action| action.is_abandoned()) else {+ return;+ };+ if self.0.is_empty() {+ return;+ }+ let presses = self+ .0+ .drain(..)+ .map(|(release, detail, _)| (release, detail));+ if let Err(error) = release_abandoned(action, presses.collect()) {+ tracing::warn!("releasing the input of an abandoned call failed: {error:#}");+ }+ }+}++fn release_abandoned(action: Arc<action::Action>, presses: Vec<(u8, u8)>) -> Result<()> {+ use x11rb::protocol::xtest::ConnectionExt as _;+ let (done, finished) = std::sync::mpsc::channel();+ std::thread::spawn(move || {+ let _desktop = action;+ let released = (|| -> Result<()> {+ let (conn, _) = RustConnection::connect(None)?;+ for &(kind, detail) in presses.iter().rev() {+ conn.xtest_fake_input(kind, detail, 0, x11rb::NONE, 0, 0, 0)?;+ }+ conn.get_input_focus()?.reply()?;+ Ok(())+ })();+ let _ = done.send(released);+ });+ finished.recv_timeout(ABANDONED_RELEASE_WAIT).map_err(|_| {+ anyhow!("the X server did not take the releases within {ABANDONED_RELEASE_WAIT:?}")+ })?}/// Send a synthetic FocusIn event to a window without changing the actual X11 input focus.@@ -3288,7 +3383,7 @@ pub fn send_focus_in(xid: u64) -> Result<()> {mode: NotifyMode::NORMAL,};- conn.send_event(false, window, EventMask::FOCUS_CHANGE, focus_in)?;+ admitted(|| conn.send_event(false, window, EventMask::FOCUS_CHANGE, focus_in))?;conn.flush()?;Ok(())}@@ -3306,7 +3401,7 @@ pub fn send_focus_out(xid: u64) -> Result<()> {mode: NotifyMode::NORMAL,};- conn.send_event(false, window, EventMask::FOCUS_CHANGE, focus_out)?;+ admitted(|| conn.send_event(false, window, EventMask::FOCUS_CHANGE, focus_out))?;conn.flush()?;Ok(())}@@ -3380,14 +3475,14 @@ pub fn send_click_with_modifiers(// The event mask selects recipients independently of the payload type.// Select the press client for both payloads so a press-only toolkit// binding (such as Tk's <Button-1>) receives one coherent click.- conn.send_event(false, target.window, EventMask::BUTTON_PRESS, press)?+ admitted(|| conn.send_event(false, target.window, EventMask::BUTTON_PRESS, press))?.check()?;- sleep(Duration::from_millis(CLICK_DELAY_MS));- conn.send_event(false, target.window, EventMask::BUTTON_PRESS, release)?+ crate::interaction::pause(Duration::from_millis(CLICK_DELAY_MS));+ admitted(|| conn.send_event(false, target.window, EventMask::BUTTON_PRESS, release))?.check()?;if count > 1 {- sleep(Duration::from_millis(80));+ crate::interaction::pause(Duration::from_millis(80));}}settle_synthetic_events(&conn);@@ -3412,8 +3507,7 @@ pub fn send_button_down(xid: u64, x: i32, y: i32, button: u8) -> Result<()> {state: KeyButMask::from(0u16),same_screen: true,};- conn.send_event(false, target.window, EventMask::BUTTON_PRESS, &press)?- .check()?;+ admitted(|| conn.send_event(false, target.window, EventMask::BUTTON_PRESS, &press))?.check()?;settle_synthetic_events(&conn);Ok(())}@@ -3439,7 +3533,7 @@ pub fn send_motion(xid: u64, x: i32, y: i32, button: Option<u8>) -> Result<()> {.unwrap_or_else(|| KeyButMask::from(0u16)),same_screen: true,};- conn.send_event(false, target.window, EventMask::POINTER_MOTION, motion)?;+ admitted(|| conn.send_event(false, target.window, EventMask::POINTER_MOTION, motion))?;conn.flush()?;settle_synthetic_events(&conn);Ok(())@@ -3628,14 +3722,14 @@ pub fn send_type_text_with_delay(xid: u64, text: &str, inter_char_ms: u64) -> Resame_screen: true,};- conn.send_event(false, window, EventMask::KEY_PRESS, press)?;+ admitted(|| conn.send_event(false, window, EventMask::KEY_PRESS, press))?;// Start the hold interval after sending the press, not while it is buffered.conn.flush()?;- sleep(Duration::from_millis(KEY_DELAY_MS));+ crate::interaction::pause(Duration::from_millis(KEY_DELAY_MS));conn.send_event(false, window, EventMask::KEY_RELEASE, release)?;conn.flush()?;if inter_char_ms > 0 {- sleep(Duration::from_millis(inter_char_ms));+ crate::interaction::pause(Duration::from_millis(inter_char_ms));}Ok(())},@@ -3672,16 +3766,19 @@ pub fn send_type_text_xtest(text: &str) -> Result<()> {|ch, mapping| text_keycode(&mut client.borrow_mut(), mapping, ch),|keycode, needs_shift| {client.borrow_mut().record_recipient(keycode);- if needs_shift {- conn.xtest_fake_input(KEY_PRESS_EVENT, shift_kc, 0, x11rb::NONE, 0, 0, 0)?;+ let mut delivered = Delivered::default();+ for kc in needs_shift.then_some(shift_kc).into_iter().chain([keycode]) {+ delivered.press(KEY_PRESS_EVENT, kc, || {+ conn.xtest_fake_input(KEY_PRESS_EVENT, kc, 0, x11rb::NONE, 0, 0, 0)+ })?;}- conn.xtest_fake_input(KEY_PRESS_EVENT, keycode, 0, x11rb::NONE, 0, 0, 0)?;conn.xtest_fake_input(KEY_RELEASE_EVENT, keycode, 0, x11rb::NONE, 0, 0, 0)?;if needs_shift {conn.xtest_fake_input(KEY_RELEASE_EVENT, shift_kc, 0, x11rb::NONE, 0, 0, 0)?;}conn.flush()?;- sleep(Duration::from_millis(KEY_DELAY_MS));+ delivered.confirmed();+ crate::interaction::pause(Duration::from_millis(KEY_DELAY_MS));Ok(())},|| client.borrow_mut().release(),@@ -3760,16 +3857,20 @@ pub fn send_key_xtest(key: &str, modifiers: &[&str]) -> Result<()> {};// Press modifiers (+ auto-Shift), tap the key, release in reverse order.- for &kc in &mod_keycodes {- conn.xtest_fake_input(KEY_PRESS_EVENT, kc, 0, x11rb::NONE, 0, 0, 0)?;- }- if let Some(sk) = auto_shift_kc {- conn.xtest_fake_input(KEY_PRESS_EVENT, sk, 0, x11rb::NONE, 0, 0, 0)?;+ let mut delivered = Delivered::default();+ for kc in mod_keycodes+ .iter()+ .copied()+ .chain(auto_shift_kc)+ .chain([keycode])+ {+ delivered.press(KEY_PRESS_EVENT, kc, || {+ conn.xtest_fake_input(KEY_PRESS_EVENT, kc, 0, x11rb::NONE, 0, 0, 0)+ })?;}- conn.xtest_fake_input(KEY_PRESS_EVENT, keycode, 0, x11rb::NONE, 0, 0, 0)?;// Flush modifiers and key-down before measuring the delivered hold interval.conn.flush()?;- sleep(Duration::from_millis(KEY_DELAY_MS));+ crate::interaction::pause(Duration::from_millis(KEY_DELAY_MS));conn.xtest_fake_input(KEY_RELEASE_EVENT, keycode, 0, x11rb::NONE, 0, 0, 0)?;if let Some(sk) = auto_shift_kc {conn.xtest_fake_input(KEY_RELEASE_EVENT, sk, 0, x11rb::NONE, 0, 0, 0)?;@@ -3778,6 +3879,7 @@ pub fn send_key_xtest(key: &str, modifiers: &[&str]) -> Result<()> {conn.xtest_fake_input(KEY_RELEASE_EVENT, kc, 0, x11rb::NONE, 0, 0, 0)?;}conn.flush()?;+ delivered.confirmed();// Round-trip so the server actually PROCESSES (delivers) the injected XTEST// key events before this function returns and drops its short-lived@@ -3836,59 +3938,46 @@ pub fn send_click_xtest_desktop_with_modifiers(}modifier_keycodes.push(keycode);}- let mut pressed = Vec::new();- let mut button_pressed = false;+ let mut delivered = Delivered::default();let gesture_result = (|| -> Result<()> {for &keycode in &modifier_keycodes {- conn.xtest_fake_input(KEY_PRESS_EVENT, keycode, 0, x11rb::NONE, 0, 0, 0)?;- pressed.push(keycode);+ delivered.press(KEY_PRESS_EVENT, keycode, || {+ conn.xtest_fake_input(KEY_PRESS_EVENT, keycode, 0, x11rb::NONE, 0, 0, 0)+ })?;}// Absolute pointer warp (MotionNotify, detail=0 => absolute) so the// button events that follow are delivered at (x, y).- conn.xtest_fake_input(MOTION_NOTIFY_EVENT, 0, 0, root, x as i16, y as i16, 0)?;+ admitted(|| conn.xtest_fake_input(MOTION_NOTIFY_EVENT, 0, 0, root, x as i16, y as i16, 0))?;let count = count.max(1);for click_index in 0..count {- let press =- conn.xtest_fake_input(BUTTON_PRESS_EVENT, button, 0, root, x as i16, y as i16, 0)?;+ let press = delivered.press(BUTTON_PRESS_EVENT, button, || {+ conn.xtest_fake_input(BUTTON_PRESS_EVENT, button, 0, root, x as i16, y as i16, 0)+ })?;// Confirm delivery before sleeping; buffering both edges produces a zero-ms click.press.check()?;- button_pressed = true;- sleep(Duration::from_millis(CLICK_DELAY_MS));+ crate::interaction::pause(Duration::from_millis(CLICK_DELAY_MS));conn.xtest_fake_input(BUTTON_RELEASE_EVENT, button, 0, root, x as i16, y as i16, 0)?.check()?;- button_pressed = false;+ delivered.released(BUTTON_RELEASE_EVENT, button);if click_index + 1 < count {// Keep the existing gap between completed click pairs.- sleep(Duration::from_millis(DOUBLE_CLICK_DELAY_MS));+ crate::interaction::pause(Duration::from_millis(DOUBLE_CLICK_DELAY_MS));}}Ok(())})();// Attempt all releases after a partial failure. Do not strand buttons or modifiers.- let mut release_result: Result<()> = Ok(());- if button_pressed {- release_result = (|| -> Result<()> {- conn.xtest_fake_input(BUTTON_RELEASE_EVENT, button, 0, root, x as i16, y as i16, 0)?- .check()?;- Ok(())- })();- }- for &keycode in pressed.iter().rev() {- if let Err(error) =- conn.xtest_fake_input(KEY_RELEASE_EVENT, keycode, 0, x11rb::NONE, 0, 0, 0)- {- if release_result.is_ok() {- release_result = Err(error.into());- }- }- }+ let release_result = delivered.release_held(&conn);// Complete queued modifier releases before the short-lived connection closes,// while preserving the primary gesture or cleanup error when one exists.let completion_result = (|| -> Result<()> {conn.get_input_focus()?.reply()?;Ok(())})();+ if completion_result.is_ok() {+ delivered.confirmed();+ }drop(guards);gesture_result?;release_result?;@@ -3901,7 +3990,7 @@ pub fn send_move_xtest_desktop(x: i32, y: i32) -> Result<()> {use x11rb::protocol::xtest::ConnectionExt as _;let (conn, screen_num) = connect_x11_for_input()?;let root = conn.setup().roots[screen_num].root;- conn.xtest_fake_input(MOTION_NOTIFY_EVENT, 0, 0, root, x as i16, y as i16, 0)?;+ admitted(|| conn.xtest_fake_input(MOTION_NOTIFY_EVENT, 0, 0, root, x as i16, y as i16, 0))?;conn.flush()?;let _ = conn.get_input_focus()?.reply();Ok(())@@ -3920,12 +4009,16 @@ pub fn send_scroll_xtest_desktop(x: i32, y: i32, direction: &str, amount: usize)};let (conn, screen_num) = connect_x11_for_input()?;let root = conn.setup().roots[screen_num].root;- conn.xtest_fake_input(MOTION_NOTIFY_EVENT, 0, 0, root, x as i16, y as i16, 0)?;+ admitted(|| conn.xtest_fake_input(MOTION_NOTIFY_EVENT, 0, 0, root, x as i16, y as i16, 0))?;+ let mut delivered = Delivered::default();for _ in 0..amount.max(1) {- conn.xtest_fake_input(BUTTON_PRESS_EVENT, button, 0, root, x as i16, y as i16, 0)?;+ delivered.press(BUTTON_PRESS_EVENT, button, || {+ conn.xtest_fake_input(BUTTON_PRESS_EVENT, button, 0, root, x as i16, y as i16, 0)+ })?;conn.xtest_fake_input(BUTTON_RELEASE_EVENT, button, 0, root, x as i16, y as i16, 0)?;}conn.flush()?;+ delivered.confirmed();let _ = conn.get_input_focus()?.reply();Ok(())}@@ -3956,50 +4049,55 @@ pub fn send_drag_xtest_desktop_with_modifiers(}let steps = steps.max(1);let delay = duration_ms / steps as u64;- let mut pressed_modifiers = Vec::new();- let mut button_pressed = false;+ let mut delivered = Delivered::default();let gesture_result = (|| -> Result<()> {for &keycode in &modifier_keycodes {- conn.xtest_fake_input(KEY_PRESS_EVENT, keycode, 0, x11rb::NONE, 0, 0, 0)?;- pressed_modifiers.push(keycode);+ delivered.press(KEY_PRESS_EVENT, keycode, || {+ conn.xtest_fake_input(KEY_PRESS_EVENT, keycode, 0, x11rb::NONE, 0, 0, 0)+ })?;}- conn.xtest_fake_input(- MOTION_NOTIFY_EVENT,- 0,- 0,- root,- from_x as i16,- from_y as i16,- 0,- )?;- conn.xtest_fake_input(- BUTTON_PRESS_EVENT,- button,- 0,- root,- from_x as i16,- from_y as i16,- 0,- )?;- button_pressed = true;- conn.flush()?;- for step in 1..=steps {- let t = step as f64 / steps as f64;- let x = from_x as f64 + (to_x - from_x) as f64 * t;- let y = from_y as f64 + (to_y - from_y) as f64 * t;+ admitted(|| {conn.xtest_fake_input(MOTION_NOTIFY_EVENT,0,0,root,- x.round() as i16,- y.round() as i16,+ from_x as i16,+ from_y as i16,0,- )?;+ )+ })?;+ delivered.press(BUTTON_PRESS_EVENT, button, || {+ conn.xtest_fake_input(+ BUTTON_PRESS_EVENT,+ button,+ 0,+ root,+ from_x as i16,+ from_y as i16,+ 0,+ )+ })?;+ conn.flush()?;+ for step in 1..=steps {+ let t = step as f64 / steps as f64;+ let x = from_x as f64 + (to_x - from_x) as f64 * t;+ let y = from_y as f64 + (to_y - from_y) as f64 * t;+ admitted(|| {+ conn.xtest_fake_input(+ MOTION_NOTIFY_EVENT,+ 0,+ 0,+ root,+ x.round() as i16,+ y.round() as i16,+ 0,+ )+ })?;conn.flush()?;if delay > 0 {- sleep(Duration::from_millis(delay));+ crate::interaction::pause(Duration::from_millis(delay));}}conn.xtest_fake_input(@@ -4011,39 +4109,18 @@ pub fn send_drag_xtest_desktop_with_modifiers(to_y as i16,0,)?;- button_pressed = false;+ delivered.released(BUTTON_RELEASE_EVENT, button);Ok(())})();// Always attempt to release the button and every modifier queued above,// including when a later motion request fails.- let mut cleanup_result: Result<()> = Ok(());- if button_pressed {- if let Err(error) = conn.xtest_fake_input(- BUTTON_RELEASE_EVENT,- button,- 0,- root,- to_x as i16,- to_y as i16,- 0,- ) {- cleanup_result = Err(error.into());- }- }- for &keycode in pressed_modifiers.iter().rev() {- if let Err(error) =- conn.xtest_fake_input(KEY_RELEASE_EVENT, keycode, 0, x11rb::NONE, 0, 0, 0)- {- if cleanup_result.is_ok() {- cleanup_result = Err(error.into());- }- }- }+ let mut cleanup_result = delivered.release_held(&conn);if let Err(error) = conn.flush() {- if cleanup_result.is_ok() {- cleanup_result = Err(error.into());- }+ cleanup_result = cleanup_result.and(Err(error.into()));+ }+ if cleanup_result.is_ok() {+ delivered.confirmed();}let _ = conn.get_input_focus()?.reply();drop(guards);@@ -4120,7 +4197,7 @@ fn send_key_to_target(state,same_screen: true,};- conn.send_event(false, window, event_mask, event)+ admitted(|| conn.send_event(false, window, event_mask, event))};let mut state_bits = 0u16;@@ -4137,7 +4214,7 @@ fn send_key_to_target(send_key_event(KEY_PRESS_EVENT, keycode, state, EventMask::KEY_PRESS)?;// Otherwise X11 receives both transitions together after the sleep.conn.flush()?;- sleep(Duration::from_millis(KEY_DELAY_MS));+ crate::interaction::pause(Duration::from_millis(KEY_DELAY_MS));send_key_event(KEY_RELEASE_EVENT, keycode, state, EventMask::KEY_RELEASE)?;for &(modifier_keycode, modifier_mask) in modifier_keycodes.iter().rev() {send_key_event(@@ -4251,6 +4328,74 @@ pub(crate) fn key_name_to_keysym(key: &str) -> Result<u32> {Ok(keysym)}+#[cfg(test)]+mod admission_tests {+ use super::*;++ #[test]+ fn an_x11_send_past_its_action_is_refused_before_it_is_queued() {+ let action = action::Action::new(Duration::from_secs(60));+ let _bound = action::bind(action.clone());+ assert_eq!(admitted(|| Ok::<_, anyhow::Error>(1)).unwrap(), 1);+ action.abandon();+ admitted(|| -> Result<()> { panic!("queued after abandon") }).unwrap_err();++ let _expired = action::bind(action::Action::new(Duration::ZERO));+ admitted(|| -> Result<()> { panic!("queued past the deadline") }).unwrap_err();+ }++ #[test]+ fn a_drag_abandoned_after_its_press_still_releases_the_button() {+ use std::io::{BufRead, BufReader};+ struct Server(std::process::Child);+ impl Drop for Server {+ fn drop(&mut self) {+ let _ = self.0.kill();+ let _ = self.0.wait();+ }+ }+ let mut server = Server(+ std::process::Command::new("Xvfb")+ .args(["-displayfd", "1", "-nolisten", "tcp"])+ .stdout(std::process::Stdio::piped())+ .stderr(std::process::Stdio::null())+ .spawn()+ .expect("install xvfb to run X11 input tests"),+ );+ let mut number = String::new();+ BufReader::new(server.0.stdout.take().unwrap())+ .read_line(&mut number)+ .unwrap();+ let display = format!(":{}", number.trim());+ let _display = crate::test_env::x11_display(&display);+ let action = action::Action::new(Duration::from_secs(60));+ let dragging = {+ let action = action.clone();+ std::thread::spawn(move || {+ let _bound = action::bind(action);+ send_drag_xtest_desktop_with_modifiers(10, 10, 200, 200, 1, 3000, 30, &[])+ })+ };+ let (observer, screen) = RustConnection::connect(Some(&display)).unwrap();+ let root = observer.setup().roots[screen].root;+ let held = || {+ let mask = observer.query_pointer(root).unwrap().reply().unwrap().mask;+ mask.contains(KeyButMask::BUTTON1)+ };+ let pressed_by = std::time::Instant::now() + Duration::from_secs(2);+ while !held() {+ assert!(+ std::time::Instant::now() < pressed_by,+ "the drag never pressed"+ );+ sleep(Duration::from_millis(20));+ }+ action.abandon();+ dragging.join().unwrap().unwrap_err();+ assert!(!held(), "the abandoned drag left button 1 held");+ }+}+#[cfg(test)]mod own_dialog_tests {use super::*;diff --git a/libs/cua-driver/rust/crates/platform-linux/src/input/mpx_keyboard.rs b/libs/cua-driver/rust/crates/platform-linux/src/input/mpx_keyboard.rsindex a8e542f140..bd9795d7a8 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/input/mpx_keyboard.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/input/mpx_keyboard.rs@@ -588,11 +588,17 @@ fn deliver(for step in steps {let emitted = {let mut device = device.lock().unwrap();- device.emit(&[InputEvent::new(+ let event = [InputEvent::new(EventType::KEY,step.evdev_code,if step.press { 1 } else { 0 },- )])+ )];+ if step.press {+ super::action::admit(|| device.emit(&event))+ .and_then(|emitted| emitted.map_err(Into::into))+ } else {+ device.emit(&event).map_err(Into::into)+ }};if let Err(error) = emitted {// Never abort with keys down: a held modifier would chord@@ -601,7 +607,7 @@ fn deliver(for code in held.iter().rev() {let _ = device.emit(&[InputEvent::new(EventType::KEY, *code, 0)]);}- return Err(error.into());+ return Err(error);}if step.press {held.push(step.evdev_code);diff --git a/libs/cua-driver/rust/crates/platform-linux/src/input/targeted.rs b/libs/cua-driver/rust/crates/platform-linux/src/input/targeted.rsindex ee6f41af2b..31feb70b78 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/input/targeted.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/input/targeted.rs@@ -182,6 +182,7 @@ pub(crate) fn connect() -> Result<(RustConnection, Window), TargetedInputError>std::env::var("DISPLAY").ok()))})?;+ crate::input::action::fence(conn.stream()).map_err(failed)?;let root = conn.setup().roots[screen].root;Ok((conn, root))}@@ -351,12 +352,15 @@ pub(crate) fn activate(if let Some(message_type) = atom(conn, "_NET_ACTIVE_WINDOW") {let event =ClientMessageEvent::new(32, window, message_type, [1, x11rb::CURRENT_TIME, 0, 0, 0]);- conn.send_event(- false,- root,- EventMask::SUBSTRUCTURE_REDIRECT | EventMask::SUBSTRUCTURE_NOTIFY,- event,- )+ crate::input::action::admit(|| {+ conn.send_event(+ false,+ root,+ EventMask::SUBSTRUCTURE_REDIRECT | EventMask::SUBSTRUCTURE_NOTIFY,+ event,+ )+ })+ .map_err(failed)?.map_err(failed)?;}let _ = conn.configure_window(@@ -538,7 +542,8 @@ pub(crate) fn fake_motion() -> Result<(), TargetedInputError> {let x = x.clamp(i16::MIN.into(), i16::MAX.into()) as i16;let y = y.clamp(i16::MIN.into(), i16::MAX.into()) as i16;- conn.xtest_fake_input(MOTION_NOTIFY_EVENT, 0, 0, root, x, y, 0)+ crate::input::action::admit(|| conn.xtest_fake_input(MOTION_NOTIFY_EVENT, 0, 0, root, x, y, 0))+ .map_err(failed)?.map_err(failed)?;Ok(())}@@ -554,8 +559,13 @@ pub(crate) fn fake_button(} else {BUTTON_RELEASE_EVENT};- conn.xtest_fake_input(kind, button, 0, root, 0, 0, 0)- .map_err(failed)?;+ let send = || conn.xtest_fake_input(kind, button, 0, root, 0, 0, 0);+ if press {+ crate::input::action::admit(send).map_err(failed)?+ } else {+ send()+ }+ .map_err(failed)?;Ok(())}@@ -570,8 +580,13 @@ pub(crate) fn fake_key(} else {KEY_RELEASE_EVENT};- conn.xtest_fake_input(kind, keycode, 0, root, 0, 0, 0)- .map_err(failed)?;+ let send = || conn.xtest_fake_input(kind, keycode, 0, root, 0, 0, 0);+ if press {+ crate::input::action::admit(send).map_err(failed)?+ } else {+ send()+ }+ .map_err(failed)?;Ok(())}diff --git a/libs/cua-driver/rust/crates/platform-linux/src/interaction/facts.rs b/libs/cua-driver/rust/crates/platform-linux/src/interaction/facts.rsnew file mode 100644index 0000000000..c01f0995e6--- /dev/null+++ b/libs/cua-driver/rust/crates/platform-linux/src/interaction/facts.rs@@ -0,0 +1,99 @@+//! Facts an adapter reports to the shared policy, each one known or+//! unknown with the reason it could not be observed.++use std::time::Duration;++use anyhow::{ensure, Result};+use zbus::zvariant::OwnedObjectPath;++/// A fact an adapter observed, or why it could not.+pub enum Fact<T> {+ Known(T),+ Unknown(String),+}++/// The facts the policy admits an operation on.+pub struct Facts {+ pub locked: Fact<bool>,+ /// Whether the target's exact focus was read back right before dispatch.+ pub focused: Fact<bool>,+}++const LOCK_PROBE_CAP: Duration = Duration::from_millis(250);++/// Whether the session's screensaver reports itself active. KWin and X11+/// sessions ask `org.freedesktop.ScreenSaver`, GNOME asks+/// `org.gnome.ScreenSaver`; the answer counts only when the service's owner+/// runs in this process's logind session. niri, Sway and Hyprland expose no+/// lock state, and a missing service, a timeout or another session's owner+/// leave it unknown.+pub fn locked() -> Fact<bool> {+ use crate::wayland::{hyprland, is_wayland, niri};+ if niri::is_session()+ || (is_wayland() && hyprland::is_session())+ || std::env::var_os("SWAYSOCK").is_some()+ {+ return Fact::Unknown("this compositor reports no lock state".into());+ }+ let gnome = std::env::var("XDG_CURRENT_DESKTOP").is_ok_and(|desktop| desktop.contains("GNOME"));+ let (service, path) = if gnome {+ ("org.gnome.ScreenSaver", "/org/gnome/ScreenSaver")+ } else {+ (+ "org.freedesktop.ScreenSaver",+ "/org/freedesktop/ScreenSaver",+ )+ };+ let wait = super::operation_deadline(LOCK_PROBE_CAP)+ .saturating_duration_since(std::time::Instant::now());+ let probe = std::thread::spawn(move || -> Result<bool> {+ let runtime = tokio::runtime::Builder::new_current_thread()+ .enable_all()+ .build()?;+ runtime.block_on(async {+ tokio::time::timeout(wait, screensaver_active(service, path))+ .await+ .map_err(|_| anyhow::anyhow!("{service} did not answer within {wait:?}"))?+ })+ });+ match probe.join() {+ Ok(Ok(active)) => Fact::Known(active),+ Ok(Err(error)) => Fact::Unknown(format!("{service}.GetActive: {error:#}")),+ Err(_) => Fact::Unknown(format!("{service}.GetActive probe panicked")),+ }+}++async fn screensaver_active(service: &str, path: &str) -> Result<bool> {+ let session = zbus::Connection::session().await?;+ let bus = zbus::Proxy::new(+ &session,+ "org.freedesktop.DBus",+ "/org/freedesktop/DBus",+ "org.freedesktop.DBus",+ )+ .await?;+ let owner: String = bus.call("GetNameOwner", &(service,)).await?;+ let owner_pid: u32 = bus+ .call("GetConnectionUnixProcessID", &(owner.as_str(),))+ .await?;+ let system = zbus::Connection::system().await?;+ let logind = zbus::Proxy::new(+ &system,+ "org.freedesktop.login1",+ "/org/freedesktop/login1",+ "org.freedesktop.login1.Manager",+ )+ .await?;+ let theirs: OwnedObjectPath = logind.call("GetSessionByPID", &(owner_pid,)).await?;+ let ours: OwnedObjectPath = logind+ .call("GetSessionByPID", &(std::process::id(),))+ .await?;+ ensure!(+ theirs == ours,+ "its owner runs in logind session {}, not this process's {}",+ theirs.as_str(),+ ours.as_str()+ );+ let saver = zbus::Proxy::new(&session, owner.as_str(), path, service).await?;+ Ok(saver.call("GetActive", &()).await?)+}diff --git a/libs/cua-driver/rust/crates/platform-linux/src/interaction/mod.rs b/libs/cua-driver/rust/crates/platform-linux/src/interaction/mod.rsnew file mode 100644index 0000000000..76866c3ff8--- /dev/null+++ b/libs/cua-driver/rust/crates/platform-linux/src/interaction/mod.rs@@ -0,0 +1,462 @@+//! The layer every Linux compositor adapter shares: the lock, identity,+//! delivery, freshness, reach, motion and restore rules in [`policy`], the one+//! foreground transaction entry point, the deadline rule, the focus+//! read-back loop and the settle loop. Wayland adapters supply mechanical+//! save, focus and restore steps to the one transaction; X11 supplies+//! activate and report steps. Adapters measure geometry; niri's output and+//! placement limits stay in its pointer gate.++use std::time::{Duration, Instant};++use anyhow::Result;++use crate::input::{ForegroundOptions, ForegroundReport};++pub mod facts;+pub mod policy;++/// Refuse an operation of `scope` the session lock does not admit.+/// `focused` is the target's exact focus, read back right before dispatch.+pub fn admit_lock(scope: policy::Scope, focused: facts::Fact<bool>) -> Result<()> {+ let facts = facts::Facts {+ locked: facts::locked(),+ focused,+ };+ policy::lock(scope, &facts)+}++/// Whether a foreground transaction gives focus back once its body ran.+#[derive(Clone, Copy)]+pub enum RestoreMode {+ RestorePrevious,+ /// The body activates the target itself and leaves it focused.+ LeaveFocused,+}++/// How a window-addressed pointer gesture (double click, scroll, drag)+/// treats focus: niri restores the prior focus, as its clicks do; other+/// compositors leave the target focused.+pub fn pointer_gesture_restore() -> RestoreMode {+ if crate::wayland::niri::is_session() {+ RestoreMode::RestorePrevious+ } else {+ RestoreMode::LeaveFocused+ }+}++/// Run `body` with the exact `pid`/`window_id` pair focused. On Wayland a+/// compositor adapter confirms the pair and restores the prior focus+/// afterwards; global input is refused when no adapter can confirm it. On+/// X11 [`with_x11_foreground`] activates the window and leaves it active.+/// [`RestoreMode::LeaveFocused`] runs `body` as it is: it activates the+/// target itself and leaves it focused.+pub fn with_target_foreground<T>(+ pid: u32,+ window_id: u64,+ mode: RestoreMode,+ body: impl FnOnce() -> Result<T>,+) -> Result<T> {+ use crate::wayland::{bind_foreground_target, kwin_helper, niri, shell_helper, sway_ipc};+ if let RestoreMode::LeaveFocused = mode {+ admit_lock(+ policy::Scope::TargetBoundInput,+ facts::Fact::Unknown("the body activates the target itself".into()),+ )?;+ return body();+ }+ if std::env::var_os("WAYLAND_DISPLAY").is_none() {+ return with_x11_foreground(window_id, ForegroundOptions::pointer(), body)+ .map(|(value, _report)| value);+ }+ let body = || {+ let focused = if niri::is_session() {+ match niri::keyboard_focus() {+ Ok(focused) => facts::Fact::Known(focused == Some(window_id)),+ Err(error) => facts::Fact::Unknown(format!("{error:#}")),+ }+ } else {+ facts::Fact::Known(true)+ };+ admit_lock(policy::Scope::WindowInput, focused)?;+ body()+ };+ let _target_guard = bind_foreground_target(pid, window_id);+ let confirm = || operation_deadline(FOCUS_CONFIRM_CAP);+ if niri::is_session() {+ return transaction(+ &mut niri::ipc(),+ |ipc| niri::save_focus(ipc, pid, window_id),+ |ipc, saved| niri::focus_saved(ipc, saved, niri::FOCUS_CONFIRM_TIMEOUT),+ body,+ niri::restore_saved,+ );+ }+ if let Some(window) = sway_ipc::window_for_id(window_id) {+ policy::identity(+ &(pid, window_id),+ &facts::Fact::Known(Some((window.pid, window.id))),+ )?;+ return transaction(+ &mut (),+ |()| {+ let prior = sway_ipc::focused(confirm())?;+ if policy::focused(&(pid, window_id), &facts::Fact::Known(prior)) {+ return Ok(None);+ }+ Ok(Some(prior))+ },+ |(), _| sway_ipc::focus(pid, window_id, confirm()),+ body,+ |(), prior, cleanup| {+ prior.map_or(Ok(()), |(pid, prior)| sway_ipc::focus(pid, prior, cleanup))+ },+ );+ }+ if kwin_helper::trusted_window_for_id(pid, window_id).is_some() {+ policy::delivery(&facts::Fact::Known(kwin_helper::available()))?;+ }+ if shell_helper::trusted_window_for_id(pid, window_id).is_some() {+ return transaction(+ &mut (),+ |()| {+ let prior = shell_helper::trusted_focus(confirm())?;+ if policy::focused(&(Some(pid), window_id), &facts::Fact::Known(prior)) {+ return Ok(None);+ }+ policy::restore(facts::Fact::Known(+ prior.and_then(|(pid, id)| pid.map(|pid| (pid, id))),+ ))+ .map(Some)+ },+ |(), _| shell_helper::trusted_focus_window(pid, window_id, confirm()),+ body,+ |(), (pid, prior), cleanup| shell_helper::trusted_focus_window(pid, prior, cleanup),+ );+ }+ anyhow::bail!(+ "foreground_unavailable: no trusted Wayland compositor adapter can confirm exact target window {window_id} for pid {pid}; no global keyboard input was sent"+ )+}++/// The X11 foreground transaction (`delivery_mode:"foreground"`): activate+/// `xid` and read back that it holds the active window and input focus,+/// admit the lock for window input, run `body`, then report where focus and+/// the target process's windows ended up. The target is left active.+pub fn with_x11_foreground<T>(+ xid: u64,+ opts: ForegroundOptions,+ body: impl FnOnce() -> Result<T>,+) -> Result<(T, ForegroundReport)> {+ let activated = crate::input::foreground::activate(xid, opts)?;+ admit_lock(policy::Scope::WindowInput, facts::Fact::Known(true))?;+ let value = body()?;+ Ok((value, activated.report()))+}++/// How long a compositor may take to apply and report a focus change.+const FOCUS_CONFIRM_CAP: Duration = Duration::from_millis(500);++/// The one foreground transaction every adapter runs: save the prior focus,+/// focus the target and read it back, run `body`, then restore the saved+/// focus within [`CLEANUP_CAP`]. `save` returns `None` when the target+/// already has focus, which runs `body` with nothing to restore. Once+/// `focus` was asked, focus may have moved even when the adapter could not+/// confirm it, so the restore runs either way. One transaction runs at a+/// time: a worker whose call timed out still gives focus back late, and a+/// retry on the same window would read that window as the prior focus and+/// lose it to the late restore.+pub(crate) fn transaction<A, S, T>(+ adapter: &mut A,+ save: impl FnOnce(&mut A) -> Result<Option<S>>,+ focus: impl FnOnce(&mut A, &S) -> Result<()>,+ body: impl FnOnce() -> Result<T>,+ restore: impl FnOnce(&mut A, S, Instant) -> Result<()>,+) -> Result<T> {+ let _transaction = hold_transaction()?;+ let Some(saved) = save(adapter)? else {+ return body();+ };+ let result = focus(adapter, &saved).and_then(|()| body());+ let cleanup = Instant::now() + CLEANUP_CAP;+ match (result, restore(adapter, saved, cleanup)) {+ (Ok(value), Ok(())) => Ok(value),+ (Err(error), Ok(())) => Err(error),+ // The body sent its input: say so, so the caller does not repeat it.+ (Ok(_), Err(error)) => {+ Err(error.context("the input was sent, but the prior focus could not be restored"))+ }+ (Err(error), Err(restore)) => {+ let note = format!("{error}; the prior focus also could not be restored: {restore:#}");+ Err(error.context(note))+ }+ }+}++/// One foreground transaction runs at a time across threads.+static TRANSACTION: std::sync::Mutex<()> = std::sync::Mutex::new(());++std::thread_local! {+ /// Whether this thread already holds the transaction. A body that opens+ /// another foreground transaction — the libei fallback of+ /// [`crate::wayland::click_focused`] re-enters the guard after a blocking+ /// portal readiness wait — runs inside the one it already holds, so it+ /// must not wait for the mutex its own thread took.+ static HOLDS_TRANSACTION: std::cell::Cell<bool> = const { std::cell::Cell::new(false) };+}++/// Wait for any other foreground transaction to give focus back, then hold+/// this one until it does. The holder's waits are bounded, and a caller+/// whose own call ends stops waiting.+fn hold_transaction() -> Result<HeldTransaction> {+ let already_held = HOLDS_TRANSACTION+ .try_with(std::cell::Cell::get)+ .unwrap_or(false);+ if already_held {+ // The outer transaction already holds focus for its body, so this one+ // takes no lock and its drop must not release the outer one.+ return Ok(HeldTransaction { guard: None });+ }+ let guard = loop {+ match TRANSACTION.try_lock() {+ Ok(held) => break held,+ Err(std::sync::TryLockError::Poisoned(poisoned)) => break poisoned.into_inner(),+ Err(std::sync::TryLockError::WouldBlock) => {+ crate::input::action::admit(|| ())?;+ pause(POLL);+ }+ }+ };+ let _ = HOLDS_TRANSACTION.try_with(|held| held.set(true));+ Ok(HeldTransaction { guard: Some(guard) })+}++/// The foreground transaction this thread holds. Only the outermost+/// transaction locks [`TRANSACTION`]; a nested one holds nothing and releases+/// nothing, so the outer transaction still runs last.+struct HeldTransaction {+ guard: Option<std::sync::MutexGuard<'static, ()>>,+}++impl Drop for HeldTransaction {+ fn drop(&mut self) {+ let released = self.guard.take();+ if released.is_some() {+ let _ = HOLDS_TRANSACTION.try_with(|held| held.set(false));+ }+ drop(released);+ }+}++/// The one budget every cleanup gets once its action is over, even past+/// the action's deadline: held inputs are released first, then focus is+/// restored, all within this cap from the start of cleanup.+pub const CLEANUP_CAP: Duration = Duration::from_millis(500);++const POLL: Duration = Duration::from_millis(10);+const SETTLE_READS: usize = 5;+const SETTLE_INTERVAL: Duration = Duration::from_millis(30);+const SETTLE_CAP: Duration = Duration::from_secs(1);++/// The deadline of an operation capped at `cap`, never past the deadline of+/// the action bound to this thread. Cleanup that must outlive the action+/// (releases, focus restore) takes `Instant::now() + cap` instead.+pub fn operation_deadline(cap: Duration) -> Instant {+ let capped = Instant::now() + cap;+ crate::input::action::current().map_or(capped, |action| capped.min(action.deadline()))+}++/// Sleep for `cap`, never past the deadline of the action bound to this+/// thread. Input pacing waits here, so a call out of time stops waiting.+pub fn pause(cap: Duration) {+ std::thread::sleep(operation_deadline(cap).saturating_duration_since(Instant::now()));+}++/// Read `confirmed` back until it holds or `deadline` passes. The callback+/// receives the same absolute deadline for its own round trips.+/// A compositor that accepts a request has not applied it: only the+/// read-back says it did.+pub fn confirm<E>(+ deadline: Instant,+ mut confirmed: impl FnMut(Instant) -> Result<bool, E>,+) -> Result<bool, E> {+ while Instant::now() < deadline {+ if confirmed(deadline)? {+ return Ok(Instant::now() < deadline);+ }+ std::thread::sleep(POLL.min(deadline.saturating_duration_since(Instant::now())));+ }+ Ok(false)+}++/// Read `observe` until two consecutive reads agree on `key`, a few reads a+/// fixed interval apart. Callbacks receive the action-capped deadline and+/// must bound their round trips by it; late results refuse with `unsettled`.+/// Agreeing reads do not prove an animation finished.+pub fn settled<T, K: PartialEq, E>(+ mut observe: impl FnMut(Instant) -> Result<T, E>,+ key: impl Fn(&T) -> K,+ unsettled: impl FnOnce() -> E,+) -> Result<T, E> {+ let deadline = operation_deadline(SETTLE_CAP);+ if Instant::now() >= deadline {+ return Err(unsettled());+ }+ let mut last = observe(deadline)?;+ for _ in 1..SETTLE_READS {+ if Instant::now() + SETTLE_INTERVAL > deadline {+ break;+ }+ std::thread::sleep(SETTLE_INTERVAL);+ if Instant::now() >= deadline {+ break;+ }+ let next = observe(deadline)?;+ if Instant::now() < deadline+ && policy::settled(&key(&last), &facts::Fact::Known(key(&next)))+ {+ return Ok(next);+ }+ last = next;+ }+ Err(unsettled())+}++#[cfg(test)]+mod tests {+ use super::*;+ use crate::input::action::{bind, Action};++ #[test]+ fn the_operation_deadline_never_passes_the_bound_action() {+ let unbound = operation_deadline(Duration::from_secs(5));+ assert!(unbound > Instant::now() + Duration::from_secs(4));+ let action = Action::new(Duration::from_millis(20));+ let _bound = bind(action.clone());+ assert!(operation_deadline(Duration::from_secs(5)) <= action.deadline());+ }++ #[test]+ fn leave_focused_runs_the_body_without_a_compositor_transaction() {+ let ran = with_target_foreground(1, 2, RestoreMode::LeaveFocused, || Ok("sent"));+ assert_eq!(ran.unwrap(), "sent");+ }++ #[test]+ fn confirm_reads_back_until_it_holds_and_never_starts_past_the_deadline() {+ let mut reads = 0;+ let deadline = Instant::now() + Duration::from_secs(1);+ let held = confirm(deadline, |received| {+ assert_eq!(received, deadline);+ reads += 1;+ Ok::<_, ()>(reads == 3)+ });+ assert_eq!((held, reads), (Ok(true), 3));+ let mut reads = 0;+ let expired = confirm(Instant::now(), |_| {+ reads += 1;+ Ok::<_, ()>(false)+ });+ assert_eq!((expired, reads), (Ok(false), 0));+ assert_eq!(+ confirm(Instant::now() + Duration::from_secs(1), |_| Err::<bool, _>(+ "unknown"+ )),+ Err("unknown")+ );+ }++ #[test]+ fn a_late_focus_read_back_does_not_confirm_delivery() {+ let deadline = Instant::now() + Duration::from_millis(10);+ assert_eq!(+ confirm(deadline, |received| {+ assert_eq!(received, deadline);+ std::thread::sleep(Duration::from_millis(20));+ Ok::<_, ()>(true)+ }),+ Ok(false)+ );+ }++ #[test]+ fn settle_stops_at_the_action_deadline() {+ let _bound = bind(Action::new(Duration::ZERO));+ let mut reads = 0;+ let moving = settled(+ |_| {+ reads += 1;+ Ok::<_, &str>(reads)+ },+ |read| *read,+ || "unsettled",+ );+ assert_eq!((moving, reads), (Err("unsettled"), 0));+ }++ #[test]+ fn a_nested_transaction_runs_inside_the_one_its_thread_holds() {+ // A GNOME click reaches `click_focused`'s libei fallback from inside+ // the foreground guard it already holds and re-enters it there. The+ // action is bound only around the nested call, as the outer caller's+ // is: binding it around the outer transaction instead would let a+ // slower test holding the process-wide lock fail this one.+ let sent = transaction(+ &mut (),+ |()| Ok(Some("prior")),+ |(), _| Ok(()),+ || {+ let _bound = bind(Action::new(Duration::from_millis(200)));+ transaction(+ &mut (),+ |()| Ok(Some("prior")),+ |(), _| Ok(()),+ || Ok("click sent"),+ |(), _, _| Ok(()),+ )+ },+ |(), _, _| Ok(()),+ );+ assert_eq!(sent.unwrap(), "click sent");+ }++ #[test]+ fn only_the_outermost_transaction_locks_and_its_drop_releases_it() {+ let outer = hold_transaction().unwrap();+ assert!(outer.guard.is_some());+ // This thread already holds the transaction, so the nested hold never+ // waits on another thread. The action bounds it anyway: a hold that+ // waited for its own lock fails here instead of hanging the suite.+ let _bound = bind(Action::new(Duration::from_millis(200)));+ drop(hold_transaction().unwrap());+ assert!(+ outer.guard.is_some(),+ "a nested hold released the outer transaction"+ );+ let (entered, entered_rx) = std::sync::mpsc::channel();+ let other = std::thread::spawn(move || {+ let held = hold_transaction();+ entered.send(held.is_ok()).unwrap();+ });+ assert!(+ entered_rx.recv_timeout(Duration::from_millis(100)).is_err(),+ "a second thread entered the transaction its holder still holds"+ );+ drop(outer);+ assert!(entered_rx.recv_timeout(Duration::from_secs(2)).unwrap());+ other.join().unwrap();+ }++ #[test]+ fn settle_returns_the_first_of_two_agreeing_reads() {+ let mut reads = 0;+ let settling = settled(+ |_| {+ reads += 1;+ Ok::<_, &str>(reads.min(2))+ },+ |read| *read,+ || "unsettled",+ );+ assert_eq!(settling, Ok(2));+ }+}diff --git a/libs/cua-driver/rust/crates/platform-linux/src/interaction/policy.rs b/libs/cua-driver/rust/crates/platform-linux/src/interaction/policy.rsnew file mode 100644index 0000000000..d79f12d44c--- /dev/null+++ b/libs/cua-driver/rust/crates/platform-linux/src/interaction/policy.rs@@ -0,0 +1,384 @@+//! What each primitive decides from the facts an adapter reports. Every+//! decision states what an unknown fact means for each scope.++use anyhow::{bail, Result};++use super::facts::{Fact, Facts};+use crate::input::FocusAfter;++/// What an operation addresses.+#[derive(Clone, Copy)]+pub enum Scope {+ WindowInput,+ /// Window input whose route activates or addresses the target itself,+ /// without the shared transaction's focus read-back.+ TargetBoundInput,+ DesktopInput,+ Capture,+}++/// Whether the session lock admits the operation. A known lock refuses+/// everything. An unknown lock admits window input only when the target's+/// exact focus was read back as `Known(true)`, and admits target-bound input, desktop input+/// and capture on the adapter's other checks, as before this rule existed.+pub fn lock(scope: Scope, facts: &Facts) -> Result<()> {+ match (&facts.locked, scope) {+ (Fact::Known(true), _) => {+ bail!("screen_locked: the session screensaver reports itself active, so nothing was sent or captured")+ }+ (Fact::Known(false), _)+ | (+ Fact::Unknown(_),+ Scope::TargetBoundInput | Scope::DesktopInput | Scope::Capture,+ ) => Ok(()),+ (Fact::Unknown(_), Scope::WindowInput) if matches!(facts.focused, Fact::Known(true)) => {+ Ok(())+ }+ (Fact::Unknown(why), Scope::WindowInput) => bail!(+ "foreground_unavailable: the lock state is unknown ({why}) and the target's focus was not read back"+ ),+ }+}++/// Require an exact observed identity. Missing, mismatched and unknown+/// identities refuse; focus or geometry cannot substitute for ownership.+pub fn identity<T: PartialEq>(expected: &T, observed: &Fact<Option<T>>) -> Result<()> {+ match observed {+ Fact::Known(Some(actual)) if actual == expected => Ok(()),+ Fact::Known(_) => {+ bail!("foreground_unavailable: the exact target identity was not observed")+ }+ Fact::Unknown(why) => bail!("foreground_unavailable: target identity is unknown ({why})"),+ }+}++/// Whether the adapter's input transport can bind delivery to the confirmed+/// target. A transport that only reaches whatever surface is focused when+/// the compositor processes the event refuses, as does an unknown one;+/// more focus checks before dispatch do not bind the input.+pub fn delivery(target_bound: &Fact<bool>) -> Result<()> {+ match target_bound {+ Fact::Known(true) => Ok(()),+ Fact::Known(false) => bail!(+ "foreground_unavailable: the target window is verified, but this compositor offers only focus-bound input; refusing raw input until a target-bound input path exists"+ ),+ Fact::Unknown(why) => {+ bail!("foreground_unavailable: the input binding is unknown ({why}); refusing raw input")+ }+ }+}++/// An exact focus read-back permits delivery. A known different or absent+/// focus needs activation and another read-back; unknown focus cannot+/// confirm delivery. Command acceptance alone is not a focus fact.+pub fn focused<T: PartialEq>(expected: &T, observed: &Fact<Option<T>>) -> bool {+ identity(expected, observed).is_ok()+}++/// Whether an admitted observation still holds. No adapter counts layout+/// events yet, so this is the weaker snapshot check: the current geometry+/// equals the admitted one, which cannot see a move away and back between+/// the two reads. A changed or unknown current geometry is stale.+pub fn fresh<G: PartialEq>(admitted: &G, current: &Fact<G>) -> bool {+ matches!(current, Fact::Known(current) if current == admitted)+}++/// What an adapter observed about one point of window-coordinate input+/// against the target window's current geometry.+pub struct Point {+ /// The window still has the geometry the point was computed on.+ pub fresh: bool,+ /// The point lies on the window.+ pub on_window: bool,+ /// The output shows the window at the point.+ pub shown: bool,+ /// No other window may be above the window at the point.+ pub unobscured: bool,+}++/// Why a point of window-coordinate input is refused.+#[derive(Clone, Copy, Debug, PartialEq, Eq)]+pub enum Miss {+ Stale,+ OffWindow,+ Hidden,+ Obscured,+}++/// Whether a point reaches the target window: it must have been computed on+/// the window's current geometry, lie on the window, be shown, and not be+/// possibly covered by another window. The first fact that fails, in that+/// order, is the refusal.+pub fn reach(point: &Point) -> Result<(), Miss> {+ match point {+ Point { fresh: false, .. } => Err(Miss::Stale),+ Point {+ on_window: false, ..+ } => Err(Miss::OffWindow),+ Point { shown: false, .. } => Err(Miss::Hidden),+ Point {+ unobscured: false, ..+ } => Err(Miss::Obscured),+ _ => Ok(()),+ }+}++/// Whether observable geometry stopped moving: two consecutive reads agree.+/// A read that differs, or one that could not be taken, is still moving;+/// agreeing reads do not prove a rendered animation finished.+pub fn settled<K: PartialEq>(last: &K, next: &Fact<K>) -> bool {+ matches!(next, Fact::Known(next) if next == last)+}++/// The saved focus a `RestorePrevious` transaction gives back. Without a+/// valid saved recipe, or with an unknown one, the transaction does not+/// begin, so nothing is focused that could not be restored.+pub fn restore<R>(saved: Fact<Option<R>>) -> Result<R> {+ match saved {+ Fact::Known(Some(recipe)) => Ok(recipe),+ Fact::Known(None) => bail!(+ "foreground_unavailable: the compositor exposes no restorable prior focus, so nothing was focused"+ ),+ Fact::Unknown(why) => {+ bail!("foreground_unavailable: {why}")+ }+ }+}++/// The focus a restore leaves alone: focus that moved during the operation+/// to something that is neither the target nor the saved prior is the+/// user's newer choice. Returns that choice; `None` means restore.+pub fn newer_choice<T: PartialEq>(+ current: Option<T>,+ target: Option<T>,+ prior: Option<T>,+) -> Option<T> {+ current.filter(|current| Some(current) != target.as_ref() && Some(current) != prior.as_ref())+}++/// What a background action's focus guard does with focus that changed.+#[derive(Clone, Copy, Debug, PartialEq, Eq)]+pub enum GuardRestore {+ /// The action closed the window that held the focus: nothing to restore.+ PriorClosed,+ /// The target app owned the focus before and still does: it opened one+ /// of its own windows (a dialog). Not an intrusion; leave it.+ SameAppDialog,+ /// A popup the action opened holds a keyboard grab: restoring cannot+ /// take effect until it closes, and closing it would undo the action.+ GrabHeld,+ Restore,+}++/// Whether a background action's changed focus is restored. The same-app+/// rule applies only when the target pid owned the focus before the action+/// *and* owns it now: a move from a decoy (another pid) into the target's+/// dialog is a steal, as is a move from the target to anything else.+/// Unknown owners never qualify.+pub fn guard_restore(+ prior_closed: bool,+ target_pid: Option<u32>,+ previous_owner: Option<u32>,+ new_owner: Option<u32>,+ grab_held: bool,+) -> GuardRestore {+ match (target_pid, previous_owner, new_owner) {+ _ if prior_closed => GuardRestore::PriorClosed,+ (Some(target), Some(previous), Some(new)) if previous == target && new == target => {+ GuardRestore::SameAppDialog+ }+ _ if grab_held => GuardRestore::GrabHeld,+ _ => GuardRestore::Restore,+ }+}++/// Where focus landed after an X11 foreground body, from where the focused+/// window sits: inside the target, inside another window of the target+/// process, or owned by the target's pid. The pid was read before the body,+/// so a body that closed the target (Escape on a dialog, alt+F4, a popup+/// item click) still compares. Unknown owners are elsewhere.+pub fn focus_after(+ in_target: bool,+ in_target_process_window: bool,+ target_pid: Option<u32>,+ focused_pid: Option<u32>,+) -> FocusAfter {+ match (target_pid, focused_pid) {+ _ if in_target => FocusAfter::Target,+ _ if in_target_process_window => FocusAfter::SamePid,+ (Some(target), Some(focused)) if target == focused => FocusAfter::SamePid,+ _ => FocusAfter::Elsewhere,+ }+}++#[cfg(test)]+mod tests {+ use super::*;++ #[test]+ fn focus_after_prefers_the_target_then_its_process_and_never_guesses_an_owner() {+ assert_eq!(focus_after(true, false, None, None), FocusAfter::Target);+ assert_eq!(focus_after(false, true, None, None), FocusAfter::SamePid);+ assert_eq!(+ focus_after(false, false, Some(7), Some(7)),+ FocusAfter::SamePid+ );+ for (target, focused) in [(Some(7), Some(8)), (Some(7), None), (None, Some(7))] {+ assert_eq!(+ focus_after(false, false, target, focused),+ FocusAfter::Elsewhere+ );+ }+ }++ #[test]+ fn a_restore_leaves_focus_only_on_a_third_choice() {+ assert_eq!(newer_choice(Some(3), Some(1), Some(2)), Some(3));+ assert_eq!(newer_choice(Some(1), Some(1), Some(2)), None);+ assert_eq!(newer_choice(Some(2), Some(1), Some(2)), None);+ assert_eq!(newer_choice(None, Some(1), Some(2)), None);+ assert_eq!(newer_choice(Some(3), None, None), Some(3));+ }++ #[test]+ fn same_app_rule_only_when_the_target_owned_the_focus_before_and_after() {+ assert_eq!(+ guard_restore(false, Some(7), Some(7), Some(7), true),+ GuardRestore::SameAppDialog+ );+ assert_eq!(+ guard_restore(true, Some(7), Some(7), Some(7), true),+ GuardRestore::PriorClosed+ );+ for (target, previous, new) in [+ (Some(7), Some(3), Some(7)),+ (Some(7), Some(7), Some(3)),+ (Some(7), None, Some(7)),+ (Some(7), Some(7), None),+ (None, Some(7), Some(7)),+ ] {+ assert_eq!(+ guard_restore(false, target, previous, new, false),+ GuardRestore::Restore+ );+ assert_eq!(+ guard_restore(false, target, previous, new, true),+ GuardRestore::GrabHeld+ );+ }+ }++ fn unknown<T>() -> Fact<T> {+ Fact::Unknown("no source".into())+ }++ fn facts(locked: Fact<bool>, focused: Fact<bool>) -> Facts {+ Facts { locked, focused }+ }++ #[test]+ fn a_known_lock_refuses_every_scope_and_a_known_unlock_admits_every_scope() {+ for scope in [+ Scope::WindowInput,+ Scope::TargetBoundInput,+ Scope::DesktopInput,+ Scope::Capture,+ ] {+ let refused = lock(scope, &facts(Fact::Known(true), Fact::Known(true))).unwrap_err();+ assert!(refused.to_string().starts_with("screen_locked:"));+ lock(scope, &facts(Fact::Known(false), Fact::Known(false))).unwrap();+ }+ }++ #[test]+ fn an_unknown_lock_admits_window_input_only_after_focus_read_back() {+ lock(Scope::WindowInput, &facts(unknown(), Fact::Known(true))).unwrap();+ assert!(lock(Scope::WindowInput, &facts(unknown(), unknown())).is_err());+ assert_eq!(+ lock(Scope::WindowInput, &facts(unknown(), Fact::Known(false))).unwrap_err().to_string(),+ "foreground_unavailable: the lock state is unknown (no source) and the target's focus was not read back"+ );+ }++ #[test]+ fn an_unknown_lock_admits_target_bound_input_desktop_input_and_capture() {+ lock(Scope::TargetBoundInput, &facts(unknown(), unknown())).unwrap();+ lock(Scope::DesktopInput, &facts(unknown(), unknown())).unwrap();+ lock(Scope::Capture, &facts(unknown(), unknown())).unwrap();+ }++ #[test]+ fn delivery_requires_a_known_target_bound_transport() {+ delivery(&Fact::Known(true)).unwrap();+ for binding in [Fact::Known(false), unknown()] {+ assert!(delivery(&binding)+ .unwrap_err()+ .to_string()+ .starts_with("foreground_unavailable:"));+ }+ }++ #[test]+ fn only_an_unchanged_known_geometry_is_fresh() {+ assert!(fresh(&(100, 60), &Fact::Known((100, 60))));+ assert!(!fresh(&(100, 60), &Fact::Known((120, 60))));+ assert!(!fresh(&(100, 60), &unknown()));+ }++ #[test]+ fn a_point_reaches_the_window_only_when_every_fact_holds_and_the_first_failure_refuses() {+ let point = |fresh, on_window, shown, unobscured| Point {+ fresh,+ on_window,+ shown,+ unobscured,+ };+ assert_eq!(reach(&point(true, true, true, true)), Ok(()));+ assert_eq!(reach(&point(false, false, false, false)), Err(Miss::Stale));+ assert_eq!(+ reach(&point(true, false, false, false)),+ Err(Miss::OffWindow)+ );+ assert_eq!(reach(&point(true, true, false, false)), Err(Miss::Hidden));+ assert_eq!(reach(&point(true, true, true, false)), Err(Miss::Obscured));+ }++ #[test]+ fn only_two_agreeing_known_reads_are_settled() {+ assert!(settled(&3, &Fact::Known(3)));+ assert!(!settled(&3, &Fact::Known(4)));+ assert!(!settled(&3, &unknown()));+ }++ #[test]+ fn restore_needs_a_known_saved_focus_before_focusing() {+ assert_eq!(restore(Fact::Known(Some(5))).unwrap(), 5);+ assert_eq!(+ restore::<u64>(Fact::Known(None)).unwrap_err().to_string(),+ "foreground_unavailable: the compositor exposes no restorable prior focus, so nothing was focused"+ );+ assert_eq!(+ restore::<u64>(unknown()).unwrap_err().to_string(),+ "foreground_unavailable: no source"+ );+ }++ #[test]+ fn identity_requires_both_the_process_and_window() {+ let target = (107, 7);+ identity(&target, &Fact::Known(Some(target))).unwrap();+ for observed in [+ Fact::Known(Some((108, 7))),+ Fact::Known(Some((107, 8))),+ Fact::Known(None),+ unknown(),+ ] {+ assert!(identity(&target, &observed)+ .unwrap_err()+ .to_string()+ .starts_with("foreground_unavailable:"));+ assert!(!focused(&target, &observed));+ }+ assert!(focused(&target, &Fact::Known(Some(target))));+ }+}diff --git a/libs/cua-driver/rust/crates/platform-linux/src/lib.rs b/libs/cua-driver/rust/crates/platform-linux/src/lib.rsindex fc866c39a6..0a84635850 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/lib.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/lib.rs@@ -61,6 +61,8 @@ pub mod capture;mod capture_action_frame;#[cfg(target_os = "linux")]mod clipboard;+#[cfg(target_os = "linux")]+mod interaction;#[cfg(target_os = "linux")]pub mod atspi;@@ -136,6 +138,11 @@ pub(crate) mod test_env {/// `WAYLAND_DISPLAY`, so the X11 path is taken and its connect fails/// regardless of the host session.pub(crate) fn unreachable_x11_display() -> DisplayOverride {+ x11_display(":9999999")+ }++ /// Point `DISPLAY` at `display` and hide `WAYLAND_DISPLAY`.+ pub(crate) fn x11_display(display: &str) -> DisplayOverride {let lock = ENV_LOCK.lock().unwrap_or_else(|poisoned| poisoned.into_inner());@@ -144,7 +151,7 @@ pub(crate) mod test_env {wayland_display: std::env::var_os("WAYLAND_DISPLAY"),_lock: lock,};- std::env::set_var("DISPLAY", ":9999999");+ std::env::set_var("DISPLAY", display);std::env::remove_var("WAYLAND_DISPLAY");guard}diff --git a/libs/cua-driver/rust/crates/platform-linux/src/recording_hooks.rs b/libs/cua-driver/rust/crates/platform-linux/src/recording_hooks.rsindex a87cd8ee79..9fff9eb906 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/recording_hooks.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/recording_hooks.rs@@ -208,7 +208,10 @@ fn resolve_window_for_recording(window_id: Option<u64>,) -> Option<crate::x11::WindowInfo> {let windows = crate::wayland::list_windows_dispatch(Some(pid));- if crate::wayland::is_wayland() && crate::wayland::hyprland::is_session() {+ // Hyprland addresses and niri window ids are stable compositor ids.+ if crate::wayland::is_wayland()+ && (crate::wayland::hyprland::is_session() || crate::wayland::niri::is_session())+ {return match window_id {Some(id) => windows.into_iter().find(|window| window.xid == id),None if windows.len() == 1 => windows.into_iter().next(),diff --git a/libs/cua-driver/rust/crates/platform-linux/src/tools/impl_.rs b/libs/cua-driver/rust/crates/platform-linux/src/tools/impl_.rsindex bd51d47a33..02b72b5e29 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/tools/impl_.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/tools/impl_.rs@@ -17,6 +17,7 @@ use std::path::PathBuf;use std::sync::{Arc, RwLock};use crate::atspi::Snapshots;+use crate::wayland::AimedPoint;use cursor_overlay::CursorRegistry;fn window_target_candidates_for_pid(@@ -54,6 +55,9 @@ fn desktop_point_window_resolver() -> cua_driver_core::window_target::DesktopPoilet (x, y) = if crate::wayland::is_wayland() && crate::wayland::hyprland::is_session() {let frame = crate::wayland::hyprland::desktop_frame().ok()?;(x + f64::from(frame.x), y + f64::from(frame.y))+ } else if crate::wayland::is_wayland() && crate::wayland::niri::is_session() {+ let frame = crate::wayland::niri::desktop_snapshot().ok()?.frame;+ (x + f64::from(frame.x), y + f64::from(frame.y))} else {(x, y)};@@ -93,6 +97,9 @@ fn topmost_window_at(fn pid_fallback_window_resolver() -> cua_driver_core::window_target::PidFallbackWindowResolver {Arc::new(|pid| {let pid = u32::try_from(pid).ok()?;+ if crate::wayland::is_wayland() && crate::wayland::niri::is_session() {+ return crate::wayland::niri::window_in_use(pid);+ }let windows: Vec<crate::x11::WindowInfo> = crate::wayland::list_windows_dispatch(Some(pid)).into_iter().filter(|w| w.pid == Some(pid))@@ -262,6 +269,14 @@ fn screenshot_scale(state.snapshots.screenshot_scale(pid as i32, window_id, args)+ .map_err(|missing| {+ // A tiled niri window never has a screenshot: say why instead of+ // asking for the get_window_state read that cannot give one.+ window_id+ .filter(|_| crate::wayland::is_wayland() && crate::wayland::niri::is_session())+ .and_then(crate::wayland::niri::tiled_refusal)+ .map_or(missing, input_error_result)+ })}// ── list_apps ────────────────────────────────────────────────────────────────@@ -2694,35 +2709,269 @@ fn desktop_frame_requested(args: &Value) -> bool {/// Convert desktop (screen) coordinates to `xid`'s window-local frame so the/// rest of the pointer pipeline (which is window-local by contract) is reused.fn desktop_to_window_local(xid: u64, x: f64, y: f64) -> anyhow::Result<(f64, f64)> {+ // A niri window id is no X11 window: never ask XWayland about it.+ if crate::wayland::is_wayland() && crate::wayland::niri::is_session() {+ return Err(crate::wayland::niri::desktop_frame_window_refusal(xid));+ }let (ox, oy) = window_local_to_screen(xid, 0.0, 0.0)?;Ok((x - ox, y - oy))}+/// A failed desktop-to-window mapping; a coded refusal keeps its structure.+fn desktop_frame_unmapped(xid: u64, error: anyhow::Error) -> ToolResult {+ if crate::input::foreground::error_code(&error).is_some() {+ return input_error_result(error);+ }+ ToolResult::error(format!(+ "desktop-frame coordinates could not be mapped into window {xid}: {error}"+ ))+}++#[cfg(test)]+#[test]+fn a_niri_desktop_frame_window_refusal_is_structured() {+ let refused =+ desktop_frame_unmapped(31, crate::wayland::niri::desktop_frame_window_refusal(31));+ let structured = refused.structured_content.unwrap();+ assert_eq!(structured["code"], "foreground_unavailable");+ assert_eq!(structured["effect"], "none");+ assert!(structured["suggestion"]+ .as_str()+ .unwrap()+ .starts_with("Drop pid and window_id"));+}+/// Upper bound for one foreground input transaction: focus confirmation plus/// the injection itself. Typing sleeps per character, so scale with `chars`.fn foreground_budget(chars: usize) -> std::time::Duration {std::time::Duration::from_millis(12_000 + (chars as u64) * 40)}-/// `spawn_blocking` with a hard deadline. A wedged X server (or a client-/// holding a server grab) must surface as a structured `foreground_timeout`-/// error instead of a tool call that never returns. The blocking task is not-/// cancellable; it is left to finish (or leak) on the blocking pool.-async fn spawn_blocking_bounded<T: Send + 'static>(- label: &'static str,- budget: std::time::Duration,- f: impl FnOnce() -> anyhow::Result<T> + Send + 'static,-) -> Result<anyhow::Result<T>, tokio::task::JoinError> {- match tokio::time::timeout(budget, tokio::task::spawn_blocking(f)).await {- Ok(joined) => joined,- Err(_elapsed) => Ok(Err(anyhow::anyhow!(- "{}: {label} did not complete within {budget:?}; the X server or the target \- application may be wedged. Take a screenshot before retrying",- crate::input::foreground::CODE_TIMEOUT- ))),+/// The budget of a `type_text` call that types `chars` characters. niri's+/// keyboard gives itself its own deadline, which starts once the window is+/// focused and stops the call with how far it got; the call allows that+/// plus the foreground setup, so that deadline ends it first.+fn typing_budget(chars: usize, niri: bool) -> std::time::Duration {+ let budget = foreground_budget(chars);+ if niri {+ budget.max(foreground_budget(0) + crate::wayland::niri_keyboard_budget(chars))+ } else {+ budget+ }+}++#[cfg(test)]+#[test]+fn niri_typing_outlasts_the_keyboard_deadline() {+ // niri's keyboard allows itself a deadline per key and reports how far+ // it got when that passes. A call budget that ran out first would cut+ // the text off with a bare timeout, and a retry would type its start+ // again.+ for chars in [0, 1, 150, 1_000, 100_000] {+ assert!(+ typing_budget(chars, true) > crate::wayland::niri_keyboard_budget(chars),+ "{chars} characters"+ );+ assert_eq!(typing_budget(chars, false), foreground_budget(chars));+ }+}++/// [`foreground_budget`] for `chars` characters with `delay_ms` between them.+/// The delay comes from the caller unbounded.+fn paced_typing_budget(chars: usize, delay_ms: u64) -> std::time::Duration {+ foreground_budget(chars)+ + std::time::Duration::from_millis(delay_ms.saturating_mul(chars as u64))+}++#[cfg(test)]+#[test]+fn any_typing_delay_gives_a_budget() {+ assert_eq!(+ paced_typing_budget(2, 30),+ foreground_budget(2) + std::time::Duration::from_millis(60)+ );+ // Sizing the call must not panic, however long the caller's delay.+ let longest = paced_typing_budget(3, u64::MAX);+ assert!(longest >= std::time::Duration::from_millis(u64::MAX));+ crate::input::action::Action::new(longest);+}++use crate::input::action::spawn_blocking_bounded;++#[cfg(test)]+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]+async fn a_timed_out_call_holds_the_desktop_until_its_late_cleanup_is_done() {+ use cua_driver_core::tool::DesktopCall;+ // niri: call 1 focused its window, then waited in niri past its budget.+ // Its worker gives focus back afterwards. Desktop typing that started in+ // between would have focus moved away under it.+ let events = Arc::new(std::sync::Mutex::new(Vec::new()));+ let call_1 = {+ let events = events.clone();+ DesktopCall::acquire()+ .await+ .unwrap()+ .scope(spawn_blocking_bounded(+ "call 1",+ std::time::Duration::from_millis(50),+ move || {+ std::thread::sleep(std::time::Duration::from_millis(300));+ // The next key is refused, but focus still goes back.+ assert!(crate::input::action::admit(|| ()).is_err());+ events.lock().unwrap().push("call 1 gave focus back");+ Ok(())+ },+ ))+ .await+ .unwrap()+ };+ assert!(call_1.is_err());+ DesktopCall::acquire()+ .await+ .unwrap()+ .scope(async { events.lock().unwrap().push("call 2 typed") })+ .await;+ assert_eq!(+ *events.lock().unwrap(),+ ["call 1 gave focus back", "call 2 typed"]+ );+}++#[cfg(test)]+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]+async fn input_that_cannot_be_cancelled_never_overlaps_the_next_desktop_action() {+ use cua_driver_core::tool::DesktopCall;+ // A wtype child does not read the call's action, and an abandoned XTest+ // call still sends its releases: a worker whose call timed out, or whose+ // caller went away, can still send input. That input must not reach+ // whatever the next desktop action targets.+ for budget in [50, 60_000] {+ let events = Arc::new(std::sync::Mutex::new(Vec::new()));+ let worker_events = events.clone();+ let call = DesktopCall::acquire().await.unwrap();+ let call_1 = async move {+ call.scope(spawn_blocking_bounded(+ "call 1",+ std::time::Duration::from_millis(budget),+ move || {+ std::thread::sleep(std::time::Duration::from_millis(300));+ worker_events.lock().unwrap().push("call 1 typed");+ Ok(())+ },+ ))+ .await+ };+ // A 50 ms budget times out; with 60 s the caller is dropped first.+ let _ = tokio::time::timeout(std::time::Duration::from_millis(100), call_1).await;+ DesktopCall::acquire()+ .await+ .unwrap()+ .scope(async { events.lock().unwrap().push("call 2 typed") })+ .await;+ assert_eq!(+ *events.lock().unwrap(),+ ["call 1 typed", "call 2 typed"],+ "budget {budget} ms"+ );}}+#[cfg(test)]+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]+async fn a_cancelled_x11_press_is_recorded_before_the_next_call_on_its_cursor() {+ use cua_driver_core::tool::DesktopCall;+ let state = ToolState::new();+ let hold = |xid| MouseHoldState {+ pid: 1,+ xid,+ button: 1,+ x: 0.0,+ y: 0.0,+ };+ // Call 1: XTest is slow, and its client cancels the call.+ let call = DesktopCall::acquire().await.unwrap();+ let first = {+ let state = state.clone();+ async move {+ call.scope(press_and_record_x11_hold(+ state,+ "default".to_owned(),+ hold(1),+ || {+ std::thread::sleep(std::time::Duration::from_millis(300));+ Ok(())+ },+ ))+ .await+ }+ };+ let _ = tokio::time::timeout(std::time::Duration::from_millis(50), first).await;+ // Call 2 on the same cursor checks for a hold once it has the desktop.+ let _call = DesktopCall::acquire().await.unwrap();+ let held = state+ .mouse_hold+ .lock()+ .unwrap()+ .get("default")+ .map(|hold| hold.xid);+ assert_eq!(+ held,+ Some(1),+ "call 2 would find no hold, press window 2, and have call 1 overwrite its hold"+ );+}++#[cfg(test)]+#[tokio::test]+async fn a_late_x11_drag_press_is_followed_by_its_release() {+ // The X server answers the press only after the call timed out. The+ // drag then sends its own release, which arrives first.+ let events = Arc::new(std::sync::Mutex::new(Vec::new()));+ let pressed = press_x11_drag_start(+ std::time::Duration::from_millis(50),+ {+ let events = events.clone();+ move || {+ std::thread::sleep(std::time::Duration::from_millis(200));+ events.lock().unwrap().push("press");+ Ok(())+ }+ },+ {+ let events = events.clone();+ move || {+ events.lock().unwrap().push("release");+ Ok(())+ }+ },+ )+ .await+ .unwrap();+ assert!(pressed.is_err());+ events.lock().unwrap().push("release");+ tokio::time::sleep(std::time::Duration::from_millis(400)).await;+ let events = events.lock().unwrap();+ assert!(events.contains(&"press"), "{events:?}");+ assert_eq!(events.last(), Some(&"release"), "{events:?}");+}++/// The `steps` of a `drag` or `mouse_drag`, held to the advertised 1..=200+/// before the button goes down: every step is a motion sent with the+/// button held.+fn drag_steps(args: &Value) -> usize {+ args.u64_or("steps", 20).clamp(1, 200) as usize+}++#[cfg(test)]+#[test]+fn drag_steps_stay_within_the_advertised_bounds() {+ // Without the bound, steps = u32::MAX holds the button through about+ // 4.3 billion motions.+ assert_eq!(drag_steps(&json!({ "steps": u32::MAX })), 200);+ assert_eq!(drag_steps(&json!({ "steps": 0 })), 1);+ assert_eq!(drag_steps(&json!({})), 20);+}+/// Map an input-path error to a tool error, attaching the structured/// foreground `code` (`foreground_unavailable` / `foreground_timeout`) when the/// message carries one so the caller can branch instead of parsing prose.@@ -2749,13 +2998,29 @@ fn input_error_result(e: anyhow::Error) -> ToolResult {"suggestion": if code == crate::input::foreground::CODE_TIMEOUT {"Take a screenshot (get_desktop_state) to confirm the desktop is responsive, then retry once."} else {- "Call bring_to_front on the window (or its modal dialog), confirm with get_window_state, then retry."+ crate::wayland::niri::refusal_suggestion(&text).unwrap_or(+ "Call bring_to_front on the window (or its modal dialog), confirm with get_window_state, then retry.",+ )},})),None => linux_input_error(e),}}+#[cfg(test)]+#[test]+fn a_niri_refusal_suggests_its_own_remedy_instead_of_bring_to_front() {+ let refused = input_error_result(anyhow::anyhow!(+ "foreground_unavailable: the niri layout changed after the get_desktop_state screenshot, so its pixels may be elsewhere now; take a new screenshot"+ ));+ let structured = refused.structured_content.unwrap();+ assert_eq!(structured["code"], "foreground_unavailable");+ assert!(structured["suggestion"]+ .as_str()+ .unwrap()+ .starts_with("Take a new screenshot"));+}+/// Structured payload for a completed foreground transaction.fn foreground_structured(path: &str,@@ -4594,8 +4859,13 @@ fn hyprland_input_result(result: anyhow::Result<Value>, foreground: bool) -> Too.as_str().unwrap_or("input refused").to_owned();+ let refusal = if code == "stale_geometry" {+ "stale_observation"+ } else {+ unavailable+ };value["reason"] = json!(code);- value["code"] = json!(unavailable);+ value["code"] = json!(refusal);value["route"] = json!(if foreground {"global_input"} else {@@ -4623,7 +4893,7 @@ fn hyprland_input_result(result: anyhow::Result<Value>, foreground: bool) -> Too.detail(&detail).build().expect("isolated input outcome is valid");- ToolResult::error(format!("{unavailable} ({code}): {detail}"))+ ToolResult::error(format!("{refusal} ({code}): {detail}")).with_structured(value).with_action_record(outcome)}@@ -5182,14 +5452,14 @@ fn coordinate_less_mouse_button_up_survives_snapshot_replacement() {fn persistent_wayland_pointer_coordinates_include_window_origin() {let output = persistent_pointer_coordinates(true, 7, 10, 20, |window_id, x, y| {assert_eq!(window_id, 7);- (x + 100, y + 200)+ Ok((x + 100, y + 200).into())});- assert_eq!(output, (110, 220));+ assert_eq!(output.unwrap().point, (110, 220));let x11 = persistent_pointer_coordinates(false, 7, 10, 20, |_, _, _| {panic!("X11 coordinates must remain window-local")});- assert_eq!(x11, (10, 20));+ assert_eq!(x11.unwrap().point, (10, 20));}#[cfg(test)]@@ -5319,6 +5589,24 @@ fn isolated_hyprland_refused_partial_and_unknown_outcomes_stay_distinct() {assert!(public.delivery.is_none());public.validate_invariants().unwrap();+ for foreground in [false, true] {+ let stale = hyprland_input_result(+ Ok(json!({"ok":false,"code":"stale_geometry","detail":"stale_geometry"})),+ foreground,+ );+ let value = stale.structured_content.as_ref().unwrap();+ assert_eq!(value["code"], "stale_observation");+ assert_eq!(value["reason"], "stale_geometry");+ assert_eq!(value["effect"], "refused");+ assert!(stale+ .action_record+ .unwrap()+ .public_result()+ .unwrap()+ .delivery+ .is_none());+ }+let partial = isolated_hyprland_result(Ok(json!({"ok":false,"code":"cancelled","detail":"cancelled","effect":"partial","delivery":{"mode":"background","delivered_count":1}}),@@ -5587,12 +5875,12 @@ fn persistent_pointer_coordinates(window_id: u64,x: i32,y: i32,- window_local_to_output: impl FnOnce(u64, i32, i32) -> (i32, i32),-) -> (i32, i32) {+ window_local_to_output: impl FnOnce(u64, i32, i32) -> anyhow::Result<AimedPoint>,+) -> anyhow::Result<AimedPoint> {if is_wayland {window_local_to_output(window_id, x, y)} else {- (x, y)+ Ok((x, y).into())}}@@ -5684,12 +5972,18 @@ async fn overlay_glide_to_target_for(cursor_id: &str, sx: f64, sy: f64, target:/// snapshot converts the action's points, places the overlay and sizes the/// virtual pointer, so they cannot disagree about the monitor layout.async fn desktop_input_space() -> anyhow::Result<crate::wayland::DesktopInputSpace> {- if !crate::wayland::wayland_input_enabled() {- return Ok(crate::wayland::DesktopInputSpace::default());- }- tokio::task::spawn_blocking(crate::wayland::DesktopInputSpace::current)- .await- .map_err(|error| anyhow::anyhow!("task error: {error}"))?+ tokio::task::spawn_blocking(|| {+ crate::interaction::admit_lock(+ crate::interaction::policy::Scope::DesktopInput,+ crate::interaction::facts::Fact::Unknown("desktop input addresses no window".into()),+ )?;+ if !crate::wayland::wayland_input_enabled() {+ return Ok(crate::wayland::DesktopInputSpace::default());+ }+ crate::wayland::DesktopInputSpace::current()+ })+ .await+ .map_err(|error| anyhow::anyhow!("task error: {error}"))?}/// Keep the logical cursor position in sync with every visibly targeted@@ -6125,6 +6419,59 @@ fn capture_admission_error(error: anyhow::Error) -> ToolResult {}static CLICK_DEF: std::sync::OnceLock<ToolDef> = std::sync::OnceLock::new();+/// Wheel detents for a compositor scroll: one per line. On niri a page is+/// the detent approximation the Hyprland route documents; other compositors+/// keep one detent per page, which no run has measured against.+fn wheel_ticks(amount: usize, by: cua_driver_contract::ScrollBy, niri: bool) -> u32 {+ let detents = match by {+ cua_driver_contract::ScrollBy::Page if niri => {+ amount * crate::wayland::hyprland_input::PAGE_WHEEL_DETENTS+ }+ _ => amount,+ };+ detents as u32+}++#[cfg(test)]+#[test]+fn a_niri_page_scroll_sends_more_than_one_detent_per_page() {+ // `by: "page"` used to send `amount` detents, the same as lines: two+ // pages moved a WebKitGTK scroller 80 px under niri, less than its+ // 128 px viewport.+ use cua_driver_contract::ScrollBy;+ assert_eq!(wheel_ticks(2, ScrollBy::Line, true), 2);+ assert_eq!(+ wheel_ticks(2, ScrollBy::Page, true),+ 2 * crate::wayland::hyprland_input::PAGE_WHEEL_DETENTS as u32+ );+ // Sway, KWin and GNOME keep their earlier page scroll.+ assert_eq!(wheel_ticks(2, ScrollBy::Page, false), 2);+}++/// Presses in one click train: 1 to 3, as the contract states. A niri click+/// train is one gated action, so a larger count would mean seconds of+/// presses no recheck covers.+fn click_train(requested: u64) -> usize {+ requested.clamp(1, 3) as usize+}++#[cfg(test)]+#[test]+fn click_trains_stop_at_the_advertised_triple_click() {+ // A niri click train is rechecked once, before its first press, so a+ // `count: 20` must not become seconds of ungated presses.+ let tool = ClickTool {+ state: ToolState::new(),+ };+ let count = &tool.def().input_schema["properties"]["count"];+ assert_eq!(+ (count["minimum"].as_u64(), count["maximum"].as_u64()),+ (Some(1), Some(3))+ );+ assert_eq!(click_train(20), 3);+ assert_eq!(click_train(2), 2);+}+/// Where a foreground element click may put the real pointer.enum ForegroundElementPlacement {/// The element centre lies inside the target window (or a popup of the@@ -6242,7 +6589,7 @@ impl ClickTool {move || {let modifier_refs: Vec<&str> =modifier_owned.iter().map(String::as_str).collect();- crate::input::with_x11_foreground_opts(+ crate::interaction::with_x11_foreground(xid,crate::input::ForegroundOptions::pointer(),|| {@@ -6336,7 +6683,7 @@ impl ClickTool {"foreground element action",foreground_budget(0),move || {- crate::input::with_x11_foreground_opts(+ crate::interaction::with_x11_foreground(xid,crate::input::ForegroundOptions::pointer(),|| crate::atspi::perform_action_observed(&observed_for_ax),@@ -6554,7 +6901,7 @@ impl Tool for ClickTool {// [left,right,middle]); kept inline to carry the Linux/Wayland// back-compat prose the click button-schema test asserts on."button":{"type":"string","enum":["left","right","middle"],"description":"Mouse button. Default: \"left\" (legacy back-compat). X11: routed via ButtonPress/Release with the matching evdev code. Native Wayland: only left-button is supported via the virtual-pointer protocol; right/middle return an error."},- "count":{"type":"integer","minimum":1,"description":"Number of clicks in one press train: 1 (default), 2 = double-click, 3 = triple-click (line/paragraph selection). There is no separate triple_click tool — pass count: 3 here."},+ "count":{"type":"integer","minimum":1,"maximum":3,"description":"Number of clicks in one press train: 1 (default), 2 = double-click, 3 = triple-click (line/paragraph selection). There is no separate triple_click tool — pass count: 3 here."},"modifier": cua_driver_core::tool_schema::modifier_schema(),"from_zoom":{"type":"boolean","description":"Set true after a zoom call to auto-translate zoom-image pixel coordinates back to full-window space."},"scope":{"type":"string","enum":["window","desktop"],"default":"window","description":"Coordinate frame (default \"window\"). Pass \"desktop\" with x,y and no pid/window_id for a screen-absolute click in get_desktop_state coordinates."},@@ -6600,11 +6947,12 @@ impl Tool for ClickTool {Err(result) => return result,};let button = parse_mouse_button(input.button.unwrap_or(ClickButton::Left).as_str());- let n = input.count.unwrap_or(1) as usize;+ let n = input.count.unwrap_or(1);if n == 0 {return ToolResult::error("click.count must be at least 1.").with_structured(json!({ "code": "invalid_arguments" }));}+ let n = click_train(u64::from(n));let space = match desktop_input_space().await {Ok(space) => space,Err(error) => {@@ -6641,7 +6989,7 @@ impl Tool for ClickTool {true,).await;- let r = tokio::task::spawn_blocking(move || {+ let r = spawn_blocking_bounded("desktop click", foreground_budget(0), move || {if crate::wayland::wayland_input_enabled() {if !modifiers.is_empty() {anyhow::bail!(@@ -6671,6 +7019,9 @@ impl Tool for ClickTool {.with_structured(json!({ "path": if crate::wayland::wayland_input_enabled() { "wayland_desktop" } else { "xtest_desktop" }, "verified": false, "effect": "unverifiable" }),),+ Ok(Err(e)) if crate::input::foreground::error_code(&e).is_some() => {+ input_error_result(e)+ }Ok(Err(e)) => ToolResult::error(format!("desktop-scope click failed: {e}")),Err(e) => ToolResult::error(format!("task error: {e}")),};@@ -6681,7 +7032,7 @@ impl Tool for ClickTool {Err(e) => return e,};let delivery = crate::input::delivery::DeliveryMode::from_args(&args);- let count = args.u64_or("count", 1) as usize;+ let count = click_train(args.u64_or("count", 1));let isolated_background = isolated_hyprland_background(delivery);// Surface 5: reject unknown buttons so a typo can't silently fall through// to a left-click. Empty string keeps back-compat with old clients.@@ -7019,54 +7370,66 @@ impl Tool for ClickTool {// click at the element (no focus steal), then to a target-addressed// X11 event for toolkits that accept it.let cursor_id_for_fallback = cursor_id.clone();- let result = tokio::task::spawn_blocking(move || -> anyhow::Result<PointerRoute> {- let (xid2, lx, ly) = checked_element_local_coords(pid, &token, xid_hint)?;- let modifier_refs: Vec<&str> = modifiers.iter().map(String::as_str).collect();- if crate::wayland::wayland_input_enabled() && !modifier_refs.is_empty() {- anyhow::bail!(- "modified element clicks are unavailable on native Wayland: \+ let result = spawn_blocking_bounded(+ "element click",+ foreground_budget(0),+ move || -> anyhow::Result<PointerRoute> {+ // Refuse before resolving coordinates: the refusal does not+ // depend on where the element is, and a window without+ // compositor geometry must not turn it into a lookup error.+ let modifier_refs: Vec<&str> = modifiers.iter().map(String::as_str).collect();+ if crate::wayland::wayland_input_enabled() && !modifier_refs.is_empty() {+ anyhow::bail!(+ "modified element clicks are unavailable on native Wayland: \the pointer route cannot carry keyboard modifier state"- );- }- // An explicit X11 foreground request needs the real XTest path- // even for a plain click. Some native widgets (notably GTK- // selectable rows) expose bounds but no AT-SPI Action and- // ignore a targeted XSendEvent; limiting XTest to modified- // clicks made those rows addressable but not selectable.- if delivery.is_foreground() && !crate::wayland::wayland_input_enabled() {- let (_, sx, sy) = placement- .ok_or_else(|| anyhow::anyhow!("element screen bounds unavailable"))?;- crate::input::with_x11_foreground(xid2, 80, || {- crate::input::send_click_xtest_desktop_with_modifiers(- sx.round() as i32,- sy.round() as i32,+ );+ }+ let (xid2, lx, ly) = checked_element_local_coords(pid, &token, xid_hint)?;+ // An explicit X11 foreground request needs the real XTest path+ // even for a plain click. Some native widgets (notably GTK+ // selectable rows) expose bounds but no AT-SPI Action and+ // ignore a targeted XSendEvent; limiting XTest to modified+ // clicks made those rows addressable but not selectable.+ if delivery.is_foreground() && !crate::wayland::wayland_input_enabled() {+ let (_, sx, sy) = placement+ .ok_or_else(|| anyhow::anyhow!("element screen bounds unavailable"))?;+ crate::interaction::with_x11_foreground(+ xid2,+ crate::input::ForegroundOptions::pointer(),+ || {+ crate::input::send_click_xtest_desktop_with_modifiers(+ sx.round() as i32,+ sy.round() as i32,+ button,+ count,+ &modifier_refs,+ )+ },+ )+ .map(|(value, _report)| value)+ .map(|()| PointerRoute::Foreground)+ } else if modifier_refs.is_empty() && !crate::wayland::wayland_input_enabled() {+ x11_pixel_click_no_focus_steal(+ &cursor_id_for_fallback,+ xid2,+ lx as i32,+ ly as i32,button,count,+ )+ } else {+ crate::input::send_click_with_modifiers(+ xid2,+ lx as i32,+ ly as i32,+ count,+ button,&modifier_refs,)- })- .map(|()| PointerRoute::Foreground)- } else if modifier_refs.is_empty() && !crate::wayland::wayland_input_enabled() {- x11_pixel_click_no_focus_steal(- &cursor_id_for_fallback,- xid2,- lx as i32,- ly as i32,- button,- count,- )- } else {- crate::input::send_click_with_modifiers(- xid2,- lx as i32,- ly as i32,- count,- button,- &modifier_refs,- )- .map(|()| PointerRoute::Synthetic)- }- })+ .map(|()| PointerRoute::Synthetic)+ }+ },+ ).await;let mode_label = if delivery.is_foreground() {"foreground"@@ -7177,11 +7540,7 @@ impl Tool for ClickTool {x = lx;y = ly;}- Ok(Err(e)) => {- return ToolResult::error(format!(- "desktop-frame coordinates could not be mapped into window {xid}: {e}"- ))- }+ Ok(Err(e)) => return desktop_frame_unmapped(xid, e),Err(e) => return ToolResult::error(format!("Task error: {e}")),}}@@ -7194,15 +7553,17 @@ impl Tool for ClickTool {// always window-local screenshot pixels; native Wayland translates// through compositor/AT-SPI geometry while X11 uses XTranslateCoordinates.let wayland_output_point = if crate::wayland::wayland_input_enabled() {- Some(crate::wayland::window_local_to_output(- xid,- x.round() as i32,- y.round() as i32,- ))+ match crate::wayland::window_local_to_output(xid, x.round() as i32, y.round() as i32) {+ Ok(point) => Some(point),+ Err(error) => return input_error_result(error),+ }} else {None};- let glide_target = if let Some((sx, sy)) = wayland_output_point {+ let glide_target = if let Some(AimedPoint {+ point: (sx, sy), ..+ }) = wayland_output_point+ {Some((sx as f64, sy as f64))} else {tokio::task::spawn_blocking(move || window_local_to_screen(xid, x, y))@@ -7217,7 +7578,8 @@ impl Tool for ClickTool {let (xi, yi) = (x as i32, y as i32);let pixel_screen_point =glide_target.map(|(sx, sy)| (sx.round() as i32, sy.round() as i32));- let (output_x, output_y) = wayland_output_point.unwrap_or((xi, yi));+ let aimed = wayland_output_point.unwrap_or_else(|| (xi, yi).into());+ let (output_x, output_y) = aimed.point;if hyprland_foreground(delivery) {if !modifiers.is_empty() {return foreground_hyprland_refusal("modified clicks are unsupported");@@ -7306,8 +7668,8 @@ impl Tool for ClickTool {// Native Wayland: focus+raise the target toplevel// (foreign-toplevel `activate`), then drive `count` virtual-pointer// button events. Wayland injection routes to the compositor focus.- crate::wayland::with_target_foreground(pid, xid, || {- crate::wayland::click_focused(output_x, output_y, count as u32, button)+ crate::interaction::with_target_foreground(pid, xid, crate::interaction::RestoreMode::RestorePrevious, || {+ crate::wayland::click_focused(aimed, count as u32, button)})?;return Ok(("wayland_activate", None, None, None));}@@ -7400,7 +7762,7 @@ impl Tool for ClickTool {Ok((if fg { "x11_pixel_fg" } else { "x11_pixel" }, None, None))};if delivery.is_foreground() {- crate::input::with_x11_foreground_opts(+ crate::interaction::with_x11_foreground(xid,crate::input::ForegroundOptions::pointer(),|| inject(true),@@ -7616,13 +7978,17 @@ impl Tool for TypeTextTool {.await;let wayland = crate::wayland::wayland_input_enabled();let path = if wayland { "wayland_focused" } else { "xtest" };- let result = tokio::task::spawn_blocking(move || {- if wayland {- crate::wayland::type_text_focused(&text)- } else {- crate::input::send_type_text_xtest(&text)- }- })+ let result = spawn_blocking_bounded(+ "desktop type_text",+ typing_budget(text.chars().count(), crate::wayland::niri::is_session()),+ move || {+ if wayland {+ crate::wayland::type_text_focused(&text)+ } else {+ crate::input::send_type_text_xtest(&text)+ }+ },+ ).await;return match result {Ok(Ok(())) => ToolResult::text("Typed text into the focused desktop application.")@@ -7964,16 +8330,25 @@ impl Tool for TypeTextTool {if let Some(element) = token.clone() {let idx = element.index;let text_w = text.clone();- let result = tokio::task::spawn_blocking(move || {- crate::wayland::with_target_foreground(pid, xid, || {- if !crate::atspi::focus_token(pid, &element)? {- anyhow::bail!(- "AT-SPI Component.GrabFocus returned false for element {idx}"- );- }- crate::wayland::type_text_focused(&text_w)- })- })+ let result = spawn_blocking_bounded(+ "type_text",+ typing_budget(text_w.chars().count(), crate::wayland::niri::is_session()),+ move || {+ crate::interaction::with_target_foreground(+ pid,+ xid,+ crate::interaction::RestoreMode::RestorePrevious,+ || {+ if !crate::atspi::focus_token(pid, &element)? {+ anyhow::bail!(+ "AT-SPI Component.GrabFocus returned false for element {idx}"+ );+ }+ crate::wayland::type_text_focused(&text_w)+ },+ )+ },+ ).await;return match result {Ok(Ok(())) => ToolResult::text(format!(@@ -8039,19 +8414,28 @@ impl Tool for TypeTextTool {}let text_w = text.clone();let element = token.clone();- let result = tokio::task::spawn_blocking(move || {- crate::wayland::with_target_foreground(pid, xid, || {- if let Some(element) = &element {- if !crate::atspi::focus_token(pid, element)? {- anyhow::bail!(- "AT-SPI Component.GrabFocus returned false for element {}",- element.index- );- }- }- crate::wayland::type_text_focused(&text_w)- })- })+ let result = spawn_blocking_bounded(+ "type_text",+ typing_budget(text_w.chars().count(), crate::wayland::niri::is_session()),+ move || {+ crate::interaction::with_target_foreground(+ pid,+ xid,+ crate::interaction::RestoreMode::RestorePrevious,+ || {+ if let Some(element) = &element {+ if !crate::atspi::focus_token(pid, element)? {+ anyhow::bail!(+ "AT-SPI Component.GrabFocus returned false for element {}",+ element.index+ );+ }+ }+ crate::wayland::type_text_focused(&text_w)+ },+ )+ },+ ).await;return match result {Ok(Ok(())) => ToolResult::text(format!(@@ -8089,7 +8473,7 @@ impl Tool for TypeTextTool {let cursor_id = resolve_cursor_key(&args);let result = spawn_blocking_bounded("type_text terminal",- foreground_budget(text_len),+ typing_budget(text_len, crate::wayland::niri::is_session()),move || -> anyhow::Result<(&'static str, Option<crate::input::KeyboardDeliveryReport>)> {// pty-master injection is preferred — it skips the X event// queue entirely. Falls through to key events if the terminal@@ -8098,9 +8482,9 @@ impl Tool for TypeTextTool {return Ok(("pty", None));}if foreground {- crate::input::with_x11_foreground(xid, 80, || {+ crate::interaction::with_x11_foreground(xid, crate::input::ForegroundOptions::pointer(), || {crate::input::send_type_text_xtest(&text_t)- })?;+ }).map(|(value, _report)| value)?;return Ok(("key_events_fg", None));}// Background: real key events through the XI2 virtual master@@ -8135,9 +8519,9 @@ impl Tool for TypeTextTool {let element = token.clone();let result = spawn_blocking_bounded("foreground type_text",- foreground_budget(text_len),+ typing_budget(text_len, crate::wayland::niri::is_session()),move || {- crate::input::with_x11_foreground_opts(+ crate::interaction::with_x11_foreground(xid,crate::input::ForegroundOptions::keyboard(),|| {@@ -8190,7 +8574,7 @@ impl Tool for TypeTextTool {let cursor_id = resolve_cursor_key(&args);let result = spawn_blocking_bounded("background type_text",- foreground_budget(text_len),+ typing_budget(text_len, crate::wayland::niri::is_session()),move || background_text_route(&cursor_id, pid, xid, &text_b),).await;@@ -8228,7 +8612,7 @@ impl Tool for TypeTextTool {let cursor_id = resolve_cursor_key(&args);let result = spawn_blocking_bounded("background type_text",- foreground_budget(text_len),+ typing_budget(text_len, crate::wayland::niri::is_session()),move || -> anyhow::Result<Option<KeyRoute>> {if inject_terminal_input(pid, xid, &text_b)? {return Ok(Some(KeyRoute::Terminal));@@ -8249,21 +8633,28 @@ impl Tool for TypeTextTool {};}let text_f = text.clone();- let result = tokio::task::spawn_blocking(move || {- if inject_terminal_input(pid, xid, &text_f)? {- return Ok(());- }- // XTest (real input to the focused window), NOT XSendEvent: GTK/Qt- // drop synthetic key events, so a spreadsheet cell / canvas would- // stay empty. The click that gave this widget focus already put it- // under the X input focus, so XTest-to-focus lands correctly.- crate::input::with_x11_foreground(xid, 80, || {- crate::input::send_type_text_xtest(&text_f)- })- })- .await;- return match result {- Ok(Ok(())) => ToolResult::text(format!(+ let result = spawn_blocking_bounded(+ "foreground type_text",+ typing_budget(text_len, crate::wayland::niri::is_session()),+ move || {+ if inject_terminal_input(pid, xid, &text_f)? {+ return Ok(());+ }+ // XTest (real input to the focused window), NOT XSendEvent: GTK/Qt+ // drop synthetic key events, so a spreadsheet cell / canvas would+ // stay empty. The click that gave this widget focus already put it+ // under the X input focus, so XTest-to-focus lands correctly.+ crate::interaction::with_x11_foreground(+ xid,+ crate::input::ForegroundOptions::pointer(),+ || crate::input::send_type_text_xtest(&text_f),+ )+ .map(|(value, _report)| value)+ },+ )+ .await;+ return match result {+ Ok(Ok(())) => ToolResult::text(format!("Typed {text_len} character(s) into the focused widget.")).with_structured(type_text_structured(@@ -8299,22 +8690,26 @@ impl Tool for TypeTextTool {// expose the widget tree, type via AT-SPI, then send FocusOut.// This doesn't change the X11 active window, so the test's focus check passes.let text_clone2 = text.clone();- let qt5_result = tokio::task::spawn_blocking(move || {- if !is_qt5_process(pid) {- anyhow::bail!("not a Qt5 process; synthetic-FocusIn bridge workaround skipped");- }- // Send FocusIn to trigger Qt5's bridge- crate::input::send_focus_in(xid)?;- std::thread::sleep(std::time::Duration::from_millis(100));+ let qt5_result = spawn_blocking_bounded(+ "Qt5 type_text",+ typing_budget(text_len, crate::wayland::niri::is_session()),+ move || {+ if !is_qt5_process(pid) {+ anyhow::bail!("not a Qt5 process; synthetic-FocusIn bridge workaround skipped");+ }+ // Send FocusIn to trigger Qt5's bridge+ crate::input::send_focus_in(xid)?;+ std::thread::sleep(std::time::Duration::from_millis(100));- // Try AT-SPI again now that widgets should be exposed- let result = crate::atspi::type_into_editable(pid, &text_clone2);+ // Try AT-SPI again now that widgets should be exposed+ let result = crate::atspi::type_into_editable(pid, &text_clone2);- // Restore state with FocusOut- crate::input::send_focus_out(xid)?;+ // Restore state with FocusOut+ crate::input::send_focus_out(xid)?;- result- })+ result+ },+ ).await;match qt5_result {@@ -8337,7 +8732,7 @@ impl Tool for TypeTextTool {let cursor_id = resolve_cursor_key(&args);let result = spawn_blocking_bounded("type_text",- foreground_budget(text_len),+ typing_budget(text_len, crate::wayland::niri::is_session()),move || -> anyhow::Result<(&'static str, Option<crate::input::KeyboardDeliveryReport>)> {// Terminals: write to the pty master (focus-free, below the toolkit).if inject_terminal_input(pid, xid, &text)? {@@ -8356,9 +8751,9 @@ impl Tool for TypeTextTool {return Ok(("key_events", None));}if delivery.is_foreground() {- crate::input::with_x11_foreground(xid, 80, || {+ crate::interaction::with_x11_foreground(xid, crate::input::ForegroundOptions::pointer(), || {crate::input::send_type_text_xtest(&text)- })?;+ }).map(|(value, _report)| value)?;return Ok(("key_events_fg", None));}// No a11y editable, no pty: real key events through the XI2 virtual@@ -8557,19 +8952,20 @@ impl Tool for PressKeyTool {.await;let wayland = crate::wayland::wayland_input_enabled();let path = if wayland { "wayland_focused" } else { "xtest" };- let result = tokio::task::spawn_blocking(move || {- if wayland && modifiers.is_empty() {- crate::wayland::press_key_focused(&key)- } else if wayland {- let mut keys = modifiers;- keys.push(key);- crate::wayland::hotkey_focused(&keys)- } else {- let refs: Vec<&str> = modifiers.iter().map(String::as_str).collect();- crate::input::send_key_xtest(&key, &refs)- }- })- .await;+ let result =+ spawn_blocking_bounded("desktop press_key", foreground_budget(1), move || {+ if wayland && modifiers.is_empty() {+ crate::wayland::press_key_focused(&key)+ } else if wayland {+ let mut keys = modifiers;+ keys.push(key);+ crate::wayland::hotkey_focused(&keys)+ } else {+ let refs: Vec<&str> = modifiers.iter().map(String::as_str).collect();+ crate::input::send_key_xtest(&key, &refs)+ }+ })+ .await;return match result {Ok(Ok(())) => ToolResult::text(format!("Pressed desktop key '{display}'.")).with_structured(@@ -8827,21 +9223,26 @@ impl Tool for PressKeyTool {let key_w = key.clone();let chord = press_key_chord(&mods, &key);let element = token.clone();- let result = tokio::task::spawn_blocking(move || {- crate::wayland::with_target_foreground(pid, xid, || {- if let Some(element) = &element {- if !crate::atspi::focus_token(pid, element)? {- anyhow::bail!(- "AT-SPI Component.GrabFocus returned false for element {}",- element.index- );+ let result = spawn_blocking_bounded("press_key", foreground_budget(1), move || {+ crate::interaction::with_target_foreground(+ pid,+ xid,+ crate::interaction::RestoreMode::RestorePrevious,+ || {+ if let Some(element) = &element {+ if !crate::atspi::focus_token(pid, element)? {+ anyhow::bail!(+ "AT-SPI Component.GrabFocus returned false for element {}",+ element.index+ );+ }}- }- match chord {- Some(keys) => crate::wayland::hotkey_focused(&keys),- None => crate::wayland::press_key_focused(&key_w),- }- })+ match chord {+ Some(keys) => crate::wayland::hotkey_focused(&keys),+ None => crate::wayland::press_key_focused(&key_w),+ }+ },+ )}).await;return match result {@@ -8880,7 +9281,7 @@ impl Tool for PressKeyTool {// events through the XI2 virtual master keyboard (no focus steal),// else direct XSendEvent for apps that accept it.if deliver_fg {- return crate::input::with_x11_foreground_opts(+ return crate::interaction::with_x11_foreground(xid,crate::input::ForegroundOptions::keyboard(),|| {@@ -8999,15 +9400,16 @@ impl Tool for HotkeyTool {.await;let wayland = crate::wayland::wayland_input_enabled();let path = if wayland { "wayland_focused" } else { "xtest" };- let result = tokio::task::spawn_blocking(move || {- if wayland {- crate::wayland::hotkey_focused(&keys)- } else {- let refs: Vec<&str> = modifiers.iter().map(String::as_str).collect();- crate::input::send_key_xtest(&key, &refs)- }- })- .await;+ let result =+ spawn_blocking_bounded("desktop hotkey", foreground_budget(1), move || {+ if wayland {+ crate::wayland::hotkey_focused(&keys)+ } else {+ let refs: Vec<&str> = modifiers.iter().map(String::as_str).collect();+ crate::input::send_key_xtest(&key, &refs)+ }+ })+ .await;return match result {Ok(Ok(())) => ToolResult::text(format!("Pressed desktop hotkey {display}.")).with_structured(@@ -9350,7 +9752,7 @@ impl Tool for HotkeyTool {// (`send_key`) are dropped by GTK/Qt/Chromium/Firefox, so the// foreground rung must use XTest, which reaches the focused window.if deliver_fg {- return crate::input::with_x11_foreground_opts(+ return crate::interaction::with_x11_foreground(xid,crate::input::ForegroundOptions::keyboard(),|| crate::input::send_key_xtest(&key, &m),@@ -9534,7 +9936,7 @@ impl Tool for SetValueTool {}if foreground_route {let (sx, sy) = window_local_to_screen(win, lx, ly)?;- crate::input::with_x11_foreground_opts(+ crate::interaction::with_x11_foreground(win,crate::input::ForegroundOptions::pointer(),|| {@@ -9632,7 +10034,7 @@ fn pick_from_popup(return Ok(());}if foreground {- crate::input::with_x11_foreground_opts(+ crate::interaction::with_x11_foreground(win,crate::input::ForegroundOptions::keyboard(),|| crate::input::send_key_xtest("Escape", &[]),@@ -9665,7 +10067,7 @@ fn pick_from_popup(};if foreground {let (sx, sy) = window_local_to_screen(win, lx, ly)?;- crate::input::with_x11_foreground_opts(+ crate::interaction::with_x11_foreground(win,crate::input::ForegroundOptions::pointer(),|| crate::input::send_click_xtest_desktop(sx.round() as i32, sy.round() as i32, 1, 1),@@ -9893,6 +10295,11 @@ impl Tool for ScrollTool {let x = x.round() as i32;let y = y.round() as i32;let amount = input.amount.unwrap_or(3).clamp(1, 50) as usize;+ let ticks = wheel_ticks(+ amount,+ input.by.unwrap_or(cua_driver_contract::ScrollBy::Line),+ crate::wayland::niri::is_session(),+ );let display = direction.clone();let wayland = crate::wayland::wayland_input_enabled();let path = if wayland { "wayland_desktop" } else { "xtest" };@@ -9911,19 +10318,26 @@ impl Tool for ScrollTool {).await;}- let result = tokio::task::spawn_blocking(move || {- if wayland {- crate::wayland::scroll_desktop(&space, x, y, &direction, amount as u32)- } else {- crate::input::send_scroll_xtest_desktop(x, y, &direction, amount)- }- })+ let result = spawn_blocking_bounded(+ "desktop scroll",+ foreground_budget(ticks as usize),+ move || {+ if wayland {+ crate::wayland::scroll_desktop(&space, x, y, &direction, ticks)+ } else {+ crate::input::send_scroll_xtest_desktop(x, y, &direction, amount)+ }+ },+ ).await;return match result {Ok(Ok(())) => ToolResult::text(format!("Scrolled desktop {display} × {amount}.")).with_structured(json!({"scope":"desktop","path":path,"effect":"unverifiable"}),),+ Ok(Err(error)) if crate::input::foreground::error_code(&error).is_some() => {+ input_error_result(error)+ }Ok(Err(error)) => ToolResult::error(error.to_string()),Err(error) => ToolResult::error(format!("Task error: {error}")),};@@ -10024,11 +10438,7 @@ impl Tool for ScrollTool {match tokio::task::spawn_blocking(move || desktop_to_window_local(xid, x, y)).await{Ok(Ok(local)) => Some(local),- Ok(Err(e)) => {- return ToolResult::error(format!(- "desktop-frame coordinates could not be mapped into window {xid}: {e}"- ))- }+ Ok(Err(e)) => return desktop_frame_unmapped(xid, e),Err(e) => return ToolResult::error(format!("Task error: {e}")),}}@@ -10246,16 +10656,36 @@ impl Tool for ScrollTool {}(None, None) => None,};- let output_point = local_point.map(|(x, y)| {- crate::wayland::window_local_to_output(xid, x.round() as i32, y.round() as i32)- });- let result = tokio::task::spawn_blocking(move || {- crate::wayland::scroll_at(xid, output_point, &direction_for_wayland, amount as u32)- })- .await;+ let output_point = match local_point+ .map(|(x, y)| {+ crate::wayland::window_local_to_output(xid, x.round() as i32, y.round() as i32)+ })+ .transpose()+ {+ Ok(point) => point,+ Err(error) => return input_error_result(error),+ };+ let ticks = wheel_ticks(amount, by, crate::wayland::niri::is_session());+ let result =+ spawn_blocking_bounded("scroll", foreground_budget(ticks as usize), move || {+ crate::interaction::with_target_foreground(+ pid,+ xid,+ crate::interaction::pointer_gesture_restore(),+ || {+ crate::wayland::scroll_at(+ xid,+ output_point,+ &direction_for_wayland,+ ticks,+ )+ },+ )+ })+ .await;return match result {Ok(Ok(())) => ToolResult::text(format!(- "Scrolled {direction} {amount} ticks (delivery_mode=foreground)."+ "Scrolled {direction} {ticks} ticks (delivery_mode=foreground).")).with_structured(json!({ "verified": false, "delivery_mode": "foreground" })),Ok(Err(error)) => ToolResult::error(error.to_string()),@@ -10315,74 +10745,90 @@ impl Tool for ScrollTool {let cursor_id_for_task = cursor_id.clone();let direction_for_wayland = direction.clone();let amount_u32 = amount as u32;- let result = tokio::task::spawn_blocking(move || -> anyhow::Result<()> {- if crate::wayland::wayland_input_enabled() {- return crate::wayland::scroll(xid, &direction_for_wayland, amount_u32);- }- // foreground: activate the window, then scroll, then restore — for- // surfaces that only route wheel events to the active window.- let x11_scroll = || -> anyhow::Result<()> {- // X11: synthetic Button4-7 XSendEvents are dropped by XInput2- // toolkits (GTK never scrolls). On a real Xorg host, drive a real- // wheel detent through the MPX uinput pointer over the window's- // center — libinput turns it into the XI2 smooth-scroll GTK reads —- // without stealing focus. Falls back to the legacy XSendEvent- // Button4-7 path on Xvfb / unsupported servers.- // Button → axis/sign: 4=up(+v) 5=down(-v) 6=left(-h) 7=right(+h).- if crate::input::real_pointer_input_available() {- let pointer_point = element_point- .and_then(|(_, screen)| screen)- .map(|(x, y)| (x as i32, y as i32))- .or_else(|| window_screen_center(xid).ok());- if let Some((cx, cy)) = pointer_point {- let horizontal = matches!(button, 6 | 7);- let ticks = match button {- 4 | 7 => amount as i32,- _ => -(amount as i32), // 5 (down) and 6 (left)- };- match crate::input::send_virtual_pointer_scroll(- &cursor_id_for_task,- &crate::input::VirtualPointerScroll {- target_window: xid,- x: cx,- y: cy,- horizontal,- ticks,- },- ) {- Ok(()) => return Ok(()),- Err(error) if crate::input::is_uinput_unavailable(&error) => {- return Err(error)+ let result = spawn_blocking_bounded(+ "scroll",+ foreground_budget(amount_u32 as usize),+ move || -> anyhow::Result<()> {+ if crate::wayland::wayland_input_enabled() {+ return crate::interaction::with_target_foreground(+ pid,+ xid,+ crate::interaction::pointer_gesture_restore(),+ || crate::wayland::scroll(xid, &direction_for_wayland, amount_u32),+ );+ }+ // foreground: activate the window, then scroll, then restore — for+ // surfaces that only route wheel events to the active window.+ let x11_scroll = || -> anyhow::Result<()> {+ // X11: synthetic Button4-7 XSendEvents are dropped by XInput2+ // toolkits (GTK never scrolls). On a real Xorg host, drive a real+ // wheel detent through the MPX uinput pointer over the window's+ // center — libinput turns it into the XI2 smooth-scroll GTK reads —+ // without stealing focus. Falls back to the legacy XSendEvent+ // Button4-7 path on Xvfb / unsupported servers.+ // Button → axis/sign: 4=up(+v) 5=down(-v) 6=left(-h) 7=right(+h).+ if crate::input::real_pointer_input_available() {+ let pointer_point = element_point+ .and_then(|(_, screen)| screen)+ .map(|(x, y)| (x as i32, y as i32))+ .or_else(|| window_screen_center(xid).ok());+ if let Some((cx, cy)) = pointer_point {+ let horizontal = matches!(button, 6 | 7);+ let ticks = match button {+ 4 | 7 => amount as i32,+ _ => -(amount as i32), // 5 (down) and 6 (left)+ };+ match crate::input::send_virtual_pointer_scroll(+ &cursor_id_for_task,+ &crate::input::VirtualPointerScroll {+ target_window: xid,+ x: cx,+ y: cy,+ horizontal,+ ticks,+ },+ ) {+ Ok(()) => return Ok(()),+ Err(error) if crate::input::is_uinput_unavailable(&error) => {+ return Err(error)+ }+ Err(e) => tracing::warn!("MPX scroll fell back to XSendEvent: {e}"),}- Err(e) => tracing::warn!("MPX scroll fell back to XSendEvent: {e}"),}}- }- let (local_x, local_y) =- element_point.map(|(local, _)| local).unwrap_or((0.0, 0.0));- crate::input::send_click(xid, local_x as i32, local_y as i32, amount, button)- };- if delivery.is_foreground() {- let point = element_point- .and_then(|(_, screen)| screen)- .or_else(|| {- window_screen_center(xid)- .ok()- .map(|(x, y)| (x as f64, y as f64))- })- .ok_or_else(|| anyhow::anyhow!("could not resolve foreground scroll point"))?;- crate::input::with_x11_foreground(xid, 80, || {- crate::input::send_click_xtest_desktop(- point.0 as i32,- point.1 as i32,- button,- amount,+ let (local_x, local_y) =+ element_point.map(|(local, _)| local).unwrap_or((0.0, 0.0));+ crate::input::send_click(xid, local_x as i32, local_y as i32, amount, button)+ };+ if delivery.is_foreground() {+ let point = element_point+ .and_then(|(_, screen)| screen)+ .or_else(|| {+ window_screen_center(xid)+ .ok()+ .map(|(x, y)| (x as f64, y as f64))+ })+ .ok_or_else(|| {+ anyhow::anyhow!("could not resolve foreground scroll point")+ })?;+ crate::interaction::with_x11_foreground(+ xid,+ crate::input::ForegroundOptions::pointer(),+ || {+ crate::input::send_click_xtest_desktop(+ point.0 as i32,+ point.1 as i32,+ button,+ amount,+ )+ },)- })- } else {- x11_scroll()- }- })+ .map(|(value, _report)| value)+ } else {+ x11_scroll()+ }+ },+ ).await;let mode_label = if delivery.is_foreground() {"foreground"@@ -10514,36 +10960,54 @@ impl Tool for DoubleClickTool {}let lxi = lx as i32;let lyi = ly as i32;- let wayland_point = crate::wayland::wayland_input_enabled()- .then(|| crate::wayland::window_local_to_output(xid, lxi, lyi));+ let wayland_point = match crate::wayland::wayland_input_enabled()+ .then(|| crate::wayland::window_local_to_output(xid, lxi, lyi))+ .transpose()+ {+ Ok(point) => point,+ Err(error) => return input_error_result(error),+ };let cursor_id_for_task = cursor_id.clone();- let click_result =- tokio::task::spawn_blocking(move || -> anyhow::Result<PointerRoute> {+ let click_result = spawn_blocking_bounded(+ "double_click",+ foreground_budget(0),+ move || -> anyhow::Result<PointerRoute> {if crate::wayland::is_inject_mode() {return crate::wayland::inject_click(pid, xid, lx, ly, 2, 1).map(|()| PointerRoute::Wayland);}if crate::wayland::wayland_input_enabled() {- let (output_x, output_y) = wayland_point.unwrap_or((lxi, lyi));- return crate::wayland::click(xid, output_x, output_y, 2, 1)- .map(|()| PointerRoute::Wayland);+ let aimed = wayland_point.unwrap_or_else(|| (lxi, lyi).into());+ return crate::interaction::with_target_foreground(+ pid,+ xid,+ crate::interaction::pointer_gesture_restore(),+ || crate::wayland::click(xid, aimed, 2, 1),+ )+ .map(|()| PointerRoute::Wayland);}if delivery.is_foreground() {- return crate::input::with_x11_foreground(xid, 80, || {- let (sx, sy) =- window_local_to_screen(xid, lxi as f64, lyi as f64)?;- crate::input::send_click_xtest_desktop(- sx.round() as i32,- sy.round() as i32,- 1,- 2,- )- })+ return crate::interaction::with_x11_foreground(+ xid,+ crate::input::ForegroundOptions::pointer(),+ || {+ let (sx, sy) =+ window_local_to_screen(xid, lxi as f64, lyi as f64)?;+ crate::input::send_click_xtest_desktop(+ sx.round() as i32,+ sy.round() as i32,+ 1,+ 2,+ )+ },+ )+ .map(|(value, _report)| value).map(|()| PointerRoute::Foreground);}x11_pixel_click_no_focus_steal(&cursor_id_for_task, xid, lxi, lyi, 1, 2)- })- .await;+ },+ )+ .await;let mode_label = if delivery.is_foreground() {"foreground"} else {@@ -10599,11 +11063,7 @@ impl Tool for DoubleClickTool {x = lx;y = ly;}- Ok(Err(e)) => {- return ToolResult::error(format!(- "desktop-frame coordinates could not be mapped into window {xid}: {e}"- ))- }+ Ok(Err(e)) => return desktop_frame_unmapped(xid, e),Err(e) => return ToolResult::error(format!("Task error: {e}")),}}@@ -10612,15 +11072,17 @@ impl Tool for DoubleClickTool {cursor_overlay::OverlayCommand::PinAbove(xid),);let wayland_output_point = if crate::wayland::wayland_input_enabled() {- Some(crate::wayland::window_local_to_output(- xid,- x.round() as i32,- y.round() as i32,- ))+ match crate::wayland::window_local_to_output(xid, x.round() as i32, y.round() as i32) {+ Ok(point) => Some(point),+ Err(error) => return input_error_result(error),+ }} else {None};- let glide_target = if let Some((sx, sy)) = wayland_output_point {+ let glide_target = if let Some(AimedPoint {+ point: (sx, sy), ..+ }) = wayland_output_point+ {Some((sx as f64, sy as f64))} else {tokio::task::spawn_blocking(move || window_local_to_screen(xid, x, y))@@ -10633,38 +11095,53 @@ impl Tool for DoubleClickTool {}let (xi, yi) = (x as i32, y as i32);let cursor_id_for_task = cursor_id.clone();- let result = tokio::task::spawn_blocking(move || -> anyhow::Result<PointerRoute> {- if crate::wayland::is_inject_mode() {- return crate::wayland::inject_click(pid, xid, x, y, 2, 1)- .map(|()| PointerRoute::Wayland);- }- if crate::wayland::wayland_input_enabled() {- let (output_x, output_y) = wayland_output_point.unwrap_or((xi, yi));- return crate::wayland::click(xid, output_x, output_y, 2, 1)+ let result = spawn_blocking_bounded(+ "double_click",+ foreground_budget(0),+ move || -> anyhow::Result<PointerRoute> {+ if crate::wayland::is_inject_mode() {+ return crate::wayland::inject_click(pid, xid, x, y, 2, 1)+ .map(|()| PointerRoute::Wayland);+ }+ if crate::wayland::wayland_input_enabled() {+ let aimed = wayland_output_point.unwrap_or_else(|| (xi, yi).into());+ return crate::interaction::with_target_foreground(+ pid,+ xid,+ crate::interaction::pointer_gesture_restore(),+ || crate::wayland::click(xid, aimed, 2, 1),+ ).map(|()| PointerRoute::Wayland);- }- if delivery.is_foreground() {- return crate::input::with_x11_foreground(xid, 80, || {- // Real XTest double-click at the screen point — synthetic- // XSendEvent button events are dropped by GTK/Qt and the MPX- // uinput path needs /dev/uinput (absent on Xvfb/Xtigervnc), so- // neither lands. Mirrors the single-click foreground path.- if let Ok((sx, sy)) = window_local_to_screen(xid, xi as f64, yi as f64) {- crate::input::send_click_xtest_desktop(- sx.round() as i32,- sy.round() as i32,- 1,- 2,- )?;- return Ok(());- }- x11_pixel_click_no_focus_steal(&cursor_id_for_task, xid, xi, yi, 1, 2)- .map(|_| ())- })- .map(|()| PointerRoute::Foreground);- }- x11_pixel_click_no_focus_steal(&cursor_id_for_task, xid, xi, yi, 1, 2)- })+ }+ if delivery.is_foreground() {+ return crate::interaction::with_x11_foreground(+ xid,+ crate::input::ForegroundOptions::pointer(),+ || {+ // Real XTest double-click at the screen point — synthetic+ // XSendEvent button events are dropped by GTK/Qt and the MPX+ // uinput path needs /dev/uinput (absent on Xvfb/Xtigervnc), so+ // neither lands. Mirrors the single-click foreground path.+ if let Ok((sx, sy)) = window_local_to_screen(xid, xi as f64, yi as f64)+ {+ crate::input::send_click_xtest_desktop(+ sx.round() as i32,+ sy.round() as i32,+ 1,+ 2,+ )?;+ return Ok(());+ }+ x11_pixel_click_no_focus_steal(&cursor_id_for_task, xid, xi, yi, 1, 2)+ .map(|_| ())+ },+ )+ .map(|(value, _report)| value)+ .map(|()| PointerRoute::Foreground);+ }+ x11_pixel_click_no_focus_steal(&cursor_id_for_task, xid, xi, yi, 1, 2)+ },+ ).await;let mode_label = if delivery.is_foreground() {"foreground"@@ -10801,36 +11278,49 @@ impl Tool for RightClickTool {}let lxi = lx as i32;let lyi = ly as i32;- let wayland_point = crate::wayland::wayland_input_enabled()- .then(|| crate::wayland::window_local_to_output(xid, lxi, lyi));+ let wayland_point = match crate::wayland::wayland_input_enabled()+ .then(|| crate::wayland::window_local_to_output(xid, lxi, lyi))+ .transpose()+ {+ Ok(point) => point,+ Err(error) => return input_error_result(error),+ };let cursor_id_for_task = cursor_id.clone();- let click_result =- tokio::task::spawn_blocking(move || -> anyhow::Result<PointerRoute> {+ let click_result = spawn_blocking_bounded(+ "right_click",+ foreground_budget(0),+ move || -> anyhow::Result<PointerRoute> {if crate::wayland::is_inject_mode() {return crate::wayland::inject_click(pid, xid, lx, ly, 1, 3).map(|()| PointerRoute::Wayland);}if crate::wayland::wayland_input_enabled() {- let (output_x, output_y) = wayland_point.unwrap_or((lxi, lyi));- return crate::wayland::click(xid, output_x, output_y, 1, 3)+ let aimed = wayland_point.unwrap_or_else(|| (lxi, lyi).into());+ return crate::wayland::click(xid, aimed, 1, 3).map(|()| PointerRoute::Wayland);}if delivery.is_foreground() {- return crate::input::with_x11_foreground(xid, 80, || {- let (sx, sy) =- window_local_to_screen(xid, lxi as f64, lyi as f64)?;- crate::input::send_click_xtest_desktop(- sx.round() as i32,- sy.round() as i32,- 3,- 1,- )- })+ return crate::interaction::with_x11_foreground(+ xid,+ crate::input::ForegroundOptions::pointer(),+ || {+ let (sx, sy) =+ window_local_to_screen(xid, lxi as f64, lyi as f64)?;+ crate::input::send_click_xtest_desktop(+ sx.round() as i32,+ sy.round() as i32,+ 3,+ 1,+ )+ },+ )+ .map(|(value, _report)| value).map(|()| PointerRoute::Foreground);}x11_pixel_click_no_focus_steal(&cursor_id_for_task, xid, lxi, lyi, 3, 1)- })- .await;+ },+ )+ .await;let mode_label = if delivery.is_foreground() {"foreground"} else {@@ -10886,11 +11376,7 @@ impl Tool for RightClickTool {x = lx;y = ly;}- Ok(Err(e)) => {- return ToolResult::error(format!(- "desktop-frame coordinates could not be mapped into window {xid}: {e}"- ))- }+ Ok(Err(e)) => return desktop_frame_unmapped(xid, e),Err(e) => return ToolResult::error(format!("Task error: {e}")),}}@@ -10899,15 +11385,17 @@ impl Tool for RightClickTool {cursor_overlay::OverlayCommand::PinAbove(xid),);let wayland_output_point = if crate::wayland::wayland_input_enabled() {- Some(crate::wayland::window_local_to_output(- xid,- x.round() as i32,- y.round() as i32,- ))+ match crate::wayland::window_local_to_output(xid, x.round() as i32, y.round() as i32) {+ Ok(point) => Some(point),+ Err(error) => return input_error_result(error),+ }} else {None};- let glide_target = if let Some((sx, sy)) = wayland_output_point {+ let glide_target = if let Some(AimedPoint {+ point: (sx, sy), ..+ }) = wayland_output_point+ {Some((sx as f64, sy as f64))} else {tokio::task::spawn_blocking(move || window_local_to_screen(xid, x, y))@@ -10920,37 +11408,46 @@ impl Tool for RightClickTool {}let (xi, yi) = (x as i32, y as i32);let cursor_id_for_task = cursor_id.clone();- let result = tokio::task::spawn_blocking(move || -> anyhow::Result<PointerRoute> {- if crate::wayland::is_inject_mode() {- return crate::wayland::inject_click(pid, xid, x, y, 1, 3)- .map(|()| PointerRoute::Wayland);- }- if crate::wayland::wayland_input_enabled() {- let (output_x, output_y) = wayland_output_point.unwrap_or((xi, yi));- return crate::wayland::click(xid, output_x, output_y, 1, 3)- .map(|()| PointerRoute::Wayland);- }- if delivery.is_foreground() {- return crate::input::with_x11_foreground(xid, 80, || {- // Real XTest right-click at the screen point (synthetic- // XSendEvent is dropped by GTK/Qt; MPX needs /dev/uinput).- // Mirrors the single-click foreground path.- if let Ok((sx, sy)) = window_local_to_screen(xid, xi as f64, yi as f64) {- crate::input::send_click_xtest_desktop(- sx.round() as i32,- sy.round() as i32,- 3,- 1,- )?;- return Ok(());- }- x11_pixel_click_no_focus_steal(&cursor_id_for_task, xid, xi, yi, 3, 1)- .map(|_| ())- })- .map(|()| PointerRoute::Foreground);- }- x11_pixel_click_no_focus_steal(&cursor_id_for_task, xid, xi, yi, 3, 1)- })+ let result = spawn_blocking_bounded(+ "right_click",+ foreground_budget(0),+ move || -> anyhow::Result<PointerRoute> {+ if crate::wayland::is_inject_mode() {+ return crate::wayland::inject_click(pid, xid, x, y, 1, 3)+ .map(|()| PointerRoute::Wayland);+ }+ if crate::wayland::wayland_input_enabled() {+ let aimed = wayland_output_point.unwrap_or_else(|| (xi, yi).into());+ return crate::wayland::click(xid, aimed, 1, 3).map(|()| PointerRoute::Wayland);+ }+ if delivery.is_foreground() {+ return crate::interaction::with_x11_foreground(+ xid,+ crate::input::ForegroundOptions::pointer(),+ || {+ // Real XTest right-click at the screen point (synthetic+ // XSendEvent is dropped by GTK/Qt; MPX needs /dev/uinput).+ // Mirrors the single-click foreground path.+ if let Ok((sx, sy)) = window_local_to_screen(xid, xi as f64, yi as f64)+ {+ crate::input::send_click_xtest_desktop(+ sx.round() as i32,+ sy.round() as i32,+ 3,+ 1,+ )?;+ return Ok(());+ }+ x11_pixel_click_no_focus_steal(&cursor_id_for_task, xid, xi, yi, 3, 1)+ .map(|_| ())+ },+ )+ .map(|(value, _report)| value)+ .map(|()| PointerRoute::Foreground);+ }+ x11_pixel_click_no_focus_steal(&cursor_id_for_task, xid, xi, yi, 3, 1)+ },+ ).await;let mode_label = if delivery.is_foreground() {"foreground"@@ -11077,32 +11574,37 @@ impl Tool for DragTool {};let (overlay_from, overlay_to) =(overlay_point(from_x, from_y), overlay_point(to_x, to_y));- let result = tokio::task::spawn_blocking(move || {- if wayland {- crate::wayland::drag_desktop(- &space,- from_x.round() as i32,- from_y.round() as i32,- to_x.round() as i32,- to_y.round() as i32,- steps as u32,- duration_ms,- button,- )- } else {- let modifier_refs: Vec<&str> = modifiers.iter().map(String::as_str).collect();- crate::input::send_drag_xtest_desktop_with_modifiers(- from_x.round() as i32,- from_y.round() as i32,- to_x.round() as i32,- to_y.round() as i32,- button,- duration_ms,- steps,- &modifier_refs,- )- }- });+ let result = spawn_blocking_bounded(+ "desktop drag",+ foreground_budget(steps) + std::time::Duration::from_millis(duration_ms),+ move || {+ if wayland {+ crate::wayland::drag_desktop(+ &space,+ from_x.round() as i32,+ from_y.round() as i32,+ to_x.round() as i32,+ to_y.round() as i32,+ steps as u32,+ duration_ms,+ button,+ )+ } else {+ let modifier_refs: Vec<&str> =+ modifiers.iter().map(String::as_str).collect();+ crate::input::send_drag_xtest_desktop_with_modifiers(+ from_x.round() as i32,+ from_y.round() as i32,+ to_x.round() as i32,+ to_y.round() as i32,+ button,+ duration_ms,+ steps,+ &modifier_refs,+ )+ }+ },+ );let visual_drag = track_overlay_drag_for(cursor_id.clone(),overlay_from,@@ -11120,6 +11622,9 @@ impl Tool for DragTool {Ok(Ok(())) => ToolResult::text("Dragged on the desktop.").with_structured(json!({"scope":"desktop","path":path,"effect":"unverifiable"}),),+ Ok(Err(error)) if crate::input::foreground::error_code(&error).is_some() => {+ input_error_result(error)+ }Ok(Err(error)) => ToolResult::error(error.to_string()),Err(error) => ToolResult::error(format!("Task error: {error}")),};@@ -11203,7 +11708,7 @@ impl Tool for DragTool {};let duration_ms = args.u64_or("duration_ms", 500);- let steps = args.u64_or("steps", 20) as usize;+ let steps = drag_steps(&args);let button_str = args.str_or("button", "left");let button = parse_mouse_button(button_str.as_str());match context {@@ -11233,11 +11738,7 @@ impl Tool for DragTool {to_x = tx;to_y = ty;}- Ok(Err(e)) => {- return ToolResult::error(format!(- "desktop-frame coordinates could not be mapped into window {xid}: {e}"- ))- }+ Ok(Err(e)) => return desktop_frame_unmapped(xid, e),Err(e) => return ToolResult::error(format!("Task error: {e}")),}}@@ -11254,16 +11755,23 @@ impl Tool for DragTool {);}let owner = named_session_cursor_key(&args);- let from = crate::wayland::window_local_to_output(+ let points = crate::wayland::window_local_to_output(xid,from_x.round() as i32,from_y.round() as i32,- );- let to = crate::wayland::window_local_to_output(- xid,- to_x.round() as i32,- to_y.round() as i32,- );+ )+ .and_then(|from| {+ crate::wayland::window_local_to_output(+ xid,+ to_x.round() as i32,+ to_y.round() as i32,+ )+ .map(|to| (from.point, to.point))+ });+ let (from, to) = match points {+ Ok(points) => points,+ Err(error) => return input_error_result(error),+ };let (started, acknowledged) = tokio::sync::oneshot::channel();let (_cancellation, mut dispatch) =match spawn_isolated_hyprland(&args, move |cancellation| {@@ -11317,16 +11825,23 @@ impl Tool for DragTool {);}let owner = named_session_cursor_key(&args);- let from = crate::wayland::window_local_to_output(+ let points = crate::wayland::window_local_to_output(xid,from_x.round() as i32,from_y.round() as i32,- );- let to = crate::wayland::window_local_to_output(- xid,- to_x.round() as i32,- to_y.round() as i32,- );+ )+ .and_then(|from| {+ crate::wayland::window_local_to_output(+ xid,+ to_x.round() as i32,+ to_y.round() as i32,+ )+ .map(|to| (from.point, to.point))+ });+ let (from, to) = match points {+ Ok(points) => points,+ Err(error) => return input_error_result(error),+ };let (started, acknowledged) = tokio::sync::oneshot::channel();let (_cancellation, mut dispatch) =match spawn_isolated_hyprland(&args, move |cancellation| {@@ -11369,22 +11884,29 @@ impl Tool for DragTool {cursor_id.clone(),cursor_overlay::OverlayCommand::PinAbove(xid),);- let wayland_points = crate::wayland::wayland_input_enabled().then(|| {- (+ let wayland_points = match crate::wayland::wayland_input_enabled()+ .then(|| {crate::wayland::window_local_to_output(xid,from_x.round() as i32,from_y.round() as i32,- ),- crate::wayland::window_local_to_output(- xid,- to_x.round() as i32,- to_y.round() as i32,- ),- )- });+ )+ .and_then(|from| {+ crate::wayland::window_local_to_output(+ xid,+ to_x.round() as i32,+ to_y.round() as i32,+ )+ .map(|to| (from, to))+ })+ })+ .transpose()+ {+ Ok(points) => points,+ Err(error) => return input_error_result(error),+ };let screen_from = if let Some((from, _)) = wayland_points {- Some((from.0 as f64, from.1 as f64))+ Some((from.point.0 as f64, from.point.1 as f64))} else {tokio::task::spawn_blocking(move || window_local_to_screen(xid, from_x, from_y)).await@@ -11410,38 +11932,46 @@ impl Tool for DragTool {// coords. Returns early so we don't fall into the X11 XSendEvent loop below.if crate::wayland::wayland_input_enabled() {let steps_u32 = steps as u32;- let drag_result = if crate::wayland::is_inject_mode() {- tokio::task::spawn_blocking(move || {- crate::wayland::inject_drag(+ let inject = crate::wayland::is_inject_mode();+ let (from_aimed, to_aimed) = wayland_points.unwrap_or_else(|| {+ (+ (from_x.round() as i32, from_y.round() as i32).into(),+ (to_x.round() as i32, to_y.round() as i32).into(),+ )+ });+ let drag_result = spawn_blocking_bounded(+ "drag",+ foreground_budget(steps) + std::time::Duration::from_millis(duration_ms),+ move || {+ if inject {+ return crate::wayland::inject_drag(+ pid,+ xid,+ (from_x, from_y),+ (to_x, to_y),+ steps,+ button as u32,+ );+ }+ crate::interaction::with_target_foreground(pid,xid,- (from_x, from_y),- (to_x, to_y),- steps,- button as u32,- )- })- } else {- let ((fxi, fyi), (txi, tyi)) = wayland_points.unwrap_or((- (from_x.round() as i32, from_y.round() as i32),- (to_x.round() as i32, to_y.round() as i32),- ));- tokio::task::spawn_blocking(move || {- crate::wayland::drag(- xid,- (fxi, fyi),- (txi, tyi),- steps_u32,- duration_ms,- button,+ crate::interaction::pointer_gesture_restore(),+ || {+ crate::wayland::drag(+ xid,+ from_aimed,+ to_aimed,+ steps_u32,+ duration_ms,+ button,+ )+ },)- })- };- let ((from_output_x, from_output_y), (to_output_x, to_output_y)) = wayland_points- .unwrap_or((- (from_x.round() as i32, from_y.round() as i32),- (to_x.round() as i32, to_y.round() as i32),- ));+ },+ );+ let ((from_output_x, from_output_y), (to_output_x, to_output_y)) =+ (from_aimed.point, to_aimed.point);let visual_drag = track_overlay_drag_for(cursor_id.clone(),(f64::from(from_output_x), f64::from(from_output_y)),@@ -11481,21 +12011,30 @@ impl Tool for DragTool {Ok(Err(e)) => return ToolResult::error(e.to_string()),Err(e) => return ToolResult::error(format!("Task error: {e}")),};- let drag_result = tokio::task::spawn_blocking(move || {- let modifier_refs: Vec<&str> = modifiers.iter().map(String::as_str).collect();- crate::input::with_x11_foreground(xid, 80, || {- crate::input::send_drag_xtest_desktop_with_modifiers(- screen_from_x.round() as i32,- screen_from_y.round() as i32,- screen_to_x.round() as i32,- screen_to_y.round() as i32,- button,- duration_ms,- steps,- &modifier_refs,+ let drag_result = spawn_blocking_bounded(+ "foreground drag",+ foreground_budget(steps) + std::time::Duration::from_millis(duration_ms),+ move || {+ let modifier_refs: Vec<&str> = modifiers.iter().map(String::as_str).collect();+ crate::interaction::with_x11_foreground(+ xid,+ crate::input::ForegroundOptions::pointer(),+ || {+ crate::input::send_drag_xtest_desktop_with_modifiers(+ screen_from_x.round() as i32,+ screen_from_y.round() as i32,+ screen_to_x.round() as i32,+ screen_to_y.round() as i32,+ button,+ duration_ms,+ steps,+ &modifier_refs,+ )+ },)- })- });+ .map(|(value, _report)| value)+ },+ );let visual_drag = track_overlay_drag_for(cursor_id.clone(),(screen_from_x, screen_from_y),@@ -11548,28 +12087,32 @@ impl Tool for DragTool {cursor_overlay::OverlayCommand::SetPressed(true),);let cursor_id_for_task = cursor_id.clone();- let drag_result = tokio::task::spawn_blocking(move || {- let (outcome, guard) =- crate::input::focus_guard::guarded_settled(Some(pid), || {- Ok(crate::input::send_virtual_pointer_drag(- &cursor_id_for_task,- &crate::input::VirtualPointerDrag {- target_window: xid,- button,- path: vec![- (sfx.round() as i32, sfy.round() as i32),- (stx.round() as i32, sty.round() as i32),- ],- duration_ms,- steps,- },- ))- })?;- outcome.map(|mut effect| {- effect.focus_guard = guard;- effect- })- })+ let drag_result = spawn_blocking_bounded(+ "drag",+ foreground_budget(steps) + std::time::Duration::from_millis(duration_ms),+ move || {+ let (outcome, guard) =+ crate::input::focus_guard::guarded_settled(Some(pid), || {+ Ok(crate::input::send_virtual_pointer_drag(+ &cursor_id_for_task,+ &crate::input::VirtualPointerDrag {+ target_window: xid,+ button,+ path: vec![+ (sfx.round() as i32, sfy.round() as i32),+ (stx.round() as i32, sty.round() as i32),+ ],+ duration_ms,+ steps,+ },+ ))+ })?;+ outcome.map(|mut effect| {+ effect.focus_guard = guard;+ effect+ })+ },+ ).await;crate::overlay::send_command_for(cursor_id.clone(),@@ -11611,14 +12154,25 @@ impl Tool for DragTool {cursor_id.clone(),cursor_overlay::OverlayCommand::SetPressed(true),);- let press_result = tokio::task::spawn_blocking(move || {- crate::input::send_button_down(- xid,- from_x.round() as i32,- from_y.round() as i32,- button,- )- })+ let press_result = press_x11_drag_start(+ foreground_budget(0),+ move || {+ crate::input::send_button_down(+ xid,+ from_x.round() as i32,+ from_y.round() as i32,+ button,+ )+ },+ move || {+ crate::input::send_button_up(+ xid,+ from_x.round() as i32,+ from_y.round() as i32,+ button,+ )+ },+ ).await;let mut result: anyhow::Result<()> = match press_result {Ok(Ok(())) => Ok(()),@@ -11636,15 +12190,16 @@ impl Tool for DragTool {let t = i as f64 / steps.max(1) as f64;let ix = from_x + (to_x - from_x) * t;let iy = from_y + (to_y - from_y) * t;- let motion_result = tokio::task::spawn_blocking(move || {- crate::input::send_motion(- xid,- ix.round() as i32,- iy.round() as i32,- Some(button),- )- })- .await;+ let motion_result =+ spawn_blocking_bounded("drag", foreground_budget(0), move || {+ crate::input::send_motion(+ xid,+ ix.round() as i32,+ iy.round() as i32,+ Some(button),+ )+ })+ .await;match motion_result {Ok(Ok(())) => {if let Ok(Ok((sx, sy))) =@@ -11676,7 +12231,9 @@ impl Tool for DragTool {}}- let release_result = tokio::task::spawn_blocking(move || {+ // XSendEvent does not read the call's action, so the release still+ // goes out if the call ends first.+ let release_result = spawn_blocking_bounded("drag", foreground_budget(0), move || {crate::input::send_button_up(xid, to_x.round() as i32, to_y.round() as i32, button)}).await;@@ -11806,11 +12363,7 @@ impl Tool for MouseButtonDownTool {x = lx;y = ly;}- Ok(Err(e)) => {- return ToolResult::error(format!(- "desktop-frame coordinates could not be mapped into window {xid}: {e}"- ))- }+ Ok(Err(e)) => return desktop_frame_unmapped(xid, e),Err(e) => return ToolResult::error(format!("Task error: {e}")),}}@@ -11837,20 +12390,42 @@ impl Tool for MouseButtonDownTool {// the existing input::send_button_down behaviour.let result = if is_wl {let cid = cursor_id.clone();- let (output_x, output_y) = persistent_pointer_coordinates(+ let aimed = match persistent_pointer_coordinates(is_wl,xid,xi,yi,crate::wayland::window_local_to_output,- );- tokio::task::spawn_blocking(move || {- crate::wayland::persistent_vptr::press(&cid, xid, output_x, output_y, button)+ ) {+ Ok(point) => point,+ Err(error) => return input_error_result(error),+ };+ let state = self.state.clone();+ let hold = MouseHoldState {+ pid,+ xid,+ button,+ x,+ y,+ };+ spawn_blocking_bounded("mouse_button_down", foreground_budget(0), move || {+ crate::wayland::persistent_vptr::press(&cid, xid, aimed, button, || {+ state.mouse_hold.lock().unwrap().insert(cid.clone(), hold);+ })}).await} else {- tokio::task::spawn_blocking(move || crate::input::send_button_down(xid, xi, yi, button))- .await+ let hold = MouseHoldState {+ pid,+ xid,+ button,+ x,+ y,+ };+ press_and_record_x11_hold(self.state.clone(), cursor_id.clone(), hold, move || {+ crate::input::send_button_down(xid, xi, yi, button)+ })+ .await};match result {Ok(Ok(())) => {@@ -11861,11 +12436,6 @@ impl Tool for MouseButtonDownTool {x,y,};- self.state- .mouse_hold- .lock()- .unwrap()- .insert(cursor_id.clone(), hold.clone());if let Ok(Ok((sx, sy))) =tokio::task::spawn_blocking(move || window_local_to_screen(xid, x, y)).await{@@ -11898,6 +12468,44 @@ impl Tool for MouseButtonDownTool {}}+/// The press of an X11 step-by-step drag. A press that goes down after its+/// call ended is released here, in the same worker: the drag's own release+/// runs in another worker on its own X connection and can reach the app+/// first, which would leave the app's last event a press.+async fn press_x11_drag_start(+ budget: std::time::Duration,+ press: impl FnOnce() -> anyhow::Result<()> + Send + 'static,+ release: impl FnOnce() -> anyhow::Result<()> + Send + 'static,+) -> Result<anyhow::Result<()>, tokio::task::JoinError> {+ spawn_blocking_bounded("drag", budget, move || {+ press()?;+ if crate::input::action::admit(|| ()).is_err() {+ release()?;+ }+ Ok(())+ })+ .await+}++/// Press an X11 button and record its hold as soon as it is down, so a caller+/// that goes away before the await resumes still leaves a hold that+/// mouse_button_up can release. The worker keeps the call's desktop lease+/// through both, so the next call on the cursor sees this hold instead of+/// pressing and recording its own, which this one would then overwrite.+async fn press_and_record_x11_hold(+ state: Arc<ToolState>,+ cursor_id: String,+ hold: MouseHoldState,+ press: impl FnOnce() -> anyhow::Result<()> + Send + 'static,+) -> Result<anyhow::Result<()>, tokio::task::JoinError> {+ spawn_blocking_bounded("mouse_button_down", foreground_budget(0), move || {+ press()?;+ state.mouse_hold.lock().unwrap().insert(cursor_id, hold);+ Ok(())+ })+ .await+}+pub struct MouseDragTool {state: Arc<ToolState>,}@@ -11975,7 +12583,7 @@ impl Tool for MouseDragTool {let from_x = hold.x;let from_y = hold.y;let duration_ms = args.u64_or("duration_ms", 500);- let steps = args.u64_or("steps", 20).max(1) as usize;+ let steps = drag_steps(&args);crate::overlay::send_command_for(cursor_id.clone(),cursor_overlay::OverlayCommand::PinAbove(xid),@@ -12001,6 +12609,7 @@ impl Tool for MouseDragTool {duration_ms};let mut result: anyhow::Result<()> = Ok(());+ let mut moved = 0;let mut prev_x = from_x;let mut prev_y = from_y;let is_wl = crate::wayland::is_wayland();@@ -12013,19 +12622,26 @@ impl Tool for MouseDragTool {// X11 keeps the existing input::send_motion path.let cid_inner = cursor_id.clone();let move_result = if is_wl {- let (output_x, output_y) = persistent_pointer_coordinates(+ match persistent_pointer_coordinates(is_wl,xid,ix.round() as i32,iy.round() as i32,crate::wayland::window_local_to_output,- );- tokio::task::spawn_blocking(move || {- crate::wayland::persistent_vptr::move_to(&cid_inner, output_x, output_y)- })- .await+ ) {+ Ok(AimedPoint {+ point: (output_x, output_y),+ ..+ }) => {+ spawn_blocking_bounded("mouse_drag", foreground_budget(0), move || {+ crate::wayland::persistent_vptr::move_to(&cid_inner, output_x, output_y)+ })+ .await+ }+ Err(error) => Ok(Err(error)),+ }} else {- tokio::task::spawn_blocking(move || {+ spawn_blocking_bounded("mouse_drag", foreground_budget(0), move || {crate::input::send_motion(xid,ix.round() as i32,@@ -12055,6 +12671,7 @@ impl Tool for MouseDragTool {}prev_x = ix;prev_y = iy;+ moved = i;if step_delay_ms > 0 {tokio::time::sleep(std::time::Duration::from_millis(step_delay_ms)).await;}@@ -12103,6 +12720,21 @@ impl Tool for MouseDragTool {)).with_structured(mouse_hold_json(&cursor_id, Some(&hold)))}+ Err(e) if e.is::<crate::wayland::persistent_vptr::HeldPointerEnded>() => {+ // The persistent pointer released the button and discarded+ // the cursor: it holds nothing any more.+ self.state.mouse_hold.lock().unwrap().remove(&cursor_id);+ crate::overlay::send_command_for(+ cursor_id.clone(),+ cursor_overlay::OverlayCommand::SetPressed(false),+ );+ let mut structured = mouse_hold_json(&cursor_id, None);+ structured["effect"] = json!("partial");+ ToolResult::error(format!(+ "mouse_drag stopped after {moved} of {steps} steps: {e}"+ ))+ .with_structured(structured)+ }Err(e) => ToolResult::error(e.to_string()).with_structured(mouse_hold_json(&cursor_id, Some(&hold))),}@@ -12147,6 +12779,35 @@ impl Tool for MouseButtonUpTool {.get(&cursor_id).cloned()else {+ // A press whose recording failed and whose release niri did not+ // confirm stays with the persistent pointer: release it there.+ if crate::wayland::is_wayland() {+ let cid = cursor_id.clone();+ let released =+ spawn_blocking_bounded("mouse_button_up", foreground_budget(0), move || {+ crate::wayland::persistent_vptr::release_unrecorded(&cid)+ })+ .await;+ let unheld = mouse_hold_json(&cursor_id, None);+ match released {+ Ok(Ok(true)) => {+ crate::overlay::send_command_for(+ cursor_id.clone(),+ cursor_overlay::OverlayCommand::SetPressed(false),+ );+ return ToolResult::text(format!(+ "✅ Cursor '{cursor_id}' released a button an earlier call pressed but could not record."+ ))+ .with_structured(unheld);+ }+ Ok(Ok(false)) => {}+ Ok(Err(e)) => return ToolResult::error(e.to_string()).with_structured(unheld),+ Err(e) => {+ return ToolResult::error(format!("Task error: {e}"))+ .with_structured(unheld)+ }+ }+ }return ToolResult::error(format!("No mouse button is currently held for cursor '{cursor_id}'."))@@ -12207,13 +12868,17 @@ impl Tool for MouseButtonUpTool {// (single logical drag rather than a click pair).let result = if crate::wayland::is_wayland() {let cid = cursor_id.clone();- tokio::task::spawn_blocking(move || {+ spawn_blocking_bounded("mouse_button_up", foreground_budget(0), move || {crate::wayland::persistent_vptr::release(&cid, button)}).await} else {- tokio::task::spawn_blocking(move || crate::input::send_button_up(xid, xi, yi, button))- .await+ // XSendEvent does not read the call's action, so the release+ // still goes out if the call ends first.+ spawn_blocking_bounded("mouse_button_up", foreground_budget(0), move || {+ crate::input::send_button_up(xid, xi, yi, button)+ })+ .await};match result {Ok(Ok(())) => {@@ -12519,7 +13184,15 @@ impl Tool for ParallelMouseDragTool {}let drags_for_task = drags.clone();- let result = tokio::task::spawn_blocking(move || {+ // The drags run side by side, so the longest one bounds the call.+ let budget = drags+ .iter()+ .map(|(_, drag)| {+ foreground_budget(drag.steps) + std::time::Duration::from_millis(drag.duration_ms)+ })+ .max()+ .unwrap_or_default();+ let result = spawn_blocking_bounded("parallel_mouse_drag", budget, move || {crate::input::send_parallel_virtual_pointer_drags(&drags_for_task)}).await;@@ -12621,6 +13294,12 @@ impl Tool for GetScreenSizeTool {let (frame, monitors) = crate::wayland::hyprland::screen_report()?;return Ok((frame.width, frame.height, frame.scale, Some(monitors)));}+ if crate::wayland::is_wayland() && crate::wayland::niri::is_session() {+ // niri reports logical output geometry and its fractional+ // scale over attested IPC, for a single-output layout.+ let (frame, monitors) = crate::wayland::niri::screen_report()?;+ return Ok((frame.width, frame.height, frame.scale, Some(monitors)));+ }// X11 reports pixel dimensions; scale factor on X11 is not// well-defined per-monitor, so report 1.0 (matches DPI-unaware// assumption). Other Wayland compositors retain their existing@@ -12792,6 +13471,11 @@ impl Tool for GetDesktopStateTool {} else {None};+ // niri: the single output's frame, read before the capture and+ // compared after it.+ let niri_frame = (crate::wayland::is_wayland() && crate::wayland::niri::is_session())+ .then(|| crate::wayland::niri::desktop_snapshot().map(|snapshot| snapshot.frame))+ .transpose()?;let (native_png, overlay_capture) =cursor_overlay::capture_exclusion::capture_excluding_overlays(&crate::overlay_capture::OverlayExcluder,@@ -12867,6 +13551,14 @@ impl Tool for GetDesktopStateTool {"the Hyprland monitor layout changed during get_desktop_state; call it again");frame.rebase_windows(&mut windows);+ } else if let Some(frame) = &niri_frame {+ // niri window geometry is in layout coordinates; screenshot+ // pixel (0, 0) is the single output's logical origin.+ anyhow::ensure!(+ crate::wayland::niri::desktop_snapshot()?.frame == *frame,+ "the niri output layout changed during get_desktop_state; call it again"+ );+ frame.rebase_windows(&mut windows);}let capture_id = crate::capture_action_frame::publish_desktop(&capture_service,@@ -13143,14 +13835,16 @@ impl Tool for MoveCursorTool {"xtest_desktop"};let result = if wayland {- tokio::task::spawn_blocking(move || {+ spawn_blocking_bounded("move_cursor", foreground_budget(0), move || {let space = crate::wayland::DesktopInputSpace::current()?;crate::wayland::move_cursor_desktop(&space, xi, yi)}).await} else {- tokio::task::spawn_blocking(move || crate::input::send_move_xtest_desktop(xi, yi))- .await+ spawn_blocking_bounded("move_cursor", foreground_budget(0), move || {+ crate::input::send_move_xtest_desktop(xi, yi)+ })+ .await};return match result {Ok(Ok(())) => ToolResult::text(format!(@@ -13159,6 +13853,9 @@ impl Tool for MoveCursorTool {.with_structured(json!({"scope":"desktop","path":path,"x":xi,"y":yi,"effect":"unverifiable"}),),+ Ok(Err(error)) if crate::input::foreground::error_code(&error).is_some() => {+ input_error_result(error)+ }Ok(Err(error)) => ToolResult::error(error.to_string()),Err(error) => ToolResult::error(format!("Task error: {error}")),};@@ -13953,23 +14650,27 @@ impl Tool for TypeTextCharsTool {}};let text_len = text.chars().count();- let result = tokio::task::spawn_blocking(move || {- if crate::wayland::wayland_input_enabled() {- // Per-char `wtype` loop with the requested delay — mirrors the- // X11 XSendEvent per-char path. Sleeping here is fine because- // we're inside spawn_blocking.- let mut buf = [0u8; 4];- for ch in text.chars() {- let s = ch.encode_utf8(&mut buf);- crate::wayland::type_text(xid, s)?;- if delay_ms > 0 {- std::thread::sleep(std::time::Duration::from_millis(delay_ms));+ let result = spawn_blocking_bounded(+ "type_text_chars",+ paced_typing_budget(text_len, delay_ms),+ move || {+ if crate::wayland::wayland_input_enabled() {+ // Per-char `wtype` loop with the requested delay — mirrors the+ // X11 XSendEvent per-char path. Sleeping here is fine because+ // we're inside spawn_blocking.+ let mut buf = [0u8; 4];+ for ch in text.chars() {+ let s = ch.encode_utf8(&mut buf);+ crate::wayland::type_text(xid, s)?;+ if delay_ms > 0 {+ std::thread::sleep(std::time::Duration::from_millis(delay_ms));+ }}+ return Ok(());}- return Ok(());- }- crate::input::send_type_text_with_delay(xid, &text, delay_ms)- })+ crate::input::send_type_text_with_delay(xid, &text, delay_ms)+ },+ ).await;match result {Ok(Ok(())) => ToolResult::text(format!(@@ -14692,9 +15393,16 @@ pub fn build_registry_with_provider(&state_for_session_end,session_id,|cursor_id, held| {- crate::wayland::persistent_vptr::release(cursor_id, held.button)+ crate::wayland::persistent_vptr::release_at_session_end(+ cursor_id,+ held.button,+ )},)?;+ // A press the tool never recorded (its recording failed+ // and its release was not confirmed) is released too.+ crate::wayland::persistent_vptr::release_unrecorded_at_session_end(session_id)+ .map_err(|error| error.to_string())?;} else {clear_mouse_hold_after_release(&state_for_session_end, session_id);}diff --git a/libs/cua-driver/rust/crates/platform-linux/src/wayland/compositor_peer.rs b/libs/cua-driver/rust/crates/platform-linux/src/wayland/compositor_peer.rsnew file mode 100644index 0000000000..4af87713fa--- /dev/null+++ b/libs/cua-driver/rust/crates/platform-linux/src/wayland/compositor_peer.rs@@ -0,0 +1,70 @@+//! Compositor peer attestation shared by the IPC adapters.+//!+//! An IPC socket and `WAYLAND_DISPLAY` must name the same compositor process+//! before IPC identity or geometry is applied to Wayland input or pixels.++use anyhow::{bail, Context, Result};+use std::os::fd::RawFd;+use std::path::PathBuf;+use std::time::Duration;++/// Process id of a same-user socket peer.+pub(super) fn peer_pid(fd: RawFd) -> Result<libc::pid_t> {+ let mut cred: libc::ucred = unsafe { std::mem::zeroed() };+ let mut len = std::mem::size_of::<libc::ucred>() as libc::socklen_t;+ let result = unsafe {+ libc::getsockopt(+ fd,+ libc::SOL_SOCKET,+ libc::SO_PEERCRED,+ (&mut cred as *mut libc::ucred).cast(),+ &mut len,+ )+ };+ if result != 0+ || len as usize != std::mem::size_of::<libc::ucred>()+ || cred.pid <= 0+ || cred.uid != unsafe { libc::geteuid() }+ {+ bail!("could not verify same-user compositor peer");+ }+ Ok(cred.pid)+}++/// Bound the socket connection as well as protocol dispatch. Inherited+/// WAYLAND_SOCKET descriptors are deliberately unsupported here: adapters+/// open independent, attested connections for each observation.+pub(super) fn wayland_connection_with_timeout(+ timeout: Duration,+) -> Result<wayland_client::Connection> {+ wayland_client::Connection::from_socket(wayland_socket_with_timeout(timeout)?)+ .context("Wayland connection failed")+}++/// A raw client socket to the `WAYLAND_DISPLAY` compositor, for a peer check+/// before a Wayland client (in or out of process) uses it.+pub(super) fn wayland_socket_with_timeout(+ timeout: Duration,+) -> Result<std::os::unix::net::UnixStream> {+ if std::env::var_os("WAYLAND_SOCKET").is_some() {+ bail!("compositor observation requires a named WAYLAND_DISPLAY socket");+ }+ let display =+ PathBuf::from(std::env::var_os("WAYLAND_DISPLAY").context("missing WAYLAND_DISPLAY")?);+ let path = if display.is_absolute() {+ display+ } else {+ if display.components().count() != 1 {+ bail!("invalid WAYLAND_DISPLAY socket name");+ }+ let runtime =+ PathBuf::from(std::env::var_os("XDG_RUNTIME_DIR").context("missing XDG_RUNTIME_DIR")?);+ if !runtime.is_absolute() {+ bail!("XDG_RUNTIME_DIR must be absolute");+ }+ runtime.join(display)+ };+ let socket = socket2::Socket::new(socket2::Domain::UNIX, socket2::Type::STREAM, None)?;+ socket.connect_timeout(&socket2::SockAddr::unix(path)?, timeout)?;+ Ok(socket.into())+}diff --git a/libs/cua-driver/rust/crates/platform-linux/src/wayland/ext_screencopy.rs b/libs/cua-driver/rust/crates/platform-linux/src/wayland/ext_screencopy.rsindex b103a21948..85b52f1d00 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/wayland/ext_screencopy.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/wayland/ext_screencopy.rs@@ -4,8 +4,9 @@//! on wlroots compositors (sway, labwc) but is missing from compositors//! that don't speak wlroots protocols (GNOME mutter, KDE Plasma <6.2).//! The staging `ext-image-copy-capture-v1` protocol replaces wlr-screencopy-//! across the ecosystem — sway 1.10+, labwc 0.8+, niri 0.1.6+, hyprland-//! 0.45+, KDE Plasma 6.2+, GNOME mutter 47+ all implement it.+//! across the ecosystem — sway 1.10+, labwc 0.8+, hyprland 0.45+, KDE+//! Plasma 6.2+, GNOME mutter 47+ all implement it. niri (26.04) does not; it+//! keeps wlr-screencopy.//!//! This module captures the FIRST `wl_output` advertised by the compositor//! via the chain:diff --git a/libs/cua-driver/rust/crates/platform-linux/src/wayland/hyprland.rs b/libs/cua-driver/rust/crates/platform-linux/src/wayland/hyprland.rsindex 9aaa2a0535..7f0370111c 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/wayland/hyprland.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/wayland/hyprland.rs@@ -7,11 +7,13 @@ use anyhow::{bail, Context, Result};use serde::Deserialize;use std::collections::HashSet;use std::io::{self, Read, Write};-use std::os::fd::{AsRawFd, RawFd};+use std::os::fd::AsRawFd;use std::os::unix::net::UnixStream;use std::path::PathBuf;use std::time::{Duration, Instant};+use super::compositor_peer::{peer_pid, wayland_connection_with_timeout};+const QUERY_TIMEOUT: Duration = Duration::from_secs(1);const QUERY_TOTAL_TIMEOUT: Duration = Duration::from_secs(3);const QUERY_RETRY_BACKOFF: Duration = Duration::from_millis(50);@@ -229,30 +231,8 @@ fn ipc_path() -> Result<PathBuf> {Ok(runtime.join("hypr").join(signature).join(".socket.sock"))}-pub(super) fn peer_pid(fd: RawFd) -> Result<libc::pid_t> {- let mut cred: libc::ucred = unsafe { std::mem::zeroed() };- let mut len = std::mem::size_of::<libc::ucred>() as libc::socklen_t;- let result = unsafe {- libc::getsockopt(- fd,- libc::SOL_SOCKET,- libc::SO_PEERCRED,- (&mut cred as *mut libc::ucred).cast(),- &mut len,- )- };- if result != 0- || len as usize != std::mem::size_of::<libc::ucred>()- || cred.pid <= 0- || cred.uid != unsafe { libc::geteuid() }- {- bail!("could not verify same-user compositor peer");- }- Ok(cred.pid)-}-fn ipc_connection() -> Result<UnixStream> {- ipc_connection_with_timeout(QUERY_TIMEOUT)+ ipc_connection_with_timeout(action_capped(QUERY_TIMEOUT)?)}fn ipc_connection_with_timeout(timeout: Duration) -> Result<UnixStream> {@@ -265,31 +245,7 @@ fn ipc_connection_with_timeout(timeout: Duration) -> Result<UnixStream> {/// WAYLAND_SOCKET descriptors are deliberately unsupported here: this adapter/// opens independent, attested connections for each observation.pub(super) fn wayland_connection() -> Result<wayland_client::Connection> {- wayland_connection_with_timeout(QUERY_TIMEOUT)-}--fn wayland_connection_with_timeout(timeout: Duration) -> Result<wayland_client::Connection> {- if std::env::var_os("WAYLAND_SOCKET").is_some() {- bail!("Hyprland observation requires a named WAYLAND_DISPLAY socket");- }- let display =- PathBuf::from(std::env::var_os("WAYLAND_DISPLAY").context("missing WAYLAND_DISPLAY")?);- let path = if display.is_absolute() {- display- } else {- if display.components().count() != 1 {- bail!("invalid WAYLAND_DISPLAY socket name");- }- let runtime =- PathBuf::from(std::env::var_os("XDG_RUNTIME_DIR").context("missing XDG_RUNTIME_DIR")?);- if !runtime.is_absolute() {- bail!("XDG_RUNTIME_DIR must be absolute");- }- runtime.join(display)- };- let socket = socket2::Socket::new(socket2::Domain::UNIX, socket2::Type::STREAM, None)?;- socket.connect_timeout(&socket2::SockAddr::unix(path)?, timeout)?;- wayland_client::Connection::from_socket(socket.into()).context("Wayland connection failed")+ wayland_connection_with_timeout(action_capped(QUERY_TIMEOUT)?)}/// Bind the Wayland connection to the IPC compositor before accepting pixels.@@ -326,6 +282,11 @@ fn query<T: serde::de::DeserializeOwned>(command: &str) -> Result<T> {)}+/// What is left of `cap` once capped by the action's deadline.+fn action_capped(cap: Duration) -> Result<Duration> {+ query_time_remaining(crate::interaction::operation_deadline(cap))+}+fn query_time_remaining(deadline: Instant) -> Result<Duration> {let remaining = deadline.saturating_duration_since(Instant::now());if remaining.is_zero() {@@ -352,7 +313,7 @@ fn query_with<T: serde::de::DeserializeOwned>(backoff: Duration,mut connect: impl FnMut(Instant) -> Result<UnixStream>,) -> Result<T> {- let deadline = Instant::now() + total;+ let deadline = crate::interaction::operation_deadline(total);// Keep this a closed list: a generic "j/" prefix also admits JSON-formatted// dispatch commands. JSON output does not imply a read-only operation.let read_only = matches!(@@ -387,7 +348,7 @@ fn query_with<T: serde::de::DeserializeOwned>(attempt,"Hyprland IPC observation timed out; retrying on a fresh connection");- std::thread::sleep(backoff);+ crate::interaction::pause(backoff);}}}@@ -823,7 +784,7 @@ pub fn move_cursor(x: f64, y: f64) -> Result<()> {x.round() as i64,y.round() as i64);- let reply = read_reply(&mut ipc, command.as_bytes(), QUERY_TIMEOUT)?;+ let reply = read_reply(&mut ipc, command.as_bytes(), action_capped(QUERY_TIMEOUT)?)?;if reply.trim_ascii() != b"ok" {bail!("Hyprland movecursor refused: {}",@@ -888,10 +849,14 @@ pub fn target_is_active(address: u64, pid: Option<u32>) -> Result<bool> {bail!("Hyprland target belongs to a different process");}let active: serde_json::Value = query("j/activewindow")?;- Ok(- active.get("address").and_then(|v| v.as_str()) == Some(format!("0x{address:x}").as_str())- && active.get("pid").and_then(|v| v.as_u64()) == Some(u64::from(target.pid)),- )+ let observed = active+ .get("address")+ .and_then(|v| v.as_str())+ .zip(active.get("pid").and_then(|v| v.as_u64()));+ Ok(crate::interaction::policy::focused(+ &(format!("0x{address:x}").as_str(), u64::from(target.pid)),+ &crate::interaction::facts::Fact::Known(observed),+ ))}fn capture_target(windows: &[Window], address: u64, pid: Option<u32>) -> Result<Window> {diff --git a/libs/cua-driver/rust/crates/platform-linux/src/wayland/hyprland_input.rs b/libs/cua-driver/rust/crates/platform-linux/src/wayland/hyprland_input.rsindex 5f6624f51e..92d2db2eaf 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/wayland/hyprland_input.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/wayland/hyprland_input.rs@@ -10,6 +10,7 @@ use std::sync::atomic::{AtomicBool, Ordering};use std::sync::{Arc, Mutex, MutexGuard, OnceLock, TryLockError};use std::time::{Duration, Instant};+use crate::interaction::operation_deadline;use anyhow::{bail, ensure, Context, Result};use serde_json::{json, Value};@@ -18,13 +19,6 @@ const CANCELLATION_POLL: Duration = Duration::from_millis(25);const TEXT_ACTION_GAP: Duration = Duration::from_millis(25);const MAX_PACKET: usize = 2048;const MAX_LANES: usize = 2;-/// Retries of an action the plugin refused as `stale_geometry` (the target's-/// surface box changed between TARGET and the action). Hyprland animates a-/// window for a few hundred milliseconds after a click activates it or the-/// layout changes, so an immediate retry lands in the same animation: wait-/// `STALE_GEOMETRY_BACKOFF * attempt` before each (at most ~0.6 s in all).-const MAX_STALE_GEOMETRY_RETRIES: usize = 4;-const STALE_GEOMETRY_BACKOFF: Duration = Duration::from_millis(60);#[derive(Clone, Copy, PartialEq, Eq)]enum DeliveryRoute {@@ -462,8 +456,8 @@ impl Client {let wayland = super::hyprland::wayland_connection()?;super::hyprland::verify_capture_peer(&wayland)?;ensure!(- super::hyprland::peer_pid(self.socket.as_raw_fd())?- == super::hyprland::peer_pid(wayland.backend().poll_fd().as_raw_fd())?,+ super::compositor_peer::peer_pid(self.socket.as_raw_fd())?+ == super::compositor_peer::peer_pid(wayland.backend().poll_fd().as_raw_fd())?,"input plugin and Wayland name different compositor processes");Ok(())@@ -552,7 +546,7 @@ impl Client {}fn request(&self, packet: &str) -> Result<Value> {- let deadline = Instant::now() + TIMEOUT;+ let deadline = operation_deadline(TIMEOUT);self.send_packet(packet, deadline)?;self.receive(deadline)}@@ -653,53 +647,36 @@ impl Client {self.delivery_route = Some(route);let is_drag = matches!(&action, Action::Drag { .. });let mut started = started;- for attempt in 0..=MAX_STALE_GEOMETRY_RETRIES {- // Foreground's first attestation covers the whole logical call.- // A retry must establish fresh compositor identity before it can- // request another exact target grant. Background attests here on- // every attempt because it has no call-level attestation.- if route == DeliveryRoute::Background || attempt > 0 {- attest(self)?;- }- let target = self.bind_target(&action, route)?;- if target["ok"] == false {- return Ok(target);- }- let token = hex_field(&target, "target")?;- let revision = target["revision"]- .as_u64()- .context("missing target revision")?;- let width = target["width"].as_f64().context("missing target width")?;- let height = target["height"].as_f64().context("missing target height")?;- ensure!(- width.is_finite() && height.is_finite() && width > 0.0 && height > 0.0,- "invalid target geometry"- );- self.sequence = self.sequence.checked_add(1).context("sequence exhausted")?;- let packet = action.packet(self.sequence, &token, revision, width, height)?;- let mut reply =- self.dispatch_routed_with_started(&packet, is_drag, &mut started, route)?;- if self.protocol == InputProtocol::Production- && reply["ok"] == false- && reply["code"] == "stale_geometry"- && reply["detail"] == "stale_geometry"- && reply.get("effect").is_none()- && reply.get("delivery").is_none()- && attempt < MAX_STALE_GEOMETRY_RETRIES- {- std::thread::sleep(STALE_GEOMETRY_BACKOFF * (attempt as u32 + 1));- continue;- }- if reply["ok"] == false && self.protocol == InputProtocol::Experiment {- // Never report a pending operator grant as successful dispatch.- reply["epoch"] = json!(self.epoch);- reply["challenge"] = json!(self.challenge);- reply["target"] = json!(token);- reply["revision"] = json!(revision);- }- return Ok(reply);+ // Foreground's attestation covers the whole logical call. Background+ // attests here because it has no call-level attestation.+ if route == DeliveryRoute::Background {+ attest(self)?;+ }+ let target = self.bind_target(&action, route)?;+ if target["ok"] == false {+ return Ok(target);+ }+ let token = hex_field(&target, "target")?;+ let revision = target["revision"]+ .as_u64()+ .context("missing target revision")?;+ let width = target["width"].as_f64().context("missing target width")?;+ let height = target["height"].as_f64().context("missing target height")?;+ ensure!(+ width.is_finite() && height.is_finite() && width > 0.0 && height > 0.0,+ "invalid target geometry"+ );+ self.sequence = self.sequence.checked_add(1).context("sequence exhausted")?;+ let packet = action.packet(self.sequence, &token, revision, width, height)?;+ let mut reply = self.dispatch_routed_with_started(&packet, is_drag, &mut started, route)?;+ if reply["ok"] == false && self.protocol == InputProtocol::Experiment {+ // Never report a pending operator grant as successful dispatch.+ reply["epoch"] = json!(self.epoch);+ reply["challenge"] = json!(self.challenge);+ reply["target"] = json!(token);+ reply["revision"] = json!(revision);}- unreachable!("bounded stale geometry retry loop always returns")+ Ok(reply)}fn dispatch_routed_with_started(@@ -709,7 +686,7 @@ impl Client {started: &mut Option<tokio::sync::oneshot::Sender<()>>,route: DeliveryRoute,) -> Result<Value> {- let deadline = Instant::now() + TIMEOUT;+ let deadline = operation_deadline(TIMEOUT);self.send_packet(packet, deadline).map_err(|error| {// Cancellation here precedes sending the action. Once sent, even// cancellation while waiting for its first reply is indeterminate.@@ -734,7 +711,7 @@ impl Client {let _ = started.send(());}reply = self- .receive(Instant::now() + TIMEOUT)+ .receive(operation_deadline(TIMEOUT)).map_err(|error| unknown_dispatch(error, 1))?;}if route == DeliveryRoute::Foreground@@ -915,7 +892,10 @@ impl SessionClient {}return Ok(slot);}- Err(TryLockError::WouldBlock) => std::thread::sleep(CANCELLATION_POLL),+ Err(TryLockError::WouldBlock) => {+ crate::input::action::admit(|| ())?;+ crate::interaction::pause(CANCELLATION_POLL);+ }Err(TryLockError::Poisoned(_)) => bail!("input connection poisoned"),}}@@ -1312,7 +1292,8 @@ fn execute_text_actions(// fence. Give the target event loop a bounded turn before the// next character so native GTK clients do not drop a burst.if index + 1 < action_count {- std::thread::sleep(TEXT_ACTION_GAP);+ let gap = operation_deadline(TEXT_ACTION_GAP);+ std::thread::sleep(gap.saturating_duration_since(Instant::now()));}}Err(error) => {@@ -1900,66 +1881,6 @@ mod tests {}}- #[test]- fn production_entry_bounds_stale_geometry_retry_with_fresh_authority() {- for (route, succeeds) in [- (DeliveryRoute::Foreground, true),- (DeliveryRoute::Background, true),- (DeliveryRoute::Foreground, false),- (DeliveryRoute::Background, false),- ] {- reset_test_attestations();- let (client, peer) = production_test_client();- let acknowledgement = route.acknowledgement();- // Success on the second attempt, or every attempt refused.- let attempts = if succeeds {- 2- } else {- MAX_STALE_GEOMETRY_RETRIES as u64 + 1- };- let server = std::thread::spawn(move || {- for sequence in 1..=attempts {- serve_key_target(&peer, route, sequence, sequence * 11);- let reply = if sequence == attempts && succeeds {- json!({"ok":true,"effect":"unverifiable","route":acknowledgement})- } else {- json!({"ok":false,"code":"stale_geometry","detail":"stale_geometry"})- };- peer.send(reply.to_string().as_bytes()).unwrap();- }- let mut byte = [0u8];- assert_eq!(- unsafe { libc::recv(peer.as_raw_fd(), byte.as_mut_ptr().cast(), 1, 0) },- 0- );- });-- let mut slot = Some(client);- let started = Instant::now();- let reply = dispatch_production_key(&mut slot, route).unwrap();- assert_eq!(reply["ok"], succeeds);- // Each retry waits for the window to settle, a little longer- // every time.- let waited: Duration = (1..attempts as u32)- .map(|n| STALE_GEOMETRY_BACKOFF * n)- .sum();- assert!(- started.elapsed() >= waited,- "{:?} < {waited:?}",- started.elapsed()- );- if !succeeds {- assert_eq!(reply["code"], "stale_geometry");- assert!(reply.get("effect").is_none());- assert!(reply.get("delivery").is_none());- }- assert_eq!(slot.as_ref().unwrap().sequence, attempts);- assert_eq!(test_attestations(), attempts as usize);- drop(slot);- server.join().unwrap();- }- }-#[test]fn production_entry_never_retries_an_uncertain_dispatch() {for route in [DeliveryRoute::Foreground, DeliveryRoute::Background] {@@ -1985,13 +1906,13 @@ mod tests {}#[test]- fn production_entry_retries_only_the_canonical_stale_geometry_refusal() {+ fn production_entry_never_retries_a_stale_geometry_refusal() {for route in [DeliveryRoute::Foreground, DeliveryRoute::Background] {reset_test_attestations();let (client, peer) = production_test_client();let server = std::thread::spawn(move || {serve_key_target(&peer, route, 1, 11);- peer.send(br#"{"ok":false,"code":"stale_geometry","detail":"geometry_changed"}"#)+ peer.send(br#"{"ok":false,"code":"stale_geometry","detail":"stale_geometry"}"#).unwrap();let mut byte = [0u8];assert_eq!(@@ -2004,7 +1925,7 @@ mod tests {let reply = dispatch_production_key(&mut slot, route).unwrap();assert_eq!(reply["ok"], false);assert_eq!(reply["code"], "stale_geometry");- assert_eq!(reply["detail"], "geometry_changed");+ assert_eq!(reply["detail"], "stale_geometry");assert_eq!(slot.as_ref().unwrap().sequence, 1);assert_eq!(test_attestations(), 1);drop(slot);diff --git a/libs/cua-driver/rust/crates/platform-linux/src/wayland/kwin_helper.rs b/libs/cua-driver/rust/crates/platform-linux/src/wayland/kwin_helper.rsindex c6e607f183..50f7a2e24b 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/wayland/kwin_helper.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/wayland/kwin_helper.rs@@ -8,8 +8,9 @@//!//! A focus check followed by global input therefore has an unavoidable TOCTOU//! race. Until Cua has a KWin input API that binds each mutation to the target-//! token/window, this module intentionally refuses raw target-addressed input.-//! Window discovery remains available; input authorization does not.+//! token/window, [`available`] reports no target-bound input and the shared+//! delivery policy refuses raw target-addressed input. Window discovery+//! remains available; input authorization does not.use std::collections::HashSet;use std::os::unix::fs::{MetadataExt, PermissionsExt};@@ -84,24 +85,6 @@ pub fn trusted_window_for_id(pid: u32, token: u64) -> Option<KwinWindow> {.find(|window| window.pid == pid && window.token == token)}-/// Refuse a focus-bound KWin foreground transaction.-///-/// `body` is deliberately never invoked. The current helper can identify an-/// exact KWin window, but its D-Bus API is read-only; even a separate activation-/// step would not bind a later libei event to that target. More pre-dispatch-/// focus checks therefore cannot make the operation target-safe.-pub fn with_focused_window<T>(- pid: u32,- token: u64,- _body: impl FnOnce() -> anyhow::Result<T>,-) -> anyhow::Result<T> {- anyhow::bail!(- "foreground_unavailable: KWin target pid={pid} token={token} is verified, but only \- focus-bound portal/libei input is available; refusing raw input until a target-bound \- KWin input path is implemented"- )-}-fn helper_owner() -> Option<String> {let owner_raw = dbus_call("GetNameOwner", &[DEST.to_owned()]);let owner = parse_quoted_string(&owner_raw?)?;@@ -290,7 +273,7 @@ fn wait_timeout(mut child: std::process::Child, timeout: Duration) -> Option<Strstdout.read_to_end(&mut bytes).ok()?;String::from_utf8(bytes).ok()});- let deadline = std::time::Instant::now() + timeout;+ let deadline = crate::interaction::operation_deadline(timeout);loop {match child.try_wait() {Ok(Some(status)) if status.success() => return reader.join().ok().flatten(),@@ -301,7 +284,7 @@ fn wait_timeout(mut child: std::process::Child, timeout: Duration) -> Option<Strlet _ = reader.join();return None;}- Ok(None) => std::thread::sleep(Duration::from_millis(10)),+ Ok(None) => crate::interaction::pause(Duration::from_millis(10)),}}}diff --git a/libs/cua-driver/rust/crates/platform-linux/src/wayland/libei.rs b/libs/cua-driver/rust/crates/platform-linux/src/wayland/libei.rsindex 6f9985d26f..5a30b63fee 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/wayland/libei.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/wayland/libei.rs@@ -1043,7 +1043,7 @@ impl EisState {let _ = context.flush();}if !delay.is_zero() {- std::thread::sleep(delay);+ crate::interaction::pause(delay);}}btn.button(button.to_evdev(), reis::ei::button::ButtonState::Released);diff --git a/libs/cua-driver/rust/crates/platform-linux/src/wayland/mod.rs b/libs/cua-driver/rust/crates/platform-linux/src/wayland/mod.rsindex 7538b4f71e..167440c8ad 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/wayland/mod.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/wayland/mod.rs@@ -9,6 +9,7 @@//! available, window-scoped screenshots use output crops only for visible,//! compositor-attested surfaces and return a typed identity error otherwise.+mod compositor_peer;pub mod ext_screencopy;pub mod ext_toplevel;pub mod hyprland;@@ -16,6 +17,9 @@ pub mod hyprland_capture;mod hyprland_compatibility;pub mod hyprland_input;pub mod kwin_helper;+pub mod niri;+pub use niri::keyboard::call_budget as niri_keyboard_budget;+pub(crate) use niri::AimedPoint;pub mod overlay;pub mod persistent_vptr;pub(crate) mod portal;@@ -41,6 +45,7 @@ use wayland_client::{event_created_child,protocol::{wl_buffer::WlBuffer,+ wl_callback::WlCallback,wl_output::{self, WlOutput},wl_pointer::{Axis, AxisSource, ButtonState},wl_registry,@@ -79,7 +84,7 @@ thread_local! {const { std::cell::Cell::new(None) };}-struct ForegroundTargetGuard(Option<(u32, u64)>);+pub(crate) struct ForegroundTargetGuard(Option<(u32, u64)>);impl Drop for ForegroundTargetGuard {fn drop(&mut self) {@@ -87,12 +92,11 @@ impl Drop for ForegroundTargetGuard {}}-fn bind_foreground_target(pid: u32, window_id: u64) -> ForegroundTargetGuard {+pub(crate) fn bind_foreground_target(pid: u32, window_id: u64) -> ForegroundTargetGuard {let previous = CURRENT_FOREGROUND_TARGET.with(|target| target.replace(Some((pid, window_id))));ForegroundTargetGuard(previous)}-#[cfg(feature = "portal-input")]fn current_foreground_target() -> Option<(u32, u64)> {CURRENT_FOREGROUND_TARGET.with(std::cell::Cell::get)}@@ -394,6 +398,8 @@ struct State {// button press at the output centre (over the just-activated window).vptr_manager: Option<ZwlrVirtualPointerManagerV1>,output: Option<WlOutput>,+ /// wl_output v4 name of `output`, the output screencopy copies.+ output_name: Option<String>,// Logical (post-transform) output size: virtual-pointer absolute motion// maps onto the output's logical box, so rotated outputs swap axes.output_w: u32,@@ -406,6 +412,129 @@ struct State {capture: CaptureState,}+wayland_client::delegate_dispatch!(State: [WlCallback: Answered] => Answered);++// ── Bounded Wayland waits (niri) ─────────────────────────────────────────────++/// How long one round trip with niri may take.+const NIRI_ROUNDTRIP_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(1);+/// A niri pointer action's deadline when its worker serves no tool call+/// with one (see [`crate::input::action`]).+const NIRI_POINTER_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(12);++/// niri did not answer a bounded round trip in time; its connection may+/// still work.+#[derive(Debug)]+struct RoundTripTimedOut;++impl std::fmt::Display for RoundTripTimedOut {+ fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {+ f.write_str("niri did not answer a Wayland round trip in time")+ }+}++impl std::error::Error for RoundTripTimedOut {}++/// Set when the `wl_display.sync` of a [`bounded_roundtrip`] is answered.+#[derive(Clone, Default)]+struct Answered(std::sync::Arc<std::sync::atomic::AtomicBool>);++impl<D: Dispatch<WlCallback, Answered>> Dispatch<WlCallback, Answered, D> for Answered {+ fn event(+ _: &mut D,+ _: &WlCallback,+ _: wayland_client::protocol::wl_callback::Event,+ answered: &Answered,+ _: &Connection,+ _: &QueueHandle<D>,+ ) {+ answered.0.store(true, std::sync::atomic::Ordering::SeqCst);+ }+}++/// Send `wl_display.sync` and dispatch until niri answers or `deadline`+/// (at most [`NIRI_ROUNDTRIP_TIMEOUT`] away) passes. `EventQueue::roundtrip`+/// blocks until the compositor answers, so a stalled niri would hang the+/// call and leave a pressed button held.+fn bounded_roundtrip<D>(+ connection: &Connection,+ queue: &mut wayland_client::EventQueue<D>,+ state: &mut D,+ deadline: std::time::Instant,+) -> anyhow::Result<()>+where+ D: Dispatch<WlCallback, Answered> + 'static,+{+ use anyhow::Context as _;+ use std::os::fd::AsRawFd as _;+ let deadline = deadline.min(std::time::Instant::now() + NIRI_ROUNDTRIP_TIMEOUT);+ let answered = Answered::default();+ connection.display().sync(&queue.handle(), answered.clone());+ loop {+ queue+ .dispatch_pending(state)+ .context("niri Wayland dispatch failed")?;+ if answered.0.load(std::sync::atomic::Ordering::SeqCst) {+ return Ok(());+ }+ let remaining = deadline.saturating_duration_since(std::time::Instant::now());+ if remaining.is_zero() {+ return Err(RoundTripTimedOut.into());+ }+ queue.flush().context("niri Wayland flush failed")?;+ let Some(guard) = queue.prepare_read() else {+ continue;+ };+ let mut pfd = libc::pollfd {+ fd: guard.connection_fd().as_raw_fd(),+ events: libc::POLLIN,+ revents: 0,+ };+ let timeout = remaining.as_millis().clamp(1, i32::MAX as u128) as i32;+ // SAFETY: one valid pollfd that outlives the call.+ let ready = unsafe { libc::poll(&mut pfd, 1, timeout) };+ if ready < 0 {+ let error = std::io::Error::last_os_error();+ if error.kind() == std::io::ErrorKind::Interrupted {+ continue;+ }+ return Err(error).context("poll on niri's Wayland socket failed");+ }+ if ready == 0 {+ continue;+ }+ match guard.read() {+ Ok(_) => {}+ Err(wayland_client::backend::WaylandError::Io(error))+ if error.kind() == std::io::ErrorKind::WouldBlock => {}+ Err(error) => return Err(error).context("niri Wayland read failed"),+ }+ }+}++/// A round trip bounded by `deadline` on niri; elsewhere the blocking one+/// other compositors have always used.+fn sync_by(+ connection: &Connection,+ queue: &mut wayland_client::EventQueue<State>,+ state: &mut State,+ deadline: Option<std::time::Instant>,+) -> anyhow::Result<()> {+ match deadline {+ Some(deadline) => bounded_roundtrip(connection, queue, state, deadline),+ None => queue.roundtrip(state).map(drop).map_err(Into::into),+ }+}++/// The Wayland connection for capture or pointer input: on niri, a socket+/// connected within a deadline and proven to reach the recognised niri.+fn wayland_connection(niri: bool) -> anyhow::Result<Connection> {+ if niri {+ return Ok(Connection::from_socket(niri::attested_wayland_socket()?)?);+ }+ Ok(Connection::connect_to_env()?)+}+impl Dispatch<wl_registry::WlRegistry, ()> for State {fn event(state: &mut Self,@@ -474,12 +603,18 @@ impl Dispatch<WlSeat, ()> for State {impl Dispatch<WlOutput, ()> for State {fn event(state: &mut Self,- _: &WlOutput,+ output: &WlOutput,event: wl_output::Event,_: &(),_: &Connection,_: &QueueHandle<Self>,) {+ if let wl_output::Event::Name { name } = &event {+ if state.output.as_ref() == Some(output) {+ state.output_name = Some(name.clone());+ }+ return;+ }// Remember the output resolution so `click` can aim at its centre.// Modes are reported in the panel's native orientation; the geometry// transform turns them into the logical frame input coordinates use.@@ -769,14 +904,26 @@ fn capture_via_grim() -> anyhow::Result<Vec<u8>> {/// request a copy, wait for Ready, swap channels, encode PNG. Returns an error/// if any global is missing or the compositor flags the capture as failed.fn capture_via_screencopy() -> anyhow::Result<Vec<u8>> {- let conn = Connection::connect_to_env()?;+ screencopy_first_output().map(|(png, _)| png)+}++/// [`capture_via_screencopy`], plus the wl_output v4 name of the copied+/// output when the compositor reports one.+fn screencopy_first_output() -> anyhow::Result<(Vec<u8>, Option<String>)> {+ // The whole copy, connection included, waits at most this long.+ let deadline = std::time::Instant::now() + std::time::Duration::from_secs(5);+ // On niri every round trip is bounded by it as well, so a stalled niri+ // ends the capture at the deadline instead of hanging it.+ let niri = niri::is_session();+ let bounded = niri.then_some(deadline);+ let conn = wayland_connection(niri)?;let mut queue = conn.new_event_queue::<State>();let qh = queue.handle();conn.display().get_registry(&qh, ());let mut state = State::default();- queue.roundtrip(&mut state)?;- queue.roundtrip(&mut state)?; // outputs report their Mode+ sync_by(&conn, &mut queue, &mut state, bounded)?;+ sync_by(&conn, &mut queue, &mut state, bounded)?; // outputs report their Modelet manager = state.scrcopy_manager@@ -793,7 +940,6 @@ fn capture_via_screencopy() -> anyhow::Result<Vec<u8>> {let frame = manager.capture_output(0, &output, &qh, ());// Drain Buffer / Flags events; spin until Ready or Failed (or timeout).- let deadline = std::time::Instant::now() + std::time::Duration::from_secs(5);let mut buffer: Option<WlBuffer> = None;let mut pool: Option<WlShmPool> = None;let mut mmap_ptr: *mut libc::c_void = std::ptr::null_mut();@@ -801,7 +947,10 @@ fn capture_via_screencopy() -> anyhow::Result<Vec<u8>> {let mut fd: i32 = -1;loop {- queue.roundtrip(&mut state)?;+ if let Err(error) = sync_by(&conn, &mut queue, &mut state, bounded) {+ cleanup_mmap(mmap_ptr, mmap_len, fd);+ return Err(error);+ }if state.capture.failed {anyhow::bail!("compositor signalled screencopy failure");}@@ -891,10 +1040,15 @@ fn capture_via_screencopy() -> anyhow::Result<Vec<u8>> {p.destroy();}frame.destroy();- let _ = queue.roundtrip(&mut state);+ let _ = sync_by(+ &conn,+ &mut queue,+ &mut state,+ niri.then(|| std::time::Instant::now() + crate::interaction::CLEANUP_CAP),+ );cleanup_mmap(mmap_ptr, mmap_len, fd);- result+ result.map(|png| (png, state.output_name.take()))}/// Allocate an anonymous shared-memory file of `size` bytes and mmap it RW.@@ -970,7 +1124,7 @@ pub(crate) unsafe fn borrowed_fd(fd: i32) -> std::os::fd::OwnedFd {#[derive(Debug)]pub struct SurfaceIdentityUnproven {window_id: u64,- reason: &'static str,+ reason: std::borrow::Cow<'static, str>,}impl std::fmt::Display for SurfaceIdentityUnproven {@@ -997,8 +1151,15 @@ struct WaylandWindowCrop {height: u32,}-fn surface_identity_unproven(window_id: u64, reason: &'static str) -> anyhow::Error {- SurfaceIdentityUnproven { window_id, reason }.into()+fn surface_identity_unproven(+ window_id: u64,+ reason: impl Into<std::borrow::Cow<'static, str>>,+) -> anyhow::Error {+ SurfaceIdentityUnproven {+ window_id,+ reason: reason.into(),+ }+ .into()}fn attested_wayland_crop(@@ -1110,6 +1271,9 @@ fn screenshot_dispatch_for_pid(xid: u64, pid: Option<u32>) -> anyhow::Result<Vec)});}+ if niri::is_session() {+ return niri_window_capture(xid, pid);+ }if is_wayland() {// A GNOME helper with API 9+ snapshots the window's own actor, so a// covering window never shows. It is then the only GNOME route: a@@ -1136,6 +1300,67 @@ fn screenshot_dispatch_for_pid(xid: u64, pid: Option<u32>) -> anyhow::Result<Vec)}+/// A niri display capture: only the attested wlroots screencopy, oriented by+/// the copied output's own transform. grim, ext-image-copy and the portal+/// are not tied to the attested compositor or orient differently, so niri+/// never falls back to them.+fn niri_display_capture() -> anyhow::Result<niri::DisplayCapture> {+ // Read the transforms on both sides of the copy: a change in between+ // would orient the buffer with a transform it was not taken in.+ let before = niri::output_transforms()?;+ let (png, output) = screencopy_first_output()?;+ let output = output.ok_or_else(|| {+ anyhow::anyhow!("niri capture needs the wl_output name (wl_output v4) of the copied output")+ })?;+ let transform = niri::output_transforms()?+ .get(&output)+ .copied()+ .ok_or_else(|| anyhow::anyhow!("niri reports no enabled output {output:?}"))?;+ if before.get(&output) != Some(&transform) {+ anyhow::bail!("the transform of niri output {output:?} changed during capture");+ }+ let png = orient_png_for_niri_transform(png, transform)?;+ let size = image::ImageReader::new(std::io::Cursor::new(&png))+ .with_guessed_format()?+ .into_dimensions()?;+ Ok(niri::DisplayCapture {+ png,+ output,+ transform,+ size,+ })+}++/// niri has no per-toplevel capture protocol: crop a settled display capture+/// to a placed, unoverlapped floating window and scale it to the window's+/// logical size, the window-local coordinate frame input uses. A tiled+/// window refuses: niri does not report where it is.+fn niri_window_capture(xid: u64, pid: Option<u32>) -> anyhow::Result<Vec<u8>> {+ let (png, plan) = niri::capture_window(xid, pid, niri_display_capture)+ .map_err(|reason| surface_identity_unproven(xid, format!("niri: {reason}")))?;+ let pixels = plan.pixels;+ let crop = crop_png_to_rect(+ &png,+ pixels.x,+ pixels.y,+ pixels.width,+ pixels.height,+ &format!("niri window {xid}"),+ )?;+ let image = image::load_from_memory_with_format(&crop, image::ImageFormat::Png)?;+ if (image.width(), image.height()) == (plan.width, plan.height) {+ return Ok(crop);+ }+ let image = image.resize_exact(+ plan.width,+ plan.height,+ image::imageops::FilterType::Triangle,+ );+ let mut out = std::io::Cursor::new(Vec::new());+ image.write_to(&mut out, image::ImageFormat::Png)?;+ Ok(out.into_inner())+}+fn crop_png_to_rect(output_png: &[u8],rect_x: i32,@@ -1176,6 +1401,9 @@ fn crop_png_to_rect(/// request has a 30-second deadline, after which the cascade continues./// 5. X11: existing root-window path.pub fn screenshot_display_dispatch() -> anyhow::Result<Vec<u8>> {+ if niri::is_session() {+ return niri_display_capture().map(|capture| capture.png);+ }if is_wayland() && hyprland::is_session() {// The desktop action frame spans every powered output. The cascade// below copies only the first output, so a multi-monitor layout is@@ -1198,17 +1426,17 @@ pub fn screenshot_display_dispatch() -> anyhow::Result<Vec<u8>> {}// Tier 2: native wlroots screencopy (fast, zero consent).match screenshot_bytes() {- Ok(bytes) => return orient_hyprland_display_png(bytes),+ Ok(bytes) => return orient_display_png(bytes),Err(e) => {tracing::debug!("wlroots screencopy unavailable ({e}); trying ext-image-copy-capture-v1");}}- // Tier 3: ext-image-copy-capture-v1 (sway 1.10+, labwc 0.8+, niri,+ // Tier 3: ext-image-copy-capture-v1 (sway 1.10+, labwc 0.8+,// hyprland, KDE 6.2+, GNOME 47+).match ext_screencopy::screenshot_via_ext_copy() {- Ok(bytes) => return orient_hyprland_display_png(bytes),+ Ok(bytes) => return orient_display_png(bytes),Err(e) => {tracing::debug!("ext-image-copy-capture-v1 unavailable ({e}); trying xdg-desktop-portal"@@ -1255,15 +1483,33 @@ fn rotate_png_for_output_transform(png: Vec<u8>, transform: u32) -> anyhow::Resu}/// Full-display screencopy returns the output buffer in the panel's native-/// orientation. On Hyprland, turn it into the logical desktop frame so the-/// screenshot matches what the user sees and what desktop-scope input uses.-fn orient_hyprland_display_png(png: Vec<u8>) -> anyhow::Result<Vec<u8>> {+/// orientation. On Hyprland, turn it into the logical desktop frame+/// so the screenshot matches what the user sees and what desktop-scope input+/// uses.+fn orient_display_png(png: Vec<u8>) -> anyhow::Result<Vec<u8>> {if !hyprland::is_session() {return Ok(png);}rotate_png_for_output_transform(png, hyprland::single_output_transform()?)}+/// niri output transforms: rotations as on Hyprland, plus the two pure+/// flips. A nested niri forces `Flipped180` (a vertical flip) on its winit+/// output, so its screencopy buffer is upside down. Flips combined with a+/// quarter turn are refused rather than guessed.+fn orient_png_for_niri_transform(png: Vec<u8>, transform: u32) -> anyhow::Result<Vec<u8>> {+ let flip: fn(&image::DynamicImage) -> image::DynamicImage = match transform {+ 0..=3 => return rotate_png_for_output_transform(png, transform),+ 4 => image::DynamicImage::fliph,+ 6 => image::DynamicImage::flipv,+ other => anyhow::bail!("niri output transform {other} is not supported for capture"),+ };+ let img = image::load_from_memory_with_format(&png, image::ImageFormat::Png)?;+ let mut out = Vec::new();+ flip(&img).write_to(&mut std::io::Cursor::new(&mut out), image::ImageFormat::Png)?;+ Ok(out)+}+fn checked_shell_helper_capture(available: bool,capture: impl FnOnce() -> Option<Vec<u8>>,@@ -1316,11 +1562,23 @@ fn is_no_vptr(err: &anyhow::Error) -> bool {/// feature) propagates unchanged. This is the single seam through which #1982's/// KDE/GNOME input recovery flows.fn with_libei_fallback<T>(+ f: impl FnOnce() -> anyhow::Result<T>,+ fallback: impl FnOnce() -> anyhow::Result<T>,+) -> anyhow::Result<T> {+ libei_recovery(niri::is_session(), f, fallback)+}++/// [`with_libei_fallback`] for a session known to be (or not be) niri.+/// libei finds its own EIS socket or portal, which nothing ties to the+/// attested niri, so under niri the failure is refused instead.+fn libei_recovery<T>(+ on_niri: bool,f: impl FnOnce() -> anyhow::Result<T>,#[allow(unused_variables)] fallback: impl FnOnce() -> anyhow::Result<T>,) -> anyhow::Result<T> {match f() {Ok(v) => Ok(v),+ Err(e) if on_niri && is_no_vptr(&e) => Err(niri_refuses_libei(e.to_string())),Err(e) if is_no_vptr(&e) => {#[cfg(feature = "portal-input")]{@@ -1354,9 +1612,74 @@ pub struct VptrSession {/// coordinates; [`VptrSession::abs`] converts them for `motion_absolute`.pub origin_x: i32,pub origin_y: i32,+ /// On niri, the deadline every round trip is bounded by.+ deadline: Option<std::time::Instant>,+ /// Buttons pressed and not released yet: dropping the session releases+ /// them, whatever stopped the action.+ held: Vec<u32>,+ /// Destroyed by [`VptrSession::finish`], or handed on by+ /// [`VptrSession::keep`].+ done: bool,}impl VptrSession {+ /// Wait until the compositor has handled every request sent so far.+ fn sync(&mut self) -> anyhow::Result<()> {+ sync_by(&self.conn, &mut self.queue, &mut self.state, self.deadline)+ }++ /// Send pointer events, unless the tool call this action serves is over+ /// (see [`crate::input::action`]), then wait for the compositor.+ fn send(&mut self, events: impl FnOnce(&ZwlrVirtualPointerV1)) -> anyhow::Result<()> {+ let (conn, vptr) = (&self.conn, &self.vptr);+ crate::input::action::admit(|| {+ events(vptr);+ vptr.frame();+ conn.flush()+ })??;+ self.sync()+ }++ fn motion(&mut self, px: u32, py: u32) -> anyhow::Result<()> {+ let (w, h) = (self.output_w, self.output_h);+ self.send(|vptr| vptr.motion_absolute(event_time_ms(), px, py, w, h))+ }++ fn press(&mut self, button: u32) -> anyhow::Result<()> {+ let (conn, vptr) = (&self.conn, &self.vptr);+ crate::input::action::admit(|| {+ vptr.button(event_time_ms(), button, ButtonState::Pressed);+ vptr.frame();+ conn.flush()+ })??;+ self.held.push(button);+ self.sync()+ }++ /// Release `button`. Cleanup: it goes out even once the call is over.+ fn release(&mut self, button: u32) -> anyhow::Result<()> {+ self.held.retain(|held| *held != button);+ self.vptr+ .button(event_time_ms(), button, ButtonState::Released);+ self.vptr.frame();+ self.sync()+ }++ /// Destroy the virtual pointer once the action is complete.+ fn finish(mut self) -> anyhow::Result<()> {+ self.done = true;+ self.vptr.destroy();+ self.sync()+ }++ /// Hand the connection and the virtual pointer, with its pressed+ /// buttons, to an owner that keeps them across calls.+ fn keep(mut self) -> (Connection, ZwlrVirtualPointerV1) {+ self.done = true;+ self.held.clear();+ (self.conn.clone(), self.vptr.clone())+ }+/// Layout point -> clamped `motion_absolute` coordinates.pub fn abs(&self, x: i32, y: i32) -> (u32, u32) {pointer_abs(@@ -1386,6 +1709,13 @@ pub(crate) fn pointer_abs()}+/// Whether `motion_absolute` reaches layout point `(x, y)` itself rather+/// than a point [`pointer_abs`] clamped it to.+fn pointer_reaches(origin_x: i32, origin_y: i32, w: u32, h: u32, x: i32, y: i32) -> bool {+ let (px, py) = pointer_abs(origin_x, origin_y, w, h, x, y);+ (origin_x + px as i32, origin_y + py as i32) == (x, y)+}+/// How one desktop-scope action is addressed: desktop-frame points (from/// `get_desktop_state`) convert to layout coordinates, and the virtual pointer/// is sized, from a single compositor snapshot. Other compositors keep the@@ -1399,6 +1729,8 @@ impl DesktopInputSpace {pub fn current() -> anyhow::Result<Self> {let snapshot = if is_wayland() && hyprland::is_session() {Some(hyprland::desktop_snapshot()?)+ } else if niri::is_session() {+ Some(niri::desktop_snapshot()?)} else {None};@@ -1438,15 +1770,33 @@ impl VptrSession {/// from the keyboard) never got the wheel or the button that followed./// Approaching from the next pixel always moves the pointer.fn point_at(&mut self, px: u32, py: u32) -> anyhow::Result<()> {- let (w, h) = (self.output_w, self.output_h);- self.vptr- .motion_absolute(event_time_ms(), approach_pixel(px, w), py, w, h);- self.vptr.frame();- self.queue.roundtrip(&mut self.state)?;- self.vptr.motion_absolute(event_time_ms(), px, py, w, h);- self.vptr.frame();- self.queue.roundtrip(&mut self.state)?;- Ok(())+ self.motion(approach_pixel(px, self.output_w), py)?;+ self.motion(px, py)+ }+}++impl Drop for VptrSession {+ /// An action that stopped early still releases what it pressed, on a+ /// short deadline of its own on niri.+ fn drop(&mut self) {+ if self.done {+ return;+ }+ for button in std::mem::take(&mut self.held) {+ self.vptr+ .button(event_time_ms(), button, ButtonState::Released);+ self.vptr.frame();+ }+ self.vptr.destroy();+ let _ = match self.deadline {+ Some(_) => bounded_roundtrip(+ &self.conn,+ &mut self.queue,+ &mut self.state,+ std::time::Instant::now() + crate::interaction::CLEANUP_CAP,+ ),+ None => self.conn.flush().map_err(Into::into),+ };}}@@ -1465,6 +1815,11 @@ fn approach_pixel(px: u32, extent: u32) -> u32 {/// and the virtual pointer both read this, so a desktop screenshot pixel and a/// pointer coordinate always share one frame.pub(crate) fn compositor_logical_frame() -> Option<anyhow::Result<(u32, u32)>> {+ if niri::is_session() {+ return Some(+ niri::desktop_snapshot().map(|snapshot| (snapshot.frame.width, snapshot.frame.height)),+ );+ }hyprland::is_session().then(|| hyprland::screen_size().map(|(width, height, _scale)| (width, height)))}@@ -1482,6 +1837,8 @@ pub(crate) fn compositor_logical_frame() -> Option<anyhow::Result<(u32, u32)>> {fn virtual_pointer_space(output_mode: (u32, u32)) -> anyhow::Result<(i32, i32, u32, u32)> {let layout = if hyprland::is_session() {hyprland::pointer_space()?+ } else if niri::is_session() {+ Some(niri::pointer_space()?)} else {None};@@ -1512,15 +1869,23 @@ fn open_vptr_session_in(activate_window_id: Option<u64>,snapshot: Option<&hyprland::DesktopSnapshot>,) -> anyhow::Result<VptrSession> {- let conn = Connection::connect_to_env()?;+ // A niri window is activated by the pointer gate, once the action is+ // admitted; its id never reaches the foreign-toplevel match below.+ let niri = niri::is_session();+ let deadline = niri.then(|| {+ crate::input::action::current().map_or_else(+ || std::time::Instant::now() + NIRI_POINTER_TIMEOUT,+ |action| action.deadline(),+ )+ });+ let conn = wayland_connection(niri)?;let mut queue = conn.new_event_queue::<State>();let qh = queue.handle();conn.display().get_registry(&qh, ());let mut state = State::default();- queue.roundtrip(&mut state)?;- for _ in 0..4 {- queue.roundtrip(&mut state)?;+ for _ in 0..5 {+ sync_by(&conn, &mut queue, &mut state, deadline)?;}// Evaluate the virtual-pointer / NO_VPTR_MARKER path FIRST: on compositors@@ -1555,7 +1920,7 @@ fn open_vptr_session_in(// foreign-toplevel is only needed to activate a specific window before// synthesising input; require it only when a caller actually asks for that.- if activate_window_id.is_some() && state.manager.is_none() {+ if activate_window_id.is_some() && !niri && state.manager.is_none() {anyhow::bail!("compositor does not expose zwlr_foreign_toplevel_manager_v1");}@@ -1563,11 +1928,11 @@ fn open_vptr_session_in(anyhow::anyhow!("compositor exposed no wl_seat for virtual-pointer input")})?;- if let Some(id) = activate_window_id {+ if let Some(id) = activate_window_id.filter(|_| !niri) {let handle = matching_handle(&state, id).ok_or_else(|| anyhow::anyhow!("no native Wayland toplevel for window_id {id}"))?;handle.activate(&seat);- queue.roundtrip(&mut state)?;+ sync_by(&conn, &mut queue, &mut state, deadline)?;}let vptr = mgr.create_virtual_pointer(Some(&seat), &qh, ());@@ -1587,6 +1952,9 @@ fn open_vptr_session_in(output_h,origin_x,origin_y,+ deadline,+ held: Vec::new(),+ done: false,})}@@ -1619,6 +1987,11 @@ pub fn activate_window_for_input_target(}anyhow::bail!("foreground_unavailable: exact-address Hyprland activation is not implemented; refusing title-based activation");}+ if niri::is_session() {+ // A niri id never enters the foreign-toplevel title route: focus the+ // exact window and read the focus back.+ return niri::activate(window_id, target_pid);+ }if is_inject_mode() {let pid = target_pid.ok_or_else(|| {anyhow::anyhow!(@@ -1627,7 +2000,7 @@ pub fn activate_window_for_input_target(})?;inject_send(&[format!("f {pid}")])?;remember_inject_focused_target(pid, window_id);- std::thread::sleep(std::time::Duration::from_millis(60));+ crate::interaction::pause(std::time::Duration::from_millis(60));return Ok(());}@@ -1648,25 +2021,24 @@ pub fn activate_window_for_input_target() {let protocol_id = handle.id().protocol_id();handle.activate(&seat);- let deadline = std::time::Instant::now() + std::time::Duration::from_millis(500);- loop {+ let deadline =+ crate::interaction::operation_deadline(std::time::Duration::from_millis(500));+ let activated = crate::interaction::confirm(deadline, |_| {queue.roundtrip(&mut state)?;- if state- .toplevels- .get(&protocol_id)- .is_some_and(|toplevel| toplevel.activated)- {- return Ok(());- }- if std::time::Instant::now() >= deadline {- break;- }- std::thread::sleep(std::time::Duration::from_millis(10));+ anyhow::Ok(+ state+ .toplevels+ .get(&protocol_id)+ .is_some_and(|toplevel| toplevel.activated),+ )+ })?;+ if activated {+ return Ok(());}}if shell_helper::activate_window(window_id) {- std::thread::sleep(std::time::Duration::from_millis(60));+ crate::interaction::pause(std::time::Duration::from_millis(60));return Ok(());}@@ -1677,37 +2049,6 @@ pub fn activate_window_for_input_target()}-/// Run a focus-bound keyboard transaction only after a compositor adapter has-/// confirmed the exact PID/window pair, and restore the previously focused-/// toplevel afterward. Global virtual-keyboard/libei input is refused when the-/// compositor cannot provide this read-back contract.-pub fn with_target_foreground<T>(- pid: u32,- window_id: u64,- body: impl FnOnce() -> anyhow::Result<T>,-) -> anyhow::Result<T> {- let _target_guard = bind_foreground_target(pid, window_id);- if let Some(window) = sway_ipc::window_for_id(window_id) {- if window.pid != pid {- anyhow::bail!(- "foreground_unavailable: Sway window {window_id} belongs to pid {}, not pid {pid}",- window.pid- );- }- return sway_ipc::with_focused_container(window_id, body);- }- if kwin_helper::trusted_window_for_id(pid, window_id).is_some() {- return kwin_helper::with_focused_window(pid, window_id, body);- }- if shell_helper::trusted_window_for_id(pid, window_id).is_some() {- return shell_helper::with_focused_window(pid, window_id, body);- }- anyhow::bail!(- "foreground_unavailable: no trusted Wayland compositor adapter can confirm exact \- target window {window_id} for pid {pid}; no global keyboard input was sent"- )-}-/// Query one `wl_output`'s pixel dimensions via a short Wayland roundtrip,/// independent of the virtual-pointer protocol. Only used by the libei click to/// resolve its `(0, 0)` centre default; it is not the desktop layout, so it@@ -1752,31 +2093,38 @@ fn event_time_ms() -> u32 {/// the legacy focus-based behaviour is preserved when callers can't supply/// real coords. A short delay between iterations gives the compositor time/// to discriminate single vs. double clicks.-pub fn click(window_id: u64, x: i32, y: i32, count: u32, button: u8) -> anyhow::Result<()> {+pub fn click(window_id: u64, at: AimedPoint, count: u32, button: u8) -> anyhow::Result<()> {with_libei_fallback(- || click_vptr(Some(window_id), x, y, count, button, None),+ || click_vptr(Some(window_id), at, count, button, None),|| {libei_wait_pointer_ready()?;activate_window_for_input(window_id)?;- libei_click(x, y, count, button)+ libei_click(at.point.0, at.point.1, count, button)},)}/// Click through a foreground target already guarded by-/// [`with_target_foreground`]. wlroots keeps its native virtual-pointer path.+/// [`crate::interaction::with_target_foreground`]. wlroots keeps its native+/// virtual-pointer path./// If that path is unavailable and libei needs a potentially blocking portal/// readiness wait, re-enter the exact foreground guard *after* readiness so/// KWin/GNOME/Sway ownership and focus are freshly verified immediately before/// the actual libei dispatch.#[cfg(feature = "portal-input")]-pub fn click_focused(x: i32, y: i32, count: u32, button: u8) -> anyhow::Result<()> {+pub fn click_focused(at: AimedPoint, count: u32, button: u8) -> anyhow::Result<()> {+ let (x, y) = at.point;with_libei_fallback(- || click_vptr(None, x, y, count, button, None),+ || click_vptr(None, at, count, button, None),|| {libei_wait_pointer_ready()?;if let Some((pid, window_id)) = current_foreground_target() {- return with_target_foreground(pid, window_id, || libei_click(x, y, count, button));+ return crate::interaction::with_target_foreground(+ pid,+ window_id,+ crate::interaction::RestoreMode::RestorePrevious,+ || libei_click(x, y, count, button),+ );}libei_click(x, y, count, button)},@@ -1784,8 +2132,8 @@ pub fn click_focused(x: i32, y: i32, count: u32, button: u8) -> anyhow::Result<(}#[cfg(not(feature = "portal-input"))]-pub fn click_focused(x: i32, y: i32, count: u32, button: u8) -> anyhow::Result<()> {- click_vptr(None, x, y, count, button, None)+pub fn click_focused(at: AimedPoint, count: u32, button: u8) -> anyhow::Result<()> {+ click_vptr(None, at, count, button, None)}/// Click a desktop-absolute point without selecting or activating a toplevel.@@ -1804,57 +2152,187 @@ pub fn click_desktop(}let (x, y) = space.input_layout(x, y)?;with_libei_fallback(- || click_vptr(None, x, y, count, button, space.snapshot.as_ref()),+ || click_vptr(None, (x, y).into(), count, button, Some(space)),|| libei_click(x, y, count, button),)}+/// The niri pointer space now, read on an attested connection, or `None`+/// off niri.+fn niri_pointer_space_now() -> anyhow::Result<Option<(i32, i32, u32, u32)>> {+ if !niri::is_session() {+ return Ok(None);+ }+ niri::pointer_space()+ .map(Some)+ .map_err(niri::input_unavailable)+}++/// Refuse when the compositor's pointer space no longer matches the extent+/// the session scales `motion_absolute` by: every motion would land elsewhere.+fn pointer_space_holds(+ session: (i32, i32, u32, u32),+ current: (i32, i32, u32, u32),+) -> anyhow::Result<()> {+ let current = select_virtual_pointer_space((1, 1), Some(current));+ anyhow::ensure!(+ session == current,+ "foreground_unavailable: the niri pointer space changed from {session:?} to {current:?}; take a new screenshot"+ );+ Ok(())+}++/// Deliver the `count` events of one gesture, the presses of a click train+/// or the ticks of a scroll, rechecking right before each one. `between`+/// separates two events (a pause, and any motion that re-aims the pointer)+/// and runs before the recheck, so nothing waits between the check and the+/// event. A first press can close its target or open another window over+/// the point, so the next press would reach whatever is there now: it stops+/// instead. A refusal after an event landed says how many did, without the+/// `foreground_unavailable` prefix that would report no effect.+fn gated_events<S>(+ sess: &mut S,+ count: u32,+ (gesture, events): (&str, &str),+ mut between: impl FnMut(&mut S) -> anyhow::Result<()>,+ mut recheck: impl FnMut(&S) -> anyhow::Result<()>,+ mut deliver: impl FnMut(&mut S, u32) -> anyhow::Result<()>,+) -> anyhow::Result<()> {+ let count = count.max(1);+ for i in 0..count {+ if i > 0 {+ between(sess)?;+ }+ if let Err(refusal) = recheck(sess) {+ if i == 0 {+ return Err(refusal);+ }+ anyhow::bail!("{gesture} stopped after {i} of {count} {events}: {refusal}");+ }+ deliver(sess, i)?;+ }+ Ok(())+}++/// Move a pressed button through `path`, rechecking right before each motion.+/// The layout or the pointer space can change while the button is held (an+/// output rescaled by the press), and the rest of the path would then land+/// elsewhere: a refusal releases the button where the pointer is and says how+/// far the drag got. The press went out, so it never reads as no effect.+fn held_motions<S>(+ sess: &mut S,+ path: impl ExactSizeIterator<Item = (u32, u32)>,+ mut recheck: impl FnMut(&S) -> anyhow::Result<()>,+ mut motion: impl FnMut(&mut S, (u32, u32)) -> anyhow::Result<()>,+ release: impl FnOnce(&mut S) -> anyhow::Result<()>,+) -> anyhow::Result<()> {+ let total = path.len();+ for (i, point) in path.enumerate() {+ if let Err(refusal) = recheck(sess) {+ match release(sess) {+ Ok(()) => anyhow::bail!(+ "drag stopped after {i} of {total} motions, and the button was released: {refusal}"+ ),+ Err(error) => anyhow::bail!(+ "drag stopped after {i} of {total} motions, and releasing the button failed ({error:#}): {refusal}"+ ),+ }+ }+ motion(sess, point)?;+ }+ Ok(())+}++/// The legacy wlroots route sends a click at `(0, 0)` to the output centre.+/// On Hyprland and niri callers address the global layout, where `(0, 0)` is+/// a real point (a window or bar at the top-left corner).+fn centres_origin_click(x: i32, y: i32, layout_addressed: bool) -> bool {+ x == 0 && y == 0 && !layout_addressed+}++/// On niri, admit the action's pointer input through [`niri::prepare_target`]:+/// input addressed to a window, by id or by an outer foreground target,+/// proves each layout point lands on that window and then activates it.+/// Desktop-scope input names no window: its points are pixels of the single+/// output. Recheck the gate with [`niri_gate_holds`] before each event.+fn niri_pointer_gate(+ window_id: Option<u64>,+ desktop: Option<&DesktopInputSpace>,+ sess: &VptrSession,+ points: &[AimedPoint],+) -> anyhow::Result<Option<niri::PointerGate>> {+ if !niri::is_session() {+ return Ok(None);+ }+ let window_id = window_id.or_else(|| {+ desktop+ .is_none()+ .then(current_foreground_target)+ .flatten()+ .map(|(_, id)| id)+ });+ let target = match (window_id, desktop) {+ (Some(id), _) => niri::Target::Window(id),+ (None, Some(_)) => niri::Target::Desktop,+ (None, None) => niri::Target::Unaddressed,+ };+ let space = (sess.origin_x, sess.origin_y, sess.output_w, sess.output_h);+ niri::prepare_target(target, space, points)+}++fn niri_gate_holds(gate: &mut Option<niri::PointerGate>) -> anyhow::Result<()> {+ gate.as_mut().map_or(Ok(()), niri::PointerGate::holds)+}+/// wlroots virtual-pointer implementation of [`click`]. Falls back to libei via/// [`with_libei_fallback`] when the compositor exposes no virtual-pointer.fn click_vptr(window_id: Option<u64>,- x: i32,- y: i32,+ at: AimedPoint,count: u32,button: u8,- snapshot: Option<&hyprland::DesktopSnapshot>,+ desktop: Option<&DesktopInputSpace>,) -> anyhow::Result<()> {- let mut sess = open_vptr_session_in(window_id, snapshot)?;- std::thread::sleep(std::time::Duration::from_millis(40));+ let mut sess = open_vptr_session_in(window_id, desktop.and_then(|d| d.snapshot.as_ref()))?;+ crate::interaction::pause(std::time::Duration::from_millis(40));let (w, h) = (sess.output_w, sess.output_h);- let (px, py) = if x == 0 && y == 0 && !hyprland::is_session() {+ let (x, y) = at.point;+ let centre = centres_origin_click(x, y, hyprland::is_session() || niri::is_session());+ let (px, py) = if centre {(w / 2, h / 2)} else {sess.abs(x, y)};+ let aimed = if centre {+ (sess.origin_x + px as i32, sess.origin_y + py as i32).into()+ } else {+ at+ };+ let mut gate = niri_pointer_gate(window_id, desktop, &sess, &[aimed])?;let btn = evdev_pointer_button(button);- for i in 0..count.max(1) {- if i > 0 {- std::thread::sleep(std::time::Duration::from_millis(80));- }- if i == 0 {- sess.point_at(px, py)?;- } else {- sess.vptr.motion_absolute(event_time_ms(), px, py, w, h);- sess.vptr.frame();- sess.queue.roundtrip(&mut sess.state)?;- }- std::thread::sleep(std::time::Duration::from_millis(15));- sess.vptr.button(event_time_ms(), btn, ButtonState::Pressed);- sess.vptr.frame();- sess.queue.roundtrip(&mut sess.state)?;- std::thread::sleep(std::time::Duration::from_millis(20));- sess.vptr- .button(event_time_ms(), btn, ButtonState::Released);- sess.vptr.frame();- sess.queue.roundtrip(&mut sess.state)?;- }+ sess.point_at(px, py)?;+ crate::interaction::pause(std::time::Duration::from_millis(15));+ gated_events(+ &mut sess,+ count,+ ("click", "presses"),+ |sess| {+ crate::interaction::pause(std::time::Duration::from_millis(80));+ sess.motion(px, py)?;+ crate::interaction::pause(std::time::Duration::from_millis(15));+ Ok(())+ },+ |_| niri_gate_holds(&mut gate),+ |sess, _| {+ sess.press(btn)?;+ crate::interaction::pause(std::time::Duration::from_millis(20));+ sess.release(btn)+ },+ )?;// Keep the synthetic-cursor registry in sync with the warp we just// performed so a subsequent `get_cursor_position` reflects reality.record_synth_cursor(sess.origin_x + px as i32, sess.origin_y + py as i32);- sess.vptr.destroy();- sess.queue.roundtrip(&mut sess.state)?;- Ok(())+ sess.finish()}/// Synthesize a vertical or horizontal scroll on the activated toplevel. Each@@ -1867,10 +2345,18 @@ pub fn scroll(window_id: u64, direction: &str, amount: u32) -> anyhow::Result<()/// Translate window-local screenshot coordinates into compositor output/// coordinates when the active compositor exposes the target geometry.-pub fn window_local_to_output(window_id: u64, x: i32, y: i32) -> (i32, i32) {- window_geometry(window_id)+/// On niri a window without a reported place (a tiled one) refuses: passing+/// `(x, y)` through would aim window-local coordinates at the layout origin.+/// On niri the point carries the rectangle used, so the pointer gate refuses+/// if the window moved before the input is sent.+pub fn window_local_to_output(window_id: u64, x: i32, y: i32) -> anyhow::Result<AimedPoint> {+ if niri::is_session() {+ return niri::aim(window_id, x, y);+ }+ Ok(window_geometry(window_id).map(|(window_x, window_y, _, _)| (window_x.saturating_add(x), window_y.saturating_add(y))).unwrap_or((x, y))+ .into())}/// Resolve geometry through stable title/app identity when a foreign-toplevel@@ -1882,6 +2368,17 @@ pub fn window_geometry(window_id: u64) -> Option<(i32, i32, u32, u32)> {// an explicit native Hyprland address is missing.return hyprland::window_for_address(window_id).map(|w| (w.x, w.y, w.width, w.height));}+ if niri::is_session() {+ // Only a placed floating niri window has geometry. A tiled one has+ // none, and no title or app-id match may supply it.+ let placed = niri::window_for_id(window_id).ok()?.placement.ok()?;+ return Some((+ placed.rect.x,+ placed.rect.y,+ placed.rect.width,+ placed.rect.height,+ ));+ }if let Some(window) = sway_ipc::window_for_id(window_id) {return Some((window.x, window.y, window.width, window.height));}@@ -1949,7 +2446,7 @@ pub fn window_geometry(window_id: u64) -> Option<(i32, i32, u32, u32)> {/// pixel-addressed scrolls must not inherit an unrelated cursor position.pub fn scroll_at(window_id: u64,- point: Option<(i32, i32)>,+ point: Option<AimedPoint>,direction: &str,amount: u32,) -> anyhow::Result<()> {@@ -1959,7 +2456,7 @@ pub fn scroll_at(|| {libei_wait_scroll_ready()?;activate_window_for_input(window_id)?;- if let Some((x, y)) = point {+ if let Some(AimedPoint { point: (x, y), .. }) = point {libei_move_absolute(x, y)?;}libei_scroll(&direction, amount)@@ -1981,15 +2478,7 @@ pub fn scroll_desktop(let (x, y) = space.input_layout(x, y)?;let direction = direction.to_string();with_libei_fallback(- || {- scroll_vptr(- None,- Some((x, y)),- &direction,- amount,- space.snapshot.as_ref(),- )- },+ || scroll_vptr(None, Some((x, y).into()), &direction, amount, Some(space)),|| {libei_wait_scroll_ready()?;libei_move_absolute(x, y)?;@@ -2001,17 +2490,33 @@ pub fn scroll_desktop(/// wlroots virtual-pointer implementation of [`scroll`].fn scroll_vptr(window_id: Option<u64>,- point: Option<(i32, i32)>,+ point: Option<AimedPoint>,direction: &str,amount: u32,- snapshot: Option<&hyprland::DesktopSnapshot>,+ desktop: Option<&DesktopInputSpace>,) -> anyhow::Result<()> {- let mut sess = open_vptr_session_in(window_id, snapshot)?;- if let Some((x, y)) = point {- let (px, py) = sess.abs(x, y);+ // A niri window scroll without a point aims at the window's centre, so+ // the wheel provably reaches the window rather than whatever is under the+ // pointer.+ let point = match (point, window_id) {+ (None, Some(id)) if niri::is_session() => {+ let (_, _, width, height) = window_geometry(id).ok_or_else(|| {+ anyhow::anyhow!(+ "foreground_unavailable: niri reports no on-screen place for window {id}; pass a point inside a floating window"+ )+ })?;+ Some(niri::aim(id, (width / 2) as i32, (height / 2) as i32)?)+ }+ (point, _) => point,+ };+ let mut sess = open_vptr_session_in(window_id, desktop.and_then(|d| d.snapshot.as_ref()))?;+ let mut gate = None;+ if let Some(aimed) = point {+ let (px, py) = sess.abs(aimed.point.0, aimed.point.1);+ gate = niri_pointer_gate(window_id, desktop, &sess, &[aimed])?;sess.point_at(px, py)?;record_synth_cursor(sess.origin_x + px as i32, sess.origin_y + py as i32);- std::thread::sleep(std::time::Duration::from_millis(15));+ crate::interaction::pause(std::time::Duration::from_millis(15));}let (axis, sign): (Axis, i32) = match direction.to_ascii_lowercase().as_str() {"up" => (Axis::VerticalScroll, -1),@@ -2023,18 +2528,23 @@ fn scroll_vptr(// axis_discrete: `value` is logical units (the wayland-rs wrapper// converts to wl_fixed internally); `discrete` is the tick count.let value: f64 = (sign as f64) * 10.0;- for i in 0..amount.max(1) {- if i > 0 {- std::thread::sleep(std::time::Duration::from_millis(25));- }- sess.vptr.axis_source(AxisSource::Wheel);- sess.vptr.axis_discrete(event_time_ms(), axis, value, sign);- sess.vptr.frame();- sess.queue.roundtrip(&mut sess.state)?;- }- sess.vptr.destroy();- sess.queue.roundtrip(&mut sess.state)?;- Ok(())+ gated_events(+ &mut sess,+ amount,+ ("scroll", "ticks"),+ |_| {+ crate::interaction::pause(std::time::Duration::from_millis(25));+ Ok(())+ },+ |_| niri_gate_holds(&mut gate),+ |sess, _| {+ sess.send(|vptr| {+ vptr.axis_source(AxisSource::Wheel);+ vptr.axis_discrete(event_time_ms(), axis, value, sign);+ })+ },+ )?;+ sess.finish()}/// Last cursor position the agent warped to via `move_cursor_absolute`.@@ -2082,7 +2592,7 @@ pub fn move_cursor_absolute(window_id: Option<u64>, x: i32, y: i32) -> anyhow::Rpub fn move_cursor_desktop(space: &DesktopInputSpace, x: i32, y: i32) -> anyhow::Result<()> {let (x, y) = space.input_layout(x, y)?;with_libei_fallback(- || move_cursor_absolute_vptr(None, x, y, space.snapshot.as_ref()),+ || move_cursor_absolute_vptr(None, x, y, Some(space)),|| libei_move_absolute(x, y),)}@@ -2092,18 +2602,14 @@ fn move_cursor_absolute_vptr(window_id: Option<u64>,x: i32,y: i32,- snapshot: Option<&hyprland::DesktopSnapshot>,+ desktop: Option<&DesktopInputSpace>,) -> anyhow::Result<()> {- let mut sess = open_vptr_session_in(window_id, snapshot)?;- let (w, h) = (sess.output_w, sess.output_h);+ let mut sess = open_vptr_session_in(window_id, desktop.and_then(|d| d.snapshot.as_ref()))?;let (px, py) = sess.abs(x, y);- sess.vptr.motion_absolute(event_time_ms(), px, py, w, h);- sess.vptr.frame();- sess.queue.roundtrip(&mut sess.state)?;+ niri_pointer_gate(window_id, desktop, &sess, &[(x, y).into()])?;+ sess.motion(px, py)?;record_synth_cursor(sess.origin_x + px as i32, sess.origin_y + py as i32);- sess.vptr.destroy();- sess.queue.roundtrip(&mut sess.state)?;- Ok(())+ sess.finish()}/// Press-drag-release on a native Wayland toplevel. Emits one button press at@@ -2113,26 +2619,15 @@ fn move_cursor_absolute_vptr(/// injection socket (`CUA_INJECT_SOCKET`).pub fn drag(window_id: u64,- from: (i32, i32),- to: (i32, i32),+ from: AimedPoint,+ to: AimedPoint,steps: u32,duration_ms: u64,button: u8,) -> anyhow::Result<()> {- let ((from_x, from_y), (to_x, to_y)) = (from, to);+ let ((from_x, from_y), (to_x, to_y)) = (from.point, to.point);with_libei_fallback(- || {- drag_vptr(- Some(window_id),- from_x,- from_y,- to_x,- to_y,- steps,- button,- None,- )- },+ || drag_vptr(Some(window_id), from, to, steps, button, None),|| {libei_wait_pointer_ready()?;activate_window_for_input(window_id)?;@@ -2158,13 +2653,11 @@ pub fn drag_desktop(|| {drag_vptr(None,- from_x,- from_y,- to_x,- to_y,+ (from_x, from_y).into(),+ (to_x, to_y).into(),steps,button,- space.snapshot.as_ref(),+ Some(space),)},|| {@@ -2174,57 +2667,73 @@ pub fn drag_desktop()}+/// The most intermediate motions a drag sends: the `steps` maximum the drag+/// tools advertise.+const MAX_DRAG_STEPS: u32 = 200;++/// A drag's motions after its press: `steps` points from `from` toward `to`,+/// each aimed through `aim`, then `end`. `steps` is clamped to+/// `1..=MAX_DRAG_STEPS`, and each point is computed only when it is sent.+fn drag_path(+ from: (i32, i32),+ to: (i32, i32),+ steps: u32,+ end: (u32, u32),+ mut aim: impl FnMut(i32, i32) -> (u32, u32),+) -> impl ExactSizeIterator<Item = (u32, u32)> {+ let n = steps.clamp(1, MAX_DRAG_STEPS) as usize;+ (1..n + 2).map(move |s| {+ if s > n {+ return end;+ }+ let t = s as f64 / n as f64;+ // In i64, the distance between any two i32 points cannot overflow.+ let x = (f64::from(from.0) + (i64::from(to.0) - i64::from(from.0)) as f64 * t).round();+ let y = (f64::from(from.1) + (i64::from(to.1) - i64::from(from.1)) as f64 * t).round();+ aim(x as i32, y as i32)+ })+}+/// wlroots virtual-pointer implementation of [`drag`].-#[allow(clippy::too_many_arguments)]fn drag_vptr(window_id: Option<u64>,- from_x: i32,- from_y: i32,- to_x: i32,- to_y: i32,+ from: AimedPoint,+ to: AimedPoint,steps: u32,button: u8,- snapshot: Option<&hyprland::DesktopSnapshot>,+ desktop: Option<&DesktopInputSpace>,) -> anyhow::Result<()> {- let mut sess = open_vptr_session_in(window_id, snapshot)?;- std::thread::sleep(std::time::Duration::from_millis(40));+ let mut sess = open_vptr_session_in(window_id, desktop.and_then(|d| d.snapshot.as_ref()))?;+ crate::interaction::pause(std::time::Duration::from_millis(40));let (w, h) = (sess.output_w, sess.output_h);let btn = evdev_pointer_button(button);let (origin_x, origin_y) = (sess.origin_x, sess.origin_y);let clamp_xy = |x: i32, y: i32| pointer_abs(origin_x, origin_y, w, h, x, y);+ let ((from_x, from_y), (to_x, to_y)) = (from.point, to.point);let (fx, fy) = clamp_xy(from_x, from_y);- sess.vptr.motion_absolute(event_time_ms(), fx, fy, w, h);- sess.vptr.frame();- sess.queue.roundtrip(&mut sess.state)?;- std::thread::sleep(std::time::Duration::from_millis(15));- sess.vptr.button(event_time_ms(), btn, ButtonState::Pressed);- sess.vptr.frame();- sess.queue.roundtrip(&mut sess.state)?;- let n = steps.max(1);- for s in 1..=n {- let t = s as f64 / n as f64;- let ix = (from_x as f64 + (to_x - from_x) as f64 * t).round() as i32;- let iy = (from_y as f64 + (to_y - from_y) as f64 * t).round() as i32;- let (cx, cy) = clamp_xy(ix, iy);- sess.vptr.motion_absolute(event_time_ms(), cx, cy, w, h);- sess.vptr.frame();- sess.queue.roundtrip(&mut sess.state)?;- std::thread::sleep(std::time::Duration::from_millis(8));- }let (tx, ty) = clamp_xy(to_x, to_y);- sess.vptr.motion_absolute(event_time_ms(), tx, ty, w, h);- sess.vptr.frame();- sess.queue.roundtrip(&mut sess.state)?;- sess.vptr- .button(event_time_ms(), btn, ButtonState::Released);- sess.vptr.frame();+ let mut gate = niri_pointer_gate(window_id, desktop, &sess, &[from, to])?;+ sess.motion(fx, fy)?;+ crate::interaction::pause(std::time::Duration::from_millis(15));+ niri_gate_holds(&mut gate)?;+ sess.press(btn)?;+ let path = drag_path((from_x, from_y), (to_x, to_y), steps, (tx, ty), clamp_xy);+ held_motions(+ &mut sess,+ path,+ |_| niri_gate_holds(&mut gate),+ |sess, (x, y)| {+ sess.motion(x, y)?;+ crate::interaction::pause(std::time::Duration::from_millis(8));+ Ok(())+ },+ |sess| sess.release(btn),+ )?;// Sync the synthetic-cursor registry with the drag endpoint so a// subsequent `get_cursor_position` reports where we left the pointer.record_synth_cursor(origin_x + tx as i32, origin_y + ty as i32);- sess.queue.roundtrip(&mut sess.state)?;- sess.vptr.destroy();- sess.queue.roundtrip(&mut sess.state)?;- Ok(())+ sess.release(btn)?;+ sess.finish()}/// Type Unicode text into the focused Wayland surface via `wtype` (the@@ -2233,21 +2742,35 @@ fn drag_vptr(/// backend's XSendEvent typing and the capture slice's shell-out to `grim`./// foreign-toplevel exposes no pid and Wayland delivers keys to the *focused*/// surface, so this is window_id-free; pair it with `click`/`activate` to put-/// the intended window in focus first.+/// the intended window in focus first. Under niri, text goes out on niri's+/// own keymap instead (see [`niri::keyboard`]).pub fn type_text(window_id: u64, text: &str) -> anyhow::Result<()> {+ type_into(Some(window_id), text)+}++/// Type into a surface whose exact Sway container is already held focused by+/// the caller. This avoids re-resolving a title that can change mid-sequence+/// (for example after opening a Chromium tab). Under niri, text goes out on+/// niri's own keymap and stops if focus leaves the window that had it.+pub fn type_text_focused(text: &str) -> anyhow::Result<()> {+ type_into(None, text)+}++/// [`type_text`] into `window`, activated first, or [`type_text_focused`].+fn type_into(window: Option<u64>, text: &str) -> anyhow::Result<()> {if text.is_empty() {return Ok(());}- activate_window_for_input(window_id)?;+ window.map_or(Ok(()), activate_window_for_input)?;+ if niri::is_session() {+ return niri::keyboard::type_text(window.or_else(niri_expected_window), text);+ }// Lead with a no-op Shift_L tap: on a freshly-focused window under a headless// seat (notably sway), the compositor needs the first virtual-keyboard event// to wire up keyboard routing, and that first key is dropped. Sacrificing a// modifier tap (no character) absorbs the drop so the real text lands intact;// it's harmless where routing is already live (labwc).- let result = std::process::Command::new("wtype")- .args(["-k", "Shift_L", "--"])- .arg(text)- .output();+ let result = wtype_output(&["-k", "Shift_L", "--", text]);match result {Ok(out) if out.status.success() => Ok(()),// `wtype` relies on `zwp_virtual_keyboard_v1`, which KWin/Plasma and@@ -2257,7 +2780,7 @@ pub fn type_text(window_id: u64, text: &str) -> anyhow::Result<()> {other => with_wtype_libei_fallback(|| {libei_wait_keyboard_ready()?;- activate_window_for_input(window_id)?;+ window.map_or(Ok(()), activate_window_for_input)?;libei_type_text(text)},other.map(|o| String::from_utf8_lossy(&o.stderr).into_owned()),@@ -2265,40 +2788,27 @@ pub fn type_text(window_id: u64, text: &str) -> anyhow::Result<()> {}}-/// Type into a surface whose exact Sway container is already held focused by-/// the caller. This avoids re-resolving a title that can change mid-sequence-/// (for example after opening a Chromium tab).-pub fn type_text_focused(text: &str) -> anyhow::Result<()> {- if text.is_empty() {- return Ok(());- }- let result = std::process::Command::new("wtype")- .args(["-k", "Shift_L", "--"])- .arg(text)- .output();- match result {- Ok(out) if out.status.success() => Ok(()),- other => with_wtype_libei_fallback(- || {- libei_wait_keyboard_ready()?;- libei_type_text(text)- },- other.map(|out| String::from_utf8_lossy(&out.stderr).into_owned()),- ),- }-}-/// Type text and press one key while an outer exact-container focus guard is/// active. Keeping both operations in one virtual-keyboard lifetime avoids a/// headless wlroots seat dropping the first event from a second `wtype`/// process after the text has landed.pub fn type_text_then_key_focused(text: &str, key: &str) -> anyhow::Result<()> {let keysym = key_to_keysym(key);- let result = std::process::Command::new("wtype")- .args(["-k", "Shift_L", "-s", "30"])- .arg(text)- .args(["-s", "50", "-k", &keysym])- .output();+ if niri::is_session() {+ niri::keyboard::type_text(niri_expected_window(), text)?;+ return niri::keyboard::press(niri_expected_window(), &[], &keysym);+ }+ let result = wtype_output(&[+ "-k",+ "Shift_L",+ "-s",+ "30",+ text,+ "-s",+ "50",+ "-k",+ keysym.as_str(),+ ]);match result {Ok(out) if out.status.success() => Ok(()),other => with_wtype_libei_fallback(@@ -2314,25 +2824,7 @@ pub fn type_text_then_key_focused(text: &str, key: &str) -> anyhow::Result<()> {/// Press a single named key into the focused Wayland surface via `wtype -k`.pub fn press_key(window_id: u64, key: &str) -> anyhow::Result<()> {- activate_window_for_input(window_id)?;- let keysym = key_to_keysym(key);- // Keep the sacrificial modifier and requested key in one virtual-keyboard- // lifetime. Starting a second wtype process creates a fresh protocol object,- // causing headless seats to drop the requested key as their first event.- let result = std::process::Command::new("wtype")- .args(["-k", "Shift_L", "-k", &keysym])- .output();- match result {- Ok(out) if out.status.success() => Ok(()),- other => with_wtype_libei_fallback(- || {- libei_wait_keyboard_ready()?;- activate_window_for_input(window_id)?;- libei_press_key(key)- },- other.map(|o| String::from_utf8_lossy(&o.stderr).into_owned()),- ),- }+ press_into(Some(window_id), key)}/// Press one key while an outer exact-container focus guard is active.@@ -2341,18 +2833,29 @@ pub fn press_key_focused(key: &str) -> anyhow::Result<()> {let (pid, window_id) = inject_focused_target()?;return inject_press_key(pid, window_id, key);}+ press_into(None, key)+}++/// [`press_key`] into `window`, activated first, or [`press_key_focused`].+fn press_into(window: Option<u64>, key: &str) -> anyhow::Result<()> {+ window.map_or(Ok(()), activate_window_for_input)?;let keysym = key_to_keysym(key);- let result = std::process::Command::new("wtype")- .args(["-k", "Shift_L", "-k", &keysym])- .output();+ if niri::is_session() {+ return niri::keyboard::press(window.or_else(niri_expected_window), &[], &keysym);+ }+ // Keep the sacrificial modifier and requested key in one virtual-keyboard+ // lifetime. Starting a second wtype process creates a fresh protocol object,+ // causing headless seats to drop the requested key as their first event.+ let result = wtype_output(&["-k", "Shift_L", "-k", keysym.as_str()]);match result {Ok(out) if out.status.success() => Ok(()),other => with_wtype_libei_fallback(|| {libei_wait_keyboard_ready()?;+ window.map_or(Ok(()), activate_window_for_input)?;libei_press_key(key)},- other.map(|out| String::from_utf8_lossy(&out.stderr).into_owned()),+ other.map(|o| String::from_utf8_lossy(&o.stderr).into_owned()),),}}@@ -2363,39 +2866,7 @@ pub fn press_key_focused(key: &str) -> anyhow::Result<()> {/// straight to wtype's `-k` so single-character keys and X keysym names work/// as-is. This is the Wayland equivalent of the X11 `send_key` modifier mask.pub fn hotkey(window_id: u64, keys: &[String]) -> anyhow::Result<()> {- activate_window_for_input(window_id)?;- let (mods, final_key) = partition_modifiers(keys)?;- if let Ok(()) = virtual_keyboard::hotkey(&mods, &final_key) {- return Ok(());- }- let keysym = key_to_keysym(&final_key);- let args = wtype_hotkey_args(&mods, &keysym);- let result = std::process::Command::new("wtype").args(&args).output();- match result {- Ok(out) if out.status.success() => Ok(()),- other => {- let stderr = other.map(|o| String::from_utf8_lossy(&o.stderr).into_owned());- #[cfg(feature = "portal-input")]- {- return with_wtype_libei_fallback(- || {- libei::wait_keyboard_ready()?;- activate_window_for_input(window_id)?;- libei_hotkey(&mods, &final_key)- },- stderr,- );- }- #[cfg(not(feature = "portal-input"))]- {- anyhow::bail!(- "wtype {} failed: {}",- args.join(" "),- stderr.unwrap_or_else(|_| "wtype unavailable".into())- );- }- }- }+ chord_into(Some(window_id), keys)}/// Send a chord while an outer exact-container focus guard is active.@@ -2404,22 +2875,33 @@ pub fn hotkey_focused(keys: &[String]) -> anyhow::Result<()> {let (pid, window_id) = inject_focused_target()?;return inject_hotkey(pid, window_id, keys);}+ chord_into(None, keys)+}++/// [`hotkey`] into `window`, activated first, or [`hotkey_focused`].+fn chord_into(window: Option<u64>, keys: &[String]) -> anyhow::Result<()> {+ window.map_or(Ok(()), activate_window_for_input)?;let (mods, final_key) = partition_modifiers(keys)?;+ if niri::is_session() {+ let expected = window.or_else(niri_expected_window);+ return niri::keyboard::press(expected, &mods, &chord_keysym(&final_key));+ }if let Ok(()) = virtual_keyboard::hotkey(&mods, &final_key) {return Ok(());}let keysym = key_to_keysym(&final_key);let args = wtype_hotkey_args(&mods, &keysym);- let result = std::process::Command::new("wtype").args(&args).output();+ let result = wtype_output(&args);match result {Ok(out) if out.status.success() => Ok(()),other => {- let stderr = other.map(|out| String::from_utf8_lossy(&out.stderr).into_owned());+ let stderr = other.map(|o| String::from_utf8_lossy(&o.stderr).into_owned());#[cfg(feature = "portal-input")]{return with_wtype_libei_fallback(|| {libei::wait_keyboard_ready()?;+ window.map_or(Ok(()), activate_window_for_input)?;libei_hotkey(&mods, &final_key)},stderr,@@ -2437,6 +2919,11 @@ pub fn hotkey_focused(keys: &[String]) -> anyhow::Result<()> {}}+/// Run `wtype` with `args`.+fn wtype_output<S: AsRef<std::ffi::OsStr>>(args: &[S]) -> std::io::Result<std::process::Output> {+ std::process::Command::new("wtype").args(args).output()+}+fn wtype_hotkey_args(mods: &[String], keysym: &str) -> Vec<String> {// Keep the same harmless first-event primer used by `press_key`. A fresh// virtual-keyboard object on headless seats can drop its first event. Give@@ -2485,6 +2972,23 @@ fn partition_modifiers(keys: &[String]) -> anyhow::Result<(Vec<String>, String)>Ok((mods, final_key))}+/// The niri window focused keyboard input must reach: the target an outer+/// [`crate::interaction::with_target_foreground`] focused, or `None` for+/// desktop-scoped input into whatever has focus.+fn niri_expected_window() -> Option<u64> {+ current_foreground_target().map(|(_, window_id)| window_id)+}++/// The keysym of a chord's final key. A letter names its key whatever its+/// case, as the virtual-keyboard chord route reads it: `ctrl+A` is `ctrl+a`,+/// and Shift is held only when asked for.+fn chord_keysym(key: &str) -> String {+ match key.as_bytes() {+ [letter] if letter.is_ascii_alphabetic() => key.to_ascii_lowercase(),+ _ => key_to_keysym(key),+ }+}+/// Map cua key names to X keysym names that `wtype -k` understands. Unknown/// values pass through (single characters and valid keysym names work as-is).fn key_to_keysym(key: &str) -> String {@@ -2555,6 +3059,15 @@ fn with_wtype_libei_fallback(}}+/// Under niri, input goes only to the attested compositor: a failed+/// attested route is refused, keeping its detail, rather than recovered.+fn niri_refuses_libei(detail: String) -> anyhow::Error {+ anyhow::anyhow!(+ "foreground_unavailable: {}; under niri the libei/portal fallback is refused because it is not attested",+ detail.trim_end()+ )+}+// Stubs for the no-feature build: the dispatch seams never call these (the// libei branch in `with_libei_fallback` / `with_wtype_libei_fallback` is// `#[cfg]`-d out), but the closures still need them to exist to type-check.@@ -2650,7 +3163,7 @@ fn libei_click_with(libei::move_absolute(px as f64, py as f64)?;for i in 0..count.max(1) {if i > 0 {- std::thread::sleep(std::time::Duration::from_millis(80));+ crate::interaction::pause(std::time::Duration::from_millis(80));}libei::click(px as f64, py as f64, btn)?;}@@ -2670,7 +3183,7 @@ fn libei_scroll(direction: &str, amount: u32) -> anyhow::Result<()> {};for i in 0..amount.max(1) {if i > 0 {- std::thread::sleep(std::time::Duration::from_millis(25));+ crate::interaction::pause(std::time::Duration::from_millis(25));}libei::scroll(dx, dy)?;}@@ -3093,33 +3606,40 @@ fn inject_exchange(lines: &[String]) -> anyhow::Result<Vec<String>> {use std::os::unix::net::UnixStream;let path = inject_socket_path().ok_or_else(|| anyhow::anyhow!("CUA_INJECT_SOCKET not set"))?;// The nested compositor may still be starting; retry the connect briefly.- let mut stream = None;- for _ in 0..60 {+ let connect_by = crate::interaction::operation_deadline(std::time::Duration::from_secs(3));+ let stream = loop {match UnixStream::connect(&path) {- Ok(s) => {- stream = Some(s);- break;+ Ok(s) => break s,+ Err(_) if std::time::Instant::now() < connect_by => {+ crate::interaction::pause(std::time::Duration::from_millis(50))}- Err(_) => std::thread::sleep(std::time::Duration::from_millis(50)),+ Err(_) => anyhow::bail!("could not connect to inject socket {path}"),}- }- let stream =- stream.ok_or_else(|| anyhow::anyhow!("could not connect to inject socket {path}"))?;- stream.set_read_timeout(Some(std::time::Duration::from_secs(5)))?;+ };+ let read_by = crate::interaction::operation_deadline(std::time::Duration::from_secs(5));let mut writer = stream.try_clone()?;let mut reader = BufReader::new(stream);+ let read_line = |reader: &mut BufReader<UnixStream>| {+ let remaining = read_by.saturating_duration_since(std::time::Instant::now());+ anyhow::ensure!(+ !remaining.is_zero(),+ "cua-compositor did not respond within the timeout"+ );+ reader.get_ref().set_read_timeout(Some(remaining))?;+ read_inject_line(reader)+ };// v1 handshake: send our banner and require the compositor to echo it.writeln!(writer, "{INJECT_PROTO_HELLO}")?;writer.flush()?;- parse_inject_hello(&read_inject_line(&mut reader)?)?;+ parse_inject_hello(&read_line(&mut reader)?)?;let mut replies = Vec::with_capacity(lines.len());// One command per line; block on its response before the next.for l in lines {writeln!(writer, "{l}")?;writer.flush()?;- replies.push(read_inject_line(&mut reader)?);+ replies.push(read_line(&mut reader)?);}Ok(replies)}@@ -3483,6 +4003,33 @@ fn wayland_atspi_windows(filter_pid: Option<u32>) -> Vec<WindowInfo> {windows}+/// A niri window as a listing entry. Only a placed floating window has+/// geometry; any other window lists as 0x0 at the origin, so no caller can+/// mistake it for a real rectangle.+fn niri_window_info(window: niri::Window) -> WindowInfo {+ let (x, y, width, height) = window.placement.map_or((0, 0, 0, 0), |placed| {+ (+ placed.rect.x,+ placed.rect.y,+ placed.rect.width,+ placed.rect.height,+ )+ });+ WindowInfo {+ xid: window.id,+ pid: window.pid,+ app_name: window.app_id,+ title: window.title,+ is_on_screen: window.on_screen,+ // niri reports no stacking order; focus recency is not one.+ z_index: None,+ x,+ y,+ width,+ height,+ }+}+fn apply_pid_filter(mut windows: Vec<WindowInfo>, filter_pid: Option<u32>) -> Vec<WindowInfo> {if let Some(pid) = filter_pid {windows.retain(|window| window.pid == Some(pid));@@ -3521,6 +4068,21 @@ pub fn list_windows_dispatch(filter_pid: Option<u32>) -> Vec<WindowInfo> {}};}+ if niri::is_session() {+ // niri IPC attests PIDs and its own window ids. Once niri is+ // recognised, an IPC failure must not fall back to the+ // foreign-toplevel title join, which loses that id namespace.+ return match niri::list_windows() {+ Ok(windows) => listed_windows(apply_pid_filter(+ windows.into_iter().map(niri_window_info).collect(),+ filter_pid,+ )),+ Err(error) => {+ tracing::warn!("niri window enumeration unavailable: {error:#}");+ Vec::new()+ }+ };+ }// A verified GNOME helper already supplies exact IDs and ownership.// Do not probe unrelated compositor protocols for every preview frame.if let Some(ws) = shell_helper::list_windows(filter_pid) {@@ -3719,7 +4281,7 @@ pub struct WaylandManagers {pub virtual_pointer: bool,pub wl_shm: bool,/// Staging `ext-image-copy-capture-v1` manager — sway 1.10+, labwc- /// 0.8+, niri, KDE 6.2+, GNOME mutter 47+.+ /// 0.8+, KDE 6.2+, GNOME mutter 47+.pub ext_image_copy_capture: bool,/// Companion `ext-output-image-capture-source-v1` source manager —/// required to capture a `wl_output` via the staging protocol.@@ -3805,7 +4367,9 @@ const _BTN_LEFT_ALIAS: u32 = BTN_LEFT;#[cfg(test)]mod output_transform_tests {- use super::{logical_output_size, rotate_png_for_output_transform};+ use super::{+ logical_output_size, orient_png_for_niri_transform, rotate_png_for_output_transform,+ };#[test]fn logical_output_size_swaps_axes_for_quarter_turns() {@@ -3856,10 +4420,278 @@ mod output_transform_tests {);assert!(rotate_png_for_output_transform(two_pixel_png(), 5).is_err());}++ #[test]+ fn niri_capture_undoes_flipped_output_transforms() {+ // Native frame: red above blue, as a nested niri's Flipped180 winit+ // output delivers an upright window with red at its top.+ let mut img = image::RgbaImage::new(2, 2);+ let (red, blue) = (image::Rgba([255, 0, 0, 255]), image::Rgba([0, 0, 255, 255]));+ img.put_pixel(0, 0, red);+ img.put_pixel(1, 1, blue);+ let mut png = Vec::new();+ image::DynamicImage::ImageRgba8(img)+ .write_to(&mut std::io::Cursor::new(&mut png), image::ImageFormat::Png)+ .unwrap();+ let flipped180 = decode(&orient_png_for_niri_transform(png.clone(), 6).unwrap());+ assert_eq!(+ (*flipped180.get_pixel(0, 1), *flipped180.get_pixel(1, 0)),+ (red, blue)+ );+ let flipped = decode(&orient_png_for_niri_transform(png.clone(), 4).unwrap());+ assert_eq!(+ (*flipped.get_pixel(1, 0), *flipped.get_pixel(0, 1)),+ (red, blue)+ );+ assert_eq!(orient_png_for_niri_transform(png.clone(), 0).unwrap(), png);+ assert_eq!(+ orient_png_for_niri_transform(png, 5)+ .unwrap_err()+ .to_string(),+ "niri output transform 5 is not supported for capture"+ );+ }}#[cfg(test)]mod tests {+ /// A point past the pointer space is clamped onto another pixel; the+ /// niri gate must see that rather than admit the clamped point.+ #[test]+ fn layout_addressed_origin_clicks_stay_at_the_origin() {+ assert!(super::centres_origin_click(0, 0, false));+ assert!(!super::centres_origin_click(0, 0, true));+ assert!(!super::centres_origin_click(0, 1, false));+ }++ #[test]+ fn clamped_points_are_not_reached() {+ // The lane's 1280x720 logical output at scale 1.5.+ assert!(super::pointer_reaches(0, 0, 1280, 720, 100, 719));+ assert!(!super::pointer_reaches(0, 0, 1280, 720, 100, 750));+ assert!(!super::pointer_reaches(0, 0, 1280, 720, -1, 10));+ assert!(super::pointer_reaches(-100, 0, 2773, 1080, -100, 0));+ }++ #[test]+ fn niri_never_recovers_input_through_libei() {+ let pointed: anyhow::Result<()> = super::libei_recovery(+ true,+ || anyhow::bail!("{}: none bound", super::NO_VPTR_MARKER),+ || panic!("libei must not run under niri"),+ );+ assert_eq!(+ pointed.unwrap_err().to_string(),+ format!(+ "foreground_unavailable: {}: none bound; under niri the libei/portal fallback is refused because it is not attested",+ super::NO_VPTR_MARKER+ )+ );+ }++ #[test]+ fn niri_focused_keyboard_input_expects_the_held_foreground_window() {+ assert_eq!(super::niri_expected_window(), None);+ {+ let _held = super::bind_foreground_target(7, 42);+ assert_eq!(super::niri_expected_window(), Some(42));+ }+ assert_eq!(super::niri_expected_window(), None);+ }++ #[test]+ fn a_chord_letter_names_its_key_in_either_case() {+ assert_eq!(super::chord_keysym("A"), "a");+ assert_eq!(super::chord_keysym("enter"), "Return");+ assert_eq!(super::chord_keysym("F5"), "F5");+ }++ #[test]+ fn a_niri_that_stops_answering_ends_the_round_trip_at_its_deadline() {+ // A compositor end that reads nothing and answers nothing.+ let (client, _server) = std::os::unix::net::UnixStream::pair().unwrap();+ let connection = super::Connection::from_socket(client).unwrap();+ let mut queue = connection.new_event_queue::<super::State>();+ let mut state = super::State::default();+ let started = std::time::Instant::now();+ let stalled = super::bounded_roundtrip(+ &connection,+ &mut queue,+ &mut state,+ started + std::time::Duration::from_millis(200),+ )+ .unwrap_err();+ assert_eq!(+ stalled.to_string(),+ "niri did not answer a Wayland round trip in time"+ );+ assert!(started.elapsed() < std::time::Duration::from_secs(1));+ }++ #[test]+ fn a_gesture_stops_at_the_first_refused_event_and_reports_the_partial_effect() {+ // The first press of a double click closes the target window, which+ // uncovers another one under the point: the second press must not+ // reach it, and the error must not read as "nothing was delivered".+ let mut target_open = true;+ let mut delivered = 0;+ let stopped = super::gated_events(+ &mut target_open,+ 2,+ ("click", "presses"),+ |_| Ok(()),+ |open| {+ anyhow::ensure!(*open, "foreground_unavailable: layout changed");+ Ok(())+ },+ |open, _| {+ delivered += 1;+ *open = false;+ Ok(())+ },+ )+ .unwrap_err();+ assert_eq!(delivered, 1);+ assert_eq!(+ stopped.to_string(),+ "click stopped after 1 of 2 presses: foreground_unavailable: layout changed"+ );+ assert_eq!(crate::input::foreground::error_code(&stopped), None);++ // A refusal before the first event still means no effect.+ let mut delivered = 0;+ let refused = super::gated_events(+ &mut (),+ 5,+ ("scroll", "ticks"),+ |_| Ok(()),+ |_| anyhow::bail!("foreground_unavailable: layout changed"),+ |_, _| {+ delivered += 1;+ Ok(())+ },+ )+ .unwrap_err();+ assert_eq!(delivered, 0);+ assert_eq!(+ crate::input::foreground::error_code(&refused),+ Some(crate::input::foreground::CODE_UNAVAILABLE)+ );+ }++ #[test]+ fn a_target_that_closes_while_the_pointer_re_aims_gets_no_further_press() {+ // The first press makes the target close itself a moment later,+ // while the pause and motion before the second press run: the check+ // must see that, not the state from before the gap.+ let mut target_open = true;+ let mut presses = 0;+ let stopped = super::gated_events(+ &mut target_open,+ 3,+ ("click", "presses"),+ |open| {+ *open = false;+ Ok(())+ },+ |open| {+ anyhow::ensure!(*open, "foreground_unavailable: niri window 7 closed");+ Ok(())+ },+ |_, _| {+ presses += 1;+ Ok(())+ },+ )+ .unwrap_err();+ assert_eq!(presses, 1);+ assert_eq!(+ stopped.to_string(),+ "click stopped after 1 of 3 presses: foreground_unavailable: niri window 7 closed"+ );+ }++ #[test]+ fn a_drag_sends_at_most_the_advertised_steps_and_ends_on_its_endpoint() {+ // `steps` comes from the caller, after the button is already down.+ let path = super::drag_path((0, 0), (100, 50), u32::MAX, (100, 50), |x, y| {+ (x as u32, y as u32)+ });+ assert_eq!(path.len(), 201);+ let points: Vec<_> = path.collect();+ assert_eq!(points[99], (50, 25));+ assert_eq!(points.last(), Some(&(100, 50)));+ let path = super::drag_path((0, 0), (10, 0), 0, (10, 0), |x, y| (x as u32, y as u32));+ assert_eq!(path.collect::<Vec<_>>(), [(10, 0), (10, 0)]);+ }++ #[test]+ fn a_drag_between_opposite_coordinate_extremes_aims_every_point() {+ // The path is interpolated after the press, so its arithmetic must+ // not overflow for any pair of points the caller can pass.+ let (min, max) = (i32::MIN, i32::MAX);+ let mut aimed = Vec::new();+ let path = super::drag_path((min, max), (max, min), 2, (0, 0), |x, y| {+ aimed.push((x, y));+ (0, 0)+ });+ assert_eq!(path.count(), 3);+ assert_eq!(aimed, [(-1, -1), (max, min)]);+ }++ #[test]+ fn a_drag_whose_layout_changes_mid_path_releases_and_reports_how_far_it_got() {+ // The press rescales the output: the remaining motions would land+ // elsewhere, so the drag releases the button instead of finishing.+ #[derive(Default)]+ struct Held {+ moved: Vec<(u32, u32)>,+ released: bool,+ }+ let mut held = Held::default();+ let stopped = super::held_motions(+ &mut held,+ [(10, 10), (20, 10), (30, 10)].into_iter(),+ |held| {+ anyhow::ensure!(+ held.moved.len() < 2,+ "foreground_unavailable: the niri pointer space changed"+ );+ Ok(())+ },+ |held, point| {+ held.moved.push(point);+ Ok(())+ },+ |held| {+ held.released = true;+ Ok(())+ },+ )+ .unwrap_err();+ assert_eq!(held.moved, [(10, 10), (20, 10)]);+ assert!(held.released);+ assert_eq!(+ stopped.to_string(),+ "drag stopped after 2 of 3 motions, and the button was released: \+ foreground_unavailable: the niri pointer space changed"+ );+ assert_eq!(crate::input::foreground::error_code(&stopped), None);+ }++ #[test]+ fn an_output_added_while_settling_refuses_the_pointer_input() {+ super::pointer_space_holds((0, 0, 854, 480), (0, 0, 854, 480)).unwrap();+ // The session scaled for one output; a second one doubles the extent,+ // so (350, 80) would land near (700, 80).+ assert_eq!(+ super::pointer_space_holds((0, 0, 854, 480), (0, 0, 1707, 480))+ .unwrap_err()+ .to_string(),+ "foreground_unavailable: the niri pointer space changed from (0, 0, 854, 480) to (0, 0, 1707, 480); take a new screenshot"+ );+ }+/// The pointer always moves before a desktop click or scroll, even when/// it already sits on the target pixel (corners and 1-pixel outputs too).#[test]diff --git a/libs/cua-driver/rust/crates/platform-linux/src/wayland/niri/capture.rs b/libs/cua-driver/rust/crates/platform-linux/src/wayland/niri/capture.rsnew file mode 100644index 0000000000..a69bfdd066--- /dev/null+++ b/libs/cua-driver/rust/crates/platform-linux/src/wayland/niri/capture.rs@@ -0,0 +1,360 @@+//! Window capture: the output region a niri window occupies, and its capture.++use super::*;++// ── Window capture ───────────────────────────────────────────────────────────++/// Where a window is in a single-output display capture, and the output+/// geometry that mapping depends on.+#[derive(Clone, Debug, PartialEq)]+pub struct CapturePlan {+ /// Capture pixels, after the capture is oriented to the logical layout.+ pub pixels: Rect,+ /// The window's logical size, the size of the returned window image.+ pub width: u32,+ pub height: u32,+ /// The output the capture must come from, its logical rectangle and+ /// scale, and the oriented capture's size in pixels.+ pub output: String,+ pub area: Rect,+ pub scale: f64,+ pub transform: u32,+ pub image: (u32, u32),+}++/// An oriented display capture, the output it copied and the transform the+/// buffer was oriented with.+pub struct DisplayCapture {+ pub png: Vec<u8>,+ pub output: String,+ pub transform: u32,+ pub size: (u32, u32),+}++pub(super) fn capture_plan_in(+ scene: &Scene,+ id: u64,+ pid: Option<u32>,+) -> Result<CapturePlan, &'static str> {+ let window = scene+ .window(id)+ .map_err(|_| "niri reports no such window; refresh list_windows")?;+ if pid.is_some_and(|pid| window.pid != Some(pid)) {+ return Err("niri reports the window belongs to a different process");+ }+ let placed = window+ .placement+ .as_ref()+ .map_err(|unplaced| unplaced.reason())?;+ let (output, logical) = scene+ .single_output()+ .map_err(|_| "niri window capture supports a single output, and this layout has several")?;+ if placed.output != output.name {+ return Err("the window is not on the captured output");+ }+ if !placed.rect.inside(&placed.area) {+ return Err("the window extends past its output's edge");+ }+ if placed.overlapped() {+ return Err("another floating window overlaps it, and niri reports no stacking order");+ }+ let mode = output+ .current_mode+ .and_then(|index| output.modes.get(index))+ .ok_or("niri reports no current mode for the output")?;+ let transform =+ transform_code(&logical.transform).map_err(|_| "unknown niri output transform")?;+ let scale = logical.scale;+ // Round only in pixel space: the fractional origin is what niri drew.+ let pixels = Rect::from_f64(+ (placed.origin.0 - f64::from(logical.x)) * scale,+ (placed.origin.1 - f64::from(logical.y)) * scale,+ f64::from(placed.rect.width) * scale,+ f64::from(placed.rect.height) * scale,+ )+ .ok_or("invalid niri window geometry")?;+ Ok(CapturePlan {+ pixels,+ width: placed.rect.width,+ height: placed.rect.height,+ output: output.name.clone(),+ area: placed.area,+ scale,+ transform,+ image: super::super::logical_output_size((mode.width, mode.height), transform),+ })+}++/// Capture one floating window by cropping a display capture. The window+/// and its output must read the same before and after the capture; a move+/// away and back between the two reads is not seen. Errors are refusal reasons for+/// `surface_identity_unproven`.+pub fn capture_window(+ id: u64,+ pid: Option<u32>,+ capture_display: impl FnOnce() -> Result<DisplayCapture>,+) -> Result<(Vec<u8>, CapturePlan), String> {+ let mut ipc = ipc();+ capture_window_in(+ |deadline| ipc.scene_until(deadline),+ id,+ pid,+ capture_display,+ )+}++fn capture_window_in(+ mut scene: impl FnMut(Instant) -> Result<Scene>,+ id: u64,+ pid: Option<u32>,+ capture_display: impl FnOnce() -> Result<DisplayCapture>,+) -> Result<(Vec<u8>, CapturePlan), String> {+ let mut plan_now = |deadline| -> Result<CapturePlan, String> {+ Ok(capture_plan_in(+ &scene(deadline).map_err(|e| format!("{e:#}"))?,+ id,+ pid,+ )?)+ };+ let plan = settled(&mut plan_now, CapturePlan::clone, || {+ "the niri window or its output is still changing".to_owned()+ })?;+ let capture = capture_display().map_err(|e| format!("{e:#}"))?;+ let now = plan_now(interaction::operation_deadline(QUERY_TIMEOUT))?;+ if !interaction::policy::fresh(&plan, &interaction::facts::Fact::Known(now)) {+ return Err("the niri window or its output changed during capture".into());+ }+ if capture.output != plan.output {+ return Err(format!(+ "the display capture copied output {:?}, not the window's output {:?}",+ capture.output, plan.output+ ));+ }+ if capture.transform != plan.transform {+ return Err(format!(+ "the capture was oriented for output transform {}, but the window's output has {}",+ capture.transform, plan.transform+ ));+ }+ if capture.size != plan.image {+ return Err(format!(+ "the capture is {}x{} pixels, but niri reports {}x{} for output {:?}",+ capture.size.0, capture.size.1, plan.image.0, plan.image.1, plan.output+ ));+ }+ Ok((capture.png, plan))+}++#[cfg(test)]+mod tests {+ use super::*;++ // ── Capture plan ────────────────────────────────────────────────────────++ #[test]+ fn crop_maps_global_logical_geometry_to_output_pixels() {+ let scene = scene(+ vec![floating(7, 1, (100.0, 60.0), (300, 200))],+ vec![workspace(1, "DP-2", true, None)],+ vec![output("DP-2", 1000, 40, (1920, 1080), 1.5, "Normal")],+ false,+ );+ assert_eq!(+ capture_plan_in(&scene, 7, Some(107)),+ Ok(CapturePlan {+ pixels: Rect {+ x: 150,+ y: 90,+ width: 450,+ height: 300+ },+ width: 300,+ height: 200,+ output: "DP-2".into(),+ area: Rect {+ x: 1000,+ y: 40,+ width: 1280,+ height: 720+ },+ scale: 1.5,+ transform: 0,+ image: (1920, 1080),+ })+ );+ }++ #[test]+ fn a_one_physical_pixel_overlap_at_a_fractional_scale_refuses() {+ // At 1.5 the target spans physical x 152..602 and the other tile+ // starts at physical 601. Rounded to logical pixels (101 and 401)+ // they would only touch.+ let scene = one_output(vec![+ floating(7, 1, (152.0 / 1.5, 60.0), (300, 200)),+ floating(8, 1, (601.0 / 1.5, 60.0), (100, 100)),+ ]);+ assert_eq!(+ capture_plan_in(&scene, 7, None),+ Err("another floating window overlaps it, and niri reports no stacking order")+ );+ let placed = placement(&scene, 7).unwrap();+ assert!(placed.hits(400, 100));+ assert!(+ !placed.hits(401, 100),+ "x=401 is also on the tile from 400.667"+ );+ }++ #[test]+ fn crop_keeps_a_pixel_aligned_fractional_origin() {+ // At scale 1.5 niri places a window on physical x=151, which IPC+ // reports as logical 100.667; rounding it first would crop from 152.+ let scene = one_output(vec![floating(7, 1, (151.0 / 1.5, 60.0), (300, 200))]);+ let plan = capture_plan_in(&scene, 7, None).unwrap();+ assert_eq!((plan.pixels.x, plan.pixels.width), (151, 450));+ let placed = placement(&scene, 7).unwrap();+ assert!(!placed.covers(100, 100), "x=100 is left of 100.667");+ assert!(placed.covers(101, 100));+ }++ #[test]+ fn capture_refuses_an_output_change_niri_sends_no_event_for() {+ let window = || floating(7, 1, (100.0, 60.0), (300, 200));+ let mut calls = 0;+ let error = capture_window_in(+ |_| {+ calls += 1;+ // The scale changes during the capture; the layout does not.+ let scale = if calls <= 2 { 1.5 } else { 1.0 };+ Ok(scene(+ vec![window()],+ vec![workspace(1, "winit", true, None)],+ vec![output("winit", 0, 0, (1280, 720), scale, "Normal")],+ false,+ ))+ },+ 7,+ None,+ || display("winit", (1280, 720)),+ )+ .unwrap_err();+ assert_eq!(+ error,+ "the niri window or its output changed during capture"+ );+ }++ #[test]+ fn capture_refuses_a_transform_change_that_keeps_every_size() {+ // one_output's winit output is Flipped180 (6); Normal (0) keeps the+ // name, area, scale and image size.+ let one = |_| Ok(one_output(vec![floating(7, 1, (100.0, 60.0), (300, 200))]));+ assert_eq!(+ capture_window_in(one, 7, None, || oriented("winit", 0, (1280, 720))).unwrap_err(),+ "the capture was oriented for output transform 0, but the window's output has 6"+ );+ let mut calls = 0;+ let error = capture_window_in(+ |_| {+ calls += 1;+ let transform = if calls <= 2 { "Flipped180" } else { "Normal" };+ Ok(scene(+ vec![floating(7, 1, (100.0, 60.0), (300, 200))],+ vec![workspace(1, "winit", true, None)],+ vec![output("winit", 0, 0, (1280, 720), 1.5, transform)],+ false,+ ))+ },+ 7,+ None,+ || display("winit", (1280, 720)),+ )+ .unwrap_err();+ assert_eq!(+ error,+ "the niri window or its output changed during capture"+ );+ }++ #[test]+ fn capture_waits_for_a_settled_window_and_refuses_one_that_keeps_moving() {+ let at = |x: f64| Ok(one_output(vec![floating(7, 1, (x, 60.0), (300, 200))]));+ let mut calls = 0;+ let settling = capture_window_in(+ |_| {+ calls += 1;+ at(if calls == 1 { 90.0 } else { 100.0 })+ },+ 7,+ None,+ || display("winit", (1280, 720)),+ );+ assert_eq!(settling.unwrap().1.area.width, 853);+ let mut x = 0.0;+ let moving = capture_window_in(+ |_| {+ x += 10.0;+ at(x)+ },+ 7,+ None,+ || display("winit", (1280, 720)),+ );+ assert_eq!(+ moving.unwrap_err(),+ "the niri window or its output is still changing"+ );+ }++ #[test]+ fn capture_refuses_another_output_or_size() {+ let one = |_| Ok(one_output(vec![floating(7, 1, (100.0, 60.0), (300, 200))]));+ assert_eq!(+ capture_window_in(one, 7, None, || display("DP-1", (1280, 720))).unwrap_err(),+ "the display capture copied output \"DP-1\", not the window's output \"winit\""+ );+ assert_eq!(+ capture_window_in(one, 7, None, || display("winit", (1920, 1080))).unwrap_err(),+ "the capture is 1920x1080 pixels, but niri reports 1280x720 for output \"winit\""+ );+ assert!(capture_window_in(one, 7, None, || display("winit", (1280, 720))).is_ok());+ }++ #[test]+ fn crop_refuses_other_outputs_overlap_and_other_pids() {+ let two_outputs = scene(+ vec![floating(7, 2, (100.0, 60.0), (300, 200))],+ vec![+ workspace(1, "DP-1", true, None),+ workspace(2, "DP-2", true, None),+ ],+ vec![+ output("DP-1", 0, 0, (1920, 1080), 1.0, "Normal"),+ output("DP-2", 1920, 0, (1920, 1080), 1.0, "Normal"),+ ],+ false,+ );+ assert_eq!(+ capture_plan_in(&two_outputs, 7, None),+ Err("niri window capture supports a single output, and this layout has several")+ );+ let overlapped = one_output(vec![+ floating(7, 1, (100.0, 60.0), (300, 200)),+ floating(8, 1, (390.0, 250.0), (100, 100)),+ ]);+ assert_eq!(+ capture_plan_in(&overlapped, 7, None),+ Err("another floating window overlaps it, and niri reports no stacking order")+ );+ let window = one_output(vec![floating(7, 1, (100.0, 60.0), (300, 200))]);+ assert_eq!(+ capture_plan_in(&window, 7, Some(1)),+ Err("niri reports the window belongs to a different process")+ );+ let past_edge = one_output(vec![floating(7, 1, (700.0, 60.0), (300, 200))]);+ assert_eq!(+ capture_plan_in(&past_edge, 7, None),+ Err("the window extends past its output's edge")+ );+ }+}diff --git a/libs/cua-driver/rust/crates/platform-linux/src/wayland/niri/focus.rs b/libs/cua-driver/rust/crates/platform-linux/src/wayland/niri/focus.rsnew file mode 100644index 0000000000..df6b775b41--- /dev/null+++ b/libs/cua-driver/rust/crates/platform-linux/src/wayland/niri/focus.rs@@ -0,0 +1,807 @@+//! Keyboard focus on niri: focusing, confirming and restoring windows, and+//! the save, focus and restore steps of the shared foreground transaction.++use super::*;++/// How long niri may take to apply and report a focus change.+pub(crate) const FOCUS_CONFIRM_TIMEOUT: Duration = Duration::from_millis(500);++// ── Focus ────────────────────────────────────────────────────────────────────++pub(super) fn focus_window_action(id: u64) -> serde_json::Value {+ serde_json::json!({ "FocusWindow": { "id": id } })+}++pub(super) fn foreground_unavailable(detail: impl std::fmt::Display) -> anyhow::Error {+ anyhow!("foreground_unavailable: {detail}")+}++/// The refusal for desktop-frame coordinates addressed to window `id`.+/// Mapping them through the window's current place would aim at the same+/// desktop pixel even after the window moved under it, and niri does not+/// say whether it did since the `get_desktop_state` screenshot.+pub fn desktop_frame_window_refusal(id: u64) -> anyhow::Error {+ foreground_unavailable(format!(+ "niri cannot map desktop-frame coordinates into window {id}, because it does not report whether the window moved since the get_desktop_state screenshot; act at desktop scope without pid and window_id, or pass window-local coordinates"+ ))+}++/// The next step for a niri input refusal: its detail names the remedy.+/// `None` for refusals that do not come from niri, which keep the generic+/// suggestion.+pub fn refusal_suggestion(detail: &str) -> Option<&'static str> {+ if detail.contains("without pid and window_id") {+ return Some("Drop pid and window_id to act at desktop scope on the latest get_desktop_state screenshot, or pass window-local coordinates from get_window_state.");+ }+ detail.contains("niri").then_some(+ "Take a new screenshot (get_desktop_state for desktop-scope points, get_window_state for a floating window) after the remedy the detail names, and retry with its coordinates.",+ )+}++/// An IPC failure on an input route, with its whole cause chain. A refusal+/// that already carries the code is kept as it is.+pub(crate) fn input_unavailable(error: anyhow::Error) -> anyhow::Error {+ if crate::input::foreground::error_code(&error).is_some() {+ return error;+ }+ foreground_unavailable(format!("{error:#}"))+}++/// Check that `id` can take focus for `pid`. Returns the window when it is+/// not focused yet, `None` when it already is.+fn focus_target<C: FnMut(Instant) -> Result<UnixStream>>(+ ipc: &mut Ipc<C>,+ id: u64,+ pid: Option<u32>,+) -> Result<Option<IpcWindow>> {+ let overview: Overview = ipc.query(Request::OverviewState)?;+ if overview.is_open {+ return Err(foreground_unavailable(+ "the niri overview is open, so input would go to the overview instead of the window",+ ));+ }+ let windows: Vec<IpcWindow> = ipc.query(Request::Windows)?;+ let target = windows+ .into_iter()+ .find(|window| window.id == id)+ .ok_or_else(|| foreground_unavailable(format!("niri window {id} no longer exists")))?;+ let target_pid = target.pid.and_then(|pid| u32::try_from(pid).ok());+ if let Some(pid) = pid {+ interaction::policy::identity(+ &(Some(pid), id),+ &interaction::facts::Fact::Known(Some((target_pid, target.id))),+ )+ .map_err(|_| {+ foreground_unavailable(format!(+ "niri window {id} belongs to pid {}, not pid {pid}",+ target_pid.map_or_else(|| "unknown".to_owned(), |pid| pid.to_string())+ ))+ })?;+ }+ Ok((!target.is_focused).then_some(target))+}++/// Focus exactly `id` and read the focus back. niri answers `Handled` even+/// for an unknown id, and reports no focused window while the overview, a+/// layer-shell surface or the lock screen holds focus, so the read-back is+/// the guarantee.+pub(super) fn activate_in<C: FnMut(Instant) -> Result<UnixStream>>(+ ipc: &mut Ipc<C>,+ id: u64,+ pid: Option<u32>,+ confirm: Duration,+) -> Result<()> {+ match focus_target(ipc, id, pid)? {+ None => Ok(()),+ Some(target) => focus_window(ipc, id, target.pid, confirm),+ }+}++fn focus_window<C: FnMut(Instant) -> Result<UnixStream>>(+ ipc: &mut Ipc<C>,+ id: u64,+ pid: Option<i32>,+ confirm: Duration,+) -> Result<()> {+ ipc.action(focus_window_action(id))?;+ confirm_focus(ipc, id, pid, interaction::operation_deadline(confirm))+}++pub(super) fn confirm_focus<C: FnMut(Instant) -> Result<UnixStream>>(+ ipc: &mut Ipc<C>,+ id: u64,+ pid: Option<i32>,+ deadline: Instant,+) -> Result<()> {+ let mut focused: Option<IpcWindow> = None;+ if interaction::confirm(deadline, |deadline| {+ focused = ipc.query_until(Request::FocusedWindow, deadline)?;+ anyhow::Ok(interaction::policy::focused(+ &(pid, id),+ &interaction::facts::Fact::Known(+ focused.as_ref().map(|window| (window.pid, window.id)),+ ),+ ))+ })? {+ return Ok(());+ }+ Err(foreground_unavailable(format!(+ "niri did not confirm focus on window {id} (focused: {})",+ focused.map_or_else(|| "none".to_owned(), |focused| focused.id.to_string())+ )))+}++/// Focus an exact niri window for input, confirmed by read-back.+pub fn activate(id: u64, pid: Option<u32>) -> Result<()> {+ activate_in(&mut ipc(), id, pid, FOCUS_CONFIRM_TIMEOUT).map_err(input_unavailable)+}++/// The window niri sends keyboard input to, or `None` when no window has it.+/// niri marks a window focused only while its layout holds keyboard focus,+/// so the lock screen, the overview, the exit dialog, the screenshot UI and+/// a focused layer-shell surface all read as `None`.+pub fn keyboard_focus() -> Result<Option<u64>> {+ Ok(ipc().focused_window()?.map(|window| window.id))+}++/// The index of niri's active keyboard layout: the xkb group keys go out in.+pub fn keyboard_layout() -> Result<u32> {+ let layouts: KeyboardLayouts = ipc().query(Request::KeyboardLayouts)?;+ if usize::try_from(layouts.current_idx).map_or(true, |idx| idx >= layouts.names.len()) {+ bail!(+ "niri's active keyboard layout {} is not one of its {} layouts",+ layouts.current_idx,+ layouts.names.len()+ );+ }+ Ok(layouts.current_idx)+}++/// What focusing a niri window changes and its restore gives back: the+/// previously focused window, or the previously focused workspace when no+/// window had focus, and each output's previously active workspace.+/// Focusing a window on another workspace switches to it; the restore+/// switches back, also after a focus niri did not confirm.+pub(crate) struct SavedFocus {+ prior: Option<IpcWindow>,+ workspaces: Vec<IpcWorkspace>,+ target: IpcWindow,+}++/// Save the focus `id` would take for `pid`, or `None` when it already has+/// it.+pub(crate) fn save_focus<C: FnMut(Instant) -> Result<UnixStream>>(+ ipc: &mut Ipc<C>,+ pid: u32,+ id: u64,+) -> Result<Option<SavedFocus>> {+ let prior = ipc.focused_window().map_err(input_unavailable)?;+ let workspaces: Vec<IpcWorkspace> =+ ipc.query(Request::Workspaces).map_err(input_unavailable)?;+ Ok(focus_target(ipc, id, Some(pid))+ .map_err(input_unavailable)?+ .map(|target| SavedFocus {+ prior,+ workspaces,+ target,+ }))+}++/// Focus the saved target and read the focus back.+pub(crate) fn focus_saved<C: FnMut(Instant) -> Result<UnixStream>>(+ ipc: &mut Ipc<C>,+ saved: &SavedFocus,+ confirm: Duration,+) -> Result<()> {+ focus_window(ipc, saved.target.id, saved.target.pid, confirm).map_err(input_unavailable)+}++/// Give back what [`focus_saved`] changed, within the `cleanup` deadline.+pub(crate) fn restore_saved<C: FnMut(Instant) -> Result<UnixStream>>(+ ipc: &mut Ipc<C>,+ saved: SavedFocus,+ cleanup: Instant,+) -> Result<()> {+ restore_focus(+ ipc,+ saved.prior.as_ref(),+ &saved.workspaces,+ &saved.target,+ cleanup,+ )+}++/// Undo what focusing `target` changed: the active window of its workspace,+/// the active workspace of every output, then the focused window or+/// workspace.+fn restore_focus<C: FnMut(Instant) -> Result<UnixStream>>(+ ipc: &mut Ipc<C>,+ prior: Option<&IpcWindow>,+ before: &[IpcWorkspace],+ target: &IpcWindow,+ cleanup: Instant,+) -> Result<()> {+ let prior_id = prior.map(|prior| prior.id);+ let focused = ipc+ .query_until::<Option<IpcWindow>>(Request::FocusedWindow, cleanup)?+ .map(|focused| focused.id);+ let workspaces: Vec<IpcWorkspace> = ipc.query_until(Request::Workspaces, cleanup)?;+ // Nothing moved, as when the call ran out of time before it focused the+ // target: there is nothing to give back.+ if focused == prior_id && workspaces == before {+ return Ok(());+ }+ // Someone focused another window meanwhile: theirs is the newer choice.+ if let Some(chosen) = interaction::policy::newer_choice(focused, Some(target.id), prior_id) {+ tracing::info!(+ window = chosen,+ "niri focus moved to another window during input; leaving it there"+ );+ return Ok(());+ }+ // The same for a workspace, which may hold no window to focus.+ let prior_workspace = before+ .iter()+ .find(|workspace| workspace.is_focused)+ .map(|workspace| workspace.id);+ if let Some(chosen) = interaction::policy::newer_choice(+ workspaces+ .into_iter()+ .find(|workspace| workspace.is_focused)+ .map(|workspace| workspace.id),+ target.workspace_id,+ prior_workspace,+ ) {+ tracing::info!(+ workspace = chosen,+ "niri focus moved to another workspace during input; leaving it there"+ );+ return Ok(());+ }+ if let Some(active) = before+ .iter()+ .find(|workspace| Some(workspace.id) == target.workspace_id)+ .and_then(|workspace| workspace.active_window_id)+ .filter(|active| *active != target.id && Some(*active) != prior_id)+ {+ ipc.restore_action(focus_window_action(active), cleanup)+ .with_context(|| format!("niri could not restore active window {active}"))?;+ }+ let now: Vec<IpcWorkspace> = ipc.query_until(Request::Workspaces, cleanup)?;+ for was in before+ .iter()+ .filter(|was| was.is_active && !was.is_focused)+ .filter(|was| now.iter().any(|w| w.id == was.id && !w.is_active))+ {+ ipc.restore_action(focus_workspace_action(was.id), cleanup)+ .with_context(|| format!("niri could not restore workspace {}", was.id))?;+ }+ match (prior, before.iter().find(|workspace| workspace.is_focused)) {+ (Some(prior), _) => ipc+ .restore_action(focus_window_action(prior.id), cleanup)+ .and_then(|()| confirm_focus(ipc, prior.id, prior.pid, cleanup))+ .with_context(|| format!("niri could not restore prior window {}", prior.id))?,+ (None, Some(workspace)) => restore_workspace(ipc, workspace.id, cleanup)+ .with_context(|| format!("niri could not restore prior workspace {}", workspace.id))?,+ (None, None) => {}+ }+ // niri applies workspace switches asynchronously: poll like the+ // single-workspace restore does.+ let mut pending = None;+ if interaction::confirm(cleanup, |deadline| {+ let now: Vec<IpcWorkspace> = ipc.query_until(Request::Workspaces, deadline)?;+ pending = before+ .iter()+ .filter(|was| was.is_active)+ .find(|was| now.iter().any(|w| w.id == was.id && !w.is_active));+ anyhow::Ok(pending.is_none())+ })? {+ return Ok(());+ }+ match pending {+ Some(was) => bail!(+ "niri did not restore workspace {} on output {}",+ was.id,+ was.output.as_deref().unwrap_or("unknown")+ ),+ None => bail!("niri did not confirm the workspace restore within the cleanup budget"),+ }+}++fn focus_workspace_action(id: u64) -> serde_json::Value {+ serde_json::json!({ "FocusWorkspace": { "reference": { "Id": id } } })+}++fn restore_workspace<C: FnMut(Instant) -> Result<UnixStream>>(+ ipc: &mut Ipc<C>,+ workspace: u64,+ cleanup: Instant,+) -> Result<()> {+ ipc.restore_action(focus_workspace_action(workspace), cleanup)?;+ let focused = interaction::confirm(cleanup, |deadline| {+ let workspaces: Vec<IpcWorkspace> = ipc.query_until(Request::Workspaces, deadline)?;+ anyhow::Ok(+ workspaces+ .iter()+ .any(|candidate| candidate.id == workspace && candidate.is_focused),+ )+ })?;+ if !focused {+ bail!("niri did not confirm focus on workspace {workspace}");+ }+ Ok(())+}++#[cfg(test)]+mod tests {+ use super::*;+ use serde_json::{json, Value};+ use std::sync::{Arc, Mutex};++ fn with_focused_window_in<C: FnMut(Instant) -> Result<UnixStream>, T>(+ ipc: &mut Ipc<C>,+ pid: u32,+ id: u64,+ confirm: Duration,+ body: impl FnOnce() -> Result<T>,+ ) -> Result<T> {+ interaction::transaction(+ ipc,+ |ipc| save_focus(ipc, pid, id),+ |ipc, saved| focus_saved(ipc, saved, confirm),+ body,+ restore_saved,+ )+ }++ #[test]+ fn a_niri_refusal_suggests_the_remedy_its_detail_names() {+ let tiled = aim_at(&tiled_scene(Some(5)).window(5).unwrap(), 10, 20).unwrap_err();+ assert!(tiled.to_string().contains("float the window"));+ assert!(refusal_suggestion(&tiled.to_string())+ .unwrap()+ .starts_with("Take a new screenshot"));+ let edge = desktop_admission(&tiled_scene(Some(5)), &[(10, 100).into()]).unwrap_err();+ assert!(refusal_suggestion(&edge.to_string()).is_some());+ // A refusal that is not niri's keeps the generic suggestion.+ assert_eq!(+ refusal_suggestion("foreground_unavailable: no window"),+ None+ );+ }++ #[test]+ fn focus_transaction_confirms_runs_and_restores() {+ let fake = focus_fixture();+ let seen = with_focused_window_in(+ &mut fake_ipc(&fake),+ 107,+ 7,+ Duration::from_millis(100),+ || Ok(fake.lock().unwrap().focused()),+ )+ .unwrap();+ assert_eq!(seen, Some(7));+ assert_eq!(fake.lock().unwrap().focused(), Some(5));+ }++ #[test]+ fn unconfirmed_focus_refuses_before_the_body() {+ let fake = focus_fixture();+ fake.lock().unwrap().ignore_focus = true;+ let mut ran = false;+ let error = with_focused_window_in(+ &mut fake_ipc(&fake),+ 107,+ 7,+ Duration::from_millis(50),+ || {+ ran = true;+ Ok(())+ },+ )+ .unwrap_err();+ assert!(!ran);+ assert_eq!(+ error.to_string(),+ "foreground_unavailable: niri did not confirm focus on window 7 (focused: 5)"+ );+ }++ #[test]+ fn a_lost_niri_refuses_focus_bound_input_with_the_typed_code() {+ let mut ran = false;+ let error = with_focused_window_in(+ &mut ipc_with(+ |_| bail!("niri was recognised as pid 20, but the compositor is now pid 40"),+ Duration::from_millis(50),+ ),+ 107,+ 7,+ Duration::from_millis(50),+ || {+ ran = true;+ Ok(())+ },+ )+ .unwrap_err();+ assert!(!ran);+ assert_eq!(+ error.to_string(),+ "foreground_unavailable: niri was recognised as pid 20, but the compositor is now pid 40"+ );+ }++ #[test]+ fn another_pid_refuses_without_focusing() {+ let fake = focus_fixture();+ let error = with_focused_window_in(+ &mut fake_ipc(&fake),+ 999,+ 7,+ Duration::from_millis(50),+ || Ok(()),+ )+ .unwrap_err();+ assert_eq!(+ error.to_string(),+ "foreground_unavailable: niri window 7 belongs to pid 107, not pid 999"+ );+ assert!(!fake+ .lock()+ .unwrap()+ .requests+ .iter()+ .any(|r| r.contains("FocusWindow")));+ }++ #[test]+ fn null_pid_window_refuses_a_pid_bound_transaction() {+ let fake = focus_fixture();+ fake.lock().unwrap().windows[1]["pid"] = Value::Null;+ let error = with_focused_window_in(+ &mut fake_ipc(&fake),+ 107,+ 7,+ Duration::from_millis(50),+ || Ok(()),+ )+ .unwrap_err();+ assert_eq!(+ error.to_string(),+ "foreground_unavailable: niri window 7 belongs to pid unknown, not pid 107"+ );+ }++ #[test]+ fn open_overview_refuses_focus_bound_input() {+ let fake = focus_fixture();+ fake.lock().unwrap().overview = true;+ let error = with_focused_window_in(+ &mut fake_ipc(&fake),+ 107,+ 7,+ Duration::from_millis(50),+ || Ok(()),+ )+ .unwrap_err();+ assert_eq!(+ error.to_string(),+ "foreground_unavailable: the niri overview is open, so input would go to the overview instead of the window"+ );+ }++ #[test]+ fn a_call_that_ran_out_of_time_moves_no_niri_focus() {+ let fake = focus_fixture();+ let action = crate::input::action::Action::new(Duration::from_secs(60));+ action.abandon();+ let _bound = crate::input::action::bind(action);+ let mut ran = false;+ let error = with_focused_window_in(+ &mut fake_ipc(&fake),+ 107,+ 7,+ Duration::from_millis(50),+ || {+ ran = true;+ Ok(())+ },+ )+ .unwrap_err();+ assert!(!ran);+ assert_eq!(+ crate::input::foreground::error_code(&error),+ Some(crate::input::foreground::CODE_TIMEOUT)+ );+ let fake = fake.lock().unwrap();+ assert!(+ !fake+ .requests+ .iter()+ .any(|request| request.contains("Action")),+ "a timed-out call sent {:?}",+ fake.requests+ );+ assert_eq!(fake.focused(), Some(5));+ }++ #[test]+ fn a_call_that_ran_out_of_time_after_focusing_its_target_still_gives_focus_back() {+ let timed_out = || {+ anyhow!(+ "{}: the call ran out of time, so no further input was sent",+ crate::input::foreground::CODE_TIMEOUT+ )+ };+ let fake = focus_fixture();+ let action = crate::input::action::Action::new(Duration::from_secs(60));+ let _bound = crate::input::action::bind(action.clone());+ let error = with_focused_window_in(+ &mut fake_ipc(&fake),+ 107,+ 7,+ Duration::from_millis(100),+ || {+ action.abandon();+ Err::<(), _>(timed_out())+ },+ )+ .unwrap_err();+ assert_eq!(+ crate::input::foreground::error_code(&error),+ Some(crate::input::foreground::CODE_TIMEOUT)+ );+ assert_eq!(fake.lock().unwrap().focused(), Some(5));+ }++ #[test]+ fn a_late_restore_does_not_take_focus_from_the_next_call_on_its_target() {+ // Call 1 focused window 7 and is still running (a worker past its+ // timeout, about to give focus back). Call 2, a retry on window 7,+ // must not find 7 focused by call 1 and then lose it to call 1's+ // restore while it types.+ let fake = focus_fixture();+ let (in_body, call_1_in_body) = std::sync::mpsc::channel();+ let (release, released) = std::sync::mpsc::channel::<()>();+ let (finished, call_1_finished) = std::sync::mpsc::channel();+ let call_1 = {+ let fake = fake.clone();+ std::thread::spawn(move || {+ with_focused_window_in(+ &mut fake_ipc(&fake),+ 107,+ 7,+ Duration::from_millis(100),+ || {+ in_body.send(()).unwrap();+ released.recv().unwrap();+ Ok(())+ },+ )+ .unwrap();+ finished.send(()).unwrap();+ })+ };+ call_1_in_body.recv().unwrap();+ let call_2 = {+ let fake = fake.clone();+ std::thread::spawn(move || {+ with_focused_window_in(+ &mut fake_ipc(&fake),+ 107,+ 7,+ Duration::from_millis(100),+ || {+ // Typing outlasts call 1's restore.+ let _ = call_1_finished.recv_timeout(Duration::from_secs(2));+ Ok(fake.lock().unwrap().focused())+ },+ )+ .unwrap()+ })+ };+ std::thread::sleep(Duration::from_millis(200));+ release.send(()).unwrap();+ call_1.join().unwrap();+ assert_eq!(call_2.join().unwrap(), Some(7));+ assert_eq!(fake.lock().unwrap().focused(), Some(5));+ }++ #[test]+ fn failed_restore_is_reported_after_the_body_ran() {+ let fake = focus_fixture();+ fake.lock().unwrap().ignore_focus_on = vec![5];+ let mut ran = false;+ let error = with_focused_window_in(+ &mut fake_ipc(&fake),+ 107,+ 7,+ Duration::from_millis(50),+ || {+ ran = true;+ Ok(())+ },+ )+ .unwrap_err();+ assert!(ran);+ assert_eq!(+ format!("{error:#}"),+ "the input was sent, but the prior focus could not be restored: niri could not restore prior window 5: foreground_unavailable: niri did not confirm focus on window 5 (focused: 7)"+ );+ assert_eq!(+ format!("{:#}", error.root_cause()),+ "foreground_unavailable: niri did not confirm focus on window 5 (focused: 7)"+ );+ }++ #[test]+ fn body_error_keeps_the_restore_error_as_context() {+ let fake = focus_fixture();+ fake.lock().unwrap().ignore_focus_on = vec![5];+ let error = with_focused_window_in(+ &mut fake_ipc(&fake),+ 107,+ 7,+ Duration::from_millis(50),+ || Err::<(), _>(anyhow!("wtype failed")),+ )+ .unwrap_err();+ assert!(error.to_string().starts_with(+ "wtype failed; the prior focus also could not be restored: niri could not restore prior window 5"+ ));+ assert_eq!(error.root_cause().to_string(), "wtype failed");+ }++ #[test]+ fn focus_niri_did_not_confirm_is_still_undone() {+ let fake = focus_fixture();+ fake.lock().unwrap().focus_taken_on = vec![7];+ let error = with_focused_window_in(+ &mut fake_ipc(&fake),+ 107,+ 7,+ Duration::from_millis(50),+ || Ok(()),+ )+ .unwrap_err();+ assert_eq!(+ error.to_string(),+ "foreground_unavailable: niri did not confirm focus on window 7 (focused: none)"+ );+ let fake = fake.lock().unwrap();+ assert!(fake.active(1) && !fake.active(2));+ assert_eq!(fake.focused(), Some(5));+ }++ #[test]+ fn a_window_focused_by_someone_else_during_input_keeps_focus() {+ let fake = focus_fixture();+ fake.lock()+ .unwrap()+ .windows+ .push(floating(9, 1, (500.0, 60.0), (100, 100)));+ with_focused_window_in(+ &mut fake_ipc(&fake),+ 107,+ 7,+ Duration::from_millis(100),+ || {+ fake.lock()+ .unwrap()+ .reply(r#"{"Action":{"FocusWindow":{"id":9}}}"#);+ Ok(())+ },+ )+ .unwrap();+ let fake = fake.lock().unwrap();+ assert_eq!(fake.focused(), Some(9));+ assert!(!fake+ .requests+ .iter()+ .any(|r| r.contains(r#""FocusWindow":{"id":5}"#)));+ }++ #[test]+ fn an_empty_workspace_chosen_during_input_keeps_focus() {+ let fake = focus_fixture();+ fake.lock()+ .unwrap()+ .workspaces+ .push(workspace(3, "winit", false, None));+ with_focused_window_in(+ &mut fake_ipc(&fake),+ 107,+ 7,+ Duration::from_millis(100),+ || {+ fake.lock()+ .unwrap()+ .reply(r#"{"Action":{"FocusWorkspace":{"reference":{"Id":3}}}}"#);+ Ok(())+ },+ )+ .unwrap();+ let fake = fake.lock().unwrap();+ assert!(fake.active(3) && !fake.active(1));+ assert_eq!(fake.focused(), None);+ }++ #[test]+ fn every_output_gets_its_active_workspace_back() {+ let mut prior = tiled(5, 1, (402.0, 448.0));+ prior["is_focused"] = json!(true);+ let fake = Arc::new(Mutex::new(Fake {+ windows: vec![prior, floating(7, 4, (100.0, 60.0), (300, 200))],+ workspaces: vec![+ workspace(1, "left", true, Some(5)),+ workspace(3, "right", true, None),+ workspace(4, "right", false, Some(7)),+ ],+ ..Fake::default()+ }));+ {+ let mut fake = fake.lock().unwrap();+ fake.workspaces[1]["is_focused"] = json!(false);+ fake.workspace_lag = 2;+ }+ let seen = with_focused_window_in(+ &mut fake_ipc(&fake),+ 107,+ 7,+ Duration::from_millis(100),+ || {+ let fake = fake.lock().unwrap();+ Ok((fake.focused(), fake.active(4)))+ },+ )+ .unwrap();+ assert_eq!(seen, (Some(7), true));+ let fake = fake.lock().unwrap();+ assert!(fake.active(1) && fake.active(3) && !fake.active(4));+ assert_eq!(fake.focused(), Some(5));+ }++ #[test]+ fn no_prior_window_restores_the_prior_workspace() {+ let fake = focus_fixture();+ {+ let mut fake = fake.lock().unwrap();+ fake.windows[0]["is_focused"] = json!(false);+ fake.workspaces.push(workspace(3, "winit", false, None));+ for ws in &mut fake.workspaces {+ ws["is_focused"] = json!(ws["id"] == 3);+ ws["is_active"] = json!(ws["id"] == 3);+ }+ }+ with_focused_window_in(+ &mut fake_ipc(&fake),+ 107,+ 7,+ Duration::from_millis(100),+ || Ok(()),+ )+ .unwrap();+ let fake = fake.lock().unwrap();+ assert!(fake+ .requests+ .iter()+ .any(|r| r.contains(r#""FocusWorkspace":{"reference":{"Id":3}}"#)));+ assert!(fake+ .workspaces+ .iter()+ .any(|ws| ws["id"] == 3 && ws["is_focused"] == true));+ }++ #[test]+ fn an_unknown_window_refuses_instead_of_trusting_handled() {+ // niri answers FocusWindow on an unknown id with Handled.+ let fake = focus_fixture();+ let error =+ activate_in(&mut fake_ipc(&fake), 999, None, Duration::from_millis(50)).unwrap_err();+ assert_eq!(+ error.to_string(),+ "foreground_unavailable: niri window 999 no longer exists"+ );+ }+}diff --git a/libs/cua-driver/rust/crates/platform-linux/src/wayland/niri/gate.rs b/libs/cua-driver/rust/crates/platform-linux/src/wayland/niri/gate.rsnew file mode 100644index 0000000000..0d92e9a0ff--- /dev/null+++ b/libs/cua-driver/rust/crates/platform-linux/src/wayland/niri/gate.rs@@ -0,0 +1,472 @@+//! The pointer gate that checks clicks and drags against niri's current+//! layout. [`prepare_target`] is the one entry point for pointer input;+//! [`PointerGate::holds`] rechecks before every later event.++use super::*;+use crate::interaction::policy::{self, Miss, Point};++/// A layout point for pointer input and, when it was translated from window+/// coordinates on niri, the window rectangle that translation used. The+/// pointer gate refuses the point if the window no longer has that rectangle.+#[derive(Clone, Copy, Debug, PartialEq, Eq)]+pub struct AimedPoint {+ pub point: (i32, i32),+ pub rect: Option<Rect>,+}++impl From<(i32, i32)> for AimedPoint {+ fn from(point: (i32, i32)) -> Self {+ Self { point, rect: None }+ }+}++fn placed_or_refuse(window: &Window) -> Result<&Placed> {+ window.placement.as_ref().map_err(|unplaced| {+ foreground_unavailable(format!("niri window {}: {}", window.id, unplaced.reason()))+ })+}++/// The refusal for window-coordinate input on `window_id` when niri shows it+/// tiled, so a caller without a screenshot of it learns why it has none.+pub fn tiled_refusal(window_id: u64) -> Option<anyhow::Error> {+ let scene = ipc().scene().ok()?;+ let window = scene.window(window_id).ok()?;+ match &window.placement {+ Err(Unplaced::Tiled) => placed_or_refuse(&window).err(),+ _ => None,+ }+}++/// Translate window coordinates into the global layout. A floating window's+/// coordinates are window-local; the rectangle used travels with the point so+/// the pointer gate can prove it still holds. A window without a place (a+/// tiled one) refuses instead of passing `(x, y)` through as output+/// coordinates.+pub fn aim(window_id: u64, x: i32, y: i32) -> Result<AimedPoint> {+ let scene = ipc().scene().map_err(input_unavailable)?;+ single_output_pointer(&scene)?;+ let window = scene.window(window_id).map_err(input_unavailable)?;+ aim_at(&window, x, y)+}++/// Window-addressed pointer input needs a single output: with several, the+/// virtual pointer's absolute motion spans their bounding box, and nothing+/// has shown that niri maps it the way the driver computes it.+pub(super) fn single_output_pointer(scene: &Scene) -> Result<()> {+ match scene.laid_out().len() {+ 0 | 1 => Ok(()),+ outputs => Err(foreground_unavailable(format!(+ "niri window-coordinate pointer input supports a single output; this layout has {outputs}"+ ))),+ }+}++pub(super) fn aim_at(window: &Window, x: i32, y: i32) -> Result<AimedPoint> {+ let rect = placed_or_refuse(window)?.rect;+ Ok(AimedPoint {+ point: (rect.x.saturating_add(x), rect.y.saturating_add(y)),+ rect: Some(rect),+ })+}++/// A pointer action admitted on the current layout. [`Self::holds`] reads+/// the layout again before every event, so a window that moved, or a pointer+/// space that changed, refuses the next one. Two equal snapshots cannot see a+/// move away and back between them, nor an animation niri does not report.+pub struct PointerGate<C = fn(Instant) -> Result<UnixStream>> {+ ipc: Ipc<C>,+ window: Option<u64>,+ /// The points, each carrying the rectangle it was admitted on.+ points: Vec<AimedPoint>,+ /// The pointer space the action's session scales motions by.+ space: (i32, i32, u32, u32),+}++impl<C: FnMut(Instant) -> Result<UnixStream>> PointerGate<C> {+ /// Recheck the admitted action right before an event that acts on the+ /// target: the first motion, the first press, and each wheel tick. A+ /// window target must also still hold keyboard focus, read back from+ /// niri: a lock or layer-shell surface can take it without moving it.+ pub fn holds(&mut self) -> Result<()> {+ if let Some(id) = self.window {+ let scene = self.ipc.scene().map_err(input_unavailable)?;+ let window = scene.window(id).map_err(input_unavailable)?;+ pointer_admission(&window, &self.points)?;+ let pid = scene+ .windows+ .iter()+ .find(|w| w.id == id)+ .and_then(|w| w.pid);+ confirm_focus(+ &mut self.ipc,+ id,+ pid,+ interaction::operation_deadline(FOCUS_CONFIRM_TIMEOUT),+ )?;+ }+ let current =+ pointer_space_of(&self.ipc.query(Request::Outputs)?).map_err(input_unavailable)?;+ super::super::pointer_space_holds(self.space, current)+ }+}++/// What a niri pointer action addresses.+pub enum Target {+ /// A window by id, focused once the action is admitted.+ Window(u64),+ /// Pixels of the single output, as `get_desktop_state` shows it.+ Desktop,+ /// Nothing to check beyond the session and the pointer space.+ Unaddressed,+}++/// The one entry point for niri pointer input: refuse points the virtual+/// pointer in `space` cannot reach (`motion_absolute`+/// clamps into it, so a moved point lands elsewhere), admit `target` on the+/// current layout, and recheck it right before the first event. Call+/// [`PointerGate::holds`] before every later one.+pub fn prepare_target(+ target: Target,+ space: (i32, i32, u32, u32),+ points: &[AimedPoint],+) -> Result<Option<PointerGate>> {+ let (origin_x, origin_y, width, height) = space;+ if let Some((x, y)) = points+ .iter()+ .map(|aimed| aimed.point)+ .find(|&(x, y)| !super::super::pointer_reaches(origin_x, origin_y, width, height, x, y))+ {+ bail!("foreground_unavailable: point ({x}, {y}) is outside the niri pointer space");+ }+ let mut gate = PointerGate {+ ipc: ipc(),+ window: None,+ points: points.to_vec(),+ space,+ };+ let scene = match target {+ Target::Window(id) => settled_window(|deadline| gate.ipc.scene_until(deadline), id),+ _ => gate.ipc.scene(),+ }+ .map_err(input_unavailable)?;+ match target {+ Target::Window(id) => {+ let rect = gate_pointer(&scene, id, points)?;+ activate_in(&mut gate.ipc, id, None, FOCUS_CONFIRM_TIMEOUT)+ .map_err(input_unavailable)?;+ gate.window = Some(id);+ gate.points = admitted_on(points, rect);+ }+ Target::Desktop => desktop_admission(&scene, &gate.points)?,+ Target::Unaddressed => return Ok(None),+ }+ gate.holds()?;+ Ok(Some(gate))+}++fn settled_window(scene: impl FnMut(Instant) -> Result<Scene>, id: u64) -> Result<Scene> {+ settled(+ scene,+ |scene| scene.window(id).map(|window| window.placement).ok(),+ || {+ foreground_unavailable(format!(+ "niri window {id} is still moving; take a new screenshot"+ ))+ },+ )+}++/// Admit window-addressed pointer input on a floating window: the target+/// must still have the rectangle each point was translated against, and+/// every point must land on it rather than on another floating window. A+/// point given in layout coordinates carries no rectangle to compare. A+/// window without a place (a tiled one) refuses. Returns the rectangle+/// admitted.+fn gate_pointer(scene: &Scene, window_id: u64, points: &[AimedPoint]) -> Result<Rect> {+ single_output_pointer(scene)?;+ let window = scene.window(window_id).map_err(input_unavailable)?;+ pointer_admission(&window, points)?;+ Ok(placed_or_refuse(&window)?.rect)+}++/// `points` as admitted on `rect`: later rechecks compare every point with it.+fn admitted_on(points: &[AimedPoint], rect: Rect) -> Vec<AimedPoint> {+ points+ .iter()+ .map(|aimed| AimedPoint {+ rect: Some(rect),+ ..*aimed+ })+ .collect()+}++fn pointer_admission(window: &Window, points: &[AimedPoint]) -> Result<()> {+ let placed = placed_or_refuse(window)?;+ for &AimedPoint {+ point: (x, y),+ rect,+ } in points+ {+ let moved_from = rect.filter(|expected| {+ !policy::fresh(expected, &interaction::facts::Fact::Known(placed.rect))+ });+ let point = Point {+ fresh: moved_from.is_none(),+ on_window: placed.covers(x, y),+ shown: placed.area.contains(x, y),+ unobscured: placed.hits(x, y),+ };+ let reason = match policy::reach(&point) {+ Ok(()) => continue,+ Err(Miss::Stale) => format!(+ "niri window {} moved from {:?} to {:?} after its coordinates were computed; take a new screenshot",+ window.id,+ moved_from.unwrap_or(placed.rect),+ placed.rect+ ),+ Err(Miss::OffWindow) => format!(+ "point ({x}, {y}) is outside niri window {} at {:?}",+ window.id, placed.rect+ ),+ Err(Miss::Hidden) => format!(+ "point ({x}, {y}) is past the edge of niri output {}, where window {} is not shown",+ placed.output, window.id+ ),+ Err(Miss::Obscured) => format!(+ "point ({x}, {y}) lies under another floating window, and niri reports no stacking order"+ ),+ };+ return Err(foreground_unavailable(reason));+ }+ Ok(())+}++/// Admit desktop-scope points: pixels of the single output, the frame a+/// `get_desktop_state` screenshot shows.+pub(super) fn desktop_admission(scene: &Scene, points: &[AimedPoint]) -> Result<()> {+ let (output, logical) = scene.single_output().map_err(input_unavailable)?;+ let area = Rect {+ x: logical.x,+ y: logical.y,+ width: logical.width,+ height: logical.height,+ };+ match points+ .iter()+ .map(|aimed| aimed.point)+ .find(|&(x, y)| !area.contains(x, y))+ {+ Some((x, y)) => Err(foreground_unavailable(format!(+ "point ({x}, {y}) is past the edge of niri output {}, which the get_desktop_state screenshot shows",+ output.name+ ))),+ None => Ok(()),+ }+}++#[cfg(test)]+mod tests {+ use super::*;+ use serde_json::json;+ use std::sync::{Arc, Mutex};++ fn aimed_on(point: (i32, i32), rect: Rect) -> AimedPoint {+ AimedPoint {+ point,+ rect: Some(rect),+ }+ }++ #[test]+ fn a_window_that_moved_after_admission_refuses_the_next_event() {+ let admitted = one_output(vec![floating(7, 1, (100.0, 60.0), (300, 200))]);+ let rect = gate_pointer(&admitted, 7, &[(150, 100).into()]).unwrap();+ let points = admitted_on(&[(150, 100).into()], rect);+ let moved = one_output(vec![floating(7, 1, (120.0, 60.0), (300, 200))]);+ assert_eq!(+ pointer_admission(&placed_window(&moved, 7), &points)+ .unwrap_err()+ .to_string(),+ "foreground_unavailable: niri window 7 moved from Rect { x: 100, y: 60, width: 300, height: 200 } to Rect { x: 120, y: 60, width: 300, height: 200 } after its coordinates were computed; take a new screenshot"+ );+ let covered = one_output(vec![+ floating(7, 1, (100.0, 60.0), (300, 200)),+ floating(8, 1, (140.0, 90.0), (50, 50)),+ ]);+ assert!(pointer_admission(&placed_window(&covered, 7), &points).is_err());+ }++ #[test]+ fn focus_taken_without_any_move_refuses_the_next_event() {+ let mut target = floating(7, 1, (100.0, 60.0), (300, 200));+ target["is_focused"] = json!(true);+ let fake = Arc::new(Mutex::new(Fake {+ windows: vec![target],+ workspaces: vec![workspace(1, "winit", true, Some(7))],+ outputs: vec![output("winit", 0, 0, (1280, 720), 1.5, "Flipped180")],+ ..Fake::default()+ }));+ let admitted = one_output(fake.lock().unwrap().windows.clone());+ let rect = gate_pointer(&admitted, 7, &[(150, 100).into()]).unwrap();+ let mut gate = PointerGate {+ ipc: fake_ipc(&fake),+ window: Some(7),+ points: admitted_on(&[(150, 100).into()], rect),+ space: admitted.pointer_space().unwrap(),+ };+ gate.holds().unwrap();+ fake.lock().unwrap().windows[0]["is_focused"] = json!(false);+ assert_eq!(+ gate.holds().unwrap_err().to_string(),+ "foreground_unavailable: niri did not confirm focus on window 7 (focused: none)"+ );+ }++ #[test]+ fn window_pointer_input_waits_for_a_settled_window_and_refuses_one_that_keeps_moving() {+ let at = |x: f64| Ok(one_output(vec![floating(7, 1, (x, 60.0), (300, 200))]));+ let mut reads = 0;+ let settling = settled_window(+ |_| {+ reads += 1;+ at(if reads == 1 { 90.0 } else { 100.0 })+ },+ 7,+ );+ assert_eq!(+ placed_window(&settling.unwrap(), 7)+ .placement+ .unwrap()+ .rect+ .x,+ 100+ );+ let mut x = 0.0;+ let moving = settled_window(+ |_| {+ x += 10.0;+ at(x)+ },+ 7,+ );+ assert_eq!(+ moving.err().unwrap().to_string(),+ "foreground_unavailable: niri window 7 is still moving; take a new screenshot"+ );+ }++ // ── Pointer admission ───────────────────────────────────────────────────++ #[test]+ fn window_coordinate_pointer_input_refuses_several_outputs() {+ let two_outputs = scene(+ vec![floating(7, 1, (100.0, 60.0), (300, 200))],+ vec![+ workspace(1, "DP-1", true, None),+ workspace(2, "DP-2", true, None),+ ],+ vec![+ output("DP-1", 0, 0, (1920, 1080), 1.0, "Normal"),+ output("DP-2", 1920, 0, (1920, 1080), 1.0, "Normal"),+ ],+ false,+ );+ assert_eq!(+ single_output_pointer(&two_outputs).unwrap_err().to_string(),+ "foreground_unavailable: niri window-coordinate pointer input supports a single output; this layout has 2"+ );+ let one = one_output(vec![floating(7, 1, (100.0, 60.0), (300, 200))]);+ assert!(single_output_pointer(&one).is_ok());+ }++ #[test]+ fn floating_window_coordinates_add_the_window_origin() {+ let scene = one_output(vec![floating(7, 1, (100.0, 60.0), (300, 200))]);+ let aimed = aim_at(&scene.window(7).unwrap(), 10, 20).unwrap();+ assert_eq!(+ aimed,+ aimed_on(+ (110, 80),+ Rect {+ x: 100,+ y: 60,+ width: 300,+ height: 200+ }+ )+ );+ }++ fn placed_window(scene: &Scene, id: u64) -> Window {+ scene.window(id).unwrap()+ }++ #[test]+ fn pointer_admission_refuses_moved_windows_and_ambiguous_points() {+ let scene = one_output(vec![+ floating(7, 1, (100.0, 60.0), (300, 200)),+ floating(8, 1, (350.0, 200.0), (200, 100)),+ ]);+ let window = placed_window(&scene, 7);+ let rect = window.placement.as_ref().unwrap().rect;+ assert!(pointer_admission(&window, &[(150, 100).into()]).is_ok());+ assert!(pointer_admission(&window, &[aimed_on((150, 100), rect)]).is_ok());+ let earlier = Rect { x: 0, ..rect };+ assert_eq!(+ pointer_admission(&window, &[aimed_on((150, 100), earlier)]).unwrap_err().to_string(),+ "foreground_unavailable: niri window 7 moved from Rect { x: 0, y: 60, width: 300, height: 200 } to Rect { x: 100, y: 60, width: 300, height: 200 } after its coordinates were computed; take a new screenshot"+ );+ assert_eq!(+ pointer_admission(&window, &[(360, 210).into()]).unwrap_err().to_string(),+ "foreground_unavailable: point (360, 210) lies under another floating window, and niri reports no stacking order"+ );+ assert_eq!(+ pointer_admission(&window, &[(150, 100).into(), (5, 5).into()]).unwrap_err().to_string(),+ "foreground_unavailable: point (5, 5) is outside niri window 7 at Rect { x: 100, y: 60, width: 300, height: 200 }"+ );+ // Taller than the 853x480 output: its bottom is not shown.+ let scene = one_output(vec![floating(9, 1, (100.0, 60.0), (300, 500))]);+ let tall = placed_window(&scene, 9);+ assert_eq!(+ pointer_admission(&tall, &[(150, 500).into()]).unwrap_err().to_string(),+ "foreground_unavailable: point (150, 500) is past the edge of niri output winit, where window 9 is not shown"+ );+ let tiled = one_output(vec![tiled(5, 1, (402.0, 448.0))])+ .window(5)+ .unwrap();+ assert_eq!(+ pointer_admission(&tiled, &[aimed_on((150, 100), rect)])+ .unwrap_err()+ .to_string(),+ "foreground_unavailable: niri window 5: niri does not report where a tiled window is; use get_desktop_state and a desktop-scope action (needs the display grant), or float the window"+ );+ }++ #[test]+ fn a_tiled_window_refuses_window_capture_and_window_coordinates() {+ // Even the focused active tiled window: its neighbour shares the+ // output, and niri cannot say which pixels are whose.+ let scene = tiled_scene(Some(5));+ assert_eq!(+ capture_plan_in(&scene, 5, Some(105)),+ Err("niri does not report where a tiled window is; use get_desktop_state and a desktop-scope action (needs the display grant), or float the window")+ );+ let window = scene.window(5).unwrap();+ assert_eq!(+ aim_at(&window, 10, 20).unwrap_err().to_string(),+ "foreground_unavailable: niri window 5: niri does not report where a tiled window is; use get_desktop_state and a desktop-scope action (needs the display grant), or float the window"+ );+ }++ #[test]+ fn desktop_pixels_are_admitted_only_on_the_single_output() {+ let scene = tiled_scene(Some(5));+ desktop_admission(&scene, &[(2500, 400).into(), (2100, 100).into()]).unwrap();+ assert_eq!(+ desktop_admission(&scene, &[(10, 100).into()])+ .unwrap_err()+ .to_string(),+ "foreground_unavailable: point (10, 100) is past the edge of niri output winit, which the get_desktop_state screenshot shows"+ );+ }+}diff --git a/libs/cua-driver/rust/crates/platform-linux/src/wayland/niri/ipc.rs b/libs/cua-driver/rust/crates/platform-linux/src/wayland/niri/ipc.rsnew file mode 100644index 0000000000..36f8fc25d6--- /dev/null+++ b/libs/cua-driver/rust/crates/platform-linux/src/wayland/niri/ipc.rs@@ -0,0 +1,854 @@+//! The niri IPC transport: socket attestation, requests and the serde subset of+//! niri's replies.++use super::*;++pub(super) const QUERY_TIMEOUT: Duration = Duration::from_secs(1);+const MAX_REPLY_BYTES: usize = 4 * 1024 * 1024;+const MAX_LOGICAL_PIXELS: u64 = 64 * 1024 * 1024;++// ── Session ──────────────────────────────────────────────────────────────────++/// The compositor process recognised as niri for this driver process.+pub(super) static RECOGNIZED: OnceLock<libc::pid_t> = OnceLock::new();+/// Set once a probe found `NIRI_SOCKET` lastingly unrelated to this session,+/// so a stale socket on another desktop costs one probe, not one per call.+static UNRELATED: OnceLock<()> = OnceLock::new();++#[derive(Debug, PartialEq, Eq)]+pub(super) enum Attestation {+ /// `NIRI_SOCKET` belongs to another session: not niri for this process.+ Unrelated(String),+ Attested(libc::pid_t),+ /// niri was recognised, but this connection cannot be proven to reach it.+ Refused(String),+}++/// Session identity is tri-state. Before recognition, IPC and Wayland peers+/// that differ (an inherited socket of another desktop) mean an unrelated+/// session. After recognition, any loss, mismatch or new compositor process+/// refuses rather than falling through to generic discovery.+fn attest(+ pinned: Option<libc::pid_t>,+ ipc_peer: Result<libc::pid_t>,+ wayland_peer: Result<libc::pid_t>,+) -> Attestation {+ let refuse = |reason| match pinned {+ None => Attestation::Unrelated(reason),+ Some(_) => Attestation::Refused(reason),+ };+ let (ipc_peer, wayland_peer) = match (ipc_peer, wayland_peer) {+ (Ok(ipc), Ok(wayland)) => (ipc, wayland),+ (ipc, wayland) => {+ return refuse(format!(+ "niri IPC or WAYLAND_DISPLAY peer is unavailable (IPC: {}, Wayland: {})",+ ipc.map_or_else(|e| format!("{e:#}"), |pid| format!("pid {pid}")),+ wayland.map_or_else(|e| format!("{e:#}"), |pid| format!("pid {pid}")),+ ));+ }+ };+ if ipc_peer != wayland_peer {+ return refuse(format!(+ "NIRI_SOCKET is served by pid {ipc_peer} but WAYLAND_DISPLAY by pid {wayland_peer}"+ ));+ }+ match pinned {+ Some(pinned) if pinned != ipc_peer => Attestation::Refused(format!(+ "niri was recognised as pid {pinned}, but the compositor is now pid {ipc_peer}"+ )),+ _ => Attestation::Attested(ipc_peer),+ }+}++fn socket_path() -> Result<PathBuf> {+ let path = PathBuf::from(std::env::var_os("NIRI_SOCKET").context("missing NIRI_SOCKET")?);+ if !path.is_absolute() {+ bail!("NIRI_SOCKET must be an absolute path");+ }+ Ok(path)+}++fn connect(timeout: Duration) -> Result<UnixStream> {+ let socket = socket2::Socket::new(socket2::Domain::UNIX, socket2::Type::STREAM, None)?;+ socket.connect_timeout(&socket2::SockAddr::unix(socket_path()?)?, timeout)?;+ Ok(socket.into())+}++/// The attestation, the IPC connection, and whether the answer is lasting:+/// a socket nobody serves, or two different live peers. A timeout is not.+pub(super) fn probe(+ pinned: Option<libc::pid_t>,+ deadline: Instant,+) -> (Attestation, Option<UnixStream>, bool) {+ let ipc = time_remaining(deadline).and_then(connect);+ let ipc_peer = ipc+ .as_ref()+ .map_err(|e| anyhow!("{e:#}"))+ .and_then(|ipc| peer_pid(ipc.as_raw_fd()));+ // Without an IPC peer there is nothing to compare the Wayland peer with.+ let wayland_peer = match &ipc_peer {+ Ok(_) => time_remaining(deadline)+ .and_then(wayland_connection_with_timeout)+ .and_then(|wayland| peer_pid(wayland.backend().poll_fd().as_raw_fd())),+ Err(_) => Err(anyhow!("not checked")),+ };+ let lasting = match &ipc {+ Err(error) => error.downcast_ref::<io::Error>().is_some_and(|error| {+ matches!(+ error.kind(),+ io::ErrorKind::NotFound | io::ErrorKind::ConnectionRefused+ )+ }),+ Ok(_) => ipc_peer.is_ok() && wayland_peer.is_ok(),+ };+ (attest(pinned, ipc_peer, wayland_peer), ipc.ok(), lasting)+}++/// True when this process drives a niri session: `NIRI_SOCKET` and+/// `WAYLAND_DISPLAY` were attested to be the same niri process, now or+/// earlier in this process. Recognition is remembered, so a later socket+/// loss refuses instead of turning this off. Also true while a probe cannot+/// tell (see [`Probed::Unknown`]), so niri's routes refuse.+pub fn is_session() -> bool {+ if let Some(known) = session_check(+ !super::super::is_inject_mode() && super::super::is_wayland(),+ RECOGNIZED.get().is_some(),+ std::env::var_os("NIRI_SOCKET").is_some_and(|socket| !socket.is_empty()),+ ) {+ return known;+ }+ if UNRELATED.get().is_some() {+ return false;+ }+ let (attestation, _, lasting) =+ probe(None, crate::interaction::operation_deadline(QUERY_TIMEOUT));+ match probed(attestation, lasting) {+ Probed::Niri(pid) => {+ let pinned = *RECOGNIZED.get_or_init(|| pid);+ tracing::info!(pid = pinned, "niri session recognised");+ true+ }+ Probed::Unknown(reason) => {+ tracing::warn!("niri session probe failed, refusing until one passes: {reason}");+ true+ }+ Probed::NotNiri(reason) => {+ tracing::warn!("NIRI_SOCKET is not this Wayland session's niri: {reason}");+ let _ = UNRELATED.set(());+ false+ }+ }+}++#[derive(Debug, PartialEq)]+enum Probed {+ Niri(libc::pid_t),+ /// A failure that may pass, such as a timeout or a failed peer read:+ /// the session may be niri. Its routes refuse until a later probe+ /// recognises niri, instead of the generic Wayland routes the adapter+ /// replaces running on a niri session.+ Unknown(String),+ /// Another session's socket, or none listening: lasting, so remembered.+ NotNiri(String),+}++/// What a probe before recognition decides; `lasting` as [`probe`] reports.+fn probed(attestation: Attestation, lasting: bool) -> Probed {+ match attestation {+ Attestation::Attested(pid) => Probed::Niri(pid),+ Attestation::Unrelated(reason) | Attestation::Refused(reason) if lasting => {+ Probed::NotNiri(reason)+ }+ Attestation::Unrelated(reason) | Attestation::Refused(reason) => Probed::Unknown(reason),+ }+}++/// The recognised niri process, or the refusal while there is none.+pub(super) fn recognised() -> Result<libc::pid_t> {+ RECOGNIZED+ .get()+ .copied()+ .context("niri session has not been recognised")+}++/// Whether this is a niri session without probing, or `None` to probe. A+/// niri recognised earlier stays niri even if the socket is gone, so routes+/// refuse instead of falling through to generic Wayland discovery.+fn session_check(adapter_on: bool, recognized: bool, socket_set: bool) -> Option<bool> {+ match (adapter_on, recognized, socket_set) {+ (false, ..) => Some(false),+ (true, true, _) => Some(true),+ (true, false, socket_set) => (!socket_set).then_some(false),+ }+}++/// A connection to the recognised niri process, re-attested each time.+pub(super) fn attested_connection(deadline: Instant) -> Result<UnixStream> {+ match probe(Some(recognised()?), deadline) {+ (Attestation::Attested(_), Some(ipc), _) => Ok(ipc),+ (Attestation::Unrelated(reason) | Attestation::Refused(reason), ..) => bail!("{reason}"),+ (Attestation::Attested(_), None, _) => bail!("niri IPC connection unavailable"),+ }+}++/// A Wayland client socket proven to reach the recognised niri process, for+/// a keyboard helper that must not rediscover the display on its own.+pub(in crate::wayland) fn attested_wayland_socket() -> Result<UnixStream> {+ let pinned = recognised()?;+ let socket = super::super::compositor_peer::wayland_socket_with_timeout(time_remaining(+ crate::interaction::operation_deadline(QUERY_TIMEOUT),+ )?)?;+ let peer = peer_pid(socket.as_raw_fd())?;+ if peer != pinned {+ bail!("WAYLAND_DISPLAY is served by pid {peer}, not the recognised niri pid {pinned}");+ }+ Ok(socket)+}++/// IPC to the recognised niri process. One value serves one action: its+/// requests share one attested connection.+pub(crate) type AttestedIpc = Ipc<fn(Instant) -> Result<UnixStream>>;++pub(crate) fn ipc() -> AttestedIpc {+ Ipc {+ connect: attested_connection,+ timeout: QUERY_TIMEOUT,+ lines: None,+ }+}++// ── IPC transport ────────────────────────────────────────────────────────────++fn time_remaining(deadline: Instant) -> Result<Duration> {+ let remaining = deadline.saturating_duration_since(Instant::now());+ if remaining.is_zero() {+ return Err(io::Error::new(io::ErrorKind::TimedOut, "niri IPC timed out").into());+ }+ Ok(remaining)+}++/// niri replies one JSON line per request; the event stream sends one line+/// per event.+pub(super) struct Lines {+ pub(super) stream: UnixStream,+ buffer: Vec<u8>,+ /// Bytes of `buffer` already known to hold no line end.+ scanned: usize,+}++impl Lines {+ pub(super) fn new(stream: UnixStream) -> Self {+ Self {+ stream,+ buffer: Vec::new(),+ scanned: 0,+ }+ }++ pub(super) fn send(&mut self, line: &str, deadline: Instant) -> Result<()> {+ self.stream+ .set_write_timeout(Some(time_remaining(deadline)?))?;+ self.stream.write_all(line.as_bytes())?;+ self.stream.write_all(b"\n")?;+ Ok(())+ }++ /// The next complete line, or `None` once `deadline` passes first.+ pub(super) fn next(&mut self, deadline: Instant) -> Result<Option<Vec<u8>>> {+ let mut chunk = [0u8; 8192];+ loop {+ if let Some(end) = self.buffer[self.scanned..]+ .iter()+ .position(|byte| *byte == b'\n')+ {+ let mut line: Vec<u8> = self.buffer.drain(..=self.scanned + end).collect();+ line.pop();+ self.scanned = 0;+ return Ok(Some(line));+ }+ self.scanned = self.buffer.len();+ let remaining = deadline.saturating_duration_since(Instant::now());+ if remaining.is_zero() {+ return Ok(None);+ }+ self.stream.set_read_timeout(Some(remaining))?;+ match self.stream.read(&mut chunk) {+ Ok(0) => bail!("niri closed the IPC connection"),+ Ok(count) => {+ if self.buffer.len() + count > MAX_REPLY_BYTES {+ bail!("niri IPC reply exceeds size limit");+ }+ self.buffer.extend_from_slice(&chunk[..count]);+ }+ Err(error)+ if matches!(+ error.kind(),+ io::ErrorKind::WouldBlock | io::ErrorKind::TimedOut+ ) =>+ {+ return Ok(None)+ }+ Err(error) => return Err(error).context("niri IPC reply unavailable"),+ }+ }+ }++ /// One request and its reply line.+ fn exchange(&mut self, request: &str, deadline: Instant) -> Result<serde_json::Value> {+ self.send(request, deadline)?;+ self.reply(deadline)+ }++ /// The reply line to the request sent last.+ fn reply(&mut self, deadline: Instant) -> Result<serde_json::Value> {+ let line = self.next(deadline)?.ok_or_else(|| {+ anyhow::Error::from(io::Error::new(+ io::ErrorKind::TimedOut,+ "niri IPC reply timed out",+ ))+ })?;+ serde_json::from_slice(&line).context("invalid niri IPC JSON")+ }+}++/// The successful payload of a reply, or niri's own error text.+pub(super) fn reply_ok(request: &str, reply: serde_json::Value) -> Result<serde_json::Value> {+ let serde_json::Value::Object(mut reply) = reply else {+ bail!("unexpected niri reply to {request}");+ };+ if let Some(error) = reply.remove("Err") {+ bail!(+ "niri refused {request}: {}",+ error+ .as_str()+ .map_or_else(|| error.to_string(), str::to_owned)+ );+ }+ reply+ .remove("Ok")+ .with_context(|| format!("unexpected niri reply to {request}"))+}++/// The niri IPC requests this adapter reads.+#[derive(Clone, Copy, Debug)]+pub(super) enum Request {+ Version,+ Windows,+ Workspaces,+ Outputs,+ FocusedWindow,+ OverviewState,+ KeyboardLayouts,+}++pub(crate) struct Ipc<C> {+ pub(super) connect: C,+ /// The deadline of each request and its reply.+ pub(super) timeout: Duration,+ /// The connection earlier requests used. A connected socket keeps its+ /// peer, so it stays attested; niri answers one connection's requests+ /// in order. A failed exchange drops it, so a late reply is never read+ /// as the answer to a later request.+ pub(super) lines: Option<Lines>,+}++impl<C: FnMut(Instant) -> Result<UnixStream>> Ipc<C> {+ fn connection(&mut self, deadline: Instant) -> Result<Lines> {+ match self.lines.take() {+ Some(lines) => Ok(lines),+ None => Ok(Lines::new((self.connect)(deadline)?)),+ }+ }++ /// A read-only request; a failure is final.+ pub(super) fn query<T: DeserializeOwned>(&mut self, request: Request) -> Result<T> {+ let deadline = interaction::operation_deadline(self.timeout);+ self.query_until(request, deadline)+ }++ pub(super) fn query_until<T: DeserializeOwned>(+ &mut self,+ request: Request,+ deadline: Instant,+ ) -> Result<T> {+ let name = format!("{request:?}");+ let name = name.as_str();+ let mut lines = self.connection(deadline)?;+ let reply = lines+ .exchange(&format!("\"{name}\""), deadline)+ .with_context(|| format!("niri IPC {name} failed"))?;+ self.lines = Some(lines);+ let mut payload = reply_ok(name, reply)?;+ let value = payload+ .get_mut(name)+ .map(serde_json::Value::take)+ .with_context(|| format!("niri reply to {name} has no {name}"))?;+ serde_json::from_value(value).with_context(|| format!("invalid niri {name} reply"))+ }++ /// One compositor action. Never retried: after a timeout niri may have+ /// applied it. Like input, it is not sent once the tool call bound to+ /// this thread ran out of time (see [`crate::input::action`]): focus+ /// must not move after the caller stopped waiting.+ pub(super) fn action(&mut self, action: serde_json::Value) -> Result<()> {+ self.send_action(action, None)+ }++ /// An action that gives back focus the call took. It is cleanup, like+ /// a button release, so it goes out also once the call ran out of time,+ /// within the transaction's one `cleanup` deadline.+ pub(super) fn restore_action(+ &mut self,+ action: serde_json::Value,+ cleanup: Instant,+ ) -> Result<()> {+ self.send_action(action, Some(cleanup))+ }++ fn send_action(&mut self, action: serde_json::Value, cleanup: Option<Instant>) -> Result<()> {+ let deadline = cleanup.unwrap_or_else(|| interaction::operation_deadline(self.timeout));+ let request = serde_json::json!({ "Action": action }).to_string();+ let mut lines = self.connection(deadline)?;+ if cleanup.is_some() {+ lines.send(&request, deadline)?;+ } else {+ crate::input::action::admit(|| lines.send(&request, deadline))??;+ }+ let reply = lines+ .reply(deadline)+ .context("niri action outcome is unknown")?;+ self.lines = Some(lines);+ match reply_ok(&request, reply)? {+ serde_json::Value::String(handled) if handled == "Handled" => Ok(()),+ other => bail!("unexpected niri reply to {request}: {other}"),+ }+ }++ pub(super) fn focused_window(&mut self) -> Result<Option<IpcWindow>> {+ self.query(Request::FocusedWindow)+ }++ /// One snapshot of the layout, its four requests sharing one deadline.+ pub(super) fn scene(&mut self) -> Result<Scene> {+ self.scene_until(interaction::operation_deadline(self.timeout))+ }++ pub(super) fn scene_until(&mut self, deadline: Instant) -> Result<Scene> {+ let windows = self.query_until(Request::Windows, deadline)?;+ let workspaces = self.query_until(Request::Workspaces, deadline)?;+ let outputs = self.query_until(Request::Outputs, deadline)?;+ let overview: Overview = self.query_until(Request::OverviewState, deadline)?;+ Ok(Scene {+ windows,+ workspaces,+ outputs,+ overview_open: overview.is_open,+ })+ }+}++// ── Replies ──────────────────────────────────────────────────────────────────++#[derive(Clone, Debug, Deserialize, PartialEq)]+pub(super) struct IpcWindow {+ pub(super) id: u64,+ #[serde(default)]+ pub(super) title: Option<String>,+ #[serde(default)]+ pub(super) app_id: Option<String>,+ pub(super) pid: Option<i32>,+ pub(super) workspace_id: Option<u64>,+ pub(super) is_focused: bool,+ /// When the window last had focus; absent on older releases.+ #[serde(default)]+ pub(super) focus_timestamp: Option<IpcTimestamp>,+ /// Added in niri 25.08; absent on older releases.+ #[serde(default)]+ pub(super) layout: Option<IpcLayout>,+}++#[derive(Clone, Copy, Debug, Deserialize, PartialEq, Eq, PartialOrd, Ord)]+pub(super) struct IpcTimestamp {+ secs: u64,+ nanos: u32,+}++#[derive(Clone, Debug, Deserialize, PartialEq)]+pub(super) struct IpcLayout {+ pub(super) pos_in_scrolling_layout: Option<(u64, u64)>,+ pub(super) tile_size: (f64, f64),+ pub(super) window_size: (i32, i32),+ pub(super) tile_pos_in_workspace_view: Option<(f64, f64)>,+ pub(super) window_offset_in_tile: (f64, f64),+}++#[derive(Clone, Debug, Deserialize, PartialEq)]+pub(super) struct IpcWorkspace {+ pub(super) id: u64,+ pub(super) output: Option<String>,+ pub(super) is_active: bool,+ pub(super) is_focused: bool,+ pub(super) active_window_id: Option<u64>,+}++#[derive(Clone, Debug, Deserialize, PartialEq)]+pub(super) struct IpcOutput {+ pub(super) name: String,+ pub(super) modes: Vec<IpcMode>,+ pub(super) current_mode: Option<usize>,+ pub(super) logical: Option<IpcLogical>,+}++#[derive(Clone, Copy, Debug, Deserialize, PartialEq)]+pub(super) struct IpcMode {+ pub(super) width: u32,+ pub(super) height: u32,+}++#[derive(Clone, Debug, Deserialize, PartialEq)]+pub(super) struct IpcLogical {+ pub(super) x: i32,+ pub(super) y: i32,+ pub(super) width: u32,+ pub(super) height: u32,+ pub(super) scale: f64,+ pub(super) transform: String,+}++#[derive(Deserialize)]+pub(super) struct Overview {+ pub(super) is_open: bool,+}++#[derive(Deserialize)]+pub(super) struct KeyboardLayouts {+ pub(super) names: Vec<String>,+ pub(super) current_idx: u32,+}++/// wl_output transform number of a niri transform name.+pub(super) fn transform_code(name: &str) -> Result<u32> {+ Ok(match name {+ "Normal" => 0,+ "90" => 1,+ "180" => 2,+ "270" => 3,+ "Flipped" => 4,+ "Flipped90" => 5,+ "Flipped180" => 6,+ "Flipped270" => 7,+ other => bail!("unknown niri output transform {other:?}"),+ })+}++pub(super) fn valid_dimensions(width: u32, height: u32) -> bool {+ width > 0 && height > 0 && u64::from(width) * u64::from(height) <= MAX_LOGICAL_PIXELS+}++impl IpcLogical {+ pub(super) fn checked(&self) -> Result<&Self> {+ if !self.scale.is_finite()+ || self.scale <= 0.0+ || !valid_dimensions(self.width, self.height)+ {+ bail!("invalid niri output geometry");+ }+ Ok(self)+ }+}++impl IpcOutput {+ /// The output's size in niri's pointer space. Absolute virtual-pointer+ /// motion uses Smithay's output geometry, which rounds `mode / scale` up,+ /// while IPC truncates it (1280 px at 1.5 is 853 in IPC, 854 here).+ pub(super) fn pointer_rect(&self) -> Result<Option<Rect>> {+ let Some(logical) = &self.logical else {+ return Ok(None);+ };+ let logical = logical.checked()?;+ let mode = self+ .current_mode+ .and_then(|index| self.modes.get(index))+ .with_context(|| format!("niri output {} has no current mode", self.name))?;+ let (width, height) = super::super::logical_output_size(+ (mode.width, mode.height),+ transform_code(&logical.transform)?,+ );+ let width = (f64::from(width) / logical.scale).ceil();+ let height = (f64::from(height) / logical.scale).ceil();+ if !(1.0..=f64::from(u32::MAX)).contains(&width)+ || !(1.0..=f64::from(u32::MAX)).contains(&height)+ {+ bail!("invalid niri output mode");+ }+ Ok(Some(Rect {+ x: logical.x,+ y: logical.y,+ width: width as u32,+ height: height as u32,+ }))+ }+}++#[cfg(test)]+mod tests {+ use super::*;+ use serde_json::{json, Value};++ /// A connection whose peer sends `reply` (raw bytes) and then holds it open.+ fn replying(reply: Vec<u8>) -> UnixStream {+ let (client, mut server) = UnixStream::pair().unwrap();+ std::thread::spawn(move || {+ let _ = server.write_all(&reply);+ std::thread::sleep(Duration::from_secs(2));+ });+ client+ }++ // ── Session attestation ─────────────────────────────────────────────────++ #[test]+ fn foreign_socket_before_recognition_is_an_unrelated_session() {+ // A host desktop's inherited NIRI_SOCKET next to a nested compositor.+ assert_eq!(+ attest(None, Ok(10), Ok(20)),+ Attestation::Unrelated(+ "NIRI_SOCKET is served by pid 10 but WAYLAND_DISPLAY by pid 20".into()+ )+ );+ assert!(matches!(+ attest(None, Err(anyhow!("connection refused")), Ok(20)),+ Attestation::Unrelated(_)+ ));+ assert_eq!(attest(None, Ok(20), Ok(20)), Attestation::Attested(20));+ }++ #[test]+ fn a_probe_that_may_pass_leaves_the_session_unknown_and_refusing() {+ let timed_out = "niri IPC or WAYLAND_DISPLAY peer is unavailable (IPC: niri IPC timed out, Wayland: not checked)";+ assert_eq!(+ probed(Attestation::Unrelated(timed_out.into()), false),+ Probed::Unknown(timed_out.into())+ );+ // Nothing listening, or another session's niri: not niri.+ assert_eq!(+ probed(Attestation::Unrelated("refused".into()), true),+ Probed::NotNiri("refused".into())+ );+ assert_eq!(probed(Attestation::Attested(20), true), Probed::Niri(20));+ }++ #[test]+ fn a_recognised_niri_stays_niri_after_its_socket_disappears() {+ assert_eq!(session_check(true, true, false), Some(true));+ assert_eq!(session_check(true, false, false), Some(false));+ assert_eq!(session_check(true, false, true), None);+ assert_eq!(session_check(false, true, true), Some(false));+ }++ #[test]+ fn recognised_niri_refuses_loss_mismatch_and_a_new_compositor() {+ assert_eq!(attest(Some(20), Ok(20), Ok(20)), Attestation::Attested(20));+ match attest(Some(20), Err(anyhow!("No such file or directory")), Ok(20)) {+ Attestation::Refused(reason) => assert!(reason.contains("No such file or directory")),+ other => panic!("socket loss must refuse, got {other:?}"),+ }+ assert_eq!(+ attest(Some(20), Ok(20), Ok(30)),+ Attestation::Refused(+ "NIRI_SOCKET is served by pid 20 but WAYLAND_DISPLAY by pid 30".into()+ )+ );+ assert_eq!(+ attest(Some(20), Ok(40), Ok(40)),+ Attestation::Refused(+ "niri was recognised as pid 20, but the compositor is now pid 40".into()+ )+ );+ }++ // ── Transport ───────────────────────────────────────────────────────────++ #[test]+ fn stalled_niri_hits_the_deadline_without_a_retry() {+ let mut attempts = 0;+ let mut held = Vec::new();+ let started = Instant::now();+ let error = ipc_with(+ |_| {+ attempts += 1;+ let (client, server) = UnixStream::pair()?;+ held.push(server);+ Ok(client)+ },+ Duration::from_millis(40),+ )+ .query::<Value>(Request::Windows)+ .unwrap_err();+ assert!(format!("{error:#}").contains("timed out"), "{error:#}");+ assert_eq!(attempts, 1);+ // Bounded by the deadlines, not by the peer; loose for loaded CI.+ assert!(started.elapsed() < Duration::from_secs(2));+ }++ #[test]+ fn one_action_attests_one_connection_for_all_its_requests() {+ let fake = focus_fixture();+ let mut ipc = fake_ipc(&fake);+ ipc.focused_window().unwrap();+ activate_in(&mut ipc, 7, Some(107), Duration::from_millis(200)).unwrap();+ let fake = fake.lock().unwrap();+ assert_eq!(fake.requests.len(), 5, "{:?}", fake.requests);+ assert_eq!(fake.connections, 1);+ }++ #[test]+ fn an_action_is_never_retried_after_an_ambiguous_timeout() {+ let mut attempts = 0;+ let mut held = Vec::new();+ let error = ipc_with(+ |_| {+ attempts += 1;+ let (client, server) = UnixStream::pair()?;+ held.push(server);+ Ok(client)+ },+ Duration::from_millis(40),+ )+ .action(focus_window_action(7))+ .unwrap_err();+ assert_eq!(attempts, 1);+ assert_eq!(error.to_string(), "niri action outcome is unknown");+ }++ #[test]+ fn the_four_scene_requests_share_one_deadline() {+ let fake = std::sync::Arc::new(std::sync::Mutex::new(Fake {+ outputs: vec![output("winit", 0, 0, (1280, 720), 1.0, "Normal")],+ ..Fake::default()+ }));+ let slow = fake.clone();+ let mut ipc = ipc_with(+ move |_| {+ let (client, server) = UnixStream::pair()?;+ let fake = slow.clone();+ std::thread::spawn(move || {+ let mut lines = Lines::new(server);+ while let Ok(Some(line)) = lines.next(Instant::now() + Duration::from_secs(2)) {+ std::thread::sleep(Duration::from_millis(40));+ let reply = fake+ .lock()+ .unwrap()+ .reply(std::str::from_utf8(&line).unwrap());+ if lines+ .stream+ .write_all(format!("{reply}\n").as_bytes())+ .is_err()+ {+ return;+ }+ }+ });+ Ok(client)+ },+ Duration::from_millis(100),+ );+ ipc.query::<Value>(Request::Windows).unwrap();+ assert!(ipc.scene().is_err());+ }++ #[test]+ fn ordinary_round_trips_use_the_action_deadline_but_cleanup_keeps_its_own() {+ let action = crate::input::action::Action::new(Duration::from_secs(1));+ let _bound = crate::input::action::bind(action.clone());+ let mut deadlines = Vec::new();+ let cleanup = Instant::now() + Duration::from_secs(2);+ {+ let mut ipc = ipc_with(+ |deadline| {+ deadlines.push(deadline);+ bail!("no connection")+ },+ Duration::from_secs(5),+ );+ assert!(ipc.query::<Value>(Request::Windows).is_err());+ assert!(ipc.action(focus_window_action(7)).is_err());+ assert!(ipc.scene().is_err());+ action.abandon();+ assert!(ipc.restore_action(focus_window_action(5), cleanup).is_err());+ }+ assert_eq!(+ deadlines,+ vec![+ action.deadline(),+ action.deadline(),+ action.deadline(),+ cleanup+ ]+ );+ }++ #[test]+ fn connection_and_attestation_errors_are_not_retried() {+ let mut attempts = 0;+ let error = ipc_with(+ |_| {+ attempts += 1;+ bail!("NIRI_SOCKET is served by pid 10 but WAYLAND_DISPLAY by pid 20")+ },+ QUERY_TIMEOUT,+ )+ .query::<Value>(Request::Windows)+ .unwrap_err();+ assert_eq!(attempts, 1);+ assert_eq!(+ error.to_string(),+ "NIRI_SOCKET is served by pid 10 but WAYLAND_DISPLAY by pid 20"+ );+ }++ #[test]+ fn niri_error_text_is_kept_verbatim() {+ let error = ipc_with(+ |_| Ok(replying(b"{\"Err\":\"error parsing request\"}\n".to_vec())),+ QUERY_TIMEOUT,+ )+ .action(focus_window_action(7))+ .unwrap_err();+ assert!(error.to_string().ends_with(+ "niri refused {\"Action\":{\"FocusWindow\":{\"id\":7}}}: error parsing request"+ ));+ }++ #[test]+ fn oversize_and_malformed_replies_refuse() {+ let reply = |bytes: Vec<u8>| {+ ipc_with(move |_| Ok(replying(bytes.clone())), QUERY_TIMEOUT)+ .query::<Vec<IpcWindow>>(Request::Windows)+ .unwrap_err()+ };+ let oversize = reply(vec![b' '; MAX_REPLY_BYTES + 1]);+ assert!(format!("{oversize:#}").contains("niri IPC reply exceeds size limit"));+ let id_as_string =+ br#"{"Ok":{"Windows":[{"id":"7","pid":1,"workspace_id":1,"is_focused":false}]}}+"#;+ assert!(+ format!("{:#}", reply(id_as_string.to_vec())).contains("invalid niri Windows reply")+ );+ let other_variant = b"{\"Ok\":{\"Workspaces\":[]}}\n";+ assert!(format!("{:#}", reply(other_variant.to_vec()))+ .contains("niri reply to Windows has no Windows"));+ }++ #[test]+ fn unknown_fields_are_ignored_and_null_pid_has_no_owner() {+ let mut window = floating(7, 1, (100.0, 60.0), (300, 200));+ window["pid"] = Value::Null;+ window["is_new_in_a_later_release"] = json!({"nested": [1, 2]});+ window["layout"]["also_new"] = json!(true);+ let scene = one_output(vec![window]);+ let window = scene.window(7).unwrap();+ assert_eq!(window.pid, None);+ assert!(window.placement.is_ok());+ }+}diff --git a/libs/cua-driver/rust/crates/platform-linux/src/wayland/niri/keyboard.rs b/libs/cua-driver/rust/crates/platform-linux/src/wayland/niri/keyboard.rsnew file mode 100644index 0000000000..43237ff4a0--- /dev/null+++ b/libs/cua-driver/rust/crates/platform-linux/src/wayland/niri/keyboard.rs@@ -0,0 +1,1218 @@+//! Text input on niri through a virtual keyboard that carries niri's own+//! keymap, so every character goes out on the key the user's layout has it+//! on. A tool like `wtype` instead numbers the n-th distinct character of a+//! call as evdev code n, and Chromium and Electron take the DOM `code` from+//! the evdev code: a punctuation mark on code 1 (Escape) is dropped, one on+//! code 14 (Backspace) deletes the character before it, and one on code 28+//! (Enter) can submit a form.+//!+//! Characters the active layout lacks go on spare keys of an extended copy+//! of the keymap: keys without symbols that Chromium knows, then F13 to F24.+//! niri sends a virtual keyboard's keymap to every client with the device's+//! next key, so clients read each key with the map it was sent in; the+//! compositor's map is uploaded again before the device goes away.+//!+//! Before each key, niri is asked which window has keyboard focus; typing+//! stops when it is no longer the window that had it when typing started. A+//! locked niri reports no focused window, so this also stops at the lock+//! screen. Every round trip and the whole call have a deadline, and held+//! keys are released before returning: destroying the device releases+//! nothing in niri.+//!+//! Text outside the Basic Multilingual Plane, such as most emoji, goes out+//! as its Unicode keysym like any other character, as `wtype` sends it.+//! Chromium cuts a keysym's character to 16 bits, so it types a wrong+//! character there whatever the input route.++use std::fs::File;+use std::io::{Seek, SeekFrom, Write};+use std::ops::RangeInclusive;+use std::os::fd::{AsFd, OwnedFd};+use std::os::unix::fs::FileExt;+use std::time::{Duration, Instant};++use anyhow::{anyhow, bail, Context, Result};+use wayland_client::protocol::{wl_callback, wl_keyboard, wl_registry, wl_seat};+use wayland_client::{Connection, Dispatch, EventQueue, Proxy, QueueHandle, WEnum};+use xkbcommon::xkb;++use super::super::virtual_keyboard::protocol::zwp_virtual_keyboard_manager_v1::ZwpVirtualKeyboardManagerV1;+use super::super::virtual_keyboard::protocol::zwp_virtual_keyboard_v1::ZwpVirtualKeyboardV1;++/// The whole call's budget: setup, then this much per key.+const CALL_BASE: Duration = Duration::from_secs(3);+const PER_KEY: Duration = Duration::from_millis(100);++/// How long a call that sends `keys` keys gives itself, from when its+/// keyboard session begins. Past it, the call stops and says how far it got.+pub fn call_budget(keys: usize) -> Duration {+ CALL_BASE + PER_KEY * u32::try_from(keys).unwrap_or(u32::MAX)+}+const MAX_KEYMAP: u32 = 1024 * 1024;++const KEY_RELEASED: u32 = 0;+const KEY_PRESSED: u32 = 1;++// ── Planning ─────────────────────────────────────────────────────────────────++/// One key: its evdev code, and the layout and modifiers it is sent in.+#[derive(Clone, Copy, Debug, PartialEq, Eq)]+struct Key {+ code: u32,+ modifiers: u32,+ group: u32,+}++/// Consecutive keys typed with one keymap: niri's own when `extended` is+/// `None`, otherwise that copy with missing symbols on spare keys.+#[derive(Debug)]+struct Part {+ keys: Vec<Key>,+ extended: Option<String>,+}++/// The highest keycode Xwayland clients can receive.+const MAX_X11_KEYCODE: u32 = 255;++/// The evdev codes up to `MAX_X11_KEYCODE` that Chromium gives a DOM `code`+/// (the rows of `ui/events/keycodes/dom/dom_code_data.inc` with an evdev+/// code and a code string). Chromium and Electron drop a key without one.+const CHROMIUM_CODES: [RangeInclusive<u32>; 22] = [+ 1..=83,+ 85..=94,+ 96..=100,+ 102..=111,+ 113..=117,+ 119..=129,+ 131..=138,+ 140..=140,+ 142..=144,+ 155..=156,+ 158..=159,+ 161..=161,+ 163..=168,+ 171..=173,+ 179..=180,+ 183..=194,+ 201..=201,+ 207..=208,+ 217..=217,+ 224..=225,+ 227..=227,+ 231..=233,+];++/// F13 to F24: Chromium knows them, keymaps give them symbols, and few+/// keyboards have them, so a keymap may borrow them once its keys without+/// symbols run out.+const BORROWABLE: RangeInclusive<u32> = 183..=194;++fn evdev_in(code: xkb::Keycode, ranges: &[RangeInclusive<u32>]) -> bool {+ code.raw()+ .checked_sub(8)+ .is_some_and(|evdev| ranges.iter().any(|range| range.contains(&evdev)))+}++/// Whether Chromium receives evdev `code` (all of these are within+/// `MAX_X11_KEYCODE`). A symbol the layout has only on other keys, such as+/// US's `EuroSign` on `<I443>` (evdev 435), counts as missing and goes on a+/// spare key.+fn chromium_receives(code: u32) -> bool {+ CHROMIUM_CODES.iter().any(|range| range.contains(&code))+}++fn refused(detail: impl std::fmt::Display) -> anyhow::Error {+ anyhow!("foreground_unavailable: {detail}; nothing was typed")+}++/// `text`'s keys in layout `group` of `map`, cut into parts whose symbols+/// the layout lacks fit on the keymap's spare keys.+fn plan_text(map: &str, group: u32, text: &str) -> Result<Vec<Part>> {+ let keymap = compile(map)?;+ if group >= keymap.num_layouts() {+ return Err(refused(format!(+ "niri's active layout {group} is absent from its keymap"+ )));+ }+ let spare = spare_keys(&keymap).len();+ split(&keymap, group, text, spare)?+ .into_iter()+ .map(|(part, missing)| {+ if missing.is_empty() {+ return Ok(Part {+ keys: characters(&keymap, group, part)?,+ extended: None,+ });+ }+ let (extended, serialized) = extend(map, &keymap, &missing)?;+ Ok(Part {+ keys: characters(&extended, group, part)?,+ extended: Some(serialized),+ })+ })+ .collect()+}++/// The keysym that types `c`. A line feed and a tab are the Return and Tab+/// keys, as `wtype` types them; any other control character is refused.+fn text_sym(c: char) -> Result<xkb::Keysym> {+ match c {+ '\n' => Ok(xkb::Keysym::Return),+ '\t' => Ok(xkb::Keysym::Tab),+ c if c.is_control() => Err(refused(format!(+ "control character U+{:04X} cannot be typed",+ u32::from(c)+ ))),+ c => match xkb::utf32_to_keysym(u32::from(c)) {+ sym if sym.raw() == 0 => Err(refused(format!("U+{:04X} has no keysym", u32::from(c)))),+ sym => Ok(sym),+ },+ }+}++fn characters(keymap: &xkb::Keymap, group: u32, text: &str) -> Result<Vec<Key>> {+ text.chars()+ .map(|c| {+ find(keymap, group, text_sym(c)?)+ .ok_or_else(|| refused(format!("U+{:04X} is absent from the keymap", u32::from(c))))+ })+ .collect()+}++/// `text` cut at character boundaries into parts, each with the distinct+/// symbols the layout lacks, in order of use, at most `spare` of them.+fn split<'t>(+ keymap: &xkb::Keymap,+ group: u32,+ text: &'t str,+ spare: usize,+) -> Result<Vec<(&'t str, Vec<xkb::Keysym>)>> {+ let mut parts = Vec::new();+ let (mut rest, mut start) = (text, 0);+ let mut missing = Vec::new();+ for (at, c) in text.char_indices() {+ let sym = text_sym(c)?;+ let present = find(keymap, group, sym).is_some_and(|key| chromium_receives(key.code));+ if present || missing.contains(&sym) {+ continue;+ }+ if spare == 0 {+ return Err(refused(format!(+ "U+{:04X} is absent from the active layout and niri's keymap has no spare key",+ u32::from(c)+ )));+ }+ if missing.len() == spare {+ let (part, after) = rest.split_at(at - start);+ parts.push((part, std::mem::take(&mut missing)));+ (rest, start) = (after, at);+ }+ missing.push(sym);+ }+ parts.push((rest, missing));+ Ok(parts)+}++/// Put each of `syms` on a spare key of `map` and prove every other key, the+/// layouts and the modifiers unchanged. Returns the extended keymap compiled+/// and the exact text that was proven.+fn extend(map: &str, keymap: &xkb::Keymap, syms: &[xkb::Keysym]) -> Result<(xkb::Keymap, String)> {+ let spare = spare_keys(keymap);+ if syms.len() > spare.len() {+ return Err(refused("more missing symbols than spare keys"));+ }+ let entries: String = spare+ .iter()+ .zip(syms)+ .map(|((_, name), &sym)| {+ format!(+ "replace key <{name}> {{ [ {} ] }};\n",+ xkb::keysym_get_name(sym)+ )+ })+ .collect();+ let end = symbols_end(map).ok_or_else(|| refused("niri's keymap has an unexpected layout"))?;+ let (head, tail) = map.split_at(end);+ let text = format!("{head}{entries}{tail}");+ let extended = compile(&text)?;+ let used: Vec<xkb::Keycode> = spare+ .iter()+ .take(syms.len())+ .map(|(code, _)| *code)+ .collect();+ if !unchanged(keymap, &extended, &used) {+ return Err(refused(+ "extending niri's keymap would change existing keys",+ ));+ }+ Ok((extended, text))+}++/// The keys up to `MAX_X11_KEYCODE` a keymap may put missing symbols on, in+/// order: those without symbols that Chromium knows, then the borrowable+/// ones the keymap has. Only a keymap with neither uses keys without symbols+/// that Chromium does not know.+fn spare_keys(keymap: &xkb::Keymap) -> Vec<(xkb::Keycode, String)> {+ let first = keymap.min_keycode().raw().max(9);+ let last = keymap.max_keycode().raw().min(MAX_X11_KEYCODE);+ let (empty, defined): (Vec<_>, Vec<_>) = (first..=last)+ .map(xkb::Keycode::new)+ .filter_map(|code| Some((code, keymap.key_get_name(code)?.to_owned())))+ .partition(|(code, _)| keymap.num_layouts_for_key(*code) == 0);+ let (known, unknown): (Vec<_>, Vec<_>) = empty+ .into_iter()+ .partition(|(code, _)| evdev_in(*code, &CHROMIUM_CODES));+ let chromium: Vec<_> = known+ .into_iter()+ .chain(+ defined+ .into_iter()+ .filter(|(code, _)| evdev_in(*code, &[BORROWABLE])),+ )+ .collect();+ if chromium.is_empty() {+ unknown+ } else {+ chromium+ }+}++/// Where the symbols section closes: the last `};` before the keymap's own.+fn symbols_end(map: &str) -> Option<usize> {+ let keymap_end = map.rfind("};")?;+ let end = map.get(..keymap_end)?.rfind("};")?;+ let symbols = map.find("xkb_symbols")?;+ let after = map.get(symbols..end)?;+ let other = ["xkb_keycodes", "xkb_types", "xkb_compat", "xkb_geometry"];+ (!other.iter().any(|section| after.contains(section))).then_some(end)+}++/// Every key but `added` is the same in both keymaps, and each of `added`+/// had no symbols or is borrowable, and now types only its one symbol.+fn unchanged(original: &xkb::Keymap, extended: &xkb::Keymap, added: &[xkb::Keycode]) -> bool {+ let mods = |map: &xkb::Keymap| -> Vec<String> {+ (0..map.num_mods())+ .map(|index| map.mod_get_name(index).to_owned())+ .collect()+ };+ original.num_layouts() == extended.num_layouts()+ && mods(original) == mods(extended)+ && (original.min_keycode().raw()..=original.max_keycode().raw())+ .map(xkb::Keycode::new)+ .filter(|code| !added.contains(code))+ .all(|code| same_key(original, extended, code))+ && added.iter().all(|&code| {+ (original.num_layouts_for_key(code) == 0 || evdev_in(code, &[BORROWABLE]))+ && inert(extended, code)+ })+}++fn same_key(original: &xkb::Keymap, extended: &xkb::Keymap, code: xkb::Keycode) -> bool {+ let layouts = original.num_layouts_for_key(code);+ layouts == extended.num_layouts_for_key(code)+ && (0..layouts).all(|layout| {+ let levels = original.num_levels_for_key(code, layout);+ levels == extended.num_levels_for_key(code, layout)+ && (0..levels).all(|level| {+ original.key_get_syms_by_level(code, layout, level)+ == extended.key_get_syms_by_level(code, layout, level)+ && level_mods(original, code, layout, level)+ == level_mods(extended, code, layout, level)+ })+ })+}++fn level_mods(map: &xkb::Keymap, code: xkb::Keycode, layout: u32, level: u32) -> Vec<u32> {+ let mut masks = [0_u32; 32];+ let count = map.key_get_mods_for_level(code, layout, level, &mut masks);+ masks.iter().take(count).copied().collect()+}++/// An added key has one symbol, in one layout and level, and changes no+/// modifier or layout.+fn inert(map: &xkb::Keymap, code: xkb::Keycode) -> bool {+ let mut state = xkb::State::new(map);+ state.update_key(code, xkb::KeyDirection::Down);+ map.num_layouts_for_key(code) == 1+ && map.num_levels_for_key(code, 0) == 1+ && map.key_get_syms_by_level(code, 0, 0).len() == 1+ && state.serialize_mods(xkb::STATE_MODS_EFFECTIVE) == 0+ && state.serialize_layout(xkb::STATE_LAYOUT_EFFECTIVE) == 0+}++fn compile(map: &str) -> Result<xkb::Keymap> {+ let context =+ xkb::Context::new(xkb::CONTEXT_NO_DEFAULT_INCLUDES | xkb::CONTEXT_NO_ENVIRONMENT_NAMES);+ xkb::Keymap::new_from_string(+ &context,+ map.to_owned(),+ xkb::KEYMAP_FORMAT_TEXT_V1,+ xkb::KEYMAP_COMPILE_NO_FLAGS,+ )+ .ok_or_else(|| refused("niri supplied a keymap libxkbcommon cannot compile"))+}++/// The key and modifiers that type `sym` in layout `group`: one Chromium+/// receives first, then the fewest modifiers, then the lowest code.+fn find(map: &xkb::Keymap, group: u32, sym: xkb::Keysym) -> Option<Key> {+ let mut candidates = Vec::new();+ for raw in map.min_keycode().raw().max(8)..=map.max_keycode().raw() {+ let code = xkb::Keycode::new(raw);+ for level in 0..map.num_levels_for_key(code, group) {+ if map.key_get_syms_by_level(code, group, level) != [sym] {+ continue;+ }+ let masks = level_mods(map, code, group, level);+ candidates.extend(masks.into_iter().filter_map(|modifiers| {+ let mut state = xkb::State::new(map);+ state.update_mask(modifiers, 0, 0, 0, 0, group);+ (state.key_get_one_sym(code) == sym).then_some(Key {+ code: raw - 8,+ modifiers,+ group,+ })+ }));+ }+ }+ candidates.into_iter().min_by_key(|key| {+ (+ !chromium_receives(key.code),+ key.modifiers.count_ones(),+ key.code,+ )+ })+}++/// The key `name` (an X keysym name, exact or else in any case) with+/// `modifiers` held: in the active layout `group`, or else in the first other+/// layout that has it, so `ctrl+c` still copies while a layout without Latin+/// letters is active.+fn plan_key(map: &str, group: u32, modifiers: &[String], name: &str) -> Result<Key> {+ let keymap = compile(map)?;+ let sym = [xkb::KEYSYM_NO_FLAGS, xkb::KEYSYM_CASE_INSENSITIVE]+ .into_iter()+ .map(|flags| xkb::keysym_from_name(name, flags))+ .find(|sym| sym.raw() != 0)+ .ok_or_else(|| refused(format!("{name:?} is not a key name")))?;+ let mut key = find(&keymap, group, sym)+ .or_else(|| {+ (0..keymap.num_layouts())+ .filter(|&other| other != group)+ .find_map(|other| find(&keymap, other, sym))+ })+ .ok_or_else(|| refused(format!("key {name:?} is in no layout of niri's keymap")))?;+ for modifier in modifiers {+ let name = match modifier.as_str() {+ "ctrl" => xkb::MOD_NAME_CTRL,+ "shift" => xkb::MOD_NAME_SHIFT,+ "alt" => xkb::MOD_NAME_ALT,+ "logo" => xkb::MOD_NAME_LOGO,+ other => return Err(refused(format!("unsupported modifier {other:?}"))),+ };+ key.modifiers |= 1_u32+ .checked_shl(keymap.mod_get_index(name))+ .ok_or_else(|| refused(format!("niri's keymap has no {name} modifier")))?;+ }+ Ok(key)+}++// ── Typing ───────────────────────────────────────────────────────────────────++/// Type `text` into the niri window that has keyboard focus now, which must+/// be `expected` when the caller activated one, stopping if focus leaves it.+/// A stop after some characters went out reports how many, without the+/// `foreground_unavailable` code that means nothing happened.+pub(in crate::wayland) fn type_text(expected: Option<u64>, text: &str) -> Result<()> {+ if text.is_empty() {+ return Ok(());+ }+ let (total, sent, typed, finished) =+ run_session(expected, text.chars().count(), |map, group| {+ plan_text(map, group, text)+ })?;+ outcome(sent, total, typed, finished)+}++/// Press the key `name` (an X keysym name) with `modifiers` (`ctrl`,+/// `shift`, `alt`, `logo`) held, into the niri window that has keyboard+/// focus, which must be `expected` when the caller activated one. The+/// modifiers are held through the keyboard's modifier state.+pub(in crate::wayland) fn press(+ expected: Option<u64>,+ modifiers: &[String],+ name: &str,+) -> Result<()> {+ let (_, sent, typed, finished) = run_session(expected, 1, |map, group| {+ Ok(vec![Part {+ keys: vec![plan_key(map, group, modifiers, name)?],+ extended: None,+ }])+ })?;+ match (typed, finished) {+ (Ok(()), Ok(())) => Ok(()),+ (Err(error), _) if sent.underway => Err(anyhow!("the key may have gone out: {error:#}")),+ (Err(error), _) => Err(error),+ (Ok(()), Err(error)) => Err(error.context("pressed the key, but could not finish")),+ }+}++/// How long before its tool call's deadline a session stops: time for the+/// keyboard's cleanup and the focus restore after it, so the call returns+/// the keyboard's report of how far it got rather than a bare timeout.+const REPORT_MARGIN: Duration = Duration::from_secs(2);++/// When a session sending `keys` keys from `now` stops, given the deadline+/// of the tool call it serves.+fn session_deadline(now: Instant, keys: usize, call: Option<Instant>) -> Instant {+ let own = now + call_budget(keys);+ call.map_or(own, |call| {+ own.min(call.checked_sub(REPORT_MARGIN).unwrap_or(call))+ })+}++/// One call's keyboard session: bind a keyboard to the window with keyboard+/// focus, send the keys `plan` makes from niri's keymap and active layout,+/// then always finish in the layout active by then: the user may have+/// switched layouts, or focus moved to a window with its own, during the+/// call. Returns the planned key count, how far the send got, its result+/// and the finish's.+fn run_session(+ expected: Option<u64>,+ keys: usize,+ plan: impl FnOnce(&str, u32) -> Result<Vec<Part>>,+) -> Result<(usize, Sent, Result<()>, Result<()>)> {+ let deadline = session_deadline(+ Instant::now(),+ keys,+ crate::input::action::current().map(|action| action.deadline()),+ );+ let focused = super::keyboard_focus().map_err(super::input_unavailable)?;+ let target = admitted_target(expected, focused)?;+ let group = super::keyboard_layout().map_err(super::input_unavailable)?;+ let mut keyboard = Keyboard::bind(deadline).map_err(|error| refused(format!("{error:#}")))?;+ let parts = plan(&keyboard.base.text, group)?;+ let total = parts.iter().map(|part| part.keys.len()).sum();+ let mut sent = Sent::default();+ let typed = type_parts(&mut keyboard, &parts, group, deadline, &mut sent, |sent| {+ let observed = super::keyboard_focus().map_err(super::input_unavailable)?;+ if crate::interaction::policy::focused(+ &target,+ &crate::interaction::facts::Fact::Known(observed),+ ) {+ return Ok(());+ }+ Err(focus_moved(target, observed, sent))+ });+ let finished = keyboard.finish(closing_group(super::keyboard_layout(), group));+ Ok((total, sent, typed, finished))+}++/// The layout group a call leaves the keyboard in: niri's active one at the+/// end, or the one it started in when niri cannot say.+fn closing_group(current: Result<u32>, start: u32) -> u32 {+ current.unwrap_or(start)+}++/// The window a call types into: the one with keyboard focus, which must be+/// `expected` when the caller activated a window, since another window may+/// have taken focus after the activation was confirmed.+fn admitted_target(expected: Option<u64>, focused: Option<u64>) -> Result<u64> {+ match (expected, focused) {+ (Some(expected), focused)+ if crate::interaction::policy::focused(+ &expected,+ &crate::interaction::facts::Fact::Known(focused),+ ) =>+ {+ Ok(expected)+ }+ (_, None) => Err(anyhow!(+ "foreground_unavailable: no niri window has keyboard focus (the screen may be \+ locked, or the overview, a layer-shell surface or a niri dialog has it); nothing \+ was typed"+ )),+ (Some(expected), Some(focused)) => Err(anyhow!(+ "foreground_unavailable: niri keyboard focus is on window {focused}, not window \+ {expected}; nothing was typed"+ )),+ (None, Some(focused)) => Ok(focused),+ }+}++/// Focus left `target` for `focused`. niri can move focus between its answer+/// and the next key, so once keys went out the last one may have reached the+/// window that has focus now; niri offers no check that is atomic with a key.+fn focus_moved(target: u64, focused: Option<u64>, sent: usize) -> anyhow::Error {+ let now = focused.map_or_else(|| "no window".to_owned(), |id| format!("window {id}"));+ if sent == 0 {+ return anyhow!(+ "foreground_unavailable: niri keyboard focus moved from window {target} to {now}"+ );+ }+ anyhow!(+ "foreground_unavailable: niri keyboard focus moved from window {target} to {now}; \+ the last key sent may have reached {now}"+ )+}++/// How far a call got: the keys niri confirmed, and whether one more was+/// sent but not confirmed, so may have gone out.+#[derive(Clone, Copy, Debug, Default)]+struct Sent {+ done: usize,+ underway: bool,+}++/// A stop before any key keeps its refusal; a stop after some keys went out,+/// or with one that may have, says how many, without the+/// `foreground_unavailable` code that would report no effect, as a partial+/// scroll does.+fn outcome(sent: Sent, total: usize, typed: Result<()>, finished: Result<()>) -> Result<()> {+ let done = sent.done;+ let typed = typed.map_err(|error| match done {+ _ if sent.underway => anyhow!(+ "type_text stopped after {done} of {total} characters, and character {} may have \+ gone out: {error:#}",+ done + 1+ ),+ 0 => error,+ _ => anyhow!("type_text stopped after {done} of {total} characters: {error:#}"),+ });+ match (typed, finished) {+ (typed, Ok(())) => typed,+ (Ok(()), Err(error)) => Err(error.context(format!(+ "typed all {total} characters, but could not put niri's keymap back"+ ))),+ (Err(typed), Err(error)) => Err(anyhow!(+ "{typed:#}; also could not put niri's keymap back: {error:#}"+ )),+ }+}++/// What sends keys: the virtual keyboard, or a recorder in tests.+trait Strokes {+ fn upload(&mut self, map: &str) -> Result<()>;+ /// Send `key` in its layout, then return to the active layout `group`.+ fn stroke(&mut self, key: Key, group: u32, deadline: Instant) -> Result<()>;+}++/// Send each key of `parts`, uploading a part's extended keymap before its+/// first key, once `still_focused` (told how many keys went out) admits the+/// key and the deadline has not passed. `sent` counts the keys that went out.+fn type_parts(+ keyboard: &mut impl Strokes,+ parts: &[Part],+ group: u32,+ deadline: Instant,+ sent: &mut Sent,+ mut still_focused: impl FnMut(usize) -> Result<()>,+) -> Result<()> {+ for part in parts {+ for (index, key) in part.keys.iter().enumerate() {+ if Instant::now() >= deadline {+ bail!("foreground_unavailable: niri text input ran out of time");+ }+ still_focused(sent.done)?;+ if let (0, Some(map)) = (index, &part.extended) {+ keyboard.upload(map)?;+ }+ sent.underway = true;+ keyboard.stroke(*key, group, deadline)?;+ sent.underway = false;+ sent.done += 1;+ }+ }+ Ok(())+}++/// A keymap niri sent: the file to upload it again from, and its text.+struct Keymap {+ file: File,+ size: u32,+ text: String,+}++#[derive(Default)]+struct State {+ seats: super::super::primary_seat::Seats<wl_seat::WlSeat>,+ manager: Option<ZwpVirtualKeyboardManagerV1>,+ keymap: Option<Keymap>,+ error: Option<String>,+}++/// A virtual keyboard on the attested niri, holding niri's keymap.+struct Keyboard {+ connection: Connection,+ queue: EventQueue<State>,+ state: State,+ device: ZwpVirtualKeyboardV1,+ /// The compositor's keymap, uploaded when the device was made.+ base: Keymap,+ /// Whether the device holds an extended keymap.+ extended: bool,+ pressed: Vec<u32>,+}++impl Keyboard {+ fn bind(deadline: Instant) -> Result<Self> {+ let socket = super::attested_wayland_socket()?;+ let connection =+ Connection::from_socket(socket).context("connect to niri's Wayland display")?;+ let mut queue = connection.new_event_queue::<State>();+ let handle = queue.handle();+ connection.display().get_registry(&handle, ());+ let mut state = State::default();+ super::super::bounded_roundtrip(&connection, &mut queue, &mut state, deadline)?;+ // The seats' names.+ super::super::bounded_roundtrip(&connection, &mut queue, &mut state, deadline)?;+ let seat = state.seats.selected().context("niri exposes no wl_seat")?;+ let manager = state+ .manager+ .clone()+ .context("niri exposes no zwp_virtual_keyboard_manager_v1")?;+ let wl_keyboard = seat.get_keyboard(&handle, ());+ super::super::bounded_roundtrip(&connection, &mut queue, &mut state, deadline)?;+ wl_keyboard.release();+ if let Some(error) = state.error.take() {+ bail!("{error}");+ }+ let base = state+ .keymap+ .take()+ .context("niri sent no keyboard keymap")?;+ let device = manager.create_virtual_keyboard(&seat, &handle, ());+ device.keymap(1, base.file.as_fd(), base.size);+ Ok(Self {+ connection,+ queue,+ state,+ device,+ base,+ extended: false,+ pressed: Vec::new(),+ })+ }++ /// Release anything a failed stroke may have left held, put niri's own+ /// keymap back for clients if the device sent another one, and destroy+ /// the device. Runs on its own short deadline, after the call's.+ fn finish(mut self, group: u32) -> Result<()> {+ let deadline = Instant::now() + crate::interaction::CLEANUP_CAP;+ for code in std::mem::take(&mut self.pressed) {+ self.device+ .key(super::super::event_time_ms(), code, KEY_RELEASED);+ }+ if self.extended {+ self.device+ .keymap(1, self.base.file.as_fd(), self.base.size);+ }+ self.device.modifiers(0, 0, 0, group);+ let restored = super::super::bounded_roundtrip(+ &self.connection,+ &mut self.queue,+ &mut self.state,+ deadline,+ );+ self.device.destroy();+ let flushed = self.queue.flush().context("flush to niri");+ restored.and(flushed)+ }+}++impl Strokes for Keyboard {+ /// Upload `map` for the following keys. niri sends it to clients with+ /// the next key.+ fn upload(&mut self, map: &str) -> Result<()> {+ let file = keymap_file(map)?;+ let size = u32::try_from(map.len() + 1)+ .ok()+ .filter(|size| *size <= MAX_KEYMAP)+ .context("the extended keymap is too large")?;+ self.device.keymap(1, file.as_fd(), size);+ self.extended = true;+ Ok(())+ }++ fn stroke(&mut self, key: Key, group: u32, deadline: Instant) -> Result<()> {+ // Nothing goes out once the tool call is over (see+ // `crate::input::action`).+ let (device, queue) = (&self.device, &self.queue);+ crate::input::action::admit(|| {+ device.modifiers(key.modifiers, 0, 0, key.group);+ device.key(super::super::event_time_ms(), key.code, KEY_PRESSED);+ device.key(super::super::event_time_ms(), key.code, KEY_RELEASED);+ device.modifiers(0, 0, 0, group);+ queue.flush()+ })?+ .context("flush to niri")?;+ self.pressed.push(key.code);+ super::super::bounded_roundtrip(+ &self.connection,+ &mut self.queue,+ &mut self.state,+ deadline,+ )?;+ self.pressed.clear();+ Ok(())+ }+}++fn keymap_file(map: &str) -> Result<File> {+ let name = std::ffi::CString::new("cua-driver-niri-keymap")?;+ // SAFETY: memfd_create with a valid C string; the fd is owned below.+ let fd = unsafe { libc::memfd_create(name.as_ptr(), libc::MFD_CLOEXEC) };+ if fd < 0 {+ return Err(std::io::Error::last_os_error().into());+ }+ // SAFETY: fd is a freshly created, owned memfd.+ let mut file = unsafe { <File as std::os::fd::FromRawFd>::from_raw_fd(fd) };+ file.write_all(map.as_bytes())?;+ file.write_all(&[0])?;+ file.seek(SeekFrom::Start(0))?;+ Ok(file)+}++fn read_keymap(fd: OwnedFd, size: u32) -> Result<Keymap, String> {+ if size == 0 || size > MAX_KEYMAP {+ return Err(format!(+ "niri's keymap size {size} is outside 1..={MAX_KEYMAP}"+ ));+ }+ let file = File::from(fd);+ let mut bytes = vec![0; size as usize];+ file.read_exact_at(&mut bytes, 0)+ .map_err(|error| format!("read niri's keymap: {error}"))?;+ if bytes.pop() != Some(0) {+ return Err("niri's keymap lacks its NUL terminator".into());+ }+ let text = String::from_utf8(bytes).map_err(|error| format!("niri's keymap: {error}"))?;+ Ok(Keymap { file, size, text })+}++impl Dispatch<wl_registry::WlRegistry, ()> for State {+ fn event(+ state: &mut Self,+ registry: &wl_registry::WlRegistry,+ event: wl_registry::Event,+ _: &(),+ _: &Connection,+ handle: &QueueHandle<Self>,+ ) {+ if let wl_registry::Event::Global {+ name,+ interface,+ version,+ } = event+ {+ if interface == wl_seat::WlSeat::interface().name {+ state+ .seats+ .add(registry.bind(name, version.min(7), handle, ()));+ } else if interface == ZwpVirtualKeyboardManagerV1::interface().name {+ state.manager = Some(registry.bind(name, 1, handle, ()));+ }+ }+ }+}++impl Dispatch<wl_seat::WlSeat, ()> for State {+ fn event(+ state: &mut Self,+ seat: &wl_seat::WlSeat,+ event: wl_seat::Event,+ _: &(),+ _: &Connection,+ _: &QueueHandle<Self>,+ ) {+ if let wl_seat::Event::Name { name } = event {+ state.seats.name(seat, name);+ }+ }+}++impl Dispatch<wl_keyboard::WlKeyboard, ()> for State {+ fn event(+ state: &mut Self,+ _: &wl_keyboard::WlKeyboard,+ event: wl_keyboard::Event,+ _: &(),+ _: &Connection,+ _: &QueueHandle<Self>,+ ) {+ if let wl_keyboard::Event::Keymap { format, fd, size } = event {+ if format != WEnum::Value(wl_keyboard::KeymapFormat::XkbV1) {+ state.error = Some("niri sent a keymap that is not xkb v1".into());+ return;+ }+ match read_keymap(fd, size) {+ Ok(keymap) => state.keymap = Some(keymap),+ Err(error) => state.error = Some(error),+ }+ }+ }+}++wayland_client::delegate_dispatch!(State: [wl_callback::WlCallback: super::super::Answered] => super::super::Answered);++wayland_client::delegate_noop!(State: ZwpVirtualKeyboardManagerV1);+wayland_client::delegate_noop!(State: ZwpVirtualKeyboardV1);++#[cfg(test)]+mod tests {+ use super::*;++ /// niri 26.04's keymap for `layout "us,de"`, as its `wl_keyboard` sends it.+ const US_DE: &str = include_str!("../../../tests/fixtures/niri-us-de.xkb");++ // Evdev codes `wtype` gives the first, 14th and 28th distinct characters+ // of a call, which Chromium reads as Escape, Backspace and Enter.+ const ESC: u32 = 1;+ const BACKSPACE: u32 = 14;+ const ENTER: u32 = 28;++ fn codes(group: u32, text: &str) -> Vec<u32> {+ let parts = plan_text(US_DE, group, text).unwrap();+ assert!(+ parts.iter().all(|part| part.extended.is_none()),+ "{text:?} is all in layout {group}"+ );+ parts+ .iter()+ .flat_map(|part| part.keys.iter().map(|key| key.code))+ .collect()+ }++ #[test]+ fn every_character_goes_out_on_its_own_key_in_the_layout() {+ // `wtype` would put `.` on code 14 (Backspace) here, deleting the m.+ let typed = codes(0, "abcdefghijklm.n");+ assert_eq!(+ typed,+ [30, 48, 46, 32, 18, 33, 34, 35, 23, 36, 37, 38, 50, 52, 49]+ );+ // A second call starting with a space: `wtype` puts it on code 1+ // (Escape), so Chromium drops it.+ let typed = codes(0, " a. b, c");+ assert_eq!(typed, [57, 30, 52, 57, 48, 51, 57, 46]);+ assert!(!typed.contains(&ESC) && !typed.contains(&BACKSPACE) && !typed.contains(&ENTER));+ }++ #[test]+ fn text_uses_the_active_layout_and_its_shift_level() {+ let shift = plan_text(US_DE, 0, "Z").unwrap()[0].keys[0].modifiers;+ assert_ne!(shift, 0);+ // German swaps Y and Z.+ assert_eq!(codes(0, "yz"), [21, 44]);+ assert_eq!(codes(1, "yz"), [44, 21]);+ assert_eq!(codes(1, "ä"), [40]);+ }++ #[test]+ fn line_feed_and_tab_are_the_return_and_tab_keys() {+ assert_eq!(codes(0, "a\nb\t"), [30, ENTER, 48, 15]);+ }++ #[test]+ fn a_control_character_refuses_before_anything_is_typed() {+ let error = plan_text(US_DE, 0, "ok\u{7}").unwrap_err();+ assert_eq!(+ crate::input::foreground::error_code(&error),+ Some(crate::input::foreground::CODE_UNAVAILABLE)+ );+ assert!(error.to_string().ends_with("nothing was typed"), "{error}");+ }++ #[test]+ fn a_layout_niri_does_not_have_refuses() {+ assert!(plan_text(US_DE, 2, "a").is_err());+ }++ #[test]+ fn a_symbol_the_layout_lacks_goes_on_a_spare_key_and_every_other_key_stays() {+ let original = compile(US_DE).unwrap();+ let [part] = <[Part; 1]>::try_from(plan_text(US_DE, 0, "a→😀").unwrap()).unwrap();+ let extended = compile(part.extended.as_deref().unwrap()).unwrap();+ assert_eq!(part.keys[0].code, 30);+ let state = xkb::State::new(&extended);+ for (key, c) in part.keys.iter().zip("a→😀".chars()).skip(1) {+ let code = xkb::Keycode::new(key.code + 8);+ assert!(evdev_in(code, &CHROMIUM_CODES), "{c} on code {}", key.code);+ assert_eq!(state.key_get_one_sym(code), text_sym(c).unwrap());+ }+ let used: Vec<xkb::Keycode> = part.keys[1..]+ .iter()+ .map(|key| xkb::Keycode::new(key.code + 8))+ .collect();+ assert!(unchanged(&original, &extended, &used));+ }++ #[test]+ fn a_symbol_only_on_a_key_chromium_drops_goes_on_a_spare_key() {+ // US has `EuroSign` only on <I443> (evdev 435): past Xwayland's last+ // keycode and without a DOM code, so Chromium dropped the €.+ let [part] = <[Part; 1]>::try_from(plan_text(US_DE, 0, "a€").unwrap()).unwrap();+ assert!(part.extended.is_some());+ let euro = xkb::Keycode::new(part.keys[1].code + 8);+ assert!(+ chromium_receives(part.keys[1].code),+ "€ on code {}",+ part.keys[1].code+ );+ let extended = compile(part.extended.as_deref().unwrap()).unwrap();+ assert_eq!(+ xkb::State::new(&extended).key_get_one_sym(euro),+ text_sym('€').unwrap()+ );+ // German has it on AltGr+E, which every client receives.+ let [part] = <[Part; 1]>::try_from(plan_text(US_DE, 1, "€").unwrap()).unwrap();+ assert!(part.extended.is_none());+ assert_eq!(part.keys[0].code, 18);+ }++ #[test]+ fn more_missing_symbols_than_spare_keys_take_several_keymaps_in_order() {+ let spare = spare_keys(&compile(US_DE).unwrap()).len();+ let text: String = (0..=spare as u32)+ .map(|i| char::from_u32(0x2190 + i).unwrap())+ .collect();+ let parts = plan_text(US_DE, 0, &text).unwrap();+ assert_eq!(parts.len(), 2);+ assert_eq!(+ parts.iter().map(|part| part.keys.len()).sum::<usize>(),+ spare + 1+ );+ assert!(parts.iter().all(|part| part.extended.is_some()));+ }++ fn ctrl() -> u32 {+ let keymap = compile(US_DE).unwrap();+ 1 << keymap.mod_get_index(xkb::MOD_NAME_CTRL)+ }++ #[test]+ fn a_chord_goes_out_on_the_layouts_key_with_its_modifier() {+ let copy = plan_key(US_DE, 0, &["ctrl".into()], "c").unwrap();+ assert_eq!((copy.code, copy.modifiers, copy.group), (46, ctrl(), 0));+ // German has Z where US has Y.+ let undo = plan_key(US_DE, 1, &["ctrl".into()], "z").unwrap();+ assert_eq!((undo.code, undo.group), (21, 1));+ let enter = plan_key(US_DE, 0, &[], "Return").unwrap();+ assert_eq!((enter.code, enter.modifiers), (ENTER, 0));+ }++ #[test]+ fn a_key_the_active_layout_lacks_goes_out_in_one_that_has_it() {+ // German has no plain `grave` keysym (only a dead key).+ let grave = plan_key(US_DE, 1, &[], "grave").unwrap();+ assert_eq!((grave.code, grave.group), (41, 0));+ }++ #[test]+ fn an_unknown_key_or_modifier_refuses() {+ for error in [+ plan_key(US_DE, 0, &[], "NoSuchKey").unwrap_err(),+ plan_key(US_DE, 0, &["hyper".into()], "a").unwrap_err(),+ ] {+ assert_eq!(+ crate::input::foreground::error_code(&error),+ Some(crate::input::foreground::CODE_UNAVAILABLE),+ "{error}"+ );+ }+ }++ #[derive(Default)]+ struct Recorder {+ events: Vec<String>,+ /// The stroke whose round trip fails, as when niri does not answer.+ fails: Option<usize>,+ }++ impl Strokes for Recorder {+ fn upload(&mut self, _: &str) -> Result<()> {+ self.events.push("keymap".into());+ Ok(())+ }+ fn stroke(&mut self, key: Key, _: u32, _: Instant) -> Result<()> {+ let index = self.events.len();+ self.events.push(key.code.to_string());+ anyhow::ensure!(+ self.fails != Some(index),+ "niri did not answer a Wayland round trip in time"+ );+ Ok(())+ }+ }++ fn later() -> Instant {+ Instant::now() + Duration::from_secs(60)+ }++ #[test]+ fn an_extended_keymap_goes_up_before_its_first_key() {+ let parts = plan_text(US_DE, 0, "a→").unwrap();+ let mut recorder = Recorder::default();+ let mut sent = Sent::default();+ type_parts(&mut recorder, &parts, 0, later(), &mut sent, |_| Ok(())).unwrap();+ assert_eq!(recorder.events[0], "keymap");+ assert_eq!(recorder.events.len(), 3);+ assert_eq!(sent.done, 2);+ }++ #[test]+ fn typing_stops_when_focus_leaves_and_reports_the_partial_effect() {+ let parts = plan_text(US_DE, 0, "hello").unwrap();+ let mut recorder = Recorder::default();+ let mut sent = Sent::default();+ let mut checks = 0;+ let typed = type_parts(&mut recorder, &parts, 0, later(), &mut sent, |_| {+ checks += 1;+ anyhow::ensure!(+ checks <= 2,+ "foreground_unavailable: niri keyboard focus moved from window 1 to window 2"+ );+ Ok(())+ });+ assert_eq!(recorder.events, ["35", "18"]);+ let stopped = outcome(sent, 5, typed, Ok(())).unwrap_err();+ assert_eq!(+ stopped.to_string(),+ "type_text stopped after 2 of 5 characters: foreground_unavailable: niri keyboard \+ focus moved from window 1 to window 2"+ );+ assert_eq!(crate::input::foreground::error_code(&stopped), None);++ // Focus gone before the first key: nothing went out.+ let mut sent = Sent::default();+ let typed = type_parts(&mut recorder, &parts, 0, later(), &mut sent, |_| {+ anyhow::bail!("foreground_unavailable: niri keyboard focus moved")+ });+ let refused = outcome(sent, 5, typed, Ok(())).unwrap_err();+ assert_eq!(+ crate::input::foreground::error_code(&refused),+ Some(crate::input::foreground::CODE_UNAVAILABLE)+ );+ }++ #[test]+ fn a_failed_keymap_restore_is_reported_with_the_typing_error() {+ let both = outcome(+ Sent {+ done: 2,+ underway: false,+ },+ 5,+ Err(anyhow!("foreground_timeout: deadline passed")),+ Err(anyhow!("keymap upload failed")),+ )+ .unwrap_err()+ .to_string();+ assert_eq!(+ both,+ "type_text stopped after 2 of 5 characters: foreground_timeout: deadline passed; \+ also could not put niri's keymap back: keymap upload failed"+ );+ }++ #[test]+ fn a_focus_move_after_some_keys_says_the_last_may_have_reached_the_new_window() {+ let before = focus_moved(3, Some(4), 0).to_string();+ assert!(!before.contains("may have reached"), "{before}");+ let after = outcome(+ Sent {+ done: 7,+ underway: false,+ },+ 400,+ Err(focus_moved(3, Some(4), 7)),+ Ok(()),+ )+ .unwrap_err();+ assert!(+ after+ .to_string()+ .ends_with("the last key sent may have reached window 4"),+ "{after}"+ );+ }++ #[test]+ fn a_window_scoped_call_types_only_into_the_window_it_activated() {+ assert_eq!(admitted_target(Some(5), Some(5)).unwrap(), 5);+ assert_eq!(admitted_target(None, Some(6)).unwrap(), 6);+ for (expected, focused) in [(Some(5), Some(6)), (Some(5), None), (None, None)] {+ let error = admitted_target(expected, focused).unwrap_err();+ assert_eq!(+ crate::input::foreground::error_code(&error),+ Some(crate::input::foreground::CODE_UNAVAILABLE),+ "{error}"+ );+ assert!(error.to_string().ends_with("nothing was typed"), "{error}");+ }+ assert_eq!(+ admitted_target(Some(5), Some(6)).unwrap_err().to_string(),+ "foreground_unavailable: niri keyboard focus is on window 6, not window 5; \+ nothing was typed"+ );+ }++ #[test]+ fn a_call_ends_in_the_layout_active_when_it_ends() {+ assert_eq!(closing_group(Ok(1), 0), 1);+ assert_eq!(closing_group(Err(anyhow!("niri did not answer")), 1), 1);+ }++ #[test]+ fn a_stroke_that_fails_after_sending_says_its_key_may_have_gone_out() {+ let parts = plan_text(US_DE, 0, "hello").unwrap();+ for (fails, message) in [+ (+ 2,+ "type_text stopped after 2 of 5 characters, and character 3 may have gone out: \+ niri did not answer a Wayland round trip in time",+ ),+ (+ 0,+ "type_text stopped after 0 of 5 characters, and character 1 may have gone out: \+ niri did not answer a Wayland round trip in time",+ ),+ ] {+ let mut recorder = Recorder {+ fails: Some(fails),+ ..Recorder::default()+ };+ let mut sent = Sent::default();+ let typed = type_parts(&mut recorder, &parts, 0, later(), &mut sent, |_| Ok(()));+ let stopped = outcome(sent, 5, typed, Ok(())).unwrap_err();+ assert_eq!(stopped.to_string(), message);+ assert_eq!(crate::input::foreground::error_code(&stopped), None);+ }+ }++ #[test]+ fn typing_stops_and_cleans_up_before_its_call_times_out() {+ // A call that waited for another's focus transaction has less time+ // left than the keyboard's own budget. The keyboard must stop, put+ // niri's keymap back and say how far it got before the call's+ // timeout replaces that with a bare foreground_timeout.+ let now = Instant::now();+ let call = now + Duration::from_secs(5);+ assert!(session_deadline(now, 1_000, Some(call)) + crate::interaction::CLEANUP_CAP < call);+ // With time to spare, the keyboard keeps its own budget.+ let far = now + Duration::from_secs(600);+ assert_eq!(session_deadline(now, 10, Some(far)), now + call_budget(10));+ assert_eq!(session_deadline(now, 10, None), now + call_budget(10));+ }++ #[test]+ fn typing_stops_at_the_deadline() {+ let parts = plan_text(US_DE, 0, "hello").unwrap();+ let mut recorder = Recorder::default();+ let mut sent = Sent::default();+ let typed = type_parts(&mut recorder, &parts, 0, Instant::now(), &mut sent, |_| {+ Ok(())+ });+ assert!(recorder.events.is_empty());+ assert!(typed.unwrap_err().to_string().contains("ran out of time"));+ }+}diff --git a/libs/cua-driver/rust/crates/platform-linux/src/wayland/niri/mod.rs b/libs/cua-driver/rust/crates/platform-linux/src/wayland/niri/mod.rsnew file mode 100644index 0000000000..a9b504f11f--- /dev/null+++ b/libs/cua-driver/rust/crates/platform-linux/src/wayland/niri/mod.rs@@ -0,0 +1,356 @@+//! niri identity, geometry and focus over niri's IPC socket.+//!+//! Window ids are niri's own window ids. Every IPC connection is attested:+//! the `NIRI_SOCKET` peer and the `WAYLAND_DISPLAY` peer must both be the+//! compositor process first recognised for this driver process. Once niri is+//! recognised, an IPC failure refuses instead of falling back to the generic+//! wlr-foreign-toplevel title join, which would lose the niri id namespace.+//!+//! niri 26.04 reports a position only for floating windows+//! (`tile_pos_in_workspace_view`): a tiled window's place depends on the+//! view's scroll position, which IPC does not report. Window capture and+//! window-coordinate pointer input therefore refuse a tiled window; the+//! desktop scope reaches it with pixels of the single output.+//!+//! Pointer input and window capture first wait, a few short reads at most,+//! for two consecutive reads that agree on the target, then check the layout+//! as niri reports it again during the action ([`PointerGate`]). Nothing keeps a+//! history of layout changes: a `get_desktop_state` screenshot is not proven+//! fresh when desktop-scope input uses its pixels, a move away and back+//! between two reads is not seen, and IPC positions omit niri's animation+//! offset, so a window still animating is checked at its destination.++use anyhow::{anyhow, bail, Context, Result};+use serde::de::DeserializeOwned;+use serde::Deserialize;+use std::collections::{BTreeMap, HashMap, HashSet};+use std::io::{self, Read, Write};+use std::os::fd::AsRawFd;+use std::os::unix::net::UnixStream;+use std::path::PathBuf;+use std::sync::OnceLock;+use std::time::{Duration, Instant};++use super::compositor_peer::{peer_pid, wayland_connection_with_timeout};+use crate::interaction::{self, settled};++mod capture;+mod focus;+mod gate;+mod ipc;+pub(super) mod keyboard;+mod scene;+#[cfg(test)]+mod test_support;++pub use self::ipc::is_session;+#[cfg(test)]+use self::test_support::*;+pub(crate) use self::{capture::*, focus::*, gate::*, ipc::*, scene::*};++// ── Observation ──────────────────────────────────────────────────────────────++pub fn list_windows() -> Result<Vec<Window>> {+ ipc().scene()?.windows()+}++pub fn window_for_id(id: u64) -> Result<Window> {+ ipc().scene()?.window(id)+}++/// The process's only niri window. A process with several windows needs an+/// exact window id; none is guessed.+pub fn window_for_pid(pid: u32) -> Option<Window> {+ let mut owned = list_windows()+ .ok()?+ .into_iter()+ .filter(|window| window.pid == Some(pid));+ let first = owned.next()?;+ owned.next().is_none().then_some(first)+}++/// The window a pid-only keyboard action means: the pid's on-screen window+/// focused most recently (the focused one first). This is focus recency,+/// not stacking, so it is used only to pick a keyboard target.+pub fn window_in_use(pid: u32) -> Option<u64> {+ pick_window_in_use(&list_windows().ok()?, pid)+}++fn pick_window_in_use(windows: &[Window], pid: u32) -> Option<u64> {+ windows+ .iter()+ .filter(|window| window.pid == Some(pid) && window.on_screen)+ .max_by_key(|window| window.focus_rank)+ .map(|window| window.id)+}++pub fn version() -> Result<String> {+ ipc().query(Request::Version)+}++/// Pointer space for `motion_absolute`: `(origin_x, origin_y, width, height)`.+/// Only outputs are queried: the gate rechecks this inside a double click.+pub fn pointer_space() -> Result<(i32, i32, u32, u32)> {+ pointer_space_of(&ipc().query(Request::Outputs)?)+}++/// The desktop frame (the single output's logical rectangle) and the pointer+/// layout, from one snapshot.+pub fn desktop_snapshot() -> Result<super::hyprland::DesktopSnapshot> {+ desktop_snapshot_in(&ipc().scene()?)+}++fn desktop_snapshot_in(scene: &Scene) -> Result<super::hyprland::DesktopSnapshot> {+ let (output, logical) = scene.single_output()?;+ Ok(super::hyprland::DesktopSnapshot {+ frame: super::hyprland::DesktopFrame {+ x: logical.x,+ y: logical.y,+ width: logical.width,+ height: logical.height,+ scale: logical.scale,+ outputs: vec![frame_output(output, logical)],+ },+ pointer: Some(scene.pointer_space()?),+ })+}++fn frame_output(output: &IpcOutput, logical: &IpcLogical) -> super::hyprland::FrameOutput {+ super::hyprland::FrameOutput {+ name: output.name.clone(),+ x: logical.x,+ y: logical.y,+ width: logical.width,+ height: logical.height,+ }+}++/// Every enabled output's logical rectangle, and their bounding box with the+/// largest scale, as Hyprland reports its frame. Desktop-scope input still+/// needs a single output; reporting sizes does not.+fn screen_frame_in(scene: &Scene) -> Result<super::hyprland::DesktopFrame> {+ let mut outputs = Vec::new();+ let mut scale: f64 = 0.0;+ for (output, logical) in scene.laid_out() {+ let logical = logical.checked()?;+ scale = scale.max(logical.scale);+ outputs.push(frame_output(output, logical));+ }+ let rects: Vec<Rect> = outputs+ .iter()+ .map(|output| Rect {+ x: output.x,+ y: output.y,+ width: output.width,+ height: output.height,+ })+ .collect();+ let (x, y, width, height) = bounding_box(&rects).context("niri has no enabled output")?;+ Ok(super::hyprland::DesktopFrame {+ x,+ y,+ width,+ height,+ scale,+ outputs,+ })+}++/// The output frame plus the monitor list `get_screen_size` reports, in the+/// shape the Hyprland adapter publishes.+pub fn screen_report() -> Result<(super::hyprland::DesktopFrame, serde_json::Value)> {+ let scene = ipc().scene()?;+ let frame = screen_frame_in(&scene)?;+ let monitors = frame+ .outputs+ .iter()+ .map(|output| {+ serde_json::json!({+ "name": output.name,+ "width": output.width,+ "height": output.height,+ "scale": scene.outputs[&output.name]+ .logical+ .as_ref()+ .map_or(frame.scale, |logical| logical.scale),+ "powered": true,+ "frame_x": output.x - frame.x,+ "frame_y": output.y - frame.y,+ })+ })+ .collect();+ Ok((frame, serde_json::Value::Array(monitors)))+}++/// wl_output transform of every enabled output, by name, for orienting a+/// capture of one of them.+pub fn output_transforms() -> Result<HashMap<String, u32>> {+ ipc()+ .scene()?+ .laid_out()+ .into_iter()+ .map(|(output, logical)| Ok((output.name.clone(), transform_code(&logical.transform)?)))+ .collect()+}++/// Doctor note: `None` without `NIRI_SOCKET`, otherwise whether this process+/// drives niri, its version and the single-output limit.+pub fn health_note() -> Option<String> {+ if std::env::var_os("NIRI_SOCKET").is_none_or(|socket| socket.is_empty()) {+ return None;+ }+ if !is_session() {+ return Some(inactive_health(+ probe(+ RECOGNIZED.get().copied(),+ crate::interaction::operation_deadline(QUERY_TIMEOUT),+ )+ .0,+ ));+ }+ if let Err(reason) = recognised() {+ return Some(format!(+ "niri: {reason:#}; niri window ids, focus and geometry refuse."+ ));+ }+ let version = version().unwrap_or_else(|error| format!("version unavailable: {error:#}"));+ Some(describe_health(+ &version,+ ipc().scene().map(|scene| scene.laid_out().len()),+ ))+}++fn inactive_health(attestation: Attestation) -> String {+ match attestation {+ Attestation::Unrelated(reason) | Attestation::Refused(reason) => format!(+ "niri: NIRI_SOCKET is not this session's compositor ({reason}); niri window ids, \+ focus and geometry are off."+ ),+ Attestation::Attested(_) => "niri: NIRI_SOCKET is this session's niri, but this process \+ does not use the niri adapter (inject mode or the native Wayland backend is off)."+ .to_owned(),+ }+}++fn describe_health(version: &str, outputs: Result<usize>) -> String {+ match outputs {+ Ok(1) => format!("niri {version}: IPC attested to WAYLAND_DISPLAY."),+ Ok(count) => format!(+ "niri {version}: IPC attested to WAYLAND_DISPLAY; window capture and desktop \+ input refuse because {count} outputs are enabled (one is supported)."+ ),+ Err(error) => format!(+ "niri {version}: IPC attested to WAYLAND_DISPLAY; the output layout is \+ unavailable: {error:#}."+ ),+ }+}++#[cfg(test)]+mod tests {+ use super::*;+ use serde_json::{json, Value};++ // ── Placement ───────────────────────────────────────────────────────────++ #[test]+ fn the_focused_then_most_recently_focused_window_ranks_highest() {+ // A main window focused at 805 s and its floating dialog at 804 s.+ let main = tiled(5, 1, (402.0, 448.0));+ let mut dialog = floating(7, 1, (100.0, 60.0), (300, 200));+ let rank = |windows: Vec<Value>, id| one_output(windows).window(id).unwrap().focus_rank;+ assert!(+ rank(vec![main.clone(), dialog.clone()], 5)+ > rank(vec![main.clone(), dialog.clone()], 7)+ );+ dialog["is_focused"] = json!(true);+ assert!(rank(vec![main.clone(), dialog.clone()], 7) > rank(vec![main, dialog], 5));+ }++ #[test]+ fn a_pid_only_keyboard_action_goes_to_the_window_in_use() {+ // A main window focused at 805 s and its focused floating dialog.+ let main = tiled(5, 1, (402.0, 448.0));+ let mut dialog = floating(7, 1, (100.0, 60.0), (300, 200));+ dialog["pid"] = json!(105);+ dialog["is_focused"] = json!(true);+ let mut scene = one_output(vec![main, dialog]);+ scene.workspaces[0].active_window_id = Some(5);+ let windows = scene.windows().unwrap();+ assert_eq!(pick_window_in_use(&windows, 105), Some(7));+ assert_eq!(pick_window_in_use(&windows, 999), None);+ }++ #[test]+ fn desktop_snapshot_uses_the_single_output_and_refuses_several() {+ let one = scene(+ vec![],+ vec![],+ vec![output("DP-2", 100, 50, (1280, 720), 1.5, "Normal")],+ false,+ );+ let snapshot = desktop_snapshot_in(&one).unwrap();+ assert_eq!(+ (+ snapshot.frame.x,+ snapshot.frame.y,+ snapshot.frame.width,+ snapshot.frame.height+ ),+ (100, 50, 853, 480)+ );+ assert_eq!(snapshot.frame.scale, 1.5);+ assert_eq!(snapshot.pointer, Some((100, 50, 854, 480)));+ let two = scene(+ vec![],+ vec![],+ vec![+ output("DP-1", 0, 0, (1920, 1080), 1.0, "Normal"),+ output("DP-2", 1920, 0, (1920, 1080), 1.0, "Normal"),+ ],+ false,+ );+ assert_eq!(+ desktop_snapshot_in(&two).unwrap_err().to_string(),+ "niri window capture and desktop input support a single output; this layout has 2"+ );+ }++ #[test]+ fn screen_size_spans_every_output_with_the_largest_scale() {+ let two = scene(+ vec![],+ vec![],+ vec![+ output("DP-1", 0, 0, (1920, 1080), 1.0, "Normal"),+ output("DP-2", 1920, 0, (2560, 1440), 2.0, "Normal"),+ ],+ false,+ );+ let frame = screen_frame_in(&two).unwrap();+ assert_eq!(+ (frame.x, frame.y, frame.width, frame.height, frame.scale),+ (0, 0, 3200, 1080, 2.0)+ );+ assert_eq!(frame.outputs.len(), 2);+ }++ #[test]+ fn health_note_states_attestation_and_the_single_output_limit() {+ assert_eq!(+ describe_health("26.04 (8ed0da4)", Ok(1)),+ "niri 26.04 (8ed0da4): IPC attested to WAYLAND_DISPLAY."+ );+ assert!(describe_health("26.04", Ok(2)).contains("refuse because 2 outputs are enabled"));+ assert_eq!(+ inactive_health(Attestation::Unrelated(+ "NIRI_SOCKET is served by pid 1 but WAYLAND_DISPLAY by pid 2".into()+ )),+ "niri: NIRI_SOCKET is not this session's compositor (NIRI_SOCKET is served by pid 1 \+ but WAYLAND_DISPLAY by pid 2); niri window ids, focus and geometry are off."+ );+ assert!(inactive_health(Attestation::Attested(1)).starts_with(+ "niri: NIRI_SOCKET is this session's niri, but this process does not use"+ ));+ }+}diff --git a/libs/cua-driver/rust/crates/platform-linux/src/wayland/niri/scene.rs b/libs/cua-driver/rust/crates/platform-linux/src/wayland/niri/scene.rsnew file mode 100644index 0000000000..02267af91e--- /dev/null+++ b/libs/cua-driver/rust/crates/platform-linux/src/wayland/niri/scene.rs@@ -0,0 +1,640 @@+//! Geometry: window placement in layout coordinates, built from one IPC+//! snapshot.++use super::*;++// ── Geometry ─────────────────────────────────────────────────────────────────++/// A rectangle in niri's global logical layout.+#[derive(Clone, Copy, Debug, PartialEq, Eq)]+pub struct Rect {+ pub x: i32,+ pub y: i32,+ pub width: u32,+ pub height: u32,+}++impl Rect {+ fn right(&self) -> i64 {+ i64::from(self.x) + i64::from(self.width)+ }++ fn bottom(&self) -> i64 {+ i64::from(self.y) + i64::from(self.height)+ }++ pub fn contains(&self, x: i32, y: i32) -> bool {+ i64::from(x) >= i64::from(self.x)+ && i64::from(y) >= i64::from(self.y)+ && i64::from(x) < self.right()+ && i64::from(y) < self.bottom()+ }++ pub(super) fn inside(&self, outer: &Rect) -> bool {+ self.x >= outer.x+ && self.y >= outer.y+ && self.right() <= outer.right()+ && self.bottom() <= outer.bottom()+ }++ pub(super) fn from_f64(x: f64, y: f64, width: f64, height: f64) -> Option<Self> {+ let (x, y, width, height) = (x.round(), y.round(), width.round(), height.round());+ let in_i32 = |v: f64| (f64::from(i32::MIN)..=f64::from(i32::MAX)).contains(&v);+ (in_i32(x) && in_i32(y) && width >= 1.0 && height >= 1.0 && in_i32(width) && in_i32(height))+ .then_some(Self {+ x: x as i32,+ y: y as i32,+ width: width as u32,+ height: height as u32,+ })+ .filter(|rect| valid_dimensions(rect.width, rect.height))+ }+}++/// What a tiled window's window capture and window-coordinate pointer input+/// refuse with: no pixel can be proven to be the window's.+const TILED_REFUSAL: &str = "niri does not report where a tiled window is; use get_desktop_state and a desktop-scope action (needs the display grant), or float the window";++/// Why niri reports no on-screen place for a window.+#[derive(Clone, Copy, Debug, PartialEq, Eq)]+pub enum Unplaced {+ /// In the scrolling layout: niri does not report where the view is.+ Tiled,+ /// On an inactive workspace, on no output, or the overview is open.+ NotShowing,+ /// A tiled window as large as the output, likely fullscreen, hides the+ /// workspace's floating windows.+ Covered,+ /// This niri release reports no window layout.+ NoLayout,+}++impl Unplaced {+ pub fn reason(self) -> &'static str {+ match self {+ Self::Tiled => TILED_REFUSAL,+ Self::NotShowing => {+ "the window is not showing: its workspace is not active on an output, or the niri overview is open"+ }+ Self::Covered => {+ "a tiled window as large as the output, likely fullscreen, covers the workspace's floating windows"+ }+ Self::NoLayout => "this niri release does not report window layout (niri 25.08 or later is required)",+ }+ }+}++/// A floating window's place on screen, from one compositor snapshot.+#[derive(Clone, Debug, PartialEq)]+pub struct Placed {+ /// The window's geometry in the global logical layout, rounded to whole+ /// logical pixels.+ pub rect: Rect,+ /// The exact logical origin niri reports: niri aligns windows to+ /// physical pixels, so at a fractional scale it is fractional.+ pub origin: (f64, f64),+ pub output: String,+ /// The output's logical rectangle: niri shows no part of the window+ /// outside it.+ pub area: Rect,+ /// The exact tiles of the workspace's other floating windows. IPC+ /// reports no stacking order, so any of them may be above this window.+ pub others: Vec<Exact>,+}++/// A rectangle in exact, possibly fractional, logical coordinates: at a+/// fractional scale niri aligns windows to physical pixels, and rounding to+/// whole logical pixels can hide a one-pixel overlap.+#[derive(Clone, Copy, Debug, PartialEq)]+pub struct Exact {+ pub x: f64,+ pub y: f64,+ pub width: f64,+ pub height: f64,+}++impl Exact {+ pub(super) fn contains(&self, x: f64, y: f64) -> bool {+ x >= self.x && y >= self.y && x < self.x + self.width && y < self.y + self.height+ }++ fn intersects(&self, other: &Exact) -> bool {+ self.x < other.x + other.width+ && other.x < self.x + self.width+ && self.y < other.y + other.height+ && other.y < self.y + self.height+ }+}++impl Placed {+ /// The window's exact geometry.+ pub fn exact(&self) -> Exact {+ Exact {+ x: self.origin.0,+ y: self.origin.1,+ width: f64::from(self.rect.width),+ height: f64::from(self.rect.height),+ }+ }++ pub fn overlapped(&self) -> bool {+ let exact = self.exact();+ self.others.iter().any(|other| other.intersects(&exact))+ }++ /// Whether a layout point lies inside the exact window geometry rather+ /// than only inside its rounded rectangle.+ pub fn covers(&self, x: i32, y: i32) -> bool {+ self.exact().contains(f64::from(x), f64::from(y))+ }++ /// Whether a layout point surely lands on this window.+ pub fn hits(&self, x: i32, y: i32) -> bool {+ self.covers(x, y)+ && !self+ .others+ .iter()+ .any(|other| other.contains(f64::from(x), f64::from(y)))+ }+}++#[derive(Clone, Debug, PartialEq)]+pub struct Window {+ pub id: u64,+ pub pid: Option<u32>,+ pub title: String,+ pub app_id: String,+ pub workspace: Option<u64>,+ pub focused: bool,+ /// Focus recency among all windows, higher is more recent and the+ /// focused window highest. niri reports no stacking order; this is the+ /// best available "which of a process's windows is in use".+ pub focus_rank: usize,+ /// niri shows it: a placed floating window, or the active tiled window of+ /// an active workspace while the overview is closed.+ pub on_screen: bool,+ /// The window's logical size, which niri reports for tiled windows too.+ /// Identity only: a tiled window has no position, so this is never input+ /// or capture geometry.+ pub size: Option<(u32, u32)>,+ pub placement: Result<Placed, Unplaced>,+}++pub(super) struct Scene {+ pub(super) windows: Vec<IpcWindow>,+ pub(super) workspaces: Vec<IpcWorkspace>,+ pub(super) outputs: BTreeMap<String, IpcOutput>,+ pub(super) overview_open: bool,+}++impl Scene {+ fn logical(&self, workspace: &IpcWorkspace) -> Option<&IpcLogical> {+ let output = self.outputs.get(workspace.output.as_ref()?)?;+ output.logical.as_ref()+ }++ fn workspace(&self, window: &IpcWindow) -> Option<&IpcWorkspace> {+ let id = window.workspace_id?;+ self.workspaces.iter().find(|workspace| workspace.id == id)+ }++ fn placement(&self, window: &IpcWindow) -> Result<Placed, Unplaced> {+ let layout = window.layout.as_ref().ok_or(Unplaced::NoLayout)?;+ if layout.pos_in_scrolling_layout.is_some() {+ return Err(Unplaced::Tiled);+ }+ if self.overview_open {+ return Err(Unplaced::NotShowing);+ }+ let workspace = self+ .workspace(window)+ .filter(|workspace| workspace.is_active)+ .ok_or(Unplaced::NotShowing)?;+ let logical = self.logical(workspace).ok_or(Unplaced::NotShowing)?;+ if self.covered(workspace, logical) {+ return Err(Unplaced::Covered);+ }+ let (tile_x, tile_y) = layout+ .tile_pos_in_workspace_view+ .ok_or(Unplaced::NotShowing)?;+ let (offset_x, offset_y) = layout.window_offset_in_tile;+ let (width, height) = layout.window_size;+ let origin = (+ f64::from(logical.x) + tile_x + offset_x,+ f64::from(logical.y) + tile_y + offset_y,+ );+ let rect = Rect::from_f64(origin.0, origin.1, f64::from(width), f64::from(height))+ .ok_or(Unplaced::NotShowing)?;+ let others = self+ .windows+ .iter()+ .filter(|other| other.id != window.id && other.workspace_id == window.workspace_id)+ .filter_map(|other| floating_tile(logical, other.layout.as_ref()?))+ .collect();+ Ok(Placed {+ rect,+ origin,+ output: workspace.output.clone().unwrap_or_default(),+ area: Rect {+ x: logical.x,+ y: logical.y,+ width: logical.width,+ height: logical.height,+ },+ others,+ })+ }++ /// niri hides a workspace's floating windows while its active column is+ /// fullscreen. IPC has no fullscreen flag, so an active tiled tile as+ /// large as the output (give or take a pixel) counts as covering them.+ /// This is refusal evidence only, never proof that a window is visible.+ fn covered(&self, workspace: &IpcWorkspace, logical: &IpcLogical) -> bool {+ self.windows+ .iter()+ .filter(|window| Some(window.id) == workspace.active_window_id)+ .filter_map(|window| window.layout.as_ref())+ .any(|layout| {+ layout.pos_in_scrolling_layout.is_some()+ && layout.tile_size.0 >= f64::from(logical.width) - 1.0+ && layout.tile_size.1 >= f64::from(logical.height) - 1.0+ })+ }++ fn on_screen(&self, window: &IpcWindow, placement: &Result<Placed, Unplaced>) -> bool {+ match placement {+ Ok(_) => true,+ Err(Unplaced::Tiled) => {+ !self.overview_open+ && self.workspace(window).is_some_and(|workspace| {+ workspace.is_active && workspace.active_window_id == Some(window.id)+ })+ }+ Err(_) => false,+ }+ }++ pub(super) fn windows(&self) -> Result<Vec<Window>> {+ let mut seen = HashSet::new();+ let mut windows = Vec::with_capacity(self.windows.len());+ for window in &self.windows {+ if !seen.insert(window.id) {+ bail!("duplicate niri window id {}", window.id);+ }+ let placement = self.placement(window);+ windows.push(Window {+ id: window.id,+ pid: window+ .pid+ .and_then(|pid| u32::try_from(pid).ok())+ .filter(|pid| *pid > 0),+ title: window.title.clone().unwrap_or_default(),+ app_id: window.app_id.clone().unwrap_or_default(),+ workspace: window.workspace_id,+ focused: window.is_focused,+ focus_rank: self+ .windows+ .iter()+ .filter(|other| {+ (other.is_focused, other.focus_timestamp)+ < (window.is_focused, window.focus_timestamp)+ })+ .count(),+ on_screen: self.on_screen(window, &placement),+ size: window.layout.as_ref().and_then(|layout| {+ let (width, height) = layout.window_size;+ Some((u32::try_from(width).ok()?, u32::try_from(height).ok()?))+ }),+ placement,+ });+ }+ Ok(windows)+ }++ pub(super) fn window(&self, id: u64) -> Result<Window> {+ self.windows()?+ .into_iter()+ .find(|window| window.id == id)+ .with_context(|| format!("niri window {id} no longer exists; refresh list_windows"))+ }++ /// Outputs niri lays out (enabled ones have a logical rectangle).+ pub(super) fn laid_out(&self) -> Vec<(&IpcOutput, &IpcLogical)> {+ self.outputs+ .values()+ .filter_map(|output| Some((output, output.logical.as_ref()?)))+ .collect()+ }++ /// Absolute virtual-pointer motion from an unbound pointer maps over the+ /// bounding box of every output's pointer rectangle.+ pub(super) fn pointer_space(&self) -> Result<(i32, i32, u32, u32)> {+ pointer_space_of(&self.outputs)+ }++ /// The one laid-out output. Display capture copies a single output, so+ /// output-pixel routes need a layout where that output is the only one.+ pub(super) fn single_output(&self) -> Result<(&IpcOutput, &IpcLogical)> {+ match self.laid_out().as_slice() {+ [(output, logical)] => Ok((*output, logical.checked()?)),+ [] => bail!("niri has no enabled output"),+ outputs => bail!(+ "niri window capture and desktop input support a single output; this layout has {}",+ outputs.len()+ ),+ }+ }+}++pub(super) fn pointer_space_of(+ outputs: &BTreeMap<String, IpcOutput>,+) -> Result<(i32, i32, u32, u32)> {+ let mut rects = Vec::new();+ for output in outputs.values() {+ rects.extend(output.pointer_rect()?);+ }+ bounding_box(&rects).context("niri has no enabled output")+}++pub(super) fn floating_tile(logical: &IpcLogical, layout: &IpcLayout) -> Option<Exact> {+ if layout.pos_in_scrolling_layout.is_some() {+ return None;+ }+ let (x, y) = layout.tile_pos_in_workspace_view?;+ let tile = Exact {+ x: f64::from(logical.x) + x,+ y: f64::from(logical.y) + y,+ width: layout.tile_size.0,+ height: layout.tile_size.1,+ };+ [tile.x, tile.y, tile.width, tile.height]+ .iter()+ .all(|v| v.is_finite())+ .then_some(tile)+}++/// The rectangle around `rects`, or `None` when it spans more than+/// `i32::MAX`: pointer motion converts the span to `i32`.+pub(super) fn bounding_box(rects: &[Rect]) -> Option<(i32, i32, u32, u32)> {+ let min_x = rects.iter().map(|r| r.x).min()?;+ let min_y = rects.iter().map(|r| r.y).min()?;+ let max_x = rects.iter().map(Rect::right).max()?;+ let max_y = rects.iter().map(Rect::bottom).max()?;+ let span = |len: i64| u32::try_from(i32::try_from(len).ok()?).ok();+ Some((+ min_x,+ min_y,+ span(max_x - i64::from(min_x))?,+ span(max_y - i64::from(min_y))?,+ ))+}++#[cfg(test)]+mod tests {+ use super::*;+ use serde_json::{json, Value};++ #[test]+ fn unknown_output_transform_refuses_pointer_space() {+ let scene = scene(+ vec![],+ vec![workspace(1, "winit", true, None)],+ vec![output("winit", 0, 0, (1280, 720), 1.5, "Diagonal")],+ false,+ );+ assert_eq!(+ scene.pointer_space().unwrap_err().to_string(),+ "unknown niri output transform \"Diagonal\""+ );+ }++ #[test]+ fn duplicate_window_ids_refuse_the_snapshot() {+ let scene = one_output(vec![+ floating(7, 1, (0.0, 0.0), (10, 10)),+ floating(7, 1, (50.0, 0.0), (10, 10)),+ ]);+ assert_eq!(+ scene.windows().unwrap_err().to_string(),+ "duplicate niri window id 7"+ );+ }++ #[test]+ fn floating_window_geometry_adds_output_origin_tile_and_offset() {+ let mut window = floating(7, 1, (100.25, 60.0), (300, 200));+ window["layout"]["window_offset_in_tile"] = json!([4.0, 4.0]);+ let scene = scene(+ vec![window],+ vec![workspace(1, "DP-2", true, None)],+ vec![output("DP-2", 1920, 0, (2560, 1440), 1.25, "Normal")],+ false,+ );+ assert_eq!(+ placement(&scene, 7).unwrap().rect,+ Rect {+ x: 2024,+ y: 64,+ width: 300,+ height: 200+ }+ );+ }++ #[test]+ fn tiled_windows_get_no_geometry() {+ let scene = one_output(vec![tiled(5, 1, (402.0, 448.0))]);+ assert_eq!(placement(&scene, 5), Err(Unplaced::Tiled));+ assert!(+ !scene.window(5).unwrap().on_screen,+ "an inactive tiled window is not proven on screen"+ );+ }++ #[test]+ fn active_tiled_window_is_on_screen_without_a_place() {+ let scene = scene(+ vec![tiled(5, 1, (402.0, 448.0))],+ vec![workspace(1, "winit", true, Some(5))],+ vec![output("winit", 0, 0, (1280, 720), 1.5, "Normal")],+ false,+ );+ let window = scene.window(5).unwrap();+ assert!(window.on_screen);+ assert_eq!(window.placement, Err(Unplaced::Tiled));+ // The size still identifies it, for AT-SPI root correlation.+ assert_eq!(window.size, Some((402, 448)));+ }++ #[test]+ fn floating_window_on_an_inactive_workspace_is_not_showing() {+ let scene = one_output(vec![floating(7, 2, (100.0, 60.0), (300, 200))]);+ assert_eq!(placement(&scene, 7), Err(Unplaced::NotShowing));+ assert!(!scene.window(7).unwrap().on_screen);+ }++ #[test]+ fn open_overview_hides_every_place() {+ let scene = scene(+ vec![floating(7, 1, (100.0, 60.0), (300, 200))],+ vec![workspace(1, "winit", true, None)],+ vec![output("winit", 0, 0, (1280, 720), 1.5, "Normal")],+ true,+ );+ assert_eq!(placement(&scene, 7), Err(Unplaced::NotShowing));+ }++ #[test]+ fn output_sized_active_tile_covers_floating_windows() {+ // Logical output 853x480; a fullscreen tile reports about that size.+ for (tile, covered) in [+ ((853.0, 480.0), true),+ ((852.4, 479.2), true),+ ((402.0, 448.0), false),+ ] {+ let scene = scene(+ vec![floating(7, 1, (100.0, 60.0), (300, 200)), tiled(5, 1, tile)],+ vec![workspace(1, "winit", true, Some(5))],+ vec![output("winit", 0, 0, (1280, 720), 1.5, "Normal")],+ false,+ );+ assert_eq!(+ placement(&scene, 7) == Err(Unplaced::Covered),+ covered,+ "{tile:?}"+ );+ }+ }++ #[test]+ fn layoutless_niri_refuses_geometry() {+ let mut window = floating(7, 1, (100.0, 60.0), (300, 200));+ window.as_object_mut().unwrap().remove("layout");+ assert_eq!(+ placement(&one_output(vec![window]), 7),+ Err(Unplaced::NoLayout)+ );+ }++ #[test]+ fn any_floating_overlap_is_ambiguous_whatever_the_focus_order() {+ let stamps = [+ (+ json!({"secs": 1, "nanos": 0}),+ json!({"secs": 2, "nanos": 0}),+ ),+ (+ json!({"secs": 2, "nanos": 0}),+ json!({"secs": 1, "nanos": 0}),+ ),+ (+ json!({"secs": 1, "nanos": 0}),+ json!({"secs": 1, "nanos": 0}),+ ),+ (Value::Null, Value::Null),+ ];+ for (target_stamp, other_stamp) in stamps {+ for target_focused in [false, true] {+ let mut target = floating(7, 1, (100.0, 60.0), (300, 200));+ target["focus_timestamp"] = target_stamp.clone();+ target["is_focused"] = json!(target_focused);+ // A dialog (child) over its parent looks the same in IPC.+ let mut other = floating(8, 1, (350.0, 200.0), (200, 100));+ other["focus_timestamp"] = other_stamp.clone();+ let placed = placement(&one_output(vec![target, other]), 7).unwrap();+ assert!(placed.overlapped());+ assert!(+ !placed.hits(360, 210),+ "a point under the other tile is ambiguous"+ );+ assert!(placed.hits(110, 70));+ }+ }+ }++ #[test]+ fn floating_windows_on_other_workspaces_do_not_overlap() {+ let placed = placement(+ &one_output(vec![+ floating(7, 1, (100.0, 60.0), (300, 200)),+ floating(8, 2, (100.0, 60.0), (300, 200)),+ tiled(5, 1, (402.0, 448.0)),+ ]),+ 7,+ )+ .unwrap();+ assert!(!placed.overlapped());+ }++ #[test]+ fn pointer_space_rounds_each_output_up_and_spans_all_outputs() {+ let single = |mode, scale| {+ scene(+ vec![],+ vec![],+ vec![output("winit", 0, 0, mode, scale, "Normal")],+ false,+ )+ .pointer_space()+ .unwrap()+ };+ assert_eq!(single((1280, 720), 1.0), (0, 0, 1280, 720));+ assert_eq!(single((1280, 720), 1.25), (0, 0, 1024, 576));+ assert_eq!(single((1280, 720), 1.5), (0, 0, 854, 480));+ assert_eq!(single((1280, 720), 2.0), (0, 0, 640, 360));+ let pair = scene(+ vec![],+ vec![],+ vec![+ output("eDP-1", 0, 0, (1280, 720), 1.5, "Normal"),+ output("DP-2", 853, -100, (1920, 1080), 1.0, "Normal"),+ ],+ false,+ );+ assert_eq!(pair.pointer_space().unwrap(), (0, -100, 2773, 1080));+ let rotated = scene(+ vec![],+ vec![],+ vec![output("DP-2", 0, 0, (1920, 1080), 1.0, "90")],+ false,+ );+ assert_eq!(rotated.pointer_space().unwrap(), (0, 0, 1080, 1920));+ }++ #[test]+ fn a_pointer_space_wider_than_i32_refuses() {+ // Pointer motion clamps into `width as i32 - 1`, which panics once+ // the span passes i32::MAX.+ let far = scene(+ vec![],+ vec![],+ vec![+ output("DP-1", i32::MIN, 0, (1920, 1080), 1.0, "Normal"),+ output("DP-2", i32::MAX - 1920, 0, (1920, 1080), 1.0, "Normal"),+ ],+ false,+ );+ assert!(far.pointer_space().is_err());+ let edge = Rect {+ x: 0,+ y: 0,+ width: i32::MAX as u32,+ height: 1,+ };+ assert_eq!(bounding_box(&[edge]), Some((0, 0, i32::MAX as u32, 1)));+ }++ #[test]+ fn disabled_outputs_are_left_out_of_pointer_space() {+ let mut off = output("HDMI-A-1", 0, 0, (1920, 1080), 1.0, "Normal");+ off["logical"] = Value::Null;+ off["current_mode"] = Value::Null;+ let scene = scene(+ vec![],+ vec![],+ vec![output("winit", 0, 0, (1280, 720), 1.5, "Normal"), off],+ false,+ );+ assert_eq!(scene.pointer_space().unwrap(), (0, 0, 854, 480));+ }+}diff --git a/libs/cua-driver/rust/crates/platform-linux/src/wayland/niri/test_support.rs b/libs/cua-driver/rust/crates/platform-linux/src/wayland/niri/test_support.rsnew file mode 100644index 0000000000..a7f547148f--- /dev/null+++ b/libs/cua-driver/rust/crates/platform-linux/src/wayland/niri/test_support.rs@@ -0,0 +1,299 @@+//! Fixtures shared by the niri tests.++use super::*;+use serde_json::{json, Value};+use std::sync::{Arc, Mutex};++// Shapes recorded from niri 26.04 (8ed0da4) in a nested session.+pub(super) fn floating(id: u64, workspace: u64, at: (f64, f64), size: (i32, i32)) -> Value {+ json!({"id": id, "title": format!("Float {id}"), "app_id": "float.a", "pid": 100 + id,+ "workspace_id": workspace, "is_focused": false, "is_floating": true, "is_urgent": false,+ "layout": {"pos_in_scrolling_layout": null, "tile_size": [f64::from(size.0), f64::from(size.1)],+ "window_size": [size.0, size.1], "tile_pos_in_workspace_view": [at.0, at.1],+ "window_offset_in_tile": [0.0, 0.0]},+ "focus_timestamp": {"secs": 804, "nanos": 995946831}})+}++pub(super) fn tiled(id: u64, workspace: u64, tile: (f64, f64)) -> Value {+ json!({"id": id, "title": "Tiled", "app_id": "tiled.b", "pid": 100 + id,+ "workspace_id": workspace, "is_focused": false, "is_floating": false, "is_urgent": false,+ "layout": {"pos_in_scrolling_layout": [1, 1], "tile_size": [tile.0, tile.1],+ "window_size": [tile.0 as i32, tile.1 as i32], "tile_pos_in_workspace_view": null,+ "window_offset_in_tile": [0.0, 0.0]},+ "focus_timestamp": {"secs": 805, "nanos": 828275861}})+}++pub(super) fn workspace(id: u64, output: &str, active: bool, active_window: Option<u64>) -> Value {+ json!({"id": id, "idx": id, "name": null, "output": output, "is_urgent": false,+ "is_active": active, "is_focused": active, "active_window_id": active_window})+}++pub(super) fn output(+ name: &str,+ x: i32,+ y: i32,+ mode: (u32, u32),+ scale: f64,+ transform: &str,+) -> Value {+ let width = (f64::from(mode.0) / scale) as u32;+ let height = (f64::from(mode.1) / scale) as u32;+ json!({"name": name, "make": "Smithay", "model": "Winit", "serial": null, "physical_size": null,+ "modes": [{"width": mode.0, "height": mode.1, "refresh_rate": 60000, "is_preferred": true}],+ "current_mode": 0, "is_custom_mode": true, "vrr_supported": false, "vrr_enabled": false,+ "logical": {"x": x, "y": y, "width": width, "height": height, "scale": scale, "transform": transform}})+}++pub(super) fn scene(+ windows: Vec<Value>,+ workspaces: Vec<Value>,+ outputs: Vec<Value>,+ overview: bool,+) -> Scene {+ Scene {+ windows: serde_json::from_value(Value::Array(windows)).unwrap(),+ workspaces: serde_json::from_value(Value::Array(workspaces)).unwrap(),+ outputs: outputs+ .into_iter()+ .map(|o| {+ (+ o["name"].as_str().unwrap().to_owned(),+ serde_json::from_value(o).unwrap(),+ )+ })+ .collect(),+ overview_open: overview,+ }+}++pub(super) fn one_output(windows: Vec<Value>) -> Scene {+ scene(+ windows,+ vec![+ workspace(1, "winit", true, None),+ workspace(2, "winit", false, None),+ ],+ vec![output("winit", 0, 0, (1280, 720), 1.5, "Flipped180")],+ false,+ )+}++pub(super) fn placement(scene: &Scene, id: u64) -> Result<Placed, Unplaced> {+ scene.window(id).unwrap().placement+}++// ── Fake niri ───────────────────────────────────────────────────────────++#[derive(Default)]+pub(super) struct Fake {+ pub(super) windows: Vec<Value>,+ pub(super) workspaces: Vec<Value>,+ pub(super) outputs: Vec<Value>,+ pub(super) overview: bool,+ /// FocusWindow is answered `Handled` but changes nothing.+ pub(super) ignore_focus: bool,+ /// FocusWindow on these ids changes nothing (a window that closed).+ pub(super) ignore_focus_on: Vec<u64>,+ /// FocusWindow on these ids switches their workspace, but keyboard+ /// focus stays elsewhere (a layer-shell surface took it).+ pub(super) focus_taken_on: Vec<u64>,+ /// After a FocusWorkspace, this many Workspaces replies still show+ /// the layout from before it (niri applies it asynchronously).+ pub(super) workspace_lag: usize,+ pub(super) stale: Option<(usize, Vec<Value>)>,+ pub(super) requests: Vec<String>,+ pub(super) connections: usize,+}++impl Fake {+ pub(super) fn reply(&mut self, request: &str) -> Value {+ self.requests.push(request.to_owned());+ let request: Value = serde_json::from_str(request).unwrap();+ match request.as_str() {+ Some("Windows") => json!({"Ok": {"Windows": self.windows}}),+ Some("Workspaces") => match &mut self.stale {+ Some((reads, stale)) if *reads > 0 => {+ *reads -= 1;+ json!({"Ok": {"Workspaces": stale}})+ }+ _ => json!({"Ok": {"Workspaces": self.workspaces}}),+ },+ Some("Outputs") => {+ let outputs: serde_json::Map<String, Value> = self+ .outputs+ .iter()+ .map(|o| (o["name"].as_str().unwrap().to_owned(), o.clone()))+ .collect();+ json!({"Ok": {"Outputs": outputs}})+ }+ Some("OverviewState") => {+ json!({"Ok": {"OverviewState": {"is_open": self.overview}}})+ }+ Some("FocusedWindow") => json!({"Ok": {"FocusedWindow":+ self.windows.iter().find(|w| w["is_focused"] == true)}}),+ _ => {+ let action = &request["Action"];+ if let Some(id) = action["FocusWindow"]["id"].as_u64() {+ if !self.ignore_focus && !self.ignore_focus_on.contains(&id) {+ let workspace = self+ .windows+ .iter()+ .find(|w| w["id"] == id)+ .map(|w| w["workspace_id"].clone());+ if let Some(workspace) = workspace.and_then(|w| w.as_u64()) {+ self.show_workspace(workspace, Some(id));+ }+ if self.focus_taken_on.contains(&id) {+ for window in &mut self.windows {+ window["is_focused"] = json!(false);+ }+ }+ }+ } else if let Some(id) = action["FocusWorkspace"]["reference"]["Id"].as_u64() {+ let active = self+ .workspaces+ .iter()+ .find(|w| w["id"] == id)+ .and_then(|w| w["active_window_id"].as_u64());+ if self.workspace_lag > 0 {+ self.stale = Some((self.workspace_lag, self.workspaces.clone()));+ }+ self.show_workspace(id, active);+ } else {+ return json!({"Err": "error parsing request"});+ }+ json!({"Ok": "Handled"})+ }+ }+ }++ /// niri's focus change: the workspace becomes active on its output+ /// and focused, and `window` its active, focused window.+ fn show_workspace(&mut self, id: u64, window: Option<u64>) {+ let output = self+ .workspaces+ .iter()+ .find(|w| w["id"] == id)+ .map(|w| w["output"].clone());+ for workspace in &mut self.workspaces {+ let this = workspace["id"] == id;+ if Some(&workspace["output"]) == output.as_ref() {+ workspace["is_active"] = json!(this);+ }+ workspace["is_focused"] = json!(this);+ if this && window.is_some() {+ workspace["active_window_id"] = json!(window);+ }+ }+ for w in &mut self.windows {+ w["is_focused"] = json!(window.is_some() && w["id"] == json!(window));+ }+ }++ pub(super) fn active(&self, id: u64) -> bool {+ self.workspaces+ .iter()+ .any(|w| w["id"] == id && w["is_active"] == true)+ }++ pub(super) fn focused(&self) -> Option<u64> {+ self.windows+ .iter()+ .find(|w| w["is_focused"] == true)+ .and_then(|w| w["id"].as_u64())+ }+}++pub(super) fn fake_ipc(fake: &Arc<Mutex<Fake>>) -> Ipc<impl FnMut(Instant) -> Result<UnixStream>> {+ let fake = fake.clone();+ Ipc {+ connect: move |_| {+ let (client, server) = UnixStream::pair()?;+ let fake = fake.clone();+ fake.lock().unwrap().connections += 1;+ std::thread::spawn(move || {+ let mut lines = Lines::new(server);+ while let Ok(Some(line)) = lines.next(Instant::now() + Duration::from_secs(2)) {+ let reply = fake+ .lock()+ .unwrap()+ .reply(std::str::from_utf8(&line).unwrap());+ if lines+ .stream+ .write_all(format!("{reply}\n").as_bytes())+ .is_err()+ {+ return;+ }+ }+ });+ Ok(client)+ },+ timeout: Duration::from_secs(1),+ lines: None,+ }+}++pub(super) fn ipc_with(+ connect: impl FnMut(Instant) -> Result<UnixStream>,+ timeout: Duration,+) -> Ipc<impl FnMut(Instant) -> Result<UnixStream>> {+ Ipc {+ connect,+ timeout,+ lines: None,+ }+}++pub(super) fn display(output: &str, size: (u32, u32)) -> Result<DisplayCapture> {+ oriented(output, 6, size)+}++pub(super) fn oriented(output: &str, transform: u32, size: (u32, u32)) -> Result<DisplayCapture> {+ Ok(DisplayCapture {+ png: vec![1],+ output: output.into(),+ transform,+ size,+ })+}++// ── Focus transactions ──────────────────────────────────────────────────++pub(super) fn focus_fixture() -> Arc<Mutex<Fake>> {+ let mut prior = tiled(5, 1, (402.0, 448.0));+ prior["is_focused"] = json!(true);+ Arc::new(Mutex::new(Fake {+ windows: vec![prior, floating(7, 2, (100.0, 60.0), (300, 200))],+ workspaces: vec![+ workspace(1, "winit", true, Some(5)),+ workspace(2, "winit", false, Some(7)),+ ],+ ..Fake::default()+ }))+}++// ── Tiled windows and desktop frames ────────────────────────────────────++/// Tiled window 5 (active) and 6 beside it, and floating window 7, on+/// workspace 1 of output winit placed at x=1920; workspace 2 hidden.+pub(super) fn tiled_scene(focused: Option<u64>) -> Scene {+ let mut windows = vec![+ tiled(5, 1, (402.0, 448.0)),+ tiled(6, 1, (402.0, 448.0)),+ floating(7, 1, (100.0, 60.0), (300, 200)),+ tiled(8, 2, (402.0, 448.0)),+ ];+ for window in &mut windows {+ window["is_focused"] = json!(Some(window["id"].as_u64().unwrap()) == focused);+ }+ scene(+ windows,+ vec![+ workspace(1, "winit", true, Some(5)),+ workspace(2, "winit", false, Some(8)),+ ],+ vec![output("winit", 1920, 0, (1280, 720), 1.5, "Flipped180")],+ false,+ )+}diff --git a/libs/cua-driver/rust/crates/platform-linux/src/wayland/persistent_vptr.rs b/libs/cua-driver/rust/crates/platform-linux/src/wayland/persistent_vptr.rsindex be594ba20f..7bf06f8eb5 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/wayland/persistent_vptr.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/wayland/persistent_vptr.rs@@ -28,29 +28,33 @@//! command on a fresh one, emitting a typed error for the in-flight call.use std::collections::{HashMap, HashSet};-use std::sync::OnceLock;+use std::sync::{Arc, OnceLock};use std::thread;use crossbeam_channel::{bounded, Receiver, Sender};use wayland_client::{protocol::wl_pointer::ButtonState, Connection};use wayland_protocols_wlr::virtual_pointer::v1::client::zwlr_virtual_pointer_v1::ZwlrVirtualPointerV1;-use super::{evdev_pointer_button, open_vptr_session};+use super::{evdev_pointer_button, open_vptr_session, AimedPoint};+use crate::input::action::Action;/// One in-flight command from the public API to the owner thread.enum Cmd {Press {cursor_id: String,window_id: u64,- x: i32,- y: i32,+ at: AimedPoint,button: u8,+ /// The tool call the press serves: bound on the owner thread, so a+ /// call that has ended presses nothing.+ action: Option<Arc<Action>>,reply: Sender<anyhow::Result<()>>,},MoveTo {cursor_id: String,x: i32,y: i32,+ action: Option<Arc<Action>>,reply: Sender<anyhow::Result<()>>,},Release {@@ -58,6 +62,12 @@ enum Cmd {button: u8,reply: Sender<anyhow::Result<()>>,},+ /// Release every button a cursor still holds, whether or not the tool+ /// recorded its hold. Replies whether the cursor held any.+ ReleaseUnrecorded {+ cursor_id: String,+ reply: Sender<anyhow::Result<bool>>,+ },/// Drop the entry for a cursor_id without sending wire events — used to/// recover when the compositor disconnected mid-life.Forget {@@ -66,9 +76,159 @@ enum Cmd {},}+/// What the owner thread does with each command. [`Pointers`] drives the+/// compositor's virtual pointers.+trait Owner {+ fn press(+ &mut self,+ cursor_id: &str,+ window_id: u64,+ at: AimedPoint,+ button: u8,+ ) -> anyhow::Result<()>;+ fn move_to(&mut self, cursor_id: &str, x: i32, y: i32) -> anyhow::Result<()>;+ fn release(&mut self, cursor_id: &str, button: u8) -> anyhow::Result<()>;+ fn release_unrecorded(&mut self, cursor_id: &str) -> anyhow::Result<bool>;+ fn forget(&mut self, cursor_id: &str);+ /// Whether `cursor_id` still holds a button.+ fn holds(&self, cursor_id: &str) -> bool;+}++/// The held cursors, by cursor_id.+#[derive(Default)]+struct Pointers(HashMap<String, ActivePointer>);++impl Owner for Pointers {+ fn press(+ &mut self,+ cursor_id: &str,+ window_id: u64,+ at: AimedPoint,+ button: u8,+ ) -> anyhow::Result<()> {+ // A press that went down but was never recorded (its release failed)+ // stays here until released; replacing it would leave it down for good.+ if self.0.contains_key(cursor_id) {+ anyhow::bail!(+ "cursor '{cursor_id}' still holds a button from an earlier call whose release niri did not confirm; call mouse_button_up first"+ );+ }+ // Open a fresh session for this press — this binds the seat, the foreign-+ // toplevel manager, activates the target window, and creates a new vptr.+ // Keep the (out_w, out_h) but drop the queue + state at end of scope; the+ // vptr itself remains alive (Wayland objects survive their original queue+ // as long as the Connection is alive).+ let mut sess = open_vptr_session(Some(window_id))?;+ let (w, h) = (sess.output_w, sess.output_h);+ let (px, py) = sess.abs(at.point.0, at.point.1);+ super::niri_pointer_gate(Some(window_id), None, &sess, &[at])?;+ let btn = evdev_pointer_button(button);+ sess.motion(px, py)?;+ // A press whose round trip fails is released when `sess` drops.+ sess.press(btn)?;++ // The vptr proxy stays valid while its connection lives: keep both for+ // the cursor's later commands.+ let (origin_x, origin_y) = (sess.origin_x, sess.origin_y);+ let (conn, vptr) = sess.keep();+ persist_conn(cursor_id, conn);++ self.0.insert(+ cursor_id.to_string(),+ ActivePointer {+ vptr,+ held: HashSet::from([btn]),+ out_w: w,+ out_h: h,+ origin_x,+ origin_y,+ },+ );+ Ok(())+ }++ fn move_to(&mut self, cursor_id: &str, x: i32, y: i32) -> anyhow::Result<()> {+ move_held(+ &mut self.0,+ cursor_id,+ (x, y),+ super::niri_pointer_space_now,+ roundtrip_on_persistent,+ )+ }++ fn release(&mut self, cursor_id: &str, button: u8) -> anyhow::Result<()> {+ release_held(+ &mut self.0,+ cursor_id,+ evdev_pointer_button(button),+ roundtrip_on_persistent,+ forget_conn,+ )+ }++ fn release_unrecorded(&mut self, cursor_id: &str) -> anyhow::Result<bool> {+ release_all_held(&mut self.0, cursor_id, roundtrip_on_persistent, forget_conn)+ }++ fn forget(&mut self, cursor_id: &str) {+ if let Some(p) = self.0.remove(cursor_id) {+ p.vptr.destroy();+ }+ }++ fn holds(&self, cursor_id: &str) -> bool {+ self.0.contains_key(cursor_id)+ }+}++/// The device a held cursor drives: the virtual pointer, or a recording+/// fake in tests. Each call sends one event and its frame.+trait HeldDevice {+ fn motion(&self, x: u32, y: u32, extent: (u32, u32));+ fn button(&self, button: u32, state: ButtonState);+ fn destroy(&self);+}++impl HeldDevice for ZwlrVirtualPointerV1 {+ fn motion(&self, x: u32, y: u32, (w, h): (u32, u32)) {+ self.motion_absolute(0, x, y, w, h);+ self.frame();+ }++ fn button(&self, button: u32, state: ButtonState) {+ ZwlrVirtualPointerV1::button(self, 0, button, state);+ self.frame();+ }++ fn destroy(&self) {+ ZwlrVirtualPointerV1::destroy(self);+ }+}++/// A held cursor's motion refused because the pointer space it scales by+/// changed, or could not be read. The owner sent the release of its buttons+/// where the pointer was, so the app may have seen a drop, and discarded the+/// cursor: it holds nothing any more, whether or not niri confirmed the+/// release.+#[derive(Debug)]+pub struct HeldPointerEnded {+ detail: String,+ /// niri answered the round trip after the release.+ pub release_confirmed: bool,+}++impl std::fmt::Display for HeldPointerEnded {+ fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {+ f.write_str(&self.detail)+ }+}++impl std::error::Error for HeldPointerEnded {}+/// State held inside the owner thread for one cursor_id.-struct ActivePointer {- vptr: ZwlrVirtualPointerV1,+struct ActivePointer<D = ZwlrVirtualPointerV1> {+ vptr: D,/// evdev codes of buttons currently held down. When this set becomes/// empty the vptr is destroyed and the entry dropped from the map.held: HashSet<u32>,@@ -87,117 +247,185 @@ static TX: OnceLock<Sender<Cmd>> = OnceLock::new();fn tx() -> &'static Sender<Cmd> {TX.get_or_init(|| {let (tx, rx) = bounded::<Cmd>(32);+ let held = held_cursors().clone();thread::Builder::new().name("cua-persistent-vptr".into())- .spawn(move || owner_thread(rx))+ .spawn(move || owner_thread(rx, Pointers::default(), held)).expect("spawn cua-persistent-vptr thread");tx})}-fn owner_thread(rx: Receiver<Cmd>) {- let mut active: HashMap<String, ActivePointer> = HashMap::new();+fn held_cursors() -> &'static Arc<HeldCursors> {+ static HELD: OnceLock<Arc<HeldCursors>> = OnceLock::new();+ HELD.get_or_init(Default::default)+}++fn owner_thread(rx: Receiver<Cmd>, mut owner: impl Owner, held: Arc<HeldCursors>) {while let Ok(cmd) = rx.recv() {+ // Each command publishes whether its cursor still holds a button+ // before it replies, so a caller that got its reply sees it.match cmd {Cmd::Press {cursor_id,window_id,- x,- y,+ at,button,+ action,reply,} => {- let r = handle_press(&mut active, &cursor_id, window_id, x, y, button);- let _ = reply.send(r);+ let _bound = action.map(crate::input::action::bind);+ let pressed = owner.press(&cursor_id, window_id, at, button);+ held.publish(&cursor_id, &owner, true);+ let _ = reply.send(pressed);}Cmd::MoveTo {cursor_id,x,y,+ action,reply,} => {- let r = handle_move(&mut active, &cursor_id, x, y);- let _ = reply.send(r);+ let _bound = action.map(crate::input::action::bind);+ let moved = owner.move_to(&cursor_id, x, y);+ held.publish(&cursor_id, &owner, false);+ let _ = reply.send(moved);}+ // Releases are cleanup: they go out also once the call is over.Cmd::Release {cursor_id,button,reply,} => {- let r = handle_release(&mut active, &cursor_id, button);- let _ = reply.send(r);+ let released = owner.release(&cursor_id, button);+ held.publish(&cursor_id, &owner, false);+ let _ = reply.send(released);+ }+ Cmd::ReleaseUnrecorded { cursor_id, reply } => {+ let released = owner.release_unrecorded(&cursor_id);+ held.publish(&cursor_id, &owner, false);+ let _ = reply.send(released);}Cmd::Forget { cursor_id, reply } => {- if let Some(p) = active.remove(&cursor_id) {- p.vptr.destroy();- }+ owner.forget(&cursor_id);+ held.publish(&cursor_id, &owner, false);let _ = reply.send(Ok(()));}}}}-fn handle_press(- active: &mut HashMap<String, ActivePointer>,- cursor_id: &str,- window_id: u64,- x: i32,- y: i32,- button: u8,-) -> anyhow::Result<()> {- // Open a fresh session for this press — this binds the seat, the foreign-- // toplevel manager, activates the target window, and creates a new vptr.- // Keep the (out_w, out_h) but drop the queue + state at end of scope; the- // vptr itself remains alive (Wayland objects survive their original queue- // as long as the Connection is alive).- let mut sess = open_vptr_session(Some(window_id))?;- let (w, h) = (sess.output_w, sess.output_h);- let (px, py) = sess.abs(x, y);- let btn = evdev_pointer_button(button);-- sess.vptr.motion_absolute(0, px, py, w, h);- sess.vptr.frame();- sess.vptr.button(0, btn, ButtonState::Pressed);- sess.vptr.frame();- sess.queue.roundtrip(&mut sess.state)?;-- // Take ownership of the vptr handle by extracting it from the session.- // ZwlrVirtualPointerV1 is a Wayland proxy — cloning it gives another- // handle to the same wire object; destroying it sends the destructor.- let vptr = sess.vptr.clone();- // Persist the live connection so the proxy stays valid after this fn returns- // (the session goes out of scope; we need the conn alive).- // We do this by leaking the connection into a process-static slot keyed by- // cursor_id. Subsequent commands on the same cursor reuse this conn.- persist_conn(cursor_id, sess.conn);-- let mut held = HashSet::new();- held.insert(btn);- active.insert(- cursor_id.to_string(),- ActivePointer {- vptr,- held,- out_w: w,- out_h: h,- origin_x: sess.origin_x,- origin_y: sess.origin_y,- },- );- Ok(())+/// The cursors that may hold a button, readable without waiting on the+/// owner thread: those it holds one for, and those with a press queued or+/// under way, which may send input at any moment.+#[derive(Default)]+struct HeldCursors(std::sync::Mutex<Cursors>);++#[derive(Default)]+struct Cursors {+ held: HashSet<String>,+ /// Presses queued and not yet handled, by cursor.+ pressing: HashMap<String, usize>,}-fn handle_move(- active: &mut HashMap<String, ActivePointer>,+impl Cursors {+ fn press_dropped(&mut self, cursor_id: &str) {+ if let Some(count) = self.pressing.get_mut(cursor_id) {+ *count -= 1;+ if *count == 0 {+ self.pressing.remove(cursor_id);+ }+ }+ }+}++impl HeldCursors {+ fn lock(&self) -> std::sync::MutexGuard<'_, Cursors> {+ self.0.lock().unwrap_or_else(|e| e.into_inner())+ }++ /// A press for `cursor_id` is about to be queued.+ fn press_queued(&self, cursor_id: &str) {+ *self+ .lock()+ .pressing+ .entry(cursor_id.to_owned())+ .or_default() += 1;+ }++ /// The owner handled a command for `cursor_id`, a press when `pressed`.+ fn publish(&self, cursor_id: &str, owner: &dyn Owner, pressed: bool) {+ let holds = owner.holds(cursor_id);+ let mut cursors = self.lock();+ if pressed {+ cursors.press_dropped(cursor_id);+ }+ if holds {+ cursors.held.insert(cursor_id.to_owned());+ } else {+ cursors.held.remove(cursor_id);+ }+ }++ /// A queued press for `cursor_id` is no longer pending.+ fn press_dropped(&self, cursor_id: &str) {+ self.lock().press_dropped(cursor_id);+ }++ fn contains(&self, cursor_id: &str) -> bool {+ let cursors = self.lock();+ cursors.held.contains(cursor_id) || cursors.pressing.contains_key(cursor_id)+ }+}++/// Move a held cursor. On niri the pointer space is read again first: the+/// motion is scaled by the extent saved at the press, and niri sends no+/// event when an output is added or rescaled. If it changed, no motion goes+/// out; the held buttons are released and the cursor is dropped.+fn move_held<D: HeldDevice>(+ active: &mut HashMap<String, ActivePointer<D>>,cursor_id: &str,- x: i32,- y: i32,+ (x, y): (i32, i32),+ space_now: impl FnOnce() -> anyhow::Result<Option<(i32, i32, u32, u32)>>,+ mut roundtrip: impl FnMut(&str) -> anyhow::Result<()>,) -> anyhow::Result<()> {let entry = active.get_mut(cursor_id).ok_or_else(|| {anyhow::anyhow!("no held mouse button for cursor '{cursor_id}'; call mouse_button_down first")})?;+ let saved = (entry.origin_x, entry.origin_y, entry.out_w, entry.out_h);+ if let Some(refusal) = space_now()+ .and_then(|now| now.map_or(Ok(()), |now| super::pointer_space_holds(saved, now)))+ .err()+ {+ let entry = active.remove(cursor_id).expect("found above");+ for button in &entry.held {+ entry.vptr.button(*button, ButtonState::Released);+ }+ entry.vptr.destroy();+ let released = roundtrip(cursor_id);+ forget_conn(cursor_id);+ let refusal = refusal+ .to_string()+ .trim_start_matches("foreground_unavailable: ")+ .to_owned();+ return Err(match released {+ Ok(()) => HeldPointerEnded {+ detail: format!(+ "{refusal}; no motion was sent, and the held button was released where the pointer was, which the app may take as a drop"+ ),+ release_confirmed: true,+ },+ Err(error) => HeldPointerEnded {+ detail: format!(+ "{refusal}; no motion was sent, and the held button's release was sent but not confirmed ({error:#}); the cursor no longer holds it"+ ),+ release_confirmed: false,+ },+ }+ .into());+ }let (px, py) = super::pointer_abs(entry.origin_x,entry.origin_y,@@ -206,49 +434,126 @@ fn handle_move(x,y,);- entry+ crate::input::action::admit(|| entry.vptr.motion(px, py, (entry.out_w, entry.out_h)))?;+ roundtrip(cursor_id)+}++/// Release `btn` on a held cursor, dropping the cursor once it holds nothing.+/// If niri only did not answer in time, the cursor stays for another try.+/// If the connection failed otherwise, the compositor has already dropped+/// the virtual pointer and its buttons, so the cursor is dropped too.+fn release_held<D: HeldDevice>(+ active: &mut HashMap<String, ActivePointer<D>>,+ cursor_id: &str,+ btn: u32,+ mut roundtrip: impl FnMut(&str) -> anyhow::Result<()>,+ forget: impl FnOnce(&str),+) -> anyhow::Result<()> {+ active+ .get(cursor_id)+ .ok_or_else(|| anyhow::anyhow!("no held mouse button for cursor '{cursor_id}'"))?.vptr- .motion_absolute(0, px, py, entry.out_w, entry.out_h);- entry.vptr.frame();- roundtrip_on_persistent(cursor_id)?;+ .button(btn, ButtonState::Released);+ match roundtrip(cursor_id) {+ Err(error) if answers_later(&error) => return Err(error),+ Err(_) => {+ drop_cursor(active, cursor_id, forget);+ return Ok(());+ }+ Ok(()) => {}+ }+ let entry = active.get_mut(cursor_id).expect("found above");+ entry.held.remove(&btn);+ if entry.held.is_empty() {+ let entry = active.remove(cursor_id).expect("found above");+ entry.vptr.destroy();+ roundtrip(cursor_id).ok();+ forget(cursor_id);+ }Ok(())}-fn handle_release(- active: &mut HashMap<String, ActivePointer>,+/// Release every button `cursor_id` holds and drop the cursor. If niri only+/// did not answer in time, the cursor stays for another try; a connection+/// that failed otherwise took the virtual pointer and its buttons with it.+fn release_all_held<D: HeldDevice>(+ active: &mut HashMap<String, ActivePointer<D>>,cursor_id: &str,- button: u8,-) -> anyhow::Result<()> {- let btn = evdev_pointer_button(button);- let drop_entry = {- let entry = active- .get_mut(cursor_id)- .ok_or_else(|| anyhow::anyhow!("no held mouse button for cursor '{cursor_id}'"))?;- entry.vptr.button(0, btn, ButtonState::Released);- entry.vptr.frame();- roundtrip_on_persistent(cursor_id)?;- entry.held.remove(&btn);- entry.held.is_empty()+ mut roundtrip: impl FnMut(&str) -> anyhow::Result<()>,+ forget: impl FnOnce(&str),+) -> anyhow::Result<bool> {+ let Some(entry) = active.get(cursor_id) else {+ return Ok(false);};- if drop_entry {- if let Some(p) = active.remove(cursor_id) {- p.vptr.destroy();- roundtrip_on_persistent(cursor_id).ok();+ for button in &entry.held {+ entry.vptr.button(*button, ButtonState::Released);+ }+ match roundtrip(cursor_id) {+ Err(error) if answers_later(&error) => Err(error),+ Err(_) => {+ drop_cursor(active, cursor_id, forget);+ Ok(true)+ }+ Ok(()) => {+ let entry = active.remove(cursor_id).expect("found above");+ entry.vptr.destroy();+ roundtrip(cursor_id).ok();+ forget(cursor_id);+ Ok(true)}- forget_conn(cursor_id);}- Ok(())}-// Process-static slots for Connection + EventQueue keyed by cursor_id. The-// EventQueue is !Send but we only touch these on the owner thread, so wrap-// in a thread-local-by-construction pattern: store inside the same map so-// the owner thread is the sole accessor.-//-// We use a per-thread static rather than a Mutex<HashMap> because the owner-// thread is the only accessor (no contention possible).+/// Drop a cursor whose connection is gone, as `forget` does, and its+/// connection with it.+fn drop_cursor<D: HeldDevice>(+ active: &mut HashMap<String, ActivePointer<D>>,+ cursor_id: &str,+ forget: impl FnOnce(&str),+) {+ if let Some(entry) = active.remove(cursor_id) {+ entry.vptr.destroy();+ }+ forget(cursor_id);+}++/// A held cursor's failed round trip. `timed_out` when niri only did not+/// answer in time, so the connection may still work.+#[derive(Debug)]+struct RoundTripFailed {+ detail: String,+ timed_out: bool,+}++impl std::fmt::Display for RoundTripFailed {+ fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {+ f.write_str(&self.detail)+ }+}++impl std::error::Error for RoundTripFailed {}++fn round_trip_failed(error: anyhow::Error) -> anyhow::Error {+ RoundTripFailed {+ timed_out: error.is::<super::RoundTripTimedOut>(),+ detail: format!("compositor roundtrip failed: {error:#}"),+ }+ .into()+}++/// Whether a failed round trip left a connection that may still answer.+fn answers_later(error: &anyhow::Error) -> bool {+ error+ .downcast_ref::<RoundTripFailed>()+ .is_some_and(|failed| failed.timed_out)+}++/// A held cursor's connection and its queue, kept thread-local because the+/// queue is `!Send` and only the owner thread touches it.+type Persisted = (Connection, wayland_client::EventQueue<super::State>);+thread_local! {- static CONNS: std::cell::RefCell<HashMap<String, (Connection, wayland_client::EventQueue<super::State>)>>+ static CONNS: std::cell::RefCell<HashMap<String, Persisted>>= std::cell::RefCell::new(HashMap::new());}@@ -268,66 +573,275 @@ fn forget_conn(cursor_id: &str) {fn roundtrip_on_persistent(cursor_id: &str) -> anyhow::Result<()> {CONNS.with(|c| {let mut b = c.borrow_mut();- let (_conn, queue) = b+ let (conn, queue) = b.get_mut(cursor_id).ok_or_else(|| anyhow::anyhow!("no persistent connection for cursor '{cursor_id}'"))?;let mut tmp = super::State::default();- queue- .roundtrip(&mut tmp)- .map_err(|e| anyhow::anyhow!("compositor roundtrip failed: {e}"))?;- Ok(())+ let deadline = std::time::Instant::now() + super::NIRI_ROUNDTRIP_TIMEOUT;+ super::sync_by(conn, queue, &mut tmp, Some(deadline)).map_err(round_trip_failed)})}// ── public API ────────────────────────────────────────────────────────────+/// Send one command to the owner thread on `tx` and wait for its reply, until+/// the deadline of the action bound to this thread, or for+/// [`SESSION_END_WAIT`] without one, queueing included.+fn request<T>(+ tx: &Sender<Cmd>,+ cmd: impl FnOnce(Sender<anyhow::Result<T>>) -> Cmd,+) -> anyhow::Result<T> {+ let deadline = crate::input::action::current().map_or_else(+ || std::time::Instant::now() + SESSION_END_WAIT,+ |action| action.deadline(),+ );+ request_by(tx, deadline, cmd)?+}++fn request_by<T>(+ tx: &Sender<Cmd>,+ deadline: std::time::Instant,+ cmd: impl FnOnce(Sender<anyhow::Result<T>>) -> Cmd,+) -> anyhow::Result<anyhow::Result<T>> {+ let (tx_r, rx_r) = bounded(1);+ tx.send_deadline(cmd(tx_r), deadline)+ .map_err(|error| match error {+ crossbeam_channel::SendTimeoutError::Timeout(_) => {+ anyhow::anyhow!("cua-persistent-vptr owner is busy: its queue stayed full")+ }+ crossbeam_channel::SendTimeoutError::Disconnected(_) => {+ anyhow::anyhow!("cua-persistent-vptr thread is dead")+ }+ })?;+ rx_r.recv_deadline(deadline)+ .map_err(|e| anyhow::Error::new(e).context("cua-persistent-vptr owner did not answer"))+}+/// Press and HOLD `button` (evdev code) at output coordinates `(x, y)` on the/// toplevel identified by `window_id`. Subsequent `move_to` / `release` calls/// targeting the same `cursor_id` reuse the same virtual-pointer device, so/// the compositor treats the sequence as one logical drag rather than as/// independent clicks. Errors if `cursor_id` already has a held button.-pub fn press(cursor_id: &str, window_id: u64, x: i32, y: i32, button: u8) -> anyhow::Result<()> {- let (tx_r, rx_r) = bounded(1);- tx().send(Cmd::Press {+///+/// The press serves the tool call bound to this thread: a call that has+/// ended presses nothing. Once the button is down, `record` notes the hold+/// for the caller, under that call; if the call ended meanwhile, or `record`+/// panics, the button is released instead, so no press stays down that+/// nobody recorded. If that release fails, the hold is recorded anyway, so+/// `mouse_button_up` can release it; when `record` itself panicked, the+/// cursor stays with the owner for [`release_unrecorded`].+pub fn press(+ cursor_id: &str,+ window_id: u64,+ at: AimedPoint,+ button: u8,+ record: impl FnOnce(),+) -> anyhow::Result<()> {+ press_on(+ tx(),+ held_cursors(),+ cursor_id,+ window_id,+ at,+ button,+ record,+ )+}++fn press_on(+ tx: &Sender<Cmd>,+ held: &HeldCursors,+ cursor_id: &str,+ window_id: u64,+ at: AimedPoint,+ button: u8,+ record: impl FnOnce(),+) -> anyhow::Result<()> {+ // Session cleanup must see this press before it can send input.+ held.press_queued(cursor_id);+ request(tx, |reply| Cmd::Press {cursor_id: cursor_id.to_string(),window_id,- x,- y,+ at,button,- reply: tx_r,+ action: crate::input::action::current(),+ reply,})- .map_err(|e| anyhow::anyhow!("cua-persistent-vptr thread is dead: {e}"))?;- rx_r.recv()- .map_err(|e| anyhow::anyhow!("reply channel closed: {e}"))?+ .inspect_err(|error| {+ if !error+ .downcast_ref::<crossbeam_channel::RecvTimeoutError>()+ .is_some_and(crossbeam_channel::RecvTimeoutError::is_timeout)+ {+ held.press_dropped(cursor_id);+ }+ })?;+ // The button is down. Until the hold is recorded, release it on every+ // way out, a panicking `record` included.+ let pressed = UnrecordedPress {+ tx,+ cursor_id,+ button,+ };+ let mut record = Some(record);+ // A call that ended while the button went down cannot record the hold:+ // its caller stopped waiting. Release the button instead.+ let admitted = crate::input::action::admit(|| record.take().expect("recorded once")());+ pressed.disarm();+ let Err(ended) = admitted else {+ return Ok(());+ };+ match release_on(tx, cursor_id, button) {+ Ok(()) => Err(anyhow::anyhow!(+ "{ended}; the button went down as the call ended, and it was released"+ )),+ Err(error) => {+ // The button may still be down, and the owner keeps it: record+ // the hold so that mouse_button_up can release it.+ if let Some(record) = record.take() {+ record();+ }+ Err(anyhow::anyhow!(+ "{ended}; the button went down as the call ended, and its release failed: {error:#}; the hold is kept, so mouse_button_up can release it"+ ))+ }+ }+}++/// A confirmed press whose hold nobody recorded yet: released when dropped.+struct UnrecordedPress<'a> {+ tx: &'a Sender<Cmd>,+ cursor_id: &'a str,+ button: u8,+}++impl UnrecordedPress<'_> {+ /// The hold was recorded, or the caller handles the release itself.+ fn disarm(self) {+ std::mem::forget(self);+ }+}++impl Drop for UnrecordedPress<'_> {+ fn drop(&mut self) {+ // If niri does not confirm this release, the owner keeps the cursor:+ // mouse_button_up and session cleanup release it through+ // [`release_unrecorded`], and a new press on it is refused.+ let _ = release_on(self.tx, self.cursor_id, self.button);+ }}-/// Emit motion_absolute on the held cursor's virtual-pointer. Errors if there-/// is no held button for `cursor_id`.+/// Emit motion_absolute on the held cursor's virtual-pointer, for the tool+/// call bound to this thread. Errors if there is no held button for+/// `cursor_id`.pub fn move_to(cursor_id: &str, x: i32, y: i32) -> anyhow::Result<()> {- let (tx_r, rx_r) = bounded(1);- tx().send(Cmd::MoveTo {+ move_on(tx(), cursor_id, x, y)+}++fn move_on(tx: &Sender<Cmd>, cursor_id: &str, x: i32, y: i32) -> anyhow::Result<()> {+ request(tx, |reply| Cmd::MoveTo {cursor_id: cursor_id.to_string(),x,y,- reply: tx_r,+ action: crate::input::action::current(),+ reply,})- .map_err(|e| anyhow::anyhow!("cua-persistent-vptr thread is dead: {e}"))?;- rx_r.recv()- .map_err(|e| anyhow::anyhow!("reply channel closed: {e}"))?}/// Release `button` on the held cursor. If no other buttons remain held the/// virtual-pointer is destroyed and its Wayland connection torn down.pub fn release(cursor_id: &str, button: u8) -> anyhow::Result<()> {- let (tx_r, rx_r) = bounded(1);- tx().send(Cmd::Release {+ release_on(tx(), cursor_id, button)+}++fn release_on(tx: &Sender<Cmd>, cursor_id: &str, button: u8) -> anyhow::Result<()> {+ request(tx, |reply| Cmd::Release {+ cursor_id: cursor_id.to_string(),+ button,+ reply,+ })+}++/// Release whatever `cursor_id` still holds although the tool has no record+/// of it: a press whose recording failed and whose release niri did not+/// confirm. Returns whether the cursor held anything.+pub fn release_unrecorded(cursor_id: &str) -> anyhow::Result<bool> {+ release_unrecorded_on(tx(), cursor_id)+}++fn release_unrecorded_on(tx: &Sender<Cmd>, cursor_id: &str) -> anyhow::Result<bool> {+ request(tx, |reply| Cmd::ReleaseUnrecorded {+ cursor_id: cursor_id.to_string(),+ reply,+ })+}++/// How long session cleanup waits for the owner thread to take and answer+/// its release: two bounded round trips (the release and the device's+/// destroy). Another cursor's command can keep the thread longer; the hook+/// then fails and is retried.+const SESSION_END_WAIT: std::time::Duration = std::time::Duration::from_secs(2);++/// [`release_unrecorded`] for session cleanup. A cursor the owner holds+/// nothing for and has no press queued for, as for most sessions, is+/// answered without waiting on the owner thread; otherwise the wait is+/// bounded by [`SESSION_END_WAIT`].+pub fn release_unrecorded_at_session_end(cursor_id: &str) -> anyhow::Result<bool> {+ release_unrecorded_within(held_cursors(), tx, cursor_id, SESSION_END_WAIT)+}++fn release_unrecorded_within<'a>(+ held: &HeldCursors,+ tx: impl FnOnce() -> &'a Sender<Cmd>,+ cursor_id: &str,+ wait: std::time::Duration,+) -> anyhow::Result<bool> {+ if !held.contains(cursor_id) {+ return Ok(false);+ }+ request_within(tx(), cursor_id, wait, |reply| Cmd::ReleaseUnrecorded {+ cursor_id: cursor_id.to_string(),+ reply,+ })+}++/// [`release`] for session cleanup, bounded by [`SESSION_END_WAIT`]. On+/// expiry the caller keeps its hold record and retries. A cursor the owner+/// no longer holds and has no press queued for is already released: a+/// release that timed out ran late.+pub fn release_at_session_end(cursor_id: &str, button: u8) -> anyhow::Result<()> {+ release_within(held_cursors(), tx, cursor_id, button, SESSION_END_WAIT)+}++fn release_within<'a>(+ held: &HeldCursors,+ tx: impl FnOnce() -> &'a Sender<Cmd>,+ cursor_id: &str,+ button: u8,+ wait: std::time::Duration,+) -> anyhow::Result<()> {+ if !held.contains(cursor_id) {+ return Ok(());+ }+ request_within(tx(), cursor_id, wait, |reply| Cmd::Release {cursor_id: cursor_id.to_string(),button,- reply: tx_r,+ reply,})- .map_err(|e| anyhow::anyhow!("cua-persistent-vptr thread is dead: {e}"))?;- rx_r.recv()- .map_err(|e| anyhow::anyhow!("reply channel closed: {e}"))?+}++/// [`request`] that gives up after `wait`, queueing included. The command+/// may still run once the owner thread is free.+fn request_within<T>(+ tx: &Sender<Cmd>,+ cursor_id: &str,+ wait: std::time::Duration,+ cmd: impl FnOnce(Sender<anyhow::Result<T>>) -> Cmd,+) -> anyhow::Result<T> {+ request_by(tx, std::time::Instant::now() + wait, cmd).map_err(|error| {+ error.context(format!(+ "the persistent pointer did not release cursor '{cursor_id}' within {wait:?}: another pointer command or the compositor is holding it up; retry"+ ))+ })?}/// Drop the entry for `cursor_id` without emitting any Wayland events.@@ -335,12 +849,646 @@ pub fn release(cursor_id: &str, button: u8) -> anyhow::Result<()> {/// compositor disagrees, this clears the local state without trying to/// send a release that would error.pub fn forget(cursor_id: &str) -> anyhow::Result<()> {- let (tx_r, rx_r) = bounded(1);- tx().send(Cmd::Forget {+ request(tx(), |reply| Cmd::Forget {cursor_id: cursor_id.to_string(),- reply: tx_r,+ reply,})- .map_err(|e| anyhow::anyhow!("cua-persistent-vptr thread is dead: {e}"))?;- rx_r.recv()- .map_err(|e| anyhow::anyhow!("reply channel closed: {e}"))?+}++#[cfg(test)]+mod tests {+ use super::*;+ use std::cell::RefCell;++ #[derive(Debug, PartialEq)]+ enum Event {+ Motion(u32, u32, (u32, u32)),+ Release(u32),+ Destroy,+ }++ #[derive(Default)]+ struct Recorder(RefCell<Vec<Event>>);++ impl HeldDevice for &Recorder {+ fn motion(&self, x: u32, y: u32, extent: (u32, u32)) {+ self.0.borrow_mut().push(Event::Motion(x, y, extent));+ }++ fn button(&self, button: u32, state: ButtonState) {+ assert_eq!(state, ButtonState::Released);+ self.0.borrow_mut().push(Event::Release(button));+ }++ fn destroy(&self) {+ self.0.borrow_mut().push(Event::Destroy);+ }+ }++ fn held(device: &Recorder) -> HashMap<String, ActivePointer<&Recorder>> {+ HashMap::from([(+ "drag".to_owned(),+ ActivePointer {+ vptr: device,+ held: HashSet::from([0x110]),+ out_w: 1280,+ out_h: 720,+ origin_x: 0,+ origin_y: 0,+ },+ )])+ }++ /// An owner that records what reaches the compositor, sending input as+ /// the virtual pointer does: through the action bound to its thread.+ struct Wire {+ sent: Arc<std::sync::Mutex<Vec<&'static str>>>,+ /// The caller stops waiting while the press is under way.+ caller_leaves_during_press: bool,+ /// niri does not confirm a release.+ release_fails: Arc<std::sync::atomic::AtomicBool>,+ /// Pressed and not yet released, as the owner keeps it.+ held: bool,+ }++ impl Owner for Wire {+ fn press(&mut self, _: &str, _: u64, _: AimedPoint, _: u8) -> anyhow::Result<()> {+ crate::input::action::admit(|| self.sent.lock().unwrap().push("press"))?;+ self.held = true;+ if self.caller_leaves_during_press {+ crate::input::action::current().unwrap().abandon();+ }+ Ok(())+ }++ fn move_to(&mut self, _: &str, _: i32, _: i32) -> anyhow::Result<()> {+ crate::input::action::admit(|| self.sent.lock().unwrap().push("motion"))+ }++ fn release(&mut self, cursor_id: &str, _: u8) -> anyhow::Result<()> {+ if !self.held {+ anyhow::bail!("no held mouse button for cursor '{cursor_id}'");+ }+ self.sent.lock().unwrap().push("release");+ if self.release_fails.load(std::sync::atomic::Ordering::SeqCst) {+ anyhow::bail!("niri did not answer a Wayland round trip in time");+ }+ self.held = false;+ Ok(())+ }++ fn release_unrecorded(&mut self, cursor_id: &str) -> anyhow::Result<bool> {+ if !self.held {+ return Ok(false);+ }+ self.release(cursor_id, 1).map(|()| true)+ }++ fn forget(&mut self, _: &str) {}++ fn holds(&self, _: &str) -> bool {+ self.held+ }+ }++ /// What reached the compositor, in order.+ type Sent = Arc<std::sync::Mutex<Vec<&'static str>>>;++ /// The owner thread's channel, with `wire` as its owner.+ fn owner(+ caller_leaves_during_press: bool,+ release_fails: bool,+ ) -> (Sender<Cmd>, Arc<std::sync::Mutex<Vec<&'static str>>>) {+ owner_with(+ caller_leaves_during_press,+ Arc::new(std::sync::atomic::AtomicBool::new(release_fails)),+ )+ }++ fn owner_with(+ caller_leaves_during_press: bool,+ release_fails: Arc<std::sync::atomic::AtomicBool>,+ ) -> (Sender<Cmd>, Arc<std::sync::Mutex<Vec<&'static str>>>) {+ let (tx, sent, _) = owner_publishing(caller_leaves_during_press, release_fails);+ (tx, sent)+ }++ fn owner_publishing(+ caller_leaves_during_press: bool,+ release_fails: Arc<std::sync::atomic::AtomicBool>,+ ) -> (Sender<Cmd>, Sent, Arc<HeldCursors>) {+ let sent = Arc::new(std::sync::Mutex::new(Vec::new()));+ let (tx, rx) = bounded(32);+ let wire = Wire {+ sent: sent.clone(),+ caller_leaves_during_press,+ release_fails,+ held: false,+ };+ let held = Arc::new(HeldCursors::default());+ thread::spawn({+ let held = held.clone();+ move || owner_thread(rx, wire, held)+ });+ (tx, sent, held)+ }++ fn timed_out(error: &anyhow::Error) -> bool {+ crate::input::foreground::error_code(error) == Some(crate::input::foreground::CODE_TIMEOUT)+ }++ #[test]+ fn a_held_pointer_sends_nothing_for_a_call_that_has_ended() {+ // mouse_button_down and mouse_drag hand their input to the owner+ // thread. A call that timed out or whose caller went away must not+ // press or move there afterwards.+ let (tx, sent) = owner(false, false);+ let action = Action::new(std::time::Duration::from_secs(60));+ action.abandon();+ let _bound = crate::input::action::bind(action);+ let mut recorded = false;+ let pressed = press_on(+ &tx,+ &HeldCursors::default(),+ "drag",+ 1,+ (10, 10).into(),+ 1,+ || recorded = true,+ )+ .unwrap_err();+ assert!(timed_out(&pressed), "{pressed:#}");+ let moved = move_on(&tx, "drag", 20, 20).unwrap_err();+ assert!(timed_out(&moved), "{moved:#}");+ assert!(sent.lock().unwrap().is_empty());+ assert!(!recorded);+ }++ #[test]+ fn a_press_whose_caller_left_meanwhile_is_released_not_left_held() {+ // The caller stopped waiting while the button went down, so it will+ // never record the hold: the press must not stay down.+ let (tx, sent) = owner(true, false);+ let _bound = crate::input::action::bind(Action::new(std::time::Duration::from_secs(60)));+ let mut recorded = false;+ let pressed = press_on(+ &tx,+ &HeldCursors::default(),+ "drag",+ 1,+ (10, 10).into(),+ 1,+ || recorded = true,+ )+ .unwrap_err();+ assert!(timed_out(&pressed), "{pressed:#}");+ assert_eq!(*sent.lock().unwrap(), ["press", "release"]);+ assert!(!recorded);++ // A caller still waiting records the hold, and the button stays down.+ let (tx, sent) = owner(false, false);+ let _bound = crate::input::action::bind(Action::new(std::time::Duration::from_secs(60)));+ let mut recorded = false;+ press_on(+ &tx,+ &HeldCursors::default(),+ "drag",+ 1,+ (10, 10).into(),+ 1,+ || recorded = true,+ )+ .unwrap();+ assert_eq!(*sent.lock().unwrap(), ["press"]);+ assert!(recorded);+ }++ #[test]+ fn a_press_whose_recording_panics_is_released() {+ let (tx, sent) = owner(false, false);+ let _bound = crate::input::action::bind(Action::new(std::time::Duration::from_secs(60)));+ let pressed = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {+ press_on(+ &tx,+ &HeldCursors::default(),+ "drag",+ 1,+ (10, 10).into(),+ 1,+ || panic!("recording failed"),+ )+ }));+ assert!(pressed.is_err());+ assert_eq!(*sent.lock().unwrap(), ["press", "release"]);+ }++ #[test]+ fn a_press_whose_recording_panics_and_whose_release_fails_stays_releasable() {+ let release_fails = Arc::new(std::sync::atomic::AtomicBool::new(true));+ let (tx, sent) = owner_with(false, release_fails.clone());+ let _bound = crate::input::action::bind(Action::new(std::time::Duration::from_secs(60)));+ let pressed = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {+ press_on(+ &tx,+ &HeldCursors::default(),+ "drag",+ 1,+ (10, 10).into(),+ 1,+ || panic!("recording failed"),+ )+ }));+ assert!(pressed.is_err());+ assert_eq!(*sent.lock().unwrap(), ["press", "release"]);+ // niri answers again: mouse_button_up or session cleanup finds the+ // press that nobody recorded and releases it.+ release_fails.store(false, std::sync::atomic::Ordering::SeqCst);+ assert!(release_unrecorded_on(&tx, "drag").unwrap());+ assert_eq!(*sent.lock().unwrap(), ["press", "release", "release"]);+ assert!(!release_unrecorded_on(&tx, "drag").unwrap());+ }++ #[test]+ fn a_press_whose_release_fails_after_its_call_ended_keeps_its_hold() {+ // The button may still be down: mouse_button_up needs the hold to+ // release it.+ let (tx, sent) = owner(true, true);+ let _bound = crate::input::action::bind(Action::new(std::time::Duration::from_secs(60)));+ let mut recorded = false;+ let pressed = press_on(+ &tx,+ &HeldCursors::default(),+ "drag",+ 1,+ (10, 10).into(),+ 1,+ || recorded = true,+ )+ .unwrap_err();+ assert!(timed_out(&pressed), "{pressed:#}");+ assert!(+ pressed.to_string().contains("the hold is kept"),+ "{pressed:#}"+ );+ assert_eq!(*sent.lock().unwrap(), ["press", "release"]);+ assert!(recorded);+ }++ #[test]+ fn session_end_does_not_wait_on_the_owner_for_a_session_without_a_pointer() {+ // Another session's command is stuck in a Sway round trip.+ let held = HeldCursors::default();+ held.lock().held.insert("other-session".to_owned());+ let released = release_unrecorded_within(+ &held,+ || -> &Sender<Cmd> { unreachable!("asked the stalled owner") },+ "empty-session",+ std::time::Duration::from_secs(60),+ );+ assert!(!released.unwrap());+ }++ #[test]+ fn session_end_gives_up_on_a_stalled_owner_within_its_bound() {+ let held = Arc::new(HeldCursors::default());+ held.lock().held.insert("session".to_owned());+ // The owner thread is stuck in another command: nothing reads the+ // channel. With room in the queue the reply never comes; with the+ // queue full the command cannot even be queued.+ for queue_full in [false, true] {+ let (tx, rx) = bounded::<Cmd>(1);+ if queue_full {+ tx.send(Cmd::Forget {+ cursor_id: "other".to_owned(),+ reply: bounded(1).0,+ })+ .unwrap();+ }+ let wait = std::time::Duration::from_millis(200);+ let (done, finished) = bounded(2);+ let started = std::time::Instant::now();+ thread::spawn({+ let held = held.clone();+ move || {+ // The recorded hold's release, then the unrecorded one.+ let recorded = release_within(&held, || &tx, "session", 1, wait);+ let _ = done.send(recorded.map_err(|error| error.to_string()));+ let unrecorded = release_unrecorded_within(&held, || &tx, "session", wait);+ let _ = done.send(unrecorded.map(drop).map_err(|error| error.to_string()));+ }+ });+ for release in ["recorded", "unrecorded"] {+ let released = finished+ .recv_timeout(std::time::Duration::from_secs(5))+ .unwrap_or_else(|_| panic!("the {release} release waited on a stalled owner"));+ let refused = released.unwrap_err();+ assert!(refused.ends_with("retry"), "{refused}");+ }+ assert!(started.elapsed() < std::time::Duration::from_secs(2));+ drop(rx);+ }+ }++ #[test]+ fn session_end_retry_completes_after_a_late_recorded_release() {+ // Session end's release of a recorded hold timed out; the release+ // ran late and lifted the button. The retry finds nothing held.+ let (tx, sent, held) = owner_publishing(false, Default::default());+ press_on(&tx, &held, "session", 1, (10, 10).into(), 1, || ()).unwrap();+ release_on(&tx, "session", 1).unwrap();+ let wait = std::time::Duration::from_secs(5);+ let retried = release_within(&held, || &tx, "session", 1, wait);+ assert!(retried.is_ok(), "{retried:?}");+ assert_eq!(*sent.lock().unwrap(), ["press", "release"]);+ }++ #[test]+ fn a_press_the_owner_never_handles_stops_counting_as_under_way() {+ let held = HeldCursors::default();+ let (tx, rx) = bounded::<Cmd>(1);+ drop(rx);+ press_on(&tx, &held, "dead", 1, (10, 10).into(), 1, || ()).unwrap_err();+ assert!(!held.contains("dead"));+ }++ #[test]+ fn a_request_to_a_full_queue_gives_up_with_or_without_an_action() {+ let (tx, rx) = bounded::<Cmd>(1);+ let forget = |reply| Cmd::Forget {+ cursor_id: "stuck".into(),+ reply,+ };+ tx.send(forget(bounded(1).0)).unwrap();+ let (done, finished) = bounded(2);+ thread::spawn(move || {+ let _ = done.send(request::<()>(&tx, forget).is_err());+ let action = crate::input::action::Action::new(std::time::Duration::from_millis(100));+ let _bound = crate::input::action::bind(action);+ let _ = done.send(request::<()>(&tx, forget).is_err());+ });+ for _ in 0..2 {+ let refused = finished.recv_timeout(SESSION_END_WAIT * 3);+ assert_eq!(refused, Ok(true));+ }+ drop(rx);+ }++ #[test]+ fn a_request_waits_for_the_owner_only_until_the_action_deadline() {+ let (tx, rx) = bounded::<Cmd>(1);+ let action = crate::input::action::Action::new(std::time::Duration::from_millis(100));+ let _bound = crate::input::action::bind(action);+ let started = std::time::Instant::now();+ request::<()>(&tx, |reply| Cmd::Forget {+ cursor_id: "stuck".into(),+ reply,+ })+ .unwrap_err();+ assert!(started.elapsed() < std::time::Duration::from_secs(5));+ drop(rx);+ }++ #[test]+ fn cleanup_sees_a_handled_press_pending_while_its_hold_is_published() {+ struct Holding(+ std::sync::mpsc::SyncSender<()>,+ std::sync::Mutex<std::sync::mpsc::Receiver<()>>,+ );+ impl Owner for Holding {+ fn press(&mut self, _: &str, _: u64, _: AimedPoint, _: u8) -> anyhow::Result<()> {+ unreachable!()+ }+ fn move_to(&mut self, _: &str, _: i32, _: i32) -> anyhow::Result<()> {+ unreachable!()+ }+ fn release(&mut self, _: &str, _: u8) -> anyhow::Result<()> {+ unreachable!()+ }+ fn release_unrecorded(&mut self, _: &str) -> anyhow::Result<bool> {+ unreachable!()+ }+ fn forget(&mut self, _: &str) {}+ fn holds(&self, _: &str) -> bool {+ self.0.send(()).unwrap();+ self.1.lock().unwrap().recv().unwrap();+ true+ }+ }+ let held = Arc::new(HeldCursors::default());+ held.press_queued("c");+ let (asked, asking) = std::sync::mpsc::sync_channel(0);+ let (answer, answering) = std::sync::mpsc::channel();+ let publisher = {+ let held = held.clone();+ thread::spawn(move || {+ held.publish("c", &Holding(asked, std::sync::Mutex::new(answering)), true)+ })+ };+ asking.recv().unwrap();+ let (seen, seeing) = std::sync::mpsc::channel();+ thread::spawn({+ let held = held.clone();+ move || seen.send(held.contains("c"))+ });+ let visible = seeing.recv_timeout(std::time::Duration::from_secs(2));+ answer.send(()).unwrap();+ assert_eq!(visible, Ok(true));+ publisher.join().unwrap();+ assert!(held.contains("c"));+ }++ #[test]+ fn session_end_waits_for_a_press_still_under_way() {+ // The client cancelled mouse_button_down while its press waited on+ // the owner, then ended the session. The press can still go down.+ let held = Arc::new(HeldCursors::default());+ let (tx, rx) = bounded::<Cmd>(1);+ let pressing = thread::spawn({+ let (tx, held) = (tx.clone(), held.clone());+ move || {+ let _ = press_on(&tx, &held, "session", 1, (10, 10).into(), 1, || ());+ }+ });+ while tx.is_empty() {+ thread::yield_now();+ }+ let wait = std::time::Duration::from_millis(200);+ let released = release_unrecorded_within(&held, || &tx, "session", wait);+ assert!(+ released+ .as_ref()+ .is_err_and(|error| error.to_string().ends_with("retry")),+ "{released:?}"+ );+ // Answer nothing: dropping the queued press ends its wait.+ drop(rx.recv().unwrap());+ pressing.join().unwrap();+ }++ #[test]+ fn session_end_releases_the_unrecorded_press_of_its_own_session() {+ let release_fails = Arc::new(std::sync::atomic::AtomicBool::new(true));+ let (tx, sent, held) = owner_publishing(false, release_fails.clone());+ let pressed = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {+ press_on(&tx, &held, "session", 1, (10, 10).into(), 1, || {+ panic!("recording failed")+ })+ }));+ assert!(pressed.is_err());+ release_fails.store(false, std::sync::atomic::Ordering::SeqCst);+ let wait = std::time::Duration::from_secs(5);+ assert!(release_unrecorded_within(&held, || &tx, "session", wait).unwrap());+ assert_eq!(*sent.lock().unwrap(), ["press", "release", "release"]);+ assert!(!held.contains("session"));+ }++ #[test]+ fn a_held_motion_for_a_call_that_has_ended_is_not_sent() {+ let device = Recorder::default();+ let mut active = held(&device);+ let action = Action::new(std::time::Duration::from_secs(60));+ action.abandon();+ let _bound = crate::input::action::bind(action);+ let refused = move_held(+ &mut active,+ "drag",+ (200, 200),+ || Ok(Some((0, 0, 1280, 720))),+ |_| Ok(()),+ )+ .unwrap_err();+ assert!(timed_out(&refused), "{refused:#}");+ assert!(device.0.borrow().is_empty());+ // The button stays held for the next call's motion or release.+ assert!(active.contains_key("drag"));+ }++ /// Release `drag`'s button with `release_held`, or with+ /// `release_all_held` when `unrecorded`, through `roundtrip`.+ fn release_drag(+ active: &mut HashMap<String, ActivePointer<&Recorder>>,+ unrecorded: bool,+ roundtrip: impl FnMut(&str) -> anyhow::Result<()>,+ forgotten: &mut bool,+ ) -> anyhow::Result<()> {+ let forget = |_: &str| *forgotten = true;+ if unrecorded {+ release_all_held(active, "drag", roundtrip, forget).map(|held| assert!(held))+ } else {+ release_held(active, "drag", 0x110, roundtrip, forget)+ }+ }++ #[test]+ fn a_release_niri_did_not_answer_in_time_keeps_the_cursor_for_another_try() {+ for unrecorded in [false, true] {+ let device = Recorder::default();+ let mut active = held(&device);+ let mut forgotten = false;+ let refused = release_drag(+ &mut active,+ unrecorded,+ |_| Err(round_trip_failed(crate::wayland::RoundTripTimedOut.into())),+ &mut forgotten,+ )+ .unwrap_err();+ assert_eq!(+ refused.to_string(),+ "compositor roundtrip failed: niri did not answer a Wayland round trip in time"+ );+ assert!(active.contains_key("drag"), "unrecorded: {unrecorded}");+ assert!(!forgotten);+ }+ }++ #[test]+ fn a_release_on_a_dead_connection_drops_the_cursor() {+ // The compositor dropped the client, and with it the virtual pointer+ // and its buttons: keeping the cursor would refuse every new press.+ for unrecorded in [false, true] {+ let device = Recorder::default();+ let mut active = held(&device);+ let mut forgotten = false;+ release_drag(+ &mut active,+ unrecorded,+ |_| {+ Err(round_trip_failed(anyhow::anyhow!(+ "niri Wayland flush failed: Broken pipe"+ )))+ },+ &mut forgotten,+ )+ .unwrap();+ assert!(active.is_empty(), "unrecorded: {unrecorded}");+ assert!(forgotten);+ assert_eq!(*device.0.borrow(), [Event::Release(0x110), Event::Destroy]);+ }+ }++ #[test]+ fn a_held_motion_after_the_pointer_space_changed_releases_instead_of_moving() {+ let device = Recorder::default();+ let mut active = held(&device);+ // Unchanged: the motion goes out in the saved extent.+ move_held(+ &mut active,+ "drag",+ (200, 200),+ || Ok(Some((0, 0, 1280, 720))),+ |_| Ok(()),+ )+ .unwrap();+ assert_eq!(*device.0.borrow(), [Event::Motion(200, 200, (1280, 720))]);++ // The output went from scale 1 to 1.5 between two motions: scaling+ // by the old extent would land far from the aimed point.+ let stopped = move_held(+ &mut active,+ "drag",+ (267, 267),+ || Ok(Some((0, 0, 854, 480))),+ |_| Ok(()),+ )+ .unwrap_err();+ assert!(stopped+ .downcast_ref::<HeldPointerEnded>()+ .is_some_and(|ended| ended.release_confirmed));+ assert_eq!(+ stopped.to_string(),+ "the niri pointer space changed from (0, 0, 1280, 720) to (0, 0, 854, 480); take a new screenshot; no motion was sent, and the held button was released where the pointer was, which the app may take as a drop"+ );+ assert_eq!(+ device.0.borrow()[1..],+ [Event::Release(0x110), Event::Destroy]+ );+ assert!(active.is_empty());+ }++ #[test]+ fn a_held_motion_whose_cleanup_niri_never_answers_still_ends_the_hold() {+ // niri stopped answering: the pointer space cannot be read and the+ // release is not confirmed. The owner discards the cursor anyway, so+ // the tool must learn that the hold ended, not see a plain failure.+ let device = Recorder::default();+ let mut active = held(&device);+ let stopped = move_held(+ &mut active,+ "drag",+ (200, 200),+ || anyhow::bail!("niri did not answer in time"),+ |_| anyhow::bail!("niri did not answer a Wayland round trip in time"),+ )+ .unwrap_err();+ let ended = stopped+ .downcast_ref::<HeldPointerEnded>()+ .expect("the hold ended");+ assert!(!ended.release_confirmed);+ assert_eq!(+ stopped.to_string(),+ "niri did not answer in time; no motion was sent, and the held button's release was sent but not confirmed (niri did not answer a Wayland round trip in time); the cursor no longer holds it"+ );+ assert_eq!(*device.0.borrow(), [Event::Release(0x110), Event::Destroy]);+ assert!(active.is_empty());+ }}diff --git a/libs/cua-driver/rust/crates/platform-linux/src/wayland/shell_helper.rs b/libs/cua-driver/rust/crates/platform-linux/src/wayland/shell_helper.rsindex 43f18f16fb..0ce650af71 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/wayland/shell_helper.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/wayland/shell_helper.rs@@ -26,10 +26,15 @@use std::os::unix::fs::{MetadataExt, PermissionsExt};mod transport;-use std::time::Duration;+use std::time::{Duration, Instant};+use crate::interaction::operation_deadline;use crate::x11::WindowInfo;+const FOCUS_CONFIRM: Duration = Duration::from_millis(500);+const CALL_CAP: Duration = Duration::from_millis(800);+const CAPTURE_CAP: Duration = Duration::from_secs(5);+const DEST: &str = "org.cua.WinRects";const COMPOSITOR_DEST: &str = "org.gnome.Shell";const PATH: &str = "/org/cua/WinRects";@@ -51,7 +56,7 @@ struct ShellWindow {}pub fn available() -> bool {- shell_owner(false).is_some()+ shell_owner(false, operation_deadline(CALL_CAP)).is_some()}/// Whether GNOME Shell is this session's compositor: the desktop name says@@ -63,48 +68,31 @@ pub fn gnome_shell_session() -> bool {desktops.split(':').any(|desktop| desktop.eq_ignore_ascii_case("gnome"))- }) || SessionShellProbe.name_owner(COMPOSITOR_DEST).is_some()+ }) || SessionShellProbe {+ deadline: operation_deadline(CALL_CAP),+ }+ .name_owner(COMPOSITOR_DEST)+ .is_some()}pub fn semantic_cursor_available() -> bool {- shell_owner_with_min_version(Some(SEMANTIC_CURSOR_API_VERSION)).is_some()-}--fn gdbus_call(method: &str, args: &[String]) -> Option<String> {- let owner = shell_owner(false)?;- gdbus_call_to(- &owner,- PATH,- &format!("{IFACE}.{method}"),- args,- Duration::from_millis(800),+ shell_owner_with_min_version(+ Some(SEMANTIC_CURSOR_API_VERSION),+ operation_deadline(CALL_CAP),)+ .is_some()}-fn gdbus_call_with_timeout(method: &str, args: &[String], timeout: Duration) -> Option<String> {- let owner = shell_owner(false)?;- gdbus_call_to(&owner, PATH, &format!("{IFACE}.{method}"), args, timeout)+/// Call `method` on the verified helper; the owner checks and the call+/// share one `deadline`.+fn gdbus_call(method: &str, args: &[String], deadline: Instant) -> Option<String> {+ let owner = shell_owner(false, deadline)?;+ transport::call(&owner, PATH, &format!("{IFACE}.{method}"), args, deadline)}-fn trusted_gdbus_call(method: &str, args: &[String]) -> Option<String> {- let owner = shell_owner(true)?;- gdbus_call_to(- &owner,- PATH,- &format!("{IFACE}.{method}"),- args,- Duration::from_millis(800),- )-}--fn gdbus_call_to(- destination: &str,- object_path: &str,- method: &str,- args: &[String],- timeout: Duration,-) -> Option<String> {- transport::call(destination, object_path, method, args, timeout)+fn trusted_gdbus_call(method: &str, args: &[String], deadline: Instant) -> Option<String> {+ let owner = shell_owner(true, deadline)?;+ transport::call(&owner, PATH, &format!("{IFACE}.{method}"), args, deadline)}/// Resolve the helper's immutable unique bus name and prove that it is hosted@@ -112,12 +100,15 @@ fn gdbus_call_to(/// callers additionally require the current helper API. Calling the unique/// name closes the race where another process replaces the well-known name/// after ownership is checked.-fn shell_owner(require_browser_api: bool) -> Option<String> {- shell_owner_with_min_version(require_browser_api.then_some(BROWSER_HELPER_API_VERSION))+fn shell_owner(require_browser_api: bool, deadline: Instant) -> Option<String> {+ shell_owner_with_min_version(+ require_browser_api.then_some(BROWSER_HELPER_API_VERSION),+ deadline,+ )}-fn shell_owner_with_min_version(min_version: Option<u32>) -> Option<String> {- select_shell_owner(&SessionShellProbe, min_version)+fn shell_owner_with_min_version(min_version: Option<u32>, deadline: Instant) -> Option<String> {+ select_shell_owner(&SessionShellProbe { deadline }, min_version)}/// Session-bus and procfs observations behind [`select_shell_owner`]. The@@ -188,30 +179,33 @@ fn attest_shell_owner<P: ShellOwnerProbe>(Some((owner, version))}-struct SessionShellProbe;+/// The live session bus, with every call sharing the caller's `deadline`.+struct SessionShellProbe {+ deadline: Instant,+}impl ShellOwnerProbe for SessionShellProbe {type ProcessPin = std::fs::File;fn name_owner(&self, name: &str) -> Option<String> {- let raw = gdbus_call_to(+ let raw = transport::call(DBUS_DEST,DBUS_PATH,&format!("{DBUS_IFACE}.GetNameOwner"),&[name.to_owned()],- Duration::from_millis(800),+ self.deadline,)?;parse_quoted_string(&raw)}fn connection_identity(&self, owner: &str) -> Option<(u32, u32)> {let scalar = |method: &str| {- let raw = gdbus_call_to(+ let raw = transport::call(DBUS_DEST,DBUS_PATH,&format!("{DBUS_IFACE}.{method}"),&[owner.to_owned()],- Duration::from_millis(800),+ self.deadline,)?;parse_first_u32(&raw)};@@ -249,12 +243,12 @@ impl ShellOwnerProbe for SessionShellProbe {}fn helper_version(&self, owner: &str) -> Option<u32> {- let raw = gdbus_call_to(+ let raw = transport::call(owner,PATH,&format!("{IFACE}.GetVersion"),&[],- Duration::from_millis(800),+ self.deadline,)?;parse_first_u32(&raw)}@@ -313,15 +307,16 @@ pub enum WindowCapture {}/// The verified helper owner and the API version that same owner reported.-fn attested_owner_with_version() -> Option<(String, u32)> {- let (owner, version) = attest_shell_owner(&SessionShellProbe, true)?;+fn attested_owner_with_version(deadline: Instant) -> Option<(String, u32)> {+ let (owner, version) = attest_shell_owner(&SessionShellProbe { deadline }, true)?;Some((owner, version?))}/// Capture one window's compositor actor without activating it. Covered/// windows keep their own content, so no stage crop is involved.pub fn screenshot_window(id: u64, pid: Option<u32>) -> WindowCapture {- let Some((owner, version)) = attested_owner_with_version() else {+ let deadline = operation_deadline(CAPTURE_CAP);+ let Some((owner, version)) = attested_owner_with_version(deadline) else {return WindowCapture::Unsupported;};if version < WINDOW_CAPTURE_API_VERSION {@@ -330,12 +325,12 @@ pub fn screenshot_window(id: u64, pid: Option<u32>) -> WindowCapture {let Ok(id) = u32::try_from(id) else {return WindowCapture::Unavailable;};- gdbus_call_to(+ transport::call(&owner,PATH,&format!("{IFACE}.CaptureWindow"),&[id.to_string(), pid.unwrap_or(0).to_string()],- Duration::from_secs(5),+ deadline,).and_then(|raw| decode_capture(&raw)).map_or(WindowCapture::Unavailable, WindowCapture::Png)@@ -350,13 +345,14 @@ pub fn screenshot_window_preview(max_dimension: u32,) -> Option<(Vec<u8>, u32, u32)> {let id = u32::try_from(id).ok()?;- let owner = shell_owner_with_min_version(Some(WINDOW_PREVIEW_API_VERSION))?;- let raw = gdbus_call_to(+ let deadline = operation_deadline(CAPTURE_CAP);+ let owner = shell_owner_with_min_version(Some(WINDOW_PREVIEW_API_VERSION), deadline)?;+ let raw = transport::call(&owner,PATH,&format!("{IFACE}.CaptureWindowPreview"),&[id.to_string(), pid.to_string(), max_dimension.to_string()],- Duration::from_secs(5),+ deadline,)?;let png = strip_preview_timestamp(decode_capture(&raw)?)?;let (width, height) = parse_preview_dimensions(&raw)?;@@ -406,21 +402,14 @@ fn strip_preview_timestamp(mut png: Vec<u8>) -> Option<Vec<u8>> {/// can use Shell's screenshot API without confusing a stable Wayland window id/// for an X11 drawable.pub fn screenshot_display() -> Option<Vec<u8>> {- let raw = gdbus_call_with_timeout("Capture", &[], Duration::from_secs(5))?;+ let raw = gdbus_call("Capture", &[], operation_deadline(CAPTURE_CAP))?;decode_capture(&raw)}/// Capture the GNOME stage only when the helper owner and current browser API/// have passed the same compositor-attestation checks used for mutation.pub fn trusted_screenshot_display() -> Option<Vec<u8>> {- let owner = shell_owner(true)?;- let raw = gdbus_call_to(- &owner,- PATH,- &format!("{IFACE}.Capture"),- &[],- Duration::from_secs(5),- )?;+ let raw = trusted_gdbus_call("Capture", &[], operation_deadline(CAPTURE_CAP))?;decode_capture(&raw)}@@ -446,7 +435,7 @@ fn decode_capture(raw: &str) -> Option<Vec<u8>> {/// is floating, so pixel actions derived from the returned screenshot miss/// their target.pub fn window_origin(pid: u32, window_id: u64) -> Option<(i32, i32)> {- let raw = gdbus_call("GetRects", &[])?;+ let raw = gdbus_call("GetRects", &[], operation_deadline(CALL_CAP))?;parse_window_origin(&raw, pid, window_id)}@@ -476,15 +465,20 @@ fn select_window(windows: Vec<WindowInfo>, window_id: Option<u64>) -> Option<Win/// shell already owns the authoritative stacking list, geometry, visibility,/// title, and PID, so use that metadata directly for `list_windows`.pub fn list_windows(filter_pid: Option<u32>) -> Option<Vec<WindowInfo>> {- let raw = gdbus_call("GetRects", &[])?;+ let raw = gdbus_call("GetRects", &[], operation_deadline(CALL_CAP))?;parse_windows(&raw, filter_pid)}/// Return one compositor-attested GNOME window only when the current helper/// API is hosted by the verified Shell owner.pub fn trusted_window_for_id(pid: u32, window_id: u64) -> Option<WindowInfo> {- exact_shell_window(&trusted_shell_windows(Some(pid))?, pid, window_id)- .map(|window| window.info.clone())+ shell_sequence_id(window_id)?;+ exact_shell_window(+ &trusted_shell_windows(Some(pid), operation_deadline(CALL_CAP))?,+ pid,+ window_id,+ )+ .map(|window| window.info.clone())}/// The one Shell window with this PID and stable sequence ID. A same-PID@@ -497,24 +491,6 @@ fn exact_shell_window(windows: &[ShellWindow], pid: u32, window_id: u64) -> Optimatches.next().is_none().then_some(window)}-/// Decide how [`with_focused_window`] reaches the exact target: `None` when it-/// already has focus, otherwise the focused window to restore afterwards.-fn plan_focus(before: &[ShellWindow], pid: u32, window_id: u64) -> anyhow::Result<Option<u64>> {- let target = exact_shell_window(before, pid, window_id)- .ok_or_else(|| anyhow::anyhow!("no exact GNOME Shell window owns the approved target"))?;- if shell_sequence_id(window_id).is_none() {- anyhow::bail!("the approved target is not a GNOME Shell window sequence");- }- if target.focused {- return Ok(None);- }- before- .iter()- .find(|window| window.focused)- .map(|window| Some(window.info.xid))- .ok_or_else(|| anyhow::anyhow!("GNOME Shell did not expose a restorable focused window"))-}-/// Shell stable sequences are u32. Wider values, such as the AT-SPI fallback/// handles (`pid << 16`), are never passed to `Activate`.fn shell_sequence_id(window_id: u64) -> Option<u32> {@@ -534,44 +510,40 @@ fn exact_window_focused(windows: &[ShellWindow], window_id: u32) -> bool {/// vector means the trusted helper found no owned windows.pub fn trusted_window_ids_for_pid(pid: u32) -> Option<Vec<u64>> {Some(- trusted_shell_windows(Some(pid))?+ trusted_shell_windows(Some(pid), operation_deadline(CALL_CAP))?.into_iter().map(|window| window.info.xid).collect(),)}-/// Briefly activate an exact compositor-owned window, execute one bounded-/// focus-sensitive operation, and restore the prior exact Shell focus.-pub fn with_focused_window<T>(- pid: u32,- window_id: u64,- body: impl FnOnce() -> anyhow::Result<T>,-) -> anyhow::Result<T> {- let before = trusted_shell_windows(None)- .ok_or_else(|| anyhow::anyhow!("the verified GNOME Shell helper API is unavailable"))?;- let Some(previous) = plan_focus(&before, pid, window_id)? else {- return body();+/// The window the verified Shell owner reports focused, `None` when none is.+/// A focused ID the snapshot lists twice is not authoritative.+pub fn trusted_focus(deadline: Instant) -> anyhow::Result<Option<(Option<u32>, u64)>> {+ focused_window(+ &trusted_shell_windows(None, deadline)+ .ok_or_else(|| anyhow::anyhow!("the verified GNOME Shell helper API is unavailable"))?,+ )+}++fn focused_window(windows: &[ShellWindow]) -> anyhow::Result<Option<(Option<u32>, u64)>> {+ let Some(focused) = windows.iter().find(|window| window.focused) else {+ return Ok(None);};- trusted_activate_window(window_id)- .then_some(())- .ok_or_else(|| anyhow::anyhow!("GNOME Shell did not confirm exact target activation"))?;- let body_result = body();- let restored = trusted_activate_window(previous);- match (body_result, restored) {- (Ok(value), true) => Ok(value),- (Err(error), true) => Err(error),- (Ok(_), false) => {- anyhow::bail!("GNOME Shell did not restore the previously focused window")- }- (Err(error), false) => Err(anyhow::anyhow!(- "{error}; GNOME Shell also failed to restore the previously focused window"- )),+ let id = focused.info.xid;+ if windows+ .iter()+ .filter(|window| window.info.xid == id)+ .count()+ > 1+ {+ anyhow::bail!("GNOME Shell listed focused window {id} more than once");}+ Ok(Some((focused.info.pid, id)))}-fn trusted_shell_windows(filter_pid: Option<u32>) -> Option<Vec<ShellWindow>> {- let raw = trusted_gdbus_call("GetRects", &[])?;+fn trusted_shell_windows(filter_pid: Option<u32>, deadline: Instant) -> Option<Vec<ShellWindow>> {+ let raw = trusted_gdbus_call("GetRects", &[], deadline)?;parse_shell_windows(&raw, filter_pid)}@@ -585,48 +557,40 @@ pub fn activate_window(window_id: u64) -> bool {let Some(window_id) = shell_sequence_id(window_id) else {return false;};- let accepted = gdbus_call("Activate", &[window_id.to_string()])+ let deadline = operation_deadline(FOCUS_CONFIRM);+ let accepted = gdbus_call("Activate", &[window_id.to_string()], deadline).is_some_and(|output| output.trim_start().starts_with("(true,"));- if !accepted {- return false;- }- let deadline = std::time::Instant::now() + Duration::from_millis(500);- loop {- if window_is_focused(window_id) {- return true;- }- if std::time::Instant::now() >= deadline {- return false;- }- std::thread::sleep(Duration::from_millis(10));- }+ accepted && focus_confirmed(deadline, |deadline| window_is_focused(window_id, deadline))}-fn trusted_activate_window(window_id: u64) -> bool {- let Some(window_id) = shell_sequence_id(window_id) else {- return false;- };- let accepted = trusted_gdbus_call("Activate", &[window_id.to_string()])- .is_some_and(|output| output.trim_start().starts_with("(true,"));- if !accepted {- return false;- }- let deadline = std::time::Instant::now() + Duration::from_millis(500);- loop {- if trusted_shell_windows(None)- .is_some_and(|windows| exact_window_focused(&windows, window_id))- {- return true;- }- if std::time::Instant::now() >= deadline {- return false;- }- std::thread::sleep(Duration::from_millis(10));+fn focus_confirmed(deadline: Instant, mut focused: impl FnMut(Instant) -> bool) -> bool {+ crate::interaction::confirm(deadline, |deadline| Ok::<_, ()>(focused(deadline)))+ .unwrap_or(false)+}++/// Activate exact window `window_id` of `pid` through the verified Shell+/// owner and read the focus back by `deadline`.+pub fn trusted_focus_window(pid: u32, window_id: u64, deadline: Instant) -> anyhow::Result<()> {+ let accepted = shell_sequence_id(window_id).is_some_and(|id| {+ trusted_gdbus_call("Activate", &[id.to_string()], deadline)+ .is_some_and(|output| output.trim_start().starts_with("(true,"))+ });+ let confirmed = accepted+ && focus_confirmed(deadline, |deadline| {+ let observed = match trusted_focus(deadline) {+ Ok(focused) => crate::interaction::facts::Fact::Known(focused),+ Err(error) => crate::interaction::facts::Fact::Unknown(error.to_string()),+ };+ crate::interaction::policy::focused(&(Some(pid), window_id), &observed)+ });+ if !confirmed {+ anyhow::bail!("GNOME Shell did not confirm activation of exact window {window_id}");}+ Ok(())}-fn window_is_focused(window_id: u32) -> bool {- let Some(raw) = gdbus_call("GetRects", &[]) else {+fn window_is_focused(window_id: u32, deadline: Instant) -> bool {+ let Some(raw) = gdbus_call("GetRects", &[], deadline) else {return false;};parse_shell_windows(&raw, None).is_some_and(|windows| exact_window_focused(&windows, window_id))@@ -700,17 +664,29 @@ fn parse_shell_windows(raw: &str, filter_pid: Option<u32>) -> Option<Vec<ShellWi/// Glide the agent cursor to screen `(x, y)`.pub fn move_cursor(x: i32, y: i32) {- let _ = gdbus_call("MoveCursor", &[x.to_string(), y.to_string()]);+ let _ = gdbus_call(+ "MoveCursor",+ &[x.to_string(), y.to_string()],+ operation_deadline(CALL_CAP),+ );}/// Snap + pulse the agent cursor at screen `(x, y)` (a click indicator).pub fn click_pulse(x: i32, y: i32) {- let _ = gdbus_call("ClickPulse", &[x.to_string(), y.to_string()]);+ let _ = gdbus_call(+ "ClickPulse",+ &[x.to_string(), y.to_string()],+ operation_deadline(CALL_CAP),+ );}/// Set the stable session-specific fill color for the compositor cursor.pub fn set_cursor_color(fill_color: &str) {- let _ = gdbus_call("SetCursorColor", &[fill_color.to_owned()]);+ let _ = gdbus_call(+ "SetCursorColor",+ &[fill_color.to_owned()],+ operation_deadline(CALL_CAP),+ );}/// Update the compositor-owned cursor's semantic action state.@@ -726,17 +702,22 @@ pub fn set_cursor_state(action: &str, delivery: &str, target: &str, active: booltarget.to_owned(),active.to_string(),],+ operation_deadline(CALL_CAP),);}/// Set the renderer-visible public session label for the compositor cursor.pub fn set_session_label(label: &str) {- let _ = gdbus_call("SetSessionLabel", &[label.to_owned()]);+ let _ = gdbus_call(+ "SetSessionLabel",+ &[label.to_owned()],+ operation_deadline(CALL_CAP),+ );}/// Hide the agent cursor.pub fn hide_cursor() {- let _ = gdbus_call("HideCursor", &[]);+ let _ = gdbus_call("HideCursor", &[], operation_deadline(CALL_CAP));}#[cfg(test)]@@ -1066,12 +1047,9 @@ mod tests {for windows in [&siblings[..], &reversed[..]] {assert_eq!(exact_shell_window(windows, 6079, 438).unwrap().info.x, 600);assert_eq!(exact_shell_window(windows, 6079, 363).unwrap().info.x, 0);- assert_eq!(plan_focus(windows, 6079, 438).unwrap(), Some(47));- assert_eq!(plan_focus(windows, 6079, 363).unwrap(), Some(47));- assert_eq!(plan_focus(windows, 6080, 47).unwrap(), None);+ assert_eq!(focused_window(windows).unwrap(), Some((Some(6080), 47)));for (pid, id) in [(6079, 999), (6080, 438), (6079, 6079 << 16), (6079, 0)] {assert!(exact_shell_window(windows, pid, id).is_none());- assert!(plan_focus(windows, pid, id).is_err());}}let duplicate = [@@ -1079,13 +1057,15 @@ mod tests {shell_window(438, 6079, 600, false),shell_window(47, 6080, 1200, true),];- assert!(plan_focus(&duplicate, 6079, 438).is_err());+ assert!(exact_shell_window(&duplicate, 6079, 438).is_none());let nothing_focused = [shell_window(438, 6079, 600, false)];- assert!(plan_focus(¬hing_focused, 6079, 438).is_err());- // An ID wider than a Shell sequence never plans activation, even if a- // malformed snapshot lists it as already focused.- let wide = [shell_window(1 << 33, 6079, 0, true)];- assert!(plan_focus(&wide, 6079, 1 << 33).is_err());+ assert_eq!(focused_window(¬hing_focused).unwrap(), None);+ // A focused ID listed twice cannot name the prior focus to restore.+ let duplicate_focus = [+ shell_window(47, 6080, 0, true),+ shell_window(47, 6080, 1200, false),+ ];+ assert!(focused_window(&duplicate_focus).is_err());}#[test]@@ -1115,7 +1095,8 @@ mod tests {let handle = 3_592_163u64 << 16;assert_eq!(shell_sequence_id(handle), None);assert!(!activate_window(handle));- assert!(!trusted_activate_window(handle));+ let deadline = Instant::now() + Duration::from_secs(1);+ assert!(trusted_focus_window(3_592_163, handle, deadline).is_err());assert_eq!(shell_sequence_id(u64::from(u32::MAX)), Some(u32::MAX));assert_eq!(shell_sequence_id(33), Some(33));}diff --git a/libs/cua-driver/rust/crates/platform-linux/src/wayland/shell_helper/transport.rs b/libs/cua-driver/rust/crates/platform-linux/src/wayland/shell_helper/transport.rsindex b40bd37117..b4d6af5c7c 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/wayland/shell_helper/transport.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/wayland/shell_helper/transport.rs@@ -1,7 +1,7 @@//! Persistent session-bus transport for synchronous compositor helpers.//! A separate runtime also serves callers already inside Tokio without nested block_on.use std::sync::{mpsc, OnceLock};-use std::time::Duration;+use std::time::Instant;use tokio::sync::Mutex;use zbus::{Connection, Message};@@ -23,8 +23,12 @@ pub(super) fn call(path: &str,method: &str,args: &[String],- timeout: Duration,+ deadline: Instant,) -> Option<String> {+ let timeout = deadline.saturating_duration_since(Instant::now());+ if timeout.is_zero() {+ return None;+ }let destination = destination.to_owned();let path = path.to_owned();let method = method.to_owned();@@ -56,7 +60,7 @@ pub(super) fn call(}let _ = tx.send(result.ok());});- rx.recv_timeout(timeout + Duration::from_millis(100))+ rx.recv_timeout(deadline.saturating_duration_since(Instant::now())).ok().flatten()}diff --git a/libs/cua-driver/rust/crates/platform-linux/src/wayland/sway_ipc.rs b/libs/cua-driver/rust/crates/platform-linux/src/wayland/sway_ipc.rsindex ca2e538a3a..eabf70b2fb 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/wayland/sway_ipc.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/wayland/sway_ipc.rs@@ -1,13 +1,110 @@-//! Best-effort Sway/i3-compatible compositor metadata.+//! Sway metadata and focus over the native i3-ipc socket.//!//! The wlroots foreign-toplevel protocol exposes titles and app ids, but not//! process ids or geometry. Sway's IPC tree supplies those missing fields and-//! a compositor-stable container id. Other compositors simply return no data.+//! a compositor-stable container id. Each request opens a connection to+//! `SWAYSOCK`, checks that its peer is the process serving `WAYLAND_DISPLAY`,+//! and finishes by its deadline. Other compositors, or a socket that fails+//! that check, return no data.-use std::process::{Command, Stdio};+use std::io::{Read, Write};+use std::os::fd::AsRawFd;+use std::os::unix::net::UnixStream;+use std::time::{Duration, Instant};+use anyhow::{bail, Context, Result};use serde::Deserialize;+use super::compositor_peer::{peer_pid, wayland_socket_with_timeout};++const MAGIC: &[u8; 6] = b"i3-ipc";+const RUN_COMMAND: u32 = 0;+const GET_TREE: u32 = 4;+const QUERY_CAP: Duration = Duration::from_secs(1);+const MAX_REPLY_BYTES: usize = 16 * 1024 * 1024;++fn remaining(deadline: Instant) -> Result<Duration> {+ let left = deadline.saturating_duration_since(Instant::now());+ if left.is_zero() {+ bail!("Sway IPC timed out");+ }+ Ok(left)+}++fn frame(kind: u32, payload: &[u8]) -> Vec<u8> {+ let mut bytes = Vec::with_capacity(14 + payload.len());+ bytes.extend_from_slice(MAGIC);+ bytes.extend_from_slice(&(payload.len() as u32).to_ne_bytes());+ bytes.extend_from_slice(&kind.to_ne_bytes());+ bytes.extend_from_slice(payload);+ bytes+}++fn read_reply(stream: &mut impl Read, kind: u32) -> Result<Vec<u8>> {+ let mut header = [0u8; 14];+ stream+ .read_exact(&mut header)+ .context("Sway IPC reply unavailable")?;+ if &header[..6] != MAGIC {+ bail!("Sway IPC reply has no i3-ipc header");+ }+ let length = u32::from_ne_bytes(header[6..10].try_into()?) as usize;+ if u32::from_ne_bytes(header[10..14].try_into()?) != kind {+ bail!("Sway IPC replied to a different request");+ }+ if length > MAX_REPLY_BYTES {+ bail!("Sway IPC reply exceeds size limit");+ }+ let mut payload = vec![0u8; length];+ stream+ .read_exact(&mut payload)+ .context("Sway IPC reply truncated")?;+ Ok(payload)+}++/// Send one `kind` request over a newly attested `SWAYSOCK` connection and+/// read its reply by `deadline`.+fn request(kind: u32, payload: &[u8], deadline: Instant) -> Result<Vec<u8>> {+ let path = std::env::var_os("SWAYSOCK").context("missing SWAYSOCK")?;+ let socket = socket2::Socket::new(socket2::Domain::UNIX, socket2::Type::STREAM, None)?;+ socket.connect_timeout(&socket2::SockAddr::unix(path)?, remaining(deadline)?)?;+ let mut stream: UnixStream = socket.into();+ let wayland = wayland_socket_with_timeout(remaining(deadline)?)?;+ if peer_pid(stream.as_raw_fd())? != peer_pid(wayland.as_raw_fd())? {+ bail!("SWAYSOCK and WAYLAND_DISPLAY name different compositor processes");+ }+ stream.set_write_timeout(Some(remaining(deadline)?))?;+ stream.write_all(&frame(kind, payload))?;+ stream.set_read_timeout(Some(remaining(deadline)?))?;+ read_reply(&mut stream, kind)+}++#[derive(Deserialize)]+struct Outcome {+ success: bool,+ #[serde(default)]+ error: Option<String>,+}++fn command_outcome(command: &str, reply: &[u8]) -> Result<()> {+ let outcomes: Vec<Outcome> =+ serde_json::from_slice(reply).context("invalid Sway command reply")?;+ if outcomes.is_empty() {+ bail!("Sway ran no command for `{command}`");+ }+ if let Some(failed) = outcomes.iter().find(|outcome| !outcome.success) {+ bail!(+ "Sway refused `{command}`: {}",+ failed.error.as_deref().unwrap_or("no reason given")+ );+ }+ Ok(())+}++fn tree(deadline: Instant) -> Result<Vec<Window>> {+ parse_tree(&request(GET_TREE, b"", deadline)?).context("invalid Sway IPC tree")+}+#[derive(Clone, Debug, Default, Deserialize)]struct Rect {x: i32,@@ -117,16 +214,7 @@ fn parse_tree(bytes: &[u8]) -> Option<Vec<Window>> {pub fn list_windows() -> Option<Vec<Window>> {std::env::var_os("SWAYSOCK")?;- let output = Command::new("swaymsg")- .args(["-r", "-t", "get_tree"])- .stdin(Stdio::null())- .stderr(Stdio::null())- .output()- .ok()?;- if !output.status.success() {- return None;- }- parse_tree(&output.stdout)+ tree(crate::interaction::operation_deadline(QUERY_CAP)).ok()}pub fn window_for_id(id: u64) -> Option<Window> {@@ -169,69 +257,34 @@ pub fn window_for_app_id(app_id: &str) -> Option<Window> {.max_by_key(|window| (window.focused, window.visible))}-fn focus_container(id: u64) -> bool {- let selector = format!("[con_id={id}]");- Command::new("swaymsg")- .args([selector.as_str(), "focus"])- .stdin(Stdio::null())- .stdout(Stdio::null())- .stderr(Stdio::null())- .status()- .is_ok_and(|status| status.success())-}--fn wait_for_container_focus(id: u64, timeout: std::time::Duration) -> bool {- let deadline = std::time::Instant::now() + timeout;- loop {- if window_for_id(id).is_some_and(|window| window.focused) {- return true;- }- if std::time::Instant::now() >= deadline {- return false;- }- std::thread::sleep(std::time::Duration::from_millis(10));- }-}--/// Briefly focus one compositor-attested container, run `body`, then restore-/// the previously focused container. The caller must already have verified the-/// target container's PID and id through [`window_for_id`].-pub fn with_focused_container<T>(- id: u64,- body: impl FnOnce() -> anyhow::Result<T>,-) -> anyhow::Result<T> {- let prior = list_windows()- .ok_or_else(|| anyhow::anyhow!("Sway IPC tree is unavailable"))?+/// The focused container's PID and id by `deadline`, `None` when no window+/// has focus.+pub fn focused(deadline: Instant) -> Result<Option<(u32, u64)>> {+ Ok(tree(deadline)?.into_iter().find(|window| window.focused)- .map(|window| window.id);- if !focus_container(id) {- anyhow::bail!("Sway refused to focus exact container {id}");- }- if !wait_for_container_focus(id, std::time::Duration::from_millis(500)) {- anyhow::bail!("Sway did not confirm focus on exact container {id}");- }- let result = body();- let restore = prior- .filter(|prior| *prior != id)- .map(|prior| {- if !focus_container(prior) {- anyhow::bail!("Sway could not restore prior container {prior}");- }- if !wait_for_container_focus(prior, std::time::Duration::from_millis(500)) {- anyhow::bail!("Sway did not confirm restored container {prior}");- }- Ok(())- })- .transpose();- match (result, restore) {- (Ok(value), Ok(_)) => Ok(value),- (Err(error), Ok(_)) => Err(error),- (Ok(_), Err(error)) => Err(error),- (Err(error), Err(restore)) => Err(error.context(format!(- "the prior Sway focus also could not be restored: {restore}"- ))),+ .map(|window| (window.pid, window.id)))+}++/// Focus exact container `id` of `pid` and read the focus back by+/// `deadline`.+pub fn focus(pid: u32, id: u64, deadline: Instant) -> Result<()> {+ let command = format!("[con_id={id}] focus");+ request(RUN_COMMAND, command.as_bytes(), deadline)+ .and_then(|reply| command_outcome(&command, &reply))+ .with_context(|| format!("Sway refused to focus exact container {id}"))?;+ let confirmed = crate::interaction::confirm(deadline, |deadline| {+ let observed = match focused(deadline) {+ Ok(focused) => crate::interaction::facts::Fact::Known(focused),+ Err(error) => crate::interaction::facts::Fact::Unknown(error.to_string()),+ };+ Ok::<_, ()>(crate::interaction::policy::focused(&(pid, id), &observed))+ })+ .unwrap_or(false);+ if !confirmed {+ bail!("Sway did not confirm focus on exact container {id}");}+ Ok(())}pub fn window_origin_for_pid(pid: u32) -> Option<(i32, i32)> {@@ -247,6 +300,34 @@ pub fn window_origin_for_title(title: &str) -> Option<(i32, i32)> {mod tests {use super::*;+ #[test]+ fn frames_round_trip_and_reject_foreign_headers() {+ let framed = frame(GET_TREE, b"{}");+ assert_eq!(&framed[..6], MAGIC);+ assert_eq!(read_reply(&mut &framed[..], GET_TREE).unwrap(), b"{}");+ assert!(read_reply(&mut &framed[..], RUN_COMMAND).is_err());+ assert!(read_reply(&mut &framed[..framed.len() - 1], GET_TREE).is_err());+ let mut foreign = framed.clone();+ foreign[0] = b'x';+ assert!(read_reply(&mut &foreign[..], GET_TREE).is_err());+ let mut oversized = frame(GET_TREE, b"");+ oversized[6..10].copy_from_slice(&(MAX_REPLY_BYTES as u32 + 1).to_ne_bytes());+ assert!(read_reply(&mut &oversized[..], GET_TREE).is_err());+ }++ #[test]+ fn a_command_succeeds_only_when_every_outcome_succeeds() {+ assert!(command_outcome("focus", br#"[{"success":true}]"#).is_ok());+ let refused = command_outcome(+ "focus",+ br#"[{"success":true},{"success":false,"error":"No matching node"}]"#,+ )+ .unwrap_err();+ assert!(refused.to_string().contains("No matching node"));+ assert!(command_outcome("focus", b"[]").is_err());+ assert!(command_outcome("focus", b"not json").is_err());+ }+#[test]fn parses_nested_and_floating_windows() {let tree = br#"{diff --git a/libs/cua-driver/rust/crates/platform-linux/src/wayland/virtual_keyboard.rs b/libs/cua-driver/rust/crates/platform-linux/src/wayland/virtual_keyboard.rsindex 9a1dabb523..4a1d46ee80 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/wayland/virtual_keyboard.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/wayland/virtual_keyboard.rs@@ -11,7 +11,7 @@ use wayland_client::{Connection, Dispatch, Proxy, QueueHandle};use self::protocol::zwp_virtual_keyboard_manager_v1::ZwpVirtualKeyboardManagerV1;use self::protocol::zwp_virtual_keyboard_v1::ZwpVirtualKeyboardV1;-mod protocol {+pub(super) mod protocol {use wayland_client;use wayland_client::protocol::*;@@ -160,7 +160,7 @@ fn send_key() -> anyhow::Result<()> {keyboard.key(super::event_time_ms(), keycode, key_state);queue.roundtrip(state)?;- std::thread::sleep(std::time::Duration::from_millis(4));+ crate::interaction::pause(std::time::Duration::from_millis(4));Ok(())}diff --git a/libs/cua-driver/rust/crates/platform-linux/src/x11/mod.rs b/libs/cua-driver/rust/crates/platform-linux/src/x11/mod.rsindex 166cd512da..90e944ade3 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/x11/mod.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/x11/mod.rs@@ -355,6 +355,7 @@ pub fn set_window_frame(let requested = (x, y, width, height);let mut observed = None;+ let deadline = crate::interaction::operation_deadline(std::time::Duration::from_millis(320));for _ in 0..8 {observed = list_windows(Some(pid)).into_iter()@@ -365,7 +366,13 @@ pub fn set_window_frame({break;}- std::thread::sleep(std::time::Duration::from_millis(40));+ if std::time::Instant::now() >= deadline {+ break;+ }+ std::thread::sleep(+ std::time::Duration::from_millis(40)+ .min(deadline.saturating_duration_since(std::time::Instant::now())),+ );}let confirmed = observed.as_ref()diff --git a/libs/cua-driver/rust/crates/platform-linux/tests/fixtures/niri-us-de.xkb b/libs/cua-driver/rust/crates/platform-linux/tests/fixtures/niri-us-de.xkbnew file mode 100644index 0000000000..bcc1135727--- /dev/null+++ b/libs/cua-driver/rust/crates/platform-linux/tests/fixtures/niri-us-de.xkb@@ -0,0 +1,1723 @@+xkb_keymap {+xkb_keycodes "evdev_aliases(qwerty)" {+ minimum = 8;+ maximum = 709;+ <ESC> = 9;+ <AE01> = 10;+ <AE02> = 11;+ <AE03> = 12;+ <AE04> = 13;+ <AE05> = 14;+ <AE06> = 15;+ <AE07> = 16;+ <AE08> = 17;+ <AE09> = 18;+ <AE10> = 19;+ <AE11> = 20;+ <AE12> = 21;+ <BKSP> = 22;+ <TAB> = 23;+ <AD01> = 24;+ <AD02> = 25;+ <AD03> = 26;+ <AD04> = 27;+ <AD05> = 28;+ <AD06> = 29;+ <AD07> = 30;+ <AD08> = 31;+ <AD09> = 32;+ <AD10> = 33;+ <AD11> = 34;+ <AD12> = 35;+ <RTRN> = 36;+ <LCTL> = 37;+ <AC01> = 38;+ <AC02> = 39;+ <AC03> = 40;+ <AC04> = 41;+ <AC05> = 42;+ <AC06> = 43;+ <AC07> = 44;+ <AC08> = 45;+ <AC09> = 46;+ <AC10> = 47;+ <AC11> = 48;+ <TLDE> = 49;+ <LFSH> = 50;+ <BKSL> = 51;+ <AB01> = 52;+ <AB02> = 53;+ <AB03> = 54;+ <AB04> = 55;+ <AB05> = 56;+ <AB06> = 57;+ <AB07> = 58;+ <AB08> = 59;+ <AB09> = 60;+ <AB10> = 61;+ <RTSH> = 62;+ <KPMU> = 63;+ <LALT> = 64;+ <SPCE> = 65;+ <CAPS> = 66;+ <FK01> = 67;+ <FK02> = 68;+ <FK03> = 69;+ <FK04> = 70;+ <FK05> = 71;+ <FK06> = 72;+ <FK07> = 73;+ <FK08> = 74;+ <FK09> = 75;+ <FK10> = 76;+ <NMLK> = 77;+ <SCLK> = 78;+ <KP7> = 79;+ <KP8> = 80;+ <KP9> = 81;+ <KPSU> = 82;+ <KP4> = 83;+ <KP5> = 84;+ <KP6> = 85;+ <KPAD> = 86;+ <KP1> = 87;+ <KP2> = 88;+ <KP3> = 89;+ <KP0> = 90;+ <KPDL> = 91;+ <LVL3> = 92;+ <ZEHA> = 93;+ <LSGT> = 94;+ <FK11> = 95;+ <FK12> = 96;+ <AB11> = 97;+ <KATA> = 98;+ <HIRA> = 99;+ <HENK> = 100;+ <HKTG> = 101;+ <MUHE> = 102;+ <JPCM> = 103;+ <KPEN> = 104;+ <RCTL> = 105;+ <KPDV> = 106;+ <PRSC> = 107;+ <RALT> = 108;+ <LNFD> = 109;+ <HOME> = 110;+ <UP> = 111;+ <PGUP> = 112;+ <LEFT> = 113;+ <RGHT> = 114;+ <END> = 115;+ <DOWN> = 116;+ <PGDN> = 117;+ <INS> = 118;+ <DELE> = 119;+ <I120> = 120;+ <MUTE> = 121;+ <VOL-> = 122;+ <VOL+> = 123;+ <POWR> = 124;+ <KPEQ> = 125;+ <I126> = 126;+ <PAUS> = 127;+ <I128> = 128;+ <I129> = 129;+ <HNGL> = 130;+ <HJCV> = 131;+ <AE13> = 132;+ <LWIN> = 133;+ <RWIN> = 134;+ <COMP> = 135;+ <STOP> = 136;+ <AGAI> = 137;+ <PROP> = 138;+ <UNDO> = 139;+ <FRNT> = 140;+ <COPY> = 141;+ <OPEN> = 142;+ <PAST> = 143;+ <FIND> = 144;+ <CUT> = 145;+ <HELP> = 146;+ <I147> = 147;+ <I148> = 148;+ <I149> = 149;+ <I150> = 150;+ <I151> = 151;+ <I152> = 152;+ <I153> = 153;+ <I154> = 154;+ <I155> = 155;+ <I156> = 156;+ <I157> = 157;+ <I158> = 158;+ <I159> = 159;+ <I160> = 160;+ <I161> = 161;+ <I162> = 162;+ <I163> = 163;+ <I164> = 164;+ <I165> = 165;+ <I166> = 166;+ <I167> = 167;+ <I168> = 168;+ <I169> = 169;+ <I170> = 170;+ <I171> = 171;+ <I172> = 172;+ <I173> = 173;+ <I174> = 174;+ <I175> = 175;+ <I176> = 176;+ <I177> = 177;+ <I178> = 178;+ <I179> = 179;+ <I180> = 180;+ <I181> = 181;+ <I182> = 182;+ <I183> = 183;+ <I184> = 184;+ <I185> = 185;+ <I186> = 186;+ <I187> = 187;+ <I188> = 188;+ <I189> = 189;+ <I190> = 190;+ <FK13> = 191;+ <FK14> = 192;+ <FK15> = 193;+ <FK16> = 194;+ <FK17> = 195;+ <FK18> = 196;+ <FK19> = 197;+ <FK20> = 198;+ <FK21> = 199;+ <FK22> = 200;+ <FK23> = 201;+ <FK24> = 202;+ <LVL5> = 203;+ <ALT> = 204;+ <META> = 205;+ <SUPR> = 206;+ <HYPR> = 207;+ <I208> = 208;+ <I209> = 209;+ <I210> = 210;+ <I211> = 211;+ <I212> = 212;+ <I213> = 213;+ <I214> = 214;+ <I215> = 215;+ <I216> = 216;+ <I217> = 217;+ <I218> = 218;+ <I219> = 219;+ <I220> = 220;+ <I221> = 221;+ <I222> = 222;+ <I223> = 223;+ <I224> = 224;+ <I225> = 225;+ <I226> = 226;+ <I227> = 227;+ <I228> = 228;+ <I229> = 229;+ <I230> = 230;+ <I231> = 231;+ <I232> = 232;+ <I233> = 233;+ <I234> = 234;+ <I235> = 235;+ <I236> = 236;+ <I237> = 237;+ <I238> = 238;+ <I239> = 239;+ <I240> = 240;+ <I241> = 241;+ <I242> = 242;+ <I243> = 243;+ <I244> = 244;+ <I245> = 245;+ <I246> = 246;+ <I247> = 247;+ <I248> = 248;+ <I249> = 249;+ <I250> = 250;+ <I251> = 251;+ <I252> = 252;+ <I253> = 253;+ <I254> = 254;+ <I255> = 255;+ <I256> = 256;+ <I360> = 360;+ <I361> = 361;+ <I362> = 362;+ <I363> = 363;+ <I364> = 364;+ <I365> = 365;+ <I366> = 366;+ <I367> = 367;+ <I368> = 368;+ <I369> = 369;+ <I370> = 370;+ <I371> = 371;+ <I372> = 372;+ <I373> = 373;+ <I374> = 374;+ <I375> = 375;+ <I376> = 376;+ <I377> = 377;+ <I378> = 378;+ <I379> = 379;+ <I380> = 380;+ <I381> = 381;+ <I382> = 382;+ <I383> = 383;+ <I384> = 384;+ <I385> = 385;+ <I386> = 386;+ <I387> = 387;+ <I388> = 388;+ <I389> = 389;+ <I390> = 390;+ <I391> = 391;+ <I392> = 392;+ <I393> = 393;+ <I394> = 394;+ <I395> = 395;+ <I396> = 396;+ <I397> = 397;+ <I398> = 398;+ <I399> = 399;+ <I400> = 400;+ <I401> = 401;+ <I402> = 402;+ <I403> = 403;+ <I404> = 404;+ <I405> = 405;+ <I406> = 406;+ <I407> = 407;+ <I408> = 408;+ <I409> = 409;+ <I410> = 410;+ <I411> = 411;+ <I412> = 412;+ <I413> = 413;+ <I414> = 414;+ <I415> = 415;+ <I416> = 416;+ <I417> = 417;+ <I418> = 418;+ <I419> = 419;+ <I420> = 420;+ <I421> = 421;+ <I422> = 422;+ <I423> = 423;+ <I424> = 424;+ <I425> = 425;+ <I426> = 426;+ <I427> = 427;+ <I428> = 428;+ <I429> = 429;+ <I430> = 430;+ <I431> = 431;+ <I432> = 432;+ <I433> = 433;+ <I434> = 434;+ <I435> = 435;+ <I436> = 436;+ <I437> = 437;+ <I438> = 438;+ <I439> = 439;+ <I440> = 440;+ <I441> = 441;+ <I442> = 442;+ <I443> = 443;+ <I444> = 444;+ <I445> = 445;+ <I446> = 446;+ <I447> = 447;+ <I448> = 448;+ <I449> = 449;+ <I450> = 450;+ <I452> = 452;+ <I453> = 453;+ <I454> = 454;+ <I455> = 455;+ <I456> = 456;+ <I457> = 457;+ <I458> = 458;+ <I459> = 459;+ <I472> = 472;+ <I473> = 473;+ <I474> = 474;+ <I475> = 475;+ <I476> = 476;+ <I477> = 477;+ <I478> = 478;+ <I479> = 479;+ <I480> = 480;+ <I481> = 481;+ <I482> = 482;+ <I483> = 483;+ <I484> = 484;+ <I485> = 485;+ <I486> = 486;+ <I487> = 487;+ <I488> = 488;+ <I489> = 489;+ <I490> = 490;+ <I491> = 491;+ <I492> = 492;+ <I493> = 493;+ <I505> = 505;+ <I506> = 506;+ <I507> = 507;+ <I508> = 508;+ <I509> = 509;+ <I510> = 510;+ <I511> = 511;+ <I512> = 512;+ <I513> = 513;+ <I514> = 514;+ <I520> = 520;+ <I521> = 521;+ <I522> = 522;+ <I523> = 523;+ <I524> = 524;+ <I525> = 525;+ <I526> = 526;+ <I527> = 527;+ <I528> = 528;+ <I529> = 529;+ <I530> = 530;+ <I531> = 531;+ <I532> = 532;+ <I533> = 533;+ <I534> = 534;+ <I535> = 535;+ <I536> = 536;+ <I537> = 537;+ <I538> = 538;+ <I539> = 539;+ <I540> = 540;+ <I541> = 541;+ <I542> = 542;+ <I543> = 543;+ <I544> = 544;+ <I545> = 545;+ <I546> = 546;+ <I547> = 547;+ <I548> = 548;+ <I549> = 549;+ <I550> = 550;+ <I568> = 568;+ <I569> = 569;+ <I570> = 570;+ <I584> = 584;+ <I585> = 585;+ <I586> = 586;+ <I587> = 587;+ <I588> = 588;+ <I589> = 589;+ <I590> = 590;+ <I591> = 591;+ <I592> = 592;+ <I593> = 593;+ <I594> = 594;+ <I595> = 595;+ <I596> = 596;+ <I597> = 597;+ <I598> = 598;+ <I599> = 599;+ <I600> = 600;+ <I601> = 601;+ <I602> = 602;+ <I603> = 603;+ <I604> = 604;+ <I605> = 605;+ <I606> = 606;+ <I616> = 616;+ <I617> = 617;+ <I618> = 618;+ <I619> = 619;+ <I620> = 620;+ <I621> = 621;+ <I622> = 622;+ <I623> = 623;+ <I624> = 624;+ <I625> = 625;+ <I626> = 626;+ <I627> = 627;+ <I628> = 628;+ <I629> = 629;+ <I630> = 630;+ <I631> = 631;+ <I632> = 632;+ <I633> = 633;+ <I634> = 634;+ <I635> = 635;+ <I636> = 636;+ <I637> = 637;+ <I638> = 638;+ <I639> = 639;+ <I640> = 640;+ <I641> = 641;+ <I642> = 642;+ <I643> = 643;+ <I644> = 644;+ <I645> = 645;+ <I646> = 646;+ <I647> = 647;+ <I648> = 648;+ <I649> = 649;+ <I650> = 650;+ <I651> = 651;+ <I652> = 652;+ <I653> = 653;+ <I654> = 654;+ <I655> = 655;+ <I656> = 656;+ <I657> = 657;+ <I664> = 664;+ <I665> = 665;+ <I666> = 666;+ <I667> = 667;+ <I668> = 668;+ <I669> = 669;+ <I670> = 670;+ <I671> = 671;+ <I672> = 672;+ <I673> = 673;+ <I674> = 674;+ <I675> = 675;+ <I676> = 676;+ <I677> = 677;+ <I678> = 678;+ <I679> = 679;+ <I680> = 680;+ <I681> = 681;+ <I682> = 682;+ <I683> = 683;+ <I684> = 684;+ <I685> = 685;+ <I686> = 686;+ <I687> = 687;+ <I688> = 688;+ <I689> = 689;+ <I690> = 690;+ <I691> = 691;+ <I692> = 692;+ <I693> = 693;+ <I696> = 696;+ <I697> = 697;+ <I698> = 698;+ <I699> = 699;+ <I700> = 700;+ <I701> = 701;+ <I704> = 704;+ <I705> = 705;+ <I706> = 706;+ <I707> = 707;+ <I708> = 708;+ <I709> = 709;+ indicator 1 = "Caps Lock";+ indicator 2 = "Num Lock";+ indicator 3 = "Scroll Lock";+ indicator 4 = "Compose";+ indicator 5 = "Kana";+ indicator 6 = "Sleep";+ indicator 7 = "Suspend";+ indicator 8 = "Mute";+ indicator 9 = "Misc";+ indicator 10 = "Mail";+ indicator 11 = "Charging";+ indicator 12 = "Shift Lock";+ indicator 13 = "Group 2";+ indicator 14 = "Mouse Keys";+ alias <AC12> = <BKSL>;+ alias <ALGR> = <RALT>;+ alias <MENU> = <COMP>;+ alias <HZTG> = <TLDE>;+ alias <LMTA> = <LWIN>;+ alias <RMTA> = <RWIN>;+ alias <OUTP> = <I235>;+ alias <KITG> = <I236>;+ alias <KIDN> = <I237>;+ alias <KIUP> = <I238>;+ alias <I121> = <MUTE>;+ alias <I122> = <VOL->;+ alias <I123> = <VOL+>;+ alias <I124> = <POWR>;+ alias <I125> = <KPEQ>;+ alias <I127> = <PAUS>;+ alias <I130> = <HNGL>;+ alias <I131> = <HJCV>;+ alias <I132> = <AE13>;+ alias <I133> = <LWIN>;+ alias <I134> = <RWIN>;+ alias <I135> = <COMP>;+ alias <I136> = <STOP>;+ alias <I137> = <AGAI>;+ alias <I138> = <PROP>;+ alias <I139> = <UNDO>;+ alias <I140> = <FRNT>;+ alias <I141> = <COPY>;+ alias <I142> = <OPEN>;+ alias <I143> = <PAST>;+ alias <I144> = <FIND>;+ alias <I145> = <CUT>;+ alias <I146> = <HELP>;+ alias <I191> = <FK13>;+ alias <I192> = <FK14>;+ alias <I193> = <FK15>;+ alias <I194> = <FK16>;+ alias <I195> = <FK17>;+ alias <I196> = <FK18>;+ alias <I197> = <FK19>;+ alias <I198> = <FK20>;+ alias <I199> = <FK21>;+ alias <I200> = <FK22>;+ alias <I201> = <FK23>;+ alias <I202> = <FK24>;+ alias <MDSW> = <LVL5>;+ alias <KPPT> = <I129>;+ alias <LatQ> = <AD01>;+ alias <LatW> = <AD02>;+ alias <LatE> = <AD03>;+ alias <LatR> = <AD04>;+ alias <LatT> = <AD05>;+ alias <LatY> = <AD06>;+ alias <LatU> = <AD07>;+ alias <LatI> = <AD08>;+ alias <LatO> = <AD09>;+ alias <LatP> = <AD10>;+ alias <LatA> = <AC01>;+ alias <LatS> = <AC02>;+ alias <LatD> = <AC03>;+ alias <LatF> = <AC04>;+ alias <LatG> = <AC05>;+ alias <LatH> = <AC06>;+ alias <LatJ> = <AC07>;+ alias <LatK> = <AC08>;+ alias <LatL> = <AC09>;+ alias <LatZ> = <AB01>;+ alias <LatX> = <AB02>;+ alias <LatC> = <AB03>;+ alias <LatV> = <AB04>;+ alias <LatB> = <AB05>;+ alias <LatN> = <AB06>;+ alias <LatM> = <AB07>;+};++xkb_types "complete" {+ virtual_modifiers NumLock,Alt,LevelThree,Super,LevelFive,Meta,Hyper=0x4000,ScrollLock=0x8000;++ type "ONE_LEVEL" {+ modifiers= none;+ level_name[1]= "Any";+ };+ type "TWO_LEVEL" {+ modifiers= Shift;+ map[Shift]= 2;+ level_name[1]= "Base";+ level_name[2]= "Shift";+ };+ type "ALPHABETIC" {+ modifiers= Shift+Lock;+ map[Shift]= 2;+ map[Lock]= 2;+ level_name[1]= "Base";+ level_name[2]= "Caps";+ };+ type "PC_ALT_LEVEL2" {+ modifiers= Alt;+ map[Alt]= 2;+ level_name[1]= "Base";+ level_name[2]= "Alt";+ };+ type "PC_SHIFT_SUPER_LEVEL2" {+ modifiers= Shift+Super;+ map[Shift+Super]= 2;+ map[Shift]= 1;+ preserve[Shift]= Shift;+ map[Super]= 1;+ preserve[Super]= Super;+ level_name[1]= "Base";+ level_name[2]= "Super";+ };+ type "PC_CONTROL_LEVEL2" {+ modifiers= Control;+ map[Control]= 2;+ level_name[1]= "Base";+ level_name[2]= "Control";+ };+ type "PC_CONTROL_SUPER_LEVEL2" {+ modifiers= Control+Super;+ map[Control+Super]= 2;+ map[Control]= 1;+ preserve[Control]= Control;+ map[Super]= 1;+ preserve[Super]= Super;+ level_name[1]= "Base";+ level_name[2]= "Control + Super";+ };+ type "CTRL+ALT" {+ modifiers= Shift+Control+Alt+LevelThree;+ map[Shift]= 2;+ map[LevelThree]= 3;+ map[Shift+LevelThree]= 4;+ map[Control]= 1;+ preserve[Control]= Control;+ map[Shift+Control]= 2;+ preserve[Shift+Control]= Control;+ map[Control+LevelThree]= 3;+ preserve[Control+LevelThree]= Control;+ map[Shift+Control+LevelThree]= 4;+ preserve[Shift+Control+LevelThree]= Control;+ map[Alt]= 1;+ preserve[Alt]= Alt;+ map[Shift+Alt]= 2;+ preserve[Shift+Alt]= Alt;+ map[Alt+LevelThree]= 3;+ preserve[Alt+LevelThree]= Alt;+ map[Shift+Alt+LevelThree]= 4;+ preserve[Shift+Alt+LevelThree]= Alt;+ map[Control+Alt]= 5;+ map[Shift+Control+Alt]= 2;+ preserve[Shift+Control+Alt]= Control+Alt;+ map[Control+Alt+LevelThree]= 3;+ preserve[Control+Alt+LevelThree]= Control+Alt;+ map[Shift+Control+Alt+LevelThree]= 4;+ preserve[Shift+Control+Alt+LevelThree]= Control+Alt;+ level_name[1]= "Base";+ level_name[2]= "Shift";+ level_name[3]= "AltGr";+ level_name[4]= "Shift AltGr";+ level_name[5]= "Ctrl+Alt";+ };+ type "FOUR_LEVEL" {+ modifiers= Shift+LevelThree;+ map[Shift]= 2;+ map[LevelThree]= 3;+ map[Shift+LevelThree]= 4;+ level_name[1]= "Base";+ level_name[2]= "Shift";+ level_name[3]= "AltGr";+ level_name[4]= "Shift AltGr";+ };+ type "FOUR_LEVEL_ALPHABETIC" {+ modifiers= Shift+Lock+LevelThree;+ map[Shift]= 2;+ map[Lock]= 2;+ map[LevelThree]= 3;+ map[Shift+LevelThree]= 4;+ map[Lock+LevelThree]= 4;+ map[Shift+Lock+LevelThree]= 3;+ level_name[1]= "Base";+ level_name[2]= "Shift";+ level_name[3]= "AltGr";+ level_name[4]= "Shift AltGr";+ };+ type "FOUR_LEVEL_SEMIALPHABETIC" {+ modifiers= Shift+Lock+LevelThree;+ map[Shift]= 2;+ map[Lock]= 2;+ map[LevelThree]= 3;+ map[Shift+LevelThree]= 4;+ map[Lock+LevelThree]= 3;+ preserve[Lock+LevelThree]= Lock;+ map[Shift+Lock+LevelThree]= 4;+ preserve[Shift+Lock+LevelThree]= Lock;+ level_name[1]= "Base";+ level_name[2]= "Shift";+ level_name[3]= "AltGr";+ level_name[4]= "Shift AltGr";+ };+ type "FOUR_LEVEL_PLUS_LOCK" {+ modifiers= Shift+Lock+LevelThree;+ map[Shift]= 2;+ map[LevelThree]= 3;+ map[Shift+LevelThree]= 4;+ map[Lock]= 5;+ map[Shift+Lock]= 2;+ map[Lock+LevelThree]= 3;+ map[Shift+Lock+LevelThree]= 4;+ level_name[1]= "Base";+ level_name[2]= "Shift";+ level_name[3]= "AltGr";+ level_name[4]= "Shift AltGr";+ level_name[5]= "Lock";+ };+ type "KEYPAD" {+ modifiers= NumLock;+ map[NumLock]= 2;+ level_name[1]= "Base";+ level_name[2]= "Number";+ };+};++xkb_compatibility "complete" {+ virtual_modifiers NumLock,Alt,LevelThree,Super,LevelFive,Meta,Hyper=0x4000,ScrollLock=0x8000;++ interpret.useModMapMods= AnyLevel;+ interpret.repeat= False;+ interpret Num_Lock+AnyOf(all) {+ virtualModifier= NumLock;+ action= LockMods(modifiers=NumLock);+ };+ interpret ISO_Level3_Shift+AnyOf(all) {+ virtualModifier= LevelThree;+ useModMapMods=level1;+ action= SetMods(modifiers=LevelThree,clearLocks);+ };+ interpret Alt_L+AnyOf(all) {+ virtualModifier= Alt;+ action= SetMods(modifiers=modMapMods,clearLocks);+ };+ interpret Alt_R+AnyOf(all) {+ virtualModifier= Alt;+ action= SetMods(modifiers=modMapMods,clearLocks);+ };+ interpret Meta_L+AnyOf(all) {+ virtualModifier= Meta;+ action= SetMods(modifiers=modMapMods,clearLocks);+ };+ interpret Super_L+AnyOf(all) {+ virtualModifier= Super;+ action= SetMods(modifiers=modMapMods,clearLocks);+ };+ interpret Super_R+AnyOf(all) {+ virtualModifier= Super;+ action= SetMods(modifiers=modMapMods,clearLocks);+ };+ interpret ISO_Level5_Shift+AnyOf(all) {+ virtualModifier= LevelFive;+ useModMapMods=level1;+ action= SetMods(modifiers=LevelFive,clearLocks);+ };+ interpret ISO_Next_Group+AnyOfOrNone(all) {+ action= LockGroup(group=+1);+ };+ interpret KP_1+AnyOfOrNone(all) {+ repeat= True;+ action= MovePtr(x=-1,y=+1);+ };+ interpret KP_End+AnyOfOrNone(all) {+ repeat= True;+ action= MovePtr(x=-1,y=+1);+ };+ interpret KP_2+AnyOfOrNone(all) {+ repeat= True;+ action= MovePtr(x=+0,y=+1);+ };+ interpret KP_Down+AnyOfOrNone(all) {+ repeat= True;+ action= MovePtr(x=+0,y=+1);+ };+ interpret KP_3+AnyOfOrNone(all) {+ repeat= True;+ action= MovePtr(x=+1,y=+1);+ };+ interpret KP_Next+AnyOfOrNone(all) {+ repeat= True;+ action= MovePtr(x=+1,y=+1);+ };+ interpret KP_4+AnyOfOrNone(all) {+ repeat= True;+ action= MovePtr(x=-1,y=+0);+ };+ interpret KP_Left+AnyOfOrNone(all) {+ repeat= True;+ action= MovePtr(x=-1,y=+0);+ };+ interpret KP_6+AnyOfOrNone(all) {+ repeat= True;+ action= MovePtr(x=+1,y=+0);+ };+ interpret KP_Right+AnyOfOrNone(all) {+ repeat= True;+ action= MovePtr(x=+1,y=+0);+ };+ interpret KP_7+AnyOfOrNone(all) {+ repeat= True;+ action= MovePtr(x=-1,y=-1);+ };+ interpret KP_Home+AnyOfOrNone(all) {+ repeat= True;+ action= MovePtr(x=-1,y=-1);+ };+ interpret KP_8+AnyOfOrNone(all) {+ repeat= True;+ action= MovePtr(x=+0,y=-1);+ };+ interpret KP_Up+AnyOfOrNone(all) {+ repeat= True;+ action= MovePtr(x=+0,y=-1);+ };+ interpret KP_9+AnyOfOrNone(all) {+ repeat= True;+ action= MovePtr(x=+1,y=-1);+ };+ interpret KP_Prior+AnyOfOrNone(all) {+ repeat= True;+ action= MovePtr(x=+1,y=-1);+ };+ interpret KP_5+AnyOfOrNone(all) {+ repeat= True;+ action= PtrBtn(button=default);+ };+ interpret KP_Begin+AnyOfOrNone(all) {+ repeat= True;+ action= PtrBtn(button=default);+ };+ interpret KP_Divide+AnyOfOrNone(all) {+ repeat= True;+ action= SetPtrDflt(affect=button,button=1);+ };+ interpret KP_Multiply+AnyOfOrNone(all) {+ repeat= True;+ action= SetPtrDflt(affect=button,button=2);+ };+ interpret KP_Subtract+AnyOfOrNone(all) {+ repeat= True;+ action= SetPtrDflt(affect=button,button=3);+ };+ interpret KP_Separator+AnyOfOrNone(all) {+ repeat= True;+ action= PtrBtn(button=default,count=2);+ };+ interpret KP_Add+AnyOfOrNone(all) {+ repeat= True;+ action= PtrBtn(button=default,count=2);+ };+ interpret KP_0+AnyOfOrNone(all) {+ repeat= True;+ action= LockPtrBtn(button=default,affect=lock);+ };+ interpret KP_Insert+AnyOfOrNone(all) {+ repeat= True;+ action= LockPtrBtn(button=default,affect=lock);+ };+ interpret KP_Decimal+AnyOfOrNone(all) {+ repeat= True;+ action= LockPtrBtn(button=default,affect=unlock);+ };+ interpret KP_Delete+AnyOfOrNone(all) {+ repeat= True;+ action= LockPtrBtn(button=default,affect=unlock);+ };+ interpret Hyper_L+AnyOfOrNone(all) {+ action= SetMods(modifiers=Hyper,clearLocks);+ };+ interpret Shift_L+AnyOfOrNone(all) {+ action= SetMods(modifiers=Shift,clearLocks);+ };+ interpret Shift_R+AnyOfOrNone(all) {+ action= SetMods(modifiers=Shift,clearLocks);+ };+ interpret XF86Switch_VT_1+AnyOfOrNone(all) {+ repeat= True;+ action= SwitchScreen(screen=1,!same);+ };+ interpret XF86Switch_VT_2+AnyOfOrNone(all) {+ repeat= True;+ action= SwitchScreen(screen=2,!same);+ };+ interpret XF86Switch_VT_3+AnyOfOrNone(all) {+ repeat= True;+ action= SwitchScreen(screen=3,!same);+ };+ interpret XF86Switch_VT_4+AnyOfOrNone(all) {+ repeat= True;+ action= SwitchScreen(screen=4,!same);+ };+ interpret XF86Switch_VT_5+AnyOfOrNone(all) {+ repeat= True;+ action= SwitchScreen(screen=5,!same);+ };+ interpret XF86Switch_VT_6+AnyOfOrNone(all) {+ repeat= True;+ action= SwitchScreen(screen=6,!same);+ };+ interpret XF86Switch_VT_7+AnyOfOrNone(all) {+ repeat= True;+ action= SwitchScreen(screen=7,!same);+ };+ interpret XF86Switch_VT_8+AnyOfOrNone(all) {+ repeat= True;+ action= SwitchScreen(screen=8,!same);+ };+ interpret XF86Switch_VT_9+AnyOfOrNone(all) {+ repeat= True;+ action= SwitchScreen(screen=9,!same);+ };+ interpret XF86Switch_VT_10+AnyOfOrNone(all) {+ repeat= True;+ action= SwitchScreen(screen=10,!same);+ };+ interpret XF86Switch_VT_11+AnyOfOrNone(all) {+ repeat= True;+ action= SwitchScreen(screen=11,!same);+ };+ interpret XF86Switch_VT_12+AnyOfOrNone(all) {+ repeat= True;+ action= SwitchScreen(screen=12,!same);+ };+ interpret XF86Next_VMode+AnyOfOrNone(all) {+ repeat= True;+ action= Private(type=0x86,data[0]=0x2b,data[1]=0x56,data[2]=0x4d,data[3]=0x6f,data[4]=0x64,data[5]=0x65,data[6]=0x00);+ };+ interpret XF86Prev_VMode+AnyOfOrNone(all) {+ repeat= True;+ action= Private(type=0x86,data[0]=0x2d,data[1]=0x56,data[2]=0x4d,data[3]=0x6f,data[4]=0x64,data[5]=0x65,data[6]=0x00);+ };+ interpret Caps_Lock+AnyOfOrNone(all) {+ action= LockMods(modifiers=Lock);+ };+ interpret Any+AnyOf(all) {+ action= SetMods(modifiers=modMapMods,clearLocks);+ };+ indicator "Caps Lock" {+ whichModState= locked;+ modifiers= Lock;+ };+ indicator "Num Lock" {+ whichModState= locked;+ modifiers= NumLock;+ };+ indicator "Scroll Lock" {+ whichModState= locked;+ modifiers= ScrollLock;+ };+ indicator "Shift Lock" {+ whichModState= locked;+ modifiers= Shift;+ };+ indicator "Group 2" {+ groups= 0xfffffffe;+ };+ indicator "Mouse Keys" {+ controls= MouseKeys;+ };+};++xkb_symbols "pc_us_de_2_inet(evdev)" {+ name[1]="English (US)";+ name[2]="German";++ key <ESC> { [ Escape ] };+ key <AE01> {+ symbols[1]= [ 1, exclam ],+ symbols[2]= [ 1, exclam, onesuperior, exclamdown ]+ };+ key <AE02> {+ symbols[1]= [ 2, at ],+ symbols[2]= [ 2, quotedbl, twosuperior, oneeighth ]+ };+ key <AE03> {+ symbols[1]= [ 3, numbersign ],+ symbols[2]= [ 3, section, threesuperior, sterling ]+ };+ key <AE04> {+ symbols[1]= [ 4, dollar ],+ symbols[2]= [ 4, dollar, onequarter, currency ]+ };+ key <AE05> {+ symbols[1]= [ 5, percent ],+ symbols[2]= [ 5, percent, onehalf, threeeighths ]+ };+ key <AE06> {+ symbols[1]= [ 6, asciicircum ],+ symbols[2]= [ 6, ampersand, notsign, fiveeighths ]+ };+ key <AE07> {+ symbols[1]= [ 7, ampersand ],+ symbols[2]= [ 7, slash, braceleft, seveneighths ]+ };+ key <AE08> {+ symbols[1]= [ 8, asterisk ],+ symbols[2]= [ 8, parenleft, bracketleft, trademark ]+ };+ key <AE09> {+ symbols[1]= [ 9, parenleft ],+ symbols[2]= [ 9, parenright, bracketright, plusminus ]+ };+ key <AE10> {+ symbols[1]= [ 0, parenright ],+ symbols[2]= [ 0, equal, braceright, degree ]+ };+ key <AE11> {+ type[2]= "FOUR_LEVEL_PLUS_LOCK",+ symbols[1]= [ minus, underscore ],+ symbols[2]= [ ssharp, question, backslash, questiondown, SSHARP ]+ };+ key <AE12> {+ symbols[1]= [ equal, plus ],+ symbols[2]= [ dead_acute, dead_grave, dead_cedilla, dead_ogonek ]+ };+ key <BKSP> { [ BackSpace, BackSpace ] };+ key <TAB> { [ Tab, ISO_Left_Tab ] };+ key <AD01> {+ symbols[1]= [ q, Q ],+ symbols[2]= [ q, Q, at, Greek_OMEGA ]+ };+ key <AD02> {+ symbols[1]= [ w, W ],+ symbols[2]= [ w, W, U017F, section ]+ };+ key <AD03> {+ symbols[1]= [ e, E ],+ symbols[2]= [ e, E, EuroSign, EuroSign ]+ };+ key <AD04> {+ symbols[1]= [ r, R ],+ symbols[2]= [ r, R, paragraph, registered ]+ };+ key <AD05> {+ symbols[1]= [ t, T ],+ symbols[2]= [ t, T, tslash, Tslash ]+ };+ key <AD06> {+ symbols[1]= [ y, Y ],+ symbols[2]= [ z, Z, leftarrow, yen ]+ };+ key <AD07> {+ symbols[1]= [ u, U ],+ symbols[2]= [ u, U, downarrow, uparrow ]+ };+ key <AD08> {+ symbols[1]= [ i, I ],+ symbols[2]= [ i, I, rightarrow, idotless ]+ };+ key <AD09> {+ symbols[1]= [ o, O ],+ symbols[2]= [ o, O, oslash, Oslash ]+ };+ key <AD10> {+ symbols[1]= [ p, P ],+ symbols[2]= [ p, P, thorn, THORN ]+ };+ key <AD11> {+ symbols[1]= [ bracketleft, braceleft ],+ symbols[2]= [ udiaeresis, Udiaeresis, dead_diaeresis, dead_abovering ]+ };+ key <AD12> {+ symbols[1]= [ bracketright, braceright ],+ symbols[2]= [ plus, asterisk, asciitilde, macron ]+ };+ key <RTRN> { [ Return ] };+ key <LCTL> { [ Control_L ] };+ key <AC01> {+ symbols[1]= [ a, A ],+ symbols[2]= [ a, A, ae, AE ]+ };+ key <AC02> {+ symbols[1]= [ s, S ],+ symbols[2]= [ s, S, U017F, SSHARP ]+ };+ key <AC03> {+ symbols[1]= [ d, D ],+ symbols[2]= [ d, D, eth, ETH ]+ };+ key <AC04> {+ symbols[1]= [ f, F ],+ symbols[2]= [ f, F, dstroke, ordfeminine ]+ };+ key <AC05> {+ symbols[1]= [ g, G ],+ symbols[2]= [ g, G, eng, ENG ]+ };+ key <AC06> {+ symbols[1]= [ h, H ],+ symbols[2]= [ h, H, hstroke, Hstroke ]+ };+ key <AC07> {+ symbols[1]= [ j, J ],+ symbols[2]= [ j, J, dead_belowdot, dead_abovedot ]+ };+ key <AC08> {+ symbols[1]= [ k, K ],+ symbols[2]= [ k, K, kra, ampersand ]+ };+ key <AC09> {+ symbols[1]= [ l, L ],+ symbols[2]= [ l, L, lstroke, Lstroke ]+ };+ key <AC10> {+ symbols[1]= [ semicolon, colon ],+ symbols[2]= [ odiaeresis, Odiaeresis, dead_doubleacute, dead_belowdot ]+ };+ key <AC11> {+ symbols[1]= [ apostrophe, quotedbl ],+ symbols[2]= [ adiaeresis, Adiaeresis, dead_circumflex, dead_caron ]+ };+ key <TLDE> {+ symbols[1]= [ grave, asciitilde ],+ symbols[2]= [ dead_circumflex, degree, U2032, U2033 ]+ };+ key <LFSH> { [ Shift_L ] };+ key <BKSL> {+ symbols[1]= [ backslash, bar ],+ symbols[2]= [ numbersign, apostrophe, rightsinglequotemark, dead_breve ]+ };+ key <AB01> {+ symbols[1]= [ z, Z ],+ symbols[2]= [ y, Y, guillemotright, rightsingleanglequotemark ]+ };+ key <AB02> {+ symbols[1]= [ x, X ],+ symbols[2]= [ x, X, guillemotleft, leftsingleanglequotemark ]+ };+ key <AB03> {+ symbols[1]= [ c, C ],+ symbols[2]= [ c, C, cent, copyright ]+ };+ key <AB04> {+ symbols[1]= [ v, V ],+ symbols[2]= [ v, V, doublelowquotemark, singlelowquotemark ]+ };+ key <AB05> {+ symbols[1]= [ b, B ],+ symbols[2]= [ b, B, leftdoublequotemark, leftsinglequotemark ]+ };+ key <AB06> {+ symbols[1]= [ n, N ],+ symbols[2]= [ n, N, rightdoublequotemark, rightsinglequotemark ]+ };+ key <AB07> {+ symbols[1]= [ m, M ],+ symbols[2]= [ m, M, mu, masculine ]+ };+ key <AB08> {+ symbols[1]= [ comma, less ],+ symbols[2]= [ comma, semicolon, periodcentered, multiply ]+ };+ key <AB09> {+ symbols[1]= [ period, greater ],+ symbols[2]= [ period, colon, U2026, division ]+ };+ key <AB10> {+ symbols[1]= [ slash, question ],+ symbols[2]= [ minus, underscore, endash, emdash ]+ };+ key <RTSH> { [ Shift_R ] };+ key <KPMU> {+ type= "CTRL+ALT",+ symbols[1]= [ KP_Multiply, KP_Multiply, KP_Multiply, KP_Multiply, XF86ClearGrab ]+ };+ key <LALT> { [ Alt_L ] };+ key <SPCE> { [ space ] };+ key <CAPS> { [ Caps_Lock ] };+ key <FK01> {+ type= "CTRL+ALT",+ symbols[1]= [ F1, F1, F1, F1, XF86Switch_VT_1 ]+ };+ key <FK02> {+ type= "CTRL+ALT",+ symbols[1]= [ F2, F2, F2, F2, XF86Switch_VT_2 ]+ };+ key <FK03> {+ type= "CTRL+ALT",+ symbols[1]= [ F3, F3, F3, F3, XF86Switch_VT_3 ]+ };+ key <FK04> {+ type= "CTRL+ALT",+ symbols[1]= [ F4, F4, F4, F4, XF86Switch_VT_4 ]+ };+ key <FK05> {+ type= "CTRL+ALT",+ symbols[1]= [ F5, F5, F5, F5, XF86Switch_VT_5 ]+ };+ key <FK06> {+ type= "CTRL+ALT",+ symbols[1]= [ F6, F6, F6, F6, XF86Switch_VT_6 ]+ };+ key <FK07> {+ type= "CTRL+ALT",+ symbols[1]= [ F7, F7, F7, F7, XF86Switch_VT_7 ]+ };+ key <FK08> {+ type= "CTRL+ALT",+ symbols[1]= [ F8, F8, F8, F8, XF86Switch_VT_8 ]+ };+ key <FK09> {+ type= "CTRL+ALT",+ symbols[1]= [ F9, F9, F9, F9, XF86Switch_VT_9 ]+ };+ key <FK10> {+ type= "CTRL+ALT",+ symbols[1]= [ F10, F10, F10, F10, XF86Switch_VT_10 ]+ };+ key <NMLK> { [ Num_Lock ] };+ key <SCLK> { [ Scroll_Lock ] };+ key <KP7> { [ KP_Home, KP_7 ] };+ key <KP8> { [ KP_Up, KP_8 ] };+ key <KP9> { [ KP_Prior, KP_9 ] };+ key <KPSU> {+ type= "CTRL+ALT",+ symbols[1]= [ KP_Subtract, KP_Subtract, KP_Subtract, KP_Subtract, XF86Prev_VMode ]+ };+ key <KP4> { [ KP_Left, KP_4 ] };+ key <KP5> { [ KP_Begin, KP_5 ] };+ key <KP6> { [ KP_Right, KP_6 ] };+ key <KPAD> {+ type= "CTRL+ALT",+ symbols[1]= [ KP_Add, KP_Add, KP_Add, KP_Add, XF86Next_VMode ]+ };+ key <KP1> { [ KP_End, KP_1 ] };+ key <KP2> { [ KP_Down, KP_2 ] };+ key <KP3> { [ KP_Next, KP_3 ] };+ key <KP0> { [ KP_Insert, KP_0 ] };+ key <KPDL> {+ type= "KEYPAD",+ symbols[1]= [ KP_Delete, KP_Decimal ],+ symbols[2]= [ KP_Delete, KP_Separator ]+ };+ key <LVL3> { [ ISO_Level3_Shift ] };+ key <ZEHA> {+ type= "PC_CONTROL_SUPER_LEVEL2",+ symbols[1]= [ F24, XF86TouchpadToggle ]+ };+ key <LSGT> {+ symbols[1]= [ less, greater, bar, brokenbar ],+ symbols[2]= [ less, greater, bar, dead_belowmacron ]+ };+ key <FK11> {+ type= "CTRL+ALT",+ symbols[1]= [ F11, F11, F11, F11, XF86Switch_VT_11 ]+ };+ key <FK12> {+ type= "CTRL+ALT",+ symbols[1]= [ F12, F12, F12, F12, XF86Switch_VT_12 ]+ };+ key <KATA> { [ Katakana ] };+ key <HIRA> { [ Hiragana ] };+ key <HENK> { [ Henkan_Mode ] };+ key <HKTG> { [ Hiragana_Katakana ] };+ key <MUHE> { [ Muhenkan ] };+ key <KPEN> { [ KP_Enter ] };+ key <RCTL> { [ Control_R ] };+ key <KPDV> {+ type= "CTRL+ALT",+ symbols[1]= [ KP_Divide, KP_Divide, KP_Divide, KP_Divide, XF86Ungrab ]+ };+ key <PRSC> {+ type= "PC_ALT_LEVEL2",+ symbols[1]= [ Print, Sys_Req ]+ };+ key <RALT> {+ type= "ONE_LEVEL",+ symbols[1]= [ Alt_R ],+ symbols[2]= [ ISO_Level3_Shift ]+ };+ key <LNFD> { [ Linefeed ] };+ key <HOME> { [ Home ] };+ key <UP> { [ Up ] };+ key <PGUP> { [ Prior ] };+ key <LEFT> { [ Left ] };+ key <RGHT> { [ Right ] };+ key <END> { [ End ] };+ key <DOWN> { [ Down ] };+ key <PGDN> { [ Next ] };+ key <INS> { [ Insert ] };+ key <DELE> { [ Delete ] };+ key <MUTE> { [ XF86AudioMute ] };+ key <VOL-> { [ XF86AudioLowerVolume ] };+ key <VOL+> { [ XF86AudioRaiseVolume ] };+ key <POWR> { [ XF86PowerOff ] };+ key <KPEQ> { [ KP_Equal ] };+ key <I126> { [ plusminus ] };+ key <PAUS> {+ type= "PC_CONTROL_LEVEL2",+ symbols[1]= [ Pause, Break ]+ };+ key <I128> { [ XF86LaunchA ] };+ key <I129> { [ KP_Decimal, KP_Decimal ] };+ key <HNGL> { [ Hangul ] };+ key <HJCV> { [ Hangul_Hanja ] };+ key <LWIN> { [ Super_L ] };+ key <RWIN> { [ Super_R ] };+ key <COMP> { [ Menu ] };+ key <STOP> { [ Cancel ] };+ key <AGAI> { [ Redo ] };+ key <PROP> { [ SunProps ] };+ key <UNDO> { [ Undo ] };+ key <FRNT> { [ SunFront ] };+ key <COPY> { [ XF86Copy ] };+ key <OPEN> { [ XF86Open ] };+ key <PAST> { [ XF86Paste ] };+ key <FIND> { [ Find ] };+ key <CUT> { [ XF86Cut ] };+ key <HELP> { [ Help ] };+ key <I147> { [ XF86MenuKB ] };+ key <I148> { [ XF86Calculator ] };+ key <I150> { [ XF86Sleep ] };+ key <I151> { [ XF86WakeUp ] };+ key <I152> { [ XF86Explorer ] };+ key <I153> { [ XF86Send ] };+ key <I155> { [ XF86Xfer ] };+ key <I156> { [ XF86Launch1 ] };+ key <I157> { [ XF86Launch2 ] };+ key <I158> { [ XF86WWW ] };+ key <I159> { [ XF86DOS ] };+ key <I160> { [ XF86ScreenSaver ] };+ key <I161> { [ XF86RotateWindows ] };+ key <I162> { [ XF86TaskPane ] };+ key <I163> { [ XF86Mail ] };+ key <I164> { [ XF86Favorites ] };+ key <I165> { [ XF86MyComputer ] };+ key <I166> { [ XF86Back ] };+ key <I167> { [ XF86Forward ] };+ key <I169> { [ XF86Eject ] };+ key <I170> { [ XF86Eject ] };+ key <I171> { [ XF86AudioNext ] };+ key <I172> { [ XF86AudioPlay, XF86AudioPause ] };+ key <I173> { [ XF86AudioPrev ] };+ key <I174> { [ XF86AudioStop, XF86Eject ] };+ key <I175> { [ XF86AudioRecord ] };+ key <I176> { [ XF86AudioRewind ] };+ key <I177> { [ XF86Phone ] };+ key <I179> { [ XF86Tools ] };+ key <I180> { [ XF86HomePage ] };+ key <I181> { [ XF86Reload ] };+ key <I182> { [ XF86Close ] };+ key <I185> { [ XF86ScrollUp ] };+ key <I186> { [ XF86ScrollDown ] };+ key <I187> { [ parenleft ] };+ key <I188> { [ parenright ] };+ key <I189> { [ XF86New ] };+ key <I190> { [ Redo ] };+ key <FK13> { [ XF86Tools ] };+ key <FK14> { [ XF86Launch5 ] };+ key <FK15> { [ XF86Launch6 ] };+ key <FK16> { [ XF86Launch7 ] };+ key <FK17> { [ XF86Launch8 ] };+ key <FK18> { [ XF86Launch9 ] };+ key <FK19> { [ F19 ] };+ key <FK20> { [ XF86AudioMicMute ] };+ key <FK21> { [ XF86TouchpadToggle ] };+ key <FK22> { [ XF86TouchpadOn ] };+ key <FK23> {+ type= "PC_SHIFT_SUPER_LEVEL2",+ symbols[1]= [ F23, XF86Assistant ]+ };+ key <FK24> {+ type= "PC_CONTROL_SUPER_LEVEL2",+ symbols[1]= [ F24, XF86TouchpadToggle ]+ };+ key <LVL5> { [ ISO_Level5_Shift ] };+ key <ALT> { [ NoSymbol, Alt_L ] };+ key <META> { [ NoSymbol, Meta_L ] };+ key <SUPR> { [ NoSymbol, Super_L ] };+ key <HYPR> { [ NoSymbol, Hyper_L ] };+ key <I208> { [ XF86AudioPlay ] };+ key <I209> { [ XF86AudioPause ] };+ key <I210> { [ XF86Launch3 ] };+ key <I211> { [ XF86Launch4 ] };+ key <I212> { [ XF86LaunchB ] };+ key <I213> { [ XF86Suspend ] };+ key <I214> { [ XF86Close ] };+ key <I215> { [ XF86AudioPlay ] };+ key <I216> { [ XF86AudioForward ] };+ key <I217> { [ XF86AudioBassBoost ] };+ key <I218> { [ Print ] };+ key <I220> { [ XF86WebCam ] };+ key <I221> { [ XF86AudioPreset ] };+ key <I223> { [ XF86Mail ] };+ key <I224> { [ XF86Messenger ] };+ key <I225> { [ XF86Search ] };+ key <I226> { [ XF86Go ] };+ key <I227> { [ XF86Finance ] };+ key <I228> { [ XF86Game ] };+ key <I229> { [ XF86Shop ] };+ key <I231> { [ Cancel ] };+ key <I232> { [ XF86MonBrightnessDown ] };+ key <I233> { [ XF86MonBrightnessUp ] };+ key <I234> { [ XF86AudioMedia ] };+ key <I235> { [ XF86Display ] };+ key <I236> { [ XF86KbdLightOnOff ] };+ key <I237> { [ XF86KbdBrightnessDown ] };+ key <I238> { [ XF86KbdBrightnessUp ] };+ key <I239> { [ XF86Send ] };+ key <I240> { [ XF86Reply ] };+ key <I241> { [ XF86MailForward ] };+ key <I242> { [ XF86Save ] };+ key <I243> { [ XF86Documents ] };+ key <I244> { [ XF86Battery ] };+ key <I245> { [ XF86Bluetooth ] };+ key <I246> { [ XF86WLAN ] };+ key <I247> { [ XF86UWB ] };+ key <I249> { [ XF86Next_VMode ] };+ key <I250> { [ XF86Prev_VMode ] };+ key <I251> { [ XF86MonBrightnessCycle ] };+ key <I252> { [ XF86BrightnessAuto ] };+ key <I253> { [ XF86DisplayOff ] };+ key <I254> { [ XF86WWAN ] };+ key <I255> { [ XF86RFKill ] };+ key <I256> { [ XF86AudioMicMute ] };+ key <I360> { [ XF86OK ] };+ key <I361> { [ XF86Select ] };+ key <I362> { [ XF86GoTo ] };+ key <I363> { [ XF86Clear ] };+ key <I365> { [ XF86Option ] };+ key <I366> { [ XF86Info ] };+ key <I367> { [ XF86Time ] };+ key <I368> { [ XF86VendorLogo ] };+ key <I370> { [ XF86MediaSelectProgramGuide ] };+ key <I371> { [ XF86NextFavorite ] };+ key <I372> { [ XF86Favorites ] };+ key <I373> { [ XF86MediaSelectProgramGuide ] };+ key <I374> { [ XF86MediaSelectHome ] };+ key <I376> { [ XF86MediaLanguageMenu ] };+ key <I377> { [ XF86MediaTitleMenu ] };+ key <I378> { [ XF86Subtitle ] };+ key <I379> { [ XF86CycleAngle ] };+ key <I380> { [ XF86FullScreen ] };+ key <I381> { [ XF86AudioChannelMode ] };+ key <I382> { [ XF86Keyboard ] };+ key <I383> { [ XF86AspectRatio ] };+ key <I384> { [ XF86MediaSelectPC ] };+ key <I385> { [ XF86MediaSelectTV ] };+ key <I386> { [ XF86MediaSelectCable ] };+ key <I387> { [ XF86MediaSelectVCR ] };+ key <I388> { [ XF86MediaSelectVCRPlus ] };+ key <I389> { [ XF86MediaSelectSatellite ] };+ key <I391> { [ XF86CD ] };+ key <I392> { [ XF86MediaSelectTape ] };+ key <I393> { [ XF86MediaSelectRadio ] };+ key <I394> { [ XF86MediaSelectTuner ] };+ key <I395> { [ XF86MediaPlayer ] };+ key <I396> { [ XF86MediaSelectTeletext ] };+ key <I397> { [ XF86DVD ] };+ key <I398> { [ XF86MediaSelectAuxiliary ] };+ key <I400> { [ XF86Audio ] };+ key <I401> { [ XF86Video ] };+ key <I404> { [ XF86Memo ] };+ key <I405> { [ XF86Calendar ] };+ key <I406> { [ XF86Red ] };+ key <I407> { [ XF86Green ] };+ key <I408> { [ XF86Yellow ] };+ key <I409> { [ XF86Blue ] };+ key <I410> { [ XF86ChannelUp ] };+ key <I411> { [ XF86ChannelDown ] };+ key <I417> { [ XF86MediaPlaySlow ] };+ key <I418> { [ XF86AudioRandomPlay ] };+ key <I419> { [ XF86Break ] };+ key <I421> { [ XF86NumberEntryMode ] };+ key <I424> { [ XF86VideoPhone ] };+ key <I425> { [ XF86Game ] };+ key <I426> { [ XF86ZoomIn ] };+ key <I427> { [ XF86ZoomOut ] };+ key <I428> { [ XF86ZoomReset ] };+ key <I429> { [ XF86Word ] };+ key <I430> { [ XF86Editor ] };+ key <I431> { [ XF86Excel ] };+ key <I432> { [ XF86GraphicsEditor ] };+ key <I433> { [ XF86Presentation ] };+ key <I434> { [ XF86Database ] };+ key <I435> { [ XF86News ] };+ key <I436> { [ XF86Voicemail ] };+ key <I437> { [ XF86Addressbook ] };+ key <I438> { [ XF86Messenger ] };+ key <I439> { [ XF86DisplayToggle ] };+ key <I440> { [ XF86SpellCheck ] };+ key <I441> { [ XF86LogOff ] };+ key <I442> { [ dollar ] };+ key <I443> { [ EuroSign ] };+ key <I444> { [ XF86FrameBack ] };+ key <I445> { [ XF86FrameForward ] };+ key <I446> { [ XF86ContextMenu ] };+ key <I447> { [ XF86MediaRepeat ] };+ key <I448> { [ XF8610ChannelsUp ] };+ key <I449> { [ XF8610ChannelsDown ] };+ key <I450> { [ XF86Images ] };+ key <I452> { [ XF86NotificationCenter ] };+ key <I453> { [ XF86PickupPhone ] };+ key <I454> { [ XF86HangupPhone ] };+ key <I455> { [ XF86LinkPhone ] };+ key <I472> { [ XF86Fn ] };+ key <I473> { [ XF86Fn_Esc ] };+ key <I474> { [ XF86Fn_F1 ] };+ key <I475> { [ XF86Fn_F2 ] };+ key <I476> { [ XF86Fn_F3 ] };+ key <I477> { [ XF86Fn_F4 ] };+ key <I478> { [ XF86Fn_F5 ] };+ key <I479> { [ XF86Fn_F6 ] };+ key <I480> { [ XF86Fn_F7 ] };+ key <I481> { [ XF86Fn_F8 ] };+ key <I482> { [ XF86Fn_F9 ] };+ key <I483> { [ XF86Fn_F10 ] };+ key <I484> { [ XF86Fn_F11 ] };+ key <I485> { [ XF86Fn_F12 ] };+ key <I486> { [ XF86Fn_1 ] };+ key <I487> { [ XF86Fn_2 ] };+ key <I488> { [ XF86Fn_D ] };+ key <I489> { [ XF86Fn_E ] };+ key <I490> { [ XF86Fn_F ] };+ key <I491> { [ XF86Fn_S ] };+ key <I492> { [ XF86Fn_B ] };+ key <I493> { [ XF86FnRightShift ] };+ key <I505> { [ braille_dot_1 ] };+ key <I506> { [ braille_dot_2 ] };+ key <I507> { [ braille_dot_3 ] };+ key <I508> { [ braille_dot_4 ] };+ key <I509> { [ braille_dot_5 ] };+ key <I510> { [ braille_dot_6 ] };+ key <I511> { [ braille_dot_7 ] };+ key <I512> { [ braille_dot_8 ] };+ key <I513> { [ braille_dot_9 ] };+ key <I514> { [ braille_dot_1 ] };+ key <I520> { [ XF86Numeric0 ] };+ key <I521> { [ XF86Numeric1 ] };+ key <I522> { [ XF86Numeric2 ] };+ key <I523> { [ XF86Numeric3 ] };+ key <I524> { [ XF86Numeric4 ] };+ key <I525> { [ XF86Numeric5 ] };+ key <I526> { [ XF86Numeric6 ] };+ key <I527> { [ XF86Numeric7 ] };+ key <I528> { [ XF86Numeric8 ] };+ key <I529> { [ XF86Numeric9 ] };+ key <I530> { [ XF86NumericStar ] };+ key <I531> { [ XF86NumericPound ] };+ key <I532> { [ XF86NumericA ] };+ key <I533> { [ XF86NumericB ] };+ key <I534> { [ XF86NumericC ] };+ key <I535> { [ XF86NumericD ] };+ key <I536> { [ XF86CameraFocus ] };+ key <I537> { [ XF86WPSButton ] };+ key <I538> { [ XF86TouchpadToggle ] };+ key <I539> { [ XF86TouchpadOn ] };+ key <I540> { [ XF86TouchpadOff ] };+ key <I541> { [ XF86CameraZoomIn ] };+ key <I542> { [ XF86CameraZoomOut ] };+ key <I543> { [ XF86CameraUp ] };+ key <I544> { [ XF86CameraDown ] };+ key <I545> { [ XF86CameraLeft ] };+ key <I546> { [ XF86CameraRight ] };+ key <I547> { [ XF86AttendantOn ] };+ key <I548> { [ XF86AttendantOff ] };+ key <I549> { [ XF86AttendantToggle ] };+ key <I550> { [ XF86LightsToggle ] };+ key <I568> { [ XF86ALSToggle ] };+ key <I569> { [ XF86RotationLockToggle ] };+ key <I570> { [ XF86RefreshRateToggle ] };+ key <I584> { [ XF86Buttonconfig ] };+ key <I585> { [ XF86Taskmanager ] };+ key <I586> { [ XF86Journal ] };+ key <I587> { [ XF86ControlPanel ] };+ key <I588> { [ XF86AppSelect ] };+ key <I589> { [ XF86Screensaver ] };+ key <I590> { [ XF86VoiceCommand ] };+ key <I591> { [ XF86Assistant ] };+ key <I592> { [ ISO_Next_Group ] };+ key <I593> { [ XF86EmojiPicker ] };+ key <I594> { [ XF86Dictate ] };+ key <I595> { [ XF86CameraAccessEnable ] };+ key <I596> { [ XF86CameraAccessDisable ] };+ key <I597> { [ XF86CameraAccessToggle ] };+ key <I598> { [ XF86Accessibility ] };+ key <I599> { [ XF86DoNotDisturb ] };+ key <I600> { [ XF86BrightnessMin ] };+ key <I601> { [ XF86BrightnessMax ] };+ key <I602> { [ XF86ElectronicPrivacyScreenOn ] };+ key <I603> { [ XF86ElectronicPrivacyScreenOff ] };+ key <I604> { [ XF86ActionOnSelection ] };+ key <I605> { [ XF86ContextualInsert ] };+ key <I606> { [ XF86ContextualQuery ] };+ key <I616> { [ XF86KbdInputAssistPrev ] };+ key <I617> { [ XF86KbdInputAssistNext ] };+ key <I618> { [ XF86KbdInputAssistPrevgroup ] };+ key <I619> { [ XF86KbdInputAssistNextgroup ] };+ key <I620> { [ XF86KbdInputAssistAccept ] };+ key <I621> { [ XF86KbdInputAssistCancel ] };+ key <I622> { [ XF86RightUp ] };+ key <I623> { [ XF86RightDown ] };+ key <I624> { [ XF86LeftUp ] };+ key <I625> { [ XF86LeftDown ] };+ key <I626> { [ XF86RootMenu ] };+ key <I627> { [ XF86MediaTopMenu ] };+ key <I628> { [ XF86Numeric11 ] };+ key <I629> { [ XF86Numeric12 ] };+ key <I630> { [ XF86AudioDesc ] };+ key <I631> { [ XF863DMode ] };+ key <I632> { [ XF86NextFavorite ] };+ key <I633> { [ XF86StopRecord ] };+ key <I634> { [ XF86PauseRecord ] };+ key <I635> { [ XF86VOD ] };+ key <I636> { [ XF86Unmute ] };+ key <I637> { [ XF86FastReverse ] };+ key <I638> { [ XF86SlowReverse ] };+ key <I639> { [ XF86Data ] };+ key <I640> { [ XF86OnScreenKeyboard ] };+ key <I641> { [ XF86PrivacyScreenToggle ] };+ key <I642> { [ XF86SelectiveScreenshot ] };+ key <I643> { [ XF86NextElement ] };+ key <I644> { [ XF86PreviousElement ] };+ key <I645> { [ XF86AutopilotEngageToggle ] };+ key <I646> { [ XF86MarkWaypoint ] };+ key <I647> { [ XF86Sos ] };+ key <I648> { [ XF86NavChart ] };+ key <I649> { [ XF86FishingChart ] };+ key <I650> { [ XF86SingleRangeRadar ] };+ key <I651> { [ XF86DualRangeRadar ] };+ key <I652> { [ XF86RadarOverlay ] };+ key <I653> { [ XF86TraditionalSonar ] };+ key <I654> { [ XF86ClearvuSonar ] };+ key <I655> { [ XF86SidevuSonar ] };+ key <I656> { [ XF86NavInfo ] };+ key <I657> { [ XF86BrightnessAdjust ] };+ key <I664> { [ XF86Macro1 ] };+ key <I665> { [ XF86Macro2 ] };+ key <I666> { [ XF86Macro3 ] };+ key <I667> { [ XF86Macro4 ] };+ key <I668> { [ XF86Macro5 ] };+ key <I669> { [ XF86Macro6 ] };+ key <I670> { [ XF86Macro7 ] };+ key <I671> { [ XF86Macro8 ] };+ key <I672> { [ XF86Macro9 ] };+ key <I673> { [ XF86Macro10 ] };+ key <I674> { [ XF86Macro11 ] };+ key <I675> { [ XF86Macro12 ] };+ key <I676> { [ XF86Macro13 ] };+ key <I677> { [ XF86Macro14 ] };+ key <I678> { [ XF86Macro15 ] };+ key <I679> { [ XF86Macro16 ] };+ key <I680> { [ XF86Macro17 ] };+ key <I681> { [ XF86Macro18 ] };+ key <I682> { [ XF86Macro19 ] };+ key <I683> { [ XF86Macro20 ] };+ key <I684> { [ XF86Macro21 ] };+ key <I685> { [ XF86Macro22 ] };+ key <I686> { [ XF86Macro23 ] };+ key <I687> { [ XF86Macro24 ] };+ key <I688> { [ XF86Macro25 ] };+ key <I689> { [ XF86Macro26 ] };+ key <I690> { [ XF86Macro27 ] };+ key <I691> { [ XF86Macro28 ] };+ key <I692> { [ XF86Macro29 ] };+ key <I693> { [ XF86Macro30 ] };+ key <I696> { [ XF86MacroRecordStart ] };+ key <I697> { [ XF86MacroRecordStop ] };+ key <I698> { [ XF86MacroPresetCycle ] };+ key <I699> { [ XF86MacroPreset1 ] };+ key <I700> { [ XF86MacroPreset2 ] };+ key <I701> { [ XF86MacroPreset3 ] };+ key <I704> { [ XF86KbdLcdMenu1 ] };+ key <I705> { [ XF86KbdLcdMenu2 ] };+ key <I706> { [ XF86KbdLcdMenu3 ] };+ key <I707> { [ XF86KbdLcdMenu4 ] };+ key <I708> { [ XF86KbdLcdMenu5 ] };+ key <I709> { [ XF86PerformanceMode ] };+ modifier_map Shift { <LFSH>, <RTSH> };+ modifier_map Lock { <CAPS> };+ modifier_map Control { <LCTL>, <RCTL> };+ modifier_map Mod1 { <LALT>, <RALT>, <ALT>, <META> };+ modifier_map Mod2 { <NMLK> };+ modifier_map Mod3 { <LVL5> };+ modifier_map Mod4 { <LWIN>, <RWIN>, <SUPR> };+ modifier_map Mod5 { <LVL3> };+};++};diff --git a/libs/cua-driver/rust/crates/platform-linux/tests/kwin_helper_contract.rs b/libs/cua-driver/rust/crates/platform-linux/tests/kwin_helper_contract.rsindex 7c06fe4453..7328453bb6 100644--- a/libs/cua-driver/rust/crates/platform-linux/tests/kwin_helper_contract.rs+++ b/libs/cua-driver/rust/crates/platform-linux/tests/kwin_helper_contract.rs@@ -1,6 +1,6 @@#![cfg(target_os = "linux")]-use platform_linux::wayland::kwin_helper::{available, parse_snapshot, with_focused_window};+use platform_linux::wayland::kwin_helper::{available, parse_snapshot};#[test]fn parses_versioned_kwin_snapshot() {@@ -70,22 +70,9 @@ fn rejects_multiple_active_records() {}#[test]-fn raw_kwin_input_is_refused_before_dispatch() {+fn kwin_reports_no_target_bound_input() {assert!(!available(),"focus-bound KWin input must not be advertised as safe");-- let called = std::cell::Cell::new(false);- let error = with_focused_window(1200, 41, || {- called.set(true);- Ok(())- })- .expect_err("focus-bound KWin input must refuse");-- assert!(- !called.get(),- "input body must not run on the focus-only KWin path"- );- assert!(error.to_string().contains("target-bound KWin input path"));}diff --git a/libs/cua-driver/tests/fixtures/apps/cross-platform/electron/main.js b/libs/cua-driver/tests/fixtures/apps/cross-platform/electron/main.jsindex 2800fcbfb7..8ab091dad4 100644--- a/libs/cua-driver/tests/fixtures/apps/cross-platform/electron/main.js+++ b/libs/cua-driver/tests/fixtures/apps/cross-platform/electron/main.js@@ -13,6 +13,7 @@ const customCuaCompositor = process.env.CUA_E2E_WAYLAND_SESSION === 'cua-composiconst fixtureJournalUrl = process.env.CUA_E2E_FIXTURE_JOURNAL_URL || '';const sentinelJournalPath = process.env.CUA_E2E_SENTINEL_JOURNAL || '';const sentinelControlPath = process.env.CUA_E2E_SENTINEL_CONTROL || '';+const sentinelStatePath = process.env.CUA_E2E_SENTINEL_STATE || '';if (process.env.CUA_E2E_USER_DATA_DIR) {app.setPath('userData', process.env.CUA_E2E_USER_DATA_DIR);}@@ -158,6 +159,22 @@ function createWindow() {}}mainWindow.webContents.send('cua-e2e-sentinel-heartbeat-probe');+ if (sentinelStatePath) {+ // Fullscreen as the compositor's last configure set it; niri's+ // IPC cannot tell a fullscreen tile from a maximized one. The+ // event journal is reset during setup, so this state is kept+ // in its own file, replaced whole on every tick.+ const state = JSON.stringify({+ at_ms: Date.now(),+ fullscreen: mainWindow.isFullScreen(),+ });+ // A failed write leaves the report stale, which the observer+ // already treats as no fullscreen evidence.+ try {+ fs.writeFileSync(`${sentinelStatePath}.tmp`, state, 'utf8');+ fs.renameSync(`${sentinelStatePath}.tmp`, sentinelStatePath);+ } catch {}+ }}}, 100);sentinelHeartbeatTimer.unref();diff --git a/libs/cua-driver/tests/fixtures/shared/web/index.html b/libs/cua-driver/tests/fixtures/shared/web/index.htmlindex 1e32fa497d..1a20545a5d 100644--- a/libs/cua-driver/tests/fixtures/shared/web/index.html+++ b/libs/cua-driver/tests/fixtures/shared/web/index.html@@ -325,6 +325,26 @@window.__CUA_E2E_FIXTURE_JOURNAL_URL || '';if (!journalUrl) return;+ const collapsedLayout = element => {+ const style = getComputedStyle(element);+ const root = document.documentElement.getBoundingClientRect();+ return {+ innerWidth: window.innerWidth,+ innerHeight: window.innerHeight,+ devicePixelRatio: window.devicePixelRatio,+ readyState: document.readyState,+ rootRect: { x: root.x, y: root.y, width: root.width, height: root.height },+ scrollStyle: {+ display: style.display,+ visibility: style.visibility,+ position: style.position,+ width: style.width,+ height: style.height,+ overflowY: style.overflowY,+ },+ };+ };+let queued = false;const publish = () => {queued = false;@@ -337,6 +357,10 @@// Scroll-event mirrors can lag the actual DOM offset.entry.scrollTop = element.scrollTop;entry.clientHeight = element.clientHeight;+ // A fixed-height scroller with no height means the page has no+ // layout; record what the host renderer reports so the harness+ // can explain the collapse.+ if (element.clientHeight === 0) entry.layout = collapsedLayout(element);}state[element.dataset.cuaId] = entry;});diff --git a/libs/cua-driver/tests/fixtures/shared/web/journal.test.cjs b/libs/cua-driver/tests/fixtures/shared/web/journal.test.cjsindex 5e402185ad..e1ab42ca3f 100644--- a/libs/cua-driver/tests/fixtures/shared/web/journal.test.cjs+++ b/libs/cua-driver/tests/fixtures/shared/web/journal.test.cjs@@ -75,3 +75,49 @@ test('journal samples live scroll geometry without an event or DOM mutation', ()assert.equal(publications[1]['scroll-tall'].scrollTop, 320);assert.deepEqual(publications[1]['lbl-scroll-offset'], { text: 'scroll_offset=104' });});++test('a collapsed scroller publishes the layout the renderer reports', () => {+ const html = readFileSync(join(__dirname, 'index.html'), 'utf8');+ const source = html.match(/\(function startFixtureJournal\(\) \{[\s\S]*?\n \}\)\(\);/);+ const scroll = { dataset: { cuaId: 'scroll-tall' }, textContent: 'contents', scrollTop: 0, clientHeight: 0 };+ const publications = [];+ const timers = [];+ runInNewContext(source[0], {+ document: {+ body: {},+ readyState: 'complete',+ documentElement: { getBoundingClientRect: () => ({ x: 0, y: 0, width: 0, height: 0 }) },+ querySelectorAll: () => [scroll],+ addEventListener() {},+ },+ window: {+ innerWidth: 0,+ innerHeight: 0,+ devicePixelRatio: 1,+ cuaE2E: {+ journalUrl: 'http://fixture.invalid',+ publishFixtureState(state) { publications.push(JSON.parse(JSON.stringify(state))); },+ },+ addEventListener(name, callback) { callback(); },+ },+ getComputedStyle: () => ({+ display: 'block', visibility: 'visible', position: 'static',+ width: '260px', height: '120px', overflowY: 'auto',+ }),+ MutationObserver: class { observe() {} },+ setTimeout(callback) { timers.push(callback); },+ setInterval() {},+ });+ timers.shift()();+ assert.deepEqual(publications[0]['scroll-tall'].layout, {+ innerWidth: 0,+ innerHeight: 0,+ devicePixelRatio: 1,+ readyState: 'complete',+ rootRect: { x: 0, y: 0, width: 0, height: 0 },+ scrollStyle: {+ display: 'block', visibility: 'visible', position: 'static',+ width: '260px', height: '120px', overflowY: 'auto',+ },+ });+});diff --git a/libs/cua-spacesd/Cargo.lock b/libs/cua-spacesd/Cargo.lockindex 0d682229b6..31f0b29e04 100644--- a/libs/cua-spacesd/Cargo.lock+++ b/libs/cua-spacesd/Cargo.lock@@ -5331,6 +5331,7 @@ dependencies = ["wayland-scanner","x11","x11rb",+ "xkbcommon","zbus",]@@ -8775,6 +8776,23 @@ dependencies = ["rustix",]+[[package]]+name = "xkbcommon"+version = "0.9.0"+source = "registry+https://github.com/rust-lang/crates.io-index"+checksum = "a7a974f48060a14e95705c01f24ad9c3345022f4d97441b8a36beb7ed5c4a02d"+dependencies = [+ "libc",+ "memmap2",+ "xkeysym",+]++[[package]]+name = "xkeysym"+version = "0.2.1"+source = "registry+https://github.com/rust-lang/crates.io-index"+checksum = "b9cc00251562a284751c9973bace760d86c0276c471b4be569fe6b068ee97a56"+[[package]]name = "xmlparser"version = "0.13.6"diff --git a/libs/cua-spacesd/tests/spaces-e2e/Cargo.lock b/libs/cua-spacesd/tests/spaces-e2e/Cargo.lockindex e33a01ef9c..ff8eb0e6fe 100644--- a/libs/cua-spacesd/tests/spaces-e2e/Cargo.lock+++ b/libs/cua-spacesd/tests/spaces-e2e/Cargo.lock@@ -6247,6 +6247,7 @@ dependencies = ["wayland-scanner","x11","x11rb",+ "xkbcommon","zbus",]@@ -9856,6 +9857,23 @@ dependencies = ["rustix",]+[[package]]+name = "xkbcommon"+version = "0.9.0"+source = "registry+https://github.com/rust-lang/crates.io-index"+checksum = "a7a974f48060a14e95705c01f24ad9c3345022f4d97441b8a36beb7ed5c4a02d"+dependencies = [+ "libc",+ "memmap2",+ "xkeysym",+]++[[package]]+name = "xkeysym"+version = "0.2.1"+source = "registry+https://github.com/rust-lang/crates.io-index"+checksum = "b9cc00251562a284751c9973bace760d86c0276c471b4be569fe6b068ee97a56"+[[package]]name = "xmlparser"version = "0.13.6"diff --git a/libs/cua-spacesd/tests/teleport-e2e/Cargo.lock b/libs/cua-spacesd/tests/teleport-e2e/Cargo.lockindex 820abd3ed0..c2ee520292 100644--- a/libs/cua-spacesd/tests/teleport-e2e/Cargo.lock+++ b/libs/cua-spacesd/tests/teleport-e2e/Cargo.lock@@ -4411,6 +4411,7 @@ dependencies = ["wayland-scanner","x11","x11rb",+ "xkbcommon","zbus",]@@ -7567,6 +7568,23 @@ dependencies = ["rustix",]+[[package]]+name = "xkbcommon"+version = "0.9.0"+source = "registry+https://github.com/rust-lang/crates.io-index"+checksum = "a7a974f48060a14e95705c01f24ad9c3345022f4d97441b8a36beb7ed5c4a02d"+dependencies = [+ "libc",+ "memmap2",+ "xkeysym",+]++[[package]]+name = "xkeysym"+version = "0.2.1"+source = "registry+https://github.com/rust-lang/crates.io-index"+checksum = "b9cc00251562a284751c9973bace760d86c0276c471b4be569fe6b068ee97a56"+[[package]]name = "yamux"version = "0.13.10"diff --git a/nix/cua-driver/tests/rust-unit.nix b/nix/cua-driver/tests/rust-unit.nixindex 6a77169078..634074d5a7 100644--- a/nix/cua-driver/tests/rust-unit.nix+++ b/nix/cua-driver/tests/rust-unit.nix@@ -60,6 +60,8 @@ pkgs.rustPlatform.buildRustPackage {libxextpipewirelibei+ # platform-linux resolves niri and libei text against the XKB keymap.+ libxkbcommon];# Focus unit tests launch private Xvfb servers. The ignored desktopdiff --git a/scripts/ci/README.md b/scripts/ci/README.mdindex e609b96e45..8ea7811b96 100644--- a/scripts/ci/README.md+++ b/scripts/ci/README.md@@ -122,7 +122,7 @@ never certify.| `libs/cua-driver/tests/runners/macos-lume/run-all.sh` | Logged-in, TCC-authorized Lume worker; every complete run includes the standalone-browser matrix | Canonical: macOS || `scripts/ci/macos/run-rust-e2e.sh` | Called by the Lume wrapper and the hosted macOS script | Matrix implementation behind the macOS rows; refuses to run without the wrapper's unrestricted daemon || `scripts/ci/run-rust-standalone-browser-e2e.sh`, `scripts/ci/windows/run-rust-standalone-browser-e2e.ps1` | `e2e-rust-standalone-browsers.yml`; the macOS Lume wrapper runs it as well | Scoped gate: installed Chrome and Edge, required for browser-facing changes |-| `scripts/ci/linux/run-rust-e2e-wayland.sh`, `run-rust-e2e-inject.sh` | `e2e-rust-linux-wayland.yml` (`sway`, `sway-xwayland`, `cua-compositor`) | Scoped gate: native Wayland compositor lanes, kept separate from X11 |+| `scripts/ci/linux/run-rust-e2e-wayland.sh`, `run-rust-e2e-inject.sh` | `e2e-rust-linux-wayland.yml` (`sway`, `sway-xwayland`, `cua-compositor`, `niri`) | Scoped gate: native Wayland compositor lanes, kept separate from X11 || `run-rust-e2e.sh` in a native Hyprland desktop, plus the allowlisted `hyprland_*` rows | Maintainer's prepared native Hyprland desktop | Scoped gate: native Hyprland; no automated lane || `scripts/ci/macos/run-hosted-rust-e2e.sh` | `e2e-rust-macos.yml` with `mode=hosted` | Supporting: supplemental hosted macOS matrix; does not replace Lume || `scripts/ci/linux/run-rust-e2e-desktop.sh` | Representative GNOME or KDE maintainer desktop | Supporting: environment coverage |@@ -258,7 +258,7 @@ with each measurement.| Runner | Session | Canonical command || ------------------------------- | ------------------------------------------------------------------ | ----------------- || `linux/run-rust-e2e.sh` | Existing Linux X11 or Wayland desktop | no selector |-| `linux/run-rust-e2e-wayland.sh` | Headless native Sway session | no selector |+| `linux/run-rust-e2e-wayland.sh` | Headless native Sway, or nested niri with `CUA_E2E_WAYLAND_SESSION=niri` | no selector || `linux/run-rust-e2e-inject.sh` | Nested `cua-compositor` session | no selector || `linux/run-rust-e2e-desktop.sh` | Existing representative Linux desktop | no selector || `windows/run-rust-e2e.ps1` | Windows console/RDP user session | `-RequireGui` |@@ -339,6 +339,32 @@ active output, and requires `get_window_state` to returnthe fixture accessibility state are external focus, workspace, and mutationoracles for that targeted regression.+Run the niri lane with `CUA_E2E_WAYLAND_SESSION=niri` through+`nix develop .#cua-driver-niri-e2e`. niri has no headless backend, so the+wrapper starts a headless `cage` in a private runtime directory, sizes its+output to 1920x1080 with `wlr-randr`, and runs niri's winit backend inside it.+Before delegating to `run-rust-e2e.sh` it requires that niri drives only its+`winit` output and that one niri process serves both the exported+`WAYLAND_DISPLAY` and `NIRI_SOCKET`, and it writes `niri-preflight.json` with+the niri and cage versions. Harness windows float, the sentinel opens+fullscreen, animations are off, and `prefer-no-csd` keeps GTK 3 and Chromium+frames the size niri reports, as Sway's server-side decorations do. `CUA_E2E_NIRI_SCALE` changes the output+scale for focused fractional-scale checks; harness windows keep 940x780+physical pixels, so at 1.5 they are 626x520 logical inside a 1280x720 output. The shell patches `wf-recorder`+0.6.0, which otherwise binds `linux-dmabuf` version 4 that a GPU-less niri does+not offer.++Every Wayland mode unsets inherited `WAYLAND_SOCKET`, `NIRI_SOCKET`,+`SWAYSOCK`, `I3SOCK`, `HYPRLAND_INSTANCE_SIGNATURE`,+`DBUS_SESSION_BUS_ADDRESS`, and `AT_SPI_BUS_ADDRESS` before it starts its+compositor, then starts its own session bus, AT-SPI bus, and Secret Service+with keyrings under its runtime directory. It asks the AT-SPI launcher for+dbus-daemon through `ATSPI_DBUS_IMPLEMENTATION`, because inside a systemd user+unit a launcher built to prefer dbus-broker would activate services through+the host's systemd user manager. at-spi2-core 2.60.1 in the Nix shells honours+it. A launcher from another source that ignores the variable can still reach+the host's activation, so run the wrapper from the Nix shells.+Run the nested compositor wrapper through`nix develop .#cua-driver-inject-e2e`. This environment is experimental andproves only the private compositor-owned route. Use `run-rust-e2e-desktop.sh`diff --git a/scripts/ci/linux/run-rust-e2e-wayland.sh b/scripts/ci/linux/run-rust-e2e-wayland.shindex 58cc8b696b..b70ecd56fc 100755--- a/scripts/ci/linux/run-rust-e2e-wayland.sh+++ b/scripts/ci/linux/run-rust-e2e-wayland.sh@@ -1,11 +1,15 @@#!/usr/bin/env bash-# Start one native, headless Sway session and run the complete Rust E2E matrix.+# Start one native, headless Wayland session and run the complete Rust E2E+# matrix. CUA_E2E_WAYLAND_SESSION selects sway (default), sway-xwayland,+# cua-compositor, or niri. niri has no headless backend, so the niri session+# runs niri's winit backend inside a headless cage.set -euo pipefailSCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"REPO_ROOT="$(cd "${SCRIPT_DIR}/../../.." && pwd)"RUNTIME_DIR="$(mktemp -d)"SWAY_CONFIG="$(mktemp)"+NIRI_CONFIG="$(mktemp)"SESSION_KIND="${CUA_E2E_WAYLAND_SESSION:-sway}"export CUA_E2E_WAYLAND_SESSION="${SESSION_KIND}"if [[ "${SESSION_KIND}" == sway-xwayland ]]; then@@ -16,23 +20,52 @@ if [[ "${SESSION_KIND}" == sway-xwayland ]]; thenfiCOMPOSITOR_LOG="${CUA_E2E_ARTIFACT_DIR:-${REPO_ROOT}/artifacts/cua-driver/linux}/${SESSION_KIND}.log"ATSPI_LOG="${CUA_E2E_ARTIFACT_DIR:-${REPO_ROOT}/artifacts/cua-driver/linux}/at-spi-bus.log"+# cage serves only niri's winit window. Its private runtime directory keeps its+# Wayland socket out of the directory the tests search and inherit.+CAGE_RUNTIME_DIR="${RUNTIME_DIR}/cage"+CAGE_LOG="${CUA_E2E_ARTIFACT_DIR:-${REPO_ROOT}/artifacts/cua-driver/linux}/cage.log"+CAGE_PID=""COMPOSITOR_PID=""DBUS_PID=""ATSPI_PID=""+KEYRING_PID=""++# SIGTERM, then SIGKILL after a 5 second grace, so a wedged or stopped+# process cannot hold cleanup. Children of this shell are also reaped.+stop_process() {+ local pid="$1"+ kill -TERM "${pid}" 2>/dev/null || return 0+ local deadline=$((SECONDS + 5))+ while kill -0 "${pid}" 2>/dev/null \+ && [[ "$(ps -o stat= -p "${pid}" 2>/dev/null)" != Z* ]] \+ && ((SECONDS < deadline)); do+ sleep 0.1+ done+ kill -KILL "${pid}" 2>/dev/null || true+ wait "${pid}" 2>/dev/null || true+}cleanup() {if [[ -n "${COMPOSITOR_PID}" ]]; then- kill "${COMPOSITOR_PID}" 2>/dev/null || true- wait "${COMPOSITOR_PID}" 2>/dev/null || true+ stop_process "${COMPOSITOR_PID}"+ fi+ if [[ -n "${CAGE_PID}" ]]; then+ # cage exits once its client does, and a SIGTERM exit removes the X11+ # socket and lock it binds in /tmp.+ pkill -TERM -P "${CAGE_PID}" 2>/dev/null || true+ stop_process "${CAGE_PID}"+ fi+ if [[ -n "${KEYRING_PID}" ]]; then+ stop_process "${KEYRING_PID}"fiif [[ -n "${DBUS_PID}" ]]; then- kill "${DBUS_PID}" 2>/dev/null || true+ stop_process "${DBUS_PID}"fiif [[ -n "${ATSPI_PID}" ]]; then- kill "${ATSPI_PID}" 2>/dev/null || true- wait "${ATSPI_PID}" 2>/dev/null || true+ stop_process "${ATSPI_PID}"fi- rm -f "${SWAY_CONFIG}"+ rm -f "${SWAY_CONFIG}" "${NIRI_CONFIG}"+ rm -rf "${RUNTIME_DIR}"}trap cleanup EXIT@@ -51,13 +84,76 @@ EOFif [[ "${SESSION_KIND}" == sway-xwayland ]]; thensed -i 's/^xwayland disable$/xwayland force/' "${SWAY_CONFIG}"fi+# The niri rules mirror the Sway ones: harness windows float at the top-left+# at 940x780 and the sentinel opens fullscreen. Animations stay off because+# niri's IPC reports destination geometry, not the animated position. Focus+# does not follow the mouse, so floating windows stay hidden behind the focused+# fullscreen sentinel; the testkit observer relies on that. CUA_E2E_NIRI_SCALE+# sets the output scale for a focused fractional-scale smoke. Harness sizes are+# logical, so divide them by the scale: the windows keep Sway's 940x780+# physical pixels and stay inside the smaller logical output. prefer-no-csd+# matches Sway's server-side decorations: with client-side ones a GTK 3 or+# Chromium accessible frame includes shadow margins that niri does not report,+# and the driver then publishes no element frames for that window.+NIRI_SCALE="${CUA_E2E_NIRI_SCALE:-1}"+NIRI_HARNESS_WIDTH="$(awk -v scale="${NIRI_SCALE}" 'BEGIN { printf "%d", 940 / scale }')"+NIRI_HARNESS_HEIGHT="$(awk -v scale="${NIRI_SCALE}" 'BEGIN { printf "%d", 780 / scale }')"+cat > "${NIRI_CONFIG}" <<EOF+hotkey-overlay {+ skip-at-startup+}+prefer-no-csd+xwayland-satellite {+ off+}+animations {+ off+}+output "winit" {+ scale ${NIRI_SCALE}+}+layout {+ focus-ring {+ off+ }+ border {+ off+ }+}+window-rule {+ match title="^CuaTestHarness"+ exclude title="^CuaTestHarness Sentinel"+ open-floating true+ default-floating-position x=0 y=0 relative-to="top-left"+ default-column-width {+ fixed ${NIRI_HARNESS_WIDTH}+ }+ default-window-height {+ fixed ${NIRI_HARNESS_HEIGHT}+ }+}+window-rule {+ match title="^CuaTestHarness Sentinel"+ open-fullscreen true+}+EOFunset DISPLAYunset WAYLAND_DISPLAY+# A runner started from a desktop inherits that session's sockets and buses.+# Export only this session's own so no test can reach the host compositor,+# Secret Service, accessibility bus, or portals.+unset WAYLAND_SOCKET NIRI_SOCKET SWAYSOCK I3SOCK HYPRLAND_INSTANCE_SIGNATURE+unset DBUS_SESSION_BUS_ADDRESS AT_SPI_BUS_ADDRESSexport XDG_RUNTIME_DIR="${RUNTIME_DIR}"export XDG_SESSION_TYPE=wayland-export XDG_CURRENT_DESKTOP=sway-export XDG_SESSION_DESKTOP=sway+if [[ "${SESSION_KIND}" == niri ]]; then+ export XDG_CURRENT_DESKTOP=niri+ export XDG_SESSION_DESKTOP=niri+else+ export XDG_CURRENT_DESKTOP=sway+ export XDG_SESSION_DESKTOP=sway+fiexport WLR_BACKENDS=headlessexport WLR_RENDERER=pixmanexport WLR_RENDERER_ALLOW_SOFTWARE=1@@ -69,6 +165,9 @@ if [[ "${SESSION_KIND}" == cua-compositor ]]; thenexport CUA_E2E_INPUT_BACKENDS=atspi,cua-compositor-injectexport CUA_E2E_HARNESS_FILTER=electronexport CUA_INJECT_SOCKET="${XDG_RUNTIME_DIR}/cua-inject.sock"+elif [[ "${SESSION_KIND}" == niri ]]; then+ export CUA_E2E_COMPOSITOR=niri+ export CUA_E2E_INPUT_BACKENDS=atspi,wlr-virtual-pointerelseexport CUA_E2E_COMPOSITOR=swayexport CUA_E2E_INPUT_BACKENDS=atspi,wlr-virtual-pointer@@ -76,7 +175,11 @@ elseexport CUA_E2E_WAYLAND_XWAYLAND_REGRESSION=1fifi-export CUA_WAYLAND_RECORDING_OUTPUT=HEADLESS-1+if [[ "${SESSION_KIND}" == niri ]]; then+ export CUA_WAYLAND_RECORDING_OUTPUT=winit+else+ export CUA_WAYLAND_RECORDING_OUTPUT=HEADLESS-1+fiexport ELECTRON_OZONE_PLATFORM_HINT=waylandexport GDK_BACKEND=waylandexport QT_QPA_PLATFORM=wayland@@ -92,18 +195,16 @@ export WEBKIT_DISABLE_SANDBOX_THIS_IS_DANGEROUS=1export NO_AT_BRIDGE=0export ACCESSIBILITY_ENABLED=1-if [[ -z "${DBUS_SESSION_BUS_ADDRESS:-}" ]]; then- dbus_daemon="$(command -v dbus-daemon)"- dbus_prefix="$(dirname "$(dirname "$(readlink -f "${dbus_daemon}")")")"- dbus_config="${dbus_prefix}/share/dbus-1/session.conf"- if [[ ! -f "${dbus_config}" ]]; then- echo "DBus session config is missing: ${dbus_config}" >&2- exit 1- fi- dbus_info="$(dbus-daemon --config-file="${dbus_config}" --fork --print-address=1 --print-pid=1)"- export DBUS_SESSION_BUS_ADDRESS="$(sed -n '1p' <<< "${dbus_info}")"- DBUS_PID="$(sed -n '2p' <<< "${dbus_info}")"+dbus_daemon="$(command -v dbus-daemon)"+dbus_prefix="$(dirname "$(dirname "$(readlink -f "${dbus_daemon}")")")"+dbus_config="${dbus_prefix}/share/dbus-1/session.conf"+if [[ ! -f "${dbus_config}" ]]; then+ echo "DBus session config is missing: ${dbus_config}" >&2+ exit 1fi+dbus_info="$(dbus-daemon --config-file="${dbus_config}" --fork --print-address=1 --print-pid=1)"+export DBUS_SESSION_BUS_ADDRESS="$(sed -n '1p' <<< "${dbus_info}")"+DBUS_PID="$(sed -n '2p' <<< "${dbus_info}")"# The shared lane exercises the encrypted Computer History lifecycle. Give its# Linux key provider the same unlocked Secret Service that the canonical X11@@ -114,8 +215,15 @@ if [[ "${CUA_E2E_INTERNAL_LANE:-all}" == "shared" \echo "gnome-keyring-daemon is required for the shared Wayland history gate" >&2exit 1fi+ # The history gate purges the cua-driver-local credential from whichever+ # Secret Service it reaches. Give the daemon its own home and data+ # directories so the disposable keyring never touches the user's.+ mkdir -m 700 "${RUNTIME_DIR}/keyring-home"keyring_environment="$(- printf 'test-password' | gnome-keyring-daemon --unlock --components=secrets+ printf 'test-password' \+ | HOME="${RUNTIME_DIR}/keyring-home" \+ XDG_DATA_HOME="${RUNTIME_DIR}/keyring-home/.local/share" \+ gnome-keyring-daemon --unlock --components=secrets)"eval "${keyring_environment}"deadline=$((SECONDS + 15))@@ -135,6 +243,14 @@ if [[ "${CUA_E2E_INTERNAL_LANE:-all}" == "shared" \echo "Secret Service did not become ready on the private Wayland session bus" >&2exit 1fi+ # The daemon detaches from this shell; stop it by the bus name it owns.+ KEYRING_PID="$(+ gdbus call --session \+ --dest org.freedesktop.DBus \+ --object-path /org/freedesktop/DBus \+ --method org.freedesktop.DBus.GetConnectionUnixProcessID org.freedesktop.secrets \+ | sed -n 's/^(uint32 \([0-9]*\),)$/\1/p'+ )"fi# A private session bus does not activate the desktop accessibility stack by@@ -145,7 +261,11 @@ if [[ ! -x "${ATSPI_LAUNCHER}" ]]; thenecho "AT-SPI bus launcher is unavailable: ${ATSPI_LAUNCHER:-<not found>}" >&2exit 1fi-"${ATSPI_LAUNCHER}" --launch-immediately > "${ATSPI_LOG}" 2>&1 &+# Inside a systemd user unit, as in any desktop terminal, the launcher prefers+# dbus-broker, whose service activation goes through the host's systemd user+# manager instead of this private bus.+ATSPI_DBUS_IMPLEMENTATION=dbus-daemon \+ "${ATSPI_LAUNCHER}" --launch-immediately > "${ATSPI_LOG}" 2>&1 &ATSPI_PID=$!deadline=$((SECONDS + 15))while ((SECONDS < deadline)); do@@ -208,6 +328,48 @@ if [[ "${SESSION_KIND}" == cua-compositor ]]; thenexit 1}cua-compositor > "${COMPOSITOR_LOG}" 2>&1 &+elif [[ "${SESSION_KIND}" == niri ]]; then+ for tool in cage niri wlr-randr; do+ command -v "${tool}" >/dev/null || {+ echo "${tool} is required for the nested niri lane; use nix develop .#cua-driver-niri-e2e" >&2+ exit 1+ }+ done+ if ! niri validate --config "${NIRI_CONFIG}" > "${COMPOSITOR_LOG}" 2>&1; then+ cat "${COMPOSITOR_LOG}" >&2+ exit 1+ fi+ mkdir -m 700 "${CAGE_RUNTIME_DIR}"+ XDG_RUNTIME_DIR="${CAGE_RUNTIME_DIR}" cage -d -- sleep infinity > "${CAGE_LOG}" 2>&1 &+ CAGE_PID=$!+ deadline=$((SECONDS + 20))+ while [[ ! -S "${CAGE_RUNTIME_DIR}/wayland-0" ]] && ((SECONDS < deadline)) \+ && kill -0 "${CAGE_PID}" 2>/dev/null; do+ sleep 0.2+ done+ # niri's winit output takes the size of cage's headless output.+ if [[ ! -S "${CAGE_RUNTIME_DIR}/wayland-0" ]] \+ || ! XDG_RUNTIME_DIR="${CAGE_RUNTIME_DIR}" WAYLAND_DISPLAY=wayland-0 \+ timeout 5 wlr-randr --output HEADLESS-1 --custom-mode 1920x1080 >> "${CAGE_LOG}" 2>&1; then+ echo "cage did not provide a 1920x1080 headless output for niri" >&2+ cat "${CAGE_LOG}" >&2+ exit 1+ fi+ # Without a GPU, niri redraws the whole output in llvmpipe for every frame+ # the behaviour recording copies, and llvmpipe's thread per CPU starves the+ # apps under test: Electron AT-SPI walks ran about 4x slower while recording+ # on the 4-vCPU hosted runner. In the shared lane, one render thread keeps+ # the recording and leaves the other CPUs to the apps. Other lanes keep+ # llvmpipe's default: a single thread on that runner makes each niri frame+ # take longer than the agent cursor's 50 ms animation step, so the cursor+ # animates in slow motion past the overlay idle test's settle time. It has+ # no effect on a hardware GPU.+ niri_render_env=()+ if [[ "${CUA_E2E_INTERNAL_LANE:-all}" == shared ]]; then+ niri_render_env=(LP_NUM_THREADS=1)+ fi+ WAYLAND_DISPLAY="${CAGE_RUNTIME_DIR}/wayland-0" env "${niri_render_env[@]}" \+ niri --config "${NIRI_CONFIG}" >> "${COMPOSITOR_LOG}" 2>&1 &elsesway --unsupported-gpu --config "${SWAY_CONFIG}" > "${COMPOSITOR_LOG}" 2>&1 &fi@@ -234,7 +396,59 @@ if [[ -z "${WAYLAND_DISPLAY:-}" ]]; thenexit 1fi-if [[ "${SESSION_KIND}" == cua-compositor ]]; then+if [[ "${SESSION_KIND}" == niri ]]; then+ # niri names its IPC socket niri.<wayland display>.<pid>.sock. Accept only+ # the socket of the niri this runner started, for the display found above.+ export NIRI_SOCKET="${XDG_RUNTIME_DIR}/niri.${WAYLAND_DISPLAY}.${COMPOSITOR_PID}.sock"+ deadline=$((SECONDS + 10))+ while [[ ! -S "${NIRI_SOCKET}" ]] && ((SECONDS < deadline)); do+ sleep 0.2+ done+ if [[ ! -S "${NIRI_SOCKET}" ]]; then+ echo "niri did not expose its IPC socket ${NIRI_SOCKET}" >&2+ cat "${COMPOSITOR_LOG}" >&2+ exit 1+ fi+ # Both sockets must be served by this niri, and niri must drive only its+ # nested winit output: a host niri would list real outputs. Every niri msg+ # has a deadline because a wedged niri accepts connections without replying.+ socket_peers="$(python3 -I - "${XDG_RUNTIME_DIR}/${WAYLAND_DISPLAY}" "${NIRI_SOCKET}" <<'PY'+import socket+import struct+import sys++for path in sys.argv[1:]:+ with socket.socket(socket.AF_UNIX) as connection:+ connection.settimeout(5)+ connection.connect(path)+ credentials = connection.getsockopt(+ socket.SOL_SOCKET, socket.SO_PEERCRED, struct.calcsize("3i")+ )+ print(struct.unpack("3i", credentials)[0])+PY+)"+ if [[ "$(sort -u <<< "${socket_peers}")" != "${COMPOSITOR_PID}" ]]; then+ echo "niri sockets are not served by niri pid ${COMPOSITOR_PID}: ${socket_peers//$'\n'/ }" >&2+ exit 1+ fi+ niri_outputs="$(timeout 5 niri msg --json outputs)"+ if [[ "$(jq -c 'keys' <<< "${niri_outputs}")" != '["winit"]' ]] \+ || [[ "$(jq -c '.winit.logical | [.width, .height]' <<< "${niri_outputs}")" \+ != "$(jq -nc --argjson scale "${NIRI_SCALE}" '[1920, 1080] | map(. / $scale | floor)')" ]]; then+ echo "nested niri must drive only its 1920x1080 winit output: ${niri_outputs}" >&2+ exit 1+ fi+ jq -n \+ --arg source_sha "${CUA_E2E_SOURCE_SHA:-unknown}" \+ --arg session_type "${XDG_SESSION_TYPE}" \+ --arg compositor "${CUA_E2E_COMPOSITOR}" \+ --arg wayland_display "${WAYLAND_DISPLAY}" \+ --arg cage "$(cage -v 2>&1)" \+ --argjson niri "$(timeout 5 niri msg --json version)" \+ --argjson outputs "${niri_outputs}" \+ '{source_sha:$source_sha,session_type:$session_type,compositor:$compositor,wayland_display:$wayland_display,cage:$cage,niri:$niri,outputs:$outputs}' \+ > "${REPO_ROOT}/artifacts/cua-driver/linux/niri-preflight.json"+elif [[ "${SESSION_KIND}" == cua-compositor ]]; thendeadline=$((SECONDS + 10))while [[ ! -S "${CUA_INJECT_SOCKET}" ]] && ((SECONDS < deadline)); dosleep 0.2@@ -291,4 +505,10 @@ set -eif [[ "${status}" != 0 && "${SESSION_KIND}" == sway* ]]; thenswaymsg -t get_tree > "${REPO_ROOT}/artifacts/cua-driver/linux/sway-tree.json" 2>/dev/null || truefi+if [[ "${status}" != 0 && "${SESSION_KIND}" == niri ]]; then+ for request in windows workspaces outputs; do+ timeout 5 niri msg --json "${request}" \+ > "${REPO_ROOT}/artifacts/cua-driver/linux/niri-${request}.json" 2>/dev/null || true+ done+fiexit "${status}"diff --git a/scripts/docs-generators/cli-specs/cua-driver-mcp-linux.json b/scripts/docs-generators/cli-specs/cua-driver-mcp-linux.jsonindex 07f9b2fa53..c52e97a41e 100644--- a/scripts/docs-generators/cli-specs/cua-driver-mcp-linux.json+++ b/scripts/docs-generators/cli-specs/cua-driver-mcp-linux.json@@ -510,6 +510,7 @@},"count": {"description": "Number of clicks in one press train: 1 (default), 2 = double-click, 3 = triple-click (line/paragraph selection). There is no separate triple_click tool — pass count: 3 here.",+ "maximum": 3,"minimum": 1,"type": "integer"},