Skip to content

armgit / cua

Public Imported from GitHub
0

Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.

feat(cua-driver): add trusted KDE KWin Wayland target identity adapter (#3336)

Piotr Matela9596fb334f3e
diff --git a/libs/cua-driver/docs/action-support.md b/libs/cua-driver/docs/action-support.md
index 526ea5401c..a728cf9ec7 100644
--- a/libs/cua-driver/docs/action-support.md
+++ b/libs/cua-driver/docs/action-support.md
@@ -77,7 +77,7 @@ are capabilities, not one uniform API.
| X11/Openbox | Exact run `29254936043` passed all 116 declared outcomes: 75 deliveries and 41 exact refusals. GTK3 AT-SPI actions and values, foreground XTest pointer/keyboard routes, capture, desktop scope, and the complete Electron/Tauri catalog all reached fixture-owned oracles. | Xvfb does not prove the real-Xorg MPX/uinput background pointer route. Toolkits that reject XSendEvent retain exact refusals. |
| Sway/wlroots | Runs `29255208927` and `29257961614` passed the complete 116 outcomes: native and capture 36/36, then shared 80/80. The Electron, Tauri, GTK3, capture, and desktop-scope catalogs use live Sway identity/geometry and external fixture oracles. | Stock Wayland cannot target raw focus-bound input at an occluded surface. Those rows retain exact refusals; another wlroots compositor is not assumed equivalent until a material divergence is reported and tested. Exact cursor preservation remains unproven and is tracked in [issue #2194](https://github.com/trycua/cua/issues/2194). |
| GNOME/Mutter | A real GNOME 46 Wayland run passed the full GTK3 matrix, 31/31. The WinRects helper supplies stable window ids, frame and buffer geometry, stacking, verified activation, stage capture, and the compositor cursor. AT-SPI handles semantic actions; persistent portal/libei sessions deliver foreground PX click, right/double click, drag, scroll, type, key, and hotkey. Background rows either deliver through AT-SPI with focus and leak guards or return the declared exact refusal. | The helper requires installation plus one Shell-session restart. Without it, target-bound foreground input refuses. Portal video and a shared Electron/Tauri GNOME run remain open. |
-| KDE/KWin | A Plasma 6 session reached GTK AT-SPI discovery, generic toplevel discovery, and portal-interface preflight. Portal input is compiled into release binaries. | No behavioral matrix is accepted. A target-addressable KWin activation adapter is not implemented, so foreground portal/libei input refuses instead of injecting into the wrong focused app. |
+| KDE/KWin | A Plasma 6 Wayland session has a live-validated Cua-owned native KWin identity adapter: trusted in-process helper, opaque KWin-lifetime tokens, PID/geometry correlation, active/minimized metadata, and live-window token pruning. | Raw target-addressed keyboard and pointer actions are intentionally refused. KWin portal/libei delivery is focus-bound, so activation plus read-back cannot prevent focus changing before compositor EIS processing. Delivery remains disabled until a KWin input path binds the mutation itself to the target token/window. The ABI-specific helper remains optional and is currently obtained by a host build. |
| Nested `cua-compositor` | The optional backend owns its nested session and is covered by typed shared, native, and capture catalogs. Run `29197643541` passed native GTK3 31/31; run `29197887387` passed capture/scope 5/5. Full shared run `29199596935` passed 26/36 Electron rows before the occluded-subtree and wheel-frame repairs. | The lane remains experimental. The accepted full shared run still has 10 failures; focused replacement runs reduce that set but do not promote the environment. Unicode text and a canonical parallel-drag row remain unproven. This private route is not evidence of a stock-Wayland capability. |
PX background left-click rows may resolve the screen point to an actionable
diff --git a/libs/cua-driver/kwin-target-helper/CMakeLists.txt b/libs/cua-driver/kwin-target-helper/CMakeLists.txt
new file mode 100644
index 0000000000..77a11a7d27
--- /dev/null
+++ b/libs/cua-driver/kwin-target-helper/CMakeLists.txt
@@ -0,0 +1,20 @@
+cmake_minimum_required(VERSION 3.21)
+project(cua_kwin_target_helper LANGUAGES CXX)
+
+set(CMAKE_AUTOMOC ON)
+set(BUILD_SHARED_LIBS ON)
+set(CMAKE_LIBRARY_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}")
+
+find_package(Qt6 REQUIRED COMPONENTS Core DBus Gui Widgets)
+find_package(KF6CoreAddons REQUIRED)
+find_package(KWin REQUIRED)
+include(KDEInstallDirs)
+include(KDECMakeSettings)
+set(KDE_INSTALL_PLUGINDIR "${CMAKE_INSTALL_LIBDIR}/qt6/plugins")
+set(KDE_INSTALL_QTPLUGINDIR "${CMAKE_INSTALL_LIBDIR}/qt6/plugins")
+
+kcoreaddons_add_plugin(cua_kwin_target_helper
+ SOURCES kwin_target_helper.cpp
+ INSTALL_NAMESPACE "kwin/effects/plugins")
+target_compile_features(cua_kwin_target_helper PRIVATE cxx_std_20)
+target_link_libraries(cua_kwin_target_helper PRIVATE KWin::kwin Qt6::Core Qt6::DBus)
diff --git a/libs/cua-driver/kwin-target-helper/README.md b/libs/cua-driver/kwin-target-helper/README.md
new file mode 100644
index 0000000000..1e199d2760
--- /dev/null
+++ b/libs/cua-driver/kwin-target-helper/README.md
@@ -0,0 +1,77 @@
+# Cua KWin target helper
+
+This is an **optional KDE Plasma/KWin Wayland integration**. It is not part of
+the portable `cua-driver` binary and must be built against the user's installed
+KWin/Qt/KF6 development files. Do not use a helper built for a different KWin
+or Qt ABI.
+
+The helper is intentionally **read-only**. It provides trusted KWin window
+identity and state metadata: stable opaque tokens, PID, geometry,
+active/minimized state, and stacking order. It does not expose window activation
+or input mutation methods. KWin portal/libei input is focus-bound, so focus can
+change between any focus check and compositor-side input processing. Cua
+therefore refuses raw target-addressed KWin keyboard/pointer actions until a
+target-bound KWin input path exists.
+
+## Requirements
+
+- KDE Plasma/KWin 6 with a matching KWin development package
+- Qt 6 development files
+- ECM/KF6CoreAddons development files
+- CMake 3.21 or newer
+- a running Wayland KWin session for loading/testing
+
+On Arch-derived systems, install the distribution's matching packages (package
+names vary by distribution), then build from this directory:
+
+```bash
+cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
+cmake --build build --parallel
+```
+
+The module is installed in KWin's effect plugin namespace:
+
+```text
+kwin/effects/plugins/cua_kwin_target_helper.so
+```
+
+For a user-local installation, use a prefix supported by the local Qt/KWin
+plugin search path and verify discovery before loading:
+
+```bash
+cmake --install build --prefix "$HOME/.local"
+```
+
+If the local KWin installation does not search that prefix, use the
+distribution's native packaging mechanism or an explicitly approved
+system-local installation. Never copy the module into an arbitrary plugin
+directory and never overwrite an existing file.
+
+Load/reload through KWin's supported Effects interface where available:
+
+```bash
+qdbus6 org.kde.KWin /Effects loadEffect cua_kwin_target_helper
+```
+
+The helper exposes the Cua-owned service `org.cua.KWinTarget` only while the
+effect is loaded by the current `kwin_wayland` process. Verify the service
+owner, protocol version, and target snapshots before trusting identity data.
+The exported D-Bus surface is deliberately limited to `GetVersion` and
+`GetWindows`.
+
+## Input safety boundary
+
+Portal/libei events are delivered according to compositor focus when KWin
+processes them. A target focus check immediately before dispatch cannot bind the
+later event to that target, so adding more checks does not remove the race.
+Until a future KWin integration can bind the mutation itself to an exact target,
+Cua fails closed rather than sending raw focus-based input to a requested KWin
+window.
+
+## ABI and rollback
+
+The module is compositor/Qt ABI-specific. Rebuild after Plasma/KWin or Qt
+upgrades. To roll back a user-local test installation, unload the effect and
+remove only the exact module installed by that test; do not remove KWin or
+change the desktop session. The portable Cua Driver continues to build without
+this optional helper and without KWin development packages.
diff --git a/libs/cua-driver/kwin-target-helper/kwin_target_helper.cpp b/libs/cua-driver/kwin-target-helper/kwin_target_helper.cpp
new file mode 100644
index 0000000000..4efd0ccd3e
--- /dev/null
+++ b/libs/cua-driver/kwin-target-helper/kwin_target_helper.cpp
@@ -0,0 +1,128 @@
+// SPDX-License-Identifier: MIT
+// Cua-owned KWin effect. The Rust driver verifies that this D-Bus name is
+// owned by the running kwin_wayland process before it trusts any response.
+// Protocol v1 is intentionally read-only: only version and window snapshots
+// are exported; input and activation remain outside this helper.
+
+#include <kwin/effect/effect.h>
+#include <kwin/window.h>
+#include <kwin/workspace.h>
+
+#include <QDBusConnection>
+#include <QHash>
+#include <QJsonArray>
+#include <QJsonDocument>
+#include <QJsonObject>
+#include <QSet>
+#include <QtMath>
+#include <QUuid>
+
+namespace
+{
+constexpr auto kService = "org.cua.KWinTarget";
+constexpr auto kPath = "/org/cua/KWinTarget";
+constexpr quint32 kProtocolVersion = 1;
+
+class TargetApi : public QObject
+{
+ Q_OBJECT
+ Q_CLASSINFO("D-Bus Interface", "org.cua.KWinTarget")
+
+public slots:
+ Q_SCRIPTABLE quint32 GetVersion()
+ {
+ return kProtocolVersion;
+ }
+
+ Q_SCRIPTABLE QString GetWindows()
+ {
+ const auto liveWindows = KWin::Workspace::self()->windows();
+ QSet<QUuid> liveIds;
+ for (KWin::Window *window : liveWindows) {
+ if (window && window->isClient() && !window->isDeleted()) {
+ liveIds.insert(window->internalId());
+ }
+ }
+ for (auto it = m_tokens.begin(); it != m_tokens.end();) {
+ if (!liveIds.contains(it.key())) {
+ it = m_tokens.erase(it);
+ } else {
+ ++it;
+ }
+ }
+
+ QJsonArray windows;
+ for (KWin::Window *window : liveWindows) {
+ if (!window || !window->isClient() || window->isDeleted()) {
+ continue;
+ }
+ const auto frame = window->frameGeometry();
+ QJsonObject record;
+ record["token"] = static_cast<qint64>(tokenFor(window->internalId()));
+ record["pid"] = window->pid();
+ record["title"] = window->captionNormal();
+ record["app_id"] = window->resourceClass();
+ record["x"] = qRound(frame.x());
+ record["y"] = qRound(frame.y());
+ record["w"] = qRound(frame.width());
+ record["h"] = qRound(frame.height());
+ record["active"] = window->isActive();
+ record["minimized"] = window->isMinimized();
+ record["stacking"] = window->stackingOrder();
+ windows.push_back(record);
+ }
+ return QString::fromUtf8(QJsonDocument(windows).toJson(QJsonDocument::Compact));
+ }
+
+private:
+ quint64 tokenFor(const QUuid &id)
+ {
+ if (const auto token = m_tokens.constFind(id); token != m_tokens.cend()) {
+ return *token;
+ }
+ const quint64 token = m_nextToken++;
+ m_tokens.insert(id, token);
+ return token;
+ }
+
+ QHash<QUuid, quint64> m_tokens;
+ quint64 m_nextToken = 1;
+};
+
+class KwinTargetHelperEffect : public KWin::Effect
+{
+public:
+ KwinTargetHelperEffect()
+ {
+ auto bus = QDBusConnection::sessionBus();
+ if (!bus.registerService(QString::fromLatin1(kService))) {
+ return;
+ }
+ m_exported = bus.registerObject(
+ QString::fromLatin1(kPath),
+ &m_api,
+ QDBusConnection::ExportScriptableSlots);
+ if (!m_exported) {
+ bus.unregisterService(QString::fromLatin1(kService));
+ }
+ }
+
+ ~KwinTargetHelperEffect() override
+ {
+ if (!m_exported) {
+ return;
+ }
+ auto bus = QDBusConnection::sessionBus();
+ bus.unregisterObject(QString::fromLatin1(kPath));
+ bus.unregisterService(QString::fromLatin1(kService));
+ }
+
+private:
+ TargetApi m_api;
+ bool m_exported = false;
+};
+}
+
+KWIN_EFFECT_FACTORY(KwinTargetHelperEffect, "kwin_target_helper.json")
+
+#include "kwin_target_helper.moc"
diff --git a/libs/cua-driver/kwin-target-helper/kwin_target_helper.json b/libs/cua-driver/kwin-target-helper/kwin_target_helper.json
new file mode 100644
index 0000000000..7449f261a3
--- /dev/null
+++ b/libs/cua-driver/kwin-target-helper/kwin_target_helper.json
@@ -0,0 +1,11 @@
+{
+ "KPlugin": {
+ "Name": "Cua KWin Target Helper",
+ "Description": "Provides read-only trusted KWin window identity metadata for Cua Driver; raw focus-bound input remains disabled.",
+ "License": "MIT",
+ "EnabledByDefault": false
+ },
+ "org.kde.kwin.effect": {
+ "enabledByDefaultMethod": false
+ }
+}
diff --git a/libs/cua-driver/rust/crates/platform-linux/Cargo.toml b/libs/cua-driver/rust/crates/platform-linux/Cargo.toml
index b6f74fa209..5021c634f8 100644
--- a/libs/cua-driver/rust/crates/platform-linux/Cargo.toml
+++ b/libs/cua-driver/rust/crates/platform-linux/Cargo.toml
@@ -145,9 +145,14 @@ tempfile = "3"
# falls through.
tracing-subscriber = { workspace = true }
-# Cargo example that exercises the libei input path directly. Lives under
+# Cargo example that probes portal/libei device readiness without sending input.
+# Lives under
# the `portal-input` feature gate so a stock `cargo build --examples`
# doesn't try to compile against the (now-optional) reis/xkbcommon deps.
+[[example]]
+name = "libei_probe"
+required-features = ["portal-input"]
+
[[example]]
name = "libei_input"
required-features = ["portal-input"]
diff --git a/libs/cua-driver/rust/crates/platform-linux/examples/libei_probe.rs b/libs/cua-driver/rust/crates/platform-linux/examples/libei_probe.rs
new file mode 100644
index 0000000000..c6fa344f5a
--- /dev/null
+++ b/libs/cua-driver/rust/crates/platform-linux/examples/libei_probe.rs
@@ -0,0 +1,13 @@
+//! Portal/libei probe that establishes the session and enumerates readiness
+//! without sending keyboard or pointer input.
+
+fn main() {
+ platform_linux::wayland::libei::ensure_started().expect("start libei worker");
+ let keyboard = platform_linux::wayland::libei::wait_keyboard_ready();
+ let pointer = platform_linux::wayland::libei::wait_pointer_ready();
+ let scroll = platform_linux::wayland::libei::wait_scroll_ready();
+ println!("keyboard={keyboard:?}");
+ println!("pointer={pointer:?}");
+ println!("scroll={scroll:?}");
+ platform_linux::wayland::libei::shutdown();
+}
diff --git a/libs/cua-driver/rust/crates/platform-linux/src/browser_consent_ui.rs b/libs/cua-driver/rust/crates/platform-linux/src/browser_consent_ui.rs
index 24806906dc..64ec8ab15c 100644
--- a/libs/cua-driver/rust/crates/platform-linux/src/browser_consent_ui.rs
+++ b/libs/cua-driver/rust/crates/platform-linux/src/browser_consent_ui.rs
@@ -180,13 +180,7 @@ fn with_target_foreground<T>(
body: impl FnOnce() -> anyhow::Result<T>,
) -> anyhow::Result<T> {
if std::env::var_os("WAYLAND_DISPLAY").is_some() {
- if let Some(window) =
- crate::wayland::sway_ipc::window_for_id(window_id).filter(|window| window.pid == pid)
- {
- crate::wayland::sway_ipc::with_focused_container(window.id, body)
- } else {
- crate::wayland::shell_helper::with_focused_window(pid, window_id, body)
- }
+ crate::wayland::with_target_foreground(pid, window_id, body)
} else {
crate::input::with_x11_foreground(window_id, 80, body)
}
@@ -219,7 +213,7 @@ fn trusted_allow_click(pid: u32, window_id: u64) -> anyhow::Result<()> {
})?;
let (center_x, center_y) = exact_button_center(&tree.bounds, index)?;
if std::env::var_os("WAYLAND_DISPLAY").is_some() {
- crate::wayland::click_desktop(center_x, center_y, 1, 1)
+ crate::wayland::click_focused(center_x, center_y, 1, 1)
} else {
crate::input::send_click_xtest_desktop(center_x, center_y, 1, 1)
}
diff --git a/libs/cua-driver/rust/crates/platform-linux/src/health_report.rs b/libs/cua-driver/rust/crates/platform-linux/src/health_report.rs
index 0907a35294..d92013910f 100644
--- a/libs/cua-driver/rust/crates/platform-linux/src/health_report.rs
+++ b/libs/cua-driver/rust/crates/platform-linux/src/health_report.rs
@@ -521,7 +521,8 @@ fn portal_input_enabled() -> bool {
#[cfg(target_os = "linux")]
fn target_activation_available() -> bool {
- crate::wayland::shell_helper::list_windows(None).is_some()
+ crate::wayland::kwin_helper::available()
+ || crate::wayland::shell_helper::list_windows(None).is_some()
}
#[cfg(not(target_os = "linux"))]
diff --git a/libs/cua-driver/rust/crates/platform-linux/src/tools/impl_.rs b/libs/cua-driver/rust/crates/platform-linux/src/tools/impl_.rs
index e9b0bfdff9..a148640ac3 100644
--- a/libs/cua-driver/rust/crates/platform-linux/src/tools/impl_.rs
+++ b/libs/cua-driver/rust/crates/platform-linux/src/tools/impl_.rs
@@ -2756,7 +2756,9 @@ impl Tool for ClickTool {
// Native Wayland: focus+raise the target toplevel
// (foreign-toplevel `activate`), then drive `count` virtual-pointer
// button events. Wayland injection routes to the compositor focus.
- crate::wayland::click(xid, output_x, output_y, count as u32, button)?;
+ crate::wayland::with_target_foreground(pid, xid, || {
+ crate::wayland::click_focused(output_x, output_y, count as u32, button)
+ })?;
return Ok("wayland_activate");
}
// X11 injection. Tiered no-focus-steal delivery (background):
diff --git a/libs/cua-driver/rust/crates/platform-linux/src/wayland/kwin_helper.rs b/libs/cua-driver/rust/crates/platform-linux/src/wayland/kwin_helper.rs
new file mode 100644
index 0000000000..2c94e567a1
--- /dev/null
+++ b/libs/cua-driver/rust/crates/platform-linux/src/wayland/kwin_helper.rs
@@ -0,0 +1,442 @@
+//! Trusted KWin window identity adapter.
+//!
+//! The optional in-process KWin effect exposes stable window tokens, process
+//! identity, geometry, active/minimized state, and stacking metadata through a
+//! deliberately read-only D-Bus surface. It does not expose activation or input
+//! mutation methods. KWin's portal/libei input path is still focus-bound: libei
+//! delivers to whichever surface is focused when KWin processes the event.
+//!
+//! A focus check followed by global input therefore has an unavoidable TOCTOU
+//! race. Until Cua has a KWin input API that binds each mutation to the target
+//! token/window, this module intentionally refuses raw target-addressed input.
+//! Window discovery remains available; input authorization does not.
+
+use std::collections::HashSet;
+use std::os::unix::fs::{MetadataExt, PermissionsExt};
+use std::process::Command;
+use std::time::Duration;
+
+use crate::x11::WindowInfo;
+
+const DEST: &str = "org.cua.KWinTarget";
+const KWIN_DEST: &str = "org.kde.KWin";
+const PATH: &str = "/org/cua/KWinTarget";
+const IFACE: &str = "org.cua.KWinTarget";
+const DBUS_DEST: &str = "org.freedesktop.DBus";
+const DBUS_PATH: &str = "/org/freedesktop/DBus";
+const DBUS_IFACE: &str = "org.freedesktop.DBus";
+pub const PROTOCOL_VERSION: u32 = 1;
+const CALL_TIMEOUT: Duration = Duration::from_secs(3);
+const GEOMETRY_DELTA_PX: i32 = 64;
+
+#[derive(Clone, Debug, Eq, PartialEq)]
+pub struct KwinWindow {
+ pub token: u64,
+ pub pid: u32,
+ pub title: String,
+ pub app_name: String,
+ pub x: i32,
+ pub y: i32,
+ pub width: u32,
+ pub height: u32,
+ pub active: bool,
+ pub minimized: bool,
+ pub stacking: usize,
+}
+
+#[derive(Clone, Copy, Debug, Eq, PartialEq)]
+pub enum CorrelationError {
+ NoMatch,
+ Ambiguous,
+ WrongActiveTarget,
+}
+
+impl std::fmt::Display for CorrelationError {
+ fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+ match self {
+ Self::NoMatch => f.write_str("no exact KWin window matches the AT-SPI target"),
+ Self::Ambiguous => f.write_str("multiple KWin windows match the AT-SPI target"),
+ Self::WrongActiveTarget => f.write_str("KWin reports a different active target"),
+ }
+ }
+}
+
+impl std::error::Error for CorrelationError {}
+
+/// Whether KWin currently offers a target-bound raw-input capability that can
+/// safely authorize portal/libei foreground mutations.
+///
+/// This is intentionally distinct from helper presence: the read-only helper
+/// may be installed and usable for discovery while target-bound raw input is
+/// unavailable. Keep this false until the input operation itself is bound to
+/// the KWin target token/window.
+pub fn available() -> bool {
+ false
+}
+
+pub fn list_windows() -> Option<Vec<KwinWindow>> {
+ snapshot_for_owner(&helper_owner()?)
+}
+
+pub fn list_window_infos() -> Option<Vec<WindowInfo>> {
+ Some(
+ list_windows()?
+ .into_iter()
+ .map(|window| WindowInfo {
+ xid: window.token,
+ pid: Some(window.pid),
+ app_name: window.app_name,
+ title: window.title,
+ is_on_screen: !window.minimized && window.width > 0 && window.height > 0,
+ z_index: Some(window.stacking),
+ x: window.x,
+ y: window.y,
+ width: window.width,
+ height: window.height,
+ })
+ .collect(),
+ )
+}
+
+pub fn trusted_window_for_id(pid: u32, token: u64) -> Option<KwinWindow> {
+ snapshot_for_owner(&helper_owner()?)?
+ .into_iter()
+ .find(|window| window.pid == pid && window.token == token)
+}
+
+pub fn correlate_atspi_window(
+ atspi: &WindowInfo,
+ windows: &[KwinWindow],
+) -> Result<KwinWindow, CorrelationError> {
+ let pid = atspi.pid.ok_or(CorrelationError::NoMatch)?;
+ let matches: Vec<_> = windows
+ .iter()
+ .filter(|window| window.pid == pid && !window.minimized)
+ .filter(|window| geometry_matches(atspi, window))
+ .cloned()
+ .collect();
+ match matches.as_slice() {
+ [window] => Ok(window.clone()),
+ [] => Err(CorrelationError::NoMatch),
+ _ => Err(CorrelationError::Ambiguous),
+ }
+}
+
+pub fn require_active_target(windows: &[KwinWindow], token: u64) -> Result<(), CorrelationError> {
+ windows
+ .iter()
+ .find(|window| window.active)
+ .filter(|window| window.token == token)
+ .map(|_| ())
+ .ok_or(CorrelationError::WrongActiveTarget)
+}
+
+/// Refuse a focus-bound KWin foreground transaction.
+///
+/// `body` is deliberately never invoked. The current helper can identify an
+/// exact KWin window, but its D-Bus API is read-only; even a separate activation
+/// step would not bind a later libei event to that target. More pre-dispatch
+/// focus checks therefore cannot make the operation target-safe.
+pub fn with_focused_window<T>(
+ pid: u32,
+ token: u64,
+ _body: impl FnOnce() -> anyhow::Result<T>,
+) -> anyhow::Result<T> {
+ anyhow::bail!(
+ "foreground_unavailable: KWin target pid={pid} token={token} is verified, but only \
+ focus-bound portal/libei input is available; refusing raw input until a target-bound \
+ KWin input path is implemented"
+ )
+}
+
+fn geometry_matches(atspi: &WindowInfo, kwin: &KwinWindow) -> bool {
+ let right = atspi
+ .x
+ .saturating_add(atspi.width.min(i32::MAX as u32) as i32);
+ let bottom = atspi
+ .y
+ .saturating_add(atspi.height.min(i32::MAX as u32) as i32);
+ let kwin_right = kwin
+ .x
+ .saturating_add(kwin.width.min(i32::MAX as u32) as i32);
+ let kwin_bottom = kwin
+ .y
+ .saturating_add(kwin.height.min(i32::MAX as u32) as i32);
+ (atspi.x - kwin.x).abs() <= GEOMETRY_DELTA_PX
+ && (atspi.y - kwin.y).abs() <= GEOMETRY_DELTA_PX
+ && (right - kwin_right).abs() <= GEOMETRY_DELTA_PX
+ && (bottom - kwin_bottom).abs() <= GEOMETRY_DELTA_PX
+}
+
+fn helper_owner() -> Option<String> {
+ let owner_raw = dbus_call("GetNameOwner", &[DEST.to_owned()]);
+ let owner = parse_quoted_string(&owner_raw?)?;
+ if !owner.starts_with(':') {
+ return None;
+ }
+
+ let pid_raw = gdbus_call_to(
+ DBUS_DEST,
+ DBUS_PATH,
+ &format!("{DBUS_IFACE}.GetConnectionUnixProcessID"),
+ &[owner.clone()],
+ );
+ let uid_raw = gdbus_call_to(
+ DBUS_DEST,
+ DBUS_PATH,
+ &format!("{DBUS_IFACE}.GetConnectionUnixUser"),
+ &[owner.clone()],
+ );
+ let pid = parse_first_u32(&pid_raw?)?;
+ let uid = parse_first_u32(&uid_raw?)?;
+ if uid != current_uid() || !is_trusted_kwin(pid, &owner) {
+ return None;
+ }
+
+ let version_raw = call_to(&owner, "GetVersion", &[]);
+ let version = parse_first_u32(&version_raw?)?;
+ (version == PROTOCOL_VERSION).then_some(owner)
+}
+
+fn snapshot_for_owner(owner: &str) -> Option<Vec<KwinWindow>> {
+ // Keep this synchronous API safe to call from either sync or async driver
+ // code by running the zbus/Tokio client on its own OS thread. In
+ // particular, do not use zbus's blocking wrapper directly from a Tokio
+ // task: that would create an async-sandwich nested-runtime hazard.
+ let owner = owner.to_owned();
+ std::thread::spawn(move || {
+ let runtime = tokio::runtime::Builder::new_current_thread()
+ .enable_all()
+ .build()
+ .ok()?;
+
+ runtime.block_on(async move {
+ let connection = tokio::time::timeout(CALL_TIMEOUT, zbus::Connection::session())
+ .await
+ .ok()?
+ .ok()?;
+ let proxy = tokio::time::timeout(
+ CALL_TIMEOUT,
+ zbus::Proxy::new(&connection, owner.as_str(), PATH, IFACE),
+ )
+ .await
+ .ok()?
+ .ok()?;
+
+ for attempt in 0..3 {
+ let raw = tokio::time::timeout(
+ CALL_TIMEOUT,
+ proxy.call::<_, _, String>("GetWindows", &()),
+ )
+ .await
+ .ok()
+ .and_then(Result::ok);
+ if let Some(snapshot) = raw.as_deref().and_then(parse_snapshot) {
+ return Some(snapshot);
+ }
+ if attempt < 2 {
+ tokio::time::sleep(Duration::from_millis(50)).await;
+ }
+ }
+ None
+ })
+ })
+ .join()
+ .ok()
+ .flatten()
+}
+
+pub fn parse_snapshot(raw: &str) -> Option<Vec<KwinWindow>> {
+ // `raw` is the actual QString payload decoded by zbus, not gdbus's
+ // human-readable GVariant rendering. Parsing the display representation is
+ // unsafe because GLib may change its quoting/escaping for valid titles.
+ let values: Vec<serde_json::Value> = serde_json::from_str(raw).ok()?;
+
+ let mut tokens = HashSet::new();
+ let mut active_count = 0usize;
+ values
+ .into_iter()
+ .map(|value| {
+ let token = value
+ .get("token")
+ .and_then(serde_json::Value::as_u64)
+ .filter(|token| *token > 0)?;
+ if !tokens.insert(token) {
+ return None;
+ }
+
+ let active = value.get("active")?.as_bool()?;
+ if active {
+ active_count += 1;
+ if active_count > 1 {
+ return None;
+ }
+ }
+
+ Some(KwinWindow {
+ token,
+ pid: u32::try_from(value.get("pid")?.as_u64()?).ok()?,
+ title: value
+ .get("title")
+ .and_then(serde_json::Value::as_str)
+ .unwrap_or_default()
+ .to_owned(),
+ app_name: value
+ .get("app_id")
+ .and_then(serde_json::Value::as_str)
+ .unwrap_or_default()
+ .to_owned(),
+ x: json_i32(&value, "x")?,
+ y: json_i32(&value, "y")?,
+ width: json_u32(&value, "w")?,
+ height: json_u32(&value, "h")?,
+ active,
+ minimized: value.get("minimized")?.as_bool()?,
+ stacking: usize::try_from(value.get("stacking")?.as_u64()?).ok()?,
+ })
+ })
+ .collect()
+}
+
+fn json_i32(value: &serde_json::Value, key: &str) -> Option<i32> {
+ let number = value.get(key)?.as_f64()?;
+ (number.is_finite() && number >= i32::MIN as f64 && number <= i32::MAX as f64)
+ .then(|| number.round() as i32)
+}
+
+fn json_u32(value: &serde_json::Value, key: &str) -> Option<u32> {
+ let number = value.get(key)?.as_f64()?;
+ (number.is_finite() && number >= 0.0 && number <= u32::MAX as f64)
+ .then(|| number.round() as u32)
+}
+
+fn dbus_call(method: &str, args: &[String]) -> Option<String> {
+ gdbus_call_to(
+ DBUS_DEST,
+ DBUS_PATH,
+ &format!("{DBUS_IFACE}.{method}"),
+ args,
+ )
+}
+
+fn call_to(owner: &str, method: &str, args: &[String]) -> Option<String> {
+ gdbus_call_to(owner, PATH, &format!("{IFACE}.{method}"), args)
+}
+
+fn gdbus_call_to(
+ destination: &str,
+ object_path: &str,
+ method: &str,
+ args: &[String],
+) -> Option<String> {
+ let mut command = Command::new("gdbus");
+ command
+ .arg("call")
+ .arg("--session")
+ .arg("--dest")
+ .arg(destination)
+ .arg("--object-path")
+ .arg(object_path)
+ .arg("--method")
+ .arg(method)
+ .args(args)
+ .stdin(std::process::Stdio::null())
+ .stdout(std::process::Stdio::piped())
+ .stderr(std::process::Stdio::null());
+ let child = command.spawn().ok()?;
+ wait_timeout(child, CALL_TIMEOUT)
+}
+
+fn wait_timeout(mut child: std::process::Child, timeout: Duration) -> Option<String> {
+ use std::io::Read;
+
+ let mut stdout = child.stdout.take()?;
+ let reader = std::thread::spawn(move || {
+ let mut bytes = Vec::new();
+ stdout.read_to_end(&mut bytes).ok()?;
+ String::from_utf8(bytes).ok()
+ });
+ let deadline = std::time::Instant::now() + timeout;
+ loop {
+ match child.try_wait() {
+ Ok(Some(status)) if status.success() => return reader.join().ok().flatten(),
+ Ok(Some(_)) | Err(_) => return None,
+ Ok(None) if std::time::Instant::now() >= deadline => {
+ let _ = child.kill();
+ let _ = child.wait();
+ let _ = reader.join();
+ return None;
+ }
+ Ok(None) => std::thread::sleep(Duration::from_millis(10)),
+ }
+ }
+}
+
+fn parse_quoted_string(raw: &str) -> Option<String> {
+ let start = raw.find('\'')? + 1;
+ let end = raw[start..].find('\'')? + start;
+ (end > start).then(|| raw[start..end].to_owned())
+}
+
+fn parse_first_u32(raw: &str) -> Option<u32> {
+ let payload = raw.split_once("uint32").map_or(raw, |(_, payload)| payload);
+ payload
+ .split(|character: char| !character.is_ascii_digit())
+ .find(|part| !part.is_empty())?
+ .parse()
+ .ok()
+}
+
+fn current_uid() -> u32 {
+ std::fs::metadata(format!("/proc/{}", std::process::id()))
+ .map(|meta| meta.uid())
+ .unwrap_or(u32::MAX)
+}
+
+fn is_trusted_kwin(pid: u32, helper_owner: &str) -> bool {
+ let comm = std::fs::read_to_string(format!("/proc/{pid}/comm")).ok();
+ if comm.as_deref().map(str::trim) != Some("kwin_wayland") {
+ return false;
+ }
+
+ let Some(owner_raw) = dbus_call("GetNameOwner", &[KWIN_DEST.to_owned()]) else {
+ return false;
+ };
+ let Some(kwin_owner) = parse_quoted_string(&owner_raw) else {
+ return false;
+ };
+ if kwin_owner != helper_owner {
+ return false;
+ }
+
+ let executable = std::fs::read_link(format!("/proc/{pid}/exe")).ok();
+ match executable {
+ Some(path) => {
+ let metadata = std::fs::metadata(&path).ok();
+ path.file_name().and_then(|name| name.to_str()) == Some("kwin_wayland")
+ && metadata
+ .is_some_and(|meta| meta.uid() == 0 && meta.permissions().mode() & 0o022 == 0)
+ }
+ None => true,
+ }
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ #[test]
+ fn raw_input_capability_is_fail_closed() {
+ assert!(!available());
+
+ let called = std::cell::Cell::new(false);
+ let error = with_focused_window(42, 7, || {
+ called.set(true);
+ Ok(())
+ })
+ .expect_err("focus-bound KWin input must refuse");
+
+ assert!(!called.get());
+ assert!(error.to_string().contains("target-bound KWin input path"));
+ }
+}
diff --git a/libs/cua-driver/rust/crates/platform-linux/src/wayland/mod.rs b/libs/cua-driver/rust/crates/platform-linux/src/wayland/mod.rs
index 631a47337c..5c1cb614a8 100644
--- a/libs/cua-driver/rust/crates/platform-linux/src/wayland/mod.rs
+++ b/libs/cua-driver/rust/crates/platform-linux/src/wayland/mod.rs
@@ -11,6 +11,7 @@
pub mod ext_screencopy;
pub mod ext_toplevel;
+pub mod kwin_helper;
pub mod overlay;
pub mod persistent_vptr;
pub(crate) mod portal;
@@ -64,6 +65,32 @@ const BTN_LEFT: u32 = 0x110;
use crate::x11::WindowInfo;
+thread_local! {
+ /// Exact foreground target currently held by an outer compositor guard.
+ /// Focus-bound helpers use this only to re-enter the guard after a blocking
+ /// portal/libei readiness wait, so the actual input is preceded by a fresh
+ /// compositor verification rather than relying on a stale pre-wait check.
+ static CURRENT_FOREGROUND_TARGET: std::cell::Cell<Option<(u32, u64)>> =
+ const { std::cell::Cell::new(None) };
+}
+
+struct ForegroundTargetGuard(Option<(u32, u64)>);
+
+impl Drop for ForegroundTargetGuard {
+ fn drop(&mut self) {
+ CURRENT_FOREGROUND_TARGET.with(|target| target.set(self.0));
+ }
+}
+
+fn bind_foreground_target(pid: u32, window_id: u64) -> ForegroundTargetGuard {
+ let previous = CURRENT_FOREGROUND_TARGET.with(|target| target.replace(Some((pid, window_id))));
+ ForegroundTargetGuard(previous)
+}
+
+fn current_foreground_target() -> Option<(u32, u64)> {
+ CURRENT_FOREGROUND_TARGET.with(std::cell::Cell::get)
+}
+
/// Name of the opt-in env var that unlocks the experimental native-Wayland
/// backend.
pub const ENABLE_WAYLAND_ENV: &str = "CUA_DRIVER_RS_ENABLE_WAYLAND";
@@ -1377,6 +1404,7 @@ pub fn with_target_foreground<T>(
window_id: u64,
body: impl FnOnce() -> anyhow::Result<T>,
) -> anyhow::Result<T> {
+ let _target_guard = bind_foreground_target(pid, window_id);
if let Some(window) = sway_ipc::window_for_id(window_id) {
if window.pid != pid {
anyhow::bail!(
@@ -1386,6 +1414,9 @@ pub fn with_target_foreground<T>(
}
return sway_ipc::with_focused_container(window_id, body);
}
+ if kwin_helper::trusted_window_for_id(pid, window_id).is_some() {
+ return kwin_helper::with_focused_window(pid, window_id, body);
+ }
if shell_helper::trusted_window_for_id(pid, window_id).is_some() {
return shell_helper::with_focused_window(pid, window_id, body);
}
@@ -1468,6 +1499,31 @@ pub fn click(window_id: u64, x: i32, y: i32, count: u32, button: u8) -> anyhow::
)
}
+/// Click through a foreground target already guarded by
+/// [`with_target_foreground`]. wlroots keeps its native virtual-pointer path.
+/// If that path is unavailable and libei needs a potentially blocking portal
+/// readiness wait, re-enter the exact foreground guard *after* readiness so
+/// KWin/GNOME/Sway ownership and focus are freshly verified immediately before
+/// the actual libei dispatch.
+#[cfg(feature = "portal-input")]
+pub fn click_focused(x: i32, y: i32, count: u32, button: u8) -> anyhow::Result<()> {
+ with_libei_fallback(
+ || click_vptr(None, x, y, count, button),
+ || {
+ libei_wait_pointer_ready()?;
+ if let Some((pid, window_id)) = current_foreground_target() {
+ return with_target_foreground(pid, window_id, || libei_click(x, y, count, button));
+ }
+ libei_click(x, y, count, button)
+ },
+ )
+}
+
+#[cfg(not(feature = "portal-input"))]
+pub fn click_focused(x: i32, y: i32, count: u32, button: u8) -> anyhow::Result<()> {
+ click_vptr(None, x, y, count, button)
+}
+
/// Click a desktop-absolute point without selecting or activating a toplevel.
/// This is the Wayland peer of an XTest root-window click and is used only by
/// the explicit desktop capture scope.
@@ -3075,10 +3131,20 @@ fn wayland_atspi_windows(filter_pid: Option<u32>) -> Vec<WindowInfo> {
windows
}
+fn apply_pid_filter(mut windows: Vec<WindowInfo>, filter_pid: Option<u32>) -> Vec<WindowInfo> {
+ if let Some(pid) = filter_pid {
+ windows.retain(|window| window.pid == Some(pid));
+ }
+ windows
+}
+
/// Window-enumeration dispatcher: native Wayland when available, else X11.
pub fn list_windows_dispatch(filter_pid: Option<u32>) -> Vec<WindowInfo> {
if wayland_enabled() && std::env::var_os("WAYLAND_DISPLAY").is_some() {
- // Prefer the richer wlroots protocol. The generic staging protocol is
+ if let Some(ws) = kwin_helper::list_window_infos() {
+ return listed_windows(apply_pid_filter(ws, filter_pid));
+ }
+ // Prefer the richer wlroots protocol when no trusted KWin helper is available.
// only consulted when wlroots yields no windows (including when its
// manager global is absent).
let native = match list_windows() {
@@ -3390,6 +3456,21 @@ mod tests {
assert_eq!(windows[2].pid, None);
}
+ #[test]
+ fn pid_filter_excludes_other_process_windows() {
+ let filtered = apply_pid_filter(
+ vec![
+ window(10, Some(100), "wanted"),
+ window(20, Some(200), "other"),
+ window(30, None, "unknown"),
+ ],
+ Some(100),
+ );
+ assert_eq!(filtered.len(), 1);
+ assert_eq!(filtered[0].pid, Some(100));
+ assert_eq!(filtered[0].xid, 10);
+ }
+
#[test]
fn zero_geometry_does_not_replace_a_real_observed_origin() {
let pid = u32::MAX - 17;
diff --git a/libs/cua-driver/rust/crates/platform-linux/tests/kwin_helper_contract.rs b/libs/cua-driver/rust/crates/platform-linux/tests/kwin_helper_contract.rs
new file mode 100644
index 0000000000..bc9192c769
--- /dev/null
+++ b/libs/cua-driver/rust/crates/platform-linux/tests/kwin_helper_contract.rs
@@ -0,0 +1,159 @@
+use platform_linux::wayland::kwin_helper::{
+ available, correlate_atspi_window, parse_snapshot, require_active_target, with_focused_window,
+ CorrelationError,
+};
+use platform_linux::x11::WindowInfo;
+
+fn atspi_window(pid: u32, x: i32, y: i32, width: u32, height: u32) -> WindowInfo {
+ WindowInfo {
+ xid: 99,
+ pid: Some(pid),
+ app_name: "org.example.Editor".into(),
+ title: "Editor".into(),
+ is_on_screen: true,
+ z_index: None,
+ x,
+ y,
+ width,
+ height,
+ }
+}
+
+#[test]
+fn parses_versioned_kwin_snapshot() {
+ let snapshot = parse_snapshot(
+ r#"[{"token":41,"pid":1200,"x":100,"y":80,"w":800,"h":600,"active":false,"minimized":false,"stacking":3}]"#,
+ )
+ .expect("valid KWin helper snapshot");
+
+ assert_eq!(snapshot.len(), 1);
+ assert_eq!(snapshot[0].token, 41);
+ assert_eq!(snapshot[0].pid, 1200);
+ assert_eq!((snapshot[0].x, snapshot[0].y), (100, 80));
+}
+
+#[test]
+fn preserves_title_with_apostrophe_from_dbus_string() {
+ let snapshot = parse_snapshot(
+ r#"[{"token":41,"pid":1200,"x":100,"y":80,"w":800,"h":600,"active":false,"minimized":false,"stacking":3,"title":"owner's editor","app_id":"org.example.Editor"}]"#,
+ )
+ .expect("apostrophe in the decoded D-Bus string must remain valid JSON");
+
+ assert_eq!(snapshot[0].title, "owner's editor");
+}
+
+#[test]
+fn parses_snapshot_when_title_contains_closing_bracket() {
+ let snapshot = parse_snapshot(
+ r#"[{"token":41,"pid":1200,"x":100,"y":80,"w":800,"h":600,"active":false,"minimized":false,"stacking":3,"title":"A ] title","app_id":"org.example.Editor"}]"#,
+ )
+ .expect("title brackets must not break snapshot parsing");
+
+ assert_eq!(snapshot[0].title, "A ] title");
+}
+
+#[test]
+fn parses_fractional_plasma_geometry() {
+ let snapshot = parse_snapshot(
+ r#"[{"token":41,"pid":1200,"x":100,"y":80,"w":332,"h":99.33333333333336,"active":false,"minimized":false,"stacking":3}]"#,
+ )
+ .expect("finite fractional geometry is valid metadata");
+
+ assert_eq!(snapshot[0].height, 99);
+}
+
+#[test]
+fn correlates_one_same_pid_window_with_bounded_frame_delta() {
+ let snapshot = parse_snapshot(
+ r#"[{"token":41,"pid":1200,"x":100,"y":80,"w":800,"h":600,"active":false,"minimized":false,"stacking":3}]"#,
+ )
+ .unwrap();
+
+ let target = correlate_atspi_window(&atspi_window(1200, 104, 112, 792, 560), &snapshot)
+ .expect("one bounded geometry match");
+ assert_eq!(target.token, 41);
+}
+
+#[test]
+fn rejects_duplicate_same_pid_geometry_candidates() {
+ let snapshot = parse_snapshot(
+ r#"[{"token":41,"pid":1200,"x":100,"y":80,"w":800,"h":600,"active":false,"minimized":false,"stacking":3},{"token":42,"pid":1200,"x":102,"y":82,"w":798,"h":598,"active":false,"minimized":false,"stacking":4}]"#,
+ )
+ .unwrap();
+
+ assert_eq!(
+ correlate_atspi_window(&atspi_window(1200, 104, 112, 792, 560), &snapshot),
+ Err(CorrelationError::Ambiguous)
+ );
+}
+
+#[test]
+fn refuses_minimized_same_pid_window() {
+ let snapshot = parse_snapshot(
+ r#"[{"token":41,"pid":1200,"x":100,"y":80,"w":800,"h":600,"active":false,"minimized":true,"stacking":3}]"#,
+ )
+ .unwrap();
+
+ assert_eq!(
+ correlate_atspi_window(&atspi_window(1200, 104, 112, 792, 560), &snapshot),
+ Err(CorrelationError::NoMatch)
+ );
+}
+
+#[test]
+fn rejects_zero_token() {
+ assert!(parse_snapshot(
+ r#"[{"token":0,"pid":1200,"x":0,"y":0,"w":10,"h":10,"active":false,"minimized":false,"stacking":0}]"#,
+ )
+ .is_none());
+}
+
+#[test]
+fn rejects_duplicate_tokens() {
+ assert!(parse_snapshot(
+ r#"[{"token":41,"pid":1200,"x":0,"y":0,"w":10,"h":10,"active":false,"minimized":false,"stacking":0},{"token":41,"pid":1300,"x":20,"y":20,"w":10,"h":10,"active":false,"minimized":false,"stacking":1}]"#,
+ )
+ .is_none());
+}
+
+#[test]
+fn rejects_multiple_active_records() {
+ assert!(parse_snapshot(
+ r#"[{"token":41,"pid":1200,"x":0,"y":0,"w":10,"h":10,"active":true,"minimized":false,"stacking":0},{"token":42,"pid":1300,"x":20,"y":20,"w":10,"h":10,"active":true,"minimized":false,"stacking":1}]"#,
+ )
+ .is_none());
+}
+
+#[test]
+fn refuses_when_a_different_token_is_active() {
+ let snapshot = parse_snapshot(
+ r#"[{"token":41,"pid":1200,"x":100,"y":80,"w":800,"h":600,"active":false,"minimized":false,"stacking":3},{"token":42,"pid":1300,"x":0,"y":0,"w":800,"h":600,"active":true,"minimized":false,"stacking":4}]"#,
+ )
+ .unwrap();
+
+ assert_eq!(
+ require_active_target(&snapshot, 41),
+ Err(CorrelationError::WrongActiveTarget)
+ );
+}
+
+#[test]
+fn raw_kwin_input_is_refused_before_dispatch() {
+ assert!(
+ !available(),
+ "focus-bound KWin input must not be advertised as safe"
+ );
+
+ let called = std::cell::Cell::new(false);
+ let error = with_focused_window(1200, 41, || {
+ called.set(true);
+ Ok(())
+ })
+ .expect_err("focus-bound KWin input must refuse");
+
+ assert!(
+ !called.get(),
+ "input body must not run on the focus-only KWin path"
+ );
+ assert!(error.to_string().contains("target-bound KWin input path"));
+}