Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.
Rust agentai-agentapplecomputer-usecomputer-use-agentcontainerizationcuadesktop-automationhacktoberfestlumemacosmanusoperatorswiftvirtualizationvirtualization-frameworkwindowswindows-sandbox
fix(hyprland-plugin): keep background popups and drags off the user's seat (CUA-3582) (#6705)
cua-agents[bot]354ed93a5cb0
diff --git a/docs/content/docs/cua-driver/concepts/platform-support.mdx b/docs/content/docs/cua-driver/concepts/platform-support.mdxindex 9296688f7c..1c1be5ef8b 100644--- a/docs/content/docs/cua-driver/concepts/platform-support.mdx+++ b/docs/content/docs/cua-driver/concepts/platform-support.mdx@@ -70,7 +70,10 @@ a narrow qualification:Evidence for this feature is recorded in PRs[#3547](https://github.com/trycua/cua/pull/3547),[#3557](https://github.com/trycua/cua/pull/3557), and-[#3572](https://github.com/trycua/cua/pull/3572). General Hyprland support+[#3572](https://github.com/trycua/cua/pull/3572). A menu or list an app opens+in the background, for example from a combo box, opens without taking the+user's keyboard, and a drag-and-drop it starts is cancelled rather than handed+the user's pointer. General Hyprland supportstill needs an accepted input and permission design, packaged install andupgrade proof, and the full native catalog on an integrated candidate.diff --git a/libs/cua-driver/hyprland-plugin/CMakeLists.txt b/libs/cua-driver/hyprland-plugin/CMakeLists.txtindex 836bf5ea36..6645e40663 100644--- a/libs/cua-driver/hyprland-plugin/CMakeLists.txt+++ b/libs/cua-driver/hyprland-plugin/CMakeLists.txt@@ -53,6 +53,12 @@ if(BUILD_TESTING)target_compile_options(cua_hyprland_foreground_route_test PRIVATE -Wall -Wextra -Wpedantic -Werror)add_test(NAME cua_hyprland_foreground_route_test COMMAND cua_hyprland_foreground_route_test)+ add_executable(cua_hyprland_user_seat_policy_test tests/user_seat_policy_test.cpp)+ target_compile_features(cua_hyprland_user_seat_policy_test PRIVATE cxx_std_26)+ target_include_directories(cua_hyprland_user_seat_policy_test PRIVATE src)+ target_compile_options(cua_hyprland_user_seat_policy_test PRIVATE -Wall -Wextra -Wpedantic -Werror)+ add_test(NAME cua_hyprland_user_seat_policy_test COMMAND cua_hyprland_user_seat_policy_test)+# Keyboard layout and foreground modifier tests compile real XKB keymaps.# Set CUA_HYPRLAND_REQUIRE_XKBCOMMON in the environment to fail instead of# skipping them when the xkbcommon development files are missing.@@ -256,7 +262,7 @@ if(CUA_HYPRLAND_BUILD_PLUGIN)message(FATAL_ERROR"Hyprland ${HYPRLAND_VERSION} headers expose no supported window input API")endif()- target_sources(cua_hyprland_plugin PRIVATE src/input_experiment.cpp)+ target_sources(cua_hyprland_plugin PRIVATE src/input_experiment.cpp src/user_seat_guard.cpp)target_compile_definitions(cua_hyprland_plugin PRIVATECUA_HYPRLAND_WINDOW_BACKEND_API=${cua_hyprland_window_backend_api})endif()diff --git a/libs/cua-driver/hyprland-plugin/README.md b/libs/cua-driver/hyprland-plugin/README.mdindex 76da33c520..ec826c30e0 100644--- a/libs/cua-driver/hyprland-plugin/README.md+++ b/libs/cua-driver/hyprland-plugin/README.md@@ -229,6 +229,16 @@ and operations outside its qualification scope.a refusal, never permission to try a weaker route.- The plugin never redirects a target-addressed request to the primary seat,focused window, pointer location, or another target.+- Two client requests take the user's whole seat in Hyprland 0.56 and 0.57,+ whatever seat and serial they name: a popup grab (a menu, a combo-box list)+ moves the user's keyboard to the popup until it closes, and a drag start makes+ the drag follow the user's pointer and drop on their next click. A background+ app can make either request when an agent seat or an accessibility action+ works in it. Input builds hook both (`src/user_seat_guard.cpp`) and grant them+ only to the client that holds the user's keyboard or pointer focus, as every+ popup and drag the user starts does. Other popups open without the grab;+ other drags are cancelled. `cua:status` reports `user_seat_guard`,+ `popup_grabs_withheld` and `drags_withheld`.- A transport acknowledgement proves only receipt and validation. Applicationbehavior requires a separate typed result and independent state evidence.diff --git a/libs/cua-driver/hyprland-plugin/packaging/release/bundle.py b/libs/cua-driver/hyprland-plugin/packaging/release/bundle.pyindex 24c78bae5a..68cbb23697 100644--- a/libs/cua-driver/hyprland-plugin/packaging/release/bundle.py+++ b/libs/cua-driver/hyprland-plugin/packaging/release/bundle.py@@ -41,6 +41,9 @@ src/protocol.cppsrc/seat_lifetime.hppsrc/session.cppsrc/status.cpp+src/user_seat_guard.cpp+src/user_seat_guard.hpp+src/user_seat_policy.hpptests/agent_key_repeat_test.pytests/cmake-api/CMakeLists.txttests/cmake-api/include/src/Compositor.hpp@@ -66,6 +69,7 @@ tests/protocol_test.cpptests/seat_lifetime_test.cpptests/status_test.cpptests/transport_test.cpp+tests/user_seat_policy_test.cpp""".split()diff --git a/libs/cua-driver/hyprland-plugin/src/plugin.cpp b/libs/cua-driver/hyprland-plugin/src/plugin.cppindex 4e92a2a03d..af764306c2 100644--- a/libs/cua-driver/hyprland-plugin/src/plugin.cpp+++ b/libs/cua-driver/hyprland-plugin/src/plugin.cpp@@ -1,6 +1,7 @@#include "inject_server.hpp"#if defined(CUA_HYPRLAND_TEST_INPUT) || defined(CUA_HYPRLAND_INPUT)#include "input_experiment.hpp"+#include "user_seat_guard.hpp"#endif#include "cua_hyprland/protocol.hpp"@@ -218,6 +219,14 @@ std::string status_output(bool json) {if (json) {auto result = cua::hyprland::render_status_json(report);+#if defined(CUA_HYPRLAND_TEST_INPUT) || defined(CUA_HYPRLAND_INPUT)+ // Whether popups and drags the user did not start are kept off the+ // user's seat (user_seat_policy.hpp), and how many it withheld.+ result.pop_back();+ result += std::format(R"(,"user_seat_guard":{},"popup_grabs_withheld":{},"drags_withheld":{}}})",+ cua::hyprland::user_seat_guard_active(), cua::hyprland::popup_grabs_withheld(),+ cua::hyprland::drags_withheld());+#endif#ifdef CUA_HYPRLAND_INPUT// Compiled keyboard behavior of this loaded module, reported whether or// not input is enabled, so an installer can tell it from an older module@@ -341,6 +350,13 @@ APICALL EXPORT PLUGIN_DESCRIPTION_INFO PLUGIN_INIT(HANDLE handle) {}});+#if defined(CUA_HYPRLAND_TEST_INPUT) || defined(CUA_HYPRLAND_INPUT)+ // The agent seats can open popups and start drags in a background app, and+ // so can an accessibility action; neither may take the user's seat. Best+ // effort: without the hooks the plugin still works, and status says so.+ static_cast<void>(cua::hyprland::install_user_seat_guard(handle));+#endif+return {"cua-hyprland-plugin","Discovery foundation for isolated background computer use",@@ -351,6 +367,9 @@ APICALL EXPORT PLUGIN_DESCRIPTION_INFO PLUGIN_INIT(HANDLE handle) {APICALL EXPORT void PLUGIN_EXIT() {g_config_listener.reset();+#if defined(CUA_HYPRLAND_TEST_INPUT) || defined(CUA_HYPRLAND_INPUT)+ cua::hyprland::remove_user_seat_guard();+#endif#if defined(CUA_HYPRLAND_TEST_INPUT) || defined(CUA_HYPRLAND_INPUT)// Retire input directly so an in-flight action receives plugin_shutdown,// not the config-disable reason from stop_server()'s suspend path.diff --git a/libs/cua-driver/hyprland-plugin/src/user_seat_guard.cpp b/libs/cua-driver/hyprland-plugin/src/user_seat_guard.cppnew file mode 100644index 0000000000..0db8763306--- /dev/null+++ b/libs/cua-driver/hyprland-plugin/src/user_seat_guard.cpp@@ -0,0 +1,115 @@+#include "user_seat_guard.hpp"++#include "user_seat_policy.hpp"++#include <src/managers/SeatManager.hpp>+#include <src/plugins/PluginAPI.hpp>+#include <src/protocols/XDGShell.hpp>+#include <src/protocols/core/Compositor.hpp>+#include <src/protocols/core/DataDevice.hpp>++#include <atomic>+#include <string>++namespace cua::hyprland {+namespace {++void* g_plugin = nullptr;+CFunctionHook* g_add_or_start_grab = nullptr;+CFunctionHook* g_initiate_drag = nullptr;+std::atomic<std::uint64_t> g_grabs_withheld = 0;+std::atomic<std::uint64_t> g_drags_withheld = 0;++// The one place an xdg_popup.grab request reaches the seat, and the one place a+// wl_data_device.start_drag request does, in Hyprland 0.56.2 and 0.57.+constexpr auto kAddOrStartGrab =+ "_ZN17CXDGShellProtocol14addOrStartGrabEN9Hyprutils6Memory14CSharedPointerI17CXDGPopupResourceEE";+constexpr auto kInitiateDrag =+ "_ZN21CWLDataDeviceProtocol12initiateDragEN9Hyprutils6Memory12CWeakPointerI21CWLDataSourceResourceEENS1_"+ "14CSharedPointerI18CWLSurfaceResourceEES7_";++using AddOrStartGrab = void (*)(CXDGShellProtocol*, SP<CXDGPopupResource>);+using InitiateDrag = void (*)(CWLDataDeviceProtocol*, WP<CWLDataSourceResource>, SP<CWLSurfaceResource>,+ SP<CWLSurfaceResource>);++const wl_client* client_of(const SP<CWLSurfaceResource>& surface) {+ return surface ? surface->client() : nullptr;+}++bool user_holds(const SP<CWLSurfaceResource>& surface) {+ return user_seat_request_allowed(client_of(surface), client_of(g_pSeatManager->m_state.keyboardFocus.lock()),+ client_of(g_pSeatManager->m_state.pointerFocus.lock()));+}++// A withheld popup still maps and works, without the grab.+void guarded_add_or_start_grab(CXDGShellProtocol* protocol, SP<CXDGPopupResource> popup) {+ const auto xdg = popup ? popup->m_surface.lock() : nullptr;+ if (!user_holds(xdg ? xdg->m_surface.lock() : nullptr)) {+ g_grabs_withheld.fetch_add(1, std::memory_order_relaxed);+ return;+ }+ reinterpret_cast<AddOrStartGrab>(g_add_or_start_grab->m_original)(protocol, popup);+}++// A withheld drag is cancelled at once, as wl_data_source.cancelled tells the+// app, rather than left to follow the user's pointer.+void guarded_initiate_drag(CWLDataDeviceProtocol* protocol, WP<CWLDataSourceResource> source,+ SP<CWLSurfaceResource> icon, SP<CWLSurfaceResource> origin) {+ if (!user_holds(origin)) {+ g_drags_withheld.fetch_add(1, std::memory_order_relaxed);+ if (const auto live = source.lock())+ live->cancelled();+ return;+ }+ reinterpret_cast<InitiateDrag>(g_initiate_drag->m_original)(protocol, source, icon, origin);+}++CFunctionHook* hook_exact(void* plugin, const std::string& symbol, void* replacement) {+ const auto matches = HyprlandAPI::findFunctionsByName(plugin, symbol);+ if (matches.size() != 1)+ return nullptr;+ auto* hook = HyprlandAPI::createFunctionHook(plugin, matches.front().address, replacement);+ if (hook && !hook->hook()) {+ HyprlandAPI::removeFunctionHook(plugin, hook);+ return nullptr;+ }+ return hook;+}++} // namespace++bool install_user_seat_guard(void* plugin) {+ if (user_seat_guard_active())+ return true;+ g_plugin = plugin;+ g_add_or_start_grab =+ hook_exact(plugin, kAddOrStartGrab, reinterpret_cast<void*>(&guarded_add_or_start_grab));+ g_initiate_drag = hook_exact(plugin, kInitiateDrag, reinterpret_cast<void*>(&guarded_initiate_drag));+ // All or nothing: half a guard would still let one kind of request through.+ if (!user_seat_guard_active())+ remove_user_seat_guard();+ return user_seat_guard_active();+}++void remove_user_seat_guard() {+ for (auto** hook : {&g_add_or_start_grab, &g_initiate_drag}) {+ if (*hook)+ HyprlandAPI::removeFunctionHook(g_plugin, *hook);+ *hook = nullptr;+ }+ g_plugin = nullptr;+}++bool user_seat_guard_active() {+ return g_add_or_start_grab && g_initiate_drag;+}++std::uint64_t popup_grabs_withheld() {+ return g_grabs_withheld.load(std::memory_order_relaxed);+}++std::uint64_t drags_withheld() {+ return g_drags_withheld.load(std::memory_order_relaxed);+}++} // namespace cua::hyprlanddiff --git a/libs/cua-driver/hyprland-plugin/src/user_seat_guard.hpp b/libs/cua-driver/hyprland-plugin/src/user_seat_guard.hppnew file mode 100644index 0000000000..49df4577b4--- /dev/null+++ b/libs/cua-driver/hyprland-plugin/src/user_seat_guard.hpp@@ -0,0 +1,17 @@+#pragma once++#include <cstdint>++namespace cua::hyprland {++// Hooks Hyprland's xdg_popup grab and wl_data_device drag start so they take+// the user's seat only under user_seat_request_allowed (user_seat_policy.hpp).+// Returns whether both hooks are active.+bool install_user_seat_guard(void* plugin);+void remove_user_seat_guard();+bool user_seat_guard_active();+// Requests withheld since the guard was installed.+std::uint64_t popup_grabs_withheld();+std::uint64_t drags_withheld();++} // namespace cua::hyprlanddiff --git a/libs/cua-driver/hyprland-plugin/src/user_seat_policy.hpp b/libs/cua-driver/hyprland-plugin/src/user_seat_policy.hppnew file mode 100644index 0000000000..672ee35b97--- /dev/null+++ b/libs/cua-driver/hyprland-plugin/src/user_seat_policy.hpp@@ -0,0 +1,25 @@+#pragma once++struct wl_client;++namespace cua::hyprland {++// Two client requests take the user's whole seat in Hyprland 0.56 and 0.57,+// whatever seat and input serial they name:+//+// - xdg_popup.grab: the user's keyboard focus moves to the popup until it+// closes;+// - wl_data_device.start_drag: the drag follows the user's pointer and drops+// on the user's next button release, and ending it re-sends the user's+// keyboard focus.+//+// A popup or drag the person did not start (one an accessibility action or an+// agent seat started in a background app) then takes over their keyboard and+// pointer. One the person starts always comes from the client that holds+// their keyboard or pointer focus, so only that client may take the seat.+inline bool user_seat_request_allowed(const wl_client* requester, const wl_client* keyboard_focus,+ const wl_client* pointer_focus) {+ return requester && (requester == keyboard_focus || requester == pointer_focus);+}++} // namespace cua::hyprlanddiff --git a/libs/cua-driver/hyprland-plugin/tests/plugin_input_lifetime_test.cpp b/libs/cua-driver/hyprland-plugin/tests/plugin_input_lifetime_test.cppindex 135f07e4df..af4ab11fa6 100644--- a/libs/cua-driver/hyprland-plugin/tests/plugin_input_lifetime_test.cpp+++ b/libs/cua-driver/hyprland-plugin/tests/plugin_input_lifetime_test.cpp@@ -1,5 +1,6 @@#include "mock-hyprland/mock.hpp"#include "input_experiment.hpp"+#include "user_seat_guard.hpp"#include <array>#include <cstdlib>@@ -46,6 +47,12 @@ void InputExperiment::resume() {if (fail_resume) throw std::runtime_error("test transport failure");}std::string InputExperiment::status_json() const { return "{}"; }+unsigned seat_guard_installs = 0, seat_guard_removals = 0;+bool install_user_seat_guard(void*) { ++seat_guard_installs; return true; }+void remove_user_seat_guard() { ++seat_guard_removals; }+bool user_seat_guard_active() { return seat_guard_installs > seat_guard_removals; }+std::uint64_t popup_grabs_withheld() { return 0; }+std::uint64_t drags_withheld() { return 0; }}int main() {@@ -67,6 +74,8 @@ int main() {const auto initial_status = HyprlandAPI::registered_legacy_command->fn(FORMAT_JSON, {});check(created == 0 && initial_status.find(R"("configured":false)") != std::string::npos,"plugin starts with input disabled");+ check(initial_status.find(R"("user_seat_guard":true,"popup_grabs_withheld":0,"drags_withheld":0)") != std::string::npos,+ "user seat guard is installed with the plugin and reported");#ifdef CUA_HYPRLAND_INPUTcheck(advertises_keyboard_behavior(initial_status),"disabled v3 module advertises its compiled keyboard behavior before enable");@@ -121,6 +130,7 @@ int main() {"unload retires directly without first sending config-disable cancellation");check(Event::bus()->m_events.config.reloaded.active_count() == 0,"unload detaches configuration listener");+ check(!cua::hyprland::user_seat_guard_active(), "unload removes the user seat guard");check(rmdir(instance.c_str()) == 0 && rmdir(hypr.c_str()) == 0 &&rmdir(directory) == 0,"remove test-owned runtime");diff --git a/libs/cua-driver/hyprland-plugin/tests/user_seat_policy_test.cpp b/libs/cua-driver/hyprland-plugin/tests/user_seat_policy_test.cppnew file mode 100644index 0000000000..bcdfcf52bf--- /dev/null+++ b/libs/cua-driver/hyprland-plugin/tests/user_seat_policy_test.cpp@@ -0,0 +1,34 @@+#include "user_seat_policy.hpp"++#include <cstdio>+#include <cstdlib>++namespace {++void check(bool condition, const char* what) {+ if (!condition) {+ std::fprintf(stderr, "FAIL: %s\n", what);+ std::exit(1);+ }+}++} // namespace++int main() {+ using cua::hyprland::user_seat_request_allowed;+ // Opaque client identities: only their addresses are compared.+ char storage[3]{};+ const auto* app = reinterpret_cast<const wl_client*>(&storage[0]);+ const auto* user_app = reinterpret_cast<const wl_client*>(&storage[1]);+ const auto* panel = reinterpret_cast<const wl_client*>(&storage[2]);++ check(user_seat_request_allowed(app, app, nullptr), "keyboard-focused client opens a menu from a key");+ check(user_seat_request_allowed(app, user_app, app), "client under the user's pointer starts a drag");+ check(user_seat_request_allowed(panel, user_app, panel), "a panel the user clicked opens its menu");+ check(!user_seat_request_allowed(app, user_app, user_app), "background app: no grab or drag");+ check(!user_seat_request_allowed(app, user_app, nullptr), "background app, pointer over nothing: no grab");+ check(!user_seat_request_allowed(app, nullptr, nullptr), "no user focus at all: no grab");+ check(!user_seat_request_allowed(nullptr, nullptr, nullptr), "request without a surface: no grab");+ std::puts("user seat policy: ok");+ return 0;+}