Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.
Rust agentai-agentapplecomputer-usecomputer-use-agentcontainerizationcuadesktop-automationhacktoberfestlumemacosmanusoperatorswiftvirtualizationvirtualization-frameworkwindowswindows-sandbox
fix(cua-driver): hard-bound Windows UIA provider calls (#2117)
injaneityc9c660770a26
diff --git a/libs/cua-driver/rust/Skills/cua-driver/WINDOWS.md b/libs/cua-driver/rust/Skills/cua-driver/WINDOWS.mdindex 64f8157e3b..028c8ddfbf 100644--- a/libs/cua-driver/rust/Skills/cua-driver/WINDOWS.md+++ b/libs/cua-driver/rust/Skills/cua-driver/WINDOWS.md@@ -412,6 +412,11 @@ When a cua-driver call surprises you, diagnose cua-driver first:`capture_mode` param is **deprecated and ignored** — it's still acceptedso old callers don't error, but both the tree and the image come backregardless of what you pass.+- **`list_windows` returns Win32 windows but misses UWP / WebView2+ windows?** UIA desktop enumeration may be degraded because a provider+ is unresponsive. `list_windows` falls back to Win32-only output instead+ of hanging; run `cua-driver doctor` and retry after the provider+ recovers.- **`get_desktop_state` returns `desktop_scope_disabled`?** That'sintended: full-display capture is a **desktop-scope** operation, gatedby the caller-declared session policy. To verify a specific window use@@ -482,8 +487,9 @@ your prior tool calls earned.schtasks /Run /TN cua-driver-serve```3. **Run `cua-driver doctor`** — reports session ID, COM apartment- status, UIA reachability, install paths, version. If anything reads- `false` / `error`, fix that before tool-calling.+ status, UIA desktop-enumeration reachability, install paths,+ version. If anything reads `false` / `error`, fix that before+ tool-calling.4. **Permissions** — Windows has no TCC equivalent. cua-driver-rsneeds:- No admin elevation for normal use (UIA, PostMessage, UWP@@ -828,7 +834,7 @@ typed browser tools yet.- Daemon version and install paths- Current session ID (must be ≥1)- COM apartment status (STA / MTA / uninitialized)-- UIA reachability (can we connect to `CUIAutomation`?)+- UIA reachability (can we create `CUIAutomation` and enumerate desktop children?)- AppX broker reachability (for packaged-app activation)- PATH state (is `cua-driver` actually on PATH?)- Autostart Scheduled Task statusdiff --git a/libs/cua-driver/rust/crates/platform-windows/src/diagnostics.rs b/libs/cua-driver/rust/crates/platform-windows/src/diagnostics.rsindex 46c38504fa..a1ab5bce9d 100644--- a/libs/cua-driver/rust/crates/platform-windows/src/diagnostics.rs+++ b/libs/cua-driver/rust/crates/platform-windows/src/diagnostics.rs@@ -21,11 +21,6 @@#[cfg(target_os = "windows")]use windows::Win32::Foundation::HANDLE;#[cfg(target_os = "windows")]-use windows::Win32::System::Com::{- CoCreateInstance, CoInitializeEx, CoUninitialize, CLSCTX_INPROC_SERVER,- COINIT_APARTMENTTHREADED,-};-#[cfg(target_os = "windows")]use windows::Win32::System::RemoteDesktop::ProcessIdToSessionId;#[cfg(target_os = "windows")]use windows::Win32::System::StationsAndDesktops::{@@ -35,8 +30,6 @@ use windows::Win32::System::StationsAndDesktops::{#[cfg(target_os = "windows")]use windows::Win32::System::Threading::{GetCurrentProcessId, GetCurrentThreadId};#[cfg(target_os = "windows")]-use windows::Win32::UI::Accessibility::{CUIAutomation, IUIAutomation};-#[cfg(target_os = "windows")]use windows::Win32::UI::WindowsAndMessaging::GetForegroundWindow;/// Snapshot of the desktop context visible to the calling process.@@ -225,45 +218,10 @@ fn classify_interactive_desktop(state: &DesktopState) -> Result<bool, String> {Ok(state.has_foreground_window())}-/// Probe whether `CoCreateInstance(CUIAutomation)` succeeds — the same-/// COM call used by `get_window_state` / `list_windows` element-walks.-///-/// Initialises COM (apartment-threaded) for the duration of the probe-/// and uninitialises before returning so the rest of the doctor run is-/// unaffected.-///-/// **COM lifecycle invariant** — the IUIAutomation interface produced by-/// CoCreateInstance must be released BEFORE CoUninitialize tears down-/// the apartment, or `IUnknown::Release` will dereference freed COM-/// infrastructure and segfault (0xC0000005 ACCESS_VIOLATION). We flatten-/// the probe result into a plain `Result<(), String>` first so the-/// `IUIAutomation` is dropped at the end of the statement, then call-/// CoUninitialize on the next line.+/// Probe the same bounded desktop child enumeration used by window tools.#[cfg(target_os = "windows")]pub fn ui_automation_available() -> Result<(), String> {- unsafe {- // Failure here means COM is already initialised in this thread- // (likely apartment-threaded too); we only call CoUninitialize- // when our CoInitializeEx actually paired with a fresh init.- let init_result = CoInitializeEx(None, COINIT_APARTMENTTHREADED);- let need_uninit = init_result.is_ok();-- // Bind to `_` immediately so the IUIAutomation interface is- // dropped (and `Release()` runs while COM is still up) before- // we proceed to CoUninitialize. Holding the binding past- // CoUninitialize and letting Drop run at function return- // causes a use-after-free in the COM apartment.- let result =- CoCreateInstance::<_, IUIAutomation>(&CUIAutomation, None, CLSCTX_INPROC_SERVER)- .map(|_| ())- .map_err(|e| format!("{e}"));-- if need_uninit {- CoUninitialize();- }-- result- }+ crate::uia::windows_enum::probe_desktop_availability()}// ── Non-Windows stubs so the cua-driver crate can call into usdiff --git a/libs/cua-driver/rust/crates/platform-windows/src/tools/impl_.rs b/libs/cua-driver/rust/crates/platform-windows/src/tools/impl_.rsindex 5999e35322..a5fe4f6707 100644--- a/libs/cua-driver/rust/crates/platform-windows/src/tools/impl_.rs+++ b/libs/cua-driver/rust/crates/platform-windows/src/tools/impl_.rs@@ -4934,23 +4934,13 @@ impl Tool for HotkeyTool {// bound the call so a hung provider returns an error instead of// blocking the daemon indefinitely. 4 s matches the budget the// rest of this file uses for similar UIA scans.- let blocking = tokio::task::spawn_blocking(move || {+ let result = tokio::task::spawn_blocking(move || {crate::uia::windows_enum::try_invoke_accelerator_in_window(hwnd as isize,&accelerator_combo,)- });- let result =- match tokio::time::timeout(std::time::Duration::from_secs(4), blocking).await {- Ok(join_result) => join_result,- Err(_) => {- return ToolResult::error(format!(- "hotkey timed out after 4s while scanning UIA accelerators on pid \- {raw_pid} (hwnd {hwnd}). A UIA provider in this app is likely \- unresponsive."- ));- }- };+ })+ .await;return match result {Ok(Ok((true, _))) => ToolResult::text(format!("✅ Pressed {key_display_for_result} on pid {raw_pid} via UIA \diff --git a/libs/cua-driver/rust/crates/platform-windows/src/uia/windows_enum.rs b/libs/cua-driver/rust/crates/platform-windows/src/uia/windows_enum.rsindex 81a7d2e445..ac3de88d66 100644--- a/libs/cua-driver/rust/crates/platform-windows/src/uia/windows_enum.rs+++ b/libs/cua-driver/rust/crates/platform-windows/src/uia/windows_enum.rs@@ -18,7 +18,10 @@// with a fresh local binding and break the match. Mirrors overlay.rs:12.#![allow(non_upper_case_globals)]-use std::cell::RefCell;+use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};+use std::sync::{mpsc, Arc, OnceLock};+use std::thread;+use std::time::{Duration, Instant};use anyhow::{bail, Context};use windows::core::{Interface, BSTR};@@ -27,7 +30,7 @@ use windows::Win32::Graphics::Dwm::{DwmGetWindowAttribute, DWMWA_CLOAKED, DWMWA_EXTENDED_FRAME_BOUNDS,};use windows::Win32::System::Com::{- CoCreateInstance, CoInitializeEx, CLSCTX_INPROC_SERVER, COINIT_APARTMENTTHREADED,+ CoCreateInstance, CoInitializeEx, CoUninitialize, CLSCTX_INPROC_SERVER, COINIT_MULTITHREADED,};use windows::Win32::UI::Accessibility::{CUIAutomation, IUIAutomation, IUIAutomationElement, IUIAutomationInvokePattern,@@ -50,51 +53,202 @@ use crate::win32::windows::WindowInfo;/// same task, or a library on the same OS thread) picked a different/// apartment. Safe to ignore — COM is up either way.const RPC_E_CHANGED_MODE: i32 = -2147417850; // 0x80010106+/// Desktop child enumeration: one wedged provider must not stall list_windows.+const DESKTOP_CALL_TIMEOUT: Duration = Duration::from_secs(2);+/// Subtree scans: keep the pre-existing interactive operation budget.+const SUBTREE_OP_TIMEOUT: Duration = Duration::from_secs(4);+const UIA_RECOVERY_COOLDOWN: Duration = Duration::from_secs(30);-thread_local! {- /// Per-thread IUIAutomation instance. UIA / COM objects are- /// apartment-bound, so we deliberately do NOT share one across threads —- /// each OS thread that calls `enumerate_top_level_windows` initializes- /// COM as STA exactly once (the first time the cell is `None`) and caches- /// its own IUIAutomation. On failure the cell stays `None`, so the next- /// call retries from scratch instead of being stuck with a permanent- /// "init failed" sentinel.- static UIA_THREAD_LOCAL: RefCell<Option<IUIAutomation>> = const { RefCell::new(None) };+enum UiaDeadlineError {+ Timeout,+ Busy,+ Unavailable,}-/// Build or fetch the per-thread IUIAutomation instance.+/// Process-wide gate for blocking UIA calls.///-/// On the first successful call per OS thread this also initializes COM as-/// STA (UIA's in-process server requires an STA; `RPC_E_CHANGED_MODE` from a-/// pre-existing apartment is treated as "already up"). Any failure leaves-/// the thread-local empty so subsequent calls retry.-fn get_uia() -> Option<IUIAutomation> {- UIA_THREAD_LOCAL.with(|cell| {- if let Some(uia) = cell.borrow().as_ref() {- return Some(uia.clone());+/// Windows offers no safe way to cancel a COM provider call in another thread.+/// The gate therefore remains owned by a timed-out worker until that worker+/// actually returns. Retries fail fast instead of stranding more threads. Once+/// the provider recovers, the worker releases the gate after a short cooldown.+struct UiaSingleFlight {+ in_flight: AtomicBool,+ cooldown_until_ms: AtomicU64,+ cooldown_ms: u64,+}++impl UiaSingleFlight {+ const fn new(cooldown_ms: u64) -> Self {+ Self {+ in_flight: AtomicBool::new(false),+ cooldown_until_ms: AtomicU64::new(0),+ cooldown_ms,}- // First (or post-failure) call on this thread: init COM + build UIA.- unsafe {- let hr = CoInitializeEx(None, COINIT_APARTMENTTHREADED);- if hr.is_err() && hr.0 != RPC_E_CHANGED_MODE {- tracing::debug!(target: "uia_windows_enum", "CoInitializeEx returned {hr:?}");- }+ }++ fn run<T, F>(+ self: &Arc<Self>,+ stage: &'static str,+ timeout: Duration,+ fallback: &'static str,+ f: F,+ ) -> Result<T, UiaDeadlineError>+ where+ T: Send + 'static,+ F: FnOnce(Arc<AtomicBool>) -> T + Send + 'static,+ {+ let now = now_ms();+ if now < self.cooldown_until_ms.load(Ordering::Acquire)+ || self+ .in_flight+ .compare_exchange(false, true, Ordering::AcqRel, Ordering::Acquire)+ .is_err()+ {+ tracing::debug!(+ target: "uia_windows_enum",+ "UIA {stage} skipped while another provider call is in flight or cooling down"+ );+ return Err(UiaDeadlineError::Busy);}- let inst: IUIAutomation = match unsafe {- CoCreateInstance(&CUIAutomation, None, CLSCTX_INPROC_SERVER)- } {- Ok(a) => a,- Err(e) => {- tracing::warn!(target: "uia_windows_enum", "CoCreateInstance(CUIAutomation) failed: {e}");- return None;++ let (tx, rx) = mpsc::channel();+ let cancelled = Arc::new(AtomicBool::new(false));+ let worker_cancelled = Arc::clone(&cancelled);+ let worker_gate = Arc::clone(self);+ let spawn = thread::Builder::new()+ .name(format!("cua-uia-{stage}"))+ .spawn(move || {+ let _guard = InFlightGuard {+ gate: worker_gate,+ cancelled: Arc::clone(&worker_cancelled),+ };+ let result = f(worker_cancelled);+ let _ = tx.send(result);+ });++ if let Err(e) = spawn {+ self.in_flight.store(false, Ordering::Release);+ tracing::warn!(target: "uia_windows_enum", "failed to spawn UIA {stage} thread: {e}");+ return Err(UiaDeadlineError::Unavailable);+ }++ match rx.recv_timeout(timeout) {+ Ok(result) => Ok(result),+ Err(mpsc::RecvTimeoutError::Timeout) => {+ cancelled.store(true, Ordering::Release);+ tracing::warn!(+ target: "uia_windows_enum",+ "UIA {stage} exceeded {}ms; falling back to {fallback}; no other UIA worker will start until this provider call returns",+ timeout.as_millis()+ );+ Err(UiaDeadlineError::Timeout)}- };- let dup = inst.clone();- *cell.borrow_mut() = Some(inst);- Some(dup)+ Err(mpsc::RecvTimeoutError::Disconnected) => {+ tracing::warn!(target: "uia_windows_enum", "UIA {stage} thread exited without a result");+ Err(UiaDeadlineError::Unavailable)+ }+ }+ }+}++struct InFlightGuard {+ gate: Arc<UiaSingleFlight>,+ cancelled: Arc<AtomicBool>,+}++impl Drop for InFlightGuard {+ fn drop(&mut self) {+ if self.cancelled.load(Ordering::Acquire) {+ self.gate.cooldown_until_ms.store(+ now_ms().saturating_add(self.gate.cooldown_ms),+ Ordering::Release,+ );+ tracing::warn!(+ target: "uia_windows_enum",+ "timed-out UIA provider call returned; cooling down for {}ms before recovery probe",+ self.gate.cooldown_ms+ );+ }+ self.gate.in_flight.store(false, Ordering::Release);+ }+}++fn uia_single_flight() -> &'static Arc<UiaSingleFlight> {+ static GATE: OnceLock<Arc<UiaSingleFlight>> = OnceLock::new();+ GATE.get_or_init(|| {+ Arc::new(UiaSingleFlight::new(+ UIA_RECOVERY_COOLDOWN.as_millis() as u64+ ))})}+fn now_ms() -> u64 {+ // Monotonic: cooldown must not depend on wall-clock jumps.+ static START: OnceLock<Instant> = OnceLock::new();+ START.get_or_init(Instant::now).elapsed().as_millis() as u64+}++fn run_uia_with_deadline<T, F>(+ stage: &'static str,+ timeout: Duration,+ fallback: &'static str,+ f: F,+) -> Result<T, UiaDeadlineError>+where+ T: Send + 'static,+ F: FnOnce() -> T + Send + 'static,+{+ run_uia_with_deadline_cancelable(stage, timeout, fallback, |_| f())+}++fn run_uia_with_deadline_cancelable<T, F>(+ stage: &'static str,+ timeout: Duration,+ fallback: &'static str,+ f: F,+) -> Result<T, UiaDeadlineError>+where+ T: Send + 'static,+ F: FnOnce(Arc<AtomicBool>) -> T + Send + 'static,+{+ uia_single_flight().run(stage, timeout, fallback, f)+}++struct ComInit {+ needs_uninit: bool,+}++impl ComInit {+ fn new() -> Self {+ let hr = unsafe { CoInitializeEx(None, COINIT_MULTITHREADED) };+ if hr.is_err() && hr.0 != RPC_E_CHANGED_MODE {+ tracing::debug!(target: "uia_windows_enum", "CoInitializeEx returned {hr:?}");+ }+ Self {+ needs_uninit: hr.is_ok(),+ }+ }+}++impl Drop for ComInit {+ fn drop(&mut self) {+ if self.needs_uninit {+ unsafe { CoUninitialize() };+ }+ }+}++/// Build an IUIAutomation instance after COM is initialized.+fn get_uia() -> Option<IUIAutomation> {+ match unsafe { CoCreateInstance(&CUIAutomation, None, CLSCTX_INPROC_SERVER) } {+ Ok(a) => Some(a),+ Err(e) => {+ tracing::warn!(target: "uia_windows_enum", "CoCreateInstance(CUIAutomation) failed: {e}");+ None+ }+ }+}+/// Enumerate top-level windows visible to UI Automation.////// Returns one `WindowInfo` per non-offscreen child of the UIA desktop root@@ -105,6 +259,22 @@ fn get_uia() -> Option<IUIAutomation> {/// Returns an empty vec on any UIA failure — callers should treat UIA as a/// best-effort source and union with `EnumWindows`.pub fn enumerate_top_level_windows() -> Vec<WindowInfo> {+ match run_uia_with_deadline(+ "desktop enumeration",+ DESKTOP_CALL_TIMEOUT,+ "Win32-only window list",+ enumerate_top_level_windows_unbounded,+ ) {+ Ok(windows) => windows,+ Err(UiaDeadlineError::Timeout | UiaDeadlineError::Busy | UiaDeadlineError::Unavailable) => {+ Vec::new()+ }+ }+}++fn enumerate_top_level_windows_unbounded() -> Vec<WindowInfo> {+ // Keep this first so COM interfaces drop before CoUninitialize.+ let _com = ComInit::new();let uia = match get_uia() {Some(u) => u,None => return Vec::new(),@@ -148,6 +318,50 @@ pub fn enumerate_top_level_windows() -> Vec<WindowInfo> {}}+/// Exercise the real desktop enumeration call shape for `cua-driver doctor`+/// without allowing health checks to bypass the process-wide single-flight+/// bound used by window and interaction tools.+pub(crate) fn probe_desktop_availability() -> Result<(), String> {+ match run_uia_with_deadline(+ "health probe",+ DESKTOP_CALL_TIMEOUT,+ "Win32-only window tools",+ probe_desktop_availability_unbounded,+ ) {+ Ok(result) => result,+ Err(UiaDeadlineError::Timeout) => Err(format!(+ "UI Automation desktop enumeration exceeded {}ms; a UIA provider may be hung. Window tools will fall back to Win32-only enumeration until it recovers.",+ DESKTOP_CALL_TIMEOUT.as_millis()+ )),+ Err(UiaDeadlineError::Busy) => Err(+ "UI Automation is busy with an earlier timed-out provider call; window tools are temporarily using Win32-only enumeration."+ .to_owned(),+ ),+ Err(UiaDeadlineError::Unavailable) => {+ Err("UI Automation health worker is unavailable".to_owned())+ }+ }+}++fn probe_desktop_availability_unbounded() -> Result<(), String> {+ // Declared first so UIA interfaces are released before COM is uninitialized.+ let _com = ComInit::new();+ let automation =+ get_uia().ok_or_else(|| "CoCreateInstance(CUIAutomation) failed".to_owned())?;+ unsafe {+ let root = automation+ .GetRootElement()+ .map_err(|e| format!("GetRootElement: {e}"))?;+ let condition = automation+ .CreateTrueCondition()+ .map_err(|e| format!("CreateTrueCondition: {e}"))?;+ let _children = root+ .FindAll(TreeScope_Children, &condition)+ .map_err(|e| format!("FindAll(TreeScope_Children): {e}"))?;+ }+ Ok(())+}+/// Hit-test screen point `(sx, sy)` against the UIA subtree rooted at/// `hwnd` and fire `Invoke` on the deepest descendant whose bounding/// rect contains the point AND which supports `InvokePattern`. Returns@@ -218,6 +432,23 @@ fn is_coord_independent_action(elem: &IUIAutomationElement) -> bool {}pub fn try_invoke_in_window_at_point(hwnd: isize, sx: i32, sy: i32) -> bool {+ run_uia_with_deadline_cancelable(+ "window hit-test invoke",+ SUBTREE_OP_TIMEOUT,+ "PostMessage click delivery",+ move |cancelled| try_invoke_in_window_at_point_unbounded(hwnd, sx, sy, &cancelled),+ )+ .unwrap_or(false)+}++fn try_invoke_in_window_at_point_unbounded(+ hwnd: isize,+ sx: i32,+ sy: i32,+ cancelled: &AtomicBool,+) -> bool {+ // Keep this first so COM interfaces drop before CoUninitialize.+ let _com = ComInit::new();if hwnd == 0 {return false;}@@ -308,6 +539,10 @@ pub fn try_invoke_in_window_at_point(hwnd: isize, sx: i32, sy: i32) -> bool {return false;}};+ if cancelled.load(Ordering::Acquire) {+ tracing::debug!(target: "click", "UIA hit-test invoke cancelled before activation");+ return false;+ }// Pattern preference for menu items: when both Invoke AND// ExpandCollapse are advertised, the element is almost always a// top-level MenuItem whose intended click behaviour is "open the@@ -318,10 +553,16 @@ pub fn try_invoke_in_window_at_point(hwnd: isize, sx: i32, sy: i32) -> bool {.GetCurrentPattern(windows::Win32::UI::Accessibility::UIA_ExpandCollapsePatternId).is_ok();let winner_has_invoke = winner.GetCurrentPattern(UIA_InvokePatternId).is_ok();+ if cancelled.load(Ordering::Acquire) {+ return false;+ }// UWP foreground-steal bypass: gate the entire activation block on// `is_xaml_host_hwnd(hwnd)`. For non-XAML hosts the closure is a// straight passthrough.crate::uia::fg_bypass::run_with_uwp_bypass(hwnd, || {+ if cancelled.load(Ordering::Acquire) {+ return false;+ }if winner_has_expand && winner_has_invoke {// Try Expand first, fall back to Invoke if Expand fails.if let Ok(pat) = winner.GetCurrentPattern(@@ -330,6 +571,9 @@ pub fn try_invoke_in_window_at_point(hwnd: isize, sx: i32, sy: i32) -> bool {if let Ok(ec) = pat.cast::<windows::Win32::UI::Accessibility::IUIAutomationExpandCollapsePattern>(){+ if cancelled.load(Ordering::Acquire) {+ return false;+ }if ec.Expand().is_ok() {return true;}@@ -343,6 +587,9 @@ pub fn try_invoke_in_window_at_point(hwnd: isize, sx: i32, sy: i32) -> bool {if let Ok(ec) = pat.cast::<windows::Win32::UI::Accessibility::IUIAutomationExpandCollapsePattern>(){+ if cancelled.load(Ordering::Acquire) {+ return false;+ }return ec.Expand().is_ok();}}@@ -356,6 +603,9 @@ pub fn try_invoke_in_window_at_point(hwnd: isize, sx: i32, sy: i32) -> bool {Ok(i) => i,Err(_) => return false,};+ if cancelled.load(Ordering::Acquire) {+ return false;+ }match inv.Invoke() {Ok(()) => true,Err(e) => {@@ -375,6 +625,28 @@ pub fn try_invoke_in_window_at_point(hwnd: isize, sx: i32, sy: i32) -> bool {/// This helper keeps that routing narrow by requiring an advertised/// AcceleratorKey match before invoking anything.pub fn try_invoke_accelerator_in_window(hwnd: isize, combo: &str) -> anyhow::Result<(bool, usize)> {+ let combo = combo.to_owned();+ run_uia_with_deadline_cancelable(+ "accelerator invoke",+ SUBTREE_OP_TIMEOUT,+ "keyboard message delivery",+ move |cancelled| try_invoke_accelerator_in_window_unbounded(hwnd, &combo, &cancelled),+ )+ .unwrap_or_else(|_| {+ Err(anyhow::anyhow!(+ "UIA accelerator scan exceeded {}ms; a UIA provider in the target app is likely unresponsive",+ SUBTREE_OP_TIMEOUT.as_millis()+ ))+ })+}++fn try_invoke_accelerator_in_window_unbounded(+ hwnd: isize,+ combo: &str,+ cancelled: &AtomicBool,+) -> anyhow::Result<(bool, usize)> {+ // Keep this first so COM interfaces drop before CoUninitialize.+ let _com = ComInit::new();if hwnd == 0 {bail!("invalid target hwnd 0");}@@ -435,6 +707,13 @@ pub fn try_invoke_accelerator_in_window(hwnd: isize, combo: &str) -> anyhow::Rescontinue;}+ if cancelled.load(Ordering::Acquire) {+ tracing::debug!(+ target: "uia_windows_enum",+ "UIA accelerator invoke cancelled before activation"+ );+ return Ok((false, count as usize));+ }// Pattern fallback chain. Different XAML controls expose// different "activation" patterns: a Save button uses Invoke, a// Bold toggle uses Toggle, a list item uses SelectionItem. Try@@ -443,7 +722,7 @@ pub fn try_invoke_accelerator_in_window(hwnd: isize, combo: &str) -> anyhow::Res// as a TogglePattern button — calling .Invoke on it returns the// misleading "operation completed successfully (0x00000000)"// error because Invoke isn't supported on the element.- match try_invoke_via_patterns(&elem, hwnd) {+ match try_invoke_via_patterns(&elem, hwnd, cancelled) {Ok(true) => return Ok((true, count as usize)),Ok(false) => {matched_failure = Some(format!(@@ -518,11 +797,7 @@ fn canonical_accelerator_token(value: &str) -> String {if value == " " {return "space".to_owned();}- let compact = value- .trim()- .to_ascii_lowercase()- .replace(' ', "")- .replace('-', "");+ let compact = value.trim().to_ascii_lowercase().replace([' ', '-'], "");match compact.as_str() {"control" => "ctrl".to_owned(),"windows" | "window" | "meta" | "cmd" | "command" => "win".to_owned(),@@ -562,11 +837,18 @@ fn accelerator_modifier_rank(value: &str) -> Option<usize> {unsafe fn try_invoke_via_patterns(elem: &IUIAutomationElement,host_hwnd: isize,+ cancelled: &AtomicBool,) -> anyhow::Result<bool> {// Invoke first — that's what most accelerator-targeted controls advertise.if let Ok(pattern) = elem.GetCurrentPattern(UIA_InvokePatternId) {if let Ok(inv) = pattern.cast::<IUIAutomationInvokePattern>() {+ if cancelled.load(Ordering::Acquire) {+ return Ok(false);+ }return crate::uia::fg_bypass::run_with_uwp_bypass(host_hwnd, || {+ if cancelled.load(Ordering::Acquire) {+ return Ok(false);+ }inv.Invoke().map(|()| true).map_err(|e| anyhow::anyhow!("InvokePattern.Invoke: {e}"))@@ -576,7 +858,13 @@ unsafe fn try_invoke_via_patterns(// Toggle next — Bold/Italic/Underline-style toolbar buttons sit here.if let Ok(pattern) = elem.GetCurrentPattern(UIA_TogglePatternId) {if let Ok(tog) = pattern.cast::<IUIAutomationTogglePattern>() {+ if cancelled.load(Ordering::Acquire) {+ return Ok(false);+ }return crate::uia::fg_bypass::run_with_uwp_bypass(host_hwnd, || {+ if cancelled.load(Ordering::Acquire) {+ return Ok(false);+ }tog.Toggle().map(|()| true).map_err(|e| anyhow::anyhow!("TogglePattern.Toggle: {e}"))@@ -794,8 +1082,9 @@ fn window_bounds(hwnd: HWND, prefer_win32: bool) -> Option<(i32, i32, i32, i32)>}#[cfg(test)]-mod bounds_fallback_tests {- use super::select_window_rect;+mod tests {+ use super::*;+ use std::sync::atomic::AtomicUsize;use windows::Win32::Foundation::RECT;fn rect() -> RECT {@@ -831,4 +1120,82 @@ mod bounds_fallback_tests {);assert!(select_window_rect(true, Some(win32), Some(empty)).is_none());}++ #[test]+ fn wedged_provider_has_bounded_worker_growth_and_recovers() {+ let gate = Arc::new(UiaSingleFlight::new(40));+ let (started_tx, started_rx) = mpsc::channel();+ let (release_tx, release_rx) = mpsc::channel();+ let starts = Arc::new(AtomicUsize::new(0));+ let active = Arc::new(AtomicUsize::new(0));+ let max_active = Arc::new(AtomicUsize::new(0));+ let side_effects = Arc::new(AtomicUsize::new(0));++ let first_start = Instant::now();+ let result = gate.run("test wedge", Duration::from_millis(20), "test", {+ let starts = Arc::clone(&starts);+ let active = Arc::clone(&active);+ let max_active = Arc::clone(&max_active);+ let side_effects = Arc::clone(&side_effects);+ move |cancelled| {+ starts.fetch_add(1, Ordering::AcqRel);+ let current = active.fetch_add(1, Ordering::AcqRel) + 1;+ max_active.fetch_max(current, Ordering::AcqRel);+ started_tx.send(()).unwrap();+ release_rx.recv().unwrap();+ if !cancelled.load(Ordering::Acquire) {+ side_effects.fetch_add(1, Ordering::AcqRel);+ }+ active.fetch_sub(1, Ordering::AcqRel);+ }+ });+ assert!(matches!(result, Err(UiaDeadlineError::Timeout)));+ assert!(first_start.elapsed() < Duration::from_millis(500));+ started_rx.recv_timeout(Duration::from_secs(1)).unwrap();++ let retries_start = Instant::now();+ for _ in 0..100 {+ let starts = Arc::clone(&starts);+ let retry = gate.run("test retry", Duration::from_secs(1), "test", move |_| {+ starts.fetch_add(1, Ordering::AcqRel);+ });+ assert!(matches!(retry, Err(UiaDeadlineError::Busy)));+ }+ assert!(retries_start.elapsed() < Duration::from_millis(500));+ assert_eq!(starts.load(Ordering::Acquire), 1);+ assert_eq!(max_active.load(Ordering::Acquire), 1);++ release_tx.send(()).unwrap();+ let recovery_deadline = Instant::now() + Duration::from_secs(1);+ while gate.in_flight.load(Ordering::Acquire) && Instant::now() < recovery_deadline {+ thread::yield_now();+ }+ assert!(!gate.in_flight.load(Ordering::Acquire));+ assert_eq!(side_effects.load(Ordering::Acquire), 0);++ let during_cooldown = gate.run("test cooldown", Duration::from_secs(1), "test", |_| 1);+ assert!(matches!(during_cooldown, Err(UiaDeadlineError::Busy)));+ let cooldown_deadline = Instant::now() + Duration::from_secs(1);+ while now_ms() < gate.cooldown_until_ms.load(Ordering::Acquire)+ && Instant::now() < cooldown_deadline+ {+ thread::yield_now();+ }++ let recovered = gate.run("test recovery", Duration::from_secs(1), "test", {+ let starts = Arc::clone(&starts);+ let active = Arc::clone(&active);+ let max_active = Arc::clone(&max_active);+ move |_| {+ starts.fetch_add(1, Ordering::AcqRel);+ let current = active.fetch_add(1, Ordering::AcqRel) + 1;+ max_active.fetch_max(current, Ordering::AcqRel);+ active.fetch_sub(1, Ordering::AcqRel);+ 42+ }+ });+ assert!(matches!(recovered, Ok(42)));+ assert_eq!(starts.load(Ordering::Acquire), 2);+ assert_eq!(max_active.load(Ordering::Acquire), 1);+ }}