Skip to content

armgit / cua

Public Imported from GitHub
0

Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.

fix(cua-driver-rs)(windows): no-foreground audit — launch_app focus restore + bookmark-exec bail (#1668)

Francesco Bonacci7436ba89becb
diff --git a/libs/cua-driver-rs/PARITY.md b/libs/cua-driver-rs/PARITY.md
index 9a97b987e1..089684ff09 100644
--- a/libs/cua-driver-rs/PARITY.md
+++ b/libs/cua-driver-rs/PARITY.md
@@ -571,10 +571,15 @@ Windows's `click` takes `{button: enum}` instead. Rationale:
- windows: VERIFIED for Win32 path; UWP path requires interactive session
(returns descriptive error in Session 0 — never hangs). Win32 launches
use `ShellExecuteExW` + `SW_SHOWNOACTIVATE` (no focus steal, matches
- macOS oapp). UWP launches use `IApplicationActivationManager` +
- best-effort `GetForegroundWindow` snapshot/restore (best-effort
- because `SetForegroundWindow` is subject to Windows' foreground-lock
- restrictions — visual confirmation in Session 1+ recommended).
+ macOS oapp) AND now schedule a best-effort polling
+ `GetForegroundWindow`/`SetForegroundWindow` restore (≤3s, 100ms cadence)
+ that flips the user's prior foreground back if the spawned app
+ activates. URLs-only invocations skip the restore (the user explicitly
+ asked the default browser to come up with that page). UWP launches use
+ `IApplicationActivationManager` + best-effort `GetForegroundWindow`
+ snapshot/restore (best-effort because `SetForegroundWindow` is subject
+ to Windows' foreground-lock restrictions — visual confirmation in
+ Session 1+ recommended).
- macOS: VERIFIED (full focus-steal contract — see [Focus-steal prevention](#focus-steal-prevention))
- linux: OPEN
- Tests:
@@ -627,6 +632,25 @@ Windows's `click` takes `{button: enum}` instead. Rationale:
Windows-equivalent of Swift's background-launch invariant.
9. **Description** — multi-paragraph port from Swift with explicit
Windows-specific notes (path takes precedence; bundle_id alias).
+10. **Polling foreground-restore for the Win32 path** — mirrors the macOS
+ `FocusRestoreGuard`. `LaunchAppTool::invoke` captures
+ `GetForegroundWindow()` before the launch dispatch and, for the
+ `ShellExecuteExW` branch, spawns a tokio task that polls every 100ms
+ (up to 3s) for "the spawned app actually grabbed foreground" and then
+ flips the prior HWND back via `SetForegroundWindow`. The UWP/AUMID
+ branch keeps its existing synchronous restore in
+ `launch_uwp::restore_foreground_best_effort` — the polling task is
+ gated on the Win32 branch to avoid double-restoring.
+
+ URLs-only invocations (`{urls: [...]}` with no app-identifying field)
+ skip the restore: the user asked for that page to come up in the
+ default browser, restoring would hide it. The decision is a pure
+ function (`should_restore_foreground_after_launch`) with unit
+ coverage in `launch_focus_restore_decision_tests`.
+
+ `SetForegroundWindow` from non-UIAccess processes is restricted by
+ Windows' foreground-lock; failures are logged at `tracing::trace!`
+ and not surfaced — the launch itself already succeeded.
### Intentional Rust-only fields accepted (no-op)
diff --git a/libs/cua-driver-rs/Skills/cua-driver-rs/WINDOWS.md b/libs/cua-driver-rs/Skills/cua-driver-rs/WINDOWS.md
index 162b522b21..bfa9cf9be3 100644
--- a/libs/cua-driver-rs/Skills/cua-driver-rs/WINDOWS.md
+++ b/libs/cua-driver-rs/Skills/cua-driver-rs/WINDOWS.md
@@ -145,6 +145,18 @@ Windows by the activation-deny dance: the launcher pumps a
`SetForegroundWindow` call per [Windows focus-stealing prevention
rules](https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-setforegroundwindow).
+**`launch_app` foreground-restore behaviour (Windows).** In addition to
+`SW_SHOWNOACTIVATE`, `launch_app` captures `GetForegroundWindow()` before
+the spawn and schedules a post-spawn polling restore (every 100 ms for up
+to 3 s) that flips the foreground back to the prior window once the
+spawned app has actually activated. This mirrors the macOS
+`FocusRestoreGuard`. For `urls`-only invocations (open these links in the
+default browser, no app-identifying field) the restore is **skipped**,
+because the user explicitly asked for the page to come up. The restore is
+best-effort: `SetForegroundWindow` from non-UIAccess processes is subject
+to Windows' foreground-lock and may silently no-op — failures are logged
+at `tracing::trace!` and not surfaced as errors.
+
## Defaults — always prefer cua-driver over shell shims
**Default transport is the `cua-driver` CLI** — `Bash` shelling out
@@ -640,14 +652,31 @@ browser instance identified by `(pid, window_id)`:
Automatic creation (drive the omnibox to `edge://favorites`,
click "Add favorite", fill the dialog) is not yet wired up —
create it manually.
- - The Favorites bar must be reachable. If hidden, the driver
- sends Ctrl+Shift+B to the Edge HWND and re-walks; if that
- still doesn't summon it (custom keyboard remap, group
- policy), bookmark exec fails and we fall through to CDP.
+ - **The Favorites bar must already be visible.** This is a
+ one-time setup: press `Ctrl+Shift+B` once inside the browser
+ and the setting persists across sessions. cua-driver does
+ **not** synthesize Ctrl+Shift+B for you any more, because the
+ synthesizer routed via `SetForegroundWindow(target) →
+ SendInput → SetForegroundWindow(prev_fg)` produces a visible
+ foreground flicker and the restore can fail silently under
+ UIPI / UIAccess constraints — both violations of the
+ no-foreground contract. If the bar is hidden when
+ `execute_javascript` is called, the bookmark path bails with a
+ clear error and falls through to the CDP path (if configured).
- The user's expression should be a single statement or block;
`return` inside the IIFE is honored. Bookmarks strip line
breaks, so multi-line expressions are joined with spaces.
+ **Known limitation — transient browser activation on Invoke.**
+ When the bookmark is invoked via UIA `InvokePattern::Invoke`,
+ Chromium briefly raises the browser window because clicking a
+ bookmark is a user-initiated navigation in Chromium's input
+ model. This is a Chromium-side activation we cannot suppress
+ from the UIA caller. We document it rather than mask it; the
+ rest of the page-tool actions (`get_text`, `query_dom`) remain
+ no-foreground. If this matters for your flow, use the CDP path
+ instead — `Runtime.evaluate` does not trigger window activation.
+
2. **CDP fallback** — `Runtime.evaluate` via raw WebSocket against
`--remote-debugging-port=N`. Requires the browser launched with
that flag and `CUA_DRIVER_CDP_PORT=N` exported before the daemon
diff --git a/libs/cua-driver-rs/crates/platform-windows/src/tools/impl_.rs b/libs/cua-driver-rs/crates/platform-windows/src/tools/impl_.rs
index dae21c5d77..36c382e125 100644
--- a/libs/cua-driver-rs/crates/platform-windows/src/tools/impl_.rs
+++ b/libs/cua-driver-rs/crates/platform-windows/src/tools/impl_.rs
@@ -740,6 +740,203 @@ fn inject_chromium_anti_throttling_flags(extra_args: &mut Vec<String>) {
// ── launch_app ───────────────────────────────────────────────────────────────
+/// Shape of the resolved launch-target params, used to decide whether the
+/// best-effort foreground-restore guard should fire.
+///
+/// The decision is intentionally a pure function of the params (not of any
+/// HWND state) so it can be unit-tested. See
+/// [`should_restore_foreground_after_launch`].
+#[derive(Clone, Copy, Debug)]
+struct LaunchTargetShape {
+ has_aumid: bool,
+ has_bundle_id: bool,
+ has_name: bool,
+ has_path: bool,
+ has_launch_path: bool,
+ has_urls: bool,
+}
+
+/// True iff the launch is "open an app in the background" (so we want to
+/// restore the user's foreground after the spawn activates the new window)
+/// rather than "open a URL in the default browser" (where the user
+/// explicitly asked for that page to come up).
+///
+/// Rule, matching the audit spec:
+/// - URLs-only call (no app-identifying field set) → SKIP restore. The
+/// user asked for `https://…` to be openable in their default browser;
+/// a browser instance freshly launched for that URL is the legitimate
+/// foreground.
+/// - Any app-identifying field present (`aumid` / `bundle_id` / `name` /
+/// `path` / `launch_path`) → RESTORE. The intent is hidden-launch +
+/// background drive; the no-foreground contract applies.
+fn should_restore_foreground_after_launch(shape: LaunchTargetShape) -> bool {
+ let has_app_identifier = shape.has_aumid
+ || shape.has_bundle_id
+ || shape.has_name
+ || shape.has_path
+ || shape.has_launch_path;
+ // URLs-only carve-out: when the caller passed urls AND no
+ // app-identifying field, skip the restore.
+ if shape.has_urls && !has_app_identifier {
+ return false;
+ }
+ has_app_identifier
+}
+
+/// Async polling restore of the prior foreground window, used by
+/// `LaunchAppTool` to mirror the macOS `FocusRestoreGuard` for the legacy
+/// `ShellExecuteExW` launch path. The UWP/AUMID branch has its own
+/// synchronous restoration in `launch_uwp::restore_foreground_best_effort`
+/// and is NOT routed through here.
+///
+/// Approach:
+/// 1. Try a short `WaitForInputIdle` on the spawned pid (if the handle
+/// is openable) — this gets us past the worst of the new app's
+/// window-creation window before we start sampling foreground state.
+/// 2. Poll every 100ms for up to ~3s; when `GetForegroundWindow()` is no
+/// longer the prior foreground (i.e. the spawned app actually grabbed
+/// focus), call `SetForegroundWindow(prior)`. We avoid restoring
+/// eagerly so we don't race the new app's own activation and lose.
+///
+/// `SetForegroundWindow` from non-UIAccess processes is restricted by
+/// Windows' foreground-lock; the call may silently fail. We log the
+/// failure at trace level and proceed — no error is surfaced, the launch
+/// itself already succeeded.
+///
+/// Note: `prior_foreground_addr` is the raw HWND value as `usize` (rather
+/// than `HWND` directly) because `HWND` wraps `*mut c_void` which is
+/// `!Send` and would prevent this future from being scheduled on the
+/// multi-threaded tokio runtime.
+async fn restore_foreground_polling_best_effort(
+ prior_foreground_addr: usize,
+ spawned_pid: u32,
+) {
+ use windows::Win32::Foundation::{CloseHandle, HWND};
+ use windows::Win32::System::Threading::{
+ OpenProcess, WaitForInputIdle, PROCESS_QUERY_LIMITED_INFORMATION, PROCESS_SYNCHRONIZE,
+ };
+ use windows::Win32::UI::WindowsAndMessaging::{
+ GetForegroundWindow, GetWindowThreadProcessId, IsWindow, SetForegroundWindow,
+ };
+
+ if prior_foreground_addr == 0 {
+ tracing::trace!(
+ target: "launch_app.focus_restore",
+ "no prior foreground HWND to restore — skipping"
+ );
+ return;
+ }
+
+ // Without a captured spawned pid we have no ownership signal — any
+ // foreground change during the 3 s window might be the user
+ // legitimately Alt-Tabbing, and we shouldn't yank focus back. Skip
+ // the entire guard. This is the launcher-stub fallback case
+ // (rare for fresh launches).
+ if spawned_pid == 0 {
+ tracing::trace!(
+ target: "launch_app.focus_restore",
+ "no spawned pid captured — skipping focus-restore guard (no ownership signal)"
+ );
+ return;
+ }
+
+ // Phase 1: best-effort wait for the spawned process to finish initial
+ // window creation. WaitForInputIdle returns when the process pumps
+ // its first message — that's typically when the main window has
+ // registered. Capped at 1s; the polling loop below covers the rest.
+ if spawned_pid != 0 {
+ // PROCESS_SYNCHRONIZE is required by WaitForInputIdle;
+ // PROCESS_QUERY_LIMITED_INFORMATION is a no-op extra that some
+ // antivirus filters tolerate better than QUERY_INFORMATION.
+ // OpenProcess result is also `!Send` (HANDLE), so do the open +
+ // wait + close in one blocking task.
+ let _ = tokio::task::spawn_blocking(move || unsafe {
+ if let Ok(handle) = OpenProcess(
+ PROCESS_SYNCHRONIZE | PROCESS_QUERY_LIMITED_INFORMATION,
+ false,
+ spawned_pid,
+ ) {
+ if !handle.is_invalid() {
+ let _ = WaitForInputIdle(handle, 1000);
+ }
+ // Always close the handle, even if `is_invalid()` (cheap;
+ // matches the OpenProcess pairing convention used elsewhere
+ // in this crate).
+ let _ = CloseHandle(handle);
+ }
+ // Silently no-op if OpenProcess failed (target may have already
+ // exited; rare for a fresh launch but not impossible for a
+ // launcher-stub).
+ })
+ .await;
+ }
+
+ // Phase 2: poll for "spawned app became foreground", then flip back.
+ // Reconstruct HWND only inside synchronous probes so the future state
+ // never holds an `!Send` value across an await point.
+ let deadline = std::time::Instant::now() + std::time::Duration::from_secs(3);
+ loop {
+ if std::time::Instant::now() >= deadline {
+ tracing::trace!(
+ target: "launch_app.focus_restore",
+ "spawned pid {} did not steal foreground within deadline — no restore needed",
+ spawned_pid
+ );
+ return;
+ }
+ // Only restore when the current foreground window is owned by
+ // the spawned pid — otherwise the user may have legitimately
+ // Alt-Tabbed during the polling window and we'd be yanking
+ // focus away from them. Resolves fg_now's owning PID via
+ // GetWindowThreadProcessId and compares against spawned_pid.
+ let activated_then_restored: Option<bool> = {
+ let prior = HWND(prior_foreground_addr as *mut _);
+ let fg_now = unsafe { GetForegroundWindow() };
+ if fg_now == prior {
+ None
+ } else {
+ let mut fg_pid: u32 = 0;
+ let _ = unsafe { GetWindowThreadProcessId(fg_now, Some(&mut fg_pid)) };
+ if fg_pid != spawned_pid {
+ // Foreground changed but to something we didn't
+ // spawn (e.g. user Alt-Tabbed). Don't touch it.
+ None
+ } else if !unsafe { IsWindow(prior) }.as_bool() {
+ tracing::trace!(
+ target: "launch_app.focus_restore",
+ "prior foreground HWND no longer valid — skipping restore (spawned pid {} owns fg)",
+ spawned_pid
+ );
+ Some(true)
+ } else {
+ let ok = unsafe { SetForegroundWindow(prior) }.as_bool();
+ if !ok {
+ tracing::trace!(
+ target: "launch_app.focus_restore",
+ "SetForegroundWindow returned FALSE — likely foreground-lock denial; \
+ prior HWND 0x{:x}, spawned pid {}",
+ prior_foreground_addr,
+ spawned_pid
+ );
+ } else {
+ tracing::debug!(
+ target: "launch_app.focus_restore",
+ "restored prior foreground HWND 0x{:x} after spawned pid {} activated",
+ prior_foreground_addr,
+ spawned_pid
+ );
+ }
+ Some(ok)
+ }
+ }
+ };
+ if activated_then_restored.is_some() {
+ return;
+ }
+ tokio::time::sleep(std::time::Duration::from_millis(100)).await;
+ }
+}
+
pub struct LaunchAppTool;
static LAUNCH_DEF: std::sync::OnceLock<ToolDef> = std::sync::OnceLock::new();
@@ -808,6 +1005,37 @@ impl Tool for LaunchAppTool {
let urls: Vec<String> = args.get("urls").and_then(|v| v.as_array())
.map(|a| a.iter().filter_map(|v| v.as_str().map(str::to_owned)).collect())
.unwrap_or_default();
+
+ // Capture the foreground window BEFORE any launch path runs so the
+ // post-spawn polling restore (below) has a target HWND to flip back
+ // to. Mirrors the macOS `FocusRestoreGuard` behavior. Best-effort —
+ // see `restore_foreground_polling_best_effort` for the actual
+ // restoration semantics.
+ //
+ // The UWP/AUMID path already restores foreground synchronously
+ // (see `launch_uwp::restore_foreground_best_effort`), so this only
+ // gates the legacy ShellExecuteExW path. We capture early so the
+ // value is consistent regardless of which sub-path runs.
+ //
+ // Stored as `usize` (not `HWND`) so this `invoke` future stays
+ // `Send` — `HWND` wraps `*mut c_void` and would taint the future
+ // for the multi-threaded tokio runtime if held across `.await`.
+ let foreground_before_addr: usize = unsafe {
+ windows::Win32::UI::WindowsAndMessaging::GetForegroundWindow().0 as usize
+ };
+
+ // Pure-function decision on whether to fire the post-launch
+ // restore. Shape mirrors the resolved params — see
+ // `should_restore_foreground_after_launch` for the rule.
+ let launch_shape = LaunchTargetShape {
+ has_aumid: aumid_opt.is_some(),
+ has_bundle_id: bundle_id_opt.is_some(),
+ has_name: name_opt.is_some(),
+ has_path: path_opt.is_some(),
+ has_launch_path: launch_path_opt.is_some(),
+ has_urls: !urls.is_empty(),
+ };
+ let restore_foreground = should_restore_foreground_after_launch(launch_shape);
let mut extra_args: Vec<String> = args.get("additional_arguments").and_then(|v| v.as_array())
.map(|a| a.iter().filter_map(|v| v.as_str().map(str::to_owned)).collect())
.unwrap_or_default();
@@ -1017,6 +1245,30 @@ impl Tool for LaunchAppTool {
}
};
+ // Best-effort foreground-restore for the legacy ShellExecuteExW
+ // path. The UWP/AUMID branch already restores foreground
+ // synchronously inside `launch_uwp::launch_uwp`, so we only spawn
+ // the polling task when we went through the ShellExecuteExW
+ // branch. URLs-only invocations are excluded via `restore_foreground`
+ // (see `should_restore_foreground_after_launch`).
+ //
+ // `HWND` wraps a raw `*mut c_void` which is `!Send`, so we marshal
+ // the foreground handle across the spawn boundary as a `usize` and
+ // reconstruct the `HWND` only inside the synchronous probes (see
+ // `restore_foreground_polling_best_effort`). Mirrors how
+ // `launch_uwp.rs` carries its restore target.
+ if aumid_for_uwp.is_none() && restore_foreground {
+ let spawned_pid_for_restore = pid;
+ let target_foreground_addr = foreground_before_addr;
+ tokio::spawn(async move {
+ restore_foreground_polling_best_effort(
+ target_foreground_addr,
+ spawned_pid_for_restore,
+ )
+ .await;
+ });
+ }
+
// When the packaged path was taken we still need to fan out any
// extra URLs to the default browser (ShellExecuteEx — no pid
// capture, mirroring Swift's NSWorkspace secondary-URL flow).
@@ -4019,6 +4271,95 @@ pub fn build_registry() -> ToolRegistry {
r
}
+#[cfg(test)]
+mod launch_focus_restore_decision_tests {
+ use super::{should_restore_foreground_after_launch, LaunchTargetShape};
+
+ fn empty() -> LaunchTargetShape {
+ LaunchTargetShape {
+ has_aumid: false,
+ has_bundle_id: false,
+ has_name: false,
+ has_path: false,
+ has_launch_path: false,
+ has_urls: false,
+ }
+ }
+
+ #[test]
+ fn urls_only_skips_restore() {
+ // `launch_app({urls: ["https://example.com"]})` — user explicitly
+ // asked for navigation in the default browser. The freshly-launched
+ // browser instance IS the legitimate foreground; restoring would
+ // hide the page the user wanted to see.
+ let shape = LaunchTargetShape { has_urls: true, ..empty() };
+ assert!(!should_restore_foreground_after_launch(shape));
+ }
+
+ #[test]
+ fn name_only_restores() {
+ let shape = LaunchTargetShape { has_name: true, ..empty() };
+ assert!(should_restore_foreground_after_launch(shape));
+ }
+
+ #[test]
+ fn path_only_restores() {
+ let shape = LaunchTargetShape { has_path: true, ..empty() };
+ assert!(should_restore_foreground_after_launch(shape));
+ }
+
+ #[test]
+ fn aumid_only_restores() {
+ // Note: AUMID path has its own synchronous restore in launch_uwp.rs,
+ // but the decision function still says "yes, this is an
+ // app-identifying launch" — the caller (`LaunchAppTool::invoke`)
+ // gates the polling spawn on `aumid_for_uwp.is_none()` so we don't
+ // double-restore.
+ let shape = LaunchTargetShape { has_aumid: true, ..empty() };
+ assert!(should_restore_foreground_after_launch(shape));
+ }
+
+ #[test]
+ fn bundle_id_only_restores() {
+ let shape = LaunchTargetShape { has_bundle_id: true, ..empty() };
+ assert!(should_restore_foreground_after_launch(shape));
+ }
+
+ #[test]
+ fn launch_path_only_restores() {
+ let shape = LaunchTargetShape { has_launch_path: true, ..empty() };
+ assert!(should_restore_foreground_after_launch(shape));
+ }
+
+ #[test]
+ fn name_with_urls_restores() {
+ // App-identifying field present alongside urls — the user wants
+ // the named app to open these URLs in the background, NOT for the
+ // urls to take over the default browser's foreground. Restore.
+ let shape = LaunchTargetShape { has_name: true, has_urls: true, ..empty() };
+ assert!(should_restore_foreground_after_launch(shape));
+ }
+
+ #[test]
+ fn path_with_urls_restores() {
+ let shape = LaunchTargetShape { has_path: true, has_urls: true, ..empty() };
+ assert!(should_restore_foreground_after_launch(shape));
+ }
+
+ #[test]
+ fn aumid_with_urls_restores() {
+ let shape = LaunchTargetShape { has_aumid: true, has_urls: true, ..empty() };
+ assert!(should_restore_foreground_after_launch(shape));
+ }
+
+ #[test]
+ fn no_target_does_not_restore() {
+ // Empty params (validated as an error before launch dispatch) — no
+ // restore needed because nothing was launched.
+ assert!(!should_restore_foreground_after_launch(empty()));
+ }
+}
+
#[cfg(test)]
mod chromium_flag_injection_tests {
use super::{inject_chromium_anti_throttling_flags, is_chromium_browser_target};
diff --git a/libs/cua-driver-rs/crates/platform-windows/src/tools/page_bookmark.rs b/libs/cua-driver-rs/crates/platform-windows/src/tools/page_bookmark.rs
index f6f68bc54a..11fe671828 100644
--- a/libs/cua-driver-rs/crates/platform-windows/src/tools/page_bookmark.rs
+++ b/libs/cua-driver-rs/crates/platform-windows/src/tools/page_bookmark.rs
@@ -32,6 +32,27 @@
//! cursor and so depend on z-order. Every failure here is logged and
//! returns `Err`, which the caller treats as "try CDP next". No path in
//! this module is allowed to panic.
+//!
+//! No-foreground contract
+//! ----------------------
+//! Earlier revisions of this module would synthesize `Ctrl+Shift+B` to
+//! summon the favorites bar when it was hidden. That keystroke is routed
+//! via [`crate::input::keyboard::send_key_synthesized`], which does a
+//! transient `SetForegroundWindow(target) → SendInput →
+//! SetForegroundWindow(prev_fg)` swap; the swap is visible to the user
+//! and the restore can fail silently under UIPI / UIAccess constraints.
+//! That violates the cua-driver no-foreground contract. The current code
+//! therefore *requires* the user to have shown the favorites bar once
+//! (the setting persists across sessions); if it's hidden we bail out and
+//! let `execute_javascript` fall through to the CDP path.
+//!
+//! One residual foreground-activation source remains and is documented as
+//! a known limitation: invoking the bookmark via UIA
+//! `InvokePattern::Invoke` causes Chromium to briefly raise the browser
+//! window because clicking a bookmark is a user-initiated navigation in
+//! Chromium's input model. This activation is Chromium-side and not
+//! something we can suppress from the UIA caller; see `WINDOWS.md` for
+//! the user-facing note.
use std::sync::Mutex;
use std::time::{Duration, Instant};
@@ -68,8 +89,6 @@ const BOOKMARK_NAME: &str = "cua-driver-eval";
/// Per-call timeouts.
mod budget {
use std::time::Duration;
- /// How long we wait for the favorites bar to appear after Ctrl+Shift+B.
- pub const FAV_BAR_APPEAR: Duration = Duration::from_millis(800);
/// How long we wait for a freshly opened dialog to appear in the tree.
pub const DIALOG_APPEAR: Duration = Duration::from_millis(2500);
/// How long we wait for the page's title to start with our marker.
@@ -169,30 +188,29 @@ unsafe fn try_bookmark_exec_blocking(
let edge_window = root_for_window(&automation, hwnd_raw, pid)
.context("could not resolve Edge top-level window for (pid, window_id)")?;
- // 2) Ensure the favorites bar is visible. If not, we don't get any
- // UIA-Invokable surface for the bookmark — even a created bookmark
- // isn't reachable. Send Ctrl+Shift+B to the Edge HWND and re-walk.
- let mut fav_bar = find_favorites_bar(&automation, &edge_window);
- if fav_bar.is_none() {
- tracing::debug!(
- target: "page.bookmark_exec",
- "favorites bar not present; sending Ctrl+Shift+B to hwnd 0x{hwnd_raw:x}"
- );
- toggle_favorites_bar(hwnd_raw)?;
- // The bar may take a beat to layout itself in.
- let deadline = Instant::now() + budget::FAV_BAR_APPEAR;
- while Instant::now() < deadline {
- std::thread::sleep(Duration::from_millis(60));
- fav_bar = find_favorites_bar(&automation, &edge_window);
- if fav_bar.is_some() {
- break;
- }
- }
- }
- let fav_bar = fav_bar.ok_or_else(|| {
+ // 2) Ensure the favorites bar is visible. If not, we bail.
+ //
+ // Historical note: we used to synthesize Ctrl+Shift+B here via
+ // `send_key_synthesized`, which does a transient
+ // `SetForegroundWindow(target) → SendInput → SetForegroundWindow(prev_fg)`
+ // dance. That dance is visible to the user (foreground flicker) and the
+ // restore can fail silently under UIPI / UIAccess constraints, both of
+ // which violate the cua-driver no-foreground contract. Every
+ // `page.execute_javascript` call where the user hadn't already shown
+ // the favorites bar would trigger this — unacceptable.
+ //
+ // The favorites-bar setting persists across browser sessions, so the
+ // one-time setup cost (Ctrl+Shift+B in the browser) is borne by the
+ // user once, never by the agent.
+ let fav_bar = find_favorites_bar(&automation, &edge_window).ok_or_else(|| {
anyhow!(
- "favorites bar is not present and Ctrl+Shift+B did not bring it up — \
- cannot reach the bookmark UI"
+ "Favorites bar not visible — cua-driver-rs's bookmark-URL JS \
+ exec needs an always-on favorites bar so we can invoke the \
+ `cua-driver-eval` bookmark without stealing the foreground. \
+ Show the bar once via Ctrl+Shift+B and the setting persists \
+ across sessions. The bookmark-exec path is now disabled for \
+ this call; `execute_javascript` will fall through to the CDP \
+ fallback if it's configured."
)
})?;
@@ -325,14 +343,6 @@ unsafe fn find_favorites_bar(
None
}
-/// Send Ctrl+Shift+B to the Edge top-level HWND using the existing
-/// `send_key_synthesized` helper. This is the Edge AccelKey for
-/// "Show/Hide favorites bar".
-unsafe fn toggle_favorites_bar(hwnd_raw: u64) -> Result<()> {
- crate::input::keyboard::send_key_synthesized(hwnd_raw, "b", &["ctrl", "shift"])
- .context("send_key_synthesized for Ctrl+Shift+B failed")
-}
-
unsafe fn find_bookmark(
automation: &IUIAutomation,
fav_bar: &IUIAutomationElement,