Skip to content

armgit / cua

Public Imported from GitHub
0

Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.

feat(cua-driver): let embedding hosts bound the post-action window observation (#3929)

mbfrmdao1435052d6aac
diff --git a/docs/content/docs/reference/cua-driver/embedding.mdx b/docs/content/docs/reference/cua-driver/embedding.mdx
index 2a1d01d6bc..24ec9041c3 100644
--- a/docs/content/docs/reference/cua-driver/embedding.mdx
+++ b/docs/content/docs/reference/cua-driver/embedding.mdx
@@ -138,6 +138,28 @@ Destroy the SDK client and call `await embedded.stop()` from every orderly shutd
The host refuses regular files, symlinks, and live listeners and only removes
the endpoint identity owned by the matching generation.
+## Post-action window observation
+
+After each input action, the driver watches the window list to report a menu,
+dialog, or new window the action opened. For an action that opens nothing, the
+wait lasts 1000 ms on macOS and 800 ms for Linux X11 foreground delivery. A host
+that observes its target continuously can shorten it by setting these variables
+in the daemon's launch environment. Tool arguments cannot change them.
+
+| Variable | Meaning | Accepted values |
+| ------------------------------------- | ----------------------------------------------- | ------------------------------------------------------ |
+| `CUA_DRIVER_WINDOW_CHANGE_TIMEOUT_MS` | Longest wait after each action. `0` skips it. | 0 to 10000 ms. Unset or unparsable keeps the default. |
+| `CUA_DRIVER_WINDOW_CHANGE_POLL_MS` | Interval between reads during the wait. | 5 to 1000 ms, and never longer than the wait. |
+
+On macOS, the wait also holds the lease that reactivates the previously
+frontmost app when another app steals focus. A shorter wait shortens that
+protection, and `0` releases it as soon as the action returns. Windows that
+appear after the wait are not reported. On Linux X11, a foreground action can
+report `effect: "confirmed"` only when a window change occurs within the wait.
+Windows ignores both variables because it does not run this wait. See
+[`EMBEDDING.md`](https://github.com/trycua/cua/blob/main/libs/cua-driver/rust/Skills/cua-driver/EMBEDDING.md#bounding-the-post-action-window-observation)
+for the full tradeoffs.
+
## Host requirements
- Spawn `cua-driver serve --embedded` directly from your app, for example with `Process` / `NSTask`, `posix_spawn`, or `fork` / `exec`.
diff --git a/libs/cua-driver/rust/Skills/cua-driver/EMBEDDING.md b/libs/cua-driver/rust/Skills/cua-driver/EMBEDDING.md
index f21d82cbff..c79548c271 100644
--- a/libs/cua-driver/rust/Skills/cua-driver/EMBEDDING.md
+++ b/libs/cua-driver/rust/Skills/cua-driver/EMBEDDING.md
@@ -200,6 +200,61 @@ daemon child.
- Permission changes require destroying clients, restarting the daemon, and
reconnecting. A connection from the old generation is never reusable.
+## Bounding the post-action window observation
+
+After an input action, the driver watches the window list for a short time so
+it can report a menu, dialog, or new window that the action opened. On macOS,
+that watch lasts up to 1000 ms for an action that opens nothing, which makes it
+the largest part of a background click's latency. A host that already observes
+its target continuously can shorten it through two variables set at trusted
+launch, in the environment of the `serve --embedded` child (or of the host
+process when you use the same-process runtime):
+
+| Variable | Meaning | Default | Accepted range |
+| ------------------------------------- | ---------------------------------------------------- | ------------------------------- | ----------------------------------------- |
+| `CUA_DRIVER_WINDOW_CHANGE_TIMEOUT_MS` | Longest wait for a window change after each action. | 1000 on macOS, 800 on Linux X11 | 0 to 10000; larger values are clamped |
+| `CUA_DRIVER_WINDOW_CHANGE_POLL_MS` | Interval between window-list reads during that wait. | 50 | 5 to 1000, and never longer than the wait |
+
+Unset, empty, or unparsable values, such as `-1`, `1.5`, or `100ms`, keep the
+default, so a daemon launched without these variables behaves exactly as
+before. No tool argument can change the bound. Every ingress strips
+underscore-prefixed arguments, so an agent cannot shorten its own focus
+protection.
+
+```sh
+CUA_DRIVER_EMBEDDED=1 \
+CUA_DRIVER_WINDOW_CHANGE_TIMEOUT_MS=200 \
+ cua-driver serve --embedded --socket /tmp/yourapp-cua.sock
+```
+
+A shorter wait costs something on each platform. Choose a value knowingly.
+
+- **macOS: less focus protection.** The watch also holds a focus-steal lease:
+ while it runs, the driver reactivates the app that was frontmost before the
+ action if any other app activates, such as a browser opened by a link. The
+ lease ends when the watch ends. With a nonzero value, cross-app activations
+ are reverted only for that long after the action returns. With `0`, the lease
+ is released as soon as the action returns. The separate target-pid guard
+ remains, and it still covers the action plus a 50 ms settle when the target
+ was not frontmost. An app that activates later stays frontmost, so the host
+ must detect and correct that itself.
+- **macOS: missing result suffixes.** A window that appears after the wait
+ ends is not reported. With `0`, results never carry
+ `Action opened new window(s): …` or
+ `Action caused a different app to become frontmost.`
+- **Linux X11 foreground delivery: weaker evidence.** A foreground-delivery
+ action (`"delivery_mode": "foreground"`) reports `effect: "confirmed"` when
+ the target opened or closed a window during the wait. A dialog that maps after a shorter wait
+ leaves the action `unverifiable`. Qt file dialogs can take 300 to 600 ms to
+ map. With `0`, foreground actions never report window-change evidence. Focus
+ checks and `suspected_noop` detection are unchanged.
+- **Windows: no effect.** The Windows adapter does not wait for window changes
+ after an action, so it ignores both variables.
+
+Measured on macOS with Calculator in the background and Terminal frontmost, a
+background AX click took 1153 ms with the default, 325 ms with `200`, and 78 ms
+with `0`.
+
## What embedded mode changes (and what it doesn't)
| | Standalone | Embedded (`CUA_DRIVER_EMBEDDED=1`) |
diff --git a/libs/cua-driver/rust/crates/cua-driver-core/src/lib.rs b/libs/cua-driver/rust/crates/cua-driver-core/src/lib.rs
index b1ac83baf2..c718b2d91b 100644
--- a/libs/cua-driver/rust/crates/cua-driver-core/src/lib.rs
+++ b/libs/cua-driver/rust/crates/cua-driver-core/src/lib.rs
@@ -96,6 +96,7 @@ pub mod video;
pub mod video_ffmpeg;
pub mod walk_budget;
pub mod window_inspection;
+pub mod window_observation;
pub mod window_target;
pub use cua_driver_contract::{CaptureScope, EscalationReason, TOOL_INVOCATION_FAILED_CODE};
diff --git a/libs/cua-driver/rust/crates/cua-driver-core/src/window_observation.rs b/libs/cua-driver/rust/crates/cua-driver-core/src/window_observation.rs
new file mode 100644
index 0000000000..33f1e78932
--- /dev/null
+++ b/libs/cua-driver/rust/crates/cua-driver-core/src/window_observation.rs
@@ -0,0 +1,187 @@
+//! Host-chosen bound for the post-action window-change observation.
+//!
+//! After an input action, the macOS adapter and the Linux X11 foreground
+//! transaction poll the window set for a short time to report a menu, dialog,
+//! or new window the action opened. That poll is the dominant latency of a
+//! background action that opens nothing. An embedding host that already
+//! observes its target continuously can bound it at trusted launch through the
+//! daemon environment:
+//!
+//! - [`WINDOW_CHANGE_TIMEOUT_ENV`] (`CUA_DRIVER_WINDOW_CHANGE_TIMEOUT_MS`):
+//! the observation deadline in milliseconds. `0` skips the observation.
+//! - [`WINDOW_CHANGE_POLL_ENV`] (`CUA_DRIVER_WINDOW_CHANGE_POLL_MS`): the
+//! interval between window-set reads in milliseconds.
+//!
+//! Unset, empty, or unparsable values keep each adapter's default, so a
+//! daemon launched without these variables behaves exactly as before. The
+//! values are read from the daemon process environment only; no tool
+//! argument can change them. See `Skills/cua-driver/EMBEDDING.md` for what a
+//! shorter or zero bound costs on each platform.
+
+use std::time::Duration;
+
+/// Post-action observation deadline, in milliseconds.
+pub const WINDOW_CHANGE_TIMEOUT_ENV: &str = "CUA_DRIVER_WINDOW_CHANGE_TIMEOUT_MS";
+
+/// Interval between window-set reads during the observation, in milliseconds.
+pub const WINDOW_CHANGE_POLL_ENV: &str = "CUA_DRIVER_WINDOW_CHANGE_POLL_MS";
+
+/// Largest accepted deadline. Larger values are clamped so a typo cannot make
+/// every action wait for minutes.
+pub const MAX_WINDOW_CHANGE_TIMEOUT: Duration = Duration::from_millis(10_000);
+
+/// Smallest accepted poll interval. Smaller values (including `0`) are raised
+/// so the observation never spins.
+pub const MIN_WINDOW_CHANGE_POLL: Duration = Duration::from_millis(5);
+
+/// Largest accepted poll interval.
+pub const MAX_WINDOW_CHANGE_POLL: Duration = Duration::from_millis(1_000);
+
+/// Resolved observation bounds for one action.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub struct WindowObservationBounds {
+ /// Deadline for the observation. Zero means "do not observe".
+ pub timeout: Duration,
+ /// Interval between reads. At least [`MIN_WINDOW_CHANGE_POLL`], and never
+ /// longer than a non-zero `timeout` that is itself above that minimum, so
+ /// the observation cannot overshoot a short deadline by a whole poll.
+ pub poll: Duration,
+}
+
+impl WindowObservationBounds {
+ /// True when the host asked to skip the observation entirely.
+ pub fn skips_observation(&self) -> bool {
+ self.timeout.is_zero()
+ }
+
+ /// Resolve bounds from raw environment values. Pure, so it can be tested
+ /// without touching the process environment.
+ pub fn from_raw(
+ timeout_raw: Option<&str>,
+ poll_raw: Option<&str>,
+ default_timeout: Duration,
+ default_poll: Duration,
+ ) -> Self {
+ let timeout = parse_millis(timeout_raw)
+ .unwrap_or(default_timeout)
+ .min(MAX_WINDOW_CHANGE_TIMEOUT);
+ let poll = parse_millis(poll_raw)
+ .unwrap_or(default_poll)
+ .min(timeout.max(MIN_WINDOW_CHANGE_POLL))
+ .clamp(MIN_WINDOW_CHANGE_POLL, MAX_WINDOW_CHANGE_POLL);
+ Self { timeout, poll }
+ }
+
+ /// Resolve bounds from the daemon environment, falling back to the
+ /// adapter's own defaults for anything unset or unparsable.
+ pub fn from_env(default_timeout: Duration, default_poll: Duration) -> Self {
+ let timeout = std::env::var(WINDOW_CHANGE_TIMEOUT_ENV).ok();
+ let poll = std::env::var(WINDOW_CHANGE_POLL_ENV).ok();
+ Self::from_raw(
+ timeout.as_deref(),
+ poll.as_deref(),
+ default_timeout,
+ default_poll,
+ )
+ }
+}
+
+/// Parse a non-negative whole number of milliseconds. Anything else (empty,
+/// negative, fractional, non-numeric, or overflowing `u64`) is `None`.
+fn parse_millis(raw: Option<&str>) -> Option<Duration> {
+ raw?.trim().parse::<u64>().ok().map(Duration::from_millis)
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ const DEFAULT_TIMEOUT: Duration = Duration::from_millis(1000);
+ const DEFAULT_POLL: Duration = Duration::from_millis(50);
+
+ fn bounds(timeout: Option<&str>, poll: Option<&str>) -> WindowObservationBounds {
+ WindowObservationBounds::from_raw(timeout, poll, DEFAULT_TIMEOUT, DEFAULT_POLL)
+ }
+
+ #[test]
+ fn unset_keeps_adapter_defaults() {
+ let b = bounds(None, None);
+ assert_eq!(b.timeout, DEFAULT_TIMEOUT);
+ assert_eq!(b.poll, DEFAULT_POLL);
+ assert!(!b.skips_observation());
+ }
+
+ #[test]
+ fn valid_values_are_honored() {
+ let b = bounds(Some("200"), Some("20"));
+ assert_eq!(b.timeout, Duration::from_millis(200));
+ assert_eq!(b.poll, Duration::from_millis(20));
+ let padded = bounds(Some(" 300 "), Some("\t25\n"));
+ assert_eq!(padded.timeout, Duration::from_millis(300));
+ assert_eq!(padded.poll, Duration::from_millis(25));
+ }
+
+ #[test]
+ fn zero_timeout_skips_observation() {
+ let b = bounds(Some("0"), None);
+ assert_eq!(b.timeout, Duration::ZERO);
+ assert!(b.skips_observation());
+ // The poll interval stays non-zero even though it is unused.
+ assert_eq!(b.poll, MIN_WINDOW_CHANGE_POLL);
+ }
+
+ #[test]
+ fn invalid_values_fall_back_to_defaults() {
+ for raw in [
+ "",
+ " ",
+ "abc",
+ "-1",
+ "1.5",
+ "100ms",
+ "18446744073709551616",
+ ] {
+ let b = bounds(Some(raw), Some(raw));
+ assert_eq!(b.timeout, DEFAULT_TIMEOUT, "timeout for {raw:?}");
+ assert_eq!(b.poll, DEFAULT_POLL, "poll for {raw:?}");
+ }
+ }
+
+ #[test]
+ fn too_large_values_are_clamped() {
+ let b = bounds(Some("600000"), Some("5000"));
+ assert_eq!(b.timeout, MAX_WINDOW_CHANGE_TIMEOUT);
+ assert_eq!(b.poll, MAX_WINDOW_CHANGE_POLL);
+ let max = bounds(Some(&u64::MAX.to_string()), None);
+ assert_eq!(max.timeout, MAX_WINDOW_CHANGE_TIMEOUT);
+ }
+
+ #[test]
+ fn zero_or_tiny_poll_is_raised_to_the_minimum() {
+ assert_eq!(bounds(None, Some("0")).poll, MIN_WINDOW_CHANGE_POLL);
+ assert_eq!(bounds(None, Some("1")).poll, MIN_WINDOW_CHANGE_POLL);
+ }
+
+ #[test]
+ fn poll_never_exceeds_a_short_timeout() {
+ let b = bounds(Some("30"), Some("200"));
+ assert_eq!(b.timeout, Duration::from_millis(30));
+ assert_eq!(b.poll, Duration::from_millis(30));
+ // The adapter default poll is capped the same way.
+ assert_eq!(bounds(Some("10"), None).poll, Duration::from_millis(10));
+ // A timeout below the poll floor still polls at the floor.
+ assert_eq!(bounds(Some("2"), None).poll, MIN_WINDOW_CHANGE_POLL);
+ }
+
+ #[test]
+ fn adapter_defaults_pass_through_per_platform() {
+ let linux = WindowObservationBounds::from_raw(
+ None,
+ None,
+ Duration::from_millis(800),
+ Duration::from_millis(50),
+ );
+ assert_eq!(linux.timeout, Duration::from_millis(800));
+ assert_eq!(linux.poll, Duration::from_millis(50));
+ }
+}
diff --git a/libs/cua-driver/rust/crates/platform-linux/src/input/foreground.rs b/libs/cua-driver/rust/crates/platform-linux/src/input/foreground.rs
index 3762da696a..bf4322fd79 100644
--- a/libs/cua-driver/rust/crates/platform-linux/src/input/foreground.rs
+++ b/libs/cua-driver/rust/crates/platform-linux/src/input/foreground.rs
@@ -17,6 +17,7 @@
//! Xlib's process-wide error handler.
use anyhow::{anyhow, Result};
+use cua_driver_core::window_observation::WindowObservationBounds;
use std::sync::mpsc;
use std::time::{Duration, Instant};
use x11rb::connection::Connection as _;
@@ -216,13 +217,31 @@ const WINDOW_CHANGE_TITLE_GRACE: Duration = Duration::from_millis(60);
/// a beat before the toolkit re-focuses its toplevel.
const FOCUS_RETRY: Duration = Duration::from_millis(250);
+/// Post-action observation bounds: `WINDOW_CHANGE_DEADLINE` /
+/// `WINDOW_CHANGE_POLL` unless the embedding host set
+/// `CUA_DRIVER_WINDOW_CHANGE_TIMEOUT_MS` / `CUA_DRIVER_WINDOW_CHANGE_POLL_MS`
+/// on the daemon environment (shared parsing with macOS in
+/// `cua_driver_core::window_observation`).
+fn window_change_bounds() -> WindowObservationBounds {
+ WindowObservationBounds::from_env(WINDOW_CHANGE_DEADLINE, WINDOW_CHANGE_POLL)
+}
+
/// Poll `pid`'s window set until it differs from `before` or the deadline
/// passes. Returns the last set read and the change description, if any.
+///
+/// A zero timeout reads the set once (the post-check still needs it to
+/// recognise focus inside a popup of the target process) and reports no
+/// change, so the action falls back to `unverifiable` rather than claiming
+/// `confirmed` window-change evidence it never looked for.
fn wait_for_window_change(
pid: Option<u32>,
before: &[(u64, String)],
+ bounds: WindowObservationBounds,
) -> (Vec<(u64, String)>, Option<String>) {
- let deadline = Instant::now() + WINDOW_CHANGE_DEADLINE;
+ if bounds.skips_observation() {
+ return (pid_window_set(pid), None);
+ }
+ let deadline = Instant::now() + bounds.timeout;
loop {
let after = pid_window_set(pid);
if let Some(change) = describe_window_change(before, &after) {
@@ -236,7 +255,7 @@ fn wait_for_window_change(
if Instant::now() >= deadline {
return (after, None);
}
- std::thread::sleep(WINDOW_CHANGE_POLL);
+ std::thread::sleep(bounds.poll);
}
}
@@ -664,9 +683,10 @@ pub fn with_x11_foreground_opts<T>(
// the window set / focus within the target's process, and one that
// landed elsewhere moves focus out of it. The window set is polled
// rather than read after a fixed settle: it returns on the first change.
+ let bounds = window_change_bounds();
let (windows_after, window_change) = run_with_deadline(
- WINDOW_CHANGE_DEADLINE + Duration::from_millis(1500),
- move || wait_for_window_change(target_pid, &windows_before),
+ bounds.timeout + bounds.poll + WINDOW_CHANGE_TITLE_GRACE + Duration::from_millis(1500),
+ move || wait_for_window_change(target_pid, &windows_before, bounds),
)
.unwrap_or((Vec::new(), None));
let focus_after = run_with_deadline(Duration::from_millis(1500), move || {
@@ -757,6 +777,44 @@ mod tests {
assert!(started.elapsed() < Duration::from_secs(2));
}
+ #[test]
+ fn window_change_bounds_default_to_the_linux_deadline() {
+ let unset = WindowObservationBounds::from_raw(
+ None,
+ None,
+ WINDOW_CHANGE_DEADLINE,
+ WINDOW_CHANGE_POLL,
+ );
+ assert_eq!(unset.timeout, WINDOW_CHANGE_DEADLINE);
+ assert_eq!(unset.poll, WINDOW_CHANGE_POLL);
+ let host = WindowObservationBounds::from_raw(
+ Some("120"),
+ Some("20"),
+ WINDOW_CHANGE_DEADLINE,
+ WINDOW_CHANGE_POLL,
+ );
+ assert_eq!(host.timeout, Duration::from_millis(120));
+ assert_eq!(host.poll, Duration::from_millis(20));
+ }
+
+ /// A zero host timeout never reports window-change evidence (which would
+ /// upgrade the action to `effect: confirmed`) even when the before-set
+ /// differs from the live one. Read-only: no input is sent.
+ #[test]
+ fn zero_window_change_timeout_reports_no_change() {
+ let bounds = WindowObservationBounds::from_raw(
+ Some("0"),
+ None,
+ WINDOW_CHANGE_DEADLINE,
+ WINDOW_CHANGE_POLL,
+ );
+ let before = vec![(u64::MAX, "window that never existed".to_string())];
+ let started = Instant::now();
+ let (_after, change) = wait_for_window_change(Some(u32::MAX), &before, bounds);
+ assert_eq!(change, None);
+ assert!(started.elapsed() < WINDOW_CHANGE_DEADLINE);
+ }
+
#[test]
fn deadline_returns_value_when_work_finishes() {
assert_eq!(run_with_deadline(Duration::from_secs(2), || 7), Some(7));
diff --git a/libs/cua-driver/rust/crates/platform-macos/src/tools/click.rs b/libs/cua-driver/rust/crates/platform-macos/src/tools/click.rs
index 222c1fc85c..7ce3c2679c 100644
--- a/libs/cua-driver/rust/crates/platform-macos/src/tools/click.rs
+++ b/libs/cua-driver/rust/crates/platform-macos/src/tools/click.rs
@@ -746,7 +746,7 @@ impl Tool for ClickTool {
.await;
// Drop the wildcard lease + detect window/foreground side-effects.
- let changes = super::finish_window_observation(snapshot, &args).await;
+ let changes = super::finish_window_observation(snapshot).await;
match result {
Ok(Ok((
@@ -1260,7 +1260,7 @@ impl Tool for ClickTool {
}
}
- let changes = super::finish_window_observation(snapshot, &args).await;
+ let changes = super::finish_window_observation(snapshot).await;
let button_label = match button_str.as_str() {
"right" => "right-click",
diff --git a/libs/cua-driver/rust/crates/platform-macos/src/tools/drag.rs b/libs/cua-driver/rust/crates/platform-macos/src/tools/drag.rs
index 91de626694..37dd9da6bb 100644
--- a/libs/cua-driver/rust/crates/platform-macos/src/tools/drag.rs
+++ b/libs/cua-driver/rust/crates/platform-macos/src/tools/drag.rs
@@ -397,7 +397,7 @@ impl Tool for DragTool {
.update_position(&cursor_key, to_sx, to_sy);
}
- let changes = super::finish_window_observation(snapshot, &args).await;
+ let changes = super::finish_window_observation(snapshot).await;
if let Some(wid) = window_id {
crate::cursor::overlay::send_command(
diff --git a/libs/cua-driver/rust/crates/platform-macos/src/tools/hotkey.rs b/libs/cua-driver/rust/crates/platform-macos/src/tools/hotkey.rs
index 1b33b4426f..81f4aac707 100644
--- a/libs/cua-driver/rust/crates/platform-macos/src/tools/hotkey.rs
+++ b/libs/cua-driver/rust/crates/platform-macos/src/tools/hotkey.rs
@@ -467,7 +467,7 @@ impl Tool for HotkeyTool {
)
.await;
- let changes = super::finish_window_observation(snapshot, &args).await;
+ let changes = super::finish_window_observation(snapshot).await;
match result {
Ok(Ok(())) => {
diff --git a/libs/cua-driver/rust/crates/platform-macos/src/tools/mod.rs b/libs/cua-driver/rust/crates/platform-macos/src/tools/mod.rs
index e3fb405aae..99a32faba3 100644
--- a/libs/cua-driver/rust/crates/platform-macos/src/tools/mod.rs
+++ b/libs/cua-driver/rust/crates/platform-macos/src/tools/mod.rs
@@ -288,40 +288,15 @@ pub(crate) async fn acquire_background_mutation(pid: i32) -> BackgroundMutationL
}
}
-/// Finish the post-action observation window. Embedded interactive clients
-/// that already observe the target continuously may opt out through the
-/// private registry argument to avoid adding a one-second acknowledgement
-/// delay to every input event. Regular MCP callers retain the full observer.
+/// Finish the post-action observation window and release the wildcard
+/// focus-steal lease. The observation bound is a daemon-launch setting
+/// (`CUA_DRIVER_WINDOW_CHANGE_TIMEOUT_MS` / `CUA_DRIVER_WINDOW_CHANGE_POLL_MS`),
+/// never a per-call argument: every ingress strips `_`-prefixed arguments, so
+/// a tool call cannot shorten focus protection for itself.
pub(crate) async fn finish_window_observation(
snapshot: crate::window_change_detector::Snapshot,
- args: &serde_json::Value,
) -> crate::window_change_detector::Changes {
- if args
- .get("_skip_window_change_detection")
- .and_then(serde_json::Value::as_bool)
- .unwrap_or(false)
- {
- drop(snapshot);
- crate::window_change_detector::Changes::no_change()
- } else {
- snapshot.detect_async().await
- }
-}
-
-#[cfg(test)]
-mod interactive_observation_tests {
- use super::*;
-
- #[tokio::test]
- async fn embedded_interactive_input_can_finish_without_polling() {
- let snapshot = crate::window_change_detector::WindowChangeDetector::snapshot(None);
- let changes = finish_window_observation(
- snapshot,
- &serde_json::json!({"_skip_window_change_detection": true}),
- )
- .await;
- assert!(!changes.needs_restore());
- }
+ snapshot.detect_async().await
}
/// px-focus for the keyboard family (type_text / press_key / hotkey): focus the
diff --git a/libs/cua-driver/rust/crates/platform-macos/src/tools/press_key.rs b/libs/cua-driver/rust/crates/platform-macos/src/tools/press_key.rs
index 393f17c3a5..8162ec97cd 100644
--- a/libs/cua-driver/rust/crates/platform-macos/src/tools/press_key.rs
+++ b/libs/cua-driver/rust/crates/platform-macos/src/tools/press_key.rs
@@ -451,7 +451,7 @@ impl Tool for PressKeyTool {
)
.await;
- let changes = super::finish_window_observation(snapshot, &args).await;
+ let changes = super::finish_window_observation(snapshot).await;
let delivery_outcome = match result {
Ok(result) => map_delivery_outcome(result),
diff --git a/libs/cua-driver/rust/crates/platform-macos/src/tools/scroll.rs b/libs/cua-driver/rust/crates/platform-macos/src/tools/scroll.rs
index 7d4a63aa47..76ecd2778e 100644
--- a/libs/cua-driver/rust/crates/platform-macos/src/tools/scroll.rs
+++ b/libs/cua-driver/rust/crates/platform-macos/src/tools/scroll.rs
@@ -561,7 +561,7 @@ impl Tool for ScrollTool {
)
.await;
- let changes = super::finish_window_observation(snapshot, &args).await;
+ let changes = super::finish_window_observation(snapshot).await;
let mode_label = if fg {
" (delivery_mode:foreground)"
} else {
@@ -660,7 +660,7 @@ impl Tool for ScrollTool {
)
.await;
- let changes = super::finish_window_observation(snapshot, &args).await;
+ let changes = super::finish_window_observation(snapshot).await;
match result {
Ok(Ok(())) => ToolResult::text(format!(
diff --git a/libs/cua-driver/rust/crates/platform-macos/src/tools/type_text.rs b/libs/cua-driver/rust/crates/platform-macos/src/tools/type_text.rs
index 091f7a7891..6555a5a6a4 100644
--- a/libs/cua-driver/rust/crates/platform-macos/src/tools/type_text.rs
+++ b/libs/cua-driver/rust/crates/platform-macos/src/tools/type_text.rs
@@ -399,7 +399,7 @@ impl Tool for TypeTextTool {
)
.await;
- let changes = super::finish_window_observation(snapshot, &args).await;
+ let changes = super::finish_window_observation(snapshot).await;
// Unwrap the delivery envelope: a structured refusal means no
// actuator ran and the caller gets the exact reason.
diff --git a/libs/cua-driver/rust/crates/platform-macos/src/window_change_detector.rs b/libs/cua-driver/rust/crates/platform-macos/src/window_change_detector.rs
index 3286ac6230..4eb41e454c 100644
--- a/libs/cua-driver/rust/crates/platform-macos/src/window_change_detector.rs
+++ b/libs/cua-driver/rust/crates/platform-macos/src/window_change_detector.rs
@@ -39,6 +39,8 @@
use std::collections::HashSet;
use std::time::{Duration, Instant};
+use cua_driver_core::window_observation::WindowObservationBounds;
+
use crate::apps;
use crate::focus_steal::{self, SuppressionLease};
use crate::windows::{self, WindowInfo};
@@ -154,6 +156,29 @@ const DEFAULT_TIMEOUT: Duration = Duration::from_millis(1000);
/// Default inter-poll interval. Matches Swift's 50ms.
const DEFAULT_POLL_INTERVAL: Duration = Duration::from_millis(50);
+/// Resolve the post-action observation bounds from raw host values against
+/// the macOS defaults. Pure; `cua_driver_core::window_observation` owns the
+/// parsing and clamping rules shared with the Linux adapter.
+fn observation_bounds_from(
+ timeout_raw: Option<&str>,
+ poll_raw: Option<&str>,
+) -> WindowObservationBounds {
+ WindowObservationBounds::from_raw(
+ timeout_raw,
+ poll_raw,
+ DEFAULT_TIMEOUT,
+ DEFAULT_POLL_INTERVAL,
+ )
+}
+
+/// Post-action observation bounds chosen by the embedding host through
+/// `CUA_DRIVER_WINDOW_CHANGE_TIMEOUT_MS` / `CUA_DRIVER_WINDOW_CHANGE_POLL_MS`
+/// on the daemon environment; `DEFAULT_TIMEOUT` / `DEFAULT_POLL_INTERVAL`
+/// when unset or unparsable.
+pub(crate) fn host_observation_bounds() -> WindowObservationBounds {
+ WindowObservationBounds::from_env(DEFAULT_TIMEOUT, DEFAULT_POLL_INTERVAL)
+}
+
/// Public API. Mirrors Swift `enum WindowChangeDetector` — no state of
/// its own; all state lives inside the returned `Snapshot`.
pub struct WindowChangeDetector;
@@ -249,11 +274,30 @@ impl Snapshot {
/// foreground-app change. Returns as soon as a change is detected
/// or the timeout elapses.
///
+ /// An embedding host that already observes the target continuously
+ /// can bound this window through `CUA_DRIVER_WINDOW_CHANGE_TIMEOUT_MS` /
+ /// `CUA_DRIVER_WINDOW_CHANGE_POLL_MS` on the daemon's environment.
+ /// Unset or unparsable values keep the defaults, so public callers see
+ /// no behavior change.
+ ///
/// Consumes the snapshot — the wildcard suppression lease is
/// dropped when this returns (covers the full action + detection
- /// window).
+ /// window). A shorter timeout therefore also shortens the wildcard
+ /// focus-steal protection, and a zero timeout releases it as soon as
+ /// the action returns and reports no change.
pub fn detect(self) -> Changes {
- self.detect_with(DEFAULT_TIMEOUT, DEFAULT_POLL_INTERVAL)
+ self.detect_bounded(host_observation_bounds())
+ }
+
+ /// `detect()` with explicit bounds. A zero timeout drops the snapshot
+ /// (and its wildcard suppression lease) immediately without reading
+ /// the window list again.
+ pub(crate) fn detect_bounded(self, bounds: WindowObservationBounds) -> Changes {
+ if bounds.skips_observation() {
+ drop(self);
+ return Changes::no_change();
+ }
+ self.detect_with(bounds.timeout, bounds.poll)
}
/// Async wrapper around `detect()` — runs the synchronous poll
@@ -511,6 +555,60 @@ mod tests {
);
}
+ #[test]
+ fn host_bounds_unset_keep_macos_defaults() {
+ let b = observation_bounds_from(None, None);
+ assert_eq!(b.timeout, DEFAULT_TIMEOUT);
+ assert_eq!(b.poll, DEFAULT_POLL_INTERVAL);
+ assert!(!b.skips_observation());
+ }
+
+ #[test]
+ fn host_bounds_valid_values_are_honored() {
+ let b = observation_bounds_from(Some("200"), Some("20"));
+ assert_eq!(b.timeout, Duration::from_millis(200));
+ assert_eq!(b.poll, Duration::from_millis(20));
+ }
+
+ #[test]
+ fn host_bounds_zero_timeout_skips_observation() {
+ let b = observation_bounds_from(Some("0"), None);
+ assert!(b.skips_observation());
+ assert!(!b.poll.is_zero());
+ }
+
+ #[test]
+ fn host_bounds_invalid_values_keep_macos_defaults() {
+ for raw in ["", "abc", "-5", "2.5", "99999999999999999999"] {
+ let b = observation_bounds_from(Some(raw), Some(raw));
+ assert_eq!(b.timeout, DEFAULT_TIMEOUT, "timeout for {raw:?}");
+ assert_eq!(b.poll, DEFAULT_POLL_INTERVAL, "poll for {raw:?}");
+ }
+ }
+
+ #[test]
+ fn host_bounds_too_large_values_are_clamped() {
+ use cua_driver_core::window_observation::{
+ MAX_WINDOW_CHANGE_POLL, MAX_WINDOW_CHANGE_TIMEOUT,
+ };
+ let b = observation_bounds_from(Some("3600000"), Some("60000"));
+ assert_eq!(b.timeout, MAX_WINDOW_CHANGE_TIMEOUT);
+ assert_eq!(b.poll, MAX_WINDOW_CHANGE_POLL);
+ }
+
+ /// A zero timeout returns immediately with no change instead of
+ /// sleeping a poll interval. Reads the window list once (in
+ /// `snapshot`) and sends no input.
+ #[test]
+ fn zero_timeout_detect_returns_without_polling() {
+ let snap = WindowChangeDetector::snapshot(None);
+ let started = Instant::now();
+ let changes = snap.detect_bounded(observation_bounds_from(Some("0"), None));
+ assert!(started.elapsed() < DEFAULT_POLL_INTERVAL);
+ assert!(!changes.needs_restore());
+ assert_eq!(changes.result_suffix(), "");
+ }
+
/// Regression: `snapshot(prior_front)` must store the caller's
/// captured front pid verbatim (rather than re-reading it inside
/// the function and racing with concurrent activations).