Skip to content

armgit / cua

Public Imported from GitHub
0

Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.

Organize cua-driver harness tests and docs

Francesco Bonacciadcd70a9c3c4
diff --git a/docs/content/docs/reference/cua-driver/development.mdx b/docs/content/docs/reference/cua-driver/development.mdx
new file mode 100644
index 0000000000..5e16d62068
--- /dev/null
+++ b/docs/content/docs/reference/cua-driver/development.mdx
@@ -0,0 +1,30 @@
+---
+title: Development
+description: Source layout, harnesses, and test entry points for contributors working on cua-driver.
+---
+
+cua-driver lives under `libs/cua-driver` in the repository. The hosted docs
+describe the public interface; contributor details live next to the code so
+they stay close to implementation changes.
+
+## Source Map
+
+| Path | Purpose |
+|---|---|
+| `libs/cua-driver/rust` | Rust workspace for the daemon, platform crates, testkit, and helper crates |
+| `libs/cua-driver/python` | Python package wrapper around the driver binary |
+| `libs/cua-driver/test-harness` | Source-built GUI harness apps and shared fixtures |
+| `libs/cua-driver/tests/e2e` | Manual Python e2e tests against real apps and harness apps |
+| `libs/cua-driver/scripts` | Install, uninstall, local build, and VM sync helpers |
+
+## Contributor Entry Points
+
+- `libs/cua-driver/README.md` gives the repo-local overview.
+- `libs/cua-driver/rust/README.md` maps the Cargo workspace and test classes.
+- `libs/cua-driver/rust/crates/cua-driver/tests/README.md` explains Rust test naming and ignored GUI lanes.
+- `libs/cua-driver/test-harness/README.md` explains source-built harness apps and staged outputs.
+- `libs/cua-driver/tests/e2e/README.md` explains the Python e2e suite.
+- `libs/cua-driver/scripts/README.md` explains install and VM sync helpers.
+
+Build artifacts, VM logs, and local verification journals should stay out of
+git unless they have been promoted into a stable fixture or contributor doc.
diff --git a/docs/content/docs/reference/cua-driver/meta.json b/docs/content/docs/reference/cua-driver/meta.json
index d37f264b67..2773873c78 100644
--- a/docs/content/docs/reference/cua-driver/meta.json
+++ b/docs/content/docs/reference/cua-driver/meta.json
@@ -1 +1 @@
-{ "title": "Cua Driver", "pages": ["cli-reference", "macos-permissions", "embedding", "mcp-tools", "mcp-tool-notes", "action-selection-policy", "contracts", "process-model", "limits"] }
+{ "title": "Cua Driver", "pages": ["cli-reference", "macos-permissions", "embedding", "mcp-tools", "mcp-tool-notes", "action-selection-policy", "contracts", "process-model", "limits", "development"] }
diff --git a/libs/cua-driver/README.md b/libs/cua-driver/README.md
index bd124760c9..eeab1ed0e1 100644
--- a/libs/cua-driver/README.md
+++ b/libs/cua-driver/README.md
@@ -4,6 +4,20 @@ Background computer-use driver for any agents. Speaks MCP over stdio; drives nat
**[Documentation](https://cua.ai/docs/cua-driver)** - Installation, guides, and API reference.
+## Repository Layout
+
+| Path | Purpose |
+| --- | --- |
+| `rust/` | Cargo workspace for the driver daemon, platform crates, and Rust tests |
+| `python/` | Python package wrapper and package tests |
+| `test-harness/` | Source-built GUI harness apps and shared fixtures |
+| `tests/e2e/` | Manual Python e2e tests against real apps and harness apps |
+| `scripts/` | Install, uninstall, local build, and VM sync helpers |
+| `docs/` | Small repo-local specs that are not part of the hosted docs site |
+
+Start with `rust/README.md`, `test-harness/README.md`, and
+`tests/e2e/README.md` when changing driver behavior or tests.
+
## Claude Code computer-use compatibility
Standard Claude Code MCP registration:
diff --git a/libs/cua-driver/rust/.gitignore b/libs/cua-driver/rust/.gitignore
index d70eb15e28..e69de29bb2 100644
--- a/libs/cua-driver/rust/.gitignore
+++ b/libs/cua-driver/rust/.gitignore
@@ -1,3 +0,0 @@
-# Large test assets — Electron app bundle exceeds GitHub's 100 MB file limit.
-# Download separately for local integration testing.
-tests/integration/v2/assets/electron/
diff --git a/libs/cua-driver/rust/README.md b/libs/cua-driver/rust/README.md
new file mode 100644
index 0000000000..01338b1766
--- /dev/null
+++ b/libs/cua-driver/rust/README.md
@@ -0,0 +1,58 @@
+# cua-driver Rust workspace
+
+This directory is the standalone Cargo workspace for the driver daemon, platform
+implementations, testkit, and helper crates.
+
+## Crates
+
+| Crate | Purpose |
+| --- | --- |
+| `cua-driver` | Main CLI/MCP daemon and integration tests |
+| `cua-driver-core` | Shared protocol, tool models, config, and CDP helpers |
+| `platform-macos` | macOS AX, capture, input, browser, and daemon support |
+| `platform-windows` | Windows UIA, capture, input, overlay, and diagnostics |
+| `platform-linux` | Linux AT-SPI, X11/Wayland, capture, and input support |
+| `cua-driver-testkit` | Test-only helpers for spawning the daemon and parsing responses |
+| `cua-driver-uia` | Windows UIAccess worker |
+| `focus-monitor-win` | Windows focus sentinel used by UX guard tests |
+| `cursor-overlay` | Cursor overlay support |
+| `pip-preview` | Packaging preview helper |
+
+Platform crates are selected with `cfg(target_os)` from the main driver crate.
+Keep platform-specific behavior inside the matching `platform-<os>` crate.
+
+## Build
+
+```bash
+cargo build -p cua-driver
+cargo build -p cua-driver --release
+```
+
+The Python wrapper and install scripts expect the built binary at
+`target/<profile>/cua-driver` (or `cua-driver.exe` on Windows), unless
+`CUA_DRIVER_BINARY` is set.
+
+## Tests
+
+Default tests should be headless and safe for CI:
+
+```bash
+cargo test -p cua-driver-core
+cargo test -p cua-driver --test protocol_mcp_test
+```
+
+GUI and VM-backed tests are marked `#[ignore]` and require harness apps from
+`../test-harness/build/` plus a real interactive desktop session:
+
+```bash
+cargo test -p cua-driver --test harness_appkit_test -- --ignored --nocapture
+cargo test -p cua-driver --test harness_wpf_test -- --ignored --nocapture
+```
+
+See `crates/cua-driver/tests/README.md` for the test matrix.
+
+## Generated Outputs
+
+- `target/`: Cargo output, ignored.
+- `test-apps/harness-*/`: staged harness apps, ignored.
+- `test-apps/README.md`: explains how staged harness outputs are produced.
diff --git a/libs/cua-driver/rust/crates/cua-driver/tests/README.md b/libs/cua-driver/rust/crates/cua-driver/tests/README.md
new file mode 100644
index 0000000000..13fc3e2ea7
--- /dev/null
+++ b/libs/cua-driver/rust/crates/cua-driver/tests/README.md
@@ -0,0 +1,45 @@
+# cua-driver Rust integration tests
+
+Tests in this directory exercise the public driver interface. Headless protocol
+tests run by default; GUI and modality tests are marked `#[ignore]` because they
+need staged harness apps and an interactive desktop.
+
+## Naming
+
+| Prefix | Runs by default | Purpose |
+| --- | --- | --- |
+| `protocol_*_test.rs` | yes | MCP/CLI protocol and schema behavior |
+| `schema_*_test.rs` | yes | Generated schema consistency |
+| `harness_<toolkit>_test.rs` | no, `#[ignore]` | Toolkit-specific harness apps |
+| `modality_<area>[_<os>]_test.rs` | no, `#[ignore]` | Background input, capture, desktop scope |
+| `guard_*_test.rs` | usually ignored or self-skipping | UX guard and interactive desktop checks |
+
+## Harness Requirements
+
+Build harness apps before running ignored tests:
+
+```bash
+# macOS
+../../../../test-harness/build/macos.sh
+
+# Linux
+../../../../test-harness/build/linux.sh
+```
+
+```powershell
+# Windows
+..\..\..\..\test-harness\build\windows.ps1
+```
+
+Staged outputs are read from `../../test-apps/harness-<name>/`.
+
+## Running
+
+```bash
+cargo test -p cua-driver --test protocol_mcp_test
+cargo test -p cua-driver --test harness_appkit_test -- --ignored --nocapture
+cargo test -p cua-driver --test modality_desktop_scope_macos_test -- --ignored --nocapture
+```
+
+Windows Sandbox runs use `../../sandbox/run-tests-in-sandbox.ps1`, which builds
+the Windows harness apps and maps them into the sandbox.
diff --git a/libs/cua-driver/rust/crates/cua-driver/tests/guard_ux_test.rs b/libs/cua-driver/rust/crates/cua-driver/tests/guard_ux_test.rs
index d26e572c29..a03eb5ff98 100644
--- a/libs/cua-driver/rust/crates/cua-driver/tests/guard_ux_test.rs
+++ b/libs/cua-driver/rust/crates/cua-driver/tests/guard_ux_test.rs
@@ -1,6 +1,6 @@
//! UX-guard integration tests for Windows.
//!
-//! These mirror the macOS Python tests in libs/cua-driver/Tests/integration/:
+//! These mirror the Python e2e tests in libs/cua-driver/tests/e2e/:
//! - test_background_focus.py
//! - test_click_opens_new_window.py
//! - test_launch_app_visible.py
@@ -10,9 +10,9 @@
//! The agent must be able to click, type, and launch apps in background
//! windows WITHOUT stealing focus from the user's foreground window.
//!
-//! Background target: desktop-test-app-electron (prebuilt binary at
-//! test-apps/desktop-test-app-electron.0.1.0.exe). Notepad is used only
-//! as a secondary fallback check.
+//! Background target: the repo-local Electron harness staged at
+//! test-apps/harness-electron/CuaTestHarness.Electron.exe. Notepad is used
+//! only as a secondary fallback check.
//!
//! All tests:
//! 1. Launch focus-monitor-win (the "user's foreground window").
@@ -32,6 +32,28 @@ use cua_driver_testkit::{driver_binary, spawn_in_job, workspace_root, Driver, Mc
// ── focus-monitor + test-app fixtures ────────────────────────────────────────
+fn require_interactive_desktop(context: &str) -> bool {
+ if platform_windows::diagnostics::current_session_id() == Some(0) {
+ eprintln!(
+ "{context}: running in Windows Session 0; skipping GUI UX guard. \
+ Re-run from an interactive logon session (RDP/console/scheduled task in user session)."
+ );
+ return false;
+ }
+
+ match platform_windows::diagnostics::interactive_desktop_check() {
+ Ok(true) => true,
+ Ok(false) => {
+ eprintln!("{context}: no foreground interactive desktop; skipping GUI UX guard");
+ false
+ }
+ Err(e) => {
+ eprintln!("{context}: interactive desktop check failed ({e}); skipping GUI UX guard");
+ false
+ }
+ }
+}
+
fn focus_monitor_path() -> PathBuf {
workspace_root().join("target/debug/focus-monitor-win.exe")
}
@@ -39,44 +61,34 @@ fn focus_monitor_path() -> PathBuf {
fn test_app_path() -> PathBuf {
// In sandbox, sandbox-runner.ps1 copies the exe to %TEMP% to avoid the
// ShellExecuteW zone-security dialog that blocks on mapped-folder exes.
- if let Ok(p) = std::env::var("TEST_APP_EXE") {
+ if let Ok(p) = std::env::var("HARNESS_ELECTRON_EXE") {
let pb = PathBuf::from(p);
- if pb.exists() { return pb; }
+ if pb.exists() {
+ return pb;
+ }
}
- workspace_root().join("test-apps/desktop-test-app-electron.0.1.0.exe")
+ workspace_root().join("test-apps/harness-electron/CuaTestHarness.Electron.exe")
}
-/// Launch the electron test app in the background (tied to the driver's reaper)
-/// and return its pid. Waits up to 10s for the app's HTTP health endpoint to
-/// respond. Returns None if the binary doesn't exist or the app fails to start.
+/// Launch the Electron harness in the background (tied to the driver's reaper)
+/// and return its pid. Returns None if the binary doesn't exist or the app
+/// fails to start.
fn launch_test_app(driver: &mut McpDriver) -> Option<u32> {
let exe = test_app_path();
if !exe.exists() {
- eprintln!("desktop-test-app-electron not found at {exe:?} — skipping");
+ eprintln!("CuaTestHarness.Electron not found at {exe:?} - skipping");
return None;
}
let child = spawn_in_job(
- Command::new(&exe).stdout(Stdio::null()).stderr(Stdio::null()),
+ Command::new(&exe)
+ .stdout(Stdio::null())
+ .stderr(Stdio::null()),
)
.ok()?;
let pid = child.id();
driver.reaper().push(child);
- // Poll the HTTP health endpoint until it responds or timeout.
- // Allow extra time for cold-start in sandbox (no cached Electron DLLs).
- let deadline = std::time::Instant::now() + Duration::from_secs(20);
- loop {
- if std::time::Instant::now() > deadline {
- eprintln!("desktop-test-app-electron HTTP /health did not respond within 10s");
- return Some(pid);
- }
- if let Ok(stream) = std::net::TcpStream::connect("127.0.0.1:6769") {
- drop(stream);
- break;
- }
- std::thread::sleep(Duration::from_millis(200));
- }
- // Extra settle time for the window to appear.
- std::thread::sleep(Duration::from_millis(500));
+ // Cold-start in sandbox can take a few seconds.
+ std::thread::sleep(Duration::from_secs(3));
Some(pid)
}
@@ -94,16 +106,25 @@ fn read_losses(path: &std::path::Path) -> u32 {
.unwrap_or(0)
}
-fn focus_pid_file() -> PathBuf { std::env::temp_dir().join("focus_monitor_pid.txt") }
-fn focus_hwnd_file() -> PathBuf { std::env::temp_dir().join("focus_monitor_hwnd.txt") }
+fn focus_pid_file() -> PathBuf {
+ std::env::temp_dir().join("focus_monitor_pid.txt")
+}
+fn focus_hwnd_file() -> PathBuf {
+ std::env::temp_dir().join("focus_monitor_hwnd.txt")
+}
/// Launch focus-monitor-win and return (process, hwnd, pid).
/// Reads FOCUS_PID and FOCUS_HWND from temp files written by the monitor
/// (avoids blocking on the stdout pipe if the sandbox redirects I/O).
-fn launch_focus_monitor() -> (Child, u64, u32) {
+fn launch_focus_monitor() -> Option<(Child, u64, u32)> {
+ if !require_interactive_desktop("focus-monitor-win") {
+ return None;
+ }
+
let exe = focus_monitor_path();
if !exe.exists() {
- panic!("focus-monitor-win.exe not found at {exe:?} — run `cargo build` first");
+ eprintln!("focus-monitor-win.exe not built at {exe:?} — skipping");
+ return None;
}
// Reset all sentinel files so stale values are not mistaken for new ones.
let _ = std::fs::write(loss_file(), "0");
@@ -112,7 +133,9 @@ fn launch_focus_monitor() -> (Child, u64, u32) {
let _ = std::fs::remove_file(focus_hwnd_file());
let child = spawn_in_job(
- Command::new(&exe).stdout(Stdio::null()).stderr(Stdio::null()),
+ Command::new(&exe)
+ .stdout(Stdio::null())
+ .stderr(Stdio::null()),
)
.expect("spawn focus-monitor-win");
@@ -125,24 +148,32 @@ fn launch_focus_monitor() -> (Child, u64, u32) {
}
if pid_val == 0 {
pid_val = std::fs::read_to_string(focus_pid_file())
- .ok().and_then(|s| s.trim().parse().ok()).unwrap_or(0);
+ .ok()
+ .and_then(|s| s.trim().parse().ok())
+ .unwrap_or(0);
}
if hwnd_val == 0 {
hwnd_val = std::fs::read_to_string(focus_hwnd_file())
- .ok().and_then(|s| s.trim().parse().ok()).unwrap_or(0);
+ .ok()
+ .and_then(|s| s.trim().parse().ok())
+ .unwrap_or(0);
+ }
+ if pid_val != 0 && hwnd_val != 0 {
+ break;
}
- if pid_val != 0 && hwnd_val != 0 { break; }
std::thread::sleep(Duration::from_millis(100));
}
// Give the window time to become foreground.
std::thread::sleep(Duration::from_millis(400));
- (child, hwnd_val, pid_val)
+ Some((child, hwnd_val, pid_val))
}
/// Find the first on-screen window belonging to the given pid.
fn find_window_for_pid(driver: &mut McpDriver, pid: i64) -> Option<u64> {
- let resp = driver.call("list_windows",
- serde_json::json!({"pid": pid, "on_screen_only": true}));
+ let resp = driver.call(
+ "list_windows",
+ serde_json::json!({"pid": pid, "on_screen_only": true}),
+ );
resp.structured()["windows"]
.as_array()?
.iter()
@@ -171,39 +202,64 @@ fn test_background_click_and_type_no_focus_steal() {
//! Equivalent of macOS test_background_focus.py.
//!
//! 1. Launch FocusMonitorWin (simulates the user's active window).
- //! 2. Launch desktop-test-app-electron in the background.
+ //! 2. Launch the Electron harness in the background.
//! 3. Click inside the app and type text via cua-driver.
//! 4. Assert act_losses on FocusMonitorWin stayed at 0.
- if !driver_binary().exists() { eprintln!("Binary not found — skipping"); return; }
+ if !driver_binary().exists() {
+ eprintln!("Binary not found — skipping");
+ return;
+ }
- let (mut fm_proc, _fm_hwnd, _fm_pid) = launch_focus_monitor();
+ let Some((mut fm_proc, _fm_hwnd, _fm_pid)) = launch_focus_monitor() else {
+ return;
+ };
let losses_before = read_losses(&loss_file());
- let Some(mut driver) = McpDriver::spawn() else { fm_proc.kill().ok(); return; };
+ let Some(mut driver) = McpDriver::spawn() else {
+ fm_proc.kill().ok();
+ return;
+ };
let Some(app_pid) = launch_test_app(&mut driver) else {
eprintln!("test app not available — skipping");
- fm_proc.kill().ok(); return;
+ fm_proc.kill().ok();
+ return;
};
let Some(app_wid) = find_window_for_pid(&mut driver, app_pid as i64) else {
eprintln!("test app window not found — skipping");
- fm_proc.kill().ok(); return;
+ fm_proc.kill().ok();
+ return;
};
// Click inside the app (background, via PostMessage).
- let r = driver.call("click",
- serde_json::json!({"pid": app_pid, "window_id": app_wid, "x": 200.0, "y": 200.0}));
- assert!(r.raw["error"].is_null(), "Protocol error from click: {:?}", r.raw);
+ let r = driver.call(
+ "click",
+ serde_json::json!({"pid": app_pid, "window_id": app_wid, "x": 200.0, "y": 200.0}),
+ );
+ assert!(
+ r.raw["error"].is_null(),
+ "Protocol error from click: {:?}",
+ r.raw
+ );
// Type text into the app (background, via PostMessage).
- let r = driver.call("type_text",
- serde_json::json!({"pid": app_pid, "window_id": app_wid, "text": "ux-guard-test"}));
- assert!(r.raw["error"].is_null(), "Protocol error from type_text: {:?}", r.raw);
+ let r = driver.call(
+ "type_text",
+ serde_json::json!({"pid": app_pid, "window_id": app_wid, "text": "ux-guard-test"}),
+ );
+ assert!(
+ r.raw["error"].is_null(),
+ "Protocol error from type_text: {:?}",
+ r.raw
+ );
// ux_guard: FocusMonitorWin must not have lost activation.
- assert_ux_guard(losses_before, 0,
- "background click + type_text into desktop-test-app-electron");
+ assert_ux_guard(
+ losses_before,
+ 0,
+ "background click + type_text into CuaTestHarness.Electron",
+ );
fm_proc.kill().ok();
}
@@ -218,22 +274,33 @@ fn test_launch_app_no_focus_steal() {
//!
//! launch_app (path variant) must open a window without displacing FocusMonitorWin.
- if !driver_binary().exists() { return; }
+ if !driver_binary().exists() {
+ return;
+ }
let exe = test_app_path();
- if !exe.exists() { eprintln!("test app not available — skipping"); return; }
+ if !exe.exists() {
+ eprintln!("test app not available — skipping");
+ return;
+ }
- let (mut fm_proc, _fm_hwnd, _fm_pid) = launch_focus_monitor();
+ let Some((mut fm_proc, _fm_hwnd, _fm_pid)) = launch_focus_monitor() else {
+ return;
+ };
let losses_before = read_losses(&loss_file());
- let Some(mut driver) = McpDriver::spawn() else { fm_proc.kill().ok(); return; };
+ let Some(mut driver) = McpDriver::spawn() else {
+ fm_proc.kill().ok();
+ return;
+ };
// Launch the test app via cua-driver launch_app (full path, SW_SHOWNOACTIVATE).
let path_str = exe.to_string_lossy().into_owned();
let r = driver.call("launch_app", serde_json::json!({"path": path_str}));
if r.is_error() {
eprintln!("launch_app failed — skipping: {:?}", r.raw);
- fm_proc.kill().ok(); return;
+ fm_proc.kill().ok();
+ return;
}
// Wait for the app window to appear (Electron startup ~2-3s).
@@ -243,7 +310,13 @@ fn test_launch_app_no_focus_steal() {
let r2 = driver.call("list_apps", serde_json::json!({}));
if let Some(procs) = r2.structured()["processes"].as_array() {
if let Some(p) = procs.iter().find(|p| {
- p["name"].as_str().map(|n| n.to_lowercase().contains("desktop-test-app")).unwrap_or(false)
+ p["name"]
+ .as_str()
+ .map(|n| {
+ let n = n.to_lowercase();
+ n.contains("cuatestharness.electron") || n.contains("electron")
+ })
+ .unwrap_or(false)
}) {
app_pid = p["pid"].as_i64();
break;
@@ -251,16 +324,21 @@ fn test_launch_app_no_focus_steal() {
}
}
if app_pid.is_none() {
- eprintln!("desktop-test-app not found in process list after launch_app — skipping");
- fm_proc.kill().ok(); return;
+ eprintln!("CuaTestHarness.Electron not found in process list after launch_app - skipping");
+ fm_proc.kill().ok();
+ return;
}
// ux_guard: FocusMonitorWin must not have lost activation.
- assert_ux_guard(losses_before, 0, "launch_app desktop-test-app-electron");
+ assert_ux_guard(losses_before, 0, "launch_app CuaTestHarness.Electron");
// Kill the launched app by exe name.
- Command::new("taskkill").args(["/F", "/T", "/IM", "desktop-test-app-electron.0.1.0.exe"])
- .stdout(Stdio::null()).stderr(Stdio::null()).spawn().ok();
+ Command::new("taskkill")
+ .args(["/F", "/T", "/IM", "CuaTestHarness.Electron.exe"])
+ .stdout(Stdio::null())
+ .stderr(Stdio::null())
+ .spawn()
+ .ok();
fm_proc.kill().ok();
}
@@ -272,32 +350,51 @@ fn test_launch_app_no_focus_steal() {
fn test_background_hotkey_no_focus_steal() {
//! Equivalent of macOS test_background_menu_shortcut.py.
//!
- //! Send Ctrl+A to a background desktop-test-app-electron window.
+ //! Send Ctrl+A to a background Electron harness window.
//! FocusMonitorWin must never lose activation.
- if !driver_binary().exists() { return; }
+ if !driver_binary().exists() {
+ return;
+ }
- let (mut fm_proc, _fm_hwnd, _fm_pid) = launch_focus_monitor();
+ let Some((mut fm_proc, _fm_hwnd, _fm_pid)) = launch_focus_monitor() else {
+ return;
+ };
let losses_before = read_losses(&loss_file());
- let Some(mut driver) = McpDriver::spawn() else { fm_proc.kill().ok(); return; };
+ let Some(mut driver) = McpDriver::spawn() else {
+ fm_proc.kill().ok();
+ return;
+ };
let Some(app_pid) = launch_test_app(&mut driver) else {
eprintln!("test app not available — skipping");
- fm_proc.kill().ok(); return;
+ fm_proc.kill().ok();
+ return;
};
let Some(app_wid) = find_window_for_pid(&mut driver, app_pid as i64) else {
eprintln!("test app window not found — skipping");
- fm_proc.kill().ok(); return;
+ fm_proc.kill().ok();
+ return;
};
// Send Ctrl+A hotkey to background app (PostMessage, no focus steal).
- let r = driver.call("hotkey",
- serde_json::json!({"pid": app_pid, "window_id": app_wid, "keys": ["ctrl", "a"]}));
- assert!(r.raw["error"].is_null(), "Protocol error from hotkey: {:?}", r.raw);
+ let r = driver.call(
+ "hotkey",
+ serde_json::json!({"pid": app_pid, "window_id": app_wid, "keys": ["ctrl", "a"]}),
+ );
+ assert!(
+ r.raw["error"].is_null(),
+ "Protocol error from hotkey: {:?}",
+ r.raw
+ );
// ux_guard: FocusMonitorWin must not have lost activation.
- assert_ux_guard(losses_before, 0, "background hotkey ctrl+a to desktop-test-app-electron");
+ assert_ux_guard(
+ losses_before,
+ 0,
+ "background hotkey ctrl+a to CuaTestHarness.Electron",
+ );
fm_proc.kill().ok();
}
@@ -311,39 +408,57 @@ fn test_background_click_opens_new_window_focus_preserved() {
//! Equivalent of macOS test_click_opens_new_window.py.
//!
//! 1. FocusMonitorWin is foreground.
- //! 2. Click a link in desktop-test-app-electron — a new window/tab may appear.
+ //! 2. Click in CuaTestHarness.Electron - a new window/tab may appear.
//! 3. FocusMonitorWin must remain active throughout (UX guard).
//!
//! Verifies PostMessage doesn't inadvertently activate any new window that
//! appears as a side-effect of the click.
- if !driver_binary().exists() { return; }
+ if !driver_binary().exists() {
+ return;
+ }
- let (mut fm_proc, _fm_hwnd, _fm_pid) = launch_focus_monitor();
+ let Some((mut fm_proc, _fm_hwnd, _fm_pid)) = launch_focus_monitor() else {
+ return;
+ };
let losses_before = read_losses(&loss_file());
- let Some(mut driver) = McpDriver::spawn() else { fm_proc.kill().ok(); return; };
+ let Some(mut driver) = McpDriver::spawn() else {
+ fm_proc.kill().ok();
+ return;
+ };
let Some(app_pid) = launch_test_app(&mut driver) else {
eprintln!("test app not available — skipping");
- fm_proc.kill().ok(); return;
+ fm_proc.kill().ok();
+ return;
};
let Some(app_wid) = find_window_for_pid(&mut driver, app_pid as i64) else {
eprintln!("test app window not found — skipping");
- fm_proc.kill().ok(); return;
+ fm_proc.kill().ok();
+ return;
};
// Click somewhere in the app content area (may trigger navigation/new window).
- let r = driver.call("click",
- serde_json::json!({"pid": app_pid, "window_id": app_wid, "x": 400.0, "y": 350.0}));
- assert!(r.raw["error"].is_null(), "Protocol error from click: {:?}", r.raw);
+ let r = driver.call(
+ "click",
+ serde_json::json!({"pid": app_pid, "window_id": app_wid, "x": 400.0, "y": 350.0}),
+ );
+ assert!(
+ r.raw["error"].is_null(),
+ "Protocol error from click: {:?}",
+ r.raw
+ );
// Brief wait for any side-effect windows to appear.
std::thread::sleep(Duration::from_millis(500));
// ux_guard: FocusMonitorWin must not have lost activation.
- assert_ux_guard(losses_before, 0,
- "background click in desktop-test-app-electron (may open new window)");
+ assert_ux_guard(
+ losses_before,
+ 0,
+ "background click in CuaTestHarness.Electron (may open new window)",
+ );
// Verify FocusMonitorWin is still alive.
assert!(
@@ -362,28 +477,45 @@ fn test_background_click_opens_new_window_focus_preserved() {
fn test_background_screenshot_no_focus_steal() {
//! PrintWindow captures a background window without activating it.
- if !driver_binary().exists() { return; }
+ if !driver_binary().exists() {
+ return;
+ }
- let (mut fm_proc, _fm_hwnd, _fm_pid) = launch_focus_monitor();
+ let Some((mut fm_proc, _fm_hwnd, _fm_pid)) = launch_focus_monitor() else {
+ return;
+ };
let losses_before = read_losses(&loss_file());
- let Some(mut driver) = McpDriver::spawn() else { fm_proc.kill().ok(); return; };
+ let Some(mut driver) = McpDriver::spawn() else {
+ fm_proc.kill().ok();
+ return;
+ };
let Some(app_pid) = launch_test_app(&mut driver) else {
eprintln!("test app not available — skipping");
- fm_proc.kill().ok(); return;
+ fm_proc.kill().ok();
+ return;
};
let Some(app_wid) = find_window_for_pid(&mut driver, app_pid as i64) else {
eprintln!("test app window not found — skipping");
- fm_proc.kill().ok(); return;
+ fm_proc.kill().ok();
+ return;
};
// Screenshot via PrintWindow — must not activate the window.
let r = driver.call("screenshot", serde_json::json!({"window_id": app_wid}));
- assert!(r.raw["error"].is_null(), "Protocol error from screenshot: {:?}", r.raw);
+ assert!(
+ r.raw["error"].is_null(),
+ "Protocol error from screenshot: {:?}",
+ r.raw
+ );
// ux_guard
- assert_ux_guard(losses_before, 0, "screenshot of background desktop-test-app-electron");
+ assert_ux_guard(
+ losses_before,
+ 0,
+ "screenshot of background CuaTestHarness.Electron",
+ );
fm_proc.kill().ok();
}
@@ -404,17 +536,32 @@ fn test_agent_cursor_visible_on_screen() {
//! 4. Decode the PNG and sample a 40×40 px patch centred on the cursor.
//! 5. Assert the patch contains cursor-like pixels (bright or saturated).
- if !driver_binary().exists() { eprintln!("Binary not found — skipping"); return; }
+ if !driver_binary().exists() {
+ eprintln!("Binary not found — skipping");
+ return;
+ }
+ if !require_interactive_desktop("agent cursor visibility") {
+ return;
+ }
- let Some(mut driver) = McpDriver::spawn() else { return };
+ let Some(mut driver) = McpDriver::spawn() else {
+ return;
+ };
// Safe centre-ish position on primary monitor.
let cx = 640.0_f64;
let cy = 400.0_f64;
// Enable cursor overlay and glide to target.
- let r = driver.call("set_agent_cursor_enabled", serde_json::json!({"enabled": true}));
- assert!(r.raw["error"].is_null(), "set_agent_cursor_enabled failed: {:?}", r.raw);
+ let r = driver.call(
+ "set_agent_cursor_enabled",
+ serde_json::json!({"enabled": true}),
+ );
+ assert!(
+ r.raw["error"].is_null(),
+ "set_agent_cursor_enabled failed: {:?}",
+ r.raw
+ );
let r = driver.call("move_cursor", serde_json::json!({"x": cx, "y": cy}));
assert!(r.raw["error"].is_null(), "move_cursor failed: {:?}", r.raw);
@@ -444,7 +591,9 @@ fn test_agent_cursor_visible_on_screen() {
for py in y0..y1 {
for px in x0..x1 {
let [r, g, b, a] = rgba.get_pixel(px, py).0;
- if a < 10 { continue; }
+ if a < 10 {
+ continue;
+ }
let brightness = r as u32 + g as u32 + b as u32;
let saturation = r.max(g).max(b) as u32 - r.min(g).min(b) as u32;
// Accept bright-white stroke pixels OR coloured gradient pixels.
diff --git a/libs/cua-driver/rust/crates/cua-driver/tests/modality_input_e2e_test.rs b/libs/cua-driver/rust/crates/cua-driver/tests/modality_input_e2e_test.rs
index 511522449d..9efc8e7ef9 100644
--- a/libs/cua-driver/rust/crates/cua-driver/tests/modality_input_e2e_test.rs
+++ b/libs/cua-driver/rust/crates/cua-driver/tests/modality_input_e2e_test.rs
@@ -18,15 +18,15 @@
//! held ports. Broker-spawned window pids (packaged apps / Electron) are
//! tree-killed via `reaper().track_pid`.
//!
-//! Targets (trycua test apps, auto-downloaded to %TEMP% if not provided):
-//! - Electron (Chromium content) — `trycua/desktop-test-app-electron`.
-//! - Tauri (WebView2 content) — `trycua/desktop-test-app`.
+//! Targets (repo-local harness apps staged by `test-harness/build/windows.ps1`):
+//! - Electron (Chromium content) — `CuaTestHarness.Electron.exe`.
+//! - Tauri (WebView2 content) — `CuaTestHarness.Tauri.exe`.
//! - Win32 baseline — `notepad.exe`.
-//! Override with `CUA_ELECTRON_EXE` / `CUA_TAURI_EXE`, or drop the exe in
-//! `test-apps/`. Auto-download uses `curl.exe`.
+//! Override with `HARNESS_ELECTRON_EXE` / `HARNESS_TAURI_EXE`
+//! (or legacy `CUA_ELECTRON_EXE` / `CUA_TAURI_EXE`).
//!
//! All tests are `#[ignore]` (GUI, real desktop session). Run explicitly,
-//! serially (apps bind a fixed HTTP port, so never in parallel):
+//! serially (GUI focus oracle, so never in parallel):
//! cargo test -p cua-driver --test e2e_windows_bg_input_test -- --ignored --nocapture --test-threads=1
#![cfg(target_os = "windows")]
@@ -39,10 +39,7 @@ use std::time::{Duration, Instant};
use cua_driver_testkit::{ax, workspace_root, Driver, McpDriver};
-const ELECTRON_URL: &str = "https://github.com/trycua/desktop-test-app-electron/releases/download/v0.1.0/desktop-test-app-electron.0.1.0.exe";
-const TAURI_URL: &str = "https://github.com/trycua/desktop-test-app/releases/download/v0.2.2/desktop-test-app-windows-x86_64.exe";
-
-// ── paths / downloads ─────────────────────────────────────────────────────────
+// ── paths ────────────────────────────────────────────────────────────────────
fn focus_monitor_binary() -> PathBuf {
workspace_root().join("target/debug/focus-monitor-win.exe")
@@ -60,56 +57,30 @@ fn kill_prior_by_name(exe: &Path) {
}
}
-fn curl_download(url: &str, dst: &Path) -> bool {
- eprintln!("[e2e] downloading {url}\n -> {dst:?}");
- let ok = Command::new("curl.exe")
- .args(["-L", "--fail", "--silent", "--show-error", "-o"])
- .arg(dst)
- .arg(url)
- .status()
- .map(|s| s.success())
- .unwrap_or(false);
- ok && fs::metadata(dst).map(|m| m.len() > 4096).unwrap_or(false)
-}
-fn resolve_or_download(env_var: &str, prefix: &str, url: &str, tmp_name: &str) -> Option<PathBuf> {
- if let Ok(p) = std::env::var(env_var) {
- let pb = PathBuf::from(p);
- if pb.exists() {
- return Some(pb);
- }
- }
- if let Ok(entries) = fs::read_dir(workspace_root().join("test-apps")) {
- for e in entries.flatten() {
- let name = e.file_name().to_string_lossy().to_lowercase();
- if name.starts_with(prefix) && name.ends_with(".exe") && !name.contains("setup") {
- return Some(e.path());
+fn resolve_harness(primary_env: &str, legacy_env: &str, rel_path: &str) -> Option<PathBuf> {
+ for env_var in [primary_env, legacy_env] {
+ if let Ok(p) = std::env::var(env_var) {
+ let pb = PathBuf::from(p);
+ if pb.exists() {
+ return Some(pb);
}
}
}
- let dst = std::env::temp_dir().join(tmp_name);
- if dst.exists() && fs::metadata(&dst).map(|m| m.len() > 4096).unwrap_or(false) {
- return Some(dst);
- }
- if curl_download(url, &dst) {
- Some(dst)
- } else {
- None
- }
+ let pb = workspace_root().join(rel_path);
+ pb.exists().then_some(pb)
}
fn electron_exe() -> Option<PathBuf> {
- resolve_or_download(
+ resolve_harness(
+ "HARNESS_ELECTRON_EXE",
"CUA_ELECTRON_EXE",
- "desktop-test-app-electron",
- ELECTRON_URL,
- "desktop-test-app-electron.0.1.0.exe",
+ "test-apps/harness-electron/CuaTestHarness.Electron.exe",
)
}
fn tauri_exe() -> Option<PathBuf> {
- resolve_or_download(
+ resolve_harness(
+ "HARNESS_TAURI_EXE",
"CUA_TAURI_EXE",
- "desktop-test-app-windows",
- TAURI_URL,
- "desktop-test-app-windows-x86_64.exe",
+ "test-apps/harness-tauri/CuaTestHarness.Tauri.exe",
)
}
@@ -126,7 +97,10 @@ fn focus_hwnd_file() -> PathBuf {
std::env::temp_dir().join("focus_monitor_hwnd.txt")
}
fn read_count(p: &Path) -> u32 {
- fs::read_to_string(p).ok().and_then(|s| s.trim().parse::<u32>().ok()).unwrap_or(0)
+ fs::read_to_string(p)
+ .ok()
+ .and_then(|s| s.trim().parse::<u32>().ok())
+ .unwrap_or(0)
}
fn window_ids(driver: &mut McpDriver) -> HashSet<u64> {
@@ -174,7 +148,11 @@ fn setup(target_exe: &Path, _title_hint: &str) -> Option<E2eFixture> {
let before = window_ids(&mut driver);
if driver
.reaper()
- .spawn(Command::new(target_exe).stdout(Stdio::null()).stderr(Stdio::null()))
+ .spawn(
+ Command::new(target_exe)
+ .stdout(Stdio::null())
+ .stderr(Stdio::null()),
+ )
.is_err()
{
eprintln!("[e2e] target spawn failed"); // driver drops → kills itself
@@ -189,7 +167,9 @@ fn setup(target_exe: &Path, _title_hint: &str) -> Option<E2eFixture> {
let r = driver.call("list_windows", serde_json::json!({}));
if let Some(arr) = r.structured()["windows"].as_array() {
for w in arr {
- let Some(wid) = w["window_id"].as_u64() else { continue };
+ let Some(wid) = w["window_id"].as_u64() else {
+ continue;
+ };
let pid = w["pid"].as_u64().unwrap_or(0) as u32;
let title = w["title"].as_str().unwrap_or("");
if !before.contains(&wid) && !title.is_empty() && pid != 0 {
@@ -218,7 +198,11 @@ fn setup(target_exe: &Path, _title_hint: &str) -> Option<E2eFixture> {
// 4. Sentinel grabs foreground; app drops to z+1 (the background target).
if driver
.reaper()
- .spawn(Command::new(&fm_bin).stdout(Stdio::null()).stderr(Stdio::null()))
+ .spawn(
+ Command::new(&fm_bin)
+ .stdout(Stdio::null())
+ .stderr(Stdio::null()),
+ )
.is_err()
{
eprintln!("[e2e] sentinel spawn failed — skipping");
@@ -340,7 +324,9 @@ fn webview_click_case(label: &str, exe: PathBuf) {
let mut needs_foreground = false;
assert_target_stays_background(label, pid, || {
let args = match elem {
- Some(idx) => serde_json::json!({"pid": pid as i64, "window_id": wid, "element_index": idx}),
+ Some(idx) => {
+ serde_json::json!({"pid": pid as i64, "window_id": wid, "element_index": idx})
+ }
None => serde_json::json!({"pid": pid as i64, "window_id": wid, "x": 200, "y": 200}),
};
let last = fx.driver.call("click", args);
@@ -358,7 +344,9 @@ fn webview_click_case(label: &str, exe: PathBuf) {
// only — it does NOT capture accessibility-driven UIA Invoke, so a `[]`
// there is expected for the UIA path. The hard invariant is no-foreground-steal.
std::thread::sleep(Duration::from_millis(300));
- let reg = registered_since_reset().map(|b| b.to_string()).unwrap_or_else(|| "n/a".into());
+ let reg = registered_since_reset()
+ .map(|b| b.to_string())
+ .unwrap_or_else(|| "n/a".into());
println!("{label}: delivered={delivered:?} pointer-events-logged={reg}");
}
diff --git a/libs/cua-driver/rust/sandbox/run-tests-in-sandbox.ps1 b/libs/cua-driver/rust/sandbox/run-tests-in-sandbox.ps1
index 2299ab2d4a..d532dfef9a 100644
--- a/libs/cua-driver/rust/sandbox/run-tests-in-sandbox.ps1
+++ b/libs/cua-driver/rust/sandbox/run-tests-in-sandbox.ps1
@@ -53,18 +53,6 @@ if ($running) {
Write-Host "[OK] Existing sandbox closed. Proceeding." -ForegroundColor Green
}
-# ── 0. Download test-app binaries if missing ─────────────────────────────────
-$testAppExe = "$wsRoot\test-apps\desktop-test-app-electron.0.1.0.exe"
-if (-not (Test-Path $testAppExe)) {
- Write-Host "`n[FETCH] Downloading desktop-test-app-electron..." -ForegroundColor Yellow
- New-Item -ItemType Directory -Force "$wsRoot\test-apps" | Out-Null
- $url = "https://github.com/trycua/desktop-test-app-electron/releases/download/v0.1.0/desktop-test-app-electron.0.1.0.exe"
- Invoke-WebRequest -Uri $url -OutFile $testAppExe -UseBasicParsing
- Write-Host " Downloaded: $testAppExe" -ForegroundColor Green
-} else {
- Write-Host "`n[FETCH] test-app already present: $testAppExe" -ForegroundColor Green
-}
-
# ── 1. Build ──────────────────────────────────────────────────────────────────
Push-Location $wsRoot
try {
@@ -97,10 +85,10 @@ try {
if ($LASTEXITCODE -ne 0) { throw "cargo test --no-run (harness_bg_modality_test) failed" }
} finally { Pop-Location }
-# ── 1.5. Build the .NET test-harness if dotnet is on PATH ────────────────────
-# The harness produces test-apps/harness-{wpf,winui3}/ which the sandbox
+# ── 1.5. Build the test-harness if dependencies are on PATH ──────────────────
+# The harness produces test-apps/harness-*/ which the sandbox
# runner picks up via the existing mapped-folder route. Skip silently if
-# dotnet isn't available — the smoke tests degrade to "skipped" inside
+# a dependency isn't available — the smoke tests degrade to "skipped" inside
# the sandbox rather than failing the whole run.
if (Get-Command dotnet -ErrorAction SilentlyContinue) {
$harnessBuild = Join-Path $wsRoot "..\test-harness\build\windows.ps1"
diff --git a/libs/cua-driver/rust/sandbox/sandbox-runner.ps1 b/libs/cua-driver/rust/sandbox/sandbox-runner.ps1
index 08b15db307..805452a948 100644
--- a/libs/cua-driver/rust/sandbox/sandbox-runner.ps1
+++ b/libs/cua-driver/rust/sandbox/sandbox-runner.ps1
@@ -18,19 +18,6 @@ Log "sandbox-runner.ps1 starting"
$env:CARGO_MANIFEST_DIR = "C:\cua-driver-rs\crates\cua-driver"
Log "CARGO_MANIFEST_DIR = $env:CARGO_MANIFEST_DIR"
-# ── copy test-app exe to %TEMP% so ShellExecuteW doesn't hit a zone dialog ──
-# Files on Sandbox-mapped folders are Zone 3 (Internet); ShellExecuteW blocks
-# on the "Do you want to run this file?" dialog. A local %TEMP% copy is Zone 1.
-$testAppSrc = "C:\cua-driver-rs\test-apps\desktop-test-app-electron.0.1.0.exe"
-$testAppDst = "$env:TEMP\desktop-test-app-electron.0.1.0.exe"
-if (Test-Path $testAppSrc) {
- Copy-Item $testAppSrc $testAppDst -Force
- $env:TEST_APP_EXE = $testAppDst
- Log "test-app copied to $testAppDst (TEST_APP_EXE=$testAppDst)"
-} else {
- Log "WARNING: test-app not found at $testAppSrc"
-}
-
$driverExe = "C:\cua-driver-rs\target\debug\cua-driver.exe"
if (-not (Test-Path $driverExe)) {
Log "ERROR: $driverExe not found"
@@ -57,7 +44,8 @@ $harnessRoots = @(
@{ Src = "C:\cua-driver-rs\test-apps\harness-wpf"; EnvVar = "HARNESS_WPF_EXE"; Exe = "CuaTestHarness.Wpf.exe" },
@{ Src = "C:\cua-driver-rs\test-apps\harness-winui3"; EnvVar = "HARNESS_WINUI3_EXE"; Exe = "CuaTestHarness.WinUI3.exe" },
@{ Src = "C:\cua-driver-rs\test-apps\harness-webview"; EnvVar = "HARNESS_WEBVIEW_EXE"; Exe = "CuaTestHarness.WebView.exe" },
- @{ Src = "C:\cua-driver-rs\test-apps\harness-electron"; EnvVar = "HARNESS_ELECTRON_EXE"; Exe = "CuaTestHarness.Electron.exe" }
+ @{ Src = "C:\cua-driver-rs\test-apps\harness-electron"; EnvVar = "HARNESS_ELECTRON_EXE"; Exe = "CuaTestHarness.Electron.exe" },
+ @{ Src = "C:\cua-driver-rs\test-apps\harness-tauri"; EnvVar = "HARNESS_TAURI_EXE"; Exe = "CuaTestHarness.Tauri.exe" }
)
foreach ($h in $harnessRoots) {
if (-not (Test-Path $h.Src)) {
diff --git a/libs/cua-driver/rust/test-apps/.gitignore b/libs/cua-driver/rust/test-apps/.gitignore
index e3ba2f39e6..56b007e740 100644
--- a/libs/cua-driver/rust/test-apps/.gitignore
+++ b/libs/cua-driver/rust/test-apps/.gitignore
@@ -15,3 +15,4 @@ harness-wpf/
harness-winui3/
harness-webview/
harness-electron/
+harness-tauri/
diff --git a/libs/cua-driver/rust/test-apps/README.md b/libs/cua-driver/rust/test-apps/README.md
index 6b2419a5ad..ff1e4f8d13 100644
--- a/libs/cua-driver/rust/test-apps/README.md
+++ b/libs/cua-driver/rust/test-apps/README.md
@@ -1,15 +1,30 @@
# test-apps
-Pre-built test application binaries used by `ux_guard_test`.
+Local staging directory for cua-driver test harness applications.
-## Contents
+Build scripts under `../test-harness/build/` write `harness-<name>/`
+directories here. The staged outputs are intentionally git-ignored; rebuild
+them from source instead of committing binaries.
-- `desktop-test-app-electron.0.1.0.exe` — Electron test app from
- [trycua/desktop-test-app-electron v0.1.0](https://github.com/trycua/desktop-test-app-electron/releases/tag/v0.1.0).
- **Not committed to git** (76 MB). Downloaded automatically by
- `sandbox\run-tests-in-sandbox.ps1` on first run.
+Common outputs:
-## Updating
+- `harness-wpf/`
+- `harness-winui3/`
+- `harness-webview/`
+- `harness-electron/`
+- `harness-tauri/`
+- `harness-appkit/`
+- `harness-swiftui/`
+- `harness-wkwebview/`
+- `harness-gtk3/`
-To update to a new version, edit the download URL in
-`sandbox\run-tests-in-sandbox.ps1` and delete the old binary.
+Run the host build script for your platform:
+
+```bash
+../../test-harness/build/macos.sh
+../../test-harness/build/linux.sh
+```
+
+```powershell
+..\..\test-harness\build\windows.ps1
+```
diff --git a/libs/cua-driver/rust/tests/integration/v2/assets/tauri/desktop-test-app-macos-arm64 b/libs/cua-driver/rust/tests/integration/v2/assets/tauri/desktop-test-app-macos-arm64
deleted file mode 100755
index 412f68af9a..0000000000
Binary files a/libs/cua-driver/rust/tests/integration/v2/assets/tauri/desktop-test-app-macos-arm64 and /dev/null differ
diff --git a/libs/cua-driver/rust/tests/integration/v2/assets/test_page.html b/libs/cua-driver/rust/tests/integration/v2/assets/test_page.html
deleted file mode 120000
index 57fd293961..0000000000
--- a/libs/cua-driver/rust/tests/integration/v2/assets/test_page.html
+++ /dev/null
@@ -1 +0,0 @@
-../../../../../cua-driver-fixtures/test_page.html
\ No newline at end of file
diff --git a/libs/cua-driver/rust/tests/integration/v2/test_tauri.py b/libs/cua-driver/rust/tests/integration/v2/test_tauri.py
deleted file mode 100644
index e1bb465f56..0000000000
--- a/libs/cua-driver/rust/tests/integration/v2/test_tauri.py
+++ /dev/null
@@ -1,155 +0,0 @@
-"""Tauri desktop-test-app background automation tests — v2 harness.
-
-The Tauri test app (v0.2.2) renders a 4-panel WKWebView UI and exposes
-an HTTP API on port 6769 that logs all mouse/keyboard/clipboard events.
-
-Primary assertion: AX tree state changes.
-Cross-check: HTTP event log.
-
-Run: python3 -m pytest test_tauri.py -v
-"""
-
-from __future__ import annotations
-
-import subprocess
-import sys
-import time
-import os
-
-import pytest
-
-try:
- import requests as _requests
- _REQUESTS_AVAILABLE = True
-except ImportError:
- _REQUESTS_AVAILABLE = False
-
-_HERE = os.path.dirname(os.path.abspath(__file__))
-sys.path.insert(0, _HERE)
-from harness.driver import Driver
-from harness import tree as Tree
-
-_TAURI_API = "http://localhost:6769"
-
-
-def _http_events(base_url: str = _TAURI_API) -> list[dict]:
- if not _REQUESTS_AVAILABLE:
- return []
- try:
- resp = _requests.get(f"{base_url}/events", timeout=2)
- return resp.json() if resp.ok else []
- except Exception:
- return []
-
-
-def _http_reset(base_url: str = _TAURI_API) -> None:
- if not _REQUESTS_AVAILABLE:
- return
- try:
- _requests.post(f"{base_url}/reset", timeout=2)
- except Exception:
- pass
-
-
[email protected](autouse=True)
-def _reactivate_focus(focus_monitor):
- _, pid = focus_monitor
- subprocess.run(
- ["osascript", "-e",
- f'tell application "System Events" to set frontmost of (first process whose unix id is {pid}) to true'],
- check=False,
- )
- time.sleep(0.4)
-
-
-# ── tests ─────────────────────────────────────────────────────────────────────
-
-class TestTauriButton:
- def test_button_click(self, driver, tauri_app, ux_guard):
- _, pid, base_url = tauri_app
- if pid == 0:
- pytest.skip("Tauri app pid not found")
-
- _http_reset(base_url)
- wid = driver.find_window(pid)
-
- before = driver.get_window_state(pid, wid, query="button")
- idx = before.find_element("click")
- if idx is None:
- idx = Tree.find(before.tree, role="AXButton")
- assert idx is not None, f"Button not found:\n{before.tree}"
-
- driver.click(pid, wid, element_index=idx)
- time.sleep(0.5)
-
- after = driver.get_window_state(pid, wid)
- # Verify some state change happened (counter or AX value)
- changed = after.tree != before.tree
- print(f"\n tauri tree changed: {changed}")
-
- # Cross-check: HTTP events should contain a mouse event
- events = _http_events(base_url)
- print(f" tauri events: {len(events)}")
- # Either AX tree changed or HTTP logged a mouse event
- assert changed or any(
- e.get("type") in ("mouse_down", "click", "mousedown") for e in events
- ), f"No observable change after button click:\n{after.tree}"
-
-
-class TestTauriTextInput:
- def test_type_text(self, driver, tauri_app, ux_guard):
- _, pid, base_url = tauri_app
- if pid == 0:
- pytest.skip("Tauri app pid not found")
-
- _http_reset(base_url)
- wid = driver.find_window(pid)
-
- before = driver.get_window_state(pid, wid)
- idx = before.find_text_field(skip_url_bar=False)
- if idx is None:
- idx = Tree.find(before.tree, role="AXTextField")
- if idx is None:
- idx = Tree.find(before.tree, role="AXTextArea")
- assert idx is not None, f"Text input not found:\n{before.tree}"
-
- driver.click(pid, wid, element_index=idx)
- time.sleep(0.3)
- driver.hotkey(pid, ["cmd", "a"])
- time.sleep(0.1)
- driver.type_text_chars(pid, "tauri test")
- time.sleep(0.6)
-
- after = driver.get_window_state(pid, wid)
- text_in_tree = after.has_text("tauri test")
- print(f"\n text in AX tree: {text_in_tree}")
-
- events = _http_events(base_url)
- key_events = [e for e in events if e.get("type") in ("key_down", "keydown", "input")]
- print(f" key events: {len(key_events)}")
-
- assert text_in_tree or len(key_events) >= len("tauri test"), (
- f"Text not found in AX tree and insufficient key events:\n{after.tree}"
- )
-
-
-class TestTauriNoFocusSteal:
- def test_multiple_actions_no_focus_steal(self, driver, tauri_app, ux_guard):
- _, pid, base_url = tauri_app
- if pid == 0:
- pytest.skip("Tauri app pid not found")
-
- wid = driver.find_window(pid)
-
- # Take 3 actions in sequence — UX guard checks no focus steal
- before = driver.get_window_state(pid, wid)
- idx = Tree.find(before.tree, role="AXButton")
- if idx is not None:
- driver.click(pid, wid, element_index=idx)
- time.sleep(0.3)
-
- driver.press_key(pid, "tab")
- time.sleep(0.2)
- driver.press_key(pid, "tab")
- time.sleep(0.2)
- # ux_guard teardown will assert_clean()
diff --git a/libs/cua-driver/scripts/README.md b/libs/cua-driver/scripts/README.md
new file mode 100644
index 0000000000..f995a109b7
--- /dev/null
+++ b/libs/cua-driver/scripts/README.md
@@ -0,0 +1,16 @@
+# cua-driver scripts
+
+Install, uninstall, local-build, and VM sync helpers for cua-driver.
+
+| Script | Purpose |
+| --- | --- |
+| `install.sh` / `install.ps1` | Install released cua-driver binaries |
+| `install-local.sh` / `install-local.ps1` | Build and install from this checkout |
+| `uninstall.sh` / `uninstall.ps1` | Remove installed driver artifacts |
+| `_install-common.sh` / `_install-common.psm1` | Shared install helper logic |
+| `_install-rust.sh` / `_install-local-rust.sh` | Rust build/install internals |
+| `sync-vm-worktree.sh` | Sync this checkout to verification VMs and pull artifacts back |
+| `post-install-hints.txt` | User-facing hints printed by install scripts |
+
+Keep source commits host-owned. Verification machines should sync from this
+checkout and return artifacts, not push code.
diff --git a/libs/cua-driver/scripts/sync-vm-worktree.sh b/libs/cua-driver/scripts/sync-vm-worktree.sh
new file mode 100755
index 0000000000..c110abab30
--- /dev/null
+++ b/libs/cua-driver/scripts/sync-vm-worktree.sh
@@ -0,0 +1,164 @@
+#!/usr/bin/env bash
+# Sync this host checkout to verification VMs and pull logs/artifacts back.
+#
+# Source changes should flow from this host to VMs. The guarded pull-code mode
+# exists only for recovering intentional VM-side edits.
+set -euo pipefail
+
+usage() {
+ cat <<'USAGE'
+Usage:
+ sync-vm-worktree.sh push <ssh-target> [remote-dir]
+ sync-vm-worktree.sh pull-artifacts <ssh-target> [remote-dir]
+ ALLOW_PULL_CODE=1 sync-vm-worktree.sh pull-code <ssh-target> [remote-dir]
+
+Modes:
+ push
+ Sync this host checkout to a VM. This is the normal way to update Linux
+ and Windows verification machines.
+
+ pull-artifacts
+ Pull VM logs/artifacts from <remote-dir>/vm-out into this host checkout.
+
+ pull-code
+ Pull source changes made on a VM back to this host checkout. This is
+ intentionally gated by ALLOW_PULL_CODE=1 because this host is the only
+ checkout that should commit or push.
+
+Environment:
+ REMOTE_ARTIFACT_DIR Remote artifact directory, relative to remote-dir.
+ Default: vm-out
+ RSYNC_SSH SSH command for rsync. Default: ssh
+ SYNC_TRANSPORT rsync or tar. Default: rsync
+ REMOTE_OS posix or windows. Only needed for tar push mkdir.
+ Default: posix
+USAGE
+}
+
+if [[ $# -lt 2 || $# -gt 3 ]]; then
+ usage >&2
+ exit 2
+fi
+
+mode="$1"
+target="$2"
+remote_dir="${3:-~/cua}"
+remote_artifact_dir="${REMOTE_ARTIFACT_DIR:-vm-out}"
+rsync_ssh="${RSYNC_SSH:-ssh}"
+transport="${SYNC_TRANSPORT:-rsync}"
+remote_os="${REMOTE_OS:-posix}"
+
+script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+repo_root="$(cd "$script_dir/../../.." && pwd)"
+artifact_root="$repo_root/libs/cua-driver/docs/vm-artifacts"
+target_slug="$(printf '%s' "$target" | tr -c 'A-Za-z0-9_.-' '_')"
+timestamp="$(date -u +%Y%m%dT%H%M%SZ)"
+
+exclude_args=(
+ --exclude=.git/
+ --exclude=.DS_Store
+ --exclude='._*'
+ --exclude=target/
+ --exclude=node_modules/
+ --exclude=.venv/
+ --exclude=__pycache__/
+ --exclude='*.pyc'
+ --exclude=dist/
+)
+
+tar_exclude_args=(
+ --exclude ./.git
+ --exclude ./.DS_Store
+ --exclude '._*'
+ --exclude ./target
+ --exclude ./node_modules
+ --exclude ./.venv
+ --exclude '*/__pycache__'
+ --exclude '*.pyc'
+ --exclude ./dist
+)
+
+remote_mkdir() {
+ case "$remote_os" in
+ windows)
+ "$rsync_ssh" "$target" \
+ "powershell -NoProfile -Command \"New-Item -ItemType Directory -Force '$remote_dir' | Out-Null\""
+ ;;
+ posix)
+ "$rsync_ssh" "$target" "mkdir -p $remote_dir"
+ ;;
+ *)
+ echo "REMOTE_OS must be posix or windows, got: $remote_os" >&2
+ exit 2
+ ;;
+ esac
+}
+
+push_rsync() {
+ ssh_cmd=${remote_dir/#\~/"\$HOME"}
+ "$rsync_ssh" "$target" "mkdir -p $ssh_cmd"
+ rsync -az --delete -e "$rsync_ssh" "${exclude_args[@]}" "$repo_root/" "$target:$remote_dir/"
+}
+
+push_tar() {
+ remote_mkdir
+ COPYFILE_DISABLE=1 tar "${tar_exclude_args[@]}" -czf - -C "$repo_root" . \
+ | "$rsync_ssh" "$target" "tar -xzf - -C \"$remote_dir\""
+}
+
+pull_artifacts_rsync() {
+ rsync -az -e "$rsync_ssh" "$target:$remote_dir/$remote_artifact_dir/" "$dest/"
+}
+
+pull_artifacts_tar() {
+ "$rsync_ssh" "$target" "tar -czf - -C \"$remote_dir/$remote_artifact_dir\" ." \
+ | tar -xzf - -C "$dest"
+}
+
+pull_code_rsync() {
+ rsync -az --backup --suffix=".vm-backup-$timestamp" -e "$rsync_ssh" \
+ "${exclude_args[@]}" "$target:$remote_dir/" "$repo_root/"
+}
+
+pull_code_tar() {
+ "$rsync_ssh" "$target" "tar -czf - -C \"$remote_dir\" ." \
+ | COPYFILE_DISABLE=1 tar "${tar_exclude_args[@]}" -xzf - -C "$repo_root"
+}
+
+case "$mode" in
+ push)
+ case "$transport" in
+ rsync) push_rsync ;;
+ tar) push_tar ;;
+ *) echo "SYNC_TRANSPORT must be rsync or tar, got: $transport" >&2; exit 2 ;;
+ esac
+ ;;
+
+ pull-artifacts)
+ dest="$artifact_root/$target_slug/$timestamp"
+ mkdir -p "$dest"
+ case "$transport" in
+ rsync) pull_artifacts_rsync ;;
+ tar) pull_artifacts_tar ;;
+ *) echo "SYNC_TRANSPORT must be rsync or tar, got: $transport" >&2; exit 2 ;;
+ esac
+ echo "Pulled artifacts into $dest"
+ ;;
+
+ pull-code)
+ if [[ "${ALLOW_PULL_CODE:-}" != "1" ]]; then
+ echo "Refusing to pull VM source without ALLOW_PULL_CODE=1." >&2
+ exit 2
+ fi
+ case "$transport" in
+ rsync) pull_code_rsync ;;
+ tar) pull_code_tar ;;
+ *) echo "SYNC_TRANSPORT must be rsync or tar, got: $transport" >&2; exit 2 ;;
+ esac
+ ;;
+
+ *)
+ usage >&2
+ exit 2
+ ;;
+esac
diff --git a/libs/cua-driver/test-harness/MODALITY_TEST_SUITE.md b/libs/cua-driver/test-harness/MODALITY_TEST_SUITE.md
deleted file mode 100644
index 8594983a7b..0000000000
--- a/libs/cua-driver/test-harness/MODALITY_TEST_SUITE.md
+++ /dev/null
@@ -1,627 +0,0 @@
-# Modality test suite and harnesses
-
-This is an internal contributor runbook for the Cua Driver modality test suite.
-It used to live in the public docs reference tree, but it describes the harness,
-CI lanes, recorder scripts, and regression matrices rather than public product
-behavior. User-facing limits harvested from this page belong in
-`docs/content/docs/reference/cua-driver/limits.mdx`.
-
-This document describes the cua-driver test-harness: a cross-OS, cross-toolkit
-rig that exercises every driver action against a controlled application and
-scores each action **twice** — did it land an effect, and did it keep the app in
-the background. It is the authoritative map of *which driver mechanisms work
-where*, and why each limitation manifests in the mode/scope it does.
-
-## 1. Overview — what the suite validates
-
-The suite validates two things, per driver action, per surface:
-
-1. **The no-foreground / background contract** — cua-driver's differentiator.
- The driver is supposed to drive an app **without bringing it to the
- foreground** and **without moving the visible cursor** (it dispatches via the
- accessibility tree or via background coordinate injection, and paints a
- per-session overlay cursor instead of warping the real pointer). The suite
- measures, for every action, whether the target app stayed in the background
- (`held`) or jumped frontmost (`STOLE FOCUS`).
-
-2. **Cross-toolkit dispatch coverage** — whether an action *actually lands* on
- each UI toolkit. The same 8-action × 6-control matrix runs against WPF,
- WinUI3, WebView2, Electron (Windows), GTK3 + Electron (Linux), and AppKit,
- SwiftUI, WKWebView, Electron (macOS). Toolkits route input differently (UIA
- Invoke vs AT-SPI `doAction` vs AX actions vs synthetic pixels vs composition
- input-sites), so "click works" is a per-toolkit, per-dispatch-path fact, not
- a global one.
-
-**Scope caveat — this tests driver *mechanisms*, not agent task success.** A cell
-that reads ✓ means the driver delivered the action and the target's own
-instrumented state changed (a checkbox toggled, a slider moved, `last_action=`
-updated). It does **not** mean an LLM agent located the control correctly or
-chose the right action. Agent locator quality and agent self-judgment are
-explicitly kept on separate axes (see §7) so a bad locator can never mask — or be
-masked by — a driver regression.
-
----
-
-## 2. How the suite works — the modality recorder
-
-The core instrument is the **modality recorder** (one per OS:
-`windows/wpf-recorder.ps1`, `linux/lin-rec.py` + `lin-rec-electron.py`,
-`macos/mac-rec.py`). Each run produces one annotated screen recording:
-
-- **Test harness on the LEFT**, **`cua-driver-panel` dashboard on the RIGHT**
- (always-on-top). The dashboard is a tiny HTML page served over loopback and
- shown via a `chrome --app` (Windows/macOS) or WebKit (Linux) window; it polls a
- `status.json` the recorder writes (~5 Hz) and renders the live verdict.
-- The dashboard shows the run's **modality**, a live **foreground/background
- indicator** (current frontmost window title), and a **per-action** dual verdict.
-- The pink/cyan overlay is the driver's **per-session agent cursor** — proof the
- driver is acting without a real pointer move.
-
-### The 5 modality modes (per surface)
-
-Action rung × foreground/background × scope = 5 single-modality runs:
-
-| Mode | Action rung | Dispatch | Foreground? | Scope |
-|------------------|-----------------------|-----------------|-------------|-----------|
-| `ax-fg` | element `ax` action (accessibility) | `element_index` | app kept FRONT | window |
-| `ax-bg` | element `ax` action | `element_index` | app stays BACKGROUND (contract measured) | window |
-| `px-fg` | element `px` action (pixel) | pixel coords | app kept FRONT | window |
-| `px-bg` | element `px` action | pixel coords | app stays BACKGROUND (contract measured) | window |
-| `px-desktop` | element `px` action | screen pixels (window-less) | FRONT | desktop |
-
-The run names use `ax`/`px` as labels for the
-element-dispatch vs pixel-dispatch runs; **`capture_mode` itself is now
-deprecated and ignored** — `get_window_state` returns both the tree and a
-screenshot regardless of what you pass. The distinction the recorder exercises
-is the **action rung** (the element `ax` action addressing by `element_index`
-vs the element `px` action addressing by pixel `x,y`), not a capture toggle.
-
-The fg/bg split only matters for the contract (measured in the `*-bg` runs).
-`px-desktop` sets `capture_scope=desktop` and issues window-less,
-screen-absolute actions.
-
-### Dual scoring per action
-
-Every action row carries **two independent measurements** (rendered as two
-badges on the dashboard):
-
-1. **✓ worked / ✗ no-op (effect)** — did the action change the harness's *own*
- state? Read back after the action via the harness's instrumented status
- labels (`agreed=`, `slider_value=`, `last_action=`, `mirror=`, `menu_action=`,
- `scroll_offset=`).
-2. **held / STOLE FOCUS (contract)** — sampled for ~1.5 s after the action: did
- the target window's title become the frontmost window? In fg modes this badge
- is replaced by `foreground` (steal is expected/intentional).
-
-The dashboard tally at the bottom is the live count: `effects: N/M actions
-changed the app` and, in bg modes, `focus contract: S/A stole focus`.
-
-### How the verifier reads each toolkit's instrumented labels
-
-The harnesses all expose the **same set of status labels** (single source of
-truth: `shared/scenarios.json` + `shared/web/index.html`), but the verifier
-reads them differently per platform:
-
-| Platform / toolkit | How the verifier reads state |
-|-------------------------------|------------------------------|
-| Windows WPF / WinUI3 | **UIAutomation** — finds the harness window by title substring, scans `ControlType.Text` descendants, regex-extracts `agreed=`, `slider_value=`, `last_action=`, `mirror=`, `menu_action=`, `scroll_offset=` |
-| Windows WebView2 / Electron | Same UIA scan, but the window is matched by **substring** because Chromium titles carry a `[cdp=NNNN]` suffix; the web-AX tree only surfaces with `--force-renderer-accessibility` |
-| Linux GTK3 | The harness writes a **state file** (`/tmp/cua-lin-state.json`); the recorder's `hstate()` reads it (AT-SPI text isn't reliably enumerable for all labels) |
-| Linux Electron | Chromium **web-AX over AT-SPI** (`--force-renderer-accessibility`), read from the `get_window_state` tree |
-| macOS AppKit / SwiftUI | **AX** — status labels render as AX text nodes in `tree_markdown`; `do_native()` drives the action, then re-reads the labels |
-| macOS WKWebView / Electron | Chromium/WebKit web-AX text (`WEBISH` path), same label contract |
-
-The per-action `verify(action, before, after)` logic is consistent across
-recorders, e.g.: `click` → `agreed` flipped; `double` → `last_action ==
-double_click`; `right` → `last_action == right_click` **or** `menu_action`
-changed; `drag` → `slider` increased; `scroll` → `scroll_offset` increased;
-`setval` → `mirror == set-by-cua`; `type` → `mirror` contains `typed-by-cua`;
-`press-key` (Tab) → no asserted effect (`na`).
-
-> **Honesty principle (FINDINGS.md):** the harness/recorder stays honest — real
-> driver gaps are documented as driver work, fixed in the driver first, then
-> re-recorded. The recorder does not paper over a no-op.
-
-### Session lifecycle & disposal
-
-Each recorder declares a **session** (the agent-cursor identity, e.g. `d1` /
-`macd-…`) via `start_session`, drives every action under it, and must
-`end_session` it on teardown so the cursor + per-session state are disposed —
-not leaked.
-
-- **Disposal is double-guarded.** `end_session` and the **idle-TTL sweep**
- (`evict_idle`, default **300 s**, overridable via
- `CUA_DRIVER_RS_SESSION_IDLE_TTL_SECS`, swept every 30 s) both dispose through
- the same `fire_session_end` hook fan-out: agent cursor removed, recording
- stopped, per-session config cleared. A reused id starts **fresh, not poisoned**.
-- **Test:** `cua-driver-core/tests/session_lifecycle.rs` asserts start → use →
- `end_session` disposes (and a reused id starts fresh), and that idle-TTL
- eviction disposes the **same** way — verified via the disposal-hook contract
- both paths fan out to (the concrete cursor/config registries are a daemon-layer
- assertion). It passes a short TTL straight to `evict_idle`; the **300 s
- production default is never lowered** for tests.
-- **The recorders dispose explicitly** — `end_session` in a `try/finally` before
- they kill the daemon (`mac-rec.py` via an `atexit` net, since it leaves the
- daemon running) — so a session can't leak even if a run errors.
-
----
-
-## 3. Test-app harnesses
-
-Each harness is a self-contained host app the driver drives. They share fixtures
-via `shared/scenarios.json` (AutomationIds / AX identifiers / DOM ids /
-window titles) and, for the web hosts, `shared/web/index.html`, so identifiers
-never drift between apps and the Rust integration tests under
-`rust/crates/cua-driver/tests/harness_*.rs`. Built outputs stage into
-`rust/test-apps/harness-{name}/` via `build/{macos.sh,windows.ps1,linux.sh}`.
-
-### The 6 parity controls
-
-The "control-parity pass" brought every surface up to the WPF baseline of **6
-controls**, so the same 8-action matrix is exercisable everywhere:
-
-1. **checkbox** (`chk-agree*`) → `agreed=`
-2. **click-target** (left / right / double) → `last_action=`, `clicks=`
-3. **slider** (`sld-value`) → `slider_value=`
-4. **scroll-target** (tall `scroll-tall` region) → `scroll_offset=`
-5. **text-input** (`txt-input`, mirrored) → `mirror=`
-6. **context-menu** (`btn-context`) → `menu_action=`
-
-(Plus a `counter=` increment button used as a generic AX-press probe.) WPF
-carries an extended scenario set on top (modal MessageBox, owned/layered popups,
-native Win32 child HWND, menu bar, combo/list boxes) — it is the "fullest
-harness".
-
-### Per OS / toolkit
-
-| OS | Harness | What it is | Build | Source |
-|---------|-----------|------------|-------|--------|
-| Windows | **WPF** | WPF / UIA host — popups, layered windows, modal MessageBox, HwndHost native child | `build/windows.ps1` → `harness-wpf` | `apps/windows/wpf/` (C#/XAML) |
-| Windows | **WinUI3** | WinUI3 unpackaged — DirectComposition popups, XAML Popup primitive, CommandBarFlyout; HWND subclassed to translate `WM_VSCROLL` → `ChangeView` | `build/windows.ps1` → `harness-winui3` | `apps/windows/winui3/` |
-| Windows | **WebView2** | WPF host + `Microsoft.Edge.WebView2` loading the shared DOM; CDP on `:9222` | `build/windows.ps1` → `harness-webview` | `apps/windows/webview2/` |
-| Windows | **Electron** | Electron host loading the same shared DOM; CDP on `:9223` | `apps/cross-platform/electron/build.ps1` | `apps/cross-platform/electron/` |
-| Linux | **GTK3** | PyGObject GTK3 app; AT-SPI exposes accessible **NAME** (not an AutomationId); writes `/tmp/cua-lin-state.json` for the verifier | `build/linux.sh` → `harness-gtk3` | `apps/linux/gtk3/main.py` |
-| Linux | **Electron** | Same Electron bundle run from `node_modules` under Xvfb + dbus with `--force-renderer-accessibility` | run via `lin-run-electron.sh` | `apps/cross-platform/electron/` |
-| macOS | **AppKit** | Cocoa — NSButton, NSScrollView, NSMenu, NSSlider, menubar item | `build/macos.sh` → `harness-appkit` | `apps/macos/appkit/main.swift` |
-| macOS | **SwiftUI** | SwiftUI — `.popover()`, `.contextMenu`, declarative `Slider`/`Toggle`/`ScrollView` | `build/macos.sh` → `harness-swiftui` | `apps/macos/swiftui/main.swift` |
-| macOS | **WKWebView** | Apple WebKit host loading the shared DOM — the native analogue of WebView2 | `build/macos.sh` → `harness-wkwebview` | `apps/macos/wkwebview/main.swift` |
-| macOS | **Electron** | Same shared DOM under Chromium-on-macOS | `apps/cross-platform/electron/build.sh` | `apps/cross-platform/electron/` |
-
-### Instrumented labels (the verifier contract)
-
-Every harness updates these labels in response to a successful action; the
-verifier asserts on them:
-
-| Label | Set by | Verifies |
-|------------------|--------------------------------|----------|
-| `last_action=` | click-target (left/right/double)| click / double / right landed |
-| `clicks=` | click-target | click count |
-| `slider_value=` | slider | drag / set_value on a range control |
-| `scroll_offset=` | scroll-target | scroll landed |
-| `agreed=` | checkbox | click toggled a toggle |
-| `menu_action=` | context-menu item | right-click → context menu → item invoked |
-| `mirror=` | text-input | type / set_value into a text field (`typed-by-cua` / `set-by-cua`) |
-| `counter=` | increment button | generic AX-press probe |
-
----
-
-## 4. Action rungs & scopes
-
-### Action rung (how the action addresses the target)
-
-`get_window_state` is perception-mode-agnostic: it returns **both** the
-accessibility tree and a screenshot by default (`include_screenshot:false` skips
-the screenshot grab — a perf opt-out, not a modality choice). `capture_mode` is
-**deprecated and ignored** — there is no `ax`/`vision` capture toggle anymore.
-The modality is chosen at **action time** by how you address the target, and
-that one choice selects the rung:
-
-| Rung | How the agent addresses the target | How actions dispatch |
-|------|------------------------------------|----------------------|
-| element `ax` action | by `element_index` (read off the tree) | UIA Invoke (Windows) / AT-SPI `doAction` (Linux) / AX actions (macOS) — backgroundable, driver-verifiable |
-| element `px` action | by pixel `x,y` (read off the screenshot in the same response) | synthetic/injected pointer events at (x,y) — best-effort, caller-confirmed |
-
-> **Deprecated aliases.** `som` (the old set-of-marks mode) and `screenshot` are
-> no longer advertised capture modes — like `ax`/`vision`/`capture_mode` itself,
-> they still decode but are **ignored** (both the tree and the screenshot come
-> back regardless). The modality recorder's runs exercise the element-dispatch
-> path and the pixel-dispatch path, so the recorder runs and the results matrices
-> below are unchanged — they are framed here as the element `ax` action and the
-> element `px` action rungs.
-
-### Capture scope (`capture_scope`)
-
-| Scope | Meaning |
-|-----------|--------------------------------|
-| `window` | single target window (default) |
-| `desktop` | full display |
-
-### Why mode/scope determines where a limitation shows
-
-This is the central organizing idea (from `CUA_DRIVER_LIMITATIONS_AND_TEST_MATRIX.md`):
-
-- **The element `ax` action (`element_index`) path.** Limits in the element path
- (a toolkit with no `doAction` for right-click, a WinUI3 composition input-site
- the WPF path can't reach, an AXValue a SwiftUI slider rejects) surface in this
- rung — the `ax-` runs.
-- **The element `px` action (pixel) path.** Limits in the pixel path (GTK
- dropping synthetic X events, NSButton ignoring a synthetic mouseDown, a 2×
- Retina coordinate conversion) surface in this rung — the `px-` runs.
-- **fg/bg** isolates the *contract* (only `*-bg` measures focus steal).
-- **`desktop` scope** is where window-less / coordinate-space-conversion bugs
- surface (e.g. the macOS 2× Retina desktop path, the macOS no-windowless-click
- gap).
-
----
-
-## 5. Action × control matrix
-
-**8 actions** × **6 controls**. Each modality run filters the plan:
-`set_value` is dropped in the `px` modes (the pixel-dispatch runs — it's
-AX-only); `px-desktop` keeps only `{click, scroll, type, press-key}`
-(window-less screen actions).
-
-| Action ↓ / Control → | checkbox | click-target | slider | scroll-target | text-input | context-menu |
-|----------------------|:--------:|:------------:|:------:|:-------------:|:----------:|:------------:|
-| **click** | ● (toggle) | ● (left) | | | | |
-| **double-click** | | ● | | | | |
-| **right-click** | | ● (web/WinUI3 records `last_action`) | | | | ● (opens menu) |
-| **drag** | | | ● (thumb→X) | | | |
-| **scroll** | | | | ● | | |
-| **set_value** (AX) | | | ● (range value) | | ● (`set-by-cua`) | |
-| **type** | | | | | ● (`typed-by-cua`) | |
-| **press-key** (Tab) | | | | | ● (focus move; no asserted effect) | |
-
-Notes:
-- WPF has a **dedicated context-menu control**; WinUI3 / web harnesses have none,
- so right-click is aimed at the click-target and verified via
- `last_action=right_click`.
-- SwiftUI's AX has no distinct right-press action, so right-click on its button
- coerces to a normal AXPress — right-click semantics there are covered by the
- dedicated `context_menu` scenario.
-
----
-
-## 6. RESULTS MATRIX (the core)
-
-Per cell: **✓ landed · ✗ no-op · ⚠ guarded/fail-loud** for the *effect*, with the
-reason. The **Contract** column is the focus verdict measured in the `*-bg` runs.
-"land" = the harness's own instrumented label changed. Verification status is
-drawn from FINDINGS.md, the limitations doc, and the fix-commit bodies; runtime
-status is flagged where a commit deferred it.
-
-> Legend: ✓ lands · ✗ no-op (honest) · ⚠ guarded clean-fail / fail-loud ·
-> — not exercised · n/a not in plan / no asserted effect.
-
-### macOS
-
-Contract: **HOLDS — 0/8 stole** on AppKit in both `ax-bg` and `px-bg`
-(pixel left-click/double/drag landed while Chrome stayed frontmost). WKWebView
-also holds — confirming the Windows Chromium steal is Windows-specific, not
-WebKit-on-macOS.
-
-| Harness | mode | click | double | right | drag | scroll | set_value | type | press-key | Contract | Notes |
-|---------|------|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|------|
-| AppKit | `ax` | ✓ | ✓ (fixed bg, `click_at_xy_with_window_local`, `80b4e3d7`) | ✓ (AXShowMenu→delivering-pixel fallback, `80b4e3d7`) | ✓ | ⚠ element scroll no-ops on content-height containers (use x,y) | ✓ CFNumber NSSlider (`92d8aebf`, verified 0→50) | ✓ | n/a | **0/8 stole** | ax-bg 5/7 effects |
-| AppKit | `px` | ⚠ **no-op on standard NSButton even frontmost** (modal mouseDown reads window-server queue, not `postToPid` → use `element_index`) | ✓ | ✓ now fires `rightMouseDown` (window-number + button + primer, `e7145c18`) | ✓ | ✓ via per-pid pixel-wheel `scroll_wheel_at_xy` (`e7145c18`) | n/a | ✓ | n/a | **0/8 stole** | needs target frontmost; px-bg 4/6 |
-| SwiftUI | `ax` | ✓ | ✓ | ✓ (via dedicated context-menu) | ✗ slider drag is a composition no-op | ⚠ nested scroll no-op | ⚠ **AXValue unsettable** (`-25200`); needs AXIncrement/AXDecrement stepping (`d2b29230`) | ✓ | n/a | **0/8 stole** | full 6-control parity confirmed live |
-| WKWebView | `ax`+`px` | ✓ | ✓ | ✓ | ✓ slider (pixel drag at live frame) | ⚠ **page scroll works, nested `overflow:auto` div no-op** (div has no `tabindex` → never keyboard-focusable; needs pixel-wheel) | ✗ **AXValue set, no DOM `input` event** (web mirror unchanged) | ✓ | n/a | **HOLDS** | native analogue of WebView2. Recorder fix (`mac-rec.py`): web dispatch now uses the **live element frame**, not a stale hardcoded coord map that was ~185px off → every WKWebView pixel action used to miss. ax-fg re-measured **5/6** (click/double/right/drag/type land; only `set_value` the honest web no-op). |
-| Electron (mac) | `ax`/`px` | ✓ | ✓ | — | — | ✗ (web scroll, same as WKWebView) | — | ✓ | n/a | holds (Chromium-on-mac does **not** steal like Windows) | local-only recordings |
-
-macOS gotchas filed as driver bugs: **`end_session` poisons a reused session id**
-(subsequent actions silently no-op); **AppKit window height drifts between
-launches** (store targets as window-local points, convert to live screenshot px).
-
-### Linux
-
-| Harness | mode | click | double | right | drag | scroll | set_value | type | press-key | Contract | Notes |
-|---------|------|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|------|
-| GTK3 | `ax` (AT-SPI) | ✓ left-click via hit-test→`doAction` (`7d358283`) | ✗ no `doAction` equiv | ✗ no `doAction` equiv | ⚠ value-only (no Action) — driven via `set_value` | ⚠ surfaced now (`is_indexable = actions \|\| has_value`); driven via `set_value` | ✓ slider/scroll value widgets now surface | ✓ | n/a | ax-bg **1/8 stole** (only `set_value`; corrected via genuine-anchor baseline `5c0a1d3c` — was 3/8) | left/set_value/type land |
-| GTK3 | `px` (Xvfb) | ✗ | ✗ | ✗ | ✗ | ✗ | n/a | ✗ | n/a | **0/7 stole** | **GTK drops synthetic XSendEvent**; XTEST core events don't reach its XInput2 path |
-| GTK3 | `px` (**real Xorg**) | ✓ | ✓ | ✓ | ✓ | ✓ | n/a | ✓ | n/a | holds focus | right/double/middle-click + scroll **land via uinput/XInput2-MPX + shield-grab** (`79e546ca`); capability auto-detected via `real_pointer_input_available()` — **runtime-verified on a real Xorg server (dummy-driver): the probe flips TRUE, all four actions LAND and HOLD focus (`_NET_ACTIVE_WINDOW` unchanged), confirmed by the harness oracle + a middle-click PRIMARY paste. Xvfb can't bind uinput as an X slave, so the path auto-skips there.** |
-| Electron (Linux) | `ax` | ✓ click | ✗ | ✗ | ✓ drag | ✗ scroll resolves but no-op (AT-SPI synthetic) | ✗ | ✓ type | n/a | ax-bg **2/8 stole** (`set_value` + `drag`; was 3/8 — recorder baseline artifact, `5c0a1d3c`) | drag lands a value-change but its synthetic window-coord activates Chromium (never reaches the slider thumb); still holds far better than Windows Electron's reported 7/8 (also an artifact) |
-| Electron (Linux) | `px` | ✗ | ✓ pixel double-click fires on click-target | ✗ | ✗ | ✗ | n/a | ✗ | n/a | px-bg **6/7 stole** (pixel dispatch foregrounds Chromium) | |
-| GTK4 (gnome-calculator) | `ax` (AT-SPI) | ✓ via `doAction` (coordinate-free; lands without a valid `frame`) | ✗ no `doAction` equiv | ✗ no `doAction` equiv | ⚠ value-only (no Action) — driven via `set_value` | ⚠ driven via `set_value` | ✓ | ✓ | n/a | holds | **GTK4 coord fix required for `frame`/agent-cursor.** GTK4's AT-SPI bridge returns `Component.GetExtents(SCREEN)=(0,0)` for every widget (GNOME/gtk #1564/#1739 a11y rework). Fix: queries `CoordType::Window` (GTK4 reports correctly per-widget) and reconstructs screen as `x11_window_origin + _GTK_FRAME_EXTENTS.(left,top) + WINDOW_xy`, gated on presence of `_GTK_FRAME_EXTENTS` so non-GTK toolkits (Qt) keep their correct SCREEN path. Verified live: button "7" = window(55,27) + inset(61,55) + WINDOW(16,293) = screen(132,375). `doAction` element clicks land regardless (coordinate-free). GNOME VM lane only. |
-| GTK4 (gnome-calculator) | `px` (real Xorg) | ✓ (after GTK4 coord fix) | ✓ | ✓ | ✓ | ✓ | n/a | ✓ | n/a | holds | Pixel coords reliable **only after** WINDOW+`_GTK_FRAME_EXTENTS` reconstruction; without the fix, cursor/frame collapses to the window corner. GNOME Shell requires real console Xorg (software GLX) — not Xvnc. |
-
-### Windows
-
-Contract: **HOLDS on WPF / WinUI3 / WebView2.** **Electron** — see the caveat
-below (the prior "7/8 stole" is a recorder measurement artifact).
-
-| Harness | mode | click | double | right | drag | scroll | set_value | type | press-key | Contract | Notes |
-|---------|------|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|:--:|------|
-| WPF | `ax` | ✓ (UIA Toggle) | ✓ | ⚠ off-screen at 556px reflow → clean no-op (guarded; UIA Invoke works off-screen) | ✓ slider 0→48 | ⚠ off-screen reflow → guarded no-op | ✓ (`mirror=set-by-cua`) | ✓ (recorder focuses the box first) | n/a | **2/8 stole** | "fullest harness"; ax-bg ~5/7 effects |
-| WPF | `px` | ✓ | ✓ | ⚠ guarded (`point_in_window_bounds` refuses off-screen point; off-screen controls now ScrollIntoView + actuate via ancestor ScrollPattern, `c3efb587`, runtime-verified counter 0→1) | ✓ | ⚠ off-screen reflow | n/a | ✓ | n/a | 1–2/7 stole | off-screen guard prevents taskbar misfire |
-| WinUI3 | `ax` | ✓ left-click (UIA Invoke) | ✓ **now LANDS** — double UIA-Invoke under `WS_EX_NOACTIVATE` guard (`531aa6de`, runtime-verified live) | ⚠ **fail-loud `background_unavailable`** — no contract-safe path (pen injection steals; WM_*BUTTON doesn't land) | ⚠ fail-loud (same composition input-site gap) | ✓ (HWND subclass: `WM_VSCROLL`→`ChangeView`) | ✓ | ✓ | n/a | **0/7 stole** | gap needs a WinUI3-specific composition InputSite path |
-| WebView2 | `ax` | ✗ checkbox below the fold (web won't scroll in ax) | ✓ | ✓ records `last_action=right_click` | ✓ | ✗ host HWND doesn't route scroll to renderer | ✓ | ✓ | n/a | **0/7 stole** | needs `--force-renderer-accessibility`; 4/7 effects |
-| Electron | `ax` | ✓ | ✓ | — | ✓ | ✗ web scroll | ✓ | ✓ | n/a | **see caveat** | 5/7 effects |
-
-Other Windows: **UIA element-cache use-after-free** (concurrent `ax` sessions on
-the same window) — fixed with a `RetainedElement` retain-under-lock guard
-(`d95b89a1`); proven by a deterministic `cache_uaf_repro` test (pre-fix path
-takes a real access violation, fixed path survives a 6-thread stress loop,
-`531aa6de`). Recording the VM needs **Session 2 attached** (`tscon … /dest:console`)
-and **ffmpeg on PATH before `serve`**.
-
-> **Windows Electron contract caveat — RESOLVED.** FINDINGS.md records Electron as
-> stealing **7/8** in `ax-bg`. That was a **recorder MEASUREMENT ARTIFACT**: the
-> recorder re-asserted its dashboard panel with `SetWindowPos` (z-order only, no
-> activation), so it never held a real foreground baseline — after the first inject
-> action click-activated Chromium, the harness stayed frontmost and every later
-> step false-positived as a steal. The recorder was fixed (`49bdb41b`) to genuinely
-> `SetForegroundWindow` a real non-harness anchor (mspaint) before each step.
-> **Re-measured: Electron `ax-bg` = 0/8 (held)** — corroborated by `baseline.log`
-> (anchor held ×8), an independent `metric.log` `MEASURE=0/8`, and frame f03 (actions
-> landed while the harness stayed non-foreground). So **Electron HOLDS the contract**,
-> consistent with Linux/macOS Electron. (The committed `matrix-electron-ax-bg.mp4`
-> still shows the old 7/8 until re-recorded with the fixed recorder.)
-
----
-
-## 7. Vision-agent coordinate test
-
-`vision-agent-test/` (`vision_agent_test.py`, `010fdd78`) is a newer test that
-hits the driver **the way a vision agent actually does** — and removes the
-modality recorder's overfit (hand-tuned window-local points run through a private
-ratio, which never exercises the driver's image→screen mapping).
-
-**Invariant under test:** *the pixel an agent reads off the returned screenshot is
-the pixel that gets clicked* — verified by the target's own instrumented state
-changing.
-
-The loop (no cheating in locate or click):
-1. **capture** — `get_window_state` (window — reads the screenshot it returns by
- default) / `get_desktop_state` (desktop, true pixels): the exact image an agent
- receives.
-2. **locate** — a deterministic pixel **in the returned-image coordinate space**
- (`PixelRegistryLocator`: a pre-measured pixel read off the real PNG, with a
- dims-guard that fails loud if the pinned geometry drifts). No `element_index`,
- no hand-converted window-local points. `locate(image, target, dims) → (x,y)`
- is pluggable (an LLM locator can drop in later).
-3. **act** — `click`/`right_click`/`scroll` at that pixel (scope matches capture).
-4. **verify** — the harness oracle (`last_action=`, `clicks=`, …) → objective
- pass/fail. A coordinate mis-map leaves the oracle unchanged → FAIL.
-
-Run: `python3 vision_agent_test.py {wkwebview-click-window|wkwebview-click-desktop|appkit-click-window|safari-learnmore-desktop|all}`.
-
-**Three axes kept separate** so none can mask the others: (a) the **driver
-coordinate invariant** (this test, deterministic → the regression guard); (b)
-**agent locator quality** (future, LLM, same `locate()` signature, scored
-separately, never gates the regression); (c) **agent self-judged success with no
-oracle** (future, isolated track).
-
-What the deterministic version caught that the modality suite structurally
-couldn't:
-- **2× Retina desktop path now correct + guarded** — pixel (340,1358) in the
- 3024×1964 desktop PNG converts to screen-point (170,679) and lands (oracle +
- a real Safari navigation confirm it). This turns the Retina escape into a
- permanently-guarded one-liner.
-- **Pixel-click is a no-op on AppKit `NSButton` even frontmost** (modal
- mouseDown reads the window-server queue, not the per-pid `postToPid` queue) —
- the modality suite drives this via AXPress, so it never saw it.
-- **The pixel path requires the target app frontmost** (the AX path doesn't).
-- **The AppKit harness window AX returns only the menu bar** — its `clicks=`
- oracle is unreachable that way; WKWebView exposes it fine.
-
----
-
-## 8. Known overfitting caveats
-
-The suite's own authors flag two ways the modality recorder could have lied, and
-the de-risking work for each:
-
-**(a) Recorder hand-coords masked the desktop 2× Retina bug.** The modality
-recorder stores targets as hand-tuned window-local points and converts them
-through a private ratio. That conversion is *self-consistent* with the driver's
-own assumption, so it never exercised the driver's real image→screen mapping —
-and the **desktop-scope 2× Retina off-by-backing-scale bug** (a center-pixel pick
-warping to the corner) slipped through every `px-desktop` run. De-risk: the
-**vision-agent coordinate test** (§7) reads a pixel straight off the returned PNG
-with a dims-guard, caught the bug, and now guards it (`80b4e3d7` fixes the desktop
-branch to divide x,y by the native/logical ratio; `010fdd78` adds the regression
-test).
-
-**(b) Recorder contract measurement had no real foreground baseline → the false
-"Electron 7/8 stole".** The recorder judges the contract by checking whether the
-target's title is the frontmost window, but it never established a genuine,
-distinct foreground window to begin with — so Windows Electron read as stealing
-**7/8** when the corrected understanding is that Chromium-on-Windows holds the
-contract (matching macOS/Linux Electron). De-risk: the **recorder-contract fix**
-(`49bdb41b`) now genuinely `SetForegroundWindow`s a real anchor before each step;
-**re-measured Electron `ax-bg` = 0/8 (held)**, so the claim is now verified and the
-legacy FINDINGS "7/8" is a confirmed measurement artifact. The Linux recorders had
-the same flaw — also fixed (`5c0a1d3c`): GTK3 + Linux-Electron `ax-bg` corrected
-3/8 → 1/8 (only `set_value` genuinely steals); macOS was already fine.
-
-> Both caveats reflect the same lesson: a recorder that hand-feeds the driver its
-> own assumptions can hide bugs in exactly the path a real agent uses. The
-> deterministic, oracle-checked vision-agent test and the re-baselined contract
-> measurement are the two structural fixes.
-
----
-
-## 9. Edge cases — real closed-source apps
-
-The synthetic harnesses pin down the heuristics; real apps surface behaviours a
-single-process test window never can. These were found driving live closed-source
-apps (Finder, System Settings, Calculator, Safari on macOS; Calculator/Notepad UWP,
-Edge, Explorer on Windows) and are why the suite is a floor, not the ceiling.
-
-### macOS (Finder, System Settings, Calculator, Safari)
-
-1. **Pixel click hits the right pixel but the wrong window.** With ~8 overlapping
- same-pid Finder windows, a crosshair dead-centre on the target file still posts to
- *screen* coordinates, so an occluding sibling intercepts it → no-op. A per-window
- screenshot masks it. Strong argument for the `ax`/element path, which is z-order
- independent.
-2. **`set_value` and `type_text` both falsely succeed on a background search field**
- (System Settings, SwiftUI). `set_value` writes `AXValue` (text appears, the search
- action never fires); `type_text` posts keys that a non-first-responder window drops.
- Neither drives the field; **both report success.**
-3. **Finder column filenames don't advertise `AXPress`** → a default click is an honest
- no-op; the driver surfaces `AXOpen`/`AXShowMenu`/`AXConfirm`, so an agent must pick
- `action:"open"`/`"pick"`.
-4. **The Calculator result is AX-invisible** (no AX node for the display) → an AX-only
- agent can't read the answer; the keypad itself is labelled. A `vision` readout is
- required.
-5. **AppKit AX-tree duplication** (System Settings ~half duplicated, Safari a duplicated
- toolbar) plus whole-menu-bar walking inflates real-app context versus synthetic.
-
-### Windows (Calculator/Notepad UWP, Edge, Explorer)
-
-A. **UWP window-identity split.** `launch_app("Calculator")` returns the package backing
- PID and a `window_id` that's stale by the next call; the real top-level HWND belongs to
- `ApplicationFrameHost.exe`. Driving by the returned pid+window_id errors `No window with
- window_id … exists`. The driver should resolve UWP windows to their `ApplicationFrameHost`
- host / relink the churned HWND.
-B. **Real UWP is drivable via the element path without the uiAccess worker.** Calculator's
- `num5Button` drove the display 0→5 via UIA Invoke from the Medium-IL daemon with no
- `cua-driver-uia.exe` running. This refines the `#1602`/`serve.rs` assumption: only the
- **pixel/SendInput** path needs the uiAccess worker for AppContainer apps — the
- **`element_index` UIA Invoke/ValuePattern path works on real UWP as-is.**
-C. **Real Edge (Chromium) holds the foreground contract.** With Notepad pinned top, 3×
- click + double-click + right-click on background Edge left the z-order unchanged
- (`(background, no foreground swap)`). The shield validated on synthetic Electron (0/8)
- **generalises to a real closed-source Chromium** — and the double/right-click that needed
- the WinUI3 composition fix synthetically just work on real Chromium in the background.
-D. **`element_index` requires `pid` (+`window_id`).** Element actions with `element_index`
- alone fail-fast with `Missing required integer field: pid`. Correct, but the MCP tool
- descriptions under-emphasise it — an agent that omits `pid` and filters stderr perceives a
- silent no-op. The `element_index` tool schemas should state `pid`'s necessity explicitly.
-E. **`get_screen_size` under-reports desktop width** (1024 reported vs 1824 actual span,
- likely an RDP dynamic-resolution artifact). The element path is unaffected (window-local
- frames stay consistent); a pixel/`vision` agent computing against 1024 width misplaces
- clicks in the right ~800 px band.
-
-F. **⚠ Unhandled-protocol `launch_app` deadlocks the whole daemon (DoS-class).** Launching an
- app whose protocol has no registered handler (e.g. `bingmaps:` with Maps uninstalled) spawns
- Windows' "you'll need a new app" modal **on the daemon's session desktop** and blocks the
- worker thread inside the shell-launch call. The client wedges indefinitely and a subsequent
- `stop` reports "daemon is not running" because the wedged daemon can't service the pipe;
- recovery needs the modal dismissed from inside that session (Session-0 SSH can't see it —
- per-session window stations). **Any bad app name/protocol is a full-daemon DoS.** The driver
- should launch via a non-blocking path with a timeout and/or validate handler registration
- before `ShellExecute`. (Needs a fix + a tracking issue.)
-G. **One `ApplicationFrameHost` pid multiplexes N unrelated UWP apps** (Settings, Calculator,
- and Store were all hosted under the same pid). Anything keying state/cursor/element-cache by
- `pid` alone is ambiguous across apps — only `window_id` is a real identity. (Generalises A.)
-H. **`launch_app`'s return contract is effectively per-app-architecture** — three topologies
- seen across five apps: brokered/`pid:0` (Settings, Store), real-pid-but-empty-`windows` race
- (Photos exposes its pid a beat before its HWND), and clean pid+window (Snipping Tool, the
- well-behaved baseline). Store is a three-way split: launch-pid `0` ≠ window-pid (AFH) ≠
- content-pid (`WinStore.App`), yet UIA still walks fully into the content provider.
-I. **The AFH UIA root is a caption-only ~188×32 strip** for Settings and Store while the content
- frames are full and correct — the AppFrame chrome and the XAML content are different UIA
- providers stitched at the window. A single-provider synthetic WinUI harness won't reproduce
- this dual-provider root.
-
-> Methodology note: on a disconnected-RDP console session, `GetForegroundWindow` returns `0`
-> (no foreground window), so focus-steal can't be measured by that probe there — verify "landed"
-> by content-state change instead. A single-process harness never hits this.
-
----
-
-## Linux dispatch ladder — container and VM lanes
-
-Linux (X11/XFCE and real-desktop VMs) now validates the same `delivery_mode` background/foreground dispatch ladder that macOS and Windows exercise, closing the parity gap. The matrix below shows each modality's path and whether the driver can verify the action landed without a screenshot.
-
-### Validated modality matrix
-
-| Modality | `delivery_mode` | Path reported | Driver-verifiable? |
-|----------|----------------|---------------|-------------------|
-| Element click (`element_index`) | background | `x11_atspi` (AT-SPI `doAction`) | yes — a11y action confirmed |
-| Element `px` action (pixel) | background | `x11_atspi` (AT-SPI `doAction`-at-point) for AX apps; else MPX `x11_pixel` | yes when AT-SPI-at-point lands; best-effort otherwise |
-| Element `px` action (escalated) | foreground | `x11_pixel_fg` (EWMH activate → inject → restore) | no — confirm via screenshot |
-| `type_text` into editable | background | `ax` (AT-SPI `insertText`) | yes (`verified: true`) |
-| `type_text`, non-editable focus | background / foreground | `key_events` / `key_events_fg` | no — confirm via screenshot |
-| `press_key` / `hotkey` | foreground | `x11_xtest_fg` (XSetInputFocus → XTEST key) | no — confirm via screenshot / effect (`keytest`) |
-
-Background element `px` actions (pixel clicks) do land on X11: apps that expose AT-SPI take the focus-free `doAction`-at-point path (`x11_atspi`), matching the macOS/Windows background-click behavior. The fallback to MPX `x11_pixel` (which requires a real Xorg server with `/dev/uinput` — unavailable under Xvnc or minimal containers) fires only for non-AX surfaces; escalate to `delivery_mode: foreground` there.
-
-**Foreground keyboard (`press_key` / `hotkey`, and `type_text` foreground) lands under Xvnc.** The XTEST keyboard path injects on a short-lived X connection; it now **round-trips after injection** so the server delivers the key before the connection closes — under Xvnc a connection that closed right after `flush()` had its queued KeyPress/KeyRelease dropped (pointer events survived the same close, which masked the bug as "foreground clicks work but keys don't"). Two companion fixes: shifted-level symbols (`*`, `+`, `(`) **auto-hold Shift** via slot-0/slot-1 keysym resolution (a bare keycode press emitted the unshifted glyph, e.g. `*`→`8`); and the foreground rung now **`XSetInputFocus`es the target** (KWin honours EWMH `_NET_ACTIVE_WINDOW` as raise-only — fine for clicks that land by stacking, but keys route to the X input focus). `derec.sh keytest` effect-confirms the whole path across toolkits (AC, `3*4=` → display reads `12`).
-
-### The XFCE container lane
-
-The `trycua/cua-xfce` Docker image is a reproducible, WM-equipped X11 target — xfwm4 + EWMH + full AT-SPI stack — that exercises both the foreground/EWMH and background AT-SPI modalities. It is richer than the existing Xvfb harness (no window manager, no EWMH) for those paths.
-
-The harness script lives at `libs/cua-driver/test-harness/linux-container/calc.sh`. It drives galculator through four modalities: background element `ax` click, background element `px` (pixel) click, foreground EWMH type, and background type with focus fallback.
-
-**Gotchas:**
-
-- **AT-SPI session-bus auto-discovery.** AT-SPI lives on the desktop session's D-Bus (`DBUS_SESSION_BUS_ADDRESS`). When the daemon starts outside the session (container entrypoint, headless, `runuser`/`su`, systemd system unit, VNC ad-hoc bus), that variable is unset, yielding an empty AT-SPI tree. The driver now auto-discovers the bus at startup — preferring `/run/user/<uid>/bus`, else reading it from a running desktop-session process's `/proc/<pid>/environ` (`xfce4-session`, `gnome-session`, etc.). An a11y bus must be running with toolkit-accessibility enabled, and the daemon must run as the desktop user — running the daemon as root against a user session is the Linux analogue of the Windows Session 0 isolation problem.
-- **`install-local` refuses root.** Use `runuser -u <desktop-user> -- cua-driver serve` to start the daemon as the session owner.
-- **Single-instance apps + zombie children.** Unreaped zombie children can pollute `pgrep`; a full daemon restart reaps them.
-- **Stale daemon, stale tool schema.** A daemon that was not restarted after an update can serve an outdated tool schema. Verify with `cua-driver describe <tool>` from a fresh shell.
-
-### Toolkit coverage
-
-| Toolkit | App under test | Lane | AT-SPI coord path |
-|---------|---------------|------|-------------------|
-| GTK3 | galculator | Container (XFCE / Kasm, `az exec`) | `SCREEN` — AT-SPI returns correct screen coords |
-| GTK4 | gnome-calculator | VM (GNOME Shell, SSH) | `WINDOW` + `_GTK_FRAME_EXTENTS` reconstruction (see below) |
-| Qt | kcalc | VM (KDE Plasma, SSH) | `SCREEN` — Qt AT-SPI reports correct screen coords |
-| Electron / Chromium | shared harness | Container (XFCE, `az exec`) | web-AX + pixel (`--force-renderer-accessibility`) |
-
-### Lane matrix
-
-| Lane | Desktop | Access method | Test app | Harness driver |
-|------|---------|---------------|----------|----------------|
-| XFCE container | `trycua/cua-xfce` (xfwm4) | `az exec` | galculator (GTK3), Electron | `libs/cua-driver/test-harness/linux-container/calc.sh` |
-| Kasm container | `trycua/cua-ubuntu` (Kasm/XFCE) | `az exec` | galculator (GTK3) | `libs/cua-driver/test-harness/linux-container/calc.sh` |
-| KDE Plasma VM | Azure VM, KDE Plasma X11 | SSH → `derec.sh` | kcalc (Qt) | `libs/cua-driver/test-harness/linux-container/derec.sh` |
-| GNOME Shell VM | Azure VM, GNOME Shell X11 (console Xorg) | SSH → `derec.sh` | gnome-calculator (GTK4) | `libs/cua-driver/test-harness/linux-container/derec.sh` |
-| GNOME Wayland VM | Azure VM, GNOME Mutter **Wayland** | SSH → `derec.sh` | gnome-calculator (GTK4) | `libs/cua-driver/test-harness/linux-container/derec.sh` |
-
-The VM lanes are driven over SSH by `derec.sh`, in contrast to the container lanes which use `az exec`. GNOME Shell requires the real console Xorg session (software GLX) and is not drivable over VNC (no GLX). KDE Plasma works over both console Xorg and VNC.
-
-### GTK4 coordinate reconstruction — invariant
-
-GTK4's AT-SPI bridge returns `Component.GetExtents(SCREEN) = (0, 0)` for every widget (GNOME/gtk #1564/#1739, a11y rework). The driver fix:
-
-1. Detects `_GTK_FRAME_EXTENTS` on the X11 window (present for GTK4; absent for Qt and GTK3).
-2. Queries `CoordType::Window` (which GTK4 reports correctly, per-widget).
-3. Reconstructs: `screen_xy = x11_window_origin + _GTK_FRAME_EXTENTS.(left, top) + WINDOW_xy`.
-
-Verified live: gnome-calculator button "7" = window-origin (55, 27) + inset (61, 55) + WINDOW (16, 293) = screen (132, 375).
-
-Non-GTK toolkits (Qt) keep the existing SCREEN path — the gate is the presence of `_GTK_FRAME_EXTENTS`.
-
-**Invariant worth guarding:** `frame_center ≈ x11_window_origin + _GTK_FRAME_EXTENTS + atspi_WINDOW_xy`. Any divergence means the reconstruction is broken. Before the fix, pixel clicks silently collapsed to the window corner; `doAction` element clicks landed regardless (coordinate-free).
-
-### The Wayland lane (GNOME Mutter / KDE Plasma)
-
-On a native Wayland session the **background AX rung is the same coordinate-free AT-SPI `doAction`** as on X11, and it is the one rung that works unchanged: `Action.DoAction` is dispatched toolkit-side (GTK/Qt invoke the widget's own action), so it needs no coordinates, no portal, no cursor move, and no focus steal. Element-index clicks therefore land on Mutter/KWin exactly as on X11.
-
-**Element `px` (pixel) coordinate clicks now land too — without pointer injection.** There is no global coordinate space on Wayland and Mutter drops synthetic virtual-pointer events, so a "click at (x,y)" can't be delivered as input. Instead the driver resolves the screen pixel back to the accessible element under it and fires *that* element's action — the proven `element_index` rung, reached by coordinates. The screen frame of every element is reconstructed the same way `get_window_state` exposes it (the `org.cua.WinRects` GNOME Shell helper supplies the window origin; GTK4's correct `CoordType::Window` extents supply the per-widget offset). The covering element is chosen **role-aware**: GTK4 nests a `label` inside every `button` with a near-identical, slightly *smaller* frame, so an area-only hit-test lands on the inert label and `do_action` silently no-ops (a "false success"); the selector prefers the smallest covering *real actuator* and only falls back to a passive label if nothing else covers the point. Validated live on Mutter Wayland: an element `px` action targeting buttons by `{x,y}` (no `element_index`) types `789` into gnome-calculator (rung `wayland_atspi`). The same role-aware fix hardened the X11 `perform_action_at_point` rung, which had the identical GTK4 false-success.
-
-Two Wayland-specific facts the lane exercises:
-
-- **Window enumeration via AT-SPI.** Native Wayland apps have no X11 XID, and GNOME Mutter / KDE KWin do **not** implement `zwlr_foreign_toplevel_management` (that is wlroots-only), so `wayland::list_windows` is empty there. `list_windows_dispatch` now falls back to **`atspi::list_windows`**, which enumerates top-level frames from the AT-SPI registry and hands back a synthetic, stable `window_id`. Downstream `get_window_state` / `click` walk the AT-SPI tree by **pid** (the xid is unused for the walk), so the full `list_windows → get_window_state → click` flow works. Validated live on Mutter Wayland: `7+8=` via `element_index` background clicks computes `15`.
-- **Engaging the Wayland backend.** `is_wayland()` requires the opt-in `CUA_DRIVER_RS_ENABLE_WAYLAND=1`, `WAYLAND_DISPLAY` set, **and** `DISPLAY` unset — otherwise the daemon stays on the X11 path. Start the daemon with all three for a pure-Wayland session.
-
-Bringing up a headless GNOME **Wayland** VM (the lane's host) has two gotchas worth recording, neither cua-driver's concern but both block the lane:
-
-- **GDM disables Wayland for the Azure virtual GPU.** The `hyperv_drm` DRM driver exposes no `parameters/modeset`, so GDM's `61-gdm.rules` udev gate (`ATTR{parameters/modeset}!="Y"`) force-selects Xorg. Override by shadowing the rule in `/etc/udev/rules.d/61-gdm.rules` (drop the modeset + bare `GOTO="gdm_disable_wayland"` lines) and set `Session=ubuntu-wayland` in `/var/lib/AccountsService/users/<user>`, then restart gdm.
-- **GTK4 apps need software GL on a GPU-less box.** Launch with `LIBGL_ALWAYS_SOFTWARE=1 GSK_RENDERER=cairo`, else the GL renderer fails (EGL/ZINK) and the app exits.
-
----
-
-## Appendix — file map
-
-| Path | Role |
-|------|------|
-| `shared/scenarios.json` | single source of truth: control ids, window titles, scenarios |
-| `shared/web/index.html` | shared DOM for WebView2 / Electron / WKWebView harnesses |
-| `apps/` | per-OS/toolkit harness sources (see §3 table) |
-| `build/{macos.sh,windows.ps1,linux.sh}` | stage harnesses into `rust/test-apps/harness-*` |
-| `modality-recordings/FINDINGS.md` | authoritative per-action findings |
-| `modality-recordings/windows/wpf-recorder.ps1` | Windows recorder (`-Mode`, `-Toolkit`) |
-| `modality-recordings/windows/run-one.ps1` | single-run launcher |
-| `modality-recordings/linux/lin-rec.py`, `lin-rec-electron.py` | Linux recorders (+ `lin-run*.sh`, `lin-dash*`, `lin-harness.py`) |
-| `modality-recordings/macos/mac-rec.py` | macOS recorder (`MODE SURFACE`) |
-| `vision-agent-test/vision_agent_test.py` + `README.md` | coordinate-invariant regression test |
-| `smoke/macos.sh` | macOS smoke check |
-| `rust/crates/cua-driver/tests/harness_*.rs` | Rust integration tests consuming staged harnesses |
-| `TEST_SUITE.md` (this dir) | sibling doc: the Rust integration-test taxonomy / `cua-driver-testkit` |
-| `CUA_DRIVER_LIMITATIONS_AND_TEST_MATRIX.md` (repo root) | limitations + mode/scope mapping |
diff --git a/libs/cua-driver/test-harness/README.md b/libs/cua-driver/test-harness/README.md
index 340b275263..69dbca7d8e 100644
--- a/libs/cua-driver/test-harness/README.md
+++ b/libs/cua-driver/test-harness/README.md
@@ -1,184 +1,103 @@
# cua-driver test harness
-Deterministic host apps that present a fixed scenario set for cua-driver
-to drive. One layout, three buckets: cross-OS reusable assets in
-`shared/`, OS-specific (or cross-platform) target apps in `apps/`,
-host-OS build scripts in `build/`, host-OS CLI smoke runners in
-`smoke/`.
+Deterministic host apps and shared fixtures used to verify cua-driver across
+Windows, macOS, and Linux. The harness is source-first: build scripts stage
+local outputs into `../rust/test-apps/harness-<name>/`; binaries are not
+committed.
## Layout
```text
test-harness/
-├── shared/ # cross-OS reusable
-│ ├── scenarios.json # single SoT for AutomationIds / window titles
-│ └── web/index.html # WebView2 + Electron load this same file
-├── apps/ # the test-target host apps
+├── shared/
+│ ├── scenarios.json # scenario ids, titles, and expected controls
+│ └── web/index.html # shared DOM for webview-style harnesses
+├── apps/
│ ├── cross-platform/
-│ │ └── electron/ # CuaTestHarness.Electron (CDP host, runs on any OS)
+│ │ ├── electron/ # Chromium/Electron host, CDP port 9223
+│ │ └── tauri/ # native webview/Tauri host
+│ ├── linux/
+│ │ └── gtk3/ # PyGObject GTK3 app
│ ├── macos/
-│ │ ├── appkit/ # CuaTestHarness.AppKit (single-file Swift)
-│ │ └── swiftui/ # CuaTestHarness.SwiftUI (single-file Swift)
+│ │ ├── appkit/ # single-file Swift AppKit app
+│ │ ├── swiftui/ # single-file SwiftUI app
+│ │ └── wkwebview/ # native WKWebView host for shared DOM
│ └── windows/
-│ ├── wpf/ # CuaTestHarness.Wpf (.NET 8 WPF)
-│ ├── winui3/ # CuaTestHarness.WinUI3 (.NET 8 WinUI3 unpackaged)
-│ └── webview2/ # CuaTestHarness.WebView (WPF + Microsoft.Edge.WebView2)
-├── build/ # host-OS build scripts (publish into ../rust/test-apps/)
-│ ├── macos.sh # builds appkit + swiftui + wkwebview + electron
-│ └── windows.ps1 # builds wpf + winui3 + webview2 + electron
-└── smoke/ # host-OS per-tool CLI smoke runners
- ├── macos.sh # spawns appkit harness, iterates every tool
- └── results/macos.txt # checked-in baseline for diff-based regression detection
+│ ├── wpf/ # .NET WPF app
+│ ├── winui3/ # unpackaged WinUI3 app
+│ └── webview2/ # WPF + WebView2 host for shared DOM
+├── build/ # host build scripts
+├── smoke/ # lightweight local smoke runners
+├── linux-container/ # Azure/XFCE Linux verification helpers
+└── modality-recordings/ # recorder scripts and dashboards
```
-`shared/scenarios.json` is the single source of truth — both the host
-apps and the Rust integration tests read it so AutomationIds, AX
-identifiers, and window titles never drift between the two halves.
-
-Published build outputs land in `../rust/test-apps/harness-{wpf,winui3,webview,electron,appkit,swiftui}/`
-so the existing sandbox runner and Rust integration tests pick them up
-without further wiring.
-
-## What's exercised
-
-WPF (`apps/windows/wpf`):
-- Plain XAML controls + AutomationIds (`counter`, `text_body`)
-- TextBox + mirrored label (`text_input`)
-- Right/double-click target (`click_target`)
-- Scroll body with VerticalOffset label + WM_VSCROLL hook (`scroll_target`)
-- Modal `MessageBox` (`message_box`)
-- Save/Cancel bottom strip — regression guard for PR #1696 (`bottom_strip`)
-- Native Win32 child HWNDs via `HwndHost` (`child_hwnd`)
-- Owned secondary `Window` (`owned_popup`)
-- Layered transparent window (`layered_popup`)
-- Keyboard accelerators — F5 + Ctrl+Shift+H (`accelerator`)
-
-WinUI3 (`apps/windows/winui3`):
-- `counter` / `text_body` / `exit` (parity with WPF)
-- TextBox + mirror (`text_input` — UIA `ValuePattern`)
-- `CommandBarFlyout` — popup in same HWND, DComp-rendered
-- XAML `Popup` — primitive, not a separate HWND
-
-WebView2 (`apps/windows/webview2`):
-- Loads `shared/web/index.html` — same page as the Electron harness.
-- Exposes CDP on `default_cdp_port=9222` so cua-driver's `page` tool can drive it.
-
-Electron (`apps/cross-platform/electron`):
-- Loads the same `shared/web/index.html`. CDP on `default_cdp_port=9223`.
-
-AppKit (`apps/macos/appkit`):
-- Plain NSButton + NSTextField counter (`counter`)
-- NSTextField label with shared marker (`text_body`)
-- Editable NSTextField + mirror label (`text_input`)
-- Custom NSView click-target (`click_target`)
-- NSScrollView with tall NSTextView body + offset label (`scroll_target`)
-- Top NSMenu item with known title (`ns_menubar`)
-- Exit button
-
-SwiftUI (`apps/macos/swiftui`):
-- Counter / text_body / text_input parity with AppKit
-- SwiftUI `.popover()` — Mac analogue of WinUI3 CommandBarFlyout / XAML Popup
-- Exit button
-
-## Coverage matrices (Rust integration tests)
-
-Windows:
-
-| Capability | WPF | WinUI3 |
-|-------------------------------------|-----------------|--------------|
-| Smoke (UIA tree + AutomationIds) | yes | yes |
-| UIA Invoke click | yes | (implicit) |
-| `right_click` | yes | — |
-| `double_click` | yes | — |
-| `type_text` (PostMessage / UIA) | yes | yes |
-| `set_value` (UIA ValuePattern) | yes | (covered) |
-| `scroll` (WM_VSCROLL via hook) | yes | — |
-| `press_key` accelerator (F5) | yes | — |
-| Modal `MessageBox` open + dismiss | yes | — |
-| Owned popup open + parse | yes | — |
-| Layered popup capture | yes | — |
-| XAML Popup open + parse | — | yes |
-
-macOS:
-
-| Capability | AppKit | SwiftUI |
-|-------------------------------------|-----------------|--------------|
-| Smoke (AX tree + identifiers) | yes | yes |
-| Counter click via element_index | yes | (implicit) |
-| `set_value` (AXSetAttribute) | yes | (covered) |
-| Popover open + body assert | — | yes |
-| Per-tool CLI smoke | `smoke/macos.sh`| — |
+`shared/scenarios.json` is the source of truth for AutomationIds, AX
+identifiers, expected window titles, and fixture names. Webview-style harnesses
+load `shared/web/index.html`, either directly or through a build-staged copy.
## Build
-macOS (requires Xcode CLT for `xcrun swiftc`; Node.js + npm for Electron):
+Build scripts stage outputs into `../rust/test-apps/`.
```bash
-libs/cua-driver/test-harness/build/macos.sh # all macOS-runnable harnesses
-libs/cua-driver/test-harness/build/macos.sh --skip swiftui # skip one target
-libs/cua-driver/test-harness/build/macos.sh --skip electron # skip Electron / npm
-libs/cua-driver/test-harness/build/macos.sh --clean # nuke stage dirs first
-```
+# macOS: AppKit, SwiftUI, WKWebView, Electron, Tauri
+libs/cua-driver/test-harness/build/macos.sh
+libs/cua-driver/test-harness/build/macos.sh --skip electron
-Windows (requires .NET 8 SDK on PATH; Node.js + npm for the Electron app):
+# Linux: GTK3, Electron, Tauri
+libs/cua-driver/test-harness/build/linux.sh
+libs/cua-driver/test-harness/build/linux.sh --skip tauri
+```
```powershell
+# Windows: WPF, WinUI3, WebView2, Electron, Tauri
cd libs\cua-driver\test-harness
-.\build\windows.ps1 # all four apps
-.\build\windows.ps1 -Skip winui3 # skip a specific one
-```
-
-Linux (requires Node.js + npm for the Electron app):
-
-```bash
-cd libs/cua-driver/test-harness/apps/cross-platform/electron
-./build.sh
-```
-
-## Run the integration tests
-
-```bash
-# macOS
-cd libs/cua-driver/rust
-cargo test --release --test harness_appkit_test -- --ignored --nocapture
-cargo test --release --test harness_swiftui_test -- --ignored --nocapture
-
-# Windows (locally)
-cd libs\cua-driver\rust
-cargo test --test harness_wpf_test -- --ignored --nocapture
-cargo test --test harness_winui3_test -- --ignored --nocapture
-
-# Windows Sandbox (matches CI)
-.\rust\sandbox\run-tests-in-sandbox.ps1 harness_wpf
-.\rust\sandbox\run-tests-in-sandbox.ps1 harness_winui3
-```
-
-Tests are `#[ignore]` so they don't run in plain `cargo test` (they
-require the harness apps to be built and, on macOS, TCC Accessibility
-granted to the test runner).
-
-## Per-tool CLI smoke
-
-Companion to the MCP integration tests. Spawns the AppKit harness as a
-victim, then iterates every cua-driver tool with sensible JSON args and
-reports PASS / FAIL / SKIP in a sorted table. Runs in ~25 seconds.
-
-```bash
-libs/cua-driver/test-harness/smoke/macos.sh
+.\build\windows.ps1
+.\build\windows.ps1 -Skip winui3
```
-A baseline result file (`smoke/results/macos.txt`) is checked in for the
-current driver version — diff against it to catch regressions in tool
-coverage.
-
-## AppKit / SwiftUI AX quirks captured in the tests
-
-- `accessibilityIdentifier` on AXStaticText doesn't propagate.
- NSTextField label-mode + SwiftUI `Text` views render as AXStaticText
- leaves and the OS drops the identifier slot. Tests assert AX-id only
- on actionable controls and assert on text-content for labels.
-- SwiftUI popovers may live in a separate AXWindow. The popover test
- walks `list_windows` for any new window owned by the harness pid if
- the trigger window doesn't contain the marker.
-- AX element budget caps tree depth + width. The scroll-body is capped
- at 30 lines so later scenarios don't get truncated.
+Host requirements:
+
+- macOS: Xcode command line tools; Node.js/npm for Electron; Rust for Tauri.
+- Linux: GTK3/PyGObject/AT-SPI; Node.js/npm for Electron; Rust plus WebKitGTK
+ build/runtime deps for Tauri.
+- Windows: .NET 8 SDK; Node.js/npm for Electron; Rust for Tauri.
+
+## Test Mapping
+
+Rust tests under `../rust/crates/cua-driver/tests/` consume the staged
+`rust/test-apps/harness-<name>/` outputs:
+
+- `harness_<toolkit>_test.rs`: toolkit-specific app coverage, usually ignored.
+- `modality_<area>[_<os>]_test.rs`: background input, capture, and desktop
+ modality coverage, ignored/manual or VM-backed.
+- `protocol_*_test.rs` and schema tests: headless protocol coverage, default.
+
+The Python e2e suite under `../tests/e2e/` drives real apps and the shared
+Electron/Tauri harnesses through the public MCP interface.
+
+## App Coverage
+
+| Harness | Source | Staged output | Primary surface |
+| --- | --- | --- | --- |
+| AppKit | `apps/macos/appkit` | `harness-appkit` | AX/AppKit controls |
+| SwiftUI | `apps/macos/swiftui` | `harness-swiftui` | AX/SwiftUI controls |
+| WKWebView | `apps/macos/wkwebview` | `harness-wkwebview` | WebKit AX tree |
+| WPF | `apps/windows/wpf` | `harness-wpf` | UIA/WPF controls |
+| WinUI3 | `apps/windows/winui3` | `harness-winui3` | UIA/XAML controls |
+| WebView2 | `apps/windows/webview2` | `harness-webview` | Chromium web UIA |
+| GTK3 | `apps/linux/gtk3` | `harness-gtk3` | AT-SPI/GTK controls |
+| Electron | `apps/cross-platform/electron` | `harness-electron` | Chromium web AX/UIA/AT-SPI |
+| Tauri | `apps/cross-platform/tauri` | `harness-tauri` | native webview AX/UIA/AT-SPI |
+
+## Maintenance Rules
+
+- Do not commit staged binaries, `node_modules`, copied `web/` folders, or Tauri
+ `target/` output.
+- Add new shared ids to `shared/scenarios.json` before asserting on them in
+ tests.
+- Keep platform-specific quirks in the relevant Rust test or app source, not in
+ the shared fixture.
+- If a recorder finding graduates into an invariant, move it into a Rust or
+ Python test and summarize it here.
diff --git a/libs/cua-driver/test-harness/TEST_HARNESS_STRUCTURE.md b/libs/cua-driver/test-harness/TEST_HARNESS_STRUCTURE.md
deleted file mode 100644
index 5ef85ef18c..0000000000
--- a/libs/cua-driver/test-harness/TEST_HARNESS_STRUCTURE.md
+++ /dev/null
@@ -1,117 +0,0 @@
-# cua-driver test harnesses — how they're structured
-
-The harness is built around **one principle**: a single source of truth (`shared/scenarios.json`) that both the **test apps** and the **Rust integration tests** read, so they can never drift. The directory tree itself encodes *which app runs on which OS* and *what's shared* (this layout landed in #1727).
-
-## Directory layout
-
-```
-libs/cua-driver/test-harness/
-│
-├── shared/ ◀── cross-OS single source of truth
-│ ├── scenarios.json # defines every app + scenario (titles, AutomationIds,
-│ │ # expected tree shape). Read by BOTH the apps and the tests.
-│ └── web/index.html # the page WebView2 AND Electron both load
-│
-├── apps/ ◀── the test-target GUI apps, grouped by where they run
-│ ├── cross-platform/
-│ │ └── electron/ # main.js · package.json (Win / macOS / Linux)
-│ ├── macos/
-│ │ ├── appkit/ # main.swift (macOS)
-│ │ └── swiftui/ # main.swift (macOS)
-│ └── windows/
-│ ├── wpf/ # WPF: MainWindow, Owned/Layered popups,
-│ │ # NativeButtonHost, Scenarios.cs (Windows)
-│ ├── winui3/ # WinUI3: App, MainWindow, Program.cs (Windows)
-│ └── webview2/ # WebView2 host (WPF) → loads shared/web (Windows)
-│
-├── build/ ◀── host-OS build scripts
-│ ├── macos.sh # swiftc the macOS apps + electron
-│ └── windows.ps1 # dotnet publish the .NET apps + electron
-│
-├── smoke/ ◀── per-tool CLI smoke runner + baseline
-│ ├── macos.sh
-│ └── results/macos.txt
-│
-├── CuaTestHarness.sln # .NET solution tying the Windows apps together
-└── README.md
-
-libs/cua-driver/rust/crates/cua-driver/tests/ ◀── the CONSUMERS (Rust integration tests)
-│ (all build on the cua-driver-testkit crate; named by 4-family taxonomy —
-│ harness_ / modality_ / protocol_ / guard_. See TEST_SUITE.md.)
-├── harness_wpf_test.rs (Windows)
-├── harness_winui3_test.rs (Windows)
-├── harness_web_test.rs (Windows — WebView2 + Electron via CDP)
-├── harness_libreoffice_test.rs (Windows — LibreOffice VCL/SAL via MSAA)
-├── harness_appkit_test.rs (macOS)
-└── harness_swiftui_test.rs (macOS)
- (the focus-steal sentinel + deprecated capture_mode compatibility tests now live under
- modality_background_test.rs — see TEST_SUITE.md for the full inventory)
-```
-
-## How the pieces connect
-
-```mermaid
-flowchart TD
- SoT["shared/scenarios.json<br/>(single source of truth:<br/>titles · AutomationIds · expected tree)"]
- WEB["shared/web/index.html"]
-
- subgraph APPS["apps/ — test-target GUI apps"]
- direction LR
- WPF["windows/wpf"]
- WINUI["windows/winui3"]
- WV["windows/webview2"]
- ELEC["cross-platform/electron"]
- AK["macos/appkit"]
- SW["macos/swiftui"]
- end
-
- BUILD["build/ → test-apps/harness-*<br/>(macos.sh · windows.ps1)"]
-
- subgraph TESTS["Rust integration tests (crates/cua-driver/tests/)"]
- direction LR
- T1["harness_wpf · winui3 · web_windows<br/>bg_modality · lo_vcl (Windows)"]
- T2["harness_appkit · swiftui (macOS)"]
- end
-
- DRIVER(["cua-driver tools<br/>(click / type / get_window_state / page / …)"])
-
- SoT -->|apps read at startup:<br/>set window titles & AutomationIds| APPS
- WEB --> WV
- WEB --> ELEC
- APPS --> BUILD
- BUILD -->|each test launches its built app| TESTS
- SoT -->|tests read: assert expected shape<br/>without hardcoding strings| TESTS
- TESTS -->|drive| DRIVER
- DRIVER -->|act on| APPS
- TESTS -->|re-snapshot & assert| TESTS
-
- classDef sot fill:#1f883d,stroke:#116329,color:#fff;
- class SoT,WEB sot;
-```
-
-The loop per test: **launch the built app → read `scenarios.json` for the expected shape → drive cua-driver against the app → re-snapshot and assert.** Because both sides read the same `scenarios.json`, a changed AutomationId or title updates the apps and the assertions together.
-
-## Coverage matrix
-
-| App (`apps/…`) | OS | `scenarios.json` key | What it exercises | Rust test(s) |
-|---|---|---|---|---|
-| `windows/wpf` | Windows | `wpf` | UIA Invoke, PostMessage type, right/double-click, scroll, modal MessageBox, owned + layered popups, native child HWNDs, accelerators | `harness_wpf_test`, `modality_background_test` |
-| `windows/winui3` | Windows | `winui3` | UIA ValuePattern text, CommandBarFlyout, XAML Popup primitive | `harness_winui3_test` |
-| `windows/webview2` | Windows | `webview` | Chromium DevTools Protocol (CDP) `page` tool over the shared web page | `harness_web_test` |
-| `cross-platform/electron` | Win / macOS / Linux | `electron` | CDP `page` tool over the shared web page | `harness_web_test` (+ macOS) |
-| `macos/appkit` | macOS | `appkit` | AX tree, NSButton AXPress, NSTextField, NSScrollView, NSMenu | `harness_appkit_test` |
-| `macos/swiftui` | macOS | `swiftui` | AX tree, SwiftUI `.popover()` (separate AXWindow) | `harness_swiftui_test` |
-| *(LibreOffice — not a harness app)* | Windows | — | VCL/SAL SplitButton, modal dialogs, color picker via MSAA fallback | `harness_libreoffice_test` |
-| *(WPF + focus sentinel)* | Windows | `wpf` | background-modality / no-focus-steal sentinel + `capture_mode` (deprecated/ignored) | `modality_background_test` |
-
-## Runtime requirements (why CI coverage is uneven)
-
-- **Linux** (Electron via CDP, plus the Nix GUI scenarios) → runs **headless in CI** (Xvfb / NixOS VMs on `ubuntu-latest`).
-- **Windows** (WPF / WinUI3 / WebView2 / bg-modality) and **macOS** (AppKit / SwiftUI) → need a **real interactive desktop** (UIA / AX / SendInput / z-order / TCC), so they need self-hosted runners; they're not in CI today.
-
-## TL;DR
-
-- **`shared/scenarios.json` is the contract.** Apps and tests both read it ⇒ no drift.
-- **Tree = topology:** `apps/{cross-platform,macos,windows}/…` says exactly where each runs; `shared/` is reusable.
-- **Tests live with the driver** (`crates/cua-driver/tests/harness_*`), launch the built app, and assert against the same scenarios.
-- The layout itself is the standardization — landed in **#1727**; Linux parity is still being extended in **#1693**.
diff --git a/libs/cua-driver/test-harness/TEST_SUITE.md b/libs/cua-driver/test-harness/TEST_SUITE.md
deleted file mode 100644
index c128ceba28..0000000000
--- a/libs/cua-driver/test-harness/TEST_SUITE.md
+++ /dev/null
@@ -1,128 +0,0 @@
-# cua-driver test suite — how the tests look
-
-The integration tests in `libs/cua-driver/rust/crates/cua-driver/tests/` all sit
-on one shared foundation — the **`cua-driver-testkit`** crate — and are named by
-a four-family taxonomy so `ls tests/` reads as a map. Companion:
-`TEST_HARNESS_STRUCTURE.md` (the harness *apps* the `harness_*` tests drive).
-
-## The shared foundation: `cua-driver-testkit`
-
-A dev-only crate (`crates/cua-driver-testkit`) that every test builds on, so no
-test re-implements process plumbing:
-
-- **Two transports, one shape.** `McpDriver` (one long-lived `cua-driver` server
- over stdio JSON-RPC, session-scoped state) and `CliDriver` (stateless
- `cua-driver call` per action) both implement the `Driver` trait and normalize
- their differing payloads — the CLI prints `structuredContent` directly, MCP
- returns the JSON-RPC envelope — into one `ToolResponse` (`text()` /
- `structured()` / `is_error()`). `RawDriver` adds raw send/recv with no
- auto-initialize for the protocol tests that drive the handshake themselves.
-- **`ChildReaper`** kills every spawned child on drop; on Windows it assigns them
- to a kill-on-close Job Object (no orphaned windows / held ports).
-- **`paths`** (`driver_binary()` resolves release-or-debug + `.exe`), **`ax`**
- (snapshot-text parsers), `McpDriver::find_window`.
-
-## The loop every behavior test runs
-
-```mermaid
-flowchart LR
- T["test (tests/*.rs)"]
- TK(["cua-driver-testkit<br/>McpDriver / CliDriver / RawDriver"])
- D(["cua-driver"])
- A["target app / harness app"]
- T -->|"Driver::call(tool, args)"| TK
- TK -->|"spawn + drive"| D
- D -->|"acts on"| A
- T -->|"re-snapshot & assert (ToolResponse)"| T
-```
-
-## Inventory by family (18 files)
-
-| Family | File | Platform | What it covers |
-|---|---|---|---|
-| **protocol_** | `protocol_handshake_test` | mac+win | initialize handshake, tools/list registration, per-tool capability + version fields, unknown-method/tool errors |
-| | `protocol_tools_call_test` | mac+win | per-tool `tools/call` shape: list_apps, get_config, permissions, AX tree, list_windows, screen size, scroll, type_text, hotkey, press_key, click, set_value |
-| | `protocol_schema_test` | mac+win | inputSchema shape for the `type_text` chars variants |
-| | `protocol_media_test` | mac+win | screenshot / zoom (jpeg) / recording / replay / screenshot-resize |
-| | `protocol_session_test` | mac+win | concurrent clients, multi-driver isolation, cursor-overlay liveness |
-| | `protocol_element_token_test` | mac/linux | Surface 6 opaque `element_token` in schema + capabilities |
-| **transport_** | `transport_config_persistence_test` | any | `set_config` persists to disk across stateless **CLI** invocations (#2034) vs visible within an **MCP** session |
-| **harness_** | `harness_wpf_test` | win | WPF: UIA Invoke, type, clicks, scroll, modal, owned/layered popups, native child HWNDs |
-| | `harness_winui3_test` | win | WinUI3: ValuePattern, CommandBarFlyout, XAML Popup |
-| | `harness_web_test` | win | WebView2 + Electron via CDP `page` |
-| | `harness_libreoffice_test` | win | LibreOffice VCL/SAL via MSAA (some `dispatch:"foreground"`) |
-| | `harness_appkit_test` | mac | AppKit: AX tree, AXPress, NSTextField, NSScrollView, NSMenu |
-| | `harness_swiftui_test` | mac | SwiftUI: AX tree, `.popover()` |
-| **modality_** | `modality_background_test` | win | background-modality / no-focus-steal sentinel + deprecated `capture_mode` values accepted but ignored |
-| | `modality_capture_mode_test` | mac+win+linux | deprecated `capture_mode` parameter on each native harness: values are accepted but ignored |
-| | `modality_input_e2e_test` | win | unified background input across Electron/Tauri/Win32, no z-raise |
-| | `modality_desktop_scope_test` | win | desktop-scope (foreground): `capture_scope=desktop`, `get_desktop_state`, window-less screen-absolute actions |
-| | `modality_focus_test` | mac | background automation does not steal focus |
-| **guard_** | `guard_ux_test` | win | UX guards: background focus, click-opens-window, launch-visible, menu shortcut |
-
-The `protocol_*` family is the renamed + split successor of the old
-`mcp_protocol_test` (3,412 lines / 65 tests). The split deduped 24 macOS↔Windows
-mirror pairs into single `cfg!`-branching tests, sharing one `RawDriver`.
-
-## Coverage on three axes: transport × modality × platform
-
-```
- ┌────────────── MODALITY ──────────────┐
- │ background foreground/desktop-scope│
- ─────────────────┼───────────────────────────────────────
- MCP transport │ ✅ thorough ✅ desktop_scope (win) │
- │ (background, focus (mac) │
- │ input_e2e, │
- │ focus, guard) │
- CLI transport │ ⬚ ✅ config-persistence │
- │ (transport_ test) │
- └───────────────────────────────────────┘
- Per-app AX (MCP): harness_{wpf,winui3,web,libreoffice}=win · {appkit,swiftui}=mac
- Protocol (RawDriver): mac+win, deduped; element_token = mac/linux
-```
-
-- **Background modality** is the most-covered: the sentinel oracle
- (`modality_background`), unified input (`modality_input_e2e`), the macOS
- no-focus-steal check (`modality_focus`), and the UX guards.
-- **Foreground / desktop-scope** is covered by `modality_desktop_scope`.
-- **Transport** is now a first-class axis: `transport_config_persistence`
- exercises CLI (disk) vs MCP (session) directly; most other tests run over MCP.
-
-### The modality matrix: action target × delivery mode × `capture_scope`
-
-The user-facing matrix is documented in
-`docs/content/docs/concepts/capture-and-delivery-modalities.mdx`. `get_window_state`
-always returns both the accessibility tree and a screenshot; AX vs pixel modality is
-chosen at action time by using `element_index` vs `x,y`. Coverage of its valid cells,
-per platform:
-
-| Cell (`scope`/action target/`delivery_mode`) | Windows | macOS | Linux |
-|---|---|---|---|
-| `window`/`element_index`/`background` (default) | `harness_*`, `modality_background` | `harness_{appkit,swiftui}`, `modality_focus`, `modality_capture_mode` | `harness_gtk3`, `modality_capture_mode` |
-| `window`/`element_index`/`foreground` | `harness_wpf` (`delivery_mode:"foreground"`) | activation differs | activation differs |
-| `window`/`x,y`/`background` | `modality_background`, `modality_capture_mode`, `modality_input_e2e` | `modality_capture_mode` | `modality_capture_mode` |
-| `window`/`x,y`/`foreground` | gap | n/a (no `bring_to_front`) | n/a (stubbed) |
-| `desktop`/`x,y`/`foreground` | `modality_desktop_scope` | rolling out | rolling out |
-| **negative gate** (`desktop_scope_disabled`) | `window_scope_rejects_windowless_click` | — | — |
-
-`capture_mode` is deprecated and ignored: `get_window_state` returns both the
-accessibility tree and a screenshot regardless of the parameter. The
-`modality_capture_mode_test` now exercises compatibility only: deprecated
-`ax`/`vision`/`som` values are accepted but do not change the returned shape.
-
-- **Remaining gaps** (honest): `window`/`x,y`/`foreground` has no explicit
- test on any platform; the desktop-scope window-less actuator is Windows-only
- (macOS click still requires `pid`), so the desktop cell and its negative gate
- are Windows-only until that loop lands on macOS/Linux; CLI transport is
- otherwise lightly exercised.
-
-## Where they run
-
-| Lane | Tests | Runner |
-|---|---|---|
-| **Runs without `#[ignore]`** | `protocol_*` (mac/win dev machines; linux excludes them via cfg) | local / VM |
-| **Interactive Windows** (`#[ignore]`) | `harness_{wpf,winui3,web,libreoffice}`, `modality_*` (win), `guard_ux`, `transport_*` | real desktop / VM |
-| **Interactive macOS** (`#[ignore]`) | `harness_{appkit,swiftui}`, `modality_focus` | logged-in session + TCC |
-
-Run an `#[ignore]` suite explicitly:
-`cargo test -p cua-driver --test <name> -- --ignored --nocapture --test-threads=1`
diff --git a/libs/cua-driver/test-harness/apps/README.md b/libs/cua-driver/test-harness/apps/README.md
index e0da6574b6..fabff8c5f9 100644
--- a/libs/cua-driver/test-harness/apps/README.md
+++ b/libs/cua-driver/test-harness/apps/README.md
@@ -1,30 +1,29 @@
# test-harness apps
-Each subdirectory is one self-contained host app cua-driver drives. They
-share fixtures via `../shared/scenarios.json` and (where relevant)
-`../shared/web/index.html` so AutomationIds, AX identifiers, and DOM
-ids never drift between the apps and the integration tests.
+Each subdirectory is one deterministic host app cua-driver can drive. Apps
+share fixtures through `../shared/scenarios.json` and, for webview hosts,
+`../shared/web/index.html`.
-| Path | Host OS | What it tests |
-|-------------------------------|-----------------|-----------------------------------------------------------------|
-| `cross-platform/electron` | any | Electron + CDP — exercises cua-driver's `page` tool |
-| `macos/appkit` | macOS 13+ | AppKit / Cocoa hosting — NSButton, NSScrollView, NSMenu, etc. |
-| `macos/swiftui` | macOS 13+ | SwiftUI hosting — `.popover()`, declarative state |
-| `windows/wpf` | Windows | WPF / UIA — popups, layered windows, modal MessageBox, hwndhost |
-| `windows/winui3` | Windows | WinUI3 unpackaged — DComp popups, XAML Popup primitive |
-| `windows/webview2` | Windows | Microsoft.Edge.WebView2 hosting the same DOM as Electron |
+| Path | Host OS | What it tests |
+| --- | --- | --- |
+| `cross-platform/electron` | any | Electron/Chromium accessibility and CDP |
+| `cross-platform/tauri` | any | Tauri/native-webview accessibility |
+| `linux/gtk3` | Linux | GTK3/AT-SPI widgets |
+| `macos/appkit` | macOS | AppKit controls and menus |
+| `macos/swiftui` | macOS | SwiftUI controls and popovers |
+| `macos/wkwebview` | macOS | WKWebView loading the shared DOM |
+| `windows/wpf` | Windows | WPF/UIA controls, popups, child HWNDs |
+| `windows/winui3` | Windows | WinUI3/XAML controls and popups |
+| `windows/webview2` | Windows | WebView2 loading the shared DOM |
-Built outputs land in `../../rust/test-apps/harness-{appkit,swiftui,wpf,winui3,webview,electron}/`
-via `../build/macos.sh` or `..\build\windows.ps1`. The Rust integration
-tests under `../../rust/crates/cua-driver/tests/harness_*.rs` consume
-those staged outputs.
+Built outputs land in `../../rust/test-apps/harness-<name>/` via
+`../build/<host>`.
Adding a new app:
-1. Pick the right OS bucket (or `cross-platform/` if Electron-equivalent).
-2. Mirror an existing app's layout — Swift `main.swift` for macOS,
- `.csproj` + XAML for Windows.
-3. Add a section to `../shared/scenarios.json` with the AutomationIds /
- AX identifiers / window title your tests will assert on.
-4. Wire it into `../build/{macos.sh,windows.ps1}` so it stages into
- `rust/test-apps/harness-<name>/`.
+1. Put native source under `apps/<platform>/<toolkit>/`, or
+ `apps/cross-platform/<toolkit>/` when the same source runs on multiple OSes.
+2. Reuse `shared/scenarios.json` and `shared/web/index.html` instead of copying
+ private ids.
+3. Stage into `rust/test-apps/harness-<toolkit>/`.
+4. Add or update the matching Rust/Python tests and this table.
diff --git a/libs/cua-driver/test-harness/apps/cross-platform/tauri/.gitignore b/libs/cua-driver/test-harness/apps/cross-platform/tauri/.gitignore
new file mode 100644
index 0000000000..3922fb1e49
--- /dev/null
+++ b/libs/cua-driver/test-harness/apps/cross-platform/tauri/.gitignore
@@ -0,0 +1,4 @@
+web/
+src-tauri/gen/
+src-tauri/target/
+src-tauri/Cargo.lock
diff --git a/libs/cua-driver/test-harness/apps/cross-platform/tauri/build.ps1 b/libs/cua-driver/test-harness/apps/cross-platform/tauri/build.ps1
new file mode 100644
index 0000000000..d7d82a6b85
--- /dev/null
+++ b/libs/cua-driver/test-harness/apps/cross-platform/tauri/build.ps1
@@ -0,0 +1,44 @@
+# build.ps1 - stage the Tauri test harness for cua-driver tests on Windows.
+#
+# Output: ../../../../rust/test-apps/harness-tauri/CuaTestHarness.Tauri.exe
+
+Set-StrictMode -Version Latest
+$ErrorActionPreference = "Stop"
+
+$tauriDir = Split-Path -Parent $MyInvocation.MyCommand.Definition
+$crossDir = Split-Path -Parent $tauriDir
+$appsDir = Split-Path -Parent $crossDir
+$harnessDir = Split-Path -Parent $appsDir
+$cuaDriverDir = Split-Path -Parent $harnessDir
+$testAppsDir = Join-Path $cuaDriverDir "rust\test-apps"
+$outDir = Join-Path $testAppsDir "harness-tauri"
+
+if (-not (Get-Command cargo -ErrorAction SilentlyContinue)) {
+ Write-Host "[ERROR] cargo not on PATH. Install Rust first." -ForegroundColor Red
+ exit 1
+}
+
+$webDir = Join-Path $tauriDir "web"
+if (-not (Test-Path $webDir)) { New-Item -ItemType Directory $webDir | Out-Null }
+Copy-Item (Join-Path $harnessDir "shared\web\*") $webDir -Recurse -Force
+
+$manifest = Join-Path $tauriDir "src-tauri\Cargo.toml"
+Write-Host "[BUILD] cargo build --release --manifest-path $manifest" -ForegroundColor Cyan
+$oldTargetDir = $env:CARGO_TARGET_DIR
+$env:CARGO_TARGET_DIR = Join-Path $tauriDir "src-tauri\target"
+try {
+ & cargo build --release --manifest-path $manifest
+} finally {
+ $env:CARGO_TARGET_DIR = $oldTargetDir
+}
+if ($LASTEXITCODE -ne 0) { throw "cargo build failed for Tauri harness" }
+
+$srcExe = Join-Path $tauriDir "src-tauri\target\release\cua-test-harness-tauri.exe"
+if (-not (Test-Path $srcExe)) {
+ throw "Tauri exe not found after build: $srcExe"
+}
+
+if (Test-Path $outDir) { Remove-Item $outDir -Recurse -Force }
+New-Item -ItemType Directory -Force $outDir | Out-Null
+Copy-Item $srcExe (Join-Path $outDir "CuaTestHarness.Tauri.exe") -Force
+Write-Host "[OK] Staged: $outDir\CuaTestHarness.Tauri.exe" -ForegroundColor Green
diff --git a/libs/cua-driver/test-harness/apps/cross-platform/tauri/build.sh b/libs/cua-driver/test-harness/apps/cross-platform/tauri/build.sh
new file mode 100755
index 0000000000..4aa7f363ac
--- /dev/null
+++ b/libs/cua-driver/test-harness/apps/cross-platform/tauri/build.sh
@@ -0,0 +1,69 @@
+#!/usr/bin/env bash
+# Stage the Tauri test harness for cua-driver tests on Linux and macOS.
+#
+# The app embeds the same shared/web/index.html DOM used by Electron,
+# WebView2, and WKWebView. Build output is copied into
+# rust/test-apps/harness-tauri/ with deterministic executable names.
+set -euo pipefail
+
+tauriDir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+crossDir="$(dirname "$tauriDir")"
+appsDir="$(dirname "$crossDir")"
+harnessDir="$(dirname "$appsDir")"
+cuaDriverDir="$(dirname "$harnessDir")"
+outDir="$cuaDriverDir/rust/test-apps/harness-tauri"
+platform="$(uname -s)"
+
+if ! command -v cargo >/dev/null 2>&1; then
+ echo "[ERROR] cargo not on PATH. Install Rust first." >&2
+ exit 1
+fi
+
+mkdir -p "$tauriDir/web"
+cp -r "$harnessDir/shared/web/." "$tauriDir/web/"
+
+echo "[BUILD] cargo build --release ..."
+CARGO_TARGET_DIR="$tauriDir/src-tauri/target" \
+ cargo build --release --manifest-path "$tauriDir/src-tauri/Cargo.toml"
+
+rm -rf "$outDir"
+mkdir -p "$outDir"
+
+if [ "$platform" = "Darwin" ]; then
+ srcBin="$tauriDir/src-tauri/target/release/cua-test-harness-tauri"
+ [ -x "$srcBin" ] || { echo "[ERROR] Tauri binary missing after build" >&2; exit 1; }
+
+ bundle="$outDir/CuaTestHarness.Tauri.app"
+ appMacOS="$bundle/Contents/MacOS"
+ mkdir -p "$appMacOS" "$bundle/Contents/Resources"
+ cp "$srcBin" "$appMacOS/CuaTestHarness.Tauri"
+ chmod +x "$appMacOS/CuaTestHarness.Tauri"
+
+ cat > "$bundle/Contents/Info.plist" <<'PLIST'
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
+ "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+ <key>CFBundleExecutable</key><string>CuaTestHarness.Tauri</string>
+ <key>CFBundleIdentifier</key><string>com.trycua.harness.tauri</string>
+ <key>CFBundleName</key><string>CuaTestHarness.Tauri</string>
+ <key>CFBundleDisplayName</key><string>CuaTestHarness.Tauri</string>
+ <key>CFBundlePackageType</key><string>APPL</string>
+ <key>CFBundleShortVersionString</key><string>1.0.0</string>
+ <key>CFBundleVersion</key><string>1</string>
+ <key>LSMinimumSystemVersion</key><string>13.0</string>
+ <key>NSHighResolutionCapable</key><true/>
+</dict>
+</plist>
+PLIST
+ xattr -cr "$bundle" 2>/dev/null || true
+ echo "[OK] Staged: $bundle"
+else
+ srcBin="$tauriDir/src-tauri/target/release/cua-test-harness-tauri"
+ [ -x "$srcBin" ] || { echo "[ERROR] Tauri binary missing after build" >&2; exit 1; }
+
+ cp "$srcBin" "$outDir/CuaTestHarness.Tauri"
+ chmod +x "$outDir/CuaTestHarness.Tauri"
+ echo "[OK] Staged: $outDir/CuaTestHarness.Tauri"
+fi
diff --git a/libs/cua-driver/test-harness/apps/cross-platform/tauri/package.json b/libs/cua-driver/test-harness/apps/cross-platform/tauri/package.json
new file mode 100644
index 0000000000..d487ea07e0
--- /dev/null
+++ b/libs/cua-driver/test-harness/apps/cross-platform/tauri/package.json
@@ -0,0 +1,10 @@
+{
+ "name": "cua-test-harness-tauri",
+ "version": "1.0.0",
+ "private": true,
+ "description": "cua-driver Tauri test harness loading the shared web DOM",
+ "scripts": {
+ "build": "cargo build --release --manifest-path src-tauri/Cargo.toml",
+ "dev": "cargo run --manifest-path src-tauri/Cargo.toml"
+ }
+}
diff --git a/libs/cua-driver/test-harness/apps/cross-platform/tauri/src-tauri/Cargo.toml b/libs/cua-driver/test-harness/apps/cross-platform/tauri/src-tauri/Cargo.toml
new file mode 100644
index 0000000000..2b6401a089
--- /dev/null
+++ b/libs/cua-driver/test-harness/apps/cross-platform/tauri/src-tauri/Cargo.toml
@@ -0,0 +1,12 @@
+[package]
+name = "cua-test-harness-tauri"
+version = "1.0.0"
+description = "cua-driver Tauri test harness"
+edition = "2021"
+rust-version = "1.77"
+
+[build-dependencies]
+tauri-build = { version = "2" }
+
+[dependencies]
+tauri = { version = "2", features = [] }
diff --git a/libs/cua-driver/test-harness/apps/cross-platform/tauri/src-tauri/build.rs b/libs/cua-driver/test-harness/apps/cross-platform/tauri/src-tauri/build.rs
new file mode 100644
index 0000000000..d860e1e6a7
--- /dev/null
+++ b/libs/cua-driver/test-harness/apps/cross-platform/tauri/src-tauri/build.rs
@@ -0,0 +1,3 @@
+fn main() {
+ tauri_build::build()
+}
diff --git a/libs/cua-driver/test-harness/apps/cross-platform/tauri/src-tauri/icons/icon.png b/libs/cua-driver/test-harness/apps/cross-platform/tauri/src-tauri/icons/icon.png
new file mode 100644
index 0000000000..4c53796631
Binary files /dev/null and b/libs/cua-driver/test-harness/apps/cross-platform/tauri/src-tauri/icons/icon.png differ
diff --git a/libs/cua-driver/test-harness/apps/cross-platform/tauri/src-tauri/src/main.rs b/libs/cua-driver/test-harness/apps/cross-platform/tauri/src-tauri/src/main.rs
new file mode 100644
index 0000000000..2331526d10
--- /dev/null
+++ b/libs/cua-driver/test-harness/apps/cross-platform/tauri/src-tauri/src/main.rs
@@ -0,0 +1,7 @@
+#![cfg_attr(not(debug_assertions), windows_subsystem = "windows")]
+
+fn main() {
+ tauri::Builder::default()
+ .run(tauri::generate_context!())
+ .expect("error while running CuaTestHarness.Tauri");
+}
diff --git a/libs/cua-driver/test-harness/apps/cross-platform/tauri/src-tauri/tauri.conf.json b/libs/cua-driver/test-harness/apps/cross-platform/tauri/src-tauri/tauri.conf.json
new file mode 100644
index 0000000000..c994485b39
--- /dev/null
+++ b/libs/cua-driver/test-harness/apps/cross-platform/tauri/src-tauri/tauri.conf.json
@@ -0,0 +1,28 @@
+{
+ "$schema": "https://schema.tauri.app/config/2",
+ "productName": "CuaTestHarness.Tauri",
+ "version": "1.0.0",
+ "identifier": "com.trycua.harness.tauri",
+ "build": {
+ "beforeBuildCommand": "",
+ "beforeDevCommand": "",
+ "frontendDist": "../web"
+ },
+ "app": {
+ "windows": [
+ {
+ "label": "main",
+ "title": "CuaTestHarness Tauri",
+ "width": 900,
+ "height": 760,
+ "resizable": true
+ }
+ ],
+ "security": {
+ "csp": null
+ }
+ },
+ "bundle": {
+ "active": false
+ }
+}
diff --git a/libs/cua-driver/test-harness/build/README.md b/libs/cua-driver/test-harness/build/README.md
index d7aa5cd09a..156654ecc5 100644
--- a/libs/cua-driver/test-harness/build/README.md
+++ b/libs/cua-driver/test-harness/build/README.md
@@ -1,22 +1,23 @@
# test-harness build scripts
-Run `macos.sh` or `windows.ps1` to build every harness app for that host
-OS. Outputs are staged into `../../rust/test-apps/harness-<name>/` so
-the Rust integration tests and the Windows Sandbox runner pick them up
-unchanged.
+These host-specific scripts stage source-built harness apps into
+`../../rust/test-apps/harness-<name>/`.
-```bash
-# macOS — builds appkit + swiftui via xcrun swiftc into .app bundles
-./macos.sh
+| Script | Host | Builds |
+| --- | --- | --- |
+| `macos.sh` | macOS | AppKit, SwiftUI, WKWebView, Electron, Tauri |
+| `linux.sh` | Linux | GTK3, Electron, Tauri |
+| `windows.ps1` | Windows | WPF, WinUI3, WebView2, Electron, Tauri |
+
+Each script supports skipping one target while iterating locally:
-# Windows — dotnet publish for wpf/winui3/webview2 + electron-builder for electron
-.\windows.ps1
+```bash
+./macos.sh --skip electron
+./linux.sh --skip tauri
```
-Both scripts accept a `--skip` / `-Skip` argument to build one app at a
-time during local iteration; see the file headers for the full surface.
+```powershell
+.\windows.ps1 -Skip webview
+```
-Cross-platform Electron is built by `windows.ps1` today (which runs
-`apps/cross-platform/electron/build.ps1`). When a non-Windows test path
-needs the Electron host, lift the staging step into `macos.sh` (or
-`linux.sh`).
+Build products are local artifacts. Rebuild them instead of committing them.
diff --git a/libs/cua-driver/test-harness/build/linux.sh b/libs/cua-driver/test-harness/build/linux.sh
index ac22a184e4..4f8fabf586 100755
--- a/libs/cua-driver/test-harness/build/linux.sh
+++ b/libs/cua-driver/test-harness/build/linux.sh
@@ -1,27 +1,54 @@
#!/usr/bin/env bash
-# Stage the Linux GTK3 test-harness app into ../../rust/test-apps/harness-gtk3/ —
-# the Linux peer of macos.sh (AppKit/SwiftUI) and windows.ps1 (WPF/WinUI3).
+# Stage Linux-runnable test-harness apps into ../../rust/test-apps/harness-*/.
#
-# The app is PyGObject (GTK3): a real GTK3 widget tree (identical AT-SPI exposure
-# to a C app), so there's no compile step — we stage main.py plus an executable
-# launcher named CuaTestHarness.Gtk3 (matching scenarios.json's exe_relative_path).
+# GTK3 is PyGObject: a real GTK3 widget tree (identical AT-SPI exposure to a C
+# app), so there's no compile step. Electron and Tauri are shared cross-platform
+# harnesses built from apps/cross-platform/.
#
-# Runtime deps (NOT installed here): python3-gi, gir1.2-gtk-3.0, at-spi2-core.
-# apt-get install -y python3-gi gir1.2-gtk-3.0 at-spi2-core
+# Runtime deps (NOT installed here): python3-gi, gir1.2-gtk-3.0, at-spi2-core,
+# Node.js/npm for Electron, Rust + WebKitGTK build deps for Tauri.
#
-# Usage: ./linux.sh [--clean]
+# Usage:
+# ./linux.sh
+# ./linux.sh --skip gtk3 # skip one target (gtk3|electron|tauri)
+# ./linux.sh --clean
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
HARNESS_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
-STAGE="$(cd "$HARNESS_DIR/../rust/test-apps" && pwd)/harness-gtk3"
+TEST_APPS_DIR="$(cd "$HARNESS_DIR/../rust/test-apps" && pwd)"
+STAGE="$TEST_APPS_DIR/harness-gtk3"
SRC="$HARNESS_DIR/apps/linux/gtk3"
+SKIP="none"
+CLEAN=0
-rm -rf "$STAGE"
-mkdir -p "$STAGE"
-cp "$SRC/main.py" "$STAGE/main.py"
+while [[ $# -gt 0 ]]; do
+ case "$1" in
+ --skip)
+ SKIP="${2:-}"
+ shift 2
+ ;;
+ --clean)
+ CLEAN=1
+ shift
+ ;;
+ *)
+ echo "Usage: $0 [--skip gtk3|electron|tauri] [--clean]" >&2
+ exit 2
+ ;;
+ esac
+done
-cat > "$STAGE/CuaTestHarness.Gtk3" <<'LAUNCHER'
+if [[ "$CLEAN" == "1" ]]; then
+ rm -rf "$TEST_APPS_DIR/harness-gtk3" "$TEST_APPS_DIR/harness-electron" "$TEST_APPS_DIR/harness-tauri"
+fi
+
+if [[ "$SKIP" != "gtk3" ]]; then
+ rm -rf "$STAGE"
+ mkdir -p "$STAGE"
+ cp "$SRC/main.py" "$STAGE/main.py"
+
+ cat > "$STAGE/CuaTestHarness.Gtk3" <<'LAUNCHER'
#!/usr/bin/env bash
# Force the X11 backend so the window is enumerable via cua-driver's X11
# list_windows (_NET_CLIENT_LIST) — under a Wayland session this routes through
@@ -29,11 +56,20 @@ cat > "$STAGE/CuaTestHarness.Gtk3" <<'LAUNCHER'
export GDK_BACKEND=x11
exec python3 "$(dirname "$(readlink -f "$0")")/main.py" "$@"
LAUNCHER
-chmod +x "$STAGE/CuaTestHarness.Gtk3"
+ chmod +x "$STAGE/CuaTestHarness.Gtk3"
+
+ echo "==> Staged GTK3 harness -> $STAGE/CuaTestHarness.Gtk3"
+ if python3 -c "import gi; gi.require_version('Gtk','3.0'); from gi.repository import Gtk" 2>/dev/null; then
+ echo "==> PyGObject/GTK3 present"
+ else
+ echo "WARNING: PyGObject/GTK3 not importable - install python3-gi gir1.2-gtk-3.0 at-spi2-core" >&2
+ fi
+fi
+
+if [[ "$SKIP" != "electron" ]]; then
+ "$HARNESS_DIR/apps/cross-platform/electron/build.sh"
+fi
-echo "==> Staged GTK3 harness → $STAGE/CuaTestHarness.Gtk3"
-if python3 -c "import gi; gi.require_version('Gtk','3.0'); from gi.repository import Gtk" 2>/dev/null; then
- echo "==> PyGObject/GTK3 present"
-else
- echo "WARNING: PyGObject/GTK3 not importable — install python3-gi gir1.2-gtk-3.0 at-spi2-core" >&2
+if [[ "$SKIP" != "tauri" ]]; then
+ "$HARNESS_DIR/apps/cross-platform/tauri/build.sh"
fi
diff --git a/libs/cua-driver/test-harness/build/macos.sh b/libs/cua-driver/test-harness/build/macos.sh
index 1bd8e0a756..233e464b72 100755
--- a/libs/cua-driver/test-harness/build/macos.sh
+++ b/libs/cua-driver/test-harness/build/macos.sh
@@ -1,13 +1,14 @@
#!/usr/bin/env bash
-# Build the macOS test-harness apps (AppKit + SwiftUI + WKWebView + Electron)
+# Build the macOS test-harness apps (AppKit + SwiftUI + WKWebView + Electron + Tauri)
# into app bundles under ../../rust/test-apps/harness-* —
# matching the convention used by windows.ps1 for the Windows WPF + WinUI3 +
# WebView2 apps. WKWebView is the Apple-WebKit host; Electron is the shared
-# Chromium harness used across Windows, Linux, and macOS.
+# Chromium harness used across Windows, Linux, and macOS. Tauri is the shared
+# native-webview harness used across Windows, Linux, and macOS.
#
# Usage:
# ./macos.sh # build all macOS-runnable harnesses
-# ./macos.sh --skip swiftui # skip one target (appkit|swiftui|wkwebview|electron)
+# ./macos.sh --skip swiftui # skip one target (appkit|swiftui|wkwebview|electron|tauri)
# ./macos.sh --clean # remove staged outputs first
set -euo pipefail
@@ -28,8 +29,8 @@ while [[ $# -gt 0 ]]; do
done
if [[ "$CLEAN" == "1" ]]; then
- rm -rf "$STAGE_DIR/harness-appkit" "$STAGE_DIR/harness-swiftui" "$STAGE_DIR/harness-wkwebview" "$STAGE_DIR/harness-electron"
- mkdir -p "$STAGE_DIR/harness-appkit" "$STAGE_DIR/harness-swiftui" "$STAGE_DIR/harness-wkwebview" "$STAGE_DIR/harness-electron"
+ rm -rf "$STAGE_DIR/harness-appkit" "$STAGE_DIR/harness-swiftui" "$STAGE_DIR/harness-wkwebview" "$STAGE_DIR/harness-electron" "$STAGE_DIR/harness-tauri"
+ mkdir -p "$STAGE_DIR/harness-appkit" "$STAGE_DIR/harness-swiftui" "$STAGE_DIR/harness-wkwebview" "$STAGE_DIR/harness-electron" "$STAGE_DIR/harness-tauri"
echo "==> Cleaned stage dirs"
fi
@@ -108,6 +109,10 @@ if [[ "$SKIP" != "electron" ]]; then
"$HARNESS_DIR/apps/cross-platform/electron/build.sh"
fi
+if [[ "$SKIP" != "tauri" ]]; then
+ "$HARNESS_DIR/apps/cross-platform/tauri/build.sh"
+fi
+
echo ""
echo "Built apps:"
find "$STAGE_DIR" -maxdepth 3 -name "*.app" -type d
diff --git a/libs/cua-driver/test-harness/build/windows.ps1 b/libs/cua-driver/test-harness/build/windows.ps1
index 23e0f8127e..aafe772a2b 100644
--- a/libs/cua-driver/test-harness/build/windows.ps1
+++ b/libs/cua-driver/test-harness/build/windows.ps1
@@ -1,6 +1,6 @@
# windows.ps1 - publish the cua-driver test harness binaries.
#
-# Outputs land in libs/cua-driver/rust/test-apps/harness-{wpf,winui3,webview,electron}/
+# Outputs land in libs/cua-driver/rust/test-apps/harness-{wpf,winui3,webview,electron,tauri}/
# so the existing sandbox runner picks them up via its mapped folder.
#
# Usage:
@@ -8,10 +8,10 @@
# .\windows.ps1 -Skip wpf # skip WPF
# .\windows.ps1 -Skip winui3
#
-# Requires: .NET 8 SDK on PATH (Node.js + npm for the Electron app).
+# Requires: .NET 8 SDK on PATH (Node.js + npm for Electron, Rust for Tauri).
param(
- [ValidateSet("none","wpf","winui3","webview","electron")]
+ [ValidateSet("none","wpf","winui3","webview","electron","tauri")]
[string]$Skip = "none"
)
@@ -92,6 +92,23 @@ if ($Skip -ne "electron") {
Write-Host "[SKIP] Electron project not present yet - skipping." -ForegroundColor Yellow
}
}
+if ($Skip -ne "tauri") {
+ $tauriBuild = Join-Path $harnessDir "apps\cross-platform\tauri\build.ps1"
+ if (Test-Path $tauriBuild) {
+ Write-Host ""
+ Write-Host "[BUILD] tauri -> $testAppsDir\harness-tauri\" -ForegroundColor Cyan
+ try {
+ & $tauriBuild
+ if ($LASTEXITCODE -ne 0) {
+ Write-Host "[WARN] Tauri build failed (exit $LASTEXITCODE)" -ForegroundColor Yellow
+ }
+ } catch {
+ Write-Host "[WARN] Tauri build errored: $($_.Exception.Message)" -ForegroundColor Yellow
+ }
+ } else {
+ Write-Host "[SKIP] Tauri project not present yet - skipping." -ForegroundColor Yellow
+ }
+}
Write-Host ""
Write-Host "[DONE] Test harness build complete." -ForegroundColor Green
diff --git a/libs/cua-driver/test-harness/smoke/results/macos.txt b/libs/cua-driver/test-harness/smoke/results/macos.txt
deleted file mode 100644
index d18911ccff..0000000000
--- a/libs/cua-driver/test-harness/smoke/results/macos.txt
+++ /dev/null
@@ -1,72 +0,0 @@
-============================================================
-macOS cua-driver smoke test
-Driver: ~/cua/libs/cua-driver/rust/target/release/cua-driver
-Version: cua-driver 0.2.18
-Harness: ~/cua/libs/cua-driver/rust/test-apps/harness-appkit/CuaTestHarness.AppKit.app
-Date: Tue May 26 01:00:13 CEST 2026
-============================================================
-
-==> Spawning AppKit harness as victim
- Harness pid=52884
- Window id=18195
-
-=== group 1: no-arg / config tools ===
-
-=== group 2: setters ===
-
-=== group 3: app lifecycle ===
- TextEdit pid=55979
-
-=== group 4: per-window state (harness pid=52884 wid=18195) ===
-
-=== group 5: input synthesis (against harness) ===
-
-=== group 6: aliases + special ===
-
-==> cleanup
-~/cua/scripts/mac-smoke.sh: line 184: 52884 Killed: 9 "$HARNESS_EXE" > /dev/null 2>&1
-
-============================================================
-SUMMARY
-============================================================
-TOOL VERDICT DETAIL
-----------------------------------------------------------------
-bring_to_front SKIP intentional cross-platform stub on macOS
-check_permissions PASS {
-click PASS ✅ Posted click to pid 52884.
-double_click PASS ✅ Double-clicked at (670.0, 102.0).
-drag PASS ✅ Posted drag to pid 52884 from window-pixel (120, 80) → (180, 120), screen (670
-get_accessibility_tree PASS {
-get_agent_cursor_state PASS {
-get_config PASS {
-get_cursor_position PASS {
-get_recording_state PASS {
-get_screen_size PASS {
-get_window_state PASS {
-hotkey PASS Pressed cmd+a on pid 52884.
-kill_app PASS ✅ Sent SIGKILL to pid 55979.
-launch_app PASS {
-list_apps PASS {
-list_windows PASS {
-move_cursor PASS Agent cursor 'default' moved to (400.0, 400.0).
-page SKIP needs Chromium with --remote-debugging-port
-press_key PASS ✅ Pressed a on pid 52884.
-replay_trajectory SKIP needs a recorded trajectory file
-right_click PASS Right-clicked at (670.0, 102.0).
-scroll PASS Scrolled down by line × 3.
-set_agent_cursor_enabled PASS Agent cursor 'default' enabled.
-set_agent_cursor_motion PASS {
-set_agent_cursor_style PASS ✅ cursor style: gradient_colors=(unchanged) bloom_color=(unchanged) image_path=(
-set_config PASS Config updated: capture_mode=som, max_image_dimension=1024
-set_recording PASS {
-set_value SKIP exercised by harness_appkit_text_input integration test instead
-type_text PASS ✅ Inserted 2 char(s) into focused AXTextField "".
-type_text_chars SKIP deprecated alias resolved by mcp-server, not registered
-zoom PASS {
-
-Tools probed: 32 PASS=27 FAIL=0 SKIP=5
-
-Interpretation:
- PASS = tool ran cleanly (exit 0, no ❌ in output)
- FAIL = error or non-zero exit
- SKIP = intentionally not probed (covered by integration tests, missing fixture, etc.)
diff --git a/libs/cua-driver/rust/tests/integration/v2/PHILOSOPHY.md b/libs/cua-driver/tests/e2e/PHILOSOPHY.md
similarity index 82%
rename from libs/cua-driver/rust/tests/integration/v2/PHILOSOPHY.md
rename to libs/cua-driver/tests/e2e/PHILOSOPHY.md
index ec1b83144e..5e0d62aeba 100644
--- a/libs/cua-driver/rust/tests/integration/v2/PHILOSOPHY.md
+++ b/libs/cua-driver/tests/e2e/PHILOSOPHY.md
@@ -1,4 +1,4 @@
-# cua-driver v2 Integration Test Philosophy
+# cua-driver Python E2E Test Philosophy
## Core Principle
@@ -65,12 +65,12 @@ fails with a list of every incident.
| App | Primary oracle | Cross-check |
|-----|---------------|-------------|
-| Safari | AX tree text values | — |
-| Chrome | AX tree text values | — |
-| Electron | AX tree text values | HTTP API on port 6769 |
-| Tauri | AX tree text values | HTTP API on port 6769 |
+| Safari | AX tree text values | - |
+| Chrome | AX tree text values | - |
+| Electron | AX tree text values from shared DOM | - |
+| Tauri | AX tree text values from shared DOM | - |
| Blender | Screenshot + NCC template match | AX tree window title |
-| Calculator | AX tree static text | — |
+| Calculator | AX tree static text | - |
For browser apps the test page (`assets/test_page.html`) provides a
self-contained oracle: every interactive element reflects its state back into
@@ -95,28 +95,13 @@ assert cv.diff_ratio(before_crop, after_crop) > 0.001
---
-## Tauri / Electron HTTP API
-
-The HTTP API is a **cross-check**, not the primary assertion:
-
-```python
-# Primary: AX tree
-after = get_window_state(tauri_pid, wid)
-assert ax_value(after.tree, field_idx) == "tauri test"
-
-# Cross-check: HTTP event log
-events = requests.get("http://localhost:6769/events").json()
-assert any(e["type"] == "key_down" for e in events)
-```
-
----
-
## Test File Structure
```
-v2/
+tests/e2e/
PHILOSOPHY.md ← this file
conftest.py ← pytest fixtures (driver, ux_guard, sentinel, html_server, ...)
+ driver_client.py ← raw MCP client wrapper
harness/
__init__.py
monitor.py ← UXMonitor (5 ms PyObjC thread)
@@ -126,12 +111,13 @@ v2/
assets/
test_page.html ← all-elements test page for browser tests
blender/
- bboxes.json ← pre-measured bounding boxes for template targets
+ toolbar_ref.png ← stored template target
test_safari.py
test_chrome.py
test_electron.py
test_tauri.py
test_blender.py
+ test_*.py ← older e2e coverage kept here until audited/ported
```
---
@@ -139,8 +125,8 @@ v2/
## Running
```bash
-cd tests/integration/v2
-export CUA_DRIVER_BINARY=../../../target/debug/cua-driver
+cd libs/cua-driver/tests/e2e
+export CUA_DRIVER_BINARY=../../rust/target/debug/cua-driver
# Single file
python3 -m pytest test_safari.py -v
diff --git a/libs/cua-driver/tests/e2e/README.md b/libs/cua-driver/tests/e2e/README.md
new file mode 100644
index 0000000000..463f03bf71
--- /dev/null
+++ b/libs/cua-driver/tests/e2e/README.md
@@ -0,0 +1,46 @@
+# cua-driver Python e2e tests
+
+Python e2e tests drive the public MCP tool interface against real apps and
+shared harness apps. They are manual/VM-backed tests, not default CI tests.
+
+## Layout
+
+```text
+tests/e2e/
+├── driver_client.py # raw MCP client
+├── conftest.py # pytest fixtures
+├── harness/ # typed wrappers, tree helpers, CV helpers, UX monitor
+├── assets/ # symlinks/references for browser and Blender tests
+├── PHILOSOPHY.md # black-box test pattern and UX invariants
+├── run_tests.sh # pytest runner that builds cua-driver if needed
+└── test_*.py # app and legacy e2e modules
+```
+
+## Running
+
+```bash
+cd libs/cua-driver/tests/e2e
+./run_tests.sh test_electron -v
+python3 -m pytest test_tauri.py -v
+```
+
+Set `CUA_DRIVER_BINARY` to test a specific binary. Without it, `run_tests.sh`
+builds `libs/cua-driver/rust` and uses `rust/target/debug/cua-driver`.
+
+## Harness Apps
+
+`test_electron.py` and `test_tauri.py` use repo-local shared harnesses:
+
+- `test-harness/apps/cross-platform/electron`
+- `test-harness/apps/cross-platform/tauri`
+
+The fixtures build missing staged apps on demand with each harness `build.sh`.
+They no longer download prebuilt app releases.
+
+## Rules
+
+- Assert through observable state in the AX/UIA/AT-SPI tree or screenshots.
+- Do not call internal browser APIs from tests.
+- Keep app-specific setup in fixtures, not in driver code.
+- Treat older root-level tests as legacy coverage until they are audited and
+ either ported into the shared harness pattern or removed.
diff --git a/libs/cua-driver/rust/tests/integration/v2/assets/blender/toolbar_ref.png b/libs/cua-driver/tests/e2e/assets/blender/toolbar_ref.png
similarity index 100%
rename from libs/cua-driver/rust/tests/integration/v2/assets/blender/toolbar_ref.png
rename to libs/cua-driver/tests/e2e/assets/blender/toolbar_ref.png
diff --git a/libs/cua-driver/tests/e2e/assets/test_page.html b/libs/cua-driver/tests/e2e/assets/test_page.html
new file mode 120000
index 0000000000..2771775117
--- /dev/null
+++ b/libs/cua-driver/tests/e2e/assets/test_page.html
@@ -0,0 +1 @@
+../../../../cua-driver-fixtures/test_page.html
\ No newline at end of file
diff --git a/libs/cua-driver/rust/tests/integration/v2/conftest.py b/libs/cua-driver/tests/e2e/conftest.py
similarity index 82%
rename from libs/cua-driver/rust/tests/integration/v2/conftest.py
rename to libs/cua-driver/tests/e2e/conftest.py
index 8580c8f429..d6f818dfe6 100644
--- a/libs/cua-driver/rust/tests/integration/v2/conftest.py
+++ b/libs/cua-driver/tests/e2e/conftest.py
@@ -1,4 +1,4 @@
-"""pytest fixtures for cua-driver v2 integration tests.
+"""pytest fixtures for cua-driver Python e2e tests.
Fixtures:
binary → str — path to cua-driver binary
@@ -6,7 +6,7 @@ Fixtures:
focus_monitor → (proc, pid) — FocusMonitorApp running as frontmost sentinel
ux_guard → UXMonitor — started before the test, assert_clean() after
html_server → str — base URL of a local HTTP server serving assets/
- tauri_app → (proc, pid, base_url) — Tauri desktop-test-app
+ tauri_app → (proc, pid, base_url) — repo-local Tauri harness
electron_app → (proc, pid, base_url) — repo-local Electron harness
"""
@@ -17,20 +17,18 @@ import subprocess
import sys
import time
import threading
-import urllib.request
from http.server import HTTPServer, SimpleHTTPRequestHandler
import pytest
_HERE = os.path.dirname(os.path.abspath(__file__))
-_INTEG_DIR = os.path.dirname(_HERE)
-_TESTS_DIR = os.path.dirname(_INTEG_DIR)
-_DRIVER_RS_ROOT = os.path.dirname(_TESTS_DIR)
-_LIBS_ROOT = os.path.dirname(_DRIVER_RS_ROOT)
-_CUA_DRIVER_ROOT = os.path.join(_LIBS_ROOT, "cua-driver")
+_TESTS_ROOT = os.path.dirname(_HERE)
+_CUA_DRIVER_ROOT = os.path.dirname(_TESTS_ROOT)
+_DRIVER_RS_ROOT = os.path.join(_CUA_DRIVER_ROOT, "rust")
+_LIBS_ROOT = os.path.dirname(_CUA_DRIVER_ROOT)
_TEST_HARNESS_ROOT = os.path.join(_CUA_DRIVER_ROOT, "test-harness")
-sys.path.insert(0, _INTEG_DIR)
+sys.path.insert(0, _HERE)
from driver_client import default_binary_path # noqa: E402
_ASSETS_DIR = os.path.join(_HERE, "assets")
@@ -39,16 +37,20 @@ _FOCUS_APP_DIR = os.path.join(_LIBS_ROOT, "cua-driver", "Tests", "FocusMonitorAp
_FOCUS_APP_BUNDLE = os.path.join(_FOCUS_APP_DIR, "FocusMonitorApp.app")
_FOCUS_APP_EXE = os.path.join(_FOCUS_APP_BUNDLE, "Contents", "MacOS", "FocusMonitorApp")
-# Tauri app download config
-# Tauri releases a raw Mach-O binary (no .app bundle, no zip).
-_TAURI_VERSION = "v0.2.2"
-_TAURI_BINARY_NAME = "desktop-test-app-macos-arm64"
-_TAURI_DOWNLOAD_URL = (
- f"https://github.com/trycua/desktop-test-app/releases/download/"
- f"{_TAURI_VERSION}/{_TAURI_BINARY_NAME}"
+# Tauri is built from the repo-local shared harness rather than downloaded.
+_TAURI_HARNESS_DIR = os.path.join(
+ _TEST_HARNESS_ROOT, "apps", "cross-platform", "tauri"
+)
+_TAURI_BUILD_SCRIPT = os.path.join(_TAURI_HARNESS_DIR, "build.sh")
+_TAURI_APP_BUNDLE = os.path.join(
+ _DRIVER_RS_ROOT,
+ "test-apps",
+ "harness-tauri",
+ "CuaTestHarness.Tauri.app",
+)
+_TAURI_APP_EXE = os.path.join(
+ _TAURI_APP_BUNDLE, "Contents", "MacOS", "CuaTestHarness.Tauri"
)
-_TAURI_APP_DIR = os.path.join(_HERE, "assets", "tauri")
-_TAURI_BINARY_PATH = os.path.join(_TAURI_APP_DIR, _TAURI_BINARY_NAME)
# Electron is built from the repo-local shared harness rather than downloaded.
_ELECTRON_HARNESS_DIR = os.path.join(
@@ -201,42 +203,39 @@ def html_server():
server.shutdown()
-# ── Tauri / Electron helpers ──────────────────────────────────────────────────
-
-
-def _download_binary(url: str, dest_path: str) -> None:
- """Download a raw binary; skip if it already exists."""
- if os.path.exists(dest_path):
- return
- os.makedirs(os.path.dirname(dest_path), exist_ok=True)
- print(f"\n Downloading {url} …")
- urllib.request.urlretrieve(url, dest_path)
- os.chmod(dest_path, 0o755)
-
-
# ── Tauri app ─────────────────────────────────────────────────────────────────
@pytest.fixture(scope="session")
def tauri_app():
- """Download, launch, and yield (proc, pid, base_url) for the Tauri test app.
+ """Build, launch, and yield (proc, pid, base_url) for the Tauri harness.
- The Tauri release ships a raw Mach-O binary (no .app bundle, no zip).
- We launch it directly via subprocess.Popen.
+ This uses test-harness/apps/cross-platform/tauri, the same shared DOM
+ harness used by the Electron/WebView/WKWebView harnesses. It does not
+ download a prebuilt Tauri app release.
"""
- _download_binary(_TAURI_DOWNLOAD_URL, _TAURI_BINARY_PATH)
+ if not os.path.exists(_TAURI_APP_EXE):
+ if not os.path.exists(_TAURI_BUILD_SCRIPT):
+ pytest.skip("repo-local Tauri harness build script not found")
+ try:
+ subprocess.run([_TAURI_BUILD_SCRIPT], check=True)
+ except (OSError, subprocess.CalledProcessError) as e:
+ pytest.skip(f"Tauri harness build failed: {e}")
+
+ if not os.path.exists(_TAURI_APP_EXE):
+ pytest.skip("Tauri harness app not found after build")
- subprocess.run(["pkill", "-f", _TAURI_BINARY_NAME], check=False)
+ subprocess.run(["pkill", "-f", "CuaTestHarness.Tauri"], check=False)
time.sleep(0.5)
proc = subprocess.Popen(
- [_TAURI_BINARY_PATH],
+ [_TAURI_APP_EXE],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
)
pid = proc.pid
time.sleep(3.0)
- base_url = "http://localhost:6769"
+ base_url = "file://shared-web-harness"
yield proc, pid, base_url
proc.terminate()
@@ -244,7 +243,7 @@ def tauri_app():
proc.wait(timeout=5)
except subprocess.TimeoutExpired:
proc.kill()
- subprocess.run(["pkill", "-f", _TAURI_BINARY_NAME], check=False)
+ subprocess.run(["pkill", "-f", "CuaTestHarness.Tauri"], check=False)
# ── Electron app ──────────────────────────────────────────────────────────────
diff --git a/libs/cua-driver/rust/tests/integration/driver_client.py b/libs/cua-driver/tests/e2e/driver_client.py
similarity index 99%
rename from libs/cua-driver/rust/tests/integration/driver_client.py
rename to libs/cua-driver/tests/e2e/driver_client.py
index 850b16abeb..e6aefd43d2 100644
--- a/libs/cua-driver/rust/tests/integration/driver_client.py
+++ b/libs/cua-driver/tests/e2e/driver_client.py
@@ -153,6 +153,7 @@ def default_binary_path() -> str:
os.path.dirname(os.path.abspath(__file__)),
"..",
"..",
+ "rust",
"target",
"debug",
"cua-driver",
diff --git a/libs/cua-driver/rust/tests/integration/fixtures/interactive.html b/libs/cua-driver/tests/e2e/fixtures/interactive.html
similarity index 100%
rename from libs/cua-driver/rust/tests/integration/fixtures/interactive.html
rename to libs/cua-driver/tests/e2e/fixtures/interactive.html
diff --git a/libs/cua-driver/rust/tests/integration/fixtures/test_cursor.png b/libs/cua-driver/tests/e2e/fixtures/test_cursor.png
similarity index 100%
rename from libs/cua-driver/rust/tests/integration/fixtures/test_cursor.png
rename to libs/cua-driver/tests/e2e/fixtures/test_cursor.png
diff --git a/libs/cua-driver/rust/tests/integration/v2/harness/__init__.py b/libs/cua-driver/tests/e2e/harness/__init__.py
similarity index 100%
rename from libs/cua-driver/rust/tests/integration/v2/harness/__init__.py
rename to libs/cua-driver/tests/e2e/harness/__init__.py
diff --git a/libs/cua-driver/rust/tests/integration/v2/harness/cv.py b/libs/cua-driver/tests/e2e/harness/cv.py
similarity index 100%
rename from libs/cua-driver/rust/tests/integration/v2/harness/cv.py
rename to libs/cua-driver/tests/e2e/harness/cv.py
diff --git a/libs/cua-driver/rust/tests/integration/v2/harness/driver.py b/libs/cua-driver/tests/e2e/harness/driver.py
similarity index 97%
rename from libs/cua-driver/rust/tests/integration/v2/harness/driver.py
rename to libs/cua-driver/tests/e2e/harness/driver.py
index cdec18f487..1409768f5b 100644
--- a/libs/cua-driver/rust/tests/integration/v2/harness/driver.py
+++ b/libs/cua-driver/tests/e2e/harness/driver.py
@@ -1,4 +1,4 @@
-"""Typed MCP driver wrapper for v2 integration tests.
+"""Typed MCP driver wrapper for Python e2e tests.
Wraps DriverClient with:
- Typed WindowState return value
@@ -23,11 +23,10 @@ import time
from dataclasses import dataclass, field
from typing import Optional
-# Allow importing from parent directory (old driver_client.py).
+# Allow importing from the e2e root.
_HERE = os.path.dirname(os.path.abspath(__file__))
-_V2_DIR = os.path.dirname(_HERE)
-_INTEG_DIR = os.path.dirname(_V2_DIR)
-sys.path.insert(0, _INTEG_DIR)
+_E2E_DIR = os.path.dirname(_HERE)
+sys.path.insert(0, _E2E_DIR)
from driver_client import DriverClient, default_binary_path, MCPCallError # noqa: E402
diff --git a/libs/cua-driver/rust/tests/integration/v2/harness/monitor.py b/libs/cua-driver/tests/e2e/harness/monitor.py
similarity index 100%
rename from libs/cua-driver/rust/tests/integration/v2/harness/monitor.py
rename to libs/cua-driver/tests/e2e/harness/monitor.py
diff --git a/libs/cua-driver/rust/tests/integration/v2/harness/tree.py b/libs/cua-driver/tests/e2e/harness/tree.py
similarity index 100%
rename from libs/cua-driver/rust/tests/integration/v2/harness/tree.py
rename to libs/cua-driver/tests/e2e/harness/tree.py
diff --git a/libs/cua-driver/rust/tests/integration/run_tests.sh b/libs/cua-driver/tests/e2e/run_tests.sh
similarity index 81%
rename from libs/cua-driver/rust/tests/integration/run_tests.sh
rename to libs/cua-driver/tests/e2e/run_tests.sh
index 0a1d3c6052..2f98162239 100755
--- a/libs/cua-driver/rust/tests/integration/run_tests.sh
+++ b/libs/cua-driver/tests/e2e/run_tests.sh
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
-# Run cua-driver-rs integration tests.
+# Run cua-driver Python e2e tests.
#
# Usage:
# ./run_tests.sh # run all tests (Rust binary)
@@ -11,7 +11,7 @@
# CUA_DRIVER_BINARY is already set in the environment.
#
# Options:
-# -v / --verbose Pass -v to unittest for verbose output.
+# -v / --verbose Pass -v to pytest for verbose output.
# --release Build with --release (sets CUA_DRIVER_BINARY to release binary).
# --parity Run test_api_parity.py against both Swift and Rust binaries.
# Requires CUA_SWIFT_BINARY or ~/.local/bin/cua-driver.
@@ -19,7 +19,8 @@
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-WORKSPACE_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
+CUA_DRIVER_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
+RUST_ROOT="$CUA_DRIVER_ROOT/rust"
VERBOSE=""
PROFILE="debug"
@@ -41,8 +42,8 @@ if [ -z "${CUA_DRIVER_BINARY:-}" ]; then
echo "==> Building cua-driver ($PROFILE)..."
CARGO_FLAGS=""
[ "$PROFILE" = "release" ] && CARGO_FLAGS="--release"
- (cd "$WORKSPACE_ROOT" && cargo build $CARGO_FLAGS -p cua-driver 2>&1)
- export CUA_DRIVER_BINARY="$WORKSPACE_ROOT/target/$PROFILE/cua-driver"
+ (cd "$RUST_ROOT" && cargo build $CARGO_FLAGS -p cua-driver 2>&1)
+ export CUA_DRIVER_BINARY="$RUST_ROOT/target/$PROFILE/cua-driver"
fi
echo "==> Using Rust binary: $CUA_DRIVER_BINARY"
@@ -72,7 +73,7 @@ cd "$SCRIPT_DIR"
if [ "$PARITY_MODE" -eq 1 ]; then
echo "==> Running parity suite against both binaries..."
- python3 -m unittest ${VERBOSE} test_api_parity
+ python3 -m pytest ${VERBOSE} test_api_parity.py
exit $?
fi
@@ -84,4 +85,13 @@ fi
echo "==> Running ${#MODULES[@]} test module(s): ${MODULES[*]}"
echo ""
-python3 -m unittest ${VERBOSE} "${MODULES[@]}"
+FILES=()
+for module in "${MODULES[@]}"; do
+ if [[ "$module" == *.py ]]; then
+ FILES+=("$module")
+ else
+ FILES+=("$module.py")
+ fi
+done
+
+python3 -m pytest ${VERBOSE} "${FILES[@]}"
diff --git a/libs/cua-driver/rust/tests/integration/test_api_parity.py b/libs/cua-driver/tests/e2e/test_api_parity.py
similarity index 100%
rename from libs/cua-driver/rust/tests/integration/test_api_parity.py
rename to libs/cua-driver/tests/e2e/test_api_parity.py
diff --git a/libs/cua-driver/rust/tests/integration/test_background_focus.py b/libs/cua-driver/tests/e2e/test_background_focus.py
similarity index 100%
rename from libs/cua-driver/rust/tests/integration/test_background_focus.py
rename to libs/cua-driver/tests/e2e/test_background_focus.py
diff --git a/libs/cua-driver/rust/tests/integration/v2/test_blender.py b/libs/cua-driver/tests/e2e/test_blender.py
similarity index 100%
rename from libs/cua-driver/rust/tests/integration/v2/test_blender.py
rename to libs/cua-driver/tests/e2e/test_blender.py
diff --git a/libs/cua-driver/rust/tests/integration/v2/test_chrome.py b/libs/cua-driver/tests/e2e/test_chrome.py
similarity index 100%
rename from libs/cua-driver/rust/tests/integration/v2/test_chrome.py
rename to libs/cua-driver/tests/e2e/test_chrome.py
diff --git a/libs/cua-driver/rust/tests/integration/test_cli.py b/libs/cua-driver/tests/e2e/test_cli.py
similarity index 100%
rename from libs/cua-driver/rust/tests/integration/test_cli.py
rename to libs/cua-driver/tests/e2e/test_cli.py
diff --git a/libs/cua-driver/rust/tests/integration/test_concurrent_drivers.py b/libs/cua-driver/tests/e2e/test_concurrent_drivers.py
similarity index 100%
rename from libs/cua-driver/rust/tests/integration/test_concurrent_drivers.py
rename to libs/cua-driver/tests/e2e/test_concurrent_drivers.py
diff --git a/libs/cua-driver/rust/tests/integration/test_cursor_visibility.py b/libs/cua-driver/tests/e2e/test_cursor_visibility.py
similarity index 100%
rename from libs/cua-driver/rust/tests/integration/test_cursor_visibility.py
rename to libs/cua-driver/tests/e2e/test_cursor_visibility.py
diff --git a/libs/cua-driver/rust/tests/integration/test_double_click_delivery.py b/libs/cua-driver/tests/e2e/test_double_click_delivery.py
similarity index 100%
rename from libs/cua-driver/rust/tests/integration/test_double_click_delivery.py
rename to libs/cua-driver/tests/e2e/test_double_click_delivery.py
diff --git a/libs/cua-driver/rust/tests/integration/v2/test_electron.py b/libs/cua-driver/tests/e2e/test_electron.py
similarity index 100%
rename from libs/cua-driver/rust/tests/integration/v2/test_electron.py
rename to libs/cua-driver/tests/e2e/test_electron.py
diff --git a/libs/cua-driver/rust/tests/integration/test_focus_steal_parity.py b/libs/cua-driver/tests/e2e/test_focus_steal_parity.py
similarity index 99%
rename from libs/cua-driver/rust/tests/integration/test_focus_steal_parity.py
rename to libs/cua-driver/tests/e2e/test_focus_steal_parity.py
index 47c4221f7c..51ff506f6d 100644
--- a/libs/cua-driver/rust/tests/integration/test_focus_steal_parity.py
+++ b/libs/cua-driver/tests/e2e/test_focus_steal_parity.py
@@ -38,7 +38,7 @@ built helper made the integration tests flaky to bootstrap on a
fresh runner — Finder removes that dependency.)
Run:
- cd libs/cua-driver/rust/tests/integration
+ cd libs/cua-driver/tests/e2e
python3 -m unittest test_focus_steal_parity -v
"""
diff --git a/libs/cua-driver/rust/tests/integration/test_health_report_mcp.py b/libs/cua-driver/tests/e2e/test_health_report_mcp.py
similarity index 100%
rename from libs/cua-driver/rust/tests/integration/test_health_report_mcp.py
rename to libs/cua-driver/tests/e2e/test_health_report_mcp.py
diff --git a/libs/cua-driver/rust/tests/integration/test_list_windows.py b/libs/cua-driver/tests/e2e/test_list_windows.py
similarity index 100%
rename from libs/cua-driver/rust/tests/integration/test_list_windows.py
rename to libs/cua-driver/tests/e2e/test_list_windows.py
diff --git a/libs/cua-driver/rust/tests/integration/test_overlay_z_order.py b/libs/cua-driver/tests/e2e/test_overlay_z_order.py
similarity index 100%
rename from libs/cua-driver/rust/tests/integration/test_overlay_z_order.py
rename to libs/cua-driver/tests/e2e/test_overlay_z_order.py
diff --git a/libs/cua-driver/rust/tests/integration/test_pixel_click_delivery.py b/libs/cua-driver/tests/e2e/test_pixel_click_delivery.py
similarity index 100%
rename from libs/cua-driver/rust/tests/integration/test_pixel_click_delivery.py
rename to libs/cua-driver/tests/e2e/test_pixel_click_delivery.py
diff --git a/libs/cua-driver/rust/tests/integration/v2/test_safari.py b/libs/cua-driver/tests/e2e/test_safari.py
similarity index 100%
rename from libs/cua-driver/rust/tests/integration/v2/test_safari.py
rename to libs/cua-driver/tests/e2e/test_safari.py
diff --git a/libs/cua-driver/tests/e2e/test_tauri.py b/libs/cua-driver/tests/e2e/test_tauri.py
new file mode 100644
index 0000000000..5845c83824
--- /dev/null
+++ b/libs/cua-driver/tests/e2e/test_tauri.py
@@ -0,0 +1,97 @@
+"""Tauri shared-harness background automation tests.
+
+The Tauri test app is built from test-harness/apps/cross-platform/tauri
+and loads test-harness/shared/web/index.html, the same DOM used by the
+Electron, WebView2, and WKWebView harnesses.
+
+Run: python3 -m pytest test_tauri.py -v
+"""
+
+from __future__ import annotations
+
+import subprocess
+import sys
+import time
+import os
+
+import pytest
+
+_HERE = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, _HERE)
+
+
[email protected](autouse=True)
+def _reactivate_focus(focus_monitor):
+ _, pid = focus_monitor
+ subprocess.run(
+ ["osascript", "-e",
+ f'tell application "System Events" to set frontmost of (first process whose unix id is {pid}) to true'],
+ check=False,
+ )
+ time.sleep(0.4)
+
+
+# ── tests ─────────────────────────────────────────────────────────────────────
+
+class TestTauriButton:
+ def test_button_click(self, driver, tauri_app, ux_guard):
+ _, pid, _ = tauri_app
+ if pid == 0:
+ pytest.skip("Tauri app pid not found")
+
+ wid = driver.find_window(pid)
+
+ before = driver.get_window_state(pid, wid, query="Increment")
+ idx = before.find_button("Increment") or before.find_element("Increment")
+ assert idx is not None, f"'Increment' not found:\n{before.tree}"
+
+ driver.click(pid, wid, element_index=idx)
+ time.sleep(0.5)
+
+ after = driver.get_window_state(pid, wid)
+ assert after.has_text("counter=1"), f"Counter not incremented:\n{after.tree}"
+
+
+class TestTauriTextInput:
+ def test_type_text(self, driver, tauri_app, ux_guard):
+ _, pid, _ = tauri_app
+ if pid == 0:
+ pytest.skip("Tauri app pid not found")
+
+ wid = driver.find_window(pid)
+
+ before = driver.get_window_state(pid, wid)
+ idx = before.find_text_field(skip_url_bar=False)
+ assert idx is not None, f"Text input not found:\n{before.tree}"
+
+ driver.click(pid, wid, element_index=idx)
+ time.sleep(0.3)
+ driver.hotkey(pid, ["cmd", "a"])
+ time.sleep(0.1)
+ driver.type_text_chars(pid, "tauri test")
+ time.sleep(0.6)
+
+ after = driver.get_window_state(pid, wid)
+ assert after.has_text("mirror=tauri test"), f"Text not mirrored in AX tree:\n{after.tree}"
+
+
+class TestTauriNoFocusSteal:
+ def test_multiple_actions_no_focus_steal(self, driver, tauri_app, ux_guard):
+ _, pid, _ = tauri_app
+ if pid == 0:
+ pytest.skip("Tauri app pid not found")
+
+ wid = driver.find_window(pid)
+
+ # Take 3 actions in sequence — UX guard checks no focus steal
+ before = driver.get_window_state(pid, wid, query="Increment")
+ idx = before.find_button("Increment") or before.find_element("Increment")
+ if idx is not None:
+ driver.click(pid, wid, element_index=idx)
+ time.sleep(0.3)
+
+ driver.press_key(pid, "tab")
+ time.sleep(0.2)
+ driver.press_key(pid, "tab")
+ time.sleep(0.2)
+ # ux_guard teardown will assert_clean()