Skip to content

armgit / cua

Public Imported from GitHub
0

Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.

Add Rust-only harness runner and optional app tests

Francesco Bonacci4a9fecc90dc1
diff --git a/libs/cua-driver/rust/crates/cua-driver/tests/README.md b/libs/cua-driver/rust/crates/cua-driver/tests/README.md
index bdb9651d0e..6b59ddba52 100644
--- a/libs/cua-driver/rust/crates/cua-driver/tests/README.md
+++ b/libs/cua-driver/rust/crates/cua-driver/tests/README.md
@@ -36,11 +36,16 @@ Staged outputs are read from `../../test-apps/harness-<name>/`.
## Running
```bash
-cargo test -p cua-driver --test protocol_mcp_test
+cargo test -p cua-driver --test protocol_handshake_test
cargo test -p cua-driver --test harness_appkit_test -- --ignored --nocapture
cargo test -p cua-driver --test modality_desktop_scope_macos_test -- --ignored --nocapture
```
+Windows Rust run-all uses
+`../../../../tests/runners/windows/run-all.ps1`. It builds repo-local fixtures
+and runs the default, guard, harness, and modality suites. It intentionally
+excludes optional external-app suites.
+
Legacy Windows Sandbox runs use
`../../../../tests/runners/windows-sandbox/run-tests-in-sandbox.ps1`, which
builds selected Windows harness apps and maps them into the sandbox. The current
@@ -52,3 +57,28 @@ Windows GUI modality tests require a foreground interactive desktop where
scheduled tasks, and PsExec-launched commands can enter the right user session
while still exposing no foreground desktop; those tests self-skip in that state
because the focus oracle would otherwise report meaningless pid-0 results.
+
+The Windows probe distinguishes two no-foreground states:
+
+- `input_desktop=Default, foreground_hwnd=0`: the desktop is usable but idle.
+ Tests now launch `focus-monitor-win` and require that sentinel HWND to become
+ foreground before assertions start.
+- `input_desktop` is not `Default` or cannot be opened: the session is usually
+ locked/disconnected, for example after an RDP client drops. Reconnect, use
+ `tscon /dest:console` on a disposable GUI VM, or boot the VM into an unlocked
+ console session before running ignored GUI tests.
+
+Set `CUA_REQUIRE_GUI=1` on dedicated GUI runners to turn these desktop
+self-skips into hard failures with the full desktop-state diagnostic.
+
+## Optional External Apps
+
+These suites are Rust source-of-truth coverage for real installed apps, but they
+are not part of the canonical run-all path because they depend on software or
+desktop state outside the repo-local fixtures:
+
+- `harness_libreoffice_test.rs`: Windows LibreOffice Writer/Calc. Requires
+ LibreOffice installed, or `LO_SWRITER_EXE` / `LO_SCALC_EXE` pointing at the
+ executables.
+- `modality_launch_focus_macos_test.rs`: macOS Calculator/TextEdit launch focus
+ checks. Requires a logged-in GUI session and usable System Events scripting.
diff --git a/libs/cua-driver/rust/crates/cua-driver/tests/harness_libreoffice_test.rs b/libs/cua-driver/rust/crates/cua-driver/tests/harness_libreoffice_test.rs
index 7743110951..532531c053 100644
--- a/libs/cua-driver/rust/crates/cua-driver/tests/harness_libreoffice_test.rs
+++ b/libs/cua-driver/rust/crates/cua-driver/tests/harness_libreoffice_test.rs
@@ -1,4 +1,8 @@
-//! Integration tests against a live LibreOffice Writer instance.
+//! Optional integration tests against a live LibreOffice Writer instance.
+//!
+//! This file is intentionally not part of the canonical Windows run-all path:
+//! it requires LibreOffice to be installed on the machine. Run it explicitly on
+//! images that include LibreOffice.
//!
//! Regression-guards the VCL/SAL gaps the vision-only LO Writer flow
//! surfaced (manually validated during the PR #1708 follow-up
@@ -39,7 +43,7 @@
//! ## How to run
//!
//! Local (requires LibreOffice installed):
-//! cargo test --test harness_lo_vcl_test -- --ignored --nocapture
+//! cargo test -p cua-driver --test harness_libreoffice_test -- --ignored --nocapture
//!
//! Tests skip cleanly if `swriter.exe` isn't on disk at one of the
//! standard install locations (override via `LO_SWRITER_EXE`).
diff --git a/libs/cua-driver/rust/crates/cua-driver/tests/modality_launch_focus_macos_test.rs b/libs/cua-driver/rust/crates/cua-driver/tests/modality_launch_focus_macos_test.rs
new file mode 100644
index 0000000000..7fee4f4deb
--- /dev/null
+++ b/libs/cua-driver/rust/crates/cua-driver/tests/modality_launch_focus_macos_test.rs
@@ -0,0 +1,140 @@
+//! Optional macOS real-app launch/focus checks.
+//!
+//! These are Rust ports of the old Python focus-steal parity coverage for
+//! built-in macOS apps. They are not part of the canonical harness run-all path:
+//! they exercise external apps and a live user desktop, so run them explicitly.
+//!
+//! Run:
+//! cargo test -p cua-driver --test modality_launch_focus_macos_test -- --ignored --nocapture --test-threads=1
+
+#![cfg(target_os = "macos")]
+
+use std::process::Command;
+use std::thread::sleep;
+use std::time::{Duration, Instant};
+
+use cua_driver_testkit::{Driver, McpDriver};
+
+const FINDER_BUNDLE: &str = "com.apple.finder";
+const CALCULATOR_BUNDLE: &str = "com.apple.calculator";
+const TEXTEDIT_BUNDLE: &str = "com.apple.TextEdit";
+
+fn osascript(script: &str) -> Option<String> {
+ let out = Command::new("osascript")
+ .arg("-e")
+ .arg(script)
+ .output()
+ .inspect_err(|e| eprintln!("[launch-focus] osascript spawn failed: {e}"))
+ .ok()?;
+ if !out.status.success() {
+ eprintln!(
+ "[launch-focus] osascript failed: {}",
+ String::from_utf8_lossy(&out.stderr).trim()
+ );
+ return None;
+ }
+ Some(String::from_utf8_lossy(&out.stdout).trim().to_string())
+}
+
+fn frontmost_bundle_id() -> Option<String> {
+ osascript(
+ r#"tell application "System Events" to bundle identifier of first application process whose frontmost is true"#,
+ )
+ .filter(|s| !s.is_empty())
+}
+
+fn activate_bundle(bundle_id: &str) {
+ let _ = osascript(&format!(r#"tell application id "{bundle_id}" to activate"#));
+}
+
+fn wait_for_frontmost(bundle_id: &str, timeout: Duration) -> bool {
+ let deadline = Instant::now() + timeout;
+ while Instant::now() < deadline {
+ if frontmost_bundle_id().as_deref() == Some(bundle_id) {
+ return true;
+ }
+ sleep(Duration::from_millis(100));
+ }
+ false
+}
+
+fn ensure_finder_frontmost() -> bool {
+ for _ in 0..3 {
+ activate_bundle(FINDER_BUNDLE);
+ if wait_for_frontmost(FINDER_BUNDLE, Duration::from_secs(2)) {
+ return true;
+ }
+ sleep(Duration::from_millis(500));
+ }
+ eprintln!(
+ "[launch-focus] could not make Finder frontmost; skipping optional real-app test"
+ );
+ false
+}
+
+fn kill_app_process(process_name: &str) {
+ let _ = Command::new("pkill").args(["-x", process_name]).output();
+ sleep(Duration::from_millis(800));
+}
+
+fn launch_and_assert_frontmost_unchanged(driver: &mut McpDriver, bundle_id: &str, label: &str) {
+ let before = frontmost_bundle_id();
+ assert_eq!(
+ before.as_deref(),
+ Some(FINDER_BUNDLE),
+ "{label}: expected Finder frontmost before launch, got {before:?}"
+ );
+
+ let launch = driver.call("launch_app", serde_json::json!({ "bundle_id": bundle_id }));
+ assert!(
+ !launch.is_error(),
+ "{label}: launch_app({bundle_id}) failed: {}",
+ launch.text()
+ );
+ assert!(
+ launch.structured()["pid"].as_i64().unwrap_or(0) > 0,
+ "{label}: launch_app({bundle_id}) returned no pid: {:?}",
+ launch.raw
+ );
+
+ sleep(Duration::from_millis(1200));
+ let after = frontmost_bundle_id();
+ assert_eq!(
+ after.as_deref(),
+ Some(FINDER_BUNDLE),
+ "{label}: launch_app({bundle_id}) stole focus; frontmost after launch was {after:?}"
+ );
+}
+
+fn driver() -> Option<McpDriver> {
+ McpDriver::spawn_macos_daemon_proxy().or_else(McpDriver::spawn)
+}
+
+#[test]
+#[ignore]
+fn textedit_launch_preserves_finder_frontmost() {
+ if !ensure_finder_frontmost() {
+ return;
+ }
+ kill_app_process("TextEdit");
+ if !ensure_finder_frontmost() {
+ return;
+ }
+
+ let Some(mut driver) = driver() else { return };
+ launch_and_assert_frontmost_unchanged(&mut driver, TEXTEDIT_BUNDLE, "TextEdit launch");
+}
+
+#[test]
+#[ignore]
+fn calculator_then_textedit_launches_preserve_finder_frontmost() {
+ kill_app_process("Calculator");
+ kill_app_process("TextEdit");
+ if !ensure_finder_frontmost() {
+ return;
+ }
+
+ let Some(mut driver) = driver() else { return };
+ launch_and_assert_frontmost_unchanged(&mut driver, CALCULATOR_BUNDLE, "Calculator launch");
+ launch_and_assert_frontmost_unchanged(&mut driver, TEXTEDIT_BUNDLE, "TextEdit launch");
+}
diff --git a/libs/cua-driver/tests/runners/windows/README.md b/libs/cua-driver/tests/runners/windows/README.md
new file mode 100644
index 0000000000..ffb2e409c0
--- /dev/null
+++ b/libs/cua-driver/tests/runners/windows/README.md
@@ -0,0 +1,15 @@
+# Windows Rust Runner
+
+Canonical Windows Rust harness runner for an interactive user desktop.
+
+Run from `libs/cua-driver` in an RDP or console session:
+
+```powershell
+.\tests\runners\windows\run-all.ps1
+.\tests\runners\windows\run-all.ps1 -RequireGui
+```
+
+The runner builds repo-local Windows fixtures and runs the Rust default,
+guard, harness, and modality suites. It intentionally skips optional
+external-app suites such as LibreOffice because those require extra software
+on the VM image.
diff --git a/libs/cua-driver/tests/runners/windows/run-all.ps1 b/libs/cua-driver/tests/runners/windows/run-all.ps1
new file mode 100644
index 0000000000..4661ec5494
--- /dev/null
+++ b/libs/cua-driver/tests/runners/windows/run-all.ps1
@@ -0,0 +1,106 @@
+# Canonical Windows Rust harness runner.
+#
+# Run from libs/cua-driver in an interactive RDP/console session:
+# .\tests\runners\windows\run-all.ps1
+# .\tests\runners\windows\run-all.ps1 -RequireGui
+
+param(
+ [switch]$NoBuild,
+ [switch]$RequireGui
+)
+
+Set-StrictMode -Version Latest
+$ErrorActionPreference = "Stop"
+
+$runnerDir = Split-Path -Parent $MyInvocation.MyCommand.Definition
+$runnersDir = Split-Path -Parent $runnerDir
+$testsDir = Split-Path -Parent $runnersDir
+$cuaDriverRoot = Split-Path -Parent $testsDir
+$rustRoot = Join-Path $cuaDriverRoot "rust"
+$fixtureBuild = Join-Path $cuaDriverRoot "tests\fixtures\build\windows.ps1"
+
+Write-Host "=== cua-driver Windows Rust run-all ===" -ForegroundColor Cyan
+Write-Host "cua-driver root: $cuaDriverRoot"
+Write-Host "Rust workspace : $rustRoot"
+
+if ($RequireGui) {
+ $env:CUA_REQUIRE_GUI = "1"
+ Write-Host "CUA_REQUIRE_GUI=1" -ForegroundColor Yellow
+}
+
+$results = New-Object System.Collections.Generic.List[object]
+
+function Add-Result {
+ param([string]$Name, [int]$ExitCode)
+ $status = if ($ExitCode -eq 0) { "PASS" } else { "FAIL" }
+ $results.Add([pscustomobject]@{
+ Name = $Name
+ Status = $status
+ ExitCode = $ExitCode
+ }) | Out-Null
+}
+
+function Run-Step {
+ param(
+ [string]$Name,
+ [string]$WorkingDirectory,
+ [string[]]$CommandArgs
+ )
+
+ Write-Host "`n[RUN] $Name" -ForegroundColor Yellow
+ Push-Location $WorkingDirectory
+ try {
+ & cargo @CommandArgs
+ $code = if ($null -eq $LASTEXITCODE) { 0 } else { $LASTEXITCODE }
+ } finally {
+ Pop-Location
+ }
+ Add-Result $Name $code
+ if ($code -ne 0) {
+ Write-Host "[FAIL] $Name exited $code" -ForegroundColor Red
+ }
+}
+
+if (-not $NoBuild) {
+ if (-not (Test-Path $fixtureBuild)) {
+ throw "Windows fixture build script not found: $fixtureBuild"
+ }
+ Write-Host "`n[BUILD] Windows fixtures" -ForegroundColor Yellow
+ & $fixtureBuild
+ if ($LASTEXITCODE -ne 0) {
+ Add-Result "windows fixtures" $LASTEXITCODE
+ throw "Windows fixture build failed with exit $LASTEXITCODE"
+ }
+ Add-Result "windows fixtures" 0
+}
+
+Run-Step "default Rust tests" $rustRoot @(
+ "test", "-p", "cua-driver", "-p", "platform-windows", "--", "--nocapture"
+)
+Run-Step "guard UX" $rustRoot @(
+ "test", "-p", "cua-driver", "--test", "guard_ux_test", "--", "--nocapture", "--test-threads=1"
+)
+Run-Step "WPF harness" $rustRoot @(
+ "test", "-p", "cua-driver", "--test", "harness_wpf_test", "--", "--ignored", "--nocapture", "--test-threads=1"
+)
+Run-Step "WinUI3 harness" $rustRoot @(
+ "test", "-p", "cua-driver", "--test", "harness_winui3_test", "--", "--ignored", "--nocapture", "--test-threads=1"
+)
+Run-Step "WebView2/Electron harness" $rustRoot @(
+ "test", "-p", "cua-driver", "--test", "harness_web_test", "--", "--ignored", "--nocapture", "--test-threads=1"
+)
+Run-Step "Windows modality input e2e" $rustRoot @(
+ "test", "-p", "cua-driver", "--test", "modality_input_e2e_test", "--", "--ignored", "--nocapture", "--test-threads=1"
+)
+
+Write-Host "`n=== Summary ===" -ForegroundColor Cyan
+$failed = 0
+foreach ($r in $results) {
+ $color = if ($r.Status -eq "PASS") { "Green" } else { "Red" }
+ Write-Host ("{0,-32} {1} ({2})" -f $r.Name, $r.Status, $r.ExitCode) -ForegroundColor $color
+ if ($r.ExitCode -ne 0) { $failed++ }
+}
+
+if ($failed -gt 0) {
+ exit 1
+}