Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.
Rust agentai-agentapplecomputer-usecomputer-use-agentcontainerizationcuadesktop-automationhacktoberfestlumemacosmanusoperatorswiftvirtualizationvirtualization-frameworkwindowswindows-sandbox
Add Rust-only harness runner and optional app tests
Francesco Bonacci4a9fecc90dc1
diff --git a/libs/cua-driver/rust/crates/cua-driver/tests/README.md b/libs/cua-driver/rust/crates/cua-driver/tests/README.mdindex bdb9651d0e..6b59ddba52 100644--- a/libs/cua-driver/rust/crates/cua-driver/tests/README.md+++ b/libs/cua-driver/rust/crates/cua-driver/tests/README.md@@ -36,11 +36,16 @@ Staged outputs are read from `../../test-apps/harness-<name>/`.## Running```bash-cargo test -p cua-driver --test protocol_mcp_test+cargo test -p cua-driver --test protocol_handshake_testcargo test -p cua-driver --test harness_appkit_test -- --ignored --nocapturecargo test -p cua-driver --test modality_desktop_scope_macos_test -- --ignored --nocapture```+Windows Rust run-all uses+`../../../../tests/runners/windows/run-all.ps1`. It builds repo-local fixtures+and runs the default, guard, harness, and modality suites. It intentionally+excludes optional external-app suites.+Legacy Windows Sandbox runs use`../../../../tests/runners/windows-sandbox/run-tests-in-sandbox.ps1`, whichbuilds selected Windows harness apps and maps them into the sandbox. The current@@ -52,3 +57,28 @@ Windows GUI modality tests require a foreground interactive desktop wherescheduled tasks, and PsExec-launched commands can enter the right user sessionwhile still exposing no foreground desktop; those tests self-skip in that statebecause the focus oracle would otherwise report meaningless pid-0 results.++The Windows probe distinguishes two no-foreground states:++- `input_desktop=Default, foreground_hwnd=0`: the desktop is usable but idle.+ Tests now launch `focus-monitor-win` and require that sentinel HWND to become+ foreground before assertions start.+- `input_desktop` is not `Default` or cannot be opened: the session is usually+ locked/disconnected, for example after an RDP client drops. Reconnect, use+ `tscon /dest:console` on a disposable GUI VM, or boot the VM into an unlocked+ console session before running ignored GUI tests.++Set `CUA_REQUIRE_GUI=1` on dedicated GUI runners to turn these desktop+self-skips into hard failures with the full desktop-state diagnostic.++## Optional External Apps++These suites are Rust source-of-truth coverage for real installed apps, but they+are not part of the canonical run-all path because they depend on software or+desktop state outside the repo-local fixtures:++- `harness_libreoffice_test.rs`: Windows LibreOffice Writer/Calc. Requires+ LibreOffice installed, or `LO_SWRITER_EXE` / `LO_SCALC_EXE` pointing at the+ executables.+- `modality_launch_focus_macos_test.rs`: macOS Calculator/TextEdit launch focus+ checks. Requires a logged-in GUI session and usable System Events scripting.diff --git a/libs/cua-driver/rust/crates/cua-driver/tests/harness_libreoffice_test.rs b/libs/cua-driver/rust/crates/cua-driver/tests/harness_libreoffice_test.rsindex 7743110951..532531c053 100644--- a/libs/cua-driver/rust/crates/cua-driver/tests/harness_libreoffice_test.rs+++ b/libs/cua-driver/rust/crates/cua-driver/tests/harness_libreoffice_test.rs@@ -1,4 +1,8 @@-//! Integration tests against a live LibreOffice Writer instance.+//! Optional integration tests against a live LibreOffice Writer instance.+//!+//! This file is intentionally not part of the canonical Windows run-all path:+//! it requires LibreOffice to be installed on the machine. Run it explicitly on+//! images that include LibreOffice.//!//! Regression-guards the VCL/SAL gaps the vision-only LO Writer flow//! surfaced (manually validated during the PR #1708 follow-up@@ -39,7 +43,7 @@//! ## How to run//!//! Local (requires LibreOffice installed):-//! cargo test --test harness_lo_vcl_test -- --ignored --nocapture+//! cargo test -p cua-driver --test harness_libreoffice_test -- --ignored --nocapture//!//! Tests skip cleanly if `swriter.exe` isn't on disk at one of the//! standard install locations (override via `LO_SWRITER_EXE`).diff --git a/libs/cua-driver/rust/crates/cua-driver/tests/modality_launch_focus_macos_test.rs b/libs/cua-driver/rust/crates/cua-driver/tests/modality_launch_focus_macos_test.rsnew file mode 100644index 0000000000..7fee4f4deb--- /dev/null+++ b/libs/cua-driver/rust/crates/cua-driver/tests/modality_launch_focus_macos_test.rs@@ -0,0 +1,140 @@+//! Optional macOS real-app launch/focus checks.+//!+//! These are Rust ports of the old Python focus-steal parity coverage for+//! built-in macOS apps. They are not part of the canonical harness run-all path:+//! they exercise external apps and a live user desktop, so run them explicitly.+//!+//! Run:+//! cargo test -p cua-driver --test modality_launch_focus_macos_test -- --ignored --nocapture --test-threads=1++#![cfg(target_os = "macos")]++use std::process::Command;+use std::thread::sleep;+use std::time::{Duration, Instant};++use cua_driver_testkit::{Driver, McpDriver};++const FINDER_BUNDLE: &str = "com.apple.finder";+const CALCULATOR_BUNDLE: &str = "com.apple.calculator";+const TEXTEDIT_BUNDLE: &str = "com.apple.TextEdit";++fn osascript(script: &str) -> Option<String> {+ let out = Command::new("osascript")+ .arg("-e")+ .arg(script)+ .output()+ .inspect_err(|e| eprintln!("[launch-focus] osascript spawn failed: {e}"))+ .ok()?;+ if !out.status.success() {+ eprintln!(+ "[launch-focus] osascript failed: {}",+ String::from_utf8_lossy(&out.stderr).trim()+ );+ return None;+ }+ Some(String::from_utf8_lossy(&out.stdout).trim().to_string())+}++fn frontmost_bundle_id() -> Option<String> {+ osascript(+ r#"tell application "System Events" to bundle identifier of first application process whose frontmost is true"#,+ )+ .filter(|s| !s.is_empty())+}++fn activate_bundle(bundle_id: &str) {+ let _ = osascript(&format!(r#"tell application id "{bundle_id}" to activate"#));+}++fn wait_for_frontmost(bundle_id: &str, timeout: Duration) -> bool {+ let deadline = Instant::now() + timeout;+ while Instant::now() < deadline {+ if frontmost_bundle_id().as_deref() == Some(bundle_id) {+ return true;+ }+ sleep(Duration::from_millis(100));+ }+ false+}++fn ensure_finder_frontmost() -> bool {+ for _ in 0..3 {+ activate_bundle(FINDER_BUNDLE);+ if wait_for_frontmost(FINDER_BUNDLE, Duration::from_secs(2)) {+ return true;+ }+ sleep(Duration::from_millis(500));+ }+ eprintln!(+ "[launch-focus] could not make Finder frontmost; skipping optional real-app test"+ );+ false+}++fn kill_app_process(process_name: &str) {+ let _ = Command::new("pkill").args(["-x", process_name]).output();+ sleep(Duration::from_millis(800));+}++fn launch_and_assert_frontmost_unchanged(driver: &mut McpDriver, bundle_id: &str, label: &str) {+ let before = frontmost_bundle_id();+ assert_eq!(+ before.as_deref(),+ Some(FINDER_BUNDLE),+ "{label}: expected Finder frontmost before launch, got {before:?}"+ );++ let launch = driver.call("launch_app", serde_json::json!({ "bundle_id": bundle_id }));+ assert!(+ !launch.is_error(),+ "{label}: launch_app({bundle_id}) failed: {}",+ launch.text()+ );+ assert!(+ launch.structured()["pid"].as_i64().unwrap_or(0) > 0,+ "{label}: launch_app({bundle_id}) returned no pid: {:?}",+ launch.raw+ );++ sleep(Duration::from_millis(1200));+ let after = frontmost_bundle_id();+ assert_eq!(+ after.as_deref(),+ Some(FINDER_BUNDLE),+ "{label}: launch_app({bundle_id}) stole focus; frontmost after launch was {after:?}"+ );+}++fn driver() -> Option<McpDriver> {+ McpDriver::spawn_macos_daemon_proxy().or_else(McpDriver::spawn)+}++#[test]+#[ignore]+fn textedit_launch_preserves_finder_frontmost() {+ if !ensure_finder_frontmost() {+ return;+ }+ kill_app_process("TextEdit");+ if !ensure_finder_frontmost() {+ return;+ }++ let Some(mut driver) = driver() else { return };+ launch_and_assert_frontmost_unchanged(&mut driver, TEXTEDIT_BUNDLE, "TextEdit launch");+}++#[test]+#[ignore]+fn calculator_then_textedit_launches_preserve_finder_frontmost() {+ kill_app_process("Calculator");+ kill_app_process("TextEdit");+ if !ensure_finder_frontmost() {+ return;+ }++ let Some(mut driver) = driver() else { return };+ launch_and_assert_frontmost_unchanged(&mut driver, CALCULATOR_BUNDLE, "Calculator launch");+ launch_and_assert_frontmost_unchanged(&mut driver, TEXTEDIT_BUNDLE, "TextEdit launch");+}diff --git a/libs/cua-driver/tests/runners/windows/README.md b/libs/cua-driver/tests/runners/windows/README.mdnew file mode 100644index 0000000000..ffb2e409c0--- /dev/null+++ b/libs/cua-driver/tests/runners/windows/README.md@@ -0,0 +1,15 @@+# Windows Rust Runner++Canonical Windows Rust harness runner for an interactive user desktop.++Run from `libs/cua-driver` in an RDP or console session:++```powershell+.\tests\runners\windows\run-all.ps1+.\tests\runners\windows\run-all.ps1 -RequireGui+```++The runner builds repo-local Windows fixtures and runs the Rust default,+guard, harness, and modality suites. It intentionally skips optional+external-app suites such as LibreOffice because those require extra software+on the VM image.diff --git a/libs/cua-driver/tests/runners/windows/run-all.ps1 b/libs/cua-driver/tests/runners/windows/run-all.ps1new file mode 100644index 0000000000..4661ec5494--- /dev/null+++ b/libs/cua-driver/tests/runners/windows/run-all.ps1@@ -0,0 +1,106 @@+# Canonical Windows Rust harness runner.+#+# Run from libs/cua-driver in an interactive RDP/console session:+# .\tests\runners\windows\run-all.ps1+# .\tests\runners\windows\run-all.ps1 -RequireGui++param(+ [switch]$NoBuild,+ [switch]$RequireGui+)++Set-StrictMode -Version Latest+$ErrorActionPreference = "Stop"++$runnerDir = Split-Path -Parent $MyInvocation.MyCommand.Definition+$runnersDir = Split-Path -Parent $runnerDir+$testsDir = Split-Path -Parent $runnersDir+$cuaDriverRoot = Split-Path -Parent $testsDir+$rustRoot = Join-Path $cuaDriverRoot "rust"+$fixtureBuild = Join-Path $cuaDriverRoot "tests\fixtures\build\windows.ps1"++Write-Host "=== cua-driver Windows Rust run-all ===" -ForegroundColor Cyan+Write-Host "cua-driver root: $cuaDriverRoot"+Write-Host "Rust workspace : $rustRoot"++if ($RequireGui) {+ $env:CUA_REQUIRE_GUI = "1"+ Write-Host "CUA_REQUIRE_GUI=1" -ForegroundColor Yellow+}++$results = New-Object System.Collections.Generic.List[object]++function Add-Result {+ param([string]$Name, [int]$ExitCode)+ $status = if ($ExitCode -eq 0) { "PASS" } else { "FAIL" }+ $results.Add([pscustomobject]@{+ Name = $Name+ Status = $status+ ExitCode = $ExitCode+ }) | Out-Null+}++function Run-Step {+ param(+ [string]$Name,+ [string]$WorkingDirectory,+ [string[]]$CommandArgs+ )++ Write-Host "`n[RUN] $Name" -ForegroundColor Yellow+ Push-Location $WorkingDirectory+ try {+ & cargo @CommandArgs+ $code = if ($null -eq $LASTEXITCODE) { 0 } else { $LASTEXITCODE }+ } finally {+ Pop-Location+ }+ Add-Result $Name $code+ if ($code -ne 0) {+ Write-Host "[FAIL] $Name exited $code" -ForegroundColor Red+ }+}++if (-not $NoBuild) {+ if (-not (Test-Path $fixtureBuild)) {+ throw "Windows fixture build script not found: $fixtureBuild"+ }+ Write-Host "`n[BUILD] Windows fixtures" -ForegroundColor Yellow+ & $fixtureBuild+ if ($LASTEXITCODE -ne 0) {+ Add-Result "windows fixtures" $LASTEXITCODE+ throw "Windows fixture build failed with exit $LASTEXITCODE"+ }+ Add-Result "windows fixtures" 0+}++Run-Step "default Rust tests" $rustRoot @(+ "test", "-p", "cua-driver", "-p", "platform-windows", "--", "--nocapture"+)+Run-Step "guard UX" $rustRoot @(+ "test", "-p", "cua-driver", "--test", "guard_ux_test", "--", "--nocapture", "--test-threads=1"+)+Run-Step "WPF harness" $rustRoot @(+ "test", "-p", "cua-driver", "--test", "harness_wpf_test", "--", "--ignored", "--nocapture", "--test-threads=1"+)+Run-Step "WinUI3 harness" $rustRoot @(+ "test", "-p", "cua-driver", "--test", "harness_winui3_test", "--", "--ignored", "--nocapture", "--test-threads=1"+)+Run-Step "WebView2/Electron harness" $rustRoot @(+ "test", "-p", "cua-driver", "--test", "harness_web_test", "--", "--ignored", "--nocapture", "--test-threads=1"+)+Run-Step "Windows modality input e2e" $rustRoot @(+ "test", "-p", "cua-driver", "--test", "modality_input_e2e_test", "--", "--ignored", "--nocapture", "--test-threads=1"+)++Write-Host "`n=== Summary ===" -ForegroundColor Cyan+$failed = 0+foreach ($r in $results) {+ $color = if ($r.Status -eq "PASS") { "Green" } else { "Red" }+ Write-Host ("{0,-32} {1} ({2})" -f $r.Name, $r.Status, $r.ExitCode) -ForegroundColor $color+ if ($r.ExitCode -ne 0) { $failed++ }+}++if ($failed -gt 0) {+ exit 1+}