Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.
Rust agentai-agentapplecomputer-usecomputer-use-agentcontainerizationcuadesktop-automationhacktoberfestlumemacosmanusoperatorswiftvirtualizationvirtualization-frameworkwindowswindows-sandbox
Move legacy Windows Sandbox runner
Francesco Bonaccied4fd87ddd0b
diff --git a/libs/cua-driver/rust/crates/cua-driver/tests/README.md b/libs/cua-driver/rust/crates/cua-driver/tests/README.mdindex 326097381e..bdb9651d0e 100644--- a/libs/cua-driver/rust/crates/cua-driver/tests/README.md+++ b/libs/cua-driver/rust/crates/cua-driver/tests/README.md@@ -41,8 +41,11 @@ cargo test -p cua-driver --test harness_appkit_test -- --ignored --nocapturecargo test -p cua-driver --test modality_desktop_scope_macos_test -- --ignored --nocapture```-Windows Sandbox runs use `../../sandbox/run-tests-in-sandbox.ps1`, which builds-the Windows harness apps and maps them into the sandbox.+Legacy Windows Sandbox runs use+`../../../../tests/runners/windows-sandbox/run-tests-in-sandbox.ps1`, which+builds selected Windows harness apps and maps them into the sandbox. The current+Windows GUI validation path should use a real user desktop session through RDP+or an interactive scheduled task.Windows GUI modality tests require a foreground interactive desktop where`GetForegroundWindow` returns a real user window. SSH-launched commands,diff --git a/libs/cua-driver/rust/crates/cua-driver/tests/guard_ux_test.rs b/libs/cua-driver/rust/crates/cua-driver/tests/guard_ux_test.rsindex a03eb5ff98..dbded6e185 100644--- a/libs/cua-driver/rust/crates/cua-driver/tests/guard_ux_test.rs+++ b/libs/cua-driver/rust/crates/cua-driver/tests/guard_ux_test.rs@@ -20,7 +20,7 @@//! 3. Assert focus-monitor-win's act_losses counter stayed at 0.//!//! Run in sandbox via:-//! .\sandbox\run-tests-in-sandbox.ps1 ux_guard+//! ..\tests\runners\windows-sandbox\run-tests-in-sandbox.ps1 guard_ux#![cfg(target_os = "windows")]diff --git a/libs/cua-driver/rust/crates/cua-driver/tests/harness_winui3_test.rs b/libs/cua-driver/rust/crates/cua-driver/tests/harness_winui3_test.rsindex 841e8cb015..2731fb9dc7 100644--- a/libs/cua-driver/rust/crates/cua-driver/tests/harness_winui3_test.rs+++ b/libs/cua-driver/rust/crates/cua-driver/tests/harness_winui3_test.rs@@ -14,7 +14,7 @@//! track of in-window flyouts.//!//! Run via:-//! .\sandbox\run-tests-in-sandbox.ps1 harness_winui3+//! ..\tests\runners\windows-sandbox\run-tests-in-sandbox.ps1 harness_winui3//! or locally://! cargo test --test harness_winui3_test -- --ignored --nocapturediff --git a/libs/cua-driver/rust/crates/cua-driver/tests/harness_wpf_test.rs b/libs/cua-driver/rust/crates/cua-driver/tests/harness_wpf_test.rsindex f7d0c4c6fb..a75016e4ef 100644--- a/libs/cua-driver/rust/crates/cua-driver/tests/harness_wpf_test.rs+++ b/libs/cua-driver/rust/crates/cua-driver/tests/harness_wpf_test.rs@@ -2,8 +2,7 @@//!//! Pairs of test apps live under `libs/cua-driver/tests/fixtures/`. They//! publish into `libs/cua-driver/rust/test-apps/harness-{wpf,winui3}/` and-//! get mapped into the sandbox via the existing run-tests-in-sandbox.ps1-//! mapped folder.+//! get mapped into the sandbox by the legacy Windows Sandbox runner.//!//! Each scenario covers a Win32 hosting pattern the agent should handle://! - counter : UIA Invoke on a plain WPF button@@ -17,7 +16,7 @@//! - child_hwnd : native Win32 BUTTON child HWND visible in tree//!//! Run via the sandbox runner:-//! .\sandbox\run-tests-in-sandbox.ps1 harness_wpf+//! ..\tests\runners\windows-sandbox\run-tests-in-sandbox.ps1 harness_wpf//!//! Or locally (requires .NET 8 SDK + `tests/fixtures/build/windows.ps1`)://! cargo test --test harness_wpf_test -- --ignored --nocapturediff --git a/libs/cua-driver/tests/runners/windows-sandbox/README.md b/libs/cua-driver/tests/runners/windows-sandbox/README.mdnew file mode 100644index 0000000000..66098ef7de--- /dev/null+++ b/libs/cua-driver/tests/runners/windows-sandbox/README.md@@ -0,0 +1,19 @@+# Windows Sandbox Runner++Legacy Windows Sandbox runner for local smoke checks. It predates the Azure/RDP+GUI validation flow and should not be treated as the canonical Windows desktop+test entrypoint.++Run it from `libs/cua-driver` on a Windows host with Windows Sandbox enabled:++```powershell+.\tests\runners\windows-sandbox\run-tests-in-sandbox.ps1+.\tests\runners\windows-sandbox\run-tests-in-sandbox.ps1 harness_wpf+```++The host script builds selected Rust test binaries and Windows fixtures, maps+`libs/cua-driver` into the sandbox as `C:\cua-driver`, and streams logs from the+inside-sandbox runner.++For current GUI validation, prefer a real user desktop session such as the+Azure RDP/scheduled-task runner described in the Rust test README.diff --git a/libs/cua-driver/rust/sandbox/run-tests-in-sandbox.ps1 b/libs/cua-driver/tests/runners/windows-sandbox/run-tests-in-sandbox.ps1similarity index 77%rename from libs/cua-driver/rust/sandbox/run-tests-in-sandbox.ps1rename to libs/cua-driver/tests/runners/windows-sandbox/run-tests-in-sandbox.ps1index dca3c31a09..b4ac011867 100644--- a/libs/cua-driver/rust/sandbox/run-tests-in-sandbox.ps1+++ b/libs/cua-driver/tests/runners/windows-sandbox/run-tests-in-sandbox.ps1@@ -1,7 +1,9 @@# run-tests-in-sandbox.ps1 - host-side launcher with live log streaming#-# Usage (from workspace root):-# .\sandbox\run-tests-in-sandbox.ps1 [<test-filter>]+# Legacy Windows Sandbox runner.+#+# Usage (from libs/cua-driver):+# .\tests\runners\windows-sandbox\run-tests-in-sandbox.ps1 [<test-filter>]## Steps:# 1. cargo build + cargo test --no-run@@ -31,11 +33,15 @@ function Close-Sandbox {Write-Host " Sandbox closed." -ForegroundColor Green}-$sandboxDir = Split-Path -Parent $MyInvocation.MyCommand.Definition-$wsRoot = Split-Path -Parent $sandboxDir+$runnerDir = Split-Path -Parent $MyInvocation.MyCommand.Definition+$runnersDir = Split-Path -Parent $runnerDir+$testsDir = Split-Path -Parent $runnersDir+$cuaDriverRoot = Split-Path -Parent $testsDir+$rustRoot = Join-Path $cuaDriverRoot "rust"-Write-Host "=== cua-driver-rs Windows Sandbox Test Runner ===" -ForegroundColor Cyan-Write-Host "Workspace: $wsRoot"+Write-Host "=== cua-driver Windows Sandbox Test Runner ===" -ForegroundColor Cyan+Write-Host "cua-driver root: $cuaDriverRoot"+Write-Host "Rust workspace : $rustRoot"# -- Pre-flight: fail fast if a sandbox is already running --------------------$running = Get-Process "WindowsSandboxClient","WindowsSandbox" -ErrorAction SilentlyContinue@@ -54,35 +60,35 @@ if ($running) {}# -- 1. Build -------------------------------------------------------------------Push-Location $wsRoot+Push-Location $rustRoottry {Write-Host "`n[BUILD] cargo build..." -ForegroundColor Yellow- cargo build+ cargo build -p cua-driverif ($LASTEXITCODE -ne 0) { throw "cargo build failed" }- Write-Host "`n[BUILD] cargo test --no-run (mcp_protocol_test)..." -ForegroundColor Yellow- cargo test --test mcp_protocol_test --no-run- if ($LASTEXITCODE -ne 0) { throw "cargo test --no-run (mcp_protocol_test) failed" }+ Write-Host "`n[BUILD] cargo test --no-run (protocol_handshake_test)..." -ForegroundColor Yellow+ cargo test -p cua-driver --test protocol_handshake_test --no-run+ if ($LASTEXITCODE -ne 0) { throw "cargo test --no-run (protocol_handshake_test) failed" }- Write-Host "`n[BUILD] cargo test --no-run (ux_guard_test)..." -ForegroundColor Yellow- cargo test --test ux_guard_test --no-run- if ($LASTEXITCODE -ne 0) { throw "cargo test --no-run (ux_guard_test) failed" }+ Write-Host "`n[BUILD] cargo test --no-run (guard_ux_test)..." -ForegroundColor Yellow+ cargo test -p cua-driver --test guard_ux_test --no-run+ if ($LASTEXITCODE -ne 0) { throw "cargo test --no-run (guard_ux_test) failed" }Write-Host "`n[BUILD] cargo test --no-run (harness_wpf_test)..." -ForegroundColor Yellow- cargo test --test harness_wpf_test --no-run+ cargo test -p cua-driver --test harness_wpf_test --no-runif ($LASTEXITCODE -ne 0) { throw "cargo test --no-run (harness_wpf_test) failed" }Write-Host "`n[BUILD] cargo test --no-run (harness_winui3_test)..." -ForegroundColor Yellow- cargo test --test harness_winui3_test --no-run+ cargo test -p cua-driver --test harness_winui3_test --no-runif ($LASTEXITCODE -ne 0) { throw "cargo test --no-run (harness_winui3_test) failed" }Write-Host "`n[BUILD] cargo test --no-run (harness_web_test)..." -ForegroundColor Yellow- cargo test --test harness_web_test --no-run+ cargo test -p cua-driver --test harness_web_test --no-runif ($LASTEXITCODE -ne 0) { throw "cargo test --no-run (harness_web_test) failed" }- Write-Host "`n[BUILD] cargo test --no-run (harness_bg_modality_test)..." -ForegroundColor Yellow- cargo test --test harness_bg_modality_test --no-run- if ($LASTEXITCODE -ne 0) { throw "cargo test --no-run (harness_bg_modality_test) failed" }+ Write-Host "`n[BUILD] cargo test --no-run (modality_input_e2e_test)..." -ForegroundColor Yellow+ cargo test -p cua-driver --test modality_input_e2e_test --no-run+ if ($LASTEXITCODE -ne 0) { throw "cargo test --no-run (modality_input_e2e_test) failed" }} finally { Pop-Location }# -- 1.5. Build the test fixtures if dependencies are on PATH ------------------@@ -91,7 +97,7 @@ try {# a dependency isn't available - the smoke tests degrade to "skipped" inside# the sandbox rather than failing the whole run.if (Get-Command dotnet -ErrorAction SilentlyContinue) {- $harnessBuild = Join-Path $wsRoot "..\tests\fixtures\build\windows.ps1"+ $harnessBuild = Join-Path $cuaDriverRoot "tests\fixtures\build\windows.ps1"if (Test-Path $harnessBuild) {Write-Host "`n[BUILD] test fixtures (dotnet publish)..." -ForegroundColor Yellow# build.ps1 sets $ErrorActionPreference=Stop and throws on@@ -112,14 +118,14 @@ if (Get-Command dotnet -ErrorAction SilentlyContinue) {Write-Host "`n[SKIP] dotnet CLI not on PATH - test fixture build skipped, harness tests will degrade." -ForegroundColor Yellow}-$mcpBin = Get-ChildItem "$wsRoot\target\debug\deps\mcp_protocol_test-*.exe" |+$protocolBin = Get-ChildItem "$rustRoot\target\debug\deps\protocol_handshake_test-*.exe" |Sort-Object LastWriteTime -Descending | Select-Object -First 1-if (-not $mcpBin) { throw "mcp_protocol_test-*.exe not found" }-Write-Host "mcp_protocol_test: $($mcpBin.Name)"+if (-not $protocolBin) { throw "protocol_handshake_test-*.exe not found" }+Write-Host "protocol_handshake_test: $($protocolBin.Name)"-$uxBin = Get-ChildItem "$wsRoot\target\debug\deps\ux_guard_test-*.exe" |+$guardBin = Get-ChildItem "$rustRoot\target\debug\deps\guard_ux_test-*.exe" |Sort-Object LastWriteTime -Descending | Select-Object -First 1-if ($uxBin) { Write-Host "ux_guard_test : $($uxBin.Name)" } else { Write-Host "ux_guard_test : (not found, will skip)" }+if ($guardBin) { Write-Host "guard_ux_test : $($guardBin.Name)" } else { Write-Host "guard_ux_test : (not found, will skip)" }# -- 2. Prepare shared output folder ------------------------------------------$outputDir = "$env:TEMP\cua-sandbox-output"@@ -132,8 +138,8 @@ $wsbContent = @"<Configuration><MappedFolders><MappedFolder>- <HostFolder>$wsRoot</HostFolder>- <SandboxFolder>C:\cua-driver-rs</SandboxFolder>+ <HostFolder>$cuaDriverRoot</HostFolder>+ <SandboxFolder>C:\cua-driver</SandboxFolder><ReadOnly>true</ReadOnly></MappedFolder><MappedFolder>@@ -143,7 +149,7 @@ $wsbContent = @"</MappedFolder></MappedFolders><LogonCommand>- <Command>powershell -ExecutionPolicy Bypass -NonInteractive -File C:\cua-driver-rs\sandbox\sandbox-runner.ps1</Command>+ <Command>powershell -ExecutionPolicy Bypass -NonInteractive -File C:\cua-driver\tests\runners\windows-sandbox\sandbox-runner.ps1</Command></LogonCommand></Configuration>"@diff --git a/libs/cua-driver/rust/sandbox/sandbox-runner.ps1 b/libs/cua-driver/tests/runners/windows-sandbox/sandbox-runner.ps1similarity index 77%rename from libs/cua-driver/rust/sandbox/sandbox-runner.ps1rename to libs/cua-driver/tests/runners/windows-sandbox/sandbox-runner.ps1index 35828f98f4..30dc7f2593 100644--- a/libs/cua-driver/rust/sandbox/sandbox-runner.ps1+++ b/libs/cua-driver/tests/runners/windows-sandbox/sandbox-runner.ps1@@ -15,10 +15,14 @@ function Log($msg) {Log "sandbox-runner.ps1 starting"# -- set env so binary_path() resolves cua-driver.exe correctly --------------$env:CARGO_MANIFEST_DIR = "C:\cua-driver-rs\crates\cua-driver"+$repoRoot = "C:\cua-driver"+$rustRoot = Join-Path $repoRoot "rust"+$targetDeps = Join-Path $rustRoot "target\debug\deps"++$env:CARGO_MANIFEST_DIR = Join-Path $rustRoot "crates\cua-driver"Log "CARGO_MANIFEST_DIR = $env:CARGO_MANIFEST_DIR"-$driverExe = "C:\cua-driver-rs\target\debug\cua-driver.exe"+$driverExe = Join-Path $rustRoot "target\debug\cua-driver.exe"if (-not (Test-Path $driverExe)) {Log "ERROR: $driverExe not found""1" | Out-File $exitFile -Encoding utf8 -NoNewline@@ -28,24 +32,24 @@ if (-not (Test-Path $driverExe)) {Log "cua-driver : $driverExe"# -- find test binaries --------------------------------------------------------# Run mcp_protocol_test first, then ux_guard_test (UX guard needs a real+# Run protocol_handshake_test first, then guard_ux_test (UX guard needs a real# desktop session and spawns visible windows, so it runs second).$testSuites = @(- @{ Pattern = "mcp_protocol_test-*.exe"; Label = "mcp_protocol_test" },- @{ Pattern = "ux_guard_test-*.exe"; Label = "ux_guard_test" },+ @{ Pattern = "protocol_handshake_test-*.exe"; Label = "protocol_handshake_test" },+ @{ Pattern = "guard_ux_test-*.exe"; Label = "guard_ux_test" },@{ Pattern = "harness_wpf_test-*.exe"; Label = "harness_wpf_test"; Extra = @("--ignored") },@{ Pattern = "harness_winui3_test-*.exe"; Label = "harness_winui3_test"; Extra = @("--ignored") },@{ Pattern = "harness_web_test-*.exe"; Label = "harness_web_test"; Extra = @("--ignored") },- @{ Pattern = "harness_bg_modality_test-*.exe";Label = "harness_bg_modality_test";Extra = @("--ignored") }+ @{ Pattern = "modality_input_e2e_test-*.exe"; Label = "modality_input_e2e_test"; Extra = @("--ignored") })# -- stage harness binaries to %TEMP% (same Zone-3 ShellExecute workaround) --$harnessRoots = @(- @{ Src = "C:\cua-driver-rs\test-apps\harness-wpf"; EnvVar = "HARNESS_WPF_EXE"; Exe = "CuaTestHarness.Wpf.exe" },- @{ Src = "C:\cua-driver-rs\test-apps\harness-winui3"; EnvVar = "HARNESS_WINUI3_EXE"; Exe = "CuaTestHarness.WinUI3.exe" },- @{ Src = "C:\cua-driver-rs\test-apps\harness-webview"; EnvVar = "HARNESS_WEBVIEW_EXE"; Exe = "CuaTestHarness.WebView.exe" },- @{ Src = "C:\cua-driver-rs\test-apps\harness-electron"; EnvVar = "HARNESS_ELECTRON_EXE"; Exe = "CuaTestHarness.Electron.exe" },- @{ Src = "C:\cua-driver-rs\test-apps\harness-tauri"; EnvVar = "HARNESS_TAURI_EXE"; Exe = "CuaTestHarness.Tauri.exe" }+ @{ Src = (Join-Path $rustRoot "test-apps\harness-wpf"); EnvVar = "HARNESS_WPF_EXE"; Exe = "CuaTestHarness.Wpf.exe" },+ @{ Src = (Join-Path $rustRoot "test-apps\harness-winui3"); EnvVar = "HARNESS_WINUI3_EXE"; Exe = "CuaTestHarness.WinUI3.exe" },+ @{ Src = (Join-Path $rustRoot "test-apps\harness-webview"); EnvVar = "HARNESS_WEBVIEW_EXE"; Exe = "CuaTestHarness.WebView.exe" },+ @{ Src = (Join-Path $rustRoot "test-apps\harness-electron"); EnvVar = "HARNESS_ELECTRON_EXE"; Exe = "CuaTestHarness.Electron.exe" },+ @{ Src = (Join-Path $rustRoot "test-apps\harness-tauri"); EnvVar = "HARNESS_TAURI_EXE"; Exe = "CuaTestHarness.Tauri.exe" })foreach ($h in $harnessRoots) {if (-not (Test-Path $h.Src)) {@@ -86,7 +90,7 @@ foreach ($suite in $testSuites) {continue}- $testBin = Get-ChildItem "C:\cua-driver-rs\target\debug\deps\$($suite.Pattern)" |+ $testBin = Get-ChildItem "$targetDeps\$($suite.Pattern)" |Sort-Object LastWriteTime -Descending | Select-Object -First 1if (-not $testBin) {Log "WARNING: $($suite.Pattern) not found, skipping $label"