Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.
Rust agentai-agentapplecomputer-usecomputer-use-agentcontainerizationcuadesktop-automationhacktoberfestlumemacosmanusoperatorswiftvirtualizationvirtualization-frameworkwindowswindows-sandbox
fix(cua-driver): say macOS list_apps omits unknown fields, and test it (#4925)
Francesco Bonacci8cd4d2cac58b
diff --git a/docs/content/docs/cua-driver/guides/operate.mdx b/docs/content/docs/cua-driver/guides/operate.mdxindex 494d9d7546..0be1d92c06 100644--- a/docs/content/docs/cua-driver/guides/operate.mdx+++ b/docs/content/docs/cua-driver/guides/operate.mdx@@ -214,6 +214,13 @@ they are only affected by item 3.a tab moves focus. Remove them from text that should not submit, or writethe value with `set_value` where the field supports it.+Scripts that read raw `list_apps` JSON on macOS should also treat+`bundle_id`, `launch_path`, `kind` and `last_used` as optional: since+[#4916](https://github.com/trycua/cua/pull/4916) they are omitted when unknown+instead of `null`, and the always-empty `windows: []` is gone. The typed SDKs+already treat them as optional. For an app's windows, call+`list_windows({app: "Name"})`.+### Before updating to token-only element targetingReleases that include [#3873](https://github.com/trycua/cua/pull/3873) addressdiff --git a/docs/content/docs/cua-driver/reference/mcp-tools/apps-and-windows.mdx b/docs/content/docs/cua-driver/reference/mcp-tools/apps-and-windows.mdxindex 4f7ba88301..fb835f5305 100644--- a/docs/content/docs/cua-driver/reference/mcp-tools/apps-and-windows.mdx+++ b/docs/content/docs/cua-driver/reference/mcp-tools/apps-and-windows.mdx@@ -38,7 +38,9 @@ List macOS apps, both currently running and installed-but-not-running, with per-- active: is it the system-frontmost app? (implies running)- launch_path: filesystem path to the `.app` bundle, when known. Pass this to `launch_app` to start the app cold.- kind: `"desktop"` for `.app` bundles on macOS.-- last_used: RFC3339 timestamp from the bundle's filesystem mtime, when readable; otherwise null.+- last_used: RFC3339 timestamp from the bundle's filesystem mtime, when readable.++Optional fields (bundle_id, launch_path, kind, last_used) are omitted when unknown, never null.Standalone running entries include only apps with NSApplicationActivationPolicyRegular: background helpers and system UI agents are filtered out. Installed apps resolve their running/pid state against all live processes by bundle identifier, so an installed app whose process runs as an accessory (LSUIElement / menu-bar apps, e.g. Cua Driver itself) still reports its live pid. Installed apps come from scanning /Applications, /Applications/Utilities, ~/Applications, /System/Applications, and /System/Applications/Utilities.diff --git a/libs/cua-driver/rust/crates/platform-macos/src/tools/list_apps.rs b/libs/cua-driver/rust/crates/platform-macos/src/tools/list_apps.rsindex 925145aa04..65cb5f6320 100644--- a/libs/cua-driver/rust/crates/platform-macos/src/tools/list_apps.rs+++ b/libs/cua-driver/rust/crates/platform-macos/src/tools/list_apps.rs@@ -20,7 +20,9 @@ fn def() -> &'static ToolDef {Pass this to `launch_app` to start the app cold.\n\- kind: `\"desktop\"` for `.app` bundles on macOS.\n\- last_used: RFC3339 timestamp from the bundle's filesystem mtime, \- when readable; otherwise null.\n\n\+ when readable.\n\n\+ Optional fields (bundle_id, launch_path, kind, last_used) are \+ omitted when unknown, never null.\n\n\Standalone running entries include only apps with \NSApplicationActivationPolicyRegular — background helpers and \system UI agents are filtered out. Installed apps resolve their \@@ -61,30 +63,80 @@ impl Tool for ListAppsTool {.await.unwrap_or_default();let text = crate::apps::format_app_list(&apps);- // Single flat array. Each entry is the unified shape — existing- // fields (`pid`, `name`, `bundle_id`, `running`, `active`) are- // unchanged for backwards compatibility; `launch_path`, `kind` and- // `last_used` are additive and omitted when unknown, as the contract- // allows. Every agent reads this payload, so it carries no field that- // is always empty.let structured = serde_json::json!({- "apps": apps.iter().map(|a| {- let mut entry = serde_json::json!({- "pid": a.pid,- "name": a.name,- "bundle_id": a.bundle_id,- "active": a.active,- "running": a.running,- "launch_path": a.launch_path,- "kind": a.kind,- "last_used": a.last_used,- });- if let Some(entry) = entry.as_object_mut() {- entry.retain(|_, value| !value.is_null());- }- entry- }).collect::<Vec<_>>()+ "apps": apps.iter().map(app_record_json).collect::<Vec<_>>()});ToolResult::text(text).with_structured(structured)}}++/// One `list_apps` record. `pid`, `name`, `running` and `active` are always+/// present; `bundle_id`, `launch_path`, `kind` and `last_used` are omitted+/// when unknown, as the contract allows. Every agent reads this payload, so it+/// carries no null or always-empty field (0.34 sent `windows: []`).+fn app_record_json(a: &crate::apps::AppInfo) -> Value {+ let mut entry = serde_json::json!({+ "pid": a.pid,+ "name": a.name,+ "bundle_id": a.bundle_id,+ "active": a.active,+ "running": a.running,+ "launch_path": a.launch_path,+ "kind": a.kind,+ "last_used": a.last_used,+ });+ if let Some(entry) = entry.as_object_mut() {+ entry.retain(|_, value| !value.is_null());+ }+ entry+}++#[cfg(test)]+mod tests {+ use super::*;+ use crate::apps::AppInfo;++ fn app(bundle_id: Option<&str>, launch_path: Option<&str>) -> AppInfo {+ AppInfo {+ name: "Example".into(),+ pid: 0,+ bundle_id: bundle_id.map(Into::into),+ running: false,+ active: false,+ launch_path: launch_path.map(Into::into),+ kind: Some("desktop".into()),+ last_used: None,+ }+ }++ #[test]+ fn record_omits_unknown_fields_and_the_empty_windows_list() {+ let record = app_record_json(&app(None, None));+ assert_eq!(+ record,+ serde_json::json!({+ "pid": 0,+ "name": "Example",+ "active": false,+ "running": false,+ "kind": "desktop",+ })+ );+ assert!(record.get("windows").is_none());+ }++ #[test]+ fn record_keeps_known_fields_and_matches_the_contract() {+ let record = app_record_json(&app(+ Some("org.example.App"),+ Some("/Applications/Example.app"),+ ));+ assert_eq!(record["bundle_id"], "org.example.App");+ assert_eq!(record["launch_path"], "/Applications/Example.app");+ assert!(record.get("last_used").is_none());+ let typed: cua_driver_contract::ListAppsOutput =+ serde_json::from_value(serde_json::json!({ "apps": [record] })).unwrap();+ assert_eq!(typed.apps[0].bundle_id.as_deref(), Some("org.example.App"));+ assert_eq!(typed.apps[0].last_used, None);+ }+}diff --git a/scripts/docs-generators/cli-specs/cua-driver-mcp-macos.json b/scripts/docs-generators/cli-specs/cua-driver-mcp-macos.jsonindex 42e40dffcf..107f1f0950 100644--- a/scripts/docs-generators/cli-specs/cua-driver-mcp-macos.json+++ b/scripts/docs-generators/cli-specs/cua-driver-mcp-macos.json@@ -1,7 +1,7 @@{"tools": [{- "description": "List macOS apps — both currently running and installed-but-not-running — with per-app state flags:\n\n- running: is a process for this app live? (pid is 0 when false)\n- active: is it the system-frontmost app? (implies running)\n- launch_path: filesystem path to the `.app` bundle, when known. Pass this to `launch_app` to start the app cold.\n- kind: `\"desktop\"` for `.app` bundles on macOS.\n- last_used: RFC3339 timestamp from the bundle's filesystem mtime, when readable; otherwise null.\n\nStandalone running entries include only apps with NSApplicationActivationPolicyRegular — background helpers and system UI agents are filtered out. Installed apps resolve their running/pid state against all live processes by bundle identifier, so an installed app whose process runs as an accessory (LSUIElement / menu-bar apps, e.g. Cua Driver itself) still reports its live pid. Installed apps come from scanning /Applications, /Applications/Utilities, ~/Applications, /System/Applications, and /System/Applications/Utilities.\n\nUse this for \"is X installed?\" as well as \"is X running?\"; the list includes every installed app. To get an open app's pid and window_id, call list_windows({app: \"Name\"}) instead, which also gives per-window state (on-screen, on-current-Space, minimized, window titles). For just opening an app — running or not — call launch_app({bundle_id: ...}) directly; list_apps is not a prerequisite.",+ "description": "List macOS apps — both currently running and installed-but-not-running — with per-app state flags:\n\n- running: is a process for this app live? (pid is 0 when false)\n- active: is it the system-frontmost app? (implies running)\n- launch_path: filesystem path to the `.app` bundle, when known. Pass this to `launch_app` to start the app cold.\n- kind: `\"desktop\"` for `.app` bundles on macOS.\n- last_used: RFC3339 timestamp from the bundle's filesystem mtime, when readable.\n\nOptional fields (bundle_id, launch_path, kind, last_used) are omitted when unknown, never null.\n\nStandalone running entries include only apps with NSApplicationActivationPolicyRegular — background helpers and system UI agents are filtered out. Installed apps resolve their running/pid state against all live processes by bundle identifier, so an installed app whose process runs as an accessory (LSUIElement / menu-bar apps, e.g. Cua Driver itself) still reports its live pid. Installed apps come from scanning /Applications, /Applications/Utilities, ~/Applications, /System/Applications, and /System/Applications/Utilities.\n\nUse this for \"is X installed?\" as well as \"is X running?\"; the list includes every installed app. To get an open app's pid and window_id, call list_windows({app: \"Name\"}) instead, which also gives per-window state (on-screen, on-current-Space, minimized, window titles). For just opening an app — running or not — call launch_app({bundle_id: ...}) directly; list_apps is not a prerequisite.","destructive": false,"idempotent": true,"input_schema": {