Skip to content

armgit / cua

Public Imported from GitHub
0

Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.

docs(cua-driver): fix stale references after recent tool consolidation (#1417)

Francesco Bonacci4ae26cfde325
diff --git a/docs/content/docs/cua-driver/guide/getting-started/quickstart.mdx b/docs/content/docs/cua-driver/guide/getting-started/quickstart.mdx
index 912cbcda03..f3915cc994 100644
--- a/docs/content/docs/cua-driver/guide/getting-started/quickstart.mdx
+++ b/docs/content/docs/cua-driver/guide/getting-started/quickstart.mdx
@@ -103,7 +103,7 @@ Pixel clicks are for surfaces the AX tree doesn't reach: canvases, video players
```bash
# Write the screenshot to disk. Works in every capture mode.
-cua-driver get_window_state '{"pid":844,"window_id":10725}' --image-out /tmp/shot.png
+cua-driver get_window_state '{"pid":844,"window_id":10725}' --screenshot-out-file /tmp/shot.png
# Look at /tmp/shot.png, pick a target pixel, then:
cua-driver click '{"pid":844,"window_id":10725,"x":120,"y":240}'
diff --git a/docs/content/docs/cua-driver/reference/cli-reference.mdx b/docs/content/docs/cua-driver/reference/cli-reference.mdx
index c5af31770c..da22b2e458 100644
--- a/docs/content/docs/cua-driver/reference/cli-reference.mdx
+++ b/docs/content/docs/cua-driver/reference/cli-reference.mdx
@@ -56,7 +56,7 @@ cua-driver <tool-name> '<JSON-args>'
**Flags:**
- `--raw` — Print the raw `CallTool.Result` JSON (content + structuredContent + isError) instead of unwrapping `structuredContent`.
-- `--image-out <path>` — Write the first image content block from the response to `path` (PNG bytes). The default text formatter would drop image content otherwise.
+- `--screenshot-out-file <path>` — Write the first image content block from the response to `path` (PNG bytes). The default text formatter would drop image content otherwise.
- `--compact` — Emit minified JSON instead of pretty-printed.
- `--no-daemon` — Skip the running daemon and run the tool in-process. Element-indexed workflows fail without a daemon because the per-pid cache dies between CLI invocations.
- `--socket <path>` — Override the daemon Unix socket path.
@@ -67,7 +67,7 @@ cua-driver <tool-name> '<JSON-args>'
cua-driver call list_apps
cua-driver call launch_app '{"bundle_id":"com.apple.finder"}'
echo '{"pid":844,"window_id":1234}' | cua-driver call get_window_state
-cua-driver get_window_state '{"pid":844,"window_id":1234}' --image-out /tmp/shot.png
+cua-driver get_window_state '{"pid":844,"window_id":1234}' --screenshot-out-file /tmp/shot.png
```
### cua-driver list-tools
@@ -380,6 +380,7 @@ The following MCP tools are callable via `cua-driver <tool>`. For input schemas
- `set_agent_cursor_enabled` — toggle the visual overlay.
- `set_agent_cursor_motion` — tune the Bezier-arc + spring motion knobs.
+- `set_agent_cursor_style` — customize the cursor's visual appearance (gradient colors, bloom color, image path).
- `get_agent_cursor_state` — read the current overlay configuration.
**Config**
diff --git a/docs/content/docs/cua-driver/reference/mcp-tools.mdx b/docs/content/docs/cua-driver/reference/mcp-tools.mdx
index 2b110dfceb..e00f452577 100644
--- a/docs/content/docs/cua-driver/reference/mcp-tools.mdx
+++ b/docs/content/docs/cua-driver/reference/mcp-tools.mdx
@@ -7,7 +7,7 @@ import { Callout } from 'fumadocs-ui/components/callout';
`cua-driver` exposes 28 MCP tools through a single stdio server (`cua-driver mcp`). Every tool is also callable from the shell as `cua-driver <name> '<JSON-args>'`.
-Tool names are `snake_case`. Responses are MCP `CallTool.Result` envelopes: a text content block prefixed with a `✅` summary (or the error reason on failure), plus optional image or structured-content blocks on tools that produce them. See the [CLI reference](/cua-driver/reference/cli-reference) for CLI-specific flags like `--raw` and `--image-out`.
+Tool names are `snake_case`. Responses are MCP `CallTool.Result` envelopes: a text content block prefixed with a `✅` summary (or the error reason on failure), plus optional image or structured-content blocks on tools that produce them. See the [CLI reference](/cua-driver/reference/cli-reference) for CLI-specific flags like `--raw` and `--screenshot-out-file`.
<Callout type="info">
Tool names here match the CLI form exactly. `cua-driver list_apps` and the MCP `list_apps` tool run the same code path.
@@ -81,6 +81,7 @@ Snapshot a single window: AX element tree plus a screenshot. Populates the per-p
- `pid` (integer, required): Process ID from `list_apps`.
- `window_id` (integer, required): CGWindowID of the target window. Must belong to `pid`. Enumerate via `list_windows` or read from `launch_app`'s `windows` array.
- `query` (string, optional): Case-insensitive substring. When set, `tree_markdown` only contains matching lines plus their ancestor chain; element indices and `element_count` are unchanged.
+- `screenshot_out_file` (string, optional): Absolute path to write the screenshot to. When set, the screenshot bytes are written to this file, the MCP image content block is omitted, and `screenshot_file_path` is returned instead. Useful for agents that cannot consume inline base64 without saturating their context window.
```json
{"pid": 844, "window_id": 10725, "query": "save"}
diff --git a/libs/cua-driver/Skills/cua-driver/SKILL.md b/libs/cua-driver/Skills/cua-driver/SKILL.md
index e442825136..5bec2c32e2 100644
--- a/libs/cua-driver/Skills/cua-driver/SKILL.md
+++ b/libs/cua-driver/Skills/cua-driver/SKILL.md
@@ -167,11 +167,11 @@ failure mode and it steals focus every time.
When a cua-driver call surprises you, diagnose cua-driver first:
- **Tiny screenshot / empty `tree_markdown`?** Check
- `cua-driver get_config` → `capture_mode`. Default `"vision"` omits
- the AX tree (PNG only), `"ax"` omits the PNG, `"som"` returns
- both. If a snapshot lacks a tree, `capture_mode` is almost
- certainly `"vision"` — either reason purely from the PNG or flip
- to `"som"` / `"ax"` via `set_config`.
+ `cua-driver get_config` → `capture_mode`. Default `"som"` returns
+ both the AX tree and screenshot. `"vision"` omits the AX tree
+ (PNG only), `"ax"` omits the PNG. If a snapshot lacks a tree,
+ `capture_mode` is almost certainly `"vision"` — either reason
+ purely from the PNG or flip to `"som"` / `"ax"` via `set_config`.
- **`has_screenshot: false`?** The window capture failed (transient
race against a close, or the window has no backing store yet).
Re-snapshot; if persistent, pick a different `window_id` via
@@ -414,17 +414,14 @@ single-window case you can skip `list_windows` entirely and read the
Call `get_window_state({pid, window_id})` with the `window_id` from
`launch_app`'s `windows` array (or a fresh `list_windows({pid})` if
-you're interacting with a long-lived process). In the default
-`vision` capture_mode the response carries **only the screenshot**
-— no AX tree — so the canonical loop is `list_windows →
-get_window_state → reason over PNG → pixel click`. When you need
-`element_index` dispatch (AX-addressable elements, backgrounded
-clicks), flip to `som` first: `cua-driver set_config '{"key":
-"capture_mode", "value": "som"}'`. The rest of this section walks through `som` mode, which
-is what you want once you've decided element-indexed addressing is
-required.
-
-In `som` mode the response carries:
+you're interacting with a long-lived process). The default `som`
+capture_mode returns **both the AX tree and screenshot**, so the
+canonical loop works immediately without any config change. The rest
+of this section walks through `som` mode. If you're in `vision` mode
+(PNG only, no AX tree), flip back: `cua-driver set_config '{"key":
+"capture_mode", "value": "som"}'`.
+
+In `som` mode (the default) the response carries:
- `tree_markdown` — every actionable element tagged `[N]`. That `N`
is the `element_index`. The tree can be very large (Finder is
diff --git a/libs/cua-driver/Skills/cua-driver/TESTS.md b/libs/cua-driver/Skills/cua-driver/TESTS.md
index fdc835b329..98b949c25e 100644
--- a/libs/cua-driver/Skills/cua-driver/TESTS.md
+++ b/libs/cua-driver/Skills/cua-driver/TESTS.md
@@ -59,7 +59,7 @@ backgrounded throughout, and Claude uses the canonical
### 4. Notes — native text entry
**Prompt:** `Create a new note in Notes titled 'cua-driver test' with the body 'native text entry verification'.`
-**Exercises:** `type_text_in` via `kAXSelectedText` against native AppKit text fields.
+**Exercises:** `type_text` via `kAXSelectedText` against native AppKit text fields.
**Success:**
- Re-snapshot of Notes shows a note row in the sidebar whose title contains `cua-driver test`.
@@ -72,7 +72,7 @@ backgrounded throughout, and Claude uses the canonical
### 5. Numbers — cell-addressable UI
**Prompt:** `Open ~/Documents/test.numbers and put 42 in cell B2.`
-**Exercises:** unusual AX shape (spreadsheet cells), click + `set_value` or `type_text_in` + Tab/Return.
+**Exercises:** unusual AX shape (spreadsheet cells), click + `set_value` or `type_text` + Tab/Return.
**Success:**
- Re-snapshot shows cell B2's AXValue is `42`.