Skip to content

armgit / cua

Public Imported from GitHub
0

Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.

docs(cua-driver): sync to cua-driver-rs 0.3.2 (#1742)

Francesco Bonacci79ea7b449ec5
diff --git a/docs/content/docs/cua-driver/guide/getting-started/autostart.mdx b/docs/content/docs/cua-driver/guide/getting-started/autostart.mdx
index 559838a519..44a2ea0023 100644
--- a/docs/content/docs/cua-driver/guide/getting-started/autostart.mdx
+++ b/docs/content/docs/cua-driver/guide/getting-started/autostart.mdx
@@ -19,8 +19,8 @@ cua-driver autostart disable # remove the entry
| Platform | Mechanism | Where it lives |
|---|---|---|
| Windows | Scheduled Task `cua-driver-serve` with `LogonType: Interactive` | Task Scheduler (`schtasks /Query /TN cua-driver-serve`) |
-| macOS | Not yet implemented in the Rust port — use the manual `LaunchAgent` recipe ([`scripts/install-local.sh --autostart`](https://github.com/trycua/cua/blob/main/libs/cua-driver/rust/scripts/install-local.sh)) | `~/Library/LaunchAgents/com.trycua.cua-driver-rs.plist` |
-| Linux | Not yet implemented in the Rust port — use the manual `systemd --user` unit ([`scripts/install-local.sh --autostart`](https://github.com/trycua/cua/blob/main/libs/cua-driver/rust/scripts/install-local.sh)) | `~/.config/systemd/user/cua-driver-rs.service` |
+| macOS | Not yet implemented in the Rust port — use the manual `LaunchAgent` recipe ([`scripts/install-local.sh --autostart`](https://github.com/trycua/cua/blob/main/libs/cua-driver/scripts/install-local.sh)) | `~/Library/LaunchAgents/com.trycua.cua-driver-rs.plist` |
+| Linux | Not yet implemented in the Rust port — use the manual `systemd --user` unit ([`scripts/install-local.sh --autostart`](https://github.com/trycua/cua/blob/main/libs/cua-driver/scripts/install-local.sh)) | `~/.config/systemd/user/cua-driver-rs.service` |
<Callout type="info">
**Platform parity status.** `cua-driver autostart` is currently a Windows-only verb in the Rust port — the macOS / Linux cases return an error pointing at the manual recipe. A cross-platform native impl is tracked as a follow-up. The verb is documented as the cross-platform interface so existing scripts and skill-pack instructions land in the right place once parity ships.
@@ -148,11 +148,7 @@ Some apps (Calculator, Settings, older UWPs) host their top-level window as a `W
## Why not just `cua-driver serve &` over SSH?
-A naively-spawned `cua-driver serve &` over SSH inherits the SSH session — which on Windows OpenSSH is **Session 0**, the non-interactive services session. The daemon comes up, but every GUI tool it answers (`list_windows`, `click`, `screenshot`, ...) bottoms out in Win32 APIs scoped to the calling session's WindowStation + Desktop, which Session 0 doesn't have. Tools silently return empty arrays.
-
-`autostart enable && kick` sidesteps this by running the daemon under the user's interactive logon (Session 1+) instead of inheriting whatever session called `serve`. The daemon then has a real desktop attached and GUI tools work.
-
-See [Running cua-driver under SSH on Windows](/cua-driver/guide/getting-started/windows-ssh) for the end-to-end SSH workflow.
+A `cua-driver serve &` spawned over SSH inherits the SSH session — Session 0 on Windows OpenSSH — which has no interactive desktop, so every GUI tool returns empty. `autostart enable && kick` runs the daemon under the user's interactive logon (Session 1+) instead, so it has a real desktop attached. [Running cua-driver under SSH on Windows](/cua-driver/guide/getting-started/windows-ssh) has the full reasoning and the end-to-end workflow.
## Uninstall behaviour
diff --git a/docs/content/docs/cua-driver/guide/getting-started/comparison.mdx b/docs/content/docs/cua-driver/guide/getting-started/comparison.mdx
index aac71fadfc..0da2ee5b0d 100644
--- a/docs/content/docs/cua-driver/guide/getting-started/comparison.mdx
+++ b/docs/content/docs/cua-driver/guide/getting-started/comparison.mdx
@@ -1,18 +1,18 @@
---
title: Comparison
-description: How Cua Driver compares to other macOS computer-use tools
+description: How Cua Driver compares to other computer-use tools
---
import { Callout } from 'fumadocs-ui/components/callout';
-This page compares Cua Driver with other macOS computer-use tools. All are quality projects. The best choice depends on your workload.
+This page compares Cua Driver with other computer-use tools. The best choice depends on your workload. (Several of the alternatives below are macOS-specific; Cua Driver itself runs on macOS, Windows, and Linux.)
## Quick comparison
| Feature | Cua Driver | Codex Computer Use | Claude Computer Use | Lume |
| -------------------- | ----------------------- | -------------------- | -------------------- | ----------------------- |
| **License** | MIT | Closed source | Closed source | MIT |
-| **Drives host Mac** | Yes | Yes | No (sandbox only) | No (hosts VMs) |
+| **Drives host machine** | Yes | Yes | No (sandbox only) | No (hosts VMs) |
| **Sandbox / VM** | No | No | Yes (Cowork VM) | Yes (macOS + Linux VMs) |
| **Backgrounded** | Default | Default | N/A (sandbox) | N/A |
| **MCP server** | Yes (stdio) | No | No | Yes |
@@ -69,7 +69,7 @@ This is a different product shape. Cowork is sandbox-first: Claude operates an i
Cua Driver fits when all of these are true:
-- You want an agent to drive real apps on your own Mac (not a VM).
+- You want an agent to drive real apps on your own machine (not a VM).
- The user needs to keep working in another app while the agent operates.
- You're building against any MCP-capable agent, not locked to one vendor.
- MIT licensing matters.
diff --git a/docs/content/docs/cua-driver/guide/getting-started/faq.mdx b/docs/content/docs/cua-driver/guide/getting-started/faq.mdx
index 49a188ff9d..1fce755437 100644
--- a/docs/content/docs/cua-driver/guide/getting-started/faq.mdx
+++ b/docs/content/docs/cua-driver/guide/getting-started/faq.mdx
@@ -20,10 +20,10 @@ Two common causes:
```bash
# Populate the cache for this exact (pid, window_id).
-cua-driver get_window_state '{"pid":844,"window_id":10725}'
+cua-driver call get_window_state '{"pid":844,"window_id":10725}'
# Then act with the same window_id.
-cua-driver click '{"pid":844,"window_id":10725,"element_index":14}'
+cua-driver call click '{"pid":844,"window_id":10725,"element_index":14}'
```
If you're running one-shot CLI invocations without a daemon, the cache lives for one process lifetime. Start the daemon first:
@@ -107,8 +107,8 @@ A known Chromium renderer-IPC limit: the filter coerces synthetic right-click su
HTML5's click-to-play handler rejects some synthetic click paths. Use keyboard instead:
```bash
-cua-driver press_key '{"pid":<pid>,"key":"k"}' # YouTube play/pause
-cua-driver press_key '{"pid":<pid>,"key":"space"}' # generic video play/pause
+cua-driver call press_key '{"pid":<pid>,"key":"k"}' # YouTube play/pause
+cua-driver call press_key '{"pid":<pid>,"key":"space"}' # generic video play/pause
```
Keyboard events travel through a different auth envelope and reach the page.
@@ -118,7 +118,7 @@ Keyboard events travel through a different auth envelope and reach the page.
Pass the URL to `launch_app`:
```bash
-cua-driver launch_app '{"bundle_id":"com.google.Chrome","urls":["https://trycua.com"]}'
+cua-driver call launch_app '{"bundle_id":"com.google.Chrome","urls":["https://trycua.com"]}'
```
The URL opens in a new window via Chrome's `application(_:open:)` delegate. The driver's focus-restore guard catches Chrome's internal activation and clobbers the frontmost back to what it was before the call.
@@ -133,7 +133,7 @@ The URL opens in a new window via Chrome's `application(_:open:)` delegate. The
### How do I disable the visual cursor overlay?
```bash
-cua-driver set_agent_cursor_enabled '{"enabled":false}'
+cua-driver call set_agent_cursor_enabled '{"enabled":false}'
```
Or via config:
@@ -149,7 +149,7 @@ The overlay only renders when the driver has an AppKit run loop (inside `cua-dri
Tune the motion knobs:
```bash
-cua-driver set_agent_cursor_motion '{"glide_duration_ms":300}'
+cua-driver call set_agent_cursor_motion '{"glide_duration_ms":300}'
```
See `set_agent_cursor_motion` in the [MCP tools reference](/cua-driver/reference/mcp-tools) for every knob.
@@ -158,13 +158,13 @@ See `set_agent_cursor_motion` in the [MCP tools reference](/cua-driver/reference
### `check_permissions` says `NOT granted` but I granted both.
-TCC checks the calling process, not `CuaDriver.app`. Inside IDE terminals (Claude Code, Cursor, VS Code, Conductor), the shell inherits the IDE's TCC responsibility chain. So running `cua-driver check_permissions` in one of those shells reads against the IDE's bundle, not `com.trycua.driver`.
+TCC checks the calling process, not `CuaDriver.app`. Inside IDE terminals (Claude Code, Cursor, VS Code, Conductor), the shell inherits the IDE's TCC responsibility chain. So running `cua-driver call check_permissions` in one of those shells reads against the IDE's bundle, not `com.trycua.driver`.
Start the daemon first, which runs through LaunchServices under the CuaDriver bundle:
```bash
open -n -g -a CuaDriver --args serve
-cua-driver check_permissions # forwards to the daemon — authoritative answer
+cua-driver call check_permissions # forwards to the daemon — authoritative answer
```
### `cua-driver mcp` from an IDE terminal can't see Accessibility.
diff --git a/docs/content/docs/cua-driver/guide/getting-started/installation.mdx b/docs/content/docs/cua-driver/guide/getting-started/installation.mdx
index 6a1ae74fc9..e73df49099 100644
--- a/docs/content/docs/cua-driver/guide/getting-started/installation.mdx
+++ b/docs/content/docs/cua-driver/guide/getting-started/installation.mdx
@@ -69,10 +69,10 @@ irm https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/in
```
<Callout type="warn">
- **Known issue on Windows PowerShell 5.1**: `install.ps1` currently fails to parse on stock Windows PowerShell 5.1 (the version shipped with Windows 10 / Server 2019 / Server 2025) due to backtick + ampersand escaping in the post-install hint block — see [#1626](https://github.com/trycua/cua/issues/1626). Workaround until that lands: manual zip install:
+ **If `install.ps1` fails to parse on stock Windows PowerShell 5.1** (the version shipped with Windows 10 / Server 2019 / Server 2025; see [#1626](https://github.com/trycua/cua/issues/1626)), fall back to a manual zip install:
```powershell
- $version = "0.2.9"
+ $version = "0.3.2"
$arch = if ([System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture -eq 'Arm64') { 'arm64' } else { 'x86_64' }
$url = "https://github.com/trycua/cua/releases/download/cua-driver-rs-v$version/cua-driver-rs-$version-windows-$arch-binary.zip"
iwr $url -OutFile $env:TEMP\cua.zip -UseBasicParsing
@@ -171,9 +171,9 @@ Clear `$env:CUA_DRIVER_RS_VERSION` and re-run to roll forward to the newest rele
| Env var | Default (Linux/macOS) | Default (Windows) | What it controls |
|---|---|---|---|
| `CUA_DRIVER_RS_VERSION` | unset → use baked version | unset → use baked version | Pin a specific release (e.g. `0.2.0`). |
-| `CUA_DRIVER_RS_INSTALL_DIR` | `~/.local/bin` | `%LOCALAPPDATA%\Programs\trycua\cua-driver-rs\bin` | The visible PATH-entry directory. On Windows this is itself a junction. |
-| `CUA_DRIVER_RS_HOME` | `~/.cua-driver-rs` | `%USERPROFILE%\.cua-driver-rs` | Package home — holds `packages/releases/<v>/` and `packages/current`. |
-| `CUA_DRIVER_RS_NO_MODIFY_PATH` *(Linux/macOS only)* | `0` | use `-NoPathUpdate` switch on `install.ps1` | Skip the PATH append. On Windows the installer appends `%LOCALAPPDATA%\Programs\trycua\cua-driver-rs\bin` to the User-scope `Path` by default; use the fetch+invoke form with `-NoPathUpdate` to opt out (see the Windows install section above for the exact one-liner). |
+| `CUA_DRIVER_RS_INSTALL_DIR` | `~/.local/bin` | `%LOCALAPPDATA%\Programs\Cua\cua-driver\bin` | The visible PATH-entry directory. On Windows this is itself a junction. |
+| `CUA_DRIVER_RS_HOME` | `~/.cua-driver-rs` | `%USERPROFILE%\.cua-driver` | Package home — holds `packages/releases/<v>/` and `packages/current`. |
+| `CUA_DRIVER_RS_NO_MODIFY_PATH` *(Linux/macOS only)* | `0` | use `-NoPathUpdate` switch on `install.ps1` | Skip the PATH append. On Windows the installer appends `%LOCALAPPDATA%\Programs\Cua\cua-driver\bin` to the User-scope `Path` by default; use the fetch+invoke form with `-NoPathUpdate` to opt out (see the Windows install section above for the exact one-liner). |
| `CUA_DRIVER_RS_KEEP_VERSIONS` *(Linux/Windows only)* | `5` | `5` | Keep the N most recent per-version release dirs after install (`0` disables GC). See the **Old-version cleanup** callout below for per-target and active-install invariants. |
<Callout type="info">
@@ -225,7 +225,7 @@ cat ~/.cua-driver-rs/packages/.install.lock.d/info
```
```powershell
-Get-Content $env:USERPROFILE\.cua-driver-rs\install.lock
+Get-Content $env:USERPROFILE\.cua-driver\install.lock
# pid=43210
# started=2026-05-17T09:14:22.123Z
# invocation=install.ps1 -Release latest
@@ -247,17 +247,17 @@ Both primitives are unprivileged — no sudo, no admin elevation, no Developer M
```bash
cua-driver --version
-# cua-driver 0.1.0
+# cua-driver 0.3.2
cua-driver --help
-# OVERVIEW: macOS Accessibility-driven computer-use agent — MCP stdio server.
+# cua-driver 0.3.2 — cross-platform computer-use automation driver
```
For a structured environment + install report on any platform, run:
```bash
cua-driver doctor
-# [ok ] binary: cua-driver 0.2.1 (aarch64-macos)
+# [ok ] binary: cua-driver 0.3.2 (aarch64-macos)
# [ok ] install dir: /Users/you/.local/bin/cua-driver
# [ok ] home dir: /Users/you/.cua-driver-rs (3 release dirs cached)
# [ok ] telemetry: enabled (install-id present)
@@ -313,7 +313,7 @@ The Windows-native equivalent of macOS's LaunchAgent is a **Scheduled Task trigg
`install.ps1` prints the exact one-shot registration block on a successful install. To register it manually (substitute your install path if you built from source):
```powershell
-$exe = "$env:LOCALAPPDATA\Programs\trycua\cua-driver-rs\bin\cua-driver.exe"
+$exe = "$env:LOCALAPPDATA\Programs\Cua\cua-driver\bin\cua-driver.exe"
$user = "$env:COMPUTERNAME\$env:USERNAME"
$action = New-ScheduledTaskAction -Execute $exe -Argument 'serve' -WorkingDirectory $env:USERPROFILE
$trigger = New-ScheduledTaskTrigger -AtLogOn -User $user
@@ -355,13 +355,13 @@ open -n -g -a CuaDriver --args serve
Then trigger the prompts:
```bash
-cua-driver check_permissions
+cua-driver call check_permissions
```
macOS raises the Accessibility and Screen Recording system dialogs. Grant both, then re-run the command to confirm:
```bash
-cua-driver check_permissions
+cua-driver call check_permissions
# ✅ Accessibility: granted.
# ✅ Screen Recording: granted.
```
@@ -425,9 +425,9 @@ If a grant still reads `NOT granted` after granting in the dialog, open **System
two-line banner to stderr when a newer `cua-driver-rs-v*` release exists:
```
- ✨ cua-driver v0.1.4 is available (you have v0.1.3).
+ ✨ cua-driver v0.3.2 is available (you have v0.3.1).
Update with: cua-driver update
- Release notes: https://github.com/trycua/cua/releases/tag/cua-driver-rs-v0.1.4
+ Release notes: https://github.com/trycua/cua/releases/tag/cua-driver-rs-v0.3.2
```
The answer is cached at `~/.cua-driver-rs/version_check.json` for ~20
@@ -489,7 +489,7 @@ cua-driver stop
## Register with an MCP client (optional)
-The MCP server is one of two ways to use cua-driver — skip this section if you only want the CLI (`cua-driver list_apps`, etc.). You can register both later, MCP and CLI work side-by-side.
+The MCP server is one of two ways to use cua-driver — skip this section if you only want the CLI (`cua-driver call list_apps`, etc.). You can register both later, MCP and CLI work side-by-side.
Cua Driver speaks MCP over stdio. Use `cua-driver mcp-config --client <name>` to print the right command for your client, or copy one of the snippets below directly:
@@ -536,7 +536,7 @@ cua-driver mcp-config | pbcopy
```
<Callout type="info">
-**Pi (badlogic/pi-mono)** does not support MCP natively. Use cua-driver as a plain CLI from inside Pi instead — `cua-driver list_apps`, `cua-driver click '{...}'`, etc. — which is exactly the shape Pi is built around. Run `cua-driver mcp-config --client pi` for the full guidance, or see the [Quickstart](/cua-driver/guide/getting-started/quickstart) for CLI usage.
+**Pi (badlogic/pi-mono)** does not support MCP natively. Use cua-driver as a plain CLI from inside Pi instead — `cua-driver call list_apps`, `cua-driver call click '{...}'`, etc. — which is exactly the shape Pi is built around. Run `cua-driver mcp-config --client pi` for the full guidance, or see the [Quickstart](/cua-driver/guide/getting-started/quickstart) for CLI usage.
</Callout>
The client spawns `cua-driver mcp` on demand once registered.
diff --git a/docs/content/docs/cua-driver/guide/getting-started/integrations.mdx b/docs/content/docs/cua-driver/guide/getting-started/integrations.mdx
index 6ed65a5a45..fb6ab861d8 100644
--- a/docs/content/docs/cua-driver/guide/getting-started/integrations.mdx
+++ b/docs/content/docs/cua-driver/guide/getting-started/integrations.mdx
@@ -8,8 +8,8 @@ import { Callout } from 'fumadocs-ui/components/callout';
`cua-driver mcp` is a stdio MCP server. Any agent that supports MCP can use it — no extra setup beyond adding it to the agent's MCP config.
<Callout type="warn">
- Grant Accessibility and Screen Recording permissions to cua-driver before connecting any agent.
- Run `cua-driver check_permissions` to verify.
+ Grant the required permissions before connecting any agent — on macOS that's Accessibility and Screen
+ Recording (Windows and Linux have no equivalent prompt). Run `cua-driver call check_permissions` to verify.
</Callout>
## Claude Code
@@ -132,7 +132,7 @@ If you are using a vision-capable model via Ollama, you must also declare its in
"npm": "@ai-sdk/openai-compatible",
"options": { "baseURL": "http://localhost:11434/v1" },
"models": {
- "gemma4:26b": {
+ "llama3.2-vision": {
"modalities": {
"input": ["text", "image"],
"output": ["text"]
diff --git a/docs/content/docs/cua-driver/guide/getting-started/introduction.mdx b/docs/content/docs/cua-driver/guide/getting-started/introduction.mdx
index 2c14126674..5220526e27 100644
--- a/docs/content/docs/cua-driver/guide/getting-started/introduction.mdx
+++ b/docs/content/docs/cua-driver/guide/getting-started/introduction.mdx
@@ -1,11 +1,13 @@
---
title: What is Cua Driver?
-description: Background computer-use driver for any agent on macOS
+description: Background computer-use driver for any agent — macOS, Windows, and Linux
---
import { Callout } from 'fumadocs-ui/components/callout';
-Cua Driver is a macOS computer-use driver that speaks MCP over stdio. It lets any agent (Claude, GPT, Gemini, Codex, custom loops) click, type, scroll, and snapshot a native macOS app without bringing the target to the foreground. Your frontmost app stays where it is; the user keeps typing in their editor while the agent drives something else in the background.
+Cua Driver is a computer-use driver that speaks MCP over stdio. It lets any agent (Claude, GPT, Gemini, Codex, custom loops) click, type, scroll, and snapshot a native app without bringing the target to the foreground. Your frontmost app stays where it is; you keep typing in your editor while the agent drives something else in the background.
+
+It runs on macOS, Windows, and Linux from one tool surface. macOS ships the Swift driver (Accessibility / `CGEvent` / ScreenCaptureKit); Windows and Linux ship `cua-driver-rs`, the Rust port (UI Automation / `PostMessage` / AT-SPI). The tool names and arguments are the same everywhere — see the [MCP tools reference](/cua-driver/reference/mcp-tools).
<Callout type="info" title="MIT License">
Cua Driver is open-source and MIT licensed. If you find it useful, we'd appreciate a [star on
@@ -13,56 +15,57 @@ Cua Driver is a macOS computer-use driver that speaks MCP over stdio. It lets an
</Callout>
```bash
-cua-driver launch_app '{"bundle_id":"com.apple.calculator"}'
-cua-driver get_window_state '{"pid":844,"window_id":10725}'
-cua-driver click '{"pid":844,"window_id":10725,"element_index":14}'
+cua-driver call launch_app '{"name":"Calculator"}'
+cua-driver call get_window_state '{"pid":844,"window_id":10725}'
+cua-driver call click '{"pid":844,"window_id":10725,"element_index":14}'
```
-A single binary. Launch it as an MCP stdio server, a long-running daemon, or invoke any tool directly from the shell.
+A single binary. Run it as an MCP stdio server (`cua-driver mcp`), a long-running daemon (`cua-driver serve`), or invoke any tool from the shell (`cua-driver call <tool>`).
## The no-foreground contract
-One rule: the user's frontmost app does not change. Not during launch, not during a click, not during a keystroke, not during a re-snapshot. Three corollaries follow:
+One rule: the user's frontmost window does not change. Not during launch, not during a click, not during a keystroke, not during a re-snapshot. So:
- The real cursor stays where the user left it. No warp.
-- The target window stays at its current z-rank. No raise.
-- The user's Space does not follow the target. No bounce.
+- The target window stays at its current z-order. No raise, no focus steal.
+- On macOS, the user's Space does not follow the target. No bounce.
-Every dispatch path inside the driver honors those four invariants. `launch_app` runs hidden. Keyboard tools post via `CGEvent.postToPid` scoped to a named pid, so a driver-backgrounded app cannot leak keystrokes into the user's foreground app. Pixel clicks route through an auth-signed `SLEventPostToPid` recipe that borrows yabai's focus-without-raise pattern. Element-indexed clicks go through `AXUIElementPerformAction` directly and skip event synthesis entirely.
+Backgrounded dispatch is the default. `launch_app` runs hidden; element-indexed actions go through the accessibility API directly (no event synthesis); keyboard and pixel events are posted to a specific target rather than the system input queue. When a target's input stack would silently drop a backgrounded event — Chromium DOM content, some GTK widgets — the tool says so (`background_unavailable`) instead of failing quietly, and you opt into a one-time foreground swap with [`bring_to_front`](/cua-driver/reference/mcp-tools#bring_to_front) plus `dispatch: "foreground"`.
-## Three modalities
+## Three capture modes
`capture_mode` controls what `get_window_state` returns. Pick based on what the agent needs:
-- **`som`** (default) — set-of-mark. Both the AX tree and the screenshot. Tree for dispatch, screenshot for visual disambiguation when labels repeat or stay empty. Works out of the box for element-indexed clicks.
-- **`ax`** — accessibility tree only. No Screen Recording cost, deterministic element addressing. Best for structured loops over apps with real AX coverage.
-- **`vision`** — window PNG only. No AX walk. Best for vision-first models that ground on pixels and don't use element_index. Pair with pixel-addressed clicks.
+- **`som`** (default) — set-of-mark: both the accessibility tree and a screenshot. The tree for element addressing, the screenshot for visual disambiguation when labels repeat or are empty.
+- **`ax`** — accessibility tree only. No screenshot cost; deterministic element addressing. Best for structured loops over apps with real accessibility coverage.
+- **`vision`** — window screenshot only, no tree walk. Best for vision-first models that ground on pixels. Pair with pixel-addressed clicks.
-Backgrounded drive is the default across all three, not a mode you toggle. Switch modalities with `cua-driver config set capture_mode som`.
+Switch with `cua-driver config set capture_mode som`.
## How it works
-Three dispatch paths, one per modality the target exposes.
+The accessibility tree is the primary addressing surface on every platform; the driver walks it, tags each actionable node with an index, and caches the element against `(pid, window_id)`. Element-indexed clicks then dispatch through the accessibility API with no cursor move and no focus steal.
-- **Accessibility elements, via public AX.** Where the target has a real AX tree, the driver walks it, tags every actionable node with an index, and caches the `AXUIElement` ref against `(pid, window_id)`. Clicks go through `AXUIElementPerformAction` directly.
-- **Chromium and Electron trees, via an AX observer SPI.** Chrome, Slack, VS Code, Discord, and every Electron app pause AX tree updates when occluded unless the observer is registered via a private SPI variant. The driver uses that variant so the tree stays populated through the full launch-snapshot-act loop without bringing the target forward.
-- **Non-AX surfaces, via `SLEventPostToPid`.** Canvas, WebView, HTML5 video, custom-drawn controls. A backgrounded click recipe stamps events through SkyLight's auth-signed per-pid path. The cursor never moves, the window never rises, Spaces never follows.
+- **macOS** — the public Accessibility API for the tree and `AXUIElementPerformAction` for clicks; keyboard via `CGEvent.postToPid`; backgrounded pixel events via SkyLight's per-pid path. Chromium and Electron trees stay populated while occluded via an AX observer SPI.
+- **Windows** — UI Automation for the tree, `PostMessage` for input. XAML / WinUI3 / UWP hosts (Calculator, Settings, modern Notepad) are driven through a focus-steal bypass; an optional UIAccess worker (`cua-driver-uia.exe`) lifts UIPI for packaged apps.
+- **Linux** — AT-SPI for the tree over X11 / XWayland. See [Running on Linux](/cua-driver/guide/getting-started/linux).
-Keyboard is simpler. Every key goes through `CGEvent.postToPid` scoped to the named pid. There is no frontmost-routed variant in the API surface.
+Non-accessible surfaces (canvas, WebGL, custom-drawn controls) fall back to pixel coordinates against the same window-local screenshot space.
## Who it's for
-- **Agent builders** who want their agent to operate macOS without stealing the user's focus. Any MCP-capable client works: Claude Code, Cursor, Codex, Gemini, custom loops.
-- **Dev-loop automation.** An agent drives an app, reads the pixels or the AX tree, edits source, rebuilds, verifies. The editor stays frontmost the entire time.
-- **Demo capture.** Record a trajectory while the user works in another app. Because the clicks are backgrounded, the overlay cursor the driver paints is the only cursor in the final video.
+- **Agent builders** who want their agent to operate a desktop without stealing the user's focus. Any MCP-capable client works: Claude Code, Cursor, Codex, Gemini, custom loops.
+- **Dev-loop automation.** An agent drives an app, reads the pixels or the accessibility tree, edits source, rebuilds, verifies — while the editor stays frontmost.
+- **Demo capture.** Record a trajectory while the user works elsewhere. Because the clicks are backgrounded, the overlay cursor the driver paints is the only cursor in the final video.
## What it doesn't do
-- Requires macOS 14 (Sonoma) or later. Works on Apple Silicon and Intel.
-- Not a VM. Cua Driver operates the real host, so grant Accessibility and Screen Recording with intent.
-- No right-click on Chromium web content through pixel synthesis: the renderer-IPC filter drops right-click subtype on non-HID-tap paths. Use `right_click({pid, element_index})` on AX-addressable targets. See [Limits](/cua-driver/reference/limits).
-- Canvas apps (Blender, Unity, games) need a brief frontmost activation because their event loops filter per-pid-routed events. Everything else stays backgrounded.
+- Not a VM. Cua Driver operates the real host, so grant accessibility permissions with intent. (macOS also needs Screen Recording for screenshots; Windows and Linux have no equivalent prompt.)
+- Right-click on Chromium **web content** via pixel coordinates fires as a left-click — the renderer drops the right-click subtype on synthetic paths. Use element-indexed `right_click` on accessible targets. See [Limits](/cua-driver/reference/limits).
+- Canvas apps (Blender, Unity, games) filter per-target events, so they need a brief foreground activation. Everything else stays backgrounded.
+
+See [Installation](/cua-driver/guide/getting-started/installation) for per-platform requirements.
## Get started
-Ready to try it? [Install Cua Driver](/cua-driver/guide/getting-started/installation) and drive your first app in the [Quickstart](/cua-driver/guide/getting-started/quickstart).
+[Install Cua Driver](/cua-driver/guide/getting-started/installation), then drive your first app in the [Quickstart](/cua-driver/guide/getting-started/quickstart).
diff --git a/docs/content/docs/cua-driver/guide/getting-started/linux.mdx b/docs/content/docs/cua-driver/guide/getting-started/linux.mdx
index 6efec9120b..dd89cffcd2 100644
--- a/docs/content/docs/cua-driver/guide/getting-started/linux.mdx
+++ b/docs/content/docs/cua-driver/guide/getting-started/linux.mdx
@@ -63,7 +63,7 @@ The `cua-driver autostart` verb family is **Windows-only today**. On Linux it cu
```text
cua-driver autostart is currently Windows-only. macOS users: see
-libs/cua-driver/rust/scripts/install-local.sh --autostart for the
+libs/cua-driver/scripts/install-local.sh --autostart for the
LaunchAgent recipe. Linux users: same script registers a systemd
--user unit. A cross-platform impl is tracked as a follow-up.
```
@@ -72,7 +72,7 @@ The two working alternatives:
### Option A — Use `install-local.sh --autostart`
-The dev-loop install script registers a systemd user unit when invoked with `--autostart`. See [`libs/cua-driver/rust/scripts/install-local.sh`](https://github.com/trycua/cua/blob/main/libs/cua-driver/rust/scripts/install-local.sh) — built for local development off a git checkout, but the systemd unit it writes is the same shape you'd use in production.
+The dev-loop install script registers a systemd user unit when invoked with `--autostart`. See [`libs/cua-driver/scripts/install-local.sh`](https://github.com/trycua/cua/blob/main/libs/cua-driver/scripts/install-local.sh) — built for local development off a git checkout, but the systemd unit it writes is the same shape you'd use in production.
### Option B — Write your own systemd user unit
@@ -154,5 +154,5 @@ See [`libs/cua-driver/rust/PARITY.md`](https://github.com/trycua/cua/blob/main/l
- [Installation](/cua-driver/guide/getting-started/installation) — canonical one-liner that handles Linux automatically
- [Autostart](/cua-driver/guide/getting-started/autostart) — concept page (Windows-only verb family; Linux uses systemd)
-- [Process attribution](/cua-driver/explanation/process-attribution) — Linux has none of the macOS-TCC or Windows-Session-0 weirdness; this page explains what those problems are and why Linux sidesteps them
+- Process attribution — Linux has none of the macOS-TCC or Windows-Session-0 weirdness. The [macOS process model](/cua-driver/guide/getting-started/process-model) and [SSH on Windows](/cua-driver/guide/getting-started/windows-ssh) pages explain those problems and why Linux sidesteps them
- [`PARITY.md`](https://github.com/trycua/cua/blob/main/libs/cua-driver/rust/PARITY.md) — per-tool platform verification matrix
diff --git a/docs/content/docs/cua-driver/guide/getting-started/pip-preview.mdx b/docs/content/docs/cua-driver/guide/getting-started/pip-preview.mdx
index 4074e7afdc..05ae69c0fb 100644
--- a/docs/content/docs/cua-driver/guide/getting-started/pip-preview.mdx
+++ b/docs/content/docs/cua-driver/guide/getting-started/pip-preview.mdx
@@ -30,7 +30,7 @@ the args list.
```
Both keys are optional; `experimental_pip_geometry` defaults to
-`320x200` in the top-right corner. Restart your MCP client (or kill
+`480x360` in the top-right corner. Restart your MCP client (or kill
the running `cua-driver` daemon) for the new config to take effect.
### B. One-off — CLI flag
diff --git a/docs/content/docs/cua-driver/guide/getting-started/process-model.mdx b/docs/content/docs/cua-driver/guide/getting-started/process-model.mdx
index f178e1ffa8..c6bd667258 100644
--- a/docs/content/docs/cua-driver/guide/getting-started/process-model.mdx
+++ b/docs/content/docs/cua-driver/guide/getting-started/process-model.mdx
@@ -7,6 +7,14 @@ import { Callout } from 'fumadocs-ui/components/callout';
`cua-driver mcp` runs in one of two modes depending on the TCC context it inherits from its parent process. Both expose the **exact same MCP tool surface** — same names, same arguments, same response shapes — so MCP clients never need to care which mode is active. This page documents how the two modes differ at the process level, so wrapper authors (Hermes, custom MCP shims, etc.) and operators debugging the lifecycle have a precise mental model.
+<Callout type="info">
+ This page describes the **macOS** driver, where the daemon proxy exists to inherit `CuaDriver.app`'s TCC
+ grants. The Rust port (`cua-driver-rs`) has the same daemon-proxy mechanism on Windows and Linux, but it
+ solves a different problem — reaching the interactive desktop from a Session 0 SSH login — and uses
+ `CUA_DRIVER_RS_MCP_NO_RELAUNCH` rather than `CUA_DRIVER_MCP_NO_RELAUNCH`. See
+ [Running cua-driver under SSH on Windows](/cua-driver/guide/getting-started/windows-ssh).
+</Callout>
+
## The two modes
### In-process mode (default when TCC context is correct)
@@ -18,7 +26,7 @@ cua-driver.app (LaunchServices) → cua-driver mcp (stdio)
attributed to com.trycua.driver
```
-When `cua-driver mcp` is spawned by `CuaDriver.app` directly (LaunchServices-attributed), the process inherits the bundle's TCC grants. Every tool runs in-process: AX walks, screenshots, click synthesis, the agent-cursor overlay, all of it. Lifecycle is dead simple — the process lives for the duration of the stdio session and exits when stdin closes.
+When `cua-driver mcp` is spawned by `CuaDriver.app` directly (LaunchServices-attributed), the process inherits the bundle's TCC grants. Every tool runs in-process: AX walks, screenshots, click synthesis, the agent-cursor overlay, all of it. The lifecycle is simple: the process lives for the duration of the stdio session and exits when stdin closes.
### Daemon-proxy mode (auto-engaged from IDE terminals)
@@ -129,7 +137,7 @@ MCPError.internalError("cua-driver daemon not reachable on <socket>. Start it wi
`open -n -g -a CuaDriver --args serve` and retry.")
```
-The MCP client surfaces this as a tool error. The mcp-client process **does not** auto-restart the daemon mid-session. The recommended client behavior is: catch the error, optionally restart the daemon yourself (`open -n -g -a CuaDriver --args serve`), reconnect the MCP transport, and retry. Hermes' wrapper in [trycua/hermes#22821](https://github.com/trycua/hermes/pull/22821) implements exactly this pattern.
+The MCP client surfaces this as a tool error. The mcp-client process **does not** auto-restart the daemon mid-session. The recommended client behavior is: catch the error, optionally restart the daemon yourself (`open -n -g -a CuaDriver --args serve`), reconnect the MCP transport, and retry.
### Socket path ownership
diff --git a/docs/content/docs/cua-driver/guide/getting-started/quickstart.mdx b/docs/content/docs/cua-driver/guide/getting-started/quickstart.mdx
index f3915cc994..f3bddee9fc 100644
--- a/docs/content/docs/cua-driver/guide/getting-started/quickstart.mdx
+++ b/docs/content/docs/cua-driver/guide/getting-started/quickstart.mdx
@@ -5,7 +5,14 @@ description: Drive your first macOS app with Cua Driver
import { Callout } from 'fumadocs-ui/components/callout';
-In under 5 minutes, you'll launch a macOS app in the background, snapshot its AX tree, click a button by its element index, and verify the action landed.
+You'll launch an app in the background, snapshot its accessibility tree, click a button by element index, and verify the action landed.
+
+<Callout type="info">
+ The commands below use macOS (`open -n -g -a CuaDriver`, `⌘Q`). On Windows start the daemon with
+ [`cua-driver autostart`](/cua-driver/guide/getting-started/autostart) and on Linux with `cua-driver serve &`
+ ([Linux guide](/cua-driver/guide/getting-started/linux)); the `cua-driver call <tool>` invocations are identical
+ across platforms.
+</Callout>
## Start the daemon
@@ -33,7 +40,7 @@ cua-driver config set capture_mode som
## Launch an app (hidden)
```bash
-cua-driver launch_app '{"bundle_id":"com.apple.calculator"}'
+cua-driver call launch_app '{"bundle_id":"com.apple.calculator"}'
```
Output:
@@ -51,7 +58,7 @@ The app's pid and a `windows` array come back in one call. `launch_app` is idemp
## Snapshot the window
```bash
-cua-driver get_window_state '{"pid":844,"window_id":10725}'
+cua-driver call get_window_state '{"pid":844,"window_id":10725}'
```
Output (trimmed):
@@ -75,13 +82,13 @@ Every actionable element is tagged with `[N]` — that's the `element_index` you
<Callout type="info">
Large trees (Finder is ~1600 elements) exceed most LLM context limits. Pass a `query` field to filter
- the Markdown to matching lines plus their ancestors: `get_window_state '{"pid":844,"window_id":10725,"query":"Three"}'`.
+ the Markdown to matching lines plus their ancestors: `cua-driver call get_window_state '{"pid":844,"window_id":10725,"query":"Three"}'`.
</Callout>
## Click by element_index
```bash
-cua-driver click '{"pid":844,"window_id":10725,"element_index":14}'
+cua-driver call click '{"pid":844,"window_id":10725,"element_index":14}'
# ✅ Performed AXPress on [14] AXButton "Three".
```
@@ -92,7 +99,7 @@ No cursor moved. Calculator never came to the foreground. The click went through
Re-snapshot and check the AX tree diff:
```bash
-cua-driver get_window_state '{"pid":844,"window_id":10725}'
+cua-driver call get_window_state '{"pid":844,"window_id":10725}'
```
Look for the display `AXStaticText` reading `3`. If the tree didn't change, the action failed silently, and you should say so rather than assume success.
@@ -103,10 +110,10 @@ Pixel clicks are for surfaces the AX tree doesn't reach: canvases, video players
```bash
# Write the screenshot to disk. Works in every capture mode.
-cua-driver get_window_state '{"pid":844,"window_id":10725}' --screenshot-out-file /tmp/shot.png
+cua-driver call get_window_state '{"pid":844,"window_id":10725}' --screenshot-out-file /tmp/shot.png
# Look at /tmp/shot.png, pick a target pixel, then:
-cua-driver click '{"pid":844,"window_id":10725,"x":120,"y":240}'
+cua-driver call click '{"pid":844,"window_id":10725,"x":120,"y":240}'
# ✅ Posted click to pid 844.
```
@@ -121,7 +128,7 @@ The PNG is capped at 1568 px long-side by default (matching Anthropic's multimod
```bash
# Quit the target app via a hotkey to its pid.
-cua-driver hotkey '{"pid":844,"keys":["cmd","q"]}'
+cua-driver call hotkey '{"pid":844,"keys":["cmd","q"]}'
# Stop the daemon.
cua-driver stop
@@ -135,16 +142,16 @@ Every multi-step workflow follows the same shape:
open -n -g -a CuaDriver --args serve
# 1. Launch the target (idempotent; returns pid + windows).
-cua-driver launch_app '{"bundle_id":"..."}'
+cua-driver call launch_app '{"bundle_id":"..."}'
# 2. Snapshot the window you care about (populates the element cache).
-cua-driver get_window_state '{"pid":<pid>,"window_id":<wid>}'
+cua-driver call get_window_state '{"pid":<pid>,"window_id":<wid>}'
# 3. Dispatch an action by element_index or pixel coordinates.
-cua-driver click '{"pid":<pid>,"window_id":<wid>,"element_index":<n>}'
+cua-driver call click '{"pid":<pid>,"window_id":<wid>,"element_index":<n>}'
# 4. Re-snapshot to verify the action landed.
-cua-driver get_window_state '{"pid":<pid>,"window_id":<wid>}'
+cua-driver call get_window_state '{"pid":<pid>,"window_id":<wid>}'
cua-driver stop
```
diff --git a/docs/content/docs/cua-driver/guide/getting-started/windows-ssh.mdx b/docs/content/docs/cua-driver/guide/getting-started/windows-ssh.mdx
index 874f7d81f8..b20b8e2df8 100644
--- a/docs/content/docs/cua-driver/guide/getting-started/windows-ssh.mdx
+++ b/docs/content/docs/cua-driver/guide/getting-started/windows-ssh.mdx
@@ -71,7 +71,7 @@ The daemon runs in the user's interactive session (registered via `cua-driver au
cua-driver autostart enable # register the Scheduled Task
cua-driver autostart kick # start it now, don't wait for next logon
-# ── from SSH (or any session — Just Works) ───────────────────────────
+# ── from SSH (or any session) ────────────────────────────────────────
cua-driver status
# cua-driver daemon is running
# socket: \\.\pipe\cua-driver
diff --git a/docs/content/docs/cua-driver/reference/cli-reference.mdx b/docs/content/docs/cua-driver/reference/cli-reference.mdx
index 99e3bdfc43..99366a7493 100644
--- a/docs/content/docs/cua-driver/reference/cli-reference.mdx
+++ b/docs/content/docs/cua-driver/reference/cli-reference.mdx
@@ -4,399 +4,189 @@ description: Command Line Interface reference for Cua Driver
---
{/*
- AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY
- Generated by: npx tsx scripts/docs-generators/cua-driver.ts
- Source: recursive Swift sources under libs/cua-driver/Sources
- Version: 0.2.0
+ Synced to the cua-driver-rs 0.3.2 binary (cross-platform).
+ Source of truth: `cua-driver --help`. Per-verb flag introspection is not
+ exposed by the Rust CLI, so this page is maintained by hand against --help.
*/}
import { Callout } from 'fumadocs-ui/components/callout';
import { VersionHeader } from '@/components/version-selector';
<VersionHeader
- versions={[{"version":"0.2","href":"/cua-driver/reference/cli-reference","isCurrent":true}]}
- currentVersion="0.2"
- fullVersion="0.2.0"
+ versions={[{"version":"0.3","href":"/cua-driver/reference/cli-reference","isCurrent":true}]}
+ currentVersion="0.3"
+ fullVersion="0.3.2"
packageName="cua-driver"
installCommand="curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.sh | bash"
/>
-macOS Accessibility-driven computer-use agent — MCP stdio server.
+Cross-platform computer-use automation driver — MCP stdio server plus a shell CLI. The same binary serves macOS (Swift driver) and Windows / Linux (`cua-driver-rs`).
-## Tool dispatch
-
-### cua-driver call
-
-Invoke an MCP tool directly from the shell.
+`cua-driver --help` prints the authoritative subcommand list; `cua-driver --version` prints the version. The subcommands are: `mcp`, `list-tools`, `describe`, `call`, `serve`, `stop`, `status`, `mcp-config`, `config`, `recording`, `update`, `check-update`, `doctor`, `diagnose`, `autostart`, `skills`.
-Runs the same ToolHandler the MCP server uses. JSON-ARGS is a JSON
-object mapping to the tool's inputSchema. When JSON-ARGS is omitted
-and stdin is a pipe, JSON is read from stdin. When neither is
-provided, the tool is called with no arguments.
+<Callout type="warn">
+**Passing JSON on Windows PowerShell 5.1.** PowerShell 5.1 strips the quotes around JSON field names in multi-field arguments. Pipe the JSON via stdin instead, or use PowerShell 7+ (`pwsh`):
-Examples:
- cua-driver call list_apps
- cua-driver call launch_app '&#123;"bundle_id":"com.apple.finder"&#125;'
- echo '&#123;"pid":844,"window_id":1234&#125;' | cua-driver call get_window_state
+```powershell
+'{"pid":1234,"window_id":5678}' | cua-driver call get_window_state
+```
+</Callout>
-**Arguments:**
+## Tool dispatch
-| Name | Type | Required | Description |
-| ---- | ---- | -------- | ----------- |
-| `<tool-name>` | String | Yes | Name of the tool to invoke (see `cua-driver list-tools`). |
-| `<json-args>` | String | No | JSON object string for the tool's inputSchema. If omitted, reads from stdin when stdin is a pipe. |
+### cua-driver call
-**Options:**
+Invoke an MCP tool directly from the shell — the same code path the MCP server uses.
-| Name | Type | Default | Description |
-| ---- | ---- | ------- | ----------- |
-| `--screenshot-out-file` | String | — | Write the first image content block from the response to this file path. |
-| `--socket` | String | — | Override the daemon Unix socket path. |
+```bash
+cua-driver call <tool-name> '<json-args>'
+```
-**Flags:**
+`<json-args>` is a JSON object matching the tool's input schema. When it's omitted and stdin is a pipe, the JSON is read from stdin; when neither is provided, the tool is called with no arguments. When a daemon is running, `call` forwards to it (so the per-pid `element_index` cache is shared across invocations); use `--socket` to target a non-default daemon socket / pipe.
-| Name | Description |
-| ---- | ----------- |
-| `--raw` | Print the raw CallTool.Result JSON (content + structuredContent + isError) instead of unwrapping structuredContent. |
-| `--compact` | Emit minified JSON instead of pretty-printed. |
-| `--no-daemon` | Skip the cua-driver daemon even if one is running. |
+```bash
+cua-driver call list_apps
+cua-driver call launch_app '{"name":"Calculator"}'
+echo '{"pid":844,"window_id":10725}' | cua-driver call get_window_state
+```
### cua-driver list-tools
-List every tool exposed by cua-driver with its one-line description.
-
-**Options:**
-
-| Name | Type | Default | Description |
-| ---- | ---- | ------- | ----------- |
-| `--socket` | String | — | Override the daemon Unix socket path. |
-
-**Flags:**
-
-| Name | Description |
-| ---- | ----------- |
-| `--no-daemon` | Skip the cua-driver daemon even if one is running. |
+List every MCP tool with its one-line description.
### cua-driver describe
Print a tool's full description and JSON input schema.
-**Arguments:**
-
-| Name | Type | Required | Description |
-| ---- | ---- | -------- | ----------- |
-| `<tool-name>` | String | Yes | Name of the tool to describe. |
-
-**Options:**
-
-| Name | Type | Default | Description |
-| ---- | ---- | ------- | ----------- |
-| `--socket` | String | — | Override the daemon Unix socket path. |
-
-**Flags:**
-
-| Name | Description |
-| ---- | ----------- |
-| `--compact` | Emit minified JSON instead of pretty-printed. |
-| `--no-daemon` | Skip the cua-driver daemon even if one is running. |
+```bash
+cua-driver describe click
+```
## Daemon management
### cua-driver mcp
-Run the stdio MCP server.
+Run the stdio MCP server. When spawned from an IDE terminal, `mcp` may proxy tool calls to a running `cua-driver serve` daemon so they execute under the right process identity — on macOS to inherit `CuaDriver.app`'s TCC grants, on Windows to reach the interactive desktop from a Session 0 SSH login. Tool semantics are identical to the in-process path. See the [process model](/cua-driver/guide/getting-started/process-model) and [SSH on Windows](/cua-driver/guide/getting-started/windows-ssh).
-When invoked from a shell or IDE terminal (Claude Code, Cursor,
-VS Code, Warp), macOS TCC attributes the process to the parent
-terminal — not to CuaDriver.app — so AX probes silently fail
-against the wrong bundle id. To sidestep this without breaking
-the stdio MCP transport, `mcp` detects the context, ensures a
-`cua-driver serve` daemon is running under LaunchServices
-(relaunching via `open -n -g -a CuaDriver --args serve` if not),
-and proxies every MCP tool call through the daemon's Unix
-socket. Tool semantics are identical to the in-process path.
-Pass `--no-daemon-relaunch` (or set CUA_DRIVER_MCP_NO_RELAUNCH=1)
-to force in-process execution — useful when the calling context
-already has the right TCC grants (e.g. spawned from CuaDriver.app
-directly), or for diagnosing in-process failures.
-
-**Options:**
-
-| Name | Type | Default | Description |
-| ---- | ---- | ------- | ----------- |
-| `--socket` | String | — | Override the daemon Unix socket path used by the proxy fallback. |
-
-**Flags:**
+**Flags / options:**
| Name | Description |
| ---- | ----------- |
-| `--claude-code-computer-use-compat` | Expose normal CuaDriver tools, replacing only `screenshot` with a Claude Code-friendly window-only screenshot that establishes the vision coordinate frame. |
-| `--no-daemon-relaunch` | Stay in the current process instead of auto-launching a daemon and proxying through its Unix socket when invoked from a shell without CuaDriver.app's TCC grants. Also toggleable via CUA_DRIVER_MCP_NO_RELAUNCH=1. |
+| `--no-daemon-relaunch` | Stay in-process; skip the daemon proxy. Also settable via `CUA_DRIVER_RS_MCP_NO_RELAUNCH=1`. |
+| `--socket <path>` | Override the daemon socket / pipe path used by the proxy. |
+| `--experimental-pip` | Show an always-on-top window with the latest post-action screenshot + a one-line label. macOS only today; Windows / Linux print a not-yet-implemented notice. See [PiP Preview](/cua-driver/guide/getting-started/pip-preview). |
+| `--experimental-pip-geometry <WxH[+X+Y]>` | Override the PiP window size / position. Default `480x360` in the top-right corner. |
### cua-driver serve
-Run cua-driver as a long-running daemon on a Unix domain socket.
-
-Listens on --socket (default: ~/Library/Caches/cua-driver/cua-driver.sock).
-Subsequent `cua-driver call/list-tools/describe` invocations auto-detect
-the socket and forward their requests, so the AppStateEngine's per-pid
-element_index cache survives across CLI calls.
-
-**Options:**
-
-| Name | Type | Default | Description |
-| ---- | ---- | ------- | ----------- |
-| `--socket` | String | — | Override the Unix socket path. |
-
-**Flags:**
-
-| Name | Description |
-| ---- | ----------- |
-| `--no-relaunch` | Stay in the current process instead of re-execing via `open -n -g -a CuaDriver`. |
+Run cua-driver as a long-running daemon. Subsequent `cua-driver call` / `mcp` invocations auto-detect the daemon and forward to it, so the per-pid `element_index` cache survives across CLI calls. Accepts the same `--experimental-pip` / `--experimental-pip-geometry` flags as `mcp`.
### cua-driver stop
-Ask the running cua-driver daemon to exit gracefully.
-
-**Options:**
-
-| Name | Type | Default | Description |
-| ---- | ---- | ------- | ----------- |
-| `--socket` | String | — | Override the Unix socket path. |
+Ask the running daemon to exit gracefully.
### cua-driver status
-Report whether a cua-driver daemon is running.
-
-**Options:**
-
-| Name | Type | Default | Description |
-| ---- | ---- | ------- | ----------- |
-| `--socket` | String | — | Override the Unix socket path. |
-| `--pid-file` | String | — | Override the daemon pid-file path. |
+Report whether a daemon is running, and its socket / pipe path and pid.
### cua-driver mcp-config
-Print MCP server config or a client-specific install command.
-
-**Options:**
-
-| Name | Type | Default | Description |
-| ---- | ---- | ------- | ----------- |
-| `--client` | String | — | Client to print the install command for: claude \| codex \| cursor \| antigravity \| openclaw \| opencode \| hermes \| pi. (`gemini` accepted as a legacy alias for `antigravity` — both read the same `~/.gemini/config/mcp_config.json` after Antigravity inherited the path in May 2026.) Omit for the generic JSON snippet. |
+Print the MCP server config JSON, or a client-specific snippet with `--client <name>` (`claude`, `codex`, `cursor`, `antigravity`, `openclaw`, `opencode`, `hermes`, `pi`; `gemini` is a legacy alias for `antigravity`). See [Integrations](/cua-driver/guide/getting-started/integrations).
-**Flags:**
-
-| Name | Description |
-| ---- | ----------- |
-| `--claude-code-computer-use-compat` | Print config for Claude Code's window-scoped screenshot compatibility mode registered as `cua-computer-use`. |
+```bash
+cua-driver mcp-config --client cursor
+```
## Trajectory recording
### cua-driver recording
-Control the trajectory recorder on a running cua-driver daemon.
-
-Wraps the `set_recording` and `get_recording_state` tools
-with human-friendly output. All subcommands require a
-running daemon (`cua-driver serve`) because recording
-state lives in-process and doesn't survive CLI-process
-lifetimes.
-
-#### cua-driver recording start
-
-Enable trajectory recording to the given directory.
-
-**Arguments:**
-
-| Name | Type | Required | Description |
-| ---- | ---- | -------- | ----------- |
-| `<output-dir>` | String | Yes | Directory to write turn folders into. Expands `~`; created if missing. |
-
-**Options:**
-
-| Name | Type | Default | Description |
-| ---- | ---- | ------- | ----------- |
-| `--socket` | String | — | Override the daemon Unix socket path. |
-
-**Flags:**
-
-| Name | Description |
-| ---- | ----------- |
-| `--video-experimental` | Also capture the main display to `<output-dir>/recording.mp4` via SCStream. Experimental, off by default. |
-
-#### cua-driver recording stop
-
-Disable trajectory recording.
+Control the trajectory recorder on a running daemon (recording state lives in the daemon, so these require `cua-driver serve`). Wraps the `start_recording` / `stop_recording` / `get_recording_state` tools with human-friendly output.
-**Options:**
+```bash
+cua-driver recording start ~/cua-trajectories/demo1 # video on by default
+cua-driver recording status
+cua-driver recording stop
+```
-| Name | Type | Default | Description |
-| ---- | ---- | ------- | ----------- |
-| `--socket` | String | — | Override the daemon Unix socket path. |
-
-#### cua-driver recording status
-
-Report whether recording is currently enabled.
-
-**Options:**
-
-| Name | Type | Default | Description |
-| ---- | ---- | ------- | ----------- |
-| `--socket` | String | — | Override the daemon Unix socket path. |
-
-#### cua-driver recording render
-
-Render a captured recording directory to a zoomed MP4.
-
-Reads `<input-dir>/session.json`, `<input-dir>/recording.mp4`,
-`<input-dir>/cursor.jsonl`, and every `<input-dir>/turn-*/action.json`,
-then produces a single zoom-on-click MP4 at `--output`.
-
-**Arguments:**
-
-| Name | Type | Required | Description |
-| ---- | ---- | -------- | ----------- |
-| `<input-dir>` | String | Yes | Recording directory (contains session.json + recording.mp4 + cursor.jsonl + turn-*/). |
-
-**Options:**
-
-| Name | Type | Default | Description |
-| ---- | ---- | ------- | ----------- |
-| `--output` | String | — | Destination path for the rendered MP4. Overwrites any existing file. |
-| `--scale` | Double | 2.0 | Zoom factor applied to each click event. 1.0 disables zoom; 2.0 is 2× magnification. |
-
-**Flags:**
-
-| Name | Description |
-| ---- | ----------- |
-| `--no-zoom` | Skip the zoom curve and re-encode the input as-is. Useful as a baseline check. |
+Video capture is on by default (macOS uses ScreenCaptureKit; Windows / Linux use an ffmpeg subprocess). See the [`start_recording` tool](/cua-driver/reference/mcp-tools#start_recording) for the per-turn folder layout and the `record_video` opt-out.
## Configuration
### cua-driver config
-Read and write persistent cua-driver settings.
-
-Persistent config lives at
-`~/Library/Application Support/<app-name>/config.json` and
-survives daemon restarts.
-
-Examples:
- cua-driver config # print full config
- cua-driver config get agent_cursor.enabled
- cua-driver config set agent_cursor.enabled false
- cua-driver config set agent_cursor.motion.arc_size 0.4
- cua-driver config reset # overwrite with defaults
-
-#### cua-driver config show
-
-Print the full current config as JSON.
-
-#### cua-driver config get
-
-Print a single config value by key.
-
-**Arguments:**
-
-| Name | Type | Required | Description |
-| ---- | ---- | -------- | ----------- |
-| `<key>` | String | Yes | Config key to read (e.g. `schema_version`, `agent_cursor.enabled`). |
-
-#### cua-driver config set
-
-Write a single config value by dotted snake_case key.
-
-**Arguments:**
+Read and write the persistent driver config. Values take effect immediately and survive daemon restarts.
-| Name | Type | Required | Description |
-| ---- | ---- | -------- | ----------- |
-| `<key>` | String | Yes | Dotted snake_case path, e.g. `agent_cursor.enabled` or `agent_cursor.motion.arc_size`. |
-| `<value>` | String | Yes | New value. JSON type depends on the key. |
+```bash
+cua-driver config get capture_mode
+cua-driver config set capture_mode som
+cua-driver config set update_check_enabled false
+```
-**Options:**
+Known keys: `capture_mode` (`vision` | `ax` | `som`), `max_image_dimension` (integer), `update_check_enabled` (boolean), `experimental_pip` (boolean), `experimental_pip_geometry` (string). Telemetry can also be toggled per-environment with `CUA_DRIVER_RS_TELEMETRY_ENABLED=0`.
-| Name | Type | Default | Description |
-| ---- | ---- | ------- | ----------- |
-| `--socket` | String | — | Override the daemon Unix socket path. |
+## Updating
-#### cua-driver config reset
+### cua-driver check-update
-Overwrite the config file with defaults.
+Ask GitHub whether a newer release is available. Read-only — never installs. Human-friendly text by default.
-#### cua-driver config telemetry
-
-Manage anonymous telemetry settings.
-
-##### cua-driver config telemetry status
-
-Show whether anonymous telemetry is enabled.
-
-##### cua-driver config telemetry enable
-
-Enable anonymous telemetry.
-
-##### cua-driver config telemetry disable
-
-Disable anonymous telemetry.
-
-#### cua-driver config updates
-
-Manage automatic update settings.
-
-##### cua-driver config updates status
-
-Show whether automatic updates are enabled.
+| Name | Description |
+| ---- | ----------- |
+| `--json` | Emit a machine-readable payload (same shape as the `check_for_update` MCP tool). |
+| `--no-cache` | Skip the 20-hour on-disk cache and force a fresh GitHub round-trip. |
-##### cua-driver config updates enable
+### cua-driver update
-Enable automatic updates.
+The same check as `check-update`, then suggests `--apply` when outdated.
-##### cua-driver config updates disable
+| Name | Description |
+| ---- | ----------- |
+| `--apply` | Download and install the latest release via the canonical installer. |
+| `--json` | Emit the structured check payload (does not change `--apply` behavior). |
-Disable automatic updates.
+See [Updating](/cua-driver/guide/getting-started/updating) for the full workflow.
## Diagnostics
-### cua-driver update
-
-Check for a newer cua-driver release and apply it.
+### cua-driver doctor
-**Flags:**
-
-| Name | Description |
-| ---- | ----------- |
-| `--apply` | Download and apply the update without prompting. |
+Run a structured environment + install-health report — version, install layout, telemetry, and platform-specific probes (TCC + LaunchAgent on macOS; UI Automation + interactive desktop session on Windows; X11/Wayland + AT-SPI on Linux). Pass `--json` for a scripting-friendly shape; the exit code is non-zero only when a probe reports an error.
### cua-driver diagnose
-Print a paste-able bundle-path / cdhash / TCC-status report for support.
-
-### cua-driver doctor
+Print a paste-able support report (bundle path / cdhash / TCC status on macOS; equivalent identity + permission details elsewhere) for attaching to issues.
-Clean up stale install bits left from older cua-driver versions.
+## Autostart
-## Other commands
+### cua-driver autostart
-### cua-driver dump-docs
+Keep a `cua-driver serve` daemon running in the user's interactive session across logons (Windows-only today; macOS / Linux point at the manual recipe). See [Autostart](/cua-driver/guide/getting-started/autostart).
-Output CLI and MCP tool documentation as JSON for tooling and integrations.
+| Subcommand | Description |
+| ---- | ----------- |
+| `enable` | Register a logon Scheduled Task so `serve` starts at every interactive logon. |
+| `disable` | Remove the autostart entry. No-op if not registered. |
+| `status` | Print whether the entry is registered and whether the daemon is running. |
+| `kick` | Start the entry now without re-logging. |
-**Options:**
+## Skills
-| Name | Type | Default | Description |
-| ---- | ---- | ------- | ----------- |
-| `--type` | String | all | Documentation type: cli, mcp, or all |
+### cua-driver skills
-**Flags:**
+Manage the agent skill pack — symlink the versioned SKILL.md into each detected agent's skills directory (Claude Code, Codex, OpenClaw, OpenCode).
-| Name | Description |
+| Subcommand | Description |
| ---- | ----------- |
-| `--pretty` | Pretty-print JSON output |
+| `install` | Fetch the skill pack from GitHub Releases and symlink it into each agent. Idempotent; never overwrites existing user links. Add `--from main` to fetch from the main branch instead of the tagged release. |
+| `update` | Re-fetch the skill pack, refreshing the local copy and links. |
+| `uninstall` | Remove the agent symlinks. Add `--all` to also delete the local copy. |
+| `status` | Report local install state and per-agent link state. Read-only. |
+| `path` | Print where the local skill pack lives. |
## Global options
Available on all commands:
-- `--help` — Show help information.
-- `--version` — Show version number.
+- `--help` — Show help information (the authoritative subcommand + flag list).
+- `--version` — Show the version number.
diff --git a/docs/content/docs/cua-driver/reference/limits.mdx b/docs/content/docs/cua-driver/reference/limits.mdx
index 55845dd4e9..ff49a65d56 100644
--- a/docs/content/docs/cua-driver/reference/limits.mdx
+++ b/docs/content/docs/cua-driver/reference/limits.mdx
@@ -1,6 +1,6 @@
---
title: Known limits
-description: What cua-driver can't do in v0.1, and the workarounds
+description: What cua-driver can't do, and the workarounds
---
import { Callout } from 'fumadocs-ui/components/callout';
@@ -63,7 +63,7 @@ If you're automating Blender or a game, the no-foreground-steal contract does no
3. Un-minimize the window (`hotkey({pid, keys: ["cmd", "m"]})` or click the Dock icon). Breaks the background contract for that window.
<Callout type="info">
-`set_value` is the right answer 90% of the time. It sidesteps both the minimized-focus issue and the general "which event commits this field" ambiguity.
+`set_value` is usually the right answer. It sidesteps both the minimized-focus issue and the general "which event commits this field" ambiguity.
</Callout>
## Permission boundaries
diff --git a/docs/content/docs/cua-driver/reference/mcp-tools.mdx b/docs/content/docs/cua-driver/reference/mcp-tools.mdx
index 4f4bf10d72..6b797b36cc 100644
--- a/docs/content/docs/cua-driver/reference/mcp-tools.mdx
+++ b/docs/content/docs/cua-driver/reference/mcp-tools.mdx
@@ -4,1197 +4,543 @@ description: Reference for every MCP tool cua-driver exposes
---
{/*
- AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY
- Generated by: npx tsx scripts/docs-generators/cua-driver.ts
- Source: recursive Swift sources under libs/cua-driver/Sources
- Version: 0.2.0
+ Synced to the cua-driver-rs 0.3.2 binary (cross-platform).
+ Regenerate with: cua-driver list-tools + cua-driver describe <tool>
+ Tool descriptions are emitted by the running binary, so they reflect the
+ platform you introspected on (this file was synced from the Windows build).
+ Where the macOS Swift driver differs, it is called out inline.
*/}
import { Callout } from 'fumadocs-ui/components/callout';
-`cua-driver` exposes 29 MCP tools through a single stdio server (`cua-driver mcp`). Every tool is also callable from the shell as `cua-driver <name> '<JSON-args>'`.
+`cua-driver` exposes 34 MCP tools through a single stdio server (`cua-driver mcp`). Every tool is also callable from the shell as `cua-driver call <name> '<JSON-args>'`.
-Tool names are `snake_case`. Responses are MCP `CallTool.Result` envelopes: a text content block prefixed with a `✅` summary (or the error reason on failure), plus optional image or structured-content blocks on tools that produce them. See the [CLI reference](/cua-driver/reference/cli-reference) for CLI-specific flags like `--raw` and `--screenshot-out-file`.
+Tool names are `snake_case`. Responses are MCP `CallTool.Result` envelopes: a text block prefixed with a `✅` summary (or the error reason on failure), plus optional image or structured-content blocks on tools that produce them. See the [CLI reference](/cua-driver/reference/cli-reference) for shell flags like `--raw` and `--screenshot-out-file`.
<Callout type="info">
- Tool names here match the CLI form exactly. `cua-driver list_apps` and the MCP `list_apps` tool run the same code path.
+ Tool names match the CLI form exactly. `cua-driver call list_apps` and the MCP `list_apps` tool run the same code path.
</Callout>
<Callout type="info">
- **TCC auto-delegation.** When an MCP client spawns `cua-driver mcp` from an IDE terminal (Claude Code, Cursor, VS Code, Warp), macOS attributes the subprocess to the parent terminal — not `CuaDriver.app` — so AX probes fail against the wrong bundle id. `mcp` detects this and auto-launches a `cua-driver serve` daemon via `open -n -g -a CuaDriver --args serve`, then proxies every tool call through the daemon's Unix socket. Tool semantics are identical to the in-process path; no Python bridge is needed. Pass `--no-daemon-relaunch` (or set `CUA_DRIVER_MCP_NO_RELAUNCH=1`) to force in-process execution. See the [process model guide](/cua-driver/guide/getting-started/process-model) for the full lifecycle, failure modes, and wrapper-author guidance.
+ **Two backends, one tool surface.** macOS ships the Swift driver by default (Accessibility / `CGEvent` / ScreenCaptureKit); Windows and Linux ship `cua-driver-rs` (UI Automation / `PostMessage` / ffmpeg). The tool names, arguments, and response shapes are the same on both. Descriptions below are synced from the Rust port and note the macOS-Swift behavior where it diverges. A few tools (`bring_to_front`, `kill_app`, `debug_window_info`, `get_accessibility_tree`) are Windows-first; the macOS Swift driver returns a pointer to the platform-native alternative.
</Callout>
-### check_permissions
-
-Report TCC permission status for Accessibility and Screen Recording.
-By default also raises the system permission dialogs for any missing
-grants — Apple's request APIs are no-ops when the grant is already
-active, so this is safe to call repeatedly. Pass &#123;"prompt": false&#125;
-for a purely read-only status check.
-
-**Arguments:**
-
-- `prompt` (boolean, optional): Raise the system permission prompts for missing grants. Default true.
-
-### click
-
-Left-click against a target pid. Two addressing modes:
-
-- `element_index` + `window_id` (from the last
- `get_window_state` snapshot of that window) —
- performs an AX action on the cached element. Default
- `action` is `press` (left-click). Other values:
- `show_menu` (right-click equivalent), `pick` (open a
- menu-bar item's submenu), `confirm` (return on a
- default button), `cancel` (escape on a dismiss
- button), `open` (open a file/folder in a browser).
- Pure AX RPC, works on backgrounded / hidden windows,
- no cursor move or focus steal. Requires a prior
- `get_window_state(pid, window_id)` in this turn; the
- element_index cache is scoped per (pid, window_id) so
- indices from one window do not resolve against another.
-
-- `x`, `y` (window-local screenshot pixels, top-left
- origin of the PNG returned by `get_window_state`) —
- synthesizes mouse events via CGEvent / SkyLight and
- delivers them to the pid. AX is never consulted on
- this path. The driver converts image-pixel →
- screen-point internally using the target's window
- origin and backing scale. `count: 2` posts two
- down/up pairs ~80ms apart for a double-click.
- `modifier` holds cmd/shift/option/ctrl during the
- click (e.g. ["cmd"] for cmd-click).
-
-Exactly one of `element_index` or (`x` AND `y`) must be
-provided. `pid` is required in both modes. `window_id`
-is required when `element_index` is used (scopes the
-cache lookup). `action` is only valid with
-`element_index`; `count` and `modifier` are ignored in
-the AX path.
-
-**Windows note:** the `x, y` path layers in a control-type
-whitelist for the UIA Invoke pre-check (canvases / images
-/ custom surfaces fall through to PostMessage to preserve
-pixel precision) and routes through `SendInput` with a
-brief foreground swap when the target HWND is a Chromium
-frame. See [Windows behavior notes](#windows-behavior-notes)
-below.
-
-**Arguments:**
-
-- `action` (string, optional): AX action name (element_index path only). Default: press.
-- `count` (integer, optional): Click count — 1 (single), 2 (double), 3 (triple). Pixel path only. Default: 1.
-- `debug_image_out` (string, optional): Optional absolute path. When set on a pixel-addressed click (x, y present), the tool captures a fresh screenshot of the target window at the current max_image_dimension, draws a red crosshair at the received (x, y), and writes the PNG to this path. Use this to verify the tool received the coordinate you intended — if you drew your own blue crosshair on the PNG from get_window_state and saved it separately, the two crosshairs must land on the same pixel. Mismatch surfaces a coord-space bug. Pixel path only; requires window_id; incompatible with from_zoom.
-- `element_index` (integer, optional): Element index from the last get_window_state for the same (pid, window_id). Routes through the AX action path. Requires window_id.
-- `from_zoom` (boolean, optional): When true, x and y are pixel coordinates in the last `zoom` image for this pid. The driver automatically maps them back to the correct window coordinates. Use this after calling `zoom` to click on something you saw in the zoomed image.
-- `modifier` (array of string, optional): Modifier keys held during the click: cmd/shift/option/ctrl. Pixel path only.
-- `pid` (integer, required): Target process ID.
-- `window_id` (integer, optional): CGWindowID for the window whose get_window_state produced the element_index. Required when element_index is used; ignored in the pixel path.
-- `x` (number, optional): X in window-local screenshot pixels — same space as the PNG get_window_state returns. Top-left origin of the target's window. Must be provided together with y.
-- `y` (number, optional): Y in window-local screenshot pixels — same space as the PNG get_window_state returns. Top-left origin of the target's window. Must be provided together with x.
+<Callout type="info">
+ **Daemon proxy.** When an MCP client spawns `cua-driver mcp` from an IDE terminal, the call may need to run under a different process identity than the spawning shell — on macOS to inherit `CuaDriver.app`'s TCC grants, on Windows to reach the interactive desktop from a Session 0 SSH login. `mcp` handles this by proxying to a running `cua-driver serve` daemon; tool semantics are unchanged. See the [process model](/cua-driver/guide/getting-started/process-model) (macOS) and [SSH on Windows](/cua-driver/guide/getting-started/windows-ssh) guides.
+</Callout>
-```json
-{"pid":844}
-```
+## Dispatch modes
-### double_click
+Every pointer and keyboard tool on the Rust port takes an optional `dispatch` field controlling how the event is delivered:
-Double-click against a target pid. Two addressing modes:
-
-- `element_index` + `window_id` (from the last
- `get_window_state` snapshot of that window) —
- performs `AXOpen` when the element advertises it
- (Finder items, list rows, openable cells); otherwise
- falls back to a stamped pixel double-click at the
- element's on-screen center. Pure AX RPC in the
- AXOpen branch; pixel recipe when no AX open action
- exists. Requires a prior
- `get_window_state(pid, window_id)` in this turn.
-
-- `x`, `y` (window-local screenshot pixels, top-left
- origin of the PNG returned by `get_window_state`) —
- synthesizes a stamped double-click (two down/up
- pairs with clickState 1→2) and delivers to the pid.
- The primer-gated auth-signed recipe lands on
- backgrounded Chromium web content (dblclick events
- that trigger YouTube fullscreen toggle, Finder
- open-on-double-click, etc.). `modifier` holds
- cmd/shift/option/ctrl during the gesture.
-
-Exactly one of `element_index` or (`x` AND `y`) must be
-provided. `pid` is required in both modes. `window_id`
-is required when `element_index` is used. `modifier`
-only takes effect in the pixel path (AXOpen doesn't
-propagate modifier keys).
+- **`background`** (default) — never swaps the foreground window. Element-indexed actions and UIA-hit-tested pixel clicks deliver through the accessibility channel with no focus steal or visible flash. If a target's input stack would silently drop a backgrounded event (Chromium DOM content, GTK button widgets), the tool returns a structured `background_unavailable` error rather than failing silently.
+- **`foreground`** — accepts a brief `SetForegroundWindow` swap (the `SendInput` path). Required to drive Chromium-content or GTK-widget targets reliably. Pair with [`bring_to_front`](#bring_to_front) to pay the foreground cost once for a sequence of calls instead of per call.
+- **`auto`** — the driver's internal heuristics, with silent fallback to `SendInput` on known-problematic targets.
-**Arguments:**
+The macOS Swift driver keeps everything backgrounded by default and exposes the same field for parity.
-- `element_index` (integer, optional): Element index from the last get_window_state for the same (pid, window_id). Routes through AXOpen when advertised, else pixel double-click at the element's center. Requires window_id.
-- `modifier` (array of string, optional): Modifier keys held during the double-click: cmd/shift/option/ctrl. Pixel path only.
-- `pid` (integer, required): Target process ID.
-- `window_id` (integer, optional): CGWindowID for the window whose get_window_state produced the element_index. Required when element_index is used; ignored in the pixel path.
-- `x` (number, optional): X in window-local screenshot pixels — same space as the PNG get_window_state returns. Top-left origin of the target's window. Must be provided together with y.
-- `y` (number, optional): Y in window-local screenshot pixels — same space as the PNG get_window_state returns. Top-left origin of the target's window. Must be provided together with x.
+## Permissions and discovery
-```json
-{"pid":844}
-```
+### check_permissions
-### drag
+Check required permissions. On Windows this reports UI Automation availability, PostMessage access, and the daemon's integrity level; on macOS it reports the Accessibility and Screen Recording TCC grants and raises the system prompts for any that are missing.
-Press-drag-release gesture from (`from_x`, `from_y`) to
-(`to_x`, `to_y`) in window-local screenshot pixels —
-the same space the PNG `get_window_state` returns.
-Top-left origin of the target's window.
-
-Use this for: marquee/lasso selection, drag-and-drop
-between source and destination, resizing via a handle,
-scrubbing a slider, repositioning a panel. macOS AX
-has no semantic drag action, so drag is pixel-only —
-there is no element-indexed mode. Address an element
-with `get_window_state`, read its `bounds`, and pass
-the pixel coordinates you want.
-
-`duration_ms` (default 500) is the wall-clock budget
-for the path between mouse-down and mouse-up; `steps`
-(default 20) is the number of intermediate `mouseDragged`
-events linearly interpolated along the path. Increase
-both for slower, more "human" drags; decrease for snap
-gestures. `modifier` keys (cmd/shift/option/ctrl) are
-held across the entire gesture — option-drag duplicates,
-shift-drag constrains the axis on most surfaces.
-
-Frontmost target: posts via `.cghidEventTap`. The real
-cursor visibly traces the drag path — unavoidable for
-AppKit-style drag sources, which accept only HID-origin
-events.
-
-Backgrounded target: posts via the auth-signed pid-routed
-path. Cursor-neutral, but some surfaces (OpenGL canvases,
-Blender, Unity) filter pid-routed dragged events at the
-event-source level — those targets must be frontmost.
-
-When `from_zoom` is true, both endpoints are pixel
-coordinates in the last `zoom` image for this pid; the
-driver maps them back to window coordinates before
-dispatching.
+**Arguments:** none.
-**Arguments:**
+### get_screen_size
-- `button` (string, optional): Mouse button used for the drag. Default: left.
-- `duration_ms` (integer, optional): Wall-clock duration of the drag path between mouseDown and mouseUp. Default: 500.
-- `from_x` (number, required): Drag-start X in window-local screenshot pixels. Top-left origin.
-- `from_y` (number, required): Drag-start Y in window-local screenshot pixels. Top-left origin.
-- `from_zoom` (boolean, optional): When true, from_x/from_y/to_x/to_y are pixel coordinates in the last `zoom` image for this pid. The driver maps them back to window coordinates before dispatching.
-- `modifier` (array of string, optional): Modifier keys held across the entire gesture: cmd/shift/option/ctrl.
-- `pid` (integer, required): Target process ID.
-- `steps` (integer, optional): Number of intermediate mouseDragged events linearly interpolated along the path. Default: 20.
-- `to_x` (number, required): Drag-end X in window-local screenshot pixels.
-- `to_y` (number, required): Drag-end Y in window-local screenshot pixels.
-- `window_id` (integer, optional): CGWindowID for the window the pixel coordinates were measured against. Optional — when omitted the driver picks the frontmost window of `pid`. Pass when the target has multiple windows and you measured against a specific one.
+Return the logical size of the main display in points plus its backing scale factor. Agents click in points; Retina displays report `scale_factor` 2.0. Requires no special permissions.
-```json
-{"from_x":100,"from_y":200,"pid":844,"to_x":100,"to_y":200}
-```
+**Arguments:** none.
-### get_agent_cursor_state
+### get_cursor_position
-Report the current agent-cursor configuration: enabled flag,
-motion knobs (startHandle, endHandle, arcSize, arcFlow,
-spring), glide duration, post-click dwell, and idle-hide
-delay. Durations come back in milliseconds to match the
-setter's units. Pure read-only — no side effects.
+Return the current mouse cursor position in screen points (origin top-left).
**Arguments:** none.
-### get_config
+### list_apps
-Report the current persistent driver config. Config lives
-at `~/Library/Application Support/<app-name>/config.json`
-and survives daemon restarts, unlike session-scoped state
-(recording).
-
-Pure read-only. Returns defaults when the file doesn't
-exist or fails to decode — same fallback the daemon uses
-at startup. Sibling to `set_config` / `cua-driver config`.
-
-Current schema:
-
- &#123;
- "schema_version": 1,
- "capture_mode": "vision" | "ax" | "som",
- "agent_cursor": &#123;
- "enabled": true,
- "motion": &#123;
- "start_handle": 0.3, "end_handle": 0.3,
- "arc_size": 0.25, "arc_flow": 0.0,
- "spring": 0.72
- &#125;
- &#125;
- &#125;
+List apps — both currently running and installed-but-not-running — with per-app state flags: `running`, `active` (system-frontmost), `kind` (`"desktop"` or `"uwp"`), `launch_path` (what `launch_app` consumes), and `last_used`. Use it for "is X installed?" as well as "is X running?". For per-window state, call `list_windows` instead. Opening an app does not require this first — call `launch_app` directly.
**Arguments:** none.
-### get_cursor_position
+### list_windows
-Return the current mouse cursor position in screen points (origin top-left).
+List every top-level window known to the window manager. Each record self-contains its owning app identity, so callers never join back against `list_apps`. Per-record fields: `window_id` (HWND on Windows, CGWindowID on macOS), `pid` + `app_name`, `title`, `bounds`, `layer`, `z_index`, `is_on_screen`. The macOS-only `on_current_space` / `space_ids` fields are omitted on Windows.
-**Arguments:** none.
+**Arguments:**
-### get_recording_state
+- `pid` (integer, optional): Restrict the result to one pid's windows.
+- `on_screen_only` (boolean, optional): Drop windows that aren't currently on-screen. Default false.
-Report the current trajectory recorder state: whether
-recording is enabled, the output directory (when
-enabled), and the 1-based counter for the next turn
-folder that will be written. Counter increments on every
-recorded action tool call and resets to 1 each time
-recording is (re-)enabled.
+### get_accessibility_tree
-Pure read-only. Typical use: `cua-driver recording
-status` surfaces these fields for human consumption;
-programmatic callers use it to confirm recording is
-armed before kicking off a session.
+Return a lightweight snapshot of the desktop: running processes and on-screen visible windows with their bounds and owner pid. This is a fast discovery read — for the full UIA subtree of a single window with clickable element indices, use [`get_window_state`](#get_window_state). Windows-first.
**Arguments:** none.
-### get_screen_size
+### get_window_state
-Return the logical size of the main display in points plus its backing scale
-factor. Agents click in points; Retina displays have scale_factor 2.0.
-Requires no TCC permissions.
+Walk a running app's UIA tree (AX tree on macOS) and return a Markdown rendering of its UI, tagging every actionable element with `[element_index N]`. Pass those indices to `click`, `type_text`, `scroll`, etc. — those tools resolve the index to the cached element on the server.
-**Arguments:** none.
+INVARIANT: call `get_window_state` once per turn per `(pid, window_id)` before any element-indexed action against that window. The index map is replaced by the next snapshot of the same `(pid, window_id)`.
-### get_window_state
+The tree walked is the window's tree (HWND-scoped); the screenshot and bounds come from the same `window_id`. `window_id` must belong to `pid`, or the call returns `isError: true` — the driver never picks a window implicitly. Set `query` to a case-insensitive substring to trim `tree_markdown` to matching lines plus their ancestor chain; element indices and `element_count` are unchanged.
+
+`capture_mode` controls the response shape (default `som`):
+
+- `som` — walks the tree **and** captures a screenshot. Element-indexed clicks work out of the box; the screenshot is there for disambiguation.
+- `vision` — screenshot only; the tree walk is skipped and the element cache is not updated. Pair with pixel-addressed `click({pid, x, y})`.
+- `ax` — tree only; no screenshot (no Screen Recording cost on macOS).
-Walk a running app's AX tree and return a Markdown rendering of its
-UI, tagging every actionable element with [element_index N]. Pass
-those indices to `click`, `type_text_in`, `scroll_in`, etc. — those
-tools resolve the index to the cached AXUIElement on the server.
-
-INVARIANT: call `get_window_state` once per turn per (pid, window_id)
-before any element-indexed action against that window. The index
-map is replaced by the next snapshot of the same (pid, window_id).
-
-The AX tree walked is the pid's tree, but the screenshot and window
-bounds reported come from the specified `window_id`. This is the
-source of truth for which window the caller intends to reason about
-— the driver never picks a window implicitly. Use `list_windows` to
-enumerate candidates, or read `launch_app`'s `windows` field for
-freshly-launched apps.
-
-`window_id` MUST belong to `pid` and MUST be on the user's current
-Space; the call returns `isError: true` otherwise. The driver does
-not auto-fall-back to a different window — window selection is the
-caller's responsibility.
-
-The screenshot of the specified window is delivered as a native MCP
-image content block (not a JSON text field), so clients receive it
-as a proper image rather than raw base64 text.
-
-Set `query` to a case-insensitive substring to filter
-`tree_markdown` down to matching lines plus their ancestor
-chain (so the structural context above each match stays
-visible). The element_index values and `element_count` are
-unchanged — the filter only trims the rendered Markdown, so
-`click({pid, window_id, element_index: N})` still resolves
-against the full cached tree. When nothing matches,
-`tree_markdown` is an empty string; the caller can retry
-without the filter.
-
-Response shape is controlled by the persistent
-`capture_mode` config setting (default `som`). Each
-mode skips the half of the work it doesn't need — not
-just the half of the response. Note that `vision` names
-the capture MODE; the separate `screenshot` tool (which
-captures a raw PNG without walking AX) is unrelated and
-keeps its name:
- - `som` — walks AX tree AND captures screenshot
- (default). Element-indexed clicks work
- out of the box; screenshot is there for
- disambiguation.
- - `vision` — captures the window PNG; AX tree walk
- is skipped entirely (no Accessibility
- hit, no element_index cache update).
- `tree_markdown`, `element_count`, and
- `turn_id` omitted. Element-indexed
- clicks will fail until a non-`vision`
- snapshot runs — pair with pixel-addressed
- `click({pid, x, y})`. Previously named
- `screenshot`; the raw string
- `"screenshot"` still decodes to `vision`
- as a deprecated alias.
- - `ax` — walks AX tree; screen-capture call is
- skipped entirely (no Screen Recording
- hit). `screenshot_*` fields omitted.
-Change with `cua-driver config set capture_mode <mode>` or
-the `set_config` tool.
-
-Screenshot capture failures behave differently per mode:
-in `som`, they're non-fatal — the AX tree still ships in
-the response and the summary line carries a hint, so
-agents can keep doing element-indexed clicks against the
-same window even when the screenshot is unavailable. In
-`vision`, the screenshot IS the deliverable, so the same
-failure returns `isError: true` with an actionable hint
-(try another window, retry later, or switch to
-`capture_mode: ax`). The macOS 26.4.x SCK regression
-(SCStreamError -3801, "Could not start streaming") is
-surfaced this way. Switching to `capture_mode: ax` skips
-the capture attempt entirely on subsequent turns.
-
-Requires Accessibility and Screen Recording permissions.
+On Windows the Rust port has no standalone `screenshot` tool — the window screenshot is delivered inside `get_window_state` (`som` / `vision`). The macOS Swift driver still exposes a separate `screenshot` tool.
**Arguments:**
-- `javascript` (string, optional): Optional JavaScript to execute in the browser tab and return alongside the AX snapshot — one round-trip instead of two. Only works for Chromium-family browsers (Chrome, Brave, Edge) and Safari; requires 'Allow JavaScript from Apple Events' to be enabled first (see WEB_APPS.md). The result is appended to the response as a `## JavaScript result` section. Use for read-only queries (document.title, innerText, querySelectorAll, etc.). For mutations or side effects use the `page` tool instead.
- `pid` (integer, required): Process ID from `list_apps`.
-- `query` (string, optional): Optional case-insensitive substring. When set, `tree_markdown` only contains lines that match plus their ancestor chain; element indices and `element_count` are unchanged.
-- `screenshot_out_file` (string, optional): Optional absolute path to write the screenshot to (e.g. "/tmp/shot.jpg"). When set, the screenshot bytes are written to this file and the MCP image content block is omitted from the response — `screenshot_file_path` is returned instead of `screenshot_png_b64`. Useful for CLI callers and agents that cannot consume inline base64 without saturating their context window (e.g. OpenCode with a local Ollama model). The directory must already exist; the file is created or overwritten.
-- `window_id` (integer, required): CGWindowID of the target window. Must belong to `pid` and be on the user's current Space. Enumerate via `list_windows` or read from `launch_app`'s `windows` array.
+- `window_id` (integer, required): HWND of the target window. Must belong to `pid`. Enumerate via `list_windows` or read from `launch_app`'s `windows` array.
+- `capture_mode` (string, optional): `som` (default), `vision`, or `ax`.
+- `query` (string, optional): Case-insensitive substring filter on `tree_markdown`.
```json
{"pid":844,"window_id":10725}
```
-### hotkey
+### debug_window_info
-Press a combination of keys simultaneously — e.g.
-`["cmd", "c"]` for Copy, `["cmd", "shift", "4"]` for
-screenshot selection. The combo is posted directly to
-the target pid's event queue via `CGEvent.postToPid`;
-the target does NOT need to be frontmost.
-
-**`window_id`** (optional): when supplied, the driver
-calls `FocusWithoutRaise.activateWithoutRaise` before
-posting — making the target AppKit-active without
-raising its window. This is required for shortcuts that
-are dispatched via NSMenu key equivalents (Cmd+S,
-Cmd+N, Cmd+W, Cmd+Shift+N, …) because AppKit only
-routes menu key equivalents to the active app. Omit
-`window_id` for shortcuts handled by the renderer
-(e.g. Cmd+C in a Chromium text field). Trade-off: the
-sentinel foreground app will lose focus once.
-
-Recognized modifiers: cmd/command, shift, option/alt,
-ctrl/control, fn. Non-modifier keys use the same
-vocabulary as `press_key` (return, tab, escape,
-up/down/left/right, space, delete, home, end, pageup,
-pagedown, f1-f12, letters, digits). Order: modifiers
-first, one non-modifier last.
-
-**Windows note:** uses `SendInput` (not PostMessage) so
-the OS modifier state is correctly updated for accelerator
-dispatch. See [Windows behavior notes](#windows-behavior-notes)
-below.
+Diagnostic: dump everything cua-driver sees about a pid's top-level windows from the daemon's session perspective — window class names, owning `.exe` basename + path, and (when UI Automation succeeds) the focused element with the list of patterns it supports (`ValuePattern`, `InvokePattern`, `TextPattern`, `TogglePattern`, …). Used to design and debug input routing for XAML / UWP / WinUI3 targets. Windows-only.
**Arguments:**
-- `keys` (array of string, required): Modifier(s) and one non-modifier key, e.g. ["cmd", "c"].
-- `pid` (integer, required): Target process ID.
-- `window_id` (integer, optional): CGWindowID of the target window. When provided, the driver calls FocusWithoutRaise before posting so NSMenu key equivalents (Cmd+S, Cmd+N, …) reach the backgrounded app.
+- `pid` (integer, required): PID of the process to inspect.
```json
-{"keys":["cmd","c"],"pid":844}
+{"pid":844}
```
-### launch_app
-
-Launch a macOS app hidden — the driver never brings the
-target to the foreground, and the target's window is not
-drawn on screen. This tool is strictly a background-launch
-primitive; if a human operator wants to switch the
-foreground app, they should do so through normal macOS UI
-(Dock, Cmd-Tab, Spotlight), not through this driver.
-
-Provide either `bundle_id` (preferred — unambiguous, e.g.
-`com.apple.calculator`) or `name` (e.g. "Calculator", resolved
-against /Applications and /System/Applications). If both are
-given, bundle_id wins.
-
-Launches in the background — the app's window appears on
-screen but does not steal focus from whatever is currently
-frontmost. The AX tree is fully populated and automation
-can start immediately via `click`, `hotkey`,
-`get_window_state`, etc.
-
-Some apps (Calculator and many Electron apps) call
-`NSApp.activate(ignoringOtherApps:)` in their own
-`applicationDidFinishLaunching`, overriding
-`NSWorkspace.OpenConfiguration.activates = false`. A layer-3
-focus-steal preventer arms a short
-`NSWorkspace.didActivateApplicationNotification` watcher
-around the launch and re-activates the previously-frontmost
-app if the target self-activates. If suppression fails the
-returned summary includes a "self-activation not suppressed"
-warning.
-
-Optional `urls` are handed to the app through its
-`application(_:open:)` AppKit delegate. For Finder, passing
-a folder URL opens a Finder window rooted at that folder in
-the background. Works with any app that implements
-`application(_:open:)`; apps that ignore the delegate simply
-launch without side effects.
-
-⚠️ BROWSER WINDOW REQUIREMENT: Browsers (Safari, Chrome,
-Firefox, Arc, Brave, Edge) require at least one URL in
-`urls` — without it NSWorkspace starts the process but
-never creates a window, so subsequent `get_window_state`,
-`click`, and `screenshot` calls will fail with "no window
-found." Use `urls=["about:blank"]` for a blank window.
-Example: `{"bundle_id": "com.apple.Safari", "urls":
-["about:blank"]}`. Electron apps also follow this contract
-when their entry point depends on a URL argument.
-
-**Windows note:** when launching a Chromium-based browser,
-the driver auto-injects three anti-throttling flags so the
-hidden renderer doesn't suspend itself. See
-[Windows behavior notes](#windows-behavior-notes) below.
-
-Optional `electron_debugging_port` launches an Electron app
-with `--remote-debugging-port=<N>`, activating its Chrome
-DevTools Protocol (CDP) on that port. This gives the `page`
-tool full renderer/DOM access (`document`, `window`,
-`fetch`, etc.) via page targets rather than the restricted
-Node main-process context exposed by SIGUSR1. Use port 9222
-unless you're running multiple Electron apps simultaneously.
-Has no effect on non-Electron apps.
-
-Optional `webkit_inspector_port` launches a Tauri/WKWebView
-app with `WEBKIT_INSPECTOR_SERVER=127.0.0.1:<N>`, activating
-WebKit's remote inspector. The `page` tool will use this to
-execute JavaScript in the WKWebView renderer. Use port 9226
-(reserved WebKit range: 9226–9228). Requires
-`developerExtrasEnabled = true` in the app's WKWebView config
-(default in Tauri debug builds; production builds may disable
-it). Has no effect on non-WKWebView apps.
-
-Use this instead of shelling out to `open -a` — the shell form
-always activates the target, requires an extra permission
-prompt for Bash, and doesn't even try to respect
-background-launch intent.
-
-Returns the launched app's pid, bundle_id, name, active
-flag, AND a `windows` array — same per-window shape as
-`list_windows` returns (window_id, title, bounds,
-z_index, is_on_screen, on_current_space, space_ids).
-That lets callers skip an extra `list_windows` round-trip
-before `get_window_state(pid, window_id)` in the common
-case. When the launch settles but no window has
-materialized yet (transient; rare), `windows` comes back
-empty — call `list_windows(pid)` explicitly a moment
-later to resolve a target.
+## Pointer input
-**Arguments:**
-
-- `additional_arguments` (array of string, optional): Extra command-line arguments passed to the launched process. Passed directly as argv entries — no shell expansion. Example: ["--user-data-dir=/tmp/cua-session", "--no-first-run"] for an isolated Chrome session.
-- `bundle_id` (string, optional): App bundle identifier, e.g. com.apple.calculator.
-- `creates_new_application_instance` (boolean, optional): Force a brand-new process even if the app is already running. Useful for isolated browser sessions: pass creates_new_application_instance=true together with additional_arguments=["--user-data-dir=/tmp/session-a", "--no-first-run", "--no-default-browser-check"] to launch a sandboxed Chrome that cannot see the user's real profile, cookies, or extensions. Each session gets its own pid and window identity and can be controlled independently.
-- `electron_debugging_port` (integer, optional): Launch an Electron app with --remote-debugging-port=&lt;N&gt; so the `page` tool gets full renderer/DOM access. Use 9222 unless running multiple Electron apps. Ignored for non-Electron apps.
-- `name` (string, optional): App display name. Used only when bundle_id is absent.
-- `urls` (array of string, optional): Optional file:// or http(s):// URLs (or plain paths with ~ expansion) to hand to the launched app via application(_:open:). For Finder, pass a folder URL or path to open a backgrounded Finder window rooted at that folder — no activation. Apps that don't implement application(_:open:) launch normally and ignore these.
-- `webkit_inspector_port` (integer, optional): Launch a Tauri/WKWebView app with WEBKIT_INSPECTOR_SERVER=127.0.0.1:&lt;N&gt; so the `page` tool can reach its WebKit inspector. Use 9226 (reserved WebKit range: 9226–9228, distinct from Electron's 9222–9225). Requires developerExtrasEnabled=true in the WKWebView config (default in Tauri debug builds).
+### click
-### list_apps
+Left-click against a target pid. **Prefer `element_index` over pixel coordinates** — it works on backgrounded / minimized / hidden / off-desktop windows, surfaces a stable handle, and tells you what you're clicking via the cached element's role + label. Reach for `x, y` only on a canvas / video / WebGL / custom-drawn surface that doesn't appear in the UIA tree.
-List every app — currently running **and** installed-but-not-running —
-in a single flat array. Each entry self-describes which state it's in.
-
-Per-entry fields (same shape on macOS, Windows, Linux; platform-specific
-values are explicitly `null` rather than omitted):
-
-- `pid` (integer): the live process id when `running=true`, else `0`.
-- `name` (string): display name.
-- `bundle_id` (string|null): platform-stable identifier. macOS = bundle id
- (`com.apple.calculator`), Windows desktop = the resolved `.exe` path,
- Windows UWP = `{PackageFamilyName}` (e.g. `Microsoft.WindowsCalculator_8wekyb3d8bbwe`),
- Linux = the XDG "desktop file id" — the `.desktop` file's path relative
- to its `applications/` root with the `.desktop` suffix stripped and any
- path separators replaced with `-` (e.g. `kde4/konqbrowser.desktop` →
- `kde4-konqbrowser`).
-- `running` (bool): true when a process matching this app is live.
-- `active` (bool): true for the single system-frontmost app (always
- implies running). Always false on Linux — X11/Wayland focus is
- per-window, not per-app.
-- `kind` (string|null): `"desktop"` for `.app`/`.exe`/`.desktop`
- launchers, `"uwp"` for Windows packaged apps.
-- `launch_path` (string|null): what `launch_app` would consume to start
- this app cold. macOS = `.app` bundle path; Windows desktop = full
- `.exe` commandline (path plus any arguments preserved from the source
- `.lnk`); Windows UWP = `shell:appsFolder\{family}!{appId}` where
- `{appId}` may fall back to `App` when the package manifest does not
- expose a specific `Application.Id`; Linux = the command from `Exec=`
- with XDG field codes stripped.
-- `last_used` (string|null): RFC3339 timestamp from the launcher's
- filesystem mtime, when readable; otherwise `null`.
-- `windows` (array): reserved — empty by design to keep the call cheap.
- Use `list_windows` for per-window state.
-
-Enumeration sources per platform:
-
-- macOS: NSWorkspace running list + filesystem scan of `/Applications`,
- `/Applications/Utilities`, `/System/Applications`,
- `/System/Applications/Utilities`, `~/Applications`. Only apps with
- `NSApplicationActivationPolicyRegular` are included — background
- helpers and system UI agents are filtered out.
-- Windows: visible top-level windows (running set) + Start-Menu
- `.lnk` resolution via `IShellLinkW::GetPath` (desktop installed set)
- + WinRT `PackageManager::FindPackagesWithPackageTypes(Main)`
- (UWP installed set).
-- Linux: `/proc` walk (running set) + XDG `.desktop` files in
- `$XDG_DATA_HOME/applications` and each `$XDG_DATA_DIRS` entry's
- `applications/` subdir (installed set). `NoDisplay=true` and
- `Hidden=true` entries are filtered.
-
-Use this for "is X installed?" as well as "is X running?". For
-per-window state — on-screen, on-current-Space, minimized,
-window titles — call `list_windows` instead. For just opening an
-app — running or not — call `launch_app({bundle_id: ...})` directly;
-`list_apps` is not a prerequisite.
-
-Legacy `processes` alias: alongside the unified `apps` array, the
-Linux and Windows backends also return a `processes` field containing
-just the running subset (one record per live pid with `pid` and
-`name`). It exists so callers written against the pre-unification
-shape (running processes only) keep working without changes; new
-callers should read `apps` and filter on `running` instead.
+- `element_index` + `window_id` (from the last `get_window_state` of that window) — performs the UIA Invoke pattern on the cached element via PostMessage. No cursor move, no focus steal. Requires a prior `get_window_state(pid, window_id)` this turn.
+- `x`, `y` (window-local screenshot pixels) — on `dispatch:"background"` / `"auto"`, the driver first does a UIA hit-test at the resolved position: if the deepest invokable element exposes `InvokePattern`, it's invoked through the accessibility channel (flash-free, even on UWP / WinUI3 / Win11 packaged apps). The `PostMessage(WM_LBUTTONDOWN/UP)` fallback only runs when the hit-test misses, and returns `background_unavailable` if PostMessage is known to drop on that target — at which point switch to `dispatch:"foreground"`.
-**Arguments:** none.
-
-### list_windows
-
-List every layer-0 top-level window currently known to
-WindowServer — including off-screen ones (hidden-launched,
-minimized into the Dock, on another Space). Each record
-self-contains its owning app identity so the caller never
-has to join back against list_apps.
-
-Use this — not list_apps — for any window-level reasoning:
-"does this app have a visible window right now?", "which
-Space is this window on?", "which of this pid's windows
-is the main one?". list_apps is purely an app / launch
-surface.
-
-Per-record fields:
-
-- window_id: CGWindowID, addressable for screenshot /
- activation / Space-move APIs.
-- pid + app_name: owning app identity.
-- title: current window title (empty string for helpers
- and chromeless surfaces).
-- bounds: x / y / width / height, top-left origin in
- global screen points.
-- layer: CGWindow stratum. Always 0 in the default
- filter; reserved for future higher-layer opt-in.
-- z_index: stacking order on the current Space (higher =
- closer to front). Cross-Space ordering is undefined.
-- is_on_screen: WindowServer's "visible on the user's
- current Space right now" bit. False for hidden /
- minimized / off-Space windows.
-- on_current_space: true when the window is bound to the
- user's current Space. Omitted when the SkyLight Space
- SPIs didn't resolve (rare; sandboxed or very old OS).
-- space_ids: every managed Space id the window is a
- member of. Compare against current_space_id in the
- top-level response. Omitted in the same failure modes
- as on_current_space.
-
-Top-level fields:
-
-- windows: array of the per-window records above.
-- current_space_id: user's active Space id, or null when
- SPI unavailable.
-
-Inputs: pid (optional — restrict to one pid's windows),
-on_screen_only (bool, default false — surface off-Space /
-minimized windows by default). Layer 0 filtering is
-always applied; menubar strips and dock shields are
-noise for every current caller.
+Exactly one of `element_index` or (`x` AND `y`) must be provided; `pid` is required in both. After a `zoom` call, pass `from_zoom=true` to translate zoom-image coords back to full-window space.
**Arguments:**
-- `on_screen_only` (boolean, optional): When true, drop windows that aren't currently on the user's Space (minimized, hidden, off-Space). Default false.
-- `pid` (integer, optional): Optional pid filter. When set, only this pid's windows are returned.
+- `pid` (integer, required): Target process ID.
+- `element_index` (integer, optional): Index from the last `get_window_state` for the same `(pid, window_id)`.
+- `window_id` (integer, optional): HWND for the window that produced the `element_index`. Required when `element_index` is used.
+- `x` (number, optional): X in window-local screenshot pixels. Provide together with `y`.
+- `y` (number, optional): Y in window-local screenshot pixels. Provide together with `x`.
+- `button` (string, optional): `left` (default), `right`, or `middle`. Windows-only convenience; macOS exposes right-click as a separate `right_click` tool.
+- `count` (integer, optional): 1 (single), 2 (double), 3 (triple). Default 1.
+- `dispatch` (string, optional): `background` (default), `foreground`, or `auto`. See [Dispatch modes](#dispatch-modes).
+- `from_zoom` (boolean, optional): Coordinates are in the last `zoom` image for this pid.
-### move_cursor
+```json
+{"pid":844,"window_id":10725,"element_index":14}
+```
+
+### double_click
-Instantly move the mouse cursor to (x, y) in screen points.
-Uses CGWarpMouseCursorPosition — no drag, no click, no CGEvent.
+Double-click against a target pid. `element_index` + `window_id` synthesizes a stamped pixel double-click at the element's cached on-screen center; `x`, `y` posts two `WM_LBUTTONDOWN/UP` pairs to the deepest child window at that point. Exactly one addressing mode; `pid` required in both. The `modifier` field is accepted for macOS parity (no-op on Windows — PostMessage doesn't propagate modifier-key state).
**Arguments:**
-- `x` (integer, required): X in screen points.
-- `y` (integer, required): Y in screen points.
+- `pid` (integer, required): Target process ID.
+- `element_index` (integer, optional): Index from the last `get_window_state`. Requires `window_id`.
+- `window_id` (integer, optional): HWND for the target window.
+- `x` / `y` (number, optional): Window-local screenshot pixels; provide together.
+- `dispatch` (string, optional): See [Dispatch modes](#dispatch-modes).
+- `from_zoom` (boolean, optional): Coordinates are in the last `zoom` image for this pid.
+- `modifier` (array of string, optional): Accepted for parity; no-op on Windows.
```json
-{"x":100,"y":200}
+{"pid":844,"window_id":10725,"element_index":14}
```
-### page
+### right_click
-Browser page primitives — execute JavaScript, extract page text, or query DOM elements via CSS selector.
+Right-click against a target pid. `element_index` + `window_id` posts `WM_RBUTTONDOWN/UP` at the element's cached on-screen center; `x`, `y` posts to the deepest child window at that point. Exactly one addressing mode; `pid` required in both. `modifier` is accepted for macOS parity (no-op on Windows).
-Three actions:
+<Callout type="info">
+ Right-click on Chromium **web content** via pixel coordinates is coerced to a left-click by the renderer on every synthetic path. Use `element_index` on AX/UIA-addressable targets (links, buttons, toolbar items). See [Limits](/cua-driver/reference/limits).
+</Callout>
-• `execute_javascript` — run arbitrary JS in the active tab and return the result. Wrap in an IIFE with try-catch for safety. Don't use this for elements already indexed by `get_window_state` — prefer `click` / `set_value` there.
+**Arguments:**
-• `get_text` — returns `document.body.innerText`. Faster than walking the AX tree for plain-text content (prices, article body, table values) that the AX tree drops or truncates.
+- `pid` (integer, required): Target process ID.
+- `element_index` (integer, optional): Index from the last `get_window_state`. Requires `window_id`.
+- `window_id` (integer, optional): HWND for the target window.
+- `x` / `y` (number, optional): Window-local screenshot pixels; provide together.
+- `dispatch` (string, optional): See [Dispatch modes](#dispatch-modes).
+- `from_zoom` (boolean, optional): Coordinates are in the last `zoom` image for this pid.
+- `modifier` (array of string, optional): Accepted for parity; no-op on Windows.
-• `query_dom` — runs `querySelectorAll(css_selector)` and returns each matching element's tag, innerText, and any requested attributes as a JSON array. Useful for structured data (table rows, link hrefs, data-* attributes) that the AX tree flattens.
+```json
+{"pid":844,"window_id":10725,"element_index":14}
+```
-Supported browsers: Chrome (com.google.Chrome), Brave (com.brave.Browser), Edge (com.microsoft.edgemac), Safari (com.apple.Safari). Requires "Allow JavaScript from Apple Events" enabled — see WEB_APPS.md.
+### drag
-`pid` and `window_id` identify the target window. The browser does NOT need to be frontmost for read actions. For mutations that trigger UI changes (form submits, navigation) the browser should be frontmost so the results are visible.
+Press-drag-release gesture from (`from_x`, `from_y`) to (`to_x`, `to_y`) in window-local screenshot pixels. Use it for marquee selection, drag-and-drop, resizing via a handle, or scrubbing a slider. `duration_ms` (default 500) is the wall-clock budget; `steps` (default 20) interpolates intermediate `WM_MOUSEMOVE` events along the path. No focus steal. After a `zoom`, pass `from_zoom=true`.
**Arguments:**
-- `action` (string, required): Which page primitive to run.
-
-• execute_javascript / get_text / query_dom — see above.
-• enable_javascript_apple_events — enable 'Allow JavaScript from Apple Events' in Chrome/Brave/Edge. Quits the browser, patches each profile's Preferences JSON, then relaunches. Requires user_has_confirmed_enabling=true — you MUST ask the user for explicit permission before calling this action.
-- `attributes` (array of string, optional): Element attributes to include per match (action=query_dom only). E.g. ["href", "src", "data-id"]. tag and innerText are always included.
-- `bundle_id` (string, optional): Browser bundle ID (action=enable_javascript_apple_events only). E.g. com.google.Chrome.
-- `css_selector` (string, optional): CSS selector (action=query_dom only). Passed to querySelectorAll — standard CSS syntax.
-- `javascript` (string, optional): JS to execute (action=execute_javascript only). Should return a serialisable value. Wrap in IIFE: `(() => { … })()`.
-- `pid` (integer, required): Browser process ID.
-- `user_has_confirmed_enabling` (boolean, optional): Must be true (action=enable_javascript_apple_events only). Set this only after the user has explicitly said yes to enabling JavaScript from Apple Events.
-- `window_id` (integer, required): CGWindowID of the target browser window.
+- `pid` (integer, required): Target process ID.
+- `from_x` (number, required): Drag-start X in window-local screenshot pixels.
+- `from_y` (number, required): Drag-start Y.
+- `to_x` (number, required): Drag-end X.
+- `to_y` (number, required): Drag-end Y.
+- `button` (string, optional): `left` (default), `right`, `middle`.
+- `duration_ms` (integer, optional): Wall-clock duration of the path. Default 500.
+- `steps` (integer, optional): Intermediate move events. Default 20.
+- `dispatch` (string, optional): See [Dispatch modes](#dispatch-modes).
+- `from_zoom` (boolean, optional): Coordinates are in the last `zoom` image for this pid.
+- `modifier` (array of string, optional): Modifier keys held across the gesture.
+- `window_id` (integer, optional): HWND the coordinates were measured against. Driver picks the frontmost window of `pid` when omitted.
```json
-{"action":"get_text","pid":844,"window_id":10725}
+{"pid":844,"from_x":120,"from_y":240,"to_x":420,"to_y":520}
```
-### press_key
+### scroll
-Press and release a single key, delivered directly to the
-target pid's event queue via `CGEvent.postToPid`. The target
-does NOT need to be frontmost — no focus steal.
-
-Optional `element_index` + `window_id` (from the last
-`get_window_state` snapshot of that window) pre-focuses
-that element (via `AXSetAttribute(kAXFocused, true)`)
-before the key fires; useful for "type into field →
-press Return on field." Without `element_index`, the
-key is posted to the pid with whatever element is
-already focused receiving it — useful for scroll keys
-on a backgrounded web view.
-
-The key is delivered via SkyLight's `SLEventPostToPid`
-with an attached `SLSEventAuthenticationMessage`,
-which is the path the renderer's keyboard pipeline
-accepts as authentic input. That's what lets Return
-commit a Chromium omnibox against a backgrounded
-window. Falls back to the public `CGEventPostToPid`
-when the SkyLight SPI isn't available.
-
-Key vocabulary: return, tab, escape, up/down/left/right,
-space, delete, home, end, pageup, pagedown, f1-f12, plus
-any letter or digit. Optional `modifiers` array takes
-cmd/shift/option/ctrl/fn. For true combinations (cmd+c),
-`hotkey` is a cleaner surface.
+Scroll the target pid's focused region. On Windows the transport is `WM_VSCROLL` / `WM_HSCROLL` posted to the window — the same events the OS sends for scrollbar / trackpad scrolling, so any window that handles scrollbars responds. (macOS synthesizes PageDown / arrow keystrokes; both reach backgrounded windows.) `by: "page"` maps to `SB_PAGE*` × amount; `by: "line"` to `SB_LINE*` × amount.
**Arguments:**
-- `element_index` (integer, optional): Optional element_index from the last get_window_state for the same (pid, window_id). When present, the element is focused before the key fires. Requires window_id.
-- `key` (string, required): Key name (return, tab, escape, up, down, left, right, space, delete, home, end, pageup, pagedown, f1-f12, letter, digit).
-- `modifiers` (array of string, optional): Optional modifier names held while the key is pressed (cmd/shift/option/ctrl/fn).
- `pid` (integer, required): Target process ID.
-- `window_id` (integer, optional): CGWindowID for the window. Required when element_index is used. When supplied without element_index, triggers FocusWithoutRaise before the key fires — use this for NSMenu key equivalents (Cmd+S, Cmd+N, …) on backgrounded apps.
+- `direction` (string, required): `up`, `down`, `left`, or `right`.
+- `amount` (integer, optional): Number of scroll ticks (1–50). Default 3.
+- `by` (string, optional): `line` (default) or `page`.
+- `dispatch` (string, optional): See [Dispatch modes](#dispatch-modes).
+- `element_index` (integer, optional): Accepted for cross-platform parity; currently no-op on Windows (UIA SetFocus not wired up yet).
+- `window_id` (integer, optional): HWND of the target window. Required when `element_index` is used; otherwise auto-resolves the pid's first visible window.
```json
-{"key":"return","pid":844}
+{"pid":844,"direction":"down"}
```
-### replay_trajectory
+### move_cursor
-Replay a recorded trajectory by re-invoking every turn's
-tool call in lexical order. `dir` must point at a
-directory previously written by `set_recording` (or the
-`cua-driver recording start` CLI). Each `turn-NNNNN/`
-is parsed for `action.json`, and the recorded tool is
-called with its recorded `arguments` via the same
-dispatch path an MCP / CLI call uses.
-
-Caveats:
-
-- Element-indexed actions (`click({pid, element_index})`
- etc.) will fail because element indices are
- per-snapshot and don't survive across sessions. Pixel
- clicks (`click({pid, x, y})`) and all keyboard tools
- replay cleanly. Failures are reported but don't stop
- replay unless `stop_on_error` is true.
-- `get_window_state` and other read-only tools are NOT
- currently recorded, so replays do not re-populate the
- per-(pid, window_id) element cache. A recorded session
- consisting solely of element-indexed clicks will
- therefore replay as a stream of "No cached AX state"
- errors — fine for regression diffs, useless for
- re-driving an app.
-- If recording is ENABLED while replay runs, the replay
- itself is recorded into the currently configured
- output directory. That's deliberate: recording a
- replay against a new build and diffing the two
- trajectories is the regression-test workflow.
+Move the **agent cursor overlay** to (x, y). Does NOT move the real mouse cursor — this is purely a visual cue for demos and recordings.
**Arguments:**
-- `delay_ms` (integer, optional): Milliseconds to sleep between turns, for human-observable pacing. Default 500.
-- `dir` (string, required): Trajectory directory previously written by `set_recording`. Absolute or ~-rooted.
-- `stop_on_error` (boolean, optional): Stop replay on the first tool-call error. Default true — set false to best-effort through the full trajectory.
+- `x` (number, required): X in screen points.
+- `y` (number, required): Y in screen points.
+- `cursor_id` (string, optional): Multi-cursor instance id. Default `default`.
```json
-{"dir":"~/cua-trajectories/demo1"}
+{"x":100,"y":200}
```
-### right_click
+### zoom
-Right-click against a target pid. Two addressing modes:
-
-- `element_index` + `window_id` (from the last
- `get_window_state` snapshot of that window) —
- performs `AXShowMenu` on the cached element. This is the
- reliable path: pure AX RPC, works on backgrounded /
- hidden windows, no cursor move or focus steal. Requires
- a prior `get_window_state(pid, window_id)` in this turn.
-
-- `x`, `y` (window-local screenshot pixels, top-left
- origin of the PNG returned by `get_window_state`) —
- internally routed through AX (`AXShowMenu`) when the
- pixel resolves to an actionable element (same
- reliability as `element_index`); falls back to a
- synthesized right-mouse-down / right-mouse-up
- CGEvent pair posted via auth-signed
- `SLEventPostToPid` for non-AX surfaces (canvas /
- WebView / games). Driver converts image-pixel →
- screen-point internally. Experimental on Chromium
- web content in the CGEvent fallback: the event is
- observed to land as a left-click there (a known
- limitation matching third-party Computer Use
- implementations). `modifier` forces the CGEvent
- path (AX doesn't propagate modifier keys).
-
-Exactly one of `element_index` or (`x` AND `y`) must be
-provided. `pid` is required in both modes. `window_id`
-is required when `element_index` is used. `modifier`
-only takes effect in the pixel path (AX actions don't
-propagate modifier keys).
+Zoom into a rectangular region of a window screenshot at full (native) resolution. Use it when `get_window_state` returned a resized image and you need to read small text or verify UI details. Coordinates `x1, y1, x2, y2` are in the same pixel space as the `get_window_state` screenshot; the maximum region width is 500 px in scaled-image coordinates. A 20% padding is added on every side. After zooming, pass `from_zoom=true` to `click` / `type_text` to translate coordinates back to full-window space.
**Arguments:**
-- `element_index` (integer, optional): Element index from the last get_window_state for the same (pid, window_id). Routes through AXShowMenu. Requires window_id.
-- `modifier` (array of string, optional): Modifier keys held during the right-click: cmd/shift/option/ctrl. Pixel path only.
-- `pid` (integer, required): Target process ID.
-- `window_id` (integer, optional): CGWindowID for the window whose get_window_state produced the element_index. Required when element_index is used; ignored in the pixel path.
-- `x` (number, optional): X in window-local screenshot pixels — same space as the PNG get_window_state returns. Top-left origin of the target's window. Must be provided together with y.
-- `y` (number, optional): Y in window-local screenshot pixels — same space as the PNG get_window_state returns. Top-left origin of the target's window. Must be provided together with x.
+- `pid` (integer, required): Target process ID (scopes the zoom-translation registry).
+- `window_id` (integer, required): HWND of the target window.
+- `x1` (number, required): Left edge (resized-image pixels).
+- `y1` (number, required): Top edge.
+- `x2` (number, required): Right edge.
+- `y2` (number, required): Bottom edge.
```json
-{"pid":844}
+{"pid":844,"window_id":10725,"x1":120,"y1":240,"x2":520,"y2":480}
```
-### screenshot
+## Keyboard and text input
-Capture a screenshot using ScreenCaptureKit. Returns base64-encoded
-image data for a single window in the requested format (default png).
-
-
- `window_id` is required. Get window ids from `list_windows`.
-
-Requires the Screen Recording TCC grant — call `check_permissions`
-first if unsure.
+### press_key
-On macOS 26.4.x, ScreenCaptureKit can refuse specific windows on
-physical Macs (SCStreamError -3801, "Could not start streaming").
-The driver retries once and falls back to the legacy
-CGWindowList path before failing; if both refuse, the error
-response includes a hint to try a different `window_id` or
-switch to `capture_mode: ax` for `get_window_state` (the
-element-indexed flow doesn't need pixels).
+Press and release a single key, delivered to the target pid's top-level window via `PostMessage(WM_KEYDOWN/WM_KEYUP)`. The target does NOT need to be frontmost — no focus steal. Key vocabulary: return, tab, escape, up/down/left/right, space, delete, home, end, pageup, pagedown, f1-f12, plus any letter or digit. For true combinations (ctrl+c) use [`hotkey`](#hotkey).
**Arguments:**
-- `format` (string, optional): Image format. Default: png.
-- `quality` (integer, optional): JPEG quality 1-95; ignored for png.
-- `window_id` (integer, required): Required CGWindowID / kCGWindowNumber to capture.
+- `pid` (integer, required): Target process ID.
+- `key` (string, required): Key name.
+- `modifiers` (array of string, optional): Modifier names held while the key is pressed (ctrl/shift/alt/win on Windows; cmd/shift/option/ctrl/fn on macOS).
+- `dispatch` (string, optional): See [Dispatch modes](#dispatch-modes).
+- `element_index` (integer, optional): Accepted for parity; currently no-op on Windows.
+- `window_id` (integer, optional): HWND for the target window. Auto-resolves the pid's first visible window when omitted.
```json
-{"window_id":10725}
+{"pid":844,"key":"return"}
```
-### scroll
+### hotkey
+
+Press a combination of keys simultaneously — e.g. `["ctrl", "c"]` for Copy. Dispatch is target-aware on Windows:
-Scroll the target pid's focused region by synthesized
-keystrokes.
-
-Mapping: `by: "page"` → PageDown/PageUp × amount;
-`by: "line"` → DownArrow/UpArrow × amount. Horizontal
-variants use Left/Right arrow keys. Keys are posted via
-the same auth-signed `SLEventPostToPid` path as
-`press_key`, so backgrounded Chromium / WebKit windows
-scroll correctly.
-
-Why keyboard instead of wheel events: probe tests
-showed `CGEventCreateScrollWheelEvent2` posted via
-SkyLight's per-pid path is silently dropped by
-Chromium (no Scroll-specific auth subclass exists;
-the factory falls back to the bare parent class and
-renderers reject it). Keystrokes don't have that
-problem — that's why PageDown works when the wheel
-event didn't.
-
-Optional `element_index` + `window_id` (from the last
-`get_window_state` snapshot of that window) focuses the
-element (`AXFocused=true`) before the keystroke, so the
-keys land in the scrollable element rather than whatever
-previously had focus. Skip it if focus was already
-established by a prior click.
+- **Modern XAML / WinUI / UWP targets** route through UI Automation: the driver finds a descendant whose `AcceleratorKey` matches the combo and invokes it. No focus steal.
+- **Legacy Win32 targets with modifiers** (Ctrl+S, etc.) route through `SendInput` with a brief `SetForegroundWindow` swap, because `PostMessage(WM_KEYDOWN, VK_CONTROL)` doesn't update the OS-wide modifier state that `TranslateAccelerator` reads. Requires the daemon to have UIAccess integrity (the MCP proxy auto-prefers the `cua-driver-uia.exe` worker when both pipes are up).
+- **Legacy Win32 targets without modifiers** (plain `enter`, `f5`) route through `PostMessage` — no focus steal.
+
+Recognized modifiers: ctrl/control, shift, alt, win/windows (cmd/command, shift, option/alt, ctrl/control, fn on macOS). Order: modifiers first, one non-modifier last.
**Arguments:**
-- `amount` (integer, optional): Number of keystroke repetitions. Default: 3.
-- `by` (string, optional): Scroll granularity. Default: line.
-- `direction` (string, required):
-- `element_index` (integer, optional): Optional element_index from the last get_window_state for the same (pid, window_id) — focuses the element before the keystrokes fire. Requires window_id.
- `pid` (integer, required): Target process ID.
-- `window_id` (integer, optional): CGWindowID for the window whose get_window_state produced the element_index. Required when element_index is used.
+- `keys` (array of string, required): Modifier(s) and one non-modifier key, e.g. `["ctrl", "c"]`.
+- `dispatch` (string, optional): See [Dispatch modes](#dispatch-modes).
+- `window_id` (integer, optional): Explicit HWND when the pid owns multiple windows.
```json
-{"direction":"down","pid":844}
+{"pid":844,"keys":["ctrl","c"]}
```
-### set_agent_cursor_enabled
+### type_text
-Toggle the visual agent-cursor overlay. When enabled, future
-pointer actions animate a floating arrow to the target's
-on-screen position before firing the AX action — purely
-visual, the AX dispatch itself is unchanged. Disabling
-removes the overlay immediately.
+Insert text into the target pid via character-by-character `PostMessage(WM_CHAR)` to the focused window. No focus steal. Special keys (Return, Escape, arrows, Tab) go through `press_key` / `hotkey` — they are not text.
-Default: **enabled**. Stays on for the life of the MCP
-session / daemon; disable with `{"enabled": false}` for
-headless / CI runs where the visual isn't wanted. The
-overlay only renders when the driver has an AppKit run
-loop, which is bootstrapped by `mcp` and `serve` but not
-by one-shot CLI invocations — so this flag is a no-op in
-fresh-CLI mode.
+When the target is a XAML / WinUI3 / UWP host (modern Notepad, Calculator, Photos, Settings, …), the tool requires `element_index` + `window_id` and routes through UI Automation's `ValuePattern.SetValue` — the same backend as `set_value` — because `WM_CHAR` doesn't reach those hosts. Calling `type_text(pid, text)` on a XAML host without `element_index` returns an actionable error pointing you at `get_window_state` first. Legacy Win32 apps keep the PostMessage path.
**Arguments:**
-- `enabled` (boolean, required): True to show the overlay cursor; false to hide.
+- `pid` (integer, required): Target process ID.
+- `text` (string, required): Text to insert at the focused element's cursor.
+- `delay_ms` (integer, optional): Milliseconds between characters on the PostMessage path (0–200). Default 30. Ignored on the UIA path (SetValue is atomic).
+- `dispatch` (string, optional): See [Dispatch modes](#dispatch-modes).
+- `element_index` (integer, optional): Required for XAML / UWP hosts (UIA path); accepted-but-optional elsewhere.
+- `window_id` (integer, optional): HWND of the target window. Required when `element_index` is used.
```json
-{"enabled":false}
+{"pid":844,"text":"hello"}
```
-### set_agent_cursor_motion
+### set_value
-Tune the agent cursor's motion curve. All fields optional;
-only the knobs you pass change, the rest keep their
-current value. The knobs map to the cubic Bezier that the
-cursor traces between successive positions, plus the
-spring animation that settles the cursor's rotation on
-arrival.
-
-- start_handle: how far along the straight line the
- first control point sits, as a fraction in [0, 1].
- Lower = tight departure, higher = floppy departure.
-- end_handle: same, measured from the end point.
-- arc_size: perpendicular deflection as a fraction of
- path length. 0 = straight line. 0.25 = modest arc.
- 0.5 = dramatic swoop. Typical: 0.25.
-- arc_flow: asymmetry bias in [-1, 1]. Positive bulges
- the arc toward the destination; negative toward the
- origin; 0 is symmetric.
-- spring: settle damping in [0.3, 1.0]. 1.0 = critically
- damped (no overshoot); 0.4 = bouncy. Typical: 0.72.
-- glide_duration_ms: wall-clock duration of each cursor
- flight in milliseconds. Higher = slower, more
- visible for demos / screen recordings. Typical: 750.
-- dwell_after_click_ms: pause the tool holds after
- the click ripple before returning, letting the
- cursor visibly rest on the target so consecutive
- clicks read as human pacing. Only applied while
- the cursor is enabled. Typical: 400.
-- idle_hide_ms: how long the overlay lingers after
- the last pointer action before auto-hiding.
- Re-armed by every click, so a burst of consecutive
- actions keeps the cursor visible throughout. Higher
- = more tolerant of follow-up actions without the
- overlay popping in and out. Typical: 3000.
-
-Defaults: start_handle=0.3, end_handle=0.3, arc_size=0.25,
-arc_flow=0.0, spring=0.72, glide_duration_ms=750,
-dwell_after_click_ms=400, idle_hide_ms=3000.
+Set a value on a UIA element via the `ValuePattern` interface (`AXValue` on macOS). Two modes: **standard input** (text fields, sliders, combo-box edit) writes directly through `IUIAutomationValuePattern::SetValue`; **ComboBox / select dropdown** picks the option whose text matches `value`. For free-form text that the target only accepts via keystrokes, prefer `type_text` — ValuePattern writes are ignored by some web inputs.
**Arguments:**
-- `arc_flow` (number, optional): Asymmetry bias in [-1, 1]. Default 0.
-- `arc_size` (number, optional): Arc deflection as fraction of path length. Default 0.25.
-- `dwell_after_click_ms` (number, optional): Pause after the click ripple in ms, for human-like pacing. Default 400.
-- `end_handle` (number, optional): End-handle fraction in [0, 1]. Default 0.3.
-- `glide_duration_ms` (number, optional): Flight duration per click in ms. Higher = slower. Default 750.
-- `idle_hide_ms` (number, optional): How long the overlay lingers after the last click before auto-hiding, in ms. Default 3000.
-- `spring` (number, optional): Settle damping in [0.3, 1]. Default 0.72.
-- `start_handle` (number, optional): Start-handle fraction in [0, 1]. Default 0.3.
+- `pid` (integer, required): Target process ID.
+- `window_id` (integer, required): HWND of the window. Scopes the `element_index` cache per `(pid, window_id)`.
+- `element_index` (integer, required): Index from the last `get_window_state`.
+- `value` (string, required): New value. UIA coerces it to the element's native type.
-### set_agent_cursor_style
+```json
+{"pid":844,"window_id":10725,"element_index":14,"value":"42"}
+```
-Customize the agent-cursor's visual appearance. All fields
-optional; omitted fields keep their current value.
+## App and window lifecycle
-- gradient_colors: Array of CSS hex strings (#RRGGBB or
- #RGB) defining the arrow fill gradient from tip to tail.
- E.g. ["#FF6B6B", "#FF8E53"] for a red-orange arrow.
-- bloom_color: CSS hex string for the glow halo and the
- focus-rect highlight drawn around clicked elements.
-- image_path: Absolute or ~-rooted path to a PNG, JPEG,
- PDF, or SVG file. When set, replaces the default arrow
- with this image (drawn at shapeSize × shapeSize points,
- rotated to match the motion heading). Set to "" (empty
- string) to revert to the procedural arrow.
+### launch_app
-Example — brand-colored arrow:
- &#123;"gradient_colors": ["#A855F7", "#6366F1"], "bloom_color": "#A855F7"&#125;
+Launch an app hidden — the driver never brings the target to the foreground (`SW_SHOWNOACTIVATE` on Windows), so it does not steal focus from whatever is currently frontmost. Returns the launched pid, name, active flag, and a `windows` array (same shape as `list_windows`), so callers can skip a `list_windows` round-trip before `get_window_state`. When the launch settles but no window has materialized yet (rare), `windows` comes back empty — call `list_windows(pid)` a moment later.
-Example — custom PNG cursor:
- &#123;"image_path": "~/cursors/my-cursor.png"&#125;
+Targeting:
-Example — revert to default:
- &#123;"gradient_colors": [], "bloom_color": "", "image_path": ""&#125;
+- **Windows:** provide `bundle_id` / `name` / `aumid` (packaged apps) or `path` (full `.exe` path). An `aumid` (or `bundle_id` containing `!`) activates the packaged app via `IApplicationActivationManager` and returns the real packaged-process pid — required on Win11 for built-in apps like Notepad, Calculator, and Paint, whose `System32` `.exe` is a stub. `path` wins over `name`.
+- **macOS:** provide `bundle_id` (preferred, e.g. `com.apple.calculator`) or `name`.
+- `urls` opens each URL without activating the app.
**Arguments:**
-- `bloom_color` (string, optional): CSS hex color for the bloom halo and focus rect. Empty string reverts to default.
-- `gradient_colors` (array of string, optional): CSS hex color strings for gradient stops (tip to tail). Empty array reverts to default.
-- `image_path` (string, optional): Path to PNG/JPEG/PDF/SVG cursor image. Empty string reverts to arrow.
+- `name` (string, optional): App display name.
+- `bundle_id` (string, optional): macOS bundle id, or a Windows AUMID (`{PackageFamilyName}!{ApplicationId}`).
+- `aumid` (string, optional): Explicit Windows AUMID. Takes precedence over `bundle_id` / `name`.
+- `path` (string, optional): Full path to an executable (Windows). Wins over name/bundle_id/aumid.
+- `launch_path` (string, optional): Round-trip the `launch_path` from `list_apps`. Highest precedence.
+- `urls` (array of string, optional): URLs to open without activation.
+- `additional_arguments` (array of string, optional): Extra command-line / activation arguments.
+- `start_minimized` (boolean, optional): Launch minimized to the taskbar so the user's frontmost window stays visually on top. UIA / background dispatch still work; only screenshots and `dispatch:"foreground"` need it restored.
-### set_config
+The macOS-specific `electron_debugging_port`, `webkit_inspector_port`, and `creates_new_application_instance` fields are accepted on Windows for parity but currently no-op.
-Write a single setting into the persistent driver config
-at `~/Library/Application Support/<app-name>/config.json`.
-Values survive daemon restarts AND are propagated to the
-live session state.
+```json
+{"name":"Calculator"}
+```
-Keys are dotted snake_case paths:
- - `agent_cursor.enabled` (bool)
- - `agent_cursor.motion.start_handle` (number)
- - `agent_cursor.motion.end_handle` (number)
- - `agent_cursor.motion.arc_size` (number)
- - `agent_cursor.motion.arc_flow` (number)
- - `agent_cursor.motion.spring` (number)
- - `capture_mode` (string: `vision` | `ax` | `som`)
+### bring_to_front
-Arguments: `{"key": "<dotted.path>", "value": <json>}`.
+Activate `pid`'s window (or `window_id` if specified) — bring it to the OS foreground. **This deliberately breaks the no-foreground contract.** Use it only when an agent is about to drive a sequence of operations against a target whose input stack drops backgrounded events (Chromium DOM content, GTK widgets) and wants to pay the foreground cost once instead of per call: call `bring_to_front`, then issue subsequent input calls with `dispatch:"foreground"` (the `SendInput` swap is then a no-op since the target is already foreground). Returns `{previous_fg_hwnd, now_fg_hwnd}` so callers can restore. Windows-only; macOS / Linux return a pointer to the platform-native alternative.
**Arguments:**
-- `key` (string, required): Dotted snake_case path to a leaf config field, e.g. `agent_cursor.enabled`.
-- `value` (undefined, required): New value for `key`. JSON type depends on the key.
+- `pid` (integer, required): Target process ID.
+- `window_id` (integer, optional): HWND. Defaults to the pid's first visible top-level window.
```json
-{"key":"return","value":"value"}
+{"pid":844}
```
-### set_recording
+### kill_app
-Toggle trajectory recording. When enabled, every
-subsequent action-tool invocation (click, right_click,
-scroll, type_text, type_text_chars, press_key, hotkey,
-set_value) writes a turn folder under `output_dir`:
+Force-terminate a process by pid. Use it when the polite close path (Alt+F4 / clicking the X) fails to make the process exit — typical for UWP / WinUI3 apps (Calculator, Photos, modern Notepad) that route `WM_CLOSE` into a suspended-but-resident state. Equivalent to `taskkill /F /PID <pid>`. Unsaved state is lost; prefer the click-the-X path first.
-- `app_state.json` — post-action AX snapshot for the
- target pid (same shape as `get_window_state`).
-- `screenshot.png` — post-action per-window screenshot
- of the target's frontmost on-screen window.
-- `action.json` — tool name, full input arguments,
- result summary, pid, click point (when applicable),
- ISO-8601 timestamp.
-- `click.png` — for click-family actions only,
- `screenshot.png` with a red dot drawn at the click
- point.
+**Arguments:**
-Turn folders are named `turn-00001/`, `turn-00002/`,
-etc. Turn numbering restarts at 1 each time recording
-is (re-)enabled.
+- `pid` (integer, required): PID of the process to terminate.
-Required when `enabled=true`: `output_dir`. Expands `~`
-and creates the directory (and intermediates) if
-missing. Ignored when `enabled=false`.
+```json
+{"pid":844}
+```
+
+## Browser pages
+
+### page
+
+Interact with the browser page loaded in a running app. Supports Chrome, Brave, Edge, and Safari (AppleScript on macOS), Electron apps (CDP), Chromium/Firefox on Windows (UIA for reads; CDP for `execute_javascript` when launched with `--remote-debugging-port`), and WKWebView / Tauri / AT-SPI fallbacks. The browser does not need to be frontmost for read actions.
+
+Actions:
-State persists for the life of the daemon / MCP
-session; a restart resets to disabled with no
-on-disk state.
+- `execute_javascript` — run JS and return the result.
+- `get_text` — extract visible page text.
+- `query_dom` — find elements matching a CSS selector.
+- `click_element` — click a CSS-selected element and animate the agent cursor to its center first (visible feedback). Prefer over `execute_javascript('el.click()')` when you want the cursor cue.
+- `enable_javascript_apple_events` — macOS-only; patches the browser's Preferences to allow JS from Apple Events (requires user confirmation and a browser restart).
**Arguments:**
-- `enabled` (boolean, required): True to start recording subsequent action tool calls; false to stop.
-- `output_dir` (string, optional): Absolute or ~-rooted directory where turn folders are written. Required when enabled=true.
-- `video_experimental` (boolean, optional): Experimental: also capture the main display to `<output_dir>/recording.mp4` via SCStream (H.264, 30fps, no audio, no cursor). Capture-only — no zoom / post-process. Off by default. Ignored when enabled=false.
+- `action` (string, required): One of the actions above.
+- `pid` (integer, optional): Target process ID.
+- `window_id` (integer, optional): Target window from `list_windows`.
+- `javascript` (string, optional): JS for `execute_javascript`.
+- `css_selector` (string, optional): Selector for `query_dom`.
+- `selector` (string, optional): Selector for `click_element`.
+- `attributes` (array of string, optional): Element attributes to include in `query_dom` results.
+- `bundle_id` (string, optional): Browser bundle id (required for `enable_javascript_apple_events`, macOS only).
+- `user_has_confirmed_enabling` (boolean, optional): Must be true to proceed with `enable_javascript_apple_events`.
```json
-{"enabled":false}
+{"action":"get_text","pid":844,"window_id":10725}
```
-### set_value
-
-Set a value on a UI element. Two modes depending on element role:
-
-- **AXPopUpButton / select dropdown**: finds the child option whose
- title or value matches `value` (case-insensitive) and AXPresses it
- directly — the native macOS popup menu is never opened, so focus
- is never stolen. Use this for HTML &lt;select&gt; elements in Safari or
- any native NSPopUpButton. Pass the option's display label as `value`
- (e.g. "Blue", not "blue").
+## Agent cursor
-- **All other elements**: writes `AXValue` directly (sliders, steppers,
- date pickers, native text fields that expose settable AXValue).
+### set_agent_cursor_enabled
-For free-form text entry into web inputs, prefer `type_text_chars`
-which synthesises key events — AXValue writes are ignored by WebKit.
+Toggle the visual agent-cursor overlay. When enabled, pointer actions animate a floating arrow to the target's on-screen position before firing — purely visual, the dispatch itself is unchanged. Default: enabled. Disable with `{"enabled": false}` for headless / CI runs.
**Arguments:**
-- `element_index` (integer, required):
-- `pid` (integer, required):
-- `value` (string, required): New value. AX will coerce to the element's native type.
-- `window_id` (integer, required): CGWindowID for the window whose get_window_state produced the element_index. The element_index cache is scoped per (pid, window_id).
+- `enabled` (boolean, required): True to show the overlay; false to hide.
+- `cursor_id` (string, optional): Multi-cursor instance id. Default `default`.
```json
-{"element_index":14,"pid":844,"value":"42","window_id":10725}
+{"enabled":false}
```
-### type_text
+### set_agent_cursor_motion
+
+Configure a cursor instance's appearance and motion curve. All fields optional; only the knobs you pass change.
-Insert text into the target pid. Tries
-`AXSetAttribute(kAXSelectedText)` first (fast bulk insert —
-works for standard Cocoa text fields). If the target element
-rejects the AX write, automatically falls back to
-character-by-character `CGEvent.postToPid` synthesis —
-reaches Chromium / Electron inputs and any surface that
-doesn't implement `kAXSelectedText`.
+Appearance: `cursor_icon` (built-in `arrow`/`crosshair`/`hand`/`dot` or a PNG/SVG path), `cursor_color` (hex or CSS name), `cursor_label`, `cursor_size` (default 16), `cursor_opacity` (0.0–1.0, default 0.85).
-Special keys (Return, Escape, arrows, Tab) go through
-`press_key` / `hotkey` — they are not text.
+Motion (cubic Bezier + settle spring): `arc_size` (default 0.25), `arc_flow` (asymmetry bias [-1,1], default 0), `start_handle` / `end_handle` (default 0.3), `spring` (settle damping [0.3,1.0], default 0.72), `glide_duration_ms` (flight per move [50,5000], default 160), `dwell_after_click_ms` (default 80), `idle_hide_ms` (auto-hide delay [0,60000], 0 = never, default 20000).
-Optional `element_index` + `window_id` (from the last
-`get_window_state` snapshot of that window) pre-focuses
-that element before the write; useful for "fill this
-specific field." Without `element_index`, the write
-targets the pid's currently-focused element — useful
-after a prior click already set focus.
+**Arguments:**
-`delay_ms` (0–200, default 30) spaces successive characters
-in the CGEvent fallback path so autocomplete and IME can keep
-up; ignored when the AX path succeeds.
+- `cursor_id` (string, optional): Instance name. Default `default`.
+- `cursor_icon` / `cursor_color` / `cursor_label` (string, optional): Appearance.
+- `cursor_size` / `cursor_opacity` (number, optional): Appearance.
+- `arc_size` / `arc_flow` / `start_handle` / `end_handle` / `spring` (number, optional): Motion-curve knobs.
+- `glide_duration_ms` / `dwell_after_click_ms` / `idle_hide_ms` (number, optional): Timing knobs.
+
+### set_agent_cursor_style
-Requires Accessibility.
+Update the cursor overlay's visual style. All fields optional.
**Arguments:**
-- `delay_ms` (integer, optional): Milliseconds between characters in the CGEvent fallback path. Default 30. Ignored when the AX path succeeds.
-- `element_index` (integer, optional): Optional element_index from the last get_window_state for the same (pid, window_id). When present, the element is focused before the write. Requires window_id.
-- `pid` (integer, required): Target process ID.
-- `text` (string, required): Text to insert at the target's cursor.
-- `window_id` (integer, optional): CGWindowID for the window whose get_window_state produced the element_index. Required when element_index is used.
+- `gradient_colors` (array of string, optional): CSS hex stops tip→tail (e.g. `["#FF0000","#0000FF"]`). `[]` reverts to default.
+- `bloom_color` (string, optional): Hex color for the halo/bloom. `""` reverts to default.
+- `image_path` (string, optional): Path to a PNG/JPEG/SVG/ICO icon to use instead of the gradient arrow. `""` reverts to the arrow.
+- `cursor_id` (string, optional): Instance. Default `default`.
-```json
-{"pid":844,"text":"hello"}
-```
+### get_agent_cursor_state
-### zoom
+Report the current agent-cursor configuration: enabled flag, motion knobs, glide duration, post-click dwell, and idle-hide delay. Durations come back in milliseconds. Read-only.
+
+**Arguments:** none.
+
+## Recording and replay
+
+### start_recording
-Zoom into a rectangular region of a window screenshot at
-full (native) resolution. Use this when `get_window_state`
-returned a resized image and you need to read small text,
-identify icons, or verify UI details.
-
-Coordinates `x1, y1, x2, y2` are in the same pixel space
-as the screenshot returned by `get_window_state` (i.e. the
-resized image if `max_image_dimension` is active). The
-maximum zoom region width is 500 px in scaled-image
-coordinates.
-
-The tool maps coordinates back to original (native)
-resolution by multiplying by the compression ratio z
-(original_size / scaled_size). A 500 px-wide region in
-the scaled image becomes 500 × z native pixels, so the
-returned image will be *larger* than 500 px whenever the
-unzoomed screenshot was downscaled. This is the intended
-behaviour — the zoom undoes the compression and shows
-the region at full pixel density.
-
-A 20% padding is automatically added on every side of
-the requested region so the target remains visible even
-if the caller's coordinates are slightly off. The
-padding is clamped to the image bounds.
-
-Requires `get_window_state(pid, window_id)` earlier in this
-session so the resize ratio is known.
+Start trajectory recording. Every subsequent action-tool call (click, right_click, scroll, type_text, press_key, hotkey, set_value) writes a turn folder under `output_dir`: `app_state.json` (post-action snapshot), `screenshot.png`, `action.json` (tool name, arguments, result, pid, click point, timestamp), and `click.png` (click-family actions, with a dot at the click point). Turn folders are `turn-00001/`, `turn-00002/`, …; numbering restarts each time recording starts.
+
+**Video is on by default** — the main display is captured to `<output_dir>/recording.mp4` (H.264 / 30 fps). Pass `record_video: false` to opt out. macOS uses native ScreenCaptureKit (no extra TCC prompt, macOS 15.0+); Windows and Linux use an ffmpeg subprocess (`gdigrab` / `x11grab` + libx264, ffmpeg on PATH). When ffmpeg is missing, per-turn capture still runs and the session's `last_error` carries the diagnostic. Call `stop_recording` to finalize the mp4.
**Arguments:**
-- `pid` (integer, required): Target process ID.
-- `x1` (number, required): Left edge of the region (resized-image pixels).
-- `x2` (number, required): Right edge of the region (resized-image pixels).
-- `y1` (number, required): Top edge of the region (resized-image pixels).
-- `y2` (number, required): Bottom edge of the region (resized-image pixels).
+- `output_dir` (string, required): Absolute or `~`-rooted directory for turn folders and the video.
+- `record_video` (boolean, optional): Capture the main display to `recording.mp4`. Default true.
```json
-{"pid":844,"x1":100,"x2":100,"y1":200,"y2":200}
+{"output_dir":"~/cua-trajectories/demo1"}
```
----
+### stop_recording
-## Windows behavior notes
+Stop trajectory recording. Disables further per-turn capture and, when video was enabled, gracefully terminates ffmpeg so the mp4 is playable. A no-op on an already-stopped session. The response carries `last_video_path` when video was on.
-The tool reference above is written against the macOS reference impl. cua-driver-rs (`v0.2.9+`) layers in several Windows-only behaviors that callers need to know about to drive Chromium-based browsers and custom-drawn canvases correctly.
+**Arguments:** none.
-### `launch_app` — Chromium anti-throttling flags auto-injected
+### get_recording_state
-When `path` / `name` / `launch_path` resolves to a Chromium-based browser (`msedge`, `chrome`, `brave`, `vivaldi`, `opera`, `chromium`, `arc`, `thorium`, `iridium`, `browser` (Yandex), or a `Chrome_WidgetWin_*` / `CefBrowser*`-class Electron app), the daemon transparently prepends these three flags to `additional_arguments`:
+Report the recorder state: whether recording is enabled, the output directory, and the 1-based counter for the next turn folder. Read-only.
-```
---disable-features=CalculateNativeWinOcclusion
---disable-backgrounding-occluded-windows
---disable-renderer-backgrounding
-```
+**Arguments:** none.
-Without them, Chromium's occlusion logic suspends the renderer process because the window is launched non-foreground via `SW_SHOWNOACTIVATE` — the UIA tree returns only browser chrome (no page DOM) and `PrintWindow` returns a blank body. With them, the page renders + exposes its DOM via UIA while the window stays hidden.
+### replay_trajectory
-If you already pass `--disable-features=...`, the driver merges `CalculateNativeWinOcclusion` into your existing list instead of duplicating the flag.
+Replay a recorded trajectory by re-invoking every turn's tool call in lexical order. `dir` must point at a directory written by `start_recording`; each `turn-NNNNN/action.json` is replayed through the same dispatch path a live call uses.
-**Also worth knowing**: Chromium auto-disables page-level accessibility unless an AT-grade client probes the renderer. For UIA-based callers that want the rendered DOM in `get_window_state`, additionally pass `--force-renderer-accessibility` yourself — that's a page-level concern the driver doesn't auto-inject.
+Caveats: element-indexed actions fail (indices are per-snapshot and don't survive sessions) — pixel clicks and keyboard tools replay cleanly. Read-only tools aren't recorded, so the element cache isn't repopulated. If recording is enabled during replay, the replay is itself recorded — deliberate, for diffing a trajectory against a new build.
-### `click` — control-type whitelist for the UIA Invoke pre-check (`x, y` path)
+**Arguments:**
-The `x, y` dispatch tries UIA Invoke first (no focus steal), then falls through to PostMessage / SendInput. The UIA Invoke step only fires for elements with a **coord-independent primary action**:
+- `dir` (string, required): Trajectory directory from `start_recording`. Absolute or `~`-rooted.
+- `delay_ms` (integer, optional): Milliseconds between turns. Default 500.
+- `stop_on_error` (boolean, optional): Stop on the first error. Default true.
-`Button` · `MenuItem` · `Hyperlink` · `TabItem` · `ListItem` · `CheckBox` · `RadioButton` · `SplitButton` · `TreeItem`
+```json
+{"dir":"~/cua-trajectories/demo1"}
+```
-For canvases, paint surfaces, image maps, and custom widgets (control types `Image` / `Pane` / `Custom` / `Document` / `Group`), `click(x, y)` skips UIA Invoke even when the element advertises `InvokePattern` — invoking those fires the element's default action at the geometric centre, ignoring the requested coords. The click falls through to PostMessage or (for Chromium targets) SendInput, preserving pixel precision.
+## Config and updates
-### `click` on UWP / WinUI / XAML targets — foreground-steal bypass
+### get_config
-UWP, WinUI, and XAML-hosted apps (Calculator, Clock, Settings, Microsoft Photos, modern Notepad, etc.) self-foreground during UIA `InvokePattern.Invoke`. The XAML host calls `SetForegroundWindow(self)` unconditionally when its message-loop processes an input-like event — including UIA Invoke. Without mitigation, every `click` against a UWP target hidden behind the user's window steals focus and pulls the UWP app on top.
+Report the current persistent driver config. Read-only; returns defaults when unset. On Windows the shape is `{schema_version, version, platform, capture_mode, max_image_dimension}`.
+
+**Arguments:** none.
-The driver wraps `Invoke`, `ExpandCollapse.Expand`, and `Toggle.Toggle` calls in `EnableWindow(host, FALSE) / call / EnableWindow(host, TRUE)` when the target's top-level HWND identifies as an XAML host (window class `ApplicationFrameWindow` / `Windows.UI.Core.CoreWindow` / `WinUIDesktopWin32WindowClass` / `Microsoft.UI.Content.DesktopChildSiteBridge`, OR a known XAML-hosted exe). A disabled HWND silently rejects `SetForegroundWindow(self)`, but UIA pattern delivery still routes through (it uses the kernel accessibility channel, not the input queue gated by `EnableWindow`), so the click action still runs.
+### set_config
-Empirically: 91% of plain-Invoke samples saw the user's window drop below the target in z-order; with the bypass that drops to 0%. State changes (Calc digits, Clock tabs, Settings nav) are confirmed via UIA probes after the call.
+Write a setting into the persistent config; values take effect immediately. Two input shapes: the dotted-path form `{"key": "capture_mode", "value": "som"}` (preferred), or the bulk per-field form `{"capture_mode": "som", "max_image_dimension": 0}`.
-Classic Win32 / WPF / WinForms apps (Notepad legacy, charmap, mmc) don't self-foreground during Invoke and don't need the bypass — verified empirically. The wrap is therefore gated on `is_xaml_host_hwnd`; non-XAML targets call `Invoke` unwrapped.
+Known keys: `capture_mode` (`vision` | `ax` | `som`), `max_image_dimension` (integer), `experimental_pip` (boolean; applies on next daemon restart — Windows backend stubbed today), `experimental_pip_geometry` (string `WxH` or `WxH+X+Y`). Returns the full updated config.
-### `click(x, y)` on Chromium — SendInput with brief foreground swap
+**Arguments:**
-`PostMessage(WM_LBUTTONDOWN/UP)` on a Chromium HWND doesn't reach the DOM input pipeline (Chromium's input thread requires SendInput-queue origin). When the target HWND class is `Chrome_WidgetWin_*` or `CefBrowser*`, `click(x, y)` routes through `SendInput` with a brief `SetForegroundWindow(target)` + `SetCursorPos(x, y)` + restore-previous-foreground dance.
+- `key` (string, optional): Dotted-path leaf field. Pair with `value`.
+- `value` (any, optional): New value for `key`.
+- `capture_mode` / `max_image_dimension` / `experimental_pip` / `experimental_pip_geometry` (optional): Per-field shape.
-Trade-off: **the cursor visibly moves to the click point and the target briefly becomes foreground**. There's no Chromium-native path that delivers coord clicks to the DOM without these side effects short of launching with `--remote-debugging-port` and dispatching via the `page` tool (CDP).
+```json
+{"key":"capture_mode","value":"som"}
+```
-UIAccess requirement: `SetForegroundWindow` from a non-UIAccess daemon is rejected by Win32, in which case the tool returns the diagnostic `"SendInput inserted only 0 of 3 mouse events..."` and recommends running through the `cua-driver-uia.exe` worker. The MCP proxy auto-prefers the UIA worker's pipe when both daemons are running, so the SendInput path works out-of-the-box on a normal install.
+### check_for_update
-**When SendInput is unwanted** (zero focus steal required): use `element_index` clicks where the page exposes the target via UIA. Buttons / links / menu items in Chromium pages take the UIA Invoke path and don't incur the focus swap.
+Check whether a newer `cua-driver-rs` release is available on GitHub. Returns current and latest versions, an `update_available` boolean, the install one-liner, and the release-notes URL. Read-only — never installs. Mirror of `cua-driver check-update --json`. See [Updating](/cua-driver/guide/getting-started/updating).
-### `hotkey` — SendInput-routed for OS modifier-state propagation
+**Arguments:** none.
-The hotkey tool synthesises keystrokes via `SendInput` with a brief foreground swap (same mechanism as the Chromium `click`), so the OS modifier state visible to `GetKeyState` / `GetAsyncKeyState` is correctly updated for the duration of the keystroke. PostMessage-based paths put `WM_KEYDOWN(VK_CONTROL)` in the target's message queue but **don't** update the system modifier state — `TranslateAccelerator` then fails to recognise Ctrl+S as a Save accelerator and the `s` arrives as plain text.
+## Platform notes
-Same UIAccess constraint as Chromium clicks: `SetForegroundWindow` is restricted from non-UIAccess processes, so callers should funnel hotkey calls through the `cua-driver-uia.exe` worker. The MCP proxy already does this when both pipes are up.
+- **UWP / WinUI3 / XAML targets** (Calculator, Settings, modern Notepad) self-foreground during a naive UIA Invoke. The driver wraps Invoke / Expand / Toggle in an `EnableWindow(host, FALSE)`-gated call so the pattern still delivers through the accessibility channel without the target stealing z-order. CoreWindow-hosted apps are reached via a UIA root-walk fallback. See [Autostart → UWP automation and the uiAccess worker](/cua-driver/guide/getting-started/autostart#uwp-automation-and-the-uiaccess-worker-windows).
+- **Chromium browsers** launched hidden need anti-throttling flags so the occluded renderer keeps rendering and exposing its DOM; the driver auto-injects them. For coord clicks into DOM content, `SendInput` (`dispatch:"foreground"`) or a `--remote-debugging-port` + the `page` tool (CDP) is required — `PostMessage` doesn't reach Chromium's input thread.
+- **macOS Swift driver** uses Accessibility, `CGEvent` / SkyLight, and ScreenCaptureKit, and exposes a standalone `screenshot` tool plus Space-aware fields (`on_current_space`, `space_ids`). See the [process model](/cua-driver/guide/getting-started/process-model) and [Limits](/cua-driver/reference/limits).
diff --git a/scripts/docs-generators/config.json b/scripts/docs-generators/config.json
index ab77634b24..ec5c33ed12 100644
--- a/scripts/docs-generators/config.json
+++ b/scripts/docs-generators/config.json
@@ -26,7 +26,8 @@
"extractCommand": ".build/release/cua-driver dump-docs --type cli"
}
],
- "enabled": true
+ "enabled": false,
+ "notes": "Targets the Swift driver; the shipping cross-platform Rust port (cua-driver-rs) ships a stubbed `dump-docs --type cli`. Reference docs are hand-maintained from `cua-driver list-tools` / `describe` until the generator is repointed at the Rust binary."
},
"lume": {
"name": "Lume VM Manager",