Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.
Rust agentai-agentapplecomputer-usecomputer-use-agentcontainerizationcuadesktop-automationhacktoberfestlumemacosmanusoperatorswiftvirtualizationvirtualization-frameworkwindowswindows-sandbox
docs: land Diátaxis content rewrite (content only, fumadocs intact) (#1957)
Francesco Bonacci3ae0c3f2acc0
diff --git a/docs/content/docs/cua-driver/guide/getting-started/autostart.mdx b/docs/content/docs/cua-driver/guide/getting-started/autostart.mdxdeleted file mode 100644index 49578bc34a..0000000000--- a/docs/content/docs/cua-driver/guide/getting-started/autostart.mdx+++ /dev/null@@ -1,184 +0,0 @@-----title: Autostart-description: Keep Cua Driver's daemon running in the user's interactive session across logons, RDP disconnects, and reboots.------import { Callout } from 'fumadocs-ui/components/callout';--`cua-driver autostart` registers the platform-native "run at logon" mechanism that keeps a `cua-driver serve` daemon alive in the user's interactive session. On Windows this is the difference between GUI tools that work over SSH and GUI tools that return empty arrays — see [Running Cua Driver under SSH on Windows](/cua-driver/guide/getting-started/windows-ssh) for the full reasoning.--```bash-cua-driver autostart enable # register the platform entry-cua-driver autostart kick # start it now without waiting for next logon-cua-driver autostart status # registered? running?-cua-driver autostart disable # remove the entry-```--## What it does per platform--| Platform | Mechanism | Where it lives |-| -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------- |-| Windows | Scheduled Task `cua-driver-serve` with `LogonType: Interactive` | Task Scheduler (`schtasks /Query /TN cua-driver-serve`) |-| macOS | Not yet in the `cua-driver autostart` verb — use `install-local.sh --autostart`, which writes a `RunAtLoad` + `KeepAlive` LaunchAgent running `cua-driver serve` ([`scripts/install-local.sh`](https://github.com/trycua/cua/blob/main/libs/cua-driver/scripts/install-local.sh)) | `~/Library/LaunchAgents/com.trycua.cua-driver.plist` |-| Linux | Not yet in the `cua-driver autostart` verb — use `install-local.sh --autostart`, which writes a `systemd --user` unit ([`scripts/install-local.sh`](https://github.com/trycua/cua/blob/main/libs/cua-driver/scripts/install-local.sh)) | `~/.config/systemd/user/cua-driver-rs.service` |--<Callout type="info">- **macOS bonus — autostart fixes TCC attribution.** A LaunchAgent daemon is started by `launchd`,- so it's attributed to `com.trycua.driver` (not the terminal that would otherwise spawn it). That- means permission prompts say "Cua Driver", you grant **Accessibility + Screen Recording once**,- and the grants persist across reboots — every `cua-driver call` / `mcp` then routes through the- always-on, correctly-attributed daemon. Without autostart, a prompt raised from a terminal- attributes to the terminal and the grant never applies to the driver — run- `cua-driver permissions status` to read the driver's real grant state (it answers via the daemon,- or reports `unknown` when none is running rather than your terminal's grants), or- `cua-driver permissions grant` to grant correctly without autostart. Autostart stays opt-in- because a standing computer-control daemon with these grants is a deliberate choice, not a default.-</Callout>--<Callout type="info">- **Platform parity status.** `cua-driver autostart` is currently a Windows-only verb in the Rust- port — the macOS / Linux cases return an error pointing at the manual recipe. A cross-platform- native impl is tracked as a follow-up. The verb is documented as the cross-platform interface so- existing scripts and skill-pack instructions land in the right place once parity ships.-</Callout>--## The four verbs--### `enable`--Registers the Scheduled Task on Windows. Idempotent — any existing task with the same name is replaced, so re-running after a `cua-driver` upgrade picks up the new binary path automatically.--```powershell-cua-driver autostart enable-# Registered autostart entry 'cua-driver-serve'.-# cua-driver serve will start at every interactive logon.-```--The registered task pins to `LogonType: Interactive` — load-bearing, because the alternative (`S4U` / `Password`) would land the daemon back in Session 0. The exact PowerShell that gets run is in [`autostart.rs`](https://github.com/trycua/cua/blob/main/libs/cua-driver/rust/crates/cua-driver/src/autostart.rs) and stays in lock-step with `scripts/install.ps1`.--### `kick`--Runs the registered task immediately, without waiting for the next logon. Use this right after `enable` so the daemon is up in the current session — otherwise `enable` is a "next-logon" change and your current shell still has no daemon to talk to.--```powershell-cua-driver autostart enable-cua-driver autostart kick-cua-driver status # confirms the daemon's listening-```--### `status`--Reports the registration + liveness state in three flavours:--```text-not-registered # no Scheduled Task exists-registered (not running) # task is there but no daemon process-registered (running) # task is there AND a daemon is listening on the socket-```--The "running" check probes the daemon's named pipe directly (no `tasklist` spawn — that's slow on first run), so it reflects real liveness, not just "Task Scheduler thinks the task is registered".--### `disable`--Removes the Scheduled Task. No-op if nothing's registered, so it's safe to script unconditionally.--```powershell-cua-driver autostart disable-# Removed autostart entry 'cua-driver-serve' (no-op if it was already absent).-```--## Worked example: `enable && kick` from an interactive shell--The canonical first-time recipe — run this once from an interactive shell (RDP, local console, Windows Terminal under your logged-in user) and the daemon is up immediately _and_ will come back after every logon:--```powershell-cua-driver autostart enable-cua-driver autostart kick-cua-driver status-# Cua Driver daemon is running-# socket: \\.\pipe\cua-driver-# pid: 12345-```--After this, you can SSH into the box and `cua-driver mcp` / `cua-driver call <tool>` will proxy through the running daemon — see [Running Cua Driver under SSH on Windows](/cua-driver/guide/getting-started/windows-ssh).--## The "daemon survives RDP disconnect" property--Scheduled Tasks registered with `LogonType: Interactive` are _session-attached_ — the task and any processes it spawned live and die with the interactive session, not with the RDP client window. Disconnecting the RDP client leaves the session in `Disc` (disconnected) state, which is still a live interactive session as far as Windows is concerned:--```powershell-query session-# SESSIONNAME USERNAME ID STATE TYPE DEVICE-# console 0 Disc-# rdp-tcp#23 you 2 Disc ← still alive-```--The daemon keeps running in session `2`. A subsequent SSH connection (which lands in Session 0) can proxy through the still-listening daemon. Reconnecting RDP reattaches to the same session — the daemon was never killed.--The session only dies on explicit logoff (`logoff` / Start menu → Sign out) or reboot. On reboot, the next interactive logon re-triggers the Scheduled Task and a fresh daemon comes back up.--<Callout type="info">- **Prerequisite — an active interactive session must exist.** `kick` runs the task in *some*- session, but Task Scheduler only triggers it when an interactive logon is present. If you've never- logged in via RDP / console on a fresh box, `kick` has nowhere to land the daemon. The fix is to- RDP in once (the logon trigger fires `serve` automatically) — after that, even an RDP-disconnect- keeps the session alive in `Disc` state and the daemon along with it.-</Callout>--## UWP automation and the uiAccess worker (Windows)--Modern UWP apps (modern Notepad, Settings, Calculator, the new File Explorer) run inside an AppContainer with **User Interface Privilege Isolation (UIPI)** enabled. UIPI is an integrity-level boundary: a process at Medium integrity (like the regular `cua-driver serve` daemon) cannot inject `SendInput`, read the UI Automation tree, or `PostMessage` across the boundary into an AppContainer'd process. The OS returns `ERROR_ACCESS_DENIED` and the tool call silently fails.--Windows' answer is **`uiAccess="true"`** in the application manifest — a flag that asks the OS to elevate the process to UIAccess integrity at launch, lifting UIPI for that process. The catch: a uiAccess-marked PE can only be launched via `ShellExecute` (not `CreateProcess`), must be Authenticode-signed, and must live in a "secure" location (`\Program Files\` by default, or anywhere if the `EnableSecureUIAPaths` policy is relaxed).--cua-driver-rs satisfies these constraints with a **second sibling binary**, `cua-driver-uia.exe`, that ships alongside `cua-driver.exe` and is the only PE in the project carrying `uiAccess="true"` in its manifest. The main binary (CLI + MCP) stays a normal `asInvoker` process so all the existing CLI / stdio MCP entrypoints continue to work — UWP tool calls just route through the sibling worker over a separate named pipe.--### How it works at runtime--| Component | Integrity | Pipe |-| ---------------------------------------------- | --------- | ------------------------- |-| `cua-driver.exe serve` | Medium | `\\.\pipe\cua-driver` |-| `cua-driver-uia.exe` (auto-spawned by `serve`) | UIAccess | `\\.\pipe\cua-driver-uia` |--When `cua-driver serve` starts, it checks for `cua-driver-uia.exe` next to its own binary and `ShellExecute`s it as a child. The OS elevates the child to UIAccess integrity (provided the signing + secure-location requirements are met) and the child binds the second pipe. `cua-driver call ...` and `cua-driver mcp` both prefer `\\.\pipe\cua-driver-uia` over the main daemon pipe when both are listening — tool calls land in the uiAccess process, UIPI doesn't block them.--No additional `autostart` configuration is needed: the single `cua-driver-serve` Scheduled Task is the only registered entry. The worker is a child of the daemon, sharing its lifecycle.--### What it unlocks--- Reading the **UI Automation tree** of modern Notepad, Settings, modern File Explorer, and other XAML / UWP apps-- `type_text` via UIA `ValuePattern.SetValue` against modern Notepad's document area-- `SendInput`-based interactions that previously failed with `ERROR_ACCESS_DENIED`--### CoreWindow-class apps and the UIA root-walk fallback--Some apps (Calculator, Settings, older UWPs) host their top-level window as a `Windows.UI.Core.CoreWindow`. For these, `IUIAutomation::ElementFromHandle(hwnd)` returns an empty wrapper — the actual XAML tree is registered at the desktop root as a sibling UIA element with the same `ProcessId`, not as a child of the wrapper element. This is the same pattern `inspect.exe` and Accessibility Insights walk.--`get_window_state` already handles this transparently: when the primary `ElementFromHandle` path yields zero actionable nodes, it falls back to `GetRootElement().FindAll(Children, TrueCondition)` filtered by `ProcessId`. Callers don't need to know which path produced the result. See [#1606](https://github.com/trycua/cua/pull/1606) for the fix and [#1601](https://github.com/trycua/cua/issues/1601) for the diagnosis.--### What it does NOT unlock--- **`hotkey` accelerator shortcuts on XAML targets** (Ctrl+S, Ctrl+F, etc.). `hotkey` posts `WM_KEYDOWN`/`WM_KEYUP` via `PostMessage`, but modern Notepad's CoreInput dispatcher only reads from the system input queue. Workaround: walk the UIA tree for the matching menu item and `click` it. Tracked in [#1607](https://github.com/trycua/cua/issues/1607) — the planned fix mirrors the UIA `ValuePattern.SetValue` routing already done for `type_text`.-- Apps that explicitly opt out of UI Automation-- Cross-session injection (the worker still has to live in the same interactive session as the target app)--<Callout type="warn">- **Signing prerequisite.** `uiAccess` apps must be Authenticode-signed by a cert trusted on the- machine — Microsoft does not honor the manifest otherwise. Until the production CD pipeline ships- a signed `cua-driver-uia.exe` ([GH #1602](https://github.com/trycua/cua/issues/1602)), the worker- fails to elevate on default-policy machines, and UWP automation falls back to the Medium-integrity- behavior (UIPI blocking). The main CLI / MCP daemon is unaffected — only the worker depends on the- signature. Until then, the daemon logs `uia spawn failed` to stderr and proceeds without it.-</Callout>--## Why not just `cua-driver serve &` over SSH?--A naively-spawned `cua-driver serve &` over SSH inherits the SSH session — which on Windows OpenSSH is **Session 0**, the non-interactive services session. The daemon comes up, but every GUI tool it answers (`list_windows`, `click`, `screenshot`, ...) bottoms out in Win32 APIs scoped to the calling session's WindowStation + Desktop, which Session 0 doesn't have. Tools silently return empty arrays.--`autostart enable && kick` sidesteps this by running the daemon under the user's interactive logon (Session 1+) instead of inheriting whatever session called `serve`. The daemon then has a real desktop attached and GUI tools work.--See [Running Cua Driver under SSH on Windows](/cua-driver/guide/getting-started/windows-ssh) for the end-to-end SSH workflow.--## Uninstall behaviour--The platform uninstaller (`uninstall.ps1` on Windows, `uninstall.sh` elsewhere) removes the autostart entry automatically as part of its teardown — you don't need to call `autostart disable` explicitly before uninstalling. See the [uninstall removal matrix](/cua-driver/guide/getting-started/installation#uninstall) for the per-platform specifics.diff --git a/docs/content/docs/cua-driver/guide/getting-started/comparison.mdx b/docs/content/docs/cua-driver/guide/getting-started/comparison.mdxdeleted file mode 100644index bbb521b539..0000000000--- a/docs/content/docs/cua-driver/guide/getting-started/comparison.mdx+++ /dev/null@@ -1,80 +0,0 @@-----title: Comparison-description: How Cua Driver compares to other computer-use tools------import { Callout } from 'fumadocs-ui/components/callout';--This page compares Cua Driver with other computer-use tools. The best choice depends on your workload. Several of the alternatives below are macOS-specific; Cua Driver runs on macOS, Windows, and Linux.--## Quick comparison--| Feature | Cua Driver | Codex Computer Use | Claude Computer Use | Lume |-| -------------------- | ----------------------- | -------------------- | -------------------- | ----------------------- |-| **License** | MIT | Closed source | Closed source | MIT |-| **Drives host machine** | Yes | Yes | No (sandbox only) | No (hosts VMs) |-| **Sandbox / VM** | No | No | Yes (Cowork VM) | Yes (macOS + Linux VMs) |-| **Backgrounded** | Default | Default | N/A (sandbox) | N/A |-| **MCP server** | Yes (stdio) | No | No | Yes |-| **Agent-agnostic** | Yes (any MCP client) | Codex-only | Claude-only | Yes (HTTP API) |-| **Capture modes** | vision / ax / som | Vision | Vision | N/A |-| **Primary use case** | Agent automation | Agent automation | Safe agent execution | Ephemeral VMs |--## Codex Computer Use--[Codex](https://openai.com/codex) describes its macOS computer-use feature as: "With computer use on macOS, Codex can now use any app by seeing, clicking, and typing with its own cursor. It runs in the background without taking over your computer, working on tasks like frontend iteration, app testing, or any workflow that doesn't expose an API."--That's a fair description of what Codex does on macOS. Cua Driver differs on four axes:--- **Agent-agnostic.** Cua Driver works with any agent that speaks MCP or shells out. Codex's computer-use is Codex-only.-- **Open source, MIT licensed.** Codex is a closed product.-- **MCP-native.** Cua Driver speaks MCP over stdio. Paste `cua-driver mcp-config` into your client and it's wired up. Codex has no MCP surface.-- **Three capture modalities.** Codex is vision-only. Cua Driver ships `vision` (PNG only), `ax` (tree only), and `som` (both). AX mode skips Screen Recording entirely and gives deterministic element addressing; `som` gives both halves for disambiguation when labels repeat.--**When to choose Codex**: if you're already inside Codex and the bundled computer-use covers your task.--**When to choose Cua Driver**: if you want the same no-foreground contract across any agent, any modality, with an open source license.--## Claude Computer Use (Cowork)--[Claude Cowork](https://support.claude.com/en/articles/13345190-getting-started-with-cowork) runs Claude Code inside a sandboxed VM. The VM boots a Linux root filesystem where Claude can execute commands and drive a virtualized desktop without access to your host system.--This is a different product shape. Cowork is sandbox-first: Claude operates an isolated environment. Cua Driver is host-first: it operates your real Mac with a background-drive contract.--**When to choose Cowork**: if you want Claude to operate a disposable environment where destructive actions are contained by default.--**When to choose Cua Driver**: if you want an agent to operate your real, running apps (the editor, the browser, Finder) without taking focus away from you.--<Callout type="info">- The two aren't mutually exclusive. An agent running in Cowork could drive Cua Driver on a host via MCP- if you expose the stdio server through the sandbox boundary, but that's not a supported configuration- today.-</Callout>--## Lume--[Lume](/lume/guide/getting-started/introduction) is a macOS VM runtime that spins up Apple Virtualization Framework guests. Cua Driver operates your host Mac; Lume hosts isolated VMs.--**Key differences:**--- Lume boots a macOS or Linux VM and hands it to you. Cua Driver does not host any VMs.-- Cua Driver drives apps on your current machine without changing which one is frontmost. Lume has nothing to do with the host's foreground state.-- Both are agent-useful, for different reasons. Lume gives you a disposable macOS environment for CI, sandboxing, or cross-version testing. Cua Driver gives you backgrounded control over the real thing.--**When to choose Lume**: if you need an isolated macOS VM for automation, testing, or sandboxing untrusted workloads.--**When to choose Cua Driver**: if you want an agent to drive real apps on your host without stealing focus.--## Summary--Cua Driver fits when all of these are true:--- You want an agent to drive real apps on your own machine (not a VM).-- The user needs to keep working in another app while the agent operates.-- You're building against any MCP-capable agent, not locked to one vendor.-- MIT licensing matters.--The trade-offs you accept:--- Destructive actions hit your real filesystem. Confirm user intent before deleting, overwriting, or sending.-- A handful of app classes (Chromium web-content right-click, canvas viewports like Blender or Unity) need known workarounds. See [Limits](/cua-driver/reference/limits).diff --git a/docs/content/docs/cua-driver/guide/getting-started/faq.mdx b/docs/content/docs/cua-driver/guide/getting-started/faq.mdxdeleted file mode 100644index d857bdc490..0000000000--- a/docs/content/docs/cua-driver/guide/getting-started/faq.mdx+++ /dev/null@@ -1,278 +0,0 @@-----title: FAQ-description: Frequently asked questions about Cua Driver------import { Callout } from 'fumadocs-ui/components/callout';--Common gotchas and questions. For the full action loop and tool semantics, see the [CLI reference](/cua-driver/reference/cli-reference) and [MCP tools reference](/cua-driver/reference/mcp-tools).--## The action loop--### Why do I get `No cached AX state`?--Element-indexed actions read an in-memory cache keyed on `(pid, window_id)`. The cache is populated by `get_window_state` and replaced on every snapshot.--Two common causes:--1. You didn't call `get_window_state` in the current turn.-2. You called it with a different `window_id` than the one in the action.--```bash-# Populate the cache for this exact (pid, window_id).-cua-driver call get_window_state '{"pid":844,"window_id":10725}'--# Then act with the same window_id.-cua-driver call click '{"pid":844,"window_id":10725,"element_index":14}'-```--If you're running one-shot CLI invocations without a daemon, the cache lives for one process lifetime. Start the daemon first:--```bash-open -n -g -a CuaDriver --args serve-```--### Why did my screenshot come back empty (`has_screenshot: false`)?--The window capture raced against a close, or the window has no backing store yet. Re-snapshot. If it persists, pick a different `window_id` via `list_windows`.--### `screenshot` or `get_window_state` fails with "ScreenCaptureKit refused this window" / "Could not start streaming".--Known macOS 26.4.x ScreenCaptureKit regression on physical Macs (`SCStreamError` code -3801, sometimes localized — e.g. Japanese "オーディオ/ビデオの取り込みがうまくいかなかったため、ストリーミングを開始できませんでした"). The driver already:--1. Retries the SCK call once after a brief delay (covers transient failures).-2. Falls back to the legacy `CGWindowListCreateImage` path (works on many windows the SCK regression breaks).--If both refuse, the error surfaces with an actionable hint. Workarounds in order of preference:--1. **Try a different `window_id` on the same app.** Usually only one specific window is hit.-2. **Switch to AX-only capture for that workflow.** Element-indexed clicks don't need pixels:-- ```bash- cua-driver config set capture_mode ax- ```-- `get_window_state` then returns the AX tree without attempting a screenshot, and `click({pid, window_id, element_index: N})` works as before.--3. **Re-snapshot a moment later.** The failure is sometimes transient.--`get_window_state` does **not** hard-fail on this error: the AX tree still ships in the response with a warning line, so element-indexed clicks keep working even when the screenshot is unavailable. The standalone `screenshot` tool does hard-fail (no AX tree to fall back to).--### The AX tree is tiny. What's happening?--Check the capture mode:--```bash-cua-driver config get capture_mode-```--Default is `som` (tree + screenshot). If it reads `vision`, `get_window_state` omits the tree by design (PNG only). Switch back to `som` to get both:--```bash-cua-driver config set capture_mode som-```--If the mode is `som` or `ax` and the tree is still small, the target uses custom rendering (Blender, Unity, Electron with AX disabled). For Chromium/Electron, retry `get_window_state` once — the tree populates on the second call. For canvas-backed apps, reach for pixel clicks instead.--## Window state--### My keyboard commit (Return, Space, Tab) on a minimized window silently no-ops.--Minimized windows receive AX reads and AX-dispatched clicks normally, but keyboard commits fail because AX focus doesn't propagate to renderer focus on a minimized window. You hear the macOS system-alert beep, or nothing happens.--Workarounds in order of preference:--1. **Use `set_value` to write the field's entire value directly.** Bypasses keyboard commits.-2. **AX-click a commit-equivalent button** (Go, Submit, checkbox). Clicks route through `AXPress` and don't need renderer focus.-3. **Last resort: ask the user to un-minimize the window.** Don't deminiaturize programmatically — layout-disrupting on many apps.--### My backgrounded SwiftUI app (System Settings) returns an almost-empty AX tree.--Windows on another Space often strip their AX tree to the menu bar on SwiftUI apps. AppKit apps are usually fine.--`get_window_state` returns `off_space: true` plus `window_space_ids` when this happens, so you can detect it. Solutions:--- Ask the user to Mission-Control back to the Space that holds the target.-- Drive the app through in-window toolbar buttons (which often stay exposed) rather than deep nested controls.-- Accept the limitation for the current session.--## Browsers and Electron--### Right-click on Chromium web content fires as a left-click.--A known Chromium renderer-IPC limit: the filter coerces synthetic right-click subtype to left on every non-HID-tap path. Use `right_click({pid, element_index})` on AX-addressable targets (links, buttons, toolbar items). For web content itself (right-clicking an image or selection), there is no backgrounded path today. See [Limits](/cua-driver/reference/limits) for the full note.--### Pixel click on a YouTube video doesn't play or pause.--HTML5's click-to-play handler rejects some synthetic click paths. Use keyboard instead:--```bash-cua-driver call press_key '{"pid":<pid>,"key":"k"}' # YouTube play/pause-cua-driver call press_key '{"pid":<pid>,"key":"space"}' # generic video play/pause-```--Keyboard events travel through a different auth envelope and reach the page.--### How do I navigate to a URL in Chrome without stealing focus?--Pass the URL to `launch_app`:--```bash-cua-driver call launch_app '{"bundle_id":"com.google.Chrome","urls":["https://trycua.com"]}'-```--The URL opens in a new window via Chrome's `application(_:open:)` delegate. The driver's focus-restore guard catches Chrome's internal activation and clobbers the frontmost back to what it was before the call.--<Callout type="warn">- Don't use `hotkey ⌘L` to focus the omnibox. Even when delivered to a backgrounded pid, `⌘L` steals- focus because the receiving app interprets "user wants to type here" as activation intent.-</Callout>--## The agent cursor--### How do I disable the visual cursor overlay?--```bash-cua-driver call set_agent_cursor_enabled '{"enabled":false}'-```--Or via config:--```bash-cua-driver config set agent_cursor.enabled false-```--The overlay only renders when the driver has an AppKit run loop (inside `cua-driver serve` or `cua-driver mcp`). One-shot CLI invocations skip it entirely.--### Can I make the cursor move faster?--Tune the motion knobs:--```bash-cua-driver call set_agent_cursor_motion '{"glide_duration_ms":300}'-```--See `set_agent_cursor_motion` in the [MCP tools reference](/cua-driver/reference/mcp-tools) for every knob.--## Concurrency & multiple agents--### Two agents (or subagents) take turns instead of running in parallel. Why?--The daemon is concurrent — it handles each connection on its own task, and proves it: two raw socket connections can drive two cursors simultaneously. The bottleneck is the **stdio MCP transport**: an MCP client (e.g. Claude Code) spawns **one** `cua-driver mcp` process per server config and shares it across all subagents, and a single stdio pipe carries one in-flight request at a time. So tool calls serialize at the transport, upstream of cua-driver. Sessions give concurrent runs distinct **cursors**; they don't give them distinct **connections**, and parallelism needs distinct connections.--Claude Code only parallelizes tool calls it deems concurrency-safe (`readOnlyHint:true`). cua-driver's read-only tools (including `move_cursor`) already parallelize; mutating tools (`click`, `type_text`) serialize by design — parallelizing an ordered sequence like `3 → + → 1 → =` would race.--### How do I run multiple agents truly in parallel?--Give each agent its **own** connection. Two options:--1. **Separate `cua-driver mcp` processes** — e.g. two Claude Code instances. Each spawns its own proxy → its own daemon connection → the (concurrent) daemon runs them in parallel.-2. **The HTTP transport** — start the daemon with `CUA_DRIVER_RS_MCP_HTTP_PORT=<port>` and it also serves MCP over HTTP at `POST http://127.0.0.1:<port>/mcp` (loopback only). Point each agent's MCP client at that URL; each opens its own HTTP connection and they run concurrently (measured 3.6× on 10 parallel calls). Per-connection ordering keeps each agent's sequence correct; the per-`(pid, window_id)` cache + per-session cursor make concurrent cross-connection actions safe.--```bash-# daemon with the HTTP MCP endpoint enabled-CUA_DRIVER_RS_MCP_HTTP_PORT=8787 cua-driver serve-# sanity check-curl -s -XPOST http://127.0.0.1:8787/mcp \- -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"start_session","arguments":{"session":"agent-1"}}}'-```--## Permissions--### `check_permissions` says `NOT granted` but I granted both.--TCC checks the calling process, not `CuaDriver.app`. Inside IDE terminals (Claude Code, Cursor, VS Code, Conductor), the shell inherits the IDE's TCC responsibility chain. So running `cua-driver call check_permissions` in one of those shells reads against the IDE's bundle, not `com.trycua.driver`.--Start the daemon first, which runs through LaunchServices under the CuaDriver bundle:--```bash-open -n -g -a CuaDriver --args serve-cua-driver call check_permissions # forwards to the daemon — authoritative answer-```--Or use the dedicated verbs, which handle attribution for you: `cua-driver permissions grant` launches-CuaDriver via LaunchServices and waits for the grant, and `cua-driver permissions status` reports the-driver's real state through the daemon — when no daemon is running it answers `❓ unknown` rather than-the calling terminal's grants, so it never reports a false `granted`.--### `cua-driver mcp` from an IDE terminal can't see Accessibility.--This is the same TCC-attribution issue as the previous question, applied to the stdio MCP server. `cua-driver mcp` detects it and auto-launches the daemon via `open -n -g -a CuaDriver --args serve`, then proxies every MCP tool call through the daemon's Unix socket. From the MCP client's perspective nothing changes — the same stdio server, the same tool names, the same response shapes — but every AX probe now hits a process that LaunchServices attributed to `CuaDriver.app`. No Python bridge needed. Force the in-process path with `--no-daemon-relaunch` (or `CUA_DRIVER_MCP_NO_RELAUNCH=1`) if you really want it, e.g. when `mcp` is launched from `CuaDriver.app` directly.--### I keep seeing the permissions dialog on every launch.--macOS is attributing the process to a different bundle id than the one you granted. Run `cua-driver diagnose` and share the output when filing an issue. It reports cdhash, team id, and which bundle TCC matched against.--### After a rebuild, the driver reports `NOT granted` but System Settings still shows CuaDriver toggled ON.--This is a stale TCC grant. TCC pins each Accessibility / Screen-Recording grant to the app's **designated requirement at grant time**. If you first granted while `CuaDriver.app` was *ad-hoc* signed, the requirement is a bare `cdhash H"…"`, which changes on every rebuild — so the grant row stays `allowed` but its requirement no longer matches the new binary, and re-toggling the switch doesn't help (the row already records a decision, so the prompt never re-fires).--Release builds are CI-signed with a stable identity, so this only affects the local dev loop (`install-local.sh`). That installer now signs with a **stable self-signed certificate** and, when it detects the signing identity changed since the last install, runs `tccutil reset` for you so the next grant re-pins cleanly. After you re-grant once on the certificate-signed build, the grant survives every future rebuild.--To clear it by hand:--```bash-tccutil reset Accessibility com.trycua.driver-tccutil reset ScreenCapture com.trycua.driver-cua-driver permissions grant # re-grant once; now pinned to the stable cert-```--## Config and telemetry--### Where does config live?--```-~/Library/Application Support/Cua Driver/config.json-```--Read and write via `cua-driver config`:--```bash-cua-driver config # show full config-cua-driver config get capture_mode-cua-driver config set capture_mode som-cua-driver config reset # overwrite with defaults-```--### How do I opt out of telemetry?--```bash-cua-driver config telemetry disable-```--Or set `CUA_DRIVER_TELEMETRY_ENABLED=0` in the environment for a one-off override.--Telemetry records anonymous subcommand usage (`cua_driver_api_click`, `cua_driver_serve`, etc). No command arguments, file paths, or personal information are collected.--## Windows (cua-driver-rs)--### Why do `click` / `type_text` / `screenshot` / `list_windows` return empty results?--Your `cua-driver` daemon is probably running in **Windows Session 0** (services / SSH-launched processes). Every window-driving Win32 API — `EnumWindows`, `PostMessage`, `PrintWindow`, UI Automation tree walks — is scoped to the calling process's WindowStation + Desktop. Session 0 has no attached interactive desktop, so all these APIs silently succeed but return empty.--Run `cua-driver doctor` — it surfaces this directly:--```-[warn] interactive session: running in Session 0 (services); window-driving tools- (list_windows, click, type_text, screenshot, get_window_state) will return- empty results — these APIs need an attached interactive desktop.-```--Fix: re-launch `cua-driver serve` from an interactive logon — RDP into the host, console session, or a scheduled task configured with `/RU <user> /IT` so it runs in the user's session. The non-GUI tools (`list_apps` for Win32 entries, `get_config`, `doctor`, telemetry plumbing) work normally in Session 0 either way.--### `launch_app` with a UWP app fails with "requires an interactive session" in Session 0.--Same root cause: `IApplicationActivationManager::ActivateApplication` needs the per-user AppX runtime which only exists in interactive logons. Until you re-run the daemon from a Session 1+ logon, fall back to the Win32 path (`launch_app {"path":"C:\\Windows\\System32\\notepad.exe"}`) — `ShellExecuteEx`-based launches work in Session 0 for anything reachable via PATH.--## Testing--### Are there tests I can run against my install?--The project includes Python integration tests under `libs/cua-driver/tests/` that exercise the real `cua-driver` stdio server against `unittest`. They run as part of `scripts/test.sh`:--```bash-cd libs/cua-driver-./scripts/test.sh-```--For a quick manual smoke check, the Calculator test from `libs/cua-driver/Skills/cua-driver/TESTS.md` is a good five-minute run: launch Calculator hidden, snapshot, click 17 × 23 by element index, re-snapshot, verify the display reads `391` and Calculator never came to the foreground.diff --git a/docs/content/docs/cua-driver/guide/getting-started/installation.mdx b/docs/content/docs/cua-driver/guide/getting-started/installation.mdxdeleted file mode 100644index 34069299a8..0000000000--- a/docs/content/docs/cua-driver/guide/getting-started/installation.mdx+++ /dev/null@@ -1,690 +0,0 @@-----title: Installation-description: Install Cua Driver on macOS, Windows, or the Linux backend------import { Callout } from 'fumadocs-ui/components/callout';--Install Cua Driver with the command for your platform. macOS, Windows, and Linux are all supported install targets. (On Linux, native Wayland input and capture are opt-in and still maturing — see the [Linux guide](/cua-driver/guide/getting-started/linux).)--| Platform | Command |-| ----------------- | ------------------------------------------------------------------------------------------------------------------- |-| macOS | `/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.sh)"` |-| Linux | `/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.sh)"` |-| Windows | `irm https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.ps1 \| iex` |--**macOS / Linux**--```bash-/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.sh)"-```--**Windows (PowerShell)**--```powershell-irm https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.ps1 | iex-```--On macOS, the installer drops `CuaDriver.app` into `/Applications` and symlinks the binary at `~/.local/bin/cua-driver`. The bundle is signed under `com.trycua.driver`, so TCC grants survive every release.--On Linux, the Linux backend installer downloads the release into `~/.cua-driver/packages/releases/`, retargets `~/.cua-driver/packages/current`, and symlinks `~/.local/bin/cua-driver`.--<Callout type="info">- **Install home is `~/.cua-driver` on every platform.** Earlier `cua-driver-rs` releases (before- v0.2.16) put the Linux/macOS package home at `~/.cua-driver-rs`; the release installer now writes- to `~/.cua-driver` like the local installer and the runtime already do. Re-running the installer- over an older release **auto-sweeps** a stale `~/.cua-driver-rs`, and also cleans up any prior- `install-local` dev build (`*-local-*` release dirs + the local signing-identity marker) under the- shared home so the downloaded release is the single authoritative install. The `CUA_DRIVER_RS_HOME`- override is still honored for back-compat.-</Callout>--<Callout type="info">- **Linux:** the backend targets X11 and runs the full tool surface — input, screenshots, and- recording — in the background. Native **Wayland** input and capture are opt-in- (`CUA_DRIVER_RS_ENABLE_WAYLAND=1`) and still maturing; on Wayland sessions the default path is- XWayland. Run `cua-driver doctor` for distro prerequisites (AT-SPI bus, ffmpeg). See the- [Linux guide](/cua-driver/guide/getting-started/linux).-</Callout>--On Windows, the installer downloads the release into `%USERPROFILE%\.cua-driver\packages\releases\`, retargets the `current` junction, and exposes `cua-driver.exe` through `%LOCALAPPDATA%\Programs\Cua\cua-driver\bin`.--The install runs **without sudo/admin**. On macOS and Linux the CLI symlink lives under `$HOME`; on Windows the script updates the User-scope `Path`. If `~/.local/bin` is not on your PATH, the shell installer detects your shell (`zsh` / `bash` / `fish`) and appends an `export PATH=...` line to the right rc file. Reload with `source ~/.zshrc` (or restart your terminal).--<Callout type="info">-**Power users:** override the install path with `--bin-dir` (or `CUA_DRIVER_RS_INSTALL_DIR`). Use `--no-modify-path` if you'd rather edit your shell rc yourself.--```bash-/bin/bash -c "$(curl -fsSL .../install.sh)" -- --bin-dir=/usr/local/bin # legacy system path (needs sudo)-/bin/bash -c "$(curl -fsSL .../install.sh)" -- --no-modify-path # don't touch shell rc-```--</Callout>--<Callout type="warn">- **Upgrading from the retired Swift driver?** The Rust implementation uses the same- `/Applications/CuaDriver.app` bundle id (`com.trycua.driver`). Existing Accessibility / Screen- Recording grants usually transfer; macOS may ask for a one-time re-grant because the binary cdhash- changed.-</Callout>--Both canonical installers live under `libs/cua-driver/scripts/`. The shell installer covers macOS and the Linux backend; `install.ps1` covers Windows.--The Windows installer auto-detects host architecture (x64 / arm64) via `[System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture` and downloads the matching `cua-driver-rs-<v>-windows-{x86_64,arm64}.zip` from GitHub Releases.--It runs **without admin** and **without Developer Mode**.--It also appends `%LOCALAPPDATA%\Programs\Cua\cua-driver\bin` to your **User-scope `Path`** so `cua-driver --version` resolves in any new PowerShell window. (v0.2.13 and earlier used `Programs\trycua\cua-driver-rs\bin`; the v0.2.14 installer auto-migrates the legacy layout.) The write is idempotent (re-running the installer never duplicates the entry) and falls back to printing manual `[Environment]::SetEnvironmentVariable(...)` instructions if the registry write is blocked (group policy, locked-down account). Pass `-NoPathUpdate` to suppress the auto-add — useful when you manage `Path` out-of-band via chezmoi / dotfiles / GPO:--```powershell-# Fetch + invoke explicitly so we can pass the switch:-& ([scriptblock]::Create((irm https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.ps1))) -NoPathUpdate-```--The install layout is wired with NTFS directory junctions (`IO_REPARSE_TAG_MOUNT_POINT`), which any unprivileged user can create — no elevated symlink privilege required.--#### Versioned-dirs install layout--Linux and Windows installs land in a three-tier layout that makes upgrades and rollbacks an atomic retarget of one link, never a file overwrite. The macOS install intentionally stays on a different layout (`/Applications/CuaDriver.app`) — see the [macOS asymmetry note](#why-macos-uses-a-different-layout) below.--**Linux**--```-$CUA_DRIVER_RS_HOME/ (default: ~/.cua-driver)- packages/- releases/- 0.2.0-x86_64-unknown-linux-gnu/cua-driver (real binary, immutable)- 0.2.1-x86_64-unknown-linux-gnu/cua-driver (real binary, immutable)- current -> releases/0.2.1-x86_64-unknown-linux-gnu (symlink — active version)- current/cua-driver (= the binary inside the target dir)-$CUA_DRIVER_RS_INSTALL_DIR/cua-driver -> $CUA_DRIVER_RS_HOME/packages/current/cua-driver- (default: ~/.local/bin/cua-driver)-```--**Windows**--```-%USERPROFILE%\.cua-driver\ ($env:CUA_DRIVER_RS_HOME)- packages\- releases\- 0.2.14-x86_64-pc-windows-msvc\cua-driver.exe (real binary, immutable)- 0.2.15-x86_64-pc-windows-msvc\cua-driver.exe (real binary, immutable)- current\ (directory junction → releases\0.2.15-...)-%LOCALAPPDATA%\Programs\Cua\cua-driver\bin\ ($env:CUA_DRIVER_RS_INSTALL_DIR)- (directory junction → packages\current)-```--v0.2.13 and earlier used `%LOCALAPPDATA%\Programs\trycua\cua-driver-rs\bin\` and `%USERPROFILE%\.cua-driver-rs\`. The v0.2.14+ installer detects the legacy layout and migrates transparently — stop any running daemon, delete the legacy task / junctions / package home, prune the legacy PATH entry, then install at the new path.--Both junctions are NTFS reparse points (`IO_REPARSE_TAG_MOUNT_POINT`), so the whole PATH-resolution chain `bin\ → current\ → releases\<v>\cua-driver.exe` is two filesystem hops transparently served from whichever release the inner junction currently points at.--#### Atomic upgrades and rollback--Both platforms ship every release into its own per-version directory under `packages/releases/`. Older versions are kept on disk so you can roll back without re-downloading. The active version is whichever directory `packages/current` points at.--**Linux — roll back to a specific version**--```bash-# The symlink target is RELATIVE to ~/.cua-driver/packages/, so just-# "releases/<v>-<target>" (no leading "../") — matches what install.sh writes.-ln -sfn releases/0.2.0-x86_64-unknown-linux-gnu ~/.cua-driver/packages/.current.tmp-mv -Tf ~/.cua-driver/packages/.current.tmp ~/.cua-driver/packages/current-cua-driver --version # → 0.2.0-```--The `mv -Tf` is the rename(2) idiom: a concurrent `cua-driver` lookup sees either the old or new target, never an absent path.--**Windows — roll back to a specific version**--Re-run the install one-liner with `$env:CUA_DRIVER_RS_VERSION` pinned to the version you want active. The installer notices the per-version dir is already on disk, skips the download, and just retargets the `current` junction — so the round-trip is a single junction retarget under the hood.--```powershell-$env:CUA_DRIVER_RS_VERSION = "0.2.0"-irm https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.ps1 | iex-cua-driver --version # → 0.2.0-```--Clear `$env:CUA_DRIVER_RS_VERSION` and re-run to roll forward to the newest release.--#### Env-var overrides--| Env var | Default (Linux/macOS) | Default (Windows) | What it controls |-| ---------------------------------------------------- | ------------------------- | -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |-| `CUA_DRIVER_RS_VERSION` | unset → use baked version | unset → use baked version | Pin a specific release (e.g. `0.2.0`). |-| `CUA_DRIVER_RS_INSTALL_DIR` | `~/.local/bin` | `%LOCALAPPDATA%\Programs\Cua\cua-driver\bin` | The visible PATH-entry directory. On Windows this is itself a junction. |-| `CUA_DRIVER_RS_HOME` | `~/.cua-driver` | `%USERPROFILE%\.cua-driver` | Package home — holds `packages/releases/<v>/` and `packages/current`. (Renamed from `~/.cua-driver-rs` in v0.2.16; a stale legacy dir is auto-swept on the next install.) |-| `CUA_DRIVER_RS_NO_MODIFY_PATH` _(Linux/macOS only)_ | `0` | use `-NoPathUpdate` switch on `install.ps1` | Skip the PATH append. On Windows the installer appends `%LOCALAPPDATA%\Programs\Cua\cua-driver\bin` to the User-scope `Path` by default; use the fetch+invoke form with `-NoPathUpdate` to opt out (see the Windows install section above for the exact one-liner). |-| `CUA_DRIVER_RS_KEEP_VERSIONS` _(Linux/Windows only)_ | `5` | `5` | Keep the N most recent per-version release dirs after install (`0` disables GC). See the **Old-version cleanup** callout below for per-target and active-install invariants. |--<Callout type="info">-**How the installer picks a version.** Resolution order is:--1. `$CUA_DRIVER_RS_VERSION` (Linux/macOS) or `$env:CUA_DRIVER_RS_VERSION` (Windows) — explicit pin.-2. A **baked-in default version** carried in the install script itself, auto-updated by the release pipeline after every published `cua-driver-rs-v*` release. This is the common-case default for `curl … | bash` / `irm … | iex` against `main`: the version resolves locally with **zero network calls** to GitHub's API.-3. GitHub Releases API — fallback only, hit when the env override is absent **and** the baked line hasn't been updated yet (e.g. installing from a dev branch before its first release).--So an API outage, an unauthenticated-rate-limit (60 req/hr per IP), or a transient network blip never breaks a default install from `main`. Power users who want the very latest tip (rather than the baked default) can set `CUA_DRIVER_RS_VERSION=<x.y.z>` explicitly.--</Callout>--<Callout type="info">-**Why macOS uses a different layout.** On macOS the install still drops `CuaDriver.app` into `/Applications` and symlinks `~/.local/bin/cua-driver` into the bundle. The `.app` placement is the anchor for both **TCC attribution** (cdhash + bundle id) and **LaunchServices** (`open -a CuaDriver`); symlinking the `.app` from `/Applications` to a versioned dir under `$CUA_DRIVER_RS_HOME` would break both. The asymmetry is intentional — rollback on macOS = reinstall an older release tag with `CUA_DRIVER_RS_VERSION=<x.y.z>`.-</Callout>--<Callout type="info">-**Old-version cleanup (Linux / Windows).** Every install drops the binary into a fresh `packages/releases/<version>-<target>/` directory and retargets the `current` symlink/junction at it. To keep disk usage bounded, the installer prunes old per-version dirs after each install — by default it keeps the **5 most recent** per platform target, deleting the rest.--```bash-# Keep more (or fewer) versions on disk-CUA_DRIVER_RS_KEEP_VERSIONS=10 bash install.sh # keep 10 most recent-CUA_DRIVER_RS_KEEP_VERSIONS=0 bash install.sh # disable GC entirely-```--Two invariants worth knowing:--- **The active install is always preserved**, even if it would otherwise fall outside the keep window (e.g. you rolled back to an older version). Worst-case on-disk count is `keep + 1`.-- **Per-target filtering** — a multi-arch dev with both `aarch64-apple-darwin` and `x86_64-unknown-linux-gnu` directories under the same `$CUA_DRIVER_RS_HOME` (rare, but possible with shared/NFS homes) gets each target's history GC'd independently of the other.--The macOS install path is unaffected — `/Applications/CuaDriver.app` is an in-place replacement with no per-version accumulation.--</Callout>--<Callout type="info">-**Concurrent-install lockfile.** Running two installs at the same time — e.g. clicking the one-liner while a CI script is also installing, or two terminals racing — used to risk a partially-installed state if both hit the atomic `current` swap simultaneously. The installer now takes a process-level lock per `$CUA_DRIVER_RS_HOME` so the second install waits for the first to finish:--```-==> another cua-driver-rs install is already in progress (lock at- ~/.cua-driver/packages/.install.lock.d); waiting...-```--If you see this, the safe action is to let it block — it polls every 1 second and proceeds the moment the holding install finishes. The lock entry stamps the holder's pid, start time, and invocation args so you can confirm what's running:--```bash-cat ~/.cua-driver/packages/.install.lock.d/info-# pid=43210-# started=2026-05-17T09:14:22Z-# argv=install.sh-```--```powershell-Get-Content $env:CUA_DRIVER_RS_HOME\install.lock-# pid=43210-# started=2026-05-17T09:14:22.123Z-# invocation=install.ps1 -Release latest-```--**Stale-lock recovery.** If the holding process died without releasing (kill -9, host reboot mid-install, OOM), the lock would otherwise wedge every subsequent install. After **600 seconds** of waiting, the installer assumes the lock is stale, force-releases it with a loud `lock appears stale (>600s), forcing release` log, and proceeds. So even in the pathological case the worst recovery is a 10-minute wait, not a manual `rm -rf` on internal-looking paths.--Lock locations:--| Platform | Lock |-| -------- | ------------------------------------------------------------------- |-| Linux | `$CUA_DRIVER_RS_HOME/packages/.install.lock.d/` (mkdir-based mutex) |-| Windows | `$env:CUA_DRIVER_RS_HOME\install.lock` (`FileShare::None` mutex) |--Both primitives are unprivileged — no sudo, no admin elevation, no Developer Mode. The lock is released on every script exit path (success, error, Ctrl-C, `exit`).--</Callout>--### Verify it worked--```bash-cua-driver --version-# cua-driver 0.1.0--cua-driver --help-# OVERVIEW: macOS Accessibility-driven computer-use agent — MCP stdio server.-```--For a structured environment + install report on any platform, run:--```bash-cua-driver doctor-# [ok ] binary: cua-driver 0.2.1 (aarch64-macos)-# [ok ] install dir: /Users/you/.local/bin/cua-driver-# [ok ] home dir: /Users/you/.cua-driver (3 release dirs cached)-# [ok ] telemetry: enabled (install-id present)-# ...-```--`doctor` runs version + install-layout + telemetry checks on every platform, plus platform-specific probes (legacy LaunchAgent / updater cleanup on macOS, UI Automation + interactive desktop session on Windows, X11/Wayland + AT-SPI on Linux). For the detailed macOS TCC + cdhash report, run `cua-driver diagnose` — `doctor` only surfaces a pointer to it. Pass `--json` for a scripting-friendly shape. Exit code is `0` when every probe is `[ok]` / `[warn]`, non-zero only on `[err]` (e.g. the binary cannot resolve its own install dir).--### Windows: interactive-session requirements--Window-driving tools — `list_windows`, `click`, `type_text`, `screenshot`, `get_window_state` — all bottom out in Win32 APIs (`EnumWindows`, `GetForegroundWindow`, `PrintWindow`) that are scoped to the **calling process's WindowStation + Desktop**. A process without an attached interactive desktop sees zero windows even when the user's logged-in desktop has plenty.--**Windows session model in one paragraph.** Every Windows process runs inside a numbered session. Session `0` is reserved for services and has no interactive desktop attached. Sessions `1`, `2`, ... are interactive logons — one per console / RDP user. SSH-launched processes (OpenSSH server) **default to Session 0** unless the SSH server is explicitly configured to spawn the shell inside the interactive session. The same applies to most Windows service hosts, scheduled tasks running under SYSTEM, and background invocations from CI agents.--If `cua-driver` lands in Session 0, the window tools will silently return empty arrays and "no windows found" — that's the APIs working as designed against a session with no desktop, not a Cua Driver bug.--**Diagnose with `cua-driver doctor`.** The Windows session probe surfaces this directly:--```text-[warn] interactive session: running in Session 0 (services); window-driving- tools (list_windows, click, type_text, screenshot, get_window_state)- will return empty results — these APIs need an attached interactive- desktop.- re-run cua-driver from an interactive logon (RDP, console, or a- scheduled task in the user's session) for the GUI tools to function.-```--A healthy interactive logon reports the inverse:--```text-[ok ] interactive session: session 2 has an attached interactive desktop- (WinSta0 + foreground window)-```--**Common fixes for "all my tools return empty arrays on Windows".**--- **Run from RDP or the local console** — the simplest fix; both put your shell in your own interactive session, not Session 0.-- **Run from a Windows Terminal / PowerShell launched by your logged-in user** — same effect.-- **Register a logon Scheduled Task** (recommended, equivalent to macOS LaunchAgent) — see [Auto-start at logon](#auto-start-at-logon-windows) below, or the [Autostart concept page](/cua-driver/guide/getting-started/autostart) for the `cua-driver autostart enable` one-liner.-- **Driving over SSH?** Register the autostart task once from an interactive session, then `cua-driver mcp` and `cua-driver call` over SSH proxy through the running daemon automatically. See [Running Cua Driver under SSH on Windows](/cua-driver/guide/getting-started/windows-ssh) for the full workflow.-- **Use a session-launcher utility** — if you must invoke from a service / SSH context _without_ the daemon-proxy path, run the binary through a launcher that spawns it inside an explicit interactive session id rather than inheriting Session 0.--Run `cua-driver doctor` after switching contexts to confirm the warning has cleared.--#### Auto-start at logon (Windows)--The Windows-native equivalent of macOS's LaunchAgent is a **Scheduled Task triggered at logon, running as your user, with `Run only when user is logged on`** (i.e. `LogonType: Interactive`). Once registered, `cua-driver serve` starts automatically every time you sign in via RDP or the console — no more pasting a startup one-liner.--<Callout type="info">- **One-liner:** `cua-driver autostart enable && cua-driver autostart kick` does everything the- manual block below does. See the [Autostart concept- page](/cua-driver/guide/getting-started/autostart) for the full `enable / disable / kick / status`- verb family, and [Running Cua Driver under SSH on- Windows](/cua-driver/guide/getting-started/windows-ssh) for the recommended headless workflow on- top of it.-</Callout>--`install.ps1` prints the exact one-shot registration block on a successful install. To register it manually (substitute your install path if you built from source):--```powershell-$exe = "$env:LOCALAPPDATA\Programs\Cua\cua-driver\bin\cua-driver.exe"-$user = "$env:COMPUTERNAME\$env:USERNAME"-$action = New-ScheduledTaskAction -Execute $exe -Argument 'serve' -WorkingDirectory $env:USERPROFILE-$trigger = New-ScheduledTaskTrigger -AtLogOn -User $user-$principal = New-ScheduledTaskPrincipal -UserId $user -LogonType Interactive -RunLevel Limited-$settings = New-ScheduledTaskSettingsSet -AllowStartIfOnBatteries -DontStopIfGoingOnBatteries `- -StartWhenAvailable -RestartCount 3 -RestartInterval (New-TimeSpan -Minutes 1) `- -ExecutionTimeLimit (New-TimeSpan -Hours 0)-Register-ScheduledTask -TaskName 'cua-driver-serve' -Action $action -Trigger $trigger `- -Principal $principal -Settings $settings-```--Useful follow-ups:--```powershell-schtasks /Run /TN cua-driver-serve # start it now without re-logging-schtasks /Query /TN cua-driver-serve /V /FO LIST-schtasks /Delete /TN cua-driver-serve /F # remove-```--Notes:--- `LogonType: Interactive` is load-bearing — it pins the task to a Session 1+ logon. The alternative (`S4U` or `Password`) would land you in Session 0 again, undoing the point.-- `-ExecutionTimeLimit (New-TimeSpan -Hours 0)` means "no time limit"; without it Windows kills the task after 72h by default.-- Set `CUA_DRIVER_RS_TELEMETRY_ENABLED=0` in your User environment if you want to disable telemetry persistently: `[Environment]::SetEnvironmentVariable('CUA_DRIVER_RS_TELEMETRY_ENABLED', '0', 'User')`.--## Grant TCC permissions--Cua Driver needs two permissions:--- **Accessibility** — to walk AX trees and dispatch `AXUIElementPerformAction`.-- **Screen Recording** — to capture per-window screenshots via ScreenCaptureKit.--Start the daemon first so TCC attributes the subsequent requests to `CuaDriver.app` rather than to whatever shell parent launched the CLI:--```bash-open -n -g -a CuaDriver --args serve-```--Then trigger the prompts:--```bash-cua-driver check_permissions-```--macOS raises the Accessibility and Screen Recording system dialogs. Grant both, then re-run the command to confirm:--```bash-cua-driver check_permissions-# ✅ Accessibility: granted.-# ✅ Screen Recording: granted.-```--<Callout type="info">- `check_permissions` reports the TCC status of the *calling* process. Inside an IDE terminal- (Claude Code, Cursor, VS Code, Conductor) the shell inherits the IDE's TCC responsibility chain,- so results can read "NOT granted" even when you've granted both to `CuaDriver.app`. The- daemon-first recipe above sidesteps this because the CLI forwards through the daemon, which runs- under `CuaDriver.app`'s bundle id.-- **Prefer the `permissions` verbs.** `cua-driver permissions grant` launches CuaDriver via- LaunchServices so the dialog attributes to `com.trycua.driver` (not your terminal), then waits for- and confirms the grant — the one-step authoritative path. `cua-driver permissions status` reads- the driver's real grant state *through the daemon*; when no daemon is running it reports- `❓ unknown` rather than your terminal's grants, so it can never claim `granted` when the driver- has none.-</Callout>--If a grant still reads `NOT granted` after granting in the dialog, open **System Settings → Privacy & Security**, find `CuaDriver.app` under Accessibility and Screen Recording, and flip the toggle.--<Callout type="info">- **First-launch permissions gate (`cua-driver serve`).** On the Rust port,- `cua-driver serve` runs an interactive permissions gate at startup. If- Accessibility or Screen Recording is missing it prints a clear terminal- banner, opens the matching System Settings pane(s), and polls TCC every- second until you grant the missing items. When both grants are already- active the gate is a transparent no-op.--**Experimental native panel.** A `CuaDriver Permissions` NSPanel with-live status rows + auto-dismiss is also available behind an opt-in env-var. The terminal flow is the safer default because the panel relies on-a chain of macOS behaviours that are easy to break in dev environments-(TCC responsible-process attribution, session-level TCC caches that-survive `tccutil reset`, ad-hoc codesign identity mismatches, AppKit-modal-run-loop modes). Opt in with:--```bash-CUA_DRIVER_RS_PERMISSIONS_PANEL=1 cua-driver serve-```--Accepted "on" values (case-insensitive): `1`, `true`, `yes`, `on`. Any-other value (or unset) keeps the terminal flow. The panel also requires-the daemon to be launched from the bundled `.app` (e.g.-`open -n -g -a CuaDriver --args serve`); bare-binary launches always-use the terminal flow regardless of the env var.--**CI / headless runners** should skip the entire gate so the daemon-does not block waiting for a TTY-attached human:--```bash-# As a flag …-cua-driver serve --no-permissions-gate--# … or as an env-var.-CUA_DRIVER_RS_PERMISSIONS_GATE=0 cua-driver serve-```--Accepted "off" values for the gate env-var (case-insensitive): `0`,-`false`, `no`, `off` — so `FALSE`, `Off`, `NO` etc. all work. Any-other value (including unset) leaves the gate active.--</Callout>--<Callout type="info">- **Update-available banner (`cua-driver mcp` / `serve` / `doctor`).** The- long-running interactive entry points kick off a non-blocking background- check against the GitHub releases API at startup and print a small- two-line banner to stderr when a newer `cua-driver-rs-v*` release exists:--```-✨ cua-driver v0.1.4 is available (you have v0.1.3).- Update with: cua-driver update- Release notes: https://github.com/trycua/cua/releases/tag/cua-driver-rs-v0.1.4-```--The answer is cached at `~/.cua-driver/version_check.json` for ~20-hours, so subsequent launches reuse the cached result without a network-call. Network failures are silent — the next launch retries.--**Scripted / machine-readable entry points are NOT instrumented.**-The banner only fires from the long-running entry points (`mcp`,-`serve`, `doctor`). One-shot and machine-readable subcommands —-`--version`, `list-tools`, `describe`, `call`, `dump-docs`,-`mcp-config`, `update`, `stop`, `status`, `recording`, `config`,-`diagnose`, and `telemetry install-event` — all skip it so piped-output (`cua-driver list-tools | jq …`) stays clean.--**Disable per-invocation:**--```bash-CUA_DRIVER_RS_UPDATE_CHECK=false cua-driver serve-```--Accepted "off" values (case-insensitive): `0`, `false`, `no`, `off`.--**Disable permanently** via the persisted config:--```bash-cua-driver config set update_check_enabled false-```--Source / dev builds (any `CARGO_PKG_VERSION` with a pre-release suffix-like `-dev`) auto-skip the check — there is no matching published-release for them to recommend.--</Callout>--## Requirements--| Platform | Requirements |-| ----------------- | ---------------------------------------------------------------------------------------------------------- |-| macOS | macOS 14 (Sonoma) or later; Apple Silicon or Intel; Accessibility and Screen Recording grants. |-| Linux | x86_64 Linux desktop session; X11 or XWayland; AT-SPI for accessibility-tree tools. |-| Windows | Windows 10/11 or Server with an interactive desktop session; PowerShell; UI Automation enabled by default. |--Plan for roughly 50 MB for the app bundle or package directory, plus any retained old versions on Linux / Windows.--## Run the daemon--Most workflows benefit from a persistent daemon. Element-indexed workflows require one: the per-pid element cache lives in-process, so one-shot CLI invocations lose it between calls.--**macOS**--```bash-open -n -g -a CuaDriver --args serve-```--**Linux**--```bash-cua-driver serve &-```--**Windows**--```powershell-cua-driver autostart enable-cua-driver autostart kick-```--Confirm it is up:--```bash-cua-driver status-# Cua Driver daemon is running-# socket: /Users/you/Library/Caches/cua-driver/cua-driver.sock-# pid: 12345-```--Stop it cleanly when done:--```bash-cua-driver stop-```--On macOS, `open -n -g -a CuaDriver --args serve` is the recommended form because LaunchServices attributes the process to `CuaDriver.app`'s bundle id, which is what the user actually granted TCC against. `cua-driver serve &` also works and auto-relaunches itself via `open` when it detects the wrong TCC context.--## Register with an MCP client (optional)--The MCP server is one of two ways to use Cua Driver — skip this section if you only want the CLI (`cua-driver list_apps`, etc.). You can register both later, MCP and CLI work side-by-side.--Cua Driver speaks MCP over stdio. Use `cua-driver mcp-config --client <name>` to print the right command for your client, or copy one of the snippets below directly:--### Clients with a CLI add command--```bash-# Claude Code (add --scope project|global as needed)-claude mcp add --transport stdio cua-driver -- ~/.local/bin/cua-driver mcp--# Claude Code computer-use compatibility mode-claude mcp add --transport stdio cua-computer-use -- ~/.local/bin/cua-driver mcp --claude-code-computer-use-compat--# Codex (OpenAI)-codex mcp add cua-driver -- ~/.local/bin/cua-driver mcp--# OpenClaw-cua-driver mcp-config --client openclaw | sh-```--The Claude Code compatibility mode keeps CuaDriver's normal MCP tools, but replaces `screenshot` with a window-only screenshot shim that requires `pid` and `window_id`. Use it when you want Claude Code's vision/computer-use-style flow to ground on CuaDriver window captures.--Use MCP for that Claude Code vision/computer-use-style path. Shelling out to `cua-driver screenshot` can capture a window, but it does not expose the `mcp__cua-computer-use__screenshot` tool name that Claude Code appears to use as the image-grounding cue.--### Clients configured via a config file--Cursor, OpenCode, and Hermes all configure MCP servers via files. Use `mcp-config` to print the exact snippet, paste it into the right path:--```bash-# Cursor — paste into ~/.cursor/mcp.json (or .cursor/mcp.json for project scope)-cua-driver mcp-config --client cursor | pbcopy--# OpenCode (sst/opencode) — paste into opencode.json at the project root-cua-driver mcp-config --client opencode | pbcopy--# Hermes (NousResearch) — paste into ~/.hermes/config.yaml,-# then run `/reload-mcp` inside Hermes-cua-driver mcp-config --client hermes | pbcopy-```--For any other client that accepts the standard `mcpServers` JSON shape:--```bash-cua-driver mcp-config | pbcopy-```--<Callout type="info">-**Pi (badlogic/pi-mono)** does not support MCP natively. Use cua-driver as a plain CLI from inside Pi instead — `cua-driver list_apps`, `cua-driver click '{...}'`, etc. — which is exactly the shape Pi is built around. Run `cua-driver mcp-config --client pi` for the full guidance, or see the [Quickstart](/cua-driver/guide/getting-started/quickstart) for CLI usage.-</Callout>--The client spawns `cua-driver mcp` on demand once registered.--<Callout type="info">- **TCC auto-delegation.** When MCP clients spawn `cua-driver mcp` from an IDE terminal (Claude- Code, Cursor, VS Code, Warp), macOS attributes the subprocess to the terminal — not- `CuaDriver.app` — so AX probes would silently fail against the wrong bundle id. `mcp` detects this- and auto-launches a `cua-driver serve` daemon via `open -n -g -a CuaDriver --args serve`, then- proxies every tool call through the daemon's Unix socket. The client sees an ordinary stdio MCP- server; the daemon runs under LaunchServices with the right TCC context. No Python bridge, no- manual `serve` step required. Pass `--no-daemon-relaunch` (or set `CUA_DRIVER_MCP_NO_RELAUNCH=1`)- to opt out, e.g. when MCP is launched from CuaDriver.app directly and already has the right TCC- grants.-</Callout>--<Callout type="info">- **Daemon-proxy on Windows / Linux (`cua-driver-rs` v0.2.7+).** The same proxy mechanism applies on- Windows and Linux, addressing the Session 0 / interactive-session problem rather than TCC: when- `cua-driver mcp` starts up and detects a daemon already listening on the default socket, it- proxies every tool call through to it instead of running in-process. Crucial for SSH on Windows,- where the SSH session lands in Session 0 (no desktop) but the daemon — registered via `cua-driver- autostart enable` — lives in the user's Session 1+ interactive logon. Same `--no-daemon-relaunch`- / `CUA_DRIVER_RS_MCP_NO_RELAUNCH=1` opt-out as macOS. See [Running Cua Driver under SSH on- Windows](/cua-driver/guide/getting-started/windows-ssh) for the full workflow.-</Callout>--## Agent skills (optional)--The installer does not modify agent instruction directories by default. To fetch the matching Anthropic-format skill pack and link it into detected agents, run:--```bash-cua-driver skills install-```--| Agent | Skills directory | Linked by `skills install` when present |-| ----------- | ---------------------------------------------------------------------------------------------------------- | --------------------------------------- |-| Claude Code | `~/.claude/skills/cua-driver` | yes |-| Codex | `~/.agents/skills/cua-driver` | yes |-| OpenClaw | `~/.openclaw/skills/cua-driver` | yes |-| OpenCode | `~/.config/opencode/skills/cua-driver` on macOS / Linux, `%APPDATA%\opencode\skills\cua-driver` on Windows | yes |-| Antigravity | `~/.gemini/skills/cua-driver` | yes |--`skills install` never overwrites an existing user-managed `cua-driver` link or directory. Use `cua-driver skills status`, `cua-driver skills update`, and `cua-driver skills uninstall` to inspect, refresh, or remove the pack.--### Agents without a SKILL.md auto-loader--Cursor, Hermes, and Pi each use a single-file system prompt (different format) instead of a skills directory. We don't auto-wire these because:--- **Cursor** (`~/.cursor/rules/*.mdc`): rules expect a different frontmatter shape (`description`/`globs`/`alwaysApply`), not Anthropic's `name`/`description`. Symlinking the SKILL.md as `.mdc` works but the frontmatter renders as a noisy comment block.-- **Hermes** (`~/.hermes/SOUL.md`) and **Pi** (`~/.pi/agent/SYSTEM.md`): replace the entire system prompt — symlinking would clobber any custom prompt you've set.--For these, install the skill pack and paste its content manually into whatever instructions file the agent uses:--```bash-cua-driver skills install-cat "$(cua-driver skills path)/SKILL.md" | pbcopy-```--## Uninstall--One canonical uninstall URL per platform mirrors the install side:--| Platform | Command |-| -------- | ---------------------------------------------------------------------------------------------------------------- |-| macOS | `bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/uninstall.sh)"` |-| Linux | `bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/uninstall.sh)"` |-| Windows | `irm https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/uninstall.ps1 \| iex` |--`uninstall.sh` removes the Rust implementation by default on macOS and Linux. A legacy `--backend=swift` path remains for users explicitly removing an old Swift install.--`uninstall.ps1` self-elevates via UAC when it detects an `-AutoStart` install — the `cua-driver-serve` Scheduled Task is registered at `RunLevel=Highest` (see [Autostart](../autostart)), so the daemon spawned by it runs at High IL and a non-elevated process (even the same user that installed it) can't terminate it or delete the task. The script detects either condition upfront and re-spawns itself elevated; the in-place path runs without UAC when no autostart artifacts are present.--**What each uninstall removes**--| Platform | Symlink / bin | App bundle / package home | Autostart entry | Skill links | Claude MCP registrations |-| -------- | -------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- |-| macOS | `~/.local/bin/cua-driver` _(when it resolves into `CuaDriver.app` and a Rust marker exists)_ | `/Applications/CuaDriver.app` (current), `/Applications/CuaDriverRs.app` (legacy), `~/.cua-driver/` (legacy `~/.cua-driver-rs/` also swept) | `~/Library/LaunchAgents/com.trycua.cua-driver-rs.plist` | `cua-driver` and legacy `cua-driver-rs` links under agent skill dirs | Scrubbed from `~/.claude.json` |-| Linux | `~/.local/bin/cua-driver` _(when it resolves into `~/.cua-driver/`)_ | `~/.cua-driver/` (legacy `~/.cua-driver-rs/` also swept) | `~/.config/systemd/user/cua-driver-rs.service` (stop + disable + remove) | Same as macOS | Scrubbed from `~/.claude.json` |-| Windows | `%LOCALAPPDATA%\Programs\Cua\cua-driver\bin` _(directory junction; legacy `Programs\trycua\cua-driver-rs\bin` also swept)_ | `%USERPROFILE%\.cua-driver\` (entire tree, including `packages\current` junction; legacy `~\.cua-driver-rs\` also swept) | Scheduled Task `cua-driver-serve` (`schtasks /Delete`; script self-elevates when needed) | Junctions under `%USERPROFILE%\.claude\skills`, `.agents\skills`, `.openclaw\skills`, `%APPDATA%\opencode\skills`, `.gemini\skills` | Not auto-edited — closing message prints the manual `claude mcp remove` command |--<Callout type="info">- **Safety invariants** — every uninstall script (`uninstall.sh`, `uninstall.ps1`) refuses to- clobber a real directory at a path where the matching installer could have only created a symlink- / junction (Linux/macOS use `[[ -L ]]`; Windows checks `IO_REPARSE_TAG_MOUNT_POINT`). A user who- hand-managed any of these dirs keeps theirs untouched. Re-running an uninstall on an already-clean- system prints `nothing to remove` per item — never errors.-</Callout>--<Callout type="info">-**TCC grants persist on macOS** (Accessibility + Screen Recording). The uninstaller leaves them in **System Settings → Privacy & Security** so a re-install doesn't have to re-prompt. To reset them explicitly:--```bash-# Both Swift and Rust drivers share bundle id `com.trycua.driver`-# (the Rust port took over Swift's identity in cua-driver-rs ≥ 0.2.4).-tccutil reset Accessibility com.trycua.driver-tccutil reset ScreenCapture com.trycua.driver-```--Pre-rename Rust installs (`cua-driver-rs ≤ 0.2.3`) used a separate bundle id `com.trycua.cuadriverrs` — clean those up if you had one of those installs:--```bash-tccutil reset Accessibility com.trycua.cuadriverrs 2>/dev/null-tccutil reset ScreenCapture com.trycua.cuadriverrs 2>/dev/null-```--</Callout>--<Callout type="warn">-**Windows is interactive by default.** `uninstall.ps1` prompts before each major delete so a stray paste doesn't wipe a working install. For non-interactive / scripted teardowns, pipe through a `-Force` arg:--```powershell-& ([scriptblock]::Create((irm https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/uninstall.ps1))) -Force-```--It also doesn't auto-edit `%USERPROFILE%\.claude.json` — the closing message prints the manual `claude mcp remove` command instead. The macOS / Linux uninstaller does auto-edit `~/.claude.json` when `python3` is on `PATH`; the Windows uninstaller skips the auto-edit unconditionally to keep the script dependency-free.--</Callout>--## Troubleshooting--**`cua-driver: command not found`** — `~/.local/bin` isn't on your PATH. Add it and reload (see the first-time install callout above).--**Permissions dialogs reappear after every launch** — macOS is attributing the process to a different bundle id than the one you granted. Run `cua-driver diagnose` and paste the output when filing an issue; it reports cdhash, team id, and which bundle TCC is checking against.--**Daemon won't start** — another daemon may already be bound to the socket. Check with `cua-driver status` and stop it with `cua-driver stop`. For stale lock files after a crash, the daemon's own probe detects those and proceeds.--Ready to drive an app? Head to the [Quickstart](/cua-driver/guide/getting-started/quickstart).--After install, see [Updating](/cua-driver/guide/getting-started/updating) for how to check for and apply new releases.diff --git a/docs/content/docs/cua-driver/guide/getting-started/integrations.mdx b/docs/content/docs/cua-driver/guide/getting-started/integrations.mdxdeleted file mode 100644index 8eb7b84d12..0000000000--- a/docs/content/docs/cua-driver/guide/getting-started/integrations.mdx+++ /dev/null@@ -1,199 +0,0 @@-----title: Integrations-description: Connect Cua Driver to your AI coding agent------import { Callout } from 'fumadocs-ui/components/callout';--`cua-driver mcp` is a stdio MCP server. Any agent that supports MCP can use it — no extra setup beyond adding it to the agent's MCP config.--<Callout type="warn">- Grant the required permissions before connecting any agent — on macOS that's Accessibility and Screen- Recording (Windows and Linux have no equivalent prompt). Run `cua-driver call check_permissions` to verify.-</Callout>--## Claude Code--Standard MCP registration:--```bash-claude mcp add --transport stdio cua-driver -- cua-driver mcp-```--Verify:--```bash-claude mcp list-# cua-driver: cua-driver mcp (stdio) - ✓ Connected-```--### Claude Code computer-use compatibility mode--Claude Code vision/computer-use-style flows appear to use the presence of a screenshot tool as a cue for image-grounded operation. If you want that behavior, register the compatibility server instead:--```bash-claude mcp add --transport stdio cua-computer-use -- cua-driver mcp --claude-code-computer-use-compat-```--This mode still exposes the normal CuaDriver tools. The only changed tool is `screenshot`: it requires `pid` and `window_id`, captures that window only, and returns a window-local image coordinate frame. Start with `launch_app` or `list_windows`, then call `screenshot` with the target window.--For this Claude Code vision/computer-use-style path, use MCP rather than shelling out to the CLI. CLI screenshots can still capture windows, but they do not expose the `mcp__cua-computer-use__screenshot` tool name that Claude Code appears to use as the image-grounding cue.--<Callout type="info">- This does not call Anthropic APIs or expose Anthropic's native computer-use API tool. It is a- CuaDriver MCP compatibility mode for Claude Code.-</Callout>--## GitHub Copilot CLI--Add to `~/.copilot/mcp-config.json`:--```json-{- "mcpServers": {- "cua-driver": {- "type": "local",- "command": "cua-driver",- "args": ["mcp"],- "tools": ["*"]- }- }-}-```--Or interactively inside `gh copilot` chat:--```-/mcp add-```--Fill in: name=`cua-driver`, type=STDIO, command=`cua-driver`, args=`mcp`. Press **Ctrl+S** to save.--## Codex (OpenAI)--```bash-codex mcp add cua-driver -- cua-driver mcp-```--## Cursor--Generate the config snippet and paste it into `~/.cursor/mcp.json`:--```bash-cua-driver mcp-config --client cursor-```--## Gemini CLI--Add to `~/.gemini/settings.json`:--```json-{- "mcp": {- "servers": {- "cua-driver": {- "type": "stdio",- "command": "cua-driver",- "args": ["mcp"]- }- }- }-}-```--Tools appear prefixed as `mcp_cua-driver_*`.--## OpenCode--```bash-cua-driver mcp-config --client opencode-```--Paste the output into `~/.config/opencode/config.json` (global) or `opencode.json` at the project root.--<Callout type="warn">- **Always configure Cua Driver as an MCP server — never rely on the CLI fallback.** If MCP is not- wired up, OpenCode calls `cua-driver` as a shell subprocess. The `get_window_state` response no- longer includes base64 by default, but the screenshot image block is silently dropped — the model- receives only the AX tree with no visual context. Use `--screenshot-out-file` or the- `screenshot_out_file` param to preserve the image when using the CLI path.-</Callout>--### Local vision models (Ollama)--If you are using a vision-capable model via Ollama, you must also declare its input modalities in `config.json` — otherwise OpenCode strips images before they reach the model:--```json-{- "mcp": {- "cua-driver": {- "type": "local",- "command": ["/Users/you/.local/bin/cua-driver", "mcp"],- "enabled": true- }- },- "provider": {- "ollama": {- "npm": "@ai-sdk/openai-compatible",- "options": { "baseURL": "http://localhost:11434/v1" },- "models": {- "llama3.2-vision": {- "modalities": {- "input": ["text", "image"],- "output": ["text"]- }- }- }- }- }-}-```--The `modalities` field is required because OpenCode's `@ai-sdk/openai-compatible` provider defaults to text-only when no capabilities are declared. Without it, screenshots are replaced with an error string and never reach the model.--## Hermes (NousResearch)--```bash-cua-driver mcp-config --client hermes-```--Paste the output into `~/.hermes/config.yaml`.--## OpenClaw--```bash-cua-driver mcp-config --client openclaw-```--## Antigravity CLI (and Antigravity IDE)--Google's [Antigravity CLI](https://developers.googleblog.com/an-important-update-transitioning-gemini-cli-to-antigravity-cli/) — the `agy` binary that replaced Gemini CLI on May 19, 2026 — reads MCP server configs from `~/.gemini/config/mcp_config.json` (Unix) or `%USERPROFILE%\.gemini\config\mcp_config.json` (Windows). The same file is read by Antigravity IDE.--```bash-cua-driver mcp-config --client antigravity-```--Output is a snippet to paste under the top-level `mcpServers` object in that file. If the file doesn't exist yet, create it with `{"mcpServers": {}}` first. Antigravity CLI has no in-shell `mcp add` subcommand — edit the JSON file directly, then restart `agy` to pick up the change.--`--client gemini` is accepted as a legacy alias and emits the same instructions — Antigravity inherited the `~/.gemini/` config tree from the old Gemini CLI install on purpose, so existing setups carry over without re-registration.--## Other clients--For any client that accepts the standard `mcpServers` shape:--```bash-cua-driver mcp-config-```--Output:--```json-{- "mcpServers": {- "cua-driver": {- "command": "cua-driver",- "args": ["mcp"]- }- }-}-```diff --git a/docs/content/docs/cua-driver/guide/getting-started/introduction.mdx b/docs/content/docs/cua-driver/guide/getting-started/introduction.mdxdeleted file mode 100644index 03167aab62..0000000000--- a/docs/content/docs/cua-driver/guide/getting-started/introduction.mdx+++ /dev/null@@ -1,67 +0,0 @@-----title: What is Cua Driver?-description: Background computer-use driver for any agent on macOS, Windows, and Linux pre-release.------import { Callout } from 'fumadocs-ui/components/callout';--Cua Driver is a computer-use driver that speaks MCP over stdio and exposes the same tools as a direct CLI. It lets agents click, type, scroll, inspect accessibility trees, and capture window state on macOS and Windows while preserving the user's working context. A Linux backend is available as a pre-release target while platform testing is still in progress.--<Callout type="info" title="MIT License">- Cua Driver is open-source and MIT licensed. If you find it useful, we'd appreciate a [star on- GitHub](https://github.com/trycua/cua)!-</Callout>--```bash-cua-driver launch_app '{"bundle_id":"com.apple.calculator"}'-cua-driver get_window_state '{"pid":844,"window_id":10725}'-cua-driver click '{"pid":844,"window_id":10725,"element_index":14}'-```--A single binary can run as an MCP stdio server, a long-running daemon, or a one-shot shell command.--## The no-foreground contract--The user's frontmost app should not change. Not during launch, not during a click, not during a keystroke, not during a re-snapshot. Three corollaries follow:--- The real cursor stays where the user left it. No warp.-- The target window stays at its current z-rank unless a platform requires explicit activation for a foreground-only fallback.-- The user's desktop or Space does not follow the target.--Every dispatch path is designed around those invariants. On macOS that means scoped CoreGraphics, Accessibility, and SkyLight paths such as `CGEvent.postToPid`, `AXUIElementPerformAction`, and pid-routed mouse delivery. On Windows it means UI Automation, window-targeted messages, and `SendInput` only when the OS requires a foreground dispatch path. On Linux it means AT-SPI for element addressing and X11/XWayland for window state, capture, and input delivery.--## Three modalities--`capture_mode` controls what `get_window_state` returns. Pick based on what the agent needs:--- **`som`** (default) — set-of-mark. Accessibility tree plus screenshot. Tree for dispatch, screenshot for visual disambiguation when labels repeat or stay empty.-- **`ax`** — accessibility tree only. No screen-capture cost, deterministic element addressing. Best for structured loops over apps with real accessibility coverage.-- **`vision`** — window image only. No accessibility walk. Best for vision-first models that ground on pixels and don't use `element_index`.--Backgrounded drive is the default across all three, not a mode you toggle. Switch modalities with `cua-driver config set capture_mode som`.--## Platform backends--- **macOS** uses Accessibility, ScreenCaptureKit, LaunchServices, and a signed `CuaDriver.app` bundle so TCC grants attach to the driver instead of the parent terminal.-- **Windows** uses UI Automation, Win32 window APIs, named pipes, and an optional logon Scheduled Task so GUI tools run in the user's interactive session rather than Session 0.-- **Linux pre-release** uses AT-SPI, X11/XWayland, XTest-style input, and normal user-session processes. Native Wayland-only apps remain a known platform limit because Wayland intentionally blocks cross-app automation. Treat this backend as pre-release until Linux testing and release coverage are complete.--The tool names stay stable across platforms. A flow that uses `list_windows`, `get_window_state`, `click`, `type_text`, and `hotkey` can keep the same shape while the backend handles each OS's process, permission, and input model. Linux follows the same surface, but it is not yet an officially released or fully tested support tier.--## Who it's for--- **Agent builders** who want a local desktop driver with MCP and shell entry points.-- **Dev-loop automation** where an agent drives an app, reads pixels or accessibility state, edits source, rebuilds, and verifies while the editor stays usable.-- **Remote Windows workflows** where the daemon runs in the interactive session and SSH-side MCP clients proxy through it.-- **Trajectory capture** for replay, inspection, and training data.--## What it doesn't do--- Not a VM or sandbox. Cua Driver operates the real host, so grant Accessibility, Screen Recording, UI Automation, and desktop-session access with intent.-- Not a Wayland security bypass. Linux apps that bypass XWayland may be invisible to the current backend.-- Canvas, game-engine, and GPU-heavy apps can require a foreground-only fallback on some platforms because their event loops filter backgrounded synthetic input.-- macOS off-Space SwiftUI windows can strip their accessibility tree by OS design. See [Limits](/cua-driver/reference/limits).--## Get started--Ready to try it? [Install Cua Driver](/cua-driver/guide/getting-started/installation) and drive your first app in the [Quickstart](/cua-driver/guide/getting-started/quickstart).diff --git a/docs/content/docs/cua-driver/guide/getting-started/linux.mdx b/docs/content/docs/cua-driver/guide/getting-started/linux.mdxdeleted file mode 100644index 565f5f130b..0000000000--- a/docs/content/docs/cua-driver/guide/getting-started/linux.mdx+++ /dev/null@@ -1,191 +0,0 @@-----title: Linux-description: Display server (X11 / Wayland), AT-SPI prerequisites, autostart, and headless workflows for the Linux backend.------import { Callout } from 'fumadocs-ui/components/callout';--The Linux backend drives **X11** apps in the background — clicks and keystrokes are injected to the target window via AT-SPI and `XSendEvent`, without raising it, activating it, or moving the real pointer (the same no-foreground contract as macOS and Windows). It runs the full tool surface: click, type, scroll, screenshot, recording, `launch_app`, and window enumeration. The implementations live under [`crates/platform-linux/`](https://github.com/trycua/cua/tree/main/libs/cua-driver/rust/crates/platform-linux/src).--Native **Wayland** input and capture are opt-in (`CUA_DRIVER_RS_ENABLE_WAYLAND=1`) and still maturing; on Wayland sessions the default path is the XWayland compatibility server (see **Display server** below).--For the install one-liner itself, see [Installation](/cua-driver/guide/getting-started/installation) — the canonical script auto-detects Linux and routes to the Rust backend.--## Display server: X11 vs Wayland vs XWayland--Cua Driver's window enumeration, input synthesis, and screen capture talk to your display server directly. The Linux backend targets **X11** (including the XWayland compatibility server on Wayland sessions), and probes which one you're on at startup:--```bash-cua-driver doctor-```--The `display server` probe surfaces:--- **`DISPLAY` set, `WAYLAND_DISPLAY` unset** — pure X11 session. This is the expected path for the Linux backend.-- **`WAYLAND_DISPLAY` set, `DISPLAY` set** — Wayland session with XWayland. cua-driver talks to XWayland (treats the session as X11). Native-Wayland-only apps that bypass XWayland aren't visible to the tools — call this out as a known limitation.-- **`WAYLAND_DISPLAY` set, `DISPLAY` unset** — pure Wayland session, no XWayland. Unsupported by default. There is an **experimental native-Wayland backend** (toplevel enumeration + virtual-pointer / virtual-keyboard input on wlroots compositors like sway and labwc) behind an opt-in flag — see [Experimental native Wayland](#experimental-native-wayland). Otherwise, run an X11 session or install / enable XWayland.-- **Neither set** — headless. Window-driving tools will return errors. See [Headless workflow](#headless-workflow) for `Xvfb`.--<Callout type="warn">- **Wayland caveat.** Wayland's security model deliberately isolates apps from each other's window- state and input — exactly what cua-driver needs to do its job. The Rust port works around this by- talking to XWayland, which acts as an X11 proxy for Wayland clients. **Apps that render natively- against Wayland** (no XWayland) — most modern Firefox builds, GTK4 apps, etc. — won't show up in- `list_windows` and aren't clickable. If your target app is one of those, run an X11 session (most- distros let you pick X11 vs Wayland at the login screen).-</Callout>--### Experimental native Wayland--A native-Wayland backend is shipping behind an opt-in flag. It enumerates toplevels via `zwlr_foreign_toplevel_manager_v1` and injects input via the wlroots virtual-pointer / virtual-keyboard protocols, so on **wlroots compositors** (sway, labwc, Hyprland, …) it can `list_windows` and drive input without XWayland. Screen capture and full AT-SPI parity are still landing, so it is **off by default** and a pure-Wayland session reports as unsupported until you opt in.--Enable it by starting the daemon with the flag set:--```bash-CUA_DRIVER_RS_ENABLE_WAYLAND=1 cua-driver serve-```--`cua-driver doctor` reflects the state — a pure-Wayland session shows the backend as `OFF` (with the flag to set) or `ENABLED`. Treat it as experimental: report gaps rather than relying on it in production.--## AT-SPI prerequisite--The accessibility-tree tools (`get_window_state`, `click` with `element_index`, anything that walks UIA-equivalent structure) talk to the **AT-SPI 2** D-Bus bus. AT-SPI is the Linux accessibility framework — same role as macOS's AX or Windows' UI Automation. Cua Driver expects the AT-SPI bus to be reachable on your session bus.--cua-driver probes this at startup via `gdbus`:--```bash-cua-driver doctor-# [ok ] AT-SPI bus: org.a11y.Bus reachable on session bus-# ...or, if not:-# [warn] AT-SPI bus: org.a11y.Bus not reachable-# install at-spi2-core (or equivalent) and ensure your desktop is-# running with accessibility enabled-```--If the probe fails, install the AT-SPI service and enable accessibility:--| Distro | Package | Enable |-| ---------------- | -------------------------------------------------- | ---------------------------------------------------------------------- |-| Ubuntu / Debian | `at-spi2-core` (usually pre-installed under GNOME) | `gsettings set org.gnome.desktop.interface toolkit-accessibility true` |-| Fedora | `at-spi2-core` | same `gsettings` command |-| Arch | `at-spi2-core` (extra) | `gsettings set …` if running GNOME; KDE has its own toggle |-| Alpine / minimal | `at-spi2-core dbus dbus-x11` | run `dbus-launch --exit-with-session` from your X startup script |--You don't need accessibility enabled _system-wide_ — it needs to be on for the user session that's running Cua Driver. Most desktop environments toggle this automatically when an accessibility client connects.--### Electron, Chromium, and Chrome apps--Chromium-based apps — Chrome, and everything built on Electron or CEF (VS Code, Slack, Discord, Obsidian, including the AppImage builds) — keep their accessibility tree switched off until they detect that an assistive technology is present, so by default `get_window_state` would see an empty tree for them. When you run `cua-driver serve`, the daemon flips the session's `org.a11y.Status` flags on (the same signal a screen reader sets), which makes Chromium build its full AT-SPI tree — retroactively, for apps already open, with no relaunch and no per-app flag. GTK and Qt read the same signal, so their bridges warm up too.--This is on by default. To leave your session's accessibility status untouched, start the daemon with `CUA_DRIVER_RS_DISABLE_A11Y_ADVERTISE=1`.--<Callout type="info">- **Why AT-SPI and not raw X11 properties?** Many Linux apps don't expose their UI structure through- X11 window manager hints — only through AT-SPI. Without AT-SPI, `click` would have to fall back to- raw pixel coordinates from screenshots (the same vision-only mode that works as a fallback). With- AT-SPI, Cua Driver can address elements by their accessible name / role / index, matching the- agent ergonomics on macOS (AX) and Windows (UIA).-</Callout>--## Autostart on Linux--The `cua-driver autostart` verb family is **Windows-only today**. On Linux it currently returns:--```text-cua-driver autostart is currently Windows-only. macOS users: see-libs/cua-driver/scripts/install-local.sh --autostart for the-LaunchAgent recipe. Linux users: same script registers a systemd---user unit. A cross-platform impl is tracked as a follow-up.-```--The two working alternatives:--### Option A — Use `install-local.sh --autostart`--The dev-loop install script registers a systemd user unit when invoked with `--autostart`. See [`libs/cua-driver/scripts/install-local.sh`](https://github.com/trycua/cua/blob/main/libs/cua-driver/scripts/install-local.sh) — built for local development off a git checkout, but the systemd unit it writes is the same shape you'd use in production.--### Option B — Write your own systemd user unit--Save the following to `~/.config/systemd/user/cua-driver.service`:--```ini-[Unit]-Description=cua-driver background daemon-# Wait for the graphical session (DISPLAY / WAYLAND_DISPLAY will be set).-After=graphical-session.target-PartOf=graphical-session.target--[Service]-Type=simple-ExecStart=%h/.local/bin/cua-driver serve-Restart=on-failure-RestartSec=2--[Install]-WantedBy=graphical-session.target-```--Then enable + start:--```bash-systemctl --user daemon-reload-systemctl --user enable --now cua-driver.service-systemctl --user status cua-driver.service # confirm-cua-driver status # confirm via daemon socket-```--On most distros you'll also want `loginctl enable-linger $USER` if you intend the daemon to keep running after the user logs out (e.g. CI runners that ssh in to drive cua-driver but never have an interactive session).--See the [Autostart](/cua-driver/guide/getting-started/autostart) concept page for the cross-platform breakdown.--## Headless workflow--Linux doesn't have Windows' Session 0 isolation — `sshd` spawns processes that inherit the user's environment cleanly, including `DISPLAY` / `WAYLAND_DISPLAY` when they're set. So the "SSH + daemon proxy" dance that's necessary on Windows ([Running Cua Driver under SSH on Windows](/cua-driver/guide/getting-started/windows-ssh)) is _not_ needed on Linux: as long as a display server is running on your session, `cua-driver serve` over SSH connects to it.--For **pure headless** (no display server at all — typical CI runner), use `Xvfb` to provide a virtual X server:--```bash-# Install Xvfb (Ubuntu/Debian: xvfb; Fedora: xorg-x11-server-Xvfb; Arch: xorg-server-xvfb)-sudo apt-get install -y xvfb at-spi2-core dbus-x11--# Run Cua Driver under a virtual X server:-xvfb-run -a cua-driver serve--# Or, more explicitly, launch Xvfb yourself + point cua-driver at it:-Xvfb :99 -screen 0 1920x1080x24 &-export DISPLAY=:99-dbus-launch --exit-with-session cua-driver serve-```--This is a useful recipe for local CI experiments with the Linux backend.--<Callout type="info">- **Tools that don't need a display.** Even without `DISPLAY` set, the non-graphical tools- (`list_apps`, `launch_app` via PATH lookup, `read_clipboard` if a clipboard daemon is up) still- work. Only the desktop-touching tools (`click`, `screenshot`, `list_windows`, `get_window_state`)- require a display server. `cua-driver doctor` makes this explicit per-tool.-</Callout>--## Distro-specific notes--The canonical install script (`/bin/bash -c "$(curl -fsSL …/install.sh)"`) downloads a Linux x86_64 binary tarball from GitHub Releases, drops it into `~/.cua-driver/packages/releases/<v>-x86_64-unknown-linux-gnu/`, and symlinks `~/.local/bin/cua-driver`. Linux desktops vary, so run `cua-driver doctor` after install to confirm the distro-specific prerequisites (display server, AT-SPI bus, ffmpeg).--The bit that varies is **what accessibility / display tooling is pre-installed**:--- **Ubuntu / Debian** — GNOME ships AT-SPI by default; `at-spi2-core` is usually present. KDE Plasma needs `at-spi2-core` installed manually.-- **Fedora** — GNOME ships AT-SPI; same as Ubuntu under GNOME. Wayland is the default session on Fedora 25+; if AT-SPI feels flaky, switch to "GNOME on Xorg" at the login screen.-- **Arch / Manjaro** — minimal install; you'll likely need to explicitly install `at-spi2-core` and ensure `dbus-launch` runs as part of your session startup.-- **Alpine / busybox-style** — slimmest of all; needs `dbus`, `dbus-x11`, `at-spi2-core` plus likely an explicit `dbus-launch --exit-with-session` wrapper.--If `cua-driver doctor` is happy after install, you have a reasonable starting point for testing the Linux backend.--## Per-tool Linux implementation matrix--See [`libs/cua-driver/rust/PARITY.md`](https://github.com/trycua/cua/blob/main/libs/cua-driver/rust/PARITY.md) for the per-tool Linux implementation matrix. Treat it as the per-tool implementation matrix; native Wayland input and capture are the main area still maturing.--## See also--- [Installation](/cua-driver/guide/getting-started/installation) — canonical one-liner that handles the Linux backend-- [Autostart](/cua-driver/guide/getting-started/autostart) — concept page (Windows-only verb family; Linux uses systemd)-- [Process model](/cua-driver/guide/getting-started/process-model) — Linux has none of the macOS-TCC or Windows-Session-0 weirdness; this page explains what those problems are and why Linux sidesteps them-- [`PARITY.md`](https://github.com/trycua/cua/blob/main/libs/cua-driver/rust/PARITY.md) — per-tool platform implementation matrixdiff --git a/docs/content/docs/cua-driver/guide/getting-started/meta.json b/docs/content/docs/cua-driver/guide/getting-started/meta.jsondeleted file mode 100644index ec601a3a38..0000000000--- a/docs/content/docs/cua-driver/guide/getting-started/meta.json+++ /dev/null@@ -1,20 +0,0 @@-{- "title": "Getting Started",- "description": "Get up and running with Cua Driver",- "icon": "Rocket",- "defaultOpen": true,- "pages": [- "introduction",- "installation",- "updating",- "quickstart",- "windows-ssh",- "linux",- "autostart",- "pip-preview",- "integrations",- "process-model",- "comparison",- "faq"- ]-}diff --git a/docs/content/docs/cua-driver/guide/getting-started/pip-preview.mdx b/docs/content/docs/cua-driver/guide/getting-started/pip-preview.mdxdeleted file mode 100644index 05ae69c0fb..0000000000--- a/docs/content/docs/cua-driver/guide/getting-started/pip-preview.mdx+++ /dev/null@@ -1,94 +0,0 @@-----title: PiP Preview (Experimental)-description: Always-on-top picture-in-picture window showing the agent's per-action screenshots and a one-line action label.------import { Callout } from 'fumadocs-ui/components/callout';--<Callout type="warn">-**Experimental — opt-in, default OFF.** macOS only today; Windows and Linux ship as compile-clean stubs that print a "not yet implemented" notice when `--experimental-pip` is on argv. The flag, geometry, and frame schema may change before the feature is promoted out of `experimental`. Don't build production tooling against it yet.-</Callout>--`--experimental-pip` opens a small always-on-top window next to your work that shows what the cua-driver agent is doing in real time: the post-action screenshot of the target window, plus a one-line label describing the tool call that produced it (`click element_index=2`, `type_text "hello world"`, etc.).--It's intended as a live "agent's-eye view" — a passive observation surface, not a debugger. The window updates on every action tool call; it doesn't continuously capture the desktop.--## Enabling it--Two ways. The persistent path uses the same `~/.cua-driver/config.json`-file that the `set_config` MCP tool writes to, so PiP survives across-daemon restarts without re-running `claude mcp add` with the flag in-the args list.--### A. Persistent — edit `~/.cua-driver/config.json` (recommended)--```json-{- "experimental_pip": true,- "experimental_pip_geometry": "320x200+24+24"-}-```--Both keys are optional; `experimental_pip_geometry` defaults to-`480x360` in the top-right corner. Restart your MCP client (or kill-the running `cua-driver` daemon) for the new config to take effect.--### B. One-off — CLI flag--```bash-# MCP server (stdio)-cua-driver mcp --experimental-pip--# HTTP / Unix-socket serve daemon-cua-driver serve --experimental-pip--# Override geometry-cua-driver serve --experimental-pip --experimental-pip-geometry 640x400-cua-driver serve --experimental-pip --experimental-pip-geometry 480x300+24+24-```--The geometry string is the standard X11 `WxH+X+Y` form. `+X+Y` is the-top-left origin of the window in screen points (AppKit's bottom-left-convention is hidden inside the backend). **CLI flags override-`config.json`** — they're not additive.--On startup you'll see:--```-⚗️ PiP preview enabled (experimental — macOS only today; see https://github.com/trycua/cua/issues for follow-up)-```--## What gets pushed--PiP receives a frame for the same set of tool calls the recording pipeline writes a `turn-NNNNN/screenshot.png` for — every non-read-only, non-meta call: `click`, `double_click`, `right_click`, `type_text`, `press_key`, `hotkey`, `scroll`, `drag`, `set_value`, `launch_app`, plus the get/refresh AX tools that take a fresh screenshot anyway.--The PNG bytes themselves come from the same `SCREENSHOT_FN` callback the recorder uses, so the live view always matches what a replay would show for that turn.--## Window properties (macOS)--- **Always-on-top:** `NSFloatingWindowLevel` — above your normal apps, below menus / accessibility overlays.-- **Never key:** `setBecomesKeyOnlyIfNeeded(true)` plus a transient / no-cycle collection behavior — the window never steals keyboard focus from your frontmost app.-- **Visible across all spaces:** `CanJoinAllSpaces | FullScreenAuxiliary | Stationary` — survives space switches and full-screen apps.-- **Closeable:** the red traffic-light button dismisses the window and decouples it from the session. Re-enabling means restarting the daemon with the flag.--## Platform support--| Platform | Status | Notes |-|---|---|---|-| macOS | Working | NSWindow + NSImageView, frame updates via `dispatch_async` to the main queue |-| Windows | Stub | `--experimental-pip` is accepted; backend logs "not yet implemented" and the daemon continues without a window |-| Linux | Stub | Same as Windows |--Track Win + Linux follow-up work via the [trycua/cua issue tracker](https://github.com/trycua/cua/issues).--## Non-goals (today)--- **Continuous capture.** PiP follows tool calls, not a frame rate. If you need a video, use `start_recording`.-- **Click-to-drag repositioning.** Window position is static for the session; use `--experimental-pip-geometry` if you need to move it.-- **Audio / recording-to-disk from PiP.** The recording feature already handles those; PiP is a live view only.--## Troubleshooting--- **Window never appears.** Make sure the cursor overlay is enabled (the AppKit event loop is shared) — if you passed `--no-overlay`, the main thread is parked and AppKit doesn't run. Workaround: drop `--no-overlay`.-- **Window appears but no frames.** Confirm a real tool call is landing — bare reads (`get_window_state`, `list_windows`) skip the push path. Try `cua-driver call launch_app '{"bundle_id":"com.apple.calculator"}'` against a running daemon.-- **Label is truncated.** Increase the width via `--experimental-pip-geometry 720x400`.diff --git a/docs/content/docs/cua-driver/guide/getting-started/process-model.mdx b/docs/content/docs/cua-driver/guide/getting-started/process-model.mdxdeleted file mode 100644index 8dc947900f..0000000000--- a/docs/content/docs/cua-driver/guide/getting-started/process-model.mdx+++ /dev/null@@ -1,210 +0,0 @@-----title: MCP process model-description: How the `cua-driver mcp` command runs — in-process vs daemon-proxy modes, lifecycles, failure modes, and wrapper-author guidance------import { Callout } from 'fumadocs-ui/components/callout';--`cua-driver mcp` runs in one of two modes depending on the TCC context it inherits from its parent process. Both expose the **exact same MCP tool surface** — same names, same arguments, same response shapes — so MCP clients never need to care which mode is active. This page documents how the two modes differ at the process level, so wrapper authors (Hermes, custom MCP shims, etc.) and operators debugging the lifecycle have a precise mental model.--<Callout type="info">- This page describes the **macOS** driver, where the daemon proxy exists to inherit `CuaDriver.app`'s TCC- grants. The Rust port (`cua-driver-rs`) has the same daemon-proxy mechanism on Windows and Linux, but it- solves a different problem — reaching the interactive desktop from a Session 0 SSH login — and uses- `CUA_DRIVER_RS_MCP_NO_RELAUNCH` rather than `CUA_DRIVER_MCP_NO_RELAUNCH`. See- [Running cua-driver under SSH on Windows](/cua-driver/guide/getting-started/windows-ssh).-</Callout>--## The two modes--### In-process mode (default when TCC context is correct)--```text-cua-driver.app (LaunchServices) → cua-driver mcp (stdio)- │- └─ AX + ScreenCaptureKit calls- attributed to com.trycua.driver-```--When `cua-driver mcp` is spawned by `CuaDriver.app` directly (LaunchServices-attributed), the process inherits the bundle's TCC grants. Every tool runs in-process: AX walks, screenshots, click synthesis, the agent-cursor overlay, all of it. The lifecycle is simple: the process lives for the duration of the stdio session and exits when stdin closes.--### Daemon-proxy mode (auto-engaged from IDE terminals)--```text-Claude Code / Cursor / Warp LaunchServices- │ │- └─ cua-driver mcp (stdio client) └─ CuaDriver.app- │ │- │ Unix socket (~/Library/Caches/ │- │ cua-driver/cua-driver.sock) │- └──────────────────────────────────────────┴─ cua-driver serve (daemon)- │- └─ AX + ScreenCaptureKit calls- attributed to com.trycua.driver-```--When `cua-driver mcp` is spawned from an IDE terminal (Claude Code, Cursor, VS Code, Warp), macOS attributes the subprocess to the terminal — not `CuaDriver.app` — so AX probes would silently fail against the wrong bundle id. To sidestep this, `mcp` does three things on startup:--1. Checks whether a `cua-driver serve` daemon is already listening on `~/Library/Caches/cua-driver/cua-driver.sock`.-2. If not, runs `open -n -g -a CuaDriver --args serve` to spawn one under LaunchServices (`-n` = new instance so we never reuse an existing CuaDriver.app process without `--args serve`; `-g` = stay backgrounded). Then polls the socket for up to 10s.-3. Proxies every MCP `ListTools` / `CallTool` request through the daemon's line-delimited JSON socket protocol. The mcp client process never touches AX directly.--Two processes, one stdio session. The daemon runs under the right TCC context; the mcp client just shuttles bytes.--## Detecting which mode is active--`cua-driver mcp` emits a single stderr line on startup when it engages the daemon-proxy path:--```text-cua-driver: mcp launched without CuaDriver.app's TCC grants; auto-launching the daemon-via `open -n -g -a CuaDriver --args serve` and proxying MCP requests through it.-Pass --no-daemon-relaunch to stay in-process.-```--In-process mode is silent at startup — no extra log line. So:--- **Stderr contains that line** → daemon-proxy mode.-- **No log line, mcp serving normally** → in-process mode.--You can also infer the mode from the heuristic the code uses. Daemon-proxy mode engages when **all** of the following are true:--- `--no-daemon-relaunch` is not set.-- `CUA_DRIVER_MCP_NO_RELAUNCH` is not truthy (`1`, `true`, `yes`, `on`).-- The binary's `Bundle.main.bundlePath` does not end in `.app` (i.e. we're invoked through the `~/.local/bin/cua-driver` symlink, not the bundle's main executable directly).-- The binary resolves into an installed `CuaDriver.app` bundle via `realpath` (raw `cargo run` dev invocations fail this check because the binary's parent path is not `/Applications/CuaDriver.app/Contents/MacOS/`, and stay in-process).-- `getppid() != 1` (a ppid of 1 means launchd reparented us — we already have the right TCC context).--## Process lifecycles--### Daemon lifecycle (proxy mode only)--The daemon is **not** owned by the mcp client. Concretely:--- **Spawned once.** The first `cua-driver mcp` invocation from an IDE terminal calls `open -n -g -a CuaDriver --args serve`. macOS LaunchServices keeps a handle on the resulting process.-- **Survives mcp-client restarts.** When the MCP client (Claude Code, Cursor) closes stdio and the `cua-driver mcp` proxy exits, the daemon keeps running. The next `cua-driver mcp` invocation reuses the same daemon — it sees the socket already listening and skips the relaunch.-- **Persists across IDE restarts.** As long as nothing kills the `CuaDriver.app` process, the daemon is there to serve any number of mcp-client sessions.-- **Exits when:**- - The user runs `cua-driver stop` (sends a clean shutdown over the socket).- - The user quits `CuaDriver.app` (via the Activity Monitor, `killall CuaDriver`, or the menubar item).- - The Mac reboots / logs out.- - The daemon's own permissions gate is closed without granting Accessibility + Screen Recording (the daemon then exits with a diagnostic).--Run `cua-driver status` at any point to verify:--```bash-cua-driver status-# Cua Driver daemon is running-# socket: /Users/you/Library/Caches/cua-driver/cua-driver.sock-# pid: 12345-```--### Session identity and shared daemon state--The daemon drives **one** physical machine, and every `cua-driver mcp` client shares its `ToolState`. Some of that state is "set once, affects later calls" — the trajectory recorder is a singleton, and `set_config` mutates a process-global config — so two concurrent MCP sessions would otherwise clobber each other.--To scope that state, each `cua-driver mcp` proxy mints a **session identity** once at startup and stamps it on every request it forwards to the daemon. The daemon uses it to OWN and CLEAN UP session-scoped state:--- **Recording ownership.** `start_recording` stamps the live recording with the calling session. On `session_end` the daemon stops only the recording that session owns — so if a later client started its own recording (clobbering the singleton), the earlier client's `session_end` can't stop it. A manual `stop_recording`, the CLI, and the daemon idle-TTL backstop all stop unconditionally.-- **Per-session config.** `set_config` from a proxy-backed session writes an in-memory override keyed by the session, not the shared/persisted global — see [set_config](/cua-driver/reference/mcp-tools#set_config). The override is dropped on `session_end`. Only the anonymous CLI / one-shot path writes the persisted default.-- **Agent cursor.** Each session owns an overlay cursor keyed by its identity, removed on `session_end` — see [the per-session cursors note](/cua-driver/reference/mcp-tools#set_agent_cursor_enabled).--**How `session_end` fires — the per-session reaper.** Each proxy opens **one long-lived control connection** to the daemon at startup (separate from the per-call connections it opens per tool call) and holds it open for its entire lifetime. When the proxy exits — a clean stdin EOF **or** an ungraceful `kill -9` / crash — the kernel closes that socket, the daemon's reader hits EOF, and it fires `session_end` for that session. This makes cleanup reliable even when the proxy dies without a chance to send anything. Liveness is connection-based, not activity-based: a connected-but-idle session (zero tool calls) keeps its control connection parked open and is never reaped. The daemon idle-TTL on recordings is now only a secondary backstop for a non-proxy client.--This solves state **ownership and cleanup**, not safe concurrent screen/input control — the daemon still drives one machine, so two sessions clicking at once still contend for the same desktop. Backward-compatible: a one-shot `cua-driver call` (or an older proxy that predates the session field) carries no session identity and behaves exactly as before — anonymous/global, writing the persisted config and owning no session-scoped state.--### mcp-client lifecycle (proxy mode)--The `cua-driver mcp` process in proxy mode is a thin stdio→UDS pump:--- Lives for the duration of the stdio session — exits cleanly when the MCP client (parent) closes stdin or the host process is killed.-- Does **not** terminate the daemon on exit. The daemon stays running, ready for the next mcp client.-- Cleanup is driven by the proxy's persistent control connection, not a best-effort message: on **any** exit (clean stdin EOF, I/O error, or `kill -9` / crash) the kernel closes that connection and the daemon fires `session_end`, dropping this session's owned state (recording, config overrides, agent cursor). `session_end` is idempotent, so a graceful exit and the EOF reaper can't double-fire.-- Holds no AX caches or per-pid state of its own — everything lives on the daemon side. Restarting just the mcp client preserves whatever element-index caches the daemon has built up.--### mcp-client lifecycle (in-process mode)--Single process. Lives for the duration of the stdio session. Exits when stdin closes. All per-pid state (element-index cache, recording state, agent-cursor toggles) lives in the mcp process and is lost on exit. For element-indexed workflows that need to survive restarts, prefer daemon-proxy mode or run `cua-driver serve` explicitly.--## Failure modes--### Daemon unreachable at mcp startup--`mcp` tries to spawn the daemon via `open -n -g -a CuaDriver --args serve` and polls the socket for 10s. If the daemon never appears (CuaDriver.app missing, app refused to launch, permissions gate not yet granted), `mcp` exits with:--```text-cua-driver: daemon did not appear on <socket> within 10s. If this is the first launch,-grant Accessibility + Screen Recording to CuaDriver.app in System Settings and retry.-Pass --no-daemon-relaunch to stay in-process.-```--If `open` itself fails (CuaDriver.app not installed at all), the error is reported and the process exits with a clear pointer at the `--no-daemon-relaunch` escape hatch.--### Daemon present but refuses the initial ListTools--`makeProxy` caches the tool list with a single `list` RPC at startup. If that fails — daemon transport error, unexpected response, daemon reports failure — `mcp` fails fast with an `MCPError.internalError` carrying the daemon hint, **before** completing the MCP handshake. The MCP client sees a clear startup error rather than a successful handshake that advertises zero tools and then errors on every `CallTool`.--### Daemon dies mid-session--If the daemon is killed (user quits CuaDriver.app, `killall`, crash) while an mcp client is connected, the proxy can't tell until the next `CallTool` lands. That call raises:--```text-MCPError.internalError("Cua Driver daemon not reachable on <socket>. Start it with-`open -n -g -a CuaDriver --args serve` and retry.")-```--The MCP client surfaces this as a tool error. The mcp-client process **does not** auto-restart the daemon mid-session. The recommended client behavior is: catch the error, optionally restart the daemon yourself (`open -n -g -a CuaDriver --args serve`), reconnect the MCP transport, and retry. Hermes' wrapper in trycua/hermes#22821 implements exactly this pattern.--### Socket path ownership--The socket lives at `~/Library/Caches/cua-driver/cua-driver.sock`. macOS cache hygiene may sweep `~/Library/Caches/` periodically; the daemon recreates the socket on startup, and the `--socket` flag overrides the path if you want to pin it elsewhere. A stale socket file from a crashed daemon is detected by the next `serve` invocation's flock probe (`cua-driver.lock`) and replaced atomically.--## Forcing one mode or the other--### Force in-process mode--Useful when the calling context already has the right TCC grants (e.g. you launched `cua-driver mcp` directly from `CuaDriver.app`), or for diagnosing in-process failures:--```bash-# CLI flag-cua-driver mcp --no-daemon-relaunch--# Environment variable (same effect; useful in MCP client config)-CUA_DRIVER_MCP_NO_RELAUNCH=1 cua-driver mcp-```--Either path keeps the entire MCP server in-process. The mcp client takes care of TCC by itself — if Accessibility / Screen Recording aren't granted to the calling shell, AX probes fail.--### Force daemon-proxy mode against a specific socket--For multi-daemon setups, or pointing at a daemon running under an alternate user / install:--```bash-cua-driver mcp --socket /tmp/my-cua-driver.sock-```--`--socket` only overrides the path — the relaunch heuristic still applies (if the daemon isn't listening at that path, `mcp` runs `open -n -g -a CuaDriver --args serve`, which writes to the default socket, not the override). For non-default sockets, start the daemon explicitly first:--```bash-cua-driver serve --socket /tmp/my-cua-driver.sock &-cua-driver mcp --socket /tmp/my-cua-driver.sock-```--### Force in-process mode by launching from CuaDriver.app--Launching `cua-driver mcp` through the bundle's main executable (or any path that LaunchServices attributes to `CuaDriver.app`) makes `Bundle.main.bundlePath` end in `.app`, which trips the heuristic's first short-circuit. In-process mode runs automatically — no flag needed.--## Recommendations for wrapper authors--If you ship a tool that spawns `cua-driver mcp` as a subprocess (Hermes, custom MCP brokers, agent-platform shims):--1. **Don't try to manage the daemon yourself.** `cua-driver mcp` already auto-spawns the daemon on demand. Adding a manual `cua-driver serve` step in your install flow creates two ways to start the daemon — and the user will hit ordering bugs.-2. **Treat the stdio MCP transport as the contract.** Both modes look identical from the MCP-client side. Don't branch on which mode you think is active; the heuristic is the driver's responsibility.-3. **Handle mid-session disconnects via MCP-level reconnect.** If a `CallTool` raises `MCPError.internalError` mentioning "daemon not reachable," tear down the MCP transport and reconnect with a fresh `cua-driver mcp` subprocess. The new subprocess will either find the daemon healthy or relaunch it.-4. **Surface the stderr line.** `cua-driver mcp` writes the daemon-relaunch notice to stderr exactly once at startup. Forward your subprocess's stderr to your own logs so operators can see which mode is active when debugging.-5. **Don't pass `--no-daemon-relaunch` by default.** That flag exists for niche cases (in-process diagnostics, fully sandboxed runtimes where `open -n -g -a` won't work). Defaulting it on makes IDE-terminal users hit the wrong-bundle TCC trap.--## See also--- [Installation](./installation) — granting TCC permissions to `CuaDriver.app`.-- [FAQ](./faq) — common TCC-attribution gotchas in IDE terminals.-- [CLI reference](/cua-driver/reference/cli-reference) — full surface area of `cua-driver mcp` flags, `cua-driver serve`, `cua-driver status`, `cua-driver stop`.diff --git a/docs/content/docs/cua-driver/guide/getting-started/quickstart.mdx b/docs/content/docs/cua-driver/guide/getting-started/quickstart.mdxdeleted file mode 100644index b0ecc3dbe7..0000000000--- a/docs/content/docs/cua-driver/guide/getting-started/quickstart.mdx+++ /dev/null@@ -1,167 +0,0 @@-----title: Quickstart-description: Drive your first desktop app with Cua Driver------import { Callout } from 'fumadocs-ui/components/callout';--You'll launch an app in the background, snapshot its accessibility tree, click a button by element index, and verify the action landed.--<Callout type="info">- The commands below use macOS (`open -n -g -a CuaDriver`, `⌘Q`). On Windows start the daemon with- [`cua-driver autostart`](/cua-driver/guide/getting-started/autostart) and on Linux with `cua-driver serve &`- ([Linux guide](/cua-driver/guide/getting-started/linux)); the `cua-driver call <tool>` invocations are identical- across platforms.-</Callout>--## Start the daemon--Element-indexed workflows require a persistent daemon. The per-pid element cache lives in-process, so one-shot CLI invocations lose it between calls.--```bash-open -n -g -a CuaDriver --args serve-cua-driver status-# Cua Driver daemon is running-# socket: /Users/you/Library/Caches/cua-driver/cua-driver.sock-# pid: 12345-```--<Callout type="info">- `cua-driver serve &` also works. The CLI auto-relaunches itself via LaunchServices when your- shell's TCC context is wrong (any IDE terminal).-</Callout>--Switch to `som` mode so snapshots carry both the AX tree and a screenshot:--```bash-cua-driver config set capture_mode som-```--## Launch an app (hidden)--```bash-cua-driver call launch_app '{"bundle_id":"com.apple.calculator"}'-```--Output:--```-✅ Launched Calculator (pid 844) in background.--Windows:-- "Calculator" [window_id: 10725]-→ Call get_window_state(pid: 844, window_id) to inspect.-```--The app's pid and a `windows` array come back in one call. `launch_app` is idempotent: relaunching a running app returns the existing pid with no side effects. The window's AX tree is fully populated (clickable via `element_index`) but not drawn on screen.--## Snapshot the window--```bash-cua-driver call get_window_state '{"pid":844,"window_id":10725}'-```--Output (trimmed):--```-✅ Calculator — 34 elements, turn 1 + screenshot--- AXApplication "Calculator"- - [0] AXWindow "Calculator" actions=[AXRaise]- - [1] AXButton "All Clear"- - [2] AXButton "Plus/Minus"- - [3] AXButton "Percent"- - [4] AXButton "Divide"- - [5] AXButton "Seven"- ...- - [14] AXButton "Three"- ...-```--Every actionable element is tagged with `[N]` — that's the `element_index` you pass to `click`, `type_text`, and friends. The index map is replaced on every snapshot, keyed on `(pid, window_id)`, so always snapshot before acting.--<Callout type="info">- Large trees (Finder is ~1600 elements) exceed most LLM context limits. Pass a `query` field to filter- the Markdown to matching lines plus their ancestors: `cua-driver call get_window_state '{"pid":844,"window_id":10725,"query":"Three"}'`.-</Callout>--## Click by element_index--```bash-cua-driver call click '{"pid":844,"window_id":10725,"element_index":14}'-# ✅ Performed AXPress on [14] AXButton "Three".-```--No cursor moved. Calculator never came to the foreground. The click went through `AXUIElementPerformAction` directly.--## Verify the action landed--Re-snapshot and check the AX tree diff:--```bash-cua-driver call get_window_state '{"pid":844,"window_id":10725}'-```--Look for the display `AXStaticText` reading `3`. If the tree didn't change, the action failed silently, and you should say so rather than assume success.--## The pixel-click variant--Pixel clicks are for surfaces the AX tree doesn't reach: canvases, video players, WebGL, custom controls. Coordinates are window-local screenshot pixels (same space as the PNG `get_window_state` returns).--```bash-# Write the screenshot to disk. Works in every capture mode.-cua-driver call get_window_state '{"pid":844,"window_id":10725}' --screenshot-out-file /tmp/shot.png--# Look at /tmp/shot.png, pick a target pixel, then:-cua-driver call click '{"pid":844,"window_id":10725,"x":120,"y":240}'-# ✅ Posted click to pid 844.-```--The PNG is capped at 1568 px long-side by default (matching Anthropic's multimodal-vision downsampling limit), so the image you reason over and the coordinate space the click tool expects are the same resolution. No scaling math.--<Callout type="warn">- The `window_id` field is optional on pixel clicks but recommended. It pins the coordinate- conversion to the window whose screenshot produced the pixel, rather than letting the driver pick- heuristically.-</Callout>--## Cleanup--```bash-# Quit the target app via a hotkey to its pid.-cua-driver call hotkey '{"pid":844,"keys":["cmd","q"]}'--# Stop the daemon.-cua-driver stop-```--## The canonical loop--Every multi-step workflow follows the same shape:--```bash-open -n -g -a CuaDriver --args serve--# 1. Launch the target (idempotent; returns pid + windows).-cua-driver call launch_app '{"bundle_id":"..."}'--# 2. Snapshot the window you care about (populates the element cache).-cua-driver call get_window_state '{"pid":<pid>,"window_id":<wid>}'--# 3. Dispatch an action by element_index or pixel coordinates.-cua-driver call click '{"pid":<pid>,"window_id":<wid>,"element_index":<n>}'--# 4. Re-snapshot to verify the action landed.-cua-driver call get_window_state '{"pid":<pid>,"window_id":<wid>}'--cua-driver stop-```--The snapshot-before-AND-after invariant is not optional. Indices are stale across turns. Actions that silently no-op (disabled buttons, minimized windows that reject keyboard commits, Chromium right-clicks) are indistinguishable from successes without the post-action diff.--## What's next--- Full CLI reference: every subcommand and its flags. See [CLI reference](/cua-driver/reference/cli-reference).-- MCP tool schemas: the authoritative input shape for every tool. See [MCP tools](/cua-driver/reference/mcp-tools).-- Known limits: Chromium right-click coercion, canvas apps, off-Space SwiftUI. See [Limits](/cua-driver/reference/limits).-- Common gotchas: "No cached AX state", minimized windows + keyboard commit, disabling the agent cursor. See the [FAQ](/cua-driver/guide/getting-started/faq).diff --git a/docs/content/docs/cua-driver/guide/getting-started/updating.mdx b/docs/content/docs/cua-driver/guide/getting-started/updating.mdxdeleted file mode 100644index 1047111b4c..0000000000--- a/docs/content/docs/cua-driver/guide/getting-started/updating.mdx+++ /dev/null@@ -1,181 +0,0 @@-----title: Updating-description: Check whether a newer Cua Driver release is available, then apply it on your terms — from the CLI, an MCP client, or a script.------import { Callout } from 'fumadocs-ui/components/callout';--Cua Driver ships three update surfaces, all reading from the same GitHub releases endpoint and 20-hour on-disk cache:--| Surface | Verb | Use case |-| -------------- | -------------------------------- | ------------------------------------------------------------------------------- |-| Passive banner | (automatic) | Friendly nudge on every `mcp` / `serve` / `doctor` start. |-| CLI check | `cua-driver check-update` | Human-readable "am I outdated?" probe. |-| Scripted check | `cua-driver check-update --json` | Hermes / CI / wrapper scripts. Same payload as the `check_for_update` MCP tool. |-| MCP tool | `check_for_update` | Agents that want to ask programmatically, without spawning a subprocess. |-| Apply | `cua-driver update --apply` | Download and install the latest release. |--`check-update` and `check_for_update` are **read-only** — they never modify your install. Apply is a separate step on purpose.--## The launch-time banner--Every `cua-driver mcp` / `cua-driver serve` / `cua-driver doctor` invocation kicks off a non-blocking GitHub poll. If a strictly-newer release is found and you haven't dismissed it, a two-line banner lands on stderr (gh-CLI style):--```-✨ cua-driver v0.3.2 is available (you have v0.3.1).- Update with: cua-driver update- Release notes: https://github.com/trycua/cua/releases/tag/cua-driver-rs-v0.3.2-```--The check is cached for 20 hours, runs on a background task, and never blocks startup. To disable it set `CUA_DRIVER_RS_UPDATE_CHECK=false` in the environment, or persist the choice with:--```bash-cua-driver config set update_check_enabled false-```--## `cua-driver check-update` — human path--A pure check, never installs. Prints a short summary and exits.--```bash-$ cua-driver check-update-Current: 0.3.1-Latest: 0.3.2--Update available. Run `cua-driver update --apply` to install.-Release notes: https://github.com/trycua/cua/releases/tag/cua-driver-rs-v0.3.2-```--When you're on the latest release:--```bash-$ cua-driver check-update-Current: 0.3.1-Latest: 0.3.1--You're on the latest release.-```--### Flags--- `--json` — emit a machine-readable JSON payload (see below).-- `--no-cache` — skip the 20h on-disk cache and force a fresh GitHub round-trip. Useful when iterating right after a release.--### Exit codes--- `0` — the check itself succeeded. The `update_available` field in the JSON payload carries the "is it outdated?" signal.-- `1` — the check failed (network down, GitHub 5xx, parse error). The text path prints a one-line reason; the JSON path puts it in the `error` field.--The non-zero-on-outdated convention (à la `brew outdated`) is intentionally not used — it conflicts with every shell script's "non-zero means error" assumption. Hermes parses JSON; humans read text; the signal lives in the payload.--## `cua-driver check-update --json` — scripted path--Same check, JSON output. Use this when wiring Cua Driver updates into Hermes, CI, or any wrapper script.--```bash-$ cua-driver check-update --json-{- "current_version": "0.3.1",- "latest_version": "0.3.2",- "update_available": true,- "source": "github_releases",- "checked_at": "2026-05-27T14:30:00Z",- "cache_hit": false,- "install_command": "curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.sh | bash",- "release_notes_url": "https://github.com/trycua/cua/releases/tag/cua-driver-rs-v0.3.2",- "error": null-}-```--When you're up to date, `install_command` and `release_notes_url` are `null`:--```bash-$ cua-driver check-update --json-{- "current_version": "0.3.1",- "latest_version": "0.3.1",- "update_available": false,- "source": "github_releases",- "checked_at": "2026-05-27T14:30:00Z",- "cache_hit": true,- "install_command": null,- "release_notes_url": null,- "error": null-}-```--### JSON schema--| Field | Type | Notes |-| ------------------- | -------------- | ------------------------------------------------------------------------------------------------------------------ |-| `current_version` | string | The running binary's `CARGO_PKG_VERSION`. |-| `latest_version` | string \| null | Highest non-draft non-prerelease `cua-driver-rs-v*` tag on GitHub. `null` if the fetch failed and no cache exists. |-| `update_available` | bool | `latest > current` under strict semver. `false` when `latest_version` is `null`. |-| `source` | string | Always `"github_releases"` today. Pinned so future alternate sources can be slotted in without breaking consumers. |-| `checked_at` | string | ISO-8601 UTC timestamp of this check (not the cached fetch). |-| `cache_hit` | bool | `true` when we short-circuited via the cache, `false` when we hit GitHub. |-| `install_command` | string \| null | Shell one-liner to apply the update. `null` when up-to-date or the check failed. |-| `release_notes_url` | string \| null | Release notes page on GitHub. `null` when up-to-date or the check failed. |-| `error` | string \| null | Human-readable reason when the fetch failed and no cache was available. |--### Example consumer snippet (bash)--```bash-if cua-driver check-update --json | jq -e '.update_available' > /dev/null; then- echo "Cua Driver is outdated — applying update"- cua-driver update --apply-fi-```--## `check_for_update` — MCP tool--Same payload, surfaced over MCP so agents can probe without spawning a subprocess. Input schema is empty.--```bash-$ cua-driver call check_for_update '{}'-{- "current_version": "0.3.1",- "latest_version": "0.3.2",- "update_available": true,- "source": "github_releases",- "checked_at": "2026-05-27T14:30:00Z",- "cache_hit": false,- "install_command": "curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.sh | bash",- "release_notes_url": "https://github.com/trycua/cua/releases/tag/cua-driver-rs-v0.3.2",- "error": null-}-```--The tool is registered as `read_only`, `idempotent`, and `open_world` (the network can change the answer between calls). MCP clients see both a human summary in `content[0].text` and the JSON above in `structuredContent`.--<Callout>- The MCP tool intentionally has no `apply` variant. Installing over MCP would force a- self-replacement of the running MCP server — a UX boundary we want explicit. If your agent needs- to apply, shell out to `cua-driver update --apply` or run the `install_command` from the payload.-</Callout>--## `cua-driver update --apply` — installing--When you're ready to install, `update --apply` delegates to the canonical installer script (the same one the docs print as the one-liner):--```bash-$ cua-driver update --apply-Current version: 0.3.1-Checking for updates…-New version available: 0.3.2-Downloading and installing Cua Driver 0.3.2…-Installed Cua Driver 0.3.2.-```--If a daemon was already running, restart it to pick up the new binary:--```bash-cua-driver stop && cua-driver serve-```--`update` (without `--apply`) keeps the existing "check + suggest" text behaviour intact for backward compat. Add `--json` to either form for a structured payload — handy when the calling script wants to log the version it just installed.--```bash-$ cua-driver update --json | jq '.update_available'-true-```diff --git a/docs/content/docs/cua-driver/guide/getting-started/windows-ssh.mdx b/docs/content/docs/cua-driver/guide/getting-started/windows-ssh.mdxdeleted file mode 100644index 9a6edc65b3..0000000000--- a/docs/content/docs/cua-driver/guide/getting-started/windows-ssh.mdx+++ /dev/null@@ -1,177 +0,0 @@-----title: Running Cua Driver under SSH on Windows-description: The "daemon in interactive session + SSH-side proxy" workflow that makes Cua Driver work headlessly over SSH on Windows.------import { Callout } from 'fumadocs-ui/components/callout';--The canonical headless / remote-driving setup for Cua Driver on Windows is:--1. A `cua-driver serve` daemon runs in the user's **interactive session** (Session 1+), kept alive by [`cua-driver autostart`](/cua-driver/guide/getting-started/autostart).-2. You SSH into the same box (which lands in Session 0).-3. `cua-driver mcp` and `cua-driver call <tool>` on the SSH side **proxy through the interactive-session daemon** over the daemon's named pipe.-4. Claude Code (or any MCP client) over SSH drives real GUI apps — `list_apps`, `click`, `type_text`, `screenshot` all respond against the user's real desktop.--This page explains why the proxy is necessary, how to wire it up, and the opt-outs.--## The Session 0 problem--Windows OpenSSH server runs in **Session 0** — the non-interactive services session. Sessions `1`, `2`, ... are interactive logons (one per console / RDP user). Every process inherits the session of its parent, so a shell spawned from `sshd` lands in Session 0 too.--The Win32 APIs that Cua Driver's window tools bottom out in — `EnumWindows`, `GetForegroundWindow`, `PrintWindow`, UIA, `BitBlt` over the screen DC — are all scoped to the **calling process's WindowStation + Desktop**. Session 0 has no interactive desktop attached. So:--```powershell-# Over SSH (Session 0), with no daemon helper:-cua-driver call list_windows-# [] ← empty. The user's RDP session has 12 windows open.-```--That's not a Cua Driver bug — those APIs are working as designed against a session with no desktop. The same thing would happen to any native Win32 tool spawned the same way.--<Callout type="info">-**Confirm with `cua-driver doctor`.** The Windows session probe surfaces this directly:--```text-[warn] interactive session: running in Session 0 (services); window-driving- tools (list_windows, click, type_text, screenshot, get_window_state)- will return empty results — these APIs need an attached interactive- desktop.-```--</Callout>--## The fix: daemon in Session 1+, proxy from Session 0--The Rust port (`cua-driver-rs`) ships with a daemon-proxy path on every platform. When `cua-driver mcp` or `cua-driver call <tool>` starts up and detects a daemon already listening on the default socket, it forwards every request through to the daemon instead of running tool calls in-process.--The daemon runs in the user's interactive session (registered via `cua-driver autostart enable`, which uses a Scheduled Task with `LogonType: Interactive`). The SSH-side CLI runs in Session 0 but only does protocol-shuttling — the actual tool invocations execute inside the daemon, which has a real desktop attached.--```-┌───────────────────────────────────────────────────────────────┐-│ Session 1+ (RDP / console — has interactive desktop) │-│ │-│ cua-driver-serve (autostart Scheduled Task) │-│ ↑ │-│ │ named pipe: \\.\pipe\cua-driver │-│ │ │-└──────┼────────────────────────────────────────────────────────┘- │-┌──────┼────────────────────────────────────────────────────────┐-│ Session 0 (services / SSH — no desktop) │-│ │ │-│ cua-driver mcp │-│ cua-driver call list_windows │-│ Claude Code → MCP stdio → cua-driver mcp │-└───────────────────────────────────────────────────────────────┘-```--## The canonical recipe--```powershell-# ── ONE-TIME from your interactive session (RDP or console) ──────────-cua-driver autostart enable # register the Scheduled Task-cua-driver autostart kick # start it now, don't wait for next logon--# ── from SSH (or any session) ────────────────────────────────────────-cua-driver status-# Cua Driver daemon is running-# socket: \\.\pipe\cua-driver-# pid: 12345-# session: 2 ← daemon's in your interactive session--cua-driver call list_apps # sees real GUI apps-claude # Claude Code's MCP tool calls proxy through-```--`autostart enable` registers a Scheduled Task that re-runs `cua-driver serve` at every interactive logon — see the [Autostart concept page](/cua-driver/guide/getting-started/autostart) for the verb-by-verb breakdown and the RDP-disconnect-survives property.--<Callout type="warn">-**Prerequisite — the user must have an active interactive session.** Either:--- An RDP client is connected, OR-- An RDP client _was_ connected and has disconnected without logging off (the session stays in `Disc` state and is still considered active), OR-- The user is logged in at the local console.--Confirm with `query session` from any shell — the user row should show `Active` or `Disc`:--```powershell-query session-# SESSIONNAME USERNAME ID STATE TYPE-# rdp-tcp#23 you 2 Active ← good-# # — or —-# rdp-tcp#23 you 2 Disc ← also good-```--No interactive session means `autostart kick` has no Session 1+ to land the daemon in — `kick` will report success but the daemon won't have a desktop attached. Open an RDP session once and the autostart trigger will fire `serve` automatically next logon.--</Callout>--## How the proxy decides whether to forward--`cua-driver mcp` and `cua-driver call <tool>` share the same gating function ([`should_use_daemon_proxy` in `cli.rs`](https://github.com/trycua/cua/blob/main/libs/cua-driver/rust/crates/cua-driver/src/cli.rs)):--- If `--no-daemon-relaunch` is passed, **stay in-process**.-- If `CUA_DRIVER_RS_MCP_NO_RELAUNCH=1` is set, **stay in-process**.-- Otherwise, **proxy if a daemon is listening** on the default socket; **run in-process if none is up**.--There's no auto-spawn on Windows / Linux — unlike macOS, there's no `open -a CuaDriver` equivalent that could land a fresh daemon in the right session. If no daemon is listening when you run `cua-driver mcp`, the CLI bails with an actionable error:--```text-no Cua Driver daemon listening on \\.\pipe\cua-driver. Start one in your-interactive session — on Windows run `cua-driver autostart enable &&-cua-driver autostart kick`; on Linux run `cua-driver serve &` in the user's-session. Then re-run `cua-driver mcp`. To skip the proxy and run in-process-anyway (Session 0 attribution, GUI tools will return empty), pass---no-daemon-relaunch.-```--<Callout type="info">-**Why `call` worked over SSH before v0.2.7 but `mcp` didn't.** The `call` CLI had its own daemon-proxy shortcut baked in from the start (the daemon is the only place the per-pid element cache lives, so `call` had to proxy to share state across invocations anyway). The `mcp` entrypoint was hardcoded to run in-process on Windows / Linux on the assumption that MCP clients spawn fresh subprocesses — true, but irrelevant: a fresh subprocess over SSH is still in Session 0. `cua-driver-rs` v0.2.7 ([PR #1580](https://github.com/trycua/cua/pull/1580)) lined the two entrypoints up so `mcp` proxies on the same conditions `call` does.--If you're on v0.2.6 or earlier and saw empty MCP results over SSH while `cua-driver call list_apps` worked, upgrade — `irm https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.ps1 | iex`.--</Callout>--## Wiring Claude Code over SSH--Once the daemon is up in your interactive session, the Claude Code registration is unchanged from the local case:--```powershell-# From inside your SSH session:-claude mcp add --transport stdio cua-driver -- cua-driver.exe mcp-claude-```--Every MCP tool call Claude makes over the stdio link triggers `cua-driver mcp` on the SSH side, which detects the daemon and proxies to it. The MCP client sees an ordinary stdio server; the daemon sees a steady stream of tool calls from its Session 1+ desktop context.--## Opt-out: in-process even when a daemon is up--There are legitimate reasons to skip the proxy — running inside CI in an already-interactive runner that owns its own session, debugging the in-process MCP server, or running an integration test that boots its own daemon and wants the CLI to talk to a different socket.--Two equivalent opt-outs:--```powershell-cua-driver mcp --no-daemon-relaunch # CLI flag, per-invocation-$env:CUA_DRIVER_RS_MCP_NO_RELAUNCH = "1" # env var, per-shell-cua-driver mcp # now stays in-process-```--Either form keeps `cua-driver mcp` in-process and the tool calls execute against whatever session spawned the CLI — Session 0 over SSH (empty GUI results), or the interactive session if you're running locally.--## Diagnostics — "my MCP calls return empty over SSH"--Walk this list before opening an issue:--1. **`cua-driver --version` on the SSH side reports `0.2.7` or later.** Earlier versions don't proxy `mcp` on Windows / Linux. Upgrade with the [install one-liner](/cua-driver/guide/getting-started/installation).-2. **`cua-driver status` from SSH reports a running daemon.** If not, `cua-driver autostart status` will tell you whether the Scheduled Task is registered. Re-run `enable && kick` from an RDP session if not.-3. **`query session` shows an `Active` or `Disc` row for your user.** No interactive session means no Session 1+ for the daemon to land in.-4. **`cua-driver doctor` from RDP reports `[ok] interactive session: session N has an attached interactive desktop`** — confirms the daemon's session has a desktop attached. Run from the same session you registered autostart in.-5. **You did not pass `--no-daemon-relaunch`** and `CUA_DRIVER_RS_MCP_NO_RELAUNCH` is unset — both opt the proxy out.--If all five pass and tool calls still come back empty, run `cua-driver call list_windows` over SSH and compare with the same call from RDP. Same empty result on both means the daemon's in the wrong session; differing results mean the proxy isn't activating (file an issue with `cua-driver doctor --json` from both sessions).--## See also--- [Autostart](/cua-driver/guide/getting-started/autostart) — verb-by-verb breakdown of `cua-driver autostart`, the daemon-survives-RDP-disconnect property.-- [Installation → Windows interactive-session requirements](/cua-driver/guide/getting-started/installation#windows-interactive-session-requirements) — the underlying session model.-- [Installation → Auto-start at logon (Windows)](/cua-driver/guide/getting-started/installation#auto-start-at-logon-windows) — manual Scheduled Task registration (equivalent to `autostart enable`, kept for users who want to hand-craft the task).diff --git a/docs/content/docs/cua-driver/guide/meta.json b/docs/content/docs/cua-driver/guide/meta.jsondeleted file mode 100644index 7febd12285..0000000000--- a/docs/content/docs/cua-driver/guide/meta.json+++ /dev/null@@ -1,6 +0,0 @@-{- "title": "Guide",- "description": "Learn how to use Cua Driver",- "icon": "Book",- "pages": ["getting-started"]-}diff --git a/docs/content/docs/cua-driver/meta.json b/docs/content/docs/cua-driver/meta.jsondeleted file mode 100644index c781e7aa4d..0000000000--- a/docs/content/docs/cua-driver/meta.json+++ /dev/null@@ -1,5 +0,0 @@-{- "title": "Cua Driver",- "description": "Background computer-use driver for any agents",- "pages": ["guide", "reference"]-}diff --git a/docs/content/docs/cua-driver/reference/changelog.mdx b/docs/content/docs/cua-driver/reference/changelog.mdxdeleted file mode 100644index 8320025cf2..0000000000--- a/docs/content/docs/cua-driver/reference/changelog.mdx+++ /dev/null@@ -1,217 +0,0 @@-----title: Changelog-description: Release history for cua-driver------# cua-driver Changelog--All notable changes to cua-driver are documented here.--These are the `cua-driver-rs` (Rust port) releases. The Rust port ships the same-user-facing `cua-driver` binary for macOS and Windows, plus Linux pre-release-artifacts for early testing. All versions are currently published as GitHub-**prereleases** under the `cua-driver-rs-v*` tag prefix (distinct from the Swift-driver's `cua-driver-v*` tags so their artifacts don't collide).--The canonical, always-current source is the-[GitHub releases page](https://github.com/trycua/cua/releases). Each GitHub-release body is auto-generated per release from the commits touching-`libs/cua-driver/rust`, including SHA256 checksums and install instructions.--## 0.5.6 (unreleased)--- **Windows: fix idle CPU burn and orphaned `mcp` processes from the cursor- overlay** (#1808). The agent-cursor overlay render timer ran at ~125 Hz- unconditionally and re-composited a full virtual-screen pixmap (plus an- RGBA→BGRA copy and `UpdateLayeredWindow` blit) on every tick — so an idle- `cua-driver mcp` with no automation pinned 60–85% of a CPU core. The render- loop is now event-driven: it only composites and blits while a cursor is- actually animating or fading, and drops to a slow heartbeat (waking instantly- on the next command) once every cursor is static. The in-process `mcp` (stdio)- child now force-exits when the client disconnects (stdin EOF / pipe closed)- instead of leaving the overlay loop running as an orphan that accumulates CPU.- Use `--no-overlay` to disable the agent cursor entirely for headless runs.--## 0.5.4 (2026-06-15)--- **macOS: fix 0x0 screenshots on macOS 15+ — the release bundle now ships the- `com.apple.security.device.screen-capture` entitlement.** The macOS release- codesign steps signed the universal binary and `CuaDriver.app` with the- hardened runtime but passed no entitlements file, so on macOS 15 Sequoia and- later `SCStream.startCapture` failed with "Missing entitlements" even when TCC- Screen Recording was granted, and vision capture (`screenshot`,- `get_window_state`) returned a 0x0 image. The release pipeline now signs with- an entitlements plist granting `com.apple.security.device.screen-capture`- (plus `com.apple.security.automation.apple-events`), restoring capture- (#1893, #1862).--## 0.5.3 (2026-06-11)--- **Linux: background drag and held-button tools + MPX parallel drags** (#1871).- Adds `parallel_mouse_drag`: each session gets its own MPX master pointer (fed- by a per-session uinput slave), so multiple agent cursors can drag- concurrently in the same window. Drags follow full waypoint paths — a- straight from→to segment or a function `fn` = y(x) (e.g. `"300+150*sin(x/40)"`)- — as one continuous held glide. A device-specific XI2 shield grab keeps the- window manager blind to the drag's button press, so parallel drags no longer- steal focus from the user's active window, and drag masters are cleaned up- per call. Also adds a confirmation-gated `install_ffmpeg` tool that detects- the platform package manager and (only with `confirm: true`) installs the- ffmpeg binary the video recording backend shells out to.-- **Linux: agent cursor in background sessions + background terminal typing**- (#1789). The agent cursor overlay now renders for background (unfocused)- sessions, and typing reaches background terminals; Linux keyboard input- switched from `XSendEvent` to XTEST injection (#1805) so synthetic keys are- accepted by apps that ignore sent events.--## 0.5.2 (2026-06-04)--- **Windows: background input without z-raise** — input lands in target- windows without raising them, plus multi-cursor demos and agent cursor- overlay improvements (#1809).-- **Windows: DPI awareness manifest** so coordinates are correct on scaled- displays (#1821).-- **Python: `cua-driver` PyPI wrapper with bundled binaries** (#1849), plus- Windows ARM64 support and `cua-driver-uia.exe` extraction (#1850).--## 0.5.1 (2026-06-01)--- **Install home unified on `~/.cua-driver` + collision fix.** The release- installer (`install.sh` → `_install-rust.sh`) was still defaulting its package- home to the legacy `~/.cua-driver-rs`, while the local installer- (`install-local.sh`) and the runtime already used `~/.cua-driver` (renamed in- v0.2.16). That mismatch meant a machine could end up with two homes and two- conflicting installs. The release installer now defaults to `~/.cua-driver`- (still honoring the `CUA_DRIVER_RS_HOME` override), and on every install it (a)- cleans up a prior `install-local` dev build under the shared home — stops the- daemon, removes the `*-local-*` release dirs and the local signing-identity- marker — and (b) sweeps a stale `~/.cua-driver-rs` left by an older release.- Both steps are best-effort, idempotent, and conservative (marker-gated; they- never touch a real release dir, the `current` symlink, or unrelated user- state). TCC grants are preserved — the `/Applications/CuaDriver.app` bundle is- replaced in place, not `tccutil reset`. The Windows installer (`install.ps1`)- already used `~/.cua-driver` and migrated the legacy home, so it is unchanged.--## 0.5.0 (2026-06-01)--- **Windows: per-session agent cursors** — the per-session cursor model (macOS- #1779) is ported to Windows, so concurrent sessions are tracked per-session on- Windows too (#1801).-- **Streamable-HTTP MCP transport (parallel multi-agent).** Over stdio, one- `cua-driver mcp` process is a single pipe, so a client's tool calls — including- multiple subagents — serialize. The daemon itself is concurrent. Set- `CUA_DRIVER_RS_MCP_HTTP_PORT=<port>` and the daemon also serves MCP over HTTP- (`POST http://127.0.0.1:<port>/mcp`, loopback only), so each agent opens its- **own** connection and they run truly in parallel — measured 3.6× on 10- concurrent calls. Per-connection ordering keeps a single agent's sequence- correct; the per-`(pid, window_id)` cache + per-session cursor make concurrent- cross-connection actions safe. `move_cursor` is now `readOnlyHint:true` so MCP- clients also parallelize cursor moves on stdio (mutating tools stay serialized- on purpose) (#1799).-- **Breaking — session identity is now caller-declared.** A "session" (which- owns the agent cursor + per-session state) used to be minted per MCP- connection, so it couldn't be set from the CLI and couldn't span a run that- touches multiple apps. It's now an explicit identity you declare: pass a- `session` id (or call the new `start_session`/`end_session` tools), and the- same id drives the same cursor over MCP, the CLI, or the raw socket, following- the run across any apps/windows. **The cursor is now opt-in:** a run shows a- cursor only when it declares a `session` — anonymous calls execute without one.- Sessions are reclaimed by `end_session` or an idle-TTL (default 300s, override- `CUA_DRIVER_RS_SESSION_IDLE_TTL_SECS`) instead of connection-EOF. Per-session- config + recording keep their existing connection-scoped cleanup as a fallback- when no `session` is declared. Update agents to `start_session` at the start of- a run (the bundled skill + MCP instructions now say so).-- macOS: fix a daemon crash (`EXC_BREAKPOINT` in `AXUIElementCopyActionNames`)- when two sessions drive the same window concurrently. Element actions- (`click`, `type_text`, `set_value`, …) read the target `AXUIElementRef` out- of the per-`(pid, window_id)` cache, but a concurrent `get_window_state`- could replace that cache entry and `CFRelease` the element to zero while the- action was still using it — a use-after-free. The cache now hands out a- retained guard (`CFRetain` under the cache lock, `CFRelease` on drop), so an- in-flight action keeps the element alive across a concurrent refresh.-- macOS (local dev install only): `install-local.sh` now clears a stale TCC- grant left over from a previous signing identity. If you first granted- Accessibility / Screen Recording while the bundle was ad-hoc signed, that- grant was pinned to the per-build `cdhash` and silently stopped matching on- the next rebuild — the daemon read `NOT granted` while System Settings still- showed CuaDriver toggled ON, a dead end re-toggling couldn't fix. The- installer records the signing identity and, when it changes, runs- `tccutil reset` so the next grant re-pins cleanly to the stable self-signed- certificate (after which grants survive all future rebuilds). Release builds- are unaffected — they're CI-signed with a stable identity (#1792 follow-up).--## 0.4.3 (2026-05-31)--- macOS: the agent-cursor overlay now actually renders in the `serve` daemon.- It was only wired into the in-process `mcp` path, so in the daemon-proxy- setup (the default) every cursor command was a silent no-op and the agent- cursor never appeared (#1790).-- macOS: `cua-driver permissions grant` no longer spams the TCC prompt — it- raises a single dialog and then polls silently, and the gate honours its- deadline instead of re-prompting (and restarting the daemon) on every- ~25s re-exec (#1791).-- Dev: `install-local` now signs the bundle with a stable self-signed- identity, so TCC grants (Accessibility / Screen Recording) survive rebuilds- instead of resetting on every install (#1792).--## 0.4.2 (2026-05-31)--- macOS: guard the SkyLight auth-message selector on macOS 14 Sonoma so the- driver no longer crashes on launch (#1782, #1503).-- macOS: enable Chromium/Electron accessibility trees via `AXManualAccessibility`,- so `get_window_state` can see their window content (#1756).--## 0.4.1 (2026-05-31)--- macOS: per-session agent cursors, so concurrent sessions each get their own- visual agent cursor (#1779).-- macOS: ship `AppIcon.icns` in the bundle so `CuaDriver.app` no longer shows a- blank icon (#1780, #1496).-- macOS: guard AppKit initialization so `mcp` runs headless instead of- SIGABRT-ing (#1781, #1724).--## 0.4.0 (2026-05-31)--- Daemon session-identity model: the daemon now owns and cleans up- session-scoped recording and config, tying that state to the session that- created it (#1776).--## 0.3.6 (2026-05-30)--- macOS: fix permissions status so it reports the driver's real TCC grants (via- the daemon) instead of the calling terminal's — it no longer claims granted- when only the caller holds the grant (#1774).--## 0.3.5 (2026-05-30)--- macOS: bind the `serve` socket before the permissions gate, so clients can- connect even while permissions are still being resolved (#1761, #1773).--## 0.3.4 (2026-05-30)--- Maintenance release (no path-specific changes).--## 0.3.3 (2026-05-30)--- Maintenance release (no path-specific changes).--## 0.3.2 (2026-05-27)--- Add a `check-update` CLI verb and a `check_for_update` MCP tool so clients can- detect when a newer driver is available (#1734).--## 0.3.1 (2026-05-27)--- First release in the 0.3.x line, which began the macOS TCC / permissions- hardening series.--## 0.2.x (2026-05-17 – 2026-05-21)--- Pre-release iteration on the Rust port across versions 0.2.0–0.2.18,- including cross-platform build and packaging work for the macOS, Windows, and- Linux pre-release artifacts.--## 0.1.x (2026-05-14)--- Initial `cua-driver-rs` pre-releases. See the- [GitHub releases](https://github.com/trycua/cua/releases) for details.diff --git a/docs/content/docs/cua-driver/reference/cli-reference.mdx b/docs/content/docs/cua-driver/reference/cli-reference.mdxdeleted file mode 100644index c2fbb50c49..0000000000--- a/docs/content/docs/cua-driver/reference/cli-reference.mdx+++ /dev/null@@ -1,371 +0,0 @@-----title: CLI Reference-description: Command Line Interface reference for Cua Driver------{/*- AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY- Generated by: npx tsx scripts/docs-generators/cua-driver.ts- Source: cua-driver dump-docs- Version: 0.3.2-*/}--import { Callout } from 'fumadocs-ui/components/callout';-import { VersionHeader } from '@/components/version-selector';--<VersionHeader- versions={[{"version":"0.3","href":"/cua-driver/reference/cli-reference","isCurrent":true}]}- currentVersion="0.3"- fullVersion="0.3.2"- packageName="cua-driver"- installCommand="curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.sh | bash"-/>--Cross-platform computer-use automation driver.--## Tool dispatch--### cua-driver list-tools--List every registered MCP tool with a one-line description.--### cua-driver describe--Print a tool's full description and JSON input schema.--**Arguments:**--| Name | Type | Required | Description |-| ---- | ---- | -------- | ----------- |-| `<tool-name>` | String | Yes | Name of the MCP tool to describe. |--### cua-driver call--Invoke an MCP tool directly from the shell.--Runs the same handler the MCP server uses. JSON arguments may be passed as a positional JSON object or through stdin.--**Arguments:**--| Name | Type | Required | Description |-| ---- | ---- | -------- | ----------- |-| `<tool-name>` | String | Yes | Name of the MCP tool to invoke. |-| `<json-args>` | String | No | JSON object for the tool input schema. If omitted, stdin is read when piped. |--**Options:**--| Name | Type | Default | Description |-| ---- | ---- | ------- | ----------- |-| `--screenshot-out-file` | String | — | Write the first image content block from the response to this path. |-| `--socket` | String | — | Override the daemon socket or named-pipe path. |--## Daemon management--### cua-driver mcp--Run the stdio MCP server.--On macOS, shell-spawned MCP processes can auto-launch and proxy through a CuaDriver.app daemon so TCC grants attach to the bundle. On Windows and Linux, MCP proxies through an already-running daemon when one is listening.--**Options:**--| Name | Type | Default | Description |-| ---- | ---- | ------- | ----------- |-| `--socket` | String | — | Override the daemon socket or named-pipe path used by the proxy fallback. |--**Flags:**--| Name | Description |-| ---- | ----------- |-| `--no-daemon-relaunch` | Stay in-process instead of proxying through a daemon. |-| `--claude-code-computer-use-compat` | Expose the Claude Code computer-use compatibility screenshot surface. |--### cua-driver serve--Run Cua Driver as a long-running daemon.--The daemon owns per-process state such as element-index caches, recording state, and cursor overlay state.--**Options:**--| Name | Type | Default | Description |-| ---- | ---- | ------- | ----------- |-| `--socket` | String | — | Override the daemon socket or named-pipe path. |-| `--pid-file` | String | — | Override the pid-file path on Unix targets. |--**Flags:**--| Name | Description |-| ---- | ----------- |-| `--no-permissions-gate` | Skip the macOS first-launch permissions gate. |--### cua-driver stop--Ask the running daemon to exit gracefully.--**Options:**--| Name | Type | Default | Description |-| ---- | ---- | ------- | ----------- |-| `--socket` | String | — | Override the daemon socket or named-pipe path. |--### cua-driver status--Report whether a Cua Driver daemon is running.--**Options:**--| Name | Type | Default | Description |-| ---- | ---- | ------- | ----------- |-| `--socket` | String | — | Override the daemon socket or named-pipe path. |-| `--pid-file` | String | — | Override the pid-file path on Unix targets. |--### cua-driver mcp-config--Print MCP server config or a client-specific install command.--Supported clients include claude, codex, cursor, antigravity, openclaw, opencode, hermes, and pi.--**Options:**--| Name | Type | Default | Description |-| ---- | ---- | ------- | ----------- |-| `--client` | String | — | Client name to print configuration for. |--## Trajectory recording--### cua-driver recording--Control trajectory recording on a running daemon.--Recording state lives in-process, so use a daemon for multi-call sessions.--**Options:**--| Name | Type | Default | Description |-| ---- | ---- | ------- | ----------- |-| `--socket` | String | — | Override the daemon socket or named-pipe path. |--#### cua-driver recording start--Start trajectory recording to a directory.--**Arguments:**--| Name | Type | Required | Description |-| ---- | ---- | -------- | ----------- |-| `<output-dir>` | String | Yes | Directory to write turn folders into. |--#### cua-driver recording stop--Stop trajectory recording.--#### cua-driver recording status--Print the current recording state.--#### cua-driver recording render--Render a recorded trajectory directory to an MP4.--This pure file-to-file path does not require a running daemon.--**Arguments:**--| Name | Type | Required | Description |-| ---- | ---- | -------- | ----------- |-| `<input-dir>` | String | Yes | Trajectory directory containing recorded turn folders. |-| `<out-mp4>` | String | Yes | Output MP4 path. |--**Options:**--| Name | Type | Default | Description |-| ---- | ---- | ------- | ----------- |-| `--scale` | Number | — | Scale factor for rendered frames. |--**Flags:**--| Name | Description |-| ---- | ----------- |-| `--no-zoom` | Disable cursor/action zoom effects in the rendered video. |--## Configuration--### cua-driver config--Read or mutate persistent driver configuration.--Without a subcommand, prints the full config.--**Options:**--| Name | Type | Default | Description |-| ---- | ---- | ------- | ----------- |-| `--socket` | String | — | Override the daemon socket or named-pipe path. |--#### cua-driver config show--Print the full config.--#### cua-driver config get--Print one config key.--**Arguments:**--| Name | Type | Required | Description |-| ---- | ---- | -------- | ----------- |-| `<key>` | String | Yes | Config key to read. |--#### cua-driver config set--Set one config key.--**Arguments:**--| Name | Type | Required | Description |-| ---- | ---- | -------- | ----------- |-| `<key>` | String | Yes | Config key to write. |-| `<value>` | String | Yes | Value to store. |--#### cua-driver config reset--Reset config to defaults.--## Diagnostics--### cua-driver check-update--Check whether a newer cua-driver release is available.--Read-only. Uses the same update-state payload as the check_for_update MCP tool.--**Flags:**--| Name | Description |-| ---- | ----------- |-| `--json` | Emit a machine-readable JSON payload. |-| `--no-cache` | Skip the 20-hour on-disk cache and force a GitHub request. |--### cua-driver update--Check for an update and optionally apply it.--The apply path delegates to the canonical platform installer scripts.--**Flags:**--| Name | Description |-| ---- | ----------- |-| `--apply` | Download and install the latest release when one is available. |-| `--json` | Emit the structured update-state payload. |--### cua-driver doctor--Run platform-aware diagnostic probes.--Exit code is non-zero when any probe is an error.--**Flags:**--| Name | Description |-| ---- | ----------- |-| `--json` | Emit the probe report as JSON. |--### cua-driver diagnose--Print a pasteable install-layout and permission-attribution report.--## Other commands--### cua-driver autostart--Manage platform-native daemon autostart.--Windows registers a logon Scheduled Task. macOS and Linux currently print manual-recipe guidance.--#### cua-driver autostart enable--Register the autostart entry.--#### cua-driver autostart disable--Remove the autostart entry.--#### cua-driver autostart status--Print whether autostart is registered and running.--#### cua-driver autostart kick--Start the autostart entry now without re-logging.--### cua-driver skills--Install, update, inspect, or remove the optional agent skill pack.--The install script never touches agent skill directories automatically.--#### cua-driver skills install--Fetch the versioned skill pack and link detected agents.--**Options:**--| Name | Type | Default | Description |-| ---- | ---- | ------- | ----------- |-| `--agent` | String | — | Restrict linking to one agent. |-| `--from` | String | — | Fetch from a source such as main instead of the tagged release. |--**Flags:**--| Name | Description |-| ---- | ----------- |-| `--all-platforms` | Keep platform-specific skill files for every platform. |--#### cua-driver skills update--Refresh the local skill pack and links.--#### cua-driver skills uninstall--Remove agent skill links.--**Flags:**--| Name | Description |-| ---- | ----------- |-| `--all` | Also delete the local skill-pack copy. |--#### cua-driver skills status--Report local skill-pack and per-agent link state.--#### cua-driver skills path--Print the local skill-pack path.--### cua-driver dump-docs--Output machine-readable CLI and MCP documentation JSON.--Used by the docs generator to keep reference pages in sync with the live binary.--**Options:**--| Name | Type | Default | Description |-| ---- | ---- | ------- | ----------- |-| `--type` | String | all | Which docs to emit: all, cli, or mcp. |--**Flags:**--| Name | Description |-| ---- | ----------- |-| `-p`, `--pretty` | Pretty-print JSON. |--## Global options--Available on all commands:--- `--help` — Show help information.-- `--version` — Show version number.diff --git a/docs/content/docs/cua-driver/reference/limits.mdx b/docs/content/docs/cua-driver/reference/limits.mdxdeleted file mode 100644index 5216f57cac..0000000000--- a/docs/content/docs/cua-driver/reference/limits.mdx+++ /dev/null@@ -1,82 +0,0 @@-----title: Known limits-description: What Cua Driver can't do, and the workarounds------import { Callout } from 'fumadocs-ui/components/callout';--Cua Driver's no-foreground contract holds for every app the driver reaches via AX or via the SkyLight-routed pixel click. Four categories of target fall outside that envelope. Each is documented here with the workaround.--## Chromium coerces synthetic right-clicks on web content--**Symptom:** `right_click({pid, x, y})` on a Chrome, Edge, Brave, or Arc tab's web content fires a left-click instead of opening the context menu.--**Cause:** Chromium's renderer-IPC filter drops the right-click subtype bit on events that don't come through the HID tap. Every synthesized-event path on macOS hits this wall, not just ours.--**Workarounds, in order of preference:**--1. Use `right_click({pid, element_index})` on AX-addressable targets (links, buttons, toolbar items). AX dispatch sidesteps the renderer filter entirely.-2. For context menus on pure web content (nothing in the AX tree), activate Chrome briefly and fall back to a HID-tap right-click. Breaks the no-foreground-steal promise for that one click.--<Callout type="info">- Element-indexed right-click (`right_click({(pid, element_index)})`) works fine. The limit is- specifically pixel right-click on non-AX Chromium web content.-</Callout>--## Canvas apps need brief frontmost activation--**Affected:** Blender (GHOST event source), Unity editor / Unity games, most native games, some WebGL-heavy Electron apps.--**Symptom:** `click({pid, x, y})` on a Blender viewport silently no-ops. `launch_app` works, the window is visible, the tree says nothing actionable is there. Clicks vanish.--**Cause:** These apps only accept events from `cghidEventTap` with a leading `mouseMoved`. They explicitly filter out per-pid-routed events, which is the path Cua Driver uses for its backgrounded dispatch. There's no per-pid recipe that reaches them.--**Workaround:** The driver auto-detects these targets and falls back to a brief activation + HID-tap click. The cursor visibly warps. The activation is the shortest possible and the click fires within the same event loop turn, so focus returns to the user's previous app within a frame or two, but the warp is noticeable.--<Callout type="warn">- If you're automating Blender or a game, the no-foreground-steal contract does not apply. The- driver will tell you: the `click` response includes `dispatch: "hid_tap"` when the HID fallback- fired.-</Callout>--## Off-Space SwiftUI windows strip their AX tree--**Symptom:** `get_window_state({pid, window_id})` on a window that's on a different Space (e.g. System Settings parked on Space 2 while you're on Space 1) returns a tiny tree that only contains the menu bar, or just the AXApplication root.--**Cause:** macOS 14+ strips AX detail from non-current-Space SwiftUI windows as a privacy / performance tradeoff. AppKit apps are not affected. This is an Apple design decision; no workaround exists that keeps the window off-Space.--**Response shape:** Cua Driver surfaces this explicitly. Every `get_window_state` response on an off-current-Space window carries `off_space: true` plus the `window_space_ids` array, so callers can decide to switch Space, pick a different window, or skip the turn.--**Workarounds:**--1. Switch the user to the target's Space first (`SpaceMigrator.migrate`, exposed via the forthcoming `migrate_space` tool). Breaks the no-Space-bounce promise.-2. Target an AppKit equivalent of the app if one exists.-3. Limit off-Space automation to AppKit apps where the tree stays populated.--## Minimized windows silently drop keyboard commits--**Symptom:** `press_key({pid, element_index, key: "return"})` on a text field in a minimized window returns success, but the field doesn't commit. You hear the macOS system-alert beep, or nothing happens.--**Cause:** AX reads and AX-dispatched clicks propagate through to minimized windows normally, but keyboard-commit events (Return, Space, Tab) require renderer focus, which AX focus does not confer on a minimized window. This is a macOS-wide behavior; every automation tool hits it.--**Workarounds:**--1. Use `set_value({pid, element_index, value: "..."})` to write the field's value directly. No keyboard event involved; no focus handoff required.-2. AX-click a commit-equivalent button (Go, Submit, Send, OK) rather than relying on Return.-3. Un-minimize the window (`hotkey({pid, keys: ["cmd", "m"]})` or click the Dock icon). Breaks the background contract for that window.--<Callout type="info">- `set_value` is the right answer 90% of the time. It sidesteps both the minimized-focus issue and- the general "which event commits this field" ambiguity.-</Callout>--## Permission boundaries--Cua Driver is constrained by the usual macOS permission model. Two relevant grants:--- **Accessibility** (System Settings → Privacy & Security → Accessibility) is required for every AX read, every element-indexed click, every keyboard / text primitive. Without it, `check_permissions` returns `accessibility: false` and every tool returns a structured error.-- **Screen Recording** is required for screenshots and for the `som` / `vision` capture modes. Without it, `get_window_state` in those modes returns a tree but no PNG; pure `ax` mode still works.--Grants are tied to the `CuaDriver.app` bundle identity. Rebuilding Cua Driver from source preserves the grants because `build-app.sh` pins a stable bundle id (`com.trycua.driver`).--See [Installation](/cua-driver/guide/getting-started/installation) for the grant flow.diff --git a/docs/content/docs/cua-driver/reference/mcp-tools.mdx b/docs/content/docs/cua-driver/reference/mcp-tools.mdxdeleted file mode 100644index b79b107253..0000000000--- a/docs/content/docs/cua-driver/reference/mcp-tools.mdx+++ /dev/null@@ -1,638 +0,0 @@-----title: MCP Tools-description: Reference for every MCP tool Cua Driver exposes------{/*- AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY- Generated by: npx tsx scripts/docs-generators/cua-driver.ts- Source: cua-driver dump-docs- Version: 0.3.2-*/}--import { Callout } from 'fumadocs-ui/components/callout';--`cua-driver` exposes 33 MCP tools through a single stdio server (`cua-driver mcp`). Every tool is also callable from the shell as `cua-driver <name> '<JSON-args>'`.--Tool names are `snake_case`. Responses are MCP `CallTool.Result` envelopes: a text content block prefixed with a `✅` summary (or the error reason on failure), plus optional image or structured-content blocks on tools that produce them. See the [CLI reference](/cua-driver/reference/cli-reference) for CLI-specific options like `--socket` and `--screenshot-out-file`.--<Callout type="info">- Tool names here match the CLI form exactly. `cua-driver list_apps` and the MCP `list_apps` tool run the same code path.-</Callout>--<Callout type="info">- **TCC auto-delegation.** When an MCP client spawns `cua-driver mcp` from an IDE terminal (Claude Code, Cursor, VS Code, Warp), macOS attributes the subprocess to the parent terminal — not `CuaDriver.app` — so AX probes fail against the wrong bundle id. `mcp` detects this and auto-launches a `cua-driver serve` daemon via `open -n -g -a CuaDriver --args serve`, then proxies every tool call through the daemon's Unix socket. Tool semantics are identical to the in-process path; no Python bridge is needed. Pass `--no-daemon-relaunch` (or set `CUA_DRIVER_MCP_NO_RELAUNCH=1`) to force in-process execution. See the [process model guide](/cua-driver/guide/getting-started/process-model) for the full lifecycle, failure modes, and wrapper-author guidance.-</Callout>--## Inspection tools--### list_apps--List macOS apps — both currently running and installed-but-not-running — with per-app state flags:--- running: is a process for this app live? (pid is 0 when false)-- active: is it the system-frontmost app? (implies running)-- launch_path: filesystem path to the `.app` bundle, when known. Pass this to `launch_app` to start the app cold.-- kind: `"desktop"` for `.app` bundles on macOS.-- last_used: RFC3339 timestamp from the bundle's filesystem mtime, when readable; otherwise null.--Only apps with NSApplicationActivationPolicyRegular are included — background helpers and system UI agents are filtered out. Installed apps come from scanning /Applications, /Applications/Utilities, ~/Applications, /System/Applications, and /System/Applications/Utilities.--Use this for "is X installed?" as well as "is X running?". For per-window state — on-screen, on-current-Space, minimized, window titles — call list_windows instead. For just opening an app — running or not — call launch_app({bundle_id: ...}) directly; list_apps is not a prerequisite.--**Arguments:** none.--### list_windows--List all layer-0 top-level windows currently known to WindowServer. Includes off-screen windows (minimized, on another Space, hidden-launched). Use this to find a window_id before calling get_window_state.--Per-record fields: window_id, pid, app_name, title, bounds (x/y/width/height, top-left origin), z_index (higher = frontmost), is_on_screen, on_current_space.--**Arguments:**--- `on_screen_only` (boolean, optional): When true, drop windows not on the current Space. Default false.-- `pid` (integer, optional): Optional pid filter. When set, only this pid's windows are returned.--### get_window_state--Walk a running app's AX tree and return a Markdown rendering of its UI, tagging every actionable element with [element_index N]. Pass those indices to click, type_text, press_key, etc.--INVARIANT: call get_window_state once per turn per (pid, window_id) before any element-indexed action. The index map is replaced by the next snapshot.--Also captures a PNG screenshot of the specified window.--Optional `query` filters the tree_markdown to matching lines plus their ancestor chain (case-insensitive substring). The element_index values are unchanged — filtering only trims the rendered Markdown.--**Arguments:**--- `capture_mode` (string, optional): som=AX+screenshot (default), vision=screenshot only (no AX walk), ax=AX only (no screenshot).-- `pid` (integer, required): Target process ID.-- `query` (string, optional): Case-insensitive filter for tree_markdown.-- `screenshot_out_file` (string, optional): When set, write the PNG to this file path (~ expanded) instead of embedding base64 in the response. The structured output will contain screenshot_file_path instead.-- `window_id` (integer, required): Target window ID from list_windows.--```json-{"pid":844,"window_id":10725}-```--### get_accessibility_tree--Return a lightweight snapshot of the desktop: running regular apps and on-screen visible windows with their bounds, z-order, and owner pid.--For the full AX subtree of a single window (with interactive element indices you can click by), use `get_window_state` instead — that's the heavy per-window tool. This one is a fast discovery read that needs no TCC grants.--**Arguments:** none.--### get_screen_size--Return the logical size of the main display in points plus its backing scale factor. Agents click in points; Retina displays have scale_factor 2.0. Requires no TCC permissions.--**Arguments:** none.--### get_cursor_position--Return the current mouse cursor position in screen points (origin top-left).--**Arguments:** none.--### get_config--Return the current cua-driver-rs configuration.--**Arguments:** none.--### get_recording_state--Report the current trajectory recorder state: whether recording is enabled, the output directory (when enabled), and the 1-based counter for the next turn folder that will be written. Counter increments on every recorded action tool call and resets to 1 each time recording is (re-)enabled.--Pure read-only.--**Arguments:** none.--### get_agent_cursor_state--Return the current state of **this session's** agent cursor: position, config (color, icon, label, size, opacity), enabled flag. The result is scoped to the cursor the call resolves to (precedence **explicit `cursor_id` > session identity > `"default"`**), so concurrent sessions no longer see each other's cursors and the top-level `enabled` flag is deterministic. Pass `cursor_id` to inspect a specific instance.--**Arguments:**--- `cursor_id` (string, optional): Cursor instance to inspect. Omit it and the call targets the calling session's own cursor (macOS); the anonymous / one-shot path targets `"default"`.--## Action tools--### launch_app--Launch a macOS app in the background — the target does NOT come to the foreground.--Provide either `bundle_id` (preferred — unambiguous, e.g. `com.apple.calculator`) or `name` (e.g. "Calculator"). If both are given, bundle_id wins.--Optional `urls` are handed to the app as open targets — for Finder, pass a folder path to open a backgrounded Finder window there.--Optional `electron_debugging_port`: opens a Chrome DevTools Protocol (CDP) server on the specified port (appends --remote-debugging-port=N to the app's argv). Use this to automate Electron/VS Code/Cursor via CDP.--Optional `webkit_inspector_port`: opens a WebKit inspector server on the specified port (sets WEBKIT_INSPECTOR_SERVER=127.0.0.1:N + TAURI_WEBVIEW_AUTOMATION=1). Use this for Tauri/WebKit-based apps.--Optional `creates_new_application_instance`: when true, forces a new app instance even if one is already running (passes -n to open). Reach for this in **concurrent multi-agent/multi-session** work — it returns a fresh pid + window so each session drives its own isolated window. Without it, single-instance apps (Calculator, many utilities) hand every caller the same window, so two sessions clobber each other.--Optional `additional_arguments`: extra argv strings appended after --args.--Returns the launched app's pid, bundle_id, name, and a `windows` array (same shape as `list_windows`) so callers can skip an extra round-trip before `get_window_state(pid, window_id)`. When the focus-steal belt-and-braces demotion check ran (target pid ≠ prior frontmost), the response also includes `self_activation_suppressed: bool` — true if focus stayed with the prior frontmost, false if the launched app held focus despite the re-demote attempt.--**Arguments:**--- `additional_arguments` (array of string, optional): Extra arguments appended after --args when launching.-- `bundle_id` (string, optional): App bundle identifier, e.g. com.apple.calculator. Preferred over name.-- `creates_new_application_instance` (boolean, optional): When true, force a new app instance even if already running (open -n). Use for concurrent multi-agent/multi-session work so each session gets an isolated instance + window instead of sharing one (which makes the sessions clobber each other on single-instance apps).-- `electron_debugging_port` (integer, optional): Open a Chrome DevTools Protocol server on this port (appends --remote-debugging-port=N).-- `name` (string, optional): App display name. Used only when bundle_id is absent.-- `urls` (array of string, optional): Optional file paths or URLs to open with the app (e.g. a folder path for Finder).-- `webkit_inspector_port` (integer, optional): Open a WebKit inspector server on this port (sets WEBKIT_INSPECTOR_SERVER env var).--### kill_app--Force-terminate a process by pid (kill -9 equivalent on macOS / Linux; taskkill /F equivalent on Windows). Use as escalation when the cooperative close path (hotkey cmd+q on macOS, click-the-X on Windows) failed to make the process exit. Unsaved state is lost — prefer the cooperative path first.--**Arguments:**--- `pid` (integer, required): PID of the process to terminate.--```json-{"pid":844}-```--### bring_to_front--Activate a window so subsequent input tools with `dispatch:"foreground"` land on it without a per-call SetForegroundWindow flash. **Windows-only:** on macOS this tool returns an error pointing to the platform-native `NSRunningApplication.activate` (which the macOS input tools don't need because CGEvent.postToPid reaches backgrounded windows). On Linux this tool also stubs out; use `wmctrl -a` or `xdotool windowactivate` if you need explicit activation.--**Arguments:**--- `pid` (integer, required)-- `window_id` (integer, optional)--```json-{"pid":844}-```--### click--Left-click against a target pid. **Prefer `element_index` over pixel coordinates** — element_index works on backgrounded / minimized / hidden / off-Space windows, surfaces a stable handle that survives rebuilds, and tells you what you're clicking via the cached element's role + label. Reach for `x, y` only when the target is a canvas / video / WebGL / custom-drawn surface that doesn't appear in the AX tree.--Two addressing modes:--- element_index + window_id (from last get_window_state): AX action path. Works on backgrounded/hidden windows. No cursor move, no focus steal. element_index cache is scoped per (pid, window_id) and is replaced by the next snapshot of the same window — re-snapshot every turn before clicking.--- x, y (window-local screenshot pixels, top-left origin of the PNG returned by get_window_state): CGEvent path. Synthesizes mouse events and posts to pid. Use modifier for cmd/shift/option/ctrl. Needs a visible on-screen window to anchor the conversion.--action: press (default), show_menu, pick, confirm, cancel, open.-from_zoom: set true after a zoom call to auto-translate zoom-image pixel coordinates to full-window space.--**Arguments:**--- `action` (string, optional): AX action: press, show_menu, pick, confirm, cancel, open.-- `count` (integer, optional): Click count (pixel path only). Default 1.-- `debug_image_out` (string, optional): Optional file path. When set on a pixel-addressed click, captures a fresh screenshot, draws a red crosshair at (x, y), and writes the PNG. Use to verify coordinate spaces. Requires window_id; incompatible with from_zoom.-- `element_index` (integer, optional): Element index from last get_window_state.-- `from_zoom` (boolean, optional): When true, x and y are in the last zoom image for this pid; driver translates back to full-window coordinates.-- `modifier` (array of string, optional): Modifier keys: cmd, shift, option/alt, ctrl.-- `pid` (integer, required): Target process ID.-- `window_id` (integer, optional): Target window ID. Required for element_index.-- `x` (number, optional): Window-local screenshot X coordinate.-- `y` (number, optional): Window-local screenshot Y coordinate.--```json-{"pid":844}-```--### double_click--Double-click at (x, y) or on an AX element identified by element_index + window_id.--AX path (element_index provided): performs `AXOpen` when the element advertises it (Finder items, openable list rows/cells); otherwise resolves the element's on-screen center and falls back to a pixel double-click there.--Pixel path (x, y provided): two down/up pairs ~80 ms apart at the given coordinates.--**Arguments:**--- `element_index` (integer, optional): Element index from last get_window_state. Uses AX path.-- `pid` (integer, required)-- `window_id` (integer, optional): CGWindowID. Required when element_index is used.-- `x` (number, optional): Screen X coordinate (pixel path).-- `y` (number, optional): Screen Y coordinate (pixel path).--```json-{"pid":844}-```--### right_click--Right-click against a target pid. Two addressing modes:--- `element_index` + `window_id` (from the last `get_window_state` snapshot) — performs `AXShowMenu` on the cached element. Pure AX RPC, works on backgrounded / hidden windows, no cursor move or focus steal. Requires a prior `get_window_state(pid, window_id)` in this turn.--- `x`, `y` — synthesizes `rightMouseDown` / `rightMouseUp` CGEvent pair posted to the pid. Driver converts image-pixel → screen-point internally. `modifier` forces the CGEvent path (AX actions don't propagate modifier keys).--Exactly one of `element_index` or (`x` AND `y`) must be provided. `pid` always required. `window_id` required when `element_index` is used.--**Arguments:**--- `element_index` (integer, optional): Element index from last get_window_state. Routes through AXShowMenu. Requires window_id.-- `modifier` (array of string, optional): Modifier keys held during the right-click: cmd/shift/option/ctrl. Pixel path only.-- `pid` (integer, required): Target process ID.-- `window_id` (integer, optional): CGWindowID. Required when element_index is used.-- `x` (number, optional): X in window-local screenshot pixels. Must be provided together with y.-- `y` (number, optional): Y in window-local screenshot pixels. Must be provided together with x.--```json-{"pid":844}-```--### drag--Press-drag-release gesture from (from_x, from_y) to (to_x, to_y) in window-local screenshot pixels — the same space get_window_state returns. Top-left origin of the target's window.--Use for: marquee/lasso selection, drag-and-drop, resizing via a handle, scrubbing a slider, repositioning a panel.--`duration_ms` (default 500) is the wall-clock budget for the path between mouse-down and mouse-up; `steps` (default 20) is the number of intermediate mouseDragged events linearly interpolated along the path. Increase both for slower, more human drags; decrease for snap gestures.--`modifier` keys (cmd/shift/option/ctrl) are held across the entire gesture.--When `from_zoom` is true, coordinates are in the last zoom image for this pid; the driver maps them back to window coordinates before dispatching.--**Arguments:**--- `button` (string, optional): Mouse button used for the drag. Default: left.-- `duration_ms` (integer, optional): Wall-clock duration of the drag path between mouseDown and mouseUp. Default: 500.-- `from_x` (number, required): Drag-start X in window-local screenshot pixels. Top-left origin.-- `from_y` (number, required): Drag-start Y in window-local screenshot pixels. Top-left origin.-- `from_zoom` (boolean, optional): When true, coordinates are in the last zoom image for this pid; driver maps back to window coordinates.-- `modifier` (array of string, optional): Modifier keys held across the entire gesture: cmd/shift/option/ctrl.-- `pid` (integer, required): Target process ID.-- `steps` (integer, optional): Number of intermediate mouseDragged events linearly interpolated along the path. Default: 20.-- `to_x` (number, required): Drag-end X in window-local screenshot pixels.-- `to_y` (number, required): Drag-end Y in window-local screenshot pixels.-- `window_id` (integer, optional): CGWindowID for the window the pixel coordinates were measured against. Optional — when omitted the driver picks the frontmost window of pid.--```json-{"from_x":100,"from_y":200,"pid":844,"to_x":100,"to_y":200}-```--### type_text--Insert text into the target pid via `AXSetAttribute(kAXSelectedText)`. Works for standard Cocoa text fields and text views. No keystrokes are synthesized — special keys (Return / Escape / arrows) go through `press_key` / `hotkey`. For Chromium / Electron inputs that don't implement `kAXSelectedText`, the tool falls back to CGEvent character synthesis automatically.--Optional `element_index` + `window_id` (from the last `get_window_state` snapshot) directs the write to a specific field. Without `element_index`, the write goes to the pid's currently focused element.--**Arguments:**--- `delay_ms` (integer, optional): Milliseconds between characters in the CGEvent fallback path. Default 30. Ignored when the AX path succeeds.-- `element_index` (integer, optional): Element index from last get_window_state. Directs the write to a specific field. Requires window_id.-- `pid` (integer, required): Target process ID.-- `text` (string, required): Text to insert at the target's cursor.-- `window_id` (integer, optional): CGWindowID. Required when element_index is used.--```json-{"pid":844,"text":"hello"}-```--### press_key--Press and release a single key, delivered to the target pid via CGEventPostToPid. No focus steal.--Two delivery paths:-• window_id + element_index: focuses the AX element first, then posts via the auth-message path (Chromium-safe).-• window_id only (no element_index): NSMenu path — briefly activates the window WindowServer-frontmost via SLPSSetFrontProcessWithOptions (kCPSNoWindows, < 1 ms), posts WITHOUT the auth envelope so IOHIDPostEvent fires and NSApplication.sendEvent: dispatches NSMenu key equivalents. Restores prior frontmost immediately.-• No window_id: standard auth-message path.--Key names: return, tab, escape, up/down/left/right, space, delete, home, end, pageup, pagedown, f1-f12, plus any letter or digit.-Modifiers array: cmd, shift, option/alt, ctrl, fn.--**Arguments:**--- `element_index` (integer, optional)-- `key` (string, required): Key name: return, tab, escape, up, down, etc.-- `modifiers` (array of string, optional): Modifier keys: cmd, shift, option/alt, ctrl, fn.-- `pid` (integer, required)-- `window_id` (integer, optional)--```json-{"key":"return","pid":844}-```--### hotkey--Press a combination of keys simultaneously — e.g. `["cmd", "c"]` for Copy, `["cmd", "shift", "4"]` for screenshot selection. The combo is posted directly to the target pid's event queue; the target does NOT need to be frontmost.--Two delivery paths:-• Default (no window_id): auth-message envelope — Chromium/Electron apps accept the keystrokes as trusted live input on macOS 14+.-• With window_id: NSMenu path — briefly activates the target WindowServer-frontmost via SLPSSetFrontProcessWithOptions (kCPSNoWindows, < 1 ms), posts WITHOUT the auth envelope so IOHIDPostEvent fires and NSApplication.sendEvent: dispatches NSMenu key equivalents (e.g. Cmd+Z undo, Cmd+W close). Restores prior frontmost immediately. Use this path when you need native menu-bar actions on non-Chromium apps.--Recognized modifiers: cmd/command, shift, option/alt, ctrl/control, fn. Non-modifier keys use the same vocabulary as `press_key` (return, tab, escape, up/down/left/right, space, delete, home, end, pageup, pagedown, f1-f12, letters, digits). Order: modifiers first, one non-modifier last.--**Arguments:**--- `keys` (array of string, required): Modifier(s) and one non-modifier key, e.g. ["cmd", "c"].-- `pid` (integer, required): Target process ID.-- `window_id` (integer, optional): When set, uses NSMenu path: briefly activates the window for menu key dispatch, then restores prior frontmost.--```json-{"keys":["cmd","c"],"pid":844}-```--### set_value--Set a value on a UI element. Two modes depending on element role:--- **AXPopUpButton / select dropdown**: finds the child option whose title or value matches `value` (case-insensitive) and AXPresses it directly — the native macOS popup menu is never opened, so focus is never stolen. Use this for HTML <select> elements in Safari or any native NSPopUpButton.--- **All other elements**: writes AXValue directly (sliders, steppers, date pickers, native text fields that expose settable AXValue).--For free-form text entry into web inputs, prefer `type_text_chars` which synthesises key events — AXValue writes are ignored by WebKit.--**Arguments:**--- `element_index` (integer, required)-- `pid` (integer, required)-- `value` (string, required): New value. AX will coerce to the element's native type.-- `window_id` (integer, required): CGWindowID for the window whose get_window_state produced the element_index.--```json-{"element_index":14,"pid":844,"value":"42","window_id":10725}-```--### scroll--Scroll the target pid's focused region by synthesized keystrokes.--Mapping: by='page' → PageDown/PageUp × amount; by='line' → DownArrow/UpArrow × amount. Horizontal variants use Left/Right arrow keys.--Optional element_index + window_id pre-focuses the element before scrolling.--**Arguments:**--- `amount` (integer, optional): Number of keystroke repetitions. Default: 3.-- `by` (string, optional): Scroll granularity. Default: line.-- `direction` (string, required): Scroll direction.-- `element_index` (integer, optional)-- `pid` (integer, required)-- `window_id` (integer, optional)--```json-{"direction":"down","pid":844}-```--### move_cursor--Move the agent cursor overlay to (x, y). Does NOT move the real mouse cursor — the user's cursor stays where it is. Useful for showing the agent's attention without interrupting the user.--**Arguments:**--- `cursor_id` (string, optional): Explicit cursor-instance override. Omit it and the move targets the calling session's own cursor (macOS); the anonymous path targets 'default'. See the per-session agent cursors note under [set_agent_cursor_enabled](#set_agent_cursor_enabled).-- `x` (number, required)-- `y` (number, required)--```json-{"x":100,"y":200}-```--### zoom--Capture a cropped JPEG of a window region (x1,y1)–(x2,y2) in screenshot pixel coordinates, with 20% padding added on each side. The output image is at most 500 px wide.--After a zoom, pass `from_zoom=true` to click/type_text to auto-translate coordinates back to full-window space.--**Arguments:**--- `pid` (integer, optional): Target pid — required for from_zoom click/type translation.-- `window_id` (integer, required): CGWindowID from list_windows.-- `x1` (number, required): Left edge of region in screenshot pixels.-- `x2` (number, required): Right edge of region in screenshot pixels.-- `y1` (number, required): Top edge of region in screenshot pixels.-- `y2` (number, required): Bottom edge of region in screenshot pixels.--```json-{"window_id":10725,"x1":100,"x2":100,"y1":200,"y2":200}-```--## Browser tools--### page--Interact with the browser page loaded in a running app. Supports Chrome, Brave, Edge, Safari (via AppleScript on macOS), Electron apps (via CDP), Chromium/Firefox on Windows (via UIA for read; CDP for execute_javascript when --remote-debugging-port is set), and WKWebView/Tauri/AT-SPI fallbacks.--Actions:-- execute_javascript: Run JS and return the result.-- get_text: Extract visible text from the page.-- query_dom: Find elements matching a CSS selector.-- click_element: Click a CSS-selected element AND animate the agent cursor to its on-screen center first (so the user sees what the agent is doing). Prefer over `execute_javascript('el.click()')` whenever you want visible cursor feedback.-- enable_javascript_apple_events: macOS-only — patch the browser's Preferences to allow JS from Apple Events (Chrome/Brave/Edge, requires user confirmation and a browser restart).--**Arguments:**--- `action` (string, required): Action to perform.-- `attributes` (array of string, optional): Element attributes to include in query_dom results.-- `bundle_id` (string, optional): Bundle ID of the browser. Required for enable_javascript_apple_events (macOS only).-- `css_selector` (string, optional): CSS selector for query_dom (e.g. 'a', 'button', 'input', 'h1'-'h6', 'p', 'img', 'select', '*').-- `javascript` (string, optional): JavaScript to execute. Required for execute_javascript.-- `pid` (integer, optional): Target process ID.-- `selector` (string, optional): CSS selector for click_element (e.g. 'button.submit', '#login a').-- `user_has_confirmed_enabling` (boolean, optional): Must be true to proceed with enable_javascript_apple_events. This will quit and relaunch the browser.-- `window_id` (integer, optional): Target window ID from list_windows.--```json-{"action":"get_text"}-```--## Recording tools--### start_recording--Start trajectory recording. Every subsequent action-tool invocation (click, right_click, scroll, type_text, press_key, hotkey, set_value) writes a turn folder under `output_dir`:--- `app_state.json` — post-action AX/UIA snapshot for the target pid.-- `screenshot.png` — post-action per-window screenshot of the target's frontmost on-screen window.-- `action.json` — tool name, full input arguments, result summary, pid, click point (when applicable), ISO-8601 timestamp.-- `click.png` — for click-family actions only, `screenshot.png` with a red dot drawn at the click point.--Turn folders are named `turn-00001/`, `turn-00002/`, etc. Turn numbering restarts at 1 each time recording is (re-)started.--**Video is off by default.** Pass `record_video: true` to also capture the main display to `<output_dir>/recording.mp4` (H.264 / 30 fps) for the lifetime of the session. On the daemon-proxy path (the default macOS `cua-driver mcp` flow), recording is stopped automatically when the session ends, and the daemon stops only the recording that session owns — so a forgotten session no longer keeps writing to disk. Session-end is detected reliably even on **ungraceful proxy death** (`kill -9`, crash): each proxy holds one long-lived control connection to the daemon, and the kernel closing that socket on proxy exit (graceful **or** killed) fires `session_end`. A daemon-side idle timeout (~5 min of no tool activity) remains as a secondary backstop for a non-proxy client that started a recording and died.--**macOS uses native ScreenCaptureKit** (in-process SCStream + SCRecordingOutput) so video inherits Cua Driver's own Screen Recording grant — no extra TCC prompt, no ffmpeg subprocess. Requires macOS 15.0+.--**Windows + Linux use an ffmpeg subprocess** (`gdigrab` / `x11grab` + libx264). Requires ffmpeg on PATH (winget install Gyan.FFmpeg / apt install ffmpeg); when ffmpeg is missing or fails on startup the per-turn capture (screenshots + action.json) still runs and the session's `last_error` field carries the diagnostic.--State persists for the life of the daemon / MCP session; a restart resets to disabled with no on-disk state. Call `stop_recording` to disable + finalize the mp4.--The result's `structuredContent` includes an `owner` field — the session that owns the live recording (the proxy-minted session identity), or `null` when started anonymously. Ownership is what lets the daemon-proxy disconnect auto-stop (a `session_end` signal) stop only the recording this session started, even when a later client clobbered the singleton recorder with its own `start_recording`. (This `owner` field supersedes the earlier `generation` token.)--**Arguments:**--- `output_dir` (string, required): Absolute or ~-rooted directory where turn folders and (when enabled) the video file are written.-- `record_video` (boolean, optional): Capture the main display to <output_dir>/recording.mp4. Default: false. Set to true to also capture the main display to recording.mp4 (otherwise only the per-turn screenshots + JSON are recorded). On macOS this uses native ScreenCaptureKit (no extra TCC prompt, macOS 15.0+); on Windows + Linux it requires ffmpeg on PATH.--```json-{"output_dir":"~/cua-trajectories/demo1"}-```--### stop_recording--Stop trajectory recording. Disables further per-turn capture and, when video was enabled, gracefully terminates the ffmpeg subprocess so the mp4's moov atom is finalized (the file is playable). Calling stop on an already-stopped session is a no-op. The response carries `last_video_path` pointing at the finalized mp4 (when video was on).--A manual `stop_recording` is **unconditional** — it stops whatever recording is active regardless of which session started it. Ownership-scoped teardown (so one MCP client disconnecting can't stop a recording a later client started) is handled by the daemon's `session_end` lifecycle signal, not by this tool — so `stop_recording` takes no arguments.--**Arguments:**--_None._--### replay_trajectory--Replay a recorded trajectory by re-invoking every turn's tool call in lexical order. `dir` must point at a directory previously written by `start_recording`. Each `turn-NNNNN/` is parsed for `action.json`, and the recorded tool is called with its recorded `arguments` via the same dispatch path an MCP / CLI call uses.--Caveats:-- Element-indexed actions (`click({pid, element_index})` etc.) will fail because element indices are per-snapshot and don't survive across sessions. Pixel clicks (`click({pid, x, y})`) and all keyboard tools replay cleanly. Failures are reported but don't stop replay unless `stop_on_error` is true.-- `get_window_state` and other read-only tools are NOT currently recorded, so replays do not re-populate the per-(pid, window_id) element cache.-- If recording is ENABLED while replay runs, the replay itself is recorded into the currently configured output directory. That's deliberate: recording a replay against a new build and diffing the two trajectories is the regression-test workflow.--**Arguments:**--- `delay_ms` (integer, optional): Milliseconds to sleep between turns, for human-observable pacing. Default 500.-- `dir` (string, required): Trajectory directory previously written by `set_recording`. Absolute or ~-rooted.-- `stop_on_error` (boolean, optional): Stop replay on the first tool-call error. Default true — set false to best-effort through the full trajectory.--```json-{"dir":"~/cua-trajectories/demo1"}-```--## Configuration tools--### set_config--Update cua-driver-rs configuration. Changes to capture_mode and max_image_dimension take effect immediately. The experimental_pip keys are persisted to ~/.cua-driver/config.json and take effect on the next daemon restart (the PiP backend is initialised once at startup).--**Per-session isolation (daemon-proxy path).** The daemon is one shared process: every `cua-driver mcp` client shares its state. To stop concurrent MCP sessions from clobbering each other (and the on-disk default), `set_config` from a proxy-backed MCP session writes an **in-memory, session-scoped override** — it does NOT touch the global `DriverConfig` or persist to `~/.cua-driver/config.json`. `get_config` and the capture tools then resolve effective values as call-arg > session override > global default, and the override is dropped automatically when the client disconnects. Only the anonymous path — the `cua-driver config set` CLI and one-shot `cua-driver call` — writes the persisted global default that survives a daemon restart. (`capture_mode` / `max_image_dimension` session-scoping is macOS-only today; Windows/Linux still write the shared config — tracked as a follow-up.)--**Arguments:**--- `capture_mode` (string, optional): Default capture mode for get_window_state.-- `experimental_pip` (boolean, optional): Enable the experimental picture-in-picture preview window. Applies on next daemon restart.-- `experimental_pip_geometry` (string, optional): PiP window size + optional position in `WxH` or `WxH+X+Y` form (e.g. `320x200+24+24`). Applies on next daemon restart.-- `max_image_dimension` (integer, optional): Max dimension for screenshot resizing (0 = no limit).--**Sessions and per-session agent cursors (macOS).** A **session** is a-caller-declared identity for one agent run — not a property of the MCP-connection. Declare it with `start_session` (or just pass a `session` id on your-actions); the same id drives the same agent cursor, per-session config, and-recording over MCP, the CLI, or the raw socket, and follows the run across any-number of apps/windows. The cursor's colour is auto-derived from the id, so-concurrent runs are visually distinct and drawn simultaneously. **The cursor is-opt-in:** a run shows a cursor only when it declares a `session` — anonymous-calls (no `session`) execute without one. `cursor_id` is a legacy alias for-`session`. A session is reclaimed by `end_session` or an idle-TTL (default 300s,-override `CUA_DRIVER_RS_SESSION_IDLE_TTL_SECS`); a late in-flight command after-teardown cannot resurrect the cursor (render-side tombstone keyed on the id). For-concurrent runs/subagents, give each its own `session` (and pass-`creates_new_application_instance:true` to `launch_app` so they don't share a-window). Per-session config + recording fall back to connection-scoped cleanup-when no `session` is declared. (Windows/Linux have no overlay cursor today; the-session identity still scopes their per-session config + recording.)--### start_session--Declare a session — a named, color-coded identity for the current agent run. Pass a stable `session` id; the agent cursor, per-session config, and recording all key on it, and it follows the run across apps/windows. The cursor appears on the session's first action. Idempotent (re-calling refreshes the idle-TTL). End it with `end_session` or let the idle-TTL reclaim it.--**Arguments:**--- `session` (string, required): Stable session id for this run (e.g. `"research-run-1"`).--### end_session--End a session declared with `start_session`: removes its agent cursor, stops any recording it owns, and clears its per-session config. Call when a run finishes so its cursor doesn't linger. Idempotent.--**Arguments:**--- `session` (string, required): The session id to end.--### set_agent_cursor_enabled--Show or hide the agent cursor overlay for a cursor instance. The overlay is **ON by default** and each MCP session automatically owns its own cursor — you do not need to call this to make the cursor appear; use it only to hide (`enabled:false`) or re-show (`enabled:true`) it. With no `cursor_id`, this targets the calling session's own cursor (see the per-session note above).--> **Visibility caveat (AX runs).** On a pure accessibility-action run (clicking by `element_index`), the session cursor seeds on-screen and pulses on its very first action rather than playing a long glide, so it is easy to miss in a screen recording. For a clearly *gliding* cursor in a demo, issue a pixel `click({pid,x,y})` or a `move_agent_cursor` first to put the cursor on-screen; subsequent AX clicks then glide normally.--**Arguments:**--- `cursor_id` (string, optional): Cursor instance. Default: 'default'.-- `enabled` (boolean, required): true = show, false = hide.--```json-{"enabled":false}-```--### set_agent_cursor_motion--Configure the visual appearance and motion curve of an agent cursor instance.--Appearance (multi-cursor customization):-- cursor_id: instance name (default='default')-- cursor_icon: built-in ('arrow','crosshair','hand','dot') or PNG/SVG file path-- cursor_color: hex color e.g. '#00FFFF' or CSS name-- cursor_label: short text shown near the cursor-- cursor_size: dot radius in points (default=16)-- cursor_opacity: 0.0–1.0 (default=0.85)--Motion curve (Bezier path shape):-- start_handle: departure control-point fraction [0,1]. Default 0.3-- end_handle: arrival control-point fraction [0,1]. Default 0.3-- arc_size: perpendicular deflection as fraction of path length [0,1]. Default 0.25-- arc_flow: asymmetry [-1,1]; positive bulges toward destination. Default 0.0-- spring: settle damping [0.3,1.0]; 1.0=no overshoot. Default 0.72-- glide_duration_ms: flight duration per move [50,5000]. Default 160-- dwell_after_click_ms: pause after click ripple [0,5000]. Default 80-- idle_hide_ms: auto-hide delay [0,60000]; 0=never. Default 20000--**Arguments:**--- `arc_flow` (number, optional): Asymmetry bias in [-1, 1]. Default 0.0.-- `arc_size` (number, optional): Arc deflection as fraction of path length [0, 1]. Default 0.25.-- `cursor_color` (string, optional): Hex color (e.g. '#00FFFF') or CSS color name.-- `cursor_icon` (string, optional): Built-in icon name or file path to PNG/SVG.-- `cursor_id` (string, optional): Cursor instance name. Default: 'default'.-- `cursor_label` (string, optional): Short label near the cursor dot.-- `cursor_opacity` (number, optional): Opacity 0.0–1.0. Default: 0.85.-- `cursor_size` (number, optional): Dot radius in points. Default: 16.-- `dwell_after_click_ms` (number, optional): Pause after click ripple in ms. Default 80.-- `end_handle` (number, optional): End-handle fraction in [0, 1]. Default 0.3.-- `glide_duration_ms` (number, optional): Flight duration per move in ms. Default 160.-- `idle_hide_ms` (number, optional): Auto-hide delay in ms. 0 = never hide. Default 20000.-- `spring` (number, optional): Settle damping in [0.3, 1.0]. Default 0.72.-- `start_handle` (number, optional): Start-handle fraction in [0, 1]. Default 0.3.--### set_agent_cursor_style--Update the visual style of the agent cursor overlay.--- gradient_colors: array of CSS hex strings (e.g. ["#FF0000","#0000FF"]) used as the arrow fill gradient from tip to tail. Empty array reverts to the default palette colours.-- bloom_color: hex string for the radial halo/bloom behind the cursor (e.g. "#00FFFF"). Empty string reverts to the default.-- image_path: path to a PNG, JPEG, SVG, or ICO file to use as the cursor icon instead of the default gradient arrow. Empty string reverts to the procedural arrow.-All parameters are optional; omit any you do not want to change.--**Arguments:**--- `bloom_color` (string, optional): Hex bloom/halo colour (e.g. '#00FFFF'). '' = revert to default.-- `cursor_id` (string, optional): Cursor instance. Default: 'default'.-- `gradient_colors` (array of string, optional): CSS hex gradient stops tip→tail. [] = revert to default.-- `image_path` (string, optional): Path to PNG/JPEG/SVG/ICO cursor image. '' = revert to arrow.--## Maintenance tools--### check_permissions--Report TCC permission status for Accessibility and Screen Recording. By default also raises the system permission dialogs for any missing grants — Apple's request APIs are no-ops when the grant is already active, so this is safe to call repeatedly. Pass {"prompt": false} for a purely read-only status check.--**Arguments:**--- `prompt` (boolean, optional): Raise the system permission prompts for missing grants. Default true.--### check_for_update--Check whether a newer cua-driver-rs release is available on GitHub. Returns the current and latest versions, an `update_available` boolean, the install one-liner, and the release notes URL. Read-only — never installs. Mirror of `cua-driver check-update --json`.--**Arguments:** none.diff --git a/docs/content/docs/cua-driver/reference/meta.json b/docs/content/docs/cua-driver/reference/meta.jsondeleted file mode 100644index e96d110716..0000000000--- a/docs/content/docs/cua-driver/reference/meta.json+++ /dev/null@@ -1,6 +0,0 @@-{- "title": "Reference",- "description": "CLI and MCP reference documentation",- "icon": "FileText",- "pages": ["cli-reference", "mcp-tools", "limits", "changelog"]-}diff --git a/docs/content/docs/cua/examples/ai-models/gemini-complex-ui-navigation.mdx b/docs/content/docs/cua/examples/ai-models/gemini-complex-ui-navigation.mdxdeleted file mode 100644index 853e968113..0000000000--- a/docs/content/docs/cua/examples/ai-models/gemini-complex-ui-navigation.mdx+++ /dev/null@@ -1,634 +0,0 @@-----title: GUI Grounding with Gemini 3-description: Using Google's Gemini 3 with OmniParser for Advanced GUI Grounding Tasks------import { Step, Steps } from 'fumadocs-ui/components/steps';-import { Tab, Tabs } from 'fumadocs-ui/components/tabs';-import { Callout } from 'fumadocs-ui/components/callout';--## Overview--This example demonstrates how to use Google's Gemini 3 models with OmniParser for complex GUI grounding tasks. Gemini 3 Pro achieves exceptional performance on the [ScreenSpot-Pro benchmark](https://github.com/likaixin2000/ScreenSpot-Pro-GUI-Grounding) with a **72.7% accuracy** (compared to Claude Sonnet 4.5's 36.2%), making it ideal for precise UI element location and complex navigation tasks.--<img- src="/docs/img/grounding-with-gemini3.gif"- alt="Demo of Gemini 3 with OmniParser performing complex GUI navigation tasks"- width="800px"-/>--<Callout type="info" title="Why Gemini 3 for UI Navigation?">- According to [Google's Gemini 3 announcement](https://blog.google/products/gemini/gemini-3/),- Gemini 3 Pro achieves: - **72.7%** on ScreenSpot-Pro (vs. Gemini 2.5 Pro's 11.4%) -- Industry-leading performance on complex UI navigation tasks - Advanced multimodal understanding- for high-resolution screens-</Callout>--### What You'll Build--This guide shows how to:--- Set up Vertex AI with proper authentication-- Use OmniParser with Gemini 3 for GUI element detection-- Leverage Gemini 3-specific features like `thinking_level` and `media_resolution`-- Create agents that can perform complex multi-step UI interactions-------<Steps>--<Step>--### Set Up Google Cloud and Vertex AI--Before using Gemini 3 models, you need to enable Vertex AI in Google Cloud Console.--#### 1. Create a Google Cloud Project--1. Go to [Google Cloud Console](https://console.cloud.google.com/)-2. Click **Select a project** → **New Project**-3. Enter a project name and click **Create**-4. Note your **Project ID** (you'll need this later)--#### 2. Enable Vertex AI API--1. Navigate to [Vertex AI API](https://console.cloud.google.com/apis/library/aiplatform.googleapis.com)-2. Select your project-3. Click **Enable**--#### 3. Enable Billing--1. Go to [Billing](https://console.cloud.google.com/billing)-2. Link a billing account to your project-3. Vertex AI offers a [free tier](https://cloud.google.com/vertex-ai/pricing) for testing--#### 4. Create a Service Account--1. Go to [IAM & Admin > Service Accounts](https://console.cloud.google.com/iam-admin/serviceaccounts)-2. Click **Create Service Account**-3. Enter a name (e.g., "cua-gemini-agent")-4. Click **Create and Continue**-5. Grant the **Vertex AI User** role-6. Click **Done**--#### 5. Create and Download Service Account Key--1. Click on your newly created service account-2. Go to **Keys** tab-3. Click **Add Key** → **Create new key**-4. Select **JSON** format-5. Click **Create** (the key file will download automatically)-6. **Important**: Store this key file securely! It contains credentials for accessing your Google Cloud resources--<Callout type="warn">- Never commit your service account JSON key to version control! Add it to `.gitignore` immediately.-</Callout>--</Step>--<Step>--### Install Dependencies--Install the required packages for OmniParser and Gemini 3:--Create a `requirements.txt` file:--```text-cua-cua-som # OmniParser for GUI element detection-litellm>=1.0.0-python-dotenv>=1.0.0-google-cloud-aiplatform>=1.70.0-```--Install the dependencies:--```bash-pip install -r requirements.txt-```--</Step>--<Step>--### Configure Environment Variables--Create a `.env` file in your project root:--```text-# Google Cloud / Vertex AI credentials-GOOGLE_CLOUD_PROJECT=your-project-id-GOOGLE_APPLICATION_CREDENTIALS=/path/to/your-service-account-key.json--# Cua credentials (for cloud sandboxes)-CUA_API_KEY=sk_cua-api01...-CUA_SANDBOX_NAME=your-sandbox-name-```--Replace the values:--- `your-project-id`: Your Google Cloud Project ID from Step 1-- `/path/to/your-service-account-key.json`: Path to the JSON key file you downloaded-- `sk_cua-api01...`: Your Cua API key from the [Cua dashboard](https://cua.ai)-- `your-sandbox-name`: Your sandbox name (if using cloud sandboxes)--</Step>--<Step>--### Create Your Complex UI Navigation Script--Create a Python file (e.g., `gemini_ui_navigation.py`):--<Tabs items={['Cloud Sandbox', 'Linux on Docker', 'macOS Sandbox']}>- <Tab value="Cloud Sandbox">--```python-import asyncio-import logging-import os-import signal-import traceback--from cua import ComputerAgent, Sandbox, Image-from dotenv import load_dotenv--logging.basicConfig(level=logging.INFO)-logger = logging.getLogger(__name__)--def handle_sigint(sig, frame):- print("\n\nExecution interrupted by user. Exiting gracefully...")- exit(0)--async def complex_ui_navigation():- """- Demonstrate Gemini 3's exceptional UI grounding capabilities- with complex, multi-step navigation tasks.- """- try:- async with Sandbox.create(- Image.linux(),- name=os.environ["CUA_SANDBOX_NAME"],- api_key=os.environ["CUA_API_KEY"],- verbosity=logging.INFO,- ) as computer:-- agent = ComputerAgent(- # Use OmniParser with Gemini 3 Pro for optimal GUI grounding- model="omniparser+vertex_ai/gemini-3-pro-preview",- tools=[computer],- only_n_most_recent_images=3,- verbosity=logging.INFO,- trajectory_dir="trajectories",- use_prompt_caching=False,- max_trajectory_budget=5.0,- # Gemini 3-specific parameters- thinking_level="high", # Enables deeper reasoning (vs "low")- media_resolution="high", # High-resolution image processing (vs "low" or "medium")- )-- # Complex GUI grounding tasks inspired by ScreenSpot-Pro benchmark- # These test precise element location in professional UIs- tasks = [- # Task 1: GitHub repository navigation- {- "instruction": (- "Go to github.com/trycua/cua. "- "Find and click on the 'Issues' tab. "- "Then locate and click on the search box within the issues page "- "(not the global GitHub search). "- "Type 'omniparser' and press Enter."- ),- "description": "Tests precise UI element distinction in a complex interface",- },-- # Task 2: Search for and install Visual Studio Code- {- "instruction": (- "Open your system's app store (e.g., Microsoft Store). "- "Search for 'Visual Studio Code'. "- "In the search results, select 'Visual Studio Code'. "- "Click on 'Install' or 'Get' to begin the installation. "- "If prompted, accept any permissions or confirm the installation. "- "Wait for Visual Studio Code to finish installing."- ),- "description": "Tests the ability to search for an application and complete its installation through a step-by-step app store workflow.",- },- ]-- history = []-- for i, task_info in enumerate(tasks, 1):- task = task_info["instruction"]- print(f"\n{'='*60}")- print(f"[Task {i}/{len(tasks)}] {task_info['description']}")- print(f"{'='*60}")- print(f"\nInstruction: {task}\n")-- # Add user message to history- history.append({"role": "user", "content": task})-- # Run agent with conversation history- async for result in agent.run(history, stream=False):- history += result.get("output", [])-- # Print output for debugging- for item in result.get("output", []):- if item.get("type") == "message":- content = item.get("content", [])- for content_part in content:- if content_part.get("text"):- logger.info(f"Agent: {content_part.get('text')}")- elif item.get("type") == "computer_call":- action = item.get("action", {})- action_type = action.get("type", "")- logger.debug(f"Computer Action: {action_type}")-- print(f"\n✅ Task {i}/{len(tasks)} completed")-- print("\n🎉 All complex UI navigation tasks completed successfully!")-- except Exception as e:- logger.error(f"Error in complex_ui_navigation: {e}")- traceback.print_exc()- raise--def main():- try:- load_dotenv()-- # Validate required environment variables- required_vars = [- "GOOGLE_CLOUD_PROJECT",- "GOOGLE_APPLICATION_CREDENTIALS",- "CUA_API_KEY",- "CUA_SANDBOX_NAME",- ]-- missing_vars = [var for var in required_vars if not os.environ.get(var)]- if missing_vars:- raise RuntimeError(- f"Missing required environment variables: {', '.join(missing_vars)}\n"- f"Please check your .env file and ensure all keys are set.\n"- f"See the setup guide for details on configuring Vertex AI credentials."- )-- signal.signal(signal.SIGINT, handle_sigint)-- asyncio.run(complex_ui_navigation())-- except Exception as e:- logger.error(f"Error running automation: {e}")- traceback.print_exc()--if __name__ == "__main__":- main()-```-- </Tab>- <Tab value="Linux on Docker">--```python-import asyncio-import logging-import os-import signal-import traceback--from cua import ComputerAgent, Sandbox, Image-from dotenv import load_dotenv--logging.basicConfig(level=logging.INFO)-logger = logging.getLogger(__name__)--def handle_sigint(sig, frame):- print("\n\nExecution interrupted by user. Exiting gracefully...")- exit(0)--async def complex_ui_navigation():- """- Demonstrate Gemini 3's exceptional UI grounding capabilities- with complex, multi-step navigation tasks.- """- try:- async with Sandbox.ephemeral(- Image.linux(),- local=True,- verbosity=logging.INFO,- ) as computer:-- agent = ComputerAgent(- # Use OmniParser with Gemini 3 Pro for optimal GUI grounding- model="omniparser+vertex_ai/gemini-3-pro-preview",- tools=[computer],- only_n_most_recent_images=3,- verbosity=logging.INFO,- trajectory_dir="trajectories",- use_prompt_caching=False,- max_trajectory_budget=5.0,- # Gemini 3-specific parameters- thinking_level="high", # Enables deeper reasoning (vs "low")- media_resolution="high", # High-resolution image processing (vs "low" or "medium")- )-- # Complex GUI grounding tasks inspired by ScreenSpot-Pro benchmark- tasks = [- {- "instruction": (- "Go to github.com/trycua/cua. "- "Find and click on the 'Issues' tab. "- "Then locate and click on the search box within the issues page "- "(not the global GitHub search). "- "Type 'omniparser' and press Enter."- ),- "description": "Tests precise UI element distinction in a complex interface",- },- ]-- history = []-- for i, task_info in enumerate(tasks, 1):- task = task_info["instruction"]- print(f"\n{'='*60}")- print(f"[Task {i}/{len(tasks)}] {task_info['description']}")- print(f"{'='*60}")- print(f"\nInstruction: {task}\n")-- history.append({"role": "user", "content": task})-- async for result in agent.run(history, stream=False):- history += result.get("output", [])-- for item in result.get("output", []):- if item.get("type") == "message":- content = item.get("content", [])- for content_part in content:- if content_part.get("text"):- logger.info(f"Agent: {content_part.get('text')}")- elif item.get("type") == "computer_call":- action = item.get("action", {})- action_type = action.get("type", "")- logger.debug(f"Computer Action: {action_type}")-- print(f"\n✅ Task {i}/{len(tasks)} completed")-- print("\n🎉 All complex UI navigation tasks completed successfully!")-- except Exception as e:- logger.error(f"Error in complex_ui_navigation: {e}")- traceback.print_exc()- raise--def main():- try:- load_dotenv()-- required_vars = [- "GOOGLE_CLOUD_PROJECT",- "GOOGLE_APPLICATION_CREDENTIALS",- ]-- missing_vars = [var for var in required_vars if not os.environ.get(var)]- if missing_vars:- raise RuntimeError(- f"Missing required environment variables: {', '.join(missing_vars)}\n"- f"Please check your .env file."- )-- signal.signal(signal.SIGINT, handle_sigint)-- asyncio.run(complex_ui_navigation())-- except Exception as e:- logger.error(f"Error running automation: {e}")- traceback.print_exc()--if __name__ == "__main__":- main()-```-- </Tab>- <Tab value="macOS Sandbox">--```python-import asyncio-import logging-import os-import signal-import traceback--from cua import ComputerAgent, Sandbox, Image-from dotenv import load_dotenv--logging.basicConfig(level=logging.INFO)-logger = logging.getLogger(__name__)--def handle_sigint(sig, frame):- print("\n\nExecution interrupted by user. Exiting gracefully...")- exit(0)--async def complex_ui_navigation():- """- Demonstrate Gemini 3's exceptional UI grounding capabilities- with complex, multi-step navigation tasks.- """- try:- async with Sandbox.create(- Image.macos(),- local=True,- name="macos-sequoia-cua:latest",- verbosity=logging.INFO,- ) as computer:-- agent = ComputerAgent(- # Use OmniParser with Gemini 3 Pro for optimal GUI grounding- model="omniparser+vertex_ai/gemini-3-pro-preview",- tools=[computer],- only_n_most_recent_images=3,- verbosity=logging.INFO,- trajectory_dir="trajectories",- use_prompt_caching=False,- max_trajectory_budget=5.0,- # Gemini 3-specific parameters- thinking_level="high", # Enables deeper reasoning (vs "low")- media_resolution="high", # High-resolution image processing (vs "low" or "medium")- )-- # Complex GUI grounding tasks inspired by ScreenSpot-Pro benchmark- tasks = [- {- "instruction": (- "Go to github.com/trycua/cua. "- "Find and click on the 'Issues' tab. "- "Then locate and click on the search box within the issues page "- "(not the global GitHub search). "- "Type 'omniparser' and press Enter."- ),- "description": "Tests precise UI element distinction in a complex interface",- },- ]-- history = []-- for i, task_info in enumerate(tasks, 1):- task = task_info["instruction"]- print(f"\n{'='*60}")- print(f"[Task {i}/{len(tasks)}] {task_info['description']}")- print(f"{'='*60}")- print(f"\nInstruction: {task}\n")-- history.append({"role": "user", "content": task})-- async for result in agent.run(history, stream=False):- history += result.get("output", [])-- for item in result.get("output", []):- if item.get("type") == "message":- content = item.get("content", [])- for content_part in content:- if content_part.get("text"):- logger.info(f"Agent: {content_part.get('text')}")- elif item.get("type") == "computer_call":- action = item.get("action", {})- action_type = action.get("type", "")- logger.debug(f"Computer Action: {action_type}")-- print(f"\n✅ Task {i}/{len(tasks)} completed")-- print("\n🎉 All complex UI navigation tasks completed successfully!")-- except Exception as e:- logger.error(f"Error in complex_ui_navigation: {e}")- traceback.print_exc()- raise--def main():- try:- load_dotenv()-- required_vars = [- "GOOGLE_CLOUD_PROJECT",- "GOOGLE_APPLICATION_CREDENTIALS",- ]-- missing_vars = [var for var in required_vars if not os.environ.get(var)]- if missing_vars:- raise RuntimeError(- f"Missing required environment variables: {', '.join(missing_vars)}\n"- f"Please check your .env file."- )-- signal.signal(signal.SIGINT, handle_sigint)-- asyncio.run(complex_ui_navigation())-- except Exception as e:- logger.error(f"Error running automation: {e}")- traceback.print_exc()--if __name__ == "__main__":- main()-```-- </Tab>-</Tabs>--</Step>--<Step>--### Run Your Script--Execute your complex UI navigation automation:--```bash-python gemini_ui_navigation.py-```--The agent will:--1. Navigate to GitHub and locate specific UI elements-2. Distinguish between similar elements (e.g., global search vs. issues search)-3. Perform multi-step interactions with visual feedback-4. Use Gemini 3's advanced reasoning for precise element grounding--Monitor the output to see the agent's progress through each task.--</Step>--</Steps>-------## Understanding Gemini 3-Specific Parameters--### `thinking_level`--Controls the amount of internal reasoning the model performs:--- `"high"`: Deeper reasoning, better for complex UI navigation (recommended for ScreenSpot-like tasks)-- `"low"`: Faster responses, suitable for simpler tasks--### `media_resolution`--Controls vision processing for multimodal inputs:--- `"high"`: Best for complex UIs with many small elements (recommended)-- `"medium"`: Balanced quality and speed-- `"low"`: Faster processing for simple interfaces--<Callout type="info">- For tasks requiring precise GUI element location (like ScreenSpot-Pro), use- `thinking_level="high"` and `media_resolution="high"` for optimal performance.-</Callout>-------## Benchmark Performance--Gemini 3 Pro's performance on ScreenSpot-Pro demonstrates its exceptional UI grounding capabilities:--| Model | ScreenSpot-Pro Score |-| ----------------- | -------------------- |-| **Gemini 3 Pro** | **72.7%** |-| Claude Sonnet 4.5 | 36.2% |-| Gemini 2.5 Pro | 11.4% |-| GPT-5.1 | 3.5% |--This makes Gemini 3 the ideal choice for complex UI navigation, element detection, and professional GUI automation tasks.-------## Troubleshooting--### Authentication Issues--If you encounter authentication errors:--1. Verify your service account JSON key path is correct-2. Ensure the service account has the **Vertex AI User** role-3. Check that the Vertex AI API is enabled in your project-4. Confirm your `GOOGLE_CLOUD_PROJECT` matches your actual project ID--### "Vertex AI API not enabled" Error--Run this command to enable the API:--```bash-gcloud services enable aiplatform.googleapis.com --project=YOUR_PROJECT_ID-```--### Billing Issues--Ensure billing is enabled for your Google Cloud project. Visit the [Billing section](https://console.cloud.google.com/billing) to verify.-------## Next Steps--- Learn more about [OmniParser agent loops](/cua/guide/fundamentals/agent-loops)-- Explore [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing)-- Read about [ScreenSpot-Pro benchmark](https://github.com/likaixin2000/ScreenSpot-Pro-GUI-Grounding)-- Check out [Google's Gemini 3 announcement](https://blog.google/products/gemini/gemini-3/)-- Join our [Discord community](https://discord.com/invite/mVnXXpdE85) for helpdiff --git a/docs/content/docs/cua/examples/ai-models/meta.json b/docs/content/docs/cua/examples/ai-models/meta.jsondeleted file mode 100644index 0257e19ee6..0000000000--- a/docs/content/docs/cua/examples/ai-models/meta.json+++ /dev/null@@ -1,6 +0,0 @@-{- "title": "AI Models",- "description": "Examples using different AI models",- "icon": "Brain",- "pages": ["gemini-complex-ui-navigation"]-}diff --git a/docs/content/docs/cua/examples/automation/form-filling.mdx b/docs/content/docs/cua/examples/automation/form-filling.mdxdeleted file mode 100644index 7bc49c966c..0000000000--- a/docs/content/docs/cua/examples/automation/form-filling.mdx+++ /dev/null@@ -1,493 +0,0 @@-----title: PDF to Form Automation-description: Enhance and Automate Interactions Between Form Filling and Local File Systems------import { Step, Steps } from 'fumadocs-ui/components/steps';-import { Tab, Tabs } from 'fumadocs-ui/components/tabs';--## Overview--Cua can be used to automate interactions between form filling and local file systems over any operating system. Cua let's you interact with all the elements of a web page and local file systems to integrate between the two.--This preset usecase uses [Cua Computer Framework](/cua/reference/computer-sdk) to interact with a web page and local file systems along with [Agent Loops](/cua/guide/fundamentals/agent-loops) to run the agent in a loop with message history.-------<Steps>--<Step>--### Set Up Your Environment--First, install the required dependencies:--Create a `requirements.txt` file:--```text-cua-python-dotenv>=1.0.0-```--Install the dependencies using uv (recommended):--```bash-uv pip install -r requirements.txt-```--Or using pip:--```bash-pip install -r requirements.txt-```--Create a `.env` file with your API keys:--```text-ANTHROPIC_API_KEY=your-anthropic-api-key-CUA_API_KEY=sk_cua-api01...-```--</Step>--<Step>--### Create Your Form Filling Script--Create a Python file (e.g., `form_filling.py`) and select your environment:--<Tabs items={['Cloud Sandbox', 'Linux on Docker', 'macOS Sandbox', 'Windows Sandbox']}>- <Tab value="Cloud Sandbox">--```python-import asyncio-import logging-import os-import signal-import traceback--from cua import ComputerAgent, Sandbox, Image-from dotenv import load_dotenv--logging.basicConfig(level=logging.INFO)-logger = logging.getLogger(__name__)--def handle_sigint(sig, frame):- print("\n\nExecution interrupted by user. Exiting gracefully...")- exit(0)--async def fill_application():- try:- async with Sandbox.create(- Image.linux(),- name="your-sandbox-name", # Replace with your sandbox name- api_key=os.environ["CUA_API_KEY"],- verbosity=logging.INFO,- ) as computer:-- agent = ComputerAgent(- model="cua/anthropic/claude-sonnet-4.5",- tools=[computer],- only_n_most_recent_images=3,- verbosity=logging.INFO,- trajectory_dir="trajectories",- use_prompt_caching=True,- max_trajectory_budget=5.0,- )-- tasks = [- "Visit https://www.overleaf.com/latex/templates/jakes-resume/syzfjbzwjncs.pdf and download the pdf.",- "Visit https://form.jotform.com/252881246782264 and fill the form from the information in the pdf."- ]-- history = []-- for i, task in enumerate(tasks, 1):- print(f"\n[Task {i}/{len(tasks)}] {task}")-- # Add user message to history- history.append({"role": "user", "content": task})-- # Run agent with conversation history- async for result in agent.run(history, stream=False):- history += result.get("output", [])-- # Print output for debugging- for item in result.get("output", []):- if item.get("type") == "message":- content = item.get("content", [])- for content_part in content:- if content_part.get("text"):- logger.info(f"Agent: {content_part.get('text')}")- elif item.get("type") == "computer_call":- action = item.get("action", {})- action_type = action.get("type", "")- logger.debug(f"Computer Action: {action_type}")-- print(f"✅ Task {i}/{len(tasks)} completed")-- print("\n🎉 All tasks completed successfully!")-- except Exception as e:- logger.error(f"Error in fill_application: {e}")- traceback.print_exc()- raise--def main():- try:- load_dotenv()-- if "ANTHROPIC_API_KEY" not in os.environ:- raise RuntimeError(- "Please set the ANTHROPIC_API_KEY environment variable.\n"- "You can add it to a .env file in the project root."- )-- if "CUA_API_KEY" not in os.environ:- raise RuntimeError(- "Please set the CUA_API_KEY environment variable.\n"- "You can add it to a .env file in the project root."- )-- signal.signal(signal.SIGINT, handle_sigint)-- asyncio.run(fill_application())-- except Exception as e:- logger.error(f"Error running automation: {e}")- traceback.print_exc()--if __name__ == "__main__":- main()-```-- </Tab>- <Tab value="Linux on Docker">--```python-import asyncio-import logging-import os-import signal-import traceback--from cua import ComputerAgent, Sandbox, Image-from dotenv import load_dotenv--logging.basicConfig(level=logging.INFO)-logger = logging.getLogger(__name__)--def handle_sigint(sig, frame):- print("\n\nExecution interrupted by user. Exiting gracefully...")- exit(0)--async def fill_application():- try:- async with Sandbox.ephemeral(- Image.linux(),- local=True,- verbosity=logging.INFO,- ) as computer:-- agent = ComputerAgent(- model="cua/anthropic/claude-sonnet-4.5",- tools=[computer],- only_n_most_recent_images=3,- verbosity=logging.INFO,- trajectory_dir="trajectories",- use_prompt_caching=True,- max_trajectory_budget=5.0,- )-- tasks = [- "Visit https://www.overleaf.com/latex/templates/jakes-resume/syzfjbzwjncs.pdf and download the pdf.",- "Visit https://form.jotform.com/252881246782264 and fill the form from the information in the pdf."- ]-- history = []-- for i, task in enumerate(tasks, 1):- print(f"\n[Task {i}/{len(tasks)}] {task}")-- # Add user message to history- history.append({"role": "user", "content": task})-- # Run agent with conversation history- async for result in agent.run(history, stream=False):- history += result.get("output", [])-- # Print output for debugging- for item in result.get("output", []):- if item.get("type") == "message":- content = item.get("content", [])- for content_part in content:- if content_part.get("text"):- logger.info(f"Agent: {content_part.get('text')}")- elif item.get("type") == "computer_call":- action = item.get("action", {})- action_type = action.get("type", "")- logger.debug(f"Computer Action: {action_type}")-- print(f"✅ Task {i}/{len(tasks)} completed")-- print("\n🎉 All tasks completed successfully!")-- except Exception as e:- logger.error(f"Error in fill_application: {e}")- traceback.print_exc()- raise--def main():- try:- load_dotenv()-- if "ANTHROPIC_API_KEY" not in os.environ:- raise RuntimeError(- "Please set the ANTHROPIC_API_KEY environment variable.\n"- "You can add it to a .env file in the project root."- )-- signal.signal(signal.SIGINT, handle_sigint)-- asyncio.run(fill_application())-- except Exception as e:- logger.error(f"Error running automation: {e}")- traceback.print_exc()--if __name__ == "__main__":- main()-```-- </Tab>- <Tab value="macOS Sandbox">--```python-import asyncio-import logging-import os-import signal-import traceback--from cua import ComputerAgent, Sandbox, Image-from dotenv import load_dotenv--logging.basicConfig(level=logging.INFO)-logger = logging.getLogger(__name__)--def handle_sigint(sig, frame):- print("\n\nExecution interrupted by user. Exiting gracefully...")- exit(0)--async def fill_application():- try:- async with Sandbox.create(- Image.macos(),- local=True,- name="macos-sequoia-cua:latest",- verbosity=logging.INFO,- ) as computer:-- agent = ComputerAgent(- model="cua/anthropic/claude-sonnet-4.5",- tools=[computer],- only_n_most_recent_images=3,- verbosity=logging.INFO,- trajectory_dir="trajectories",- use_prompt_caching=True,- max_trajectory_budget=5.0,- )-- tasks = [- "Visit https://www.overleaf.com/latex/templates/jakes-resume/syzfjbzwjncs.pdf and download the pdf.",- "Visit https://form.jotform.com/252881246782264 and fill the form from the information in the pdf."- ]-- history = []-- for i, task in enumerate(tasks, 1):- print(f"\n[Task {i}/{len(tasks)}] {task}")-- # Add user message to history- history.append({"role": "user", "content": task})-- # Run agent with conversation history- async for result in agent.run(history, stream=False):- history += result.get("output", [])-- # Print output for debugging- for item in result.get("output", []):- if item.get("type") == "message":- content = item.get("content", [])- for content_part in content:- if content_part.get("text"):- logger.info(f"Agent: {content_part.get('text')}")- elif item.get("type") == "computer_call":- action = item.get("action", {})- action_type = action.get("type", "")- logger.debug(f"Computer Action: {action_type}")-- print(f"✅ Task {i}/{len(tasks)} completed")-- print("\n🎉 All tasks completed successfully!")-- except Exception as e:- logger.error(f"Error in fill_application: {e}")- traceback.print_exc()- raise--def main():- try:- load_dotenv()-- if "ANTHROPIC_API_KEY" not in os.environ:- raise RuntimeError(- "Please set the ANTHROPIC_API_KEY environment variable.\n"- "You can add it to a .env file in the project root."- )-- signal.signal(signal.SIGINT, handle_sigint)-- asyncio.run(fill_application())-- except Exception as e:- logger.error(f"Error running automation: {e}")- traceback.print_exc()--if __name__ == "__main__":- main()-```-- </Tab>- <Tab value="Windows Sandbox">--```python-import asyncio-import logging-import os-import signal-import traceback--from cua import ComputerAgent, Sandbox, Image-from dotenv import load_dotenv--logging.basicConfig(level=logging.INFO)-logger = logging.getLogger(__name__)--def handle_sigint(sig, frame):- print("\n\nExecution interrupted by user. Exiting gracefully...")- exit(0)--async def fill_application():- try:- async with Sandbox.create(- Image.windows(),- local=True,- verbosity=logging.INFO,- ) as computer:-- agent = ComputerAgent(- model="cua/anthropic/claude-sonnet-4.5",- tools=[computer],- only_n_most_recent_images=3,- verbosity=logging.INFO,- trajectory_dir="trajectories",- use_prompt_caching=True,- max_trajectory_budget=5.0,- )-- tasks = [- "Visit https://www.overleaf.com/latex/templates/jakes-resume/syzfjbzwjncs.pdf and download the pdf.",- "Visit https://form.jotform.com/252881246782264 and fill the form from the information in the pdf."- ]-- history = []-- for i, task in enumerate(tasks, 1):- print(f"\n[Task {i}/{len(tasks)}] {task}")-- # Add user message to history- history.append({"role": "user", "content": task})-- # Run agent with conversation history- async for result in agent.run(history, stream=False):- history += result.get("output", [])-- # Print output for debugging- for item in result.get("output", []):- if item.get("type") == "message":- content = item.get("content", [])- for content_part in content:- if content_part.get("text"):- logger.info(f"Agent: {content_part.get('text')}")- elif item.get("type") == "computer_call":- action = item.get("action", {})- action_type = action.get("type", "")- logger.debug(f"Computer Action: {action_type}")-- print(f"✅ Task {i}/{len(tasks)} completed")-- print("\n🎉 All tasks completed successfully!")-- except Exception as e:- logger.error(f"Error in fill_application: {e}")- traceback.print_exc()- raise--def main():- try:- load_dotenv()-- if "ANTHROPIC_API_KEY" not in os.environ:- raise RuntimeError(- "Please set the ANTHROPIC_API_KEY environment variable.\n"- "You can add it to a .env file in the project root."- )-- signal.signal(signal.SIGINT, handle_sigint)-- asyncio.run(fill_application())-- except Exception as e:- logger.error(f"Error running automation: {e}")- traceback.print_exc()--if __name__ == "__main__":- main()-```-- </Tab>-</Tabs>--</Step>--<Step>--### Run Your Script--Execute your form filling automation:--```bash-python form_filling.py-```--The agent will:--1. Download the PDF resume from Overleaf-2. Extract information from the PDF-3. Fill out the JotForm with the extracted information--Monitor the output to see the agent's progress through each task.--</Step>--</Steps>-------## Next Steps--- Learn more about [Cua Computer Framework](/cua/reference/computer-sdk) and [Computer Commands](/cua/reference/computer-sdk)-- Read about [Agent Loops](/cua/guide/fundamentals/agent-loops), [Tools](/cua/guide/advanced/custom-tools), and [Supported Model Providers](/cua/guide/fundamentals/vlms)-- Experiment with different [Models and Providers](/cua/guide/fundamentals/vlms)-- Join our [Discord community](https://discord.com/invite/mVnXXpdE85) for helpdiff --git a/docs/content/docs/cua/examples/automation/meta.json b/docs/content/docs/cua/examples/automation/meta.jsondeleted file mode 100644index 2a57724c0c..0000000000--- a/docs/content/docs/cua/examples/automation/meta.json+++ /dev/null@@ -1,7 +0,0 @@-{- "title": "Automation",- "description": "Automate repetitive tasks with Cua",- "icon": "Workflow",- "defaultOpen": true,- "pages": ["form-filling", "post-event-contact-export"]-}diff --git a/docs/content/docs/cua/examples/automation/post-event-contact-export.mdx b/docs/content/docs/cua/examples/automation/post-event-contact-export.mdxdeleted file mode 100644index ea5493e595..0000000000--- a/docs/content/docs/cua/examples/automation/post-event-contact-export.mdx+++ /dev/null@@ -1,478 +0,0 @@-----title: Post-Event Contact Export-description: Run overnight contact extraction from LinkedIn, X, or other social platforms after networking events------import { Step, Steps } from 'fumadocs-ui/components/steps';-import { Tab, Tabs } from 'fumadocs-ui/components/tabs';--## Overview--After networking events, you need to export new connections from LinkedIn, X, or other platforms into your CRM. This automation handles it for you.--**The workflow**: Kick off the script after an event and let it run overnight. Wake up to a clean CSV ready for your CRM or email tool.--This example focuses on LinkedIn but works across platforms. It uses [Cua Computer Framework](/cua/reference/computer-sdk) to interact with web interfaces and [Agent Loops](/cua/guide/fundamentals/agent-loops) to iterate through connections with conversation history.--### Why Cua is Perfect for This--**Cua's VMs save your session data**, bypassing bot detection entirely:--- **Log in once manually** through the VM browser-- **Session persists** - you appear as a regular user, not a bot-- **No captchas** - the platform treats automation like normal browsing-- **No login code** - script doesn't handle authentication-- **Run overnight** - kick off and forget--Traditional web scraping triggers anti-bot measures immediately. Cua's approach works across all platforms.--### What You Get--The script generates two files with your extracted connections:--**CSV Export** (`linkedin_connections_20250116_143022.csv`):--```csv-first,last,role,company,met_at,linkedin-John,Smith,Software Engineer,Acme Corp,Google Devfest Toronto,https://www.linkedin.com/in/johnsmith-Sarah,Johnson,Product Manager,Tech Inc,Google Devfest Toronto,https://www.linkedin.com/in/sarahjohnson-```--**Messaging Links** (`linkedin_messaging_links_20250116_143022.txt`):--```-LinkedIn Messaging Compose Links-================================================================================--1. https://www.linkedin.com/messaging/compose/?recipient=johnsmith-2. https://www.linkedin.com/messaging/compose/?recipient=sarahjohnson-```-------<Steps>--<Step>--### Set Up Your Environment--First, install the required dependencies:--Create a `requirements.txt` file:--```text-cua-python-dotenv>=1.0.0-```--Install the dependencies using uv (recommended):--```bash-uv pip install -r requirements.txt-```--Or using pip:--```bash-pip install -r requirements.txt-```--Create a `.env` file with your API keys:--```text-ANTHROPIC_API_KEY=your-anthropic-api-key # optional, BYOK. By default, this cookbook uses the Cua VLM Router-CUA_API_KEY=sk_cua-api01...-CUA_CONTAINER_NAME=m-linux-...-```--Finally, setup your sandbox. Refer to the [quickstart guide](/cua/guide/get-started/quickstart) on how to setup the computer environment.--</Step>--<Step>--### Log Into LinkedIn Manually--**Important**: Before running the script, manually log into LinkedIn through your VM:--1. Access your VM through the Cua dashboard-2. Open a browser and navigate to LinkedIn-3. Log in with your credentials (handle any captchas manually)-4. Close the browser but leave the VM running-5. Your session is now saved and ready for automation!--This one-time manual login bypasses all bot detection.--</Step>--<Step>--### Configure and Create Your Script--Create a Python file (e.g., `contact_export.py`). You can customize:--```python-# Where you met these connections (automatically added to CSV)-MET_AT_REASON = "Google Devfest Toronto"--# Number of contacts to extract (in the main loop)-for contact_num in range(1, 21): # Change 21 to extract more/fewer contacts-```--Select your environment:--<Tabs items={['Cloud Sandbox', 'Linux on Docker', 'macOS Sandbox', 'Windows Sandbox']}>- <Tab value="Cloud Sandbox">--```python-import asyncio-import csv-import logging-import os-import signal-import traceback-from datetime import datetime--from cua import ComputerAgent, Sandbox, Image-from dotenv import load_dotenv--logging.basicConfig(level=logging.INFO)-logger = logging.getLogger(__name__)--# Configuration: Define where you met these connections-MET_AT_REASON = "Google Devfest Toronto"--def handle_sigint(sig, frame):- print("\n\nExecution interrupted by user. Exiting gracefully...")- exit(0)--def extract_public_id_from_linkedin_url(linkedin_url):- """Extract public ID from LinkedIn profile URL."""- if not linkedin_url:- return None-- url = linkedin_url.split('?')[0].rstrip('/')-- if '/in/' in url:- public_id = url.split('/in/')[-1]- return public_id-- return None--def extract_contact_from_response(result_output):- """- Extract contact information from agent's response.- Expects format:- FIRST: value- LAST: value- ROLE: value- COMPANY: value- LINKEDIN: value- """- contact = {- 'first': '',- 'last': '',- 'role': '',- 'company': '',- 'met_at': MET_AT_REASON,- 'linkedin': ''- }-- for item in result_output:- if item.get("type") == "message":- content = item.get("content", [])- for content_part in content:- text = content_part.get("text", "")- if text:- for line in text.split('\n'):- line = line.strip()- line_upper = line.upper()-- if line_upper.startswith("FIRST:"):- value = line[6:].strip()- if value and value.upper() != "N/A":- contact['first'] = value- elif line_upper.startswith("LAST:"):- value = line[5:].strip()- if value and value.upper() != "N/A":- contact['last'] = value- elif line_upper.startswith("ROLE:"):- value = line[5:].strip()- if value and value.upper() != "N/A":- contact['role'] = value- elif line_upper.startswith("COMPANY:"):- value = line[8:].strip()- if value and value.upper() != "N/A":- contact['company'] = value- elif line_upper.startswith("LINKEDIN:"):- value = line[9:].strip()- if value and value.upper() != "N/A":- contact['linkedin'] = value-- return contact--async def scrape_linkedin_connections():- """Scrape LinkedIn connections and export to CSV."""-- timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")- csv_filename = f"linkedin_connections_{timestamp}.csv"- csv_path = os.path.join(os.getcwd(), csv_filename)-- # Initialize CSV file- with open(csv_path, 'w', newline='', encoding='utf-8') as csvfile:- writer = csv.DictWriter(csvfile, fieldnames=['first', 'last', 'role', 'company', 'met_at', 'linkedin'])- writer.writeheader()-- print(f"\n🚀 Starting LinkedIn connections scraper")- print(f"📁 Output file: {csv_path}")- print(f"📍 Met at: {MET_AT_REASON}")- print("=" * 80)-- try:- async with Sandbox.create(- Image.linux(),- name=os.environ["CUA_CONTAINER_NAME"], # Your sandbox name- api_key=os.environ["CUA_API_KEY"],- verbosity=logging.INFO,- ) as computer:-- agent = ComputerAgent(- model="cua/anthropic/claude-sonnet-4.5",- tools=[computer],- only_n_most_recent_images=3,- verbosity=logging.INFO,- trajectory_dir="trajectories",- use_prompt_caching=True,- max_trajectory_budget=10.0,- )-- history = []-- # Task 1: Navigate to LinkedIn connections page- navigation_task = (- "STEP 1 - NAVIGATE TO LINKEDIN CONNECTIONS PAGE:\n"- "1. Open a web browser (Chrome or Firefox)\n"- "2. Navigate to https://www.linkedin.com/mynetwork/invite-connect/connections/\n"- "3. Wait for the page to fully load\n"- "4. Confirm you can see the list of connections\n"- "5. Ready to start extracting contacts"- )-- print(f"\n[Task 1/21] Navigating to LinkedIn...")- history.append({"role": "user", "content": navigation_task})-- async for result in agent.run(history, stream=False):- history += result.get("output", [])-- print(f"✅ Navigation completed\n")-- # Extract 20 contacts- contacts_extracted = 0- linkedin_urls = []- previous_contact_name = None-- for contact_num in range(1, 21):- # Build extraction task- if contact_num == 1:- extraction_task = (- f"STEP {contact_num + 1} - EXTRACT CONTACT {contact_num} OF 20:\n"- f"1. Click on the first connection's profile\n"- f"2. Extract: FIRST, LAST, ROLE, COMPANY, LINKEDIN URL\n"- f"3. Return in exact format:\n"- f"FIRST: [value]\n"- f"LAST: [value]\n"- f"ROLE: [value]\n"- f"COMPANY: [value]\n"- f"LINKEDIN: [value]\n"- f"4. Navigate back to connections list"- )- else:- extraction_task = (- f"STEP {contact_num + 1} - EXTRACT CONTACT {contact_num} OF 20:\n"- f"1. Find '{previous_contact_name}' in the list\n"- f"2. Click on the contact BELOW them\n"- f"3. Extract: FIRST, LAST, ROLE, COMPANY, LINKEDIN URL\n"- f"4. Return in exact format:\n"- f"FIRST: [value]\n"- f"LAST: [value]\n"- f"ROLE: [value]\n"- f"COMPANY: [value]\n"- f"LINKEDIN: [value]\n"- f"5. Navigate back"- )-- print(f"[Task {contact_num + 1}/21] Extracting contact {contact_num}/20...")- history.append({"role": "user", "content": extraction_task})-- all_output = []- async for result in agent.run(history, stream=False):- output = result.get("output", [])- history += output- all_output.extend(output)-- contact_data = extract_contact_from_response(all_output)-- has_name = bool(contact_data['first'] and contact_data['last'])- has_linkedin = bool(contact_data['linkedin'] and 'linkedin.com' in contact_data['linkedin'])-- if has_name or has_linkedin:- with open(csv_path, 'a', newline='', encoding='utf-8') as csvfile:- writer = csv.DictWriter(csvfile, fieldnames=['first', 'last', 'role', 'company', 'met_at', 'linkedin'])- writer.writerow(contact_data)- contacts_extracted += 1-- if contact_data['linkedin']:- linkedin_urls.append(contact_data['linkedin'])-- if has_name:- previous_contact_name = f"{contact_data['first']} {contact_data['last']}".strip()-- name_str = f"{contact_data['first']} {contact_data['last']}" if has_name else "[No name]"- print(f"✅ Contact {contact_num}/20 saved: {name_str}")- else:- print(f"⚠️ Could not extract valid data for contact {contact_num}")-- if contact_num % 5 == 0:- print(f"\n📈 Progress: {contacts_extracted}/{contact_num} contacts extracted\n")-- # Create messaging links file- messaging_filename = f"linkedin_messaging_links_{timestamp}.txt"- messaging_path = os.path.join(os.getcwd(), messaging_filename)-- with open(messaging_path, 'w', encoding='utf-8') as txtfile:- txtfile.write("LinkedIn Messaging Compose Links\n")- txtfile.write("=" * 80 + "\n\n")-- for i, linkedin_url in enumerate(linkedin_urls, 1):- public_id = extract_public_id_from_linkedin_url(linkedin_url)- if public_id:- messaging_url = f"https://www.linkedin.com/messaging/compose/?recipient={public_id}"- txtfile.write(f"{i}. {messaging_url}\n")-- print("\n" + "="*80)- print("🎉 All tasks completed!")- print(f"📁 CSV file saved to: {csv_path}")- print(f"📊 Total contacts extracted: {contacts_extracted}/20")- print(f"💬 Messaging links saved to: {messaging_path}")- print("="*80)-- except Exception as e:- print(f"\n❌ Error: {e}")- traceback.print_exc()- raise--def main():- try:- load_dotenv()-- if "ANTHROPIC_API_KEY" not in os.environ:- raise RuntimeError("Please set ANTHROPIC_API_KEY in .env")-- if "CUA_API_KEY" not in os.environ:- raise RuntimeError("Please set CUA_API_KEY in .env")-- if "CUA_CONTAINER_NAME" not in os.environ:- raise RuntimeError("Please set CUA_CONTAINER_NAME in .env")-- signal.signal(signal.SIGINT, handle_sigint)-- asyncio.run(scrape_linkedin_connections())-- except Exception as e:- print(f"\n❌ Error: {e}")- traceback.print_exc()--if __name__ == "__main__":- main()-```-- </Tab>- <Tab value="Linux on Docker">--```python-# Same code as Cloud Sandbox, but change Sandbox initialization to:-async with Sandbox.ephemeral(- Image.linux(),- local=True,- verbosity=logging.INFO,-) as computer:-```--And remove the `CUA_API_KEY` and `CUA_CONTAINER_NAME` requirements from `.env` and the validation checks.-- </Tab>- <Tab value="macOS Sandbox">--```python-# Same code as Cloud Sandbox, but change Sandbox initialization to:-async with Sandbox.create(- Image.macos(),- local=True,- name="macos-sequoia-cua:latest",- verbosity=logging.INFO,-) as computer:-```--And remove the `CUA_API_KEY` and `CUA_CONTAINER_NAME` requirements from `.env` and the validation checks.-- </Tab>- <Tab value="Windows Sandbox">--```python-# Same code as Cloud Sandbox, but change Sandbox initialization to:-async with Sandbox.create(- Image.windows(),- local=True,- verbosity=logging.INFO,-) as computer:-```--And remove the `CUA_API_KEY` and `CUA_CONTAINER_NAME` requirements from `.env` and the validation checks.-- </Tab>-</Tabs>--</Step>--<Step>--### Run Your Script--Execute your contact extraction automation:--```bash-python contact_export.py-```--The agent will:--1. Navigate to your LinkedIn connections page-2. Extract data from 20 contacts (first name, last name, role, company, LinkedIn URL)-3. Save contacts to a timestamped CSV file-4. Generate messaging compose links for easy follow-up--Monitor the output to see the agent's progress. The script will show a progress update every 5 contacts.--</Step>--</Steps>-------## How It Works--This script demonstrates a practical workflow for extracting LinkedIn connection data:--1. **Session Persistence** - Manually log into LinkedIn through the VM once, and the VM saves your session-2. **Navigation** - The script navigates to your connections page using your saved authenticated session-3. **Data Extraction** - For each contact, the agent clicks their profile, extracts data, and navigates back-4. **Python Processing** - Python parses responses, validates data, and writes to CSV incrementally-5. **Output Files** - Generates a CSV with contact data and a text file with messaging URLs--## Next Steps--- Learn more about [Cua Computer Framework](/cua/reference/computer-sdk) and [Computer Commands](/cua/reference/computer-sdk)-- Read about [Agent Loops](/cua/guide/fundamentals/agent-loops), [Tools](/cua/guide/advanced/custom-tools), and [Supported Model Providers](/cua/guide/fundamentals/vlms)-- Experiment with different [Models and Providers](/cua/guide/fundamentals/vlms)-- Adapt this script for other platforms (Twitter/X, email extraction, etc.)-- Join our [Discord community](https://discord.com/invite/mVnXXpdE85) for helpdiff --git a/docs/content/docs/cua/examples/claude-code/cloud-sandbox.mdx b/docs/content/docs/cua/examples/claude-code/cloud-sandbox.mdxdeleted file mode 100644index c0d58ff626..0000000000--- a/docs/content/docs/cua/examples/claude-code/cloud-sandbox.mdx+++ /dev/null@@ -1,77 +0,0 @@-----title: Cloud Sandbox-description: Run Claude Code in a Cua cloud sandbox------A complete script that hands off a coding task to Claude Code running inside a Cua cloud sandbox.--## Prerequisites--```bash-pip install cua-sandbox-export CUA_API_KEY="your-cua-key"-export ANTHROPIC_API_KEY="your-anthropic-key"-```--## Script--```python-"""Run Claude Code inside a Cua cloud sandbox."""--import asyncio-import os--from cua_sandbox import Image, Sandbox--image = Image.linux("ubuntu", "24.04")---async def run(sb, cmd, **kwargs):- """Run a command and print its output."""- print(f"$ {cmd}")- result = await sb.shell.run(cmd, **kwargs)- if result.stdout:- print(result.stdout.strip())- if result.stderr:- print(result.stderr.strip())- return result---async def main():- api_key = os.environ["ANTHROPIC_API_KEY"]-- async with Sandbox.ephemeral(image) as sb:- # Install Node.js 20 + Claude Code (skip if already present)- check = await sb.shell.run("which claude", timeout=10)- if check.returncode != 0:- sudo = "sudo " if (await sb.shell.run("whoami")).stdout.strip() != "root" else ""- await run(sb, f"{sudo}apt-get update && {sudo}apt-get install -y curl git", timeout=120)- await run(sb, f"curl -fsSL https://deb.nodesource.com/setup_20.x | {sudo}bash -", timeout=60)- await run(sb, f"{sudo}apt-get install -y nodejs", timeout=60)- await run(sb, f"{sudo}npm install -g @anthropic-ai/claude-code", timeout=120)-- # Run Claude Code- result = await run(- sb,- f"ANTHROPIC_API_KEY='{api_key}' claude "- f"--dangerously-skip-permissions "- f"-p 'Create a hello world index.html file' "- f"--output-format stream-json --verbose",- timeout=300,- )-- # Verify the result- await run(sb, "cat index.html", timeout=10)---if __name__ == "__main__":- asyncio.run(main())-```--## Run it--```bash-python test_claude_code_cloud.py-```--See the [Claude Code guide](/cua/guide/agents-in-sandbox/claude-code) for more patterns like working on repositories, resuming sessions, and custom system prompts.diff --git a/docs/content/docs/cua/examples/claude-code/human-demonstrations.mdx b/docs/content/docs/cua/examples/claude-code/human-demonstrations.mdxdeleted file mode 100644index 21b4bf6f8d..0000000000--- a/docs/content/docs/cua/examples/claude-code/human-demonstrations.mdx+++ /dev/null@@ -1,106 +0,0 @@-----title: Prompting Claude Code with Human Demonstrations-description: Record human workflows and use them to guide Claude Code agents------import { Step, Steps } from 'fumadocs-ui/components/steps';-import { Callout } from 'fumadocs-ui/components/callout';--## Overview--Human demonstrations let you record a workflow once and replay it with AI assistance. This guide shows how to connect the Cua MCP server to Claude Code, record a demonstration, and use it to guide automated agents.-------## Add the Cua MCP Server--Connect Claude Code to your Cua sandboxes:--```bash-claude mcp add cua-mcp -- cua serve-mcp-```--Verify the connection:--```bash-claude mcp list-```--<Callout type="info">- Run `cua auth login` first if you haven't authenticated with your Cua API key.-</Callout>-------## Prompts to Try--Once connected, try these prompts in sequence:--<Steps>--<Step>--### Create a Sandbox--```-Create a Linux sandbox in north-america with size medium-```--Claude Code uses the `create_sandbox` tool to provision a new cloud sandbox.--</Step>--<Step>--### Record a Demonstration--```-Record a demonstration on my-sandbox called "login-flow" that shows how to log into the admin panel-```--Claude Code opens the VNC interface with recording enabled. Perform your workflow, then click **Stop Recording**. The recording is processed and saved as a skill.--</Step>--<Step>--### Replay the Demonstration--```-Read the login-flow skill and replay those steps on my-sandbox-```--Claude Code reads the skill from `~/.cua/skills/login-flow/SKILL.md` and executes each step using the computer control tools (`computer_click`, `computer_type`, etc.).--</Step>--</Steps>-------## Managing Skills--```bash-# List all skills-cua skills list--# Read a skill's full content-cua skills read my-skill--# Read as JSON (includes trajectory data)-cua skills read my-skill --format json--# Replay the recorded video-cua skills replay my-skill--# Delete a skill-cua skills delete my-skill-```-------## Next Steps--- Learn about [Agent Loops](/cua/guide/fundamentals/agent-loops) for understanding how agents process tasks-- See [Trajectories](/cua/guide/advanced/trajectories) for debugging and analyzing agent behavior-- Explore [Custom Tools](/cua/guide/advanced/custom-tools) to extend agent capabilitiesdiff --git a/docs/content/docs/cua/examples/claude-code/meta.json b/docs/content/docs/cua/examples/claude-code/meta.jsondeleted file mode 100644index 2e95cad97b..0000000000--- a/docs/content/docs/cua/examples/claude-code/meta.json+++ /dev/null@@ -1,6 +0,0 @@-{- "title": "Claude Code",- "description": "Claude Code integration examples",- "icon": "Terminal",- "pages": ["cloud-sandbox", "human-demonstrations"]-}diff --git a/docs/content/docs/cua/examples/index.mdx b/docs/content/docs/cua/examples/index.mdxdeleted file mode 100644index bf56b7338d..0000000000--- a/docs/content/docs/cua/examples/index.mdx+++ /dev/null@@ -1,57 +0,0 @@-----title: Examples-description: Real-world examples of building with Cua------import { Card, Cards } from 'fumadocs-ui/components/card';-import { Workflow, Laptop, Brain, Terminal } from 'lucide-react';--# Examples--Explore real-world examples of building computer-use agents with Cua. These examples demonstrate various use cases across different platforms and AI models.--<Cards>- <Card- icon={<Terminal className="w-6 h-6" />}- href="/cua/examples/claude-code/human-demonstrations"- title="Claude Code"- description="Use human demonstrations to guide Claude Code agents"- />- <Card- icon={<Workflow className="w-6 h-6" />}- href="/cua/examples/automation/form-filling"- title="Automation"- description="Automate repetitive tasks like form filling and data export"- />- <Card- icon={<Laptop className="w-6 h-6" />}- href="/cua/examples/platform-specific/windows-app-behind-vpn"- title="Platform-Specific"- description="Platform-specific examples and integrations"- />- <Card- icon={<Brain className="w-6 h-6" />}- href="/cua/examples/ai-models/gemini-complex-ui-navigation"- title="AI Models"- description="Examples using different AI models for GUI grounding"- />-</Cards>--## Featured Examples--### Claude Code--- **[Human Demonstrations](/cua/examples/claude-code/human-demonstrations)** - Record workflows and use them to guide Claude Code agents--### Automation--- **[PDF to Form Automation](/cua/examples/automation/form-filling)** - Extract data from PDFs and automatically fill web forms-- **[Post-Event Contact Export](/cua/examples/automation/post-event-contact-export)** - Automate contact data extraction after events--### Platform-Specific--- **[Windows App behind VPN](/cua/examples/platform-specific/windows-app-behind-vpn)** - Control Windows applications through VPN connections--### AI Models--- **[GUI Grounding with Gemini 3](/cua/examples/ai-models/gemini-complex-ui-navigation)** - Use Gemini for complex UI navigation tasksdiff --git a/docs/content/docs/cua/examples/meta.json b/docs/content/docs/cua/examples/meta.jsondeleted file mode 100644index 9c05043268..0000000000--- a/docs/content/docs/cua/examples/meta.json+++ /dev/null@@ -1,6 +0,0 @@-{- "title": "Examples",- "description": "Real-world examples of building with Cua",- "icon": "Blocks",- "pages": ["sandboxes", "automation", "ai-models", "platform-specific", "claude-code"]-}diff --git a/docs/content/docs/cua/examples/platform-specific/meta.json b/docs/content/docs/cua/examples/platform-specific/meta.jsondeleted file mode 100644index ec5306dbc2..0000000000--- a/docs/content/docs/cua/examples/platform-specific/meta.json+++ /dev/null@@ -1,6 +0,0 @@-{- "title": "Platform-Specific",- "description": "Platform-specific examples and integrations",- "icon": "Laptop",- "pages": ["windows-app-behind-vpn"]-}diff --git a/docs/content/docs/cua/examples/platform-specific/windows-app-behind-vpn.mdx b/docs/content/docs/cua/examples/platform-specific/windows-app-behind-vpn.mdxdeleted file mode 100644index 956242141b..0000000000--- a/docs/content/docs/cua/examples/platform-specific/windows-app-behind-vpn.mdx+++ /dev/null@@ -1,624 +0,0 @@-----title: Windows App behind VPN-description: Automate legacy Windows desktop applications behind VPN with Cua------import { Step, Steps } from 'fumadocs-ui/components/steps';-import { Tab, Tabs } from 'fumadocs-ui/components/tabs';--## Overview--This guide demonstrates how to automate Windows desktop applications (like eGecko HR/payroll systems) that run behind corporate VPN. This is a common enterprise scenario where legacy desktop applications require manual data entry, report generation, or workflow execution.--**Use cases:**--- HR/payroll processing (employee onboarding, payroll runs, benefits administration)-- Desktop ERP systems behind corporate networks-- Legacy financial applications requiring VPN access-- Compliance reporting from on-premise systems--**Architecture:**--- Client-side Cua agent (Python SDK or Playground UI)-- Windows VM/Sandbox with VPN client configured-- RDP/remote desktop connection to target environment-- Desktop application automation via computer vision and UI control--<Callout type="info">- **Production Deployment**: For production use, consider workflow mining and custom finetuning to- create vertical-specific actions (e.g., "Run payroll", "Onboard employee") instead of generic UI- automation. This provides better audit trails and higher success rates.-</Callout>-------## Video Demo--<div className="rounded-lg border bg-card text-card-foreground shadow-sm p-4 mb-6">- <video- src="https://github.com/user-attachments/assets/8ab07646-6018-4128-87ce-53180cfea696"- controls- className="w-full rounded"- >- Your browser does not support the video tag.- </video>- <div className="text-sm text-muted-foreground mt-2">- Demo showing Cua automating an eGecko-like desktop application on Windows behind AWS VPN- </div>-</div>-------<Steps>--<Step>--### Set Up Your Environment--Install the required dependencies:--Create a `requirements.txt` file:--```text-cua-python-dotenv>=1.0.0-```--Install the dependencies:--```bash-pip install -r requirements.txt-```--Create a `.env` file with your API keys:--```text-ANTHROPIC_API_KEY=your-anthropic-api-key-CUA_API_KEY=sk_cua-api01...-CUA_SANDBOX_NAME=your-windows-sandbox-```--</Step>--<Step>--### Configure Windows Sandbox with VPN--<Tabs items={['Cloud Sandbox (Recommended)', 'Windows Sandbox', 'Self-Hosted VM']}>- <Tab value="Cloud Sandbox (Recommended)">--For enterprise deployments, use Cua Cloud Sandbox with pre-configured VPN:--1. Go to [cua.ai/signin](https://cua.ai/signin)-2. Navigate to **Dashboard > Containers > Create Instance**-3. Create a **Windows** sandbox (Medium or Large for desktop apps)-4. Configure VPN settings:- - Upload your AWS VPN Client configuration (`.ovpn` file)- - Or configure VPN credentials directly in the dashboard-5. Note your sandbox name and API key--Your Windows sandbox will launch with VPN automatically connected.-- </Tab>- <Tab value="Windows Sandbox">--For local development on Windows 10 Pro/Enterprise or Windows 11:--1. Enable [Windows Sandbox](https://learn.microsoft.com/en-us/windows/security/application-security/application-isolation/windows-sandbox/windows-sandbox-install)-2. Install the `pywinsandbox` dependency:- ```bash- pip install -U git+git://github.com/karkason/pywinsandbox.git- ```-3. Create a VPN setup script that runs on sandbox startup-4. Configure your desktop application installation within the sandbox--<Callout type="warn">- **Manual VPN Setup**: Windows Sandbox requires manual VPN configuration each time it starts. For- production use, consider Cloud Sandbox or self-hosted VMs with persistent VPN connections.-</Callout>-- </Tab>- <Tab value="Self-Hosted VM">--For self-managed infrastructure:--1. Deploy Windows VM on your preferred cloud (AWS, Azure, GCP)-2. Install and configure VPN client (AWS VPN Client, OpenVPN, etc.)-3. Install target desktop application and any dependencies-4. Install `cua-computer-server`:- ```bash- pip install cua-computer-server- python -m computer_server- ```-5. Configure firewall rules to allow Cua agent connections-- </Tab>-</Tabs>--</Step>--<Step>--### Create Your Automation Script--Create a Python file (e.g., `hr_automation.py`):--<Tabs items={['Cloud Sandbox', 'Windows Sandbox', 'Self-Hosted']}>- <Tab value="Cloud Sandbox">--```python-import asyncio-import logging-import os-from cua import ComputerAgent, Sandbox, Image-from dotenv import load_dotenv--logging.basicConfig(level=logging.INFO)-logger = logging.getLogger(__name__)--load_dotenv()--async def automate_hr_workflow():- """- Automate HR/payroll desktop application workflow.-- This example demonstrates:- - Launching Windows desktop application- - Navigating complex desktop UI- - Data entry and form filling- - Report generation and export- """- try:- # Connect to Windows Cloud Sandbox with VPN- async with Sandbox.create(- Image.windows(),- name=os.environ["CUA_SANDBOX_NAME"],- api_key=os.environ["CUA_API_KEY"],- verbosity=logging.INFO,- ) as computer:-- # Configure agent with specialized instructions- agent = ComputerAgent(- model="cua/anthropic/claude-sonnet-4.5",- tools=[computer],- only_n_most_recent_images=3,- verbosity=logging.INFO,- trajectory_dir="trajectories",- use_prompt_caching=True,- max_trajectory_budget=10.0,- instructions="""-You are automating a Windows desktop HR/payroll application.--IMPORTANT GUIDELINES:-- Always wait for windows and dialogs to fully load before interacting-- Look for loading indicators and wait for them to disappear-- Verify each action by checking on-screen confirmation messages-- If a button or field is not visible, try scrolling or navigating tabs-- Desktop apps often have nested menus - explore systematically-- Save work frequently using File > Save or Ctrl+S-- Before closing, always verify changes were saved--COMMON UI PATTERNS:-- Menu bar navigation (File, Edit, View, etc.)-- Ribbon interfaces with tabs-- Modal dialogs that block interaction-- Data grids/tables for viewing records-- Form fields with validation-- Status bars showing operation progress- """.strip()- )-- # Define workflow tasks- tasks = [- "Launch the HR application from the desktop or start menu",- "Log in with the credentials shown in credentials.txt on the desktop",- "Navigate to Employee Management section",- "Create a new employee record with information from new_hire.xlsx on desktop",- "Verify the employee was created successfully by searching for their name",- "Generate an onboarding report for the new employee",- "Export the report as PDF to the desktop",- "Log out of the application"- ]-- history = []-- for task in tasks:- logger.info(f"\n{'='*60}")- logger.info(f"Task: {task}")- logger.info(f"{'='*60}\n")-- history.append({"role": "user", "content": task})-- async for result in agent.run(history):- for item in result.get("output", []):- if item.get("type") == "message":- content = item.get("content", [])- for block in content:- if block.get("type") == "text":- response = block.get("text", "")- logger.info(f"Agent: {response}")- history.append({"role": "assistant", "content": response})-- logger.info("\nTask completed. Moving to next task...\n")-- logger.info("\n" + "="*60)- logger.info("All tasks completed successfully!")- logger.info("="*60)-- except Exception as e:- logger.error(f"Error during automation: {e}")- import traceback- traceback.print_exc()--if __name__ == "__main__":- asyncio.run(automate_hr_workflow())-```-- </Tab>- <Tab value="Windows Sandbox">--```python-import asyncio-import logging-import os-from cua import ComputerAgent, Sandbox, Image-from dotenv import load_dotenv--logging.basicConfig(level=logging.INFO)-logger = logging.getLogger(__name__)--load_dotenv()--async def automate_hr_workflow():- try:- # Connect to Windows Sandbox- async with Sandbox.create(- Image.windows(),- local=True,- verbosity=logging.INFO,- ) as computer:-- agent = ComputerAgent(- model="cua/anthropic/claude-sonnet-4.5",- tools=[computer],- only_n_most_recent_images=3,- verbosity=logging.INFO,- trajectory_dir="trajectories",- use_prompt_caching=True,- max_trajectory_budget=10.0,- instructions="""-You are automating a Windows desktop HR/payroll application.--IMPORTANT GUIDELINES:-- Always wait for windows and dialogs to fully load before interacting-- Verify each action by checking on-screen confirmation messages-- Desktop apps often have nested menus - explore systematically-- Save work frequently using File > Save or Ctrl+S- """.strip()- )-- tasks = [- "Launch the HR application from the desktop",- "Log in with credentials from credentials.txt on desktop",- "Navigate to Employee Management and create new employee from new_hire.xlsx",- "Generate and export onboarding report as PDF",- "Log out of the application"- ]-- history = []-- for task in tasks:- logger.info(f"\nTask: {task}")- history.append({"role": "user", "content": task})-- async for result in agent.run(history):- for item in result.get("output", []):- if item.get("type") == "message":- content = item.get("content", [])- for block in content:- if block.get("type") == "text":- response = block.get("text", "")- logger.info(f"Agent: {response}")- history.append({"role": "assistant", "content": response})-- logger.info("\nAll tasks completed!")-- except Exception as e:- logger.error(f"Error: {e}")- import traceback- traceback.print_exc()--if __name__ == "__main__":- asyncio.run(automate_hr_workflow())-```-- </Tab>- <Tab value="Self-Hosted">--```python-import asyncio-import logging-import os-from cua import ComputerAgent, Sandbox, Image-from dotenv import load_dotenv--logging.basicConfig(level=logging.INFO)-logger = logging.getLogger(__name__)--load_dotenv()--async def automate_hr_workflow():- try:- # Connect to self-hosted Windows VM running computer-server- async with Sandbox.create(- use_host_computer_server=True,- base_url="http://your-windows-vm-ip:5757", # Update with your VM IP- verbosity=logging.INFO,- ) as computer:-- agent = ComputerAgent(- model="cua/anthropic/claude-sonnet-4.5",- tools=[computer],- only_n_most_recent_images=3,- verbosity=logging.INFO,- trajectory_dir="trajectories",- use_prompt_caching=True,- max_trajectory_budget=10.0,- instructions="""-You are automating a Windows desktop HR/payroll application.--IMPORTANT GUIDELINES:-- Always wait for windows and dialogs to fully load before interacting-- Verify each action by checking on-screen confirmation messages-- Save work frequently using File > Save or Ctrl+S- """.strip()- )-- tasks = [- "Launch the HR application",- "Log in with provided credentials",- "Complete the required HR workflow",- "Generate and export report",- "Log out"- ]-- history = []-- for task in tasks:- logger.info(f"\nTask: {task}")- history.append({"role": "user", "content": task})-- async for result in agent.run(history):- for item in result.get("output", []):- if item.get("type") == "message":- content = item.get("content", [])- for block in content:- if block.get("type") == "text":- response = block.get("text", "")- logger.info(f"Agent: {response}")- history.append({"role": "assistant", "content": response})-- logger.info("\nAll tasks completed!")-- except Exception as e:- logger.error(f"Error: {e}")- import traceback- traceback.print_exc()--if __name__ == "__main__":- asyncio.run(automate_hr_workflow())-```-- </Tab>-</Tabs>--</Step>--<Step>--### Run Your Automation--Execute the script:--```bash-python hr_automation.py-```--The agent will:--1. Connect to your Windows environment (with VPN if configured)-2. Launch and navigate the desktop application-3. Execute each workflow step sequentially-4. Verify actions and handle errors-5. Save trajectory logs for audit and debugging--Monitor the console output to see the agent's progress through each task.--</Step>--</Steps>-------## Key Configuration Options--### Agent Instructions--The `instructions` parameter is critical for reliable desktop automation:--```python-instructions="""-You are automating a Windows desktop HR/payroll application.--IMPORTANT GUIDELINES:-- Always wait for windows and dialogs to fully load before interacting-- Look for loading indicators and wait for them to disappear-- Verify each action by checking on-screen confirmation messages-- If a button or field is not visible, try scrolling or navigating tabs-- Desktop apps often have nested menus - explore systematically-- Save work frequently using File > Save or Ctrl+S-- Before closing, always verify changes were saved--COMMON UI PATTERNS:-- Menu bar navigation (File, Edit, View, etc.)-- Ribbon interfaces with tabs-- Modal dialogs that block interaction-- Data grids/tables for viewing records-- Form fields with validation-- Status bars showing operation progress--APPLICATION-SPECIFIC:-- Login is at top-left corner-- Employee records are under "HR Management" > "Employees"-- Reports are generated via "Tools" > "Reports" > "Generate"-- Always click "Save" before navigating away from a form-""".strip()-```--### Budget Management--For long-running workflows, adjust budget limits:--```python-agent = ComputerAgent(- model="cua/anthropic/claude-sonnet-4.5",- tools=[computer],- max_trajectory_budget=20.0, # Increase for complex workflows- # ... other params-)-```--### Image Retention--Balance context and cost by retaining only recent screenshots:--```python-agent = ComputerAgent(- # ...- only_n_most_recent_images=3, # Keep last 3 screenshots- # ...-)-```-------## Production Considerations--<Callout type="warn" title="Production Deployment">- For enterprise production deployments, consider these additional steps:-</Callout>--### 1. Workflow Mining--Before deploying, analyze your actual workflows:--- Record user interactions with the application-- Identify common patterns and edge cases-- Map out decision trees and validation requirements-- Document application-specific quirks and timing issues--### 2. Custom Finetuning--Create vertical-specific actions instead of generic UI automation:--```python-# Instead of generic steps:-tasks = ["Click login", "Type username", "Type password", "Click submit"]--# Create semantic actions:-tasks = ["onboard_employee", "run_payroll", "generate_compliance_report"]-```--This provides:--- Better audit trails-- Approval gates at business logic level-- Higher success rates-- Easier maintenance and updates--### 3. Human-in-the-Loop--Add approval gates for critical operations:--```python-agent = ComputerAgent(- model="cua/anthropic/claude-sonnet-4.5",- tools=[computer],- # Add human approval callback for sensitive operations- callbacks=[ApprovalCallback(require_approval_for=["payroll", "termination"])]-)-```--### 4. Deployment Options--Choose your deployment model:--**Managed (Recommended)**--- Cua hosts Windows sandboxes, VPN/RDP stack, and agent runtime-- You get UI/API endpoints for triggering workflows-- Automatic scaling, monitoring, and maintenance-- SLA guarantees and enterprise support--**Self-Hosted**--- You manage Windows VMs, VPN infrastructure, and agent deployment-- Full control over data and security-- Custom network configurations-- On-premise or your preferred cloud-------## Troubleshooting--### VPN Connection Issues--If the agent cannot reach the application:--1. Verify VPN is connected: Check VPN client status in the Windows sandbox-2. Test network connectivity: Try pinging internal resources-3. Check firewall rules: Ensure RDP and application ports are open-4. Review VPN logs: Look for authentication or routing errors--### Application Not Launching--If the desktop application fails to start:--1. Verify installation: Check the application is installed in the sandbox-2. Check dependencies: Ensure all required DLLs and frameworks are present-3. Review permissions: Application may require admin rights-4. Check logs: Look for error messages in Windows Event Viewer--### UI Element Not Found--If the agent cannot find buttons or fields:--1. Increase wait times: Some applications load slowly-2. Check screen resolution: UI elements may be off-screen-3. Verify DPI scaling: High DPI settings can affect element positions-4. Update instructions: Provide more specific navigation guidance--### Cost Management--If costs are higher than expected:--1. Reduce `max_trajectory_budget`-2. Decrease `only_n_most_recent_images`-3. Use prompt caching: Set `use_prompt_caching=True`-4. Optimize task descriptions: Be more specific to reduce retry attempts-------## Next Steps--- **Explore Custom Tools**: Learn how to create [Custom Tools](/cua/guide/advanced/custom-tools) for application-specific actions-- **Implement Callbacks**: Add [Monitoring and Logging](/cua/guide/fundamentals/callbacks) for production workflows-- **Join community**: Get help in our [Discord](https://discord.com/invite/mVnXXpdE85)-------## Related Examples--- [Form Filling](/cua/examples/automation/form-filling) - Web form automation-- [Post-Event Contact Export](/cua/examples/automation/post-event-contact-export) - Data extraction workflows-- [Custom Tools](/cua/guide/advanced/custom-tools) - Building application-specific functionsdiff --git a/docs/content/docs/cua/examples/sandboxes/android.mdx b/docs/content/docs/cua/examples/sandboxes/android.mdxdeleted file mode 100644index b7dd4e9dd5..0000000000--- a/docs/content/docs/cua/examples/sandboxes/android.mdx+++ /dev/null@@ -1,120 +0,0 @@-----title: Android Sandboxes-description: Run Android 14 VMs with touch, gesture, and hardware key controls-# test-sources:-# examples/sandboxes/test_android_cloud_vm.py-# examples/sandboxes/test_android_local_vm.py------import { Callout } from 'fumadocs-ui/components/callout';--Android sandboxes expose the `sb.mobile` interface for touch input, gestures, and hardware keys. Cloud Android works on any host OS. Local Android uses the Android SDK emulator.-------## Cloud Android VM--<Callout type="info">Requires `CUA_API_KEY`. No Android SDK or emulator needed.</Callout>--```python-# source: examples/sandboxes/test_android_cloud_vm.py-import asyncio-from cua import Image, Sandbox---async def main():- async with Sandbox.ephemeral(- Image.android("14"),- name="example-android-cloud-vm",- ) as sb:- screenshot = await sb.screenshot()- with open("screenshot.png", "wb") as f:- f.write(screenshot)-- w, h = await sb.screen.size()- print(f"Screen: {w}x{h}")-- # Touch controls- await sb.mobile.tap(w // 2, h // 2)- await sb.mobile.home()---asyncio.run(main())-```--## Local Android VM--<Callout type="info">Requires Android SDK with `emulator` and `adb` installed. No API key needed.</Callout>--```python-# source: examples/sandboxes/test_android_local_vm.py-import asyncio-from cua import Image, Sandbox---async def main():- async with Sandbox.ephemeral(- Image.android("14"),- local=True,- name="example-android-local-vm",- ) as sb:- screenshot = await sb.screenshot()- with open("screenshot.png", "wb") as f:- f.write(screenshot)-- w, h = await sb.screen.size()- print(f"Screen: {w}x{h}")-- await sb.mobile.tap(w // 2, h // 2)- await sb.mobile.home()---asyncio.run(main())-```-------## `sb.mobile` interface--All touch and hardware key operations go through `sb.mobile`:--```python-# Tap, long press, double tap-await sb.mobile.tap(x, y)-await sb.mobile.long_press(x, y, duration_ms=1000)-await sb.mobile.double_tap(x, y)--# Swipe and scroll-await sb.mobile.swipe(x1, y1, x2, y2, duration_ms=300)-await sb.mobile.scroll_up()-await sb.mobile.scroll_down()--# Hardware keys-await sb.mobile.home()-await sb.mobile.back()-await sb.mobile.recents()--# Text input-await sb.mobile.type_text("hello world")-```--## Multi-touch gestures--Use `sb.mobile.gesture()` for pinch-to-zoom and other multi-finger inputs. Pass alternating start/end `(x, y)` pairs — one pair per finger:--```python-w, h = await sb.screen.size()-cx, cy = w // 2, h // 2--# Pinch open (zoom in)-await sb.mobile.gesture(- (cx - 20, cy), (cx - 200, cy), # finger 0- (cx + 20, cy), (cx + 200, cy), # finger 1-)--# Pinch close (zoom out)-await sb.mobile.gesture(- (cx - 200, cy), (cx - 20, cy),- (cx + 200, cy), (cx + 20, cy),-)-```diff --git a/docs/content/docs/cua/examples/sandboxes/custom-images.mdx b/docs/content/docs/cua/examples/sandboxes/custom-images.mdxdeleted file mode 100644index 5062e68e7b..0000000000--- a/docs/content/docs/cua/examples/sandboxes/custom-images.mdx+++ /dev/null@@ -1,148 +0,0 @@-----title: Custom Images-description: Run sandboxes from OCI registry images, local disk files, and OSWorld benchmarks-# test-sources:-# examples/sandboxes/test_linux_local_registry_vm.py-# examples/sandboxes/test_macos_local_registry_vm.py-# examples/sandboxes/test_linux_local_osworld_vm.py------import { Callout } from 'fumadocs-ui/components/callout';--Beyond the built-in `Image.linux()` / `Image.macos()` / `Image.windows()` constructors, the SDK supports pulling from OCI registries and running local disk images directly.-------## OCI Registry — Linux VM via Tart--Pull a Linux OCI VM image from `ghcr.io` and run it locally with `TartRuntime`. Tart uses Apple Virtualization framework and only works on macOS.--<Callout type="info">Requires macOS host with [Tart](https://github.com/cirruslabs/tart) installed.</Callout>--```python-# source: examples/sandboxes/test_linux_local_registry_vm.py-import asyncio-from cua import Image, Sandbox-from cua.runtime import TartRuntime---async def main():- async with Sandbox.ephemeral(- Image.from_registry("ghcr.io/trycua/cua-xfce:latest"),- local=True,- runtime=TartRuntime(),- name="example-linux-local-registry-vm",- ) as sb:- result = await sb.shell.run("uname -s")- print(result.stdout.strip()) # "Linux"-- screenshot = await sb.screenshot()- with open("screenshot.png", "wb") as f:- f.write(screenshot)---asyncio.run(main())-```--## OCI Registry — macOS VM--Pull a macOS OCI image. The SDK reads the OCI manifest to auto-select Lume as the runtime.--<Callout type="info">Requires macOS host with Lume installed. For cloud macOS, use `Image.macos()` without `local=True` instead.</Callout>--```python-# source: examples/sandboxes/test_macos_local_registry_vm.py-import asyncio-from cua import Image, Sandbox---async def main():- async with Sandbox.ephemeral(- Image.from_registry("ghcr.io/trycua/macos-tahoe-cua:latest"),- local=True,- name="example-macos-local-registry-vm",- ) as sb:- result = await sb.shell.run("sw_vers")- print(result.stdout.strip())-- screenshot = await sb.screenshot()- with open("screenshot.png", "wb") as f:- f.write(screenshot)---asyncio.run(main())-```-------## Local disk image — qcow2 / vhdx / iso--Run any local disk image directly. Supported formats: `.qcow2`, `.vhdx`, `.raw`, `.img`, `.iso`.--```python-import asyncio-from cua import Image, Sandbox--async def main():- async with Sandbox.ephemeral(- Image.from_file("/path/to/disk.qcow2", os_type="linux"),- local=True,- ) as sb:- result = await sb.shell.run("uname -a")- print(result.stdout)--asyncio.run(main())-```--URLs are downloaded and cached automatically in `~/.cua/cua-sandbox/image-cache/`:--```python-Image.from_file("https://example.com/custom-linux.qcow2", os_type="linux")-```-------## OSWorld benchmark image--Run the standard [OSWorld](https://os-world.github.io/) Ubuntu image using the OSWorld Flask transport (`agent_type="osworld"`). The SDK downloads and caches the image from HuggingFace automatically.--<Callout type="info">- Requires QEMU. The first run downloads the ~8GB Ubuntu.qcow2 image — subsequent runs use the local cache at `~/.cua/cua-sandbox/image-cache/`.-</Callout>--```python-# source: examples/sandboxes/test_linux_local_osworld_vm.py-import asyncio-from cua import Image, Sandbox-from cua.runtime import QEMURuntime--OSWORLD_IMAGE = (- "https://huggingface.co/datasets/xlangai/ubuntu_osworld/resolve/main/Ubuntu.qcow2.zip"-)---async def main():- async with Sandbox.ephemeral(- Image.from_file(OSWORLD_IMAGE, os_type="linux", agent_type="osworld"),- local=True,- runtime=QEMURuntime(- mode="bare-metal",- api_port=18020,- vnc_display=20,- memory_mb=4096,- cpu_count=4,- ),- name="example-linux-local-osworld-vm",- ) as sb:- screenshot = await sb.screenshot()- with open("screenshot.png", "wb") as f:- f.write(screenshot)-- w, h = await sb.get_dimensions()- print(f"Screen: {w}x{h}") # 1920x1080---asyncio.run(main())-```--The `agent_type="osworld"` flag tells the SDK to use the OSWorld Flask API (`/screenshot`, `/execute_action`) instead of the standard computer-server transport. This matches the interface expected by the [OSWorld benchmark harness](/cua/guide/integrations/benchmarks/introduction).diff --git a/docs/content/docs/cua/examples/sandboxes/linux.mdx b/docs/content/docs/cua/examples/sandboxes/linux.mdxdeleted file mode 100644index 76ecaad986..0000000000--- a/docs/content/docs/cua/examples/sandboxes/linux.mdx+++ /dev/null@@ -1,137 +0,0 @@-----title: Linux Sandboxes-description: Run Linux containers and VMs — cloud or local-# test-sources:-# examples/sandboxes/test_linux_cloud_container.py-# examples/sandboxes/test_linux_cloud_vm.py-# examples/sandboxes/test_linux_local_container.py-# examples/sandboxes/test_linux_local_vm.py------import { Tab, Tabs } from 'fumadocs-ui/components/tabs';-import { Callout } from 'fumadocs-ui/components/callout';--Linux is available as either a lightweight **container** (Docker-backed, fast startup) or a full **VM** (QEMU, full kernel isolation). Both work locally and on Cua Cloud.--| | Container | VM |-|--|-----------|-----|-| Startup | Fast (~5s) | Slower (~30s) |-| Isolation | Process-level | Full kernel |-| Local runtime | Docker | QEMU (Docker-wrapped or bare-metal) |-| Cloud | `Image.linux()` | `Image.linux(kind="vm")` |-------## Cloud Container--<Callout type="info">Requires `CUA_API_KEY`. No local Docker or runtime needed.</Callout>--```python-# source: examples/sandboxes/test_linux_cloud_container.py-import asyncio-from cua import Image, Sandbox---async def main():- async with Sandbox.ephemeral(- Image.linux("ubuntu", "24.04"),- name="example-linux-cloud-container",- ) as sb:- result = await sb.shell.run("uname -s")- print(result.stdout.strip()) # "Linux"-- screenshot = await sb.screenshot()- with open("screenshot.png", "wb") as f:- f.write(screenshot)---asyncio.run(main())-```--## Cloud VM--<Callout type="info">Requires `CUA_API_KEY`. Full VM — heavier than a container but with full kernel.</Callout>--```python-# source: examples/sandboxes/test_linux_cloud_vm.py-import asyncio-from cua import Image, Sandbox---async def main():- async with Sandbox.ephemeral(- Image.linux("ubuntu", "24.04", kind="vm"),- name="example-linux-cloud-vm",- ) as sb:- result = await sb.shell.run("uname -s")- print(result.stdout.strip()) # "Linux"-- screenshot = await sb.screenshot()- with open("screenshot.png", "wb") as f:- f.write(screenshot)---asyncio.run(main())-```--## Local Container--<Callout type="info">Requires Docker. No API key needed.</Callout>--```python-# source: examples/sandboxes/test_linux_local_container.py-import asyncio-from cua import Image, Sandbox---async def main():- async with Sandbox.ephemeral(- Image.linux("ubuntu", "24.04"),- local=True,- name="example-linux-local-container",- ) as sb:- result = await sb.shell.run("uname -s")- print(result.stdout.strip()) # "Linux"-- screenshot = await sb.screenshot()- with open("screenshot.png", "wb") as f:- f.write(screenshot)---asyncio.run(main())-```--## Local VM--<Callout type="info">Requires Docker (QEMU runs inside a Docker container). No API key needed.</Callout>--```python-# source: examples/sandboxes/test_linux_local_vm.py-import asyncio-from cua import Image, Sandbox---async def main():- async with Sandbox.ephemeral(- Image.linux("ubuntu", "24.04", kind="vm"),- local=True,- name="example-linux-local-vm",- ) as sb:- result = await sb.shell.run("uname -s")- print(result.stdout.strip()) # "Linux"-- screenshot = await sb.screenshot()- with open("screenshot.png", "wb") as f:- f.write(screenshot)---asyncio.run(main())-```-------## Container vs. VM — when to use which--Use a **container** when you want fast iteration and don't need kernel-level isolation — CI jobs, web scraping, most agent tasks.--Use a **VM** when you need a real kernel — testing kernel modules, `/dev/kvm` workloads, OS-level benchmarks like OSWorld, or when you need stronger isolation between runs.diff --git a/docs/content/docs/cua/examples/sandboxes/macos.mdx b/docs/content/docs/cua/examples/sandboxes/macos.mdxdeleted file mode 100644index 832db7b8d1..0000000000--- a/docs/content/docs/cua/examples/sandboxes/macos.mdx+++ /dev/null@@ -1,109 +0,0 @@-----title: macOS Sandboxes-description: Run macOS VMs — cloud or local via Lume, with managed or registry images-# test-sources:-# examples/sandboxes/test_macos_cloud_vm.py-# examples/sandboxes/test_macos_local_vm.py-# examples/sandboxes/test_macos_local_registry_vm.py------import { Callout } from 'fumadocs-ui/components/callout';--macOS sandboxes are always VMs — either on Cua Cloud or locally via [Lume](https://github.com/trycua/cua/tree/main/libs/lume) (Apple Virtualization framework). Cloud macOS works on any host OS — no Mac required.--**Supported versions:** `"26"` / `"tahoe"` (default), `"15"` / `"sequoia"`-------## Cloud macOS VM--<Callout type="info">Requires `CUA_API_KEY`. Works on any host OS — Linux, Windows, or macOS.</Callout>--```python-# source: examples/sandboxes/test_macos_cloud_vm.py-import asyncio-from cua import Image, Sandbox---async def main():- async with Sandbox.ephemeral(- Image.macos("26"),- name="example-macos-cloud-vm",- ) as sb:- result = await sb.shell.run("sw_vers")- print(result.stdout.strip())-- screenshot = await sb.screenshot()- with open("screenshot.png", "wb") as f:- f.write(screenshot)---asyncio.run(main())-```--## Local macOS VM--<Callout type="info">Requires macOS host with [Lume](https://github.com/trycua/cua/tree/main/libs/lume) installed. No API key needed.</Callout>--```python-# source: examples/sandboxes/test_macos_local_vm.py-import asyncio-from cua import Image, Sandbox---async def main():- async with Sandbox.ephemeral(- Image.macos("26"),- local=True,- name="example-macos-local-vm",- ) as sb:- result = await sb.shell.run("sw_vers")- print(result.stdout.strip())-- screenshot = await sb.screenshot()- with open("screenshot.png", "wb") as f:- f.write(screenshot)---asyncio.run(main())-```--## Local macOS VM from OCI Registry--Pull a macOS image directly from the `ghcr.io/trycua` registry. The SDK reads the OCI manifest to resolve the image kind and auto-selects Lume.--<Callout type="info">Requires macOS host with Lume installed.</Callout>--```python-# source: examples/sandboxes/test_macos_local_registry_vm.py-import asyncio-from cua import Image, Sandbox---async def main():- async with Sandbox.ephemeral(- Image.from_registry("ghcr.io/trycua/macos-tahoe-cua:latest"),- local=True,- name="example-macos-local-registry-vm",- ) as sb:- result = await sb.shell.run("sw_vers")- print(result.stdout.strip())-- screenshot = await sb.screenshot()- with open("screenshot.png", "wb") as f:- f.write(screenshot)---asyncio.run(main())-```-------## Choosing a version--```python-Image.macos() # macOS Tahoe (26) — latest, default-Image.macos("26") # macOS Tahoe-Image.macos("15") # macOS Sequoia-Image.macos("sequoia") # also Sequoia-```diff --git a/docs/content/docs/cua/examples/sandboxes/meta.json b/docs/content/docs/cua/examples/sandboxes/meta.jsondeleted file mode 100644index 344b7aaf61..0000000000--- a/docs/content/docs/cua/examples/sandboxes/meta.json+++ /dev/null@@ -1,6 +0,0 @@-{- "title": "Sandboxes",- "description": "Runnable examples for every sandbox type",- "icon": "Monitor",- "pages": ["linux", "macos", "windows", "android", "custom-images"]-}diff --git a/docs/content/docs/cua/examples/sandboxes/windows.mdx b/docs/content/docs/cua/examples/sandboxes/windows.mdxdeleted file mode 100644index 8147e6f113..0000000000--- a/docs/content/docs/cua/examples/sandboxes/windows.mdx+++ /dev/null@@ -1,68 +0,0 @@-----title: Windows Sandboxes-description: Run Windows 11 VMs — cloud or local via Hyper-V / QEMU-# test-sources:-# examples/sandboxes/test_windows_cloud_vm.py-# examples/sandboxes/test_windows_local_vm.py------import { Callout } from 'fumadocs-ui/components/callout';--Windows sandboxes run as full VMs. Cloud Windows works on any host OS. Local Windows uses Hyper-V on Windows hosts, or QEMU as a fallback.-------## Cloud Windows VM--<Callout type="info">Requires `CUA_API_KEY`. Works on any host OS — no Windows needed.</Callout>--```python-# source: examples/sandboxes/test_windows_cloud_vm.py-import asyncio-from cua import Image, Sandbox---async def main():- async with Sandbox.ephemeral(- Image.windows("11"),- name="example-windows-cloud-vm",- ) as sb:- result = await sb.shell.run("ver")- print(result.stdout.strip())-- screenshot = await sb.screenshot()- with open("screenshot.png", "wb") as f:- f.write(screenshot)---asyncio.run(main())-```--## Local Windows VM--<Callout type="info">Requires a Windows host with Hyper-V, or QEMU installed. No API key needed.</Callout>--The SDK automatically selects Hyper-V on Windows hosts with Hyper-V available, otherwise falls back to QEMU.--```python-# source: examples/sandboxes/test_windows_local_vm.py-import asyncio-from cua import Image, Sandbox---async def main():- async with Sandbox.ephemeral(- Image.windows("11"),- local=True,- name="example-windows-local-vm",- ) as sb:- result = await sb.shell.run("ver")- print(result.stdout.strip())-- screenshot = await sb.screenshot()- with open("screenshot.png", "wb") as f:- f.write(screenshot)---asyncio.run(main())-```diff --git a/docs/content/docs/cua/guide/advanced/chat-history.mdx b/docs/content/docs/cua/guide/advanced/chat-history.mdxdeleted file mode 100644index 0ec4a84daf..0000000000--- a/docs/content/docs/cua/guide/advanced/chat-history.mdx+++ /dev/null@@ -1,145 +0,0 @@-----title: Chat History-description: Managing conversation history for multi-turn agent interactions------**Chat history** tracks the conversation between you and the agent across multiple turns. Each run builds on the previous context, enabling complex multi-step workflows and conversational interactions.--## Basic Multi-Turn Conversation--Pass a message history to continue from previous context:--```python-from cua import Sandbox, Image, ComputerAgent--async with Sandbox.ephemeral(Image.linux()) as computer:- agent = ComputerAgent(- model="anthropic/claude-sonnet-4-5-20250929",- tools=[computer]- )--# First turn-messages = [{"role": "user", "content": "Open Firefox"}]-async for result in agent.run(messages):- messages.extend(result["output"])--# Second turn - agent remembers Firefox is open-messages.append({"role": "user", "content": "Now go to github.com"})-async for result in agent.run(messages):- messages.extend(result["output"])-```--## Interactive Loop--Build a chat interface where each response adds to the history:--```python-history = []--while True:- user_input = input("> ")- if user_input.lower() == "quit":- break-- history.append({"role": "user", "content": user_input})-- async for result in agent.run(history):- history.extend(result["output"])-- # Print assistant responses- for item in result["output"]:- if item["type"] == "message":- print(item["content"][0]["text"])-```--## Message Structure--A conversation history contains these message types:--```python-messages = [- # User input- {"role": "user", "content": "go to trycua on github"},-- # Agent reasoning (internal thought process)- {- "type": "reasoning",- "summary": [{"type": "summary_text", "text": "Searching for Trycua GitHub"}]- },-- # Computer action- {- "type": "computer_call",- "call_id": "call_abc123",- "status": "completed",- "action": {"type": "type", "text": "Trycua GitHub"}- },-- # Screenshot result- {- "type": "computer_call_output",- "call_id": "call_abc123",- "output": {"type": "input_image", "image_url": "data:image/png;base64,..."}- },-- # Final response- {- "type": "message",- "role": "assistant",- "content": [{"type": "output_text", "text": "Done! The Trycua GitHub page is now open."}]- }-]-```--See [Message Format](/cua/guide/advanced/message-format) for complete type definitions.--## Managing Context Size--Screenshots accumulate in the history and consume context window space. Use `only_n_most_recent_images` to limit how many are kept:--```python-agent = ComputerAgent(- model="anthropic/claude-sonnet-4-5-20250929",- tools=[computer],- only_n_most_recent_images=3 # Keep only last 3 screenshots-)-```--Older screenshots are automatically removed, preventing context overflow during long sessions.--## Clearing History--Start fresh by resetting the messages array:--```python-# Reset to start a new conversation-messages = []--# Or keep system context but clear conversation-messages = [{"role": "system", "content": "You are a helpful assistant."}]-```--## Saving and Loading History--Persist conversations for later use:--```python-import json--# Save-with open("conversation.json", "w") as f:- # Filter out large image data if needed- saveable = [- {**m, "output": {"type": m["output"]["type"]}}- if m.get("type") == "computer_call_output"- else m- for m in messages- ]- json.dump(saveable, f)--# Load-with open("conversation.json", "r") as f:- messages = json.load(f)-```--For full conversation recording including screenshots, use [Trajectories](/cua/guide/advanced/trajectories) instead.diff --git a/docs/content/docs/cua/guide/advanced/composed-models.mdx b/docs/content/docs/cua/guide/advanced/composed-models.mdxdeleted file mode 100644index b389a0be06..0000000000--- a/docs/content/docs/cua/guide/advanced/composed-models.mdx+++ /dev/null@@ -1,96 +0,0 @@-----title: Composed Models-description: Combine grounding and planning models for optimized computer-use------Some models excel at **grounding** (finding UI elements) but can't reason about tasks. Others are great at **planning** but can't precisely locate elements. Composed models combine both capabilities for better results.--## How It Works--Use the `+` syntax to combine a grounding model with a planning model:--```python-from cua import Sandbox, Image, ComputerAgent--async with Sandbox.ephemeral(Image.linux()) as computer:- # Format: grounding_model+planning_model- agent = ComputerAgent(- model="huggingface-local/HelloKKMe/GTA1-7B+anthropic/claude-sonnet-4-5-20250929",- tools=[computer]- )-```--Cua uses a two-stage approach:--1. **Grounding stage** - The first model analyzes the screenshot and identifies UI element coordinates-2. **Planning stage** - The second model receives the element list and decides which to interact with--## Common Combinations--### Local Grounding + Cloud Planning--Run visual processing locally while using cloud models for reasoning:--```python-# GTA1 for grounding + Claude for planning-agent = ComputerAgent(- model="huggingface-local/HelloKKMe/GTA1-7B+anthropic/claude-sonnet-4-5-20250929",- tools=[computer]-)--# Moondream3 for grounding + Claude for planning-agent = ComputerAgent(- model="moondream3+anthropic/claude-sonnet-4-5-20250929",- tools=[computer]-)--# OmniParser for grounding + GPT-4o for planning-agent = ComputerAgent(- model="omniparser+openai/gpt-4o",- tools=[computer]-)--# UI-TARS for grounding + GPT-4o for planning-agent = ComputerAgent(- model="huggingface-local/ByteDance-Seed/UI-TARS-1.5-7B+openai/gpt-4o",- tools=[computer]-)-```--## Grounding Models--These models locate UI elements but can't reason about actions:--| Model | Description |-| ------------------------------------- | -------------------------------- |-| `huggingface-local/HelloKKMe/GTA1-7B` | General-purpose grounding |-| `omniparser` | Microsoft's UI element detection |-| `moondream3` | Lightweight, fast grounding |--## Planning Models--Any full computer-use model can serve as a planner:--| Model | Description |-| -------------------------------------- | ---------------------------------- |-| `anthropic/claude-sonnet-4-5-20250929` | Best balance of speed and accuracy |-| `anthropic/claude-haiku-4-5-20251001` | Faster, lower cost |-| `openai/gpt-4o` | Strong reasoning capabilities |--## When to Use Composed Models--**Privacy** - Run grounding locally so screenshots never leave your machine. Only text descriptions of UI elements go to the cloud.--**Cost** - Grounding models process images locally, reducing API calls to cloud providers.--**Latency** - Local grounding can be faster than sending full screenshots over the network.--**Accuracy** - Some grounding models detect UI elements more precisely than general-purpose VLMs.--## Limitations--- Requires a local GPU for the grounding model (typically 16GB+ VRAM for 7B models)-- Two-stage processing adds some overhead-- Not all grounding models work with all planning models--For simpler setups, consider using a single full computer-use model like Claude or OpenAI computer-use-preview.diff --git a/docs/content/docs/cua/guide/advanced/custom-tools.mdx b/docs/content/docs/cua/guide/advanced/custom-tools.mdxdeleted file mode 100644index 3628978328..0000000000--- a/docs/content/docs/cua/guide/advanced/custom-tools.mdx+++ /dev/null@@ -1,290 +0,0 @@-----title: Custom Tools-description: Extend agent capabilities with your own tools------import { Callout } from 'fumadocs-ui/components/callout';--**Custom tools** let you extend what your agent can do beyond the built-in Computer and Browser tools. Define functions that the agent can call to perform specific tasks like calculations, API calls, file operations, or any custom logic.--## Built-in Tools--Before creating custom tools, understand the built-in tool types:--| Tool | Actions | Use Case |-| ------------- | ---------------------------------- | -------------------------- |-| `Computer` | click, type, scroll, screenshot | Generic desktop automation |-| `BrowserTool` | visit_url, web_search, click, type | Browser-focused tasks |--### Model Tool Requirements--Some models are trained for specific toolsets. For example, FARA requires browser tools:--```python-# When you pass Computer to FARA, it auto-wraps to BrowserTool-agent = ComputerAgent(model="cua/microsoft/fara-7b", tools=[computer])-# Warning: "Model requires browser tools. Auto-wrapping Computer to BrowserTool."-```--To silence the warning, pass the correct tool explicitly:--```python-from cua_agent.tools.browser_tool import BrowserTool--browser_tool = BrowserTool(interface=computer.interface)-agent = ComputerAgent(model="cua/microsoft/fara-7b", tools=[browser_tool])-# No warning-```--General models like Claude work with any tools you provide - no auto-wrapping occurs.--<Callout type="info">- The Computer is always your base environment. Tools are built on top of it for specific use cases.-</Callout>--## Simple Function Tools--The easiest way to add a tool is to pass a function with a docstring:--```python-from cua import ComputerAgent--def calculate(a: int, b: int) -> int:- """Calculate the sum of two integers"""- return a + b--def get_weather(city: str) -> str:- """Get current weather for a city"""- # Your API call here- return f"Weather in {city}: Sunny, 72°F"--agent = ComputerAgent(- model="anthropic/claude-sonnet-4-5-20250929",- tools=[computer, calculate, get_weather]-)--await agent.run("What's 42 + 58? Also check the weather in San Francisco.")-```--The agent can now call `calculate` and `get_weather` alongside computer actions. Cua automatically extracts the function signature and docstring to create the tool schema.--## Sandboxed Tools--For tools that should run inside the sandbox rather than on your host machine, use the `@sandboxed` decorator:--```python-from cua import sandboxed--@sandboxed()-def read_file(path: str) -> str:- """Read contents of a file in the sandbox"""- with open(path, 'r') as f:- return f.read()--@sandboxed()-def save_note(filename: str, content: str) -> str:- """Save a note to a file in the sandbox"""- with open(filename, 'w') as f:- f.write(content)- return f"Saved to {filename}"--agent = ComputerAgent(- model="anthropic/claude-sonnet-4-5-20250929",- tools=[computer, read_file, save_note]-)--await agent.run("Read the file at /tmp/data.txt and summarize it")-```--Sandboxed tools execute inside the sandbox's virtual environment, which is safer and can access files on the sandbox's filesystem.--See [Sandboxed Python](/cua/guide/advanced/sandboxed-python) for more details.--## Using the BaseTool Class--For more control over tool schema and behavior, extend `BaseTool`:--```python-from cua.tools import BaseTool, register_tool-from typing import Union--@register_tool("database_query")-class DatabaseQueryTool(BaseTool):- def __init__(self, connection_string: str):- self.connection_string = connection_string-- @property- def description(self) -> str:- return "Execute a read-only SQL query against the database"-- @property- def parameters(self) -> dict:- return {- "type": "object",- "properties": {- "query": {- "type": "string",- "description": "SQL SELECT query to execute"- }- },- "required": ["query"]- }-- def call(self, params: Union[str, dict], **kwargs) -> Union[str, dict]:- if isinstance(params, str):- params = {"query": params}-- query = params["query"]- # Execute query and return results- return {"rows": [...], "count": 10}--# Use it-db_tool = DatabaseQueryTool("postgresql://...")-agent = ComputerAgent(- model="anthropic/claude-sonnet-4-5-20250929",- tools=[computer, db_tool]-)-```--The `@register_tool` decorator registers your tool class with a name that the agent uses to invoke it.--## Tool Schema--Tools need a schema so the model knows how to call them. For function tools, Cua generates the schema automatically from type hints and docstrings.--For `BaseTool` classes, you define the schema in the `parameters` property:--```python-@property-def parameters(self) -> dict:- return {- "type": "object",- "properties": {- "param_name": {- "type": "string",- "description": "What this parameter does"- },- "optional_param": {- "type": "integer",- "description": "An optional number"- }- },- "required": ["param_name"] # Only required params- }-```--This follows the JSON Schema format used by OpenAI and other providers.--## Async Tools--Both sync and async functions work as tools:--```python-import asyncio-import httpx--async def fetch_url(url: str) -> str:- """Fetch content from a URL"""- async with httpx.AsyncClient() as client:- response = await client.get(url)- return response.text[:1000] # First 1000 chars--agent = ComputerAgent(- model="anthropic/claude-sonnet-4-5-20250929",- tools=[computer, fetch_url]-)-```--Sync functions are automatically run in a thread pool to avoid blocking.--## Combining Tools--Pass multiple tools of different types:--```python-from cua import ComputerAgent-from cua.tools import BrowserTool-from cua import sandboxed--# Built-in tool-browser = BrowserTool(interface=computer)--# Simple function-def calculate(expression: str) -> float:- """Evaluate a math expression"""- return eval(expression) # Use with caution--# Sandboxed function-@sandboxed()-def process_data(data: list) -> dict:- """Process data inside the sandbox"""- import pandas as pd- df = pd.DataFrame(data)- return df.describe().to_dict()--agent = ComputerAgent(- model="anthropic/claude-sonnet-4-5-20250929",- tools=[computer, browser, calculate, process_data]-)-```--The agent decides which tool to use based on the task.--## Error Handling--Tools can raise exceptions, which are passed back to the model:--```python-from cua.tools import ToolError--def divide(a: float, b: float) -> float:- """Divide a by b"""- if b == 0:- raise ToolError("Cannot divide by zero")- return a / b-```--The model sees the error message and can adjust its approach.--## Example: File Management Tools--Here's a complete example with file management tools:--```python-from cua import ComputerAgent, sandboxed--@sandboxed()-def list_files(directory: str = ".") -> list:- """List files in a directory"""- import os- return os.listdir(directory)--@sandboxed()-def read_file(path: str) -> str:- """Read a file's contents"""- with open(path, 'r') as f:- return f.read()--@sandboxed()-def write_file(path: str, content: str) -> str:- """Write content to a file"""- with open(path, 'w') as f:- f.write(content)- return f"Written {len(content)} bytes to {path}"--@sandboxed()-def append_file(path: str, content: str) -> str:- """Append content to a file"""- with open(path, 'a') as f:- f.write(content)- return f"Appended {len(content)} bytes to {path}"--agent = ComputerAgent(- model="anthropic/claude-sonnet-4-5-20250929",- tools=[computer, list_files, read_file, write_file, append_file]-)--await agent.run("Create a file called notes.txt with today's date, then read it back")-```--The agent can now manage files directly without needing to use the UI.diff --git a/docs/content/docs/cua/guide/advanced/demonstration-guided-skills.mdx b/docs/content/docs/cua/guide/advanced/demonstration-guided-skills.mdxdeleted file mode 100644index 983b66e721..0000000000--- a/docs/content/docs/cua/guide/advanced/demonstration-guided-skills.mdx+++ /dev/null@@ -1,115 +0,0 @@-----title: Demonstration-Guided Skills-description: Use human demonstrations to guide agent behavior and improve task completion------Human demonstrations provide powerful context for guiding agent behavior. By showing an agent how a task is performed once, it can learn the workflow pattern and generalize to similar tasks with different inputs.--## Overview--The demonstration-guided workflow consists of three stages:--1. **Record** - Capture a human performing the task via [VNC recording](/cua/guide/advanced/vnc-recorder)-2. **Process** - Convert the recording into semantic action traces with VLM captions-3. **Prompt** - Use the processed trajectory to guide agent execution--This approach enables:--- **One-shot learning**: Teach a workflow once, execute it many times-- **Reduced errors**: Agents follow demonstrated patterns instead of discovering them-- **Complex workflows**: Handle multi-step tasks that require specific sequences--## Example: Using a Demonstration to Guide an Agent--### 1. Record a demonstration--```bash-cua skills record --sandbox my-sandbox-```--When prompted, name it `flight-booking` and describe it as "Book a flight from NYC to LA".--### 2. View the saved skill--The skill is saved to `~/.cua/skills/flight-booking/SKILL.md`:--```bash-cua skills read flight-booking-```--### 3. Use the skill to guide an agent--Read the SKILL.md file and prefix it to your prompt:--```python-from pathlib import Path-from cua import ComputerAgent--# Load the skill prompt from SKILL.md-skill_path = Path.home() / ".cua/skills/flight-booking/SKILL.md"-skill_prompt = skill_path.read_text()--agent = ComputerAgent(loop="anthropic")--# Prefix the skill as context, then give the actual task-messages = [- {"role": "user", "content": skill_prompt},- {"role": "user", "content": "Book a flight from Boston to Chicago for next Friday"}-]--async for result in agent.run(messages):- print(result)-```--The skill contains a natural language description of the demonstrated workflow, allowing the agent to follow the same pattern while adapting to the new parameters (Boston→Chicago instead of NYC→LA).--## Skill Format--Each skill is saved to `~/.cua/skills/<name>/` with the following structure:--```-~/.cua/skills/flight-booking/-├── SKILL.md # Skill description and agent prompt-└── trajectory/- ├── trajectory.json # Raw events and captions- ├── events.json # Input events- ├── flight-booking.mp4 # Video recording- ├── step_1_full.jpg # Full screenshot for step 1- ├── step_1_crop.jpg # Cropped screenshot with marker- └── ...-```--The `SKILL.md` file contains the skill metadata, step descriptions, and agent prompt:--```markdown-----name: flight-booking-description: Book a flight from NYC to LA------# flight-booking--Book a flight from NYC to LA--## Steps--### Step 1: Click on the departure city field--**Context:** A flight booking form with empty departure and arrival fields...-**Intent:** User wants to enter the departure city-**Expected Result:** Field becomes focused for text input--### Step 2: Type "NYC"--...--## Agent Prompt--You have been shown a demonstration of how to perform this task:-Book a flight from NYC to LA--The demonstration consisted of the following steps:-Step 1: Click on the departure city field-Step 2: Type "NYC"-...-```diff --git a/docs/content/docs/cua/guide/advanced/human-in-the-loop.mdx b/docs/content/docs/cua/guide/advanced/human-in-the-loop.mdxdeleted file mode 100644index 43815eeff1..0000000000--- a/docs/content/docs/cua/guide/advanced/human-in-the-loop.mdx+++ /dev/null@@ -1,118 +0,0 @@-----title: Human-in-the-Loop-description: Use humans as agents for evaluation, demonstrations, and interactive control------**Human-in-the-loop** lets you use a human operator as the agent instead of an AI model. This is useful for creating training data, evaluating agent performance, or taking manual control when automation needs guidance.--## Setup--Start the human agent tool:--```bash-python -m agent.human_tool-```--This opens a UI that shows pending completions. Select a completion to take control.--## Basic Usage--Use `human/human` as the model:--```python-from cua import Sandbox, Image, ComputerAgent--async with Sandbox.ephemeral(Image.linux()) as computer:- agent = ComputerAgent(- model="human/human",- tools=[computer]- )--async for result in agent.run("Navigate to settings and enable dark mode"):- print(result)-```--The agent waits for a human to perform each action through the UI.--## Composed with Grounding--Combine human planning with automated grounding:--```python-agent = ComputerAgent(- model="huggingface-local/HelloKKMe/GTA1-7B+human/human",- tools=[computer]-)--async for result in agent.run("Click the submit button"):- print(result)-```--The grounding model finds UI elements, then the human decides which to interact with.--## Use Cases--### Creating Training Data--Record human demonstrations to train or fine-tune models:--```python-agent = ComputerAgent(- model="human/human",- tools=[computer],- trajectory_dir="demonstrations"-)--# Human performs task, trajectory is saved-async for result in agent.run("Complete the checkout flow"):- pass-```--The saved trajectory contains the human's actions and screenshots, ready for training.--### Evaluating Agent Performance--Compare AI agent outputs to human ground truth:--```python-# Run AI agent-ai_agent = ComputerAgent(model="anthropic/claude-sonnet-4-5-20250929", tools=[computer])-async for result in ai_agent.run(task):- ai_results.append(result)--# Run human agent on same task-human_agent = ComputerAgent(model="human/human", tools=[computer])-async for result in human_agent.run(task):- human_results.append(result)--# Compare trajectories-```--### Interactive Control--Take over when automated agents get stuck:--```python-try:- async for result in ai_agent.run(task):- process(result)-except AgentStuckError:- # Switch to human control- async for result in human_agent.run("Continue from here"):- process(result)-```--### Testing and Debugging--Manually validate tool and environment behavior:--```python-agent = ComputerAgent(- model="human/human",- tools=[computer, custom_tool]-)--# Human can test each tool manually-async for result in agent.run("Test the database query tool"):- print(result)-```diff --git a/docs/content/docs/cua/guide/advanced/interactive-shell.mdx b/docs/content/docs/cua/guide/advanced/interactive-shell.mdxdeleted file mode 100644index b7dc2b0844..0000000000--- a/docs/content/docs/cua/guide/advanced/interactive-shell.mdx+++ /dev/null@@ -1,77 +0,0 @@-----title: Interactive Shell-description: Open a live PTY session inside a sandbox or local machine from the CLI or Python SDK------The `cua do shell` command and `computer.pty` API give you an interactive terminal (PTY) session inside any sandbox or local machine — similar to SSH, but without any network setup.--## CLI--```bash-# Switch to a target first-cua do switch docker my-container-# or-cua do switch host--# Open an interactive shell (bash / PowerShell on host)-cua do shell--# Run a specific program interactively-cua do shell python3-cua do shell vim /etc/hosts--# Override terminal dimensions-cua do shell --cols 220 --rows 50-```--The shell is fully interactive — arrow keys, tab completion, and `Ctrl+C` all work. When stdin is piped (non-TTY), the command runs non-interactively and its output is printed normally.--## Python SDK--```python-from cua import Sandbox, Image--async with Sandbox.create(provider_type="docker", name="my-container") as c:- handle = await c.pty.create(- command="bash",- cols=80,- rows=24,- on_data=lambda chunk: print(chunk.decode(errors="replace"), end="", flush=True),- )-- await handle.send_stdin(b"echo hello\n")- await handle.send_stdin(b"exit\n")- code = await handle.wait()- print(f"exited {code}")-```--### PtyHandle methods--| Method | Description |-|---|---|-| `send_stdin(data: bytes)` | Write bytes to the terminal's stdin |-| `resize(cols, rows)` | Resize the terminal window |-| `kill()` | Kill the session process |-| `disconnect()` | Close the WebSocket without killing the process |-| `wait()` | Block until the session exits; returns the exit code |--### Reconnecting--```python-# Disconnect and reconnect later (process keeps running)-await handle.disconnect()--handle2 = await c.pty.connect(- handle.pid,- on_data=lambda chunk: print(chunk.decode(errors="replace"), end=""),-)-```--## How it works--The PTY stack has four layers:--1. **`cua-auto`** — spawns a real PTY process (`pty` on Unix, `pywinpty` on Windows)-2. **`computer-server`** — exposes `/pty` REST endpoints and a WebSocket for I/O-3. **`computer`** — Python client (`PtyInterface`) connecting over HTTP + WebSocket-4. **`cua do shell`** — sets your terminal to raw mode and proxies I/O through the WebSocketdiff --git a/docs/content/docs/cua/guide/advanced/local-computer-server.mdx b/docs/content/docs/cua/guide/advanced/local-computer-server.mdxdeleted file mode 100644index bd9efb0fd5..0000000000--- a/docs/content/docs/cua/guide/advanced/local-computer-server.mdx+++ /dev/null@@ -1,177 +0,0 @@-----title: Local Computer Server-description: Run the computer server on your host machine for direct desktop control------The **computer server** is a local HTTP service that exposes your desktop for programmatic control. Use it when you want agents to control your actual machine instead of a sandbox, or when connecting to a self-hosted VM.--## When to Use--- **Development/debugging** - Test agents on your real desktop-- **Self-hosted VMs** - Connect to Windows/Linux VMs you manage yourself-- **Air-gapped environments** - Run everything locally without cloud dependencies-- **MCP integrations** - Let Claude Desktop control your computer--## Installation--Install the computer server package:--```bash-pip install cua-computer-server-```--Or install dependencies manually:--```bash-pip install uvicorn fastapi-pip install cua-```--## Starting the Server--Run the server on the default port (8000):--```bash-python -m computer_server-```--With debug logging:--```bash-python -m computer_server --log-level debug-```--Custom port:--```bash-python -m computer_server --port 9000-```--The server exposes endpoints for screenshots, mouse/keyboard input, and other computer operations.--## Connecting from Python--Use `use_host_computer_server=True` to connect to a running computer server:--```python-from cua import Sandbox, Image--# Connect to local computer server-async with Sandbox.create(use_host_computer_server=True) as computer:- # Now you can control your desktop- screenshot = await computer.interface.screenshot()- await computer.interface.left_click(500, 300)- await computer.interface.type_text("Hello from the agent!")-```--### Custom Host/Port--Connect to a computer server running on a different machine:--```python-async with Sandbox.create(- use_host_computer_server=True,- api_host="192.168.1.100",- api_port=8000-) as computer:- ...-```--## With ComputerAgent--Run agents against your desktop:--```python-from cua import Sandbox, Image, ComputerAgent--async with Sandbox.create(use_host_computer_server=True) as computer:- agent = ComputerAgent(- model="anthropic/claude-sonnet-4-5-20250929",- tools=[computer]- )-- async for result in agent.run("Open Firefox and go to github.com"):- print(result)-```--## Self-Hosted VMs--For VMs you manage yourself (e.g., Windows behind a VPN), install the computer server inside the VM and connect remotely:--```python-# Connect to Windows VM running computer-server-async with Sandbox.create(- use_host_computer_server=True,- api_host="windows-vm.local",- api_port=8000-) as computer:- ...-```--See the [Windows App Behind VPN](/cua/examples/platform-specific/windows-app-behind-vpn) example for a complete walkthrough.--## MCP Server Integration--To use with Claude Desktop, start the computer server and configure the MCP server:--```bash-# Terminal 1: Start computer server-python -m computer_server --log-level debug-```--```json-// Claude Desktop config-{- "mcpServers": {- "cua-agent": {- "command": "/bin/bash",- "args": ["~/.cua/start_mcp_server.sh"],- "env": {- "CUA_MODEL_NAME": "anthropic/claude-sonnet-4-20250514",- "ANTHROPIC_API_KEY": "your-key",- "CUA_USE_HOST_COMPUTER_SERVER": "true"- }- }- }-}-```--See [MCP Server Installation](/cua/reference/mcp-server/installation) for full setup instructions.--## Security Warning--The computer server gives full control over your desktop. Anyone who can reach the server can:--- Take screenshots of your screen-- Click anywhere on your desktop-- Type text and press keys-- Launch applications--**Recommendations:**--- Only run on localhost or trusted networks-- Don't expose to the public internet-- Stop the server when not in use-- Consider using sandboxes for untrusted workloads--## Troubleshooting--**Port already in use:**--```bash-lsof -i :8000 # Find what's using the port-kill -9 <PID> # Stop it-```--**Connection refused:**--- Ensure the server is running-- Check the host and port match-- Verify firewall allows the connection--**Permission errors (Linux):**--```bash-# May need to run with display access-DISPLAY=:0 python -m computer_server-```diff --git a/docs/content/docs/cua/guide/advanced/message-format.mdx b/docs/content/docs/cua/guide/advanced/message-format.mdxdeleted file mode 100644index 1646d2e3d2..0000000000--- a/docs/content/docs/cua/guide/advanced/message-format.mdx+++ /dev/null@@ -1,226 +0,0 @@-----title: Message Format-description: Type definitions for agent messages and actions------The Agent SDK uses a consistent message format for all conversations. This page provides type definitions for building custom integrations or processing agent outputs programmatically.--## Agent Response--Each iteration of `agent.run()` yields a response with outputs and usage:--```python-class AgentResponse(TypedDict):- output: list[AgentMessage] # Messages from this iteration- usage: Usage # Token counts and cost--class Usage(TypedDict, total=False):- prompt_tokens: int- completion_tokens: int- total_tokens: int- response_cost: float # Cost in USD-```--## Message Types--The `output` list contains different message types:--### User Message--Input from the user or system:--```python-class UserMessage(TypedDict, total=False):- type: Literal["message"]- role: Literal["user", "system", "developer"]- content: str | list[InputContent]-```--### Assistant Message--Text output from the agent:--```python-class AssistantMessage(TypedDict):- type: Literal["message"]- role: Literal["assistant"]- content: list[OutputContent]--class OutputContent(TypedDict):- type: Literal["output_text"]- text: str-```--### Reasoning Message--Agent's internal reasoning (when available):--```python-class ReasoningMessage(TypedDict):- type: Literal["reasoning"]- summary: list[SummaryContent]--class SummaryContent(TypedDict):- type: Literal["summary_text"]- text: str-```--### Computer Call--Action the agent wants to perform:--```python-class ComputerCallMessage(TypedDict):- type: Literal["computer_call"]- call_id: str- status: Literal["completed", "failed", "pending"]- action: ComputerAction-```--### Computer Call Output--Result of a computer action (screenshot):--```python-class ComputerCallOutputMessage(TypedDict):- type: Literal["computer_call_output"]- call_id: str- output: ComputerResultContent--class ComputerResultContent(TypedDict):- type: Literal["computer_screenshot", "input_image"]- image_url: str # Base64 data URL-```--### Function Call--Call to a custom tool:--```python-class FunctionCallMessage(TypedDict):- type: Literal["function_call"]- call_id: str- status: Literal["completed", "failed", "pending"]- name: str- arguments: str # JSON string--class FunctionCallOutputMessage(TypedDict):- type: Literal["function_call_output"]- call_id: str- output: str-```--## Computer Actions--Actions represent operations the agent performs. The format varies by provider.--### Common Actions--```python-class ClickAction(TypedDict):- type: Literal["click"]- button: Literal["left", "right", "wheel", "back", "forward"]- x: int- y: int--class DoubleClickAction(TypedDict):- type: Literal["double_click"]- x: int- y: int--class TypeAction(TypedDict):- type: Literal["type"]- text: str--class KeyPressAction(TypedDict):- type: Literal["keypress"]- keys: list[str] # e.g., ["ctrl", "c"]--class ScrollAction(TypedDict):- type: Literal["scroll"]- x: int- y: int- scroll_x: int- scroll_y: int--class MoveAction(TypedDict):- type: Literal["move"]- x: int- y: int--class DragAction(TypedDict):- type: Literal["drag"]- path: list[tuple[int, int]] # [(x1, y1), (x2, y2), ...]--class ScreenshotAction(TypedDict):- type: Literal["screenshot"]--class WaitAction(TypedDict):- type: Literal["wait"]-```--### Anthropic-Specific Actions--```python-class LeftMouseDownAction(TypedDict):- type: Literal["left_mouse_down"]- x: int- y: int--class LeftMouseUpAction(TypedDict):- type: Literal["left_mouse_up"]- x: int- y: int-```--## Processing Messages--Handle different message types in your code:--```python-async for result in agent.run(messages):- for item in result["output"]:- match item["type"]:- case "message":- print(f"Agent: {item['content'][0]['text']}")- case "reasoning":- print(f"Thinking: {item['summary'][0]['text']}")- case "computer_call":- print(f"Action: {item['action']['type']}")- case "computer_call_output":- print("Screenshot received")- case "function_call":- print(f"Tool call: {item['name']}")-```--## Full Type Union--For type checking, use the union of all message types:--```python-from typing import Union--AgentMessage = Union[- UserMessage,- AssistantMessage,- ReasoningMessage,- ComputerCallMessage,- ComputerCallOutputMessage,- FunctionCallMessage,- FunctionCallOutputMessage,-]--ComputerAction = Union[- ClickAction,- DoubleClickAction,- TypeAction,- KeyPressAction,- ScrollAction,- MoveAction,- DragAction,- ScreenshotAction,- WaitAction,- LeftMouseDownAction,- LeftMouseUpAction,-]-```diff --git a/docs/content/docs/cua/guide/advanced/meta.json b/docs/content/docs/cua/guide/advanced/meta.jsondeleted file mode 100644index ba5dd84457..0000000000--- a/docs/content/docs/cua/guide/advanced/meta.json+++ /dev/null@@ -1,20 +0,0 @@-{- "title": "Advanced",- "description": "Advanced topics for building with Cua",- "icon": "Layers",- "pages": [- "composed-models",- "custom-tools",- "sandboxed-python",- "local-computer-server",- "human-in-the-loop",- "vnc-recorder",- "demonstration-guided-skills",- "trajectories",- "tracing",- "chat-history",- "message-format",- "prompt-caching",- "telemetry"- ]-}diff --git a/docs/content/docs/cua/guide/advanced/prompt-caching.mdx b/docs/content/docs/cua/guide/advanced/prompt-caching.mdxdeleted file mode 100644index d21607f780..0000000000--- a/docs/content/docs/cua/guide/advanced/prompt-caching.mdx+++ /dev/null@@ -1,113 +0,0 @@-----title: Prompt Caching-description: Reduce costs and latency with provider-specific prompt caching------**Prompt caching** avoids reprocessing the same prompt content on repeated calls. This reduces costs and latency for long-running tasks or iterative workflows where the context stays mostly the same.--## Enabling Prompt Caching--Set `use_prompt_caching=True` when creating your agent:--```python-from cua import Sandbox, Image, ComputerAgent--async with Sandbox.ephemeral(Image.linux()) as computer:-- agent = ComputerAgent(- model="anthropic/claude-sonnet-4-5-20250929",- tools=[computer],- use_prompt_caching=True- )-```--## Provider Behavior--### Anthropic--When enabled, Cua adds `{"cache_control": "ephemeral"}` to messages. This tells Anthropic to cache the prompt for the duration of the session.--```python-# Required for Anthropic models-agent = ComputerAgent(- model="anthropic/claude-sonnet-4-5-20250929",- tools=[computer],- use_prompt_caching=True-)-```--### OpenAI--OpenAI automatically caches prompts over 1000 tokens. You don't need to set `use_prompt_caching`—it happens automatically.--```python-# Caching is automatic for OpenAI-agent = ComputerAgent(- model="openai/computer-use-preview",- tools=[computer]-)-```--### Other Providers--For providers that don't support caching, the parameter is ignored.--## When to Use--**Iterative tasks** - When the agent runs multiple turns with similar context:--```python-agent = ComputerAgent(- model="anthropic/claude-sonnet-4-5-20250929",- tools=[computer],- use_prompt_caching=True-)--# Each turn reuses cached context-for task in tasks:- async for result in agent.run(task):- process(result)-```--**Long system prompts** - When using detailed instructions that repeat across calls:--```python-agent = ComputerAgent(- model="anthropic/claude-sonnet-4-5-20250929",- tools=[computer],- instructions="Very long system prompt with detailed rules...",- use_prompt_caching=True-)-```--**Multi-turn conversations** - When conversation history grows but the beginning stays constant:--```python-agent = ComputerAgent(- model="anthropic/claude-sonnet-4-5-20250929",- tools=[computer],- use_prompt_caching=True-)--messages = [{"role": "system", "content": "Long context..."}]-for user_input in inputs:- messages.append({"role": "user", "content": user_input})- async for result in agent.run(messages):- messages.extend(result["output"])-```--## Cost Impact--With Anthropic's prompt caching:--- Cached input tokens cost 90% less than regular input tokens-- Cache writes have a small additional cost-- Cache hits provide significant savings on repeated runs--Check your usage in the response:--```python-async for result in agent.run(messages):- usage = result["usage"]- print(f"Tokens: {usage['total_tokens']}, Cost: ${usage['response_cost']:.4f}")-```diff --git a/docs/content/docs/cua/guide/advanced/sandboxed-python.mdx b/docs/content/docs/cua/guide/advanced/sandboxed-python.mdxdeleted file mode 100644index 7b10703bfc..0000000000--- a/docs/content/docs/cua/guide/advanced/sandboxed-python.mdx+++ /dev/null@@ -1,217 +0,0 @@-----title: Sandboxed Python-description: Execute Python functions securely in isolated virtual environments------Sometimes you don't want to rely on UI automation or shell commands—you just want to run Python code directly inside the sandbox. **Sandboxed Python** lets you execute Python functions in isolated virtual environments within your sandbox, giving you programmatic control without clicking through UIs.--## Why Run Code in the Sandbox?--- **Faster than UI** - Skip the clicks and run code directly-- **More reliable** - No flaky element detection or timing issues-- **Full Python power** - Use any library for data processing, API calls, or computation-- **Isolated** - Code runs in the sandbox, not on your host, with its own dependencies--## Running a Function in the Sandbox--Here's the basic pattern:--```python-from cua import Sandbox, Image--async with Sandbox.ephemeral(Image.linux()) as computer:- # First, install any packages you need-await computer.venv_install("my_env", ["requests", "pandas"])--# Define a function to run remotely-def fetch_data(url):- import requests- import pandas as pd-- response = requests.get(url)- data = response.json()- df = pd.DataFrame(data)- return {"rows": len(df), "columns": list(df.columns)}--# Execute it in the sandbox-result = await computer.venv_exec("my_env", fetch_data, "https://api.example.com/data")-print(result) # {"rows": 100, "columns": ["id", "name", "value"]}-```--The function runs inside the sandbox, not on your host. Cua automatically extracts the function's source code, serializes the arguments, sends everything to the sandbox, executes it, and returns the result.--## Setting Up Virtual Environments--Before running code, create a virtual environment and install packages:--```python-# Create "data_env" and install packages-await computer.venv_install("data_env", ["numpy", "scipy", "matplotlib"])--# Add more packages later-await computer.venv_install("data_env", ["seaborn", "scikit-learn"])-```--Virtual environments are stored in `~/.venvs/{name}` on Linux/macOS or `%USERPROFILE%\.venvs\{name}` on Windows sandboxes. They persist across function calls.--You can also run arbitrary shell commands with the environment activated:--```python-# Check installed packages-result = await computer.venv_cmd("data_env", "pip list")-print(result.output)--# Run a script-result = await computer.venv_cmd("data_env", "python /path/to/script.py")-```--## Using the `@sandboxed` Decorator--For cleaner code, use the decorator to mark functions for remote execution:--```python-from cua import Sandbox, Image, sandboxed--async with Sandbox.ephemeral(Image.linux()) as computer:- await computer.venv_install("api_env", ["requests"])--@sandboxed("api_env")-def call_api(endpoint, payload):- import requests- response = requests.post(endpoint, json=payload)- response.raise_for_status()- return response.json()--# Call it like a normal async function-result = await call_api("https://api.example.com/submit", {"name": "test"})-```--The decorator transforms the function so that calling it executes remotely. You can configure retry behavior and specify which computer instance to use:--```python-@sandboxed(venv_name="my_env", computer=computer, max_retries=5)-def unreliable_operation():- # Will retry up to 5 times on failure- ...-```--## How It Works Under the Hood--When you call a sandboxed function, Cua:--1. **Extracts source code** - Uses Python's AST to analyze your function-2. **Detects dependencies** - Automatically includes imports, helper functions, and closure variables-3. **Serializes arguments** - JSON-encodes and base64-wraps data for safe transport-4. **Executes remotely** - Runs the code in the sandbox's virtual environment-5. **Returns results** - Deserializes the output back to Python objects--This means your function can call other local functions, and they'll be included automatically:--```python-def helper(x):- return x * 2--@sandboxed("my_env")-def process(value):- return helper(value) + 10 # helper() is included automatically--result = await process(5) # Returns 20-```--## Handling Errors--Exceptions raised in the sandbox propagate back with full tracebacks:--```python-@sandboxed("my_env")-def might_fail(x):- if x < 0:- raise ValueError(f"Expected positive number, got {x}")- return x ** 2--try:- result = await might_fail(-5)-except ValueError as e:- print(f"Error from sandbox: {e}")- # Traceback shows the exact line in your function-```--The traceback includes both the remote stack (where the error occurred in the sandbox) and local context (where you called the function).--## Running Long Tasks in the Background--For tasks that take a while, run them in the background to avoid blocking:--```python-def long_running_analysis(data_path):- import pandas as pd- import time-- df = pd.read_csv(data_path)- # ... expensive computation ...- time.sleep(60)- return {"processed": len(df)}--# Start without waiting-pid = await computer.venv_exec_background(- "data_env",- long_running_analysis,- "/tmp/large_dataset.csv",- requirements=["pandas"] # Auto-installs if missing-)--print(f"Background task running with PID: {pid}")--# Continue with other work while it runs-```--## Example: Browser Automation with Playwright--Combine UI automation with Playwright running inside the sandbox:--```python-await computer.venv_install("browser_env", ["playwright"])-await computer.venv_cmd("browser_env", "playwright install chromium")--@sandboxed("browser_env")-def scrape_page(url):- from playwright.sync_api import sync_playwright-- with sync_playwright() as p:- browser = p.chromium.launch()- page = browser.new_page()- page.goto(url)-- title = page.title()- links = [a.get_attribute("href") for a in page.query_selector_all("a")]-- browser.close()-- return {"title": title, "link_count": len(links)}--result = await scrape_page("https://example.com")-```--## Example: Processing Agent-Scraped Data--After your agent extracts data from a UI, process it in the sandbox:--```python-# Agent scrapes a table from the UI-table_data = await agent.run("Extract the sales data table from the spreadsheet")--@sandboxed("data_env")-def analyze_sales(data):- import pandas as pd-- df = pd.DataFrame(data)- return {- "total_revenue": df["revenue"].sum(),- "top_product": df.loc[df["revenue"].idxmax(), "product"],- "monthly_avg": df.groupby("month")["revenue"].mean().to_dict()- }--analysis = await analyze_sales(table_data)-```--This pattern keeps your host clean while giving agents full computational capabilities inside isolated environments.diff --git a/docs/content/docs/cua/guide/advanced/telemetry.mdx b/docs/content/docs/cua/guide/advanced/telemetry.mdxdeleted file mode 100644index 11e06ad389..0000000000--- a/docs/content/docs/cua/guide/advanced/telemetry.mdx+++ /dev/null@@ -1,107 +0,0 @@-----title: Telemetry-description: Understanding and controlling Cua's anonymous usage collection------Cua collects anonymous usage statistics to improve the product. No personal data, screenshots, or conversation content is collected by default. You can opt out completely at any time.--## What's Collected--### By Default (Opt-Out)--Basic performance metrics:--- **System info** - OS, OS version, Python version-- **Module initialization** - When packages are imported and their versions-- **Performance** - Agent run durations, step counts, token usage, API costs-- **Session tracking** - Anonymous UUIDs for sessions and runs--### Opt-In Only--**Trajectory logging** captures full conversation history and must be explicitly enabled:--- User messages and agent responses-- Computer actions and outputs-- Agent reasoning traces--### Never Collected--- Personal information or user identifiers-- API keys or credentials-- File contents or application data-- Screenshots (unless trajectory logging is enabled)-- Text being typed or model outputs (unless trajectory logging is enabled)--## Disabling Telemetry--### Environment Variable--Set `CUA_TELEMETRY_ENABLED` to disable globally:--```bash-export CUA_TELEMETRY_ENABLED=false-```--### Per Instance--Disable on individual objects:--```python-from cua import Sandbox, Image, ComputerAgent--# Disable for Computer-async with Sandbox.ephemeral(Image.linux(), telemetry_enabled=False) as computer:- pass--# Disable for Agent-agent = ComputerAgent(- model="anthropic/claude-sonnet-4-5-20250929",- tools=[computer],- telemetry_enabled=False-)-```--### Check Status--```python-print(computer.telemetry_enabled) # True or False-print(agent.telemetry_enabled) # True, False, or dict-```--## Enabling Trajectory Logging--To opt in to full conversation logging:--```python-agent = ComputerAgent(- model="anthropic/claude-sonnet-4-5-20250929",- tools=[computer],- telemetry_enabled={"log_trajectory": True}-)-```--This sends complete conversation items to help improve Cua's models.--## Events Reference--### Computer SDK Events--| Event | Data | Trigger |-| ---------------------- | ------------------------------------ | ------------------------- |-| `computer_initialized` | OS, OS version, Python version | Computer instance created |-| `module_init` | Module name, version, Python version | Package first imported |--### Agent SDK Events--| Event | Data | Trigger |-| --------------------- | ------------------------------------------ | ----------------------- |-| `module_init` | Module name, version, Python version | Package first imported |-| `agent_session_start` | Session ID, agent type, model, system info | Agent instantiated |-| `agent_run_start` | Session/run IDs, input size, message count | `agent.run()` called |-| `agent_run_end` | Session/run IDs, duration, steps, usage | `agent.run()` completes |-| `agent_step` | Session/run IDs, step number, timestamp | Each agent iteration |-| `agent_usage` | Session/run IDs, token counts, cost | Usage info received |--## Questions--Questions about telemetry? Open an issue on [GitHub](https://github.com/trycua/cua).diff --git a/docs/content/docs/cua/guide/advanced/tracing.mdx b/docs/content/docs/cua/guide/advanced/tracing.mdxdeleted file mode 100644index 01daa6ea44..0000000000--- a/docs/content/docs/cua/guide/advanced/tracing.mdx+++ /dev/null@@ -1,269 +0,0 @@-----title: Tracing API-description: Record computer interactions for debugging, training, and analysis------The **Tracing API** records everything that happens on a Computer—screenshots, API calls, accessibility trees, and custom metadata. Unlike [Trajectories](/cua/guide/advanced/trajectories) which are specific to ComputerAgent, tracing works with any Computer usage: custom agents, scripts, human-in-the-loop workflows, or RPA.--## Basic Usage--Start and stop tracing around any computer operations:--```python-from cua import Sandbox, Image--async with Sandbox.ephemeral(Image.linux()) as computer:- # Start tracing- await computer.tracing.start()-- # Perform operations (these get recorded)- await computer.interface.left_click(100, 200)- await computer.interface.type_text("Hello, World!")- screenshot = await computer.interface.screenshot()-- # Stop and save trace- trace_path = await computer.tracing.stop()- print(f"Trace saved to: {trace_path}")-```--This creates a ZIP archive containing all recorded events, screenshots, and metadata.--## What Gets Recorded--By default, tracing captures:--- **Screenshots** - Taken automatically after key actions (clicks, typing, key presses)-- **API calls** - Every method called on `computer.interface` with arguments and results-- **Timestamps** - Absolute and relative timing for each event-- **Errors** - Any exceptions that occur during operations--You can also enable:--- **Accessibility trees** - Semantic structure of the UI (useful for debugging element detection)-- **Custom metadata** - Your own key-value data added during the session--## Configuration--Control what gets recorded with start options:--```python-await computer.tracing.start({- "name": "my-workflow", # Custom trace name- "screenshots": True, # Capture screenshots (default: True)- "api_calls": True, # Log interface calls (default: True)- "accessibility_tree": False, # Record a11y trees (default: False)- "metadata": True, # Enable custom metadata (default: True)-})-```--Control output format when stopping:--```python-# Save as ZIP archive (default)-trace_path = await computer.tracing.stop({"format": "zip"})--# Save as directory (easier to browse)-trace_path = await computer.tracing.stop({"format": "dir"})--# Custom output path-trace_path = await computer.tracing.stop({"path": "/tmp/my-trace.zip"})-```--## Output Format--Traces are saved with this structure:--```-trace_20250103_143052_abc123/-├── trace_metadata.json # Overall trace info-├── event_000001_trace_start.json-├── event_000002_api_call.json-├── event_000003_api_call.json-├── 000001_initial_screenshot.png-├── 000002_after_left_click.png-├── 000003_after_type_text.png-└── event_000004_trace_end.json-```--**trace_metadata.json:**--```json-{- "trace_id": "trace_20250103_143052_abc123",- "start_time": "2025-01-03T14:30:52Z",- "end_time": "2025-01-03T14:31:15Z",- "duration": 23.456,- "total_events": 12,- "screenshot_count": 5,- "config": {- "screenshots": true,- "api_calls": true- }-}-```--**API call event:**--```json-{- "type": "api_call",- "timestamp": 1704295852.789,- "relative_time": 3.666,- "data": {- "method": "left_click",- "args": { "x": 100, "y": 200 },- "result": null,- "error": null,- "screenshot": "000002_after_left_click.png",- "success": true- }-}-```--## Adding Custom Metadata--Add context during tracing for later analysis:--```python-await computer.tracing.start()--# Add metadata at any point-await computer.tracing.add_metadata("workflow", "login-flow")-await computer.tracing.add_metadata("user_id", "test-user-123")-await computer.tracing.add_metadata("step", "entering-credentials")--await computer.interface.type_text("username")--await computer.tracing.add_metadata("step", "clicking-submit")-await computer.interface.left_click(500, 300)--trace_path = await computer.tracing.stop()-```--Metadata appears in the trace alongside events, making it easy to correlate actions with your application context.--## Tracing with ComputerAgent--Tracing works alongside ComputerAgent—you get both agent-level information and detailed computer interaction logs:--```python-from cua import Sandbox, Image, ComputerAgent--async with Sandbox.ephemeral(Image.linux()) as computer:- # Start tracing before agent runs- await computer.tracing.start({"name": "agent-task"})-- agent = ComputerAgent(- model="anthropic/claude-sonnet-4-5-20250929",- tools=[computer],- trajectory_dir="trajectories" # Can use both!- )-- async for result in agent.run("Open Firefox and search for Cua"):- pass-- # Stop tracing after agent completes- trace_path = await computer.tracing.stop()-```--Now you have:--- **Trajectory** - Agent's reasoning, API calls to the model, action decisions-- **Trace** - Low-level computer interactions, screenshots, timing--## Tracing vs Trajectories--| Feature | Tracing API | Trajectories |-| ------------------ | ------------------------ | ------------------------- |-| Works with | Any Computer usage | ComputerAgent only |-| Records | Computer interface calls | Agent reasoning + actions |-| Screenshots | After each action | Per agent turn |-| Accessibility data | Optional | No |-| Custom metadata | Yes | No |-| Output format | ZIP or directory | Directory with turns |-| Real-time control | Start/stop anytime | Per agent run |--**Use Tracing when:**--- Building custom agents or scripts-- Recording human demonstrations-- Debugging computer interface issues-- Need fine-grained timing data-- Want accessibility tree snapshots--**Use Trajectories when:**--- Using ComputerAgent-- Need agent reasoning and model responses-- Want turn-by-turn organization-- Using the trajectory viewer--## Recording Human Demonstrations--Tracing is ideal for capturing human-driven workflows as training data:--```python-async with Sandbox.ephemeral(Image.linux()) as computer:- await computer.tracing.start({- "name": "human-demo-login",- "accessibility_tree": True # Capture UI structure- })-- # Human performs actions via VNC or other interface- # Meanwhile, your script can add context:- await computer.tracing.add_metadata("task", "login-to-dashboard")- await computer.tracing.add_metadata("demonstrator", "expert-user-1")-- # Wait for human to finish...- input("Press Enter when done with demonstration")-- trace_path = await computer.tracing.stop()- print(f"Demo saved to: {trace_path}")-```--## Privacy Considerations--Tracing is designed with privacy in mind:--- **Clipboard** - Only the length is recorded, not the actual content-- **Screenshots** - Can be disabled entirely with `screenshots: False`-- **Passwords** - Text input is recorded, so consider disabling for sensitive workflows--```python-# Privacy-conscious tracing-await computer.tracing.start({- "screenshots": False, # No visual data- "api_calls": True # Still log actions-})-```--## Debugging with Traces--When something goes wrong, traces show exactly what happened:--1. **Open trace_metadata.json** to see the overview-2. **Check event files** in order to find where things went wrong-3. **Look at screenshots** to see what the screen looked like-4. **Check error fields** in API call events for exceptions--```python-import json-from pathlib import Path--# Load a trace-trace_dir = Path("trace_20250103_143052_abc123")--with open(trace_dir / "trace_metadata.json") as f:- metadata = json.load(f)--print(f"Duration: {metadata['duration']:.2f}s")-print(f"Events: {metadata['total_events']}")-print(f"Screenshots: {metadata['screenshot_count']}")--# Find failed API calls-for event_file in sorted(trace_dir.glob("event_*.json")):- with open(event_file) as f:- event = json.load(f)- if event.get("data", {}).get("error"):- print(f"Error in {event_file.name}: {event['data']['error']}")-```diff --git a/docs/content/docs/cua/guide/advanced/trajectories.mdx b/docs/content/docs/cua/guide/advanced/trajectories.mdxdeleted file mode 100644index bca5f664cc..0000000000--- a/docs/content/docs/cua/guide/advanced/trajectories.mdx+++ /dev/null@@ -1,201 +0,0 @@-----title: Trajectories-description: Save and analyze agent execution for debugging and training------**Trajectories** record everything your agent does—every API call, response, screenshot, and action. Use them to debug issues, understand agent behavior, or create training data.--## Saving Trajectories--Enable trajectory saving with the `trajectory_dir` parameter:--```python-from cua import ComputerAgent--agent = ComputerAgent(- model="anthropic/claude-sonnet-4-5-20250929",- tools=[computer],- trajectory_dir="trajectories"-)--await agent.run("Open Firefox and search for Cua documentation")-```--This creates a directory structure with everything the agent did:--```-trajectories/-└── 2025-01-03_claude_143022_a1b2/- ├── metadata.json- ├── turn_000/- │ ├── 0000_api_start.json- │ ├── 0001_api_result.json- │ ├── 0002_screenshot.png- │ ├── 0003_agent_response.json- │ ├── 0004_computer_call_result.json- │ └── 0005_screenshot_action.png- └── turn_001/- └── ...-```--## What Gets Saved--Each trajectory includes:--- **metadata.json** - Run info, timestamps, total token usage, and costs-- **API calls** - The exact messages sent to the model-- **Model responses** - What the model decided to do-- **Screenshots** - Raw screenshots at each step-- **Action screenshots** - Screenshots with crosshairs showing where actions occurred-- **Computer call results** - Outcomes of clicks, typing, and other actions--## Trajectory Structure--The trajectory ID encodes useful information:--```-2025-01-03_claude_143022_a1b2-│ │ │ └── Unique ID (first 4 chars of UUID)-│ │ └── Time (HHMMSS)-│ └── Model name-└── Date (YYYY-MM-DD)-```--Each turn (iteration of the agent loop) gets its own directory with sequentially numbered artifacts.--## Configuration Options--Control trajectory behavior with a config dict:--```python-agent = ComputerAgent(- model="anthropic/claude-sonnet-4-5-20250929",- tools=[computer],- trajectory_dir={- "trajectory_dir": "trajectories",- "reset_on_run": False, # Continue same trajectory across runs- "screenshot_dir": "screenshots" # Save screenshots separately- }-)-```--**Options:**--| Option | Default | Description |-| ---------------- | -------- | ------------------------------------------- |-| `trajectory_dir` | Required | Base directory for trajectories |-| `reset_on_run` | `True` | Create new trajectory ID per run |-| `screenshot_dir` | `None` | Optional separate directory for screenshots |--Setting `reset_on_run=False` appends to the same trajectory across multiple `agent.run()` calls, useful for multi-step workflows.--## Viewing Trajectories--### Online Viewer--Upload your trajectory folder to the [Cua Trajectory Viewer](https://cua.ai/trajectory-viewer) for interactive exploration:--- Timeline slider to step through actions-- Screenshot gallery with action annotations-- API call and response inspection-- Usage statistics--The viewer runs entirely in your browser—no data is uploaded to any server.--### Manual Inspection--Open the JSON files directly to see exactly what happened:--**metadata.json:**--```json-{- "trajectory_id": "2025-01-03_claude_143022_a1b2",- "created_at": "2025-01-03T14:30:22Z",- "status": "completed",- "total_usage": {- "total_tokens": 5000,- "response_cost": 0.5- },- "total_turns": 5-}-```--**agent_response.json:**--```json-{- "type": "computer_call",- "action": {- "type": "click",- "coordinate": [500, 300]- },- "call_id": "call_abc123"-}-```--## Action Crosshairs--Screenshots are saved twice: once raw, once with crosshairs showing where the agent clicked. The `_action.png` suffix indicates an annotated screenshot:--- `0002_screenshot.png` - Original screenshot-- `0005_screenshot_action.png` - Screenshot with red crosshair at click location--This makes it easy to see exactly where the agent intended to click, which is invaluable for debugging misclicks.--## Using Trajectories for Debugging--When an agent fails, trajectories help you understand why:--1. **Open the trajectory folder** and find the turn where things went wrong-2. **Check the screenshot** to see what the agent saw-3. **Check the action screenshot** to see where it clicked-4. **Read the API result** to see what the model decided-5. **Compare to the computer_call_result** to see what actually happened--Common issues trajectories help identify:--- **Wrong element clicked** - Crosshair shows the agent clicked the wrong button-- **Element not visible** - Screenshot shows the target was off-screen or hidden-- **Timing issue** - Screenshot shows page wasn't fully loaded-- **Misinterpretation** - Model response shows incorrect reasoning--## Example: Saving and Analyzing--```python-from cua import ComputerAgent-from pathlib import Path-import json--agent = ComputerAgent(- model="anthropic/claude-sonnet-4-5-20250929",- tools=[computer],- trajectory_dir="trajectories"-)--await agent.run("Log into the admin panel and export user data")--# Find the latest trajectory-trajectories = sorted(Path("trajectories").iterdir())-latest = trajectories[-1]--# Load metadata-with open(latest / "metadata.json") as f:- metadata = json.load(f)--print(f"Trajectory: {metadata['trajectory_id']}")-print(f"Status: {metadata['status']}")-print(f"Turns: {metadata['total_turns']}")-print(f"Cost: ${metadata['total_usage']['response_cost']:.4f}")-```--## Training Data--Trajectories can be used as training data for fine-tuning computer-use models. The structured format includes:--- Input screenshots-- Model decisions-- Action coordinates-- Outcomes--This is the same format used for collecting human demonstrations for training.diff --git a/docs/content/docs/cua/guide/advanced/vnc-recorder.mdx b/docs/content/docs/cua/guide/advanced/vnc-recorder.mdxdeleted file mode 100644index c46ec2be4a..0000000000--- a/docs/content/docs/cua/guide/advanced/vnc-recorder.mdx+++ /dev/null@@ -1,154 +0,0 @@-----title: VNC Recorder-description: Record human demonstrations from Cua sandboxes for training and evaluation------Cua sandboxes include a [noVNC fork](https://github.com/trycua/novnc) with built-in recording capabilities. This lets you capture human demonstrations as replayable session files for training data collection, evaluation baselines, or debugging.--<video controls width="100%">- <source src="/docs/videos/demonstration-recorder.mp4" type="video/mp4" />-</video>--## Recording with the Cua CLI--The easiest way to record demonstrations is with the `cua skills` command.--### 1. Start recording--Connect to a sandbox and start recording:--```bash-# Record from a Cua Cloud sandbox-cua skills record --sandbox my-sandbox--# Or record from any VNC URL-cua skills record --vnc-url http://localhost:8006/vnc.html-```--This opens the VNC interface with recording enabled. You'll be prompted for a skill name and description after recording.--### 2. Perform your demonstration--Interact with the sandbox normally. All mouse and keyboard input is captured along with video.--### 3. Stop and save--Click **Stop Recording** in the VNC panel. The CLI processes the recording, generates captions using a VLM, and saves the skill to `~/.cua/skills/<name>/`.--### 4. Replay--View the recorded video:--```bash-cua skills replay my-skill-```--List all saved skills:--```bash-cua skills list-```--Read a skill's prompt:--```bash-cua skills read my-skill-```-------## Manual Recording--### 1. Start a sandbox and open the VNC UI--Start a sandbox using [Docker](/cua/guide/get-started/set-up-sandbox) or [Cua Cloud](/cua/guide/get-started/set-up-sandbox). The VNC UI will be available at:--- **Docker**: `http://localhost:8006`-- **Cloud**: `https://{sandbox-name}.sandbox.cua.ai/vnc.html`--### 2. Enable recording--Add `autorecord=1` to the VNC URL to automatically start recording when you connect:--```-http://localhost:8006/vnc.html?autoconnect=1&autorecord=1-```--Or click the **record button** in the control bar before connecting.--### 3. Perform your demonstration--Interact with the sandbox normally. All VNC traffic (screen updates and input events) is captured.--### 4. Download the recording--Click **Stop Recording**, then **Download** to save the session as a `.js` file.--### 5. Replay the recording--Start the playback UI:--```bash-# Clone the noVNC fork-git clone https://github.com/trycua/novnc-cd novnc--# Move your recording into the recordings folder-mv ~/Downloads/vnc-recording-*.js ./recordings/--# Start the local server-./utils/novnc_proxy-```--Then open the playback URL:--```-http://localhost:6080/tests/vnc_playback.html?data=../recordings/your-recording.js-```--## Streaming to External Server--For automated pipelines, stream recordings directly to a custom server using the `record_url` parameter. The recording data is streamed as a websockify stream over WebSocket.--Here's a simple recording server:--```python-import asyncio-import websockets--async def handle_recording(websocket):- with open("recording.bin", "wb") as f:- async for message in websocket:- f.write(message)- print("Recording saved")--async def main():- async with websockets.serve(handle_recording, "localhost", 6090):- print("Recording server running on ws://localhost:6090")- await asyncio.Future()--asyncio.run(main())-```--Then connect with streaming enabled:--```-http://localhost:8006/vnc.html?autoconnect=1&autorecord=1&record_url=ws://localhost:6090-```--Recording data streams to disk in real-time via WebSocket, so the file is ready immediately when the session ends.--## URL Parameters--| Parameter | Description |-| --------------------- | ---------------------------------------------- |-| `autorecord=1` | Start recording when connection is established |-| `record_url=ws://...` | Stream recording to external WebSocket server |--For more details on prompting agents with demonstrations, see [Demonstration-Guided Skills](/cua/guide/advanced/demonstration-guided-skills).--## Reference--For related capabilities, see:--- [ShowUI-Aloha](https://github.com/showlab/ShowUI-Aloha) - Human-taught GUI agent that learns from demonstrationsdiff --git a/docs/content/docs/cua/guide/agents-in-sandbox/claude-code.mdx b/docs/content/docs/cua/guide/agents-in-sandbox/claude-code.mdxdeleted file mode 100644index cce555baa2..0000000000--- a/docs/content/docs/cua/guide/agents-in-sandbox/claude-code.mdx+++ /dev/null@@ -1,153 +0,0 @@-----title: Claude Code-description: Run Claude Code inside a Cua cloud sandbox-icon: claude-code------import { Callout } from 'fumadocs-ui/components/callout';-import { Step, Steps } from 'fumadocs-ui/components/steps';--Hand off coding tasks to [Claude Code](https://docs.anthropic.com/en/docs/claude-code) running in an isolated Cua cloud sandbox with full filesystem, terminal, and git access.--## Setup--<Steps>--<Step>--### Install the SDK--```bash-pip install cua-sandbox-```--</Step>--<Step>--### Set your keys--```bash-export CUA_API_KEY="your-cua-key"-export ANTHROPIC_API_KEY="your-anthropic-key"-```--</Step>--</Steps>--## Run Claude Code in a cloud sandbox--This spins up an ephemeral cloud VM via the Cua API. You'll need a `CUA_API_KEY` — get one from [cua.ai/dashboard](https://cua.ai/dashboard) if you don't have one yet.--```python-import asyncio, os-from cua_sandbox import Sandbox, Image--async def main():- api_key = os.environ["ANTHROPIC_API_KEY"]-- # CUA_API_KEY is read from your environment automatically- async with Sandbox.ephemeral(Image.linux("ubuntu", "24.04")) as sb:- # Install Node.js + Claude Code- await sb.shell.run("sudo apt-get update && sudo apt-get install -y curl", timeout=60)- await sb.shell.run("curl -fsSL https://deb.nodesource.com/setup_20.x | sudo bash -", timeout=60)- await sb.shell.run("sudo apt-get install -y nodejs", timeout=60)- await sb.shell.run("sudo npm install -g @anthropic-ai/claude-code", timeout=120)-- # Hand off a task- result = await sb.shell.run(- f"ANTHROPIC_API_KEY='{api_key}' claude -p 'Create a hello world index.html' "- f"--dangerously-skip-permissions --output-format stream-json --verbose",- timeout=300,- )- print(result.stdout)--asyncio.run(main())-```--<Callout type="info">- `--dangerously-skip-permissions` is required for headless execution. `-p` passes the task as a prompt.-</Callout>--## Connect to an existing sandbox--Skip provisioning by connecting to a sandbox that's already running.--```python-sb = await Sandbox.connect("my-sandbox")-async with sb:- result = await sb.shell.run(- f"ANTHROPIC_API_KEY='{api_key}' claude -p 'List all TODO comments' "- f"--dangerously-skip-permissions",- timeout=300,- )- print(result.stdout)-```--## Work on a repository--Clone a repo, run a task, and inspect the diff.--```python-await sb.shell.run("git clone https://github.com/user/repo.git /tmp/repo", timeout=60)--result = await sb.shell.run(- f"cd /tmp/repo && ANTHROPIC_API_KEY='{api_key}' claude -p 'Fix the failing tests' "- f"--dangerously-skip-permissions --output-format stream-json --verbose",- timeout=300,-)-print(result.stdout)--diff = await sb.shell.run("cd /tmp/repo && git diff", timeout=30)-print(diff.stdout)-```--## Structured output--`--output-format stream-json --verbose` emits one JSON object per line.--| Event type | Description |-|------------|-------------|-| `system` | Session init — tools, model, config |-| `assistant` | Claude's responses and tool calls |-| `result` | Final summary — cost, duration, tokens |--```python-import json--for line in result.stdout.strip().splitlines():- event = json.loads(line)- if event["type"] == "result":- print(f"Cost: ${event['total_cost_usd']:.4f}")- print(f"Result: {event['result']}")-```--## Resume a session--Extract the `session_id` from the init event and pass it back.--```python-init = json.loads(result.stdout.strip().splitlines()[0])-session_id = init["session_id"]--followup = await sb.shell.run(- f"ANTHROPIC_API_KEY='{api_key}' claude -p 'Now add CSS styling' "- f"--dangerously-skip-permissions --output-format stream-json --verbose "- f"--session-id {session_id}",- timeout=300,-)-```--## Custom system prompt--Write a `CLAUDE.md` into the repo before running Claude Code.--```python-await sb.shell.run(- "echo '# Rules\n- Use TypeScript\n- Write tests for all changes' > /tmp/repo/CLAUDE.md",- timeout=10,-)-```--Claude Code automatically reads `CLAUDE.md` from the working directory.diff --git a/docs/content/docs/cua/guide/agents-in-sandbox/meta.json b/docs/content/docs/cua/guide/agents-in-sandbox/meta.jsondeleted file mode 100644index 17af893e65..0000000000--- a/docs/content/docs/cua/guide/agents-in-sandbox/meta.json+++ /dev/null@@ -1,7 +0,0 @@-{- "title": "Agents in Sandbox",- "description": "Run coding agents inside Cua sandboxes",- "icon": "Bot",- "defaultOpen": true,- "pages": ["claude-code"]-}diff --git a/docs/content/docs/cua/guide/fundamentals/agent-loops.mdx b/docs/content/docs/cua/guide/fundamentals/agent-loops.mdxdeleted file mode 100644index 54493f68a6..0000000000--- a/docs/content/docs/cua/guide/fundamentals/agent-loops.mdx+++ /dev/null@@ -1,142 +0,0 @@-----title: Agent Loops-description: Supported computer-using agent loops and models------<Callout>- A corresponding <a href="https://github.com/trycua/cua/blob/main/notebooks/agent_nb.ipynb" target="_blank">Jupyter Notebook</a> is available for this documentation.-</Callout>--An **agent loop** runs iteratively until a task is complete:--1. **Observe** → Take a screenshot-2. **Reason** → VLM decides the next action-3. **Act** → Execute (click, type, scroll)-4. **Repeat** → Until done--To run an agent loop simply do:--```python-from cua import ComputerAgent-import asyncio-from cua import Sandbox, Image---async def take_screenshot():- async with Sandbox.create(- Image.linux(),- name="your-sandbox-name",- api_key="your-api-key"- ) as computer:-- agent = ComputerAgent(- model="anthropic/claude-sonnet-4-5-20250929",- tools=[computer],- max_trajectory_budget=5.0- )-- messages = [{"role": "user", "content": "Take a screenshot and tell me what you see"}]-- async for result in agent.run(messages):- for item in result["output"]:- if item["type"] == "message":- print(item["content"][0]["text"])---if __name__ == "__main__":- asyncio.run(take_screenshot())-```--For a list of supported models and configurations, see the [Vision Language Models](/cua/guide/fundamentals/vlms) page.--## Configuration--Before running an agent, set the appropriate API key as an environment variable:--```bash-# For cloud sandboxes and Cua VLM Router-export CUA_API_KEY="your-cua-api-key"--# For direct provider access (BYOK - Bring Your Own Key)-export ANTHROPIC_API_KEY="your-anthropic-key"-export OPENAI_API_KEY="your-openai-key"-```--Use `CUA_API_KEY` when using Cua's cloud infrastructure or VLM Router. Use provider-specific keys when connecting directly to Anthropic or OpenAI.--## Sending Tasks--You can send tasks to the agent as a simple string or as a message list for multi-turn conversations:--```python-# Single task-async for result in agent.run("Open Firefox and go to google.com"):- process(result)--# Multi-turn conversation (continues from previous context)-messages = [- {"role": "user", "content": "Take a screenshot"},- {"role": "assistant", "content": "Done. I can see a desktop with Firefox open."},- {"role": "user", "content": "Now click the search bar"}-]-async for result in agent.run(messages):- process(result)-```--Multi-turn conversations are useful when you need the agent to build on previous actions or when implementing human-in-the-loop workflows.--## Processing Results--Each iteration yields a result containing the agent's outputs and token usage:--```python-async for result in agent.run(messages):- for item in result["output"]:- match item["type"]:- case "message":- # Agent's text response- print(item["content"][0]["text"])- case "computer_call":- # Action the agent is taking (screenshot, click, type)- print(f"Action: {item['action']['type']}")- case "computer_call_output":- # Result of the action (e.g., screenshot image)- print(f"Output received for {item['call_id']}")-- # Track costs- print(f"Cost so far: ${result['usage']['response_cost']:.4f}")-```--### Streaming Responses--For real-time feedback, enable streaming to receive partial results as they're generated:--```python-async for result in agent.run(messages, stream=True):- for item in result["output"]:- if item["type"] == "message":- # Print text as it arrives- print(item["content"][0]["text"], end="", flush=True)-```--## Handling Errors--The agent respects the `max_trajectory_budget` you set. When the budget is exceeded, the agent raises an exception:--```python-from cua_agent.exceptions import BudgetExceededException--agent = ComputerAgent(- model="anthropic/claude-sonnet-4-5-20250929",- tools=[computer],- max_trajectory_budget=5.0 # Max $5 per run-)--try:- async for result in agent.run(messages):- process(result)-except BudgetExceededException:- print("Task exceeded budget limit")-```--This prevents runaway costs when agents get stuck in loops or take longer than expected.diff --git a/docs/content/docs/cua/guide/fundamentals/browser-tool.mdx b/docs/content/docs/cua/guide/fundamentals/browser-tool.mdxdeleted file mode 100644index 9b3a9382fa..0000000000--- a/docs/content/docs/cua/guide/fundamentals/browser-tool.mdx+++ /dev/null@@ -1,190 +0,0 @@-----title: Browser Tool-description: Specialized browser automation for web-focused models------The **Browser Tool** provides a specialized interface for browser automation. While full computer-use models control the entire desktop, some models like Gemini 2.5 Computer-Use and Microsoft Fara are optimized specifically for web interactions. The Browser Tool gives these models browser-specific actions that are more efficient than simulating desktop clicks.--## When to Use Browser Tool--Use the Browser Tool when:--- You're automating web-based workflows (forms, data extraction, navigation)-- You're using a browser-optimized model (Gemini 2.5, Fara)-- You want browser-specific actions like direct URL navigation or web search--Use the standard Computer tool when:--- You need to control desktop applications-- You're using a full computer-use model (Claude, OpenAI, UI-TARS, Qwen)-- Your workflow spans multiple applications--## Basic Usage--The Browser Tool uses Playwright under the hood to control the browser:--```python-from cua import Sandbox, Image, ComputerAgent-from cua.tools import BrowserTool--# Create a computer instance-async with Sandbox.ephemeral(Image.linux()) as computer:- # Create a browser tool from the computer interface- browser = BrowserTool(interface=computer)--# Use with a browser-optimized model-agent = ComputerAgent(- model="gemini-2.5-computer-use-preview-10-2025",- tools=[browser]-)--async for result in agent.run("Go to github.com and search for 'cua'"):- print(result)-```--The agent sees the browser viewport and can issue browser-specific commands.--## Browser-Specific Actions--The Browser Tool provides actions that aren't available with standard computer-use:--### Direct URL Navigation--Instead of clicking the address bar and typing, go directly to a URL:--```python-# Agent can use visit_url action-await agent.run("Navigate to https://docs.cua.ai")-```--The model issues a `visit_url` action that immediately loads the page—no keyboard or mouse simulation needed.--### Web Search--Perform searches without navigating to a search engine first:--```python-# Agent can use web_search action-await agent.run("Search for 'python async tutorial'")-```--The `web_search` action opens a search directly, saving multiple interaction steps.--### Browser History--Navigate back without finding and clicking the back button:--```python-# Agent can use history_back action-await agent.run("Go back to the previous page")-```--## Using Browser Tool Directly--You can also call Browser Tool actions programmatically without an agent:--```python-from cua.tools import BrowserTool--browser = BrowserTool(interface=computer)--# Navigate to a URL-await browser.visit_url("https://example.com")--# Take a screenshot-screenshot = await browser.screenshot()--# Click at coordinates-await browser.click(x=500, y=300)--# Type text-await browser.type("Hello from Browser Tool")--# Scroll the page (positive = up, negative = down)-await browser.scroll(delta_x=0, delta_y=500)--# Perform a web search-await browser.web_search("Python documentation")--# Go back in history-await browser.history_back()-```--This is useful for scripted automation where you don't need AI reasoning.--## Coordinate Handling--Browser-optimized models often work with resized images for efficiency. The Browser Tool automatically handles coordinate translation:--1. **Model sees** - A resized screenshot optimized for the VLM-2. **Model outputs** - Coordinates in the resized image space-3. **Browser Tool translates** - Converts to actual viewport coordinates-4. **Playwright executes** - Click/type at the correct screen position--You don't need to handle this manually—coordinates from the model are automatically mapped to the correct viewport position.--## Memory System--The Browser Tool includes a memory feature for multi-step tasks:--```python-# Agent can memorize facts during execution-await agent.run("Find the pricing for the Pro plan and remember it, then go to the FAQ page")-```--The `pause_and_memorize_fact` action lets the agent store information for later reference within the same session.--## Example: Form Automation--Here's a complete example automating a web form:--```python-from cua import Sandbox, Image, ComputerAgent-from cua.tools import BrowserTool--async with Sandbox.ephemeral(Image.linux()) as computer:- browser = BrowserTool(interface=computer)--agent = ComputerAgent(- model="gemini-2.5-computer-use-preview-10-2025",- tools=[browser]-)--# Multi-step form automation-instructions = """-1. Go to https://example.com/contact-2. Fill in the name field with "John Doe"-3. Fill in the email field with "[email protected]"-4. Select "Sales Inquiry" from the dropdown-5. Type a message asking about enterprise pricing-6. Click the Submit button-"""--async for result in agent.run(instructions):- for item in result["output"]:- if item["type"] == "message":- print(item["content"][0]["text"])-```--## Supported Models--These models are designed for browser automation and work best with the Browser Tool:--- **Gemini 2.5 Computer-Use** (`gemini-2.5-computer-use-preview-10-2025`)-- **Microsoft Fara** (`azure_ml/Fara-7B`)--Full computer-use models (Claude, OpenAI, UI-TARS, Qwen) can also use the Browser Tool, but they're designed for broader desktop control and may not take full advantage of browser-specific actions.--## Browser Tool vs Computer Tool--| Feature | Browser Tool | Computer Tool |-| --------------------------- | ------------ | ------------------------ |-| Direct URL navigation | ✅ | ❌ (requires typing) |-| Web search action | ✅ | ❌ (requires browser UI) |-| History navigation | ✅ | ❌ (requires clicking) |-| Desktop applications | ❌ | ✅ |-| System dialogs | ❌ | ✅ |-| Multi-application workflows | ❌ | ✅ |-| Playwright integration | ✅ Native | ❌ |--Choose Browser Tool for web-focused tasks where efficiency matters. Choose Computer Tool for workflows that span multiple applications or require desktop access.diff --git a/docs/content/docs/cua/guide/fundamentals/callbacks.mdx b/docs/content/docs/cua/guide/fundamentals/callbacks.mdxdeleted file mode 100644index 03afdcbe37..0000000000--- a/docs/content/docs/cua/guide/fundamentals/callbacks.mdx+++ /dev/null@@ -1,260 +0,0 @@-----title: Callbacks-description: Hook into agent lifecycle events for logging, cost tracking, and more------**Callbacks** let you hook into the agent's execution lifecycle. Use them to log events, track costs, save trajectories, limit context size, or add custom behavior at any point during agent execution.--## Using Callbacks--Pass callbacks when creating your agent:--```python-from cua import ComputerAgent-from cua.callbacks import LoggingCallback, BudgetManagerCallback-import logging--agent = ComputerAgent(- model="anthropic/claude-sonnet-4-5-20250929",- tools=[computer],- callbacks=[- LoggingCallback(level=logging.INFO),- BudgetManagerCallback(max_budget=5.0)- ]-)-```--Many callbacks also have shorthand parameters on `ComputerAgent`:--```python-agent = ComputerAgent(- model="anthropic/claude-sonnet-4-5-20250929",- tools=[computer],- verbosity=logging.INFO, # Adds LoggingCallback- max_trajectory_budget=5.0, # Adds BudgetManagerCallback- only_n_most_recent_images=3, # Adds ImageRetentionCallback- trajectory_dir="trajectories", # Adds TrajectorySaverCallback- instructions="Be concise" # Adds PromptInstructionsCallback-)-```--## Built-in Callbacks--### LoggingCallback--Logs agent lifecycle events with configurable verbosity:--```python-from cua.callbacks import LoggingCallback-import logging--agent = ComputerAgent(- model="...",- tools=[computer],- callbacks=[LoggingCallback(level=logging.DEBUG)]-)--# Or use shorthand-agent = ComputerAgent(- model="...",- tools=[computer],- verbosity=logging.INFO-)-```--Logs include API calls, computer actions, text responses, and usage statistics. Image URLs are automatically sanitized in logs.--### BudgetManagerCallback--Tracks costs and stops execution when a budget limit is reached:--```python-from cua.callbacks import BudgetManagerCallback--agent = ComputerAgent(- model="...",- tools=[computer],- callbacks=[BudgetManagerCallback(- max_budget=10.0, # Dollar limit- reset_after_each_run=True, # Reset per run (default)- raise_error=False # Graceful stop vs exception- )]-)--# Or use shorthand-agent = ComputerAgent(- model="...",- tools=[computer],- max_trajectory_budget=5.0-)-```--When `raise_error=True`, the agent raises `BudgetExceededError` instead of stopping gracefully.--### ImageRetentionCallback--Limits how many screenshots are kept in the message history to prevent context overflow:--```python-from cua.callbacks import ImageRetentionCallback--agent = ComputerAgent(- model="...",- tools=[computer],- callbacks=[ImageRetentionCallback(only_n_most_recent_images=3)]-)--# Or use shorthand-agent = ComputerAgent(- model="...",- tools=[computer],- only_n_most_recent_images=3-)-```--Older screenshots are removed along with their associated computer call items.--### TrajectorySaverCallback--Saves complete agent conversations for debugging and analysis:--```python-from cua.callbacks import TrajectorySaverCallback--agent = ComputerAgent(- model="...",- tools=[computer],- callbacks=[TrajectorySaverCallback(- trajectory_dir="trajectories",- reset_on_run=True, # New trajectory per run- screenshot_dir="screenshots" # Optional separate screenshot dir- )]-)--# Or use shorthand-agent = ComputerAgent(- model="...",- tools=[computer],- trajectory_dir="trajectories"-)-```--See [Trajectories](/cua/guide/advanced/trajectories) for details on the saved format.--### PromptInstructionsCallback--Prepends custom instructions to every LLM call:--```python-from cua.callbacks import PromptInstructionsCallback--agent = ComputerAgent(- model="...",- tools=[computer],- callbacks=[PromptInstructionsCallback("Always confirm before clicking buttons")]-)--# Or use shorthand-agent = ComputerAgent(- model="...",- tools=[computer],- instructions="Always confirm before clicking buttons"-)-```--This is a simple way to guide agent behavior without modifying agent loops.--## Lifecycle Hooks--Callbacks can implement these hooks (all are optional):--```python-from cua_agent.callbacks.base import AsyncCallbackHandler--class MyCallback(AsyncCallbackHandler):- async def on_run_start(self, kwargs, old_items):- """Called when agent.run() begins"""- pass-- async def on_run_continue(self, kwargs, old_items, new_items) -> bool:- """Called before each iteration. Return False to stop."""- return True-- async def on_llm_start(self, messages):- """Preprocess messages before LLM call. Return modified messages."""- return messages-- async def on_llm_end(self, messages):- """Postprocess messages after LLM call. Return modified messages."""- return messages-- async def on_usage(self, usage):- """Called with token usage info after each LLM call"""- print(f"Tokens: {usage.total_tokens}, Cost: ${usage.response_cost:.4f}")-- async def on_computer_call_start(self, item):- """Called before a computer action (click, type, etc.)"""- pass-- async def on_computer_call_end(self, item, result):- """Called after a computer action completes"""- pass-- async def on_screenshot(self, screenshot, name):- """Called when a screenshot is taken"""- pass-- async def on_run_end(self, kwargs, old_items, new_items):- """Called when agent.run() completes"""- pass-```--## Creating Custom Callbacks--Build your own callback by extending `AsyncCallbackHandler`:--```python-from cua_agent.callbacks.base import AsyncCallbackHandler--class ActionCounterCallback(AsyncCallbackHandler):- def __init__(self):- self.action_count = 0-- async def on_computer_call_end(self, item, result):- self.action_count += 1- print(f"Actions taken: {self.action_count}")-- async def on_run_end(self, kwargs, old_items, new_items):- print(f"Total actions in run: {self.action_count}")- self.action_count = 0 # Reset for next run--# Use it-counter = ActionCounterCallback()-agent = ComputerAgent(- model="...",- tools=[computer],- callbacks=[counter]-)--await agent.run("Open Firefox and search for Cua")-print(f"Final count: {counter.action_count}")-```--## Callback Execution Order--Callbacks execute in this order during a run:--1. `on_run_start` - Once at the beginning-2. For each iteration:- - `on_run_continue` - Check if should continue- - `on_llm_start` - Before LLM call- - `on_api_start` - Before API request- - `on_api_end` - After API response- - `on_usage` - With usage stats- - `on_llm_end` - After LLM processing- - `on_responses` - With model responses- - `on_text` / `on_computer_call_start` / `on_computer_call_end` - Per response item- - `on_screenshot` - When screenshots are taken-3. `on_run_end` - Once at the end--Multiple callbacks are called in the order they appear in the `callbacks` list.diff --git a/docs/content/docs/cua/guide/fundamentals/cua-vlm-router.mdx b/docs/content/docs/cua/guide/fundamentals/cua-vlm-router.mdxdeleted file mode 100644index 6da44f92b9..0000000000--- a/docs/content/docs/cua/guide/fundamentals/cua-vlm-router.mdx+++ /dev/null@@ -1,102 +0,0 @@-----title: Cua VLM Router-description: Unified API access to multiple VLM providers without managing infrastructure------Instead of deploying and managing computer-use vision language models yourself on various cloud providers, Cua VLM Router gives you instant access to all supported models through a single API key. No infrastructure setup, no multiple provider accounts—just one integration for Claude, Gemini, Qwen, and more.--## Setup--```python-from cua import Sandbox, Image, ComputerAgent--async with Sandbox.ephemeral(Image.linux()) as computer:- agent = ComputerAgent(- model="cua/anthropic/claude-sonnet-4.5",- tools=[computer]- )-```--```bash-export CUA_API_KEY="your-cua-key"-```--Get your API key at [cua.ai](https://cua.ai/signin).--## Available Models--| Model | Description |-| ----------------------------------- | ----------------------------- |-| `cua/anthropic/claude-sonnet-4.5` | General-purpose (recommended) |-| `cua/anthropic/claude-haiku-4.5` | Fast, cost-effective |-| `cua/anthropic/claude-opus-4.5` | Most capable |-| `cua/google/gemini-3-flash-preview` | Fastest, cheapest |-| `cua/google/gemini-3-pro-preview` | Most powerful |-| `cua/microsoft/fara-7b` | Good at browser use |-| `cua/qwen/qwen3-vl-30b` | Open source & powerful |--## Benefits--- **Single API key** for all supported providers-- **Cost tracking** in every response-- **Unified dashboard** for usage monitoring-- **No vendor lock-in** - switch models without changing code--## Cost Tracking--Every response includes usage information:--```python-async for result in agent.run(messages):- print(f"Cost: ${result['usage']['response_cost']:.4f}")- print(f"Tokens: {result['usage']['total_tokens']}")-```--## HTTP API--Use the router directly via HTTP for integrations outside Python:--### Anthropic-compatible Messages--```bash-curl -X POST https://inference.cua.ai/v1/messages \- -H "Authorization: Bearer $CUA_API_KEY" \- -H "Content-Type: application/json" \- -d '{- "model": "anthropic/claude-sonnet-4.5",- "messages": [{"role": "user", "content": "Hello!"}],- "max_tokens": 100- }'-```--### OpenAI-compatible Chat Completions--```bash-curl -X POST https://inference.cua.ai/v1/chat/completions \- -H "Authorization: Bearer $CUA_API_KEY" \- -H "Content-Type: application/json" \- -d '{- "model": "microsoft/fara-7b",- "messages": [{"role": "user", "content": "Hello!"}],- "max_tokens": 100- }'-```--### Check Balance--```bash-curl -H "Authorization: Bearer $CUA_API_KEY" \- "https://inference.cua.ai/v1/balance"-```--## When to Use--**Multi-provider workflows** - Access Claude, Gemini, and Qwen from a single integration.--**Cost monitoring** - Track spending across all providers in one dashboard.--**Team environments** - Share a single API key with usage limits and monitoring.--**Rapid prototyping** - Try different models without setting up individual provider accounts.--For direct provider access or custom billing arrangements, use the provider-specific APIs instead. See [Vision Language Models](/cua/guide/fundamentals/vlms) for direct provider setup.diff --git a/docs/content/docs/cua/guide/fundamentals/meta.json b/docs/content/docs/cua/guide/fundamentals/meta.jsondeleted file mode 100644index 4bf64fa359..0000000000--- a/docs/content/docs/cua/guide/fundamentals/meta.json+++ /dev/null@@ -1,7 +0,0 @@-{- "title": "Agent",- "description": "Core concepts for building agents with Cua",- "icon": "Lightbulb",- "defaultOpen": false,- "pages": ["agent-loops", "vlms", "cua-vlm-router", "browser-tool", "callbacks"]-}diff --git a/docs/content/docs/cua/guide/fundamentals/vlms.mdx b/docs/content/docs/cua/guide/fundamentals/vlms.mdxdeleted file mode 100644index 6788bf2efd..0000000000--- a/docs/content/docs/cua/guide/fundamentals/vlms.mdx+++ /dev/null@@ -1,196 +0,0 @@-----title: Vision Language Models-description: VLMs supported by the Cua Agent Framework------A **Vision Language Model (VLM)** is the AI that powers your computer-use agent. It sees screenshots, reasons about what's on screen, and decides what action to take next. Cua supports models from cloud providers, open-source projects, and local inference.--## Using a Model--Specify the model when creating your agent:--```python-from cua import Sandbox, Image, ComputerAgent--async with Sandbox.ephemeral(Image.linux()) as computer:- agent = ComputerAgent(- model="anthropic/claude-sonnet-4-5-20250929",- tools=[computer]- )--async for result in agent.run("Open Firefox and search for 'Cua'"):- print(result)-```--The model string tells Cua which provider and model to use. Cua automatically selects the right agent loop implementation for each model.--## Cloud Providers--Cloud-hosted models offer the best accuracy and require no local GPU. You'll need an API key from the provider (or use Cua VLM Router for unified access).--### Anthropic Claude--Claude models are optimized for computer-use with native tool support.--```python-# Claude Sonnet 4.5 - best balance of speed and accuracy-agent = ComputerAgent(model="anthropic/claude-sonnet-4-5-20250929", tools=[computer])--# Claude Haiku 4.5 - faster, lower cost-agent = ComputerAgent(model="anthropic/claude-haiku-4-5-20251001", tools=[computer])-```--Set your API key:--```bash-export ANTHROPIC_API_KEY="your-key"-```--### OpenAI--OpenAI's computer-use preview model:--```python-agent = ComputerAgent(model="openai/computer-use-preview", tools=[computer])-```--```bash-export OPENAI_API_KEY="your-key"-```--### Google Gemini--Gemini 2.5 Computer-Use is optimized for **browser automation** rather than full desktop control. Use it with the [Browser Tool](/cua/guide/fundamentals/browser-tool) for web-based tasks:--```python-from cua.tools import BrowserTool--# Use with BrowserTool for web automation-agent = ComputerAgent(model="gemini-2.5-computer-use-preview-10-2025", tools=[browser])-```--```bash-export GOOGLE_API_KEY="your-key"-```--For full desktop computer-use, consider Anthropic Claude or OpenAI instead.--### Microsoft Fara--Fara is Microsoft's browser automation model, available via Azure ML. Like Gemini 2.5, it's designed for web tasks and works with the [Browser Tool](/cua/guide/fundamentals/browser-tool):--```python-from cua.tools import BrowserTool--agent = ComputerAgent(- model="azure_ml/Fara-7B",- tools=[browser],- api_base="https://your-azure-endpoint.inference.ml.azure.com"-)-```--## Local Models--Run models on your own hardware for privacy, cost savings, or air-gapped environments. Local models require a GPU with sufficient VRAM (typically 16GB+ for 7B models).--### Using HuggingFace Models--```python-# UI-TARS - ByteDance's full computer-use model-agent = ComputerAgent(- model="huggingface-local/ByteDance-Seed/UI-TARS-1.5-7B",- tools=[computer]-)--# Qwen 2.5 VL - Alibaba's full computer-use model-agent = ComputerAgent(- model="huggingface-local/Qwen/Qwen2.5-VL-7B-Instruct",- tools=[computer]-)-```--The first run downloads model weights from HuggingFace. Subsequent runs load from cache.--### Using MLX (Apple Silicon)--For M1/M2/M3/M4 Macs, MLX provides optimized inference:--```python-agent = ComputerAgent(- model="mlx/mlx-community/UI-TARS-1.5-7B-6bit",- tools=[computer]-)-```--MLX models run efficiently on Apple's unified memory architecture.--### Using Ollama--Run models locally via Ollama:--```python-agent = ComputerAgent(- model="ollama_chat/llama3.2:latest",- tools=[computer]-)-```--Ollama handles model downloading and serves inference locally.--## Model Capabilities--Models fall into three categories based on what they can control:--### Full Computer-Use--These models can control any desktop application—clicking, typing, scrolling across the entire screen. They work with the standard `Computer` tool:--- **Anthropic Claude** (all versions)-- **OpenAI computer-use-preview**-- **UI-TARS** (ByteDance)-- **Qwen 2.5 VL** (Alibaba)--```python-# Full desktop control-agent = ComputerAgent(model="anthropic/claude-sonnet-4-5-20250929", tools=[computer])-```--### Browser-Only--These models are optimized for web automation. They work with the `BrowserTool` and provide browser-specific actions like URL navigation and web search:--- **Gemini 2.5 Computer-Use** (Google)-- **Fara** (Microsoft)--```python-# Browser automation only-agent = ComputerAgent(model="gemini-2.5-computer-use-preview-10-2025", tools=[browser])-```--See [Browser Tool](/cua/guide/fundamentals/browser-tool) for details.--### Grounding-Only--These models can locate UI elements but can't reason about what action to take. They must be composed with a planning model:--- **GTA1** (grounding)-- **OmniParser** (Microsoft, UI element detection)-- **Moondream3** (lightweight grounding)--```python-# Must be composed with a planning model-"omniparser+openai/gpt-4o"-"huggingface-local/HelloKKMe/GTA1-7B+anthropic/claude-sonnet-4-5-20250929"-```--## Choosing a Model--**For full desktop automation:** Use Claude Sonnet 4.5 or OpenAI computer-use-preview. They handle complex UIs and multi-application workflows.--**For web automation:** Use Gemini 2.5 or Fara with the Browser Tool. They have browser-specific optimizations like direct URL navigation.--**For local development:** UI-TARS 1.5 7B or Qwen 2.5 VL run well on consumer GPUs without API keys.--**For cost optimization:** Compose a local grounding model with a cloud planner. The grounding model handles visual processing locally, reducing API calls.--**For air-gapped environments:** UI-TARS or Qwen run fully offline with no external API calls.diff --git a/docs/content/docs/cua/guide/get-started/meta.json b/docs/content/docs/cua/guide/get-started/meta.jsondeleted file mode 100644index a9fe5e0ca1..0000000000--- a/docs/content/docs/cua/guide/get-started/meta.json+++ /dev/null@@ -1,13 +0,0 @@-{- "title": "Get Started",- "description": "Get started with Cua",- "defaultOpen": true,- "icon": "Rocket",- "pages": [- "what-is-cua",- "quickstart",- "using-sandbox-sdk",- "using-agent-sdk",- "migrating-from-computer-sdk"- ]-}diff --git a/docs/content/docs/cua/guide/get-started/migrating-from-computer-sdk.mdx b/docs/content/docs/cua/guide/get-started/migrating-from-computer-sdk.mdxdeleted file mode 100644index 0c79ef23bb..0000000000--- a/docs/content/docs/cua/guide/get-started/migrating-from-computer-sdk.mdx+++ /dev/null@@ -1,136 +0,0 @@-----title: Migrating from the Computer SDK-description: Move from cua-computer to cua-sandbox — the new unified Cua SDK------import { Callout } from 'fumadocs-ui/components/callout';--<Callout type="warn" title="Computer SDK is deprecated">-`cua-computer` is deprecated and will no longer receive new features. All new development is happening in `cua-sandbox`. This guide shows you how to migrate.-</Callout>--## Why the change?--The old Computer SDK (`cua-computer`) required you to separately provision a sandbox (via the CLI or cloud dashboard) and then connect to it with a `Computer` object. This split made the API more confusing than necessary — two packages to install, two different concepts to understand.--The new Sandbox SDK (`cua-sandbox`) unifies everything: one package creates the sandbox, controls it, and cleans it up. The API surface is also cleaner — sub-objects (`sb.mouse`, `sb.keyboard`, `sb.shell`) instead of a flat `computer.interface.*` namespace.--## Package rename--```bash-# Before-pip install cua-computer--# After-pip install cua-```--## Import and instantiation--**Before:**--```python-from computer import Computer--computer = Computer(- os_type="linux",- provider_type="cloud",- name="my-sandbox-name"-)-await computer.run()-```--**After:**--```python-from cua import Sandbox, Image--# Ephemeral (create + destroy automatically)-async with Sandbox.ephemeral(Image.linux()) as sb:- ...--# Connect to an existing sandbox by name-async with Sandbox.connect("my-sandbox-name") as sb:- ...-```--## Method mapping--| Computer SDK | Sandbox SDK |-|---|---|-| `await computer.run()` | Handled by `Sandbox.ephemeral()` / `.create()` / `.connect()` |-| `await computer.disconnect()` | `await sb.disconnect()` |-| `await computer.stop()` | `await sb.delete()` |-| `await computer.interface.screenshot()` | `await sb.screenshot()` |-| `await computer.interface.left_click(x, y)` | `await sb.mouse.click(x, y)` |-| `await computer.interface.right_click(x, y)` | `await sb.mouse.right_click(x, y)` |-| `await computer.interface.double_click(x, y)` | `await sb.mouse.double_click(x, y)` |-| `await computer.interface.move(x, y)` | `await sb.mouse.move(x, y)` |-| `await computer.interface.scroll(x, y, clicks)` | `await sb.mouse.scroll(x, y, dy=clicks)` |-| `await computer.interface.type_text("text")` | `await sb.keyboard.type("text")` |-| `await computer.interface.key("Enter")` | `await sb.keyboard.press("Return")` |-| `await computer.interface.get_screen_size()` | `await sb.screen.size()` |--## Provider mapping--The old SDK used a `provider_type` string to select the runtime. The new SDK uses `Image` factory methods and the `local` flag.--| Computer SDK `provider_type` | Sandbox SDK |-|---|---|-| `"cloud"` | `Sandbox.ephemeral(Image.linux())` (default) |-| `"docker"` with Linux image | `Sandbox.ephemeral(Image.linux(), local=True)` |-| `"docker"` with QEMU Linux | `Sandbox.ephemeral(Image.linux(kind="vm"), local=True)` |-| `"docker"` with QEMU Windows | `Sandbox.ephemeral(Image.windows(), local=True)` |-| `"docker"` with QEMU Android | `Sandbox.ephemeral(Image.android(), local=True)` |-| `"lume"` | `Sandbox.ephemeral(Image.macos(), local=True)` |-| `"windows_sandbox"` | `Sandbox.ephemeral(Image.windows(), local=True)` |-| `use_host_computer_server=True` | `from cua import Localhost; Localhost.connect()` |--## Before and after — full example--**Before:**--```python-import asyncio-from computer import Computer--computer = Computer(- os_type="linux",- provider_type="docker",- image="trycua/cua-xfce:latest"-)--async def main():- await computer.run()- try:- screenshot = await computer.interface.screenshot()- await computer.interface.left_click(100, 100)- await computer.interface.type_text("Hello!")- finally:- await computer.disconnect()--asyncio.run(main())-```--**After:**--```python-import asyncio-from cua import Sandbox, Image--async def main():- async with Sandbox.ephemeral(Image.linux(), local=True) as sb:- screenshot = await sb.screenshot()- await sb.mouse.click(100, 100)- await sb.keyboard.type("Hello!")--asyncio.run(main())-```--## Agent SDK--The Agent SDK (`cua-agent`) currently uses `cua-computer` internally. Migration of the agent SDK to `cua-sandbox` is in progress. For now, you can use both packages side by side — `cua-sandbox` for sandbox control and `cua-agent` for AI agent orchestration.--## Getting help--If you run into anything not covered here, [open an issue](https://github.com/trycua/cua/issues) or check the [Sandbox SDK Reference](/cua/reference/sandbox-sdk).diff --git a/docs/content/docs/cua/guide/get-started/quickstart.mdx b/docs/content/docs/cua/guide/get-started/quickstart.mdxdeleted file mode 100644index 192d4d8a84..0000000000--- a/docs/content/docs/cua/guide/get-started/quickstart.mdx+++ /dev/null@@ -1,126 +0,0 @@-----title: Quickstart-description: Get a computer-use agent running in 5 minutes------import { Callout } from 'fumadocs-ui/components/callout';-import { Tab, Tabs } from 'fumadocs-ui/components/tabs';--This guide gets you from zero to a working computer-use agent.--<Callout type="info">- **No account? No problem.** The hello-world example below runs entirely with local Docker — no API key required. You only need a [Cua account](https://cua.ai/signin) when you want cloud sandboxes or to run the AI agent against a hosted model.-- **Prerequisites:** Python 3.12 or 3.13 and [Docker](https://docs.docker.com/get-docker/) (for local sandboxes).-</Callout>--## 1. Install--```bash-pip install cua-```--## 2. Hello World--Create `hello.py`:--```python-import asyncio-from cua import Sandbox, Image--async def main():- # local=True uses Docker — no API key needed- # remove local=True to run on Cua Cloud (requires CUA_API_KEY)- async with Sandbox.ephemeral(Image.linux(), local=True) as sb:- result = await sb.shell.run("uname -a")- print(result.stdout)-- screenshot = await sb.screenshot()- with open("screenshot.png", "wb") as f:- f.write(screenshot)- print("Screenshot saved to screenshot.png")--asyncio.run(main())-```--```bash-python hello.py-```--The sandbox is created, used, and destroyed automatically. No pre-provisioning needed.--## 3. Run an Agent--To add AI, you need a [Cua account](https://cua.ai/signin) and an API key:--```bash-pip install cua-cli-cua auth login # opens browser, saves key locally-```--Then create `agent.py`:--```python-import asyncio-from cua import Sandbox, Image, ComputerAgent--async def main():- async with Sandbox.ephemeral(Image.linux()) as sb:- agent = ComputerAgent(- model="cua/anthropic/claude-sonnet-4-5",- tools=[sb],- )- messages = [{"role": "user", "content": "Open Firefox and go to google.com"}]- async for result in agent.run(messages):- for item in result["output"]:- if item["type"] == "message":- print(item["content"][0]["text"])--asyncio.run(main())-```--```bash-python agent.py-```--## 4. Watch the Agent Live (Optional)--Call `get_display_url(share=True)` to get a shareable browser link to the live desktop — no CLI needed:--```python-import asyncio-from cua import Sandbox, Image, ComputerAgent--async def main():- async with Sandbox.ephemeral(Image.linux()) as sb:- url = await sb.get_display_url(share=True)- print(f"Watch live: {url}") # open this in your browser-- agent = ComputerAgent(- model="cua/anthropic/claude-sonnet-4-5",- tools=[sb],- )- messages = [{"role": "user", "content": "Open Firefox and go to google.com"}]- async for result in agent.run(messages):- for item in result["output"]:- if item["type"] == "message":- print(item["content"][0]["text"])--asyncio.run(main())-```--## 5. Clean Up--Ephemeral sandboxes delete themselves. For persistent sandboxes:--```bash-cua sb ls # show all running sandboxes-cua sb delete my-agent-sandbox # delete when done-```--## Next Steps--- [Sandbox Lifecycle](/cua/guide/sandbox/lifecycle) — ephemeral, persistent, and connect patterns-- [Images](/cua/guide/sandbox/images) — install packages and configure the environment-- [Parallel Sandboxes](/cua/guide/sandbox/scale-out) — run multiple agents concurrently-- [Using the Agent SDK](/cua/guide/get-started/using-agent-sdk) — advanced agent configurationdiff --git a/docs/content/docs/cua/guide/get-started/set-up-sandbox.mdx b/docs/content/docs/cua/guide/get-started/set-up-sandbox.mdxdeleted file mode 100644index 9d212df0f4..0000000000--- a/docs/content/docs/cua/guide/get-started/set-up-sandbox.mdx+++ /dev/null@@ -1,200 +0,0 @@-----title: Set Up a Sandbox-description: Create an isolated environment for your computer-use agents------import { Tab, Tabs } from 'fumadocs-ui/components/tabs';-import { Callout } from 'fumadocs-ui/components/callout';--Choose how you want to run your Cua sandbox. This will be the isolated environment where your automated tasks execute safely.--<Tabs items={['Cloud Sandbox', 'Linux on Docker', 'QEMU Docker', 'macOS Sandbox', 'Windows Sandbox']}>- <Tab value="Cloud Sandbox">-- Create and manage cloud sandboxes that run Linux (Ubuntu), Windows, or macOS.-- **First, create your API key:**-- 1. Go to [cua.ai/signin](https://cua.ai/signin)- 2. Navigate to **Dashboard > API Keys > New API Key** to create your API key- 3. **Important:** Copy and save your API key immediately - you won't be able to see it again (you'll need to regenerate if lost)-- **Then, create your sandbox using either option:**-- **Option 1: Via Website**-- 1. Navigate to **Dashboard > Sandboxes > Create Sandbox**- 2. Create a sandbox, choosing **Linux**, **Windows**, or **macOS**-- <img src="/docs/img/sandbox-create.png" alt="Create Sandbox Instance" className="w-full h-auto rounded-xl" />-- 3. Note your sandbox name-- **Option 2: Via CLI**-- 1. Install the Cua CLI:- ```bash- # macOS/Linux- curl -LsSf https://cua.ai/cli/install.sh | sh-- # Windows- powershell -ExecutionPolicy ByPass -c "irm https://cua.ai/cli/install.ps1 | iex"- ```-- 2. Login and create a sandbox:- ```bash- cua auth login- cua sb create --os linux --size small --region north-america- ```-- 3. Note your sandbox name and password from the output-- Your Cloud Sandbox will be automatically configured and ready to use.-- </Tab>- <Tab value="Linux on Docker">-- Run Linux desktop locally on macOS, Windows, or Linux hosts.-- 1. Install Docker Desktop or Docker Engine-- 2. Pull a Cua Docker image:-- ```bash- # XFCE (Lightweight) - recommended for most use cases- docker pull --platform=linux/amd64 trycua/cua-xfce:latest-- # OR KASM (Full-Featured) - full Ubuntu desktop- docker pull --platform=linux/amd64 trycua/cua-ubuntu:latest- ```-- </Tab>- <Tab value="QEMU Docker">-- Run full Linux (Ubuntu Desktop), Windows 11, or Android 11 VMs inside Docker containers using QEMU virtualization.-- <Callout type="warn">- Linux and Windows images require a **golden image preparation step** on first use. Android images start directly without preparation.- </Callout>-- <Tabs items={['Linux', 'Windows', 'Android']}>- <Tab value="Linux">-- **1. Install Docker Desktop or Docker Engine**-- **2. Pull the QEMU Linux image:**-- ```bash- docker pull trycua/cua-qemu-linux:latest- ```-- **3. Download Ubuntu 22.04 LTS Server ISO:**-- - Download the [Ubuntu 22.04 Server ISO](https://releases.ubuntu.com/22.04/ubuntu-22.04.5-live-server-amd64.iso) (~2GB)-- **4. Create golden image:**-- ```bash- docker run -it --rm \- --device=/dev/kvm \- --cap-add NET_ADMIN \- --mount type=bind,source=/path/to/ubuntu-22.04.5-live-server-amd64.iso,target=/custom.iso \- -v ~/cua-storage/linux:/storage \- -p 8006:8006 \- -p 5000:5000 \- -e RAM_SIZE=8G \- -e CPU_CORES=4 \- -e DISK_SIZE=64G \- trycua/cua-qemu-linux:latest- ```-- The container will install Ubuntu Desktop from the ISO and shut down when complete. Monitor progress at [http://localhost:8006](http://localhost:8006).-- </Tab>- <Tab value="Windows">-- **1. Install Docker Desktop or Docker Engine**-- **2. Pull the QEMU Windows image:**-- ```bash- docker pull trycua/cua-qemu-windows:latest- ```-- **3. Download Windows 11 Enterprise Evaluation ISO:**-- - Visit [Microsoft Evaluation Center](https://info.microsoft.com/ww-landing-windows-11-enterprise.html)- - Accept the Terms of Service- - Download **Windows 11 Enterprise Evaluation (90-day trial, English, United States)** ISO (~6GB)-- **4. Create golden image:**-- ```bash- docker run -it --rm \- --device=/dev/kvm \- --cap-add NET_ADMIN \- --mount type=bind,source=/path/to/windows-11-enterprise-eval.iso,target=/custom.iso \- -v ~/cua-storage/windows:/storage \- -p 8006:8006 \- -p 5000:5000 \- -e RAM_SIZE=8G \- -e CPU_CORES=4 \- -e DISK_SIZE=64G \- trycua/cua-qemu-windows:latest- ```-- The container will install Windows 11 from the ISO and shut down when complete. Monitor progress at [http://localhost:8006](http://localhost:8006).-- </Tab>- <Tab value="Android">-- **1. Install Docker Desktop or Docker Engine**-- **2. Pull the QEMU Android image:**-- ```bash- docker pull trycua/cua-qemu-android:latest- ```-- No golden image preparation needed - the Android emulator starts directly when you run it!-- </Tab>- </Tabs>-- </Tab>- <Tab value="macOS Sandbox">-- macOS hosts only - requires Lume CLI.-- 1. Install the Lume CLI:-- ```bash- /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/lume/scripts/install.sh)"- ```-- 2. Start a local Cua sandbox:-- ```bash- lume run macos-sequoia-cua:latest- ```-- </Tab>- <Tab value="Windows Sandbox">-- Windows hosts only - requires Windows 10 Pro/Enterprise or Windows 11.-- 1. Enable [Windows Sandbox](https://learn.microsoft.com/en-us/windows/security/application-security/application-isolation/windows-sandbox/windows-sandbox-install)- 2. Install the `pywinsandbox` dependency:-- ```bash- pip install -U git+git://github.com/karkason/pywinsandbox.git- ```-- 3. Windows Sandbox will be automatically configured when you run the CLI-- </Tab>-</Tabs>--## Next Step--Once your sandbox is set up, proceed to [Using the Computer SDK](/cua/guide/get-started/using-computer-sdk) to connect and start interacting with it.diff --git a/docs/content/docs/cua/guide/get-started/using-agent-sdk.mdx b/docs/content/docs/cua/guide/get-started/using-agent-sdk.mdxdeleted file mode 100644index ffa619bec8..0000000000--- a/docs/content/docs/cua/guide/get-started/using-agent-sdk.mdx+++ /dev/null@@ -1,257 +0,0 @@-----title: Using the Agent SDK-description: Add AI automation with vision-language models------import { Tab, Tabs } from 'fumadocs-ui/components/tabs';-import { Callout } from 'fumadocs-ui/components/callout';--Now that you've verified your sandbox works with the Sandbox SDK, use an Agent to automate complex tasks. The agent interacts with the computer environment using a vision-language model to understand the UI and execute actions.--Cua Agent Framework provides:--- **100+ VLM options** through Cua VLM Router and direct provider access-- **Built-in optimizations** for computer-use tasks-- **Structured agent loops** for consistent behavior--## Installation--<Tabs items={['Python', 'TypeScript']}>- <Tab value="Python">- **Using uv (recommended):**- ```bash- uv pip install cua- ```-- **Or with pip:**- ```bash- pip install cua- ```-- </Tab>- <Tab value="TypeScript">- ```bash- npm install @trycua/computer ai- ```- </Tab>-</Tabs>--## Choose Your Model Provider--<Tabs items={['Python', 'TypeScript']}>- <Tab value="Python">--<Tabs items={['Cua VLM Router', 'BYOK (Bring Your Own Key)']}>- <Tab value="Cua VLM Router">--Use Cua's inference API to access multiple model providers with a single API key (same key used for sandbox access). Cua VLM Router provides intelligent routing and cost optimization.--```python-import os-import asyncio-from cua import Sandbox, Image, ComputerAgent--os.environ["CUA_API_KEY"] = "sk_cua-api01_..."--async def main():- async with Sandbox.ephemeral(Image.linux()) as sb:- agent = ComputerAgent(- model="cua/anthropic/claude-sonnet-4.5", # Cua-routed model- tools=[sb],- max_trajectory_budget=5.0- )-- messages = [{"role": "user", "content": "Take a screenshot and tell me what you see"}]-- async for result in agent.run(messages):- for item in result["output"]:- if item["type"] == "message":- print(item["content"][0]["text"])--asyncio.run(main())-```--**Available Cua models:**--- `cua/anthropic/claude-sonnet-4.5` - Claude Sonnet 4.5 (recommended)-- `cua/anthropic/claude-opus-4.5` - Claude Opus 4.5 (enhanced agentic capabilities)-- `cua/anthropic/claude-haiku-4.5` - Claude Haiku 4.5 (faster, cost-effective)-- `cua/google/gemini-3-pro-preview` - Gemini 3 Pro Preview (most powerful multimodal)-- `cua/google/gemini-3-flash-preview` - Gemini 3 Flash Preview (fastest and cheapest)--**Available composed models:**--- `huggingface-local/HelloKKMe/GTA1-7B+anthropic/claude-sonnet-4-5-20250929` - GTA1 grounding + Claude Sonnet 4.5 planning-- `huggingface-local/HelloKKMe/GTA1-7B+openai/gpt-5` - GTA1 grounding + GPT-5 planning-- `huggingface-local/ByteDance-Seed/UI-TARS-1.5-7B+openai/gpt-4o` - UI-TARS grounding + GPT-4o planning-- `moondream3+openai/gpt-4o` - Moondream3 grounding + GPT-4o planning--**Benefits:**--- Single API key for multiple providers-- Cost tracking and optimization-- No need to manage multiple provider keys--</Tab>-<Tab value="BYOK (Bring Your Own Key)">--Use your own API keys from model providers like Anthropic, OpenAI, or others.--```python-import asyncio-from cua import Sandbox, Image, ComputerAgent--# Set your provider API key-import os-os.environ["ANTHROPIC_API_KEY"] = "sk-ant-..." # For Anthropic-# OR-os.environ["OPENAI_API_KEY"] = "sk-..." # For OpenAI--async def main():- async with Sandbox.ephemeral(Image.linux()) as sb:- agent = ComputerAgent(- model="anthropic/claude-sonnet-4-5-20250929", # Direct provider model- tools=[sb],- max_trajectory_budget=5.0- )-- messages = [{"role": "user", "content": "Take a screenshot and tell me what you see"}]-- async for result in agent.run(messages):- for item in result["output"]:- if item["type"] == "message":- print(item["content"][0]["text"])--asyncio.run(main())-```--**Supported providers:**--- `anthropic/claude-*` - Anthropic Claude models-- `openai/gpt-*` - OpenAI GPT models-- `openai/o1-*` - OpenAI o1 models-- `huggingface-local/*` - Local HuggingFace models-- And many more via LiteLLM--See [Supported Models](/cua/guide/fundamentals/vlms) for the complete list.--</Tab>-</Tabs>--</Tab>-<Tab value="TypeScript">--For TypeScript, you can build agent loops using the [Vercel AI SDK](https://ai-sdk.dev/) with the Cua Computer Framework TypeScript library.--```typescript-import Anthropic from '@anthropic-ai/sdk';-import { Computer, OSType } from '@trycua/computer';--const client = new Anthropic();-let computer: Computer;--const computerTool = {- type: 'tool' as const,- name: 'computer',- description: 'Control the computer with actions like screenshot, click, type, etc.',- inputSchema: {- type: 'object' as const,- properties: {- action: {- type: 'string' as const,- description: 'Action to perform (screenshot, click, type, key_press, etc.)',- },- coordinate: {- type: 'array' as const,- items: { type: 'number' as const },- description: 'x, y coordinates for click actions',- },- text: {- type: 'string' as const,- description: 'Text to type',- },- },- required: ['action'],- },-};--async function runAgentLoop(goal: string) {- // Initialize computer- computer = new Computer({- osType: OSType.LINUX,- provider_type: 'cloud',- name: 'your-sandbox-name',- apiKey: process.env.CUA_API_KEY!,- });-- await computer.run();-- const messages: any[] = [];- messages.push({ role: 'user', content: goal });-- // Agent loop- for (let i = 0; i < 10; i++) {- const response = await client.messages.create({- model: 'claude-opus-4-1-20250805',- max_tokens: 4096,- tools: [computerTool],- messages: messages,- });-- if (response.stop_reason === 'end_turn') {- console.log('Task completed!');- break;- }-- messages.push({ role: 'assistant', content: response.content });-- // Process tool calls- const toolResults = [];- for (const block of response.content) {- if (block.type === 'tool_use') {- let result: any;- switch (block.input.action) {- case 'screenshot':- result = await computer.interface.screenshot();- break;- case 'click':- result = await computer.interface.click(- block.input.coordinate[0],- block.input.coordinate[1]- );- break;- case 'type':- result = await computer.interface.type(block.input.text);- break;- }- toolResults.push({- type: 'tool_result',- tool_use_id: block.id,- content: JSON.stringify(result),- });- }- }-- if (toolResults.length > 0) {- messages.push({ role: 'user', content: toolResults });- }- }-- await computer.disconnect();-}--runAgentLoop('Take a screenshot and tell me what you see');-```--For more details, see the [Vercel AI SDK Computer Use Cookbook](https://ai-sdk.dev/cookbook/guides/computer-use).--</Tab>-</Tabs>--## Next Steps--- Explore [Agent Loops](/cua/guide/fundamentals/agent-loops) for advanced agent configuration-- Check out [Custom Tools](/cua/guide/advanced/custom-tools) to extend your agents-- Review [Supported Model Providers](/cua/guide/fundamentals/vlms) for more LLM options-- Try the [Form Filling](/cua/examples/automation/form-filling) example use case-- Join our [Discord community](https://discord.com/invite/cua-ai) for help and discussiondiff --git a/docs/content/docs/cua/guide/get-started/using-cloud-cli.mdx b/docs/content/docs/cua/guide/get-started/using-cloud-cli.mdxdeleted file mode 100644index aaad8ae1f9..0000000000--- a/docs/content/docs/cua/guide/get-started/using-cloud-cli.mdx+++ /dev/null@@ -1,193 +0,0 @@-----title: Using the Cloud CLI-description: Manage cloud sandboxes from your terminal------import { Tab, Tabs } from 'fumadocs-ui/components/tabs';-import { Callout } from 'fumadocs-ui/components/callout';--The Cua Cloud CLI lets you create, manage, and connect to cloud sandboxes directly from your terminal. This is the fastest way to get a sandbox running without writing any code.--## Installation--<Tabs items={['macOS/Linux', 'Windows', 'bun']}>- <Tab value="macOS/Linux">```bash curl -LsSf https://cua.ai/cli/install.sh | sh ```</Tab>- <Tab value="Windows">- ```powershell powershell -ExecutionPolicy ByPass -c "irm https://cua.ai/cli/install.ps1 | iex"- ```- </Tab>- <Tab value="bun">```bash bun install -g @trycua/cli ```</Tab>-</Tabs>--Verify the installation:--```bash-cua --version-```--## Authentication--Before managing sandboxes, authenticate with your Cua account:--```bash-# Browser-based login (opens OAuth flow)-cua auth login--# Or use an API key directly-cua auth login --api-key sk_cua-api01_...-```--To write your API key to a `.env` file for use in projects:--```bash-cua auth env-```--This creates or updates `.env` with your `CUA_API_KEY`.--## Creating a Sandbox--Create a new cloud sandbox with the `create` command:--```bash-cua sandbox create --os linux --size small --region north-america-```--**Required options:**--| Option | Values | Description |-| ---------- | ---------------------------------------------------------- | ----------------- |-| `--os` | `linux`, `windows`, `macos` | Operating system |-| `--size` | `small`, `medium`, `large` | Compute tier |-| `--region` | `north-america`, `europe`, `asia-pacific`, `south-america` | Geographic region |--**Compute tiers:**--| Size | vCPU | RAM | Best for |-| -------- | ---- | ----- | -------------------------------- |-| `small` | 1 | 4 GB | Simple tasks, testing |-| `medium` | 2 | 8 GB | General automation |-| `large` | 8 | 32 GB | Complex workflows, multiple apps |--The command returns once your sandbox is ready. You'll see the sandbox name, which you'll use for all other commands.--## Listing Sandboxes--View all your sandboxes:--```bash-cua sandbox list-```--Output shows NAME, STATUS, and HOST for each sandbox. Status values:--- `running` - Ready to use-- `stopped` - Not running (no charges)-- `pending` - Starting up-- `suspended` - Memory state preserved-- `terminated` - Deleted--Shorthand aliases: `cua ls`, `cua ps`, `cua sb list`--## Managing Sandbox Lifecycle--Start, stop, and restart sandboxes:--```bash-# Start a stopped sandbox (hot-start in under 1 second)-cua sandbox start my-sandbox--# Stop a running sandbox (preserves data)-cua sandbox stop my-sandbox--# Restart a sandbox-cua sandbox restart my-sandbox--# Suspend (preserves memory state)-cua sandbox suspend my-sandbox--# Permanently delete-cua sandbox delete my-sandbox-```--Stopping a sandbox preserves all files, installed software, and browser profiles. You only pay for compute time while the sandbox is running.--## Connecting to a Sandbox--Open the sandbox desktop in your browser:--```bash-cua sandbox vnc my-sandbox-```--This opens a NoVNC session with the password pre-filled. You can interact with the desktop directly in your browser.--Shorthand: `cua open my-sandbox`--## Getting Sandbox Details--View detailed information about a sandbox:--```bash-cua sandbox get my-sandbox-```--Add flags for more details:--```bash-# Include password in output-cua sandbox get my-sandbox --show-passwords--# Include VNC URL-cua sandbox get my-sandbox --show-vnc-url--# JSON output for scripting-cua sandbox get my-sandbox --json-```--## Command Reference--The CLI supports both grouped and flat command styles:--| Grouped Style | Flat Style | Description |-| ---------------------------- | -------------------- | --------------------- |-| `cua auth login` | `cua login` | Authenticate |-| `cua auth logout` | `cua logout` | Clear credentials |-| `cua auth env` | `cua env` | Write API key to .env |-| `cua sandbox list` | `cua list` | List sandboxes |-| `cua sandbox create` | `cua create` | Create sandbox |-| `cua sandbox get <name>` | `cua get <name>` | Get sandbox details |-| `cua sandbox start <name>` | `cua start <name>` | Start sandbox |-| `cua sandbox stop <name>` | `cua stop <name>` | Stop sandbox |-| `cua sandbox restart <name>` | `cua restart <name>` | Restart sandbox |-| `cua sandbox delete <name>` | `cua delete <name>` | Delete sandbox |-| `cua sandbox vnc <name>` | `cua vnc <name>` | Open VNC in browser |--## Example Workflow--Here's a typical workflow for setting up and using a sandbox:--```bash-# 1. Login-cua auth login--# 2. Create a Linux sandbox-cua sandbox create --os linux --size small --region north-america-# Output: Created sandbox "curious-fox-123"--# 3. Open the desktop in your browser-cua open curious-fox-123--# 4. Export your API key for SDK usage-cua auth env--# 5. Use the sandbox with Python SDK-python my_agent.py--# 6. Stop when done (preserves data, stops charges)-cua stop curious-fox-123-```--## Next Step--Once your sandbox is running, proceed to [Using the Computer SDK](/cua/guide/get-started/using-computer-sdk) to connect programmatically, or [Using the Agent SDK](/cua/guide/get-started/using-agent-sdk) to add AI automation.diff --git a/docs/content/docs/cua/guide/get-started/using-computer-sdk.mdx b/docs/content/docs/cua/guide/get-started/using-computer-sdk.mdxdeleted file mode 100644index e1f8429bbf..0000000000--- a/docs/content/docs/cua/guide/get-started/using-computer-sdk.mdx+++ /dev/null@@ -1,465 +0,0 @@-----title: Using the Computer SDK-description: Connect to your sandbox and perform basic interactions------import { Tab, Tabs } from 'fumadocs-ui/components/tabs';-import { Callout } from 'fumadocs-ui/components/callout';--<Callout type="error" title="Deprecated">- The Computer SDK (`cua-computer`) is deprecated. New projects should use [`cua-sandbox`](/cua/guide/get-started/using-sandbox-sdk) instead. See the [migration guide](/cua/guide/get-started/migrating-from-computer-sdk) to update existing code.-</Callout>--<Callout type="warn" title="Python Version Compatibility">- Cua packages require **Python 3.12 or 3.13**. Python 3.14 is not currently supported due to- dependency compatibility issues (pydantic-core/PyO3 compatibility). If you encounter build errors- on Python 3.14, please use Python 3.13 or earlier.-</Callout>--The Computer SDK lets you connect to your sandbox and perform basic interactions like taking screenshots, clicking, and typing. This is an important verification step before adding AI agents.--## Installation--<Tabs items={['Python', 'TypeScript']}>- <Tab value="Python">- **Using uv (recommended):**- ```bash- uv pip install cua-computer- ```-- **Or with pip:**- ```bash- pip install cua-computer- ```-- </Tab>- <Tab value="TypeScript">- ```bash- npm install @trycua/computer- ```- </Tab>-</Tabs>--## Connect to Your Sandbox--<Tabs items={['Python', 'TypeScript']}>- <Tab value="Python">-- <Tabs items={['Cloud Sandbox', 'Linux on Docker', 'QEMU Docker', 'macOS Sandbox', 'Windows Sandbox', 'Your host desktop']}>- <Tab value="Cloud Sandbox">- Set your Cua API key (same key used for model inference) and connect to your sandbox:- ```python- import os- from computer import Computer- import asyncio-- os.environ["CUA_API_KEY"] = "sk_cua-api01_..."-- computer = Computer(- os_type="linux", # or "windows" or "macos"- provider_type="cloud",- name="your-sandbox-name" # from CLI or website- )-- async def main():- await computer.run() # Connect to the sandbox-- try:- # Take a screenshot of the computer's current display- screenshot = await computer.interface.screenshot()- # Simulate a left-click at coordinates (100, 100)- await computer.interface.left_click(100, 100)- # Type "Hello!" into the active application- await computer.interface.type_text("Hello!")- finally:- await computer.disconnect()-- asyncio.run(main())- ```- </Tab>- <Tab value="Linux on Docker">- ```python- from computer import Computer- import asyncio-- computer = Computer(- os_type="linux",- provider_type="docker",- image="trycua/cua-xfce:latest" # or "trycua/cua-ubuntu:latest"- )-- async def main():- await computer.run() # Launch & connect to the sandbox-- try:- screenshot = await computer.interface.screenshot()- await computer.interface.left_click(100, 100)- await computer.interface.type_text("Hello!")- finally:- await computer.disconnect()-- asyncio.run(main())- ```- </Tab>- <Tab value="QEMU Docker">- <Tabs items={['Linux', 'Windows', 'Android']}>- <Tab value="Linux">- ```python- from computer import Computer- import asyncio-- computer = Computer(- os_type="linux",- provider_type="docker",- image="trycua/cua-qemu-linux:latest",- storage="~/cua-storage/linux",- run_opts={- "devices": ["/dev/kvm"], # Optional but recommended- },- )-- async def main():- await computer.run() # Boot from golden image-- try:- screenshot = await computer.interface.screenshot()- await computer.interface.left_click(100, 100)- await computer.interface.type_text("Hello!")- finally:- await computer.disconnect()-- asyncio.run(main())- ```- </Tab>- <Tab value="Windows">- ```python- from computer import Computer- import asyncio-- computer = Computer(- os_type="windows",- provider_type="docker",- image="trycua/cua-qemu-windows:latest",- storage="~/cua-storage/windows",- run_opts={- "devices": ["/dev/kvm"], # Optional but recommended- },- )-- async def main():- await computer.run() # Boot from golden image-- try:- screenshot = await computer.interface.screenshot()- await computer.interface.left_click(100, 100)- await computer.interface.type_text("Hello!")- finally:- await computer.disconnect()-- asyncio.run(main())- ```- </Tab>- <Tab value="Android">- ```python- from computer import Computer- import asyncio-- computer = Computer(- os_type="android",- provider_type="docker",- image="trycua/cua-qemu-android:latest",- timeout=150, # Emulator needs more time to boot- run_opts={- "devices": ["/dev/kvm"], # Required for Android emulator- "env": {- "EMULATOR_DEVICE": "Samsung Galaxy S10",- },- },- )-- async def main():- await computer.run() # Launch & connect to Android emulator-- try:- screenshot = await computer.interface.screenshot()- await computer.interface.left_click(100, 100)- await computer.interface.type_text("Hello!")- finally:- await computer.disconnect()-- asyncio.run(main())- ```- </Tab>- </Tabs>- </Tab>- <Tab value="macOS Sandbox">- ```python- from computer import Computer- import asyncio-- computer = Computer(- os_type="macos",- provider_type="lume",- name="macos-sequoia-cua:latest"- )-- async def main():- await computer.run() # Launch & connect to the sandbox-- try:- screenshot = await computer.interface.screenshot()- await computer.interface.left_click(100, 100)- await computer.interface.type_text("Hello!")- finally:- await computer.disconnect()-- asyncio.run(main())- ```- </Tab>- <Tab value="Windows Sandbox">- ```python- from computer import Computer- import asyncio-- computer = Computer(- os_type="windows",- provider_type="windows_sandbox"- )-- async def main():- await computer.run() # Launch & connect to the sandbox-- try:- screenshot = await computer.interface.screenshot()- await computer.interface.left_click(100, 100)- await computer.interface.type_text("Hello!")- finally:- await computer.disconnect()-- asyncio.run(main())- ```- </Tab>- <Tab value="Your host desktop">- Install and run `cua-computer-server`:- ```bash- pip install cua-computer-server- python -m computer_server- ```-- Then, use the `Computer` object to connect:- ```python- from computer import Computer- import asyncio-- computer = Computer(use_host_computer_server=True)-- async def main():- await computer.run() # Connect to the host desktop-- try:- screenshot = await computer.interface.screenshot()- await computer.interface.left_click(100, 100)- await computer.interface.type_text("Hello!")- finally:- await computer.disconnect()-- asyncio.run(main())- ```- </Tab>- </Tabs>-- </Tab>- <Tab value="TypeScript">-- <Tabs items={['Cloud Sandbox', 'Linux on Docker', 'macOS Sandbox', 'Windows Sandbox', 'Your host desktop']}>- <Tab value="Cloud Sandbox">- Set your Cua API key (same key used for model inference):- ```bash- export CUA_API_KEY="sk_cua-api01_..."- ```-- Then connect to your sandbox:- ```typescript- import { Computer, OSType } from '@trycua/computer';-- const computer = new Computer({- osType: OSType.LINUX, // or OSType.WINDOWS or OSType.MACOS- name: "your-sandbox-name" // from CLI or website- });- await computer.run(); // Connect to the sandbox- ```- </Tab>- <Tab value="Linux on Docker">- ```typescript- import { Computer, OSType, ProviderType } from '@trycua/computer';-- const computer = new Computer({- osType: OSType.LINUX,- providerType: ProviderType.DOCKER,- image: "trycua/cua-xfce:latest" // or "trycua/cua-ubuntu:latest"- });- await computer.run(); // Launch & connect to the sandbox- ```- </Tab>- <Tab value="macOS Sandbox">- ```typescript- import { Computer, OSType, ProviderType } from '@trycua/computer';-- const computer = new Computer({- osType: OSType.MACOS,- providerType: ProviderType.LUME,- name: "macos-sequoia-cua:latest"- });- await computer.run(); // Launch & connect to the sandbox- ```- </Tab>- <Tab value="Windows Sandbox">- ```typescript- import { Computer, OSType, ProviderType } from '@trycua/computer';-- const computer = new Computer({- osType: OSType.WINDOWS,- providerType: ProviderType.WINDOWS_SANDBOX- });- await computer.run(); // Launch & connect to the sandbox- ```- </Tab>- <Tab value="Your host desktop">- First, install and run `cua-computer-server`:- ```bash- pip install cua-computer-server- python -m computer_server- ```-- Then, use the `Computer` object to connect:- ```typescript- import { Computer } from '@trycua/computer';-- const computer = new Computer({ useHostComputerServer: true });- await computer.run(); // Connect to the host desktop- ```- </Tab>- </Tabs>-- Once connected, you can perform interactions:- ```typescript- try {- // Take a screenshot of the computer's current display- const screenshot = await computer.interface.screenshot();- // Simulate a left-click at coordinates (100, 100)- await computer.interface.leftClick(100, 100);- // Type "Hello!" into the active application- await computer.interface.typeText("Hello!");- } finally {- await computer.disconnect();- }- ```-- </Tab>-</Tabs>--## Common Operations--| Operation | Python | TypeScript |-| ------------ | ----------------------------------------------- | ----------------------------------------------- |-| Screenshot | `await computer.interface.screenshot()` | `await computer.interface.screenshot()` |-| Left click | `await computer.interface.left_click(x, y)` | `await computer.interface.leftClick(x, y)` |-| Right click | `await computer.interface.right_click(x, y)` | `await computer.interface.rightClick(x, y)` |-| Double click | `await computer.interface.double_click(x, y)` | `await computer.interface.doubleClick(x, y)` |-| Type text | `await computer.interface.type_text("text")` | `await computer.interface.typeText("text")` |-| Press key | `await computer.interface.key("Enter")` | `await computer.interface.key("Enter")` |-| Scroll | `await computer.interface.scroll(x, y, clicks)` | `await computer.interface.scroll(x, y, clicks)` |--## Lifecycle Methods--- `computer.run()` - Connect to an already-running sandbox-- `computer.start()` - Start and connect to the sandbox (if not running)-- `computer.disconnect()` - Disconnect but keep sandbox running-- `computer.stop()` - Fully stop the sandbox and disconnect--## Managing Cloud Sandboxes--For cloud sandboxes, you can programmatically manage VM lifecycle using the `CloudProvider`:--<Tabs items={['Python', 'TypeScript']}>-<Tab value="Python">--```python-import os-import asyncio-from computer.providers.cloud.provider import CloudProvider--os.environ["CUA_API_KEY"] = "sk_cua-api01_..."--async def main():- provider = CloudProvider()-- # List all your sandboxes- sandboxes = await provider.list_vms()- for vm in sandboxes:- print(f"{vm['name']}: {vm['status']}")-- # Start a sandbox- await provider.run_vm("my-sandbox")-- # Check sandbox status- info = await provider.get_vm("my-sandbox")- print(f"Status: {info['status']}")-- # Stop when done- await provider.stop_vm("my-sandbox")--asyncio.run(main())-```--</Tab>-<Tab value="TypeScript">--```typescript-import { CloudProvider } from '@trycua/computer';--const provider = new CloudProvider({- apiKey: process.env.CUA_API_KEY!,-});--async function main() {- // List all your sandboxes- const sandboxes = await provider.listVMs();- for (const vm of sandboxes) {- console.log(`${vm.name}: ${vm.status}`);- }-- // Start a sandbox- await provider.runVM('my-sandbox');-- // Check sandbox status- const info = await provider.getVM('my-sandbox');- console.log(`Status: ${info.status}`);-- // Stop when done- await provider.stopVM('my-sandbox');-}--main();-```--</Tab>-</Tabs>--Available operations:--| Operation | Description |-| ------------------ | --------------------------------------------- |-| `list_vms()` | List all sandboxes in your account |-| `get_vm(name)` | Get status and details for a specific sandbox |-| `run_vm(name)` | Start a sandbox (hot-start in under 1 second) |-| `stop_vm(name)` | Stop a running sandbox |-| `restart_vm(name)` | Restart a sandbox |--This is useful for orchestrating multiple sandboxes, building CI/CD pipelines, or programmatically managing sandbox lifecycle.--## Next Step--Once you've verified your sandbox connection works, proceed to [Using the Agent SDK](/cua/guide/get-started/using-agent-sdk) to add AI automation.--For more details on available commands, see the [Computer SDK Reference](/cua/reference/computer-sdk) reference.diff --git a/docs/content/docs/cua/guide/get-started/using-sandbox-sdk.mdx b/docs/content/docs/cua/guide/get-started/using-sandbox-sdk.mdxdeleted file mode 100644index 562589d05d..0000000000--- a/docs/content/docs/cua/guide/get-started/using-sandbox-sdk.mdx+++ /dev/null@@ -1,255 +0,0 @@-----title: Using the Sandbox SDK-description: Create sandboxes and perform basic interactions with cua-sandbox------import { Tab, Tabs } from 'fumadocs-ui/components/tabs';-import { Callout } from 'fumadocs-ui/components/callout';--<Callout type="info">- **Prerequisites:** Python 3.12 or 3.13 and a free Cua account at [cua.ai](https://cua.ai/signin) for cloud sandboxes.-</Callout>--The Sandbox SDK (`cua-sandbox`) lets you create and control sandboxes — take screenshots, run shell commands, click, type, and more. Use it to verify your sandbox works before adding AI agents.--## Installation--```bash-pip install cua-```--Or with uv:--```bash-uv pip install cua-```--## Create a Sandbox--<Tabs items={['Cloud Linux Container', 'Cloud Linux VM', 'Local Linux', 'Local macOS (Lume)', 'Local Windows', 'Local Android']}>- <Tab value="Cloud Linux Container">- ```python- import asyncio- from cua import Sandbox, Image-- async def main():- async with Sandbox.ephemeral(Image.linux()) as sb:- result = await sb.shell.run("uname -s")- print(result.stdout) # "Linux"-- screenshot = await sb.screenshot()- with open("screenshot.png", "wb") as f:- f.write(screenshot)-- asyncio.run(main())- ```- Requires `CUA_API_KEY` environment variable.- </Tab>- <Tab value="Cloud Linux VM">- ```python- import asyncio- from cua import Sandbox, Image-- async def main():- async with Sandbox.ephemeral(Image.linux(kind="vm")) as sb:- result = await sb.shell.run("uname -s")- print(result.stdout) # "Linux"-- screenshot = await sb.screenshot()- with open("screenshot.png", "wb") as f:- f.write(screenshot)-- asyncio.run(main())- ```- Requires `CUA_API_KEY` environment variable.- </Tab>- <Tab value="Cloud macOS">- ```python- import asyncio- from cua import Sandbox, Image-- async def main():- async with Sandbox.ephemeral(Image.macos()) as sb:- result = await sb.shell.run("sw_vers")- print(result.stdout)-- screenshot = await sb.screenshot()- with open("screenshot.png", "wb") as f:- f.write(screenshot)-- asyncio.run(main())- ```- Requires `CUA_API_KEY` environment variable. Works on any host OS — no Mac needed.- </Tab>- <Tab value="Cloud Windows">- ```python- import asyncio- from cua import Sandbox, Image-- async def main():- async with Sandbox.ephemeral(Image.windows()) as sb:- result = await sb.shell.run("ver")- print(result.stdout)-- screenshot = await sb.screenshot()- with open("screenshot.png", "wb") as f:- f.write(screenshot)-- asyncio.run(main())- ```- Requires `CUA_API_KEY` environment variable.- </Tab>- <Tab value="Local Linux">- ```python- import asyncio- from cua import Sandbox, Image-- async def main():- async with Sandbox.ephemeral(Image.linux(), local=True) as sb:- result = await sb.shell.run("uname -s")- print(result.stdout) # "Linux"-- screenshot = await sb.screenshot()- with open("screenshot.png", "wb") as f:- f.write(screenshot)-- asyncio.run(main())- ```- Requires [Docker](https://docs.docker.com/get-docker/). No API key needed.- </Tab>- <Tab value="Local macOS (Lume)">- ```python- import asyncio- from cua import Sandbox, Image-- async def main():- async with Sandbox.ephemeral(Image.macos(), local=True) as sb:- result = await sb.shell.run("sw_vers")- print(result.stdout)-- screenshot = await sb.screenshot()- with open("screenshot.png", "wb") as f:- f.write(screenshot)-- asyncio.run(main())- ```- Requires macOS host with [Lume](/lume/guide/getting-started/introduction) installed.- </Tab>- <Tab value="Local Windows">- ```python- import asyncio- from cua import Sandbox, Image-- async def main():- async with Sandbox.ephemeral(Image.windows(), local=True) as sb:- result = await sb.shell.run("ver")- print(result.stdout)-- screenshot = await sb.screenshot()- with open("screenshot.png", "wb") as f:- f.write(screenshot)-- asyncio.run(main())- ```- Requires a Windows or Linux host with QEMU installed.- </Tab>- <Tab value="Local Android">- ```python- import asyncio- from cua import Sandbox, Image-- async def main():- async with Sandbox.ephemeral(Image.android(), local=True) as sb:- result = await sb.shell.run("getprop ro.build.version.release")- print(result.stdout) # "14"-- screenshot = await sb.screenshot()- with open("screenshot.png", "wb") as f:- f.write(screenshot)-- asyncio.run(main())- ```- Requires Android SDK (emulator + platform-tools) or QEMU. No API key needed.- </Tab>-</Tabs>--## Common Operations--### Shell Commands--```python-result = await sb.shell.run("echo hello")-print(result.stdout) # "hello\n"-print(result.returncode) # 0-print(result.success) # True-```--### Screenshots--```python-png_bytes = await sb.screenshot()-with open("screenshot.png", "wb") as f:- f.write(png_bytes)-```--### Mouse & Keyboard--```python-# Click-await sb.mouse.click(100, 200) # left click-await sb.mouse.right_click(100, 200) # right click-await sb.mouse.double_click(100, 200) # double click-await sb.mouse.move(100, 200) # move without clicking-await sb.mouse.scroll(100, 200, dy=3) # scroll--# Keyboard-await sb.keyboard.type("Hello, world!")-await sb.keyboard.press("Return")-await sb.keyboard.press("ctrl+c")-```--## Persistent Sandboxes--For long-running workflows, create a named sandbox instead of ephemeral:--```python-# Launch via CLI: cua sb launch ubuntu:24.04 --local --json-# → {"name": "my-sandbox-name", "status": "ready"}--# Connect from Python-from cua import Sandbox--async with Sandbox.connect("my-sandbox-name", local=True) as sb:- await sb.shell.run("echo connected")-```--Or manage the full lifecycle in Python:--```python-from cua import Sandbox, Image--# Create and keep running-sb = await Sandbox.create(Image.linux(), local=True)-await sb.shell.run("echo running")-# ... do more work ...-await sb.delete()-```--## Image Types--| Image | Description |-|-------|-------------|-| `Image.linux()` | Ubuntu 24.04 container (default) |-| `Image.linux(kind="vm")` | Ubuntu 24.04 full VM |-| `Image.macos()` | macOS (latest) |-| `Image.windows()` | Windows 11 |-| `Image.from_registry("ghcr.io/...")` | Custom OCI image |--## Next Steps--- [Sandbox Lifecycle](/cua/guide/sandbox/lifecycle) — understand ephemeral, persistent, and connected sandboxes-- [Images](/cua/guide/sandbox/images) — install packages and customize the environment-- [Secrets & Environment Variables](/cua/guide/sandbox/secrets) — safely pass credentials into sandboxes-- [Parallel Sandboxes](/cua/guide/sandbox/scale-out) — run multiple sandboxes concurrently-- [Using the Agent SDK](/cua/guide/get-started/using-agent-sdk) — add AI automation-- [Sandbox Interfaces Reference](/cua/reference/sandbox-sdk/interfaces) — full API for `sb.shell`, `sb.mouse`, `sb.tunnel`, `sb.mobile`, and morediff --git a/docs/content/docs/cua/guide/get-started/what-is-computer-use-agent.mdx b/docs/content/docs/cua/guide/get-started/what-is-computer-use-agent.mdxdeleted file mode 100644index aa276107d6..0000000000--- a/docs/content/docs/cua/guide/get-started/what-is-computer-use-agent.mdx+++ /dev/null@@ -1,122 +0,0 @@-----title: What is a Computer-Use Agent?-description: Understanding AI agents that can see and interact with computer interfaces------A **computer-use agent** is an AI system that can perceive, understand, and interact with graphical user interfaces (GUIs) just like a human user would, as well as execute code and shell commands in isolated environments. These agents can "see" the screen and perform actions through simulated mouse and keyboard inputs, or run scripts and programs directly to accomplish tasks.--<div className="not-prose relative rounded-xl overflow-hidden my-8 w-full">- <img- src="/docs/img/what-is-cua.png"- alt="How Computer-Use Agents work"- className="w-full h-auto rounded-xl"- />-</div>--## How Computer-Use Agents Work--Computer-use agents operate through a continuous loop:--```-┌─────────────────────────────────────────┐-│ 1. OBSERVE │-│ Take a screenshot of the screen │-└──────────────────┬──────────────────────┘- │- ▼-┌─────────────────────────────────────────┐-│ 2. UNDERSTAND │-│ Vision-language model analyzes │-│ the screenshot and current goal │-└──────────────────┬──────────────────────┘- │- ▼-┌─────────────────────────────────────────┐-│ 3. DECIDE │-│ Determine the next action: │-│ click, type, scroll, etc. │-└──────────────────┬──────────────────────┘- │- ▼-┌─────────────────────────────────────────┐-│ 4. ACT │-│ Execute the action on the computer │-└──────────────────┬──────────────────────┘- │- ▼- Loop back to 1-```--This cycle repeats until the agent completes its goal or determines it cannot proceed.--## Key Components--### Vision-Language Models (VLMs)--At the core of computer-use agents are **vision-language models**—AI models that can process both images and text. These models:--- Identify UI elements (buttons, text fields, menus)-- Read text on screen-- Understand spatial relationships-- Reason about what actions to take--Popular VLMs for computer use include Claude (Anthropic), GPT-4V (OpenAI), and Gemini (Google).--### Grounding Models--**Grounding models** specialize in precisely locating UI elements on screen. Given a description like "the Submit button," they return exact pixel coordinates. Examples include:--- UI-TARS-- GTA-1-- Moondream--### Planning Models--**Planning models** handle high-level reasoning—deciding what to do next given the current state and goal. They're typically large language models that excel at:--- Breaking down complex tasks into steps-- Handling errors and unexpected situations-- Maintaining context across multiple actions--## Composed vs. End-to-End Agents--### End-to-End Agents--A single model handles both understanding and action selection:--```-Screenshot → [Single VLM] → Action-```--**Pros:** Simpler architecture, lower latency-**Cons:** May sacrifice accuracy in either perception or planning--### Composed Agents--Separate models for grounding and planning:--```-Screenshot → [Grounding Model] → UI Elements → [Planning Model] → Action-```--**Pros:** Best-in-class performance for each capability-**Cons:** Higher complexity, potentially higher cost--Cua supports both approaches, letting you choose based on your requirements.--## Challenges in Computer Use--Building effective computer-use agents involves solving several challenges:--- **Coordinate accuracy** - Clicking the right pixel matters-- **Timing** - Waiting for pages to load, animations to complete-- **Error recovery** - Handling popups, errors, and unexpected states-- **Context management** - Remembering what's been done across many steps-- **Safety** - Preventing unintended actions on real systems--Cua addresses these challenges with sandboxed environments, optimized agent loops, and built-in safety mechanisms.--## Next Steps--- Learn about [Desktop Sandboxes](/cua/guide/get-started/what-is-desktop-sandbox) where agents run safely-- [Set Up a Sandbox](/cua/guide/get-started/set-up-sandbox) to build your first agentdiff --git a/docs/content/docs/cua/guide/get-started/what-is-cua.mdx b/docs/content/docs/cua/guide/get-started/what-is-cua.mdxdeleted file mode 100644index c351fc1894..0000000000--- a/docs/content/docs/cua/guide/get-started/what-is-cua.mdx+++ /dev/null@@ -1,97 +0,0 @@-----title: What is Cua?-description: Open-source sandboxes and agent framework for computer-use AI------Cua lets you spin up isolated desktop environments — Linux, macOS, Windows, Android — and build agents that can see and control them. One SDK, any OS, cloud or local.--```python-import asyncio-from cua import Sandbox, Image, ComputerAgent--async def main():- async with Sandbox.ephemeral(Image.linux(), local=True) as sb:- agent = ComputerAgent(- model="cua/anthropic/claude-sonnet-4-5",- tools=[sb],- )- async for result in agent.run([{- "role": "user",- "content": "Open Firefox and search for 'open source computer use'",- }]):- for item in result["output"]:- if item["type"] == "message":- print(item["content"][0]["text"])--asyncio.run(main())-```--## Install--```bash-pip install cua-```--No account needed to run locally with Docker. For cloud sandboxes and hosted models, get a free API key at [cua.ai](https://cua.ai/signin).--## What it includes--**Sandbox SDK** — create and control sandboxes programmatically. Same API everywhere:--```python-async with Sandbox.ephemeral(Image.linux()) as sb: # cloud Linux container-async with Sandbox.ephemeral(Image.macos()) as sb: # cloud macOS VM-async with Sandbox.ephemeral(Image.windows()) as sb: # cloud Windows VM-async with Sandbox.ephemeral(Image.android()) as sb: # cloud Android VM-async with Sandbox.ephemeral(Image.linux(), local=True) as sb: # local Docker-```--Inside any sandbox:--```python-await sb.shell.run("npm install") # run shell commands-await sb.screenshot() # capture the screen-await sb.mouse.click(x, y) # click-await sb.keyboard.type("hello") # type-await sb.keyboard.press("ctrl+c") # key combos-await sb.mobile.tap(x, y) # Android touch-async with sb.tunnel.forward(3000) as t: # forward ports- print(t.url)-```--**Agent SDK** — plug any vision-language model into a computer-use loop:--```python-agent = ComputerAgent(- model="cua/anthropic/claude-sonnet-4-5", # or openai, gemini, qwen, ...- tools=[sb],-)-async for result in agent.run(messages): ...-```--**Image builder** — configure the environment before the sandbox starts:--```python-img = (- Image.linux()- .apt_install("curl", "ffmpeg")- .pip_install("playwright")- .env(API_KEY="...")- .run("playwright install chromium")-)-async with Sandbox.ephemeral(img) as sb: ...-```--## Use cases--- **AI coding assistants** — safe isolated environments for Claude Code, Codex, and similar tools-- **Browser and desktop automation** — agents that interact with any real UI-- **Cross-platform testing** — run the same test against Linux, macOS, Windows, and Android-- **Benchmarking** — evaluate agents on OSWorld, ScreenSpot, and custom task sets--## Next--- [Quickstart](/cua/guide/get-started/quickstart) — running in 5 minutes-- [Sandbox Lifecycle](/cua/guide/sandbox/lifecycle) — ephemeral, persistent, connect-- [Images](/cua/guide/sandbox/images) — customizing your environment-- [Examples](/cua/examples/sandboxes/linux) — runnable samples for every OSdiff --git a/docs/content/docs/cua/guide/get-started/what-is-desktop-sandbox.mdx b/docs/content/docs/cua/guide/get-started/what-is-desktop-sandbox.mdxdeleted file mode 100644index 0be41f7c55..0000000000--- a/docs/content/docs/cua/guide/get-started/what-is-desktop-sandbox.mdx+++ /dev/null@@ -1,159 +0,0 @@-----title: What is a Desktop Sandbox?-description: Understanding isolated environments for safe computer-use agent execution------A **desktop sandbox** is an isolated virtual environment that provides a complete desktop operating system where computer-use agents can safely execute tasks. Think of it as a disposable computer that your agent can control without risk to your real system.--<div className="not-prose relative rounded-xl overflow-hidden my-8 w-full">- <img- src="/docs/img/what-is-sandbox.png"- alt="What is a Desktop Sandbox"- className="w-full h-auto rounded-xl"- />-</div>--## Why Use Sandboxes?--Running AI agents directly on your computer poses risks:--- **Unintended actions** - Agents might click the wrong button or delete files-- **Security concerns** - Malicious content could affect your system-- **State pollution** - Each run might leave behind artifacts that affect future runs-- **Reproducibility** - Hard to ensure consistent starting conditions--Sandboxes solve these problems by providing:--- **Isolation** - Actions in the sandbox don't affect your host system-- **Reset capability** - Start fresh for each task-- **Safety** - Experiment freely without consequences-- **Consistency** - Identical environment every time-- **Reproducibility** - Define dependencies and configurations that create baseline environments your agents can rely on--## Sandbox Options in Cua--Cua supports multiple sandbox types to fit different needs:--<div className="not-prose relative rounded-xl overflow-hidden my-8 w-full">- <img- src="/docs/img/sandbox-options.png"- alt="Sandbox Options in Cua"- className="w-full h-auto rounded-xl"- />-</div>--### Cloud Sandboxes--Managed virtual machines hosted by Cua's infrastructure.--| Feature | Details |-| -------------- | ----------------------------------- |-| **OS Options** | Linux (Ubuntu), Windows, macOS |-| **Setup** | Instant—create via CLI or dashboard |-| **Best For** | Production workloads, teams, CI/CD |-| **Access** | API key required |--```bash-# Create a cloud sandbox-cua sb create --os linux --size small --region north-america-```--### Docker Sandboxes--Linux desktops running in Docker containers on your machine.--| Feature | Details |-| -------------- | ---------------------------- |-| **OS Options** | Linux (XFCE, Ubuntu Desktop) |-| **Setup** | Docker Desktop required |-| **Best For** | Local development, testing |-| **Access** | No API key needed |--```bash-# Pull and run a Docker sandbox-docker pull trycua/cua-xfce:latest-```--### QEMU Docker Sandboxes--Full virtual machines (Linux, Windows, Android) running inside Docker via QEMU emulation.--| Feature | Details |-| -------------- | --------------------------------- |-| **OS Options** | Linux, Windows 11, Android |-| **Setup** | Requires golden image preparation |-| **Best For** | Testing on specific OS versions |-| **Access** | No API key needed |--### macOS Sandboxes (Lume)--Native macOS virtual machines using Apple's Virtualization framework.--| Feature | Details |-| -------------- | ------------------------------------- |-| **OS Options** | macOS (various versions) |-| **Setup** | macOS host required, install Lume CLI |-| **Best For** | macOS-specific automation |-| **Access** | No API key needed |--```bash-# Start a macOS sandbox-lume run macos-sequoia-cua:latest-```--### Windows Sandbox--Native Windows sandbox using Windows Sandbox feature.--| Feature | Details |-| -------------- | --------------------------------------- |-| **OS Options** | Windows |-| **Setup** | Windows 10 Pro/Enterprise or Windows 11 |-| **Best For** | Windows-specific automation |-| **Access** | No API key needed |--## How Sandboxes Connect to Agents--Your agent interacts with sandboxes through the Computer Framework:--```-┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐-│ │ │ │ │ │-│ Your Agent │────▶│ Computer │────▶│ Sandbox │-│ (AI Logic) │ │ Framework │ │ (Desktop) │-│ │◀────│ │◀────│ │-└─────────────────┘ └─────────────────┘ └─────────────────┘- │- │ Unified API- │- ┌─────────┴─────────┐- │ │- Screenshots Actions- (what agent sees) (click, type, etc.)-```--The Computer Framework provides the same API regardless of which sandbox type you're using. This means you can:--- Develop locally with Docker-- Test on specific platforms with QEMU-- Deploy to production with Cloud Sandboxes--...all without changing your agent code.--## Choosing the Right Sandbox--| Use Case | Recommended Sandbox |-| -------------------------- | ---------------------------------- |-| Getting started / learning | Docker (Linux) |-| Production deployment | Cloud Sandbox |-| macOS automation | Lume (macOS) |-| Windows automation | Cloud (Windows) or Windows Sandbox |-| Android testing | QEMU Docker (Android) |-| CI/CD pipelines | Cloud Sandbox |-| Air-gapped environments | Docker or QEMU |--## Next Steps--- [Set Up a Sandbox](/cua/guide/get-started/set-up-sandbox) to create your first environment-- Learn about [Computer SDK](/cua/reference/computer-sdk) for controlling sandboxes-- Explore [Computer SDK Reference](/cua/reference/computer-sdk) for production deploymentsdiff --git a/docs/content/docs/cua/guide/index.mdx b/docs/content/docs/cua/guide/index.mdxdeleted file mode 100644index c0a4c87160..0000000000--- a/docs/content/docs/cua/guide/index.mdx+++ /dev/null@@ -1,67 +0,0 @@-----title: Introduction------import { Monitor, Code, BookOpen, Zap, Bot, Boxes, Rocket } from 'lucide-react';--<div className="not-prose -mt-2 mb-6">- <p className="text-fd-primary font-semibold text-sm mb-1">Welcome</p>- <h1 className="text-3xl font-bold tracking-tight md:text-4xl">Welcome to Cua</h1>-</div>--**Cua** is an open-source framework for building and evaluating Computer-Use Agents - AI systems that autonomously interact with computer interfaces by understanding visual elements and executing actions.--<div className="not-prose relative rounded-xl overflow-hidden my-8 w-full">- <img src="/docs/img/hero.png" alt="Cua" className="w-full h-auto rounded-xl" />-</div>--## What is a Computer-Use Agent?--Computer-Use Agents (CUAs) are AI systems that can autonomously interact with computer interfaces through visual understanding and action execution. They work by capturing screenshots, feeding them to a vision-language model (VLM), and letting the model determine the next action to take - such as clicking, typing, or scrolling - in a continuous loop until the task is complete.--## What is a Computer-Use Sandbox?--Computer-Use Sandboxes are isolated, controlled environments where AI agents can safely interact with computer interfaces. They provide a secure execution space for agents to perform actions such as clicking, typing, and running code, test automation workflows, and learn from interactions without affecting production systems.--## Key Features--With **Cua Computer Framework**, you can:--- Automate **Windows, Linux, macOS, and Android** sandboxes with a consistent API-- Create & manage sandboxes locally or using **Cua Cloud**--With **Cua Agent Framework**, you can:--- Run computer-use models with a consistent schema-- Benchmark on **OSWorld-Verified**, **SheetBench-V2**, and **ScreenSpot**-- Combine UI grounding models with any LLM using **composed agents**-- Use **100+ models** via API or local inference (Claude, GPT-4, Gemini, Ollama, MLX)--## Get Started--Follow the [Get Started guide](/cua/guide/get-started/what-is-cua) for step-by-step setup with Python or TypeScript.--Check out our [tutorials](https://cua.ai/blog), [examples](https://github.com/trycua/cua/tree/main/examples), and [notebooks](https://github.com/trycua/cua/tree/main/notebooks) to start building with Cua today.--<div className="grid grid-cols-2 md:grid-cols-4 gap-2 mt-4 text-sm">- <Card- icon={<Rocket className="w-4 h-4" />}- href="/cua/guide/get-started/what-is-cua"- title="Get Started"- />- <Card- icon={<Zap className="w-4 h-4" />}- href="/cua/guide/fundamentals/agent-loops"- title="Agent Loops"- />- <Card- icon={<BookOpen className="w-4 h-4" />}- href="/cua/reference/sandbox-sdk"- title="Sandbox SDK"- />- <Card- icon={<Monitor className="w-4 h-4" />}- href="/cua/examples/automation/form-filling"- title="Examples"- />-</div>diff --git a/docs/content/docs/cua/guide/integrations/benchmarks/index.mdx b/docs/content/docs/cua/guide/integrations/benchmarks/index.mdxdeleted file mode 100644index 8502b9ebed..0000000000--- a/docs/content/docs/cua/guide/integrations/benchmarks/index.mdx+++ /dev/null@@ -1,29 +0,0 @@-----title: Benchmarks-description: Computer Agent SDK benchmarks for agentic GUI tasks------The benchmark system evaluates models on GUI grounding tasks, specifically agent loop success rate and click prediction accuracy. It supports both:--- **Computer Agent SDK providers** (using model strings like `"huggingface-local/HelloKKMe/GTA1-7B"`)-- **Reference agent implementations** (custom model classes implementing the `ModelProtocol`)--## Available Benchmarks--- **[ScreenSpot-v2](./screenspot-v2)** - Standard resolution GUI grounding-- **[ScreenSpot-Pro](./screenspot-pro)** - High-resolution GUI grounding-- **[Interactive Testing](./interactive)** - Real-time testing and visualization--## Quick Start--```bash-# Clone the benchmark repository-git clone https://github.com/trycua/cua-cd libs/python/agent/benchmarks--# Install dependencies-pip install cua--# Run a benchmark-python ss-v2.py-```diff --git a/docs/content/docs/cua/guide/integrations/benchmarks/interactive.mdx b/docs/content/docs/cua/guide/integrations/benchmarks/interactive.mdxdeleted file mode 100644index 43170ca461..0000000000--- a/docs/content/docs/cua/guide/integrations/benchmarks/interactive.mdx+++ /dev/null@@ -1,21 +0,0 @@-----title: Interactive Tool-description: Real-time testing and visualization tool for GUI grounding models------This tool allows you to test multiple models interactively by providing natural language instructions. It automatically captures screenshots and tests all configured models sequentially, providing immediate feedback and visual results.--## Usage--```bash-# Start the interactive tool-cd libs/python/agent/benchmarks-python interactive.py-```--## Commands--- **Type instruction**: Screenshot + test all models-- **`screenshot`**: Take screenshot without prediction-- **`models`**: List available models-- **`quit`/`exit`**: Exit the tooldiff --git a/docs/content/docs/cua/guide/integrations/benchmarks/introduction.mdx b/docs/content/docs/cua/guide/integrations/benchmarks/introduction.mdxdeleted file mode 100644index 67a90769bd..0000000000--- a/docs/content/docs/cua/guide/integrations/benchmarks/introduction.mdx+++ /dev/null@@ -1,58 +0,0 @@-----title: Introduction-description: Overview of benchmarking in the Cua agent framework------The Cua agent framework uses benchmarks to test the performance of supported models and providers at various agentic tasks.--## Benchmark Types--Computer-Agent benchmarks evaluate two key capabilities:--- **Plan Generation**: Breaking down complex tasks into a sequence of actions-- **Coordinate Generation**: Predicting precise click locations on GUI elements--## Using State-of-the-Art Models--Let's see how to use the SOTA vision-language models in the Cua agent framework.--### Plan Generation + Coordinate Generation--**[OS-World](https://os-world.github.io/)** - Benchmark for complete computer-use agents--This leaderboard tests models that can understand instructions and automatically perform the full sequence of actions needed to complete tasks.--```python-# UI-TARS-1.5 is a SOTA unified plan generation + coordinate generation VLM-# This makes it suitable for agentic loops for computer-use-agent = ComputerAgent("huggingface-local/ByteDance-Seed/UI-TARS-1.5-7B", tools=[computer])-agent.run("Open Firefox and go to github.com")-# Success! 🎉-```--### Coordinate Generation Only--**[GUI Agent Grounding Leaderboard](https://gui-agent.github.io/grounding-leaderboard/)** - Benchmark for click prediction accuracy--This leaderboard tests models that specialize in finding exactly where to click on screen elements, but needs to be told what specific action to take.--```python-# GTA1-7B is a SOTA coordinate generation VLM-# It can only generate coordinates, not plan:-agent = ComputerAgent("huggingface-local/HelloKKMe/GTA1-7B", tools=[computer])-agent.predict_click("find the button to open the settings") # (27, 450)-# This will raise an error:-# agent.run("Open Firefox and go to github.com")-```--### Composed Agent--The Cua agent framework also supports composed agents, which combine a planning model with a clicking model for the best of both worlds. Any liteLLM model can be used as the plan generation model.--```python-# It can be paired with any LLM to form a composed agent:-# "gemini/gemini-1.5-pro" will be used as the plan generation LLM-agent = ComputerAgent("huggingface-local/HelloKKMe/GTA1-7B+gemini/gemini-1.5-pro", tools=[computer])-agent.run("Open Firefox and go to github.com")-# Success! 🎉-```diff --git a/docs/content/docs/cua/guide/integrations/benchmarks/meta.json b/docs/content/docs/cua/guide/integrations/benchmarks/meta.jsondeleted file mode 100644index 8067bb0114..0000000000--- a/docs/content/docs/cua/guide/integrations/benchmarks/meta.json+++ /dev/null@@ -1,3 +0,0 @@-{- "pages": ["introduction", "screenspot-v2", "screenspot-pro", "interactive", "osworld-verified"]-}diff --git a/docs/content/docs/cua/guide/integrations/benchmarks/osworld-verified.mdx b/docs/content/docs/cua/guide/integrations/benchmarks/osworld-verified.mdxdeleted file mode 100644index 150b7b073d..0000000000--- a/docs/content/docs/cua/guide/integrations/benchmarks/osworld-verified.mdx+++ /dev/null@@ -1,8 +0,0 @@-----title: OSWorld-Verified-description: Benchmark ComputerAgent on OSWorld tasks using HUD------OSWorld-Verified is a curated subset of OSWorld tasks that can be run using the HUD framework.--Use [ComputerAgent with HUD](../hud) to benchmark on these tasks.diff --git a/docs/content/docs/cua/guide/integrations/benchmarks/screenspot-pro.mdx b/docs/content/docs/cua/guide/integrations/benchmarks/screenspot-pro.mdxdeleted file mode 100644index 15739d9d11..0000000000--- a/docs/content/docs/cua/guide/integrations/benchmarks/screenspot-pro.mdx+++ /dev/null@@ -1,25 +0,0 @@-----title: ScreenSpot-Pro-description: High-resolution GUI grounding benchmark------ScreenSpot-Pro is a benchmark for evaluating click prediction accuracy on high-resolution GUI screenshots with complex layouts.--## Usage--```bash-# Run the benchmark-cd libs/python/agent/benchmarks-python ss-pro.py--# Run with custom sample limit-python ss-pro.py --samples 50-```--## Results--| Model | Accuracy | Failure Rate | Samples |-| ----------- | -------- | ------------ | ------- |-| Coming Soon | - | - | - |--Results will be populated after running benchmarks with various models.diff --git a/docs/content/docs/cua/guide/integrations/benchmarks/screenspot-v2.mdx b/docs/content/docs/cua/guide/integrations/benchmarks/screenspot-v2.mdxdeleted file mode 100644index ba78d5f915..0000000000--- a/docs/content/docs/cua/guide/integrations/benchmarks/screenspot-v2.mdx+++ /dev/null@@ -1,25 +0,0 @@-----title: ScreenSpot-v2-description: Standard resolution GUI grounding benchmark------ScreenSpot-v2 is a benchmark for evaluating click prediction accuracy on standard resolution GUI screenshots.--## Usage--```bash-# Run the benchmark-cd libs/python/agent/benchmarks-python ss-v2.py--# Run with custom sample limit-python ss-v2.py --samples 100-```--## Results--| Model | Accuracy | Failure Rate | Samples |-| ----------- | -------- | ------------ | ------- |-| Coming Soon | - | - | - |--Results will be populated after running benchmarks with various models.diff --git a/docs/content/docs/cua/guide/integrations/hud.mdx b/docs/content/docs/cua/guide/integrations/hud.mdxdeleted file mode 100644index 3656d69346..0000000000--- a/docs/content/docs/cua/guide/integrations/hud.mdx+++ /dev/null@@ -1,138 +0,0 @@-----title: HUD-description: Use ComputerAgent with HUD for benchmarking and evaluation-icon: hud------<Callout>- A corresponding <a href="https://github.com/trycua/cua/blob/main/notebooks/eval_osworld.ipynb" target="_blank">Jupyter Notebook</a> is available for this documentation.-</Callout>--The HUD integration allows an agent to be benchmarked using the [HUD framework](https://www.hud.ai/). Through the HUD integration, the agent controls a computer inside HUD, where tests are run to evaluate the success of each task.--## Installation--First, install the required package:--```bash-pip install "cua[hud]"-## or install hud-python directly-# pip install hud-python==0.4.12-```--## Environment Variables--Before running any evaluations, you’ll need to set up your environment variables for HUD and your model providers:--```bash-# HUD access-export HUD_API_KEY="your_hud_api_key"--# Model provider keys (at least one required)-export OPENAI_API_KEY="your_openai_key"-export ANTHROPIC_API_KEY="your_anthropic_key"-```--## Running a Single Task--You can run a single task from a HUD dataset for quick verification.--### Example--```python-from cua_agent.integrations.hud import run_single_task--await run_single_task(- dataset="hud-evals/OSWorld-Verified", # or another HUD dataset- model="openai/computer-use-preview+openai/gpt-5-nano", # any supported model string- task_id=155, # e.g., reopen last closed tab-)-```--### Parameters--- `task_id` (`int`): Default: `0`- Index of the task to run from the dataset.--## Running a Full Dataset--To benchmark your agent at scale, you can run an entire dataset (or a subset) in parallel.--### Example--```python-from cua_agent.integrations.hud import run_full_dataset--results = await run_full_dataset(- dataset="hud-evals/OSWorld-Verified", # can also pass a Dataset or list[dict]- model="openai/computer-use-preview",- split="train[:3]", # try a few tasks to start- max_concurrent=20, # tune to your infra- max_steps=50 # safety cap per task-)-```--### Parameters--- `job_name` (`str` | `None`):- Optional human-readable name for the evaluation job (shows up in HUD UI).-- `max_concurrent` (`int`): Default: `30`- Number of tasks to run in parallel. Scale this based on your infra.-- `max_steps` (`int`): Default: `50`- Safety cap on steps per task to prevent infinite loops.-- `split` (`str`): Default: `"train"`- Dataset split or subset to run. Uses the [Hugging Face split format](https://huggingface.co/docs/datasets/v1.11.0/splits.html), e.g., `"train[:10]"` for the first 10 tasks.--## Additional Parameters--Both single-task and full-dataset runs share a common set of configuration options. These let you fine-tune how the evaluation runs.--- `dataset` (`str` | `Dataset` | `list[dict]`): **Required**- HUD dataset name (e.g. `"hud-evals/OSWorld-Verified"`), a loaded `Dataset`, or a list of tasks.-- `model` (`str`): Default: `"computer-use-preview"`- Model string, e.g. `"openai/computer-use-preview+openai/gpt-5-nano"`. Supports composition with `+` (planning + grounding).-- `allowed_tools` (`list[str]`): Default: `["openai_computer"]`- Restrict which tools the agent may use.-- `tools` (`list[Any]`):- Extra tool configs to inject.-- `custom_loop` (`Callable`):- Optional custom agent loop function. If provided, overrides automatic loop selection.-- `only_n_most_recent_images` (`int`): Default: `5` for full dataset, `None` for single task.- Retain only the last N screenshots in memory.-- `callbacks` (`list[Any]`):- Hook functions for logging, telemetry, or side effects.-- `verbosity` (`int`):- Logging level. Set `2` for debugging every call/action.-- `trajectory_dir` (`str` | `dict`):- Save local copies of trajectories for replay/analysis.-- `max_retries` (`int`): Default: `3`- Number of retries for failed model/tool calls.-- `screenshot_delay` (`float` | `int`): Default: `0.5`- Delay (seconds) between screenshots to avoid race conditions.-- `use_prompt_caching` (`bool`): Default: `False`- Cache repeated prompts to reduce API calls.-- `max_trajectory_budget` (`float` | `dict`):- Limit on trajectory size/budget (e.g., tokens, steps).-- `telemetry_enabled` (`bool`): Default: `True`- Whether to send telemetry/traces to HUD.-- `**kwargs` (`any`):- Any additional keyword arguments are passed through to the agent loop or model provider.--## Available Benchmarks--HUD provides multiple benchmark datasets for realistic evaluation.--1. **[OSWorld-Verified](/cua/guide/integrations/benchmarks/osworld-verified)** – Benchmark on 369+ real-world desktop tasks across Chrome, LibreOffice, GIMP, VS Code, etc.- _Best for_: evaluating full computer-use agents in realistic environments.- _Verified variant_: fixes 300+ issues from earlier versions for reliability.--**Coming soon:** SheetBench (spreadsheet automation) and other specialized HUD datasets.--See the [HUD docs](https://docs.hud.ai/platform/environments) for more eval environments.--## Tips--- **Debugging:** set `verbosity=2` to see every model call and tool action.-- **Performance:** lower `screenshot_delay` for faster runs; raise it if you see race conditions.-- **Safety:** always set `max_steps` (defaults to 50) to prevent runaway loops.-- **Custom tools:** pass extra `tools=[...]` into the agent config if you need beyond `openai_computer`.diff --git a/docs/content/docs/cua/guide/integrations/laminar.mdx b/docs/content/docs/cua/guide/integrations/laminar.mdxdeleted file mode 100644index 4b047d5c28..0000000000--- a/docs/content/docs/cua/guide/integrations/laminar.mdx+++ /dev/null@@ -1,64 +0,0 @@-----title: Laminar-description: Trace Cua execution steps and sessions-icon: laminar------Cua has a native integration with [Laminar](https://laminar.sh/) – an open-source platform for tracing, evals, and labeling of autonomous AI agents. Read more about Laminar in the [Laminar docs](https://docs.lmnr.ai/).--## Setup--Register on [Laminar Cloud](https://laminar.sh/) or spin up a [local instance](https://github.com/lmnr-ai/lmnr) and get the key from your project settings. Set the `LMNR_PROJECT_API_KEY` environment variable to your key.--```bash-pip install lmnr[all]-export LMNR_PROJECT_API_KEY=your-key-```--## Usage--Then, initialize Laminar at the entry point of your application, register Laminar LiteLLM callback, and all steps of Cua will be automatically traced.--```python-import os--import litellm--from cua import ComputerAgent, Sandbox, Image-from lmnr import Laminar, LaminarLiteLLMCallback # [!code highlight]--Laminar.initialize() # [!code highlight]-litellm.callbacks.append(LaminarLiteLLMCallback()) # [!code highlight]--computer = await Sandbox.create(- os_type="linux",- provider_type="cloud",- name=os.getenv("CUA_CONTAINER_NAME"),- api_key=os.getenv("CUA_API_KEY"),-)--agent = ComputerAgent(- model="openai/computer-use-preview",- tools=[computer],-)--async def main():- async for step in agent.run("Create a new file called 'test.txt' in the current directory"):- print(step["output"])--if __name__ == "__main__":- asyncio.run(main())-```--## Viewing traces--You can view traces in the Laminar UI by going to the traces tab in your project. When you select a trace,-you will see all the agent execution steps, including computer actions, LLM calls, and screenshots.--For each step, you will see the LLM call, the computer action. The computer actions are highlighted in the timeline in yellow.--<img- src="/docs/img/laminar_trace_example.png"- alt="Example trace in Laminar showing the litellm.response span and its output."- width="800px"-/>diff --git a/docs/content/docs/cua/guide/integrations/meta.json b/docs/content/docs/cua/guide/integrations/meta.jsondeleted file mode 100644index 5909ddb254..0000000000--- a/docs/content/docs/cua/guide/integrations/meta.json+++ /dev/null@@ -1,6 +0,0 @@-{- "title": "Integrations",- "description": "Third-party tools and integrations",- "icon": "Plug",- "pages": ["laminar", "hud", "yutori", "benchmarks"]-}diff --git a/docs/content/docs/cua/guide/integrations/yutori.mdx b/docs/content/docs/cua/guide/integrations/yutori.mdxdeleted file mode 100644index 6ca536558b..0000000000--- a/docs/content/docs/cua/guide/integrations/yutori.mdx+++ /dev/null@@ -1,82 +0,0 @@-----title: Yutori-description: Use Yutori's n1.5 model with Cua cloud sandboxes-icon: yutori------Cua supports [Yutori](https://yutori.com/)'s `n1.5` model as a computer-use agent. This guide shows how to connect Yutori n1.5 to a Cua cloud sandbox for an interactive agent loop.--## Setup--Install the Cua SDK and `python-dotenv`:--```bash-uv pip install cua-computer cua-agent python-dotenv-```--Set the following environment variables (or add them to a `.env` file):--```bash-export CUA_API_KEY=your-cua-api-key-export YUTORI_API_KEY=your-yutori-api-key-```--- **`CUA_API_KEY`** — your Cua cloud API key. Get one from the [Cua dashboard](https://cua.ai/dashboard).-- **`YUTORI_API_KEY`** — your Yutori API key. Get one from [Yutori](https://yutori.com/).--## Usage--Create a file called `yutori_cua_script.py` with the following contents:--```python-import asyncio-import os--import dotenv--dotenv.load_dotenv()--from cua import ComputerAgent, Computer--CUA_API_KEY = os.environ["CUA_API_KEY"]-YUTORI_API_KEY = os.environ["YUTORI_API_KEY"]---async def main():- async with Computer(- os_type="linux",- provider_type="cloudv2",- name="my-sandbox",- api_key=CUA_API_KEY,- ) as computer:- agent = ComputerAgent(- model="yutori/n1.5",- tools=[computer],- api_key=YUTORI_API_KEY,- )-- history = []- while True:- user_input = input("> ")- history.append({"role": "user", "content": user_input})-- async for result in agent.run(history, stream=False):- history += result["output"]---asyncio.run(main())-```--## Running the script--```bash-uv run yutori_cua_script.py-```--Once running, you'll see a `>` prompt. Type a natural-language instruction and the Yutori n1.5 agent will execute it inside the Cua cloud sandbox. For example:--```-> Open Firefox and search for "Cua computer use agent"-```--The agent maintains conversation history, so you can issue follow-up instructions that build on previous actions.diff --git a/docs/content/docs/cua/guide/meta.json b/docs/content/docs/cua/guide/meta.jsondeleted file mode 100644index d9c32772de..0000000000--- a/docs/content/docs/cua/guide/meta.json+++ /dev/null@@ -1,6 +0,0 @@-{- "title": "Guide",- "description": "Learn how to build with Cua",- "icon": "Book",- "pages": ["get-started", "sandbox", "agents-in-sandbox", "fundamentals", "advanced", "integrations"]-}diff --git a/docs/content/docs/cua/guide/sandbox/images.mdx b/docs/content/docs/cua/guide/sandbox/images.mdxdeleted file mode 100644index 5142d113d1..0000000000--- a/docs/content/docs/cua/guide/sandbox/images.mdx+++ /dev/null@@ -1,218 +0,0 @@-----title: Images-description: Choose and customize the OS image your sandbox runs------import { Callout } from 'fumadocs-ui/components/callout';--An `Image` describes the operating system and software environment your sandbox will run. Images are immutable and chainable — each builder method returns a new `Image` value so you can fork and reuse them freely.--## Built-in images--```python-from cua import Image--Image.linux() # Ubuntu 24.04 container (default)-Image.linux(kind="vm") # Ubuntu 24.04 full VM-Image.linux("ubuntu", "22.04") # Older Ubuntu-Image.macos() # macOS Tahoe (latest)-Image.macos("15") # macOS Sequoia-Image.windows() # Windows 11-Image.android() # Android 14-```--| Image | Kind | Notes |-|-------|------|-------|-| `Image.linux()` | container | Fastest startup, Linux only |-| `Image.linux(kind="vm")` | VM | Full kernel, supports `/dev/kvm` |-| `Image.macos()` | VM | Apple Virtualization / Lume |-| `Image.windows()` | VM | QEMU or Hyper-V |-| `Image.android()` | VM | QEMU Android emulator |-------## The Image builder--Use the chainable builder to install packages and configure the environment before your sandbox starts. Each method returns a new `Image` — the original is unchanged.--```python-img = (- Image.linux()- .apt_install("curl", "git", "ffmpeg")- .pip_install("requests", "Pillow")- .env(MY_TOKEN="abc123", DEBUG="1")- .run("mkdir -p /app/data")- .expose(8080)-)--async with Sandbox.ephemeral(img) as sb:- result = await sb.shell.run("curl --version")- print(result.stdout)-```--### Package installation--**Linux — apt**--```python-Image.linux().apt_install("curl", "git", "build-essential", "ffmpeg")-```--**macOS — Homebrew**--```python-Image.macos().brew_install("ffmpeg", "jq")-```--**Windows — Chocolatey or winget**--```python-Image.windows().choco_install("nodejs", "git")-Image.windows().winget_install("Microsoft.VisualStudioCode")-```--**Android — APK sideloading**--```python-Image.android().apk_install("/path/to/app.apk")-```--**Android — PWA (Trusted Web Activity)**--`pwa_install` builds a signed TWA APK from a Web App Manifest URL using [Bubblewrap](https://github.com/GoogleChromeLabs/bubblewrap) and installs it into the emulator. The app opens full-screen with no browser chrome.--```python-Image.android().pwa_install(- "https://example.com/manifest.json",- package_name="com.example.myapp", # optional — derived from hostname if omitted- keystore="/path/to/android.keystore", # optional — auto-generated if omitted- keystore_alias="android", # default: "android"- keystore_password="android", # default: "android"-)-```--**How signing works.** Chrome will only open the PWA in TWA mode (no browser UI) when the APK's SHA-256 signing certificate matches the fingerprint served by the origin at `/.well-known/assetlinks.json`. To make this reliable across runs:--1. Generate or commit a keystore to your repo.-2. Extract the SHA-256 fingerprint with `keytool -list -v -keystore android.keystore`.-3. Serve that fingerprint from `/.well-known/assetlinks.json` on your PWA host.-4. Pass the keystore path to `pwa_install`.--If you omit `keystore`, a fresh one is generated and cached under `~/.cua/cua-sandbox/pwa-cache/` alongside the built APK — convenient for local development, but the fingerprint will differ between machines.--**System requirements.** Node.js ≥ 18 and Java ≤ 21 must be on `PATH` (Gradle does not support Java 22+). The Android SDK and Bubblewrap CLI are auto-installed on first use.--**Built APKs are cached** by `(manifest_url, package_name)` under `~/.cua/cua-sandbox/pwa-cache/` and reused on subsequent runs.--### Python packages--```python-Image.linux().pip_install("numpy", "pandas", "playwright")--# or via uv (faster, installs into cua-server project)-Image.linux().uv_install("numpy", "pandas")-```--### Environment variables--```python-Image.linux().env(- DATABASE_URL="postgres://localhost/mydb",- LOG_LEVEL="debug",-)-```--<Callout type="warn">-Avoid putting real secrets in `.env()` — anyone who can read the `Image` spec can see the values. Use [Secrets](/cua/guide/sandbox/secrets) instead.-</Callout>--### Shell commands--Run arbitrary shell commands during setup:--```python-Image.linux().run("curl -fsSL https://deb.nodesource.com/setup_20.x | bash -")-```--### Copy local files--```python-Image.linux().copy("./config.json", "/app/config.json")-```--### Expose ports--Mark ports the sandbox will serve on (used for `sb.tunnel.forward()`):--```python-Image.linux().expose(8080).expose(5432)-```-------## Chaining builder calls--Builder calls are fully composable and can be combined in any order. Each call returns a new `Image`:--```python-base = Image.linux().apt_install("curl", "git")--# Fork the base for two different use cases-dev = base.pip_install("ipython", "rich").env(DEBUG="1")-prod = base.pip_install("gunicorn").run("useradd -m appuser")-```-------## Custom OCI images--Pull any image from a public OCI registry:--```python-Image.from_registry("ghcr.io/trycua/macos-tahoe-cua:latest")-Image.from_registry("ubuntu:22.04")-```--You can also chain builder methods on top of a registry image:--```python-img = Image.from_registry("ubuntu:22.04").apt_install("curl")-```-------## Local disk images--Run from a local `.qcow2`, `.vhdx`, `.raw`, or `.iso` file — useful for bring-your-own images or OSWorld tasks:--```python-Image.from_file("/path/to/disk.qcow2", os_type="linux")-Image.from_file("/path/to/windows.vhdx", os_type="windows")-```--URLs are also supported and cached automatically:--```python-Image.from_file("https://example.com/disk.qcow2", os_type="linux")-```--Images are cached in `~/.cua/cua-sandbox/image-cache/`.-------## Inspecting an image--```python-img = Image.linux().apt_install("curl").pip_install("requests")-print(img.to_dict())-# {-# "os_type": "linux",-# "distro": "ubuntu",-# "version": "24.04",-# "kind": "container",-# "layers": [-# {"type": "apt_install", "packages": ["curl"]},-# {"type": "pip_install", "packages": ["requests"]}-# ]-# }-```diff --git a/docs/content/docs/cua/guide/sandbox/interactive-shell.mdx b/docs/content/docs/cua/guide/sandbox/interactive-shell.mdxdeleted file mode 100644index b7dc2b0844..0000000000--- a/docs/content/docs/cua/guide/sandbox/interactive-shell.mdx+++ /dev/null@@ -1,77 +0,0 @@-----title: Interactive Shell-description: Open a live PTY session inside a sandbox or local machine from the CLI or Python SDK------The `cua do shell` command and `computer.pty` API give you an interactive terminal (PTY) session inside any sandbox or local machine — similar to SSH, but without any network setup.--## CLI--```bash-# Switch to a target first-cua do switch docker my-container-# or-cua do switch host--# Open an interactive shell (bash / PowerShell on host)-cua do shell--# Run a specific program interactively-cua do shell python3-cua do shell vim /etc/hosts--# Override terminal dimensions-cua do shell --cols 220 --rows 50-```--The shell is fully interactive — arrow keys, tab completion, and `Ctrl+C` all work. When stdin is piped (non-TTY), the command runs non-interactively and its output is printed normally.--## Python SDK--```python-from cua import Sandbox, Image--async with Sandbox.create(provider_type="docker", name="my-container") as c:- handle = await c.pty.create(- command="bash",- cols=80,- rows=24,- on_data=lambda chunk: print(chunk.decode(errors="replace"), end="", flush=True),- )-- await handle.send_stdin(b"echo hello\n")- await handle.send_stdin(b"exit\n")- code = await handle.wait()- print(f"exited {code}")-```--### PtyHandle methods--| Method | Description |-|---|---|-| `send_stdin(data: bytes)` | Write bytes to the terminal's stdin |-| `resize(cols, rows)` | Resize the terminal window |-| `kill()` | Kill the session process |-| `disconnect()` | Close the WebSocket without killing the process |-| `wait()` | Block until the session exits; returns the exit code |--### Reconnecting--```python-# Disconnect and reconnect later (process keeps running)-await handle.disconnect()--handle2 = await c.pty.connect(- handle.pid,- on_data=lambda chunk: print(chunk.decode(errors="replace"), end=""),-)-```--## How it works--The PTY stack has four layers:--1. **`cua-auto`** — spawns a real PTY process (`pty` on Unix, `pywinpty` on Windows)-2. **`computer-server`** — exposes `/pty` REST endpoints and a WebSocket for I/O-3. **`computer`** — Python client (`PtyInterface`) connecting over HTTP + WebSocket-4. **`cua do shell`** — sets your terminal to raw mode and proxies I/O through the WebSocketdiff --git a/docs/content/docs/cua/guide/sandbox/lifecycle.mdx b/docs/content/docs/cua/guide/sandbox/lifecycle.mdxdeleted file mode 100644index e2989fdfcc..0000000000--- a/docs/content/docs/cua/guide/sandbox/lifecycle.mdx+++ /dev/null@@ -1,146 +0,0 @@-----title: Sandbox Lifecycle-description: Understand ephemeral, persistent, and connected sandboxes — and when to use each------import { Callout } from 'fumadocs-ui/components/callout';--A sandbox is an isolated virtual environment — a Linux container, macOS VM, Windows VM, or Android emulator — that your code can fully control. The Sandbox SDK gives you three lifecycle patterns depending on how long you need the sandbox to live.--## The three patterns--```-┌─────────────────────────────────────────────────────────┐-│ ephemeral create / delete lifecycle in one block │-│ sandbox is gone when the block exits │-│ │-│ create you manage start and stop explicitly │-│ sandbox persists across process exits │-│ │-│ connect attach to an already-running sandbox │-│ sandbox keeps running after disconnect │-└─────────────────────────────────────────────────────────┘-```-------## `Sandbox.ephemeral` — one-shot tasks--Use this for scripts, CI jobs, and any task where you want the sandbox to disappear automatically when you're done. It's an async context manager: the sandbox is created on enter and destroyed on exit, even if an exception is raised.--```python-import asyncio-from cua import Sandbox, Image--async def main():- async with Sandbox.ephemeral(Image.linux()) as sb:- result = await sb.shell.run("echo hello")- print(result.stdout) # "hello\n"- # sandbox is deleted here — no cleanup needed--asyncio.run(main())-```--**When to use:** scripts, tests, CI pipelines, one-off agent runs, any time you don't need state to survive past the current process.-------## `Sandbox.create` — persistent sandbox--Use this when you need the sandbox to outlive your Python process, or when you want to reuse it across multiple scripts without re-provisioning.--```python-import asyncio-from cua import Sandbox, Image--async def main():- # Create and get back a handle — sandbox keeps running after this script exits- sb = await Sandbox.create(Image.linux(), name="my-dev-sandbox")- await sb.shell.run("apt-get install -y vim")- await sb.disconnect() # drop the connection; sandbox stays running--asyncio.run(main())-```--Later, reconnect from any process:--```python-async with Sandbox.connect("my-dev-sandbox") as sb:- result = await sb.shell.run("vim --version")- print(result.stdout)-```--When you're finished with a persistent sandbox, delete it explicitly:--```python-async with Sandbox.connect("my-dev-sandbox") as sb:- await sb.delete()-```--**When to use:** interactive development, long-running workflows, sandboxes that accumulate installed software or state, multi-step pipelines spread across separate scripts.-------## `Sandbox.connect` — attach to existing--Use this to attach to any sandbox that's already running, whether you started it from a previous script, the CLI, or another process.--```python-# Start a sandbox from the CLI-# $ cua sb launch ubuntu:24.04 --name my-sandbox--# Then connect from Python-async with Sandbox.connect("my-sandbox") as sb:- await sb.shell.run("echo reconnected")-# connection is dropped, but sandbox keeps running-```--<Callout type="info">-`Sandbox.connect` never starts or stops the sandbox — it only manages the network connection. The sandbox continues running after `.disconnect()` or after the `async with` block exits.-</Callout>-------## Disconnect vs. Delete--| Method | Effect on sandbox | Use when |-|--------|------------------|----------|-| `await sb.disconnect()` | Keeps running | You want to reconnect later |-| `await sb.delete()` | Permanently destroyed | You're done with it entirely |--Exiting an `ephemeral` context manager calls `delete()` automatically. Exiting a `connect` context manager calls `disconnect()` automatically.-------## Listing running sandboxes--```python-sandboxes = await Sandbox.list()-for info in sandboxes:- print(info.name, info.os_type, info.status)-```--List only local sandboxes:--```python-sandboxes = await Sandbox.list(local=True)-```-------## Cloud vs. local--Every lifecycle pattern works both locally and on Cua Cloud. Pass `local=True` to run on your machine using Docker, Lume, or QEMU.--```python-# Cloud (default) — requires CUA_API_KEY-async with Sandbox.ephemeral(Image.linux()) as sb: ...--# Local Docker — no API key needed-async with Sandbox.ephemeral(Image.linux(), local=True) as sb: ...--# Local macOS VM — requires Lume on macOS-async with Sandbox.ephemeral(Image.macos(), local=True) as sb: ...-```--See [Set Up a Sandbox](/cua/guide/get-started/set-up-sandbox) for local setup instructions.diff --git a/docs/content/docs/cua/guide/sandbox/meta.json b/docs/content/docs/cua/guide/sandbox/meta.jsondeleted file mode 100644index 2603d94ce5..0000000000--- a/docs/content/docs/cua/guide/sandbox/meta.json+++ /dev/null@@ -1,6 +0,0 @@-{- "title": "Sandbox",- "description": "Working with sandboxes",- "icon": "Box",- "pages": ["lifecycle", "images", "snapshots", "secrets", "scale-out", "interactive-shell", "tunneling"]-}diff --git a/docs/content/docs/cua/guide/sandbox/scale-out.mdx b/docs/content/docs/cua/guide/sandbox/scale-out.mdxdeleted file mode 100644index 8d66adbab0..0000000000--- a/docs/content/docs/cua/guide/sandbox/scale-out.mdx+++ /dev/null@@ -1,172 +0,0 @@-----title: Parallel Sandboxes-description: Run multiple sandboxes concurrently to parallelize agent workloads------import { Callout } from 'fumadocs-ui/components/callout';--Because `Sandbox.ephemeral` and `Sandbox.create` are async, you can run many sandboxes at the same time using standard Python async patterns. This is useful for parallel scraping, batch testing, multi-agent workflows, and CI/CD pipelines.--## Run N sandboxes concurrently--`asyncio.gather` is the simplest way to run a fixed set of tasks in parallel:--```python-import asyncio-from cua import Sandbox, Image--async def run_task(task: str) -> str:- async with Sandbox.ephemeral(Image.linux()) as sb:- result = await sb.shell.run(f"echo '{task}'")- return result.stdout.strip()--async def main():- tasks = ["task-1", "task-2", "task-3", "task-4"]- results = await asyncio.gather(*[run_task(t) for t in tasks])- print(results) # ["task-1", "task-2", "task-3", "task-4"]--asyncio.run(main())-```--## Fan out with a semaphore (rate limiting)--If you have a large batch, use a semaphore to cap the number of sandboxes running at once:--```python-import asyncio-from cua import Sandbox, Image--MAX_CONCURRENT = 5--async def process_item(sem: asyncio.Semaphore, item: str) -> dict:- async with sem:- async with Sandbox.ephemeral(Image.linux()) as sb:- result = await sb.shell.run(f"python /app/process.py '{item}'")- return {"item": item, "output": result.stdout, "ok": result.success}--async def main():- items = [f"item-{i}" for i in range(20)]- sem = asyncio.Semaphore(MAX_CONCURRENT)- results = await asyncio.gather(*[process_item(sem, item) for item in items])- print(f"{sum(r['ok'] for r in results)}/{len(results)} succeeded")--asyncio.run(main())-```--## Producer-consumer with a queue--For dynamic workloads where tasks arrive at runtime, use `asyncio.Queue`:--```python-import asyncio-from cua import Sandbox, Image--async def worker(queue: asyncio.Queue, worker_id: int):- while True:- task = await queue.get()- if task is None:- break- async with Sandbox.ephemeral(Image.linux()) as sb:- result = await sb.shell.run(task)- print(f"[worker-{worker_id}] {result.stdout.strip()}")- queue.task_done()--async def main():- queue: asyncio.Queue = asyncio.Queue()- num_workers = 4-- # Start workers- workers = [asyncio.create_task(worker(queue, i)) for i in range(num_workers)]-- # Enqueue tasks- for i in range(10):- await queue.put(f"echo 'job {i}'")-- # Signal workers to stop- for _ in range(num_workers):- await queue.put(None)-- await asyncio.gather(*workers)--asyncio.run(main())-```--## Collecting screenshots in parallel--A common pattern for visual testing — take screenshots across multiple environments simultaneously:--```python-import asyncio-from cua import Sandbox, Image--async def screenshot_env(image: Image, label: str) -> tuple[str, bytes]:- async with Sandbox.ephemeral(image) as sb:- await sb.shell.run("sleep 2") # wait for desktop to settle- png = await sb.screenshot()- return label, png--async def main():- envs = [- (Image.linux(), "linux"),- (Image.linux(kind="vm"), "linux-vm"),- (Image.macos(), "macos"),- (Image.windows(), "windows"),- ]- results = await asyncio.gather(*[screenshot_env(img, label) for img, label in envs])- for label, png in results:- with open(f"screenshot-{label}.png", "wb") as f:- f.write(png)- print(f"Saved screenshot-{label}.png")--asyncio.run(main())-```--## Parallel agent runs--Run the same task across multiple sandboxes to compare model outputs or increase throughput:--```python-import asyncio-from cua import Sandbox, Image, ComputerAgent--async def run_agent(task: str, run_id: int) -> dict:- async with Sandbox.ephemeral(Image.linux()) as sb:- agent = ComputerAgent(- model="cua/anthropic/claude-sonnet-4-5",- tools=[sb],- )- messages = [{"role": "user", "content": task}]- outputs = []- async for result in agent.run(messages):- outputs.append(result)- return {"run_id": run_id, "steps": len(outputs)}--async def main():- task = "Open Firefox and navigate to google.com"- runs = await asyncio.gather(*[run_agent(task, i) for i in range(3)])- for run in runs:- print(f"Run {run['run_id']}: {run['steps']} steps")--asyncio.run(main())-```--<Callout type="info">-Cloud sandboxes are billed per second of runtime. Running many sandboxes in parallel will incur charges proportional to wall-clock time × number of sandboxes. Use `local=True` during development to avoid costs.-</Callout>--## Checking results and handling errors--`asyncio.gather` raises the first exception by default. Use `return_exceptions=True` to collect all results including failures:--```python-results = await asyncio.gather(- *[run_task(t) for t in tasks],- return_exceptions=True-)--for task, result in zip(tasks, results):- if isinstance(result, Exception):- print(f"{task}: FAILED — {result}")- else:- print(f"{task}: {result}")-```diff --git a/docs/content/docs/cua/guide/sandbox/secrets.mdx b/docs/content/docs/cua/guide/sandbox/secrets.mdxdeleted file mode 100644index 7771174362..0000000000--- a/docs/content/docs/cua/guide/sandbox/secrets.mdx+++ /dev/null@@ -1,130 +0,0 @@-----title: Secrets & Environment Variables-description: Pass credentials and configuration into sandboxes without hardcoding values------import { Callout } from 'fumadocs-ui/components/callout';--Sandboxes often need credentials — API keys, database URLs, tokens — to perform real work. There are several ways to get those values in safely.--## The `CUA_API_KEY` environment variable--The Sandbox SDK reads your Cua API key from the `CUA_API_KEY` environment variable. Set it in your shell before running scripts:--```bash-export CUA_API_KEY="sk_cua-..."-```--Or use a `.env` file with your preferred loader:--```bash-# .env-CUA_API_KEY=sk_cua-...-```--```python-from dotenv import load_dotenv-load_dotenv()--from cua import Sandbox, Image-```--You can also pass the key explicitly if you manage it programmatically:--```python-async with Sandbox.ephemeral(Image.linux(), api_key=my_key) as sb:- ...-```-------## Passing secrets into the sandbox at runtime--The recommended pattern is to read secrets from the host environment and inject them into the sandbox via `sb.shell.run`, not via `Image.env()`.--```python-import os-import asyncio-from cua import Sandbox, Image--async def main():- db_url = os.environ["DATABASE_URL"] # read from host- gh_token = os.environ["GITHUB_TOKEN"]-- async with Sandbox.ephemeral(Image.linux()) as sb:- # Set inside the sandbox for this session only- await sb.shell.run(f"export DATABASE_URL='{db_url}'")- await sb.shell.run(f"export GITHUB_TOKEN='{gh_token}'")- await sb.shell.run("python /app/main.py")--asyncio.run(main())-```--For multi-command sessions, write a wrapper script instead of exporting in each call:--```python-script = f"""-export DATABASE_URL='{db_url}'-export GITHUB_TOKEN='{gh_token}'-python /app/main.py-"""-result = await sb.shell.run(script)-```-------## `Image.env()` — bake-in non-secret config--Use `.env()` on the `Image` builder for configuration that isn't sensitive — feature flags, log levels, app settings:--```python-from cua import Image--img = (- Image.linux()- .apt_install("python3")- .env(- LOG_LEVEL="info",- APP_ENV="production",- PORT="8080",- )-)-```--<Callout type="warn">-`.env()` values are stored in the `Image` spec and visible to anyone who can inspect it. Do not use `.env()` for API keys, passwords, or tokens.-</Callout>-------## Copying secret files into the sandbox--For credentials stored as files (SSH keys, service account JSON, certificates):--```python-async with Sandbox.ephemeral(Image.linux()) as sb:- # Read the file on the host and write it in the sandbox- with open(os.path.expanduser("~/.ssh/id_rsa")) as f:- key_content = f.read()-- await sb.shell.run(f"mkdir -p /root/.ssh && chmod 700 /root/.ssh")- await sb.shell.run(f"cat > /root/.ssh/id_rsa << 'EOF'\n{key_content}\nEOF")- await sb.shell.run("chmod 600 /root/.ssh/id_rsa")-```--Alternatively, use `Image.copy()` for files that are safe to bake into the image:--```python-img = Image.linux().copy("./app-config.json", "/app/config.json")-```-------## Best practices--| Do | Don't |-|----|-------|-| Read secrets from host environment variables | Hardcode secrets in Python source |-| Inject secrets at runtime via `sb.shell.run` | Store secrets in `Image.env()` |-| Use `.env` files + `python-dotenv` locally | Commit `.env` files to version control |-| Use your CI/CD provider's secret manager | Log or print secret values |diff --git a/docs/content/docs/cua/guide/sandbox/snapshots.mdx b/docs/content/docs/cua/guide/sandbox/snapshots.mdxdeleted file mode 100644index 0fef544e71..0000000000--- a/docs/content/docs/cua/guide/sandbox/snapshots.mdx+++ /dev/null@@ -1,116 +0,0 @@-----title: Snapshots-description: Capture sandbox state and boot new sandboxes from snapshots instantly------import { Callout } from 'fumadocs-ui/components/callout';--A snapshot captures the full disk state of a running sandbox. It returns a new `Image` that you can pass to `Sandbox.create()` or `Sandbox.ephemeral()` to boot a fresh sandbox from that point — with all installed packages, files, and configuration intact.--## Quick example--```python-import asyncio-from cua import Sandbox, Image--async def main():- # 1. Create a sandbox and install software- async with Sandbox.ephemeral(Image.linux("ubuntu", "24.04")) as sb:- await sb.shell.run("apt-get update -qq && apt-get install -y -qq cowsay")-- # 2. Snapshot — returns an Image- cowsay_image = await sb.snapshot(name="with-cowsay")-- # 3. Boot from snapshot — cowsay is already installed- async with Sandbox.ephemeral(cowsay_image) as sb2:- result = await sb2.shell.run("/usr/games/cowsay hello from snapshot!")- print(result.stdout)--asyncio.run(main())-```-------## `sb.snapshot()`--```python-image = await sb.snapshot(name="my-snapshot", stateful=False)-```--| Parameter | Type | Default | Description |-|-----------|------|---------|-------------|-| `name` | `str \| None` | `None` | Optional human-readable label for the snapshot |-| `stateful` | `bool` | `False` | Capture memory state in addition to disk (VMs only) |--**Returns:** an `Image` that can be passed directly to `Sandbox.ephemeral()` or `Sandbox.create()`.--<Callout type="info">-Snapshots are currently supported on Cua Cloud sandboxes. Local sandboxes (Docker, Lume, QEMU) have limited support for snapshots.-</Callout>-------## Use cases--### Pre-baked environments--Avoid repeating slow setup steps (package installs, model downloads, database migrations) by snapshotting once and forking many times:--```python-# Build the base image once-async with Sandbox.ephemeral(Image.linux()) as sb:- await sb.shell.run("apt-get update && apt-get install -y python3-pip ffmpeg")- await sb.shell.run("pip install torch transformers")- ml_image = await sb.snapshot(name="ml-ready")--# Fork it for each experiment — no reinstall needed-for lr in [0.001, 0.01, 0.1]:- async with Sandbox.ephemeral(ml_image) as sb:- await sb.shell.run(f"python train.py --lr {lr}")-```--### Android snapshots--Snapshot works with Android sandboxes too. Sideload APKs or configure the emulator once, then fork:--```python-FDROID_APK = "https://f-droid.org/F-Droid.apk"--async with Sandbox.ephemeral(Image.android("14").apk_install(FDROID_APK)) as sb:- fdroid_image = await sb.snapshot(name="with-fdroid")--# Fork — F-Droid is already installed-async with Sandbox.ephemeral(fdroid_image) as sb2:- result = await sb2.shell.run("pm list packages org.fdroid.fdroid")- assert "org.fdroid.fdroid" in result.stdout-```--### State isolation--Each fork gets its own copy of the disk. Changes in one fork never affect another or the original snapshot:--```python-async with Sandbox.ephemeral(Image.linux()) as sb:- await sb.shell.run("echo original > /tmp/data.txt")- base = await sb.snapshot(name="base")--# Fork A writes a file-async with Sandbox.ephemeral(base) as a:- await a.shell.run("echo fork-a >> /tmp/data.txt")--# Fork B doesn't see Fork A's changes-async with Sandbox.ephemeral(base) as b:- result = await b.shell.run("cat /tmp/data.txt")- print(result.stdout) # "original\n" — no trace of fork-a-```-------## Performance--| Operation | Typical time |-|-----------|-------------|-| Create + install packages | 30–120 s |-| Fork from snapshot | 1–5 s |--On backends that use copy-on-write storage, forking is nearly instant regardless of how large the disk image is. On `dir` storage backends, the fork performs a full copy, so it may be slower for large images.diff --git a/docs/content/docs/cua/guide/sandbox/tunneling.mdx b/docs/content/docs/cua/guide/sandbox/tunneling.mdxdeleted file mode 100644index d2694ed3d5..0000000000--- a/docs/content/docs/cua/guide/sandbox/tunneling.mdx+++ /dev/null@@ -1,85 +0,0 @@-----title: Tunneling-description: Forward ports and sockets from a sandbox to your local machine------import { Callout } from 'fumadocs-ui/components/callout';--`sb.tunnel.forward()` exposes a port (or Android abstract socket) running inside a sandbox on your local host. Useful for hitting a web server, connecting a debugger, or scraping CDP targets from a headless browser.--## Basic usage--```python-from cua import Sandbox, Image--async with Sandbox.ephemeral(Image.linux()) as sb:- # Start something inside the sandbox- await sb.shell.run("python3 -m http.server 8080 &")-- async with sb.tunnel.forward(8080) as t:- print(t.url) # http://localhost:49823- print(t.host, t.port) # localhost, 49823- # make requests to t.url from your local machine-```--The tunnel is closed automatically when the `async with` block exits.--## Multiple ports--Pass multiple ports to get a `dict[port, TunnelInfo]`:--```python-async with sb.tunnel.forward(8080, 9222) as tunnels:- app_url = tunnels[8080].url- devtools_url = tunnels[9222].url-```--## Manual close--If you need the tunnel outside a `with` block, `await` it directly and close manually:--```python-t = await sb.tunnel.forward(8080)-print(t.url)-# ... do work ...-await t.close()-```--## Android abstract sockets--For Android sandboxes, pass a socket name string instead of a port number:--```python-async with sb.tunnel.forward("chrome_devtools_remote") as t:- # Connect a CDP client to t.url- print(t.url) # http://localhost:<random>-```--<Callout type="info">- Abstract socket names are Linux-only (Android). On Linux/macOS sandboxes use integer port numbers.-</Callout>--## TunnelInfo fields--| Field | Type | Description |-|---|---|---|-| `host` | `str` | Always `"localhost"` |-| `port` | `int` | The host-side port assigned |-| `sandbox_port` | `int \| str` | The original port/socket inside the sandbox |-| `url` | `str` | Shorthand `http://{host}:{port}` |--## Chrome DevTools example--Forward the DevTools port from a sandbox browser to use CDP tools locally:--```python-from cua import Sandbox, Image--async with Sandbox.ephemeral(Image.linux()) as sb:- await sb.shell.run("google-chrome --remote-debugging-port=9222 --headless &")-- async with sb.tunnel.forward(9222) as t:- import httpx- targets = httpx.get(f"{t.url}/json").json()- print(targets)-```diff --git a/docs/content/docs/cua/meta.json b/docs/content/docs/cua/meta.jsondeleted file mode 100644index 19d9599c54..0000000000--- a/docs/content/docs/cua/meta.json+++ /dev/null@@ -1,5 +0,0 @@-{- "title": "Cua",- "description": "Computer Use Agent SDK",- "pages": ["guide", "examples", "reference"]-}diff --git a/docs/content/docs/cua/reference/agent-sdk/changelog.mdx b/docs/content/docs/cua/reference/agent-sdk/changelog.mdxdeleted file mode 100644index ac90a87067..0000000000--- a/docs/content/docs/cua/reference/agent-sdk/changelog.mdx+++ /dev/null@@ -1,623 +0,0 @@-----title: Changelog-description: Release history for Agent SDK------{/*-AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY-Generated by: npx tsx scripts/docs-generators/generate-changelog.ts-Last updated: 2026-02-04-*/}--# Agent SDK Changelog--All notable changes to the Agent SDK are documented here.--## 0.7.x--### v0.7.24 (2026-02-04)--**Dependencies:** cua-computer: 0.5.12, cua-som: 0.1.3--- Initial release or no path-specific changes found--### v0.7.22 (2026-01-28)--**Dependencies:** cua-computer: 0.5.12, cua-som: 0.1.3--- Bump cua-agent to v0.7.22 by @github-actions[bot]--### v0.7.21 (2026-01-26)--**Dependencies:** cua-computer: 0.5.11, cua-som: 0.1.3--- Bump cua-agent to v0.7.21 by @github-actions[bot]--### v0.7.18 (2026-01-22)--**Dependencies:** cua-computer: 0.5.10, cua-som: 0.1.3--- Bump cua-agent to v0.7.18 by @github-actions[bot]-- fix(agent): accept 'computer_use' as valid function name for Fara model ([#865](https://github.com/trycua/cua/pull/865)) by @sarinali--### v0.7.17 (2026-01-17)--**Dependencies:** cua-computer: 0.5.7, cua-som: 0.1.3--- Bump cua-agent to v0.7.17 by @github-actions[bot]--### v0.7.16 (2026-01-17)--**Dependencies:** cua-computer: 0.5.7, cua-som: 0.1.3--- Bump cua-agent to v0.7.16 by @github-actions[bot]--### v0.7.15 (2026-01-17)--**Dependencies:** cua-computer: 0.5.7, cua-som: 0.1.3--- Bump cua-agent to v0.7.15 by @github-actions[bot]--### v0.7.14 (2026-01-17)--**Dependencies:** cua-computer: 0.5.6, cua-som: 0.1.3--- Bump cua-agent to v0.7.14 by @github-actions[bot]--### v0.7.13 (2026-01-17)--**Dependencies:** cua-computer: 0.5.6, cua-som: 0.1.3--- Bump cua-agent to v0.7.13 by @github-actions[bot]-- Fix/gradio 6.3 compatibility ([#810](https://github.com/trycua/cua/pull/810)) by @Weichen Zhang-- fix(agent): create screenshot_dir when trajectory_dir option is specified ([#813](https://github.com/trycua/cua/pull/813)) by @Harsh Verma-- fix(agent): only yield partial response if not empty ([#802](https://github.com/trycua/cua/pull/802)) by @Harsh Verma-- fix tool mismatch in uitars agent loop ([#640](https://github.com/trycua/cua/pull/640)) by @zju-lx--### v0.7.12 (2026-01-13)--**Dependencies:** cua-computer: 0.5.6, cua-som: 0.1.3--- Bump cua-agent to v0.7.12 by @github-actions[bot]-- pass browser tool ([#774](https://github.com/trycua/cua/pull/774)) by @Sarina Li-- feat(agent): add OpenTelemetry instrumentation callback ([#662](https://github.com/trycua/cua/pull/662)) by @r33drichards-- Fix Omniparser historical message conversion using per-screenshot mappings ([#706](https://github.com/trycua/cua/pull/706)) by @Fizza Mukhtar--### v0.7.11 (2026-01-12)--**Dependencies:** cua-computer: 0.5.6, cua-som: 0.1.3--- Bump cua-agent to v0.7.11 by @github-actions[bot]--### v0.7.10 (2026-01-12)--**Dependencies:** cua-computer: 0.5.6, cua-som: 0.1.3--- Bump cua-agent to v0.7.10 by @github-actions[bot]--### v0.7.9 (2026-01-12)--**Dependencies:** cua-computer: 0.5.5, cua-som: 0.1.3--- Bump cua-agent to v0.7.9 by @github-actions[bot]--### v0.7.8 (2026-01-12)--**Dependencies:** cua-computer: 0.5.4, cua-som: 0.1.3--- Bump cua-agent to v0.7.8 by @github-actions[bot]-- Bump cua-agent to v0.7.7 ([#781](https://github.com/trycua/cua/pull/781)) by @Francesco Bonacci-- Bump cua-agent to v0.7.7 ([#780](https://github.com/trycua/cua/pull/780)) by @Francesco Bonacci--### v0.7.6 (2026-01-12)--**Dependencies:** cua-computer: 0.5.1, cua-som: 0.1.3--- Bump cua-agent to v0.7.6 by @github-actions[bot]--### v0.7.5 (2026-01-12)--**Dependencies:** cua-computer: 0.5.1, cua-som: 0.1.3--- Bump cua-agent to v0.7.5 by @github-actions[bot]--### v0.7.4 (2026-01-12)--**Dependencies:** cua-computer: 0.5.1, cua-som: 0.1.3--- Bump cua-agent to v0.7.4 by @github-actions[bot]-- refactor(docs): reorganize lume/cua docs and standardize READMEs ([#752](https://github.com/trycua/cua/pull/752)) by @Francesco Bonacci-- feat(lume,ci): add unattended VM setup and reorganize CI/CD workflows ([#729](https://github.com/trycua/cua/pull/729)) by @Francesco Bonacci-- Fix linting in cua-bench module ([#727](https://github.com/trycua/cua/pull/727)) by @ddupont-- Raise clear error when using Ollama models with ComputerAgent image inputs ([#711](https://github.com/trycua/cua/pull/711)) by @Fizza Mukhtar-- update dead links due to docs update by @Sarina Li-- add vm name to start/end by @Sarina Li--## 0.4.x--### v0.4.53 (2025-11-19)--cua-agent is a general Computer-Use framework with liteLLM integration for running agentic workflows on macOS, Windows, and Linux sandboxes. It provides a unified interface for computer-use agents across multiple LLM providers with advanced callback system for extensibility.--- Jagjeevan's Fix + Merged Main for 4o Model Fix by @sarinali in https://github.com/trycua/cua/pull/522-- fix: Added GPT-4o compatibility for screenshot actions with text parameter by @JagjeevanAK in https://github.com/trycua/cua/pull/422-- Add test infrastructure with CI/CD #478 by @AceAtDev in https://github.com/trycua/cua/pull/491-- Fix/omniparser predict refactor by @sarinali in https://github.com/trycua/cua/pull/529-- Make VS Code Python interpreter path cross-platform by @skools-here in https://github.com/trycua/cua/pull/520-- Fix: sanitize shell in subprocess calls by @AdityaBavadekar in https://github.com/trycua/cua/pull/519-- [Agent] Add support for overriding api_base and api_url kwargs by @ddupont808 in https://github.com/trycua/cua/pull/504-- Update model list and schedule daily test for agent testing by @YeIIcw in https://github.com/trycua/cua/pull/540-- Add Claude Desktop Extension by @YeIIcw in https://github.com/trycua/cua/pull/521-- [AGENT] - New Model Gelato-30B-A3B added by @tamoghnokandar in https://github.com/trycua/cua/pull/558-- [AGENT] - New model UI-Ins added by @tamoghnokandar in https://github.com/trycua/cua/pull/549-- Add "cua/" LLM provider by @ddupont808 in https://github.com/trycua/cua/pull/506-- Add Local Desktop Mode for MCP Server with updated docs by @YeIIcw in https://github.com/trycua/cua/pull/493-- Fix: Enable custom function tools with Anthropic models by @LucaStngn in https://github.com/trycua/cua/pull/548-- Fix cua adapter, add UI-TARS-2 model by @ddupont808 in https://github.com/trycua/cua/pull/592-- Fix cua adapter for anthropic models by @ddupont808 in https://github.com/trycua/cua/pull/601-- Remove hud-python from cua-agent[all] extra by @f-trycua in https://github.com/trycua/cua/pull/603-- Add automatic CUA_API_KEY environment variable support by @f-trycua in https://github.com/trycua/cua/pull/604--New Contributors--- @AceAtDev made their first contribution in https://github.com/trycua/cua/pull/491-- @skools-here made their first contribution in https://github.com/trycua/cua/pull/520-- @tamoghnokandar made their first contribution in https://github.com/trycua/cua/pull/545-- @dinmukhamedm made their first contribution in https://github.com/trycua/cua/pull/404-- @LucaStngn made their first contribution in https://github.com/trycua/cua/pull/548--### v0.4.35 (2025-10-22)--**Dependencies:** cua-computer: latest, cua-som: latest--- Standardize Python version 3.12 across all packages by @AdityaBavadekar in https://github.com/trycua/cua/pull/500-- Add support for all QwenVL models by @ddupont808 in https://github.com/trycua/cua/pull/501--### v0.4.34 (2025-10-17)--**Dependencies:** cua-computer: latest, cua-som: latest--- Fix torch dependency in moondream3 loop by @ddupont808 in https://github.com/trycua/cua/pull/482--New Contributors--- @sarinali made their first contribution in https://github.com/trycua/cua/pull/477--### v0.4.33 (2025-10-15)--**Dependencies:** cua-computer: latest, cua-som: latest--- Stream MCP responses instead of buffering by @YeIIcw in https://github.com/trycua/cua/pull/427-- Add support for Gemini Cua model by @ddupont808 in https://github.com/trycua/cua/pull/472-- Add support for Claude Haiku 4.5 by @ddupont808 in https://github.com/trycua/cua/pull/474-- Add bump2version configuration for all Python packages by @r33drichards in https://github.com/trycua/cua/pull/463--### v0.4.32 (2025-09-25)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.4.31 (2025-09-13)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.4.30 (2025-09-12)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.4.29 (2025-09-12)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.4.28 (2025-09-12)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.4.27 (2025-09-10)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.4.26 (2025-09-10)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.4.25 (2025-09-05)--**Dependencies:** cua-computer: latest, cua-som: latest--- Change HUD dataset name from `OSWorld-Verified-XLang` to `OSWorld-Verified` by @jamesmurdza in https://github.com/trycua/cua/pull/392-- Fixed error when running agent with multimodal user inputs in the anthropic loop by @ddupont808 in https://github.com/trycua/cua/pull/394-- Reference documentation batch by @onel in https://github.com/trycua/cua/pull/390--### v0.4.24 (2025-08-28)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.4.23 (2025-08-28)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.4.22 (2025-08-28)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.4.21 (2025-08-28)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.4.20 (2025-08-28)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.4.19 (2025-08-28)--**Dependencies:** cua-computer: latest, cua-som: latest--- Added screenshot_dir parameter-- Bugfix: Lazy loading of MLX by @ddupont808 in https://github.com/trycua/cua/pull/373--### v0.4.18 (2025-08-28)--**Dependencies:** cua-computer: latest, cua-som: latest--- Fix example code and notebooks by @jamesmurdza in https://github.com/trycua/cua/pull/364-- Added a readme file to the notebooks folder by @onel in https://github.com/trycua/cua/pull/361-- Restrict root project to Python versions below 3.14 by @jamesmurdza in https://github.com/trycua/cua/pull/363-- Restored `mlx/` adapter by @ddupont808 in https://github.com/trycua/cua/pull/366-- Upgrade HUD SDK to 0.4.12 by @ddupont808 in https://github.com/trycua/cua/pull/371-- Added callback to normalize common tool call hallucinations seen during evals--New Contributors--- @onel made their first contribution in https://github.com/trycua/cua/pull/361--### v0.4.17 (2025-08-19)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.4.16 (2025-08-19)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.4.15 (2025-08-19)--**Dependencies:** cua-computer: latest, cua-som: latest--- Pin OpenAI version to work around BerriAI/litellm#13711 by @jamesmurdza in https://github.com/trycua/cua/pull/356-- Upgrade Claude 3.5 snapshot in web app, examples and docs by @jamesmurdza in https://github.com/trycua/cua/pull/359-- Update agent/computer SDKs to match changes in telemetry SDK by @jamesmurdza in https://github.com/trycua/cua/pull/355--### v0.4.14 (2025-08-18)--**Dependencies:** cua-computer: latest, cua-som: latest--- Upgrade Agent SDK to require Python 3.12 by @jamesmurdza in https://github.com/trycua/cua/pull/342-- Fix UI-TARS predict_click always returning None by @ddupont808 in https://github.com/trycua/cua/pull/350--### v0.4.13 (2025-08-14)--**Dependencies:** cua-computer: latest, cua-som: latest--- [Agent] Add HUD evals, OSWorld-verified docs, and support for custom computers by @ddupont808 in https://github.com/trycua/cua/pull/334--### v0.4.12 (2025-08-12)--**Dependencies:** cua-computer: latest, cua-som: latest--- [Agent] Implement left_mouse_down, left_mouse_up, and tool errors by @ddupont808 in https://github.com/trycua/cua/pull/333-- [Agent] Add GLM-4.5V support by @ddupont808 in https://github.com/trycua/cua/pull/337--### v0.4.11 (2025-08-07)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.4.10 (2025-08-06)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.4.9 (2025-08-06)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.4.8 (2025-08-05)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.4.7 (2025-08-05)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.4.6 (2025-08-04)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.4.5 (2025-07-29)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.4.4 (2025-07-29)--Bugfixes caused by `mouse_move` having the wrong name--### v0.4.3 (2025-07-29)--Bugfixes caused by `mouse_move` having the wrong name--### v0.4.2 (2025-07-29)--- [Agent] Fix Anthropic unexpected tool_use_id by @ddupont808 in https://github.com/trycua/cua/pull/325--### v0.4.1 (2025-07-28)--Updated to use latest `cua-core`--### v0.4.0 (2025-07-28)--This update refactored the Agent SDK to make it easier to implement new features and support the release of new agent models/loops.--Changelog:--- Reworked agent loop, now all agent providers share a loop (Generate, Execute, Repeat), with the only difference between loops being the implementation of the Generate function-- Replaced LLM clients with LiteLLM, now all agent providers support any provider supported by LiteLLM-- Added 2 custom LiteLLM providers for local model inference on CUDA and MLX devices: `huggingface-local/`, `mlx/`-- Reworked callback system to have hooks at every step of the lifecycle-- Converted logging, trajectory saving, image retention into callbacks-- Added new callbacks - PII Anonymization (still a W.I.P) & budget management-- Anthropic providers - Added support for explicit prompt caching-- OpenAI providers - Added support for zero data retention-- Added Agent CLI for quick testing: `python -m agent.cli <model name>`--**Breaking Changes**--- **Initialization:**- - `ComputerAgent` (v0.4.x) uses `model` as a string (e.g. "anthropic/claude-3-5-sonnet-20241022") instead of `LLM` and `AgentLoop` objects.- - `tools` is a list (can include multiple computers and decorated functions).- - `callbacks` are now first-class for extensibility (image retention, budget, trajectory, logging, etc).-- **No explicit `loop` parameter:**- - Loop is inferred from the `model` string (e.g. `anthropic/`, `openai/`, `omniparser+`, `ui-tars`).-- **No explicit `computer` parameter:**- - Computers are added to `tools` list.--Install--```bash-Before merge:-pip install --pre "cua-agent[all]==0.4.0b4"--After merge:-pip install "cua-agent[all]"--or install specific providers-pip install "cua-agent[openai]" # OpenAI computer-use-preview support-pip install "cua-agent[anthropic]" # Anthropic Claude support-pip install "cua-agent[omni]" # Omniparser + any LLM support-pip install "cua-agent[uitars]" # UI-TARS-pip install "cua-agent[uitars-mlx]" # UI-TARS + MLX support-pip install "cua-agent[uitars-hf]" # UI-TARS + Huggingface support-pip install "cua-agent[ui]" # Gradio UI support-```--Supported Models--Anthropic Claude (Computer Use API)--```python-model="anthropic/claude-3-5-sonnet-20241022"-model="anthropic/claude-3-5-sonnet-20240620"-model="anthropic/claude-opus-4-20250514"-model="anthropic/claude-sonnet-4-20250514"-```--OpenAI Computer Use Preview--```python-model="openai/computer-use-preview"-```--UI-TARS (Local or Huggingface Inference)--```python-model="huggingface-local/ByteDance-Seed/UI-TARS-1.5-7B"-model="ollama_chat/0000/ui-tars-1.5-7b"-```--Omniparser + Any LLM--```python-model="omniparser+ollama_chat/mistral-small3.2"-model="omniparser+vertex_ai/gemini-pro"-model="omniparser+anthropic/claude-3-5-sonnet-20241022"-model="omniparser+openai/gpt-4o"-```--## 0.3.x--### v0.3.2 (2025-07-15)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.3.1 (2025-07-01)--**Dependencies:** cua-computer: latest, cua-som: latest--## 0.2.x--### v0.2.15 (2025-06-25)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.2.14 (2025-06-24)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.2.13 (2025-06-24)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.2.12 (2025-06-20)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.2.11 (2025-06-18)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.2.10 (2025-06-10)--**Dependencies:** cua-computer: latest, cua-som: latest--- Fixed image retention bug by @ddupont808 in https://github.com/trycua/cua/pull/282-- Removed `torch` requirement from `cua-agent/core` and `cua-agent/anthropic` by @ddupont808 in https://github.com/trycua/cua/pull/285--### v0.2.9 (2025-06-05)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.2.7 (2025-06-05)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.2.6 (2025-05-28)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.2.5 (2025-05-28)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.2.4 (2025-05-28)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.2.3 (2025-05-28)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.2.2 (2025-05-28)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.2.1 (2025-05-16)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.2.0 (2025-05-14)--**Dependencies:** cua-computer: latest, cua-som: latest--## 0.1.x--### v0.1.44 (2025-05-13)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.1.43 (2025-05-12)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.1.42 (2025-05-11)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.1.41 (2025-05-11)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.1.40 (2025-05-11)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.1.39 (2025-05-11)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.1.38 (2025-05-11)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.1.37 (2025-05-10)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.1.35 (2025-05-10)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.1.34 (2025-05-10)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.1.33 (2025-05-10)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.1.32 (2025-05-02)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.1.31 (2025-04-29)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.1.30 (2025-04-24)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.1.29 (2025-04-22)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.1.28 (2025-04-15)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.1.27 (2025-04-15)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.1.26 (2025-04-14)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.1.25 (2025-04-06)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.1.24 (2025-04-06)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.1.23 (2025-04-06)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.1.22 (2025-04-04)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.1.21 (2025-04-02)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.1.20 (2025-03-30)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.1.19 (2025-03-30)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.1.18 (2025-03-30)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.1.17 (2025-03-24)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.1.6 (2025-03-19)--**Dependencies:** cua-computer: latest, cua-som: latest--### v0.1.5 (2025-03-17)--**Dependencies:** cua-computer: latest, cua-som: latestdiff --git a/docs/content/docs/cua/reference/agent-sdk/index.mdx b/docs/content/docs/cua/reference/agent-sdk/index.mdxdeleted file mode 100644index ece0060541..0000000000--- a/docs/content/docs/cua/reference/agent-sdk/index.mdx+++ /dev/null@@ -1,1357 +0,0 @@-----title: Agent SDK API Reference-description: Python API reference for building computer-use agents------{/*- AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY- Generated by: npx tsx scripts/docs-generators/python-sdk.ts- Source: libs/python/agent/agent- Version: 0.7.36-*/}--import { Callout } from 'fumadocs-ui/components/callout';-import { Tabs, Tab } from 'fumadocs-ui/components/tabs';-import { VersionHeader } from '@/components/version-selector';--<VersionHeader- versions={[{"version":"0.7","href":"/cua/reference/agent-sdk","isCurrent":true},{"version":"0.6","href":"/cua/reference/agent-sdk/v0.6/api","isCurrent":false},{"version":"0.5","href":"/cua/reference/agent-sdk/v0.5/api","isCurrent":false},{"version":"0.4","href":"/cua/reference/agent-sdk/v0.4/api","isCurrent":false},{"version":"0.3","href":"/cua/reference/agent-sdk/v0.3/api","isCurrent":false}]}- currentVersion="0.7"- fullVersion="0.7.36"- packageName="cua-agent"-/>--agent - Decorator-based Computer Use Agent with liteLLM integration--## Classes--| Class | Description |-|-------|-------------|-| [`ComputerAgent`](#computeragent) | Main agent class that automatically selects the appropriate agent loop |--## Functions--| Function | Description |-|----------|-------------|-| [`register_agent`](#register_agent) | Decorator to register an AsyncAgentConfig class. |-------## ComputerAgent--Main agent class that automatically selects the appropriate agent loop-based on the model and executes tool calls.--### Constructor--```python-ComputerAgent(self, model: str, tools: Optional[List[Any]] = None, custom_loop: Optional[Callable] = None, only_n_most_recent_images: Optional[int] = None, callbacks: Optional[List[Any]] = None, instructions: Optional[str] = None, verbosity: Optional[int] = None, trajectory_dir: Optional[str | Path | dict] = None, max_retries: Optional[int] = 3, screenshot_delay: Optional[float | int] = 0.5, use_prompt_caching: Optional[bool] = False, max_trajectory_budget: Optional[float | dict] = None, telemetry_enabled: Optional[bool] = True, trust_remote_code: Optional[bool] = False, api_key: Optional[str] = None, api_base: Optional[str] = None, additional_generation_kwargs = {})-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `model` | `Any` | |-| `tools` | `Any` | |-| `custom_loop` | `Any` | |-| `only_n_most_recent_images` | `Any` | |-| `callbacks` | `Any` | |-| `instructions` | `Any` | |-| `verbosity` | `Any` | |-| `trajectory_dir` | `Any` | |-| `max_retries` | `Any` | |-| `screenshot_delay` | `Any` | |-| `use_prompt_caching` | `Any` | |-| `telemetry_enabled` | `Any` | |-| `kwargs` | `Any` | |-| `trust_remote_code` | `Any` | |-| `api_key` | `Any` | |-| `api_base` | `Any` | |-| `agent_loop` | `Any` | |-| `agent_config_info` | `Any` | |-| `tool_schemas` | `Any` | |-| `computer_handler` | `Any` | |--### Methods--#### ComputerAgent.run--```python-async def run(self, messages: Messages, stream: bool = False, api_key: Optional[str] = None, api_base: Optional[str] = None, additional_generation_kwargs = {}) -> AsyncGenerator[Dict[str, Any], None]-```--Run the agent with the given messages using Computer protocol handler pattern.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `messages` | `Any` | List of message dictionaries |-| `stream` | `Any` | Whether to stream the response |-| `api_key` | `Any` | Optional API key override for the model provider |-| `api_base` | `Any` | Optional API base URL override for the model provider **additional_generation_kwargs: Additional arguments passed to the model provider |--**Returns:** AsyncGenerator that yields response chunks--#### ComputerAgent.predict_click--```python-async def predict_click(self, instruction: str, image_b64: Optional[str] = None) -> Optional[Tuple[int, int]]-```--Predict click coordinates based on image and instruction.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `instruction` | `Any` | Instruction for where to click |-| `image_b64` | `Any` | Base64 encoded image (optional, will take screenshot if not provided) |--**Returns:** None or tuple with (x, y) coordinates--#### ComputerAgent.get_capabilities--```python-def get_capabilities(self) -> List[AgentCapability]-```--Get list of capabilities supported by the current agent config.--**Returns:** List of capability strings (e.g., ["step", "click"])--#### ComputerAgent.open--```python-def open(self, port: Optional[int] = None)-```--Start the playground server and open it in the browser.--This method starts a local HTTP server that exposes the /responses endpoint-and automatically opens the Cua playground interface in the default browser.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `port` | `Any` | Port to run the server on. If None, finds an available port automatically. |--**Example:**--```python->>> agent = ComputerAgent(model="claude-sonnet-4")->>> agent.open() # Starts server and opens browser-```--## register_agent--```python-def register_agent(models: str, priority: int = 0, tool_type: Optional[str] = None)-```--Decorator to register an AsyncAgentConfig class.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `models` | `Any` | Regex pattern to match supported models |-| `priority` | `Any` | Priority for agent selection (higher = more priority) |-| `tool_type` | `Any` | Required tool type for this model ("browser" | "mobile" | None). Specialized models (like FARA) declare their required tool type, and ComputerAgent will auto-wrap tools accordingly. General models (like Claude) leave this as None for full flexibility. |-------## types--Type definitions for agent-------## ToolError--*Inherits from: RuntimeError*--Base exception for tool-related errors-------## IllegalArgumentError--*Inherits from: ToolError*--Exception raised when function arguments are invalid-------## AgentConfigInfo--*Inherits from: BaseModel*--Information about a registered agent config--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `agent_class` | `type` | |-| `models_regex` | `str` | |-| `priority` | `int` | |-| `tool_type` | `Optional[str]` | |--### Methods--#### AgentConfigInfo.matches_model--```python-def matches_model(self, model: str) -> bool-```--Check if this agent config matches the given model-------## tools--Agent tools module.-Provides base classes and registered tools for agent interactions.-------## BaseComputerTool--*Inherits from: BaseTool*--Base class for computer tools that can provide screenshots.--Computer tools must implement:-- All BaseTool requirements (name, description, parameters, call)-- screenshot() method that returns screenshot as base64 string--### Methods--#### BaseComputerTool.screenshot--```python-async def screenshot(self) -> str-```--Take a screenshot of the computer/browser.--**Returns:** Screenshot image data as base64-encoded string-------## BaseTool--*Inherits from: ABC*--Base class for all agent tools.--Tools must implement:-- name: str - The tool name (set by @register_tool decorator)-- description: property that returns str - Tool description-- parameters: property that returns dict - JSON schema for tool parameters-- call: method - Execute the tool with given parameters--### Constructor--```python-BaseTool(self, cfg: Optional[dict] = None)-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `name` | `str` | |-| `cfg` | `Any` | |-| `description` | `str` | Return the tool description. |-| `parameters` | `dict` | Return the JSON schema for tool parameters. |-| `function` | `dict` | Return the function information for this tool. |--### Methods--#### BaseTool.call--```python-def call(self, params: Union[str, dict], kwargs = {}) -> Union[str, list, dict]-```--Execute the tool with the given parameters.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `params` | `Any` | The parameters for the tool call (JSON string or dict) **kwargs: Additional keyword arguments |--**Returns:** The result of the tool execution-------## BrowserTool--*Inherits from: BaseComputerTool*--Browser tool that uses the computer SDK's interface to control a browser.-Implements a comprehensive computer_use action interface for browser control.--### Constructor--```python-BrowserTool(self, interface: GenericComputerInterface, cfg: Optional[dict] = None)-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `interface` | `Any` | |-| `viewport_width` | `Any` | |-| `viewport_height` | `Any` | |-| `resized_width` | `Any` | |-| `resized_height` | `Any` | |-| `automation` | `Any` | Get the automation interface for keyboard/mouse actions. |-| `description` | `str` | |-| `parameters` | `dict` | |--### Methods--#### BrowserTool.call--```python-def call(self, params: Union[str, dict], kwargs = {}) -> Union[str, dict]-```--Execute a browser action.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `params` | `Any` | Action parameters (JSON string or dict) **kwargs: Additional keyword arguments |--**Returns:** Result of the action execution--#### BrowserTool.visit_url--```python-async def visit_url(self, url: str) -> dict-```--Navigate to a URL.--#### BrowserTool.click--```python-async def click(self, x: int = None, y: int = None, button: str = 'left', kwargs = {}) -> dict-```--Click at coordinates. Supports both positional (x, y) and kwargs (button, x, y).--This is compatible with the normalized format from OperatorNormalizerCallback-which transforms actions like \{"type": "left_click", "coordinate": [x, y]\}-into \{"type": "click", "button": "left", "x": x, "y": y\}.--#### BrowserTool.type--```python-async def type(self, text: str) -> dict-```--Type text into the focused element.--#### BrowserTool.scroll--```python-async def scroll(self, delta_x: int = None, delta_y: int = None, scroll_x: int = None, scroll_y: int = None, x: int = None, y: int = None, pixels: int = None, coordinate = None, kwargs = {}) -> dict-```--Scroll the page. Supports multiple formats:-- Legacy: scroll(delta_x, delta_y)-- Normalized: scroll(scroll_x=0, scroll_y=100, x=500, y=300)-- FARA: scroll(pixels=100, coordinate=[500, 300])--#### BrowserTool.web_search--```python-async def web_search(self, query: str) -> dict-```--Navigate to a Google search for the query.--#### BrowserTool.screenshot--```python-async def screenshot(self) -> str-```--Take a screenshot of the current browser page.--#### BrowserTool.get_current_url--```python-async def get_current_url(self) -> str-```--Get the current URL of the browser page.--#### BrowserTool.left_click--```python-async def left_click(self, coordinate = None, x: int = None, y: int = None, kwargs = {}) -> dict-```--Left click at coordinates. Supports coordinate array or x/y kwargs.--#### BrowserTool.right_click--```python-async def right_click(self, coordinate = None, x: int = None, y: int = None, kwargs = {}) -> dict-```--Right click at coordinates. Supports coordinate array or x/y kwargs.--#### BrowserTool.middle_click--```python-async def middle_click(self, coordinate = None, x: int = None, y: int = None, kwargs = {}) -> dict-```--Middle click at coordinates. Supports coordinate array or x/y kwargs.--#### BrowserTool.double_click--```python-async def double_click(self, coordinate = None, x: int = None, y: int = None, kwargs = {}) -> dict-```--Double click at coordinates. Supports coordinate array or x/y kwargs.--#### BrowserTool.triple_click--```python-async def triple_click(self, coordinate = None, x: int = None, y: int = None, button: str = None, kwargs = {}) -> dict-```--Triple click at coordinates. Supports coordinate array or x/y kwargs.--#### BrowserTool.mouse_move--```python-async def mouse_move(self, coordinate = None, x: int = None, y: int = None, kwargs = {}) -> dict-```--Move mouse to coordinates. Supports coordinate array or x/y kwargs.--#### BrowserTool.move--```python-async def move(self, x: int = None, y: int = None, kwargs = {}) -> dict-```--Move mouse to coordinates. Alias for mouse_move with x/y kwargs.--#### BrowserTool.left_click_drag--```python-async def left_click_drag(self, coordinate = None, start_coordinate = None, end_coordinate = None, kwargs = {}) -> dict-```--Drag from start to end coordinates. FARA-compatible.--#### BrowserTool.key--```python-async def key(self, keys = None, kwargs = {}) -> dict-```--Press keys. FARA-compatible.--#### BrowserTool.keypress--```python-async def keypress(self, keys = None, kwargs = {}) -> dict-```--Press keys. Alias for key() - used by OperatorNormalizerCallback.--#### BrowserTool.hscroll--```python-async def hscroll(self, pixels = None, coordinate = None, kwargs = {}) -> dict-```--Horizontal scroll. FARA-compatible.--#### BrowserTool.wait--```python-async def wait(self, time = None, kwargs = {}) -> dict-```--Wait for specified seconds. FARA-compatible.--#### BrowserTool.history_back--```python-async def history_back(self, kwargs = {}) -> dict-```--Go back in browser history. FARA-compatible.--#### BrowserTool.terminate--```python-async def terminate(self, status = None, kwargs = {}) -> dict-```--Terminate and report status. FARA-compatible.--### get_registered_tools--```python-def get_registered_tools() -> Dict[str, type]-```--Get all registered tools.--**Returns:** Dictionary mapping tool names to tool classes--### get_tool--```python-def get_tool(name: str) -> Optional[type]-```--Get a registered tool by name.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `name` | `Any` | The tool name |--**Returns:** The tool class, or None if not found--### register_tool--```python-def register_tool(name: str, allow_overwrite: bool = False)-```--Decorator to register a tool class with a given name.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `name` | `Any` | The name to register the tool under |-| `allow_overwrite` | `Any` | Whether to allow overwriting an existing tool with the same name |--**Returns:** Decorator function that registers the class--**Example:**--```python-@register_tool("my_tool")-class MyTool(BaseTool):- ...-```-------## callbacks--Callback system for ComputerAgent preprocessing and postprocessing hooks.-------## AsyncCallbackHandler--*Inherits from: ABC*--Base class for async callback handlers that can preprocess messages before-the agent loop and postprocess output after the agent loop.--### Methods--#### AsyncCallbackHandler.on_run_start--```python-async def on_run_start(self, kwargs: Dict[str, Any], old_items: List[Dict[str, Any]]) -> None-```--Called at the start of an agent run loop.--#### AsyncCallbackHandler.on_run_end--```python-async def on_run_end(self, kwargs: Dict[str, Any], old_items: List[Dict[str, Any]], new_items: List[Dict[str, Any]]) -> None-```--Called at the end of an agent run loop.--#### AsyncCallbackHandler.on_run_continue--```python-async def on_run_continue(self, kwargs: Dict[str, Any], old_items: List[Dict[str, Any]], new_items: List[Dict[str, Any]]) -> bool-```--Called during agent run loop to determine if execution should continue.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `kwargs` | `Any` | Run arguments |-| `old_items` | `Any` | Original messages |-| `new_items` | `Any` | New messages generated during run |--**Returns:** True to continue execution, False to stop--#### AsyncCallbackHandler.on_llm_start--```python-async def on_llm_start(self, messages: List[Dict[str, Any]]) -> List[Dict[str, Any]]-```--Called before messages are sent to the agent loop.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `messages` | `Any` | List of message dictionaries to preprocess |--**Returns:** List of preprocessed message dictionaries--#### AsyncCallbackHandler.on_llm_end--```python-async def on_llm_end(self, output: List[Dict[str, Any]]) -> List[Dict[str, Any]]-```--Called after the agent loop returns output.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `output` | `Any` | List of output message dictionaries to postprocess |--**Returns:** List of postprocessed output dictionaries--#### AsyncCallbackHandler.on_computer_call_start--```python-async def on_computer_call_start(self, item: Dict[str, Any]) -> None-```--Called when a computer call is about to start.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `item` | `Any` | The computer call item dictionary |--#### AsyncCallbackHandler.on_computer_call_end--```python-async def on_computer_call_end(self, item: Dict[str, Any], result: List[Dict[str, Any]]) -> None-```--Called when a computer call has completed.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `item` | `Any` | The computer call item dictionary |-| `result` | `Any` | The result of the computer call |--#### AsyncCallbackHandler.on_function_call_start--```python-async def on_function_call_start(self, item: Dict[str, Any]) -> None-```--Called when a function call is about to start.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `item` | `Any` | The function call item dictionary |--#### AsyncCallbackHandler.on_function_call_end--```python-async def on_function_call_end(self, item: Dict[str, Any], result: List[Dict[str, Any]]) -> None-```--Called when a function call has completed.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `item` | `Any` | The function call item dictionary |-| `result` | `Any` | The result of the function call |--#### AsyncCallbackHandler.on_text--```python-async def on_text(self, item: Dict[str, Any]) -> None-```--Called when a text message is encountered.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `item` | `Any` | The message item dictionary |--#### AsyncCallbackHandler.on_api_start--```python-async def on_api_start(self, kwargs: Dict[str, Any]) -> None-```--Called when an API call is about to start.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `kwargs` | `Any` | The kwargs being passed to the API call |--#### AsyncCallbackHandler.on_api_end--```python-async def on_api_end(self, kwargs: Dict[str, Any], result: Any) -> None-```--Called when an API call has completed.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `kwargs` | `Any` | The kwargs that were passed to the API call |-| `result` | `Any` | The result of the API call |--#### AsyncCallbackHandler.on_usage--```python-async def on_usage(self, usage: Dict[str, Any]) -> None-```--Called when usage information is received.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `usage` | `Any` | The usage information |--#### AsyncCallbackHandler.on_screenshot--```python-async def on_screenshot(self, screenshot: Union[str, bytes], name: str = 'screenshot') -> None-```--Called when a screenshot is taken.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `screenshot` | `Any` | The screenshot image |-| `name` | `Any` | The name of the screenshot |--#### AsyncCallbackHandler.on_responses--```python-async def on_responses(self, kwargs: Dict[str, Any], responses: Dict[str, Any]) -> None-```--Called when responses are received.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `kwargs` | `Any` | The kwargs being passed to the agent loop |-| `responses` | `Any` | The responses received |-------## BudgetManagerCallback--*Inherits from: AsyncCallbackHandler*--Budget manager callback that tracks usage costs and can stop execution when budget is exceeded.--### Constructor--```python-BudgetManagerCallback(self, max_budget: float, reset_after_each_run: bool = True, raise_error: bool = False)-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `max_budget` | `Any` | |-| `reset_after_each_run` | `Any` | |-| `raise_error` | `Any` | |-| `total_cost` | `Any` | |--### Methods--#### BudgetManagerCallback.on_run_start--```python-async def on_run_start(self, kwargs: Dict[str, Any], old_items: List[Dict[str, Any]]) -> None-```--Reset budget if configured to do so.--#### BudgetManagerCallback.on_usage--```python-async def on_usage(self, usage: Dict[str, Any]) -> None-```--Track usage costs.--#### BudgetManagerCallback.on_run_continue--```python-async def on_run_continue(self, kwargs: Dict[str, Any], old_items: List[Dict[str, Any]], new_items: List[Dict[str, Any]]) -> bool-```--Check if budget allows continuation.-------## ImageRetentionCallback--*Inherits from: AsyncCallbackHandler*--Callback handler that applies image retention policy to limit the number-of recent images in message history to prevent context window overflow.--### Constructor--```python-ImageRetentionCallback(self, only_n_most_recent_images: Optional[int] = None)-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `only_n_most_recent_images` | `Any` | |--### Methods--#### ImageRetentionCallback.on_llm_start--```python-async def on_llm_start(self, messages: List[Dict[str, Any]]) -> List[Dict[str, Any]]-```--Apply image retention policy to messages before sending to agent loop.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `messages` | `Any` | List of message dictionaries |--**Returns:** List of messages with image retention policy applied-------## LoggingCallback--*Inherits from: AsyncCallbackHandler*--Callback handler that logs agent lifecycle events with configurable verbosity.--Logging levels:-- DEBUG: All events including API calls, message preprocessing, and detailed outputs-- INFO: Major lifecycle events (start/end, messages, outputs)-- WARNING: Only warnings and errors-- ERROR: Only errors--### Constructor--```python-LoggingCallback(self, logger: Optional[logging.Logger] = None, level: int = logging.INFO)-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `logger` | `Any` | |-| `level` | `Any` | |--### Methods--#### LoggingCallback.on_run_start--```python-async def on_run_start(self, kwargs: Dict[str, Any], old_items: List[Dict[str, Any]]) -> None-```--Called before the run starts.--#### LoggingCallback.on_usage--```python-async def on_usage(self, usage: Dict[str, Any]) -> None-```--Called when usage information is received.--#### LoggingCallback.on_run_end--```python-async def on_run_end(self, kwargs: Dict[str, Any], old_items: List[Dict[str, Any]], new_items: List[Dict[str, Any]]) -> None-```--Called after the run ends.--#### LoggingCallback.on_llm_start--```python-async def on_llm_start(self, messages: List[Dict[str, Any]]) -> List[Dict[str, Any]]-```--Called before LLM processing starts.--#### LoggingCallback.on_llm_end--```python-async def on_llm_end(self, messages: List[Dict[str, Any]]) -> List[Dict[str, Any]]-```--Called after LLM processing ends.--#### LoggingCallback.on_computer_call_start--```python-async def on_computer_call_start(self, item: Dict[str, Any]) -> None-```--Called when a computer call starts.--#### LoggingCallback.on_computer_call_end--```python-async def on_computer_call_end(self, item: Dict[str, Any], result: Any) -> None-```--Called when a computer call ends.--#### LoggingCallback.on_function_call_start--```python-async def on_function_call_start(self, item: Dict[str, Any]) -> None-```--Called when a function call starts.--#### LoggingCallback.on_function_call_end--```python-async def on_function_call_end(self, item: Dict[str, Any], result: Any) -> None-```--Called when a function call ends.--#### LoggingCallback.on_text--```python-async def on_text(self, item: Dict[str, Any]) -> None-```--Called when a text message is encountered.--#### LoggingCallback.on_api_start--```python-async def on_api_start(self, kwargs: Dict[str, Any]) -> None-```--Called when an API call is about to start.--#### LoggingCallback.on_api_end--```python-async def on_api_end(self, kwargs: Dict[str, Any], result: Any) -> None-```--Called when an API call has completed.--#### LoggingCallback.on_screenshot--```python-async def on_screenshot(self, item: Union[str, bytes], name: str = 'screenshot') -> None-```--Called when a screenshot is taken.-------## OperatorNormalizerCallback--*Inherits from: AsyncCallbackHandler*--Normalizes common computer call hallucinations / errors in computer call syntax.--### Methods--#### OperatorNormalizerCallback.on_llm_end--```python-async def on_llm_end(self, output: List[Dict[str, Any]]) -> List[Dict[str, Any]]-```-------## OtelCallback--*Inherits from: AsyncCallbackHandler*--OpenTelemetry callback handler for instrumentation.--Tracks:-- Agent session lifecycle (start/end)-- Agent run lifecycle (start/end with duration)-- Individual steps (with duration)-- Computer actions (with duration)-- Token usage-- Errors--### Constructor--```python-OtelCallback(self, agent: Any)-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `agent` | `Any` | |-| `model` | `Any` | |-| `run_start_time` | `Optional[float]` | |-| `step_start_time` | `Optional[float]` | |-| `step_count` | `Any` | |--### Methods--#### OtelCallback.on_run_start--```python-async def on_run_start(self, kwargs: Dict[str, Any], old_items: List[Dict[str, Any]]) -> None-```--Called at the start of an agent run loop.--#### OtelCallback.on_run_end--```python-async def on_run_end(self, kwargs: Dict[str, Any], old_items: List[Dict[str, Any]], new_items: List[Dict[str, Any]]) -> None-```--Called at the end of an agent run loop.--#### OtelCallback.on_responses--```python-async def on_responses(self, kwargs: Dict[str, Any], responses: Dict[str, Any]) -> None-```--Called when responses are received (each step).--#### OtelCallback.on_usage--```python-async def on_usage(self, usage: Dict[str, Any]) -> None-```--Called when usage information is received.--#### OtelCallback.on_computer_call_start--```python-async def on_computer_call_start(self, item: Dict[str, Any]) -> None-```--Called when a computer call is about to start.--#### OtelCallback.on_computer_call_end--```python-async def on_computer_call_end(self, item: Dict[str, Any], result: List[Dict[str, Any]]) -> None-```--Called when a computer call has completed.--#### OtelCallback.on_api_start--```python-async def on_api_start(self, kwargs: Dict[str, Any]) -> None-```--Called when an LLM API call is about to start.--#### OtelCallback.on_api_end--```python-async def on_api_end(self, kwargs: Dict[str, Any], result: Any) -> None-```--Called when an LLM API call has completed.-------## OtelErrorCallback--*Inherits from: AsyncCallbackHandler*--Callback that captures errors and sends them to OTEL.--Should be added early in the callback chain to catch all errors.--### Constructor--```python-OtelErrorCallback(self, agent: Any)-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `agent` | `Any` | |-| `model` | `Any` | |--### Methods--#### OtelErrorCallback.on_error--```python-async def on_error(self, error: Exception, context: Dict[str, Any]) -> None-```--Called when an error occurs during agent execution.-------## PromptInstructionsCallback--*Inherits from: AsyncCallbackHandler*--Prepend a user instructions message to the message list.--This is a minimal, non-invasive way to guide the agent's behavior without-modifying agent loops or tools. It works with any provider/loop since it-only alters the messages array before sending to the model.--### Constructor--```python-PromptInstructionsCallback(self, instructions: Optional[str]) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `instructions` | `Any` | |--### Methods--#### PromptInstructionsCallback.on_llm_start--```python-async def on_llm_start(self, messages: List[Dict[str, Any]]) -> List[Dict[str, Any]]-```-------## TelemetryCallback--*Inherits from: AsyncCallbackHandler*--Telemetry callback handler for Computer-Use Agent (cua-agent)--Tracks agent usage, performance metrics, and optionally trajectory data.--### Constructor--```python-TelemetryCallback(self, agent, log_trajectory: bool = False)-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `agent` | `Any` | |-| `log_trajectory` | `Any` | |-| `session_id` | `Any` | |-| `run_id` | `Any` | |-| `run_start_time` | `Any` | |-| `step_count` | `Any` | |-| `step_start_time` | `Any` | |-| `total_usage` | `Any` | |--### Methods--#### TelemetryCallback.on_run_start--```python-async def on_run_start(self, kwargs: Dict[str, Any], old_items: List[Dict[str, Any]]) -> None-```--Called at the start of an agent run loop.--#### TelemetryCallback.on_run_end--```python-async def on_run_end(self, kwargs: Dict[str, Any], old_items: List[Dict[str, Any]], new_items: List[Dict[str, Any]]) -> None-```--Called at the end of an agent run loop.--#### TelemetryCallback.on_usage--```python-async def on_usage(self, usage: Dict[str, Any]) -> None-```--Called when usage information is received.--#### TelemetryCallback.on_responses--```python-async def on_responses(self, kwargs: Dict[str, Any], responses: Dict[str, Any]) -> None-```--Called when responses are received.-------## TrajectorySaverCallback--*Inherits from: AsyncCallbackHandler*--Callback handler that saves agent trajectories to disk.--Saves each run as a separate trajectory with unique ID, and each turn-within the trajectory gets its own folder with screenshots and responses.--### Constructor--```python-TrajectorySaverCallback(self, trajectory_dir: str, reset_on_run: bool = True, screenshot_dir: Optional[str] = None)-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `trajectory_dir` | `Any` | |-| `trajectory_id` | `Optional[str]` | |-| `current_turn` | `int` | |-| `current_artifact` | `int` | |-| `model` | `Optional[str]` | |-| `total_usage` | `Dict[str, Any]` | |-| `reset_on_run` | `Any` | |-| `screenshot_dir` | `Optional[Path]` | |--### Methods--#### TrajectorySaverCallback.on_run_start--```python-async def on_run_start(self, kwargs: Dict[str, Any], old_items: List[Dict[str, Any]]) -> None-```--Initialize trajectory tracking for a new run.--#### TrajectorySaverCallback.on_run_end--```python-async def on_run_end(self, kwargs: Dict[str, Any], old_items: List[Dict[str, Any]], new_items: List[Dict[str, Any]]) -> None-```--Finalize run tracking by updating metadata with completion status, usage, and new items.--#### TrajectorySaverCallback.on_api_start--```python-async def on_api_start(self, kwargs: Dict[str, Any]) -> None-```--#### TrajectorySaverCallback.on_api_end--```python-async def on_api_end(self, kwargs: Dict[str, Any], result: Any) -> None-```--Save API call result.--#### TrajectorySaverCallback.on_screenshot--```python-async def on_screenshot(self, screenshot: Union[str, bytes], name: str = 'screenshot') -> None-```--Save a screenshot.--#### TrajectorySaverCallback.on_usage--```python-async def on_usage(self, usage: Dict[str, Any]) -> None-```--Called when usage information is received.--#### TrajectorySaverCallback.on_responses--```python-async def on_responses(self, kwargs: Dict[str, Any], responses: Dict[str, Any]) -> None-```--Save responses to the current turn directory and update usage statistics.--#### TrajectorySaverCallback.on_computer_call_end--```python-async def on_computer_call_end(self, item: Dict[str, Any], result: List[Dict[str, Any]]) -> None-```--Called when a computer call has completed.-Saves screenshots and computer call output.--#### TrajectorySaverCallback.on_function_call_end--```python-async def on_function_call_end(self, item: Dict[str, Any], result: List[Dict[str, Any]]) -> None-```--Called when a function call has completed.-Saves screenshots and function call output for GPT 5.4 / BrowserTool.diff --git a/docs/content/docs/cua/reference/agent-sdk/meta.json b/docs/content/docs/cua/reference/agent-sdk/meta.jsondeleted file mode 100644index 0d3799c957..0000000000--- a/docs/content/docs/cua/reference/agent-sdk/meta.json+++ /dev/null@@ -1,6 +0,0 @@-{- "title": "Agent SDK",- "description": "Python API for building computer-use agents",- "icon": "Bot",- "pages": ["changelog"]-}diff --git a/docs/content/docs/cua/reference/agent-sdk/v0.3/api.mdx b/docs/content/docs/cua/reference/agent-sdk/v0.3/api.mdxdeleted file mode 100644index f507faa81f..0000000000--- a/docs/content/docs/cua/reference/agent-sdk/v0.3/api.mdx+++ /dev/null@@ -1,70 +0,0 @@-----title: Agent SDK v0.3 API Reference-description: API reference for Agent SDK version 0.3------{/*-AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY-Generated by: npx tsx scripts/docs-generators/generate-versioned-docs.ts-Source tag: agent-v0.3.2-Version: 0.3.2-*/}--import { Callout } from 'fumadocs-ui/components/callout';--<Callout type="warn">- This is documentation for **v0.3**. [View latest version](/cua/reference/agent-sdk).-</Callout>--<div className="flex items-center gap-2 mb-6">- <span className="px-2 py-1 bg-amber-100 dark:bg-amber-900 text-amber-800 dark:text-amber-200 rounded text-sm font-mono">- v0.3.2- </span>- <span className="text-sm text-fd-muted-foreground">pip install cua-agent==0.3.2</span>-</div>--Cua (Computer Use) Agent for AI-driven computer interaction.--## Classes--| Class | Description |-| --------------- | -------------------------------------------------------------------------------------- |-| `LLMProvider` | Supported LLM providers. |-| `LLM` | Configuration for LLM model and provider. |-| `ComputerAgent` | A computer agent that can perform automated tasks using natural language instructions. |--## LLMProvider--Supported LLM providers.--## LLM--Configuration for LLM model and provider.--## ComputerAgent--A computer agent that can perform automated tasks using natural language instructions.--### Methods--#### ComputerAgent.initialize--```python-async def initialize(self) -> None-```--Initialize the agent and its components.--#### ComputerAgent.run--```python-async def run(self, task: str) -> AsyncGenerator[AgentResponse, None]-```--Run a task using the computer agent.--Args:-task: Task description--Yields:-Agent response formatdiff --git a/docs/content/docs/cua/reference/agent-sdk/v0.3/meta.json b/docs/content/docs/cua/reference/agent-sdk/v0.3/meta.jsondeleted file mode 100644index 4f4b3dc2bf..0000000000--- a/docs/content/docs/cua/reference/agent-sdk/v0.3/meta.json+++ /dev/null@@ -1,5 +0,0 @@-{- "title": "v0.3",- "description": "Agent SDK v0.3 API Reference",- "pages": ["api"]-}diff --git a/docs/content/docs/cua/reference/agent-sdk/v0.4/api.mdx b/docs/content/docs/cua/reference/agent-sdk/v0.4/api.mdxdeleted file mode 100644index b1fbe0fefd..0000000000--- a/docs/content/docs/cua/reference/agent-sdk/v0.4/api.mdx+++ /dev/null@@ -1,83 +0,0 @@-----title: Agent SDK v0.4 API Reference-description: API reference for Agent SDK version 0.4------{/*-AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY-Generated by: npx tsx scripts/docs-generators/generate-versioned-docs.ts-Source tag: agent-v0.4.53-Version: 0.4.53-*/}--import { Callout } from 'fumadocs-ui/components/callout';--<Callout type="warn">- This is documentation for **v0.4**. [View latest version](/cua/reference/agent-sdk).-</Callout>--<div className="flex items-center gap-2 mb-6">- <span className="px-2 py-1 bg-amber-100 dark:bg-amber-900 text-amber-800 dark:text-amber-200 rounded text-sm font-mono">- v0.4.53- </span>- <span className="text-sm text-fd-muted-foreground">pip install cua-agent==0.4.53</span>-</div>--agent - Decorator-based Computer Use Agent with liteLLM integration--## Classes--| Class | Description |-| --------------- | ---------------------------------------------------------------------- |-| `ComputerAgent` | Main agent class that automatically selects the appropriate agent loop |--## ComputerAgent--Main agent class that automatically selects the appropriate agent loop-based on the model and executes tool calls.--### Methods--#### ComputerAgent.run--```python-async def run(self, messages: Messages, stream: bool = False, api_key: Optional[str] = None, api_base: Optional[str] = None, additional_generation_kwargs = {}) -> AsyncGenerator[Dict[str, Any], None]-```--Run the agent with the given messages using Computer protocol handler pattern.--Args:-messages: List of message dictionaries-stream: Whether to stream the response-api_key: Optional API key override for the model provider-api_base: Optional API base URL override for the model provider-\*\*additional_generation_kwargs: Additional arguments passed to the model provider--Returns:-AsyncGenerator that yields response chunks--#### ComputerAgent.predict_click--```python-async def predict_click(self, instruction: str, image_b64: Optional[str] = None) -> Optional[Tuple[int, int]]-```--Predict click coordinates based on image and instruction.--Args:-instruction: Instruction for where to click-image_b64: Base64 encoded image (optional, will take screenshot if not provided)--Returns:-None or tuple with (x, y) coordinates--#### ComputerAgent.get_capabilities--```python-def get_capabilities(self) -> List[AgentCapability]-```--Get list of capabilities supported by the current agent config.--Returns:-List of capability strings (e.g., ["step", "click"])diff --git a/docs/content/docs/cua/reference/agent-sdk/v0.4/meta.json b/docs/content/docs/cua/reference/agent-sdk/v0.4/meta.jsondeleted file mode 100644index c1fcce1a7b..0000000000--- a/docs/content/docs/cua/reference/agent-sdk/v0.4/meta.json+++ /dev/null@@ -1,5 +0,0 @@-{- "title": "v0.4",- "description": "Agent SDK v0.4 API Reference",- "pages": ["api"]-}diff --git a/docs/content/docs/cua/reference/agent-sdk/v0.5/api.mdx b/docs/content/docs/cua/reference/agent-sdk/v0.5/api.mdxdeleted file mode 100644index 80eed1cfa3..0000000000--- a/docs/content/docs/cua/reference/agent-sdk/v0.5/api.mdx+++ /dev/null@@ -1,83 +0,0 @@-----title: Agent SDK v0.5 API Reference-description: API reference for Agent SDK version 0.5------{/*-AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY-Generated by: npx tsx scripts/docs-generators/generate-versioned-docs.ts-Source tag: agent-v0.5.2-Version: 0.5.2-*/}--import { Callout } from 'fumadocs-ui/components/callout';--<Callout type="warn">- This is documentation for **v0.5**. [View latest version](/cua/reference/agent-sdk).-</Callout>--<div className="flex items-center gap-2 mb-6">- <span className="px-2 py-1 bg-amber-100 dark:bg-amber-900 text-amber-800 dark:text-amber-200 rounded text-sm font-mono">- v0.5.2- </span>- <span className="text-sm text-fd-muted-foreground">pip install cua-agent==0.5.2</span>-</div>--agent - Decorator-based Computer Use Agent with liteLLM integration--## Classes--| Class | Description |-| --------------- | ---------------------------------------------------------------------- |-| `ComputerAgent` | Main agent class that automatically selects the appropriate agent loop |--## ComputerAgent--Main agent class that automatically selects the appropriate agent loop-based on the model and executes tool calls.--### Methods--#### ComputerAgent.run--```python-async def run(self, messages: Messages, stream: bool = False, api_key: Optional[str] = None, api_base: Optional[str] = None, additional_generation_kwargs = {}) -> AsyncGenerator[Dict[str, Any], None]-```--Run the agent with the given messages using Computer protocol handler pattern.--Args:-messages: List of message dictionaries-stream: Whether to stream the response-api_key: Optional API key override for the model provider-api_base: Optional API base URL override for the model provider-\*\*additional_generation_kwargs: Additional arguments passed to the model provider--Returns:-AsyncGenerator that yields response chunks--#### ComputerAgent.predict_click--```python-async def predict_click(self, instruction: str, image_b64: Optional[str] = None) -> Optional[Tuple[int, int]]-```--Predict click coordinates based on image and instruction.--Args:-instruction: Instruction for where to click-image_b64: Base64 encoded image (optional, will take screenshot if not provided)--Returns:-None or tuple with (x, y) coordinates--#### ComputerAgent.get_capabilities--```python-def get_capabilities(self) -> List[AgentCapability]-```--Get list of capabilities supported by the current agent config.--Returns:-List of capability strings (e.g., ["step", "click"])diff --git a/docs/content/docs/cua/reference/agent-sdk/v0.5/meta.json b/docs/content/docs/cua/reference/agent-sdk/v0.5/meta.jsondeleted file mode 100644index 0332e1eb36..0000000000--- a/docs/content/docs/cua/reference/agent-sdk/v0.5/meta.json+++ /dev/null@@ -1,5 +0,0 @@-{- "title": "v0.5",- "description": "Agent SDK v0.5 API Reference",- "pages": ["api"]-}diff --git a/docs/content/docs/cua/reference/agent-sdk/v0.6/api.mdx b/docs/content/docs/cua/reference/agent-sdk/v0.6/api.mdxdeleted file mode 100644index 871a4d41d2..0000000000--- a/docs/content/docs/cua/reference/agent-sdk/v0.6/api.mdx+++ /dev/null@@ -1,99 +0,0 @@-----title: Agent SDK v0.6 API Reference-description: API reference for Agent SDK version 0.6------{/*-AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY-Generated by: npx tsx scripts/docs-generators/generate-versioned-docs.ts-Source tag: agent-v0.6.2-Version: 0.6.2-*/}--import { Callout } from 'fumadocs-ui/components/callout';--<Callout type="warn">- This is documentation for **v0.6**. [View latest version](/cua/reference/agent-sdk).-</Callout>--<div className="flex items-center gap-2 mb-6">- <span className="px-2 py-1 bg-amber-100 dark:bg-amber-900 text-amber-800 dark:text-amber-200 rounded text-sm font-mono">- v0.6.2- </span>- <span className="text-sm text-fd-muted-foreground">pip install cua-agent==0.6.2</span>-</div>--agent - Decorator-based Computer Use Agent with liteLLM integration--## Classes--| Class | Description |-| --------------- | ---------------------------------------------------------------------- |-| `ComputerAgent` | Main agent class that automatically selects the appropriate agent loop |--## ComputerAgent--Main agent class that automatically selects the appropriate agent loop-based on the model and executes tool calls.--### Methods--#### ComputerAgent.run--```python-async def run(self, messages: Messages, stream: bool = False, api_key: Optional[str] = None, api_base: Optional[str] = None, additional_generation_kwargs = {}) -> AsyncGenerator[Dict[str, Any], None]-```--Run the agent with the given messages using Computer protocol handler pattern.--Args:-messages: List of message dictionaries-stream: Whether to stream the response-api_key: Optional API key override for the model provider-api_base: Optional API base URL override for the model provider-\*\*additional_generation_kwargs: Additional arguments passed to the model provider--Returns:-AsyncGenerator that yields response chunks--#### ComputerAgent.predict_click--```python-async def predict_click(self, instruction: str, image_b64: Optional[str] = None) -> Optional[Tuple[int, int]]-```--Predict click coordinates based on image and instruction.--Args:-instruction: Instruction for where to click-image_b64: Base64 encoded image (optional, will take screenshot if not provided)--Returns:-None or tuple with (x, y) coordinates--#### ComputerAgent.get_capabilities--```python-def get_capabilities(self) -> List[AgentCapability]-```--Get list of capabilities supported by the current agent config.--Returns:-List of capability strings (e.g., ["step", "click"])--#### ComputerAgent.open--```python-def open(self, port: Optional[int] = None)-```--Start the playground server and open it in the browser.--This method starts a local HTTP server that exposes the /responses endpoint-and automatically opens the Cua playground interface in the default browser.--Args:-port: Port to run the server on. If None, finds an available port automatically.--Example: >>> agent = ComputerAgent(model="claude-sonnet-4") >>> agent.open() # Starts server and opens browserdiff --git a/docs/content/docs/cua/reference/agent-sdk/v0.6/meta.json b/docs/content/docs/cua/reference/agent-sdk/v0.6/meta.jsondeleted file mode 100644index d141fa6c32..0000000000--- a/docs/content/docs/cua/reference/agent-sdk/v0.6/meta.json+++ /dev/null@@ -1,5 +0,0 @@-{- "title": "v0.6",- "description": "Agent SDK v0.6 API Reference",- "pages": ["api"]-}diff --git a/docs/content/docs/cua/reference/agent-sdk/v0.7/api.mdx b/docs/content/docs/cua/reference/agent-sdk/v0.7/api.mdxdeleted file mode 100644index d5ac95e669..0000000000--- a/docs/content/docs/cua/reference/agent-sdk/v0.7/api.mdx+++ /dev/null@@ -1,99 +0,0 @@-----title: Agent SDK v0.7 API Reference-description: API reference for Agent SDK version 0.7------{/*-AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY-Generated by: npx tsx scripts/docs-generators/generate-versioned-docs.ts-Source tag: agent-v0.7.23-Version: 0.7.23-*/}--import { Callout } from 'fumadocs-ui/components/callout';--<Callout type="warn">- This is documentation for **v0.7**. [View latest version](/cua/reference/agent-sdk).-</Callout>--<div className="flex items-center gap-2 mb-6">- <span className="px-2 py-1 bg-amber-100 dark:bg-amber-900 text-amber-800 dark:text-amber-200 rounded text-sm font-mono">- v0.7.23- </span>- <span className="text-sm text-fd-muted-foreground">pip install cua-agent==0.7.23</span>-</div>--agent - Decorator-based Computer Use Agent with liteLLM integration--## Classes--| Class | Description |-| --------------- | ---------------------------------------------------------------------- |-| `ComputerAgent` | Main agent class that automatically selects the appropriate agent loop |--## ComputerAgent--Main agent class that automatically selects the appropriate agent loop-based on the model and executes tool calls.--### Methods--#### ComputerAgent.run--```python-async def run(self, messages: Messages, stream: bool = False, api_key: Optional[str] = None, api_base: Optional[str] = None, additional_generation_kwargs = {}) -> AsyncGenerator[Dict[str, Any], None]-```--Run the agent with the given messages using Computer protocol handler pattern.--Args:-messages: List of message dictionaries-stream: Whether to stream the response-api_key: Optional API key override for the model provider-api_base: Optional API base URL override for the model provider-\*\*additional_generation_kwargs: Additional arguments passed to the model provider--Returns:-AsyncGenerator that yields response chunks--#### ComputerAgent.predict_click--```python-async def predict_click(self, instruction: str, image_b64: Optional[str] = None) -> Optional[Tuple[int, int]]-```--Predict click coordinates based on image and instruction.--Args:-instruction: Instruction for where to click-image_b64: Base64 encoded image (optional, will take screenshot if not provided)--Returns:-None or tuple with (x, y) coordinates--#### ComputerAgent.get_capabilities--```python-def get_capabilities(self) -> List[AgentCapability]-```--Get list of capabilities supported by the current agent config.--Returns:-List of capability strings (e.g., ["step", "click"])--#### ComputerAgent.open--```python-def open(self, port: Optional[int] = None)-```--Start the playground server and open it in the browser.--This method starts a local HTTP server that exposes the /responses endpoint-and automatically opens the Cua playground interface in the default browser.--Args:-port: Port to run the server on. If None, finds an available port automatically.--Example: >>> agent = ComputerAgent(model="claude-sonnet-4") >>> agent.open() # Starts server and opens browserdiff --git a/docs/content/docs/cua/reference/agent-sdk/v0.7/meta.json b/docs/content/docs/cua/reference/agent-sdk/v0.7/meta.jsondeleted file mode 100644index 832d2e52d9..0000000000--- a/docs/content/docs/cua/reference/agent-sdk/v0.7/meta.json+++ /dev/null@@ -1,5 +0,0 @@-{- "title": "v0.7",- "description": "Agent SDK v0.7 API Reference",- "pages": ["api"]-}diff --git a/docs/content/docs/cua/reference/cli/changelog.mdx b/docs/content/docs/cua/reference/cli/changelog.mdxdeleted file mode 100644index 9a54d7db74..0000000000--- a/docs/content/docs/cua/reference/cli/changelog.mdx+++ /dev/null@@ -1,28 +0,0 @@-----title: Changelog-description: Release history for Cua CLI------# CLI Changelog--All notable changes to the Cua CLI are documented here.--## 0.1.x--### v0.1.0 (Unreleased)--Initial release of the unified Cua CLI.--**Features:**--- Authentication management (`cua auth login/logout/env`)-- Sandbox lifecycle management (`cua sandbox create/list/start/stop/delete`)-- Cloud image management (`cua image list/push/pull/delete`)-- Skills recording and playback (`cua skills record/list/read/replay`)-- MCP server for AI assistants (`cua serve-mcp`)--**Installation:**--```bash-pip install cua-cli-```diff --git a/docs/content/docs/cua/reference/cli/commands.mdx b/docs/content/docs/cua/reference/cli/commands.mdxdeleted file mode 100644index f68dfbe7b1..0000000000--- a/docs/content/docs/cua/reference/cli/commands.mdx+++ /dev/null@@ -1,197 +0,0 @@-----title: Command Reference-description: Complete reference for all Cua CLI commands------import { Callout } from 'fumadocs-ui/components/callout';--**Package:** `cua-cli` | **Install:** `pip install cua-cli`--## Command Groups--| Group | Alias | Description |-|-------|-------|-------------|-| `cua sandbox` | `cua sb` | Sandbox lifecycle management |-| `cua auth` | | Authentication |-| `cua image` | `cua img` | Local image management |-| `cua skills` | | Skills recording and playback |-| `cua serve-mcp` | | Start MCP server |-------## Sandbox Commands--All sandbox commands support the `cua sb` alias (e.g. `cua sb launch`).--### `cua sb launch <image>`--Launch a new sandbox. The image is a positional argument; resource specs are options.--```bash-cua sb launch ubuntu:24.04-cua sb launch macos-cua sb launch windows:11-cua sb launch android:14--# Local (Docker / Lume / QEMU) — no API key needed-cua sb launch ubuntu:24.04 --local-cua sb launch macos --local--# Specify resources (cloud only)-cua sb launch ubuntu:24.04 --cpu 4 --memory 8GB --disk 50GB-cua sb launch ubuntu:24.04 --region europe--# Give it a name and get JSON output (useful in scripts)-cua sb launch ubuntu:24.04 --local --name my-sandbox --json-```--**Arguments:**--| Argument | Description |-|----------|-------------|-| `image` | Image to run: `macos`, `ubuntu:24.04`, `windows:11`, `android:14`, or a registry ref like `ghcr.io/org/img:tag` |--**Options:**--| Option | Description |-|--------|-------------|-| `--local` | Run locally (Docker/Lume/QEMU) instead of Cua Cloud |-| `--name` | Sandbox name (auto-generated if omitted) |-| `--vm` | Force VM kind for Linux images (default: container) |-| `--cpu N` | Number of vCPUs (cloud only) |-| `--memory SIZE` | Memory — e.g. `8GB`, `4096MB` (cloud only) |-| `--disk SIZE` | Disk size — e.g. `50GB` (cloud only) |-| `--region NAME` | Cloud region (cloud only) |-| `--json` | Output `{"name": "...", "status": "ready"}` |--**Image shorthands:**--| String | Resolves to |-|--------|-------------|-| `macos` | `Image.macos("26")` |-| `macos:sequoia` | `Image.macos("sequoia")` |-| `ubuntu:24.04` | `Image.linux("ubuntu", "24.04")` |-| `linux` | `Image.linux("ubuntu", "24.04")` |-| `windows:11` | `Image.windows("11")` |-| `android:14` | `Image.android("14")` |-| `ghcr.io/org/img` | `Image.from_registry("ghcr.io/org/img")` |--### `cua sb ls`--List running sandboxes.--```bash-cua sb ls # cloud sandboxes (requires API key)-cua sb ls --local # local sandboxes only-cua sb ls --all # both cloud and local-cua sb ls --json # JSON output-```--Aliases: `cua sb list`--### `cua sb info <name>`--Get details for a sandbox.--```bash-cua sb info my-sandbox-cua sb info my-sandbox --local-cua sb info my-sandbox --json-```--Alias: `cua sb get <name>`--### `cua sb delete <name>`--Delete a sandbox permanently.--```bash-cua sb delete my-sandbox-cua sb delete my-sandbox --local-cua sb delete my-sandbox --force # skip confirmation-```--<Callout type="warn">- Irreversible. All data on the sandbox is permanently lost.-</Callout>--### `cua sb vnc <name>`--Open the sandbox desktop in your browser.--```bash-cua sb vnc my-sandbox-cua sb vnc my-sandbox --local-```--### `cua sb shell <name>`--Open an interactive shell session.--```bash-cua sb shell my-sandbox-cua sb shell my-sandbox --local--# Run a single command-cua sb shell my-sandbox ls -la /-cua sb shell my-sandbox --cols 120 --rows 40-```--### `cua sb exec <name> <command>`--Execute a command non-interactively (useful for scripts).--```bash-cua sb exec my-sandbox echo hello-cua sb exec my-sandbox --json uname -a-```--### `cua sb suspend <name>` / `cua sb resume <name>`--Suspend a sandbox (preserves memory state) and resume it.--```bash-cua sb suspend my-sandbox-cua sb resume my-sandbox-```--### `cua sb restart <name>`--Restart a sandbox.--```bash-cua sb restart my-sandbox-```-------## Authentication Commands--### `cua auth login`--```bash-cua auth login # browser-based OAuth-cua auth login --api-key sk_cua-... # direct API key-```--### `cua auth env`--Write `CUA_API_KEY` to a `.env` file in the current directory.--```bash-cua auth env-```--### `cua auth logout`--```bash-cua auth logout-```-------## Environment Variables--| Variable | Description |-|----------|-------------|-| `CUA_API_KEY` | API key for cloud authentication |diff --git a/docs/content/docs/cua/reference/cli/index.mdx b/docs/content/docs/cua/reference/cli/index.mdxdeleted file mode 100644index 06fb35b9f4..0000000000--- a/docs/content/docs/cua/reference/cli/index.mdx+++ /dev/null@@ -1,249 +0,0 @@-----title: Cua CLI API Reference-description: Python API reference for the Cua command-line interface------{/*- AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY- Generated by: npx tsx scripts/docs-generators/python-sdk.ts- Source: libs/python/cua-cli/cua_cli- Version: 0.1.8-*/}--import { Callout } from 'fumadocs-ui/components/callout';-import { Tabs, Tab } from 'fumadocs-ui/components/tabs';-import { VersionHeader } from '@/components/version-selector';--<VersionHeader- versions={[{"version":"0.1","href":"/cua/reference/cli","isCurrent":true}]}- currentVersion="0.1"- fullVersion="0.1.8"- packageName="cua-cli"-/>--Cua CLI - Unified command-line interface for Computer-Use Agents.-------## main--Main entry point for Cua CLI.--### print_error--```python-def print_error(message: str) -> None-```--Print an error message to stderr.--### create_parser--```python-def create_parser() -> argparse.ArgumentParser-```--Create the main argument parser with all subcommands.--### main--```python-def main() -> int-```--Main entry point for the CLI.-------## auth--Authentication module for Cua CLI.-------## CredentialStore--SQLite-based credential store with WAL mode for concurrent access.--### Constructor--```python-CredentialStore(self, db_path: Path | None = None)-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `db_path` | `Any` | |--### Methods--#### CredentialStore.get--```python-def get(self, key: str) -> Optional[str]-```--Get a value from the store.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `key` | `Any` | The key to look up |--**Returns:** The value, or None if not found--#### CredentialStore.set--```python-def set(self, key: str, value: str) -> None-```--Set a value in the store.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `key` | `Any` | The key to set |-| `value` | `Any` | The value to store |--#### CredentialStore.delete--```python-def delete(self, key: str) -> bool-```--Delete a value from the store.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `key` | `Any` | The key to delete |--**Returns:** True if the key was deleted, False if it didn't exist--#### CredentialStore.clear--```python-def clear(self) -> None-```--Clear all stored credentials.--### clear_credentials--```python-def clear_credentials() -> None-```--Clear all stored credentials.--### get_api_key--```python-def get_api_key() -> Optional[str]-```--Get the stored API key.--First checks CUA_API_KEY environment variable, then falls back to stored credentials.--**Returns:** The API key, or None if not found--### save_api_key--```python-def save_api_key(api_key: str) -> None-```--Save an API key to the credential store.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `api_key` | `Any` | The API key to save |-------## utils--Utility modules for Cua CLI.--### run_async--```python-def run_async(coro: Coroutine[Any, Any, T]) -> T-```--Run an async coroutine synchronously.--This is the standard pattern for CLI commands that need to call async code.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `coro` | `Any` | The coroutine to run |--**Returns:** The result of the coroutine--### print_error--```python-def print_error(message: str) -> None-```--Print an error message to stderr.--### print_info--```python-def print_info(message: str) -> None-```--Print an info message.--### print_json--```python-def print_json(data: Any) -> None-```--Print data as formatted JSON.--### print_success--```python-def print_success(message: str) -> None-```--Print a success message.--### print_table--```python-def print_table(data: list[dict[str, Any]], columns: list[tuple[str, str]] | None = None, title: str | None = None) -> None-```--Print data as a formatted table.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `data` | `Any` | List of dictionaries to display |-| `columns` | `Any` | List of (key, header) tuples. If None, uses all keys from first item. |-| `title` | `Any` | Optional table title |--### print_warning--```python-def print_warning(message: str) -> None-```--Print a warning message.diff --git a/docs/content/docs/cua/reference/cli/meta.json b/docs/content/docs/cua/reference/cli/meta.jsondeleted file mode 100644index 739205a5cd..0000000000--- a/docs/content/docs/cua/reference/cli/meta.json+++ /dev/null@@ -1,6 +0,0 @@-{- "title": "CLI",- "description": "Command-line interface for Cua",- "icon": "Terminal",- "pages": ["commands", "changelog"]-}diff --git a/docs/content/docs/cua/reference/cloud-cli/commands.mdx b/docs/content/docs/cua/reference/cloud-cli/commands.mdxdeleted file mode 100644index d4be42ccc1..0000000000--- a/docs/content/docs/cua/reference/cloud-cli/commands.mdx+++ /dev/null@@ -1,530 +0,0 @@-----title: Command Reference-description: Complete reference for all Cua CLI commands------import { Tabs, Tab } from 'fumadocs-ui/components/tabs';-import { Callout } from 'fumadocs-ui/components/callout';--## Overview--The Cua CLI provides commands for authentication and sandbox management.--### Command Styles--The CLI supports **two command styles** for flexibility:--**Flat style** (quick & concise):--```bash-cua list-cua create --os linux --size small --region north-america-cua start my-sandbox-```--**Grouped style** (explicit & clear):--```bash-cua sb list # or: cua sandbox list-cua sb create # or: cua sandbox create-cua sb start # or: cua sandbox start-```--Both styles work identically - use whichever you prefer!--### Available Commands--- **Authentication** - `cua auth login`, `cua auth env`, `cua auth logout` (also available as flat commands: `cua login`, `cua env`, `cua logout`)-- **Sandbox Management** - `cua list`, `cua create`, `cua get`, `cua start`, `cua stop`, `cua restart`, `cua delete`, `cua vnc`, `cua shell`, `cua exec`--## Authentication Commands--### `cua auth login`--Authenticate with your Cua account using browser-based OAuth flow.--```bash-# Interactive browser login-cua auth login--# Direct API key login-cua auth login --api-key sk-your-api-key-here--# Alternative flat style-cua login-cua login --api-key sk-your-api-key-here-```--**Options:**--- `--api-key <key>` - Provide API key directly instead of browser flow--**Example:**--```bash-$ cua auth login-Opening browser for CLI auth...-API key saved-```--### `cua auth env`--Create or update a `.env` file in the current directory with your Cua API key.--```bash-cua auth env--# Alternative flat style-cua env-```--**Example:**--```bash-$ cua auth env-Wrote /path/to/your/project/.env-```--The generated `.env` file will contain:--```-CUA_API_KEY=sk-your-api-key-here-```--### `cua auth logout`--Remove the stored API key from your system.--```bash-cua auth logout--# Alternative flat style-cua logout-```--**Example:**--```bash-$ cua auth logout-Logged out-```--## Sandbox Commands--### `cua list`--List all your sandboxes with their current status. Passwords are hidden by default for security.--```bash-# List sandboxes (passwords hidden)-cua list--# Show passwords explicitly-cua list --show-passwords--# Alternative aliases-cua ls-cua ps-```--**Example Output (default, passwords hidden):**--```-NAME STATUS HOST-my-dev-sandbox running my-dev-sandbox.sandbox.cua.ai-test-windows stopped test-windows.sandbox.cua.ai-```--**Example Output (with --show-passwords):**--```-NAME STATUS PASSWORD HOST-my-dev-sandbox running secure-pass-123 my-dev-sandbox.sandbox.cua.ai-test-windows stopped another-pass-456 test-windows.sandbox.cua.ai-```--### `cua create`--Create a new sandbox.--```bash-cua create --os <OS> --size <SIZE> --region <REGION>-```--**Required Options:**--- `--os` - Operating system: `linux`, `windows`, `macos`-- `--size` - Sandbox size: `small`, `medium`, `large`-- `--region` - Region: `north-america`, `europe`, `asia-pacific`, `south-america`--**Examples:**--```bash-# Create a small Linux sandbox in North America-cua create --os linux --size small --region north-america--# Create a medium Windows sandbox in Europe-cua create --os windows --size medium --region europe--# Create a large macOS sandbox in Asia Pacific-cua create --os macos --size large --region asia-pacific-```--**Response Types:**--**Immediate (Status 200):**--```bash-Sandbox created and ready: my-new-sandbox-abc123-Password: secure-password-here-Host: my-new-sandbox-abc123.sandbox.cua.ai-```--**Provisioning (Status 202):**--```bash-Sandbox provisioning started: my-new-sandbox-abc123-Job ID: job-xyz789-Use 'cua list' to monitor provisioning progress-```--### `cua get`--Get detailed information about a specific sandbox, including computer-server health status.--```bash-cua get <name>--# With additional options-cua get <name> --json-cua get <name> --show-passwords-cua get <name> --show-vnc-url-```--**Options:**--- `--json` - Output all details in JSON format-- `--show-passwords` - Include password in output-- `--show-vnc-url` - Include computed NoVNC URL--**Example Output (default):**--```bash-$ cua get my-dev-sandbox-Name: my-dev-sandbox-Status: running-Host: my-dev-sandbox.containers.cloud.trycua.com-OS Type: linux-Computer Server Version: 0.1.30-Computer Server Status: healthy-```--**Example Output (with --show-passwords and --show-vnc-url):**--```bash-$ cua get my-dev-sandbox --show-passwords --show-vnc-url-Name: my-dev-sandbox-Status: running-Host: my-dev-sandbox.containers.cloud.trycua.com-Password: secure-pass-123-OS Type: linux-Computer Server Version: 0.1.30-Computer Server Status: healthy-VNC URL: https://my-dev-sandbox.containers.cloud.trycua.com/vnc.html?autoconnect=true&password=secure-pass-123-```--**Example Output (JSON format):**--```bash-$ cua get my-dev-sandbox --json-{- "name": "my-dev-sandbox",- "status": "running",- "host": "my-dev-sandbox.containers.cloud.trycua.com",- "os_type": "linux",- "computer_server_version": "0.1.30",- "computer_server_status": "healthy"-}-```--**Computer Server Health Check:**--The `cua get` command automatically probes the computer-server when the sandbox is running:--- Checks OS type via `https://{host}:8443/status`-- Checks version via `https://{host}:8443/cmd`-- Shows "Computer Server Status: healthy" when both probes succeed-- Uses a 3-second timeout for each probe--<Callout type="info">- The computer server status is only checked for running sandboxes. Stopped or suspended sandboxes- will not show computer server information.-</Callout>--### `cua start`--Start a stopped sandbox.--```bash-cua start <name>-```--**Example:**--```bash-$ cua start my-dev-sandbox-Start accepted-```--### `cua stop`--Stop a running sandbox.--```bash-cua stop <name>-```--**Example:**--```bash-$ cua stop my-dev-sandbox-stopping-```--### `cua restart`--Restart a sandbox.--```bash-cua restart <name>-```--**Example:**--```bash-$ cua restart my-dev-sandbox-restarting-```--### `cua delete`--Delete a sandbox permanently.--```bash-cua delete <name>-```--**Example:**--```bash-$ cua delete old-test-sandbox-Sandbox deletion initiated: deleting-```--<Callout type="warn">- This action is irreversible. All data on the sandbox will be permanently lost.-</Callout>--### `cua vnc`--Open the VNC interface for a sandbox in your browser.--```bash-cua vnc <name>--# Alternative alias-cua open <name>-```--**Example:**--```bash-$ cua vnc my-dev-sandbox-Opening NoVNC: https://my-dev-sandbox.sandbox.cua.ai/vnc.html?autoconnect=true&password=...-```--This command automatically opens your default browser to the VNC interface with the correct password pre-filled.--### `cua shell`--Open an interactive shell session in a running sandbox.--```bash-cua shell <name>--# Run a command through the shell-cua shell <name> <command...>--# Specify terminal size-cua shell --cols 120 --rows 40 <name>-```--**Options:**--- `--cols <number>` - Terminal width in columns (default: current terminal width or 80)-- `--rows <number>` - Terminal height in rows (default: current terminal height or 24)--**Examples:**--```bash-# Open interactive bash shell-$ cua shell my-dev-sandbox-root@sandbox:~# ls-Desktop Documents Downloads-root@sandbox:~# exit-exit--# Run a single command through the shell-$ cua shell my-dev-sandbox echo "Hello from sandbox"-Hello from sandbox--# Check system info-$ cua shell my-dev-sandbox uname -a-Linux sandbox 5.15.0 #1 SMP x86_64 GNU/Linux-```--<Callout type="info">- The shell command connects via WebSocket to the sandbox's PTY endpoint, providing a full interactive terminal experience with proper TTY handling.-</Callout>--### `cua exec`--Execute a command directly in a sandbox without an interactive shell.--```bash-cua exec <name> <command...>--# Get JSON output-cua exec --json <name> <command...>-```--**Options:**--- `--json` - Output result as JSON including stdout, stderr, and exit code--**Examples:**--```bash-# Run a command-$ cua exec my-dev-sandbox ls -la /tmp-total 8-drwxrwxrwt 2 root root 4096 Mar 13 10:00 .-drwxr-xr-x 1 root root 4096 Mar 13 09:00 ..--# Get JSON output with exit code-$ cua exec --json my-dev-sandbox echo "test"-{- "stdout": "test\n",- "stderr": "",- "return_code": 0-}--# Check exit codes for scripting-$ cua exec my-dev-sandbox test -f /etc/passwd && echo "File exists"-File exists-```--**Exit Codes:**--The `exec` command returns the exit code from the executed command, making it suitable for scripting:--```bash-# Returns 0 on success-cua exec my-sandbox true-echo $? # 0--# Returns non-zero on failure-cua exec my-sandbox false-echo $? # 1-```--<Callout type="info">- Unlike `cua shell`, the `exec` command runs commands non-interactively without PTY allocation. Use `shell` for interactive sessions and `exec` for automation scripts.-</Callout>--## Global Options--### Help--Get help for any command:--```bash-cua --help-cua auth login --help-cua create --help-cua list --help-```--## Error Handling--The CLI provides clear error messages for common issues:--### Authentication Errors--```bash-$ cua list-Unauthorized. Try 'cua auth login' again.-```--### Sandbox Not Found--```bash-$ cua start nonexistent-sandbox-Sandbox not found-```--### Invalid Configuration--```bash-$ cua create --os invalid --configuration small --region north-america-Invalid request or unsupported configuration-```--## Tips and Best Practices--### 1. Use Descriptive Sandbox Names--```bash-# Good-cua create --os linux --size small --region north-america-# Then rename or use meaningful names in the dashboard--# Better workflow-cua list # Check the generated name-# Use that name consistently-```--### 2. Environment Management--```bash-# Set up your project with API key-cd my-project-cua auth env-# Now your project has CUA_API_KEY in .env-```--### 3. Quick Sandbox Access--```bash-# Create aliases for frequently used sandboxes-alias dev-sandbox="cua vnc my-development-sandbox"-alias prod-sandbox="cua vnc my-production-sandbox"-```--### 4. Monitoring Provisioning--```bash-# For sandboxes that need provisioning time-cua create --os windows --size large --region europe-# Sandbox provisioning started: my-sandbox-abc123-# Job ID: job-xyz789--# Check status periodically-watch -n 5 cua list-```--## Next Steps--- [Get started with the quickstart guide](/cua/guide/get-started/what-is-cua)-- [Learn about Cua computers](/cua/reference/computer-sdk)-- [Explore agent automation](/cua/guide/fundamentals/agent-loops)diff --git a/docs/content/docs/cua/reference/cloud-cli/index.mdx b/docs/content/docs/cua/reference/cloud-cli/index.mdxdeleted file mode 100644index 11a6cf1c95..0000000000--- a/docs/content/docs/cua/reference/cloud-cli/index.mdx+++ /dev/null@@ -1,74 +0,0 @@-----title: Getting Started-description: Install and set up the Cua CLI------import { Tabs, Tab } from 'fumadocs-ui/components/tabs';-import { Callout } from 'fumadocs-ui/components/callout';--The Cua CLI is a command-line tool for managing your Cua cloud sandboxes. Create, start, stop, and connect to sandboxes directly from your terminal.--## Installation--<Tabs items={['macOS / Linux', 'Windows']}>- <Tab value="macOS / Linux">- ```bash- curl -LsSf https://cua.ai/cli/install.sh | sh- ```- </Tab>- <Tab value="Windows">- ```powershell- powershell -ExecutionPolicy ByPass -c "irm https://cua.ai/cli/install.ps1 | iex"- ```- </Tab>-</Tabs>--This installs [Bun](https://bun.sh) and the Cua CLI. Verify with:--```bash-cua --help-```--## Authentication--Login to your Cua account:--```bash-# Browser-based login-cua auth login--# Or with API key-cua auth login --api-key sk-your-api-key-here-```--Generate a `.env` file for your project:--```bash-cua auth env-```--## Quick Start--```bash-# Create a sandbox-cua create --os linux --size small --region north-america--# List sandboxes-cua list--# Open VNC in browser-cua vnc my-sandbox--# Open interactive shell-cua shell my-sandbox--# Run a command in the sandbox-cua exec my-sandbox ls -la--# Stop a sandbox-cua stop my-sandbox-```--## Next Steps--- [Command Reference](/cua/reference/cloud-cli/commands) - Full list of available commandsdiff --git a/docs/content/docs/cua/reference/cloud-cli/meta.json b/docs/content/docs/cua/reference/cloud-cli/meta.jsondeleted file mode 100644index 61e7dc9aaa..0000000000--- a/docs/content/docs/cua/reference/cloud-cli/meta.json+++ /dev/null@@ -1,6 +0,0 @@-{- "title": "Cloud CLI",- "description": "Command-line interface for Cua Cloud",- "icon": "Cloud",- "pages": ["index", "commands"]-}diff --git a/docs/content/docs/cua/reference/computer-sdk/changelog.mdx b/docs/content/docs/cua/reference/computer-sdk/changelog.mdxdeleted file mode 100644index 7b1ce79d1f..0000000000--- a/docs/content/docs/cua/reference/computer-sdk/changelog.mdx+++ /dev/null@@ -1,328 +0,0 @@-----title: Changelog-description: Release history for Computer SDK------{/*-AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY-Generated by: npx tsx scripts/docs-generators/generate-changelog.ts-Last updated: 2026-02-04-*/}--# Computer SDK Changelog--All notable changes to the Computer SDK are documented here.--## 0.5.x--### v0.5.12 (2026-01-26)--- Initial release or no path-specific changes found--### v0.5.11 (2026-01-24)--- Bump cua-computer to v0.5.11 by @github-actions[bot]-- Setup UV as venv manager for computer ([#789](https://github.com/trycua/cua/pull/789)) by @synacktraa--### v0.5.10 (2026-01-17)--Maintenance release.--### v0.5.7 (2026-01-17)--- Bump cua-computer to v0.5.7 by @github-actions[bot]-- fix(computer): prevent data loss when tracing.stop path conflicts with trace directory ([#814](https://github.com/trycua/cua/pull/814)) by @Harsh Verma-- Improve Computer API Server connection warnings and timeout errors ([#803](https://github.com/trycua/cua/pull/803)) by @Fizza Mukhtar-- feat(computer): add OpenTelemetry instrumentation ([#663](https://github.com/trycua/cua/pull/663)) by @r33drichards--### v0.5.6 (2026-01-12)--- Initial release or no path-specific changes found--### v0.5.5 (2026-01-12)--- Initial release or no path-specific changes found--### v0.5.4 (2026-01-12)--- Bump cua-computer to v0.5.3 by @github-actions[bot]-- Bump cua-computer to v0.5.2 by @github-actions[bot]-- refactor(docs): reorganize lume/cua docs and standardize READMEs ([#752](https://github.com/trycua/cua/pull/752)) by @Francesco Bonacci-- feat(lume,ci): add unattended VM setup and reorganize CI/CD workflows ([#729](https://github.com/trycua/cua/pull/729)) by @Francesco Bonacci-- update dead links due to docs update by @Sarina Li-- pyautogui cleanup by @Adam-- Revert "refractor docs into 6 sections" by @Adam-- refractor docs into 6 sections by @Adam--## 0.4.x--### v0.4.11 (2025-10-29)--**Dependencies:** pylume: latest--- Docs/improve readme by @jamesmurdza in https://github.com/trycua/cua/pull/512-- Feature/version bump workflows by @r33drichards in https://github.com/trycua/cua/pull/511-- Support `display` and `ephemeral` in Docker provider by @ddupont808 in https://github.com/trycua/cua/pull/514-- Fix markdown formatting in docs by @jamesmurdza in https://github.com/trycua/cua/pull/523-- Jagjeevan's Fix + Merged Main for 4o Model Fix by @sarinali in https://github.com/trycua/cua/pull/522-- fix: Added GPT-4o compatibility for screenshot actions with text parameter by @JagjeevanAK in https://github.com/trycua/cua/pull/422-- Remove PyLume code and examples by @jamesmurdza in https://github.com/trycua/cua/pull/435-- Remove PyLume library by @jamesmurdza in https://github.com/trycua/cua/pull/527-- Add test infrastructure with CI/CD #478 by @AceAtDev in https://github.com/trycua/cua/pull/491-- Feature/agent loop test by @YeIIcw in https://github.com/trycua/cua/pull/528-- Update Agent Loop Test by @YeIIcw in https://github.com/trycua/cua/pull/530-- Fix/omniparser predict refactor by @sarinali in https://github.com/trycua/cua/pull/529-- Fix agent test condition trigger by @YeIIcw in https://github.com/trycua/cua/pull/531-- Fix/agent loop test by @YeIIcw in https://github.com/trycua/cua/pull/532-- Add desktop and window management commands to computer interface by @ddupont808 in https://github.com/trycua/cua/pull/516--### v0.4.10 (2025-10-14)--- Fix XFCE Password Prompts and Lock Screen Issues by @f-trycua in https://github.com/trycua/cua/pull/466-- Add Cloud VM Management API by @ddupont808 in https://github.com/trycua/cua/pull/452--### v0.4.8 (2025-10-12)--**Dependencies:** pylume: latest--### v0.4.7 (2025-10-03)--**Dependencies:** pylume: latest--### v0.4.6 (2025-09-24)--**Dependencies:** pylume: latest--- 🔧 Improve HUD eval script: auto-discover .env and simplify usage by @YeIIcw in https://github.com/trycua/cua/pull/419-- Fix Windows Sandbox errors by @ddupont808 in https://github.com/trycua/cua/pull/424--New Contributors--- @YeIIcw made their first contribution in https://github.com/trycua/cua/pull/419-- @JagjeevanAK made their first contribution in https://github.com/trycua/cua/pull/420--### v0.4.5 (2025-09-05)--**Dependencies:** pylume: latest--- Fix x/y scrolling by @ddupont808 in https://github.com/trycua/cua/pull/393-- Fix invalid image error with some providers (i.e. WinSandbox)-- Reference documentation batch by @onel in https://github.com/trycua/cua/pull/390--### v0.4.4 (2025-08-19)--**Dependencies:** pylume: latest--- Update agent/computer SDKs to match changes in telemetry SDK by @jamesmurdza in https://github.com/trycua/cua/pull/355-- Bugfixes - Removed slashes from container name by @ddupont808 in https://github.com/trycua/cua/pull/358--### v0.4.3 (2025-08-15)--**Dependencies:** pylume: latest--- Fixed bug where the `image` kwarg would be ignored--### v0.4.2 (2025-08-14)--**Dependencies:** pylume: latest--- [Computer] Add Docker as a local VM provider by @ddupont808 in https://github.com/trycua/cua/pull/339--### v0.4.1 (2025-08-05)--**Dependencies:** pylume: latest--### v0.4.0 (2025-07-28)--Bumped `cua-computer` to use latest `cua-core`--## 0.3.x--### v0.3.7 (2025-07-16)--**Dependencies:** pylume: latest--### v0.3.6 (2025-07-16)--**Dependencies:** pylume: latest--### v0.3.5 (2025-07-10)--**Dependencies:** pylume: latest--### v0.3.4 (2025-07-03)--**Dependencies:** pylume: latest--### v0.3.3 (2025-07-03)--**Dependencies:** pylume: latest--### v0.3.2 (2025-07-03)--**Dependencies:** pylume: latest--### v0.3.1 (2025-07-01)--**Dependencies:** pylume: latest--### v0.3.0 (2025-06-25)--**Dependencies:** pylume: latest--## 0.2.x--### v0.2.13 (2025-06-24)--**Dependencies:** pylume: latest--### v0.2.12 (2025-06-20)--**Dependencies:** pylume: latest--### v0.2.11 (2025-06-18)--**Dependencies:** pylume: latest--### v0.2.10 (2025-06-10)--**Dependencies:** pylume: latest--- Removed warning when calling `start_vm` on the CloudProvider-- Add disconnect() method for Computer-- Fixed `file_exists` and `directory_exists` not being implemented in the computer-server-- Added keyboard/mouse primitives ( `mouse_up`, `mouse_down`, `key_up`, `key_down` )-- Added file system commands to computer interface-- Added file system interface pytests-- Improved computer interface accessibility tree output on macOS, now includes menubar and dock items, along with preserving z-ordering-- Increased timeout for web socket commands due to a TimeoutError if the accessibility tree takes longer than 30 seconds to crawl--**Changes reference:**--```py-... old actions haven't changed--Disconnect from computer interface / cleanup websocket resources-await computer.disconnect()--New mouse actions-await computer.interface.mouse_down(x, y, button="left") # Press and hold a mouse button-await computer.interface.mouse_up(x, y, button="left") # Release a mouse button--New keyboard actions-await computer.interface.key_down("command") # Press and hold a key-await computer.interface.key_up("command") # Release a key--New scrolling actions-await computer.interface.scroll(x, y) # Scroll the mouse wheel--New file-system commands-Fixed: await computer.interface.file_exists(path) # Check if file exists-Fixed: await computer.interface.directory_exists(path) # Check if directory exists-await computer.interface.read_text(path) # Read file content-await computer.interface.write_text(path, content) # Write file content-await computer.interface.read_bytes(path) # Read file content as bytes-await computer.interface.write_bytes(path, content) # Write file content as bytes-await computer.interface.delete_file(path) # Delete file-await computer.interface.create_dir(path) # Create directory-await computer.interface.delete_dir(path) # Delete directory-await computer.interface.list_dir(path) # List directory contents-```--### v0.2.9 (2025-06-05)--**Dependencies:** pylume: latest--### v0.2.8 (2025-05-31)--**Dependencies:** pylume: latest--### v0.2.7 (2025-05-28)--**Dependencies:** pylume: latest--### v0.2.6 (2025-05-28)--**Dependencies:** pylume: latest--### v0.2.5 (2025-05-28)--**Dependencies:** pylume: latest--### v0.2.4 (2025-05-28)--**Dependencies:** pylume: latest--### v0.2.3 (2025-05-28)--**Dependencies:** pylume: latest--### v0.2.2 (2025-05-16)--**Dependencies:** pylume: latest--### v0.2.1 (2025-05-14)--**Dependencies:** pylume: latest--### v0.2.0 (2025-05-14)--**Dependencies:** pylume: latest--## 0.1.x--### v0.1.29 (2025-05-12)--**Dependencies:** pylume: latest--### v0.1.28 (2025-05-10)--**Dependencies:** pylume: latest--### v0.1.27 (2025-05-10)--**Dependencies:** pylume: latest--### v0.1.26 (2025-05-10)--**Dependencies:** pylume: latest--### v0.1.25 (2025-05-07)--**Dependencies:** pylume: latest--### v0.1.24 (2025-04-29)--**Dependencies:** pylume: latest--### v0.1.23 (2025-04-22)--**Dependencies:** pylume: latest--### v0.1.22 (2025-04-15)--**Dependencies:** pylume: latest--### v0.1.21 (2025-03-30)--**Dependencies:** pylume: latest--### v0.1.20 (2025-03-30)--**Dependencies:** pylume: latest--### v0.1.19 (2025-03-30)--**Dependencies:** pylume: latest--### v0.1.18 (2025-03-24)--**Dependencies:** pylume: latest--### v0.1.7 (2025-03-19)--**Dependencies:** pylume: latest--### v0.1.6 (2025-03-17)--**Dependencies:** pylume: latestdiff --git a/docs/content/docs/cua/reference/computer-sdk/index.mdx b/docs/content/docs/cua/reference/computer-sdk/index.mdxdeleted file mode 100644index 40d15551f4..0000000000--- a/docs/content/docs/cua/reference/computer-sdk/index.mdx+++ /dev/null@@ -1,2196 +0,0 @@-----title: Computer SDK API Reference-description: Python API reference for controlling virtual machines and computer interfaces------{/*- AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY- Generated by: npx tsx scripts/docs-generators/python-sdk.ts- Source: libs/python/computer/computer- Version: 0.5.17-*/}--import { Callout } from 'fumadocs-ui/components/callout';-import { Tabs, Tab } from 'fumadocs-ui/components/tabs';-import { VersionHeader } from '@/components/version-selector';--<VersionHeader- versions={[{"version":"0.5","href":"/cua/reference/computer-sdk","isCurrent":true},{"version":"0.4","href":"/cua/reference/computer-sdk/v0.4/api","isCurrent":false},{"version":"0.3","href":"/cua/reference/computer-sdk/v0.3/api","isCurrent":false}]}- currentVersion="0.5"- fullVersion="0.5.17"- packageName="cua-computer"-/>--Cua Computer Interface for cross-platform computer control.--## Classes--| Class | Description |-|-------|-------------|-| [`Computer`](#computer) | Computer is the main class for interacting with the computer. |-| [`VMProviderType`](#vmprovidertype) | Enum of supported VM provider types. |-| [`PtyHandle`](#ptyhandle) | Lightweight handle to a live PTY session on a remote computer-server. |-| [`PtyInterface`](#ptyinterface) | Async HTTP+WebSocket client for the ``/pty`` endpoints on computer-server. |-------## Computer--Computer is the main class for interacting with the computer.--### Constructor--```python-Computer(self, display: Union[Display, Dict[str, int], str] = '1024x768', memory: str = '8GB', cpu: str = '4', os_type: OSType = 'macos', name: str = '', image: Optional[str] = None, shared_directories: Optional[List[str]] = None, use_host_computer_server: bool = False, verbosity: Union[int, LogLevel] = logging.INFO, telemetry_enabled: bool = True, provider_type: Union[str, VMProviderType] = VMProviderType.LUME, provider_port: Optional[int] = 7777, noVNC_port: Optional[int] = 8006, api_port: Optional[int] = None, host: str = 'localhost', api_host: Optional[str] = None, storage: Optional[str] = None, ephemeral: bool = False, api_key: Optional[str] = None, experiments: Optional[List[str]] = None, timeout: int = 100, backend: Literal['native', 'vnc'] = 'native', vnc_host: Optional[str] = None, vnc_port: int = 5900, vnc_password: str = '', run_opts: Optional[Dict[str, Any]] = None)-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `logger` | `Any` | |-| `image` | `Any` | |-| `host` | `Any` | |-| `provider_port` | `Any` | |-| `noVNC_port` | `Any` | |-| `api_port` | `Any` | |-| `api_host` | `Any` | |-| `os_type` | `Any` | |-| `provider_type` | `Any` | |-| `ephemeral` | `Any` | |-| `api_key` | `Any` | |-| `timeout` | `Any` | |-| `experiments` | `Any` | |-| `backend` | `Any` | |-| `vnc_host` | `Any` | |-| `vnc_port` | `Any` | |-| `vnc_password` | `Any` | |-| `custom_run_opts` | `Any` | |-| `storage` | `Any` | |-| `shared_path` | `Any` | |-| `verbosity` | `Any` | |-| `vm_logger` | `Any` | |-| `interface_logger` | `Any` | |-| `config` | `Any` | |-| `shared_directories` | `Any` | |-| `use_host_computer_server` | `Any` | |-| `interface` | `Any` | Get the computer interface for interacting with the VM. |-| `pty` | `PtyInterface` | Return a :class:`~computer.pty.PtyInterface` for spawning interactive PTY sessions. |-| `tracing` | `ComputerTracing` | Get the computer tracing instance for recording sessions. |-| `telemetry_enabled` | `bool` | Check if telemetry is enabled for this computer instance. |--### Methods--#### Computer.create_desktop_from_apps--```python-def create_desktop_from_apps(self, apps)-```--Create a virtual desktop from a list of app names, returning a DioramaComputer-that proxies Diorama.Interface but uses diorama_cmds via the computer interface.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `apps` | `list[str]` | List of application names to include in the desktop. |--**Returns:** DioramaComputer: A proxy object with the Diorama interface, but using diorama_cmds.--#### Computer.run--```python-async def run(self) -> Optional[str]-```--Initialize the VM and computer interface.--#### Computer.disconnect--```python-async def disconnect(self) -> None-```--Disconnect from the computer's WebSocket interface.--#### Computer.stop--```python-async def stop(self) -> None-```--Disconnect from the computer's WebSocket interface and stop the computer.--#### Computer.start--```python-async def start(self) -> None-```--Start the computer.--#### Computer.restart--```python-async def restart(self) -> None-```--Restart the computer.--If using a VM provider that supports restart, this will issue a restart-without tearing down the provider context, then reconnect the interface.-Falls back to stop()+run() when a provider restart is not available.--#### Computer.get_ip--```python-async def get_ip(self, max_retries: int = 15, retry_delay: int = 3) -> str-```--Get the IP address of the VM or localhost if using host computer server.--This method delegates to the provider's get_ip method, which waits indefinitely-until the VM has a valid IP address.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `max_retries` | `Any` | Unused parameter, kept for backward compatibility |-| `retry_delay` | `Any` | Delay between retries in seconds (default: 2) |--**Returns:** IP address of the VM or localhost if using host computer server--#### Computer.wait_vm_ready--```python-async def wait_vm_ready(self) -> Optional[Dict[str, Any]]-```--Wait for VM to be ready with an IP address.--**Returns:** VM status information or None if using host computer server.--#### Computer.update--```python-async def update(self, cpu: Optional[int] = None, memory: Optional[str] = None)-```--Update VM settings.--#### Computer.get_screenshot_size--```python-def get_screenshot_size(self, screenshot: bytes) -> Dict[str, int]-```--Get the dimensions of a screenshot.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `screenshot` | `Any` | The screenshot bytes |--**Returns:** Dict[str, int]: Dictionary containing 'width' and 'height' of the image--#### Computer.to_screen_coordinates--```python-async def to_screen_coordinates(self, x: float, y: float) -> tuple[float, float]-```--Convert normalized coordinates to screen coordinates.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `x` | `Any` | X coordinate between 0 and 1 |-| `y` | `Any` | Y coordinate between 0 and 1 |--**Returns:** tuple[float, float]: Screen coordinates (x, y)--#### Computer.to_screenshot_coordinates--```python-async def to_screenshot_coordinates(self, x: float, y: float) -> tuple[float, float]-```--Convert screen coordinates to screenshot coordinates.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `x` | `Any` | X coordinate in screen space |-| `y` | `Any` | Y coordinate in screen space |--**Returns:** tuple[float, float]: (x, y) coordinates in screenshot space--#### Computer.playwright_exec--```python-async def playwright_exec(self, command: str, params: Optional[Dict] = None) -> Dict[str, Any]-```--Execute a Playwright browser command.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `command` | `Any` | The browser command to execute (visit_url, click, type, scroll, web_search) |-| `params` | `Any` | Command parameters |--**Returns:** Dict containing the command result--**Example:**--```python-# Navigate to a URL-await computer.playwright_exec("visit_url", {"url": "https://example.com"})--# Click at coordinates-await computer.playwright_exec("click", {"x": 100, "y": 200})--# Type text-await computer.playwright_exec("type", {"text": "Hello, world!"})--# Scroll-await computer.playwright_exec("scroll", {"delta_x": 0, "delta_y": -100})--# Web search-await computer.playwright_exec("web_search", {"query": "computer use agent"})-```--#### Computer.venv_install--```python-async def venv_install(self, venv_name: str, requirements: list[str])-```--Install packages in a UV project.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `venv_name` | `Any` | Name of the UV project |-| `requirements` | `Any` | List of package requirements to install |--**Returns:** Tuple of (stdout, stderr) from the installation command--#### Computer.pip_install--```python-async def pip_install(self, requirements: list[str])-```--Install packages using the system Python with UV (no venv).--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `requirements` | `Any` | List of package requirements to install globally/user site. |--**Returns:** Tuple of (stdout, stderr) from the installation command--#### Computer.venv_cmd--```python-async def venv_cmd(self, venv_name: str, command: str)-```--Execute a shell command in a UV project.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `venv_name` | `Any` | Name of the UV project |-| `command` | `Any` | Shell command to execute in the UV project |--**Returns:** Tuple of (stdout, stderr) from the command execution--#### Computer.venv_exec--```python-async def venv_exec(self, venv_name: str, python_func, args = (), kwargs = {})-```--Execute Python function in a virtual environment using source code extraction.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `venv_name` | `Any` | Name of the virtual environment |-| `python_func` | `Any` | A callable function to execute *args: Positional arguments to pass to the function **kwargs: Keyword arguments to pass to the function |--**Returns:** The result of the function execution, or raises any exception that occurred--#### Computer.venv_exec_background--```python-async def venv_exec_background(self, venv_name: str, python_func, args = (), requirements: Optional[List[str]] = None, kwargs = {}) -> int-```--Run the Python function in the venv in the background and return the PID.--Uses a short launcher Python that spawns a detached child and exits immediately.--#### Computer.python_exec--```python-async def python_exec(self, python_func, args = (), kwargs = {})-```--Execute a Python function using the system Python (no venv).--Uses source extraction and base64 transport, mirroring venv_exec but-without virtual environment activation.--Returns the function result or raises a reconstructed exception with-remote traceback context appended.--#### Computer.python_exec_background--```python-async def python_exec_background(self, python_func, args = (), requirements: Optional[List[str]] = None, kwargs = {}) -> int-```--Run a Python function with the system interpreter in the background and return PID.--Uses a short launcher Python that spawns a detached child and exits immediately.--#### Computer.python_command--```python-def python_command(self, requirements: Optional[List[str]] = None, venv_name: str = 'default', use_system_python: bool = False, background: bool = False) -> Callable[[Callable[P, R]], Callable[P, Awaitable[R]]]-```--Decorator to execute a Python function remotely in this Computer's venv.--This mirrors `computer.helpers.sandboxed()` but binds to this instance and-optionally ensures required packages are installed before execution.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `requirements` | `Any` | Packages to install in the virtual environment. |-| `venv_name` | `Any` | Name of the virtual environment to use. |-| `use_system_python` | `Any` | If True, use the system Python/pip instead of a venv. |-| `background` | `Any` | If True, run the function detached and return the child PID immediately. |--**Returns:** A decorator that turns a local function into an async callable which runs remotely and returns the function's result.-------## VMProviderType--*Inherits from: StrEnum*--Enum of supported VM provider types.--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `LUME` | `Any` | |-| `LUMIER` | `Any` | |-| `CLOUD` | `Any` | |-| `CLOUDV2` | `Any` | |-| `WINSANDBOX` | `Any` | |-| `DOCKER` | `Any` | |-| `UNKNOWN` | `Any` | |-------## PtyHandle--Lightweight handle to a live PTY session on a remote computer-server.--### Constructor--```python-PtyHandle(self, pid: int, cols: int, rows: int, _iface: 'PtyInterface') -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `pid` | `int` | |-| `cols` | `int` | |-| `rows` | `int` | |--### Methods--#### PtyHandle.send_stdin--```python-async def send_stdin(self, data: bytes) -> None-```--Write *data* to the PTY's stdin.--#### PtyHandle.resize--```python-async def resize(self, cols: int, rows: int) -> None-```--Resize the terminal window.--#### PtyHandle.kill--```python-async def kill(self) -> bool-```--Kill the PTY session process.--#### PtyHandle.disconnect--```python-async def disconnect(self) -> None-```--Close the WebSocket connection without killing the PTY.--The session keeps running on the server; use :meth:`connect` to-re-attach later.--#### PtyHandle.wait--```python-async def wait(self) -> int-```--Block until the PTY session exits and return its exit code.--**Raises:**--- `LookupError` - If the session for this handle's pid is not tracked by the interface (e.g. the handle was never connected, or the interface was recreated after a reconnect).-------## PtyInterface--Async HTTP+WebSocket client for the ``/pty`` endpoints on computer-server.--Args:- base_url: HTTP base URL of the computer-server, e.g.- ``"http://192.168.64.10:8000"``.- api_key: Optional API key for cloud providers (passed as- ``X-API-Key`` header).- vm_name: Optional VM / container name (passed as- ``X-Container-Name`` header).--### Constructor--```python-PtyInterface(self, base_url: str, api_key: Optional[str] = None, vm_name: Optional[str] = None) -> None-```--### Methods--#### PtyInterface.create--```python-async def create(self, command: Optional[str] = None, cols: int = 80, rows: int = 24, on_data: Optional[Callable[[bytes], None]] = None, cwd: Optional[str] = None, envs: Optional[dict] = None, timeout: int = 60) -> PtyHandle-```--Spawn a new PTY session on the remote computer-server.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `command` | `Any` | Shell command (defaults to ``bash`` on the server side). |-| `cols` | `Any` | Terminal width. |-| `rows` | `Any` | Terminal height. |-| `on_data` | `Any` | Callback invoked with raw bytes whenever the PTY produces output. Called from the asyncio event loop. |-| `cwd` | `Any` | Working directory on the remote host. |-| `envs` | `Any` | Extra environment variables for the remote process. |-| `timeout` | `Any` | HTTP request timeout in seconds. |--**Returns:** :class:`PtyHandle` for the new session.--#### PtyInterface.kill--```python-async def kill(self, pid: int) -> bool-```--Kill the remote PTY session *pid*.--#### PtyInterface.resize--```python-async def resize(self, pid: int, cols: int, rows: int) -> None-```--Resize the terminal for the remote PTY session *pid*.--#### PtyInterface.send_stdin--```python-async def send_stdin(self, pid: int, data: bytes) -> None-```--Write *data* to the remote PTY session's stdin via the WebSocket.--If a WebSocket is active the data is sent through it; otherwise falls-back to the HTTP ``/stdin`` endpoint.--#### PtyInterface.connect--```python-async def connect(self, pid: int, on_data: Optional[Callable[[bytes], None]] = None) -> PtyHandle-```--Re-open a WebSocket to an existing PTY session *pid*.--Any previous connection for this pid is cancelled first.-The current terminal dimensions are fetched from the server so that-the returned :class:`PtyHandle` reflects any resizes since creation.-------## tracing--Computer tracing functionality for recording sessions.--This module provides a Computer.tracing API inspired by Playwright's tracing functionality,-allowing users to record computer interactions for debugging, training, and analysis.-------## ComputerTracing--Computer tracing class that records computer interactions and saves them to disk.--This class provides a flexible API for recording computer sessions with configurable-options for what to record (screenshots, API calls, video, etc.).--### Constructor--```python-ComputerTracing(self, computer_instance)-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `is_tracing` | `bool` | Check if tracing is currently active. |--### Methods--#### ComputerTracing.start--```python-async def start(self, config: Optional[Dict[str, Any]] = None) -> None-```--Start tracing with the specified configuration.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `config` | `Any` | Tracing configuration dict with options: - video: bool - Record video frames (default: False) - screenshots: bool - Record screenshots (default: True) - api_calls: bool - Record API calls and results (default: True) - accessibility_tree: bool - Record accessibility tree snapshots (default: False) - metadata: bool - Record custom metadata (default: True) - name: str - Custom trace name (default: auto-generated) - path: str - Custom trace directory path (default: auto-generated) |--#### ComputerTracing.stop--```python-async def stop(self, options: Optional[Dict[str, Any]] = None) -> str-```--Stop tracing and save the trace data.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `options` | `Any` | Stop options dict with: - path: str - Custom output path for the trace archive - format: str - Output format ('zip' or 'dir', default: 'zip') |--**Returns:** str: Path to the saved trace file or directory--#### ComputerTracing.record_api_call--```python-async def record_api_call(self, method: str, args: Dict[str, Any], result: Any = None, error: Optional[Exception] = None) -> None-```--Record an API call event.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `method` | `Any` | The method name that was called |-| `args` | `Any` | Arguments passed to the method |-| `result` | `Any` | Result returned by the method |-| `error` | `Any` | Exception raised by the method, if any |--#### ComputerTracing.record_accessibility_tree--```python-async def record_accessibility_tree(self) -> None-```--Record the current accessibility tree if enabled.--#### ComputerTracing.add_metadata--```python-async def add_metadata(self, key: str, value: Any) -> None-```--Add custom metadata to the trace.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `key` | `Any` | Metadata key |-| `value` | `Any` | Metadata value |-------## models--Models for computer configuration.-------## BaseVMProvider--*Inherits from: AsyncContextManager*--Base interface for VM providers.--All VM provider implementations must implement this interface.--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `provider_type` | `VMProviderType` | Get the provider type. |--### Methods--#### BaseVMProvider.get_vm--```python-async def get_vm(self, name: str, storage: Optional[str] = None) -> Dict[str, Any]-```--Get VM information by name.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `name` | `Any` | Name of the VM to get information for |-| `storage` | `Any` | Optional storage path override. If provided, this will be used instead of the provider's default storage path. |--**Returns:** Dictionary with VM information including status, IP address, etc.--#### BaseVMProvider.list_vms--```python-async def list_vms(self) -> ListVMsResponse-```--List all available VMs.--**Returns:** ListVMsResponse: A list of minimal VM objects as defined in `computer.providers.types.MinimalVM`.--#### BaseVMProvider.run_vm--```python-async def run_vm(self, image: str, name: str, run_opts: Dict[str, Any], storage: Optional[str] = None) -> Dict[str, Any]-```--Run a VM by name with the given options.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `image` | `Any` | Name/tag of the image to use |-| `name` | `Any` | Name of the VM to run |-| `run_opts` | `Any` | Dictionary of run options (memory, cpu, etc.) |-| `storage` | `Any` | Optional storage path override. If provided, this will be used instead of the provider's default storage path. |--**Returns:** Dictionary with VM run status and information--#### BaseVMProvider.stop_vm--```python-async def stop_vm(self, name: str, storage: Optional[str] = None) -> Dict[str, Any]-```--Stop a VM by name.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `name` | `Any` | Name of the VM to stop |-| `storage` | `Any` | Optional storage path override. If provided, this will be used instead of the provider's default storage path. |--**Returns:** Dictionary with VM stop status and information--#### BaseVMProvider.restart_vm--```python-async def restart_vm(self, name: str, storage: Optional[str] = None) -> Dict[str, Any]-```--Restart a VM by name.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `name` | `Any` | Name of the VM to restart |-| `storage` | `Any` | Optional storage path override. If provided, this will be used instead of the provider's default storage path. |--**Returns:** Dictionary with VM restart status and information--#### BaseVMProvider.update_vm--```python-async def update_vm(self, name: str, update_opts: Dict[str, Any], storage: Optional[str] = None) -> Dict[str, Any]-```--Update VM configuration.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `name` | `Any` | Name of the VM to update |-| `update_opts` | `Any` | Dictionary of update options (memory, cpu, etc.) |-| `storage` | `Any` | Optional storage path override. If provided, this will be used instead of the provider's default storage path. |--**Returns:** Dictionary with VM update status and information--#### BaseVMProvider.get_ip--```python-async def get_ip(self, name: str, storage: Optional[str] = None, retry_delay: int = 2) -> str-```--Get the IP address of a VM, waiting indefinitely until it's available.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `name` | `Any` | Name of the VM to get the IP for |-| `storage` | `Any` | Optional storage path override. If provided, this will be used instead of the provider's default storage path. |-| `retry_delay` | `Any` | Delay between retries in seconds (default: 2) |--**Returns:** IP address of the VM when it becomes available-------## Display--Display configuration.--### Constructor--```python-Display(self, width: int, height: int) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `width` | `int` | |-| `height` | `int` | |-------## Image--VM image configuration.--### Constructor--```python-Image(self, image: str, tag: str, name: str) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `image` | `str` | |-| `tag` | `str` | |-| `name` | `str` | |-------## Computer--Computer configuration.--### Constructor--```python-Computer(self, image: str, tag: str, name: str, display: Display, memory: str, cpu: str, vm_provider: Optional[BaseVMProvider] = None) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `image` | `str` | |-| `tag` | `str` | |-| `name` | `str` | |-| `display` | `Display` | |-| `memory` | `str` | |-| `cpu` | `str` | |-| `vm_provider` | `Optional[BaseVMProvider]` | |--### Methods--#### Computer.get_ip--```python-async def get_ip(self) -> Optional[str]-```--Get the IP address of the VM.-------## diorama_computer-------## Key--*Inherits from: Enum*--Keyboard keys that can be used with press_key.--These key names follow a consistent cross-platform keyboard key naming convention.--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `PAGE_DOWN` | `Any` | |-| `PAGE_UP` | `Any` | |-| `HOME` | `Any` | |-| `END` | `Any` | |-| `LEFT` | `Any` | |-| `RIGHT` | `Any` | |-| `UP` | `Any` | |-| `DOWN` | `Any` | |-| `RETURN` | `Any` | |-| `ENTER` | `Any` | |-| `ESCAPE` | `Any` | |-| `ESC` | `Any` | |-| `TAB` | `Any` | |-| `SPACE` | `Any` | |-| `BACKSPACE` | `Any` | |-| `DELETE` | `Any` | |-| `ALT` | `Any` | |-| `CTRL` | `Any` | |-| `SHIFT` | `Any` | |-| `WIN` | `Any` | |-| `COMMAND` | `Any` | |-| `OPTION` | `Any` | |-| `F1` | `Any` | |-| `F2` | `Any` | |-| `F3` | `Any` | |-| `F4` | `Any` | |-| `F5` | `Any` | |-| `F6` | `Any` | |-| `F7` | `Any` | |-| `F8` | `Any` | |-| `F9` | `Any` | |-| `F10` | `Any` | |-| `F11` | `Any` | |-| `F12` | `Any` | |--### Methods--#### Key.from_string--```python-def from_string(cls, key: str) -> Key | str-```--Convert a string key name to a Key enum value.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `key` | `Any` | String key name to convert |--**Returns:** Key enum value if the string matches a known key, otherwise returns the original string for single character keys-------## DioramaComputer--A Computer-compatible proxy for Diorama that sends commands over the ComputerInterface.--### Constructor--```python-DioramaComputer(self, computer, apps)-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `computer` | `Any` | |-| `apps` | `Any` | |-| `interface` | `Any` | |--### Methods--#### DioramaComputer.run--```python-async def run(self)-```--Initialize and run the DioramaComputer if not already initialized.--**Returns:** self: The DioramaComputer instance-------## DioramaComputerInterface--Diorama Interface proxy that sends diorama_cmds via the Computer's interface.--### Constructor--```python-DioramaComputerInterface(self, computer, apps)-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `computer` | `Any` | |-| `apps` | `Any` | |--### Methods--#### DioramaComputerInterface.screenshot--```python-async def screenshot(self, as_bytes = True)-```--Take a screenshot of the diorama scene.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `as_bytes` | `bool` | If True, return image as bytes; if False, return PIL Image object |--**Returns:** bytes or PIL.Image: Screenshot data in the requested format--#### DioramaComputerInterface.get_screen_size--```python-async def get_screen_size(self)-```--Get the dimensions of the diorama scene.--**Returns:** dict: Dictionary containing 'width' and 'height' keys with pixel dimensions--#### DioramaComputerInterface.move_cursor--```python-async def move_cursor(self, x, y)-```--Move the cursor to the specified coordinates.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `x` | `int` | X coordinate to move cursor to |-| `y` | `int` | Y coordinate to move cursor to |--#### DioramaComputerInterface.left_click--```python-async def left_click(self, x = None, y = None)-```--Perform a left mouse click at the specified coordinates or current cursor position.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `x` | `int, optional` | X coordinate to click at. If None, clicks at current cursor position |-| `y` | `int, optional` | Y coordinate to click at. If None, clicks at current cursor position |--#### DioramaComputerInterface.right_click--```python-async def right_click(self, x = None, y = None)-```--Perform a right mouse click at the specified coordinates or current cursor position.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `x` | `int, optional` | X coordinate to click at. If None, clicks at current cursor position |-| `y` | `int, optional` | Y coordinate to click at. If None, clicks at current cursor position |--#### DioramaComputerInterface.double_click--```python-async def double_click(self, x = None, y = None)-```--Perform a double mouse click at the specified coordinates or current cursor position.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `x` | `int, optional` | X coordinate to double-click at. If None, clicks at current cursor position |-| `y` | `int, optional` | Y coordinate to double-click at. If None, clicks at current cursor position |--#### DioramaComputerInterface.scroll_up--```python-async def scroll_up(self, clicks = 1)-```--Scroll up by the specified number of clicks.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `clicks` | `int` | Number of scroll clicks to perform upward. Defaults to 1 |--#### DioramaComputerInterface.scroll_down--```python-async def scroll_down(self, clicks = 1)-```--Scroll down by the specified number of clicks.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `clicks` | `int` | Number of scroll clicks to perform downward. Defaults to 1 |--#### DioramaComputerInterface.drag_to--```python-async def drag_to(self, x, y, duration = 0.5)-```--Drag from the current cursor position to the specified coordinates.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `x` | `int` | X coordinate to drag to |-| `y` | `int` | Y coordinate to drag to |-| `duration` | `float` | Duration of the drag operation in seconds. Defaults to 0.5 |--#### DioramaComputerInterface.get_cursor_position--```python-async def get_cursor_position(self)-```--Get the current cursor position.--**Returns:** dict: Dictionary containing the current cursor coordinates--#### DioramaComputerInterface.type_text--```python-async def type_text(self, text)-```--Type the specified text at the current cursor position.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `text` | `str` | The text to type |--#### DioramaComputerInterface.press_key--```python-async def press_key(self, key)-```--Press a single key.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `key` | `Any` | The key to press |--#### DioramaComputerInterface.hotkey--```python-async def hotkey(self, keys = ())-```--Press multiple keys simultaneously as a hotkey combination.--**Raises:**--- `ValueError` - If any key is not a Key enum or string type--#### DioramaComputerInterface.to_screen_coordinates--```python-async def to_screen_coordinates(self, x, y)-```--Convert coordinates to screen coordinates.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `x` | `int` | X coordinate to convert |-| `y` | `int` | Y coordinate to convert |--**Returns:** dict: Dictionary containing the converted screen coordinates-------## helpers--Helper functions and decorators for the Computer module.-------## DependencyInfo--*Inherits from: TypedDict*--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `import_statements` | `List[str]` | |-| `definitions` | `List[tuple[str, Any]]` | |--### set_default_computer--```python-def set_default_computer(computer: Any) -> None-```--Set the default computer instance to be used by the remote decorator.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `computer` | `Any` | The computer instance to use as default |--### sandboxed--```python-def sandboxed(venv_name: str = 'default', computer: str = 'default', max_retries: int = 3) -> Callable[[Callable[P, R]], Callable[P, Awaitable[R]]]-```--Decorator that wraps a function to be executed remotely via computer.venv_exec--The function is automatically analyzed for dependencies (imports, helper functions,-constants, etc.) and reconstructed with all necessary code in the remote sandbox.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `venv_name` | `Any` | Name of the virtual environment to execute in |-| `computer` | `Any` | The computer instance to use, or "default" to use the globally set default |-| `max_retries` | `Any` | Maximum number of retries for the remote execution |--### generate_source_code--```python-def generate_source_code(func: FunctionType) -> str-```--Generate complete source code for a function with all dependencies.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `func` | `Any` | The function to generate source code for |--**Returns:** Complete Python source code as a string-------## interface--Interface package for Computer SDK.-------## BaseComputerInterface--*Inherits from: ABC*--Base class for computer control interfaces.--### Constructor--```python-BaseComputerInterface(self, ip_address: str, username: str = 'lume', password: str = 'lume', api_key: Optional[str] = None, vm_name: Optional[str] = None)-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `ip_address` | `Any` | |-| `username` | `Any` | |-| `password` | `Any` | |-| `api_key` | `Any` | |-| `vm_name` | `Any` | |-| `logger` | `Any` | |-| `delay` | `float` | |--### Methods--#### BaseComputerInterface.wait_for_ready--```python-async def wait_for_ready(self, timeout: int = 60) -> None-```--Wait for interface to be ready.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `timeout` | `Any` | Maximum time to wait in seconds |--**Raises:**--- `TimeoutError` - If interface is not ready within timeout--#### BaseComputerInterface.close--```python-def close(self) -> None-```--Close the interface connection.--#### BaseComputerInterface.force_close--```python-def force_close(self) -> None-```--Force close the interface connection.--By default, this just calls close(), but subclasses can override-to provide more forceful cleanup.--#### BaseComputerInterface.mouse_down--```python-async def mouse_down(self, x: Optional[int] = None, y: Optional[int] = None, button: MouseButton = 'left', delay: Optional[float] = None) -> None-```--Press and hold a mouse button.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `x` | `Any` | X coordinate to press at. If None, uses current cursor position. |-| `y` | `Any` | Y coordinate to press at. If None, uses current cursor position. |-| `button` | `Any` | Mouse button to press ('left', 'middle', 'right'). |-| `delay` | `Any` | Optional delay in seconds after the action |--#### BaseComputerInterface.mouse_up--```python-async def mouse_up(self, x: Optional[int] = None, y: Optional[int] = None, button: MouseButton = 'left', delay: Optional[float] = None) -> None-```--Release a mouse button.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `x` | `Any` | X coordinate to release at. If None, uses current cursor position. |-| `y` | `Any` | Y coordinate to release at. If None, uses current cursor position. |-| `button` | `Any` | Mouse button to release ('left', 'middle', 'right'). |-| `delay` | `Any` | Optional delay in seconds after the action |--#### BaseComputerInterface.left_click--```python-async def left_click(self, x: Optional[int] = None, y: Optional[int] = None, delay: Optional[float] = None) -> None-```--Perform a left mouse button click.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `x` | `Any` | X coordinate to click at. If None, uses current cursor position. |-| `y` | `Any` | Y coordinate to click at. If None, uses current cursor position. |-| `delay` | `Any` | Optional delay in seconds after the action |--#### BaseComputerInterface.right_click--```python-async def right_click(self, x: Optional[int] = None, y: Optional[int] = None, delay: Optional[float] = None) -> None-```--Perform a right mouse button click.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `x` | `Any` | X coordinate to click at. If None, uses current cursor position. |-| `y` | `Any` | Y coordinate to click at. If None, uses current cursor position. |-| `delay` | `Any` | Optional delay in seconds after the action |--#### BaseComputerInterface.double_click--```python-async def double_click(self, x: Optional[int] = None, y: Optional[int] = None, delay: Optional[float] = None) -> None-```--Perform a double left mouse button click.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `x` | `Any` | X coordinate to double-click at. If None, uses current cursor position. |-| `y` | `Any` | Y coordinate to double-click at. If None, uses current cursor position. |-| `delay` | `Any` | Optional delay in seconds after the action |--#### BaseComputerInterface.move_cursor--```python-async def move_cursor(self, x: int, y: int, delay: Optional[float] = None) -> None-```--Move the cursor to the specified screen coordinates.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `x` | `Any` | X coordinate to move cursor to. |-| `y` | `Any` | Y coordinate to move cursor to. |-| `delay` | `Any` | Optional delay in seconds after the action |--#### BaseComputerInterface.drag_to--```python-async def drag_to(self, x: int, y: int, button: str = 'left', duration: float = 0.5, delay: Optional[float] = None) -> None-```--Drag from current position to specified coordinates.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `x` | `Any` | The x coordinate to drag to |-| `y` | `Any` | The y coordinate to drag to |-| `button` | `Any` | The mouse button to use ('left', 'middle', 'right') |-| `duration` | `Any` | How long the drag should take in seconds |-| `delay` | `Any` | Optional delay in seconds after the action |--#### BaseComputerInterface.drag--```python-async def drag(self, path: List[Tuple[int, int]], button: str = 'left', duration: float = 0.5, delay: Optional[float] = None) -> None-```--Drag the cursor along a path of coordinates.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `path` | `Any` | List of (x, y) coordinate tuples defining the drag path |-| `button` | `Any` | The mouse button to use ('left', 'middle', 'right') |-| `duration` | `Any` | Total time in seconds that the drag operation should take |-| `delay` | `Any` | Optional delay in seconds after the action |--#### BaseComputerInterface.key_down--```python-async def key_down(self, key: str, delay: Optional[float] = None) -> None-```--Press and hold a key.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `key` | `Any` | The key to press and hold (e.g., 'a', 'shift', 'ctrl'). |-| `delay` | `Any` | Optional delay in seconds after the action. |--#### BaseComputerInterface.key_up--```python-async def key_up(self, key: str, delay: Optional[float] = None) -> None-```--Release a previously pressed key.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `key` | `Any` | The key to release (e.g., 'a', 'shift', 'ctrl'). |-| `delay` | `Any` | Optional delay in seconds after the action. |--#### BaseComputerInterface.type_text--```python-async def type_text(self, text: str, delay: Optional[float] = None) -> None-```--Type the specified text string.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `text` | `Any` | The text string to type. |-| `delay` | `Any` | Optional delay in seconds after the action. |--#### BaseComputerInterface.press_key--```python-async def press_key(self, key: str, delay: Optional[float] = None) -> None-```--Press and release a single key.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `key` | `Any` | The key to press (e.g., 'a', 'enter', 'escape'). |-| `delay` | `Any` | Optional delay in seconds after the action. |--#### BaseComputerInterface.hotkey--```python-async def hotkey(self, keys: str = (), delay: Optional[float] = None) -> None-```--Press multiple keys simultaneously (keyboard shortcut).--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `delay` | `Any` | Optional delay in seconds after the action. |--#### BaseComputerInterface.scroll--```python-async def scroll(self, x: int, y: int, delay: Optional[float] = None) -> None-```--Scroll the mouse wheel by specified amounts.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `x` | `Any` | Horizontal scroll amount (positive = right, negative = left). |-| `y` | `Any` | Vertical scroll amount (positive = up, negative = down). |-| `delay` | `Any` | Optional delay in seconds after the action. |--#### BaseComputerInterface.scroll_down--```python-async def scroll_down(self, clicks: int = 1, delay: Optional[float] = None) -> None-```--Scroll down by the specified number of clicks.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `clicks` | `Any` | Number of scroll clicks to perform downward. |-| `delay` | `Any` | Optional delay in seconds after the action. |--#### BaseComputerInterface.scroll_up--```python-async def scroll_up(self, clicks: int = 1, delay: Optional[float] = None) -> None-```--Scroll up by the specified number of clicks.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `clicks` | `Any` | Number of scroll clicks to perform upward. |-| `delay` | `Any` | Optional delay in seconds after the action. |--#### BaseComputerInterface.screenshot--```python-async def screenshot(self) -> bytes-```--Take a screenshot.--**Returns:** Raw bytes of the screenshot image--#### BaseComputerInterface.get_screen_size--```python-async def get_screen_size(self) -> Dict[str, int]-```--Get the screen dimensions.--**Returns:** Dict with 'width' and 'height' keys--#### BaseComputerInterface.get_cursor_position--```python-async def get_cursor_position(self) -> Dict[str, int]-```--Get the current cursor position on screen.--**Returns:** Dict with 'x' and 'y' keys containing cursor coordinates.--#### BaseComputerInterface.copy_to_clipboard--```python-async def copy_to_clipboard(self) -> str-```--Get the current clipboard content.--**Returns:** The text content currently stored in the clipboard.--#### BaseComputerInterface.set_clipboard--```python-async def set_clipboard(self, text: str) -> None-```--Set the clipboard content to the specified text.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `text` | `Any` | The text to store in the clipboard. |--#### BaseComputerInterface.file_exists--```python-async def file_exists(self, path: str) -> bool-```--Check if a file exists at the specified path.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `path` | `Any` | The file path to check. |--**Returns:** True if the file exists, False otherwise.--#### BaseComputerInterface.directory_exists--```python-async def directory_exists(self, path: str) -> bool-```--Check if a directory exists at the specified path.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `path` | `Any` | The directory path to check. |--**Returns:** True if the directory exists, False otherwise.--#### BaseComputerInterface.list_dir--```python-async def list_dir(self, path: str) -> List[str]-```--List the contents of a directory.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `path` | `Any` | The directory path to list. |--**Returns:** List of file and directory names in the specified directory.--#### BaseComputerInterface.read_text--```python-async def read_text(self, path: str) -> str-```--Read the text contents of a file.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `path` | `Any` | The file path to read from. |--**Returns:** The text content of the file.--#### BaseComputerInterface.write_text--```python-async def write_text(self, path: str, content: str) -> None-```--Write text content to a file.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `path` | `Any` | The file path to write to. |-| `content` | `Any` | The text content to write. |--#### BaseComputerInterface.read_bytes--```python-async def read_bytes(self, path: str, offset: int = 0, length: Optional[int] = None) -> bytes-```--Read file binary contents with optional seeking support.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `path` | `Any` | Path to the file |-| `offset` | `Any` | Byte offset to start reading from (default: 0) |-| `length` | `Any` | Number of bytes to read (default: None for entire file) |--#### BaseComputerInterface.write_bytes--```python-async def write_bytes(self, path: str, content: bytes) -> None-```--Write binary content to a file.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `path` | `Any` | The file path to write to. |-| `content` | `Any` | The binary content to write. |--#### BaseComputerInterface.delete_file--```python-async def delete_file(self, path: str) -> None-```--Delete a file at the specified path.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `path` | `Any` | The file path to delete. |--#### BaseComputerInterface.create_dir--```python-async def create_dir(self, path: str) -> None-```--Create a directory at the specified path.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `path` | `Any` | The directory path to create. |--#### BaseComputerInterface.delete_dir--```python-async def delete_dir(self, path: str) -> None-```--Delete a directory at the specified path.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `path` | `Any` | The directory path to delete. |--#### BaseComputerInterface.get_file_size--```python-async def get_file_size(self, path: str) -> int-```--Get the size of a file in bytes.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `path` | `Any` | The file path to get the size of. |--**Returns:** The size of the file in bytes.--#### BaseComputerInterface.get_desktop_environment--```python-async def get_desktop_environment(self) -> str-```--Get the current desktop environment.--**Returns:** The name of the current desktop environment.--#### BaseComputerInterface.set_wallpaper--```python-async def set_wallpaper(self, path: str) -> None-```--Set the desktop wallpaper to the specified path.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `path` | `Any` | The file path to set as wallpaper |--#### BaseComputerInterface.open--```python-async def open(self, target: str) -> None-```--Open a target using the system's default handler.--Typically opens files, folders, or URLs with the associated application.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `target` | `Any` | The file path, folder path, or URL to open. |--#### BaseComputerInterface.launch--```python-async def launch(self, app: str, args: List[str] | None = None) -> Optional[int]-```--Launch an application with optional arguments.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `app` | `Any` | The application executable or bundle identifier. |-| `args` | `Any` | Optional list of arguments to pass to the application. |--**Returns:** Optional process ID (PID) of the launched application if available, otherwise None.--#### BaseComputerInterface.get_current_window_id--```python-async def get_current_window_id(self) -> int | str-```--Get the identifier of the currently active/focused window.--**Returns:** A window identifier that can be used with other window management methods.--#### BaseComputerInterface.get_application_windows--```python-async def get_application_windows(self, app: str) -> List[int | str]-```--Get all window identifiers for a specific application.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `app` | `Any` | The application name, executable, or identifier to query. |--**Returns:** A list of window identifiers belonging to the specified application.--#### BaseComputerInterface.get_window_name--```python-async def get_window_name(self, window_id: int | str) -> str-```--Get the title/name of a window.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `window_id` | `Any` | The window identifier. |--**Returns:** The window's title or name string.--#### BaseComputerInterface.get_window_size--```python-async def get_window_size(self, window_id: int | str) -> tuple[int, int]-```--Get the size of a window in pixels.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `window_id` | `Any` | The window identifier. |--**Returns:** A tuple of (width, height) representing the window size in pixels.--#### BaseComputerInterface.get_window_position--```python-async def get_window_position(self, window_id: int | str) -> tuple[int, int]-```--Get the screen position of a window.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `window_id` | `Any` | The window identifier. |--**Returns:** A tuple of (x, y) representing the window's top-left corner in screen coordinates.--#### BaseComputerInterface.set_window_size--```python-async def set_window_size(self, window_id: int | str, width: int, height: int) -> None-```--Set the size of a window in pixels.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `window_id` | `Any` | The window identifier. |-| `width` | `Any` | Desired width in pixels. |-| `height` | `Any` | Desired height in pixels. |--#### BaseComputerInterface.set_window_position--```python-async def set_window_position(self, window_id: int | str, x: int, y: int) -> None-```--Move a window to a specific position on the screen.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `window_id` | `Any` | The window identifier. |-| `x` | `Any` | X coordinate for the window's top-left corner. |-| `y` | `Any` | Y coordinate for the window's top-left corner. |--#### BaseComputerInterface.maximize_window--```python-async def maximize_window(self, window_id: int | str) -> None-```--Maximize a window.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `window_id` | `Any` | The window identifier. |--#### BaseComputerInterface.minimize_window--```python-async def minimize_window(self, window_id: int | str) -> None-```--Minimize a window.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `window_id` | `Any` | The window identifier. |--#### BaseComputerInterface.activate_window--```python-async def activate_window(self, window_id: int | str) -> None-```--Bring a window to the foreground and focus it.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `window_id` | `Any` | The window identifier. |--#### BaseComputerInterface.close_window--```python-async def close_window(self, window_id: int | str) -> None-```--Close a window.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `window_id` | `Any` | The window identifier. |--#### BaseComputerInterface.get_window_title--```python-async def get_window_title(self, window_id: int | str) -> str-```--Convenience alias for get_window_name().--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `window_id` | `Any` | The window identifier. |--**Returns:** The window's title or name string.--#### BaseComputerInterface.window_size--```python-async def window_size(self, window_id: int | str) -> tuple[int, int]-```--Convenience alias for get_window_size().--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `window_id` | `Any` | The window identifier. |--**Returns:** A tuple of (width, height) representing the window size in pixels.--#### BaseComputerInterface.run_command--```python-async def run_command(self, command: str) -> CommandResult-```--Run shell command and return structured result.--Executes a shell command using subprocess.run with shell=True and check=False.-The command is run in the target environment and captures both stdout and stderr.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `command` | `str` | The shell command to execute |--**Returns:** CommandResult: A structured result containing: - stdout (str): Standard output from the command - stderr (str): Standard error from the command - returncode (int): Exit code from the command (0 indicates success)--**Raises:**--- `RuntimeError` - If the command execution fails at the system level--**Example:**--```python-result = await interface.run_command("ls -la")-if result.returncode == 0:- print(f"Output: {result.stdout}")-else:- print(f"Error: {result.stderr}, Exit code: {result.returncode}")-```--#### BaseComputerInterface.get_accessibility_tree--```python-async def get_accessibility_tree(self) -> Dict-```--Get the accessibility tree of the current screen.--**Returns:** Dict containing the hierarchical accessibility information of screen elements.--#### BaseComputerInterface.to_screen_coordinates--```python-async def to_screen_coordinates(self, x: float, y: float) -> tuple[float, float]-```--Convert screenshot coordinates to screen coordinates.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `x` | `Any` | X coordinate in screenshot space |-| `y` | `Any` | Y coordinate in screenshot space |--**Returns:** tuple[float, float]: (x, y) coordinates in screen space--#### BaseComputerInterface.to_screenshot_coordinates--```python-async def to_screenshot_coordinates(self, x: float, y: float) -> tuple[float, float]-```--Convert screen coordinates to screenshot coordinates.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `x` | `Any` | X coordinate in screen space |-| `y` | `Any` | Y coordinate in screen space |--**Returns:** tuple[float, float]: (x, y) coordinates in screenshot space-------## InterfaceFactory--Factory for creating OS-specific computer interfaces.--### Methods--#### InterfaceFactory.create_interface_for_os--```python-def create_interface_for_os(os: OSType, ip_address: str, api_port: Optional[int] = None, api_key: Optional[str] = None, vm_name: Optional[str] = None) -> BaseComputerInterface-```--Create an interface for the specified OS.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `os` | `Any` | Operating system type ('macos', 'linux', or 'windows') |-| `ip_address` | `Any` | IP address of the computer to control |-| `api_port` | `Any` | Optional API port of the computer to control |-| `api_key` | `Any` | Optional API key for cloud authentication |-| `vm_name` | `Any` | Optional VM name for cloud authentication |--**Returns:** BaseComputerInterface: The appropriate interface for the OS--**Raises:**--- `ValueError` - If the OS type is not supported-------## MacOSComputerInterface--*Inherits from: GenericComputerInterface*--Interface for macOS.--### Constructor--```python-MacOSComputerInterface(self, ip_address: str, username: str = 'lume', password: str = 'lume', api_key: Optional[str] = None, vm_name: Optional[str] = None, api_port: Optional[int] = None)-```--### Methods--#### MacOSComputerInterface.diorama_cmd--```python-async def diorama_cmd(self, action: str, arguments: Optional[dict] = None) -> dict-```--Send a diorama command to the server (macOS only).diff --git a/docs/content/docs/cua/reference/computer-sdk/meta.json b/docs/content/docs/cua/reference/computer-sdk/meta.jsondeleted file mode 100644index 5ae7deabe4..0000000000--- a/docs/content/docs/cua/reference/computer-sdk/meta.json+++ /dev/null@@ -1,6 +0,0 @@-{- "title": "Computer SDK",- "description": "Python API for sandboxed desktop environments",- "icon": "Monitor",- "pages": ["changelog"]-}diff --git a/docs/content/docs/cua/reference/computer-sdk/v0.3/api.mdx b/docs/content/docs/cua/reference/computer-sdk/v0.3/api.mdxdeleted file mode 100644index eb129dc5da..0000000000--- a/docs/content/docs/cua/reference/computer-sdk/v0.3/api.mdx+++ /dev/null@@ -1,209 +0,0 @@-----title: Computer SDK v0.3 API Reference-description: API reference for Computer SDK version 0.3------{/*-AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY-Generated by: npx tsx scripts/docs-generators/generate-versioned-docs.ts-Source tag: computer-v0.3.7-Version: 0.3.7-*/}--import { Callout } from 'fumadocs-ui/components/callout';--<Callout type="warn">- This is documentation for **v0.3**. [View latest version](/cua/reference/computer-sdk).-</Callout>--<div className="flex items-center gap-2 mb-6">- <span className="px-2 py-1 bg-amber-100 dark:bg-amber-900 text-amber-800 dark:text-amber-200 rounded text-sm font-mono">- v0.3.7- </span>- <span className="text-sm text-fd-muted-foreground">pip install cua-computer==0.3.7</span>-</div>--Cua Computer Interface for cross-platform computer control.--## Classes--| Class | Description |-| ---------------- | ------------------------------------------------------------- |-| `Computer` | Computer is the main class for interacting with the computer. |-| `VMProviderType` | Enum of supported VM provider types. |--## Computer--Computer is the main class for interacting with the computer.--### Methods--#### Computer.create_desktop_from_apps--```python-def create_desktop_from_apps(self, apps)-```--Create a virtual desktop from a list of app names, returning a DioramaComputer-that proxies Diorama.Interface but uses diorama_cmds via the computer interface.--Args:-apps (list[str]): List of application names to include in the desktop.-Returns:-DioramaComputer: A proxy object with the Diorama interface, but using diorama_cmds.--#### Computer.run--```python-async def run(self) -> Optional[str]-```--Initialize the VM and computer interface.--#### Computer.disconnect--```python-async def disconnect(self) -> None-```--Disconnect from the computer's WebSocket interface.--#### Computer.stop--```python-async def stop(self) -> None-```--Disconnect from the computer's WebSocket interface and stop the computer.--#### Computer.get_ip--```python-async def get_ip(self, max_retries: int = 15, retry_delay: int = 3) -> str-```--Get the IP address of the VM or localhost if using host computer server.--This method delegates to the provider's get_ip method, which waits indefinitely-until the VM has a valid IP address.--Args:-max_retries: Unused parameter, kept for backward compatibility-retry_delay: Delay between retries in seconds (default: 2)--Returns:-IP address of the VM or localhost if using host computer server--#### Computer.wait_vm_ready--```python-async def wait_vm_ready(self) -> Optional[Dict[str, Any]]-```--Wait for VM to be ready with an IP address.--Returns:-VM status information or None if using host computer server.--#### Computer.update--```python-async def update(self, cpu: Optional[int] = None, memory: Optional[str] = None)-```--Update VM settings.--#### Computer.get_screenshot_size--```python-def get_screenshot_size(self, screenshot: bytes) -> Dict[str, int]-```--Get the dimensions of a screenshot.--Args:-screenshot: The screenshot bytes--Returns:-Dict[str, int]: Dictionary containing 'width' and 'height' of the image--#### Computer.to_screen_coordinates--```python-async def to_screen_coordinates(self, x: float, y: float) -> tuple[float, float]-```--Convert normalized coordinates to screen coordinates.--Args:-x: X coordinate between 0 and 1-y: Y coordinate between 0 and 1--Returns:-tuple[float, float]: Screen coordinates (x, y)--#### Computer.to_screenshot_coordinates--```python-async def to_screenshot_coordinates(self, x: float, y: float) -> tuple[float, float]-```--Convert screen coordinates to screenshot coordinates.--Args:-x: X coordinate in screen space-y: Y coordinate in screen space--Returns:-tuple[float, float]: (x, y) coordinates in screenshot space--#### Computer.venv_install--```python-async def venv_install(self, venv_name: str, requirements: list[str])-```--Install packages in a virtual environment.--Args:-venv_name: Name of the virtual environment-requirements: List of package requirements to install--Returns:-Tuple of (stdout, stderr) from the installation command--#### Computer.venv_cmd--```python-async def venv_cmd(self, venv_name: str, command: str)-```--Execute a shell command in a virtual environment.--Args:-venv_name: Name of the virtual environment-command: Shell command to execute in the virtual environment--Returns:-Tuple of (stdout, stderr) from the command execution--#### Computer.venv_exec--```python-async def venv_exec(self, venv_name: str, python_func, args = (), kwargs = {})-```--Execute Python function in a virtual environment using source code extraction.--Args:-venv_name: Name of the virtual environment-python_func: A callable function to execute-\*args: Positional arguments to pass to the function-\*\*kwargs: Keyword arguments to pass to the function--Returns:-The result of the function execution, or raises any exception that occurred--## VMProviderType--Enum of supported VM provider types.diff --git a/docs/content/docs/cua/reference/computer-sdk/v0.3/meta.json b/docs/content/docs/cua/reference/computer-sdk/v0.3/meta.jsondeleted file mode 100644index 1d882707c5..0000000000--- a/docs/content/docs/cua/reference/computer-sdk/v0.3/meta.json+++ /dev/null@@ -1,5 +0,0 @@-{- "title": "v0.3",- "description": "Computer SDK v0.3 API Reference",- "pages": ["api"]-}diff --git a/docs/content/docs/cua/reference/computer-sdk/v0.4/api.mdx b/docs/content/docs/cua/reference/computer-sdk/v0.4/api.mdxdeleted file mode 100644index a32d7d2f8d..0000000000--- a/docs/content/docs/cua/reference/computer-sdk/v0.4/api.mdx+++ /dev/null@@ -1,328 +0,0 @@-----title: Computer SDK v0.4 API Reference-description: API reference for Computer SDK version 0.4------{/*-AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY-Generated by: npx tsx scripts/docs-generators/generate-versioned-docs.ts-Source tag: computer-v0.4.19-Version: 0.4.19-*/}--import { Callout } from 'fumadocs-ui/components/callout';--<Callout type="warn">- This is documentation for **v0.4**. [View latest version](/cua/reference/computer-sdk).-</Callout>--<div className="flex items-center gap-2 mb-6">- <span className="px-2 py-1 bg-amber-100 dark:bg-amber-900 text-amber-800 dark:text-amber-200 rounded text-sm font-mono">- v0.4.19- </span>- <span className="text-sm text-fd-muted-foreground">pip install cua-computer==0.4.19</span>-</div>--Cua Computer Interface for cross-platform computer control.--## Classes--| Class | Description |-| ---------------- | ------------------------------------------------------------- |-| `Computer` | Computer is the main class for interacting with the computer. |-| `VMProviderType` | Enum of supported VM provider types. |--## Computer--Computer is the main class for interacting with the computer.--### Methods--#### Computer.create_desktop_from_apps--```python-def create_desktop_from_apps(self, apps)-```--Create a virtual desktop from a list of app names, returning a DioramaComputer-that proxies Diorama.Interface but uses diorama_cmds via the computer interface.--Args:-apps (list[str]): List of application names to include in the desktop.-Returns:-DioramaComputer: A proxy object with the Diorama interface, but using diorama_cmds.--#### Computer.run--```python-async def run(self) -> Optional[str]-```--Initialize the VM and computer interface.--#### Computer.disconnect--```python-async def disconnect(self) -> None-```--Disconnect from the computer's WebSocket interface.--#### Computer.stop--```python-async def stop(self) -> None-```--Disconnect from the computer's WebSocket interface and stop the computer.--#### Computer.start--```python-async def start(self) -> None-```--Start the computer.--#### Computer.restart--```python-async def restart(self) -> None-```--Restart the computer.--If using a VM provider that supports restart, this will issue a restart-without tearing down the provider context, then reconnect the interface.-Falls back to stop()+run() when a provider restart is not available.--#### Computer.get_ip--```python-async def get_ip(self, max_retries: int = 15, retry_delay: int = 3) -> str-```--Get the IP address of the VM or localhost if using host computer server.--This method delegates to the provider's get_ip method, which waits indefinitely-until the VM has a valid IP address.--Args:-max_retries: Unused parameter, kept for backward compatibility-retry_delay: Delay between retries in seconds (default: 2)--Returns:-IP address of the VM or localhost if using host computer server--#### Computer.wait_vm_ready--```python-async def wait_vm_ready(self) -> Optional[Dict[str, Any]]-```--Wait for VM to be ready with an IP address.--Returns:-VM status information or None if using host computer server.--#### Computer.update--```python-async def update(self, cpu: Optional[int] = None, memory: Optional[str] = None)-```--Update VM settings.--#### Computer.get_screenshot_size--```python-def get_screenshot_size(self, screenshot: bytes) -> Dict[str, int]-```--Get the dimensions of a screenshot.--Args:-screenshot: The screenshot bytes--Returns:-Dict[str, int]: Dictionary containing 'width' and 'height' of the image--#### Computer.to_screen_coordinates--```python-async def to_screen_coordinates(self, x: float, y: float) -> tuple[float, float]-```--Convert normalized coordinates to screen coordinates.--Args:-x: X coordinate between 0 and 1-y: Y coordinate between 0 and 1--Returns:-tuple[float, float]: Screen coordinates (x, y)--#### Computer.to_screenshot_coordinates--```python-async def to_screenshot_coordinates(self, x: float, y: float) -> tuple[float, float]-```--Convert screen coordinates to screenshot coordinates.--Args:-x: X coordinate in screen space-y: Y coordinate in screen space--Returns:-tuple[float, float]: (x, y) coordinates in screenshot space--#### Computer.playwright_exec--```python-async def playwright_exec(self, command: str, params: Optional[Dict] = None) -> Dict[str, Any]-```--Execute a Playwright browser command.--Args:-command: The browser command to execute (visit_url, click, type, scroll, web_search)-params: Command parameters--Returns:-Dict containing the command result--Examples: # Navigate to a URL-await computer.playwright_exec("visit_url", \{"url": "https://example.com"\})-- # Click at coordinates- await computer.playwright_exec("click", \{"x": 100, "y": 200\})-- # Type text- await computer.playwright_exec("type", \{"text": "Hello, world!"\})-- # Scroll- await computer.playwright_exec("scroll", \{"delta_x": 0, "delta_y": -100\})-- # Web search- await computer.playwright_exec("web_search", \{"query": "computer use agent"\})--#### Computer.venv_install--```python-async def venv_install(self, venv_name: str, requirements: list[str])-```--Install packages in a virtual environment.--Args:-venv_name: Name of the virtual environment-requirements: List of package requirements to install--Returns:-Tuple of (stdout, stderr) from the installation command--#### Computer.pip_install--```python-async def pip_install(self, requirements: list[str])-```--Install packages using the system Python/pip (no venv).--Args:-requirements: List of package requirements to install globally/user site.--Returns:-Tuple of (stdout, stderr) from the installation command--#### Computer.venv_cmd--```python-async def venv_cmd(self, venv_name: str, command: str)-```--Execute a shell command in a virtual environment.--Args:-venv_name: Name of the virtual environment-command: Shell command to execute in the virtual environment--Returns:-Tuple of (stdout, stderr) from the command execution--#### Computer.venv_exec--```python-async def venv_exec(self, venv_name: str, python_func, args = (), kwargs = {})-```--Execute Python function in a virtual environment using source code extraction.--Args:-venv_name: Name of the virtual environment-python_func: A callable function to execute-\*args: Positional arguments to pass to the function-\*\*kwargs: Keyword arguments to pass to the function--Returns:-The result of the function execution, or raises any exception that occurred--#### Computer.venv_exec_background--```python-async def venv_exec_background(self, venv_name: str, python_func, args = (), requirements: Optional[List[str]] = None, kwargs = {}) -> int-```--Run the Python function in the venv in the background and return the PID.--Uses a short launcher Python that spawns a detached child and exits immediately.--#### Computer.python_exec--```python-async def python_exec(self, python_func, args = (), kwargs = {})-```--Execute a Python function using the system Python (no venv).--Uses source extraction and base64 transport, mirroring venv_exec but-without virtual environment activation.--Returns the function result or raises a reconstructed exception with-remote traceback context appended.--#### Computer.python_exec_background--```python-async def python_exec_background(self, python_func, args = (), requirements: Optional[List[str]] = None, kwargs = {}) -> int-```--Run a Python function with the system interpreter in the background and return PID.--Uses a short launcher Python that spawns a detached child and exits immediately.--#### Computer.python_command--```python-def python_command(self, requirements: Optional[List[str]] = None, venv_name: str = 'default', use_system_python: bool = False, background: bool = False) -> Callable[[Callable[P, R]], Callable[P, Awaitable[R]]]-```--Decorator to execute a Python function remotely in this Computer's venv.--This mirrors `computer.helpers.sandboxed()` but binds to this instance and-optionally ensures required packages are installed before execution.--Args:-requirements: Packages to install in the virtual environment.-venv_name: Name of the virtual environment to use.-use_system_python: If True, use the system Python/pip instead of a venv.-background: If True, run the function detached and return the child PID immediately.--Returns:-A decorator that turns a local function into an async callable which-runs remotely and returns the function's result.--## VMProviderType--Enum of supported VM provider types.diff --git a/docs/content/docs/cua/reference/computer-sdk/v0.4/meta.json b/docs/content/docs/cua/reference/computer-sdk/v0.4/meta.jsondeleted file mode 100644index c526fd3ce1..0000000000--- a/docs/content/docs/cua/reference/computer-sdk/v0.4/meta.json+++ /dev/null@@ -1,5 +0,0 @@-{- "title": "v0.4",- "description": "Computer SDK v0.4 API Reference",- "pages": ["api"]-}diff --git a/docs/content/docs/cua/reference/computer-sdk/v0.5/api.mdx b/docs/content/docs/cua/reference/computer-sdk/v0.5/api.mdxdeleted file mode 100644index 8bb1dc3488..0000000000--- a/docs/content/docs/cua/reference/computer-sdk/v0.5/api.mdx+++ /dev/null@@ -1,328 +0,0 @@-----title: Computer SDK v0.5 API Reference-description: API reference for Computer SDK version 0.5------{/*-AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY-Generated by: npx tsx scripts/docs-generators/generate-versioned-docs.ts-Source tag: computer-v0.5.12-Version: 0.5.12-*/}--import { Callout } from 'fumadocs-ui/components/callout';--<Callout type="warn">- This is documentation for **v0.5**. [View latest version](/cua/reference/computer-sdk).-</Callout>--<div className="flex items-center gap-2 mb-6">- <span className="px-2 py-1 bg-amber-100 dark:bg-amber-900 text-amber-800 dark:text-amber-200 rounded text-sm font-mono">- v0.5.12- </span>- <span className="text-sm text-fd-muted-foreground">pip install cua-computer==0.5.12</span>-</div>--Cua Computer Interface for cross-platform computer control.--## Classes--| Class | Description |-| ---------------- | ------------------------------------------------------------- |-| `Computer` | Computer is the main class for interacting with the computer. |-| `VMProviderType` | Enum of supported VM provider types. |--## Computer--Computer is the main class for interacting with the computer.--### Methods--#### Computer.create_desktop_from_apps--```python-def create_desktop_from_apps(self, apps)-```--Create a virtual desktop from a list of app names, returning a DioramaComputer-that proxies Diorama.Interface but uses diorama_cmds via the computer interface.--Args:-apps (list[str]): List of application names to include in the desktop.-Returns:-DioramaComputer: A proxy object with the Diorama interface, but using diorama_cmds.--#### Computer.run--```python-async def run(self) -> Optional[str]-```--Initialize the VM and computer interface.--#### Computer.disconnect--```python-async def disconnect(self) -> None-```--Disconnect from the computer's WebSocket interface.--#### Computer.stop--```python-async def stop(self) -> None-```--Disconnect from the computer's WebSocket interface and stop the computer.--#### Computer.start--```python-async def start(self) -> None-```--Start the computer.--#### Computer.restart--```python-async def restart(self) -> None-```--Restart the computer.--If using a VM provider that supports restart, this will issue a restart-without tearing down the provider context, then reconnect the interface.-Falls back to stop()+run() when a provider restart is not available.--#### Computer.get_ip--```python-async def get_ip(self, max_retries: int = 15, retry_delay: int = 3) -> str-```--Get the IP address of the VM or localhost if using host computer server.--This method delegates to the provider's get_ip method, which waits indefinitely-until the VM has a valid IP address.--Args:-max_retries: Unused parameter, kept for backward compatibility-retry_delay: Delay between retries in seconds (default: 2)--Returns:-IP address of the VM or localhost if using host computer server--#### Computer.wait_vm_ready--```python-async def wait_vm_ready(self) -> Optional[Dict[str, Any]]-```--Wait for VM to be ready with an IP address.--Returns:-VM status information or None if using host computer server.--#### Computer.update--```python-async def update(self, cpu: Optional[int] = None, memory: Optional[str] = None)-```--Update VM settings.--#### Computer.get_screenshot_size--```python-def get_screenshot_size(self, screenshot: bytes) -> Dict[str, int]-```--Get the dimensions of a screenshot.--Args:-screenshot: The screenshot bytes--Returns:-Dict[str, int]: Dictionary containing 'width' and 'height' of the image--#### Computer.to_screen_coordinates--```python-async def to_screen_coordinates(self, x: float, y: float) -> tuple[float, float]-```--Convert normalized coordinates to screen coordinates.--Args:-x: X coordinate between 0 and 1-y: Y coordinate between 0 and 1--Returns:-tuple[float, float]: Screen coordinates (x, y)--#### Computer.to_screenshot_coordinates--```python-async def to_screenshot_coordinates(self, x: float, y: float) -> tuple[float, float]-```--Convert screen coordinates to screenshot coordinates.--Args:-x: X coordinate in screen space-y: Y coordinate in screen space--Returns:-tuple[float, float]: (x, y) coordinates in screenshot space--#### Computer.playwright_exec--```python-async def playwright_exec(self, command: str, params: Optional[Dict] = None) -> Dict[str, Any]-```--Execute a Playwright browser command.--Args:-command: The browser command to execute (visit_url, click, type, scroll, web_search)-params: Command parameters--Returns:-Dict containing the command result--Examples: # Navigate to a URL-await computer.playwright_exec("visit_url", \{"url": "https://example.com"\})-- # Click at coordinates- await computer.playwright_exec("click", \{"x": 100, "y": 200\})-- # Type text- await computer.playwright_exec("type", \{"text": "Hello, world!"\})-- # Scroll- await computer.playwright_exec("scroll", \{"delta_x": 0, "delta_y": -100\})-- # Web search- await computer.playwright_exec("web_search", \{"query": "computer use agent"\})--#### Computer.venv_install--```python-async def venv_install(self, venv_name: str, requirements: list[str])-```--Install packages in a UV project.--Args:-venv_name: Name of the UV project-requirements: List of package requirements to install--Returns:-Tuple of (stdout, stderr) from the installation command--#### Computer.pip_install--```python-async def pip_install(self, requirements: list[str])-```--Install packages using the system Python with UV (no venv).--Args:-requirements: List of package requirements to install globally/user site.--Returns:-Tuple of (stdout, stderr) from the installation command--#### Computer.venv_cmd--```python-async def venv_cmd(self, venv_name: str, command: str)-```--Execute a shell command in a UV project.--Args:-venv_name: Name of the UV project-command: Shell command to execute in the UV project--Returns:-Tuple of (stdout, stderr) from the command execution--#### Computer.venv_exec--```python-async def venv_exec(self, venv_name: str, python_func, args = (), kwargs = {})-```--Execute Python function in a virtual environment using source code extraction.--Args:-venv_name: Name of the virtual environment-python_func: A callable function to execute-\*args: Positional arguments to pass to the function-\*\*kwargs: Keyword arguments to pass to the function--Returns:-The result of the function execution, or raises any exception that occurred--#### Computer.venv_exec_background--```python-async def venv_exec_background(self, venv_name: str, python_func, args = (), requirements: Optional[List[str]] = None, kwargs = {}) -> int-```--Run the Python function in the venv in the background and return the PID.--Uses a short launcher Python that spawns a detached child and exits immediately.--#### Computer.python_exec--```python-async def python_exec(self, python_func, args = (), kwargs = {})-```--Execute a Python function using the system Python (no venv).--Uses source extraction and base64 transport, mirroring venv_exec but-without virtual environment activation.--Returns the function result or raises a reconstructed exception with-remote traceback context appended.--#### Computer.python_exec_background--```python-async def python_exec_background(self, python_func, args = (), requirements: Optional[List[str]] = None, kwargs = {}) -> int-```--Run a Python function with the system interpreter in the background and return PID.--Uses a short launcher Python that spawns a detached child and exits immediately.--#### Computer.python_command--```python-def python_command(self, requirements: Optional[List[str]] = None, venv_name: str = 'default', use_system_python: bool = False, background: bool = False) -> Callable[[Callable[P, R]], Callable[P, Awaitable[R]]]-```--Decorator to execute a Python function remotely in this Computer's venv.--This mirrors `computer.helpers.sandboxed()` but binds to this instance and-optionally ensures required packages are installed before execution.--Args:-requirements: Packages to install in the virtual environment.-venv_name: Name of the virtual environment to use.-use_system_python: If True, use the system Python/pip instead of a venv.-background: If True, run the function detached and return the child PID immediately.--Returns:-A decorator that turns a local function into an async callable which-runs remotely and returns the function's result.--## VMProviderType--Enum of supported VM provider types.diff --git a/docs/content/docs/cua/reference/computer-sdk/v0.5/meta.json b/docs/content/docs/cua/reference/computer-sdk/v0.5/meta.jsondeleted file mode 100644index 3dedd0e8d4..0000000000--- a/docs/content/docs/cua/reference/computer-sdk/v0.5/meta.json+++ /dev/null@@ -1,5 +0,0 @@-{- "title": "v0.5",- "description": "Computer SDK v0.5 API Reference",- "pages": ["api"]-}diff --git a/docs/content/docs/cua/reference/desktop-sandbox/changelog.mdx b/docs/content/docs/cua/reference/desktop-sandbox/changelog.mdxdeleted file mode 100644index f0defa0579..0000000000--- a/docs/content/docs/cua/reference/desktop-sandbox/changelog.mdx+++ /dev/null@@ -1,341 +0,0 @@-----title: Changelog-description: Release history for Desktop Sandbox------{/*-AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY-Generated by: npx tsx scripts/docs-generators/generate-changelog.ts-Last updated: 2026-02-04-*/}--# Desktop Sandbox Changelog--All notable changes to the Desktop Sandbox are documented here.--## 0.3.x--### v0.3.16 (2026-01-30)--- Initial release or no path-specific changes found--Run the server-cua-computer-server--```--### v0.3.15 (2026-01-29)--* Bump cua-computer-server to v0.3.15 by @github-actions[bot]-* fix(computer-server): initialize coordinate scaling after auto-resize in ([#936](https://github.com/trycua/cua/pull/936)) by @synacktraa-* fix(mcp-server): disable redirect for /mcp endpoint ([#935](https://github.com/trycua/cua/pull/935)) by @Fizza-Mukhtar-* feat(agent): Centralized tool resolution in ComputerAgent ([#920](https://github.com/trycua/cua/pull/920)) by @sarinali--Run the server-cua-computer-server-```--### v0.3.14 (2026-01-24)--Maintenance release.--### v0.3.13 (2026-01-23)--- Bump cua-computer-server to v0.3.13 by @github-actions[bot]-- fix(computer-server): standardize Android handler response format ([#893](https://github.com/trycua/cua/pull/893)) by @synacktraa--Run the server-cua-computer-server--```--### v0.3.12 (2026-01-21)--Maintenance release.--### v0.3.11 (2026-01-21)--* Initial release or no path-specific changes found--Run the server-cua-computer-server-```--### v0.3.9 (2026-01-21)--- Bump cua-computer-server to v0.3.9 by @github-actions[bot]-- fix(computer-server): make fastmcp a required dependency ([#875](https://github.com/trycua/cua/pull/875)) by @Francesco Bonacci--Run the server-cua-computer-server--```--### v0.3.8 (2026-01-20)--* Bump cua-computer-server to v0.3.8 by @github-actions[bot]-* fix(computer-server): correct MCP endpoint path and lifespan ([#869](https://github.com/trycua/cua/pull/869)) by @Francesco Bonacci--Run the server-cua-computer-server-```--### v0.3.7 (2026-01-20)--- Bump cua-computer-server to v0.3.7 by @github-actions[bot]-- feat(computer-server): expose HTTP and MCP interfaces simultaneously ([#861](https://github.com/trycua/cua/pull/861)) by @Francesco Bonacci--Run the server-cua-computer-server--```--### v0.3.6 (2026-01-20)--* Bump cua-computer-server to v0.3.6 by @github-actions[bot]-* feat(computer-server): auto-detect MCP mode and expose HTTP+MCP simultaneously ([#860](https://github.com/trycua/cua/pull/860)) by @Francesco Bonacci--Run the server-cua-computer-server-```--### v0.3.5 (2026-01-20)--- Bump cua-computer-server to v0.3.5 by @github-actions[bot]-- feat(computer-server): add MCP interface for Claude Code integration ([#859](https://github.com/trycua/cua/pull/859)) by @Francesco Bonacci--Run the server-cua-computer-server--```--### v0.3.2 (2026-01-11)--* Initial release or no path-specific changes found--### v0.3.1 (2026-01-11)--Maintenance release.--## 0.1.x--### v0.1.28 (2025-10-29)--**Dependencies:** cua-computer: latest--Run the server-cua-computer-server-```--- Add test infrastructure with CI/CD #478 by @AceAtDev in https://github.com/trycua/cua/pull/491-- Add desktop and window management commands to computer interface by @ddupont808 in https://github.com/trycua/cua/pull/516--New Contributors--- @AceAtDev made their first contribution in https://github.com/trycua/cua/pull/491--### v0.1.27 (2025-10-24)--**Dependencies:** cua-computer: latest--- Patch Python SSL on Windows VMs to use system cert store (Python bug: https://bugs.python.org/issue36011) ( by @ddupont808 in https://github.com/trycua/cua/pull/510--### v0.1.26 (2025-10-24)--**Dependencies:** cua-computer: latest--Run the server-cua-computer-server--```---* Moved platform-specific deps from computer-server out of optional dependencies (fixes accessibility API not working on macOS, Windows) by @ddupont808 in https://github.com/trycua/cua/pull/508--### v0.1.25 (2025-10-22)--**Dependencies:** cua-computer: latest--Run the server-cua-computer-server-```--- Fix authentication / connection errors caused by computer-server using old API url by @ddupont808 in https://github.com/trycua/cua/pull/499--New Contributors--- @masterbatcoderman10 made their first contribution in https://github.com/trycua/cua/pull/480--### v0.1.23 (2025-10-03)--**Dependencies:** cua-computer: latest--Run the server-cua-computer-server--```--### v0.1.22 (2025-09-03)--**Dependencies:** cua-computer: latest--Run the server-cua-computer-server-```--### v0.1.21 (2025-08-06)--**Dependencies:** cua-computer: latest--Run the server-cua-computer-server--```--### v0.1.20 (2025-07-29)--Replaced `pyautogui.type_text` with `pynput.KeyboardController`, fixing the bug where multi-line text would just type the letter `"v"` on Linux--### v0.1.19 (2025-07-11)--**Dependencies:** cua-computer: latest--Run the server-cua-computer-server-```--### v0.1.18 (2025-07-10)--**Dependencies:** cua-computer: latest--Run the server-cua-computer-server--```--### v0.1.17 (2025-07-09)--**Dependencies:** cua-computer: latest--Run the server-cua-computer-server-```--### v0.1.16 (2025-07-09)--**Dependencies:** cua-computer: latest--Run the server-cua-computer-server--```--### v0.1.15 (2025-07-01)--**Dependencies:** cua-computer: latest--Run the server-cua-computer-server-```--### v0.1.14 (2025-07-01)--**Dependencies:** cua-computer: latest--Run the server-cua-computer-server--```--### v0.1.13 (2025-06-18)--**Dependencies:** cua-computer: latest--Run the server-cua-computer-server-```--### v0.1.12 (2025-06-10)--**Dependencies:** cua-computer: latest--Run the server-cua-computer-server--```--### v0.1.11 (2025-06-05)--**Dependencies:** cua-computer: latest--Run the server-cua-computer-server-```--### v0.1.10 (2025-05-31)--**Dependencies:** cua-computer: latest--Run the server-cua-computer-server--```--### v0.1.9 (2025-05-28)--**Dependencies:** cua-computer: latest--Run the server-cua-computer-server-```--### v0.1.8 (2025-05-28)--**Dependencies:** cua-computer: latest--Run the server-cua-computer-server--```--### v0.1.7 (2025-05-28)--**Dependencies:** cua-computer: latest--Run the server-cua-computer-server-```--### v0.1.6 (2025-05-25)--**Dependencies:** cua-computer: latest--Run the server-cua-computer-server--```--### v0.1.5 (2025-05-16)--**Dependencies:** cua-computer: latest--Run the server-cua-computer-server-```--### v0.1.4 (2025-04-25)--**Dependencies:** cua-computer: latest--Run the server-cua-computer-server--```--### v0.1.3 (2025-04-15)--**Dependencies:** cua-computer: latest--Run the server-cua-computer-server-```diff --git a/docs/content/docs/cua/reference/desktop-sandbox/index.mdx b/docs/content/docs/cua/reference/desktop-sandbox/index.mdxdeleted file mode 100644index dbfb13002e..0000000000--- a/docs/content/docs/cua/reference/desktop-sandbox/index.mdx+++ /dev/null@@ -1,59 +0,0 @@-----title: Desktop Sandbox-description: Isolated desktop environments for Computer-Using Agents------Desktop sandbox environments provide isolated desktops for running Computer-Using Agents. Each environment includes a pre-installed computer-server for remote control via HTTP API.--## Sandbox Options--### macOS Sandbox--Native macOS virtual machines on Apple Silicon using Apple's Virtualization Framework. Near-native performance for macOS automation. [Learn more →](/cua/reference/desktop-sandbox/macos)--- **Lume** - Native CLI for VM management-- **Lumier** - Docker wrapper for containerized deployments--### Linux Container--Docker containers running Linux desktops. Fast startup, low resource usage. [Learn more →](/cua/reference/desktop-sandbox/linux-container)--| Container | Description |-| ----------------------------------------------------------- | ---------------------------------- |-| [Kasm](/cua/reference/desktop-sandbox/linux-container/kasm) | KasmWeb-based Ubuntu with XFCE |-| [XFCE](/cua/reference/desktop-sandbox/linux-container/xfce) | Vanilla XFCE, minimal dependencies |--### QEMU Container--Full virtual machines running in Docker via QEMU/KVM. Complete OS isolation, supports Windows. [Learn more →](/cua/reference/desktop-sandbox/qemu-container)--| Container | OS | Description |-| ---------------------------------------------------------------- | ------------ | ------------------------ |-| [Windows](/cua/reference/desktop-sandbox/qemu-container/windows) | Windows 11 | Windows desktop with KVM |-| [Linux](/cua/reference/desktop-sandbox/qemu-container/linux) | Ubuntu 22.04 | Full Ubuntu VM |-| [Android](/cua/reference/desktop-sandbox/qemu-container/android) | Android 11 | Android emulator |--## Quick Comparison--| Feature | macOS (Lume) | Linux Container | QEMU VM |-| -------------------- | ------------- | --------------- | ----------- |-| Performance | Near-native | Native | Virtualized |-| Startup Time | Minutes | Seconds | 30s-2min |-| Host Requirements | Apple Silicon | Docker | KVM support |-| Windows Support | No | No | Yes |-| Resource Usage | Medium | Low | High |-| Memory Snapshots | Yes | No | Yes |-| Filesystem Snapshots | Yes | Yes | Yes |--### Snapshot Capabilities--**macOS (Lume)** and **QEMU containers** support full memory snapshots, allowing you to save and restore the complete VM state including running processes and memory contents. This is useful for creating checkpoints during long-running tasks or reverting to a known state.--**Linux containers** (Kasm, XFCE) only support filesystem snapshots via `docker commit`, which saves the disk state but not running processes or memory. The container must be restarted after restoring.--## Common Access Points--All environments expose:--- **Computer Server API** - HTTP API for automation (port 5000 or 8000)-- **VNC/noVNC** - Visual desktop access via browser (port 6901 or 8006)diff --git a/docs/content/docs/cua/reference/desktop-sandbox/linux-container/index.mdx b/docs/content/docs/cua/reference/desktop-sandbox/linux-container/index.mdxdeleted file mode 100644index e385486f52..0000000000--- a/docs/content/docs/cua/reference/desktop-sandbox/linux-container/index.mdx+++ /dev/null@@ -1,37 +0,0 @@-----title: Linux Container-description: Docker containers running Linux desktops for Computer-Using Agents------Docker containers running Linux desktops with fast startup and low resource usage. Each container includes a pre-installed computer-server for remote control via HTTP API.--## Available Containers--| Container | Description |-| --- | --- |-| [Kasm](/cua/reference/desktop-sandbox/linux-container/kasm) | KasmWeb-based Ubuntu with XFCE desktop |-| [XFCE](/cua/reference/desktop-sandbox/linux-container/xfce) | Vanilla XFCE, minimal dependencies |--## Key Features--- **Fast startup** — Containers start in seconds-- **Low resource usage** — Minimal overhead compared to full VMs-- **Docker-based** — Works anywhere Docker runs-- **Filesystem snapshots** — Save state via `docker commit`--## Quick Start--```python-from computer import Computer--async with Computer(- os_type="linux",- provider_type="docker",- image="trycua/cua-xfce:latest",-) as computer:- screenshot = await computer.interface.screenshot()-```--## Limitations--Linux containers only support filesystem snapshots via `docker commit`, which saves the disk state but not running processes or memory. The container must be restarted after restoring.diff --git a/docs/content/docs/cua/reference/desktop-sandbox/linux-container/kasm.mdx b/docs/content/docs/cua/reference/desktop-sandbox/linux-container/kasm.mdxdeleted file mode 100644index b4c07bf912..0000000000--- a/docs/content/docs/cua/reference/desktop-sandbox/linux-container/kasm.mdx+++ /dev/null@@ -1,90 +0,0 @@-----title: Kasm Container-description: Kasm-based Ubuntu desktop container for Computer-Using Agents------Containerized virtual desktop for Computer-Using Agents (Cua). Utilizes Kasm's MIT-licensed Ubuntu XFCE container as a base with computer-server pre-installed.--## Features--- Ubuntu 22.04 (Jammy) with XFCE desktop environment-- Pre-installed computer-server for remote computer control-- VNC access for visual desktop interaction-- Python 3.11 with necessary libraries-- Screen capture tools (gnome-screenshot, wmctrl, ffmpeg)-- Clipboard utilities (xclip, socat)--## Quick Start--```bash-docker pull trycua/cua-ubuntu:latest--docker run --rm -it \- --shm-size=512m \- -p 6901:6901 \- -p 8000:8000 \- -e VNCOPTIONS=-disableBasicAuth \- trycua/cua-ubuntu:latest-```--**Access Points:**--- **VNC Web Interface**: `http://localhost:6901`-- **Computer Server API**: `http://localhost:8000`--## Using with Cua--```python-from computer import Computer--computer = Computer(- os_type="linux",- provider_type="docker",- image="trycua/cua-ubuntu:latest",- display="1024x768",- memory="4GB",- cpu="2"-)--async with computer:- screenshot = await computer.interface.screenshot()- await computer.interface.left_click(100, 100)- await computer.interface.type_text("Hello from Cua!")-```--## Configuration--### Ports--- **6901**: VNC web interface (noVNC)-- **8000**: Computer-server API endpoint--### Environment Variables--| Variable | Default | Description |-| --------- | ---------- | ------------ |-| `VNC_PW` | `password` | VNC password |-| `DISPLAY` | `:0` | X11 display |--### Volumes--- `/home/kasm-user/storage`: Persistent storage mount point-- `/home/kasm-user/shared`: Shared folder mount point--## Creating Snapshots--Create a filesystem snapshot of the container:--```bash-docker commit <container_id> cua-ubuntu-snapshot:latest-```--Then run the snapshot:--```bash-docker run --rm -it --shm-size=512m -p 6901:6901 -p 8000:8000 -e VNCOPTIONS=-disableBasicAuth cua-ubuntu-snapshot:latest-```--## Building from Source--To build the image manually, see the [kasm source code](https://github.com/trycua/cua/tree/main/libs/kasm).diff --git a/docs/content/docs/cua/reference/desktop-sandbox/linux-container/meta.json b/docs/content/docs/cua/reference/desktop-sandbox/linux-container/meta.jsondeleted file mode 100644index 37c26d401e..0000000000--- a/docs/content/docs/cua/reference/desktop-sandbox/linux-container/meta.json+++ /dev/null@@ -1,4 +0,0 @@-{- "title": "Linux Container",- "pages": ["---", "kasm", "xfce"]-}diff --git a/docs/content/docs/cua/reference/desktop-sandbox/linux-container/xfce.mdx b/docs/content/docs/cua/reference/desktop-sandbox/linux-container/xfce.mdxdeleted file mode 100644index 189ca99b57..0000000000--- a/docs/content/docs/cua/reference/desktop-sandbox/linux-container/xfce.mdx+++ /dev/null@@ -1,124 +0,0 @@-----title: XFCE Container-description: Vanilla XFCE desktop container for Computer-Using Agents------Vanilla XFCE desktop container for Computer-Using Agents (Cua) with noVNC and computer-server. This is a lightweight alternative to the Kasm-based container with minimal dependencies.--## Features--- Ubuntu 22.04 (Jammy) with vanilla XFCE desktop environment-- TigerVNC server for remote desktop access-- noVNC for web-based VNC access (no client required)-- Pre-installed computer-server for remote computer control-- Python 3.11 with necessary libraries-- Firefox browser with telemetry disabled--## Quick Start--```bash-docker pull trycua/cua-xfce:latest--docker run --rm -it \- --shm-size=512m \- -p 5901:5901 \- -p 6901:6901 \- -p 8000:8000 \- trycua/cua-xfce:latest-```--**Access Points:**--- **noVNC Web Interface**: `http://localhost:6901` (no password required)-- **VNC Client**: `localhost:5901` (no password required)-- **Computer Server API**: `http://localhost:8000`--### With Custom Resolution--```bash-docker run --rm -it \- --shm-size=512m \- -p 6901:6901 \- -p 8000:8000 \- -e VNC_RESOLUTION=1280x720 \- trycua/cua-xfce:latest-```--### With Persistent Storage--```bash-docker run --rm -it \- --shm-size=512m \- -p 6901:6901 \- -p 8000:8000 \- -v $(pwd)/storage:/home/cua/storage \- trycua/cua-xfce:latest-```--## Using with Cua--```python-from computer import Computer--computer = Computer(- os_type="linux",- provider_type="docker",- image="trycua/cua-xfce:latest",- display="1024x768",- memory="4GB",- cpu="2"-)--async with computer:- screenshot = await computer.interface.screenshot()- await computer.interface.left_click(100, 100)- await computer.interface.type_text("Hello from Cua!")-```--## Configuration--### Environment Variables--| Variable | Default | Description |-| ---------------- | ---------- | ------------------------ |-| `VNC_RESOLUTION` | `1024x768` | Screen resolution |-| `VNC_COL_DEPTH` | `24` | Color depth |-| `VNC_PORT` | `5901` | VNC server port |-| `NOVNC_PORT` | `6901` | noVNC web interface port |-| `API_PORT` | `8000` | Computer-server API port |--### Ports--- **5901**: TigerVNC server-- **6901**: noVNC web interface-- **8000**: Computer-server WebSocket API--### Volumes--- `/home/cua/storage`: Persistent storage mount point-- `/home/cua/shared`: Shared folder mount point--### User Credentials--- **Username**: `cua`-- **Password**: `password` (for shell login only)-- **Sudo access**: Enabled without password--## Creating Snapshots--```bash-docker commit <container_id> cua-xfce-snapshot:latest-```--## Comparison with Kasm Container--| Feature | Kasm Container | XFCE Container |-| ------------ | -------------- | -------------- |-| Base Image | KasmWeb Ubuntu | Vanilla Ubuntu |-| VNC Server | KasmVNC | TigerVNC |-| Dependencies | Higher | Lower |-| Size | Larger | Smaller |--## Building from Source--To build the image manually, see the [xfce source code](https://github.com/trycua/cua/tree/main/libs/xfce).diff --git a/docs/content/docs/cua/reference/desktop-sandbox/macos.mdx b/docs/content/docs/cua/reference/desktop-sandbox/macos.mdxdeleted file mode 100644index cb7d3f0caf..0000000000--- a/docs/content/docs/cua/reference/desktop-sandbox/macos.mdx+++ /dev/null@@ -1,81 +0,0 @@-----title: macOS Sandbox-description: macOS virtual machines for Computer-Using Agents on Apple Silicon------macOS sandbox environments run native macOS virtual machines on Apple Silicon using Apple's Virtualization Framework. These provide near-native performance for running Computer-Using Agents on macOS.--## Options--| Option | Description | Best For |-| ------------------------------------------------------------- | ---------------------------------------------- | --------------------------------- |-| [Lume](/lume/guide/getting-started/introduction) | Native CLI for creating and managing macOS VMs | Direct VM management, development |-| [Lumier](/lume/guide/advanced/lumier) | Docker wrapper around Lume | Containerized deployments |--## Requirements--- **Apple Silicon Mac** (M1, M2, M3, M4, etc.)-- **macOS 13.0 or later**-- At least 8GB RAM (16GB recommended)-- At least 30GB free disk space--## Quick Start with Lume--```bash-# Install Lume-/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/lume/scripts/install.sh)"--# Create and run a macOS VM-lume create my-vm --os macos --ipsw latest-lume run my-vm-```--## Quick Start with Lumier (Docker)--```bash-# Run macOS VM in Docker container-docker run -it --rm \- --name macos-vm \- -p 8006:8006 \- -e VM_NAME=macos-vm \- -e CPU_CORES=4 \- -e RAM_SIZE=8192 \- trycua/lumier:latest-```--Access your VM at `http://localhost:8006`.--## Using with Cua--```python-from computer import Computer--# Create computer with macOS VM-computer = Computer(- os_type="macos",- provider_type="lume",- display="1920x1080",- memory="8GB",- cpu="4"-)--async with computer:- screenshot = await computer.interface.screenshot()- await computer.interface.left_click(100, 100)- await computer.interface.type_text("Hello from Cua!")-```--## Comparison with Other Platforms--| Feature | macOS (Lume) | Linux Container | QEMU VM |-| --------------- | ------------ | --------------- | --------------------- |-| Performance | Near-native | Native | Virtualized |-| Host OS | macOS only | Any with Docker | Any with KVM |-| Setup Time | Minutes | Seconds | 15-30 min (first run) |-| Windows Support | No | No | Yes |--## Related Documentation--- [Lume Documentation](/lume/guide/getting-started/introduction) - Full Lume CLI reference-- [Lumier Documentation](/lume/guide/advanced/lumier) - Docker-based macOS VMs-- [Unattended Setup](/lume/guide/fundamentals/unattended-setup) - Automate macOS VM configurationdiff --git a/docs/content/docs/cua/reference/desktop-sandbox/meta.json b/docs/content/docs/cua/reference/desktop-sandbox/meta.jsondeleted file mode 100644index 12cf0816e1..0000000000--- a/docs/content/docs/cua/reference/desktop-sandbox/meta.json+++ /dev/null@@ -1,5 +0,0 @@-{- "title": "Desktop Sandbox",- "icon": "Container",- "pages": ["macos", "linux-container", "qemu-container", "changelog"]-}diff --git a/docs/content/docs/cua/reference/desktop-sandbox/qemu-container/android.mdx b/docs/content/docs/cua/reference/desktop-sandbox/qemu-container/android.mdxdeleted file mode 100644index 882240c09a..0000000000--- a/docs/content/docs/cua/reference/desktop-sandbox/qemu-container/android.mdx+++ /dev/null@@ -1,76 +0,0 @@-----title: QEMU Android-description: Android emulator container with QEMU/KVM for Computer-Using Agents------Docker image that runs an Android emulator using QEMU/KVM with Cua Computer Server integration, enabling programmatic control of Android devices via HTTP API.--## Features--- Android 11 Emulator (based on budtmo/docker-android)-- Cua Computer Server for automation via HTTP API-- VNC access for visual interaction via web browser-- Memory snapshots support--## Requirements--- Linux host with KVM support-- Docker with `--device=/dev/kvm` access--## Quick Start--```bash-docker pull trycua/cua-qemu-android:latest--docker run -d \- -p 6080:6080 \- -p 8000:8000 \- -e EMULATOR_DEVICE="Samsung Galaxy S10" \- -e WEB_VNC=true \- --device /dev/kvm \- --name android-container \- trycua/cua-qemu-android:latest-```--**Access Points:**--- **VNC Web UI**: `http://localhost:6080`-- **Computer Server API**: `http://localhost:8000`-- **API Documentation**: `http://localhost:8000/docs`--## API Examples--```bash-# Get screen size-curl -X POST http://localhost:8000/cmd \- -H "Content-Type: application/json" \- -d '{"command": "get_screen_size", "params": {}}'--# Launch app-curl -X POST http://localhost:8000/cmd \- -H "Content-Type: application/json" \- -d '{"command": "launch", "params": {"app": "com.android.settings"}}'--# Set wallpaper-curl -X POST http://localhost:8000/cmd \- -H "Content-Type: application/json" \- -d '{"command": "set_wallpaper", "params": {"path": "/sdcard/image.jpg", "target": "home"}}'-```--## Configuration--### Environment Variables--| Variable | Default | Description |-| ----------------- | ------- | ------------------------------------------- |-| `EMULATOR_DEVICE` | - | Device profile (e.g., "Samsung Galaxy S10") |-| `WEB_VNC` | `true` | Enable VNC web interface |--### Ports--- **6080**: VNC web interface-- **8000**: Computer Server API--## Building from Source--To build the image manually, see the [qemu-docker/android source code](https://github.com/trycua/cua/tree/main/libs/qemu-docker/android).diff --git a/docs/content/docs/cua/reference/desktop-sandbox/qemu-container/index.mdx b/docs/content/docs/cua/reference/desktop-sandbox/qemu-container/index.mdxdeleted file mode 100644index 45bd7b05d9..0000000000--- a/docs/content/docs/cua/reference/desktop-sandbox/qemu-container/index.mdx+++ /dev/null@@ -1,39 +0,0 @@-----title: QEMU Container-description: Full virtual machines running in Docker via QEMU/KVM for Computer-Using Agents------Full virtual machines running in Docker via QEMU/KVM. Complete OS isolation with support for Windows, Linux, and Android.--## Available Containers--| Container | OS | Description |-| --- | --- | --- |-| [Windows](/cua/reference/desktop-sandbox/qemu-container/windows) | Windows 11 | Windows desktop with KVM |-| [Linux](/cua/reference/desktop-sandbox/qemu-container/linux) | Ubuntu 22.04 | Full Ubuntu VM |-| [Android](/cua/reference/desktop-sandbox/qemu-container/android) | Android 11 | Android emulator |--## Key Features--- **Complete OS isolation** — Full virtual machines with dedicated kernels-- **Windows support** — Run Windows 11 in Docker-- **Memory snapshots** — Save and restore full VM state including running processes-- **KVM acceleration** — Near-native performance on Linux hosts with KVM support--## Requirements--- Linux host with KVM support-- Docker with `--device=/dev/kvm` access--## Quick Start--```python-from computer import Computer--async with Computer(- os_type="windows",- provider_type="docker",- image="trycua/cua-qemu-windows:latest",-) as computer:- screenshot = await computer.interface.screenshot()-```diff --git a/docs/content/docs/cua/reference/desktop-sandbox/qemu-container/linux.mdx b/docs/content/docs/cua/reference/desktop-sandbox/qemu-container/linux.mdxdeleted file mode 100644index 066c00e12d..0000000000--- a/docs/content/docs/cua/reference/desktop-sandbox/qemu-container/linux.mdx+++ /dev/null@@ -1,81 +0,0 @@-----title: QEMU Linux-description: Ubuntu 22.04 container with QEMU/KVM for Computer-Using Agents------Containerized Ubuntu 22.04 LTS virtual desktop for Computer-Using Agents (Cua). Utilizes QEMU/KVM with Ubuntu Desktop and computer-server pre-installed for remote computer control.--## Features--- Ubuntu 22.04 LTS Desktop running in QEMU/KVM-- Pre-installed Cua computer-server for remote computer control-- noVNC access for visual desktop interaction-- Memory snapshots support-- Full VM isolation--## Requirements--- Linux host with KVM support-- Docker with `--device=/dev/kvm` access--## Quick Start--```bash-docker pull trycua/cua-qemu-linux:latest--mkdir -p ./storage--docker run -it --rm \- --device=/dev/kvm \- --cap-add NET_ADMIN \- -v $(pwd)/storage:/storage \- -p 8006:8006 \- -p 5000:5000 \- -e RAM_SIZE=8G \- -e CPU_CORES=4 \- trycua/cua-qemu-linux:latest-```--**Access Points:**--- **Computer Server API**: `http://localhost:5000`-- **noVNC Browser**: `http://localhost:8006`--## Configuration--### Environment Variables--| Variable | Default | Description |-| ----------- | ------- | --------------------------- |-| `RAM_SIZE` | `8G` | RAM allocated to Ubuntu VM |-| `CPU_CORES` | `8` | CPU cores allocated to VM |-| `DISK_SIZE` | `64G` | VM disk size (minimum: 32G) |--### Ports--- **5000**: Cua computer-server API endpoint-- **8006**: noVNC web interface for visual desktop access--### Volumes--- `/storage`: Persistent VM storage (golden image, disk)--## Architecture--```-┌─────────────────────────────────────────────────────────┐-│ Docker Container (Linux host) │-│ │-│ • Port forwarding: localhost:5000 → VM:5000 │-│ • Exposes: 5000 (API), 8006 (noVNC) │-│ │-│ ┌────────────────────────────────────────────────────┐ │-│ │ QEMU VM (Ubuntu 22.04) │ │-│ │ • Cua computer-server listens on 5000 │ │-│ └────────────────────────────────────────────────────┘ │-└─────────────────────────────────────────────────────────┘-```--## Building from Source--To build the image manually (including creating a golden image from an Ubuntu ISO), see the [qemu-docker/linux source code](https://github.com/trycua/cua/tree/main/libs/qemu-docker/linux).diff --git a/docs/content/docs/cua/reference/desktop-sandbox/qemu-container/meta.json b/docs/content/docs/cua/reference/desktop-sandbox/qemu-container/meta.jsondeleted file mode 100644index 4d2d31eb39..0000000000--- a/docs/content/docs/cua/reference/desktop-sandbox/qemu-container/meta.json+++ /dev/null@@ -1,4 +0,0 @@-{- "title": "QEMU Container",- "pages": ["---", "windows", "linux", "android"]-}diff --git a/docs/content/docs/cua/reference/desktop-sandbox/qemu-container/windows.mdx b/docs/content/docs/cua/reference/desktop-sandbox/qemu-container/windows.mdxdeleted file mode 100644index 74962d1350..0000000000--- a/docs/content/docs/cua/reference/desktop-sandbox/qemu-container/windows.mdx+++ /dev/null@@ -1,81 +0,0 @@-----title: QEMU Windows-description: Windows 11 container with QEMU/KVM for Computer-Using Agents------Containerized Windows 11 virtual desktop for Computer-Using Agents (Cua). Utilizes QEMU/KVM with Windows 11 and computer-server pre-installed for remote computer control.--## Features--- Windows 11 Enterprise running in QEMU/KVM-- Pre-installed Cua computer-server for remote computer control-- noVNC access for visual desktop interaction-- Memory snapshots support-- Essential tools pre-installed (Chrome, LibreOffice, VLC, GIMP, VSCode, Thunderbird)--## Requirements--- Linux host with KVM support-- Docker with `--device=/dev/kvm` access--## Quick Start--```bash-docker pull trycua/cua-qemu-windows:latest--mkdir -p ./storage--docker run -it --rm \- --device=/dev/kvm \- --cap-add NET_ADMIN \- -v $(pwd)/storage:/storage \- -p 8006:8006 \- -p 5000:5000 \- -e RAM_SIZE=8G \- -e CPU_CORES=4 \- trycua/cua-qemu-windows:latest-```--**Access Points:**--- **Computer Server API**: `http://localhost:5000`-- **noVNC Browser**: `http://localhost:8006`--## Configuration--### Environment Variables--| Variable | Default | Description |-| ----------- | ------- | --------------------------- |-| `RAM_SIZE` | `8G` | RAM allocated to Windows VM |-| `CPU_CORES` | `8` | CPU cores allocated to VM |-| `DISK_SIZE` | `30G` | VM disk size (minimum: 20G) |--### Ports--- **5000**: Cua computer-server API endpoint-- **8006**: noVNC web interface for visual desktop access--### Volumes--- `/storage`: Persistent VM storage (golden image, disk, firmware)--## Architecture--```-┌─────────────────────────────────────────────────────────┐-│ Docker Container (Linux host) │-│ │-│ • Port forwarding: localhost:5000 → VM:5000 │-│ • Exposes: 5000 (API), 8006 (noVNC) │-│ │-│ ┌────────────────────────────────────────────────────┐ │-│ │ QEMU VM (Windows 11) │ │-│ │ • Cua computer-server listens on 5000 │ │-│ └────────────────────────────────────────────────────┘ │-└─────────────────────────────────────────────────────────┘-```--## Building from Source--To build the image manually (including creating a golden image from a Windows ISO), see the [qemu-docker/windows source code](https://github.com/trycua/cua/tree/main/libs/qemu-docker/windows).diff --git a/docs/content/docs/cua/reference/mcp-server/client-integrations.mdx b/docs/content/docs/cua/reference/mcp-server/client-integrations.mdxdeleted file mode 100644index e39287951e..0000000000--- a/docs/content/docs/cua/reference/mcp-server/client-integrations.mdx+++ /dev/null@@ -1,123 +0,0 @@-----title: Client Integrations------## Claude Desktop Integration--To use with Claude Desktop, add an entry to your Claude Desktop configuration (`claude_desktop_config.json`, typically found in `~/.config/claude-desktop/`):--### Package Installation Method--```json-{- "mcpServers": {- "cua-agent": {- "command": "/bin/bash",- "args": ["~/.cua/start_mcp_server.sh"],- "env": {- "CUA_MODEL_NAME": "anthropic/claude-sonnet-4-20250514",- "ANTHROPIC_API_KEY": "your-anthropic-api-key-here",- "CUA_MAX_IMAGES": "3",- "CUA_USE_HOST_COMPUTER_SERVER": "false"- }- }- }-}-```--### Development Method--If you're working with the Cua source code:--**Standard VM Mode:**--```json-{- "mcpServers": {- "cua-agent": {- "command": "/usr/bin/env",- "args": [- "bash",- "-lc",- "export CUA_MODEL_NAME='anthropic/claude-sonnet-4-20250514'; export ANTHROPIC_API_KEY='your-anthropic-api-key-here'; /path/to/cua/libs/python/mcp-server/scripts/start_mcp_server.sh"- ]- }- }-}-```--**Host Computer Control Mode:**--```json-{- "mcpServers": {- "cua-agent": {- "command": "/usr/bin/env",- "args": [- "bash",- "-lc",- "export CUA_MODEL_NAME='anthropic/claude-sonnet-4-20250514'; export ANTHROPIC_API_KEY='your-anthropic-api-key-here'; export CUA_USE_HOST_COMPUTER_SERVER='true'; export CUA_MAX_IMAGES='1'; /path/to/cua/libs/python/mcp-server/scripts/start_mcp_server.sh"- ]- }- }-}-```--**Note**: Replace `/path/to/cua` with the absolute path to your Cua repository directory.--**⚠️ Host Computer Control Setup**: When using `CUA_USE_HOST_COMPUTER_SERVER='true'`, you must also:--1. Install computer server dependencies: `python3 -m pip install uvicorn fastapi`-2. Install the computer server: `python3 -m pip install -e libs/python/computer-server --break-system-packages`-3. Start the computer server: `python -m computer_server --log-level debug`-4. The AI will have direct access to your desktop - use with caution!--For more information on MCP with Claude Desktop, see the [official MCP User Guide](https://modelcontextprotocol.io/quickstart/user).--## Cursor Integration--To use with Cursor, add an MCP configuration file in one of these locations:--- **Project-specific**: Create `.cursor/mcp.json` in your project directory-- **Global**: Create `~/.cursor/mcp.json` in your home directory--Example configuration for Cursor:--```json-{- "mcpServers": {- "cua-agent": {- "command": "/bin/bash",- "args": ["~/.cua/start_mcp_server.sh"],- "env": {- "CUA_MODEL_NAME": "anthropic/claude-sonnet-4-20250514",- "ANTHROPIC_API_KEY": "your-anthropic-api-key-here"- }- }- }-}-```--After configuration, you can simply tell Cursor's Agent to perform computer tasks by explicitly mentioning the Cua agent, such as "Use the computer control tools to open Safari."--For more information on MCP with Cursor, see the [official Cursor MCP documentation](https://docs.cursor.com/context/model-context-protocol).--## Other MCP Clients--The MCP server is compatible with any MCP-compliant client. The server exposes the following tools:--- `run_cua_task` - Execute single computer tasks-- `run_multi_cua_tasks` - Execute multiple tasks (sequential or concurrent)-- `screenshot_cua` - Capture screenshots-- `get_session_stats` - Monitor session statistics-- `cleanup_session` - Manage session lifecycle--### Configuration Options--All MCP clients can configure the server using environment variables:--- `CUA_MODEL_NAME` - Model to use for task execution-- `CUA_MAX_IMAGES` - Maximum images to keep in context-- `CUA_USE_HOST_COMPUTER_SERVER` - Use host system instead of VM--See the [Configuration](/cua/reference/mcp-server/configuration) page for detailed configuration options.diff --git a/docs/content/docs/cua/reference/mcp-server/configuration.mdx b/docs/content/docs/cua/reference/mcp-server/configuration.mdxdeleted file mode 100644index 106ce6d983..0000000000--- a/docs/content/docs/cua/reference/mcp-server/configuration.mdx+++ /dev/null@@ -1,73 +0,0 @@-----title: Configuration------The server is configured using environment variables (can be set in the Claude Desktop config):--| Variable | Description | Default |-| ------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------- |-| `CUA_MODEL_NAME` | Model string (e.g., "anthropic/claude-sonnet-4-20250514", "openai/computer-use-preview", "huggingface-local/ByteDance-Seed/UI-TARS-1.5-7B", "omniparser+litellm/gpt-4o", "omniparser+ollama_chat/gemma3") | anthropic/claude-sonnet-4-20250514 |-| `ANTHROPIC_API_KEY` | Your Anthropic API key (required for Anthropic models) | None |-| `CUA_MAX_IMAGES` | Maximum number of images to keep in context | 3 |-| `CUA_USE_HOST_COMPUTER_SERVER` | Target your local desktop instead of a VM. Set to "true" to use your host system. **Warning:** AI models may perform risky actions. | false |--## Model Configuration--The `CUA_MODEL_NAME` environment variable supports various model providers through LiteLLM integration:--### Supported Providers--- **Anthropic**: `anthropic/claude-sonnet-4-20250514`,-- **OpenAI**: `openai/computer-use-preview`, `openai/gpt-4o`-- **Local Models**: `huggingface-local/ByteDance-Seed/UI-TARS-1.5-7B`-- **Omni + LiteLLM**: `omniparser+litellm/gpt-4o`, `omniparser+litellm/claude-3-haiku`-- **Ollama**: `omniparser+ollama_chat/gemma3`--### Example Configurations--**Claude Desktop Configuration:**--```json-{- "mcpServers": {- "cua-agent": {- "command": "/bin/bash",- "args": ["~/.cua/start_mcp_server.sh"],- "env": {- "CUA_MODEL_NAME": "anthropic/claude-sonnet-4-20250514",- "ANTHROPIC_API_KEY": "your-anthropic-api-key-here",- "CUA_MAX_IMAGES": "5",- "CUA_USE_HOST_COMPUTER_SERVER": "false"- }- }- }-}-```--**Local Model Configuration:**--```json-{- "mcpServers": {- "cua-agent": {- "command": "/bin/bash",- "args": ["~/.cua/start_mcp_server.sh"],- "env": {- "CUA_MODEL_NAME": "huggingface-local/ByteDance-Seed/UI-TARS-1.5-7B",- "CUA_MAX_IMAGES": "3"- }- }- }-}-```--## Session Management Configuration--The MCP server automatically manages sessions with the following defaults:--- **Max Concurrent Sessions**: 10-- **Session Timeout**: 10 minutes of inactivity-- **Computer Pool Size**: 5 instances-- **Automatic Cleanup**: Enabled--These settings are optimized for typical usage and don't require configuration for most users.diff --git a/docs/content/docs/cua/reference/mcp-server/index.mdx b/docs/content/docs/cua/reference/mcp-server/index.mdxdeleted file mode 100644index 108437f306..0000000000--- a/docs/content/docs/cua/reference/mcp-server/index.mdx+++ /dev/null@@ -1,17 +0,0 @@-----title: MCP Server-description: Run Cua agents through Claude Desktop and other MCP clients------The MCP Server exposes Cua agents as tools for [Model Context Protocol](https://modelcontextprotocol.io/) clients like Claude Desktop. This lets you ask Claude to perform computer tasks directly from the chat interface.--```bash-pip install cua-mcp-server-```--## Key Features--- **Claude Desktop integration** - Use Cua agents directly in Claude's chat-- **Multi-client support** - Concurrent sessions with automatic resource management-- **Progress reporting** - Real-time updates during task execution-- **VM safety** - Runs in sandboxed VMs by defaultdiff --git a/docs/content/docs/cua/reference/mcp-server/installation.mdx b/docs/content/docs/cua/reference/mcp-server/installation.mdxdeleted file mode 100644index 7b5cf16197..0000000000--- a/docs/content/docs/cua/reference/mcp-server/installation.mdx+++ /dev/null@@ -1,140 +0,0 @@-----title: Installation------Install the package from PyPI:--```bash-pip install cua-mcp-server-```--This will install:--- The MCP server-- Cua agent and computer dependencies-- An executable `cua-mcp-server` script in your PATH--## Easy Setup Script--If you want to simplify installation, you can use this one-liner to download and run the installation script:--```bash-curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/python/mcp-server/scripts/install_mcp_server.sh | bash-```--This script will:--- Create the ~/.cua directory if it doesn't exist-- Generate a startup script at ~/.cua/start_mcp_server.sh-- Make the script executable-- The startup script automatically manages Python virtual environments and installs/updates the cua-mcp-server package--You can then use the script in your MCP configuration like this:--```json-{- "mcpServers": {- "cua-agent": {- "command": "/bin/bash",- "args": ["~/.cua/start_mcp_server.sh"],- "env": {- "CUA_MODEL_NAME": "anthropic/claude-sonnet-4-20250514",- "ANTHROPIC_API_KEY": "your-anthropic-api-key-here"- }- }- }-}-```--**Important**: You must include your Anthropic API key for the MCP server to work properly.--## Development Setup--If you're working with the Cua source code directly (like in the Cua repository), you can use the development script instead:--```json-{- "mcpServers": {- "cua-agent": {- "command": "/usr/bin/env",- "args": [- "bash",- "-lc",- "export CUA_MODEL_NAME='anthropic/claude-sonnet-4-20250514'; export ANTHROPIC_API_KEY='your-anthropic-api-key-here'; /path/to/cua/libs/python/mcp-server/scripts/start_mcp_server.sh"- ]- }- }-}-```--**For host computer control** (development setup):--1. **Install Computer Server Dependencies**:-- ```bash- python3 -m pip install uvicorn fastapi- python3 -m pip install -e libs/python/computer-server --break-system-packages- ```--2. **Start the Computer Server**:-- ```bash- cd /path/to/cua- python -m computer_server --log-level debug- ```-- This will start the computer server on `http://localhost:8000` that controls your actual desktop.--3. **Configure Claude Desktop**:- ```json- {- "mcpServers": {- "cua-agent": {- "command": "/usr/bin/env",- "args": [- "bash",- "-lc",- "export CUA_MODEL_NAME='anthropic/claude-sonnet-4-20250514'; export ANTHROPIC_API_KEY='your-anthropic-api-key-here'; export CUA_USE_HOST_COMPUTER_SERVER='true'; export CUA_MAX_IMAGES='1'; /path/to/cua/libs/python/mcp-server/scripts/start_mcp_server.sh"- ]- }- }- }- ```--**Note**: Replace `/path/to/cua` with the absolute path to your Cua repository directory.--**⚠️ Important**: When using host computer control (`CUA_USE_HOST_COMPUTER_SERVER='true'`), the AI will have direct access to your desktop and can perform actions like opening applications, clicking, typing, and taking screenshots. Make sure you're comfortable with this level of access.--### Troubleshooting--**Common Issues:**--1. **"Claude's response was interrupted"** - This usually means:- - Missing API key: Add `ANTHROPIC_API_KEY` to your environment variables- - Invalid model name: Use a valid model like `anthropic/claude-sonnet-4-20250514`- - Check logs for specific error messages--2. **"Missing Anthropic API Key"** - Add your API key to the configuration:-- ```json- "env": {- "ANTHROPIC_API_KEY": "your-api-key-here"- }- ```--3. **"model not found"** - Use a valid model name:- - ✅ `anthropic/claude-sonnet-4-20250514`--4. **Script not found** - If you get a `/bin/bash: ~/cua/libs/python/mcp-server/scripts/start_mcp_server.sh: No such file or directory` error, try changing the path to the script to be absolute instead of relative.--5. **Host Computer Control Issues** - If using `CUA_USE_HOST_COMPUTER_SERVER='true'`:- - **Computer Server not running**: Make sure you've started the computer server with `python -m computer_server --log-level debug`- - **Port 8000 in use**: Check if another process is using port 8000 with `lsof -i :8000`- - **Missing dependencies**: Install `uvicorn` and `fastapi` with `python3 -m pip install uvicorn fastapi`- - **Image size errors**: Use `CUA_MAX_IMAGES='1'` to reduce image context size--**Viewing Logs:**--```bash-tail -n 20 -f ~/Library/Logs/Claude/mcp*.log-```diff --git a/docs/content/docs/cua/reference/mcp-server/llm-integrations.mdx b/docs/content/docs/cua/reference/mcp-server/llm-integrations.mdxdeleted file mode 100644index 656def704f..0000000000--- a/docs/content/docs/cua/reference/mcp-server/llm-integrations.mdx+++ /dev/null@@ -1,18 +0,0 @@-----title: LLM Integrations------## LiteLLM Integration--This MCP server features comprehensive liteLLM integration, allowing you to use any supported LLM provider with a simple model string configuration.--- **Unified Configuration**: Use a single `CUA_MODEL_NAME` environment variable with a model string-- **Automatic Provider Detection**: The agent automatically detects the provider and capabilities from the model string-- **Extensive Provider Support**: Works with Anthropic, OpenAI, local models, and any liteLLM-compatible provider--### Model String Examples:--- **Anthropic**: `"anthropic/claude-sonnet-4-5-20250929"`-- **OpenAI**: `"openai/computer-use-preview"`-- **UI-TARS**: `"huggingface-local/ByteDance-Seed/UI-TARS-1.5-7B"`-- **Omni + Any LiteLLM**: `"omniparser+litellm/gpt-4o"`, `"omniparser+litellm/claude-3-haiku"`, `"omniparser+ollama_chat/gemma3"`diff --git a/docs/content/docs/cua/reference/mcp-server/meta.json b/docs/content/docs/cua/reference/mcp-server/meta.jsondeleted file mode 100644index 4cbb6aa1b1..0000000000--- a/docs/content/docs/cua/reference/mcp-server/meta.json+++ /dev/null@@ -1,12 +0,0 @@-{- "title": "MCP Server",- "icon": "Server",- "pages": [- "installation",- "configuration",- "usage",- "tools",- "client-integrations",- "llm-integrations"- ]-}diff --git a/docs/content/docs/cua/reference/mcp-server/tools.mdx b/docs/content/docs/cua/reference/mcp-server/tools.mdxdeleted file mode 100644index 0b4616ef05..0000000000--- a/docs/content/docs/cua/reference/mcp-server/tools.mdx+++ /dev/null@@ -1,66 +0,0 @@-----title: Tools------## Available Tools--The MCP server exposes the following tools to Claude:--### Core Task Execution Tools--1. **`run_cua_task`** - Run a single Computer-Use Agent task with the given instruction- - `task` (string): The task description for the agent to execute- - `session_id` (string, optional): Session ID for multi-client support. If not provided, a new session will be created- - Returns: Tuple of (combined text output, final screenshot)--2. **`run_multi_cua_tasks`** - Run multiple tasks in sequence or concurrently- - `tasks` (list of strings): List of task descriptions to execute- - `session_id` (string, optional): Session ID for multi-client support. If not provided, a new session will be created- - `concurrent` (boolean, optional): If true, run tasks concurrently. If false, run sequentially (default)- - Returns: List of tuples (combined text output, screenshot) for each task--### Utility Tools--3. **`screenshot_cua`** - Take a screenshot of the current screen- - `session_id` (string, optional): Session ID for multi-client support. If not provided, a new session will be created- - Returns: Screenshot image--4. **`get_session_stats`** - Get statistics about active sessions and resource usage- - Returns: Dictionary with session statistics including total sessions, active tasks, and session details--5. **`cleanup_session`** - Cleanup a specific session and release its resources- - `session_id` (string): The session ID to cleanup- - Returns: Confirmation message--## Session Management--The MCP server supports multi-client sessions with automatic resource management:--- **Session Isolation**: Each client can have its own session with isolated computer instances-- **Resource Pooling**: Computer instances are pooled for efficient resource usage-- **Automatic Cleanup**: Idle sessions are automatically cleaned up after 10 minutes-- **Concurrent Tasks**: Multiple tasks can run concurrently within the same session-- **Progress Reporting**: Real-time progress updates during task execution--## Usage Examples--### Basic Task Execution--```-"Open Chrome and navigate to github.com"-"Create a folder called 'Projects' on my desktop"-```--### Multi-Task Execution--```-"Run these tasks: 1) Open Finder, 2) Navigate to Documents, 3) Create a new folder called 'Work'"-```--### Session Management--```-"Take a screenshot of the current screen"-"Show me the session statistics"-"Cleanup session abc123"-```diff --git a/docs/content/docs/cua/reference/mcp-server/usage.mdx b/docs/content/docs/cua/reference/mcp-server/usage.mdxdeleted file mode 100644index b329d2b645..0000000000--- a/docs/content/docs/cua/reference/mcp-server/usage.mdx+++ /dev/null@@ -1,169 +0,0 @@-----title: Usage------## Basic Usage--Once configured, you can simply ask Claude to perform computer tasks:--- "Open Chrome and go to github.com"-- "Create a folder called 'Projects' on my desktop"-- "Find all PDFs in my Downloads folder"-- "Take a screenshot and highlight the error message"--Claude will automatically use your Cua agent to perform these tasks.--## Advanced Features--### Progress Reporting--The MCP server provides real-time progress updates during task execution:--- Task progress is reported as percentages (0-100%)-- Multi-task operations show progress for each individual task-- Progress updates are streamed to the MCP client for real-time feedback--### Error Handling--Robust error handling ensures reliable operation:--- Failed tasks return error messages with screenshots when possible-- Session state is preserved even when individual tasks fail-- Automatic cleanup prevents resource leaks-- Detailed error logging for troubleshooting--### Concurrent Task Execution--For improved performance, multiple tasks can run concurrently:--- Set `concurrent=true` in `run_multi_cua_tasks` for parallel execution-- Each task runs in its own context with isolated state-- Progress tracking works for both sequential and concurrent modes-- Resource pooling ensures efficient computer instance usage--### Session Management--Multi-client support with automatic resource management:--- Each client gets isolated sessions with separate computer instances-- Sessions automatically clean up after 10 minutes of inactivity-- Resource pooling prevents resource exhaustion-- Session statistics available for monitoring--## Target Computer Options--By default, the MCP server runs Cua in a virtual machine for safety. However, you can also configure it to run on your local system.--### Default: Using a VM (Recommended)--The MCP server will automatically start and connect to a VM based on your platform. This is the safest option as AI actions are isolated from your host system.--No additional configuration is needed - this is the default behavior.--### Option: Targeting Your Local Desktop--<Callout type="warn">- **Warning:** When targeting your local system, AI models have direct access to your desktop and- may perform risky actions. Use with caution.-</Callout>--To have the MCP server control your local desktop instead of a VM:--1. **Start the Computer Server on your host:**--```bash-pip install cua-computer-server-python -m computer_server-```--2. **Configure the MCP server to use your host system:**--Add the `CUA_USE_HOST_COMPUTER_SERVER` environment variable to your MCP client configuration:--<Tabs items={['Claude Desktop', 'Other MCP Clients']}>- <Tab value="Claude Desktop">- Update your Claude Desktop config (see [Installation](/cua/reference/mcp-server/installation)) to include the environment variable:-- ```json- {- "mcpServers": {- "cua-agent": {- "command": "/bin/bash",- "args": ["~/.cua/start_mcp_server.sh"],- "env": {- "CUA_MODEL_NAME": "anthropic/claude-sonnet-4-5-20250929",- "CUA_USE_HOST_COMPUTER_SERVER": "true"- }- }- }- }- ```-- </Tab>- <Tab value="Other MCP Clients">- Set the environment variable in your MCP client configuration:-- ```bash- export CUA_USE_HOST_COMPUTER_SERVER=true- ```-- Then start your MCP client as usual.-- </Tab>-</Tabs>--3. **Restart your MCP client** (e.g., Claude Desktop) to apply the changes.--Now Claude will control your local desktop directly when you ask it to perform computer tasks.--## Usage Examples--### Single Task Execution--```-"Open Safari and navigate to apple.com"-"Create a new folder on the desktop called 'My Projects'"-"Take a screenshot of the current screen"-```--### Multi-Task Execution (Sequential)--```-"Run these tasks in order: 1) Open Finder, 2) Navigate to Documents folder, 3) Create a new folder called 'Work'"-```--### Multi-Task Execution (Concurrent)--```-"Run these tasks simultaneously: 1) Open Chrome, 2) Open Safari, 3) Open Finder"-```--### Session Management--```-"Show me the current session statistics"-"Take a screenshot using session abc123"-"Cleanup session xyz789"-```--### Error Recovery--```-"Try to open a non-existent application and show me the error"-"Find all files with .tmp extension and delete them safely"-```--## First-time Usage Notes--**API Keys**: Ensure you have valid API keys:--- Add your Anthropic API key in the Claude Desktop config (as shown above)-- Or set it as an environment variable in your shell profile-- **Required**: The MCP server needs an API key to authenticate with the model provider--**Model Selection**: Choose the appropriate model for your needs:--- **Claude Sonnet 4**: Latest model with best performance (`anthropic/claude-sonnet-4-20250514`)-- **Computer-Use Preview**: Specialized for computer tasks (`openai/computer-use-preview`)-- **Local Models**: For privacy-sensitive environments-- **Ollama**: For offline usagediff --git a/docs/content/docs/cua/reference/meta.json b/docs/content/docs/cua/reference/meta.jsondeleted file mode 100644index 1c6f97e05d..0000000000--- a/docs/content/docs/cua/reference/meta.json+++ /dev/null@@ -1,6 +0,0 @@-{- "title": "Reference",- "description": "SDK and CLI API reference",- "icon": "FileText",- "pages": ["desktop-sandbox", "sandbox-sdk", "agent-sdk", "cli"]-}diff --git a/docs/content/docs/cua/reference/sandbox-sdk/index.mdx b/docs/content/docs/cua/reference/sandbox-sdk/index.mdxdeleted file mode 100644index 47d1d30dfc..0000000000--- a/docs/content/docs/cua/reference/sandbox-sdk/index.mdx+++ /dev/null@@ -1,2871 +0,0 @@-----title: Sandbox SDK API Reference-description: Python API reference for cua-sandbox — creating and controlling sandboxes------{/*- AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY- Generated by: npx tsx scripts/docs-generators/python-sdk.ts- Source: libs/python/cua-sandbox/cua_sandbox- Version: 0.1.0-*/}--import { Callout } from 'fumadocs-ui/components/callout';-import { Tabs, Tab } from 'fumadocs-ui/components/tabs';-import { VersionHeader } from '@/components/version-selector';--<VersionHeader- versions={[{"version":"0.1","href":"/cua/reference/sandbox-sdk","isCurrent":true}]}- currentVersion="0.1"- fullVersion="0.1.0"- packageName="cua-sandbox"-/>--cua-sandbox — ephemeral and persistent sandboxed computer environments.--Usage::-- import cua-- # Configure API access- cua.configure(api_key="sk-...")-- # Local sandbox- async with cua.sandbox(local=True) as sb:- await sb.screenshot()-- # Localhost (no sandbox, direct host control)- async with cua.localhost() as host:- await host.mouse.click(100, 200)--## Classes--| Class | Description |-|-------|-------------|-| [`Image`](#image) | Immutable, chainable image specification. |-| [`Localhost`](#localhost) | Direct host control via cua_auto — no sandboxing. |-| [`Sandbox`](#sandbox) | A sandboxed computer environment. |-| [`SandboxInfo`](#sandboxinfo) | Metadata for a local or cloud sandbox. |-| [`CloudTransport`](#cloudtransport) | Transport that provisions / connects to a Cua cloud VM. |--## Functions--| Function | Description |-|----------|-------------|-| [`login`](#login) | Open the Cua login page in a browser and store credentials. |-| [`whoami`](#whoami) | Return info about the authenticated user. |-| [`configure`](#configure) | Set global configuration for the Cua SDK. |-------## Image--Immutable, chainable image specification.--Each mutation method returns a new Image instance so that builders-can be forked at any point.--### Constructor--```python-Image(self, os_type: str, distro: str, version: str, kind: Optional[str] = None, _layers: Tuple[Dict[str, Any], ...] = (), _env: Tuple[Tuple[str, str], ...] = (), _ports: Tuple[int, ...] = (), _files: Tuple[Tuple[str, str], ...] = (), _registry: Optional[str] = None, _disk_path: Optional[str] = None, _agent_type: Optional[str] = None) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `os_type` | `str` | |-| `distro` | `str` | |-| `version` | `str` | |-| `kind` | `Optional[str]` | |--### Methods--#### Image.linux--```python-def linux(cls, distro: str = 'ubuntu', version: str = '24.04', kind: str = 'container') -> Image-```--Linux image. Defaults to 'container' (Docker/XFCE). Use kind='vm' for QEMU.--#### Image.macos--```python-def macos(cls, version: str = '26', kind: str = 'vm') -> Image-```--macOS image. Always a VM (Apple Virtualization / Lume).--Supported versions: ``"15"`` / ``"sequoia"``, ``"26"`` / ``"tahoe"``.--#### Image.windows--```python-def windows(cls, version: str = '11', kind: str = 'vm') -> Image-```--Windows image. Always a VM (QEMU or Hyper-V).--#### Image.android--```python-def android(cls, version: str = '14', kind: str = 'vm') -> Image-```--Android image. Always a VM (QEMU emulator).--#### Image.from_registry--```python-def from_registry(cls, ref: str) -> Image-```--Create an image from a registry reference. kind is resolved after pull.--#### Image.from_file--```python-def from_file(cls, path: str, os_type: str = 'windows', kind: str = 'vm', agent_type: Optional[str] = None) -> Image-```--Create an image from a local disk, ISO file, or URL.--Supported formats: qcow2, vhdx, raw, img, iso.-URLs (http/https) are downloaded automatically. Zip files are extracted.-For ISOs, the runtime will create a qcow2 disk and attach the ISO-as a CD-ROM for installation/boot.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `path` | `Any` | Local file path or URL (http/https). |-| `os_type` | `Any` | OS type hint ("linux", "windows", "macos", "android"). |-| `kind` | `Any` | "vm" or "container". |-| `agent_type` | `Any` | Agent type hint (e.g. "osworld" for OSWorld Flask server). |--#### Image.from_dict--```python-def from_dict(cls, data: Dict[str, Any]) -> Image-```--Reconstruct an Image from a serialized spec dict.--#### Image.apt_install--```python-def apt_install(self, packages: str = ()) -> Image-```--Install packages via apt (Linux only).--#### Image.brew_install--```python-def brew_install(self, packages: str = ()) -> Image-```--Install packages via Homebrew (macOS).--#### Image.choco_install--```python-def choco_install(self, packages: str = ()) -> Image-```--Install packages via Chocolatey (Windows).--#### Image.winget_install--```python-def winget_install(self, packages: str = ()) -> Image-```--Install packages via winget (Windows).--#### Image.apk_install--```python-def apk_install(self, apk_paths: str = ()) -> Image-```--Install APK files via adb (Android only).--#### Image.pwa_install--```python-def pwa_install(self, manifest_url: str, package_name: Optional[str] = None, keystore: Optional[str] = None, keystore_alias: str = 'android', keystore_password: str = 'android') -> 'Image'-```--Build an APK from a PWA manifest URL via Bubblewrap and install it (Android only).--Bubblewrap reads the Web App Manifest at *manifest_url*, generates a-Trusted Web Activity (TWA) APK signed with *keystore*, and installs it-via adb. The keystore's SHA-256 fingerprint must match what the server-serves from ``/.well-known/assetlinks.json`` so Chrome trusts the TWA-and skips the browser UI entirely.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `manifest_url` | `Any` | Full URL to the PWA's ``manifest.json``. |--**Example:**--```python-package_name: Android package ID. Defaults to a reversed-hostname- derivation (e.g. ``"com.example.app"``).-keystore: Path to a ``*.keystore`` / ``*.jks`` file. When- omitted a fresh keystore is generated and cached under- ``~/.cua/cua-sandbox/pwa-cache/``. Pass the keystore- bundled with your PWA repo so the fingerprint is- deterministic and pre-loaded into ``assetlinks.json``.-keystore_alias: Key alias inside the keystore (default ``"android"``).-keystore_password: Password for both the store and the key- (default ``"android"``).-```--#### Image.uv_install--```python-def uv_install(self, packages: str = ()) -> Image-```--Install Python packages via uv add into the cua-server project.--#### Image.pip_install--```python-def pip_install(self, packages: str = ()) -> Image-```--Install Python packages via pip.--#### Image.run--```python-def run(self, command: str) -> Image-```--Run a shell command during image build.--#### Image.env--```python-def env(self, variables: str = {}) -> Image-```--Set environment variables.--#### Image.copy--```python-def copy(self, src: str, dst: str) -> Image-```--Copy a file into the image.--#### Image.expose--```python-def expose(self, port: int) -> Image-```--Expose a port.--#### Image.to_dict--```python-def to_dict(self) -> Dict[str, Any]-```--Serialize to a plain dict suitable for JSON or cloud API.--#### Image.to_cloud_init--```python-def to_cloud_init(self) -> str-```--Generate a cloud-init user-data script from the image layers.-------## Localhost--Direct host control via cua_auto — no sandboxing.--### Constructor--```python-Localhost(self) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `screen` | `Any` | |-| `mouse` | `Any` | |-| `keyboard` | `Any` | |-| `clipboard` | `Any` | |-| `shell` | `Any` | |-| `window` | `Any` | |-| `terminal` | `Any` | |--### Methods--#### Localhost.disconnect--```python-async def disconnect(self) -> None-```--#### Localhost.screenshot--```python-async def screenshot(self, text: Optional[str] = None) -> bytes-```--#### Localhost.screenshot_base64--```python-async def screenshot_base64(self, text: Optional[str] = None) -> str-```--#### Localhost.get_environment--```python-async def get_environment(self) -> str-```--#### Localhost.get_dimensions--```python-async def get_dimensions(self) -> tuple[int, int]-```--#### Localhost.connect--```python-def connect(cls) -> '_ConnectResult'-```--Connect to the local machine.--Supports both ``await`` and ``async with``.--**Example:**--```python-# plain await-host = await Localhost.connect()-await host.shell.run("echo hello")-await host.disconnect()--# context manager-async with Localhost.connect() as host:- await host.shell.run("echo hello")-```-------## Sandbox--A sandboxed computer environment.--Provides programmatic control of a VM or container through a unified-interface: ``.mouse``, ``.keyboard``, ``.screen``, ``.clipboard``,-``.shell``, ``.window``, and ``.terminal``.--Sandboxes are always isolated — they never control the host machine-directly. For unsandboxed host control, use :func:`cua_sandbox.localhost`.--There are three ways to obtain a Sandbox:--1. **Persistent** — provision and keep alive after the script exits::-- sb = await Sandbox.create(Image.desktop("ubuntu"))- await sb.shell.run("whoami")- await sb.disconnect()--2. **Connect** — attach to an already-running sandbox by name::-- sb = await Sandbox.connect("my-sandbox")- await sb.screenshot()- await sb.disconnect()--3. **Ephemeral** — auto-destroyed when the ``async with`` block exits::-- async with Sandbox.ephemeral(Image.desktop("ubuntu")) as sb:- await sb.shell.run("whoami")--### Constructor--```python-Sandbox(self, transport: Transport, name: Optional[str] = None, _runtime: Optional[Runtime] = None, _runtime_info: Optional[RuntimeInfo] = None, _ephemeral: Optional[bool] = None)-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `name` | `Any` | |-| `screen` | `Any` | |-| `mouse` | `Any` | |-| `keyboard` | `Any` | |-| `clipboard` | `Any` | |-| `shell` | `Any` | |-| `window` | `Any` | |-| `terminal` | `Any` | |-| `mobile` | `Any` | |-| `tunnel` | `Any` | |--### Methods--#### Sandbox.disconnect--```python-async def disconnect(self) -> None-```--Drop the transport connection. The sandbox keeps running.--#### Sandbox.destroy--```python-async def destroy(self) -> None-```--Disconnect and permanently delete the sandbox (VM/container).--#### Sandbox.screenshot--```python-async def screenshot(self, text: Optional[str] = None, format: str = 'png', quality: int = 95) -> bytes-```--#### Sandbox.screenshot_base64--```python-async def screenshot_base64(self, text: Optional[str] = None, format: str = 'png', quality: int = 95) -> str-```--#### Sandbox.get_environment--```python-async def get_environment(self) -> str-```--#### Sandbox.get_display_url--```python-async def get_display_url(self, share: bool = False) -> str-```--Return a URL to view this sandbox's display.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `share` | `Any` | If True, return a public link with embedded credentials (cloud only). If False, return a direct connection URL. |--#### Sandbox.get_dimensions--```python-async def get_dimensions(self) -> tuple[int, int]-```--#### Sandbox.create--```python-async def create(cls, image: Image, name: Optional[str] = None, api_key: Optional[str] = None, local: bool = False, runtime: Optional['Runtime'] = None, cpu: Optional[int] = None, memory_mb: Optional[int] = None, disk_gb: Optional[int] = None, region: str = 'us-east-1') -> 'Sandbox'-```--Provision a new persistent sandbox and return it connected.--The sandbox is kept alive after your script exits — call ``close()``-when you are done, or use :meth:`ephemeral` if you want it destroyed-automatically.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `image` | `Any` | Image to run (e.g. ``Image.desktop("ubuntu")``). |-| `name` | `Any` | Optional name to assign to the sandbox. |-| `api_key` | `Any` | Cua API key for cloud sandboxes. |-| `local` | `Any` | Use a local runtime instead of cloud. |-| `runtime` | `Any` | Explicit runtime backend (DockerRuntime, QEMURuntime, etc.). |-| `cpu` | `Any` | Number of CPUs for the cloud sandbox. |-| `memory_mb` | `Any` | Memory in MB for the cloud sandbox. |-| `disk_gb` | `Any` | Disk size in GB for the cloud sandbox. |-| `region` | `Any` | Cloud region (default ``"us-east-1"``). |--**Example:**--```python-sb = await Sandbox.create(Image.desktop("ubuntu"))-await sb.shell.run("uname -a")-print(sb.name) # save to reconnect later-await sb.disconnect()-```--#### Sandbox.connect--```python-def connect(cls, name: str, api_key: Optional[str] = None, local: bool = False, ws_url: Optional[str] = None, http_url: Optional[str] = None, container_name: Optional[str] = None, cpu: Optional[int] = None, memory_mb: Optional[int] = None, disk_gb: Optional[int] = None, region: str = 'us-east-1') -> '_ConnectResult'-```--Connect to an existing sandbox by name.--Supports both ``await`` and ``async with``. When used as a context-manager, ``disconnect()`` is called on exit — the sandbox keeps running.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `name` | `Any` | Name of the existing sandbox. |-| `api_key` | `Any` | Cua API key for cloud sandboxes. |-| `ws_url` | `Any` | WebSocket URL for a remote computer-server. |-| `http_url` | `Any` | HTTP base URL for a remote computer-server. |-| `container_name` | `Any` | Container name for cloud auth (HTTP transport). |-| `region` | `Any` | Cloud region (default ``"us-east-1"``). |--**Example:**--```python-# plain await-sb = await Sandbox.connect("my-sandbox")-await sb.screenshot()-await sb.disconnect()--# context manager — disconnects on exit, sandbox keeps running-async with Sandbox.connect("my-sandbox") as sb:- await sb.screenshot()-```--#### Sandbox.ephemeral--```python-async def ephemeral(cls, image: Image, name: Optional[str] = None, api_key: Optional[str] = None, local: bool = False, runtime: Optional['Runtime'] = None, cpu: Optional[int] = None, memory_mb: Optional[int] = None, disk_gb: Optional[int] = None, region: str = 'us-east-1') -> AsyncIterator['Sandbox']-```--Create an ephemeral sandbox that is automatically destroyed on exit.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `image` | `Any` | Image to run (e.g. ``Image.desktop("ubuntu")``). |-| `name` | `Any` | Optional name to assign to the sandbox. |-| `api_key` | `Any` | Cua API key for cloud sandboxes. |-| `local` | `Any` | Use a local runtime instead of cloud. |-| `runtime` | `Any` | Explicit runtime backend (DockerRuntime, QEMURuntime, etc.). |-| `cpu` | `Any` | Number of CPUs for the cloud sandbox. |-| `memory_mb` | `Any` | Memory in MB for the cloud sandbox. |-| `disk_gb` | `Any` | Disk size in GB for the cloud sandbox. |-| `region` | `Any` | Cloud region (default ``"us-east-1"``). |--**Example:**--```python-async with Sandbox.ephemeral(Image.desktop("ubuntu")) as sb:- await sb.shell.run("whoami")-# sandbox is destroyed here-```--#### Sandbox.list--```python-async def list(cls, local: bool = False, api_key: Optional[str] = None) -> 'list[SandboxInfo]'-```--List running and suspended sandboxes.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `local` | `Any` | If True, list local sandboxes (Lume, Docker, QEMU). If False, list cloud sandboxes. |-| `api_key` | `Any` | Cua API key for cloud sandboxes. |--#### Sandbox.get_info--```python-async def get_info(cls, name: str, local: bool = False, api_key: Optional[str] = None) -> 'SandboxInfo'-```--Get metadata for a specific sandbox.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `name` | `Any` | Sandbox name. |-| `local` | `Any` | If True, look up in local runtimes. |-| `api_key` | `Any` | Cua API key for cloud. |--#### Sandbox.suspend--```python-async def suspend(cls, name: str, local: bool = False, api_key: Optional[str] = None) -> None-```--Suspend a running sandbox (save state).--For local QEMU bare-metal: saves a QMP snapshot then quits the process.-For local Docker/QEMU-docker: pauses the container.-For local Lume: stops the Lume VM (Lume persists state).-For cloud: calls POST /v1/vms/\{name\}/stop.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `name` | `Any` | Sandbox name. |-| `local` | `Any` | If True, operate on a local sandbox. |-| `api_key` | `Any` | Cua API key for cloud. |--#### Sandbox.resume--```python-async def resume(cls, name: str, local: bool = False, api_key: Optional[str] = None) -> 'Sandbox'-```--Resume a suspended sandbox and return a connected Sandbox.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `name` | `Any` | Sandbox name. |-| `local` | `Any` | If True, resume a local sandbox. |-| `api_key` | `Any` | Cua API key for cloud. |--**Returns:** A connected Sandbox ready to use.--#### Sandbox.restart--```python-async def restart(cls, name: str, local: bool = False, api_key: Optional[str] = None) -> 'Sandbox'-```--Restart a sandbox (suspend then resume) and return a connected Sandbox.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `name` | `Any` | Sandbox name. |-| `local` | `Any` | If True, restart a local sandbox. |-| `api_key` | `Any` | Cua API key for cloud. |--**Returns:** A connected Sandbox ready to use.--#### Sandbox.delete--```python-async def delete(cls, name: str, local: bool = False, api_key: Optional[str] = None) -> None-```--Permanently delete a sandbox.--For local sandboxes, stops the VM and removes the state file.-For cloud sandboxes, calls DELETE /v1/vms/\{name\}.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `name` | `Any` | Sandbox name. |-| `local` | `Any` | If True, delete a local sandbox. |-| `api_key` | `Any` | Cua API key for cloud. |-------## SandboxInfo--Metadata for a local or cloud sandbox.--### Constructor--```python-SandboxInfo(self, name: str, status: str, source: str, os_type: Optional[str] = None, host: Optional[str] = None, vnc_url: Optional[str] = None, api_url: Optional[str] = None, created_at: Optional[str] = None) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `name` | `str` | |-| `status` | `str` | |-| `source` | `str` | |-| `os_type` | `Optional[str]` | |-| `host` | `Optional[str]` | |-| `vnc_url` | `Optional[str]` | |-| `api_url` | `Optional[str]` | |-| `created_at` | `Optional[str]` | |-------## CloudTransport--*Inherits from: Transport*--Transport that provisions / connects to a Cua cloud VM.--### Constructor--```python-CloudTransport(self, name: Optional[str] = None, api_key: Optional[str] = None, base_url: Optional[str] = None, image: Optional[Any] = None, cpu: Optional[int] = None, memory_mb: Optional[int] = None, disk_gb: Optional[int] = None, region: str = 'us-east-1')-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `name` | `Optional[str]` | |--### Methods--#### CloudTransport.connect--```python-async def connect(self) -> None-```--#### CloudTransport.disconnect--```python-async def disconnect(self) -> None-```--#### CloudTransport.delete_vm--```python-async def delete_vm(self) -> None-```--Delete the cloud VM via the platform API.--#### CloudTransport.suspend_vm--```python-async def suspend_vm(self) -> None-```--Stop (suspend) the cloud VM.--#### CloudTransport.resume_vm--```python-async def resume_vm(self) -> None-```--Start (resume) the cloud VM.--#### CloudTransport.restart_vm--```python-async def restart_vm(self) -> None-```--Restart the cloud VM.--#### CloudTransport.send--```python-async def send(self, action: str, params: Any = {}) -> Any-```--#### CloudTransport.screenshot--```python-async def screenshot(self, format: str = 'png', quality: int = 95) -> bytes-```--#### CloudTransport.get_screen_size--```python-async def get_screen_size(self) -> Dict[str, int]-```--#### CloudTransport.get_environment--```python-async def get_environment(self) -> str-```--#### CloudTransport.get_display_url--```python-async def get_display_url(self, share: bool = False) -> str-```--## login--```python-def login(base_url: Optional[str] = None) -> None-```--Open the Cua login page in a browser and store credentials.--This initiates a Clerk-based browser authentication flow.-The user completes login in their browser, and the resulting-API key is stored in ~/.cua/credentials.--## whoami--```python-def whoami(api_key: Optional[str] = None) -> Dict[str, Any]-```--Return info about the authenticated user.--**Returns:** Dict with user info (id, email, etc.) from the Cua API.--## configure--```python-def configure(api_key: Optional[str] = None, base_url: Optional[str] = None) -> None-```--Set global configuration for the Cua SDK.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `api_key` | `Any` | API key for cloud sandboxes. |-| `base_url` | `Any` | Base URL for the Cua cloud API. |-------## localhost--Localhost — wraps cua_auto directly. No sandbox, no computer-server.--All cua_auto calls are sync; async wrappers use asyncio.to_thread().--Usage::-- from cua import localhost-- async with localhost() as host:- await host.mouse.click(100, 200)- img = await host.screen.screenshot()-------## LocalTransport--*Inherits from: Transport*--Transport that routes commands directly to cua_auto modules.--### Methods--#### LocalTransport.connect--```python-async def connect(self) -> None-```--#### LocalTransport.disconnect--```python-async def disconnect(self) -> None-```--#### LocalTransport.send--```python-async def send(self, action: str, params: Any = {}) -> Any-```--Dispatch a named command to the appropriate cua_auto module.--#### LocalTransport.screenshot--```python-async def screenshot(self, format: str = 'png', quality: int = 95) -> bytes-```--#### LocalTransport.get_screen_size--```python-async def get_screen_size(self) -> Dict[str, int]-```--#### LocalTransport.get_environment--```python-async def get_environment(self) -> str-```-------## Localhost--Direct host control via cua_auto — no sandboxing.--### Constructor--```python-Localhost(self) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `screen` | `Any` | |-| `mouse` | `Any` | |-| `keyboard` | `Any` | |-| `clipboard` | `Any` | |-| `shell` | `Any` | |-| `window` | `Any` | |-| `terminal` | `Any` | |--### Methods--#### Localhost.disconnect--```python-async def disconnect(self) -> None-```--#### Localhost.screenshot--```python-async def screenshot(self, text: Optional[str] = None) -> bytes-```--#### Localhost.screenshot_base64--```python-async def screenshot_base64(self, text: Optional[str] = None) -> str-```--#### Localhost.get_environment--```python-async def get_environment(self) -> str-```--#### Localhost.get_dimensions--```python-async def get_dimensions(self) -> tuple[int, int]-```--#### Localhost.connect--```python-def connect(cls) -> '_ConnectResult'-```--Connect to the local machine.--Supports both ``await`` and ``async with``.--**Example:**--```python-# plain await-host = await Localhost.connect()-await host.shell.run("echo hello")-await host.disconnect()--# context manager-async with Localhost.connect() as host:- await host.shell.run("echo hello")-```--### localhost--```python-async def localhost() -> AsyncIterator[Localhost]-```--Async context manager yielding a Localhost instance.--.. deprecated::- Prefer ``Localhost.connect()`` instead.-------## sandbox--Sandbox class — the primary entry point for sandboxed environments.--Exposes .mouse, .keyboard, .screen, .clipboard, .shell, .window, .terminal-as interface objects backed by a Transport.--Usage::-- from cua import Sandbox, Image-- # Provision a new persistent sandbox- sb = await Sandbox.create(Image.desktop("ubuntu"))- await sb.shell.run("uname -a")- await sb.disconnect()-- # Connect to an existing sandbox by name (plain await or async with)- sb = await Sandbox.connect("my-sandbox")- await sb.screenshot()- await sb.disconnect()-- async with Sandbox.connect("my-sandbox") as sb: # disconnects on exit- await sb.screenshot()-- # Ephemeral — auto-destroyed on exit- async with Sandbox.ephemeral(Image.desktop("ubuntu")) as sb:- await sb.shell.run("whoami")-------## Image--Immutable, chainable image specification.--Each mutation method returns a new Image instance so that builders-can be forked at any point.--### Constructor--```python-Image(self, os_type: str, distro: str, version: str, kind: Optional[str] = None, _layers: Tuple[Dict[str, Any], ...] = (), _env: Tuple[Tuple[str, str], ...] = (), _ports: Tuple[int, ...] = (), _files: Tuple[Tuple[str, str], ...] = (), _registry: Optional[str] = None, _disk_path: Optional[str] = None, _agent_type: Optional[str] = None) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `os_type` | `str` | |-| `distro` | `str` | |-| `version` | `str` | |-| `kind` | `Optional[str]` | |--### Methods--#### Image.linux--```python-def linux(cls, distro: str = 'ubuntu', version: str = '24.04', kind: str = 'container') -> Image-```--Linux image. Defaults to 'container' (Docker/XFCE). Use kind='vm' for QEMU.--#### Image.macos--```python-def macos(cls, version: str = '26', kind: str = 'vm') -> Image-```--macOS image. Always a VM (Apple Virtualization / Lume).--Supported versions: ``"15"`` / ``"sequoia"``, ``"26"`` / ``"tahoe"``.--#### Image.windows--```python-def windows(cls, version: str = '11', kind: str = 'vm') -> Image-```--Windows image. Always a VM (QEMU or Hyper-V).--#### Image.android--```python-def android(cls, version: str = '14', kind: str = 'vm') -> Image-```--Android image. Always a VM (QEMU emulator).--#### Image.from_registry--```python-def from_registry(cls, ref: str) -> Image-```--Create an image from a registry reference. kind is resolved after pull.--#### Image.from_file--```python-def from_file(cls, path: str, os_type: str = 'windows', kind: str = 'vm', agent_type: Optional[str] = None) -> Image-```--Create an image from a local disk, ISO file, or URL.--Supported formats: qcow2, vhdx, raw, img, iso.-URLs (http/https) are downloaded automatically. Zip files are extracted.-For ISOs, the runtime will create a qcow2 disk and attach the ISO-as a CD-ROM for installation/boot.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `path` | `Any` | Local file path or URL (http/https). |-| `os_type` | `Any` | OS type hint ("linux", "windows", "macos", "android"). |-| `kind` | `Any` | "vm" or "container". |-| `agent_type` | `Any` | Agent type hint (e.g. "osworld" for OSWorld Flask server). |--#### Image.from_dict--```python-def from_dict(cls, data: Dict[str, Any]) -> Image-```--Reconstruct an Image from a serialized spec dict.--#### Image.apt_install--```python-def apt_install(self, packages: str = ()) -> Image-```--Install packages via apt (Linux only).--#### Image.brew_install--```python-def brew_install(self, packages: str = ()) -> Image-```--Install packages via Homebrew (macOS).--#### Image.choco_install--```python-def choco_install(self, packages: str = ()) -> Image-```--Install packages via Chocolatey (Windows).--#### Image.winget_install--```python-def winget_install(self, packages: str = ()) -> Image-```--Install packages via winget (Windows).--#### Image.apk_install--```python-def apk_install(self, apk_paths: str = ()) -> Image-```--Install APK files via adb (Android only).--#### Image.pwa_install--```python-def pwa_install(self, manifest_url: str, package_name: Optional[str] = None, keystore: Optional[str] = None, keystore_alias: str = 'android', keystore_password: str = 'android') -> 'Image'-```--Build an APK from a PWA manifest URL via Bubblewrap and install it (Android only).--Bubblewrap reads the Web App Manifest at *manifest_url*, generates a-Trusted Web Activity (TWA) APK signed with *keystore*, and installs it-via adb. The keystore's SHA-256 fingerprint must match what the server-serves from ``/.well-known/assetlinks.json`` so Chrome trusts the TWA-and skips the browser UI entirely.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `manifest_url` | `Any` | Full URL to the PWA's ``manifest.json``. |--**Example:**--```python-package_name: Android package ID. Defaults to a reversed-hostname- derivation (e.g. ``"com.example.app"``).-keystore: Path to a ``*.keystore`` / ``*.jks`` file. When- omitted a fresh keystore is generated and cached under- ``~/.cua/cua-sandbox/pwa-cache/``. Pass the keystore- bundled with your PWA repo so the fingerprint is- deterministic and pre-loaded into ``assetlinks.json``.-keystore_alias: Key alias inside the keystore (default ``"android"``).-keystore_password: Password for both the store and the key- (default ``"android"``).-```--#### Image.uv_install--```python-def uv_install(self, packages: str = ()) -> Image-```--Install Python packages via uv add into the cua-server project.--#### Image.pip_install--```python-def pip_install(self, packages: str = ()) -> Image-```--Install Python packages via pip.--#### Image.run--```python-def run(self, command: str) -> Image-```--Run a shell command during image build.--#### Image.env--```python-def env(self, variables: str = {}) -> Image-```--Set environment variables.--#### Image.copy--```python-def copy(self, src: str, dst: str) -> Image-```--Copy a file into the image.--#### Image.expose--```python-def expose(self, port: int) -> Image-```--Expose a port.--#### Image.to_dict--```python-def to_dict(self) -> Dict[str, Any]-```--Serialize to a plain dict suitable for JSON or cloud API.--#### Image.to_cloud_init--```python-def to_cloud_init(self) -> str-```--Generate a cloud-init user-data script from the image layers.-------## Transport--*Inherits from: ABC*--Base class for all transports.--### Methods--#### Transport.connect--```python-async def connect(self) -> None-```--Establish the transport connection.--#### Transport.disconnect--```python-async def disconnect(self) -> None-```--Tear down the transport connection.--#### Transport.send--```python-async def send(self, action: str, params: Any = {}) -> Any-```--Send a command and return the result.--#### Transport.screenshot--```python-async def screenshot(self, format: str = 'png', quality: int = 95) -> bytes-```--Capture a screenshot and return raw image bytes.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `format` | `Any` | "png" (lossless, default) or "jpeg" (lossy, ~5-10x smaller). |-| `quality` | `Any` | JPEG quality 1-95, ignored for PNG. |--#### Transport.get_screen_size--```python-async def get_screen_size(self) -> Dict[str, int]-```--Return \{"width": ..., "height": ...\}.--#### Transport.get_environment--```python-async def get_environment(self) -> str-```--Return 'windows', 'mac', 'linux', or 'browser'.--#### Transport.forward_tunnel--```python-async def forward_tunnel(self, sandbox_port: int | str) -> 'TunnelInfo'-```--Forward *sandbox_port* to an available host port and return info.--Subclasses that support tunnelling must override this method.-The returned :class:`~cua_sandbox.interfaces.tunnel.TunnelInfo` must-have ``host`` and ``port`` set to the host-side address.--#### Transport.close_tunnel--```python-async def close_tunnel(self, info: 'TunnelInfo') -> None-```--Release a previously forwarded port or socket. No-op by default.--#### Transport.get_display_url--```python-async def get_display_url(self, share: bool = False) -> str-```--Return a URL to view this sandbox's display.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `share` | `Any` | If True, return a public link with embedded credentials (cloud only). If False, return a direct connection URL (localhost VNC for local runtimes; auth-gated URL for cloud). Raises NotImplementedError for transports that don't expose a display (e.g. HTTP, ADB). |-------## CloudTransport--*Inherits from: Transport*--Transport that provisions / connects to a Cua cloud VM.--### Constructor--```python-CloudTransport(self, name: Optional[str] = None, api_key: Optional[str] = None, base_url: Optional[str] = None, image: Optional[Any] = None, cpu: Optional[int] = None, memory_mb: Optional[int] = None, disk_gb: Optional[int] = None, region: str = 'us-east-1')-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `name` | `Optional[str]` | |--### Methods--#### CloudTransport.connect--```python-async def connect(self) -> None-```--#### CloudTransport.disconnect--```python-async def disconnect(self) -> None-```--#### CloudTransport.delete_vm--```python-async def delete_vm(self) -> None-```--Delete the cloud VM via the platform API.--#### CloudTransport.suspend_vm--```python-async def suspend_vm(self) -> None-```--Stop (suspend) the cloud VM.--#### CloudTransport.resume_vm--```python-async def resume_vm(self) -> None-```--Start (resume) the cloud VM.--#### CloudTransport.restart_vm--```python-async def restart_vm(self) -> None-```--Restart the cloud VM.--#### CloudTransport.send--```python-async def send(self, action: str, params: Any = {}) -> Any-```--#### CloudTransport.screenshot--```python-async def screenshot(self, format: str = 'png', quality: int = 95) -> bytes-```--#### CloudTransport.get_screen_size--```python-async def get_screen_size(self) -> Dict[str, int]-```--#### CloudTransport.get_environment--```python-async def get_environment(self) -> str-```--#### CloudTransport.get_display_url--```python-async def get_display_url(self, share: bool = False) -> str-```-------## HTTPTransport--*Inherits from: Transport*--Transport that communicates with computer-server over HTTP POST /cmd (SSE).--### Constructor--```python-HTTPTransport(self, base_url: str, api_key: Optional[str] = None, container_name: Optional[str] = None, timeout: float = 30.0)-```--### Methods--#### HTTPTransport.connect--```python-async def connect(self) -> None-```--#### HTTPTransport.disconnect--```python-async def disconnect(self) -> None-```--#### HTTPTransport.send--```python-async def send(self, action: str, params: Any = {}) -> Any-```--#### HTTPTransport.screenshot--```python-async def screenshot(self, format: str = 'png', quality: int = 95) -> bytes-```--#### HTTPTransport.get_screen_size--```python-async def get_screen_size(self) -> Dict[str, int]-```--#### HTTPTransport.get_environment--```python-async def get_environment(self) -> str-```-------## WebSocketTransport--*Inherits from: Transport*--Transport that communicates with computer-server over WebSocket.--### Constructor--```python-WebSocketTransport(self, url: str, api_key: Optional[str] = None)-```--### Methods--#### WebSocketTransport.connect--```python-async def connect(self) -> None-```--#### WebSocketTransport.disconnect--```python-async def disconnect(self) -> None-```--#### WebSocketTransport.send--```python-async def send(self, action: str, params: Any = {}) -> Any-```--#### WebSocketTransport.screenshot--```python-async def screenshot(self, format: str = 'png', quality: int = 95) -> bytes-```--#### WebSocketTransport.get_screen_size--```python-async def get_screen_size(self) -> Dict[str, int]-```--#### WebSocketTransport.get_environment--```python-async def get_environment(self) -> str-```-------## Runtime--*Inherits from: ABC*--Base class for local runtimes that create VMs/containers from an Image.--### Methods--#### Runtime.start--```python-async def start(self, image: Image, name: str, opts = {}) -> RuntimeInfo-```--Start a VM/container and return connection info.--#### Runtime.stop--```python-async def stop(self, name: str) -> None-```--Stop and clean up a VM/container.--#### Runtime.is_ready--```python-async def is_ready(self, info: RuntimeInfo, timeout: float = 120) -> bool-```--Wait until the computer-server inside is reachable.--#### Runtime.suspend--```python-async def suspend(self, name: str) -> None-```--Suspend (pause/save state of) a running VM.--#### Runtime.resume--```python-async def resume(self, image: 'Image', name: str, opts = {}) -> RuntimeInfo-```--Resume a suspended VM and return its RuntimeInfo.--#### Runtime.list--```python-async def list(self) -> list[dict]-```--List known VMs managed by this runtime.--Returns a list of dicts with at minimum: name, status.-------## RuntimeInfo--Connection info returned after a runtime spins up.--### Constructor--```python-RuntimeInfo(self, host: str, api_port: int, vnc_port: Optional[int] = None, api_key: Optional[str] = None, container_id: Optional[str] = None, name: Optional[str] = None, qmp_port: Optional[int] = None, environment: Optional[str] = None, agent_type: Optional[str] = None, guest_server_port: int = 8000, ssh_port: Optional[int] = None, ssh_username: Optional[str] = None, ssh_password: Optional[str] = None, ssh_key_filename: Optional[str] = None, vnc_host: Optional[str] = None, vnc_password: Optional[str] = None, grpc_port: Optional[int] = None) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `host` | `str` | |-| `api_port` | `int` | |-| `vnc_port` | `Optional[int]` | |-| `api_key` | `Optional[str]` | |-| `container_id` | `Optional[str]` | |-| `name` | `Optional[str]` | |-| `qmp_port` | `Optional[int]` | |-| `environment` | `Optional[str]` | |-| `agent_type` | `Optional[str]` | |-| `guest_server_port` | `int` | |-| `ssh_port` | `Optional[int]` | |-| `ssh_username` | `Optional[str]` | |-| `ssh_password` | `Optional[str]` | |-| `ssh_key_filename` | `Optional[str]` | |-| `vnc_host` | `Optional[str]` | |-| `vnc_password` | `Optional[str]` | |-| `grpc_port` | `Optional[int]` | |-------## SandboxInfo--Metadata for a local or cloud sandbox.--### Constructor--```python-SandboxInfo(self, name: str, status: str, source: str, os_type: Optional[str] = None, host: Optional[str] = None, vnc_url: Optional[str] = None, api_url: Optional[str] = None, created_at: Optional[str] = None) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `name` | `str` | |-| `status` | `str` | |-| `source` | `str` | |-| `os_type` | `Optional[str]` | |-| `host` | `Optional[str]` | |-| `vnc_url` | `Optional[str]` | |-| `api_url` | `Optional[str]` | |-| `created_at` | `Optional[str]` | |-------## Sandbox--A sandboxed computer environment.--Provides programmatic control of a VM or container through a unified-interface: ``.mouse``, ``.keyboard``, ``.screen``, ``.clipboard``,-``.shell``, ``.window``, and ``.terminal``.--Sandboxes are always isolated — they never control the host machine-directly. For unsandboxed host control, use :func:`cua_sandbox.localhost`.--There are three ways to obtain a Sandbox:--1. **Persistent** — provision and keep alive after the script exits::-- sb = await Sandbox.create(Image.desktop("ubuntu"))- await sb.shell.run("whoami")- await sb.disconnect()--2. **Connect** — attach to an already-running sandbox by name::-- sb = await Sandbox.connect("my-sandbox")- await sb.screenshot()- await sb.disconnect()--3. **Ephemeral** — auto-destroyed when the ``async with`` block exits::-- async with Sandbox.ephemeral(Image.desktop("ubuntu")) as sb:- await sb.shell.run("whoami")--### Constructor--```python-Sandbox(self, transport: Transport, name: Optional[str] = None, _runtime: Optional[Runtime] = None, _runtime_info: Optional[RuntimeInfo] = None, _ephemeral: Optional[bool] = None)-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `name` | `Any` | |-| `screen` | `Any` | |-| `mouse` | `Any` | |-| `keyboard` | `Any` | |-| `clipboard` | `Any` | |-| `shell` | `Any` | |-| `window` | `Any` | |-| `terminal` | `Any` | |-| `mobile` | `Any` | |-| `tunnel` | `Any` | |--### Methods--#### Sandbox.disconnect--```python-async def disconnect(self) -> None-```--Drop the transport connection. The sandbox keeps running.--#### Sandbox.destroy--```python-async def destroy(self) -> None-```--Disconnect and permanently delete the sandbox (VM/container).--#### Sandbox.screenshot--```python-async def screenshot(self, text: Optional[str] = None, format: str = 'png', quality: int = 95) -> bytes-```--#### Sandbox.screenshot_base64--```python-async def screenshot_base64(self, text: Optional[str] = None, format: str = 'png', quality: int = 95) -> str-```--#### Sandbox.get_environment--```python-async def get_environment(self) -> str-```--#### Sandbox.get_display_url--```python-async def get_display_url(self, share: bool = False) -> str-```--Return a URL to view this sandbox's display.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `share` | `Any` | If True, return a public link with embedded credentials (cloud only). If False, return a direct connection URL. |--#### Sandbox.get_dimensions--```python-async def get_dimensions(self) -> tuple[int, int]-```--#### Sandbox.create--```python-async def create(cls, image: Image, name: Optional[str] = None, api_key: Optional[str] = None, local: bool = False, runtime: Optional['Runtime'] = None, cpu: Optional[int] = None, memory_mb: Optional[int] = None, disk_gb: Optional[int] = None, region: str = 'us-east-1') -> 'Sandbox'-```--Provision a new persistent sandbox and return it connected.--The sandbox is kept alive after your script exits — call ``close()``-when you are done, or use :meth:`ephemeral` if you want it destroyed-automatically.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `image` | `Any` | Image to run (e.g. ``Image.desktop("ubuntu")``). |-| `name` | `Any` | Optional name to assign to the sandbox. |-| `api_key` | `Any` | Cua API key for cloud sandboxes. |-| `local` | `Any` | Use a local runtime instead of cloud. |-| `runtime` | `Any` | Explicit runtime backend (DockerRuntime, QEMURuntime, etc.). |-| `cpu` | `Any` | Number of CPUs for the cloud sandbox. |-| `memory_mb` | `Any` | Memory in MB for the cloud sandbox. |-| `disk_gb` | `Any` | Disk size in GB for the cloud sandbox. |-| `region` | `Any` | Cloud region (default ``"us-east-1"``). |--**Example:**--```python-sb = await Sandbox.create(Image.desktop("ubuntu"))-await sb.shell.run("uname -a")-print(sb.name) # save to reconnect later-await sb.disconnect()-```--#### Sandbox.connect--```python-def connect(cls, name: str, api_key: Optional[str] = None, local: bool = False, ws_url: Optional[str] = None, http_url: Optional[str] = None, container_name: Optional[str] = None, cpu: Optional[int] = None, memory_mb: Optional[int] = None, disk_gb: Optional[int] = None, region: str = 'us-east-1') -> '_ConnectResult'-```--Connect to an existing sandbox by name.--Supports both ``await`` and ``async with``. When used as a context-manager, ``disconnect()`` is called on exit — the sandbox keeps running.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `name` | `Any` | Name of the existing sandbox. |-| `api_key` | `Any` | Cua API key for cloud sandboxes. |-| `ws_url` | `Any` | WebSocket URL for a remote computer-server. |-| `http_url` | `Any` | HTTP base URL for a remote computer-server. |-| `container_name` | `Any` | Container name for cloud auth (HTTP transport). |-| `region` | `Any` | Cloud region (default ``"us-east-1"``). |--**Example:**--```python-# plain await-sb = await Sandbox.connect("my-sandbox")-await sb.screenshot()-await sb.disconnect()--# context manager — disconnects on exit, sandbox keeps running-async with Sandbox.connect("my-sandbox") as sb:- await sb.screenshot()-```--#### Sandbox.ephemeral--```python-async def ephemeral(cls, image: Image, name: Optional[str] = None, api_key: Optional[str] = None, local: bool = False, runtime: Optional['Runtime'] = None, cpu: Optional[int] = None, memory_mb: Optional[int] = None, disk_gb: Optional[int] = None, region: str = 'us-east-1') -> AsyncIterator['Sandbox']-```--Create an ephemeral sandbox that is automatically destroyed on exit.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `image` | `Any` | Image to run (e.g. ``Image.desktop("ubuntu")``). |-| `name` | `Any` | Optional name to assign to the sandbox. |-| `api_key` | `Any` | Cua API key for cloud sandboxes. |-| `local` | `Any` | Use a local runtime instead of cloud. |-| `runtime` | `Any` | Explicit runtime backend (DockerRuntime, QEMURuntime, etc.). |-| `cpu` | `Any` | Number of CPUs for the cloud sandbox. |-| `memory_mb` | `Any` | Memory in MB for the cloud sandbox. |-| `disk_gb` | `Any` | Disk size in GB for the cloud sandbox. |-| `region` | `Any` | Cloud region (default ``"us-east-1"``). |--**Example:**--```python-async with Sandbox.ephemeral(Image.desktop("ubuntu")) as sb:- await sb.shell.run("whoami")-# sandbox is destroyed here-```--#### Sandbox.list--```python-async def list(cls, local: bool = False, api_key: Optional[str] = None) -> 'list[SandboxInfo]'-```--List running and suspended sandboxes.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `local` | `Any` | If True, list local sandboxes (Lume, Docker, QEMU). If False, list cloud sandboxes. |-| `api_key` | `Any` | Cua API key for cloud sandboxes. |--#### Sandbox.get_info--```python-async def get_info(cls, name: str, local: bool = False, api_key: Optional[str] = None) -> 'SandboxInfo'-```--Get metadata for a specific sandbox.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `name` | `Any` | Sandbox name. |-| `local` | `Any` | If True, look up in local runtimes. |-| `api_key` | `Any` | Cua API key for cloud. |--#### Sandbox.suspend--```python-async def suspend(cls, name: str, local: bool = False, api_key: Optional[str] = None) -> None-```--Suspend a running sandbox (save state).--For local QEMU bare-metal: saves a QMP snapshot then quits the process.-For local Docker/QEMU-docker: pauses the container.-For local Lume: stops the Lume VM (Lume persists state).-For cloud: calls POST /v1/vms/\{name\}/stop.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `name` | `Any` | Sandbox name. |-| `local` | `Any` | If True, operate on a local sandbox. |-| `api_key` | `Any` | Cua API key for cloud. |--#### Sandbox.resume--```python-async def resume(cls, name: str, local: bool = False, api_key: Optional[str] = None) -> 'Sandbox'-```--Resume a suspended sandbox and return a connected Sandbox.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `name` | `Any` | Sandbox name. |-| `local` | `Any` | If True, resume a local sandbox. |-| `api_key` | `Any` | Cua API key for cloud. |--**Returns:** A connected Sandbox ready to use.--#### Sandbox.restart--```python-async def restart(cls, name: str, local: bool = False, api_key: Optional[str] = None) -> 'Sandbox'-```--Restart a sandbox (suspend then resume) and return a connected Sandbox.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `name` | `Any` | Sandbox name. |-| `local` | `Any` | If True, restart a local sandbox. |-| `api_key` | `Any` | Cua API key for cloud. |--**Returns:** A connected Sandbox ready to use.--#### Sandbox.delete--```python-async def delete(cls, name: str, local: bool = False, api_key: Optional[str] = None) -> None-```--Permanently delete a sandbox.--For local sandboxes, stops the VM and removes the state file.-For cloud sandboxes, calls DELETE /v1/vms/\{name\}.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `name` | `Any` | Sandbox name. |-| `local` | `Any` | If True, delete a local sandbox. |-| `api_key` | `Any` | Cua API key for cloud. |--### sandbox--```python-async def sandbox(local: bool = False, ws_url: Optional[str] = None, http_url: Optional[str] = None, api_key: Optional[str] = None, container_name: Optional[str] = None, image: Optional[Image] = None, runtime: Optional['Runtime'] = None, name: Optional[str] = None, ephemeral: Optional[bool] = None, cpu: Optional[int] = None, memory_mb: Optional[int] = None, disk_gb: Optional[int] = None, region: str = 'us-east-1') -> AsyncIterator[Sandbox]-```--Async context manager for a sandboxed environment.--.. deprecated::- Prefer ``Sandbox.create()``, ``Sandbox.connect()``, or- ``Sandbox.ephemeral()`` instead.-------## image--Image builder — pure-data immutable chained builder for sandbox images.--Supports Linux, macOS, and Windows constructors. Serializes to a spec dict-for cloud API or cloud-init consumption.--Usage::-- from cua import Image-- img = (- Image.linux("ubuntu", "24.04")- .apt_install("curl", "git", "build-essential")- .pip_install("numpy", "pandas")- .env(MY_VAR="hello")- .run("echo 'setup complete'")- .expose(8080)- )-- spec = img.to_dict()-------## Image--Immutable, chainable image specification.--Each mutation method returns a new Image instance so that builders-can be forked at any point.--### Constructor--```python-Image(self, os_type: str, distro: str, version: str, kind: Optional[str] = None, _layers: Tuple[Dict[str, Any], ...] = (), _env: Tuple[Tuple[str, str], ...] = (), _ports: Tuple[int, ...] = (), _files: Tuple[Tuple[str, str], ...] = (), _registry: Optional[str] = None, _disk_path: Optional[str] = None, _agent_type: Optional[str] = None) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `os_type` | `str` | |-| `distro` | `str` | |-| `version` | `str` | |-| `kind` | `Optional[str]` | |--### Methods--#### Image.linux--```python-def linux(cls, distro: str = 'ubuntu', version: str = '24.04', kind: str = 'container') -> Image-```--Linux image. Defaults to 'container' (Docker/XFCE). Use kind='vm' for QEMU.--#### Image.macos--```python-def macos(cls, version: str = '26', kind: str = 'vm') -> Image-```--macOS image. Always a VM (Apple Virtualization / Lume).--Supported versions: ``"15"`` / ``"sequoia"``, ``"26"`` / ``"tahoe"``.--#### Image.windows--```python-def windows(cls, version: str = '11', kind: str = 'vm') -> Image-```--Windows image. Always a VM (QEMU or Hyper-V).--#### Image.android--```python-def android(cls, version: str = '14', kind: str = 'vm') -> Image-```--Android image. Always a VM (QEMU emulator).--#### Image.from_registry--```python-def from_registry(cls, ref: str) -> Image-```--Create an image from a registry reference. kind is resolved after pull.--#### Image.from_file--```python-def from_file(cls, path: str, os_type: str = 'windows', kind: str = 'vm', agent_type: Optional[str] = None) -> Image-```--Create an image from a local disk, ISO file, or URL.--Supported formats: qcow2, vhdx, raw, img, iso.-URLs (http/https) are downloaded automatically. Zip files are extracted.-For ISOs, the runtime will create a qcow2 disk and attach the ISO-as a CD-ROM for installation/boot.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `path` | `Any` | Local file path or URL (http/https). |-| `os_type` | `Any` | OS type hint ("linux", "windows", "macos", "android"). |-| `kind` | `Any` | "vm" or "container". |-| `agent_type` | `Any` | Agent type hint (e.g. "osworld" for OSWorld Flask server). |--#### Image.from_dict--```python-def from_dict(cls, data: Dict[str, Any]) -> Image-```--Reconstruct an Image from a serialized spec dict.--#### Image.apt_install--```python-def apt_install(self, packages: str = ()) -> Image-```--Install packages via apt (Linux only).--#### Image.brew_install--```python-def brew_install(self, packages: str = ()) -> Image-```--Install packages via Homebrew (macOS).--#### Image.choco_install--```python-def choco_install(self, packages: str = ()) -> Image-```--Install packages via Chocolatey (Windows).--#### Image.winget_install--```python-def winget_install(self, packages: str = ()) -> Image-```--Install packages via winget (Windows).--#### Image.apk_install--```python-def apk_install(self, apk_paths: str = ()) -> Image-```--Install APK files via adb (Android only).--#### Image.pwa_install--```python-def pwa_install(self, manifest_url: str, package_name: Optional[str] = None, keystore: Optional[str] = None, keystore_alias: str = 'android', keystore_password: str = 'android') -> 'Image'-```--Build an APK from a PWA manifest URL via Bubblewrap and install it (Android only).--Bubblewrap reads the Web App Manifest at *manifest_url*, generates a-Trusted Web Activity (TWA) APK signed with *keystore*, and installs it-via adb. The keystore's SHA-256 fingerprint must match what the server-serves from ``/.well-known/assetlinks.json`` so Chrome trusts the TWA-and skips the browser UI entirely.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `manifest_url` | `Any` | Full URL to the PWA's ``manifest.json``. |--**Example:**--```python-package_name: Android package ID. Defaults to a reversed-hostname- derivation (e.g. ``"com.example.app"``).-keystore: Path to a ``*.keystore`` / ``*.jks`` file. When- omitted a fresh keystore is generated and cached under- ``~/.cua/cua-sandbox/pwa-cache/``. Pass the keystore- bundled with your PWA repo so the fingerprint is- deterministic and pre-loaded into ``assetlinks.json``.-keystore_alias: Key alias inside the keystore (default ``"android"``).-keystore_password: Password for both the store and the key- (default ``"android"``).-```--#### Image.uv_install--```python-def uv_install(self, packages: str = ()) -> Image-```--Install Python packages via uv add into the cua-server project.--#### Image.pip_install--```python-def pip_install(self, packages: str = ()) -> Image-```--Install Python packages via pip.--#### Image.run--```python-def run(self, command: str) -> Image-```--Run a shell command during image build.--#### Image.env--```python-def env(self, variables: str = {}) -> Image-```--Set environment variables.--#### Image.copy--```python-def copy(self, src: str, dst: str) -> Image-```--Copy a file into the image.--#### Image.expose--```python-def expose(self, port: int) -> Image-```--Expose a port.--#### Image.to_dict--```python-def to_dict(self) -> Dict[str, Any]-```--Serialize to a plain dict suitable for JSON or cloud API.--#### Image.to_cloud_init--```python-def to_cloud_init(self) -> str-```--Generate a cloud-init user-data script from the image layers.-------## builder--Image builder — build QEMU VM images from Image layer specs.-------## LayerExecutor--Execute Image layer specs against a running computer-server.--### Constructor--```python-LayerExecutor(self, base_url: str, timeout: float = 600)-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `base_url` | `Any` | |-| `timeout` | `Any` | |--### Methods--#### LayerExecutor.run_command--```python-async def run_command(self, command: str, timeout: float | None = None) -> dict-```--Run a shell command via computer-server and return the result.--#### LayerExecutor.execute_layer--```python-async def execute_layer(self, layer: dict) -> dict-```--Execute a single Image layer and return the result.--#### LayerExecutor.execute_layers--```python-async def execute_layers(self, layers: list[dict]) -> list[dict]-```--Execute all layers sequentially. Raises on first failure.--### create_overlay--```python-def create_overlay(backing: Path, overlay: Path) -> Path-```--Create a qcow2 overlay with the given backing file.-------## interfaces-------## Clipboard--Clipboard read/write.--### Constructor--```python-Clipboard(self, transport: Transport)-```--### Methods--#### Clipboard.get--```python-async def get(self) -> str-```--Return the current clipboard text.--#### Clipboard.set--```python-async def set(self, text: str) -> None-```--Set the clipboard text.-------## Keyboard--Keyboard control.--### Constructor--```python-Keyboard(self, transport: Transport)-```--### Methods--#### Keyboard.type--```python-async def type(self, text: str) -> None-```--Type a string of text.--#### Keyboard.keypress--```python-async def keypress(self, keys: Union[List[str], str]) -> None-```--Press a key combination (e.g. ["ctrl", "c"] or "enter").--#### Keyboard.key_down--```python-async def key_down(self, key: str) -> None-```--#### Keyboard.key_up--```python-async def key_up(self, key: str) -> None-```-------## Mobile--Mobile (Android) touch and hardware-key control.--All touch coordinates are in screen pixels. Single-touch methods use-``input tap/swipe`` via ``adb shell``. Multi-touch gestures delegate to-the ``multitouch_gesture`` transport action, which uses ``adb root`` +-MT Protocol B ``sendevent`` for reliable injection on both local and cloud-transports.--### Constructor--```python-Mobile(self, transport: Transport)-```--### Methods--#### Mobile.tap--```python-async def tap(self, x: int, y: int) -> None-```--#### Mobile.long_press--```python-async def long_press(self, x: int, y: int, duration_ms: int = 1000) -> None-```--#### Mobile.double_tap--```python-async def double_tap(self, x: int, y: int, delay: float = 0.1) -> None-```--#### Mobile.type_text--```python-async def type_text(self, text: str) -> None-```--#### Mobile.swipe--```python-async def swipe(self, x1: int, y1: int, x2: int, y2: int, duration_ms: int = 300) -> None-```--#### Mobile.scroll_up--```python-async def scroll_up(self, x: int, y: int, distance: int = 600, duration_ms: int = 400) -> None-```--#### Mobile.scroll_down--```python-async def scroll_down(self, x: int, y: int, distance: int = 600, duration_ms: int = 400) -> None-```--#### Mobile.scroll_left--```python-async def scroll_left(self, x: int, y: int, distance: int = 400, duration_ms: int = 300) -> None-```--#### Mobile.scroll_right--```python-async def scroll_right(self, x: int, y: int, distance: int = 400, duration_ms: int = 300) -> None-```--#### Mobile.fling--```python-async def fling(self, x1: int, y1: int, x2: int, y2: int) -> None-```--#### Mobile.gesture--```python-async def gesture(self, finger_paths: tuple[int, int] = (), duration_ms: int = 400, steps: int = 0) -> None-```--Inject an arbitrary N-finger gesture via MT Protocol B ``sendevent``.--Each positional argument is a sequence of ``(x, y)`` waypoints for one-finger. Pass an even number of ``(x, y)`` tuples and they are paired-into start/end positions per finger::-- # two-finger pinch-out- await mobile.gesture(- (cx - 20, cy), (cx - 200, cy), # finger 0- (cx + 20, cy), (cx + 200, cy), # finger 1- )--Delegates to the ``multitouch_gesture`` transport action, which uses-``adb root`` + MT Protocol B ``sendevent`` for reliable injection on-both local ADB and cloud HTTP transports.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `duration_ms` | `Any` | Total gesture duration in milliseconds. |-| `steps` | `Any` | Interpolation steps (0 = auto: duration_ms // 20, min 5). |--#### Mobile.pinch_in--```python-async def pinch_in(self, cx: int, cy: int, spread: int = 300, duration_ms: int = 400) -> None-```--Pinch-in (zoom out) with two real simultaneous fingers.--#### Mobile.pinch_out--```python-async def pinch_out(self, cx: int, cy: int, spread: int = 300, duration_ms: int = 400) -> None-```--Pinch-out (zoom in) with two real simultaneous fingers.--#### Mobile.key--```python-async def key(self, keycode: int) -> None-```--#### Mobile.home--```python-async def home(self) -> None-```--#### Mobile.back--```python-async def back(self) -> None-```--#### Mobile.recents--```python-async def recents(self) -> None-```--#### Mobile.power--```python-async def power(self) -> None-```--#### Mobile.volume_up--```python-async def volume_up(self) -> None-```--#### Mobile.volume_down--```python-async def volume_down(self) -> None-```--#### Mobile.enter--```python-async def enter(self) -> None-```--#### Mobile.backspace--```python-async def backspace(self) -> None-```--#### Mobile.wake--```python-async def wake(self) -> None-```--#### Mobile.notifications--```python-async def notifications(self) -> None-```--#### Mobile.close_notifications--```python-async def close_notifications(self) -> None-```-------## Mouse--Mouse control.--### Constructor--```python-Mouse(self, transport: Transport)-```--### Methods--#### Mouse.click--```python-async def click(self, x: int, y: int, button: str = 'left') -> None-```--#### Mouse.right_click--```python-async def right_click(self, x: int, y: int) -> None-```--#### Mouse.double_click--```python-async def double_click(self, x: int, y: int) -> None-```--#### Mouse.move--```python-async def move(self, x: int, y: int) -> None-```--#### Mouse.scroll--```python-async def scroll(self, x: int, y: int, scroll_x: int = 0, scroll_y: int = 3) -> None-```--#### Mouse.mouse_down--```python-async def mouse_down(self, x: int, y: int, button: str = 'left') -> None-```--#### Mouse.mouse_up--```python-async def mouse_up(self, x: int, y: int, button: str = 'left') -> None-```--#### Mouse.drag--```python-async def drag(self, start_x: int, start_y: int, end_x: int, end_y: int, button: str = 'left') -> None-```-------## Screen--Screen capture and info.--### Constructor--```python-Screen(self, transport: Transport)-```--### Methods--#### Screen.screenshot--```python-async def screenshot(self, format: str = 'png', quality: int = 95) -> bytes-```--Capture a screenshot and return raw image bytes.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `format` | `Any` | "png" (lossless, default) or "jpeg" (lossy, ~5-10x smaller). |-| `quality` | `Any` | JPEG quality 1-95, ignored for PNG. |--#### Screen.screenshot_base64--```python-async def screenshot_base64(self, format: str = 'png', quality: int = 95) -> str-```--Capture a screenshot and return as a base64-encoded string.--#### Screen.size--```python-async def size(self) -> Tuple[int, int]-```--Return (width, height) of the screen.-------## Shell--Shell command execution.--### Constructor--```python-Shell(self, transport: Transport)-```--### Methods--#### Shell.run--```python-async def run(self, command: str, timeout: int = 30) -> CommandResult-```--Run a shell command and return the result.-------## Terminal--PTY terminal sessions.--### Constructor--```python-Terminal(self, transport: Transport)-```--### Methods--#### Terminal.create--```python-async def create(self, shell: str = 'bash', cols: int = 80, rows: int = 24) -> dict-```--Create a new PTY session. Returns \{"pid": int, "cols": int, "rows": int\}.--#### Terminal.send_input--```python-async def send_input(self, pid: int, data: str) -> None-```--Send input to a PTY session.--#### Terminal.resize--```python-async def resize(self, pid: int, cols: int, rows: int) -> None-```--Resize a PTY session.--#### Terminal.close--```python-async def close(self, pid: int) -> Optional[int]-```--Close a PTY session. Returns exit code.-------## Tunnel--Port-forwarding interface — exposes sandbox ports on the host.--### Constructor--```python-Tunnel(self, transport: Transport)-```--### Methods--#### Tunnel.forward--```python-def forward(self, ports: int | str = ()) -> _TunnelContext-```--Forward one or more sandbox ports (or abstract sockets) to the host.--*ports* may be:-- ``int`` — TCP port inside the sandbox (e.g. ``8080``)-- ``str`` — Android abstract socket name (e.g. ``"chrome_devtools_remote"``)--Returns a context manager (or awaitable) that yields:-- a single :class:`TunnelInfo` when one target is given-- a ``dict[sandbox_port, TunnelInfo]`` when multiple targets are given--Example — Chrome DevTools over ADB::-- async with sb.tunnel.forward("chrome_devtools_remote") as t:- # http://localhost:<random>/json lists CDP targets- print(t.url)-------## TunnelInfo--A single forwarded port.--### Constructor--```python-TunnelInfo(self, host: str, port: int, sandbox_port: int) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `host` | `Any` | |-| `port` | `Any` | |-| `sandbox_port` | `Any` | |-| `url` | `str` | |--### Methods--#### TunnelInfo.close--```python-async def close(self) -> None-```--Close this tunnel (no-op if already closed or inside a context manager).-------## Window--Window management.--### Constructor--```python-Window(self, transport: Transport)-```--### Methods--#### Window.get_active_title--```python-async def get_active_title(self) -> str-```--Return the title of the currently focused window.diff --git a/docs/content/docs/cua/reference/sandbox-sdk/interfaces.mdx b/docs/content/docs/cua/reference/sandbox-sdk/interfaces.mdxdeleted file mode 100644index 68834504c5..0000000000--- a/docs/content/docs/cua/reference/sandbox-sdk/interfaces.mdx+++ /dev/null@@ -1,330 +0,0 @@-----title: Interfaces Reference-description: Auto-generated reference for all cua-sandbox interface classes. Do not edit manually � run scripts/gen_interface_docs.py to regenerate.------import { Callout } from "fumadocs-ui/components/callout";--<Callout type="info">- This page is auto-generated from the Python source docstrings.- Run `python scripts/gen_interface_docs.py` to regenerate after editing interface code.-</Callout>--The sandbox exposes the following interface objects on every `Sandbox` instance:--| Attribute | Class | Purpose |-|-----------|-------|---------|-| `sb.shell` | `Shell` | Run shell commands |-| `sb.mouse` | `Mouse` | Mouse control |-| `sb.keyboard` | `Keyboard` | Keyboard control |-| `sb.screen` | `Screen` | Screenshots and screen info |-| `sb.clipboard` | `Clipboard` | Clipboard read/write |-| `sb.tunnel` | `Tunnel` | Port forwarding |-| `sb.terminal` | `Terminal` | PTY terminal sessions |-| `sb.window` | `Window` | Window management |-| `sb.mobile` | `Mobile` | Mobile-specific actions |--------## `Shell`--Shell command execution.--### Methods--#### `async run(command: str, timeout: int = 30) -> CommandResult`--Run a shell command and return the result.---------## `CommandResult`--### Fields--| Name | Type |-|------|------|-| `stdout` | `str` |-| `stderr` | `str` |-| `returncode` | `int` |--### Methods--#### `@property-success() -> bool`---------## `Mouse`--Mouse control.--### Methods--#### `async click(x: int, y: int, button: str = 'left') -> None`--#### `async right_click(x: int, y: int) -> None`--#### `async double_click(x: int, y: int) -> None`--#### `async move(x: int, y: int) -> None`--#### `async scroll(x: int, y: int, scroll_x: int = 0, scroll_y: int = 3) -> None`--#### `async mouse_down(x: int, y: int, button: str = 'left') -> None`--#### `async mouse_up(x: int, y: int, button: str = 'left') -> None`--#### `async drag(start_x: int, start_y: int, end_x: int, end_y: int, button: str = 'left') -> None`---------## `Keyboard`--Keyboard control.--### Methods--#### `async type(text: str) -> None`--Type a string of text.--#### `async keypress(keys: Union[List[str], str]) -> None`--Press a key combination (e.g. ["ctrl", "c"] or "enter").--#### `async key_down(key: str) -> None`--#### `async key_up(key: str) -> None`---------## `Screen`--Screen capture and info.--### Methods--#### `async screenshot(format: str = 'png', quality: int = 95) -> bytes`--Capture a screenshot and return raw image bytes.--Args:- format: "png" (lossless, default) or "jpeg" (lossy, ~5-10x smaller).- quality: JPEG quality 1-95, ignored for PNG.--#### `async screenshot_base64(format: str = 'png', quality: int = 95) -> str`--Capture a screenshot and return as a base64-encoded string.--#### `async size() -> Tuple[int, int]`--Return (width, height) of the screen.---------## `Clipboard`--Clipboard read/write.--### Methods--#### `async get() -> str`--Return the current clipboard text.--#### `async set(text: str) -> None`--Set the clipboard text.---------## `Tunnel`--Port-forwarding interface — exposes sandbox ports on the host.--### Methods--#### `forward(*ports) -> _TunnelContext`--Forward one or more sandbox ports (or abstract sockets) to the host.--*ports* may be:-- ``int`` — TCP port inside the sandbox (e.g. ``8080``)-- ``str`` — Android abstract socket name (e.g. ``"chrome_devtools_remote"``)--Returns a context manager (or awaitable) that yields:-- a single :class:`TunnelInfo` when one target is given-- a ``dict[sandbox_port, TunnelInfo]`` when multiple targets are given--Example — Chrome DevTools over ADB::-- async with sb.tunnel.forward("chrome_devtools_remote") as t:- # http://localhost:<random>/json lists CDP targets- print(t.url)---------## `TunnelInfo`--A single forwarded port.--### Methods--#### `@property-url() -> str`--#### `async close() -> None`--Close this tunnel (no-op if already closed or inside a context manager).---------## `Terminal`--PTY terminal sessions.--### Methods--#### `async create(shell: str = 'bash', cols: int = 80, rows: int = 24) -> dict`--Create a new PTY session. Returns {"pid": int, "cols": int, "rows": int}.--#### `async send_input(pid: int, data: str) -> None`--Send input to a PTY session.--#### `async resize(pid: int, cols: int, rows: int) -> None`--Resize a PTY session.--#### `async close(pid: int) -> Optional[int]`--Close a PTY session. Returns exit code.---------## `Window`--Window management.--### Methods--#### `async get_active_title() -> str`--Return the title of the currently focused window.---------## `Mobile`--Mobile (Android) touch and hardware-key control.--All touch coordinates are in screen pixels. Single-touch methods use-``input tap/swipe`` via ``adb shell``. Multi-touch gestures delegate to-the ``multitouch_gesture`` transport action, which uses ``adb root`` +-MT Protocol B ``sendevent`` for reliable injection on both local and cloud-transports.--### Methods--#### `async tap(x: int, y: int) -> None`--#### `async long_press(x: int, y: int, duration_ms: int = 1000) -> None`--#### `async double_tap(x: int, y: int, delay: float = 0.1) -> None`--#### `async type_text(text: str) -> None`--#### `async swipe(x1: int, y1: int, x2: int, y2: int, duration_ms: int = 300) -> None`--#### `async scroll_up(x: int, y: int, distance: int = 600, duration_ms: int = 400) -> None`--#### `async scroll_down(x: int, y: int, distance: int = 600, duration_ms: int = 400) -> None`--#### `async scroll_left(x: int, y: int, distance: int = 400, duration_ms: int = 300) -> None`--#### `async scroll_right(x: int, y: int, distance: int = 400, duration_ms: int = 300) -> None`--#### `async fling(x1: int, y1: int, x2: int, y2: int) -> None`--#### `async gesture(duration_ms: int = 400, steps: int = 0, *finger_paths) -> None`--Inject an arbitrary N-finger gesture via MT Protocol B ``sendevent``.--Each positional argument is a sequence of ``(x, y)`` waypoints for one-finger. Pass an even number of ``(x, y)`` tuples and they are paired-into start/end positions per finger::-- # two-finger pinch-out- await mobile.gesture(- (cx - 20, cy), (cx - 200, cy), # finger 0- (cx + 20, cy), (cx + 200, cy), # finger 1- )--Delegates to the ``multitouch_gesture`` transport action, which uses-``adb root`` + MT Protocol B ``sendevent`` for reliable injection on-both local ADB and cloud HTTP transports.--Args:- *finger_paths: Alternating start/end ``(x, y)`` tuples, two per finger.- Must have an even length >= 4 (at least 2 fingers × 2 points).- duration_ms: Total gesture duration in milliseconds.- steps: Interpolation steps (0 = auto: duration_ms // 20, min 5).--#### `async pinch_in(cx: int, cy: int, spread: int = 300, duration_ms: int = 400) -> None`--Pinch-in (zoom out) with two real simultaneous fingers.--#### `async pinch_out(cx: int, cy: int, spread: int = 300, duration_ms: int = 400) -> None`--Pinch-out (zoom in) with two real simultaneous fingers.--#### `async key(keycode: int) -> None`--#### `async home() -> None`--#### `async back() -> None`--#### `async recents() -> None`--#### `async power() -> None`--#### `async volume_up() -> None`--#### `async volume_down() -> None`--#### `async enter() -> None`--#### `async backspace() -> None`--#### `async wake() -> None`--#### `async notifications() -> None`--#### `async close_notifications() -> None`-------diff --git a/docs/content/docs/cua/reference/sandbox-sdk/meta.json b/docs/content/docs/cua/reference/sandbox-sdk/meta.jsondeleted file mode 100644index 2c02f8ea4d..0000000000--- a/docs/content/docs/cua/reference/sandbox-sdk/meta.json+++ /dev/null@@ -1,5 +0,0 @@-{- "title": "Sandbox SDK",- "icon": "Box",- "pages": ["index", "interfaces"]-}diff --git a/docs/content/docs/cua/vibe-coding-mcp.mdx b/docs/content/docs/cua/vibe-coding-mcp.mdxdeleted file mode 100644index e4e06a74d9..0000000000--- a/docs/content/docs/cua/vibe-coding-mcp.mdx+++ /dev/null@@ -1,154 +0,0 @@-----title: Vibe Coding MCP-description: Connect your AI agents to Cua documentation and source code------The Cua MCP server equips AI coding agents with deep knowledge about Cua's APIs, patterns, source code, and best practices. When connected to your development environment, it enables AI assistants to:--- **Search documentation** with full-text and semantic search-- **Explore source code** across all versions and components-- **Generate accurate code** with context from the actual codebase-- **Understand Cua internals** by querying versioned implementations--## Server URL--| Endpoint | URL |-|----------|-----|-| Streamable HTTP Transport | `https://vk-mcp.cua.ai/mcp` |--## Available Tools--The MCP server exposes four read-only query tools:--### Documentation Tools--| Tool | Description |-|------|-------------|-| `query_docs_db` | SQL queries against documentation (FTS5 full-text search) |-| `query_docs_vectors` | Semantic vector search for natural language queries |--### Code Search Tools--| Tool | Description |-|------|-------------|-| `query_code_db` | SQL queries against versioned source code (FTS5 full-text search) |-| `query_code_vectors` | Semantic code search across all components and versions |--## Connect Your AI Tools--### Cursor--1. Press `Shift+Command+J` (Mac) or `Shift+Ctrl+J` (Windows/Linux) to open settings-2. Find "MCP Tools" in the left sidebar-3. Click "Add Custom MCP" to open `mcp.json`-4. Add the Cua MCP server configuration:--```json-{- "mcpServers": {- "Cua MCP": {- "url": "https://vk-mcp.cua.ai/mcp"- }- }-}-```--### Claude Code--Add the MCP server using the CLI:--```bash-claude mcp add --transport http cua-mcp https://vk-mcp.cua.ai/mcp-```--Verify the connection:--```bash-claude mcp list-```--Expected output:--```-Checking MCP server health...-cua-mcp: https://vk-mcp.cua.ai/mcp (HTTP) - ✓ Connected-```--Management commands:--```bash-# View server details-claude mcp get cua-mcp--# Remove server if needed-claude mcp remove cua-mcp -s local-```--### Claude Web / Claude Desktop--1. Go to **Settings** > **Connectors**-2. Click **Add custom connector**-3. Enter the following:- - **Name**: `Cua MCP`- - **URL**: `https://vk-mcp.cua.ai/mcp`-4. Click **Add**--### Windsurf--1. Open Windsurf Settings (bottom right settings button)-2. Navigate to **Cascade** > **Model Context Protocol**-3. Click **Add Server** > **Add custom server**-4. Select **Streamable HTTP** transport and enter: `https://vk-mcp.cua.ai/mcp`--Configuration file location:-- **macOS**: `~/.codeium/windsurf/mcp_config.json`-- **Windows**: `%USERPROFILE%\.codeium\windsurf\mcp_config.json`-- **Linux**: `~/.codeium/windsurf/mcp_config.json`--### Cline (VS Code)--1. Open the Cline extension panel in VS Code-2. Click the menu (⋮) in the top right-3. Select **MCP Servers**-4. Click **Remote Servers** tab-5. Enter:- - **Server Name**: `Cua MCP`- - **Server URL**: `https://vk-mcp.cua.ai/mcp`-6. Click **Add Server**--### GitHub Copilot--1. Enable MCP in VS Code: Settings > search "MCP" > enable `chat.mcp.enabled`-2. Create `.vscode/mcp.json` in your project:--```json-{- "servers": {- "Cua MCP": {- "url": "https://vk-mcp.cua.ai/mcp"- }- }-}-```--3. Open Copilot Chat, switch to **Agent mode**, and the tools will be available--### Other MCP-Compatible Tools--For any MCP-compatible application:--**Streamable HTTP Connection**:-```-https://vk-mcp.cua.ai/mcp-```--**Common JSON Configuration**:-```json-{- "servers": {- "Cua MCP": {- "url": "https://vk-mcp.cua.ai/mcp"- }- }-}-```\ No newline at end of filediff --git a/docs/content/docs/cuabench/examples/custom-agent.mdx b/docs/content/docs/cuabench/examples/custom-agent.mdxdeleted file mode 100644index 87d4342739..0000000000--- a/docs/content/docs/cuabench/examples/custom-agent.mdx+++ /dev/null@@ -1,138 +0,0 @@-----title: Build a Custom Agent-description: Add your own agent to cua-bench------# Build a Custom Agent--Add a custom agent to cua-bench by extending `BaseAgent` and registering it.--## 1. Create your agent file--Create `cua_bench/agents/my_agent.py`:--```python-import base64-from pathlib import Path-from typing import TYPE_CHECKING--from . import register_agent-from .base import AgentResult, BaseAgent, FailureMode--if TYPE_CHECKING:- from ..computers import DesktopSession---@register_agent("my-agent")-class MyAgent(BaseAgent):- def __init__(self, **kwargs):- super().__init__(**kwargs)- self.model = kwargs.get("model", "claude-sonnet-4-20250514")- self.max_steps = int(kwargs.get("max_steps", 50))-- @staticmethod- def name() -> str:- return "my-agent"-- async def perform_task(- self,- task_description: str,- session: "DesktopSession",- logging_dir: Path | None = None,- tracer=None,- ) -> AgentResult:- instruction = self._render_instruction(task_description)-- total_input = 0- total_output = 0-- for step in range(self.max_steps):- screenshot_bytes = await session.screenshot()-- if logging_dir:- (logging_dir / f"step_{step}.png").write_bytes(screenshot_bytes)-- # Your logic: send screenshot to an LLM, parse action, etc.- # action = await self._decide(instruction, screenshot_bytes)-- # if action is None:- # break- # await session.execute_action(action)-- return AgentResult(- total_input_tokens=total_input,- total_output_tokens=total_output,- failure_mode=FailureMode.NONE,- )-```--## 2. Register the import--Add one line to `cua_bench/agents/__init__.py`:--```python-from .my_agent import MyAgent # noqa: E402-```--This triggers `@register_agent("my-agent")` at import time.--## 3. Run it--```bash-cb run task tasks/my-task --agent my-agent-cb run task tasks/my-task --agent my-agent --agent-kwarg model=gpt-4o --agent-kwarg max_steps=100-```--## 4. Develop without rebuilding the Docker image--Use `--with` to mount your local cua-bench (or any local package) into the agent container:--```bash-cb run task tasks/my-task --agent my-agent --with ../libs/cua-bench-```--This mounts the path into the container and runs `pip install` on it before the agent starts. You can repeat `--with` for multiple packages.--## Reference--### BaseAgent interface--| Method / Property | Description |-| --- | --- |-| `name()` (static, abstract) | Return the agent name string |-| `perform_task(task_description, session, logging_dir, tracer)` (abstract) | Run the agent loop, return `AgentResult` |-| `version` | Optional version string (set via `kwargs`) |-| `_render_instruction(text)` | Render through a Jinja2 `prompt_template` if one was provided |--### AgentResult--```python-@dataclass-class AgentResult:- total_input_tokens: int = 0- total_output_tokens: int = 0- failure_mode: FailureMode = FailureMode.UNSET # NONE, UNKNOWN, MAX_STEPS_EXCEEDED-```--### Available actions--All importable from `cua_bench` (e.g. `from cua_bench import ClickAction`):--| Action | Fields |-| --- | --- |-| `ClickAction` | `x, y` |-| `RightClickAction` | `x, y` |-| `DoubleClickAction` | `x, y` |-| `MiddleClickAction` | `x, y` |-| `DragAction` | `from_x, from_y, to_x, to_y, duration=1.0` |-| `MoveToAction` | `x, y, duration=0.0` |-| `ScrollAction` | `direction="up", amount=100` |-| `TypeAction` | `text` |-| `KeyAction` | `key` |-| `HotkeyAction` | `keys` (list) |-| `WaitAction` | `seconds=1.0` |-| `DoneAction` | (none) |--### kwargs--All `--agent-kwarg` values from the CLI arrive as strings in `**kwargs`. Cast as needed (e.g. `int(kwargs.get("max_steps", 50))`). Built-in kwargs handled by `BaseAgent`: `version`, `prompt_template`.diff --git a/docs/content/docs/cuabench/examples/genai-task-generation.mdx b/docs/content/docs/cuabench/examples/genai-task-generation.mdxdeleted file mode 100644index 4d4c73c524..0000000000--- a/docs/content/docs/cuabench/examples/genai-task-generation.mdx+++ /dev/null@@ -1,391 +0,0 @@-----title: Generate Tasks with AI-description: Use Claude to create tasks from natural language descriptions------In this tutorial, you'll use AI to generate complete task environments from natural language descriptions. This is the fastest way to create new tasks.--**Time:** ~10 minutes-**Prerequisites:** Claude Code OAuth token--## What You'll Build--An AI-generated color picker task where the agent must select a specific color. You'll learn:--- Using `cb task generate` to create tasks-- Reviewing and understanding generated code-- Iterating on AI-generated tasks-- Manual refinement techniques--## How It Works--The `cb task generate` command:--1. Creates a starter task template-2. Launches Claude Code in interactive mode-3. Claude modifies the template based on your description-4. You review, test, and refine the result--```-┌──────────────────────────────────────────────────────────────┐-│ cb task generate "Create a color picker..." │-│ │ │-│ ▼ │-│ ┌────────────────────────────────────────────────────────┐ │-│ │ Starter Template │ │-│ │ ├── main.py (decorators) │ │-│ │ ├── gui/index.html │ │-│ │ └── pyproject.toml │ │-│ └────────────────────────────────────────────────────────┘ │-│ │ │-│ ▼ │-│ ┌────────────────────────────────────────────────────────┐ │-│ │ Claude Code │ │-│ │ Modifies template based on your prompt │ │-│ └────────────────────────────────────────────────────────┘ │-│ │ │-│ ▼ │-│ ┌────────────────────────────────────────────────────────┐ │-│ │ Generated Task │ │-│ │ ├── main.py (customized) │ │-│ │ ├── gui/index.html (customized) │ │-│ │ └── pyproject.toml (metadata) │ │-│ └────────────────────────────────────────────────────────┘ │-└──────────────────────────────────────────────────────────────┘-```--## Step 1: Set Up Authentication--Get your Claude Code OAuth token and set it as an environment variable:--```bash-export CLAUDE_CODE_OAUTH_TOKEN=your-token-here-```--<Callout title="Getting a Token" type="info">- Contact your administrator or visit the Claude Code documentation to obtain an OAuth token.-</Callout>--## Step 2: Generate a Task--Run the generate command with a description:--```bash-cb task generate "Create a color picker task where the user must select the color red from a palette of 6 colors"-```--Claude Code will start an interactive session. You'll see it:--1. Read the starter template-2. Modify `main.py` with task logic-3. Create `gui/index.html` with the color picker UI-4. Update `pyproject.toml` with metadata--## Step 3: Review Generated Files--After generation, examine the created files:--```bash-ls tasks/color-picker-env/-# main.py gui/ pyproject.toml-```--### Generated main.py--The generated `main.py` should look something like:--```python-import cua_bench as cb-from pathlib import Path--pid = None-[email protected]_config(split="train")-def load():- return [- cb.Task(- description="Select the red color from the color palette.",- metadata={"target_color": "red"},- computer={- "provider": "simulated",- "setup_config": {- "os_type": "macos",- "width": 512,- "height": 512,- }- }- )- ]--[email protected]_task(split="train")-async def start(task_cfg: cb.Task, session: cb.DesktopSession):- global pid- html_content = (Path(__file__).parent / "gui/index.html").read_text()- pid = await session.launch_window(- html=html_content,- title="Color Picker",- width=400,- height=300,- )--[email protected]_task(split="train")-async def evaluate(task_cfg: cb.Task, session: cb.DesktopSession) -> list[float]:- global pid- if pid is None:- return [0.0]-- selected = await session.execute_javascript(pid, "window.__selectedColor")- target = task_cfg.metadata.get("target_color", "red")-- return [1.0] if selected == target else [0.0]--[email protected]_task(split="train")-async def solve(task_cfg: cb.Task, session: cb.DesktopSession):- global pid- if pid is None:- return-- await session.click_element(pid, '[data-color="red"]')-```--### Generated gui/index.html--```html-<!DOCTYPE html>-<html>- <head>- <style>- body {- font-family: system-ui;- display: flex;- flex-direction: column;- align-items: center;- padding: 20px;- }-- h2 {- margin-bottom: 20px;- }-- .palette {- display: grid;- grid-template-columns: repeat(3, 1fr);- gap: 10px;- }-- .color-box {- width: 80px;- height: 80px;- border: 3px solid #ccc;- border-radius: 8px;- cursor: pointer;- }-- .color-box:hover {- transform: scale(1.05);- }- .color-box.selected {- border-color: #000;- }- </style>- </head>- <body>- <h2>Select the red color</h2>- <div class="palette">- <div class="color-box" data-color="red" style="background: #e74c3c;"></div>- <div class="color-box" data-color="blue" style="background: #3498db;"></div>- <div class="color-box" data-color="green" style="background: #2ecc71;"></div>- <div class="color-box" data-color="yellow" style="background: #f1c40f;"></div>- <div class="color-box" data-color="purple" style="background: #9b59b6;"></div>- <div class="color-box" data-color="orange" style="background: #e67e22;"></div>- </div>-- <script>- window.__selectedColor = null;-- document.querySelectorAll('.color-box').forEach((box) => {- box.addEventListener('click', function () {- document.querySelectorAll('.color-box').forEach((b) => b.classList.remove('selected'));- this.classList.add('selected');- window.__selectedColor = this.dataset.color;- });- });- </script>- </body>-</html>-```--## Step 4: Test the Generated Task--### Run Interactively--```bash-cb interact tasks/color-picker-env --variant-id 0-```--Try clicking different colors and see how the UI responds.--### Run with Oracle Solution--```bash-cb interact tasks/color-picker-env --oracle --no-wait-```--> **Note**: For headless servers, use `xvfb-run`:->-> ```bash-> xvfb-run -a cb interact tasks/color-picker-env --oracle --no-wait-> ```--### Run with Agent--```bash-cb interact tasks/color-picker-env --variant-id 0-# Agent will be prompted interactively-```--## Step 5: Iterate on the Task--If the generated task isn't quite right, you can:--### Option A: Regenerate with Better Prompt--```bash-cb task generate "Create a color picker with 9 colors in a 3x3 grid. The target color should be randomly selected from the palette. Show the target color name prominently above the grid."-```--### Option B: Continue in Claude Code--Run the generate command again and ask Claude to modify:--```bash-cb task generate "Modify the existing color picker to add a timer that gives bonus points for fast selections"-```--### Option C: Manual Refinement--Edit the generated files directly:--```python-# Add more task variants[email protected]_config(split="train")-def load():- colors = ["red", "blue", "green", "yellow", "purple", "orange"]-- return [- cb.Task(- description=f"Select the {color} color from the palette.",- metadata={"target_color": color},- computer={...}- )- for color in colors- ]-```--## Step 6: Generate More Complex Tasks--### Multi-Step Task--```bash-cb task generate "Create a todo list app where the user must:-1. Add a task called 'Buy groceries'-2. Mark it as complete-3. Delete the task-Track which steps are completed for partial rewards."-```--### Form-Based Task--```bash-cb task generate "Create a registration form with fields for name, email, and password.-The agent must fill in the form with valid data and submit it.-Validate that email contains @ and password is at least 8 characters."-```--### Game Task--```bash-cb task generate "Create a simple memory card matching game with 8 cards (4 pairs).-The agent must match all pairs. Give partial reward based on number of pairs found."-```--## Tips for Good Prompts--### Be Specific--```bash-# Good-cb task generate "Create a dropdown menu with 5 options (Apple, Banana, Cherry, Date, Elderberry). The agent must select 'Cherry'."--# Too vague-cb task generate "Create a dropdown task"-```--### Include Success Criteria--```bash-# Good-cb task generate "Create a slider that goes from 0 to 100. The agent must set it to exactly 75. Give reward of 1.0 for exact match, 0.5 for within 5 points."--# Missing criteria-cb task generate "Create a slider task"-```--### Specify Visual Requirements--```bash-# Good-cb task generate "Create a dark-themed calculator with large buttons (at least 50x50 pixels) and clear labels."--# No visual guidance-cb task generate "Create a calculator"-```--## What Claude Can and Can't Do--### Claude CAN:--- Create HTML/CSS/JavaScript UIs-- Implement game logic-- Write evaluation functions-- Create multiple task variants-- Add partial reward calculations--### Claude CANNOT:--- Access external APIs at generation time-- Create native application tasks (use templates for those)-- Generate images (use placeholder colors/text)-- Test the task (you need to do this)--## Complete Workflow Example--```bash-# 1. Generate initial task-cb task generate "Create a simple quiz with 3 multiple choice questions about Python"--# 2. Test interactively-cb interact tasks/python-quiz-env --variant-id 0--# 3. Verify oracle works-cb interact tasks/python-quiz-env --oracle --no-wait-# On headless servers: xvfb-run -a cb interact tasks/python-quiz-env --oracle --no-wait--# 4. Test with agent-cb interact tasks/python-quiz-env --variant-id 0--# 5. Manually refine if needed-# Edit tasks/python-quiz-env/main.py--# 6. Verify changes still work-cb interact tasks/python-quiz-env --oracle --no-wait-```--## Next Steps--- Learn about [Task Structure](/cuabench/guide/fundamentals/tasks) in depth-- Explore [Simulated Desktop](/cuabench/guide/fundamentals/simulated-desktop) for visual variants-- See [Creating Private Tasks](/cuabench/guide/advanced/creating-private-tasks) for advanced patternsdiff --git a/docs/content/docs/cuabench/examples/macos-native-task.mdx b/docs/content/docs/cuabench/examples/macos-native-task.mdxdeleted file mode 100644index fbd55bb261..0000000000--- a/docs/content/docs/cuabench/examples/macos-native-task.mdx+++ /dev/null@@ -1,78 +0,0 @@-----title: macOS Native Task-description: Build a Reminders task using osascript-based app helpers on macOS------Create a task where agents must add a reminder using the Reminders app. This shows how to use macOS app helpers that leverage `osascript` for evaluation.--## Prerequisites--macOS native provider configured (runs on actual Mac hardware).--## Create the Task--Create `tasks/add_reminder/main.py`:--```python-import cua_bench as cb--[email protected]_config(split="train")-def load():- return [- cb.Task(- description="Open Reminders and create a new reminder called 'Buy groceries' in the default list.",- metadata={"reminder_name": "Buy groceries"},- computer={- "provider": "native",- "setup_config": {"os_type": "macos", "width": 1440, "height": 900},- },- )- ]--[email protected]_task(split="train")-async def start(task_cfg: cb.Task, session: cb.DesktopSession):- """Launch Reminders app."""- await session.apps.reminders.launch()--[email protected]_task(split="train")-async def evaluate(task_cfg: cb.Task, session: cb.DesktopSession) -> list[float]:- """Check if the reminder was created using osascript getter."""- target_name = task_cfg.metadata["reminder_name"]-- # App helper uses osascript internally to query Reminders- reminders = await session.apps.reminders.get_incomplete_reminders()-- for reminder in reminders:- if reminder["name"] == target_name:- return [1.0]- return [0.0]---if __name__ == "__main__":- cb.interact(__file__)-```--## How It Works--The macOS app helpers use AppleScript via `osascript` to interact with first-party apps:--```python-# These methods run osascript commands under the hood-notes = await session.apps.notes.get_all_notes()-events = await session.apps.calendar.get_events_today()-reminders = await session.apps.reminders.get_incomplete_reminders()-```--## Run It--```bash-cb interact tasks/add_reminder-```--## Next Steps--- [Windows Native Task](/cuabench/examples/windows-native-task) - File system-based evaluation-- [App Helpers](/cuabench/guide/fundamentals/app-helpers) - Full API referencediff --git a/docs/content/docs/cuabench/examples/meta.json b/docs/content/docs/cuabench/examples/meta.jsondeleted file mode 100644index 490c3fa347..0000000000--- a/docs/content/docs/cuabench/examples/meta.json+++ /dev/null@@ -1,13 +0,0 @@-{- "title": "Examples",- "description": "Step-by-step tutorials for common workflows",- "icon": "Blocks",- "pages": [- "universal-task",- "windows-native-task",- "macos-native-task",- "genai-task-generation",- "custom-agent",- "rl-training"- ]-}diff --git a/docs/content/docs/cuabench/examples/rl-training.mdx b/docs/content/docs/cuabench/examples/rl-training.mdxdeleted file mode 100644index a2469c07f2..0000000000--- a/docs/content/docs/cuabench/examples/rl-training.mdx+++ /dev/null@@ -1,756 +0,0 @@-----title: RL Training with GRPO-description: Train a GUI agent using TRL's GRPOTrainer with CUA-Bench environments on Modal------Train a multimodal GUI agent using GRPO (Group Relative Policy Optimization) with TRL and CUA-Bench on Modal.--## How It Works--The training follows TRL's OpenEnv pattern:--```-┌─────────────────────────────────────────────────────────────┐-│ GRPO Training Loop │-├─────────────────────────────────────────────────────────────┤-│ 1. rollout_func() called with prompts │-│ ├─ Reset CUA-Bench environment │-│ ├─ Generate actions via vLLM │-│ ├─ Step environment, collect rewards │-│ └─ Return {prompt_ids, completion_ids, logprobs, ...} │-│ │-│ 2. Reward function receives env_reward from rollout │-│ │-│ 3. GRPO updates policy using rewards │-└─────────────────────────────────────────────────────────────┘-```--## Step 1: Install Modal--```bash-pip install modal-modal setup-```--Optionally, create a wandb secret for logging:--```bash-modal secret create wandb-secret WANDB_API_KEY=your_key-```--## Step 2: Create the Training Script--Create `modal_grpo_training.py`:--```python-#!/usr/bin/env python3-"""GRPO Training with CUA-Bench on Modal.--This script runs GRPO (Group Relative Policy Optimization) training using-TRL's GRPOTrainer with CUA-Bench environments on Modal's cloud infrastructure.--Usage:- # Setup Modal- pip install modal- modal setup-- # Run training- modal run modal_grpo_training.py-- # With custom settings- modal run modal_grpo_training.py --num-workers 4 --max-steps 1000-"""--from __future__ import annotations--from dataclasses import dataclass--import modal---# =============================================================================-# Training Configuration-# =============================================================================--@dataclass-class TrainerConfig:- """Configuration for GRPO training hyperparameters."""-- # Model- model_id: str = "Qwen/Qwen3-VL-2B-Instruct"- task_prompt: str = "Complete the computer task successfully."-- # Environment- num_workers: int = 2- max_steps: int = 10- max_history: int = 2-- # Generation- num_generations: int = 4- max_completion_length: int = 256- temperature: float = 0.7-- # Training- learning_rate: float = 5e-6- gradient_accumulation_steps: int = 4- per_device_train_batch_size: int = 1- num_train_epochs: int = 1- warmup_steps: int = 10-- # vLLM- use_vllm: bool = True- vllm_mode: str = "colocate"- vllm_gpu_memory_utilization: float = 0.4- vllm_max_model_length: int = 32768-- # Checkpointing- output_dir: str = "/checkpoints/grpo-output"- save_strategy: str = "steps"- save_steps: int = 100-- # Logging- logging_steps: int = 1- use_wandb: bool = True- wandb_project: str = "cua-bench-grpo"-- # Dataset- dataset_size: int = 1000-- # Misc- bf16: bool = True- debug: bool = False---DEFAULT_CONFIG = TrainerConfig()---# =============================================================================-# Modal App Configuration-# =============================================================================--app = modal.App("cua-bench-grpo-training")--image = (- modal.Image.debian_slim(python_version="3.11")- .apt_install("git", "chromium", "chromium-driver")- .uv_pip_install(- "trl[vllm]==0.27.1",- "datasets==3.5.1",- "verl",- "pillow",- "requests",- "fastapi",- "uvicorn",- "playwright",- "cua-bench",- )- .run_commands("playwright install chromium")- # .add_local_dir("cua_bench", remote_path="/root/cua_bench")-)--checkpoint_volume = modal.Volume.from_name("cua-bench-grpo-checkpoints", create_if_missing=True)--# Optional: wandb secret for logging-# Create with: modal secret create wandb-secret WANDB_API_KEY=your_key---# =============================================================================-# Constants-# =============================================================================--SYSTEM_PROMPT = """You are a GUI agent. Complete tasks by interacting with the screen.--Available actions:-- click(x, y): Click at coordinates (0-1000 range)-- type(text): Type text-- hotkey(key1, key2, ...): Press keyboard shortcut-- scroll(x, y, direction): Scroll up/down-- wait(): Wait 1 second-- done(): Mark task complete--Reply with thinking in <|think_start|>...<|think_end|> tags,-then action in <|action_start|>...<|action_end|> tags.-Once you complete the task, output a done() action.--Example:-<|think_start|>I see a blue button. I'll click it.<|think_end|>-<|action_start|>click(500, 300)<|action_end|>"""--CLICK_TARGET_HTML = """<!DOCTYPE html>-<html>-<head>- <style>- body {{ margin: 0; display: flex; justify-content: center; align-items: center; height: 100vh; background: #f0f0f0; }}- .target {{ position: absolute; left: {x}px; top: {y}px; padding: 12px 24px; background: #3b82f6; color: white; border: none; border-radius: 8px; cursor: pointer; }}- </style>-</head>-<body>- <h1>Click the Button</h1>- <button class="target" onclick="window.__clicked=true;">Click Me</button>- <script>window.__clicked = false;</script>-</body>-</html>"""--CLICK_TARGET_TASK_PY = '''-import cua_bench as cb-import random--HTML = """{html}"""-pid = None-[email protected]_config(split="train")-def get_tasks():- return [cb.Task(- description="Click the blue 'Click Me' button",- computer={{"provider": "simulated", "setup_config": {{"width": 1024, "height": 720}}}},- metadata={{"x": random.randint(100, 350), "y": random.randint(100, 250)}}- ) for _ in range(50)]-[email protected]_task(split="train")-async def setup(task, session):- global pid- html = HTML.format(x=task.metadata["x"], y=task.metadata["y"])- pid = await session.launch_window(html=html, title="Click Target", width=512, height=384)-[email protected]_task(split="train")-async def evaluate(task, session):- return [1.0 if await session.execute_javascript(pid, "window.__clicked") else 0.0]-'''---# =============================================================================-# Training Function-# =============================================================================-- image=image,- gpu="H100",- timeout=60 * 60 * 4, # 4 hours- volumes={"/checkpoints": checkpoint_volume},- secrets=[modal.Secret.from_name("wandb-secret")],-)-def train_grpo(- model_id: str = DEFAULT_CONFIG.model_id,- task_prompt: str = DEFAULT_CONFIG.task_prompt,- num_workers: int = DEFAULT_CONFIG.num_workers,- max_steps: int = DEFAULT_CONFIG.max_steps,- max_history: int = DEFAULT_CONFIG.max_history,- num_generations: int = DEFAULT_CONFIG.num_generations,- dataset_size: int = DEFAULT_CONFIG.dataset_size,- max_completion_length: int = DEFAULT_CONFIG.max_completion_length,- temperature: float = DEFAULT_CONFIG.temperature,- learning_rate: float = DEFAULT_CONFIG.learning_rate,- gradient_accumulation_steps: int = DEFAULT_CONFIG.gradient_accumulation_steps,- save_steps: int = DEFAULT_CONFIG.save_steps,- use_wandb: bool = DEFAULT_CONFIG.use_wandb,- wandb_project: str = DEFAULT_CONFIG.wandb_project,- debug: bool = DEFAULT_CONFIG.debug,-):- """Run GRPO training with CUA-Bench environments."""- import asyncio- import base64- import io- import re- import tempfile- from pathlib import Path-- from datasets import Dataset- from PIL import Image- from transformers import AutoTokenizer-- from trl import GRPOConfig, GRPOTrainer- from vllm import SamplingParams-- from cua_bench.workers import CBEnvWorkerClient, cleanup_workers, create_workers-- # -------------------------------------------------------------------------- # Multimodal rollout generation (based on TRL's generate_rollout_completions)- # --------------------------------------------------------------------------- def generate_rollout_completions_multimodal(- trainer,- prompts: list[str],- images: list[list[Image.Image]] | None = None,- ) -> list[dict]:- """- Generate completions for multimodal prompts using vLLM in colocate mode.-- Args:- trainer: GRPOTrainer instance with vLLM configured- prompts: List of text prompts- images: Optional list of image lists, one per prompt-- Returns:- List of dicts with prompt_ids, completion_ids, logprobs, and text- """- if not prompts:- return []-- if not trainer.use_vllm or trainer.vllm_mode != "colocate":- raise RuntimeError("Multimodal rollouts require vLLM in colocate mode.")-- # Build sampling params- sampling_params = SamplingParams(- n=1,- temperature=trainer.temperature,- top_k=trainer.top_k,- min_p=0.0 if trainer.min_p is None else trainer.min_p,- max_tokens=trainer.max_completion_length,- logprobs=0,- )- if trainer.repetition_penalty is not None:- sampling_params.repetition_penalty = trainer.repetition_penalty- if trainer.top_p is not None:- sampling_params.top_p = trainer.top_p-- # Wake up vLLM if sleep mode is enabled- if trainer.args.vllm_enable_sleep_mode:- trainer.llm.wake_up(tags=["kv_cache"])- trainer.llm.collective_rpc("reload_weights")-- # Build inputs with multimodal data if images provided- if images:- inputs = []- for i, prompt in enumerate(prompts):- prompt_images = images[i] if i < len(images) else []- if prompt_images:- inputs.append({- "prompt": prompt,- "multi_modal_data": {"image": prompt_images},- })- else:- inputs.append({"prompt": prompt})- vllm_outputs = trainer.llm.generate(inputs, sampling_params=sampling_params, use_tqdm=False)- else:- vllm_outputs = trainer.llm.generate(prompts, sampling_params=sampling_params, use_tqdm=False)-- # Process outputs- results = []- for request in vllm_outputs:- if not request.outputs:- results.append({- "prompt_ids": request.prompt_token_ids,- "completion_ids": [],- "logprobs": [],- "text": "",- })- continue- sequence = request.outputs[0]- logprobs = [- next(iter(token_logprob.values())).logprob- for token_logprob in sequence.logprobs- ] if sequence.logprobs else []- results.append({- "prompt_ids": request.prompt_token_ids,- "completion_ids": list(sequence.token_ids),- "logprobs": logprobs,- "text": sequence.text,- })-- # Sleep vLLM if sleep mode is enabled- if trainer.args.vllm_enable_sleep_mode:- trainer.llm.sleep(level=2)-- return results-- # -------------------------------------------------------------------------- # Helper functions- # --------------------------------------------------------------------------- def decode_image(b64_str: str) -> Image.Image:- img = Image.open(io.BytesIO(base64.b64decode(b64_str)))- return img.convert("RGB") if img.mode != "RGB" else img-- def extract_images(obs: str) -> tuple[str, list[Image.Image]]:- """Extract base64 images from obs and replace with vLLM-compatible placeholder."""- images = []- pattern = r"<\|vision_start\|>(.*?)<\|vision_end\|>"- def repl(m):- images.append(decode_image(m.group(1)))- # Use Qwen2-VL's placeholder format that vLLM expects- return "<|vision_start|><|image_pad|><|vision_end|>"- cleaned = re.sub(pattern, repl, obs, flags=re.DOTALL)- return cleaned, images-- def make_prompt(- tok,- instruction: str,- obs: str,- step: int,- history: list[tuple[str, str]] | None = None,- ) -> tuple[str, list[Image.Image]]:- """Build prompt with optional history of previous observations and actions.-- Args:- tok: Tokenizer- instruction: Task instruction- obs: Current observation (may contain base64 images)- step: Current step number- history: List of (prev_obs, prev_action) tuples- """- all_images = []-- # Build history section- history_parts = []- if history:- for i, (prev_obs, prev_action) in enumerate(history):- cleaned_prev_obs, prev_images = extract_images(prev_obs)- all_images.extend(prev_images)- history_parts.append(f"Step {step - len(history) + i + 1}:\n{cleaned_prev_obs}\nAction: {prev_action}")-- # Current observation- cleaned_obs, curr_images = extract_images(obs)- all_images.extend(curr_images)-- # Build user content- if history_parts:- history_text = "\n\n".join(history_parts)- user_content = f"Task: {instruction}\n\nHistory:\n{history_text}\n\nStep {step + 1} (current):\n{cleaned_obs}\n\nWhat action?"- else:- user_content = f"Task: {instruction}\n\nStep {step + 1}:\n{cleaned_obs}\n\nWhat action?"-- messages = [- {"role": "system", "content": SYSTEM_PROMPT},- {"role": "user", "content": user_content},- ]- prompt = tok.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)- return prompt, all_images-- def parse_action(response: str) -> str:- match = re.search(r"<\|action_start\|>(.*?)<\|action_end\|>", response, re.DOTALL)- return match.group(1).strip() if match else "wait()"-- # -------------------------------------------------------------------------- # Episode rollout- # --------------------------------------------------------------------------- def rollout_episode(trainer, env, tok, prompt_text: str, max_hist: int = 2) -> dict:- env_ret, meta = env.reset()-- # Get instruction from env.prompt (set during reset)- instruction = env.prompt["instruction"] if env.prompt else prompt_text-- all_prompt_ids, all_completion_ids, all_logprobs = [], [], []- all_images = []- step_rewards = []- history = [] # List of (screenshot, action) tuples-- for step in range(max_steps):- if env_ret.get("done", False):- break-- # Get current screenshot from env.prompt["steps"][-1]- # steps contains: [screenshot0, action0, screenshot1, action1, ...]- current_screenshot = env.prompt["steps"][-1] if env.prompt else ""-- # Get recent history (last max_hist steps)- recent_history = history[-max_hist:] if history else None-- prompt, images = make_prompt(tok, instruction, current_screenshot, step, history=recent_history)- if images:- all_images.extend(images)-- rollout_output = generate_rollout_completions_multimodal(- trainer, [prompt], images=[images] if images else None- )[0]- all_prompt_ids.extend(rollout_output["prompt_ids"])- all_completion_ids.extend(rollout_output["completion_ids"])- all_logprobs.extend(rollout_output["logprobs"])-- completion = rollout_output.get("text") or tok.decode(- rollout_output["completion_ids"], skip_special_tokens=True- )-- # Add current screenshot and action to history for next step- action = parse_action(completion)- history.append((current_screenshot, action))-- if debug:- print(f" Step {step + 1}: {action}")-- env_ret, _ = env.step(completion)- step_rewards.append(float(env_ret.get("reward", 0.0)))-- result = {- "prompt_ids": all_prompt_ids,- "completion_ids": all_completion_ids,- "logprobs": all_logprobs,- "env_reward": step_rewards[-1] if step_rewards else 0.0,- }- if all_images:- result["images"] = all_images- return result-- # -------------------------------------------------------------------------- # Reward function- # --------------------------------------------------------------------------- def reward_evaluator_func(completions: list[str], env_reward=None, **_) -> list[float]:- if env_reward is not None:- return [float(r) for r in env_reward]- return [0.0] * len(completions)-- # -------------------------------------------------------------------------- # Create task- # --------------------------------------------------------------------------- def create_task(tmp_dir: Path) -> Path:- task_dir = tmp_dir / "click-target"- task_dir.mkdir(exist_ok=True)- task_code = CLICK_TARGET_TASK_PY.format(html=CLICK_TARGET_HTML)- (task_dir / "main.py").write_text(task_code)- return task_dir-- # -------------------------------------------------------------------------- # Main training logic- # --------------------------------------------------------------------------- import os-- # Initialize wandb if available- wandb_enabled = False- if use_wandb and os.environ.get("WANDB_API_KEY"):- try:- import wandb- wandb.init(- project=wandb_project,- config={- "model_id": model_id,- "num_workers": num_workers,- "max_steps": max_steps,- "num_generations": num_generations,- "learning_rate": learning_rate,- "temperature": temperature,- "dataset_size": dataset_size,- },- )- wandb_enabled = True- print("Wandb logging enabled")- except ImportError:- print("Wandb not installed, skipping logging")-- print("=" * 60)- print("GRPO Training with CUA-Bench on Modal")- print("=" * 60)- print(f"Model: {model_id}")- print(f"Workers: {num_workers}")- print(f"Max steps: {max_steps}")- print(f"Generations: {num_generations}")- print(f"Wandb: {wandb_enabled}")- print("=" * 60)-- # Setup tokenizer- tokenizer = AutoTokenizer.from_pretrained(model_id)-- # Create temp task- tmp_dir = Path(tempfile.mkdtemp(prefix="grpo-modal-"))- task_path = str(create_task(tmp_dir))- print(f"Created task: {task_path}")-- # Start workers- print(f"Starting {num_workers} worker(s)...")- workers = asyncio.run(create_workers(- n_workers=num_workers,- allowed_ips=["127.0.0.1"],- startup_timeout=120.0,- ))- worker_urls = [w.api_url for w in workers]- print(f"Workers ready: {worker_urls}")-- # Create environment clients- task_configs = [{"env_path": task_path, "task_index": i % 10, "split": "train"} for i in range(num_workers)]- envs = [- CBEnvWorkerClient({- "server_url": url,- "task_configs": task_configs,- "max_step": max_steps,- "max_hist": 2,- "timeout": 300,- })- for url in worker_urls- ]-- # Dataset- dataset = Dataset.from_dict({"prompt": [task_prompt] * dataset_size})-- # GRPO config- config = GRPOConfig(- output_dir=DEFAULT_CONFIG.output_dir,- use_vllm=DEFAULT_CONFIG.use_vllm,- vllm_mode=DEFAULT_CONFIG.vllm_mode,- vllm_gpu_memory_utilization=DEFAULT_CONFIG.vllm_gpu_memory_utilization,- vllm_max_model_length=DEFAULT_CONFIG.vllm_max_model_length,- num_train_epochs=DEFAULT_CONFIG.num_train_epochs,- per_device_train_batch_size=DEFAULT_CONFIG.per_device_train_batch_size,- warmup_steps=DEFAULT_CONFIG.warmup_steps,- logging_steps=DEFAULT_CONFIG.logging_steps,- save_strategy=DEFAULT_CONFIG.save_strategy,- bf16=DEFAULT_CONFIG.bf16,- # From function args (overridable)- learning_rate=learning_rate,- gradient_accumulation_steps=gradient_accumulation_steps,- num_generations=num_generations,- max_completion_length=max_completion_length,- temperature=temperature,- save_steps=save_steps,- report_to="wandb" if wandb_enabled else "none",- )-- # Rollout function- env_idx = [0]-- def rollout_func(prompts: list[str], trainer: GRPOTrainer) -> dict[str, list]:- all_prompt_ids, all_completion_ids, all_logprobs = [], [], []- all_rewards, all_images = [], []-- if debug:- print(f"\n[Rollout] Processing {len(prompts)} prompts")-- for i, prompt in enumerate(prompts):- env = envs[env_idx[0] % len(envs)]- env_idx[0] += 1-- if debug:- print(f"[Rollout] Episode {i + 1}/{len(prompts)}")-- episode = rollout_episode(trainer, env, tokenizer, prompt, max_hist=max_history)-- all_prompt_ids.append(episode["prompt_ids"])- all_completion_ids.append(episode["completion_ids"])- all_logprobs.append(episode["logprobs"])- all_rewards.append(episode["env_reward"])-- if "images" in episode:- all_images.append(episode["images"])-- result = {- "prompt_ids": all_prompt_ids,- "completion_ids": all_completion_ids,- "logprobs": all_logprobs,- "env_reward": all_rewards,- }- if all_images:- result["images"] = all_images- return result-- # Create trainer- trainer = GRPOTrainer(- model=model_id,- processing_class=tokenizer,- reward_funcs=[reward_evaluator_func],- train_dataset=dataset,- args=config,- rollout_func=rollout_func,- )-- try:- print("\nStarting training...")- trainer.train()-- # Save final checkpoint- trainer.save_model("/checkpoints/final")- checkpoint_volume.commit()-- print("\nTraining complete!")- return {"status": "complete", "checkpoints": "/checkpoints/final"}-- finally:- print("Cleaning up workers...")- asyncio.run(cleanup_workers(workers))- import shutil- shutil.rmtree(tmp_dir, ignore_errors=True)---# =============================================================================-# Entrypoint-# =============================================================================-[email protected]_entrypoint()-def main(- model_id: str | None = None,- num_workers: int | None = None,- max_steps: int | None = None,- num_generations: int | None = None,- dataset_size: int | None = None,- learning_rate: float | None = None,- save_steps: int | None = None,- debug: bool = False,-):- """Run GRPO training on Modal."""- kwargs = {- k: v for k, v in {- "model_id": model_id,- "num_workers": num_workers,- "max_steps": max_steps,- "num_generations": num_generations,- "dataset_size": dataset_size,- "learning_rate": learning_rate,- "save_steps": save_steps,- "debug": debug,- }.items() if v is not None- }- result = train_grpo.remote(**kwargs)- print(f"Training result: {result}")-```--## Step 3: Run Training--```bash-modal run modal_grpo_training.py-```--With custom settings:--```bash-modal run modal_grpo_training.py \- --model-id Qwen/Qwen3-VL-2B-Instruct \- --num-workers 4 \- --max-steps 10 \- --num-generations 4 \- --learning-rate 5e-6 \- --debug-```--## CLI Options--| Option | Default | Description |-| ------------------- | --------------------------- | -------------------------------------- |-| `--model-id` | `Qwen/Qwen3-VL-2B-Instruct` | Model to train |-| `--num-workers` | `2` | Number of parallel environment workers |-| `--max-steps` | `10` | Max steps per episode |-| `--num-generations` | `4` | Rollouts per prompt |-| `--dataset-size` | `1000` | Number of training samples |-| `--learning-rate` | `5e-6` | Learning rate |-| `--save-steps` | `100` | Checkpoint save interval |-| `--debug` | `False` | Enable verbose output |--## Creating Custom Tasks--To train on your own task, create a task directory with `main.py`:--```python-import cua_bench as cb-[email protected]_config(split="train")-def get_tasks():- return [cb.Task(- description="Your task description",- computer={"provider": "simulated", "setup_config": {"width": 1024, "height": 720}},- )]-[email protected]_task(split="train")-async def setup(task, session):- await session.launch_window(url="https://example.com")-[email protected]_task(split="train")-async def evaluate(task, session):- success = await session.execute_javascript(None, "checkSuccess()")- return [1.0 if success else 0.0]-```--Then modify the script to use your task path instead of the built-in click target task.--## Reference--- [TRL GRPOTrainer Documentation](https://huggingface.co/docs/trl/grpo_trainer)-- [CUA-Bench Workers API](/cuabench/guide/advanced/rl-dataloader)diff --git a/docs/content/docs/cuabench/examples/universal-task.mdx b/docs/content/docs/cuabench/examples/universal-task.mdxdeleted file mode 100644index 6b709112f8..0000000000--- a/docs/content/docs/cuabench/examples/universal-task.mdx+++ /dev/null@@ -1,412 +0,0 @@-----title: Create a Universal Task-description: Build a cross-platform Minesweeper game that works on QEMU, simulated desktop, and Docker------In this tutorial, you'll create a Minesweeper game task using the Universal GUI API. This task is **universal** - it runs across QEMU, simulated desktop, and Docker environments without modification.--**Time:** ~15 minutes-**Prerequisites:** cua-bench installed--## What You'll Build--A fully playable Minesweeper game where agents must click cells to reveal them and flag mines. You'll learn:--- Using `launch_window()` to create GUI apps-- Communicating with your GUI via JavaScript-- Creating task variants with different difficulty levels-- Evaluating game state for rewards--## Step 1: Create Task Directory--```bash-mkdir -p tasks/minesweeper/gui-cd tasks/minesweeper-```--## Step 2: Create the Task Configuration--Create `main.py`:--```python-import cua_bench as cb-from pathlib import Path--pid = None-[email protected]_config(split="train")-def load():- """Generate Minesweeper variants with different difficulty levels."""- game_configs = [- (8, 8, 10), # Easy- (10, 10, 15), # Medium- (12, 12, 20), # Hard- ]-- return [- cb.Task(- description=f'Play Minesweeper on a {rows}x{cols} grid with {mines} mines. Click cells to reveal them, right-click to flag mines. Win by revealing all non-mine cells.',- metadata={- "rows": rows,- "cols": cols,- "mines": mines,- },- computer={- "provider": "simulated", # Use simulated desktop for preview- "setup_config": {- "os_type": "win11",- "width": 800,- "height": 800,- }- }- )- for rows, cols, mines in game_configs- ]-[email protected]_task(split="train")-async def start(task_cfg: cb.Task, session: cb.DesktopSession):- """Launch the Minesweeper GUI window."""- global pid-- # Get game parameters from task config- rows = task_cfg.metadata["rows"]- cols = task_cfg.metadata["cols"]- mines = task_cfg.metadata["mines"]-- # Calculate window size based on grid size- window_width = cols * 30 + 100- window_height = rows * 30 + 200-- # Launch window with the game GUI- pid = await session.launch_window(- html=(Path(__file__).parent / "gui/index.html").read_text('utf-8'),- title="Minesweeper",- width=window_width,- height=window_height,- )-- # Initialize the game with the task configuration- if pid is not None:- await session.execute_javascript(pid, f"window.initGame({rows}, {cols}, {mines})")-[email protected]_task(split="train")-async def evaluate(task_cfg: cb.Task, session: cb.DesktopSession) -> list[float]:- """Check if the agent won or provide partial credit."""- global pid-- if pid is None:- return [0.0]-- # Check game state via JavaScript- game_won = await session.execute_javascript(pid, "window.__gameWon")- game_lost = await session.execute_javascript(pid, "window.__gameLost")-- # Win: 1.0, Loss: 0.0- if game_won is True:- return [1.0]- elif game_lost is True:- return [0.0]- else:- # Partial credit based on revealed cells- rows = task_cfg.metadata["rows"]- cols = task_cfg.metadata["cols"]- mines = task_cfg.metadata["mines"]-- revealed_count = await session.execute_javascript(pid, "window.game.revealedCount")- total_non_mines = rows * cols - mines-- if revealed_count and total_non_mines > 0:- return [revealed_count / total_non_mines]- return [0.0]-[email protected]_task(split="train")-async def solve(task_cfg: cb.Task, session: cb.DesktopSession):- """Demonstrate a simple solution strategy."""- global pid- import asyncio-- if pid is None:- return-- rows = task_cfg.metadata["rows"]- cols = task_cfg.metadata["cols"]-- # Simple strategy: click corner first, then scan left-to-right- await session.click_element(pid, '[data-row="0"][data-col="0"]')- await asyncio.sleep(0.5)-- # Continue clicking unrevealed cells- for r in range(rows):- for c in range(cols):- game_state = await session.execute_javascript(pid, "window.__gameState")- if game_state != "playing":- break-- is_revealed = await session.execute_javascript(- pid, f"window.game.revealed[{r}][{c}]"- )- is_flagged = await session.execute_javascript(- pid, f"window.game.flagged[{r}][{c}]"- )-- if not is_revealed and not is_flagged:- await session.click_element(pid, f'[data-row="{r}"][data-col="{c}"]')- await asyncio.sleep(0.2)-- if game_state != "playing":- break-```--## Step 3: Create the HTML GUI--Create `gui/index.html` with the Minesweeper game:--```html-<div class="p-4 flex flex-col items-center">- <div class="mb-4 text-center">- <h1 class="text-2xl font-bold mb-2">Minesweeper</h1>- <div class="flex gap-4 justify-center items-center mb-2">- <div class="text-lg font-mono">🚩 <span id="flag-count">0</span></div>- <button id="reset-btn" class="px-4 py-2 bg-gray-200 hover:bg-gray-300 rounded">- 😊 New Game- </button>- <div class="text-lg font-mono">⏱️ <span id="timer">0</span></div>- </div>- <div id="status" class="text-sm font-semibold"></div>- </div>- <div id="game-board" class="inline-block border-4 border-gray-400 bg-gray-300"></div>-- <style>- .cell {- width: 30px;- height: 30px;- border: 2px outset #999;- background: #c0c0c0;- display: inline-flex;- align-items: center;- justify-content: center;- font-weight: bold;- font-size: 16px;- cursor: pointer;- user-select: none;- }- .cell.revealed {- border: 1px solid #999;- background: #e0e0e0;- cursor: default;- }- .cell.mine {- background: #ff6b6b;- }- .cell.num-1 {- color: blue;- }- .cell.num-2 {- color: green;- }- .cell.num-3 {- color: red;- }- </style>-- <script>- class Minesweeper {- constructor(rows, cols, mines) {- this.rows = rows;- this.cols = cols;- this.totalMines = mines;- this.board = [];- this.revealed = [];- this.flagged = [];- this.revealedCount = 0;- this.initBoard();- this.render();- window.__gameState = 'playing';- window.__gameWon = false;- window.__gameLost = false;- }-- initBoard() {- for (let r = 0; r < this.rows; r++) {- this.board[r] = [];- this.revealed[r] = [];- this.flagged[r] = [];- for (let c = 0; c < this.cols; c++) {- this.board[r][c] = 0;- this.revealed[r][c] = false;- this.flagged[r][c] = false;- }- }- }-- placeMines(excludeRow, excludeCol) {- let placed = 0;- while (placed < this.totalMines) {- const r = Math.floor(Math.random() * this.rows);- const c = Math.floor(Math.random() * this.cols);- if ((r === excludeRow && c === excludeCol) || this.board[r][c] === -1) {- continue;- }- this.board[r][c] = -1;- placed++;- }-- // Calculate numbers- for (let r = 0; r < this.rows; r++) {- for (let c = 0; c < this.cols; c++) {- if (this.board[r][c] !== -1) {- this.board[r][c] = this.countAdjacentMines(r, c);- }- }- }- }-- countAdjacentMines(row, col) {- let count = 0;- for (let dr = -1; dr <= 1; dr++) {- for (let dc = -1; dc <= 1; dc++) {- if (dr === 0 && dc === 0) continue;- const r = row + dr,- c = col + dc;- if (r >= 0 && r < this.rows && c >= 0 && c < this.cols && this.board[r][c] === -1) {- count++;- }- }- }- return count;- }-- reveal(row, col) {- if (this.revealed[row][col] || this.flagged[row][col]) return;-- if (!this.firstClickDone) {- this.placeMines(row, col);- this.firstClickDone = true;- }-- this.revealed[row][col] = true;- this.revealedCount++;-- if (this.board[row][col] === -1) {- this.lose();- return;- }-- if (this.board[row][col] === 0) {- for (let dr = -1; dr <= 1; dr++) {- for (let dc = -1; dc <= 1; dc++) {- const r = row + dr,- c = col + dc;- if (r >= 0 && r < this.rows && c >= 0 && c < this.cols && !this.revealed[r][c]) {- this.reveal(r, c);- }- }- }- }-- this.checkWin();- this.render();- }-- checkWin() {- if (this.revealedCount === this.rows * this.cols - this.totalMines) {- window.__gameState = 'won';- window.__gameWon = true;- document.getElementById('status').textContent = '🎉 You Won!';- }- }-- lose() {- window.__gameState = 'lost';- window.__gameLost = true;- document.getElementById('status').textContent = '💥 Game Over!';- }-- render() {- const board = document.getElementById('game-board');- board.innerHTML = '';- board.style.display = 'grid';- board.style.gridTemplateColumns = `repeat(${this.cols}, 30px)`;-- for (let r = 0; r < this.rows; r++) {- for (let c = 0; c < this.cols; c++) {- const cell = document.createElement('div');- cell.className = 'cell';- cell.dataset.row = r;- cell.dataset.col = c;-- if (this.revealed[r][c]) {- cell.classList.add('revealed');- if (this.board[r][c] === -1) {- cell.textContent = '💣';- cell.classList.add('mine');- } else if (this.board[r][c] > 0) {- cell.textContent = this.board[r][c];- cell.classList.add(`num-${this.board[r][c]}`);- }- } else if (this.flagged[r][c]) {- cell.textContent = '🚩';- }-- cell.addEventListener('click', () => this.reveal(r, c));- board.appendChild(cell);- }- }- }- }-- window.initGame = function (rows, cols, mines) {- window.game = new Minesweeper(rows, cols, mines);- };- </script>-</div>-```--## Step 4: Run the Task--### Preview Interactively--```bash-cb interact tasks/minesweeper --variant-id 0-```--A browser window will open showing the Minesweeper game in a simulated desktop environment.--### Run with Oracle--```bash-cb interact tasks/minesweeper --oracle --variant-id 0-```--### Run with Agent--```bash-export ANTHROPIC_API_KEY=sk-...-cb run task tasks/minesweeper --agent cua-agent --model anthropic/claude-sonnet-4-20250514-```--## Key Concepts--**Window Management**: `launch_window()` creates GUI apps in your tasks--**JavaScript Communication**: Use `execute_javascript()` to read/write game state--**Task Variants**: One codebase generates multiple difficulty levels--**Partial Rewards**: Reward based on progress, not just win/loss--## Why Universal?--This task works across all platforms:--- **Simulated Desktop**: Set `provider: "simulated"` for lightweight browser preview-- **Docker**: Use real Linux desktop environments-- **QEMU**: Run on actual Windows/Linux VMs--The same HTML GUI and task code runs everywhere.--## Next Steps--- Learn about [Simulated Desktop](/cuabench/guide/fundamentals/simulated-desktop) for themed previews-- See [Universal GUI](/cuabench/guide/fundamentals/universal-gui) for the full API-- Build a [Custom Agent](/cuabench/examples/custom-agent) to solve tasksdiff --git a/docs/content/docs/cuabench/examples/windows-native-task.mdx b/docs/content/docs/cuabench/examples/windows-native-task.mdxdeleted file mode 100644index 80ea199e26..0000000000--- a/docs/content/docs/cuabench/examples/windows-native-task.mdx+++ /dev/null@@ -1,72 +0,0 @@-----title: Windows Native Task-description: Build a Unity project creation task using app helpers on Windows------Create a task where agents must create a Unity project on Windows. This shows how to use app helpers for installation and file system checks for evaluation.--## Prerequisites--Create a Windows VM image:--```bash-cb image create windows-qemu --download-iso-```--## Create the Task--Create `tasks/unity_project/main.py`:--```python-import cua_bench as cb--[email protected]_config(split="train")-def load():- return [- cb.Task(- description="Create a new Unity 3D project called 'MyGame' and save it to Desktop/UnityProjects/",- metadata={"project_name": "MyGame"},- computer={- "provider": "native",- "setup_config": {"os_type": "windows", "width": 1920, "height": 1080},- },- )- ]--[email protected]_task(split="train")-async def start(task_cfg: cb.Task, session: cb.DesktopSession):- """Install Unity using the app helper."""- await session.apps.unity.install(with_shortcut=True)--[email protected]_task(split="train")-async def evaluate(task_cfg: cb.Task, session: cb.DesktopSession) -> list[float]:- """Check if the Unity project was created."""- project_name = task_cfg.metadata["project_name"]-- result = await session.run_command(- f'if exist "%USERPROFILE%\\Desktop\\UnityProjects\\{project_name}\\Assets" (echo FOUND) else (echo NOT_FOUND)',- check=False,- )-- if "FOUND" in result["stdout"]:- return [1.0]- return [0.0]---if __name__ == "__main__":- cb.interact(__file__)-```--## Run It--```bash-cb interact tasks/unity_project-```--## Next Steps--- [macOS Native Task](/cuabench/examples/macos-native-task) - Use osascript-based app helpers-- [App Helpers](/cuabench/guide/fundamentals/app-helpers) - Full API referencediff --git a/docs/content/docs/cuabench/guide/advanced/configuration.mdx b/docs/content/docs/cuabench/guide/advanced/configuration.mdxdeleted file mode 100644index 12aadc909f..0000000000--- a/docs/content/docs/cuabench/guide/advanced/configuration.mdx+++ /dev/null@@ -1,145 +0,0 @@-----title: Configuration-description: Configure cua-bench with config files and environment variables------# Configuration--Configure cua-bench behavior with `.cua/` config files and environment variables.--## Config Files--| File | Description |-| ------------------- | ------------------------------ |-| `.cua/config.yaml` | Default execution settings |-| `.cua/secrets.yaml` | Secret references for API keys |-| `.cua/agents.yaml` | Agent definitions |--## config.yaml--Default execution settings for `cb run`:--```yaml-# Where to run: "local" or "cloud"-on: local--# Parallel execution-max_parallel: 16-timeout: 600--# Default images by platform-defaults:- images:- linux-docker: linux-docker- linux-qemu: linux-qemu- windows-qemu: windows-waa- android-qemu: android-qemu- macos-lume: macos-lume-- resources:- linux-docker:- memory: 4G- cpus: 2- windows-qemu:- memory: 8G- cpus: 4--# Default agent image-default_image: trycua/cua-bench:latest-```--## secrets.yaml--Reference API keys securely (values stored in environment variables):--```yaml-secrets:- anthropic-api-key:- env: ANTHROPIC_API_KEY- openai-api-key:- env: OPENAI_API_KEY- custom-key:- env: MY_CUSTOM_KEY-```--Use in code with `Secret.from_name()`:--```python-from cua_bench import Secret-- secrets=[Secret.from_name("anthropic-api-key")],-)-async def run_task(task_index: int, env) -> dict:- import os- api_key = os.environ["ANTHROPIC_API_KEY"]-```--## agents.yaml--Define custom agents:--```yaml-agents:- my-agent:- image: my-registry/my-agent:latest- model: anthropic/claude-sonnet-4-20250514- max_steps: 50-- cua-agent:- image: trycua/cua-agent:latest- model: anthropic/claude-sonnet-4-20250514-```--Use with `--agent`:--```bash-cb run dataset datasets/cua-bench-basic --agent my-agent-```--## Environment Variables--| Variable | Description |-| ------------------- | ------------------------------------------ |-| `ANTHROPIC_API_KEY` | Anthropic API key for Claude models |-| `OPENAI_API_KEY` | OpenAI API key for GPT models |-| `CUA_CLOUD_TOKEN` | Cua Cloud authentication token |-| `CUA_CONFIG_PATH` | Custom config directory (default: `.cua/`) |--## Image Registry--Images are stored in `~/.local/state/cua-bench/images.json`:--```json-{- "version": 1,- "images": {- "linux-docker": {- "platform": "linux-docker",- "docker_image": "trycua/cua-xfce:latest",- "status": "ready"- },- "windows-qemu": {- "platform": "windows-qemu",- "path": "~/.local/state/cua-bench/images/windows-qemu",- "config": {- "memory": "8G",- "cpus": 4- },- "status": "ready"- }- }-}-```--## Authentication--For Cua Cloud access:--```bash-# Browser-based login-cb login--# Verify authentication-cb status-```diff --git a/docs/content/docs/cuabench/guide/advanced/creating-private-tasks.mdx b/docs/content/docs/cuabench/guide/advanced/creating-private-tasks.mdxdeleted file mode 100644index 3289208096..0000000000--- a/docs/content/docs/cuabench/guide/advanced/creating-private-tasks.mdx+++ /dev/null@@ -1,163 +0,0 @@-----title: Creating Private Tasks-description: How to create a private set of tasks.------Cua-Bench is a broad set of tasks meant to test the general ability of an agent to perform complex tasks in a desktop environment.--However, many agents are designed for more narrow use cases. The harness can still be leveraged by making it easy for agent developers to create a custom set of tasks for their agent.--## Creating a Custom Task--### Using the Wizard--```bash-cb task create tasks/my_custom_task-```--This starts an interactive wizard that guides you through the task creation process.--### Using AI Generation--Use Claude Code to scaffold your starter environment:--```bash-cb task generate "discord clone with channels and messages" --output tasks/discord_clone-```--### Manual Creation--Create the task structure manually:--```-tasks/my_custom_task/-├── main.py # Task implementation-├── gui/-│ └── index.html # UI assets-└── pyproject.toml # Optional metadata-```--## Task Implementation--Here's a complete example task:--```python-"""Custom task example - Click a button."""-import cua_bench as cb-from pathlib import Path-[email protected]_config(split="train")-def load():- """Define task variants."""- return [- cb.Task(- description='Click the "Submit" button to complete the task.',- metadata={"button_id": "submit"},- computer={- "provider": "computer",- "setup_config": {- "os_type": "linux",- "width": 1024,- "height": 768,- }- }- )- ]--pid = None-[email protected]_task(split="train")-async def start(task_cfg, session: cb.DesktopSession):- """Initialize the task environment."""- global pid- pid = await session.launch_window(- html=(Path(__file__).parent / "gui/index.html").read_text('utf-8'),- title="My Custom Task",- width=400,- height=300,- )-[email protected]_task(split="train")-async def evaluate(task_cfg, session: cb.DesktopSession) -> list[float]:- """Check if the task was completed successfully."""- global pid- if pid is None:- return [0.0]-- submitted = await session.execute_javascript(pid, "window.__submitted")- return [1.0] if submitted is True else [0.0]-[email protected]_task(split="train")-async def solve(task_cfg, session: cb.DesktopSession):- """Oracle solution - click the button."""- global pid- if pid is None:- return-- await session.click_element(pid, "#submit")--if __name__ == "__main__":- cb.interact(__file__)-```--<Callout title="Async Required" type="warn">- All task functions (setup, evaluate, solve) must be `async` functions, and session methods must be- called with `await`.-</Callout>--## Running Custom Tasks--### Test Locally--```bash-# Interactive mode - opens browser to view task-cb interact tasks/my_custom_task--# Run specific task variant-cb interact tasks/my_custom_task --variant-id 0--# Run oracle solution-cb interact tasks/my_custom_task --oracle--# Save screenshot-cb interact tasks/my_custom_task --oracle --screenshot output.png-```--### Evaluate with Agent--```bash-# Run with agent-cb run task tasks/my_custom_task --agent cua-agent --model anthropic/claude-sonnet-4-20250514--# Validate oracle solutions-cb run task tasks/my_custom_task --oracle-```--## Running a Custom Dataset--To run multiple tasks as a dataset:--```bash-# Run all tasks in a directory-cb run dataset tasks --agent cua-agent --model anthropic/claude-sonnet-4-20250514--# Run specific tasks-cb run dataset tasks/my_custom_task,tasks/another_task --agent cua-agent-```--## Task Development Workflow--1. **Create task structure** with `cb task create` or `cb task generate`-2. **Implement task logic** in `main.py` with async functions-3. **Test interactively** with `cb interact tasks/my_task`-4. **Validate oracle** with `cb interact tasks/my_task --oracle`-5. **Run evaluation** with `cb run task tasks/my_task --oracle` to verify evaluation returns `[1.0]`-6. **Test with agent** with `cb run task tasks/my_task --agent cua-agent`--## Tips--- Use `global pid` to share window references between setup, evaluate, and solve functions-- Always check `if pid is None` in evaluate and solve functions-- Use `await session.execute_javascript(pid, "expression")` to read app state-- Use `await session.click_element(pid, "selector")` for reliable element clicks-- Test with `--oracle` first to ensure your evaluation logic is correctdiff --git a/docs/content/docs/cuabench/guide/advanced/custom-agents.mdx b/docs/content/docs/cuabench/guide/advanced/custom-agents.mdxdeleted file mode 100644index 87b8c59cbe..0000000000--- a/docs/content/docs/cuabench/guide/advanced/custom-agents.mdx+++ /dev/null@@ -1,86 +0,0 @@-----title: Custom Agents-description: Register and run custom agents in cua-bench------# Custom Agents--## Adding an agent to the source--1. Create `cua_bench/agents/your_agent.py` — extend `BaseAgent`, decorate with `@register_agent("your-agent")`-2. Add the import to `cua_bench/agents/__init__.py`-3. Run with `cb run task <path> --agent your-agent`--See the [Build a Custom Agent](/cuabench/examples/custom-agent) example for full code.--## Local registry (without modifying source)--Register an agent via `.cua/agents.yaml` in your project root:--```yaml-agents:- - name: my-agent- import_path: my_package.agents:MyAgent- defaults:- model: claude-sonnet-4-20250514- max_steps: 50-```--The `import_path` format is `module.path:ClassName`. The module must be importable from the Python environment.--## Docker image agents--For production or CI, package your agent as a Docker image:--```yaml-agents:- - name: my-agent- image: myregistry/my-agent:latest- command: ['python', '-m', 'my_agent.main']- defaults:- model: gpt-4o-```--The agent container receives these environment variables:--| Variable | Description |-| --- | --- |-| `CUA_ENV_API_URL` | API endpoint for the environment container |-| `CUA_ENV_VNC_URL` | VNC endpoint for debugging |-| `CUA_ENV_TYPE` | OS type (`linux`, `windows`, `android`) |-| `CUA_TASK_PATH` | Mounted task config path (`/app/env`) |-| `CUA_TASK_INDEX` | Task index to run |-| `CUA_MODEL` | Model override (if specified) |-| `ANTHROPIC_API_KEY` | Passed through from host |-| `OPENAI_API_KEY` | Passed through from host |-| `GOOGLE_API_KEY` | Passed through from host |--## Development with `--with`--Mount and pip install local packages into the agent container without rebuilding:--```bash-cb run task tasks/my-task --agent my-agent --with ./path/to/local/package-```--Can be repeated for multiple packages:--```bash-cb run task tasks/my-task --agent my-agent --with ../cua-bench --with ../my-utils-```--## Agent lookup order--When you pass `--agent <name>`:--1. `.cua/agents.yaml` (local registry) — checked first-2. Built-in registry (`@register_agent` decorators) — fallback--## CLI options--```bash-cb run task <path> --agent <name> # Use a registered agent-cb run task <path> --agent-kwarg key=value # Pass kwargs to agent __init__-cb run task <path> --model <model> # Override model-cb run task <path> --with <path> # Mount local package (repeatable)-```diff --git a/docs/content/docs/cuabench/guide/advanced/images.mdx b/docs/content/docs/cuabench/guide/advanced/images.mdxdeleted file mode 100644index 7b30d5988f..0000000000--- a/docs/content/docs/cuabench/guide/advanced/images.mdx+++ /dev/null@@ -1,257 +0,0 @@-----title: Base Images-description: Setting up reproducible VM environments for benchmarks------Base images are reproducible environments for running benchmarks. Each image type provides a consistent, isolated desktop or mobile environment. Images are protected by overlays during benchmark runs, ensuring each task starts from a clean state.--## Image Types--| Type | Command | Description | Requirements |-| -------------- | --------------------------------------------- | -------------------------- | ------------- |-| `linux-docker` | `cb image create linux-docker` | Linux GUI container (XFCE) | Docker |-| `linux-qemu` | `cb image create linux-qemu` | Linux VM with QEMU/KVM | Docker + KVM |-| `windows-qemu` | `cb image create windows-qemu --download-iso` | Windows 11 VM | Docker + KVM |-| `android-qemu` | `cb image create android-qemu` | Android VM | Docker + KVM |-| `macos-lume` | `cb image create macos-lume` | macOS VM via Lume | Apple Silicon |--## Quick Setup--### linux-docker (Fastest)--Container-based Linux GUI. No KVM required, works anywhere Docker runs.--```bash-# Create image (~1 minute)-cb image create linux-docker--# Start interactive shell (protected by overlay)-cb image shell linux-docker--# Access via browser at http://localhost:8006-# Press Ctrl+C to stop-```--### windows-qemu (Windows Arena)--Full Windows 11 VM with QEMU/KVM. Required for Windows Arena benchmarks.--```bash-# Create image (~1-2 hours, downloads Windows 11 ISO)-cb image create windows-qemu --download-iso--# With WinArena benchmark apps pre-installed (Chrome, LibreOffice, VLC, etc.)-cb image create windows-qemu --download-iso --winarena-apps--# With custom resources-cb image create windows-qemu --download-iso --memory 16G --cpus 8 --disk 128G--# Run in background (detached)-cb image create windows-qemu --download-iso --detach-```--<Callout title="Requirements" type="info">- Windows QEMU requires an x86_64 Linux host with KVM support. For cloud VMs, ensure nested- virtualization is enabled.-</Callout>--### macos-lume (Apple Silicon only)--macOS VM using Apple's Virtualization framework via Lume.--```bash-# Install Lume first-/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/lume/scripts/install.sh)"--# Create macOS image-cb image create macos-lume --version sonoma-```--## Interactive Shells--Use `cb image shell` to interactively explore or debug images. By default, shells use an overlay to protect the golden image.--### Protected Mode (Default)--```bash-# Start shell with overlay protection-cb image shell windows-qemu--# Access VNC at http://localhost:8006-# Any changes are discarded when you exit-```--### Writable Mode (Image Authoring)--Use `--writable` when you need to permanently modify an image (e.g., installing apps, configuring settings):--```bash-# Clone image first (recommended workflow)-cb image clone windows-qemu windows-custom--# Modify the clone-cb image shell windows-custom --writable-# Install apps, configure settings...-# Changes persist to the image--# Use your custom image in benchmarks-cb run task my-task --image windows-custom-```--<Callout title="Warning" type="warn">- `--writable` mode directly modifies the golden image. Always clone first to preserve your original- base image.-</Callout>--### Shell Options--```bash-cb image shell <name> [options]-```--| Option | Description |-| ------------------- | ------------------------------------------ |-| `--writable` | Modify golden image directly (dangerous!) |-| `--vnc-port <port>` | VNC port (auto-allocated if not specified) |-| `--api-port <port>` | API port (auto-allocated if not specified) |-| `--memory <size>` | Memory (default: 8G) |-| `--cpus <count>` | CPU cores (default: 8) |-| `--no-kvm` | Disable KVM acceleration |-| `--detach`, `-d` | Run in background |--<Callout title="Auto Port Allocation" type="info">- Ports are automatically allocated to avoid conflicts when not specified. The CLI finds available- ports starting from the defaults (8006 for VNC, 5000 for API).-</Callout>--## Image Management--Images are stored in `~/.local/share/cua-bench/images/`. Use the `cb image` commands to manage them:--```bash-# List all images-cb image list-cb image list --format json--# Show detailed info-cb image info windows-qemu--# Clone for customization-cb image clone windows-qemu windows-custom--# Delete an image-cb image delete windows-custom-```--## Port Mappings--Different image types use different internal ports:--| Image Type | VNC Port (internal) | API Port (internal) |-| -------------- | ------------------- | ------------------- |-| `linux-docker` | 6901 | 8000 |-| `windows-qemu` | 8006 | 5000 |-| `linux-qemu` | 8006 | 5000 |-| `android-qemu` | 8006 | 5000 |--When starting shells, you can map these to custom external ports:--```bash-cb image shell linux-docker --vnc-port 8080 --api-port 9000-```--## Storage Structure--Cua-Bench follows the XDG Base Directory specification:--```-~/.local/share/cua-bench/ # Data directory-├── images/ # Base images (golden)-│ ├── linux-docker/-│ ├── windows-qemu/-│ │ ├── windows.boot # Marker file (setup complete)-│ │ └── data/ # VM disk files-│ └── ...-├── runs/ # Task run outputs (default location)-│ └── <run_id>/ # Each run gets its own directory-│ ├── run.log # Execution logs-│ └── ... # Results, traces, etc.-├── overlays/ # Task overlays (ephemeral)-│ ├── shell-windows-qemu/ # Shell overlay-│ └── task-abc123/ # Task run overlay-└── windows.iso # Downloaded Windows ISO--~/.local/state/cua-bench/ # State directory-├── images.json # Image registry-└── runs.json # Run session tracking-```--## Architecture--```-┌─────────────────────────────────────────────────────────────────────────┐-│ cua-bench │-├─────────────────────────────────┬───────────────────────────────────────┤-│ LOCAL │ CLOUD │-│ (Development & Authoring) │ (Execution & Scale) │-├─────────────────────────────────┼───────────────────────────────────────┤-│ │ │-│ cb image create windows-qemu │ cb run dataset --on cloud │-│ cb image shell │ Parallel execution │-│ cb interact │ Agent orchestration │-│ cb image clone │ Result aggregation │-│ │ │-│ ↓ │ ↑ │-│ Create base images │ Run evals at scale │-│ Develop benchmark tasks │ 100s of parallel VMs │-│ Debug & iterate │ Automatic infrastructure │-│ │ │-└─────────────────────────────────┴───────────────────────────────────────┘-```--**Local** is the authoring environment for images and tasks. **Cloud** handles scale.--## Image Protection--During benchmark runs, golden images are protected by an overlay system:--1. **Golden image** mounted read-only at `/golden`-2. **Worker overlay** mounted writable at `/storage`-3. QEMU creates a QCOW2 overlay on top--This ensures:--- Each task starts from a pristine state-- Golden images are never accidentally modified-- Failed tasks don't contaminate subsequent runs--## Troubleshooting--### KVM not available--```bash-# Check if KVM is available-ls -la /dev/kvm--# If not available on cloud VM, enable nested virtualization-# or use linux-docker which doesn't require KVM-```--### Docker not running--```bash-# Start Docker-sudo systemctl start docker--# Or on macOS-open -a Docker-```--### Port already in use--Ports are now auto-allocated by default, so conflicts are rare. If you need specific ports:--```bash-# Use custom ports-cb image shell linux-docker --vnc-port 9000 --api-port 9001-```diff --git a/docs/content/docs/cuabench/guide/advanced/meta.json b/docs/content/docs/cuabench/guide/advanced/meta.jsondeleted file mode 100644index 5ebc98286b..0000000000--- a/docs/content/docs/cuabench/guide/advanced/meta.json+++ /dev/null@@ -1,13 +0,0 @@-{- "title": "Advanced",- "description": "Build your own tasks and agents",- "icon": "Layers",- "pages": [- "images",- "creating-private-tasks",- "custom-agents",- "rl-dataloader",- "configuration",- "telemetry"- ]-}diff --git a/docs/content/docs/cuabench/guide/advanced/rl-dataloader.mdx b/docs/content/docs/cuabench/guide/advanced/rl-dataloader.mdxdeleted file mode 100644index 6e91481744..0000000000--- a/docs/content/docs/cuabench/guide/advanced/rl-dataloader.mdx+++ /dev/null@@ -1,174 +0,0 @@-----title: RL Dataloader-description: API reference for ReplayBuffer and MultiTurnDataloader------# RL Dataloader--The `MultiTurnDataloader` handles parallel environment rollout, tokenization, and batching for RL training.--## ReplayBuffer--Stores experience tuples with episode-aware reward discounting.--```python-from cua_bench.workers import ReplayBuffer--buffer = ReplayBuffer(- capacity=10000, # Max experiences to store- gamma=0.9, # Discount factor- only_keep_outcome=False, # Keep all steps or just final- balance_thres=0.5, # Threshold for balance stats-)--# Add experiences (as tuple)-buffer.add((- 0, # worker_id- {"obs": "...", "reward": 0.0, "done": False}, # env_ret- {"uid": "episode-123"} # meta_info-))--# Sample for training-samples = buffer.sample(batch_size=32)--# Get balance statistics-below, above = buffer.get_balance_stats()-```--### Reward Discounting--When an episode completes, rewards propagate backwards:--```-Episode: [step0, step1, step2 (done, reward=1.0)]-With gamma=0.9:- step2.reward = 1.0- step1.reward = 0.9- step0.reward = 0.81-```--## MultiTurnDataloader--Manages parallel environments and provides tokenized batches.--```python-from cua_bench.workers import (- MultiTurnDataloader,- CBEnvWorkerClient,- create_workers,-)--workers = await create_workers(n_workers=4, allowed_ips=["127.0.0.1"])-task_configs = [{"env_path": "./task", "task_index": 0, "split": "train"}]-env_configs = [{- "server_url": w.api_url,- "task_configs": task_configs,- "max_step": 50,- "max_hist": 10,- "timeout": 300,-} for w in workers]--dataloader = MultiTurnDataloader(- env_class=CBEnvWorkerClient,- env_configs=env_configs,- tokenizer=tokenizer,- batch_size=4,- replay_capacity=10000,- replay_reward_discount=0.9,- max_prompt_length=1024,- max_response_length=256,-)--for batch in dataloader:- # batch: input_ids, attention_mask, position_ids, worker_id, meta_info- responses = model.generate(batch['input_ids'])-- dataloader.async_step({- 'prompts': batch['input_ids'],- 'responses': responses,- 'attention_mask': combined_mask,- 'worker_id': batch['worker_id'],- 'meta_info': batch['meta_info'],- })-```--### Constructor Parameters--| Parameter | Type | Default | Description |-| ----------------------------- | ------------ | -------- | -------------------------------------- |-| `env_class` | Type | required | Environment client class |-| `env_configs` | `List[Dict]` | required | Worker configs |-| `task_configs` | `List[Dict]` | required | Task configurations |-| `tokenizer` | Any | required | HuggingFace tokenizer |-| `processor` | Any | `None` | HuggingFace processor for multimodal |-| `is_multi_modal` | bool | `False` | Enable image processing |-| `batch_size` | int | `8` | Must be less than or equal to num_envs |-| `replay_capacity` | int | `10000` | Replay buffer size |-| `replay_reward_discount` | float | `0.9` | Gamma for discounting |-| `max_prompt_length` | int | `1024` | Max prompt tokens |-| `max_response_length` | int | `1024` | Max response tokens |-| `only_keep_outcome_in_replay` | bool | `False` | Only keep final steps |--### Batch Format--From `next(dataloader)`:--```python-{- 'input_ids': torch.Tensor, # (batch, seq_len)- 'attention_mask': torch.Tensor, # (batch, seq_len)- 'position_ids': torch.Tensor, # (batch, seq_len)- 'worker_id': np.ndarray,- 'meta_info': np.ndarray,-}-```--### batch_return Format--For `async_step()`:--```python-{- 'prompts': torch.Tensor, # (batch, prompt_len)- 'responses': torch.Tensor, # (batch, response_len)- 'attention_mask': torch.Tensor, # (batch, total_len)- 'worker_id': np.ndarray,- 'meta_info': np.ndarray,-}-```--### Action Parsing--The dataloader parses action strings from responses:--```-<|action_start|>click(0.5, 0.5)<|action_end|>-<|action_start|>type("hello")<|action_end|>-<|action_start|>done()<|action_end|>-```--### Methods--```python-# Get running reward (EMA)-reward = dataloader.running_outcome_reward()--# Get balance stats-below, above = dataloader.get_balance_stats()--# Sample from replay buffer-batch = dataloader.sample_from_buffer(batch_size=32)--# Print stats-dataloader.print_stats_in_replay_buffer()--# Clear replay buffer-dataloader.clear_replay_buffer()--# Cleanup workers-dataloader.close()-```--## Example--See the [Train an Agent with GRPO](/cuabench/examples/rl-training) tutorial for a complete working example.diff --git a/docs/content/docs/cuabench/guide/advanced/telemetry.mdx b/docs/content/docs/cuabench/guide/advanced/telemetry.mdxdeleted file mode 100644index 18b0454f97..0000000000--- a/docs/content/docs/cuabench/guide/advanced/telemetry.mdx+++ /dev/null@@ -1,30 +0,0 @@-----title: Telemetry-description: Understanding and controlling cua-bench's anonymous usage collection------cua-bench collects anonymous usage statistics to help improve the product.--## What We Use It For--This data helps us understand which features are used, identify reliability issues, and prioritize development efforts.--## Never Collected--- Personal information or user identifiers-- API keys or credentials-- File contents, paths, or application data-- Screenshots or traces-- Model outputs or prompts--## Disabling Telemetry--To disable telemetry, set the environment variable:--```bash-export CUA_TELEMETRY_ENABLED=false-```--## Questions--Questions about telemetry? Open an issue on [GitHub](https://github.com/trycua/cua).diff --git a/docs/content/docs/cuabench/guide/fundamentals/adapters.mdx b/docs/content/docs/cuabench/guide/fundamentals/adapters.mdxdeleted file mode 100644index 3378aba666..0000000000--- a/docs/content/docs/cuabench/guide/fundamentals/adapters.mdx+++ /dev/null@@ -1,68 +0,0 @@-----title: Adapters-description: Running benchmarks across different operating systems and environments.------The **Cua-Bench** framework makes it easy to create adapters for existing computer-use benchmarks by implementing their setup/evaluators with the Python task specification.--# Supported Benchmarks--| Benchmark | Image | Tasks | Status | Description |-| ------------- | -------------- | ----- | ----------- | -------------------------------------------- |-| Windows Arena | `windows-qemu` | 173 | Available | Windows 11 desktop automation across 12 apps |-| OSWorld | `linux-qemu` | 369 | Coming soon | Linux desktop automation |-| WebArena | `linux-docker` | 812 | Coming soon | Web browsing tasks |-| AndroidWorld | `android-qemu` | 200+ | Coming soon | Android mobile automation |-| macOSWorld | `macos-lume` | 100+ | Coming soon | macOS desktop automation |--## Example: Run Windows Arena--Windows Arena is a benchmark with 173 tasks across 12 Windows application domains.--<Callout title="Requirements" type="info">- Windows Arena requires x86_64 Linux with KVM support (nested virtualization). Setup takes ~1-2- hours.-</Callout>--### Step 1: Create Windows Base Image--```bash-# Download Windows ISO and create base image (~1-2 hours)-cb image create windows-qemu --download-iso--# With WinArena benchmark apps pre-installed (recommended)-cb image create windows-qemu --download-iso --winarena-apps--# Monitor progress (VNC at http://localhost:8006)-docker logs -f cua-setup-windows-```--### Step 2: Verify Image (Optional)--```bash-# Start interactive shell to verify the image works-cb image shell windows-qemu--# Access VNC at http://localhost:8006-# Press Ctrl+C to stop when done verifying-```--### Step 3: Run Tasks--```bash-# Run specific task with oracle-cb interact tasks/winarena_adapter --variant-id 0 --oracle--# Run with agent (2-container architecture)-cb run task tasks/winarena_adapter \- --agent cua-agent \- --model anthropic/claude-sonnet-4-20250514 \- --image windows-qemu--# Run entire dataset in parallel-cb run dataset datasets/winarena \- --agent cua-agent \- --model anthropic/claude-sonnet-4-20250514 \- --image windows-qemu \- --max-parallel 4-```diff --git a/docs/content/docs/cuabench/guide/fundamentals/agent-traces.mdx b/docs/content/docs/cuabench/guide/fundamentals/agent-traces.mdxdeleted file mode 100644index 037f8fdca5..0000000000--- a/docs/content/docs/cuabench/guide/fundamentals/agent-traces.mdx+++ /dev/null@@ -1,140 +0,0 @@-----title: Agent Traces-description: Recording and viewing agent execution traces------When running tasks with agents, **cua-bench** automatically records detailed traces of agent execution. These traces capture screenshots, actions, reasoning, and metadata—essential for debugging, evaluation, and dataset creation.--## Trace Output Locations--After running a task with an agent, traces are saved to `~/.local/share/cua-bench/runs/<run_id>/<task>_v<variant>/`:--```-~/.local/share/cua-bench/runs/96d41b51/slack_env_v0/-├── run.log # Execution logs-├── task_0_agent_logs/ # Cua Agent SDK trajectories (if using cua-agent)-│ └── trajectories/-│ └── 2026-01-07_claudesonnet4_210819_2a23/-│ ├── metadata.json # Agent turns, model responses, usage-│ ├── turn_000/ # Screenshots and data per turn-│ ├── turn_001/-│ └── ...-└── task_0_trace/ # cua-bench trace dataset- ├── data-00000-of-00001.arrow # HuggingFace Dataset format- ├── dataset_info.json- └── state.json-```--## cua-bench Trace Format--The cua-bench trace format uses **HuggingFace Datasets** with Apache Arrow for efficient storage and loading. Each trace contains:--### Dataset Schema--```python-{- "event_name": str, # Event type: "reset", "agent_step", "agent_action", etc.- "data_json": str, # JSON-encoded event metadata- "data_images": [Image], # List of screenshots (PIL Images)- "trajectory_id": str, # Unique trajectory identifier- "timestamp": str # ISO 8601 timestamp-}-```--### Event Types--| Event | Description | Images | Metadata |-| ---------------- | ------------------------ | ----------------------- | -------------------------------- |-| `reset` | Task setup complete | Initial screenshot | Task config, index |-| `agent_step` | Agent reasoning step | Screenshot | Step number, model usage, output |-| `agent_action` | Agent performed action | Screenshot after action | Action type, arguments |-| `agent_thinking` | Model thinking/response | Screenshot | Thinking text (truncated) |-| `solve` | Oracle solution complete | Final screenshot | Completion status |-| `evaluate` | Task evaluation | - | Evaluation result |--### Loading Traces Programmatically--```python-from datasets import load_from_disk--# Load a trace dataset-trace = load_from_disk("~/.local/share/cua-bench/runs/96d41b51/slack_env_v0/task_0_trace")--# Iterate through events-for event in trace:- print(f"Event: {event['event_name']}")- print(f"Time: {event['timestamp']}")-- # Parse metadata- import json- data = json.loads(event['data_json'])- print(f"Data: {data}")-- # Access screenshots- for img in event['data_images']:- img.show() # PIL Image-```--## Viewing Traces--Use the **cua-bench trace viewer** to visualize traces in your browser:--### View Single Trace--```bash-# View a specific task trace-cb trace view <run_id>/<task>_v<variant>--# Example-cb trace view 96d41b51/slack_env_v0-```--Opens an interactive viewer showing:--- Timeline of events-- Screenshots at each step-- Action details and metadata-- Agent reasoning (if available)--### View Agent Trajectories from a Run--```bash-# Open agent trajectories in the cua.ai trajectory viewer-cb trace traj <run_id>--# Example-cb trace traj 96d41b51-```--Zips and serves the cua-agent trajectory recordings from the run artifacts folder, then opens [cua.ai/trajectory-viewer](https://cua.ai/trajectory-viewer). For runs with multiple tasks, an index page is shown so you can open each session individually.--## Recording Custom Events--When building custom agents, you can record your own events to the trace:--```python-from cua_bench.agents.base import BaseAgent, AgentResult--class MyAgent(BaseAgent):- async def perform_task(- self,- task_description: str,- session,- logging_dir=None,- tracer=None, # Tracer is passed automatically- ) -> AgentResult:- # Record custom event- if tracer:- screenshot = await session.screenshot()- tracer.record(- "my_custom_event",- {- "step": 1,- "reasoning": "Analyzing the screen...",- "confidence": 0.95,- },- [screenshot]- )-- # Your agent logic...-```diff --git a/docs/content/docs/cuabench/guide/fundamentals/app-helpers.mdx b/docs/content/docs/cuabench/guide/fundamentals/app-helpers.mdxdeleted file mode 100644index 91b7051d70..0000000000--- a/docs/content/docs/cuabench/guide/fundamentals/app-helpers.mdx+++ /dev/null@@ -1,95 +0,0 @@-----title: App Helpers-description: Reusable primitives for installing, launching, and evaluating native apps------App helpers provide reusable methods for working with native applications. Beyond install/launch, apps can define custom getters and utilities that simplify writing evaluators.--## Basic Usage--```python[email protected]_task(split="train")-async def start(task_cfg: cb.Task, session: cb.DesktopSession):- await session.apps.chrome.install(with_shortcut=True)- await session.apps.chrome.launch(url="https://example.com")-[email protected]_task(split="train")-async def evaluate(task_cfg: cb.Task, session: cb.DesktopSession) -> list[float]:- # Use app-specific getters for evaluation- current_url = await session.apps.chrome.get_current_url()- tabs = await session.apps.chrome.get_open_tabs()- bookmarks = await session.apps.chrome.get_bookmarks()-- if "google.com" in current_url and len(tabs) >= 3:- return [1.0]- return [0.0]-```--## Available Apps--| App | Name | Platforms |-| --------------- | ----------------- | --------------------- |-| Godot | `godot` | Linux, Windows, macOS |-| Unity | `unity` | Linux, Windows, macOS |-| Adobe Photoshop | `adobe_photoshop` | Linux (WINE), Windows |-| Notes | `notes` | macOS |-| Reminders | `reminders` | macOS |-| Calendar | `calendar` | macOS |--## Contributing Apps--Apps live in `cua_bench/apps/`. Each app can define:--- **install** - Install the application-- **launch** - Launch with options-- **uninstall** - Remove the application-- **Custom methods** - Getters/utilities for evaluators--```python-from .registry import App, install, launch--class Chrome(App):- name = "chrome"- description = "Google Chrome browser"-- @install("linux")- async def install_linux(self, *, with_shortcut=True):- await self.session.run_command("sudo apt install -y google-chrome-stable")-- @launch("linux", "windows")- async def launch_app(self, *, url=None):- cmd = "google-chrome &" if self.platform == "linux" else "start chrome.exe"- if url:- cmd = f"{cmd} {url}"- await self.session.run_command(cmd)-- # Custom getters for evaluators- async def get_current_url(self) -> str:- """Get Chrome's current URL."""- result = await self.session.run_command("xdotool getactivewindow getwindowname")- return result["stdout"].strip()-- async def get_open_tabs(self) -> list[str]:- """Get list of open tab titles."""- ...-- async def get_bookmarks(self) -> list[dict]:- """Get user's bookmarks."""- ...-```--Register in `cua_bench/apps/__init__.py`:--```python-from . import chrome # noqa: F401-```--## Platform Matrix--| Method | Linux | Windows | macOS |-| ----------- | ----- | ------- | ----- |-| `install` | ✅ | ✅ | ✅ |-| `launch` | ✅ | ✅ | ✅ |-| `uninstall` | ✅ | ⚠️ | ⚠️ |--✅ = Supported, ⚠️ = Partial supportdiff --git a/docs/content/docs/cuabench/guide/fundamentals/meta.json b/docs/content/docs/cuabench/guide/fundamentals/meta.jsondeleted file mode 100644index c675290a14..0000000000--- a/docs/content/docs/cuabench/guide/fundamentals/meta.json+++ /dev/null@@ -1,14 +0,0 @@-{- "title": "Fundamentals",- "description": "Core concepts of tasks and environments",- "icon": "Lightbulb",- "pages": [- "tasks",- "app-helpers",- "universal-gui",- "simulated-desktop",- "agent-traces",- "adapters",- "registry"- ]-}diff --git a/docs/content/docs/cuabench/guide/fundamentals/registry.mdx b/docs/content/docs/cuabench/guide/fundamentals/registry.mdxdeleted file mode 100644index b0e55ae2c3..0000000000--- a/docs/content/docs/cuabench/guide/fundamentals/registry.mdx+++ /dev/null@@ -1,104 +0,0 @@-----title: Registry-description: The Cua-Bench registry of tasks and datasets.------The [Cua-Bench registry](https://cua.ai/docs/cuabench/guide/fundamentals/registry) contains a collection of pre-built tasks and datasets for evaluating computer-use agents.--Datasets are now part of the main [cua repository](https://github.com/trycua/cua) at [`libs/cua-bench/datasets`](https://github.com/trycua/cua/tree/main/libs/cua-bench/datasets).--The goal of the registry is to be a community-based effort to increase the variety and quantity of useful, verifiable computer-use tasks.--## Using the Registry--### List Available Datasets--```bash-cb dataset list-```--### Run a Dataset from the Registry--```bash-# Run with agent-cb run dataset datasets/cua-bench-basic --agent cua-agent --model anthropic/claude-sonnet-4-20250514--# Run interactively-cb interact click-button --dataset cua-bench-basic--# Run oracle solutions-cb run dataset datasets/cua-bench-basic --oracle-```--### Clone Registry Locally--The registry is automatically cloned when you first use the `--dataset` flag:--```bash-# Registry location-~/.cua/cua/libs/cua-bench/datasets/--# Update datasets-cd ~/.cua/cua && git pull-```--Or set a custom location:--```bash-export CUA_REGISTRY_HOME=/path/to/your/datasets-```--## Available Datasets--| Dataset | Tasks | Description | Golden Environment |-| ----------------- | ----- | ------------------------------------------------------- | ------------------ |-| `cua-bench-basic` | 13 | Basic UI interaction tasks (clicking, typing, dragging) | `simulated` |--## Dataset Structure--Each dataset in the registry follows this structure:--```-libs/cua-bench/datasets/-└── cua-bench-basic/- ├── meta.json # Dataset metadata- └── tasks/- ├── click-button/- │ ├── main.py # Task implementation- │ └── gui/- │ └── index.html- └── ...-```--### meta.json Format--```json-{- "name": "cua-bench-basic",- "version": "1.0.0",- "description": "Basic UI interaction tasks",- "platform": "simulated",- "tasks": ["click-button", "typing-input", "drag-slider"]-}-```--## Contributing Tasks--To contribute tasks to the registry:--1. Fork the [cua repository](https://github.com/trycua/cua)-2. Create your task in `libs/cua-bench/datasets/<dataset-name>/tasks/`-3. Follow the [task structure](/cuabench/guide/fundamentals/tasks)-4. Test locally with `cb interact tasks/your_task`-5. Submit a pull request to the main repository--See [Creating Tasks](/cuabench/examples/universal-task) for task development guidance.--## Registry Location--By default, datasets are stored at:--- **Default**: `~/.cua/cua/libs/cua-bench/datasets/`-- **Custom**: Set `CUA_REGISTRY_HOME` environment variable--The registry is a shallow clone of the main cua repository, pulling only the datasets directory.diff --git a/docs/content/docs/cuabench/guide/fundamentals/simulated-desktop.mdx b/docs/content/docs/cuabench/guide/fundamentals/simulated-desktop.mdxdeleted file mode 100644index 1f705cefec..0000000000--- a/docs/content/docs/cuabench/guide/fundamentals/simulated-desktop.mdx+++ /dev/null@@ -1,90 +0,0 @@-----title: Simulated Desktop (Preview)-description: Preview feature for testing tasks with themed desktop environments (no Docker required)------<Callout type="info">This is a **preview feature** for rapid development and testing.</Callout>--When you set `provider="simulated"` in your task config, you can preview it in a lightweight HTML/CSS desktop environment without Docker. This renders OS-themed window chrome, taskbars, and desktop styling around your GUI.--## Available Themes--| Theme | Style |-| ------- | ---------------------------------------------------- |-| `win11` | Windows 11 with rounded corners and centered taskbar |-| `macos` | macOS with top menu bar and dock |--## Usage--Set `provider: "simulated"` and choose a theme:--```python-import cua_bench as cb-[email protected]_config(split="train")-def load():- return [- cb.Task(- description='Click the Submit button',- computer={- "provider": "simulated", # Enables simulated desktop- "setup_config": {- "os_type": "win11", # win11 or macos- "width": 1920,- "height": 1080,- }- }- )- ]-```--Then preview:--```bash-cb interact tasks/my_task --variant-id 0-```--## Multiple Theme Variants--Generate tasks across different desktop themes:--```python[email protected]_config(split="train")-def load():- themes = ["win11", "macos"]-- return [- cb.Task(- description=f'Complete the form',- computer={- "provider": "simulated",- "setup_config": {- "os_type": theme,- "width": 1280,- "height": 800,- }- }- )- for theme in themes- ]-```--## Limitations--Simulated desktop runs in a browser, so it has some restrictions:--- **No shell actions**: Can't execute bash commands or scripts-- **No native apps**: Can't install or run real applications-- **HTML/CSS only**: Only works with custom GUIs created via `launch_window()` or `install_app()`--For tasks requiring shell access or native applications, use Docker or QEMU environments instead.--## Why Use Simulated Desktop?--- **No Docker**: Runs directly in browser with Playwright-- **Fast**: Instant startup, no VM boot time-- **Visual Diversity**: Train agents on different OS appearances-- **Deterministic**: Same theme always renders identically-- **Lightweight**: Run thousands in parallel--Use simulated desktop for quick iteration and agent development. Switch to Docker/QEMU when you need real OS environments.diff --git a/docs/content/docs/cuabench/guide/fundamentals/tasks.mdx b/docs/content/docs/cuabench/guide/fundamentals/tasks.mdxdeleted file mode 100644index 095f83e618..0000000000--- a/docs/content/docs/cuabench/guide/fundamentals/tasks.mdx+++ /dev/null@@ -1,222 +0,0 @@-----title: Tasks-description: An in-depth guide describing the structure of a Cua-Bench task.------A Cua-Bench task is a Python module that defines a verifiable, cross-platform GUI task. Tasks use decorators to define configuration, setup, evaluation, and oracle solutions.--## Task Structure--Every task consists of a `main.py` file with four key components:--### 1. Task Configuration (`@cb.tasks_config`)--Defines the list of task variants to generate:--```python-import cua_bench as cb-[email protected]_config(split="train")-def load():- return [- cb.Task(- description='Click the "Submit" button on the page.',- metadata={- "button_text": "Submit",- },- # Configure the computer platform- computer={- "provider": provider,- "setup_config": {- "os_type": os_type,- "width": 1024,- "height": 768- }- }- )- for provider, os_type in [("native", "linux"), ("native", "windows"), ("simulated", "win11"), ("simulated", "macos")]- # More task variants...- ]-```--Each task variant can have different metadata, enabling thousands of unique scenarios from a single template.--The `native` provider **emulates** `windows` and `linux` operating systems using QEMU/Docker, while the `simulated` provider lightweight Playwright-based desktop environments with themes such as `win11` and `macos`. See [Simulated Desktop](/cuabench/guide/fundamentals/simulated-desktop) for all available themes.--### 2. Setup (`@cb.setup_task`)--Initializes the environment for a specific task, installing any necessary apps or web UI onto the desktop:--```python-# Each (setup_task, evaluate_task, and solve_task) function runs in 1 thread/context per task_cfg-# All task functions must be async-pid = None-[email protected]_task(split="train")-async def start(task_cfg: cb.Task, session: cb.DesktopSession | cb.MobileSession):- global pid- # Launch a window with HTML content- pid = await session.launch_window(- html=Path("gui/index.html").read_text(),- title="My App",- width=800,- height=600- )-```--<Callout title="Async Required" type="warn">- All task functions (setup, evaluate, solve) must be `async` functions, and session methods must be- called with `await`.-</Callout>--### 3. Evaluation (`@cb.evaluate_task`)--Checks if the task was completed successfully by observing the internal state of the apps:--```python[email protected]_task(split="train")-async def evaluate(task_cfg: cb.Task, session: cb.DesktopSession | cb.MobileSession) -> list[float]:- global pid- if pid is None:- return [0.0]-- # Check task completion- submitted = await session.execute_javascript(pid, "window.__submitted")-- # Return reward: 1.0 for success, 0.0 for failure- return [1.0] if submitted is True else [0.0]-```--### 4. Oracle Solution (`@cb.solve_task`)--Provides a reference solution for trajectory generation:--```python[email protected]_task(split="train")-async def solve(task_cfg: cb.Task, session: cb.DesktopSession | cb.MobileSession):- global pid- if pid is None:- return-- # Execute the solution- await session.click_element(pid, ".submit-button")-```--## Environment API--### Window Management--Cua-Bench provides a Python API for launching webviews and installing web apps, which makes it simple to add custom GUI applications to your tasks.--```python-# Launch a webview window (async)-pid = await session.launch_window(- html="<html>...</html>",- title="Window Title",- width=800,- height=600-)--# Execute JavaScript (async)-result = await session.execute_javascript(pid, "document.title")--# Install a webapp shortcut-await session.install_app("Slack Clone", html="<html>...</html>")-```--### Playwright-style Actions--Cua-Bench also provides a Playwright-like API, making it easy to script oracle solutions:--```python-# Click an element by CSS selector (async)-await session.click_element(pid, ".button")-await session.click_element(pid, "#submit-btn")--# Type text-await session.type_text("Hello world")--# Press keys-await session.key("Enter")-await session.key("Control+c")-```--### Low-level Actions--All actions from the agent or the oracle solution are performed via low-level keyboard and mouse actions. You can use `execute_action` to execute these actions directly:--```python-# Perform low-level actions (async)-await session.execute_action(cb.ClickAction(x=64, y=128))-await session.execute_action(cb.MoveToAction(x=16, y=100, duration=0.5))-await session.execute_action(cb.KeyAction(key="Enter"))--# Take screenshot-screenshot = await session.screenshot()-```--### Shell Actions--If your task setup requires installing binaries or packages, then you can use `session.run_command` to execute shell commands in the computer environment.--```python-# Install firefox (async)-await session.run_command("sudo apt update && sudo apt install firefox")-```--<Callout title="Compatibility Warning" type="warn">- This feature is only supported by `VMDesktopSession` and not `WebDesktopSession`. Avoid using this- API for writing tasks unless you cannot create your task with the other provided environment APIs.-</Callout>--## Structuring your task folders--All tasks must have a top-level `main.py` file. We recommend also including a `gui/` directory for any HTML/CSS/JS files:--```-tasks/my_task/-├── main.py-└── gui/- ├── index.html- ├── style.css- └── script.js-```--To reference these files in your setup function:--```python-folder_path = Path(__file__).parent / "gui"-pid = env.launch_window(folder=str(folder_path), title="My GUI", ...)-```--If you use a web framework such as React or Svelte, you can serve the `dist/` folder, or you can start a development server and launch a window pointing to it:--```python[email protected]_task(split="train")-def start(task_cfg: cb.Task, env: cb.DesktopSession | cb.MobileSession):- folder_path = Path(__file__).parent / "gui"-- # Start framework app in the background- _server_process = subprocess.Popen(["npm", "run", "dev"], cwd=folder_path)-- # Launch window pointing to the local server- pid = env.launch_window(- url="http://localhost:3000",- title="My React GUI",- width=800,- height=600- )-```--## Running Tasks--```bash-# Run interactively-cb interact tasks/my_task --variant-id 0--# Run evaluations-cb run task tasks/my_task --model anthropic/claude-opus-4-5 --agent cua-agent--# Generate oracle trajectories-cb run task tasks/my_task --oracle-```diff --git a/docs/content/docs/cuabench/guide/fundamentals/universal-gui.mdx b/docs/content/docs/cuabench/guide/fundamentals/universal-gui.mdxdeleted file mode 100644index 49b042979e..0000000000--- a/docs/content/docs/cuabench/guide/fundamentals/universal-gui.mdx+++ /dev/null@@ -1,76 +0,0 @@-----title: Universal GUI-description: Package custom GUIs with your tasks using the window management API------Cua-Bench provides an Electron-like API for packaging custom HTML/CSS/JS GUIs with your tasks. This allows you to create interactive GUI apps, GUI-based tools, or simulated clones of real applications when developing tasks.--## Window Management API--Use `session.launch_window()` to create GUI windows in any task:--```python-import cua_bench as cb-from pathlib import Path-[email protected]_task(split="train")-async def start(task_cfg: cb.Task, session: cb.DesktopSession):- # Launch a window with your HTML GUI- pid = await session.launch_window(- html=(Path(__file__).parent / "gui/index.html").read_text(),- title="My App",- width=800,- height=600,- )-- # Initialize your app with JavaScript- await session.execute_javascript(pid, "window.initApp()")-```--## When to Use--- **Custom GUI apps**: Build interactive applications (games, tools, dashboards)-- **Simulated environments**: Create clones of real apps for testing-- **GUI-based benchmarks**: Test agent capabilities with visual interfaces-- **Rapid prototyping**: Quickly iterate on task ideas with HTML/CSS--## Structure--```-my_task/-├── main.py # Task with launch_window() calls-└── gui/- └── index.html # Your HTML/CSS/JS GUI-```--## Previewing with Simulated Desktop--For lightweight previewing without Docker, set `provider: "simulated"` in your task config:--```python-computer={- "provider": "simulated", # Enables simulated desktop preview- "setup_config": {- "os_type": "win11", # win11 or macos theme- "width": 800,- "height": 800,- }-}-```--Then preview with:--```bash-cb interact tasks/my_game --variant-id 0-```--The [Simulated Desktop](/cuabench/guide/fundamentals/simulated-desktop) renders your GUI in a themed desktop environment (Linux/Windows/macOS) without requiring Docker.--## Key Features--- **JavaScript execution**: Communicate with your GUI via `execute_javascript()`-- **CSS selectors**: Click elements with `click_element(pid, "#button")`-- **State management**: Expose game/app state via `window.__variables`-- **Cross-platform**: Works in Docker containers and simulated desktop--See the [Create a Universal Task](/cuabench/examples/universal-task) example for a complete walkthrough.diff --git a/docs/content/docs/cuabench/guide/getting-started/first-steps.mdx b/docs/content/docs/cuabench/guide/getting-started/first-steps.mdxdeleted file mode 100644index 046ca5d7fe..0000000000--- a/docs/content/docs/cuabench/guide/getting-started/first-steps.mdx+++ /dev/null@@ -1,224 +0,0 @@-----title: First Steps-description: Running the benchmark and testing your own agent.------## Prerequisites--Before running benchmarks, ensure you have:--1. [Installed the CLI](/cuabench/guide/getting-started/installation)-2. Created at least one [base image](/cuabench/guide/advanced/images):--```bash-# Quick setup (Linux container, no KVM required)-cb image create linux-docker--# Check available images-cb image list-```--## Example 1: Run a Single Task--Learn the basic workflow by running a single task end-to-end.--### Step 1: Explore Interactively--Start by exploring the task interactively in a browser window to see how it works:--```bash-cb interact tasks/slack_env --variant-id 0-```--This opens a visible browser where you can manually interact with the task to understand what needs to be done.--### Step 2: Run the Task with an Agent--Now run the same task with an AI agent. Tasks run **asynchronously by default** and return immediately with a run ID:--```bash-export ANTHROPIC_API_KEY=sk-....--# Run a single task (async - returns immediately)-cb run task tasks/slack_env --agent cua-agent --model anthropic/claude-sonnet-4-20250514-```--You'll see output like:--```-✓ Run started-- Run ID: 96d41b51- Task: click_button (variant 0)- Env: simulated (Playwright)- Output: ~/.local/share/cua-bench/runs/96d41b51- Session: ~/.local/share/cua-bench/runs/96d41b51/click_button_v0--Commands:- cb run watch 96d41b51 # Watch progress in real-time- cb run info 96d41b51 # Show run details- cb run logs 96d41b51 # View logs- cb run stop 96d41b51 # Stop the run-```--### Step 3: Watch Progress--Monitor the agent's progress in real-time:--```bash-cb run watch 96d41b51-```--You'll see live updates:--```-████████████████████ 1/1-Model: anthropic/claude-sonnet-4-20250514-Agent: cua-agent-Run ID: 96d41b51--SESSION ID ENVIRONMENT VARIANT STATUS REWARD----------------- ------------ ------- ----------- -------task-96d41b51 slack_env 0 ✓ completed 1.0--✓ All sessions completed!-Average Reward: 1.000-```--### Step 4: View the Trace--Finally, explore the agent's trajectory using the trace viewer:--```bash-cb trace view 96d41b51-```--This launches an interactive viewer where you can see screenshots, actions, and debug the agent's behavior:--```-Serving trace viewer at: http://127.0.0.1:55115/-Press Enter to stop...-```--Open the URL in your browser to explore the full execution trace.-------## Example 2: Run a Basic Benchmark--Scale up to run an entire benchmark dataset with multiple tasks.--The Cua-Bench CLI can run many different computer-use benchmarks and versions of benchmarks. To see available datasets, run `cb dataset list` or visit the [registry](https://cua.ai/docs/cuabench/guide/fundamentals/registry).--### Step 1: Explore a Single Task--Start by exploring one task from the dataset interactively:--```bash-cb interact click-icon --dataset cua-bench-basic-```--### Step 2: Validate with Oracle Solutions--Verify your environment setup by running the oracle solutions (reference implementations) for all tasks:--```bash-cb run dataset datasets/cua-bench-basic --oracle-```--Oracle solutions should achieve 100% success rate. If they fail, check your environment setup.--### Step 3: Run the Entire Dataset in Parallel--Now run the full benchmark with an agent. Use `--max-parallel` to run multiple tasks simultaneously:--```bash-export ANTHROPIC_API_KEY=sk-....--# Run entire dataset in parallel-cb run dataset datasets/cua-bench-basic \- --agent cua-agent \- --model anthropic/claude-haiku-4-5 \- --max-parallel 8 \- --max-steps 10 \- --max-variants 1-```--Monitor progress with:--```bash-cb run watch <run_id>-```--You should see output similar to:--```-- ⠀⣀⣀⡀⠀⠀⠀⠀⢀⣀⣀⣀⡀⠘⠋⢉⠙⣷⠀⠀ ⠀- ⠀⠀⢀⣴⣿⡿⠋⣉⠁⣠⣾⣿⣿⣿⣿⡿⠿⣦⡈⠀⣿⡇⠃⠀- ⠀⠀⠀⣽⣿⣧⠀⠃⢰⣿⣿⡏⠙⣿⠿⢧⣀⣼⣷⠀⡿⠃⠀⠀- ⠀⠀⠀⠉⣿⣿⣦⠀⢿⣿⣿⣷⣾⡏⠀⠀⢹⣿⣿⠀⠀⠀⠀⠀⠀- ⠀⠀⠀⠀⠀⠉⠛⠁⠈⠿⣿⣿⣿⣷⣄⣠⡼⠟⠁⠀cua-bench==v0.1.0- toolkit for computer-use RL environments and benchmarks--████████████████████ 13/13-Model: anthropic/claude-haiku-4-5-Agent: cua-agent-Run ID: run-2768db7f----SESSION ID ENVIRONMENT VARIANT STATUS REWARD----------------------------------------- --------------- ------- --------------- -----------task-baecb22e typing-input 0 ✗ failed 0.0-task-05a6e068 video-player 0 ✓ completed 1.0-task-a8faeb43 spreadsheet-cell 0 ✗ failed 0.0-task-5a5f30c5 toggle-switch 0 ✓ completed 1.0-task-3555ee20 select-dropdown 0 ✗ failed 0.0-task-e9560151 click-button 0 ✓ completed 1.0-task-7ec326f2 fill-form 0 ✗ failed 0.0-task-57dae23d date-picker 0 ✗ failed 0.0-task-01276975 drag-slider 0 ✗ failed 0.0-task-4b875cd2 color-picker 0 ✓ completed 1.0-task-c933d5b0 drag-drop 0 ✗ failed 0.0-task-ec44d643 right-click-menu 0 ✗ failed 0.0-task-7c4acb4c click-icon 0 ✓ completed 1.0----✓ All sessions completed!-Average Reward: 0.385--REWARD COUNT---------------1.0 5-0.0 8-```--Results are saved to `~/.local/share/cua-bench/runs/<run_id>/` by default.-------## Managing Runs--View and manage your past runs:--```bash-# List all runs with statistics-cb run list--# Watch a run in real-time-cb run watch <run_id>--# Check status of a specific run-cb run info <run_id>--# View logs-cb run logs <run_id>--# Stop a running task-cb run stop <run_id>--# View traces from a specific run-cb trace view <run_id>-```diff --git a/docs/content/docs/cuabench/guide/getting-started/installation.mdx b/docs/content/docs/cuabench/guide/getting-started/installation.mdxdeleted file mode 100644index 06a0939625..0000000000--- a/docs/content/docs/cuabench/guide/getting-started/installation.mdx+++ /dev/null@@ -1,92 +0,0 @@-----title: Installation-description: Getting started with Cua-Bench------Cua-Bench provides a CLI for running the benchmark, creating tasks, and generating training data.--## Install Dependencies--Cua-Bench requires `git` and `Docker` to be installed on your system.--## Install the CLI--Install Cua-Bench using `uv`:--```bash-# uv tool install cua-bench--# Clone the repo-git clone https://github.com/trycua/cua.git-cd cua/libs/cua-bench--# Install cua-bench CLI-uv tool install -e .[all]--# Install playwright-uv run python -m playwright install chromium--# Build cua-bench Docker image-docker build -t cua-bench:latest .-```--After installation, you can run the CLI using `cua-bench` or `cb` for convenience.--## Install a default Base Image--Base images are reproducible computer environments for running tasks. Create at least one:--```bash-# Quick setup (Linux container, no KVM required)-cb image create linux-docker--# Or for Windows Arena benchmarks (~1-2 hours)-cb image create windows-qemu --download-iso-```--| Type | Description | Requirements |-| -------------- | -------------------------- | ------------- |-| `linux-docker` | Linux GUI container (XFCE) | Docker |-| `linux-qemu` | Linux VM with QEMU/KVM | Docker + KVM |-| `windows-qemu` | Windows 11 VM | Docker + KVM |-| `android-qemu` | Android VM | Docker + KVM |-| `macos-lume` | macOS VM via Lume | Apple Silicon |--See [Base Images](/cuabench/guide/advanced/images) for more options.--## Verify Installation--Run the help command to verify the installation:--```bash-cb --help-```--You should see output similar to:--```- ⠀⣀⣀⡀⠀⠀⠀⠀⢀⣀⣀⣀⡀⠘⠋⢉⠙⣷⠀⠀ ⠀- ⠀⠀⢀⣴⣿⡿⠋⣉⠁⣠⣾⣿⣿⣿⣿⡿⠿⣦⡈⠀⣿⡇⠃⠀- ⠀⠀⠀⣽⣿⣧⠀⠃⢰⣿⣿⡏⠙⣿⠿⢧⣀⣼⣷⠀⡿⠃⠀⠀- ⠀⠀⠀⠉⣿⣿⣦⠀⢿⣿⣿⣷⣾⡏⠀⠀⢹⣿⣿⠀⠀⠀⠀⠀⠀- ⠀⠀⠀⠀⠀⠉⠛⠁⠈⠿⣿⣿⣿⣷⣄⣠⡼⠟⠁⠀cua-bench==v0.4.0- toolkit for computer-use RL environments and benchmarks--Commands:- Available commands- run Run tasks with optional agent evaluation- interact Interactively run a task with browser visible- agent Agent management commands- platform Show available platform configurations- image Manage stored base images- status Show system status dashboard- task Inspect and manage task environments- trace View and manage trace datasets- dataset Manage datasets and build from outputs- prune Clean up cua-bench data, images, and Docker resources- login Authenticate with Cua Cloud to get API token--Options:- -h, --help Show this help message and exit- --version Show version-```diff --git a/docs/content/docs/cuabench/guide/getting-started/introduction.mdx b/docs/content/docs/cuabench/guide/getting-started/introduction.mdxdeleted file mode 100644index 3059692846..0000000000--- a/docs/content/docs/cuabench/guide/getting-started/introduction.mdx+++ /dev/null@@ -1,63 +0,0 @@-----title: Introduction-description: A benchmark to measure the capabilities of computer-use agents in desktop environments.------<Callout type="info">- Building or researching computer-use agents? Fill out the [Interest Form](https://cuabench.ai/) to- chat with us.-</Callout>--**Cua-Bench** is a framework and set of tasks for evaluating how well AI agents can accomplish complex tasks on a desktop computer using primarily the keyboard and mouse, or on a mobile device using primarily the touchscreen.--<div className="not-prose relative rounded-xl overflow-hidden my-8 w-full">- <img- src="/docs/img/cua_bench_screenshots.png"- alt="Cua"- className="w-full h-auto rounded-xl border"- />-</div>--Examples of Cua-Bench tasks are:--- Navigating a web browser to book travel reservations-- Installing design software and editing images to remove backgrounds-- Using office applications to format and merge multiple documents--Cua-Bench consists of three parts:--<div className="not-prose relative rounded-xl overflow-hidden my-8 w-full">- <img- src="/docs/img/cua-bench-architecture.png"- alt="Cua-Bench Architecture"- className="w-full h-auto rounded-xl"- />-</div>--1. **Base images** - Pre-packaged Windows, Linux, macOS, and Android environments with apps and dependencies needed for benchmark runs-2. **A dataset of tasks** - Verifiable, dynamic computer-use environments across multiple operating systems-3. **An evaluation/training harness** - Tools for running benchmarks, generating data, and testing agents--# Dataset of tasks--Each task in Cua-Bench includes:--- a description in English-- a config for the desktop/mobile environment-- a setup script to install any HTML webapps-- a test script to verify if the agent completed the task successfully-- a reference ("oracle") solution that solves the task--Check out our existing tasks [here](https://cua.ai/docs/cuabench/guide/fundamentals/registry)--# An environment harness--The harness provides familiar APIs for creating verifiable tasks. Use an Electron-like Python API to install web apps onto desktop/mobile environments, then use a Playwright-like API to write automated solutions or reward test scripts.--After [installing the package](/cuabench/guide/getting-started/installation), you can run the harness using `cb run` or `cb interact`.--# Next Steps--1. [Install the CLI](/cuabench/guide/getting-started/installation)-2. [Run your first benchmark](/cuabench/guide/getting-started/first-steps)-3. [Create custom tasks](/cuabench/guide/fundamentals/tasks)diff --git a/docs/content/docs/cuabench/guide/getting-started/meta.json b/docs/content/docs/cuabench/guide/getting-started/meta.jsondeleted file mode 100644index 651815461f..0000000000--- a/docs/content/docs/cuabench/guide/getting-started/meta.json+++ /dev/null@@ -1,7 +0,0 @@-{- "title": "Getting Started",- "description": "Get started with Cua-Bench",- "icon": "Rocket",- "defaultOpen": true,- "pages": ["introduction", "installation", "first-steps"]-}diff --git a/docs/content/docs/cuabench/guide/meta.json b/docs/content/docs/cuabench/guide/meta.jsondeleted file mode 100644index bc617a2497..0000000000--- a/docs/content/docs/cuabench/guide/meta.json+++ /dev/null@@ -1,6 +0,0 @@-{- "title": "Guide",- "description": "Learn how to use Cua-Bench",- "icon": "Book",- "pages": ["getting-started", "fundamentals", "advanced"]-}diff --git a/docs/content/docs/cuabench/index.mdx b/docs/content/docs/cuabench/index.mdxdeleted file mode 100644index f1518e8bfc..0000000000--- a/docs/content/docs/cuabench/index.mdx+++ /dev/null@@ -1,87 +0,0 @@-----title: Cua Bench-description: Framework for benchmarking and training computer-use agents------# Cua Bench--A toolkit for verifiable cross-platform computer-use RL environments and benchmarks. Supports real Windows, Linux, macOS, and Android VM environments with reproducible base images.--## Quick Start--```bash-# Install-git clone https://github.com/trycua/cua-bench.git && cd cua-bench-uv tool install -e .--# Create base image-cb image create linux-docker--# Run interactive task-cb interact click-icon --dataset cua-bench-basic--# Run single task with agent-cb run task tasks/click_button --agent cua-agent --model anthropic/claude-sonnet-4-20250514--# Run entire dataset in parallel-cb run dataset datasets/cua-bench-basic --agent cua-agent --max-parallel 4-```--## Key Features--- **Base Images** - Reproducible VM images for Windows, Linux, macOS, and Android-- **Task Authoring** - Python API with async decorators for setup, evaluation, and oracle solutions-- **Custom Agents** - Docker image agents for production, import path agents for development-- **2-Container Architecture** - Agent and environment containers communicate via Docker network-- **Benchmark Execution** - Run evaluations locally or at scale on cloud-- **Training Data** - Export trajectories to datasets for UI grounding models--## Architecture--Cua-Bench uses a **2-container architecture** where agent and environment run in isolated containers:--```-┌─────────────────────────────────────────────────────────────┐-│ Docker Network: cua-task-{id} │-│ │-│ ┌─────────────────────┐ ┌─────────────────────────────┐│-│ │ cua-agent-{id} │ │ cua-env-{id} ││-│ │ │ │ ││-│ │ Your agent image │───►│ Base image environment ││-│ │ (or built-in) │HTTP│ (linux/windows/android) ││-│ │ │ │ ││-│ └─────────────────────┘ └─────────────────────────────┘│-└─────────────────────────────────────────────────────────────┘-```--This architecture is consistent between local development and cloud deployments.--## Next Steps--<Cards>- <Card- title="Introduction"- href="/cuabench/guide/getting-started/introduction"- description="Learn about Cua-Bench concepts"- />- <Card- title="Installation"- href="/cuabench/guide/getting-started/installation"- description="Install the CLI and dependencies"- />- <Card- title="Base Images"- href="/cuabench/guide/advanced/images"- description="Set up VM environments"- />- <Card- title="First Steps"- href="/cuabench/guide/getting-started/first-steps"- description="Run your first benchmark"- />- <Card- title="Custom Agents"- href="/cuabench/guide/advanced/custom-agents"- description="Build and run custom agents"- />-</Cards>diff --git a/docs/content/docs/cuabench/meta.json b/docs/content/docs/cuabench/meta.jsondeleted file mode 100644index 0be5332970..0000000000--- a/docs/content/docs/cuabench/meta.json+++ /dev/null@@ -1,5 +0,0 @@-{- "title": "Cua-Bench",- "description": "Framework for benchmarking and training computer-use agents",- "pages": ["guide", "examples", "reference"]-}diff --git a/docs/content/docs/cuabench/reference/api.mdx b/docs/content/docs/cuabench/reference/api.mdxdeleted file mode 100644index 0814b9caed..0000000000--- a/docs/content/docs/cuabench/reference/api.mdx+++ /dev/null@@ -1,5157 +0,0 @@-----title: API Reference-description: Python API reference for the desktop automation benchmarking framework------{/*- AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY- Generated by: npx tsx scripts/docs-generators/python-sdk.ts- Source: libs/cua-bench/cua_bench- Version: 0.2.6-*/}--import { Callout } from 'fumadocs-ui/components/callout';-import { Tabs, Tab } from 'fumadocs-ui/components/tabs';-import { VersionHeader } from '@/components/version-selector';--<VersionHeader- versions={[{"version":"0.2","href":"/cuabench/reference","isCurrent":true}]}- currentVersion="0.2"- fullVersion="0.2.6"- packageName="cua-bench"-/>--cua-bench SDK - A framework for desktop automation tasks with batch processing.--## Classes--| Class | Description |-|-------|-------------|-| [`Task`](#task) | Represents a single task to be executed. |-| [`Desktop`](#desktop) | Desktop environment manager. |-| [`Environment`](#environment) | A minimal environment wrapper that delegates everything to a provider. |-| [`BenchmarkResult`](#benchmarkresult) | Result of a benchmark run. |-| [`TaskResult`](#taskresult) | Result of a single task execution. |-| [`ClickAction`](#clickaction) | No description |-| [`DoneAction`](#doneaction) | No description |-| [`DoubleClickAction`](#doubleclickaction) | No description |-| [`DragAction`](#dragaction) | No description |-| [`HotkeyAction`](#hotkeyaction) | No description |-| [`KeyAction`](#keyaction) | No description |-| [`MiddleClickAction`](#middleclickaction) | No description |-| [`MoveToAction`](#movetoaction) | No description |-| [`RightClickAction`](#rightclickaction) | No description |-| [`ScrollAction`](#scrollaction) | No description |-| [`TypeAction`](#typeaction) | No description |-| [`WaitAction`](#waitaction) | No description |--## Functions--| Function | Description |-|----------|-------------|-| [`repr_to_action`](#repr_to_action) | Parse an action from repr format string. |-| [`interact`](#interact) | Run an environment interactively with simplified output. |-| [`make`](#make) | Create an Environment by loading the env's main.py as a module. |-| [`evaluate_task`](#evaluate_task) | Decorator for the function that evaluates a task. |-| [`setup_task`](#setup_task) | Decorator for the function that sets up a task. |-| [`solve_task`](#solve_task) | Decorator for the function that solves a task. |-| [`tasks_config`](#tasks_config) | Decorator for the function that loads tasks. |-| [`run_benchmark`](#run_benchmark) | Run a benchmark on a dataset using the gym interface. |-| [`run_interactive`](#run_interactive) | Run an environment interactively using the gym interface. |-| [`run_single_task`](#run_single_task) | Run a single task using the gym interface. |-------## Task--Represents a single task to be executed.--### Constructor--```python-Task(self, description: str, task_id: Optional[str] = None, metadata: Optional[dict] = None, computer: Optional[dict] = None) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `description` | `str` | |-| `task_id` | `Optional[str]` | |-| `metadata` | `Optional[dict]` | |-| `computer` | `Optional[dict]` | |-------## Desktop--Desktop environment manager.--### Constructor--```python-Desktop(self, env)-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `env` | `Any` | |-| `state` | `Any` | |-| `template` | `Any` | |--### Methods--#### Desktop.configure--```python-def configure(self, os_type: Optional[str] = None, width: Optional[int] = None, height: Optional[int] = None, background: Optional[str] = None, dock_state: Optional[Dict[str, List[Union[str, Dict[str, str]]]]] = None, randomize_dock: bool = True, taskbar_state: Optional[Dict[str, List[Union[str, Dict[str, str]]]]] = None, randomize_taskbar: bool = True)-```--Configure desktop appearance.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `os_type` | `Any` | OS appearance (win11, win10, win7, macos, winxp, win98, android, ios) |-| `width` | `Any` | Screen width in pixels |-| `height` | `Any` | Screen height in pixels |-| `background` | `Any` | Background color |-| `dock_state` | `Any` | Explicit dock state to set with keys 'pinned_apps', 'recent_apps', 'pinned_folders' |-| `randomize_dock` | `Any` | If True, populate dock_state using macOS icon sets |-| `taskbar_state` | `Any` | Explicit taskbar state to set with keys 'pinned_apps', 'open_apps' |-| `randomize_taskbar` | `Any` | If True, populate taskbar_state using Windows 11 icon sets |--#### Desktop.launch--```python-def launch(self, content: str, title: str = 'Window', x: Optional[int] = None, y: Optional[int] = None, width: int = 600, height: int = 400, icon: Optional[str] = None, use_inner_size: bool = False, title_bar_style: str = 'default') -> Window-```--Launch a new window on the desktop.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `content` | `Any` | HTML content for the window body |-| `title` | `Any` | Window title |-| `x` | `Any` | X position (auto-calculated if None) |-| `y` | `Any` | Y position (auto-calculated if None) |-| `width` | `Any` | Window width |-| `height` | `Any` | Window height |-| `use_inner_size` | `Any` | Whether to use the inner size of the window (i.e. content size) |--**Returns:** Window instance-------## Environment--A minimal environment wrapper that delegates everything to a provider.--Functions can be injected directly, or discovered from a module via-`make_from_module` based on cua-bench decorators (`_td_type`, `_td_split`).--### Constructor--```python-Environment(self, env_name: Optional[str] = None, split: str = 'train', tasks_config_fn: Optional[Callable[..., Any]] = None, setup_task_fn: Optional[Callable[..., Any]] = None, solve_task_fn: Optional[Callable[..., Any]] = None, evaluate_task_fn: Optional[Callable[..., Any]] = None) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `session` | `Optional[Any]` | |-| `env_name` | `Optional[str]` | |-| `split` | `Optional[str]` | |-| `headless` | `bool` | |-| `print_actions` | `bool` | |-| `bot` | `Optional[Bot]` | |-| `tracing` | `Optional[Tracing]` | |-| `step_count` | `int` | |-| `max_steps` | `Optional[int]` | |-| `tasks_config_fn` | `Any` | |-| `setup_task_fn` | `Any` | |-| `solve_task_fn` | `Any` | |-| `evaluate_task_fn` | `Any` | |-| `tasks` | `Optional[list]` | |-| `current_task` | `Optional[Any]` | |-| `session_name` | `Optional[str]` | |-| `session_config` | `Dict[str, Any]` | |-| `setup_config` | `DesktopSetupConfig` | |-| `page` | `Optional[Any]` | |--### Methods--#### Environment.make_from_module--```python-def make_from_module(cls, module: Any, env_path: str | Path, split: str = 'train') -> 'Environment'-```--#### Environment.create_sandbox--```python-async def create_sandbox(self, provider: str, provider_config: Dict[str, Any] | None = None, setup_config: DesktopSetupConfig | None = None) -> None-```--#### Environment.reset--```python-async def reset(self, task_id: Optional[int] = None, run_id: Optional[str] = None) -> Tuple[bytes, Dict]-```--#### Environment.step--```python-async def step(self, action: Action, dry_run: bool | Literal['before', 'after'] = False) -> bytes-```--#### Environment.solve--```python-async def solve(self) -> bytes-```--#### Environment.evaluate--```python-async def evaluate(self) -> Any-```--#### Environment.close--```python-async def close(self) -> None-```-------## BenchmarkResult--Result of a benchmark run.--Attributes:- run_id: Unique identifier for this run- task_results: List of individual task results- total_tasks: Total number of tasks in the benchmark- success_count: Number of successful tasks- failed_count: Number of failed tasks- avg_reward: Average reward across all tasks- duration_seconds: Total duration of the benchmark- output_dir: Output directory for results (if any)--### Constructor--```python-BenchmarkResult(self, run_id: str, task_results: List[Dict[str, Any]], total_tasks: int, success_count: int, failed_count: int, avg_reward: float, duration_seconds: float, output_dir: Optional[str] = None) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `run_id` | `str` | |-| `task_results` | `List[Dict[str, Any]]` | |-| `total_tasks` | `int` | |-| `success_count` | `int` | |-| `failed_count` | `int` | |-| `avg_reward` | `float` | |-| `duration_seconds` | `float` | |-| `output_dir` | `Optional[str]` | |-------## TaskResult--Result of a single task execution.--Attributes:- task_path: Path to the task- variant_id: Task variant index- success: Whether the task succeeded- reward: Reward from evaluation- steps: Number of steps taken- error: Error message if failed--### Constructor--```python-TaskResult(self, task_path: str, variant_id: int, success: bool, reward: float, steps: int, error: Optional[str] = None) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `task_path` | `str` | |-| `variant_id` | `int` | |-| `success` | `bool` | |-| `reward` | `float` | |-| `steps` | `int` | |-| `error` | `Optional[str]` | |-------## ClickAction--### Constructor--```python-ClickAction(self, x: int, y: int) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `x` | `int` | |-| `y` | `int` | |-------## DoneAction--### Constructor--```python-DoneAction(self) -> None-```-------## DoubleClickAction--### Constructor--```python-DoubleClickAction(self, x: int, y: int) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `x` | `int` | |-| `y` | `int` | |-------## DragAction--### Constructor--```python-DragAction(self, from_x: int, from_y: int, to_x: int, to_y: int, duration: float = 1.0) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `from_x` | `int` | |-| `from_y` | `int` | |-| `to_x` | `int` | |-| `to_y` | `int` | |-| `duration` | `float` | |-------## HotkeyAction--### Constructor--```python-HotkeyAction(self, keys: List[str]) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `keys` | `List[str]` | |-------## KeyAction--### Constructor--```python-KeyAction(self, key: str) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `key` | `str` | |-------## MiddleClickAction--### Constructor--```python-MiddleClickAction(self, x: int, y: int) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `x` | `int` | |-| `y` | `int` | |-------## MoveToAction--### Constructor--```python-MoveToAction(self, x: int, y: int, duration: float = 0.0) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `x` | `int` | |-| `y` | `int` | |-| `duration` | `float` | |-------## RightClickAction--### Constructor--```python-RightClickAction(self, x: int, y: int) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `x` | `int` | |-| `y` | `int` | |-------## ScrollAction--### Constructor--```python-ScrollAction(self, direction: Literal['up', 'down'] = 'up', amount: int = 100) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `direction` | `Literal['up', 'down']` | |-| `amount` | `int` | |-------## TypeAction--### Constructor--```python-TypeAction(self, text: str) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `text` | `str` | |-------## WaitAction--### Constructor--```python-WaitAction(self, seconds: float = 1.0) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `seconds` | `float` | |--## repr_to_action--```python-def repr_to_action(action_repr: str) -> Action-```--Parse an action from repr format string.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `action_repr` | `Any` | Action string in repr format, e.g., "ClickAction(x=100, y=200)" |--**Returns:** Parsed Action object--**Raises:**--- `ValueError` - If the action string cannot be parsed--## interact--```python-def interact(env_path: str, task_id: int = 0) -> None-```--Run an environment interactively with simplified output.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `env_path` | `Any` | Path to the environment directory |-| `task_id` | `Any` | Task ID to run (default: 0) |--## make--```python-def make(env_name: str, split: str = 'train') -> Any-```--Create an Environment by loading the env's main.py as a module.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `env_name` | `Any` | Path to the environment directory (must contain main.py) |-| `split` | `Any` | Dataset split to use for decorated functions (e.g., 'train', 'test') |--**Returns:** Environment instance--## evaluate_task--```python-def evaluate_task(_arg: Optional[Callable] = None, args = (), kwargs = {}) -> Callable-```--Decorator for the function that evaluates a task.--Can be used as ``@cb.evaluate_task`` or ``@cb.evaluate_task("train")``.-The decorated function receives task_cfg and should return evaluation results.--## setup_task--```python-def setup_task(_arg: Optional[Callable] = None, args = (), kwargs = {}) -> Callable-```--Decorator for the function that sets up a task.--Can be used as ``@cb.setup_task`` or ``@cb.setup_task("train")``.-The decorated function receives task_cfg and should initialize the environment.--## solve_task--```python-def solve_task(_arg: Optional[Callable] = None, args = (), kwargs = {}) -> Callable-```--Decorator for the function that solves a task.--Can be used as ``@cb.solve_task`` or ``@cb.solve_task("train")``.-The decorated function receives task_cfg and should execute the solution.--## tasks_config--```python-def tasks_config(_arg: Optional[Callable] = None, args = (), kwargs = {}) -> Callable-```--Decorator for the function that loads tasks.--Can be used as ``@cb.tasks_config`` or ``@cb.tasks_config("train")``.-The decorated function should return a list of Task objects.--## run_benchmark--```python-async def run_benchmark(dataset_path: Path, agent_fn: Optional[Callable[[bytes, Task], Action]] = None, max_steps: int = 100, max_parallel: int = 4, oracle: bool = False, max_variants: Optional[int] = None, task_filter: Optional[str] = None, split: str = 'train') -> BenchmarkResult-```--Run a benchmark on a dataset using the gym interface.--This function runs multiple tasks in parallel using the core gym interface-(make, reset, step, evaluate).--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `dataset_path` | `Any` | Path to the dataset directory |-| `agent_fn` | `Any` | Optional agent function that takes (screenshot, task_config) and returns an Action. Required if oracle=False. |-| `max_steps` | `Any` | Maximum steps per task (default: 100) |-| `max_parallel` | `Any` | Maximum parallel workers (default: 4) |-| `oracle` | `Any` | Run oracle/solver mode (default: False) |-| `max_variants` | `Any` | Maximum variants per task (optional) |-| `task_filter` | `Any` | Glob pattern to filter tasks (optional) |-| `split` | `Any` | Dataset split (default: "train") |--**Returns:** BenchmarkResult with run statistics and task results--**Example:**--```python-# Run oracle benchmark-result = await run_benchmark(- Path("./datasets/cua-bench-basic"),- oracle=True,- max_parallel=8,-)-print(f"Success rate: {result.success_count / result.total_tasks:.2%}")--# Run with custom agent-def random_agent(screenshot: bytes, task: Task) -> Action:- import random- return random.choice([- ClickAction(x=random.randint(0, 1920), y=random.randint(0, 1080)),- DoneAction(),- ])--result = await run_benchmark(- Path("./datasets/my-dataset"),- agent_fn=random_agent,- max_parallel=4,-)-```--## run_interactive--```python-async def run_interactive(env_path: Path, task_index: int = 0, split: str = 'train', headless: bool = False) -> Tuple[Environment, bytes, Task]-```--Run an environment interactively using the gym interface.--This function sets up an environment for interactive use, returning-the environment instance, initial screenshot, and task configuration.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `env_path` | `Any` | Path to the environment directory |-| `task_index` | `Any` | Task variant index (default: 0) |-| `split` | `Any` | Dataset split (default: "train") |-| `headless` | `Any` | Run in headless mode (default: False) |--**Returns:** Tuple of (env, screenshot, task_config) - env: Environment instance (caller should call env.close() when done) - screenshot: Initial screenshot bytes - task_config: Task configuration--**Example:**--```python-env, screenshot, task_cfg = await run_interactive(Path("./task"))-print(f"Task: {task_cfg.description}")--# Execute actions...-screenshot = await env.step(ClickAction(x=100, y=200))--# Evaluate-reward = await env.evaluate()-print(f"Reward: {reward}")--# Cleanup-await env.close()-```--## run_single_task--```python-async def run_single_task(env_path: Path, task_index: int = 0, split: str = 'train', agent_fn: Optional[Callable[[bytes, Task], Action]] = None, max_steps: int = 100, oracle: bool = False) -> TaskResult-```--Run a single task using the gym interface.--This function uses the core gym interface (make, reset, step, evaluate)-to run a task with either an agent function or the oracle solver.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `env_path` | `Any` | Path to the task environment directory |-| `task_index` | `Any` | Task variant index (default: 0) |-| `split` | `Any` | Dataset split (default: "train") |-| `agent_fn` | `Any` | Optional agent function that takes (screenshot, task_config) and returns an Action. If None and oracle=False, returns after setup. |-| `max_steps` | `Any` | Maximum steps per task (default: 100) |-| `oracle` | `Any` | Run oracle/solver mode (default: False) |--**Returns:** TaskResult with execution results--**Example:**--```python-# Run with oracle-result = await run_single_task(Path("./task"), oracle=True)--# Run with custom agent-def my_agent(screenshot: bytes, task: Task) -> Action:- return DoneAction() # Simple agent that immediately finishes--result = await run_single_task(Path("./task"), agent_fn=my_agent)-```-------## tracing-------## Tracing--Lightweight trajectory tracing using Hugging Face Datasets.--Records events with arbitrary JSON metadata and a list of PIL images.-Exposes a datasets.Dataset-compatible interface for saving/pushing.--### Constructor--```python-Tracing(self, env: Any) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `env` | `Any` | |-| `trajectory_id` | `Optional[str]` | |-| `dataset` | `Dataset` | Return a HF Dataset built from current rows, constructing lazily. |--### Methods--#### Tracing.start--```python-def start(self, trajectory_id: Optional[str] = None) -> str-```--Start a new trajectory. Resets any previously recorded rows.--Returns the trajectory_id used.--#### Tracing.record--```python-def record(self, event_name: str, data_dict: Dict[str, Any], data_images: List[Image.Image | bytes] | None = None) -> None-```--#### Tracing.save_to_disk--```python-def save_to_disk(self, output_dir: str, save_pngs: bool = False, image_dir: Optional[str] = None, filter_events: Optional[List[str]] = None) -> None-```--#### Tracing.push_to_hub--```python-def push_to_hub(self, repo_id: str, private: bool | None = None) -> str-```--#### Tracing.bytes_to_image--```python-def bytes_to_image(png_bytes: bytes) -> Image.Image-```-------## actions-------## ClickAction--### Constructor--```python-ClickAction(self, x: int, y: int) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `x` | `int` | |-| `y` | `int` | |-------## DoneAction--### Constructor--```python-DoneAction(self) -> None-```-------## DoubleClickAction--### Constructor--```python-DoubleClickAction(self, x: int, y: int) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `x` | `int` | |-| `y` | `int` | |-------## DragAction--### Constructor--```python-DragAction(self, from_x: int, from_y: int, to_x: int, to_y: int, duration: float = 1.0) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `from_x` | `int` | |-| `from_y` | `int` | |-| `to_x` | `int` | |-| `to_y` | `int` | |-| `duration` | `float` | |-------## HotkeyAction--### Constructor--```python-HotkeyAction(self, keys: List[str]) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `keys` | `List[str]` | |-------## KeyAction--### Constructor--```python-KeyAction(self, key: str) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `key` | `str` | |-------## MiddleClickAction--### Constructor--```python-MiddleClickAction(self, x: int, y: int) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `x` | `int` | |-| `y` | `int` | |-------## MoveToAction--### Constructor--```python-MoveToAction(self, x: int, y: int, duration: float = 0.0) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `x` | `int` | |-| `y` | `int` | |-| `duration` | `float` | |-------## RightClickAction--### Constructor--```python-RightClickAction(self, x: int, y: int) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `x` | `int` | |-| `y` | `int` | |-------## ScrollAction--### Constructor--```python-ScrollAction(self, direction: Literal['up', 'down'] = 'up', amount: int = 100) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `direction` | `Literal['up', 'down']` | |-| `amount` | `int` | |-------## TypeAction--### Constructor--```python-TypeAction(self, text: str) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `text` | `str` | |-------## WaitAction--### Constructor--```python-WaitAction(self, seconds: float = 1.0) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `seconds` | `float` | |--### repr_to_action--```python-def repr_to_action(action_repr: str) -> Action-```--Parse an action from repr format string.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `action_repr` | `Any` | Action string in repr format, e.g., "ClickAction(x=100, y=200)" |--**Returns:** Parsed Action object--**Raises:**--- `ValueError` - If the action string cannot be parsed--### snake_case_to_action--```python-def snake_case_to_action(action_str: str) -> Action-```--Parse an action from snake_case format string.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `action_str` | `Any` | Action string in snake_case format, e.g., "click(0.5, 0.5)" |--**Returns:** Parsed Action object--**Raises:**--- `ValueError` - If the action string cannot be parsed--### parse_action_string--```python-def parse_action_string(action_str: str) -> Action-```--Parse an action from either repr or snake_case format.--This is the unified entry point for parsing action strings.-It automatically detects the format and delegates to the appropriate parser.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `action_str` | `Any` | Action string in either format: - Repr format: "ClickAction(x=100, y=200)" - Snake_case format: "click(0.5, 0.5)" |--**Returns:** Parsed Action object--**Raises:**--- `ValueError` - If the action string cannot be parsed in either format--### action_to_dict--```python-def action_to_dict(action: Action) -> Dict[str, Any]-```--Convert an Action object to a dictionary.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `action` | `Any` | Action object to convert |--**Returns:** Dictionary representation of the action with 'type' key--### dict_to_action--```python-def dict_to_action(action_dict: Dict[str, Any]) -> Action-```--Convert a dictionary to an Action object.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `action_dict` | `Any` | Dictionary with 'type' key and action parameters |--**Returns:** Action object--**Raises:**--- `ValueError` - If the action type is unknown-------## core--Core classes and functions for cua-bench.-------## Task--Represents a single task to be executed.--### Constructor--```python-Task(self, description: str, task_id: Optional[str] = None, metadata: Optional[dict] = None, computer: Optional[dict] = None) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `description` | `str` | |-| `task_id` | `Optional[str]` | |-| `metadata` | `Optional[dict]` | |-| `computer` | `Optional[dict]` | |--### make--```python-def make(env_name: str, split: str = 'train') -> Any-```--Create an Environment by loading the env's main.py as a module.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `env_name` | `Any` | Path to the environment directory (must contain main.py) |-| `split` | `Any` | Dataset split to use for decorated functions (e.g., 'train', 'test') |--**Returns:** Environment instance--### interact--```python-def interact(env_path: str, task_id: int = 0) -> None-```--Run an environment interactively with simplified output.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `env_path` | `Any` | Path to the environment directory |-| `task_id` | `Any` | Task ID to run (default: 0) |-------## types-------## WindowSnapshot--### Constructor--```python-WindowSnapshot(self, window_type: Literal['webview', 'process', 'desktop'], pid: Optional[str] = None, url: Optional[str] = None, html: Optional[str] = None, title: str = '', x: int = 0, y: int = 0, width: int = 0, height: int = 0, active: bool = False, minimized: bool = False) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `window_type` | `Literal['webview', 'process', 'desktop']` | |-| `pid` | `Optional[str]` | |-| `url` | `Optional[str]` | |-| `html` | `Optional[str]` | |-| `title` | `str` | |-| `x` | `int` | |-| `y` | `int` | |-| `width` | `int` | |-| `height` | `int` | |-| `active` | `bool` | |-| `minimized` | `bool` | |-------## Snapshot--### Constructor--```python-Snapshot(self, windows: List[WindowSnapshot]) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `windows` | `List[WindowSnapshot]` | |-------## ClickAction--### Constructor--```python-ClickAction(self, x: int, y: int) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `x` | `int` | |-| `y` | `int` | |-------## RightClickAction--### Constructor--```python-RightClickAction(self, x: int, y: int) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `x` | `int` | |-| `y` | `int` | |-------## DoubleClickAction--### Constructor--```python-DoubleClickAction(self, x: int, y: int) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `x` | `int` | |-| `y` | `int` | |-------## MiddleClickAction--### Constructor--```python-MiddleClickAction(self, x: int, y: int) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `x` | `int` | |-| `y` | `int` | |-------## DragAction--### Constructor--```python-DragAction(self, from_x: int, from_y: int, to_x: int, to_y: int, duration: float = 1.0) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `from_x` | `int` | |-| `from_y` | `int` | |-| `to_x` | `int` | |-| `to_y` | `int` | |-| `duration` | `float` | |-------## MoveToAction--### Constructor--```python-MoveToAction(self, x: int, y: int, duration: float = 0.0) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `x` | `int` | |-| `y` | `int` | |-| `duration` | `float` | |-------## ScrollAction--### Constructor--```python-ScrollAction(self, direction: Literal['up', 'down'] = 'up', amount: int = 100) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `direction` | `Literal['up', 'down']` | |-| `amount` | `int` | |-------## TypeAction--### Constructor--```python-TypeAction(self, text: str) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `text` | `str` | |-------## KeyAction--### Constructor--```python-KeyAction(self, key: str) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `key` | `str` | |-------## HotkeyAction--### Constructor--```python-HotkeyAction(self, keys: List[str]) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `keys` | `List[str]` | |-------## DoneAction--### Constructor--```python-DoneAction(self) -> None-```-------## WaitAction--### Constructor--```python-WaitAction(self, seconds: float = 1.0) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `seconds` | `float` | |-------## bot-------## ClickAction--### Constructor--```python-ClickAction(self, x: int, y: int) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `x` | `int` | |-| `y` | `int` | |-------## RightClickAction--### Constructor--```python-RightClickAction(self, x: int, y: int) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `x` | `int` | |-| `y` | `int` | |-------## Bot--Helper class for writing trajectories for task solutions.--### Constructor--```python-Bot(self, env: Any)-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `env` | `Any` | |--### Methods--#### Bot.click_element--```python-def click_element(self, pid: int, selector: str) -> None-```--Find element by CSS selector and click its center.--Uses provider's bench-ui bridge to fetch element rect in screen space-and then dispatches a ClickAction via env.step().--#### Bot.right_click_element--```python-def right_click_element(self, pid: int, selector: str) -> None-```-------## utils--Utility functions for synthetic data generation.-------## DesktopSetupConfig--*Inherits from: TypedDict*--Configuration for desktop setup provided to providers.--Fields mirror high-level desktop appearance and workspace options.--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `os_type` | `Literal['win11', 'win10', 'win7', 'winxp', 'win98', 'macos', 'linux', 'android', 'ios', 'windows']` | |-| `width` | `int` | |-| `height` | `int` | |-| `background` | `str` | |-| `wallpaper` | `str` | |-| `installed_apps` | `List[str]` | |-| `image` | `str` | |-| `storage` | `str` | |-| `memory` | `str` | |-| `cpu` | `str` | |-| `provider_type` | `str` | |-------## Environment--A minimal environment wrapper that delegates everything to a provider.--Functions can be injected directly, or discovered from a module via-`make_from_module` based on cua-bench decorators (`_td_type`, `_td_split`).--### Constructor--```python-Environment(self, env_name: Optional[str] = None, split: str = 'train', tasks_config_fn: Optional[Callable[..., Any]] = None, setup_task_fn: Optional[Callable[..., Any]] = None, solve_task_fn: Optional[Callable[..., Any]] = None, evaluate_task_fn: Optional[Callable[..., Any]] = None) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `session` | `Optional[Any]` | |-| `env_name` | `Optional[str]` | |-| `split` | `Optional[str]` | |-| `headless` | `bool` | |-| `print_actions` | `bool` | |-| `bot` | `Optional[Bot]` | |-| `tracing` | `Optional[Tracing]` | |-| `step_count` | `int` | |-| `max_steps` | `Optional[int]` | |-| `tasks_config_fn` | `Any` | |-| `setup_task_fn` | `Any` | |-| `solve_task_fn` | `Any` | |-| `evaluate_task_fn` | `Any` | |-| `tasks` | `Optional[list]` | |-| `current_task` | `Optional[Any]` | |-| `session_name` | `Optional[str]` | |-| `session_config` | `Dict[str, Any]` | |-| `setup_config` | `DesktopSetupConfig` | |-| `page` | `Optional[Any]` | |--### Methods--#### Environment.make_from_module--```python-def make_from_module(cls, module: Any, env_path: str | Path, split: str = 'train') -> 'Environment'-```--#### Environment.create_sandbox--```python-async def create_sandbox(self, provider: str, provider_config: Dict[str, Any] | None = None, setup_config: DesktopSetupConfig | None = None) -> None-```--#### Environment.reset--```python-async def reset(self, task_id: Optional[int] = None, run_id: Optional[str] = None) -> Tuple[bytes, Dict]-```--#### Environment.step--```python-async def step(self, action: Action, dry_run: bool | Literal['before', 'after'] = False) -> bytes-```--#### Environment.solve--```python-async def solve(self) -> bytes-```--#### Environment.evaluate--```python-async def evaluate(self) -> Any-```--#### Environment.close--```python-async def close(self) -> None-```-------## Snapshot--### Constructor--```python-Snapshot(self, windows: List[WindowSnapshot]) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `windows` | `List[WindowSnapshot]` | |--### render_snapshot_async--```python-async def render_snapshot_async(setup_config: Dict[str, Any], snapshot: Dict[str, Any], screenshot_delay: float = 0, provider: Literal['webtop', 'computer'] = 'webtop') -> bytes-```--Render a snapshot and return screenshot bytes (async).--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `provider` | `Any` | Provider name ("webtop" or "computer") |-| `setup_config` | `Any` | Configuration dict for create_sandbox setup_config parameter |-| `snapshot` | `Any` | Snapshot dict containing windows and other state |-| `screenshot_delay` | `Any` | Delay in seconds before taking screenshot |--**Returns:** Screenshot as bytes--### render_windows_async--```python-async def render_windows_async(setup_config: Dict[str, Any], windows: List[Dict[str, Any]], screenshot_delay: float = 0, provider: Literal['webtop', 'computer'] = 'webtop', return_snapshot: bool = False, scroll_into_view: Optional[str] = None) -> bytes | Tuple[bytes, Snapshot]-```--Render windows and return screenshot bytes (async).--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `provider` | `Any` | Provider name ("webtop" or "computer") |-| `setup_config` | `Any` | Configuration dict for create_sandbox setup_config parameter |-| `windows` | `Any` | List of window dicts to pass directly to launch_window |-| `screenshot_delay` | `Any` | Delay in seconds before taking screenshot |-| `return_snapshot` | `Any` | If True, return tuple of (bytes, Snapshot) instead of just bytes |-| `scroll_into_view` | `Any` | Optional CSS selector for an element to scroll into view |--**Returns:** Screenshot as bytes, or tuple of (bytes, Snapshot) if return_snapshot=True--### render_snapshot--```python-def render_snapshot(setup_config: Dict[str, Any], snapshot: Dict[str, Any], screenshot_delay: float = 0, provider: Literal['webtop', 'computer'] = 'webtop') -> bytes-```--Render a snapshot and return screenshot bytes (sync wrapper).--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `provider` | `Any` | Provider name ("webtop" or "computer") |-| `setup_config` | `Any` | Configuration dict for create_sandbox setup_config parameter |-| `snapshot` | `Any` | Snapshot dict containing windows and other state |-| `screenshot_delay` | `Any` | Delay in seconds before taking screenshot |--**Returns:** Screenshot as bytes--### render_windows--```python-def render_windows(setup_config: Dict[str, Any], windows: List[Dict[str, Any]], screenshot_delay: float = 0, provider: Literal['webtop', 'computer'] = 'webtop', return_snapshot: bool = False, scroll_into_view: Optional[str] = None) -> bytes | Tuple[bytes, Snapshot]-```--Render windows and return screenshot bytes (sync wrapper).--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `provider` | `Any` | Provider name ("webtop" or "computer") |-| `setup_config` | `Any` | Configuration dict for create_sandbox setup_config parameter |-| `windows` | `Any` | List of window dicts to pass directly to launch_window |-| `screenshot_delay` | `Any` | Delay in seconds before taking screenshot |-| `return_snapshot` | `Any` | If True, return tuple of (bytes, Snapshot) instead of just bytes |-| `scroll_into_view` | `Any` | Optional CSS selector for an element to scroll into view |--**Returns:** Screenshot as bytes, or tuple of (bytes, Snapshot) if return_snapshot=True-------## runners--Benchmark runner functions for cua-bench.--This module provides programmatic interfaces for running benchmarks and-interactive environments, using the core gym interface (make, reset, step, evaluate).-------## Task--Represents a single task to be executed.--### Constructor--```python-Task(self, description: str, task_id: Optional[str] = None, metadata: Optional[dict] = None, computer: Optional[dict] = None) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `description` | `str` | |-| `task_id` | `Optional[str]` | |-| `metadata` | `Optional[dict]` | |-| `computer` | `Optional[dict]` | |-------## Environment--A minimal environment wrapper that delegates everything to a provider.--Functions can be injected directly, or discovered from a module via-`make_from_module` based on cua-bench decorators (`_td_type`, `_td_split`).--### Constructor--```python-Environment(self, env_name: Optional[str] = None, split: str = 'train', tasks_config_fn: Optional[Callable[..., Any]] = None, setup_task_fn: Optional[Callable[..., Any]] = None, solve_task_fn: Optional[Callable[..., Any]] = None, evaluate_task_fn: Optional[Callable[..., Any]] = None) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `session` | `Optional[Any]` | |-| `env_name` | `Optional[str]` | |-| `split` | `Optional[str]` | |-| `headless` | `bool` | |-| `print_actions` | `bool` | |-| `bot` | `Optional[Bot]` | |-| `tracing` | `Optional[Tracing]` | |-| `step_count` | `int` | |-| `max_steps` | `Optional[int]` | |-| `tasks_config_fn` | `Any` | |-| `setup_task_fn` | `Any` | |-| `solve_task_fn` | `Any` | |-| `evaluate_task_fn` | `Any` | |-| `tasks` | `Optional[list]` | |-| `current_task` | `Optional[Any]` | |-| `session_name` | `Optional[str]` | |-| `session_config` | `Dict[str, Any]` | |-| `setup_config` | `DesktopSetupConfig` | |-| `page` | `Optional[Any]` | |--### Methods--#### Environment.make_from_module--```python-def make_from_module(cls, module: Any, env_path: str | Path, split: str = 'train') -> 'Environment'-```--#### Environment.create_sandbox--```python-async def create_sandbox(self, provider: str, provider_config: Dict[str, Any] | None = None, setup_config: DesktopSetupConfig | None = None) -> None-```--#### Environment.reset--```python-async def reset(self, task_id: Optional[int] = None, run_id: Optional[str] = None) -> Tuple[bytes, Dict]-```--#### Environment.step--```python-async def step(self, action: Action, dry_run: bool | Literal['before', 'after'] = False) -> bytes-```--#### Environment.solve--```python-async def solve(self) -> bytes-```--#### Environment.evaluate--```python-async def evaluate(self) -> Any-```--#### Environment.close--```python-async def close(self) -> None-```-------## DoneAction--### Constructor--```python-DoneAction(self) -> None-```-------## BenchmarkResult--Result of a benchmark run.--Attributes:- run_id: Unique identifier for this run- task_results: List of individual task results- total_tasks: Total number of tasks in the benchmark- success_count: Number of successful tasks- failed_count: Number of failed tasks- avg_reward: Average reward across all tasks- duration_seconds: Total duration of the benchmark- output_dir: Output directory for results (if any)--### Constructor--```python-BenchmarkResult(self, run_id: str, task_results: List[Dict[str, Any]], total_tasks: int, success_count: int, failed_count: int, avg_reward: float, duration_seconds: float, output_dir: Optional[str] = None) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `run_id` | `str` | |-| `task_results` | `List[Dict[str, Any]]` | |-| `total_tasks` | `int` | |-| `success_count` | `int` | |-| `failed_count` | `int` | |-| `avg_reward` | `float` | |-| `duration_seconds` | `float` | |-| `output_dir` | `Optional[str]` | |-------## TaskResult--Result of a single task execution.--Attributes:- task_path: Path to the task- variant_id: Task variant index- success: Whether the task succeeded- reward: Reward from evaluation- steps: Number of steps taken- error: Error message if failed--### Constructor--```python-TaskResult(self, task_path: str, variant_id: int, success: bool, reward: float, steps: int, error: Optional[str] = None) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `task_path` | `str` | |-| `variant_id` | `int` | |-| `success` | `bool` | |-| `reward` | `float` | |-| `steps` | `int` | |-| `error` | `Optional[str]` | |--### make--```python-def make(env_name: str, split: str = 'train') -> Any-```--Create an Environment by loading the env's main.py as a module.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `env_name` | `Any` | Path to the environment directory (must contain main.py) |-| `split` | `Any` | Dataset split to use for decorated functions (e.g., 'train', 'test') |--**Returns:** Environment instance--### run_single_task--```python-async def run_single_task(env_path: Path, task_index: int = 0, split: str = 'train', agent_fn: Optional[Callable[[bytes, Task], Action]] = None, max_steps: int = 100, oracle: bool = False) -> TaskResult-```--Run a single task using the gym interface.--This function uses the core gym interface (make, reset, step, evaluate)-to run a task with either an agent function or the oracle solver.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `env_path` | `Any` | Path to the task environment directory |-| `task_index` | `Any` | Task variant index (default: 0) |-| `split` | `Any` | Dataset split (default: "train") |-| `agent_fn` | `Any` | Optional agent function that takes (screenshot, task_config) and returns an Action. If None and oracle=False, returns after setup. |-| `max_steps` | `Any` | Maximum steps per task (default: 100) |-| `oracle` | `Any` | Run oracle/solver mode (default: False) |--**Returns:** TaskResult with execution results--**Example:**--```python-# Run with oracle-result = await run_single_task(Path("./task"), oracle=True)--# Run with custom agent-def my_agent(screenshot: bytes, task: Task) -> Action:- return DoneAction() # Simple agent that immediately finishes--result = await run_single_task(Path("./task"), agent_fn=my_agent)-```--### run_benchmark--```python-async def run_benchmark(dataset_path: Path, agent_fn: Optional[Callable[[bytes, Task], Action]] = None, max_steps: int = 100, max_parallel: int = 4, oracle: bool = False, max_variants: Optional[int] = None, task_filter: Optional[str] = None, split: str = 'train') -> BenchmarkResult-```--Run a benchmark on a dataset using the gym interface.--This function runs multiple tasks in parallel using the core gym interface-(make, reset, step, evaluate).--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `dataset_path` | `Any` | Path to the dataset directory |-| `agent_fn` | `Any` | Optional agent function that takes (screenshot, task_config) and returns an Action. Required if oracle=False. |-| `max_steps` | `Any` | Maximum steps per task (default: 100) |-| `max_parallel` | `Any` | Maximum parallel workers (default: 4) |-| `oracle` | `Any` | Run oracle/solver mode (default: False) |-| `max_variants` | `Any` | Maximum variants per task (optional) |-| `task_filter` | `Any` | Glob pattern to filter tasks (optional) |-| `split` | `Any` | Dataset split (default: "train") |--**Returns:** BenchmarkResult with run statistics and task results--**Example:**--```python-# Run oracle benchmark-result = await run_benchmark(- Path("./datasets/cua-bench-basic"),- oracle=True,- max_parallel=8,-)-print(f"Success rate: {result.success_count / result.total_tasks:.2%}")--# Run with custom agent-def random_agent(screenshot: bytes, task: Task) -> Action:- import random- return random.choice([- ClickAction(x=random.randint(0, 1920), y=random.randint(0, 1080)),- DoneAction(),- ])--result = await run_benchmark(- Path("./datasets/my-dataset"),- agent_fn=random_agent,- max_parallel=4,-)-```--### run_interactive--```python-async def run_interactive(env_path: Path, task_index: int = 0, split: str = 'train', headless: bool = False) -> Tuple[Environment, bytes, Task]-```--Run an environment interactively using the gym interface.--This function sets up an environment for interactive use, returning-the environment instance, initial screenshot, and task configuration.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `env_path` | `Any` | Path to the environment directory |-| `task_index` | `Any` | Task variant index (default: 0) |-| `split` | `Any` | Dataset split (default: "train") |-| `headless` | `Any` | Run in headless mode (default: False) |--**Returns:** Tuple of (env, screenshot, task_config) - env: Environment instance (caller should call env.close() when done) - screenshot: Initial screenshot bytes - task_config: Task configuration--**Example:**--```python-env, screenshot, task_cfg = await run_interactive(Path("./task"))-print(f"Task: {task_cfg.description}")--# Execute actions...-screenshot = await env.step(ClickAction(x=100, y=200))--# Evaluate-reward = await env.evaluate()-print(f"Reward: {reward}")--# Cleanup-await env.close()-```-------## environment--Simplified, provider-driven environment.-------## Bot--Helper class for writing trajectories for task solutions.--### Constructor--```python-Bot(self, env: Any)-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `env` | `Any` | |--### Methods--#### Bot.click_element--```python-def click_element(self, pid: int, selector: str) -> None-```--Find element by CSS selector and click its center.--Uses provider's bench-ui bridge to fetch element rect in screen space-and then dispatches a ClickAction via env.step().--#### Bot.right_click_element--```python-def right_click_element(self, pid: int, selector: str) -> None-```-------## Tracing--Lightweight trajectory tracing using Hugging Face Datasets.--Records events with arbitrary JSON metadata and a list of PIL images.-Exposes a datasets.Dataset-compatible interface for saving/pushing.--### Constructor--```python-Tracing(self, env: Any) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `env` | `Any` | |-| `trajectory_id` | `Optional[str]` | |-| `dataset` | `Dataset` | Return a HF Dataset built from current rows, constructing lazily. |--### Methods--#### Tracing.start--```python-def start(self, trajectory_id: Optional[str] = None) -> str-```--Start a new trajectory. Resets any previously recorded rows.--Returns the trajectory_id used.--#### Tracing.record--```python-def record(self, event_name: str, data_dict: Dict[str, Any], data_images: List[Image.Image | bytes] | None = None) -> None-```--#### Tracing.save_to_disk--```python-def save_to_disk(self, output_dir: str, save_pngs: bool = False, image_dir: Optional[str] = None, filter_events: Optional[List[str]] = None) -> None-```--#### Tracing.push_to_hub--```python-def push_to_hub(self, repo_id: str, private: bool | None = None) -> str-```--#### Tracing.bytes_to_image--```python-def bytes_to_image(png_bytes: bytes) -> Image.Image-```-------## MaxStepsExceeded--*Inherits from: Exception*--Raised when the environment's max step budget is exhausted.-------## Environment--A minimal environment wrapper that delegates everything to a provider.--Functions can be injected directly, or discovered from a module via-`make_from_module` based on cua-bench decorators (`_td_type`, `_td_split`).--### Constructor--```python-Environment(self, env_name: Optional[str] = None, split: str = 'train', tasks_config_fn: Optional[Callable[..., Any]] = None, setup_task_fn: Optional[Callable[..., Any]] = None, solve_task_fn: Optional[Callable[..., Any]] = None, evaluate_task_fn: Optional[Callable[..., Any]] = None) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `session` | `Optional[Any]` | |-| `env_name` | `Optional[str]` | |-| `split` | `Optional[str]` | |-| `headless` | `bool` | |-| `print_actions` | `bool` | |-| `bot` | `Optional[Bot]` | |-| `tracing` | `Optional[Tracing]` | |-| `step_count` | `int` | |-| `max_steps` | `Optional[int]` | |-| `tasks_config_fn` | `Any` | |-| `setup_task_fn` | `Any` | |-| `solve_task_fn` | `Any` | |-| `evaluate_task_fn` | `Any` | |-| `tasks` | `Optional[list]` | |-| `current_task` | `Optional[Any]` | |-| `session_name` | `Optional[str]` | |-| `session_config` | `Dict[str, Any]` | |-| `setup_config` | `DesktopSetupConfig` | |-| `page` | `Optional[Any]` | |--### Methods--#### Environment.make_from_module--```python-def make_from_module(cls, module: Any, env_path: str | Path, split: str = 'train') -> 'Environment'-```--#### Environment.create_sandbox--```python-async def create_sandbox(self, provider: str, provider_config: Dict[str, Any] | None = None, setup_config: DesktopSetupConfig | None = None) -> None-```--#### Environment.reset--```python-async def reset(self, task_id: Optional[int] = None, run_id: Optional[str] = None) -> Tuple[bytes, Dict]-```--#### Environment.step--```python-async def step(self, action: Action, dry_run: bool | Literal['before', 'after'] = False) -> bytes-```--#### Environment.solve--```python-async def solve(self) -> bytes-```--#### Environment.evaluate--```python-async def evaluate(self) -> Any-```--#### Environment.close--```python-async def close(self) -> None-```-------## iconify--Iconify icon processing module for cua_bench.--This module provides functionality to process HTML containing iconify-icon elements-and replace them with inline SVG content fetched from the Iconify API.--Key features:-- Processes <iconify-icon icon="prefix:name"> elements-- Supports custom icons.json for icon resolution-- Option to ignore icon set prefixes for randomization-- Caches SVG content for performance-- Preserves element attributes (width, height, class, etc.)--### process_icons--```python-def process_icons(html: str, icons_json: Optional[str] = None, ignore_iconset: bool = False) -> str-```--Process HTML containing iconify-icon elements and replace them with inline SVGs.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `html` | `Any` | HTML content containing iconify-icon elements |-| `icons_json` | `Any` | Path to custom icons.json file. If None, uses default iconsets/icons.json |-| `ignore_iconset` | `Any` | If True, ignores the iconset prefix and searches for icon name only. Useful for shuffling/randomizing icon sets. For example: - eva:people-outline becomes */people-outline - mingcute:ad-circle-line becomes */ad-circle-line |--**Returns:** HTML with iconify-icon elements replaced by inline SVG content--**Example:**--```python->>> html = '<iconify-icon icon="eva:people-outline"></iconify-icon>'->>> process_icons(html)-'<svg>...</svg>'-->>> # With ignore_iconset=True for randomization->>> process_icons(html, ignore_iconset=True) # May use different iconset-```--### clear_cache--```python-def clear_cache()-```--Clear the SVG cache. Useful for testing or memory management.--### get_cache_size--```python-def get_cache_size() -> int-```--Get the number of cached SVG entries.-------## main--Main entry point for cua-bench CLI.--### main--```python-def main()-```--Main CLI entry point.-------## desktop--Desktop environment management for cua-bench.-------## Window--Represents a window in the desktop environment.--### Constructor--```python-Window(self, x: int, y: int, width: int, height: int, title: str, content: str, focused: bool = False, icon: Optional[str] = None, title_bar_style: str = 'hidden') -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `x` | `int` | |-| `y` | `int` | |-| `width` | `int` | |-| `height` | `int` | |-| `title` | `str` | |-| `content` | `str` | |-| `focused` | `bool` | |-| `icon` | `Optional[str]` | |-| `title_bar_style` | `str` | |-------## DesktopState--State of the unified desktop environment.--### Constructor--```python-DesktopState(self, os_type: str = 'win11', width: int = 1024, height: int = 768, background: str = '#000', windows: List[Window] = list(), dock_state: Dict[str, List[Dict[str, str]]] = (lambda: {'pinned_apps': [], 'recent_apps': [], 'pinned_folders': []})(), taskbar_state: Dict[str, List[Dict[str, str]]] = (lambda: {'pinned_apps': [], 'open_apps': []})()) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `os_type` | `str` | |-| `width` | `int` | |-| `height` | `int` | |-| `background` | `str` | |-| `windows` | `List[Window]` | |-| `dock_state` | `Dict[str, List[Dict[str, str]]]` | |-| `taskbar_state` | `Dict[str, List[Dict[str, str]]]` | |-------## Desktop--Desktop environment manager.--### Constructor--```python-Desktop(self, env)-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `env` | `Any` | |-| `state` | `Any` | |-| `template` | `Any` | |--### Methods--#### Desktop.configure--```python-def configure(self, os_type: Optional[str] = None, width: Optional[int] = None, height: Optional[int] = None, background: Optional[str] = None, dock_state: Optional[Dict[str, List[Union[str, Dict[str, str]]]]] = None, randomize_dock: bool = True, taskbar_state: Optional[Dict[str, List[Union[str, Dict[str, str]]]]] = None, randomize_taskbar: bool = True)-```--Configure desktop appearance.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `os_type` | `Any` | OS appearance (win11, win10, win7, macos, winxp, win98, android, ios) |-| `width` | `Any` | Screen width in pixels |-| `height` | `Any` | Screen height in pixels |-| `background` | `Any` | Background color |-| `dock_state` | `Any` | Explicit dock state to set with keys 'pinned_apps', 'recent_apps', 'pinned_folders' |-| `randomize_dock` | `Any` | If True, populate dock_state using macOS icon sets |-| `taskbar_state` | `Any` | Explicit taskbar state to set with keys 'pinned_apps', 'open_apps' |-| `randomize_taskbar` | `Any` | If True, populate taskbar_state using Windows 11 icon sets |--#### Desktop.launch--```python-def launch(self, content: str, title: str = 'Window', x: Optional[int] = None, y: Optional[int] = None, width: int = 600, height: int = 400, icon: Optional[str] = None, use_inner_size: bool = False, title_bar_style: str = 'default') -> Window-```--Launch a new window on the desktop.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `content` | `Any` | HTML content for the window body |-| `title` | `Any` | Window title |-| `x` | `Any` | X position (auto-calculated if None) |-| `y` | `Any` | Y position (auto-calculated if None) |-| `width` | `Any` | Window width |-| `height` | `Any` | Window height |-| `use_inner_size` | `Any` | Whether to use the inner size of the window (i.e. content size) |--**Returns:** Window instance-------## decorators--Decorators for defining cua-bench environments.--### tasks_config--```python-def tasks_config(_arg: Optional[Callable] = None, args = (), kwargs = {}) -> Callable-```--Decorator for the function that loads tasks.--Can be used as ``@cb.tasks_config`` or ``@cb.tasks_config("train")``.-The decorated function should return a list of Task objects.--### setup_task--```python-def setup_task(_arg: Optional[Callable] = None, args = (), kwargs = {}) -> Callable-```--Decorator for the function that sets up a task.--Can be used as ``@cb.setup_task`` or ``@cb.setup_task("train")``.-The decorated function receives task_cfg and should initialize the environment.--### solve_task--```python-def solve_task(_arg: Optional[Callable] = None, args = (), kwargs = {}) -> Callable-```--Decorator for the function that solves a task.--Can be used as ``@cb.solve_task`` or ``@cb.solve_task("train")``.-The decorated function receives task_cfg and should execute the solution.--### evaluate_task--```python-def evaluate_task(_arg: Optional[Callable] = None, args = (), kwargs = {}) -> Callable-```--Decorator for the function that evaluates a task.--Can be used as ``@cb.evaluate_task`` or ``@cb.evaluate_task("train")``.-The decorated function receives task_cfg and should return evaluation results.-------## computers-------## DesktopSession--*Inherits from: Protocol*--Desktop session interface for environment backends.--Usage:- # Preferred: async context manager- async with get_session("native")(os_type="linux") as session:- await session.screenshot()-- # Alternative: manual lifecycle- session = get_session("native")(os_type="linux")- await session.start()- try:- await session.screenshot()- finally:- await session.close()--### Constructor--```python-DesktopSession(self, env: Any)-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `page` | `Any` | |-| `vnc_url` | `str` | Return the VNC URL for accessing the desktop environment. |-| `apps` | `'AppsProxy'` | Access registered apps via session.apps.\{app_name\}. |--### Methods--#### DesktopSession.start--```python-async def start(self, config: Optional[DesktopSetupConfig] = None, headless: Optional[bool] = None) -> None-```--Start the session and connect to the environment.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `config` | `Any` | Optional configuration to apply before starting. |-| `headless` | `Any` | If False, shows browser/VNC preview. Defaults to True. |--#### DesktopSession.serve_static--```python-async def serve_static(self, url_path: str, local_path: str) -> None-```--#### DesktopSession.launch_window--```python-async def launch_window(self, url: Optional[str] = None, html: Optional[str] = None, folder: Optional[str] = None, title: str = 'Window', x: Optional[int] = None, y: Optional[int] = None, width: int = 600, height: int = 400, icon: Optional[str] = None, use_inner_size: bool = False, title_bar_style: str = 'default') -> int | str-```--Launch a window and return its process ID.--#### DesktopSession.get_element_rect--```python-async def get_element_rect(self, pid: int | str, selector: str, space: Literal['window', 'screen'] = 'window', timeout: float = 0.5) -> dict[str, Any] | None-```--#### DesktopSession.execute_javascript--```python-async def execute_javascript(self, pid: int | str, javascript: str) -> Any-```--#### DesktopSession.execute_action--```python-async def execute_action(self, action: Any) -> None-```--#### DesktopSession.screenshot--```python-async def screenshot(self) -> bytes-```--#### DesktopSession.get_snapshot--```python-async def get_snapshot(self) -> Snapshot-```--Return a lightweight snapshot of the desktop state (windows, etc.).--Implementations should populate the list of open windows with geometry-and metadata. If not supported, raise NotImplementedError.--#### DesktopSession.close--```python-async def close(self) -> None-```--#### DesktopSession.close_all_windows--```python-async def close_all_windows(self) -> None-```--Close or clear all open windows in the desktop environment.--#### DesktopSession.click_element--```python-async def click_element(self, pid: int | str, selector: str) -> None-```--Find element by CSS selector and click its center.--Uses the session's get_element_rect to fetch element rect in screen space-and then dispatches a ClickAction.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `pid` | `Any` | Process ID of the window |-| `selector` | `Any` | CSS selector for the element |--#### DesktopSession.right_click_element--```python-async def right_click_element(self, pid: int | str, selector: str) -> None-```--Find element by CSS selector and right-click its center.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `pid` | `Any` | Process ID of the window |-| `selector` | `Any` | CSS selector for the element |--#### DesktopSession.run_command--```python-async def run_command(self, command: str, timeout: Optional[float] = None, check: bool = True) -> 'CommandResult'-```--Execute a shell command on the native desktop environment.--This method is only available with the native provider (Docker/QEMU).-It will raise NotImplementedError on simulated sessions.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `command` | `Any` | Shell command to execute |-| `timeout` | `Any` | Optional timeout in seconds |-| `check` | `Any` | If True (default), raise an exception if the command fails (non-zero return code). If False, return the result regardless. |--**Returns:** CommandResult with stdout, stderr, and return_code--**Raises:**--- `NotImplementedError` - If called on simulated provider-- `RuntimeError` - If check=True and command returns non-zero exit code--**Example:**--```python-result = await session.run_command("ls -la /home/user")-print(result.stdout)-```--#### DesktopSession.install_app--```python-async def install_app(self, app_name: str, with_shortcut: bool = True, kwargs = {}) -> None-```--Install a registered app on the native desktop environment.--Uses the app registry to find platform-specific install functions.-This method is only available with the native provider (Docker/QEMU).--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `app_name` | `Any` | Name of the app to install (e.g., "godot", "firefox") |-| `with_shortcut` | `Any` | Create desktop shortcut (default True) **kwargs: App-specific arguments (e.g., version="4.2.1") |--**Raises:**--- `ValueError` - If app is not registered-- `NotImplementedError` - If app doesn't support the current platform--**Example:**--```python-await session.install_app("godot", version="4.2.1")-await session.install_app("firefox", with_shortcut=True)-```--#### DesktopSession.launch_app--```python-async def launch_app(self, app_name: str, kwargs = {}) -> None-```--Launch a registered app on the native desktop environment.--Uses the app registry to find platform-specific launch functions.-This method is only available with the native provider (Docker/QEMU).--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `app_name` | `Any` | Name of the app to launch **kwargs: App-specific arguments (e.g., project_path="/path") |--**Raises:**--- `ValueError` - If app is not registered-- `NotImplementedError` - If app doesn't support the current platform--**Example:**--```python-await session.launch_app("godot", project_path="~/project", editor=True)-```-------## DesktopSetupConfig--*Inherits from: TypedDict*--Configuration for desktop setup provided to providers.--Fields mirror high-level desktop appearance and workspace options.--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `os_type` | `Literal['win11', 'win10', 'win7', 'winxp', 'win98', 'macos', 'linux', 'android', 'ios', 'windows']` | |-| `width` | `int` | |-| `height` | `int` | |-| `background` | `str` | |-| `wallpaper` | `str` | |-| `installed_apps` | `List[str]` | |-| `image` | `str` | |-| `storage` | `str` | |-| `memory` | `str` | |-| `cpu` | `str` | |-| `provider_type` | `str` | |-------## RemoteDesktopSession--Unified desktop session using cua-computer SDK.--Supports two modes:-1. **Full lifecycle mode** (default): Computer SDK manages container/VM- - Pass config via constructor kwargs or start(config=\{...\})- - SDK starts container, waits for boot, connects--2. **Client-only mode**: Connect to pre-existing cua-computer-server- - Pass api_url to connect to existing server- - Used by 2-container architecture, batch execution--Works with any golden environment type:-- linux-docker: trycua/cua-xfce container-- windows-qemu: Windows 11 VM-- linux-qemu: Linux VM-- android-qemu: Android VM--Supports full bench_ui integration when bench_ui is installed in the-remote environment, enabling:-- launch_window() with HTML content via pywebview-- execute_javascript() for DOM manipulation-- get_element_rect() for element location queries-- click_element() / right_click_element() for element-based interaction--### Constructor--```python-RemoteDesktopSession(self, api_url: str = '', vnc_url: str = '', width: int = 1920, height: int = 1080, os_type: str = 'linux', image: str = '', provider_type: str = 'docker', memory: str = '8GB', cpu: str = '4', name: str = '', storage: str = '', ephemeral: bool = True, headless: bool = True, kwargs = {})-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `DEFAULT_TIMEOUT` | `Any` | |-| `SCREENSHOT_TIMEOUT` | `Any` | |-| `computer` | `Any` | Get the Computer SDK instance for advanced operations. |-| `interface` | `Any` | Get the computer interface for direct SDK access. |-| `page` | `Any` | Return underlying page object - not applicable for remote. |-| `vnc_url` | `str` | Return the VNC URL for accessing the environment. |-| `apps` | `'AppsProxy'` | Access registered apps via session.apps.\{app_name\}. |-| `os_type` | `str` | Return the OS type for this session. |--### Methods--#### RemoteDesktopSession.step--```python-async def step(self, action: Action) -> None-```--Execute an action (alias for execute_action, for env.step() compatibility).--#### RemoteDesktopSession.start--```python-async def start(self, config: Optional[DesktopSetupConfig] = None, headless: Optional[bool] = None) -> None-```--Start the session and connect to the environment.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `config` | `Any` | Optional configuration to apply before starting. |-| `headless` | `Any` | If False, opens VNC preview in browser. Defaults to constructor value if not specified. |--**Example:**--```python-# Using constructor params (preferred)-async with RemoteDesktopSession(os_type="linux") as session:- await session.screenshot()--# Or with config dict-session = RemoteDesktopSession()-await session.start(config={"os_type": "linux", "width": 1920})-```--#### RemoteDesktopSession.serve_static--```python-async def serve_static(self, url_path: str, local_path: str) -> None-```--Serve static files - not applicable for remote environments.--#### RemoteDesktopSession.launch_window--```python-async def launch_window(self, url: Optional[str] = None, html: Optional[str] = None, folder: Optional[str] = None, title: str = 'Window', x: Optional[int] = None, y: Optional[int] = None, width: int = 600, height: int = 400, icon: Optional[str] = None, use_inner_size: bool = False, title_bar_style: str = 'default') -> int | str-```--Launch a window in the remote environment using bench_ui (pywebview).--Supports:-- url: Open a URL in a pywebview window-- html: Display HTML content in a pywebview window-- folder: Copy folder to remote and serve it in a pywebview window--**Returns:** Process ID of the pywebview window (int)--#### RemoteDesktopSession.get_element_rect--```python-async def get_element_rect(self, pid: int | str, selector: str, space: Literal['window', 'screen'] = 'window', timeout: float = 0.5) -> dict[str, Any] | None-```--Get element rect by CSS selector using bench_ui.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `pid` | `Any` | Process ID of the pywebview window |-| `selector` | `Any` | CSS selector for the element |-| `space` | `Any` | Coordinate space - "window" or "screen" |-| `timeout` | `Any` | Maximum time to wait for element |--**Returns:** Dict with x, y, width, height or None if not found--#### RemoteDesktopSession.execute_javascript--```python-async def execute_javascript(self, pid: int | str, javascript: str) -> Any-```--Execute JavaScript in a pywebview window using bench_ui.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `pid` | `Any` | Process ID of the pywebview window |-| `javascript` | `Any` | JavaScript code to execute |--**Returns:** Result of the JavaScript execution--#### RemoteDesktopSession.execute_action--```python-async def execute_action(self, action: Action) -> None-```--Execute an action on the remote desktop using the SDK.--#### RemoteDesktopSession.screenshot--```python-async def screenshot(self) -> bytes-```--Capture screenshot from remote environment.--**Returns:** PNG image bytes--#### RemoteDesktopSession.get_snapshot--```python-async def get_snapshot(self) -> Snapshot-```--Get snapshot of desktop state with active window info.--Uses pywinctl on remote to get active window, and if it's a webview-we launched, extracts HTML via snapshot.js.--#### RemoteDesktopSession.close--```python-async def close(self) -> None-```--Close the session and cleanup resources.--#### RemoteDesktopSession.close_all_windows--```python-async def close_all_windows(self) -> None-```--Close all windows - best effort.--#### RemoteDesktopSession.click_element--```python-async def click_element(self, pid: int | str, selector: str) -> None-```--Find element by CSS selector and click its center.--Uses get_element_rect to fetch element rect in screen space-and then dispatches a ClickAction.--#### RemoteDesktopSession.right_click_element--```python-async def right_click_element(self, pid: int | str, selector: str) -> None-```--Find element by CSS selector and right-click its center.--#### RemoteDesktopSession.get_accessibility_tree--```python-async def get_accessibility_tree(self) -> Dict[str, Any]-```--Get the accessibility tree if supported.--#### RemoteDesktopSession.shell_command--```python-async def shell_command(self, command: str, check: bool = True) -> Dict[str, Any]-```--Execute a shell command.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `command` | `Any` | Shell command to execute |-| `check` | `Any` | If True (default), raise an exception if the command fails (non-zero return code). If False, return the result regardless. |--**Returns:** Command result with stdout/stderr--**Raises:**--- `RuntimeError` - If check=True and command returns non-zero exit code--#### RemoteDesktopSession.read_file--```python-async def read_file(self, path: str) -> str-```--Read a text file from the environment.--#### RemoteDesktopSession.write_file--```python-async def write_file(self, path: str, content: str) -> None-```--Write a text file to the environment.--#### RemoteDesktopSession.read_bytes--```python-async def read_bytes(self, path: str) -> bytes-```--Read a file as bytes from the environment.--#### RemoteDesktopSession.write_bytes--```python-async def write_bytes(self, path: str, data: bytes) -> None-```--Write bytes to a file in the environment.--#### RemoteDesktopSession.file_exists--```python-async def file_exists(self, path: str) -> bool-```--Check if a file exists in the environment.--#### RemoteDesktopSession.directory_exists--```python-async def directory_exists(self, path: str) -> bool-```--Check if a directory exists in the environment.--#### RemoteDesktopSession.list_dir--```python-async def list_dir(self, path: str) -> list[str]-```--List contents of a directory in the environment.--#### RemoteDesktopSession.run_command--```python-async def run_command(self, command: str, check: bool = True) -> Dict[str, Any]-```--Execute a shell command (alias for shell_command).--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `command` | `Any` | Shell command to execute |-| `check` | `Any` | If True (default), raise an exception if the command fails (non-zero return code). If False, return the result regardless. |--**Returns:** Command result with stdout/stderr--**Raises:**--- `RuntimeError` - If check=True and command returns non-zero exit code--#### RemoteDesktopSession.launch_application--```python-async def launch_application(self, app_name: str) -> None-```--Launch an application by name.--#### RemoteDesktopSession.check_status--```python-async def check_status(self) -> bool-```--Check if the environment is responsive.--**Returns:** True if environment is ready, False otherwise--#### RemoteDesktopSession.wait_until_ready--```python-async def wait_until_ready(self, timeout: int = 60, poll_interval: float = 2.0) -> bool-```--Wait until the environment is ready.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `timeout` | `Any` | Maximum time to wait in seconds |-| `poll_interval` | `Any` | Time between status checks |--**Returns:** True if environment became ready, False if timeout--#### RemoteDesktopSession.click--```python-async def click(self, x: int, y: int) -> None-```--Click at coordinates.--#### RemoteDesktopSession.right_click--```python-async def right_click(self, x: int, y: int) -> None-```--Right-click at coordinates.--#### RemoteDesktopSession.double_click--```python-async def double_click(self, x: int, y: int) -> None-```--Double-click at coordinates.--#### RemoteDesktopSession.type--```python-async def type(self, text: str) -> None-```--Type text.--#### RemoteDesktopSession.key--```python-async def key(self, key: str) -> None-```--Press a key.--#### RemoteDesktopSession.hotkey--```python-async def hotkey(self, keys: list[str]) -> None-```--Press a key combination.--#### RemoteDesktopSession.scroll--```python-async def scroll(self, direction: str = 'down', amount: int = 300) -> None-```--Scroll the screen.--#### RemoteDesktopSession.move_to--```python-async def move_to(self, x: int, y: int) -> None-```--Move cursor to coordinates.--#### RemoteDesktopSession.drag--```python-async def drag(self, from_x: int, from_y: int, to_x: int, to_y: int) -> None-```--Drag from one position to another.--#### RemoteDesktopSession.install_app--```python-async def install_app(self, app_name: str, with_shortcut: bool = True, kwargs = {}) -> None-```--Install a registered app on the native desktop environment.--Uses the app registry to find platform-specific install functions.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `app_name` | `Any` | Name of the app to install (e.g., "godot", "firefox") |-| `with_shortcut` | `Any` | Create desktop shortcut (default True) **kwargs: App-specific arguments (e.g., version="4.2.1") |--**Raises:**--- `ValueError` - If app is not registered-- `NotImplementedError` - If app doesn't support the current platform--**Example:**--```python-await session.install_app("godot", version="4.2.1")-await session.install_app("firefox", with_shortcut=True)-```--#### RemoteDesktopSession.launch_app--```python-async def launch_app(self, app_name: str, kwargs = {}) -> None-```--Launch a registered app on the native desktop environment.--Uses the app registry to find platform-specific launch functions.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `app_name` | `Any` | Name of the app to launch **kwargs: App-specific arguments (e.g., project_path="/path") |--**Raises:**--- `ValueError` - If app is not registered-- `NotImplementedError` - If app doesn't support the current platform--**Example:**--```python-await session.launch_app("godot", project_path="~/project", editor=True)-```--### get_session--```python-def get_session(name: Optional[str] = None) -> type[DesktopSession]-```--Return session class by name.--Provider names:- - "simulated" (alias: "webtop"): Playwright-based browser simulation- Fast, no Docker required. UI is HTML/CSS rendering of desktop.- Good for web-app testing, UI benchmarks.-- - "native" (alias: "computer"): Real OS in Docker/QEMU container- Actual desktop environment with real applications.- Requires Docker. Good for real app testing, OS-level tasks.--### create_remote_session--```python-def create_remote_session(api_url: str, vnc_url: str = '', os_type: str = 'linux', width: int = 1920, height: int = 1080) -> RemoteDesktopSession-```--Create a RemoteDesktopSession.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `api_url` | `Any` | URL of the environment's API endpoint |-| `vnc_url` | `Any` | URL for VNC access |-| `os_type` | `Any` | Operating system type |-| `width` | `Any` | Screen width |-| `height` | `Any` | Screen height |--**Returns:** Configured RemoteDesktopSession instance-------## config--Configuration module for cua-bench.-------## ConfigLoader--Load and merge configuration from .cua/ directory.--### Constructor--```python-ConfigLoader(self, search_path: Path | None = None)-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `CONFIG_DIR_NAME` | `Any` | |-| `CONFIG_FILE_NAME` | `Any` | |-| `AGENTS_FILE_NAME` | `Any` | |-| `search_path` | `Any` | |--### Methods--#### ConfigLoader.find_config_dir--```python-def find_config_dir(self) -> Path | None-```--Walk up directory tree to find .cua/ directory.--**Returns:** Path to .cua/ directory if found, None otherwise.--#### ConfigLoader.load_config--```python-def load_config(self) -> CuaConfig | None-```--Load .cua/config.yaml if it exists.--**Returns:** CuaConfig object if config file exists, None otherwise.--#### ConfigLoader.load_agents--```python-def load_agents(self) -> list[CustomAgentEntry]-```--Load .cua/agents.yaml if it exists.--**Returns:** List of CustomAgentEntry objects.--#### ConfigLoader.get_agent_by_name--```python-def get_agent_by_name(self, name: str) -> CustomAgentEntry | None-```--Get a custom agent entry by name.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `name` | `Any` | Agent name to look up. |--**Returns:** CustomAgentEntry if found, None otherwise.--#### ConfigLoader.get_effective_config--```python-def get_effective_config(self, cli_args: dict[str, Any], env_type: str | None = None) -> dict[str, Any]-```--Merge configuration sources into effective config.--Priority (highest to lowest):-1. CLI arguments-2. Environment-specific overrides-3. Agent defaults from agents.yaml-4. Agent config from config.yaml-5. Defaults from config.yaml--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `cli_args` | `Any` | Command line arguments as dictionary. |-| `env_type` | `Any` | Environment type for env-specific overrides (e.g., "webtop", "winarena"). |--**Returns:** Merged configuration dictionary.-------## AgentConfig--Agent configuration from .cua/config.yaml.--### Constructor--```python-AgentConfig(self, name: str | None = None, import_path: str | None = None, model: str | None = None, max_steps: int = 100, environments: dict[str, dict[str, Any]] | None = None) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `name` | `str | None` | |-| `import_path` | `str | None` | |-| `model` | `str | None` | |-| `max_steps` | `int` | |-| `environments` | `dict[str, dict[str, Any]] | None` | |--### Methods--#### AgentConfig.from_dict--```python-def from_dict(cls, data: dict[str, Any]) -> AgentConfig-```--Create AgentConfig from dictionary.-------## AgentsConfig--Configuration from .cua/agents.yaml.--Supports two formats:-- Legacy: `custom_agents` list-- New: `agents` list (preferred)--Example .cua/agents.yaml:- agents:- - name: my-agent- image: myregistry/my-agent:latest- defaults:- model: gpt-4o-- - name: dev-agent- import_path: my_agents.dev:DevAgent--### Constructor--```python-AgentsConfig(self, custom_agents: list[CustomAgentEntry] = list()) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `custom_agents` | `list[CustomAgentEntry]` | |--### Methods--#### AgentsConfig.from_dict--```python-def from_dict(cls, data: dict[str, Any]) -> AgentsConfig-```--Create AgentsConfig from dictionary.-------## CuaConfig--Root configuration from .cua/config.yaml.--### Constructor--```python-CuaConfig(self, defaults: DefaultsConfig | None = None, agent: AgentConfig | None = None) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `defaults` | `DefaultsConfig | None` | |-| `agent` | `AgentConfig | None` | |--### Methods--#### CuaConfig.from_dict--```python-def from_dict(cls, data: dict[str, Any]) -> CuaConfig-```--Create CuaConfig from dictionary.-------## CustomAgentEntry--Entry for a custom agent in .cua/agents.yaml.--Agents can be defined in two ways:-1. Docker image (cloud-ready): Specify `image` field with a Docker image-2. Import path (local dev): Specify `import_path` for Python import--Examples:- # Docker image agent- - name: my-agent- image: myregistry/my-agent:latest-- # Import path agent (uses default cua-agent image)- - name: dev-agent- import_path: my_agents.dev:DevAgent-- # Built-in agent- - name: cua-agent- builtin: true--### Constructor--```python-CustomAgentEntry(self, name: str, image: Optional[str] = None, import_path: Optional[str] = None, builtin: bool = False, command: Optional[list[str]] = None, defaults: dict[str, Any] = dict()) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `name` | `str` | |-| `image` | `Optional[str]` | |-| `import_path` | `Optional[str]` | |-| `builtin` | `bool` | |-| `command` | `Optional[list[str]]` | |-| `defaults` | `dict[str, Any]` | |--### Methods--#### CustomAgentEntry.get_image--```python-def get_image(self) -> str-```--Get the Docker image to use for this agent.--**Returns:** Docker image name. Uses custom image if specified, otherwise returns the default cua-agent image.--#### CustomAgentEntry.is_docker_agent--```python-def is_docker_agent(self) -> bool-```--Check if this agent is defined as a Docker image.--**Returns:** True if agent has a custom Docker image specified.-------## DefaultsConfig--Default configuration values from .cua/config.yaml.--### Constructor--```python-DefaultsConfig(self, model: str | None = None, max_steps: int = 100, output_dir: str = './results') -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `model` | `str | None` | |-| `max_steps` | `int` | |-| `output_dir` | `str` | |--### Methods--#### DefaultsConfig.from_dict--```python-def from_dict(cls, data: dict[str, Any]) -> DefaultsConfig-```--Create DefaultsConfig from dictionary.--### detect_env_type--```python-def detect_env_type(env_path: str) -> str | None-```--Detect environment type from path.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `env_path` | `Any` | Path to the environment. |--**Returns:** Environment type string ("webtop" or "winarena"), or None if unknown.-------## runner--Runner module for 2-container task execution.-------## TaskResult--Result of a task execution.--### Constructor--```python-TaskResult(self, success: bool, exit_code: int, agent_logs: str, env_logs: str, output_dir: Optional[str] = None, error: Optional[str] = None) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `success` | `bool` | |-| `exit_code` | `int` | |-| `agent_logs` | `str` | |-| `env_logs` | `str` | |-| `output_dir` | `Optional[str]` | |-| `error` | `Optional[str]` | |-------## TaskRunner--Orchestrates 2-container task execution.--Architecture:-- Creates isolated Docker network per task-- Creates task overlay to protect golden image (QEMU types)-- Starts environment container (base image with QCOW2 disk)-- Starts agent container (runs solver)-- Agent connects to env via network hostname-- Waits for agent completion-- Collects results and cleans up (including overlay)--### Constructor--```python-TaskRunner(self, agent_image: str = DEFAULT_AGENT_IMAGE, env_hostname: str = 'cua-env', agent_hostname: str = 'cua-agent')-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `agent_image` | `Any` | |-| `env_hostname` | `Any` | |-| `agent_hostname` | `Any` | |--### Methods--#### TaskRunner.run_task--```python-async def run_task(self, env_path: Path, task_index: int, env_type: str, golden_name: Optional[str] = None, agent: Optional[str] = None, agent_image: Optional[str] = None, agent_command: Optional[List[str]] = None, agent_import_path: Optional[str] = None, model: Optional[str] = None, max_steps: int = 100, oracle: bool = False, memory: str = '8G', cpus: str = '8', vnc_port: Optional[int] = None, api_port: Optional[int] = None, output_dir: Optional[str] = None, stream_agent_logs: bool = False, timeout: Optional[int] = None, cleanup_before: bool = True, remove_images_after: bool = False, provider_type: Optional[str] = None, dev_paths: Optional[List[str]] = None) -> TaskResult-```--Run a task with 2-container architecture.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `env_path` | `Any` | Path to task environment directory |-| `task_index` | `Any` | Task index to run |-| `env_type` | `Any` | Environment type (linux-docker, windows-qemu, etc.) |-| `image_name` | `Any` | Image name to use (defaults to env_type). See: cb image list |-| `agent` | `Any` | Agent name (for built-in agents) |-| `agent_image` | `Any` | Docker image for agent container (overrides default) |-| `agent_command` | `Any` | Custom command for agent container |-| `agent_import_path` | `Any` | Custom agent import path |-| `model` | `Any` | Model to use |-| `max_steps` | `Any` | Maximum agent steps |-| `oracle` | `Any` | Run oracle solution instead of agent |-| `memory` | `Any` | Memory for environment (QEMU only) |-| `cpus` | `Any` | CPUs for environment (QEMU only) |-| `vnc_port` | `Any` | Host port to map VNC (for debugging) |-| `api_port` | `Any` | Host port to map API (for debugging) |-| `output_dir` | `Any` | Output directory for results |-| `stream_agent_logs` | `Any` | Stream agent logs to <output_dir>/run.log in real-time (default: False) |-| `timeout` | `Any` | Timeout in seconds (None = no timeout) |-| `cleanup_before` | `Any` | Clean up stale containers before starting (default: True) |-| `remove_images_after` | `Any` | Remove Docker images after task (default: False) Note: This removes Docker images but NOT base VM disk images. |-| `provider_type` | `Any` | Provider type ("simulated", "webtop", "native", "computer", None). If "simulated" or "webtop", the agent container will use a local Playwright session instead of connecting to a remote environment. |--**Returns:** TaskResult with execution details--#### TaskRunner.run_task_interactively--```python-async def run_task_interactively(self, env_type: str, golden_name: Optional[str] = None, env_path: Optional[Path] = None, task_index: int = 0, setup_config: Optional[dict] = None, memory: str = '8G', cpus: str = '8', vnc_port: Optional[int] = None, api_port: Optional[int] = None, auto_allocate_ports: bool = True, cleanup_before: bool = True) -> tuple[str, str, callable, Optional[dict]]-```--Start an environment container interactively (without agent).--This method starts only the environment container with VNC and API ports-exposed to the host, allowing manual interaction or agent connection.-If env_path is provided, it will also load the task and run the setup.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `env_type` | `Any` | Environment type (linux-docker, windows-qemu, etc.) |-| `golden_name` | `Any` | Image name to use (defaults to env_type) |-| `env_path` | `Any` | Path to task directory (optional, for running task setup) |-| `task_index` | `Any` | Task index to run (default: 0) |-| `memory` | `Any` | Memory for environment (QEMU only) |-| `cpus` | `Any` | CPUs for environment (QEMU only) |-| `vnc_port` | `Any` | Host port to map VNC (None = auto-allocate) |-| `api_port` | `Any` | Host port to map API (None = auto-allocate) |-| `auto_allocate_ports` | `Any` | Auto-allocate ports if not specified (default: True) |-| `cleanup_before` | `Any` | Clean up stale containers before starting (default: True) |--**Returns:** Tuple of (vnc_url, api_url, cleanup_func, task_config, env, session) - vnc_url: URL to access VNC (e.g., http://localhost:8006) - api_url: URL to access API (e.g., http://localhost:5000) - cleanup_func: Async function to call when done to cleanup resources - task_config: Task configuration dict (None if env_path not provided) - env: Environment object (None if env_path not provided) - session: RemoteDesktopSession object (None if env_path not provided)--**Example:**--```python-```python-runner = TaskRunner()-vnc_url, api_url, cleanup, task_cfg, env, session = await runner.run_task_interactively(- "linux-docker",- env_path=Path("./my_task"),- task_index=0-)-print(f"VNC: {vnc_url}")-print(f"Task: {task_cfg.get('description')}")-# ... do interactive work ...-# Evaluate before cleanup-if env and env.evaluate_task_fn:- result = await env.evaluate_task_fn(task_cfg['_task_cfg'], session)- print(f"Result: {result}")-await cleanup()-```-```--#### TaskRunner.cleanup_all--```python-async def cleanup_all(self) -> None-```--Clean up all running tasks.--#### TaskRunner.force_cleanup--```python-async def force_cleanup() -> dict-```--Force cleanup of all stale cua-bench containers and networks.--Use this when containers are left behind from previous runs.--**Returns:** Dict with counts: \{"containers": N, "networks": N\}-------## agents-------## AgentResult--Result of agent execution.--### Constructor--```python-AgentResult(self, total_input_tokens: int = 0, total_output_tokens: int = 0, failure_mode: FailureMode = FailureMode.UNSET) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `total_input_tokens` | `int` | |-| `total_output_tokens` | `int` | |-| `failure_mode` | `FailureMode` | |-------## BaseAgent--*Inherits from: ABC*--Base class for agents that can perform tasks.--### Constructor--```python-BaseAgent(self, kwargs = {})-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `version` | `str | None` | The version of the agent. Can be any string (e.g. could be |-| `prompt_template` | `str | None` | The path to a custom prompt template file. If specified, this template |--### Methods--#### BaseAgent.name--```python-def name() -> str-```--Return the name of the agent.--#### BaseAgent.perform_task--```python-async def perform_task(self, task_description: str, session: DesktopSession, logging_dir: Path | None = None, tracer = None) -> AgentResult-```--Perform a task using the agent.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `task_description` | `Any` | The task description/instruction |-| `session` | `Any` | The desktop or mobile session to interact with |-| `logging_dir` | `Any` | Optional directory for logging agent execution |-| `tracer` | `Any` | Optional tracer object for recording agent actions |--**Returns:** AgentResult with token counts and failure mode-------## FailureMode--*Inherits from: Enum*--Failure mode for agent execution.--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `UNSET` | `Any` | |-| `NONE` | `Any` | |-| `UNKNOWN` | `Any` | |-| `MAX_STEPS_EXCEEDED` | `Any` | |-------## CuaAgent--*Inherits from: BaseAgent*--Agent implementation using the Cua Computer Agent SDK.--### Constructor--```python-CuaAgent(self, kwargs = {})-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `model` | `Any` | |-| `max_steps` | `Any` | |--### Methods--#### CuaAgent.name--```python-def name() -> str-```--#### CuaAgent.perform_task--```python-async def perform_task(self, task_description: str, session: DesktopSession, logging_dir: Path | None = None, tracer = None) -> AgentResult-```--Perform a task using the Cua Computer Agent.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `task_description` | `Any` | The task description/instruction |-| `session` | `Any` | The desktop session to interact with |-| `logging_dir` | `Any` | Optional directory for logging agent execution |-| `tracer` | `Any` | Optional tracer object for recording agent actions |--**Returns:** AgentResult with token counts and failure mode-------## GeminiAgent--*Inherits from: BaseAgent*--Agent implementation using Google's Gemini API with Computer Use.--### Constructor--```python-GeminiAgent(self, kwargs = {})-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `model` | `Any` | |-| `api_key` | `Any` | |-| `thinking_level` | `Any` | |-| `media_resolution` | `Any` | |-| `max_steps` | `Any` | |--### Methods--#### GeminiAgent.name--```python-def name() -> str-```--#### GeminiAgent.perform_task--```python-async def perform_task(self, task_description: str, session: DesktopSession, logging_dir: Path | None = None, tracer = None) -> AgentResult-```--Perform a task using the Gemini Computer Use agent.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `task_description` | `Any` | The task description/instruction |-| `session` | `Any` | The desktop session to interact with |-| `logging_dir` | `Any` | Optional directory for logging agent execution |-| `tracer` | `Any` | Optional tracer object for recording agent actions |--**Returns:** AgentResult with token counts and failure mode--### register_agent--```python-def register_agent(name: str)-```--Decorator to register an agent class with a given name.--### load_agent_from_path--```python-def load_agent_from_path(import_path: str) -> type[BaseAgent]-```--Load an agent class from an import path.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `import_path` | `Any` | Import path in format 'module.path:ClassName' |--**Returns:** Agent class--**Raises:**--- `ValueError` - If import path format is invalid-- `ImportError` - If module cannot be imported-- `AttributeError` - If class is not found in module--### get_agent--```python-def get_agent(name: str, config_loader: 'ConfigLoader | None' = None) -> type[BaseAgent] | None-```--Get an agent class by name.--Lookup order:-1. Local registry (.cua/agents.yaml) - if config_loader provided-2. Built-in registry (_AGENT_REGISTRY)--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `name` | `Any` | Agent name to look up |-| `config_loader` | `Any` | Optional ConfigLoader for local registry lookup |--**Returns:** Agent class if found, None otherwise--### list_agents--```python-def list_agents(config_loader: 'ConfigLoader | None' = None) -> list[str]-```--List all registered agent names.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `config_loader` | `Any` | Optional ConfigLoader to include local agents |--**Returns:** List of agent names (local + built-in, deduplicated)-------## processors--Snapshot processors for converting batch outputs into various dataset formats.-------## AgUVisStage1Processor--*Inherits from: BaseProcessor*--Processor for aguvis-stage-1 format (action augmentation dataset).--### Methods--#### AgUVisStage1Processor.get_dataset_name--```python-def get_dataset_name(self) -> str-```--#### AgUVisStage1Processor.process--```python-def process(self) -> List[Dict[str, Any]]-```--Process snapshots into aguvis-stage-1 format.-------## BaseProcessor--*Inherits from: ABC*--Base class for snapshot processors.--A processor converts batch dump outputs (screenshots + snapshots)-into a specific dataset format.--### Constructor--```python-BaseProcessor(self, args: ProcessorArgs)-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `args` | `Any` | |--### Methods--#### BaseProcessor.process--```python-def process(self) -> List[Dict[str, Any]]-```--Process the snapshots and return a list of dataset rows.--**Returns:** List of dictionaries, where each dict is a row in the dataset. The schema depends on the specific processor implementation.--#### BaseProcessor.get_dataset_name--```python-def get_dataset_name(self) -> str-```--Get the default dataset name for this processor.--#### BaseProcessor.save_jsonl--```python-def save_jsonl(self, rows: List[Dict[str, Any]], save_dir: Path, dataset_name: str) -> Path-```--Save dataset rows as JSONL file.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `rows` | `Any` | List of dataset row dictionaries |-| `save_dir` | `Any` | Directory to save to |-| `dataset_name` | `Any` | Name of the dataset file (without extension) |--**Returns:** Path to the saved file--#### BaseProcessor.save_to_disk--```python-def save_to_disk(self, rows: List[Dict[str, Any]], save_dir: Path, dataset_name: str) -> Path-```--Save dataset rows using HuggingFace's save_to_disk method.--This method properly handles PIL images and other complex data types-that cannot be serialized to JSON.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `rows` | `Any` | List of dataset row dictionaries |-| `save_dir` | `Any` | Directory to save to |-| `dataset_name` | `Any` | Name of the dataset directory |--**Returns:** Path to the saved dataset directory--#### BaseProcessor.push_to_hub--```python-def push_to_hub(self, rows: List[Dict[str, Any]], repo_id: str, private: bool) -> None-```--Push dataset to Hugging Face Hub.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `rows` | `Any` | List of dataset row dictionaries |-| `repo_id` | `Any` | HuggingFace repository ID (e.g., "username/dataset-name") |-| `private` | `Any` | Whether to make the dataset private |-------## GuiR1Processor--*Inherits from: BaseProcessor*--Processor for gui-r1 format (low-level click instructions).--### Methods--#### GuiR1Processor.get_dataset_name--```python-def get_dataset_name(self) -> str-```--#### GuiR1Processor.process--```python-def process(self) -> List[Dict[str, Any]]-```--Process snapshots into gui-r1 format.--### get_processor--```python-def get_processor(name: str) -> type[BaseProcessor]-```--Get a processor class by name.-------## sessions--Sessions module for async container management.-------## SessionProvider--*Inherits from: ABC*--Base class for session providers (Docker, Cua Cloud, etc.).--### Methods--#### SessionProvider.start_session--```python-async def start_session(self, session_id: str, env_path: Path, container_script: str, image_uri: Optional[str] = None, output_dir: Optional[str] = None, kwargs = {}) -> Dict[str, Any]-```--Start a new session.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `session_id` | `Any` | Unique identifier for the session |-| `env_path` | `Any` | Path to the environment directory |-| `container_script` | `Any` | Script to run in the container |-| `image_uri` | `Any` | Container image to use |-| `output_dir` | `Any` | Directory to save outputs **kwargs: Additional provider-specific arguments |--**Returns:** Dict containing session metadata (container_id, status, etc.)--#### SessionProvider.get_session_status--```python-async def get_session_status(self, session_id: str) -> Dict[str, Any]-```--Get the status of a running session.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `session_id` | `Any` | Session identifier |--**Returns:** Dict containing session status information--#### SessionProvider.stop_session--```python-async def stop_session(self, session_id: str) -> None-```--Stop a running session.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `session_id` | `Any` | Session identifier |--#### SessionProvider.get_session_logs--```python-async def get_session_logs(self, session_id: str, tail: Optional[int] = None) -> str-```--Get logs from a session.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `session_id` | `Any` | Session identifier |-| `tail` | `Any` | Number of lines to return from the end (None for all) |--**Returns:** Log output as string--### list_sessions--```python-def list_sessions(provider: Optional[str] = None) -> List[Dict[str, Any]]-```--List all stored sessions.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `provider` | `Any` | Optional provider filter ("docker", "cua-cloud", etc.) |--**Returns:** List of session metadata dicts--### make--```python-def make(provider_name: str, env_type: Optional[str] = None) -> SessionProvider-```--Create a session provider for the specified provider.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `provider_name` | `Any` | Name of the provider: - "local": Run locally using Docker (webtop) or QEMU/KVM (winarena) - "cloud": Run on Cua Cloud (GCP Batch for webtop, Azure Batch for winarena) - "docker": (legacy) Alias for "local" |-| `env_type` | `Any` | Optional environment type hint ("webtop" or "winarena"). Used by local provider to select appropriate backend. |--**Returns:** SessionProvider instance--**Raises:**--- `ValueError` - If provider is not supported-------## batch--Batch integration for cua-bench.--### execute_batch--```python-async def execute_batch(job_name: str, env_path: Path, container_script: str, task_count: int = 4, task_parallelism: int = 4, run_local: bool = False, image_uri: Optional[str] = None, auto_cleanup: bool = True, output_dir: Optional[str] = None) -> List[str]-```--Execute a batch job for cua-bench environment.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `job_name` | `Any` | Name of the batch job |-| `env_path` | `Any` | Path to the environment directory |-| `container_script` | `Any` | Script to run in the container |-| `task_count` | `Any` | Number of tasks to run |-| `task_parallelism` | `Any` | Max concurrent tasks |-| `run_local` | `Any` | Run locally using Docker instead of GCP |-| `image_uri` | `Any` | Custom container image |-| `auto_cleanup` | `Any` | Clean up resources after completion |--**Returns:** List of log lines from the job--### run_local_docker--```python-async def run_local_docker(env_path: Path, container_script: str, image_uri: Optional[str] = None, output_dir: Optional[str] = None, task_count: int = 1, parallelism: int = 1) -> List[str]-```--Run the batch job locally using Docker.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `env_path` | `Any` | Path to environment directory |-| `container_script` | `Any` | Script to run |-| `image_uri` | `Any` | Docker image to use |-| `output_dir` | `Any` | Local directory to mount as /tmp/td_output for results |-| `task_count` | `Any` | Total number of tasks to run |-| `parallelism` | `Any` | Maximum number of concurrent containers |--**Returns:** List of output lines-------## workers--Worker-based gym system for parallel environment management.--This module provides a FastAPI-based worker system for running CUA-Bench-environments in parallel, enabling efficient RL training and evaluation.--Components:-- worker_server: FastAPI server wrapping Environment instances-- worker_client: HTTP client for interacting with worker servers-- worker_manager: Utilities for spawning and managing multiple workers-- dataloader: MultiTurnDataloader and ReplayBuffer for RL training-------## MultiTurnDataloader--Dataloader for RL training with parallel environment workers.--Each env_config must contain a 'task_configs' key with a list of task-configurations that the client will use internally.--### Constructor--```python-MultiTurnDataloader(self, env_class, env_configs, tokenizer, processor = None, is_multi_modal = True, batch_size = 8, replay_capacity = 10000, replay_reward_discount = 0.9, max_prompt_length = 1024, max_response_length = 1024, only_keep_outcome_in_replay = False)-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `num_envs` | `Any` | |-| `batch_size` | `Any` | |-| `replay` | `Any` | |--### Methods--#### MultiTurnDataloader.async_step--```python-def async_step(self, batch_return)-```--#### MultiTurnDataloader.sample_from_buffer--```python-def sample_from_buffer(self, batch_size = None)-```--#### MultiTurnDataloader.clear_replay_buffer--```python-def clear_replay_buffer(self)-```--#### MultiTurnDataloader.get_balance_stats--```python-def get_balance_stats(self)-```--#### MultiTurnDataloader.calculate_outcome_reward--```python-def calculate_outcome_reward(self)-```--#### MultiTurnDataloader.print_examples--```python-def print_examples(self, n = 2)-```--#### MultiTurnDataloader.print_stats_in_replay_buffer--```python-def print_stats_in_replay_buffer(self)-```--#### MultiTurnDataloader.running_outcome_reward--```python-def running_outcome_reward(self)-```--#### MultiTurnDataloader.close--```python-def close(self)-```--Close all workers and clean up resources.-------## ReplayBuffer--### Constructor--```python-ReplayBuffer(self, capacity = 10000, gamma = 1.0, only_keep_outcome = False, balance_thres = 0.1)-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `capacity` | `Any` | |-| `gamma` | `Any` | |-| `only_keep_outcome` | `Any` | |-| `balance_thres` | `Any` | |-| `ready_buffer` | `Any` | |-| `ready_position` | `Any` | |-| `ready_count` | `Any` | |-| `episode_buffer` | `Any` | |--### Methods--#### ReplayBuffer.add--```python-def add(self, data)-```--Add data to the replay buffer--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `data` | `tuple` | A tuple of (worker_id, env_ret, meta_info) |--#### ReplayBuffer.get_balance_stats--```python-def get_balance_stats(self)-```--#### ReplayBuffer.should_keep--```python-def should_keep(self, curr_below, curr_above, curr_ret)-```--#### ReplayBuffer.sample--```python-def sample(self, batch_size)-```--Sample experiences from the ready buffer--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `batch_size` | `int` | Number of experiences to sample |--**Returns:** list: List of sampled experiences--#### ReplayBuffer.clear--```python-def clear(self)-```--Clear both ready buffer and episode buffer-------## CBEnvWorkerClient--HTTP client for CUA-Bench worker servers.--This client manages communication with the worker server, image processing,-observation history tracking, and action normalization.--Args:- env_config: Configuration dict with keys:- - server_url: URL of the worker server- - task_configs: List of task configs, each with env_path, task_index, split- - img_w: Image width (default: 1920)- - img_h: Image height (default: 1080)- - max_step: Maximum steps per episode (default: 50)- - max_hist: Maximum observation history length (default: 10)- - timeout: Environment timeout in seconds (default: 300)--### Constructor--```python-CBEnvWorkerClient(self, env_config)-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `vision_start_token` | `Any` | |-| `vision_end_token` | `Any` | |-| `think_start_token` | `Any` | |-| `think_end_token` | `Any` | |-| `action_start_token` | `Any` | |-| `action_end_token` | `Any` | |-| `valid_fn_names` | `Any` | |-| `vlm_img_w` | `Any` | |-| `vlm_img_h` | `Any` | |-| `dynamic_img_size` | `Any` | |-| `env_config` | `Any` | |-| `server_url` | `Any` | |-| `max_step` | `Any` | |-| `max_hist` | `Any` | |-| `task_configs` | `List[Dict[str, Any]]` | |-| `img_h` | `Any` | |-| `img_w` | `Any` | |-| `timeout` | `Any` | |-| `env_id` | `Any` | |-| `uid` | `Any` | |-| `step_count` | `Any` | |-| `done` | `Any` | |-| `prompt` | `Any` | |--### Methods--#### CBEnvWorkerClient.reset--```python-def reset(self)-```--#### CBEnvWorkerClient.reset_attempt--```python-def reset_attempt(self)-```--#### CBEnvWorkerClient.prompt_to_input_obs--```python-def prompt_to_input_obs(self, prompt)-```--#### CBEnvWorkerClient.check_and_fix_action--```python-def check_and_fix_action(self, action_str)-```--Parse action string and return (normalized_str, Action object for server).--#### CBEnvWorkerClient.reward_shaping--```python-def reward_shaping(self, reward)-```--#### CBEnvWorkerClient.check_and_resize_image--```python-def check_and_resize_image(self, jpg_string)-```--#### CBEnvWorkerClient.step--```python-def step(self, action)-```--#### CBEnvWorkerClient.step_attempt--```python-def step_attempt(self, action)-```--#### CBEnvWorkerClient.render--```python-def render(self)-```--Renders the current state in self.prompt as a sequence of text-image pairs into a single image--**Returns:** PIL.Image: Combined image showing the instruction and interaction history-------## WorkerHandle--Handle for a running worker server.--Attributes:- worker_id: Unique identifier for this worker- port: Port the worker is listening on- process: Subprocess running the worker- api_url: Full URL for API requests--### Constructor--```python-WorkerHandle(self, worker_id: str, port: int, process: subprocess.Popen, api_url: str) -> None-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `worker_id` | `str` | |-| `port` | `int` | |-| `process` | `subprocess.Popen` | |-| `api_url` | `str` | |-| `is_running` | `bool` | Check if the worker process is still running. |--### Methods--#### WorkerHandle.health_check--```python-async def health_check(self, timeout: float = 5.0) -> bool-```--Check if the worker is healthy.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `timeout` | `Any` | Request timeout in seconds |--**Returns:** True if healthy, False otherwise--#### WorkerHandle.stop--```python-def stop(self) -> None-```--Stop the worker process.-------## WorkerPool--Context manager for a pool of worker servers.--Example:- async with WorkerPool(n_workers=4, allowed_ips=["127.0.0.1"]) as pool:- for url in pool.urls:- client = CBEnvWorkerClient(\{ "server_url": url \})- # Use client...--### Constructor--```python-WorkerPool(self, n_workers: int, allowed_ips: List[str], startup_timeout: float = 30.0, host: str = '0.0.0.0')-```--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `n_workers` | `Any` | |-| `allowed_ips` | `Any` | |-| `startup_timeout` | `Any` | |-| `host` | `Any` | |-| `workers` | `List[WorkerHandle]` | Get the list of worker handles. |-| `urls` | `List[str]` | Get the list of worker URLs. |--### Methods--#### WorkerPool.health_check_all--```python-async def health_check_all(self) -> dict-```--Check health of all workers.--**Returns:** Dict mapping worker_id to health status--### cleanup_workers--```python-async def cleanup_workers(workers: List[WorkerHandle]) -> None-```--Stop all workers.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `workers` | `Any` | List of WorkerHandle objects to stop |--### create_workers--```python-async def create_workers(n_workers: int, allowed_ips: List[str], startup_timeout: float = 30.0, host: str = '0.0.0.0') -> List[WorkerHandle]-```--Spawn N worker servers on automatically allocated free ports.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `n_workers` | `Any` | Number of worker servers to spawn |-| `allowed_ips` | `Any` | List of IPs allowed to access workers |-| `startup_timeout` | `Any` | Max time to wait for each worker to become healthy |-| `host` | `Any` | Host for workers to bind to |--**Returns:** List of WorkerHandle objects--**Raises:**--- `RuntimeError` - If any worker fails to start--**Example:**--```python-workers = await create_workers(- n_workers=4,- allowed_ips=["127.0.0.1", "10.0.0.5"],-)-# Each worker manages up to 2 envs, so 4 workers = 8 parallel envs-```-------## telemetry--Telemetry module for cua-bench.--This module provides analytics for tracking feature usage, user workflows,-and system performance. All telemetry is routed through cua-core's PostHog-infrastructure for consistency across the Cua ecosystem.--Events tracked:-- Tier 1 (Core): command_invoked, task_execution_started, task_evaluation_completed, batch_job_started-- Tier 2 (High Value): task_step_executed, batch_task_completed, dataset_processing_completed, task_execution_failed--Usage:- from cua_bench.telemetry import record_event, track_command-- # Track CLI command usage- @track_command- def my_command(args):- ...-- # Track custom events- record_event("custom_event", \{"property": "value"\})--Environment Variables:- CUA_TELEMETRY_ENABLED: Set to "false" to disable telemetry (default: "true")- CUA_TELEMETRY_DEBUG: Set to "on" for debug logging--### flush_telemetry--```python-def flush_telemetry() -> None-```--Flush pending telemetry events.--Delegates to cua-core's PostHog client.--### is_telemetry_enabled--```python-def is_telemetry_enabled() -> bool-```--Check if telemetry is enabled.--Delegates to cua-core's telemetry check.--### record_event--```python-def record_event(event_name: str, properties: Optional[Dict[str, Any]] = None) -> None-```--Record a telemetry event.--Routes through cua-core's telemetry infrastructure.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `event_name` | `Any` | Name of the event (e.g., "cb_command_invoked") |-| `properties` | `Any` | Optional dict of event properties |--### track_batch_job_started--```python-def track_batch_job_started(dataset_name: str, task_count: int, variant_count: int, parallelism: int = 1, agent: Optional[str] = None, model: Optional[str] = None, run_id: Optional[str] = None, provider_type: Optional[str] = None) -> None-```--Track batch job start.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `dataset_name` | `Any` | Name of the dataset |-| `task_count` | `Any` | Number of unique tasks |-| `variant_count` | `Any` | Total variants to run |-| `parallelism` | `Any` | Max parallel workers |-| `agent` | `Any` | Agent name if specified |-| `model` | `Any` | Model name if specified |-| `run_id` | `Any` | Run ID for correlation |-| `provider_type` | `Any` | Provider type |--### track_batch_task_completed--```python-def track_batch_task_completed(env_name: str, task_index: int, success: bool, reward: Optional[float] = None, total_steps: int = 0, duration_seconds: float = 0, run_id: Optional[str] = None, error: Optional[str] = None) -> None-```--Track individual task completion in batch.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `env_name` | `Any` | Name of the environment/task |-| `task_index` | `Any` | Task variant index |-| `success` | `Any` | Whether task succeeded |-| `reward` | `Any` | Reward/score if available |-| `total_steps` | `Any` | Steps taken |-| `duration_seconds` | `Any` | Task duration |-| `run_id` | `Any` | Run ID for correlation |-| `error` | `Any` | Error message if failed |--### track_command--```python-def track_command(func: Callable) -> Callable-```--Decorator to track command invocation.--Usage:- @track_command- def cmd_run_task(args):- ...--### track_command_async--```python-def track_command_async(func: Callable) -> Callable-```--Async decorator to track command invocation.--### track_command_invoked--```python-def track_command_invoked(command: str, subcommand: Optional[str] = None, args: Optional[Dict[str, Any]] = None) -> None-```--Track CLI command invocation.--This is the primary event for understanding feature usage.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `command` | `Any` | Main command (e.g., "run", "interact", "trace") |-| `subcommand` | `Any` | Optional subcommand (e.g., "task", "dataset", "list") |-| `args` | `Any` | Optional sanitized arguments (no sensitive data) |--### track_dataset_processing_completed--```python-def track_dataset_processing_completed(processor_mode: str, rows_processed: int, duration_seconds: float, success: bool = True, output_format: Optional[str] = None) -> None-```--Track dataset processing completion.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `processor_mode` | `Any` | Processing mode (aguvis-stage-1, gui-r1, etc.) |-| `rows_processed` | `Any` | Number of rows processed |-| `duration_seconds` | `Any` | Processing duration |-| `success` | `Any` | Whether processing succeeded |-| `output_format` | `Any` | Output format (disk, hub, jsonl) |--### track_task_evaluation_completed--```python-def track_task_evaluation_completed(env_name: str, task_index: int, result: Any, success: bool, total_steps: int, duration_seconds: float, run_id: Optional[str] = None, agent: Optional[str] = None, model: Optional[str] = None) -> None-```--Track task evaluation completion.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `env_name` | `Any` | Name of the environment/task |-| `task_index` | `Any` | Task variant index |-| `result` | `Any` | Evaluation result (reward/score) |-| `success` | `Any` | Whether task was successful |-| `total_steps` | `Any` | Total steps taken |-| `duration_seconds` | `Any` | Total duration in seconds |-| `run_id` | `Any` | Run ID for correlation |-| `agent` | `Any` | Agent name if used |-| `model` | `Any` | Model name if used |--### track_task_execution_failed--```python-def track_task_execution_failed(env_name: str, task_index: int, error_type: str, error_message: str, stage: str, run_id: Optional[str] = None) -> None-```--Track task execution failure.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `env_name` | `Any` | Name of the environment/task |-| `task_index` | `Any` | Task variant index |-| `error_type` | `Any` | Exception class name |-| `error_message` | `Any` | Error message (truncated) |-| `stage` | `Any` | Stage where error occurred |-| `run_id` | `Any` | Run ID for correlation |--### track_task_execution_started--```python-def track_task_execution_started(env_name: str, task_index: int, provider_type: Optional[str] = None, os_type: Optional[str] = None, agent: Optional[str] = None, model: Optional[str] = None, max_steps: Optional[int] = None, execution_mode: str = 'single', run_id: Optional[str] = None) -> None-```--Track task execution start.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `env_name` | `Any` | Name of the environment/task |-| `task_index` | `Any` | Task variant index |-| `provider_type` | `Any` | Provider type (simulated, webtop, native, computer) |-| `os_type` | `Any` | OS type (linux, windows, android) |-| `agent` | `Any` | Agent name if specified |-| `model` | `Any` | Model name if specified |-| `max_steps` | `Any` | Max steps budget |-| `execution_mode` | `Any` | Execution mode (single, batch, interactive) |-| `run_id` | `Any` | Run ID for correlation |--### track_task_step_executed--```python-def track_task_step_executed(action_type: str, step_count: int, duration_ms: Optional[float] = None, run_id: Optional[str] = None) -> None-```--Track individual step execution.--Note: This should be sampled to avoid high event volume.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `action_type` | `Any` | Type of action (ClickAction, TypeAction, etc.) |-| `step_count` | `Any` | Current step number |-| `duration_ms` | `Any` | Step duration in milliseconds |-| `run_id` | `Any` | Run ID for correlation |-------## apps--App Registry for cua-bench.--A decorator-based API for registering platform-specific app installers and launchers.-Makes it easy for contributors to add support for new applications.--Example - Defining an app:-- # cua_bench/apps/godot.py- from cua_bench.apps import App, install, launch-- class Godot(App):- name = "godot"- description = "Godot game engine"-- @install("linux")- async def install_linux(session, *, with_shortcut=True, version="4.2.1"):- await session.run_command(- f"cd ~/Desktop && "- f"wget -q https://github.com/godotengine/godot/releases/download/\{version\}-stable/Godot_v\{version\}-stable_linux.x86_64.zip && "- f"unzip -q Godot_v\{version\}-stable_linux.x86_64.zip"- )- if with_shortcut:- await session.run_command(- "ln -sf ~/Desktop/Godot_v*_linux.x86_64 ~/Desktop/Godot"- )-- @install("windows")- async def install_windows(session, *, with_shortcut=True, version="4.2.1"):- await session.run_command(f"choco install godot --version=\{version\} -y")-- @launch("linux", "windows")- async def launch_editor(session, *, project_path=None):- cmd = "~/Desktop/Godot" if session.os_type == "linux" else "godot"- if project_path:- cmd += f" --editor --path \{project_path\}"- await session.run_command(f"\{cmd\} &")--Example - Using in a task:-- @cb.setup_task(split="train")- async def start(task_cfg: cb.Task, session: cb.DesktopSession):- # Install app (auto-selects platform)- await session.install_app("godot", with_shortcut=True, version="4.2.1")-- # Launch app- await session.launch_app("godot", project_path="~/project")-------## App--Base class for app definitions.--Subclass this and define platform-specific methods using decorators:-- class MyApp(App):- name = "myapp"- description = "My application"-- @install("linux")- async def install_linux(session, **kwargs):- ...-- @install("windows")- async def install_windows(session, **kwargs):- ...-- @launch("linux", "windows")- async def launch(session, **kwargs):- ...--### Attributes--| Name | Type | Description |-|------|------|-------------|-| `name` | `str` | |-| `description` | `str` | |--### Methods--#### App.get_method--```python-def get_method(self, method_type: str, platform: Platform) -> Optional[AppMethod]-```--Get a method for the given type and platform.--#### App.get_install--```python-def get_install(self, platform: Platform) -> Optional[AppMethod]-```--Get the install method for a platform.--#### App.get_launch--```python-def get_launch(self, platform: Platform) -> Optional[AppMethod]-```--Get the launch method for a platform.--#### App.get_uninstall--```python-def get_uninstall(self, platform: Platform) -> Optional[AppMethod]-```--Get the uninstall method for a platform.--#### App.supported_platforms--```python-def supported_platforms(self, method_type: str = 'install') -> Set[Platform]-```--Get platforms supported for a method type.-------## AppRegistry--Registry access for DesktopSession integration.--This class provides the interface used by DesktopSession to install/launch apps.--### Methods--#### AppRegistry.install_app--```python-async def install_app(session: Any, app_name: str, with_shortcut: bool = True, kwargs = {}) -> None-```--Install an app on the session's platform.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `session` | `Any` | DesktopSession instance |-| `app_name` | `Any` | Name of the app to install |-| `with_shortcut` | `Any` | Whether to create desktop shortcut (default True) **kwargs: Additional app-specific arguments |--#### AppRegistry.launch_app--```python-async def launch_app(session: Any, app_name: str, kwargs = {}) -> None-```--Launch an app on the session's platform.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `session` | `Any` | DesktopSession instance |-| `app_name` | `Any` | Name of the app to launch **kwargs: App-specific launch arguments |--#### AppRegistry.uninstall_app--```python-async def uninstall_app(session: Any, app_name: str, kwargs = {}) -> None-```--Uninstall an app from the session's platform.--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `session` | `Any` | DesktopSession instance |-| `app_name` | `Any` | Name of the app to uninstall **kwargs: App-specific arguments |--### get_app--```python-def get_app(name: str) -> Optional[App]-```--Get a registered app by name.--### list_apps--```python-def list_apps() -> List[str]-```--List all registered app names.diff --git a/docs/content/docs/cuabench/reference/cli-reference.mdx b/docs/content/docs/cuabench/reference/cli-reference.mdxdeleted file mode 100644index 78e8d9aec5..0000000000--- a/docs/content/docs/cuabench/reference/cli-reference.mdx+++ /dev/null@@ -1,326 +0,0 @@-----title: CLI Reference-description: Complete reference for all cua-bench CLI commands------import { VersionBadge } from '@/components/version-selector';--<VersionBadge version="0.2.3" />--## Image Commands--Create and manage base images for environments.--```bash-cb image <action> [options]-```--| Command | Description |-| ---------------------------- | ---------------------------------------- |-| `cb image create <platform>` | Create a base image |-| `cb image list` | List all images |-| `cb image info <name>` | Show image details |-| `cb image clone <src> <dst>` | Clone/fork an image |-| `cb image delete <name>` | Delete an image |-| `cb image shell <name>` | Interactive shell into image (protected) |--### Create Options--```bash-cb image create <platform> [options]-```--| Option | Description |-| ------------------- | ---------------------------------------------------------------- |-| `--download-iso` | Download Windows 11 ISO (~6GB) |-| `--iso <path>` | Use existing Windows ISO |-| `--name <name>` | Image name (default: same as platform) |-| `--winarena-apps` | Install WinArena benchmark apps (Chrome, LibreOffice, VLC, etc.) |-| `--memory <size>` | Memory (default: 8G) |-| `--cpus <count>` | CPU cores (default: 8) |-| `--disk <size>` | Disk size (default: 64G) |-| `--detach`, `-d` | Run in background |-| `--force` | Force recreation |-| `--skip-pull` | Don't pull Docker image (use local) |-| `--vnc-port <port>` | VNC port (default: auto-allocate from 8006) |-| `--api-port <port>` | API port (default: auto-allocate from 5000) |--### List Options--```bash-cb image list [options]-```--| Option | Description |-| ------------------- | -------------------------- |-| `--format <fmt>` | Output format: table, json |-| `--platform <type>` | Filter by platform type |--### Shell Options--Interactive shell into an image. By default uses an overlay to protect the golden image.--```bash-cb image shell <name> [options]-```--| Option | Description |-| ------------------- | ------------------------------------------- |-| `--writable` | Modify golden image directly (dangerous!) |-| `--vnc-port <port>` | VNC port (default: auto-allocate from 8006) |-| `--api-port <port>` | API port (default: auto-allocate from 5000) |-| `--memory <size>` | Memory (default: 8G) |-| `--cpus <count>` | CPU cores (default: 8) |-| `--no-kvm` | Disable KVM acceleration |-| `--detach`, `-d` | Run in background |--<Callout title="Image Protection" type="info">- By default, `cb image shell` mounts the golden image read-only and uses an overlay for any- changes. This protects your base images from accidental modification. Use `--writable` only when- you intentionally want to modify the golden image.-</Callout>--## Platform Commands--View available platforms.--```bash-cb platform list [options]-```--| Option | Description |-| ---------------- | -------------------------- |-| `--format <fmt>` | Output format: table, json |--## Status Commands--Show system status.--```bash-cb status-```--Shows Docker status, KVM availability, running shells, and active runs.--## Run Commands--Run tasks with the 2-container architecture (agent container + environment container).--```bash-cb run <subcommand> [options]-```--| Command | Description |-| ----------------------- | ---------------------------------------- |-| `cb run task <path>` | Run a single task |-| `cb run dataset <path>` | Run all tasks in a dataset (parallel) |-| `cb run list` | List all runs with status |-| `cb run status <id>` | Show detailed run status |-| `cb run watch <id>` | Watch a run in real-time |-| `cb run stop <id>` | Stop/cancel a run |-| `cb run logs <id>` | View combined logs from a run or session |--### Task Options--Tasks run **asynchronously by default** and return immediately with a run ID. Use `--wait` for synchronous execution.--```bash-cb run task <path> [options]-```--| Option | Description |-| ---------------------------- | --------------------------------------------------------------------- |-| `--wait`, `-w` | Wait for task completion (default: run async) |-| `--debug` | Auto-allocate debug ports (VNC + API) |-| `--variant-id <n>` | Task variant index (default: 0) |-| `--agent <name>` | Agent to use (from `.cua/agents.yaml`) |-| `--agent-import-path <path>` | Import path for custom agent |-| `--model <model>` | Model to use (e.g., `anthropic/claude-sonnet-4-20250514`) |-| `--oracle` | Run with oracle solution (no agent) |-| `--max-steps <n>` | Maximum steps per task (default: 100) |-| `--image <name>` | Use specific base image |-| `--platform <type>` | Platform type: `linux-docker`, `windows-qemu`, `android-qemu` |-| `--vnc-port <port>` | Expose VNC on host port (auto-allocated if not specified) |-| `--api-port <port>` | Expose API on host port (auto-allocated if not specified) |-| `--output-dir <dir>` | Output directory (default: `~/.local/share/cua-bench/runs/<run_id>/`) |--### Dataset Options--```bash-cb run dataset <path> [options]-```--| Option | Description |-| ------------------------- | ------------------------------------------ |-| `--max-parallel <n>` | Maximum parallel task runners (default: 4) |-| `--max-variants <n>` | Maximum variants per task (default: all) |-| `--task-filter <pattern>` | Filter tasks by name pattern (glob) |-| `--agent <name>` | Agent to use (from `.cua/agents.yaml`) |-| `--model <model>` | Model to use |-| `--oracle` | Run with oracle solution (no agent) |-| `--max-steps <n>` | Maximum steps per task (default: 100) |-| `--image <name>` | Use specific base image |-| `--platform <type>` | Platform type |-| `--output-dir <dir>` | Output directory for results |--### Examples--```bash-# Run single task (async by default - returns immediately)-cb run task tasks/click_button --agent cua-agent --model anthropic/claude-sonnet-4-20250514--# Wait for task completion (synchronous)-cb run task tasks/click_button --agent cua-agent --wait--# Run with oracle solution-cb run task tasks/click_button --variant-id 0 --oracle --wait--# Auto-allocate debug ports for live viewing-cb run task tasks/click_button --agent cua-agent --debug-# Output shows: Auto-allocated debug ports: VNC=8006, API=5000--# Run Windows task with specific image-cb run task tasks/winarena --image windows-qemu --agent cua-agent--# Run all tasks in a dataset (4 parallel)-cb run dataset datasets/cua-bench-basic --agent cua-agent --max-parallel 4--# Run dataset with filtering-cb run dataset datasets/cua-bench-basic --task-filter "click*" --max-variants 1--# Monitor runs (async tasks)-cb run list # List all runs-cb run watch <run_id> # Watch progress in real-time-cb run status <run_id> # Check status-cb run logs <run_id> # View logs-cb run logs <run_id> --tail 100 # View last 100 lines-cb run stop <run_id> # Stop a run-```--## Interact Commands--Interactive task mode with visible browser.--```bash-cb interact <task> [options]-```--| Option | Description |-| ----------------------- | ----------------------------------------------------------- |-| `--variant-id <id>` | Task variant index (default: 0) |-| `--dataset <name>` | Dataset name to resolve task from registry |-| `--dataset-path <path>` | Path to dataset directory containing multiple tasks |-| `--oracle` | Run the solution after setup |-| `--max-steps <n>` | Maximum number of env.step() calls before stopping |-| `--screenshot <path>` | Save screenshot to file |-| `--trace-out <path>` | If set, start tracing and save dataset to this path on exit |-| `--view` | Open a trace viewer when done |-| `--no-wait` | Skip the interactive prompt (useful for SSH/CI testing) |--## Task Commands--Manage task environments.--```bash-cb task <action> [options]-```--| Command | Description |-| ----------------------------- | -------------------------------------- |-| `cb task info <path>` | Show task details (provider, variants) |-| `cb task list [path]` | List tasks in a directory |-| `cb task create [path]` | Scaffold a new task environment |-| `cb task generate "<prompt>"` | Generate a task using Claude |--### Generate Options--```bash-cb task generate "<description>" [options]-```--| Option | Description |-| ----------------------- | ------------------------------------------------------------------- |-| `--output <path>`, `-o` | Output directory path. If not provided, auto-generates from prompt. |-| `--no-interaction` | Skip prompts and run Claude non-interactively |--## Trace Commands--View and manage trace datasets.--```bash-cb trace <action> <id>-```--| Command | Description |-| -------------------- | -------------------------------------------------------------------- |-| `cb trace view <id>` | View a single trace in browser (accepts run_id or session_id) |-| `cb trace traj <id>` | View agent trajectories from a run in cua.ai/trajectory-viewer |--## Dataset Commands--Manage datasets and build from outputs.--```bash-cb dataset <action> [options]-```--| Command | Description |-| ---------------------------- | ------------------------------------- |-| `cb dataset list` | List available datasets from registry |-| `cb dataset build <outputs>` | Build a dataset from batch outputs |--### Build Options--```bash-cb dataset build <outputs> [options]-```--| Option | Description |-| ------------------ | -------------------------------------- |-| `--save-dir <dir>` | Output directory for datasets |-| `--mode <mode>` | Processor mode: aguvis-stage-1, gui-r1 |-| `--push-to-hub` | Push to Hugging Face Hub |-| `--repo-id <id>` | Hugging Face repo ID |-| `--private` | Create private repo on Hub |--## Prune Commands--Clean up cua-bench data, images, and Docker resources.--```bash-cb prune [options]-```--When called without options, shows an interactive overview of storage usage. Use flags to clean specific resources.--| Option | Description |-| --------------- | -------------------------------------------------- |-| `--all`, `-a` | Remove everything (images, overlays, runs, docker) |-| `--images` | Remove only stored images |-| `--overlays` | Remove only task overlays |-| `--runs` | Remove only run logs and registry |-| `--docker` | Remove only docker containers/images |-| `--dry-run` | Show what would be deleted without deleting |-| `--force`, `-f` | Skip confirmation prompts |--### Examples--```bash-# Show storage overview-cb prune--# Remove all data with confirmation-cb prune --all--# Remove all data without confirmation-cb prune --all --force--# Preview what would be deleted-cb prune --all --dry-run--# Remove only run logs-cb prune --runs--# Remove docker resources-cb prune --docker-```diff --git a/docs/content/docs/cuabench/reference/meta.json b/docs/content/docs/cuabench/reference/meta.jsondeleted file mode 100644index b1238bf890..0000000000--- a/docs/content/docs/cuabench/reference/meta.json+++ /dev/null@@ -1,6 +0,0 @@-{- "title": "Reference",- "description": "CLI and API reference",- "icon": "FileText",- "pages": ["cli-reference", "api"]-}diff --git a/docs/content/docs/cuabench/reference/sdk-reference.mdx b/docs/content/docs/cuabench/reference/sdk-reference.mdxdeleted file mode 100644index 678eeb4223..0000000000--- a/docs/content/docs/cuabench/reference/sdk-reference.mdx+++ /dev/null@@ -1,270 +0,0 @@-----title: SDK Reference-description: Session interface for interacting with desktop environments------The cua-bench Session interface provides methods for interacting with desktop environments, whether they're simulated (Playwright-based) or native (Docker/QEMU-based).--## Quick Example--```python-from cua_bench import make--# Load an environment-env = make("./my_task")--# Reset to start a task-screenshot, task_config = await env.reset(task_id=0)--# Get the session from the environment-session = env.session--# Take actions-await session.click(100, 200)-await session.type("Hello, world!")-await session.key("Enter")--# Take a screenshot-screenshot_bytes = await session.screenshot()--# Close when done-await env.close()-```-------## Session Interface--The `DesktopSession` protocol defines the interface for interacting with desktop environments.--### Lifecycle Methods--```python-# Async context manager (preferred)-async with get_session("native")(os_type="linux") as session:- await session.screenshot()--# Manual lifecycle-session = get_session("native")(os_type="linux")-await session.start()-try:- await session.screenshot()-finally:- await session.close()-```--| Method | Description |-| --------------------------- | ------------------------------------------------ |-| `start(config?, headless?)` | Start the session and connect to the environment |-| `close()` | Close the session and cleanup resources |-------### Screenshot & State--```python-# Capture screenshot-screenshot_bytes = await session.screenshot() # PNG bytes--# Get desktop state snapshot-snapshot = await session.get_snapshot()-# Returns: Snapshot(windows: List[WindowSnapshot])-```--| Method | Return Type | Description |-| ---------------- | ----------- | ----------------------------------------------------------------------- |-| `screenshot()` | `bytes` | Capture screen as PNG bytes |-| `get_snapshot()` | `Snapshot` | Get lightweight snapshot of desktop state (windows, geometry, metadata) |-------### Mouse Actions--```python-# Click actions-await session.click(x, y) # Left click-await session.right_click(x, y) # Right click-await session.double_click(x, y) # Double click-await session.middle_click(x, y) # Middle click--# Movement and drag-await session.move_to(x, y) # Move cursor-await session.drag(from_x, from_y, to_x, to_y) # Drag gesture--# Scroll-await session.scroll(direction="down", amount=100) # Scroll up/down-```--| Method | Parameters | Description |-| ---------------------------------- | ---------------------------------------------- | --------------------------------- |-| `click(x, y)` | x: int, y: int | Left click at coordinates |-| `right_click(x, y)` | x: int, y: int | Right click at coordinates |-| `double_click(x, y)` | x: int, y: int | Double click at coordinates |-| `middle_click(x, y)` | x: int, y: int | Middle click at coordinates |-| `move_to(x, y)` | x: int, y: int | Move cursor to coordinates |-| `drag(from_x, from_y, to_x, to_y)` | from_x: int, from_y: int, to_x: int, to_y: int | Drag from one position to another |-| `scroll(direction, amount)` | direction: "up" \| "down", amount: int | Scroll in given direction |-------### Keyboard Actions--```python-# Type text-await session.type("Hello, world!")--# Press single key-await session.key("Enter")-await session.key("Escape")-await session.key("Tab")--# Key combinations (hotkeys)-await session.hotkey(["ctrl", "c"]) # Copy-await session.hotkey(["ctrl", "v"]) # Paste-await session.hotkey(["ctrl", "shift", "t"]) # Multiple modifiers-```--| Method | Parameters | Description |-| -------------- | --------------- | --------------------------------------------------- |-| `type(text)` | text: str | Type a string of text |-| `key(key)` | key: str | Press a single key (e.g., "Enter", "Escape", "Tab") |-| `hotkey(keys)` | keys: List[str] | Press a key combination (e.g., ["ctrl", "c"]) |-------### Window Management--```python-# Launch a window-pid = await session.launch_window(- url="https://example.com",- title="My Window",- width=800,- height=600-)--# Execute JavaScript in a window-result = await session.execute_javascript(pid, "document.title")--# Get element position within a window-rect = await session.get_element_rect(pid, ".button-class")-# Returns: {"x": 10, "y": 20, "width": 100, "height": 30}--# Close all windows-await session.close_all_windows()-```--| Method | Parameters | Return Type | Description |-| ------------------------------------------ | ----------------------------------- | -------------- | ----------------------------------------------- |-| `launch_window(url?, html?, folder?, ...)` | Various options for window creation | `int \| str` | Launch a window and return its process ID |-| `execute_javascript(pid, js)` | pid: int \| str, js: str | `Any` | Execute JavaScript in a specific window |-| `get_element_rect(pid, selector)` | pid: int \| str, selector: str | `dict \| None` | Get element rectangle in window or screen space |-| `close_all_windows()` | - | `None` | Close or clear all open windows |-------### Action System--For more advanced use cases, you can use the action dataclasses directly:--```python-from cua_bench.types import (- ClickAction,- RightClickAction,- DoubleClickAction,- MiddleClickAction,- DragAction,- MoveToAction,- ScrollAction,- TypeAction,- KeyAction,- HotkeyAction,- WaitAction,- DoneAction,-)--# Execute an action directly-await session.execute_action(ClickAction(x=100, y=200))-await session.execute_action(TypeAction(text="Hello"))-await session.execute_action(HotkeyAction(keys=["ctrl", "c"]))-```--**Available Action Types:**--| Action Class | Fields | Description |-| ------------------- | --------------------------------------------------------------- | ------------------------ |-| `ClickAction` | x: int, y: int | Left click at position |-| `RightClickAction` | x: int, y: int | Right click at position |-| `DoubleClickAction` | x: int, y: int | Double click at position |-| `MiddleClickAction` | x: int, y: int | Middle click at position |-| `DragAction` | from_x: int, from_y: int, to_x: int, to_y: int, duration: float | Drag gesture |-| `MoveToAction` | x: int, y: int, duration: float | Move cursor |-| `ScrollAction` | direction: "up" \| "down", amount: int | Scroll action |-| `TypeAction` | text: str | Type text |-| `KeyAction` | key: str | Press single key |-| `HotkeyAction` | keys: List[str] | Key combination |-| `WaitAction` | seconds: float | Wait/pause |-| `DoneAction` | - | Signal completion |-------## Provider Types--cua-bench supports two provider types:--### Simulated Desktop (Playwright-based)--- **Name:** `"simulated"` or `"webtop"`-- **Description:** HTML/CSS desktop simulation in headless browser-- **Pros:** Fast, no Docker required-- **Use cases:** Web app testing, UI benchmarks--```python-from cua_bench.computers.base import get_session--SessionClass = get_session("simulated")-session = SessionClass(env=env)-```--### Native Desktop (Docker/QEMU-based)--- **Name:** `"native"` or `"computer"`-- **Description:** Real OS in Docker/QEMU container-- **Pros:** Actual desktop environment with real applications-- **Requires:** Docker-- **Use cases:** Real app testing, OS-level tasks--```python-from cua_bench.computers.base import get_session--SessionClass = get_session("native")-session = SessionClass(env=env)-```-------## Data Types--### Snapshot--```python-@dataclass-class WindowSnapshot:- window_type: Literal["webview", "process", "desktop"]- pid: Optional[str] = None- url: Optional[str] = None- html: Optional[str] = None- title: str = ""- x: int = 0- y: int = 0- width: int = 0- height: int = 0- active: bool = False- minimized: bool = False--@dataclass-class Snapshot:- windows: List[WindowSnapshot]-```--The `get_snapshot()` method returns a `Snapshot` object containing information about all open windows, their positions, sizes, and states.diff --git a/docs/content/docs/cuabot/guide/getting-started/installation.mdx b/docs/content/docs/cuabot/guide/getting-started/installation.mdxdeleted file mode 100644index 4fcddf5f54..0000000000--- a/docs/content/docs/cuabot/guide/getting-started/installation.mdx+++ /dev/null@@ -1,71 +0,0 @@-----title: Installation-description: Install Cua-Bot and its dependencies------import { Tab, Tabs } from 'fumadocs-ui/components/tabs';-import { Callout } from 'fumadocs-ui/components/callout';--<Tabs items={['macOS', 'Linux', 'Windows']}>- <Tab value="macOS">-- **Requirements:**- - [Node.js](https://nodejs.org/) v18+- - [Docker Desktop](https://www.docker.com/products/docker-desktop/)- - [Xpra client](https://github.com/Xpra-org/xpra/wiki/Download#-macos)-- **Install:**-- ```bash- npm install -g cuabot- ```-- <Callout type="warn">- After installing Xpra, you may need to remove quarantine:- ```bash- sudo xattr -rd com.apple.quarantine /Applications/Xpra.app- ```- </Callout>-- </Tab>- <Tab value="Linux">-- **Requirements:**- - [Node.js](https://nodejs.org/) v18+- - [Docker Engine](https://docs.docker.com/engine/install/)- - Xpra-- **Install:**-- ```bash- # Install Xpra- sudo apt install xpra-- # Install Cua-Bot- npm install -g cuabot- ```-- </Tab>- <Tab value="Windows">-- **Requirements:**- - [Node.js](https://nodejs.org/) v18+- - [Docker Desktop](https://www.docker.com/products/docker-desktop/)- - [Xpra client](https://github.com/Xpra-org/xpra/wiki/Download#-windows)-- **Install:**-- ```bash- npm install -g cuabot- ```-- </Tab>-</Tabs>--Then run:--```bash-cuabot-```--This runs the onboarding wizard which checks for dependencies and guides you through setup.diff --git a/docs/content/docs/cuabot/guide/getting-started/introduction.mdx b/docs/content/docs/cuabot/guide/getting-started/introduction.mdxdeleted file mode 100644index e41b0630b4..0000000000--- a/docs/content/docs/cuabot/guide/getting-started/introduction.mdx+++ /dev/null@@ -1,140 +0,0 @@-----title: Introduction-description: Multi-user computing with AI------<div className="not-prose flex justify-center my-8">- <img- src="/docs/img/cuabot-screenshot.png"- alt="cuabot screenshot"- style={{ maxHeight: '32em' }}- className="rounded-xl"- />-</div>--**Multi-user computing with AI**--`cuabot` gives any coding agent a seamless sandbox for computer-use. Windows stream to your desktop with clipboard sharing and audio, isolated from your system.--## Quick Start--```bash-# Start onboarding-npx cuabot--# Run your agent in the sandbox-cuabot claude-cuabot vibe-cuabot codex-```--Requirements:--- [Node.js](https://nodejs.org/) v18+-- [Docker Desktop](https://www.docker.com/products/docker-desktop/)-- [Xpra client](https://github.com/Xpra-org/xpra/wiki/Download)--## Usage--```bash-cuabot # Run default agent (or setup if not configured)-cuabot <agent> # Run a specific agent-cuabot <command> # Run any command in the sandbox-```--### Agents--Run any agent CLI in the sandboxed container:--```bash-cuabot claude # Claude Code (Anthropic)-cuabot gemini # Gemini CLI (Google)-cuabot codex # Codex CLI (OpenAI)-cuabot aider # Aider-cuabot openclaw # OpenClaw-cuabot vibe # Vibe (Mistral)-```--Agents are lazily installed on first use.--### Sandboxed Apps--Safely run scripts or GUI apps in the sandbox:--```bash-cuabot chromium # Open chromium window-cuabot xterm # Open terminal window-cuabot bash # Interactive bash-```--Terminal sessions and GUI applications are seamlessly streamed to your terminal or as a native desktop window.--### Named Sessions--Run multiple isolated sandboxes simultaneously:--```bash-cuabot -n work claude # Run Claude in a session named "work"-cuabot -n personal claude # Run Claude in a separate session named "personal"-cuabot --name dev bash # Interactive bash in session "dev"-```--Each named session gets its own container, port, and window border color. Session files are stored as `server.<name>.pid` and `server.<name>.port`.--### Commands--```bash-# Options (can be combined with any command)--n, --name <name> # Use a named session--# Control cuabotd-cuabot --serve [port] # Start cuabotd (auto-finds port)-cuabot --stop # Stop cuabotd-cuabot --status # Server status--# Control sandbox-cuabot --screenshot [path] # Take screenshot-cuabot --bash <command> # Execute bash command-cuabot --click <x> <y> [button] # Click at coordinates-cuabot --doubleclick <x> <y> # Double-click-cuabot --move <x> <y> # Move mouse-cuabot --mousedown <x> <y> # Press mouse button-cuabot --mouseup <x> <y> # Release mouse button-cuabot --drag <x1> <y1> <x2> <y2> # Drag-cuabot --scroll <x> <y> <dx> <dy> # Scroll-cuabot --type <text> # Type text-cuabot --key <key> # Press key-cuabot --keydown <key> # Key down-cuabot --keyup <key> # Key up--cuabot --help # Show help-```--## How It Works--Cua-Bot runs a Docker container with Xpra, streaming individual application windows to your desktop. Your chosen agent runs inside the container with:--- Full Ubuntu 22.04 environment-- Pre-installed: Node.js, Python, browsers, dev tools-- Computer-use MCP server for screenshot/click/type actions-- Isolated from your host system--## Configuration--Config files are stored in `~/.cuabot/`:--- `settings.json` - Default agent and preferences-- `server.pid` / `server.<name>.pid` - Server process ID-- `server.port` / `server.<name>.port` - Server port number-- `user/` - Persistent config for agent CLIs--## Development--```bash-pnpm dev # Run with file watching-pnpm build # Build TypeScript-```--## License--MITdiff --git a/docs/content/docs/cuabot/guide/getting-started/meta.json b/docs/content/docs/cuabot/guide/getting-started/meta.jsondeleted file mode 100644index 4e1aaf13ac..0000000000--- a/docs/content/docs/cuabot/guide/getting-started/meta.json+++ /dev/null@@ -1,7 +0,0 @@-{- "title": "Getting Started",- "description": "Get started with Cua-Bot",- "icon": "Rocket",- "defaultOpen": true,- "pages": ["introduction", "installation"]-}diff --git a/docs/content/docs/cuabot/guide/meta.json b/docs/content/docs/cuabot/guide/meta.jsondeleted file mode 100644index 5f712fc7b9..0000000000--- a/docs/content/docs/cuabot/guide/meta.json+++ /dev/null@@ -1,6 +0,0 @@-{- "title": "Guide",- "description": "Learn how to use Cua-Bot",- "icon": "Book",- "pages": ["getting-started"]-}diff --git a/docs/content/docs/cuabot/meta.json b/docs/content/docs/cuabot/meta.jsondeleted file mode 100644index 2e99986ace..0000000000--- a/docs/content/docs/cuabot/meta.json+++ /dev/null@@ -1,5 +0,0 @@-{- "title": "Cua-Bot",- "description": "Co-op computer-use for any agent",- "pages": ["guide", "reference"]-}diff --git a/docs/content/docs/cuabot/reference/changelog.mdx b/docs/content/docs/cuabot/reference/changelog.mdxdeleted file mode 100644index d386784e11..0000000000--- a/docs/content/docs/cuabot/reference/changelog.mdx+++ /dev/null@@ -1,132 +0,0 @@-----title: Changelog-description: Release history for Cua-Bot------{/*-AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY-Generated by: npx tsx scripts/docs-generators/generate-changelog.ts-Last updated: 2026-02-09-*/}--# Cua-Bot Changelog--All notable changes to the Cua-Bot are documented here.--## 1.0.x--### v1.0.13 (2026-02-05)--- Bump cuabot to v1.0.13 by @github-actions[bot]--Documentation--- [Getting Started](/cuabot/guide/getting-started/introduction)-- [Installation Guide](/cuabot/guide/getting-started/installation)--### v1.0.12 (2026-02-05)--- Bump cuabot to v1.0.12 by @github-actions[bot]-- fix cursor blocking input, remove start command override for cuabot ([#1020](https://github.com/trycua/cua/pull/1020)) by @ddupont808--Documentation--- [Getting Started](/cuabot/guide/getting-started/introduction)-- [Installation Guide](/cuabot/guide/getting-started/installation)--### v1.0.11 (2026-02-04)--- Bump cuabot to v1.0.11 by @github-actions[bot]--Documentation--- [Getting Started](/cuabot/guide/getting-started/introduction)-- [Installation Guide](/cuabot/guide/getting-started/installation)--### v1.0.10 (2026-02-04)--- Bump cuabot to v1.0.10 by @github-actions[bot]--Documentation--- [Getting Started](/cuabot/guide/getting-started/introduction)-- [Installation Guide](/cuabot/guide/getting-started/installation)--### v1.0.9 (2026-02-04)--- Bump cuabot to v1.0.9 by @github-actions[bot]-- Fix onboarding step missing on Windows over npx/pnpx--Documentation--- [Getting Started](/cuabot/guide/getting-started/introduction)-- [Installation Guide](/cuabot/guide/getting-started/installation)--### v1.0.8 (2026-02-04)--- Bump cuabot to v1.0.8 by @github-actions[bot]-- Add onboarding debug info and fix Xpra window detection on Windows ([#1014](https://github.com/trycua/cua/pull/1014)) by @ddupont808--Documentation--- [Getting Started](/cuabot/guide/getting-started/introduction)-- [Installation Guide](/cuabot/guide/getting-started/installation)--### v1.0.7 (2026-02-04)--- Bump cuabot to v1.0.7 by @github-actions[bot]--Documentation--- [Getting Started](/cuabot/guide/getting-started/introduction)-- [Installation Guide](/cuabot/guide/getting-started/installation)--### v1.0.6 (2026-02-04)--- Bump cuabot to v1.0.6 by @github-actions[bot]-- Bugfixes for Windows--Documentation--- [Getting Started](/cuabot/guide/getting-started/introduction)-- [Installation Guide](/cuabot/guide/getting-started/installation)--### v1.0.5 (2026-02-04)--- Bump cuabot to v1.0.5 by @github-actions[bot]-- [Cuabot] add y to npx ([#1009](https://github.com/trycua/cua/pull/1009)) by @ddupont808-- chore: bump cuabot container to 1.0.4 by @github-actions[bot]--Documentation--- [Getting Started](/cuabot/guide/getting-started/introduction)-- [Installation Guide](/cuabot/guide/getting-started/installation)--### v1.0.4 (2026-02-04)--- Initial release or no path-specific changes found--Documentation--- [Getting Started](/cuabot/guide/getting-started/introduction)-- [Installation Guide](/cuabot/guide/getting-started/installation)--### v1.0.3 (2026-02-04)--- Bump cuabot to v1.0.3 by @github-actions[bot]-- fix cuabot publish 3 ([#1003](https://github.com/trycua/cua/pull/1003)) by @ddupont808-- fix cuabot publish 2 ([#1002](https://github.com/trycua/cua/pull/1002)) by @ddupont808--Documentation--- [Getting Started](/cuabot/guide/getting-started/introduction)-- [Installation Guide](/cuabot/guide/getting-started/installation)--### v1.0.2 (2026-02-04)--- fix cuabot publish ([#1001](https://github.com/trycua/cua/pull/1001)) by @ddupont808-- Bump cuabot to v1.0.2 by @github-actions[bot]--Documentation--See [Cua-Bot documentation](/cuabot/guide/getting-started/introduction)diff --git a/docs/content/docs/cuabot/reference/index.mdx b/docs/content/docs/cuabot/reference/index.mdxdeleted file mode 100644index abe9868fd9..0000000000--- a/docs/content/docs/cuabot/reference/index.mdx+++ /dev/null@@ -1,295 +0,0 @@-----title: API Reference-description: TypeScript API reference for the Cua-Bot sandboxed agent framework------{/*- AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY- Generated by: npx tsx scripts/docs-generators/typescript-sdk.ts- Source: libs/cuabot/src- Version: 1.0.14-*/}--import { Callout } from 'fumadocs-ui/components/callout';-import { VersionHeader } from '@/components/version-selector';--<VersionHeader- versions={[{"version":"1.0","href":"/cuabot/reference","isCurrent":true}]}- currentVersion="1.0"- fullVersion="1.0.14"- packageName="cuabot"- installCommand="npm install -g cuabot"-/>-------## client--CuaBot Server Client-Connects to the CuaBot server via HTTP--### CuaBotClient--#### Constructor--```typescript-new CuaBotClient(port: number)-```--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `port` | `number` | |--#### Methods--##### CuaBotClient.for--```typescript-for(): Promise<-```--**Returns:** `Promise<`--##### CuaBotClient.request--```typescript-request(): Promise<number | null>-```--**Returns:** `Promise<number | null>`--##### CuaBotClient.status--```typescript-status(): Promise<-```--**Returns:** `Promise<`--##### CuaBotClient.request--```typescript-request(): Promise<string>-```--**Returns:** `Promise<string>`--##### CuaBotClient.request--```typescript-request(): Promise<void>-```--##### CuaBotClient.request--```typescript-request(): Promise<void>-```--##### CuaBotClient.request--```typescript-request(): Promise<void>-```--##### CuaBotClient.request--```typescript-request(): Promise<void>-```--##### CuaBotClient.request--```typescript-request(): Promise<void>-```--##### CuaBotClient.request--```typescript-request(): Promise<void>-```--##### CuaBotClient.request--```typescript-request(): Promise<void>-```--##### CuaBotClient.request--```typescript-request(): Promise<void>-```--##### CuaBotClient.request--```typescript-request(): Promise<void>-```--##### CuaBotClient.request--```typescript-request(): Promise<void>-```--##### CuaBotClient.request--```typescript-request(): Promise<void>-```--### setSessionName--```typescript-function setSessionName(name: string | null): void-```--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `name` | `string | null` | |--### getSessionName--```typescript-function getSessionName(): string | null-```--**Returns:** `string | null`--### isServerRunning--```typescript-async function isServerRunning(): Promise<-```--**Returns:** `Promise<`--### ensureServerRunning--```typescript-async function ensureServerRunning(): Promise<number>-```--**Returns:** `Promise<number>`-------## settings--CuaBot Settings Management--### Settings--```typescript-interface Settings {- defaultAgent?: string;- telemetryEnabled?: boolean;- aliasIgnored?: boolean;-}-```--| Property | Type | Description |-|----------|------|-------------|-| `defaultAgent` | `string` | *(optional)* |-| `telemetryEnabled` | `boolean` | *(optional)* |-| `aliasIgnored` | `boolean` | *(optional)* |--### AGENTS--```typescript-const AGENTS: const-```--### loadSettings--```typescript-function loadSettings(): Settings-```--**Returns:** `Settings`--### saveSettings--```typescript-function saveSettings(settings: Settings): void-```--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `settings` | `Settings` | |--### getDefaultAgent--```typescript-function getDefaultAgent(): string | undefined-```--**Returns:** `string | undefined`--### setDefaultAgent--```typescript-function setDefaultAgent(agent: string): void-```--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `agent` | `string` | |--### getTelemetryEnabled--```typescript-function getTelemetryEnabled(): boolean-```--**Returns:** `boolean`--### isTelemetryConfigured--```typescript-function isTelemetryConfigured(): boolean-```--**Returns:** `boolean`--### setTelemetryEnabled--```typescript-function setTelemetryEnabled(enabled: boolean): void-```--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `enabled` | `boolean` | |--### getAliasIgnored--```typescript-function getAliasIgnored(): boolean-```--**Returns:** `boolean`--### setAliasIgnored--```typescript-function setAliasIgnored(ignored: boolean): void-```--**Parameters:**--| Name | Type | Description |-|------|------|-------------|-| `ignored` | `boolean` | |diff --git a/docs/content/docs/cuabot/reference/meta.json b/docs/content/docs/cuabot/reference/meta.jsondeleted file mode 100644index 6680696f7e..0000000000--- a/docs/content/docs/cuabot/reference/meta.json+++ /dev/null@@ -1,6 +0,0 @@-{- "title": "Reference",- "description": "API and CLI reference for Cua-Bot",- "icon": "FileText",- "pages": ["changelog"]-}diff --git a/docs/content/docs/explanation/architecture.mdx b/docs/content/docs/explanation/architecture.mdxnew file mode 100644index 0000000000..a07fd10dce--- /dev/null+++ b/docs/content/docs/explanation/architecture.mdx@@ -0,0 +1,36 @@+---+title: Architecture+description: "How Cua Driver and Cua Sandbox give an external agent a computer-use interface: one on a real machine, one in a disposable VM."+---++Cua has two separate ways to expose a computer to an external agent. In both cases, the agent sends computer-use operations and receives observations such as screenshots, window lists, accessibility trees, and command results. The operating-system details stay behind a driver or server. The agent, model, or orchestration code remains outside that boundary and decides what to do next.++## Two paths to a computer++The real-machine path uses Cua Driver. The computer is an existing macOS, Windows, or Linux machine. Cua Driver runs on that machine and exposes OS-level operations through MCP over stdio and through a CLI. This path is for operating real native applications in the user's environment.++The disposable-computer path uses Cua Sandbox. The computer is a cloud or local VM or container created for the task. Cua Sandbox starts and controls that environment, then talks to a `computer-server` running inside it over HTTP and WebSocket. This path is for isolated workspaces that can be created, used, and discarded.++These paths are not two backends behind one shared SDK. Cua Driver and Cua Sandbox are separate products. They both provide a computer-use surface, but they use different transports, runtimes, and deployment targets.++## Cua Driver: the real machine++Cua Driver is a single Rust binary that operates a real machine. It exposes operations for window discovery, screenshots, accessibility-tree walking, input dispatch, and background operation that does not take over the user's active workspace. Tools can call it through MCP over stdio, or users can call it directly through the CLI.++Under the hood, Cua Driver uses the native platform APIs for each operating system. On macOS it uses AX plus CoreGraphics. On Windows it uses UIA. On Linux it uses AT-SPI. Those APIs provide the bridge from a model-agnostic computer-use request to concrete OS behavior, such as finding windows, reading accessibility state, capturing pixels, and sending input.++Because it runs on the real machine, Cua Driver inherits that machine's installed applications, user session, permissions, displays, and files. That is the main reason to use it. The tradeoff is that the computer is not disposable, so the caller is responsible for choosing operations that are appropriate for the user's active environment.++## Cua Sandbox: a disposable computer++Cua Sandbox creates a disposable cloud or local computer. The environment can be an isolated VM or container, depending on the backend. The Python Sandbox SDK creates and controls that sandbox, then communicates with the `computer-server` running inside it over REST and WebSocket APIs.++The `computer-server` is the component inside the sandbox that exposes the computer-use surface for that disposable machine. It receives operations over HTTP and WebSocket, performs them inside the VM or container, and returns observations. The Python sandbox stack talks to this server only over HTTP and WebSocket. It does not invoke Cua Driver and it does not use MCP.++Cua Sandbox can target several backends, including Lume and Lumier for macOS VMs, cloud environments, Windows Sandbox, QEMU, Hyper-V, and Docker. The backend determines where the disposable computer runs and what isolation boundary it uses, but the external shape remains the same: create a sandbox, connect to its `computer-server`, operate the computer, then tear it down when the task is finished.++## The model boundary++Cua Driver and `computer-server` are model-agnostic. They receive operations and return observations. They do not call language models, plan tasks, decide intent, or choose the next action. Cua does not ship a model.++That boundary keeps OS interaction separate from reasoning. The user brings an agent or model, and that system decides what operation to send next based on the observations it receives. Cua provides the computer-use interface on either a real machine through Cua Driver, or a disposable machine through Cua Sandbox and `computer-server`.diff --git a/docs/content/docs/explanation/demonstrations-skills-and-trajectories.mdx b/docs/content/docs/explanation/demonstrations-skills-and-trajectories.mdxnew file mode 100644index 0000000000..cc414888a5--- /dev/null+++ b/docs/content/docs/explanation/demonstrations-skills-and-trajectories.mdx@@ -0,0 +1,58 @@+---+title: 'Demonstrations, skills, and trajectories'+description: 'Cua has three different recording systems that get conflated. This page explains what each one records, where it lives, and when to use it.'+---++Several cua features use the words "record" and "trajectory", but they do not record the same thing. One system records a human demonstration and turns it into a reusable skill. Another records the agent's action-tool calls at the driver boundary. A third records the agent loop itself, including model outputs and computer observations.++The **actor being recorded** is the central distinction. Demonstration skills capture a human and produce a **generalizing document**. Cua Driver recordings and agent trajectories capture the agent, but at different layers: Cua Driver records the tools sent to the desktop, while `TrajectorySaverCallback` records the agent run around those tools.++## Demonstration Skills++Demonstration skills are exposed through the `cua skills record`, `list`, `read`, `replay`, `delete`, and `clean` CLI commands. The implementation lives in `cua-cli`.++This system records a human demonstrating a task by hand over VNC. During recording, the human uses the remote desktop normally. Afterward, cua extracts frames at each input event and uses a vision-language model to caption the step. Each caption describes the observation, intent, action, and expected result for that moment in the demonstration.++The output is a reusable skill under `~/.cua/skills/<name>/`. The main artifact is `SKILL.md`, which contains frontmatter with the skill name and description, a `Steps` section made from the captioned steps, and an `Agent Prompt`. The same skill directory also contains a `trajectory/` folder with the MP4 video, `events.json`, `trajectory.json`, and per-step screenshots.++Replay has two different meanings here. `cua skills replay` opens the MP4 so a person can watch the demonstration. It does not re-drive the original mouse and keyboard input. Actual reuse happens by reading `SKILL.md`, prefixing it as context to a `ComputerAgent` run, and then giving the agent a new task with new inputs. The agent follows the demonstrated pattern and adapts the parameters. This is the cua analog of OpenAI Codex record-and-replay: show a workflow once, get a reusable skill, not a verbatim playback script.++Use demonstration skills when the valuable thing is human know-how. They fit multi-step GUI workflows that should generalize across similar tasks, especially when the important knowledge is the order of operations, the visual cues, and the decision points rather than exact screen coordinates.++See [Record a demonstration](/how-to-guides/skills/record-a-demonstration) for the how-to guide.++## Cua Driver Trajectory Recording++Cua Driver trajectory recording is exposed through the `cua-driver recording start`, `stop`, `status`, and `render` subcommands, and through the equivalent MCP tools `start_recording`, `stop_recording`, `get_recording_state`, and `replay_trajectory`. It requires a running Cua Driver daemon. Recording state is per-process and kept in memory, so restarting the daemon resets it.++This system records the agent's action-tool calls. While recording is enabled, every action tool call writes a numbered `turn-NNNNN` folder under the selected `output_dir`. Recording fires for every non-read-only action tool, such as `click`, `right_click`, `scroll`, `type_text`, `press_key`, `hotkey`, and `set_value`. Read-only tools such as `get_window_state` and `list_windows` are not recorded.++Each turn folder contains `action.json`, which stores the tool name, full arguments, result, process id, click point, and timestamp. It also contains `screenshot.png`, a post-action capture of the target window, and `app_state.json`, a post-action AX or UIA tree snapshot that is omitted on Linux. Click-family actions also write `click.png`, a screenshot with a red dot at the click point. Video capture is off by default. Pass `record_video: true` to also capture the main display as `recording.mp4`, written as H.264 at 30 fps.++Replay is literal at the driver boundary. `replay_trajectory` walks the turn folders in order and re-invokes each recorded tool with its recorded arguments. The main controls are `delay_ms`, which spaces out actions, and `stop_on_error`, which decides whether replay stops on the first failed action.++The important caveat is **element addressing**. `element_index` values do not survive across sessions because indices are assigned per `get_window_state` snapshot and are keyed on `pid` and `window_id`. Element-indexed actions therefore do not resolve reliably on replay. Pixel clicks and keyboard tools replay cleanly. For that reason, Cua Driver recording is best treated as a regression and evidence artifact rather than a guaranteed durable automation script.++Use Cua Driver recording when the relevant question is exactly what the agent sent to the desktop and what the desktop looked like afterward. It is useful for demos, comparing a future run against a saved run across builds, and collecting tool-level training data.++See the Cua Driver [trajectory recording CLI reference](/reference/cua-driver/cli-reference#trajectory-recording) and [recording MCP tools](/reference/cua-driver/mcp-tools#start_recording) for the command and tool surfaces.++## Agent Trajectories++Agent trajectories are produced by `TrajectorySaverCallback` in the `cua-agent` package. The callback is attached to a `ComputerAgent`, so recording is wired into the agent run itself rather than managed by a separate daemon. The CLI surface for saved sessions is `cua trajectory ls`, `view`, and `clean`, implemented across `cua-agent` and `cua-cli`.++This system records the agent loop's `(state, action, next_state)` data as the run proceeds. The saved files include messages, model outputs, computer calls, call outputs, and screenshots. It captures the reasoning and observation context around the desktop operations, not only the driver calls themselves.++Inspection is session-oriented. `cua trajectory ls` lists saved sessions. `cua trajectory view` zips a session and opens it in the cua.ai trajectory viewer in the browser. `cua trajectory clean` removes old sessions.++Use agent trajectories when the relevant question is *why* an agent behaved as it did. They are the right artifact for debugging model decisions, reviewing the chain of observations and actions in a run, and collecting training or evaluation data from real agent sessions.++## Which One Do I Want++The systems differ by the actor recorded and the layer where the recording happens. Demonstration skills record a human and produce `SKILL.md`, which is meant to generalize. Cua Driver recordings and agent trajectories record the agent. Cua Driver records at the tool-call boundary for replay and regression. Agent trajectories record at the agent-loop boundary for debugging and data analysis.++| System | Records | Artifact | Replay means | Use for |+| --- | --- | --- | --- | --- |+| Demonstration skills | Human demo over VNC | `~/.cua/skills/<name>/SKILL.md` plus `trajectory/` assets | Watch the MP4 with `cua skills replay`, or reuse by giving `SKILL.md` to a `ComputerAgent` as context | Teaching a reusable GUI workflow from one human demonstration |+| Cua Driver recording | Agent action-tool calls | `turn-NNNNN` folders under `output_dir`, plus `recording.mp4` by default | Re-invoke recorded driver tools in order with `replay_trajectory` | Demos, regression diffs, and tool-level data collection |+| Agent trajectories | Agent loop state, actions, next states, messages, model outputs, computer calls, outputs, and screenshots | Saved trajectory session files | Inspect sessions with `cua trajectory view` in the trajectory viewer | Debugging agent behavior and collecting training or eval data |diff --git a/docs/content/docs/explanation/how-sandboxes-work.mdx b/docs/content/docs/explanation/how-sandboxes-work.mdxnew file mode 100644index 0000000000..c19caad936--- /dev/null+++ b/docs/content/docs/explanation/how-sandboxes-work.mdx@@ -0,0 +1,64 @@+---+title: "How sandboxes work"+description: "How a Cua Sandbox gives an agent one isolated computer it can both run code in and drive through the GUI."+---++A Cua Sandbox is a **full, isolated computer**, not a remote desktop session. In that one machine, an agent can run code through Python, shell commands, or a PTY, and it can also drive the graphical interface through screenshots, the accessibility tree, clicks, and typing.++Those are two complementary halves of the same computer. The code half and the GUI half share one filesystem, one set of processes, and one OS state. The value is that they live in the same place. An agent can click through an app and then run a Python function against the files or process state that app produced. It can also set up state in code and then automate the UI over that state.++## Sandboxes and real machines++Cua Driver operates a real machine you already have. It can observe and control that machine, so its actions happen in the same desktop environment you use.++A sandbox is different. It is a disposable machine created for a task. It starts from a known state, accumulates state only while it lives, and can be deleted without consequence. Actions inside a sandbox affect only that sandbox. They do not move the host mouse, change host files, or touch other sandboxes.++This isolation is what makes a sandbox useful for repeatable agent work. The agent gets a whole computer, but the effects are contained within that computer.++## The code half and the GUI half++The *code half* is how an agent runs programs inside the sandbox. It includes shell commands through `shell.run`, an interactive PTY or terminal through `computer.pty` and `cua do shell`, and sandboxed Python that runs a function inside the sandbox's own virtualenv through `venv_install`, `venv_exec`, and the `@sandboxed` decorator.++The *GUI half* is how an agent uses the sandbox like a desktop computer. It can observe the screen through screenshots and the accessibility tree, then act through clicks, typing, scrolling, keypresses, and other input events.++Because both halves are interfaces to the same machine, they can be mixed within one task. A shell command can create a file that is opened in the GUI. A GUI workflow can download data that is then inspected with Python. A PTY session can start a server, and the GUI half can open a browser against it. The important point is that all of these actions share the same filesystem and OS state.++## Containers and full VMs++A Linux container sandbox starts quickly because it shares the host kernel. It layers a Linux userspace, a desktop such as XFCE, and a remote display stack such as KasmWeb. This makes startup fast, but it is not identical to a physical Linux box. Kernel behavior, device access, and isolation come from the container host.++A full VM emulates hardware and boots its own kernel. macOS sandboxes use Apple Virtualization. Windows sandboxes use QEMU or Hyper-V. Android sandboxes use QEMU. Full VMs are slower to start than containers, but they provide higher OS fidelity because the guest OS owns its kernel and hardware model.++The trade-off is startup latency versus OS fidelity. Containers are suited to fast Linux environments. Full VMs are suited to work that depends on the behavior of a complete guest operating system.++## Images as starting-state contracts++An `Image` is the **immutable** description of the sandbox's starting environment. It is not the running sandbox. An `Image` defines the OS type, distro or OS version, packages, environment variables, copied files, and setup commands that should exist when a sandbox starts.++The image builder composes layers such as `apt_install`, `pip_install`, `run`, `copy`, and `env`. Those layers are applied once at launch to produce the sandbox's initial state. Installing another package later changes that sandbox, not the Image.++This separation makes environments reproducible. The `Image` describes what a fresh sandbox should look like. The sandbox is the live machine created from that description.++## Lifecycle patterns++Sandbox lifetime is separate from agent connection lifetime. An ephemeral sandbox exists for one block of work and is destroyed at the end. This fits CI jobs, tests, and one-shot tasks where the state has no value afterward.++A persistent or named sandbox is created once, identified by name, and can survive process exits. A later process can reconnect to the same sandbox and continue from the state it already has.++Connect mode attaches to an already-running sandbox. It does not create a sandbox and it does not delete one. Disconnecting drops the control connection. Deleting destroys the machine and its state. Those are different operations, and the distinction matters when the sandbox contains work that should survive the current process.++## Snapshots and forks++A *snapshot* captures the disk state of a running sandbox and returns a new `Image`. That `Image` can start any number of fresh sandboxes from the captured point.++Snapshots are useful when setup is expensive. Packages can be installed once, models can be downloaded once, and data can be prepared once. After the snapshot is created, new sandboxes can start from that prepared state instead of repeating the setup work.++Each *fork* starts from the same snapshot but gets its own writable disk. Changes in one fork do not affect the snapshot or sibling forks. On copy-on-write storage, forks can be near-instant because unchanged blocks are shared until a sandbox writes to them.++## Cloud and local execution++The same lifecycle concepts apply in local mode and cloud mode. The sandbox still starts from an `Image`, exposes a code half and a GUI half, can be ephemeral or persistent, and can be snapshotted or deleted.++Local mode runs on the developer's hardware. Linux containers use Docker Desktop, macOS VMs use Lume, and VM backends can use QEMU. Local execution depends on the machine's available CPU, memory, disk, and virtualization support. Local mode is selected with `local=True`.++Cloud mode runs managed VMs on Cua infrastructure with the same API surface. Provisioning, placement, and capacity are handled by the service. Cloud mode uses a Cua API key through `CUA_API_KEY`.diff --git a/docs/content/docs/explanation/index.mdx b/docs/content/docs/explanation/index.mdxnew file mode 100644index 0000000000..b581e32acc--- /dev/null+++ b/docs/content/docs/explanation/index.mdx@@ -0,0 +1,8 @@+---+title: "Explanation"+description: "Understand the design ideas behind Cua and how its main pieces fit together."+---++Explanation is for the parts of Cua that are easier to use once the model is clear. These pages are not recipes or API listings. They describe the **concepts, constraints, and tradeoffs** behind Cua's two main ways of giving an agent a computer: driving an existing machine with Cua Driver, or running an isolated cloud desktop with Cua Sandbox. Read this section when you want to understand why the system behaves the way it does, what guarantees it is trying to preserve, and where each component sits in the stack.++The section starts with [What is computer use?](/explanation/what-is-computer-use), which defines computer-use agents and where Cua fits. [The no foreground contract](/explanation/the-no-foreground-contract) explains the central invariant in Cua Driver: agent actions must not steal focus, move the real pointer, or interrupt the person using the machine. [Process model](/explanation/process-model) describes how Cua Driver runs as an MCP stdio server, daemon, and CLI. [How sandboxes work](/explanation/how-sandboxes-work) covers cloud desktop sandboxes, their lifecycle, and snapshots. [Architecture](/explanation/architecture) ties the pieces together: Cua Driver, Cua Sandbox, and the computer-server.diff --git a/docs/content/docs/explanation/meta.json b/docs/content/docs/explanation/meta.jsonnew file mode 100644index 0000000000..257002ad0d--- /dev/null+++ b/docs/content/docs/explanation/meta.json@@ -0,0 +1 @@+{ "title": "Explanation", "icon": "Lightbulb", "pages": ["index", "what-is-computer-use", "the-no-foreground-contract", "process-model", "how-sandboxes-work", "demonstrations-skills-and-trajectories", "architecture"] }diff --git a/docs/content/docs/explanation/process-model.mdx b/docs/content/docs/explanation/process-model.mdxnew file mode 100644index 0000000000..1a74caf4b5--- /dev/null+++ b/docs/content/docs/explanation/process-model.mdx@@ -0,0 +1,50 @@+---+title: "Process model"+description: "How Cua Driver maps one tool surface onto CLI, MCP, and daemon-backed processes."+---++Cua Driver has several process shapes, but they are adapters around one driver surface: list windows, read accessibility, capture the screen, click, type, record, configure, and report state. The process model places that surface in the operating-system context that can perform it.++Most users meet Cua Driver through an MCP stdio server spawned by an AI coding assistant such as Claude Code or Cursor. The same driver can also run as a long-lived daemon through `cua-driver serve`, or answer a one-shot CLI call. All three expose the same operations; the calling tool usually chooses the shape.++## Three invocation shapes++The MCP stdio shape is *client-owned*. The parent process starts `cua-driver`, keeps stdin and stdout connected, and sends MCP tool calls over that transport.++The daemon shape is *machine-owned*. A single `cua-driver serve` process listens on a local IPC endpoint, such as a Unix socket or named pipe, and keeps driver state in memory while it lives.++The one-shot CLI shape is *call-owned*. A command starts, performs one operation, prints a result, and exits.++These are invocation shapes, not separate implementations.++## Why a daemon proxy exists++The **daemon-proxy pattern** separates the process that speaks to the caller from the process that has the right desktop authority. The proxy handles the client protocol. The daemon performs the GUI work. That matters when the caller can start a shell process but cannot correctly operate the user's desktop.++On macOS, the root issue is **TCC**, Transparency Consent and Control. Accessibility and Screen Recording grants are attached to a specific app identity, represented by a bundle ID. Grants to `CuaDriver.app` do not automatically cover every subprocess named `cua-driver`.++If an IDE terminal starts `cua-driver` directly, macOS attributes that subprocess to the terminal app's bundle, not to `CuaDriver.app`. The binary is right, but the privacy identity is wrong.++The daemon path fixes the attribution. The daemon is launched through LaunchServices with `open -n -g -a CuaDriver`, so macOS treats it as part of `CuaDriver.app`. The MCP stdio process remains where the assistant spawned it, but becomes a thin proxy: it forwards tool calls to the daemon over a Unix socket and returns the daemon's responses. There are two processes, but one tool surface.++## Windows has the same shape for a different reason++On Windows, the daemon solves a session problem rather than a bundle-identity problem. When `cua-driver` is reached through SSH, the SSH-side process typically lands in Session 0, the non-interactive service session. Session 0 is not the logged-in user's GUI desktop, so it cannot see or operate those windows.++The daemon belongs in the interactive user session instead. It may be kept there by platform autostart machinery such as a Scheduled Task. An SSH-side client can then proxy requests to it. The shape is the same as on macOS: a caller-side process speaks the protocol, while a daemon-side process owns desktop access. The root cause is different.++## Session identity and shared state++A daemon drives one physical machine. Multiple MCP clients can connect at the same time, but they still share the same screen, keyboard, pointer, accessibility tree, and recording machinery. Session identity does not make concurrent control independent. Two agents clicking at once still contend for the same desktop.++Session identity solves a narrower state problem. When a proxy starts, it mints a session identity and stamps forwarded calls with it. The daemon uses that identity to scope mutable state to one client lifetime. Recording ownership, per-session config overrides, and the agent-cursor overlay are all keyed by session.++Cleanup follows the proxy connection, not a final message. The proxy keeps a long-lived control connection open to the daemon. When the proxy exits, even from an ungraceful kill, the kernel closes that connection. The daemon sees EOF and removes state owned by that session. This prevents stale recordings, cursors, and temporary config overrides.++## Lifetimes and memory++In-process MCP mode is simple: one process speaks MCP and performs the driver work directly. It lives for the stdio session. When the MCP client closes the transport, the process exits, and in-memory state exits with it. Element-index caches, active recording state, temporary config, and cursor state are process-local.++Daemon-proxy mode gives the driver a longer lifetime than the MCP client. The proxy may come and go as an assistant restarts its MCP transport, but the daemon can keep running. Element-index caches can remain warm, and daemon-held configuration can continue to apply after a tool client reconnects.++In-process mode is direct and short-lived. Daemon-proxy mode adds a local IPC hop, but it puts GUI work in the operating-system context that can perform it, and gives shared state a machine-level lifetime rather than a stdio subprocess lifetime.diff --git a/docs/content/docs/explanation/the-no-foreground-contract.mdx b/docs/content/docs/explanation/the-no-foreground-contract.mdxnew file mode 100644index 0000000000..e1db139a73--- /dev/null+++ b/docs/content/docs/explanation/the-no-foreground-contract.mdx@@ -0,0 +1,62 @@+---+title: "Background Computer-Use"+description: "How Cua Driver operates an app in the background without taking focus, moving the cursor, or raising windows."+---++# Background Computer-Use++Background computer-use means an agent operates an application without taking over the desktop. It does not change the user's frontmost app, move the real cursor, or raise the target window.++The **no-foreground contract** is the invariant and guarantee that makes background computer-use possible. Cua Driver addresses the target application through background-capable input, accessibility, and capture paths while preserving the developer's active session. An agent can operate one application while the developer keeps coding, reading logs, and using the same machine.++## Why It Matters++Most GUI automation assumes the automated app owns the desktop. It activates a window, moves the pointer, and repeats. That works for unattended jobs, but fails when **a human is using the same machine**.++If an agent raised the target window before every action, the developer's editor would lose focus every time the agent acted. If the agent moved the real pointer, the developer would lose their cursor position. If screenshots required the target window to be visible on the active desktop or Space, the workspace would jump to follow the target.++That limits the tool to fully automated, unattended scenarios: scheduled tasks, CI-style jobs, or disposable desktops. Preserving desktop state enables concurrent work instead. The agent operates an app in the background while the developer keeps working in the foreground.++## Platform Mechanisms++Each OS splits accessibility, input delivery, capture, and focus policy differently. Cua Driver upholds the contract with APIs that address a process, window, element, or message queue directly.++### macOS++On macOS, scoped CoreGraphics event routing lets synthetic events be posted to a target process. `CGEvent.postToPid` routes input directly to a process, bypassing the window server's focus check. The normal event path is focus-based.++The Accessibility API provides semantic action dispatch. `AXUIElementPerformAction` operates on an accessibility element, such as a button or menu item, regardless of whether the owning app is frontmost. When an app exposes useful structure, the driver can act on the element rather than replay motion.++ScreenCaptureKit can capture a specific window without requiring it to be visible, raised, or on the active Space. Cua Driver also uses SkyLight pid-routed mouse delivery for apps that respond better to routed pointer events than accessibility actions.++### Windows++Windows separates windows, handles, message queues, and foreground ownership. UI Automation can inspect and operate controls by window handle and automation element even when the target window is behind another window.++For input-like behavior, Cua Driver posts messages directly to a window's message queue with `PostMessage`, and uses synthetic pointer-device injection for cases UIPI blocks. That differs from global input injection, where the OS routes input to the focused window. Many standard Win32 controls can respond while the active window stays active.++Only a narrow set of operations still require foreground. Some DirectInput games and raw-input canvases poll device state or reject background messages. UAC dialogs live on a protected desktop where normal automation paths are restricted.++### Linux++Linux depends on the session stack. AT-SPI provides accessibility-tree access and action dispatch, giving the driver a semantic path for controls that expose accessibility metadata.++On X11, synthetic input and capture are window-addressable. `XSendEvent` can target a window ID without activating that window, and X11 window capture can read pixels from any mapped window. XWayland bridges help when an app still exposes an X11 surface.++**Wayland** is the notable exception. Its security model intentionally blocks one client from synthesizing arbitrary input into another client. That is the point of the design. Native Wayland-only apps remain a known gap for background input.++## Agent Cursor Overlay++The real pointer is part of the user's working state, so Cua Driver does not move it to show agent activity. Instead, the driver can render a *synthetic cursor overlay*: a visible marker for agent activity.++This is useful for supervision. The user can glance at the screen and see the agent clicking a button, selecting a field, or dragging inside a target window, while their own pointer stays where they left it. The overlay is an observation aid, not the input device.++## Honest Limits++The contract has exceptions, and they come from OS and application input models.++Canvas and game-style applications often poll raw input state rather than accepting normal window messages or accessibility actions. DirectInput games are a common Windows example. These apps may only respond when foregrounded because their event loop is built around active device state.++On macOS, SwiftUI windows that are off the active Space can lose their accessibility tree by OS design. The window may still exist, and pixels may still be capturable, but the semantic tree can disappear until the OS considers that UI present.++These cases are not bugs in Cua Driver. They are boundaries set by the host platform. The contract is the default wherever the OS and target application provide background-addressable input, accessibility, and capture paths.diff --git a/docs/content/docs/explanation/what-is-computer-use.mdx b/docs/content/docs/explanation/what-is-computer-use.mdxnew file mode 100644index 0000000000..9c347c750e--- /dev/null+++ b/docs/content/docs/explanation/what-is-computer-use.mdx@@ -0,0 +1,38 @@+---+title: "Computer-Use 2.0"+description: "Computer-Use 2.0 is how an agent operates a real computer through code, structured tool calls, and the graphical interface."+---++# Computer-Use 2.0++Computer-Use 2.0 is the way an AI agent operates a real computer by choosing among three action surfaces: writing and running code, calling tools and APIs, and driving the graphical interface a person would use. The older use of the term, where an agent looks at screenshots and clicks through a GUI, still matters as the UI surface inside a broader model of how agents get work done on computers.++## Three action surfaces++On the **coding surface**, the agent writes and runs code as the action itself. This works especially well when the task is text-native, when the same operation must run more than once, or when a small program can cover a larger body of files and state than a human would want to touch by hand.++On the **tool-use surface**, the agent makes structured calls to tools, functions, APIs, and MCP servers. The action has a typed shape, with named inputs and defined outputs, so the agent can ask an external system to perform a specific operation without inventing a script or trying to reach the same control through the screen.++On the **UI automation surface**, the agent clicks, types, scrolls, and presses keys on the real interface a person would use. This is the only surface that reaches native applications, web applications, enterprise and legacy software, and systems that have no useful API, because the GUI is often the only contract those systems expose to the user.++## Choosing a surface++A capable agent moves between these surfaces during a single task, choosing the one that fits the next piece of work. Code and structured tool calls are usually the right fit when the job is repeatable or text-heavy, especially when the system already exposes a clear interface. UI automation becomes the better choice when the work depends on visual state or when the application is unfamiliar; it is also the fallback when there is no useful API and the path exists only inside the interface.++The skill is the judgment behind that choice. An agent that edits a file with code, checks account state through an API, and then changes a setting in a desktop app is still doing one computer task; it is simply using different action surfaces as the task moves through different kinds of state.++## The UI automation loop++The observe, decide, act loop belongs specifically to the UI automation surface. Observation comes from a screenshot, an accessibility tree, or both, which gives the model either the pixels a person would see or the structured information exposed by the operating system. From there, grounding turns a target such as a button, field, menu item, or selected region into on-screen coordinates that input events can hit.++Planning carries the task across changing interface states. A click may open a dialog, a page may reflow after loading, or an application may show an error that changes the next useful action, so the model has to keep track of the goal while the computer responds. Frontier models such as Claude can handle understanding, grounding, and planning together in one call, while grounding-specialist models such as UI-TARS and Moondream can help when coordinate accuracy is the limiting factor.++The narrow origin of computer-use came in October 2024, when Anthropic introduced an agent that operated a GUI through screenshots and input events. Through 2025, coding agents were increasingly recognized as computer-use agents too, with CoAct-1 making the connection explicit, and the field began to converge on Computer-Use 2.0 as a wider concept. Francesco Bonacci traces that arc in [A Story of Computer-Use](https://github.com/trycua/cua/blob/main/blog/clawdbot-computer-use-history.md).++## Where Cua fits++You bring the agent, which already handles coding and tool-use on its own. Cua gives that agent the UI automation surface and a real computer to act in, so the same task can move from code to tools to the graphical interface when the application requires it.++**Cua Driver** drives the GUI of a real machine you already have, whether that machine runs macOS, Windows, or Linux. It is the right shape when the agent needs to work with local applications, signed-in accounts, existing files, or machine state that already lives on that computer.++**Cua Sandbox** is a fresh isolated cloud computer where the agent can run code and drive the GUI together. It is a full computer with all three action surfaces available inside the same environment rather than a remote desktop, which matters when the task needs both programmatic work and visible interaction without touching your local machine. The agent brings the model and its reasoning; Cua provides the computer where that reasoning can turn into action.diff --git a/docs/content/docs/how-to-guides/agent-context/connect-your-agent-to-cua-docs.mdx b/docs/content/docs/how-to-guides/agent-context/connect-your-agent-to-cua-docs.mdxnew file mode 100644index 0000000000..fd3a26861e--- /dev/null+++ b/docs/content/docs/how-to-guides/agent-context/connect-your-agent-to-cua-docs.mdx@@ -0,0 +1,124 @@+---+title: Connect your agent to Cua docs+description: Point your coding agent at the hosted Docs & Code MCP Server so it can search Cua's docs and source.+---++The Docs & Code MCP Server lets coding agents **ground answers and generated code** in Cua's real docs and source *instead of guessing*. Point your agent at the endpoint and it can query the actual APIs across released versions.++The Docs & Code MCP Server speaks streamable-HTTP at `https://vk-mcp.cua.ai/mcp`.++## Cursor++Open settings with `Shift+Command+J` on Mac, or `Shift+Ctrl+J` on Windows and Linux.++Find `MCP Tools` in the sidebar.++Click `Add Custom MCP` to open `mcp.json`.++Add this server entry:++```json+{+ "mcpServers": {+ "Cua Docs & Code": {+ "url": "https://vk-mcp.cua.ai/mcp"+ }+ }+}+```++## Claude Code++Add the Docs & Code MCP Server over the CLI:++```bash+claude mcp add --transport http cua-docs https://vk-mcp.cua.ai/mcp+```++Verify the connection:++```bash+claude mcp list+```++Expect a `Connected` line for `cua-docs`.++Inspect the server configuration when needed:++```bash+claude mcp get cua-docs+```++Remove the local server entry when needed:++```bash+claude mcp remove cua-docs -s local+```++## Windsurf++Open Windsurf Settings.++Go to `Cascade > Model Context Protocol`.++Click `Add Server > Add custom server`.++Select `Streamable HTTP` transport.++Enter this URL:++```text+https://vk-mcp.cua.ai/mcp+```++Edit the config file directly when needed. On macOS and Linux, use `~/.codeium/windsurf/mcp_config.json`. On Windows, use `%USERPROFILE%\.codeium\windsurf\mcp_config.json`.++## Cline (VS Code)++Open the Cline panel.++Click the menu in the top right.++Select `MCP Servers`.++Open the `Remote Servers` tab.++Enter `Cua Docs & Code` as the server name.++Enter this URL:++```text+https://vk-mcp.cua.ai/mcp+```++Click `Add Server`.++## GitHub Copilot++Enable MCP in VS Code. Open Settings, search for `MCP`, and enable `chat.mcp.enabled`.++Create `.vscode/mcp.json` in the project with this server entry:++```json+{+ "servers": {+ "Cua Docs & Code": {+ "url": "https://vk-mcp.cua.ai/mcp"+ }+ }+}+```++Open Copilot Chat.++Switch to Agent mode.++Use the Docs & Code MCP Server tools from the agent.++## Verify it works++Once connected, the agent exposes four read-only tools: `query_docs_db`, `query_docs_vectors`, `query_code_db`, and `query_code_vectors`.++Ask the agent something that forces a lookup, such as a Cua API signature in a specific release. Confirm that it calls one of the Docs & Code MCP Server tools before it answers.++See the [Docs & Code MCP reference](/reference/docs-code-mcp) for tool details.diff --git a/docs/content/docs/how-to-guides/agent-context/meta.json b/docs/content/docs/how-to-guides/agent-context/meta.jsonnew file mode 100644index 0000000000..fdde740b85--- /dev/null+++ b/docs/content/docs/how-to-guides/agent-context/meta.json@@ -0,0 +1 @@+{ "title": "Agent context", "pages": ["connect-your-agent-to-cua-docs"] }diff --git a/docs/content/docs/how-to-guides/driver/connect-your-agent.mdx b/docs/content/docs/how-to-guides/driver/connect-your-agent.mdxnew file mode 100644index 0000000000..e9bdb7006c--- /dev/null+++ b/docs/content/docs/how-to-guides/driver/connect-your-agent.mdx@@ -0,0 +1,235 @@+---+title: Connect your agent+description: Register Cua Driver as an MCP server with Claude Code, Codex, Cursor, Antigravity, OpenClaw, OpenCode, Hermes, and Pi.+---++import { Callout } from 'fumadocs-ui/components/callout';++`cua-driver mcp` runs as a stdio MCP server. Register it with an MCP-capable agent when you want that agent to **control the host desktop** through cua-driver.++<Callout type="warn">+ This verb wires the agent to Cua Driver, which is control of the HOST machine ONLY. It does NOT connect an agent to a cua sandbox. Sandbox access is a separate, SDK-first path: use the Computer SDK, or the separate cua-agent MCP server. Do not imply that mcp-config gives sandbox access.+</Callout>++<Callout type="warn">+ Grant the required permissions before you connect an agent. On macOS, grant Accessibility and Screen Recording. Verify them with `cua-driver call check_permissions`.+</Callout>++The helper command prints the exact registration snippet for each supported client:++```bash+cua-driver mcp-config --client <client>+```++<Callout type="info">+ The `claude` client can register a computer-use compatibility surface through `--claude-code-computer-use-compat`. Every client gets the same driver tools. There is no standalone screenshot tool, so window capture comes from `get_window_state` (set `capture_mode` to `vision` for a screenshot).+</Callout>++## Claude Code++Register the plain stdio server:++```bash+claude mcp add --transport stdio cua-driver -- cua-driver mcp+```++Verify:++```bash+claude mcp list+# cua-driver: cua-driver mcp (stdio) - connected+```++Claude Code also supports a computer-use compatibility mode. Generate the command:++```bash+cua-driver mcp-config --client claude+```++It emits:++```bash+claude mcp add-json cua-computer-use '{"command":"cua-driver","args":["mcp","--claude-code-computer-use-compat"]}'+```++This registers the server under the key `cua-computer-use` and passes `--claude-code-computer-use-compat`. The flag selects the Claude Code compat surface. The compat screenshot tool was removed in #1692, so the flag has no tool-surface effect today; the wiring stays in place for any future compat-gated tool. Capture the target window with `get_window_state` (set `capture_mode` to `vision` for a screenshot).++<Callout type="info">+ This is NOT Anthropic's native computer-use API. It is Cua Driver's MCP compatibility mode for Claude Code. Start with `launch_app` or `list_windows`, then capture the target window with `get_window_state`.+</Callout>++## Codex++Register the stdio server:++```bash+codex mcp add cua-driver -- cua-driver mcp+```++Verify:++```bash+codex mcp list+```++## Cursor++Generate the Cursor snippet:++```bash+cua-driver mcp-config --client cursor+```++Paste the JSON into `~/.cursor/mcp.json`, or `.cursor/mcp.json` for project scope:++```json+{+ "mcpServers": {+ "cua-driver": {+ "command": "cua-driver",+ "args": ["mcp"],+ "type": "stdio"+ }+ }+}+```++Verify: restart Cursor and confirm `cua-driver` appears in the MCP server list.++## Antigravity++Generate the Antigravity snippet:++```bash+cua-driver mcp-config --client antigravity+```++Antigravity CLI, the `agy` binary and successor to Gemini CLI, has no `agy mcp add` subcommand. Paste the printed JSON into `~/.gemini/config/mcp_config.json`, merging it under the top-level `mcpServers` object. On Windows, use `%USERPROFILE%\.gemini\config\mcp_config.json`. Restart `agy` after saving. The same file is shared with the Antigravity IDE. `--client gemini` is a legacy alias.++```json+{+ "mcpServers": {+ "cua-driver": {+ "command": "cua-driver",+ "args": ["mcp"]+ }+ }+}+```++Verify: restart `agy` or the Antigravity IDE and confirm `cua-driver` is listed as an MCP server.++## OpenClaw++Register the stdio server:++```bash+openclaw mcp set cua-driver '{"command":"cua-driver","args":["mcp"]}'+```++Verify: restart OpenClaw and confirm `cua-driver` is available in the MCP server list.++## OpenCode++Generate the OpenCode snippet:++```bash+cua-driver mcp-config --client opencode+```++Paste it under `mcp` in `opencode.json`, or in `~/.config/opencode/config.json` for global config:++```json+{+ "$schema": "https://opencode.ai/config.json",+ "mcp": {+ "cua-driver": {+ "type": "local",+ "command": ["cua-driver", "mcp"],+ "enabled": true+ }+ }+}+```++<Callout type="warn">+ Configure it as a real MCP server. If MCP is not configured, OpenCode runs Cua Driver as a shell subprocess, the screenshot image block is dropped, and the model gets only the AX tree with no visual context.+</Callout>++Verify: restart OpenCode and confirm `cua-driver` appears under MCP servers.++## Hermes++Generate the Hermes snippet:++```bash+cua-driver mcp-config --client hermes+```++Paste it under `mcp_servers` in `~/.hermes/config.yaml`:++```yaml+mcp_servers:+ cua-driver:+ command: "cua-driver"+ args: ["mcp"]+```++Reload MCP servers inside Hermes:++```text+/reload-mcp+```++Verify: confirm Hermes reports `cua-driver` after `/reload-mcp`.++## Pi++Pi, `badlogic/pi-mono`, does not support MCP natively. The author has stated MCP support will not be added for context-budget reasons, so this client is an honest caveat rather than an MCP registration path.++Run:++```bash+cua-driver mcp-config --client pi+```++The output tells you to use Cua Driver as a plain CLI from inside Pi instead:++```bash+cua-driver list_apps+cua-driver click '{"pid": 1234, "x": 100, "y": 200}'+cua-driver --help # full tool catalog+```++Each call is one-shot and returns JSON or text on stdout, which is the shape Pi is built around.++Verify: run `cua-driver --help` from the same Pi shell context.++## Any other client++For any client that accepts the standard `mcpServers` shape, print the generic config:++```bash+cua-driver mcp-config+```++Output:++```json+{+ "mcpServers": {+ "cua-driver": {+ "command": "cua-driver",+ "args": ["mcp"]+ }+ }+}+```++Verify: restart the client and confirm `cua-driver` appears in its MCP server list.++## Next steps++- [Install Cua Driver / grant permissions](/how-to-guides/driver/install): install Cua Driver and approve host permissions.+- [Keep Cua Driver running](/how-to-guides/driver/keep-running): configure autostart so the daemon comes back after reboots.+- [Update Cua Driver](/how-to-guides/driver/update): check for new releases and apply them.diff --git a/docs/content/docs/how-to-guides/driver/install.mdx b/docs/content/docs/how-to-guides/driver/install.mdxnew file mode 100644index 0000000000..2efcbcc17c--- /dev/null+++ b/docs/content/docs/how-to-guides/driver/install.mdx@@ -0,0 +1,114 @@+---+title: Install Cua Driver+description: Install Cua Driver on macOS, Windows, or Linux with a one-line script.+---++import { Tabs, Tab } from 'fumadocs-ui/components/tabs';+import { Callout } from 'fumadocs-ui/components/callout';++Cua Driver supports macOS, Windows, and Linux. Use the same **one-line installer** on each platform; the script selects the right install path for the host and *does not require administrator access*.++<Tabs items={['macOS', 'Windows', 'Linux']}>+ <Tab value="macOS">++**Requirements:** macOS 14 (Sonoma) or later on Apple Silicon or Intel.++```bash+/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.sh)"+```++The installer places `CuaDriver.app` in `/Applications` and creates the `~/.local/bin/cua-driver` symlink. The app bundle uses the `com.trycua.driver` signing identity, so macOS TCC permissions for Accessibility and Screen Recording remain attached across upgrades.++If `~/.local/bin` is missing from your PATH, the installer detects your shell (`zsh`, `bash`, or `fish`) and adds the matching `export PATH=…` line to your rc file. Reload the shell with `source ~/.zshrc` or open a new terminal window.++ </Tab>+ <Tab value="Windows">++**Requirements:** Windows 10/11 or Windows Server with an interactive desktop session and PowerShell.++```powershell+irm https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.ps1 | iex+```++The installer downloads the release under `%USERPROFILE%\.cua-driver\packages\releases\` and exposes `cua-driver.exe` from `%LOCALAPPDATA%\Programs\Cua\cua-driver\bin`. It runs without administrator privileges, detects whether the host is x64 or arm64, and appends the install directory to your User-scope `Path` so new PowerShell windows can resolve `cua-driver.exe`.++To skip the PATH change, pass the no-update flag:++```powershell+& ([scriptblock]::Create((irm https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.ps1))) -NoPathUpdate+```++ </Tab>+ <Tab value="Linux">++**Requirements:** an x86_64 Linux desktop session with X11 or XWayland, plus AT-SPI 2 for accessibility-tree tools.++```bash+/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.sh)"+```++The installer downloads the binary to `~/.cua-driver/packages/releases/`, points `~/.cua-driver/packages/current` at that release, and creates the `~/.local/bin/cua-driver` symlink. It does not use sudo.++X11 is fully supported. Native Wayland input and capture remain opt-in with `CUA_DRIVER_RS_ENABLE_WAYLAND=1` and are still maturing; Wayland sessions use XWayland by default. After installation, run `cua-driver doctor` to check distro-specific requirements such as the AT-SPI bus and ffmpeg.++ </Tab>+</Tabs>++## Verify the install++```bash+cua-driver --version+# cua-driver 0.5.x+```++For a full environment and install report:++```bash+cua-driver doctor+# [ok ] binary: cua-driver 0.5.x (aarch64-macos)+# [ok ] install dir: /Users/you/.local/bin/cua-driver+# [ok ] home dir: /Users/you/.cua-driver (3 release dirs cached)+# ...+```++`doctor` checks the version, install layout, and telemetry setup on every platform. It also runs platform probes for TCC on macOS, the interactive session on Windows, and AT-SPI plus the display server on Linux.++## Grant TCC permissions (macOS only)++**Start the daemon first** so macOS attributes the TCC request to `CuaDriver.app` instead of your terminal:++```bash+open -n -g -a CuaDriver --args serve+```++Then grant both permissions:++```bash+cua-driver permissions grant+```++That command launches CuaDriver through LaunchServices and waits while you approve Accessibility and Screen Recording in the macOS dialogs.++You can also trigger the prompts yourself:++```bash+cua-driver check_permissions+```++macOS opens the Accessibility and Screen Recording prompts. Grant both permissions, then run the check again:++```bash+cua-driver check_permissions+# ✅ Accessibility: granted.+# ✅ Screen Recording: granted.+```++<Callout type="info">+ `cua-driver permissions status` reads the driver's actual grant state through the daemon. When no daemon is running, it reports `❓ unknown` instead of reporting your terminal's grants, and it does not claim `granted` unless the driver has that permission.+</Callout>++## Next steps++- [Keep Cua Driver running](/how-to-guides/driver/keep-running): configure autostart so the daemon comes back after reboots.+- [Connect Cua Driver to an MCP client](/how-to-guides/driver/connect-your-agent): register it with Claude Code, Cursor, Codex, and other clients.+- [Update Cua Driver](/how-to-guides/driver/update): check for new releases and apply them.diff --git a/docs/content/docs/how-to-guides/driver/keep-running.mdx b/docs/content/docs/how-to-guides/driver/keep-running.mdxnew file mode 100644index 0000000000..acec889330--- /dev/null+++ b/docs/content/docs/how-to-guides/driver/keep-running.mdx@@ -0,0 +1,143 @@+---+title: Keep Cua Driver running+description: Register Cua Driver as a persistent daemon that starts automatically and survives reboots.+---++import { Tabs, Tab } from 'fumadocs-ui/components/tabs';+import { Callout } from 'fumadocs-ui/components/callout';++Use a **persistent daemon** for element-indexed workflows. The per-pid element cache lives inside the process, so one-shot CLI invocations *drop that cache between calls*. The daemon also gives macOS the right TCC attribution and gives Windows an interactive-session proxy.++<Tabs items={['macOS', 'Windows', 'Linux']}>+ <Tab value="macOS">++`cua-driver autostart` is not implemented for macOS yet. Write a LaunchAgent with `install-local.sh --autostart`:++```bash+bash libs/cua-driver/scripts/install-local.sh --autostart+```++You can also create the plist yourself. Save this file at `~/Library/LaunchAgents/com.trycua.cua-driver.plist`:++```xml+<?xml version="1.0" encoding="UTF-8"?>+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"+ "http://www.apple.com/DTDs/PropertyList-1.0.dtd">+<plist version="1.0">+<dict>+ <key>Label</key>+ <string>com.trycua.cua-driver</string>+ <key>ProgramArguments</key>+ <array>+ <string>/Applications/CuaDriver.app/Contents/MacOS/cua-driver</string>+ <string>serve</string>+ </array>+ <key>RunAtLoad</key>+ <true/>+ <key>KeepAlive</key>+ <true/>+</dict>+</plist>+```++Load the LaunchAgent:++```bash+launchctl load ~/Library/LaunchAgents/com.trycua.cua-driver.plist+```++<Callout type="info">+ A LaunchAgent daemon starts under `launchd` and is attributed to `com.trycua.driver`. Grant Accessibility and Screen Recording once, and those grants persist across reboots. If you start prompts from a terminal without a LaunchAgent, macOS attributes them to the terminal instead of the driver, so the grants do not apply to Cua Driver.+</Callout>++ </Tab>+ <Tab value="Windows">++Run these commands once from an interactive session, either RDP or the local console:++```powershell+cua-driver autostart enable+cua-driver autostart kick+```++`enable` registers a Scheduled Task named `cua-driver-serve` with `LogonType: Interactive`. That keeps the daemon in Session 1+ instead of Session 0. The command is idempotent, so running it again after an upgrade updates the binary path automatically.++`kick` starts the task immediately, so you do not need to wait for the next logon.++Check the task state when you need it:++```powershell+cua-driver autostart status+# registered (running)+```++Remove the autostart registration:++```powershell+cua-driver autostart disable+```++<Callout type="info">+ `LogonType: Interactive` is required. Alternatives such as `S4U` or `Password` put the daemon in Session 0, where GUI tools return empty arrays. See [Drive a Windows app over SSH](/how-to-guides/driver/windows-ssh) for the SSH proxy workflow that uses this setup.+</Callout>++The Scheduled Task keeps running after an RDP disconnect. A disconnected session remains in `Disc` state and is still a live interactive session. The daemon stops only on explicit logoff or reboot; the next interactive logon triggers the task again.++ </Tab>+ <Tab value="Linux">++`cua-driver autostart` is Windows-only today. Create a systemd user unit by hand.++Save this unit at `~/.config/systemd/user/cua-driver.service`:++```ini+[Unit]+Description=cua-driver background daemon+After=graphical-session.target+PartOf=graphical-session.target++[Service]+Type=simple+ExecStart=%h/.local/bin/cua-driver serve+Restart=on-failure+RestartSec=2++[Install]+WantedBy=graphical-session.target+```++Reload systemd, enable the unit, and start it:++```bash+systemctl --user daemon-reload+systemctl --user enable --now cua-driver.service+systemctl --user status cua-driver.service+```++On CI runners or headless machines where the daemon should survive user logout, enable linger:++```bash+loginctl enable-linger $USER+```++ </Tab>+</Tabs>++## Verify it's running++```bash+cua-driver status+# Cua Driver daemon is running+# socket: /Users/you/Library/Caches/cua-driver/cua-driver.sock+# pid: 12345+```++On Windows the socket path is `\\.\pipe\cua-driver` and `status` also reports the session number.++## Stop it cleanly++```bash+cua-driver stop+```++This sends a shutdown signal to the daemon process. The autostart entry, whether LaunchAgent, Scheduled Task, or systemd unit, remains registered and starts it again on the next logon or trigger.diff --git a/docs/content/docs/how-to-guides/driver/meta.json b/docs/content/docs/how-to-guides/driver/meta.jsonnew file mode 100644index 0000000000..8695def43c--- /dev/null+++ b/docs/content/docs/how-to-guides/driver/meta.json@@ -0,0 +1 @@+{ "title": "Driver", "pages": ["install", "connect-your-agent", "keep-running", "update", "windows-ssh"] }diff --git a/docs/content/docs/how-to-guides/driver/update.mdx b/docs/content/docs/how-to-guides/driver/update.mdxnew file mode 100644index 0000000000..9a6f9d4d07--- /dev/null+++ b/docs/content/docs/how-to-guides/driver/update.mdx@@ -0,0 +1,122 @@+---+title: Update Cua Driver+description: Check whether a newer Cua Driver release is available and install it.+---++import { Callout } from 'fumadocs-ui/components/callout';++Cua Driver gives you three update paths: a passive banner at launch time, a CLI command that only checks, and a separate command that installs the update. Keep the **check and apply steps separate** so you can decide when to change the installed binary.++## Check for updates++```bash+cua-driver check-update+```++When you're behind:++```+Current: 0.3.1+Latest: 0.3.2++Update available. Run `cua-driver update --apply` to install.+Release notes: https://github.com/trycua/cua/releases/tag/cua-driver-rs-v0.3.2+```++When you're current:++```+Current: 0.3.1+Latest: 0.3.1++You're on the latest release.+```++Flags:+- `--json`: return a machine-readable payload, shown below.+- `--no-cache`: skip the 20-hour on-disk cache and force a fresh GitHub check.++The command exits with `0` when the check succeeds. Read `update_available` in the JSON payload to learn whether a release is available. Non-zero exit codes mean the check failed, for example because the network was unavailable or parsing failed.++## Apply the update++```bash+cua-driver update --apply+```++Output:++```+Current version: 0.3.1+Checking for updates…+New version available: 0.3.2+Downloading and installing Cua Driver 0.3.2…+Installed Cua Driver 0.3.2.+```++`update --apply` calls the canonical installer script, the same one-line installer used for the first install.++If the daemon was running before the update, restart it so it uses the new binary:++```bash+cua-driver stop && cua-driver serve+```++On Windows with autostart enabled, the Scheduled Task uses the new binary on its next invocation. `autostart enable` is idempotent, and the task path resolves through the `current` junction.++## Scripted check (CI / agents)++```bash+cua-driver check-update --json+```++```json+{+ "current_version": "0.3.1",+ "latest_version": "0.3.2",+ "update_available": true,+ "source": "github_releases",+ "checked_at": "2026-05-27T14:30:00Z",+ "cache_hit": false,+ "install_command": "curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.sh | bash",+ "release_notes_url": "https://github.com/trycua/cua/releases/tag/cua-driver-rs-v0.3.2",+ "error": null+}+```++When the installed version is current, `install_command` and `release_notes_url` are `null`.++Use the JSON form from a bash script:++```bash+if cua-driver check-update --json | jq -e '.update_available' > /dev/null; then+ echo "Cua Driver is outdated — applying update"+ cua-driver update --apply+fi+```++The same payload is available over MCP through the `check_for_update` tool. MCP does not provide an `apply` variant because installing through MCP would replace the running server process.++## Passive banner++Every `cua-driver mcp`, `serve`, and `doctor` invocation starts a background version check. If it finds a newer release, Cua Driver prints a two-line banner to stderr:++```+✨ cua-driver v0.3.2 is available (you have v0.3.1).+ Update with: cua-driver update+ Release notes: https://github.com/trycua/cua/releases/tag/cua-driver-rs-v0.3.2+```++The check uses a 20-hour cache and never blocks startup. One-shot commands such as `--version`, `call`, and `list-tools` skip it so piped output remains clean.++Disable the check for one invocation:++```bash+CUA_DRIVER_RS_UPDATE_CHECK=false cua-driver serve+```++Disable it permanently:++```bash+cua-driver config set update_check_enabled false+```diff --git a/docs/content/docs/how-to-guides/driver/windows-ssh.mdx b/docs/content/docs/how-to-guides/driver/windows-ssh.mdxnew file mode 100644index 0000000000..5a311b5103--- /dev/null+++ b/docs/content/docs/how-to-guides/driver/windows-ssh.mdx@@ -0,0 +1,124 @@+---+title: Drive a Windows app over SSH+description: Use Cua Driver to drive GUI apps on a remote Windows machine from an SSH session.+---++import { Callout } from 'fumadocs-ui/components/callout';++## The problem++Windows OpenSSH server runs in **Session 0**, the non-interactive services session. Sessions 1, 2, ... are interactive logons, one for each console or RDP user. A process inherits its parent's session, so shells created by `sshd` also run in Session 0.++The Win32 APIs used by Cua Driver window tools, including `EnumWindows`, `GetForegroundWindow`, `PrintWindow`, UIA, and `BitBlt`, are scoped to the caller's WindowStation and Desktop. Session 0 has no attached interactive desktop, so the tools cannot see the user's windows:++```powershell+# Over SSH (Session 0), with no daemon helper:+cua-driver call list_windows+# [] ← empty, even though the user's RDP session has 12 windows open+```++<Callout type="info">+ Run `cua-driver doctor` to confirm this state. The Windows session probe reports it directly:++ ```+ [warn] interactive session: running in Session 0 (services); window-driving+ tools (list_windows, click, type_text, screenshot, get_window_state)+ will return empty results — these APIs need an attached interactive+ desktop.+ ```+</Callout>++## The solution++Run a `cua-driver serve` daemon in your **interactive session**, Session 1 or higher, through an autostart Scheduled Task. The CLI running over SSH detects that daemon and proxies tool calls through its named pipe. The SSH process only moves protocol messages; the daemon performs the actual GUI work from a session with a desktop attached.++```+┌───────────────────────────────────────────────────────────────┐+│ Session 1+ (RDP / console — has interactive desktop) │+│ │+│ cua-driver-serve (autostart Scheduled Task) │+│ ↑ │+│ │ named pipe: \\.\pipe\cua-driver │+│ │ │+└──────┼────────────────────────────────────────────────────────┘+ │+┌──────┼────────────────────────────────────────────────────────┐+│ Session 0 (services / SSH — no desktop) │+│ │ │+│ cua-driver mcp │+│ cua-driver call list_windows │+│ Claude Code → MCP stdio → cua-driver mcp │+└───────────────────────────────────────────────────────────────┘+```++## Set it up++**1. From an interactive session, either RDP or the local console, run:**++```powershell+cua-driver autostart enable+cua-driver autostart kick+```++`enable` registers a Scheduled Task with `LogonType: Interactive`. That setting is required because the alternatives would start the daemon in Session 0. `kick` starts the task immediately instead of waiting for the next logon.++<Callout type="warn">+ You need an active interactive session for `kick` to place the daemon in Session 1+. Confirm that state with `query session`; your row should show `Active` or `Disc`:++ ```powershell+ query session+ # SESSIONNAME USERNAME ID STATE TYPE+ # rdp-tcp#23 you 2 Active+ ```++ If you have never opened RDP on this machine, connect once with RDP. The autostart trigger fires automatically on the next logon.+</Callout>++**2. From your SSH session, verify that the daemon is reachable:**++```powershell+cua-driver status+# Cua Driver daemon is running+# socket: \\.\pipe\cua-driver+# pid: 12345+# session: 2 ← daemon is in your interactive session+```++**3. Call tools from SSH:**++```powershell+cua-driver call list_apps # sees real GUI apps+```++## Connect Claude Code over SSH++After the daemon is running in the interactive session, register Claude Code the same way you would locally:++```powershell+# From inside your SSH session:+claude mcp add --transport stdio cua-driver -- cua-driver.exe mcp+claude+```++Each MCP tool call starts `cua-driver mcp` on the SSH side. That process detects the daemon and proxies through it. Claude Code sees a normal stdio MCP server, while the daemon receives tool calls from the Session 1+ desktop context.++## Diagnose empty results++Check these items before opening an issue:++1. Confirm that `cua-driver --version` on the SSH side reports `0.2.7` or later. Earlier Windows builds do not proxy `mcp`. Upgrade with `irm https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.ps1 | iex`.+2. Run `cua-driver status` from SSH and confirm it reports a running daemon. If it does not, use `cua-driver autostart status` to see whether the Scheduled Task is registered.+3. Run `query session` and confirm your user has a row in `Active` or `Disc` state.+4. Run `cua-driver doctor` from RDP and confirm it reports `[ok] interactive session: session N has an attached interactive desktop`.+5. Confirm that you did not pass `--no-daemon-relaunch` and that `CUA_DRIVER_RS_MCP_NO_RELAUNCH` is unset.++## Opt out of proxying++To keep `cua-driver mcp` in the current process, such as in a CI runner that already owns an interactive session, use either opt-out:++```powershell+cua-driver mcp --no-daemon-relaunch # per-invocation flag+$env:CUA_DRIVER_RS_MCP_NO_RELAUNCH = "1" # per-shell env var+```++With either form, tool calls run directly against the current session.diff --git a/docs/content/docs/how-to-guides/index.mdx b/docs/content/docs/how-to-guides/index.mdxnew file mode 100644index 0000000000..5f3f4ccd40--- /dev/null+++ b/docs/content/docs/how-to-guides/index.mdx@@ -0,0 +1,8 @@+---+title: 'How-to guides'+description: 'Follow recipes for specific Cua goals.'+---++How-to guides are recipes for **a specific goal**. They assume you already know the basics, so do a tutorial first if not. Unlike tutorials, they are *not a lesson*; they get you to a result and stop.++They are grouped by surface: Cua Driver for operating apps on a real machine, sandbox for cloud desktops, plus recipes, skills, and agent context.diff --git a/docs/content/docs/how-to-guides/meta.json b/docs/content/docs/how-to-guides/meta.jsonnew file mode 100644index 0000000000..ec338b048b--- /dev/null+++ b/docs/content/docs/how-to-guides/meta.json@@ -0,0 +1 @@+{ "title": "How-to guides", "icon": "Wrench", "pages": ["index", "driver", "recipes", "sandbox", "skills", "agent-context"] }diff --git a/docs/content/docs/how-to-guides/recipes/automate-a-legacy-windows-app-behind-a-vpn.mdx b/docs/content/docs/how-to-guides/recipes/automate-a-legacy-windows-app-behind-a-vpn.mdxnew file mode 100644index 0000000000..ae21471569--- /dev/null+++ b/docs/content/docs/how-to-guides/recipes/automate-a-legacy-windows-app-behind-a-vpn.mdx@@ -0,0 +1,125 @@+---+title: Automate a legacy Windows app behind a VPN+description: Use Cua Driver on a VPN-connected Windows machine to drive a legacy desktop app that has no API.+---++import { Callout } from 'fumadocs-ui/components/callout';+import { Steps, Step } from 'fumadocs-ui/components/steps';++When a desktop app has **no API** and only runs **behind the corporate VPN**, drive it where it already runs, on the Windows box inside the network. **Cua Driver** runs on that machine and exposes the app through MCP stdio tools, so the desktop session, app traffic, and staged files stay inside the VPN boundary. **No data leaves the VPN boundary.**++<Steps>+ <Step>+ ### Start from the Windows machine inside the VPN++ Use a workstation, jump box, or RDP VM that is already connected to the VPN and can open the legacy app.++ Connect with RDP if needed:++ ```powershell+ mstsc /v:payroll-jumpbox.corp.example+ ```++ Confirm the app is installed or reachable from that desktop session before you install anything. For this recipe, assume the payroll client opens normally when you launch it by hand.+ </Step>++ <Step>+ ### Install Cua Driver++ Run the Windows installer in PowerShell on that same machine:++ ```powershell+ irm https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.ps1 | iex+ ```++ Open a new PowerShell window after the installer updates your user `Path`, then verify the CLI resolves:++ ```powershell+ cua-driver --version+ ```+ </Step>++ <Step>+ ### Keep the daemon in the interactive session++ Windows OpenSSH and services run in Session 0, which has no desktop. Register Cua Driver as an interactive-session daemon so window tools can see the RDP or console desktop:++ ```powershell+ cua-driver autostart enable+ cua-driver autostart kick+ ```++ Verify that the daemon is running and reports a real session number:++ ```powershell+ cua-driver status+ # Cua Driver daemon is running+ # socket: \\.\pipe\cua-driver+ # pid: 12345+ # session: 2+ ```++ <Callout type="info">+ For the full daemon setup, see [Keep Cua Driver running](/how-to-guides/driver/keep-running). If you connect to this box over SSH, also read [Drive a Windows app over SSH](/how-to-guides/driver/windows-ssh).+ </Callout>+ </Step>++ <Step>+ ### Check that Cua Driver sees the real GUI++ Run tool calls from the same Windows machine:++ ```powershell+ cua-driver call list_apps+ cua-driver call list_windows+ ```++ The output should include the legacy app, such as `Payroll Client` or `eGecko`, and its active windows. It should not be an empty array:++ ```json+ []+ ```++ Empty output usually means the command is running outside the interactive desktop. Recheck `cua-driver status`, `query session`, and the autostart task.+ </Step>++ <Step>+ ### Register Cua Driver with your agent++ Add Cua Driver as an MCP stdio server. For Claude Code on Windows:++ ```powershell+ claude mcp add --transport stdio cua-driver -- cua-driver.exe mcp+ ```++ Then verify the server entry:++ ```powershell+ claude mcp list+ ```++ <Callout type="info">+ For other MCP clients, see [Connect Cua Driver to an MCP client](/how-to-guides/driver/connect-your-agent).+ </Callout>+ </Step>++ <Step>+ ### Drive the legacy app++ Stage the input files on the Windows box first, for example `C:\creds.txt` and `C:\Users\you\Desktop\new_hire.xlsx`. Cua Driver does not read or write files directly. It drives the apps that open those files.++ Start the agent from the Windows machine, then give it a concrete task:++ ```text+ Open the payroll client, log in with the credentials in C:\creds.txt, create a new employee from new_hire.xlsx, generate the onboarding report, and export it as PDF to the Desktop.+ ```++ The agent should use `launch_app` to open the client, `list_apps` and `list_windows` to find the running process and target window, `get_window_state` to inspect the UI (it returns the accessibility tree and a screenshot), then `click` and `type_text` to complete the workflow.+ </Step>+</Steps>++## Scale this out++When the same workflow must run across many employees, regions, or tenants at once, move it onto **Cua Sandbox** cloud Windows desktops and fan out the jobs. Start with [Run sandboxes in parallel](/how-to-guides/sandbox/scale-out) and [Drive a sandbox with the SDK](/tutorials/run-an-agent-in-a-sandbox).++The driving logic stays the same. The host changes from your VPN-connected Windows box to a fleet of cloud desktops with the VPN configuration baked into the image. See [Choose and build a sandbox image](/how-to-guides/sandbox/images).diff --git a/docs/content/docs/how-to-guides/recipes/build-a-report-in-a-native-app.mdx b/docs/content/docs/how-to-guides/recipes/build-a-report-in-a-native-app.mdxnew file mode 100644index 0000000000..5a091b8171--- /dev/null+++ b/docs/content/docs/how-to-guides/recipes/build-a-report-in-a-native-app.mdx@@ -0,0 +1,116 @@+---+title: Build a report in a native app+description: Generate a financial report in Numbers on macOS by driving the native app with Cua Driver.+---++import { Callout } from 'fumadocs-ui/components/callout';+import { Steps, Step } from 'fumadocs-ui/components/steps';++Numbers is a **macOS-only** app with **no clean automation API** for building a spreadsheet, charting it, and exporting a polished PDF. With Cua Driver, the agent drives the *real Numbers app on a real Mac*, the same way a person would, through the GUI.++<Callout type="warn">+ This recipe is macOS-only. Numbers, Keynote, and Pages do not exist on Linux or Windows, so you need a real Mac now, or a cloud macOS desktop later.+</Callout>++<Steps>+ <Step>+ ### Check Numbers.app++ On your Mac, make sure Numbers.app is installed:++ ```bash+ open -a Numbers+ ```++ Numbers ships with macOS. If you removed it, install it again from the App Store before continuing.+ </Step>++ <Step>+ ### Install Cua Driver++ Install `cua-driver` with the one-line installer:++ ```bash+ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.sh)"+ ```++ Verify the CLI is available:++ ```bash+ cua-driver --version+ # cua-driver 0.5.x+ ```+ </Step>++ <Step>+ ### Grant macOS permissions++ Grant Accessibility and Screen Recording to the driver, then verify both grants:++ ```bash+ cua-driver call check_permissions+ ```++ If either permission is missing, approve it in macOS System Settings, then run the check again.++ For long-running report jobs, configure the daemon so macOS attributes the permissions to the driver process. See [Keep Cua Driver running](/how-to-guides/driver/keep-running).+ </Step>++ <Step>+ ### Register the MCP server++ Register `cua-driver` as a stdio MCP server in your agent:++ ```bash+ claude mcp add --transport stdio cua-driver -- cua-driver mcp+ ```++ Other MCP clients use the same `cua-driver mcp` command with their own config format. See [Connect Cua Driver to an MCP client](/how-to-guides/driver/connect-your-agent).+ </Step>++ <Step>+ ### Ask the agent to build the report++ Give the agent the task in plain language:++ ```text+ Fetch the last 30 days of AAPL daily closing prices. Open Numbers, create a new spreadsheet, enter Date and Close columns, add a 2D line chart, then export the document as a PDF to ~/reports/aapl-analysis.pdf.+ ```++ The agent drives Numbers with `launch_app`, `list_windows`, `click`, `type_text`, `set_value`, and `get_window_state`.++ Cua Driver **does not include HTTP or filesystem tools**. To fetch prices, the agent can drive Terminal and run `curl`, or drive a browser and copy the data from the page. For example:++ ```bash+ curl "https://query1.finance.yahoo.com/v8/finance/chart/AAPL?interval=1d&range=1mo"+ ```++ This Yahoo Finance endpoint is unofficial and may require a `User-Agent` header. If your data source needs authentication or a different format, have the agent use Terminal or the browser to retrieve it, then paste or enter the rows into Numbers.++ AppleScript can help with small app commands if the agent runs it through Terminal, but the main workflow is still the agent driving the Numbers GUI through Cua Driver.+ </Step>++ <Step>+ ### Confirm the PDF++ Confirm the exported report exists:++ ```bash+ ls -lh ~/reports/aapl-analysis.pdf+ ```++ Open it to verify the chart and table landed in the PDF:++ ```bash+ open ~/reports/aapl-analysis.pdf+ ```+ </Step>+</Steps>++## Scale this out++One Mac builds one Numbers report at a time. To generate many reports at once, for example one per ticker, client, or reporting period, move the same workflow onto **Cua Sandbox** cloud macOS desktops and fan them out.++Use [Run sandboxes in parallel](/how-to-guides/sandbox/scale-out) to run the jobs concurrently, [Choose and build a sandbox image](/how-to-guides/sandbox/images) to bake Numbers and any helper tools into the macOS image, and [Drive a sandbox with the SDK](/tutorials/run-an-agent-in-a-sandbox) to run an agent in each desktop.++A cloud macOS desktop is the only way to scale Numbers automation, since Numbers cannot run on Linux containers.diff --git a/docs/content/docs/how-to-guides/recipes/export-contacts-overnight.mdx b/docs/content/docs/how-to-guides/recipes/export-contacts-overnight.mdxnew file mode 100644index 0000000000..31e56eae4e--- /dev/null+++ b/docs/content/docs/how-to-guides/recipes/export-contacts-overnight.mdx@@ -0,0 +1,112 @@+---+title: Export contacts overnight+description: Extract LinkedIn or X contacts overnight from a logged-in browser session into a clean CSV.+---++import { Callout } from 'fumadocs-ui/components/callout';+import { Steps, Step } from 'fumadocs-ui/components/steps';++After a networking event, use Cua Driver on the machine where you already have a logged-in browser. The agent inherits that **real authenticated session** from the browser profile, so there are no credentials in the script and no anti-bot fight. Let it run overnight, then wake up to a CSV.++<Steps>++<Step>++### Log in once++On your own machine, open the browser profile that Cua Driver will drive and log into LinkedIn, X, or both. Do the normal login yourself, including any two-factor prompts. The *session persistence* is the whole trick: the overnight job starts from a browser that is already authenticated.++<Callout type="warn">+ Keep the machine awake for the whole run. Disable sleep, leave the browser profile available, and keep the network connection up so the overnight run does not pause.+</Callout>++</Step>++<Step>++### Install Cua Driver++Install `cua-driver` on the same machine:++```bash+/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.sh)"+```++Verify that the CLI is on your PATH:++```bash+cua-driver --version+```++</Step>++<Step>++### Grant macOS permissions++On macOS, grant Accessibility and Screen Recording. Then verify the driver can see its permission state:++```bash+cua-driver call check_permissions+```++For overnight runs, keep the daemon running instead of depending on a terminal session. See [Keep Cua Driver running](/how-to-guides/driver/keep-running).++</Step>++<Step>++### Register it with your agent++Add `cua-driver` as a stdio MCP server in your agent. For Claude Code:++```bash+claude mcp add --transport stdio cua-driver -- cua-driver mcp+```++For other MCP clients, see [Connect Cua Driver to an MCP client](/how-to-guides/driver/connect-your-agent).++</Step>++<Step>++### Give the overnight task++Ask the agent for the exact extraction you want. For LinkedIn, start from the connections page and cap the run with a clear `N` while you test:++```text+Open the browser to https://www.linkedin.com/mynetwork/invite-connect/connections/ .++For each of the first 50 connections:+1. Open the profile.+2. Read first name, last name, role, company, and profile URL.+3. Append one row to ~/contacts.csv.+4. Set met_at to "Google Devfest Toronto" for every row.++Use this CSV header:+first,last,role,company,met_at,linkedin++If a value is missing, leave that field empty and keep going.+```++The agent will use `page` for browser navigation, then `click`, `type_text`, and `get_window_state` to inspect and drive the browser. It may call `list_windows` first if more than one browser window is open.++Cua Driver **does not expose a filesystem write tool**. The agent writes `~/contacts.csv` by driving an app that can write files, for example a terminal running shell commands or a text editor with the CSV open.++</Step>++<Step>++### Let it run++Kick off the task before you leave. In the morning, open `~/contacts.csv` and spot-check the rows before importing them into a CRM or email tool.++</Step>++</Steps>++## Scale this out++One logged-in browser profile handles one account at a time. To process many accounts, events, or platforms in parallel overnight, move each authenticated session into its own **Cua Sandbox** cloud desktop. Log in once per sandbox, persist the session in the image, then fan the jobs out.++Use [Run sandboxes in parallel](/how-to-guides/sandbox/scale-out) to run many desktops at once, [Images](/how-to-guides/sandbox/images) to bake the logged-in session into an image, and [Run an agent in a sandbox](/tutorials/run-an-agent-in-a-sandbox) for the agent loop.diff --git a/docs/content/docs/how-to-guides/recipes/fill-a-form-from-a-local-file.mdx b/docs/content/docs/how-to-guides/recipes/fill-a-form-from-a-local-file.mdxnew file mode 100644index 0000000000..dcd56ce1bf--- /dev/null+++ b/docs/content/docs/how-to-guides/recipes/fill-a-form-from-a-local-file.mdx@@ -0,0 +1,105 @@+---+title: Fill a form from a local file+description: Drive a browser form from a PDF or CSV file on your local machine.+---++import { Callout } from 'fumadocs-ui/components/callout';+import { Steps, Step } from 'fumadocs-ui/components/steps';++Use this recipe when the data already lives on your machine, for example in a PDF resume or a CSV file, and the form lives in a browser on the same desktop. **Cua Driver** drives both, so the **data never has to be uploaded anywhere** because the agent reads it locally and types it into the form.++<Steps>+ <Step>+ ### Put the source file on the machine++ Save the source file somewhere the desktop user can open it, for example:++ ```text+ ~/Downloads/resume.pdf+ ```++ Cua Driver has **no filesystem read tool**. The agent reads local files by opening them in an app it can see, for example Preview, another PDF viewer, or a text editor, then reading the on-screen content with the accessibility tree and screenshots from `get_window_state`.++ For a CSV, open the file in a text editor, spreadsheet app, or browser tab. If the file contains many rows, start with one row and name the row or record the agent should use.+ </Step>++ <Step>+ ### Install Cua Driver++ Run the one-line installer:++ ```bash+ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.sh)"+ ```++ Verify the CLI is available:++ ```bash+ cua-driver --version+ # cua-driver 0.5.x+ ```+ </Step>++ <Step>+ ### Grant desktop permissions++ On macOS, grant Accessibility and Screen Recording so Cua Driver can inspect windows, click controls, and type into apps.++ ```bash+ cua-driver call check_permissions+ ```++ Approve both prompts, then run the check again:++ ```bash+ cua-driver call check_permissions+ # Accessibility: granted.+ # Screen Recording: granted.+ ```++ For a driver process that survives reboots and keeps the right macOS TCC attribution, see [Keep Cua Driver running](/how-to-guides/driver/keep-running).+ </Step>++ <Step>+ ### Register Cua Driver with your MCP client++ For Claude Code, add Cua Driver as a stdio MCP server:++ ```bash+ claude mcp add --transport stdio cua-driver -- cua-driver mcp+ ```++ Other MCP clients use the same `cua-driver mcp` command. See [Connect Cua Driver to an MCP client](/how-to-guides/driver/connect-your-agent) for Cursor, Codex, Gemini CLI, and other clients.+ </Step>++ <Step>+ ### Give the agent the task++ In your MCP client, describe the local file, the fields to read, and the form URL:++ ```text+ Open ~/Downloads/resume.pdf in the PDF viewer. Read the name, email,+ phone, and work history. Then open+ https://form.jotform.com/<your-form-id> in the browser, fill each matching+ field from the PDF, and submit the form.+ ```++ The agent can use `launch_app` to open the PDF viewer, `page` to drive the browser, `get_window_state` or `get_accessibility_tree` to read visible fields, and `click`, `type_text`, or `set_value` to fill the form. It can set `capture_mode` to `vision` on `get_window_state` when a screenshot is needed.+ </Step>++ <Step>+ ### Watch the field mapping++ Keep the browser visible while the agent maps source fields to form fields. If a form label is ambiguous, correct the mapping in plain language, for example "use the resume email for Work Email" or "leave Current Employer blank".++ <Callout type="info">+ Forms vary, so the agent works best when you name the exact fields to fill, such as name, email, and phone, instead of asking it to fill the whole form.+ </Callout>+ </Step>+</Steps>++## Scale this out++One desktop fills one form at a time. To submit many forms in parallel, for example one per CSV row or one per applicant, move the same workflow onto **Cua Sandbox** cloud desktops. Drop each source file into its own sandbox, start one agent job per sandbox, and fan the jobs out.++Use [Run sandboxes in parallel](/how-to-guides/sandbox/scale-out) to run many jobs concurrently. Use [Choose and build a sandbox image](/how-to-guides/sandbox/images) to stage the source files in an image or mount them into each sandbox. Use [Run an agent in a sandbox](/tutorials/run-an-agent-in-a-sandbox) when you want the same MCP-driven workflow to run inside a disposable cloud desktop.diff --git a/docs/content/docs/how-to-guides/recipes/meta.json b/docs/content/docs/how-to-guides/recipes/meta.jsonnew file mode 100644index 0000000000..d8f91a7618--- /dev/null+++ b/docs/content/docs/how-to-guides/recipes/meta.json@@ -0,0 +1 @@+{ "title": "Recipes", "pages": ["automate-a-legacy-windows-app-behind-a-vpn", "export-contacts-overnight", "build-a-report-in-a-native-app", "fill-a-form-from-a-local-file"] }diff --git a/docs/content/docs/how-to-guides/sandbox/images.mdx b/docs/content/docs/how-to-guides/sandbox/images.mdxnew file mode 100644index 0000000000..39ea45d72d--- /dev/null+++ b/docs/content/docs/how-to-guides/sandbox/images.mdx@@ -0,0 +1,145 @@+---+title: Choose and build a sandbox image+description: Select a sandbox base image and add packages, files, setup commands, ports, and inspection.+---++import { Callout } from 'fumadocs-ui/components/callout';++Use an **Image** to define the OS and software environment for a sandbox before you start it.++## Choose a base image++An Image describes the OS and software environment. Images are *immutable* and *chainable*. Each builder method returns a new Image.++Start with the built-in constructor that matches the operating system and isolation level you need.++```python+from cua import Image++Image.linux() # Ubuntu 24.04 VM (default, QEMU)+Image.linux(kind='container') # Ubuntu 24.04 container (lighter, Docker/XFCE)+Image.linux('ubuntu', '22.04') # Older Ubuntu+Image.macos() # macOS Tahoe (version '26', latest)+Image.macos('15') # macOS Sequoia+Image.windows() # Windows 11+Image.android() # Android 14+```++| Image | Kind | Notes |+|-------|------|-------|+| Image.linux() | VM | Default (QEMU), supports /dev/kvm |+| Image.linux(kind='container') | container | Lighter, Docker/XFCE |+| Image.macos() | VM | Apple Virtualization / Lume |+| Image.windows() | VM | QEMU or Hyper-V |+| Image.android() | VM | QEMU Android emulator |++## Install packages++Use the package manager for the target operating system.++```python+Image.linux().apt_install('curl', 'git', 'ffmpeg')+Image.macos().brew_install('ffmpeg', 'jq')+Image.windows().choco_install('nodejs', 'git')+Image.windows().winget_install('Microsoft.VisualStudioCode')+Image.android().apk_install('/path/to/app.apk')+```++Install Python packages with `pip_install()` or `uv_install()`.++```python+Image.linux().pip_install('numpy', 'pandas', 'playwright')+Image.linux().uv_install('numpy', 'pandas') # faster, installs into cua-server project+```++## Set environment variables++Set non-sensitive environment variables with `.env()`.++```python+Image.linux().env(DATABASE_URL='postgres://localhost/mydb', LOG_LEVEL='debug')+```++<Callout type="warn">+ Avoid putting real secrets in .env() because anyone who can read the Image spec can see the values.+</Callout>++## Run setup commands++Run arbitrary shell commands during setup with `.run()`.++```python+Image.linux().run('curl -fsSL https://deb.nodesource.com/setup_20.x | bash -')+```++## Copy files++Copy local files into the image with `.copy()`.++```python+Image.linux().copy('./config.json', '/app/config.json')+```++## Expose ports++Expose every port the sandbox workload must listen on.++```python+Image.linux().expose(8080).expose(5432)+```++## Chain builder calls++Chaining: builder calls are composable, each returns a new Image:++```python+img = (+ Image.linux()+ .apt_install('curl', 'git', 'ffmpeg')+ .pip_install('requests', 'Pillow')+ .env(MY_TOKEN='abc123', DEBUG='1')+ .run('mkdir -p /app/data')+ .expose(8080)+)+```++Fork a base image for different use cases:++```python+base = Image.linux().apt_install('curl', 'git')+dev = base.pip_install('ipython', 'rich').env(DEBUG='1')+prod = base.pip_install('gunicorn').run('useradd -m appuser')+```++## Use a custom OCI image++Use `Image.from_registry()` when your base image already exists in a registry.++```python+Image.from_registry('ghcr.io/trycua/macos-tahoe-cua:latest')+Image.from_registry('ubuntu:22.04')+# chain builder on top+img = Image.from_registry('ubuntu:22.04').apt_install('curl')+```++## Use a local disk image++Use `Image.from_file()` for qcow2, vhdx, raw, or iso disk images. URLs are also supported and cached automatically.++```python+Image.from_file('/path/to/disk.qcow2', os_type='linux')+Image.from_file('/path/to/windows.vhdx', os_type='windows')+Image.from_file('https://example.com/disk.qcow2', os_type='linux')+```++Images are cached in `~/.cua/cua-sandbox/image-cache/`.++## Inspect an image++Call `.to_dict()` to inspect the final image spec before using it.++```python+img = Image.linux().apt_install('curl').pip_install('requests')+print(img.to_dict())+# {'os_type': 'linux', 'distro': 'ubuntu', 'version': '24.04', 'kind': 'container', 'layers': [...]}+```diff --git a/docs/content/docs/how-to-guides/sandbox/interactive-shell.mdx b/docs/content/docs/how-to-guides/sandbox/interactive-shell.mdxnew file mode 100644index 0000000000..bd83b2d39f--- /dev/null+++ b/docs/content/docs/how-to-guides/sandbox/interactive-shell.mdx@@ -0,0 +1,64 @@+---+title: Open an interactive shell in a sandbox+description: Open an interactive PTY session inside the code half of a sandbox from the CLI or Python SDK.+---++A **Cua Sandbox** is a full isolated computer where an agent can run code and drive the GUI. Use `cua do shell` or the Sandbox SDK terminal API to open an interactive PTY session in the code half of that sandbox, similar to SSH but without network setup. The same sandbox is also driven through the GUI via computer-use with screenshots, clicks, and typing. Because both share one machine, you can run code and automate the UI against the same filesystem and state.++## From the CLI++Switch to the sandbox target first.++```bash+# Switch to a target first+cua do switch docker my-container+# or+cua do switch host++# Open an interactive shell (bash / PowerShell on host)+cua do shell++# Run a specific program interactively+cua do shell python3+cua do shell vim /etc/hosts++# Override terminal dimensions+cua do shell --cols 220 --rows 50+```++The shell is fully interactive: arrow keys, tab completion, and Ctrl+C work. When stdin is piped (non-TTY), the command runs non-interactively.++## From the Python SDK++Create a sandbox, open a PTY session with `sb.terminal.create`, send input as text, read session info, then close the session.++```python+from cua import Sandbox, Image++async with Sandbox.ephemeral(Image.linux()) as sb:+ session = await sb.terminal.create(cols=80, rows=24)+ pid = session['pid']++ await sb.terminal.send_input(pid, 'echo hello\n')++ info = await sb.terminal.info(pid)+ print(info)++ closed = await sb.terminal.close(pid)+ print(f'closed: {closed}')+```++Pass a `command` to run a specific program instead of the default login shell:++```python+session = await sb.terminal.create(command='python3', cols=80, rows=24)+```++## Terminal methods++| Method | Description |+|--------|-------------|+| create(command=None, cols=80, rows=24) | Open a PTY session; returns a dict with the session `pid`, `cols`, and `rows` |+| send_input(pid, data) | Send input text to the session |+| info(pid) | Return session info, or `None` if the session is gone |+| close(pid) | Kill the session; returns a bool |diff --git a/docs/content/docs/how-to-guides/sandbox/lifecycle.mdx b/docs/content/docs/how-to-guides/sandbox/lifecycle.mdxnew file mode 100644index 0000000000..5fccab22df--- /dev/null+++ b/docs/content/docs/how-to-guides/sandbox/lifecycle.mdx@@ -0,0 +1,103 @@+---+title: Sandbox lifecycle+description: Choose the right Sandbox SDK lifecycle pattern for temporary, persistent, and reconnectable sandboxes.+---++import { Callout } from 'fumadocs-ui/components/callout';++Use the **lifecycle pattern** that matches whether you want the sandbox *deleted after the job*, *kept for later*, or *attached to while it is already running*.++## Create and auto-destroy a sandbox++Use `Sandbox.ephemeral` for scripts, CI, and one-off tasks.++```python+import asyncio+from cua import Sandbox, Image++async def main():+ async with Sandbox.ephemeral(Image.linux()) as sb:+ result = await sb.shell.run('echo hello')+ print(result.stdout) # 'hello\n'+ # sandbox deleted here++asyncio.run(main())+```++`Sandbox.ephemeral` is **auto-destroyed** when the `async with` block exits. Its context manager tears the sandbox down automatically.++## Create a persistent sandbox++Use `Sandbox.create` when the sandbox **must outlive the script**. Call `await sb.disconnect()` to drop the connection while the sandbox keeps running.++```python+sb = await Sandbox.create(Image.linux(), name='my-dev-sandbox')+await sb.shell.run('apt-get install -y vim')+await sb.disconnect() # sandbox keeps running+```++Delete the sandbox when you are done with it.++```python+async with Sandbox.connect('my-dev-sandbox') as sb:+ await sb.destroy()+```++| Method | Effect | When |+|--------|--------|------|+| `await sb.disconnect()` | keeps running | reconnect later |+| `await sb.destroy()` | permanently destroyed | done with it |+| `await Sandbox.delete(name)` | permanently destroyed (by name, no connect) | classmethod |++## Reconnect to a running sandbox++Use `Sandbox.connect` to attach to an already-running sandbox.++<Callout type="info">+ Sandbox.connect never starts or stops the sandbox. It only manages the network connection.+</Callout>++Reconnect by name.++```python+async with Sandbox.connect('my-dev-sandbox') as sb:+ result = await sb.shell.run('vim --version')+ print(result.stdout)+```++`Sandbox.connect` supports both `await` and `async with`. Its context manager calls `disconnect()` automatically, so the sandbox keeps running after the connection closes.++```python+async with Sandbox.connect('my-sandbox') as sb:+ await sb.shell.run('echo reconnected')+# connection dropped, sandbox keeps running+```++## List running sandboxes++List available sandboxes, then filter to local runtimes when needed.++```python+sandboxes = await Sandbox.list()+for info in sandboxes:+ print(info.name, info.os_type, info.status)++# list only local sandboxes+sandboxes = await Sandbox.list(local=True)+```++## Cloud vs local++Use cloud by default. Set `CUA_API_KEY` in the environment before creating cloud sandboxes.++```python+# cloud (default)+async with Sandbox.ephemeral(Image.linux()) as sb: ...+```++Pass `local=True` for local runtimes such as Docker, Lume, or QEMU.++```python+# local Docker+async with Sandbox.ephemeral(Image.linux(), local=True) as sb: ...+```diff --git a/docs/content/docs/how-to-guides/sandbox/meta.json b/docs/content/docs/how-to-guides/sandbox/meta.jsonnew file mode 100644index 0000000000..323fc062b9--- /dev/null+++ b/docs/content/docs/how-to-guides/sandbox/meta.json@@ -0,0 +1 @@+{ "title": "Sandbox", "pages": ["lifecycle", "snapshots", "secrets", "scale-out", "tunneling", "images", "interactive-shell"] }diff --git a/docs/content/docs/how-to-guides/sandbox/scale-out.mdx b/docs/content/docs/how-to-guides/sandbox/scale-out.mdxnew file mode 100644index 0000000000..c9e604d01c--- /dev/null+++ b/docs/content/docs/how-to-guides/sandbox/scale-out.mdx@@ -0,0 +1,105 @@+---+title: Run sandboxes in parallel+description: Run many sandbox jobs at once, limit concurrency, and handle failures.+---++import { Callout } from 'fumadocs-ui/components/callout';++Because `Sandbox.ephemeral` and `Sandbox.create` are async, you can **run many sandboxes at the same time** using `asyncio`.++<Callout type="info">+ Cloud sandboxes are billed per second of runtime. Running many sandboxes in parallel incurs charges proportional to wall-clock time times number of sandboxes. Use local=True during development to avoid costs.+</Callout>++## Run a fixed set concurrently++Use `asyncio.gather` when you already have the full list of sandbox jobs.++```python+import asyncio+from cua import Sandbox, Image++async def run_task(task: str) -> str:+ async with Sandbox.ephemeral(Image.linux()) as sb:+ result = await sb.shell.run(f"echo '{task}'")+ return result.stdout.strip()++async def main():+ tasks = ['task-1', 'task-2', 'task-3', 'task-4']+ results = await asyncio.gather(*[run_task(t) for t in tasks])+ print(results)++asyncio.run(main())+```++## Cap the number of concurrent sandboxes++Use an `asyncio.Semaphore` when you need *rate limiting*.++```python+import asyncio+from cua import Sandbox, Image++MAX_CONCURRENT = 5++async def process_item(sem: asyncio.Semaphore, item: str) -> dict:+ async with sem:+ async with Sandbox.ephemeral(Image.linux()) as sb:+ result = await sb.shell.run(f"python /app/process.py '{item}'")+ return {'item': item, 'output': result.stdout, 'ok': result.success}++async def main():+ items = [f'item-{i}' for i in range(20)]+ sem = asyncio.Semaphore(MAX_CONCURRENT)+ results = await asyncio.gather(*[process_item(sem, item) for item in items])+ print(f"{sum(r['ok'] for r in results)}/{len(results)} succeeded")++asyncio.run(main())+```++## Process a dynamic queue of tasks++Use `asyncio.Queue` when tasks arrive at runtime.++```python+import asyncio+from cua import Sandbox, Image++async def worker(queue: asyncio.Queue, worker_id: int):+ while True:+ task = await queue.get()+ if task is None:+ break+ async with Sandbox.ephemeral(Image.linux()) as sb:+ result = await sb.shell.run(task)+ print(f'[worker-{worker_id}] {result.stdout.strip()}')+ queue.task_done()++async def main():+ queue: asyncio.Queue = asyncio.Queue()+ num_workers = 4+ workers = [asyncio.create_task(worker(queue, i)) for i in range(num_workers)]+ for i in range(10):+ await queue.put(f"echo 'job {i}'")+ for _ in range(num_workers):+ await queue.put(None)+ await asyncio.gather(*workers)++asyncio.run(main())+```++## Handle failures++Set `return_exceptions=True` when one failed sandbox job **should not cancel the rest**.++```python+results = await asyncio.gather(+ *[run_task(t) for t in tasks],+ return_exceptions=True+)+for task, result in zip(tasks, results):+ if isinstance(result, Exception):+ print(f'{task}: FAILED — {result}')+ else:+ print(f'{task}: {result}')+```diff --git a/docs/content/docs/how-to-guides/sandbox/secrets.mdx b/docs/content/docs/how-to-guides/sandbox/secrets.mdxnew file mode 100644index 0000000000..0cb7aa629e--- /dev/null+++ b/docs/content/docs/how-to-guides/sandbox/secrets.mdx@@ -0,0 +1,97 @@+---+title: Pass secrets into a sandbox+description: Read secrets from the host and inject them into a sandbox at runtime.+---++import { Callout } from 'fumadocs-ui/components/callout';++Pass secrets from the host environment into a sandbox **at runtime**, and keep image metadata for non-sensitive configuration.++## Set the Cua API key++The Cua API key is read from CUA_API_KEY environment variable. You can also pass it explicitly:++```python+async with Sandbox.ephemeral(Image.linux(), api_key=my_key) as sb:+ ...+```++## Inject secrets at runtime++Read secrets from the host environment, then inject them into the sandbox with `sb.shell.run`.++```python+import os+import asyncio+from cua import Sandbox, Image++async def main():+ db_url = os.environ['DATABASE_URL']+ gh_token = os.environ['GITHUB_TOKEN']++ async with Sandbox.ephemeral(Image.linux()) as sb:+ await sb.shell.run(f"export DATABASE_URL='{db_url}'")+ await sb.shell.run(f"export GITHUB_TOKEN='{gh_token}'")+ await sb.shell.run('python /app/main.py')++asyncio.run(main())+```++For multi-command sessions, use a wrapper script:++```python+script = f"""+export DATABASE_URL='{db_url}'+export GITHUB_TOKEN='{gh_token}'+python /app/main.py+"""+result = await sb.shell.run(script)+```++## Non-sensitive config with Image.env()++Use `Image.env()` for non-sensitive config, **not for secrets**.++```python+img = (+ Image.linux()+ .apt_install('python3')+ .env(+ LOG_LEVEL='info',+ APP_ENV='production',+ PORT='8080',+ )+)+```++<Callout type="warn">+ .env() values are stored in the Image spec and visible to anyone who can inspect it. Do not use .env() for API keys, passwords, or tokens.+</Callout>++## Copy secret files into the sandbox++Copy SSH keys, certificates, and other secret files into the sandbox at runtime, then set file permissions.++```python+async with Sandbox.ephemeral(Image.linux()) as sb:+ with open(os.path.expanduser('~/.ssh/id_rsa')) as f:+ key_content = f.read()+ await sb.shell.run('mkdir -p /root/.ssh && chmod 700 /root/.ssh')+ await sb.shell.run(f"cat > /root/.ssh/id_rsa << 'EOF'\n{key_content}\nEOF")+ await sb.shell.run('chmod 600 /root/.ssh/id_rsa')+```++For files safe to bake in, use Image.copy():++```python+img = Image.linux().copy('./app-config.json', '/app/config.json')+```++## Best practices++| Do | Don't |+|----|-------|+| Read secrets from host environment variables | Hardcode secrets in Python source |+| Inject secrets at runtime via sb.shell.run | Store secrets in Image.env() |+| Use .env files + python-dotenv locally | Commit .env files to version control |+| Use your CI/CD provider's secret manager | Log or print secret values |diff --git a/docs/content/docs/how-to-guides/sandbox/snapshots.mdx b/docs/content/docs/how-to-guides/sandbox/snapshots.mdxnew file mode 100644index 0000000000..30f4f0c7fd--- /dev/null+++ b/docs/content/docs/how-to-guides/sandbox/snapshots.mdx@@ -0,0 +1,87 @@+---+title: 'Take and restore snapshots'+description: Capture a sandbox disk state and fork fresh sandboxes from that point.+---++import { Callout } from 'fumadocs-ui/components/callout';++A *snapshot* captures the **full disk state** of a running sandbox and returns a new Image that you can pass to Sandbox.ephemeral() or Sandbox.create() to start a fresh sandbox from that point.++## Snapshot and restore++<Callout type="info">+ Snapshots are currently supported on Cua Cloud sandboxes. Local sandboxes (Docker, Lume, QEMU) have limited snapshot support.+</Callout>++Call `snapshot()` on a running sandbox after you make the changes you want to keep.++```python+import asyncio+from cua import Sandbox, Image++async def main():+ async with Sandbox.ephemeral(Image.linux('ubuntu', '24.04')) as sb:+ await sb.shell.run('apt-get update -qq && apt-get install -y -qq cowsay')+ cowsay_image = await sb.snapshot(name='with-cowsay')++ # boot from snapshot — cowsay already installed+ async with Sandbox.ephemeral(cowsay_image) as sb2:+ result = await sb2.shell.run('/usr/games/cowsay hello from snapshot!')+ print(result.stdout)++asyncio.run(main())+```++Signature:++```python+await sb.snapshot(name='my-snapshot', stateful=False)+```++Parameters:++- name: str | None: optional human-readable label+- stateful: bool (default False): capture memory state in addition to disk (VMs only)++Returns: an Image++## Pre-bake an environment once, fork many times++Snapshot after slow setup, then **reuse that image for repeated runs**.++```python+async with Sandbox.ephemeral(Image.linux()) as sb:+ await sb.shell.run('apt-get update && apt-get install -y python3-pip ffmpeg')+ await sb.shell.run('pip install torch transformers')+ ml_image = await sb.snapshot(name='ml-ready')++for lr in [0.001, 0.01, 0.1]:+ async with Sandbox.ephemeral(ml_image) as sb:+ await sb.shell.run(f'python train.py --lr {lr}')+```++## Isolate state between runs++Each fork gets **its own copy of disk**, changes in one fork never affect another.++```python+async with Sandbox.ephemeral(Image.linux()) as sb:+ await sb.shell.run('echo original > /tmp/data.txt')+ base = await sb.snapshot(name='base')++async with Sandbox.ephemeral(base) as a:+ await a.shell.run('echo fork-a >> /tmp/data.txt')++async with Sandbox.ephemeral(base) as b:+ result = await b.shell.run('cat /tmp/data.txt')+ print(result.stdout) # 'original\n' — no trace of fork-a+```++## Performance++| Operation | Typical time |+|-----------|-------------|+| Create + install packages | 30-120 s |+| Fork from snapshot | 1-5 s |++On backends that use copy-on-write storage, forking is nearly instant regardless of disk image size. On dir storage backends, the fork performs a full copy.diff --git a/docs/content/docs/how-to-guides/sandbox/tunneling.mdx b/docs/content/docs/how-to-guides/sandbox/tunneling.mdxnew file mode 100644index 0000000000..59ddacef82--- /dev/null+++ b/docs/content/docs/how-to-guides/sandbox/tunneling.mdx@@ -0,0 +1,84 @@+---+title: Forward a port from a sandbox+description: Expose a sandbox port on localhost so local tools can connect to it.+---++import { Callout } from 'fumadocs-ui/components/callout';++`sb.tunnel.forward()` exposes a port (or Android abstract socket) running inside the sandbox on the local host.++## Forward a single port++Start the service inside the sandbox, then forward its port.++```python+from cua import Sandbox, Image++async with Sandbox.ephemeral(Image.linux()) as sb:+ await sb.shell.run('python3 -m http.server 8080 &')++ async with sb.tunnel.forward(8080) as t:+ print(t.url) # http://localhost:49823+ print(t.host, t.port) # localhost, 49823+```++The tunnel **closes automatically** when the async with block exits.++## Forward multiple ports at once++Pass multiple ports to get a `dict[port, TunnelInfo]`.++```python+async with sb.tunnel.forward(8080, 9222) as tunnels:+ app_url = tunnels[8080].url+ devtools_url = tunnels[9222].url+```++## Close a tunnel manually++If you need the tunnel outside an `async with` block, close it yourself.++```python+t = await sb.tunnel.forward(8080)+print(t.url)+# ... do work ...+await t.close()+```++## Android abstract sockets++Pass a string instead of a port number.++```python+async with sb.tunnel.forward('chrome_devtools_remote') as t:+ print(t.url) # http://localhost:<random>+```++<Callout type="info">+ Abstract socket names are Linux-only (Android). On Linux and macOS sandboxes use integer port numbers.+</Callout>++## TunnelInfo fields++| Field | Type | Description |+|-------|------|-------------|+| host | str | Always 'localhost' |+| port | int | The host-side port assigned |+| sandbox_port | int or str | The original port/socket inside the sandbox |+| url | str | Shorthand `http://{host}:{port}` |++## Example: Chrome DevTools Protocol++Start Chrome with remote debugging enabled, then read the DevTools target list through the forwarded port.++```python+from cua import Sandbox, Image++async with Sandbox.ephemeral(Image.linux()) as sb:+ await sb.shell.run('google-chrome --remote-debugging-port=9222 --headless &')++ async with sb.tunnel.forward(9222) as t:+ import httpx+ targets = httpx.get(f'{t.url}/json').json()+ print(targets)+```diff --git a/docs/content/docs/how-to-guides/skills/meta.json b/docs/content/docs/how-to-guides/skills/meta.jsonnew file mode 100644index 0000000000..f115067ce5--- /dev/null+++ b/docs/content/docs/how-to-guides/skills/meta.json@@ -0,0 +1 @@+{ "title": "Skills & demonstrations", "pages": ["record-a-demonstration"] }diff --git a/docs/content/docs/how-to-guides/skills/record-a-demonstration.mdx b/docs/content/docs/how-to-guides/skills/record-a-demonstration.mdxnew file mode 100644index 0000000000..0f0f1c707b--- /dev/null+++ b/docs/content/docs/how-to-guides/skills/record-a-demonstration.mdx@@ -0,0 +1,135 @@+---+title: 'Record a demonstration -> create a skill'+description: 'Demonstrate a task once over VNC and turn it into a reusable cua skill you can read, replay, and apply with new inputs.'+---+import { Callout } from 'fumadocs-ui/components/callout';++Use `cua skills record` to demonstrate a task once in a VNC session and save it as a **reusable skill**. The result is `~/.cua/skills/<name>/SKILL.md` plus the recorded trajectory assets, which you can inspect, replay as a video, and pass to `ComputerAgent` for similar tasks with new inputs. This is *record-and-replay* in the sense of show-once-get-a-reusable-skill, the cua analog of OpenAI Codex record-and-replay, not verbatim input playback.++<Callout type='info'>+This records a HUMAN demonstration into a reusable skill. It is not the Cua Driver agent-trajectory recorder. See [Demonstrations, skills, and trajectories](/explanation/demonstrations-skills-and-trajectories) for how the three recording systems differ.+</Callout>++## Prerequisites++- Start a sandbox that exposes VNC. Use a running Cua Cloud sandbox by name, or use any VNC URL, such as a local Docker sandbox at `http://localhost:8006/vnc.html`.+- Install `ffmpeg`. The recorder fails fast if it is missing.++```bash+# macOS+brew install ffmpeg++# Linux+sudo apt install ffmpeg+```++- Set an LLM API key for captioning. The default provider is `anthropic`, which reads `ANTHROPIC_API_KEY` and uses `claude-haiku-4-5` by default.++```bash+export ANTHROPIC_API_KEY=...+```++Use `--provider openai` to read `OPENAI_API_KEY` and use `gpt-4o-mini` by default. You can also pass `--api-key` directly.++## Steps++1. Start recording.++ Use exactly one target flag: `--sandbox` or `--vnc-url`. Pass `--name` and `--description` to avoid the prompts after recording.++```bash+cua skills record \+ --sandbox my-sandbox \+ --name flight-booking \+ --description "Book a flight through the browser"+```++```bash+cua skills record \+ --vnc-url http://localhost:8006/vnc.html \+ --name flight-booking \+ --description "Book a flight through the browser"+```++ The full command shape is:++```text+cua skills record [--sandbox NAME | --vnc-url URL] [--name NAME] [--description DESC] [--provider anthropic|openai] [--model NAME] [--api-key KEY]+```++ Short flags are `-s` for `--sandbox`, `-u` for `--vnc-url`, `-n` for `--name`, `-d` for `--description`, `-p` for `--provider`, `-m` for `--model`, and `-k` for `--api-key`.++2. Perform the task by hand in the VNC window.++ The CLI starts a local recording server, opens the VNC viewer in your browser with recording already enabled, and prints:++```text+Recording will start automatically when you connect.+```++ Complete the task manually in the VNC window. The recording has a 30 minute timeout.++3. Stop recording.++ Click `Stop Recording` in the VNC panel. If you did not pass `--name` or `--description`, enter them when prompted. Skill names can contain only letters, numbers, hyphens, and underscores.++4. Inspect the skill.++ After you stop recording, the CLI extracts frames at each input event, captions each step with the VLM, and writes the skill to `~/.cua/skills/<name>/`.++```bash+cua skills read flight-booking+cua skills replay flight-booking+```++ `cua skills read` prints `SKILL.md` as markdown by default. Use `--format json` when you need trajectory data and `skill_prompt`.++```bash+cua skills read flight-booking --format json+```++ `cua skills replay` opens the recorded `.mp4` in your browser. It does not replay the original inputs.++ The skill directory contains:++```text+~/.cua/skills/flight-booking/+ SKILL.md+ trajectory/+ *.mp4+ events.json+ trajectory.json+ step_N_full.jpg+```++ `SKILL.md` contains YAML frontmatter with `name` and `description`, a `Steps` section, and an `Agent Prompt` section.++5. Apply the skill to a new task.++ Read `SKILL.md`, prefix it as context to a `ComputerAgent` run, then provide the real task with different inputs. The agent follows the demonstrated workflow pattern and adapts the parameters.++```python+from pathlib import Path+from cua import ComputerAgent++skill_prompt = (Path.home() / '.cua/skills/flight-booking/SKILL.md').read_text()+agent = ComputerAgent(loop='anthropic')+messages = [+ {'role': 'user', 'content': skill_prompt},+ {'role': 'user', 'content': 'Book a flight from Boston to Chicago for next Friday'},+]+async for result in agent.run(messages):+ print(result)+```++## Manage your skills++```bash+cua skills list+cua skills ls+cua skills read flight-booking+cua skills delete flight-booking+cua skills clean+```++`cua skills clean` deletes all skills after confirmation.diff --git a/docs/content/docs/index.mdx b/docs/content/docs/index.mdxnew file mode 100644index 0000000000..a212ff2ae1--- /dev/null+++ b/docs/content/docs/index.mdx@@ -0,0 +1,35 @@+---+title: 'Cua documentation'+description: 'Open-source computer-use automation for real machines and disposable cloud desktops.'+---++Cua is an open-source MIT-licensed platform ([github.com/trycua/cua](https://github.com/trycua/cua)) for **computer-use automation**: letting AI agents operate real computers by clicking, typing, and reading the screen and accessibility tree. The main decision is whether to drive a machine you already have with Cua Driver, or spin up a fresh, isolated cloud desktop with Cua Sandbox.++import { Card, Cards } from 'fumadocs-ui/components/card';++<Cards>+ <Card+ title="Drive a real app — Cua Driver"+ href="/tutorials/drive-your-first-app"+ >+ Run a background driver on macOS, Windows, or Linux so an agent can operate native GUI apps you already have without bringing them to the foreground or moving the real mouse. It speaks MCP over stdio and is also a plain CLI.+ </Card>+ <Card+ title="Spin up a cloud desktop — Cua Sandbox"+ href="/tutorials/your-first-cloud-sandbox"+ >+ Start disposable full Linux, Windows, or macOS VMs with a GUI for an agent to drive when you want a fresh, isolated computer in the cloud instead of your own machine.+ </Card>+</Cards>++## How these docs are organised++Tutorials: learn by doing. Start with [Tutorials](/tutorials).++Explanation: understand how and why Cua works. Start with [Explanation](/explanation).++How-to guides: recipes for a goal. Start with [How-to guides](/how-to-guides).++Reference: precise technical description. Start with [Reference](/reference).++Cua also has an Agent SDK, a Computer SDK, CLIs, and an MCP server.diff --git a/docs/content/docs/lume/examples/claude-code/homebrew-testing.mdx b/docs/content/docs/lume/examples/claude-code/homebrew-testing.mdxdeleted file mode 100644index bbb2582861..0000000000--- a/docs/content/docs/lume/examples/claude-code/homebrew-testing.mdx+++ /dev/null@@ -1,205 +0,0 @@-----title: Homebrew Testing-description: Test Homebrew formulas in a clean, isolated macOS environment------import { Callout } from 'fumadocs-ui/components/callout';--This example demonstrates testing Homebrew packages in a **clean, isolated macOS environment**. Unlike your host machine cluttered with dependencies, a fresh VM sandbox ensures reproducible testing conditions.--## Why use a sandbox?--Testing Homebrew formulas on your host has problems:--- **Dependency pollution** — Existing packages may mask missing dependencies-- **State accumulation** — Previous installs leave behind configs and files-- **Risk of breakage** — Testing can corrupt your development environment--A macOS VM sandbox solves these:--- **Clean slate** — Every test starts from a known state-- **Isolation** — Bad installs don't affect your host-- **Reproducibility** — Others can replicate your exact environment--## Prerequisites--- [Sandbox VM set up](/lume/examples/claude-code/sandbox)-- A "golden image" VM for quick resets--### Create a golden image--Before testing, create a clean snapshot you can restore:--```bash-# Stop the sandbox if running-lume stop sandbox--# Create a golden image-lume clone sandbox sandbox-golden--# Now you can always reset to this clean state-```--## Example prompt--Ask Claude Code:--> "SSH into the sandbox. Install the 'jq' Homebrew formula, verify it works by parsing some JSON, then tell me if it installed correctly."--## What Claude does--1. **Connects to the sandbox** via SSH-2. **Installs the formula** with `brew install`-3. **Tests the installation** with sample data-4. **Reports results** including version and functionality--## Example session--```-You: SSH into [email protected]. Test the 'jq' Homebrew formula -- install it, verify it parses JSON correctly, and report the results.--Claude: I'll connect to the sandbox and test the jq formula.--[Runs: ssh [email protected]]--First, let me check if Homebrew is installed:--[Runs: brew --version]-Homebrew 4.2.0--Now installing jq:--[Runs: brew install jq]-==> Downloading https://ghcr.io/v2/homebrew/core/jq/manifests/1.7.1-==> Fetching jq-==> Installing jq-==> Pouring jq--1.7.1.arm64_sonoma.bottle.tar.gz-/opt/homebrew/Cellar/jq/1.7.1: 19 files, 1.4MB--Verifying the installation:--[Runs: jq --version]-jq-1.7.1--Testing JSON parsing:--[Runs: echo '{"name": "test", "values": [1, 2, 3]}' | jq '.name']-"test"--[Runs: echo '{"name": "test", "values": [1, 2, 3]}' | jq '.values | add']-6--Testing with a more complex query:--[Runs: curl -s https://api.github.com/repos/jqlang/jq | jq '{name, stars: .stargazers_count, language}']-{- "name": "jq",- "stars": 28534,- "language": "C"-}--Test Results:-- Installation: Success-- Version: jq-1.7.1-- Basic parsing: Working-- Array operations: Working-- API response parsing: Working--The jq formula installed correctly and all functionality tests passed.-```--## Resetting for the next test--After testing, reset the sandbox to the clean state:--```bash-# Stop and delete the modified sandbox-lume stop sandbox-lume delete sandbox--# Restore from golden image-lume clone sandbox-golden sandbox--# Start fresh for the next test-lume run sandbox --no-display --shared-dir ~/my-project-```--Or automate it in a script:--```bash-#!/bin/bash-# reset-sandbox.sh--VM_NAME="sandbox"-GOLDEN_IMAGE="sandbox-golden"--lume stop $VM_NAME 2>/dev/null-lume delete $VM_NAME 2>/dev/null-lume clone $GOLDEN_IMAGE $VM_NAME-lume run $VM_NAME --no-display--echo "Sandbox reset complete"-```--## Testing your own formula--If you're developing a Homebrew formula:--```-You: SSH into the sandbox. Install my formula from ./my-formula.rb in the shared folder- and verify it works.--Claude: I'll test your local formula.--[Runs: ssh [email protected]]-[Runs: cd /Volumes/My\ Shared\ Files]-[Runs: brew install --formula ./my-formula.rb]--# Claude tests the installed tool and reports any issues-```--## Common testing scenarios--### Test a formula with dependencies--```-You: Install 'ffmpeg' and verify video conversion works.-```--Claude will install ffmpeg with all its dependencies and test encoding.--### Test formula upgrades--```-You: The sandbox has jq 1.6. Upgrade to 1.7 and verify nothing breaks.-```--### Test conflicting packages--```-You: Try installing both '[email protected]' and '[email protected]', then test importing- packages in each version.-```--### Verify formula uninstallation--```-You: Install jq, uninstall it, and verify it's completely removed with no leftover files.-```--## Why this matters--| Testing Approach | Clean Environment | Isolated | Resettable |-| ---------------- | ----------------- | -------- | ------------------ |-| Host machine | No | No | No |-| Docker | Yes | Yes | Yes (but no macOS) |-| **macOS VM** | **Yes** | **Yes** | **Yes** |--Homebrew is macOS-specific. Testing Homebrew formulas requires macOS, and testing them properly requires isolation.--## Next steps--- [Sandbox Setup](/lume/examples/claude-code/sandbox) — Initial VM configuration-- [Claude Cowork with MCP](/lume/examples/claude-cowork/sandbox) — Use MCP for native VM integration-- [Unattended Setup](/lume/guide/fundamentals/unattended-setup) — Pre-configure VMs with Homebrewdiff --git a/docs/content/docs/lume/examples/claude-code/meta.json b/docs/content/docs/lume/examples/claude-code/meta.jsondeleted file mode 100644index 0dadc14830..0000000000--- a/docs/content/docs/lume/examples/claude-code/meta.json+++ /dev/null@@ -1,6 +0,0 @@-{- "title": "Claude Code",- "description": "Run Claude Code CLI in isolated macOS VM sandboxes",- "icon": "Terminal",- "pages": ["sandbox", "homebrew-testing"]-}diff --git a/docs/content/docs/lume/examples/claude-code/sandbox.mdx b/docs/content/docs/lume/examples/claude-code/sandbox.mdxdeleted file mode 100644index 2038a826d1..0000000000--- a/docs/content/docs/lume/examples/claude-code/sandbox.mdx+++ /dev/null@@ -1,184 +0,0 @@-----title: Sandbox Setup-description: Run Claude Code CLI safely in an isolated macOS VM------import { Callout } from 'fumadocs-ui/components/callout';--Run [Claude Code](https://code.claude.com/docs) on your host while it executes commands inside a sandboxed macOS VM via SSH. No MCP required—just the `lume` CLI and SSH.--## Why sandbox Claude Code?--Claude Code is powerful—it can read files, run commands, install packages, and modify your system. Sometimes you want that power contained:--- **Experiment safely** — Let Claude install dependencies, run builds, test scripts without touching your real system-- **Isolate risky operations** — Test destructive commands, untrusted code, or experimental changes-- **Reset instantly** — Clone a fresh VM to undo everything Claude did--## Architecture--```-┌─────────────────────────────────────────────────────┐-│ Your Mac (Host) │-│ │-│ ┌───────────────────┐ ┌───────────────────┐ │-│ │ Claude Code │ │ ~/project │ │-│ │ (runs here) │ │ (shared folder) │ │-│ └─────────┬─────────┘ └─────────┬─────────┘ │-│ │ SSH │ mounted │-│ ┌─────────▼─────────────────────────▼─────────┐ │-│ │ macOS VM (Sandbox) │ │-│ │ │ │-│ │ Commands execute here, isolated from host │ │-│ │ Full macOS environment with network access │ │-│ │ Can only access /Volumes/My Shared Files │ │-│ └─────────────────────────────────────────────┘ │-└─────────────────────────────────────────────────────┘-```--## Step 1: Create the sandbox VM--```bash-# Create a VM with automated setup (15-20 min, no interaction needed)-lume create sandbox --os macos --ipsw latest --unattended tahoe-```--The `tahoe` preset creates user `lume` with password `lume` and enables SSH.--## Step 2: Start the sandbox with your project--```bash-# Run headlessly, sharing your project folder-lume run sandbox --no-display --shared-dir ~/my-project-```--Inside the VM, your files appear at `/Volumes/My Shared Files`.--## Step 3: Get the VM's IP address--```bash-lume get sandbox-# Look for the IP address in the output-```--Save this IP—you'll give it to Claude Code.--## Step 4: Install Claude Code on your host--```bash-# macOS/Linux-curl -fsSL https://claude.ai/install.sh | bash--# Or with Homebrew-brew install --cask claude-code-```--See [Claude Code docs](https://code.claude.com/docs) for other installation methods.--## Step 5: Use Claude Code with the sandbox--Now ask Claude Code to SSH into the sandbox and work there:--```bash-claude-```--Then tell Claude:--> "SSH into [email protected] (password: lume) and work on the project in /Volumes/My Shared Files.-> Install any dependencies you need, run builds, execute tests—all inside that VM."--Claude Code will:--1. SSH into the sandbox VM-2. Navigate to your shared project-3. Run commands in the isolated environment-4. Report results back to you--<Callout type="info">- Claude Code runs on your host but executes commands via SSH inside the VM. Your host system stays- untouched—only the sandbox and shared files are affected.-</Callout>--## Example session--```-You: SSH into [email protected] and set up a Python project in /Volumes/My Shared Files--Claude: I'll connect to the VM and set up the project.--[Runs: ssh [email protected]]-[Runs: cd /Volumes/My\ Shared\ Files]-[Runs: python3 -m venv venv]-[Runs: source venv/bin/activate]-[Runs: pip install -r requirements.txt]--Done! I've created a virtual environment and installed the dependencies.-The project is ready at /Volumes/My Shared Files.-```--All those commands ran inside the sandbox. Your host's Python, packages, and filesystem weren't touched.--## Resetting the sandbox--Made a mess? Reset instantly:--```bash-# Create a golden image first (do this once, after initial setup)-lume clone sandbox sandbox-golden--# Reset to clean state anytime-lume stop sandbox-lume delete sandbox-lume clone sandbox-golden sandbox-lume run sandbox --no-display --shared-dir ~/my-project-```--## Automating the workflow--Script the entire setup:--```bash-#!/bin/bash--PROJECT_DIR="$1"-VM_NAME="sandbox"--# Start the sandbox-lume run $VM_NAME --no-display --shared-dir "$PROJECT_DIR"-sleep 10 # Wait for boot--# Get IP-VM_IP=$(lume get $VM_NAME --format json | jq -r '.ip')--echo "Sandbox ready!"-echo "Run: claude"-echo "Then tell Claude to SSH into lume@$VM_IP"-```--## Security considerations--<Callout type="warn">- The shared folder is read/write. Claude can modify or delete files in that folder. For sensitive- work: - Share a copy of your project, not the original - Use git to track changes and revert if- needed - Clone the VM before risky operations-</Callout>--The VM itself is fully isolated—code running inside cannot:--- Access files outside the shared folder-- Modify your host system-- Read your host's environment variables or credentials--But it can:--- Access the network-- Install software inside the VM-- Consume CPU/memory--## Next steps--- [Homebrew Testing](/lume/examples/claude-code/homebrew-testing) — Test packages in a clean environment-- [Claude Cowork with MCP](/lume/examples/claude-cowork/sandbox) — Use MCP for native VM integration-- [Unattended Setup](/lume/guide/fundamentals/unattended-setup) — Customize the VM configuration-- [Claude Code docs](https://code.claude.com/docs) — Learn more about Claude Code's capabilitiesdiff --git a/docs/content/docs/lume/examples/claude-cowork/meta.json b/docs/content/docs/lume/examples/claude-cowork/meta.jsondeleted file mode 100644index 0ca5bbfc74..0000000000--- a/docs/content/docs/lume/examples/claude-cowork/meta.json+++ /dev/null@@ -1,6 +0,0 @@-{- "title": "Claude Cowork",- "description": "Use Claude Cowork with Lume MCP connector",- "icon": "Bot",- "pages": ["sandbox", "numbers-stock-analysis"]-}diff --git a/docs/content/docs/lume/examples/claude-cowork/numbers-stock-analysis.mdx b/docs/content/docs/lume/examples/claude-cowork/numbers-stock-analysis.mdxdeleted file mode 100644index 8680aa5acf..0000000000--- a/docs/content/docs/lume/examples/claude-cowork/numbers-stock-analysis.mdx+++ /dev/null@@ -1,174 +0,0 @@-----title: Stock Analysis with Numbers-description: Create financial reports using macOS-only Numbers.app via MCP------import { Callout } from 'fumadocs-ui/components/callout';--This example demonstrates a task that **requires macOS**: creating a stock analysis report using Apple's Numbers spreadsheet app. Numbers is only available on macOS—this workflow wouldn't work in Docker or a Linux VM.--Using the Lume MCP connector, Claude Cowork can execute commands in the VM without manual SSH setup.--## Why macOS?--Numbers.app is Apple's spreadsheet application, comparable to Excel but with tighter macOS integration. It's perfect for:--- Financial reports with charts-- Data visualization-- PDF export for sharing--Since Numbers only runs on macOS, this task showcases the unique value of macOS VM sandboxes.--## Prerequisites--- [Lume MCP connector configured](/lume/examples/claude-cowork/sandbox)-- Sandbox VM created with `--unattended tahoe`--## Example prompt--Ask Cowork:--> "Start the sandbox VM with ~/reports shared. Fetch Apple (AAPL) stock prices for the last 30 days, create a spreadsheet in Numbers with the data and a line chart, then export it as a PDF to the shared folder."--## What Cowork does--1. **Starts the VM** via `lume_run_vm` with shared directory-2. **Fetches stock data** via `lume_exec` using curl or Python-3. **Creates a Numbers spreadsheet** via `lume_exec` with AppleScript-4. **Exports to PDF** at `/Volumes/My Shared Files/aapl-analysis.pdf`--## Example session--```-You: Start the sandbox with ~/reports shared, then create a stock analysis- report for AAPL in Numbers and export it as PDF.--Cowork: I'll start the sandbox and create the stock analysis report.--[Calls lume_run_vm with name="sandbox", shared_dir="~/reports", no_display=true]-VM 'sandbox' started successfully.--[Calls lume_exec with name="sandbox", command="python3 << 'EOF'-import urllib.request-import json-import csv--# Fetch stock data from a free API-url = \"https://query1.finance.yahoo.com/v8/finance/chart/AAPL?interval=1d&range=1mo\"-req = urllib.request.Request(url, headers={'User-Agent': 'Mozilla/5.0'})-data = json.loads(urllib.request.urlopen(req).read())--# Extract timestamps and prices-timestamps = data['chart']['result'][0]['timestamp']-closes = data['chart']['result'][0]['indicators']['quote'][0]['close']--# Write to CSV-with open('/tmp/aapl_data.csv', 'w', newline='') as f:- writer = csv.writer(f)- writer.writerow(['Date', 'Close'])- for ts, close in zip(timestamps, closes):- from datetime import datetime- date = datetime.fromtimestamp(ts).strftime('%Y-%m-%d')- writer.writerow([date, f'{close:.2f}'])--print('Data saved to /tmp/aapl_data.csv')-EOF"]-Data saved to /tmp/aapl_data.csv--Now I'll create a Numbers spreadsheet using AppleScript:--[Calls lume_exec with name="sandbox", command="osascript << 'EOF'-tell application \"Numbers\"- activate- set newDoc to make new document- tell table 1 of sheet 1 of newDoc- set csvData to read POSIX file \"/tmp/aapl_data.csv\"- set csvLines to paragraphs of csvData- repeat with i from 1 to count of csvLines- set csvLine to item i of csvLines- set AppleScript's text item delimiters to \",\"- set parts to text items of csvLine- set AppleScript's text item delimiters to \"\"- set value of cell 1 of row i to item 1 of parts- set value of cell 2 of row i to item 2 of parts- end repeat- end tell- tell sheet 1 of newDoc- set newChart to make new chart with properties {chart type:line_2d}- end tell- export newDoc to POSIX file \"/Volumes/My Shared Files/aapl-analysis.pdf\" as PDF- close newDoc saving no-end tell-EOF"]-Success--Done! I've created the stock analysis report. You can find it at:-~/reports/aapl-analysis.pdf (via the shared folder)--The report contains:-- 30 days of AAPL closing prices-- A line chart showing price trends-- Data table with dates and values-```--## Key techniques--### Automating Numbers with AppleScript--Numbers can be controlled via AppleScript through `lume_exec`:--```applescript-tell application "Numbers"- -- Create documents- set newDoc to make new document-- -- Manipulate cells- tell table 1 of sheet 1 of newDoc- set value of cell 1 of row 1 to "Hello"- end tell-- -- Add charts- make new chart with properties {chart type:line_2d}-- -- Export- export newDoc to POSIX file "/path/to/output.pdf" as PDF-end tell-```--### Fetching financial data--Many free APIs provide stock data:--```bash-# Yahoo Finance (unofficial)-curl "https://query1.finance.yahoo.com/v8/finance/chart/AAPL?interval=1d&range=1mo"--# Alpha Vantage (requires free API key)-curl "https://www.alphavantage.co/query?function=TIME_SERIES_DAILY&symbol=AAPL&apikey=YOUR_KEY"-```--## Variations--Try these related tasks:--- "Create a portfolio tracker comparing AAPL, GOOGL, and MSFT"-- "Generate a monthly expense report from this CSV in Numbers"-- "Create a chart in Numbers showing my app's download stats"--## Why this matters--This example shows why macOS sandboxes are valuable:--| Approach | Can use Numbers.app? | Native macOS automation? |-| ------------ | -------------------- | ------------------------ |-| Docker | No | No |-| Linux VM | No | No |-| **macOS VM** | **Yes** | **Yes** |--Tasks requiring Numbers, Keynote, Pages, or other macOS-only apps need a macOS environment. Lume provides that in an isolated sandbox.--## Next steps--- [Sandbox with MCP](/lume/examples/claude-cowork/sandbox) — MCP connector setup-- [Claude Code Sandbox](/lume/examples/claude-code/sandbox) — SSH-based alternative-- [Homebrew Testing](/lume/examples/claude-code/homebrew-testing) — Test packages in a clean environmentdiff --git a/docs/content/docs/lume/examples/claude-cowork/sandbox.mdx b/docs/content/docs/lume/examples/claude-cowork/sandbox.mdxdeleted file mode 100644index 6096aea9a6..0000000000--- a/docs/content/docs/lume/examples/claude-cowork/sandbox.mdx+++ /dev/null@@ -1,186 +0,0 @@-----title: Sandbox with MCP-description: Use Claude Cowork with Lume MCP connector for VM management------import { Callout } from 'fumadocs-ui/components/callout';--[Claude Cowork](https://support.claude.com/en/articles/13345190-getting-started-with-cowork) can manage macOS VMs natively using Lume's MCP connector. Instead of manually starting VMs and providing SSH credentials, Cowork uses MCP tools to control the full VM lifecycle.--## How it works--```-┌─────────────────────────────────────────────────────────────────┐-│ Your Mac (Host) │-│ │-│ ┌──────────────────┐ ┌───────────────────────────┐ │-│ │ Claude Cowork │◄── MCP ──►│ Lume MCP Server │ │-│ │ (Desktop app) │ (stdio) │ (lume serve --mcp) │ │-│ └──────────────────┘ └─────────────┬─────────────┘ │-│ │ │-│ │ VM Control │-│ ▼ │-│ ┌─────────────────────────────────────────────────────────────┐│-│ │ macOS VM Sandbox ││-│ │ Cowork can: list, run, stop, clone, delete, exec commands ││-│ └─────────────────────────────────────────────────────────────┘│-└─────────────────────────────────────────────────────────────────┘-```--## Prerequisites--1. **Lume installed** with MCP support-2. **A sandbox VM** created with `--unattended tahoe`-3. **sshpass** for command execution: `brew install hudochenkov/sshpass/sshpass`--## Setup--### 1. Configure Claude Desktop--Add the Lume MCP server to your Claude Desktop config:--**`~/Library/Application Support/Claude/claude_desktop_config.json`**--```json-{- "mcpServers": {- "lume": {- "command": "lume",- "args": ["serve", "--mcp"]- }- }-}-```--<Callout type="info">- If `lume` isn't in your PATH, use the full path: `/Users/yourname/.local/bin/lume`-</Callout>--### 2. Restart Claude Desktop--After saving the config, restart Claude Desktop. The Lume tools will now be available to Cowork.--### 3. Create a sandbox VM--If you haven't already:--```bash-lume create sandbox --os macos --ipsw latest --unattended tahoe-```--## Available MCP tools--| Tool | Description |-| ---------------- | ----------------------------------------- |-| `lume_list_vms` | List all VMs with status and IP addresses |-| `lume_get_vm` | Get detailed VM info |-| `lume_run_vm` | Start a VM with optional shared directory |-| `lume_stop_vm` | Stop a running VM |-| `lume_clone_vm` | Clone a VM (for golden images) |-| `lume_delete_vm` | Delete a VM |-| `lume_exec` | Execute commands inside the VM |--## Example workflow--Ask Cowork:--> "Start my sandbox VM with ~/project shared, then run the tests in the shared folder."--Cowork will:--```-[Calls lume_run_vm with name="sandbox", shared_dir="~/project"]-VM 'sandbox' started successfully.--[Calls lume_exec with name="sandbox", command="cd /Volumes/My\\ Shared\\ Files && npm test"]--> [email protected] test-> jest--PASS src/app.test.js- 42 passing (1.2s)--All tests passed!-```--## Example session--```-You: List my VMs and start the sandbox with my project folder shared.--Cowork: I'll check your VMs and start the sandbox.--[Calls lume_list_vms]-Found 2 VMs:-- sandbox (stopped)-- dev-vm (running, IP: 192.168.64.3)--[Calls lume_run_vm with name="sandbox", shared_dir="~/my-project", no_display=true]-VM 'sandbox' started successfully.--[Calls lume_get_vm with name="sandbox"]-sandbox:- State: running- IP: 192.168.64.5- SSH: available--The sandbox is now running. Your project files are available at /Volumes/My Shared Files-inside the VM. What would you like me to do next?-```--## Resetting the sandbox--Ask Cowork:--> "Reset the sandbox to a clean state using the golden image."--```-[Calls lume_stop_vm with name="sandbox"]-VM stopped.--[Calls lume_delete_vm with name="sandbox"]-VM deleted.--[Calls lume_clone_vm with name="sandbox-golden", new_name="sandbox"]-VM cloned successfully.--[Calls lume_run_vm with name="sandbox", no_display=true]-VM 'sandbox' started from clean state.-```--## Advantages over SSH--| Feature | SSH (Claude Code) | MCP (Claude Cowork) |-| -------------- | ------------------------ | ------------------- |-| VM lifecycle | Manual (`lume run/stop`) | Automated via tools |-| IP lookup | Manual (`lume get`) | Automatic |-| Credentials | User provides | Configured once |-| Reset workflow | Script required | Natural language |--With MCP, Cowork can manage the entire VM lifecycle through conversation.--## Troubleshooting--### Cowork doesn't see Lume tools--1. Check `claude_desktop_config.json` syntax-2. Verify `lume serve --mcp` works: `npx @modelcontextprotocol/inspector lume serve --mcp`-3. Restart Claude Desktop completely--### lume_exec fails--Ensure sshpass is installed:--```bash-brew install hudochenkov/sshpass/sshpass-```--### VM has no IP address--The VM might not be fully booted. Wait a few seconds and try again, or check with `lume get sandbox`.--## Next steps--- [Stock Analysis with Numbers](/lume/examples/claude-cowork/numbers-stock-analysis) — Use macOS-only apps via MCP-- [MCP Server Reference](/lume/guide/advanced/mcp-server) — Full MCP documentation-- [Claude Code Sandbox](/lume/examples/claude-code/sandbox) — SSH-based alternativediff --git a/docs/content/docs/lume/examples/index.mdx b/docs/content/docs/lume/examples/index.mdxdeleted file mode 100644index d16ebdfefd..0000000000--- a/docs/content/docs/lume/examples/index.mdx+++ /dev/null@@ -1,48 +0,0 @@-----title: Examples-description: Step-by-step tutorials and use cases for Lume------import { Card, Cards } from 'fumadocs-ui/components/card';-import { Bot, Terminal } from 'lucide-react';--# Examples--Explore real-world examples and tutorials for using Lume to create and manage macOS virtual machines.--<Cards>- <Card- icon={<Terminal className="w-6 h-6" />}- href="/lume/examples/claude-code"- title="Claude Code"- description="Run Claude Code CLI in isolated macOS VMs using SSH"- />- <Card- icon={<Bot className="w-6 h-6" />}- href="/lume/examples/claude-cowork"- title="Claude Cowork"- description="Use Claude Cowork with Lume MCP connector"- />-</Cards>--## Available Examples--### Claude Code--Run the Claude Code CLI in isolated macOS sandboxes using SSH:--- **[Sandbox Setup](/lume/examples/claude-code/sandbox)** — Create and configure a sandboxed macOS VM-- **[Homebrew Testing](/lume/examples/claude-code/homebrew-testing)** — Test Homebrew formulas in a clean environment--### Claude Cowork--Use Claude Cowork with the Lume MCP connector for native VM management:--- **[Sandbox with MCP](/lume/examples/claude-cowork/sandbox)** — Configure the MCP connector for VM lifecycle control-- **[Stock Analysis with Numbers](/lume/examples/claude-cowork/numbers-stock-analysis)** — Use macOS-only apps like Numbers.app--## Coming Soon--- **CI/CD with GitHub Actions** - Run macOS builds and tests in ephemeral VMs-- **Multi-VM Development** - Manage multiple VMs for different project environments-- **Automated Testing** - Set up VMs for automated UI testingdiff --git a/docs/content/docs/lume/examples/meta.json b/docs/content/docs/lume/examples/meta.jsondeleted file mode 100644index 72804e2d01..0000000000--- a/docs/content/docs/lume/examples/meta.json+++ /dev/null@@ -1,6 +0,0 @@-{- "title": "Examples",- "description": "Step-by-step tutorials and use cases",- "icon": "Blocks",- "pages": ["claude-code", "claude-cowork", "openclaw"]-}diff --git a/docs/content/docs/lume/examples/openclaw/index.mdx b/docs/content/docs/lume/examples/openclaw/index.mdxdeleted file mode 100644index 5df4b55776..0000000000--- a/docs/content/docs/lume/examples/openclaw/index.mdx+++ /dev/null@@ -1,231 +0,0 @@-----title: OpenClaw Setup-description: Run OpenClaw messaging gateway in an isolated macOS VM with Lume------import { Callout } from 'fumadocs-ui/components/callout';--Run [OpenClaw](https://openclaw.ai) in a headless macOS VM to create a unified messaging gateway that bridges WhatsApp, Telegram, iMessage, and more with AI agents.--## Why run OpenClaw in a VM?--Running OpenClaw in a macOS VM provides several advantages:--- **iMessage integration** — Only macOS can run iMessage natively; a VM gives you this capability on any Apple Silicon Mac-- **Isolation** — Keep your messaging gateway separate from your main system-- **Always-on operation** — Run the VM headlessly 24/7 without affecting your desktop-- **Easy reset** — Clone and restore VMs to quickly recover from issues--## Quick path (experienced users)--```bash-# Install Lume-/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/lume/scripts/install.sh)"--# Create and setup VM-lume create openclaw --os macos --ipsw latest--# Complete Setup Assistant in VNC window, enable Remote Login (SSH)-# Then run headlessly-lume run openclaw --no-display--# SSH in, install OpenClaw, configure channels-ssh youruser@<VM_IP>-npm install -g openclaw@latest-openclaw onboard --install-daemon-```--## Requirements--- Apple Silicon Mac (M1/M2/M3/M4)-- macOS Sequoia or later on the host-- ~60 GB free disk space per VM--## Step 1: Install Lume--```bash-/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/lume/scripts/install.sh)"-```--If `~/.local/bin` isn't in your PATH:--```bash-echo 'export PATH="$PATH:$HOME/.local/bin"' >> ~/.zshrc && source ~/.zshrc-```--Verify the installation:--```bash-lume --version-```--See [Lume Installation](/lume/guide/getting-started/installation) for more details.--## Step 2: Create the macOS VM--```bash-lume create openclaw --os macos --ipsw latest-```--This downloads macOS and creates the VM. A VNC window opens automatically.--<Callout type="info">-The macOS download can take a while depending on your connection speed.-</Callout>--## Step 3: Complete Setup Assistant--In the VNC window:--1. Select language and region-2. Skip Apple ID (or sign in if you want iMessage later)-3. Create a user account (remember the username and password)-4. Skip all optional features--After setup completes, enable SSH:--1. Open **System Settings** → **General** → **Sharing**-2. Enable **Remote Login**--## Step 4: Get the VM's IP address--```bash-lume get openclaw-```--Look for the IP address in the output (usually `192.168.64.x`).--## Step 5: SSH into the VM--```bash-ssh [email protected]-```--Replace `youruser` with the account you created, and the IP with your VM's IP.--## Step 6: Install OpenClaw--Inside the VM via SSH:--```bash-npm install -g openclaw@latest-openclaw onboard --install-daemon-```--Follow the onboarding prompts to set up your model provider (Anthropic, OpenAI, etc.).--## Step 7: Configure channels--Edit the config file:--```bash-nano ~/.openclaw/openclaw.json-```--Add your channels:--```json-{- "channels": {- "whatsapp": {- "dmPolicy": "allowlist",- "allowFrom": ["+15551234567"]- },- "telegram": {- "botToken": "YOUR_BOT_TOKEN"- }- }-}-```--Then login to WhatsApp (scan QR):--```bash-openclaw channels login-```--## Step 8: Run the VM headlessly--Stop the VM and restart without display:--```bash-lume stop openclaw-lume run openclaw --no-display-```--The VM runs in the background. OpenClaw's daemon keeps the gateway running.--To check status:--```bash-ssh [email protected] "openclaw status"-```--## iMessage integration with BlueBubbles--This is the killer feature of running on macOS. Use [BlueBubbles](https://bluebubbles.app) to add iMessage to OpenClaw.--Inside the VM:--1. Download BlueBubbles from [bluebubbles.app](https://bluebubbles.app)-2. Sign in with your Apple ID-3. Enable the Web API and set a password-4. Point BlueBubbles webhooks at your gateway (example: `https://your-gateway-host:3000/bluebubbles-webhook?password=<password>`)--Add to your OpenClaw config:--```json-{- "channels": {- "bluebubbles": {- "serverUrl": "http://localhost:1234",- "password": "your-api-password",- "webhookPath": "/bluebubbles-webhook"- }- }-}-```--Restart the gateway. Now your agent can send and receive iMessages.--## Save a golden image--Before customizing further, snapshot your clean state:--```bash-lume stop openclaw-lume clone openclaw openclaw-golden-```--Reset anytime:--```bash-lume stop openclaw && lume delete openclaw-lume clone openclaw-golden openclaw-lume run openclaw --no-display-```--## Running 24/7--Keep the VM running by:--- Keeping your Mac plugged in-- Disabling sleep in **System Settings** → **Energy Saver**-- Using `caffeinate` if needed--For true always-on operation, consider a dedicated Mac mini or a small VPS.--## Troubleshooting--| Problem | Solution |-|---------|----------|-| Can't SSH into VM | Check "Remote Login" is enabled in VM's System Settings |-| VM IP not showing | Wait for VM to fully boot, run `lume get openclaw` again |-| Lume command not found | Add `~/.local/bin` to your PATH |-| WhatsApp QR not scanning | Ensure you're logged into the VM (not host) when running `openclaw channels login` |--## Next steps--- [VM Management](/lume/guide/fundamentals/vm-management) — Learn more about managing VMs-- [Unattended Setup](/lume/guide/fundamentals/unattended-setup) — Automate VM creation without manual steps-- [HTTP Server](/lume/guide/advanced/http-server) — Control VMs programmatically via APIdiff --git a/docs/content/docs/lume/examples/openclaw/meta.json b/docs/content/docs/lume/examples/openclaw/meta.jsondeleted file mode 100644index 3676077975..0000000000--- a/docs/content/docs/lume/examples/openclaw/meta.json+++ /dev/null@@ -1,5 +0,0 @@-{- "title": "OpenClaw",- "description": "Run OpenClaw messaging gateway in a macOS VM",- "pages": ["index"]-}diff --git a/docs/content/docs/lume/guide/advanced/http-server.mdx b/docs/content/docs/lume/guide/advanced/http-server.mdxdeleted file mode 100644index 009456d64f..0000000000--- a/docs/content/docs/lume/guide/advanced/http-server.mdx+++ /dev/null@@ -1,150 +0,0 @@-----title: HTTP Server-description: Using the Lume REST API server------import { Callout } from 'fumadocs-ui/components/callout';--Lume's HTTP API lets you manage VMs programmatically—useful for automation scripts, CI/CD pipelines, and the Computer Use Agent SDK.--```bash-# Start the server-lume serve--# Or on a custom port-lume serve --port 8080-```--<Callout type="info">- If you installed Lume with default settings, the server runs as a background service on port 7777.- No need to start it manually.-</Callout>--## Base URL--```-http://localhost:7777-```--## Quick examples--### List VMs--```bash-curl http://localhost:7777/lume/vms-```--### Run a VM--```bash-curl -X POST http://localhost:7777/lume/vms/my-vm/run \- -H "Content-Type: application/json" \- -d '{"noDisplay": true}'-```--### Stop a VM--```bash-curl -X POST http://localhost:7777/lume/vms/my-vm/stop-```--### Create a VM--VM creation is asynchronous—returns immediately with 202 Accepted while provisioning runs in the background:--```bash-curl -X POST http://localhost:7777/lume/vms \- -H "Content-Type: application/json" \- -d '{- "name": "my-vm",- "os": "macOS",- "cpu": 4,- "memory": "8GB",- "diskSize": "50GB",- "ipsw": "latest",- "unattended": "tahoe"- }'-```--Response:--```json-{- "message": "VM creation started",- "name": "my-vm",- "status": "provisioning"-}-```--Poll the list endpoint to check provisioning status:--```bash-# Check status (will show "provisioning (ipsw_install)" or "provisioning (unattended_setup)")-curl http://localhost:7777/lume/vms | jq '.[] | select(.name == "my-vm") | .status'-```--### Get VM details--```bash-curl http://localhost:7777/lume/vms/my-vm-```--## Common workflows--### Automation script--```bash-#!/bin/bash-# Start a VM, wait for it, run tests, stop it--curl -X POST localhost:7777/lume/vms/test-vm/run -d '{"noDisplay": true}'-sleep 30 # Wait for boot--# Run your tests here-ssh lume@$(curl -s localhost:7777/lume/vms/test-vm | jq -r '.ip') 'run-tests.sh'--curl -X POST localhost:7777/lume/vms/test-vm/stop-```--### Python integration--The Computer Use Agent SDK connects to Lume's HTTP API:--```python-from cua import Computer--async with Computer() as computer:- await computer.run("my-vm")- # Perform automation...- await computer.stop()-```--## Managing the background service--```bash-# Check if running-launchctl list | grep com.trycua.lume_daemon--# Stop-launchctl unload ~/Library/LaunchAgents/com.trycua.lume_daemon.plist--# Start-launchctl load ~/Library/LaunchAgents/com.trycua.lume_daemon.plist--# View logs-tail -f /tmp/lume_daemon.log-```--## Configuration--The server uses settings from `~/.config/lume/config.yaml`:--- VM storage locations-- Cache settings-- Default resource allocations--See the [FAQ](/lume/guide/getting-started/faq) for more on configuration.--## Full API reference--For all endpoints, request/response formats, and language-specific examples, see the [HTTP API Reference](/lume/reference/http-api).diff --git a/docs/content/docs/lume/guide/advanced/lumier/building-lumier.mdx b/docs/content/docs/lume/guide/advanced/lumier/building-lumier.mdxdeleted file mode 100644index 81a803845c..0000000000--- a/docs/content/docs/lume/guide/advanced/lumier/building-lumier.mdx+++ /dev/null@@ -1,79 +0,0 @@-----title: Building Lumier-description: Build Lumier from source or customize the container------Build Lumier from source to customize packages, hooks, or configuration.--## Build from source--```bash-# Clone the repo (if you haven't)-git clone https://github.com/trycua/cua.git-cd cua/libs/lumier--# Build-docker build -t lumier-custom:latest .--# Run your build-docker run -it --rm \- -p 8006:8006 \- -e VM_NAME=my-vm \- -e VERSION=ghcr.io/trycua/macos-sequoia-cua:latest \- -e CPU_CORES=4 \- -e RAM_SIZE=8192 \- lumier-custom:latest-```--## Customization points--| Location | What to customize |-| -------------------------- | ------------------------------------ |-| `Dockerfile` | Base image, installed packages |-| `/run/hooks/` | Scripts that run during VM lifecycle |-| `/run/config/constants.sh` | Default settings |--### Change the base image--The default is Debian Bullseye Slim. To use a different base, edit the first line of `Dockerfile`:--```dockerfile-FROM debian:bookworm-slim-```--### Add packages--Add to the `apt-get install` list in the Dockerfile:--```dockerfile-RUN apt-get update && apt-get install -y \- your-package-here \- ...-```--### Add lifecycle hooks--Scripts in `/run/hooks/` run at specific points. Create your own to customize behavior.--## Publish your build--Push to Docker Hub or another registry:--```bash-# Tag for your registry-docker build -t yourusername/lumier:custom .--# Login and push-docker login-docker push yourusername/lumier:custom-```--Then use your image:--```bash-docker run -it --rm \- -p 8006:8006 \- -e VM_NAME=my-vm \- -e VERSION=ghcr.io/trycua/macos-sequoia-cua:latest \- yourusername/lumier:custom-```diff --git a/docs/content/docs/lume/guide/advanced/lumier/docker-compose.mdx b/docs/content/docs/lume/guide/advanced/lumier/docker-compose.mdxdeleted file mode 100644index bf7ffee895..0000000000--- a/docs/content/docs/lume/guide/advanced/lumier/docker-compose.mdx+++ /dev/null@@ -1,91 +0,0 @@-----title: Docker Compose-description: Running Lumier with Docker Compose------For easier management, use Docker Compose instead of raw `docker run` commands.--## Setup--Create `docker-compose.yml`:--```yaml-version: '3'--services:- lumier:- image: trycua/lumier:latest- container_name: lumier-vm- restart: unless-stopped- ports:- - '8006:8006'- volumes:- - ./storage:/storage- - ./shared:/shared- environment:- - VM_NAME=lumier-vm- - VERSION=ghcr.io/trycua/macos-sequoia-cua:latest- - CPU_CORES=4- - RAM_SIZE=8192- - HOST_STORAGE_PATH=${PWD}/storage- - HOST_SHARED_PATH=${PWD}/shared- stop_signal: SIGINT- stop_grace_period: 2m-```--## Run it--```bash-# Create directories-mkdir -p storage shared--# Start in background-docker-compose up -d--# View logs-docker-compose logs -f--# Stop-docker-compose down-```--Access at `http://localhost:8006`.--## Configuration options--| Field | Description |-| ------------------------- | ------------------------------------ |-| `restart: unless-stopped` | Auto-restart unless manually stopped |-| `stop_grace_period: 2m` | Wait 2 minutes for graceful shutdown |-| `stop_signal: SIGINT` | Send interrupt signal to stop |--## Multiple VMs--Run multiple VMs by adding services:--```yaml-version: '3'--services:- dev-vm:- image: trycua/lumier:latest- ports:- - '8006:8006'- environment:- - VM_NAME=dev-vm- - VERSION=ghcr.io/trycua/macos-sequoia-cua:latest- - CPU_CORES=4- - RAM_SIZE=8192-- test-vm:- image: trycua/lumier:latest- ports:- - '8007:8006'- environment:- - VM_NAME=test-vm- - VERSION=ghcr.io/trycua/macos-sequoia-cua:latest- - CPU_CORES=2- - RAM_SIZE=4096-```--Access at `localhost:8006` and `localhost:8007`.diff --git a/docs/content/docs/lume/guide/advanced/lumier/docker.mdx b/docs/content/docs/lume/guide/advanced/lumier/docker.mdxdeleted file mode 100644index 3ea783482d..0000000000--- a/docs/content/docs/lume/guide/advanced/lumier/docker.mdx+++ /dev/null@@ -1,119 +0,0 @@-----title: Docker-description: Running Lumier with Docker------import { Callout } from 'fumadocs-ui/components/callout';--## Quick start (ephemeral)--Run a macOS VM that resets when you stop it:--```bash-docker run -it --rm \- -p 8006:8006 \- -e VM_NAME=my-vm \- -e VERSION=ghcr.io/trycua/macos-sequoia-cua:latest \- -e CPU_CORES=4 \- -e RAM_SIZE=8192 \- trycua/lumier:latest-```--Open `http://localhost:8006` in your browser.--<Callout type="info">- In ephemeral mode, changes are lost when you stop the container. See below for persistent storage.-</Callout>--## Persistent storage--Keep your VM state between sessions:--```bash-mkdir -p storage--docker run -it --rm \- -p 8006:8006 \- -v $(pwd)/storage:/storage \- -e VM_NAME=my-vm \- -e VERSION=ghcr.io/trycua/macos-sequoia-cua:latest \- -e CPU_CORES=4 \- -e RAM_SIZE=8192 \- -e HOST_STORAGE_PATH=$(pwd)/storage \- trycua/lumier:latest-```--The `-v` mount and `HOST_STORAGE_PATH` work together to save VM data on your Mac.--## Share files with the VM--Add a shared folder accessible at `/Volumes/My Shared Files` inside the VM:--```bash-mkdir -p storage shared--docker run -it --rm \- -p 8006:8006 \- -v $(pwd)/storage:/storage \- -v $(pwd)/shared:/shared \- -e VM_NAME=my-vm \- -e VERSION=ghcr.io/trycua/macos-sequoia-cua:latest \- -e CPU_CORES=4 \- -e RAM_SIZE=8192 \- -e HOST_STORAGE_PATH=$(pwd)/storage \- -e HOST_SHARED_PATH=$(pwd)/shared \- trycua/lumier:latest-```--Files you put in `./shared` appear in the VM, and vice versa.--## Run scripts on VM startup--Create `shared/lifecycle/on-logon.sh` to run commands when the VM boots:--```bash-mkdir -p shared/lifecycle--cat > shared/lifecycle/on-logon.sh << 'EOF'-#!/usr/bin/env bash--# Example: create a file on the desktop-echo "Hello from Lumier!" > /Users/lume/Desktop/hello.txt--# Add your setup commands here:-# - Install packages-# - Configure environment-# - Start services-EOF--chmod +x shared/lifecycle/on-logon.sh-```--The script runs as the `lume` user and can access:--- `/Users/lume` — home directory-- `/Volumes/My Shared Files` — shared folder--## Environment variables--| Variable | Description | Example |-| ------------------- | ----------------------- | ----------------------------------------- |-| `VM_NAME` | Name for the VM | `my-vm` |-| `VERSION` | macOS image to use | `ghcr.io/trycua/macos-sequoia-cua:latest` |-| `CPU_CORES` | CPU cores | `4` |-| `RAM_SIZE` | Memory in MB | `8192` |-| `HOST_STORAGE_PATH` | Persistent storage path | `$(pwd)/storage` |-| `HOST_SHARED_PATH` | Shared folder path | `$(pwd)/shared` |--## Change the port--If port 8006 is in use, map to a different port:--```bash-docker run -it --rm \- -p 9000:8006 \- # ... rest of options- trycua/lumier:latest-```--Then access at `http://localhost:9000`.diff --git a/docs/content/docs/lume/guide/advanced/lumier/index.mdx b/docs/content/docs/lume/guide/advanced/lumier/index.mdxdeleted file mode 100644index ae754c9d01..0000000000--- a/docs/content/docs/lume/guide/advanced/lumier/index.mdx+++ /dev/null@@ -1,69 +0,0 @@-----title: Lumier-description: Docker interface for running macOS virtual machines with Lume-github:- - https://github.com/trycua/cua/tree/main/libs/lumier------import { Callout } from 'fumadocs-ui/components/callout';--Lumier wraps Lume in a Docker container, giving you browser-based access to macOS VMs with zero configuration.--```bash-docker run -it --rm \- -p 8006:8006 \- -e VM_NAME=my-vm \- -e VERSION=ghcr.io/trycua/macos-sequoia-cua:latest \- trycua/lumier:latest-```--Open `http://localhost:8006` in your browser—you'll see a macOS desktop.--## When to use Lumier vs Lume directly--| Use Lumier when... | Use Lume directly when... |-| -------------------------------------- | -------------------------------------- |-| You want browser-based VNC access | You prefer native VNC clients |-| You're distributing VM environments | You're running locally for development |-| You want Docker Compose orchestration | You need maximum performance |-| You're building Docker-based workflows | You want full CLI control |--## How it works--<Callout type="info">- Docker is the delivery mechanism, not an isolation layer. Lumier connects to the Lume service on- your host Mac to create real VMs using Apple's Virtualization Framework.-</Callout>--```-┌─────────────────────────────────────┐-│ Docker Container │-│ ┌─────────────────────────────┐ │-│ │ Lumier + noVNC server │ │-│ └──────────────┬──────────────┘ │-└─────────────────┼───────────────────┘- │ connects to-┌─────────────────▼───────────────────┐-│ Lume service (port 7777) │-│ on host Mac │-└─────────────────┬───────────────────┘- │ creates-┌─────────────────▼───────────────────┐-│ macOS VM (Virtualization.framework) │-└─────────────────────────────────────┘-```--The Docker container provides:--- noVNC server for browser access-- Environment variable configuration-- Lifecycle hooks for automation--The actual VM runs natively on your Mac with hardware acceleration.--## Get started--1. [Install prerequisites](/lume/guide/advanced/lumier/installation) — Docker and Lume-2. [Run with Docker](/lume/guide/advanced/lumier/docker) — Basic usage and persistent storage-3. [Use Docker Compose](/lume/guide/advanced/lumier/docker-compose) — Multi-container setups-4. [Build from source](/lume/guide/advanced/lumier/building-lumier) — Customize the containerdiff --git a/docs/content/docs/lume/guide/advanced/lumier/installation.mdx b/docs/content/docs/lume/guide/advanced/lumier/installation.mdxdeleted file mode 100644index b8b8cf2d20..0000000000--- a/docs/content/docs/lume/guide/advanced/lumier/installation.mdx+++ /dev/null@@ -1,73 +0,0 @@-----title: Installation-description: Prerequisites for running Lumier------import { Callout } from 'fumadocs-ui/components/callout';--Lumier needs two things: Docker and Lume.--## 1. Install Docker--Download [Docker Desktop for Apple Silicon](https://desktop.docker.com/mac/main/arm64/Docker.dmg) and install it.--Verify it's running:--```bash-docker --version-# Docker version 24.0.0, build ...-```--## 2. Install Lume--```bash-/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/lume/scripts/install.sh)"-```--Verify the Lume service is running:--```bash-curl http://localhost:7777/lume/vms-# Should return [] or a list of VMs-```--<Callout type="info">- Lume runs as a background service on port 7777. Lumier connects to this service to create and- manage VMs.-</Callout>--## Verify everything works--Pull and run Lumier:--```bash-docker run -it --rm \- -p 8006:8006 \- -e VM_NAME=test-vm \- -e VERSION=ghcr.io/trycua/macos-sequoia-cua:latest \- trycua/lumier:latest-```--Open `http://localhost:8006` in your browser. You should see a macOS desktop loading.--## Troubleshooting--**"Cannot connect to Lume service"** — The Lume background service isn't running. Start it:--```bash-lume serve-```--Or check if the daemon is loaded:--```bash-launchctl list | grep com.trycua.lume_daemon-```--**Port 7777 in use** — Another service is using the port. Either stop that service or reinstall Lume with a different port:--```bash-/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/lume/scripts/install.sh)" -- --port 7778-```--Then update your Lumier commands to use the new port via the `LUME_PORT` environment variable.diff --git a/docs/content/docs/lume/guide/advanced/lumier/meta.json b/docs/content/docs/lume/guide/advanced/lumier/meta.jsondeleted file mode 100644index 4b2725b14b..0000000000--- a/docs/content/docs/lume/guide/advanced/lumier/meta.json+++ /dev/null@@ -1,4 +0,0 @@-{- "title": "Lumier",- "pages": ["installation", "docker", "docker-compose", "building-lumier"]-}diff --git a/docs/content/docs/lume/guide/advanced/mcp-server.mdx b/docs/content/docs/lume/guide/advanced/mcp-server.mdxdeleted file mode 100644index a8de20ef0f..0000000000--- a/docs/content/docs/lume/guide/advanced/mcp-server.mdx+++ /dev/null@@ -1,305 +0,0 @@-----title: MCP Server-description: Integrate Lume with AI agents via Model Context Protocol------import { Callout } from 'fumadocs-ui/components/callout';--Lume supports the [Model Context Protocol (MCP)](https://modelcontextprotocol.io/), allowing AI agents like Claude Desktop and Claude Cowork to manage VMs directly through native tool integration.--```-┌─────────────────────────────────────────────────────────────────┐-│ Your Mac (Host) │-│ │-│ ┌──────────────────┐ ┌───────────────────────────┐ │-│ │ Claude Desktop │◄── MCP ──►│ Lume MCP Server │ │-│ │ or Claude Cowork │ (stdio) │ (lume serve --mcp) │ │-│ └──────────────────┘ └─────────────┬─────────────┘ │-│ │ │-│ │ VM Control │-│ ▼ │-│ ┌─────────────────────────────────────────────────────────────┐│-│ │ macOS VM Sandbox ││-│ │ Claude can: create, list, run, stop, clone, delete, exec ││-│ └─────────────────────────────────────────────────────────────┘│-└─────────────────────────────────────────────────────────────────┘-```--## Quick setup--### 1. Configure Claude Desktop--Run this command to automatically configure Claude Desktop:--```bash-mkdir -p ~/Library/Application\ Support/Claude && cat > ~/Library/Application\ Support/Claude/claude_desktop_config.json << EOF-{- "mcpServers": {- "lume": {- "command": "$(which lume)",- "args": ["serve", "--mcp"]- }- }-}-EOF-```--Or manually edit `~/Library/Application Support/Claude/claude_desktop_config.json`:--```json-{- "mcpServers": {- "lume": {- "command": "/Users/yourname/.local/bin/lume",- "args": ["serve", "--mcp"]- }- }-}-```--<Callout type="info">- If editing manually, replace `/Users/yourname/.local/bin/lume` with the actual path to your lume- binary. You can find it by running `which lume` in your terminal.-</Callout>--### 2. Restart Claude Desktop--Claude will now have access to Lume's VM management tools.--### 3. Try it out--Ask Claude:--- "List my VMs"-- "Start the sandbox VM"-- "Run `ls -la` in the sandbox VM"-- "Stop the sandbox VM"--## Available tools--| Tool | Description |-| ---------------- | --------------------------------------------------------------- |-| `lume_create_vm` | Create a new macOS VM (async, returns immediately) |-| `lume_list_vms` | List all VMs with status, IP addresses, and resource allocation |-| `lume_get_vm` | Get detailed VM info (IP, VNC URL, SSH availability) |-| `lume_run_vm` | Start a VM with optional shared directory |-| `lume_stop_vm` | Stop a running VM gracefully |-| `lume_clone_vm` | Clone a VM (useful for golden images) |-| `lume_delete_vm` | Delete a VM and its files |-| `lume_exec` | Execute commands inside a VM via SSH |--## Resources--The MCP server provides documentation resources that AI agents can read:--| Resource | Description |-| -------------------- | ------------------------------------------------------------------------ |-| `lume://usage-guide` | Comprehensive guide with workflows, best practices, and status reference |-| `lume://credentials` | Default SSH credentials for VMs created with unattended setup |--## Prompts--Pre-defined workflow prompts guide AI agents through common tasks:--| Prompt | Description |-| ---------------- | ---------------------------------------------- |-| `create-sandbox` | Create a new macOS VM with unattended setup |-| `run-in-sandbox` | Execute a command in an existing VM |-| `reset-sandbox` | Reset a sandbox by cloning from a golden image |--## Creating VMs--The `lume_create_vm` tool creates new macOS VMs asynchronously. Since VM creation takes 15-30 minutes (downloading IPSW, installing macOS, optional unattended setup), the tool returns immediately while provisioning continues in the background.--### Parameters--| Parameter | Required | Default | Description |-| ------------ | -------- | -------- | ---------------------------------------------------------- |-| `name` | Yes | - | Name for the new VM |-| `ipsw` | No | `latest` | IPSW path or "latest" to download |-| `unattended` | No | - | Preset name (e.g., "tahoe", "sequoia") for automatic setup |-| `cpu` | No | 4 | Number of CPU cores |-| `memory` | No | `8GB` | Memory size (e.g., "8GB", "16GB") |-| `disk_size` | No | `64GB` | Disk size (e.g., "64GB", "128GB") |-| `storage` | No | - | Storage location name or path |--### Provisioning status--During creation, VMs show a `provisioning` status with the current operation:--```-NAME STATUS IP-sandbox provisioning (ipsw_install) --dev-vm running 192.168.64.3-test stopped --```--The operation progresses through:--1. `ipsw_install` — Downloading and installing macOS-2. `unattended_setup` — Running Setup Assistant automation (if `unattended` specified)--Poll `lume_list_vms` or `lume_get_vm` to monitor progress. Once complete, status changes to `stopped`.--### Example--```-You: Create a new sandbox VM with unattended setup--Claude: I'll create a new macOS VM with automatic configuration.--[Calls lume_create_vm with name="sandbox", unattended="tahoe"]--VM 'sandbox' creation started. Status: provisioning.-I'll monitor the progress...--[Calls lume_list_vms periodically]-Status: provisioning (ipsw_install)...-Status: provisioning (unattended_setup)...-Status: stopped--The sandbox VM is ready! Would you like me to start it?-```--## Command execution--The `lume_exec` tool runs commands inside VMs via SSH. It requires:--1. **SSH enabled** in the VM (automatic with `--unattended tahoe`)-2. **sshpass installed** on your host for password-based auth--```bash-# Install sshpass-brew install hudochenkov/sshpass/sshpass-```--For VMs created with `--unattended tahoe`:--- Username: `lume`-- Password: `lume`--## Example workflow--Here's what an AI agent interaction looks like:--```-You: Create a sandbox VM and run my tests--Claude: I'll set up a sandbox and run your tests.--[Calls lume_list_vms to check existing VMs]--No sandbox VM exists. I'll create one with unattended setup.--[Calls lume_create_vm with name="sandbox", unattended="tahoe"]--VM creation started. This will take 15-30 minutes.-I'll monitor the progress and let you know when it's ready.--[Polls lume_list_vms until status changes from "provisioning" to "stopped"]--The sandbox is ready! Now starting it with your project directory shared.--[Calls lume_run_vm with name="sandbox", shared_dir="~/project"]-[Calls lume_get_vm to get IP address]-[Calls lume_exec with command="cd /Volumes/My\ Shared\ Files && npm test"]--Tests completed! Here are the results:-✓ 42 tests passed-✗ 2 tests failed-...-```--## MCP vs HTTP API--| Feature | MCP Server | HTTP Server |-| --------- | ------------------------- | -------------------- |-| Command | `lume serve --mcp` | `lume serve` |-| Transport | stdio (spawned by client) | HTTP (port 7777) |-| Use case | AI agent integration | Scripts, CI/CD, SDKs |-| Lifecycle | On-demand, client-managed | Background daemon |--Use MCP for AI agent integration. Use HTTP for programmatic access from scripts and applications.--## Testing with MCP Inspector--Debug your MCP setup with the official inspector:--```bash-npx @modelcontextprotocol/inspector lume serve --mcp-```--This opens a web UI where you can:--- See all available tools-- Test tool calls interactively-- Inspect request/response payloads--## Troubleshooting--### Claude doesn't see Lume tools--1. Check the config file path is correct-2. Verify `lume` is accessible (try running `lume --version`)-3. Restart Claude Desktop completely (quit and reopen)--### lume_exec fails with "sshpass not found"--Install sshpass:--```bash-brew install hudochenkov/sshpass/sshpass-```--### lume_exec fails with "no IP address"--The VM might not be running or hasn't acquired an IP yet:--```bash-# Check VM status-lume ls--# Start the VM if needed-lume run my-vm --no-display--# Wait a few seconds for IP assignment-lume get my-vm-```--### SSH connection refused--Ensure SSH is enabled in the VM. For existing VMs:--```bash-# Connect via VNC and enable in System Settings > Sharing > Remote Login-# Or if you can SSH with a different method:-ssh lume@<ip> "sudo launchctl load -w /System/Library/LaunchDaemons/ssh.plist"-```--New VMs created with `--unattended tahoe` have SSH enabled automatically.--## Security considerations--<Callout type="warn">- The MCP server gives AI agents direct control over your VMs. Only use it with trusted AI- applications.-</Callout>--The MCP server:--- Runs locally (stdio transport, no network exposure)-- Has full access to VM lifecycle operations-- Can execute commands inside VMs with SSH access--Consider:--- Using dedicated sandbox VMs for AI agent tasks-- Cloning VMs before letting agents modify them-- Reviewing agent actions in Claude's interface--## Related--- [Claude Cowork Sandbox](/lume/examples/claude-cowork/sandbox) — MCP-based sandbox workflow-- [Claude Code Sandbox](/lume/examples/claude-code/sandbox) — SSH-based sandbox setup-- [HTTP Server](/lume/guide/advanced/http-server) — REST API for scripts and CI/CD-- [Unattended Setup](/lume/guide/fundamentals/unattended-setup) — Automated VM configurationdiff --git a/docs/content/docs/lume/guide/advanced/meta.json b/docs/content/docs/lume/guide/advanced/meta.jsondeleted file mode 100644index 972a1fd56e..0000000000--- a/docs/content/docs/lume/guide/advanced/meta.json+++ /dev/null@@ -1,6 +0,0 @@-{- "title": "Advanced",- "description": "Advanced features and automation",- "icon": "Wrench",- "pages": ["http-server", "mcp-server", "lumier"]-}diff --git a/docs/content/docs/lume/guide/fundamentals/meta.json b/docs/content/docs/lume/guide/fundamentals/meta.jsondeleted file mode 100644index f64dc270b5..0000000000--- a/docs/content/docs/lume/guide/fundamentals/meta.json+++ /dev/null@@ -1,6 +0,0 @@-{- "title": "Fundamentals",- "description": "Core concepts and usage",- "icon": "Layers",- "pages": ["vm-management", "unattended-setup"]-}diff --git a/docs/content/docs/lume/guide/fundamentals/unattended-setup.mdx b/docs/content/docs/lume/guide/fundamentals/unattended-setup.mdxdeleted file mode 100644index 4a5d5aadfd..0000000000--- a/docs/content/docs/lume/guide/fundamentals/unattended-setup.mdx+++ /dev/null@@ -1,290 +0,0 @@-----title: Unattended Setup-description: Automate macOS Setup Assistant with YAML configuration files------import { Callout } from 'fumadocs-ui/components/callout';--The macOS Setup Assistant takes 10+ minutes of clicking through screens. Unattended setup does it for you—using VNC and OCR to navigate the UI automatically.--```bash-lume create my-vm --os macos --ipsw latest --unattended tahoe-```--This creates a VM, installs macOS, and runs through the entire Setup Assistant. When it's done, you have a configured VM with user `lume` (password `lume`) and SSH enabled.--<Callout type="warn">- Unattended configs are macOS version-specific. The `tahoe` preset works with macOS Tahoe (15.x).- Setup Assistant changes between versions, so configs may need updating for new releases.-</Callout>--## Prerequisites--If your config includes an SSH health check (like the built-in presets do), you need `sshpass` installed:--```bash-brew install sshpass-```--Without it, the setup will complete but the health check will fail with `sshpass: No such file or directory`.--## Quick start--### Use the built-in preset--The `tahoe` preset handles macOS Tahoe's Setup Assistant:--```bash-# During VM creation-lume create my-vm --os macos --ipsw latest --unattended tahoe--# Or on an existing VM that's at the Setup Assistant-lume setup my-vm --unattended tahoe-```--### Use a custom config--Point to your YAML file:--```bash-lume create my-vm --os macos --ipsw latest --unattended ./my-config.yml-```--### Debug mode--Save screenshots showing what the automation sees:--```bash-lume create my-vm --os macos --ipsw latest --unattended tahoe --debug-```--Screenshots go to `/tmp/unattended-<uuid>` by default, or specify `--debug-dir /path/to/folder`.--## How it works--1. VM boots and reaches the Setup Assistant-2. Lume waits `boot_wait` seconds for the UI to stabilize-3. Commands execute sequentially: click text, type strings, press keys-4. OCR finds text on screen; VNC sends input-5. Optional health check verifies success (e.g., SSH connection)--## Writing a config file--A config has three parts:--```yaml-boot_wait: 30 # Seconds to wait after boot--boot_commands: # Sequence of automation commands- - "<wait 'Continue'>"- - "<click 'Continue'>"- - "<type 'lume'>"- - '<enter>'--health_check: # Optional verification- type: ssh- user: lume- password: lume-```--## Commands--### Wait for text (OCR)--```yaml-- "<wait 'Continue'>" # Wait up to 120s (default)-- "<wait 'Loading...', timeout=300>" # Custom timeout-```--### Click text--```yaml-- "<click 'Continue'>" # Click first occurrence-- "<click 'Agree', index=-1>" # Click last occurrence-- "<click 'Label', xoffset=50>" # Click 50px right of text-- '<click_at 960,540>' # Click exact coordinates-```--When text appears multiple times (like "Agree" in license text and button), use `index`:--- `index=0` — first (top)-- `index=-1` — last (bottom)--### Type text--```yaml-- "<type 'username'>"-- "<type 'my-password'>"-```--### Press keys--```yaml-- "<enter>" # or <return>-- "<tab>"-- "<space>"-- "<esc>"-- "<backspace>"-- "<delete>"-- "<up>" "<down>" "<left>" "<right>"-- "<f1>" through "<f12>"-```--### Hotkey combinations--```yaml-- '<cmd+space>' # Spotlight-- '<cmd+c>' # Copy-- '<cmd+v>' # Paste-- '<cmd+q>' # Quit-- '<shift+cmd+3>' # Screenshot-- '<ctrl+alt+delete>'-```--Modifiers: `cmd`/`command`/`super`, `shift`, `alt`/`option`, `ctrl`/`control`--### Delays--```yaml-- '<delay 2>' # Wait 2 seconds-- '<delay 0.5>' # Decimals work-```--## Example: Complete Setup Assistant--This walks through macOS Tahoe's Setup Assistant:--```yaml-boot_wait: 30--boot_commands:- # Dismiss greeting- - '<delay 5>'- - '<space>'- - '<delay 2>'-- # Language- - "<wait 'English', timeout=120>"- - "<type 'English'>"- - '<delay 1>'- - '<enter>'- - '<delay 2>'-- # Country- - "<wait 'Country or Region'>"- - '<click_at 960,900>'- - "<type 'United States'>"- - '<enter>'- - "<click 'Continue'>"- - '<delay 2>'-- # Transfer - select "Set up as new"- - "<wait 'Transfer Your Data'>"- - '<delay 20>'- - '<tab><tab><tab>'- - '<space>'- - "<click 'Continue'>"-- # Accessibility- - "<wait 'Accessibility'>"- - "<click 'Not Now'>"-- # Account creation- - "<wait 'Create a Mac Account'>"- - '<tab><tab><tab><tab><tab><tab>'- - "<type 'lume'>"- - '<tab><tab>'- - "<type 'lume'>"- - '<tab>'- - "<type 'lume'>"- - '<tab><tab><space>'- - "<click 'Continue'>"-- # Terms- - "<wait 'Terms and Conditions'>"- - "<click 'Agree', index=-1>"- - "<wait 'I have read and agree'>"- - "<click 'Agree', index=0>"--health_check:- type: ssh- user: lume- password: lume- timeout: 30- retries: 5- retry_delay: 10-```--## Health checks--Verify setup succeeded by testing SSH connectivity:--```yaml-health_check:- type: ssh- user: lume- password: lume- timeout: 30 # Seconds per attempt- retries: 5 # Number of attempts- retry_delay: 10 # Seconds between retries-```--The automation waits for the VM to become reachable via SSH before declaring success.--## Tips--**Timing issues** — If commands run before the screen is ready, increase `boot_wait` or add `<delay>` commands. Use `<wait 'text'>` to sync with specific screens.--**Text not found** — OCR is case-sensitive. Check exact spelling. If text is unreliable, use `<click_at X,Y>` with coordinates.--**Multiple occurrences** — When "Agree" appears in both license text and the button, use `<click 'Agree', index=-1>` to click the button (usually last).--**Keyboard navigation** — macOS doesn't Tab through all elements by default. Enable full keyboard access:--```yaml-- '<cmd+space>'-- "<type 'Keyboard'>"-- '<delay 2>'-- '<enter>'-# Then enable "Keyboard navigation" in settings-```--**Screen coordinates** — Default resolution is 1920x1440. Center is roughly `<click_at 960,720>`.--## Troubleshooting--| Problem | Solution |-| ---------------------- | ------------------------------------------------------ |-| Commands run too early | Increase `boot_wait`, add `<delay>` |-| Text not found | Check spelling/case, increase timeout, use coordinates |-| Wrong element clicked | Use `index` to select correct occurrence |-| Hotkeys ignored | Click desktop first to focus |-| Tab skips elements | Enable keyboard navigation in System Settings |--## Limitations--- **OCR depends on resolution** — Text detection accuracy varies with display settings-- **Version-specific** — Setup Assistant changes between macOS releases-- **No conditionals** — Commands run sequentially; can't branch based on screen state-- **Single display** — Coordinates assume one display at native resolution--## Reference--### Config fields--| Field | Type | Default | Description |-| --------------- | ------- | -------- | ------------------------------- |-| `boot_wait` | integer | 60 | Seconds to wait before starting |-| `boot_commands` | array | required | List of commands |-| `health_check` | object | optional | Verification config |--### Health check fields--| Field | Type | Default | Description |-| ------------- | ------- | -------- | ----------------------- |-| `type` | string | required | `ssh` |-| `user` | string | — | SSH username |-| `password` | string | — | SSH password |-| `timeout` | integer | 30 | Seconds per attempt |-| `retries` | integer | 3 | Number of attempts |-| `retry_delay` | integer | 5 | Seconds between retries |diff --git a/docs/content/docs/lume/guide/fundamentals/vm-management.mdx b/docs/content/docs/lume/guide/fundamentals/vm-management.mdxdeleted file mode 100644index 38eda3df56..0000000000--- a/docs/content/docs/lume/guide/fundamentals/vm-management.mdx+++ /dev/null@@ -1,219 +0,0 @@-----title: VM Management-description: Run, configure, and organize your VMs------import { Callout } from 'fumadocs-ui/components/callout';--Once you've [created a VM](/lume/guide/getting-started/quickstart), here's how to manage it.--## Quick reference--| Task | Command |-| ------------ | ----------------------------- |-| List VMs | `lume ls` |-| Get details | `lume get my-vm` |-| Run | `lume run my-vm` |-| Run headless | `lume run my-vm --no-display` |-| Stop | `lume stop my-vm` |-| Clone | `lume clone my-vm backup` |-| Delete | `lume delete my-vm` |--## Run a VM--```bash-# Opens VNC window-lume run my-vm--# Headless (for automation)-lume run my-vm --no-display-```--### Share files with your VM--Mount a host directory that appears in the VM at `/Volumes/My Shared Files`:--```bash-lume run my-vm --shared-dir ~/Projects-```--### Attach USB storage--Mount a disk image as USB storage (macOS 15+ only):--```bash-lume run my-vm --usb-storage /path/to/disk.img-```--## Linux VMs--Lume supports Linux VMs using ARM64 ISO images. The workflow differs slightly from macOS.--### Create and install--```bash-# Create empty VM configured for Linux-lume create ubuntu-vm --os linux --cpu 4 --memory 8GB --disk-size 50GB--# First boot: mount installer ISO-lume run ubuntu-vm --mount ~/Downloads/ubuntu-24.04-live-server-arm64.iso--# After installation: run normally-lume run ubuntu-vm-```--Complete the installation in the VNC window, shut down, then run without the ISO.--### Supported distributions--Any ARM64 Linux distribution works. Download the ARM64 (aarch64) ISO, not x86_64:--- **Ubuntu** - [ubuntu.com/download](https://ubuntu.com/download)-- **Debian** - [debian.org](https://www.debian.org/)-- **Fedora** - [fedoraproject.org](https://fedoraproject.org/)--<Callout type="info">- Linux VMs have no licensing restrictions—run as many concurrent instances as your hardware- supports. Only macOS VMs are limited to 2 per Mac.-</Callout>--## Stop a VM--```bash-# Graceful shutdown-lume stop my-vm--# Force stop (immediate)-lume stop my-vm --force-```--## Inspect VMs--```bash-# List all VMs with state, CPU, memory, disk-lume ls--# Detailed info for one VM-lume get my-vm-```--`lume get` shows state, resource allocation, network config, and display settings.--## Change VM settings--<Callout type="warn">- Stop the VM before changing settings. Disk size can only increase, not decrease.-</Callout>--```bash-# CPU cores-lume set my-vm --cpu 8--# Memory-lume set my-vm --memory 16GB--# Disk (increase only)-lume set my-vm --disk-size 100GB--# Display resolution-lume set my-vm --display 1920x1080-```--## Clone a VM--Cloning creates a full copy—useful for backups or creating variants:--```bash-lume clone my-vm my-vm-backup-```--Clone between storage locations:--```bash-lume clone my-vm external-copy \- --source-storage default \- --dest-storage external-ssd-```--## Delete a VM--```bash-lume delete my-vm--# From specific storage-lume delete my-vm --storage external-ssd-```--## Storage locations--By default, VMs live in `~/.lume`. You can add more locations—useful for external drives or separating projects.--### Configure storage--```bash-# List locations-lume config storage list--# Add a location-lume config storage add external-ssd /Volumes/ExternalSSD/lume--# Set default-lume config storage default external-ssd--# Remove a location-lume config storage remove old-location-```--### Use storage in commands--Most commands accept `--storage` to target a specific location. You can use either a **named location** or a **direct path**:--```bash-# Use a named storage location-lume create my-vm --os macos --ipsw latest --storage external-ssd-lume run my-vm --storage external-ssd--# Or use a direct path-lume create my-vm --os macos --ipsw latest --storage /Volumes/MySSD/vms-lume run my-vm --storage /Volumes/MySSD/vms/my-vm-```--Without `--storage`, Lume uses the default location or searches all locations.--### Move a VM between locations--Clone to the new location, then delete the original:--```bash-lume clone my-vm my-vm --source-storage default --dest-storage external-ssd-lume delete my-vm --storage default-```--## Image caching--When pulling images from a registry, Lume can cache layers locally for faster future pulls. Caching is **disabled by default** to save disk space.--```bash-# Enable caching-lume config caching set true--# Set cache directory-lume config cache set /path/to/cache--# View current config-lume config get--# Clear cache-lume prune-```--<Callout type="info">- Enable caching if you frequently pull the same images or images sharing base layers. With caching- disabled, each `lume pull` downloads everything fresh.-</Callout>--## What's next--- [Automate setup](/lume/guide/fundamentals/unattended-setup) with YAML configurations-- [Use the HTTP API](/lume/guide/advanced/http-server) for programmatic access-- See all options in the [CLI Reference](/lume/reference/cli-reference)diff --git a/docs/content/docs/lume/guide/getting-started/comparison.mdx b/docs/content/docs/lume/guide/getting-started/comparison.mdxdeleted file mode 100644index c88ed2ac00..0000000000--- a/docs/content/docs/lume/guide/getting-started/comparison.mdx+++ /dev/null@@ -1,100 +0,0 @@-----title: Comparison-description: How Lume compares to other macOS virtualization tools------import { Callout } from 'fumadocs-ui/components/callout';--This page compares Lume with other macOS virtualization tools. All of these are quality projects—the best choice depends on your use case.--## Quick Comparison--| Feature | Lume | Tart | Lima | UTM |-| -------------------- | ----------------------- | -------------- | ---------------- | --------------- |-| **License** | MIT | Fair Source | Apache 2.0 | Apache 2.0 |-| **macOS VMs** | Yes | Yes | No | Yes |-| **Linux VMs** | Yes | Yes | Yes | Yes |-| **HTTP API** | Yes | No | No | No |-| **MCP Server** | Yes | No | No | No |-| **Unattended Setup** | Yes (VNC + OCR) | Via Packer | N/A | No |-| **Registry Support** | OCI (GHCR, GCS) | OCI registries | N/A | No |-| **Primary Use Case** | Agent automation, CI/CD | CI/CD | Linux containers | General desktop |--## Tart--[Tart](https://github.com/cirruslabs/tart) is a mature CLI tool for macOS and Linux VMs on Apple Silicon. Like Lume, it uses Apple's Virtualization Framework, so core VM performance is identical.--**Key differences:**--- **Licensing**: Tart uses a [Fair Source](https://fair.io/) license, which is source-available but not fully open source. Lume is MIT licensed.--- **API access**: Lume includes an HTTP server (`lume serve`) for programmatic VM control. Tart is CLI-only—integrating it into applications requires wrapping CLI commands.--- **Unattended setup**: Lume automates the macOS Setup Assistant via VNC and OCR, creating ready-to-use VMs without manual clicking. Tart relies on Packer plugins for automation.--- **MCP integration**: Lume ships with an [MCP server](/lume/guide/advanced/mcp-server) for AI agent integration. Tart doesn't have this.--- **Ecosystem**: Tart has a larger community and extensive CI/CD integrations (Cirrus CI, GitHub Actions). Lume integrates with the [Cua agent framework](/cua/guide/get-started/what-is-cua) for computer-use agents.--**When to choose Tart**: If you need proven CI/CD integrations, prefer a larger community, or if the Fair Source license works for your use case.--**When to choose Lume**: If you need an HTTP API, MCP server integration, fully automated VM provisioning, MIT licensing, or are building AI agents that interact with macOS.--## Lima--[Lima](https://github.com/lima-vm/lima) is primarily designed for running Linux containers on macOS, similar to how Docker Desktop works.--**Key differences:**--- Lima doesn't support macOS VMs—it's focused on providing a Linux environment for container workloads.-- Lima automatically sets up file sharing and port forwarding between host and guest.-- Lima integrates with containerd and nerdctl for Docker-compatible workflows.--**When to choose Lima**: If you need Linux containers on macOS and want a Docker-like experience. Lima isn't suitable if you need macOS VMs.--## UTM--[UTM](https://mac.getutm.app/) provides a graphical interface for running VMs on macOS, supporting both Apple Silicon (via Virtualization Framework) and Intel emulation (via QEMU).--**Key differences:**--- UTM emphasizes GUI-based VM management, while Lume is CLI and API-first.-- UTM can emulate x86 systems on Apple Silicon (slower), while Lume only supports native ARM virtualization.-- UTM supports a wider range of guest operating systems via QEMU emulation.--**When to choose UTM**: If you prefer a graphical interface, need to run x86 operating systems, or want broad guest OS compatibility.--## OrbStack--[OrbStack](https://orbstack.dev/) is a fast, lightweight alternative to Docker Desktop for running containers and Linux machines on macOS.--**Key differences:**--- OrbStack focuses on Linux containers and doesn't support macOS VMs.-- OrbStack has excellent integration with Docker and Kubernetes workflows.-- OrbStack is a commercial product with a free tier.--**When to choose OrbStack**: If you're primarily working with Docker/Linux containers and want the best performance for that use case.--## VirtualBuddy--[VirtualBuddy](https://github.com/insidegui/VirtualBuddy) is a native macOS app for creating and managing macOS and Linux VMs with a focus on simplicity.--**Key differences:**--- VirtualBuddy is GUI-first with no CLI or API.-- Built for desktop use, not automation or CI/CD.-- Excellent for manually testing on different macOS versions.--**When to choose VirtualBuddy**: If you want a simple, native Mac app for occasional VM use without automation needs.--## Summary--Choose Lume if you need:--- An HTTP API for programmatic VM control-- MCP server integration for AI agents-- Fully automated macOS VM provisioning-- MIT-licensed tooling--The Virtualization Framework means all Swift-based tools (Lume, Tart, VirtualBuddy) have the same underlying VM performance. The differences are in the interfaces, automation capabilities, and ecosystem integration.diff --git a/docs/content/docs/lume/guide/getting-started/faq.mdx b/docs/content/docs/lume/guide/getting-started/faq.mdxdeleted file mode 100644index 1f5a1d2bba..0000000000--- a/docs/content/docs/lume/guide/getting-started/faq.mdx+++ /dev/null@@ -1,181 +0,0 @@-----title: FAQ-description: Frequently asked questions about Lume------Common questions about using Lume. For detailed guides, see [VM Management](/lume/guide/fundamentals/vm-management) and [Unattended Setup](/lume/guide/fundamentals/unattended-setup).--## Credentials--### What's the default password?--For VMs created with `--unattended tahoe`:--- **Username**: `lume`-- **Password**: `lume`--For manually configured VMs, you set credentials during the Setup Assistant.--## Storage--### Where are VMs stored?--Default location: `~/.lume`--Add more locations with:--```bash-lume config storage add external /Volumes/External/lume-lume config storage list-```--### Do VM disks use their full allocated size?--No. macOS uses sparse files—a 50GB disk might only use 20GB on your actual drive, growing as needed.--### How do I move a VM to another drive?--Clone it to the new location, then delete the original:--```bash-lume clone my-vm my-vm --source-storage default --dest-storage external-lume delete my-vm --storage default-```--## Configuration--### Where's the config file?--`~/.config/lume/config.yaml` (follows XDG Base Directory spec)--View settings with:--```bash-lume config get-```--### How do I manage cache settings?--```bash-# Check caching status-lume config caching get--# Enable/disable caching-lume config caching set true--# Set cache directory-lume config cache set /path/to/cache--# Clear cache-lume prune-```--### How do I change the display resolution?--By default, VMs use 1024x768. Change it with `lume set`:--```bash-lume set my-vm --display 1920x1080-```--The VM must be stopped for this to work. After restarting the VM, go to **System Settings → Displays**, enable **Show all resolutions**, and select your new resolution. You can also set display resolution during creation:--```bash-lume create my-vm --os macos --ipsw latest --display 1920x1080-```--## Limits--### How many macOS VMs can I run at once?--Apple's macOS Software License Agreement (section 2.B.iii) limits you to **2 concurrent macOS VMs** per physical Mac. This is a licensing restriction, not a technical one—the Virtualization Framework enforces it automatically.--Linux VMs have no such limitation. You can run as many as your hardware supports.--### Can I run VMs inside a VM (nested virtualization)?--**macOS VMs**: No. Apple's Virtualization Framework doesn't support nested virtualization for macOS guests. If you try to run `lume create` inside a macOS VM, you'll get "Virtualization is not available on this hardware."--**Linux VMs**: Yes, on macOS 15+. Linux guests can run VMs inside them using KVM or other hypervisors. This is useful for testing Kubernetes, Docker, or other virtualization workloads.--## Images--### How do I get the latest macOS IPSW URL?--```bash-lume ipsw-```--### Where can I get older macOS versions?--Apple only provides the latest IPSW officially. Community archives exist—search "macOS IPSW download" and ensure you get UniversalMac files (for Apple Silicon).--### How do I install from a specific IPSW?--Download the IPSW first, then:--```bash-lume create my-vm --os macos --ipsw /path/to/downloaded.ipsw-```--## Linux VMs--### How do I create a Linux VM?--Linux uses ISO images instead of IPSW. Create the VM, boot with the installer mounted, then run normally:--```bash-# Create empty Linux VM-lume create linux-vm --os linux--# First boot: mount installer ISO-lume run linux-vm --mount /path/to/ubuntu.iso--# After installation: run normally-lume run linux-vm-```--## VM Status--### What do the different VM statuses mean?--Check VM status with `lume ls`:--| Status | Description |-| --------------------------------- | ----------------------------------------------------- |-| `stopped` | VM is not running and ready to start |-| `running` | VM is running |-| `provisioning (ipsw_install)` | VM is being created—downloading and installing macOS |-| `provisioning (unattended_setup)` | VM is being created—running automated Setup Assistant |--VMs in `provisioning` status are being created asynchronously. Wait for status to change to `stopped` before starting them.--## Troubleshooting--### VM won't start--1. Check if another VM is running: `lume ls`-2. Stop any running VMs: `lume stop <name>`-3. Check available disk space-4. If VM shows `provisioning`, wait for creation to complete-5. Try running with `--no-display` to see if it's a VNC issue--### Can't connect to VM via SSH--1. Ensure SSH is enabled in the VM (System Settings → Sharing → Remote Login)-2. Get the VM's IP: `lume get my-vm`-3. Try: `ssh lume@<ip-address>`--For VMs created with `--unattended tahoe`, SSH is enabled automatically.--### `lume serve` port already in use--Another Lume instance is running. Either use it or kill it:--```bash-# Check what's using port 7777-lsof -i :7777--# Kill the process-kill <pid>-```diff --git a/docs/content/docs/lume/guide/getting-started/installation.mdx b/docs/content/docs/lume/guide/getting-started/installation.mdxdeleted file mode 100644index 4b683bbaeb..0000000000--- a/docs/content/docs/lume/guide/getting-started/installation.mdx+++ /dev/null@@ -1,153 +0,0 @@-----title: Installation-description: Install Lume on your Mac------import { Callout } from 'fumadocs-ui/components/callout';--Install Lume with a single command:--```bash-/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/lume/scripts/install.sh)"-```--This installs Lume with a background service (starts on login) and auto-updater (checks every 24 hours).--<Callout type="warn">-**First-time install?** If `~/.local/bin` isn't in your PATH (common on fresh macOS), add it:--```bash-echo 'export PATH="$PATH:$HOME/.local/bin"' >> ~/.zshrc-source ~/.zshrc-```--Or restart your terminal after the first command.--</Callout>--### Verify it worked--```bash-lume --version-# Lume 0.2.27--lume --help-# OVERVIEW: Lume - CLI and framework for macOS VMs...-```--If you see version output, you're ready to [create your first VM](/lume/guide/getting-started/quickstart).--## Requirements--- Apple Silicon Mac (M1/M2/M3/M4)-- macOS 13.0 or later-- 8GB RAM minimum (16GB recommended)-- 30GB free disk space--## Installation options--The default install includes a background service and auto-updater. You can disable either:--| Option | Command |-| --------------------- | --------------------------------------------------------- |-| No background service | `install.sh -- --no-background-service` |-| No auto-updates | `install.sh -- --no-auto-updater` |-| Neither | `install.sh -- --no-background-service --no-auto-updater` |--Full command with options:--```bash-/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/lume/scripts/install.sh)" -- --no-background-service-```--<Callout type="info">- Without the background service, you'll need to run `lume serve` manually when using tools that- rely on the Lume API (like the Computer Use Agent).-</Callout>--## Manual installation--Download `lume.pkg.tar.gz` from the [latest release](https://github.com/trycua/cua/releases?q=lume&expanded=true), extract it, and install the package.--## Managing services--### Background service--```bash-# Check if running-launchctl list | grep com.trycua.lume_daemon--# View logs-tail -f /tmp/lume_daemon.log--# Start manually (if service disabled)-lume serve-```--### Auto-updater--```bash-# Check if running-launchctl list | grep com.trycua.lume_updater--# View logs-tail -f /tmp/lume_updater.log--# Manual update check-~/.local/bin/lume-update-```--To disable auto-updates after installation:--```bash-launchctl unload ~/Library/LaunchAgents/com.trycua.lume_updater.plist-rm ~/Library/LaunchAgents/com.trycua.lume_updater.plist-```--## Uninstall--Remove Lume completely:--```bash-# Stop and remove background service-launchctl unload ~/Library/LaunchAgents/com.trycua.lume_daemon.plist 2>/dev/null-rm -f ~/Library/LaunchAgents/com.trycua.lume_daemon.plist--# Stop and remove auto-updater-launchctl unload ~/Library/LaunchAgents/com.trycua.lume_updater.plist 2>/dev/null-rm -f ~/Library/LaunchAgents/com.trycua.lume_updater.plist-rm -f ~/.local/bin/lume-update--# Optional: Remove cached images (run before removing binary)-lume prune--# Remove Lume binary-rm -f $(which lume)--# Optional: Remove VMs and config-rm -rf ~/.lume-rm -rf ~/.config/lume-```--## Troubleshooting--**`lume: command not found`** — The install location (`~/.local/bin`) isn't in your PATH. Add it and reload:--```bash-echo 'export PATH="$PATH:$HOME/.local/bin"' >> ~/.zshrc-source ~/.zshrc-```--Or restart your terminal after the first command.--**Background service not starting** — Check the daemon logs:--```bash-tail -50 /tmp/lume_daemon.log-```--**Permission errors** — The installer needs to create files in `~/.local/bin` and `~/Library/LaunchAgents`. Ensure you have write access to these directories.--**Intel Mac or Linux** — Lume requires Apple Silicon. It won't work on Intel Macs or other platforms.--Ready to create a VM? Head to the [Quickstart](/lume/guide/getting-started/quickstart).diff --git a/docs/content/docs/lume/guide/getting-started/introduction.mdx b/docs/content/docs/lume/guide/getting-started/introduction.mdxdeleted file mode 100644index bcbaddebfe..0000000000--- a/docs/content/docs/lume/guide/getting-started/introduction.mdx+++ /dev/null@@ -1,70 +0,0 @@-----title: What is Lume?-description: Introduction to Lume - the macOS VM CLI and framework------import { Callout } from 'fumadocs-ui/components/callout';--Lume is a VM runtime for building AI agents, running CI/CD pipelines, and automating macOS. It uses Apple's native Virtualization Framework to run macOS and Linux VMs at near-native speed on Apple Silicon.--<Callout type="info" title="MIT License">- Lume is open-source and MIT licensed. If you find it useful, we'd appreciate a [star on- GitHub](https://github.com/trycua/cua)!-</Callout>--<Callout type="info" title="Cloud macOS Sandboxes">- We're piloting a managed service for customers who want to run cloud macOS sandboxes for CI/CD and- agent workloads. [Book a demo](https://cal.com/cua/cua-demo?overlayCalendar=true) if you're- interested.-</Callout>--```bash-lume create test-vm --os macos --ipsw latest-lume run test-vm-```--A single binary with an HTTP API. Create a VM, run it headlessly, control it programmatically.--## Architecture----You can use Lume directly via CLI, or run `lume serve` to expose an HTTP API for programmatic access. The [Computer SDK](/cua/reference/computer-sdk) uses this API to automate macOS interactions.--## How it works--Lume is a thin layer over Apple's [Virtualization Framework](https://developer.apple.com/documentation/virtualization), which provides hardware-accelerated virtualization on Apple Silicon. This gives you:--- **Native speed** — CPU instructions execute directly via hardware virtualization-- **Paravirtualized graphics** — Basic GPU support via Apple's virtualization layer (limited to GPU Family 5)-- **Efficient storage** — Sparse disk files only consume actual usage, not allocated size-- **Rosetta 2 support** — Run x86 Linux binaries in ARM Linux VMs-- **Automated golden images** — Go from IPSW to fully configured macOS VM without manual intervention-- **Registry support** — Pull and push VM images from GHCR or GCS registries--## When to use Lume--**Testing across macOS versions** — Spin up a VM with a specific macOS version, test your software, tear it down. No need to maintain multiple physical machines.--**Automating macOS tasks** — Combine Lume with [Unattended Setup](/lume/guide/fundamentals/unattended-setup) to create pre-configured VMs. The setup automation uses VNC and OCR to click through the Setup Assistant without manual intervention.--**Running CI/CD locally** — Test your macOS builds in isolated VMs before pushing to remote CI. The `--no-display` flag runs VMs headlessly.--**Sandboxing risky operations** — Need to test untrusted software or destructive scripts? Run them in a VM, then delete it. Clone a known-good VM to reset to a clean state instantly.--**Building AI agents** — Lume powers the [Cua Computer SDK](/cua/reference/computer-sdk), providing VMs that AI models can interact with through screenshots and input simulation.--<Callout type="info" title="Used by Anthropic">- Apple's Virtualization Framework—the same technology Lume is built on—powers [Claude- Cowork](https://support.claude.com/en/articles/13345190-getting-started-with-cowork), Anthropic's- sandboxed environment for Claude Code. It downloads a Linux root filesystem and boots it in an- isolated VM where Claude can safely execute commands without access to your broader system.-</Callout>--## What Lume doesn't do--Lume requires Apple Silicon—it won't work on Intel Macs or other platforms.--## Get started--Ready to try it? [Install Lume](/lume/guide/getting-started/installation) and create your first VM in the [Quickstart](/lume/guide/getting-started/quickstart).diff --git a/docs/content/docs/lume/guide/getting-started/meta.json b/docs/content/docs/lume/guide/getting-started/meta.jsondeleted file mode 100644index 78349c0f4c..0000000000--- a/docs/content/docs/lume/guide/getting-started/meta.json+++ /dev/null@@ -1,7 +0,0 @@-{- "title": "Getting Started",- "description": "Get up and running with Lume",- "icon": "Rocket",- "defaultOpen": true,- "pages": ["introduction", "installation", "quickstart", "comparison", "faq"]-}diff --git a/docs/content/docs/lume/guide/getting-started/quickstart.mdx b/docs/content/docs/lume/guide/getting-started/quickstart.mdxdeleted file mode 100644index 69fd46ea67..0000000000--- a/docs/content/docs/lume/guide/getting-started/quickstart.mdx+++ /dev/null@@ -1,200 +0,0 @@-----title: Quickstart-description: Create and run your first VM with Lume------import { Callout } from 'fumadocs-ui/components/callout';--In under 5 minutes, you'll have a macOS VM running on your Mac.--## Get the macOS restore image--Before creating a VM, you need a macOS restore image (IPSW). You have two options:--### Option 1: Download manually (recommended)--Get the download URL and download the IPSW file yourself:--```bash-# Get the latest IPSW download URL-lume ipsw-# Output: https://updates.cdn-apple.com/...--# Download it (using curl, wget, or your browser)-curl -L -o ~/Downloads/macos.ipsw "$(lume ipsw 2>/dev/null | tail -1)"--# Create the VM with the downloaded file-lume create my-vm --os macos --ipsw ~/Downloads/macos.ipsw-```--This approach is better for slower connections since you can resume downloads and reuse the IPSW for multiple VMs.--### Option 2: Let Lume download it--Use `--ipsw latest` and Lume will download the restore image automatically:--```bash-lume create my-vm --os macos --ipsw latest-```--<Callout type="info">- The IPSW file is ~15GB. If you plan to create multiple VMs, downloading manually and reusing the- file saves time and bandwidth.-</Callout>--## Configure VM storage location (optional)--By default, VMs are stored in `~/.lume`. You can configure custom storage locations before creating VMs—useful for external drives or organizing VMs by project.--```bash-# Add an external drive as a storage location-lume config storage add external /Volumes/MySSD/lume--# Set it as the default for new VMs-lume config storage default external--# View all configured storage locations-lume config storage list-```--When creating a VM, specify which storage to use:--```bash-# Use a named storage location-lume create my-vm --os macos --ipsw latest --storage external--# Or use a direct path-lume create my-vm --os macos --ipsw latest --storage /path/to/vm/directory-```--<Callout type="info">- VM disk images can grow large (~50GB+). Using an external SSD keeps your internal storage free and- can improve performance for I/O-heavy workloads.-</Callout>--## Create a Linux VM--Lume also supports Linux VMs using ISO images:--```bash-# Create an empty Linux VM-lume create ubuntu-vm --os linux--# First boot: mount the installer ISO-lume run ubuntu-vm --mount ~/Downloads/ubuntu-24.04-live-server-arm64.iso--# After installation completes, run normally-lume run ubuntu-vm-```--<Callout type="info">- Linux VMs require ARM64 ISO images (not x86). Ubuntu, Debian, and Fedora all provide ARM64 server- and desktop images.-</Callout>--## Run your VM--```bash-lume run my-vm-```--A VNC window opens. For macOS VMs, you'll see the Setup Assistant—complete it manually or use [Unattended Setup](/lume/guide/fundamentals/unattended-setup) to automate it.--## Create a VM with custom resources--By default, Lume creates VMs with 4 CPU cores, 8GB memory, and a 50GB disk. Override any of these:--```bash-lume create dev-vm --os macos --ipsw latest \- --cpu 8 \- --memory 16GB \- --disk-size 100GB-```--| Resource | Default | Notes |-| -------- | ------- | -------------------------------------------- |-| CPU | 4 cores | Max is your Mac's core count |-| Memory | 8GB | Leave headroom for your host |-| Disk | 50GB | Uses sparse files—only consumes actual usage |--## Change display resolution--By default, VMs use a 1024x768 display resolution. You can change it with `lume set`:--```bash-lume set my-vm --display 1920x1080-```--The change takes effect on the next VM start. After starting the VM, go to **System Settings → Displays**, enable **Show all resolutions**, and select your new resolution. Common resolutions:--| Resolution | Aspect Ratio | Use Case |-| ---------- | ------------ | --------------------------- |-| 1024x768 | 4:3 | Default, low resource usage |-| 1280x800 | 16:10 | MacBook-style |-| 1920x1080 | 16:9 | Full HD |-| 2560x1440 | 16:9 | QHD |--## Automate the Setup Assistant--Skip manual setup entirely with the `--unattended` flag:--```bash-lume create my-vm --os macos --ipsw latest --unattended tahoe-```--The `tahoe` preset runs through the Setup Assistant automatically, creating a user `lume` with password `lume` and enabling SSH. This takes 10-15 minutes but requires no interaction.--For custom configurations, see [Unattended Setup](/lume/guide/fundamentals/unattended-setup).--## Run a VM--```bash-# With display (VNC window opens)-lume run my-vm--# Headless (for automation, CI/CD)-lume run my-vm --no-display--# Run in background (doesn't block terminal)-lume run my-vm > /dev/null 2>&1 &-```--When running headless or in background, connect via SSH (if enabled) or VNC client. Use `lume ls` to see running VMs and their connection details.--## Common operations--```bash-# List all VMs-lume ls--# Get details about a VM-lume get my-vm--# Stop a running VM-lume stop my-vm--# Clone a VM-lume clone my-vm my-vm-backup--# Delete a VM-lume delete my-vm-```--## Share files with your VM--Mount a host directory inside the VM:--```bash-lume run my-vm --shared-dir ~/Projects-```--Inside the VM, find your files at `/Volumes/My Shared Files`.--## What's next--Now that you have a VM running, you can:--- Learn more [VM operations](/lume/guide/fundamentals/vm-management) like modifying resources and managing storage locations-- Set up [automated configuration](/lume/guide/fundamentals/unattended-setup) with custom YAML files-- Integrate with your tools via the [HTTP API](/lume/guide/advanced/http-server)-- Connect AI agents like Claude Desktop via [MCP](/lume/guide/advanced/mcp-server)diff --git a/docs/content/docs/lume/guide/meta.json b/docs/content/docs/lume/guide/meta.jsondeleted file mode 100644index 900c259458..0000000000--- a/docs/content/docs/lume/guide/meta.json+++ /dev/null@@ -1,6 +0,0 @@-{- "title": "Guide",- "description": "Learn how to use Lume",- "icon": "Book",- "pages": ["getting-started", "fundamentals", "advanced"]-}diff --git a/docs/content/docs/lume/meta.json b/docs/content/docs/lume/meta.jsondeleted file mode 100644index af3275ad8e..0000000000--- a/docs/content/docs/lume/meta.json+++ /dev/null@@ -1,5 +0,0 @@-{- "title": "Lume",- "description": "CLI and framework for macOS VMs using Apple Virtualization",- "pages": ["guide", "examples", "reference"]-}diff --git a/docs/content/docs/lume/reference/changelog.mdx b/docs/content/docs/lume/reference/changelog.mdxdeleted file mode 100644index 82b9acba62..0000000000--- a/docs/content/docs/lume/reference/changelog.mdx+++ /dev/null@@ -1,441 +0,0 @@-----title: Changelog-description: Release history for Lume------{/*-AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY-Generated by: npx tsx scripts/docs-generators/generate-changelog.ts-Last updated: 2026-02-04-*/}--# Lume Changelog--All notable changes to the Lume are documented here.--## 0.2.x--### v0.2.76 (2026-02-03)--Maintenance release.--### v0.2.75 (2026-02-03)--Maintenance release.--### v0.2.74 (2026-02-03)--Maintenance release.--### v0.2.73 (2026-02-03)--Maintenance release.--### v0.2.72 (2026-02-03)--Maintenance release.--### v0.2.71 (2026-02-03)--Maintenance release.--### v0.2.70 (2026-02-03)--Maintenance release.--### v0.2.69 (2026-02-02)--Maintenance release.--### v0.2.68 (2026-02-02)--Maintenance release.--### v0.2.67 (2026-02-02)--Maintenance release.--### v0.2.66 (2026-02-02)--Maintenance release.--### v0.2.65 (2026-02-02)--Maintenance release.--### v0.2.64 (2026-02-02)--Maintenance release.--### v0.2.63 (2026-02-02)--Maintenance release.--### v0.2.62 (2026-02-02)--Maintenance release.--### v0.2.61 (2026-02-02)--Maintenance release.--### v0.2.60 (2026-02-02)--Maintenance release.--### v0.2.59 (2026-02-01)--Maintenance release.--### v0.2.58 (2026-02-01)--Maintenance release.--### v0.2.57 (2026-02-01)--Maintenance release.--### v0.2.56 (2026-02-01)--Maintenance release.--### v0.2.55 (2026-02-01)--Maintenance release.--### v0.2.54 (2026-02-01)--Maintenance release.--### v0.2.52 (2026-01-25)--Maintenance release.--### v0.2.51 (2026-01-21)--Maintenance release.--### v0.2.50 (2026-01-21)--Maintenance release.--### v0.2.49 (2026-01-21)--Maintenance release.--### v0.2.46 (2026-01-20)--Maintenance release.--### v0.2.45 (2026-01-19)--Maintenance release.--### v0.2.44 (2026-01-19)--Maintenance release.--### v0.2.43 (2026-01-19)--Maintenance release.--### v0.2.42 (2026-01-19)--Maintenance release.--### v0.2.41 (2026-01-19)--Maintenance release.--### v0.2.40 (2026-01-18)--Maintenance release.--### v0.2.39 (2026-01-18)--Maintenance release.--### v0.2.38 (2026-01-18)--Maintenance release.--### v0.2.37 (2026-01-18)--Maintenance release.--### v0.2.36 (2026-01-18)--Maintenance release.--### v0.2.34 (2026-01-18)--Maintenance release.--### v0.2.33 (2026-01-18)--Maintenance release.--### v0.2.32 (2026-01-18)--Maintenance release.--### v0.2.31 (2026-01-17)--Maintenance release.--### v0.2.30 (2026-01-17)--Maintenance release.--### v0.2.29 (2026-01-17)--Maintenance release.--### v0.2.28 (2026-01-17)--Maintenance release.--### v0.2.27 (2026-01-11)--Maintenance release.--### v0.2.26 (2026-01-11)--Maintenance release.--### v0.2.25 (2026-01-11)--Maintenance release.--### v0.2.23 (2025-12-23)--- Update README badges with sky/emerald colors and larger logo by @f-trycua in https://github.com/trycua/cua/pull/641-- Add QEMU Ubuntu 22.04 template with Cua computer-server support by @synacktraa in https://github.com/trycua/cua/pull/622-- Add QEMU Windows 11 template with Cua computer-server support by @synacktraa in https://github.com/trycua/cua/pull/551-- docs: update intro page with hero image and Geist font by @f-trycua in https://github.com/trycua/cua/pull/642-- docs: expand Cua description with screenshot-VLM-action loop by @f-trycua in https://github.com/trycua/cua/pull/643-- F trycua/docs hero image by @f-trycua in https://github.com/trycua/cua/pull/644-- Introduce cua-bench-ui submodules (webview API, configurable ports, improved python RPC) by @ddupont808 in https://github.com/trycua/cua/pull/645-- Replace PyQt6 with PySide6 by @ddupont808 in https://github.com/trycua/cua/pull/646-- Bump cua-bench-ui version in xfce Dockerfile by @ddupont808 in https://github.com/trycua/cua/pull/647-- Replace qt backend with gtk in xfce Docker image by @ddupont808 in https://github.com/trycua/cua/pull/648-- fix(docs): broken link references for docs by @sarinali in https://github.com/trycua/cua/pull/652-- fix(docs): document cloud models by @sarinali in https://github.com/trycua/cua/pull/651-- Refactor Docker publish workflow to correctly build and publish multi-arch images by @synacktraa in https://github.com/trycua/cua/pull/653-- Refactor Python quickstart examples to use asyncio by @LucaStngn in https://github.com/trycua/cua/pull/650-- fix(lume): Resolve install script downloading wrong release by @synacktraa in https://github.com/trycua/cua/pull/655--### v0.2.22 (2025-06-17)--Maintenance release.--### v0.2.16 (2025-06-16)--- Add .devcontainer and Dockerfile fixes by @ddupont808 in https://github.com/trycua/cua/pull/287-- [Agent] Removed extra screenshots in OpenAI/Anthropic tools, fixed image retention bug by @ddupont808 in https://github.com/trycua/cua/pull/291--### v0.2.15 (2025-05-23)--- Lumier computer provider takes non-trycua image by @jklapacz in https://github.com/trycua/cua/pull/181-- Added Cmake dep to playground.sh by @ddupont808 in https://github.com/trycua/cua/pull/184--### v0.2.14 (2025-05-14)--- [Computer] Add initial VM Provider by @f-trycua in https://github.com/trycua/cua/pull/162--### v0.2.13 (2025-05-11)--- Add clipboard and audio device by @f-trycua in https://github.com/trycua/cua/pull/170--### v0.2.12 (2025-05-07)--- Replaced remaining `os=` occurrences with `os_type=` by @ddupont808 in https://github.com/trycua/cua/pull/159-- Prevent install if sudo by @f-trycua in https://github.com/trycua/cua/pull/161--### v0.2.11 (2025-05-06)--Maintenance release.--### v0.2.10 (2025-05-06)--Maintenance release.--### v0.2.9 (2025-05-05)--- [Agent] Add standardized agent trajectory logging by @ddupont808 in https://github.com/trycua/cua/pull/155--### v0.2.8 (2025-05-02)--Maintenance release.--### v0.2.7 (2025-04-30)--Maintenance release.--### v0.2.6 (2025-04-29)--- docs: fix wait action in blog notebook by @FinnBorge in https://github.com/trycua/cua/pull/142-- [Lumier] Introduce Lumier - Docker-based macOS VM Management Interface by @f-trycua in https://github.com/trycua/cua/pull/144--New Contributors--- @FinnBorge made their first contribution in https://github.com/trycua/cua/pull/142--### v0.2.5 (2025-04-25)--Maintenance release.--### v0.2.4 (2025-04-24)--- [Root] Fix root py project toml by @f-trycua in https://github.com/trycua/cua/pull/133-- Update Mac identifier during clone process by @dp221125 in https://github.com/trycua/cua/pull/130--New Contributors--- @dp221125 made their first contribution in https://github.com/trycua/cua/pull/130--### v0.2.3 (2025-04-23)--Maintenance release.--### v0.2.2 (2025-04-22)--Maintenance release.--### v0.2.1 (2025-04-22)--Maintenance release.--### v0.2.0 (2025-04-22)--- [Computer] Add Gradio UI and fix interaction bugs by @ddupont808 in https://github.com/trycua/cua/pull/116-- Docs: Add EasyOCR SSL & Lume VM Startup Troubleshooting to FAQ by @trospix in https://github.com/trycua/cua/pull/114-- [Lume] Fix broken storage tests by @f-trycua in https://github.com/trycua/cua/pull/107-- docs: add trospix as a contributor for code by @allcontributors in https://github.com/trycua/cua/pull/115-- docs: add eltociear as a contributor for code by @allcontributors in https://github.com/trycua/cua/pull/121-- docs: update computer/README.md by @eltociear in https://github.com/trycua/cua/pull/119-- [Lume] Sparse Push & Pull Optimizations by @f-trycua in https://github.com/trycua/cua/pull/122--New Contributors--- @trospix made their first contribution in https://github.com/trycua/cua/pull/114-- @eltociear made their first contribution in https://github.com/trycua/cua/pull/119--## 0.1.x--### v0.1.38 (2025-04-15)--Maintenance release.--### v0.1.37 (2025-04-15)--Maintenance release.--### v0.1.36 (2025-04-15)--Maintenance release.--### v0.1.35 (2025-04-15)--- Update install_mcp_server.sh - remove piping to /dev/null by @rahulkarajgikar in https://github.com/trycua/cua/pull/110-- docs: add rahulkarajgikar as a contributor for code by @allcontributors in https://github.com/trycua/cua/pull/111-- Support py >= 3.13 by @f-trycua in https://github.com/trycua/cua/pull/112--New Contributors--- @rahulkarajgikar made their first contribution in https://github.com/trycua/cua/pull/110--### v0.1.34 (2025-04-14)--- [Lume] Add multiple VM locations and configurable cache by @f-trycua in https://github.com/trycua/cua/pull/91--### v0.1.33 (2025-04-14)--- [Agent] Improved Gradio UI by @ddupont808 in https://github.com/trycua/cua/pull/104--### v0.1.32 (2025-04-12)--- [Lume] Optimize VM image assembly by @f-trycua in https://github.com/trycua/cua/pull/102--### v0.1.31 (2025-04-11)--- [Agent] Fix leftover Gradio docs by @f-trycua in https://github.com/trycua/cua/pull/97-- docs: add RicterZ as a contributor for code by @allcontributors in https://github.com/trycua/cua/pull/98-- [Lume] Fix reassemble kill by @f-trycua in https://github.com/trycua/cua/pull/99--### v0.1.30 (2025-04-07)--Maintenance release.--### v0.1.29 (2025-04-07)--Maintenance release.--### v0.1.28 (2025-04-07)--Maintenance release.--### v0.1.27 (2025-04-07)--Maintenance release.--### v0.1.26 (2025-04-06)--- Fix anthropic format in omni loop by @f-trycua in https://github.com/trycua/cua/pull/94--### v0.1.25 (2025-04-06)--Maintenance release.--### v0.1.24 (2025-04-06)--- [Agent] Add Gradio UI & OAI-Compatible Provider by @f-trycua in https://github.com/trycua/cua/pull/93--### v0.1.23 (2025-04-04)--- Add ai-gradio notebook by @f-trycua in https://github.com/trycua/cua/pull/85-- Update README with gradio by @f-trycua in https://github.com/trycua/cua/pull/86-- [SOM] Fix overlapping bounding boxes and added GPU/MPS support by @ddupont808 in https://github.com/trycua/cua/pull/87-- Add Ollama support in Omni parser by @Lizzard1123 in https://github.com/trycua/cua/pull/76-- docs: add ddupont808 as a contributor for code by @allcontributors in https://github.com/trycua/cua/pull/88-- docs: add Lizzard1123 as a contributor for code by @allcontributors in https://github.com/trycua/cua/pull/89-- [Agent] Add ollama support by @f-trycua in https://github.com/trycua/cua/pull/90--New Contributors--- @ddupont808 made their first contribution in https://github.com/trycua/cua/pull/87-- @Lizzard1123 made their first contribution in https://github.com/trycua/cua/pull/76--### v0.1.22 (2025-04-02)--- Refactor Developer Guide by @f-trycua in https://github.com/trycua/cua/pull/77-- Add operator blogpost - part 1 by @f-trycua in https://github.com/trycua/cua/pull/80-- [Agent] Add OpenAI generate summary by @f-trycua in https://github.com/trycua/cua/pull/84--### v0.1.21 (2025-03-30)--Maintenance release.--### v0.1.20 (2025-03-30)--- [Lume] Optimize disk image reassembly for performance and storage by @f-trycua in https://github.com/trycua/cua/pull/72--### v0.1.19 (2025-03-24)--- Add `--no-cache` option in lume pull by @aktech in https://github.com/trycua/cua/pull/67--### v0.1.18 (2025-03-19)--Maintenance release.--### v0.1.17 (2025-03-17)--Maintenance release.--### v0.1.16 (2025-03-17)--Maintenance release.--### v0.1.14 (2025-03-17)--- Fix url for clone endpoint by @aktech in https://github.com/trycua/computer/pull/54diff --git a/docs/content/docs/lume/reference/cli-reference.mdx b/docs/content/docs/lume/reference/cli-reference.mdxdeleted file mode 100644index 1a0e8475d6..0000000000--- a/docs/content/docs/lume/reference/cli-reference.mdx+++ /dev/null@@ -1,292 +0,0 @@-----title: CLI Reference-description: Command Line Interface reference for Lume------{/*- AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY- Generated by: npx tsx scripts/docs-generators/lume.ts- Source: libs/lume/src/Commands/*.swift- Version: 0.3.0-*/}--import { Callout } from 'fumadocs-ui/components/callout';-import { VersionHeader } from '@/components/version-selector';--<VersionHeader- versions={[{"version":"0.3","href":"/lume/reference/cli-reference","isCurrent":true},{"version":"0.2","href":"/lume/reference/v0.2/cli-reference","isCurrent":false}]}- currentVersion="0.3"- fullVersion="0.3.0"- packageName="lume"- installCommand="curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/lume/scripts/install.sh | bash"-/>--A lightweight CLI and local API server to build, run and manage macOS VMs.--## Quick Start--```bash-# Run a prebuilt macOS VM-lume run macos-tahoe-vanilla:latest--# Create a custom VM-lume create my-vm --cpu 4 --memory 8GB --disk-size 50GB-```--## VM Management--### lume create--Create a new virtual machine--**Arguments:**--- `<name>` - Name for the virtual machine--**Options:**--- `--os` - Operating system to install (macOS or linux) (default: macOS)-- `--cpu` - Number of CPU cores (default: 4)-- `--memory` - Memory size (e.g., 8GB) (default: 8GB)-- `--disk-size` - Disk size (e.g., 50GB) (default: 50GB)-- `--display` - Display resolution (e.g., 1024x768) (default: 1024x768)-- `--ipsw` - Path to IPSW file or 'latest' for macOS VMs-- `--storage` - VM storage location to use-- `--unattended` - [Preview] Preset name or path to YAML config file for unattended macOS Setup Assistant automation. Built-in presets: sequoia, tahoe. Only supported for macOS VMs.-- `--debug-dir` - Custom directory for debug screenshots during unattended setup (defaults to unique folder in system temp)-- `--vnc-port` - Port to use for the VNC server during unattended setup. Defaults to 0 (auto-assign) (default: 0)--**Flags:**--- `--debug` - Enable debug mode for unattended setup - saves screenshots with click coordinates-- `--no-display` - Do not open the VNC client during unattended setup (default: true for unattended)--### lume clone--Clone an existing virtual machine--**Arguments:**--- `<name>` - Name of the source VM-- `<new-name>` - Name for the cloned VM--**Options:**--- `--source-storage` - Source VM storage location-- `--dest-storage` - Destination VM storage location--### lume run--Run a virtual machine--**Arguments:**--- `<name>` - Name of the VM or image to run (format: name or name:tag)--**Options:**--- `--shared-dir` - Directory to share with the VM (format: path or path:ro or path:rw)-- `--mount` - For Linux VMs only, attach a read-only disk image-- `--usb-storage` - Disk image to attach as USB mass storage device-- `--registry` - Container registry URL (default: ghcr.io)-- `--organization` - Organization to pull from (default: trycua)-- `--vnc-port` - Port for VNC server (0 for auto-assign) (default: 0)-- `--recovery-mode` - For macOS VMs only, boot in recovery mode (default: false)-- `--storage` - VM storage location to use--**Flags:**--- `-d, --no-display` - Do not start the VNC client--### lume stop--Stop a virtual machine--**Arguments:**--- `<name>` - Name of the VM to stop--**Options:**--- `--storage` - VM storage location to use--### lume delete--Delete a virtual machine--**Arguments:**--- `<name>` - Name of the VM to delete--**Options:**--- `--storage` - VM storage location to use--**Flags:**--- `--force` - Force deletion without confirmation--## VM Information and Configuration--### lume get--Get detailed information about a virtual machine--**Arguments:**--- `<name>` - Name of the VM--**Options:**--- `-f, --format` - Output format (default: text)-- `--storage` - VM storage location to use--### lume set--Set new values for CPU, memory, and disk size of a virtual machine--**Arguments:**--- `<name>` - Name of the VM--**Options:**--- `--cpu` - New number of CPU cores-- `--memory` - New memory size (e.g., 8GB)-- `--disk-size` - New disk size (e.g., 100GB)-- `--display` - New display resolution-- `--storage` - VM storage location to use--### lume ls--List virtual machines--**Options:**--- `-f, --format` - Output format (json or text) (default: text)-- `--storage` - Filter by storage location name--## Image Management--### lume pull--Pull a macOS image from GitHub Container Registry--**Arguments:**--- `<image>` - Image to pull (format: name:tag)-- `<name>` - Name for the resulting VM (optional)--**Options:**--- `--registry` - Container registry URL (default: ghcr.io)-- `--organization` - Organization to pull from (default: trycua)-- `--storage` - VM storage location to use--### lume push--Push a macOS VM to GitHub Container Registry--**Arguments:**--- `<name>` - Name of VM to push-- `<image>` - Image tag (format: name:tag)--**Options:**--- `--additional-tags` - Additional tags to push-- `--registry` - Container registry URL (default: ghcr.io)-- `--organization` - Organization to push to (default: trycua)-- `--storage` - VM storage location to use-- `--chunk-size-mb` - Chunk size for upload in MB (default: 512)--**Flags:**--- `--verbose` - Enable verbose logging-- `--dry-run` - Prepare files without uploading-- `--reassemble` - Verify integrity by reassembling chunks--### lume images--List available macOS images from local cache--**Options:**--- `--organization` - Organization to list images for (default: trycua)--### lume ipsw--Get macOS restore image IPSW URL--### lume prune--Remove cached images--## Configuration and Server--### lume serve--Start the VM management server--**Options:**--- `--port` - Port to listen on (default: 7777)--### lume config--Get or set lume configuration--**Subcommands:**--- `lume config get` - Get current configuration-- `lume config storage` - Manage VM storage locations- - `lume config storage add` - Add a new VM storage location- - `lume config storage remove` - Remove a VM storage location- - `lume config storage list` - List all VM storage locations- - `lume config storage default` - Set the default VM storage location-- `lume config cache` - Manage cache settings- - `lume config cache get` - Get current cache directory- - `lume config cache set` - Set cache directory-- `lume config caching` - Manage image caching settings- - `lume config caching get` - Show current caching status- - `lume config caching set` - Enable or disable image caching--### lume logs--View lume serve logs--**Subcommands:**--- `lume logs info` - View info logs from the daemon- - `-n, --lines` - Number of lines to display-- `lume logs error` - View error logs from the daemon- - `-n, --lines` - Number of lines to display-- `lume logs all` - View both info and error logs- - `-n, --lines` - Number of lines to display--### lume setup--[Preview] Run unattended Setup Assistant automation on a macOS VM--**Arguments:**--- `<name>` - Name of the virtual machine--**Options:**--- `--unattended` - Preset name or path to YAML config file for unattended macOS Setup Assistant automation. Built-in presets: sequoia, tahoe.-- `--storage` - VM storage location to use or direct path to VM location-- `--vnc-port` - Port to use for the VNC server. Defaults to 0 (auto-assign) (default: 0)-- `--debug-dir` - Custom directory for debug screenshots (defaults to unique folder in system temp)--**Flags:**--- `--no-display` - Do not open the VNC client automatically-- `--debug` - Enable debug mode - saves screenshots with click coordinates--## Global Options--These options are available for all commands:--- `--help` - Show help information-- `--version` - Show version numberdiff --git a/docs/content/docs/lume/reference/http-api.mdx b/docs/content/docs/lume/reference/http-api.mdxdeleted file mode 100644index 79f38961df..0000000000--- a/docs/content/docs/lume/reference/http-api.mdx+++ /dev/null@@ -1,1072 +0,0 @@-----title: API Reference-description: HTTP API reference for Lume server------{/*- AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY- Generated by: npx tsx scripts/docs-generators/lume.ts- Source: libs/lume/src/Server/*.swift- Version: 0.3.0-*/}--import { Callout } from 'fumadocs-ui/components/callout';-import { Tabs, Tab } from 'fumadocs-ui/components/tabs';-import { VersionHeader } from '@/components/version-selector';--<VersionHeader- versions={[{"version":"0.3","href":"/lume/reference/cli-reference","isCurrent":true},{"version":"0.2","href":"/lume/reference/v0.2/cli-reference","isCurrent":false}]}- currentVersion="0.3"- fullVersion="0.3.0"- packageName="lume"- installCommand="curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/lume/scripts/install.sh | bash"-/>--HTTP API for managing macOS and Linux virtual machines--## Default URL--```-http://localhost:7777-```--Start the server with `lume serve` or specify a custom port with `lume serve --port <port>`.--## VM Management--### All virtual machines--List all virtual machines--`GET: /lume/vms`--#### Parameters--| Name | Type | Required | Description |-| ---- | ---- | -------- | ----------- |-| storage | string | No | Filter by storage location name |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl "http://localhost:7777/lume/vms"-```- </Tab>- <Tab value="Python">-```python-import requests--response = requests.get("http://localhost:7777/lume/vms")-print(response.json())-```- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/vms`);-const data = await response.json();-```- </Tab>-</Tabs>--#### Response--- **200**: Success-- **400**: Bad request-------### Detailed information about a specific virtual machine--Get detailed information about a specific virtual machine--`GET: /lume/vms/:name`--#### Parameters--| Name | Type | Required | Description |-| ---- | ---- | -------- | ----------- |-| name | string | Yes | Name of the VM |-| storage | string | No | VM storage location to use |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl "http://localhost:7777/lume/vms/{name}"-```- </Tab>- <Tab value="Python">-```python-import requests--response = requests.get("http://localhost:7777/lume/vms/{name}")-print(response.json())-```- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/vms/${name}`);-const data = await response.json();-```- </Tab>-</Tabs>--#### Response--- **200**: Success-- **400**: VM not found or invalid request-------### A new virtual machine--Create a new virtual machine--`POST: /lume/vms`--#### Request Body--| Name | Type | Required | Description |-| ---- | ---- | -------- | ----------- |-| name | string | Yes | Name for the virtual machine |-| os | string | Yes | Operating system to install (macOS or linux) |-| cpu | integer | Yes | Number of CPU cores |-| memory | string | Yes | Memory size (e.g., 8GB) |-| diskSize | string | Yes | Disk size (e.g., 50GB) |-| display | string | Yes | Display resolution (e.g., 1024x768) |-| ipsw | string | No | Path to IPSW file or 'latest' for macOS VMs |-| storage | string | No | VM storage location to use |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl -X POST "http://localhost:7777/lume/vms" \- -H "Content-Type: application/json" \- -d '{- "name": "my-vm",- "os": "macOS",- "cpu": 4,- "memory": "8GB",- "diskSize": "50GB",- "display": "1024x768"-}'-```- </Tab>- <Tab value="Python">-```python-import requests--data = {- "name": "my-vm",- "os": "macOS",- "cpu": 4,- "memory": "8GB",- "diskSize": "50GB",- "display": "1024x768",-}--response = requests.post("http://localhost:7777/lume/vms", json=data)-print(response.json())-```- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/vms`, {- method: "POST",- headers: { "Content-Type": "application/json" },- body: JSON.stringify({- name: "my-vm",- os: "macOS",- cpu: 4,- memory: "8GB",- diskSize: "50GB",- display: "1024x768",- }),-});-const data = await response.json();-```- </Tab>-</Tabs>--#### Response--- **200**: VM created successfully-- **400**: Invalid request body or VM creation failed-------### A virtual machine and its associated files--Delete a virtual machine and its associated files--`DELETE: /lume/vms/:name`--#### Parameters--| Name | Type | Required | Description |-| ---- | ---- | -------- | ----------- |-| name | string | Yes | Name of the VM to delete |-| storage | string | No | VM storage location |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl -X DELETE "http://localhost:7777/lume/vms/{name}"-```- </Tab>- <Tab value="Python">-```python-import requests--response = requests.delete("http://localhost:7777/lume/vms/{name}")-print(response.json())-```- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/vms/${name}`, {- method: "DELETE",-});-const data = await response.json();-```- </Tab>-</Tabs>--#### Response--- **200**: VM deleted successfully-- **400**: VM not found or deletion failed-------### A copy of an existing virtual machine--Create a copy of an existing virtual machine--`POST: /lume/vms/clone`--#### Request Body--| Name | Type | Required | Description |-| ---- | ---- | -------- | ----------- |-| name | string | Yes | Name of the source VM |-| newName | string | Yes | Name for the cloned VM |-| sourceLocation | string | No | Source VM storage location |-| destLocation | string | No | Destination VM storage location |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl -X POST "http://localhost:7777/lume/vms/clone" \- -H "Content-Type: application/json" \- -d '{- "name": "my-vm",- "newName": "example"-}'-```- </Tab>- <Tab value="Python">-```python-import requests--data = {- "name": "my-vm",- "newName": "example",-}--response = requests.post("http://localhost:7777/lume/vms/clone", json=data)-print(response.json())-```- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/vms/clone`, {- method: "POST",- headers: { "Content-Type": "application/json" },- body: JSON.stringify({- name: "my-vm",- newName: "example",- }),-});-const data = await response.json();-```- </Tab>-</Tabs>--#### Response--- **200**: VM cloned successfully-- **400**: Clone operation failed-------### Virtual machine configuration settings--Update virtual machine configuration settings--`PATCH: /lume/vms/:name`--#### Parameters--| Name | Type | Required | Description |-| ---- | ---- | -------- | ----------- |-| name | string | Yes | Name of the VM to update |--#### Request Body--| Name | Type | Required | Description |-| ---- | ---- | -------- | ----------- |-| cpu | integer | No | New number of CPU cores |-| memory | string | No | New memory size (e.g., 16GB) |-| diskSize | string | No | New disk size (e.g., 100GB) |-| display | string | No | New display resolution |-| storage | string | No | VM storage location |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl -X PATCH "http://localhost:7777/lume/vms/{name}" \- -H "Content-Type: application/json" \- -d '{}'-```- </Tab>- <Tab value="Python">-```python-import requests--data = {-}--response = requests.patch("http://localhost:7777/lume/vms/{name}", json=data)-print(response.json())-```- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/vms/${name}`, {- method: "PATCH",- headers: { "Content-Type": "application/json" },- body: JSON.stringify({- }),-});-const data = await response.json();-```- </Tab>-</Tabs>--#### Response--- **200**: Settings updated successfully-- **400**: Invalid settings or update failed-------### Start a virtual machine--Start a virtual machine--`POST: /lume/vms/:name/run`--#### Parameters--| Name | Type | Required | Description |-| ---- | ---- | -------- | ----------- |-| name | string | Yes | Name of the VM to start |--#### Request Body--| Name | Type | Required | Description |-| ---- | ---- | -------- | ----------- |-| noDisplay | boolean | No | Run without VNC display (default: false) |-| sharedDirectories | array | No | Directories to share with the VM |-| recoveryMode | boolean | No | Boot macOS VM in recovery mode (default: false) |-| storage | string | No | VM storage location |-| clipboard | boolean | No | Enable bidirectional clipboard sync via SSH (experimental) (default: false) |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl -X POST "http://localhost:7777/lume/vms/{name}/run" \- -H "Content-Type: application/json" \- -d '{}'-```- </Tab>- <Tab value="Python">-```python-import requests--data = {-}--response = requests.post("http://localhost:7777/lume/vms/{name}/run", json=data)-print(response.json())-```- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/vms/${name}/run`, {- method: "POST",- headers: { "Content-Type": "application/json" },- body: JSON.stringify({- }),-});-const data = await response.json();-```- </Tab>-</Tabs>--#### Response--- **202**: VM start initiated (async operation)-- **400**: Invalid request or VM not found-------### A running virtual machine--Stop a running virtual machine--`POST: /lume/vms/:name/stop`--#### Parameters--| Name | Type | Required | Description |-| ---- | ---- | -------- | ----------- |-| name | string | Yes | Name of the VM to stop |--#### Request Body--| Name | Type | Required | Description |-| ---- | ---- | -------- | ----------- |-| storage | string | No | VM storage location |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl -X POST "http://localhost:7777/lume/vms/{name}/stop" \- -H "Content-Type: application/json" \- -d '{}'-```- </Tab>- <Tab value="Python">-```python-import requests--data = {-}--response = requests.post("http://localhost:7777/lume/vms/{name}/stop", json=data)-print(response.json())-```- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/vms/${name}/stop`, {- method: "POST",- headers: { "Content-Type": "application/json" },- body: JSON.stringify({- }),-});-const data = await response.json();-```- </Tab>-</Tabs>--#### Response--- **200**: VM stopped successfully-- **400**: Stop operation failed-------## Image Management--### Available images from local cache--List available images from local cache--`GET: /lume/images`--#### Parameters--| Name | Type | Required | Description |-| ---- | ---- | -------- | ----------- |-| organization | string | No | Organization to list images for (default: trycua) |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl "http://localhost:7777/lume/images"-```- </Tab>- <Tab value="Python">-```python-import requests--response = requests.get("http://localhost:7777/lume/images")-print(response.json())-```- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/images`);-const data = await response.json();-```- </Tab>-</Tabs>--#### Response--- **200**: Success-- **400**: Failed to list images-------### The latest macOS restore image (IPSW) URL--Get the latest macOS restore image (IPSW) URL--`GET: /lume/ipsw`--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl "http://localhost:7777/lume/ipsw"-```- </Tab>- <Tab value="Python">-```python-import requests--response = requests.get("http://localhost:7777/lume/ipsw")-print(response.json())-```- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/ipsw`);-const data = await response.json();-```- </Tab>-</Tabs>--#### Response--- **200**: Success-- **400**: Failed to get IPSW URL-------### A VM image from a container registry--Pull a VM image from a container registry--`POST: /lume/pull`--#### Request Body--| Name | Type | Required | Description |-| ---- | ---- | -------- | ----------- |-| image | string | Yes | Image to pull (format: name:tag) |-| name | string | No | Name for the resulting VM |-| registry | string | No | Container registry URL (default: ghcr.io) |-| organization | string | No | Organization to pull from (default: trycua) |-| storage | string | No | VM storage location |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl -X POST "http://localhost:7777/lume/pull" \- -H "Content-Type: application/json" \- -d '{- "image": "macos-tahoe-vanilla:latest"-}'-```- </Tab>- <Tab value="Python">-```python-import requests--data = {- "image": "macos-tahoe-vanilla:latest",-}--response = requests.post("http://localhost:7777/lume/pull", json=data)-print(response.json())-```- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/pull`, {- method: "POST",- headers: { "Content-Type": "application/json" },- body: JSON.stringify({- image: "macos-tahoe-vanilla:latest",- }),-});-const data = await response.json();-```- </Tab>-</Tabs>--#### Response--- **200**: Image pulled successfully-- **400**: Pull operation failed-------### A VM image to a container registry--Push a VM image to a container registry--`POST: /lume/vms/push`--#### Request Body--| Name | Type | Required | Description |-| ---- | ---- | -------- | ----------- |-| name | string | Yes | Name of the local VM to push |-| imageName | string | Yes | Base name for the image in the registry |-| tags | array | Yes | List of tags to push |-| registry | string | No | Container registry URL (default: ghcr.io) |-| organization | string | No | Organization to push to (default: trycua) |-| storage | string | No | VM storage location |-| chunkSizeMb | integer | No | Chunk size for upload in MB (default: 512) |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl -X POST "http://localhost:7777/lume/vms/push" \- -H "Content-Type: application/json" \- -d '{- "name": "my-vm",- "imageName": "example",- "tags": [- "latest"- ]-}'-```- </Tab>- <Tab value="Python">-```python-import requests--data = {- "name": "my-vm",- "imageName": "example",- "tags": ["latest"],-}--response = requests.post("http://localhost:7777/lume/vms/push", json=data)-print(response.json())-```- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/vms/push`, {- method: "POST",- headers: { "Content-Type": "application/json" },- body: JSON.stringify({- name: "my-vm",- imageName: "example",- tags: ["latest"],- }),-});-const data = await response.json();-```- </Tab>-</Tabs>--#### Response--- **202**: Push initiated (async operation)-- **400**: Invalid request-------### Cached images to free up disk space--Remove cached images to free up disk space--`POST: /lume/prune`--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl -X POST "http://localhost:7777/lume/prune"-```- </Tab>- <Tab value="Python">-```python-import requests--response = requests.post("http://localhost:7777/lume/prune")-print(response.json())-```- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/prune`);-const data = await response.json();-```- </Tab>-</Tabs>--#### Response--- **200**: Images pruned successfully-- **400**: Prune operation failed-------## Configuration--### Current Lume configuration settings--Get current Lume configuration settings--`GET: /lume/config`--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl "http://localhost:7777/lume/config"-```- </Tab>- <Tab value="Python">-```python-import requests--response = requests.get("http://localhost:7777/lume/config")-print(response.json())-```- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/config`);-const data = await response.json();-```- </Tab>-</Tabs>--#### Response--- **200**: Success-- **400**: Failed to get config-------### Lume configuration settings--Update Lume configuration settings--`POST: /lume/config`--#### Request Body--| Name | Type | Required | Description |-| ---- | ---- | -------- | ----------- |-| homeDirectory | string | No | VM home directory path |-| cacheDirectory | string | No | Cache directory path |-| cachingEnabled | boolean | No | Enable or disable image caching |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl -X POST "http://localhost:7777/lume/config" \- -H "Content-Type: application/json" \- -d '{}'-```- </Tab>- <Tab value="Python">-```python-import requests--data = {-}--response = requests.post("http://localhost:7777/lume/config", json=data)-print(response.json())-```- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/config`, {- method: "POST",- headers: { "Content-Type": "application/json" },- body: JSON.stringify({- }),-});-const data = await response.json();-```- </Tab>-</Tabs>--#### Response--- **200**: Configuration updated successfully-- **400**: Invalid request-------### All VM storage locations--List all VM storage locations--`GET: /lume/config/locations`--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl "http://localhost:7777/lume/config/locations"-```- </Tab>- <Tab value="Python">-```python-import requests--response = requests.get("http://localhost:7777/lume/config/locations")-print(response.json())-```- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/config/locations`);-const data = await response.json();-```- </Tab>-</Tabs>--#### Response--- **200**: Success-- **400**: Failed to get locations-------### A new VM storage location--Add a new VM storage location--`POST: /lume/config/locations`--#### Request Body--| Name | Type | Required | Description |-| ---- | ---- | -------- | ----------- |-| name | string | Yes | Storage location name |-| path | string | Yes | Path to storage directory |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl -X POST "http://localhost:7777/lume/config/locations" \- -H "Content-Type: application/json" \- -d '{- "name": "my-vm",- "path": "/path/to/storage"-}'-```- </Tab>- <Tab value="Python">-```python-import requests--data = {- "name": "my-vm",- "path": "/path/to/storage",-}--response = requests.post("http://localhost:7777/lume/config/locations", json=data)-print(response.json())-```- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/config/locations`, {- method: "POST",- headers: { "Content-Type": "application/json" },- body: JSON.stringify({- name: "my-vm",- path: "/path/to/storage",- }),-});-const data = await response.json();-```- </Tab>-</Tabs>--#### Response--- **200**: Location added successfully-- **400**: Invalid request or location already exists-------### A VM storage location--Remove a VM storage location--`DELETE: /lume/config/locations/:name`--#### Parameters--| Name | Type | Required | Description |-| ---- | ---- | -------- | ----------- |-| name | string | Yes | Name of the location to remove |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl -X DELETE "http://localhost:7777/lume/config/locations/{name}"-```- </Tab>- <Tab value="Python">-```python-import requests--response = requests.delete("http://localhost:7777/lume/config/locations/{name}")-print(response.json())-```- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/config/locations/${name}`, {- method: "DELETE",-});-const data = await response.json();-```- </Tab>-</Tabs>--#### Response--- **200**: Location removed successfully-- **400**: Location not found or cannot be removed-------### The default VM storage location--Set the default VM storage location--`POST: /lume/config/locations/default/:name`--#### Parameters--| Name | Type | Required | Description |-| ---- | ---- | -------- | ----------- |-| name | string | Yes | Name of the location to set as default |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl -X POST "http://localhost:7777/lume/config/locations/default/{name}"-```- </Tab>- <Tab value="Python">-```python-import requests--response = requests.post("http://localhost:7777/lume/config/locations/default/{name}")-print(response.json())-```- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/config/locations/default/${name}`);-const data = await response.json();-```- </Tab>-</Tabs>--#### Response--- **200**: Default location set successfully-- **400**: Location not found-------## Logs--### Lume server logs--Retrieve Lume server logs--`GET: /lume/logs`--#### Parameters--| Name | Type | Required | Description |-| ---- | ---- | -------- | ----------- |-| type | string | No | Log type: 'info', 'error', or 'all' (default: all) |-| lines | integer | No | Number of lines to return from end of log |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl "http://localhost:7777/lume/logs"-```- </Tab>- <Tab value="Python">-```python-import requests--response = requests.get("http://localhost:7777/lume/logs")-print(response.json())-```- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/logs`);-const data = await response.json();-```- </Tab>-</Tabs>--#### Response--- **200**: Success-- **400**: Failed to read logs-----diff --git a/docs/content/docs/lume/reference/meta.json b/docs/content/docs/lume/reference/meta.jsondeleted file mode 100644index 8af3a48a14..0000000000--- a/docs/content/docs/lume/reference/meta.json+++ /dev/null@@ -1,6 +0,0 @@-{- "title": "Reference",- "description": "CLI and API reference documentation",- "icon": "FileText",- "pages": ["cli-reference", "http-api", "changelog"]-}diff --git a/docs/content/docs/lume/reference/v0.2/cli-reference.mdx b/docs/content/docs/lume/reference/v0.2/cli-reference.mdxdeleted file mode 100644index 2842b77473..0000000000--- a/docs/content/docs/lume/reference/v0.2/cli-reference.mdx+++ /dev/null@@ -1,295 +0,0 @@-----title: Lume CLI Reference-description: Command Line Interface reference for Lume------{/*-AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY-Generated by: npx tsx scripts/docs-generators/lume.ts-Source: libs/lume/src/Commands/*.swift-Version: 0.2.75-*/}--import { Callout } from 'fumadocs-ui/components/callout';-import { VersionHeader } from '@/components/version-selector';--<Callout type="warn">- This is documentation for **v0.2**. [View latest version](/lume/reference/cli-reference).-</Callout>--<div className="flex items-center gap-2 mb-6">- <span className="px-2 py-1 bg-amber-100 dark:bg-amber-900 text-amber-800 dark:text-amber-200 rounded text-sm font-mono">- v0.2.75- </span>- <span className="text-sm text-fd-muted-foreground">curl -fsSL .../install.sh | bash</span>-</div>--A lightweight CLI and local API server to build, run and manage macOS VMs.--## Quick Start--```bash-# Run a prebuilt macOS VM-lume run macos-sequoia-vanilla:latest--# Create a custom VM-lume create my-vm --cpu 4 --memory 8GB --disk-size 50GB-```--## VM Management--### lume create--Create a new virtual machine--**Arguments:**--- `<name>` - Name for the virtual machine--**Options:**--- `--os` - Operating system to install (macOS or linux) (default: macOS)-- `--cpu` - Number of CPU cores (default: 4)-- `--memory` - Memory size (e.g., 8GB) (default: 8GB)-- `--disk-size` - Disk size (e.g., 50GB) (default: 50GB)-- `--display` - Display resolution (e.g., 1024x768) (default: 1024x768)-- `--ipsw` - Path to IPSW file or 'latest' for macOS VMs-- `--storage` - VM storage location to use-- `--unattended` - [Preview] Preset name or path to YAML config file for unattended macOS Setup Assistant automation. Built-in presets: sequoia, tahoe. Only supported for macOS VMs.-- `--debug-dir` - Custom directory for debug screenshots during unattended setup (defaults to unique folder in system temp)-- `--vnc-port` - Port to use for the VNC server during unattended setup. Defaults to 0 (auto-assign) (default: 0)--**Flags:**--- `--debug` - Enable debug mode for unattended setup - saves screenshots with click coordinates-- `--no-display` - Do not open the VNC client during unattended setup (default: true for unattended)--### lume clone--Clone an existing virtual machine--**Arguments:**--- `<name>` - Name of the source VM-- `<new-name>` - Name for the cloned VM--**Options:**--- `--source-storage` - Source VM storage location-- `--dest-storage` - Destination VM storage location--### lume run--Run a virtual machine--**Arguments:**--- `<name>` - Name of the VM or image to run (format: name or name:tag)--**Options:**--- `--shared-dir` - Directory to share with the VM (format: path or path:ro or path:rw)-- `--mount` - For Linux VMs only, attach a read-only disk image-- `--usb-storage` - Disk image to attach as USB mass storage device-- `--registry` - Container registry URL (default: ghcr.io)-- `--organization` - Organization to pull from (default: trycua)-- `--vnc-port` - Port for VNC server (0 for auto-assign) (default: 0)-- `--recovery-mode` - For macOS VMs only, boot in recovery mode (default: false)-- `--storage` - VM storage location to use--**Flags:**--- `-d, --no-display` - Do not start the VNC client--### lume stop--Stop a virtual machine--**Arguments:**--- `<name>` - Name of the VM to stop--**Options:**--- `--storage` - VM storage location to use--### lume delete--Delete a virtual machine--**Arguments:**--- `<name>` - Name of the VM to delete--**Options:**--- `--storage` - VM storage location to use--**Flags:**--- `--force` - Force deletion without confirmation--## VM Information and Configuration--### lume get--Get detailed information about a virtual machine--**Arguments:**--- `<name>` - Name of the VM--**Options:**--- `-f, --format` - Output format (default: text)-- `--storage` - VM storage location to use--### lume set--Set new values for CPU, memory, and disk size of a virtual machine--**Arguments:**--- `<name>` - Name of the VM--**Options:**--- `--cpu` - New number of CPU cores-- `--memory` - New memory size (e.g., 8GB)-- `--disk-size` - New disk size (e.g., 100GB)-- `--display` - New display resolution-- `--storage` - VM storage location to use--### lume ls--List virtual machines--**Options:**--- `-f, --format` - Output format (json or text) (default: text)-- `--storage` - Filter by storage location name--## Image Management--### lume pull--Pull a macOS image from GitHub Container Registry--**Arguments:**--- `<image>` - Image to pull (format: name:tag)-- `<name>` - Name for the resulting VM (optional)--**Options:**--- `--registry` - Container registry URL (default: ghcr.io)-- `--organization` - Organization to pull from (default: trycua)-- `--storage` - VM storage location to use--### lume push--Push a macOS VM to GitHub Container Registry--**Arguments:**--- `<name>` - Name of VM to push-- `<image>` - Image tag (format: name:tag)--**Options:**--- `--additional-tags` - Additional tags to push-- `--registry` - Container registry URL (default: ghcr.io)-- `--organization` - Organization to push to (default: trycua)-- `--storage` - VM storage location to use-- `--chunk-size-mb` - Chunk size for upload in MB (default: 512)--**Flags:**--- `--verbose` - Enable verbose logging-- `--dry-run` - Prepare files without uploading-- `--reassemble` - Verify integrity by reassembling chunks--### lume images--List available macOS images from local cache--**Options:**--- `--organization` - Organization to list images for (default: trycua)--### lume ipsw--Get macOS restore image IPSW URL--### lume prune--Remove cached images--## Configuration and Server--### lume serve--Start the VM management server--**Options:**--- `--port` - Port to listen on (default: 7777)--### lume config--Get or set lume configuration--**Subcommands:**--- `lume config get` - Get current configuration-- `lume config storage` - Manage VM storage locations- - `lume config storage add` - Add a new VM storage location- - `lume config storage remove` - Remove a VM storage location- - `lume config storage list` - List all VM storage locations- - `lume config storage default` - Set the default VM storage location-- `lume config cache` - Manage cache settings- - `lume config cache get` - Get current cache directory- - `lume config cache set` - Set cache directory-- `lume config caching` - Manage image caching settings- - `lume config caching get` - Show current caching status- - `lume config caching set` - Enable or disable image caching--### lume logs--View lume serve logs--**Subcommands:**--- `lume logs info` - View info logs from the daemon- - `-n, --lines` - Number of lines to display-- `lume logs error` - View error logs from the daemon- - `-n, --lines` - Number of lines to display-- `lume logs all` - View both info and error logs- - `-n, --lines` - Number of lines to display--### lume setup--[Preview] Run unattended Setup Assistant automation on a macOS VM--**Arguments:**--- `<name>` - Name of the virtual machine--**Options:**--- `--unattended` - Preset name or path to YAML config file for unattended macOS Setup Assistant automation. Built-in presets: sequoia, tahoe.-- `--storage` - VM storage location to use or direct path to VM location-- `--vnc-port` - Port to use for the VNC server. Defaults to 0 (auto-assign) (default: 0)-- `--debug-dir` - Custom directory for debug screenshots (defaults to unique folder in system temp)--**Flags:**--- `--no-display` - Do not open the VNC client automatically-- `--debug` - Enable debug mode - saves screenshots with click coordinates--## Global Options--These options are available for all commands:--- `--help` - Show help information-- `--version` - Show version numberdiff --git a/docs/content/docs/lume/reference/v0.2/http-api.mdx b/docs/content/docs/lume/reference/v0.2/http-api.mdxdeleted file mode 100644index fa9715f6a3..0000000000--- a/docs/content/docs/lume/reference/v0.2/http-api.mdx+++ /dev/null@@ -1,1114 +0,0 @@-----title: Lume HTTP API Reference-description: HTTP API reference for Lume server------{/*-AUTO-GENERATED FILE - DO NOT EDIT DIRECTLY-Generated by: npx tsx scripts/docs-generators/lume.ts-Source: libs/lume/src/Server/*.swift-Version: 0.2.75-*/}--import { Callout } from 'fumadocs-ui/components/callout';-import { Tabs, Tab } from 'fumadocs-ui/components/tabs';-import { VersionHeader } from '@/components/version-selector';--<Callout type="warn">- This is documentation for **v0.2**. [View latest version](/lume/reference/cli-reference).-</Callout>--<div className="flex items-center gap-2 mb-6">- <span className="px-2 py-1 bg-amber-100 dark:bg-amber-900 text-amber-800 dark:text-amber-200 rounded text-sm font-mono">- v0.2.75- </span>- <span className="text-sm text-fd-muted-foreground">curl -fsSL .../install.sh | bash</span>-</div>--HTTP API for managing macOS and Linux virtual machines--## Default URL--```-http://localhost:7777-```--Start the server with `lume serve` or specify a custom port with `lume serve --port <port>`.--## VM Management--### All virtual machines--List all virtual machines--`GET: /lume/vms`--#### Parameters--| Name | Type | Required | Description |-| ------- | ------ | -------- | ------------------------------- |-| storage | string | No | Filter by storage location name |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl "http://localhost:7777/lume/vms"-```- </Tab>- <Tab value="Python">-```python-import requests--response = requests.get("http://localhost:7777/lume/vms")-print(response.json())--````- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/vms`);-const data = await response.json();-````-- </Tab>-</Tabs>--#### Response--- **200**: Success-- **400**: Bad request-------### Detailed information about a specific virtual machine--Get detailed information about a specific virtual machine--`GET: /lume/vms/:name`--#### Parameters--| Name | Type | Required | Description |-| ------- | ------ | -------- | -------------------------- |-| name | string | Yes | Name of the VM |-| storage | string | No | VM storage location to use |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl "http://localhost:7777/lume/vms/{name}"-```- </Tab>- <Tab value="Python">-```python-import requests--response = requests.get("http://localhost:7777/lume/vms/{name}")-print(response.json())--````- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/vms/${name}`);-const data = await response.json();-````-- </Tab>-</Tabs>--#### Response--- **200**: Success-- **400**: VM not found or invalid request-------### A new virtual machine--Create a new virtual machine--`POST: /lume/vms`--#### Request Body--| Name | Type | Required | Description |-| -------- | ------- | -------- | -------------------------------------------- |-| name | string | Yes | Name for the virtual machine |-| os | string | Yes | Operating system to install (macOS or linux) |-| cpu | integer | Yes | Number of CPU cores |-| memory | string | Yes | Memory size (e.g., 8GB) |-| diskSize | string | Yes | Disk size (e.g., 50GB) |-| display | string | Yes | Display resolution (e.g., 1024x768) |-| ipsw | string | No | Path to IPSW file or 'latest' for macOS VMs |-| storage | string | No | VM storage location to use |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl -X POST "http://localhost:7777/lume/vms" \- -H "Content-Type: application/json" \- -d '{- "name": "my-vm",- "os": "macOS",- "cpu": 4,- "memory": "8GB",- "diskSize": "50GB",- "display": "1024x768"-}'-```- </Tab>- <Tab value="Python">-```python-import requests--data = {-"name": "my-vm",-"os": "macOS",-"cpu": 4,-"memory": "8GB",-"diskSize": "50GB",-"display": "1024x768",-}--response = requests.post("http://localhost:7777/lume/vms", json=data)-print(response.json())--````- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/vms`, {- method: "POST",- headers: { "Content-Type": "application/json" },- body: JSON.stringify({- name: "my-vm",- os: "macOS",- cpu: 4,- memory: "8GB",- diskSize: "50GB",- display: "1024x768",- }),-});-const data = await response.json();-````-- </Tab>-</Tabs>--#### Response--- **200**: VM created successfully-- **400**: Invalid request body or VM creation failed-------### A virtual machine and its associated files--Delete a virtual machine and its associated files--`DELETE: /lume/vms/:name`--#### Parameters--| Name | Type | Required | Description |-| ------- | ------ | -------- | ------------------------ |-| name | string | Yes | Name of the VM to delete |-| storage | string | No | VM storage location |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl -X DELETE "http://localhost:7777/lume/vms/{name}"-```- </Tab>- <Tab value="Python">-```python-import requests--response = requests.delete("http://localhost:7777/lume/vms/{name}")-print(response.json())--````- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/vms/${name}`, {- method: "DELETE",-});-const data = await response.json();-````-- </Tab>-</Tabs>--#### Response--- **200**: VM deleted successfully-- **400**: VM not found or deletion failed-------### A copy of an existing virtual machine--Create a copy of an existing virtual machine--`POST: /lume/vms/clone`--#### Request Body--| Name | Type | Required | Description |-| -------------- | ------ | -------- | ------------------------------- |-| name | string | Yes | Name of the source VM |-| newName | string | Yes | Name for the cloned VM |-| sourceLocation | string | No | Source VM storage location |-| destLocation | string | No | Destination VM storage location |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl -X POST "http://localhost:7777/lume/vms/clone" \- -H "Content-Type: application/json" \- -d '{- "name": "my-vm",- "newName": "example"-}'-```- </Tab>- <Tab value="Python">-```python-import requests--data = {-"name": "my-vm",-"newName": "example",-}--response = requests.post("http://localhost:7777/lume/vms/clone", json=data)-print(response.json())--````- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/vms/clone`, {- method: "POST",- headers: { "Content-Type": "application/json" },- body: JSON.stringify({- name: "my-vm",- newName: "example",- }),-});-const data = await response.json();-````-- </Tab>-</Tabs>--#### Response--- **200**: VM cloned successfully-- **400**: Clone operation failed-------### Virtual machine configuration settings--Update virtual machine configuration settings--`PATCH: /lume/vms/:name`--#### Parameters--| Name | Type | Required | Description |-| ---- | ------ | -------- | ------------------------ |-| name | string | Yes | Name of the VM to update |--#### Request Body--| Name | Type | Required | Description |-| -------- | ------- | -------- | ---------------------------- |-| cpu | integer | No | New number of CPU cores |-| memory | string | No | New memory size (e.g., 16GB) |-| diskSize | string | No | New disk size (e.g., 100GB) |-| display | string | No | New display resolution |-| storage | string | No | VM storage location |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl -X PATCH "http://localhost:7777/lume/vms/{name}" \- -H "Content-Type: application/json" \- -d '{}'-```- </Tab>- <Tab value="Python">-```python-import requests--data = {-}--response = requests.patch("http://localhost:7777/lume/vms/{name}", json=data)-print(response.json())--````- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/vms/${name}`, {- method: "PATCH",- headers: { "Content-Type": "application/json" },- body: JSON.stringify({- }),-});-const data = await response.json();-````-- </Tab>-</Tabs>--#### Response--- **200**: Settings updated successfully-- **400**: Invalid settings or update failed-------### Start a virtual machine--Start a virtual machine--`POST: /lume/vms/:name/run`--#### Parameters--| Name | Type | Required | Description |-| ---- | ------ | -------- | ----------------------- |-| name | string | Yes | Name of the VM to start |--#### Request Body--| Name | Type | Required | Description |-| ----------------- | ------- | -------- | ----------------------------------------------- |-| noDisplay | boolean | No | Run without VNC display (default: false) |-| sharedDirectories | array | No | Directories to share with the VM |-| recoveryMode | boolean | No | Boot macOS VM in recovery mode (default: false) |-| storage | string | No | VM storage location |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl -X POST "http://localhost:7777/lume/vms/{name}/run" \- -H "Content-Type: application/json" \- -d '{}'-```- </Tab>- <Tab value="Python">-```python-import requests--data = {-}--response = requests.post("http://localhost:7777/lume/vms/{name}/run", json=data)-print(response.json())--````- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/vms/${name}/run`, {- method: "POST",- headers: { "Content-Type": "application/json" },- body: JSON.stringify({- }),-});-const data = await response.json();-````-- </Tab>-</Tabs>--#### Response--- **202**: VM start initiated (async operation)-- **400**: Invalid request or VM not found-------### A running virtual machine--Stop a running virtual machine--`POST: /lume/vms/:name/stop`--#### Parameters--| Name | Type | Required | Description |-| ---- | ------ | -------- | ---------------------- |-| name | string | Yes | Name of the VM to stop |--#### Request Body--| Name | Type | Required | Description |-| ------- | ------ | -------- | ------------------- |-| storage | string | No | VM storage location |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl -X POST "http://localhost:7777/lume/vms/{name}/stop" \- -H "Content-Type: application/json" \- -d '{}'-```- </Tab>- <Tab value="Python">-```python-import requests--data = {-}--response = requests.post("http://localhost:7777/lume/vms/{name}/stop", json=data)-print(response.json())--````- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/vms/${name}/stop`, {- method: "POST",- headers: { "Content-Type": "application/json" },- body: JSON.stringify({- }),-});-const data = await response.json();-````-- </Tab>-</Tabs>--#### Response--- **200**: VM stopped successfully-- **400**: Stop operation failed-------## Image Management--### Available images from local cache--List available images from local cache--`GET: /lume/images`--#### Parameters--| Name | Type | Required | Description |-| ------------ | ------ | -------- | ------------------------------------------------- |-| organization | string | No | Organization to list images for (default: trycua) |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl "http://localhost:7777/lume/images"-```- </Tab>- <Tab value="Python">-```python-import requests--response = requests.get("http://localhost:7777/lume/images")-print(response.json())--````- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/images`);-const data = await response.json();-````-- </Tab>-</Tabs>--#### Response--- **200**: Success-- **400**: Failed to list images-------### The latest macOS restore image (IPSW) URL--Get the latest macOS restore image (IPSW) URL--`GET: /lume/ipsw`--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl "http://localhost:7777/lume/ipsw"-```- </Tab>- <Tab value="Python">-```python-import requests--response = requests.get("http://localhost:7777/lume/ipsw")-print(response.json())--````- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/ipsw`);-const data = await response.json();-````-- </Tab>-</Tabs>--#### Response--- **200**: Success-- **400**: Failed to get IPSW URL-------### A VM image from a container registry--Pull a VM image from a container registry--`POST: /lume/pull`--#### Request Body--| Name | Type | Required | Description |-| ------------ | ------ | -------- | ------------------------------------------- |-| image | string | Yes | Image to pull (format: name:tag) |-| name | string | No | Name for the resulting VM |-| registry | string | No | Container registry URL (default: ghcr.io) |-| organization | string | No | Organization to pull from (default: trycua) |-| storage | string | No | VM storage location |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl -X POST "http://localhost:7777/lume/pull" \- -H "Content-Type: application/json" \- -d '{- "image": "macos-sequoia-vanilla:latest"-}'-```- </Tab>- <Tab value="Python">-```python-import requests--data = {-"image": "macos-sequoia-vanilla:latest",-}--response = requests.post("http://localhost:7777/lume/pull", json=data)-print(response.json())--````- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/pull`, {- method: "POST",- headers: { "Content-Type": "application/json" },- body: JSON.stringify({- image: "macos-sequoia-vanilla:latest",- }),-});-const data = await response.json();-````-- </Tab>-</Tabs>--#### Response--- **200**: Image pulled successfully-- **400**: Pull operation failed-------### A VM image to a container registry--Push a VM image to a container registry--`POST: /lume/vms/push`--#### Request Body--| Name | Type | Required | Description |-| ------------ | ------- | -------- | ------------------------------------------ |-| name | string | Yes | Name of the local VM to push |-| imageName | string | Yes | Base name for the image in the registry |-| tags | array | Yes | List of tags to push |-| registry | string | No | Container registry URL (default: ghcr.io) |-| organization | string | No | Organization to push to (default: trycua) |-| storage | string | No | VM storage location |-| chunkSizeMb | integer | No | Chunk size for upload in MB (default: 512) |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl -X POST "http://localhost:7777/lume/vms/push" \- -H "Content-Type: application/json" \- -d '{- "name": "my-vm",- "imageName": "example",- "tags": [- "latest"- ]-}'-```- </Tab>- <Tab value="Python">-```python-import requests--data = {-"name": "my-vm",-"imageName": "example",-"tags": ["latest"],-}--response = requests.post("http://localhost:7777/lume/vms/push", json=data)-print(response.json())--````- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/vms/push`, {- method: "POST",- headers: { "Content-Type": "application/json" },- body: JSON.stringify({- name: "my-vm",- imageName: "example",- tags: ["latest"],- }),-});-const data = await response.json();-````-- </Tab>-</Tabs>--#### Response--- **202**: Push initiated (async operation)-- **400**: Invalid request-------### Cached images to free up disk space--Remove cached images to free up disk space--`POST: /lume/prune`--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl -X POST "http://localhost:7777/lume/prune"-```- </Tab>- <Tab value="Python">-```python-import requests--response = requests.post("http://localhost:7777/lume/prune")-print(response.json())--````- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/prune`);-const data = await response.json();-````-- </Tab>-</Tabs>--#### Response--- **200**: Images pruned successfully-- **400**: Prune operation failed-------## Configuration--### Current Lume configuration settings--Get current Lume configuration settings--`GET: /lume/config`--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl "http://localhost:7777/lume/config"-```- </Tab>- <Tab value="Python">-```python-import requests--response = requests.get("http://localhost:7777/lume/config")-print(response.json())--````- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/config`);-const data = await response.json();-````-- </Tab>-</Tabs>--#### Response--- **200**: Success-- **400**: Failed to get config-------### Lume configuration settings--Update Lume configuration settings--`POST: /lume/config`--#### Request Body--| Name | Type | Required | Description |-| -------------- | ------- | -------- | ------------------------------- |-| homeDirectory | string | No | VM home directory path |-| cacheDirectory | string | No | Cache directory path |-| cachingEnabled | boolean | No | Enable or disable image caching |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl -X POST "http://localhost:7777/lume/config" \- -H "Content-Type: application/json" \- -d '{}'-```- </Tab>- <Tab value="Python">-```python-import requests--data = {-}--response = requests.post("http://localhost:7777/lume/config", json=data)-print(response.json())--````- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/config`, {- method: "POST",- headers: { "Content-Type": "application/json" },- body: JSON.stringify({- }),-});-const data = await response.json();-````-- </Tab>-</Tabs>--#### Response--- **200**: Configuration updated successfully-- **400**: Invalid request-------### All VM storage locations--List all VM storage locations--`GET: /lume/config/locations`--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl "http://localhost:7777/lume/config/locations"-```- </Tab>- <Tab value="Python">-```python-import requests--response = requests.get("http://localhost:7777/lume/config/locations")-print(response.json())--````- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/config/locations`);-const data = await response.json();-````-- </Tab>-</Tabs>--#### Response--- **200**: Success-- **400**: Failed to get locations-------### A new VM storage location--Add a new VM storage location--`POST: /lume/config/locations`--#### Request Body--| Name | Type | Required | Description |-| ---- | ------ | -------- | ------------------------- |-| name | string | Yes | Storage location name |-| path | string | Yes | Path to storage directory |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl -X POST "http://localhost:7777/lume/config/locations" \- -H "Content-Type: application/json" \- -d '{- "name": "my-vm",- "path": "/path/to/storage"-}'-```- </Tab>- <Tab value="Python">-```python-import requests--data = {-"name": "my-vm",-"path": "/path/to/storage",-}--response = requests.post("http://localhost:7777/lume/config/locations", json=data)-print(response.json())--````- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/config/locations`, {- method: "POST",- headers: { "Content-Type": "application/json" },- body: JSON.stringify({- name: "my-vm",- path: "/path/to/storage",- }),-});-const data = await response.json();-````-- </Tab>-</Tabs>--#### Response--- **200**: Location added successfully-- **400**: Invalid request or location already exists-------### A VM storage location--Remove a VM storage location--`DELETE: /lume/config/locations/:name`--#### Parameters--| Name | Type | Required | Description |-| ---- | ------ | -------- | ------------------------------ |-| name | string | Yes | Name of the location to remove |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl -X DELETE "http://localhost:7777/lume/config/locations/{name}"-```- </Tab>- <Tab value="Python">-```python-import requests--response = requests.delete("http://localhost:7777/lume/config/locations/{name}")-print(response.json())--````- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/config/locations/${name}`, {- method: "DELETE",-});-const data = await response.json();-````-- </Tab>-</Tabs>--#### Response--- **200**: Location removed successfully-- **400**: Location not found or cannot be removed-------### The default VM storage location--Set the default VM storage location--`POST: /lume/config/locations/default/:name`--#### Parameters--| Name | Type | Required | Description |-| ---- | ------ | -------- | -------------------------------------- |-| name | string | Yes | Name of the location to set as default |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl -X POST "http://localhost:7777/lume/config/locations/default/{name}"-```- </Tab>- <Tab value="Python">-```python-import requests--response = requests.post("http://localhost:7777/lume/config/locations/default/{name}")-print(response.json())--````- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/config/locations/default/${name}`);-const data = await response.json();-````-- </Tab>-</Tabs>--#### Response--- **200**: Default location set successfully-- **400**: Location not found-------## Logs--### Lume server logs--Retrieve Lume server logs--`GET: /lume/logs`--#### Parameters--| Name | Type | Required | Description |-| ----- | ------- | -------- | -------------------------------------------------- |-| type | string | No | Log type: 'info', 'error', or 'all' (default: all) |-| lines | integer | No | Number of lines to return from end of log |--#### Example Request--<Tabs groupId="language" persist items={['Curl', 'Python', 'TypeScript']}>- <Tab value="Curl">-```bash-curl "http://localhost:7777/lume/logs"-```- </Tab>- <Tab value="Python">-```python-import requests--response = requests.get("http://localhost:7777/lume/logs")-print(response.json())--````- </Tab>- <Tab value="TypeScript">-```typescript-const response = await fetch(`http://localhost:7777/lume/logs`);-const data = await response.json();-````-- </Tab>-</Tabs>--#### Response--- **200**: Success-- **400**: Failed to read logs-----diff --git a/docs/content/docs/lume/reference/v0.2/meta.json b/docs/content/docs/lume/reference/v0.2/meta.jsondeleted file mode 100644index a31a729a53..0000000000--- a/docs/content/docs/lume/reference/v0.2/meta.json+++ /dev/null@@ -1,5 +0,0 @@-{- "title": "v0.2",- "description": "Lume v0.2 Reference",- "pages": ["cli-reference", "http-api"]-}diff --git a/docs/content/docs/meta.json b/docs/content/docs/meta.jsonindex 4cd2e7bcab..e77424b694 100644--- a/docs/content/docs/meta.json+++ b/docs/content/docs/meta.json@@ -1,5 +1,5 @@{- "title": "Home",- "description": "Documentation Home",- "pages": ["cua-driver", "cua", "cuabench", "cuabot", "lume"]+ "title": "Documentation",+ "root": true,+ "pages": ["index", "---Learn---", "tutorials", "explanation", "---Do---", "how-to-guides", "reference"]}diff --git a/docs/content/docs/reference/cua-driver/cli-reference.mdx b/docs/content/docs/reference/cua-driver/cli-reference.mdxnew file mode 100644index 0000000000..4d7a59c030--- /dev/null+++ b/docs/content/docs/reference/cua-driver/cli-reference.mdx@@ -0,0 +1,380 @@+---+title: CLI Reference+description: Command-line interface specification for Cua Driver+---++Cross-platform computer-use automation driver. Install via the official script:++```sh+curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.sh | bash+```++Documented against Cua Driver **0.5.7**. Run `cua-driver --version` for your installed version.++---++## Tool dispatch++### `cua-driver list-tools`++List every registered MCP tool with a one-line description.++**Arguments:** none.++---++### `cua-driver describe`++Print a tool's full description and JSON input schema.++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `<tool-name>` | String | Yes | Name of the MCP tool to describe. |++---++### `cua-driver call`++Invoke an MCP tool directly from the shell. Runs the same handler the MCP server uses. JSON arguments may be passed as a positional JSON object or through stdin.++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `<tool-name>` | String | Yes | Name of the MCP tool to invoke. |+| `<json-args>` | String | No | JSON object for the tool input schema. If omitted, stdin is read when piped. |++**Options:**++| Option | Type | Default | Description |+| ------ | ---- | ------- | ----------- |+| `--screenshot-out-file` | String | — | Write the first image content block from the response to this path. |+| `--socket` | String | — | Override the daemon socket or named-pipe path. |++---++## Daemon management++### `cua-driver mcp`++Run the stdio MCP server. On macOS, shell-spawned MCP processes can auto-launch and proxy through a CuaDriver.app daemon so TCC grants attach to the bundle. On Windows and Linux, MCP proxies through an already-running daemon when one is listening.++**Options:**++| Option | Type | Default | Description |+| ------ | ---- | ------- | ----------- |+| `--socket` | String | — | Override the daemon socket or named-pipe path used by the proxy fallback. |++**Flags:**++| Flag | Description |+| ---- | ----------- |+| `--no-daemon-relaunch` | Stay in-process instead of proxying through a daemon. |+| `--claude-code-computer-use-compat` | Expose the Claude Code computer-use compatibility screenshot surface. |++---++### `cua-driver serve`++Run Cua Driver as a long-running daemon. The daemon owns per-process state such as element-index caches, recording state, and cursor overlay state.++**Options:**++| Option | Type | Default | Description |+| ------ | ---- | ------- | ----------- |+| `--socket` | String | — | Override the daemon socket or named-pipe path. |+| `--pid-file` | String | — | Override the pid-file path on Unix targets. |++**Flags:**++| Flag | Description |+| ---- | ----------- |+| `--no-permissions-gate` | Skip the macOS first-launch permissions gate. |++---++### `cua-driver stop`++Ask the running daemon to exit gracefully.++**Options:**++| Option | Type | Default | Description |+| ------ | ---- | ------- | ----------- |+| `--socket` | String | — | Override the daemon socket or named-pipe path. |++---++### `cua-driver status`++Report whether a Cua Driver daemon is running.++**Options:**++| Option | Type | Default | Description |+| ------ | ---- | ------- | ----------- |+| `--socket` | String | — | Override the daemon socket or named-pipe path. |+| `--pid-file` | String | — | Override the pid-file path on Unix targets. |++---++### `cua-driver mcp-config`++Print MCP server config or a client-specific install command.++Supported clients: `claude`, `codex`, `cursor`, `antigravity`, `openclaw`, `opencode`, `hermes`, `pi`.++**Options:**++| Option | Type | Default | Description |+| ------ | ---- | ------- | ----------- |+| `--client` | String | — | Client name to print configuration for. |++---++## Trajectory recording++### `cua-driver recording`++Control trajectory recording on a running daemon. Recording state lives in-process; use a daemon for multi-call sessions.++**Options:**++| Option | Type | Default | Description |+| ------ | ---- | ------- | ----------- |+| `--socket` | String | — | Override the daemon socket or named-pipe path. |++#### `cua-driver recording start`++Start trajectory recording to a directory.++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `<output-dir>` | String | Yes | Directory to write turn folders into. |++#### `cua-driver recording stop`++Stop trajectory recording.++#### `cua-driver recording status`++Print the current recording state.++#### `cua-driver recording render`++Render a recorded trajectory directory to an MP4. This pure file-to-file path does not require a running daemon.++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `<input-dir>` | String | Yes | Trajectory directory containing recorded turn folders. |+| `<out-mp4>` | String | Yes | Output MP4 path. |++**Options:**++| Option | Type | Default | Description |+| ------ | ---- | ------- | ----------- |+| `--scale` | Number | `2.0` | Scale factor for rendered frames. |++**Flags:**++| Flag | Description |+| ---- | ----------- |+| `--no-zoom` | Disable cursor/action zoom effects in the rendered video. |++---++## Configuration++### `cua-driver config`++Read or mutate persistent driver configuration. Without a subcommand, prints the full config.++**Options:**++| Option | Type | Default | Description |+| ------ | ---- | ------- | ----------- |+| `--socket` | String | — | Override the daemon socket or named-pipe path. |++#### `cua-driver config show`++Print the full config.++#### `cua-driver config get`++Print one config key.++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `<key>` | String | Yes | Config key to read. |++#### `cua-driver config set`++Set one config key.++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `<key>` | String | Yes | Config key to write. |+| `<value>` | String | Yes | Value to store. |++#### `cua-driver config reset`++Reset config to defaults.++---++## Diagnostics++### `cua-driver check-update`++Check whether a newer Cua Driver release is available. Read-only. Uses the same update-state payload as the `check_for_update` MCP tool.++**Flags:**++| Flag | Description |+| ---- | ----------- |+| `--json` | Emit a machine-readable JSON payload. |+| `--no-cache` | Skip the 20-hour on-disk cache and force a GitHub request. |++---++### `cua-driver update`++Check for an update and optionally apply it. The apply path delegates to the canonical platform installer scripts.++**Flags:**++| Flag | Description |+| ---- | ----------- |+| `--apply` | Download and install the latest release when one is available. |+| `--json` | Emit the structured update-state payload. |++---++### `cua-driver doctor`++Run platform-aware diagnostic probes. Exit code is non-zero when any probe is an error.++**Flags:**++| Flag | Description |+| ---- | ----------- |+| `--json` | Emit the probe report as JSON. |++---++### `cua-driver diagnose`++Print a pasteable install-layout and permission-attribution report.++---++## Other commands++### `cua-driver autostart`++Manage platform-native daemon autostart. Windows registers a logon Scheduled Task. macOS and Linux currently print manual-recipe guidance.++#### `cua-driver autostart enable`++Register the autostart entry.++#### `cua-driver autostart disable`++Remove the autostart entry.++#### `cua-driver autostart status`++Print whether autostart is registered and running.++#### `cua-driver autostart kick`++Start the autostart entry now without re-logging.++---++### `cua-driver skills`++Install, update, inspect, or remove the optional agent skill pack. The install script never touches agent skill directories automatically.++#### `cua-driver skills install`++Fetch the versioned skill pack and link detected agents.++**Options:**++| Option | Type | Default | Description |+| ------ | ---- | ------- | ----------- |+| `--agent` | String | — | Restrict linking to one agent. |+| `--from` | String | — | Fetch from a source such as `main` instead of the tagged release. |++**Flags:**++| Flag | Description |+| ---- | ----------- |+| `--all-platforms` | Keep platform-specific skill files for every platform. |++#### `cua-driver skills update`++Refresh the local skill pack and links.++#### `cua-driver skills uninstall`++Remove agent skill links.++**Flags:**++| Flag | Description |+| ---- | ----------- |+| `--all` | Also delete the local skill-pack copy. |++#### `cua-driver skills status`++Report local skill-pack and per-agent link state.++#### `cua-driver skills path`++Print the local skill-pack path.++---++### `cua-driver dump-docs`++Output machine-readable CLI and MCP documentation JSON. Used by the docs generator to keep reference pages in sync with the live binary.++**Options:**++| Option | Type | Default | Description |+| ------ | ---- | ------- | ----------- |+| `--type` | String | `all` | Which docs to emit: `all`, `cli`, or `mcp`. |++**Flags:**++| Flag | Description |+| ---- | ----------- |+| `-p`, `--pretty` | Pretty-print JSON. |++---++### `cua-driver permissions` (macOS)++Inspect or request the macOS TCC grants the driver needs (Accessibility and Screen Recording).++```bash+cua-driver permissions status # report grant status; read-only, no prompt+cua-driver permissions grant # launch CuaDriver via LaunchServices so the prompt attributes to the app+```++**Flags:**++| Flag | Description |+| ---- | ----------- |+| `--json` | Machine-readable status output. |++---++## Global options++Available on all commands:++| Option | Description |+| ------ | ----------- |+| `--help` | Show help information. |+| `--version` | Show version number. |diff --git a/docs/content/docs/reference/cua-driver/limits.mdx b/docs/content/docs/reference/cua-driver/limits.mdxnew file mode 100644index 0000000000..87390e4a80--- /dev/null+++ b/docs/content/docs/reference/cua-driver/limits.mdx@@ -0,0 +1,89 @@+---+title: Known Limits+description: Documented behavioral limits of Cua Driver and available workarounds+---++import { Callout } from 'fumadocs-ui/components/callout';++Cua Driver's *no-foreground contract* holds for every app it reaches via AX or via the SkyLight-routed pixel click. Four categories of target fall outside that envelope.++---++## Chromium coerces synthetic right-clicks on web content++**Symptom:** `right_click({pid, x, y})` on a Chrome, Edge, Brave, or Arc tab's web content fires a left-click instead of opening the context menu.++**Cause:** Chromium's renderer-IPC filter drops the right-click subtype bit on events that don't come through the HID tap. Every synthesized-event path on macOS hits this wall.++**Workarounds, in order of preference:**++1. Use `right_click({pid, element_index})` on AX-addressable targets (links, buttons, toolbar items). AX dispatch sidesteps the renderer filter entirely.+2. For context menus on pure web content (nothing in the AX tree), activate Chrome briefly and fall back to a HID-tap right-click. This breaks the no-foreground-steal promise for that one click.++<Callout type="info">+ Element-indexed right-click (`right_click` with `element_index`) works fine. The limit is+ specifically pixel right-click on non-AX Chromium web content.+</Callout>++---++## Canvas apps need brief frontmost activation++**Affected:** Blender (GHOST event source), Unity editor / Unity games, most native games, some WebGL-heavy Electron apps.++**Symptom:** `click({pid, x, y})` on a Blender viewport silently no-ops. The window is visible and `launch_app` works, but clicks vanish.++**Cause:** These apps only accept events from `cghidEventTap` with a leading `mouseMoved`. They explicitly filter out per-pid-routed events, which is the path Cua Driver uses for backgrounded dispatch. There is no per-pid recipe that reaches them.++**Workaround:** Bring the app to the foreground before clicking, then use pixel `click({pid, x, y})`. Where the target exposes AX-addressable controls, prefer `right_click` or element actions, which sidestep the renderer filter without foregrounding.++<Callout type="warn">+ When automating Blender or a native game, the no-foreground-steal contract does not apply: these apps must be foregrounded to receive clicks, so do this only when the user is not actively working on the machine.+</Callout>++---++## Off-Space SwiftUI windows strip their AX tree++**Symptom:** `get_window_state({pid, window_id})` on a window on a different Space (e.g. System Settings parked on Space 2 while on Space 1) returns a minimal tree that contains only the menu bar, or just the `AXApplication` root.++**Cause:** macOS 14+ strips AX detail from non-current-Space SwiftUI windows as a privacy/performance tradeoff. AppKit apps are not affected. There is no workaround that keeps the window off-Space.++**Response shape:** Every `get_window_state` response on an off-current-Space window carries `off_space: true`, so callers can decide to switch Space, pick a different window, or skip the turn.++**Workarounds:**++1. Switch the user to the target's Space first. This breaks the no-Space-bounce promise.+2. Target an AppKit equivalent of the app if one exists.+3. Limit off-Space automation to AppKit apps where the tree stays populated.++---++## Minimized windows silently drop keyboard commits++**Symptom:** `press_key({pid, element_index, key: "return"})` on a text field in a minimized window returns success, but the field doesn't commit. The macOS system-alert beep fires, or nothing happens.++**Cause:** AX reads and AX-dispatched clicks propagate through to minimized windows normally, but keyboard-commit events (Return, Space, Tab) require renderer focus, which AX focus does not confer on a minimized window. This is a macOS-wide behavior.++**Workarounds:**++1. Use `set_value({pid, element_index, value: "..."})` to write the field's value directly. No keyboard event involved; no focus handoff required.+2. AX-click a commit-equivalent button (Go, Submit, Send, OK) rather than relying on Return.+3. Un-minimize the window (`hotkey({pid, keys: ["cmd", "m"]})` or click the Dock icon). This breaks the background contract for that window.++<Callout type="info">+ `set_value` is the correct approach 90% of the time. It sidesteps both the minimized-focus issue and the general "which event commits this field" ambiguity.+</Callout>++---++## Permission boundaries++Cua Driver is constrained by the macOS permission model. Two relevant grants:++| Grant | Required for |+| ----- | ------------ |+| **Accessibility** (System Settings → Privacy & Security → Accessibility) | Every AX read, every element-indexed click, every keyboard/text primitive. Without it, `check_permissions` returns `accessibility: false` and every tool returns a structured error. |+| **Screen Recording** | Screenshots and the `som`/`vision` capture modes. Without it, `get_window_state` in those modes returns a tree but no PNG. Pure `ax` mode still works. |++Grants are tied to the `CuaDriver.app` bundle identity (`com.trycua.driver`). Rebuilding Cua Driver from source preserves the grants because `build-app.sh` pins a stable bundle id.diff --git a/docs/content/docs/reference/cua-driver/mcp-tools.mdx b/docs/content/docs/reference/cua-driver/mcp-tools.mdxnew file mode 100644index 0000000000..7ab5babf10--- /dev/null+++ b/docs/content/docs/reference/cua-driver/mcp-tools.mdx@@ -0,0 +1,643 @@+---+title: MCP Tools+description: Specification for every MCP tool exposed by Cua Driver+---++import { Callout } from 'fumadocs-ui/components/callout';++Cua Driver exposes 36 MCP tools through a single stdio server (`cua-driver mcp`). Every tool is also callable from the shell as `cua-driver call <tool-name> '<JSON-args>'`.++Tool names are `snake_case`. Responses are MCP `CallTool.Result` envelopes: a text content block prefixed with a `✅` summary (or the error reason on failure), plus optional image or structured-content blocks on tools that produce them.++<Callout type="info">+ Tool names match the CLI dispatch form exactly. `cua-driver call list_apps '{}'` and the MCP `list_apps` tool run the same code path.+</Callout>++<Callout type="info">+ **TCC auto-delegation.** When an MCP client spawns `cua-driver mcp` from an IDE terminal (Claude Code, Cursor, VS Code, Warp), macOS attributes the subprocess to the parent terminal, not `CuaDriver.app`, so AX probes fail against the wrong bundle id. `mcp` detects this, auto-launches a `cua-driver serve` daemon via `open -n -g -a CuaDriver --args serve`, and proxies every tool call through the daemon's Unix socket. Tool semantics are identical to the in-process path. Pass `--no-daemon-relaunch` (or set `CUA_DRIVER_MCP_NO_RELAUNCH=1`) to force in-process execution.+</Callout>++---++## Inspection tools++### `list_apps`++List macOS apps, both currently running and installed-but-not-running, with per-app state flags.++Per-record fields:++| Field | Description |+| ----- | ----------- |+| `running` | Whether a process for this app is live. `pid` is 0 when false. |+| `active` | Whether it is the system-frontmost app (implies `running`). |+| `launch_path` | Filesystem path to the `.app` bundle, when known. |+| `kind` | `"desktop"` for `.app` bundles on macOS. |+| `last_used` | RFC 3339 timestamp from the bundle's filesystem mtime, or null. |++Only apps with `NSApplicationActivationPolicyRegular` are included. Installed apps are scanned from `/Applications`, `/Applications/Utilities`, `~/Applications`, `/System/Applications`, and `/System/Applications/Utilities`.++**Arguments:** none.++---++### `list_windows`++List all layer-0 top-level windows currently known to WindowServer, including off-screen windows (minimized, on another Space, hidden-launched).++Per-record fields: `window_id`, `pid`, `app_name`, `title`, `bounds` (x/y/width/height, top-left origin), `z_index` (higher = frontmost), `is_on_screen`, `on_current_space`.++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `on_screen_only` | boolean | No | When true, drop windows not on the current Space. Default false. |+| `pid` | integer | No | Optional pid filter. When set, only this pid's windows are returned. |++---++### `get_window_state`++Walk a running app's AX tree and return a Markdown rendering of its UI, tagging every actionable element with `[element_index N]`. Also captures a PNG screenshot of the specified window.++**Invariant:** call `get_window_state` once per turn per `(pid, window_id)` before any element-indexed action. The index map is replaced by the next snapshot.++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `pid` | integer | Yes | Target process ID. |+| `window_id` | integer | Yes | Target window ID from `list_windows`. |+| `capture_mode` | string | No | `som` = AX + screenshot (default); `vision` = screenshot only; `ax` = AX only. |+| `query` | string | No | Case-insensitive filter for `tree_markdown`. Element indices are unchanged. |+| `screenshot_out_file` | string | No | When set, write the PNG to this file path (`~` expanded) instead of embedding base64 in the response. |++```json+{"pid": 844, "window_id": 10725}+```++---++### `get_accessibility_tree`++Return a lightweight snapshot of the desktop: running regular apps and on-screen visible windows with their bounds, z-order, and owner pid. No TCC grants required.++**Arguments:** none.++---++### `get_screen_size`++Return the logical size of the main display in points plus its backing scale factor. Agents click in points; Retina displays have `scale_factor` 2.0. Requires no TCC permissions.++**Arguments:** none.++---++### `get_cursor_position`++Return the current mouse cursor position in screen points (origin top-left).++**Arguments:** none.++---++### `get_config`++Return the current cua-driver-rs configuration.++**Arguments:** none.++---++### `get_recording_state`++Report the current trajectory recorder state: whether recording is enabled, the output directory (when enabled), and the 1-based counter for the next turn folder. Counter increments on every recorded action tool call and resets to 1 each time recording is (re-)enabled. Pure read-only.++**Arguments:** none.++---++### `get_agent_cursor_state`++Return the current state of this session's agent cursor: position, config (color, icon, label, size, opacity), enabled flag. Scoped to the cursor the call resolves to (precedence: explicit `cursor_id` > session identity > `"default"`).++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `cursor_id` | string | No | Cursor instance to inspect. Omit to target the calling session's own cursor (macOS); the anonymous path targets `"default"`. |++---++## Action tools++### `launch_app`++Launch a macOS app in the background. The target does NOT come to the foreground.++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `bundle_id` | string | No | App bundle identifier, e.g. `com.apple.calculator`. Preferred over `name`. |+| `name` | string | No | App display name. Used only when `bundle_id` is absent. |+| `urls` | array of string | No | File paths or URLs to open with the app. |+| `electron_debugging_port` | integer | No | Open a Chrome DevTools Protocol server on this port (appends `--remote-debugging-port=N`). |+| `webkit_inspector_port` | integer | No | Open a WebKit inspector server on this port (sets `WEBKIT_INSPECTOR_SERVER` env var). |+| `creates_new_application_instance` | boolean | No | When true, force a new app instance even if already running (`open -n`). Use for concurrent multi-agent/multi-session work. |+| `additional_arguments` | array of string | No | Extra arguments appended after `--args` when launching. |++Returns the launched app's `pid`, `bundle_id`, `name`, and a `windows` array (same shape as `list_windows`). When the focus-steal demotion check ran, the response also includes `self_activation_suppressed: bool`.++---++### `kill_app`++Force-terminate a process by pid (kill -9 equivalent on macOS / Linux; `taskkill /F` equivalent on Windows). Unsaved state is lost.++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `pid` | integer | Yes | PID of the process to terminate. |++```json+{"pid": 844}+```++---++### `bring_to_front`++Activate a window so subsequent input tools with `dispatch:"foreground"` land on it without a per-call `SetForegroundWindow` flash. Windows-only. On macOS this tool returns an error. On Linux this tool stubs out.++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `pid` | integer | Yes | Target process ID. |+| `window_id` | integer | No | Optional target window ID. |++```json+{"pid": 844}+```++---++### `click`++Left-click against a target pid.++Two addressing modes:++- **Element index** (`element_index` + `window_id`): AX action path. Works on backgrounded/hidden windows. No cursor move, no focus steal. Cache is scoped per `(pid, window_id)` and is replaced by the next snapshot.+- **Pixel** (`x`, `y`): CGEvent path. Synthesizes mouse events posted to pid. Needs a visible on-screen window.++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `pid` | integer | Yes | Target process ID. |+| `element_index` | integer | No | Element index from last `get_window_state`. |+| `window_id` | integer | No | Target window ID. Required for `element_index`. |+| `x` | number | No | Window-local screenshot X coordinate. |+| `y` | number | No | Window-local screenshot Y coordinate. |+| `action` | string | No | AX action: `press` (default), `show_menu`, `pick`, `confirm`, `cancel`, `open`. |+| `count` | integer | No | Click count (pixel path only). Default 1. |+| `modifier` | array of string | No | Modifier keys: `cmd`, `shift`, `option`/`alt`, `ctrl`. |+| `from_zoom` | boolean | No | When true, `x` and `y` are in the last zoom image for this pid; driver translates back to full-window coordinates. |+| `debug_image_out` | string | No | File path for a diagnostic screenshot with a red crosshair at (x, y). |++```json+{"pid": 844}+```++---++### `double_click`++Double-click at `(x, y)` or on an AX element identified by `element_index` + `window_id`.++AX path: performs `AXOpen` when the element advertises it; otherwise resolves the element's on-screen center and falls back to a pixel double-click.++Pixel path: two down/up pairs ~80 ms apart.++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `pid` | integer | Yes | Target process ID. |+| `element_index` | integer | No | Element index from last `get_window_state`. Uses AX path. |+| `window_id` | integer | No | CGWindowID. Required when `element_index` is used. |+| `x` | number | No | Screen X coordinate (pixel path). |+| `y` | number | No | Screen Y coordinate (pixel path). |++```json+{"pid": 844}+```++---++### `right_click`++Right-click against a target pid.++Two addressing modes:++- **Element index** (`element_index` + `window_id`): performs `AXShowMenu`. Pure AX RPC, works on backgrounded/hidden windows.+- **Pixel** (`x`, `y`): synthesizes `rightMouseDown`/`rightMouseUp` CGEvent pair posted to the pid.++Exactly one of `element_index` or (`x` AND `y`) must be provided. `pid` always required.++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `pid` | integer | Yes | Target process ID. |+| `element_index` | integer | No | Element index from last `get_window_state`. Routes through `AXShowMenu`. Requires `window_id`. |+| `window_id` | integer | No | CGWindowID. Required when `element_index` is used. |+| `x` | number | No | X in window-local screenshot pixels. |+| `y` | number | No | Y in window-local screenshot pixels. |+| `modifier` | array of string | No | Modifier keys held during right-click: `cmd`/`shift`/`option`/`ctrl`. Pixel path only. |++```json+{"pid": 844}+```++---++### `drag`++Press-drag-release gesture from `(from_x, from_y)` to `(to_x, to_y)` in window-local screenshot pixels.++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `pid` | integer | Yes | Target process ID. |+| `from_x` | number | Yes | Drag-start X in window-local screenshot pixels. |+| `from_y` | number | Yes | Drag-start Y in window-local screenshot pixels. |+| `to_x` | number | Yes | Drag-end X in window-local screenshot pixels. |+| `to_y` | number | Yes | Drag-end Y in window-local screenshot pixels. |+| `window_id` | integer | No | CGWindowID for the window the pixel coordinates were measured against. |+| `duration_ms` | integer | No | Wall-clock duration of the drag path between mouseDown and mouseUp. Default 500. |+| `steps` | integer | No | Number of intermediate mouseDragged events. Default 20. |+| `modifier` | array of string | No | Modifier keys held across the entire gesture: `cmd`/`shift`/`option`/`ctrl`. |+| `button` | string | No | Mouse button used for the drag. Default: `left`. |+| `from_zoom` | boolean | No | When true, coordinates are in the last zoom image for this pid. |++```json+{"from_x": 100, "from_y": 200, "pid": 844, "to_x": 300, "to_y": 400}+```++---++### `type_text`++Insert text into the target pid via `AXSetAttribute(kAXSelectedText)`. Works for standard Cocoa text fields and text views. No keystrokes are synthesized. For Chromium/Electron inputs that don't implement `kAXSelectedText`, the tool falls back to CGEvent character synthesis automatically.++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `pid` | integer | Yes | Target process ID. |+| `text` | string | Yes | Text to insert at the target's cursor. |+| `element_index` | integer | No | Element index from last `get_window_state`. Directs the write to a specific field. Requires `window_id`. |+| `window_id` | integer | No | CGWindowID. Required when `element_index` is used. |+| `delay_ms` | integer | No | Milliseconds between characters in the CGEvent fallback path. Default 30. Ignored when the AX path succeeds. |++```json+{"pid": 844, "text": "hello"}+```++---++### `press_key`++Press and release a single key, delivered to the target pid via `CGEventPostToPid`. No focus steal.++Three delivery paths:+- `window_id` + `element_index`: focuses the AX element first, then posts via the auth-message path (Chromium-safe).+- `window_id` only (no `element_index`): NSMenu path. Briefly activates the window, posts without the auth envelope so `IOHIDPostEvent` fires and `NSApplication.sendEvent:` dispatches NSMenu key equivalents. Restores prior frontmost immediately.+- No `window_id`: standard auth-message path.++Key names: `return`, `tab`, `escape`, `up`/`down`/`left`/`right`, `space`, `delete`, `home`, `end`, `pageup`, `pagedown`, `f1`–`f12`, plus any letter or digit.++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `pid` | integer | Yes | Target process ID. |+| `key` | string | Yes | Key name: `return`, `tab`, `escape`, `up`, `down`, etc. |+| `modifiers` | array of string | No | Modifier keys: `cmd`, `shift`, `option`/`alt`, `ctrl`, `fn`. |+| `element_index` | integer | No | Element index. |+| `window_id` | integer | No | Target window ID. |++```json+{"key": "return", "pid": 844}+```++---++### `hotkey`++Press a combination of keys simultaneously. The combo is posted directly to the target pid's event queue; the target does NOT need to be frontmost.++Two delivery paths:+- Default (no `window_id`): auth-message envelope. Chromium/Electron apps accept the keystrokes as trusted live input on macOS 14+.+- With `window_id`: NSMenu path. Briefly activates the target WindowServer-frontmost, posts without the auth envelope so `IOHIDPostEvent` fires and NSMenu key equivalents dispatch (e.g. Cmd+Z undo, Cmd+W close). Restores prior frontmost immediately.++Recognized modifiers: `cmd`/`command`, `shift`, `option`/`alt`, `ctrl`/`control`, `fn`. Order: modifiers first, one non-modifier last.++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `pid` | integer | Yes | Target process ID. |+| `keys` | array of string | Yes | Modifier(s) and one non-modifier key, e.g. `["cmd", "c"]`. |+| `window_id` | integer | No | When set, uses NSMenu path. |++```json+{"keys": ["cmd", "c"], "pid": 844}+```++---++### `set_value`++Set a value on a UI element.++Two modes:++- **`AXPopUpButton` / select dropdown**: finds the child option whose title or value matches `value` (case-insensitive) and `AXPress`es it directly. No native popup menu is opened.+- **All other elements**: writes `AXValue` directly (sliders, steppers, date pickers, native text fields that expose a settable `AXValue`).++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `pid` | integer | Yes | Target process ID. |+| `element_index` | integer | Yes | Element index from last `get_window_state`. |+| `window_id` | integer | Yes | CGWindowID for the window whose `get_window_state` produced the element index. |+| `value` | string | Yes | New value. AX will coerce to the element's native type. |++```json+{"element_index": 14, "pid": 844, "value": "42", "window_id": 10725}+```++---++### `scroll`++Scroll the target pid's focused region by synthesized keystrokes.++Mapping: `by='page'` → PageDown/PageUp × amount; `by='line'` → DownArrow/UpArrow × amount. Horizontal variants use Left/Right arrow keys.++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `pid` | integer | Yes | Target process ID. |+| `direction` | string | Yes | Scroll direction: `up`, `down`, `left`, `right`. |+| `by` | string | No | Scroll granularity: `line` (default) or `page`. |+| `amount` | integer | No | Number of keystroke repetitions. Default 3. |+| `element_index` | integer | No | Pre-focuses this element before scrolling. |+| `window_id` | integer | No | Required when `element_index` is set. |++```json+{"direction": "down", "pid": 844}+```++---++### `move_cursor`++Move the agent cursor overlay to `(x, y)`. Does NOT move the real mouse cursor.++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `x` | number | Yes | Target X coordinate in screen points. |+| `y` | number | Yes | Target Y coordinate in screen points. |+| `cursor_id` | string | No | Explicit cursor-instance override. Omit to target the calling session's own cursor. |++```json+{"x": 100, "y": 200}+```++---++### `zoom`++Capture a cropped JPEG of a window region `(x1, y1)`–`(x2, y2)` in screenshot pixel coordinates, with 20% padding added on each side. The output image is at most 500 px wide.++After a zoom, pass `from_zoom=true` to `click`/`type_text` to auto-translate coordinates back to full-window space.++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `window_id` | integer | Yes | CGWindowID from `list_windows`. |+| `x1` | number | Yes | Left edge of region in screenshot pixels. |+| `y1` | number | Yes | Top edge of region in screenshot pixels. |+| `x2` | number | Yes | Right edge of region in screenshot pixels. |+| `y2` | number | Yes | Bottom edge of region in screenshot pixels. |+| `pid` | integer | No | Target pid, required for `from_zoom` click/type translation. |++```json+{"window_id": 10725, "x1": 100, "y1": 200, "x2": 400, "y2": 500}+```++---++## Browser tools++### `page`++Interact with the browser page loaded in a running app. Supports Chrome, Brave, Edge, Safari (via AppleScript on macOS), Electron apps (via CDP), Chromium/Firefox on Windows (via UIA for read; CDP for `execute_javascript` when `--remote-debugging-port` is set), and WKWebView/Tauri/AT-SPI fallbacks.++**Actions:**++| Action | Description |+| ------ | ----------- |+| `execute_javascript` | Run JS and return the result. |+| `get_text` | Extract visible text from the page. |+| `query_dom` | Find elements matching a CSS selector. |+| `click_element` | Click a CSS-selected element and animate the agent cursor to its on-screen center. |+| `enable_javascript_apple_events` | macOS-only: patch the browser's Preferences to allow JS from Apple Events. Requires `user_has_confirmed_enabling: true` and a browser restart. |++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `action` | string | Yes | Action to perform (see table above). |+| `pid` | integer | No | Target process ID. |+| `window_id` | integer | No | Target window ID from `list_windows`. |+| `javascript` | string | No | JavaScript to execute. Required for `execute_javascript`. |+| `css_selector` | string | No | CSS selector for `query_dom`. |+| `attributes` | array of string | No | Element attributes to include in `query_dom` results. |+| `selector` | string | No | CSS selector for `click_element`. |+| `bundle_id` | string | No | Bundle ID of the browser. Required for `enable_javascript_apple_events` (macOS only). |+| `user_has_confirmed_enabling` | boolean | No | Must be true to proceed with `enable_javascript_apple_events`. |++```json+{"action": "get_text"}+```++---++## Recording tools++### `start_recording`++Start trajectory recording. Every subsequent action-tool invocation writes a turn folder under `output_dir`.++Turn folder contents:++| File | Description |+| ---- | ----------- |+| `app_state.json` | Post-action AX/UIA snapshot for the target pid. |+| `screenshot.png` | Post-action per-window screenshot of the target's frontmost on-screen window. |+| `action.json` | Tool name, full input arguments, result summary, pid, click point (when applicable), ISO-8601 timestamp. |+| `click.png` | For click-family actions only: `screenshot.png` with a red dot drawn at the click point. |++Turn folders are named `turn-00001/`, `turn-00002/`, etc. Numbering restarts at 1 each time recording is (re-)started.++**Video recording:**++- Default: off. Pass `record_video: true` to also capture the main display to `<output_dir>/recording.mp4` (H.264, 30 fps).+- On macOS: uses native ScreenCaptureKit (requires macOS 15.0+, no extra TCC prompt).+- On Windows + Linux: uses an ffmpeg subprocess (`gdigrab` / `x11grab` + libx264). Requires ffmpeg on PATH.+- Recording stops automatically when the session ends on the daemon-proxy path.++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `output_dir` | string | Yes | Absolute or `~`-rooted directory where turn folders and the video file are written. |+| `record_video` | boolean | No | Capture the main display to `<output_dir>/recording.mp4`. Default false. |++```json+{"output_dir": "~/cua-trajectories/demo1"}+```++---++### `stop_recording`++Stop trajectory recording. When video was enabled, gracefully terminates the ffmpeg subprocess so the mp4's `moov` atom is finalized (file is playable). Calling stop on an already-stopped session is a no-op. The response carries `last_video_path` pointing at the finalized mp4 when video was on.++A manual `stop_recording` is unconditional. It stops whatever recording is active regardless of which session started it.++**Arguments:** none.++---++### `replay_trajectory`++Replay a recorded trajectory by re-invoking every turn's tool call in lexical order. `dir` must point at a directory previously written by `start_recording`.++Caveats:+- Element-indexed actions fail because element indices are per-snapshot. Pixel clicks and keyboard tools replay cleanly.+- `get_window_state` and other read-only tools are not recorded, so replays do not re-populate the element cache.+- When recording is enabled while replay runs, the replay itself is recorded into the currently configured output directory.++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `dir` | string | Yes | Trajectory directory previously written by `start_recording`. Absolute or `~`-rooted. |+| `delay_ms` | integer | No | Milliseconds to sleep between turns. Default 500. |+| `stop_on_error` | boolean | No | Stop replay on the first tool-call error. Default true. |++```json+{"dir": "~/cua-trajectories/demo1"}+```++---++## Configuration tools++### `set_config`++Update cua-driver-rs configuration. Changes to `capture_mode` and `max_image_dimension` take effect immediately. The `experimental_pip` keys are persisted to `~/.cua-driver/config.json` and take effect on the next daemon restart.++**Per-session isolation (daemon-proxy path).** On the daemon-proxy path, `set_config` writes an in-memory, session-scoped override that does not touch the global `DriverConfig` or persist to disk. `get_config` and capture tools resolve effective values as: call-arg > session override > global default. The override is dropped automatically when the client disconnects. Only the anonymous path (`cua-driver config set` CLI, one-shot `cua-driver call`) writes the persisted global default.++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `capture_mode` | string | No | Default capture mode for `get_window_state`. |+| `max_image_dimension` | integer | No | Max dimension for screenshot resizing. 0 = no limit. |+| `experimental_pip` | boolean | No | Enable the experimental picture-in-picture preview window. Applies on next daemon restart. |+| `experimental_pip_geometry` | string | No | PiP window size + optional position in `WxH` or `WxH+X+Y` form (e.g. `320x200+24+24`). Applies on next daemon restart. |++---++### `start_session`++Declare a session: a named, color-coded identity for the current agent run. The agent cursor, per-session config, and recording all key on the session id, and it follows the run across apps/windows. Idempotent (re-calling refreshes the idle-TTL). End it with `end_session` or let the idle-TTL reclaim it.++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `session` | string | Yes | Stable session id for this run (e.g. `"research-run-1"`). |++---++### `end_session`++End a session declared with `start_session`: removes its agent cursor, stops any recording it owns, and clears its per-session config. Idempotent.++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `session` | string | Yes | The session id to end. |++---++### `set_agent_cursor_enabled`++Show or hide the agent cursor overlay for a cursor instance. The overlay is on by default for each MCP session.++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `enabled` | boolean | Yes | `true` = show, `false` = hide. |+| `cursor_id` | string | No | Cursor instance. Default: `"default"`. |++```json+{"enabled": false}+```++---++### `set_agent_cursor_motion`++Configure the visual appearance and motion curve of an agent cursor instance.++**Appearance parameters:**++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `cursor_id` | string | No | Instance name. Default: `"default"`. |+| `cursor_icon` | string | No | Built-in (`arrow`, `crosshair`, `hand`, `dot`) or PNG/SVG file path. |+| `cursor_color` | string | No | Hex color (e.g. `"#00FFFF"`) or CSS name. |+| `cursor_label` | string | No | Short text shown near the cursor. |+| `cursor_size` | number | No | Dot radius in points. Default 16. |+| `cursor_opacity` | number | No | Opacity 0.0–1.0. Default 0.85. |++**Motion curve parameters:**++| Argument | Type | Range | Default | Description |+| -------- | ---- | ----- | ------- | ----------- |+| `start_handle` | number | [0, 1] | 0.3 | Departure control-point fraction. |+| `end_handle` | number | [0, 1] | 0.3 | Arrival control-point fraction. |+| `arc_size` | number | [0, 1] | 0.25 | Perpendicular deflection as fraction of path length. |+| `arc_flow` | number | [-1, 1] | 0.0 | Asymmetry bias; positive bulges toward destination. |+| `spring` | number | [0.3, 1.0] | 0.72 | Settle damping; 1.0 = no overshoot. |+| `turn_radius` | number | [1, 1000] | 80 | Corner rounding radius at direction changes. |+| `glide_duration_ms` | number | [50, 5000] | — | Flight duration per move in ms. Omit for speed-based timing. |+| `dwell_after_click_ms` | number | [0, 5000] | 80 | Pause after click ripple in ms. |+| `idle_hide_ms` | number | [0, 60000] | 20000 | Auto-hide delay in ms. 0 = never hide. |++---++### `set_agent_cursor_style`++Update the visual style of the agent cursor overlay.++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `cursor_id` | string | No | Cursor instance. Default: `"default"`. |+| `gradient_colors` | array of string | No | CSS hex gradient stops tip→tail. Empty array reverts to default palette colours. |+| `bloom_color` | string | No | Hex bloom/halo colour (e.g. `"#00FFFF"`). Empty string reverts to default. |+| `image_path` | string | No | Path to PNG/JPEG/SVG/ICO cursor image. Empty string reverts to procedural arrow. |++---++## Maintenance tools++### `check_permissions`++Report TCC permission status for Accessibility and Screen Recording. By default also raises the system permission dialogs for any missing grants (Apple's request APIs are no-ops when the grant is already active). Pass `{"prompt": false}` for a read-only status check.++| Argument | Type | Required | Description |+| -------- | ---- | -------- | ----------- |+| `prompt` | boolean | No | Raise the system permission prompts for missing grants. Default true. |++---++### `check_for_update`++Check whether a newer cua-driver-rs release is available on GitHub. Returns the current and latest versions, an `update_available` boolean, the install one-liner, and the release notes URL. Read-only. Mirror of `cua-driver check-update --json`.++**Arguments:** none.++### `install_ffmpeg`++Install ffmpeg, which the recording video backend shells out to. Two-step and confirmed: called without `confirm` it only reports the command it would run (a read-only preview); pass `confirm: true` to actually run it. No-op if ffmpeg is already on `PATH`.++**Arguments:**++| Argument | Type | Description |+| -------- | ---- | ----------- |+| `confirm` | boolean (optional) | When `true`, runs the install. Omit or set `false` for a read-only preview of the command. |diff --git a/docs/content/docs/reference/cua-driver/meta.json b/docs/content/docs/reference/cua-driver/meta.jsonnew file mode 100644index 0000000000..22be9dae07--- /dev/null+++ b/docs/content/docs/reference/cua-driver/meta.json@@ -0,0 +1 @@+{ "title": "Cua Driver", "pages": ["cli-reference", "mcp-tools", "limits"] }diff --git a/docs/content/docs/reference/docs-code-mcp/index.mdx b/docs/content/docs/reference/docs-code-mcp/index.mdxnew file mode 100644index 0000000000..cf99b48c30--- /dev/null+++ b/docs/content/docs/reference/docs-code-mcp/index.mdx@@ -0,0 +1,107 @@+---+title: Docs & Code MCP Server+description: "Reference for the hosted Cua Docs & Code MCP Server: endpoint, the four read-only query tools, database tables, and version indexing."+---++The Docs & Code MCP Server is a hosted MCP server exposing **read-only** search over Cua's documentation and versioned source code. The endpoint is `https://vk-mcp.cua.ai/mcp` over the streamable-HTTP transport. The Docs & Code MCP Server is backed by SQLite FTS5 for keyword search and LanceDB embeddings using `all-MiniLM-L6-v2`, 384-dimensional vectors, for semantic search. It is re-crawled daily via Modal. All tools are read-only. SQL tools accept SELECT queries only.++## Tools++### query_docs_db++SQL query against the documentation database, read-only and SELECT only.++Parameters:++| Name | Type | Description |+| --- | --- | --- |+| `sql` | string | SQL SELECT query against the docs database. |++Returns a list of rows as objects keyed by column name.++This tool targets the docs database, including the `pages` table and the `pages_fts` FTS5 virtual table.++### query_docs_vectors++Vector similarity search over the documentation embeddings, read-only.++Parameters:++| Name | Type | Description |+| --- | --- | --- |+| `query` | string | Natural language query, embedded and matched against documentation vectors. |+| `limit` | int | Maximum number of matches. Default `10`, maximum `100`. |+| `where` | string, optional | SQL-like filter, for example `category = 'cua'`. |+| `select` | list of strings, optional | Columns to return. The vector column is always excluded. |++Returns matching document chunks with a `_distance` similarity score.++### query_code_db++SQL query against the code database, read-only and SELECT only.++Parameters:++| Name | Type | Description |+| --- | --- | --- |+| `sql` | string | SQL SELECT query against the code database. |++Returns a list of rows as objects keyed by column name.++This tool targets the code database, including the `code_files` table and the `code_files_fts` FTS5 virtual table.++### query_code_vectors++Vector similarity search over the code embeddings, read-only.++Parameters:++| Name | Type | Description |+| --- | --- | --- |+| `query` | string | Natural language query, embedded and matched against code vectors. |+| `limit` | int | Maximum number of matches. Default `10`, maximum `100`. |+| `where` | string, optional | SQL-like filter, for example `version = '0.7.3'`. |+| `select` | list of strings, optional | Columns to return. |+| `component` | string, optional | Component name to restrict the search. |++Returns matching code chunks with a `_distance` similarity score.++## Database tables++The SQL tools query two SQLite tables and their FTS5 companions.++### pages++Documentation pages.++| Column | Type | Description |+| --- | --- | --- |+| `id` | INTEGER PRIMARY KEY | Row identifier. |+| `url` | TEXT | Unique full URL of the page. |+| `title` | TEXT | Page title. |+| `category` | TEXT | Category, for example `cua`, `cuabench`, or `llms.txt`. |+| `content` | TEXT | Plain text content with markdown stripped. |++The `pages_fts` FTS5 virtual table mirrors `pages` for full text search. The `content` column is indexed. The `url`, `title`, and `category` columns are unindexed.++### code_files++Versioned source code files.++| Column | Type | Description |+| --- | --- | --- |+| `id` | INTEGER PRIMARY KEY | Row identifier. |+| `component` | TEXT | Component name, for example `agent` or `computer`. |+| `version` | TEXT | Version string, for example `0.7.3`. |+| `file_path` | TEXT | Path to the file within the component. |+| `content` | TEXT | Full source content. |+| `language` | TEXT | Programming language. |+| `UNIQUE(component, version, file_path)` | constraint | Unique source file identity within a component version. |++The `code_files_fts` FTS5 virtual table mirrors `code_files` for full text search. The `content` column is indexed. The `component`, `version`, and `file_path` columns are unindexed.++## Version indexing++Source code is indexed per released version. Each row in `code_files` is keyed by `(component, version, file_path)`, so the same file exists once per version it appeared in. Components are named units such as `agent` and `computer`. Versions are release strings such as `0.7.3`, drawn from the project's git tags.++A query can target one version, compare across versions, or enumerate them, for example `SELECT DISTINCT version FROM code_files WHERE component = 'agent'`. The index is rebuilt daily.diff --git a/docs/content/docs/reference/docs-code-mcp/meta.json b/docs/content/docs/reference/docs-code-mcp/meta.jsonnew file mode 100644index 0000000000..7765264a91--- /dev/null+++ b/docs/content/docs/reference/docs-code-mcp/meta.json@@ -0,0 +1 @@+{ "title": "Docs & Code MCP", "pages": ["index"] }diff --git a/docs/content/docs/reference/index.mdx b/docs/content/docs/reference/index.mdxnew file mode 100644index 0000000000..935d97b7c7--- /dev/null+++ b/docs/content/docs/reference/index.mdx@@ -0,0 +1,11 @@+---+title: 'Reference'+description: 'Index of reference material for Cua Driver and the Sandbox SDK.'+---++Reference documents the full technical specification for Cua's subsystems. It covers `cua-driver` and `sandbox-sdk`. The pages define commands, APIs, exposed tools, types, and stated limits.++| Section | What it documents |+|---------|-------------------|+| [`cua-driver`](/reference/cua-driver/cli-reference) | CLI commands, 35 MCP tools exposed via the stdio MCP server, and known behavioral limits, including platform-specific constraints on the no-foreground-steal contract. |+| [`sandbox-sdk`](/reference/sandbox-sdk) | Sandbox SDK API, documented by a separate agent. |diff --git a/docs/content/docs/reference/meta.json b/docs/content/docs/reference/meta.jsonnew file mode 100644index 0000000000..0ccc7d0d66--- /dev/null+++ b/docs/content/docs/reference/meta.json@@ -0,0 +1 @@+{ "title": "Reference", "icon": "BookText", "pages": ["index", "cua-driver", "sandbox-sdk", "docs-code-mcp"] }diff --git a/docs/content/docs/reference/sandbox-sdk/image.mdx b/docs/content/docs/reference/sandbox-sdk/image.mdxnew file mode 100644index 0000000000..d37589f22c--- /dev/null+++ b/docs/content/docs/reference/sandbox-sdk/image.mdx@@ -0,0 +1,276 @@+---+title: Image reference+description: "API reference for the Image class: the immutable, chainable image specification used to configure sandbox environments."+---++`Image` is an *immutable, chainable image specification*. Each method returns a new `Image` instance; the original is unchanged. Import from `cua`:++```python+from cua import Image+```++## Constructors++### Image.linux++```python+@classmethod+def linux(cls, distro: str = 'ubuntu', version: str = '24.04', kind: str = 'vm') -> Image+```++Linux image. Defaults to `kind='vm'` (QEMU). `kind='container'` uses Docker/XFCE.++| Parameter | Type | Default | Description |+|-----------|------|---------|-------------|+| `distro` | `str` | `'ubuntu'` | Linux distribution |+| `version` | `str` | `'24.04'` | Distribution version |+| `kind` | `str` | `'vm'` | `'vm'` or `'container'` |++### Image.macos++```python+@classmethod+def macos(cls, version: str = '26', kind: str = 'vm') -> Image+```++macOS image. Always a VM (Apple Virtualization / Lume). Supported versions: `'15'` / `'sequoia'`, `'26'` / `'tahoe'`.++| Parameter | Type | Default | Description |+|-----------|------|---------|-------------|+| `version` | `str` | `'26'` | macOS version string or name |+| `kind` | `str` | `'vm'` | Always `'vm'` |++### Image.windows++```python+@classmethod+def windows(cls, version: str = '11', kind: str = 'vm') -> Image+```++Windows image. Always a VM (QEMU or Hyper-V).++| Parameter | Type | Default | Description |+|-----------|------|---------|-------------|+| `version` | `str` | `'11'` | Windows version |+| `kind` | `str` | `'vm'` | Always `'vm'` |++### Image.android++```python+@classmethod+def android(cls, version: str = '14', kind: str = 'vm') -> Image+```++Android image. Always a VM (QEMU emulator).++| Parameter | Type | Default | Description |+|-----------|------|---------|-------------|+| `version` | `str` | `'14'` | Android version |+| `kind` | `str` | `'vm'` | Always `'vm'` |++### Image.from_registry++```python+@classmethod+def from_registry(cls, ref: str) -> Image+```++Image from an OCI registry reference. `kind` is resolved after pull.++| Parameter | Type | Default | Description |+|-----------|------|---------|-------------|+| `ref` | `str` | required | Registry reference, e.g. `'ghcr.io/trycua/macos-tahoe-cua:latest'` or `'ubuntu:22.04'` |++### Image.from_file++```python+@classmethod+def from_file(cls, path: str, os_type: str = 'windows', kind: str = 'vm', agent_type: Optional[str] = None) -> Image+```++Image from a local disk file, ISO, or URL. Supported formats: `qcow2`, `vhdx`, `raw`, `img`, `iso`. HTTP/HTTPS URLs are downloaded and cached automatically. Zip files are extracted. For ISOs, the runtime creates a `qcow2` disk and attaches the ISO as a CD-ROM. Downloaded images are cached in `~/.cua/cua-sandbox/image-cache/`.++| Parameter | Type | Default | Description |+|-----------|------|---------|-------------|+| `path` | `str` | required | Local file path or `http`/`https` URL |+| `os_type` | `str` | `'windows'` | `'linux'`, `'windows'`, `'macos'`, or `'android'` |+| `kind` | `str` | `'vm'` | `'vm'` or `'container'` |+| `agent_type` | `str or None` | `None` | e.g. `'osworld'` for OSWorld Flask server |++### Image.from_dict++```python+@classmethod+def from_dict(cls, data: Dict[str, Any]) -> Image+```++Reconstruct an `Image` from a serialized spec dict (e.g. the output of `to_dict()`).++---++## Builder methods++Each method returns a new `Image`. Builder calls are composable and can be chained in any order.++### Image.apt_install++```python+def apt_install(self, *packages: str) -> Image+```++Install packages via `apt`. Linux only.++### Image.brew_install++```python+def brew_install(self, *packages: str) -> Image+```++Install packages via Homebrew. macOS only.++### Image.choco_install++```python+def choco_install(self, *packages: str) -> Image+```++Install packages via Chocolatey. Windows only.++### Image.winget_install++```python+def winget_install(self, *packages: str) -> Image+```++Install packages via `winget`. Windows only.++### Image.apk_install++```python+def apk_install(self, *apk_paths: str) -> Image+```++Install APK files via `adb`. Android only.++### Image.pwa_install++```python+def pwa_install(+ self,+ manifest_url: str,+ package_name: Optional[str] = None,+ keystore: Optional[str] = None,+ keystore_alias: str = 'android',+ keystore_password: str = 'android',+ builder: str = 'pwa2apk',+ push_timeout: Optional[float] = None,+) -> Image+```++Build an APK from a PWA manifest URL and install it via `adb`. Android only. The default `builder='pwa2apk'` produces a lightweight WebView APK. With `builder='bubblewrap'` it builds a Trusted Web Activity (TWA), which additionally requires the keystore SHA-256 fingerprint to match the server's `/.well-known/assetlinks.json` so Chrome opens the PWA without browser chrome.++| Parameter | Type | Default | Description |+|-----------|------|---------|-------------|+| `manifest_url` | `str` | required | Full URL to the PWA's `manifest.json` |+| `package_name` | `str or None` | `None` | Android package ID; derived from hostname if omitted |+| `keystore` | `str or None` | `None` | Path to `.keystore`/`.jks` file; auto-generated and cached if omitted |+| `keystore_alias` | `str` | `'android'` | Key alias inside the keystore |+| `keystore_password` | `str` | `'android'` | Password for both the store and the key |+| `builder` | `str` | `'pwa2apk'` | `'pwa2apk'` (WebView APK) or `'bubblewrap'` (TWA) |+| `push_timeout` | `float or None` | `None` | Timeout for the `adb install` push |++Built APKs are cached by `(manifest_url, package_name)`. Requirements: Node.js >= 18, Java <= 21 on `PATH` (Gradle does not support Java 22+).++### Image.pip_install++```python+def pip_install(self, *packages: str) -> Image+```++Install Python packages via `pip`.++### Image.uv_install++```python+def uv_install(self, *packages: str) -> Image+```++Install Python packages via `uv add` into the cua-server project. Faster than `pip_install`.++### Image.run++```python+def run(self, command: str) -> Image+```++Run an arbitrary shell command during image setup.++### Image.env++```python+def env(self, **variables: str) -> Image+```++Set environment variables. Values are stored in the `Image` spec in plaintext. Do not use for secrets.++### Image.copy++```python+def copy(self, src: str, dst: str) -> Image+```++Copy a local file into the image at the specified destination path.++### Image.expose++```python+def expose(self, port: int) -> Image+```++Mark a port the sandbox will serve on. Used with `sb.tunnel.forward()`.++---++## Serialization++### Image.to_dict++```python+def to_dict(self) -> Dict[str, Any]+```++Serialize to a plain dict suitable for JSON or the cloud API.++Example output:++```python+{+ 'os_type': 'linux',+ 'distro': 'ubuntu',+ 'version': '24.04',+ 'kind': 'container',+ 'layers': [+ {'type': 'apt_install', 'packages': ['curl']},+ {'type': 'pip_install', 'packages': ['requests']},+ ]+}+```++### Image.to_cloud_init++```python+def to_cloud_init(self) -> str+```++Generate a cloud-init user-data script from the image layers.++---++## Attributes++| Name | Type | Description |+|------|------|-------------|+| `os_type` | `str` | `'linux'`, `'macos'`, `'windows'`, or `'android'` |+| `distro` | `str` | Distribution name (e.g. `'ubuntu'`) |+| `version` | `str` | Version string (e.g. `'24.04'`) |+| `kind` | `str or None` | `'container'` or `'vm'` |diff --git a/docs/content/docs/reference/sandbox-sdk/index.mdx b/docs/content/docs/reference/sandbox-sdk/index.mdxnew file mode 100644index 0000000000..713958d3da--- /dev/null+++ b/docs/content/docs/reference/sandbox-sdk/index.mdx@@ -0,0 +1,173 @@+---+title: Sandbox SDK reference+description: "API reference for the Cua Sandbox Python package: Sandbox, Image, Localhost, and top-level functions."+---++## Package++`cua-sandbox` is a Python package imported as `cua`.++| Field | Value |+|-------|-------|+| Package | `cua-sandbox` |+| Import name | `cua` |+| Python version | 3.12+ |+| Install | `pip install cua` |++## Top-level exports++| Export | Description |+|--------|-------------|+| `Sandbox` | VM or container control class |+| `Image` | Image builder class |+| `Localhost` | Direct host control class |+| `SandboxInfo` | Sandbox metadata structure |+| `configure()` | Global SDK configuration function |+| `login()` | Browser login function |+| `whoami()` | Authenticated user lookup function |++## Environment variables++| Variable | Description |+|----------|-------------|+| `CUA_API_KEY` | Cua API key for cloud sandboxes |++## Sandbox class++`Sandbox` controls a VM or container through `.mouse`, `.keyboard`, `.screen`, `.clipboard`, `.shell`, `.window`, `.terminal`, `.mobile`, and `.tunnel`.++### Sandbox.ephemeral++Async context manager. The sandbox is destroyed automatically on exit.++```python+async def ephemeral(cls, image: Image, name=None, api_key=None, local=False, runtime=None, cpu=None, memory_mb=None, disk_gb=None, region='us-east-1') -> AsyncIterator[Sandbox]+```++| Parameter | Type | Default | Description |+|-----------|------|---------|-------------|+| `image` | `Image` | required | Image to run |+| `name` | `str or None` | `None` | Optional name |+| `api_key` | `str or None` | `None` | Cua API key; reads `CUA_API_KEY` if `None` |+| `local` | `bool` | `False` | Use local runtime (Docker/Lume/QEMU) |+| `runtime` | `Runtime or None` | `None` | Explicit runtime backend |+| `cpu` | `int or None` | `None` | CPU count (cloud) |+| `memory_mb` | `int or None` | `None` | Memory in MB (cloud) |+| `disk_gb` | `int or None` | `None` | Disk size in GB (cloud) |+| `region` | `str` | `'us-east-1'` | Cloud region |++### Sandbox.create++Provisions a persistent sandbox. The sandbox keeps running after the script exits.++```python+async def create(cls, image: Image, name=None, api_key=None, local=False, runtime=None, cpu=None, memory_mb=None, disk_gb=None, region='us-east-1') -> Sandbox+```++Same parameters as `ephemeral`.++### Sandbox.connect++Connects to an existing sandbox by name. Supports both `await` and `async with`. When used as a context manager, `disconnect()` is called on exit. The sandbox keeps running.++```python+def connect(cls, name: str, api_key=None, local=False, ws_url=None, http_url=None, container_name=None, cpu=None, memory_mb=None, disk_gb=None, region='us-east-1') -> _ConnectResult+```++| Parameter | Type | Default | Description |+|-----------|------|---------|-------------|+| `name` | `str` | required | Sandbox name |+| `api_key` | `str or None` | `None` | Cua API key; reads `CUA_API_KEY` if `None` |+| `local` | `bool` | `False` | Use local runtime (Docker/Lume/QEMU) |+| `ws_url` | `str or None` | `None` | WebSocket URL |+| `http_url` | `str or None` | `None` | HTTP URL |+| `container_name` | `str or None` | `None` | Container name |+| `cpu` | `int or None` | `None` | CPU count (cloud) |+| `memory_mb` | `int or None` | `None` | Memory in MB (cloud) |+| `disk_gb` | `int or None` | `None` | Disk size in GB (cloud) |+| `region` | `str` | `'us-east-1'` | Cloud region |++### Sandbox instance methods++| Method | Signature | Description |+|--------|-----------|-------------|+| `disconnect` | `async disconnect() -> None` | Drop transport connection. Sandbox keeps running. |+| `destroy` | `async destroy() -> None` | Disconnect and permanently delete the sandbox. |+| `screenshot` | `async screenshot(text=None, format='png', quality=95) -> bytes` | Capture screenshot. `format`: `'png'` or `'jpeg'`. `quality`: 1-95 (JPEG only). |+| `screenshot_base64` | `async screenshot_base64(text=None, format='png', quality=95) -> str` | Screenshot as base64 string. |+| `get_environment` | `async get_environment() -> str` | Returns `'linux'`, `'mac'`, `'windows'`, or `'browser'`. |+| `get_display_url` | `async get_display_url(share=False) -> str` | URL to view the sandbox display. `share=True` returns a public link (cloud only). |+| `get_dimensions` | `async get_dimensions() -> tuple[int, int]` | Screen width and height in pixels. |++### Sandbox class methods++| Method | Signature | Description |+|--------|-----------|-------------|+| `list` | `async list(local=False, api_key=None) -> list[SandboxInfo]` | List running and suspended sandboxes. |+| `get_info` | `async get_info(name, local=False, api_key=None) -> SandboxInfo` | Metadata for a specific sandbox. |+| `suspend` | `async suspend(name, local=False, api_key=None) -> None` | Suspend a sandbox (save state). Local QEMU: QMP snapshot. Local Docker: pause. Local Lume: stop. Cloud: `POST /v1/vms/{name}/stop`. |+| `resume` | `async resume(name, local=False, api_key=None) -> Sandbox` | Resume a suspended sandbox. Returns connected `Sandbox`. |+| `restart` | `async restart(name, local=False, api_key=None) -> Sandbox` | Restart (suspend then resume). Returns connected `Sandbox`. |+| `delete` | `async delete(name, local=False, api_key=None) -> None` | Permanently delete. Local: stops and removes state file. Cloud: `DELETE /v1/vms/{name}`. |++### Sandbox attributes++| Attribute | Type | Description |+|-----------|------|-------------|+| `name` | `str or None` | Sandbox name |+| `shell` | `Shell` | Run shell commands |+| `mouse` | `Mouse` | Mouse control |+| `keyboard` | `Keyboard` | Keyboard control |+| `screen` | `Screen` | Screenshots and screen info |+| `clipboard` | `Clipboard` | Clipboard read/write |+| `tunnel` | `Tunnel` | Port forwarding |+| `terminal` | `Terminal` | PTY sessions |+| `window` | `Window` | Window management |+| `mobile` | `Mobile` | Mobile (Android) touch and hardware keys |++## SandboxInfo++Returned by `Sandbox.list()` and `Sandbox.get_info()`.++| Field | Type | Description |+|-------|------|-------------|+| `name` | `str` | Sandbox name |+| `status` | `str` | Current status |+| `source` | `str` | Runtime source |+| `os_type` | `str or None` | OS type |+| `host` | `str or None` | Host address |+| `vnc_url` | `str or None` | VNC URL |+| `api_url` | `str or None` | API URL |+| `created_at` | `str or None` | ISO timestamp |++## configure, login, whoami++```python+def configure(api_key=None, base_url=None) -> None+# Set global SDK config. Equivalent to setting CUA_API_KEY.++def login(base_url=None) -> None+# Open Cua login page in browser; store credentials in ~/.cua/credentials (Clerk flow).++def whoami(api_key=None) -> Dict[str, Any]+# Return dict with user info (id, email, etc.) from Cua API.+```++## Localhost++Direct host control via `cua_auto`. No sandbox. Attributes: `screen`, `mouse`, `keyboard`, `clipboard`, `shell`, `window`, `terminal`.++```python+# plain await+host = await Localhost.connect()+await host.shell.run('echo hello')+await host.disconnect()++# context manager+async with Localhost.connect() as host:+ await host.shell.run('echo hello')+```++Methods: same signatures as `Sandbox` (`screenshot`, `screenshot_base64`, `get_environment`, `get_dimensions`, `disconnect`).++See [Sandbox interfaces reference](/reference/sandbox-sdk/interfaces) for the full API of sb.shell, sb.mouse, sb.keyboard, sb.screen, sb.clipboard, sb.tunnel, sb.terminal, sb.window, and sb.mobile. See [Image reference](/reference/sandbox-sdk/image) for the full Image builder API.diff --git a/docs/content/docs/reference/sandbox-sdk/interfaces.mdx b/docs/content/docs/reference/sandbox-sdk/interfaces.mdxnew file mode 100644index 0000000000..593ab2853f--- /dev/null+++ b/docs/content/docs/reference/sandbox-sdk/interfaces.mdx@@ -0,0 +1,167 @@+---+title: Sandbox interfaces reference+description: API reference for sb.shell, sb.mouse, sb.keyboard, sb.screen, sb.clipboard, sb.tunnel, sb.terminal, sb.window, and sb.mobile.+---++| Attribute | Class | Purpose |+|-----------|-------|---------|+| sb.shell | Shell | Run shell commands |+| sb.mouse | Mouse | Mouse control |+| sb.keyboard | Keyboard | Keyboard control |+| sb.screen | Screen | Screenshots and screen info |+| sb.clipboard | Clipboard | Clipboard read/write |+| sb.tunnel | Tunnel | Port forwarding |+| sb.terminal | Terminal | PTY terminal sessions |+| sb.window | Window | Window management |+| sb.mobile | Mobile | Mobile (Android) touch and hardware-key control |++## Shell++```python+async def run(command: str, timeout: int = 30) -> CommandResult+```++### CommandResult++| Field | Type | Description |+|-------|------|-------------|+| stdout | str | Standard output |+| stderr | str | Standard error |+| returncode | int | Exit code |+| success | bool (property) | True if returncode == 0 |++## Mouse++```python+async def click(x: int, y: int, button: str = 'left') -> None+async def right_click(x: int, y: int) -> None+async def double_click(x: int, y: int) -> None+async def move(x: int, y: int) -> None+async def scroll(x: int, y: int, scroll_x: int = 0, scroll_y: int = 3) -> None+async def mouse_down(x: int, y: int, button: str = 'left') -> None+async def mouse_up(x: int, y: int, button: str = 'left') -> None+async def drag(start_x: int, start_y: int, end_x: int, end_y: int, button: str = 'left') -> None+```++## Keyboard++```python+async def type(text: str) -> None+# Type a string of text.++async def keypress(keys: Union[List[str], str]) -> None+# Press a key combination, e.g. ['ctrl', 'c'] or 'enter'.++async def key_down(key: str) -> None+async def key_up(key: str) -> None+```++## Screen++```python+async def screenshot(format: str = 'png', quality: int = 95) -> bytes+# format: 'png' (lossless) or 'jpeg' (lossy, ~5-10x smaller). quality: 1-95, ignored for PNG.++async def screenshot_base64(format: str = 'png', quality: int = 95) -> str+# Screenshot as base64-encoded string.++async def size() -> Tuple[int, int]+# Returns (width, height) in pixels.+```++## Clipboard++```python+async def get() -> str+# Returns current clipboard text.++async def set(text: str) -> None+# Sets clipboard text.+```++## Tunnel++```python+def forward(*ports) -> _TunnelContext+```++Forwards one or more sandbox ports (or Android abstract sockets) to the host.++*ports* may be int (TCP port) or str (Android abstract socket name, e.g. 'chrome_devtools_remote').++Returns:+- single TunnelInfo when one target is given+- dict[sandbox_port, TunnelInfo] when multiple targets are given++### TunnelInfo++| Field | Type | Description |+|-------|------|-------------|+| host | str | Always 'localhost' |+| port | int | Host-side port assigned |+| sandbox_port | int or str | Original port/socket inside the sandbox |+| url | str (property) | `http://{host}:{port}` |++```python+async def close() -> None+# Close this tunnel. No-op if already closed or inside a context manager.+```++## Terminal++```python+async def create(command: Optional[str] = None, cols: int = 80, rows: int = 24) -> dict+# Create a PTY session (default command is the login shell). Returns {'pid': int, 'cols': int, 'rows': int}.++async def send_input(pid: int, data: str) -> None+# Send input to a PTY session.++async def info(pid: int) -> Optional[dict]+# Return PTY session info, or None if the session is gone.++async def close(pid: int) -> bool+# Kill a PTY session. Returns True on success.+```++## Window++```python+async def get_active_title() -> str+# Returns the title of the currently focused window.+```++## Mobile++Mobile (Android) touch and hardware-key control. Coordinates are in screen pixels. Single-touch methods use input tap/swipe via adb shell. Multi-touch gestures use adb root + MT Protocol B sendevent.++```python+async def tap(x: int, y: int) -> None+async def long_press(x: int, y: int, duration_ms: int = 1000) -> None+async def double_tap(x: int, y: int, delay: float = 0.1) -> None+async def type_text(text: str) -> None+async def swipe(x1: int, y1: int, x2: int, y2: int, duration_ms: int = 300) -> None+async def scroll_up(x: int, y: int, distance: int = 600, duration_ms: int = 400) -> None+async def scroll_down(x: int, y: int, distance: int = 600, duration_ms: int = 400) -> None+async def scroll_left(x: int, y: int, distance: int = 400, duration_ms: int = 300) -> None+async def scroll_right(x: int, y: int, distance: int = 400, duration_ms: int = 300) -> None+async def fling(x1: int, y1: int, x2: int, y2: int) -> None+async def gesture(duration_ms: int = 400, steps: int = 0, *finger_paths) -> None+# N-finger gesture via MT Protocol B sendevent. Each positional arg is (x,y) waypoints for one finger.+# steps=0 = auto (duration_ms // 20, min 5).+async def pinch_in(cx: int, cy: int, spread: int = 300, duration_ms: int = 400) -> None+# Pinch-in (zoom out), two simultaneous fingers.+async def pinch_out(cx: int, cy: int, spread: int = 300, duration_ms: int = 400) -> None+# Pinch-out (zoom in), two simultaneous fingers.+async def key(keycode: int) -> None+async def home() -> None+async def back() -> None+async def recents() -> None+async def power() -> None+async def volume_up() -> None+async def volume_down() -> None+async def enter() -> None+async def backspace() -> None+async def wake() -> None+async def notifications() -> None+async def close_notifications() -> None+```diff --git a/docs/content/docs/reference/sandbox-sdk/meta.json b/docs/content/docs/reference/sandbox-sdk/meta.jsonnew file mode 100644index 0000000000..d11948586d--- /dev/null+++ b/docs/content/docs/reference/sandbox-sdk/meta.json@@ -0,0 +1 @@+{ "title": "Sandbox SDK", "pages": ["index", "interfaces", "image"] }diff --git a/docs/content/docs/tutorials/drive-your-first-app.mdx b/docs/content/docs/tutorials/drive-your-first-app.mdxnew file mode 100644index 0000000000..3ea2baeada--- /dev/null+++ b/docs/content/docs/tutorials/drive-your-first-app.mdx@@ -0,0 +1,164 @@+---+title: Drive your first app+description: Install Cua Driver, connect your agent, and ask it in plain English to drive a desktop app in the background on macOS, Windows, or Linux.+---++import { Tabs, Tab } from 'fumadocs-ui/components/tabs';+import { Callout } from 'fumadocs-ui/components/callout';++# Drive your first app++You do not type **Cua Driver** CLI commands by hand. Cua Driver is a backend that an agent harness (Claude Code, Codex, Hermes, and others) drives for you. In this tutorial you install Cua Driver, connect your agent, and ask it in plain English to open a calculator and read a result back. Works the same on macOS, Windows, and Linux.++## 1. Install Cua Driver++Use the same one-line installer on every platform; it picks the right path for the host and needs no administrator access.++<Tabs items={['macOS', 'Windows', 'Linux']}>+ <Tab value="macOS">+ Requires macOS 14 (Sonoma) or later.++ ```bash+ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.sh)"+ ```++ Start the daemon through the app bundle so macOS attributes permission prompts to CuaDriver.app:++ ```bash+ open -n -g -a CuaDriver --args serve+ ```++ Then grant Accessibility and Screen Recording:++ ```bash+ cua-driver permissions grant+ ```+ </Tab>+ <Tab value="Windows">+ Requires Windows 10/11 with an interactive desktop session and PowerShell.++ ```powershell+ irm https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.ps1 | iex+ ```++ Start the daemon from an interactive desktop session, not an SSH session. **The daemon must run in Session 1 or later**; a daemon started over SSH usually lands in Session 0 and cannot drive the GUI.++ ```powershell+ cua-driver serve+ ```+ </Tab>+ <Tab value="Linux">+ Requires an x86_64 Linux desktop session with X11 or XWayland, plus AT-SPI 2.++ ```bash+ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.sh)"+ ```++ Start the daemon from inside your graphical desktop session:++ ```bash+ cua-driver serve+ ```+ </Tab>+</Tabs>++<Callout type="info">+ Full install steps, PATH setup, and permission details live in the install guide. See [Install Cua Driver](/how-to-guides/driver/install).+</Callout>++## 2. Verify it is working++Before you hand control to an agent, confirm the driver can see your desktop. Run the cross-platform probe:++```bash+cua-driver doctor+```++Then list the running GUI apps the driver can reach:++```bash+cua-driver call list_apps+```++If you see your running apps listed, the plumbing works. That is the only CLI you run by hand; from here the agent does the driving.++## 3. Connect your agent++Register Cua Driver with your agent harness once.++<Tabs items={['Claude Code', 'Codex', 'Hermes']}>+ <Tab value="Claude Code">+ Install the Claude Code skill. This is turnkey: the skill teaches Claude Code how to drive apps through Cua Driver, and you then ask in plain English.++ ```bash+ cua-driver skills install+ ```++ Prefer plain MCP instead of the skill? Generate the registration command:++ ```bash+ cua-driver mcp-config --client claude+ ```+ </Tab>+ <Tab value="Codex">+ Print the Codex registration command:++ ```bash+ cua-driver mcp-config --client codex+ ```++ It prints a command you run to register the server:++ ```bash+ codex mcp add cua-driver -- cua-driver mcp+ ```+ </Tab>+ <Tab value="Hermes">+ Print the Hermes snippet:++ ```bash+ cua-driver mcp-config --client hermes+ ```++ Paste it under mcp_servers in ~/.hermes/config.yaml:++ ```yaml+ mcp_servers:+ cua-driver:+ command: "cua-driver"+ args: ["mcp"]+ ```++ Then reload MCP servers inside Hermes:++ ```text+ /reload-mcp+ ```+ </Tab>+</Tabs>++Using Cursor, Gemini/Antigravity, OpenCode, OpenClaw, or Pi instead? See [Connect your agent](/how-to-guides/driver/connect-your-agent).++## 4. Ask your agent to drive an app++Now ask your agent in plain English. Type this prompt to Claude Code, Codex, or Hermes:++```text+Open the calculator and compute 17 x 23, then tell me the answer.+```++The agent uses Cua Driver to launch the calculator (Calculator on macOS and Windows, the system calculator on Linux), read the window's accessibility tree, click the number and operator keys, and read the result back from the display. All of it runs in the background. The agent picks the right calculator app for your OS, so the same prompt works on all three platforms.++## 5. Confirm what happened++The agent reports 391. Cua Driver kept the calculator in the background the whole time. It never stole your keyboard focus and never moved your cursor. That is the *no-foreground contract*: the agent drives the app while you keep working. See [The no-foreground contract](/explanation/the-no-foreground-contract).++## What you did++You installed Cua Driver, verified it could see your desktop, connected your agent harness, and asked in plain English for a result computed inside a real desktop app. The agent did the driving through Cua Driver, in the background, on your platform.++## Next steps++- [Connect your agent](/how-to-guides/driver/connect-your-agent): register Cua Driver with Cursor, Antigravity, OpenCode, OpenClaw, Pi, and more.+- [The no-foreground contract](/explanation/the-no-foreground-contract): why the agent never steals focus or moves your cursor.+- [How-to guides](/how-to-guides): keep the driver running, update it, and more.diff --git a/docs/content/docs/tutorials/index.mdx b/docs/content/docs/tutorials/index.mdxnew file mode 100644index 0000000000..bb9e396cd9--- /dev/null+++ b/docs/content/docs/tutorials/index.mdx@@ -0,0 +1,12 @@+---+title: 'Tutorials'+description: 'Learn Cua by following end-to-end builds.'+---++Tutorials teach Cua by building something end to end. Start here if you are new to Cua. Each tutorial is *one happy path* you can follow from start to finish.++- [Drive your first app](/tutorials/drive-your-first-app) - operate a native app in the background with Cua Driver.+- [Your first cloud sandbox](/tutorials/your-first-cloud-sandbox) - spin up a disposable cloud computer.+- [Run code and drive the GUI in a sandbox](/tutorials/run-an-agent-in-a-sandbox) - run code and drive the GUI in a cloud sandbox via the SDK.++Once you are comfortable, [How-to guides](/how-to-guides) cover specific goals and [Reference](/reference) has the exact APIs.diff --git a/docs/content/docs/tutorials/meta.json b/docs/content/docs/tutorials/meta.jsonnew file mode 100644index 0000000000..5d4fa1c0ea--- /dev/null+++ b/docs/content/docs/tutorials/meta.json@@ -0,0 +1 @@+{ "title": "Tutorials", "icon": "GraduationCap", "pages": ["index", "drive-your-first-app", "your-first-cloud-sandbox", "run-an-agent-in-a-sandbox"] }diff --git a/docs/content/docs/tutorials/run-an-agent-in-a-sandbox.mdx b/docs/content/docs/tutorials/run-an-agent-in-a-sandbox.mdxnew file mode 100644index 0000000000..a6f0bf0dcc--- /dev/null+++ b/docs/content/docs/tutorials/run-an-agent-in-a-sandbox.mdx@@ -0,0 +1,99 @@+---+title: Run code and drive the GUI in a sandbox+description: Spin up a cloud Linux sandbox, run code and a GUI action against it with the Python SDK, then let it tear down.+---++import { Callout } from 'fumadocs-ui/components/callout';++# Run code and drive the GUI in a sandbox++In this tutorial, you create an *ephemeral* cloud Linux sandbox, run Python code inside it, set and read the sandbox clipboard, and save a screenshot from the sandbox to your current directory.++<Callout type="info">+ Prerequisites: Python 3.12 or 3.13. A Cua API key from cua.ai under **Dashboard > API Keys > New API Key**. No Anthropic key is needed.+</Callout>++## Sign in and create an API key++Sign in at cua.ai.++Open **Dashboard > API Keys > New API Key**.++Copy the API key immediately. You will use it in your terminal in the next step.++## Install the SDK++Open a terminal and install the Python SDK:++```bash+pip install cua+```++## Set your API key++Set `CUA_API_KEY` in the same terminal:++```bash+export CUA_API_KEY=sk_cua-...+```++Replace `sk_cua-...` with the API key you copied from the dashboard.++## Create the script++Create a file named `drive_sandbox.py`:++```python+import asyncio+from cua import Sandbox, Image++async def main():+ async with Sandbox.ephemeral(Image.linux()) as sb:+ result = await sb.shell.run("python3 -c 'print(1 + 1)'")+ print(result.stdout)++ await sb.clipboard.set("Hello from the sandbox clipboard")+ value = await sb.clipboard.get()+ print(value)++ screenshot = await sb.screenshot()+ with open("screenshot.png", "wb") as f:+ f.write(screenshot)+ print("Screenshot saved to screenshot.png")++asyncio.run(main())+```++## Run it++Run the script from the same directory:++```bash+python drive_sandbox.py+```++You should see output like this:++```text+2++Hello from the sandbox clipboard+Screenshot saved to screenshot.png+```++You should also see a new file named `screenshot.png` in the current directory.++## What just happened++`Sandbox.ephemeral(Image.linux())` created a cloud Linux sandbox. `sb.shell.run("python3 -c 'print(1 + 1)'")` executed code inside the cloud container and returned a `CommandResult` with `stdout`, `stderr`, `returncode`, and the `success` boolean property. The script printed `result.stdout`.++`sb.clipboard.set(...)` changed the sandbox desktop clipboard state. `sb.clipboard.get()` read that GUI state back as a string. `sb.screenshot()` captured the sandbox display as PNG bytes and wrote them to `screenshot.png`.++When the `async with` block exited, **the sandbox destroyed itself**. No agent was involved.++## Next steps++- [Manage sandbox lifecycle](/how-to-guides/sandbox/lifecycle)+- [Run an interactive shell](/how-to-guides/sandbox/interactive-shell)+- [Forward a sandbox port](/how-to-guides/sandbox/tunneling)+- [Sandbox SDK API reference](/reference/sandbox-sdk)diff --git a/docs/content/docs/tutorials/your-first-cloud-sandbox.mdx b/docs/content/docs/tutorials/your-first-cloud-sandbox.mdxnew file mode 100644index 0000000000..2af9213247--- /dev/null+++ b/docs/content/docs/tutorials/your-first-cloud-sandbox.mdx@@ -0,0 +1,87 @@+---+title: Your first cloud sandbox+description: Create a cloud Linux sandbox with the Python SDK, run one command, and save a screenshot.+---++import { Callout } from 'fumadocs-ui/components/callout';++# Your first cloud sandbox++In this tutorial, you create an *ephemeral* cloud Linux sandbox, run `uname -a` inside it, and save a screenshot from the sandbox to your current directory.++<Callout type="info">+ Prerequisites: Python 3.12 or 3.13. A free account at cua.ai for cloud sandboxes.+</Callout>++## Sign in and create an API key++Sign in at cua.ai.++Open **Dashboard > API Keys > New API Key**.++Copy the API key immediately. You will use it in your terminal in the next step.++## Install the SDK++Open a terminal and install the Python SDK:++```bash+pip install cua+```++## Set your API key++Set `CUA_API_KEY` in the same terminal:++```bash+export CUA_API_KEY=sk_cua-...+```++Replace `sk_cua-...` with the API key you copied from the dashboard.++## Create the script++Create a file named `first_sandbox.py`:++```python+import asyncio+from cua import Sandbox, Image++async def main():+ async with Sandbox.ephemeral(Image.linux()) as sb:+ result = await sb.shell.run("uname -a")+ print(result.stdout)++ screenshot = await sb.screenshot()+ with open("screenshot.png", "wb") as f:+ f.write(screenshot)+ print("Screenshot saved to screenshot.png")++asyncio.run(main())+```++## Run it++Run the script from the same directory:++```bash+python first_sandbox.py+```++You should see a Linux kernel string printed in your terminal.++You should also see a new file named `screenshot.png` in the current directory.++## What just happened++`Sandbox.ephemeral(Image.linux())` created a cloud Linux container. The script ran `uname -a` inside that container, printed the command output, took a screenshot, and wrote it to `screenshot.png`.++When the `async with` block exited, **the sandbox destroyed itself**. No cleanup is needed.++## Next steps++- [Manage sandbox lifecycle](/how-to-guides/sandbox/lifecycle)+- [Create and restore snapshots](/how-to-guides/sandbox/snapshots)+- [Pass secrets to sandboxes](/how-to-guides/sandbox/secrets)+- [Run parallel sandboxes](/how-to-guides/sandbox/scale-out)+- [Sandbox SDK API reference](/reference/sandbox-sdk)