Skip to content

armgit / cua

Public Imported from GitHub
0

Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.

docs(cua-driver): add integration chooser (#3190)

Francesco Bonacci80b05014c22c
diff --git a/docs/content/docs/concepts/choose-a-cua-driver-integration.mdx b/docs/content/docs/concepts/choose-a-cua-driver-integration.mdx
new file mode 100644
index 0000000000..aae59d5214
--- /dev/null
+++ b/docs/content/docs/concepts/choose-a-cua-driver-integration.mdx
@@ -0,0 +1,152 @@
+---
+title: 'Choose a Cua Driver integration'
+description: 'Choose between MCP, the SDK, a private worker, and an app-hosted service based on who owns the runtime.'
+---
+
+import { Callout } from 'fumadocs-ui/components/callout';
+
+Most agent harnesses should connect to Cua Driver through MCP. Embed the SDK
+when computer use is part of your product, and use an app-hosted service when a
+desktop application must own the operating-system permissions.
+
+## Start with who owns the runtime
+
+The runtime is the process that owns Cua Driver's permissions, browser
+connections, recordings, and lifecycle state. Choose its owner before choosing
+the transport:
+
+| You are building | Start with | Runtime owner |
+| -------------------------------------------------- | ----------------------------------------------------------------------- | ------------------------------------------------- |
+| An agent or harness | [MCP](/how-to-guides/driver/connect-your-agent) | The MCP process, or an explicitly selected daemon |
+| An application with built-in computer use | [Same-process SDK](/how-to-guides/driver/use-sdk-in-process) | Your application process |
+| A signed desktop app that serves an external agent | [App-hosted service](/how-to-guides/driver/expose-mcp-from-desktop-app) | The permission-owning desktop app |
+
+These are starting points. Add a private worker or shared daemon only when you
+need the process boundary it provides.
+
+## Connect an agent through MCP
+
+Use MCP when an existing agent or harness needs Cua Driver's tool catalog.
+
+```text
+Agent or harness -> cua-driver mcp -> Cua runtime -> desktop
+```
+
+The MCP client starts `cua-driver mcp` and keeps its standard input and output
+open. That authenticated transport receives a private implicit lifecycle
+session. Closing the transport releases its session state.
+
+This is the normal route for Codex, Claude Code, Cursor, Hermes, and a standard
+OpenClaw gateway. Generate the client-specific configuration with
+`cua-driver mcp-config --client <client>`.
+
+On Windows and Linux, bare `cua-driver mcp` owns its runtime. On macOS, it
+normally proxies to the installed `CuaDriver.app` daemon so Accessibility and
+Screen Recording remain attached to the app identity.
+
+## Embed the SDK in an application
+
+Use the Python or TypeScript SDK when your product owns computer use as a
+feature and wants a typed API instead of an agent protocol.
+
+```text
+Your application -> CuaDriver.create() -> desktop
+```
+
+The application owns runtime startup, permissions, sessions, cancellation, and
+shutdown. It can expose Cua operations directly or map them into a smaller
+product-owned tool.
+
+Use a private worker when the application needs the same SDK contract in a
+supervised child process:
+
+```text
+Your application -> inherited pipes -> private Cua worker -> desktop
+```
+
+The worker has no listener or reconnect path. Closing its private channel ends
+the runtime. This gives one host process isolation without creating a shared
+desktop service.
+
+## Host Cua Driver from a desktop app
+
+Use an app-hosted service when a signed desktop app owns Accessibility and
+Screen Recording permissions while a backend or external agent needs MCP.
+
+```text
+Desktop app -> private Cua service -> generated MCP connection -> agent
+```
+
+The desktop app starts `EmbeddedCuaDriverHost` and passes its returned MCP
+connection to the backend. The backend launches that connection unchanged. It
+must not start a second host.
+
+This distinction matters on macOS. The permission-owning app must start the
+private service directly so the child stays in the app's TCC responsibility
+chain. Starting it from a gateway or backend gives the service that process's
+identity instead.
+
+## Less common variants
+
+Use one-shot CLI calls when an agent can run shell commands but cannot keep an
+MCP transport open:
+
+```text
+Agent -> cua-driver call <tool> -> Cua Driver daemon -> desktop
+```
+
+Use an explicit shared daemon when several clients must reconnect to one
+long-lived desktop service or when the MCP process cannot reach the interactive
+desktop:
+
+```text
+Trusted launcher -> cua-driver serve --socket <endpoint>
+Agent A ---------> cua-driver mcp --socket <endpoint>
+Agent B ---------> cua-driver mcp --socket <endpoint>
+```
+
+The daemon owns the runtime. Each MCP transport still receives its own private
+lifecycle session.
+
+## Permissions belong to the runtime
+
+<Callout type="info">
+ The trusted host fixes the permission mode, capability manifest, launch grants, and authorization
+ callbacks when it starts the runtime. An agent can request actions, but it cannot widen the
+ runtime's permissions.
+</Callout>
+
+The optional public session name is a label for display and explicit lifecycle
+control. It is not a credential and does not carry permission. Two MCP
+transports that use the same public label still have separate private lifecycle
+sessions.
+
+## Existing browser sessions
+
+A driver-owned isolated browser profile is the default. Attaching to an
+existing Chrome or Edge profile is more sensitive because it exposes signed-in
+tabs, cookies, and storage through CDP.
+
+Browser integrations that depend on launching Chromium with a remote debugging
+flag cannot attach that way to a profile already running without the flag. Cua
+Driver can instead use the browser's per-instance remote debugging control in
+the exact approved window, complete the browser-owned consent flow, and bind
+the endpoint to that browser process and lifecycle session. It does not copy,
+edit, restart, or terminate the selected profile.
+
+Standard mode requires the trusted launcher to pass `--grant
+existing-profile`, or an embedding application to approve the exact request.
+The old `browser-approve` token is not part of this model, and an MCP tool
+argument cannot create the grant.
+
+See [Browser Profile Attachment](/reference/cua-driver/browser-profile-attachment)
+for the supported browsers, setup effects, grant lifetime, and refusal codes.
+
+## Go deeper
+
+- [Connect your agent](/how-to-guides/driver/connect-your-agent): configure an MCP client.
+- [Use Cua Driver in process](/how-to-guides/driver/use-sdk-in-process): embed the typed SDK.
+- [Expose MCP from a desktop app](/how-to-guides/driver/expose-mcp-from-desktop-app): host a private service from a signed app.
+- [SDK, MCP, and process hosting](/concepts/sdk-mcp-and-hosting): understand the complete interface and topology grid.
+- [Process model](/reference/cua-driver/process-model): check platform-specific runtime ownership.
+- [Permission modes](/reference/cua-driver/permission-modes): check the authorization contract.
diff --git a/docs/content/docs/concepts/index.mdx b/docs/content/docs/concepts/index.mdx
index 6e7b9dba98..608638f86f 100644
--- a/docs/content/docs/concepts/index.mdx
+++ b/docs/content/docs/concepts/index.mdx
@@ -5,4 +5,4 @@ description: "Understand the design ideas behind Cua and how its main pieces fit
Use these pages when you want the model behind Cua rather than a step-by-step guide or API table. They explain what computer use means in Cua, how Cua Driver keeps the desktop usable while it acts, and how Cua Sandbox gives an agent a disposable local computer.
-Start with [What is computer use?](/concepts/what-is-computer-use) for the basic model. Read [Best-effort background](/concepts/the-no-foreground-contract) to understand Cua Driver's default behavior on a shared machine, then [Capture and delivery modalities](/concepts/capture-and-delivery-modalities) for the action axes. [How we continuously validate Cua Driver](/concepts/how-cua-driver-is-validated) explains why desktop support requires application-owned E2E evidence in addition to unit tests. Read [How sandboxes work](/concepts/how-sandboxes-work) when you need the model for disposable computers. Read [How Lume creates local macOS sandboxes](/concepts/how-lume-unattended-setup-works) for the local VM setup model, and [How SIP works in Lume VMs](/concepts/how-sip-works-in-lume-vms) for signed boot-policy changes. For evaluation, start with [What is Cua-Bench?](/concepts/what-is-cua-bench), then read [The Cua-Bench task lifecycle](/concepts/cua-bench-task-lifecycle).
+Start with [What is computer use?](/concepts/what-is-computer-use) for the basic model. If you are adding Cua Driver to an agent or product, use [Choose a Cua Driver integration](/concepts/choose-a-cua-driver-integration) before reading the deeper [SDK, MCP, and process hosting](/concepts/sdk-mcp-and-hosting) model. Read [Best-effort background](/concepts/the-no-foreground-contract) to understand Cua Driver's default behavior on a shared machine, then [Capture and delivery modalities](/concepts/capture-and-delivery-modalities) for the action axes. [How we continuously validate Cua Driver](/concepts/how-cua-driver-is-validated) explains why desktop support requires application-owned E2E evidence in addition to unit tests. Read [How sandboxes work](/concepts/how-sandboxes-work) when you need the model for disposable computers. Read [How Lume creates local macOS sandboxes](/concepts/how-lume-unattended-setup-works) for the local VM setup model, and [How SIP works in Lume VMs](/concepts/how-sip-works-in-lume-vms) for signed boot-policy changes. For evaluation, start with [What is Cua-Bench?](/concepts/what-is-cua-bench), then read [The Cua-Bench task lifecycle](/concepts/cua-bench-task-lifecycle).
diff --git a/docs/content/docs/concepts/meta.json b/docs/content/docs/concepts/meta.json
index 030f917219..22bfee672a 100644
--- a/docs/content/docs/concepts/meta.json
+++ b/docs/content/docs/concepts/meta.json
@@ -5,6 +5,7 @@
"index",
"what-is-computer-use",
"---Cua Driver---",
+ "choose-a-cua-driver-integration",
"sdk-mcp-and-hosting",
"the-no-foreground-contract",
"capture-and-delivery-modalities",
diff --git a/docs/content/docs/how-to-guides/driver/connect-your-agent.mdx b/docs/content/docs/how-to-guides/driver/connect-your-agent.mdx
index a9dfe8ee74..0d2d89e1e8 100644
--- a/docs/content/docs/how-to-guides/driver/connect-your-agent.mdx
+++ b/docs/content/docs/how-to-guides/driver/connect-your-agent.mdx
@@ -7,6 +7,10 @@ import { Callout } from 'fumadocs-ui/components/callout';
Cua Driver lets a [computer-use agent](/concepts/what-is-computer-use) drive the **host desktop**: installed apps, signed-in browser sessions, local files opened in apps, and the current OS user session. MCP-capable agents connect through `cua-driver mcp`; skill-based agents such as Prime Agent call the CLI directly. [Grok Bot](/use-cua-with/grok-bot) is a separate path; it is not Grok Build and does not use stdio MCP.
+Building Cua Driver into an application instead? Start with [Choose a Cua
+Driver integration](/concepts/choose-a-cua-driver-integration) to compare MCP,
+the direct SDK, private workers, and app-hosted services.
+
<Callout type="info">
This connects an agent to Cua Driver on the current machine. To create a disposable local desktop,
use [Cua Sandbox](/tutorials/your-first-local-sandbox) instead.
diff --git a/docs/content/docs/how-to-guides/driver/install.mdx b/docs/content/docs/how-to-guides/driver/install.mdx
index 9b9b74841e..ce38a52997 100644
--- a/docs/content/docs/how-to-guides/driver/install.mdx
+++ b/docs/content/docs/how-to-guides/driver/install.mdx
@@ -253,6 +253,7 @@ cua-driver permissions status
## Next steps
+- [Choose a Cua Driver integration](/concepts/choose-a-cua-driver-integration): decide whether your agent or product should use MCP, the SDK, or an app-hosted service.
- [Permission modes](/reference/cua-driver/permission-modes): what standard allows, and how bounded and unrestricted differ.
- [Install the Cua Driver agent skill](/how-to-guides/driver/install-agent-skill): add the cross-platform instructions from ClawHub or install them directly for another agent.
- [Keep Cua Driver running](/how-to-guides/driver/keep-running): configure autostart so the daemon comes back after reboots.
diff --git a/docs/content/docs/index.mdx b/docs/content/docs/index.mdx
index 77d4ce96db..8ad8ae750c 100644
--- a/docs/content/docs/index.mdx
+++ b/docs/content/docs/index.mdx
@@ -25,6 +25,10 @@ import { Card, Cards } from 'fumadocs-ui/components/card';
</Card>
</Cards>
+Building Cua Driver into your own product? Start with [Choose a Cua Driver
+integration](/concepts/choose-a-cua-driver-integration) to decide whether MCP,
+the SDK, or an app-hosted service should own the runtime.
+
## Cua Driver in action
These recordings show agents operating desktop apps while the user's active window stays in place. Each demo was recorded by the Cua team on the named platform.
diff --git a/docs/content/docs/tutorials/drive-your-first-app.mdx b/docs/content/docs/tutorials/drive-your-first-app.mdx
index 68feb48699..56c6affe5f 100644
--- a/docs/content/docs/tutorials/drive-your-first-app.mdx
+++ b/docs/content/docs/tutorials/drive-your-first-app.mdx
@@ -13,6 +13,10 @@ Linux.
New to agents that operate applications? Start with [What is computer use?](/concepts/what-is-computer-use) for the model behind the workflow, then return here to build one.
+This tutorial connects an existing agent to Cua Driver. If you are embedding
+Cua Driver in a product, start with [Choose a Cua Driver
+integration](/concepts/choose-a-cua-driver-integration) instead.
+
<VideoDemo
src="https://github.com/user-attachments/assets/11b248e5-d505-4349-aaae-ad9f95a1162f"
poster="https://github.com/user-attachments/assets/0fb62243-56ec-4d51-8771-35640a5e6031"