Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.
Rust agentai-agentapplecomputer-usecomputer-use-agentcontainerizationcuadesktop-automationhacktoberfestlumemacosmanusoperatorswiftvirtualizationvirtualization-frameworkwindowswindows-sandbox
docs(cua-driver): add Grok Bot integration guide (#3112)
Francesco Bonaccic2ee81fd9f4d
diff --git a/docs/content/docs/use-cua-with/grok-bot.mdx b/docs/content/docs/use-cua-with/grok-bot.mdxnew file mode 100644index 0000000000..6611840230--- /dev/null+++ b/docs/content/docs/use-cua-with/grok-bot.mdx@@ -0,0 +1,93 @@+---+title: Grok Bot+description: Let Grok Bot operate desktop apps through Cua Driver on your local computer.+icon: Bot+---++import { Callout } from 'fumadocs-ui/components/callout';++[Grok Bot](https://docs.x.ai/grok-bot/overview) runs on a persistent cloud computer. Cua Driver can give it a separate path to native apps on your Mac or Windows computer.++Use Grok Bot's local-command execution for the first setup. This keeps Cua Driver on the computer it controls and leaves each command behind Grok Bot's local-computer approval policy.++## Before you start++Install Cua Driver on the computer with the apps you want to control, grant the required operating-system permissions, and start its daemon. Verify the local CLI before involving Grok Bot:++```bash+cua-driver --version+cua-driver doctor+cua-driver call list_apps '{}'+```++See [Install Cua Driver](/how-to-guides/driver/install) and [Keep Cua Driver running](/how-to-guides/driver/keep-running) if these commands do not succeed.++## Allow local commands++In Grok Bot, open **Settings → General → Agent → Execution on Local Computer** and choose **Ask every time**. This is xAI's default and lets you review each local command before it runs.++The Bot's cloud computer and your local computer are separate. Make this boundary explicit in the task:++```text+Use command execution on my local computer for every cua-driver command.+Do not install software, change Cua Driver permissions, or start an unrestricted+daemon. Before each UI action, get fresh window state and use an element token+from that response. Get fresh state after the action and verify the result.+Stop for approval before sending, publishing, purchasing, deleting, changing+permissions, or modifying a production system.+```++Start with a read-only request:++```text+Run `cua-driver call list_apps '{}'` on my local computer and tell me which apps+have visible windows. Do not interact with them yet.+```++After that succeeds, give the Bot a narrow UI task. Cua Driver actions should follow this loop:++1. Find the target app and window.+2. Call `get_window_state` for that exact window.+3. Act through an `element_token` from the latest state when one is available.+4. Call `get_window_state` again and verify the expected result.++Read [Agent action policy](/reference/cua-driver/action-selection-policy) for the full selection and escalation rules.++<Callout type="warn">+ Grok Bot's local-command approval controls whether a command may run. Cua Driver's permission mode controls what that command may do. Keep both boundaries enabled. Do not set Grok Bot to **Always allowed** or run Cua Driver in unrestricted mode for routine use.+</Callout>++## Save the workflow as a Grok Bot skill++Once a safe task succeeds, ask Grok Bot to save the method as a private skill:++```text+Save the Cua Driver process we just used as a skill. Preserve the fresh-state,+snapshot-bound element-token, and post-action verification rules. Require my+approval for consequential actions. Keep Cua Driver commands on my local+computer and report any refusal instead of bypassing it.+```++Test the saved skill on a disposable example before attaching it to a routine. Grok Bot routines can run while your laptop is closed, but a routine that depends on local commands also depends on your computer being reachable and Cua Driver running.++## Custom MCP is an advanced route++Grok Bot can use connectors and MCP servers. Cua Driver exposes MCP over local stdio and can also enable an authenticated Streamable HTTP endpoint. xAI requires a custom MCP server to be reachable from the public internet, so connecting the two requires an authenticated TLS tunnel or gateway.++This route adds a public network boundary to desktop control. Use it only when you can provide all of the following:++- A host-generated bearer token of at least 32 characters+- A Cua Driver bounded capability manifest for the required tools and apps+- A TLS endpoint that forwards only to Cua Driver's loopback MCP listener+- A separate MCP connection for each Bot that needs an independent session+- A way to revoke the endpoint and token when the task ends++Cua Driver does not provide a managed public gateway or a Grok Bot connector preset. Review [SDK, MCP, and hosting](/concepts/sdk-mcp-and-hosting#service-transport-hardening) and [Restrict tool access](/how-to-guides/driver/restrict-tool-access) before building this route. xAI documents the other side in [Connectors](https://docs.x.ai/grok/connectors) and [Custom MCP Server Tunneling](https://docs.x.ai/grok/connectors/custom-mcp-tunneling).++## Shared-state boundaries++All Bots on one Grok Bot account share its cloud computer, browser sessions, files, and command-line credentials. Each screen is a separate work surface within the same security boundary. Grok Bot also installs connectors account-wide.++Cua Driver sessions isolate transient state such as element cursors. They do not isolate the underlying desktop account or the apps running in it. Use separate operating-system accounts or machines when tasks require separate trust boundaries.++Upstream: [Grok Bot overview](https://docs.x.ai/grok-bot/overview), [computer and apps](https://docs.x.ai/grok-bot/computer-and-apps), and [approvals, security, and privacy](https://docs.x.ai/grok-bot/approvals-security-and-privacy).diff --git a/docs/content/docs/use-cua-with/index.mdx b/docs/content/docs/use-cua-with/index.mdxindex 4633230a01..7810b07f71 100644--- a/docs/content/docs/use-cua-with/index.mdx+++ b/docs/content/docs/use-cua-with/index.mdx@@ -34,7 +34,7 @@ These pages show the verified harnesses and local runtimes that can put each modUse Antigravity CLI as the MCP-capable harness.</Card><Card icon={<Orbit />} title="xAI / Grok" href="/use-cua-with/xai">- Connect Grok Build to Cua Driver as a local MCP server.+ Connect Grok Build or Grok Bot to Cua Driver.</Card><Card icon={<SiMeta />} title="Meta" href="/use-cua-with/meta">Run Muse Glimmer locally with a harness such as Claude Code.@@ -56,6 +56,7 @@ Harnesses own the agent loop and invoke Cua Driver. Some are tied to a model pro<Card icon={<SiOpenai />} title="OpenAI Codex" href="/use-cua-with/codex" /><Card icon={<Rocket />} title="Antigravity CLI" href="/use-cua-with/antigravity-cli" /><Card icon={<TerminalSquare />} title="Grok Build" href="/use-cua-with/grok-build" />+ <Card icon={<Bot />} title="Grok Bot" href="/use-cua-with/grok-bot" /><Card icon={<Waypoints />} title="Hermes" href="/use-cua-with/hermes" /><Card icon={<Bot />} title="OpenClaw" href="/use-cua-with/openclaw" /><Card icon={<PanelsTopLeft />} title="T3 Code" href="/use-cua-with/t3-code" />diff --git a/docs/content/docs/use-cua-with/meta.json b/docs/content/docs/use-cua-with/meta.jsonindex 05df730496..776ca570d5 100644--- a/docs/content/docs/use-cua-with/meta.json+++ b/docs/content/docs/use-cua-with/meta.json@@ -16,6 +16,7 @@"codex","antigravity-cli","grok-build",+ "grok-bot","hermes","openclaw","t3-code",diff --git a/docs/content/docs/use-cua-with/xai.mdx b/docs/content/docs/use-cua-with/xai.mdxindex 2bf1c7886c..79477c93aa 100644--- a/docs/content/docs/use-cua-with/xai.mdx+++ b/docs/content/docs/use-cua-with/xai.mdx@@ -1,17 +1,18 @@---title: xAI / Grok-description: Use Grok models with Cua Driver through Grok Build.+description: Use Grok Build or Grok Bot with Cua Driver.icon: Orbit----Grok Build is xAI's coding agent and supports local MCP servers. It is the shortest verified route from a Grok model to Cua Driver.+Use Grok Build when the agent and Cua Driver run on the same computer. Use Grok Bot when you want a persistent cloud agent to operate apps on your local computer.| Start here when… | Recommended path || ----------------------------------------- | -------------------------------------------------------------------------------- || You use Grok Build | [Connect Grok Build to Cua Driver](/use-cua-with/grok-build) |+| You use Grok Bot | [Let Grok Bot operate local apps](/use-cua-with/grok-bot) || You are building a custom Grok agent | Call Cua Driver through [MCP](/reference/cua-driver/mcp-tools) or the CLI || You want to manage the agent from T3 Code | Configure Grok Build first, then [use it through T3 Code](/use-cua-with/t3-code) |-Cua Driver does not call the Grok API directly. Grok Build owns the agent loop and launches Cua Driver as a local tool server.+Cua Driver does not call the Grok API directly. Grok Build can launch Cua Driver as a local tool server. Grok Bot can invoke the local CLI under its local-computer policy or reach an authenticated Streamable HTTP MCP endpoint through a gateway.-See xAI's [Grok Build overview](https://docs.x.ai/build/overview) and [MCP server guide](https://docs.x.ai/build/features/mcp-servers).+See xAI's [Grok Build overview](https://docs.x.ai/build/overview), [MCP server guide](https://docs.x.ai/build/features/mcp-servers), and [Grok Bot overview](https://docs.x.ai/grok-bot/overview).