Skip to content

armgit / cua

Public Imported from GitHub
0

Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.

docs(cua-driver): add Grok Bot integration guide (#3112)

Francesco Bonaccic2ee81fd9f4d
diff --git a/docs/content/docs/use-cua-with/grok-bot.mdx b/docs/content/docs/use-cua-with/grok-bot.mdx
new file mode 100644
index 0000000000..6611840230
--- /dev/null
+++ b/docs/content/docs/use-cua-with/grok-bot.mdx
@@ -0,0 +1,93 @@
+---
+title: Grok Bot
+description: Let Grok Bot operate desktop apps through Cua Driver on your local computer.
+icon: Bot
+---
+
+import { Callout } from 'fumadocs-ui/components/callout';
+
+[Grok Bot](https://docs.x.ai/grok-bot/overview) runs on a persistent cloud computer. Cua Driver can give it a separate path to native apps on your Mac or Windows computer.
+
+Use Grok Bot's local-command execution for the first setup. This keeps Cua Driver on the computer it controls and leaves each command behind Grok Bot's local-computer approval policy.
+
+## Before you start
+
+Install Cua Driver on the computer with the apps you want to control, grant the required operating-system permissions, and start its daemon. Verify the local CLI before involving Grok Bot:
+
+```bash
+cua-driver --version
+cua-driver doctor
+cua-driver call list_apps '{}'
+```
+
+See [Install Cua Driver](/how-to-guides/driver/install) and [Keep Cua Driver running](/how-to-guides/driver/keep-running) if these commands do not succeed.
+
+## Allow local commands
+
+In Grok Bot, open **Settings → General → Agent → Execution on Local Computer** and choose **Ask every time**. This is xAI's default and lets you review each local command before it runs.
+
+The Bot's cloud computer and your local computer are separate. Make this boundary explicit in the task:
+
+```text
+Use command execution on my local computer for every cua-driver command.
+Do not install software, change Cua Driver permissions, or start an unrestricted
+daemon. Before each UI action, get fresh window state and use an element token
+from that response. Get fresh state after the action and verify the result.
+Stop for approval before sending, publishing, purchasing, deleting, changing
+permissions, or modifying a production system.
+```
+
+Start with a read-only request:
+
+```text
+Run `cua-driver call list_apps '{}'` on my local computer and tell me which apps
+have visible windows. Do not interact with them yet.
+```
+
+After that succeeds, give the Bot a narrow UI task. Cua Driver actions should follow this loop:
+
+1. Find the target app and window.
+2. Call `get_window_state` for that exact window.
+3. Act through an `element_token` from the latest state when one is available.
+4. Call `get_window_state` again and verify the expected result.
+
+Read [Agent action policy](/reference/cua-driver/action-selection-policy) for the full selection and escalation rules.
+
+<Callout type="warn">
+ Grok Bot's local-command approval controls whether a command may run. Cua Driver's permission mode controls what that command may do. Keep both boundaries enabled. Do not set Grok Bot to **Always allowed** or run Cua Driver in unrestricted mode for routine use.
+</Callout>
+
+## Save the workflow as a Grok Bot skill
+
+Once a safe task succeeds, ask Grok Bot to save the method as a private skill:
+
+```text
+Save the Cua Driver process we just used as a skill. Preserve the fresh-state,
+snapshot-bound element-token, and post-action verification rules. Require my
+approval for consequential actions. Keep Cua Driver commands on my local
+computer and report any refusal instead of bypassing it.
+```
+
+Test the saved skill on a disposable example before attaching it to a routine. Grok Bot routines can run while your laptop is closed, but a routine that depends on local commands also depends on your computer being reachable and Cua Driver running.
+
+## Custom MCP is an advanced route
+
+Grok Bot can use connectors and MCP servers. Cua Driver exposes MCP over local stdio and can also enable an authenticated Streamable HTTP endpoint. xAI requires a custom MCP server to be reachable from the public internet, so connecting the two requires an authenticated TLS tunnel or gateway.
+
+This route adds a public network boundary to desktop control. Use it only when you can provide all of the following:
+
+- A host-generated bearer token of at least 32 characters
+- A Cua Driver bounded capability manifest for the required tools and apps
+- A TLS endpoint that forwards only to Cua Driver's loopback MCP listener
+- A separate MCP connection for each Bot that needs an independent session
+- A way to revoke the endpoint and token when the task ends
+
+Cua Driver does not provide a managed public gateway or a Grok Bot connector preset. Review [SDK, MCP, and hosting](/concepts/sdk-mcp-and-hosting#service-transport-hardening) and [Restrict tool access](/how-to-guides/driver/restrict-tool-access) before building this route. xAI documents the other side in [Connectors](https://docs.x.ai/grok/connectors) and [Custom MCP Server Tunneling](https://docs.x.ai/grok/connectors/custom-mcp-tunneling).
+
+## Shared-state boundaries
+
+All Bots on one Grok Bot account share its cloud computer, browser sessions, files, and command-line credentials. Each screen is a separate work surface within the same security boundary. Grok Bot also installs connectors account-wide.
+
+Cua Driver sessions isolate transient state such as element cursors. They do not isolate the underlying desktop account or the apps running in it. Use separate operating-system accounts or machines when tasks require separate trust boundaries.
+
+Upstream: [Grok Bot overview](https://docs.x.ai/grok-bot/overview), [computer and apps](https://docs.x.ai/grok-bot/computer-and-apps), and [approvals, security, and privacy](https://docs.x.ai/grok-bot/approvals-security-and-privacy).
diff --git a/docs/content/docs/use-cua-with/index.mdx b/docs/content/docs/use-cua-with/index.mdx
index 4633230a01..7810b07f71 100644
--- a/docs/content/docs/use-cua-with/index.mdx
+++ b/docs/content/docs/use-cua-with/index.mdx
@@ -34,7 +34,7 @@ These pages show the verified harnesses and local runtimes that can put each mod
Use Antigravity CLI as the MCP-capable harness.
</Card>
<Card icon={<Orbit />} title="xAI / Grok" href="/use-cua-with/xai">
- Connect Grok Build to Cua Driver as a local MCP server.
+ Connect Grok Build or Grok Bot to Cua Driver.
</Card>
<Card icon={<SiMeta />} title="Meta" href="/use-cua-with/meta">
Run Muse Glimmer locally with a harness such as Claude Code.
@@ -56,6 +56,7 @@ Harnesses own the agent loop and invoke Cua Driver. Some are tied to a model pro
<Card icon={<SiOpenai />} title="OpenAI Codex" href="/use-cua-with/codex" />
<Card icon={<Rocket />} title="Antigravity CLI" href="/use-cua-with/antigravity-cli" />
<Card icon={<TerminalSquare />} title="Grok Build" href="/use-cua-with/grok-build" />
+ <Card icon={<Bot />} title="Grok Bot" href="/use-cua-with/grok-bot" />
<Card icon={<Waypoints />} title="Hermes" href="/use-cua-with/hermes" />
<Card icon={<Bot />} title="OpenClaw" href="/use-cua-with/openclaw" />
<Card icon={<PanelsTopLeft />} title="T3 Code" href="/use-cua-with/t3-code" />
diff --git a/docs/content/docs/use-cua-with/meta.json b/docs/content/docs/use-cua-with/meta.json
index 05df730496..776ca570d5 100644
--- a/docs/content/docs/use-cua-with/meta.json
+++ b/docs/content/docs/use-cua-with/meta.json
@@ -16,6 +16,7 @@
"codex",
"antigravity-cli",
"grok-build",
+ "grok-bot",
"hermes",
"openclaw",
"t3-code",
diff --git a/docs/content/docs/use-cua-with/xai.mdx b/docs/content/docs/use-cua-with/xai.mdx
index 2bf1c7886c..79477c93aa 100644
--- a/docs/content/docs/use-cua-with/xai.mdx
+++ b/docs/content/docs/use-cua-with/xai.mdx
@@ -1,17 +1,18 @@
---
title: xAI / Grok
-description: Use Grok models with Cua Driver through Grok Build.
+description: Use Grok Build or Grok Bot with Cua Driver.
icon: Orbit
---
-Grok Build is xAI's coding agent and supports local MCP servers. It is the shortest verified route from a Grok model to Cua Driver.
+Use Grok Build when the agent and Cua Driver run on the same computer. Use Grok Bot when you want a persistent cloud agent to operate apps on your local computer.
| Start here when… | Recommended path |
| ----------------------------------------- | -------------------------------------------------------------------------------- |
| You use Grok Build | [Connect Grok Build to Cua Driver](/use-cua-with/grok-build) |
+| You use Grok Bot | [Let Grok Bot operate local apps](/use-cua-with/grok-bot) |
| You are building a custom Grok agent | Call Cua Driver through [MCP](/reference/cua-driver/mcp-tools) or the CLI |
| You want to manage the agent from T3 Code | Configure Grok Build first, then [use it through T3 Code](/use-cua-with/t3-code) |
-Cua Driver does not call the Grok API directly. Grok Build owns the agent loop and launches Cua Driver as a local tool server.
+Cua Driver does not call the Grok API directly. Grok Build can launch Cua Driver as a local tool server. Grok Bot can invoke the local CLI under its local-computer policy or reach an authenticated Streamable HTTP MCP endpoint through a gateway.
-See xAI's [Grok Build overview](https://docs.x.ai/build/overview) and [MCP server guide](https://docs.x.ai/build/features/mcp-servers).
+See xAI's [Grok Build overview](https://docs.x.ai/build/overview), [MCP server guide](https://docs.x.ai/build/features/mcp-servers), and [Grok Bot overview](https://docs.x.ai/grok-bot/overview).