Skip to content

armgit / cua

Public Imported from GitHub
0

Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.

docs: update architecture images and improve documentation (#827)

Francesco Bonacci0b9cf2ff62f2
diff --git a/README.md b/README.md
index 282cacb12a..afc4ed772f 100644
--- a/README.md
+++ b/README.md
@@ -7,7 +7,7 @@
</picture>
</a>
- <p align="center">Any OS. Any agent. Self-hostable.</p>
+ <p align="center">Build, benchmark, and deploy agents that use computers</p>
<p align="center">
<a href="https://cua.ai" target="_blank" rel="noopener noreferrer"><img src="https://img.shields.io/badge/cua.ai-0ea5e9" alt="cua.ai"></a>
@@ -28,11 +28,11 @@
## Choose Your Path
-| If you want to... | Check out | For |
-| ---------------------------------------------- | --------------------------------------------------------- | ---------------------------- |
-| Build AI agents or run isolated code execution | [**Cua**](#cua---agentic-ui-automation--code-execution) | AI Engineers |
-| Benchmark or train computer-use models with RL | [**Cua-Bench**](#cua-bench---benchmarks--rl-environments) | Researchers |
-| Run macOS/Linux VMs on Apple Silicon | [**Lume**](#lume---macos-virtualization) | Developers, System Engineers |
+| | If you want to... | Check out |
+| - | ---------------------------------------------- | --------------------------------------------------------- |
+| [<img src="img/cua-architecture.png" width="150">](#cua---agentic-ui-automation--code-execution) | Build AI agents or run isolated code execution | [**Cua**](#cua---agentic-ui-automation--code-execution) |
+| [<img src="img/cua-bench-architecture.png" width="150">](#cua-bench---benchmarks--rl-environments) | Benchmark or train computer-use models with RL | [**Cua-Bench**](#cua-bench---benchmarks--rl-environments) |
+| [<img src="img/lume-architecture.png" width="150">](#lume---macos-virtualization) | Run macOS/Linux VMs on Apple Silicon | [**Lume**](#lume---macos-virtualization) |
---
@@ -40,6 +40,8 @@
Build agents that see screens, click buttons, and complete tasks autonomously. Run isolated code execution environments for AI coding assistants like Claude Code, Codex CLI, or OpenCode.
+<img src="img/cua-architecture.png" alt="Cua Architecture" width="100%">
+
```python
# Requires Python 3.12 or 3.13
from computer import Computer
@@ -60,6 +62,8 @@ async for result in agent.run([{"role": "user", "content": "Open Firefox and sea
Evaluate computer-use agents on OSWorld, ScreenSpot, Windows Arena, and custom tasks. Export trajectories for training.
+<img src="img/cua-bench-architecture.png" alt="Cua-Bench Architecture" width="100%">
+
```bash
# Install and create base image
cd cua-bench
@@ -77,6 +81,8 @@ cb run dataset datasets/cua-bench-basic --agent cua-agent --max-parallel 4
Create and manage macOS/Linux VMs with near-native performance on Apple Silicon using Apple's Virtualization.Framework.
+<img src="img/lume-architecture.png" alt="Lume Architecture" width="100%">
+
```bash
# Install Lume
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/lume/scripts/install.sh)"
diff --git a/docs/content/docs/cua/guide/get-started/what-is-cua.mdx b/docs/content/docs/cua/guide/get-started/what-is-cua.mdx
index 5f9b64eef8..b25568f097 100644
--- a/docs/content/docs/cua/guide/get-started/what-is-cua.mdx
+++ b/docs/content/docs/cua/guide/get-started/what-is-cua.mdx
@@ -3,7 +3,7 @@ title: What is Cua?
description: Learn about Cua, the open-source platform for building computer-use agents
---
-Cua is an **open-source platform** for building, deploying, and managing computer-use agents. It provides everything you need to create AI agents that can interact with desktop environments just like a human would—clicking, typing, navigating, and completing tasks autonomously.
+Cua is an open-source platform for building, benchmarking, and deploying agents that can use any computer, with isolated, self-hostable sandboxes (Docker, QEMU, Apple Vz).
<div className="not-prose relative rounded-xl overflow-hidden my-8 w-full">
<img src="/docs/img/hero.png" alt="Cua" className="w-full h-auto rounded-xl" />
@@ -13,6 +13,10 @@ Cua is an **open-source platform** for building, deploying, and managing compute
Cua consists of three main components:
+<div className="not-prose relative rounded-xl overflow-hidden my-8 w-full">
+ <img src="/docs/img/cua-architecture.png" alt="Cua Architecture" className="w-full h-auto rounded-xl" />
+</div>
+
### 1. Desktop Sandboxes
Isolated virtual environments where your agents can safely execute tasks. Cua supports:
@@ -27,11 +31,12 @@ A unified SDK for controlling desktop environments programmatically:
- Take screenshots and observe the screen
- Simulate mouse clicks, movements, and scrolling
- Type text and press keyboard shortcuts
+- Run code and shell commands
- Works identically across all sandbox types
### 3. Agent Framework
-Tools for building AI agents that understand and interact with GUIs:
+Build agents that see screens, click buttons, and complete tasks autonomously. Run isolated code execution environments for AI coding assistants like Claude Code, Codex CLI, or OpenCode.
- **100+ vision-language model options** through Cua VLM Router or direct provider access
- **Pre-built agent loops** optimized for computer-use tasks
@@ -40,24 +45,23 @@ Tools for building AI agents that understand and interact with GUIs:
## Why Cua?
-Traditional automation tools require you to write explicit scripts for every action. Cua agents can:
+Cua provides isolated, reproducible environments for AI agents to operate safely:
-- **Understand visual context** - See what's on screen and make decisions
-- **Adapt to changes** - Handle UI variations without breaking
-- **Complete complex tasks** - Chain multiple actions to achieve goals
-- **Work across applications** - Navigate between different programs seamlessly
+- **Secure execution** - Run AI coding assistants and computer-use agents in sandboxed environments
+- **Self-hostable** - Deploy locally with Docker, QEMU, or Apple Virtualization
+- **Cross-platform** - Same API across Linux, Windows, and macOS sandboxes
+- **Visual understanding** - Agents see screens, adapt to UI changes, and complete complex tasks
## Use Cases
Cua is ideal for:
+- **AI coding assistants** - Isolated code execution environments for Claude Code, Codex CLI, OpenCode, and other AI coding tools
+- **Computer-use agents** - Build agents that interact with any desktop application autonomously
- **Workflow automation** - Automate repetitive tasks across any application
- **Testing** - Run end-to-end tests that interact with real UIs
-- **Data extraction** - Scrape information from applications that don't have APIs
-- **RPA (Robotic Process Automation)** - Enterprise automation with AI understanding
-- **Benchmarks** - Evaluate computer-use agents on standardized tasks (OSWorld, ScreenSpot, etc.)
-- **RL/Gym Environments** - Train reinforcement learning agents with desktop environments as Gym-compatible envs
-- **Research** - Build and evaluate computer-use AI agents
+- **Benchmarks** - Evaluate agents on OSWorld, ScreenSpot, and other standardized tasks
+- **Research** - Build, evaluate, and train computer-use AI agents
## Getting Started
diff --git a/docs/content/docs/cuabench/guide/getting-started/introduction.mdx b/docs/content/docs/cuabench/guide/getting-started/introduction.mdx
index 9c042a793a..f1f4618c6a 100644
--- a/docs/content/docs/cuabench/guide/getting-started/introduction.mdx
+++ b/docs/content/docs/cuabench/guide/getting-started/introduction.mdx
@@ -19,10 +19,19 @@ Examples of Cua-Bench tasks are:
- Installing design software and editing images to remove backgrounds
- Using office applications to format and merge multiple documents
-Cua-Bench consists of two parts:
+Cua-Bench consists of three parts:
-1. **A dataset of tasks** - Verifiable, dynamic computer-use environments across multiple operating systems
-2. **An evaluation/training harness** - Tools for running benchmarks, generating data, and testing agents
+<div className="not-prose relative rounded-xl overflow-hidden my-8 w-full">
+ <img
+ src="/docs/img/cua-bench-architecture.png"
+ alt="Cua-Bench Architecture"
+ className="w-full h-auto rounded-xl"
+ />
+</div>
+
+1. **Base images** - Pre-packaged Windows, Linux, macOS, and Android environments with apps and dependencies needed for benchmark runs
+2. **A dataset of tasks** - Verifiable, dynamic computer-use environments across multiple operating systems
+3. **An evaluation/training harness** - Tools for running benchmarks, generating data, and testing agents
# Dataset of tasks
diff --git a/docs/content/docs/lume/guide/getting-started/introduction.mdx b/docs/content/docs/lume/guide/getting-started/introduction.mdx
index 8a98f167bc..4ed66d3883 100644
--- a/docs/content/docs/lume/guide/getting-started/introduction.mdx
+++ b/docs/content/docs/lume/guide/getting-started/introduction.mdx
@@ -7,6 +7,10 @@ import { Callout } from 'fumadocs-ui/components/callout';
Lume is a VM runtime for building AI agents, running CI/CD pipelines, and automating macOS. It uses Apple's native Virtualization Framework to run macOS and Linux VMs at near-native speed on Apple Silicon.
+<Callout type="info" title="Cloud macOS Sandboxes">
+We're piloting a managed service for customers who want to run cloud macOS sandboxes for CI/CD and agent workloads. [Book a demo](https://cal.com/cua/cua-demo?overlayCalendar=true) if you're interested.
+</Callout>
+
```bash
lume create test-vm --os macos --ipsw latest
lume run test-vm
@@ -49,9 +53,7 @@ Apple's Virtualization Framework—the same technology Lume is built on—powers
## What Lume doesn't do
-Lume runs VMs locally on your Mac. It's not a cloud service, container runtime, or cluster orchestrator. If you need to run VMs in the cloud or across multiple machines, look elsewhere.
-
-Lume also requires Apple Silicon—it won't work on Intel Macs or other platforms.
+Lume requires Apple Silicon—it won't work on Intel Macs or other platforms.
## Get started
diff --git a/docs/public/img/cua-architecture.png b/docs/public/img/cua-architecture.png
new file mode 100644
index 0000000000..7276f43e15
Binary files /dev/null and b/docs/public/img/cua-architecture.png differ
diff --git a/docs/public/img/cua-bench-architecture.png b/docs/public/img/cua-bench-architecture.png
new file mode 100644
index 0000000000..b937a2314a
Binary files /dev/null and b/docs/public/img/cua-bench-architecture.png differ
diff --git a/docs/public/img/hero.png b/docs/public/img/hero.png
index 17cb6e1e8e..f76a84d8d8 100644
Binary files a/docs/public/img/hero.png and b/docs/public/img/hero.png differ
diff --git a/docs/public/img/lume-architecture.png b/docs/public/img/lume-architecture.png
index 79f1d90078..c1661aa712 100644
Binary files a/docs/public/img/lume-architecture.png and b/docs/public/img/lume-architecture.png differ
diff --git a/docs/src/app/global.css b/docs/src/app/global.css
index 53c9ad28c2..124bb67330 100644
--- a/docs/src/app/global.css
+++ b/docs/src/app/global.css
@@ -11,6 +11,12 @@
--color-fd-accent: hsl(152, 76%, 92%); /* emerald-100 */
--font-sans: var(--font-geist-sans);
--font-mono: var(--font-geist-mono);
+ --font-heading: var(--font-urbanist);
+}
+
+/* Urbanist for headings */
+h1, h2, h3, h4 {
+ font-family: var(--font-heading), var(--font-sans), sans-serif;
}
.dark {
diff --git a/img/cua-architecture.png b/img/cua-architecture.png
new file mode 100644
index 0000000000..7276f43e15
Binary files /dev/null and b/img/cua-architecture.png differ
diff --git a/img/cua-bench-architecture.png b/img/cua-bench-architecture.png
new file mode 100644
index 0000000000..b937a2314a
Binary files /dev/null and b/img/cua-bench-architecture.png differ
diff --git a/img/lume-architecture.png b/img/lume-architecture.png
new file mode 100644
index 0000000000..c1661aa712
Binary files /dev/null and b/img/lume-architecture.png differ