Skip to content

armgit / cua

Public Imported from GitHub
0

Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.

Move images to Advanced, move WAA to Adapters

Dillon DuPont91da0ad4ec91
diff --git a/docs/content/docs/cuabench/guide/getting-started/images.mdx b/docs/content/docs/cuabench/guide/advanced/images.mdx
similarity index 100%
rename from docs/content/docs/cuabench/guide/getting-started/images.mdx
rename to docs/content/docs/cuabench/guide/advanced/images.mdx
diff --git a/docs/content/docs/cuabench/guide/advanced/meta.json b/docs/content/docs/cuabench/guide/advanced/meta.json
index faba4a499b..c01c6c1823 100644
--- a/docs/content/docs/cuabench/guide/advanced/meta.json
+++ b/docs/content/docs/cuabench/guide/advanced/meta.json
@@ -2,5 +2,10 @@
"title": "Advanced",
"description": "Build your own tasks and agents",
"icon": "Layers",
- "pages": ["creating-private-tasks", "custom-agents", "configuration"]
-}
+ "pages": [
+ "images",
+ "creating-private-tasks",
+ "custom-agents",
+ "configuration"
+ ]
+}
\ No newline at end of file
diff --git a/docs/content/docs/cuabench/guide/fundamentals/adapters.mdx b/docs/content/docs/cuabench/guide/fundamentals/adapters.mdx
new file mode 100644
index 0000000000..44bbadc9c3
--- /dev/null
+++ b/docs/content/docs/cuabench/guide/fundamentals/adapters.mdx
@@ -0,0 +1,67 @@
+---
+title: Adapters
+description: Running benchmarks across different operating systems and environments.
+---
+
+The **Cua-Bench** framework makes it easy to create adapters for existing computer-use benchmarks by implementing their setup/evaluators with the Python task specification.
+
+# Supported Benchmarks
+
+| Benchmark | Image | Tasks | Status | Description |
+|-----------|--------------|-------|--------|-------------|
+| Windows Arena | `windows-qemu` | 173 | Available | Windows 11 desktop automation across 12 apps |
+| OSWorld | `linux-qemu` | 369 | Coming soon | Linux desktop automation |
+| WebArena | `linux-docker` | 812 | Coming soon | Web browsing tasks |
+| AndroidWorld | `android-qemu` | 200+ | Coming soon | Android mobile automation |
+| macOSWorld | `macos-lume` | 100+ | Coming soon | macOS desktop automation |
+
+## Example: Run Windows Arena
+
+Windows Arena is a benchmark with 173 tasks across 12 Windows application domains.
+
+<Callout title="Requirements" type="info">
+ Windows Arena requires x86_64 Linux with KVM support (nested virtualization). Setup takes ~1-2 hours.
+</Callout>
+
+### Step 1: Create Windows Base Image
+
+```bash
+# Download Windows ISO and create base image (~1-2 hours)
+cb image create windows-qemu --download-iso
+
+# With WinArena benchmark apps pre-installed (recommended)
+cb image create windows-qemu --download-iso --winarena-apps
+
+# Monitor progress (VNC at http://localhost:8006)
+docker logs -f cua-setup-windows
+```
+
+### Step 2: Verify Image (Optional)
+
+```bash
+# Start interactive shell to verify the image works
+cb image shell windows-qemu
+
+# Access VNC at http://localhost:8006
+# Press Ctrl+C to stop when done verifying
+```
+
+### Step 3: Run Tasks
+
+```bash
+# Run specific task with oracle
+cb interact tasks/winarena_adapter --variant-id 0 --oracle
+
+# Run with agent (2-container architecture)
+cb run task tasks/winarena_adapter \
+ --agent cua-agent \
+ --model anthropic/claude-sonnet-4-20250514 \
+ --image windows-qemu
+
+# Run entire dataset in parallel
+cb run dataset datasets/winarena \
+ --agent cua-agent \
+ --model anthropic/claude-sonnet-4-20250514 \
+ --image windows-qemu \
+ --max-parallel 4
+```
diff --git a/docs/content/docs/cuabench/guide/fundamentals/meta.json b/docs/content/docs/cuabench/guide/fundamentals/meta.json
index 3cf887ba85..c5d4163dca 100644
--- a/docs/content/docs/cuabench/guide/fundamentals/meta.json
+++ b/docs/content/docs/cuabench/guide/fundamentals/meta.json
@@ -2,5 +2,5 @@
"title": "Fundamentals",
"description": "Core concepts of tasks and environments",
"icon": "Lightbulb",
- "pages": ["tasks", "simulated-tasks", "os-themes", "native-tasks", "registry"]
+ "pages": ["tasks", "simulated-tasks", "os-themes", "native-tasks", "adapters", "registry"]
}
diff --git a/docs/content/docs/cuabench/guide/fundamentals/tasks.mdx b/docs/content/docs/cuabench/guide/fundamentals/tasks.mdx
index 0a27646799..dc22e8ff4e 100644
--- a/docs/content/docs/cuabench/guide/fundamentals/tasks.mdx
+++ b/docs/content/docs/cuabench/guide/fundamentals/tasks.mdx
@@ -3,11 +3,7 @@ title: Tasks
description: An in-depth guide describing the structure of a Cua-Bench task.
---
-<div className="not-prose relative rounded-xl overflow-hidden my-8 w-full">
- <img src="/docs/img/cua_bench_overview.png" alt="Cua" className="w-full h-auto rounded-xl border" />
-</div>
-
-A Cua-Bench task is a Python module that defines a verifiable computer-use environment. Tasks use decorators to define configuration, setup, evaluation, and oracle solutions.
+A Cua-Bench task is a Python module that defines a verifiable, cross-platform GUI task. Tasks use decorators to define configuration, setup, evaluation, and oracle solutions.
## Task Structure
diff --git a/docs/content/docs/cuabench/guide/getting-started/first-steps.mdx b/docs/content/docs/cuabench/guide/getting-started/first-steps.mdx
index 7b559d02cb..47150d507f 100644
--- a/docs/content/docs/cuabench/guide/getting-started/first-steps.mdx
+++ b/docs/content/docs/cuabench/guide/getting-started/first-steps.mdx
@@ -114,57 +114,6 @@ REWARD COUNT
You can see an overview table of your past runs with `cb run list`
-## Run Windows Arena
-
-Windows Arena is a benchmark with 173 tasks across 12 Windows application domains.
-
-<Callout title="Requirements" type="info">
- Windows Arena requires x86_64 Linux with KVM support (nested virtualization). Setup takes ~1-2 hours.
-</Callout>
-
-### Step 1: Create Windows Base Image
-
-```bash
-# Download Windows ISO and create base image (~1-2 hours)
-cb image create windows-qemu --download-iso
-
-# With WinArena benchmark apps pre-installed (recommended)
-cb image create windows-qemu --download-iso --winarena-apps
-
-# Monitor progress (VNC at http://localhost:8006)
-docker logs -f cua-setup-windows
-```
-
-### Step 2: Verify Image (Optional)
-
-```bash
-# Start interactive shell to verify the image works
-cb image shell windows-qemu
-
-# Access VNC at http://localhost:8006
-# Press Ctrl+C to stop when done verifying
-```
-
-### Step 3: Run Tasks
-
-```bash
-# Run specific task with oracle
-cb interact tasks/winarena_adapter --variant-id 0 --oracle
-
-# Run with agent (2-container architecture)
-cb run task tasks/winarena_adapter \
- --agent cua-agent \
- --model anthropic/claude-sonnet-4-20250514 \
- --image windows-qemu
-
-# Run entire dataset in parallel
-cb run dataset datasets/winarena \
- --agent cua-agent \
- --model anthropic/claude-sonnet-4-20250514 \
- --image windows-qemu \
- --max-parallel 4
-```
-
## View Past Runs
```bash
diff --git a/docs/content/docs/cuabench/guide/getting-started/installation.mdx b/docs/content/docs/cuabench/guide/getting-started/installation.mdx
index a50eb6493f..3ad16bf19d 100644
--- a/docs/content/docs/cuabench/guide/getting-started/installation.mdx
+++ b/docs/content/docs/cuabench/guide/getting-started/installation.mdx
@@ -33,9 +33,9 @@ docker build -t cua-bench:latest .
After installation, you can run the CLI using `cua-bench` or `cb` for convenience.
-## Create Base Image
+## Install a default Base Image
-Base images are reproducible environments for running benchmarks. Create at least one:
+Base images are reproducible computer environments for running tasks. Create at least one:
```bash
# Quick setup (Linux container, no KVM required)
@@ -45,6 +45,15 @@ cb image create linux-docker
cb image create windows-qemu --download-iso
```
+| Type | Description | Requirements |
+|------|-------------|--------------|
+| `linux-docker` | Linux GUI container (XFCE) | Docker |
+| `linux-qemu` | Linux VM with QEMU/KVM | Docker + KVM |
+| `windows-qemu` | Windows 11 VM | Docker + KVM |
+| `android-qemu` | Android VM | Docker + KVM |
+| `macos-lume` | macOS VM via Lume | Apple Silicon |
+
+
See [Base Images](/cuabench/guide/getting-started/images) for more options.
## Verify Installation
diff --git a/docs/content/docs/cuabench/guide/getting-started/introduction.mdx b/docs/content/docs/cuabench/guide/getting-started/introduction.mdx
index 48aeb9d38a..0fee429386 100644
--- a/docs/content/docs/cuabench/guide/getting-started/introduction.mdx
+++ b/docs/content/docs/cuabench/guide/getting-started/introduction.mdx
@@ -38,33 +38,8 @@ The harness provides familiar APIs for creating verifiable tasks. Use an Electro
After [installing the package](/cuabench/guide/getting-started/installation), you can run the harness using `cb run` or `cb interact`.
-# Base Images
-
-Cua-Bench uses **base images** - reproducible environments that provide consistent, isolated desktop or mobile VMs. Images are protected by overlays during benchmark runs, ensuring each task starts from a clean state.
-
-| Type | Description | Requirements |
-|------|-------------|--------------|
-| `linux-docker` | Linux GUI container (XFCE) | Docker |
-| `linux-qemu` | Linux VM with QEMU/KVM | Docker + KVM |
-| `windows-qemu` | Windows 11 VM | Docker + KVM |
-| `android-qemu` | Android VM | Docker + KVM |
-| `macos-lume` | macOS VM via Lume | Apple Silicon |
-
-See [Base Images](/cuabench/guide/getting-started/images) for setup instructions.
-
-# Supported Benchmarks
-
-| Benchmark | Image | Tasks | Status | Description |
-|-----------|--------------|-------|--------|-------------|
-| Windows Arena | `windows-qemu` | 173 | Available | Windows 11 desktop automation across 12 apps |
-| OSWorld | `linux-qemu` | 369 | Coming soon | Linux desktop automation |
-| WebArena | `linux-docker` | 812 | Coming soon | Web browsing tasks |
-| AndroidWorld | `android-qemu` | 200+ | Coming soon | Android mobile automation |
-| macOSWorld | `macos-lume` | 100+ | Coming soon | macOS desktop automation |
-
# Next Steps
1. [Install the CLI](/cuabench/guide/getting-started/installation)
-2. [Create a base image](/cuabench/guide/getting-started/images)
-3. [Run your first benchmark](/cuabench/guide/getting-started/first-steps)
-4. [Create custom tasks](/cuabench/guide/fundamentals/tasks)
\ No newline at end of file
+2. [Run your first benchmark](/cuabench/guide/getting-started/first-steps)
+3. [Create custom tasks](/cuabench/guide/fundamentals/tasks)
\ No newline at end of file
diff --git a/docs/content/docs/cuabench/guide/getting-started/meta.json b/docs/content/docs/cuabench/guide/getting-started/meta.json
index 66aa74847f..283267d460 100644
--- a/docs/content/docs/cuabench/guide/getting-started/meta.json
+++ b/docs/content/docs/cuabench/guide/getting-started/meta.json
@@ -3,5 +3,9 @@
"description": "Get started with Cua-Bench",
"icon": "Rocket",
"defaultOpen": true,
- "pages": ["introduction", "installation", "images", "first-steps"]
-}
+ "pages": [
+ "introduction",
+ "installation",
+ "first-steps"
+ ]
+}
\ No newline at end of file