Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.
Rust agentai-agentapplecomputer-usecomputer-use-agentcontainerizationcuadesktop-automationhacktoberfestlumemacosmanusoperatorswiftvirtualizationvirtualization-frameworkwindowswindows-sandbox
Move images to Advanced, move WAA to Adapters
Dillon DuPont91da0ad4ec91
diff --git a/docs/content/docs/cuabench/guide/getting-started/images.mdx b/docs/content/docs/cuabench/guide/advanced/images.mdxsimilarity index 100%rename from docs/content/docs/cuabench/guide/getting-started/images.mdxrename to docs/content/docs/cuabench/guide/advanced/images.mdxdiff --git a/docs/content/docs/cuabench/guide/advanced/meta.json b/docs/content/docs/cuabench/guide/advanced/meta.jsonindex faba4a499b..c01c6c1823 100644--- a/docs/content/docs/cuabench/guide/advanced/meta.json+++ b/docs/content/docs/cuabench/guide/advanced/meta.json@@ -2,5 +2,10 @@"title": "Advanced","description": "Build your own tasks and agents","icon": "Layers",- "pages": ["creating-private-tasks", "custom-agents", "configuration"]-}+ "pages": [+ "images",+ "creating-private-tasks",+ "custom-agents",+ "configuration"+ ]+}\ No newline at end of filediff --git a/docs/content/docs/cuabench/guide/fundamentals/adapters.mdx b/docs/content/docs/cuabench/guide/fundamentals/adapters.mdxnew file mode 100644index 0000000000..44bbadc9c3--- /dev/null+++ b/docs/content/docs/cuabench/guide/fundamentals/adapters.mdx@@ -0,0 +1,67 @@+---+title: Adapters+description: Running benchmarks across different operating systems and environments.+---++The **Cua-Bench** framework makes it easy to create adapters for existing computer-use benchmarks by implementing their setup/evaluators with the Python task specification.++# Supported Benchmarks++| Benchmark | Image | Tasks | Status | Description |+|-----------|--------------|-------|--------|-------------|+| Windows Arena | `windows-qemu` | 173 | Available | Windows 11 desktop automation across 12 apps |+| OSWorld | `linux-qemu` | 369 | Coming soon | Linux desktop automation |+| WebArena | `linux-docker` | 812 | Coming soon | Web browsing tasks |+| AndroidWorld | `android-qemu` | 200+ | Coming soon | Android mobile automation |+| macOSWorld | `macos-lume` | 100+ | Coming soon | macOS desktop automation |++## Example: Run Windows Arena++Windows Arena is a benchmark with 173 tasks across 12 Windows application domains.++<Callout title="Requirements" type="info">+ Windows Arena requires x86_64 Linux with KVM support (nested virtualization). Setup takes ~1-2 hours.+</Callout>++### Step 1: Create Windows Base Image++```bash+# Download Windows ISO and create base image (~1-2 hours)+cb image create windows-qemu --download-iso++# With WinArena benchmark apps pre-installed (recommended)+cb image create windows-qemu --download-iso --winarena-apps++# Monitor progress (VNC at http://localhost:8006)+docker logs -f cua-setup-windows+```++### Step 2: Verify Image (Optional)++```bash+# Start interactive shell to verify the image works+cb image shell windows-qemu++# Access VNC at http://localhost:8006+# Press Ctrl+C to stop when done verifying+```++### Step 3: Run Tasks++```bash+# Run specific task with oracle+cb interact tasks/winarena_adapter --variant-id 0 --oracle++# Run with agent (2-container architecture)+cb run task tasks/winarena_adapter \+ --agent cua-agent \+ --model anthropic/claude-sonnet-4-20250514 \+ --image windows-qemu++# Run entire dataset in parallel+cb run dataset datasets/winarena \+ --agent cua-agent \+ --model anthropic/claude-sonnet-4-20250514 \+ --image windows-qemu \+ --max-parallel 4+```diff --git a/docs/content/docs/cuabench/guide/fundamentals/meta.json b/docs/content/docs/cuabench/guide/fundamentals/meta.jsonindex 3cf887ba85..c5d4163dca 100644--- a/docs/content/docs/cuabench/guide/fundamentals/meta.json+++ b/docs/content/docs/cuabench/guide/fundamentals/meta.json@@ -2,5 +2,5 @@"title": "Fundamentals","description": "Core concepts of tasks and environments","icon": "Lightbulb",- "pages": ["tasks", "simulated-tasks", "os-themes", "native-tasks", "registry"]+ "pages": ["tasks", "simulated-tasks", "os-themes", "native-tasks", "adapters", "registry"]}diff --git a/docs/content/docs/cuabench/guide/fundamentals/tasks.mdx b/docs/content/docs/cuabench/guide/fundamentals/tasks.mdxindex 0a27646799..dc22e8ff4e 100644--- a/docs/content/docs/cuabench/guide/fundamentals/tasks.mdx+++ b/docs/content/docs/cuabench/guide/fundamentals/tasks.mdx@@ -3,11 +3,7 @@ title: Tasksdescription: An in-depth guide describing the structure of a Cua-Bench task.----<div className="not-prose relative rounded-xl overflow-hidden my-8 w-full">- <img src="/docs/img/cua_bench_overview.png" alt="Cua" className="w-full h-auto rounded-xl border" />-</div>--A Cua-Bench task is a Python module that defines a verifiable computer-use environment. Tasks use decorators to define configuration, setup, evaluation, and oracle solutions.+A Cua-Bench task is a Python module that defines a verifiable, cross-platform GUI task. Tasks use decorators to define configuration, setup, evaluation, and oracle solutions.## Task Structurediff --git a/docs/content/docs/cuabench/guide/getting-started/first-steps.mdx b/docs/content/docs/cuabench/guide/getting-started/first-steps.mdxindex 7b559d02cb..47150d507f 100644--- a/docs/content/docs/cuabench/guide/getting-started/first-steps.mdx+++ b/docs/content/docs/cuabench/guide/getting-started/first-steps.mdx@@ -114,57 +114,6 @@ REWARD COUNTYou can see an overview table of your past runs with `cb run list`-## Run Windows Arena--Windows Arena is a benchmark with 173 tasks across 12 Windows application domains.--<Callout title="Requirements" type="info">- Windows Arena requires x86_64 Linux with KVM support (nested virtualization). Setup takes ~1-2 hours.-</Callout>--### Step 1: Create Windows Base Image--```bash-# Download Windows ISO and create base image (~1-2 hours)-cb image create windows-qemu --download-iso--# With WinArena benchmark apps pre-installed (recommended)-cb image create windows-qemu --download-iso --winarena-apps--# Monitor progress (VNC at http://localhost:8006)-docker logs -f cua-setup-windows-```--### Step 2: Verify Image (Optional)--```bash-# Start interactive shell to verify the image works-cb image shell windows-qemu--# Access VNC at http://localhost:8006-# Press Ctrl+C to stop when done verifying-```--### Step 3: Run Tasks--```bash-# Run specific task with oracle-cb interact tasks/winarena_adapter --variant-id 0 --oracle--# Run with agent (2-container architecture)-cb run task tasks/winarena_adapter \- --agent cua-agent \- --model anthropic/claude-sonnet-4-20250514 \- --image windows-qemu--# Run entire dataset in parallel-cb run dataset datasets/winarena \- --agent cua-agent \- --model anthropic/claude-sonnet-4-20250514 \- --image windows-qemu \- --max-parallel 4-```-## View Past Runs```bashdiff --git a/docs/content/docs/cuabench/guide/getting-started/installation.mdx b/docs/content/docs/cuabench/guide/getting-started/installation.mdxindex a50eb6493f..3ad16bf19d 100644--- a/docs/content/docs/cuabench/guide/getting-started/installation.mdx+++ b/docs/content/docs/cuabench/guide/getting-started/installation.mdx@@ -33,9 +33,9 @@ docker build -t cua-bench:latest .After installation, you can run the CLI using `cua-bench` or `cb` for convenience.-## Create Base Image+## Install a default Base Image-Base images are reproducible environments for running benchmarks. Create at least one:+Base images are reproducible computer environments for running tasks. Create at least one:```bash# Quick setup (Linux container, no KVM required)@@ -45,6 +45,15 @@ cb image create linux-dockercb image create windows-qemu --download-iso```+| Type | Description | Requirements |+|------|-------------|--------------|+| `linux-docker` | Linux GUI container (XFCE) | Docker |+| `linux-qemu` | Linux VM with QEMU/KVM | Docker + KVM |+| `windows-qemu` | Windows 11 VM | Docker + KVM |+| `android-qemu` | Android VM | Docker + KVM |+| `macos-lume` | macOS VM via Lume | Apple Silicon |++See [Base Images](/cuabench/guide/getting-started/images) for more options.## Verify Installationdiff --git a/docs/content/docs/cuabench/guide/getting-started/introduction.mdx b/docs/content/docs/cuabench/guide/getting-started/introduction.mdxindex 48aeb9d38a..0fee429386 100644--- a/docs/content/docs/cuabench/guide/getting-started/introduction.mdx+++ b/docs/content/docs/cuabench/guide/getting-started/introduction.mdx@@ -38,33 +38,8 @@ The harness provides familiar APIs for creating verifiable tasks. Use an ElectroAfter [installing the package](/cuabench/guide/getting-started/installation), you can run the harness using `cb run` or `cb interact`.-# Base Images--Cua-Bench uses **base images** - reproducible environments that provide consistent, isolated desktop or mobile VMs. Images are protected by overlays during benchmark runs, ensuring each task starts from a clean state.--| Type | Description | Requirements |-|------|-------------|--------------|-| `linux-docker` | Linux GUI container (XFCE) | Docker |-| `linux-qemu` | Linux VM with QEMU/KVM | Docker + KVM |-| `windows-qemu` | Windows 11 VM | Docker + KVM |-| `android-qemu` | Android VM | Docker + KVM |-| `macos-lume` | macOS VM via Lume | Apple Silicon |--See [Base Images](/cuabench/guide/getting-started/images) for setup instructions.--# Supported Benchmarks--| Benchmark | Image | Tasks | Status | Description |-|-----------|--------------|-------|--------|-------------|-| Windows Arena | `windows-qemu` | 173 | Available | Windows 11 desktop automation across 12 apps |-| OSWorld | `linux-qemu` | 369 | Coming soon | Linux desktop automation |-| WebArena | `linux-docker` | 812 | Coming soon | Web browsing tasks |-| AndroidWorld | `android-qemu` | 200+ | Coming soon | Android mobile automation |-| macOSWorld | `macos-lume` | 100+ | Coming soon | macOS desktop automation |-# Next Steps1. [Install the CLI](/cuabench/guide/getting-started/installation)-2. [Create a base image](/cuabench/guide/getting-started/images)-3. [Run your first benchmark](/cuabench/guide/getting-started/first-steps)-4. [Create custom tasks](/cuabench/guide/fundamentals/tasks)\ No newline at end of file+2. [Run your first benchmark](/cuabench/guide/getting-started/first-steps)+3. [Create custom tasks](/cuabench/guide/fundamentals/tasks)\ No newline at end of filediff --git a/docs/content/docs/cuabench/guide/getting-started/meta.json b/docs/content/docs/cuabench/guide/getting-started/meta.jsonindex 66aa74847f..283267d460 100644--- a/docs/content/docs/cuabench/guide/getting-started/meta.json+++ b/docs/content/docs/cuabench/guide/getting-started/meta.json@@ -3,5 +3,9 @@"description": "Get started with Cua-Bench","icon": "Rocket","defaultOpen": true,- "pages": ["introduction", "installation", "images", "first-steps"]-}+ "pages": [+ "introduction",+ "installation",+ "first-steps"+ ]+}\ No newline at end of file