Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.
Rust agentai-agentapplecomputer-usecomputer-use-agentcontainerizationcuadesktop-automationhacktoberfestlumemacosmanusoperatorswiftvirtualizationvirtualization-frameworkwindowswindows-sandbox
Suggest installing cua-bench[all]
Dillon DuPont915adc8cb4e2
diff --git a/docs/content/docs/cuabench/guide/fundamentals/registry.mdx b/docs/content/docs/cuabench/guide/fundamentals/registry.mdxindex 87dda11199..c12169649c 100644--- a/docs/content/docs/cuabench/guide/fundamentals/registry.mdx+++ b/docs/content/docs/cuabench/guide/fundamentals/registry.mdx@@ -5,6 +5,8 @@ description: The Cua-Bench registry of tasks and datasets.The [Cua-Bench registry](https://cuabench.ai/cuabench/registry) contains a collection of pre-built tasks and datasets for evaluating computer-use agents.+Datasets are now part of the main [cua repository](https://github.com/trycua/cua) at [`libs/cua-bench/datasets`](https://github.com/trycua/cua/tree/main/libs/cua-bench/datasets).+The goal of the registry is to be a community-based effort to increase the variety and quantity of useful, verifiable computer-use tasks.## Using the Registry@@ -34,36 +36,38 @@ The registry is automatically cloned when you first use the `--dataset` flag:```bash# Registry location-~/.cua/cbregistry/+~/.cua/cua/libs/cua-bench/datasets/++# Update datasets+cd ~/.cua/cua && git pull+```++Or set a custom location:-# Update registry-cd ~/.cua/cbregistry && git pull+```bash+export CUA_REGISTRY_HOME=/path/to/your/datasets```## Available Datasets| Dataset | Tasks | Description | Golden Environment ||---------|-------|-------------|-------------------|-| `cua-bench-basic` | 13 | Basic UI interaction tasks (clicking, typing, dragging) | `linux-docker` |-| `winarena` | 173 | Windows 11 desktop automation | `windows-qemu` |-| `osworld` | 369 | Linux desktop automation | `linux-qemu` |-| `webarena` | 812 | Web browsing tasks | `linux-docker` |+| `cua-bench-basic` | 13 | Basic UI interaction tasks (clicking, typing, dragging) | `simulated` |## Dataset StructureEach dataset in the registry follows this structure:```-registry/-└── datasets/- └── cua-bench-basic/- ├── meta.json # Dataset metadata- └── tasks/- ├── click-button/- │ ├── main.py # Task implementation- │ └── gui/- │ └── index.html- └── ...+libs/cua-bench/datasets/+└── cua-bench-basic/+ ├── meta.json # Dataset metadata+ └── tasks/+ ├── click-button/+ │ ├── main.py # Task implementation+ │ └── gui/+ │ └── index.html+ └── ...```### meta.json Format@@ -73,7 +77,7 @@ registry/"name": "cua-bench-basic","version": "1.0.0","description": "Basic UI interaction tasks",- "platform": "linux-docker",+ "platform": "simulated","tasks": ["click-button","typing-input",@@ -86,9 +90,18 @@ registry/To contribute tasks to the registry:-1. Fork the [cua-bench-registry](https://github.com/trycua/cua-bench-registry) repository-2. Create your task following the [task structure](/cuabench/guide/fundamentals/tasks)-3. Test locally with `cb interact tasks/your_task`-4. Submit a pull request+1. Fork the [cua repository](https://github.com/trycua/cua)+2. Create your task in `libs/cua-bench/datasets/<dataset-name>/tasks/`+3. Follow the [task structure](/cuabench/guide/fundamentals/tasks)+4. Test locally with `cb interact tasks/your_task`+5. Submit a pull request to the main repository++See [Creating Tasks](/cuabench/guide/creating-tasks) for task development guidance.++## Registry Location++By default, datasets are stored at:+- **Default**: `~/.cua/cua/libs/cua-bench/datasets/`+- **Custom**: Set `CUA_REGISTRY_HOME` environment variable-See [Creating Private Tasks](/cuabench/guide/advanced/creating-private-tasks) for task development guidance.\ No newline at end of file+The registry is a shallow clone of the main cua repository, pulling only the datasets directory.diff --git a/docs/content/docs/cuabench/guide/getting-started/installation.mdx b/docs/content/docs/cuabench/guide/getting-started/installation.mdxindex 19c87d7bdd..eeb20eb181 100644--- a/docs/content/docs/cuabench/guide/getting-started/installation.mdx+++ b/docs/content/docs/cuabench/guide/getting-started/installation.mdx@@ -21,7 +21,7 @@ git clone https://github.com/trycua/cua.gitcd cua/libs/cua-bench# Install cua-bench CLI-uv tool install -e .+uv tool install -e .[all]# Install playwrightuv run python -m playwright install chromiumdiff --git a/libs/cua-bench/pyproject.toml b/libs/cua-bench/pyproject.tomlindex 363023c011..67e1c8b369 100644--- a/libs/cua-bench/pyproject.toml+++ b/libs/cua-bench/pyproject.toml@@ -47,6 +47,8 @@ dependencies = ["numpy>=2.0.0","python-dotenv>=1.0.0","psutil>=5.9.0",+ "datasets>=3.0.0",+ "matplotlib>=3.8.0",# Docker integration"docker>=7.0.0",# CUA SDK@@ -56,7 +58,7 @@ dependencies = [[project.optional-dependencies]# Full installation with all featuresall = [- "cua-bench[browser,agents,cloud,tracing]",+ "cua-bench[browser,agents,cloud]",]# Browser automation (for simulated environments)@@ -79,12 +81,6 @@ cloud = ["grpcio>=1.60.0",]-# Tracing & datasets-tracing = [- "datasets>=3.0.0",- "matplotlib>=3.8.0",-]-# API server (for custom environments)server = ["fastapi>=0.100.0",