Skip to content

armgit / cua

Public Imported from GitHub
0

Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.

Suggest installing cua-bench[all]

Dillon DuPont915adc8cb4e2
diff --git a/docs/content/docs/cuabench/guide/fundamentals/registry.mdx b/docs/content/docs/cuabench/guide/fundamentals/registry.mdx
index 87dda11199..c12169649c 100644
--- a/docs/content/docs/cuabench/guide/fundamentals/registry.mdx
+++ b/docs/content/docs/cuabench/guide/fundamentals/registry.mdx
@@ -5,6 +5,8 @@ description: The Cua-Bench registry of tasks and datasets.
The [Cua-Bench registry](https://cuabench.ai/cuabench/registry) contains a collection of pre-built tasks and datasets for evaluating computer-use agents.
+Datasets are now part of the main [cua repository](https://github.com/trycua/cua) at [`libs/cua-bench/datasets`](https://github.com/trycua/cua/tree/main/libs/cua-bench/datasets).
+
The goal of the registry is to be a community-based effort to increase the variety and quantity of useful, verifiable computer-use tasks.
## Using the Registry
@@ -34,36 +36,38 @@ The registry is automatically cloned when you first use the `--dataset` flag:
```bash
# Registry location
-~/.cua/cbregistry/
+~/.cua/cua/libs/cua-bench/datasets/
+
+# Update datasets
+cd ~/.cua/cua && git pull
+```
+
+Or set a custom location:
-# Update registry
-cd ~/.cua/cbregistry && git pull
+```bash
+export CUA_REGISTRY_HOME=/path/to/your/datasets
```
## Available Datasets
| Dataset | Tasks | Description | Golden Environment |
|---------|-------|-------------|-------------------|
-| `cua-bench-basic` | 13 | Basic UI interaction tasks (clicking, typing, dragging) | `linux-docker` |
-| `winarena` | 173 | Windows 11 desktop automation | `windows-qemu` |
-| `osworld` | 369 | Linux desktop automation | `linux-qemu` |
-| `webarena` | 812 | Web browsing tasks | `linux-docker` |
+| `cua-bench-basic` | 13 | Basic UI interaction tasks (clicking, typing, dragging) | `simulated` |
## Dataset Structure
Each dataset in the registry follows this structure:
```
-registry/
-└── datasets/
- └── cua-bench-basic/
- ├── meta.json # Dataset metadata
- └── tasks/
- ├── click-button/
- │ ├── main.py # Task implementation
- │ └── gui/
- │ └── index.html
- └── ...
+libs/cua-bench/datasets/
+└── cua-bench-basic/
+ ├── meta.json # Dataset metadata
+ └── tasks/
+ ├── click-button/
+ │ ├── main.py # Task implementation
+ │ └── gui/
+ │ └── index.html
+ └── ...
```
### meta.json Format
@@ -73,7 +77,7 @@ registry/
"name": "cua-bench-basic",
"version": "1.0.0",
"description": "Basic UI interaction tasks",
- "platform": "linux-docker",
+ "platform": "simulated",
"tasks": [
"click-button",
"typing-input",
@@ -86,9 +90,18 @@ registry/
To contribute tasks to the registry:
-1. Fork the [cua-bench-registry](https://github.com/trycua/cua-bench-registry) repository
-2. Create your task following the [task structure](/cuabench/guide/fundamentals/tasks)
-3. Test locally with `cb interact tasks/your_task`
-4. Submit a pull request
+1. Fork the [cua repository](https://github.com/trycua/cua)
+2. Create your task in `libs/cua-bench/datasets/<dataset-name>/tasks/`
+3. Follow the [task structure](/cuabench/guide/fundamentals/tasks)
+4. Test locally with `cb interact tasks/your_task`
+5. Submit a pull request to the main repository
+
+See [Creating Tasks](/cuabench/guide/creating-tasks) for task development guidance.
+
+## Registry Location
+
+By default, datasets are stored at:
+- **Default**: `~/.cua/cua/libs/cua-bench/datasets/`
+- **Custom**: Set `CUA_REGISTRY_HOME` environment variable
-See [Creating Private Tasks](/cuabench/guide/advanced/creating-private-tasks) for task development guidance.
\ No newline at end of file
+The registry is a shallow clone of the main cua repository, pulling only the datasets directory.
diff --git a/docs/content/docs/cuabench/guide/getting-started/installation.mdx b/docs/content/docs/cuabench/guide/getting-started/installation.mdx
index 19c87d7bdd..eeb20eb181 100644
--- a/docs/content/docs/cuabench/guide/getting-started/installation.mdx
+++ b/docs/content/docs/cuabench/guide/getting-started/installation.mdx
@@ -21,7 +21,7 @@ git clone https://github.com/trycua/cua.git
cd cua/libs/cua-bench
# Install cua-bench CLI
-uv tool install -e .
+uv tool install -e .[all]
# Install playwright
uv run python -m playwright install chromium
diff --git a/libs/cua-bench/pyproject.toml b/libs/cua-bench/pyproject.toml
index 363023c011..67e1c8b369 100644
--- a/libs/cua-bench/pyproject.toml
+++ b/libs/cua-bench/pyproject.toml
@@ -47,6 +47,8 @@ dependencies = [
"numpy>=2.0.0",
"python-dotenv>=1.0.0",
"psutil>=5.9.0",
+ "datasets>=3.0.0",
+ "matplotlib>=3.8.0",
# Docker integration
"docker>=7.0.0",
# CUA SDK
@@ -56,7 +58,7 @@ dependencies = [
[project.optional-dependencies]
# Full installation with all features
all = [
- "cua-bench[browser,agents,cloud,tracing]",
+ "cua-bench[browser,agents,cloud]",
]
# Browser automation (for simulated environments)
@@ -79,12 +81,6 @@ cloud = [
"grpcio>=1.60.0",
]
-# Tracing & datasets
-tracing = [
- "datasets>=3.0.0",
- "matplotlib>=3.8.0",
-]
-
# API server (for custom environments)
server = [
"fastapi>=0.100.0",