Skip to content

armgit / cua

Public Imported from GitHub
0

Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.

renamed HTML Desktops to Simulated Desktops

Dillon DuPont0e615556b03b
diff --git a/docs/content/docs/cuabench/guide/examples/genai-task-generation.mdx b/docs/content/docs/cuabench/guide/examples/genai-task-generation.mdx
index c1bc607312..a173aec9e8 100644
--- a/docs/content/docs/cuabench/guide/examples/genai-task-generation.mdx
+++ b/docs/content/docs/cuabench/guide/examples/genai-task-generation.mdx
@@ -382,5 +382,5 @@ cb interact tasks/python-quiz-env --oracle --no-wait
## Next Steps
- Learn about [Task Structure](/cuabench/guide/fundamentals/tasks) in depth
-- Explore [HTML Desktops](/cuabench/guide/fundamentals/html-desktops) for visual variants
+- Explore [Simulated Desktop](/cuabench/guide/fundamentals/simulated-desktop) for visual variants
- See [Creating Private Tasks](/cuabench/guide/advanced/creating-private-tasks) for advanced patterns
diff --git a/docs/content/docs/cuabench/guide/examples/universal-task.mdx b/docs/content/docs/cuabench/guide/examples/universal-task.mdx
index 8102e0d859..3839bb1dcd 100644
--- a/docs/content/docs/cuabench/guide/examples/universal-task.mdx
+++ b/docs/content/docs/cuabench/guide/examples/universal-task.mdx
@@ -1,9 +1,9 @@
---
title: Create a Universal Task
-description: Build a cross-platform Minesweeper game that works on QEMU, HTML desktops, and Docker
+description: Build a cross-platform Minesweeper game that works on QEMU, simulated desktop, and Docker
---
-In this tutorial, you'll create a Minesweeper game task using the Universal GUI API. This task is **universal** - it runs across QEMU, HTML desktops, and Docker environments without modification.
+In this tutorial, you'll create a Minesweeper game task using the Universal GUI API. This task is **universal** - it runs across QEMU, simulated desktop, and Docker environments without modification.
**Time:** ~15 minutes
**Prerequisites:** cua-bench installed
@@ -52,7 +52,7 @@ def load():
"mines": mines,
},
computer={
- "provider": "simulated", # Use HTML desktop for preview
+ "provider": "simulated", # Use simulated desktop for preview
"setup_config": {
"os_type": "linux",
"width": 800,
@@ -358,7 +358,7 @@ Create `gui/index.html` with the Minesweeper game:
cb interact tasks/minesweeper --task-id 0
```
-A browser window will open showing the Minesweeper game in an HTML desktop environment.
+A browser window will open showing the Minesweeper game in a simulated desktop environment.
### Run with Oracle
@@ -386,7 +386,7 @@ cb run task tasks/minesweeper --agent cua-agent --model anthropic/claude-sonnet-
## Why Universal?
This task works across all platforms:
-- **HTML Desktops**: Set `provider: "simulated"` for lightweight browser preview
+- **Simulated Desktop**: Set `provider: "simulated"` for lightweight browser preview
- **Docker**: Use real Linux desktop environments
- **QEMU**: Run on actual Windows/Linux VMs
@@ -394,6 +394,6 @@ The same HTML GUI and task code runs everywhere.
## Next Steps
-- Learn about [HTML Desktops](/cuabench/guide/fundamentals/html-desktops) for themed previews
+- Learn about [Simulated Desktop](/cuabench/guide/fundamentals/simulated-desktop) for themed previews
- See [Universal GUI](/cuabench/guide/fundamentals/universal-gui) for the full API
- Build a [Custom Agent](/cuabench/guide/examples/custom-agent) to solve tasks
diff --git a/docs/content/docs/cuabench/guide/fundamentals/meta.json b/docs/content/docs/cuabench/guide/fundamentals/meta.json
index ee7b09d754..c4abb69162 100644
--- a/docs/content/docs/cuabench/guide/fundamentals/meta.json
+++ b/docs/content/docs/cuabench/guide/fundamentals/meta.json
@@ -2,5 +2,5 @@
"title": "Fundamentals",
"description": "Core concepts of tasks and environments",
"icon": "Lightbulb",
- "pages": ["tasks", "universal-gui", "html-desktops", "adapters", "registry"]
+ "pages": ["tasks", "universal-gui", "simulated-desktop", "adapters", "registry"]
}
diff --git a/docs/content/docs/cuabench/guide/fundamentals/html-desktops.mdx b/docs/content/docs/cuabench/guide/fundamentals/simulated-desktop.mdx
similarity index 65%
rename from docs/content/docs/cuabench/guide/fundamentals/html-desktops.mdx
rename to docs/content/docs/cuabench/guide/fundamentals/simulated-desktop.mdx
index 75c2239c4c..017af20e0d 100644
--- a/docs/content/docs/cuabench/guide/fundamentals/html-desktops.mdx
+++ b/docs/content/docs/cuabench/guide/fundamentals/simulated-desktop.mdx
@@ -1,9 +1,11 @@
---
-title: HTML Desktops
-description: Preview tasks with themed desktop environments (no Docker required)
+title: Simulated Desktop (Preview)
+description: Preview feature for testing tasks with themed desktop environments (no Docker required)
---
-# HTML Desktops
+<Callout type="info">
+This is a **preview feature** for rapid development and testing.
+</Callout>
When you set `provider="simulated"` in your task config, you can preview it in a lightweight HTML/CSS desktop environment without Docker. This renders OS-themed window chrome, taskbars, and desktop styling around your GUI.
@@ -28,7 +30,7 @@ def load():
cb.Task(
description='Click the Submit button',
computer={
- "provider": "simulated", # Enables HTML desktop
+ "provider": "simulated", # Enables simulated desktop
"setup_config": {
"os_type": "win11", # linux, win11, or macos
"width": 1920,
@@ -70,12 +72,22 @@ def load():
]
```
-## Why Use HTML Desktops?
+## Limitations
+
+Simulated desktop runs in a browser, so it has some restrictions:
+
+- **No shell actions**: Can't execute bash commands or scripts
+- **No native apps**: Can't install or run real applications
+- **HTML/CSS only**: Only works with custom GUIs created via `launch_window()` or `install_app()`
+
+For tasks requiring shell access or native applications, use Docker or QEMU environments instead.
+
+## Why Use Simulated Desktop?
- **No Docker**: Runs directly in browser with Playwright
- **Fast**: Instant startup, no VM boot time
- **Visual Diversity**: Train agents on different OS appearances
- **Deterministic**: Same theme always renders identically
-- **Lightweight**: Run hundreds in parallel
+- **Lightweight**: Run thousands in parallel
-Use HTML desktops for quick iteration and agent development. Switch to Docker/QEMU when you need real OS environments.
+Use simulated desktop for quick iteration and agent development. Switch to Docker/QEMU when you need real OS environments.
diff --git a/docs/content/docs/cuabench/guide/fundamentals/tasks.mdx b/docs/content/docs/cuabench/guide/fundamentals/tasks.mdx
index 78e5611f5c..f0f64e4112 100644
--- a/docs/content/docs/cuabench/guide/fundamentals/tasks.mdx
+++ b/docs/content/docs/cuabench/guide/fundamentals/tasks.mdx
@@ -34,14 +34,14 @@ def load():
}
}
)
- for provider, os_type in [("computer", "linux"), ("computer", "windows"), ("webtop", "linux"), ("webtop", "win11"), ("webtop", "macos")]
+ for provider, os_type in [("native", "linux"), ("native", "windows"), ("simulated", "linux"), ("simulated", "win11"), ("simulated", "macos")]
# More task variants...
]
```
Each task variant can have different metadata, enabling thousands of unique scenarios from a single template.
-The `computer` provider **emulates** `windows` and `linux` operating systems, while the `simulated` provider renders HTML desktop themes including `linux`, `win11`, and `macos`. See [HTML Desktops](/cuabench/guide/fundamentals/html-desktops) for all available themes.
+The `native` provider **emulates** `windows` and `linux` operating systems using QEMU/Docker, while the `simulated` provider lightweight Playwright-based desktop environments with themes such as `linux`, `win11`, and `macos`. See [Simulated Desktop](/cuabench/guide/fundamentals/simulated-desktop) for all available themes.
### 2. Setup (`@cb.setup_task`)
diff --git a/docs/content/docs/cuabench/guide/fundamentals/universal-gui.mdx b/docs/content/docs/cuabench/guide/fundamentals/universal-gui.mdx
index fea64d1264..86a8e9a91c 100644
--- a/docs/content/docs/cuabench/guide/fundamentals/universal-gui.mdx
+++ b/docs/content/docs/cuabench/guide/fundamentals/universal-gui.mdx
@@ -3,8 +3,6 @@ title: Universal GUI
description: Package custom GUIs with your tasks using the window management API
---
-# Universal GUI
-
Cua-Bench provides an Electron-like API for packaging custom HTML/CSS/JS GUIs with your tasks. This allows you to create interactive GUI apps, GUI-based tools, or simulated clones of real applications when developing tasks.
## Window Management API
@@ -45,13 +43,13 @@ my_task/
└── index.html # Your HTML/CSS/JS GUI
```
-## Previewing with HTML Desktops
+## Previewing with Simulated Desktop
For lightweight previewing without Docker, set `provider: "simulated"` in your task config:
```python
computer={
- "provider": "simulated", # Enables HTML desktop preview
+ "provider": "simulated", # Enables simulated desktop preview
"setup_config": {
"os_type": "linux", # linux, win11, or macos theme
"width": 800,
@@ -66,13 +64,13 @@ Then preview with:
cb interact tasks/my_game --task-id 0
```
-The [HTML Desktop](/cuabench/guide/fundamentals/html-desktops) renders your GUI in a themed desktop environment (Linux/Windows/macOS) without requiring Docker.
+The [Simulated Desktop](/cuabench/guide/fundamentals/simulated-desktop) renders your GUI in a themed desktop environment (Linux/Windows/macOS) without requiring Docker.
## Key Features
- **JavaScript execution**: Communicate with your GUI via `execute_javascript()`
- **CSS selectors**: Click elements with `click_element(pid, "#button")`
- **State management**: Expose game/app state via `window.__variables`
-- **Cross-platform**: Works in Docker containers and HTML desktops
+- **Cross-platform**: Works in Docker containers and simulated desktop
See the [Create a Universal Task](/cuabench/guide/examples/universal-task) example for a complete walkthrough.