Skip to content

armgit / cua

Public Imported from GitHub
0

Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.

updated docs

Dillon DuPont5902be2917dd
diff --git a/libs/python/agent/benchmarks/README.md b/libs/python/agent/benchmarks/README.md
index 3df840c154..03d1a78973 100644
--- a/libs/python/agent/benchmarks/README.md
+++ b/libs/python/agent/benchmarks/README.md
@@ -25,8 +25,8 @@ The benchmark system evaluates models on GUI grounding tasks, specifically click
### 3. Interactive Testing (`interactive.py`)
- **Real-time testing**: Take screenshots and visualize model predictions
- **Commands**:
- - Type instruction → screenshot + test all models
- - `screenshot` → take screenshot without prediction
+ - Type instruction → test all models on last screenshot
+ - `screenshot` → take screenshot
- `models` → list available models
- `quit`/`exit` → exit tool
- **Output**: Visual predictions with crosshairs for each model
@@ -63,21 +63,6 @@ Model Results:
- **Visualizations**: `output/` directory with prediction visualizations
- **Interactive Output**: `interactive_output/` for interactive session results
-## Metrics Tracked
-
-- **Accuracy**: Percentage of clicks within bounding boxes
-- **Timing**: Average, min, max prediction times
-- **VRAM Usage**: Maximum and average GPU memory usage
-- **Per-sample Results**: Detailed breakdown for debugging
-
-## Architecture
-
-The benchmark system is designed for:
-- **Modularity**: Easy to add new models and benchmarks
-- **Flexibility**: Works with any iterator of dicts with `image`, `bbox`, `instruction` keys
-- **Performance**: VRAM tracking and timing analysis
-- **Visualization**: Automatic generation of prediction visualizations
-
## Contributing
To add a new reference model, follow the instructions in [contrib.md](contrib.md).
\ No newline at end of file