Skip to content

armgit / cua

Public Imported from GitHub
0

Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.

Update README.md

ddupont0a007b8148e5
diff --git a/libs/computer/README.md b/libs/computer/README.md
index 1b86d12294..7690ff4a20 100644
--- a/libs/computer/README.md
+++ b/libs/computer/README.md
@@ -67,30 +67,78 @@ Refer to this notebook for a step-by-step guide on how to use the Computer-Use I
## Using the Gradio Computer UI
-The computer module includes a Gradio UI for creating and sharing demonstration data. The UI provides built-in integration with Hugging Face Datasets for sharing demonstrations and incorporating them into CUA ML pipelines.
+The computer module includes a Gradio UI for creating and sharing demonstration data. We make it easy for people to build community datasets for better computer use models with an upload to Huggingface feature.
```bash
# Install with UI support
pip install "cua-computer[ui]"
```
+> **Note:** For precise control of the computer, we recommend using VNC or Screen Sharing instead of the Computer Gradio UI.
-<details open>
-<summary>View demonstration video</summary>
-<video src="https://github.com/user-attachments/assets/7c683b58-f04d-4e8c-b63f-6ef36e9637d5" controls width="600"></video>
-</details>
+### Building and Sharing Demonstrations with Huggingface
-> **Note:** For precise control of the computer, we recommend using VNC or Screen Sharing instead of the Computer Gradio UI.
+Follow these steps to contribute your own demonstrations:
+
+#### 1. Set up Huggingface Access
+Set your HF_TOKEN in a .env file or in your environment variables:
+
+```bash
+# In .env file
+HF_TOKEN=your_huggingface_token
+```
-### Launch the UI
+#### 2. Launch the Computer UI
```python
# launch_ui.py
from computer.ui.gradio.app import create_gradio_ui
+from dotenv import load_dotenv
+load_dotenv('.env')
app = create_gradio_ui()
app.launch(share=False)
```
For examples, see [Computer UI Examples](../../examples/computer_ui_examples.py)
+
+#### 3. Record Your Tasks
+
+<details open>
+<summary>View demonstration video</summary>
+<video src="https://github.com/user-attachments/assets/de3c3477-62fe-413c-998d-4063e48de176" controls width="600"></video>
+</details>
+
+Record yourself performing various computer tasks using the UI.
+
+#### 4. Save Your Demonstrations
+
+<details open>
+<summary>View demonstration video</summary>
+<video src="https://github.com/user-attachments/assets/5ad1df37-026a-457f-8b49-922ae805faef" controls width="600"></video>
+</details>
+
+Save each task by picking a descriptive name and adding relevant tags (e.g., "office", "web-browsing", "coding").
+
+#### 5. Record Additional Demonstrations
+
+Repeat steps 3 and 4 until you have a good amount of demonstrations covering different tasks and scenarios.
+
+#### 6. Upload to Huggingface
+
+<details open>
+<summary>View demonstration video</summary>
+<video src="https://github.com/user-attachments/assets/c586d460-3877-4b5f-a736-3248886d2134" controls width="600"></video>
+</details>
+
+Upload your dataset to Huggingface by:
+- Naming it as `{your_username}/{dataset_name}`
+- Choosing public or private visibility
+- Optionally selecting specific tags to upload only tasks with certain tags
+
+#### Examples and Resources
+
+- Example Dataset: [ddupont/test-dataset](https://huggingface.co/datasets/ddupont/test-dataset)
+- Find Community Datasets: 🔍 [Browse CUA Datasets on Huggingface](https://huggingface.co/datasets?other=cua)
+