Skip to content

armgit / cua

Public Imported from GitHub
0

Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.

added gpt-5 + gpta1 examples

Dillon DuPontf45f6b84e94f
diff --git a/docs/content/docs/agent-sdk/integrations/hud.mdx b/docs/content/docs/agent-sdk/integrations/hud.mdx
index cebd36be0a..b517121e0f 100644
--- a/docs/content/docs/agent-sdk/integrations/hud.mdx
+++ b/docs/content/docs/agent-sdk/integrations/hud.mdx
@@ -28,6 +28,8 @@ taskset = TaskSet(tasks=taskset[:10]) # limit to 10 tasks instead of all 370
# Run benchmark job
job = await run_job(
model="openai/computer-use-preview",
+ # model="anthropic/claude-3-5-sonnet-20241022",
+ # model="huggingface-local/HelloKKMe/GTA1-7B+openai/gpt-5",
task_or_taskset=taskset,
job_name="test-computeragent-job",
max_concurrent_tasks=5,
diff --git a/docs/content/docs/agent-sdk/supported-agents/composed-agents.mdx b/docs/content/docs/agent-sdk/supported-agents/composed-agents.mdx
index 50160fd8cf..8040d2e541 100644
--- a/docs/content/docs/agent-sdk/supported-agents/composed-agents.mdx
+++ b/docs/content/docs/agent-sdk/supported-agents/composed-agents.mdx
@@ -28,41 +28,41 @@ Any model that supports `predict_click()` can be used as the grounding component
Any vision-enabled LiteLLM-compatible model can be used as the thinking component:
- **Anthropic**: `anthropic/claude-3-5-sonnet-20241022`, `anthropic/claude-3-opus-20240229`
-- **OpenAI**: `openai/gpt-4o`, `openai/gpt-4-vision-preview`
+- **OpenAI**: `openai/gpt-5`, `openai/gpt-o3`, `openai/gpt-4o`
- **Google**: `gemini/gemini-1.5-pro`, `vertex_ai/gemini-pro-vision`
- **Local models**: Any Hugging Face vision-language model
## Usage Examples
-### GTA1 + Claude 3.5 Sonnet
+### GTA1 + GPT-5
-Combine state-of-the-art grounding with powerful reasoning:
+Use Google's Gemini for planning with specialized grounding:
```python
agent = ComputerAgent(
- "huggingface-local/HelloKKMe/GTA1-7B+anthropic/claude-3-5-sonnet-20241022",
+ "huggingface-local/HelloKKMe/GTA1-7B+openai/gpt-5",
tools=[computer]
)
-async for _ in agent.run("Open Firefox, navigate to github.com, and search for 'computer-use'"):
+async for _ in agent.run("Take a screenshot, analyze the UI, and click on the most prominent button"):
pass
-# Success! πŸŽ‰
-# - Claude 3.5 Sonnet plans the sequence of actions
-# - GTA1-7B provides precise click coordinates for each UI element
```
-### GTA1 + Gemini Pro
+### GTA1 + Claude 3.5 Sonnet
-Use Google's Gemini for planning with specialized grounding:
+Combine state-of-the-art grounding with powerful reasoning:
```python
agent = ComputerAgent(
- "huggingface-local/HelloKKMe/GTA1-7B+gemini/gemini-1.5-pro",
+ "huggingface-local/HelloKKMe/GTA1-7B+anthropic/claude-3-5-sonnet-20241022",
tools=[computer]
)
-async for _ in agent.run("Take a screenshot, analyze the UI, and click on the most prominent button"):
+async for _ in agent.run("Open Firefox, navigate to github.com, and search for 'computer-use'"):
pass
+# Success! πŸŽ‰
+# - Claude 3.5 Sonnet plans the sequence of actions
+# - GTA1-7B provides precise click coordinates for each UI element
```
### UI-TARS + GPT-4o