Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.
Rust agentai-agentapplecomputer-usecomputer-use-agentcontainerizationcuadesktop-automationhacktoberfestlumemacosmanusoperatorswiftvirtualizationvirtualization-frameworkwindowswindows-sandbox
Improve notebook structure
James Murdza981a081672fd
diff --git a/notebooks/hud_hackathon.ipynb b/notebooks/hud_hackathon.ipynbindex 0f295b081e..fd6e59d4e9 100644--- a/notebooks/hud_hackathon.ipynb+++ b/notebooks/hud_hackathon.ipynb@@ -115,15 +115,7 @@"source": ["## 🤖 Create a computer use agent\n","\n",- "Create and run a computer use agent using the Cua SDK."- ]- },- {- "cell_type": "markdown",- "id": "54338496",- "metadata": {},- "source": [- "Make sure Docker is running and create a Docker-based agent."+ "Create and a computer use agent using the Cua SDK."]},{@@ -136,15 +128,6 @@"import logging\n","from pathlib import Path\n","from agent import ComputerAgent\n",- "from computer import Computer, VMProviderType\n",- "\n",- "# Connect to your existing cloud container\n",- "computer = Computer(\n",- " os_type=\"linux\",\n",- " provider_type=VMProviderType.DOCKER,\n",- " verbosity=logging.INFO\n",- ")\n",- "await computer.run()\n","\n","# Here you can set the model and tools for your agent.\n","# Computer use models: https://www.trycua.com/docs/agent-sdk/supported-agents/computer-use-agents\n",@@ -152,7 +135,6 @@"# Custom tools: https://www.trycua.com/docs/agent-sdk/custom-tools\n","agent_config = {\n"," \"model\": \"openai/computer-use-preview\",\n",- " \"tools\": [computer],\n"," \"trajectory_dir\": str(Path(\"trajectories\")),\n"," \"only_n_most_recent_images\": 3,\n"," \"verbosity\": logging.INFO\n",@@ -177,9 +159,40 @@"id": "12b9c22c","metadata": {},"source": [- "Try running the computer use agent on a simple task.\n",+ "Make sure Docker is running to launch the computer.\n",+ "\n",+ "You can view the live VNC stream from the Docker container at `http://localhost:8006/`"+ ]+ },+ {+ "cell_type": "code",+ "execution_count": null,+ "id": "a210e959",+ "metadata": {},+ "outputs": [],+ "source": [+ "from computer import Computer, VMProviderType\n",+ "import webbrowser \n","\n",- "You can view the VNC stream from the Docker container at `http://localhost:8006/`\n",+ "# Connect to your existing cloud container\n",+ "computer = Computer(\n",+ " os_type=\"linux\",\n",+ " provider_type=VMProviderType.DOCKER,\n",+ " verbosity=logging.INFO\n",+ ")\n",+ "await computer.run()\n",+ "\n",+ "agent_config[\"tools\"] = computer\n",+ "\n",+ "webbrowser.open(\"http://localhost:8006/\", new=0, autoraise=True)"+ ]+ },+ {+ "cell_type": "markdown",+ "id": "87a307e3",+ "metadata": {},+ "source": [+ "Try running the computer use agent on a simple task.\n","\n","Trajectories are saved in the format: `trajectories/YYYY-MM-DD_computer-use-pre_XXX`."]