Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.
Rust agentai-agentapplecomputer-usecomputer-use-agentcontainerizationcuadesktop-automationhacktoberfestlumemacosmanusoperatorswiftvirtualizationvirtualization-frameworkwindowswindows-sandbox
Merge branch 'main' into docs/mcp-server-locally
Adam043e3f231883
diff --cc docs/content/docs/libraries/mcp-server/client-integrations.mdxindex 6a79f5b372,4ad0c6a6cc..a95df6a9a1--- a/docs/content/docs/libraries/mcp-server/client-integrations.mdx+++ b/docs/content/docs/libraries/mcp-server/client-integrations.mdx@@@ -96,23 -18,3 +96,23 @@@ Example configuration for CursorAfter configuration, you can simply tell Cursor's Agent to perform computer tasks by explicitly mentioning the CUA agent, such as "Use the computer control tools to open Safari."For more information on MCP with Cursor, see the [official Cursor MCP documentation](https://docs.cursor.com/context/model-context-protocol).++## Other MCP Clients++The MCP server is compatible with any MCP-compliant client. The server exposes the following tools:++- `run_cua_task` - Execute single computer tasks+- `run_multi_cua_tasks` - Execute multiple tasks (sequential or concurrent)+- `screenshot_cua` - Capture screenshots+- `get_session_stats` - Monitor session statistics+- `cleanup_session` - Manage session lifecycle++### Configuration Options++All MCP clients can configure the server using environment variables:++- `CUA_MODEL_NAME` - Model to use for task execution+- `CUA_MAX_IMAGES` - Maximum images to keep in context+- `CUA_USE_HOST_COMPUTER_SERVER` - Use host system instead of VM+- See the [Configuration](/docs/libraries/mcp-server/configuration) page for detailed configuration options.++See the [Configuration](/docs/libraries/mcp-server/configuration) page for detailed configuration options.diff --cc docs/content/docs/libraries/mcp-server/index.mdxindex a20b5d0980,e79d6b1ea9..7f2c7684c2--- a/docs/content/docs/libraries/mcp-server/index.mdx+++ b/docs/content/docs/libraries/mcp-server/index.mdx@@@ -7,21 -7,3 +7,21 @@@ github---**cua-mcp-server** is a MCP server for the Computer-Use Agent (CUA), allowing you to run CUA through Claude Desktop or other MCP clients.++## Features++- **Multi-Client Support**: Concurrent sessions with automatic resource management+- **Progress Reporting**: Real-time progress updates during task execution+- **Error Handling**: Robust error recovery with screenshot capture+- **Concurrent Execution**: Run multiple tasks in parallel for improved performance+- **Session Management**: Automatic cleanup and resource pooling+- **LiteLLM Integration**: Support for multiple model providers+- **VM Safety**: Default VM execution with optional host system control++## Quick Start++1. **Install**: `pip install cua-mcp-server`+2. **Configure**: Add to your MCP client configuration+3. **Use**: Ask Claude to perform computer tasks+- See the [Installation](/docs/libraries/mcp-server/installation) guide for detailed setup instructions.++See the [Installation](/docs/libraries/mcp-server/installation) guide for detailed setup instructions.diff --cc docs/content/docs/libraries/mcp-server/installation.mdxindex ce4f87a6fe,9c0d281fc0..e3e11a6bb3--- a/docs/content/docs/libraries/mcp-server/installation.mdx+++ b/docs/content/docs/libraries/mcp-server/installation.mdx@@@ -44,90 -45,12 +46,90 @@@ You can then use the script in your MC}```-### Troubleshooting+**Important**: You must include your Anthropic API key for the MCP server to work properly.-If you get a `/bin/bash: ~/cua/libs/python/mcp-server/scripts/start_mcp_server.sh: No such file or directory` error, try changing the path to the script to be absolute instead of relative.+## Development Setup-To see the logs:+If you're working with the CUA source code directly (like in the CUA repository), you can use the development script instead:+```json+{+ "mcpServers": {+ "cua-agent": {+ "command": "/usr/bin/env",+ "args": [+ "bash", "-lc",+ "export CUA_MODEL_NAME='anthropic/claude-sonnet-4-20250514'; export ANTHROPIC_API_KEY='your-anthropic-api-key-here'; /path/to/cua/libs/python/mcp-server/scripts/start_mcp_server.sh"+ ]+ }+ }+}```++**For host computer control** (development setup):++1. **Install Computer Server Dependencies**:+ ```bash+ python3 -m pip install uvicorn fastapi+ python3 -m pip install -e libs/python/computer-server --break-system-packages+ ```++2. **Start the Computer Server**:+ ```bash+ cd /path/to/cua+ python -m computer_server --log-level debug+ ```+ This will start the computer server on `http://localhost:8000` that controls your actual desktop.++3. **Configure Claude Desktop**:+ ```json+ {+ "mcpServers": {+ "cua-agent": {+ "command": "/usr/bin/env",+ "args": [+ "bash", "-lc",+ "export CUA_MODEL_NAME='anthropic/claude-sonnet-4-20250514'; export ANTHROPIC_API_KEY='your-anthropic-api-key-here'; export CUA_USE_HOST_COMPUTER_SERVER='true'; export CUA_MAX_IMAGES='1'; /path/to/cua/libs/python/mcp-server/scripts/start_mcp_server.sh"+ ]+ }+ }+ }+ ```++**Note**: Replace `/path/to/cua` with the absolute path to your CUA repository directory.++**⚠️ Important**: When using host computer control (`CUA_USE_HOST_COMPUTER_SERVER='true'`), the AI will have direct access to your desktop and can perform actions like opening applications, clicking, typing, and taking screenshots. Make sure you're comfortable with this level of access.++### Troubleshooting++**Common Issues:**++1. **"Claude's response was interrupted"** - This usually means:+ - Missing API key: Add `ANTHROPIC_API_KEY` to your environment variables+ - Invalid model name: Use a valid model like `anthropic/claude-sonnet-4-20250514`+ - Check logs for specific error messages++2. **"Missing Anthropic API Key"** - Add your API key to the configuration:+ ```json+ "env": {+ "ANTHROPIC_API_KEY": "your-api-key-here"+ }+ ```++3. **"model not found"** - Use a valid model name:+ - ✅ `anthropic/claude-sonnet-4-20250514`+ - ✅ `anthropic/claude-3-5-sonnet-20240620`+ - ❌ `anthropic/claude-3-5-sonnet-20241022` (doesn't exist)++4. **Script not found** - If you get a `/bin/bash: ~/cua/libs/python/mcp-server/scripts/start_mcp_server.sh: No such file or directory` error, try changing the path to the script to be absolute instead of relative.++5. **Host Computer Control Issues** - If using `CUA_USE_HOST_COMPUTER_SERVER='true'`:+ - **Computer Server not running**: Make sure you've started the computer server with `python -m computer_server --log-level debug`+ - **Port 8000 in use**: Check if another process is using port 8000 with `lsof -i :8000`+ - **Missing dependencies**: Install `uvicorn` and `fastapi` with `python3 -m pip install uvicorn fastapi`+ - **Image size errors**: Use `CUA_MAX_IMAGES='1'` to reduce image context size++**Viewing Logs:**+```bashtail -n 20 -f ~/Library/Logs/Claude/mcp*.log- ```+ ```diff --cc docs/content/docs/libraries/mcp-server/tools.mdxindex 20e91311f1,fd09b366e6..14901057a6--- a/docs/content/docs/libraries/mcp-server/tools.mdx+++ b/docs/content/docs/libraries/mcp-server/tools.mdx@@@ -6,58 -6,5 +6,58 @@@ title: ToolThe MCP server exposes the following tools to Claude:-1. `run_cua_task` - Run a single Computer-Use Agent task with the given instruction-2. `run_multi_cua_tasks` - Run multiple tasks in sequence+### Core Task Execution Tools++1. **`run_cua_task`** - Run a single Computer-Use Agent task with the given instruction+ - `task` (string): The task description for the agent to execute+ - `session_id` (string, optional): Session ID for multi-client support. If not provided, a new session will be created+ - Returns: Tuple of (combined text output, final screenshot)++2. **`run_multi_cua_tasks`** - Run multiple tasks in sequence or concurrently+ - `tasks` (list of strings): List of task descriptions to execute+ - `session_id` (string, optional): Session ID for multi-client support. If not provided, a new session will be created+ - `concurrent` (boolean, optional): If true, run tasks concurrently. If false, run sequentially (default)+ - Returns: List of tuples (combined text output, screenshot) for each task++### Utility Tools++3. **`screenshot_cua`** - Take a screenshot of the current screen+ - `session_id` (string, optional): Session ID for multi-client support. If not provided, a new session will be created+ - Returns: Screenshot image++4. **`get_session_stats`** - Get statistics about active sessions and resource usage+ - Returns: Dictionary with session statistics including total sessions, active tasks, and session details++5. **`cleanup_session`** - Cleanup a specific session and release its resources+ - `session_id` (string): The session ID to cleanup+ - Returns: Confirmation message++## Session Management++The MCP server supports multi-client sessions with automatic resource management:++- **Session Isolation**: Each client can have its own session with isolated computer instances+- **Resource Pooling**: Computer instances are pooled for efficient resource usage+- **Automatic Cleanup**: Idle sessions are automatically cleaned up after 10 minutes+- **Concurrent Tasks**: Multiple tasks can run concurrently within the same session+- **Progress Reporting**: Real-time progress updates during task execution++## Usage Examples++### Basic Task Execution+```+"Open Chrome and navigate to github.com"+"Create a folder called 'Projects' on my desktop"+```++### Multi-Task Execution+```+"Run these tasks: 1) Open Finder, 2) Navigate to Documents, 3) Create a new folder called 'Work'"+```++### Session Management+```+"Take a screenshot of the current screen"+"Show me the session statistics"+"Cleanup session abc123"- ```++```