Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.
Rust agentai-agentapplecomputer-usecomputer-use-agentcontainerizationcuadesktop-automationhacktoberfestlumemacosmanusoperatorswiftvirtualizationvirtualization-frameworkwindowswindows-sandbox
Refactor Developer Guide
f-trycuaabfa795af520
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.mdindex 5a479add6f..0f34d9d874 100644--- a/CONTRIBUTING.md+++ b/CONTRIBUTING.md@@ -26,6 +26,23 @@ We're always looking for suggestions to make lume better. If you have an idea:- Any potential implementation details- Why this enhancement would benefit lume users+## Code Formatting++We follow strict code formatting guidelines to ensure consistency across the codebase. Before submitting any code:++1. **Review Our Format Guide**: Please review our [Code Formatting Standards](docs/Getting-Started.md#code-formatting-standards) section in the Getting Started guide.+2. **Configure Your IDE**: We recommend using the workspace settings provided in `.vscode/` for automatic formatting.+3. **Run Formatting Tools**: Always run the formatting tools before submitting a PR:+ ```bash+ # For Python code+ pdm run black .+ pdm run ruff check --fix .+ ```+4. **Validate Your Code**: Ensure your code passes all checks:+ ```bash+ pdm run mypy .+ ```+## DocumentationDocumentation improvements are always welcome. You can:@@ -34,6 +51,6 @@ Documentation improvements are always welcome. You can:- Improve API documentation- Add tutorials or guides-For detailed instructions on setting up your development environment and submitting code contributions, please see our [Developer-Guide.md](docs/Developer-Guide.md) guide.+For detailed instructions on setting up your development environment and submitting code contributions, please see our [Developer-Guide](./docs/Developer-Guide.md) guide.Feel free to join our [Discord community](https://discord.com/invite/mVnXXpdE85) to discuss ideas or get help with your contributions.\ No newline at end of filediff --git a/README.md b/README.mdindex 100c65418e..c4cb94f175 100644--- a/README.md+++ b/README.md@@ -59,10 +59,46 @@ If you want to use AI agents with virtualized environments:2. Install the Python libraries:```bash- pip install cua-computer cua-agent+ pip install cua-computer cua-agent[all]```-Check out our [Getting Started Guide](./docs/Getting-Started.md) or explore the [Notebooks](./notebooks/) for interactive examples.+3. Use the libraries in your Python code:+ ```python+ from cua.computer import Computer+ from cua.agent import ComputerAgent, LLM, AgentLoop, LLMProvider++ async with Computer(verbosity=logging.DEBUG) as macos_computer:+ agent = ComputerAgent(+ computer=macos_computer,+ loop=AgentLoop.OPENAI, # or AgentLoop.ANTHROPIC, or AgentLoop.OMNI+ model=LLM(provider=LLMProvider.OPENAI) # or LLM(provider=LLMProvider.ANTHROPIC)+ )++ tasks = [+ "Look for a repository named trycua/cua on GitHub.",+ ]++ for task in tasks:+ async for result in agent.run(task):+ print(result)+ ```++ Explore the [Notebooks](./notebooks/) for ready-to-run examples.++4. For Developers (contribute and use latest features):+ ```bash+ # Clone the repository+ git clone https://github.com/trycua/cua.git+ cd cua++ # Open the project in VSCode+ code ./vscode/py.code-workspace++ # Build the project+ ./scripts/build.sh+ ```++ See our [Developer-Guide](./docs/Developer-Guide.md) for more information.## Monorepo Librariesdiff --git a/docs/Getting-Started.md b/docs/Developer-Guide.mdsimilarity index 53%rename from docs/Getting-Started.mdrename to docs/Developer-Guide.mdindex cd93ac3c3d..bfcb3a60a6 100644--- a/docs/Getting-Started.md+++ b/docs/Developer-Guide.md@@ -35,18 +35,7 @@ ANTHROPIC_API_KEY=your_anthropic_key_hereOPENAI_API_KEY=your_openai_key_here```-4. Run the build script to set up all packages:-```bash-./scripts/build.sh-```--This will:-- Create a virtual environment for the project-- Install all packages in development mode-- Set up the correct Python path-- Install development tools--5. Open the workspace in VSCode or Cursor:+4. Open the workspace in VSCode or Cursor:```bash# For Cua Python developmentcode .vscode/py.code-workspace@@ -61,14 +50,44 @@ Using the workspace file is strongly recommended as it:- Enables debugging configurations- Maintains consistent settings across packages-### Docker Development Environment+### Lume Development++Refer to the [Lume README](../libs/lume/docs/Development.md) for instructions on how to develop the Lume CLI.-As an alternative to running directly on your host machine, you can use Docker for development. This approach has several advantages:+### Python Development++Run the build script to set up all packages:+```bash+./scripts/build.sh+```-- Ensures consistent development environment across different machines-- Isolates dependencies from your host system-- Works well for cross-platform development-- Avoids conflicts with existing Python installations+The build script creates a shared virtual environment for all packages. The workspace configuration automatically handles import paths with the correct Python path settings.++This will:+- Create a virtual environment for the project+- Install all packages in development mode+- Set up the correct Python path+- Install development tools++### Running Examples++The Python workspace includes launch configurations for all packages:++- "Run Computer Examples" - Runs computer examples+- "Run Computer API Server" - Runs the computer-server+- "Run Agent Examples" - Runs agent examples+- "SOM" configurations - Various settings for running SOM++To run examples from VSCode / Cursor:+1. Press F5 or use the Run/Debug view+2. Select the desired configuration++The workspace also includes compound launch configurations:+- "Run Computer Examples + Server" - Runs both the Computer Examples and Server simultaneously++### Docker Development Environment++As an alternative to installing directly on your host machine, you can use Docker for development. This approach has several advantages:#### Prerequisites@@ -122,77 +141,110 @@ This will:- Clean PDM-related files- Reset environment configurations-### Package Virtual Environments+## Code Formatting Standards-The build script creates a shared virtual environment for all packages. The workspace configuration automatically handles import paths with the correct Python path settings.+The cua project follows strict code formatting standards to ensure consistency across all packages.-### Running Examples+### Python Code Formatting-The Python workspace includes launch configurations for all packages:+#### Tools-- "Run Computer Examples" - Runs computer examples-- "Run Computer API Server" - Runs the computer-server-- "Run Omni Agent Examples" - Runs agent examples-- "SOM" configurations - Various settings for running SOM+The project uses the following tools for code formatting and linting:-To run examples:-1. Open the workspace file (`.vscode/py.code-workspace`)-2. Press F5 or use the Run/Debug view-3. Select the desired configuration+- **[Black](https://black.readthedocs.io/)**: Code formatter+- **[Ruff](https://beta.ruff.rs/docs/)**: Fast linter and formatter+- **[MyPy](https://mypy.readthedocs.io/)**: Static type checker-The workspace also includes compound launch configurations:-- "Run Computer Examples + Server" - Runs both the Computer Examples and Server simultaneously+These tools are automatically installed when you set up the development environment using the `./scripts/build.sh` script.-## Release and Publishing Process+#### Configuration-This monorepo contains multiple Python packages that can be published to PyPI. The packages-have dependencies on each other in the following order:+The formatting configuration is defined in the root `pyproject.toml` file:-1. `pylume` - Base package for VM management-2. `cua-computer` - Computer control interface (depends on pylume)-3. `cua-som` - Parser for UI elements (independent, formerly omniparser)-4. `cua-agent` - AI agent (depends on cua-computer and optionally cua-som)-5. `computer-server` - Server component installed on the sandbox+```toml+[tool.black]+line-length = 100+target-version = ["py310"]-#### Workflow Structure+[tool.ruff]+line-length = 100+target-version = "py310"+select = ["E", "F", "B", "I"]+fix = true-The publishing process is managed by these GitHub workflow files:+[tool.ruff.format]+docstring-code-format = true-- **Package-specific workflows**:- - `.github/workflows/publish-pylume.yml`- - `.github/workflows/publish-computer.yml`- - `.github/workflows/publish-som.yml`- - `.github/workflows/publish-agent.yml`- - `.github/workflows/publish-computer-server.yml`+[tool.mypy]+strict = true+python_version = "3.10"+ignore_missing_imports = true+disallow_untyped_defs = true+check_untyped_defs = true+warn_return_any = true+show_error_codes = true+warn_unused_ignores = false+```++#### Key Formatting Rules++- **Line Length**: Maximum of 100 characters+- **Python Version**: Code should be compatible with Python 3.10++- **Imports**: Automatically sorted (using Ruff's "I" rule)+- **Type Hints**: Required for all function definitions (strict mypy mode)++#### IDE Integration-- **Coordinator workflow**:- - `.github/workflows/publish-all.yml` - Manages global releases and manual selections+The repository includes VSCode workspace configurations that enable automatic formatting. When you open the workspace files (as recommended in the setup instructions), the correct formatting settings are automatically applied.-### Version Management+Python-specific settings in the workspace files:-#### Special Considerations for Pylume+```json+"[python]": {+ "editor.formatOnSave": true,+ "editor.defaultFormatter": "ms-python.black-formatter",+ "editor.codeActionsOnSave": {+ "source.organizeImports": "explicit"+ }+}+```++Recommended VS Code extensions:+- Black Formatter (ms-python.black-formatter)+- Ruff (charliermarsh.ruff)+- Pylance (ms-python.vscode-pylance)-The `pylume` package requires special handling as it incorporates the binary executable from the [lume repository](https://github.com/trycua/lume):+#### Manual Formatting-- When releasing `pylume`, ensure the version matches a corresponding release in the lume repository-- The workflow automatically downloads the matching lume binary and includes it in the pylume package-- If you need to release a new version of pylume, make sure to coordinate with a matching lume release+To manually format code:-## Development Workspaces+```bash+# Format all Python files using Black+pdm run black .-This monorepo includes multiple VS Code workspace configurations to optimize the development experience based on which components you're working with:+# Run Ruff linter with auto-fix+pdm run ruff check --fix .-### Available Workspace Files+# Run type checking with MyPy+pdm run mypy .+```-- **[py.code-workspace](.vscode/py.code-workspace)**: For Python package development (Computer, Agent, SOM, etc.)-- **[lume.code-workspace](.vscode/lume.code-workspace)**: For Swift-based Lume development+#### Pre-commit Validation-To open a specific workspace:+Before submitting a pull request, ensure your code passes all formatting checks:```bash-# For Python development-code .vscode/py.code-workspace+# Run all checks+pdm run black --check .+pdm run ruff check .+pdm run mypy .+```++### Swift Code (Lume)++For Swift code in the `libs/lume` directory:++- Follow the [Swift API Design Guidelines](https://www.swift.org/documentation/api-design-guidelines/)+- Use SwiftFormat for consistent formatting+- Code will be automatically formatted on save when using the lume workspace-# For Lume (Swift) development-code .vscode/lume.code-workspace-```\ No newline at end of file