Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.
Rust agentai-agentapplecomputer-usecomputer-use-agentcontainerizationcuadesktop-automationhacktoberfestlumemacosmanusoperatorswiftvirtualizationvirtualization-frameworkwindowswindows-sandbox
Merge branch 'main' into feat/docs/init
Morgan Dean6ccf507c4125
diff --cc docs/content/docs/api/computer/index.mdxindex 0d57d60da6,0000000000..ba1873ff39mode 100644,000000..100644--- a/docs/content/docs/api/computer/index.mdx+++ b/docs/content/docs/api/computer/index.mdx@@@ -1,85 -1,0 +1,94 @@@+---+title: Computer+---++The Computer API reference documentation is currently under development.++## Overview++The Computer library provides programmatic interfaces for computer automation and control.++## Reference++```python+# Shell Actions+result = await computer.interface.run_command(cmd) # Run shell command+# result.stdout, result.stderr, result.returncode++# Mouse Actions+await computer.interface.left_click(x, y) # Left click at coordinates+await computer.interface.right_click(x, y) # Right click at coordinates+await computer.interface.double_click(x, y) # Double click at coordinates+await computer.interface.move_cursor(x, y) # Move cursor to coordinates+await computer.interface.drag_to(x, y, duration) # Drag to coordinates+await computer.interface.get_cursor_position() # Get current cursor position+await computer.interface.mouse_down(x, y, button="left") # Press and hold a mouse button+await computer.interface.mouse_up(x, y, button="left") # Release a mouse button++# Keyboard Actions+await computer.interface.type_text("Hello") # Type text+await computer.interface.press_key("enter") # Press a single key+await computer.interface.hotkey("command", "c") # Press key combination+await computer.interface.key_down("command") # Press and hold a key+await computer.interface.key_up("command") # Release a key++# Scrolling Actions+await computer.interface.scroll(x, y) # Scroll the mouse wheel+await computer.interface.scroll_down(clicks) # Scroll down+await computer.interface.scroll_up(clicks) # Scroll up++# Screen Actions+await computer.interface.screenshot() # Take a screenshot+await computer.interface.get_screen_size() # Get screen dimensions++# Clipboard Actions+await computer.interface.set_clipboard(text) # Set clipboard content+await computer.interface.copy_to_clipboard() # Get clipboard content++# File System Operations+await computer.interface.file_exists(path) # Check if file exists+await computer.interface.directory_exists(path) # Check if directory exists+await computer.interface.read_text(path, encoding="utf-8") # Read file content+await computer.interface.write_text(path, content, encoding="utf-8") # Write file content+await computer.interface.read_bytes(path) # Read file content as bytes+await computer.interface.write_bytes(path, content) # Write file content as bytes+await computer.interface.delete_file(path) # Delete file+await computer.interface.create_dir(path) # Create directory+await computer.interface.delete_dir(path) # Delete directory+await computer.interface.list_dir(path) # List directory contents++# Accessibility+await computer.interface.get_accessibility_tree() # Get accessibility tree+++# Delay Configuration++# Set default delay between all actions (in seconds)++computer.interface.delay = 0.5 # 500ms delay between actions++++# Or specify delay for individual actions++await computer.interface.left_click(x, y, delay=1.0) # 1 second delay after click++await computer.interface.type_text("Hello", delay=0.2) # 200ms delay after typing++await computer.interface.press_key("enter", delay=0.5) # 500ms delay after key press+++# Python Virtual Environment Operations+await computer.venv_install("demo_venv", ["requests", "macos-pyxa"]) # Install packages in a virtual environment+await computer.venv_cmd("demo_venv", "python -c 'import requests; print(requests.get(`https://httpbin.org/ip`).json())'") # Run a shell command in a virtual environment+await computer.venv_exec("demo_venv", python_function_or_code, *args, **kwargs) # Run a Python function in a virtual environment and return the result / raise an exception++# Example: Use sandboxed functions to execute code in a C/ua Container+from computer.helpers import sandboxed++@sandboxed("demo_venv")+def greet_and_print(name):+ """Get the HTML of the current Safari tab"""+ import PyXA+ safari = PyXA.Application("Safari")+ html = safari.current_document.source()+ print(f"Hello from inside the container, {name}!")+ return {"greeted": name, "safari_html": html}++# When a @sandboxed function is called, it will execute in the container+result = await greet_and_print("C/ua")+# Result: {"greeted": "C/ua", "safari_html": "<html>...</html>"}+# stdout and stderr are also captured and printed / raised+print("Result from sandboxed function:", result)+```