Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.
Rust agentai-agentapplecomputer-usecomputer-use-agentcontainerizationcuadesktop-automationhacktoberfestlumemacosmanusoperatorswiftvirtualizationvirtualization-frameworkwindowswindows-sandbox
Add references to standalone examples and notebooks
James Murdza80153541fb72
diff --git a/docs/content/docs/agent-sdk/agent-loops.mdx b/docs/content/docs/agent-sdk/agent-loops.mdxindex 6a18f064c8..33bf66e2d9 100644--- a/docs/content/docs/agent-sdk/agent-loops.mdx+++ b/docs/content/docs/agent-sdk/agent-loops.mdx@@ -3,6 +3,8 @@ title: Agent Loopsdescription: Supported computer-using agent loops and models---+<Callout>A corresponding <a href="https://github.com/trycua/cua/blob/main/notebooks/agent_nb.ipynb" target="_blank">Jupyter Notebook</a> is available for this documentation.</Callout>+An agent can be thought of as a loop - it generates actions, executes them, and repeats until done:1. **Generate**: Your `model` generates `output_text`, `computer_call`, `function_call`diff --git a/docs/content/docs/agent-sdk/integrations/hud.mdx b/docs/content/docs/agent-sdk/integrations/hud.mdxindex 35236746b8..23ce86a6b2 100644--- a/docs/content/docs/agent-sdk/integrations/hud.mdx+++ b/docs/content/docs/agent-sdk/integrations/hud.mdx@@ -3,6 +3,8 @@ title: HUD Evalsdescription: Use ComputerAgent with HUD for benchmarking and evaluation---+<Callout>A corresponding <a href="https://github.com/trycua/cua/blob/main/notebooks/eval_osworld.ipynb" target="_blank">Jupyter Notebook</a> is available for this documentation.</Callout>+The HUD integration allows an agent to be benchmarked using the [HUD framework](https://www.hud.so/). Through the HUD integration, the agent controls a computer inside HUD, where tests are run to evaluate the success of each task.## Installationdiff --git a/docs/content/docs/computer-sdk/computers.mdx b/docs/content/docs/computer-sdk/computers.mdxindex 9b920aee84..0b11d20d75 100644--- a/docs/content/docs/computer-sdk/computers.mdx+++ b/docs/content/docs/computer-sdk/computers.mdx@@ -3,6 +3,8 @@ title: Cua Computersdescription: Understanding cua computer types and connection methods---+<Callout>A corresponding <a href="https://github.com/trycua/cua/blob/main/notebooks/computer_nb.ipynb" target="_blank">Jupyter Notebook</a> and <a href="https://github.com/trycua/cua/tree/main/examples/computer-example-ts" target="_blank">NodeJS project</a> are available for this documentation.</Callout>+Before we can automate apps using AI, we need to first connect to a Computer Server to give the AI a safe environment to execute workflows in.Cua Computers are preconfigured virtual machines running the Computer Server. They can be either macOS, Linux, or Windows. They're found in either a cloud-native container, or on your host desktop.diff --git a/docs/content/docs/computer-sdk/sandboxed-python.mdx b/docs/content/docs/computer-sdk/sandboxed-python.mdxindex 5f1687bf83..6d70e9a689 100644--- a/docs/content/docs/computer-sdk/sandboxed-python.mdx+++ b/docs/content/docs/computer-sdk/sandboxed-python.mdx@@ -3,6 +3,8 @@ title: Sandboxed Pythonslug: sandboxed-python---+<Callout>A corresponding <a href="https://github.com/trycua/cua/blob/main/examples/sandboxed_functions_examples.py" target="_blank">Python example</a> is available for this documentation.</Callout>+You can run Python functions securely inside a sandboxed virtual environment on a remote Cua Computer. This is useful for executing untrusted user code, isolating dependencies, or providing a safe environment for automation tasks.## How It Worksdiff --git a/docs/content/docs/libraries/computer-server/index.mdx b/docs/content/docs/libraries/computer-server/index.mdxindex c2a9c43cbb..fcf265daaf 100644--- a/docs/content/docs/libraries/computer-server/index.mdx+++ b/docs/content/docs/libraries/computer-server/index.mdx@@ -6,6 +6,8 @@ github:- https://github.com/trycua/cua/tree/main/libs/python/computer-server---+<Callout>A corresponding <a href="https://github.com/trycua/cua/blob/main/notebooks/computer_server_nb.ipynb" target="_blank">Jupyter Notebook</a> is available for this documentation.</Callout>+The Computer Server API reference documentation is currently under development.## Overviewdiff --git a/docs/content/docs/libraries/som/index.mdx b/docs/content/docs/libraries/som/index.mdxindex e61f7a057d..ceba6e62a5 100644--- a/docs/content/docs/libraries/som/index.mdx+++ b/docs/content/docs/libraries/som/index.mdx@@ -6,6 +6,8 @@ github:- https://github.com/trycua/cua/tree/main/libs/python/som---+<Callout>A corresponding <a href="https://github.com/trycua/cua/blob/main/examples/som_examples.py" target="_blank">Python example</a> is available for this documentation.</Callout>+## OverviewThe SOM library provides visual element detection and interaction capabilities. It is based on the [Set-of-Mark](https://arxiv.org/abs/2310.11441) research paper and the [OmniParser](https://github.com/microsoft/OmniParser) model.