Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.
Rust agentai-agentapplecomputer-usecomputer-use-agentcontainerizationcuadesktop-automationhacktoberfestlumemacosmanusoperatorswiftvirtualizationvirtualization-frameworkwindowswindows-sandbox
feat(cua-driver): add root Package.swift shim for SwiftPM consumption (#1390)
Francesco Bonacci6e04d161b034
diff --git a/.gitignore b/.gitignoreindex 6d16337472..5806dbb21c 100644--- a/.gitignore+++ b/.gitignore@@ -162,6 +162,7 @@ playground.xcworkspace# hence it is not needed unless you have added a package configuration file to your project.swiftpm/.build/+/Package.resolved# CocoaPods## We recommend against adding the Pods directory to your .gitignore. Howeverdiff --git a/Package.swift b/Package.swiftnew file mode 100644index 0000000000..c951688d4a--- /dev/null+++ b/Package.swift@@ -0,0 +1,60 @@+// swift-tools-version: 6.0+import PackageDescription++// Root shim: re-exports CuaDriverCore and CuaDriverServer so Swift packages+// can consume them directly from the trycua/cua monorepo without knowing the+// internal layout. Sources live in libs/cua-driver/Sources/; this file uses+// path: to forward there.+//+// IMPORTANT — SPM version resolution:+// SPM's `from:` / `upToNextMajor` only recognises semver tags ("0.1.0",+// "v0.1.0"). This repo uses "cua-driver-v*" tags for the CLI releases, which+// SPM cannot parse. Until plain semver tags are published, pin by revision:+//+// .package(url: "https://github.com/trycua/cua.git", .revision("cua-driver-v0.1.0"))+//+// When the repo starts publishing semver tags alongside the CLI tags, use:+//+// .package(url: "https://github.com/trycua/cua.git", from: "0.1.0")+//+// Then in your target's dependencies:+//+// .product(name: "CuaDriverCore", package: "cua") // AX, input, capture, recording+// .product(name: "CuaDriverServer", package: "cua") // MCP tool handlers + daemon layer++let package = Package(+ name: "cua",+ platforms: [.macOS(.v14)],+ products: [+ // Accessibility, input, capture, app-launch, recording primitives.+ // No external dependencies — system frameworks only.+ .library(name: "CuaDriverCore", targets: ["CuaDriverCore"]),++ // MCP tool handlers and daemon server built on top of CuaDriverCore.+ // Depends on modelcontextprotocol/swift-sdk for the MCP protocol types.+ .library(name: "CuaDriverServer", targets: ["CuaDriverServer"]),+ ],+ dependencies: [+ .package(+ url: "https://github.com/modelcontextprotocol/swift-sdk.git",+ from: "0.9.0"+ ),+ ],+ targets: [+ // NOTE: if libs/cua-driver/Package.swift ever gains resources:,+ // swiftSettings:, linkerSettings:, or exclude: on these targets,+ // mirror those changes here to avoid a silent build mismatch.+ .target(+ name: "CuaDriverCore",+ path: "libs/cua-driver/Sources/CuaDriverCore"+ ),+ .target(+ name: "CuaDriverServer",+ dependencies: [+ "CuaDriverCore",+ .product(name: "MCP", package: "swift-sdk"),+ ],+ path: "libs/cua-driver/Sources/CuaDriverServer"+ ),+ ]+)diff --git a/docs/content/docs/cua-driver/guide/getting-started/meta.json b/docs/content/docs/cua-driver/guide/getting-started/meta.jsonindex 3c9c8b84e0..1ca74ebad1 100644--- a/docs/content/docs/cua-driver/guide/getting-started/meta.json+++ b/docs/content/docs/cua-driver/guide/getting-started/meta.json@@ -3,5 +3,5 @@"description": "Get up and running with Cua Driver","icon": "Rocket","defaultOpen": true,- "pages": ["introduction", "installation", "quickstart", "comparison", "faq"]+ "pages": ["introduction", "installation", "quickstart", "swift-integration", "comparison", "faq"]}diff --git a/docs/content/docs/cua-driver/guide/getting-started/swift-integration.mdx b/docs/content/docs/cua-driver/guide/getting-started/swift-integration.mdxnew file mode 100644index 0000000000..36019a9a18--- /dev/null+++ b/docs/content/docs/cua-driver/guide/getting-started/swift-integration.mdx@@ -0,0 +1,98 @@+---+title: Swift Integration+description: Embed CuaDriverCore or CuaDriverServer in a Swift package+---++import { Callout } from 'fumadocs-ui/components/callout';++`CuaDriverCore` and `CuaDriverServer` are available as Swift library products directly from the `trycua/cua` repository. Use this when you want to embed accessibility automation, window capture, or MCP tool handling into your own macOS Swift app or package — without shelling out to the `cua-driver` CLI.++## Add the dependency++In your `Package.swift`:++```swift+dependencies: [+ .package(+ url: "https://github.com/trycua/cua.git",+ from: "cua-driver-v0.1.0"+ ),+],+```++Then add the products you need to your target:++```swift+targets: [+ .target(+ name: "MyApp",+ dependencies: [+ // Accessibility, input synthesis, window capture, recording.+ // No external dependencies — system frameworks only.+ .product(name: "CuaDriverCore", package: "cua"),++ // MCP tool handlers and daemon layer (adds swift-sdk dependency).+ // Use this if you want to expose cua-driver tools over MCP.+ .product(name: "CuaDriverServer", package: "cua"),+ ]+ ),+]+```++<Callout type="info">+ `CuaDriverCore` depends only on system frameworks (AppKit, CoreGraphics, ScreenCaptureKit, etc.). `CuaDriverServer` adds [modelcontextprotocol/swift-sdk](https://github.com/modelcontextprotocol/swift-sdk) for the MCP protocol types.+</Callout>++## What's in each product++### CuaDriverCore++Low-level macOS automation primitives:++| Module area | What it gives you |+|---|---|+| `AppStateEngine` | AX tree snapshots with element-index caching |+| `AppLauncher` | Launch apps without focus steal |+| `AXInput` | Perform AX actions, read attributes |+| `MouseInput` / `KeyboardInput` | Synthesize CGEvents |+| `WindowCapture` | Per-window screenshots via ScreenCaptureKit |+| `RecordingSession` | Trajectory recording and replay |+| `AgentCursor` | Animated overlay cursor |+| `BrowserJS` / `ElectronJS` | Execute JS in browser tabs via Apple Events or CDP |+| `AXPageReader` | Extract text / query DOM from WKWebView AX trees |++### CuaDriverServer++Ready-made MCP tool handlers built on `CuaDriverCore`:++```swift+import CuaDriverServer++let registry = ToolRegistry.default+// registry.handlers["click"], ["get_window_state"], ["page"], etc.+```++Embed the full tool set in your own MCP server:++```swift+import MCP+import CuaDriverServer++let server = MCPServer(tools: ToolRegistry.default.allTools) { name, args in+ try await ToolRegistry.default.call(name, arguments: args)+}+```++## Minimum requirements++- macOS 14 (Sonoma) or later+- Swift 6.0+++## TCC permissions++Calling AX or screen-capture APIs from an embedded library still requires the host app to hold the relevant TCC grants:++- **Accessibility** — for any `AXUIElement` work (snapshots, clicks, AX actions).+- **Screen Recording** — for `WindowCapture` screenshots.++Grant these under **System Settings → Privacy & Security** against your app's bundle identifier, the same as for the standalone `CuaDriver.app`.