👾 Open Computer Use – Open-Source Alternative to Codex Computer Use
Swift accessibilityai-agentai-agentsclaude-codecodexcodex-computer-usecomputer-usecomputer-use-agentcross-platformcuadesktop-automationgemini-cligui-automationmacosmcpmodel-context-protocolopen-computer-useopen-computer-use-mcpopenai-codexopencode
Release 0.1.11
ifuryst870554bcecc8
diff --git a/README.md b/README.mdindex 6da6fe1..6ee5810 100644--- a/README.md+++ b/README.md@@ -1,5 +1,7 @@# open-computer-use+中文说明见 [README.zh-CN.md](./README.zh-CN.md)。+`open-computer-use` is an open-source `Computer Use` service exposed over `MCP`, so any AI agent or MCP client can call it directly and use computer interaction capabilities on macOS.This project was inspired by OpenAI's recently released [Codex Computer Use](https://openai.com/index/codex-for-almost-everything/). It showed that non-intrusive CUA can be built on top of macOS Accessibility, which is why I decided to build an open-source version.@@ -17,7 +19,7 @@ npm i -g open-computer-useBefore first use, grant macOS `Accessibility` and `Screen Recording` permission to the `Open Computer Use.app` installed by `npm install -g open-computer-use`. That global npm install location should be treated as the long-term stable permission target. The development copy at `dist/Open Computer Use.app` should only be used as a local debugging fallback, not as the long-term app identity. If you are not sure about the current state, run:```bash-open-computer-use doctor+open-computer-use```Then add it to your MCP client:diff --git a/README.zh-CN.md b/README.zh-CN.mdnew file mode 100644index 0000000..a502c31--- /dev/null+++ b/README.zh-CN.md@@ -0,0 +1,64 @@+# open-computer-use++English version: [README.md](./README.md)++`open-computer-use` 是一个开源的 `Computer Use` 服务,已经包装成 `MCP` 协议,支持所有的 AI Agent 或 MCP Client 快速调用,实现 macOS 上的 `Computer Use` 能力。++项目的背后是 OpenAI 刚发布的 [Codex Computer Use](https://openai.com/index/codex-for-almost-everything/),让我看到了基于 Accessibility 可以实现非抢占式 CUA 能力,因此决定复刻一个开源版本。++在这期间我利用了之前写的 [harness 模版](https://github.com/iFurySt/harness-template) 开启了这个新项目。这是一个可以快速拉起面向 AI 仓库的 template,非常适合 100% AI-Generated 的项目,也是这一个月来我们最大的实践和收获。现在我们可以基于这套方法论快速实现很多东西;如果你有兴趣,我也写了一篇[文章](https://www.ifuryst.com/blog/2026/speedrunning-the-ai-era/)专门介绍这套方法论。++## Quick Start++先全局安装:++```bash+npm i -g open-computer-use+```++第一次使用前,给 `Open Computer Use.app` 授予 macOS 的 `Accessibility` 和 `Screen Recording` 权限。++```bash+open-computer-use+```++接着把它配到你的 MCP client 里:++```json+{+ "mcpServers": {+ "open-computer-use": {+ "command": "open-computer-use",+ "args": ["mcp"]+ }+ }+}+```++## 更多++除了直接用上面的 MCP JSON 配置,你也可以用一些内置子命令:++```bash+# 一键安装到 Claude Code,写到 ~/.claude.json 中+open-computer-use install-claude-mcp++# 一键安装到 Codex,写到 ~/.codex/config.toml 中+open-computer-use install-codex-mcp++# 一键安装到 Codex 插件,主要方便在 Codex App 中使用+open-computer-use install-codex-plugin++# 直接启动 MCP server+open-computer-use mcp++# 检查权限;只有缺失时才会拉起引导,已全部授权则只打印状态并退出+open-computer-use doctor++# 查看帮助+open-computer-use -h+```++## License++[MIT](./LICENSE)。diff --git a/apps/OpenComputerUse/Sources/OpenComputerUse/PermissionOnboardingApp.swift b/apps/OpenComputerUse/Sources/OpenComputerUse/PermissionOnboardingApp.swiftindex e3ffb14..a9bbd3e 100644--- a/apps/OpenComputerUse/Sources/OpenComputerUse/PermissionOnboardingApp.swift+++ b/apps/OpenComputerUse/Sources/OpenComputerUse/PermissionOnboardingApp.swift@@ -425,12 +425,16 @@ final class GuidancePlaceholderView: NSView {@MainActorfinal class PermissionAccessoryPanelController {+ private let bootstrapRetryInterval: TimeInterval = 0.15+ private let bootstrapRetryLimit = 24private var panel: NSPanel?private var currentPermission: SystemPermissionKind?private var workspaceObserver: NSObjectProtocol?private var globalDragMonitor: Any?private var localDragMonitor: Any?private var orderedWindowNumber: Int?+ private var bootstrapRetryTimer: Timer?+ private var bootstrapRetriesRemaining = 0private enum Layout {static let screenHorizontalInset: CGFloat = 16@@ -466,9 +470,11 @@ final class PermissionAccessoryPanelController {installObserversIfNeeded()position(panel: panel)updatePanelVisibility()+ startBootstrapRetries()}func hide() {+ stopBootstrapRetries()removeObservers()panel?.orderOut(nil)currentPermission = nil@@ -540,6 +546,35 @@ final class PermissionAccessoryPanelController {}}+ private func startBootstrapRetries() {+ stopBootstrapRetries()+ bootstrapRetriesRemaining = bootstrapRetryLimit+ bootstrapRetryTimer = Timer.scheduledTimer(withTimeInterval: bootstrapRetryInterval, repeats: true) { [weak self] _ in+ Task { @MainActor [weak self] in+ self?.handleBootstrapRetryTick()+ }+ }+ bootstrapRetryTimer?.tolerance = 0.03+ }++ private func stopBootstrapRetries() {+ bootstrapRetryTimer?.invalidate()+ bootstrapRetryTimer = nil+ bootstrapRetriesRemaining = 0+ }++ private func handleBootstrapRetryTick() {+ updatePanelVisibility()+ refreshPosition()++ let shouldStop = bootstrapRetriesRemaining <= 0 || (panel?.isVisible ?? false)+ bootstrapRetriesRemaining -= 1++ if shouldStop {+ stopBootstrapRetries()+ }+ }+private func refreshPosition() {guard let panel, currentPermission != nil, panel.isVisible else {return@@ -562,6 +597,10 @@ final class PermissionAccessoryPanelController {panel.order(.above, relativeTo: windowContext.windowNumber)orderedWindowNumber = windowContext.windowNumber}++ if panel.isVisible {+ stopBootstrapRetries()+ }}private func position(panel: NSPanel) {diff --git a/apps/OpenComputerUseSmokeSuite/Sources/OpenComputerUseSmokeSuite/main.swift b/apps/OpenComputerUseSmokeSuite/Sources/OpenComputerUseSmokeSuite/main.swiftindex cdac287..47e10bf 100644--- a/apps/OpenComputerUseSmokeSuite/Sources/OpenComputerUseSmokeSuite/main.swift+++ b/apps/OpenComputerUseSmokeSuite/Sources/OpenComputerUseSmokeSuite/main.swift@@ -34,7 +34,7 @@ final class MCPClient {_ = try request(method: "initialize", params: ["clientInfo": ["name": "OpenComputerUseSmokeSuite",- "version": "0.1.10",+ "version": "0.1.11",],"capabilities": [:],"protocolVersion": "2025-03-26",diff --git a/docs/histories/2026-04/20260417-2350-fix-permission-accessory-panel-ordering.md b/docs/histories/2026-04/20260417-2350-fix-permission-accessory-panel-ordering.mdindex 3989abe..c98694e 100644--- a/docs/histories/2026-04/20260417-2350-fix-permission-accessory-panel-ordering.md+++ b/docs/histories/2026-04/20260417-2350-fix-permission-accessory-panel-ordering.md@@ -22,6 +22,7 @@- **[Permission Identity Alignment]**: 权限状态查询不再只看 `Bundle.main.bundleIdentifier`;当 onboarding 实际引导用户拖入仓库内打包好的 `.app` 时,TCC 查询会改用那个真实 app bundle 的 identifier,避免列表里已经出现 `Open Computer Use` 但 UI 仍然不显示 `Done`。- **[Valid Bundle Guard]**: 回退到 `dist/Open Computer Use.app` 时会先验证它是否包含 `Info.plist`、可执行文件且 bundle id 正确,避免把空壳目录拖进系统设置后出现无图标、拖拽角标异常和权限状态不收敛的问题。- **[Path-Based TCC Detection]**: 适配 macOS TCC 把 `Open Computer Use.app` 授权记录存成 `client_type=1` 路径项的情况;权限查询现在会同时检查 app 路径和 bundle identifier,不再因为系统设置里已授权但数据库 key 不是 bundle id 而卡在 `Allow`。+- **[Cold Launch Bootstrap Retry]**: 当 `Allow` 会首次冷启动 `System Settings` 时,辅助 panel 现在会在短时间内重试挂载,直到系统设置窗口真正 ready;避免“第一次拉起系统设置看不到浮窗,切走再切回才出现”的时序问题。- **[Docs Sync]**: 更新架构文档,补充这块权限引导面板现在会保持在 `System Settings` 窗口之上的行为说明。### 🧠 Design Intent (Why)diff --git a/docs/histories/2026-04/20260418-0100-bump-open-computer-use-to-0.1.11.md b/docs/histories/2026-04/20260418-0100-bump-open-computer-use-to-0.1.11.mdnew file mode 100644index 0000000..c271b28--- /dev/null+++ b/docs/histories/2026-04/20260418-0100-bump-open-computer-use-to-0.1.11.md@@ -0,0 +1,30 @@+## [2026-04-18 01:00] | Task: 发布 0.1.11++### 🤖 Execution Context+* **Agent ID**: `codex`+* **Base Model**: `gpt-5.4`+* **Runtime**: `Codex CLI on macOS`++### 📥 User Query+> 加个小版本,提交所有改动,git tag推送++### 🛠 Changes Overview+**Scope:** `apps/`, `docs/`, `packages/`, `plugins/`, `scripts/`++**Key Actions:**+- **[Version Bump]**: 把插件 manifest、Swift/Go 版本常量、smoke suite 初始化版本和单测中的 client version 统一提升到 `0.1.11`。+- **[Release Scope]**: 把首次冷启动 `System Settings` 时权限浮窗不出现的修复,以及根目录新增中文 README 入口,一并纳入 patch release。+- **[Release Notes]**: 更新功能发布记录,为 `0.1.11` 补充用户价值与变更摘要。++### 🧠 Design Intent (Why)+这轮剩余改动虽然不大,但都直接影响首次授权体验和仓库入口文档。单独补一个 patch release,可以把“首次冷启动可见性修复”和“中文 README 恢复”从 `0.1.10` 的权限身份收口里切开,避免后续 tag 和实际用户可见行为继续混在一起。++### 📁 Files Modified+- `plugins/open-computer-use/.codex-plugin/plugin.json`+- `packages/OpenComputerUseKit/Sources/OpenComputerUseKit/OpenComputerUseVersion.swift`+- `apps/OpenComputerUseSmokeSuite/Sources/OpenComputerUseSmokeSuite/main.swift`+- `packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/OpenComputerUseKitTests.swift`+- `scripts/computer-use-cli/main.go`+- `scripts/computer-use-cli/README.md`+- `docs/releases/feature-release-notes.md`+- `docs/histories/2026-04/20260418-0100-bump-open-computer-use-to-0.1.11.md`diff --git a/docs/releases/feature-release-notes.md b/docs/releases/feature-release-notes.mdindex 30766b8..e93815d 100644--- a/docs/releases/feature-release-notes.md+++ b/docs/releases/feature-release-notes.md@@ -4,6 +4,7 @@| 日期 | 功能域 | 用户价值 | 变更摘要 || --- | --- | --- | --- |+| 2026-04-18 | 权限浮窗与文档入口 | 第一次冷启动系统设置时权限浮窗能直接出现,仓库根目录也重新补回中文文档入口。 | 发布 `0.1.11`,修复首次 `Allow` 冷启动 `System Settings` 时辅助浮窗需要切窗后才显示的时序问题,并新增根目录 `README.zh-CN.md` 承载中文说明。 || 2026-04-18 | 权限身份与 onboarding | npm 安装后的权限身份更稳定,已授权用户不会被重复弹窗打扰。 | 发布 `0.1.10`,统一 bundle identifier 为 `com.ifuryst.opencomputeruse`,让权限检测兼容路径型 TCC 记录并优先认 npm 全局安装后的 app;同时让 `doctor` / 默认启动在权限齐全时不再弹出 onboarding,完成授权后自动关窗。 || 2026-04-17 | 发布稳定性 | release workflow 不再因为 Xcode 26 的 CoreFoundation 类型检查而在构建阶段提前失败。 | 发布 `0.1.9`,修复权限引导窗口里 `AXUIElement` 属性读取在 `macos-26` / Xcode 26.2 下的编译错误,恢复 npm release artifact 构建链路。 || 2026-04-17 | 权限引导与安装 | 权限授权浮窗在 `Allow` 后不再掉到屏幕底部,且仓库继续提供稳定的一键安装/发布版本。 | 发布 `0.1.8`,收口 `System Settings` 跟随 panel 的定位修复,并同步更新插件、CLI、smoke/test 与发布文档中的版本号。 |diff --git a/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/OpenComputerUseVersion.swift b/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/OpenComputerUseVersion.swiftindex eb89a62..6256a76 100644--- a/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/OpenComputerUseVersion.swift+++ b/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/OpenComputerUseVersion.swift@@ -1,6 +1,6 @@import Foundation-public let openComputerUseVersion = "0.1.10"+public let openComputerUseVersion = "0.1.11"public func resolvedOpenComputerUseVersion(bundle: Bundle = .main) -> String {if let version = bundle.object(forInfoDictionaryKey: "CFBundleShortVersionString") as? String,diff --git a/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/OpenComputerUseKitTests.swift b/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/OpenComputerUseKitTests.swiftindex a6ac8aa..51c1d2e 100644--- a/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/OpenComputerUseKitTests.swift+++ b/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/OpenComputerUseKitTests.swift@@ -83,7 +83,7 @@ final class OpenComputerUseKitTests: XCTestCase {func testInitializeResponseContainsToolsCapability() throws {let server = StdioMCPServer(service: ComputerUseService())- let response = server.handle(line: #"{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","clientInfo":{"name":"test","version":"0.1.10"},"capabilities":{}}}"#)+ let response = server.handle(line: #"{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","clientInfo":{"name":"test","version":"0.1.11"},"capabilities":{}}}"#)XCTAssertNotNil(response)XCTAssertTrue(response!.contains(#""name":"open-computer-use""#))XCTAssertTrue(response!.contains(#""tools":{"listChanged":false}"#))@@ -92,7 +92,7 @@ final class OpenComputerUseKitTests: XCTestCase {func testInitializeResponseContainsComputerUseInstructions() throws {let server = StdioMCPServer(service: ComputerUseService())let response = try XCTUnwrap(- server.handle(line: #"{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","clientInfo":{"name":"test","version":"0.1.10"},"capabilities":{}}}"#)+ server.handle(line: #"{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","clientInfo":{"name":"test","version":"0.1.11"},"capabilities":{}}}"#))let data = try XCTUnwrap(response.data(using: .utf8))let json = try XCTUnwrap(JSONSerialization.jsonObject(with: data) as? [String: Any])diff --git a/plugins/open-computer-use/.codex-plugin/plugin.json b/plugins/open-computer-use/.codex-plugin/plugin.jsonindex 9df8c0e..2c71392 100644--- a/plugins/open-computer-use/.codex-plugin/plugin.json+++ b/plugins/open-computer-use/.codex-plugin/plugin.json@@ -1,6 +1,6 @@{"name": "open-computer-use",- "version": "0.1.10",+ "version": "0.1.11","description": "Open-source macOS Computer Use MCP server packaged as a Codex plugin.","author": {"name": "Leo",diff --git a/scripts/computer-use-cli/README.md b/scripts/computer-use-cli/README.mdindex b6671d0..482a1c9 100644--- a/scripts/computer-use-cli/README.md+++ b/scripts/computer-use-cli/README.md@@ -85,7 +85,7 @@ Example working invocation against `open-computer-use`:```bashgo run . list-tools \--transport direct \- --server-bin ~/.codex/plugins/cache/open-computer-use-local/open-computer-use/0.1.10/scripts/launch-open-computer-use.sh+ --server-bin ~/.codex/plugins/cache/open-computer-use-local/open-computer-use/0.1.11/scripts/launch-open-computer-use.sh```Example working invocation against the official bundled `computer-use`:diff --git a/scripts/computer-use-cli/main.go b/scripts/computer-use-cli/main.goindex 73c8a74..4d30f93 100644--- a/scripts/computer-use-cli/main.go+++ b/scripts/computer-use-cli/main.go@@ -21,7 +21,7 @@ import (const (cliName = "computer-use-cli"- cliVersion = "0.1.10"+ cliVersion = "0.1.11"defaultTimeout = 60 * time.SecondpluginRootEnvVar = "COMPUTER_USE_PLUGIN_ROOT"serverBinEnvVar = "COMPUTER_USE_SERVER_BIN"