Skip to content

👾 Open Computer Use – Open-Source Alternative to Codex Computer Use

Match official no-window error text

Leo19040fcf3589
diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md
index be89942..bfcb994 100644
--- a/docs/ARCHITECTURE.md
+++ b/docs/ARCHITECTURE.md
@@ -60,7 +60,7 @@
- `ComputerUseService` 负责把 Computer Use tool 请求映射到本地能力,`ComputerUseToolDispatcher` 则把 9 个 tool 的参数解析与 service 方法分发收敛成 MCP server 和 `open-computer-use call` 共用的一层。
- `list_apps` 通过 Spotlight metadata query 拉取标准 application 目录里的 app bundle,并读取 `kMDItemUseCount` / `kMDItemLastUsedDate_Ranking` 这类系统元数据;再与 `NSWorkspace` 的运行态 app 合并,输出“当前运行中 + 近 14 天用过”的视图。
-- `get_app_state` 优先走真实 AX / 窗口截图,但不再为了读状态而显式 `activate` 目标 app;真实 app 必须同时有未最小化的 `AXWindow` 和可匹配的 on-screen `CGWindow`,否则返回明确的窗口不可用错误,不再把 application 根节点或无截图窗口伪装成可操作状态;当目标是仓库内 fixture app 时,回退到 fixture 导出的合成状态。真实 AX tree 对 Electron/WebView 这类深层 UI 会放宽遍历深度,同时压缩空 `AXGroup` / `AXUnknown` wrapper、过滤 `AXScrollToVisible` 噪音和空字符串属性,避免 action-critical 的输入框被无语义容器挤出节点预算。
+- `get_app_state` 优先走真实 AX / 窗口截图,但不再为了读状态而显式 `activate` 目标 app;真实 app 必须同时有未最小化的 `AXWindow` 和可匹配的 on-screen `CGWindow`,否则返回官方风格的 `Apple event error -10005: cgWindowNotFound`,不再把 application 根节点或无截图窗口伪装成可操作状态;当目标是仓库内 fixture app 时,回退到 fixture 导出的合成状态。真实 AX tree 对 Electron/WebView 这类深层 UI 会放宽遍历深度,同时压缩空 `AXGroup` / `AXUnknown` wrapper、过滤 `AXScrollToVisible` 噪音和空字符串属性,避免 action-critical 的输入框被无语义容器挤出节点预算。
- MCP `tools/list` 的 description / input schema 当前按官方 `computer-use` 的 9 个 tools 文案和参数面收敛,尽量减少 host 侧提示词和 tool surface 偏差。
- `open-computer-use call <tool> --args '{...}'` 会直接输出 MCP-style JSON result;`open-computer-use call --calls '[...]'` / `--calls-file <path>` 会在同一进程里顺序执行 JSON 数组里的 tool calls,并复用同一个 `ComputerUseService` 内存态,因此 `get_app_state` 之后的 action tool 可以继续使用同一轮 snapshot 的 `element_index`。序列执行默认会在成功的相邻操作之间 sleep 1 秒,也可以用 `--sleep <seconds>` 覆盖;遇到 `isError=true` 的 tool result 后停止。
- 对真实 app 的 `get_app_state` / action tool 入口,当前只保留一层密码管理器 bundle denylist:bundle-id 直传时直接返回 safety denial;名称匹配时默认不解析到这些 app。终端、Chrome / Atlas 和系统组件不再属于内置阻止目标。
diff --git a/docs/histories/2026-05/20260507-1905-require-ax-window-for-snapshot.md b/docs/histories/2026-05/20260507-1905-require-ax-window-for-snapshot.md
index ac5989e..7676db9 100644
--- a/docs/histories/2026-05/20260507-1905-require-ax-window-for-snapshot.md
+++ b/docs/histories/2026-05/20260507-1905-require-ax-window-for-snapshot.md
@@ -36,3 +36,15 @@ Official `computer-use` returns an unavailable-window error for the observed Lar
- `git diff --check`
- `./scripts/run-tool-smoke-tests.sh`
- Manual comparison: official `computer-use` and current source build both return unavailable-window errors for the current Lark / Chrome / System Settings no-window state.
+
+### 🔁 Follow-up (2026-05-07, match official no-window text)
+
+**Additional changes:**
+- Introduced a shared `computerUseNoWindowFoundMessage` constant with the official observed text: `Apple event error -10005: cgWindowNotFound`.
+- Reused that message for both missing AX window and missing visible CG window paths.
+- Added a focused unit test for the exact no-window message.
+- Synchronized `docs/ARCHITECTURE.md` to call out the official-style no-window error text.
+
+**Validation:**
+- `swift test --filter NoWindowErrorMessageMatchesOfficialShape`
+- Manual comparison: source build now returns the same no-window text as official `computer-use` for current Lark and Chrome no-window states.
diff --git a/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/AccessibilitySnapshot.swift b/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/AccessibilitySnapshot.swift
index f573e5a..0e5f41b 100644
--- a/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/AccessibilitySnapshot.swift
+++ b/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/AccessibilitySnapshot.swift
@@ -92,12 +92,12 @@ enum SnapshotBuilder {
let systemWide = AXUIElementCreateSystemWide()
let focusedApplication = copyElement(systemWide, attribute: kAXFocusedApplicationAttribute)
guard let focusedWindow = preferredFocusedWindow(appElement: appElement, appPID: app.pid, focusedApplication: focusedApplication, systemWide: systemWide) else {
- throw ComputerUseError.stateUnavailable("No key window is available for \(app.bundleIdentifier ?? app.name). Bring the app window on screen and call get_app_state again.")
+ throw ComputerUseError.stateUnavailable(computerUseNoWindowFoundMessage)
}
let windowTitle = stringValue(of: focusedWindow, attribute: kAXTitleAttribute)
guard let windowCapture = WindowCapture.resolve(for: app.pid, titleHint: windowTitle) else {
- throw ComputerUseError.stateUnavailable("No visible window is available for \(app.bundleIdentifier ?? app.name). Bring the app window on screen and call get_app_state again.")
+ throw ComputerUseError.stateUnavailable(computerUseNoWindowFoundMessage)
}
let rootElement = focusedWindow
diff --git a/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/Errors.swift b/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/Errors.swift
index 0e8a3e4..88209a2 100644
--- a/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/Errors.swift
+++ b/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/Errors.swift
@@ -1,5 +1,7 @@
import Foundation
+let computerUseNoWindowFoundMessage = "Apple event error -10005: cgWindowNotFound"
+
public enum ComputerUseError: Error, LocalizedError {
case message(String)
case unsupportedTool(String)
diff --git a/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/OpenComputerUseKitTests.swift b/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/OpenComputerUseKitTests.swift
index 00be210..3d503e6 100644
--- a/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/OpenComputerUseKitTests.swift
+++ b/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/OpenComputerUseKitTests.swift
@@ -701,6 +701,14 @@ final class OpenComputerUseKitTests: XCTestCase {
XCTAssertTrue(ComputerUseError.invalidArguments("bad").toolResultIsError)
}
+ func testNoWindowErrorMessageMatchesOfficialShape() {
+ XCTAssertEqual(computerUseNoWindowFoundMessage, "Apple event error -10005: cgWindowNotFound")
+ XCTAssertEqual(
+ ComputerUseError.stateUnavailable(computerUseNoWindowFoundMessage).errorDescription,
+ "Apple event error -10005: cgWindowNotFound"
+ )
+ }
+
func testAppSafetyPolicyDoesNotBlockNonPasswordApps() {
XCTAssertFalse(AppSafetyPolicy.isBlocked(bundleIdentifier: "com.google.Chrome"))
XCTAssertFalse(AppSafetyPolicy.isBlocked(bundleIdentifier: "com.googlecode.iterm2"))