👾 Open Computer Use – Open-Source Alternative to Codex Computer Use
Swift accessibilityai-agentai-agentsclaude-codecodexcodex-computer-usecomputer-usecomputer-use-agentcross-platformcuadesktop-automationgemini-cligui-automationmacosmcpmodel-context-protocolopen-computer-useopen-computer-use-mcpopenai-codexopencode
feat(snapshot): unify tree budget defaults
tisfeng405bc4392b0b
diff --git a/apps/OpenComputerUseLinux/main_test.go b/apps/OpenComputerUseLinux/main_test.goindex 6e28938..a43f751 100644--- a/apps/OpenComputerUseLinux/main_test.go+++ b/apps/OpenComputerUseLinux/main_test.go@@ -148,6 +148,15 @@ func TestLinuxRuntimeTextLimitSupportsFullTextMode(t *testing.T) {}}+func TestLinuxRuntimeTreeBudgetDefaultsMatchMacOS(t *testing.T) {+ if !strings.Contains(linuxRuntimeScript, "MAX_ELEMENTS = 1200") {+ t.Fatal("Linux runtime should default to the shared 1200 node tree budget")+ }+ if !strings.Contains(linuxRuntimeScript, "MAX_DEPTH = 64") {+ t.Fatal("Linux runtime should default to the shared 64 level tree depth")+ }+}+func TestLinuxRuntimeEnvironmentDiscoversDesktopSession(t *testing.T) {runtimeDir := shortTempDir(t)listenUnixSocket(t, filepath.Join(runtimeDir, "bus"))diff --git a/apps/OpenComputerUseLinux/runtime.py b/apps/OpenComputerUseLinux/runtime.pyindex a147234..32d8a82 100644--- a/apps/OpenComputerUseLinux/runtime.py+++ b/apps/OpenComputerUseLinux/runtime.py@@ -24,7 +24,7 @@ except (ImportError, ValueError):from gi.repository import Atspi-MAX_ELEMENTS = 500+MAX_ELEMENTS = 1200MAX_DEPTH = 64TEXT_CHARACTER_LIMIT = 500diff --git a/apps/OpenComputerUseWindows/main_test.go b/apps/OpenComputerUseWindows/main_test.goindex a2adcf3..0739433 100644--- a/apps/OpenComputerUseWindows/main_test.go+++ b/apps/OpenComputerUseWindows/main_test.go@@ -158,6 +158,18 @@ func TestWindowsRuntimeTextLimitSupportsFullTextMode(t *testing.T) {}}+func TestWindowsRuntimeTreeBudgetDefaultsMatchMacOS(t *testing.T) {+ if !strings.Contains(windowsRuntimeScript, "$AccessibilityTreeMaxNodeCount = 1200") {+ t.Fatal("Windows runtime should default to the shared 1200 node tree budget")+ }+ if !strings.Contains(windowsRuntimeScript, "$AccessibilityTreeMaxDepth = 64") {+ t.Fatal("Windows runtime should default to the shared 64 level tree depth")+ }+ if !strings.Contains(windowsRuntimeScript, "$script:nextIndex -ge $script:AccessibilityTreeMaxNodeCount -or $depth -gt $script:AccessibilityTreeMaxDepth") {+ t.Fatal("Windows runtime should use shared tree budget constants while rendering")+ }+}+func findToolDefinition(t *testing.T, name string) toolDefinition {t.Helper()for _, tool := range toolDefinitions() {diff --git a/apps/OpenComputerUseWindows/runtime.ps1 b/apps/OpenComputerUseWindows/runtime.ps1index 2a38ad3..47c28b9 100644--- a/apps/OpenComputerUseWindows/runtime.ps1+++ b/apps/OpenComputerUseWindows/runtime.ps1@@ -5,6 +5,8 @@ param($ErrorActionPreference = "Stop"$TextCharacterLimit = 500+$AccessibilityTreeMaxNodeCount = 1200+$AccessibilityTreeMaxDepth = 64# Set output encoding to UTF-8 to properly handle non-ASCII characters$OutputEncoding = [System.Text.Encoding]::UTF8@@ -479,7 +481,7 @@ function Render-Tree($element, $windowBounds, [bool]$ShowFullText = $false) {$nextIndex = 0function Visit($node, [int]$depth) {- if ($script:nextIndex -ge 500 -or $depth -gt 16) {+ if ($script:nextIndex -ge $script:AccessibilityTreeMaxNodeCount -or $depth -gt $script:AccessibilityTreeMaxDepth) {return}$runtime = ""diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.mdindex 234452b..6127364 100644--- a/docs/ARCHITECTURE.md+++ b/docs/ARCHITECTURE.md@@ -62,7 +62,7 @@- `ComputerUseService` 负责把 Computer Use tool 请求映射到本地能力,`ComputerUseToolDispatcher` 则把 9 个 tool 的参数解析与 service 方法分发收敛成 MCP server 和 `open-computer-use call` 共用的一层。- `list_apps` 通过 Spotlight metadata query 拉取标准 application 目录里的 app bundle,并读取 `kMDItemUseCount` / `kMDItemLastUsedDate_Ranking` 这类系统元数据;再与 `NSWorkspace` 的运行态 app 合并,输出“当前运行中 + 近 14 天用过”的视图。-- `get_app_state` 优先走真实 AX / 窗口截图;真实 app 必须同时有未最小化的 `AXWindow` 和可匹配的 on-screen `CGWindow`。如果目标 app 只是隐藏或暂时没有 on-screen window,会先 best-effort unhide / activate / `open -b` / `AXRaise` 并短暂重试,以贴近官方 `computer-use` 会把 Lark / Electron 窗口拉回再采集的行为;恢复后仍无法匹配时返回官方风格的 `Apple event error -10005: cgWindowNotFound`,不再把 application 根节点或无截图窗口伪装成可操作状态。当目标是仓库内 fixture app 时,回退到 fixture 导出的合成状态。真实 AX tree 对 Electron/WebView 这类深层 UI 会放宽遍历深度,同时压缩空 `AXGroup` / `AXUnknown` wrapper、过滤 `AXScrollToVisible` 噪音和空字符串属性,避免 action-critical 的输入框被无语义容器挤出节点预算;对原生 open panel / Finder column view 这类把内容放在 `AXContents` / `AXVisibleChildren` 里的控件,也会把可见文件项纳入元素树。+- `get_app_state` 优先走真实 AX / 窗口截图;真实 app 必须同时有未最小化的 `AXWindow` 和可匹配的 on-screen `CGWindow`。如果目标 app 只是隐藏或暂时没有 on-screen window,会先 best-effort unhide / activate / `open -b` / `AXRaise` 并短暂重试,以贴近官方 `computer-use` 会把 Lark / Electron 窗口拉回再采集的行为;恢复后仍无法匹配时返回官方风格的 `Apple event error -10005: cgWindowNotFound`,不再把 application 根节点或无截图窗口伪装成可操作状态。当目标是仓库内 fixture app 时,回退到 fixture 导出的合成状态。真实 AX tree 默认在 macOS、Linux、Windows 上最多渲染 1200 个节点、64 层深度;对 Electron/WebView 这类深层 UI 会压缩空 `AXGroup` / `AXUnknown` wrapper、过滤 `AXScrollToVisible` 噪音和空字符串属性,避免 action-critical 的输入框被无语义容器挤出节点预算;对原生 open panel / Finder column view 这类把内容放在 `AXContents` / `AXVisibleChildren` 里的控件,也会把可见文件项纳入元素树。- MCP `tools/list` 的 description / input schema 当前按官方 `computer-use` 的 9 个 tools 文案和参数面收敛,尽量减少 host 侧提示词和 tool surface 偏差。- `open-computer-use call <tool> --args '{...}'` 会直接输出 MCP-style JSON result;`open-computer-use call --calls '[...]'` / `--calls-file <path>` 会在同一进程里顺序执行 JSON 数组里的 tool calls,并复用同一个 `ComputerUseService` 内存态,因此 `get_app_state` 之后的 action tool 可以继续使用同一轮 snapshot 的 `element_index`。序列执行默认会在成功的相邻操作之间 sleep 1 秒,也可以用 `--sleep <seconds>` 覆盖;遇到 `isError=true` 的 tool result 后停止。- 对真实 app 的 `get_app_state` / action tool 入口,当前只保留一层密码管理器 bundle denylist:bundle-id 直传时直接返回 safety denial;名称匹配时默认不解析到这些 app。终端、Chrome / Atlas 和系统组件不再属于内置阻止目标。@@ -119,7 +119,7 @@- Go runtime 通过 `go:embed` 带上 `runtime.py`,执行 tool call 时临时落盘并调用 `python3`。Python bridge 使用 GNOME/GObject Introspection 暴露的 AT-SPI2 接口做 app/window discovery、accessibility tree rendering、semantic action、editable text、value set,以及 best-effort 的 key/mouse fallback。- Linux 上最接近 macOS AX 的是 AT-SPI2/D-Bus accessibility,而不是一套统一的后台键鼠输入模型。第一版优先使用元素暴露的 AT-SPI action、EditableText 和 Value 接口;coordinate `click` / `drag` 与 `press_key` 使用 AT-SPI event synthesis fallback,在 Wayland 下只能按 best-effort 处理。- Linux runtime 需要运行在已登录桌面用户 session 里。缺少 `XDG_RUNTIME_DIR`、`DBUS_SESSION_BUS_ADDRESS` 或 display 环境时,Go runtime 会在启动 Python AT-SPI bridge 前尝试从 `/run/user/<uid>` 和常见桌面进程自动发现当前用户的 session bus、display / Wayland 值;纯 SSH tty 如果找不到已登录桌面 session,可以启动二进制,但不能直接 inspect 或操作 GUI session。-- `get_app_state` 的 accessibility tree 在 GTK/GNOME app 上可能很深,Linux bridge 当前把 traversal depth 放宽到 64。截图通过 GDK root window best-effort capture;GNOME Wayland 可能返回黑图,bridge 会检测全黑采样并省略 image block。+- `get_app_state` 的 accessibility tree 在 GTK/GNOME app 上可能很深,Linux bridge 使用与 macOS / Windows 一致的 1200 节点、64 层默认 tree budget。截图通过 GDK root window best-effort capture;GNOME Wayland 可能返回黑图,bridge 会检测全黑采样并省略 image block。- 这 9 个 tool 的协议面与 macOS / Windows 保持一致:`list_apps`、`get_app_state`、`click`、`perform_secondary_action`、`scroll`、`drag`、`type_text`、`press_key`、`set_value`。其中 element-targeted action 会优先复用上一轮 `get_app_state` 的 runtime path metadata,coordinate action 使用 screenshot/window-relative 坐标。- 当前 Linux 侧仍是功能性第一版:没有 visual cursor overlay、没有 installer/desktop entry,也没有独立 Linux fixture。后续 TODO 记录在 `docs/exec-plans/active/20260422-linux-computer-use-runtime.md`。diff --git a/docs/histories/2026-06/20260630-1635-unify-snapshot-tree-budget-defaults.md b/docs/histories/2026-06/20260630-1635-unify-snapshot-tree-budget-defaults.mdnew file mode 100644index 0000000..a67cdc5--- /dev/null+++ b/docs/histories/2026-06/20260630-1635-unify-snapshot-tree-budget-defaults.md@@ -0,0 +1,29 @@+## [2026-06-30 16:35] | Task: Unify snapshot tree budget defaults++### Execution Context+* **Agent ID**: `Codex`+* **Base Model**: `GPT-5`+* **Runtime**: `macOS local CLI`++### User Query+> 先修改 默认统一为 1200/64。++### Changes Overview+**Scope:** Linux and Windows snapshot tree rendering defaults, tests, and docs.++**Key Actions:**+- Unified Linux and Windows default accessibility tree node/depth budgets with macOS at 1200 nodes and 64 levels.+- Replaced the Windows renderer's hard-coded `500/16` guard with named tree budget constants.+- Added Linux and Windows regression tests that lock the shared `1200/64` defaults.+- Updated architecture and skill troubleshooting docs so agents can distinguish text truncation from tree budget limits.++### Design Intent+The previous platform defaults had drifted: macOS used `1200/64`, Linux used `500/64`, and Windows used `500/16`. The lower Linux/Windows budgets could hide visible content on deep or long accessibility trees, while the macOS defaults had already been tuned for Electron/WebView-style apps. The shared default keeps bounded snapshots while reducing cross-platform surprise.++### Files Modified+- `apps/OpenComputerUseLinux/runtime.py`+- `apps/OpenComputerUseLinux/main_test.go`+- `apps/OpenComputerUseWindows/runtime.ps1`+- `apps/OpenComputerUseWindows/main_test.go`+- `docs/ARCHITECTURE.md`+- `skills/open-computer-use/references/troubleshooting.md`diff --git a/skills/open-computer-use/references/troubleshooting.md b/skills/open-computer-use/references/troubleshooting.mdindex 0d079d1..9e0b6b3 100644--- a/skills/open-computer-use/references/troubleshooting.md+++ b/skills/open-computer-use/references/troubleshooting.md@@ -48,7 +48,7 @@ open-computer-use call get_app_state --args '{"app":"TextEdit","show_full_text":open-computer-use snapshot --show-full-text TextEdit```-`show_full_text` only disables the text character limit. It does not remove node count, tree depth, screenshot size, permission, or desktop-session protections.+`show_full_text` only disables the text character limit. It does not remove the default 1200 node count limit, 64 level tree depth limit, screenshot size, permission, or desktop-session protections.## Element Action Fails