Skip to content

👾 Open Computer Use – Open-Source Alternative to Codex Computer Use

docs: refresh readme demos and badges

Leo87d7bafe76ae
diff --git a/README.md b/README.md
index dce6374..9280b9d 100644
--- a/README.md
+++ b/README.md
@@ -1,36 +1,59 @@
# open-computer-use
-[中文说明](./README.zh-CN.md)
+[![English](https://img.shields.io/badge/English-Click-yellow)](./README.md)
+[![简体中文](https://img.shields.io/badge/简体中文-点击查看-orange)](./README.zh-CN.md)
+[![Release](https://img.shields.io/github/v/release/iFurySt/open-codex-computer-use)](https://github.com/iFurySt/open-codex-computer-use/releases)
+[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/iFurySt/open-codex-computer-use)
+<a href="https://llmapis.com?source=https%3A%2F%2Fgithub.com%2FiFurySt%2Fopen-codex-computer-use" target="_blank"><img src="https://llmapis.com/api/badge/iFurySt/open-codex-computer-use" alt="LLMAPIS" width="20" /></a>
+
+---
+
+`open-computer-use` is an open-source `Computer Use` service wrapped as `MCP`. Any AI agent or MCP client can use it to run Computer Use on macOS, Linux, and Windows.
+
+This project was inspired by OpenAI's [Codex Computer Use](https://openai.com/index/codex-for-almost-everything/). It showed that non-intrusive CUA can be built on top of Accessibility, so I decided to build an open-source version.
+
+I started this repo with my [harness template](https://github.com/iFurySt/harness-template), a template for quickly spinning up AI-first projects. It has been one of our most useful workflows lately, especially for nearly 100% AI-generated projects. I also wrote [a post](https://www.ifuryst.com/blog/2026/speedrunning-the-ai-era/) about the methodology behind it.
+
+## Demos
+
+### Codex App and Codex CLI
[![Open Computer Use custom demo cover](./docs/generated/readme-assets/open-computer-use-demo-cover.png)](https://youtu.be/2s6aVpGiwaQ)
+<sub><em>`open-computer-use` used as Computer Use in Codex App and Codex CLI, matching the official experience.</em></sub>
+
+### Gemini CLI
+
https://github.com/user-attachments/assets/eacb3b15-f939-46c7-b3b3-6f876977a58d
-<sub><em>Gemini CLI connected to `open-computer-use` through MCP, driving real Computer Use actions end to end.</em></sub>
+<sub><em>Gemini CLI connects to `open-computer-use` through MCP and runs full Computer Use actions.</em></sub>
-`open-computer-use` is an open-source `Computer Use` service exposed over `MCP`, so any AI agent or MCP client can call it directly and use computer interaction capabilities on macOS, Linux, and Windows. The macOS runtime ships as an app bundle, while the Windows and Linux runtimes expose the same 9-tool surface through standalone Go-built binaries.
+### Linux
-This project was inspired by OpenAI's recently released [Codex Computer Use](https://openai.com/index/codex-for-almost-everything/). It showed that non-intrusive CUA can be built on top of macOS Accessibility, which is why I decided to build an open-source version.
+https://github.com/user-attachments/assets/e036b1c8-2200-4896-abd4-19225915cf66
-I bootstrapped this repo with my earlier [harness template](https://github.com/iFurySt/harness-template). It is a template for spinning up an AI-oriented repository quickly, especially for projects that are close to 100% AI-generated. This has been one of our most useful workflows over the past month, and it now lets us ship new ideas very quickly. If you are interested, I also wrote [a post](https://www.ifuryst.com/blog/2026/speedrunning-the-ai-era/) about the methodology behind it.
+<sub><em>`open-computer-use` running on Linux.</em></sub>
## Quick Start
-The npm package bundles the macOS, Linux, and Windows native runtimes, then the root launcher selects the artifact that matches your current `os-arch`:
-
```bash
npm i -g open-computer-use
```
-On macOS, grant `Accessibility` and `Screen Recording` permission to the `Open Computer Use.app` you actually plan to keep installed. The CI-built release package remains the stable identity for distribution. Local debug/dev builds are intentionally packaged as `Open Computer Use (Dev).app`, so System Settings shows them as a separate development app instead of another indistinguishable `Open Computer Use`.
+**On macOS, run it once and grant `Accessibility` and `Screen Recording`. Windows and Linux do not need this step.**
+
+```bash
+open-computer-use
+```
-On Linux and Windows, run it in the signed-in desktop session so AT-SPI2 or UI Automation can see GUI apps. To confirm the bundled native runtime is wired correctly, run:
+Before using it, install it into your agent:
```bash
-open-computer-use --version
+# Install into Codex by writing to ~/.codex/config.toml
+open-computer-use install-codex-mcp
```
-Then add it to your MCP client:
+Or add it to your own client manually:
```json
{
@@ -45,77 +68,46 @@ Then add it to your MCP client:
## More
-Besides using the MCP JSON config above, you can also use the built-in subcommands:
+Besides the MCP JSON config above, you can also use the built-in commands:
```bash
+# Install into Codex by writing to ~/.codex/config.toml
+open-computer-use install-codex-mcp
+
+# Install as a Codex plugin, mainly for Codex App
+open-computer-use install-codex-plugin
+
# Install into Claude Code by writing to ~/.claude.json
open-computer-use install-claude-mcp
+
# Install into Gemini CLI for the current project by writing to ./.gemini/settings.json
open-computer-use install-gemini-mcp
+
# Install into Gemini CLI user config instead
open-computer-use install-gemini-mcp --scope user
-# Install into Codex by writing to ~/.codex/config.toml
-open-computer-use install-codex-mcp
+
# Install into opencode by writing to ~/.config/opencode/opencode.json (or the active config file)
open-computer-use install-opencode-mcp
-# Install as a Codex plugin, mainly for Codex App usage; if you use this, you usually do not need install-codex-mcp as well
-open-computer-use install-codex-plugin
-# Start the MCP server directly
-open-computer-use mcp
+
# Call a single Computer Use tool and print the MCP-style JSON result
open-computer-use call list_apps
open-computer-use call get_app_state --args '{"app":"TextEdit"}'
+
# Run a sequence in one process so element_index state can be reused
# Sequence runs sleep 1s between successful operations by default
open-computer-use call --calls '[{"tool":"get_app_state","args":{"app":"TextEdit"}},{"tool":"press_key","args":{"app":"TextEdit","key":"Return"}}]'
open-computer-use call --calls-file examples/textedit-overlay-seq.json --sleep 0.5
+
# Check permissions; onboarding only opens when something is missing
open-computer-use doctor
+
# Show help
open-computer-use -h
```
-## Windows Runtime
-
-The Windows runtime is intentionally separate from the macOS Swift `.app`. It is built from `apps/OpenComputerUseWindows` and uses Windows UI Automation first, then Win32 window messages for fallback input.
-
-```bash
-# Build a Windows arm64 executable from this repo
-./scripts/build-open-computer-use-windows.sh --arch arm64
-
-# On Windows, run it directly
-open-computer-use.exe mcp
-open-computer-use.exe call list_apps
-open-computer-use.exe call --calls "[{\"tool\":\"get_app_state\",\"args\":{\"app\":\"notepad\"}},{\"tool\":\"type_text\",\"args\":{\"app\":\"notepad\",\"text\":\"hello\"}}]"
-```
-
-Run the `.exe` in the signed-in desktop session. Running it as a Windows service or a detached SSH-only process may not expose top-level UI Automation windows.
-
-By default, the Windows runtime only attaches to already running apps, does not call `SetFocus`, and avoids the UIA `ValuePattern.SetValue` fallback for `type_text` because some apps bring themselves forward from that path. If you explicitly want the old foreground behavior, set `OPEN_COMPUTER_USE_WINDOWS_ALLOW_APP_LAUNCH=1` to allow app launch fallback, `OPEN_COMPUTER_USE_WINDOWS_ALLOW_FOCUS_ACTIONS=1` to allow the `SetFocus` secondary action, and `OPEN_COMPUTER_USE_WINDOWS_ALLOW_UIA_TEXT_FALLBACK=1` to allow UIA text fallback.
-
-## Linux Runtime
-
-The Linux runtime is also separate from the macOS Swift `.app`. It is built from `apps/OpenComputerUseLinux` and uses AT-SPI2 accessibility over the signed-in desktop session's D-Bus bus. Semantic actions, editable text, and value updates are preferred; coordinate mouse, drag, and key synthesis are best-effort fallbacks rather than a universal Wayland background input model.
-
-```bash
-# Build a Linux arm64 binary from this repo
-./scripts/build-open-computer-use-linux.sh --arch arm64
-
-# On Linux, run it in the signed-in desktop session
-open-computer-use mcp
-open-computer-use call list_apps
-open-computer-use call --calls '[{"tool":"get_app_state","args":{"app":"gnome-text-editor"}},{"tool":"type_text","args":{"app":"gnome-text-editor","text":"hello"}}]'
-```
-
-The runtime needs the desktop user's D-Bus and display session. When those variables are missing, it tries to discover the current user's signed-in desktop session at launch from `/run/user/<uid>` and common desktop processes, so the normal Codex setup remains `npm i -g open-computer-use`, `open-computer-use install-codex-mcp`, then restart Codex from the same desktop user. A pure SSH tty without a discoverable signed-in desktop session can build and launch the binary, but it cannot inspect or operate the GUI session. Screenshot capture is best-effort on GNOME Wayland and may be omitted when the compositor returns a black frame.
-
## Cursor Motion
-Cursor Motion is an open-source cursor motion system for macOS, based on public information shared by members of the Software.Inc team. You can run it from source or download the app from the [Releases page](https://github.com/iFurySt/open-codex-computer-use/releases).
-
-```bash
-swift run CursorMotion
-```
+Cursor Motion is an open-source cursor motion system for macOS, based on public information shared by members of the Software.Inc team. You can download the app from the [Releases page](https://github.com/iFurySt/open-codex-computer-use/releases).
[![Cursor Motion custom demo cover](./docs/generated/readme-assets/cursor-motion-demo-cover.png)](https://youtu.be/KRUq5GUHv1Q)
diff --git a/README.zh-CN.md b/README.zh-CN.md
index cb9aa8b..cbfaf65 100644
--- a/README.zh-CN.md
+++ b/README.zh-CN.md
@@ -1,36 +1,57 @@
# open-computer-use
-[English Version](./README.md)
+[![English](https://img.shields.io/badge/English-Click-yellow)](./README.md)
+[![简体中文](https://img.shields.io/badge/简体中文-点击查看-orange)](./README.zh-CN.md)
+[![Release](https://img.shields.io/github/v/release/iFurySt/open-codex-computer-use)](https://github.com/iFurySt/open-codex-computer-use/releases)
+[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/iFurySt/open-codex-computer-use)
+<a href="https://llmapis.com?source=https%3A%2F%2Fgithub.com%2FiFurySt%2Fopen-codex-computer-use" target="_blank"><img src="https://llmapis.com/api/badge/iFurySt/open-codex-computer-use" alt="LLMAPIS" width="20" /></a>
+
+---
+
+`open-computer-use` 是一个开源的 `Computer Use` 服务,已经包装成 `MCP` 协议,支持所有的 AI Agent 或 MCP Client 快速调用,实现 macOS、Linux 和 Windows 上的 `Computer Use` 能力。
+
+项目的背后是 OpenAI 刚发布的 [Codex Computer Use](https://openai.com/index/codex-for-almost-everything/),让我看到了基于 Accessibility 可以实现非抢占式 CUA 能力,因此决定复刻一个开源版本
+
+在这期间我利用了之前写的 [harness 模版](https://github.com/iFurySt/harness-template) 开启了这个新项目。这是一个可以快速拉起面向 AI 仓库的 template,非常适合 100% AI-Generated 的项目,也是这一个月来我们最大的实践和收获。现在我们可以基于这套方法论快速实现很多东西;如果你有兴趣,我也写了一篇[文章](https://www.ifuryst.com/blog/2026/speedrunning-the-ai-era/)专门介绍这套方法论
+
+## 演示
+
+### Codex App 和 Codex CLI
[![open-computer-use 自定义演示封面](./docs/generated/readme-assets/open-computer-use-demo-cover.png)](https://youtu.be/2s6aVpGiwaQ)
+<sub><em>`open-computer-use` 作为 Computer Use,在 Codex App 和 Codex CLI 里使用,和官方体验一致。</em></sub>
+
+### Gemini CLI
+
https://github.com/user-attachments/assets/eacb3b15-f939-46c7-b3b3-6f876977a58d
-<sub><em>Gemini CLI 作为 host 接入 `open-computer-use` MCP,并完整触发真实的 Computer Use 操作。</em></sub>
+<sub><em>Gemini CLI 通过MCP接入使用 `open-computer-use`,实现完整的 Computer Use 操作。</em></sub>
-`open-computer-use` 是一个开源的 `Computer Use` 服务,已经包装成 `MCP` 协议,支持所有的 AI Agent 或 MCP Client 快速调用,实现 macOS、Linux 和 Windows 上的 `Computer Use` 能力。macOS 侧以 app bundle 分发,Windows 和 Linux runtime 用 Go 生成独立二进制,暴露同样的 9 个 tool。
+### Linux
-项目的背后是 OpenAI 刚发布的 [Codex Computer Use](https://openai.com/index/codex-for-almost-everything/),让我看到了基于 Accessibility 可以实现非抢占式 CUA 能力,因此决定复刻一个开源版本
+https://github.com/user-attachments/assets/e036b1c8-2200-4896-abd4-19225915cf66
-在这期间我利用了之前写的 [harness 模版](https://github.com/iFurySt/harness-template) 开启了这个新项目。这是一个可以快速拉起面向 AI 仓库的 template,非常适合 100% AI-Generated 的项目,也是这一个月来我们最大的实践和收获。现在我们可以基于这套方法论快速实现很多东西;如果你有兴趣,我也写了一篇[文章](https://www.ifuryst.com/blog/2026/speedrunning-the-ai-era/)专门介绍这套方法论
+<sub><em>`open-computer-use` 在 Linux 里使用</em></sub>
## Quick Start
-当前 npm 包会内置 macOS、Linux、Windows 的 native runtime,并由 root launcher 按当前 `os-arch` 拉起对应制品:
-
```bash
npm i -g open-computer-use
```
-macOS 第一次使用前,给你实际准备长期保留的那个 `Open Computer Use.app` 授予 `Accessibility` 和 `Screen Recording` 权限。CI 产出的 release 包继续作为正式分发身份;本地 debug/dev 构建会故意打成 `Open Computer Use (Dev).app`,这样系统设置里会明确显示成一个开发版 app,而不是再出现两个同名的 `Open Computer Use`。
-
-Linux 和 Windows 需要跑在已登录桌面 session 里,这样 AT-SPI2 或 UI Automation 才能看到 GUI app。先用下面的命令确认内置 native runtime 已经接好:
+**macOS 第一次使用前,需要授权 `Accessibility` 和 `Screen Recording` 的权限,windows和linux无需执行**
+```bash
+open-computer-use
+```
+开始用前可以通过一键安装到主流的Agent里:
```bash
-open-computer-use --version
+# 一键安装到 Codex,写到 ~/.codex/config.toml 中
+open-computer-use install-codex-mcp
```
-接着把它配到你的 MCP client 里:
+也可以手动配置到你自己的客户端里:
```json
{
@@ -48,6 +69,12 @@ open-computer-use --version
除了直接用上面的 MCP JSON 配置,你也可以用一些内置子命令:
```bash
+# 一键安装到 Codex,写到 ~/.codex/config.toml 中
+open-computer-use install-codex-mcp
+
+# 一键安装到 Codex 插件,主要方便在 Codex App 中使用
+open-computer-use install-codex-plugin
+
# 一键安装到 Claude Code,写到 ~/.claude.json 中
open-computer-use install-claude-mcp
@@ -57,18 +84,9 @@ open-computer-use install-gemini-mcp
# 一键安装到 Gemini CLI 用户级配置
open-computer-use install-gemini-mcp --scope user
-# 一键安装到 Codex,写到 ~/.codex/config.toml 中
-open-computer-use install-codex-mcp
-
# 一键安装到 opencode,写到 ~/.config/opencode/opencode.json(或当前生效的配置文件)
open-computer-use install-opencode-mcp
-# 一键安装到 Codex 插件,主要方便在 Codex App 中使用
-open-computer-use install-codex-plugin
-
-# 直接启动 MCP server
-open-computer-use mcp
-
# 直接调用单个 Computer Use tool,输出 MCP 风格的 JSON result
open-computer-use call list_apps
open-computer-use call get_app_state --args '{"app":"TextEdit"}'
@@ -85,47 +103,9 @@ open-computer-use doctor
open-computer-use -h
```
-## Windows Runtime
-
-Windows 侧不复用 Swift `.app`,而是放在 `apps/OpenComputerUseWindows` 里独立构建;执行时优先走 Windows UI Automation,必要时用 Win32 window message 做 fallback。
-
-```bash
-# 从仓库里构建 Windows arm64 exe
-./scripts/build-open-computer-use-windows.sh --arch arm64
-
-# 在 Windows 里直接运行
-open-computer-use.exe mcp
-open-computer-use.exe call list_apps
-open-computer-use.exe call --calls "[{\"tool\":\"get_app_state\",\"args\":{\"app\":\"notepad\"}},{\"tool\":\"type_text\",\"args\":{\"app\":\"notepad\",\"text\":\"hello\"}}]"
-```
-
-这个 `.exe` 需要跑在已登录的桌面 session 里。作为 Windows service 或纯 SSH 脱离桌面运行时,系统可能不给它暴露顶层 UI Automation 窗口。
-
-默认情况下,Windows runtime 只连接已经运行的 app,不会自动启动目标 app,也不会执行 `SetFocus`;`type_text` 也会避开 UIA `ValuePattern.SetValue` fallback,因为有些 app 会在这条路径里主动把自己带到前台。如果确实需要旧的前台行为,可以设置 `OPEN_COMPUTER_USE_WINDOWS_ALLOW_APP_LAUNCH=1` 允许启动 fallback,设置 `OPEN_COMPUTER_USE_WINDOWS_ALLOW_FOCUS_ACTIONS=1` 允许 `SetFocus` secondary action,设置 `OPEN_COMPUTER_USE_WINDOWS_ALLOW_UIA_TEXT_FALLBACK=1` 允许 UIA text fallback。
-
-## Linux Runtime
-
-Linux 侧也不复用 Swift `.app`,而是放在 `apps/OpenComputerUseLinux` 里独立构建;执行时走已登录桌面 session 的 AT-SPI2 / D-Bus accessibility。默认优先用语义化 action、editable text 和 value 接口;坐标鼠标、拖拽和键盘合成只是 best-effort fallback,不等价于一套通用的 Wayland 后台输入模型。
-
-```bash
-# 从仓库里构建 Linux arm64 binary
-./scripts/build-open-computer-use-linux.sh --arch arm64
-
-# 在 Linux 已登录桌面 session 里直接运行
-open-computer-use mcp
-open-computer-use call list_apps
-open-computer-use call --calls '[{"tool":"get_app_state","args":{"app":"gnome-text-editor"}},{"tool":"type_text","args":{"app":"gnome-text-editor","text":"hello"}}]'
-```
-
-runtime 需要桌面用户的 D-Bus 和 display session。缺少这些变量时,它会在启动时尝试从 `/run/user/<uid>` 和常见桌面进程自动发现当前用户已登录的桌面 session,所以 Codex 的正常接入路径仍然是 `npm i -g open-computer-use`、`open-computer-use install-codex-mcp`,然后用同一个桌面用户重启 Codex。纯 SSH tty 如果找不到已登录桌面 session,可以构建和启动二进制,但不能直接 inspect 或操作 GUI session。GNOME Wayland 下截图是 best-effort,如果 compositor 返回黑图,Linux bridge 会省略 image block。
-
## Cursor Motion
-Cursor Motion 是一个面向 macOS 的开源光标运动系统,基于 Software.Inc 几位大佬的公开信息实现的开源版本,可以代码里运行,也可以到 [Releases 页面](https://github.com/iFurySt/open-codex-computer-use/releases) 下载 app 运行。
-
-```bash
-swift run CursorMotion
-```
+Cursor Motion 是一个面向 macOS 的开源光标运动系统,基于 Software.Inc 几位大佬的公开信息实现的开源版本,也可以到 [Releases 页面](https://github.com/iFurySt/open-codex-computer-use/releases) 下载 app 运行。
[![Cursor Motion 自定义演示封面](./docs/generated/readme-assets/cursor-motion-demo-cover.png)](https://youtu.be/KRUq5GUHv1Q)
diff --git a/docs/histories/2026-04/20260423-1417-organize-readme-demo-videos.md b/docs/histories/2026-04/20260423-1417-organize-readme-demo-videos.md
new file mode 100644
index 0000000..8cb456f
--- /dev/null
+++ b/docs/histories/2026-04/20260423-1417-organize-readme-demo-videos.md
@@ -0,0 +1,26 @@
+## [2026-04-23 14:17] | Task: Organize README demo videos
+
+### Execution Context
+* **Agent ID**: `Codex`
+* **Base Model**: `GPT-5`
+* **Runtime**: `codex-cli`
+
+### User Query
+> Add the new Linux demo video to both READMEs and split the three top demo videos into independent second-level sections.
+
+### Changes Overview
+**Scope:** root README documentation
+
+**Key Actions:**
+- **Organized demo area**: Added a dedicated demos section at the top of both English and Chinese READMEs.
+- **Separated demos**: Split the Codex App / Codex CLI, Gemini CLI, and Linux runtime videos into their own third-level subsections with captions.
+- **Added Linux demo**: Embedded the new GitHub user-attachment video URL in both README variants.
+- **Aligned English copy**: Reworked the English README to follow the shorter Chinese README structure and removed redundant runtime detail sections.
+- **Added README badges**: Replaced the top language links in both READMEs with repository-specific language, release, DeepWiki, and LLMAPIS badges.
+
+### Design Intent (Why)
+The top of the README now makes the three usage contexts scan-friendly and avoids mixing unrelated demo videos without labels.
+
+### Files Modified
+- `README.md`
+- `README.zh-CN.md`