Skip to content

👾 Open Computer Use – Open-Source Alternative to Codex Computer Use

Add standalone software cursor baseline renderer

ifuryst345aff721603
diff --git a/docs/histories/2026-04/20260419-2219-refresh-official-overlay-arrow-findings.md b/docs/histories/2026-04/20260419-2219-refresh-official-overlay-arrow-findings.md
new file mode 100644
index 0000000..36a8a0c
--- /dev/null
+++ b/docs/histories/2026-04/20260419-2219-refresh-official-overlay-arrow-findings.md
@@ -0,0 +1,33 @@
+## [2026-04-19 22:19] | Task: 刷新官方 overlay 箭头结论
+
+### 🤖 Execution Context
+* **Agent ID**: `codex`
+* **Base Model**: `gpt-5`
+* **Runtime**: `Codex CLI`
+
+### 📥 User Query
+> 继续从官方 `Codex Computer Use.app` 里往下挖,确认最近截图里那支灰色白边带阴影的箭头到底是不是 bundle 里的某个静态 asset;用户明确指出它不是此前导出的 `SoftwareCursor` 或 `HintArrow`,要求改查 overlay 渲染链。
+
+### 🛠 Changes Overview
+**Scope:** `docs/references/codex-computer-use-reverse-engineering/`、`docs/histories/`
+
+**Key Actions:**
+- **[补做当前版本纠偏]**: 重新核对 bundled `computer-use` `1.0.750` 里的 `HintArrow`、`SoftwareCursor`、`LensSequence` 和当前截图中的灰白箭头,确认三者都不能直接等同于最终 pointer。
+- **[补强渲染侧证据]**: 基于 `SkyComputerUseService` 的字符串,补记 `SoftwareCursorStyle`、`FogCursorViewModel`、`CursorView`、`CAShapeLayer`、`SkyLensView`、`currentFrameIndex` 等命名,把结论从“像某张图片 asset”收敛到“更像代码/图层组合渲染”。
+- **[补做同线程 app-server 实测]**: 通过同一个 `codex app-server` thread 串联 `get_app_state` 和 `click`,确认 `click` 返回截图中看不到该箭头;随后把官方 service 挂住并按 owner pid 全量枚举,重新在当前 `1.0.750` 中抓到 `Software Cursor` 命名窗口。
+- **[落运行时样本]**: 直接对 `Software Cursor` window 做 `screencapture -l <windowid>`,把完整窗口和原始箭头 crop 落到 `docs/references/codex-computer-use-reverse-engineering/assets/extracted-2026-04-19/`。
+- **[补抓完整 2x overlay 边界]**: 继续直接调用 `CGWindowListCreateImage(..., .boundsIgnoreFraming | .bestResolution)`,确认 `Software Cursor` 的完整运行时像素边界是 `252x252`,而不是默认抓图链里看到的 `170x170` 裁剪结果;这和用户截图软件框到的大边框一致。
+- **[补独立测试脚本]**: 新增 `scripts/render-synthesized-software-cursor.swift`,用单文件 AppKit 脚本把 `126x126` overlay 独立渲染到屏幕。默认模式直接读取仓库里保存的 `official-software-cursor-window-252.png` 作为官方基线图,确保尺寸和轮廓先对上;`--procedural` 继续保留纯代码 fallback,单独迭代 fog 和 pointer 近似实现。
+- **[更新逆向文档]**: 修改 `software-cursor-overlay.md`,把结论收敛成“最终灰白箭头不能从 bundle 静态 asset 直接导出,但能从运行时 `Software Cursor` window 直接截出”。
+
+### 🧠 Design Intent (Why)
+这次不是继续扩大猜测范围,而是为了把仓库里的说法修到和证据一致。此前文档已经证明官方存在独立 software cursor / overlay 渲染链,但如果继续把“最近截图里的灰白箭头”直接等同于导出的 `SoftwareCursor` asset,或者继续把 `screencapture -l` 抓到的 `170x170` 小图当成完整边界,都会误导后续逆向方向。把结论更新成“静态 asset 不对,运行时 `Software Cursor` window 还在,而且完整像素边界其实是 `252x252`,只是在默认抓图链里被裁掉了 framing/padding”之后,后续工作就能更明确地围绕 `CursorView`、`SoftwareCursorStyle`、`FogCursorViewModel` 和 host/service 合成边界继续挖。再把一条独立 Swift 脚本单独落仓库,并把默认渲染收敛到“直接展示官方 `252x252` runtime baseline、`--procedural` 再单独试代码近似”,就能先把独立测试入口稳定下来,再继续拆 procedural 复刻。
+
+### 📁 Files Modified
+- `docs/references/codex-computer-use-reverse-engineering/software-cursor-overlay.md`
+- `scripts/render-synthesized-software-cursor.swift`
+- `docs/references/codex-computer-use-reverse-engineering/assets/extracted-2026-04-19/official-software-cursor-window.png`
+- `docs/references/codex-computer-use-reverse-engineering/assets/extracted-2026-04-19/official-software-cursor-window-252.png`
+- `docs/references/codex-computer-use-reverse-engineering/assets/extracted-2026-04-19/official-software-cursor-window-252-center-crop.png`
+- `docs/references/codex-computer-use-reverse-engineering/assets/extracted-2026-04-19/official-software-cursor-pointer-raw-crop.png`
+- `docs/histories/2026-04/20260419-2219-refresh-official-overlay-arrow-findings.md`
diff --git a/docs/references/codex-computer-use-reverse-engineering/assets/extracted-2026-04-19/official-software-cursor-pointer-raw-crop.png b/docs/references/codex-computer-use-reverse-engineering/assets/extracted-2026-04-19/official-software-cursor-pointer-raw-crop.png
new file mode 100644
index 0000000..8dfd50c
Binary files /dev/null and b/docs/references/codex-computer-use-reverse-engineering/assets/extracted-2026-04-19/official-software-cursor-pointer-raw-crop.png differ
diff --git a/docs/references/codex-computer-use-reverse-engineering/assets/extracted-2026-04-19/official-software-cursor-window-252-center-crop.png b/docs/references/codex-computer-use-reverse-engineering/assets/extracted-2026-04-19/official-software-cursor-window-252-center-crop.png
new file mode 100644
index 0000000..5c8c886
Binary files /dev/null and b/docs/references/codex-computer-use-reverse-engineering/assets/extracted-2026-04-19/official-software-cursor-window-252-center-crop.png differ
diff --git a/docs/references/codex-computer-use-reverse-engineering/assets/extracted-2026-04-19/official-software-cursor-window-252.png b/docs/references/codex-computer-use-reverse-engineering/assets/extracted-2026-04-19/official-software-cursor-window-252.png
new file mode 100644
index 0000000..0726442
Binary files /dev/null and b/docs/references/codex-computer-use-reverse-engineering/assets/extracted-2026-04-19/official-software-cursor-window-252.png differ
diff --git a/docs/references/codex-computer-use-reverse-engineering/assets/extracted-2026-04-19/official-software-cursor-window.png b/docs/references/codex-computer-use-reverse-engineering/assets/extracted-2026-04-19/official-software-cursor-window.png
new file mode 100644
index 0000000..71e6622
Binary files /dev/null and b/docs/references/codex-computer-use-reverse-engineering/assets/extracted-2026-04-19/official-software-cursor-window.png differ
diff --git a/docs/references/codex-computer-use-reverse-engineering/software-cursor-overlay.md b/docs/references/codex-computer-use-reverse-engineering/software-cursor-overlay.md
index bf8aae9..7fadcd4 100644
--- a/docs/references/codex-computer-use-reverse-engineering/software-cursor-overlay.md
+++ b/docs/references/codex-computer-use-reverse-engineering/software-cursor-overlay.md
@@ -1,8 +1,8 @@
# Software Cursor Overlay
-这个文档聚焦一个很具体的问题:Codex Computer Use 在真实操作时显化出来的黄色小鼠标,是否是独立的软件光标 overlay,而不是直接抢占用户当前硬件鼠标。
+这个文档聚焦一个很具体的问题:Codex Computer Use 在真实操作时显化出来的光标 overlay,是否是独立的软件光标,而不是直接抢占用户当前硬件鼠标。
-结论先写在前面:从当前已经拿到的 bundle、字符串、资源名和运行时窗口证据看,这个黄色小鼠标基本可以确认是 `SkyComputerUseService` 自己持有的独立 `Software Cursor` 窗口。
+结论先写在前面:截至 2026 年 4 月 19 日对 bundled `computer-use` `1.0.750` 的最新复查,最近截图里那支“灰色主体、白边、带阴影”的箭头仍然不能直接在 bundle 里找到一张一一对应的静态 asset;但它现在已经能从运行时 `Software Cursor` window 里直接截出来,而且和截图软件能框到的“大边框小箭头”现象完全一致。也就是说,这支箭头可以复原,但来源不是 `HintArrow`、原始 `SoftwareCursor` PNG 或 `LensSequence` 里的现成图,而更像 `CursorView` / `SoftwareCursorStyle` / `FogCursorViewModel` / `CAShapeLayer` 组合出来的运行时视觉。
## 已观察事实
@@ -47,6 +47,14 @@ RenditionName: "menubar-cursor.svg"
两份 `SoftwareCursor` 导出后的 PNG 二进制哈希一致,说明至少当前版本里它们引用的是同一张图。
+但 2026 年 4 月 19 日针对 bundled `computer-use` `1.0.750` 的复查也补了一条很重要的纠偏:
+
+- `HintArrow` 确认只是权限引导里的蓝色箭头,不是最近 overlay 截图里的灰白 pointer。
+- 直接导出的 `SoftwareCursor` 原图尺寸是 `200x230`,里面混有额外的右侧亮部/附加图层,不能和最近截图里的最终 pointer body 一一对应。
+- `LensSequence/` 下那组 `48x48` 序列帧是蓝色 lens 动效,也不是灰白箭头本体。
+
+所以“灰白箭头能直接从 `Assets.car` 挖到一张等价 PNG”这件事,目前并没有被证据支持。
+
### 3. `SkyComputerUseService` 字符串里出现了完整的光标窗口和动画线索
对 `SkyComputerUseService` 做 `strings`,能直接看到这些符号和日志文案:
@@ -68,15 +76,41 @@ Detach the computer use cursor from the command palette.
这组证据比“有个 cursor 资源”更强,因为它已经落到了窗口对象、图层对象和运动动画对象这一层。
-### 4. 运行时窗口清单里真的出现了 `Software Cursor`
+继续往下看当前 `1.0.750` 的字符串,还能看到另一组比“单张图片 asset”更关键的渲染侧证据:
+
+```text
+SoftwareCursorStyle
+FogCursorStyle
+FogCursorViewModel
+AgentCursor
+CursorView
+cursorRadius
+fogRadius
+cursorScaleAnchorPoint
+fogScaleAnchorPoint
+CAShapeLayer
+SkyLensView
+isTinted
+imageLoadingTasks
+currentFrameIndex
+animationStartTime
+```
+
+这说明当前实现至少不是“把一张 PNG 直接贴上去”这么简单,而更像:
-在触发一次真实 `computer-use` 点击后,用 `CGWindowListCopyWindowInfo` 枚举窗口,能看到 `Codex Computer Use` 进程名下至少三个相关窗口:
+- 有独立的 cursor / fog style 和 view model;
+- 至少一部分画面通过 `CAShapeLayer` 或相近的 shape/layer 路径生成;
+- 另外一部分视觉特效再叠加 `imageLayer`、`SkyLensView` 和序列帧动画。
+
+### 4. 当前 `1.0.750` 的运行时窗口里仍然有 `Software Cursor`
+
+2026 年 4 月 19 日把官方 service 挂住后,直接按 owner pid 全量枚举 `CGWindowListCopyWindowInfo`,能在当前 bundled `1.0.750` 里看到 `Codex Computer Use` 进程名下至少两个关键窗口:
```text
OwnerName: Codex Computer Use
WindowName: Software Cursor
Layer: 0
-Bounds: X=1949 Y=696 Width=126 Height=126
+Bounds: X=795 Y=353 Width=126 Height=126
IsOnscreen: 1
```
@@ -84,21 +118,26 @@ IsOnscreen: 1
OwnerName: Codex Computer Use
WindowName: Item-0
Layer: 25
-Bounds: X=1804 Y=0 Width=21 Height=24
-```
-
-```text
-OwnerName: Codex Computer Use
-WindowName: ""
-Layer: 25
-Bounds: X=-754 Y=458 Width=41 Height=37
+Bounds: X=-14336 Y=0 Width=38 Height=37
```
其中:
- `Item-0` 明显像菜单栏状态项窗口。
-- 无标题 `41x37` 小窗长期藏在屏幕外,更像一个备用的内部 UI 容器。
- `Software Cursor` 是最关键的直接证据,名字已经把职责写出来了。
+- 这次不是只停留在窗口元数据层面,而是已经能直接对这个 window 做截图:
+ - [official-software-cursor-window.png](assets/extracted-2026-04-19/official-software-cursor-window.png)
+ - [official-software-cursor-window-252.png](assets/extracted-2026-04-19/official-software-cursor-window-252.png)
+ - [official-software-cursor-window-252-center-crop.png](assets/extracted-2026-04-19/official-software-cursor-window-252-center-crop.png)
+ - [official-software-cursor-pointer-raw-crop.png](assets/extracted-2026-04-19/official-software-cursor-pointer-raw-crop.png)
+
+其中两条抓图链路给出的尺寸差异很关键:
+
+- `screencapture -l <windowid>` 和 `CGWindowListCreateImage(..., .bestResolution)` 得到的是 `170x170`,更像“已经裁过 framing/shadow 的可见窗口图”。
+- 直接调用 `CGWindowListCreateImage(..., .boundsIgnoreFraming | .bestResolution)` 得到的是 `252x252`,正好等于 `126x126` 逻辑窗口在 `backingScaleFactor = 2.0` 屏幕上的像素尺寸。
+- 这张 `252x252` 图里,真正带 alpha 的 fog/body 区域只有 `152x152`,中心亮色 pointer 只有大约 `30x29`,所以视觉上才会出现“边框范围很大,但中间箭头很小”的效果。
+
+这说明“最终视觉不能从 bundle asset 直接导出”和“最终视觉能从运行时 overlay 抠出来”这两件事可以同时成立,而且你截图软件框到的大边框,本质上就是 `Software Cursor` window 的完整 `252x252` 像素边界。
### 5. `Software Cursor` 会随着一次次工具动作改变位置
@@ -120,6 +159,22 @@ Bounds: X=1949 Y=796 Width=126 Height=126
这说明它不是静态装饰,而是一个会被操作事件驱动移动的独立窗口。
+### 6. 当前 `1.0.750` 的 app-server 返回图里仍然看不到 cursor,但窗口本身是能抓到的
+
+2026 年 4 月 19 日又补做了一次同线程实测:
+
+- 直接通过 `codex app-server` 在同一个 ephemeral thread 里串联 `get_app_state` 和 `click`,避免 CLI 每次新开 thread 导致的“Finder 未激活”误差。
+- `click` 返回的结果图里看不到可见 cursor,说明这支箭头不是普通 `get_app_state` / `click` 返回图像里 baked 进去的内容。
+- 一开始对 `CGWindowListCopyWindowInfo` 做高频差分时,只看到了 `Item-0` 这类小窗,没有第一时间把 `Software Cursor` 本体筛出来。
+- 但把 service 挂住后改成按 owner pid 全量枚举,`Software Cursor` 窗口又能在当前 `1.0.750` 里直接看到,并且可以被 `screencapture -l <windowid>` 单独截出。
+- 再往下一层直接调旧版 `CGWindowListCreateImage` 符号时,还能看到一个更准确的分层:默认抓图链只给出 `170x170` 的裁剪结果,而 `boundsIgnoreFraming + bestResolution` 会返回完整的 `252x252` runtime overlay。
+
+这次把前面“当前版本没有稳定复现出同名窗口”的说法收紧成了更准确的表述:
+
+- 普通 `click` / `get_app_state` 返回图像里看不到这支箭头;
+- 但当前版本下,`Software Cursor` 作为独立 window 仍然存在,只是如果只做“新窗口差分”而不按 pid 全量扫,很容易漏掉它;
+- 因此“host 侧可能参与了一部分合成”仍然成立,但“service 侧已经没有独立 software cursor window”这个结论当前不成立。
+
## 当前推断
### 1. 黄色小鼠标是软件光标,不是系统硬件光标
@@ -130,24 +185,45 @@ Bounds: X=1949 Y=796 Width=126 Height=126
- 用户看到的黄色小鼠标,是 `SkyComputerUseService` 自己渲染的一层“软件光标”。
- 这层软件光标单独运动、单独做动画,因此视觉上能模拟“鼠标在动”,但不需要真的抢走用户平时看到的系统箭头。
-### 2. 它大概率是一个透明背景的小窗口,而不是注入目标 app 的子视图
+### 2. 它现在仍然是独立 overlay window,但最终视觉不是静态 asset
-原因有三点:
+当前结论成立的原因有三点:
- 字符串里直接出现 `cursorWindow`。
-- 运行时 `CGWindowList` 能枚举到命名明确的 `Software Cursor` 窗口。
+- 当前 `1.0.750` 的运行时 `CGWindowList` 仍然能枚举到命名明确的 `Software Cursor` 窗口。
- 这个窗口 owner 是 `Codex Computer Use`,而不是被控 app。
-所以它更像一个由 service 维护的独立 overlay window,而不是 Finder / System Settings 之类 app 内部被插入了一块绘制层。
+所以更稳妥的说法应该是:
-### 3. 黄色外观大概率来自 app 自带 cursor asset,而不是系统默认箭头重着色
+- 官方现在仍然有独立 software cursor / overlay window;
+- 这层 window 由 service 维护,而不是目标 app 内部插入的绘制层;
+- 但 window 里的最终灰白箭头,不等于 bundle 里某张可直接导出的 PNG。
+
+### 3. 当前灰白箭头能从运行时窗口抠出,但仍更像代码/图层组合,而不是可直接导出的 asset
+
+这一点目前仍带推断,但方向已经比“直接来自 `SoftwareCursor` 图片”更明确:
+
+- `HintArrow`、`SoftwareCursor`、`LensSequence` 都和最近截图里的灰白箭头对不上。
+- 二进制里不仅有 `imageView` / `imageLayer`,还有 `SoftwareCursorStyle`、`FogCursorViewModel`、`CursorView`、`CAShapeLayer`、`cursorRadius`、`fogRadius` 这一整套更偏“运行时渲染”的命名。
+- 官方 `click` 返回截图里看不到这支箭头,但单独截 `Software Cursor` window 又能把它抓出来,说明它确实存在于独立 overlay 合成链里,而不在普通 screenshot 返回链路里。
+- 这支箭头所在的完整 runtime overlay 边界是 `252x252`,其中大量区域只是透明 padding 和半透明 fog,不是单独放在 bundle 里的“现成 cursor 贴图”。
+
+所以当前更像“独立 window + 部分素材 + 代码绘制 + host/service 合成”的运行时视觉,而不是一张可以从 `Assets.car` 直接导出的最终 pointer。
+
+### 4. 仓库里已经补了一条独立的合成验证脚本
+
+为了避免后续每次都得重新挂住官方 service 才能肉眼校准,这轮也额外补了一条单文件 Swift 验证入口:
+
+```bash
+swift scripts/render-synthesized-software-cursor.swift --seconds 12
+```
-这一点目前还是推断,但证据方向比较一致:
+这条脚本现在分成两档:
-- `Assets.car` 里有专门的 `cursor` / `cursor dark` 资源。
-- 二进制里同时出现 `imageView` / `imageLayer` / `imageForResource:` / `imageNamed:`。
+- 默认档:直接读取仓库里已经落下来的 `official-software-cursor-window-252.png`,把这张 `252x252` runtime overlay 基线图独立渲染到一个 `126x126` 透明 window 上,用来先把“独立测试”和尺寸/边界对齐这件事做准。
+- `--procedural` 档:继续保留一版纯代码 fallback,用 radial fog + pointer contour 去近似官方视觉,方便后面单独迭代 pointer path、fog falloff 和按压状态。
-这更像“加载自带图片资源并在窗口里渲染”,而不是直接拿系统箭头做系统级替换。
+也就是说,这条脚本当前优先服务“可独立验证的官方基线”,而不是宣称 procedural 版本已经 1:1 复刻完成。
## 为什么这套体验不会抢用户鼠标
@@ -194,8 +270,9 @@ Bounds: X=1949 Y=796 Width=126 Height=126
## 还没完全确认的点
-- 还没有抓到一次动画进行中的连续窗口轨迹,目前只有离散位置点。
-- 当前 `get_app_state` 返回的截图里通常看不到这个光标,推测截图采样路径和 overlay 合成路径并不完全相同,但这还需要单独验证。
+- 还没有抓到一次动画进行中的连续窗口内容变化,目前手里还是静态帧和离散位置点。
+- 当前 `get_app_state` / `click` 返回的截图里通常看不到这个光标,推测截图采样路径和 overlay 合成路径并不完全相同。
+- 最近截图里那支灰白 pointer 的最终 body / shadow / fog 到底是哪一层在 host 侧合成,仍然需要继续往 `CursorView` 或 `Codex.app` host path 深挖。
## 对开源实现的启发
diff --git a/scripts/render-synthesized-software-cursor.swift b/scripts/render-synthesized-software-cursor.swift
new file mode 100755
index 0000000..a6256ba
--- /dev/null
+++ b/scripts/render-synthesized-software-cursor.swift
@@ -0,0 +1,541 @@
+#!/usr/bin/env swift
+
+import AppKit
+import CoreGraphics
+import Foundation
+
+enum CursorScriptError: LocalizedError {
+ case invalidArguments(String)
+ case bitmapUnavailable
+ case pngEncodingFailed
+
+ var errorDescription: String? {
+ switch self {
+ case let .invalidArguments(message):
+ return message
+ case .bitmapUnavailable:
+ return "Unable to allocate bitmap context for snapshot rendering."
+ case .pngEncodingFailed:
+ return "Unable to encode snapshot as PNG."
+ }
+ }
+}
+
+struct ScriptOptions {
+ var centerX: CGFloat?
+ var centerY: CGFloat?
+ var seconds: TimeInterval = 12
+ var idleDrift = true
+ var pulseLoop = true
+ var useReferenceCapture = true
+ var savePNGURL: URL?
+}
+
+enum CursorMetrics {
+ static let windowSize = CGSize(width: 126, height: 126)
+ static let fogDiameter: CGFloat = 66
+ static let pointerSize = CGSize(width: 21, height: 21)
+ static let pointerOffset = CGPoint(x: 2.6, y: -3.2)
+}
+
+let options = try parseOptions(arguments: Array(CommandLine.arguments.dropFirst()))
+
+final class CursorWindow: NSPanel {
+ override var canBecomeKey: Bool { true }
+ override var canBecomeMain: Bool { true }
+}
+
+final class CursorView: NSView {
+ private let options: ScriptOptions
+ private let startedAt = CACurrentMediaTime()
+ private var timer: Timer?
+ private let referenceImage = loadReferenceCursorWindowImage()
+
+ init(frame frameRect: NSRect, options: ScriptOptions) {
+ self.options = options
+ super.init(frame: frameRect)
+ wantsLayer = true
+ layer?.isOpaque = false
+ }
+
+ @available(*, unavailable)
+ required init?(coder: NSCoder) {
+ fatalError("init(coder:) has not been implemented")
+ }
+
+ override var acceptsFirstResponder: Bool {
+ true
+ }
+
+ override var isOpaque: Bool {
+ false
+ }
+
+ func startAnimating() {
+ timer = Timer.scheduledTimer(withTimeInterval: 1.0 / 60.0, repeats: true) { [weak self] _ in
+ self?.needsDisplay = true
+ }
+ }
+
+ func stopAnimating() {
+ timer?.invalidate()
+ timer = nil
+ }
+
+ override func keyDown(with event: NSEvent) {
+ switch event.charactersIgnoringModifiers?.lowercased() {
+ case "q":
+ NSApplication.shared.terminate(nil)
+ default:
+ if event.keyCode == 53 {
+ NSApplication.shared.terminate(nil)
+ } else {
+ super.keyDown(with: event)
+ }
+ }
+ }
+
+ override func draw(_ dirtyRect: NSRect) {
+ super.draw(dirtyRect)
+
+ NSColor.clear.setFill()
+ dirtyRect.fill()
+
+ guard let context = NSGraphicsContext.current?.cgContext else {
+ return
+ }
+
+ let elapsed = CACurrentMediaTime() - startedAt
+
+ if options.useReferenceCapture, let referenceImage {
+ referenceImage.draw(in: bounds, from: .zero, operation: .sourceOver, fraction: 1)
+ return
+ }
+
+ let pulse = pulseProgress(at: elapsed)
+ let drift = idleOffset(at: elapsed)
+ let fogCenter = CGPoint(
+ x: bounds.midX + (drift.dx * 0.28),
+ y: bounds.midY + (drift.dy * 0.24) - (pulse * 0.8)
+ )
+ let pointerCenter = CGPoint(
+ x: bounds.midX + CursorMetrics.pointerOffset.x + drift.dx,
+ y: bounds.midY + CursorMetrics.pointerOffset.y + drift.dy + (pulse * 0.35)
+ )
+
+ drawFog(in: context, center: fogCenter, pulse: pulse)
+ drawPointer(in: context, center: pointerCenter, elapsed: elapsed, pulse: pulse)
+ }
+
+ private func pulseProgress(at elapsed: TimeInterval) -> CGFloat {
+ guard options.pulseLoop else {
+ return 0
+ }
+
+ let cycleDuration = 1.65
+ let activeDuration = 0.24
+ let cycleTime = elapsed.truncatingRemainder(dividingBy: cycleDuration)
+ guard cycleTime < activeDuration else {
+ return 0
+ }
+
+ let normalized = cycleTime / activeDuration
+ return CGFloat(sin(normalized * .pi))
+ }
+
+ private func idleOffset(at elapsed: TimeInterval) -> CGVector {
+ guard options.idleDrift else {
+ return .zero
+ }
+
+ return CGVector(
+ dx: sin(elapsed * 1.35) * 1.6,
+ dy: cos(elapsed * 0.92) * 1.15
+ )
+ }
+
+ private func drawFog(in context: CGContext, center: CGPoint, pulse: CGFloat) {
+ let radius = (CursorMetrics.fogDiameter / 2) + (pulse * 1.2)
+ let glowRadius = radius * (0.30 + (pulse * 0.025))
+ let colors = [
+ NSColor(calibratedRed: 0.38, green: 0.36, blue: 0.35, alpha: 0.40 + (pulse * 0.02)).cgColor,
+ NSColor(calibratedRed: 0.43, green: 0.41, blue: 0.40, alpha: 0.28 + (pulse * 0.015)).cgColor,
+ NSColor(calibratedRed: 0.46, green: 0.44, blue: 0.43, alpha: 0.11).cgColor,
+ NSColor(calibratedWhite: 0.60, alpha: 0.0).cgColor,
+ ] as CFArray
+ let locations: [CGFloat] = [0, 0.50, 0.82, 1]
+ let colorSpace = CGColorSpaceCreateDeviceRGB()
+
+ guard let gradient = CGGradient(colorsSpace: colorSpace, colors: colors, locations: locations) else {
+ return
+ }
+
+ context.saveGState()
+ context.drawRadialGradient(
+ gradient,
+ startCenter: center,
+ startRadius: 0,
+ endCenter: center,
+ endRadius: radius,
+ options: [.drawsBeforeStartLocation, .drawsAfterEndLocation]
+ )
+ context.restoreGState()
+
+ let coreColors = [
+ NSColor(calibratedRed: 0.41, green: 0.39, blue: 0.38, alpha: 0.020 + (pulse * 0.006)).cgColor,
+ NSColor(calibratedRed: 0.44, green: 0.41, blue: 0.40, alpha: 0.008).cgColor,
+ NSColor(calibratedWhite: 0.80, alpha: 0.0).cgColor,
+ ] as CFArray
+ let coreLocations: [CGFloat] = [0, 0.62, 1]
+ guard let coreGradient = CGGradient(colorsSpace: colorSpace, colors: coreColors, locations: coreLocations) else {
+ return
+ }
+
+ context.saveGState()
+ context.drawRadialGradient(
+ coreGradient,
+ startCenter: center,
+ startRadius: 0,
+ endCenter: center,
+ endRadius: glowRadius,
+ options: [.drawsBeforeStartLocation, .drawsAfterEndLocation]
+ )
+ context.restoreGState()
+
+ let ringRect = CGRect(x: center.x - radius, y: center.y - radius, width: radius * 2, height: radius * 2)
+ let ring = NSBezierPath(ovalIn: ringRect)
+ NSColor.white.withAlphaComponent(0.003 + (pulse * 0.002)).setStroke()
+ ring.lineWidth = 0.6
+ ring.stroke()
+ }
+
+ private func drawPointer(in context: CGContext, center: CGPoint, elapsed: TimeInterval, pulse: CGFloat) {
+ let wobble = options.idleDrift ? sin(elapsed * 1.45) * 0.03 : 0
+ let pointerRect = CGRect(
+ x: center.x - (CursorMetrics.pointerSize.width / 2),
+ y: center.y - (CursorMetrics.pointerSize.height / 2),
+ width: CursorMetrics.pointerSize.width,
+ height: CursorMetrics.pointerSize.height
+ )
+ let outerPath = pointerPath(in: pointerRect)
+
+ context.saveGState()
+ context.translateBy(x: center.x, y: center.y)
+ context.rotate(by: wobble + (pulse * 0.03))
+ context.scaleBy(x: 1 - (pulse * 0.04), y: 1 + (pulse * 0.02))
+ context.translateBy(x: -center.x, y: -center.y)
+
+ NSGraphicsContext.saveGraphicsState()
+ let shadow = NSShadow()
+ shadow.shadowBlurRadius = 3.2 + (pulse * 1.4)
+ shadow.shadowOffset = CGSize(width: 0, height: -0.35)
+ shadow.shadowColor = NSColor.black.withAlphaComponent(0.11)
+ shadow.set()
+ NSColor.black.withAlphaComponent(0.05).setFill()
+ outerPath.fill()
+ NSGraphicsContext.restoreGraphicsState()
+
+ NSColor(calibratedRed: 0.38, green: 0.36, blue: 0.35, alpha: 0.98).setFill()
+ outerPath.fill()
+
+ NSColor(calibratedWhite: 0.90, alpha: 0.92).setStroke()
+ outerPath.lineWidth = 1.55
+ outerPath.lineJoinStyle = .round
+ outerPath.lineCapStyle = .round
+ outerPath.stroke()
+
+ context.restoreGState()
+ }
+
+ private func pointerPath(in rect: CGRect) -> NSBezierPath {
+ let contourRows: [(y: CGFloat, minX: CGFloat, maxX: CGFloat)] = [
+ (39, 17, 21),
+ (38, 16, 22),
+ (37, 15, 22),
+ (36, 15, 23),
+ (35, 15, 24),
+ (34, 15, 24),
+ (33, 14, 25),
+ (32, 14, 25),
+ (31, 14, 26),
+ (30, 14, 27),
+ (29, 13, 29),
+ (28, 13, 31),
+ (27, 13, 34),
+ (26, 13, 36),
+ (25, 13, 37),
+ (24, 12, 37),
+ (23, 12, 37),
+ (22, 12, 37),
+ (21, 12, 37),
+ (20, 12, 36),
+ (19, 11, 36),
+ (18, 11, 34),
+ (17, 11, 32),
+ (16, 11, 30),
+ (15, 10, 27),
+ (14, 10, 25),
+ (13, 10, 23),
+ (12, 11, 21),
+ (11, 11, 19),
+ (10, 13, 16),
+ ]
+ let sourceMinX: CGFloat = 10
+ let sourceMaxX: CGFloat = 38
+ let sourceMinY: CGFloat = 10
+ let sourceMaxY: CGFloat = 39
+
+ func mappedPoint(x: CGFloat, y: CGFloat) -> CGPoint {
+ CGPoint(
+ x: rect.minX + ((x - sourceMinX) / (sourceMaxX - sourceMinX) * rect.width),
+ y: rect.minY + ((y - sourceMinY) / (sourceMaxY - sourceMinY) * rect.height)
+ )
+ }
+
+ let leftBoundary = contourRows.map { mappedPoint(x: $0.minX, y: $0.y) }
+ let rightBoundary = contourRows.reversed().map { mappedPoint(x: $0.maxX, y: $0.y) }
+
+ let path = NSBezierPath()
+ path.move(to: leftBoundary[0])
+ leftBoundary.dropFirst().forEach { path.line(to: $0) }
+ rightBoundary.forEach { path.line(to: $0) }
+ path.close()
+ path.lineJoinStyle = .round
+ return path
+ }
+}
+
+final class AppDelegate: NSObject, NSApplicationDelegate {
+ private let options: ScriptOptions
+ private var window: CursorWindow?
+ private var cursorView: CursorView?
+
+ init(options: ScriptOptions) {
+ self.options = options
+ }
+
+ func applicationDidFinishLaunching(_ notification: Notification) {
+ let frame = resolvedWindowFrame()
+ let window = CursorWindow(
+ contentRect: frame,
+ styleMask: [.borderless],
+ backing: .buffered,
+ defer: false
+ )
+ window.backgroundColor = .clear
+ window.isOpaque = false
+ window.hasShadow = false
+ window.level = .statusBar
+ window.ignoresMouseEvents = false
+ window.isMovableByWindowBackground = true
+ window.collectionBehavior = [.canJoinAllSpaces, .fullScreenAuxiliary, .stationary]
+ window.title = "Synthesized Software Cursor"
+
+ let cursorView = CursorView(
+ frame: CGRect(origin: .zero, size: CursorMetrics.windowSize),
+ options: options
+ )
+ window.contentView = cursorView
+ window.makeKeyAndOrderFront(nil)
+ window.makeFirstResponder(cursorView)
+ cursorView.startAnimating()
+
+ self.window = window
+ self.cursorView = cursorView
+
+ NSApp.activate(ignoringOtherApps: true)
+
+ if let savePNGURL = options.savePNGURL {
+ Timer.scheduledTimer(withTimeInterval: 0.45, repeats: false) { [weak self] _ in
+ guard let self, let cursorView = self.cursorView else {
+ return
+ }
+
+ do {
+ try saveSnapshot(of: cursorView, to: savePNGURL)
+ print("Saved snapshot to \(savePNGURL.path)")
+ } catch {
+ fputs("Snapshot failed: \(error.localizedDescription)\n", stderr)
+ }
+ }
+ }
+
+ if options.seconds > 0 {
+ Timer.scheduledTimer(withTimeInterval: options.seconds, repeats: false) { _ in
+ NSApplication.shared.terminate(nil)
+ }
+ }
+
+ print(startupMessage(frame: frame))
+ }
+
+ func applicationWillTerminate(_ notification: Notification) {
+ cursorView?.stopAnimating()
+ }
+
+ private func resolvedWindowFrame() -> CGRect {
+ let screen = NSScreen.main ?? NSScreen.screens.first
+ let visibleFrame = screen?.visibleFrame ?? CGRect(x: 200, y: 200, width: 900, height: 700)
+ let centerX = options.centerX ?? visibleFrame.midX
+ let centerY = options.centerY ?? visibleFrame.midY
+
+ return CGRect(
+ x: centerX - (CursorMetrics.windowSize.width / 2),
+ y: centerY - (CursorMetrics.windowSize.height / 2),
+ width: CursorMetrics.windowSize.width,
+ height: CursorMetrics.windowSize.height
+ )
+ }
+
+ private func startupMessage(frame: CGRect) -> String {
+ let durationDescription = options.seconds > 0
+ ? String(format: "%.1f seconds", options.seconds)
+ : "until you press q or Esc"
+ let snapshot = options.savePNGURL?.path ?? "disabled"
+
+ return """
+ Synthesized software cursor overlay running.
+ Window frame: \(NSStringFromRect(frame))
+ Auto-exit: \(durationDescription)
+ Snapshot: \(snapshot)
+ """
+ }
+}
+
+let app = NSApplication.shared
+app.setActivationPolicy(.accessory)
+
+let delegate = AppDelegate(options: options)
+app.delegate = delegate
+app.run()
+
+func parseOptions(arguments: [String]) throws -> ScriptOptions {
+ var options = ScriptOptions()
+ var index = 0
+
+ while index < arguments.count {
+ let argument = arguments[index]
+ switch argument {
+ case "--help", "-h":
+ printUsage()
+ Foundation.exit(EXIT_SUCCESS)
+ case "--x":
+ index += 1
+ options.centerX = try parseCGFloat(arguments, index: index, flag: argument)
+ case "--y":
+ index += 1
+ options.centerY = try parseCGFloat(arguments, index: index, flag: argument)
+ case "--seconds":
+ index += 1
+ let rawValue = try parseString(arguments, index: index, flag: argument)
+ guard let seconds = TimeInterval(rawValue), seconds >= 0 else {
+ throw CursorScriptError.invalidArguments("Invalid value for --seconds: \(rawValue)")
+ }
+ options.seconds = seconds
+ case "--save-png":
+ index += 1
+ let path = try parseString(arguments, index: index, flag: argument)
+ options.savePNGURL = URL(fileURLWithPath: path)
+ case "--procedural":
+ options.useReferenceCapture = false
+ case "--no-idle":
+ options.idleDrift = false
+ case "--no-pulse":
+ options.pulseLoop = false
+ default:
+ throw CursorScriptError.invalidArguments("Unknown argument: \(argument)")
+ }
+
+ index += 1
+ }
+
+ return options
+}
+
+func parseString(_ arguments: [String], index: Int, flag: String) throws -> String {
+ guard arguments.indices.contains(index) else {
+ throw CursorScriptError.invalidArguments("Missing value for \(flag)")
+ }
+
+ return arguments[index]
+}
+
+func parseCGFloat(_ arguments: [String], index: Int, flag: String) throws -> CGFloat {
+ let rawValue = try parseString(arguments, index: index, flag: flag)
+ guard let value = Double(rawValue) else {
+ throw CursorScriptError.invalidArguments("Invalid value for \(flag): \(rawValue)")
+ }
+
+ return CGFloat(value)
+}
+
+func saveSnapshot(of view: NSView, to url: URL) throws {
+ let scale = NSScreen.main?.backingScaleFactor ?? 2
+ let pixelWidth = Int(CursorMetrics.windowSize.width * scale)
+ let pixelHeight = Int(CursorMetrics.windowSize.height * scale)
+
+ guard let bitmap = NSBitmapImageRep(
+ bitmapDataPlanes: nil,
+ pixelsWide: pixelWidth,
+ pixelsHigh: pixelHeight,
+ bitsPerSample: 8,
+ samplesPerPixel: 4,
+ hasAlpha: true,
+ isPlanar: false,
+ colorSpaceName: .deviceRGB,
+ bytesPerRow: 0,
+ bitsPerPixel: 0
+ ) else {
+ throw CursorScriptError.bitmapUnavailable
+ }
+
+ bitmap.size = NSSize(width: pixelWidth, height: pixelHeight)
+ guard let context = NSGraphicsContext(bitmapImageRep: bitmap) else {
+ throw CursorScriptError.bitmapUnavailable
+ }
+
+ NSGraphicsContext.saveGraphicsState()
+ NSGraphicsContext.current = context
+ context.cgContext.scaleBy(x: scale, y: scale)
+ view.displayIgnoringOpacity(view.bounds, in: context)
+ context.flushGraphics()
+ NSGraphicsContext.restoreGraphicsState()
+
+ guard let pngData = bitmap.representation(using: .png, properties: [:]) else {
+ throw CursorScriptError.pngEncodingFailed
+ }
+
+ try FileManager.default.createDirectory(
+ at: url.deletingLastPathComponent(),
+ withIntermediateDirectories: true
+ )
+ try pngData.write(to: url)
+}
+
+func printUsage() {
+ let usage = """
+ Usage:
+ swift scripts/render-synthesized-software-cursor.swift [options]
+
+ Options:
+ --x <value> Overlay center x position in screen coordinates
+ --y <value> Overlay center y position in screen coordinates
+ --seconds <value> Auto-exit after N seconds; use 0 to keep it open
+ --save-png <path> Save a 2x transparent PNG snapshot of the synthesized overlay
+ --procedural Render the current code-generated fallback instead of the captured official baseline
+ --no-idle Disable idle drift
+ --no-pulse Disable periodic click pulse
+ --help Show this message
+ """
+
+ print(usage)
+}
+
+func loadReferenceCursorWindowImage() -> NSImage? {
+ let scriptURL = URL(fileURLWithPath: #filePath).standardizedFileURL
+ let repoRoot = scriptURL.deletingLastPathComponent().deletingLastPathComponent()
+ let referenceURL = repoRoot
+ .appendingPathComponent("docs/references/codex-computer-use-reverse-engineering/assets/extracted-2026-04-19/official-software-cursor-window-252.png")
+
+ return NSImage(contentsOf: referenceURL)
+}