Skip to content

👾 Open Computer Use – Open-Source Alternative to Codex Computer Use

Restore CursorMotion glyph boundary

Leocdc450c9ba14
diff --git a/Package.swift b/Package.swift
index 6ff8a7e..32ecdf1 100644
--- a/Package.swift
+++ b/Package.swift
@@ -34,13 +34,8 @@ let package = Package(
),
],
targets: [
- .target(
- name: "SoftwareCursorGlyphKit",
- path: "packages/SoftwareCursorGlyphKit/Sources/SoftwareCursorGlyphKit"
- ),
.target(
name: "OpenComputerUseKit",
- dependencies: ["SoftwareCursorGlyphKit"],
path: "packages/OpenComputerUseKit/Sources/OpenComputerUseKit"
),
.executableTarget(
@@ -60,7 +55,6 @@ let package = Package(
),
.executableTarget(
name: "CursorMotion",
- dependencies: ["SoftwareCursorGlyphKit"],
path: "experiments/CursorMotion/Sources/CursorMotion"
),
.target(
@@ -74,7 +68,7 @@ let package = Package(
),
.testTarget(
name: "OpenComputerUseKitTests",
- dependencies: ["OpenComputerUseKit", "SoftwareCursorGlyphKit"],
+ dependencies: ["OpenComputerUseKit"],
path: "packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests"
),
.testTarget(
diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md
index 475a4dc..5061b15 100644
--- a/docs/ARCHITECTURE.md
+++ b/docs/ARCHITECTURE.md
@@ -58,7 +58,7 @@
- 对真实 app 的 `get_app_state` / action tool 入口,当前新增了一层官方风格的高风险 bundle denylist:bundle-id 直传时直接返回 safety denial;名称匹配时默认不解析到这些 app,尽量贴近官方对终端、密码管理器、Chrome 与少量系统敏感组件的防护行为。
- 普通 app 的 element frame 当前按“窗口左上角为原点”的 window-relative 坐标输出,便于后续把 `element_index` 和截图坐标统一到同一套参考系。
- `click` / `set_value` 在执行真实动作前后,会额外驱动一层透明 `SoftwareCursorOverlay` window:两者的移动阶段现在共用一条 heading-driven 的官方风格 motion 内核,显式把“当前 cursor 朝向”和“最终 resting pose”一起喂给选路器,优先生成需要时先掉头、再沿车头方向推进的 C 形/单侧大弧轨迹;真正显示出来的 cursor 不再直接等于 path sample,而是经过一层独立的 visual dynamics 状态,把 visible tip、velocity、angle 和 fog/offset 持续推进。`click` 结尾会衔接 click pulse、idle sway 和自动淡出,`set_value` 则只做 settle / idle / hide,不给 pulse。
-- overlay 的 visual style 不再自己从官方 app bundle 裁 `SoftwareCursor` 小图;runtime 现在直接复用一个中立 `SoftwareCursorGlyphKit` target 里的程序化 pointer/fog glyph 校准,命中点 anchor 固定在 `126x126` 画布里的同一组 tip-offset 上,避免实验室和主 MCP runtime 各自维护一套 cursor 轮廓。
+- overlay 的 visual style 不再自己从官方 app bundle 裁 `SoftwareCursor` 小图;主 MCP runtime 现在在 `OpenComputerUseKit` 内部复刻 `CursorMotion` procedural fallback 的 pointer/fog glyph 校准,命中点 anchor 固定在 `126x126` 画布里的同一组 tip-offset 上,但不让实验线依赖 runtime 代码。
- overlay 的层级不再固定 `.floating`;现在会跟随 snapshot 命中的目标 window id / layer,把自己排到该目标 window 之上,而不是粗暴压到所有前台 app 最上层。
- overlay 的曲线路径不再只按固定 Bezier 模板生成;当前主线采用 reverse-engineering 约束下的 heading-driven candidate 族,候选只保留 `direct` / `turn` / `brake` / `orbit` 这些能稳定产出单侧主弧的 family,并继续保留 target-window 命中策略作为同类候选间的 tie-break。原始 binary lift 恢复出来的 `20` 条路径和 score 仍然保留在独立的 `StandaloneCursor` viewer / Python 重建脚本里,用于对照分析,不再直接作为 runtime 默认 chooser。
- overlay 的 progress 曲线也不再是固定 `easeInOut`;主线现在复用官方 `response=1.4`、`dampingFraction=0.9`、`dt=1/240` 的 spring/`VelocityVerlet` 形状,但 wall-clock duration 仍做本地校准,因为官方 transaction-level 时长映射还没有完全恢复。
@@ -86,7 +86,7 @@
- 当前它刻意不引入 speculative 的 wall-clock duration 映射,也不复用 `CursorMotion` 里更偏视觉手感试验的 pose dynamics。
- `CursorMotion` 是一个单独的 SwiftUI demo target,可通过 `swift run CursorMotion` 本地启动。
- 这条线优先验证 motion model 本身:当前主线是 heading-driven 的 turn / brake / orbit / direct candidate 族、spring progress、独立 visual dynamics 和 debug UI;moving 阶段真正画出来的箭头角度会持续跟随 visual dynamics 的主 heading,接近停住后再平滑回到默认 resting pose,并在 idle 阶段保留原地小摆角。
-- lab 的 cursor 视觉现在和主 runtime 一样,统一走 `SoftwareCursorGlyphKit` 里的程序化 pointer/fog glyph;独立脚本 `scripts/render-synthesized-software-cursor.swift` 仍保留 reference-vs-procedural 的逆向对照用途,但 demo/runtime 本身已经不再依赖 PNG baseline 图。settle 态也改成中心固定的小幅摆角,而不是继续沿 XY 轻微漂移。
+- lab 的 cursor 视觉继续以 `scripts/render-synthesized-software-cursor.swift` 为参考:优先使用仓库里保存的官方 `252x252` runtime baseline 图,缺失时再退回脚本里的 procedural pointer/fog 近似;settle 态也改成中心固定的小幅摆角,而不是继续沿 XY 轻微漂移。
- 当前它不接真实 tool call,也不回写主 `SoftwareCursorOverlay`,目的是把实验噪音与产品行为边界隔离开。
## 关键边界
diff --git a/docs/exec-plans/active/20260418-standalone-cursor-lab.md b/docs/exec-plans/active/20260418-standalone-cursor-lab.md
index 8112933..2fa56c3 100644
--- a/docs/exec-plans/active/20260418-standalone-cursor-lab.md
+++ b/docs/exec-plans/active/20260418-standalone-cursor-lab.md
@@ -65,7 +65,7 @@
- 2026-04-19:已把 lab 从 speculative slider 驱动的曲线/收尾模型,重构为 recovered 的 `20` candidate path + 官方风格 spring progress + 独立 visual dynamics;收尾不再靠 endpoint 锁住后原地翻角。
- 2026-04-19:在对照官方视频后发现 guide/arc 相关常量不能直接按屏幕坐标向量使用;当前已改成先投到 start→end 的局部基底,再生成候选路径,默认样例和反向斜移都不再出现起点附近打结式的扭曲回环。
- 2026-04-19:继续对照官方视频后,确认 lab 主线不能直接拿 raw reverse-engineered `20` candidates 当 chooser;当前已改成 heading-driven 选路,把当前可见朝向和最终 resting pose 一起参与选路,主路径重新收敛到“需要掉头时走单侧 C 形,不需要掉头时近直线”的分布。
-- 2026-04-20:在对照 `scripts/render-synthesized-software-cursor.swift` 与用户截图后,确认共享 glyph renderer 的亮白 asset 风格并不对;当前已把 lab 改成优先显示仓库里的官方 `252x252` runtime baseline 图,fallback 才走脚本同款 procedural pointer/fog,同时把 idle 从 XY 漂移收紧为中心固定的小幅摆角。
+- 2026-04-20:在对照 `scripts/render-synthesized-software-cursor.swift` 与用户截图后,确认上一版亮白 asset 风格并不对;当前已把 lab 改成优先显示仓库里的官方 `252x252` runtime baseline 图,fallback 才走脚本同款 procedural pointer/fog,同时把 idle 从 XY 漂移收紧为中心固定的小幅摆角。
- 2026-04-20:曾按用户反馈把“内部 heading”与“可见箭头角度”分离,短暂把 moving 阶段的可见箭头收紧成轻微 lean;这层假设随后已在对照官方抽帧后撤回。
- 2026-04-20:按用户反馈把左上角 5 个 slider 恢复回来,并重新接到 heading-driven 路径几何与 progress spring;当前 slider 明确只作为本地调参入口,不宣称是已完全确认的官方字段映射。
- 2026-04-20:继续按用户反馈把左上角控件区裁成纯 slider,并把 slider label / panel accent 收回当前中性灰紫主色系;不再保留 `REPLAY` / `RESET` 按钮和额外 metrics 文案,避免信息噪音和低对比白字。
@@ -87,7 +87,6 @@
- 2026-04-20:已补一条 tag 驱动的 `Cursor Motion` 分发链路;本地可以通过 `scripts/build-cursor-motion-dmg.sh` 构建 DMG,GitHub Actions 在推送 release tag 后会自动生成 `CursorMotion-<version>.dmg` 并上传到 GitHub Releases。
- 2026-04-20:继续修正“打包版和 `swift run CursorMotion` 观感不一致”的问题;当前已确认 release app 之前没把官方 `252x252` baseline cursor PNG 带进 bundle,导致退回 procedural glyph。现在 `.app` 会优先从 `Bundle.main` 读官方 cursor 图,DMG 打包脚本也会把这张图复制进 `Contents/Resources`,并显式打开 `NSHighResolutionCapable`。
- 2026-04-20:继续修正 packaged `Cursor Motion` 的 app icon 资产链;当前已经收口到仓库内 checked-in 的 `1024x1024` master PNG,并在 DMG 打包阶段通过 `sips + iconutil` 生成 `CursorMotion.icns`,避免继续把 Dock 光学尺寸调校耦合在临时 icon render 脚本里。
-- 2026-04-20:继续收口 glyph 边界;`CursorMotion` 已不再优先走 `official-software-cursor-window-252.png`,而是固定使用代码绘制的程序化 pointer/fog glyph,并把这份 renderer 抽到中立 `SoftwareCursorGlyphKit` target,避免实验线重新直接依赖 `OpenComputerUseKit`。
## 决策记录
@@ -106,4 +105,3 @@
- 2026-04-20:对 `Cursor Motion` 的对外交付,当前采用“源码运行 + GitHub Releases 分发 ad-hoc signed DMG”的策略;先把 tag 驱动的可复现封装链路稳定下来,不在这一轮提前引入 notarization 和 Developer ID 签名复杂度。
- 2026-04-20:对 packaged `Cursor Motion` 的 glyph 资源,当前采用“bundle 内官方 baseline 图优先,仓库 reference 路径兜底”的策略;这样 release app 不再因为缺资源而静默退回低保真的 procedural glyph。
- 2026-04-20:对 packaged `Cursor Motion` 的 app icon,当前采用“先直接复用 `Open Computer Use` 的现有 `.icns` 渲染脚本”的策略;先解决 Finder / DMG 里的无图标问题,后续如果需要再单独设计专属 icon。
-- 2026-04-20:对 `CursorMotion` 与主 runtime 的 glyph 共享,当前采用“抽独立 `SoftwareCursorGlyphKit` target、两边都依赖这份程序化 renderer”的策略;这样既能统一 cursor 轮廓,又不会把实验线重新绑回 `OpenComputerUseKit`。
diff --git a/docs/histories/2026-04/20260420-2033-integrate-set-value-visual-cursor.md b/docs/histories/2026-04/20260420-2033-integrate-set-value-visual-cursor.md
index a238388..26c2a5c 100644
--- a/docs/histories/2026-04/20260420-2033-integrate-set-value-visual-cursor.md
+++ b/docs/histories/2026-04/20260420-2033-integrate-set-value-visual-cursor.md
@@ -76,3 +76,22 @@
- `docs/ARCHITECTURE.md`
- `docs/exec-plans/active/20260418-standalone-cursor-lab.md`
- `docs/histories/2026-04/20260420-2033-integrate-set-value-visual-cursor.md`
+
+### 🔁 Follow-up (2026-04-20, restore `CursorMotion` and scope glyph reuse to runtime)
+
+- **[恢复 CursorMotion 边界]**: 用户明确反馈 `CursorMotion` 本身已经正常,不应被主 runtime 集成改动牵连;当前已撤回 `CursorMotion -> SoftwareCursorGlyphKit` 的 package 依赖,恢复它原来的独立 target 形态。
+- **[恢复 PNG-first lab glyph]**: `SynthesizedCursorGlyphView` 重新回到“优先读取官方 `official-software-cursor-window-252.png`,缺失时再使用本地 procedural fallback”的实现,DMG 打包脚本也继续把这张参考图复制进 bundle。
+- **[runtime 内部复刻即可]**: `click` / `set_value` 需要的程序化 glyph renderer 只保留在 `OpenComputerUseKit` 内部,作为主 MCP runtime overlay 的实现细节;不再新增中立共享 target,也不要求 `CursorMotion` 复用 runtime 代码。
+- **[同步文档口径]**: README、架构说明和 active execution plan 都改回“CursorMotion 独立、PNG-first;OpenComputerUseKit 自己参考 CursorMotion fallback 绘制”的边界,避免继续误导后续改动。
+
+**Follow-up Files:**
+- `Package.swift`
+- `packages/OpenComputerUseKit/Sources/OpenComputerUseKit/SoftwareCursorGlyphRenderer.swift`
+- `packages/OpenComputerUseKit/Sources/OpenComputerUseKit/SoftwareCursorOverlay.swift`
+- `experiments/CursorMotion/Sources/CursorMotion/SynthesizedCursorGlyphView.swift`
+- `packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/OpenComputerUseKitTests.swift`
+- `scripts/build-cursor-motion-dmg.sh`
+- `experiments/CursorMotion/README.md`
+- `docs/ARCHITECTURE.md`
+- `docs/exec-plans/active/20260418-standalone-cursor-lab.md`
+- `docs/histories/2026-04/20260420-2033-integrate-set-value-visual-cursor.md`
diff --git a/experiments/CursorMotion/README.md b/experiments/CursorMotion/README.md
index 10c371b..9cc311a 100644
--- a/experiments/CursorMotion/README.md
+++ b/experiments/CursorMotion/README.md
@@ -9,7 +9,6 @@
- 主线 `packages/OpenComputerUseKit/.../SoftwareCursorOverlay.swift` 已经承担产品行为,不适合继续塞大量实验代码。
- 用户提供的视频和官方字符串都说明 cursor motion 有独立参数模型,适合先做一个 lab。
- 这块后续可能单独开源,先在目录边界上收口更干净。
-- 当前与主 runtime 共享的只有一个中立 `SoftwareCursorGlyphKit` target,用来承载程序化 cursor glyph;`CursorMotion` 不再直接依赖 `OpenComputerUseKit`。
## 当前模块边界
@@ -20,7 +19,7 @@
- `Sources/CursorLabRootView.swift`
- 本地 demo UI、slider 调参面板、候选路径 overlay 与点击交互
- `Sources/SynthesizedCursorGlyphView.swift`
- - 复用共享 `SoftwareCursorGlyphKit` 的程序化 pointer/fog glyph renderer
+ - 参考 `scripts/render-synthesized-software-cursor.swift` 的 baseline/procedural cursor renderer
## 当前参考
@@ -56,7 +55,7 @@ swift run CursorMotion
- 可见 cursor 不再直接贴在 path sample 上,而是经过独立 visual dynamics 状态,再输出 `rotation + cursorBodyOffset + fogOffset + fogScale`。
- 候选路径现在显式约束“先顺车头方向掉头,再沿主轴推进,再按 resting pose 收尾”;因此大多数跨向移动会呈现单侧 C 形,需要直接切入时才会退化为近直线,而不会再出现两侧乱甩的 S 形扭曲。
- 箭头的可见角度现在重新对齐官方抽帧与逆向证据:moving 阶段持续跟随当前 move heading,接近停住后再平滑回到默认 resting pose,并继续做原地小摆角。
-- cursor glyph 现在固定走代码绘制的程序化 pointer/fog renderer,不再依赖 `252x252` PNG baseline 图或其它图片资产。
+- cursor glyph 不再走之前那套亮白 asset;当前优先直接显示仓库里的官方 `252x252` runtime baseline 图,缺失时才退回脚本同款 procedural pointer/fog。
- settle 态不再做 XY 漂移;现在改成和参考脚本一致的中心固定小摆角,让“停住以后原地轻微转动”的观感先对齐。
后续实现应优先保持:
diff --git a/experiments/CursorMotion/Sources/CursorMotion/SynthesizedCursorGlyphView.swift b/experiments/CursorMotion/Sources/CursorMotion/SynthesizedCursorGlyphView.swift
index ddcaa2b..f2f85a1 100644
--- a/experiments/CursorMotion/Sources/CursorMotion/SynthesizedCursorGlyphView.swift
+++ b/experiments/CursorMotion/Sources/CursorMotion/SynthesizedCursorGlyphView.swift
@@ -1,7 +1,11 @@
import AppKit
import CoreGraphics
import Foundation
-import SoftwareCursorGlyphKit
+
+private enum SynthesizedCursorColors {
+ static let pointerFill = NSColor(calibratedRed: 0.38, green: 0.36, blue: 0.35, alpha: 0.98)
+ static let pointerStroke = NSColor(calibratedWhite: 0.90, alpha: 0.92)
+}
@MainActor
final class SynthesizedCursorGlyphView: NSView {
@@ -29,6 +33,8 @@ final class SynthesizedCursorGlyphView: NSView {
didSet { needsDisplay = true }
}
+ private let referenceImage = loadReferenceCursorWindowImage()
+
override init(frame frameRect: NSRect) {
super.init(frame: frameRect)
wantsLayer = true
@@ -54,22 +60,170 @@ final class SynthesizedCursorGlyphView: NSView {
return
}
+ if let referenceImage {
+ drawReferenceImage(referenceImage, in: context)
+ return
+ }
+
drawProceduralGlyph(in: context)
}
+ private func drawReferenceImage(_ image: NSImage, in context: CGContext) {
+ let drawingBodyOffset = drawingVector(from: cursorBodyOffset)
+ let motionCompression = min(hypot(cursorBodyOffset.dx, cursorBodyOffset.dy) * 0.008, 0.018)
+ let pulseCompression = clickProgress * 0.03
+
+ context.saveGState()
+ context.translateBy(
+ x: bounds.midX + drawingBodyOffset.dx,
+ y: bounds.midY + drawingBodyOffset.dy
+ )
+ context.rotate(by: drawingAngle(from: rotation - CursorGlyphCalibration.restingRotation))
+ context.scaleBy(
+ x: 1 - motionCompression - pulseCompression,
+ y: 1 + (pulseCompression * 0.4)
+ )
+ context.translateBy(x: -bounds.midX, y: -bounds.midY)
+ image.draw(in: bounds, from: .zero, operation: .sourceOver, fraction: 1)
+ context.restoreGState()
+ }
+
private func drawProceduralGlyph(in context: CGContext) {
- SoftwareCursorGlyphRenderer.draw(
- in: bounds,
- context: context,
- state: SoftwareCursorGlyphRenderState(
- rotation: drawingAngle(from: rotation - CursorGlyphCalibration.restingRotation),
- cursorBodyOffset: drawingVector(from: cursorBodyOffset),
- fogOffset: drawingVector(from: fogOffset),
- fogOpacity: fogOpacity,
- fogScale: fogScale,
- clickProgress: clickProgress
- )
+ let pulse = clickProgress
+ let drawingFogOffset = drawingVector(from: fogOffset)
+ let drawingBodyOffset = drawingVector(from: cursorBodyOffset)
+ let fogCenter = CGPoint(
+ x: bounds.midX + drawingFogOffset.dx,
+ y: bounds.midY + drawingFogOffset.dy
)
+ let pointerCenter = CGPoint(
+ x: bounds.midX + 2.6 + drawingBodyOffset.dx,
+ y: bounds.midY - 3.2 + drawingBodyOffset.dy + (pulse * 0.35)
+ )
+
+ drawFog(in: context, center: fogCenter, pulse: pulse)
+ drawPointer(in: context, center: pointerCenter, pulse: pulse)
+ }
+
+ private func drawFog(in context: CGContext, center: CGPoint, pulse: CGFloat) {
+ let radius = ((66 * fogScale) / 2) + (pulse * 1.2)
+ let glowRadius = radius * (0.30 + (pulse * 0.025))
+ let opacityMultiplier = max(0.28, min(fogOpacity / 0.12, 2.2))
+ let colors = [
+ NSColor(calibratedRed: 0.38, green: 0.36, blue: 0.35, alpha: (0.40 + (pulse * 0.02)) * opacityMultiplier).cgColor,
+ NSColor(calibratedRed: 0.43, green: 0.41, blue: 0.40, alpha: (0.28 + (pulse * 0.015)) * opacityMultiplier).cgColor,
+ NSColor(calibratedRed: 0.46, green: 0.44, blue: 0.43, alpha: 0.11 * opacityMultiplier).cgColor,
+ NSColor(calibratedWhite: 0.60, alpha: 0.0).cgColor,
+ ] as CFArray
+ let locations: [CGFloat] = [0, 0.50, 0.82, 1]
+ let colorSpace = CGColorSpaceCreateDeviceRGB()
+
+ guard let gradient = CGGradient(colorsSpace: colorSpace, colors: colors, locations: locations) else {
+ return
+ }
+
+ context.saveGState()
+ context.drawRadialGradient(
+ gradient,
+ startCenter: center,
+ startRadius: 0,
+ endCenter: center,
+ endRadius: radius,
+ options: [.drawsBeforeStartLocation, .drawsAfterEndLocation]
+ )
+ context.restoreGState()
+
+ let coreColors = [
+ NSColor(calibratedRed: 0.41, green: 0.39, blue: 0.38, alpha: (0.020 + (pulse * 0.006)) * opacityMultiplier).cgColor,
+ NSColor(calibratedRed: 0.44, green: 0.41, blue: 0.40, alpha: 0.008 * opacityMultiplier).cgColor,
+ NSColor(calibratedWhite: 0.80, alpha: 0.0).cgColor,
+ ] as CFArray
+ let coreLocations: [CGFloat] = [0, 0.62, 1]
+ guard let coreGradient = CGGradient(colorsSpace: colorSpace, colors: coreColors, locations: coreLocations) else {
+ return
+ }
+
+ context.saveGState()
+ context.drawRadialGradient(
+ coreGradient,
+ startCenter: center,
+ startRadius: 0,
+ endCenter: center,
+ endRadius: glowRadius,
+ options: [.drawsBeforeStartLocation, .drawsAfterEndLocation]
+ )
+ context.restoreGState()
+ }
+
+ private func drawPointer(in context: CGContext, center: CGPoint, pulse: CGFloat) {
+ let pointerRect = CGRect(
+ x: center.x - 10.5,
+ y: center.y - 10.5,
+ width: 21,
+ height: 21
+ )
+ let outerPath = pointerPath(in: pointerRect)
+
+ context.saveGState()
+ let drawingBodyOffset = drawingVector(from: cursorBodyOffset)
+ context.translateBy(x: bounds.midX + drawingBodyOffset.dx, y: bounds.midY + drawingBodyOffset.dy)
+ context.rotate(by: drawingAngle(from: rotation - CursorGlyphCalibration.restingRotation))
+ context.scaleBy(x: 1 - (pulse * 0.04), y: 1 + (pulse * 0.02))
+ context.translateBy(x: -(bounds.midX + drawingBodyOffset.dx), y: -(bounds.midY + drawingBodyOffset.dy))
+
+ NSGraphicsContext.saveGraphicsState()
+ let shadow = NSShadow()
+ shadow.shadowBlurRadius = 3.2 + (pulse * 1.4)
+ shadow.shadowOffset = CGSize(width: 0, height: -0.35)
+ shadow.shadowColor = NSColor.black.withAlphaComponent(0.11)
+ shadow.set()
+ NSColor.black.withAlphaComponent(0.05).setFill()
+ outerPath.fill()
+ NSGraphicsContext.restoreGraphicsState()
+
+ SynthesizedCursorColors.pointerFill.setFill()
+ outerPath.fill()
+
+ SynthesizedCursorColors.pointerStroke.setStroke()
+ outerPath.lineWidth = 1.55
+ outerPath.lineJoinStyle = .round
+ outerPath.lineCapStyle = .round
+ outerPath.stroke()
+
+ context.restoreGState()
+ }
+
+ private func pointerPath(in rect: CGRect) -> NSBezierPath {
+ let contourRows: [(y: CGFloat, minX: CGFloat, maxX: CGFloat)] = [
+ (39, 17, 21), (38, 16, 22), (37, 15, 22), (36, 15, 23), (35, 15, 24),
+ (34, 15, 24), (33, 14, 25), (32, 14, 25), (31, 14, 26), (30, 14, 27),
+ (29, 13, 29), (28, 13, 31), (27, 13, 34), (26, 13, 36), (25, 13, 37),
+ (24, 12, 37), (23, 12, 37), (22, 12, 37), (21, 12, 37), (20, 12, 36),
+ (19, 11, 36), (18, 11, 34), (17, 11, 32), (16, 11, 30), (15, 10, 27),
+ (14, 10, 25), (13, 10, 23), (12, 11, 21), (11, 11, 19), (10, 13, 16),
+ ]
+ let sourceMinX: CGFloat = 10
+ let sourceMaxX: CGFloat = 38
+ let sourceMinY: CGFloat = 10
+ let sourceMaxY: CGFloat = 39
+
+ func mappedPoint(x: CGFloat, y: CGFloat) -> CGPoint {
+ CGPoint(
+ x: rect.minX + ((x - sourceMinX) / (sourceMaxX - sourceMinX) * rect.width),
+ y: rect.minY + ((y - sourceMinY) / (sourceMaxY - sourceMinY) * rect.height)
+ )
+ }
+
+ let leftBoundary = contourRows.map { mappedPoint(x: $0.minX, y: $0.y) }
+ let rightBoundary = contourRows.reversed().map { mappedPoint(x: $0.maxX, y: $0.y) }
+
+ let path = NSBezierPath()
+ path.move(to: leftBoundary[0])
+ leftBoundary.dropFirst().forEach { path.line(to: $0) }
+ rightBoundary.forEach { path.line(to: $0) }
+ path.close()
+ path.lineJoinStyle = .round
+ return path
}
private func drawingVector(from screenVector: CGVector) -> CGVector {
@@ -80,3 +234,25 @@ final class SynthesizedCursorGlyphView: NSView {
-screenAngle
}
}
+
+private func loadReferenceCursorWindowImage() -> NSImage? {
+ if let bundledReference = Bundle.main.url(
+ forResource: "official-software-cursor-window-252",
+ withExtension: "png"
+ ), let image = NSImage(contentsOf: bundledReference) {
+ return image
+ }
+
+ let fileURL = URL(fileURLWithPath: #filePath).standardizedFileURL
+ let repoRoot = fileURL
+ .deletingLastPathComponent()
+ .deletingLastPathComponent()
+ .deletingLastPathComponent()
+ .deletingLastPathComponent()
+ .deletingLastPathComponent()
+
+ let referenceURL = repoRoot
+ .appendingPathComponent("docs/references/codex-computer-use-reverse-engineering/assets/extracted-2026-04-19/official-software-cursor-window-252.png")
+
+ return NSImage(contentsOf: referenceURL)
+}
diff --git a/packages/SoftwareCursorGlyphKit/Sources/SoftwareCursorGlyphKit/SoftwareCursorGlyphRenderer.swift b/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/SoftwareCursorGlyphRenderer.swift
similarity index 93%
rename from packages/SoftwareCursorGlyphKit/Sources/SoftwareCursorGlyphKit/SoftwareCursorGlyphRenderer.swift
rename to packages/OpenComputerUseKit/Sources/OpenComputerUseKit/SoftwareCursorGlyphRenderer.swift
index 370ac30..d558f20 100644
--- a/packages/SoftwareCursorGlyphKit/Sources/SoftwareCursorGlyphKit/SoftwareCursorGlyphRenderer.swift
+++ b/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/SoftwareCursorGlyphRenderer.swift
@@ -2,15 +2,15 @@ import AppKit
import CoreGraphics
import Foundation
-package struct SoftwareCursorGlyphRenderState {
- package let rotation: CGFloat
- package let cursorBodyOffset: CGVector
- package let fogOffset: CGVector
- package let fogOpacity: CGFloat
- package let fogScale: CGFloat
- package let clickProgress: CGFloat
-
- package init(
+struct SoftwareCursorGlyphRenderState {
+ let rotation: CGFloat
+ let cursorBodyOffset: CGVector
+ let fogOffset: CGVector
+ let fogOpacity: CGFloat
+ let fogScale: CGFloat
+ let clickProgress: CGFloat
+
+ init(
rotation: CGFloat,
cursorBodyOffset: CGVector,
fogOffset: CGVector,
@@ -27,9 +27,9 @@ package struct SoftwareCursorGlyphRenderState {
}
}
-package enum SoftwareCursorGlyphMetrics {
- package static let windowSize = CGSize(width: 126, height: 126)
- package static let tipAnchor = CGPoint(x: 60.35, y: 70.3)
+enum SoftwareCursorGlyphMetrics {
+ static let windowSize = CGSize(width: 126, height: 126)
+ static let tipAnchor = CGPoint(x: 60.35, y: 70.3)
static let pointerSize = CGSize(width: 21, height: 21)
static let pointerOffset = CGPoint(x: 2.6, y: -3.2)
@@ -40,8 +40,8 @@ private enum SoftwareCursorGlyphColors {
static let pointerStroke = NSColor(calibratedWhite: 0.90, alpha: 0.92)
}
-package enum SoftwareCursorGlyphRenderer {
- package static func draw(
+enum SoftwareCursorGlyphRenderer {
+ static func draw(
in bounds: CGRect,
context: CGContext,
state: SoftwareCursorGlyphRenderState
diff --git a/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/SoftwareCursorOverlay.swift b/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/SoftwareCursorOverlay.swift
index a328089..ff36d99 100644
--- a/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/SoftwareCursorOverlay.swift
+++ b/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/SoftwareCursorOverlay.swift
@@ -2,7 +2,6 @@ import AppKit
import CoreGraphics
import Foundation
import QuartzCore
-import SoftwareCursorGlyphKit
public enum VisualCursorSupport {
public static var isEnabled: Bool {
diff --git a/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/OpenComputerUseKitTests.swift b/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/OpenComputerUseKitTests.swift
index ba1cf29..7585378 100644
--- a/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/OpenComputerUseKitTests.swift
+++ b/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/OpenComputerUseKitTests.swift
@@ -1,5 +1,4 @@
import AppKit
-import SoftwareCursorGlyphKit
import XCTest
@testable import OpenComputerUseKit
@@ -315,7 +314,7 @@ final class OpenComputerUseKitTests: XCTestCase {
XCTAssertEqual(geometry.tipPosition(forOrigin: CGPoint(x: 1156, y: 712)), tipPosition)
}
- func testSoftwareCursorGlyphMetricsMatchSharedProceduralCalibration() {
+ func testSoftwareCursorGlyphMetricsMatchRuntimeProceduralCalibration() {
XCTAssertEqual(SoftwareCursorGlyphMetrics.windowSize, CGSize(width: 126, height: 126))
XCTAssertEqual(SoftwareCursorGlyphMetrics.tipAnchor.x, 60.35, accuracy: 0.01)
XCTAssertEqual(SoftwareCursorGlyphMetrics.tipAnchor.y, 70.3, accuracy: 0.01)
diff --git a/scripts/build-cursor-motion-dmg.sh b/scripts/build-cursor-motion-dmg.sh
index 8a068a1..b85cca4 100755
--- a/scripts/build-cursor-motion-dmg.sh
+++ b/scripts/build-cursor-motion-dmg.sh
@@ -216,6 +216,7 @@ bundle_version="${CURSOR_MOTION_BUNDLE_VERSION:-${GITHUB_RUN_NUMBER:-$(git -C "$
bundle_icon_name="CursorMotion.icns"
icon_master_png="${repo_root}/assets/app-icons/open-computer-use-1024.png"
iconset_build_script="${repo_root}/scripts/build-apple-iconset.sh"
+cursor_reference_source="${repo_root}/docs/references/codex-computer-use-reverse-engineering/assets/extracted-2026-04-19/official-software-cursor-window-252.png"
app_root="${output_dir}/${app_name}"
contents_dir="${app_root}/Contents"
macos_dir="${contents_dir}/MacOS"
@@ -248,6 +249,13 @@ esac
chmod +x "${macos_dir}/CursorMotion"
+if [[ ! -f "${cursor_reference_source}" ]]; then
+ echo "Missing cursor reference asset: ${cursor_reference_source}" >&2
+ exit 1
+fi
+
+cp "${cursor_reference_source}" "${resources_dir}/official-software-cursor-window-252.png"
+
if [[ ! -f "${icon_master_png}" ]]; then
echo "Missing icon master PNG: ${icon_master_png}" >&2
exit 1