👾 Open Computer Use – Open-Source Alternative to Codex Computer Use
Swift accessibilityai-agentai-agentsclaude-codecodexcodex-computer-usecomputer-usecomputer-use-agentcross-platformcuadesktop-automationgemini-cligui-automationmacosmcpmodel-context-protocolopen-computer-useopen-computer-use-mcpopenai-codexopencode
Tune standalone cursor idle wobble
ifurystc98910831d32
diff --git a/docs/histories/2026-04/20260419-2219-refresh-official-overlay-arrow-findings.md b/docs/histories/2026-04/20260419-2219-refresh-official-overlay-arrow-findings.mdindex 36a8a0c..c9fe7ae 100644--- a/docs/histories/2026-04/20260419-2219-refresh-official-overlay-arrow-findings.md+++ b/docs/histories/2026-04/20260419-2219-refresh-official-overlay-arrow-findings.md@@ -18,6 +18,9 @@- **[落运行时样本]**: 直接对 `Software Cursor` window 做 `screencapture -l <windowid>`,把完整窗口和原始箭头 crop 落到 `docs/references/codex-computer-use-reverse-engineering/assets/extracted-2026-04-19/`。- **[补抓完整 2x overlay 边界]**: 继续直接调用 `CGWindowListCreateImage(..., .boundsIgnoreFraming | .bestResolution)`,确认 `Software Cursor` 的完整运行时像素边界是 `252x252`,而不是默认抓图链里看到的 `170x170` 裁剪结果;这和用户截图软件框到的大边框一致。- **[补独立测试脚本]**: 新增 `scripts/render-synthesized-software-cursor.swift`,用单文件 AppKit 脚本把 `126x126` overlay 独立渲染到屏幕。默认模式直接读取仓库里保存的 `official-software-cursor-window-252.png` 作为官方基线图,确保尺寸和轮廓先对上;`--procedural` 继续保留纯代码 fallback,单独迭代 fog 和 pointer 近似实现。+- **[补默认档晃动效果]**: 继续在独立脚本里给默认 reference-baseline 模式接入轻微的 center-fixed angle wobble,并补 `--snapshot-delay`,方便直接导出不同时间相位的独立样本。+- **[按二进制证据收紧 wobble]**: 随后又重新对齐仓库里的逆向文档和 `SkyComputerUseService` 的 runtime 证据,确认 `CursorView` 侧能直接看到的是 `_animatedAngleOffsetDegrees` / `_loadingAnimationToken`,`FogCursorViewModel` 侧是 velocity / pressed / activity / angle;因此把默认档从“整图平移 + 呼吸缩放 + pulse”收紧回“中心固定、顺时针/逆时针轻微摆角”的小幅 angle wobble。+- **[调整中心摆角振幅]**: 根据后续对官方视觉的回看,又把默认档的中心摆角调到接近“时钟 `55` 分到 `00` 分”的总摆幅,让独立脚本更接近用户观察到的钟摆式旋转范围。- **[更新逆向文档]**: 修改 `software-cursor-overlay.md`,把结论收敛成“最终灰白箭头不能从 bundle 静态 asset 直接导出,但能从运行时 `Software Cursor` window 直接截出”。### 🧠 Design Intent (Why)diff --git a/docs/references/codex-computer-use-reverse-engineering/software-cursor-overlay.md b/docs/references/codex-computer-use-reverse-engineering/software-cursor-overlay.mdindex 7fadcd4..785eae9 100644--- a/docs/references/codex-computer-use-reverse-engineering/software-cursor-overlay.md+++ b/docs/references/codex-computer-use-reverse-engineering/software-cursor-overlay.md@@ -220,11 +220,26 @@ swift scripts/render-synthesized-software-cursor.swift --seconds 12这条脚本现在分成两档:-- 默认档:直接读取仓库里已经落下来的 `official-software-cursor-window-252.png`,把这张 `252x252` runtime overlay 基线图独立渲染到一个 `126x126` 透明 window 上,用来先把“独立测试”和尺寸/边界对齐这件事做准。+- 默认档:直接读取仓库里已经落下来的 `official-software-cursor-window-252.png`,把这张 `252x252` runtime overlay 基线图独立渲染到一个 `126x126` 透明 window 上,并只额外叠一层“中心固定、像钟摆一样左右摆角”的 angle wobble。当前独立脚本把这段 wobble 收到接近“时钟 `55` 分到 `00` 分”的总摆幅,用来先把“独立测试”和官方视觉姿态做准。- `--procedural` 档:继续保留一版纯代码 fallback,用 radial fog + pointer contour 去近似官方视觉,方便后面单独迭代 pointer path、fog falloff 和按压状态。也就是说,这条脚本当前优先服务“可独立验证的官方基线”,而不是宣称 procedural 版本已经 1:1 复刻完成。+这条 idle wobble 后来又按二进制证据收紧过一轮:++- `CursorView` 一侧目前能直接确认的是 `cursorRadius`、`_animatedAngleOffsetDegrees`、`_loadingAnimationToken`、`fogRadius` 和两套 scale anchor;+- `FogCursorViewModel` 一侧当前能确认的是 `_velocityX`、`_velocityY`、`_isPressed`、`_activityState`、`_isAttached`、`_angle`;+- 所以独立脚本默认档现在不再对整张 runtime baseline 图做“呼吸式整体缩放”或额外平移,而是只保留一个以图案中心为轴的小幅 angle offset。++如果需要抓不同 wobble 相位的静态样本,可以直接加:++```bash+swift scripts/render-synthesized-software-cursor.swift \+ --seconds 1.4 \+ --snapshot-delay 0.9 \+ --save-png /tmp/software-cursor-wobble.png+```+## 为什么这套体验不会抢用户鼠标如果这个判断成立,那么体验上“看起来像有一个额外的小黄鼠标在代操作,但用户自己的真实鼠标仍然自由”就很好解释了:diff --git a/scripts/render-synthesized-software-cursor.swift b/scripts/render-synthesized-software-cursor.swiftindex a6256ba..7580423 100755--- a/scripts/render-synthesized-software-cursor.swift+++ b/scripts/render-synthesized-software-cursor.swift@@ -25,6 +25,7 @@ struct ScriptOptions {var centerX: CGFloat?var centerY: CGFloat?var seconds: TimeInterval = 12+ var snapshotDelay: TimeInterval = 0.45var idleDrift = truevar pulseLoop = truevar useReferenceCapture = true@@ -108,7 +109,7 @@ final class CursorView: NSView {let elapsed = CACurrentMediaTime() - startedAtif options.useReferenceCapture, let referenceImage {- referenceImage.draw(in: bounds, from: .zero, operation: .sourceOver, fraction: 1)+ drawReferenceImage(referenceImage, in: context, elapsed: elapsed)return}@@ -127,6 +128,21 @@ final class CursorView: NSView {drawPointer(in: context, center: pointerCenter, elapsed: elapsed, pulse: pulse)}+ private func drawReferenceImage(_ image: NSImage, in context: CGContext, elapsed: TimeInterval) {+ let idlePhase = CGFloat(elapsed * 3.0)+ let idleRotationAmplitude: CGFloat = .pi / 12+ let rotation = options.idleDrift+ ? sin(idlePhase * 0.8) * idleRotationAmplitude+ : 0++ context.saveGState()+ context.translateBy(x: bounds.midX, y: bounds.midY)+ context.rotate(by: rotation)+ context.translateBy(x: -bounds.midX, y: -bounds.midY)+ image.draw(in: bounds, from: .zero, operation: .sourceOver, fraction: 1)+ context.restoreGState()+ }+private func pulseProgress(at elapsed: TimeInterval) -> CGFloat {guard options.pulseLoop else {return 0@@ -346,7 +362,7 @@ final class AppDelegate: NSObject, NSApplicationDelegate {NSApp.activate(ignoringOtherApps: true)if let savePNGURL = options.savePNGURL {- Timer.scheduledTimer(withTimeInterval: 0.45, repeats: false) { [weak self] _ in+ Timer.scheduledTimer(withTimeInterval: options.snapshotDelay, repeats: false) { [weak self] _ inguard let self, let cursorView = self.cursorView else {return}@@ -432,6 +448,13 @@ func parseOptions(arguments: [String]) throws -> ScriptOptions {throw CursorScriptError.invalidArguments("Invalid value for --seconds: \(rawValue)")}options.seconds = seconds+ case "--snapshot-delay":+ index += 1+ let rawValue = try parseString(arguments, index: index, flag: argument)+ guard let snapshotDelay = TimeInterval(rawValue), snapshotDelay >= 0 else {+ throw CursorScriptError.invalidArguments("Invalid value for --snapshot-delay: \(rawValue)")+ }+ options.snapshotDelay = snapshotDelaycase "--save-png":index += 1let path = try parseString(arguments, index: index, flag: argument)@@ -521,6 +544,8 @@ func printUsage() {--x <value> Overlay center x position in screen coordinates--y <value> Overlay center y position in screen coordinates--seconds <value> Auto-exit after N seconds; use 0 to keep it open+ --snapshot-delay <value>+ Delay before writing --save-png, useful for sampling wobble phases--save-png <path> Save a 2x transparent PNG snapshot of the synthesized overlay--procedural Render the current code-generated fallback instead of the captured official baseline--no-idle Disable idle drift