Skip to content

👾 Open Computer Use – Open-Source Alternative to Codex Computer Use

Align runtime cursor speed with recovered timing

Leoc2ee2db3c573
diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md
index a89a1b4..f95ecb6 100644
--- a/docs/ARCHITECTURE.md
+++ b/docs/ARCHITECTURE.md
@@ -64,7 +64,7 @@
- overlay 的 visual style 不再自己从官方 app bundle 裁 `SoftwareCursor` 小图;主 MCP runtime 现在和 `CursorMotion` 一样优先渲染仓库里沉淀的 `official-software-cursor-window-252.png` baseline,只有资源缺失时才退回 `OpenComputerUseKit` 内部的程序化 pointer/fog fallback。命中点 anchor 仍固定在 `126x126` 画布里的同一组 tip-offset 上;glyph 自身的 neutral heading 继续沿用 `CursorMotion` / 官方 baseline 的 `-3π/4`。主 runtime overlay window 按 AppKit 全局坐标移动;路径选路用屏幕上实际可见的 AppKit forward heading,进入 visual dynamics / render state 前则把 velocity 的 y 轴翻回 CursorMotion 的 y-down screen state,再交给 AppKit 绘制层做角度和 `dy` 翻转。程序化 fallback 保留 neutral artwork correction,把它的天然轮廓轴对齐到 `CursorMotion` / 官方 baseline 的 `-3π/4` forward 方向,但不让实验线依赖 runtime 代码。
- overlay 的层级不再固定 `.floating`;现在会跟随 snapshot 命中的目标 window id / layer,把自己排到该目标 window 之上,而不是粗暴压到所有前台 app 最上层。
- overlay 的曲线路径不再只按固定 Bezier 模板生成;当前主线采用 reverse-engineering 约束下的 heading-driven candidate 族,候选只保留 `direct` / `turn` / `brake` / `orbit` 这些能稳定产出单侧主弧的 family,并继续保留 target-window 命中策略作为同类候选间的 tie-break。原始 binary lift 恢复出来的 `20` 条路径和 score 仍然保留在独立的 `StandaloneCursor` viewer / Python 重建脚本里,用于对照分析,不再直接作为 runtime 默认 chooser。
-- overlay 的 progress 曲线也不再是固定 `easeInOut`;主线现在复用官方 `response=1.4`、`dampingFraction=0.9`、`dt=1/240` 的 spring/`VelocityVerlet` 形状,但 wall-clock duration 仍做本地校准,因为官方 transaction-level 时长映射还没有完全恢复。
+- overlay 的 progress 曲线也不再是固定 `easeInOut`;主线现在复用官方 `response=1.4`、`dampingFraction=0.9`、`dt=1/240` 的 spring/`VelocityVerlet` 形状,默认 move 时长对齐已恢复出的 close-enough endpoint-lock 时间 `343 / 240 = 1.4291667s`,不再按路径距离额外压缩。
- overlay 不再依赖临时 `terminal settle` 补丁来修尾;主线现在统一改成“路径层给目标点,visual dynamics 层给可见姿态”的双层模型,所以 move 末段、pulse 和 idle 共用同一套状态,不会再出现 endpoint 锁住后只剩原地翻角的收尾。
- overlay 的渲染输入也从单一 `rotation` 扩展成 `rotation + cursorBodyOffset + fogOffset + fogScale`,让速度滞后能真正体现在画面上,而不是只存在于主循环内部状态;其中 `rotation` 现在按二进制里 `SoftwareCursorStyle.angle + CursorView._animatedAngleOffsetDegrees` 的分层去近似,不再把“跟随运动方向的主朝向”和“小幅 wiggle offset”压成同一个受限小角度。
- 动作型 tools 对普通 app 采用“非侵入优先,物理指针路径显式 opt-in”策略:
diff --git a/docs/histories/2026-04/20260422-1156-align-runtime-cursor-speed.md b/docs/histories/2026-04/20260422-1156-align-runtime-cursor-speed.md
new file mode 100644
index 0000000..645ce3e
--- /dev/null
+++ b/docs/histories/2026-04/20260422-1156-align-runtime-cursor-speed.md
@@ -0,0 +1,26 @@
+## [2026-04-22 11:56] | Task: 对齐 runtime overlay cursor 移动速度
+
+### Execution Context
+* **Agent ID**: `Codex`
+* **Base Model**: `GPT-5.4`
+* **Runtime**: `Codex CLI on macOS + SwiftPM`
+
+### User Query
+> `open-computer-use` 里的 overlay cursor 移动速度好像太快;对照官方速度和 `Cursor Motion` 里的可调速度。
+
+### Changes Overview
+**Scope:** `OpenComputerUseKit` cursor motion timing、文档、测试
+
+**Key Actions:**
+- **[确认差异]**: `Cursor Motion` 默认档已经对齐官方 `response=1.4 / damping=0.9` 的 `343 / 240 = 1.4291667s` endpoint-lock 时间,但 runtime 仍使用旧的距离压缩公式,实际常落在 `0.23s+`,导致中长距离移动明显偏快。
+- **[Runtime timing 对齐]**: `OfficialCursorMotionModel.calibratedTravelDuration` 改为直接返回 recovered close-enough 时间,不再按路径距离和曲率压缩 wall-clock duration。
+- **[回归测试]**: 新增测试锁定 runtime travel duration 等于 recovered endpoint-lock timing,避免后续重新引入距离压缩。
+- **[文档同步]**: 架构文档和逆向 motion model 文档更新为默认 move 时长对齐 `343 / 240`。
+
+### Files Modified
+- `packages/OpenComputerUseKit/Sources/OpenComputerUseKit/CursorMotionModel.swift`
+- `packages/OpenComputerUseKit/Sources/OpenComputerUseKit/SoftwareCursorOverlay.swift`
+- `packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/OpenComputerUseKitTests.swift`
+- `docs/ARCHITECTURE.md`
+- `docs/references/codex-computer-use-reverse-engineering/software-cursor-motion-model.md`
+- `docs/histories/2026-04/20260422-1156-align-runtime-cursor-speed.md`
diff --git a/docs/references/codex-computer-use-reverse-engineering/software-cursor-motion-model.md b/docs/references/codex-computer-use-reverse-engineering/software-cursor-motion-model.md
index 49347cf..236d1f2 100644
--- a/docs/references/codex-computer-use-reverse-engineering/software-cursor-motion-model.md
+++ b/docs/references/codex-computer-use-reverse-engineering/software-cursor-motion-model.md
@@ -281,6 +281,8 @@ CursorView
`VelocityVerletSimulation` 这个名字尤其关键,因为它已经把“用什么数值方法跑 spring”暴露出来了。就现有证据看,官方更像是在做逐帧积分,而不是单纯调用一个现成的 `CASpringAnimation` 然后交给系统黑盒求值。
+当前已经恢复出的 shipping 默认 progress spring 为 `response=1.4`、`dampingFraction=0.9`、`dt=1/240`。按 `CloseEnoughConfiguration(progressThreshold=1.0, distanceThreshold=0.01)` 计算,endpoint-lock / close-enough 时间约为 `343 / 240 = 1.4291667s`。这意味着默认可见移动不应该再叠加一层本地距离压缩;短距离和长距离都会复用同一条 spring progress 时间线,只是路径几何和可见姿态的速度感不同。
+
### 3. 什么时候允许下一次交互开始
`CloseEnoughConfiguration(progressThreshold, distanceThreshold)` 和 `CursorNextInteractionTiming(closeEnough, finished)` 这组类型,说明官方把“动作已足够接近,可继续下一步”和“动作完全结束”明确区分开了。
diff --git a/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/CursorMotionModel.swift b/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/CursorMotionModel.swift
index 02d4467..3c13f41 100644
--- a/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/CursorMotionModel.swift
+++ b/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/CursorMotionModel.swift
@@ -490,12 +490,8 @@ enum OfficialCursorMotionModel {
}
}
- static func calibratedTravelDuration(distance: CGFloat, measurement: CursorMotionMeasurement) -> CGFloat {
- let weightedRatio = max((measurement.length / max(distance, 1)) - 1, 0)
- let distanceTerm = 0.18 + min(distance / 1300, 0.35)
- let curvatureTerm = min(weightedRatio, 1.5) * 0.18
- let settleTerm = min(weightedRatio, 1.0) * 0.04
- return max(distanceTerm + curvatureTerm + settleTerm, 0.23)
+ static func calibratedTravelDuration(distance _: CGFloat, measurement _: CursorMotionMeasurement) -> CGFloat {
+ closeEnoughTime
}
private static func makeCandidate(
diff --git a/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/SoftwareCursorOverlay.swift b/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/SoftwareCursorOverlay.swift
index 58149ce..fe6a614 100644
--- a/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/SoftwareCursorOverlay.swift
+++ b/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/SoftwareCursorOverlay.swift
@@ -273,9 +273,9 @@ enum SoftwareCursorOverlay {
private static func animateMove(from start: CGPoint, to end: CGPoint, relativeTo targetWindow: CursorTargetWindow?) {
let candidate = bestMotionCandidate(from: start, to: end, relativeTo: targetWindow)
let path = candidate.path
- // Binary-backed geometry and spring shape are used directly, but the final
- // wall-clock duration is still calibrated locally because the official
- // transaction-level duration mapping has not been fully recovered yet.
+ // Use the recovered official progress spring timing instead of the older
+ // distance-compressed local duration, otherwise medium and long moves feel
+ // noticeably faster than the bundled app.
let duration = OfficialCursorMotionModel.calibratedTravelDuration(
distance: distanceBetween(start, end),
measurement: candidate.measurement
diff --git a/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/OpenComputerUseKitTests.swift b/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/OpenComputerUseKitTests.swift
index 0a78a5c..d9c294f 100644
--- a/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/OpenComputerUseKitTests.swift
+++ b/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/OpenComputerUseKitTests.swift
@@ -669,6 +669,26 @@ final class OpenComputerUseKitTests: XCTestCase {
XCTAssertEqual(OfficialCursorMotionModel.closeEnoughTime, 1.429166666666663, accuracy: 0.000_001)
}
+ func testOfficialCursorMotionTravelDurationUsesRecoveredEndpointLockTiming() {
+ let curvedMeasurement = CursorMotionMeasurement(
+ length: 1280,
+ angleChangeEnergy: 8,
+ maxAngleChange: 1.2,
+ totalTurn: 4,
+ staysInBounds: true
+ )
+
+ XCTAssertEqual(
+ OfficialCursorMotionModel.calibratedTravelDuration(distance: 140, measurement: curvedMeasurement),
+ OfficialCursorMotionModel.closeEnoughTime,
+ accuracy: 0.000_001
+ )
+ XCTAssertGreaterThan(
+ OfficialCursorMotionModel.calibratedTravelDuration(distance: 900, measurement: curvedMeasurement),
+ 1.0
+ )
+ }
+
func testHeadingDrivenMotionPrefersNearDirectPathWhenHeadingsAlreadyAlign() throws {
let start = CGPoint(x: 120, y: 120)
let end = CGPoint(x: 920, y: 320)