👾 Open Computer Use – Open-Source Alternative to Codex Computer Use
Swift accessibilityai-agentai-agentsclaude-codecodexcodex-computer-usecomputer-usecomputer-use-agentcross-platformcuadesktop-automationgemini-cligui-automationmacosmcpmodel-context-protocolopen-computer-useopen-computer-use-mcpopenai-codexopencode
Refine standalone cursor lab slider behavior
Leof55fcd7d54bf
diff --git a/docs/exec-plans/active/20260418-standalone-cursor-lab.md b/docs/exec-plans/active/20260418-standalone-cursor-lab.mdindex 85655d4..7d6917b 100644--- a/docs/exec-plans/active/20260418-standalone-cursor-lab.md+++ b/docs/exec-plans/active/20260418-standalone-cursor-lab.md@@ -74,6 +74,12 @@- 2026-04-20:对照用户提供的官方视频抽帧后,确认 moving 阶段的可见箭头确实会持续跟随当前 move heading,而不是只保留一个轻微 lean;lab 现已撤回那层 `displayRotation` 分离,glyph 渲染重新直接使用 visual dynamics 的主 `rotation`。- 2026-04-20:继续对照 `official-software-cursor-window-252.png` 与独立脚本的静止朝向后,确认 lab 之前额外加的 `-26.5°` glyph 补偿会让 moving heading 固定偏掉;当前已把静止基线收敛到和主运行时一致的“零旋转即左上朝向”,也就是 y-down 画布里的 `-3π/4`。- 2026-04-20:继续排查“屁股超前”后,确认真正的问题是 lab 的 motion/heading 运行在 SwiftUI 的 y-down 坐标,但 glyph 渲染落在 AppKit 默认 y-up `NSView`;当前已在 glyph 渲染层对 angle、body offset 和 fog offset 统一做 y-down -> y-up 转换,避免 moving 姿态被垂直镜像。+- 2026-04-20:把刚确认的 slider 语义进一步收回到 `StandaloneCursorLab` 主线上;`START HANDLE` 现在主要调起步段的 guide / reach / normal,`END HANDLE` 主要调收尾段的 guide / reach / normal,不再只是对整条曲线做对称缩放。同时移除了过紧的 corridor clipping,改用内缩 canvas bounds,避免默认样例里 `END HANDLE` 被提前裁没。+- 2026-04-20:继续按同样方法收紧 `ARC SIZE`;当前它明确表示轨迹弧度而不是 cursor 大小,并且已经同时接到局部 path 的弧高/控制点侧向偏移,以及 chooser 对 `direct` 和 `arched` family 的偏好。默认点位采样下,`arcSize=0.04 -> 0.12` 会让 `curveScale` 从约 `10.3` 提升到 `47.7`,中段 `y` 也从约 `326.8` 抬到 `345.5`。+- 2026-04-20:继续收紧 `ARC FLOW`;当前它明确表示“最宽弧段在 chord 上更靠前还是更靠后”,实现上不再只是改 start/end reach 的抽象 bias,而是改到单段 cubic 控制点的前后相位偏置。+- 2026-04-20:继续收紧 `SPRING`;当前它明确表示 progress spring 的 `response / damping` 与 endpoint-lock 时间,默认 `spring=0.5` 会精确回到官方 `.official` 配置。采样下,`spring=0.25 / 0.5 / 0.75` 分别对应约 `1.0958s / 1.4292s / 1.8750s` 的 endpoint-lock,语义已经收成稳定的“左快右慢”。+- 2026-04-20:修正 `ARC FLOW` 首轮实现引入的中段“节点感”;问题不在 `SPRING`,而在那版显式中间锚点的双段曲线把路径本身做出了 join,叠加 debug overlay 又把这个中点强调出来。当前已经收回到单段 cubic,相位控制继续保留,但几何上不再存在中间 join。+- 2026-04-20:继续修正 slider 调参状态流;当前已把“当前 cursor 位置”和“当前会话的 reference path”拆开,slider 重算时用最近一次真实 move 的 origin / startRotation + `queuedTarget` 更新整条调试曲线,同时只把 cursor snap 在当前位置,因此 settled 后调参也不会把 DEBUG 线重建成 `target -> target` 的零长度路径。## 决策记录@@ -84,3 +90,8 @@- 2026-04-19:对 `swift_once` 恢复出的 guide 系数,当前默认采用“常量已确认、世界坐标解释不成立、局部基底投影更贴近官方视频”的实现策略;后续如果拿到更强的二进制级证据,再继续下沉这层解释。- 2026-04-19:raw binary lift 的 `20` candidate pool 保留在 `StandaloneCursor` 这条分析线;`StandaloneCursorLab` 和主 runtime overlay 则统一切到 heading-driven 主线,实现上优先保证“朝向约束 + 单侧转弯”这个更贴近官方视频的结构行为。- 2026-04-20:恢复 slider UI 时,继续把“调参入口”和“binary-confirmed 结构”分开表述;lab 可以暴露 `start/end handle`、`arc size/flow`、`spring` 这些测试旋钮,但文档和代码都不把它们说成官方一一字段对照。+- 2026-04-20:对 `start/end handle` 这两个旋钮,当前实验线采用“作用在局部 start/end control 几何,而不是全局统一缩放”的策略;这更接近 binary lift 里 `startControl/endControl` 与 `startExtent/endExtent` 的边界。+- 2026-04-20:对 `ARC SIZE`,当前实验线采用“作用在局部 arc height / normal bias / family chooser,而不是 cursor glyph 尺寸”的策略;这更接近 binary lift 里 `handleExtent / arcExtent / tableA / tableB` 对曲线宽度的作用边界。+- 2026-04-20:对 `ARC FLOW`,当前实验线采用“作用在单段 cubic 控制点的前后相位,而不是单纯抽象 reach bias”的策略;这更接近 reverse-engineering 里“最宽弧段被沿 guide 方向推前/推后”的边界。+- 2026-04-20:对 `SPRING`,当前实验线采用“围绕官方 `.official` 的 centered response/damping remap,而不是再叠加额外 distance-based duration 层”的策略;这更接近当前 binary-backed 证据里“主链直接吃 `1.4 / 0.9` spring config”的边界。+- 2026-04-20:调左上角 slider 时,path 重建现在统一基于当前会话里最近一次 move 的 reference origin / startRotation / `queuedTarget`,而不是外层 `@State start/end` 或 settled 后的 live endpoint;这样参数调节既不会把曲线起点误拉回初始位置,也不会把 DEBUG 反馈线消成零长度。diff --git a/docs/exec-plans/active/20260420-cursor-slider-binary-investigation.md b/docs/exec-plans/active/20260420-cursor-slider-binary-investigation.mdnew file mode 100644index 0000000..75ac630--- /dev/null+++ b/docs/exec-plans/active/20260420-cursor-slider-binary-investigation.md@@ -0,0 +1,68 @@+# Cursor Slider Binary Investigation++## 目标++继续围绕官方 `Codex Computer Use.app` 的 cursor motion 做一轮更聚焦的逆向分析,回答 5 个 slider (`start handle`、`end handle`、`arc size`、`arc flow`、`spring`) 在 shipping binary 中是否存在直接证据;如果没有,则把它们和当前已经 binary-confirmed 的路径几何 / spring 链路做一层明确标注的映射分析,并产出可重复运行的敏感性分析输出。++## 范围++- 包含:+ - 检查官方 shipping bundle 中是否存在 slider 文案、独立参数类型或明显的调试 UI 证据。+ - 复用 `scripts/cursor-motion-re/official_cursor_motion.py` 中已经确认的候选几何与 spring 常量,补一版参数敏感性分析。+ - 把“binary-confirmed 证据”和“根据路径几何做的 slider 映射推断”清晰分开写回文档。+- 不包含:+ - 不把新的 slider 映射直接宣称为官方一一字段对照。+ - 不修改主 MCP runtime 的 `SoftwareCursorOverlay` 行为。+ - 不要求在这轮里恢复内部 debug UI 的完整实现来源。++## 背景++- 相关文档:+ - `docs/references/codex-computer-use-reverse-engineering/software-cursor-motion-model.md`+ - `docs/references/codex-computer-use-reverse-engineering/software-cursor-motion-reconstruction.md`+ - `docs/exec-plans/active/20260419-official-cursor-motion-reconstruction.md`+- 相关代码路径:+ - `scripts/cursor-motion-re/`+ - `experiments/StandaloneCursorLab/`+- 已知约束:+ - 当前本机可用的官方 binary 为 `~/.codex/plugins/cache/openai-bundled/computer-use/1.0.750/Codex Computer Use.app/Contents/MacOS/SkyComputerUseService`。+ - 用户看到的 slider 来自视频 / 调试构建,不代表 shipping bundle 一定保留相同 UI 文案。+ - 当前仓库已经明确要求区分 `confirmed_from_binary` 与 `reconstructed`。++## 风险++- 风险:把 shipping bundle 中不存在的 slider 文案误写成“官方 release 里已确认存在”。+ - 缓解方式:先做全 bundle 文本扫描;若未命中,只记为“debug-build evidence from video, not string-confirmed in release bundle”。+- 风险:把参数敏感性分析误写成“已经定位到真实调参接口”。+ - 缓解方式:分析输出与文档都显式区分“直接命中 binary 的字段/常量”和“基于这些量做的 slider 映射推断”。+- 风险:分析脚本只对某个起终点样本成立,结论过拟合。+ - 缓解方式:让脚本支持传入任意 `start/end/bounds`,输出具体 measurement / chosen candidate 变化。++## 里程碑++1. 收敛 shipping binary 中与 slider 相关的直接证据。+2. 实现可重复运行的 slider sensitivity / parameter mapping analysis。+3. 文档沉淀、history 与验证。++## 验证方式++- 命令:+ - `python3 scripts/cursor-motion-re/reconstruct_cursor_motion.py inspect --pretty`+ - `python3 scripts/cursor-motion-re/reconstruct_cursor_motion.py slider-study --start 220 440 --end 860 260 --bounds 0 0 1120 760 --pretty`+- 手工检查:+ - 确认 shipping bundle 中的字符串扫描没有把 slider 文案误报为已存在。+ - 确认每个 slider 的分析输出都能落到具体几何字段、measurement 或 spring timeline 变化。+- 观测检查:+ - 文档里明确标出“release bundle string evidence”和“geometry/timing inference”两层边界。++## 进度记录++- [x] 里程碑 1+- [x] 里程碑 2+- [x] 里程碑 3++## 决策记录++- 2026-04-20:这轮不继续追“视频里的 slider UI 本身怎么渲染”,而是优先回答 shipping binary 中是否保留对应证据,以及这些 knob 更接近哪些已确认几何 / spring 量。+- 2026-04-20:整包 phrase scan 没命中 `START HANDLE`、`END HANDLE`、`ARC SIZE`、`ARC FLOW`,所以仓库文档改成“视频证据仍成立,但更像内部调试构建而非 release bundle 直接可见 UI”。+- 2026-04-20:新增 `slider-study` CLI,而不是把参数敏感性分析继续塞进 `inspect` / `demo`;这样可以把“shipping phrase evidence”“binary-confirmed motion terms”“slider mapping inference”三层输出保持分离。diff --git a/docs/histories/2026-04/20260418-1430-standalone-cursor-lab.md b/docs/histories/2026-04/20260418-1430-standalone-cursor-lab.mdindex 818a0e3..0a03f8f 100644--- a/docs/histories/2026-04/20260418-1430-standalone-cursor-lab.md+++ b/docs/histories/2026-04/20260418-1430-standalone-cursor-lab.md@@ -163,6 +163,62 @@- `experiments/StandaloneCursorLab/README.md`- `docs/exec-plans/active/20260418-standalone-cursor-lab.md`+### 🔁 Follow-up (2026-04-20, apply arc-size semantics from slider investigation)+**Scope:** `experiments/StandaloneCursorLab/`、`docs/`++**Key Actions:**+- **[明确 ARC SIZE 不是 cursor 大小]**: `ARC SIZE` 这次明确收口为“轨迹弧度”旋钮,不再给出任何会让人误解成 cursor glyph 尺寸的语义空间。+- **[接到弧高与控制点侧向偏移]**: heading-driven path builder 现在会用 `ARC SIZE` 同时调 `baseArcHeight`、guide normal bias 和 start/end normal scale,让 slider 直接影响中段离 chord 的偏移和整条曲线的开口宽度。+- **[接到 chooser 偏好]**: scoring 也补了 `ARC SIZE` 对 family 选择的影响;arc 更小时更偏 `direct/tight`,arc 更大时会更愿意保留 `turn/brake/orbit` 这类更宽的弧线路径。+- **[补默认样例采样]**: 用 lab 默认点位验证后,`arcSize=0.04` 时选中 `brake-primary-tight`、`curveScale≈10.3`、中段 `y≈326.8`;调到 `0.12` 后仍在主 family 内,但 `curveScale≈47.7`、中段 `y≈345.5`,能直接看出弧度抬高。++### 📁 Additional Files Modified+- `experiments/StandaloneCursorLab/Sources/StandaloneCursorLab/CursorMotionModel.swift`+- `experiments/StandaloneCursorLab/README.md`+- `docs/exec-plans/active/20260418-standalone-cursor-lab.md`++### 🔁 Follow-up (2026-04-20, remove mid-curve point feel introduced by arc-flow)+**Scope:** `experiments/StandaloneCursorLab/`、`docs/`++**Key Actions:**+- **[定位到不是 SPRING 问题]**: 重新检查后确认 `SPRING` 只影响 progress timing,不参与路径几何;用户看到的“中间多了一个点”来自 `ARC FLOW` 首轮实现。+- **[直接移除中间 join]**: `ARC FLOW` 当前不再通过显式中间锚点双段曲线实现,而是收回到单段 cubic 的控制点前后相位偏置,从根上消掉“曲线中间多了一个节点”的几何来源。+- **[移除中点 debug 强调]**: `DEBUG` overlay 也不再单独渲染那个中间锚点,避免视觉上继续像“曲线中间有个控制节点”。+- **[保留 ARC FLOW 的作用边界]**: 修完后 `ARC FLOW` 仍然会影响路径前后相位,但不再靠一条显式 join 来表达。++### 📁 Additional Files Modified+- `experiments/StandaloneCursorLab/Sources/StandaloneCursorLab/CursorMotionModel.swift`+- `experiments/StandaloneCursorLab/Sources/StandaloneCursorLab/CursorLabRootView.swift`+- `docs/exec-plans/active/20260418-standalone-cursor-lab.md`++### 🔁 Follow-up (2026-04-20, apply spring semantics from slider investigation)+**Scope:** `experiments/StandaloneCursorLab/`、`docs/`++**Key Actions:**+- **[把 SPRING 收回官方中心档]**: `SPRING` slider 不再走之前那套“拖高以后反而更快收口”的映射;现在改成围绕官方 `response=1.4 / damping=0.9` 的 centered remap,并在 `spring=0.5` 时精确返回 `.official`。+- **[保持 timing 只由 spring 决定]**: 这次没有再引入任何新的距离驱动时长层;`travelDuration` 继续直接取 spring 的 endpoint-lock 时间,保持和当前 reverse-engineering 边界一致。+- **[把语义收成左快右慢]**: 当前档位已经稳定成“往左更快更硬,往右更慢更稳”。采样下,`spring=0.25 / 0.5 / 0.75` 分别对应约 `1.0958s / 1.4292s / 1.8750s` 的 endpoint-lock。+- **[补默认档验证]**: 额外验证了 `spring=0.5` 时 `progressSpringConfiguration == .official`,因此默认档会继续命中 `343/240` 的官方 endpoint-lock 快路,而不是只算出一个近似值。++### 📁 Additional Files Modified+- `experiments/StandaloneCursorLab/Sources/StandaloneCursorLab/CursorMotionModel.swift`+- `experiments/StandaloneCursorLab/README.md`+- `docs/exec-plans/active/20260418-standalone-cursor-lab.md`++### 🔁 Follow-up (2026-04-20, apply arc-flow semantics from slider investigation)+**Scope:** `experiments/StandaloneCursorLab/`、`docs/`++**Key Actions:**+- **[明确 ARC FLOW 不是改弧度大小]**: `ARC FLOW` 这次明确收口为“最宽弧段沿主轴的前后相位位置”,不再和 `ARC SIZE` 混成同一个“弯曲程度”旋钮。+- **[先把 ARC FLOW 接到显式中间锚点]**: 首轮实现里,heading-driven path builder 曾短暂把 `turn / brake / orbit` family 切到显式中间锚点的双段曲线,让 `ARC FLOW` 能更直接前后移动弧顶。+- **[保留 start/end handle 与 arc size 的局部几何语义]**: 这次没有回退前面已经确认的 `START/END HANDLE` 和 `ARC SIZE` 语义,而是在其上叠加 `ARC FLOW` 对 apex phase 的控制。+- **[随后收回成单段 cubic]**: 用户确认显式中间锚点会让曲线读起来像“中间多了一个点”后,`ARC FLOW` 已收回到单段 cubic 的前后相位偏置;当前继续保留相位控制,但不再通过可见 join 来实现。++### 📁 Additional Files Modified+- `experiments/StandaloneCursorLab/Sources/StandaloneCursorLab/CursorMotionModel.swift`+- `experiments/StandaloneCursorLab/README.md`+- `docs/exec-plans/active/20260418-standalone-cursor-lab.md`+### 🔁 Follow-up (2026-04-20, align default move speed with official endpoint-lock timing)**Scope:** `experiments/StandaloneCursorLab/`、`docs/`@@ -201,3 +257,43 @@### 📁 Additional Files Modified- `experiments/StandaloneCursorLab/Sources/StandaloneCursorLab/CursorLabRootView.swift`- `docs/exec-plans/active/20260418-standalone-cursor-lab.md`++### 🔁 Follow-up (2026-04-20, preserve live path origin while tuning sliders)+**Scope:** `experiments/StandaloneCursorLab/`、`docs/`++**Key Actions:**+- **[修正 slider 调参回跳]**: `motionParameters` 变化时不再把外层 `@State start/end` 重新喂回 `updateParameters`;当前会话的调试曲线现在改由最近一次真实 move 的 reference origin 和 `queuedTarget` 重建,避免曲线起点偶发回到初始点。+- **[保留当前 session 的 start heading]**: slider 调参时会继续沿用这次 move 开始时记录下来的 `startRotation`,而不是临时取 settled 后的 endpoint 姿态;这样 candidate chooser 不会因为调参时机不同而突然换成另一套起步朝向。+- **[同步记录]**: active plan 补充这次 bugfix,明确“参数调节不应改写当前会话位置”已经和此前的 resize 修复收口到同一原则。++### 📁 Additional Files Modified+- `experiments/StandaloneCursorLab/Sources/StandaloneCursorLab/CursorLabRootView.swift`+- `docs/exec-plans/active/20260418-standalone-cursor-lab.md`++### 🔁 Follow-up (2026-04-20, keep debug path visible while tuning sliders)+**Scope:** `experiments/StandaloneCursorLab/`、`docs/`++**Key Actions:**+- **[修正 DEBUG 线消失]**: 上一轮把 slider 调参直接绑到 `currentState.point -> queuedTarget` 后,在 settled 态会退化成 `target -> target` 的零长度 path;现在把“cursor 当前点位”和“reference path”分离,DEBUG 模式下调参仍会显示完整曲线反馈。+- **[保留当前 cursor 不回跳]**: 调参时仍然只把 simulator snap 在当前位置,不会因为恢复完整 reference path 而把 cursor 本体重新拉回起点。+- **[同步文档口径]**: active plan 和 README 都补充这次调整,明确 slider 的可视反馈来自当前 session path,而不是简单拿 live endpoint 直接重建。++### 📁 Additional Files Modified+- `experiments/StandaloneCursorLab/Sources/StandaloneCursorLab/CursorLabRootView.swift`+- `experiments/StandaloneCursorLab/README.md`+- `docs/exec-plans/active/20260418-standalone-cursor-lab.md`++### 🔁 Follow-up (2026-04-20, apply start/end handle semantics from slider investigation)+**Scope:** `experiments/StandaloneCursorLab/`、`docs/`++**Key Actions:**+- **[收紧 START HANDLE 语义]**: `START HANDLE` 不再只是参与全局 curve reach 的对称缩放;现在优先改变起步段的 guide line/heading mix、start reach 和 start-side normal 偏置,让前段轨迹能更明确地体现“先甩出去多远、多久才回咬主轴”。+- **[收紧 END HANDLE 语义]**: `END HANDLE` 也改为只优先作用在 end guide / end reach / end-side normal 上,使末段收束钩子的长度和贴回目标的时机能单独变化,而不是跟起步段一起等比例变化。+- **[放宽默认样例的 bounds clipping]**: lab 选路不再使用过紧的 corridor bounds,而是改用画布内缩后的实际 canvas bounds;这样默认样例下 `END HANDLE` 不会因为提前 clipping 而看起来“几乎没反应”。+- **[补验证采样]**: 除了 `swift build --product StandaloneCursorLab` 之外,还用默认点位做了小脚本采样,确认 `START HANDLE` 和 `END HANDLE` 在接近默认档的小范围调节下也会改变 early/late path sample 与 control geometry。++### 📁 Additional Files Modified+- `experiments/StandaloneCursorLab/Sources/StandaloneCursorLab/CursorMotionModel.swift`+- `experiments/StandaloneCursorLab/Sources/StandaloneCursorLab/CursorLabRootView.swift`+- `experiments/StandaloneCursorLab/README.md`+- `docs/exec-plans/active/20260418-standalone-cursor-lab.md`diff --git a/docs/histories/2026-04/20260420-1151-investigate-cursor-slider-binary-mapping.md b/docs/histories/2026-04/20260420-1151-investigate-cursor-slider-binary-mapping.mdnew file mode 100644index 0000000..def68e1--- /dev/null+++ b/docs/histories/2026-04/20260420-1151-investigate-cursor-slider-binary-mapping.md@@ -0,0 +1,30 @@+## [2026-04-20 11:51] | Task: 深挖 cursor slider 的 binary 映射++### 🤖 Execution Context+* **Agent ID**: `codex`+* **Base Model**: `gpt-5`+* **Runtime**: `Codex CLI`++### 📥 User Query+> 现在我们有了这些参数调节,现在我们看看之前逆向分析的,再去二进制里挖掘一下,这几个参数调节的,看看二进制里是否有涉及这个,以及他们调节后分别对实际的曲线有什么影响++### 🛠 Changes Overview+**Scope:** `scripts/cursor-motion-re/`、`docs/references/`、`docs/exec-plans/`++**Key Actions:**+- **[新增 slider-study 分析入口]**: 在 `scripts/cursor-motion-re/official_cursor_motion.py` / `reconstruct_cursor_motion.py` 中新增 `slider-study` 子命令,输出 shipping bundle phrase scan、binary-confirmed motion terms,以及 5 个 slider 的参数敏感性分析。+- **[补 shipping bundle 证据边界]**: 明确记录当前 release bundle 未命中 `START HANDLE`、`END HANDLE`、`ARC SIZE`、`ARC FLOW` 这些完整 phrase;`SPRING` / `DEBUG` / `MAIL` / `CLICK` 仅作为歧义 token 命中,不再被误写成“debug UI 仍然随 release shipping”。+- **[沉淀 slider 映射文档]**: 新增 `software-cursor-slider-parameter-investigation.md`,把 `start/end handle`、`arc size/flow`、`spring` 各自对应的 binary-confirmed 几何 / timing 量,以及默认样例 / 居中样例下的实际曲线影响单独沉淀。+- **[同步旧文档口径]**: `software-cursor-motion-model.md` 不再只写“视频里有 slider UI”,而是补充说明当前 shipping bundle phrase scan 没有直接命中这组 label。+- **[新增独立执行计划]**: 为这轮参数映射调查新增 active execution plan,避免继续挤在之前的 cursor reconstruction 或 StandaloneCursorLab UI 任务里。++### 🧠 Design Intent (Why)+这轮的关键不是继续猜 5 个 slider 在 UI 上长什么样,而是把“release binary 里还能直接确认什么”和“我们如何基于这些量推断 slider 对曲线的影响”明确分层。这样后续继续挖 `ARC FLOW` 或 spring remap 时,就不会把 shipping 证据、视频证据和本地 lab 调参混成一团。++### 📁 Files Modified+- `scripts/cursor-motion-re/official_cursor_motion.py`+- `scripts/cursor-motion-re/README.md`+- `docs/references/codex-computer-use-reverse-engineering/software-cursor-slider-parameter-investigation.md`+- `docs/references/codex-computer-use-reverse-engineering/software-cursor-motion-model.md`+- `docs/references/codex-computer-use-reverse-engineering/README.md`+- `docs/exec-plans/active/20260420-cursor-slider-binary-investigation.md`diff --git a/docs/histories/2026-04/20260420-1238-confirm-official-overlay-trigger-matrix.md b/docs/histories/2026-04/20260420-1238-confirm-official-overlay-trigger-matrix.mdnew file mode 100644index 0000000..0c52cda--- /dev/null+++ b/docs/histories/2026-04/20260420-1238-confirm-official-overlay-trigger-matrix.md@@ -0,0 +1,24 @@+## [2026-04-20 12:38] | Task: 确认官方 overlay cursor 的 tool 触发矩阵++### 🤖 Execution Context+* **Agent ID**: `codex`+* **Base Model**: `gpt-5.4`+* **Runtime**: `Codex CLI + official computer-use app-server probe`++### 📥 User Query+> 通过 `scripts/computer-use-cli/` 的 `app-server` 连到官方的 `computer-use`,结合 bundled app 二进制分析,确认原来在调用什么 tool 的时候会出现 overlay cursor;并进一步确认 `type_text / press_key` 会不会触发。++### 🛠 Changes Overview+**Scope:** `docs/references`, `docs/histories`++**Key Actions:**+- **[补做官方路径实测]**: 用隔离 `HOME` 强制只启用 bundled `computer-use`,避免把本地 `open-computer-use` 和官方实现混淆。+- **[沉淀 tool 触发矩阵]**: 把 `set_value / scroll / drag / perform_secondary_action / click / type_text / press_key` 的 overlay 触发结论补进 reverse-engineering 文档。+- **[记录 click 分叉]**: 明确 element-scoped `click` 和 coordinate `click` 不一定走同一条执行链。++### 🧠 Design Intent (Why)+这类结论如果只留在临时日志和聊天上下文里,下次很容易重复踩坑,尤其当前机器默认配置里官方插件是关的,本地开源实现是开的。把“必须先切到官方路径”和“哪些 tool 真会拉起 `Software Cursor`”一起落文档,能直接减少后续逆向分析误判。++### 📁 Files Modified+- `docs/references/codex-computer-use-reverse-engineering/software-cursor-overlay.md`+- `docs/histories/2026-04/20260420-1238-confirm-official-overlay-trigger-matrix.md`diff --git a/docs/references/codex-computer-use-reverse-engineering/README.md b/docs/references/codex-computer-use-reverse-engineering/README.mdindex 8190234..6167aae 100644--- a/docs/references/codex-computer-use-reverse-engineering/README.md+++ b/docs/references/codex-computer-use-reverse-engineering/README.md@@ -20,6 +20,8 @@- 结合视频样本、官方字符串和当前开源实现,对 cursor motion 参数模型的推断。- `software-cursor-motion-reconstruction.md`- 继续下钻到函数级后,对 `CursorMotionPath.sample(progress)`、`CursorMotionPathMeasurement`、`CursorMotionPath/Segment` 布局、20 条候选几何、score 公式、in-bounds 优先选择策略,以及 `SpringAnimation -> VelocityVerletSimulation` timing 链、finished predicate 和 endpoint-lock 观测的重建说明。+- `software-cursor-slider-parameter-investigation.md`+ - 针对视频里 5 个 slider,记录 shipping bundle 中是否还带对应 label phrase、它们与当前 binary-confirmed 几何 / spring 量的映射关系,以及对实际曲线的敏感性分析。- `permission-onboarding.md`- 对 Accessibility / Screen Recording 权限引导和 System Settings accessory window 的分析。- `assets/README.md`diff --git a/docs/references/codex-computer-use-reverse-engineering/software-cursor-motion-model.md b/docs/references/codex-computer-use-reverse-engineering/software-cursor-motion-model.mdindex 33248ed..2438768 100644--- a/docs/references/codex-computer-use-reverse-engineering/software-cursor-motion-model.md+++ b/docs/references/codex-computer-use-reverse-engineering/software-cursor-motion-model.md@@ -28,6 +28,10 @@这说明至少在官方内部调试构建里,cursor motion 不是黑盒常量,而是一组可实时调的参数。+需要补一条边界:当前本机 shipping bundle+`~/.codex/plugins/cache/openai-bundled/computer-use/1.0.750/Codex Computer Use.app`+做整包 phrase scan 后,并没有命中 `START HANDLE`、`END HANDLE`、`ARC SIZE`、`ARC FLOW` 这些完整 label。也就是说,视频里的 slider UI 仍然是有效证据,但更像内部调试构建或未发布调试面板,而不是当前 release app 里可直接字符串恢复出来的现成界面。+### 2. `SkyComputerUseService` 里不仅有字符串,还有可恢复的 Swift motion 类型这次除了 `strings`,还额外做了两步:diff --git a/docs/references/codex-computer-use-reverse-engineering/software-cursor-overlay.md b/docs/references/codex-computer-use-reverse-engineering/software-cursor-overlay.mdindex 785eae9..5ae5053 100644--- a/docs/references/codex-computer-use-reverse-engineering/software-cursor-overlay.md+++ b/docs/references/codex-computer-use-reverse-engineering/software-cursor-overlay.md@@ -175,6 +175,44 @@ Bounds: X=1949 Y=796 Width=126 Height=126- 但当前版本下,`Software Cursor` 作为独立 window 仍然存在,只是如果只做“新窗口差分”而不按 pid 全量扫,很容易漏掉它;- 因此“host 侧可能参与了一部分合成”仍然成立,但“service 侧已经没有独立 software cursor window”这个结论当前不成立。+### 7. 2026 年 4 月 20 日补做 tool 级触发矩阵后,可以确认 overlay 不是只绑定 `click`++这轮又补做了一次更细的官方路径实测,重点不是“有没有 `Software Cursor`”,而是“哪些 public tool 会把它真正拉出来”。++这里有一个前提很重要:++- 这台机器的日常 Codex 用户态配置里,官方 bundled `computer-use@openai-bundled` 默认是关的,而本地 `open-computer-use` MCP 是开的。+- 所以这次所有样本都先切到一个隔离的临时 `HOME`,只启用官方 bundled `computer-use`,再通过 `codex app-server` 走同一条 signed host 路径。+- 否则很容易把本地开源实现和官方闭源实现混在一起,得出错误结论。++在这个前提下,对 `1.0.750` 做同线程 tool call,并同时抓 `SkyComputerUseService` 日志与 `CGWindowList` 后,结论可以收成下面这张表:++| Tool | 2026-04-20 结果 | 观察到的直接证据 |+| --- | --- | --- |+| `set_value` | 会触发 | 命中 `Prepare to interact with element ...`、`Move to location ...`、`Move cursor to ...`;对 `Activity Monitor` 搜索框样本还能看到 `Start Bezier cursor animation ...` |+| `scroll` | 会触发 | 命中 `Prepare to interact with element 1`、`Move cursor to ...`、`Start Bezier cursor animation ...`、`Moving mouse to ...` |+| `drag` | 会触发 | 命中 `Move cursor to ...`、`Start Bezier cursor animation ...`、`Moving mouse to ...`、`Dragging from ... to ...` |+| `perform_secondary_action` | 会触发 | 对 `TextEdit` window 的 `Raise` 样本命中 `Prepare to interact with element 0` 和 `Move cursor to ...` |+| `click` | 分路径 | 坐标点击会命中 `Move cursor to ...`、`Start Bezier cursor animation ...`、`Moving mouse to ...`、`Clicking at ...`;但 element-scoped `click(element_index)` 不一定会 |+| `type_text` | 这轮未触发 | 前台 `TextEdit`、前台 `Activity Monitor`、以及“Finder 在前台但向后台 `TextEdit` 投递文字”三组样本里,都没看到 `Computer Use Cursor` 日志,也没枚举到 `Software Cursor` window |+| `press_key` | 这轮未触发 | 前台 `TextEdit` 的 `Return`,以及“Finder 在前台但向后台 `TextEdit` 投递 `Return`”两组样本里,都没看到 cursor motion 日志 |+| `get_app_state` | 未触发 | 没有看到 `Computer Use Cursor` 相关日志 |+| `list_apps` | 未单独重放,但静态上不支持它会触发 | 当前没有任何与 cursor motion 相关的运行证据 |++其中有三条更关键的收敛:++- `set_value` 确认不是纯“AX 直接赋值然后结束”这么简单。至少在部分控件上,它会先走一段 element preparation,再把软件光标移动到目标附近。+- `type_text` 和 `press_key` 当前更像纯键盘注入路径。尤其在“目标 app 不在前台”的样本里,`TextEdit` 文本内容确实被改动了,但整个时间窗里仍然没有 `Software Cursor` window,也没有任何 `Computer Use Cursor` motion 日志。+- `click` 需要拆成两类看:坐标点击更像“真实鼠标模拟”,而 element-scoped `click` 有时会走更偏 Accessibility action 的路径,因此不一定显出 overlay。++这一点和二进制里的 feature flag 也对得上:++- `feature/computerUseCursor`+- `feature/computerUseAlwaysSimulateClick`+- `Prefer simulating physical clicks over Accessibility actions.`++也就是说,官方实现更像是“按这次 interaction 最终走的是哪条执行链”来决定是否出 overlay,而不是简单按 public tool 名字一刀切。+## 当前推断### 1. 黄色小鼠标是软件光标,不是系统硬件光标diff --git a/docs/references/codex-computer-use-reverse-engineering/software-cursor-slider-parameter-investigation.md b/docs/references/codex-computer-use-reverse-engineering/software-cursor-slider-parameter-investigation.mdnew file mode 100644index 0000000..9c554a2--- /dev/null+++ b/docs/references/codex-computer-use-reverse-engineering/software-cursor-slider-parameter-investigation.md@@ -0,0 +1,272 @@+# Software Cursor Slider Parameter Investigation++这份文档专门回答一个更聚焦的问题:++- 视频里看到的 5 个 slider (`START HANDLE`、`END HANDLE`、`ARC SIZE`、`ARC FLOW`、`SPRING`) 在当前 shipping 的 `Codex Computer Use.app` 里有没有直接证据?+- 如果 shipping bundle 里没有这组调试 UI 文案,那么它们更接近哪些已经 binary-confirmed 的几何 / timing 量?+- 这些量一旦变化,实际会把曲线的哪一段拉长、收紧或后移?++## 结论先写在前面++当前最稳的结论有 4 条:++1. 在 shipping bundle 中,没有扫到 `START HANDLE`、`END HANDLE`、`ARC SIZE`、`ARC FLOW` 这 4 个完整 slider phrase。+2. `SPRING` / `DEBUG` / `MAIL` / `CLICK` 这类单词在 shipping bundle 里能扫到,但它们都是高度歧义的 token,不能据此宣称“视频里的 debug UI 还保留在 release app 里”。+3. 虽然 slider 文案没有直接出现在 release bundle 中,但 binary 里明确保留了对应的 motion 结构:+ - `CursorMotionPath.startControl`+ - `CursorMotionPath.arc`+ - `CursorMotionPath.arcIn`+ - `CursorMotionPath.arcOut`+ - `CursorMotionPath.endControl`+ - `Animation.SpringParameters(response, dampingFraction)`+4. 目前最合理的说法是:+ - 视频里的 slider 更像是内部调试构建对这些底层几何 / timing 量做的一层调参 UI。+ - shipping binary 里保留的是“固定常量 + 候选表 + 分段逻辑”,不是同名 slider label。++## 证据分层++### 1. shipping bundle phrase scan++对本机 shipping bundle++`~/.codex/plugins/cache/openai-bundled/computer-use/1.0.750/Codex Computer Use.app`++做整包字节扫描后,当前结果是:++- `START HANDLE`: 未命中+- `END HANDLE`: 未命中+- `ARC SIZE`: 未命中+- `ARC FLOW`: 未命中+- `SPRING`: 命中,但属于歧义单词+- `DEBUG`: 命中,但属于歧义单词+- `MAIL`: 命中,但属于歧义单词+- `CLICK`: 命中,但属于歧义单词++这里最关键的是前四项。它们作为完整 phrase 没出现在 shipping bundle 里,所以不能把“release binary 里存在 slider label”当成既成事实。++### 2. motion struct / timing struct 证据++虽然 phrase 没命中,但 `SkyComputerUseService` 的 Swift metadata 和字符串里已经能直接恢复出这批 motion 结构:++- `CursorMotionPath`+ - `start`+ - `end`+ - `startControl`+ - `arc`+ - `arcIn`+ - `arcOut`+ - `endControl`+ - `segments`+- `CursorMotionPathMeasurement`+ - `length`+ - `angleChangeEnergy`+ - `maxAngleChange`+ - `totalTurn`+ - `staysInBounds`+- `Animation.SpringParameters`+ - `response`+ - `dampingFraction`++因此“底层曲线确实可拆成 handle / arc / spring 这些量”已经是 binary-backed 结论;当前没坐实的,是“内部调试 UI 上的 5 个 slider 与这些字段之间的一一映射关系”。++## 当前最合理的 slider 映射++下面这层仍然带 inference,但每一项都尽量只挂到已经确认的 binary 量。++### `START HANDLE`++当前最接近:++- `CursorMotionPath.startControl`+- candidate builder 中的 `startExtent`++在当前 binary-lift 里,`startExtent` 来自一条 piecewise:++```text+48+distance * 0.41960295031576633+640+```++再叠加 bounds clipping 后,写到 `startControl`。++直观影响:++- 主要改变起步阶段“先顺车头方向甩出去多远”。+- handle 更大时,曲线前段更长、更晚才往目标收。+- handle 更小时,起步更快回咬主轴。++### `END HANDLE`++当前最接近:++- `CursorMotionPath.endControl`+- candidate builder 中的 `endExtent`++它控制终点前那一段导向量拉多长,直观上更像“刹车和收尾的手柄长度”。++直观影响:++- 更大时,末段更容易拉出更长的收束钩子。+- 更小时,末段更早贴回目标。++但这项特别容易被 bounds clipping 吃掉,所以某些样例里会看起来“几乎没变”。++### `ARC SIZE`++当前最接近:++- `handleExtent`+- `arcExtent`+- `tableA`+- `tableB`++其中已确认的主尺度是:++```text+handleExtent = piecewise(distance * 0.2765523188064277)+arcExtent = clamp(distance * 0.5783555327868779, 38, 440)+tableA = [0.55, 0.8, 1.05]+tableB = [0.65, 1.0, 1.35]+```++直观影响:++- 更大时,arched family 的 apex 离 chord 更远,曲线更宽、更弯、长度更长。+- 更小时,arched family 更像收紧后的椭圆或浅弧。++### `ARC FLOW`++当前没有恢复出一个独立的 `flow` 字段。++shipping binary 里最接近的固定量是:++```text+arcAnchorBias = guide * (startExtent * 0.65)+```++也就是 arc anchor 会被往 guide 方向推一段,而不是严格落在 chord midpoint 上。++因此当前最保守的说法是:++- `ARC FLOW` 更像在调“apex 沿路径前后偏移多少”,而不是单纯调“弧有多大”。+- 它主要改变的是“最宽的那一段出现在路径更前还是更后”。++这条判断比 `START/END HANDLE` 和 `ARC SIZE` 更弱,因为 shipping binary 当前没有恢复出一个明确叫 `flow` 的独立字段。++### `SPRING`++这项是目前 5 个里 binary 证据最直接的。++cursor move 的 timing 链已经直接确认会走:++- `Animation.SpringParameters(response=1.4, dampingFraction=0.9)`+- `Animation.VelocityVerletSimulation.Configuration(dt=1/240, idleVelocityThreshold=28800)`++所以至少可以确认:++- “spring 确实是 cursor motion 的一等 timing 输入”。+- shipping 默认档的 release 常量就是 `response=1.4`、`dampingFraction=0.9`。++当前没恢复的是:++- 内部 debug UI 那个单一 `SPRING` slider,究竟怎么映射成 `response/dampingFraction` 这对值。++动画库附近还有一个 `0x1005879a4` 的 piecewise remap helper,但当前证据仍指向“cursor move 主链直接用 1.4 / 0.9”,而不是先经过那条 helper。++## 对实际曲线的影响++下面是对两个样例做 `slider-study` 后,最值得记的几条行为结论。++### 样例 A:lab 默认点位++输入:++```text+start = (220, 440)+end = (860, 260)+bounds = (0, 0, 1120, 760)+```++baseline 选中的仍然是 `base-scaled-guide`,不是 arched family。++因此:++- `START HANDLE`+ - `-25%` 时,chosen path 长度约 `744.1`,最大离 chord 偏移约 `44.6`+ - `+25%` 时,chosen path 长度约 `776.9`,最大离 chord 偏移约 `49.8`+ - 说明它直接影响当前可见主路径+- `END HANDLE`+ - 这组样例里几乎没变+ - 原因不是“binary 里没有 end handle”,而是 `endControl` 已经被 bounds clipping 钉住+- `ARC SIZE`+ - 当前 chosen path 仍然没变+ - 但 best arched candidate 的离 chord 偏移会从约 `90.1` 涨到约 `152.9`+ - 说明这项当前更像在改 arched family 的竞争力和宽度+- `ARC FLOW`+ - 当前 chosen path 仍然没变+ - 但 best arched candidate 的 apex progress 会从约 `0.558` 挪到约 `0.863`+ - 说明它主要在改“最宽的弧出现在更前还是更后”+- `SPRING`+ - baseline endpoint-lock 约 `1.4291667s`+ - `response -15%` 时,endpoint-lock 提前到约 `1.225s`+ - `response +15%` 时,endpoint-lock 延后到约 `1.6375s`++### 样例 B:更居中的 end-handle 样例++输入:++```text+start = (240, 420)+end = (760, 360)+bounds = (0, 0, 1280, 900)+```++这组更能看出 `END HANDLE` 的实际作用:++- `end_extent -25%`+ - chosen path 长度约 `632.4`+ - 最大离 chord 偏移约 `55.2`+ - `endControl ≈ (899.55, 177.60)`+- `end_extent +25%`+ - chosen path 长度约 `666.4`+ - 最大离 chord 偏移约 `75.1`+ - `endControl ≈ (939.02, 126.0)`++也就是说:++- end-handle 不是“没参与 shipping binary”+- 而是它的可见影响非常依赖当前路径是否先被 bounds clip 住++## 当前最稳的边界++可以直接说的:++- shipping binary 里没有 `START HANDLE` / `END HANDLE` / `ARC SIZE` / `ARC FLOW` 这些完整 label phrase。+- shipping binary 里明确存在 `startControl / arc / arcIn / arcOut / endControl / SpringParameters(response,dampingFraction)`。+- `SPRING` 对 timing 的参与是 direct binary evidence。+- `START HANDLE` / `END HANDLE` / `ARC SIZE` / `ARC FLOW` 当前更像是对 release builder 中固定几何量的一层调试 UI 映射。++还不能直接说死的:++- 5 个 slider 与 shipping binary 内部字段之间已经一一对上。+- `ARC FLOW` 已经恢复到一个独立字段。+- 单一 `SPRING` slider 的 release 映射已经确认一定经过 `0x1005879a4`。++## 可重复命令++查看当前 binary 的 motion 结构与常量:++```bash+python3 scripts/cursor-motion-re/reconstruct_cursor_motion.py inspect --pretty+```++查看 slider 敏感性分析:++```bash+python3 scripts/cursor-motion-re/reconstruct_cursor_motion.py slider-study \+ --start 220 440 \+ --end 860 260 \+ --bounds 0 0 1120 760 \+ --pretty+```diff --git a/experiments/StandaloneCursorLab/README.md b/experiments/StandaloneCursorLab/README.mdindex ccc1dad..c5efa70 100644--- a/experiments/StandaloneCursorLab/README.md+++ b/experiments/StandaloneCursorLab/README.md@@ -39,6 +39,11 @@ swift run StandaloneCursorLab- 点击画布任意位置,先预览当前 heading-driven candidate 族,再自动选一路径并驱动 cursor 过去。- 左上角保留 `START HANDLE`、`END HANDLE`、`ARC SIZE`、`ARC FLOW`、`SPRING` 5 个 slider,面板本身不再附带 `REPLAY` / `RESET` 按钮或额外指标文案,便于直接对照当前轨迹和画面观感。+- slider 调参时会重算当前 session 的整条 reference path,同时保持 cursor 本体停在当前位置;因此 `DEBUG` 开着时,settled 后继续拖 slider 也还能看到完整曲线反馈,不会退化成零长度路径。+- `START HANDLE` 现在优先改变起步段的 guide / reach / normal 偏置,`END HANDLE` 则优先改变收尾段的 guide / reach / normal 偏置;两者不再只是一起放大整条曲线。+- `ARC SIZE` 现在明确表示轨迹弧度本身,不是 cursor glyph 的尺寸;它会同时改变弧高/控制点侧向偏移,以及 chooser 对更直路径和更宽弧线路径的偏好。+- `ARC FLOW` 现在明确表示“最宽弧段沿 start→end 主轴更靠前还是更靠后”;它不负责把弧变大,而是优先改单段 cubic 控制点的前后相位偏置。+- `SPRING` 现在明确表示 progress spring 本身的快慢与阻尼,不再叠加额外 distance-based duration fudge;`0.5` 档会精确回到官方 `response=1.4`、`damping=0.9`、`343/240` endpoint-lock 时间,往左更快,往右更慢。- debug overlay 会显示控制点、arc handle 和当前选中的 candidate id / score。- 关闭 `DEBUG` 后不会展示任何轨迹线或目标点,只保留 cursor 本体,便于单独观察最终运动观感。- lab 主线不再直接复用 raw binary lift 的 `20` 条 candidate + score;当前改为 reverse-engineering 约束下的 heading-driven chooser,把起始朝向和最终 resting pose 一起喂给路径选择器,让默认曲线更稳定收敛到单侧 C 形或近直线。@@ -54,7 +59,10 @@ swift run StandaloneCursorLab- 不要再把未验证的 slider 参数语义伪装成“官方实现”。- slider 可以作为本地调参入口保留,但要明确它们是 heading-driven lab 的测试旋钮,不是已经 binary-confirmed 的一一字段映射。-- `SPRING` slider 可以改变 spring 本身的 response / damping 与对应 settle 时间,但默认档应继续保持官方 `1.4291667s` endpoint-lock 节奏。+- 为了避免默认样例里 `END HANDLE` 被过紧的 corridor bounds 提前裁掉,lab 现在直接以画布内缩后的实际 canvas bounds 做选路和 control clipping。+- 当前 `ARC SIZE` 的实现边界是“局部 heading-driven path 的弧高和 arched family 倾向”;它不是在调 cursor 资源尺寸,也不宣称已经和 release binary 的 `tableA/tableB/arcExtent` 一一对上。+- 当前 `ARC FLOW` 的实现边界是“单段 cubic 的前后相位偏置”;它更接近 reverse-engineering 里 `arcAnchorBias` 这类几何前后偏置,不宣称已经和 release binary 内部某个独立 `flow` 字段一一对应。+- 当前 `SPRING` 的实现边界是“围绕官方 `1.4 / 0.9` 的 centered spring remap”;它直接改变 progress spring 的 `response / damping` 与 endpoint-lock 时间,但不宣称已经恢复出 release app 内部 debug slider 的精确 remap helper。- 路径层、progress 层和 visible pose 层继续保持分离。- 没有真实 target window 的场景里,要明确区分 `StandaloneCursor` 的 raw reverse-engineered pool 和 `StandaloneCursorLab` 的 heading-driven 主线实现。- demo host 可以替换,但 motion model 和 visual dynamics 应保持可单独复用。diff --git a/experiments/StandaloneCursorLab/Sources/StandaloneCursorLab/CursorLabRootView.swift b/experiments/StandaloneCursorLab/Sources/StandaloneCursorLab/CursorLabRootView.swiftindex 03d3af9..1e87733 100644--- a/experiments/StandaloneCursorLab/Sources/StandaloneCursorLab/CursorLabRootView.swift+++ b/experiments/StandaloneCursorLab/Sources/StandaloneCursorLab/CursorLabRootView.swift@@ -52,7 +52,7 @@ struct CursorLabRootView: View {model.queueMove(to: newValue)}.onChange(of: motionParameters) { _, newValue in- model.updateParameters(newValue, start: start, end: end)+ model.updateParameters(newValue)}}.overlay(alignment: .topLeading) {@@ -133,6 +133,8 @@ final class CursorLabViewModel: ObservableObject {private var lastTimestamp: CFTimeInterval?private var previewRemaining: CGFloat = 0private var queuedTarget = CGPoint(x: 860, y: 260)+ private var pathReferenceOrigin = CGPoint(x: 220, y: 440)+ private var pathReferenceStartRotation = CursorGlyphCalibration.restingRotationprivate var canvasBounds = CGRect(x: 0, y: 0, width: 1080, height: 720)private var motionParameters = CursorMotionParameters.default@@ -172,6 +174,8 @@ final class CursorLabViewModel: ObservableObject {motionParameters = parameterssimulator.updateParameters(parameters)queuedTarget = end+ pathReferenceOrigin = start+ pathReferenceStartRotation = CursorGlyphCalibration.restingRotationcanvasBounds = CGRect(origin: .zero, size: canvasSize)let selection = bestSelection(from: start,@@ -192,15 +196,14 @@ final class CursorLabViewModel: ObservableObject {canvasBounds = CGRect(origin: .zero, size: canvasSize)}- func updateParameters(_ parameters: CursorMotionParameters, start: CGPoint, end: CGPoint) {+ func updateParameters(_ parameters: CursorMotionParameters) {motionParameters = parameterssimulator.updateParameters(parameters)- queuedTarget = endlet selection = bestSelection(- from: start,- to: end,- startRotation: CursorGlyphCalibration.restingRotation+ from: pathReferenceOrigin,+ to: queuedTarget,+ startRotation: pathReferenceStartRotation)candidates = selection.candidatesselectedCandidateID = selection.selected.id@@ -212,10 +215,13 @@ final class CursorLabViewModel: ObservableObject {}func updateStart(_ value: CGPoint) {+ let startRotation = currentState.rotation+ pathReferenceOrigin = value+ pathReferenceStartRotation = startRotationlet selection = bestSelection(from: value,to: queuedTarget,- startRotation: currentState.rotation+ startRotation: startRotation)candidates = selection.candidatesselectedCandidateID = selection.selected.id@@ -235,6 +241,8 @@ final class CursorLabViewModel: ObservableObject {private func scheduleMove(from origin: CGPoint, to target: CGPoint, snapOrigin: Bool) {queuedTarget = targetlet startRotation = snapOrigin ? CursorGlyphCalibration.restingRotation : currentState.rotation+ pathReferenceOrigin = origin+ pathReferenceStartRotation = startRotationlet selection = bestSelection(from: origin, to: target, startRotation: startRotation)candidates = selection.candidates@@ -317,7 +325,7 @@ final class CursorLabViewModel: ObservableObject {to end: CGPoint,startRotation: CGFloat) -> (candidates: [CursorMotionCandidate], selected: CursorMotionCandidate) {- let selectionBounds = syntheticMotionBounds(from: start, to: end)+ let selectionBounds = motionBounds()let candidates = HeadingDrivenCursorMotionModel.makeCandidates(start: start,end: end,@@ -341,26 +349,17 @@ final class CursorLabViewModel: ObservableObject {return (candidates, defaultCandidate)}- private func syntheticMotionBounds(from start: CGPoint, to end: CGPoint) -> CGRect? {- guard canvasBounds.isNull == false else {+ private func motionBounds() -> CGRect? {+ guard canvasBounds.isNull == false, canvasBounds.isEmpty == false else {return nil}- let distance = hypot(end.x - start.x, end.y - start.y)- let margin = min(max(distance * 0.14, 90), 180)- let corridorBounds = CGRect(- x: min(start.x, end.x),- y: min(start.y, end.y),- width: abs(end.x - start.x),- height: abs(end.y - start.y)- ).insetBy(dx: -margin, dy: -margin)-- let resolved = canvasBounds.intersection(corridorBounds)- if resolved.isNull || resolved.isEmpty {+ let paddedBounds = canvasBounds.insetBy(dx: 24, dy: 24)+ if paddedBounds.isNull || paddedBounds.isEmpty {return canvasBounds}- return resolved+ return paddedBounds}private func headingVector(rotation: CGFloat) -> CGVector {@@ -460,11 +459,6 @@ private struct CursorLabCanvas: View {context.fill(Path(ellipseIn: rect), with: .color(CursorLabPalette.ink.opacity(0.40)))}- if let arc = model.path.arc {- let rect = CGRect(x: arc.x - 5, y: arc.y - 5, width: 10, height: 10)- context.fill(Path(ellipseIn: rect), with: .color(CursorLabPalette.mainStrong.opacity(0.78)))- }-if let selectedCandidate = model.candidates.first(where: { $0.id == model.selectedCandidateID }) {let labelPoint = CGPoint(x: selectedCandidate.path.point(at: 0.54).x + 18,diff --git a/experiments/StandaloneCursorLab/Sources/StandaloneCursorLab/CursorMotionModel.swift b/experiments/StandaloneCursorLab/Sources/StandaloneCursorLab/CursorMotionModel.swiftindex d64bef4..d36f10f 100644--- a/experiments/StandaloneCursorLab/Sources/StandaloneCursorLab/CursorMotionModel.swift+++ b/experiments/StandaloneCursorLab/Sources/StandaloneCursorLab/CursorMotionModel.swift@@ -2,6 +2,8 @@ import CoreGraphicsimport Foundationstruct CursorMotionParameters: Equatable {+ private static let springBaselineEpsilon: CGFloat = 0.0001+var startHandle: CGFloatvar endHandle: CGFloatvar arcSize: CGFloat@@ -31,9 +33,15 @@ struct CursorMotionParameters: Equatable {}var progressSpringConfiguration: CursorMotionSpringConfiguration {- CursorMotionSpringConfiguration(- response: 0.96 + (0.88 * spring),- dampingFraction: 0.98 - (0.16 * spring)+ if abs(spring - Self.default.spring) <= Self.springBaselineEpsilon {+ return .official+ }++ let tuning = centeredSpringTuning+ return CursorMotionSpringConfiguration(+ response: CursorMotionSpringConfiguration.official.response * (1 + (tuning * 0.30)),+ dampingFraction: (CursorMotionSpringConfiguration.official.dampingFraction + (tuning * 0.04))+ .clamped(to: 0.86...0.94))}@@ -45,6 +53,17 @@ struct CursorMotionParameters: Equatable {// not by an extra distance-based wall-clock scaling layer.CursorMotionProgressAnimator.closeEnoughTime(configuration: progressSpringConfiguration)}++ private var centeredSpringTuning: CGFloat {+ let baseline = Self.default.spring+ if spring >= baseline {+ let upperSpan = max(1 - baseline, Self.springBaselineEpsilon)+ return (spring - baseline) / upperSpan+ }++ let lowerSpan = max(baseline, Self.springBaselineEpsilon)+ return (spring - baseline) / lowerSpan+ }}struct CursorMotionSegment: Equatable {@@ -675,7 +694,11 @@ enum HeadingDrivenCursorMotionModel {metrics: metrics,startForward: resolvedStartForward,endForward: resolvedEndForward,- preferredSide: preferredSide+ preferredSide: preferredSide,+ arcSizeTuning: signedSliderOffset(+ parameters.arcSize,+ baseline: CursorMotionParameters.default.arcSize+ ))return descriptors(for: metrics, preferredSide: preferredSide).map { descriptor in@@ -730,14 +753,44 @@ enum HeadingDrivenCursorMotionModel {let direction = metrics.directionlet normal = metrics.normallet resolvedFlow = (parameters.arcFlow + descriptor.flowShift).clamped(to: 0...1)- let flowBias = (resolvedFlow - 0.5) * distance * 0.18+ let resolvedDefaultFlow = (CursorMotionParameters.default.arcFlow + descriptor.flowShift).clamped(to: 0...1)+ let flowPhaseTuning = signedSliderOffset(+ resolvedFlow,+ baseline: resolvedDefaultFlow+ )+ let flowBias = flowPhaseTuning * distance * 0.18+ let startHandleTuning = signedSliderOffset(+ parameters.startHandle,+ baseline: CursorMotionParameters.default.startHandle+ )+ let endHandleTuning = signedSliderOffset(+ parameters.endHandle,+ baseline: CursorMotionParameters.default.endHandle+ )+ let arcSizeTuning = signedSliderOffset(+ parameters.arcSize,+ baseline: CursorMotionParameters.default.arcSize+ )+ let arcGeometryScale = tunedArcScale(+ arcSizeTuning,+ delta: descriptor.family == "direct" ? 0.32 : 0.82+ )+ let flowStartNormalBiasDelta = descriptor.family == "direct" ? -0.04 : -0.14+ let flowEndNormalBiasDelta = descriptor.family == "direct" ? 0.04 : 0.16+ let flowNormalScaleDelta = descriptor.family == "direct" ? 0.24 : 0.78+ let startFlowChordBias = direction.scaled(+ by: distance * flowPhaseTuning * (descriptor.family == "direct" ? -0.04 : -0.12)+ )+ let endFlowChordBias = direction.scaled(+ by: distance * flowPhaseTuning * (descriptor.family == "direct" ? 0.05 : 0.16)+ )let baseStartReach = distance * (0.10 + parameters.startHandle * 0.56)let baseEndReach = distance * (0.11 + parameters.endHandle * 0.62)let distanceLift = 0.68 + (metrics.farFactor * 0.56)let baseArcHeight = min(- max(distance * (0.10 + parameters.arcSize * 0.92) * descriptor.arcScale * distanceLift, 20),- distance * 0.96+ max(distance * (0.10 + parameters.arcSize * 0.92) * descriptor.arcScale * arcGeometryScale * distanceLift, 12),+ distance * (0.84 + max(arcSizeTuning, 0) * 0.18))let sideSign = CGFloat(descriptor.side)@@ -751,30 +804,104 @@ enum HeadingDrivenCursorMotionModel {forward: startForward,normal: normal,sideSign: sideSign,- lineWeight: descriptor.startLineWeight,- headingWeight: descriptor.startHeadingWeight,- normalBias: descriptor.startGuideNormalBias+ lineWeight: tunedHandleComponent(+ descriptor.startLineWeight,+ tuning: startHandleTuning,+ delta: -0.22,+ range: -0.56...1.30+ ),+ headingWeight: tunedHandleComponent(+ descriptor.startHeadingWeight,+ tuning: startHandleTuning,+ delta: 0.52,+ range: 0.04...2.24+ ),+ normalBias: tunedHandleComponent(+ tunedFlowComponent(+ tunedArcComponent(+ descriptor.startGuideNormalBias,+ tuning: arcSizeTuning,+ delta: descriptor.family == "direct" ? 0.04 : 0.12,+ range: -0.24...0.74+ ),+ tuning: flowPhaseTuning,+ delta: flowStartNormalBiasDelta,+ range: -0.32...0.82+ ),+ tuning: startHandleTuning,+ delta: 0.20,+ range: -0.20...0.82+ ))let endGuide = resolvedGuide(line: direction,forward: endForward,normal: normal,sideSign: sideSign,- lineWeight: descriptor.endLineWeight,- headingWeight: descriptor.endHeadingWeight,- normalBias: descriptor.endGuideNormalBias+ lineWeight: tunedHandleComponent(+ descriptor.endLineWeight,+ tuning: endHandleTuning,+ delta: -0.18,+ range: -0.48...1.16+ ),+ headingWeight: tunedHandleComponent(+ descriptor.endHeadingWeight,+ tuning: endHandleTuning,+ delta: 0.48,+ range: 0.08...2.28+ ),+ normalBias: tunedHandleComponent(+ tunedFlowComponent(+ tunedArcComponent(+ descriptor.endGuideNormalBias,+ tuning: arcSizeTuning,+ delta: descriptor.family == "direct" ? 0.05 : 0.14,+ range: -0.20...0.82+ ),+ tuning: flowPhaseTuning,+ delta: flowEndNormalBiasDelta,+ range: -0.28...0.90+ ),+ tuning: endHandleTuning,+ delta: 0.22,+ range: -0.16...0.90+ ))- let startReach = max(baseStartReach * descriptor.startReachScale + flowBias * descriptor.startFlowWeight, 12)- let endReach = max(baseEndReach * descriptor.endReachScale - flowBias * descriptor.endFlowWeight, 12)- let control1Base = start + startGuide.scaled(by: startReach)- let control2Base = end - endGuide.scaled(by: endReach)-- let control1 = control1Base + arcVector.scaled(by: descriptor.startNormalScale)- let control2 = control2Base + arcVector.scaled(by: descriptor.endNormalScale)+ let startReach = max(+ (baseStartReach+ * descriptor.startReachScale+ * tunedHandleScale(startHandleTuning, delta: 0.58))+ + flowBias * descriptor.startFlowWeight,+ 12+ )+ let endReach = max(+ (baseEndReach+ * descriptor.endReachScale+ * tunedHandleScale(endHandleTuning, delta: 0.68))+ - flowBias * descriptor.endFlowWeight,+ 12+ )+ let control1Base = start+ + startGuide.scaled(by: startReach)+ + startFlowChordBias+ let control2Base = end+ - endGuide.scaled(by: endReach)+ + endFlowChordBias+ let startNormalScale = descriptor.startNormalScale+ * tunedHandleScale(startHandleTuning, delta: 0.40)+ * tunedArcScale(arcSizeTuning, delta: descriptor.family == "direct" ? 0.46 : 0.94)+ * tunedFlowScale(flowPhaseTuning, direction: -1, delta: flowNormalScaleDelta)+ let endNormalScale = descriptor.endNormalScale+ * tunedHandleScale(endHandleTuning, delta: 0.44)+ * tunedArcScale(arcSizeTuning, delta: descriptor.family == "direct" ? 0.46 : 0.94)+ * tunedFlowScale(flowPhaseTuning, direction: 1, delta: flowNormalScaleDelta)++ let control1 = control1Base + arcVector.scaled(by: startNormalScale)+ let control2 = control2Base + arcVector.scaled(by: endNormalScale)let resolvedArcHeight = baseArcHeight * max(- abs(descriptor.startNormalScale),- abs(descriptor.endNormalScale),+ abs(startNormalScale),+ abs(endNormalScale),0.12)@@ -855,12 +982,20 @@ enum HeadingDrivenCursorMotionModel {switch descriptor.family {case "turn":score += (1 - context.turnDemand) * 55+ score += max(-context.arcSizeTuning, 0) * 48+ score -= max(context.arcSizeTuning, 0) * 24case "brake":score += (1 - context.arrivalDemand) * 40+ score += max(-context.arcSizeTuning, 0) * 42+ score -= max(context.arcSizeTuning, 0) * 20case "orbit":score += context.directness * 70+ score += max(-context.arcSizeTuning, 0) * 68+ score -= max(context.arcSizeTuning, 0) * 40case "direct":score += max(context.turnDemand - 0.12, 0) * 80+ score += max(context.arcSizeTuning, 0) * 110+ score -= max(-context.arcSizeTuning, 0) * 44default:break}@@ -1124,6 +1259,59 @@ enum HeadingDrivenCursorMotionModel {return CGVector(dx: vector.dx / length, dy: vector.dy / length)}+ private static func signedSliderOffset(_ value: CGFloat, baseline: CGFloat) -> CGFloat {+ if value >= baseline {+ let upperSpan = max(1 - baseline, normalizationEpsilon)+ return (value - baseline) / upperSpan+ }++ let lowerSpan = max(baseline, normalizationEpsilon)+ return (value - baseline) / lowerSpan+ }++ private static func tunedHandleComponent(+ _ base: CGFloat,+ tuning: CGFloat,+ delta: CGFloat,+ range: ClosedRange<CGFloat>+ ) -> CGFloat {+ (base + (tuning * delta)).clamped(to: range)+ }++ private static func tunedHandleScale(_ tuning: CGFloat, delta: CGFloat) -> CGFloat {+ max(0.28, 1 + (tuning * delta))+ }++ private static func tunedArcComponent(+ _ base: CGFloat,+ tuning: CGFloat,+ delta: CGFloat,+ range: ClosedRange<CGFloat>+ ) -> CGFloat {+ (base + (tuning * delta)).clamped(to: range)+ }++ private static func tunedArcScale(_ tuning: CGFloat, delta: CGFloat) -> CGFloat {+ max(0.16, 1 + (tuning * delta))+ }++ private static func tunedFlowComponent(+ _ base: CGFloat,+ tuning: CGFloat,+ delta: CGFloat,+ range: ClosedRange<CGFloat>+ ) -> CGFloat {+ (base + (tuning * delta)).clamped(to: range)+ }++ private static func tunedFlowScale(+ _ tuning: CGFloat,+ direction: CGFloat,+ delta: CGFloat+ ) -> CGFloat {+ max(0.18, 1 + (tuning * direction * delta))+ }+private static func signedAngle(from lhs: CGVector, to rhs: CGVector) -> CGFloat {atan2((lhs.dx * rhs.dy) - (lhs.dy * rhs.dx), (lhs.dx * rhs.dx) + (lhs.dy * rhs.dy))}@@ -1133,6 +1321,7 @@ enum HeadingDrivenCursorMotionModel {let startForward: CGVectorlet endForward: CGVectorlet preferredSide: Int+ let arcSizeTuning: CGFloatvar turnDemand: CGFloat {min(abs(HeadingDrivenCursorMotionModel.signedAngle(from: startForward, to: metrics.direction)) / .pi, 1)diff --git a/scripts/cursor-motion-re/README.md b/scripts/cursor-motion-re/README.mdindex 2444950..5155862 100644--- a/scripts/cursor-motion-re/README.md+++ b/scripts/cursor-motion-re/README.md@@ -47,6 +47,16 @@ python3 scripts/cursor-motion-re/reconstruct_cursor_motion.py demo \--pretty```+分析视频里那 5 个 slider 在 shipping bundle 中是否还有直接证据,并输出它们和当前 binary-confirmed 几何 / spring 量的敏感性分析:++```bash+python3 scripts/cursor-motion-re/reconstruct_cursor_motion.py slider-study \+ --start 220 440 \+ --end 860 260 \+ --bounds 0 0 1120 760 \+ --pretty+```+## 输出说明- `inspect`:@@ -64,3 +74,7 @@ python3 scripts/cursor-motion-re/reconstruct_cursor_motion.py demo \- 当前仍未完全恢复的是 duration / wall-clock timing、`0x1005934b0` 第二段里几个泛型 buffer 的精确语义命名,以及调用前那层 runtime bounds 自动发现。- `first_endpoint_lock_*` 依赖一个已确认前提:`sample(progress)` 会 clamp 到 `0...1`;但它和 `SpringAnimation` finished optional-return 的最终联动,当前仍按“多段已确认证据拼接出的 inference”标注。- 当前输出的 `speed_units_per_progress` 是几何速度,不是带真实 duration 的时间速度;duration 仍在继续逆向。+- `slider-study`:+ - 先扫描 shipping bundle,确认 `START HANDLE` / `END HANDLE` / `ARC SIZE` / `ARC FLOW` 这些完整 phrase 是否还存在。+ - 再把这 5 个 knob 分别映射到当前已经 binary-confirmed 的 `startControl` / `endControl` / `arc*` / `SpringParameters` 相关量,并输出 baseline 与扰动后的 chosen candidate / best arched candidate / endpoint-lock timing 变化。+ - 输出里会明确区分“release bundle phrase evidence”和“基于 binary-confirmed 几何做的 slider mapping inference”。diff --git a/scripts/cursor-motion-re/official_cursor_motion.py b/scripts/cursor-motion-re/official_cursor_motion.pyindex a513ced..1ffff62 100644--- a/scripts/cursor-motion-re/official_cursor_motion.py+++ b/scripts/cursor-motion-re/official_cursor_motion.py@@ -684,6 +684,14 @@ class CandidatePath:}+@dataclass(frozen=True)+class BinaryGuidedPathOverrides:+ start_extent_scale: float = 1.0+ end_extent_scale: float = 1.0+ arc_size_scale: float = 1.0+ arc_flow_scale: float = 1.0++def _sample_cubic(start: Vec2, control1: Vec2, control2: Vec2, end: Vec2, t: float) -> Vec2:omt = 1.0 - tomt2 = omt * omt@@ -716,9 +724,23 @@ def clamp(value: float, minimum: float, maximum: float) -> float:return max(minimum, min(maximum, value))+def signed_angle(lhs: Vec2, rhs: Vec2) -> float:+ return math.atan2((lhs.x * rhs.y) - (lhs.y * rhs.x), (lhs.x * rhs.x) + (lhs.y * rhs.y))++def build_scalar_spring_configuration() -> ScalarSpringConfiguration:response = CONFIRMED_CURSOR_TIMING_CONSTANTS["cursor_path_spring_response"]damping_fraction = CONFIRMED_CURSOR_TIMING_CONSTANTS["cursor_path_spring_damping_fraction"]+ return build_scalar_spring_configuration_for(+ response=response,+ damping_fraction=damping_fraction,+ )+++def build_scalar_spring_configuration_for(+ response: float,+ damping_fraction: float,+) -> ScalarSpringConfiguration:dt = CONFIRMED_CURSOR_TIMING_CONSTANTS["velocity_verlet_dt"]idle_velocity_threshold = CONFIRMED_CURSOR_TIMING_CONSTANTS["velocity_verlet_idle_velocity_threshold"]@@ -795,10 +817,11 @@ def advance_scalar_velocity_verlet_to_time(def build_timed_cursor_timeline(path: CursorMotionPath,+ config: ScalarSpringConfiguration | None = None,report_every_steps: int = 4,max_duration_seconds: float = 2.0,) -> dict[str, object]:- config = build_scalar_spring_configuration()+ config = config or build_scalar_spring_configuration()current = 0.0target = 1.0state = ScalarVelocityVerletState(time=0.0, velocity=0.0, force=0.0)@@ -954,7 +977,14 @@ class BinaryGuidedPathGenerator:self.tables = tablesself.guide_local = Vec2(-0.6946583704589973, 0.7193398003386512)- def build_candidates(self, start: Vec2, end: Vec2, bounds: Bounds | None) -> list[CandidatePath]:+ def build_candidates(+ self,+ start: Vec2,+ end: Vec2,+ bounds: Bounds | None,+ overrides: BinaryGuidedPathOverrides | None = None,+ ) -> list[CandidatePath]:+ overrides = overrides or BinaryGuidedPathOverrides()delta = end - startdistance = max(delta.length(), self.constants["normalization_epsilon"])direction = delta.normalized()@@ -967,8 +997,8 @@ class BinaryGuidedPathGenerator:constants=self.constants,guide=guide,)- start_extent = min(start_extent_pre, _clip_positive_ray(start, guide, bounds))- end_extent = min(end_extent_pre, _clip_positive_ray(end, reverse_guide, bounds))+ start_extent = min(start_extent_pre * overrides.start_extent_scale, _clip_positive_ray(start, guide, bounds))+ end_extent = min(end_extent_pre * overrides.end_extent_scale, _clip_positive_ray(end, reverse_guide, bounds))start_extent_scaled = min(max(start_extent * self.constants["side_bias_scale"], 0.0),@@ -984,9 +1014,9 @@ class BinaryGuidedPathGenerator:scaled_start_control = start + guide.scale(start_extent_scaled)scaled_end_control = end - guide.scale(end_extent_scaled)- raw_handle_extent = _binary_piecewise_handle_extent(distance, self.constants)+ raw_handle_extent = _binary_piecewise_handle_extent(distance, self.constants) * overrides.arc_size_scaleraw_arc_extent = clamp(- distance * self.constants["distance_scale_tertiary"],+ distance * self.constants["distance_scale_tertiary"] * overrides.arc_size_scale,self.constants["candidate_arc_min"],self.constants["candidate_arc_max"],)@@ -996,7 +1026,7 @@ class BinaryGuidedPathGenerator:cross = (guide.y * direction.x) - (guide.x * direction.y)if cross < 0.0:signed_normal = signed_normal.scale(-1.0)- arc_anchor_bias = guide.scale(start_extent * self.constants["side_bias_scale"])+ arc_anchor_bias = guide.scale(start_extent * self.constants["side_bias_scale"] * overrides.arc_flow_scale)forward_unit = _normalized_or_default(direction.scale(distance) + signed_normal.scale(raw_arc_extent),minimum_length=raw_handle_extent,@@ -1146,6 +1176,320 @@ class BinaryGuidedPathGenerator:)+def _derive_bundle_root(binary_path: Path) -> Path:+ if binary_path.parent.name == "MacOS" and binary_path.parent.parent.name == "Contents":+ return binary_path.parent.parent.parent+ return binary_path.parent+++def scan_bundle_for_slider_labels(binary_path: Path) -> dict[str, object]:+ bundle_root = _derive_bundle_root(binary_path)+ exact_slider_phrases = [+ "START HANDLE",+ "END HANDLE",+ "ARC SIZE",+ "ARC FLOW",+ ]+ ambiguous_single_tokens = [+ "SPRING",+ "DEBUG",+ "MAIL",+ "CLICK",+ ]+ exact_findings = {label: False for label in exact_slider_phrases}+ ambiguous_findings = {label: False for label in ambiguous_single_tokens}+ files_scanned = 0++ for file_path in bundle_root.rglob("*"):+ if not file_path.is_file():+ continue+ files_scanned += 1+ try:+ payload = file_path.read_bytes().lower()+ except OSError:+ continue++ for label in exact_slider_phrases:+ if exact_findings[label]:+ continue+ if label.lower().encode("utf-8") in payload:+ exact_findings[label] = True++ for label in ambiguous_single_tokens:+ if ambiguous_findings[label]:+ continue+ if label.lower().encode("utf-8") in payload:+ ambiguous_findings[label] = True++ return {+ "bundle_root": str(bundle_root),+ "files_scanned": files_scanned,+ "exact_slider_phrases": exact_findings,+ "ambiguous_single_tokens": ambiguous_findings,+ "notes": {+ "exact_slider_phrase_scan": "used to check whether the shipping bundle still carries the specific slider UI labels from the video.",+ "ambiguous_single_token_scan": "single words like SPRING / DEBUG / MAIL / CLICK can appear for unrelated reasons in a release bundle, so these hits are not enough to claim the debug UI still ships.",+ },+ }+++def summarize_path_shape(path: CursorMotionPath, sample_count: int = 128) -> dict[str, float | list[float]]:+ chord = path.end - path.start+ chord_length = chord.length()+ chord_direction = chord.normalized() if chord_length > 1e-9 else Vec2(1.0, 0.0)+ normal = chord_direction.perpendicular()++ max_signed_offset = 0.0+ apex_progress = 0.0+ for sample in path.sample_points(sample_count):+ offset_vector = sample.point - path.start+ signed_offset = (offset_vector.x * normal.x) + (offset_vector.y * normal.y)+ if abs(signed_offset) > abs(max_signed_offset):+ max_signed_offset = signed_offset+ apex_progress = sample.progress++ start_tangent = path.sample(0.04)[1]+ end_tangent = path.sample(0.96)[1]+ return {+ "chord_length": chord_length,+ "max_signed_offset_from_chord": max_signed_offset,+ "max_offset_from_chord": abs(max_signed_offset),+ "apex_progress": apex_progress,+ "start_heading_error_degrees": math.degrees(signed_angle(chord_direction, start_tangent)),+ "end_heading_error_degrees": math.degrees(signed_angle(chord_direction, end_tangent)),+ }+++def summarize_candidate_geometry(candidate: CandidatePath, sample_count: int = 128) -> dict[str, object]:+ return {+ "identifier": candidate.identifier,+ "kind": candidate.kind,+ "side": candidate.side,+ "measurement": candidate.measurement.to_dict(),+ "shape_metrics": summarize_path_shape(candidate.path, sample_count=sample_count),+ "control_points": {+ "start_control": candidate.path.start_control.to_list() if candidate.path.start_control else None,+ "arc": candidate.path.arc.to_list() if candidate.path.arc else None,+ "arc_in": candidate.path.arc_in.to_list() if candidate.path.arc_in else None,+ "arc_out": candidate.path.arc_out.to_list() if candidate.path.arc_out else None,+ "end_control": candidate.path.end_control.to_list() if candidate.path.end_control else None,+ },+ }+++def _find_best_arched_candidate(candidates: list[CandidatePath]) -> CandidatePath | None:+ arched = [candidate for candidate in candidates if candidate.kind == "arched"]+ if not arched:+ return None+ in_bounds = [candidate for candidate in arched if candidate.measurement.stays_in_bounds]+ pool = in_bounds if in_bounds else arched+ return min(pool, key=lambda item: item.score)+++def _build_geometry_slider_variants(+ generator: BinaryGuidedPathGenerator,+ start: Vec2,+ end: Vec2,+ bounds: Bounds | None,+ sample_count: int,+ variant_overrides: list[tuple[str, BinaryGuidedPathOverrides]],+) -> list[dict[str, object]]:+ payload: list[dict[str, object]] = []+ for name, overrides in variant_overrides:+ candidates = generator.build_candidates(start=start, end=end, bounds=bounds, overrides=overrides)+ chosen_candidate, selection_policy = generator.choose_candidate(candidates)+ best_arched = _find_best_arched_candidate(candidates)+ payload.append(+ {+ "variant": name,+ "selection_policy": selection_policy,+ "chosen_candidate": summarize_candidate_geometry(chosen_candidate, sample_count=sample_count)+ if chosen_candidate+ else None,+ "best_arched_candidate": summarize_candidate_geometry(best_arched, sample_count=sample_count)+ if best_arched+ else None,+ }+ )+ return payload+++def build_slider_study_output(+ binary: MachOBinary,+ start: Vec2,+ end: Vec2,+ bounds: Bounds | None,+ sample_count: int,+) -> dict[str, object]:+ constants = build_model_constants(binary)+ tables = extract_candidate_tables(binary)+ generator = BinaryGuidedPathGenerator(constants=constants, tables=tables)+ baseline_candidates = generator.build_candidates(start=start, end=end, bounds=bounds)+ baseline_choice, selection_policy = generator.choose_candidate(baseline_candidates)+ baseline_best_arched = _find_best_arched_candidate(baseline_candidates)++ if baseline_choice is None:+ raise ValueError("no baseline candidate produced for slider study")++ baseline_timeline = build_timed_cursor_timeline(path=baseline_choice.path)+ spring_response_baseline = CONFIRMED_CURSOR_TIMING_CONSTANTS["cursor_path_spring_response"]+ spring_damping_baseline = CONFIRMED_CURSOR_TIMING_CONSTANTS["cursor_path_spring_damping_fraction"]+ spring_variants = [+ (+ "response_minus_15pct",+ build_scalar_spring_configuration_for(+ response=spring_response_baseline * 0.85,+ damping_fraction=spring_damping_baseline,+ ),+ ),+ (+ "response_plus_15pct",+ build_scalar_spring_configuration_for(+ response=spring_response_baseline * 1.15,+ damping_fraction=spring_damping_baseline,+ ),+ ),+ ]+ spring_variant_payload = []+ for name, config in spring_variants:+ timeline = build_timed_cursor_timeline(path=baseline_choice.path, config=config)+ spring_variant_payload.append(+ {+ "variant": name,+ "spring_configuration": config.to_dict(),+ "timed_cursor_timeline": {+ "first_endpoint_lock_time": timeline["first_endpoint_lock_time"],+ "close_enough_first_time": timeline["close_enough_first_time"],+ "raw_progress_first_ge_target_time": timeline["raw_progress_first_ge_target_time"],+ },+ }+ )++ return {+ "binary_path": str(binary.path),+ "input": {+ "start": start.to_list(),+ "end": end.to_list(),+ "bounds": asdict(bounds) if bounds else None,+ "sample_count": sample_count,+ },+ "shipping_bundle_label_evidence": scan_bundle_for_slider_labels(binary.path),+ "binary_confirmed_motion_terms": {+ "path_fields": ["startControl", "arc", "arcIn", "arcOut", "endControl"],+ "timing_fields": ["Animation.SpringParameters.response", "Animation.SpringParameters.dampingFraction"],+ "candidate_tables": extract_candidate_tables(binary),+ "fixed_geometry_constants": {+ "guide_vector": [-0.6946583704589973, 0.7193398003386512],+ "start_extent_piecewise": "48 / d*0.41960295031576633 / 640 depending on region",+ "end_extent_piecewise": "48 / d*0.9 / 640 depending on region",+ "handle_extent_piecewise": "50 / d*0.2765523188064277 / 520",+ "arc_extent_piecewise": "clamp(d*0.5783555327868779, 38, 440)",+ "arc_anchor_bias": "guide * (startExtent * 0.65)",+ },+ "spring_defaults": {+ "response": spring_response_baseline,+ "damping_fraction": spring_damping_baseline,+ "dt": CONFIRMED_CURSOR_TIMING_CONSTANTS["velocity_verlet_dt"],+ },+ },+ "baseline": {+ "selection_policy": selection_policy,+ "chosen_candidate": summarize_candidate_geometry(baseline_choice, sample_count=sample_count),+ "best_arched_candidate": summarize_candidate_geometry(baseline_best_arched, sample_count=sample_count)+ if baseline_best_arched+ else None,+ "timed_cursor_timeline": {+ "spring_configuration": baseline_timeline["spring_configuration"],+ "first_endpoint_lock_time": baseline_timeline["first_endpoint_lock_time"],+ "close_enough_first_time": baseline_timeline["close_enough_first_time"],+ },+ },+ "slider_mapping_analysis": {+ "start_handle": {+ "release_bundle_label_found": False,+ "mapping_confidence": "inferred_from_binary_fields",+ "affects_current_baseline_choice_directly": True,+ "binary_terms_touched": ["CursorMotionPath.startControl", "guide startExtent piecewise"],+ "effect_summary": "changes how far the launch-side control point projects along the guide direction, so it mainly changes the first third of the curve and the initial heading commitment.",+ "variants": _build_geometry_slider_variants(+ generator=generator,+ start=start,+ end=end,+ bounds=bounds,+ sample_count=sample_count,+ variant_overrides=[+ ("start_extent_minus_25pct", BinaryGuidedPathOverrides(start_extent_scale=0.75)),+ ("start_extent_plus_25pct", BinaryGuidedPathOverrides(start_extent_scale=1.25)),+ ],+ ),+ },+ "end_handle": {+ "release_bundle_label_found": False,+ "mapping_confidence": "inferred_from_binary_fields",+ "affects_current_baseline_choice_directly": True,+ "binary_terms_touched": ["CursorMotionPath.endControl", "guide endExtent piecewise"],+ "effect_summary": "changes how far the arrival-side control point projects before the endpoint, so it mainly changes the terminal braking / hook-in segment and the final tangent alignment.",+ "variants": _build_geometry_slider_variants(+ generator=generator,+ start=start,+ end=end,+ bounds=bounds,+ sample_count=sample_count,+ variant_overrides=[+ ("end_extent_minus_25pct", BinaryGuidedPathOverrides(end_extent_scale=0.75)),+ ("end_extent_plus_25pct", BinaryGuidedPathOverrides(end_extent_scale=1.25)),+ ],+ ),+ },+ "arc_size": {+ "release_bundle_label_found": False,+ "mapping_confidence": "inferred_from_binary_fields",+ "affects_current_baseline_choice_directly": baseline_choice.kind == "arched",+ "binary_terms_touched": ["handleExtent piecewise", "arcExtent piecewise", "table_a", "table_b"],+ "effect_summary": "changes how far the arched family pushes the apex off the chord and how wide the arc tangents open, so it mainly changes peak bowing, total turn, and path length.",+ "variants": _build_geometry_slider_variants(+ generator=generator,+ start=start,+ end=end,+ bounds=bounds,+ sample_count=sample_count,+ variant_overrides=[+ ("arc_size_minus_30pct", BinaryGuidedPathOverrides(arc_size_scale=0.7)),+ ("arc_size_plus_30pct", BinaryGuidedPathOverrides(arc_size_scale=1.3)),+ ],+ ),+ },+ "arc_flow": {+ "release_bundle_label_found": False,+ "mapping_confidence": "inferred_from_binary_fields",+ "affects_current_baseline_choice_directly": baseline_choice.kind == "arched",+ "binary_terms_touched": ["arc_anchor_bias = guide * (startExtent * 0.65)"],+ "effect_summary": "the current lift has no dedicated flow field, but the fixed arc-anchor bias shifts the apex forward along the guide direction; perturbing it mainly moves where the curve reaches its widest bow, making the turn feel earlier or later.",+ "variants": _build_geometry_slider_variants(+ generator=generator,+ start=start,+ end=end,+ bounds=bounds,+ sample_count=sample_count,+ variant_overrides=[+ ("arc_flow_minus_25pct", BinaryGuidedPathOverrides(arc_flow_scale=0.75)),+ ("arc_flow_plus_25pct", BinaryGuidedPathOverrides(arc_flow_scale=1.25)),+ ],+ ),+ },+ "spring": {+ "release_bundle_label_found": False,+ "mapping_confidence": "direct_binary_timing_fields_confirmed_but_one_dimensional_ui_mapping_unrecovered",+ "affects_current_baseline_choice_directly": True,+ "binary_terms_touched": ["Animation.SpringParameters.response", "Animation.SpringParameters.dampingFraction"],+ "effect_summary": "shipping cursor motion definitely uses SpringParameters(response=1.4, dampingFraction=0.9); varying response changes endpoint-lock timing and how quickly progress approaches 1.0, but the exact single-slider mapping used by the internal debug UI is still unrecovered.",+ "normalization_helper_note": CONFIRMED_TIMING_PIPELINE["spring_parameter_normalization_helper"],+ "variants": spring_variant_payload,+ },+ },+ }++def build_inspect_output(binary: MachOBinary) -> dict[str, object]:guide_vector = [-0.6946583704589973, 0.7193398003386512]return {@@ -1327,6 +1671,21 @@ def build_parser() -> argparse.ArgumentParser:help="Include full path and sample output for every candidate instead of only the chosen candidate.",)+ slider_study = subparsers.add_parser(+ "slider-study",+ help="Study how slider-like parameter hypotheses map onto binary-confirmed geometry and spring terms.",+ )+ slider_study.add_argument("--pretty", dest="pretty_local", action="store_true", help="Pretty-print JSON output.")+ slider_study.add_argument("--start", nargs=2, metavar=("X", "Y"), required=True, help="Start position.")+ slider_study.add_argument("--end", nargs=2, metavar=("X", "Y"), required=True, help="End position.")+ slider_study.add_argument(+ "--bounds",+ nargs=4,+ metavar=("MIN_X", "MIN_Y", "MAX_X", "MAX_Y"),+ help="Optional bounds used for stays_in_bounds measurement.",+ )+ slider_study.add_argument("--samples", type=int, default=96, help="Number of sample points used for geometry summaries.")+return parser@@ -1349,6 +1708,17 @@ def main(argv: list[str] | None = None) -> int:sample_count=args.samples,include_all_candidates=args.include_all_candidates,)+ elif args.command == "slider-study":+ start = parse_vec2(args.start)+ end = parse_vec2(args.end)+ bounds = parse_bounds(args.bounds) if args.bounds else None+ payload = build_slider_study_output(+ binary,+ start=start,+ end=end,+ bounds=bounds,+ sample_count=args.samples,+ )else:raise AssertionError(f"unsupported command: {args.command}")