👾 Open Computer Use – Open-Source Alternative to Codex Computer Use
Swift accessibilityai-agentai-agentsclaude-codecodexcodex-computer-usecomputer-usecomputer-use-agentcross-platformcuadesktop-automationgemini-cligui-automationmacosmcpmodel-context-protocolopen-computer-useopen-computer-use-mcpopenai-codexopencode
fix(cursor): follow live windows throughout the overlay lifecycle
ifuryst9bc2bd762d4f
diff --git a/apps/OpenComputerUseSmokeSuite/Sources/OpenComputerUseSmokeSuite/main.swift b/apps/OpenComputerUseSmokeSuite/Sources/OpenComputerUseSmokeSuite/main.swiftindex 7c9ab1d..677d935 100644--- a/apps/OpenComputerUseSmokeSuite/Sources/OpenComputerUseSmokeSuite/main.swift+++ b/apps/OpenComputerUseSmokeSuite/Sources/OpenComputerUseSmokeSuite/main.swift@@ -196,8 +196,13 @@ enum OpenComputerUseSmokeSuite {try client.initialize()let tools = try client.listTools()- guard tools.count == 9 else {- throw SmokeError.message("Expected 9 tools, got \(tools.count)")+ let expectedTools: Set<String> = [+ "list_apps", "get_app_state", "query", "click", "perform_secondary_action",+ "scroll", "drag", "type_text", "press_key", "set_value",+ ]+ let toolNames = Set(tools.compactMap { $0["name"] as? String })+ guard tools.count == expectedTools.count, toolNames == expectedTools else {+ throw SmokeError.message("Expected tools \(expectedTools.sorted()), got \(toolNames.sorted())")}print("1. list_apps")diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.mdindex 85ee038..9f72683 100644--- a/docs/ARCHITECTURE.md+++ b/docs/ARCHITECTURE.md@@ -75,6 +75,7 @@- 对真实 app 的 `get_app_state` / action tool 入口,当前只保留一层密码管理器 bundle denylist:bundle-id 直传时直接返回 safety denial;名称匹配时默认不解析到这些 app。终端、Chrome / Atlas 和系统组件不再属于内置阻止目标。- 普通 app 的 element frame 当前按“窗口左上角为原点”的 window-relative 坐标输出,便于后续把 `element_index` 和截图坐标统一到同一套参考系。- `click` / `set_value` 在执行真实动作前后,会额外驱动一层透明 `SoftwareCursorOverlay` window:两者的移动阶段现在共用一条 heading-driven 的官方风格 motion 内核,显式把“当前 cursor 朝向”和“最终 resting pose”一起喂给选路器,优先生成需要时先掉头、再沿车头方向推进的 C 形/单侧大弧轨迹;首次显示时按官方 binary 的 fresh state 从 AppKit 全局 `(0,0)` window origin 生成起点,后续动作继续复用上一帧 visible tip。真正显示出来的 cursor 不再直接等于 path sample,而是经过一层独立的 visual dynamics 状态,把 visible tip、velocity、angle 和 fog/offset 持续推进。`click` 结尾会衔接 click pulse 和更明显但仍然很小的 rotate wobble,`set_value` 则只做 settle / idle,不给 pulse;两者收尾后会在目标点继续保持 idle 状态,等待下一次动作时 tip 保持 anchored、只保留可感知的小角度摆动;只有连续 30 秒没有新动作时才做 cleanup,这样连续 tool call 不会反复从 fresh `(0,0)` 起步;如果宿主在任务 / turn 结束时发出 `turn-ended`,cursor 会立即消失并清掉本轮位置状态。+- overlay 用目标点相对窗口的锚点跟随窗口移动;行进、settle、点击反馈和 idle 都读取最新缓存帧,发现位置变化时直接重定位可见 tip 并清掉旧速度。窗口帧在后台串行读取,最多 30 Hz、同时最多一个请求,避免 AX 阻塞渲染线程;优先读取快照持有的 AX 窗口,仅在没有 AX 元素时使用窗口列表,读取失败保留最后有效帧。只保留当前窗口的追踪状态,reset / idle 超时后释放;这层跟随只调整显示位置,输入投递仍走既有路径。- overlay 的 visual style 不再自己从官方 app bundle 裁 `SoftwareCursor` 小图;主 MCP runtime 现在和 `CursorMotion` 一样优先渲染仓库里沉淀的 `official-software-cursor-window-252.png` baseline,只有资源缺失时才退回 `OpenComputerUseKit` 内部的程序化 pointer/fog fallback。命中点 anchor 仍固定在 `126x126` 画布里的同一组 tip-offset 上;glyph 自身的 neutral heading 继续沿用 `CursorMotion` / 官方 baseline 的 `-3π/4`。主 runtime overlay window 按 AppKit 全局坐标移动,因此在把 AX / `CGWindowList` 产出的 y-down screen-space 点击目标喂给 overlay 之前,会先转换成对应屏幕的 AppKit 全局坐标;路径选路用屏幕上实际可见的 AppKit forward heading,进入 visual dynamics / render state 前则把 velocity 的 y 轴翻回 CursorMotion 的 y-down screen state,再交给 AppKit 绘制层做角度和 `dy` 翻转。程序化 fallback 保留 neutral artwork correction,把它的天然轮廓轴对齐到 `CursorMotion` / 官方 baseline 的 `-3π/4` forward 方向,但不让实验线依赖 runtime 代码。- overlay 的层级不再固定 `.floating`;现在会跟随 snapshot 命中的目标 window id / layer,把自己排到该目标 window 之上,而不是粗暴压到所有前台 app 最上层。- overlay 的曲线路径不再只按固定 Bezier 模板生成;当前主线采用 reverse-engineering 约束下的 heading-driven candidate 族,候选只保留 `direct` / `turn` / `brake` / `orbit` 这些能稳定产出单侧主弧的 family,并继续保留 target-window 命中策略作为同类候选间的 tie-break。原始 binary lift 恢复出来的 `20` 条路径和 score 仍然保留在独立的 `StandaloneCursor` viewer / Python 重建脚本里,用于对照分析,不再直接作为 runtime 默认 chooser。diff --git a/docs/exec-plans/completed/20261017-pr95-live-cursor-follow.md b/docs/exec-plans/completed/20261017-pr95-live-cursor-follow.mdnew file mode 100644index 0000000..ae3d9f9--- /dev/null+++ b/docs/exec-plans/completed/20261017-pr95-live-cursor-follow.md@@ -0,0 +1,32 @@+# PR #95: verify live cursor following on current main++## Objective++Review #95 against current main, reproduce remaining cursor-follow defects, fix them on the contributor branch and report merge readiness. Preserve the original contribution. Do not merge the PR as part of this task.++## Scope++- Follow the live window frame during travel, settle, click feedback and idle.+- Keep Accessibility IPC out of the overlay render loop; bound tracking state and outstanding reads.+- Preserve input delivery and window focus policies. This is visual cursor positioning, not a new input-routing mode.+- Keep existing motion dynamics for stationary windows and clear tracking at cursor/session cleanup.++## Progress++- [x] Fetch PR head and merge current main without conflicts.+- [x] Reproduce stale idle/click targets with production-overlay regression tests.+- [x] Fix live placement, tracking lifecycle and bounded frame acquisition.+- [x] Run Swift/Node checks, smoke and isolated live-window validation where available.+- [x] Update history/docs and deliver follow-up commits on the existing PR branch.++## Validation++Exercise the actual overlay loop with controlled frame changes, including repeated moves, a move back to the original frame, unanchored targets and reset. Verify slow frame reads do not block rendering or accumulate work. Use only isolated test windows for live checks; do not move personal app windows or synthesize user typing. Distinguish injected tests from real Accessibility frame observations in the final evidence.++## Evidence and decisions++- The original overlay failed three of four new regression tests (16 assertions). Fixes now cover idle/pulse targets, the actual visible travel endpoint, stale anchors and orphan idle timers.+- Frame reads run on a serial worker at up to 30 Hz with one outstanding request. Retain only the current source and last valid frame; discard results for replaced sources.+- Swift: 351 tests, 8 skipped, 0 failures. Node: 36 passed. Fixture action smoke and cursor idle smoke passed after updating its old tool-list assertion for main's query tool.+- The production overlay followed an isolated real AX window during travel, pulse and idle. Only one physical display was available. The full service-click test explicitly skips on this machine because the XCTest host does not satisfy the installed Locked Use identity check; retain that protection.+- No input-routing changes or version bump. Preserve the contributor's original commit and update the existing PR.diff --git a/docs/histories/2026-10/20261010-1745-cursor-live-frame-anchor.md b/docs/histories/2026-10/20261010-1745-cursor-live-frame-anchor.mdindex d35c2ea..055c8c7 100644--- a/docs/histories/2026-10/20261010-1745-cursor-live-frame-anchor.md+++ b/docs/histories/2026-10/20261010-1745-cursor-live-frame-anchor.md@@ -25,3 +25,26 @@## 教训第一次把锚点接在错误的函数上,单测全绿但真机无效。是**临时诊断**(无条件写日志,先验证调用路径)定位到真实构造点。结论:**先证明代码路径会被走到,再写实现**;单测覆盖不到调用路径本身。++## 2026-10-17:基于当前 main 复查 #95++用户要求在当前 main 上重新检查和测试,修复之前仍有问题的跟随行为,并补充提交到原 PR。先保留原提交,再合入 main 的 1.1.1 基线。++### 补充修复++- 原实现的 idle timer 和 click pulse 保存旧落点,窗口移动后仍在旧位置绘制。现在行进、settle、pulse 和 idle 共用最新缓存帧,并能处理连续移动和返回原位置。+- 行进中止原本只更新 spring 目标,旧速度仍让可见 tip 留在旧位置。现在同时重置 dynamics 并直接绘制新位置;窗口在行进开始前已移动时,仍保留正常行进动画。+- 新的无锚点目标会清掉旧窗口追踪。开始 idle / pulse 前停止旧计时器,避免重复 timer 在 reset 后继续绘制。+- AX 查询改为后台串行读取:最多 30 Hz、同时一个请求,渲染只读缓存。慢窗口不阻塞主线程,也不会堆积请求。+- 只保留当前窗口的 AX 引用;切换窗口和清理后丢弃在途旧结果。短暂读取失败保留最后有效帧,不回退到旧快照或滞后的窗口列表。非法 / 无限 frame 被拒绝。+- 同步架构、skill 使用说明和功能记录。冒烟脚本的旧 9-tool 数量断言不兼容 main 新增的 query,改为检查完整的 10 个工具名称;现有 fixture 动作覆盖仍为原 9 个工具。++### 验证与边界++- 修复前在原 overlay 上跑新增回归,4 项中的 3 项失败、共 16 个断言失败,复现 idle / pulse 和可见 tip 问题。+- 完整 Swift:351 项,8 项跳过,0 失败;原基线为 341 项、7 项跳过、0 失败。+- Node REPL / CLI / 配置:36 项通过。完整 fixture 动作 smoke 和 cursor idle smoke 通过;文档检查与 diff whitespace 检查通过。+- 新测试直接驱动生产 overlay,覆盖连续移动、返回原位置、双击期间移动、行进中止、无锚点目标、reset 和行进前已经移动;后台读取测试覆盖慢响应、请求不堆积、旧结果丢弃和读取失败保留缓存。+- 真实独立 NSWindow 的 AX 测试通过,包含行进、双击反馈、idle 和当前所有显示器。当前测试机仅一块显示器,未重做物理双屏拖动;上文双屏实测属于原贡献者的验证。+- 新增完整 ComputerUseService.click 集成测试,但本机 XCTest 宿主未通过已有 Locked Use 身份校验,明确跳过;没有绕过保护。这项跳过与 fixture smoke、真实窗口 AX / overlay 测试区分记录。+- 该修改调整软件光标显示位置,不改变输入路由,也不处理窗口内部控件重新排版后的定位。diff --git a/docs/releases/feature-release-notes.md b/docs/releases/feature-release-notes.mdindex c50f580..33f366c 100644--- a/docs/releases/feature-release-notes.md+++ b/docs/releases/feature-release-notes.md@@ -4,6 +4,7 @@| 日期 | 功能域 | 用户价值 | 变更摘要 || --- | --- | --- | --- |+| 2026-10-17 | macOS 软件光标窗口跟随 | 目标窗口移动后,行进、点击反馈和等待中的软件光标继续跟随,避免停在旧位置。 | 接入窗口相对锚点,修复 idle / pulse 的旧落点、行进中止后的可见位置及重复计时器;AX 帧读取改为后台串行缓存,限制为最多 30 Hz / 一个请求,并在清理时释放追踪。 || 2026-10-15 | 1.1.1 修复发布 | Locked Use 安装更新更可靠,降低认证与后台 IPC 开销。 | 统一版本为 1.1.1,发布授权宿主、安装加载检查、旧组件卸载与连接生命周期修复;完整双屏遮罩、自动解锁、点击验证和重锁实测通过。 || 2026-10-15 | Locked Use 安装与性能修复 | 减少认证和 IPC 开销,旧系统组件也能正常移除并重装。 | 非 UI 插件使用 privileged host,安装前验证真实加载;修复系统缓存卸载与转交 socket 的生命周期,降低空闲唤醒,status 提示组件版本差异。 || 2026-10-14 | 1.1.0 正式发布 | 用户可直接安装 Locked Use,在锁屏遮罩期间继续 GUI 操作,也可用定向查询和持久截图配置减少工具开销。 | 统一版本为 1.1.0;签名 universal macOS App 内嵌服务、Guardian、安装器和授权插件,普通 enable 请求权限,补齐冷启动握手隔离和发布制品检查;新增 app.query()、doctor --json 与 ocu config。 |diff --git a/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/CursorWindowAnchor.swift b/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/CursorWindowAnchor.swiftindex 4903e26..299ce42 100644--- a/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/CursorWindowAnchor.swift+++ b/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/CursorWindowAnchor.swift@@ -22,7 +22,7 @@ struct CursorRestingAnchor: Equatable, Sendable {/// Returns `nil` when the live frame carries no usable size, so callers can/// fall back to the point they already have.func screenStatePoint(liveFrame: CGRect) -> CGPoint? {- guard liveFrame.width > 0, liveFrame.height > 0 else {+ guard Self.isUsable(liveFrame), windowLocalPoint.x.isFinite, windowLocalPoint.y.isFinite else {return nil}@@ -30,6 +30,11 @@ struct CursorRestingAnchor: Equatable, Sendable {let localY = windowLocalPoint.y.clamped(to: 0...liveFrame.height)return CGPoint(x: liveFrame.minX + localX, y: liveFrame.minY + localY)}++ static func isUsable(_ frame: CGRect) -> Bool {+ !frame.isInfinite && !frame.isNull && frame.origin.x.isFinite && frame.origin.y.isFinite &&+ frame.width.isFinite && frame.height.isFinite && frame.width > 0 && frame.height > 0+ }}/// Live frames for target windows.@@ -38,40 +43,74 @@ struct CursorRestingAnchor: Equatable, Sendable {/// second after a window is dragged to another display — measured while fixing/// this: 83 identical frames at the cursor travel's 120 Hz pump. The/// accessibility element the snapshot already holds reports the move-/// immediately, so it is the source of truth here and the window list is only a-/// fallback for windows the snapshot never registered.+/// immediately, so it is the preferred source. Read it on a serial worker,+/// cache the last usable frame, and limit requests to 30 Hz. The window list is+/// used only when no accessibility element was registered; a failed AX read+/// must not replace a good frame with the window list's stale origin.enum CursorWindowFrameTracker {+ private final class Source: @unchecked Sendable {+ let windowID: CGWindowID+ let element: AXUIElement?+ init(windowID: CGWindowID, element: AXUIElement?) {+ self.windowID = windowID+ self.element = element+ }+ }+/// Lock-protected box so the tracker can be used from the service thread/// and the overlay's main-thread pump alike.private final class State: @unchecked Sendable {let lock = NSLock()- var elements: [CGWindowID: AXUIElement] = [:]+ // Only one software cursor is displayed. Do not retain every window+ // ever observed by a long-lived App Agent.+ var source: Source?+ var frame: CGRect?+ var nextRefresh: TimeInterval = 0+ var refreshing = false/// Test seam: consulted before any real lookup.var override: ((CGWindowID) -> CGRect?)?+ var reader: (@Sendable (CGWindowID) -> CGRect?)?}private static let state = State()+ private static let queue = DispatchQueue(label: "com.ifuryst.ocu.cursor-frame", qos: .userInitiated)+ private static let refreshInterval: TimeInterval = 1 / 30static func register(windowID: CGWindowID?, element: AXUIElement?) {- guard let windowID, let element else {+ guard let windowID else {return}state.lock.lock()defer { state.lock.unlock() }- state.elements[windowID] = element+ if let existing = state.source, existing.windowID == windowID {+ switch (existing.element, element) {+ case let (lhs?, rhs?) where CFEqual(lhs, rhs): return+ case (nil, nil): return+ default: break+ }+ }+ state.source = Source(windowID: windowID, element: element)+ state.frame = nil+ state.nextRefresh = 0}static func forget(windowID: CGWindowID) {state.lock.lock()defer { state.lock.unlock() }- state.elements.removeValue(forKey: windowID)+ if state.source?.windowID == windowID {+ state.source = nil+ state.frame = nil+ }}static func forgetAll() {state.lock.lock()defer { state.lock.unlock() }- state.elements.removeAll()+ state.source = nil+ state.frame = nil+ // An in-flight IPC read cannot be cancelled safely. Keep its slot busy+ // until it returns; its result will be discarded by source identity.}static func installFrameOverrideForTesting(_ body: ((CGWindowID) -> CGRect?)?) {@@ -80,35 +119,61 @@ enum CursorWindowFrameTracker {state.override = body}- static func liveFrame(for windowID: CGWindowID) -> CGRect? {+ static func installFrameReaderForTesting(_ body: (@Sendable (CGWindowID) -> CGRect?)?) {state.lock.lock()- let body = state.override- state.lock.unlock()+ defer { state.lock.unlock() }+ state.reader = body+ }- if let body {+ static func liveFrame(for windowID: CGWindowID) -> CGRect? {+ state.lock.lock()+ if let body = state.override {+ state.lock.unlock()return body(windowID)}-- return accessibilityFrame(for: windowID) ?? windowListFrame(for: windowID)+ guard let source = state.source, source.windowID == windowID else {+ state.lock.unlock()+ return nil+ }+ let frame = state.frame+ let now = ProcessInfo.processInfo.systemUptime+ let refresh = !state.refreshing && now >= state.nextRefresh+ let reader = state.reader+ if refresh {+ state.refreshing = true+ state.nextRefresh = now + refreshInterval+ }+ state.lock.unlock()+ if refresh {+ queue.async {+ // Accessibility can block on an unresponsive app. It must not+ // run on the render thread or accumulate a queue of requests.+ let observed: CGRect?+ if let reader { observed = reader(source.windowID) }+ else if let element = source.element { observed = accessibilityFrame(element) }+ else { observed = windowListFrame(for: source.windowID) }+ state.lock.lock()+ if state.source === source, let observed, CursorRestingAnchor.isUsable(observed) {+ state.frame = observed+ }+ state.refreshing = false+ state.lock.unlock()+ }+ }+ return frame}/// Frame read from the registered accessibility element, which reflects a/// move immediately.- static func accessibilityFrame(for windowID: CGWindowID) -> CGRect? {- state.lock.lock()- let element = state.elements[windowID]- state.lock.unlock()-- guard let element else {- return nil- }-+ private static func accessibilityFrame(_ element: AXUIElement) -> CGRect? {var positionValue: CFTypeRef?var sizeValue: CFTypeRef?guard AXUIElementCopyAttributeValue(element, kAXPositionAttribute as CFString, &positionValue) == .success,AXUIElementCopyAttributeValue(element, kAXSizeAttribute as CFString, &sizeValue) == .success,let positionValue,- let sizeValue+ let sizeValue,+ CFGetTypeID(positionValue) == AXValueGetTypeID(),+ CFGetTypeID(sizeValue) == AXValueGetTypeID()else {return nil}@@ -117,8 +182,7 @@ enum CursorWindowFrameTracker {var size = CGSize.zeroguard AXValueGetValue(positionValue as! AXValue, .cgPoint, &position),AXValueGetValue(sizeValue as! AXValue, .cgSize, &size),- size.width > 0,- size.height > 0+ CursorRestingAnchor.isUsable(CGRect(origin: position, size: size))else {return nil}diff --git a/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/SoftwareCursorOverlay.swift b/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/SoftwareCursorOverlay.swiftindex 791e39d..436c1a3 100644--- a/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/SoftwareCursorOverlay.swift+++ b/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/SoftwareCursorOverlay.swift@@ -195,6 +195,7 @@ enum SoftwareCursorOverlay {/// Window frame the current target point was derived from; lets a move/// under the cursor be detected and re-derived instead of ignored.private static var restingAnchor: CursorRestingAnchor?+ private static var lastAnchorFrame: CGRect?private static var visualDynamicsState: CursorVisualDynamicsState?private static var idleTimer: Timer?private static var hideTimer: Timer?@@ -215,7 +216,8 @@ enum SoftwareCursorOverlay {cancelPendingHide()configureOrdering(relativeTo: targetWindow)- restingAnchor = anchor+ adoptAnchor(anchor)+ lastAnchorFrame = anchor.flatMap { CursorWindowFrameTracker.liveFrame(for: $0.windowID) } ?? anchor?.windowBoundslet constrainedTarget = clampTipPosition(liveTargetPoint(targetPoint, anchor: anchor))let isFreshStart = displayedTipPosition == nillet startPoint = displayedTipPosition ?? defaultInitialTipPosition()@@ -254,7 +256,9 @@ enum SoftwareCursorOverlay {}configureOrdering(relativeTo: targetWindow)- restingAnchor = anchor ?? restingAnchor+ stopIdleAnimation()+ adoptAnchor(anchor)+ _ = reanchorIfWindowMoved()let constrainedTarget = clampTipPosition(liveTargetPoint(targetPoint, anchor: anchor))let now = CACurrentMediaTime()seedVisualDynamicsIfNeeded(at: constrainedTarget, time: now)@@ -275,7 +279,8 @@ enum SoftwareCursorOverlay {}configureOrdering(relativeTo: targetWindow)- restingAnchor = anchor ?? restingAnchor+ adoptAnchor(anchor)+ _ = reanchorIfWindowMoved()let constrainedTarget = clampTipPosition(liveTargetPoint(targetPoint, anchor: anchor))restingTipPosition = constrainedTargetobservationPhase = "settling"@@ -297,6 +302,8 @@ enum SoftwareCursorOverlay {restingTipPosition = nilactiveTargetWindow = nilrestingAnchor = nil+ lastAnchorFrame = nil+ CursorWindowFrameTracker.forgetAll()visualDynamicsState = nilobservationPhase = "hidden"writeObservationSnapshot(tipPosition: nil, rotation: nil)@@ -309,7 +316,6 @@ enum SoftwareCursorOverlay {private static func liveTargetPoint(_ targetPoint: CGPoint, anchor: CursorRestingAnchor?) -> CGPoint {guard let anchor,let liveFrame = CursorWindowFrameTracker.liveFrame(for: anchor.windowID),- liveFrame != anchor.windowBounds,let screenStatePoint = anchor.screenStatePoint(liveFrame: liveFrame)else {return targetPoint@@ -321,26 +327,24 @@ enum SoftwareCursorOverlay {)}- /// Whether the anchored window has moved since the anchor was taken.- private static func anchoredWindowMoved() -> Bool {- guard let anchor = restingAnchor,- let liveFrame = CursorWindowFrameTracker.liveFrame(for: anchor.windowID)- else {- return false+ private static func adoptAnchor(_ anchor: CursorRestingAnchor?) {+ if restingAnchor != anchor {+ restingAnchor = anchor+ lastAnchorFrame = anchor?.windowBounds}-- return liveFrame != anchor.windowBounds+ if anchor == nil { CursorWindowFrameTracker.forgetAll() }}- /// Stop a travel whose window moved under it: land the cursor on the- /// window's live frame instead of finishing the path towards the display- /// the window just left.- private static func abortTravelOntoObservedWindow() {+ /// Relocate the visible tip, not just its spring target. Otherwise an old+ /// travel velocity can leave it on the previous display after reanchoring.+ @discardableResult+ private static func reanchorIfWindowMoved() -> Bool {guard let anchor = restingAnchor,let liveFrame = CursorWindowFrameTracker.liveFrame(for: anchor.windowID),+ liveFrame != lastAnchorFrame,let screenStatePoint = anchor.screenStatePoint(liveFrame: liveFrame)else {- return+ return false}let constrainedTarget = clampTipPosition(@@ -349,12 +353,15 @@ enum SoftwareCursorOverlay {screenMappings: currentVisualCursorScreenMappings()))+ lastAnchorFrame = liveFramerestingTipPosition = constrainedTarget+ visualDynamicsState = CursorVisualDynamicsAnimator.state(at: constrainedTarget, time: CGFloat(CACurrentMediaTime()))observationPhase = "reanchored"placeCursor(- using: advanceVisualDynamics(toward: constrainedTarget, at: CACurrentMediaTime()),+ using: initialRenderState(at: constrainedTarget),clickProgress: 0)+ return true}private static var canPresentOverlay: Bool {@@ -438,8 +445,7 @@ enum SoftwareCursorOverlay {while true {refreshActiveOrderingIfNeeded()- if anchoredWindowMoved() {- abortTravelOntoObservedWindow()+ if reanchorIfWindowMoved() {return}@@ -632,13 +638,14 @@ enum SoftwareCursorOverlay {let startTime = CACurrentMediaTime()while true {+ _ = reanchorIfWindowMoved()let elapsed = CACurrentMediaTime() - startTimelet rawProgress = min(max(elapsed / duration, 0), 1)let clickProgress = sin(rawProgress * .pi) * pulseBiasplaceCursor(using: advanceVisualDynamics(- toward: point,+ toward: restingTipPosition ?? point,at: CACurrentMediaTime()),clickProgress: clickProgress@@ -658,7 +665,7 @@ enum SoftwareCursorOverlay {placeCursor(using: advanceVisualDynamics(- toward: point,+ toward: restingTipPosition ?? point,at: CACurrentMediaTime()),clickProgress: 0@@ -666,6 +673,7 @@ enum SoftwareCursorOverlay {}private static func startIdleAnimation() {+ stopIdleAnimation()guard canPresentOverlay, let restingTipPosition else {return}@@ -679,11 +687,13 @@ enum SoftwareCursorOverlay {}refreshActiveOrderingIfNeeded()+ _ = reanchorIfWindowMoved()+ guard let currentTipPosition = self.restingTipPosition else { return }observationPhase = "idle"idlePhase += 0.05let idlePose = visualCursorIdlePose(- restingTipPosition: restingTipPosition,+ restingTipPosition: currentTipPosition,phase: idlePhase)@@ -750,6 +760,9 @@ enum SoftwareCursorOverlay {displayedTipPosition = nilrestingTipPosition = nilactiveTargetWindow = nil+ restingAnchor = nil+ lastAnchorFrame = nil+ CursorWindowFrameTracker.forgetAll()visualDynamicsState = nilobservationPhase = "hidden"writeObservationSnapshot(tipPosition: nil, rotation: nil)diff --git a/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/CursorWindowAnchorTests.swift b/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/CursorWindowAnchorTests.swiftindex 8b0e1a7..a4f9960 100644--- a/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/CursorWindowAnchorTests.swift+++ b/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/CursorWindowAnchorTests.swift@@ -52,6 +52,8 @@ final class CursorWindowAnchorTests: XCTestCase {func testAnchorRejectsALiveFrameWithoutSize() {XCTAssertNil(anchor().screenStatePoint(liveFrame: .zero))XCTAssertNil(anchor().screenStatePoint(liveFrame: CGRect(x: 10, y: 10, width: 0, height: 40)))+ XCTAssertNil(anchor().screenStatePoint(liveFrame: .infinite))+ XCTAssertNil(anchor().screenStatePoint(liveFrame: CGRect(x: CGFloat.nan, y: 0, width: 50, height: 50)))}func testTrackerPrefersTheRegisteredLiveFrame() {@@ -64,11 +66,81 @@ final class CursorWindowAnchorTests: XCTestCase {XCTAssertNil(CursorWindowFrameTracker.liveFrame(for: 1))}- /// Registering nothing must not produce a frame out of thin air; the- /// window list is the only remaining source and an unknown id has no entry.+ /// An unknown window must not produce a frame out of thin air.func testTrackerReportsNothingForAnUnknownWindow() {CursorWindowFrameTracker.forgetAll()XCTAssertNil(CursorWindowFrameTracker.liveFrame(for: CGWindowID(0)))}++ func testFailedRefreshKeepsLastUsableFrame() throws {+ let reads = DispatchSemaphore(value: 0), failedRead = DispatchSemaphore(value: 0)+ let frame = mainDisplayFrame+ CursorWindowFrameTracker.forgetAll()+ CursorWindowFrameTracker.installFrameReaderForTesting { _ in+ reads.signal()+ return frame+ }+ defer {+ CursorWindowFrameTracker.forgetAll()+ CursorWindowFrameTracker.installFrameReaderForTesting(nil)+ }+ CursorWindowFrameTracker.register(windowID: 1, element: nil)+ _ = CursorWindowFrameTracker.liveFrame(for: 1)+ XCTAssertEqual(reads.wait(timeout: .now() + 1), .success)+ let deadline = ProcessInfo.processInfo.systemUptime + 1+ while CursorWindowFrameTracker.liveFrame(for: 1) == nil && ProcessInfo.processInfo.systemUptime < deadline {+ Thread.sleep(forTimeInterval: 0.001)+ }+ XCTAssertEqual(CursorWindowFrameTracker.liveFrame(for: 1), frame)+ CursorWindowFrameTracker.installFrameReaderForTesting { _ in+ failedRead.signal()+ return nil+ }+ Thread.sleep(forTimeInterval: 0.04)+ XCTAssertEqual(CursorWindowFrameTracker.liveFrame(for: 1), frame)+ XCTAssertEqual(failedRead.wait(timeout: .now() + 1), .success)+ Thread.sleep(forTimeInterval: 0.01)+ XCTAssertEqual(CursorWindowFrameTracker.liveFrame(for: 1), frame)+ }++ func testSlowReadDoesNotBlockOrQueueWorkAndCannotPublishAfterReplacement() throws {+ let entered = DispatchSemaphore(value: 0), release = DispatchSemaphore(value: 0)+ let first = builtInDisplayFrame, second = mainDisplayFrame+ CursorWindowFrameTracker.forgetAll()+ CursorWindowFrameTracker.installFrameReaderForTesting { id in+ if id == 1 {+ entered.signal()+ _ = release.wait(timeout: .now() + 2)+ return first+ }+ return second+ }+ defer {+ release.signal()+ CursorWindowFrameTracker.forgetAll()+ CursorWindowFrameTracker.installFrameReaderForTesting(nil)+ }+ CursorWindowFrameTracker.register(windowID: 1, element: nil)+ XCTAssertNil(CursorWindowFrameTracker.liveFrame(for: 1))+ XCTAssertEqual(entered.wait(timeout: .now() + 1), .success)+ let start = ProcessInfo.processInfo.systemUptime+ for _ in 0..<1000 { XCTAssertNil(CursorWindowFrameTracker.liveFrame(for: 1)) }+ XCTAssertLessThan(ProcessInfo.processInfo.systemUptime - start, 0.25)+ CursorWindowFrameTracker.forgetAll()+ CursorWindowFrameTracker.register(windowID: 2, element: nil)+ XCTAssertNil(CursorWindowFrameTracker.liveFrame(for: 1))+ XCTAssertNil(CursorWindowFrameTracker.liveFrame(for: 2))+ release.signal()+ let deadline = ProcessInfo.processInfo.systemUptime + 1+ var observed: CGRect?+ while observed == nil && ProcessInfo.processInfo.systemUptime < deadline {+ observed = CursorWindowFrameTracker.liveFrame(for: 2)+ Thread.sleep(forTimeInterval: 0.001)+ }+ XCTAssertEqual(observed, second)+ XCTAssertNil(CursorWindowFrameTracker.liveFrame(for: 1))+ // No stale queued requests from the 1,000 render ticks may remain.+ XCTAssertEqual(entered.wait(timeout: .now() + 0.05), .timedOut)+ }}diff --git a/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/CursorWindowFollowTests.swift b/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/CursorWindowFollowTests.swiftnew file mode 100644index 0000000..8415c8d--- /dev/null+++ b/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/CursorWindowFollowTests.swift@@ -0,0 +1,264 @@+import AppKit+import XCTest+@testable import OpenComputerUseKit++final class CursorWindowFollowTests: XCTestCase {+ @MainActor+ private final class Counter: NSObject {+ var count = 0+ @objc func increment() { count += 1 }+ }+ private final class Frame: @unchecked Sendable {+ private let lock = NSLock()+ private var stored: CGRect+ init(_ value: CGRect) { stored = value }+ var value: CGRect {+ get { lock.lock(); defer { lock.unlock() }; return stored }+ set { lock.lock(); defer { lock.unlock() }; stored = newValue }+ }+ }+ @MainActor+ private func withOverlay(_ body: (URL, CGRect) throws -> Void) throws {+ guard let screen = currentVisualCursorScreenMappings().first?.screenStateFrame else {+ throw XCTSkip("A WindowServer display is required for overlay rendering")+ }+ _ = NSApplication.shared+ let file = FileManager.default.temporaryDirectory.appendingPathComponent("ocu-cursor-follow-\(UUID().uuidString).json")+ let previous = ProcessInfo.processInfo.environment+ setenv("OPEN_COMPUTER_USE_VISUAL_CURSOR", "1", 1)+ setenv("OPEN_COMPUTER_USE_VISUAL_CURSOR_OBSERVATION_FILE", file.path, 1)+ setenv("OPEN_COMPUTER_USE_ACTION_READ_BACK", "0", 1)+ SoftwareCursorOverlay.moveCursor(to: defaultVisualCursorInitialTipPosition(), in: nil)+ SoftwareCursorOverlay.reset()+ defer {+ SoftwareCursorOverlay.reset()+ CursorWindowFrameTracker.installFrameOverrideForTesting(nil)+ CursorWindowFrameTracker.forgetAll()+ for key in ["OPEN_COMPUTER_USE_VISUAL_CURSOR", "OPEN_COMPUTER_USE_VISUAL_CURSOR_OBSERVATION_FILE", "OPEN_COMPUTER_USE_ACTION_READ_BACK"] {+ if let value = previous[key] { setenv(key, value, 1) } else { unsetenv(key) }+ }+ try? FileManager.default.removeItem(at: file)+ }+ try body(file, CGRect(x: screen.minX + 100, y: screen.minY + 100, width: 200, height: 160))+ }++ private func anchor(_ frame: CGRect) -> CursorRestingAnchor {+ .init(windowID: 4242, layer: 0, windowLocalPoint: CGPoint(x: 40, y: 50), windowBounds: frame)+ }++ @MainActor+ private func point(_ frame: CGRect) -> CGPoint {+ screenStatePointToAppKitGlobalPoint(fromScreenStatePoint: CGPoint(x: frame.minX + 40, y: frame.minY + 50))+ }++ @MainActor+ private func assertPosition(_ file: URL, frame: CGRect, file testFile: StaticString = #filePath, line: UInt = #line) throws {+ let state = try JSONDecoder().decode(VisualCursorObservationSnapshot.self, from: Data(contentsOf: file))+ let expected = point(frame)+ XCTAssertEqual(try XCTUnwrap(state.restingTipPosition).x, expected.x, accuracy: 1, file: testFile, line: line)+ XCTAssertEqual(try XCTUnwrap(state.restingTipPosition).y, expected.y, accuracy: 1, file: testFile, line: line)+ XCTAssertEqual(try XCTUnwrap(state.tipPosition).x, expected.x, accuracy: 1, file: testFile, line: line)+ XCTAssertEqual(try XCTUnwrap(state.tipPosition).y, expected.y, accuracy: 1, file: testFile, line: line)+ }++ @MainActor+ func testIdleFollowsRepeatedMovesAndReturnToOriginalFrame() throws {+ try withOverlay { file, original in+ let live = Frame(original)+ CursorWindowFrameTracker.installFrameOverrideForTesting { _ in live.value }+ SoftwareCursorOverlay.settle(at: point(original), in: nil, anchor: anchor(original))+ for frame in [original.offsetBy(dx: 240, dy: 80), original.offsetBy(dx: 80, dy: 160), original] {+ live.value = frame+ RunLoop.main.run(until: Date().addingTimeInterval(0.08))+ try assertPosition(file, frame: frame)+ }+ }+ }++ @MainActor+ func testClickPulseTracksMovesDuringBothClicks() throws {+ try withOverlay { file, original in+ let live = Frame(original)+ CursorWindowFrameTracker.installFrameOverrideForTesting { _ in live.value }+ SoftwareCursorOverlay.settle(at: point(original), in: nil, anchor: anchor(original))+ let first = Timer(timeInterval: 0.04, repeats: false) { _ in live.value = original.offsetBy(dx: 180, dy: 40) }+ let second = Timer(timeInterval: 0.24, repeats: false) { _ in live.value = original.offsetBy(dx: 260, dy: 100) }+ RunLoop.main.add(first, forMode: .common)+ RunLoop.main.add(second, forMode: .common)+ defer { first.invalidate(); second.invalidate() }+ SoftwareCursorOverlay.pulseClick(at: point(original), clickCount: 2, mouseButton: .left, in: nil, anchor: anchor(original))+ try assertPosition(file, frame: live.value)+ }+ }++ @MainActor+ func testTravelAbortActuallyPlacesVisibleTipAtMovedWindow() throws {+ try withOverlay { file, original in+ let live = Frame(original)+ CursorWindowFrameTracker.installFrameOverrideForTesting { _ in live.value }+ let timer = Timer(timeInterval: 0.04, repeats: false) { _ in live.value = original.offsetBy(dx: 240, dy: 100) }+ RunLoop.main.add(timer, forMode: .common)+ defer { timer.invalidate() }+ SoftwareCursorOverlay.moveCursor(to: point(original), in: nil, anchor: anchor(original))+ try assertPosition(file, frame: live.value)+ // The next pipeline stage must not restore the snapshot's old point.+ SoftwareCursorOverlay.settle(at: point(original), in: nil, anchor: anchor(original))+ try assertPosition(file, frame: live.value)+ }+ }++ @MainActor+ func testUnanchoredTargetDoesNotKeepPreviousWindowTracking() throws {+ try withOverlay { file, original in+ let live = Frame(original)+ CursorWindowFrameTracker.installFrameOverrideForTesting { _ in live.value }+ SoftwareCursorOverlay.settle(at: point(original), in: nil, anchor: anchor(original))+ SoftwareCursorOverlay.settle(at: point(original), in: nil)+ live.value = original.offsetBy(dx: 240, dy: 100)+ RunLoop.main.run(until: Date().addingTimeInterval(0.08))+ try assertPosition(file, frame: original)+ }+ }++ @MainActor+ func testResetAfterPulseDoesNotLeaveAnIdleTimerRepainting() throws {+ try withOverlay { file, original in+ CursorWindowFrameTracker.installFrameOverrideForTesting { _ in original }+ SoftwareCursorOverlay.settle(at: point(original), in: nil, anchor: anchor(original))+ SoftwareCursorOverlay.pulseClick(at: point(original), clickCount: 1, mouseButton: .left, in: nil, anchor: anchor(original))+ SoftwareCursorOverlay.reset()+ RunLoop.main.run(until: Date().addingTimeInterval(0.08))+ let state = try JSONDecoder().decode(VisualCursorObservationSnapshot.self, from: Data(contentsOf: file))+ XCTAssertEqual(state.phase, "hidden")+ XCTAssertNil(state.tipPosition)+ XCTAssertNil(state.restingTipPosition)+ }+ }++ @MainActor+ func testWindowAlreadyMovedBeforeTravelDoesNotAbortAgainstOriginalSnapshot() throws {+ try withOverlay { _, original in+ let moved = original.offsetBy(dx: 240, dy: 100)+ CursorWindowFrameTracker.installFrameOverrideForTesting { _ in moved }+ let start = ProcessInfo.processInfo.systemUptime+ SoftwareCursorOverlay.moveCursor(to: point(original), in: nil, anchor: anchor(original))+ XCTAssertGreaterThan(ProcessInfo.processInfo.systemUptime - start, 0.5)+ }+ }++ @MainActor+ func testServiceClickRegistersLiveWindowAndFeedbackMatchesMovedButton() throws {+ do { try requireUsableComputerUseSession() }+ catch { throw XCTSkip("Service integration requires an unlocked session and a trusted test host: \(error)") }+ try withOverlay { file, original in+ let counter = Counter()+ let window = NSWindow(contentRect: original, styleMask: [.titled], backing: .buffered, defer: false)+ window.title = "OCU service click fixture"+ window.isReleasedWhenClosed = false+ let button = NSButton(title: "OCU follow click", target: counter, action: #selector(Counter.increment))+ button.frame = CGRect(x: 20, y: 20, width: 140, height: 32)+ window.contentView?.addSubview(button)+ NSApplication.shared.finishLaunching()+ window.orderFront(nil)+ defer { window.close() }+ RunLoop.main.run(until: Date().addingTimeInterval(0.05))+ guard AXIsProcessTrusted(), let app = AppDiscovery.runningApps().first(where: { $0.pid == getpid() }) else {+ throw XCTSkip("Live AX test application is unavailable")+ }+ let service = ComputerUseService()+ let result = try service.query(app: app.bundleIdentifier ?? app.name, text: button.title, role: "button",+ exact: true, windowID: CGWindowID(window.windowNumber))+ let matches = try XCTUnwrap(result["matches"] as? [[String: Any]])+ XCTAssertEqual(matches.count, 1)+ let index = try XCTUnwrap(matches.first?["index"] as? Int)+ let timer = Timer(timeInterval: 0.08, repeats: false) { _ in+ MainActor.assumeIsolated {+ window.setFrameOrigin(CGPoint(x: original.minX + 240, y: original.minY + 80))+ }+ }+ RunLoop.main.add(timer, forMode: .common)+ defer { timer.invalidate() }+ _ = try service.click(app: app.bundleIdentifier ?? app.name, elementIndex: String(index),+ x: nil, y: nil, clickCount: 1, mouseButton: "left", clickMethod: .accessibility)+ XCTAssertEqual(counter.count, 1, "The real AX button action must still execute exactly once")+ let state = try JSONDecoder().decode(VisualCursorObservationSnapshot.self, from: Data(contentsOf: file))+ let expected = window.convertPoint(toScreen: CGPoint(x: button.frame.midX, y: button.frame.midY))+ XCTAssertEqual(try XCTUnwrap(state.tipPosition).x, expected.x, accuracy: 1)+ XCTAssertEqual(try XCTUnwrap(state.tipPosition).y, expected.y, accuracy: 1)+ XCTAssertEqual(try XCTUnwrap(state.restingTipPosition).x, expected.x, accuracy: 1)+ XCTAssertEqual(try XCTUnwrap(state.restingTipPosition).y, expected.y, accuracy: 1)+ }+ }++ @MainActor+ func testRealAccessibilityWindowFollowsTravelPulseAndIdleAcrossDisplays() throws {+ try withOverlay { file, original in+ let window = NSWindow(contentRect: original, styleMask: [.titled], backing: .buffered, defer: false)+ window.title = "OCU cursor follow fixture \(UUID().uuidString)"+ window.isReleasedWhenClosed = false+ NSApplication.shared.finishLaunching()+ window.orderFront(nil)+ defer { window.close() }+ RunLoop.main.run(until: Date().addingTimeInterval(0.05))+ let application = AXUIElementCreateApplication(getpid())+ var raw: CFTypeRef?+ let status = AXUIElementCopyAttributeValue(application, kAXWindowsAttribute as CFString, &raw)+ guard status == .success,+ let windows = raw as? [AXUIElement],+ let element = windows.first(where: { element in+ var title: CFTypeRef?+ return AXUIElementCopyAttributeValue(element, kAXTitleAttribute as CFString, &title) == .success &&+ title as? String == window.title+ })+ else { throw XCTSkip("Live self-AX window access unavailable (AX status \(status.rawValue), trusted \(AXIsProcessTrusted()))") }+ let id = CGWindowID(window.windowNumber)+ CursorWindowFrameTracker.register(windowID: id, element: element)+ func awaitFrame(_ predicate: (CGRect) -> Bool) throws -> CGRect {+ let deadline = Date().addingTimeInterval(1)+ while Date() < deadline {+ if let frame = CursorWindowFrameTracker.liveFrame(for: id), predicate(frame) { return frame }+ RunLoop.main.run(until: Date().addingTimeInterval(0.01))+ }+ XCTFail("Real AX frame was not observed before deadline")+ throw NSError(domain: "CursorFollowTests", code: 1)+ }+ let initial = try awaitFrame { _ in true }+ let targetAnchor = CursorRestingAnchor(windowID: id, layer: 0, windowLocalPoint: CGPoint(x: 40, y: 50), windowBounds: initial)+ let move = Timer(timeInterval: 0.08, repeats: false) { _ in+ MainActor.assumeIsolated {+ window.setFrameOrigin(CGPoint(x: original.minX + 200, y: original.minY + 40))+ }+ }+ RunLoop.main.add(move, forMode: .common)+ defer { move.invalidate() }+ SoftwareCursorOverlay.moveCursor(to: point(initial), in: nil, anchor: targetAnchor)+ let afterTravel = try awaitFrame { abs($0.minX - (initial.minX + 200)) < 1 }+ try assertPosition(file, frame: afterTravel)+ SoftwareCursorOverlay.settle(at: point(initial), in: nil, anchor: targetAnchor)+ try assertPosition(file, frame: afterTravel)+ let duringPulse = Timer(timeInterval: 0.06, repeats: false) { _ in+ MainActor.assumeIsolated {+ window.setFrameOrigin(CGPoint(x: original.minX + 260, y: original.minY + 100))+ }+ }+ RunLoop.main.add(duringPulse, forMode: .common)+ defer { duringPulse.invalidate() }+ SoftwareCursorOverlay.pulseClick(at: point(initial), clickCount: 2, mouseButton: .left, in: nil, anchor: targetAnchor)+ let afterPulse = try awaitFrame { abs($0.minX - (initial.minX + 260)) < 1 }+ try assertPosition(file, frame: afterPulse)+ for offset in [CGPoint(x: 220, y: 60), CGPoint(x: 100, y: 140), .zero] {+ window.setFrameOrigin(CGPoint(x: original.minX + offset.x, y: original.minY + offset.y))+ let moved = try awaitFrame { abs($0.minX - (initial.minX + offset.x)) < 1 }+ RunLoop.main.run(until: Date().addingTimeInterval(0.04))+ try assertPosition(file, frame: moved)+ }+ for mapping in currentVisualCursorScreenMappings() {+ window.setFrameOrigin(CGPoint(x: mapping.appKitFrame.minX + 120, y: mapping.appKitFrame.minY + 120))+ let moved = try awaitFrame { abs($0.minX - (mapping.screenStateFrame.minX + 120)) < 1 }+ RunLoop.main.run(until: Date().addingTimeInterval(0.04))+ try assertPosition(file, frame: moved)+ }+ print("Live AX cursor following verified on \(currentVisualCursorScreenMappings().count) display(s).")+ }+ }+}diff --git a/skills/open-computer-use/references/usage.md b/skills/open-computer-use/references/usage.mdindex 5c8426c..9a710b5 100644--- a/skills/open-computer-use/references/usage.md+++ b/skills/open-computer-use/references/usage.md@@ -230,15 +230,18 @@ When the gate is not enabled, treat window-server drags as unavailable and reach## Software Cursor-The software cursor follows the target window when it moves. Its point is derived from the snapshot, so it-also carries an anchor: the same point expressed relative to the window frame the snapshot reported. Before-drawing, and on every frame of a travel, the overlay re-derives the point from the window's live frame, so a-window dragged to another display mid-action takes the cursor with it instead of leaving it on the display-the window just left.--The live frame comes from the window's accessibility element — the element the snapshot already holds — and-only falls back to `CGWindowListCopyWindowInfo`, which can keep reporting the pre-move origin for about a-second (measured: 83 identical frames at the travel's 120 Hz pump).+The software cursor keeps a window-relative anchor for its target point. During travel, settle, click+feedback and idle, it follows changes to the target window's frame. If the window moves during travel,+the visible tip lands at the new position immediately, without continuing toward the old position.++Window frames are read on a background worker at up to 30 Hz, with at most one read in flight. Rendering+uses the latest cached frame, so a slow Accessibility response does not block the cursor. The snapshot's+AX window element is preferred; `CGWindowListCopyWindowInfo` is used only when no AX element is available,+because it can report an old origin after a move. A failed read keeps the last usable frame. Tracking is+released when the cursor is cleared or its idle timeout expires.++This follows the window's position, not layout changes inside it. Refresh app state after a layout change+before choosing another target. Input delivery continues to use the existing action paths.## Platform Notes