👾 Open Computer Use – Open-Source Alternative to Codex Computer Use
Swift accessibilityai-agentai-agentsclaude-codecodexcodex-computer-usecomputer-usecomputer-use-agentcross-platformcuadesktop-automationgemini-cligui-automationmacosmcpmodel-context-protocolopen-computer-useopen-computer-use-mcpopenai-codexopencode
fix(cursor): 窗口移动时按实时帧重定位软件光标
wicgee5a4515979853
diff --git a/docs/histories/2026-10/20261010-1745-cursor-live-frame-anchor.md b/docs/histories/2026-10/20261010-1745-cursor-live-frame-anchor.mdnew file mode 100644index 0000000..d35c2ea--- /dev/null+++ b/docs/histories/2026-10/20261010-1745-cursor-live-frame-anchor.md@@ -0,0 +1,27 @@+# 软件光标:窗口移动时按实时帧重定位(v1.0.0 基线)++## 背景++在 v1.0.0 上重做此前那条跨屏光标修复。双屏环境(内建屏给 Agent、主屏给人)下,目标窗口在动作途中被拖到另一块屏时,软件光标会停在窗口原来的屏幕上。++## 根因++1. `CGWindowListCopyWindowInfo` 在窗口移动后仍返回移动前的 frame(实测连续 83 帧、约 1 秒),只有 AX 元素的位置是即时的。+2. 光标目标点是在快照时刻算出来的;窗口之后移动,落点随之失效。+3. **v1.0.0 的点击路径不经过 `visualCursorTarget(for:snapshot:)`**:它在 `ComputerUseService` 的 click 分支里直接调 `makeVisualCursorTarget(at:...)`。因此最初把锚点接在 `visualCursorTarget(for:snapshot:)` 上,单测全过而真机完全无效——诊断日志证明 `register`/`liveFrame` 一次都没被调用。++## 改动++- 新增 `CursorWindowAnchor.swift`:`CursorRestingAnchor`(元素相对窗口帧的锚点,纯函数、可单测)+ `CursorWindowFrameTracker`(AX 元素优先、窗口列表兜底,带测试注入点)。+- `VisualCursorTarget` 增加 `anchor`;点击分支用 `clickPoint` 得到的窗口内坐标构造锚点,并在构造锚点处注册快照携带的 AX 窗口元素。+- Overlay:新增 `restingAnchor`、`liveTargetPoint()`、`anchoredWindowMoved()`、`abortTravelOntoObservedWindow()`;`moveCursor` / `settle` / `pulseClick` 均按实时帧重算,行进循环逐帧检查窗口是否移动,移动即中止并落到窗口的新位置。++## 验证++- 单测:`CursorWindowAnchorTests` 7 项(含用本次复现的真实数字构造的用例);全套 199 tests / 7 skipped / 0 failures;`make check-docs` 通过。+- 真机(双屏):窗口在行进途中从内建屏移到主屏,**连续两次**光标跟随到主屏(光标 `x=1057`,窗口 `x=600`)。+- 帧来源日志:76–79 次读取全部走 AX 元素,窗口列表兜底 **0** 次——即不再依赖那条会滞后的路径。++## 教训++第一次把锚点接在错误的函数上,单测全绿但真机无效。是**临时诊断**(无条件写日志,先验证调用路径)定位到真实构造点。结论:**先证明代码路径会被走到,再写实现**;单测覆盖不到调用路径本身。diff --git a/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/ComputerUseService.swift b/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/ComputerUseService.swiftindex 179019e..cf1060a 100644--- a/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/ComputerUseService.swift+++ b/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/ComputerUseService.swift@@ -6,6 +6,11 @@ import ImageIOstruct VisualCursorTarget: Equatable {let point: CGPointlet window: CursorTargetWindow?+ /// The same point expressed relative to the window frame the snapshot+ /// reported. The overlay keeps it so a window that moves before or during+ /// the action can be re-derived instead of drawing on the display the+ /// window just left.+ var anchor: CursorRestingAnchor? = nil}public enum ClickMethod: String, CaseIterable, Sendable {@@ -149,18 +154,43 @@ func inputEventPoint(point}+/// Anchor for a cursor target built from a window-local point.+///+/// The click paths derive their point from the snapshot, so they carry the+/// window frame that point belongs to; the overlay needs both to follow the+/// window if it moves before or during the action.+func makeCursorWindowAnchor(snapshot: AppSnapshot, windowLocalPoint: CGPoint) -> CursorRestingAnchor? {+ guard let windowID = snapshot.targetWindowID, let windowBounds = snapshot.windowBounds else {+ return nil+ }++ // Registering here keeps the live-frame source next to the anchor that+ // depends on it: the window list can lag a move by about a second, so the+ // accessibility element is what makes the re-derivation immediate.+ CursorWindowFrameTracker.register(windowID: windowID, element: snapshot.windowElement)++ return CursorRestingAnchor(+ windowID: windowID,+ layer: snapshot.targetWindowLayer ?? 0,+ windowLocalPoint: windowLocalPoint,+ windowBounds: windowBounds+ )+}+func makeVisualCursorTarget(at point: CGPoint,targetWindowID: CGWindowID?,targetWindowLayer: Int?,- screenMappings: [VisualCursorScreenMapping] = currentVisualCursorScreenMappings()+ screenMappings: [VisualCursorScreenMapping] = currentVisualCursorScreenMappings(),+ anchor: CursorRestingAnchor? = nil) -> VisualCursorTarget {VisualCursorTarget(point: screenStatePointToAppKitGlobalPoint(fromScreenStatePoint: point,screenMappings: screenMappings),- window: targetWindowID.map { CursorTargetWindow(windowID: $0, layer: targetWindowLayer ?? 0) }+ window: targetWindowID.map { CursorTargetWindow(windowID: $0, layer: targetWindowLayer ?? 0) },+ anchor: anchor)}@@ -183,7 +213,15 @@ func makeVisualCursorTarget(at: point,targetWindowID: targetWindowID,targetWindowLayer: targetWindowLayer,- screenMappings: screenMappings+ screenMappings: screenMappings,+ anchor: targetWindowID.map { windowID in+ CursorRestingAnchor(+ windowID: windowID,+ layer: targetWindowLayer ?? 0,+ windowLocalPoint: CGPoint(x: localFrame.midX, y: localFrame.midY),+ windowBounds: windowBounds+ )+ })}@@ -625,7 +663,8 @@ public final class ComputerUseService {let cursorTarget = makeVisualCursorTarget(at: targetPoint,targetWindowID: snapshot.targetWindowID,- targetWindowLayer: snapshot.targetWindowLayer+ targetWindowLayer: snapshot.targetWindowLayer,+ anchor: makeCursorWindowAnchor(snapshot: snapshot, windowLocalPoint: windowPoint))moveVisualCursor(to: cursorTarget)@@ -686,7 +725,8 @@ public final class ComputerUseService {let cursorTarget = makeVisualCursorTarget(at: targetPoint,targetWindowID: snapshot.targetWindowID,- targetWindowLayer: snapshot.targetWindowLayer+ targetWindowLayer: snapshot.targetWindowLayer,+ anchor: makeCursorWindowAnchor(snapshot: snapshot, windowLocalPoint: point))moveVisualCursor(to: cursorTarget)@@ -1943,7 +1983,15 @@ public final class ComputerUseService {}private func visualCursorTarget(for record: ElementRecord, snapshot: AppSnapshot) -> VisualCursorTarget? {- makeVisualCursorTarget(+ // The accessibility element is the overlay's only immediate source of+ // truth for the window frame: the window list can keep reporting the+ // pre-move origin for about a second.+ CursorWindowFrameTracker.register(+ windowID: snapshot.targetWindowID,+ element: snapshot.windowElement+ )++ return makeVisualCursorTarget(localFrame: record.localFrame,windowBounds: snapshot.windowBounds,targetWindowID: snapshot.targetWindowID,@@ -1962,7 +2010,7 @@ public final class ComputerUseService {}VisualCursorSupport.performOnMain {- SoftwareCursorOverlay.moveCursor(to: target.point, in: target.window)+ SoftwareCursorOverlay.moveCursor(to: target.point, in: target.window, anchor: target.anchor)}}@@ -1972,7 +2020,7 @@ public final class ComputerUseService {}VisualCursorSupport.performOnMain {- SoftwareCursorOverlay.settle(at: target.point, in: target.window)+ SoftwareCursorOverlay.settle(at: target.point, in: target.window, anchor: target.anchor)}}@@ -1986,7 +2034,8 @@ public final class ComputerUseService {at: target.point,clickCount: clickCount,mouseButton: mouseButton,- in: target.window+ in: target.window,+ anchor: target.anchor)}}diff --git a/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/CursorWindowAnchor.swift b/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/CursorWindowAnchor.swiftnew file mode 100644index 0000000..4903e26--- /dev/null+++ b/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/CursorWindowAnchor.swift@@ -0,0 +1,147 @@+import ApplicationServices+import CoreGraphics+import Foundation++/// The point a cursor target was derived from, expressed relative to the+/// window frame the snapshot reported.+///+/// A window that moves between the snapshot and the action keeps its+/// window-local geometry, so this is what lets the overlay re-derive the point+/// from the window's live frame instead of drawing on the display the window+/// just left.+struct CursorRestingAnchor: Equatable, Sendable {+ let windowID: CGWindowID+ let layer: Int+ /// Element centre relative to the window's top-left corner, screen-state space.+ let windowLocalPoint: CGPoint+ /// Window frame the anchor was derived from, screen-state space.+ let windowBounds: CGRect++ /// The anchored point re-expressed against a live window frame.+ ///+ /// Returns `nil` when the live frame carries no usable size, so callers can+ /// fall back to the point they already have.+ func screenStatePoint(liveFrame: CGRect) -> CGPoint? {+ guard liveFrame.width > 0, liveFrame.height > 0 else {+ return nil+ }++ let localX = windowLocalPoint.x.clamped(to: 0...liveFrame.width)+ let localY = windowLocalPoint.y.clamped(to: 0...liveFrame.height)+ return CGPoint(x: liveFrame.minX + localX, y: liveFrame.minY + localY)+ }+}++/// Live frames for target windows.+///+/// `CGWindowListCopyWindowInfo` keeps reporting the pre-move frame for about a+/// second after a window is dragged to another display — measured while fixing+/// this: 83 identical frames at the cursor travel's 120 Hz pump. The+/// accessibility element the snapshot already holds reports the move+/// immediately, so it is the source of truth here and the window list is only a+/// fallback for windows the snapshot never registered.+enum CursorWindowFrameTracker {+ /// Lock-protected box so the tracker can be used from the service thread+ /// and the overlay's main-thread pump alike.+ private final class State: @unchecked Sendable {+ let lock = NSLock()+ var elements: [CGWindowID: AXUIElement] = [:]+ /// Test seam: consulted before any real lookup.+ var override: ((CGWindowID) -> CGRect?)?+ }++ private static let state = State()++ static func register(windowID: CGWindowID?, element: AXUIElement?) {+ guard let windowID, let element else {+ return+ }++ state.lock.lock()+ defer { state.lock.unlock() }+ state.elements[windowID] = element+ }++ static func forget(windowID: CGWindowID) {+ state.lock.lock()+ defer { state.lock.unlock() }+ state.elements.removeValue(forKey: windowID)+ }++ static func forgetAll() {+ state.lock.lock()+ defer { state.lock.unlock() }+ state.elements.removeAll()+ }++ static func installFrameOverrideForTesting(_ body: ((CGWindowID) -> CGRect?)?) {+ state.lock.lock()+ defer { state.lock.unlock() }+ state.override = body+ }++ static func liveFrame(for windowID: CGWindowID) -> CGRect? {+ state.lock.lock()+ let body = state.override+ state.lock.unlock()++ if let body {+ return body(windowID)+ }++ return accessibilityFrame(for: windowID) ?? windowListFrame(for: windowID)+ }++ /// Frame read from the registered accessibility element, which reflects a+ /// move immediately.+ static func accessibilityFrame(for windowID: CGWindowID) -> CGRect? {+ state.lock.lock()+ let element = state.elements[windowID]+ state.lock.unlock()++ guard let element else {+ return nil+ }++ var positionValue: CFTypeRef?+ var sizeValue: CFTypeRef?+ guard AXUIElementCopyAttributeValue(element, kAXPositionAttribute as CFString, &positionValue) == .success,+ AXUIElementCopyAttributeValue(element, kAXSizeAttribute as CFString, &sizeValue) == .success,+ let positionValue,+ let sizeValue+ else {+ return nil+ }++ var position = CGPoint.zero+ var size = CGSize.zero+ guard AXValueGetValue(positionValue as! AXValue, .cgPoint, &position),+ AXValueGetValue(sizeValue as! AXValue, .cgSize, &size),+ size.width > 0,+ size.height > 0+ else {+ return nil+ }++ return CGRect(origin: position, size: size)+ }++ /// Fallback frame from the window server. Kept last because it can report+ /// the pre-move origin for about a second.+ static func windowListFrame(for windowID: CGWindowID) -> CGRect? {+ guard let info = CGWindowListCopyWindowInfo(+ [.optionIncludingWindow, .excludeDesktopElements],+ windowID+ ) as? [[String: Any]],+ let entry = info.first,+ let boundsDictionary = entry[kCGWindowBounds as String] as? [String: Any],+ let frame = CGRect(dictionaryRepresentation: boundsDictionary as CFDictionary),+ frame.width > 0,+ frame.height > 0+ else {+ return nil+ }++ return frame+ }+}diff --git a/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/SoftwareCursorOverlay.swift b/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/SoftwareCursorOverlay.swiftindex 2dfe3ad..791e39d 100644--- a/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/SoftwareCursorOverlay.swift+++ b/packages/OpenComputerUseKit/Sources/OpenComputerUseKit/SoftwareCursorOverlay.swift@@ -192,13 +192,20 @@ enum SoftwareCursorOverlay {private static var restingTipPosition: CGPoint?private static var displayedTipPosition: CGPoint?private static var activeTargetWindow: CursorTargetWindow?+ /// Window frame the current target point was derived from; lets a move+ /// under the cursor be detected and re-derived instead of ignored.+ private static var restingAnchor: CursorRestingAnchor?private static var visualDynamicsState: CursorVisualDynamicsState?private static var idleTimer: Timer?private static var hideTimer: Timer?private static var idlePhase: CGFloat = 0private static var observationPhase = "hidden"- static func moveCursor(to targetPoint: CGPoint, in targetWindow: CursorTargetWindow?) {+ static func moveCursor(+ to targetPoint: CGPoint,+ in targetWindow: CursorTargetWindow?,+ anchor: CursorRestingAnchor? = nil+ ) {guard VisualCursorSupport.isEnabled, canPresentOverlay else {return}@@ -208,7 +215,8 @@ enum SoftwareCursorOverlay {cancelPendingHide()configureOrdering(relativeTo: targetWindow)- let constrainedTarget = clampTipPosition(targetPoint)+ restingAnchor = anchor+ let constrainedTarget = clampTipPosition(liveTargetPoint(targetPoint, anchor: anchor))let isFreshStart = displayedTipPosition == nillet startPoint = displayedTipPosition ?? defaultInitialTipPosition()let now = CACurrentMediaTime()@@ -234,13 +242,20 @@ enum SoftwareCursorOverlay {}}- static func pulseClick(at targetPoint: CGPoint, clickCount: Int, mouseButton: MouseButtonKind, in targetWindow: CursorTargetWindow?) {+ static func pulseClick(+ at targetPoint: CGPoint,+ clickCount: Int,+ mouseButton: MouseButtonKind,+ in targetWindow: CursorTargetWindow?,+ anchor: CursorRestingAnchor? = nil+ ) {guard VisualCursorSupport.isEnabled, canPresentOverlay else {return}configureOrdering(relativeTo: targetWindow)- let constrainedTarget = clampTipPosition(targetPoint)+ restingAnchor = anchor ?? restingAnchor+ let constrainedTarget = clampTipPosition(liveTargetPoint(targetPoint, anchor: anchor))let now = CACurrentMediaTime()seedVisualDynamicsIfNeeded(at: constrainedTarget, time: now)restingTipPosition = constrainedTarget@@ -250,13 +265,18 @@ enum SoftwareCursorOverlay {scheduleHide(after: visualCursorPostInteractionIdleTimeout())}- static func settle(at targetPoint: CGPoint, in targetWindow: CursorTargetWindow?) {+ static func settle(+ at targetPoint: CGPoint,+ in targetWindow: CursorTargetWindow?,+ anchor: CursorRestingAnchor? = nil+ ) {guard VisualCursorSupport.isEnabled, canPresentOverlay else {return}configureOrdering(relativeTo: targetWindow)- let constrainedTarget = clampTipPosition(targetPoint)+ restingAnchor = anchor ?? restingAnchor+ let constrainedTarget = clampTipPosition(liveTargetPoint(targetPoint, anchor: anchor))restingTipPosition = constrainedTargetobservationPhase = "settling"placeCursor(@@ -276,12 +296,67 @@ enum SoftwareCursorOverlay {displayedTipPosition = nilrestingTipPosition = nilactiveTargetWindow = nil+ restingAnchor = nilvisualDynamicsState = nilobservationPhase = "hidden"writeObservationSnapshot(tipPosition: nil, rotation: nil)panel?.orderOut(nil)}+ /// Re-derive a target point that was computed from a snapshot taken before+ /// the window moved. Element frames are window-relative, so the anchored+ /// point follows the window instead of staying on the display it just left.+ private static func liveTargetPoint(_ targetPoint: CGPoint, anchor: CursorRestingAnchor?) -> CGPoint {+ guard let anchor,+ let liveFrame = CursorWindowFrameTracker.liveFrame(for: anchor.windowID),+ liveFrame != anchor.windowBounds,+ let screenStatePoint = anchor.screenStatePoint(liveFrame: liveFrame)+ else {+ return targetPoint+ }++ return screenStatePointToAppKitGlobalPoint(+ fromScreenStatePoint: screenStatePoint,+ screenMappings: currentVisualCursorScreenMappings()+ )+ }++ /// Whether the anchored window has moved since the anchor was taken.+ private static func anchoredWindowMoved() -> Bool {+ guard let anchor = restingAnchor,+ let liveFrame = CursorWindowFrameTracker.liveFrame(for: anchor.windowID)+ else {+ return false+ }++ return liveFrame != anchor.windowBounds+ }++ /// Stop a travel whose window moved under it: land the cursor on the+ /// window's live frame instead of finishing the path towards the display+ /// the window just left.+ private static func abortTravelOntoObservedWindow() {+ guard let anchor = restingAnchor,+ let liveFrame = CursorWindowFrameTracker.liveFrame(for: anchor.windowID),+ let screenStatePoint = anchor.screenStatePoint(liveFrame: liveFrame)+ else {+ return+ }++ let constrainedTarget = clampTipPosition(+ screenStatePointToAppKitGlobalPoint(+ fromScreenStatePoint: screenStatePoint,+ screenMappings: currentVisualCursorScreenMappings()+ )+ )+ restingTipPosition = constrainedTarget+ observationPhase = "reanchored"+ placeCursor(+ using: advanceVisualDynamics(toward: constrainedTarget, at: CACurrentMediaTime()),+ clickProgress: 0+ )+ }+private static var canPresentOverlay: Bool {!NSScreen.screens.isEmpty}@@ -363,6 +438,11 @@ enum SoftwareCursorOverlay {while true {refreshActiveOrderingIfNeeded()+ if anchoredWindowMoved() {+ abortTravelOntoObservedWindow()+ return+ }+let elapsed = CGFloat(CACurrentMediaTime() - startTime)let normalizedElapsed = (elapsed / max(duration, 0.001)).clamped(to: 0...1)let springTime = normalizedElapsed * springTargetDurationdiff --git a/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/CursorWindowAnchorTests.swift b/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/CursorWindowAnchorTests.swiftnew file mode 100644index 0000000..8b0e1a7--- /dev/null+++ b/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/CursorWindowAnchorTests.swift@@ -0,0 +1,74 @@+import CoreGraphics+import XCTest+@testable import OpenComputerUseKit++/// Element frames are window-relative, so a cursor point derived from a+/// snapshot stops being valid the moment the window moves. These numbers are+/// the ones the bug was reproduced with: the same window on the built-in+/// display (`x = -1400`) and on the main display (`x = 600`).+final class CursorWindowAnchorTests: XCTestCase {+ private let builtInDisplayFrame = CGRect(x: -1400, y: 200, width: 500, height: 400)+ private let mainDisplayFrame = CGRect(x: 600, y: 200, width: 500, height: 400)++ private func anchor() -> CursorRestingAnchor {+ CursorRestingAnchor(+ windowID: 4242,+ layer: 0,+ windowLocalPoint: CGPoint(x: 250, y: 200),+ windowBounds: builtInDisplayFrame+ )+ }++ func testAnchorStaysPutWhileTheWindowDoesNotMove() {+ let point = anchor().screenStatePoint(liveFrame: builtInDisplayFrame)+ XCTAssertEqual(point, CGPoint(x: -1150, y: 400))+ }++ /// The reported bug: the window is dragged to the main display while the+ /// cursor is travelling, and the cursor finishes on the display the window+ /// just left.+ func testAnchorFollowsTheWindowOntoTheOtherDisplay() {+ let point = anchor().screenStatePoint(liveFrame: mainDisplayFrame)+ XCTAssertEqual(point, CGPoint(x: 850, y: 400))+ }++ /// Only the frame origin moves; the window-local offset must survive, so a+ /// click still lands on the same control inside the window.+ func testAnchorPreservesTheWindowLocalOffset() {+ let moved = CGRect(x: -1400, y: -600, width: 500, height: 400)+ let point = anchor().screenStatePoint(liveFrame: moved)+ XCTAssertEqual(point, CGPoint(x: -1150, y: -400))+ }++ /// A window that shrank must not put the cursor outside itself.+ func testAnchorClampsIntoASmallerLiveFrame() {+ let shrunk = CGRect(x: 600, y: 200, width: 120, height: 80)+ let point = anchor().screenStatePoint(liveFrame: shrunk)+ XCTAssertEqual(point, CGPoint(x: 720, y: 280))+ }++ /// No usable size means no re-derivation: the caller keeps the point it has+ /// rather than guessing.+ func testAnchorRejectsALiveFrameWithoutSize() {+ XCTAssertNil(anchor().screenStatePoint(liveFrame: .zero))+ XCTAssertNil(anchor().screenStatePoint(liveFrame: CGRect(x: 10, y: 10, width: 0, height: 40)))+ }++ func testTrackerPrefersTheRegisteredLiveFrame() {+ CursorWindowFrameTracker.installFrameOverrideForTesting { windowID in+ windowID == 4242 ? self.mainDisplayFrame : nil+ }+ defer { CursorWindowFrameTracker.installFrameOverrideForTesting(nil) }++ XCTAssertEqual(CursorWindowFrameTracker.liveFrame(for: 4242), mainDisplayFrame)+ XCTAssertNil(CursorWindowFrameTracker.liveFrame(for: 1))+ }++ /// Registering nothing must not produce a frame out of thin air; the+ /// window list is the only remaining source and an unknown id has no entry.+ func testTrackerReportsNothingForAnUnknownWindow() {+ CursorWindowFrameTracker.forgetAll()++ XCTAssertNil(CursorWindowFrameTracker.liveFrame(for: CGWindowID(0)))+ }+}diff --git a/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/OpenComputerUseKitTests.swift b/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/OpenComputerUseKitTests.swiftindex 443772e..564acee 100644--- a/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/OpenComputerUseKitTests.swift+++ b/packages/OpenComputerUseKit/Tests/OpenComputerUseKitTests/OpenComputerUseKitTests.swift@@ -2006,11 +2006,20 @@ final class OpenComputerUseKitTests: XCTestCase {screenMappings: screenMappings)+ // The target also carries the anchor: the element centre relative to+ // the window frame it was derived from, which is what lets the overlay+ // follow the window if it moves before the action lands.XCTAssertEqual(target,VisualCursorTarget(point: CGPoint(x: 484, y: 724),- window: CursorTargetWindow(windowID: 321, layer: 8)+ window: CursorTargetWindow(windowID: 321, layer: 8),+ anchor: CursorRestingAnchor(+ windowID: 321,+ layer: 8,+ windowLocalPoint: CGPoint(x: 84, y: 56),+ windowBounds: CGRect(x: 400, y: 220, width: 900, height: 640)+ )))}diff --git a/skills/open-computer-use/references/usage.md b/skills/open-computer-use/references/usage.mdindex 40a896e..5c8426c 100644--- a/skills/open-computer-use/references/usage.md+++ b/skills/open-computer-use/references/usage.md@@ -228,6 +228,18 @@ Every non-fixture `drag` result includes a text item that begins `Drag deliveredWhen the gate is not enabled, treat window-server drags as unavailable and reach the same outcome another way: copy or move files with a shell command instead of a Finder drag, use `set_value` or keyboard selection instead of drag-selecting text, and use the app's own window controls instead of dragging a title bar.+## Software Cursor++The software cursor follows the target window when it moves. Its point is derived from the snapshot, so it+also carries an anchor: the same point expressed relative to the window frame the snapshot reported. Before+drawing, and on every frame of a travel, the overlay re-derives the point from the window's live frame, so a+window dragged to another display mid-action takes the cursor with it instead of leaving it on the display+the window just left.++The live frame comes from the window's accessibility element — the element the snapshot already holds — and+only falls back to `CGWindowListCopyWindowInfo`, which can keep reporting the pre-move origin for about a+second (measured: 83 identical frames at the travel's 120 Hz pump).+## Platform Notes### macOS