Skip to content

armgit / cua

Public Imported from GitHub
0

Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.

test(cua-driver)(windows): pin PostMessage LPARAM coord packing for negative client-area coords (#1979) (#1983)

Francesco Bonaccieb15e8e32276
diff --git a/libs/cua-driver/rust/crates/platform-windows/src/input/mouse.rs b/libs/cua-driver/rust/crates/platform-windows/src/input/mouse.rs
index 1cad286fff..7f8100c7ea 100644
--- a/libs/cua-driver/rust/crates/platform-windows/src/input/mouse.rs
+++ b/libs/cua-driver/rust/crates/platform-windows/src/input/mouse.rs
@@ -223,8 +223,35 @@ pub fn post_drag_screen(
}
/// Pack two 16-bit integers into a LPARAM (low word = x, high word = y).
+///
+/// Delegates the bit-math to [`crate::lparam::pack_xy`] so the
+/// receiver-side `GET_X_LPARAM` / `GET_Y_LPARAM` sign-extension contract is
+/// covered by cross-platform unit tests (see #1979's audit: the PostMessage
+/// path packing was a suspect for the multi-monitor wrong-screen symptom,
+/// turned out to be correct, and now has regression coverage).
+///
+/// On the (currently unreachable) out-of-range path we log + clamp rather
+/// than panic — every existing call site passes post-`ScreenToClient`
+/// window-local coords that fit in `i16` by construction, but if a future
+/// caller passes a raw screen coord on a >32k-px virtual desktop, clamping
+/// is at least visible in the log instead of silently wrapping.
fn make_lparam(x: i32, y: i32) -> LPARAM {
- LPARAM((((y as u16 as u32) << 16) | (x as u16 as u32)) as isize)
+ match crate::lparam::pack_xy(x, y) {
+ Ok(packed) => LPARAM(packed as isize),
+ Err(err) => {
+ tracing::warn!(
+ target: "click",
+ "make_lparam: {err}; clamping to i16 range. \
+ If you see this, the caller is passing non-window-local coords."
+ );
+ let clamp = |v: i32| v.clamp(i16::MIN as i32, i16::MAX as i32);
+ let cx = clamp(x);
+ let cy = clamp(y);
+ let packed = crate::lparam::pack_xy(cx, cy)
+ .expect("clamped values always fit in i16 range");
+ LPARAM(packed as isize)
+ }
+ }
}
/// Returns `true` when `hwnd` is a top-level frame of a Chromium-based browser
diff --git a/libs/cua-driver/rust/crates/platform-windows/src/lib.rs b/libs/cua-driver/rust/crates/platform-windows/src/lib.rs
index c087b58ee8..3addf38d7e 100644
--- a/libs/cua-driver/rust/crates/platform-windows/src/lib.rs
+++ b/libs/cua-driver/rust/crates/platform-windows/src/lib.rs
@@ -32,6 +32,14 @@ pub mod terminal;
// where the multi-monitor normalization bug was diagnosable by pure math.
pub mod virtualdesk;
+// Cross-platform: pure math for packing `(x, y)` into the `LPARAM` payload
+// every `WM_MOUSE*` / `WM_*BUTTON*` message carries. Lives outside the
+// Windows-only `input` module for the same reason as `virtualdesk` — the
+// receiver-side `GET_X_LPARAM` sign-extension contract is a pure-math
+// property we want to pin with cross-platform unit tests (the #1979 audit
+// pass that lives in this commit).
+pub mod lparam;
+
#[cfg(target_os = "windows")]
pub mod win32;
diff --git a/libs/cua-driver/rust/crates/platform-windows/src/lparam.rs b/libs/cua-driver/rust/crates/platform-windows/src/lparam.rs
new file mode 100644
index 0000000000..0042d18d0a
--- /dev/null
+++ b/libs/cua-driver/rust/crates/platform-windows/src/lparam.rs
@@ -0,0 +1,282 @@
+//! Pure math for packing/unpacking `LPARAM` coordinate pairs in window
+//! messages — `WM_MOUSEMOVE`, `WM_LBUTTONDOWN`, `WM_RBUTTONUP`, and friends.
+//!
+//! Windows packs the click `(x, y)` into a single `LPARAM` (`MAKELPARAM` in
+//! `windowsx.h`): low 16 bits hold `x`, high 16 bits hold `y`. The receiver
+//! extracts with `GET_X_LPARAM` / `GET_Y_LPARAM`, which sign-extend each half
+//! via `(int)(short)LOWORD(lp)` — so negative client-area coordinates
+//! (legitimately produced by `ScreenToClient` for clicks on a window that
+//! straddles a monitor edge, or for non-client clicks above the title bar)
+//! round-trip back to the same `i32` value, as long as each coordinate fits in
+//! `i16` range.
+//!
+//! This module exists alongside [`crate::virtualdesk`] for the same reason:
+//! the coordinate-handoff is the kind of pure math that needs cross-platform
+//! unit tests on every host (issue #1979 — the agent reported correct
+//! screen-points being delivered to the wrong monitor on multi-monitor setups
+//! with negative virtual-desktop X origins, so we pin both the SendInput
+//! [VIRTUALDESK normalization] and the PostMessage [LPARAM packing] paths
+//! with regression tests on the macOS CI host).
+//!
+//! Audit conclusion for #1979: the existing PostMessage packing is correct
+//! for every screen / client coordinate that fits in `i16` range, including
+//! the reporter's `(-1198, 292)` and `(-1795, 383)` examples — the round-trip
+//! through `GET_X_LPARAM` / `GET_Y_LPARAM` recovers the input exactly. The
+//! tests below lock that invariant in so a future "simplification" (e.g.
+//! dropping the implicit `i16` truncation and going `x as u32` directly)
+//! cannot regress it without turning a test red.
+
+/// Pack a `(x, y)` coordinate pair into the 32-bit `LPARAM` payload used by
+/// every `WM_MOUSE*` and `WM_*BUTTON*` message: `x` in the low 16 bits,
+/// `y` in the high 16 bits, each truncated to `i16` width.
+///
+/// Returned as `u32` — the caller is responsible for the final `LPARAM`
+/// (`isize`) cast. Splitting the pure-bit-math from the Win32 `LPARAM`
+/// newtype lets this run in `cargo test` on any host.
+///
+/// Returns `Err` when either coordinate falls outside the `i16` range
+/// `[-32768, 32767]`. Win32 packs the LPARAM as two `WORD`s (16 bits each),
+/// so a coordinate outside that range would silently truncate to a different
+/// value on the receiver — and the receiver has no way to distinguish that
+/// from a legitimate small-magnitude coord. Every existing call site in
+/// `mouse.rs` passes client-area coords that come out of `ScreenToClient`
+/// (window-local, capped at the window's extents) so they're always inside
+/// `i16` range in practice — the explicit fail-fast is the cheap insurance
+/// against a future caller passing screen-direct coords on a >32k-pixel
+/// virtual desktop.
+#[inline]
+pub fn pack_xy(x: i32, y: i32) -> Result<u32, LParamPackError> {
+ if !fits_in_i16(x) {
+ return Err(LParamPackError::XOutOfRange(x));
+ }
+ if !fits_in_i16(y) {
+ return Err(LParamPackError::YOutOfRange(y));
+ }
+ // `i32 as u16` truncates to the low 16 bits, preserving the two's-complement
+ // bit pattern of any i16-fitting value. The receiver re-interprets each
+ // half as `(short)` (signed 16-bit) — see `unpack_x` / `unpack_y`.
+ Ok(((y as u16 as u32) << 16) | (x as u16 as u32))
+}
+
+/// Errors that [`pack_xy`] can surface. Currently only out-of-range coord
+/// errors; carries the bad value for diagnostics.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub enum LParamPackError {
+ /// `x` cannot be represented as `i16`. Carries the offending value.
+ XOutOfRange(i32),
+ /// `y` cannot be represented as `i16`. Carries the offending value.
+ YOutOfRange(i32),
+}
+
+impl core::fmt::Display for LParamPackError {
+ fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
+ match self {
+ Self::XOutOfRange(v) => write!(
+ f,
+ "LPARAM pack: x={v} is outside i16 range [-32768, 32767]; \
+ PostMessage coords must be window-local (post-ScreenToClient)"
+ ),
+ Self::YOutOfRange(v) => write!(
+ f,
+ "LPARAM pack: y={v} is outside i16 range [-32768, 32767]; \
+ PostMessage coords must be window-local (post-ScreenToClient)"
+ ),
+ }
+ }
+}
+
+impl std::error::Error for LParamPackError {}
+
+#[inline]
+fn fits_in_i16(v: i32) -> bool {
+ (i16::MIN as i32..=i16::MAX as i32).contains(&v)
+}
+
+/// Mirror of Win32's `GET_X_LPARAM` macro: `(int)(short)LOWORD(lp)`. Used by
+/// the round-trip tests; the production code never decodes an LPARAM it
+/// packed itself.
+#[inline]
+#[cfg(test)]
+pub fn unpack_x(lp: u32) -> i32 {
+ (lp & 0xFFFF) as i16 as i32
+}
+
+/// Mirror of Win32's `GET_Y_LPARAM` macro: `(int)(short)HIWORD(lp)`.
+#[inline]
+#[cfg(test)]
+pub fn unpack_y(lp: u32) -> i32 {
+ ((lp >> 16) & 0xFFFF) as i16 as i32
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ /// Round-trip every coordinate in `cases` through `pack_xy` and the
+ /// `GET_X_LPARAM` / `GET_Y_LPARAM` mirrors. The packing contract is that
+ /// any `i16`-fitting input is recovered exactly by the receiver.
+ fn assert_round_trips(cases: &[(i32, i32)]) {
+ for &(x, y) in cases {
+ let lp = pack_xy(x, y).expect("inputs all fit in i16");
+ let rx = unpack_x(lp);
+ let ry = unpack_y(lp);
+ assert_eq!(
+ (rx, ry),
+ (x, y),
+ "round-trip mismatch: in=({x}, {y}) lp=0x{lp:08x} out=({rx}, {ry})"
+ );
+ }
+ }
+
+ #[test]
+ fn reporter_negative_coords_round_trip() {
+ // The two coordinate pairs the reporter saw in #1979's PostMessage path
+ // log lines:
+ // "Posted double-click to [10] at screen-point (-1198, 292)"
+ // "Posted double-click to pid X at window-pixel (38, 40) → screen-point (-1795, 383)"
+ //
+ // The screen-point form goes through post_click_screen → deepest_child
+ // → ScreenToClient before reaching make_lparam, so by the time we pack
+ // the coords they're window-local and almost always small positives —
+ // but on a child window that straddles the monitor edge, ScreenToClient
+ // can produce a small negative for the non-client side. Lock both signs
+ // in.
+ assert_round_trips(&[
+ (-1198, 292),
+ (-1795, 383),
+ (38, 40),
+ (-39, 383), // representative post-ScreenToClient coord on the
+ // secondary-monitor window edge
+ ]);
+ }
+
+ #[test]
+ fn full_i16_range_round_trips() {
+ // Both signs at the i16 fence-posts. If a future change accidentally
+ // drops the implicit i16 cast (e.g. someone "simplifies" `x as u16` to
+ // `(x as u32) & 0xFFFF`, which evaluates the same for valid inputs but
+ // hides intent for the next reader), the round-trip still holds inside
+ // the i16 range — what we'd want catching here is a sign-flip at the
+ // boundary, which `i16::MIN ↔ i16::MAX` exercises.
+ assert_round_trips(&[
+ (i16::MIN as i32, 0),
+ (i16::MAX as i32, 0),
+ (0, i16::MIN as i32),
+ (0, i16::MAX as i32),
+ (i16::MIN as i32, i16::MAX as i32),
+ (i16::MAX as i32, i16::MIN as i32),
+ (-1, -1),
+ (1, 1),
+ ]);
+ }
+
+ #[test]
+ fn out_of_range_coords_rejected() {
+ // Outside i16, packing must surface an error rather than silently
+ // truncating — the truncated bit pattern is indistinguishable on the
+ // receiving side from a legitimate small value, so a silent truncation
+ // would be the worst possible outcome.
+ let bad_x = pack_xy(i16::MAX as i32 + 1, 0);
+ assert!(matches!(bad_x, Err(LParamPackError::XOutOfRange(v)) if v == 32768));
+ let bad_neg_x = pack_xy(i16::MIN as i32 - 1, 0);
+ assert!(matches!(bad_neg_x, Err(LParamPackError::XOutOfRange(v)) if v == -32769));
+ let bad_y = pack_xy(0, 65000);
+ assert!(matches!(bad_y, Err(LParamPackError::YOutOfRange(v)) if v == 65000));
+ }
+
+ #[test]
+ fn round_trip_negative_x_does_not_sign_flip_to_primary_monitor() {
+ // The structural worry behind #1979's PostMessage half: a packing bug
+ // that maps a negative-X click (secondary-monitor window-local coord)
+ // to a positive X (primary-monitor coord) would manifest as the agent
+ // reporting "click at (-1198, 292)" while the actual click lands on
+ // the primary monitor.
+ //
+ // The receiver-side macro `GET_X_LPARAM` sign-extends via
+ // `(int)(short)LOWORD(lp)`. As long as our packed `x` keeps its
+ // two's-complement bit pattern in the low 16 bits, the receiver
+ // recovers the original negative value. This test fails if anyone
+ // ever replaces the implicit `i16` truncation with a zero-extending
+ // cast (e.g. `(x as u32) & 0xFFFF` from an `i32::abs()`-style change).
+ for x in [-32768_i32, -10000, -1795, -1198, -1, 0, 1, 1000] {
+ let lp = pack_xy(x, 0).unwrap();
+ let rx = unpack_x(lp);
+ assert_eq!(rx, x, "sign-flip: x={x} round-tripped to {rx}");
+ // And the same `x` must produce a strictly smaller LPARAM's low
+ // word when its neighbour does — i.e. the bit pattern is
+ // monotone in the i16 sense. We can check that by re-decoding
+ // adjacent samples.
+ if x < i16::MAX as i32 {
+ let lp2 = pack_xy(x + 1, 0).unwrap();
+ let rx2 = unpack_x(lp2);
+ assert_eq!(rx2, x + 1, "monotonicity decode failed");
+ }
+ }
+ }
+
+ #[test]
+ fn screen_coord_to_window_local_then_pack_monotonic() {
+ // Approximate the full coord-translation pipeline for the reporter's
+ // window: a window at screen origin (-1834, 0) (the DWM frame the
+ // reporter's `bitmap_to_screen` math implies). For two adjacent screen
+ // pixels (sx, y) and (sx+1, y), the window-local x increases by 1, so
+ // the packed LPARAM's low word (when re-interpreted as i16) must also
+ // increase by 1.
+ //
+ // This mirrors the #1980 `monotonic_in_x_and_y` test for the SendInput
+ // path: any sign-mangling bug that folded negatives back into the
+ // positive band would break the monotonicity here. Adjacent screen
+ // pixels must produce adjacent packed coords for any sx along the
+ // secondary monitor (`sx in [-1834, -915)`).
+ let window_origin_x: i32 = -1834;
+ let window_origin_y: i32 = 0;
+ // Screen pixels sweeping across the secondary monitor (1920 px wide).
+ let mut prev: Option<i32> = None;
+ for sx in window_origin_x..(window_origin_x + 1920) {
+ let local_x = sx - window_origin_x; // 0..1920 — well inside i16
+ let local_y = 383 - window_origin_y;
+ let lp = pack_xy(local_x, local_y)
+ .expect("post-ScreenToClient coords always fit in i16");
+ let rx = unpack_x(lp);
+ if let Some(p) = prev {
+ assert_eq!(
+ rx,
+ p + 1,
+ "non-monotonic at sx={sx}: prev decoded x={p}, current decoded x={rx}"
+ );
+ }
+ prev = Some(rx);
+ }
+ }
+
+ #[test]
+ fn full_reporter_scenario_screen_minus_1795_through_postmessage_pipeline() {
+ // End-to-end reconstruction of the reporter's `(-1795, 383)` path:
+ //
+ // 1. Agent passes window_id with DWM frame at screen X=-1834 and
+ // bitmap pixel (38, 40) → screen (-1795, 383) (per bitmap_to_screen).
+ // 2. post_click_screen → deepest_child → ScreenToClient(window, (-1795, 383))
+ // = (39, 383) (window-local; positive; well inside i16).
+ // 3. ChildWindowFromPointEx finds the deepest descendant; we'll assume the
+ // target is the same window (a single-HWND app) for this pure-math test.
+ // 4. make_lparam(39, 383) → LPARAM payload.
+ // 5. Receiver decodes via GET_X_LPARAM / GET_Y_LPARAM.
+ //
+ // The assertion: the decoded coords must match step (2) exactly. Any
+ // sign-flip or truncation in steps (4)-(5) would land the click on the
+ // wrong screen — which is the reporter's symptom.
+ let window_screen_left: i32 = -1834;
+ let click_screen_x: i32 = -1795;
+ let click_screen_y: i32 = 383;
+ let local_x = click_screen_x - window_screen_left; // 39
+ let local_y = click_screen_y; // window top at y=0
+ assert_eq!(local_x, 39);
+ let lp = pack_xy(local_x, local_y).unwrap();
+ let (rx, ry) = (unpack_x(lp), unpack_y(lp));
+ assert_eq!(
+ (rx, ry),
+ (39, 383),
+ "PostMessage LPARAM packing dropped the reporter's coords on the floor"
+ );
+ }
+}