Skip to content

armgit / cua

Public Imported from GitHub
0

Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.

test(cua-driver): qualify the Omarchy Edge kit for Driver 0.33.2 (#4395)

Francesco Bonacci7d69b8d80ccf
diff --git a/libs/cua-driver/hyprland-plugin/docs/omarchy-edge-20261001-validation.md b/libs/cua-driver/hyprland-plugin/docs/omarchy-edge-20261001-validation.md
new file mode 100644
index 0000000000..ddf2d51bcb
--- /dev/null
+++ b/libs/cua-driver/hyprland-plugin/docs/omarchy-edge-20261001-validation.md
@@ -0,0 +1,166 @@
+# Omarchy Edge profile candidate: 2026-10-01
+
+Refs [#4216](https://github.com/trycua/cua/issues/4216).
+
+Superseded by the [2026-10-04 kit qualification](omarchy-edge-20261004-validation.md). This record documents the earlier candidate and is kept for history.
+
+This is an in-progress qualification record, not a release or compatibility
+claim. The target is an upgraded disposable x86_64 Omarchy Edge guest, not a
+fresh ISO installation or Omarchy's independent Omabot acceptance run.
+
+## Candidate identity
+
+- Driver source: `5272e492d61b96caf08e3bf434d91126c1f3dccc` (`0.31.0`).
+- Profile tooling: `54805907461376c8574996583b948500ce038d1a`, merged in #4383.
+- Hyprland: `0.56.2-3`; Aquamarine: `0.15.1-1`.
+- GCC: `16.2.1 20260810`; package `16.2.1+r23+gd564253eb6c8-1`.
+- Inkscape: `1.4.4-6`. Use the documented Inkscape-only app profile;
+ installed LibreOffice `26.8.0-2` is outside background admission.
+- Profile SHA-256: `a7289e169914eda41ee4bb49e2077c5e4af36cb228390cd9104f2a24ceb02891`.
+- Kit provenance SHA-256: `07f9f2462af9db1b5d8d076212eb4e26c34c5fcec4262e7802a19a82b720fe8d`.
+- Trace-disabled module SHA-256: `1469e1f8aaf141199b4080b5bd755d68e56e8eac245730e7f893c717519d72cb`.
+
+## Evidence so far
+
+The reviewed-profile recipe builds without rewriting the source archive or
+relaxing ABI checks. Its bundled CTests pass. The consumer verifier passes
+after installation, and a fresh compositor loads the module with matching
+compiled/runtime ABI hashes and enabled input endpoints.
+
+The isolated ALPM lifecycle runner passes install, removal, reinstallation,
+and rejection of a mismatched Hyprland dependency. Its rebuilt module matches
+the installed module hash. The package archives have distinct identities:
+the initial package is `02fe5dc2293eb5b66a7ec9c837d1f9a5dd149aff624ffdd4d82d218ecef6fbc4`,
+and the lifecycle rebuild is `c689cc41a9cbece8b67cdfff18074269dac21c6ea745fe789484301753586d0d`.
+Do not substitute one archive's lifecycle evidence for the other's native proof.
+
+The bounded single-app production smoke now passes. Initial attempts stopped before raw
+input because the default accessibility walk was truncated or the grounding
+expected absent closed-menu children. The repaired harness requires a complete
+15-second-budget observation, prepares the Objects panel through accessibility,
+and accepts exact semantic geometry labels without relaxing numeric values.
+An intermediate attempt selected the rectangle, moved it two pixels right,
+and saved the SVG, but failed its immediate post-save identity observation
+during a compositor/AT-SPI title transition. Saved-file evidence alone did not
+pass the smoke. The native replay with bounded observation-only retries passes
+strict identity, semantic geometry and saved SVG checks on the published Driver
+and trace-disabled module. This proves the recorded single-app operations, not
+two-lane isolation or concurrency. Failures after raw input are reported as failed,
+not inspection-only. The portable harness suite passes 623 tests, with two skips.
+
+The unchanged complete native Linux baseline finished with failures in Tauri
+page scrolling, keyboard-first cursor placement, GTK3 pixel geometry, desktop
+scope, and encrypted history setup. Product repairs are isolated in #4396;
+the guest function-key helper and Secret Service setup were also corrected.
+The manual foreground safety suite passed three tests. The updated manual
+observation test starts behavioral recording; its latest replay stopped before
+input because two fixture windows overlapped with ambiguous compositor z-order.
+A non-overlapping fixture layout is required for that separate replay.
+
+The first repaired candidate, `420c87d87fbd423d5f9da098b437946bc13098ea`,
+passes the complete hosted X11 matrix and every native Hyprland behavioral
+suite, including GTK3 37/37, capture 4/4, desktop scope 5/5 and encrypted history.
+Its final native evidence validator fails because an intentional stale-zoom
+refusal was classified as an unverifiable click and expected a click marker.
+The shared producer fix in candidate
+`e20386e4551c791c66a61e7e4693909cfdda430a` preserves legacy fields and reports
+refused-before-dispatch; the validator is unchanged. That exact candidate passes
+the complete native Hyprland Linux runner, including final video validation:
+90 delivered, 42 expected refusals, zero failed or skipped cases. The complete
+hosted X11 and Windows gates also pass. Hosted macOS passes as supplemental
+evidence; the canonical Lume gate remains in progress.
+Final manual foreground safety passes 3/3 and native observation passes 1/1
+with behavioral video after the fixture windows are arranged without overlap.
+Focused XWayland geometry on
+the prior candidate matches an independent AT-SPI Screen oracle and delivers
+a background accessibility click.
+
+Two-lane production setup exposed an observer assumption that rejected a stable
+primary locked-modifier mask despite no held keys. The proof now records that
+baseline mask, still refuses depressed/latched modifiers and nonzero groups,
+and still rejects every keyboard event during the interval, including a lock
+change and return. The native production replay passes with two independent
+Inkscape processes and Driver runtimes: each selects, moves and saves its own
+rectangle, and both saved SVGs verify a two-pixel horizontal translation.
+Independent primary-client observation finds no interference. The separate
+negative control detects a deliberate pointer excursion and return. This does
+not substitute for diagnostic overlap, compositor attribution or cleanup proof.
+
+The first diagnostic overlap attempt stops before dragging: semantic Inkscape
+geometry labels did not match the legacy numeric-label parser, and the Fill and
+Stroke panel made the blue-rectangle pixel oracle ambiguous. The parser now
+accepts only exact axis-specific semantic labels while retaining cross-projection,
+numeric, visibility and uniqueness checks. Fresh setup closes the unrelated
+panel through accessibility. A later run verifies both drags and saved outputs,
+but fails final attribution because identical window-local coordinates match
+both lanes. Harness `1b4c066ce` binds each agent to the lane independently
+observed during its serial selection and verifies retained epoch/reservation
+before subsequent input. A fresh run passes with 1501 ms of drag overlap,
+per-lane wire endpoints, saved SVG effects, primary isolation and cleanup.
+Capacity also passes: two persistent owners are admitted and a third receives
+`lane_busy` without dispatch. Runtime cancellation passes with the sibling
+finishing and synthetic input released; reacquisition remains unproven.
+The integrated portable suite passes 628 tests with two skips.
+
+The replacement Edge guest has Mesa `1:26.2.3-2`, while the first profile pins
+`1:26.2.3-1`. The old recipe correctly refuses that mismatch. The newly measured
+profile differs only in that runtime package plus its explicit labels:
+`omarchy-edge-20261001-mesa2`, kit `1.2.1`, package release `3`.
+Its profile digest is
+`d3ff786deb7bbde5d2368dcb58966d9ddf0c705c37e51d0aac60901d7d983c84`;
+kit provenance is `28a3ccebccb539fc625b972082d3426f41a403e872d358b8eff130a4f80913f8`.
+The rebuilt production module is byte-identical to the original module, but
+the package is distinct:
+`fa68bd906e8bfe6473d790d28728e35efad978f548f48cf61d08a668d3841a30`.
+Its live install, upgrade to release 4, rollback to release 3, removal and
+reinstallation pass with the compositor stopped for every package mutation.
+Fresh sessions after each installed stage map the production module with matching
+ABI and enabled input. Removal leaves no module or loaded plugin; the retained
+test-only enable setting reports an unknown key until reinstallation and
+deliberate activation clear it. The current-profile production smoke passes
+on published Driver 0.31.0 bytes, with strict semantic and saved-SVG readback.
+Full reboot verification passes on kernel `7.2.7-arch1-1`: a new boot identity,
+fresh compositor activation, matching ABI, enabled input, and the consumer
+verifier all pass. The post-reboot production smoke also passes on published
+Driver 0.31.0, including semantic readback and the saved two-pixel SVG translation.
+The complete unchanged Linux runner also passes on the updated Mesa and kernel
+with exact candidate `e20386e4551c791c66a61e7e4693909cfdda430a`: 90 delivered,
+42 expected refusals, zero failures, and zero skips. Final video validation
+passes. The retained private evidence archive has SHA-256
+`34518c7cfc25653d299611e22763e3ceb473cb3a588aea58b725fafde76a6269`.
+Published-source package smoke and candidate Driver matrix results remain
+separately identified; neither changes the source contained in the package.
+
+## Remaining gates
+
+Additional diagnostic fault runs on the replacement guest pass primary-client
+initial refusal with a new-action recovery, and same-client passive-hover
+agent conflict with owner recovery. These do not establish active-drag primary
+conflict, active-lease contention, other-lane recovery, or same-process sibling
+window behavior. Separate floating-window move and resize episodes pass
+mid-drag cancellation, own-seat input release, continuous primary isolation,
+and freshly grounded recovery. Initial target-destruction episodes fail because
+the tiled replacement changes bounds after target removal. A fresh episode with
+fixed, non-overlapping floating windows passes exact-target destruction during
+a drag, connection retirement, and a new action on a prepared distinct process.
+The test uses an advertised 2560x1080 virtual display mode because each Inkscape
+window expands to a 1078-pixel minimum width with the fixture panels open.
+Same-client recovery, PID/address reuse and active-sibling behavior remain
+unproven by this bounded case.
+
+A second current-environment package, release `4`, builds with all 20 CTests
+passing. Its measured profile differs from release `3` only in explicit labels
+and package revision. Package SHA-256 is
+`4bf905f05590a9f2bd4b25fa7d302c63dc17c71ebd2bd726b67235248d561982`;
+kit provenance is `0947837380c9af82ca9dc0d2ac4f7b1e0238a74f7677495fee4685278c8af0d3`.
+Its live upgrade and rollback checks pass as described above; it is not a
+published release asset.
+
+- Canonical Lume macOS verification of the shared refusal-metadata repair.
+ Canonical hosted Linux and Windows pass; hosted macOS is supplemental only.
+- Fault evidence is limited to the cases stated above; it is not arbitrary
+ application or physical-hardware certification.
+- Bind any published kit/package to the exact passing evidence.
+- Omarchy-owned downstream acceptance and package promotion.
+
+No native-certified profile or release asset is published by this record.
diff --git a/libs/cua-driver/hyprland-plugin/docs/omarchy-edge-20261004-validation.md b/libs/cua-driver/hyprland-plugin/docs/omarchy-edge-20261004-validation.md
new file mode 100644
index 0000000000..471978c58a
--- /dev/null
+++ b/libs/cua-driver/hyprland-plugin/docs/omarchy-edge-20261004-validation.md
@@ -0,0 +1,128 @@
+# Omarchy Edge kit qualification: 2026-10-04
+
+Refs [#4216](https://github.com/trycua/cua/issues/4216). This supersedes the
+in-progress [2026-10-01 record](omarchy-edge-20261001-validation.md).
+
+The target is an upgraded, disposable x86_64 Omarchy Edge guest. It is not a
+fresh ISO installation or Omarchy's Omabot acceptance run.
+
+## Identity
+
+| Item | Value |
+| --- | --- |
+| Driver release | [`cua-driver-rs-v0.33.2`](https://github.com/trycua/cua/releases/tag/cua-driver-rs-v0.33.2), source `c82d32e3e1adbc6578a148962002ddf6e3e8a15a` |
+| Plugin source archive | `cua-hyprland-plugin-0.33.2-c82d32e3e1adbc6578a148962002ddf6e3e8a15a.tar.gz`, SHA-256 `79596a9a5e7300546ad4ab0646430edcf3e0c0456a541458de2a52b8c44a8fad` (matches the published asset digest) |
+| Packaging tooling | `c82d32e3e1adbc6578a148962002ddf6e3e8a15a` |
+| Profile | [`omarchy-edge-20261004`](../packaging/release/profiles/omarchy-edge-20261004.json), kit `1.3.0`, package release `2`, SHA-256 `0ada2c227d0817b0228e9d8127e2363a785c13175a9b74fbebaedc9825d3a4fd` |
+| `KIT-PROVENANCE.json` | SHA-256 `e1c1653045149af9e1fba077a5bd91008c3178a180e62ac06dac9f19c6db5fbe` |
+| Kit archive | SHA-256 `ac560bba4bc656004564fd9cc09daea1206402148825ca7d7ac27ccdd56345b1` |
+| Production package | `cua-hyprland-plugin-0.33.2-2-x86_64.pkg.tar.zst`, SHA-256 `669773904b972855cf9d08b181d6c13b2b8ddc1726f22c2e4e2ff28da970e7b3` |
+| Module | SHA-256 `9cc1ea1bb7fe2b025f9980c38b142cab1ffd0e13a3d74cf6fbdfc36e95aba837`; production input on, experimental input and tracing off |
+| Published Driver binary | `cua-driver-rs-0.33.2-linux-x86_64-binary.tar.gz`, SHA-256 `845d0c4eb15d9baaf1343e53074dbfbbef5adcc4e415526b929e549d250f221b`; executable `a818162f9c997548473598b9c73081365830156360e3f6e4a77723ab47f5e78e` |
+| Proof harness | `bbea0acf875bfe113f052fa65720e05880b21ecd` (this branch before the main merge) |
+
+## Environment
+
+Measured on Omarchy Edge, 2026-10-04:
+
+- **Compositor:** Hyprland `0.56.2-4` with headers `0.56.2`. The header-inventory hash is `1fdefe6ac027a159d04a5dfee4928ec7ebd15544a9a25b5f66d2f5a46fcf364a`, the same value Omarchy measured for its own `-4` profile.
+- **Compiler:** GCC `16.2.1 20260810`.
+- **Runtime:** Aquamarine `0.15.1-1`, glibc `2.44+r50+g1848099f063e-1`, Mesa `1:26.2.4-1`. The profile pins all 77 runtime ABI package versions.
+- **Session:** kernel `7.2.8-arch1-2`, display 1280x800.
+- **Application:** Inkscape `1.4.4-6`.
+
+Against the 2026-10-01 profile, Edge changed Hyprland (`-3` to `-4`), Mesa,
+glslang, spirv-tools and harfbuzz, so a new profile was required. The compiler
+is unchanged.
+
+## Evidence
+
+All results below are for the identities above.
+
+1. **Build and isolated lifecycle.** `lifecycle.py` passes: the native recipe, all 20 bundled CTests, install, removal and reinstallation in isolated ALPM roots, and the paired dependency-refusal control. Its rebuild (package `a790fd7a…`) maps the same module bytes as the production package.
+2. **Live package transactions**, each with SDDM stopped and a fresh session verified afterwards. In every fresh session the exact module is mapped, the ABI matches, input protocol 3 is transport-ready, and there are no config errors:
+ - Install `0.33.2-2` over Omarchy's own signed `cua-hyprland-plugin 0.32.0-3`.
+ - Roll back to that signed `0.32.0-3` package.
+ - Upgrade to `0.33.2-2` again.
+ - Remove: no module is mapped. The sandbox image's own plugin configuration reports one unknown-key error until reinstallation.
+ - Reinstall `0.33.2-2`.
+ - Reboot: new boot ID, fresh activation.
+3. **Consumer verifier.** `profile_verify.py --consumer` with the reviewed provenance digest passes after the install and after the reboot.
+4. **Production single-app smoke** with the published Driver binary and the installed module passes before and after the reboot. A separate GTK foreground fixture holds focus. Inkscape is driven in the background, and its saved SVG verifies a two-pixel move of the rectangle.
+5. **Two-lane production proof** with the published Driver binary, the trace-disabled module, two independent Inkscape processes and two Driver runtimes passes:
+ - Select, move and save run in parallel across the two lanes, six actions in total.
+ - Both saved SVGs verify the move.
+ - The independent parked-primary observer records zero violations.
+ - The negative control detects a deliberate primary pointer excursion.
+ - Continuous isolation and synthetic-input cleanup stay unproven on this trace-disabled path; they belong to the instrumented diagnostics.
+6. **Complete native Linux runner** (`scripts/ci/linux/run-rust-e2e.sh`, all suites) passes with the kit module loaded and the Driver built from the tag source: 90 delivered, 42 expected refusals, 0 failed, 0 skipped. The manual foreground-safety (3/3) and native-observation (1/1) gates also pass.
+7. **Driver certification.** The Hyprland fixes in 0.33.2 came from #4396. That PR's candidate passed the complete native runner on this Edge environment, together with Omarchy's `0.32.0-3` plugin. It also passed the hosted Linux, Windows and macOS lanes. The canonical Lume gate found no regression against `main`; #4607 tracks the one test that still fails there. The 0.33.2 release gate passed.
+
+Disclosed setup and harness issues:
+
+- The first production smoke stopped as `inspection_only` because no separate foreground window was active.
+- The first attempt in a cold session, and the first after the reboot, failed the smoke's 10-second `git status` provenance check before any input. That check is slow on the guest's partial clone. A warm rerun passed.
+- The native runner needed an unlocked Secret Service keyring, as the hosted lanes use, and the sentinel `no_anim` rule from the test-harness guide.
+- The background smoke also used the Inkscape `no_initial_focus` fixture rule.
+
+No assertion was changed. Evidence archives are retained privately:
+
+- native runner: `8d5c83caf5d3bdfa684ac70bc30f7aa78a53044c96aaae4b331678bafe77cc5e`
+- live transactions, smokes and two-lane proof: `43ec60db9210563ca455f64803a1dff3301a3d249945106a32e56a42e58a30e4`
+
+## Supported pairing
+
+The plugin built from this kit pairs with Cua Driver `0.33.2`, input
+protocol 3. The evidence uses the published `0.33.2` Driver binary for
+application input and the tag source for the native runner. It covers no other
+Driver version. The kit plugin reports `keyboard_layout_independent: true` and
+`foreground_numlock_compatible: true` from `hyprctl -j cua:status` (#3970).
+
+## Limits
+
+- Native Wayland Inkscape `1.4.4-6` is the only background-input application.
+ Background input keeps the canonical US-keymap requirement. Foreground typing
+ keeps the upstream equivalence check, so remapped keymaps such as
+ `ctrl:swapcaps` remain refused.
+- Not covered: Calc 26.8, Chromium/Electron raw background input, XWayland,
+ Unicode/IME, non-US layouts and multi-output capture (#4161, #4305).
+- Faults beyond the 2026-10-01 bounded cases are not repeated here: active-drag
+ primary conflict, active-lease contention, other-lane recovery, same-process
+ siblings, PID/address reuse and reacquisition remain unproven.
+- The evidence comes from one disposable guest, not arbitrary applications or
+ physical hardware.
+
+## Rebuild and requalification
+
+Run `profile_measure.py measure` on the target, then
+`profile_measure.py reuse --reviewed` with this profile:
+
+- `profile-unchanged`: only the profile data matches. Tooling, kit, module and
+ package bytes, and their evidence, still need a separate check.
+- `relabel-rebuild`: build with a higher package release and repeat the package
+ lifecycle evidence.
+- `rebuild`: a changed compositor build, headers, compiler or runtime package
+ needs a new reviewed profile and the affected native qualification.
+
+Every runtime ABI package is pinned exactly, so an Edge update to any of them
+makes this package uninstallable until a new profile lands. A new Driver release
+needs a new kit bound to its own tag, even when the plugin source is unchanged.
+A matching Hyprland version string alone is not sufficient after an Arch rebuild.
+
+## Downstream patches
+
+Omarchy's `downstream.patch` (omacom/omarchy-pkgs#772) adds two behaviours that
+this kit's upstream source does not have:
+
+- a typing check that admits keymap remaps by testing only the chords Driver types;
+- an IME popup guard that hooks Hyprland's input-method popup functions.
+
+Keep the patch until both land upstream and ship in a qualified kit. Applying
+it to this kit changes the source, module and package bytes, so the patched
+package needs its own evidence, as the downstream recipe already requires.
+
+## Ownership
+
+Cua owns the source, tooling and the native evidence above. Omarchy owns Omabot
+replay, package maintenance and signing, and edge-to-RC-to-stable promotion.
+This record does not publish a package into Omarchy's repositories.
diff --git a/libs/cua-driver/hyprland-plugin/docs/production-inkscape-profile.md b/libs/cua-driver/hyprland-plugin/docs/production-inkscape-profile.md
index 201b921dc0..10b633842f 100644
--- a/libs/cua-driver/hyprland-plugin/docs/production-inkscape-profile.md
+++ b/libs/cua-driver/hyprland-plugin/docs/production-inkscape-profile.md
@@ -18,7 +18,12 @@ Prepare two distinct native Inkscape processes for `purpose: "apps"`, and
three for `purpose: "capacity"`. This runner consumes prelaunched targets.
Verify the installed application's supported independent-process launch method;
do not assume a `--new-instance` flag. The single-app smoke already launches a
-positional document and independently verifies that its process is new.
+positional document with a run-specific `--app-id-tag` and independently
+verifies that its process is new and carries the exact tag and document.
+The smoke requests a complete accessibility walk with a 15-second budget.
+When the Objects panel is closed, it opens the uniquely identified button
+through a background accessibility action, then verifies the fixture row
+before sending raw keys. This preparation is not plugin input evidence.
The profile checks the canonical executable `/usr/bin/inkscape`, exact ALPM
ownership/version, GTK3 mappings, one native Wayland client per PID, exact
Hyprland address, document title, and the absolute document argument in the
@@ -153,7 +158,11 @@ record, changed file, stalled event loop, unknown wire record, or exceeded limit
fails qualification. Raw begin/end journal and wire evidence are retained.
Baseline wire evidence must show one primary pointer and keyboard on the same
-surface, a held left button, and no held keyboard keys/modifiers. During the
+surface, a held left button, no held keyboard keys, no depressed or latched
+modifiers, and layout group zero. A stable locked-modifier mask is retained as
+baseline state without interpreting its keymap-defined bits. Every keyboard
+event during the interval, including a lock change and return, still fails.
+During the
parked interval, an exact same-position `wl_pointer.motion` notification on the
same primary object may pass only with unchanged focus, held input, and
foreground interaction. Retain and count these duplicates; they do not certify
diff --git a/libs/cua-driver/hyprland-plugin/packaging/release/profiles/omarchy-edge-20261004.json b/libs/cua-driver/hyprland-plugin/packaging/release/profiles/omarchy-edge-20261004.json
new file mode 100644
index 0000000000..324a1cf6ef
--- /dev/null
+++ b/libs/cua-driver/hyprland-plugin/packaging/release/profiles/omarchy-edge-20261004.json
@@ -0,0 +1,107 @@
+{
+ "architecture": "x86_64",
+ "compiler": {
+ "comment": "GCC: (GNU) 16.2.1 20260810",
+ "sha256": "f04191f6a7b2cd7d9a62e1745872b8a6088791e5af6955488c69c9b2c4668bc9",
+ "version": "16.2.1 20260810"
+ },
+ "hyprland": {
+ "header_version": "0.56.2",
+ "headers_sha256": "1fdefe6ac027a159d04a5dfee4928ec7ebd15544a9a25b5f66d2f5a46fcf364a",
+ "package_version": "0.56.2-4",
+ "sha256": "55da553be71222566ee73b973d2f56dbb9939044d8f22f81aa54b66c83f2f6d1"
+ },
+ "kit_version": "1.3.0",
+ "package_release": 2,
+ "profile_id": "omarchy-edge-20261004",
+ "runtime": {
+ "basename": "libstdc++.so.6.0.36",
+ "packages": {
+ "abseil-cpp": "20260817.0-2",
+ "aquamarine": "0.15.1-1",
+ "brotli": "1.2.0-1",
+ "bzip2": "1.0.8-6",
+ "cairo": "1.18.6-1",
+ "dav1d": "1.5.4-1",
+ "expat": "2.8.5-1",
+ "file": "5.48-1",
+ "fontconfig": "2:2.18.3-2",
+ "freetype2": "2.14.3-1",
+ "fribidi": "1.0.17-1",
+ "gdk-pixbuf2": "2.44.7-1",
+ "glib2": "2.88.3-1",
+ "glibc": "2.44+r50+g1848099f063e-1",
+ "glslang": "1:1.4.363.0-1",
+ "glycin": "2.1.5-2",
+ "graphite": "1:1.3.15-1",
+ "harfbuzz": "14.5.1-1",
+ "highway": "1.4.0-1",
+ "hyprcursor": "0.1.13-7",
+ "hyprgraphics": "0.5.1-4",
+ "hyprlang": "0.6.8-5",
+ "hyprutils": "0.14.2-1",
+ "icu": "78.3-1",
+ "lcms2": "2.19.1-1",
+ "libdatrie": "0.2.14-1",
+ "libdisplay-info": "0.3.0-1",
+ "libdrm": "2.4.134-1",
+ "libei": "1.6.0-1",
+ "libevdev": "1.13.7-1",
+ "libffi": "3.8.0-1",
+ "libgcc": "16.2.1+r23+gd564253eb6c8-1",
+ "libglvnd": "1.7.0-3",
+ "libgomp": "16.2.1+r23+gd564253eb6c8-1",
+ "libgudev": "238-3",
+ "libinput": "1.32.0-1",
+ "libjpeg-turbo": "3.2.0-2",
+ "libjxl": "0.12.0-1",
+ "libpng": "1.6.59-1",
+ "librsvg": "2:2.62.4-1",
+ "libseccomp": "2.6.0-1",
+ "libstdc++": "16.2.1+r23+gd564253eb6c8-1",
+ "libthai": "0.1.30-1",
+ "libwacom": "2.20.0-1",
+ "libwebp": "1.6.0-2",
+ "libx11": "1.8.13-1",
+ "libxau": "1.0.12-1",
+ "libxcb": "1.17.0-1",
+ "libxcursor": "1.2.3-1",
+ "libxdmcp": "1.1.5-2",
+ "libxext": "1.3.7-1",
+ "libxfixes": "6.0.2-1",
+ "libxkbcommon": "1.13.2-1",
+ "libxml2": "2.15.4-1",
+ "libxrender": "0.9.12-1",
+ "libzip": "1.11.4-1",
+ "lua": "5.5.1-1",
+ "lua54": "5.4.9-1",
+ "mesa": "1:26.2.4-1",
+ "mtdev": "1.1.7-1",
+ "muparser": "2.3.5-2",
+ "openssl": "3.6.5-1",
+ "pango": "1:1.58.2-1",
+ "pcre2": "10.49-1",
+ "pixman": "0.46.4-1",
+ "re2": "2:2025.11.05-6",
+ "seatd": "0.9.3-1",
+ "spirv-tools": "1:1.4.363.0-1",
+ "systemd-libs": "262-1",
+ "tomlplusplus": "3.4.0-2",
+ "util-linux-libs": "2.42.4-1",
+ "wayland": "1.26.0-1",
+ "xcb-util-errors": "1.0.1-2",
+ "xcb-util-wm": "0.4.2-2",
+ "xz": "5.8.4-1",
+ "zlib": "1:1.3.2-3",
+ "zstd": "1.5.7-3"
+ },
+ "sha256": "f5fc7380f2ae46fa4053a64be04e7b98109f1066a4bbfff3c37042488aa0be0e"
+ },
+ "schema": 2,
+ "source": {
+ "archive_sha256": "79596a9a5e7300546ad4ab0646430edcf3e0c0456a541458de2a52b8c44a8fad",
+ "driver_version": "0.33.2",
+ "manifest_sha256": "9a7eba9fd642d6697213729311da620fad292cd3f693c8b49b8a07163ef22a1d",
+ "revision": "c82d32e3e1adbc6578a148962002ddf6e3e8a15a"
+ }
+}
diff --git a/libs/cua-driver/hyprland-plugin/proofs/primary_observer.py b/libs/cua-driver/hyprland-plugin/proofs/primary_observer.py
index acb0dfd1f1..b97bd13685 100644
--- a/libs/cua-driver/hyprland-plugin/proofs/primary_observer.py
+++ b/libs/cua-driver/hyprland-plugin/proofs/primary_observer.py
@@ -152,10 +152,16 @@ def primary_wire_state(data):
keyboard_id, keyboard = keyboards[0]
assert pointer['surface'] == keyboard['surface'], 'primary pointer/keyboard focus differs'
assert pointer['buttons'] == {272}, 'primary wire does not prove the held left-button grab'
- assert not keyboard['keys'] and keyboard['modifiers'] == [0, 0, 0, 0], 'primary keyboard is not idle'
+ # A stable locked mask (e.g. a user's Num Lock) is baseline state, not
+ # activity. Held keys and depressed, latched or group state stay refused.
+ # The bit meaning is keymap-defined, so it is retained verbatim, never named.
+ modifiers = keyboard['modifiers']
+ assert not keyboard['keys'] and modifiers is not None and modifiers[0] == modifiers[1] == modifiers[3] == 0 \
+ and 0 <= modifiers[2] < 2**32, 'primary keyboard is not idle'
assert pointer['position'] is not None and all(math.isfinite(value) for value in pointer['position'])
return {'pointer': pointer_id, 'keyboard': keyboard_id, 'surface': pointer['surface'],
- 'position': pointer['position'], 'held_button': 272}
+ 'position': pointer['position'], 'held_button': 272,
+ 'baseline_locked_modifiers': modifiers[2]}
def analyze(before, after, journal, wire, intervals, primary_before, primary_after):
diff --git a/libs/cua-driver/hyprland-plugin/proofs/production_app_smoke.py b/libs/cua-driver/hyprland-plugin/proofs/production_app_smoke.py
index af103fe1c8..632f43376b 100644
--- a/libs/cua-driver/hyprland-plugin/proofs/production_app_smoke.py
+++ b/libs/cua-driver/hyprland-plugin/proofs/production_app_smoke.py
@@ -33,6 +33,12 @@ NS = {'office': 'urn:oasis:names:tc:opendocument:xmlns:office:1.0',
'table': 'urn:oasis:names:tc:opendocument:xmlns:table:1.0',
'text': 'urn:oasis:names:tc:opendocument:xmlns:text:1.0',
'svg': 'http://www.w3.org/2000/svg'}
+# Default 1000 ms truncates the pinned Inkscape tree; a full walk took ~2.2 s.
+OBSERVATION_TIMEOUT_MS = 15000
+OBSERVATION_ATTEMPTS = 2
+IDENTITY_UNPROVEN_REASON = 'accessibility_window_identity_unproven'
+OPEN_OBJECTS_DESCRIPTION = 'Open Objects\nView Objects (Shift+Ctrl+L)'
+OFFSCREEN_DESCRIPTION = '; off-screen: scroll it into view before a pixel action; element actions still reach it'
LIMITS = {'cursor_isolation': False, 'concurrency': False,
'plugin_transport_attribution': False,
'all_operations': False, 'full_desktop_matrix': False}
@@ -383,23 +389,39 @@ def calc_formula_selection(snapshot, elements, expected='A1'):
for row in elements))
+def inkscape_objects(elements):
+ return [row for row in elements if row.get('role') == 'table cell'
+ and row.get('label') == 'smoke-rectangle' and row.get('enabled') is True]
+
+
+def open_objects_button(elements, lines):
+ """The one exactly described, enabled, actionable Open Objects button."""
+ descriptions = (OPEN_OBJECTS_DESCRIPTION, OPEN_OBJECTS_DESCRIPTION + OFFSCREEN_DESCRIPTION)
+ described = [row for row in elements if row.get('description') in descriptions]
+ if len(described) != 1:
+ return None
+ button = described[0]
+ index = button.get('element_index')
+ if (button.get('role') != 'button' or button.get('enabled') is not True
+ or type(index) is not int or not button.get('actions')
+ or not isinstance(button.get('element_token'), str) or not button['element_token']
+ or sum(line.startswith(f'- [{index}] button ') for line in lines) != 1):
+ return None
+ return button
+
+
def inkscape_selection_command(snapshot, elements):
"""Ground Ctrl+A in the pinned document UI, which exposes no canvas role.
The non-actionable selection status exists only in tree_markdown. Require
- the exact initial status, the fixture's object row, and Edit > Select All
- in both projections; a label mentioning a canvas is not evidence.
+ the exact initial status, the fixture's unselected object row, and the Edit
+ menu in both projections. A closed menu omits its children, so Select All
+ is not required; a label mentioning a canvas is not evidence.
"""
menus = [row for row in elements if row.get('role') == 'menu'
and row.get('label') == 'Edit' and row.get('enabled') is True]
- objects = [row for row in elements if row.get('role') == 'table cell'
- and row.get('label') == 'smoke-rectangle' and row.get('enabled') is True]
- if len(menus) != 1 or len(objects) != 1:
- return False
- commands = [row for row in elements if row.get('role') == 'menu item'
- and row.get('label') == 'Select All' and row.get('enabled') is True
- and row.get('parent_index') == menus[0].get('element_index')]
- if len(commands) != 1:
+ objects = inkscape_objects(elements)
+ if len(menus) != 1 or len(objects) != 1 or objects[0].get('selected') is True:
return False
lines = [line.strip() for line in snapshot.get('tree_markdown', '').splitlines()]
status = ('- label = "No objects selected. Click, Shift+click, Alt+scroll mouse '
@@ -407,7 +429,7 @@ def inkscape_selection_command(snapshot, elements):
return lines.count(status) == 1 and all(
sum(line.startswith(f'- [{row.get("element_index")}] {row["role"]} "{row["label"]}" ')
for line in lines) == 1
- for row in (menus[0], objects[0], commands[0]))
+ for row in (menus[0], objects[0]))
def inkscape_selected_rectangle(snapshot, elements):
@@ -423,23 +445,28 @@ def inkscape_selected_rectangle(snapshot, elements):
'scale/rotation handles."')
if lines.count(status) != 1 or any('No objects selected.' in line for line in lines):
return False
- objects = [row for row in elements if row.get('role') == 'table cell'
- and row.get('label') == 'smoke-rectangle' and row.get('enabled') is True]
+ objects = inkscape_objects(elements)
if len(objects) != 1 or sum(line.startswith(
f'- [{objects[0].get("element_index")}] table cell "smoke-rectangle" ')
for line in lines) != 1:
return False
- for axis, value in (('X', 40), ('Y', 60), ('W', 80), ('H', 50)):
+ for axis, value, semantic in (('X', 40, 'Horizontal coordinate of selection'),
+ ('Y', 60, 'Vertical coordinate of selection'),
+ ('W', 80, 'Width of selection'),
+ ('H', 50, 'Height of selection')):
+ # The pinned Inkscape labels each control with its value or, in newer
+ # builds, with its axis meaning; accept only this axis's own label.
+ labels = (f'{value:.3f}', semantic)
controls = [row for row in elements if row.get('role') == 'spin button'
- and row.get('label') == f'{value:.3f}'
+ and row.get('label') in labels
and row.get('value') == f'{value:.1f}' and row.get('enabled') is True]
matches = []
for row in controls:
- prefix = (f'- [{row.get("element_index")}] spin button "{value:.3f}" '
+ prefix = (f'- [{row.get("element_index")}] spin button "{row["label"]}" '
f'value="{value:.1f}" ')
matches.extend(index for index, line in enumerate(lines) if index > 0
and line.startswith(prefix) and lines[index - 1] == f'- label = "{axis}:"')
- if len(matches) != 1:
+ if len(controls) != 1 or len(matches) != 1:
return False
return True
@@ -461,29 +488,125 @@ def ground(snapshot, app, stage):
raise GroundingUnavailable('cannot prove the single rectangle is selected before Right')
-def snapshot(mcp, target, filename, app, stage):
- result = content(mcp.tool('get_window_state', target))
- assert filename in result.get('window_title', ''), 'snapshot is not the synthetic document'
+def require_complete(state):
+ """A truncated, degraded or unreported tree cannot authorize any action."""
+ if (state.get('truncated') is not False or state.get('elements_complete') is not True
+ or state.get('degraded') or state.get('timeout_ms') != OBSERVATION_TIMEOUT_MS):
+ raise GroundingUnavailable('accessibility tree is truncated, degraded or not proven complete')
+
+
+def identity_unproven_only(state):
+ """A complete walk whose only defect is a not-yet-reconciled window identity."""
+ reason = state.get('degraded_reason')
+ return (state.get('truncated') is False and state.get('elements_complete') is True
+ and state.get('timeout_ms') == OBSERVATION_TIMEOUT_MS and state.get('degraded') is True
+ and isinstance(reason, str) and reason.startswith(IDENTITY_UNPROVEN_REASON))
+
+
+def require_single_window(mcp, target):
windows = content(mcp.tool('list_windows', {}))['windows']
owned = [row for row in windows if row.get('pid') == target['pid']]
if len(owned) != 1 or owned[0]['window_id'] != target['window_id']:
raise GroundingUnavailable('extra app window or dialog; inspect fresh snapshot and window list')
+
+
+def observe(mcp, target, filename):
+ """One complete, single-window observation with no stage grounding.
+
+ Right after a save the compositor title and the AT-SPI title can briefly
+ disagree, so the Driver refuses exact identity. That refusal alone is
+ re-observed (read-only, fixed bound); the Driver stays strict and no
+ input is ever replayed.
+ """
+ attempts = []
+ for attempt in range(1, OBSERVATION_ATTEMPTS + 1):
+ result = content(mcp.tool('get_window_state', {**target, 'timeout_ms': OBSERVATION_TIMEOUT_MS}))
+ assert filename in result.get('window_title', ''), 'snapshot is not the synthetic document'
+ retryable = identity_unproven_only(result)
+ if not retryable:
+ require_complete(result)
+ require_single_window(mcp, target)
+ if not retryable:
+ rows(result)
+ return result
+ attempts.append({'attempt': attempt, 'degraded_reason': result['degraded_reason']})
+ directory = getattr(mcp, 'directory', None)
+ if isinstance(directory, Path):
+ save_json(directory, 'observation-retries.json',
+ {'max_attempts': OBSERVATION_ATTEMPTS, 'attempts': attempts})
+ raise GroundingUnavailable(
+ f'window identity still unproven after {OBSERVATION_ATTEMPTS} observations')
+
+
+def snapshot(mcp, target, filename, app, stage):
+ result = observe(mcp, target, filename)
ground(result, app, stage)
return result
+def retain_unknown_outcome(mcp):
+ # DirectMCP forbids all RPC after an unknown result, including a snapshot.
+ # A best-effort compositor observation retains state without replaying input.
+ try:
+ save_json(mcp.directory, 'unknown-outcome-windows.json',
+ json.loads(read(['hyprctl', '-j', 'clients'])))
+ except Exception:
+ pass
+
+
+def prepare_inkscape_objects(mcp, target, filename, directory):
+ """Open the Objects panel semantically only when the fixture row is absent.
+
+ This is accessibility setup, not plugin input proof: it never counts toward
+ the raw-input stages. The result is checked from a fresh observation before
+ any raw key is sent, and the click is never retried.
+ """
+ before = observe(mcp, target, filename)
+ elements = before['elements']
+ named = [row for row in elements if row.get('label') == 'smoke-rectangle']
+ if named:
+ if len(named) != 1 or len(inkscape_objects(elements)) != 1:
+ raise GroundingUnavailable('ambiguous or disabled smoke-rectangle object row')
+ return {'performed': False}
+ lines = [line.strip() for line in before.get('tree_markdown', '').splitlines()]
+ button = open_objects_button(elements, lines)
+ if button is None:
+ raise GroundingUnavailable('Objects panel absent and Open Objects button is not exactly grounded')
+ setup = {'performed': False, 'attempted': True, 'kind': 'accessibility_setup',
+ 'plugin_input_proof': False, 'action': 'click Open Objects button',
+ 'element_index': button['element_index'], 'outcome': 'unknown'}
+ save_json(directory, 'objects-panel-setup.json', setup)
+ try:
+ response = mcp.tool('click', {**target, 'element_token': button['element_token'],
+ 'delivery_mode': 'background'})
+ except BaseException:
+ retain_unknown_outcome(mcp)
+ raise
+ value = content(response)
+ delivery = value.get('delivery')
+ setup.update(performed=True, outcome='returned', delivery=delivery,
+ effect=value.get('effect'), route=value.get('route'))
+ save_json(directory, 'objects-panel-setup.json', setup)
+ assert value.get('route') == 'accessibility', 'setup click did not use accessibility'
+ assert value.get('effect') in ('confirmed', 'unverifiable'), 'partial/refused setup cannot pass'
+ assert isinstance(delivery, dict) and delivery.get('mode') == 'background', 'setup click was not background'
+ after = observe(mcp, target, filename)
+ named = [row for row in after['elements'] if row.get('label') == 'smoke-rectangle']
+ if len(named) != 1 or len(inkscape_objects(after['elements'])) != 1:
+ raise GroundingUnavailable('Objects panel did not expose exactly one smoke-rectangle row')
+ setup['outcome'] = 'verified'
+ save_json(directory, 'objects-panel-setup.json', setup)
+ return setup
+
+
def input_step(mcp, target, filename, app, stage, tool, arguments):
snapshot(mcp, target, filename, app, stage)
+ # Attempted is not delivered: mark before dispatch so any later failure reports it.
+ setattr(mcp, 'raw_input_attempted', True)
try:
response = mcp.tool(tool, {**target, **arguments, 'delivery_mode': 'background'})
except BaseException:
- # DirectMCP forbids all RPC after an unknown result, including a snapshot.
- # A best-effort compositor observation retains state without replaying input.
- try:
- save_json(mcp.directory, 'unknown-outcome-windows.json',
- json.loads(read(['hyprctl', '-j', 'clients'])))
- except Exception:
- pass
+ retain_unknown_outcome(mcp)
raise
try:
snapshot(mcp, target, filename, app, 'after')
@@ -492,7 +615,7 @@ def input_step(mcp, target, filename, app, stage, tool, arguments):
check_delivery(response)
-def discover(mcp, app, document, old_pids):
+def discover(mcp, app, document, old_pids, arguments=()):
deadline = time.monotonic() + 20
while time.monotonic() < deadline:
windows = content(mcp.tool('list_windows', {}))['windows']
@@ -504,8 +627,10 @@ def discover(mcp, app, document, old_pids):
pid = window['pid']
executable = Path(f'/proc/{pid}/exe').resolve(strict=True)
assert executable == EXECUTABLES[app].resolve(strict=True), 'noncanonical running executable'
- assert str(document).encode() in Path(f'/proc/{pid}/cmdline').read_bytes().split(b'\0'), \
- 'running process is not bound to the new document'
+ argv = Path(f'/proc/{pid}/cmdline').read_bytes().split(b'\0')
+ assert str(document).encode() in argv, 'running process is not bound to the new document'
+ assert all(argument.encode() in argv for argument in arguments), \
+ 'running process is not bound to its independent launch arguments'
clients = json.loads(read(['hyprctl', '-j', 'clients']))
native = [row for row in clients if row.get('pid') == pid]
assert len(native) == 1 and native[0].get('xwayland') is False, 'not one native Wayland app window'
@@ -527,12 +652,17 @@ def discover(mcp, app, document, old_pids):
try:
executable = Path(f'/proc/{pid}/exe').resolve(strict=True)
if executable == EXECUTABLES[app].resolve(strict=True):
- mcp.tool('get_window_state', {key: window[key] for key in ('pid', 'window_id')})
+ mcp.tool('get_window_state', {**{key: window[key] for key in ('pid', 'window_id')},
+ 'timeout_ms': OBSERVATION_TIMEOUT_MS})
except OSError:
continue
raise GroundingUnavailable('launched document window not discovered; inspect launch and window evidence')
+def inkscape_app_id_tag(directory):
+ return 'cua-smoke-' + hashlib.sha256(str(Path(directory).resolve()).encode()).hexdigest()[:16]
+
+
def launch_arguments(app, document, directory):
launch = {'launch_path': str(EXECUTABLES[app])}
if app == 'calc':
@@ -540,9 +670,11 @@ def launch_arguments(app, document, directory):
launch['additional_arguments'] = [f'-env:UserInstallation={(directory / "calc-profile").as_uri()}',
'--norestore', '--nologo', '--calc', str(document)]
else:
- # Inkscape 1.4.4 accepts positional documents, not --new-instance.
- # discover() still requires a new PID, exact executable and document.
- launch['additional_arguments'] = [str(document)]
+ # Inkscape 1.4.4 accepts positional documents, not --new-instance. A
+ # run-unique --app-id-tag keeps D-Bus from handing the document to a
+ # retained earlier process; discover() still requires a new PID,
+ # exact executable, tag and document.
+ launch['additional_arguments'] = [f'--app-id-tag={inkscape_app_id_tag(directory)}', str(document)]
return launch
@@ -564,11 +696,14 @@ def run_app(mcp, app, document, directory):
old_pids = {int(path.name) for path in Path('/proc').iterdir() if path.name.isdigit()}
before = document.read_bytes()
(directory / ('before' + document.suffix)).write_bytes(before)
- content(mcp.tool('launch_app', launch_arguments(app, document, directory)))
- target, identity = discover(mcp, app, document, old_pids)
+ launch = launch_arguments(app, document, directory)
+ content(mcp.tool('launch_app', launch))
+ target, identity = discover(mcp, app, document, old_pids,
+ launch['additional_arguments'][:-1] if app == 'inkscape' else ())
save_json(directory, 'target.json', identity)
try:
require_background_target(target, directory)
+ setup = prepare_inkscape_objects(mcp, target, document.name, directory) if app == 'inkscape' else None
if app == 'calc':
steps = [('insert', 'type_text', {'text': 'abc'}),
('commit', 'press_key', {'key': 'Return'}),
@@ -591,12 +726,20 @@ def run_app(mcp, app, document, directory):
raise
time.sleep(0.1)
snapshot(mcp, target, document.name, app, 'saved')
- return {'result': 'passed', 'target': target, 'saved_document': result}
+ return {'result': 'passed', 'target': target, 'saved_document': result,
+ **({'setup': setup} if setup is not None else {})}
finally:
if document.exists():
(directory / ('after' + document.suffix)).write_bytes(document.read_bytes())
+def grounding_outcome(mcp, error):
+ """Grounding loss after a raw input attempt is a failure, not inspection."""
+ if getattr(mcp, 'raw_input_attempted', False) is True:
+ return {'result': 'failed', 'blocker': str(error), 'raw_input_attempted': True}
+ return {'result': 'inspection_only', 'blocker': str(error)}
+
+
def run(args):
if not __debug__:
raise RuntimeError('assertions must be enabled')
@@ -626,7 +769,7 @@ def run(args):
{'mode': 'unrestricted', 'acknowledge_unrestricted': True})
result['apps'][app] = run_app(mcp, app, document, directory)
except GroundingUnavailable as error:
- result['apps'][app] = {'result': 'inspection_only', 'blocker': str(error)}
+ result['apps'][app] = grounding_outcome(mcp, error)
except Exception as error:
result['apps'][app] = {'result': 'failed', 'error': str(error)}
finally:
diff --git a/libs/cua-driver/hyprland-plugin/proofs/production_cancel_proof.py b/libs/cua-driver/hyprland-plugin/proofs/production_cancel_proof.py
index 729b8f25b0..b31337afcb 100644
--- a/libs/cua-driver/hyprland-plugin/proofs/production_cancel_proof.py
+++ b/libs/cua-driver/hyprland-plugin/proofs/production_cancel_proof.py
@@ -6,7 +6,7 @@ telemetry proves two active drags; preserve unknown delivery without replay.
"""
import argparse
from harness_paths import harness_file # Also puts tests/ helpers on sys.path.
-from production_app_smoke import add_provenance_arguments
+from production_app_smoke import OBSERVATION_TIMEOUT_MS, add_provenance_arguments
from concurrent.futures import ThreadPoolExecutor
import hashlib
import json
@@ -40,7 +40,7 @@ MAX_GROUNDING_ATTEMPTS = 2
# recovered the status and geometry in about 1.2 seconds per independent app.
# Keep the walk bounded and depth uncapped (the object row is deeply nested).
# Missing oracle evidence and the unchanged five-second age limit fail closed.
-POINTER_SNAPSHOT_LIMITS = {'inkscape': {'max_elements': 3000}}
+POINTER_SNAPSHOT_LIMITS = {'inkscape': {'max_elements': 3000, 'timeout_ms': OBSERVATION_TIMEOUT_MS}}
def validate_app_profile(plan, *, require_drag=True):
diff --git a/libs/cua-driver/hyprland-plugin/proofs/production_pointer_grounding.py b/libs/cua-driver/hyprland-plugin/proofs/production_pointer_grounding.py
index 207a693bd1..667c17a3d2 100644
--- a/libs/cua-driver/hyprland-plugin/proofs/production_pointer_grounding.py
+++ b/libs/cua-driver/hyprland-plugin/proofs/production_pointer_grounding.py
@@ -19,6 +19,11 @@ STAGES = {
}
+INKSCAPE_AXIS_LABELS = {'X': 'Horizontal coordinate of selection',
+ 'Y': 'Vertical coordinate of selection',
+ 'W': 'Width of selection', 'H': 'Height of selection'}
+
+
class Pixels:
def __init__(self, width, height, data, stride, channels=3):
self.width, self.height = width, height
@@ -173,7 +178,7 @@ def inkscape_geometry(snapshot, *, allow_transform_center=False):
for index, line in enumerate(lines):
if index == 0 or lines[index - 1] != f'- label = "{axis}:"':
continue
- match = re.match(r'^- \[(\d+)\] spin button "([-\d.]+)" value="([-\d.]+)" ', line)
+ match = re.match(r'^- \[(\d+)\] spin button "([^"]+)" value="([-\d.]+)" ', line)
if not match:
continue
controls = [row for row in elements if row.get('element_index') == int(match[1])
@@ -181,7 +186,16 @@ def inkscape_geometry(snapshot, *, allow_transform_center=False):
and row.get('value') == match[3] and row.get('enabled') is True
and row.get('frame', {}).get('h', 0) > 0
and 0 <= row.get('frame', {}).get('y', -1) - snapshot['window_bounds']['y'] < 90]
- if len(controls) == 1 and math.isclose(float(match[2]), float(match[3]), abs_tol=.001):
+ # A legacy numeric label must equal the value; otherwise the label
+ # must be this axis's own semantic name, never another axis's.
+ if match[2] == INKSCAPE_AXIS_LABELS[axis]:
+ label_ok = True
+ else:
+ try:
+ label_ok = math.isclose(float(match[2]), float(match[3]), abs_tol=.001)
+ except ValueError:
+ label_ok = False
+ if len(controls) == 1 and label_ok:
matches.append(float(match[3]))
if len(matches) != 1 or not math.isfinite(matches[0]):
raise GroundingUnavailable('selection geometry differs between semantic projections')
diff --git a/libs/cua-driver/hyprland-plugin/proofs/production_realapp_proof.py b/libs/cua-driver/hyprland-plugin/proofs/production_realapp_proof.py
index 546c419676..50180ccaca 100644
--- a/libs/cua-driver/hyprland-plugin/proofs/production_realapp_proof.py
+++ b/libs/cua-driver/hyprland-plugin/proofs/production_realapp_proof.py
@@ -24,7 +24,8 @@ from driver_input_live import state, wait_for, wm
from primary_trace import Trace, analyze
from primary_observer import PrimaryObserver, verify_negative_control as verify_primary_control
from production_app_smoke import (EXECUTABLES, NS, add_provenance_arguments, digest, ground, package_owner, profile_packages,
- provenance as runtime_provenance)
+ provenance as runtime_provenance, OBSERVATION_ATTEMPTS, OBSERVATION_TIMEOUT_MS, identity_unproven_only,
+ require_complete)
from production_mcp import DirectMCP, assert_distinct_runtimes, stop_process
import production_pointer_grounding as pointer_grounding
from realapp_proof import cleanup_all, rect_position, released_synthetic_input
@@ -147,6 +148,17 @@ def validate_plan(plan):
assert Path(oracle['path']).resolve() == documents[oracle['agent']].resolve(), \
'saved SVG oracle does not belong to its target lane'
assert plan['phases'], 'empty plan cannot pass'
+ if inkscape_only and plan['purpose'] == 'apps' and plan.get('require_overlap'):
+ selected = set()
+ for phase in plan['phases']:
+ for step in phase.get('parallel', [phase]):
+ if step.get('smoke_stage') == 'select':
+ assert 'parallel' not in phase, 'lane binding requires serial selection'
+ assert step['agent'] not in selected, 'lane binding requires one selection per agent'
+ selected.add(step['agent'])
+ if step.get('pointer_stage') == 'move_rectangle':
+ assert selected == {0, 1}, 'both lane bindings must precede overlapping drags'
+ assert selected == {0, 1}, 'overlap requires both serial lane selections'
for phase in plan['phases']:
if phase.get('negative_control'):
assert plan['purpose'] == 'negative_control'
@@ -309,6 +321,21 @@ def capacity_lane(before, after, tool):
return lanes.pop()
+def bound_drag_lanes(actions):
+ """Map agents to lanes proven by their own serial select hotkey, never by order."""
+ bound = {}
+ for row in actions:
+ # Only bound plans record lane evidence; its absence is rejected where required.
+ if row.get('tool') != 'hotkey' or row.get('smoke_stage') != 'select' or 'compositor_lane' not in row:
+ continue
+ lane = row['compositor_lane']
+ assert type(lane) is int and lane in (1, 2), 'invalid select compositor lane evidence'
+ assert row['agent'] not in bound, 'agent bound to a lane twice'
+ assert lane not in bound.values(), 'agents share a compositor lane'
+ bound[row['agent']] = lane
+ return bound
+
+
def verify_capacity(actions):
assert len(actions) == 3 and [row['agent'] for row in actions] == [0, 1, 2], 'incomplete capacity actions'
assert all(row['expected'] == 'dispatched' for row in actions[:2])
@@ -604,6 +631,7 @@ def run(args):
commands, motion_errors, action_intervals = [], [], []
capacity_traces = []
capacity_owners = {}
+ bound_owners = {}
policy_cache_traces = []
trajectory = None
recording = False
@@ -629,18 +657,28 @@ def run(args):
directory.mkdir()
return DirectMCP(args.driver, directory, profile)
def snapshot(mcp, target, session=None, full=False, pixels=False):
- if capacity or policy_cache or full:
- windows = mcp.tool('list_windows', {})
- assert not windows.get('isError'), windows
- matches = [window for window in windows['structuredContent']['windows']
- if window.get('pid') == target['pid']]
- assert len(matches) == 1 and matches[0].get('window_id') == target['window_id'], \
- 'reviewed PID/window identity is stale or ambiguous'
- result = mcp.tool('get_window_state', {**target,
- **({} if full else {'max_elements': 100, 'max_depth': 6}),
- **({'session': session} if session else {})})
- assert not result.get('isError'), result
- content = result['structuredContent']
+ # Only a complete walk whose sole defect is a not-yet-reconciled window
+ # identity is re-observed (read-only, fixed bound, exact PID/window on
+ # every attempt). Input is never replayed; every strict gate still applies.
+ for attempt in range(1, (OBSERVATION_ATTEMPTS if full else 1) + 1):
+ if capacity or policy_cache or full:
+ windows = mcp.tool('list_windows', {})
+ assert not windows.get('isError'), windows
+ matches = [window for window in windows['structuredContent']['windows']
+ if window.get('pid') == target['pid']]
+ assert len(matches) == 1 and matches[0].get('window_id') == target['window_id'], \
+ 'reviewed PID/window identity is stale or ambiguous'
+ result = mcp.tool('get_window_state', {**target,
+ **({'timeout_ms': OBSERVATION_TIMEOUT_MS} if full else {'max_elements': 100, 'max_depth': 6}),
+ **({'session': session} if session else {})})
+ assert not result.get('isError'), result
+ content = result['structuredContent']
+ if not (full and attempt < OBSERVATION_ATTEMPTS and identity_unproven_only(content)):
+ break
+ mark('observation_retry', pid=target['pid'], window_id=target['window_id'], attempt=attempt,
+ max_attempts=OBSERVATION_ATTEMPTS, degraded_reason=content['degraded_reason'])
+ if full:
+ require_complete(content)
assert content.get('screenshot_width', 0) > 0, 'missing grounding image'
if pixels:
images = [row for row in result.get('content', []) if row.get('type') == 'image']
@@ -687,8 +725,17 @@ def run(args):
status = read_input_status()
save(f'capacity-agent-{index}-owners-before.json', status)
capacity_reservations(status, capacity_owners, capacity_owners)
+ bind_select = bool(trace) and plan['purpose'] == 'apps' and plan.get('app_profile') == 'inkscape-only' \
+ and plan.get('require_overlap', False) \
+ and step['tool'] == 'hotkey' and smoke_stage == 'select'
+ if bound_owners:
+ # Quiet pre-dispatch check (parallel siblings dispatch only after the
+ # barrier): a reassigned lane would invalidate the agent->lane binding.
+ status = read_input_status()
+ save(f'bound-agent-{index}-{mcp.counter}-owners-before.json', status)
+ capacity_reservations(status, bound_owners, bound_owners)
if not policy_cache:
- trace_before = trace.collect() if trace and (capacity or expected['kind'] == 'refused') else None
+ trace_before = trace.collect() if trace and (capacity or bind_select or expected['kind'] == 'refused') else None
if capacity:
assert_no_dispatch(capacity_traces[-1] if capacity_traces else trace_before, trace_before)
if barrier:
@@ -766,6 +813,17 @@ def run(args):
result['compositor_lane'] = capacity_lane(trace_before, trace_after, step['tool'])
assert result['compositor_lane'] not in {row.get('compositor_lane') for row in report['actions']}, \
'capacity needs distinct compositor lanes'
+ if bind_select:
+ # Serial select: the only exercised lane is this agent's lane.
+ trace_after = trace.collect()
+ save(f'bound-agent-{index}-trace.json', {'before': trace_before, 'after': trace_after})
+ lane = capacity_lane(trace_before, trace_after, step['tool'])
+ assert lane not in bound_owners, 'agents share a compositor lane'
+ result['compositor_lane'] = lane
+ status = read_input_status()
+ save(f'bound-agent-{index}-owners-after.json', status)
+ bound_owners.update(capacity_reservations(status, {*bound_owners, lane}, bound_owners))
+ result['persistent_owners'] = dict(bound_owners)
if expected['kind'] == 'refused' and not policy_cache:
result['no_dispatch'] = 'unproven'
if trace:
@@ -991,10 +1049,14 @@ def run(args):
report['synthetic_cleanup'] = 'verified'
if focus_before is not None:
report['passive_focus'] = passive_focus_evidence(focus_before, focus_after, data)
+ lanes = bound_drag_lanes(report['actions'])
for action_result in report['actions']:
if action_result.get('pointer_stage') in ('select_range', 'move_rectangle'):
+ if plan.get('require_overlap') and plan.get('app_profile') == 'inkscape-only':
+ assert action_result['agent'] in lanes, 'overlapping drag lacks an independent lane binding'
action_result['pointer_delivery'] = pointer_grounding.verify_drag_trace(
- data, action_result['arguments'], action_result['pointer_effect'])
+ data, action_result['arguments'], action_result['pointer_effect'],
+ expected_lane=lanes.get(action_result['agent']))
operations += [('finish_trace', finish_trace), ('close_trace', trace.close)]
def release_primary():
if grab:
diff --git a/libs/cua-driver/hyprland-plugin/tests/production_app_smoke_test.py b/libs/cua-driver/hyprland-plugin/tests/production_app_smoke_test.py
index 50ee12da2a..49c687d9e8 100644
--- a/libs/cua-driver/hyprland-plugin/tests/production_app_smoke_test.py
+++ b/libs/cua-driver/hyprland-plugin/tests/production_app_smoke_test.py
@@ -1,5 +1,6 @@
"""No native applications or input: fixtures and fail-closed orchestration only."""
import copy
+import json
import os
from pathlib import Path
import tempfile
@@ -9,11 +10,12 @@ import zipfile
import proofs_path # noqa: F401 Puts ../proofs on sys.path.
from production_app_smoke import (
- GroundingUnavailable, check_delivery, create_documents, ground, input_step,
- kernel_file_identity, launch_arguments, mapped_plugin, package_owner,
+ GroundingUnavailable, OBSERVATION_TIMEOUT_MS, OPEN_OBJECTS_DESCRIPTION, OFFSCREEN_DESCRIPTION, check_delivery,
+ OBSERVATION_ATTEMPTS, create_documents, grounding_outcome, ground, inkscape_app_id_tag, input_step, kernel_file_identity,
+ launch_arguments, mapped_plugin, observe, package_owner, prepare_inkscape_objects,
require_background_target, require_enabled_plugin, run_app, verify_calc, verify_inkscape,
)
-from proof_fixtures import CALC, INKSCAPE, INKSCAPE_SELECTED, changed_ods
+from proof_fixtures import CALC as CALC_STATE, INKSCAPE_SELECTED, changed_ods
TARGET = {'pid': 123, 'window_id': 456}
@@ -23,6 +25,44 @@ GOOD_DELIVERY = {'structuredContent': {'route': 'synthetic_events',
WINDOWS = {'structuredContent': {'windows': [TARGET]}}
+def complete(state):
+ """Mark a fixture as a full walk under the explicit observation budget."""
+ return {**state, 'truncated': False, 'elements_complete': True,
+ 'timeout_ms': OBSERVATION_TIMEOUT_MS}
+
+
+def observed(state):
+ return [{'structuredContent': state}, WINDOWS]
+
+
+CALC = complete(CALC_STATE)
+# The pinned closed Edit menu omits its Select All child.
+INKSCAPE = complete({
+ 'window_title': 'cua-smoke-inkscape.svg - Inkscape',
+ 'elements': [
+ {'element_index': 10, 'role': 'menu', 'label': 'Edit', 'enabled': True},
+ {'element_index': 12, 'role': 'table cell', 'label': 'smoke-rectangle', 'enabled': True,
+ 'selected': False}],
+ 'tree_markdown': '\n'.join([
+ ' - [10] menu "Edit" [actions=[click]]',
+ ' - [12] table cell "smoke-rectangle" [actions=[activate]]',
+ ' - label = "No objects selected. Click, Shift+click, Alt+scroll mouse on top of '
+ 'objects, or drag around objects to select."'])})
+# Objects panel not yet open: only the semantic opener exists.
+INKSCAPE_NO_OBJECTS = complete({
+ 'window_title': 'cua-smoke-inkscape.svg - Inkscape',
+ 'elements': [
+ {'element_index': 10, 'role': 'menu', 'label': 'Edit', 'enabled': True},
+ {'element_index': 20, 'element_token': 'token-20', 'role': 'button',
+ 'label': OPEN_OBJECTS_DESCRIPTION, 'description': OPEN_OBJECTS_DESCRIPTION,
+ 'enabled': True, 'actions': ['click']}],
+ 'tree_markdown': '\n'.join([
+ ' - [10] menu "Edit" [actions=[click]]',
+ ' - [20] button "Open Objects" [actions=[click]]'])})
+SETUP_CLICK = {'structuredContent': {'route': 'accessibility', 'effect': 'unverifiable',
+ 'delivery': {'mode': 'background'}}}
+
+
class FixtureTests(unittest.TestCase):
def test_enabled_option_uses_exact_pinned_v2_boolean_contract(self):
enabled = {'option': 'plugin:cua:enabled', 'bool': True, 'set': True}
@@ -174,10 +214,16 @@ class InputTests(unittest.TestCase):
require_background_target(TARGET, Path('/evidence'))
read.assert_called_once_with(['hyprctl', '-j', 'activewindow'])
- def test_inkscape_launch_uses_supported_positional_document(self):
- self.assertEqual(launch_arguments('inkscape', Path('/docs/smoke.svg'), Path('/evidence')),
- {'launch_path': '/usr/bin/inkscape',
- 'additional_arguments': ['/docs/smoke.svg']})
+ def test_inkscape_launch_uses_run_unique_app_id_tag_and_positional_document(self):
+ launch = launch_arguments('inkscape', Path('/docs/smoke.svg'), Path('/evidence/a/inkscape'))
+ tag = inkscape_app_id_tag(Path('/evidence/a/inkscape'))
+ self.assertRegex(tag, r'^cua-smoke-[0-9a-f]{16}$')
+ self.assertEqual(launch, {'launch_path': '/usr/bin/inkscape',
+ 'additional_arguments': [f'--app-id-tag={tag}', '/docs/smoke.svg']})
+ self.assertNotEqual(tag, inkscape_app_id_tag(Path('/evidence/b/inkscape')))
+ self.assertTrue(all('app-id-tag' not in word for word in
+ launch_arguments('calc', Path('/docs/a.ods'), Path('/evidence/calc'))
+ ['additional_arguments']))
def test_calc_formula_name_field_requires_matching_toolbar_and_rows(self):
state = {'elements': [
@@ -212,15 +258,19 @@ class InputTests(unittest.TestCase):
def test_inkscape_requires_document_command_and_exact_initial_status(self):
ground(INKSCAPE, 'inkscape', 'select')
+ # A closed menu has no Select All child; an open one with it still grounds.
+ ground({**INKSCAPE, 'elements': INKSCAPE['elements'] + [
+ {'element_index': 11, 'parent_index': 10, 'role': 'menu item',
+ 'label': 'Select All', 'enabled': True}]}, 'inkscape', 'select')
for index, replacement in [(0, {'role': 'label'}), (0, {'enabled': False}),
- (1, {'parent_index': 99}), (1, {'enabled': False}),
- (1, {'label': 'Select All in All Layers'}),
- (2, {'label': 'other-rectangle'}), (2, {'role': 'label'})]:
+ (0, {'label': 'File'}),
+ (1, {'label': 'other-rectangle'}), (1, {'role': 'label'}),
+ (1, {'enabled': False}), (1, {'selected': True})]:
bad = copy.deepcopy(INKSCAPE)
bad['elements'][index].update(replacement)
with self.subTest(index=index, replacement=replacement), self.assertRaises(GroundingUnavailable):
ground(bad, 'inkscape', 'select')
- for old, new in [('[10]', '[99]'), ('[11]', '[99]'), ('[12]', '[99]'),
+ for old, new in [('[10]', '[99]'), ('[12]', '[99]'),
('No objects selected.', '1 object selected.'), ('- label =', '- button =')]:
with self.subTest(old=old), self.assertRaises(GroundingUnavailable):
ground({**INKSCAPE, 'tree_markdown': INKSCAPE['tree_markdown'].replace(old, new)},
@@ -274,6 +324,36 @@ class InputTests(unittest.TestCase):
with self.assertRaises(GroundingUnavailable):
ground(state, 'inkscape', 'move')
+ def test_selected_rectangle_accepts_only_per_axis_semantic_labels(self):
+ names = ['Horizontal coordinate of selection', 'Vertical coordinate of selection',
+ 'Width of selection', 'Height of selection']
+
+ def relabel(state, labels):
+ state = copy.deepcopy(state)
+ markdown = state['tree_markdown']
+ for row, label in zip(state['elements'][1:], labels):
+ markdown = markdown.replace(f'spin button "{row["label"]}"', f'spin button "{label}"')
+ row['label'] = label
+ return {**state, 'tree_markdown': markdown}
+
+ ground(relabel(INKSCAPE_SELECTED, names), 'inkscape', 'move')
+ # Swapped axes, a duplicate label, or a wrong value must not ground.
+ for labels in (names[1::-1] + names[2:], [names[0]] * 2 + names[2:],
+ names[:3] + ['Depth of selection']):
+ with self.subTest(labels=labels), self.assertRaises(GroundingUnavailable):
+ ground(relabel(INKSCAPE_SELECTED, labels), 'inkscape', 'move')
+ good = relabel(INKSCAPE_SELECTED, names)
+ bad = copy.deepcopy(good)
+ bad['elements'][3]['value'] = '81.0'
+ bad['tree_markdown'] = bad['tree_markdown'].replace('value="80.0"', 'value="81.0"')
+ with self.assertRaises(GroundingUnavailable):
+ ground(bad, 'inkscape', 'move')
+ with self.assertRaises(GroundingUnavailable):
+ ground({**good, 'elements': good['elements'] + [good['elements'][1]]}, 'inkscape', 'move')
+ with self.assertRaises(GroundingUnavailable):
+ ground({**good, 'tree_markdown': good['tree_markdown'].replace('"Y:"', '"X:"')},
+ 'inkscape', 'move')
+
def test_calc_grounding_rejects_empty_tree_dialog_and_wrong_selection(self):
# Inkscape stages are owned by the two Inkscape grounding tables above.
ground(CALC, 'calc', 'insert')
@@ -292,6 +372,32 @@ class InputTests(unittest.TestCase):
self.assertEqual([call.args[0] for call in calls],
['get_window_state', 'list_windows', 'type_text', 'get_window_state', 'list_windows'])
self.assertEqual(calls[2].args[1], {**TARGET, 'text': 'abc', 'delivery_mode': 'background'})
+ for call in (calls[0], calls[3]):
+ self.assertEqual(call.args[1], {**TARGET, 'timeout_ms': OBSERVATION_TIMEOUT_MS})
+ self.assertEqual(OBSERVATION_TIMEOUT_MS, 15000)
+
+ def test_truncated_or_unproven_tree_sends_no_input(self):
+ for change in ({'truncated': True}, {'elements_complete': False}, {'degraded': True},
+ {'timeout_ms': 1000}, {'truncated': None}, {'timeout_ms': None}):
+ mcp = Mock()
+ mcp.tool.side_effect = observed({**CALC, **change})
+ with self.subTest(change=change), self.assertRaisesRegex(GroundingUnavailable, 'not proven complete'):
+ input_step(mcp, TARGET, 'cua-smoke-calc.ods', 'calc', 'insert', 'type_text', {'text': 'abc'})
+ self.assertEqual([call.args[0] for call in mcp.tool.call_args_list], ['get_window_state'])
+ for missing in ('truncated', 'elements_complete', 'timeout_ms'):
+ state = {key: value for key, value in CALC.items() if key != missing}
+ mcp = Mock()
+ mcp.tool.side_effect = observed(state)
+ with self.subTest(missing=missing), self.assertRaises(GroundingUnavailable):
+ input_step(mcp, TARGET, 'cua-smoke-calc.ods', 'calc', 'insert', 'type_text', {'text': 'abc'})
+ self.assertEqual(mcp.tool.call_count, 1)
+
+ def test_truncated_tree_after_input_still_fails_delivery_check_first(self):
+ mcp = Mock()
+ partial = {'structuredContent': {**GOOD_DELIVERY['structuredContent'], 'effect': 'partial'}}
+ mcp.tool.side_effect = [*observed(CALC), partial, *observed({**CALC, 'truncated': True})]
+ with self.assertRaisesRegex(AssertionError, 'partial/refused input cannot pass'):
+ input_step(mcp, TARGET, 'cua-smoke-calc.ods', 'calc', 'insert', 'type_text', {'text': 'abc'})
def test_missing_grounding_sends_no_input(self):
mcp = Mock()
@@ -319,5 +425,278 @@ class InputTests(unittest.TestCase):
self.assertEqual(mcp.tool.call_count, 3)
+class ObjectsSetupTests(unittest.TestCase):
+ def setUp(self):
+ self.temp = tempfile.TemporaryDirectory()
+ self.addCleanup(self.temp.cleanup)
+ self.directory = Path(self.temp.name).resolve()
+
+ def prepare(self, side_effect):
+ mcp = Mock()
+ mcp.directory = self.directory
+ mcp.tool.side_effect = side_effect
+ return mcp, lambda: prepare_inkscape_objects(mcp, TARGET, 'cua-smoke-inkscape.svg', self.directory)
+
+ @staticmethod
+ def names(mcp):
+ return [call.args[0] for call in mcp.tool.call_args_list]
+
+ def test_existing_object_needs_no_setup_click(self):
+ mcp, run = self.prepare(observed(INKSCAPE))
+ self.assertEqual(run(), {'performed': False})
+ self.assertEqual(self.names(mcp), ['get_window_state', 'list_windows'])
+
+ def test_missing_object_uses_one_semantic_background_click_then_fresh_verification(self):
+ mcp, run = self.prepare([*observed(INKSCAPE_NO_OBJECTS), SETUP_CLICK, *observed(INKSCAPE)])
+ setup = run()
+ self.assertEqual(self.names(mcp), ['get_window_state', 'list_windows', 'click',
+ 'get_window_state', 'list_windows'])
+ self.assertEqual(mcp.tool.call_args_list[2].args[1],
+ {**TARGET, 'element_token': 'token-20', 'delivery_mode': 'background'})
+ self.assertEqual((setup['kind'], setup['plugin_input_proof'], setup['performed']),
+ ('accessibility_setup', False, True))
+ self.assertTrue((self.directory / 'objects-panel-setup.json').is_file())
+
+ def test_ungrounded_opener_sends_nothing(self):
+ def mutate(**change):
+ state = copy.deepcopy(INKSCAPE_NO_OBJECTS)
+ state['elements'][1].update(change)
+ return state
+ duplicate = copy.deepcopy(INKSCAPE_NO_OBJECTS)
+ duplicate['elements'].append({**duplicate['elements'][1], 'element_index': 21})
+ cases = [mutate(description='Open Objects'), mutate(description=OPEN_OBJECTS_DESCRIPTION + ' '),
+ mutate(role='menu item'), mutate(enabled=False), mutate(actions=[]),
+ mutate(element_token=''), mutate(element_index=99), duplicate]
+ cases.append({**INKSCAPE_NO_OBJECTS, 'elements': INKSCAPE_NO_OBJECTS['elements'][:1]})
+ cases.append({**INKSCAPE_NO_OBJECTS, 'tree_markdown': INKSCAPE_NO_OBJECTS['tree_markdown'].replace(
+ 'button', 'label')})
+ for state in cases:
+ mcp, run = self.prepare(observed(state))
+ with self.subTest(state=state), self.assertRaises(GroundingUnavailable):
+ run()
+ self.assertEqual(self.names(mcp), ['get_window_state', 'list_windows'])
+
+ def test_exact_offscreen_annotation_allows_semantic_setup(self):
+ state = copy.deepcopy(INKSCAPE_NO_OBJECTS)
+ state['elements'][1]['description'] += OFFSCREEN_DESCRIPTION
+ mcp, run = self.prepare([*observed(state), SETUP_CLICK, *observed(INKSCAPE)])
+ self.assertTrue(run()['performed'])
+ self.assertEqual(self.names(mcp).count('click'), 1)
+
+ def test_setup_refuses_incomplete_or_nonsemantic_delivery(self):
+ for change in ({'effect': 'partial'}, {'effect': 'refused'}, {'effect': 'unknown'},
+ {'route': 'synthetic_events'}, {'delivery': None},
+ {'delivery': {'mode': 'unknown'}}):
+ response = {'structuredContent': {**SETUP_CLICK['structuredContent'], **change}}
+ mcp, run = self.prepare([*observed(INKSCAPE_NO_OBJECTS), response])
+ with self.subTest(change=change), self.assertRaises(AssertionError):
+ run()
+ self.assertEqual(self.names(mcp).count('click'), 1)
+
+ def test_setup_requires_complete_tree_single_window_and_no_dialog(self):
+ mcp, run = self.prepare(observed({**INKSCAPE_NO_OBJECTS, 'truncated': True}))
+ with self.assertRaisesRegex(GroundingUnavailable, 'not proven complete'):
+ run()
+ self.assertEqual(self.names(mcp), ['get_window_state'])
+ extra = {'structuredContent': {'windows': [TARGET, {'pid': 123, 'window_id': 789}]}}
+ mcp, run = self.prepare([{'structuredContent': INKSCAPE_NO_OBJECTS}, extra])
+ with self.assertRaisesRegex(GroundingUnavailable, 'extra app window'):
+ run()
+ self.assertNotIn('click', self.names(mcp))
+ dialog = {**INKSCAPE_NO_OBJECTS,
+ 'elements': INKSCAPE_NO_OBJECTS['elements'] + [{'role': 'dialog'}]}
+ mcp, run = self.prepare(observed(dialog))
+ with self.assertRaisesRegex(GroundingUnavailable, 'unexpected dialog'):
+ run()
+ self.assertNotIn('click', self.names(mcp))
+
+ def test_ambiguous_or_disabled_existing_object_is_not_setup(self):
+ for rows in ([INKSCAPE['elements'][1], {**INKSCAPE['elements'][1], 'element_index': 13}],
+ [{**INKSCAPE['elements'][1], 'enabled': False}]):
+ mcp, run = self.prepare(observed({**INKSCAPE, 'elements': rows}))
+ with self.assertRaisesRegex(GroundingUnavailable, 'ambiguous or disabled'):
+ run()
+ self.assertEqual(self.names(mcp), ['get_window_state', 'list_windows'])
+
+ def test_click_result_must_be_background_and_never_retried(self):
+ for response in ({'structuredContent': {'delivery': {'mode': 'foreground'}}},
+ {'isError': True, 'structuredContent': {}}):
+ mcp, run = self.prepare([*observed(INKSCAPE_NO_OBJECTS), response])
+ with self.assertRaises(AssertionError):
+ run()
+ self.assertEqual(self.names(mcp).count('click'), 1)
+ mcp, run = self.prepare([*observed(INKSCAPE_NO_OBJECTS), TimeoutError('unknown')])
+ with patch('production_app_smoke.read', side_effect=OSError('unavailable')), \
+ self.assertRaises(TimeoutError):
+ run()
+ self.assertEqual(self.names(mcp).count('click'), 1)
+ self.assertEqual(mcp.tool.call_count, 3)
+
+ def test_unverified_result_after_click_fails_closed(self):
+ duplicate = {**INKSCAPE, 'elements': INKSCAPE['elements'] + [
+ {**INKSCAPE['elements'][1], 'element_index': 13}]}
+ for after in (INKSCAPE_NO_OBJECTS, duplicate, {**INKSCAPE, 'truncated': True},
+ {**INKSCAPE, 'elements': [{**INKSCAPE['elements'][1], 'enabled': False}]}):
+ mcp, run = self.prepare([*observed(INKSCAPE_NO_OBJECTS), SETUP_CLICK, *observed(after)])
+ with self.subTest(after=after), self.assertRaises(GroundingUnavailable):
+ run()
+ self.assertEqual(self.names(mcp).count('click'), 1)
+ self.assertNotIn('hotkey', self.names(mcp))
+
+ def test_inkscape_run_sets_up_before_raw_input_and_marks_it_not_plugin_proof(self):
+ document = create_documents(self.directory, 'inkscape-only')['inkscape']
+ mcp = Mock()
+ mcp.directory = self.directory
+ mcp.tool.side_effect = [{'structuredContent': {}},
+ *observed(INKSCAPE_NO_OBJECTS), SETUP_CLICK, *observed(INKSCAPE),
+ *observed(INKSCAPE), TimeoutError('unknown hotkey')]
+ with patch('production_app_smoke.Path.iterdir', return_value=iter([])), \
+ patch('production_app_smoke.discover', return_value=(TARGET, {})) as discover, \
+ patch('production_app_smoke.read', return_value='{"pid": 999}'), \
+ self.assertRaises(TimeoutError):
+ run_app(mcp, 'inkscape', document, self.directory)
+ self.assertEqual(self.names(mcp), ['launch_app', 'get_window_state', 'list_windows', 'click',
+ 'get_window_state', 'list_windows',
+ 'get_window_state', 'list_windows', 'hotkey'])
+ tag = inkscape_app_id_tag(self.directory)
+ launch = mcp.tool.call_args_list[0].args[1]
+ self.assertEqual(launch['additional_arguments'], [f'--app-id-tag={tag}', str(document)])
+ self.assertEqual(discover.call_args.args[4], [f'--app-id-tag={tag}'])
+ self.assertEqual(mcp.tool.call_args_list[8].args[1],
+ {**TARGET, 'keys': ['ctrl', 'a'], 'delivery_mode': 'background'})
+
+ def test_calc_run_has_no_setup_click(self):
+ document = create_documents(self.directory)['calc']
+ mcp = Mock()
+ mcp.directory = self.directory
+ mcp.tool.side_effect = [{'structuredContent': {}}, *observed(CALC), TimeoutError('unknown')]
+ with patch('production_app_smoke.Path.iterdir', return_value=iter([])), \
+ patch('production_app_smoke.discover', return_value=(TARGET, {})), \
+ patch('production_app_smoke.read', return_value='{"pid": 999}'), \
+ self.assertRaises(TimeoutError):
+ run_app(mcp, 'calc', document, self.directory)
+ self.assertNotIn('click', self.names(mcp))
+
+ def test_observation_helper_requests_explicit_budget(self):
+ mcp = Mock()
+ mcp.tool.side_effect = observed(INKSCAPE)
+ observe(mcp, TARGET, 'cua-smoke-inkscape.svg')
+ self.assertEqual(mcp.tool.call_args_list[0].args,
+ ('get_window_state', {**TARGET, 'timeout_ms': 15000}))
+
+
+UNPROVEN = {'degraded': True,
+ 'degraded_reason': 'accessibility_window_identity_unproven: tree is application-scoped'}
+
+
+class ObservationRetryTests(unittest.TestCase):
+ FILENAME = 'cua-smoke-calc.ods'
+
+ def names(self, mcp):
+ return [call.args[0] for call in mcp.tool.call_args_list]
+
+ def test_transient_identity_refusal_succeeds_on_second_observation(self):
+ with tempfile.TemporaryDirectory() as temp:
+ mcp = Mock()
+ mcp.directory = Path(temp)
+ mcp.tool.side_effect = [*observed({**CALC, **UNPROVEN}), *observed(CALC)]
+ self.assertEqual(observe(mcp, TARGET, self.FILENAME), CALC)
+ self.assertEqual(self.names(mcp), ['get_window_state', 'list_windows'] * 2)
+ record = json.loads((Path(temp) / 'observation-retries.json').read_text())
+ self.assertEqual(record['max_attempts'], OBSERVATION_ATTEMPTS)
+ self.assertEqual([row['attempt'] for row in record['attempts']], [1])
+ self.assertTrue(record['attempts'][0]['degraded_reason'].startswith(
+ 'accessibility_window_identity_unproven'))
+
+ def test_persistent_identity_refusal_stops_after_fixed_bound(self):
+ mcp = Mock()
+ mcp.tool.side_effect = [*observed({**CALC, **UNPROVEN})] * 5
+ with self.assertRaisesRegex(GroundingUnavailable, 'still unproven'):
+ observe(mcp, TARGET, self.FILENAME)
+ self.assertEqual(OBSERVATION_ATTEMPTS, 2)
+ self.assertEqual(self.names(mcp).count('get_window_state'), 2)
+
+ def test_non_retryable_states_fail_immediately(self):
+ for change in ({'truncated': True, **UNPROVEN}, {'elements_complete': False, **UNPROVEN},
+ {'timeout_ms': 1000, **UNPROVEN}, {'degraded': True, 'degraded_reason': 'other'},
+ {'degraded': True}, {'truncated': True}):
+ mcp = Mock()
+ mcp.tool.side_effect = observed({**CALC, **change})
+ with self.subTest(change=change), self.assertRaisesRegex(GroundingUnavailable, 'not proven complete'):
+ observe(mcp, TARGET, self.FILENAME)
+ self.assertEqual(self.names(mcp), ['get_window_state'])
+
+ def test_wrong_title_fails_immediately(self):
+ mcp = Mock()
+ mcp.tool.side_effect = observed({**CALC, **UNPROVEN, 'window_title': 'other.ods'})
+ with self.assertRaisesRegex(AssertionError, 'not the synthetic document'):
+ observe(mcp, TARGET, self.FILENAME)
+ self.assertEqual(self.names(mcp), ['get_window_state'])
+
+ def test_wrong_windows_fail_on_every_attempt_without_retrying(self):
+ extra = {'structuredContent': {'windows': [TARGET, {'pid': 123, 'window_id': 789}]}}
+ wrong = {'structuredContent': {'windows': [{'pid': 123, 'window_id': 999}]}}
+ for windows in (extra, wrong):
+ mcp = Mock()
+ mcp.tool.side_effect = [{'structuredContent': {**CALC, **UNPROVEN}}, windows]
+ with self.subTest(windows=windows), self.assertRaisesRegex(GroundingUnavailable, 'extra app window'):
+ observe(mcp, TARGET, self.FILENAME)
+ self.assertEqual(mcp.tool.call_count, 2)
+ # The second attempt is checked too, even when the first window list was fine.
+ mcp = Mock()
+ mcp.tool.side_effect = [*observed({**CALC, **UNPROVEN}),
+ {'structuredContent': CALC}, extra]
+ with self.assertRaisesRegex(GroundingUnavailable, 'extra app window'):
+ observe(mcp, TARGET, self.FILENAME)
+
+ def test_retry_never_replays_input(self):
+ mcp = Mock()
+ mcp.tool.side_effect = [*observed(CALC), GOOD_DELIVERY,
+ *observed({**CALC, **UNPROVEN}), *observed(CALC)]
+ input_step(mcp, TARGET, self.FILENAME, 'calc', 'insert', 'type_text', {'text': 'abc'})
+ self.assertEqual(self.names(mcp), ['get_window_state', 'list_windows', 'type_text',
+ 'get_window_state', 'list_windows',
+ 'get_window_state', 'list_windows'])
+ self.assertEqual(self.names(mcp).count('type_text'), 1)
+
+
+class RawInputReportTests(unittest.TestCase):
+ def test_input_step_marks_attempt_before_dispatch(self):
+ mcp = Mock()
+ seen = []
+ responses = [*observed(CALC)]
+
+ def tool(name, arguments):
+ if name == 'type_text':
+ seen.append(mcp.raw_input_attempted)
+ raise TimeoutError('unknown')
+ return responses.pop(0)
+ mcp.tool.side_effect = tool
+ mcp.raw_input_attempted = False
+ with patch('production_app_smoke.read', side_effect=OSError('unavailable')), self.assertRaises(TimeoutError):
+ input_step(mcp, TARGET, 'cua-smoke-calc.ods', 'calc', 'insert', 'type_text', {'text': 'abc'})
+ self.assertEqual(seen, [True])
+
+ def test_grounding_loss_after_raw_attempt_is_failed_not_delivered(self):
+ mcp = Mock()
+ mcp.raw_input_attempted = True
+ outcome = grounding_outcome(mcp, GroundingUnavailable('lost'))
+ self.assertEqual(outcome, {'result': 'failed', 'blocker': 'lost', 'raw_input_attempted': True})
+ self.assertNotIn('actions_delivered', outcome)
+
+ def test_grounding_loss_without_raw_attempt_stays_inspection_only(self):
+ for mcp in (Mock(), None): # A bare Mock attribute is truthy but not True.
+ with self.subTest(mcp=mcp):
+ self.assertEqual(grounding_outcome(mcp, GroundingUnavailable('lost')),
+ {'result': 'inspection_only', 'blocker': 'lost'})
+
+ def test_post_input_grounding_failure_is_reported_failed(self):
+ mcp = Mock()
+ mcp.tool.side_effect = [*observed(CALC), GOOD_DELIVERY, *observed({**CALC, 'truncated': True})]
+ with self.assertRaises(GroundingUnavailable) as caught:
+ input_step(mcp, TARGET, 'cua-smoke-calc.ods', 'calc', 'insert', 'type_text', {'text': 'abc'})
+ self.assertEqual(grounding_outcome(mcp, caught.exception)['result'], 'failed')
+
+
if __name__ == '__main__':
unittest.main()
diff --git a/libs/cua-driver/hyprland-plugin/tests/production_cancel_proof_test.py b/libs/cua-driver/hyprland-plugin/tests/production_cancel_proof_test.py
index 13531a41ee..3089026643 100644
--- a/libs/cua-driver/hyprland-plugin/tests/production_cancel_proof_test.py
+++ b/libs/cua-driver/hyprland-plugin/tests/production_cancel_proof_test.py
@@ -556,6 +556,7 @@ class OwnershipTests(unittest.TestCase):
grounded_snapshot(mcp, spec['target'], spec)
def test_inkscape_pointer_snapshot_bounds_walk_without_limiting_depth(self):
+ self.assertEqual(POINTER_SNAPSHOT_LIMITS['inkscape'], {'max_elements': 3000, 'timeout_ms': 15000})
spec = {**plan()['agents'][1], 'drag': {}, 'pointer_stage': 'move_rectangle'}
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
diff --git a/libs/cua-driver/hyprland-plugin/tests/production_pointer_grounding_test.py b/libs/cua-driver/hyprland-plugin/tests/production_pointer_grounding_test.py
index 05c13cadd3..347508359e 100644
--- a/libs/cua-driver/hyprland-plugin/tests/production_pointer_grounding_test.py
+++ b/libs/cua-driver/hyprland-plugin/tests/production_pointer_grounding_test.py
@@ -269,6 +269,55 @@ class PointerGroundingTests(unittest.TestCase):
with self.assertRaises(pointer.GroundingUnavailable):
pointer.verify(after, after_image, drag)
+ @staticmethod
+ def relabel(state, labels):
+ """Replace spin-button labels (by element index) in both projections."""
+ for row in state['elements']:
+ if row.get('element_index') in labels:
+ old = f'[{row["element_index"]}] spin button "{row["label"]}"'
+ row['label'] = labels[row['element_index']]
+ state['tree_markdown'] = state['tree_markdown'].replace(
+ old, f'[{row["element_index"]}] spin button "{row["label"]}"')
+ return state
+
+ SEMANTIC = {2: 'Horizontal coordinate of selection', 3: 'Vertical coordinate of selection',
+ 4: 'Width of selection', 5: 'Height of selection'}
+
+ def test_inkscape_geometry_accepts_semantic_and_mixed_axis_labels(self):
+ expected = {'X': 40.0, 'Y': 60.0, 'W': 80.0, 'H': 50.0}
+ state, _ = ink()
+ self.assertEqual(pointer.inkscape_geometry(self.relabel(state, self.SEMANTIC)), expected)
+ state, _ = ink()
+ mixed = {index: self.SEMANTIC[index] for index in (3, 5)}
+ self.assertEqual(pointer.inkscape_geometry(self.relabel(state, mixed)), expected)
+
+ def test_inkscape_geometry_rejects_wrong_ambiguous_or_mismatched_labels(self):
+ semantic = self.SEMANTIC
+ for failure in ('wrong_axis', 'arbitrary', 'numeric_mismatch', 'nonfinite_label',
+ 'nonfinite_value', 'duplicate_control', 'markdown_mismatch'):
+ with self.subTest(failure=failure):
+ state, _ = ink()
+ if failure == 'wrong_axis':
+ self.relabel(state, {**semantic, 2: semantic[3], 3: semantic[2]})
+ elif failure == 'arbitrary':
+ self.relabel(state, {**semantic, 4: 'Width'})
+ elif failure == 'numeric_mismatch':
+ self.relabel(state, {**semantic, 2: '41.000'})
+ elif failure == 'nonfinite_label':
+ self.relabel(state, {**semantic, 2: 'nan'})
+ elif failure == 'nonfinite_value':
+ self.relabel(state, semantic)
+ state['elements'][1]['value'] = 'nan'
+ state['tree_markdown'] = state['tree_markdown'].replace('value="40.0"', 'value="nan"')
+ elif failure == 'duplicate_control':
+ self.relabel(state, semantic)
+ state['elements'].append(copy.deepcopy(state['elements'][1]))
+ else:
+ self.relabel(state, semantic)
+ state['elements'][1]['label'] = '40.000'
+ with self.assertRaises(pointer.GroundingUnavailable):
+ pointer.inkscape_geometry(state)
+
def test_inkscape_hover_does_not_admit_ambiguous_unselected_or_in_progress_state(self):
selected = 'Rectangle in root. Click selection again to toggle scale/rotation handles.'
center = ('Center of transformation: drag to reposition; scaling, rotation '
diff --git a/libs/cua-driver/hyprland-plugin/tests/production_primary_observer_test.py b/libs/cua-driver/hyprland-plugin/tests/production_primary_observer_test.py
index 630807db52..79d027d6b0 100644
--- a/libs/cua-driver/hyprland-plugin/tests/production_primary_observer_test.py
+++ b/libs/cua-driver/hyprland-plugin/tests/production_primary_observer_test.py
@@ -248,6 +248,45 @@ class ObserverAnalysisTests(unittest.TestCase):
with self.assertRaises(AssertionError):
primary_wire_state(data)
+ def test_stable_baseline_lock_passes_and_is_reported_but_held_state_still_refuses(self):
+ locked = BASE.replace(b'modifiers(3, 0, 0, 0, 0)', b'modifiers(3, 0, 0, 16, 0)')
+ self.assertEqual(primary_wire_state(BASE)['baseline_locked_modifiers'], 0)
+ self.assertEqual(primary_wire_state(locked)['baseline_locked_modifiers'], 16)
+ for modifiers in (b'1, 0, 16, 0', b'0, 2, 16, 0', b'0, 0, 16, 1', b'4, 0, 0, 0',
+ b'0, 0, -1, 0', b'0, 0, 4294967296, 0'):
+ with self.subTest(modifiers=modifiers), self.assertRaisesRegex(AssertionError, 'not idle'):
+ primary_wire_state(BASE.replace(b'modifiers(3, 0, 0, 0, 0)', b'modifiers(3, ' + modifiers + b')'))
+ with self.assertRaisesRegex(AssertionError, 'not idle'):
+ primary_wire_state(locked + wire('wl_keyboard#6.key(5, 101, 69, 1)'))
+ with self.assertRaisesRegex(AssertionError, 'not idle'):
+ primary_wire_state(wire('wl_pointer#5.enter(1, wl_surface#10, 300.0, 300.0)',
+ 'wl_keyboard#6.enter(2, wl_surface#10, array[0])',
+ 'wl_pointer#5.button(4, 100, 272, 1)'))
+
+ def test_lock_changes_and_return_during_interval_still_fail(self):
+ base = BASE.replace(b'modifiers(3, 0, 0, 0, 0)', b'modifiers(3, 0, 0, 16, 0)')
+ def run_with(events):
+ values = list(observation(events))
+ data = base + values[3][len(BASE):]
+ offset = len(base) - len(BASE)
+ for marker in values[:2]:
+ marker = marker['marker']
+ marker['wire_start'] += offset
+ marker['wire_end'] += offset
+ values[3] = data
+ return analyze(*values)
+ self.assertEqual(run_with(b'')['result'], 'passed')
+ self.assertEqual(run_with(b'')['primary']['baseline_locked_modifiers'], 16)
+ for events in (wire('wl_keyboard#6.modifiers(5, 0, 0, 0, 0)'),
+ wire('wl_keyboard#6.modifiers(5, 0, 0, 48, 0)'),
+ wire('wl_keyboard#6.modifiers(5, 0, 0, 0, 0)', 'wl_keyboard#6.modifiers(6, 0, 0, 16, 0)'),
+ wire('wl_keyboard#6.modifiers(5, 0, 0, 16, 0)')):
+ with self.subTest(events=events):
+ result = run_with(events)
+ self.assertEqual(result['result'], 'failed')
+ self.assertTrue(any(row['interface'] == 'wl_keyboard' for row in result['violations']
+ if row['kind'] == 'wire_event'))
+
def test_raw_evidence_cannot_be_partial_malformed_reordered_or_from_another_producer(self):
rows = observation()[2]
data = b''.join((json.dumps(row) + '\n').encode() for row in rows)
diff --git a/libs/cua-driver/hyprland-plugin/tests/production_realapp_proof_test.py b/libs/cua-driver/hyprland-plugin/tests/production_realapp_proof_test.py
index 9bc9f37678..19786d4745 100644
--- a/libs/cua-driver/hyprland-plugin/tests/production_realapp_proof_test.py
+++ b/libs/cua-driver/hyprland-plugin/tests/production_realapp_proof_test.py
@@ -12,9 +12,10 @@ from unittest.mock import Mock, patch
import zipfile
import proofs_path # noqa: F401 Puts ../proofs on sys.path.
+import production_pointer_grounding as pointer
from production_realapp_proof import (SMOKE_STEPS, app_process_identity, provenance,
assert_no_dispatch, assert_primary_state, check_response,
- capacity_lane, verify_capacity, check_manifest_refusal,
+ bound_drag_lanes, capacity_lane, verify_capacity, check_manifest_refusal,
manifest_tool_messages, verify_policy_cache,
passive_focus_evidence,
expected_primary_motion, move_primary, primary_acknowledgement,
@@ -22,8 +23,9 @@ from production_realapp_proof import (SMOKE_STEPS, app_process_identity, provena
PRIMARY_LIFETIME_MS, POINTER_EPISODES, run, trace_interval, validate_plan,
verify_output)
from proof_fixtures import (INKSCAPE, INKSCAPE_SELECTED, START, STOP, assert_rejects, capacity_events, capacity_plan,
- harness_args, inkscape_plan, lane, lane_status, patch_module, plan,
+ harness_args, ink, inkscape_plan, lane, lane_status, patch_module, plan,
policy_cache_plan, primary_trace as trace, run_replacements)
+from production_app_smoke import OBSERVATION_TIMEOUT_MS
def focus_evidence(rows):
@@ -385,6 +387,70 @@ class PolicyCacheTests(unittest.TestCase):
self.assertEqual(mutations, ['click', 'press_key'])
+class DragLaneBindingTests(unittest.TestCase):
+ def test_overlap_requires_serial_unique_selections_before_drags(self):
+ candidate = inkscape_plan()
+ candidate['require_overlap'] = True
+ selections = [{'agent': i, 'tool': 'hotkey', 'arguments': {'keys': ['ctrl', 'a']},
+ 'smoke_stage': 'select'} for i in range(2)]
+ drags = {'parallel': [{'agent': i, 'tool': 'drag', 'arguments': {},
+ 'pointer_stage': 'move_rectangle'} for i in range(2)]}
+ candidate['phases'] = [*selections, drags]
+ validate_plan(candidate)
+ for phases in ([drags, *selections], [selections[0], drags],
+ [*selections, selections[0], drags], [{'parallel': selections}, drags]):
+ with self.subTest(phases=phases), self.assertRaises(AssertionError):
+ validate_plan({**candidate, 'phases': phases})
+
+ @staticmethod
+ def selects(*lanes):
+ return [{'agent': i, 'tool': 'hotkey', 'smoke_stage': 'select', 'compositor_lane': lane}
+ for i, lane in enumerate(lanes)]
+
+ def test_binding_comes_from_each_agents_own_select_evidence_not_order(self):
+ self.assertEqual(bound_drag_lanes(self.selects(1, 2)), {0: 1, 1: 2})
+ self.assertEqual(bound_drag_lanes(self.selects(2, 1)), {0: 2, 1: 1})
+ # Unbound plans record no lane evidence; overlap enforcement rejects that at cleanup.
+ rows = [{'agent': 0, 'tool': 'hotkey', 'smoke_stage': 'select'},
+ {'agent': 1, 'tool': 'drag', 'pointer_stage': 'move_rectangle'}]
+ self.assertEqual(bound_drag_lanes(rows), {})
+
+ def test_shared_duplicate_or_invalid_lane_evidence_is_rejected(self):
+ for rows in (self.selects(1, 1), self.selects(1, 2) + self.selects(1), self.selects(0),
+ self.selects(3), self.selects(True), self.selects('1'), self.selects(None)):
+ with self.subTest(rows=rows), self.assertRaises(AssertionError):
+ bound_drag_lanes(rows)
+
+ def test_identical_coordinate_drags_are_ambiguous_without_but_exact_with_binding(self):
+ args, oracle = pointer.action(*ink(), 'inkscape', 'move_rectangle')
+ effect = pointer.verify(*ink(dx=35, dy=27), oracle)
+ rows = []
+ for lane_number in (1, 2):
+ rows += [('pointer_motion', 100, 200, lane_number, 0), ('agent_drag_start', 100, 200, lane_number, 0),
+ ('pointer_button', 100, 200, lane_number, 1)] + \
+ [('pointer_motion', 100, 200, lane_number, 0)] * 3 + \
+ [('pointer_button', 100, 200, lane_number, 0), ('agent_drag_end', 100, 200, lane_number, 0)]
+ data = trace(START, *rows, STOP)
+ for base in (1, 9): # Same local coordinates in both windows.
+ for offset, xy in ((0, [145, 165]), (3, [149, 168]), (4, [165, 180]), (5, [185, 195])):
+ data['events'][base + offset].extend(xy)
+ with self.assertRaisesRegex(AssertionError, 'missing or ambiguous'):
+ pointer.verify_drag_trace(data, args, effect)
+ # Swapped independently proven bindings select swapped lanes, never agent order.
+ for agents in (self.selects(1, 2), self.selects(2, 1)):
+ lanes = bound_drag_lanes(agents)
+ for agent, lane_number in lanes.items():
+ self.assertEqual(pointer.verify_drag_trace(data, args, effect, expected_lane=lanes[agent])['lane'],
+ lane_number)
+
+ def test_absent_or_ambiguous_select_trace_cannot_bind_a_lane(self):
+ before = {**trace(START), 'active': True}
+ two_lanes = {**trace(START, *capacity_events(1), *capacity_events(2)), 'active': True}
+ for after in (before, two_lanes):
+ with self.subTest(after=after['events']), self.assertRaisesRegex(AssertionError, 'exactly one compositor lane'):
+ capacity_lane(before, after, 'click')
+
+
class CapacityTests(unittest.TestCase):
def test_plan_requires_three_serial_independent_qualified_targets(self):
for app_profile, factory in (('calc-inkscape', capacity_plan),
@@ -673,11 +739,22 @@ class SmokeStageTests(unittest.TestCase):
def test_runner_grounds_full_snapshots_and_never_recovers_or_replays(self):
failures = {None: None, 'unmarked': None,
+ 'truncated': 'not proven complete', 'incomplete': 'not proven complete',
+ 'degraded': 'not proven complete', 'missing_budget': 'not proven complete',
'no_elements': 'snapshot has no semantic elements',
'dialog': 'unexpected dialog; no dismissal keys were sent',
'extra_window': 'reviewed PID/window identity is stale or ambiguous',
'after_dialog': 'unexpected dialog; no dismissal keys were sent',
- 'transport': 'unknown input outcome'}
+ 'transport': 'unknown input outcome',
+ 'identity_before_once': None, 'identity_after_once': None,
+ 'identity_before_permanent': 'not proven complete',
+ 'identity_after_permanent': 'not proven complete',
+ 'identity_window_changed': 'reviewed PID/window identity is stale or ambiguous',
+ 'identity_truncated': 'not proven complete',
+ 'identity_incomplete': 'not proven complete'}
+ no_input = ('no_elements', 'dialog', 'extra_window', 'truncated', 'incomplete', 'degraded',
+ 'missing_budget', 'identity_before_permanent', 'identity_window_changed',
+ 'identity_truncated', 'identity_incomplete')
for app, stage, elements in (
('calc', 'insert', [{'role': 'table cell', 'label': 'A1', 'selected': True}]),
('inkscape', 'select', INKSCAPE['elements']),
@@ -705,10 +782,13 @@ class SmokeStageTests(unittest.TestCase):
observer = Mock()
targets = [candidate['foreground']] + [spec['target'] for spec in candidate['agents']]
+ identity_seen, phase_count = [0], {False: 0, True: 0}
+
def tool(name, parameters):
if name == 'list_windows':
windows = copy.deepcopy(targets)
- if failure == 'extra_window':
+ if failure == 'extra_window' or (failure == 'identity_window_changed'
+ and identity_seen[0]):
windows.append({**candidate['agents'][index]['target'], 'window_id': 999})
return {'structuredContent': {'windows': windows}}
if name == 'get_window_state':
@@ -719,7 +799,23 @@ class SmokeStageTests(unittest.TestCase):
rows = []
if failure == 'dialog' or (failure == 'after_dialog' and sent):
rows = [{'role': 'dialog'}]
+ phase_count[sent] += parameters.get('timeout_ms') == OBSERVATION_TIMEOUT_MS
+ identity = ((failure in ('identity_truncated', 'identity_incomplete',
+ 'identity_window_changed') and not sent)
+ or ((failure or '').startswith('identity_before') and not sent)
+ or ((failure or '').startswith('identity_after') and sent))
+ if (failure or '').endswith('_once') and phase_count[sent] > 1:
+ identity = False
+ identity = identity and parameters.get('timeout_ms') == OBSERVATION_TIMEOUT_MS
+ identity_seen[0] += identity
return {'structuredContent': {'screenshot_width': 600, 'window_bounds': bounds,
+ 'truncated': failure in ('truncated', 'identity_truncated'),
+ 'elements_complete': failure not in ('incomplete',
+ 'identity_incomplete'),
+ 'degraded': failure == 'degraded' or identity,
+ **({'degraded_reason': 'accessibility_window_identity_unproven'
+ ': title transition'} if identity else {}),
+ 'timeout_ms': None if failure == 'missing_budget' else OBSERVATION_TIMEOUT_MS,
'tree_markdown': (INKSCAPE_SELECTED if stage == 'move'
else INKSCAPE)['tree_markdown'],
'elements': rows}}
@@ -740,7 +836,16 @@ class SmokeStageTests(unittest.TestCase):
verify_output=Mock(return_value={'verified': True})))
self.assertEqual(run(args), 0 if expected_error is None else 1)
inputs = [i for i, row in enumerate(calls) if row[0] == 'input']
- self.assertEqual(len(inputs), 0 if failure in ('no_elements', 'dialog', 'extra_window') else 1)
+ self.assertEqual(len(inputs), 0 if failure in no_input else 1)
+ snapshots = [row for row in calls if row[0] == 'snapshot'
+ and row[1].get('timeout_ms') == OBSERVATION_TIMEOUT_MS]
+ bound = {'identity_before_once': 3, 'identity_after_once': 3,
+ 'identity_before_permanent': 2, 'identity_after_permanent': 3,
+ 'identity_window_changed': 1, 'identity_truncated': 1,
+ 'identity_incomplete': 1, 'truncated': 1, 'incomplete': 1,
+ 'degraded': 1, 'missing_budget': 1}
+ if failure in bound:
+ self.assertEqual(len(snapshots), bound[failure])
if inputs:
i = inputs[0]
self.assertEqual(calls[i - 1][0], 'snapshot')
@@ -750,6 +855,8 @@ class SmokeStageTests(unittest.TestCase):
for _, parameters in (calls[i - 1], calls[i + 1]):
self.assertEqual('max_elements' in parameters, failure == 'unmarked')
self.assertEqual('max_depth' in parameters, failure == 'unmarked')
+ if failure != 'unmarked':
+ self.assertEqual(parameters['timeout_ms'], OBSERVATION_TIMEOUT_MS)
for mcp in agents + [observer]:
mcp.close.assert_called_once()
self.assertFalse(held[0])
@@ -854,7 +961,9 @@ class PointerStageTests(unittest.TestCase):
calls.append(('snapshot', parameters, folder.name, mcp.counter))
image = folder / f'{mcp.counter}.png'
image.write_bytes(b'synthetic image read by a stub')
- return {'structuredContent': {'screenshot_width': 600, 'window_bounds': bounds},
+ return {'structuredContent': {'screenshot_width': 600, 'window_bounds': bounds,
+ 'truncated': False, 'elements_complete': True,
+ 'timeout_ms': OBSERVATION_TIMEOUT_MS},
'content': [] if failure == 'missing_image' else
[{'type': 'image', 'image_file': image.name}]}
if name == 'get_desktop_state':
diff --git a/libs/cua-driver/rust/crates/cua-driver-e2e/tests/hyprland_native_observation_test.rs b/libs/cua-driver/rust/crates/cua-driver-e2e/tests/hyprland_native_observation_test.rs
index cc970e42de..58507967c7 100644
--- a/libs/cua-driver/rust/crates/cua-driver-e2e/tests/hyprland_native_observation_test.rs
+++ b/libs/cua-driver/rust/crates/cua-driver-e2e/tests/hyprland_native_observation_test.rs
@@ -95,6 +95,7 @@ fn native_window_capture_and_background_ax_keep_exact_identity() {
baseline.target_z,
TargetZ::BackgroundVisible | TargetZ::BackgroundOccluded
));
+ driver.start_behavior_recording();
let (clicked, delta) = observer
.observe(
&[OracleKind::Focus, OracleKind::ZOrder, OracleKind::Cursor],