Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.
Rust agentai-agentapplecomputer-usecomputer-use-agentcontainerizationcuadesktop-automationhacktoberfestlumemacosmanusoperatorswiftvirtualizationvirtualization-frameworkwindowswindows-sandbox
fix(cua-driver): avoid retrying deferred text writes
Chris Hynes7e0d6bcb5d7c
diff --git a/.github/workflows/ci-rust-windows.yml b/.github/workflows/ci-rust-windows.ymlindex d0f9cc838c..376f881678 100644--- a/.github/workflows/ci-rust-windows.yml+++ b/.github/workflows/ci-rust-windows.yml@@ -64,6 +64,13 @@ jobs:# Compile every Rust target without executing desktop-dependent integration# tests; interactive behavior belongs in e2e-rust-windows.yml.run: cargo test -p cua-driver -p cua-driver-core -p cua-driver-sdk -p cua-driver-testkit -p cursor-overlay -p cursor-theme-cli -p platform-windows -p cua-driver-uia --all-targets --no-run --locked+ - name: Run deferred text publication regressions+ working-directory: libs/cua-driver/rust+ run: |+ cargo test -p platform-windows post_message_readback_tests:: --lib --locked+ if ($LASTEXITCODE -ne 0) { throw "PostMessage read-back tests failed with exit $LASTEXITCODE" }+ cargo test -p platform-windows value_write_readback_tests:: --lib --locked+ if ($LASTEXITCODE -ne 0) { throw "ValuePattern read-back tests failed with exit $LASTEXITCODE" }- name: Run named-pipe readiness regressionworking-directory: libs/cua-driver/rustrun: cargo test -p cua-driver-testkit stalled_pipe_attempt_is_bounded_and_does_not_block_the_next_probe --lib --lockeddiff --git a/libs/cua-driver/rust/Skills/cua-driver/SKILL.md b/libs/cua-driver/rust/Skills/cua-driver/SKILL.mdindex 97cf20a84f..57706fbb73 100644--- a/libs/cua-driver/rust/Skills/cua-driver/SKILL.md+++ b/libs/cua-driver/rust/Skills/cua-driver/SKILL.md@@ -502,6 +502,13 @@ success” above. The old `verified`, `path`, coordinates, scope, andThe full wire contract and 0.14 migration notes are in`../../../docs/action-result-contract.md`.+A successful accessibility value write can still return+`effect:"unverifiable"` when the provider publishes its new value only after+the action call unwinds. Take a fresh snapshot before retrying; an immediate+retry can duplicate text. An explicit pixel escalation is reserved for a web+surface whose accessibility layer echoed the write without proving that the+renderer observed it.+`get_window_state` itself, when the AX tree comes back empty (a non-AXsurface like Electron/Chromium/canvas), returns `degraded: true`plus an observation-specific escalation hint — normally pointing at pixels (youdiff --git a/libs/cua-driver/rust/Skills/cua-driver/WINDOWS.md b/libs/cua-driver/rust/Skills/cua-driver/WINDOWS.mdindex 028c8ddfbf..226d3ea456 100644--- a/libs/cua-driver/rust/Skills/cua-driver/WINDOWS.md+++ b/libs/cua-driver/rust/Skills/cua-driver/WINDOWS.md@@ -787,21 +787,26 @@ typed browser tools yet.- **`hotkey` / `press_key`** (keystroke + key-combo): `delivery_mode:"background"`surfaces a `background_unavailable` error for VCL.- **`type_text`** does a **UIA read-back** and returns the shared- `ActionResult`: `effect:"confirmed"` with `evidence:[{"kind":- "value_readback"}]` when the value reflects the text, and- `effect:"unverifiable"` when the value is unchanged or unreadable. Use the- optional `escalation` to choose the next rung. **Pass an- `element_index`** for reliable verification: the read-back then reads- _that specific element_ by handle (ValuePattern → TextPattern), which is- **focus-independent** — it can confirm or disprove a change whether or not- the target is foreground. (Verified live against the WPF harness: typed- via element_index, read back confirmed, value independently present in- the next snapshot — app never fronted.) **Without** an element_index it- falls back to system-wide `GetFocusedElement`, which on Windows only- resolves when the target is the **foreground** app (no per-app- `AXFocusedUIElement` like macOS); a backgrounded target then reads- an unverifiable result even when the text actually landed — so it is NOT a- failure signal; call `verify_state` or inspect a fresh screenshot.+ `ActionResult`. With an `element_index`, the ValuePattern path returns+ `effect:"confirmed"` with `evidence:[{"kind":"value_readback"}]` only+ when the complete expected value is synchronously visible and differs from+ the prior value. If SetValue succeeded but the provider still exposes the+ old or no value, the result is `effect:"unverifiable"` with no escalation.+ This is common for deferred providers such as AccessKit: take a fresh+ snapshot before retrying because the value may publish only after+ `type_text` returns and an immediate retry can duplicate text. A pixel+ escalation is reserved for an Electron/web accessibility echo that does+ not prove the renderer observed the write. The PostMessage fallback keeps+ its separate delivery/read-back behavior. Even when the value changes and+ contains the requested text, the result stays `effect:"unverifiable"` because+ WM_CHAR does not expose the insertion point. Take a fresh snapshot before+ retrying. It may recommend foreground when a background insert appears+ dropped. Passing an `element_index` makes the+ read-back target that exact element by handle (ValuePattern → TextPattern),+ independent of foreground focus. Without one, PostMessage verification+ falls back to system-wide `GetFocusedElement`, which normally resolves only+ for the foreground app; an unreadable result is therefore not proof of+ failure.Escalate to `delivery_mode:"foreground"` for both (SendInput Unicode /accelerator). **But** foreground needs the swap to actually land — if thedaemon lacks UIAccess and `bring_to_front` returns `landed_on_target:false`diff --git a/libs/cua-driver/rust/crates/cua-driver/tests/harness_wpf_test.rs b/libs/cua-driver/rust/crates/cua-driver/tests/harness_wpf_test.rsindex 33e1847824..3726992c75 100644--- a/libs/cua-driver/rust/crates/cua-driver/tests/harness_wpf_test.rs+++ b/libs/cua-driver/rust/crates/cua-driver/tests/harness_wpf_test.rs@@ -824,6 +824,60 @@ fn harness_wpf_set_value() {);}+#[test]+#[ignore]+fn harness_wpf_deferred_type_text_requires_fresh_snapshot_before_retry() {+ execute_case(+ background_case("deferred-type-text", DriverRoute::UiaValue),+ |evidence| {+ with_named_session(+ "windows-wpf-deferred-type-text-ax-background",+ |pid, wid, driver| {+ *evidence = recording_evidence(driver.recording_dir());+ let snap = snapshot(driver, pid, wid);+ let idx = ax::element_index_by_id(snap.text(), "txt-deferred-input")+ .expect("txt-deferred-input not in snapshot");+ let (response, passed) = observe_background(driver, pid, wid, |driver| {+ driver.call(+ "type_text",+ serde_json::json!({+ "pid": pid as i64,+ "window_id": wid,+ "element_index": idx,+ "snapshot_id": snap.snapshot_id(),+ "text": "deferred-once",+ "delivery_mode": "background"+ }),+ )+ });+ assert!(+ !response.is_error(),+ "type_text failed: {}",+ response.text()+ );+ assert_eq!(response.action_effect(), Some("unverifiable"));+ assert_eq!(response.action_route(), Some("accessibility"));+ assert!(+ response.structured().get("escalation").is_none(),+ "deferred publication must not recommend an immediate retry: {}",+ response.structured()+ );++ std::thread::sleep(Duration::from_millis(700));+ let post = snapshot(driver, pid, wid);+ assert!(+ post.text().contains("deferred_mirror=deferred-once"),+ "fresh snapshot did not observe exactly one deferred write: {}",+ post.text().chars().take(700).collect::<String>()+ );+ delivered_with_fixture_state(passed)+ },+ )+ .expect("required WPF session did not start")+ },+ );+}+// In test-batch mode (many harnesses launched/killed in sequence) the WPF// window's input pump occasionally misses background PostMessage events —// reproducibly passes in isolation, intermittently fails in batch.diff --git a/libs/cua-driver/rust/crates/platform-windows/src/tools/impl_.rs b/libs/cua-driver/rust/crates/platform-windows/src/tools/impl_.rsindex 44c99290bd..3d24348ed3 100644--- a/libs/cua-driver/rust/crates/platform-windows/src/tools/impl_.rs+++ b/libs/cua-driver/rust/crates/platform-windows/src/tools/impl_.rs@@ -3919,7 +3919,7 @@ impl Tool for TypeTextTool {"pid":{"type":"integer","description":"Target process ID."},"text":{"type":"string","description":"Text to insert at the focused element's cursor."},"window_id":{"type":"integer","description":"HWND of the target window. Required when element_index is used. Optional when element_token is supplied (the token carries it)."},- "element_index":{"type":"integer","description":"Element index from the last get_window_state for the same (pid, window_id). When supplied, type_text (1) writes via UIA ValuePattern.SetValue on that element (works for WPF/WinForms/UWP/XAML without focus steal) and (2) confirms by reading that element's value back by handle — focus-independent. A matching read-back produces effect:\"confirmed\" with value_readback evidence; an unchanged or unreadable value remains effect:\"unverifiable\". Strongly preferred over typing into 'whatever is focused' (no element_index), which falls back to PostMessage WM_CHAR plus a foreground-only focused-element read-back. Requires window_id."},+ "element_index":{"type":"integer","description":"Element index from the last get_window_state for the same (pid, window_id). When supplied, type_text writes through UIA ValuePattern and reads that exact element back by handle. The shared ActionResult is confirmed only when the complete expected value is synchronously visible. If SetValue succeeds but read-back is stale or unavailable, the result is unverifiable with no escalation; take a fresh snapshot before retrying because deferred providers may publish only after this call returns. Requires window_id."},"element_token": cua_driver_core::tool_schema::element_token_schema(),"snapshot_id": cua_driver_core::tool_schema::snapshot_id_schema(),"x":{"type":"number","description":"Window-local screenshot-pixel X of the field to type into — the element px action form. Pass x,y (no element_index) and the tool pixel-clicks there to establish real renderer focus, then types. Use for Chromium/Electron inputs the UIA/WM_CHAR path can't reach. Read straight off the get_window_state PNG, same convention as click."},@@ -4250,15 +4250,12 @@ impl Tool for TypeTextTool {let idx = idx as usize;let state = self.state.clone();let text_for_uia = text.clone();- let set_ok = tokio::task::spawn_blocking(move || -> bool {+ let set_result = tokio::task::spawn_blocking(move || {// Retain the element under the cache lock so a concurrent// get_window_state snapshot-replace on the same (pid, hwnd)// can't Release it to zero while this SetValue is in flight.// The guard is held for the whole closure.- let Some(element_guard) = state.element_cache.get_element_retained(pid, hwnd, idx)- else {- return false;- };+ let element_guard = state.element_cache.get_element_retained(pid, hwnd, idx)?;let ptr = element_guard.as_ptr();use windows::core::{Interface, BSTR};use windows::Win32::UI::Accessibility::{@@ -4266,7 +4263,7 @@ impl Tool for TypeTextTool {};let elem: IUIAutomationElement =unsafe { IUIAutomationElement::from_raw(ptr as *mut _) };- let ok = (|| -> anyhow::Result<()> {+ let result = (|| -> anyhow::Result<(String, String)> {let pattern = unsafe { elem.GetCurrentPattern(UIA_ValuePatternId) }?;let vp: IUIAutomationValuePattern = pattern.cast()?;// Shield the SetValue against host self-foreground. A@@ -4285,32 +4282,28 @@ impl Tool for TypeTextTool {.map(|value| value.to_string()).unwrap_or_default();let inserted = format!("{current}{text_for_uia}");- vp.SetValue(&BSTR::from(inserted.as_str()))- })?;- Ok(())+ vp.SetValue(&BSTR::from(inserted.as_str()))?;+ Ok((current, inserted))+ })})()- .is_ok();+ .ok();std::mem::forget(elem);- ok+ result}).await- .unwrap_or(false);- if set_ok {+ .unwrap_or(None);+ if let Some((before, expected)) = set_result {// Read the value back by element handle — focus-independent// (works regardless of which window is foreground), proving the// a11y write actually took rather than trusting SetValue's// return alone.let state_rb = self.state.clone();- let text_rb = text.clone();let verify = tokio::task::spawn_blocking(move || {- match read_cached_element_value(&state_rb, pid, hwnd, idx) {- Some(v) if v.contains(text_rb.as_str()) => "confirmed",- Some(_) => "unchanged",- None => "unreadable",- }+ let value = read_cached_element_value(&state_rb, pid, hwnd, idx);+ classify_value_write_readback(value.as_deref(), &before, &expected)}).await- .unwrap_or("unreadable");+ .unwrap_or("pending");// SURFACE-AWARE VERIFICATION (mirrors macOS type_text). On// Electron/Chromium web inputs the UIA layer accepts a// `ValuePattern.SetValue` and echoes it straight back through@@ -4340,8 +4333,9 @@ impl Tool for TypeTextTool {)} else {(- "📨 Sent (unverified)",- " — driver could not confirm; verify via screenshot.".to_string(),+ "📨 Accepted (verification pending)",+ " — the provider may publish its new value only after this call returns. Take a fresh snapshot before retrying."+ .to_string(),)};return ToolResult::text(format!(@@ -4350,15 +4344,10 @@ impl Tool for TypeTextTool {)).with_structured({// `effect` mirrors the UIA read-back: a TRUSTED positive- // read-back is "confirmed"; an unchanged/unreadable value, or an- // Electron UIA echo we refuse to trust, is "unverifiable".- let mut s = serde_json::json!({- "path": "ax",- "characters": text_len,- "verified": verified,- "verify": verify,- "effect": if verified { "confirmed" } else { "unverifiable" },- });+ // read-back is "confirmed"; a deferred provider or an Electron UIA echo+ // that we refuse to trust is "unverifiable". Only the echo+ // recommends pixel escalation.+ let mut s = value_write_structured_result(text_len, verify, verified);if ax_echo_surface {// Electron UIA echo → the element px action is the next rung,// not foreground (it's a renderer-focus problem, not an@@ -4371,15 +4360,6 @@ impl Tool for TypeTextTool {re-type with the element px action (pass x,y to \pixel-focus the field, then type)."});- } else if !verified {- // The field IS in the UIA tree (it's a focus/accept problem,- // not a missing element), so escalation points at foreground.- s["escalation"] = serde_json::json!({- "recommended": "foreground",- "reason": "background insert could not be confirmed — re-call \- with delivery_mode:\"foreground\" if a screenshot \- shows the text didn't land."- });}s});@@ -4427,13 +4407,13 @@ impl Tool for TypeTextTool {let post_res = crate::input::post_type_text(hwnd, &text_for_post);std::thread::sleep(std::time::Duration::from_millis(40));let after = read(verify_idx);- let confirmed =- post_message_readback_confirms(before.as_deref(), after.as_deref(), &text_for_post);- (post_res, before, after, confirmed)+ let observed =+ post_message_readback_observed(before.as_deref(), after.as_deref(), &text_for_post);+ (post_res, before, after, observed)}).await;match result {- Ok((Ok(()), before, after, confirmed)) => {+ Ok((Ok(()), before, after, observed)) => {// Three-way verdict. Crucially, "couldn't read the field" is NOT// the same as "the text didn't land": on Windows, UIA's// GetFocusedElement is system-wide (unlike macOS's per-app@@ -4441,19 +4421,18 @@ impl Tool for TypeTextTool {// app we cannot read it back even though a PostMessage WM_CHAR// may have landed fine. Treating unreadable as failure would// spuriously push agents to foreground and break the- // background-first contract. Confirm only when the value- // changed and the resulting value contains the requested- // text; every other readable result remains unverified.+ // background-first contract. A changed value containing the+ // requested text is useful evidence, but the insertion point is+ // unknowable on WM_CHAR. Keep the action unverifiable so callers+ // prove final state from a fresh snapshot before any retry.match (&before, &after) {- (Some(_), Some(_)) if confirmed => ToolResult::text(format!(- "✅ Typed {text_len} char(s) on pid {raw_pid} via PostMessage \- ({_delay_ms}ms delay; verified via UIA read-back)."+ (Some(_), Some(_)) if observed => ToolResult::text(format!(+ "📨 Sent {text_len} char(s) to pid {raw_pid} via PostMessage \+ ({_delay_ms}ms delay; the field changed and now contains the \+ requested text, but the insertion point is not known). Take a \+ fresh snapshot to verify the intended final value before retrying."))- .with_structured(serde_json::json!({- "path": "key_events", "characters": text_len,- "verified": true, "verify": "confirmed",- "effect": "confirmed",- })),+ .with_structured(changed_contains_post_message_result(text_len)),(Some(_), Some(_)) => ToolResult::text(format!("📨 Sent {text_len} char(s) to pid {raw_pid} via PostMessage, but \the focused field's value did not contain the requested text \@@ -4498,7 +4477,7 @@ impl Tool for TypeTextTool {}}-fn post_message_readback_confirms(+fn post_message_readback_observed(before: Option<&str>,after: Option<&str>,requested_text: &str,@@ -4510,34 +4489,93 @@ fn post_message_readback_confirms()}+fn value_write_structured_result(+ text_len: usize,+ verify: &str,+ verified: bool,+) -> serde_json::Value {+ serde_json::json!({+ "path": "ax",+ "characters": text_len,+ "verified": verified,+ "verify": verify,+ "effect": if verified { "confirmed" } else { "unverifiable" },+ })+}++fn changed_contains_post_message_result(text_len: usize) -> serde_json::Value {+ serde_json::json!({+ "path": "key_events",+ "characters": text_len,+ "verified": false,+ "verify": "changed_contains",+ "effect": "unverifiable",+ })+}+#[cfg(test)]mod post_message_readback_tests {- use super::post_message_readback_confirms;+ use super::{changed_contains_post_message_result, post_message_readback_observed};#[test]- fn confirms_only_a_changed_value_containing_the_requested_text() {- assert!(post_message_readback_confirms(+ fn observes_only_a_changed_value_containing_the_requested_text() {+ assert!(post_message_readback_observed(Some("prefix"),Some("prefixhello"),"hello"));- assert!(!post_message_readback_confirms(None, None, "hello"));- assert!(!post_message_readback_confirms(+ assert!(!post_message_readback_observed(None, None, "hello"));+ assert!(!post_message_readback_observed(Some("hello"),Some("hello"),"hello"));- assert!(!post_message_readback_confirms(+ assert!(!post_message_readback_observed(Some(""),Some("h"),"hello"));- assert!(!post_message_readback_confirms(+ assert!(!post_message_readback_observed(Some("before"),Some("different"),"hello"));}++ #[test]+ fn changed_contains_remains_unverifiable_without_retry_escalation() {+ let result = changed_contains_post_message_result(5);+ assert_eq!(result["effect"], "unverifiable");+ assert_eq!(result["verify"], "changed_contains");+ assert_eq!(result["verified"], false);+ assert!(result.get("escalation").is_none());++ let record = cua_driver_core::action_record::ActionExecutionRecord::from_legacy(+ "type_text",+ &serde_json::json!({"delivery_mode": "background"}),+ &result,+ )+ .expect("changed-and-contains PostMessage result should normalize");+ let public = serde_json::to_value(record.public_result().expect("valid ActionResult"))+ .expect("serialize ActionResult");+ assert_eq!(public["effect"], "unverifiable");+ assert!(public.get("escalation").is_none());+ assert!(public.get("evidence").is_none());+ }+}++fn classify_value_write_readback(+ value: Option<&str>,+ before: &str,+ expected: &str,+) -> &'static str {+ match value {+ Some(value) if value == expected && value != before => "confirmed",+ // SetValue returned success. Some providers publish their new Value only+ // after this tool call unwinds, so an old or unreadable synchronous value+ // is pending verification, not evidence that the write failed.+ _ => "pending",+ }}/// Read a **specific cached element's** text value by its `element_index`,@@ -9959,3 +9997,65 @@ mod pid_window_target_tests {));}}++#[cfg(test)]+mod value_write_readback_tests {+ use super::{classify_value_write_readback, value_write_structured_result};++ #[test]+ fn confirms_when_the_expected_value_replaces_the_prior_value() {+ assert_eq!(+ classify_value_write_readback(Some("beforeinserted"), "before", "beforeinserted"),+ "confirmed"+ );+ }++ #[test]+ fn treats_stale_or_unreadable_value_as_pending() {+ assert_eq!(+ classify_value_write_readback(Some("old value"), "old value", "old valueinserted"),+ "pending"+ );+ assert_eq!(+ classify_value_write_readback(None, "old value", "old valueinserted"),+ "pending"+ );+ }++ #[test]+ fn deferred_publication_remains_unverifiable_without_retry_escalation() {+ let result = value_write_structured_result(8, "pending", false);+ assert_eq!(result["effect"], "unverifiable");+ assert_eq!(result["verify"], "pending");+ assert_eq!(result["verified"], false);+ assert!(result.get("escalation").is_none());++ let record = cua_driver_core::action_record::ActionExecutionRecord::from_legacy(+ "type_text",+ &serde_json::json!({"delivery_mode": "background"}),+ &result,+ )+ .expect("deferred ValuePattern result should normalize");+ let public = serde_json::to_value(record.public_result().expect("valid ActionResult"))+ .expect("serialize ActionResult");+ assert_eq!(public["effect"], "unverifiable");+ assert!(public.get("escalation").is_none());+ assert!(public.get("evidence").is_none());+ }++ #[test]+ fn does_not_confirm_when_stale_value_contains_the_typed_text() {+ assert_eq!(+ classify_value_write_readback(Some("10.00"), "10.00", "10.000"),+ "pending"+ );+ }++ #[test]+ fn does_not_confirm_an_empty_write() {+ assert_eq!(+ classify_value_write_readback(Some("unchanged"), "unchanged", "unchanged"),+ "pending"+ );+ }+}diff --git a/libs/cua-driver/tests/fixtures/apps/windows/wpf/DeferredValueTextBox.cs b/libs/cua-driver/tests/fixtures/apps/windows/wpf/DeferredValueTextBox.csnew file mode 100644index 0000000000..bb476c31cd--- /dev/null+++ b/libs/cua-driver/tests/fixtures/apps/windows/wpf/DeferredValueTextBox.cs@@ -0,0 +1,50 @@+using System;+using System.Threading.Tasks;+using System.Windows.Automation.Peers;+using System.Windows.Automation.Provider;+using System.Windows.Controls;++namespace CuaTestHarness.Wpf;++/// <summary>+/// Deterministic fixture for UIA providers that accept SetValue synchronously+/// but publish the new Value only after the provider callback has unwound.+/// </summary>+public sealed class DeferredValueTextBox : TextBox+{+ protected override AutomationPeer OnCreateAutomationPeer() =>+ new DeferredValueTextBoxAutomationPeer(this);+}++internal sealed class DeferredValueTextBoxAutomationPeer : TextBoxAutomationPeer, IValueProvider+{+ private readonly DeferredValueTextBox _owner;++ internal DeferredValueTextBoxAutomationPeer(DeferredValueTextBox owner) : base(owner)+ {+ _owner = owner;+ }++ public override object GetPattern(PatternInterface patternInterface) =>+ patternInterface == PatternInterface.Value ? this : base.GetPattern(patternInterface);++ bool IValueProvider.IsReadOnly => _owner.Dispatcher.CheckAccess()+ ? _owner.IsReadOnly+ : _owner.Dispatcher.Invoke(() => _owner.IsReadOnly);++ string IValueProvider.Value => _owner.Dispatcher.CheckAccess()+ ? _owner.Text+ : _owner.Dispatcher.Invoke(() => _owner.Text);++ void IValueProvider.SetValue(string value)+ {+ // Start the delay on the UI thread, then publish later. The driver's+ // immediate CurrentValue read therefore sees the old value, while a+ // separate snapshot after the action call observes the accepted write.+ _owner.Dispatcher.BeginInvoke(new Action(async () =>+ {+ await Task.Delay(400);+ _owner.Text = value;+ }));+ }+}diff --git a/libs/cua-driver/tests/fixtures/apps/windows/wpf/MainWindow.xaml b/libs/cua-driver/tests/fixtures/apps/windows/wpf/MainWindow.xamlindex 1f797c410c..190f9a3d28 100644--- a/libs/cua-driver/tests/fixtures/apps/windows/wpf/MainWindow.xaml+++ b/libs/cua-driver/tests/fixtures/apps/windows/wpf/MainWindow.xaml@@ -3,6 +3,7 @@xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"xmlns:sys="clr-namespace:System;assembly=mscorlib"xmlns:auto="clr-namespace:System.Windows.Automation;assembly=PresentationCore"+ xmlns:local="clr-namespace:CuaTestHarness.Wpf"Title="CuaTestHarness WPF"AutomationProperties.AutomationId="wnd-main"Width="900" Height="650"@@ -12,10 +13,8 @@Keyboard.Modifiers is sourced from GetKeyState(), so theforeground SendInput press_key case uses Ctrl+Shift+H to verifymodifier-bearing letter delivery. -->- <KeyBinding Key="F5" Command="{x:Static local:MainWindow.AccelCmd}"- xmlns:local="clr-namespace:CuaTestHarness.Wpf"/>- <KeyBinding Modifiers="Control+Shift" Key="H" Command="{x:Static local:MainWindow.AccelCmd}"- xmlns:local="clr-namespace:CuaTestHarness.Wpf"/>+ <KeyBinding Key="F5" Command="{x:Static local:MainWindow.AccelCmd}"/>+ <KeyBinding Modifiers="Control+Shift" Key="H" Command="{x:Static local:MainWindow.AccelCmd}"/></Window.InputBindings><DockPanel LastChildFill="True">@@ -46,16 +45,36 @@stays within the visible window for PostMessage clickseven when the main ScrollViewer hasn't been scrolled). --><GroupBox Header="text_input" Padding="10" Margin="0,0,0,12">- <StackPanel>- <TextBox x:Name="TxtInput"- AutomationProperties.AutomationId="txt-input"- Width="300" HorizontalAlignment="Left"- TextChanged="OnInputChanged"/>- <TextBlock x:Name="LblInputMirror"- AutomationProperties.AutomationId="lbl-input-mirror"- Margin="0,6,0,0" FontFamily="Consolas"- Text="mirror="/>- </StackPanel>+ <Grid>+ <Grid.ColumnDefinitions>+ <ColumnDefinition Width="Auto"/>+ <ColumnDefinition Width="Auto"/>+ </Grid.ColumnDefinitions>+ <StackPanel Grid.Column="0">+ <TextBox x:Name="TxtInput"+ AutomationProperties.AutomationId="txt-input"+ Width="300" HorizontalAlignment="Left"+ TextChanged="OnInputChanged"/>+ <TextBlock x:Name="LblInputMirror"+ AutomationProperties.AutomationId="lbl-input-mirror"+ Margin="0,6,0,0" FontFamily="Consolas"+ Text="mirror="/>+ </StackPanel>+ <!-- Models providers that accept ValuePattern.SetValue but+ publish their value only after the UIA call unwinds.+ Keep it beside the original input so existing controls+ retain their vertical coordinates in the E2E fixture. -->+ <StackPanel Grid.Column="1" Margin="24,0,0,0">+ <local:DeferredValueTextBox x:Name="TxtDeferredInput"+ AutomationProperties.AutomationId="txt-deferred-input"+ Width="300" HorizontalAlignment="Left"+ TextChanged="OnDeferredInputChanged"/>+ <TextBlock x:Name="LblDeferredInputMirror"+ AutomationProperties.AutomationId="lbl-deferred-input-mirror"+ Margin="0,6,0,0" FontFamily="Consolas"+ Text="deferred_mirror="/>+ </StackPanel>+ </Grid></GroupBox><!-- Click target (Button) — also at top for visibility. -->diff --git a/libs/cua-driver/tests/fixtures/apps/windows/wpf/MainWindow.xaml.cs b/libs/cua-driver/tests/fixtures/apps/windows/wpf/MainWindow.xaml.csindex e49ce58257..8799710807 100644--- a/libs/cua-driver/tests/fixtures/apps/windows/wpf/MainWindow.xaml.cs+++ b/libs/cua-driver/tests/fixtures/apps/windows/wpf/MainWindow.xaml.cs@@ -153,6 +153,11 @@ public partial class MainWindow : WindowLblInputMirror.Text = $"mirror={TxtInput.Text}";}+ private void OnDeferredInputChanged(object sender, TextChangedEventArgs e)+ {+ LblDeferredInputMirror.Text = $"deferred_mirror={TxtDeferredInput.Text}";+ }+private void OnTargetLeftDown(object sender, MouseButtonEventArgs e){_targetPointerSeen = true;