Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.
Rust agentai-agentapplecomputer-usecomputer-use-agentcontainerizationcuadesktop-automationhacktoberfestlumemacosmanusoperatorswiftvirtualizationvirtualization-frameworkwindowswindows-sandbox
fix(cua-driver): accept every Enter-key spelling in the Linux terminal gate (#3658)
bennybuoyf1d8388714bb
diff --git a/docs/content/docs/reference/cua-driver/mcp-tools-linux.mdx b/docs/content/docs/reference/cua-driver/mcp-tools-linux.mdxindex 3e3b040b3d..86bd61736d 100644--- a/docs/content/docs/reference/cua-driver/mcp-tools-linux.mdx+++ b/docs/content/docs/reference/cua-driver/mcp-tools-linux.mdx@@ -369,7 +369,7 @@ Press a key via XSendEvent to a window. No focus steal.- `delivery_mode` (string, optional): Input delivery mode. 'background' (default) never activates or raises the target window. On X11 it injects via XTEST / the XInput2 master pointer (no focus steal). On Wayland it goes through libei + xdg-desktop-portal, which injects to the compositor's input focus — Wayland's security model has no per-window background targeting, so a specific non-focused window cannot be aimed at; when no libei backend is available the tool returns a structured background_unavailable error. 'foreground' is the explicit escalation: activate the target (X11 _NET_ACTIVE_WINDOW; Wayland compositor activate), inject, then restore the prior active window — a brief focus swap unless the target was already active. Matches the macOS / Windows delivery_mode surface. default: `"background"`- `element_index` (integer, optional): Element index from get_window_state. Requires the matching `snapshot_id` alongside it. Prefer `element_token`, which carries both values.- `element_token` (string, optional): Opaque per-snapshot element handle from `structuredContent.elements[].element_token`. If element_index, snapshot_id, or window_id are also supplied they must agree. Returns an explicit stale error once a newer snapshot supersedes it.-- `key` (string, required)+- `key` (string, required): Key name: enter/return, tab, escape, space, backspace, delete, insert, home, end, pageup, pagedown, up, down, left, right, f1-f12, or any single ASCII character.- `modifiers` (array of string, optional)- `pid` (integer, optional)- `scope` (string, optional): default: `"window"`diff --git a/libs/cua-driver/rust/crates/platform-linux/src/input/mod.rs b/libs/cua-driver/rust/crates/platform-linux/src/input/mod.rsindex 3d34fc0759..447450917d 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/input/mod.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/input/mod.rs@@ -4016,7 +4016,11 @@ fn char_to_keycode_shift(mapping: &GetKeyboardMappingReply, keysym: u32) -> Opti/// Map a human key name (e.g. "Return", "F5", "a") to its X11 keysym. Pure name/// resolution — no server interaction — split out from keycode lookup so the/// keysym can be remapped onto a spare keycode when the keymap lacks it.-fn key_name_to_keysym(key: &str) -> Result<u32> {+/// Resolve a key name in the shared X keysym vocabulary, or error on an+/// unknown name. Exposed crate-wide so input-adjacent paths (e.g. the+/// press_key terminal short-circuit gate) can test key identity against+/// this canonical mapping instead of a local string comparison.+pub(crate) fn key_name_to_keysym(key: &str) -> Result<u32> {// Common X11 keysym names.let keysym: u32 = match key.to_lowercase().as_str() {"return" | "enter" => 0xFF0D,diff --git a/libs/cua-driver/rust/crates/platform-linux/src/tools/impl_.rs b/libs/cua-driver/rust/crates/platform-linux/src/tools/impl_.rsindex fcc7de5655..a56354bd21 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/tools/impl_.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/tools/impl_.rs@@ -5641,6 +5641,15 @@ fn terminal_tty_for_window(pid: u32, xid: u64) -> Option<PathBuf> {ttys.get(window_index).cloned()}+/// True when `key` names the Enter key in the shared X keysym vocabulary+/// (`key_name_to_keysym`). The terminal pty short-circuit below applies to+/// every spelling of that physical key — `enter`, `return`, any case — so an+/// agent following the documented key names cannot silently lose the keypress+/// on a terminal window (terminals discard synthetic XSendEvent keys).+fn is_enter_key(key: &str) -> bool {+ crate::input::key_name_to_keysym(key).ok() == Some(0xFF0D)+}+/// Type into a terminal window without touching X focus. Resolves the window's/// pty, then borrows the emulator's master fd and writes to it (see/// `crate::tty`). Returns `Ok(false)` when the target isn't a terminal we can@@ -7995,7 +8004,7 @@ impl Tool for PressKeyTool {"session": cua_driver_core::tool_schema::session_schema(),"pid":{"type":"integer"},"window_id":{"type":"integer"},- "key":{"type":"string"},+ "key":{"type":"string","description":"Key name: enter/return, tab, escape, space, backspace, delete, insert, home, end, pageup, pagedown, up, down, left, right, f1-f12, or any single ASCII character."},"modifiers":{"type":"array","items":{"type":"string"}},"element_index": cua_driver_core::tool_schema::element_index_schema(),"element_token": cua_driver_core::tool_schema::element_token_schema(),@@ -8333,7 +8342,7 @@ impl Tool for PressKeyTool {move || -> anyhow::Result<KeyRoute> {if resolved_element_index.is_none()&& mods.is_empty()- && key_for_task.eq_ignore_ascii_case("enter")+ && is_enter_key(&key_for_task){if inject_terminal_input(pid, xid, "\n")? {return Ok(KeyRoute::Terminal);diff --git a/libs/cua-driver/rust/crates/platform-linux/src/tools/impl_/press_key_tests.rs b/libs/cua-driver/rust/crates/platform-linux/src/tools/impl_/press_key_tests.rsindex 88f4576fda..f4911628a7 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/tools/impl_/press_key_tests.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/tools/impl_/press_key_tests.rs@@ -1,3 +1,4 @@+use super::is_enter_key;use super::press_key_chord;#[test]@@ -5,6 +6,32 @@ fn unmodified_press_stays_on_the_single_key_route() {assert_eq!(press_key_chord(&[], "return"), None);}+#[test]+fn enter_key_gate_accepts_the_documented_enter_spelling() {+ assert!(is_enter_key("enter"));+ assert!(is_enter_key("Enter"));+ assert!(is_enter_key("ENTER"));+}++#[test]+fn enter_key_gate_accepts_the_documented_return_spelling() {+ // `key_name_to_keysym` resolves "return" and "enter" to the same+ // physical key (XK_Return, 0xFF0D); the pty short-circuit must not+ // drop a documented spelling on terminals (they discard synthetic+ // XSendEvent keys, so a dropped short-circuit loses the keypress).+ assert!(is_enter_key("return"));+ assert!(is_enter_key("Return"));+ assert!(is_enter_key("RETURN"));+}++#[test]+fn enter_key_gate_rejects_non_enter_keys() {+ assert!(!is_enter_key("tab"));+ assert!(!is_enter_key("escape"));+ assert!(!is_enter_key("space"));+ assert!(!is_enter_key(""));+}+#[test]fn modifiers_are_promoted_to_a_chord_in_order() {assert_eq!(