Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.
Rust agentai-agentapplecomputer-usecomputer-use-agentcontainerizationcuadesktop-automationhacktoberfestlumemacosmanusoperatorswiftvirtualizationvirtualization-frameworkwindowswindows-sandbox
feat(cua-driver): drop the session orb from the cursor badge (#2842)
Francesco Bonacci59f280e485be
diff --git a/libs/cua-driver/docs/cursor-modifier-badge-restructuring-plan.md b/libs/cua-driver/docs/cursor-modifier-badge-restructuring-plan.mdindex f29a8c5a7d..6965b6ced7 100644--- a/libs/cua-driver/docs/cursor-modifier-badge-restructuring-plan.md+++ b/libs/cua-driver/docs/cursor-modifier-badge-restructuring-plan.md@@ -13,7 +13,7 @@ Cua Driver should present two kinds of cursor information in two clear places:- delivery: background or foreground- target: AX, pixel, browser, or desktop-The badge remains lightweight. The session name fades using the existing two-second hold and 400-millisecond fade. Modifier chips appear only while their action is active, followed by a 400-millisecond trailing fade. When the name is hidden, the badge contracts to a compact session orb and active modifier chips. Hovering the pointer restores the full session name where hover observation is supported.+The badge remains lightweight. The session name fades using the existing two-second hold and 400-millisecond fade. Modifier chips appear only while their action is active, followed by a 400-millisecond trailing fade. When the name is hidden, the badge contracts to a compact session-colored capsule and active modifier chips. Hovering the pointer restores the full session name where hover observation is supported.This is visual feedback only. It is not an authorization indicator, a security boundary, or evidence that an action succeeded.@@ -40,13 +40,13 @@ Use one pill rather than adding a second persistent status capsule.| Badge content | Visibility || ------------------------- | --------------------------------------------------------------------------------------- |-| Session orb and name | Existing reveal, two-second hold, 400-millisecond fade; hover restores full opacity |+| Session name | Existing reveal, two-second hold, 400-millisecond fade; hover restores full opacity || Delivery and target chips | Full opacity while modifiers are active; 400-millisecond trailing fade after they clear || Badge chrome | Visible while either the name or modifier chips are visible |Starting an action must not restart the session-name timer. Otherwise a long session name would flash on every tool call. Modifier changes reveal or update the chips only.-When the name has faded, the pill contracts to the session orb and active chips. If no public session label exists, modifier-bearing actions still get this compact session-colored capsule. This preserves context without requiring a display name.+When the name has faded, the pill contracts to the session-colored capsule and active chips. If no public session label exists, modifier-bearing actions still get this compact session-colored capsule. This preserves context without requiring a display name.### Host-owned modifier glyphs@@ -127,7 +127,6 @@ pub struct SessionBadgeInput<'a> {pub struct SessionBadgeLayout {pub rect: Rect,- pub orb: Rect,pub label: Option<BadgeLabelLayout>,pub delivery_chip: Option<BadgeChip>,pub target_chip: Option<BadgeChip>,@@ -284,8 +283,8 @@ Keep the D-Bus action payload unchanged. Bump the bundled helper version and its3. Add:- full label with one chip- full label with two chips- - contracted orb with one chip- - contracted orb with two chips+ - contracted capsule with one chip+ - contracted capsule with two chips- no-label modifier capsule4. Regenerate the public delivery and target GIF.5. Update cursor personalization documentation and theme author instructions.diff --git a/libs/cua-driver/rust/crates/cursor-overlay/src/session_badge.rs b/libs/cua-driver/rust/crates/cursor-overlay/src/session_badge.rsindex 65f301c86e..b37a2c8576 100644--- a/libs/cua-driver/rust/crates/cursor-overlay/src/session_badge.rs+++ b/libs/cua-driver/rust/crates/cursor-overlay/src/session_badge.rs@@ -29,8 +29,6 @@ pub const BADGE_CHIP_GROUP_GAP: f32 = 7.0;const FONT_BYTES: &[u8] = include_bytes!("../assets/Inter.ttf");const FONT_SIZE: f32 = 11.5;const HORIZONTAL_PADDING: f32 = 10.0;-const ORB_SIZE: f32 = 10.0;-const ORB_GAP: f32 = 7.0;const TEXT_OPTICAL_Y_OFFSET: f32 = 1.0;#[derive(Debug, Clone, Copy, PartialEq)]@@ -50,7 +48,6 @@ pub struct BadgeLabelLayout {pub struct SessionBadgeLayout {pub rect: Rect,pub corner_radius: f32,- pub orb_center: (f32, f32),pub label: Option<BadgeLabelLayout>,pub delivery_chip: Option<BadgeChip>,pub target_chip: Option<BadgeChip>,@@ -337,7 +334,7 @@ pub fn session_badge_layout(input: SessionBadgeInput<'_>) -> Option<SessionBadge1 => BADGE_CHIP_SIZE,_ => BADGE_CHIP_SIZE * 2.0 + BADGE_CHIP_GAP,} * scale;- let fixed_width = (HORIZONTAL_PADDING * 2.0 + ORB_SIZE + ORB_GAP) * scale+ let fixed_width = HORIZONTAL_PADDING * 2.0 * scale+ chip_width+ if show_label && chip_count > 0 {BADGE_CHIP_GROUP_GAP * scale@@ -354,7 +351,7 @@ pub fn session_badge_layout(input: SessionBadgeInput<'_>) -> Option<SessionBadge};let text_width = label_data.as_ref().map_or(0.0, |(_, bounds)| bounds.width);let minimum_width = if show_label {- 72.0 * scale+ 55.0 * scale} else {fixed_width};@@ -381,11 +378,7 @@ pub fn session_badge_layout(input: SessionBadgeInput<'_>) -> Option<SessionBadgelet rect = Rect::from_xywh(x, y, badge_width, badge_height)?;let corner_radius = badge_height * 0.5;- let orb_center = (- x + HORIZONTAL_PADDING * scale + ORB_SIZE * scale * 0.5,- y + badge_height * 0.5,- );- let mut cursor_x = x + (HORIZONTAL_PADDING + ORB_SIZE + ORB_GAP) * scale;+ let mut cursor_x = x + HORIZONTAL_PADDING * scale;let label = label_data.map(|(text, bounds)| {let origin = (cursor_x - bounds.min_x,@@ -426,7 +419,6 @@ pub fn session_badge_layout(input: SessionBadgeInput<'_>) -> Option<SessionBadgeSome(SessionBadgeLayout {rect,corner_radius,- orb_center,label,delivery_chip,target_chip,@@ -486,15 +478,15 @@ pub fn paint_session_badge(vec![GradientStop::new(0.0,- session_tinted_color([94, 151, 178], session_fill, 0.52, (236.0 * opacity) as u8),+ session_tinted_color([94, 151, 178], session_fill, 0.66, (236.0 * opacity) as u8),),GradientStop::new(0.46,- session_tinted_color([43, 92, 119], session_fill, 0.40, (239.0 * opacity) as u8),+ session_tinted_color([43, 92, 119], session_fill, 0.52, (239.0 * opacity) as u8),),GradientStop::new(1.0,- session_tinted_color([13, 27, 38], session_fill, 0.18, (245.0 * opacity) as u8),+ session_tinted_color([13, 27, 38], session_fill, 0.26, (245.0 * opacity) as u8),),],SpreadMode::Pad,@@ -514,12 +506,14 @@ pub fn paint_session_badge(Transform::identity(),None,);+ // The rim carries session identity now that the orb is gone. Tinting it+ // costs no width and leaves label contrast untouched.let outline = Paint {- shader: tiny_skia::Shader::SolidColor(Color::from_rgba8(- 255,- 255,- 255,- (160.0 * opacity) as u8,+ shader: tiny_skia::Shader::SolidColor(session_tinted_color(+ [255, 255, 255],+ session_fill,+ 0.55,+ (190.0 * opacity) as u8,)),anti_alias: true,..Default::default()@@ -535,50 +529,6 @@ pub fn paint_session_badge(None,);- if let Some(orb) = PathBuilder::from_circle(- layout.orb_center.0,- layout.orb_center.1,- ORB_SIZE * scale * 0.5,- ) {- let orb_paint = Paint {- shader: tiny_skia::Shader::SolidColor(Color::from_rgba8(- session_fill[0],- session_fill[1],- session_fill[2],- (255.0 * opacity) as u8,- )),- anti_alias: true,- ..Default::default()- };- pixmap.fill_path(- &orb,- &orb_paint,- tiny_skia::FillRule::Winding,- Transform::identity(),- None,- );- let orb_outline = Paint {- shader: tiny_skia::Shader::SolidColor(Color::from_rgba8(- 255,- 255,- 255,- (226.0 * opacity) as u8,- )),- anti_alias: true,- ..Default::default()- };- pixmap.stroke_path(- &orb,- &orb_outline,- &Stroke {- width: 1.5 * scale,- ..Default::default()- },- Transform::identity(),- None,- );- }-if let Some(label) = &layout.label {let label_opacity = (alpha_scale * layout.label_alpha).clamp(0.0, 1.0);let text = text_layout(font, &label.text, FONT_SIZE * scale);@@ -699,6 +649,23 @@ mod tests {assert!(delivery.rect.x() < target.rect.x());}+ #[test]+ fn label_starts_at_the_leading_padding_with_no_reserved_marker() {+ let layout = session_badge_layout(SessionBadgeInput {+ label: Some("Research run"),+ delivery: None,+ target: None,+ cursor: (120.0, 60.0),+ backing_scale: 1.0,+ label_alpha: 1.0,+ chip_alpha: 0.0,+ clip: None,+ })+ .unwrap();+ let label = layout.label.unwrap();+ assert!((label.origin.0 - (layout.rect.x() + HORIZONTAL_PADDING)).abs() < 1.5);+ }+#[test]fn long_label_stays_inside_the_bounded_badge() {let layout = session_badge_layout(SessionBadgeInput {diff --git a/libs/cua-driver/rust/crates/platform-linux/src/wayland/shell_helper.rs b/libs/cua-driver/rust/crates/platform-linux/src/wayland/shell_helper.rsindex 19e92e6aa5..b32e4aa984 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/wayland/shell_helper.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/wayland/shell_helper.rs@@ -643,13 +643,14 @@ mod tests {assert!(EXTENSION_SOURCE.contains("createGlowSurface(this._fillColor)"));assert!(EXTENSION_SOURCE.contains("cr.translate(-GLOW_PADDING, -GLOW_PADDING);"));assert!(EXTENSION_SOURCE.contains("function drawBadgeChip"));- assert!(EXTENSION_SOURCE.contains("this._badge.add_child(this._badgeDot)"));+ assert!(EXTENSION_SOURCE.contains("function badgeStyle(fillColor)"));assert!(EXTENSION_SOURCE.contains("this._badge.add_child(this._badgeLabel)"));assert!(EXTENSION_SOURCE.contains("this._deliveryChip"));assert!(EXTENSION_SOURCE.contains("this._targetChip"));- assert!(EXTENSION_SOURCE.contains("const badgeAlpha = Math.max(labelAlpha, chipAlpha)"));+ assert!(EXTENSION_SOURCE.contains("if (labelAlpha > 0.001 || chipAlpha > 0.001)"));assert!(EXTENSION_SOURCE.contains("this._badgeLabel.hide()"));assert!(!EXTENSION_SOURCE.contains("this._badgeIdentity"));+ assert!(!EXTENSION_SOURCE.contains("this._badgeDot"));assert!(!EXTENSION_SOURCE.contains("function drawModifiers"));let metadata: serde_json::Value =serde_json::from_str(EXTENSION_METADATA).expect("valid bundled helper metadata");diff --git a/libs/cua-driver/wayland-helper/winrects@cua/extension.js b/libs/cua-driver/wayland-helper/winrects@cua/extension.jsindex dd2d6044bc..743acc2f5c 100644--- a/libs/cua-driver/wayland-helper/winrects@cua/extension.js+++ b/libs/cua-driver/wayland-helper/winrects@cua/extension.js@@ -75,15 +75,18 @@ function mixChannel(base, accent, weight) {}function badgeStyle(fillColor) {- const start = fillColor.map((channel, index) => mixChannel([94, 151, 178][index], channel, 0.52));- const end = fillColor.map((channel, index) => mixChannel([13, 27, 38][index], channel, 0.18));+ const start = fillColor.map((channel, index) => mixChannel([94, 151, 178][index], channel, 0.66));+ const end = fillColor.map((channel, index) => mixChannel([13, 27, 38][index], channel, 0.26));+ // The rim carries session identity now that the orb is gone, matching+ // paint_session_badge in the Rust renderer.+ const rim = fillColor.map((channel) => mixChannel(255, channel, 0.55));return ['spacing: 7px','padding: 6px 10px',`background-gradient-start: rgba(${start[0]}, ${start[1]}, ${start[2]}, 0.93)`,`background-gradient-end: rgba(${end[0]}, ${end[1]}, ${end[2]}, 0.96)`,'background-gradient-direction: horizontal',- 'border: 1px solid rgba(255, 255, 255, 0.63)',+ `border: 1px solid rgba(${rim[0]}, ${rim[1]}, ${rim[2]}, 0.75)`,'border-radius: 14px','box-shadow: 0 2px 9px rgba(0, 0, 0, 0.30)',].join(';');@@ -721,12 +724,6 @@ export default class WinRectsExtension extends Extension {});this._cursor.set_pivot_point(0.5, 0.5);Main.layoutManager.addTopChrome(this._cursor);- this._badgeDot = new St.Widget({- width: 10,- height: 10,- style: `background-color: ${this._fillColorCss}; border: 1px solid white; border-radius: 99px;`,- y_align: Clutter.ActorAlign.CENTER,- });this._badgeLabel = new St.Label({text: '',visible: false,@@ -759,7 +756,6 @@ export default class WinRectsExtension extends Extension {can_focus: false,style: badgeStyle(this._fillColor),});- this._badge.add_child(this._badgeDot);this._badge.add_child(this._badgeLabel);this._badge.add_child(this._deliveryChip);this._badge.add_child(this._targetChip);@@ -804,8 +800,6 @@ export default class WinRectsExtension extends Extension {this._updateModifierChips();}}- const badgeAlpha = Math.max(labelAlpha, chipAlpha);- if (this._badgeDot) this._badgeDot.opacity = Math.round(255 * badgeAlpha);if (this._badgeLabel) {this._badgeLabel.opacity = Math.round(255 * labelAlpha);if (labelAlpha <= 0.001 && this._badgeLabel.visible) {@@ -840,7 +834,6 @@ export default class WinRectsExtension extends Extension {this._badge.destroy();this._badge = null;}- this._badgeDot = null;this._badgeLabel = null;this._deliveryChip = null;this._targetChip = null;@@ -966,10 +959,6 @@ export default class WinRectsExtension extends Extension {const rgb = Number.parseInt(match[1], 16);this._fillColor = [(rgb >> 16) & 0xff, (rgb >> 8) & 0xff, rgb & 0xff];this._fillColorCss = `#${match[1].toLowerCase()}`;- if (this._badgeDot)- this._badgeDot.set_style(- `background-color: ${this._fillColorCss}; border: 1px solid white; border-radius: 99px;`- );if (this._badge) this._badge.set_style(badgeStyle(this._fillColor));this._deliveryChip?.queue_repaint();this._targetChip?.queue_repaint();