Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.
Rust agentai-agentapplecomputer-usecomputer-use-agentcontainerizationcuadesktop-automationhacktoberfestlumemacosmanusoperatorswiftvirtualizationvirtualization-frameworkwindowswindows-sandbox
revert(cua-driver): remove native Agent View presentation (#3452)
Francesco Bonacci47452be2da82
diff --git a/docs/content/docs/reference/cua-driver/mcp-tools.mdx b/docs/content/docs/reference/cua-driver/mcp-tools.mdxindex 91187102b7..3dd5405122 100644--- a/docs/content/docs/reference/cua-driver/mcp-tools.mdx+++ b/docs/content/docs/reference/cua-driver/mcp-tools.mdx@@ -661,14 +661,14 @@ Caveats:### `set_config`-Update cua-driver-rs configuration. Changes to max_image_dimension take effect immediately. The agent_view keys are persisted to ~/.cua-driver/config.json and take effect on the next daemon restart (Agent View is initialised once at startup).+Update cua-driver-rs configuration. Changes to max_image_dimension take effect immediately. The experimental_pip keys are persisted to ~/.cua-driver/config.json and take effect on the next daemon restart (the PiP backend is initialised once at startup).Note: capture_mode is a per-call param (on get_window_state / click), not a stored setting. Capture modality is selected by each action's target; the old capture_scope config key is retired.**Arguments:**-- `agent_view` (boolean, optional): Enable the multi-target Agent View. Exact native windows and Chrome tabs are grouped automatically by session; no target claiming is involved. Applies on next daemon restart.-- `agent_view_geometry` (string, optional): Agent View size + optional position in `WxH` or `WxH+X+Y` form (e.g. `640x420+24+24`). Applies on next daemon restart.+- `experimental_pip` (boolean, optional): Enable the experimental picture-in-picture preview window. Applies on next daemon restart.+- `experimental_pip_geometry` (string, optional): PiP window size + optional position in `WxH` or `WxH+X+Y` form (e.g. `320x200+24+24`). Applies on next daemon restart.- `key` (string, optional): Name of a single config field to write ({key, value} shape, matching the CLI `config set` and the Windows/Linux tools). Pair with `value`. Equivalent to passing the field directly.- `max_image_dimension` (integer, optional): Max dimension for screenshot resizing (0 = no limit).- `value` (unknown, optional): New value for `key`. JSON type depends on the key.diff --git a/libs/cua-driver/README.md b/libs/cua-driver/README.mdindex 40d859a01b..710afcdfa7 100644--- a/libs/cua-driver/README.md+++ b/libs/cua-driver/README.md@@ -131,9 +131,6 @@ Contributor documentation:- `docs/cursor-themes.md` documents the default semantic cursor and customdotLottie authoring contract.-- `docs/agent-view.md` documents the cross-platform miniature desktop (enabled- by default, with an explicit opt-out), its selected-session model, and- exact-target membership rules.- `docs/test-matrix.md` maps unit and canonical harness E2E suites.- `docs/action-support.md` is the empirical platform behavior ledger.- `docs/test-harnesses-guide.md` explains fixture and runner ownership.diff --git a/libs/cua-driver/docs/agent-view.md b/libs/cua-driver/docs/agent-view.mddeleted file mode 100644index e5f1144d10..0000000000--- a/libs/cua-driver/docs/agent-view.md+++ /dev/null@@ -1,79 +0,0 @@-# Agent View--Agent View is an optional, always-on-top miniature desktop that shows the exact-application windows and browser tabs traversed by one Cua Driver agent session.-It is disabled by default. Enable it for one daemon invocation with:--```console-cua-driver --agent-view-```--The durable config equivalent is `{"agent_view": true}` in-`~/.cua-driver/config.json`.--`--no-agent-view` remains available to override a persisted enable for one-daemon invocation.--Use `--agent-view-geometry WxH[+X+Y]` to override its initial size and optional-top-left position. The default is `640x420` near the top-right of the main-display. The view is resizable on macOS, Windows, and Linux X11/XWayland.-Pure Wayland uses a native top-right layer surface when the compositor exposes-`zwlr_layer_shell_v1`; its position and size are compositor-managed. GNOME-sessions without layer-shell use the supported XWayland presentation path. A-pure layer-shell view remains click-through and follows the most-recent session;-local tab selection, dragging, and manual resizing require the X11/XWayland-window path because standard Wayland does not expose those global window-management operations to clients.--## Session model--- One Agent View window belongs to one daemon and displays one selected private- runtime session at a time. Multiple sessions appear as local tabs in that- same window; the tab strip is hidden when only one session has cards.-- Until a person selects a session locally, Agent View follows the session with- the most recent exact target activity.-- When multiple sessions have cards, the native Agent View window exposes a- local session switcher. Selecting a session pins the view there until that- session ends or is removed. There is no agent-facing selection tool.-- Public session names are display labels only. The private runtime session ID- remains the isolation identity, so equal labels never combine sessions.-- Ending, disconnecting, revoking, or idle-expiring a session removes its cards.- Reusing a revived session starts with an empty view.--Agent View does not claim targets and does not add claim/release semantics to-the CLI, MCP tools, SDKs, or target schemas. Switching the presentation never-focuses, moves, resizes, closes, or otherwise controls an underlying target.-During physical Cua Driver actions, the always-on-top surface temporarily-becomes input-transparent so an overlapping Agent View cannot intercept input-intended for the represented application. Local switching and resizing resume-as soon as the action finishes.--## What becomes a card--A target is added or refreshed only after Cua Driver receives exact target-evidence through one of these paths:--- `get_window_state` with both `pid` and `window_id`;-- an exact native action carrying both `pid` and `window_id`;-- `get_browser_state` with both `target_id` and `tab_id`; or-- an exact browser-tab action carrying both `target_id` and `tab_id`.--Broad discovery such as `list_windows`, `list_browser_tabs`, whole-desktop-snapshots, ambient foreground activity, and actions performed outside Cua-Driver do not populate Agent View.--Browser cards use the page's stable internal CDP identity, so navigation and a-repeated browser-window bind refresh the same card. Multiple tabs in one Chrome-or Edge window remain distinct, while their native container window is hidden-to avoid duplicating the browser.--When a target is proven absent, only its exact card is removed. When a session-ends, all cards belonging to that private session are removed. The underlying-application or tab is never changed by presentation cleanup.--## Public contract--Agent View uses the existing daemon flag, geometry option, runtime sessions,-and exact target arguments. This session-oriented behavior does not introduce-new MCP tools, CLI flags, SDK types, capability IDs, claim tokens, or public-target/session fields.diff --git a/libs/cua-driver/rust/crates/cua-driver-core/src/browser/engine.rs b/libs/cua-driver/rust/crates/cua-driver-core/src/browser/engine.rsindex a38ada51f1..5f4d1a0760 100644--- a/libs/cua-driver/rust/crates/cua-driver-core/src/browser/engine.rs+++ b/libs/cua-driver/rust/crates/cua-driver-core/src/browser/engine.rs@@ -410,15 +410,6 @@ pub(crate) struct BrowserTabScreenshot {pub pixel_to_css_scale_y: f64,}-pub(crate) struct BrowserTabPreview {- pub screenshot: BrowserTabScreenshot,- pub title: String,- pub url: String,- pub cdp_target_id: String,- pub pid: i64,- pub window_id: u64,-}-#[derive(Debug, Clone, Copy)]struct BrowserScreenshotViewport {page_x: f64,@@ -2023,36 +2014,6 @@ impl BrowserEngine {})}- /// Capture one exact tab for a trusted local presentation surface while- /// retaining the tab metadata already proved by the session-scoped store.- /// Like `capture_tab_screenshot`, this never activates the tab or brings- /// its native window to the foreground.- pub(crate) async fn capture_tab_preview(- &self,- session: &str,- target_id: &str,- tab_id: &str,- ) -> Result<BrowserTabPreview, BrowserRefusal> {- let record = self.store.get_target(session, target_id)?;- let tab = record.tabs.get(tab_id).cloned().ok_or_else(|| {- refuse(- BrowserRefusalCode::BrowserTabNotFound,- format!("tab {tab_id} is not known for target {target_id}"),- )- })?;- let screenshot = self- .capture_tab_screenshot(session, target_id, tab_id)- .await?;- Ok(BrowserTabPreview {- screenshot,- title: tab.title,- url: tab.url,- cdp_target_id: tab.cdp_target_id,- pid: record.pid,- window_id: record.window_id,- })- }-/// Snapshot one tab's composed DOM (main frame + shadow DOM +/// same-process iframes + capability-tested OOPIFs) and mint/// `p<snap>:<index>` refs for interactive elements. Read-onlydiff --git a/libs/cua-driver/rust/crates/cua-driver-core/src/browser/tools.rs b/libs/cua-driver/rust/crates/cua-driver-core/src/browser/tools.rsindex 194018f72b..f16fb1793c 100644--- a/libs/cua-driver/rust/crates/cua-driver-core/src/browser/tools.rs+++ b/libs/cua-driver/rust/crates/cua-driver-core/src/browser/tools.rs@@ -28,7 +28,6 @@ use super::types::BindingQuality;/// crates call this from their `register_all` after constructing the/// engine with their adapter.pub fn register_browser_tools(engine: &Arc<BrowserEngine>, registry: &mut ToolRegistry) {- registry.set_browser_engine(engine.clone());registry.register(Box::new(GetBrowserStateTool::new(engine.clone())));registry.register(Box::new(BrowserPrepareTool::new(engine.clone())));registry.register(Box::new(BrowserNavigateTool::new(engine.clone())));diff --git a/libs/cua-driver/rust/crates/cua-driver-core/src/pip_hook.rs b/libs/cua-driver/rust/crates/cua-driver-core/src/pip_hook.rsindex a0f15c9423..d2c426169f 100644--- a/libs/cua-driver/rust/crates/cua-driver-core/src/pip_hook.rs+++ b/libs/cua-driver/rust/crates/cua-driver-core/src/pip_hook.rs@@ -1,5 +1,5 @@-//! Agent View event hook, registered once by `main.rs` when `--agent-view`-//! is enabled.+//! PiP frame-push hook — registered once by `main.rs` when the+//! `--experimental-pip` flag is on argv.//!//! The trait + factory live in the `pip-preview` crate so the platform//! backends can implement them without depending on `cua-driver-core`.@@ -19,114 +19,31 @@ use std::sync::OnceLock;/// to `pip_preview::PipFrame` — duplicated here to keep `cua-driver-core`/// from importing `pip-preview` (the dependency would be circular once/// platform backends pull both crates in).-#[derive(Clone, Copy)]-pub enum PipHookTargetKind {- NativeWindow,- BrowserTab,-}--pub struct PipHookTarget {- pub workspace_id: String,- pub workspace_label: String,- pub target_id: String,- pub identity_key: String,- pub target_kind: PipHookTargetKind,- pub target_label: String,- pub native_container: Option<PipHookNativeContainer>,-}--#[derive(Clone, Copy)]-pub struct PipHookNativeContainer {- pub pid: i64,- pub window_id: u64,-}-pub struct PipHookFrame {- pub target: PipHookTarget,pub png_bytes: Vec<u8>,pub action_label: String,pub timestamp_ms: u64,- pub cursor_position: Option<(f64, f64)>,-}--pub enum PipHookEvent {- Upsert(PipHookFrame),- SetInputPassthrough {- passthrough: bool,- },- RemoveTarget {- workspace_id: String,- identity_key: String,- },- RemoveWorkspace {- workspace_id: String,- },-}--/// Restores Agent View interactivity even when an action exits early or-/// unwinds. Physical desktop actions are serialized by the dispatcher, so one-/// process-global passthrough scope is sufficient.-pub struct PipInputPassthroughGuard;--impl Drop for PipInputPassthroughGuard {- fn drop(&mut self) {- if let Some(f) = PIP_EVENT_FN.get() {- if let Err(error) = f(PipHookEvent::SetInputPassthrough { passthrough: false }) {- tracing::error!(%error, "failed to restore Agent View input handling");- }- }- }}-type PipEventFnBox = Box<dyn Fn(PipHookEvent) -> Result<(), String> + Send + Sync>;-static PIP_EVENT_FN: OnceLock<PipEventFnBox> = OnceLock::new();+type PipPushFnBox = Box<dyn Fn(PipHookFrame) + Send + Sync>;+static PIP_PUSH_FN: OnceLock<PipPushFnBox> = OnceLock::new();/// Register the platform-side push callback. `main.rs` calls this/// once after starting the PiP backend.-pub fn set_pip_event_fn(f: impl Fn(PipHookEvent) -> Result<(), String> + Send + Sync + 'static) {- let _ = PIP_EVENT_FN.set(Box::new(f));+pub fn set_pip_push_fn(f: impl Fn(PipHookFrame) + Send + Sync + 'static) {+ let _ = PIP_PUSH_FN.set(Box::new(f));}/// True when a PiP backend is wired up. Tool dispatcher uses this to/// skip the screenshot-bytes path when nothing would consume the-/// frame (avoiding wasted capture work when Agent View is disabled).+/// frame (avoiding wasted capture work in the common --pip-off case).pub fn pip_enabled() -> bool {- PIP_EVENT_FN.get().is_some()+ PIP_PUSH_FN.get().is_some()}-/// Temporarily make Agent View input-transparent. The registered backend-/// applies the native state synchronously before this returns.-pub fn begin_pip_input_passthrough() -> Result<Option<PipInputPassthroughGuard>, String> {- let Some(f) = PIP_EVENT_FN.get() else {- return Ok(None);- };- f(PipHookEvent::SetInputPassthrough { passthrough: true })?;- Ok(Some(PipInputPassthroughGuard))-}--/// Upsert one exact-target frame. No-op when no backend is registered.+/// Push a frame to the PiP window. No-op when no backend is registered.pub fn push_pip_frame(frame: PipHookFrame) {- if let Some(f) = PIP_EVENT_FN.get() {- let _ = f(PipHookEvent::Upsert(frame));- }-}--/// Remove one exact target card. Presentation only; never closes the target.-pub fn remove_pip_target(workspace_id: impl Into<String>, identity_key: impl Into<String>) {- if let Some(f) = PIP_EVENT_FN.get() {- let _ = f(PipHookEvent::RemoveTarget {- workspace_id: workspace_id.into(),- identity_key: identity_key.into(),- });- }-}--/// Remove presentation state for an ended workspace. This never closes the-/// workspace's native windows or browser tabs.-pub fn remove_pip_workspace(workspace_id: impl Into<String>) {- if let Some(f) = PIP_EVENT_FN.get() {- let _ = f(PipHookEvent::RemoveWorkspace {- workspace_id: workspace_id.into(),- });+ if let Some(f) = PIP_PUSH_FN.get() {+ f(frame);}}diff --git a/libs/cua-driver/rust/crates/cua-driver-core/src/recording.rs b/libs/cua-driver/rust/crates/cua-driver-core/src/recording.rsindex 8a0c3a7cdf..362f76118a 100644--- a/libs/cua-driver/rust/crates/cua-driver-core/src/recording.rs+++ b/libs/cua-driver/rust/crates/cua-driver-core/src/recording.rs@@ -646,7 +646,7 @@ fn capture_turn(window_id: Option<u64>, pid: Option<i64>) -> TurnCapture {}}-pub(crate) fn resolve_click_point(+fn resolve_click_point(tool_name: &str,args: &Value,window_id: Option<u64>,diff --git a/libs/cua-driver/rust/crates/cua-driver-core/src/tool.rs b/libs/cua-driver/rust/crates/cua-driver-core/src/tool.rsindex fee4b0cfa1..c9d82805f0 100644--- a/libs/cua-driver/rust/crates/cua-driver-core/src/tool.rs+++ b/libs/cua-driver/rust/crates/cua-driver-core/src/tool.rs@@ -6,7 +6,6 @@ use std::sync::{Arc, Mutex, OnceLock};use std::time::Duration;use async_trait::async_trait;-use base64::{engine::general_purpose::STANDARD as BASE64, Engine as _};use serde_json::Value;thread_local! {@@ -641,7 +640,6 @@ pub struct ToolRegistry {approval_broker: Arc<crate::consent::ApprovalBroker>,protected_resource_grants: Arc<crate::consent::ProtectedResourceGrants>,protected_resource_ownership: Arc<crate::consent::ProtectedResourceOwnershipStore>,- browser_engine: Option<Arc<crate::browser::BrowserEngine>>,}impl ToolRegistry {@@ -705,7 +703,6 @@ impl ToolRegistry {approval_broker,protected_resource_grants,protected_resource_ownership,- browser_engine: None,}}@@ -793,10 +790,6 @@ impl ToolRegistry {self.tools.insert(name, tool);}- pub(crate) fn set_browser_engine(&mut self, engine: Arc<crate::browser::BrowserEngine>) {- self.browser_engine = Some(engine);- }-pub fn retain_session_end_hook(&mut self,registration: crate::session::SessionEndHookRegistration,@@ -1517,18 +1510,6 @@ impl ToolRegistry {} else {None};- let pip_input_passthrough = if _desktop_action.is_some() {- match pip_hook::begin_pip_input_passthrough() {- Ok(guard) => guard,- Err(error) => {- return ToolResult::error(format!(- "Agent View could not yield pointer input before {resolved_name}: {error}"- ));- }- }- } else {- None- };let pending_turn = should_record.then(|| {if private_consent_turn {@@ -1542,7 +1523,6 @@ impl ToolRegistry {.flatten();let mut result = tool.invoke(args.clone()).await;- drop(pip_input_passthrough);drop(lifecycle_dispatch);// The platform worker has exited, so another text operation for this// pid may now start even while result projection and evidence capture@@ -1666,122 +1646,26 @@ impl ToolRegistry {);}- // Agent View keeps one latest frame per exact native window or browser- // tab. Exact read-side snapshots seed cards as well as action tools,- // so a user can inspect several targets before any mutation occurs.- if pip_hook::pip_enabled()- && !private_consent_turn- && result.is_error == Some(true)- && pip_result_proves_target_absent(&result)- {- if let (Some(workspace_id), Some(target)) =- (runtime_session.as_deref(), pip_capture_target(&args))- {- pip_hook::remove_pip_target(workspace_id, pip_target_public_key(&target));- }- }- if pip_hook::pip_enabled()- && !private_consent_turn- && (should_record || is_exact_visual_snapshot(name, &args))- {- if let Some(frame) = self- .capture_pip_frame(name, &args, &public_args, runtime_session.as_deref())- .await- {- pip_hook::push_pip_frame(frame);+ // Experimental PiP push — only when --experimental-pip is on argv+ // (otherwise `pip_enabled()` is false and we skip the screenshot+ // entirely to avoid wasted capture work). We push for the same set+ // of action tools the recording pipeline cares about (non-read-only,+ // not the recording-control meta-tools) so the live view matches+ // what the recorder would have captured for the turn.+ if pip_hook::pip_enabled() && should_record && !private_consent_turn {+ let window_id = args.opt_u64("window_id");+ let pid = args.opt_i64("pid");+ if let Some(png_bytes) = screenshot_for(window_id, pid) {+ let label = synthesize_action_label(name, &public_args);+ pip_hook::push_pip_frame(pip_hook::PipHookFrame {+ png_bytes,+ action_label: label,+ timestamp_ms: now_ms(),+ });}}- result- }-- async fn capture_pip_frame(- &self,- tool_name: &str,- args: &Value,- public_args: &Value,- runtime_session: Option<&str>,- ) -> Option<pip_hook::PipHookFrame> {- let workspace_id = runtime_session?.to_owned();- let workspace_label = pip_workspace_label(public_args, runtime_session?);- let action_label = synthesize_action_label(tool_name, public_args);- let timestamp_ms = now_ms();-- if let (PipCaptureTarget::Browser { target_id, tab_id }, Some(engine)) =- (pip_capture_target(args)?, self.browser_engine.as_ref())- {- let browser_session = args- .get("session")- .and_then(Value::as_str)- .or(runtime_session)?;- let preview = match engine- .capture_tab_preview(browser_session, &target_id, &tab_id)- .await- {- Ok(preview) => preview,- Err(error)- if matches!(- error.code,- crate::browser::refusal::BrowserRefusalCode::BrowserTabNotFound- | crate::browser::refusal::BrowserRefusalCode::BrowserBindingStale- | crate::browser::refusal::BrowserRefusalCode::BrowserWrongTargetRefused- ) =>- {- pip_hook::remove_pip_target(- &workspace_id,- format!("browser:{target_id}:{tab_id}"),- );- return None;- }- Err(_) => return None,- };- let png_bytes = BASE64.decode(preview.screenshot.data_base64).ok()?;- let cursor_position = pip_cursor_position(tool_name, args, &png_bytes, None, None);- let target_label = if preview.title.trim().is_empty() {- preview.url- } else {- preview.title- };- return Some(pip_hook::PipHookFrame {- target: pip_hook::PipHookTarget {- workspace_id,- workspace_label,- target_id: format!("browser:{target_id}:{tab_id}"),- identity_key: format!("browser-cdp:{}", preview.cdp_target_id),- target_kind: pip_hook::PipHookTargetKind::BrowserTab,- target_label,- native_container: Some(pip_hook::PipHookNativeContainer {- pid: preview.pid,- window_id: preview.window_id,- }),- },- png_bytes,- action_label,- timestamp_ms,- cursor_position,- });- }- let PipCaptureTarget::NativeWindow { pid, window_id } = pip_capture_target(args)? else {- return None;- };- let png_bytes = screenshot_for(Some(window_id), Some(pid))?;- let cursor_position =- pip_cursor_position(tool_name, args, &png_bytes, Some(window_id), Some(pid));- Some(pip_hook::PipHookFrame {- target: pip_hook::PipHookTarget {- workspace_id,- workspace_label,- target_id: format!("window:{pid}:{window_id}"),- identity_key: format!("window:{pid}:{window_id}"),- target_kind: pip_hook::PipHookTargetKind::NativeWindow,- target_label: format!("Window {window_id}"),- native_container: Some(pip_hook::PipHookNativeContainer { pid, window_id }),- },- png_bytes,- action_label,- timestamp_ms,- cursor_position,- })+ result}async fn authorize_private_observation(@@ -3181,7 +3065,7 @@ mod runtime_isolation_tests {"key": {"type": "string"},"value": {},"max_image_dimension": {"type": "integer"},- "agent_view": {"type": "boolean"}+ "experimental_pip": {"type": "boolean"}},"additionalProperties": false})@@ -4847,144 +4731,7 @@ impl Default for ToolRegistry {}}-#[derive(Debug, Clone, PartialEq, Eq)]-enum PipCaptureTarget {- NativeWindow { pid: i64, window_id: u64 },- Browser { target_id: String, tab_id: String },-}--fn pip_capture_target(args: &Value) -> Option<PipCaptureTarget> {- if let (Some(target_id), Some(tab_id)) = (- args.get("target_id").and_then(Value::as_str),- args.get("tab_id").and_then(Value::as_str),- ) {- return Some(PipCaptureTarget::Browser {- target_id: target_id.to_owned(),- tab_id: tab_id.to_owned(),- });- }- Some(PipCaptureTarget::NativeWindow {- pid: args.opt_i64("pid")?,- window_id: args.opt_u64("window_id")?,- })-}--fn pip_target_public_key(target: &PipCaptureTarget) -> String {- match target {- PipCaptureTarget::NativeWindow { pid, window_id } => format!("window:{pid}:{window_id}"),- PipCaptureTarget::Browser { target_id, tab_id } => {- format!("browser:{target_id}:{tab_id}")- }- }-}--fn pip_cursor_position(- tool_name: &str,- args: &Value,- png_bytes: &[u8],- window_id: Option<u64>,- pid: Option<i64>,-) -> Option<(f64, f64)> {- use crate::tool_args::ArgsExt;-- let point = if tool_name == "drag" || tool_name == "browser_drag" {- Some((args.opt_f64("to_x")?, args.opt_f64("to_y")?))- } else if matches!(- tool_name,- "click"- | "double_click"- | "right_click"- | "move_cursor"- | "browser_click"- | "browser_double_click"- | "browser_right_click"- ) {- match (args.opt_f64("x"), args.opt_f64("y")) {- (Some(x), Some(y)) => Some((x, y)),- _ => crate::recording::resolve_click_point(- tool_name,- args,- window_id,- pid,- args.opt_u64("element_index"),- ),- }- } else {- None- }?;- let (width, height) = pip_png_dimensions(png_bytes)?;- Some((- (point.0 / width).clamp(0.0, 1.0),- (point.1 / height).clamp(0.0, 1.0),- ))-}--fn pip_png_dimensions(bytes: &[u8]) -> Option<(f64, f64)> {- const PNG_SIGNATURE: &[u8; 8] = b"\x89PNG\r\n\x1a\n";- if bytes.len() < 24 || &bytes[..8] != PNG_SIGNATURE || &bytes[12..16] != b"IHDR" {- return None;- }- let width = u32::from_be_bytes(bytes[16..20].try_into().ok()?) as f64;- let height = u32::from_be_bytes(bytes[20..24].try_into().ok()?) as f64;- (width > 0.0 && height > 0.0).then_some((width, height))-}--fn pip_result_proves_target_absent(result: &ToolResult) -> bool {- let code = result- .structured_content- .as_ref()- .and_then(|payload| payload.get("code"))- .and_then(Value::as_str);- matches!(- code,- Some(- "window_target_not_found"- | "window_not_found"- | "browser_tab_not_found"- | "browser_binding_stale"- | "browser_wrong_target_refused"- )- )-}--fn is_exact_visual_snapshot(tool_name: &str, args: &Value) -> bool {- matches!(- (tool_name, pip_capture_target(args)),- (- "get_window_state",- Some(PipCaptureTarget::NativeWindow { .. })- ) | ("get_browser_state", Some(PipCaptureTarget::Browser { .. }))- )-}--fn pip_workspace_label(public_args: &Value, runtime_session: &str) -> String {- if let Some(label) = public_args- .get("session")- .and_then(Value::as_str)- .filter(|label| !label.is_empty() && *label != "default")- {- return label.to_owned();- }- let suffix = runtime_session- .rsplit(':')- .next()- .unwrap_or(runtime_session)- .chars()- .filter(|ch| ch.is_ascii_alphanumeric())- .rev()- .take(6)- .collect::<String>()- .chars()- .rev()- .collect::<String>();- if suffix.is_empty() {- "Agent".to_owned()- } else {- format!("Agent {suffix}")- }-}--/// Build a short, human-friendly label for the Agent View from the+/// Build a short, human-friendly label for the PiP overlay from the/// tool name + raw args. Kept under ~60 chars so the macOS NSTextField/// has room without truncation at default geometry.fn synthesize_action_label(tool_name: &str, args: &Value) -> String {@@ -5031,133 +4778,6 @@ fn synthesize_action_label(tool_name: &str, args: &Value) -> String {}}-#[cfg(test)]-mod pip_routing_tests {- use super::*;-- fn png_header(width: u32, height: u32) -> Vec<u8> {- let mut bytes = b"\x89PNG\r\n\x1a\n\0\0\0\rIHDR".to_vec();- bytes.extend_from_slice(&width.to_be_bytes());- bytes.extend_from_slice(&height.to_be_bytes());- bytes- }-- #[test]- fn pointer_coordinates_are_normalized_to_the_captured_target() {- assert_eq!(- pip_cursor_position(- "click",- &serde_json::json!({"x": 50, "y": 25}),- &png_header(200, 100),- None,- None,- ),- Some((0.25, 0.25))- );- assert_eq!(- pip_cursor_position(- "drag",- &serde_json::json!({"to_x": 180, "to_y": 90}),- &png_header(200, 100),- None,- None,- ),- Some((0.9, 0.9))- );- assert_eq!(- pip_cursor_position(- "type_text",- &serde_json::json!({"x": 50, "y": 25}),- &png_header(200, 100),- None,- None,- ),- None- );- }-- #[test]- fn browser_tab_identity_wins_over_its_native_container_window() {- let target = pip_capture_target(&serde_json::json!({- "pid": 42,- "window_id": 7,- "target_id": "bt-one",- "tab_id": "tab-two",- }));- assert_eq!(- target,- Some(PipCaptureTarget::Browser {- target_id: "bt-one".to_owned(),- tab_id: "tab-two".to_owned(),- })- );- }-- #[test]- fn native_window_identity_requires_the_exact_pair() {- assert_eq!(- pip_capture_target(&serde_json::json!({"pid": 42, "window_id": 7})),- Some(PipCaptureTarget::NativeWindow {- pid: 42,- window_id: 7,- })- );- assert!(pip_capture_target(&serde_json::json!({"pid": 42})).is_none());- assert!(pip_capture_target(&serde_json::json!({"window_id": 7})).is_none());- }-- #[test]- fn only_exact_state_reads_seed_agent_view_cards() {- assert!(is_exact_visual_snapshot(- "get_window_state",- &serde_json::json!({"pid": 42, "window_id": 7})- ));- assert!(is_exact_visual_snapshot(- "get_browser_state",- &serde_json::json!({"target_id": "bt-one", "tab_id": "tab-two"})- ));- assert!(!is_exact_visual_snapshot(- "get_browser_state",- &serde_json::json!({"pid": 42, "window_id": 7})- ));- }-- #[test]- fn public_session_label_is_preferred_for_presentation() {- assert_eq!(- pip_workspace_label(- &serde_json::json!({"session": "research"}),- "__cua_runtime_x:private"- ),- "research"- );- assert_eq!(- pip_workspace_label(&serde_json::json!({}), "__cua_runtime_x:agent-123456"),- "Agent 123456"- );- }-- #[test]- fn proven_absent_results_remove_only_the_exact_public_target() {- let absent = ToolResult::error("gone").with_structured(serde_json::json!({- "code": "window_target_not_found"- }));- assert!(pip_result_proves_target_absent(&absent));- assert_eq!(- pip_target_public_key(&PipCaptureTarget::NativeWindow {- pid: 42,- window_id: 7,- }),- "window:42:7"- );-- let permission = ToolResult::error("denied").with_structured(serde_json::json!({- "code": "permission_denied"- }));- assert!(!pip_result_proves_target_absent(&permission));- }-}-#[cfg(test)]mod capability_tests {//! Unit tests for the per-tool `capabilities` array and thediff --git a/libs/cua-driver/rust/crates/cua-driver/src/cli.rs b/libs/cua-driver/rust/crates/cua-driver/src/cli.rsindex baec2e6c49..f3e021b74e 100644--- a/libs/cua-driver/rust/crates/cua-driver/src/cli.rs+++ b/libs/cua-driver/rust/crates/cua-driver/src/cli.rs@@ -232,10 +232,10 @@ const VALUE_FLAGS: &[&str] = &["--session","--profile-mode","--profile-name",- // Agent View value flag for the optional geometry override- // (`--agent-view` itself is a bare flag and doesn't+ // Experimental PiP preview — value flag for the optional geometry+ // override (--experimental-pip itself is a bare flag and doesn't// need to be listed here).- "--agent-view-geometry",+ "--experimental-pip-geometry",];/// Authorization selectors are trusted-daemon startup inputs. Direct MCP@@ -634,7 +634,7 @@ pub fn parse_command() -> Command {println!("doctor options:");println!(" --json Emit the probe report as JSON for scripting.");println!();- println!("experimental options:");+ println!("experimental options (default: off):");println!(" --experimental-history Admit encrypted local Computer History for this daemon.");@@ -643,30 +643,17 @@ pub fn parse_command() -> Command {);println!(" cua-driver history enable Opt in and initialize encrypted local history.");println!(" cua-driver history status|pause|resume|flush|list|show|disable|delete");- println!(" --agent-view Enable the optional Agent View.");- println!(" Exact native windows and");+ println!(" --experimental-pip Show a small always-on-top window with the latest");println!(- " Chrome tabs traversed through Cua Driver become cards."+ " post-action screenshot + a 1-line label. macOS only");println!(- " Recent activity is followed until you locally select"+ " today; Win/Linux print a not-yet-implemented notice.");println!(- " another session in Agent View; sessions never mix."+ " --experimental-pip-geometry WxH[+X+Y] Override window size (and optional top-left");- println!(- " Closed targets and ended sessions are removed. This is"- );- println!(- " presentation only; no claim/release API is involved."- );- println!(- " Native presentation is available on macOS, Windows,"- );- println!(" and Linux (X11/XWayland).");- println!(" --no-agent-view Disable Agent View, including a persisted enable.");- println!(" --agent-view-geometry WxH[+X+Y] Override window size (and optional top-left");- println!(" origin). Defaults to 640x420 in the top-right");+ println!(" origin). Defaults to 480x360 in the top-right");println!(" corner of the main display.");std::process::exit(0);}diff --git a/libs/cua-driver/rust/crates/cua-driver/src/main.rs b/libs/cua-driver/rust/crates/cua-driver/src/main.rsindex e95c1a3ba8..7ce3e7c072 100644--- a/libs/cua-driver/rust/crates/cua-driver/src/main.rs+++ b/libs/cua-driver/rust/crates/cua-driver/src/main.rs@@ -216,31 +216,23 @@ fn run_cursor_theme_command(args: &[String]) -> ! {std::process::exit(status.code().unwrap_or(1));}-/// Wire up the multi-target Agent View.+/// Wire up the experimental picture-in-picture preview window.////// Called from every long-running entry point (Serve and Mcp on all/// platforms; the `Call` arm intentionally skips PiP since the/// per-call binaries don't keep an AppKit/event loop alive long/// enough to be useful).///-fn maybe_init_pip() -> bool {+/// No-op when `--experimental-pip` is not on argv. On Windows / Linux+/// the factory returns "not yet implemented" — we log and continue+/// without a window so the rest of the daemon keeps working.+fn maybe_init_pip() {let cfg = match pip_preview::default_config_path() {Some(p) => pip_preview::PipConfig::from_args_and_file(&p),None => pip_preview::PipConfig::from_args(),};if !cfg.enabled {- return false;- }-- // AppKit's shared application registration aborts the process when the- // daemon has no Window Server session (for example, an SSH-launched or- // headless service). Do not select the Agent View run loop unless the- // native surface can safely exist; the socket-serving thread must remain- // the main lifetime owner in that case.- #[cfg(target_os = "macos")]- if !platform_macos::session::has_graphic_access() {- eprintln!("⚗️ Agent View requested but unavailable: no Window Server graphic session");- return false;+ return;}// Register the platform factory. The set is idempotent so multiple@@ -264,68 +256,24 @@ fn maybe_init_pip() -> bool {StdMutex<Option<Box<dyn pip_preview::PipBackend>>>,> = std::sync::OnceLock::new();let _ = BACKEND.set(StdMutex::new(Some(backend)));- cua_driver_core::pip_hook::set_pip_event_fn(|event| {+ cua_driver_core::pip_hook::set_pip_push_fn(|frame| {if let Some(slot) = BACKEND.get() {if let Some(b) = slot.lock().unwrap().as_ref() {- match event {- cua_driver_core::pip_hook::PipHookEvent::Upsert(frame) => {- let target_kind = match frame.target.target_kind {- cua_driver_core::pip_hook::PipHookTargetKind::NativeWindow => {- pip_preview::PipTargetKind::NativeWindow- }- cua_driver_core::pip_hook::PipHookTargetKind::BrowserTab => {- pip_preview::PipTargetKind::BrowserTab- }- };- b.push_frame(pip_preview::PipFrame {- target: pip_preview::PipTarget {- workspace_id: frame.target.workspace_id,- workspace_label: frame.target.workspace_label,- target_id: frame.target.target_id,- identity_key: frame.target.identity_key,- target_kind,- target_label: frame.target.target_label,- native_container: frame.target.native_container.map(- |container| pip_preview::PipNativeContainer {- pid: container.pid,- window_id: container.window_id,- },- ),- },- png_bytes: frame.png_bytes,- action_label: frame.action_label,- timestamp_ms: frame.timestamp_ms,- cursor_position: frame.cursor_position,- });- }- cua_driver_core::pip_hook::PipHookEvent::SetInputPassthrough {- passthrough,- } => {- return b- .set_input_passthrough(passthrough)- .map_err(|error| error.to_string());- }- cua_driver_core::pip_hook::PipHookEvent::RemoveWorkspace {- workspace_id,- } => b.remove_workspace(&workspace_id),- cua_driver_core::pip_hook::PipHookEvent::RemoveTarget {- workspace_id,- identity_key,- } => b.remove_target(&workspace_id, &identity_key),- }+ b.push_frame(pip_preview::PipFrame {+ png_bytes: frame.png_bytes,+ action_label: frame.action_label,+ timestamp_ms: frame.timestamp_ms,+ });}}- Ok(())});- cua_driver_core::session::register_session_end_hook(|workspace_id| {- cua_driver_core::pip_hook::remove_pip_workspace(workspace_id);- });- eprintln!("⚗️ Agent View enabled (native macOS, Windows, and Linux presentation)");- true+ eprintln!(+ "⚗️ PiP preview enabled (experimental — macOS only today; \+ see https://github.com/trycua/cua/issues for follow-up)"+ );}Err(e) => {- eprintln!("⚗️ Agent View requested but unavailable: {e}");- false+ eprintln!("⚗️ PiP preview requested but unavailable: {e}");}}}@@ -419,28 +367,6 @@ fn history_admission_requested(explicit: bool, persisted: bool) -> bool {explicit || persisted}-#[cfg(target_os = "macos")]-#[derive(Debug, PartialEq, Eq)]-enum MacosMainLoopHost {- CursorOverlay,- AgentView,- ServeThread,-}--#[cfg(target_os = "macos")]-fn macos_main_loop_host(cursor_enabled: bool, agent_view_enabled: bool) -> MacosMainLoopHost {- if cursor_enabled {- // The cursor host also owns NSApplication's main run loop. Agent View- // posts its window updates to that same queue, so both surfaces can- // coexist without leaving cursor animations waiting forever.- MacosMainLoopHost::CursorOverlay- } else if agent_view_enabled {- MacosMainLoopHost::AgentView- } else {- MacosMainLoopHost::ServeThread- }-}-#[cfg(test)]mod history_admission_tests {use super::history_admission_requested;@@ -458,31 +384,6 @@ mod history_admission_tests {}}-#[cfg(all(test, target_os = "macos"))]-mod macos_main_loop_host_tests {- use super::{macos_main_loop_host, MacosMainLoopHost};-- #[test]- fn cursor_overlay_hosts_the_shared_appkit_loop_when_agent_view_is_enabled() {- assert_eq!(- macos_main_loop_host(true, true),- MacosMainLoopHost::CursorOverlay- );- }-- #[test]- fn agent_view_hosts_appkit_only_when_the_cursor_is_disabled() {- assert_eq!(- macos_main_loop_host(false, true),- MacosMainLoopHost::AgentView- );- assert_eq!(- macos_main_loop_host(false, false),- MacosMainLoopHost::ServeThread- );- }-}-fn mcp_uses_direct_runtime(socket: Option<&str>, direct: bool) -> anyhow::Result<bool> {mcp_uses_direct_runtime_for(cua_driver_core::embedded_mode(),@@ -697,7 +598,11 @@ fn main() {// before any blocking work so the banner can land on stderr// early in the serve lifecycle.version_check::maybe_announce_update();- let pip_available = maybe_init_pip();+ let pip_cfg = match pip_preview::default_config_path() {+ Some(p) => pip_preview::PipConfig::from_args_and_file(&p),+ None => pip_preview::PipConfig::from_args(),+ };+ maybe_init_pip();// Agent-cursor overlay. The DAEMON is the process that actually// performs clicks / AX presses, so the overlay NSWindow + render@@ -804,24 +709,24 @@ fn main() {// Keep the main thread alive for the daemon.//- // Cursor and Agent View share one AppKit application/main queue.- // When both are enabled, the cursor host must drain its command- // receiver as well as NSApplication events; the Agent View-only- // loop would leave cursor-bearing actions waiting indefinitely.- match macos_main_loop_host(cursor_cfg.enabled, pip_available) {- MacosMainLoopHost::CursorOverlay => {- // `run_on_main_thread` self-guards on graphic-session- // access. If it returns, keep the daemon alive by joining- // the serve thread.- platform_macos::cursor::overlay::run_on_main_thread();- let _ = serve_handle.join();- }- MacosMainLoopHost::AgentView => {- platform_macos::pip::run_appkit_main_loop();- }- MacosMainLoopHost::ServeThread => {- let _ = serve_handle.join();- }+ // PiP needs the AppKit main run loop to process the+ // dispatch_async_f calls that push frames into NSImageView;+ // park main in NSApplication.run() when --experimental-pip is+ // on. Otherwise just join the serve thread so the process+ // stays up as long as the daemon does.+ if pip_cfg.enabled {+ platform_macos::pip::run_appkit_main_loop();+ } else if cursor_cfg.enabled {+ // Render the agent-cursor overlay: park the main thread in the+ // AppKit run loop so the overlay NSWindow draws. `run_on_main_thread`+ // self-guards on `has_graphic_access()` and returns immediately+ // when the daemon has no Window Server session — fall through to+ // join so the daemon still serves headless. The serve thread runs+ // on its background thread regardless.+ platform_macos::cursor::overlay::run_on_main_thread();+ let _ = serve_handle.join();+ } else {+ let _ = serve_handle.join();}}cli::Command::Stop {@@ -1088,7 +993,7 @@ fn main() -> anyhow::Result<()> {claude_code_compat,cua_driver_core::embedded_mode(),)?;- let _ = maybe_init_pip();+ maybe_init_pip();let sp = socket.unwrap_or_else(serve::default_socket_path);let pid_path = serve::default_pid_file_path();// run_serve_cmd builds its own runtime; must run on a fresh thread.diff --git a/libs/cua-driver/rust/crates/cursor-overlay/src/lib.rs b/libs/cua-driver/rust/crates/cursor-overlay/src/lib.rsindex 2943ccf4a1..be79e5b15d 100644--- a/libs/cua-driver/rust/crates/cursor-overlay/src/lib.rs+++ b/libs/cua-driver/rust/crates/cursor-overlay/src/lib.rs@@ -27,10 +27,10 @@ pub use render_state::{SESSION_BADGE_HOLD_SECS,};pub use session_badge::{- paint_session_badge, rasterize_inter_text, sanitize_session_label, session_badge_extents,- session_badge_layout, BadgeExtents, BadgeLabelLayout, SessionBadgeInput, SessionBadgeLayout,- TextRaster, BADGE_CHIP_GAP, BADGE_CHIP_GROUP_GAP, BADGE_CHIP_SIZE, BADGE_CURSOR_GAP,- BADGE_HEIGHT, BADGE_MAX_WIDTH, MAX_SESSION_LABEL_CHARS,+ paint_session_badge, sanitize_session_label, session_badge_extents, session_badge_layout,+ BadgeExtents, BadgeLabelLayout, SessionBadgeInput, SessionBadgeLayout, BADGE_CHIP_GAP,+ BADGE_CHIP_GROUP_GAP, BADGE_CHIP_SIZE, BADGE_CURSOR_GAP, BADGE_HEIGHT, BADGE_MAX_WIDTH,+ MAX_SESSION_LABEL_CHARS,};pub use theme::{session_fill_hex, session_fill_rgba, CursorAction, CursorVisualState, DeliveryModifier,diff --git a/libs/cua-driver/rust/crates/cursor-overlay/src/session_badge.rs b/libs/cua-driver/rust/crates/cursor-overlay/src/session_badge.rsindex d2f3139ed6..b37a2c8576 100644--- a/libs/cua-driver/rust/crates/cursor-overlay/src/session_badge.rs+++ b/libs/cua-driver/rust/crates/cursor-overlay/src/session_badge.rs@@ -202,56 +202,6 @@ pub fn sanitize_session_label(input: &str) -> Option<String> {Some(result)}-/// Tightly cropped 8-bit coverage mask for one line of rasterized text.-#[derive(Debug, Clone, PartialEq)]-pub struct TextRaster {- pub width: u32,- pub height: u32,- /// Row-major coverage, exactly `width * height` bytes.- pub coverage: Vec<u8>,-}--/// Rasterize one line of UI text with the bundled Inter face.-///-/// Renderers that do not draw through `tiny_skia` — the Windows Agent View-/// canvas paints straight into a DIB — can compose the returned mask themselves-/// and still share one typeface with the cursor badge, so no second font asset-/// or drawing dependency is needed. Returns `None` for text that rasterizes to-/// nothing, such as an empty or whitespace-only string.-pub fn rasterize_inter_text(text: &str, font_size: f32) -> Option<TextRaster> {- let font = font()?;- let layout = text_layout(font, text, font_size.max(1.0));- let bounds = text_bounds(&layout)?;- let width = bounds.width.ceil().max(1.0) as u32;- let height = bounds.height.ceil().max(1.0) as u32;- let mut coverage = vec![0u8; width as usize * height as usize];- for glyph in layout.glyphs() {- let (metrics, bitmap) = font.rasterize_config(glyph.key);- if metrics.width == 0 || metrics.height == 0 {- continue;- }- let origin_x = (glyph.x - bounds.min_x).round() as i32;- let origin_y = (glyph.y - bounds.min_y).round() as i32;- for row in 0..metrics.height {- for column in 0..metrics.width {- let x = origin_x + column as i32;- let y = origin_y + row as i32;- if x < 0 || y < 0 || x >= width as i32 || y >= height as i32 {- continue;- }- let at = y as usize * width as usize + x as usize;- coverage[at] = coverage[at].max(bitmap[row * metrics.width + column]);- }- }- }- let has_ink = coverage.iter().any(|&sample| sample > 0);- has_ink.then_some(TextRaster {- width,- height,- coverage,- })-}-fn rounded_rect(rect: Rect, radius: f32) -> Option<Path> {const K: f32 = 0.552_284_8;let x = rect.x();@@ -634,28 +584,6 @@ mod tests {assert!(long.ends_with('…'));}- #[test]- fn rasterized_ui_text_is_cropped_and_reuses_the_badge_typeface() {- let raster = rasterize_inter_text("10:42 AM", 12.0).unwrap();- assert_eq!(- raster.coverage.len(),- raster.width as usize * raster.height as usize- );- assert!(raster.width > raster.height);- assert!(raster.coverage.iter().any(|&coverage| coverage > 128));- // A tight crop means the first and last columns both carry ink.- let column_has_ink = |column: u32| {- (0..raster.height)- .any(|row| raster.coverage[(row * raster.width + column) as usize] > 0)- };- assert!(column_has_ink(0));- assert!((raster.width.saturating_sub(2)..raster.width).any(column_has_ink));-- let larger = rasterize_inter_text("10:42 AM", 20.0).unwrap();- assert!(larger.width > raster.width);- assert!(rasterize_inter_text(" ", 12.0).is_none());- }-#[test]fn badge_paints_at_one_and_two_x_backing_scales() {for scale in [1.0, 2.0] {diff --git a/libs/cua-driver/rust/crates/pip-preview/src/desktop_layout.rs b/libs/cua-driver/rust/crates/pip-preview/src/desktop_layout.rsdeleted file mode 100644index 86ea2437b0..0000000000--- a/libs/cua-driver/rust/crates/pip-preview/src/desktop_layout.rs+++ /dev/null@@ -1,631 +0,0 @@-//! Platform-neutral geometry for the Agent View miniature desktop.--#[derive(Debug, Clone, Copy, PartialEq)]-pub struct LayoutRect {- pub x: f64,- pub y: f64,- pub width: f64,- pub height: f64,-}--impl LayoutRect {- #[cfg(test)]- fn right(self) -> f64 {- self.x + self.width- }-- #[cfg(test)]- fn bottom(self) -> f64 {- self.y + self.height- }-}--#[derive(Debug, Clone, Copy, PartialEq, Eq)]-pub struct TargetSize {- pub width: u32,- pub height: u32,-}--impl TargetSize {- fn aspect(self) -> f64 {- if self.width == 0 || self.height == 0 {- return 16.0 / 10.0;- }- (self.width as f64 / self.height as f64).clamp(0.4, 2.8)- }-}--#[derive(Debug, Clone, Copy, PartialEq)]-pub struct TargetLayout {- pub window: LayoutRect,- pub content: LayoutRect,-}--/// Shell chrome idiom for the miniature desktop.-///-/// Only the dock band differs between styles. Target geometry is identical-/// under every style, so a session's cards never move when the host platform-/// swaps its shell.-#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]-pub enum ShellStyle {- /// No permanent launcher or taskbar chrome.- #[default]- None,- /// Centered dock floating clear of every edge (macOS and Linux).- FloatingDock,- /// Full-width bar flush with the bottom screen edge (Windows).- EdgeTaskbar,-}--#[derive(Debug, Clone, PartialEq)]-pub struct DesktopLayout {- pub desktop: LayoutRect,- pub dock: LayoutRect,- pub dock_icons: Vec<LayoutRect>,- pub targets: Vec<TargetLayout>,- pub shell: ShellStyle,- /// Leading launcher slot. `EdgeTaskbar` only.- pub start_button: Option<LayoutRect>,- /// Trailing status band reserved for a clock and tray glyphs.- /// `EdgeTaskbar` only.- pub tray: Option<LayoutRect>,- /// Running-app marks, one per dock icon in the same order.- /// `EdgeTaskbar` only.- pub indicators: Vec<LayoutRect>,-}--#[derive(Debug, Clone, PartialEq)]-struct ShellChrome {- dock: LayoutRect,- dock_icons: Vec<LayoutRect>,- start_button: Option<LayoutRect>,- tray: Option<LayoutRect>,- indicators: Vec<LayoutRect>,-}--const DOCK_ICON_GAP: f64 = 8.0;--/// Lay out a miniature desktop with the floating dock shell.-///-/// Targets keep a stable row-major order. Columns receive widths based on the-/// average aspect ratio of their targets, which gives landscape windows more-/// room while keeping portrait and square windows compact. Narrow containers-/// collapse to one column instead of shrinking previews beyond usefulness.-pub fn layout_desktop(width: f64, height: f64, targets: &[TargetSize]) -> DesktopLayout {- layout_desktop_with_shell(width, height, targets, ShellStyle::None)-}--/// Lay out a miniature desktop for one platform shell idiom.-///-/// The desktop band, and therefore every `TargetLayout`, is derived from the-/// dock's top edge alone. Both shells compute that edge identically, so the-/// shell only decides how the dock band itself is drawn.-pub fn layout_desktop_with_shell(- width: f64,- height: f64,- targets: &[TargetSize],- shell: ShellStyle,-) -> DesktopLayout {- let width = width.max(1.0);- let height = height.max(1.0);- let short_edge = width.min(height);- // Keep target shadows and rounded corners visibly separated from the- // miniature desktop frame, including at the minimum supported panel size.- let outer_gap = (short_edge * 0.045).clamp(16.0, 26.0);- let dock_height = (height * 0.12).clamp(38.0, 58.0).min(height * 0.22);- let dock_bottom_margin = (height * 0.028).clamp(10.0, 16.0);- let dock_y = if shell == ShellStyle::None {- height - outer_gap- } else {- (height - dock_height - dock_bottom_margin).max(outer_gap)- };- let desktop_y = outer_gap;- let desktop_height = (dock_y - outer_gap - desktop_y).max(1.0);- let desktop = LayoutRect {- x: outer_gap,- y: desktop_y,- width: (width - 2.0 * outer_gap).max(1.0),- height: desktop_height,- };-- let chrome = match shell {- ShellStyle::None => ShellChrome {- dock: LayoutRect {- x: 0.0,- y: height,- width: 0.0,- height: 0.0,- },- dock_icons: Vec::new(),- start_button: None,- tray: None,- indicators: Vec::new(),- },- ShellStyle::FloatingDock => {- floating_dock_chrome(width, outer_gap, dock_y, dock_height, targets.len())- }- ShellStyle::EdgeTaskbar => edge_taskbar_chrome(width, height, dock_y, targets.len()),- };-- if targets.is_empty() {- return DesktopLayout::assemble(desktop, shell, chrome, Vec::new());- }-- let columns = if targets.len() == 1 || desktop.width < 430.0 {- 1- } else if targets.len() <= 4 || desktop.width < 760.0 {- 2- } else {- 3- };- let rows = targets.len().div_ceil(columns);- let cell_gap = (short_edge * 0.014).clamp(5.0, 10.0);- let usable_width = (desktop.width - cell_gap * (columns - 1) as f64).max(1.0);-- let mut column_weights = vec![0.0; columns];- let mut column_counts = vec![0usize; columns];- for (index, target) in targets.iter().enumerate() {- let column = index % columns;- column_weights[column] += target.aspect().clamp(0.65, 2.0);- column_counts[column] += 1;- }- for column in 0..columns {- column_weights[column] = if column_counts[column] == 0 {- 1.0- } else {- (column_weights[column] / column_counts[column] as f64).clamp(0.7, 1.8)- };- }- let total_column_weight = column_weights.iter().sum::<f64>().max(1.0);- let column_widths = column_weights- .iter()- .map(|weight| usable_width * weight / total_column_weight)- .collect::<Vec<_>>();-- let mut row_weights = vec![1.0; rows];- for (row, row_weight) in row_weights.iter_mut().enumerate() {- let mut desired_height: f64 = 1.0;- for (column, column_width) in column_widths.iter().enumerate() {- let index = row * columns + column;- let Some(target) = targets.get(index) else {- continue;- };- desired_height = desired_height.max(column_width / target.aspect());- }- *row_weight = desired_height;- }- let usable_height = (desktop.height - cell_gap * (rows - 1) as f64).max(1.0);- let total_row_weight = row_weights.iter().sum::<f64>().max(1.0);- let row_heights = row_weights- .iter()- .map(|weight| usable_height * weight / total_row_weight)- .collect::<Vec<_>>();-- let mut column_x = Vec::with_capacity(columns);- let mut x = desktop.x;- for column_width in &column_widths {- column_x.push(x);- x += column_width + cell_gap;- }- let mut row_y = Vec::with_capacity(rows);- let mut y = desktop.y;- for row_height in &row_heights {- row_y.push(y);- y += row_height + cell_gap;- }-- let target_layouts = targets- .iter()- .enumerate()- .map(|(index, target)| {- let column = index % columns;- let row = index / columns;- fit_target(- LayoutRect {- x: column_x[column],- y: row_y[row],- width: column_widths[column],- height: row_heights[row],- },- target.aspect(),- )- })- .collect::<Vec<_>>();-- DesktopLayout::assemble(desktop, shell, chrome, target_layouts)-}--impl DesktopLayout {- fn assemble(- desktop: LayoutRect,- shell: ShellStyle,- chrome: ShellChrome,- targets: Vec<TargetLayout>,- ) -> Self {- Self {- desktop,- dock: chrome.dock,- dock_icons: chrome.dock_icons,- targets,- shell,- start_button: chrome.start_button,- tray: chrome.tray,- indicators: chrome.indicators,- }- }-}--fn floating_dock_chrome(- width: f64,- outer_gap: f64,- dock_y: f64,- dock_height: f64,- target_count: usize,-) -> ShellChrome {- let icon_count = target_count.max(1);- let max_icon_width = ((width - 4.0 * outer_gap - DOCK_ICON_GAP * (icon_count - 1) as f64)- / icon_count as f64)- .max(16.0);- let icon_size = (dock_height - 16.0).min(max_icon_width).clamp(20.0, 42.0);- let dock_width =- (icon_count as f64 * icon_size + (icon_count - 1) as f64 * DOCK_ICON_GAP + 28.0)- .min(width - 2.0 * outer_gap)- .max(icon_size + 20.0);- let dock = LayoutRect {- x: (width - dock_width) / 2.0,- y: dock_y,- width: dock_width,- height: dock_height,- };- let icons_width = icon_count as f64 * icon_size + (icon_count - 1) as f64 * DOCK_ICON_GAP;- let icon_x = dock.x + (dock.width - icons_width) / 2.0;- let icon_y = dock.y + (dock.height - icon_size) / 2.0 - 1.0;- let dock_icons = (0..target_count)- .map(|index| LayoutRect {- x: icon_x + index as f64 * (icon_size + DOCK_ICON_GAP),- y: icon_y,- width: icon_size,- height: icon_size,- })- .collect::<Vec<_>>();- ShellChrome {- dock,- dock_icons,- start_button: None,- tray: None,- indicators: Vec::new(),- }-}--/// Windows 11-style bar: full width, flush with the bottom edge, with the-/// launcher and app icons centered as one group and the tray trailing.-///-/// The bar takes its top edge from the shared `dock_y` and reaches the bottom-/// by absorbing the floating dock's bottom margin, so it gains height without-/// moving the desktop band above it.-fn edge_taskbar_chrome(width: f64, height: f64, dock_y: f64, target_count: usize) -> ShellChrome {- let dock = LayoutRect {- x: 0.0,- y: dock_y,- width,- height: (height - dock_y).max(1.0),- };- let tray_width = (width * 0.16).clamp(56.0, 104.0).min(width * 0.34);- let tray = LayoutRect {- x: (width - tray_width).max(0.0),- y: dock.y,- width: tray_width,- height: dock.height,- };-- // Reserve the tray band on both sides so the icon group stays centered on- // the whole bar rather than on the space left of the clock, and let icons- // shrink instead of sliding under the tray on a very narrow panel.- let slots = target_count + 1;- let slot_span =- (width - 2.0 * (tray_width + 6.0) - DOCK_ICON_GAP * (slots - 1) as f64) / slots as f64;- let icon_size = (dock.height * 0.55).min(slot_span).clamp(8.0, 32.0);- let group_width = slots as f64 * icon_size + (slots - 1) as f64 * DOCK_ICON_GAP;- let group_x = ((width - group_width) / 2.0).max(4.0);- // Sit the icons slightly high in the band to leave room for the marks.- let icon_y = dock.y + (dock.height - icon_size) / 2.0 - 2.0;- let slot_x = |index: usize| group_x + index as f64 * (icon_size + DOCK_ICON_GAP);-- let start_button = LayoutRect {- x: slot_x(0),- y: icon_y,- width: icon_size,- height: icon_size,- };- let dock_icons = (0..target_count)- .map(|index| LayoutRect {- x: slot_x(index + 1),- y: icon_y,- width: icon_size,- height: icon_size,- })- .collect::<Vec<_>>();-- let indicator_height = (icon_size * 0.1).clamp(2.0, 3.0);- let indicator_width = (icon_size * 0.42).max(4.0);- let indicator_y = (icon_y + icon_size + 3.0).min(dock.y + dock.height - indicator_height - 1.0);- let indicators = dock_icons- .iter()- .map(|icon| LayoutRect {- x: icon.x + (icon.width - indicator_width) / 2.0,- y: indicator_y,- width: indicator_width,- height: indicator_height,- })- .collect::<Vec<_>>();-- ShellChrome {- dock,- dock_icons,- start_button: Some(start_button),- tray: Some(tray),- indicators,- }-}--fn fit_target(cell: LayoutRect, aspect: f64) -> TargetLayout {- let inset = 1.0;- let available_width = (cell.width - 2.0 * inset).max(1.0);- let available_content_height = (cell.height - 2.0 * inset).max(1.0);- let content_width = available_width.min(available_content_height * aspect);- let content_height = (content_width / aspect).min(available_content_height);- let window_width = content_width;- let window_height = content_height;- let window = LayoutRect {- x: cell.x + (cell.width - window_width) / 2.0,- y: cell.y + (cell.height - window_height) / 2.0,- width: window_width,- height: window_height,- };- let content = LayoutRect {- x: window.x,- y: window.y,- width: content_width,- height: content_height,- };- TargetLayout { window, content }-}--pub fn png_dimensions(bytes: &[u8]) -> Option<TargetSize> {- const PNG_SIGNATURE: &[u8; 8] = b"\x89PNG\r\n\x1a\n";- if bytes.len() < 24 || &bytes[..8] != PNG_SIGNATURE || &bytes[12..16] != b"IHDR" {- return None;- }- let width = u32::from_be_bytes(bytes[16..20].try_into().ok()?);- let height = u32::from_be_bytes(bytes[20..24].try_into().ok()?);- (width > 0 && height > 0).then_some(TargetSize { width, height })-}--#[cfg(test)]-mod tests {- use super::*;-- fn size(width: u32, height: u32) -> TargetSize {- TargetSize { width, height }- }-- fn intersects(a: LayoutRect, b: LayoutRect) -> bool {- a.x < b.right() && a.right() > b.x && a.y < b.bottom() && a.bottom() > b.y- }-- #[test]- fn mixed_form_factors_make_the_landscape_column_wider() {- let layout = layout_desktop(- 720.0,- 520.0,- &[- size(1600, 900),- size(700, 1200),- size(1400, 900),- size(900, 900),- ],- );- assert_eq!(layout.targets.len(), 4);- assert!(layout.targets[0].window.width > layout.targets[1].window.width);- assert!(layout.targets[2].window.width > layout.targets[3].window.width);- }-- #[test]- fn narrow_desktop_reflows_to_one_column() {- let layout = layout_desktop(380.0, 620.0, &[size(1600, 900), size(700, 1200)]);- assert!(layout.targets[1].window.y > layout.targets[0].window.y);- assert!((layout.targets[0].window.x - layout.targets[1].window.x).abs() < 80.0);- }-- #[test]- fn target_windows_stay_inside_the_desktop_and_do_not_overlap() {- let layout = layout_desktop(- 840.0,- 560.0,- &[- size(1600, 900),- size(700, 1200),- size(1400, 900),- size(900, 900),- size(1000, 700),- size(800, 1100),- ],- );- for target in &layout.targets {- assert!(target.window.x >= layout.desktop.x - 0.01);- assert!(target.window.y >= layout.desktop.y - 0.01);- assert!(target.window.right() <= layout.desktop.right() + 0.01);- assert!(target.window.bottom() <= layout.desktop.bottom() + 0.01);- }- for (index, target) in layout.targets.iter().enumerate() {- for other in layout.targets.iter().skip(index + 1) {- assert!(!intersects(target.window, other.window));- }- }- }-- #[test]- fn content_preserves_the_source_aspect_ratio() {- let targets = [size(1600, 900), size(700, 1200), size(900, 900)];- let layout = layout_desktop(720.0, 520.0, &targets);- for (target, source) in layout.targets.iter().zip(targets) {- let rendered_aspect = target.content.width / target.content.height;- assert!((rendered_aspect - source.aspect()).abs() < 0.001);- assert_eq!(target.window, target.content);- }- }-- #[test]- fn layout_reserves_padding_but_no_menu_or_target_title_bars() {- let layout = layout_desktop(720.0, 520.0, &[size(1600, 900)]);- assert!((16.0..=26.0).contains(&layout.desktop.y));- assert_eq!(layout.targets[0].window, layout.targets[0].content);- }-- #[test]- fn minimum_panel_keeps_targets_away_from_the_frame() {- let layout = layout_desktop(360.0, 260.0, &[size(900, 700)]);- let target = layout.targets[0].window;- assert!(target.x >= 16.0);- assert!(target.y >= 16.0);- assert!(target.right() <= 344.0);- }-- #[test]- fn dock_stays_below_targets_and_inside_the_panel() {- let height = 520.0;- let layout = layout_desktop_with_shell(- 720.0,- height,- &[size(1600, 900), size(700, 1200), size(900, 900)],- ShellStyle::FloatingDock,- );- let lowest_target = layout- .targets- .iter()- .map(|target| target.window.bottom())- .fold(0.0, f64::max);- assert!(layout.dock.y >= lowest_target);- assert!(layout.dock.bottom() <= height - 10.0);- }-- #[test]- fn no_shell_is_the_default_and_carries_no_launcher_metadata() {- let layout = layout_desktop(720.0, 520.0, &[size(1600, 900), size(900, 900)]);- assert_eq!(layout.shell, ShellStyle::None);- assert_eq!(layout.shell, ShellStyle::default());- assert!(layout.start_button.is_none());- assert!(layout.tray.is_none());- assert!(layout.indicators.is_empty());- assert_eq!(layout.dock.width, 0.0);- assert!(layout.dock_icons.is_empty());- }-- #[test]- fn edge_taskbar_spans_the_full_width_and_reaches_the_bottom_edge() {- let width = 720.0;- let height = 520.0;- let targets = [size(1600, 900), size(900, 900), size(700, 1200)];- let floating = layout_desktop_with_shell(width, height, &targets, ShellStyle::FloatingDock);- let taskbar = layout_desktop_with_shell(width, height, &targets, ShellStyle::EdgeTaskbar);-- assert_eq!(taskbar.shell, ShellStyle::EdgeTaskbar);- assert_eq!(taskbar.dock.x, 0.0);- assert_eq!(taskbar.dock.width, width);- assert_eq!(taskbar.dock.bottom(), height);- // The bar keeps the floating dock's top edge and only absorbs the- // margin that used to sit below it.- assert_eq!(taskbar.dock.y, floating.dock.y);- assert!(taskbar.dock.height > floating.dock.height);- assert!(taskbar.dock.height - floating.dock.height <= 16.0 + f64::EPSILON);- }-- #[test]- fn edge_taskbar_preserves_every_floating_dock_target_layout() {- let targets = [- size(1600, 900),- size(700, 1200),- size(1400, 900),- size(900, 900),- size(1000, 700),- ];- for (width, height) in [(720.0, 520.0), (380.0, 620.0), (360.0, 260.0)] {- let floating =- layout_desktop_with_shell(width, height, &targets, ShellStyle::FloatingDock);- let taskbar =- layout_desktop_with_shell(width, height, &targets, ShellStyle::EdgeTaskbar);- assert_eq!(floating.desktop, taskbar.desktop);- assert_eq!(floating.targets, taskbar.targets);- }- }-- #[test]- fn edge_taskbar_centers_the_start_slot_with_the_app_icons() {- let width = 720.0;- let targets = [size(1600, 900), size(900, 900)];- let layout = layout_desktop_with_shell(width, 520.0, &targets, ShellStyle::EdgeTaskbar);- let start = layout.start_button.unwrap();- let tray = layout.tray.unwrap();-- assert_eq!(layout.dock_icons.len(), targets.len());- assert!(start.x < layout.dock_icons[0].x);- assert_eq!(start.y, layout.dock_icons[0].y);- assert_eq!(start.width, layout.dock_icons[0].width);-- let group_left = start.x;- let group_right = layout.dock_icons.last().unwrap().right();- assert!(((group_left + group_right) / 2.0 - width / 2.0).abs() < 1.0);- assert!(group_right <= tray.x);- assert_eq!(tray.right(), width);- assert_eq!(tray.y, layout.dock.y);- assert_eq!(tray.height, layout.dock.height);- }-- #[test]- fn edge_taskbar_marks_every_running_icon_inside_the_bar() {- let targets = [size(1600, 900), size(900, 900), size(700, 1200)];- let layout = layout_desktop_with_shell(720.0, 520.0, &targets, ShellStyle::EdgeTaskbar);- assert_eq!(layout.indicators.len(), layout.dock_icons.len());- for (indicator, icon) in layout.indicators.iter().zip(&layout.dock_icons) {- assert!(indicator.width < icon.width);- assert!(indicator.y >= icon.bottom());- assert!(indicator.bottom() <= layout.dock.bottom());- assert!(- ((indicator.x + indicator.width / 2.0) - (icon.x + icon.width / 2.0)).abs() < 1.0- );- }- }-- #[test]- fn edge_taskbar_keeps_the_start_slot_when_no_target_has_a_card() {- let layout = layout_desktop_with_shell(640.0, 420.0, &[], ShellStyle::EdgeTaskbar);- assert!(layout.start_button.is_some());- assert!(layout.tray.is_some());- assert!(layout.dock_icons.is_empty());- assert!(layout.indicators.is_empty());- assert!(layout.targets.is_empty());- }-- #[test]- fn crowded_narrow_taskbar_shrinks_icons_instead_of_sliding_under_the_tray() {- let targets = vec![size(1200, 800); 12];- let layout = layout_desktop_with_shell(320.0, 260.0, &targets, ShellStyle::EdgeTaskbar);- let tray = layout.tray.unwrap();- assert!(layout.start_button.unwrap().x >= 0.0);- assert!(layout.dock_icons.last().unwrap().right() <= tray.x);- for icon in &layout.dock_icons {- assert!(icon.width >= 8.0);- }- }-- #[test]- fn reads_png_ihdr_dimensions_without_decoding_the_image() {- let mut png = vec![0u8; 24];- png[..8].copy_from_slice(b"\x89PNG\r\n\x1a\n");- png[12..16].copy_from_slice(b"IHDR");- png[16..20].copy_from_slice(&1440u32.to_be_bytes());- png[20..24].copy_from_slice(&900u32.to_be_bytes());- assert_eq!(png_dimensions(&png), Some(size(1440, 900)));- assert_eq!(png_dimensions(b"not png"), None);- }-}diff --git a/libs/cua-driver/rust/crates/pip-preview/src/lib.rs b/libs/cua-driver/rust/crates/pip-preview/src/lib.rsindex 22f453f4a8..f8efa30b7e 100644--- a/libs/cua-driver/rust/crates/pip-preview/src/lib.rs+++ b/libs/cua-driver/rust/crates/pip-preview/src/lib.rs@@ -1,28 +1,21 @@-//! Shared model and platform contract for Agent View.+//! pip-preview — shared types + trait for the experimental+//! picture-in-picture agent preview window.//!-//! Agent View is an opt-in surface rather than one process-global "latest screenshot".-//! Frames carry an existing session/workspace identity and an exact native-//! window or browser-tab identity. Platform backends can therefore keep-//! several target cards visible without introducing target claims or leases.+//! The PiP window is an opt-in, always-on-top floating window that+//! shows what the cua-driver agent just did: a post-action screenshot+//! of the target window plus a one-line label summarising the tool+//! call. It mirrors the architecture used by `cursor-overlay` (shared+//! config/types here, platform-specific renderer in each `platform-*`+//! crate) and the registration pattern used by `cua_driver_core::video`+//! (a `OnceLock` factory set once at startup by `main.rs`).//!-//! Native backends render the shared model on macOS, Windows, and Linux-//! X11/XWayland while keeping platform-specific window-system code isolated.+//! macOS is the first working implementation (NSWindow + NSImageView).+//! Windows + Linux ship as compile-clean stubs whose `start()` returns+//! a clear "not yet implemented" error so the rest of the daemon+//! continues without a PiP window.use std::sync::OnceLock;-use std::collections::HashMap;--mod desktop_layout;-mod session_tabs;--pub use desktop_layout::{- layout_desktop, layout_desktop_with_shell, png_dimensions, DesktopLayout, LayoutRect,- ShellStyle, TargetLayout, TargetSize,-};-pub use session_tabs::{layout_session_tabs, session_accent, SessionTab, SessionTabsLayout};--pub const AGENT_VIEW_DEFAULT_ENABLED: bool = false;-/// Canonical `~/.cua-driver/config.json` path matching what the per-platform/// `set_config` tools write to. Resolves `$HOME` first (Unix/macOS) and falls/// back to `%USERPROFILE%` (Windows, where `HOME` is usually unset). Returns@@ -51,8 +44,8 @@ pub fn read_config_value(key: &str) -> Option<serde_json::Value> {/// Merge a single `key`/`value` into `~/.cua-driver/config.json`,/// preserving any other keys that are already there. Used by the-/// per-platform `set_config` tools to persist `agent_view` /-/// `agent_view_geometry` so the next daemon restart picks them up.+/// per-platform `set_config` tools to persist `experimental_pip` /+/// `experimental_pip_geometry` so the next daemon restart picks them up.pub fn write_config_key(key: &str, value: serde_json::Value) -> Result<(), String> {let path = default_config_path().ok_or_else(|| "$HOME is not set".to_string())?;let mut json: serde_json::Value = path@@ -70,29 +63,29 @@ pub fn write_config_key(key: &str, value: serde_json::Value) -> Result<(), StrinOk(())}-/// Read `agent_view` + `agent_view_geometry` from the+/// Read `experimental_pip` + `experimental_pip_geometry` from the/// config file, falling back to defaults when missing or malformed./// Surfaced by the per-platform `get_config` tools alongside the/// in-memory `DriverConfig` fields.-pub fn read_agent_view_keys_from_file() -> (bool, Option<String>) {+pub fn read_pip_keys_from_file() -> (bool, Option<String>) {let path = match default_config_path() {Some(p) => p,- None => return (AGENT_VIEW_DEFAULT_ENABLED, None),+ None => return (false, None),};let text = match std::fs::read_to_string(&path) {Ok(t) => t,- Err(_) => return (AGENT_VIEW_DEFAULT_ENABLED, None),+ Err(_) => return (false, None),};let json: serde_json::Value = match serde_json::from_str(&text) {Ok(v) => v,- Err(_) => return (AGENT_VIEW_DEFAULT_ENABLED, None),+ Err(_) => return (false, None),};let enabled = json- .get("agent_view")+ .get("experimental_pip").and_then(|v| v.as_bool())- .unwrap_or(AGENT_VIEW_DEFAULT_ENABLED);+ .unwrap_or(false);let geometry = json- .get("agent_view_geometry")+ .get("experimental_pip_geometry").and_then(|v| v.as_str()).map(|s| s.to_owned());(enabled, geometry)@@ -100,7 +93,7 @@ pub fn read_agent_view_keys_from_file() -> (bool, Option<String>) {/// Geometry of the PiP window, in screen points (top-left origin).///-/// Parsed from `--agent-view-geometry WxH+X+Y`. `x` / `y` are+/// Parsed from `--experimental-pip-geometry WxH+X+Y`. `x` / `y` are/// optional; when `None` the platform backend picks a sensible/// "top-right corner with a small inset" default so a user enabling/// the feature without any geometry flags still sees a window.@@ -115,8 +108,8 @@ pub struct PipGeometry {impl Default for PipGeometry {fn default() -> Self {Self {- width: 640,- height: 420,+ width: 320,+ height: 200,x: None,y: None,}@@ -156,32 +149,33 @@ impl PipGeometry {/// flags and handed to `PipBackendFactory::start`.#[derive(Debug, Clone)]pub struct PipConfig {- /// `--agent-view` is on argv. The factory is only consulted+ /// `--experimental-pip` is on argv. The factory is only consulted/// when this is true; the field is kept here so backends that/// share a `start()` path can early-return.pub enabled: bool,pub geometry: PipGeometry,- /// Native window title shared by all platform backends.+ /// Window title — kept here so the "experimental" label stays in+ /// one place. Defaults to "cua-driver — PiP preview (experimental)".pub title: String,}impl Default for PipConfig {fn default() -> Self {Self {- enabled: AGENT_VIEW_DEFAULT_ENABLED,+ enabled: false,geometry: PipGeometry::default(),- title: "Cua Agent View".to_owned(),+ title: "cua-driver — PiP preview (experimental)".to_owned(),}}}impl PipConfig {- /// Parse the Agent View CLI flags out of `std::env::args()`.- /// Recognised flags:+ /// Parse the PiP-related CLI flags out of `std::env::args()`.+ /// Recognised flags (all opt-in):/// ```text- /// --agent-view- /// --no-agent-view- /// --agent-view-geometry WxH | WxH+X+Y+ /// --experimental-pip+ /// --pip (short alias)+ /// --experimental-pip-geometry WxH | WxH+X+Y/// ```/// Unknown flags are ignored so this never conflicts with the/// other arg-parser passes (CursorConfig, the subcommand router).@@ -192,27 +186,13 @@ impl PipConfig {pub fn parse(args: &[String]) -> Self {let mut cfg = PipConfig::default();- let (enabled, geometry) = Self::parse_overrides(args);- if let Some(enabled) = enabled {- cfg.enabled = enabled;- }- if let Some(geometry) = geometry {- cfg.geometry = geometry;- }- cfg- }-- fn parse_overrides(args: &[String]) -> (Option<bool>, Option<PipGeometry>) {- let mut enabled = None;- let mut geometry = None;let mut i = 0usize;while i < args.len() {match args[i].as_str() {- "--agent-view" => enabled = Some(true),- "--no-agent-view" => enabled = Some(false),- "--agent-view-geometry" => {+ "--experimental-pip" | "--pip" => cfg.enabled = true,+ "--experimental-pip-geometry" => {if let Some(geom) = args.get(i + 1).and_then(|s| PipGeometry::parse(s)) {- geometry = Some(geom);+ cfg.geometry = geom;i += 1;}}@@ -220,32 +200,30 @@ impl PipConfig {}i += 1;}- (enabled, geometry)+ cfg}/// Resolve the config from (in order of precedence, low → high):////// defaults → `~/.cua-driver/config.json` keys- /// (`agent_view` bool, `agent_view_geometry` string)+ /// (`experimental_pip` bool, `experimental_pip_geometry` string)/// → CLI flags///- /// Lets users persist `--agent-view` across daemon restarts by+ /// Lets users persist `--experimental-pip` across daemon restarts by/// editing `~/.cua-driver/config.json` once, instead of re-running/// `claude mcp add` with the flag baked into the args list./// Malformed or missing file falls back to the next layer silently.pub fn from_args_and_file(config_path: &std::path::Path) -> Self {- let args: Vec<String> = std::env::args().skip(1).collect();- Self::from_file_and_args(config_path, &args)- }-- fn from_file_and_args(config_path: &std::path::Path, args: &[String]) -> Self {let mut cfg = PipConfig::default();if let Ok(text) = std::fs::read_to_string(config_path) {if let Ok(json) = serde_json::from_str::<serde_json::Value>(&text) {- if let Some(b) = json.get("agent_view").and_then(|v| v.as_bool()) {+ if let Some(b) = json.get("experimental_pip").and_then(|v| v.as_bool()) {cfg.enabled = b;}- if let Some(s) = json.get("agent_view_geometry").and_then(|v| v.as_str()) {+ if let Some(s) = json+ .get("experimental_pip_geometry")+ .and_then(|v| v.as_str())+ {if let Some(g) = PipGeometry::parse(s) {cfg.geometry = g;}@@ -253,73 +231,26 @@ impl PipConfig {}}// CLI args override anything in the file.- let (enabled, geometry) = PipConfig::parse_overrides(args);- if let Some(enabled) = enabled {- cfg.enabled = enabled;+ let args: Vec<String> = std::env::args().collect();+ let cli = PipConfig::parse(&args[1..]);+ if cli.enabled {+ cfg.enabled = true;}- if let Some(geometry) = geometry {- cfg.geometry = geometry;+ // CLI geometry only overrides when explicitly passed — detect by+ // diffing against PipGeometry::default() (the parse() entry point+ // returns the default when no flag is present).+ if cli.geometry.width != PipGeometry::default().width+ || cli.geometry.height != PipGeometry::default().height+ || cli.geometry.x.is_some()+ || cli.geometry.y.is_some()+ {+ cfg.geometry = cli.geometry;}cfg}}-/// The exact surface represented by an Agent View card.-#[derive(Debug, Clone, Copy, PartialEq, Eq)]-pub enum PipTargetKind {- NativeWindow,- BrowserTab,-}--impl PipTargetKind {- pub fn as_str(self) -> &'static str {- match self {- Self::NativeWindow => "native_window",- Self::BrowserTab => "browser_tab",- }- }-}--/// Stable target metadata used to group frames in the Agent View.-#[derive(Debug, Clone, PartialEq, Eq)]-pub struct PipTarget {- /// Runtime-unique session key. This is grouping metadata, not an ownership- /// or authorization claim.- pub workspace_id: String,- /// Short public session label shown to the user.- pub workspace_label: String,- /// Stable exact-target key within the workspace.- pub target_id: String,- /// Process-local identity used to de-duplicate equivalent bindings. This- /// may be more stable than the public target reference (for example a CDP- /// page target across repeated browser-window binds).- pub identity_key: String,- pub target_kind: PipTargetKind,- pub target_label: String,- /// Native window containing this target, when proven. Browser tabs use it- /// to avoid a duplicate native Chrome/Edge window card.- pub native_container: Option<PipNativeContainer>,-}--impl PipTarget {- pub fn view_id(&self) -> String {- view_id(&self.workspace_id, &self.identity_key)- }-}--fn view_id(workspace_id: &str, identity_key: &str) -> String {- // Both identities may contain `:`. Prefixing the workspace byte length- // prevents cross-session concatenation aliases.- format!("{}:{workspace_id}{identity_key}", workspace_id.len())-}--#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]-pub struct PipNativeContainer {- pub pid: i64,- pub window_id: u64,-}--/// A single exact-target frame pushed into the Agent View after a tool call.+/// A single frame pushed into the PiP window after a tool call lands.////// `png_bytes` are the raw PNG bytes produced by the platform/// screenshot callback — the same path that powers `screenshot.png`@@ -327,7 +258,6 @@ pub struct PipNativeContainer {/// sees.#[derive(Debug, Clone)]pub struct PipFrame {- pub target: PipTarget,pub png_bytes: Vec<u8>,/// One-line summary shown overlayed on the frame, e.g./// `click element_index=2` or `type_text "hello world"`.@@ -335,351 +265,6 @@ pub struct PipFrame {/// Wall-clock timestamp (ms since Unix epoch) — used by backends/// that want to show "last update Xs ago" in the title bar.pub timestamp_ms: u64,- /// Normalized location of the synthetic agent pointer within the target image.- pub cursor_position: Option<(f64, f64)>,-}--/// Platform-neutral latest-frame model with bounded target retention.-///-/// Backends use this to keep one card per exact target. The oldest-updated-/// card is evicted when the configured limit is reached; ending a workspace-/// removes all of its cards without closing the underlying applications.-pub struct PipViewModel {- max_targets: usize,- frames: HashMap<String, PipFrame>,- workspace_activity_ms: HashMap<String, u64>,- active_targets: HashMap<String, String>,- selected_workspace_id: Option<String>,- selection_pinned: bool,-}--#[derive(Debug, Clone, PartialEq, Eq)]-pub struct PipWorkspaceSummary {- pub workspace_id: String,- pub workspace_label: String,- pub target_count: usize,- pub updated_ms: u64,-}--impl PipViewModel {- pub fn new(max_targets: usize) -> Self {- Self {- max_targets: max_targets.max(1),- frames: HashMap::new(),- workspace_activity_ms: HashMap::new(),- active_targets: HashMap::new(),- selected_workspace_id: None,- selection_pinned: false,- }- }-- pub fn upsert(&mut self, frame: PipFrame) -> Option<String> {- let mut frame = frame;- let workspace_id = frame.target.workspace_id.clone();- self.workspace_activity_ms- .entry(workspace_id.clone())- .and_modify(|timestamp| *timestamp = (*timestamp).max(frame.timestamp_ms))- .or_insert(frame.timestamp_ms);- if self.selected_workspace_id.is_none() || !self.selection_pinned {- self.selected_workspace_id = Some(workspace_id.clone());- }- if frame.target.target_kind == PipTargetKind::NativeWindow- && frame.target.native_container.is_some_and(|container| {- self.frames.values().any(|existing| {- existing.target.workspace_id == workspace_id- && existing.target.target_kind == PipTargetKind::BrowserTab- && existing.target.native_container == Some(container)- })- })- {- return None;- }- if frame.target.target_kind == PipTargetKind::BrowserTab {- if let Some(container) = frame.target.native_container {- self.frames.retain(|_, existing| {- existing.target.workspace_id != workspace_id- || existing.target.target_kind != PipTargetKind::NativeWindow- || existing.target.native_container != Some(container)- });- }- }- let view_id = frame.target.view_id();- if frame.cursor_position.is_none() {- frame.cursor_position = self- .frames- .get(&view_id)- .and_then(|previous| previous.cursor_position);- }- self.frames.insert(view_id.clone(), frame);- self.active_targets- .insert(workspace_id.clone(), view_id.clone());- if self.frames.len() <= self.max_targets {- return None;- }- let counts = self- .frames- .values()- .fold(HashMap::new(), |mut counts, frame| {- *counts- .entry(frame.target.workspace_id.as_str())- .or_insert(0usize) += 1;- counts- });- let largest_workspace = counts- .iter()- .max_by(|(left_id, left_count), (right_id, right_count)| {- left_count- .cmp(right_count)- .then_with(|| right_id.cmp(left_id))- })- .map(|(id, _)| *id);- let evicted = self- .frames- .iter()- .filter(|(id, _)| *id != &view_id)- .min_by_key(|(_, candidate)| {- (- candidate.target.workspace_id.as_str() != largest_workspace.unwrap_or(""),- candidate.timestamp_ms,- )- })- .map(|(id, _)| id.clone());- if let Some(id) = evicted.as_ref() {- let workspace_id = self- .frames- .get(id)- .map(|frame| frame.target.workspace_id.clone());- self.frames.remove(id);- self.clear_active_view(id);- if let Some(workspace_id) = workspace_id.as_deref() {- self.reconcile_active_target(workspace_id);- }- }- self.prune_empty_workspace_activity();- self.reconcile_selection();- evicted- }-- pub fn remove_target(&mut self, workspace_id: &str, identity_key: &str) -> bool {- let direct = view_id(workspace_id, identity_key);- let view_id = self- .frames- .contains_key(&direct)- .then_some(direct)- .or_else(|| {- self.frames- .iter()- .find(|(_, frame)| {- frame.target.workspace_id == workspace_id- && frame.target.target_id == identity_key- })- .map(|(id, _)| id.clone())- });- let removed = view_id- .as_ref()- .is_some_and(|view_id| self.frames.remove(view_id).is_some());- if removed {- if let Some(view_id) = view_id.as_deref() {- self.clear_active_view(view_id);- }- self.reconcile_active_target(workspace_id);- self.prune_empty_workspace_activity();- self.reconcile_selection();- }- removed- }-- pub fn remove_workspace(&mut self, workspace_id: &str) -> Vec<String> {- let removed = self- .frames- .iter()- .filter(|(_, frame)| frame.target.workspace_id == workspace_id)- .map(|(id, _)| id.clone())- .collect::<Vec<_>>();- for id in &removed {- self.frames.remove(id);- }- self.workspace_activity_ms.remove(workspace_id);- self.active_targets.remove(workspace_id);- if self.selected_workspace_id.as_deref() == Some(workspace_id) {- self.selection_pinned = false;- }- self.reconcile_selection();- removed- }-- pub fn ordered_frames(&self) -> Vec<&PipFrame> {- let mut frames = self.frames.values().collect::<Vec<_>>();- frames.sort_by(|a, b| {- a.target- .workspace_label- .cmp(&b.target.workspace_label)- .then_with(|| b.timestamp_ms.cmp(&a.timestamp_ms))- .then_with(|| a.target.target_id.cmp(&b.target.target_id))- });- frames- }-- pub fn selected_workspace_id(&self) -> Option<&str> {- self.selected_workspace_id.as_deref()- }-- pub fn selected_frames(&self) -> Vec<&PipFrame> {- let Some(selected) = self.selected_workspace_id() else {- return Vec::new();- };- let mut frames = self- .frames- .values()- .filter(|frame| frame.target.workspace_id == selected)- .collect::<Vec<_>>();- frames.sort_by(|a, b| {- b.timestamp_ms- .cmp(&a.timestamp_ms)- .then_with(|| a.target.identity_key.cmp(&b.target.identity_key))- });- frames- }-- pub fn active_view_id(&self) -> Option<&str> {- let workspace_id = self.selected_workspace_id()?;- self.active_targets.get(workspace_id).map(String::as_str)- }-- pub fn workspaces(&self) -> Vec<PipWorkspaceSummary> {- let mut summaries = HashMap::<&str, PipWorkspaceSummary>::new();- for frame in self.frames.values() {- let entry = summaries- .entry(&frame.target.workspace_id)- .or_insert_with(|| PipWorkspaceSummary {- workspace_id: frame.target.workspace_id.clone(),- workspace_label: sanitize_label(&frame.target.workspace_label, 48),- target_count: 0,- updated_ms: self- .workspace_activity_ms- .get(&frame.target.workspace_id)- .copied()- .unwrap_or(frame.timestamp_ms),- });- entry.target_count += 1;- }- let mut summaries = summaries.into_values().collect::<Vec<_>>();- summaries.sort_by(|a, b| {- b.updated_ms- .cmp(&a.updated_ms)- .then_with(|| a.workspace_label.cmp(&b.workspace_label))- .then_with(|| a.workspace_id.cmp(&b.workspace_id))- });- summaries- }-- pub fn select_workspace(&mut self, workspace_id: &str) -> bool {- if !self- .frames- .values()- .any(|frame| frame.target.workspace_id == workspace_id)- {- return false;- }- self.selected_workspace_id = Some(workspace_id.to_owned());- self.selection_pinned = true;- true- }-- pub fn select_next_workspace(&mut self) -> bool {- let workspaces = self.workspaces();- if workspaces.len() <= 1 {- return false;- }- let selected = self.selected_workspace_id();- let index = workspaces- .iter()- .position(|workspace| Some(workspace.workspace_id.as_str()) == selected)- .unwrap_or(0);- self.select_workspace(&workspaces[(index + 1) % workspaces.len()].workspace_id)- }-- pub fn selection_is_pinned(&self) -> bool {- self.selection_pinned- }-- fn reconcile_selection(&mut self) {- let selected_exists = self- .selected_workspace_id- .as_deref()- .is_some_and(|selected| {- self.frames- .values()- .any(|frame| frame.target.workspace_id == selected)- });- if selected_exists {- return;- }- self.selection_pinned = false;- self.selected_workspace_id = self- .workspaces()- .into_iter()- .next()- .map(|workspace| workspace.workspace_id);- }-- fn prune_empty_workspace_activity(&mut self) {- self.workspace_activity_ms.retain(|workspace_id, _| {- self.frames- .values()- .any(|frame| frame.target.workspace_id == *workspace_id)- });- }-- fn clear_active_view(&mut self, view_id: &str) {- self.active_targets.retain(|_, active| active != view_id);- }-- fn reconcile_active_target(&mut self, workspace_id: &str) {- if self- .active_targets- .get(workspace_id)- .is_some_and(|active| self.frames.contains_key(active))- {- return;- }- let fallback = self- .frames- .iter()- .filter(|(_, frame)| frame.target.workspace_id == workspace_id)- .max_by(|(left_id, left), (right_id, right)| {- left.timestamp_ms- .cmp(&right.timestamp_ms)- .then_with(|| right_id.cmp(left_id))- })- .map(|(id, _)| id.clone());- if let Some(fallback) = fallback {- self.active_targets- .insert(workspace_id.to_owned(), fallback);- } else {- self.active_targets.remove(workspace_id);- }- }-- pub fn len(&self) -> usize {- self.frames.len()- }-- pub fn is_empty(&self) -> bool {- self.frames.is_empty()- }-}--fn sanitize_label(value: &str, max_chars: usize) -> String {- let mut label = value- .chars()- .filter(|character| !character.is_control())- .take(max_chars)- .collect::<String>();- if label.trim().is_empty() {- label = "Agent".to_owned();- }- label}/// A live PiP window. Owned by `main.rs` for the lifetime of the@@ -691,24 +276,6 @@ pub trait PipBackend: Send + Sync {/// queue via `dispatch_async`).fn push_frame(&self, frame: PipFrame);- /// Remove every card associated with an ended session/workspace. This- /// changes only Agent View presentation; it does not close applications.- fn remove_workspace(&self, workspace_id: &str);-- /// Remove one exact target from presentation. This never closes or- /// otherwise mutates the underlying native window or browser tab.- fn remove_target(&self, _workspace_id: &str, _identity_key: &str) {}-- /// Make the floating surface ignore or resume native pointer input.- ///- /// Implementations must not return until the native window-system state is- /// applied. Cua Driver uses this around physical desktop actions so an- /// overlapping always-on-top Agent View cannot intercept the action meant- /// for an underlying target.- fn set_input_passthrough(&self, _passthrough: bool) -> anyhow::Result<()> {- Ok(())- }-/// Close the window and release native resources. Called from/// `main.rs` on shutdown.fn shutdown(self: Box<Self>);@@ -739,494 +306,3 @@ pub fn start_pip(cfg: &PipConfig) -> anyhow::Result<Box<dyn PipBackend>> {.ok_or_else(|| anyhow::anyhow!("no PiP backend registered for this platform"))?;factory.start(cfg)}--#[cfg(test)]-mod tests {- use super::*;-- fn frame(workspace: &str, target: &str, timestamp_ms: u64) -> PipFrame {- PipFrame {- target: PipTarget {- workspace_id: workspace.to_owned(),- workspace_label: workspace.to_owned(),- target_id: target.to_owned(),- identity_key: target.to_owned(),- target_kind: PipTargetKind::NativeWindow,- target_label: target.to_owned(),- native_container: None,- },- png_bytes: vec![1],- action_label: "click".to_owned(),- timestamp_ms,- cursor_position: None,- }- }-- fn browser_frame(- workspace: &str,- target: &str,- identity: &str,- container: PipNativeContainer,- timestamp_ms: u64,- ) -> PipFrame {- PipFrame {- target: PipTarget {- workspace_id: workspace.to_owned(),- workspace_label: workspace.to_owned(),- target_id: target.to_owned(),- identity_key: identity.to_owned(),- target_kind: PipTargetKind::BrowserTab,- target_label: target.to_owned(),- native_container: Some(container),- },- png_bytes: vec![1],- action_label: "get_browser_state".to_owned(),- timestamp_ms,- cursor_position: None,- }- }-- fn native_frame(- workspace: &str,- target: &str,- container: PipNativeContainer,- timestamp_ms: u64,- ) -> PipFrame {- let mut frame = frame(workspace, target, timestamp_ms);- frame.target.native_container = Some(container);- frame- }-- #[test]- fn keeps_distinct_targets_in_the_same_workspace() {- let mut model = PipViewModel::new(4);- model.upsert(frame("agent-a", "window:1:2", 1));- model.upsert(frame("agent-a", "browser:bt:tab", 2));- assert_eq!(model.len(), 2);- }-- #[test]- fn non_pointer_updates_preserve_the_last_cursor_position() {- let mut model = PipViewModel::new(4);- let mut clicked = frame("agent-a", "window:1:2", 1);- clicked.cursor_position = Some((0.25, 0.75));- model.upsert(clicked);- model.upsert(frame("agent-a", "window:1:2", 2));- assert_eq!(- model.selected_frames()[0].cursor_position,- Some((0.25, 0.75))- );- }-- #[test]- fn updating_one_target_does_not_replace_another() {- let mut model = PipViewModel::new(4);- model.upsert(frame("agent-a", "window:1:2", 1));- model.upsert(frame("agent-a", "browser:bt:tab", 2));- model.upsert(frame("agent-a", "window:1:2", 3));- assert_eq!(model.len(), 2);- assert_eq!(model.ordered_frames()[0].target.target_id, "window:1:2");- }-- #[test]- fn ending_a_workspace_removes_only_its_cards() {- let mut model = PipViewModel::new(4);- model.upsert(frame("agent-a", "window:1:2", 1));- model.upsert(frame("agent-b", "window:3:4", 2));- assert_eq!(model.remove_workspace("agent-a").len(), 1);- assert_eq!(model.len(), 1);- assert_eq!(model.ordered_frames()[0].target.workspace_id, "agent-b");- }-- #[test]- fn evicts_the_oldest_other_target_at_capacity() {- let mut model = PipViewModel::new(2);- model.upsert(frame("agent-a", "old", 1));- model.upsert(frame("agent-a", "newer", 2));- assert_eq!(- model.upsert(frame("agent-b", "newest", 3)),- Some(view_id("agent-a", "old"))- );- assert_eq!(model.len(), 2);- }-- #[test]- fn follows_recent_activity_until_the_user_pins_a_workspace() {- let mut model = PipViewModel::new(4);- model.upsert(frame("agent-a", "a", 1));- model.upsert(frame("agent-b", "b", 2));- assert_eq!(model.selected_workspace_id(), Some("agent-b"));-- assert!(model.select_workspace("agent-a"));- model.upsert(frame("agent-b", "b", 3));- assert_eq!(model.selected_workspace_id(), Some("agent-a"));- assert!(model.selection_is_pinned());- assert_eq!(model.selected_frames().len(), 1);- }-- #[test]- fn active_target_is_scoped_to_the_selected_workspace() {- let mut model = PipViewModel::new(6);- model.upsert(frame("agent-a", "a-old", 1));- model.upsert(frame("agent-a", "a-active", 2));- model.upsert(frame("agent-b", "b-old", 3));- model.upsert(frame("agent-b", "b-active", 4));-- assert_eq!(- model.active_view_id(),- Some(view_id("agent-b", "b-active").as_str())- );- assert!(model.select_workspace("agent-a"));- assert_eq!(- model.active_view_id(),- Some(view_id("agent-a", "a-active").as_str())- );- assert!(model.select_workspace("agent-b"));- assert_eq!(- model.active_view_id(),- Some(view_id("agent-b", "b-active").as_str())- );- }-- #[test]- fn removing_an_active_target_falls_back_within_its_workspace() {- let mut model = PipViewModel::new(6);- model.upsert(frame("agent-a", "older", 1));- model.upsert(frame("agent-a", "newer", 2));- model.upsert(frame("agent-b", "other", 3));- assert!(model.select_workspace("agent-a"));-- assert!(model.remove_target("agent-a", "newer"));-- assert_eq!(- model.active_view_id(),- Some(view_id("agent-a", "older").as_str())- );- assert_eq!(model.selected_workspace_id(), Some("agent-a"));- }-- #[test]- fn evicting_an_active_target_falls_back_within_its_workspace() {- let mut model = PipViewModel::new(3);- model.upsert(frame("agent-a", "fallback", 10));- model.upsert(frame("agent-a", "active", 1));- model.upsert(frame("agent-b", "other", 20));- assert!(model.select_workspace("agent-a"));-- assert_eq!(- model.upsert(frame("agent-c", "new", 30)),- Some(view_id("agent-a", "active"))- );- assert_eq!(- model.active_view_id(),- Some(view_id("agent-a", "fallback").as_str())- );- }-- #[test]- fn removing_the_selected_workspace_falls_back_to_the_most_recent() {- let mut model = PipViewModel::new(4);- model.upsert(frame("agent-a", "a", 1));- model.upsert(frame("agent-b", "b", 2));- assert!(model.select_workspace("agent-a"));- model.remove_workspace("agent-a");- assert_eq!(model.selected_workspace_id(), Some("agent-b"));- assert!(!model.selection_is_pinned());- }-- #[test]- fn exact_target_removal_uses_the_internal_identity_key() {- let mut model = PipViewModel::new(4);- model.upsert(frame("agent-a", "window:1:2", 1));- assert!(model.remove_target("agent-a", "window:1:2"));- assert!(model.is_empty());- assert_eq!(model.selected_workspace_id(), None);- }-- #[test]- fn equal_public_labels_do_not_merge_private_workspaces() {- let mut model = PipViewModel::new(4);- let mut first = frame("private-a", "one", 1);- first.target.workspace_label = "research".to_owned();- let mut second = frame("private-b", "two", 2);- second.target.workspace_label = "research".to_owned();- model.upsert(first);- model.upsert(second);-- let workspaces = model.workspaces();- assert_eq!(workspaces.len(), 2);- assert!(workspaces- .iter()- .all(|workspace| workspace.workspace_label == "research"));- assert_eq!(model.selected_workspace_id(), Some("private-b"));- }-- #[test]- fn stable_browser_identity_updates_across_repeated_bindings() {- let mut model = PipViewModel::new(4);- let container = PipNativeContainer {- pid: 42,- window_id: 7,- };- model.upsert(browser_frame(- "agent-a",- "browser:first:tab-a",- "browser-cdp:page-1",- container,- 1,- ));- model.upsert(browser_frame(- "agent-a",- "browser:second:tab-b",- "browser-cdp:page-1",- container,- 2,- ));-- assert_eq!(model.len(), 1);- assert_eq!(- model.selected_frames()[0].target.target_id,- "browser:second:tab-b"- );- assert_eq!(model.selected_frames()[0].timestamp_ms, 2);- }-- #[test]- fn browser_tabs_replace_and_suppress_their_native_container_card() {- let mut model = PipViewModel::new(6);- let container = PipNativeContainer {- pid: 42,- window_id: 7,- };- model.upsert(native_frame("agent-a", "window:42:7", container, 1));- model.upsert(browser_frame(- "agent-a",- "browser:binding:tab-a",- "browser-cdp:page-a",- container,- 2,- ));- model.upsert(browser_frame(- "agent-a",- "browser:binding:tab-b",- "browser-cdp:page-b",- container,- 3,- ));- model.upsert(native_frame("agent-a", "window:42:7", container, 4));-- let frames = model.selected_frames();- assert_eq!(frames.len(), 2);- assert!(frames- .iter()- .all(|frame| frame.target.target_kind == PipTargetKind::BrowserTab));- }-- #[test]- fn suppressed_native_container_activity_still_drives_auto_follow() {- let mut model = PipViewModel::new(6);- let first_container = PipNativeContainer {- pid: 42,- window_id: 7,- };- let second_container = PipNativeContainer {- pid: 84,- window_id: 9,- };- model.upsert(browser_frame(- "agent-a",- "browser:first:tab-a",- "browser-cdp:page-a",- first_container,- 1,- ));- model.upsert(browser_frame(- "agent-b",- "browser:second:tab-b",- "browser-cdp:page-b",- second_container,- 2,- ));- assert_eq!(model.selected_workspace_id(), Some("agent-b"));-- model.upsert(native_frame("agent-a", "window:42:7", first_container, 3));-- assert_eq!(model.selected_workspace_id(), Some("agent-a"));- assert_eq!(model.workspaces()[0].workspace_id, "agent-a");- assert_eq!(model.workspaces()[0].updated_ms, 3);- assert!(model- .selected_frames()- .iter()- .all(|frame| frame.target.target_kind == PipTargetKind::BrowserTab));- }-- #[test]- fn private_session_and_target_delimiters_cannot_alias() {- let mut model = PipViewModel::new(4);- model.upsert(frame("private:a", "b", 1));- model.upsert(frame("private", "a:b", 2));-- assert_eq!(model.len(), 2);- assert!(model.select_workspace("private:a"));- assert_eq!(model.selected_frames()[0].target.target_id, "b");- assert!(model.select_workspace("private"));- assert_eq!(model.selected_frames()[0].target.target_id, "a:b");- }-- #[test]- fn eviction_prefers_the_most_populated_workspace() {- let mut model = PipViewModel::new(3);- model.upsert(frame("agent-a", "old-a", 1));- model.upsert(frame("agent-a", "new-a", 2));- model.upsert(frame("agent-b", "only-b", 3));- assert_eq!(- model.upsert(frame("agent-c", "only-c", 4)),- Some(view_id("agent-a", "old-a"))- );- assert!(model- .ordered_frames()- .iter()- .any(|frame| frame.target.workspace_id == "agent-b"));- }-- #[test]- fn parses_agent_view_cli_names() {- let cfg = PipConfig::parse(&[- "--agent-view".to_owned(),- "--agent-view-geometry".to_owned(),- "640x420+24+36".to_owned(),- ]);- assert!(cfg.enabled);- assert_eq!(cfg.geometry.width, 640);- assert_eq!(cfg.geometry.height, 420);- assert_eq!(cfg.geometry.x, Some(24));- assert_eq!(cfg.geometry.y, Some(36));- }-- #[test]- fn agent_view_is_disabled_by_default() {- let cfg = PipConfig::parse(&[]);- assert!(!cfg.enabled);- assert!(!PipConfig::default().enabled);- }-- #[test]- fn ignores_removed_pip_cli_names() {- let cfg = PipConfig::parse(&[- "--experimental-pip".to_owned(),- "--experimental-pip-geometry".to_owned(),- "640x420".to_owned(),- "--pip".to_owned(),- ]);- assert!(!cfg.enabled);- assert_eq!(cfg.geometry.width, PipGeometry::default().width);- assert_eq!(cfg.geometry.height, PipGeometry::default().height);- }-- #[test]- fn reads_agent_view_config_names() {- let path = std::env::temp_dir().join(format!(- "cua-agent-view-config-{}-new.json",- std::process::id()- ));- std::fs::write(- &path,- r#"{"agent_view":true,"agent_view_geometry":"800x600+12+24"}"#,- )- .unwrap();-- let cfg = PipConfig::from_args_and_file(&path);- assert!(cfg.enabled);- assert_eq!(cfg.geometry.width, 800);- assert_eq!(cfg.geometry.height, 600);- assert_eq!(cfg.geometry.x, Some(12));- assert_eq!(cfg.geometry.y, Some(24));-- let _ = std::fs::remove_file(path);- }-- #[test]- fn config_file_overrides_defaults_without_cli_flags() {- let path = std::env::temp_dir().join(format!(- "cua-agent-view-config-{}-file-precedence.json",- std::process::id()- ));- std::fs::write(- &path,- r#"{"agent_view":false,"agent_view_geometry":"800x600+12+24"}"#,- )- .unwrap();-- let cfg = PipConfig::from_file_and_args(&path, &[]);- assert!(!cfg.enabled);- assert_eq!(cfg.geometry.width, 800);- assert_eq!(cfg.geometry.height, 600);- assert_eq!(cfg.geometry.x, Some(12));- assert_eq!(cfg.geometry.y, Some(24));-- let _ = std::fs::remove_file(path);- }-- #[test]- fn agent_view_cli_overrides_disabled_config_file() {- let path = std::env::temp_dir().join(format!(- "cua-agent-view-config-{}-cli-enable.json",- std::process::id()- ));- std::fs::write(&path, r#"{"agent_view":false}"#).unwrap();-- let cfg = PipConfig::from_file_and_args(&path, &["--agent-view".to_owned()]);- assert!(cfg.enabled);-- let _ = std::fs::remove_file(path);- }-- #[test]- fn cli_flags_override_config_file_values() {- let path = std::env::temp_dir().join(format!(- "cua-agent-view-config-{}-cli-precedence.json",- std::process::id()- ));- std::fs::write(- &path,- r#"{"agent_view":true,"agent_view_geometry":"800x600+12+24"}"#,- )- .unwrap();-- let cfg = PipConfig::from_file_and_args(- &path,- &[- "--no-agent-view".to_owned(),- "--agent-view-geometry".to_owned(),- "720x480+30+40".to_owned(),- ],- );- assert!(!cfg.enabled);- assert_eq!(cfg.geometry.width, 720);- assert_eq!(cfg.geometry.height, 480);- assert_eq!(cfg.geometry.x, Some(30));- assert_eq!(cfg.geometry.y, Some(40));-- let _ = std::fs::remove_file(path);- }-- #[test]- fn ignores_removed_pip_config_names() {- let path = std::env::temp_dir().join(format!(- "cua-agent-view-config-{}-old.json",- std::process::id()- ));- std::fs::write(- &path,- r#"{"experimental_pip":true,"experimental_pip_geometry":"800x600"}"#,- )- .unwrap();-- let cfg = PipConfig::from_args_and_file(&path);- assert!(!cfg.enabled);- assert_eq!(cfg.geometry.width, PipGeometry::default().width);- assert_eq!(cfg.geometry.height, PipGeometry::default().height);-- let _ = std::fs::remove_file(path);- }-}diff --git a/libs/cua-driver/rust/crates/pip-preview/src/session_tabs.rs b/libs/cua-driver/rust/crates/pip-preview/src/session_tabs.rsdeleted file mode 100644index fe85c8d7ba..0000000000--- a/libs/cua-driver/rust/crates/pip-preview/src/session_tabs.rs+++ /dev/null@@ -1,136 +0,0 @@-use crate::{LayoutRect, PipWorkspaceSummary};--const MAX_VISIBLE_TABS: usize = 6;--#[derive(Debug, Clone, PartialEq)]-pub struct SessionTab {- pub workspace_id: String,- pub label: String,- pub rect: LayoutRect,- pub selected: bool,- pub accent: (u8, u8, u8),-}--#[derive(Debug, Clone, PartialEq, Default)]-pub struct SessionTabsLayout {- pub tabs: Vec<SessionTab>,- pub overflow: usize,-}--impl SessionTabsLayout {- pub fn hit_test(&self, x: f64, y: f64) -> Option<&str> {- self.tabs- .iter()- .find(|tab| {- x >= tab.rect.x- && x <= tab.rect.x + tab.rect.width- && y >= tab.rect.y- && y <= tab.rect.y + tab.rect.height- })- .map(|tab| tab.workspace_id.as_str())- }-}--pub fn layout_session_tabs(- panel: LayoutRect,- workspaces: &[PipWorkspaceSummary],- selected_workspace_id: Option<&str>,-) -> SessionTabsLayout {- if workspaces.len() <= 1 {- return SessionTabsLayout::default();- }- let visible = workspaces.len().min(MAX_VISIBLE_TABS);- let gap = 6.0;- let height = (panel.height * 0.09).clamp(28.0, 42.0);- let width = ((panel.width - gap * (visible.saturating_sub(1)) as f64) / visible as f64)- .clamp(90.0, 190.0);- let tabs = workspaces- .iter()- .take(visible)- .enumerate()- .map(|(index, workspace)| SessionTab {- workspace_id: workspace.workspace_id.clone(),- label: workspace.workspace_label.clone(),- rect: LayoutRect {- x: panel.x + index as f64 * (width + gap),- y: panel.y,- width,- height,- },- selected: selected_workspace_id == Some(workspace.workspace_id.as_str()),- accent: session_accent(&workspace.workspace_id),- })- .collect();- SessionTabsLayout {- tabs,- overflow: workspaces.len().saturating_sub(visible),- }-}--pub fn session_accent(workspace_id: &str) -> (u8, u8, u8) {- let hash = workspace_id- .bytes()- .fold(0u32, |hash, byte| hash.wrapping_mul(16777619) ^ byte as u32);- const COLORS: [(u8, u8, u8); 6] = [- (55, 148, 255),- (255, 166, 54),- (54, 190, 160),- (228, 96, 135),- (139, 112, 246),- (66, 181, 225),- ];- COLORS[hash as usize % COLORS.len()]-}--#[cfg(test)]-mod tests {- use super::*;-- fn workspace(id: &str, updated_ms: u64) -> PipWorkspaceSummary {- PipWorkspaceSummary {- workspace_id: id.into(),- workspace_label: id.into(),- target_count: 1,- updated_ms,- }- }-- #[test]- fn hides_single_session_and_directly_hit_tests_tabs() {- let panel = LayoutRect {- x: 10.0,- y: 12.0,- width: 600.0,- height: 400.0,- };- assert!(layout_session_tabs(panel, &[workspace("a", 1)], Some("a"))- .tabs- .is_empty());- let layout = layout_session_tabs(panel, &[workspace("b", 2), workspace("a", 1)], Some("a"));- assert_eq!(- layout.hit_test(layout.tabs[0].rect.x + 2.0, layout.tabs[0].rect.y + 2.0),- Some("b")- );- assert!(layout.tabs[1].selected);- }-- #[test]- fn caps_visible_tabs_and_reports_overflow() {- let workspaces = (0..8)- .map(|i| workspace(&format!("agent-{i}"), i))- .collect::<Vec<_>>();- let layout = layout_session_tabs(- LayoutRect {- x: 0.0,- y: 0.0,- width: 900.0,- height: 500.0,- },- &workspaces,- None,- );- assert_eq!(layout.tabs.len(), 6);- assert_eq!(layout.overflow, 2);- assert_eq!(session_accent("stable"), session_accent("stable"));- }-}diff --git a/libs/cua-driver/rust/crates/platform-linux/src/pip/mod.rs b/libs/cua-driver/rust/crates/platform-linux/src/pip/mod.rsindex df35fe672a..805c0eb4e6 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/pip/mod.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/pip/mod.rs@@ -1,1348 +1,23 @@-//! Linux multi-target Agent View.+//! Linux picture-in-picture preview stub.//!-//! The backend uses a regular, resizable X11 utility window (and therefore-//! also works through XWayland) with a software-rendered GNOME-like overview.-//! It only presents frames supplied by the shared Agent View model; it never-//! moves, resizes, focuses, or closes the represented applications.--use std::sync::mpsc::{self, Receiver, Sender};-use std::time::Duration;--#[cfg(target_os = "linux")]-mod wayland;--#[cfg(target_os = "linux")]-use image::imageops::FilterType;-use pip_preview::{- layout_desktop, layout_session_tabs, png_dimensions, LayoutRect, PipBackend, PipBackendFactory,- PipConfig, PipFrame, PipTargetKind, PipViewModel, PipWorkspaceSummary, SessionTabsLayout,- TargetSize,-};--const MIN_WIDTH: u16 = 360;-const MIN_HEIGHT: u16 = 260;--pub(super) enum UiMessage {- Frame(PipFrame),- RemoveTarget(String, String),- RemoveWorkspace(String),- SetInputTransparent(bool, mpsc::SyncSender<anyhow::Result<()>>),- Shutdown,-}--pub struct LinuxPipBackend {- tx: Sender<UiMessage>,-}--impl LinuxPipBackend {- /// Synchronously include or exclude Agent View from X11 pointer hit-testing.- ///- /// Cua actions should enable transparency before injecting input and restore- /// interaction afterward. Waiting for the X11 thread to flush the shape- /// change prevents an always-on-top Agent View from intercepting the action.- pub fn set_input_transparent(&self, transparent: bool) -> anyhow::Result<()> {- let (reply_tx, reply_rx) = mpsc::sync_channel(1);- if self- .tx- .send(UiMessage::SetInputTransparent(transparent, reply_tx))- .is_err()- {- return Ok(());- }- reply_rx- .recv_timeout(Duration::from_secs(2))- .map_err(|error| {- anyhow::anyhow!("timed out updating Agent View input shape: {error}")- })?- }-}--impl PipBackend for LinuxPipBackend {- fn push_frame(&self, frame: PipFrame) {- tracing::debug!(- target: "pip",- view_id = %frame.target.view_id(),- "queueing Linux Agent View frame"- );- let _ = self.tx.send(UiMessage::Frame(frame));- }-- fn remove_workspace(&self, workspace_id: &str) {- let _ = self- .tx- .send(UiMessage::RemoveWorkspace(workspace_id.to_owned()));- }-- fn set_input_passthrough(&self, passthrough: bool) -> anyhow::Result<()> {- self.set_input_transparent(passthrough)- }-- fn remove_target(&self, workspace_id: &str, identity_key: &str) {- let _ = self.tx.send(UiMessage::RemoveTarget(- workspace_id.to_owned(),- identity_key.to_owned(),- ));- }+//! The Linux implementation is a follow-up to the experimental macOS+//! drop. Plan: GTK4 always-on-top utility window under X11/XWayland,+//! with a Wayland-native fallback via `wlr-layer-shell` where+//! supported. Tracking issue linked in the docs.+//!+//! Until that lands, `start()` returns a clear error so `main.rs`+//! can log "PiP unavailable on this platform" and continue without+//! the window.- fn shutdown(self: Box<Self>) {- let _ = self.tx.send(UiMessage::Shutdown);- }-}+use pip_preview::{PipBackend, PipBackendFactory, PipConfig};pub struct LinuxPipBackendFactory;impl PipBackendFactory for LinuxPipBackendFactory {- fn start(&self, cfg: &PipConfig) -> anyhow::Result<Box<dyn PipBackend>> {- let (tx, rx) = mpsc::channel();- let (ready_tx, ready_rx) = mpsc::sync_channel(1);- let cfg = cfg.clone();- std::thread::Builder::new()- .name("cua-agent-view-linux".to_owned())- .spawn(move || {- #[cfg(target_os = "linux")]- if should_use_native_wayland() {- wayland::run_window(cfg, rx, ready_tx);- return;- }- run_x11_window(cfg, rx, ready_tx)- })?;-- ready_rx- .recv()- .map_err(|_| anyhow::anyhow!("Agent View X11 thread exited during startup"))??;- Ok(Box::new(LinuxPipBackend { tx }))- }-}--#[cfg(target_os = "linux")]-fn should_use_native_wayland() -> bool {- std::env::var_os("WAYLAND_DISPLAY").is_some()- && (crate::wayland::is_wayland() || std::env::var_os("DISPLAY").is_none())-}--#[cfg(target_os = "linux")]-fn run_x11_window(- cfg: PipConfig,- rx: Receiver<UiMessage>,- ready: mpsc::SyncSender<anyhow::Result<()>>,-) {- use x11rb::connection::Connection;- use x11rb::protocol::xproto::{- AtomEnum, ConnectionExt as _, CreateGCAux, CreateWindowAux, EventMask, PropMode,- WindowClass,- };- use x11rb::protocol::Event;- use x11rb::wrapper::ConnectionExt as _;-- let setup = (|| -> anyhow::Result<_> {- let (conn, screen_num) = x11rb::connect(None)?;- let screen = &conn.setup().roots[screen_num];- let root = screen.root;- let root_depth = screen.root_depth;- let root_visual = screen.root_visual;- let screen_width = screen.width_in_pixels;- let width = cfg- .geometry- .width- .clamp(u32::from(MIN_WIDTH), u32::from(u16::MAX)) as u16;- let height = cfg- .geometry- .height- .clamp(u32::from(MIN_HEIGHT), u32::from(u16::MAX)) as u16;- let inset = 24i16;- let x = cfg.geometry.x.map(clamp_i16).unwrap_or_else(|| {- clamp_i16(i32::from(screen_width) - i32::from(width) - i32::from(inset))- });- let y = cfg.geometry.y.map(clamp_i16).unwrap_or(inset);- let window = conn.generate_id()?;- conn.create_window(- root_depth,- window,- root,- x,- y,- width,- height,- 0,- WindowClass::INPUT_OUTPUT,- root_visual,- &CreateWindowAux::new()- .background_pixel(0x1820_27)- .border_pixel(0)- .event_mask(- EventMask::EXPOSURE- | EventMask::STRUCTURE_NOTIFY- | EventMask::PROPERTY_CHANGE- | EventMask::BUTTON_PRESS,- ),- )?- .check()?;-- let gc = conn.generate_id()?;- conn.create_gc(gc, window, &CreateGCAux::new())?.check()?;- conn.change_property8(- PropMode::REPLACE,- window,- AtomEnum::WM_NAME,- AtomEnum::STRING,- cfg.title.as_bytes(),- )?;- conn.change_property8(- PropMode::REPLACE,- window,- AtomEnum::WM_CLASS,- AtomEnum::STRING,- b"cua-agent-view\0CuaAgentView\0",- )?;- let net_wm_pid = conn.intern_atom(false, b"_NET_WM_PID")?.reply()?.atom;- conn.change_property32(- PropMode::REPLACE,- window,- net_wm_pid,- AtomEnum::CARDINAL,- &[std::process::id()],- )?;-- let wm_protocols = conn.intern_atom(false, b"WM_PROTOCOLS")?.reply()?.atom;- let wm_delete = conn.intern_atom(false, b"WM_DELETE_WINDOW")?.reply()?.atom;- conn.change_property32(- PropMode::REPLACE,- window,- wm_protocols,- AtomEnum::ATOM,- &[wm_delete],- )?;- let net_wm_state = conn.intern_atom(false, b"_NET_WM_STATE")?.reply()?.atom;- let net_wm_state_above = conn- .intern_atom(false, b"_NET_WM_STATE_ABOVE")?- .reply()?- .atom;- conn.change_property32(- PropMode::REPLACE,- window,- net_wm_state,- AtomEnum::ATOM,- &[net_wm_state_above],- )?;- let net_wm_window_type = conn- .intern_atom(false, b"_NET_WM_WINDOW_TYPE")?- .reply()?- .atom;- let net_wm_window_type_utility = conn- .intern_atom(false, b"_NET_WM_WINDOW_TYPE_UTILITY")?- .reply()?- .atom;- conn.change_property32(- PropMode::REPLACE,- window,- net_wm_window_type,- AtomEnum::ATOM,- &[net_wm_window_type_utility],- )?;- conn.map_window(window)?.check()?;- conn.flush()?;- Ok((- conn,- window,- gc,- root_depth,- wm_protocols,- wm_delete,- width,- height,+ fn start(&self, _cfg: &PipConfig) -> anyhow::Result<Box<dyn PipBackend>> {+ Err(anyhow::anyhow!(+ "PiP preview is not yet implemented on Linux — track \+ trycua/cua follow-up issue for status"))- })();-- let (conn, window, gc, depth, wm_protocols, wm_delete, mut width, mut height) = match setup {- Ok(values) => {- let _ = ready.send(Ok(()));- values- }- Err(error) => {- let _ = ready.send(Err(error));- return;- }- };-- let mut model = PipViewModel::new(12);- let mut dirty = true;- let mut running = true;- while running {- while let Ok(message) = rx.try_recv() {- handle_x11_message(- &conn,- window,- width,- height,- message,- &mut model,- &mut dirty,- &mut running,- );- }-- loop {- match conn.poll_for_event() {- Ok(Some(Event::Expose(_))) => dirty = true,- Ok(Some(Event::ConfigureNotify(event))) => {- let next_width = event.width.max(1);- let next_height = event.height.max(1);- if next_width != width || next_height != height {- width = next_width;- height = next_height;- dirty = true;- }- }- Ok(Some(Event::ButtonPress(event))) => {- let workspaces = model.workspaces();- let tabs = session_tabs_for_bounds(- width,- height,- &workspaces,- model.selected_workspace_id(),- );- if let Some(workspace_id) =- tabs.hit_test(f64::from(event.event_x), f64::from(event.event_y))- {- dirty |= model.select_workspace(workspace_id);- }- }- Ok(Some(Event::ClientMessage(event)))- if is_delete_message(&event, wm_protocols, wm_delete) =>- {- running = false;- }- Ok(Some(_)) => {}- Ok(None) => break,- Err(error) => {- tracing::warn!(target: "pip", "Agent View X11 event error: {error}");- running = false;- break;- }- }- }-- if dirty && running {- let frames = model.selected_frames();- let workspaces = model.workspaces();- let pixels = render_agent_view(- width,- height,- &frames,- &workspaces,- model.selected_workspace_id(),- model.active_view_id(),- );- if let Err(error) = upload_image(&conn, window, gc, depth, width, height, &pixels) {- tracing::warn!(target: "pip", "Agent View X11 paint failed: {error}");- running = false;- }- dirty = false;- }-- if running {- match rx.recv_timeout(Duration::from_millis(16)) {- Ok(message) => handle_x11_message(- &conn,- window,- width,- height,- message,- &mut model,- &mut dirty,- &mut running,- ),- Err(mpsc::RecvTimeoutError::Timeout) => {}- Err(mpsc::RecvTimeoutError::Disconnected) => running = false,- }- }- }-- let _ = conn.free_gc(gc);- let _ = conn.destroy_window(window);- let _ = conn.flush();-}--#[cfg(target_os = "linux")]-fn handle_x11_message(- conn: &impl x11rb::connection::Connection,- window: u32,- width: u16,- height: u16,- message: UiMessage,- model: &mut PipViewModel,- dirty: &mut bool,- running: &mut bool,-) {- match message {- UiMessage::SetInputTransparent(transparent, reply) => {- let result = set_x11_input_transparent(conn, window, width, height, transparent);- let _ = reply.send(result);- }- message => handle_message(message, model, dirty, running),- }-}--#[cfg(target_os = "linux")]-fn set_x11_input_transparent(- conn: &impl x11rb::connection::Connection,- window: u32,- width: u16,- height: u16,- transparent: bool,-) -> anyhow::Result<()> {- use x11rb::protocol::shape::{ConnectionExt as ShapeConnectionExt, SK, SO};- use x11rb::protocol::xproto::{ClipOrdering, Rectangle};-- let interactive_region = [Rectangle {- x: 0,- y: 0,- width,- height,- }];- let rectangles = if transparent {- &[][..]- } else {- &interactive_region[..]- };- conn.shape_rectangles(- SO::SET,- SK::INPUT,- ClipOrdering::UNSORTED,- window,- 0,- 0,- rectangles,- )?- .check()?;- conn.flush()?;- Ok(())-}--#[cfg(target_os = "linux")]-fn upload_image(- conn: &impl x11rb::connection::Connection,- window: u32,- gc: u32,- depth: u8,- width: u16,- height: u16,- pixels: &[u8],-) -> anyhow::Result<()> {- use x11rb::protocol::xproto::{ConnectionExt as _, ImageFormat};-- let stride = usize::from(width) * 4;- // Keep each PutImage below the common 256 KiB X11 request limit.- let rows_per_request = (200_000usize / stride.max(1)).max(1);- for start_row in (0..usize::from(height)).step_by(rows_per_request) {- let rows = rows_per_request.min(usize::from(height) - start_row);- let start = start_row * stride;- let end = start + rows * stride;- conn.put_image(- ImageFormat::Z_PIXMAP,- window,- gc,- width,- rows as u16,- 0,- start_row as i16,- 0,- depth,- &pixels[start..end],- )?- .check()?;- }- conn.flush()?;- Ok(())-}--fn handle_message(- message: UiMessage,- model: &mut PipViewModel,- dirty: &mut bool,- running: &mut bool,-) {- match message {- UiMessage::Frame(frame) => {- tracing::debug!(- target: "pip",- view_id = %frame.target.view_id(),- "rendering Linux Agent View frame"- );- model.upsert(frame);- *dirty = true;- }- UiMessage::RemoveTarget(workspace_id, identity_key) => {- *dirty |= model.remove_target(&workspace_id, &identity_key);- }- UiMessage::RemoveWorkspace(workspace_id) => {- *dirty |= !model.remove_workspace(&workspace_id).is_empty();- }- UiMessage::SetInputTransparent(_, reply) => {- let _ = reply.send(Err(anyhow::anyhow!(- "Agent View input shape update reached a non-X11 handler"- )));- }- UiMessage::Shutdown => *running = false,- }-}--#[cfg(not(target_os = "linux"))]-fn run_x11_window(- _cfg: PipConfig,- _rx: Receiver<UiMessage>,- ready: mpsc::SyncSender<anyhow::Result<()>>,-) {- let _ = ready.send(Err(anyhow::anyhow!(- "Linux Agent View can only start on Linux"- )));-}--#[cfg(target_os = "linux")]-fn is_delete_message(- event: &x11rb::protocol::xproto::ClientMessageEvent,- wm_protocols: u32,- wm_delete: u32,-) -> bool {- event.type_ == wm_protocols && event.data.as_data32()[0] == wm_delete-}--fn clamp_i16(value: i32) -> i16 {- value.clamp(i32::from(i16::MIN), i32::from(i16::MAX)) as i16-}--pub(super) fn render_agent_view(- width: u16,- height: u16,- frames: &[&PipFrame],- workspaces: &[PipWorkspaceSummary],- selected_workspace_id: Option<&str>,- active_view_id: Option<&str>,-) -> Vec<u8> {- let mut canvas = Canvas::new(width, height);- canvas.vertical_gradient(Color::rgb(45, 57, 66), Color::rgb(24, 29, 34));- canvas.radial_glow(- width as f64 * 0.18,- height as f64 * 0.02,- width as f64 * 0.72,- Color::rgba(93, 129, 145, 70),- );- canvas.radial_glow(- width as f64 * 0.88,- height as f64 * 0.72,- width as f64 * 0.55,- Color::rgba(113, 90, 72, 42),- );- canvas.rounded_rect(- Rect::new(1.0, 1.0, width as f64 - 2.0, height as f64 - 2.0),- 15.0,- Color::rgba(15, 18, 21, 142),- );- canvas.stroke_rounded_rect(- Rect::new(1.5, 1.5, width as f64 - 3.0, height as f64 - 3.0),- 14.5,- 1.0,- Color::rgba(210, 221, 226, 92),- );-- let sizes = frames- .iter()- .map(|frame| {- png_dimensions(&frame.png_bytes).unwrap_or(TargetSize {- width: 16,- height: 10,- })- })- .collect::<Vec<_>>();- let selector_height = if workspaces.len() > 1 { 34.0 } else { 0.0 };- let mut layout = layout_desktop(- width.into(),- (f64::from(height) - selector_height).max(1.0),- &sizes,- );- offset_layout_y(&mut layout, selector_height);- if frames.is_empty() {- render_waiting_state(&mut canvas, layout.desktop);- } else {- for (frame, target) in frames.iter().zip(&layout.targets) {- render_target(&mut canvas, frame, target.window);- if active_view_id == Some(frame.target.view_id().as_str()) {- canvas.stroke_rounded_rect(- Rect::from_layout(target.window).expand(3.0),- 13.0,- 2.0,- Color::rgba(55, 148, 255, 242),- );- }- }- // Agent View has no permanent launcher; session tabs remain the only- // presentation-level navigation surface.- }- render_session_selector(- &mut canvas,- width,- height,- workspaces,- selected_workspace_id,- );- render_resize_affordance(&mut canvas, width, height);- canvas.into_bgrx()-}--fn render_waiting_state(canvas: &mut Canvas, desktop: LayoutRect) {- let center_x = desktop.x + desktop.width / 2.0;- let center_y = desktop.y + desktop.height / 2.0;- canvas.circle(center_x, center_y - 7.0, 22.0, Color::rgba(48, 59, 67, 220));- canvas.stroke_circle(- center_x,- center_y - 7.0,- 22.0,- 1.0,- Color::rgba(223, 230, 233, 88),- );- for offset in [-8.0, 0.0, 8.0] {- canvas.circle(- center_x + offset,- center_y - 7.0,- 2.1,- Color::rgba(235, 239, 241, 170),- );- }- canvas.rounded_rect(- Rect::new(center_x - 58.0, center_y + 25.0, 116.0, 3.0),- 1.5,- Color::rgba(224, 230, 232, 45),- );-}--fn render_target(canvas: &mut Canvas, frame: &PipFrame, rect: LayoutRect) {- let rect = Rect::from_layout(rect);- canvas.shadow(rect, 9.0, 6.0, Color::rgba(0, 0, 0, 118));- canvas.rounded_rect(rect.expand(1.0), 8.0, Color::rgba(220, 225, 227, 68));- canvas.rounded_rect(rect, 7.0, Color::rgb(21, 25, 28));- if !draw_frame_image(canvas, frame, rect) {- canvas.vertical_gradient_in(rect, Color::rgb(47, 55, 61), Color::rgb(27, 31, 35));- let accent = target_accent(frame.target.target_kind);- canvas.circle(- rect.x + rect.width / 2.0,- rect.y + rect.height / 2.0,- 12.0,- accent,- );- }- canvas.stroke_rounded_rect(rect, 7.0, 1.0, Color::rgba(225, 230, 232, 76));-}--#[cfg(target_os = "linux")]-fn draw_frame_image(canvas: &mut Canvas, frame: &PipFrame, rect: Rect) -> bool {- let Ok(image) = image::load_from_memory(&frame.png_bytes) else {- return false;- };- canvas.draw_image_contain(&image.to_rgba8(), rect, 6.5);- true-}--#[cfg(not(target_os = "linux"))]-fn draw_frame_image(_canvas: &mut Canvas, _frame: &PipFrame, _rect: Rect) -> bool {- false-}--fn session_tabs_for_bounds(- width: u16,- height: u16,- workspaces: &[PipWorkspaceSummary],- selected_workspace_id: Option<&str>,-) -> SessionTabsLayout {- let visible = workspaces.len().min(6);- let desired_width = visible as f64 * 190.0 + visible.saturating_sub(1) as f64 * 6.0;- let panel_width = desired_width.min((f64::from(width) - 16.0).max(1.0));- layout_session_tabs(- LayoutRect {- x: ((f64::from(width) - panel_width) / 2.0).max(0.0),- y: 4.0_f64.min(f64::from(height).max(0.0)),- width: panel_width,- height: 34.0,- },- workspaces,- selected_workspace_id,- )-}--fn render_session_selector(- canvas: &mut Canvas,- width: u16,- height: u16,- workspaces: &[PipWorkspaceSummary],- selected_workspace_id: Option<&str>,-) {- let tabs = session_tabs_for_bounds(width, height, workspaces, selected_workspace_id);- let Some(first) = tabs.tabs.first() else {- return;- };- let last = tabs.tabs.last().expect("session tabs are non-empty");- let dock = Rect::new(- first.rect.x - 6.0,- first.rect.y - 3.0,- last.rect.x + last.rect.width - first.rect.x + 12.0,- first.rect.height + 6.0,- );- canvas.shadow(dock, 12.0, 5.0, Color::rgba(0, 0, 0, 100));- canvas.rounded_rect(dock, dock.height * 0.27, Color::rgba(67, 70, 72, 195));- canvas.stroke_rounded_rect(- dock,- dock.height * 0.27,- 1.0,- Color::rgba(231, 235, 236, 66),- );- for (workspace, tab) in workspaces.iter().zip(&tabs.tabs) {- let icon = Rect::from_layout(tab.rect);- let accent = Color::rgb(tab.accent.0, tab.accent.1, tab.accent.2);- if tab.selected {- canvas.rounded_rect(- icon.expand(2.0),- icon.width * 0.27,- Color::rgba(245, 248, 249, 220),- );- }- canvas.rounded_rect(icon, icon.width * 0.23, accent);- let glyph = icon.inset(icon.width * 0.23);- canvas.circle(- glyph.x + glyph.width / 2.0,- glyph.y + glyph.height * 0.36,- glyph.width * 0.22,- Color::rgba(250, 252, 253, 235),- );- canvas.rounded_rect(- Rect::new(- glyph.x + glyph.width * 0.16,- glyph.y + glyph.height * 0.62,- glyph.width * 0.68,- glyph.height * 0.28,- ),- glyph.width * 0.14,- Color::rgba(250, 252, 253, 225),- );- let indicators = workspace.target_count.min(3);- for index in 0..indicators {- canvas.circle(- icon.x + icon.width / 2.0 + (index as f64 - (indicators - 1) as f64 / 2.0) * 5.0,- dock.y + dock.height - 4.0,- 1.4,- Color::rgba(242, 245, 246, if tab.selected { 245 } else { 145 }),- );- }- }-}--fn offset_layout_y(layout: &mut pip_preview::DesktopLayout, offset: f64) {- layout.desktop.y += offset;- layout.dock.y += offset;- for icon in &mut layout.dock_icons {- icon.y += offset;- }- for target in &mut layout.targets {- target.window.y += offset;- target.content.y += offset;- }-}--fn render_dash(canvas: &mut Canvas, frames: &[&PipFrame], dock: &LayoutRect, icons: &[LayoutRect]) {- let dock = Rect::from_layout(*dock);- canvas.shadow(dock, 12.0, 5.0, Color::rgba(0, 0, 0, 100));- canvas.rounded_rect(dock, dock.height * 0.27, Color::rgba(67, 70, 72, 195));- canvas.stroke_rounded_rect(- dock,- dock.height * 0.27,- 1.0,- Color::rgba(231, 235, 236, 66),- );- for (frame, icon) in frames.iter().zip(icons) {- let icon = Rect::from_layout(*icon);- let accent = target_accent(frame.target.target_kind);- canvas.rounded_rect(icon, icon.width * 0.23, accent);- let glyph = icon.inset(icon.width * 0.23);- match frame.target.target_kind {- PipTargetKind::BrowserTab => {- canvas.stroke_circle(- glyph.x + glyph.width / 2.0,- glyph.y + glyph.height / 2.0,- glyph.width * 0.44,- (glyph.width * 0.08).max(1.0),- Color::rgba(250, 252, 253, 235),- );- canvas.line(- glyph.x + glyph.width / 2.0,- glyph.y,- glyph.x + glyph.width / 2.0,- glyph.y + glyph.height,- (glyph.width * 0.07).max(1.0),- Color::rgba(250, 252, 253, 215),- );- }- PipTargetKind::NativeWindow => {- canvas.rounded_rect(glyph, glyph.width * 0.12, Color::rgba(250, 252, 253, 228));- canvas.rect(- Rect::new(- glyph.x + glyph.width * 0.12,- glyph.y + glyph.height * 0.25,- glyph.width * 0.76,- (glyph.height * 0.09).max(1.0),- ),- accent,- );- }- }- canvas.circle(- icon.x + icon.width / 2.0,- dock.y + dock.height - 4.0,- 1.6,- Color::rgba(242, 245, 246, 225),- );- }-}--fn render_resize_affordance(canvas: &mut Canvas, width: u16, height: u16) {- let x = f64::from(width) - 15.0;- let y = f64::from(height) - 8.0;- let color = Color::rgba(226, 231, 233, 116);- canvas.line(x, y, x + 7.0, y - 7.0, 1.2, color);- canvas.line(x + 5.0, y, x + 9.0, y - 4.0, 1.2, color);-}--fn target_accent(kind: PipTargetKind) -> Color {- match kind {- PipTargetKind::BrowserTab => Color::rgb(53, 154, 220),- PipTargetKind::NativeWindow => Color::rgb(239, 112, 54),- }-}--#[derive(Clone, Copy, Debug)]-struct Rect {- x: f64,- y: f64,- width: f64,- height: f64,-}--impl Rect {- fn new(x: f64, y: f64, width: f64, height: f64) -> Self {- Self {- x,- y,- width: width.max(0.0),- height: height.max(0.0),- }- }-- fn from_layout(rect: LayoutRect) -> Self {- Self::new(rect.x, rect.y, rect.width, rect.height)- }-- fn inset(self, amount: f64) -> Self {- Self::new(- self.x + amount,- self.y + amount,- self.width - amount * 2.0,- self.height - amount * 2.0,- )- }-- fn expand(self, amount: f64) -> Self {- Self::new(- self.x - amount,- self.y - amount,- self.width + amount * 2.0,- self.height + amount * 2.0,- )- }-}--#[derive(Clone, Copy, Debug, PartialEq, Eq)]-struct Color {- r: u8,- g: u8,- b: u8,- a: u8,-}--impl Color {- const fn rgb(r: u8, g: u8, b: u8) -> Self {- Self { r, g, b, a: 255 }- }-- const fn rgba(r: u8, g: u8, b: u8, a: u8) -> Self {- Self { r, g, b, a }- }-}--struct Canvas {- width: usize,- height: usize,- rgba: Vec<u8>,-}--impl Canvas {- fn new(width: u16, height: u16) -> Self {- Self {- width: usize::from(width),- height: usize::from(height),- rgba: vec![0; usize::from(width) * usize::from(height) * 4],- }- }-- fn into_bgrx(mut self) -> Vec<u8> {- for pixel in self.rgba.chunks_exact_mut(4) {- pixel.swap(0, 2);- pixel[3] = 0;- }- self.rgba- }-- fn blend(&mut self, x: i32, y: i32, color: Color) {- if x < 0 || y < 0 || x >= self.width as i32 || y >= self.height as i32 {- return;- }- let index = (y as usize * self.width + x as usize) * 4;- let alpha = u32::from(color.a);- let inverse = 255 - alpha;- self.rgba[index] =- ((u32::from(color.r) * alpha + u32::from(self.rgba[index]) * inverse) / 255) as u8;- self.rgba[index + 1] =- ((u32::from(color.g) * alpha + u32::from(self.rgba[index + 1]) * inverse) / 255) as u8;- self.rgba[index + 2] =- ((u32::from(color.b) * alpha + u32::from(self.rgba[index + 2]) * inverse) / 255) as u8;- self.rgba[index + 3] = 255;- }-- fn vertical_gradient(&mut self, top: Color, bottom: Color) {- self.vertical_gradient_in(- Rect::new(0.0, 0.0, self.width as f64, self.height as f64),- top,- bottom,- );- }-- fn vertical_gradient_in(&mut self, rect: Rect, top: Color, bottom: Color) {- let (x0, y0, x1, y1) = self.bounds(rect);- let height = (y1 - y0).max(1) as f64;- for y in y0..y1 {- let t = f64::from(y - y0) / height;- let color = lerp_color(top, bottom, t);- for x in x0..x1 {- self.blend(x, y, color);- }- }- }-- fn radial_glow(&mut self, cx: f64, cy: f64, radius: f64, color: Color) {- let rect = Rect::new(cx - radius, cy - radius, radius * 2.0, radius * 2.0);- let (x0, y0, x1, y1) = self.bounds(rect);- for y in y0..y1 {- for x in x0..x1 {- let distance = ((f64::from(x) - cx).powi(2) + (f64::from(y) - cy).powi(2)).sqrt();- if distance < radius {- let mut glow = color;- glow.a = (f64::from(color.a) * (1.0 - distance / radius).powi(2)) as u8;- self.blend(x, y, glow);- }- }- }- }-- fn rect(&mut self, rect: Rect, color: Color) {- let (x0, y0, x1, y1) = self.bounds(rect);- for y in y0..y1 {- for x in x0..x1 {- self.blend(x, y, color);- }- }- }-- fn rounded_rect(&mut self, rect: Rect, radius: f64, color: Color) {- let (x0, y0, x1, y1) = self.bounds(rect);- for y in y0..y1 {- for x in x0..x1 {- if rounded_contains(rect, radius, f64::from(x) + 0.5, f64::from(y) + 0.5) {- self.blend(x, y, color);- }- }- }- }-- fn stroke_rounded_rect(&mut self, rect: Rect, radius: f64, width: f64, color: Color) {- let inner = rect.inset(width);- let (x0, y0, x1, y1) = self.bounds(rect);- for y in y0..y1 {- for x in x0..x1 {- let px = f64::from(x) + 0.5;- let py = f64::from(y) + 0.5;- if rounded_contains(rect, radius, px, py)- && !rounded_contains(inner, (radius - width).max(0.0), px, py)- {- self.blend(x, y, color);- }- }- }- }-- fn shadow(&mut self, rect: Rect, radius: f64, y_offset: f64, color: Color) {- let steps = 6;- for step in (1..=steps).rev() {- let spread = radius * f64::from(step) / f64::from(steps);- let mut layer = color;- layer.a = color.a / (steps as u8 + 1);- self.rounded_rect(- Rect::new(- rect.x - spread,- rect.y - spread + y_offset,- rect.width + spread * 2.0,- rect.height + spread * 2.0,- ),- 7.0 + spread,- layer,- );- }- }-- fn circle(&mut self, cx: f64, cy: f64, radius: f64, color: Color) {- let (x0, y0, x1, y1) = self.bounds(Rect::new(- cx - radius,- cy - radius,- radius * 2.0,- radius * 2.0,- ));- let radius_squared = radius * radius;- for y in y0..y1 {- for x in x0..x1 {- if (f64::from(x) + 0.5 - cx).powi(2) + (f64::from(y) + 0.5 - cy).powi(2)- <= radius_squared- {- self.blend(x, y, color);- }- }- }- }-- fn stroke_circle(&mut self, cx: f64, cy: f64, radius: f64, width: f64, color: Color) {- let inner = (radius - width).max(0.0);- let (x0, y0, x1, y1) = self.bounds(Rect::new(- cx - radius,- cy - radius,- radius * 2.0,- radius * 2.0,- ));- for y in y0..y1 {- for x in x0..x1 {- let distance =- ((f64::from(x) + 0.5 - cx).powi(2) + (f64::from(y) + 0.5 - cy).powi(2)).sqrt();- if distance >= inner && distance <= radius {- self.blend(x, y, color);- }- }- }- }-- fn line(&mut self, x0: f64, y0: f64, x1: f64, y1: f64, width: f64, color: Color) {- let steps = ((x1 - x0).abs().max((y1 - y0).abs()).ceil() as usize).max(1);- for step in 0..=steps {- let t = step as f64 / steps as f64;- self.circle(x0 + (x1 - x0) * t, y0 + (y1 - y0) * t, width / 2.0, color);- }- }-- #[cfg(target_os = "linux")]- fn draw_image_contain(&mut self, image: &image::RgbaImage, rect: Rect, radius: f64) {- let target_width = rect.width.ceil().max(1.0) as u32;- let target_height = rect.height.ceil().max(1.0) as u32;- let scale = (target_width as f64 / image.width() as f64)- .min(target_height as f64 / image.height() as f64);- let scaled_width = (image.width() as f64 * scale).floor().max(1.0) as u32;- let scaled_height = (image.height() as f64 * scale).floor().max(1.0) as u32;- let scaled =- image::imageops::resize(image, scaled_width, scaled_height, FilterType::Triangle);- let image_rect = Rect::new(- rect.x + (rect.width - f64::from(scaled_width)) / 2.0,- rect.y + (rect.height - f64::from(scaled_height)) / 2.0,- f64::from(scaled_width),- f64::from(scaled_height),- );- let (x0, y0, x1, y1) = self.bounds(image_rect);- for y in y0..y1 {- for x in x0..x1 {- let px = f64::from(x) + 0.5;- let py = f64::from(y) + 0.5;- if !rounded_contains(rect, radius, px, py) {- continue;- }- let source_x = (f64::from(x) + 0.5 - image_rect.x).floor().max(0.0) as u32;- let source_y = (f64::from(y) + 0.5 - image_rect.y).floor().max(0.0) as u32;- if source_x < scaled.width() && source_y < scaled.height() {- let pixel = scaled.get_pixel(source_x, source_y);- self.blend(x, y, Color::rgba(pixel[0], pixel[1], pixel[2], pixel[3]));- }- }- }- }-- fn bounds(&self, rect: Rect) -> (i32, i32, i32, i32) {- (- rect.x.floor().max(0.0) as i32,- rect.y.floor().max(0.0) as i32,- (rect.x + rect.width).ceil().min(self.width as f64) as i32,- (rect.y + rect.height).ceil().min(self.height as f64) as i32,- )- }-}--fn rounded_contains(rect: Rect, radius: f64, x: f64, y: f64) -> bool {- if x < rect.x || y < rect.y || x >= rect.x + rect.width || y >= rect.y + rect.height {- return false;- }- let radius = radius.min(rect.width / 2.0).min(rect.height / 2.0).max(0.0);- // Equivalent half-width bounds can differ by a sub-ULP after arithmetic;- // max/min preserves the rounded geometry without f64::clamp panicking.- let nearest_x = x.max(rect.x + radius).min(rect.x + rect.width - radius);- let nearest_y = y.max(rect.y + radius).min(rect.y + rect.height - radius);- (x - nearest_x).powi(2) + (y - nearest_y).powi(2) <= radius * radius-}--fn lerp_color(from: Color, to: Color, t: f64) -> Color {- let channel = |a: u8, b: u8| (f64::from(a) + (f64::from(b) - f64::from(a)) * t) as u8;- Color::rgba(- channel(from.r, to.r),- channel(from.g, to.g),- channel(from.b, to.b),- channel(from.a, to.a),- )-}--#[cfg(test)]-mod tests {- use super::*;-- fn workspace(id: &str) -> PipWorkspaceSummary {- PipWorkspaceSummary {- workspace_id: id.to_owned(),- workspace_label: id.to_owned(),- target_count: 1,- updated_ms: 0,- }- }-- #[test]- fn renderer_emits_x11_bgrx_at_the_requested_size() {- let rendered = render_agent_view(360, 260, &[], &[], None, None);- assert_eq!(rendered.len(), 360 * 260 * 4);- assert!(rendered.chunks_exact(4).all(|pixel| pixel[3] == 0));- }-- #[test]- fn session_selector_is_hidden_for_zero_or_one_workspace() {- assert!(session_tabs_for_bounds(720, 520, &[], None).tabs.is_empty());- assert!(- session_tabs_for_bounds(720, 520, &[workspace("one")], Some("one"))- .tabs- .is_empty()- );- assert_eq!(- session_tabs_for_bounds(720, 520, &[workspace("one"), workspace("two")], Some("one"),)- .tabs- .len(),- 2- );- }-- #[test]- fn session_selector_hit_testing_only_accepts_session_tabs() {- let workspaces = [workspace("one"), workspace("two"), workspace("three")];- let tabs = session_tabs_for_bounds(720, 520, &workspaces, Some("one"));- let first = tabs.tabs[0].rect;- assert_eq!(- tabs.hit_test(first.x + first.width / 2.0, first.y + first.height / 2.0),- Some("one")- );- assert_eq!(- tabs.hit_test(first.x - 2.0, first.y + first.height / 2.0),- None- );- }-- #[cfg(target_os = "linux")]- #[test]- #[ignore = "requires a live pure-Wayland compositor with layer-shell"]- fn native_wayland_agent_view_starts_and_stops() {- assert!(std::env::var_os("WAYLAND_DISPLAY").is_some());- assert!(std::env::var_os("DISPLAY").is_none());- let backend = LinuxPipBackendFactory- .start(&PipConfig::default())- .expect("native Wayland Agent View should start");- backend.shutdown();- }-- #[test]- fn target_kind_uses_distinct_gnome_dash_accents() {- assert_ne!(- target_accent(PipTargetKind::NativeWindow),- target_accent(PipTargetKind::BrowserTab)- );- }-- #[test]- fn rounded_rect_clips_its_corner() {- let rect = Rect::new(10.0, 10.0, 40.0, 30.0);- assert!(!rounded_contains(rect, 8.0, 10.0, 10.0));- assert!(rounded_contains(rect, 8.0, 18.0, 10.5));- assert!(rounded_contains(rect, 8.0, 30.0, 25.0));- }-- #[test]- fn default_position_clamps_to_x11_coordinates() {- assert_eq!(clamp_i16(i32::MAX), i16::MAX);- assert_eq!(clamp_i16(i32::MIN), i16::MIN);- assert_eq!(clamp_i16(24), 24);- }-- #[cfg(target_os = "linux")]- #[test]- #[ignore = "requires a live X11 server (run under xvfb-run)"]- fn x11_input_transparency_toggle_changes_input_shape() -> anyhow::Result<()> {- use x11rb::connection::Connection;- use x11rb::protocol::shape::{ConnectionExt as ShapeConnectionExt, SK};- use x11rb::protocol::xproto::{ConnectionExt as _, CreateWindowAux, WindowClass};-- let (conn, screen_num) = x11rb::connect(None)?;- let screen = &conn.setup().roots[screen_num];- let window = conn.generate_id()?;- let width = 320;- let height = 240;- conn.create_window(- screen.root_depth,- window,- screen.root,- 0,- 0,- width,- height,- 0,- WindowClass::INPUT_OUTPUT,- screen.root_visual,- &CreateWindowAux::new(),- )?- .check()?;- conn.map_window(window)?.check()?;- conn.flush()?;-- set_x11_input_transparent(&conn, window, width, height, true)?;- let transparent = conn.shape_get_rectangles(window, SK::INPUT)?.reply()?;- assert!(transparent.rectangles.is_empty());-- set_x11_input_transparent(&conn, window, width, height, false)?;- let interactive = conn.shape_get_rectangles(window, SK::INPUT)?.reply()?;- assert_eq!(interactive.rectangles.len(), 1);- assert_eq!(interactive.rectangles[0].width, width);- assert_eq!(interactive.rectangles[0].height, height);-- conn.destroy_window(window)?.check()?;- conn.flush()?;- Ok(())- }-- #[cfg(target_os = "linux")]- #[test]- #[ignore = "requires an interactive X11 display"]- fn x11_visual_smoke_presents_multiple_targets() {- use std::io::Cursor;-- use image::{DynamicImage, ImageFormat, Rgba, RgbaImage};- use pip_preview::{PipGeometry, PipTarget};-- fn png(width: u32, height: u32, base: [u8; 3]) -> Vec<u8> {- let mut image = RgbaImage::new(width, height);- for (x, y, pixel) in image.enumerate_pixels_mut() {- let shade = ((x * 31 / width.max(1) + y * 23 / height.max(1)) % 44) as u8;- *pixel = Rgba([- base[0].saturating_add(shade),- base[1].saturating_add(shade),- base[2].saturating_add(shade),- 255,- ]);- }- let mut bytes = Cursor::new(Vec::new());- DynamicImage::ImageRgba8(image)- .write_to(&mut bytes, ImageFormat::Png)- .unwrap();- bytes.into_inner()- }-- let config = PipConfig {- enabled: true,- geometry: PipGeometry {- width: 720,- height: 520,- x: Some(1080),- y: Some(80),- },- title: "Cua Agent View Linux visual smoke".to_owned(),- };- let backend = LinuxPipBackendFactory.start(&config).unwrap();- for (index, (workspace_id, workspace_label, width, height, base, kind)) in [- (- "linux-browser",- "Browser session",- 1280,- 760,- [20, 72, 92],- PipTargetKind::BrowserTab,- ),- (- "linux-browser",- "Browser session",- 760,- 760,- [25, 91, 67],- PipTargetKind::BrowserTab,- ),- (- "linux-native",- "Native session",- 840,- 1060,- [47, 58, 65],- PipTargetKind::NativeWindow,- ),- (- "linux-native",- "Native session",- 1440,- 800,- [91, 59, 34],- PipTargetKind::NativeWindow,- ),- ]- .into_iter()- .enumerate()- {- backend.push_frame(PipFrame {- target: PipTarget {- workspace_id: workspace_id.to_owned(),- workspace_label: workspace_label.to_owned(),- target_id: format!("target-{index}"),- identity_key: format!("target-{index}"),- target_kind: kind,- target_label: format!("Target {index}"),- native_container: None,- },- png_bytes: png(width, height, base),- action_label: "observe".to_owned(),- timestamp_ms: index as u64,- cursor_position: None,- });- }- let seconds = std::env::var("CUA_AGENT_VIEW_SMOKE_SECONDS")- .ok()- .and_then(|value| value.parse().ok())- .unwrap_or(3);- std::thread::sleep(Duration::from_secs(seconds));- backend.shutdown();}}diff --git a/libs/cua-driver/rust/crates/platform-linux/src/pip/wayland.rs b/libs/cua-driver/rust/crates/platform-linux/src/pip/wayland.rsdeleted file mode 100644index 47271e9f4e..0000000000--- a/libs/cua-driver/rust/crates/platform-linux/src/pip/wayland.rs+++ /dev/null@@ -1,307 +0,0 @@-//! Native Wayland Agent View for compositors exposing layer-shell.--use std::collections::HashMap;-use std::sync::mpsc::{Receiver, SyncSender};-use std::time::Duration;--use pip_preview::{PipConfig, PipViewModel};-use wayland_client::{- protocol::{- wl_buffer::{self, WlBuffer},- wl_compositor::WlCompositor,- wl_region::WlRegion,- wl_registry,- wl_shm::{Format, WlShm},- wl_shm_pool::WlShmPool,- wl_surface::WlSurface,- },- Connection, Dispatch, Proxy, QueueHandle,-};-use wayland_protocols_wlr::layer_shell::v1::client::{- zwlr_layer_shell_v1::{Layer, ZwlrLayerShellV1},- zwlr_layer_surface_v1::{self, Anchor, KeyboardInteractivity, ZwlrLayerSurfaceV1},-};--use super::{render_agent_view, UiMessage};--struct State {- compositor: Option<WlCompositor>,- shm: Option<WlShm>,- layer_shell: Option<ZwlrLayerShellV1>,- surface: Option<WlSurface>,- layer_surface: Option<ZwlrLayerSurfaceV1>,- configured: bool,- width: u32,- height: u32,- pending_buffers: HashMap<u32, (*mut libc::c_void, usize, i32)>,-}--unsafe impl Send for State {}--impl State {- fn new(width: u32, height: u32) -> Self {- Self {- compositor: None,- shm: None,- layer_shell: None,- surface: None,- layer_surface: None,- configured: false,- width,- height,- pending_buffers: HashMap::new(),- }- }-}--pub(super) fn run_window(- cfg: PipConfig,- rx: Receiver<UiMessage>,- ready: SyncSender<anyhow::Result<()>>,-) {- if let Err(error) = run(cfg, rx, &ready) {- let _ = ready.send(Err(error));- }-}--fn run(- cfg: PipConfig,- rx: Receiver<UiMessage>,- ready: &SyncSender<anyhow::Result<()>>,-) -> anyhow::Result<()> {- let connection = Connection::connect_to_env()?;- let mut queue = connection.new_event_queue::<State>();- let qh = queue.handle();- let _registry = connection.display().get_registry(&qh, ());- let mut state = State::new(cfg.geometry.width.max(360), cfg.geometry.height.max(260));- queue.roundtrip(&mut state)?;-- let compositor = state- .compositor- .clone()- .ok_or_else(|| anyhow::anyhow!("Wayland compositor does not expose wl_compositor"))?;- let layer_shell = state.layer_shell.clone().ok_or_else(|| {- anyhow::anyhow!(- "native Wayland Agent View requires zwlr_layer_shell_v1; use XWayland on this compositor"- )- })?;- let surface = compositor.create_surface(&qh, ());- let layer_surface = layer_shell.get_layer_surface(- &surface,- None,- Layer::Overlay,- "cua-agent-view".to_owned(),- &qh,- (),- );- layer_surface.set_anchor(Anchor::Top | Anchor::Right);- layer_surface.set_size(state.width, state.height);- layer_surface.set_margin(24, 24, 24, 24);- layer_surface.set_exclusive_zone(-1);- layer_surface.set_keyboard_interactivity(KeyboardInteractivity::None);- // Layer-shell cannot provide a normal draggable/resizable window contract.- // Keep the presentation click-through so it can never block automation;- // session selection continues to follow MRU on this native-only path.- let input_region = compositor.create_region(&qh, ());- surface.set_input_region(Some(&input_region));- input_region.destroy();- surface.commit();- state.surface = Some(surface);- state.layer_surface = Some(layer_surface);-- for _ in 0..8 {- queue.roundtrip(&mut state)?;- if state.configured {- break;- }- }- anyhow::ensure!(state.configured, "Wayland Agent View was never configured");- ready- .send(Ok(()))- .map_err(|_| anyhow::anyhow!("Agent View startup receiver was dropped"))?;-- let mut model = PipViewModel::new(12);- let mut running = true;- let mut dirty = true;- while running {- match rx.recv_timeout(Duration::from_millis(16)) {- Ok(UiMessage::Frame(frame)) => {- model.upsert(frame);- dirty = true;- }- Ok(UiMessage::RemoveTarget(workspace, target)) => {- dirty |= model.remove_target(&workspace, &target);- }- Ok(UiMessage::RemoveWorkspace(workspace)) => {- dirty |= !model.remove_workspace(&workspace).is_empty();- }- Ok(UiMessage::SetInputTransparent(_, reply)) => {- // Layer-shell presentation does not cover the automated target- // at its top-right inset, so no compositor input-region swap is- // required for injected actions.- let _ = reply.send(Ok(()));- }- Ok(UiMessage::Shutdown) => running = false,- Err(std::sync::mpsc::RecvTimeoutError::Timeout) => {}- Err(std::sync::mpsc::RecvTimeoutError::Disconnected) => break,- }- if dirty && running {- redraw(&mut state, &model, &qh)?;- dirty = false;- }- queue.roundtrip(&mut state)?;- }- Ok(())-}--fn redraw(state: &mut State, model: &PipViewModel, qh: &QueueHandle<State>) -> anyhow::Result<()> {- let surface = state- .surface- .as_ref()- .ok_or_else(|| anyhow::anyhow!("Wayland Agent View surface is unavailable"))?;- let shm = state- .shm- .as_ref()- .ok_or_else(|| anyhow::anyhow!("Wayland compositor does not expose wl_shm"))?;- let width = state.width.max(1).min(u16::MAX as u32) as u16;- let height = state.height.max(1).min(u16::MAX as u32) as u16;- let frames = model.selected_frames();- let workspaces = model.workspaces();- let pixels = render_agent_view(- width,- height,- &frames,- &workspaces,- model.selected_workspace_id(),- model.active_view_id(),- );- let stride = i32::from(width) * 4;- let size = pixels.len();- let (fd, ptr) = crate::wayland::anon_shm(size)?;- unsafe { std::ptr::copy_nonoverlapping(pixels.as_ptr(), ptr.cast::<u8>(), size) };- use std::os::fd::AsFd as _;- let pool_fd = unsafe { crate::wayland::borrowed_fd(fd) };- let pool = shm.create_pool(pool_fd.as_fd(), size as i32, qh, ());- let buffer = pool.create_buffer(- 0,- i32::from(width),- i32::from(height),- stride,- Format::Xrgb8888,- qh,- (),- );- pool.destroy();- state- .pending_buffers- .insert(buffer.id().protocol_id(), (ptr, size, fd));- surface.attach(Some(&buffer), 0, 0);- surface.damage_buffer(0, 0, i32::from(width), i32::from(height));- surface.commit();- Ok(())-}--impl Dispatch<wl_registry::WlRegistry, ()> for State {- fn event(- state: &mut Self,- registry: &wl_registry::WlRegistry,- event: wl_registry::Event,- _: &(),- _: &Connection,- qh: &QueueHandle<Self>,- ) {- if let wl_registry::Event::Global {- name,- interface,- version,- } = event- {- match interface.as_str() {- "wl_compositor" => {- state.compositor =- Some(registry.bind::<WlCompositor, _, _>(name, version.min(6), qh, ()))- }- "wl_shm" => {- state.shm = Some(registry.bind::<WlShm, _, _>(name, version.min(1), qh, ()))- }- "zwlr_layer_shell_v1" => {- state.layer_shell =- Some(registry.bind::<ZwlrLayerShellV1, _, _>(name, version.min(4), qh, ()))- }- _ => {}- }- }- }-}--impl Dispatch<ZwlrLayerSurfaceV1, ()> for State {- fn event(- state: &mut Self,- layer: &ZwlrLayerSurfaceV1,- event: zwlr_layer_surface_v1::Event,- _: &(),- _: &Connection,- _: &QueueHandle<Self>,- ) {- match event {- zwlr_layer_surface_v1::Event::Configure {- serial,- width,- height,- } => {- layer.ack_configure(serial);- if width > 0 {- state.width = width;- }- if height > 0 {- state.height = height;- }- state.configured = true;- }- zwlr_layer_surface_v1::Event::Closed => state.surface = None,- _ => {}- }- }-}--impl Dispatch<WlBuffer, ()> for State {- fn event(- state: &mut Self,- buffer: &WlBuffer,- event: wl_buffer::Event,- _: &(),- _: &Connection,- _: &QueueHandle<Self>,- ) {- if matches!(event, wl_buffer::Event::Release) {- if let Some((ptr, size, fd)) = state.pending_buffers.remove(&buffer.id().protocol_id())- {- crate::wayland::cleanup_mmap(ptr, size, fd);- }- buffer.destroy();- }- }-}--macro_rules! empty_dispatch {- ($ty:ty) => {- impl Dispatch<$ty, ()> for State {- fn event(- _: &mut Self,- _: &$ty,- _: <$ty as Proxy>::Event,- _: &(),- _: &Connection,- _: &QueueHandle<Self>,- ) {- }- }- };-}--empty_dispatch!(WlCompositor);-empty_dispatch!(WlShm);-empty_dispatch!(WlShmPool);-empty_dispatch!(WlSurface);-empty_dispatch!(WlRegion);-empty_dispatch!(ZwlrLayerShellV1);diff --git a/libs/cua-driver/rust/crates/platform-linux/src/tools/impl_.rs b/libs/cua-driver/rust/crates/platform-linux/src/tools/impl_.rsindex 9b509284a6..a148640ac3 100644--- a/libs/cua-driver/rust/crates/platform-linux/src/tools/impl_.rs+++ b/libs/cua-driver/rust/crates/platform-linux/src/tools/impl_.rs@@ -7114,16 +7114,15 @@ impl Tool for GetConfigTool {}async fn invoke(&self, _args: Value) -> ToolResult {let cfg = self.state.config.read().unwrap();- let (agent_view_enabled, agent_view_geometry) =- pip_preview::read_agent_view_keys_from_file();+ let (pip_enabled, pip_geometry) = pip_preview::read_pip_keys_from_file();ToolResult::text("cua-driver-rs configuration").with_structured(json!({"version": env!("CARGO_PKG_VERSION"),"source_sha": option_env!("CUA_DRIVER_SOURCE_SHA"),"platform": "linux","capture_mode": cfg.capture_mode,"max_image_dimension": cfg.max_image_dimension,- "agent_view": agent_view_enabled,- "agent_view_geometry": agent_view_geometry+ "experimental_pip": pip_enabled,+ "experimental_pip_geometry": pip_geometry}))}}@@ -7146,15 +7145,16 @@ impl Tool for SetConfigTool {- **{key, value}** (preferred): `{\"key\": \"max_image_dimension\", \"value\": 800}` \— single leaf write.\n\- **Legacy per-field**: `{\"capture_mode\": \"som\", \"max_image_dimension\": 0}`.\n\n\- The agent_view keys persist to ~/.cua-driver/config.json and apply on next \- daemon restart (Agent View is initialised once at startup).".into(),+ The experimental_pip keys persist to ~/.cua-driver/config.json and apply on next \+ daemon restart (the PiP backend is initialised once at startup; \+ Linux ships only the trait stub today — see issue #1729).".into(),input_schema: json!({"type":"object","properties":{"key":{"type":"string","description":"Name of a single config field to write ({key, value} shape). Pair with `value`."},"value":{"description":"New value for `key`. JSON type depends on the key."},"capture_mode":{"type":"string","enum":["ax","vision"],"description":"Legacy per-field shape. Default capture mode for get_window_state. (\"som\"/\"screenshot\" still decode as deprecated aliases.)"},"max_image_dimension":{"type":"integer","description":"Legacy per-field shape. Max dimension for screenshot resizing (0 = no limit)."},- "agent_view":{"type":"boolean","description":"Enable the multi-target Agent View (applies next restart)."},- "agent_view_geometry":{"type":"string","description":"Agent View size + optional position in `WxH` or `WxH+X+Y` form."}+ "experimental_pip":{"type":"boolean","description":"Enable the experimental PiP preview window (applies next restart; Linux backend stubbed)."},+ "experimental_pip_geometry":{"type":"string","description":"PiP window size + optional position in `WxH` or `WxH+X+Y` form."}},"additionalProperties":false}),read_only: false, destructive: false, idempotent: true, open_world: false,})@@ -7203,31 +7203,31 @@ impl Tool for SetConfigTool {}None => return ToolResult::error(format!("`max_image_dimension` must be an integer, got {val}.")),},- "agent_view" => match val.as_bool() {+ "experimental_pip" => match val.as_bool() {Some(b) => {- if let Err(e) = pip_preview::write_config_key("agent_view", Value::Bool(b)) {- return ToolResult::error(format!("failed to persist agent_view: {e}"));+ if let Err(e) = pip_preview::write_config_key("experimental_pip", Value::Bool(b)) {+ return ToolResult::error(format!("failed to persist experimental_pip: {e}"));}- parts.push(format!("agent_view={b} (next restart)"));+ parts.push(format!("experimental_pip={b} (next restart)"));}- None => return ToolResult::error(format!("`agent_view` must be a boolean, got {val}.")),+ None => return ToolResult::error(format!("`experimental_pip` must be a boolean, got {val}.")),},- "agent_view_geometry" => match val.as_str() {+ "experimental_pip_geometry" => match val.as_str() {Some(s) => {if pip_preview::PipGeometry::parse(s).is_none() {return ToolResult::error(format!(- "agent_view_geometry `{s}` is not a valid WxH or WxH+X+Y string"+ "experimental_pip_geometry `{s}` is not a valid WxH or WxH+X+Y string"));}- if let Err(e) = pip_preview::write_config_key("agent_view_geometry", Value::String(s.to_owned())) {- return ToolResult::error(format!("failed to persist agent_view_geometry: {e}"));+ if let Err(e) = pip_preview::write_config_key("experimental_pip_geometry", Value::String(s.to_owned())) {+ return ToolResult::error(format!("failed to persist experimental_pip_geometry: {e}"));}- parts.push(format!("agent_view_geometry={s} (next restart)"));+ parts.push(format!("experimental_pip_geometry={s} (next restart)"));}- None => return ToolResult::error(format!("`agent_view_geometry` must be a string, got {val}.")),+ None => return ToolResult::error(format!("`experimental_pip_geometry` must be a string, got {val}.")),},other => return ToolResult::error(format!(- "Unknown config key `{other}`. Known: capture_mode, max_image_dimension, agent_view, agent_view_geometry."+ "Unknown config key `{other}`. Known: capture_mode, max_image_dimension, experimental_pip, experimental_pip_geometry.")),}}@@ -7248,37 +7248,40 @@ impl Tool for SetConfigTool {}parts.push(format!("max_image_dimension={dim}"));}- if let Some(enabled) = args.get("agent_view").and_then(|v| v.as_bool()) {- if let Err(e) = pip_preview::write_config_key("agent_view", Value::Bool(enabled)) {- return ToolResult::error(format!("failed to persist agent_view: {e}"));+ if let Some(enabled) = args.get("experimental_pip").and_then(|v| v.as_bool()) {+ if let Err(e) = pip_preview::write_config_key("experimental_pip", Value::Bool(enabled))+ {+ return ToolResult::error(format!("failed to persist experimental_pip: {e}"));}- parts.push(format!("agent_view={enabled} (next restart)"));+ parts.push(format!("experimental_pip={enabled} (next restart)"));}- if let Some(geom) = args.opt_str("agent_view_geometry") {+ if let Some(geom) = args.opt_str("experimental_pip_geometry") {if pip_preview::PipGeometry::parse(&geom).is_none() {return ToolResult::error(format!(- "agent_view_geometry `{geom}` is not a valid WxH or WxH+X+Y string"+ "experimental_pip_geometry `{geom}` is not a valid WxH or WxH+X+Y string"));}- if let Err(e) =- pip_preview::write_config_key("agent_view_geometry", Value::String(geom.clone()))- {- return ToolResult::error(format!("failed to persist agent_view_geometry: {e}"));+ if let Err(e) = pip_preview::write_config_key(+ "experimental_pip_geometry",+ Value::String(geom.clone()),+ ) {+ return ToolResult::error(format!(+ "failed to persist experimental_pip_geometry: {e}"+ ));}- parts.push(format!("agent_view_geometry={geom} (next restart)"));+ parts.push(format!("experimental_pip_geometry={geom} (next restart)"));}let msg = if parts.is_empty() {"Config unchanged (no known parameters).".to_owned()} else {format!("Config updated: {}", parts.join(", "))};- let (agent_view_enabled, agent_view_geometry) =- pip_preview::read_agent_view_keys_from_file();+ let (pip_enabled, pip_geometry) = pip_preview::read_pip_keys_from_file();ToolResult::text(msg).with_structured(json!({"capture_mode": cfg.capture_mode,"max_image_dimension": cfg.max_image_dimension,- "agent_view": agent_view_enabled,- "agent_view_geometry": agent_view_geometry+ "experimental_pip": pip_enabled,+ "experimental_pip_geometry": pip_geometry}))}}diff --git a/libs/cua-driver/rust/crates/platform-macos/src/pip/mod.rs b/libs/cua-driver/rust/crates/platform-macos/src/pip/mod.rsindex 4bb349e555..bf873a403f 100644--- a/libs/cua-driver/rust/crates/platform-macos/src/pip/mod.rs+++ b/libs/cua-driver/rust/crates/platform-macos/src/pip/mod.rs@@ -1,61 +1,90 @@-//! macOS multi-target Agent View.+//! macOS picture-in-picture preview window.//!-//! A floating, resizable miniature desktop that uses the host wallpaper and-//! presents exact native windows and browser tabs as aspect-aware macOS-like-//! windows. Existing lifecycle sessions group presentation only; Agent View-//! never claims, moves, resizes, or closes the underlying targets.--use std::ffi::{c_void, CStr};-use std::sync::atomic::{AtomicBool, Ordering};-use std::sync::{Arc, Mutex, OnceLock};--use pip_preview::{- layout_desktop, layout_session_tabs, png_dimensions, LayoutRect, PipBackend, PipBackendFactory,- PipConfig, PipFrame, PipTargetKind, PipViewModel, PipWorkspaceSummary, SessionTabsLayout,- TargetSize,-};+//! Floating NSWindow with an NSImageView showing the most recent+//! post-action screenshot and an NSTextField with a one-line label+//! describing the action that produced it.+//!+//! ## Threading model+//!+//! Mirrors `cursor/overlay.rs`:+//!+//! - The MCP/tokio server runs on a background thread.+//! - AppKit MUST run on the main thread, which `cua-driver/src/main.rs`+//! parks in `NSApplication.run()` for the cursor overlay.+//! - `push_frame()` is called from arbitrary tokio tasks. It packages+//! the frame into a heap-allocated `Box` and posts the actual UI+//! update onto the main queue via `dispatch_async_f`. The block+//! then constructs an `NSImage` from the PNG bytes and calls+//! `[imageView setImage:]` + `[label setStringValue:]`.+//!+//! ## Window properties+//!+//! - `NSWindowCollectionBehaviorCanJoinAllSpaces | FullScreenAuxiliary |+//! Stationary | Transient | IgnoresCycle`+//! - `level = .floating` (kCGFloatingWindowLevel, between normal apps+//! and dock; high enough to stay visible, low enough not to obscure+//! menus or accessibility overlays).+//! - `setIgnoresMouseEvents(false)` — user can click the red close+//! button. Backend cleanup happens on `shutdown()`; closing the+//! window manually decouples it from the session as the spec+//! requires.+//! - No activation: `setHidesOnDeactivate(false)` and+//! `setBecomesKeyOnlyIfNeeded(true)` so the window never steals+//! keyboard focus from the user's frontmost app.+//!+//! ## Init lifecycle+//!+//! Because the cursor overlay already owns the main thread when+//! enabled, `MacosPipBackend::start` cannot block on it. Instead it+//! posts the window-creation block onto the main queue and returns+//! immediately. The first frame may arrive before the window exists;+//! that's fine — the push path reads the window pointer from a+//! `Mutex<Option<usize>>` and silently no-ops until init finishes.++use std::ffi::c_void;+use std::sync::Mutex;++use pip_preview::{PipBackend, PipBackendFactory, PipConfig, PipFrame};++// ── CGColor objc2 encoding shim ────────────────────────────────────────────+//+// `[NSColor CGColor]` returns a `CGColorRef` whose Objective-C type encoding+// is `^{CGColor=}`. objc2's strict msg_send! enforcement rejects bare+// `*mut c_void` (`^v`) for both sides of that call. Declare a phantom+// struct with the matching encoding so we can typed-cast through it+// without pulling in a wider CGColor binding crate.#[repr(C)]struct CGColor {_opaque: [u8; 0],}+// RefEncode supplies an automatic Encode impl for `*mut CGColor` /+// `*const CGColor` via objc2's blanket — that's the route msg_send! needs+// for both setting layer.backgroundColor and reading [NSColor CGColor].+// `ENCODING_REF` is the encoding for one level of indirection, so the+// pointer wrap goes here (objc encoding `^{CGColor=}`).unsafe impl objc2::RefEncode for CGColor {const ENCODING_REF: objc2::Encoding =objc2::Encoding::Pointer(&objc2::Encoding::Struct("CGColor", &[]));}+// ── Native AppKit pointer cell ─────────────────────────────────────────────+//+// Window, image view, and label pointers are stashed as `usize` so+// `Send` works (raw `*mut AnyObject` is `!Send`). The actual deref ++// `msg_send!` happens only on the main queue inside the dispatched+// block, so there is no thread-safety hazard from the Send promise.+struct NativeHandles {window: usize,- canvas_view: usize,- shell_wallpaper_container: usize,- shell_wallpaper_view: usize,- wallpaper_container: usize,- wallpaper_view: usize,- delegate: usize,+ image_view: usize,+ label: usize,}-/// Outer corner radius of the Agent View container chrome.-const CONTAINER_RADIUS: f64 = 15.0;-const PIP_CORNER_RADIUS: f64 = 12.0;-/// The miniature desktop sits inside an asymmetric hardware-like glass shell.-const SHELL_SIDE_INSET: f64 = 9.0;-const SHELL_TOP_INSET: f64 = 20.0;-const SHELL_BOTTOM_INSET: f64 = 10.0;-const SESSION_SELECTOR_HEIGHT: f64 = 34.0;-const RESIZE_HIT_INSET: f64 = 7.0;--// Keep the PiP shell visually clean for now; invisible hit zones still provide-// dragging and resizing without exposing a native or custom frame.-const SHOW_SHELL_BORDER: bool = false;--const RESIZE_LEFT: isize = 1;-const RESIZE_RIGHT: isize = 2;-const RESIZE_BOTTOM: isize = 4;-const RESIZE_TOP: isize = 8;-static HANDLES: Mutex<Option<NativeHandles>> = Mutex::new(None);-static VIEW_MODEL: Mutex<Option<PipViewModel>> = Mutex::new(None);++// ── libdispatch glue — same shape as cursor::overlay ──────────────────────#[link(name = "dispatch", kind = "dylib")]extern "C" {@@ -65,11 +94,6 @@ extern "C" {context: *mut c_void,work: unsafe extern "C" fn(*mut c_void),);- fn dispatch_sync_f(- queue: *const c_void,- context: *mut c_void,- work: unsafe extern "C" fn(*mut c_void),- );}fn dispatch_to_main<T: Send + 'static>(payload: T, cb: unsafe extern "C" fn(*mut c_void)) {@@ -80,1082 +104,74 @@ fn dispatch_to_main<T: Send + 'static>(payload: T, cb: unsafe extern "C" fn(*mut}}-fn dispatch_to_main_sync<T>(payload: T, cb: unsafe extern "C" fn(*mut c_void)) {- let context = Box::into_raw(Box::new(payload)) as *mut c_void;- unsafe {- if libc::pthread_main_np() != 0 {- cb(context);- } else {- let main_queue = &raw const _dispatch_main_q as *const c_void;- dispatch_sync_f(main_queue, context, cb);- }- }-}+// ── Backend impl ──────────────────────────────────────────────────────────pub struct MacosPipBackend;-struct InputPassthroughRequest {- passthrough: bool,- applied: Arc<AtomicBool>,-}-impl PipBackend for MacosPipBackend {fn push_frame(&self, frame: PipFrame) {- // init_cb was queued first on the serial main queue, so retaining the- // frame here avoids dropping exact activity during daemon startup.+ // No window yet? Drop the frame silently — start() dispatches+ // the create block onto the main queue and the very first+ // tool call can race that block.+ if HANDLES.lock().unwrap().is_none() {+ return;+ }dispatch_to_main(frame, push_frame_cb);}- fn remove_workspace(&self, workspace_id: &str) {- dispatch_to_main(workspace_id.to_owned(), remove_workspace_cb);- }-- fn remove_target(&self, workspace_id: &str, identity_key: &str) {- dispatch_to_main(- (workspace_id.to_owned(), identity_key.to_owned()),- remove_target_cb,- );- }-- fn set_input_passthrough(&self, passthrough: bool) -> anyhow::Result<()> {- let applied = Arc::new(AtomicBool::new(false));- dispatch_to_main_sync(- InputPassthroughRequest {- passthrough,- applied: Arc::clone(&applied),- },- set_input_passthrough_cb,- );- // The user may close the optional presentation window while the- // daemon continues serving tools; that must never break input.- let _ = applied.load(Ordering::Acquire);- Ok(())- }-fn shutdown(self: Box<Self>) {dispatch_to_main((), shutdown_cb);}}-unsafe extern "C" fn set_input_passthrough_cb(ctx: *mut c_void) {- use objc2::msg_send;- use objc2::runtime::AnyObject;-- let request: InputPassthroughRequest = *Box::from_raw(ctx as *mut InputPassthroughRequest);- let handles = HANDLES.lock().unwrap();- if let Some(handles) = handles.as_ref() {- let window = handles.window as *mut AnyObject;- let _: () = msg_send![window, setIgnoresMouseEvents: request.passthrough];- request.applied.store(true, Ordering::Release);- }-}-unsafe extern "C" fn push_frame_cb(ctx: *mut c_void) {- let frame: PipFrame = *Box::from_raw(ctx as *mut PipFrame);- let snapshot = {- let mut model = VIEW_MODEL.lock().unwrap();- let model = model.get_or_insert_with(|| PipViewModel::new(12));- model.upsert(frame);- clone_snapshot(model)- };- render_snapshot(&snapshot);-}--unsafe extern "C" fn remove_workspace_cb(ctx: *mut c_void) {- let workspace_id: String = *Box::from_raw(ctx as *mut String);- let snapshot = {- let mut model = VIEW_MODEL.lock().unwrap();- let model = model.get_or_insert_with(|| PipViewModel::new(12));- model.remove_workspace(&workspace_id);- clone_snapshot(model)- };- render_snapshot(&snapshot);-}--unsafe extern "C" fn remove_target_cb(ctx: *mut c_void) {- let (workspace_id, identity_key): (String, String) =- *Box::from_raw(ctx as *mut (String, String));- let snapshot = {- let mut model = VIEW_MODEL.lock().unwrap();- let model = model.get_or_insert_with(|| PipViewModel::new(12));- model.remove_target(&workspace_id, &identity_key);- clone_snapshot(model)- };- render_snapshot(&snapshot);-}--struct ViewSnapshot {- frames: Vec<PipFrame>,- workspaces: Vec<PipWorkspaceSummary>,- selected_workspace_id: Option<String>,- active_view_id: Option<String>,-}--fn clone_snapshot(model: &PipViewModel) -> ViewSnapshot {- ViewSnapshot {- frames: model.selected_frames().into_iter().cloned().collect(),- workspaces: model.workspaces(),- selected_workspace_id: model.selected_workspace_id().map(str::to_owned),- active_view_id: model.active_view_id().map(str::to_owned),- }-}--fn current_snapshot() -> ViewSnapshot {- VIEW_MODEL- .lock()- .unwrap()- .as_ref()- .map(clone_snapshot)- .unwrap_or_else(|| ViewSnapshot {- frames: Vec::new(),- workspaces: Vec::new(),- selected_workspace_id: None,- active_view_id: None,- })-}--unsafe fn ns_string(value: &str) -> *mut objc2::runtime::AnyObject {- use objc2::{class, msg_send};-- let sanitized = value.replace('\0', " ");- let Ok(cstr) = std::ffi::CString::new(sanitized) else {- return std::ptr::null_mut();- };- msg_send![- class!(NSString),- stringWithUTF8String: cstr.as_ptr() as *const u8- ]-}--unsafe fn rust_string(value: *mut objc2::runtime::AnyObject) -> Option<String> {- use objc2::msg_send;-- if value.is_null() {- return None;- }- let utf8: *const std::os::raw::c_char = msg_send![value, UTF8String];- (!utf8.is_null()).then(|| CStr::from_ptr(utf8).to_string_lossy().into_owned())-}--#[derive(Clone, Copy)]-enum LabelTone {- Muted,- Dark,-}--unsafe fn color(red: f64, green: f64, blue: f64, alpha: f64) -> *mut objc2::runtime::AnyObject {- use objc2::{class, msg_send};-- msg_send![- class!(NSColor),- colorWithCalibratedRed: red- green: green- blue: blue- alpha: alpha- ]-}--unsafe fn set_layer_background(- layer: *mut objc2::runtime::AnyObject,- background: *mut objc2::runtime::AnyObject,-) {- use objc2::msg_send;-- let cg_color: *mut CGColor = msg_send![background, CGColor];- let _: () = msg_send![layer, setBackgroundColor: cg_color];-}--/// Opt a container layer into the macOS squircle corner curve so the chrome-/// reads as continuous rather than as a circular-arc rectangle.-unsafe fn set_continuous_corners(layer: *mut objc2::runtime::AnyObject) {- use objc2::msg_send;-- let responds: bool = msg_send![layer, respondsToSelector: objc2::sel!(setCornerCurve:)];- if !responds {- return;- }- let curve = ns_string("continuous");- if curve.is_null() {- return;- }- let _: () = msg_send![layer, setCornerCurve: curve];-}--unsafe fn set_layer_border(- layer: *mut objc2::runtime::AnyObject,- width: f64,- border: *mut objc2::runtime::AnyObject,-) {- use objc2::msg_send;-- let cg_color: *mut CGColor = msg_send![border, CGColor];- let _: () = msg_send![layer, setBorderWidth: width];- let _: () = msg_send![layer, setBorderColor: cg_color];-}--unsafe fn add_text_label(- parent: *mut objc2::runtime::AnyObject,- frame: objc2_foundation::NSRect,- text: &str,- font_size: f64,- bold: bool,- tone: LabelTone,- alignment: isize,-) {- use objc2::runtime::AnyObject;- use objc2::{class, msg_send};-- let label: *mut AnyObject = {- let alloc: *mut AnyObject = msg_send![class!(NSTextField), alloc];- msg_send![alloc, initWithFrame: frame]- };- let _: () = msg_send![label, setBezeled: false];- let _: () = msg_send![label, setDrawsBackground: false];- let _: () = msg_send![label, setEditable: false];- let _: () = msg_send![label, setSelectable: false];- let _: () = msg_send![label, setAlignment: alignment];- let text_color = match tone {- LabelTone::Muted => color(1.0, 1.0, 1.0, 0.66),- LabelTone::Dark => color(0.12, 0.13, 0.15, 0.94),- };- let _: () = msg_send![label, setTextColor: text_color];- let font: *mut AnyObject = if bold {- msg_send![class!(NSFont), boldSystemFontOfSize: font_size]- } else {- msg_send![class!(NSFont), systemFontOfSize: font_size]- };- let _: () = msg_send![label, setFont: font];- let value = ns_string(text);- if !value.is_null() {- let _: () = msg_send![label, setStringValue: value];- }- let _: () = msg_send![parent, addSubview: label];-}--unsafe fn rounded_view(- frame: objc2_foundation::NSRect,- radius: f64,- background: *mut objc2::runtime::AnyObject,- clips: bool,-) -> *mut objc2::runtime::AnyObject {- use objc2::runtime::AnyObject;- use objc2::{class, msg_send};-- let view: *mut AnyObject = {- let alloc: *mut AnyObject = msg_send![class!(NSView), alloc];- msg_send![alloc, initWithFrame: frame]- };- let _: () = msg_send![view, setWantsLayer: true];- let layer: *mut AnyObject = msg_send![view, layer];- let _: () = msg_send![layer, setCornerRadius: radius];- let _: () = msg_send![layer, setMasksToBounds: clips];- set_layer_background(layer, background);- view-}--/// A layer-hosting view whose backing layer is a vertical `CAGradientLayer`.-///-/// The chrome body needs a top-to-bottom falloff to read as a lit glass-/// surface rather than as a flat outline. AppKit keeps a view-assigned layer-/// sized to its view, so this survives live resize without a manual pass.-/// Layer-hosting views must not take subviews; overlay strips go on the-/// enclosing chrome view instead.-unsafe fn gradient_view(- frame: objc2_foundation::NSRect,- radius: f64,- top: *mut objc2::runtime::AnyObject,- bottom: *mut objc2::runtime::AnyObject,-) -> *mut objc2::runtime::AnyObject {- use objc2::runtime::AnyObject;- use objc2::{class, msg_send};-- let view: *mut AnyObject = {- let alloc: *mut AnyObject = msg_send![class!(NSView), alloc];- msg_send![alloc, initWithFrame: frame]- };- let layer: *mut AnyObject = {- let alloc: *mut AnyObject = msg_send![class!(CAGradientLayer), alloc];- msg_send![alloc, init]- };- let top_cg: *mut CGColor = msg_send![top, CGColor];- let bottom_cg: *mut CGColor = msg_send![bottom, CGColor];- let stops: [*mut CGColor; 2] = [top_cg, bottom_cg];- let colors: *mut AnyObject = msg_send![- class!(NSArray),- arrayWithObjects: stops.as_ptr() as *const *mut AnyObject- count: 2usize- ];- let _: () = msg_send![layer, setColors: colors];- let _: () = msg_send![layer, setStartPoint: objc2_foundation::NSPoint::new(0.5, 1.0)];- let _: () = msg_send![layer, setEndPoint: objc2_foundation::NSPoint::new(0.5, 0.0)];- let _: () = msg_send![layer, setCornerRadius: radius];- let _: () = msg_send![layer, setMasksToBounds: true];- set_continuous_corners(layer);- let _: () = msg_send![view, setLayer: layer];- let _: () = msg_send![view, setWantsLayer: true];- view-}--unsafe fn visual_effect_view(- frame: objc2_foundation::NSRect,- radius: f64,- material: isize,- blending_mode: isize,- clips: bool,-) -> *mut objc2::runtime::AnyObject {- use objc2::runtime::AnyObject;- use objc2::{class, msg_send};-- let view: *mut AnyObject = {- let alloc: *mut AnyObject = msg_send![class!(NSVisualEffectView), alloc];- msg_send![alloc, initWithFrame: frame]- };- let _: () = msg_send![view, setMaterial: material];- let _: () = msg_send![view, setBlendingMode: blending_mode];- let _: () = msg_send![view, setState: 1isize];- let _: () = msg_send![view, setWantsLayer: true];- let layer: *mut AnyObject = msg_send![view, layer];- let _: () = msg_send![layer, setCornerRadius: radius];- let _: () = msg_send![layer, setMasksToBounds: clips];- view-}--unsafe fn add_circle(- parent: *mut objc2::runtime::AnyObject,- x: f64,- y: f64,- diameter: f64,- fill: *mut objc2::runtime::AnyObject,-) {- use objc2::msg_send;- use objc2_foundation::{NSPoint, NSRect, NSSize};-- let circle = rounded_view(- NSRect::new(NSPoint::new(x, y), NSSize::new(diameter, diameter)),- diameter / 2.0,- fill,- true,- );- let _: () = msg_send![parent, addSubview: circle];-}--fn appkit_rect(rect: pip_preview::LayoutRect, bounds_height: f64) -> objc2_foundation::NSRect {- use objc2_foundation::{NSPoint, NSRect, NSSize};-- NSRect::new(- NSPoint::new(rect.x, bounds_height - rect.y - rect.height),- NSSize::new(rect.width, rect.height),- )-}--fn parse_native_pid(target_id: &str) -> Option<i32> {- let mut parts = target_id.split(':');- (parts.next()? == "window")- .then(|| parts.next()?.parse::<i32>().ok())- .flatten()-}--fn truncate_label(value: &str, max_chars: usize) -> String {- let mut chars = value.chars();- let prefix = chars.by_ref().take(max_chars).collect::<String>();- if chars.next().is_some() {- format!("{prefix}...")- } else {- prefix- }-}--unsafe fn target_identity(frame: &PipFrame) -> (String, *mut objc2::runtime::AnyObject, char) {use objc2::runtime::AnyObject;use objc2::{class, msg_send};- if frame.target.target_kind == PipTargetKind::NativeWindow {- if let Some(pid) = parse_native_pid(&frame.target.target_id) {- let app: *mut AnyObject = msg_send![- class!(NSRunningApplication),- runningApplicationWithProcessIdentifier: pid- ];- if !app.is_null() {- let name: *mut AnyObject = msg_send![app, localizedName];- let icon: *mut AnyObject = msg_send![app, icon];- if let Some(name) = rust_string(name) {- let fallback = name.chars().next().unwrap_or('A').to_ascii_uppercase();- return (truncate_label(&name, 28), icon, fallback);- }- }- }- }+ let frame: PipFrame = *Box::from_raw(ctx as *mut PipFrame);- let label = match frame.target.target_kind {- PipTargetKind::BrowserTab => {- if frame.target.target_label.trim().is_empty() {- "Browser".to_owned()- } else {- truncate_label(&frame.target.target_label, 28)- }+ let (image_view_ptr, label_ptr) = {+ let guard = HANDLES.lock().unwrap();+ match guard.as_ref() {+ Some(h) => (h.image_view, h.label),+ None => return,}- PipTargetKind::NativeWindow => truncate_label(&frame.target.workspace_label, 28),- };- let fallback = match frame.target.target_kind {- PipTargetKind::BrowserTab => 'B',- PipTargetKind::NativeWindow => label.chars().next().unwrap_or('A').to_ascii_uppercase(),};- (label, std::ptr::null_mut(), fallback)-}-unsafe fn image_from_png(bytes: &[u8]) -> *mut objc2::runtime::AnyObject {- use objc2::runtime::AnyObject;- use objc2::{class, msg_send};-- let data: *mut AnyObject = msg_send![+ // Construct NSData from the PNG bytes, then NSImage from NSData.+ // `dataWithBytes:length:` copies into a fresh NSData so the input+ // `Vec<u8>` can be freed at the end of this block.+ let png_ptr = frame.png_bytes.as_ptr() as *const c_void;+ let png_len = frame.png_bytes.len();+ let ns_data: *mut AnyObject = msg_send![class!(NSData),- dataWithBytes: bytes.as_ptr() as *const c_void- length: bytes.len()+ dataWithBytes: png_ptr+ length: png_len];- if data.is_null() {- return std::ptr::null_mut();- }- let alloc: *mut AnyObject = msg_send![class!(NSImage), alloc];- msg_send![alloc, initWithData: data]-}--unsafe fn render_target_window(- canvas: *mut objc2::runtime::AnyObject,- frame: &PipFrame,- layout: pip_preview::TargetLayout,- bounds_height: f64,- active: bool,-) {- use objc2::runtime::AnyObject;- use objc2::{class, msg_send};- use objc2_foundation::{NSPoint, NSRect, NSSize};-- let window_frame = appkit_rect(layout.window, bounds_height);- let radius = (window_frame.size.width.min(window_frame.size.height) * 0.035).clamp(5.0, 9.0);- let shadow = rounded_view(window_frame, radius, color(0.0, 0.0, 0.0, 0.0), false);- let shadow_layer: *mut AnyObject = msg_send![shadow, layer];- let _: () = msg_send![shadow_layer, setShadowOpacity: 0.30_f32];- let _: () = msg_send![shadow_layer, setShadowRadius: 16.0_f64];- let _: () = msg_send![shadow_layer, setShadowOffset: NSSize::new(0.0, -6.0)];- let black = color(0.0, 0.0, 0.0, 0.90);- let black_cg: *mut CGColor = msg_send![black, CGColor];- let _: () = msg_send![shadow_layer, setShadowColor: black_cg];-- let window = rounded_view(- NSRect::new(- NSPoint::new(0.0, 0.0),- NSSize::new(window_frame.size.width, window_frame.size.height),- ),- radius,- color(0.0, 0.0, 0.0, 0.0),- true,- );- let window_layer: *mut AnyObject = msg_send![window, layer];- if active {- set_layer_border(window_layer, 2.2, color(0.25, 0.65, 1.0, 0.96));- } else if SHOW_SHELL_BORDER {- set_layer_border(window_layer, 0.55, color(0.0, 0.0, 0.0, 0.22));- }- let image_view: *mut AnyObject = {- let alloc: *mut AnyObject = msg_send![class!(NSImageView), alloc];- msg_send![- alloc,- initWithFrame: NSRect::new(- NSPoint::new(0.0, 0.0),- NSSize::new(window_frame.size.width, window_frame.size.height),- )- ]- };- let _: () = msg_send![image_view, setImageScaling: 3u64];- let image = image_from_png(&frame.png_bytes);- if !image.is_null() {- let _: () = msg_send![image_view, setImage: image];- }- let _: () = msg_send![window, addSubview: image_view];- if let Some((normalized_x, normalized_y)) = frame.cursor_position {- let pointer_size = NSSize::new(18.0, 22.0);- let x = (normalized_x.clamp(0.0, 1.0) * window_frame.size.width)- .clamp(0.0, (window_frame.size.width - pointer_size.width).max(0.0));- let y_from_top = normalized_y.clamp(0.0, 1.0) * window_frame.size.height;- let y = (window_frame.size.height - y_from_top - pointer_size.height).clamp(- 0.0,- (window_frame.size.height - pointer_size.height).max(0.0),- );- let pointer: *mut AnyObject = {- let alloc: *mut AnyObject = msg_send![class!(NSImageView), alloc];- msg_send![- alloc,- initWithFrame: NSRect::new(NSPoint::new(x, y), pointer_size)- ]- };- let arrow_cursor: *mut AnyObject = msg_send![class!(NSCursor), arrowCursor];- let arrow_image: *mut AnyObject = msg_send![arrow_cursor, image];- let _: () = msg_send![pointer, setImage: arrow_image];- let _: () = msg_send![pointer, setImageScaling: 0u64];- let _: () = msg_send![pointer, setWantsLayer: true];- let pointer_layer: *mut AnyObject = msg_send![pointer, layer];- let _: () = msg_send![pointer_layer, setShadowOpacity: 0.72_f32];- let _: () = msg_send![pointer_layer, setShadowRadius: 2.5_f64];- let _: () = msg_send![pointer_layer, setShadowOffset: NSSize::new(0.0, -1.0)];- let shadow_color = color(0.0, 0.0, 0.0, 0.92);- let shadow_cg: *mut CGColor = msg_send![shadow_color, CGColor];- let _: () = msg_send![pointer_layer, setShadowColor: shadow_cg];- let _: () = msg_send![window, addSubview: pointer];- }- let _: () = msg_send![shadow, addSubview: window];- let _: () = msg_send![canvas, addSubview: shadow];-}--unsafe fn render_dock(- canvas: *mut objc2::runtime::AnyObject,- frames: &[PipFrame],- layout: &pip_preview::DesktopLayout,- bounds_height: f64,-) {- use objc2::runtime::AnyObject;- use objc2::{class, msg_send};- use objc2_foundation::{NSPoint, NSRect, NSSize};-- let dock_frame = appkit_rect(layout.dock, bounds_height);- let dock = visual_effect_view(dock_frame, dock_frame.size.height * 0.30, 6, 1, true);- let appearance_name = ns_string("NSAppearanceNameVibrantDark");- let dark_appearance: *mut AnyObject =- msg_send![class!(NSAppearance), appearanceNamed: appearance_name];- if !dark_appearance.is_null() {- let _: () = msg_send![dock, setAppearance: dark_appearance];- }- let dock_layer: *mut AnyObject = msg_send![dock, layer];- set_layer_background(dock_layer, color(0.30, 0.32, 0.34, 0.22));- set_layer_border(dock_layer, 0.65, color(1.0, 1.0, 1.0, 0.20));- let _: () = msg_send![dock_layer, setShadowOpacity: 0.34_f32];- let _: () = msg_send![dock_layer, setShadowRadius: 14.0_f64];- let _: () = msg_send![dock_layer, setShadowOffset: NSSize::new(0.0, -6.0)];- let dock_shadow = color(0.0, 0.0, 0.0, 0.75);- let dock_shadow_cg: *mut CGColor = msg_send![dock_shadow, CGColor];- let _: () = msg_send![dock_layer, setShadowColor: dock_shadow_cg];-- for (frame, icon_layout) in frames.iter().zip(layout.dock_icons.iter()) {- let icon_global = appkit_rect(*icon_layout, bounds_height);- let icon_frame = NSRect::new(- NSPoint::new(- icon_global.origin.x - dock_frame.origin.x,- icon_global.origin.y - dock_frame.origin.y,- ),- icon_global.size,- );- let (_, icon, fallback) = target_identity(frame);- if icon.is_null() {- let tile = rounded_view(- icon_frame,- icon_frame.size.width * 0.22,- color(0.98, 0.98, 0.99, 0.86),- true,- );- add_text_label(- tile,- NSRect::new(- NSPoint::new(0.0, (icon_frame.size.height - 24.0) / 2.0),- NSSize::new(icon_frame.size.width, 24.0),- ),- &fallback.to_string(),- (icon_frame.size.width * 0.46).clamp(12.0, 22.0),- true,- LabelTone::Dark,- 1,- );- let _: () = msg_send![dock, addSubview: tile];- } else {- let image_view: *mut AnyObject = {- let alloc: *mut AnyObject = msg_send![class!(NSImageView), alloc];- msg_send![- alloc,- initWithFrame: NSRect::new(- icon_frame.origin,- icon_frame.size,- )- ]- };- let _: () = msg_send![image_view, setImageScaling: 3u64];- let _: () = msg_send![image_view, setImage: icon];- let _: () = msg_send![dock, addSubview: image_view];- }-- let dot_size = 3.2_f64.min(icon_frame.size.width * 0.10);- let dot_x = icon_frame.origin.x + (icon_frame.size.width - dot_size) / 2.0;- add_circle(dock, dot_x, 2.2, dot_size, color(0.20, 0.72, 0.38, 0.92));- }- let _: () = msg_send![canvas, addSubview: dock];-}--fn session_tabs_layout(- width: f64,- height: f64,- workspaces: &[PipWorkspaceSummary],- selected_workspace_id: Option<&str>,-) -> SessionTabsLayout {- layout_session_tabs(- LayoutRect {- x: (width - 600.0).max(0.0) / 2.0,- y: (height - SESSION_SELECTOR_HEIGHT).max(0.0),- width: width.min(600.0),- height: SESSION_SELECTOR_HEIGHT,- },- workspaces,- selected_workspace_id,- )-}--unsafe fn render_session_selector(- canvas: *mut objc2::runtime::AnyObject,- bounds: objc2_foundation::NSRect,- workspaces: &[PipWorkspaceSummary],- selected_workspace_id: Option<&str>,-) {- use objc2::runtime::AnyObject;- use objc2::{class, msg_send};-- let tabs = session_tabs_layout(- bounds.size.width,- bounds.size.height,- workspaces,- selected_workspace_id,- );- let Some(first) = tabs.tabs.first() else {- return;- };- let last = tabs.tabs.last().expect("tabs are non-empty");- let selector_frame = objc2_foundation::NSRect::new(- objc2_foundation::NSPoint::new(first.rect.x - 6.0, first.rect.y - 3.0),- objc2_foundation::NSSize::new(- last.rect.x + last.rect.width - first.rect.x + 12.0,- first.rect.height + 6.0,- ),- );- let selector = visual_effect_view(selector_frame, 14.0, 6, 1, true);- let selector_layer: *mut AnyObject = msg_send![selector, layer];- set_layer_background(selector_layer, color(0.10, 0.12, 0.15, 0.30));- set_layer_border(selector_layer, 0.6, color(1.0, 1.0, 1.0, 0.26));-- let delegate = HANDLES- .lock()- .unwrap()- .as_ref()- .map(|handles| handles.delegate as *mut AnyObject)- .unwrap_or(std::ptr::null_mut());- for (index, tab) in tabs.tabs.iter().enumerate() {- let workspace = &workspaces[index];- let selected = tab.selected;- let (red, green, blue) = (- f64::from(tab.accent.0) / 255.0,- f64::from(tab.accent.1) / 255.0,- f64::from(tab.accent.2) / 255.0,- );- let icon_frame = objc2_foundation::NSRect::new(- objc2_foundation::NSPoint::new(tab.rect.x, tab.rect.y),- objc2_foundation::NSSize::new(tab.rect.width, tab.rect.height),- );- let button: *mut AnyObject = {- let allocated: *mut AnyObject = msg_send![class!(NSButton), alloc];- msg_send![allocated, initWithFrame: icon_frame]- };- let _: () = msg_send![button, setBordered: false];- let _: () = msg_send![button, setRefusesFirstResponder: true];- let _: () = msg_send![button, setWantsLayer: true];- let layer: *mut AnyObject = msg_send![button, layer];- let _: () = msg_send![layer, setCornerRadius: 11.0_f64];- set_continuous_corners(layer);- set_layer_background(- layer,- color(red, green, blue, if selected { 0.96 } else { 0.62 }),- );- set_layer_border(- layer,- if selected { 1.4 } else { 0.5 },- color(1.0, 1.0, 1.0, if selected { 0.94 } else { 0.38 }),- );- let initial = workspace- .workspace_label- .chars()- .find(|character| character.is_alphanumeric())- .unwrap_or('A')- .to_uppercase()- .collect::<String>();- let title = ns_string(&initial);- let tooltip = ns_string(&workspace.workspace_label);- if !title.is_null() {- let _: () = msg_send![button, setTitle: title];- }- if !tooltip.is_null() {- let _: () = msg_send![button, setToolTip: tooltip];- }- let font: *mut AnyObject = msg_send![class!(NSFont), boldSystemFontOfSize: 10.0_f64];- let _: () = msg_send![button, setFont: font];- let _: () = msg_send![button, setTag: index as isize];- if !delegate.is_null() {- let _: () = msg_send![button, setTarget: delegate];- let _: () = msg_send![button, setAction: objc2::sel!(selectWorkspace:)];- }- let _: () = msg_send![selector, addSubview: button];- }- let _: () = msg_send![canvas, addSubview: selector];-}--fn desktop_frame(bounds: objc2_foundation::NSRect) -> objc2_foundation::NSRect {- use objc2_foundation::{NSPoint, NSRect, NSSize};-- if !SHOW_SHELL_BORDER {- return bounds;- }-- NSRect::new(- NSPoint::new(SHELL_SIDE_INSET, SHELL_BOTTOM_INSET),- NSSize::new(- (bounds.size.width - 2.0 * SHELL_SIDE_INSET).max(1.0),- (bounds.size.height - SHELL_TOP_INSET - SHELL_BOTTOM_INSET).max(1.0),- ),- )-}--unsafe fn install_shell_details(- content_view: *mut objc2::runtime::AnyObject,- bounds: objc2_foundation::NSRect,-) {- use objc2::msg_send;- use objc2_foundation::{NSPoint, NSRect, NSSize};-- if !SHOW_SHELL_BORDER {- return;- }-- let grip_width = 24.0;- let grip = rounded_view(- NSRect::new(- NSPoint::new(- (bounds.size.width - grip_width) / 2.0,- bounds.size.height - 9.0,- ),- NSSize::new(grip_width, 3.0),- ),- 1.5,- color(0.08, 0.10, 0.13, 0.52),- true,- );- // Keep the grip centred and pinned to the shell's top edge while resizing.- let _: () = msg_send![grip, setAutoresizingMask: 13u64];- let grip_layer: *mut objc2::runtime::AnyObject = msg_send![grip, layer];- let _: () = msg_send![grip_layer, setShadowOpacity: 0.32_f32];- let _: () = msg_send![grip_layer, setShadowRadius: 0.8_f64];- let _: () = msg_send![grip_layer, setShadowOffset: NSSize::new(0.0, 1.0)];- let _: () = msg_send![content_view, addSubview: grip];-- let marker = color(1.0, 1.0, 1.0, 0.48);- for (index, length) in [10.0_f64, 6.0].into_iter().enumerate() {- let bar = rounded_view(- NSRect::new(- NSPoint::new(- (bounds.size.width - 16.0 + index as f64 * 4.0).max(0.0),- 4.5 + index as f64 * 2.0,- ),- NSSize::new(length, 1.2),- ),- 0.6,- marker,- false,- );- let _: () = msg_send![bar, setFrameCenterRotation: -45.0_f64];- // Move with the right edge while remaining in the bottom shell rail.- let _: () = msg_send![bar, setAutoresizingMask: 1u64];- let _: () = msg_send![content_view, addSubview: bar];- }-}--fn resized_window_frame(- start: objc2_foundation::NSRect,- delta_x: f64,- delta_y: f64,- direction: isize,- minimum: objc2_foundation::NSSize,-) -> objc2_foundation::NSRect {- use objc2_foundation::{NSPoint, NSRect, NSSize};-- let mut x = start.origin.x;- let mut y = start.origin.y;- let mut width = start.size.width;- let mut height = start.size.height;- if direction & RESIZE_LEFT != 0 {- let applied = delta_x.min(width - minimum.width);- x += applied;- width -= applied;- }- if direction & RESIZE_RIGHT != 0 {- width = (width + delta_x).max(minimum.width);- }- if direction & RESIZE_BOTTOM != 0 {- let applied = delta_y.min(height - minimum.height);- y += applied;- height -= applied;- }- if direction & RESIZE_TOP != 0 {- height = (height + delta_y).max(minimum.height);- }- NSRect::new(NSPoint::new(x, y), NSSize::new(width, height))-}--fn agent_view_shell_hit_class() -> &'static objc2::runtime::AnyClass {- use objc2::class;- use objc2::declare::ClassBuilder;-- static CLASS: OnceLock<&'static objc2::runtime::AnyClass> = OnceLock::new();- CLASS.get_or_init(|| {- let mut builder = ClassBuilder::new("CuaDriverAgentViewShellHitView", class!(NSView))- .expect("CuaDriverAgentViewShellHitView already registered");- unsafe {- builder.add_method(- objc2::sel!(acceptsFirstMouse:),- shell_accepts_first_mouse as extern "C" fn(_, _, _) -> objc2::runtime::Bool,- );- builder.add_method(- objc2::sel!(mouseDown:),- shell_mouse_down as extern "C" fn(_, _, _),- );- }- builder.register()- })-}--extern "C" fn shell_accepts_first_mouse(- _view: *mut objc2::runtime::AnyObject,- _selector: objc2::runtime::Sel,- _event: *mut objc2::runtime::AnyObject,-) -> objc2::runtime::Bool {- objc2::runtime::Bool::YES-}--extern "C" fn shell_mouse_down(- view: *mut objc2::runtime::AnyObject,- _selector: objc2::runtime::Sel,- event: *mut objc2::runtime::AnyObject,-) {- use objc2::msg_send;- use objc2::runtime::AnyObject;-- if view.is_null() || event.is_null() {+ if ns_data.is_null() {return;}- unsafe {- let window: *mut AnyObject = msg_send![view, window];- if window.is_null() {- return;- }- let direction = rust_string(msg_send![view, toolTip])- .and_then(|value| value.parse::<isize>().ok())- .unwrap_or(0);- if direction == 0 {- let _: () = msg_send![window, performWindowDragWithEvent: event];- return;- }-- let start_frame: objc2_foundation::NSRect = msg_send![window, frame];- let minimum: objc2_foundation::NSSize = msg_send![window, minSize];- let start_mouse: objc2_foundation::NSPoint =- msg_send![objc2::class!(NSEvent), mouseLocation];- let event_mask: u64 = (1 << 2) | (1 << 6);- let distant_future: *mut AnyObject = msg_send![objc2::class!(NSDate), distantFuture];- let default_mode = ns_string("kCFRunLoopDefaultMode");- loop {- let next: *mut AnyObject = msg_send![- window,- nextEventMatchingMask: event_mask- untilDate: distant_future- inMode: default_mode- dequeue: true- ];- if next.is_null() {- break;- }- let event_type: usize = msg_send![next, type];- if event_type == 2 {- break;- }- let mouse: objc2_foundation::NSPoint = msg_send![objc2::class!(NSEvent), mouseLocation];- let frame = resized_window_frame(- start_frame,- mouse.x - start_mouse.x,- mouse.y - start_mouse.y,- direction,- minimum,- );- let _: () = msg_send![window, setFrame: frame display: true];- }- }-}--unsafe fn add_shell_hit_view(- parent: *mut objc2::runtime::AnyObject,- frame: objc2_foundation::NSRect,- direction: isize,- autoresizing_mask: u64,-) {- use objc2::msg_send;- use objc2::runtime::AnyObject;-- let allocated: *mut AnyObject = msg_send![agent_view_shell_hit_class(), alloc];- let view: *mut AnyObject = msg_send![allocated, initWithFrame: frame];- let direction_string = ns_string(&direction.to_string());- let _: () = msg_send![view, setToolTip: direction_string];- let _: () = msg_send![view, setAutoresizingMask: autoresizing_mask];- let _: () = msg_send![parent, addSubview: view];-}--unsafe fn install_shell_interaction(- content_view: *mut objc2::runtime::AnyObject,- bounds: objc2_foundation::NSRect,-) {- use objc2_foundation::{NSPoint, NSRect, NSSize};-- let edge = RESIZE_HIT_INSET;- let width = bounds.size.width;- let height = bounds.size.height;- add_shell_hit_view(- content_view,- NSRect::new(- NSPoint::new(edge, height - edge),- NSSize::new(width - 2.0 * edge, edge),- ),- RESIZE_TOP,- 10,- );- add_shell_hit_view(- content_view,- NSRect::new(- NSPoint::new(edge, 0.0),- NSSize::new(width - 2.0 * edge, edge),- ),- RESIZE_BOTTOM,- 10,- );- add_shell_hit_view(- content_view,- NSRect::new(- NSPoint::new(0.0, edge),- NSSize::new(edge, height - 2.0 * edge),- ),- RESIZE_LEFT,- 20,- );- add_shell_hit_view(- content_view,- NSRect::new(- NSPoint::new(width - edge, edge),- NSSize::new(edge, height - 2.0 * edge),- ),- RESIZE_RIGHT,- 17,- );- for (origin, direction, mask) in [- (NSPoint::new(0.0, 0.0), RESIZE_LEFT | RESIZE_BOTTOM, 4),- (- NSPoint::new(width - edge, 0.0),- RESIZE_RIGHT | RESIZE_BOTTOM,- 1,- ),- (- NSPoint::new(0.0, height - edge),- RESIZE_LEFT | RESIZE_TOP,- 8,- ),- (- NSPoint::new(width - edge, height - edge),- RESIZE_RIGHT | RESIZE_TOP,- 2,- ),- ] {- add_shell_hit_view(- content_view,- NSRect::new(origin, NSSize::new(edge, edge)),- direction,- mask,- );- }- add_shell_hit_view(- content_view,- NSRect::new(- NSPoint::new(edge, height - SHELL_TOP_INSET),- NSSize::new(width - 2.0 * edge, SHELL_TOP_INSET - edge),- ),- 0,- 10,- );-}--unsafe fn render_snapshot(snapshot: &ViewSnapshot) {- use objc2::runtime::AnyObject;- use objc2::{class, msg_send};- use objc2_foundation::{NSPoint, NSRect, NSSize};-- let canvas = {- let guard = HANDLES.lock().unwrap();- match guard.as_ref() {- Some(handles) => handles.canvas_view as *mut AnyObject,- None => return,- }+ let img: *mut AnyObject = {+ let alloc: *mut AnyObject = msg_send![class!(NSImage), alloc];+ msg_send![alloc, initWithData: ns_data]};- let empty: *mut AnyObject = msg_send![class!(NSArray), array];- let _: () = msg_send![canvas, setSubviews: empty];+ if !img.is_null() {+ let image_view = image_view_ptr as *mut AnyObject;+ let _: () = msg_send![image_view, setImage: img];+ }- let bounds: NSRect = msg_send![canvas, bounds];- let selector_height = if snapshot.workspaces.len() > 1 {- SESSION_SELECTOR_HEIGHT- } else {- 0.0- };- let content_height = (bounds.size.height - selector_height).max(1.0);- let target_sizes = snapshot- .frames- .iter()- .map(|frame| {- png_dimensions(&frame.png_bytes).unwrap_or(TargetSize {- width: 16,- height: 10,- })- })- .collect::<Vec<_>>();- let layout = layout_desktop(bounds.size.width, content_height, &target_sizes);- if snapshot.frames.is_empty() {- let waiting = appkit_rect(layout.desktop, content_height);- add_text_label(- canvas,- NSRect::new(- NSPoint::new(- waiting.origin.x + 16.0,- waiting.origin.y + waiting.size.height / 2.0 - 12.0,- ),- NSSize::new((waiting.size.width - 32.0).max(40.0), 24.0),- ),- "Waiting for an exact window or browser tab...",- 12.0,- false,- LabelTone::Muted,- 1,- );- } else {- for (frame, target_layout) in snapshot.frames.iter().zip(layout.targets.iter().copied()) {- render_target_window(- canvas,- frame,- target_layout,- content_height,- snapshot.active_view_id.as_deref() == Some(frame.target.view_id().as_str()),- );+ // Update the label. NSString::stringWithUTF8String requires NUL+ // termination; copy into a CString so we hand a clean buffer.+ if let Ok(cstr) = std::ffi::CString::new(frame.action_label) {+ let ns_str: *mut AnyObject = msg_send![+ class!(NSString),+ stringWithUTF8String: cstr.as_ptr() as *const u8+ ];+ if !ns_str.is_null() {+ let label = label_ptr as *mut AnyObject;+ let _: () = msg_send![label, setStringValue: ns_str];}- // Keep the resting Agent View free of a redundant internal Dock.}- render_session_selector(- canvas,- bounds,- &snapshot.workspaces,- snapshot.selected_workspace_id.as_deref(),- );}unsafe extern "C" fn shutdown_cb(_ctx: *mut c_void) {@@ -1163,90 +179,27 @@ unsafe extern "C" fn shutdown_cb(_ctx: *mut c_void) {use objc2::runtime::AnyObject;let handles = HANDLES.lock().unwrap().take();- VIEW_MODEL.lock().unwrap().take();- if let Some(handles) = handles {- let window = handles.window as *mut AnyObject;- let _: () = msg_send![window, setDelegate: std::ptr::null_mut::<AnyObject>()];- let _: () = msg_send![window, orderOut: std::ptr::null_mut::<AnyObject>()];- let _: () = msg_send![window, close];- let _ = handles.delegate;- }-}--fn agent_view_delegate_class() -> &'static objc2::runtime::AnyClass {- use objc2::class;- use objc2::declare::ClassBuilder;-- static CLASS: OnceLock<&'static objc2::runtime::AnyClass> = OnceLock::new();- CLASS.get_or_init(|| {- let superclass = class!(NSObject);- let mut builder = ClassBuilder::new("CuaDriverAgentViewDelegate", superclass)- .expect("CuaDriverAgentViewDelegate already registered");- unsafe {- builder.add_method(- objc2::sel!(windowDidResize:),- on_window_did_resize as extern "C" fn(_, _, _),- );- builder.add_method(- objc2::sel!(selectWorkspace:),- on_select_workspace as extern "C" fn(_, _, _),- );+ if let Some(h) = handles {+ let win = h.window as *mut AnyObject;+ if !win.is_null() {+ let _: () = msg_send![win, orderOut: std::ptr::null_mut::<AnyObject>()];+ let _: () = msg_send![win, close];}- builder.register()- })-}--unsafe fn agent_view_delegate_instance() -> *mut objc2::runtime::AnyObject {- use objc2::msg_send;- use objc2::runtime::AnyObject;-- let class = agent_view_delegate_class();- let allocated: *mut AnyObject = msg_send![class, alloc];- msg_send![allocated, init]-}--extern "C" fn on_window_did_resize(- _delegate: *mut objc2::runtime::AnyObject,- _selector: objc2::runtime::Sel,- _notification: *mut objc2::runtime::AnyObject,-) {- let snapshot = current_snapshot();- unsafe {- update_wallpaper_frame();- render_snapshot(&snapshot);- };-}--extern "C" fn on_select_workspace(- _delegate: *mut objc2::runtime::AnyObject,- _selector: objc2::runtime::Sel,- sender: *mut objc2::runtime::AnyObject,-) {- if sender.is_null() {- return;- }- let index: isize = unsafe { objc2::msg_send![sender, tag] };- if index < 0 {- return;}- let snapshot = {- let mut model = VIEW_MODEL.lock().unwrap();- let Some(model) = model.as_mut() else {- return;- };- let workspaces = model.workspaces();- let Some(workspace) = workspaces.get(index as usize) else {- return;- };- let workspace_id = workspace.workspace_id.clone();- model.select_workspace(&workspace_id);- clone_snapshot(model)- };- unsafe { render_snapshot(&snapshot) };}-/// Park the main thread in `NSApplication.run()` for the main-queue AppKit-/// work used by Agent View when the cursor overlay is disabled.+// ── AppKit main loop helper for Serve mode ───────────────────────────────++/// Park the main thread in `NSApplication.run()`. Used by `cua-driver+/// serve --experimental-pip` so the dispatch_async_f → main queue+/// path PiP frames go through can be drained. Mirrors the cursor+/// overlay's `run_appkit` startup (Accessory activation policy →+/// finishLaunching → run) without installing the overlay's+/// CALayer-backed window itself.+///+/// Never returns — the background `serve::run_serve_cmd` thread calls+/// `std::process::exit` when it finishes, which tears down NSApp at+/// the same time.pub fn run_appkit_main_loop() {use objc2::runtime::AnyObject;use objc2::{class, msg_send};@@ -1255,514 +208,218 @@ pub fn run_appkit_main_loop() {.expect("run_appkit_main_loop must be called from the main thread");unsafe {let app: *mut AnyObject = msg_send![class!(NSApplication), sharedApplication];+ // Accessory policy: no Dock icon, no menu bar. Keeps the+ // daemon out of the user's application switcher, same as+ // the cursor overlay's NSApp setup.let _: bool = msg_send![app, setActivationPolicy: 1i64];let _: () = msg_send![app, finishLaunching];let _: () = msg_send![app, run];}}+// ── Factory ──────────────────────────────────────────────────────────────+pub struct MacosPipBackendFactory;impl PipBackendFactory for MacosPipBackendFactory {fn start(&self, cfg: &PipConfig) -> anyhow::Result<Box<dyn PipBackend>> {- dispatch_to_main(cfg.clone(), init_cb);+ // Window construction must happen on the main thread. We hand+ // off via dispatch_async_f and return immediately — the first+ // few frames may be dropped while init races, which is fine+ // for a live-preview UX.+ let cfg_clone = cfg.clone();+ dispatch_to_main(cfg_clone, init_cb);Ok(Box::new(MacosPipBackend))}}-unsafe fn install_wallpaper(- content_view: *mut objc2::runtime::AnyObject,- screen: *mut objc2::runtime::AnyObject,- bounds: objc2_foundation::NSRect,-) -> (- *mut objc2::runtime::AnyObject,- *mut objc2::runtime::AnyObject,- *mut objc2::runtime::AnyObject,- *mut objc2::runtime::AnyObject,-) {- use objc2::runtime::AnyObject;- use objc2::{class, msg_send};-- // The same wallpaper continues beneath the shell rails. The centre is- // covered by a sharp inset copy while the outer copy is blurred and tinted.- let glass_frame = rounded_view(- bounds,- if SHOW_SHELL_BORDER {- CONTAINER_RADIUS- } else {- PIP_CORNER_RADIUS- },- color(0.18, 0.21, 0.25, if SHOW_SHELL_BORDER { 0.16 } else { 0.0 }),- true,- );- let _: () = msg_send![glass_frame, setAutoresizingMask: 18u64];- let glass_layer: *mut AnyObject = msg_send![glass_frame, layer];- set_continuous_corners(glass_layer);- if SHOW_SHELL_BORDER {- set_layer_border(glass_layer, 0.8, color(1.0, 1.0, 1.0, 0.50));- }- let _: () = msg_send![content_view, addSubview: glass_frame];-- let glass_bounds: objc2_foundation::NSRect = msg_send![glass_frame, bounds];- let shell_wallpaper_view: *mut AnyObject = {- let allocated: *mut AnyObject = msg_send![class!(NSImageView), alloc];- msg_send![allocated, initWithFrame: glass_bounds]- };- let _: () = msg_send![shell_wallpaper_view, setImageScaling: 2u64];- if SHOW_SHELL_BORDER {- let _: () = msg_send![glass_frame, addSubview: shell_wallpaper_view];- }-- let backdrop: *mut AnyObject = {- let allocated: *mut AnyObject = msg_send![class!(NSVisualEffectView), alloc];- msg_send![- allocated,- initWithFrame: objc2_foundation::NSRect::new(- objc2_foundation::NSPoint::new(0.0, 0.0),- bounds.size,- )- ]- };- let _: () = msg_send![backdrop, setAutoresizingMask: 18u64];- let _: () = msg_send![backdrop, setMaterial: 13i64]; // HUD window material.- let _: () = msg_send![backdrop, setBlendingMode: 1i64]; // Blur the wallpaper within the shell.- let _: () = msg_send![backdrop, setState: 1i64]; // Keep the blur active while non-key.- let _: () = msg_send![backdrop, setEmphasized: true];- if SHOW_SHELL_BORDER {- let _: () = msg_send![glass_frame, addSubview: backdrop];- }-- // A restrained silver-to-graphite tint keeps the blurred backdrop legible- // as a shell instead of exposing raw, high-contrast shapes behind it.- let body = gradient_view(- objc2_foundation::NSRect::new(objc2_foundation::NSPoint::new(0.0, 0.0), bounds.size),- CONTAINER_RADIUS,- color(0.68, 0.71, 0.76, 0.34),- color(0.15, 0.18, 0.22, 0.52),- );- let _: () = msg_send![body, setAutoresizingMask: 18u64];- let body_layer: *mut AnyObject = msg_send![body, layer];- let scale: f64 = msg_send![screen, backingScaleFactor];- if scale > 0.0 {- // Layer-hosting views do not inherit the backing scale, and the chrome- // hairlines are sub-point.- let _: () = msg_send![body_layer, setContentsScale: scale];- }- if SHOW_SHELL_BORDER {- let _: () = msg_send![glass_frame, addSubview: body];- }-- // Light-from-above specular along the top edge of the chrome.- let highlight = rounded_view(- objc2_foundation::NSRect::new(- objc2_foundation::NSPoint::new(CONTAINER_RADIUS, bounds.size.height - 1.7),- objc2_foundation::NSSize::new(- (bounds.size.width - 2.0 * CONTAINER_RADIUS).max(1.0),- 1.0,- ),- ),- 0.5,- color(1.0, 1.0, 1.0, 0.38),- true,- );- let _: () = msg_send![highlight, setAutoresizingMask: 10u64];- if SHOW_SHELL_BORDER {- let _: () = msg_send![glass_frame, addSubview: highlight];- }-- let container_frame = desktop_frame(bounds);- let container_radius = if SHOW_SHELL_BORDER {- 8.5- } else {- PIP_CORNER_RADIUS- };- let wallpaper_shadow = rounded_view(- container_frame,- container_radius,- color(0.0, 0.0, 0.0, 0.01),- false,- );- let _: () = msg_send![wallpaper_shadow, setAutoresizingMask: 18u64];- let shadow_layer: *mut AnyObject = msg_send![wallpaper_shadow, layer];- set_continuous_corners(shadow_layer);- // The inset shadow separates the desktop from the thicker shell rails.- let _: () = msg_send![shadow_layer, setShadowOpacity: 0.62_f32];- let _: () = msg_send![shadow_layer, setShadowRadius: 4.0_f64];- let _: () = msg_send![shadow_layer, setShadowOffset: objc2_foundation::NSSize::new(0.0, -1.5)];- let shadow_color = color(0.0, 0.0, 0.0, 0.85);- let shadow_cg: *mut CGColor = msg_send![shadow_color, CGColor];- let _: () = msg_send![shadow_layer, setShadowColor: shadow_cg];- if SHOW_SHELL_BORDER {- let _: () = msg_send![content_view, addSubview: wallpaper_shadow];- }-- // Machined inner edge. It hugs the seam from the chrome side, so the- // miniature desktop stays separated from the graphite body whether the- // content under it is a bright wallpaper or a dark window.- let seam_outset = 0.75;- let seam = rounded_view(- objc2_foundation::NSRect::new(- objc2_foundation::NSPoint::new(- container_frame.origin.x - seam_outset,- container_frame.origin.y - seam_outset,- ),- objc2_foundation::NSSize::new(- container_frame.size.width + 2.0 * seam_outset,- container_frame.size.height + 2.0 * seam_outset,- ),- ),- container_radius + seam_outset,- color(0.0, 0.0, 0.0, 0.0),- false,- );- let _: () = msg_send![seam, setAutoresizingMask: 18u64];- let seam_layer: *mut AnyObject = msg_send![seam, layer];- set_continuous_corners(seam_layer);- if SHOW_SHELL_BORDER {- set_layer_border(seam_layer, seam_outset, color(1.0, 1.0, 1.0, 0.20));- }- let _: () = msg_send![content_view, addSubview: seam];-- let wallpaper_container = rounded_view(- container_frame,- container_radius,- color(0.02, 0.03, 0.04, if SHOW_SHELL_BORDER { 0.68 } else { 0.0 }),- true,- );- let _: () = msg_send![wallpaper_container, setAutoresizingMask: 18u64];- let container_layer: *mut AnyObject = msg_send![wallpaper_container, layer];- set_continuous_corners(container_layer);- // Dark hairline where the miniature desktop meets the chrome. Softer than- // the flat-outline pass, because the graphite body now carries the seam.- if SHOW_SHELL_BORDER {- set_layer_border(container_layer, 0.8, color(0.0, 0.0, 0.0, 0.30));- }- let container_bounds: objc2_foundation::NSRect = msg_send![wallpaper_container, bounds];- let wallpaper_view: *mut AnyObject = {- let allocated: *mut AnyObject = msg_send![class!(NSImageView), alloc];- msg_send![allocated, initWithFrame: container_bounds]- };- let _: () = msg_send![wallpaper_view, setImageScaling: 2u64];- let workspace: *mut AnyObject = msg_send![class!(NSWorkspace), sharedWorkspace];- let url: *mut AnyObject = msg_send![workspace, desktopImageURLForScreen: screen];- if !url.is_null() {- let allocated: *mut AnyObject = msg_send![class!(NSImage), alloc];- let wallpaper: *mut AnyObject = msg_send![allocated, initWithContentsOfURL: url];- if !wallpaper.is_null() {- let _: () = msg_send![shell_wallpaper_view, setImage: wallpaper];- let _: () = msg_send![wallpaper_view, setImage: wallpaper];- }- }- let _: () = msg_send![wallpaper_container, addSubview: wallpaper_view];- let _: () = msg_send![content_view, addSubview: wallpaper_container];-- (- glass_frame,- shell_wallpaper_view,- wallpaper_container,- wallpaper_view,- )-}--fn aspect_fill_frame(- bounds: objc2_foundation::NSRect,- image_size: objc2_foundation::NSSize,-) -> objc2_foundation::NSRect {- use objc2_foundation::{NSPoint, NSRect, NSSize};-- if image_size.width <= 0.0 || image_size.height <= 0.0 {- return bounds;- }- let scale = (bounds.size.width / image_size.width).max(bounds.size.height / image_size.height);- let width = image_size.width * scale;- let height = image_size.height * scale;- NSRect::new(- NSPoint::new(- bounds.origin.x + (bounds.size.width - width) / 2.0,- bounds.origin.y + (bounds.size.height - height) / 2.0,- ),- NSSize::new(width, height),- )-}--unsafe fn update_wallpaper_frame() {- use objc2::msg_send;- use objc2::runtime::AnyObject;- use objc2_foundation::{NSRect, NSSize};-- let (shell_container, shell_view, wallpaper_container, wallpaper_view) = {- let guard = HANDLES.lock().unwrap();- let Some(handles) = guard.as_ref() else {- return;- };- (- handles.shell_wallpaper_container as *mut AnyObject,- handles.shell_wallpaper_view as *mut AnyObject,- handles.wallpaper_container as *mut AnyObject,- handles.wallpaper_view as *mut AnyObject,- )- };- for (container, view) in [- (shell_container, shell_view),- (wallpaper_container, wallpaper_view),- ] {- let bounds: NSRect = msg_send![container, bounds];- let image: *mut AnyObject = msg_send![view, image];- let image_size = if image.is_null() {- NSSize::new(0.0, 0.0)- } else {- msg_send![image, size]- };- let frame = aspect_fill_frame(bounds, image_size);- let _: () = msg_send![view, setFrame: frame];- }-}-unsafe extern "C" fn init_cb(ctx: *mut c_void) {use objc2::runtime::AnyObject;use objc2::{class, msg_send};use objc2_foundation::{NSPoint, NSRect, NSSize};let cfg: PipConfig = *Box::from_raw(ctx as *mut PipConfig);++ // Idempotency guard — `start()` should only be called once per+ // process, but cheap to defend against duplicate calls.if HANDLES.lock().unwrap().is_some() {return;}+ // ── Resolve geometry ──+ // AppKit windows use a bottom-left origin in screen coordinates.+ // The CLI flag uses a top-left X11-style origin (since that's the+ // mental model agents have for screenshots). Flip Y here so a+ // `+0+0` flag puts the window in the top-left corner.let screen: *mut AnyObject = msg_send![class!(NSScreen), mainScreen];if screen.is_null() {+ // Headless environment (CI) — skip silently. The daemon keeps+ // running without a PiP window.return;}let screen_frame: NSRect = msg_send![screen, frame];- let minimum = NSSize::new(360.0, 260.0);- let width = (cfg.geometry.width as f64).max(minimum.width);- let height = (cfg.geometry.height as f64).max(minimum.height);++ let w = cfg.geometry.width as f64;+ let h = cfg.geometry.height as f64;+ // Default placement: top-right corner with a 24pt inset, mirroring+ // the macOS conventions for floating utility windows.let inset = 24.0_f64;let (top_left_x, top_left_y) = match (cfg.geometry.x, cfg.geometry.y) {(Some(x), Some(y)) => (x as f64, y as f64),- _ => (screen_frame.size.width - width - inset, inset),+ _ => (screen_frame.size.width - w - inset, inset),};- let bottom_y = screen_frame.size.height - top_left_y - height;- let rect = NSRect::new(- NSPoint::new(top_left_x, bottom_y),- NSSize::new(width, height),- );-- let style_mask: u64 = (1 << 3) | (1 << 7);+ // Convert top-left → bottom-left for AppKit.+ let bottom_y = screen_frame.size.height - top_left_y - h;+ let rect = NSRect::new(NSPoint::new(top_left_x, bottom_y), NSSize::new(w, h));++ // ── NSWindow ──+ // Borderless so the image owns the whole rectangle. No close button+ // / title bar — the window is owned by the daemon session lifecycle.+ // The rounded-corner look comes from a CALayer-backed content view+ // with cornerRadius + masksToBounds; the window itself stays+ // transparent outside the rounded rect.+ // NSWindowStyleMaskBorderless = 0+ let style_mask: u64 = 0;let backing_store_buffered: u64 = 2;- let window: *mut AnyObject = {- let allocated: *mut AnyObject = msg_send![class!(NSPanel), alloc];+ let win: *mut AnyObject = {+ let alloc: *mut AnyObject = msg_send![class!(NSWindow), alloc];msg_send![- allocated,+ alloc,initWithContentRect: rectstyleMask: style_maskbacking: backing_store_buffereddefer: false]};- if window.is_null() {+ if win.is_null() {return;}+ // Transparent backing so the corners outside the CALayer-clipped+ // content view show whatever's underneath — gives the floating-pill+ // look. The shadow comes from AppKit's default `hasShadow: true`.let clear: *mut AnyObject = msg_send![class!(NSColor), clearColor];- let _: () = msg_send![window, setBackgroundColor: clear];- let _: () = msg_send![window, setOpaque: false];- let _: () = msg_send![window, setHasShadow: SHOW_SHELL_BORDER];- // Screen sharing and recordings should include Agent View itself.- let _: () = msg_send![window, setSharingType: 1u64];- let _: () = msg_send![window, setMovableByWindowBackground: true];- let _: () = msg_send![window, setIgnoresMouseEvents: false];- let _: () = msg_send![window, setBecomesKeyOnlyIfNeeded: true];- let _: () = msg_send![window, setFloatingPanel: true];- let _: () = msg_send![window, setLevel: 3i64];- let _: () = msg_send![window, setMinSize: minimum];+ let _: () = msg_send![win, setBackgroundColor: clear];+ let _: () = msg_send![win, setOpaque: false];+ let _: () = msg_send![win, setHasShadow: true];+ // Draggable from anywhere since there's no title bar.+ let _: () = msg_send![win, setMovableByWindowBackground: true];++ // Floating window level (NSFloatingWindowLevel = 3).+ let _: () = msg_send![win, setLevel: 3i64];++ // Collection behavior: visible across all spaces, no Mission+ // Control affordance, never the main / key window.+ // 1<<0 CanJoinAllSpaces | 1<<4 Stationary | 1<<8 FullScreenAuxiliary+ // 1<<6 Transient | 1<<7 IgnoresCyclelet behavior: u64 = (1 << 0) | (1 << 4) | (1 << 8) | (1 << 6) | (1 << 7);- let _: () = msg_send![window, setCollectionBehavior: behavior];- let _: () = msg_send![window, setReleasedWhenClosed: false];- let _: () = msg_send![window, setHidesOnDeactivate: false];+ let _: () = msg_send![win, setCollectionBehavior: behavior];- let content_view: *mut AnyObject = msg_send![window, contentView];+ let _: () = msg_send![win, setReleasedWhenClosed: false];+ let _: () = msg_send![win, setHidesOnDeactivate: false];++ // ── Content view: rounded-corner black backing ──+ // wantsLayer + masksToBounds clips the image view to the rounded+ // rect. The backing CALayer color shows wherever the (proportionally+ // scaled) image leaves gaps above/below or left/right.+ let content_view: *mut AnyObject = msg_send![win, contentView];let _: () = msg_send![content_view, setWantsLayer: true];let content_layer: *mut AnyObject = msg_send![content_view, layer];- let _: () = msg_send![- content_layer,- setCornerRadius: if SHOW_SHELL_BORDER {- CONTAINER_RADIUS- } else {- PIP_CORNER_RADIUS- }+ let _: () = msg_send![content_layer, setCornerRadius: 12.0_f64];+ let _: () = msg_send![content_layer, setMasksToBounds: true];+ let black: *mut AnyObject = msg_send![+ class!(NSColor),+ colorWithCalibratedRed: 0.0_f64+ green: 0.0_f64+ blue: 0.0_f64+ alpha: 1.0_f64];- set_continuous_corners(content_layer);- // The chrome view clips its own children. Masking here too would clip the- // chrome's own outer rim stroke in half and flatten the frame.- let _: () = msg_send![content_layer, setMasksToBounds: false];- set_layer_background(content_layer, color(0.0, 0.0, 0.0, 0.0));- let bounds: NSRect = msg_send![content_view, bounds];- let (shell_wallpaper_container, shell_wallpaper_view, wallpaper_container, wallpaper_view) =- install_wallpaper(content_view, screen, bounds);- let inner_frame = desktop_frame(bounds);+ let black_cg: *mut CGColor = msg_send![black, CGColor];+ let _: () = msg_send![content_layer, setBackgroundColor: black_cg];- let canvas: *mut AnyObject = {- let allocated: *mut AnyObject = msg_send![class!(NSView), alloc];- msg_send![allocated, initWithFrame: inner_frame]+ // ── NSImageView: fills the entire content view ──+ let image_rect = NSRect::new(NSPoint::new(0.0, 0.0), NSSize::new(w, h));+ let image_view: *mut AnyObject = {+ let alloc: *mut AnyObject = msg_send![class!(NSImageView), alloc];+ msg_send![alloc, initWithFrame: image_rect]};- let _: () = msg_send![canvas, setAutoresizingMask: 18u64];- // The content view no longer masks, so the canvas clips its own contents to- // the container silhouette.- let _: () = msg_send![canvas, setWantsLayer: true];- let canvas_layer: *mut AnyObject = msg_send![canvas, layer];- let _: () = msg_send![- canvas_layer,- setCornerRadius: if SHOW_SHELL_BORDER {- 8.5_f64- } else {- PIP_CORNER_RADIUS- }- ];- set_continuous_corners(canvas_layer);- let _: () = msg_send![canvas_layer, setMasksToBounds: true];- let _: () = msg_send![content_view, addSubview: canvas];- install_shell_details(content_view, bounds);- install_shell_interaction(content_view, bounds);-- let delegate = agent_view_delegate_instance();- let _: () = msg_send![window, setDelegate: delegate];- *HANDLES.lock().unwrap() = Some(NativeHandles {- window: window as usize,- canvas_view: canvas as usize,- shell_wallpaper_container: shell_wallpaper_container as usize,- shell_wallpaper_view: shell_wallpaper_view as usize,- wallpaper_container: wallpaper_container as usize,- wallpaper_view: wallpaper_view as usize,- delegate: delegate as usize,- });- *VIEW_MODEL.lock().unwrap() = Some(PipViewModel::new(12));- update_wallpaper_frame();- render_snapshot(&ViewSnapshot {- frames: Vec::new(),- workspaces: Vec::new(),- selected_workspace_id: None,- active_view_id: None,- });- let _: () = msg_send![window, orderFrontRegardless];+ // NSImageScaleProportionallyUpOrDown = 3 (preserve aspect ratio).+ // AppKit types this as NSUInteger — passing signed i64 triggers+ // an objc2 type-encoding panic on macOS 26+.+ let _: () = msg_send![image_view, setImageScaling: 3u64];- tracing::info!(- target: "pip",- "Agent View miniature desktop initialised ({}x{})",- width,- height+ // ── Label overlay: pill at bottom-center ──+ // Container NSView with semi-transparent black backing + rounded+ // corners (half its height for a fully rounded pill). NSTextField+ // sits inside, centered, white text on the dark backing.+ let pill_height = 22.0_f64;+ let pill_inset_x = 16.0_f64;+ let pill_inset_bottom = 10.0_f64;+ let pill_w = (w - pill_inset_x * 2.0).max(60.0);+ let pill_rect = NSRect::new(+ NSPoint::new(pill_inset_x, pill_inset_bottom),+ NSSize::new(pill_w, pill_height),);-}--#[cfg(test)]-mod tests {- use super::*;-- fn workspace(id: &str, target_count: usize, updated_ms: u64) -> PipWorkspaceSummary {- PipWorkspaceSummary {- workspace_id: id.to_owned(),- workspace_label: id.to_owned(),- target_count,- updated_ms,- }- }-- #[test]- fn parses_native_pid_from_exact_window_target() {- assert_eq!(parse_native_pid("window:501:90210"), Some(501));- assert_eq!(parse_native_pid("browser:target:tab"), None);- assert_eq!(parse_native_pid("window:not-a-pid:7"), None);- }-- #[test]- fn truncates_long_titles_without_touching_short_ones() {- assert_eq!(truncate_label("Calculator", 28), "Calculator");- assert_eq!(- truncate_label("abcdefghijklmnopqrstuvwxyz", 8),- "abcdefgh..."- );- }-- #[test]- fn aspect_fill_centers_and_crops_without_distortion() {- use objc2_foundation::{NSPoint, NSRect, NSSize};-- let wide = aspect_fill_frame(- NSRect::new(NSPoint::new(0.0, 0.0), NSSize::new(640.0, 420.0)),- NSSize::new(1440.0, 900.0),- );- assert_eq!(wide.size, NSSize::new(672.0, 420.0));- assert_eq!(wide.origin, NSPoint::new(-16.0, 0.0));-- let tall = aspect_fill_frame(- NSRect::new(NSPoint::new(0.0, 0.0), NSSize::new(380.0, 660.0)),- NSSize::new(1440.0, 900.0),- );- assert_eq!(tall.size, NSSize::new(1056.0, 660.0));- assert_eq!(tall.origin, NSPoint::new(-338.0, 0.0));- }-- #[test]- fn desktop_frame_uses_full_bounds_when_shell_border_is_hidden() {- use objc2_foundation::{NSPoint, NSRect, NSSize};-- let outer = NSRect::new(NSPoint::new(0.0, 0.0), NSSize::new(620.0, 420.0));- let inner = desktop_frame(outer);-- assert_eq!(inner, outer);- }+ let pill: *mut AnyObject = {+ let alloc: *mut AnyObject = msg_send![class!(NSView), alloc];+ msg_send![alloc, initWithFrame: pill_rect]+ };+ let _: () = msg_send![pill, setWantsLayer: true];+ let pill_layer: *mut AnyObject = msg_send![pill, layer];+ let _: () = msg_send![pill_layer, setCornerRadius: pill_height / 2.0];+ let _: () = msg_send![pill_layer, setMasksToBounds: true];+ let pill_bg: *mut AnyObject = msg_send![+ class!(NSColor),+ colorWithCalibratedRed: 0.0_f64+ green: 0.0_f64+ blue: 0.0_f64+ alpha: 0.62_f64+ ];+ let pill_bg_cg: *mut CGColor = msg_send![pill_bg, CGColor];+ let _: () = msg_send![pill_layer, setBackgroundColor: pill_bg_cg];- #[test]- fn session_selector_is_local_compact_and_hidden_for_one_session() {- let one = [workspace("one", 1, 1)];- assert!(session_tabs_layout(602.0, 390.0, &one, Some("one"))- .tabs- .is_empty());- let workspaces = [- workspace("one", 1, 1),- workspace("two", 1, 1),- workspace("three", 1, 1),+ // NSTextField inside the pill — horizontal padding via frame inset.+ let label_rect = NSRect::new(+ NSPoint::new(10.0, 0.0),+ NSSize::new(pill_w - 20.0, pill_height),+ );+ let label: *mut AnyObject = {+ let alloc: *mut AnyObject = msg_send![class!(NSTextField), alloc];+ msg_send![alloc, initWithFrame: label_rect]+ };+ let _: () = msg_send![label, setBezeled: false];+ let _: () = msg_send![label, setDrawsBackground: false];+ let _: () = msg_send![label, setEditable: false];+ let _: () = msg_send![label, setSelectable: false];+ // NSTextAlignment.center = 2 (NSInteger).+ let _: () = msg_send![label, setAlignment: 2i64];+ let white: *mut AnyObject = msg_send![class!(NSColor), whiteColor];+ let _: () = msg_send![label, setTextColor: white];+ let font: *mut AnyObject = msg_send![class!(NSFont), systemFontOfSize: 11.0_f64];+ let _: () = msg_send![label, setFont: font];+ // Initial placeholder text — overwritten on the first frame.+ if let Ok(cstr) = std::ffi::CString::new("waiting for first action…") {+ let ns_str: *mut AnyObject = msg_send![+ class!(NSString),+ stringWithUTF8String: cstr.as_ptr() as *const u8];- let tabs = session_tabs_layout(602.0, 390.0, &workspaces, Some("one"));- assert_eq!(tabs.tabs.len(), 3);- assert!(tabs.tabs.iter().all(|tab| {- tab.rect.x >= 0.0- && tab.rect.y >= 0.0- && tab.rect.x + tab.rect.width <= 602.0- && tab.rect.y + tab.rect.height <= 390.0- }));+ if !ns_str.is_null() {+ let _: () = msg_send![label, setStringValue: ns_str];+ }}- #[test]- fn every_resize_direction_keeps_the_opposite_edges_anchored() {- use objc2_foundation::{NSPoint, NSRect, NSSize};+ let _: () = msg_send![content_view, addSubview: image_view];+ let _: () = msg_send![pill, addSubview: label];+ let _: () = msg_send![content_view, addSubview: pill];- let start = NSRect::new(NSPoint::new(100.0, 100.0), NSSize::new(600.0, 400.0));- let minimum = NSSize::new(360.0, 260.0);- let left = resized_window_frame(start, 40.0, 0.0, RESIZE_LEFT, minimum);- assert_eq!(left.origin.x, 140.0);- assert_eq!(left.size.width, 560.0);- let right = resized_window_frame(start, 40.0, 0.0, RESIZE_RIGHT, minimum);- assert_eq!(right.origin.x, 100.0);- assert_eq!(right.size.width, 640.0);- let bottom = resized_window_frame(start, 0.0, 30.0, RESIZE_BOTTOM, minimum);- assert_eq!(bottom.origin.y, 130.0);- assert_eq!(bottom.size.height, 370.0);- let top = resized_window_frame(start, 0.0, 30.0, RESIZE_TOP, minimum);- assert_eq!(top.origin.y, 100.0);- assert_eq!(top.size.height, 430.0);- let corner = resized_window_frame(start, -25.0, 35.0, RESIZE_LEFT | RESIZE_TOP, minimum);- assert_eq!(corner.origin, NSPoint::new(75.0, 100.0));- assert_eq!(corner.size, NSSize::new(625.0, 435.0));- }+ // Show the window without making it key or activating the app.+ let _: () = msg_send![win, orderFrontRegardless];- #[test]- fn resize_geometry_enforces_the_minimum_without_moving_far_edges() {- use objc2_foundation::{NSPoint, NSRect, NSSize};+ *HANDLES.lock().unwrap() = Some(NativeHandles {+ window: win as usize,+ image_view: image_view as usize,+ label: label as usize,+ });- let start = NSRect::new(NSPoint::new(100.0, 100.0), NSSize::new(600.0, 400.0));- let minimum = NSSize::new(360.0, 260.0);- let frame = resized_window_frame(start, 500.0, 500.0, RESIZE_LEFT | RESIZE_BOTTOM, minimum);- assert_eq!(frame.origin, NSPoint::new(340.0, 240.0));- assert_eq!(frame.size, minimum);- }+ tracing::info!(target: "pip", "PiP window initialised ({}x{})", cfg.geometry.width, cfg.geometry.height);}diff --git a/libs/cua-driver/rust/crates/platform-macos/src/tools/get_config.rs b/libs/cua-driver/rust/crates/platform-macos/src/tools/get_config.rsindex 03feba03ab..a1dc0cbab1 100644--- a/libs/cua-driver/rust/crates/platform-macos/src/tools/get_config.rs+++ b/libs/cua-driver/rust/crates/platform-macos/src/tools/get_config.rs@@ -63,12 +63,11 @@ impl Tool for GetConfigTool {.or_else(|| self.state.cursor_registry.get("default")).map(|s| s.config.enabled).unwrap_or(true);- // Agent View values aren't in DriverConfig — they're file-only since the+ // PiP values aren't in DriverConfig — they're file-only since the// backend is initialised once at startup. Read fresh so the// response reflects whatever set_config (or a direct JSON edit)// last wrote.- let (agent_view_enabled, agent_view_geometry) =- pip_preview::read_agent_view_keys_from_file();+ let (pip_enabled, pip_geometry) = pip_preview::read_pip_keys_from_file();ToolResult::text("cua-driver-rs configuration").with_structured(serde_json::json!({"version": env!("CARGO_PKG_VERSION"),// Maintainer E2E builds set this at compile time so the@@ -83,8 +82,8 @@ impl Tool for GetConfigTool {"agent_cursor": {"enabled": cursor_enabled,},- "agent_view": agent_view_enabled,- "agent_view_geometry": agent_view_geometry,+ "experimental_pip": pip_enabled,+ "experimental_pip_geometry": pip_geometry,}))}}diff --git a/libs/cua-driver/rust/crates/platform-macos/src/tools/set_config.rs b/libs/cua-driver/rust/crates/platform-macos/src/tools/set_config.rsindex 31516025a0..dbf3d225f3 100644--- a/libs/cua-driver/rust/crates/platform-macos/src/tools/set_config.rs+++ b/libs/cua-driver/rust/crates/platform-macos/src/tools/set_config.rs@@ -25,8 +25,8 @@ fn def() -> &'static ToolDef {name: "set_config".into(),description: "Update cua-driver-rs configuration. Changes to \max_image_dimension take effect immediately. The \- agent_view keys are persisted to ~/.cua-driver/config.json and \- take effect on the next daemon restart (Agent View is \+ experimental_pip keys are persisted to ~/.cua-driver/config.json and \+ take effect on the next daemon restart (the PiP backend is \initialised once at startup).\n\nNote: capture_mode is a per-call \param (on get_window_state / click), not a stored setting. Capture \modality is selected by each action's target; the old \@@ -47,16 +47,15 @@ fn def() -> &'static ToolDef {"type": "integer","description": "Max dimension for screenshot resizing (0 = no limit)."},- "agent_view": {+ "experimental_pip": {"type": "boolean",- "description": "Enable the multi-target Agent View. Exact native \- windows and Chrome tabs are grouped automatically by session; no target \- claiming is involved. Applies on next daemon restart."+ "description": "Enable the experimental picture-in-picture preview window. \+ Applies on next daemon restart."},- "agent_view_geometry": {+ "experimental_pip_geometry": {"type": "string",- "description": "Agent View size + optional position in `WxH` or `WxH+X+Y` \- form (e.g. `640x420+24+24`). Applies on next daemon restart."+ "description": "PiP window size + optional position in `WxH` or `WxH+X+Y` \+ form (e.g. `320x200+24+24`). Applies on next daemon restart."}},"additionalProperties": false@@ -105,17 +104,6 @@ impl Tool for SetConfigTool {.filter(|(k, _)| k == name).and_then(|(_, v)| v.as_u64())};- let kv_bool = |name: &str| -> Option<bool> {- kv.as_ref()- .filter(|(k, _)| k == name)- .and_then(|(_, v)| v.as_bool())- };- let kv_str = |name: &str| -> Option<String> {- kv.as_ref()- .filter(|(k, _)| k == name)- .and_then(|(_, v)| v.as_str())- .map(ToOwned::to_owned)- };// Validate max_image_dimension up front so both branches share the// u32 check and we never half-apply.@@ -157,38 +145,36 @@ impl Tool for SetConfigTool {}cfg.max_image_dimension};- // Agent View keys persist to the same config.json but take effect only on+ // PiP keys persist to the same config.json but take effect only on// next daemon restart — the backend is initialised once at startup.- let mut agent_view_note = String::new();- if let Some(enabled) = args- .get("agent_view")- .and_then(|v| v.as_bool())- .or_else(|| kv_bool("agent_view"))- {- if let Err(e) = pip_preview::write_config_key("agent_view", Value::Bool(enabled)) {- return ToolResult::error(format!("failed to persist agent_view: {e}"));+ let mut pip_note = String::new();+ if let Some(enabled) = args.get("experimental_pip").and_then(|v| v.as_bool()) {+ if let Err(e) = pip_preview::write_config_key("experimental_pip", Value::Bool(enabled))+ {+ return ToolResult::error(format!("failed to persist experimental_pip: {e}"));}- agent_view_note =- format!(" — restart cua-driver for agent_view={enabled} to take effect");+ pip_note =+ format!(" — restart cua-driver for experimental_pip={enabled} to take effect");}- if let Some(geom) = args- .opt_str("agent_view_geometry")- .or_else(|| kv_str("agent_view_geometry"))- {+ if let Some(geom) = args.opt_str("experimental_pip_geometry") {// Validate before persisting so the user gets an immediate error.if pip_preview::PipGeometry::parse(&geom).is_none() {return ToolResult::error(format!(- "agent_view_geometry `{geom}` is not a valid WxH or WxH+X+Y string"+ "experimental_pip_geometry `{geom}` is not a valid WxH or WxH+X+Y string"));}- if let Err(e) =- pip_preview::write_config_key("agent_view_geometry", Value::String(geom.clone()))- {- return ToolResult::error(format!("failed to persist agent_view_geometry: {e}"));+ if let Err(e) = pip_preview::write_config_key(+ "experimental_pip_geometry",+ Value::String(geom.clone()),+ ) {+ return ToolResult::error(format!(+ "failed to persist experimental_pip_geometry: {e}"+ ));}- if agent_view_note.is_empty() {- agent_view_note =- format!(" — restart cua-driver for agent_view_geometry={geom} to take effect");+ if pip_note.is_empty() {+ pip_note = format!(+ " — restart cua-driver for experimental_pip_geometry={geom} to take effect"+ );}}let scope_note = if session_id.is_some() {@@ -198,7 +184,7 @@ impl Tool for SetConfigTool {};ToolResult::text(format!("Config updated: max_image_dimension={}{}{}",- effective_dim, scope_note, agent_view_note+ effective_dim, scope_note, pip_note)).with_structured(serde_json::json!({"version": env!("CARGO_PKG_VERSION"),diff --git a/libs/cua-driver/rust/crates/platform-windows/src/pip/mod.rs b/libs/cua-driver/rust/crates/platform-windows/src/pip/mod.rsindex 58e001ce0f..6f8666fced 100644--- a/libs/cua-driver/rust/crates/platform-windows/src/pip/mod.rs+++ b/libs/cua-driver/rust/crates/platform-windows/src/pip/mod.rs@@ -1,1495 +1,24 @@-//! Windows multi-target Agent View.+//! Windows picture-in-picture preview stub.//!-//! The preview owns a Win32 message-loop thread and never activates, so it can-//! remain visible without stealing focus from the application being automated.+//! The Win32 implementation is a follow-up to the experimental macOS+//! drop. The plan is a layered, no-activate `WS_EX_NOACTIVATE |+//! WS_EX_TOPMOST` HWND with a child `STATIC` showing the bitmap of+//! the latest screenshot, fed via `SetWindowPos` for the never-key+//! behavior the spec requires. Tracking issue is linked in the docs.+//!+//! Until that lands, `start()` returns a clear error so `main.rs`+//! can log "PiP unavailable on this platform" and continue without+//! the window.-#[cfg(not(target_os = "windows"))]use pip_preview::{PipBackend, PipBackendFactory, PipConfig};-#[cfg(not(target_os = "windows"))]-impl PipBackendFactory for WindowsPipBackendFactory {- fn start(&self, _cfg: &PipConfig) -> anyhow::Result<Box<dyn PipBackend>> {- Err(anyhow::anyhow!("Windows Agent View requires Windows"))- }-}-pub struct WindowsPipBackendFactory;-#[cfg(target_os = "windows")]-mod native {- use std::sync::atomic::{AtomicBool, AtomicIsize, Ordering};- use std::sync::{mpsc, Arc, Mutex};-- use cursor_overlay::{rasterize_inter_text, TextRaster};- use image::imageops::FilterType;- use pip_preview::{- layout_desktop_with_shell, layout_session_tabs, png_dimensions, LayoutRect, PipBackend,- PipBackendFactory, PipConfig, PipFrame, PipTargetKind, PipViewModel, PipWorkspaceSummary,- ShellStyle, TargetSize,- };- use windows::core::PCWSTR;- use windows::Win32::Foundation::{HWND, LPARAM, LRESULT, POINT, RECT, WPARAM};- use windows::Win32::Graphics::Dwm::{- DwmSetWindowAttribute, DWMWA_SYSTEMBACKDROP_TYPE, DWMWA_WINDOW_CORNER_PREFERENCE,- };- use windows::Win32::Graphics::Gdi::{- BeginPaint, EndPaint, InvalidateRect, ScreenToClient, StretchDIBits, UpdateWindow,- BITMAPINFO, BITMAPINFOHEADER, BI_RGB, DIB_RGB_COLORS, PAINTSTRUCT, SRCCOPY,- };- use windows::Win32::System::LibraryLoader::GetModuleHandleW;- use windows::Win32::UI::WindowsAndMessaging::{- CreateWindowExW, DefWindowProcW, DestroyWindow, DispatchMessageW, GetClientRect,- GetMessageW, GetSystemMetrics, GetWindowLongPtrW, GetWindowRect, LoadCursorW, PostMessageW,- PostQuitMessage, RegisterClassExW, SendMessageW, SetWindowLongPtrW, ShowWindow,- TranslateMessage, CREATESTRUCTW, CS_HREDRAW, CS_VREDRAW, GWLP_USERDATA, GWL_EXSTYLE,- HTBOTTOM, HTBOTTOMLEFT, HTBOTTOMRIGHT, HTCAPTION, HTCLIENT, HTLEFT, HTRIGHT, HTTOP,- HTTOPLEFT, HTTOPRIGHT, HTTRANSPARENT, IDC_ARROW, MA_NOACTIVATE, MSG, SM_CXSCREEN,- SM_CYSCREEN, SW_SHOWNOACTIVATE, WM_APP, WM_CLOSE, WM_DESTROY, WM_ERASEBKGND, WM_LBUTTONUP,- WM_MOUSEACTIVATE, WM_NCCREATE, WM_NCDESTROY, WM_NCHITTEST, WM_PAINT, WM_SIZE, WNDCLASSEXW,- WS_CLIPCHILDREN, WS_EX_NOACTIVATE, WS_EX_TOOLWINDOW, WS_EX_TOPMOST, WS_EX_TRANSPARENT,- WS_POPUP, WS_THICKFRAME,- };-- use super::WindowsPipBackendFactory;-- const REDRAW: u32 = WM_APP + 1;- const SHUTDOWN: u32 = WM_APP + 2;- const SET_INPUT_PASSTHROUGH: u32 = WM_APP + 3;-- struct State {- hwnd: AtomicIsize,- input_passthrough: AtomicBool,- model: Mutex<PipViewModel>,- }-- impl State {- fn post(&self, message: u32) {- let hwnd = self.hwnd.load(Ordering::Acquire);- if hwnd != 0 {- let _ =- unsafe { PostMessageW(HWND(hwnd as *mut _), message, WPARAM(0), LPARAM(0)) };- }- }-- fn set_input_passthrough(&self, passthrough: bool) -> anyhow::Result<()> {- let hwnd = self.hwnd.load(Ordering::Acquire);- if hwnd == 0 {- return Ok(());- }- let applied = unsafe {- SendMessageW(- HWND(hwnd as *mut _),- SET_INPUT_PASSTHROUGH,- WPARAM(usize::from(passthrough)),- LPARAM(0),- )- };- anyhow::ensure!(- applied.0 == 1,- "Agent View rejected the synchronous input-passthrough update"- );- Ok(())- }- }-- struct WindowsPipBackend {- state: Arc<State>,- }-- impl PipBackend for WindowsPipBackend {- fn push_frame(&self, frame: PipFrame) {- self.state.model.lock().unwrap().upsert(frame);- self.state.post(REDRAW);- }-- fn remove_workspace(&self, workspace_id: &str) {- self.state- .model- .lock()- .unwrap()- .remove_workspace(workspace_id);- self.state.post(REDRAW);- }-- fn remove_target(&self, workspace_id: &str, identity_key: &str) {- self.state- .model- .lock()- .unwrap()- .remove_target(workspace_id, identity_key);- self.state.post(REDRAW);- }-- fn set_input_passthrough(&self, passthrough: bool) -> anyhow::Result<()> {- self.state.set_input_passthrough(passthrough)- }-- fn shutdown(self: Box<Self>) {- self.state.post(SHUTDOWN);- }- }-- impl PipBackendFactory for WindowsPipBackendFactory {- fn start(&self, cfg: &PipConfig) -> anyhow::Result<Box<dyn PipBackend>> {- let state = Arc::new(State {- hwnd: AtomicIsize::new(0),- input_passthrough: AtomicBool::new(false),- model: Mutex::new(PipViewModel::new(12)),- });- let thread_state = Arc::clone(&state);- let cfg = cfg.clone();- let (ready_tx, ready_rx) = mpsc::sync_channel(1);- std::thread::Builder::new()- .name("cua-agent-view-windows".to_owned())- .spawn(move || run_window(cfg, thread_state, ready_tx))?;- ready_rx- .recv()- .map_err(|_| anyhow::anyhow!("Agent View UI thread exited during startup"))??;- Ok(Box::new(WindowsPipBackend { state }))- }- }-- fn wide(value: &str) -> Vec<u16> {- value.encode_utf16().chain(std::iter::once(0)).collect()- }-- fn run_window(cfg: PipConfig, state: Arc<State>, ready: mpsc::SyncSender<anyhow::Result<()>>) {- let hwnd = match unsafe { create_window(&cfg, &state) } {- Ok(hwnd) => hwnd,- Err(error) => {- let _ = ready.send(Err(error));- return;- }- };- state.hwnd.store(hwnd.0 as isize, Ordering::Release);- let _ = ready.send(Ok(()));- unsafe {- let _ = ShowWindow(hwnd, SW_SHOWNOACTIVATE);- let _ = UpdateWindow(hwnd);- let mut message = MSG::default();- while GetMessageW(&mut message, None, 0, 0).as_bool() {- let _ = TranslateMessage(&message);- DispatchMessageW(&message);- }- }- state.hwnd.store(0, Ordering::Release);- }-- unsafe fn create_window(cfg: &PipConfig, state: &Arc<State>) -> anyhow::Result<HWND> {- let instance = GetModuleHandleW(None)?;- let class_name = wide("CuaAgentViewWindow");- let class = WNDCLASSEXW {- cbSize: std::mem::size_of::<WNDCLASSEXW>() as u32,- style: CS_HREDRAW | CS_VREDRAW,- lpfnWndProc: Some(window_proc),- hInstance: instance.into(),- hCursor: LoadCursorW(None, IDC_ARROW)?,- lpszClassName: PCWSTR(class_name.as_ptr()),- ..Default::default()- };- if RegisterClassExW(&class) == 0 {- let error = windows::core::Error::from_win32();- if error.code().0 != 0x8007_0582u32 as i32 {- return Err(error.into());- }- }-- let width = cfg.geometry.width.max(320) as i32;- let height = cfg.geometry.height.max(240) as i32;- let x = cfg- .geometry- .x- .unwrap_or_else(|| (GetSystemMetrics(SM_CXSCREEN) - width - 24).max(0));- let y = cfg- .geometry- .y- .unwrap_or(24)- .min((GetSystemMetrics(SM_CYSCREEN) - height).max(0));- let title = wide(&cfg.title);- let state_ptr = Box::into_raw(Box::new(Arc::clone(state)));- let hwnd = match CreateWindowExW(- WS_EX_TOPMOST | WS_EX_NOACTIVATE | WS_EX_TOOLWINDOW,- PCWSTR(class_name.as_ptr()),- PCWSTR(title.as_ptr()),- WS_POPUP | WS_THICKFRAME | WS_CLIPCHILDREN,- x,- y,- width,- height,- None,- None,- instance,- Some(state_ptr.cast()),- ) {- Ok(hwnd) => hwnd,- Err(error) => {- drop(Box::from_raw(state_ptr));- return Err(error.into());- }- };- let corner = 2i32; // DWMWCP_ROUND- let _ = DwmSetWindowAttribute(- hwnd,- DWMWA_WINDOW_CORNER_PREFERENCE,- (&corner as *const i32).cast(),- std::mem::size_of_val(&corner) as u32,- );- let backdrop = 3i32; // DWMSBT_TRANSIENTWINDOW- let _ = DwmSetWindowAttribute(- hwnd,- DWMWA_SYSTEMBACKDROP_TYPE,- (&backdrop as *const i32).cast(),- std::mem::size_of_val(&backdrop) as u32,- );- Ok(hwnd)- }-- unsafe extern "system" fn window_proc(- hwnd: HWND,- message: u32,- wparam: WPARAM,- lparam: LPARAM,- ) -> LRESULT {- if message == WM_NCCREATE {- let create = &*(lparam.0 as *const CREATESTRUCTW);- SetWindowLongPtrW(hwnd, GWLP_USERDATA, create.lpCreateParams as isize);- return LRESULT(1);- }- let state_ptr = GetWindowLongPtrW(hwnd, GWLP_USERDATA) as *mut Arc<State>;- match message {- REDRAW | WM_SIZE => {- let _ = InvalidateRect(hwnd, None, false);- LRESULT(0)- }- WM_PAINT => {- if !state_ptr.is_null() {- paint(hwnd, &*state_ptr);- }- LRESULT(0)- }- WM_ERASEBKGND => LRESULT(1),- WM_MOUSEACTIVATE => LRESULT(MA_NOACTIVATE as isize),- SET_INPUT_PASSTHROUGH => {- let passthrough = wparam.0 != 0;- if !state_ptr.is_null() {- (&*state_ptr)- .input_passthrough- .store(passthrough, Ordering::Release);- }- let style = GetWindowLongPtrW(hwnd, GWL_EXSTYLE);- SetWindowLongPtrW(- hwnd,- GWL_EXSTYLE,- extended_style_with_input_passthrough(style, passthrough),- );- LRESULT(1)- }- WM_LBUTTONUP => {- if !state_ptr.is_null() {- let x = lparam.0 as i16 as i32;- let y = (lparam.0 >> 16) as i16 as i32;- let mut client = RECT::default();- if GetClientRect(hwnd, &mut client).is_ok() {- let state = &*state_ptr;- let mut model = state.model.lock().unwrap();- let workspaces = model.workspaces();- let tabs = session_tabs_for_width(- client.right - client.left,- client.bottom - client.top,- &workspaces,- model.selected_workspace_id(),- );- let changed = tabs- .hit_test(f64::from(x), f64::from(y))- .map(|workspace_id| model.select_workspace(workspace_id))- .unwrap_or(false);- if changed {- let _ = InvalidateRect(hwnd, None, false);- }- }- }- LRESULT(0)- }- WM_NCHITTEST => {- if !state_ptr.is_null() && (&*state_ptr).input_passthrough.load(Ordering::Acquire) {- return LRESULT(HTTRANSPARENT as isize);- }- let hit = DefWindowProcW(hwnd, message, wparam, lparam);- if hit.0 == HTCLIENT as isize {- let screen_x = lparam.0 as i16 as i32;- let screen_y = (lparam.0 >> 16) as i16 as i32;- let mut window = RECT::default();- if GetWindowRect(hwnd, &mut window).is_ok() {- let border = 8;- let left = screen_x - window.left < border;- let right = window.right - screen_x <= border;- let top = screen_y - window.top < border;- let bottom = window.bottom - screen_y <= border;- let resize_hit = match (left, right, top, bottom) {- (true, _, true, _) => Some(HTTOPLEFT),- (_, true, true, _) => Some(HTTOPRIGHT),- (true, _, _, true) => Some(HTBOTTOMLEFT),- (_, true, _, true) => Some(HTBOTTOMRIGHT),- (true, _, _, _) => Some(HTLEFT),- (_, true, _, _) => Some(HTRIGHT),- (_, _, true, _) => Some(HTTOP),- (_, _, _, true) => Some(HTBOTTOM),- _ => None,- };- if let Some(resize_hit) = resize_hit {- return LRESULT(resize_hit as isize);- }- let mut client_point = POINT {- x: screen_x,- y: screen_y,- };- let mut client = RECT::default();- let tabs_hit = if !state_ptr.is_null()- && ScreenToClient(hwnd, &mut client_point).as_bool()- && GetClientRect(hwnd, &mut client).is_ok()- {- let model = (&*state_ptr).model.lock().unwrap();- let workspaces = model.workspaces();- session_tabs_for_width(- client.right - client.left,- client.bottom - client.top,- &workspaces,- model.selected_workspace_id(),- )- .hit_test(f64::from(client_point.x), f64::from(client_point.y))- .is_some()- } else {- false- };- if tabs_hit {- return LRESULT(HTCLIENT as isize);- }- if screen_y - window.top < 28 {- return LRESULT(HTCAPTION as isize);- }- }- }- hit- }- SHUTDOWN | WM_CLOSE => {- let _ = DestroyWindow(hwnd);- LRESULT(0)- }- WM_DESTROY => {- PostQuitMessage(0);- LRESULT(0)- }- WM_NCDESTROY => {- SetWindowLongPtrW(hwnd, GWLP_USERDATA, 0);- if !state_ptr.is_null() {- drop(Box::from_raw(state_ptr));- }- DefWindowProcW(hwnd, message, wparam, lparam)- }- _ => DefWindowProcW(hwnd, message, wparam, lparam),- }- }-- unsafe fn paint(hwnd: HWND, state: &Arc<State>) {- let mut ps = PAINTSTRUCT::default();- let hdc = BeginPaint(hwnd, &mut ps);- let mut rect = RECT::default();- if GetClientRect(hwnd, &mut rect).is_err() {- let _ = EndPaint(hwnd, &ps);- return;- }- let width = (rect.right - rect.left).max(1) as u32;- let height = (rect.bottom - rect.top).max(1) as u32;- let (frames, workspaces, selected_workspace_id, active_view_id) = {- let model = state.model.lock().unwrap();- (- model- .selected_frames()- .into_iter()- .cloned()- .collect::<Vec<_>>(),- model.workspaces(),- model.selected_workspace_id().map(str::to_owned),- model.active_view_id().map(str::to_owned),- )- };- let pixels = render_view(- width,- height,- &frames,- &workspaces,- selected_workspace_id.as_deref(),- active_view_id.as_deref(),- );- let info = BITMAPINFO {- bmiHeader: BITMAPINFOHEADER {- biSize: std::mem::size_of::<BITMAPINFOHEADER>() as u32,- biWidth: width as i32,- biHeight: -(height as i32),- biPlanes: 1,- biBitCount: 32,- biCompression: BI_RGB.0,- biSizeImage: pixels.len() as u32,- ..Default::default()- },- ..Default::default()- };- StretchDIBits(- hdc,- 0,- 0,- width as i32,- height as i32,- 0,- 0,- width as i32,- height as i32,- Some(pixels.as_ptr().cast()),- &info,- DIB_RGB_COLORS,- SRCCOPY,- );- let _ = EndPaint(hwnd, &ps);- }-- fn render_view(- width: u32,- height: u32,- frames: &[PipFrame],- workspaces: &[PipWorkspaceSummary],- selected_workspace_id: Option<&str>,- active_view_id: Option<&str>,- ) -> Vec<u8> {- let mut canvas = Canvas::new(width, height);- canvas.smoked_shell();- let show_switcher = workspaces.len() > 1;- let desktop = desktop_rect(width, height, show_switcher);- canvas.wallpaper(desktop);- let sizes = frames- .iter()- .map(|frame| {- png_dimensions(&frame.png_bytes).unwrap_or(TargetSize {- width: 16,- height: 10,- })- })- .collect::<Vec<_>>();- let layout =- layout_desktop_with_shell(desktop.2 as f64, desktop.3 as f64, &sizes, ShellStyle::None);- canvas.border((0, 0, width as i32, height as i32), 14, [19, 12, 7, 225]);- canvas.border(- (1, 1, width as i32 - 2, height as i32 - 2),- 13,- [211, 184, 151, 118],- );- canvas.border(- (2, 2, width as i32 - 4, height as i32 - 4),- 12,- [108, 82, 57, 105],- );- canvas.border(desktop, 10, [181, 151, 122, 138]);- if show_switcher {- canvas.switcher(- switcher_rect(width as i32),- workspaces,- selected_workspace_id,- );- }- for (frame, target) in frames.iter().zip(&layout.targets) {- let rect = offset_pixels(target.content, desktop.0, desktop.1);- canvas.shadow(rect);- canvas.fill(rect, 7, [244, 247, 251, 255]);- if let Ok(image) = image::load_from_memory(&frame.png_bytes) {- let image = image- .resize_exact(rect.2 as u32, rect.3 as u32, FilterType::Lanczos3)- .to_rgba8();- canvas.blit(&image, rect, 7);- }- canvas.border(rect, 7, [40, 52, 72, 76]);- if active_view_id == Some(frame.target.view_id().as_str()) {- canvas.border(- (rect.0 - 3, rect.1 - 3, rect.2 + 6, rect.3 + 6),- 9,- [230, 158, 73, 232],- );- }- }- canvas.data- }-- /// Index of the card whose capture is newest, which the taskbar marks as- /// the foreground app.- ///- /// Ties resolve to the earliest card so one repaint never reorders the- /// highlight for an unchanged set of frames.- fn active_target_index(frames: &[PipFrame]) -> Option<usize> {- frames- .iter()- .enumerate()- .max_by_key(|(index, frame)| (frame.timestamp_ms, std::cmp::Reverse(*index)))- .map(|(index, _)| index)- }-- /// `HH:MM` for one epoch-millisecond capture stamp, in UTC.- ///- /// The clock reads the newest card's capture time rather than the wall- /// clock so a rendered frame stays reproducible from its inputs alone.- fn clock_label(timestamp_ms: u64) -> String {- let minutes = timestamp_ms / 60_000;- format!("{:02}:{:02}", minutes / 60 % 24, minutes % 60)- }-- /// Separable box blur over a tightly packed 3-channel buffer.- fn box_blur(source: &[u8], width: usize, height: usize, radius: usize) -> Vec<u8> {- let mut pass = source.to_vec();- let mut out = vec![0u8; source.len()];- for vertical in [false, true] {- let (major, minor) = if vertical {- (width, height)- } else {- (height, width)- };- for outer in 0..major {- for inner in 0..minor {- let low = inner.saturating_sub(radius);- let high = (inner + radius).min(minor - 1);- let taps = (high - low + 1) as u32;- for channel in 0..3 {- let mut sum = 0u32;- for tap in low..=high {- let at = if vertical {- tap * width + outer- } else {- outer * width + tap- };- sum += pass[at * 3 + channel] as u32;- }- let at = if vertical {- inner * width + outer- } else {- outer * width + inner- };- out[at * 3 + channel] = (sum / taps) as u8;- }- }- }- std::mem::swap(&mut pass, &mut out);- }- pass- }-- fn desktop_rect(width: u32, height: u32, show_switcher: bool) -> (i32, i32, i32, i32) {- const SIDE_INSET: i32 = 8;- const BOTTOM_INSET: i32 = 9;- let top_inset = if show_switcher { 36 } else { 14 };- (- SIDE_INSET,- top_inset,- (width as i32 - SIDE_INSET * 2).max(1),- (height as i32 - top_inset - BOTTOM_INSET).max(1),- )- }-- fn switcher_rect(width: i32) -> (i32, i32, i32, i32) {- (12, 8, (width - 24).max(1), 28)- }-- fn session_tabs_for_width(- width: i32,- height: i32,- workspaces: &[PipWorkspaceSummary],- selected_workspace_id: Option<&str>,- ) -> pip_preview::SessionTabsLayout {- layout_session_tabs(- LayoutRect {- x: 12.0,- y: 8.0,- width: f64::from((width - 24).max(1)),- height: f64::from(height.max(1)),- },- workspaces,- selected_workspace_id,- )- }-- fn contains(rect: (i32, i32, i32, i32), x: i32, y: i32) -> bool {- x >= rect.0 && y >= rect.1 && x < rect.0 + rect.2 && y < rect.1 + rect.3- }-- fn extended_style_with_input_passthrough(style: isize, passthrough: bool) -> isize {- let transparent = WS_EX_TRANSPARENT.0 as isize;- if passthrough {- style | transparent- } else {- style & !transparent- }- }-- fn pixels(rect: LayoutRect) -> (i32, i32, i32, i32) {- (- rect.x.round() as i32,- rect.y.round() as i32,- rect.width.round().max(1.0) as i32,- rect.height.round().max(1.0) as i32,- )- }-- fn offset_pixels(rect: LayoutRect, x: i32, y: i32) -> (i32, i32, i32, i32) {- let rect = pixels(rect);- (rect.0 + x, rect.1 + y, rect.2, rect.3)- }-- struct Canvas {- width: u32,- height: u32,- data: Vec<u8>,- }-- impl Canvas {- fn new(width: u32, height: u32) -> Self {- Self {- width,- height,- data: vec![0; width as usize * height as usize * 4],- }- }-- fn wallpaper(&mut self, rect: (i32, i32, i32, i32)) {- let w = rect.2.max(1) as f32;- let h = rect.3.max(1) as f32;- for y in 0..rect.3 {- for x in 0..rect.2 {- if !Self::inside(x, y, rect.2, rect.3, 10) {- continue;- }- let nx = x as f32 / w;- let ny = y as f32 / h;- let upper_glow =- (1.0 - (((nx - 0.72).powi(2) + (ny - 0.08).powi(2)).sqrt() * 1.5)).max(0.0);- let lower_glow =- (1.0 - (((nx - 0.18).powi(2) + (ny - 0.88).powi(2)).sqrt() * 1.8)).max(0.0);- let ribbon =- (1.0 - ((ny - 0.64 + 0.08 * (nx * 5.4).sin()).abs() * 6.5)).max(0.0);- self.set(- rect.0 + x,- rect.1 + y,- [- (35.0 + 45.0 * upper_glow + 25.0 * lower_glow + 19.0 * ribbon) as u8,- (25.0 + 36.0 * upper_glow + 17.0 * lower_glow + 11.0 * ribbon) as u8,- (18.0 + 22.0 * upper_glow + 10.0 * lower_glow + 5.0 * ribbon) as u8,- 255,- ],- );- }- }- }-- fn smoked_shell(&mut self) {- let width = self.width as i32;- let height = self.height as i32;- self.fill((0, 0, width, height), 0, [15, 10, 7, 255]);- self.fill((0, 0, width, height), 14, [24, 16, 10, 255]);-- let header_height = (height / 12).clamp(24, 42);- self.fill(- (2, 2, width.saturating_sub(4), header_height),- 12,- [66, 50, 36, 92],- );- self.fill((16, 3, width.saturating_sub(32), 1), 0, [235, 211, 181, 92]);-- // Fine, low-contrast grain keeps the painted fallback from looking- // flat when the system backdrop is unavailable (for example RDP).- for y in (8..height.saturating_sub(8)).step_by(4) {- for x in ((y & 7)..width.saturating_sub(8)).step_by(8) {- self.blend(x, y, [205, 181, 155, 7]);- }- }- }-- fn switcher(- &mut self,- rect: (i32, i32, i32, i32),- workspaces: &[PipWorkspaceSummary],- selected_workspace_id: Option<&str>,- ) {- let tabs = layout_session_tabs(- LayoutRect {- x: f64::from(rect.0),- y: f64::from(rect.1),- width: f64::from(rect.2),- height: f64::from(rect.3),- },- workspaces,- selected_workspace_id,- );- for tab in tabs.tabs {- let tab_rect = offset_pixels(tab.rect, 0, 0);- self.shadow(tab_rect);- self.fill(- tab_rect,- 9,- [42, 31, 24, if tab.selected { 242 } else { 204 }],- );- self.border(- tab_rect,- 9,- [- tab.accent.0,- tab.accent.1,- tab.accent.2,- if tab.selected { 235 } else { 128 },- ],- );- }- }-- fn set(&mut self, x: i32, y: i32, color: [u8; 4]) {- if x < 0 || y < 0 || x >= self.width as i32 || y >= self.height as i32 {- return;- }- let at = (y as usize * self.width as usize + x as usize) * 4;- self.data[at..at + 4].copy_from_slice(&color);- }-- fn blend(&mut self, x: i32, y: i32, color: [u8; 4]) {- if x < 0 || y < 0 || x >= self.width as i32 || y >= self.height as i32 {- return;- }- let at = (y as usize * self.width as usize + x as usize) * 4;- let alpha = color[3] as u16;- for channel in 0..3 {- self.data[at + channel] = ((color[channel] as u16 * alpha- + self.data[at + channel] as u16 * (255 - alpha))- / 255) as u8;- }- self.data[at + 3] = 255;- }-- fn inside(x: i32, y: i32, width: i32, height: i32, radius: i32) -> bool {- let dx = if x < radius {- radius - x- } else if x >= width - radius {- x - width + radius + 1- } else {- 0- };- let dy = if y < radius {- radius - y- } else if y >= height - radius {- y - height + radius + 1- } else {- 0- };- dx == 0 || dy == 0 || dx * dx + dy * dy <= radius * radius- }-- fn fill(&mut self, rect: (i32, i32, i32, i32), radius: i32, color: [u8; 4]) {- for y in 0..rect.3 {- for x in 0..rect.2 {- if Self::inside(x, y, rect.2, rect.3, radius) {- self.blend(rect.0 + x, rect.1 + y, color);- }- }- }- }-- fn border(&mut self, rect: (i32, i32, i32, i32), radius: i32, color: [u8; 4]) {- for y in 0..rect.3 {- for x in 0..rect.2 {- let outer = Self::inside(x, y, rect.2, rect.3, radius);- let inner = x > 0- && y > 0- && x < rect.2 - 1- && y < rect.3 - 1- && Self::inside(x - 1, y - 1, rect.2 - 2, rect.3 - 2, radius - 1);- if outer && !inner {- self.blend(rect.0 + x, rect.1 + y, color);- }- }- }- }-- fn shadow(&mut self, rect: (i32, i32, i32, i32)) {- for spread in (1..=8).rev() {- self.border(- (- rect.0 - spread,- rect.1 - spread + 2,- rect.2 + spread * 2,- rect.3 + spread * 2,- ),- 7 + spread,- [8, 24, 49, (26 / spread) as u8],- );- }- }-- fn blit(&mut self, image: &image::RgbaImage, rect: (i32, i32, i32, i32), radius: i32) {- for y in 0..rect.3 {- for x in 0..rect.2 {- if Self::inside(x, y, rect.2, rect.3, radius) {- let p = image.get_pixel(x as u32, y as u32).0;- self.blend(rect.0 + x, rect.1 + y, [p[2], p[1], p[0], p[3]]);- }- }- }- }-- fn icon(&mut self, rect: (i32, i32, i32, i32), kind: PipTargetKind, index: usize) {- let colors = match kind {- PipTargetKind::BrowserTab => ([31, 180, 210, 255], [205, 91, 28, 255]),- PipTargetKind::NativeWindow => [- ([205, 91, 138, 255], [131, 53, 92, 255]),- ([78, 166, 66, 255], [35, 104, 26, 255]),- ([43, 145, 235, 255], [21, 84, 180, 255]),- ][index % 3],- };- self.fill(rect, (rect.2 / 4).max(3), colors.0);- let inset = (rect.2 / 4).max(3);- self.fill(- (- rect.0 + inset,- rect.1 + inset,- rect.2 - inset * 2,- rect.3 - inset * 2,- ),- 3,- colors.1,- );- }-- /// Acrylic-like backdrop for the edge-anchored taskbar band.- ///- /// The band is a material, not a card: it blurs whatever Agent View- /// already composited beneath it, tints the result, and adds fine- /// noise, which is why it runs after the wallpaper and the target- /// cards. Painting is clipped to the miniature desktop so the bar- /// keeps the desktop's rounded bottom corners.- fn taskbar_material(&mut self, rect: (i32, i32, i32, i32), clip: (i32, i32, i32, i32)) {- const BLUR_RADIUS: usize = 5;- /// Tint strength out of 255. Leaving some backdrop through is what- /// separates acrylic from a flat fill.- const TINT_ALPHA: u32 = 196;- const TINT: [u32; 3] = [44, 34, 27];-- let width = rect.2.max(0) as usize;- let height = rect.3.max(0) as usize;- if width == 0 || height == 0 {- return;- }- // Sample with clamped coordinates so the blur never smears the- // surrounding shell frame into the band.- let mut backdrop = vec![0u8; width * height * 3];- for y in 0..height {- for x in 0..width {- let sx = (rect.0 + x as i32).clamp(0, self.width as i32 - 1);- let sy = (rect.1 + y as i32).clamp(0, self.height as i32 - 1);- let from = (sy as usize * self.width as usize + sx as usize) * 4;- let to = (y * width + x) * 3;- backdrop[to..to + 3].copy_from_slice(&self.data[from..from + 3]);- }- }- let blurred = box_blur(&backdrop, width, height, BLUR_RADIUS);-- for y in 0..height {- for x in 0..width {- let px = rect.0 + x as i32;- let py = rect.1 + y as i32;- if !Self::inside(px - clip.0, py - clip.1, clip.2, clip.3, 10) {- continue;- }- let sample = (y * width + x) * 3;- let mut color = [0u8; 4];- for channel in 0..3 {- color[channel] = ((blurred[sample + channel] as u32 * (255 - TINT_ALPHA)- + TINT[channel] * TINT_ALPHA)- / 255) as u8;- }- color[3] = 255;- self.set(px, py, color);- // Deterministic grain, the visual signature of acrylic.- if (x ^ y) & 3 == 0 {- self.blend(px, py, [188, 170, 150, 6]);- }- }- }-- // A single bright hairline along the top edge reads as the lit- // lip of the bar and keeps it separated from the wallpaper.- for x in 0..rect.2 {- let px = rect.0 + x;- if Self::inside(px - clip.0, rect.1 - clip.1, clip.2, clip.3, 10) {- self.blend(px, rect.1, [214, 190, 162, 96]);- }- }- }-- /// Four-pane launcher glyph anchored at the head of the icon cluster.- fn start_glyph(&mut self, rect: (i32, i32, i32, i32)) {- let gap = (rect.2 / 9).max(1);- let pane_width = ((rect.2 - gap) / 2).max(1);- let pane_height = ((rect.3 - gap) / 2).max(1);- // Center the glyph inside its slot, which is sized like an app icon.- let x = rect.0 + (rect.2 - (pane_width * 2 + gap)) / 2;- let y = rect.1 + (rect.3 - (pane_height * 2 + gap)) / 2;- for row in 0..2 {- for column in 0..2 {- self.fill(- (- x + column * (pane_width + gap),- y + row * (pane_height + gap),- pane_width,- pane_height,- ),- 1,- [230, 158, 73, 235],- );- }- }- }-- /// Running-app mark beneath one taskbar icon.- ///- /// The foreground card gets the full accent pill; the rest get a- /// shorter, dimmer mark, matching how a Windows taskbar distinguishes- /// the active window from other running ones.- fn running_indicator(&mut self, rect: (i32, i32, i32, i32), active: bool) {- let (width, color) = if active {- (rect.2, [230, 158, 73, 255])- } else {- ((rect.2 / 2).max(2), [176, 158, 138, 190])- };- self.fill(- (rect.0 + (rect.2 - width) / 2, rect.1, width, rect.3),- rect.3 / 2,- color,- );- }-- /// Trailing status band: a chevron, a battery mark, and the clock.- ///- /// Every element is dropped rather than crowded when the band is too- /// narrow, so a small Agent View degrades instead of overlapping the- /// centered icon cluster.- fn tray(&mut self, rect: (i32, i32, i32, i32), timestamp_ms: Option<u64>) {- const INK: [u8; 4] = [224, 210, 190, 235];- let padding = 8;- let mut right = rect.0 + rect.2 - padding;-- let font_size = (rect.3 as f32 * 0.2).clamp(8.0, 13.0);- let clock = timestamp_ms- .map(clock_label)- .and_then(|label| rasterize_inter_text(&label, font_size));- if let Some(raster) = clock {- if raster.width as i32 + padding * 2 <= rect.2 {- right -= raster.width as i32;- let y = rect.1 + (rect.3 - raster.height as i32) / 2;- self.text(&raster, right, y, INK);- }- }-- let center_y = rect.1 + rect.3 / 2;- // Battery: an outline pill with a nub, drawn only when it fits- // clear of the clock.- let battery_width = 14;- if right - (battery_width + 10) >= rect.0 + padding {- right -= battery_width + 10;- self.border((right, center_y - 4, battery_width, 8), 2, INK);- self.fill((right + battery_width, center_y - 2, 2, 4), 1, INK);- }- // Chevron: the "show hidden icons" affordance.- if right - 12 >= rect.0 + padding {- right -= 12;- for step in 0..4 {- self.blend(right + step, center_y + 1 - step, INK);- self.blend(right + 6 - step, center_y + 1 - step, INK);- }- }- }-- /// Composite one rasterized text mask in the buffer's channel order.- fn text(&mut self, raster: &TextRaster, x: i32, y: i32, color: [u8; 4]) {- for row in 0..raster.height {- for column in 0..raster.width {- let coverage = raster.coverage[(row * raster.width + column) as usize];- if coverage == 0 {- continue;- }- let alpha = (color[3] as u16 * coverage as u16 / 255) as u8;- self.blend(- x + column as i32,- y + row as i32,- [color[0], color[1], color[2], alpha],- );- }- }- }- }-- #[cfg(test)]- mod tests {- use std::io::Cursor;-- use super::*;- use image::{ImageFormat, Rgba, RgbaImage};- use pip_preview::PipTarget;-- fn solid_png(width: u32, height: u32, color: [u8; 4]) -> Vec<u8> {- let image = RgbaImage::from_pixel(width, height, Rgba(color));- let mut bytes = Cursor::new(Vec::new());- image.write_to(&mut bytes, ImageFormat::Png).unwrap();- bytes.into_inner()- }-- fn pixel(data: &[u8], width: u32, x: u32, y: u32) -> [u8; 4] {- let offset = ((y * width + x) * 4) as usize;- data[offset..offset + 4].try_into().unwrap()- }-- fn frame(- workspace: &str,- id: &str,- kind: PipTargetKind,- png_bytes: Vec<u8>,- timestamp_ms: u64,- ) -> PipFrame {- PipFrame {- target: PipTarget {- workspace_id: workspace.to_owned(),- workspace_label: workspace.to_owned(),- target_id: id.to_owned(),- identity_key: id.to_owned(),- target_kind: kind,- target_label: id.to_owned(),- native_container: None,- },- png_bytes,- action_label: "click".to_owned(),- timestamp_ms,- cursor_position: None,- }- }-- fn workspace(id: &str, target_count: usize, updated_ms: u64) -> PipWorkspaceSummary {- PipWorkspaceSummary {- workspace_id: id.to_owned(),- workspace_label: id.to_owned(),- target_count,- updated_ms,- }- }-- #[test]- fn renderer_tracks_each_target_and_requested_size() {- let workspaces = [workspace("workspace", 2, 1)];- let data = render_view(- 480,- 320,- &[- frame(- "workspace",- "edge",- PipTargetKind::BrowserTab,- Vec::new(),- 1,- ),- frame(- "workspace",- "notes",- PipTargetKind::NativeWindow,- Vec::new(),- 1,- ),- ],- &workspaces,- Some("workspace"),- None,- );- assert_eq!(data.len(), 480 * 320 * 4);- assert!(data.chunks_exact(4).all(|pixel| pixel[3] == 255));- }-- #[test]- fn renderer_preserves_target_pixels_across_mixed_form_factors() {- let frames = [- frame(- "workspace",- "wide",- PipTargetKind::BrowserTab,- solid_png(80, 32, [220, 30, 20, 255]),- 1,- ),- frame(- "workspace",- "tall",- PipTargetKind::NativeWindow,- solid_png(24, 72, [20, 90, 230, 255]),- 1,- ),- ];- let width = 520;- let height = 360;- let workspaces = [workspace("workspace", 2, 1)];- let data = render_view(width, height, &frames, &workspaces, Some("workspace"), None);- let desktop = desktop_rect(width, height, false);- let layout = layout_desktop_with_shell(- desktop.2 as f64,- desktop.3 as f64,- &[- TargetSize {- width: 80,- height: 32,- },- TargetSize {- width: 24,- height: 72,- },- ],- ShellStyle::EdgeTaskbar,- );-- let wide = offset_pixels(layout.targets[0].content, desktop.0, desktop.1);- let tall = offset_pixels(layout.targets[1].content, desktop.0, desktop.1);- assert_eq!(- pixel(- &data,- width,- (wide.0 + wide.2 / 2) as u32,- (wide.1 + wide.3 / 2) as u32,- ),- [20, 30, 220, 255]- );- assert_eq!(- pixel(- &data,- width,- (tall.0 + tall.2 / 2) as u32,- (tall.1 + tall.3 / 2) as u32,- ),- [230, 90, 20, 255]- );- assert!(wide.2 > wide.3);- assert!(tall.3 > tall.2);- assert!(wide.0 >= desktop.0 && wide.1 >= desktop.1);- assert!(tall.0 + tall.2 <= desktop.0 + desktop.2);- assert!(tall.1 + tall.3 <= desktop.1 + desktop.3);- }-- #[test]- fn renderer_separates_dark_shell_from_inset_desktop() {- let width = 360;- let data = render_view(width, 240, &[], &[], None, None);- let shell = pixel(&data, width, 4, 100);- let desktop = pixel(&data, width, 180, 100);- let highlight = pixel(&data, width, 180, 1);- let brightness = |color: [u8; 4]| color[0] as u16 + color[1] as u16 + color[2] as u16;-- assert_ne!(shell, desktop);- assert!(brightness(shell) < brightness(desktop));- assert!(shell[0] > shell[1] && shell[1] > shell[2]);- assert!(brightness(highlight) > brightness(shell));- assert!(highlight[0] > highlight[1] && highlight[1] > highlight[2]);- }-- #[test]- fn switcher_is_hidden_for_one_workspace_and_cycles_without_target_activation() {- let width = 420;- let one = [workspace("agent-a", 1, 1)];- let two = [workspace("agent-b", 1, 2), workspace("agent-a", 1, 1)];- let single = render_view(width, 280, &[], &one, Some("agent-a"), None);- let multiple = render_view(width, 280, &[], &two, Some("agent-a"), None);- let switcher = switcher_rect(width as i32);- let sample_x = (switcher.0 + switcher.2 / 2) as u32;- let sample_y = (switcher.1 + switcher.3 / 2) as u32;-- assert_eq!(desktop_rect(width, 280, false).1, 14);- assert_eq!(desktop_rect(width, 280, true).1, 36);- assert_ne!(- pixel(&single, width, sample_x, sample_y),- pixel(&multiple, width, sample_x, sample_y)- );-- let mut model = PipViewModel::new(4);- model.upsert(frame(- "agent-a",- "window-a",- PipTargetKind::NativeWindow,- Vec::new(),- 1,- ));- model.upsert(frame(- "agent-b",- "window-b",- PipTargetKind::NativeWindow,- Vec::new(),- 2,- ));- assert!(model.select_workspace("agent-a"));- assert_eq!(model.selected_frames()[0].target.workspace_id, "agent-a");- assert!(model.select_next_workspace());- assert_eq!(model.selected_frames()[0].target.workspace_id, "agent-b");- }-- #[test]- fn exact_target_removal_and_workspace_cleanup_redraw_the_selected_session() {- let mut model = PipViewModel::new(6);- model.upsert(frame(- "agent-a",- "wide",- PipTargetKind::BrowserTab,- Vec::new(),- 1,- ));- model.upsert(frame(- "agent-a",- "tall",- PipTargetKind::NativeWindow,- Vec::new(),- 2,- ));- model.upsert(frame(- "agent-b",- "other",- PipTargetKind::NativeWindow,- Vec::new(),- 3,- ));- assert!(model.select_workspace("agent-a"));- assert!(model.remove_target("agent-a", "wide"));- assert_eq!(model.selected_frames().len(), 1);- assert_eq!(model.selected_frames()[0].target.identity_key, "tall");- assert_eq!(model.remove_workspace("agent-a").len(), 1);- assert_eq!(model.selected_workspace_id(), Some("agent-b"));- }-- fn taskbar_layout(width: u32, height: u32, targets: usize) -> pip_preview::DesktopLayout {- let desktop = desktop_rect(width, height, false);- let sizes = vec![- TargetSize {- width: 16,- height: 10,- };- targets- ];- layout_desktop_with_shell(- desktop.2 as f64,- desktop.3 as f64,- &sizes,- ShellStyle::EdgeTaskbar,- )- }-- /// Mean of one channel over a small window, which averages out the- /// material's deterministic grain.- fn mean_channel(data: &[u8], width: u32, x: u32, y: u32, channel: usize) -> f32 {- let mut sum = 0u32;- for dy in 0..4 {- for dx in 0..4 {- sum += pixel(data, width, x + dx, y + dy)[channel] as u32;- }- }- sum as f32 / 16.0- }-- #[test]- fn taskbar_is_edge_anchored_and_spans_the_miniature_desktop() {- let (width, height) = (520u32, 360u32);- let desktop = desktop_rect(width, height, false);- let layout = taskbar_layout(width, height, 2);-- assert_eq!(layout.shell, ShellStyle::EdgeTaskbar);- let bar = offset_pixels(layout.dock, desktop.0, desktop.1);- assert_eq!(bar.0, desktop.0);- assert_eq!(bar.2, desktop.2);- assert!(((bar.1 + bar.3) - (desktop.1 + desktop.3)).abs() <= 1);-- // The band is a distinct material, so the wallpaper just above the- // bar's top edge never matches the pixels just inside it.- let data = render_view(- width,- height,- &[- frame("w", "a", PipTargetKind::BrowserTab, Vec::new(), 1),- frame("w", "b", PipTargetKind::NativeWindow, Vec::new(), 2),- ],- &[workspace("w", 2, 2)],- Some("w"),- None,- );- let center_x = (desktop.0 + desktop.2 / 2) as u32;- let above = pixel(&data, width, center_x, (bar.1 - 6) as u32);- let inside = pixel(&data, width, center_x, (bar.1 + 6) as u32);- assert_ne!(above, inside);- }-- #[test]- fn taskbar_material_carries_the_blurred_backdrop_instead_of_a_flat_fill() {- let (width, height) = (520u32, 360u32);- let desktop = desktop_rect(width, height, false);- let layout = taskbar_layout(width, height, 1);- let bar = offset_pixels(layout.dock, desktop.0, desktop.1);- let data = render_view(- width,- height,- &[frame("w", "a", PipTargetKind::BrowserTab, Vec::new(), 1)],- &[workspace("w", 1, 1)],- Some("w"),- None,- );-- // The wallpaper's lower glow sits left of center, so a material- // that samples what it covers stays brighter on the left. A flat- // fill would make these identical.- let sample_y = (bar.1 + bar.3 / 3) as u32;- let left = mean_channel(&data, width, (desktop.0 + 80) as u32, sample_y, 0);- let right = mean_channel(- &data,- width,- (desktop.0 + desktop.2 - 90) as u32,- sample_y,- 0,- );- assert!(- left - right >= 3.0,- "expected a blurred backdrop gradient, got {left} vs {right}"- );- // It is still a heavy tint, not a window into the wallpaper.- assert!(left < 90.0);- }-- #[test]- fn taskbar_marks_the_newest_card_more_strongly_than_the_others() {- let (width, height) = (560u32, 380u32);- let desktop = desktop_rect(width, height, false);- let layout = taskbar_layout(width, height, 2);- let data = render_view(- width,- height,- &[- frame("w", "a", PipTargetKind::BrowserTab, Vec::new(), 1),- frame("w", "b", PipTargetKind::NativeWindow, Vec::new(), 9),- ],- &[workspace("w", 2, 9)],- Some("w"),- None,- );- assert_eq!(layout.indicators.len(), 2);- let brightness = |index: usize| {- let mark = offset_pixels(layout.indicators[index], desktop.0, desktop.1);- let color = pixel(- &data,- width,- (mark.0 + mark.2 / 2) as u32,- (mark.1 + mark.3 / 2) as u32,- );- color[0] as i32 + color[1] as i32 + color[2] as i32- };- // Index 1 is newest, so it wears the accent pill.- assert!(brightness(1) != brightness(0));- let active = offset_pixels(layout.indicators[1], desktop.0, desktop.1);- let accent = pixel(- &data,- width,- (active.0 + active.2 / 2) as u32,- (active.1 + active.3 / 2) as u32,- );- assert!(accent[0] > accent[1] && accent[1] > accent[2]);- }-- #[test]- fn start_slot_and_tray_paint_inside_the_bar() {- let (width, height) = (560u32, 380u32);- let desktop = desktop_rect(width, height, false);- let layout = taskbar_layout(width, height, 2);- let bar = offset_pixels(layout.dock, desktop.0, desktop.1);- let start = offset_pixels(layout.start_button.unwrap(), desktop.0, desktop.1);- let tray = offset_pixels(layout.tray.unwrap(), desktop.0, desktop.1);- assert!(start.1 >= bar.1 && start.1 + start.3 <= bar.1 + bar.3);- assert_eq!(tray.0 + tray.2, bar.0 + bar.2);-- let data = render_view(- width,- height,- &[- frame("w", "a", PipTargetKind::BrowserTab, Vec::new(), 42 * 60_000),- frame("w", "b", PipTargetKind::NativeWindow, Vec::new(), 1),- ],- &[workspace("w", 2, 42 * 60_000)],- Some("w"),- None,- );- let empty = render_view(width, height, &[], &[workspace("w", 0, 1)], Some("w"), None);- let glyph_region = |data: &[u8], rect: (i32, i32, i32, i32)| {- let mut ink = 0u32;- for y in rect.1..rect.1 + rect.3 {- for x in rect.0..rect.0 + rect.2 {- ink += pixel(data, width, x as u32, y as u32)[0] as u32;- }- }- ink- };- // Both the launcher glyph and the tray band paint measurably- // brighter than the bare material behind them.- assert!(glyph_region(&data, start) > glyph_region(&empty, start));- assert!(glyph_region(&data, tray) > glyph_region(&empty, tray));- }-- #[test]- fn clock_and_foreground_selection_are_derived_only_from_frame_inputs() {- assert_eq!(clock_label(0), "00:00");- assert_eq!(clock_label(9 * 3_600_000 + 5 * 60_000), "09:05");- assert_eq!(clock_label(23 * 3_600_000 + 59 * 60_000), "23:59");- // Wrapping past a day keeps a valid time rather than overflowing.- assert_eq!(clock_label(25 * 3_600_000), "01:00");-- let make = |stamps: [u64; 3]| {- stamps- .iter()- .enumerate()- .map(|(index, stamp)| {- frame(- "w",- &index.to_string(),- PipTargetKind::NativeWindow,- Vec::new(),- *stamp,- )- })- .collect::<Vec<_>>()- };- assert_eq!(active_target_index(&[]), None);- assert_eq!(active_target_index(&make([1, 7, 3])), Some(1));- // Ties resolve to the earliest card so repaints stay stable.- assert_eq!(active_target_index(&make([7, 7, 7])), Some(0));- }-- #[test]- fn box_blur_flattens_an_impulse_without_changing_a_uniform_field() {- let uniform = vec![120u8; 9 * 9 * 3];- assert_eq!(box_blur(&uniform, 9, 9, 2), uniform);-- let mut impulse = vec![0u8; 9 * 9 * 3];- let center = (4 * 9 + 4) * 3;- impulse[center..center + 3].copy_from_slice(&[255, 255, 255]);- let blurred = box_blur(&impulse, 9, 9, 2);- assert!(blurred[center] < 255);- // Energy spreads to the neighbours the impulse never touched.- assert!(blurred[((4 * 9 + 6) * 3)] > 0);- assert!(blurred[((2 * 9 + 4) * 3)] > 0);- // ...but not past the kernel radius.- assert_eq!(blurred[((4 * 9 + 8) * 3)], 0);- }-- #[test]- fn input_passthrough_only_toggles_the_transparent_extended_style() {- let base = (WS_EX_TOPMOST | WS_EX_NOACTIVATE | WS_EX_TOOLWINDOW).0 as isize;- let passthrough = extended_style_with_input_passthrough(base, true);- assert_ne!(passthrough & WS_EX_TRANSPARENT.0 as isize, 0);- assert_eq!(passthrough & base, base);-- let interactive = extended_style_with_input_passthrough(passthrough, false);- assert_eq!(interactive, base);- }+impl PipBackendFactory for WindowsPipBackendFactory {+ fn start(&self, _cfg: &PipConfig) -> anyhow::Result<Box<dyn PipBackend>> {+ Err(anyhow::anyhow!(+ "PiP preview is not yet implemented on Windows — track \+ trycua/cua follow-up issue for status"+ ))}}diff --git a/libs/cua-driver/rust/crates/platform-windows/src/tools/impl_.rs b/libs/cua-driver/rust/crates/platform-windows/src/tools/impl_.rsindex f4383cb3a6..ef1477e1d6 100644--- a/libs/cua-driver/rust/crates/platform-windows/src/tools/impl_.rs+++ b/libs/cua-driver/rust/crates/platform-windows/src/tools/impl_.rs@@ -8031,8 +8031,7 @@ impl Tool for GetConfigTool {// and read it from the overlay deterministically — `all_states().first()`// was a nondeterministic HashMap read across sessions (macOS BUG 3).let cursor_enabled = crate::overlay::is_enabled(&resolve_cursor_key(&args));- let (agent_view_enabled, agent_view_geometry) =- pip_preview::read_agent_view_keys_from_file();+ let (pip_enabled, pip_geometry) = pip_preview::read_pip_keys_from_file();let payload = json!({"schema_version": 1,"version": env!("CARGO_PKG_VERSION"),@@ -8041,8 +8040,8 @@ impl Tool for GetConfigTool {"capture_mode": cfg.capture_mode,"max_image_dimension": cfg.max_image_dimension,"agent_cursor": { "enabled": cursor_enabled },- "agent_view": agent_view_enabled,- "agent_view_geometry": agent_view_geometry,+ "experimental_pip": pip_enabled,+ "experimental_pip_geometry": pip_geometry,});// Match Swift's text format 1:1: `"✅ <pretty JSON>"`.let pretty = serde_json::to_string_pretty(&payload).unwrap_or_else(|_| payload.to_string());@@ -8077,16 +8076,16 @@ impl Tool for SetConfigTool {`get_window_state` always returns both the UIA tree and a screenshot. Still \accepted/persisted for back-compat but has no effect.\n\- `max_image_dimension` (integer)\n\- - `agent_view` (boolean; enables the multi-target Agent View after restart)\n\- - `agent_view_geometry` (Agent View size as `WxH` or `WxH+X+Y`; persisted; applies on next daemon restart)\n\n\+ - `experimental_pip` (boolean; persisted to config.json, applies on next daemon restart — Windows backend stubbed today, see issue #1729)\n\+ - `experimental_pip_geometry` (string `WxH` or `WxH+X+Y`; persisted; applies on next daemon restart)\n\n\Returns the full updated config in the same shape as `get_config`.".into(),input_schema: json!({"type":"object","properties":{"key":{"type":"string","description":"Dotted snake_case path to a leaf config field (Swift-compatible shape). Pair with `value`."},"value":{"description":"New value for `key`. JSON type depends on the key."},"capture_mode":{"type":"string","enum":["ax","vision"],"description":"DEPRECATED and ignored — get_window_state always returns both the UIA tree and a screenshot. Still accepted/persisted for back-compat but has no effect. (\"som\"/\"screenshot\" still decode as deprecated aliases.)"},"max_image_dimension":{"type":"integer","description":"Legacy per-field shape."},- "agent_view":{"type":"boolean","description":"Per-field shape. Enables the multi-target Agent View (applies next restart)."},- "agent_view_geometry":{"type":"string","description":"Per-field shape. Agent View size + optional position."}+ "experimental_pip":{"type":"boolean","description":"Legacy per-field shape. Enables PiP preview (applies next restart)."},+ "experimental_pip_geometry":{"type":"string","description":"Legacy per-field shape. PiP window size + optional position."}},"additionalProperties":false}),read_only: false, destructive: false, idempotent: true, open_world: false,})@@ -8131,31 +8130,31 @@ impl Tool for SetConfigTool {}None => return ToolResult::error(format!("`max_image_dimension` must be an integer, got {val}.")),},- "agent_view" => match val.as_bool() {+ "experimental_pip" => match val.as_bool() {Some(b) => {- if let Err(e) = pip_preview::write_config_key("agent_view", Value::Bool(b)) {- return ToolResult::error(format!("failed to persist agent_view: {e}"));+ if let Err(e) = pip_preview::write_config_key("experimental_pip", Value::Bool(b)) {+ return ToolResult::error(format!("failed to persist experimental_pip: {e}"));}applied = true;}- None => return ToolResult::error(format!("`agent_view` must be a boolean, got {val}.")),+ None => return ToolResult::error(format!("`experimental_pip` must be a boolean, got {val}.")),},- "agent_view_geometry" => match val.as_str() {+ "experimental_pip_geometry" => match val.as_str() {Some(s) => {if pip_preview::PipGeometry::parse(s).is_none() {return ToolResult::error(format!(- "agent_view_geometry `{s}` is not a valid WxH or WxH+X+Y string"+ "experimental_pip_geometry `{s}` is not a valid WxH or WxH+X+Y string"));}- if let Err(e) = pip_preview::write_config_key("agent_view_geometry", Value::String(s.to_owned())) {- return ToolResult::error(format!("failed to persist agent_view_geometry: {e}"));+ if let Err(e) = pip_preview::write_config_key("experimental_pip_geometry", Value::String(s.to_owned())) {+ return ToolResult::error(format!("failed to persist experimental_pip_geometry: {e}"));}applied = true;}- None => return ToolResult::error(format!("`agent_view_geometry` must be a string, got {val}.")),+ None => return ToolResult::error(format!("`experimental_pip_geometry` must be a string, got {val}.")),},other => return ToolResult::error(format!(- "Unknown config key `{other}`. Known: capture_mode, max_image_dimension, agent_view, agent_view_geometry."+ "Unknown config key `{other}`. Known: capture_mode, max_image_dimension, experimental_pip, experimental_pip_geometry.")),}}@@ -8176,22 +8175,29 @@ impl Tool for SetConfigTool {}applied = true;}- if let Some(enabled) = args.get("agent_view").and_then(|v| v.as_bool()) {- if let Err(e) = pip_preview::write_config_key("agent_view", Value::Bool(enabled)) {- return ToolResult::error(format!("failed to persist agent_view: {e}"));+ if let Some(enabled) = args.get("experimental_pip").and_then(|v| v.as_bool()) {+ if let Err(e) = pip_preview::write_config_key("experimental_pip", Value::Bool(enabled))+ {+ return ToolResult::error(format!("failed to persist experimental_pip: {e}"));}applied = true;}- if let Some(geom) = args.get("agent_view_geometry").and_then(|v| v.as_str()) {+ if let Some(geom) = args+ .get("experimental_pip_geometry")+ .and_then(|v| v.as_str())+ {if pip_preview::PipGeometry::parse(geom).is_none() {return ToolResult::error(format!(- "agent_view_geometry `{geom}` is not a valid WxH or WxH+X+Y string"+ "experimental_pip_geometry `{geom}` is not a valid WxH or WxH+X+Y string"));}- if let Err(e) =- pip_preview::write_config_key("agent_view_geometry", Value::String(geom.to_owned()))- {- return ToolResult::error(format!("failed to persist agent_view_geometry: {e}"));+ if let Err(e) = pip_preview::write_config_key(+ "experimental_pip_geometry",+ Value::String(geom.to_owned()),+ ) {+ return ToolResult::error(format!(+ "failed to persist experimental_pip_geometry: {e}"+ ));}applied = true;}@@ -8209,8 +8215,7 @@ impl Tool for SetConfigTool {.first().map(|s| s.config.enabled).unwrap_or(true);- let (agent_view_enabled, agent_view_geometry) =- pip_preview::read_agent_view_keys_from_file();+ let (pip_enabled, pip_geometry) = pip_preview::read_pip_keys_from_file();let payload = json!({"schema_version": 1,"version": env!("CARGO_PKG_VERSION"),@@ -8219,8 +8224,8 @@ impl Tool for SetConfigTool {"capture_mode": cfg.capture_mode,"max_image_dimension": cfg.max_image_dimension,"agent_cursor": { "enabled": cursor_enabled },- "agent_view": agent_view_enabled,- "agent_view_geometry": agent_view_geometry,+ "experimental_pip": pip_enabled,+ "experimental_pip_geometry": pip_geometry,});let pretty = serde_json::to_string_pretty(&payload).unwrap_or_else(|_| payload.to_string());ToolResult::text(format!("✅ {pretty}")).with_structured(payload)