Computer use for the Pi coding agent — Windows apps, browsers and desktops through native UI trees, screenshots and real input.
Windows source identity and action evidence
Window discovery rows, observations and native action receipts include source_identity. Captures also include the same fields in source, alongside image geometry and read/input scope.
domain: "windows-host"names the current backend, not a sandbox.identity.session_idnames this agent's ComputerSession. It is not a Windows login/session ID.identity.window_refis this session's reference to a native window lifetime. PID/HWND describe the native target; models still address the window by its reference.- Window
membership.windows_session_idandmonitor_idremain null.virtual_desktop_idis known only from a validated native shell sample; older runtimes leave it unknown. See shell desktop membership for timestamps, independent failures and qualification status. An exact selected-display read reports its explicit native monitor ID; this is not evidence about nearby windows. Negative coordinates, desktop position and theprimarycapture alias do not establish membership. - Movement or resizing changes geometry, not lifetime. Closing/reusing a native handle retires the reference. Reload/close retires all session resources.
These fields describe an address. They do not grant input. Input still needs the latest authorized window overview. Display, detail and temporal pixels remain read-only. Foreign-session references or observations cannot authorize another agent's input.
Read action receipts, not toolkit promises
A native action receipt includes action_capability. In a bounded input sequence, each attempted row in state.sequence.steps has its own projection. There is no batch-wide capability promise.
The projection applies only to that attempted action:
operation,native_toolandrequested_deliveryidentify the attempt.actual_deliverycomes from nativedelivery.mode. Missing evidence staysunknown; requested mode does not fill the gap.background/foregroundsaydeliveredonly for a delivered native result with explicit matching mode. Partial, uncertain or suspected-no-op input does not establish support.- A native background refusal makes
background: "unavailable". Native foreground escalation makesforeground: "suggested", not proven foreground-only support. Foreground may also fail. unsupported: truerecords a known held-path/key-hold runtime refusal. False means no such evidence, not guaranteed support.route,event_kindandcoderetain explicit native labels or null.
Pre-dispatch validation failures have no native action capability receipt. Neither native delivery nor this projection proves that the app handled the input or repainted. Observe again. Never cache one successful action as permission or support for the whole app, auto-promote to foreground, or replay an uncertain prefix.
Native basis and limits
The public window state carries PID/HWND, screenshot/window bounds, snapshot tokens and UIA pattern names. Monitors come from selected displays and Task View desktops from shell desktop membership; internal virtual-screen coordinate checks are not Windows virtual-desktop membership.
CUA's validated action result publishes route, effect and actual delivery. Background refusal publishes a code and optional foreground escalation. The adapter projects these fields without predicting capabilities from app names or UIA pattern labels.
This contract does not cover virtual-desktop management or concurrent input arbitration.