Accessibility-first browser & computer use for AI agents — one MCP server for every harness (Codex, Claude Code, OpenCode, Antigravity). Instruments the DOM/accessibility tree via CDP instead of screenshot loops: diffed snapshots, stable element refs, trusted input, typed errors. Zero dependencies
JavaScript agentic-automationai-agentsbrowser-automationbrowser-usecdpchrome-devtools-protocolcomputer-usecuamcpmodel-context-protocolrpaweb-automation
Linux (Ubuntu): test, harden, and bring to SOTA parity — X11 desktop driver + AT-SPI structure layer
CTOL Team BJ7414a42aa328
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.ymlindex d6b78fb..fc92b19 100644--- a/.github/workflows/ci.yml+++ b/.github/workflows/ci.yml@@ -53,7 +53,7 @@ jobs:AXUSE_TEST_MACOS_DESKTOP: "1"run: |set -o pipefail- node --test --test-reporter=tap test/*.test.js | tee "$RUNNER_TEMP/tap.txt"+ node --test --test-timeout=90000 --test-reporter=tap test/*.test.js | tee "$RUNNER_TEMP/tap.txt"- name: Skip audit (every skip explicit, typed, CI-visible)run: node tools/skip-audit.mjs "$RUNNER_TEMP/tap.txt"- name: Doctor smoke@@ -125,12 +125,19 @@ jobs:curl --fail --silent --show-error --location "$url" --output "$RUNNER_TEMP/cft.zip"unzip -q "$RUNNER_TEMP/cft.zip" -d "$RUNNER_TEMP/cft"echo "AXUSE_BROWSER=$RUNNER_TEMP/cft/chrome-linux64/chrome" >> "$GITHUB_ENV"+ - name: Install AT-SPI sandbox deps (GTK app for the structure-layer E2E)+ run: sudo apt-get update -qq && sudo apt-get install -y -qq python3-gi gir1.2-gtk-3.0- name: Full suite (TAP for the skip audit)env:DISPLAY: ":99"run: |set -o pipefail- node --test --test-reporter=tap test/*.test.js | tee "$RUNNER_TEMP/tap.txt"+ # A REAL X server on :99: the Linux desktop driver mounts against it+ # (input synthesis is verified on the suite's own private Xvfb+ # displays; :99 only lets the MCP layer mount the live tools).+ Xvfb :99 -screen 0 1280x800x24 -ac -nolisten tcp &+ sleep 2+ node --test --test-timeout=90000 --test-reporter=tap test/*.test.js | tee "$RUNNER_TEMP/tap.txt"- name: Skip auditrun: node tools/skip-audit.mjs "$RUNNER_TEMP/tap.txt"- name: Doctor smokediff --git a/README.md b/README.mdindex 5a2918b..4853beb 100644--- a/README.md+++ b/README.md@@ -4,7 +4,8 @@server plugs into Codex, Claude Code, OpenCode, Antigravity, or anyMCP-capable host. Zero dependencies (Node ≥ 22). Windows desktop driver isverified; macOS desktop control is available as an explicit opt-in with-native TCC permission checks.+native TCC permission checks; Linux desktop control runs over X11/XWayland+(XTEST input + capture + EWMH windows, zero extra packages).The design is distilled from the two best implementations of this problem —OpenAI's Codex Chrome extension and the ZCode harness — and exists to prove a@@ -12,11 +13,12 @@ point: **a screenshot loop is not the right interface for a machine**. TheDOM tree, the accessibility tree, and CDP are. axuse instruments the targetinstead of photographing it.-If you are a journalist or other non-developer using Codex or ChatGPT Work,-start with the [non-developer onboarding guide](docs/non-developer-onboarding.md).-It explains the one-time connector setup, the first verification prompt, and-copy-and-paste research prompts. The source repository's setup below is for-developers and administrators.+**Not a developer?** You don't need anything below. In the Claude desktop+app, download `axuse.mcpb` from [ax-use.com](https://ax-use.com) and+double-click it. In ChatGPT's Codex app, Claude Code, Cursor or any other+agent, paste the [install prompt](https://ax-use.com/install.md) and the+agent sets axuse up for you. See the+[non-developer guide](docs/non-developer-onboarding.md).## Proven in production@@ -232,7 +234,8 @@ The ones that save the most steps:- JS dialogs and new tabs opened by an action are reported in its result.**Desktop** (Windows: verified. macOS: opt-in with-`AXUSE_ENABLE_MACOS_CUA=1` and host Accessibility/Screen Recording grants):+`AXUSE_ENABLE_MACOS_CUA=1` and host Accessibility/Screen Recording grants.+Linux: automatic on X11/XWayland sessions with `DISPLAY` set):`ax_desktop_screenshot`, `ax_desktop_screen_info`,`ax_desktop_click`, `ax_desktop_type`, `ax_desktop_press`, `ax_desktop_scroll`,`ax_desktop_windows`, `ax_desktop_activate`, `ax_desktop_drag`.@@ -294,8 +297,13 @@ Field notes (all established live against Edge 155 / Chrome 155):`AXUSE_ENABLE_MACOS_CUA=1`, grant the host Accessibility and Screen Recordingin System Settings → Privacy & Security, then run `node src/cli.js doctor`.Coordinates are macOS global display points and returned desktop JPEGs are- resized to the same point dimensions for direct click mapping. Linux has no- desktop driver yet (browser tools still work).+ resized to the same point dimensions for direct click mapping. Linux desktop+ control mounts automatically on X11/XWayland sessions (needs `DISPLAY`;+ Xvfb works for headless hosts) — and it is structure-first too: the AT-SPI+ tree (via axuse's zero-dependency D-Bus client) mounts+ ax_desktop_ax_snapshot/ax_click/ax_set/ax_read like macOS, plus EWMH+ windows, RandR monitor layout, a screenshot budget ladder that downscales+ instead of erroring, and clipboard-paste typing when xclip is installed.- **The extension backend has no Target domain** (chrome.debugger limits), soOOPIF frames are listed but not entered there; the CDP backend enters them.- **Anti-bot reality**: axuse runs a real Chromium with a real profile, but@@ -317,7 +325,7 @@ summaries.## Development```bash-node --test test/diff.test.js test/browser.test.js test/mcp.test.js test/desktop.test.js test/oopif.test.js test/bridge.test.js test/plugin.test.js+node --test test/diff.test.js test/browser.test.js test/mcp.test.js test/desktop.test.js test/desktop-linux.test.js test/oopif.test.js test/bridge.test.js test/plugin.test.js```On macOS, enable the passive desktop capability gates explicitly:@@ -334,7 +342,9 @@ macOS builds is honestly skipped because it ignores automated`--load-extension`); desktop tests exercise passive screen/window discovery onmacOS with `AXUSE_TEST_MACOS_DESKTOP=1`. A real macOS screenshot test isexplicit with `AXUSE_TEST_MACOS_SCREENSHOT=1` because Screen Recording is-TCC-protected. No test moves your real mouse or sends keys.+TCC-protected. The Linux desktop tests boot their own private Xvfb displays+(and skip typed when Xvfb is not installed); they never touch your session.+No test moves your real mouse or sends keys.Debugging: `AXUSE_BRIDGE_DEBUG=1` traces hub + native-host messages;`self.__axuseDiag` in the extension SW reports connect/disconnect history.diff --git a/SKILL.md b/SKILL.mdindex 3b9095c..5f9b394 100644--- a/SKILL.md+++ b/SKILL.md@@ -132,6 +132,18 @@ the tabs axuse created — never your browser.pixels — get bounds from `ax_desktop_screen_info`/`ax_desktop_windows`).Verified live.+**Linux** mounts the same tools automatically on X11/XWayland sessions+(`DISPLAY` set): XTEST input, GetImage JPEG capture, EWMH window list and+activation, RandR per-monitor layout in `ax_desktop_screen_info` — and the+**structure-first layer through AT-SPI** (zero-dependency D-Bus client):+`ax_desktop_ax_snapshot/ax_click/ax_set/ax_read` with refs, roles, names,+bounds and states, like macOS System Events. No extra packages; `xvfb-run`+covers headless hosts. Coordinates are root-window pixels. Screenshots honor+the byte budget with a quality → quality → downscale ladder (vision keeps a+usable image instead of a dead end), and `ax_desktop_type` pastes long or+non-keymap text through the clipboard when xclip is present (reported+honestly as `via: "clipboard-paste"`).+**macOS** ships an opt-in driver (System Events keystroke, correctedCoreGraphics mouse/scroll events via JXA, and `screencapture` + `sips` pixels)that loads with `AXUSE_ENABLE_MACOS_CUA=1`. It reports TCC permission failuresdiff --git a/docs/macos-support.md b/docs/macos-support.mdindex ac75e9d..661e62c 100644--- a/docs/macos-support.md+++ b/docs/macos-support.md@@ -199,7 +199,7 @@ the `ax_*` browser tools — never a hang or a generic failure.| Lane | Where | What it proves ||---|---|---|| `macos-26`, `macos-15`, `macos-15-intel` | CI (`ci.yml`) | Full suite × 3 runners (2 versions × arm64 + Intel), CfT installed, gates on, skip audit, doctor, bridge-security assertion |-| `windows-latest`, `ubuntu-latest` | CI (`ci.yml`) | Cross-platform guard (Windows desktop driver live; Linux browser tools with CfT) |+| `windows-latest`, `ubuntu-latest` | CI (`ci.yml`) | Cross-platform guard (Windows desktop driver live; Linux browser tools with CfT, Linux desktop driver against a CI Xvfb) || Plugin E2E | CI-private runner or dev machine | `AXUSE_TEST_LIVE_*=1 node --test test/plugins.test.js` — every plugin's unit + live suite on macOS || `tcc-manual` | `tcc-manual.yml`, self-hosted | Granted-state proofs: real (corner) capture, inert zero-delta CGEvent, recovery re-probe. Fails closed with typed instructions if the runner's grants are missing. Zero skips allowed. |@@ -263,7 +263,10 @@ Measured on Apple M1, macOS 26.6, Node 26 (`bench/macos-hotpaths.mjs`):7. **Vision-gated plugin live demos** (challenge-handler B–E) — require`CHALLENGE_VISION_KEY` / `CHALLENGE_CF_DEMO_URL`; they self-skip typed andare listed in the allowlist. Not macOS-specific.-8. **Linux desktop driver** — does not exist (by design; browser tools only).+8. **Linux desktop driver** — shipped (X11/XWayland). Structure-first via+ AT-SPI (ax_desktop_ax_snapshot/click/set/read over axuse's zero-dep+ D-Bus client) plus coordinate+vision input/capture, EWMH windows and+ RandR monitor layout — the macOS pattern, on Linux.## Release checklistdiff --git a/package.json b/package.jsonindex f40d15c..38756b7 100644--- a/package.json+++ b/package.json@@ -11,7 +11,7 @@"node": ">=22"},"scripts": {- "test": "node --test test/*.test.js",+ "test": "node --test --test-timeout=90000 test/*.test.js","doctor": "node src/cli.js doctor"},"keywords": [diff --git a/src/cli.js b/src/cli.jsindex 4003407..8313a2c 100644--- a/src/cli.js+++ b/src/cli.js@@ -148,8 +148,11 @@ function openExtensionsPage(browser, browserPath) { const url = "chrome://extenconst app = apps[browser] ?? "Google Chrome";return launch("open", ["-a", app, url], app);}- const command = browserPath ?? (process.platform === "win32" ? "chrome.exe" : "google-chrome");- return launch(command, [url], browserPath ?? "browser");+ const command = browserPath+ ?? (process.platform === "win32" ? "chrome.exe" : undefined)+ ?? (process.platform !== "darwin" ? locateBrowser() : undefined)+ ?? "google-chrome";+ return launch(command, [url], browserPath ?? command);}async function setupCmd() {@@ -293,6 +296,25 @@ async function doctor() {out(`native WebSocket: ${hasWs ? "yes" : "NO — need Node >= 22"}`);const browser = locateBrowser();out(`browser: ${browser ?? "not found (set AXUSE_BROWSER)"}`);+ if (process.platform === "linux") {+ // Chrome keeps shared memory in /dev/shm; a full tmpfs kills headed AND+ // headless launches with a bare crash. axuse auto-passes+ // --disable-dev-shm-usage, but the underlying pressure is worth knowing.+ try {+ const { statfsSync } = await import("node:fs");+ const st = statfsSync("/dev/shm");+ const free = (BigInt(st.bavail) * BigInt(st.bsize)) / 1024n / 1024n;+ const total = (BigInt(st.blocks) * BigInt(st.bsize)) / 1024n / 1024n;+ const pct = total > 0n ? Number((free * 100n) / total) : 100;+ if (free < 128n) {+ out(`/dev/shm: ${free} MB free of ${total} MB — Chrome needs ≥64MB; axuse adds --disable-dev-shm-usage automatically, but free space if launches still fail`);+ } else if (pct < 5) {+ out(`/dev/shm: low (${free} MB free, ${pct.toFixed(0)}%) — axuse mitigates with --disable-dev-shm-usage`);+ }+ } catch {+ // no /dev/shm mount — the launch path mitigates silently+ }+ }if (process.platform === "win32" || process.platform === "darwin") {if (process.platform === "darwin" && !/^(1|true)$/i.test(process.env.AXUSE_ENABLE_MACOS_CUA ?? "")) {out("desktop driver: disabled (set AXUSE_ENABLE_MACOS_CUA=1; Accessibility + Screen Recording grants are required for OS control)");@@ -333,6 +355,29 @@ async function doctor() {out(`desktop driver: FAILED — ${e.message}`);}}+ } else if (process.platform === "linux") {+ if (!process.env.DISPLAY) {+ out(`desktop driver: not available — DISPLAY is not set (X11/XWayland session required; browser tools still work${process.env.WAYLAND_DISPLAY ? "; this looks like a native Wayland session — XWayland usually provides DISPLAY like :0/:1" : ""})`);+ } else {+ try {+ const d = await resolveDesktopDriver();+ const info = await d.screenInfo();+ out(`desktop driver: ok (X11/XWayland, XTEST + EWMH) — root window ${info.w}x${info.h} at (${info.x},${info.y}) (pixels)`);+ try {+ const clip = await d.constructor.clipboardAvailable?.();+ out(`desktop clipboard paste: ${clip === undefined ? "unknown" : clip ? "available (xclip — long/untypeable text pastes via ctrl+v)" : "absent (type() synthesizes per key; sudo apt install xclip for the paste path)"}`);+ } catch {}+ try {+ await d.screenshot({ x: info.x, y: info.y, w: 1, h: 1, budgetBytes: 20_000 });+ out("desktop screenshot: ok");+ } catch (e) {+ out(`desktop screenshot: unavailable — ${e.message}`);+ }+ d.dispose();+ } catch (e) {+ out(`desktop driver: FAILED — ${e.message}`);+ }+ }} else {out("desktop driver: not implemented on this platform (browser tools still work)");}diff --git a/src/core/cdp.js b/src/core/cdp.jsindex 6e03119..fb21b41 100644--- a/src/core/cdp.js+++ b/src/core/cdp.js@@ -9,13 +9,29 @@* dependency-free — important for a library that harnesses vendor via MCP.*/import { spawn } from "node:child_process";-import { accessSync, mkdirSync, mkdtempSync, readdirSync, rmSync, statSync } from "node:fs";+import { accessSync, existsSync, mkdirSync, mkdtempSync, readdirSync, rmSync, statfsSync, statSync } from "node:fs";import { homedir, tmpdir } from "node:os";import { delimiter, join } from "node:path";import { AxError, Codes } from "./errors.js";const SEND_TIMEOUT_MS = 30_000;+/** Chromium keeps its discardable/shared-memory segment in /dev/shm (tmpfs).+ * When that mount is full or missing, the browser aborts during startup —+ * on Linux with a bare "Browser exited during startup (code null)" and one+ * stderr line ("Less than 64MB of free space…"). Detect the condition and+ * steer Chrome to the disk-backed temp dir instead (the same thing the+ * --disable-dev-shm-usage flag does in every headless-container guide). */+export function linuxShmUnusable() {+ if (process.platform !== "linux") return false;+ try {+ const st = statfsSync("/dev/shm");+ return BigInt(st.bavail) * BigInt(st.bsize) < 128n * 1024n * 1024n;+ } catch {+ return true; // no /dev/shm mount at all (some container/minimal images)+ }+}+function isMacBrandedChrome(executable) {// ALL branded Chrome channels (stable/Beta/Dev/Canary) ignore// --load-extension on current macOS builds — live-verified for stable and@@ -24,33 +40,62 @@ function isMacBrandedChrome(executable) {&& /(?:^|\/)Google Chrome(?: Beta| Dev| Canary)?\.app\/Contents\/MacOS\/Google Chrome(?: Beta| Dev| Canary)?$/.test(executable);}+/** Branded-Chrome builds that ignore --load-extension, per platform:+ * macOS .app bundles (live-verified) and the Linux distro packages+ * (live-verified on google-chrome-stable 151 / Ubuntu 22.04: the flag is+ * silently ignored — only built-in component extensions appear in+ * /json/list). Chrome-for-Testing's chrome-linux64 layout is NOT branded. */+export function isBrandedChrome(executable) {+ if (isMacBrandedChrome(executable)) return true;+ if (process.platform === "linux") {+ return /^\/usr\/bin\/google-chrome(-stable|-beta|-unstable)?$/.test(executable)+ || /^\/opt\/google\/chrome\/chrome$/.test(executable);+ }+ return false;+}+/** Candidate Chromium-family executables, best first. */export function browserCandidates({ extensionCapable = false, browser, ignoreEnv = false, exact = false } = {}) {const out = [];if (!ignoreEnv && process.env.AXUSE_BROWSER && process.env.AXUSE_BROWSER.trim()) {const configured = process.env.AXUSE_BROWSER.trim();- // Current macOS branded Chrome builds silently ignore --load-extension.- // Do not let a general AXUSE_BROWSER override defeat the explicit- // extension-capable selection used by the bridge test/setup lane.- if (!(extensionCapable && isMacBrandedChrome(configured))) {+ // Branded Chrome builds (macOS .app bundles; Linux distro packages)+ // silently ignore --load-extension. Do not let a general AXUSE_BROWSER+ // override defeat the explicit extension-capable selection used by the+ // bridge test/setup lane.+ if (!(extensionCapable && isBrandedChrome(configured))) {out.push(configured);return out;}}if (process.platform === "win32") {const chrome = [];+ const chromeBeta = [];+ const chromeCanary = [];const edge = [];+ const edgeBeta = [];for (const envVar of ["ProgramFiles", "ProgramFiles(x86)", "LOCALAPPDATA"]) {const base = process.env[envVar];if (!base) continue;chrome.push(join(base, "Google", "Chrome", "Application", "chrome.exe"));+ // Per-user installs of the preview channels (the default for Beta/Canary).+ if (envVar === "LOCALAPPDATA" || envVar === "ProgramFiles") {+ chromeBeta.push(join(base, "Google", "Chrome Beta", "Application", "chrome.exe"));+ chromeCanary.push(join(base, "Google", "Chrome SxS", "Application", "chrome.exe"));+ }}for (const envVar of ["ProgramFiles", "ProgramFiles(x86)"]) {const base = process.env[envVar];if (!base) continue;edge.push(join(base, "Microsoft", "Edge", "Application", "msedge.exe"));+ edgeBeta.push(join(base, "Microsoft", "Edge Beta", "Application", "msedge.exe"));}- out.push(...(browser === "edge" ? edge : browser === "chrome" ? chrome : [...chrome, ...edge]));+ out.push(...(browser === "edge" ? edge+ : browser === "edge-beta" ? edgeBeta+ : browser === "chrome-beta" ? chromeBeta+ : browser === "chrome-canary" ? chromeCanary+ : browser === "chrome" ? [...chrome, ...edge]+ : [...chrome, ...edge, ...edgeBeta, ...chromeBeta, ...chromeCanary]));} else if (process.platform === "darwin") {const app = (name, executable) => [`/Applications/${name}.app/Contents/MacOS/${executable}`,@@ -100,20 +145,47 @@ export function browserCandidates({ extensionCapable = false, browser, ignoreEnvout.push(...chrome, ...cft, ...preview, ...chromium, ...edge);}} else {+ // Linux: distro packages live in /usr/bin; snap Chromium installs to+ // /snap/bin (Ubuntu's default Chromium is a snap). Branded Chrome on+ // Linux IGNORES --load-extension (live-verified on google-chrome-stable+ // 151 / Ubuntu 22.04 — only built-in component extensions appear), so it+ // is excluded from the extension lane exactly like macOS branded builds.const chrome = ["/usr/bin/google-chrome","/usr/bin/google-chrome-stable",];- const chromium = ["/usr/bin/chromium", "/usr/bin/chromium-browser"];+ const chromium = ["/usr/bin/chromium", "/usr/bin/chromium-browser", "/snap/bin/chromium"];const cft = ["/usr/bin/google-chrome-for-testing", "/usr/bin/chrome-for-testing"];- const edge = ["/usr/bin/microsoft-edge"];+ const edge = ["/usr/bin/microsoft-edge", "/usr/bin/microsoft-edge-stable"];+ const edgeBeta = ["/usr/bin/microsoft-edge-beta"];const chromeBeta = ["/usr/bin/google-chrome-beta"];- out.push(...(browser === "edge" ? edge- : browser === "chromium" ? chromium- : browser === "chrome-for-testing" ? cft- : browser === "chrome-beta" ? chromeBeta- : browser === "chrome" ? (exact ? chrome : [...chrome, ...cft, ...chromeBeta, ...chromium, ...edge])- : [...chrome, ...cft, ...chromeBeta, ...chromium, ...edge]));+ const preview = [...chromeBeta, ...edgeBeta];+ if (extensionCapable) {+ const preferred = browser === "edge" ? [...edge, ...edgeBeta]+ : browser === "edge-beta" ? edgeBeta+ : browser === "chromium" ? chromium+ : browser === "chrome-for-testing" ? cft+ : [...edge, ...edgeBeta, ...chromium, ...cft];+ out.push(...preferred);+ for (const dir of (process.env.PATH ?? "").split(delimiter).filter(Boolean)) {+ if (!browser || browser === "chromium") out.push(join(dir, "chromium"));+ if (!browser || browser === "chrome-for-testing") out.push(join(dir, "google-chrome-for-testing"));+ }+ } else if (browser === "edge") {+ out.push(...edge);+ } else if (browser === "edge-beta") {+ out.push(...edgeBeta);+ } else if (browser === "chromium") {+ out.push(...chromium);+ } else if (browser === "chrome-for-testing") {+ out.push(...cft);+ } else if (browser === "chrome-beta") {+ out.push(...chromeBeta);+ } else if (browser === "chrome") {+ out.push(...(exact ? chrome : [...chrome, ...cft, ...preview, ...chromium, ...edge]));+ } else {+ out.push(...chrome, ...cft, ...preview, ...chromium, ...edge, ...edgeBeta);+ }}return [...new Set(out)];}@@ -257,6 +329,11 @@ export class BrowserHandle {"--remote-allow-origins=*", // modern Chrome rejects unknown ws Origins];if (headless) args.push("--headless");+ if (linuxShmUnusable() && !args.includes("--disable-dev-shm-usage")) {+ // /dev/shm full/missing: without this the browser dies with SIGTRAP+ // before DevTools ever comes up (verified on Ubuntu 22.04).+ args.push("--disable-dev-shm-usage");+ }args.push(...extraArgs);const proc = spawn(exe, args, { stdio: ["ignore", "pipe", "pipe"], windowsHide: false });let wsUrl;@@ -264,17 +341,29 @@ export class BrowserHandle {wsUrl = await new Promise((resolve, reject) => {let buf = "";let settled = false;+ let tail = ""; // last stderr/stdout chars — the only real evidence when a browser dies young+ const noteTail = () => {+ const t = tail.trim();+ if (!t) return "";+ const hint = /Less than \d+MB of free space/.test(t)+ ? " Chrome ran out of shared-memory space (full /dev/shm or temp filesystem). Free space, or raise it, then retry."+ : /X server|DISPLAY|display .*not/.test(t) && !process.env.DISPLAY && process.platform === "linux"+ ? " Headed Chrome needs an X server: run inside the desktop session or under xvfb-run (or pass headless: true)."+ : "";+ return `${hint}\nBrowser output (tail): ${t}`;+ };const timer = setTimeout(() => {try { proc.kill(); } catch {}if (!settled) {settled = true;reject(new AxError(Codes.LAUNCH_FAILED,- `Timed out after ${launchTimeoutMs}ms waiting for the DevTools WebSocket URL (first launch of a browser can be slow; raise AXUSE_LAUNCH_TIMEOUT_MS if this recurs)`,+ `Timed out after ${launchTimeoutMs}ms waiting for the DevTools WebSocket URL (first launch of a browser can be slow; raise AXUSE_LAUNCH_TIMEOUT_MS if this recurs)${noteTail()}`,{ retry: "retry" }));}}, launchTimeoutMs);const onData = (d) => {buf += String(d);+ tail = (tail + String(d)).slice(-800);const m = buf.match(/DevTools listening on (ws:\/\/\S+)/);if (m) {settled = true;@@ -290,11 +379,13 @@ export class BrowserHandle {settled = true;reject(new AxError(Codes.LAUNCH_FAILED, `Could not start browser: ${e.message}`, { retry: "never" }));});- proc.on("exit", (code) => {+ proc.on("exit", (code, signal) => {clearTimeout(timer);if (settled) return;settled = true;- reject(new AxError(Codes.LAUNCH_FAILED, `Browser exited during startup (code ${code})`, { retry: "retry" }));+ reject(new AxError(Codes.LAUNCH_FAILED,+ `Browser exited during startup (code ${code}${signal ? `, signal ${signal}` : ""})${noteTail()}`,+ { retry: "retry" }));});});} catch (e) {@@ -386,7 +477,11 @@ export class BrowserHandle {/** Chrome's helper processes can hold profile files briefly after exit. */async function removeDirWithRetry(dir, attempts = 10) {for (let i = 0; i < attempts; i++) {- try { rmSync(dir, { recursive: true, force: true }); return true; } catch {}+ try { rmSync(dir, { recursive: true, force: true }); } catch {}+ // Verify, don't assume: Chrome's dying children can re-create files+ // moments after a clean delete (late crashpad dumps) — the dir would+ // silently reappear and look like a leak.+ if (!existsSync(dir)) return true;await new Promise((ok) => setTimeout(ok, 200 * (i + 1)));}return false;diff --git a/src/desktop/atspi.js b/src/desktop/atspi.jsnew file mode 100644index 0000000..1e82a05--- /dev/null+++ b/src/desktop/atspi.js@@ -0,0 +1,239 @@+/**+ * axuse — AT-SPI2 client over the zero-dependency D-Bus transport.+ *+ * The macOS driver gets its platform AX tree from System Events; on Linux+ * the same tree lives at AT-SPI2, exposed over a private D-Bus socket. This+ * module speaks the AT-SPI interface set directly (verified against the+ * live at-spi2 registry on Xorg/GNOME, Ubuntu 22.04):+ *+ * - objects are (bus name, object path) pairs — children of the desktop+ * live on each application's own connection name+ * - Accessible: GetChildren → a(so), GetChildAtIndex(i), GetRoleName,+ * GetState → (au), GetInterfaces → as, GetApplication → (so);+ * Name/ChildCount/Description are D-Bus PROPERTIES (the old GetX+ * methods do not exist on current registries)+ * - Component.GetExtents(coordType 0) → (iiii) screen-absolute bounds+ * - Action: nActions property, GetActionName(i), DoAction(i)+ * - Text: GetCharCount, GetText(start, end); EditableText.SetTextContents+ * - Value: current/minimum/maximum as variants+ */+import { AxError, Codes } from "../core/errors.js";+import { DbusClient } from "./dbus.js";++const A11Y = "org.a11y.atspi.Accessible";+const REGISTRY_NAME = "org.a11y.atspi.Registry";+const DESKTOP_PATH = "/org/a11y/atspi/accessible/root";++/** AT-SPI state bits (atspi-state.h) — the operationally meaningful ones. */+export const STATE_NAMES = [+ "invalid", "active", "armed", "busy", "checked", "collapsed", "defunct",+ "editable", "enabled", "expandable", "expanded", "focusable", "focused",+ "has-tooltip", "horizontal", "iconified", "modal", "multi-line",+ "multiselectable", "opaque", "pressed", "resizable", "selectable",+ "selected", "sensitive", "showing", "single-line", "stale", "transient",+ "vertical", "visible", "manages-descendants", "indeterminate",+];++function decodeStates(words) {+ const out = [];+ for (let w = 0; w < words.length; w++) {+ let bits = words[w] >>> 0;+ let b = 0;+ while (bits) {+ if (bits & 1) {+ const name = STATE_NAMES[w * 32 + b];+ if (name) out.push(name);+ }+ bits >>>= 1;+ b++;+ }+ }+ return out;+}++export class AtspiClient {+ /** @param {{sessionBusAddress?: string}} [opts] — the session bus to+ * discover org.a11y.Bus on (defaults to the ambient environment; the+ * sandbox test passes a private bus explicitly). */+ constructor(opts = {}) {+ this.sessionBusAddress = opts.sessionBusAddress ?? process.env.DBUS_SESSION_BUS_ADDRESS;+ this.bus = null; // the AT-SPI bus connection+ this.sessionBus = null; // temporary connection used to discover it+ }++ /** Discover and connect the AT-SPI bus. */+ async connect() {+ const direct = process.env.AXUSE_ATSPI_BUS;+ let address = direct ?? null;+ if (!address) {+ const sessionAddress = this.sessionBusAddress;+ if (!sessionAddress) {+ throw new AxError(Codes.DESKTOP_DRIVER_UNAVAILABLE,+ "No D-Bus session bus (DBUS_SESSION_BUS_ADDRESS unset) — the AT-SPI structure layer needs the session's accessibility bus.",+ { hint: "Run inside the desktop session; structure-first desktop control needs AT-SPI.", retry: "never" });+ }+ try {+ this.sessionBus = await new DbusClient(sessionAddress).connect();+ const [a] = await this.sessionBus.call("org.a11y.Bus", "/org/a11y/bus", "org.a11y.Bus", "GetAddress", "", []);+ address = String(a);+ } catch (e) {+ try { this.sessionBus?.close(); } catch {}+ this.sessionBus = null;+ throw new AxError(Codes.DESKTOP_DRIVER_UNAVAILABLE,+ `The session bus has no org.a11y.Bus (is at-spi2 running?): ${e.message}`,+ { hint: "Install/enable at-spi2-core; coordinate+vision tools keep working without it.", retry: "never" });+ }+ }+ try {+ this.bus = await new DbusClient(address).connect();+ } catch (e) {+ try { this.sessionBus?.close(); } catch {}+ this.sessionBus = null;+ throw e;+ }+ return this;+ }++ get connected() {+ return !!this.bus?.socket && !this.bus.dead;+ }++ close() {+ try { this.bus?.close(); } catch {}+ try { this.sessionBus?.close(); } catch {}+ this.bus = null;+ this.sessionBus = null;+ }++ /* ------------------------------------------------------ object helpers */++ /** The desktop root (registry-owned). */+ desktop() {+ return [REGISTRY_NAME, DESKTOP_PATH];+ }++ /** Accessible children as (name, path) pairs. */+ async children(ref) {+ const [list] = await this.bus.call(ref[0], ref[1], A11Y, "GetChildren", "", []);+ return list;+ }++ async childCount(ref) {+ const v = await this.bus.prop(ref[0], ref[1], A11Y, "ChildCount");+ return Number(v);+ }++ async name(ref) {+ const v = await this.bus.prop(ref[0], ref[1], A11Y, "Name");+ return String(v ?? "");+ }++ async description(ref) {+ const v = await this.bus.prop(ref[0], ref[1], A11Y, "Description");+ return String(v ?? "");+ }++ async role(ref) {+ // Method on most at-spi2 apps; a few roots only expose the Role+ // property (role enum) — report it honestly as role#N then.+ try {+ const [r] = await this.bus.call(ref[0], ref[1], A11Y, "GetRoleName", "", []);+ return String(r);+ } catch {+ const v = await this.bus.prop(ref[0], ref[1], A11Y, "Role").catch(() => null);+ return v === null ? "unknown" : `role#${Number(v)}`;+ }+ }++ async states(ref) {+ const [words] = await this.bus.call(ref[0], ref[1], A11Y, "GetState", "", []);+ return decodeStates(Array.isArray(words) ? words : [words]);+ }++ async interfaces(ref) {+ const [list] = await this.bus.call(ref[0], ref[1], A11Y, "GetInterfaces", "", []);+ return list;+ }++ async application(ref) {+ const [app] = await this.bus.call(ref[0], ref[1], A11Y, "GetApplication", "", []);+ return app;+ }++ /** Screen-absolute bounds via Component.GetExtents(coordType 0). */+ async bounds(ref) {+ const [tuple] = await this.bus.call(ref[0], ref[1], "org.a11y.atspi.Component", "GetExtents", "u", [0]);+ return { x: tuple[0], y: tuple[1], w: tuple[2], h: tuple[3] };+ }++ /* --------------------------------------------------------------- Action */+ // CAUTION: GetActions is never called internally — GTK's atk-bridge (2.4x)+ // crashes marshalling its a(sss) reply and takes the app's bus connection+ // with it. DoAction(i) and GetActionName(i) are the safe surface.++ async actionCount(ref) {+ try {+ const v = await this.bus.prop(ref[0], ref[1], "org.a11y.atspi.Action", "nActions");+ if (v !== undefined && v !== null) return Number(v);+ } catch {}+ // No nActions property (GTK): a resolvable action name implies ≥1 action.+ try { await this.actionName(ref, 0); return 1; } catch { return 0; }+ }++ async actionName(ref, index) {+ const [n] = await this.bus.call(ref[0], ref[1], "org.a11y.atspi.Action", "GetActionName", "i", [index]);+ return String(n);+ }++ async doAction(ref, index) {+ const [ok] = await this.bus.call(ref[0], ref[1], "org.a11y.atspi.Action", "DoAction", "i", [index]);+ return !!ok;+ }++ async doAction(ref, index) {+ const [ok] = await this.bus.call(ref[0], ref[1], "org.a11y.atspi.Action", "DoAction", "i", [index]);+ return !!ok;+ }++ /* ----------------------------------------------------------------- Text */++ async charCount(ref) {+ const [n] = await this.bus.call(ref[0], ref[1], "org.a11y.atspi.Text", "GetCharCount", "", []);+ return Number(n);+ }++ async text(ref, start = 0, end = -1) {+ const [s] = await this.bus.call(ref[0], ref[1], "org.a11y.atspi.Text", "GetText", "ii", [start, end]);+ return String(s);+ }++ async setTextContents(ref, contents) {+ const [ok] = await this.bus.call(ref[0], ref[1], "org.a11y.atspi.EditableText", "SetTextContents", "s", [String(contents)]);+ return !!ok;+ }++ /* ---------------------------------------------------------------- Value */++ async value(ref) {+ try {+ const [cur] = await this.bus.call(ref[0], ref[1], "org.a11y.atspi.Value", "GetCurrentValue", "", []);+ return cur?.sig === "d" ? cur.value : cur?.value ?? null;+ } catch {+ return null;+ }+ }++ /* ------------------------------------------------------------- registry */++ /** Applications registered on the desktop: [{ref, name}]. */+ async apps() {+ const kids = await this.children(this.desktop());+ const out = [];+ for (const ref of kids) {+ let n = "";+ try { n = await this.name(ref); } catch {}+ out.push({ ref, name: n });+ }+ return out;+ }+}diff --git a/src/desktop/dbus.js b/src/desktop/dbus.jsnew file mode 100644index 0000000..6a43009--- /dev/null+++ b/src/desktop/dbus.js@@ -0,0 +1,486 @@+/**+ * axuse — minimal zero-dependency D-Bus client (AT-SPI desktop layer).+ *+ * The macOS driver gets its platform AX tree from System Events over JXA;+ * Linux exposes the same thing — AT-SPI2 — over D-Bus at a Unix socket.+ * Node ships no D-Bus client, so this speaks the wire protocol directly:+ *+ * - SASL EXTERNAL handshake (uid cookie — the standard user-bus auth)+ * - Hello(), then method_call / method_return / error routing by serial+ * - a general little-endian marshaller for the type grammar used by the+ * AT-SPI interfaces: basics (y b n q i u x t d s o g), arrays, structs,+ * variants and dict entries, with the alignment rules from the spec+ *+ * Scope: method calls + property access on the AT-SPI bus. No signals, no+ * file-descriptor passing, no peer-to-peer container gymnastics.+ */+import { connect as netConnect } from "node:net";+import { userInfo } from "node:os";+import { AxError, Codes } from "../core/errors.js";++const LITTLE = 0x6c; // 'l'++const MSG_METHOD_CALL = 1;+const MSG_METHOD_RETURN = 2;+const MSG_ERROR = 3;++// Header field codes.+const FIELD_PATH = 1, FIELD_INTERFACE = 2, FIELD_MEMBER = 3,+ FIELD_ERROR_NAME = 4, FIELD_REPLY_SERIAL = 5, FIELD_DESTINATION = 6,+ FIELD_SENDER = 7, FIELD_SIGNATURE = 8;++/* ------------------------------------------------------- type marshalling */++class Writer {+ constructor() {+ this.chunks = [];+ this.length = 0;+ this.alignBuf = { 1: Buffer.from([0]), 2: Buffer.alloc(2), 4: Buffer.alloc(4), 8: Buffer.alloc(8) };+ }+ push(buf) {+ this.chunks.push(buf);+ this.length += buf.length;+ }+ align(n) {+ const pad = (n - (this.length % n)) % n;+ if (pad) this.push(Buffer.alloc(pad));+ }+ byte(v) { this.align(1); this.push(Buffer.from([v & 0xff])); }+ bool(v) { this.align(4); this.push(Buffer.from([v ? 1 : 0, 0, 0, 0])); }+ u16(v) { this.align(2); const b = Buffer.alloc(2); b.writeUInt16LE(v); this.push(b); }+ i16(v) { this.align(2); const b = Buffer.alloc(2); b.writeInt16LE(v); this.push(b); }+ u32(v) { this.align(4); const b = Buffer.alloc(4); b.writeUInt32LE(v >>> 0); this.push(b); }+ i32(v) { this.align(4); const b = Buffer.alloc(4); b.writeInt32LE(v | 0); this.push(b); }+ u64(v) { this.align(8); const b = Buffer.alloc(8); b.writeBigUInt64LE(BigInt(v)); this.push(b); }+ string(v) {+ const s = Buffer.from(String(v), "utf8");+ this.u32(s.length);+ this.push(s);+ this.push(Buffer.from([0]));+ }+ signature(v) {+ const s = Buffer.from(String(v), "utf8");+ if (s.length > 255) throw new Error("signature too long");+ this.byte(s.length);+ this.push(s);+ this.push(Buffer.from([0]));+ }+ /** Write one value of the given single-complete-type signature. */+ value(sig, v) {+ switch (sig) {+ case "y": return this.byte(v);+ case "b": return this.bool(v);+ case "n": return this.i16(v);+ case "q": return this.u16(v);+ case "i": return this.i32(v);+ case "u": return this.u32(v);+ case "x": case "t": return this.u64(v);+ case "d": {+ this.align(8);+ const b = Buffer.alloc(8);+ b.writeDoubleLE(Number(v));+ this.push(b);+ return;+ }+ case "s": return this.string(v);+ case "o": return this.string(v);+ case "g": return this.signature(v);+ case "v": {+ const inner = v ?? { sig: "s", value: "" };+ this.signature(inner.sig);+ return this.value(inner.sig, inner.value);+ }+ }+ if (sig.startsWith("a")) {+ const elem = sig.slice(1);+ const body = new Writer();+ const elemAlign = alignmentOf(elem);+ for (const item of Array.isArray(v) ? v : []) {+ body.align(elemAlign); // every element starts at its alignment+ body.value(elem, item);+ }+ const bytes = body.finish();+ this.u32(bytes.length);+ this.align(elemAlign);+ return this.push(bytes);+ }+ if (sig.startsWith("(") || sig.startsWith("{")) {+ const inner = sig.slice(1, -1);+ this.align(8);+ if (sig.startsWith("{")) {+ // Dict entries are written as {key, value} objects (the reader's shape).+ const obj = Array.isArray(v) ? { key: v[0], value: v[1] } : v;+ return this.writeValues(inner, [obj?.key, obj?.value]);+ }+ return this.writeValues(inner, Array.isArray(v) ? v : []);+ }+ throw new Error(`unsupported D-Bus signature: ${sig}`);+ }+ /** Write a sequence of values for a multi-type signature ("is", "a(so)"…). */+ writeValues(sig, values) {+ let vi = 0;+ for (const [type] of splitSignature(sig)) {+ this.value(type, values[vi++]);+ }+ }+ finish() {+ return Buffer.concat(this.chunks);+ }+}++class Reader {+ constructor(buf) {+ this.buf = buf;+ this.pos = 0;+ }+ align(n) {+ this.pos += (n - (this.pos % n)) % n;+ }+ byte() { this.align(1); return this.buf[this.pos++]; }+ u16() { this.align(2); const v = this.buf.readUInt16LE(this.pos); this.pos += 2; return v; }+ i16() { this.align(2); const v = this.buf.readInt16LE(this.pos); this.pos += 2; return v; }+ u32() { this.align(4); const v = this.buf.readUInt32LE(this.pos); this.pos += 4; return v; }+ i32() { this.align(4); const v = this.buf.readInt32LE(this.pos); this.pos += 4; return v; }+ u64() { this.align(8); const v = this.buf.readBigUInt64LE(this.pos); this.pos += 8; return Number(v); }+ string() {+ const len = this.u32();+ const s = this.buf.toString("utf8", this.pos, this.pos + len);+ this.pos += len + 1;+ return s;+ }+ signature() {+ const len = this.byte();+ const s = this.buf.toString("latin1", this.pos, this.pos + len);+ this.pos += len + 1;+ return s;+ }+ value(sig) {+ switch (sig) {+ case "y": return this.byte();+ case "b": { this.align(4); const v = this.u32(); return !!v; }+ case "n": return this.i16();+ case "q": return this.u16();+ case "i": return this.i32();+ case "u": return this.u32();+ case "x": case "t": return this.u64();+ case "d": { this.align(8); const v = this.buf.readDoubleLE(this.pos); this.pos += 8; return v; }+ case "s": case "o": return this.string();+ case "g": return this.signature();+ case "v": {+ const inner = this.signature();+ return { sig: inner, value: this.value(inner) };+ }+ }+ if (sig.startsWith("a")) {+ const elem = sig.slice(1);+ const len = this.u32();+ this.align(alignmentOf(elem));+ const end = this.pos + len;+ const out = [];+ // Guard against malformed lengths from hostile peers.+ if (end > this.buf.length + 8) throw new Error("bad array length in D-Bus message");+ while (this.pos < end) {+ out.push(this.value(elem));+ this.align(alignmentOf(elem)); // elements restart at their alignment+ }+ this.pos = end;+ return out;+ }+ if (sig.startsWith("(") || sig.startsWith("{")) {+ this.align(8);+ const inner = sig.slice(1, -1);+ const types = splitSignature(inner);+ const out = [];+ for (const [type] of types) out.push(this.value(type));+ return sig.startsWith("(") ? out : { key: out[0], value: out[1] };+ }+ throw new Error(`unsupported D-Bus signature while reading: ${sig}`);+ }+ values(sig) {+ const out = [];+ for (const [type] of splitSignature(sig)) out.push(this.value(type));+ return out;+ }+}++/** Split a signature into its top-level single-complete types. */+function splitSignature(sig) {+ const out = [];+ let i = 0;+ while (i < sig.length) {+ const start = i;+ let depth = 0;+ // Consume one complete type.+ for (;;) {+ const c = sig[i];+ if (c === "a") { i++; continue; }+ if (c === "(" || c === "{") { depth++; i++; continue; }+ if (c === ")" || c === "}") { depth--; i++; if (depth === 0) break; continue; }+ i++;+ if (depth === 0) break;+ }+ out.push([sig.slice(start, i)]);+ }+ return out;+}++function alignmentOf(sig) {+ if (sig.startsWith("a")) return alignmentOf(sig.slice(1));+ if (sig.startsWith("(") || sig.startsWith("{")) return 8;+ switch (sig) {+ case "y": case "g": return 1;+ case "n": case "q": return 2;+ case "b": case "i": case "u": case "s": case "o": return 4;+ case "x": case "t": case "d": case "v": return 8;+ default: return 1;+ }+}++/** Marshal a single value of `sig` (for tests and future direct users). */+export function marshalValue(sig, value) {+ const w = new Writer();+ w.value(sig, value);+ return w.finish();+}++/** Unmarshal a single value of `sig`; `offset` positions the reader+ * (alignment is relative to the buffer + offset, like a message body). */+export function unmarshalValue(sig, buf, offset = 0) {+ const r = new Reader(buf);+ r.pos = offset;+ return r.value(sig);+}++/* ------------------------------------------------------------ connection */++/** Parse a D-Bus address like unix:path=/run/user/1000/bus. */+export function parseDbusAddress(address) {+ const m = /^unix:(path=[^,]+|abstract=[^,]+)/.exec(String(address ?? ""));+ if (!m) return null;+ const [, kv] = m;+ const [k, v] = kv.split("=");+ return k === "path" ? { path: v } : { abstract: v };+}++const MSG_TIMEOUT_MS = 12_000;++export class DbusClient {+ /**+ * @param {string} address D-Bus server address (unix:path=…)+ */+ constructor(address) {+ this.address = address;+ this.socket = null;+ this.serial = 0;+ this.pending = new Map(); // serial → {resolve, reject, timer}+ this.buffer = Buffer.alloc(0);+ this.dead = false;+ }++ async connect() {+ const parsed = parseDbusAddress(this.address);+ if (!parsed) {+ throw new AxError(Codes.DESKTOP_DRIVER_UNAVAILABLE,+ `Cannot parse D-Bus address: ${this.address}`,+ { retry: "never" });+ }+ this.socket = await new Promise((resolve, reject) => {+ const t = setTimeout(() => {+ // Never leave a half-created socket dangling on the event loop.+ try { sock.destroy(); } catch {}+ reject(new Error("D-Bus socket connect timed out"));+ }, 5_000);+ const sock = parsed.path !== undefined+ ? netConnect(parsed.path)+ : netConnect(`\0${parsed.abstract}`);+ sock.once("error", (e) => { clearTimeout(t); try { sock.destroy(); } catch {} reject(e); });+ sock.once("connect", () => { clearTimeout(t); sock.off("error", reject); resolve(sock); });+ });+ this.socket.on("close", () => this.failAll("D-Bus connection closed"));+ this.socket.on("error", () => this.failAll("D-Bus connection error"));++ // SASL EXTERNAL: leading NUL (needed on the transport), then the cookie.+ // BEGIN has NO reply — the next server bytes are already D-Bus messages,+ // so the message listeners attach only after the handshake completes.+ const uidHex = Buffer.from(String(userInfo().uid)).toString("hex");+ try {+ await this.sasl(`\0AUTH EXTERNAL ${uidHex}\r\n`, /^OK /);+ } catch (e) {+ try { this.socket.destroy(); } catch {}+ this.socket = null;+ throw e;+ }+ this.socket.write("BEGIN\r\n");+ this.socket.on("data", (d) => this.route(d));+ await this.call("org.freedesktop.DBus", "/org/freedesktop/DBus", "org.freedesktop.DBus", "Hello", "", []);+ return this;+ }++ /** One SASL line exchange (BEFORE message framing starts), time-bounded:+ * a wedged daemon must not dangle the socket on the event loop. */+ async sasl(request, expect) {+ const reply = await new Promise((resolve, reject) => {+ const t = setTimeout(() => {+ cleanup();+ try { this.socket?.destroy(); } catch {}+ reject(new Error("D-Bus SASL handshake timed out"));+ }, 5_000);+ let buf = "";+ const onData = (d) => {+ buf += d.toString("latin1");+ const i = buf.indexOf("\r\n");+ if (i >= 0) {+ clearTimeout(t);+ cleanup();+ resolve(buf.slice(0, i));+ }+ };+ const onErr = (e) => { clearTimeout(t); cleanup(); reject(e); };+ const cleanup = () => {+ this.socket?.off("data", onData);+ this.socket?.off("error", onErr);+ };+ this.socket.on("data", onData);+ this.socket.on("error", onErr);+ this.socket.write(request);+ });+ if (expect && !expect.test(reply)) {+ throw new AxError(Codes.DESKTOP_DRIVER_UNAVAILABLE,+ `D-Bus SASL handshake failed: ${reply.trim()}`,+ { retry: "never" });+ }+ return reply;+ }++ route(chunk) {+ this.buffer = Buffer.concat([this.buffer, chunk]);+ for (;;) {+ if (this.buffer.length < 16) return;+ // Header "yyyyuua(yv)": [4..8) body length, [8..12) serial,+ // [12..16) fields-array length; elements start at 16 (8-aligned).+ const bodyLen = this.buffer.readUInt32LE(4);+ const arrLen = this.buffer.readUInt32LE(12);+ const headerLen = 16 + arrLen + ((8 - ((16 + arrLen) % 8)) % 8);+ const total = headerLen + bodyLen;+ if (this.buffer.length < total) return;+ const msg = Buffer.from(this.buffer.subarray(0, total));+ this.buffer = this.buffer.subarray(total);+ this.dispatch(msg, headerLen, arrLen);+ }+ }++ dispatch(msg, bodyStart, arrLen) {+ const type = msg[1];+ // Fields elements span [16, 16+arrLen) — the u32 length prefix sits at+ // 12..16 and the elements start 8-aligned at 16.+ const reader = new Reader(msg.subarray(16));+ let replySer = null, errorName = null, sig = "";+ try {+ const end = arrLen;+ for (;;) {+ if (reader.pos >= end) break;+ reader.align(8); // each (yv) struct element is 8-aligned+ const code = reader.byte();+ const v = reader.value("v");+ if (code === FIELD_REPLY_SERIAL) replySer = v.value;+ else if (code === FIELD_ERROR_NAME) errorName = String(v.value);+ else if (code === FIELD_SIGNATURE) sig = String(v.value);+ }+ } catch {+ return; // unparseable header — drop+ }+ const p = replySer !== null ? this.pending.get(replySer) : null;+ if (!p) return;+ this.pending.delete(replySer);+ clearTimeout(p.timer);+ if (type === MSG_ERROR) {+ const body = new Reader(msg.subarray(bodyStart));+ p.reject(new AxError(Codes.DESKTOP_OP_FAILED,+ `D-Bus error: ${errorName ?? "unknown"}${sig ? ` (${body.value("s")})` : ""}`,+ { retry: "retry" }));+ return;+ }+ if (type === MSG_METHOD_RETURN) {+ const body = new Reader(msg.subarray(bodyStart));+ p.resolve(sig ? body.values(sig) : []);+ }+ // Signals (type 4) are ignored — v1 polls.+ }++ failAll(reason) {+ this.dead = true;+ for (const [, p] of this.pending) {+ clearTimeout(p.timer);+ p.reject(new AxError(Codes.DESKTOP_OP_FAILED, reason));+ }+ this.pending.clear();+ }++ /**+ * Method call. `argsSig` is the argument signature, `args` the values.+ * Resolves with the array of reply values.+ */+ call(destination, path, iface, member, argsSig, args) {+ if (!this.socket || this.dead) {+ return Promise.reject(new AxError(Codes.DESKTOP_OP_FAILED, "D-Bus connection is not open"));+ }+ const serial = ++this.serial;+ const body = new Writer();+ if (argsSig) body.writeValues(argsSig, args);+ const bodyBytes = body.finish();++ // Header "yyyyuua(yv)": [4..8) body length, [8..12) serial,+ // [12..16) fields-array length, elements from 16 (8-aligned),+ // header padded to a multiple of 8 before the body.+ const fields = new Writer();+ const field = (code, sig, value) => {+ fields.align(8); // struct element alignment+ fields.byte(code);+ fields.value("v", { sig, value });+ };+ field(FIELD_PATH, "o", path);+ field(FIELD_DESTINATION, "s", destination);+ field(FIELD_INTERFACE, "s", iface);+ field(FIELD_MEMBER, "s", member);+ if (argsSig) field(FIELD_SIGNATURE, "g", argsSig);+ const fieldsBytes = fields.finish();++ const head = Buffer.alloc(16);+ head[0] = LITTLE;+ head[1] = MSG_METHOD_CALL;+ head[2] = 0; // flags+ head[3] = 1; // protocol version+ head.writeUInt32LE(bodyBytes.length, 4);+ head.writeUInt32LE(serial, 8);+ head.writeUInt32LE(fieldsBytes.length, 12);+ const headerPad = (8 - ((16 + fieldsBytes.length) % 8)) % 8;+ const msg = Buffer.concat([+ head,+ fieldsBytes,+ Buffer.alloc(headerPad),+ bodyBytes,+ ]);+ return new Promise((resolve, reject) => {+ const timer = setTimeout(() => {+ this.pending.delete(serial);+ reject(new AxError(Codes.TIMEOUT, `D-Bus call ${iface}.${member} timed out after ${MSG_TIMEOUT_MS}ms`, { retry: "retry" }));+ }, MSG_TIMEOUT_MS);+ this.pending.set(serial, { resolve, reject, timer });+ this.socket.write(msg);+ });+ }++ /** org.freedesktop.DBus.Properties.Get — returns the variant value. */+ async prop(destination, path, iface, name) {+ const [variant] = await this.call(destination, path, "org.freedesktop.DBus.Properties", "Get", "ss", [iface, name]);+ return variant?.value;+ }++ close() {+ this.dead = true;+ this.failAll("D-Bus client closed");+ try { this.socket?.end(); } catch {}+ try { this.socket?.destroy(); } catch {}+ this.socket = null;+ }+}diff --git a/src/desktop/index.js b/src/desktop/index.jsindex e8f969d..a648ce4 100644--- a/src/desktop/index.js+++ b/src/desktop/index.js@@ -22,7 +22,12 @@* loads with AXUSE_ENABLE_MACOS_CUA=1; reports TCC failures clearly.* AX-tree ops need the Accessibility grant; capture needs Screen* Recording.- * - Linux: no driver yet (browser tools still work).+ * - Linux: zero-dependency X11/XWayland driver (src/desktop/linux.js):+ * XTEST FakeInput for trusted input, GetImage+baseline JPEG for+ * capture, EWMH for the window list and activation. Mounts whenever+ * DISPLAY is set (verified on Xorg 1.21 / Ubuntu 22.04+ and Xvfb);+ * native Wayland sessions without XWayland fail typed with guidance.+ * Coordinate+vision mode, like Windows — no AT-SPI AX tree in v1.** The driver interface is the seam: a Windows UIA driver slots in behind* the same tools without touching the contract.@@ -241,7 +246,9 @@ class WindowsDriver {let cached = null;/** Get the platform driver, or null when unsupported. Tools hide when null.- * macOS remains explicitly opt-in because it can synthesize OS input. */+ * macOS remains explicitly opt-in because it can synthesize OS input; the+ * Linux X11 driver mounts when a DISPLAY is configured (same trust level as+ * the always-on Windows console-session driver). */export function getDesktopDriver() {if (process.platform === "win32") {if (!cached) cached = new WindowsDriver();@@ -260,6 +267,20 @@ export function getDesktopDriver() {}return cached;}+ if (process.platform === "linux") {+ if (!process.env.DISPLAY) return null; // no X11/XWayland to drive+ if (!cached) {+ // Lazy import keeps the X11/JPEG modules out of non-Linux paths; the+ // driver resolves through resolveDesktopDriver() like the macOS one.+ cached = import("./linux.js")+ .then(({ LinuxDriver }) => new LinuxDriver())+ .catch((e) => {+ cached = null;+ throw e;+ });+ }+ return cached;+ }return null;}@@ -279,9 +300,14 @@ export function desktopUnavailable() {const macNote = process.platform === "darwin"? " The macOS driver is disabled: set AXUSE_ENABLE_MACOS_CUA=1; desktop input/capture also needs Accessibility and Screen Recording grants for the host.": "";+ const linuxNote = process.platform === "linux" && !process.env.DISPLAY+ ? " The Linux driver speaks X11/XWayland: set DISPLAY (usually :0/:1) — or run under xvfb-run — and the ax_desktop_* tools mount automatically."+ : "";throw new AxError(Codes.DESKTOP_DRIVER_UNAVAILABLE,process.platform === "darwin"? `Desktop control is disabled on macOS until explicitly enabled.${macNote}`- : `Desktop control is not implemented for ${process.platform} yet.${macNote}`,+ : process.platform === "linux"+ ? `Desktop control is unavailable on this Linux session.${linuxNote}`+ : `Desktop control is not implemented for ${process.platform} yet.${macNote}`,{ hint: "Prefer the ax_* browser tools whenever the target is a web page.", retry: "never" });}diff --git a/src/desktop/jpeg.js b/src/desktop/jpeg.jsnew file mode 100644index 0000000..b826368--- /dev/null+++ b/src/desktop/jpeg.js@@ -0,0 +1,359 @@+/**+ * axuse — baseline JPEG encoder (Linux desktop driver screenshots).+ *+ * The Windows driver gets JPEG from GDI+ and macOS from sips; Node ships no+ * image encoder, so this is a compact baseline-sequential encoder written+ * against the JPEG spec: standard Annex K quantization tables scaled by the+ * IJG quality curve, standard Huffman tables, 4:2:0 chroma subsampling,+ * BT.601 full-range RGB→YCbCr. Plain Buffer math, zero dependencies.+ */++// Standard luminance/chrominance quantization tables (Annex K, quality 50).+const STD_LUM_Q = [+ 16, 11, 10, 16, 24, 40, 51, 61,+ 12, 12, 14, 19, 26, 58, 60, 55,+ 14, 13, 16, 24, 40, 57, 69, 56,+ 14, 17, 22, 29, 51, 87, 80, 62,+ 18, 22, 37, 56, 68, 109, 103, 77,+ 24, 35, 55, 64, 81, 104, 113, 92,+ 49, 64, 78, 87, 103, 121, 120, 101,+ 72, 92, 95, 98, 112, 100, 103, 99,+];+const STD_CHROMA_Q = [+ 17, 18, 24, 47, 99, 99, 99, 99,+ 18, 21, 26, 66, 99, 99, 99, 99,+ 24, 26, 56, 99, 99, 99, 99, 99,+ 47, 66, 99, 99, 99, 99, 99, 99,+ 99, 99, 99, 99, 99, 99, 99, 99,+ 99, 99, 99, 99, 99, 99, 99, 99,+ 99, 99, 99, 99, 99, 99, 99, 99,+ 99, 99, 99, 99, 99, 99, 99, 99,+];++// Zigzag order for 8x8 blocks.+const ZIGZAG = [+ 0, 1, 8, 16, 9, 2, 3, 10,+ 17, 24, 32, 25, 18, 11, 4, 5,+ 12, 19, 26, 33, 40, 48, 41, 34,+ 27, 20, 13, 6, 7, 14, 21, 28,+ 35, 42, 49, 56, 57, 50, 43, 36,+ 29, 22, 15, 23, 30, 37, 44, 51,+ 58, 59, 52, 45, 38, 31, 39, 46,+ 53, 60, 61, 54, 47, 55, 62, 63,+];++// Standard Huffman tables (Annex K): [16 code lengths, symbols].+const DC_LUM_LENGTHS = [0, 1, 5, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0];+const DC_LUM_SYMBOLS = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11];+const DC_CHROMA_LENGTHS = [0, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0];+const DC_CHROMA_SYMBOLS = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11];++const AC_LUM_LENGTHS = [0, 2, 1, 3, 3, 2, 4, 3, 5, 5, 4, 4, 0, 0, 1, 125];+const AC_LUM_SYMBOLS = [+ 0x01, 0x02, 0x03, 0x00, 0x04, 0x11, 0x05, 0x12, 0x21, 0x31, 0x41, 0x06, 0x13, 0x51, 0x61, 0x07,+ 0x22, 0x71, 0x14, 0x32, 0x81, 0x91, 0xa1, 0x08, 0x23, 0x42, 0xb1, 0xc1, 0x15, 0x52, 0xd1, 0xf0,+ 0x24, 0x33, 0x62, 0x72, 0x82, 0x09, 0x0a, 0x16, 0x17, 0x18, 0x19, 0x1a, 0x25, 0x26, 0x27, 0x28,+ 0x29, 0x2a, 0x34, 0x35, 0x36, 0x37, 0x38, 0x39, 0x3a, 0x43, 0x44, 0x45, 0x46, 0x47, 0x48, 0x49,+ 0x4a, 0x53, 0x54, 0x55, 0x56, 0x57, 0x58, 0x59, 0x5a, 0x63, 0x64, 0x65, 0x66, 0x67, 0x68, 0x69,+ 0x6a, 0x73, 0x74, 0x75, 0x76, 0x77, 0x78, 0x79, 0x7a, 0x83, 0x84, 0x85, 0x86, 0x87, 0x88, 0x89,+ 0x8a, 0x92, 0x93, 0x94, 0x95, 0x96, 0x97, 0x98, 0x99, 0x9a, 0xa2, 0xa3, 0xa4, 0xa5, 0xa6, 0xa7,+ 0xa8, 0xa9, 0xaa, 0xb2, 0xb3, 0xb4, 0xb5, 0xb6, 0xb7, 0xb8, 0xb9, 0xba, 0xc2, 0xc3, 0xc4, 0xc5,+ 0xc6, 0xc7, 0xc8, 0xc9, 0xca, 0xd2, 0xd3, 0xd4, 0xd5, 0xd6, 0xd7, 0xd8, 0xd9, 0xda, 0xe1, 0xe2,+ 0xe3, 0xe4, 0xe5, 0xe6, 0xe7, 0xe8, 0xe9, 0xea, 0xf1, 0xf2, 0xf3, 0xf4, 0xf5, 0xf6, 0xf7, 0xf8,+ 0xf9, 0xfa,+];++const AC_CHROMA_LENGTHS = [0, 2, 1, 2, 4, 4, 3, 4, 7, 5, 4, 4, 0, 1, 2, 119];+const AC_CHROMA_SYMBOLS = [+ 0x00, 0x01, 0x02, 0x03, 0x11, 0x04, 0x05, 0x21, 0x31, 0x06, 0x12, 0x41, 0x51, 0x07, 0x61, 0x71,+ 0x13, 0x22, 0x32, 0x81, 0x08, 0x14, 0x42, 0x91, 0xa1, 0xb1, 0xc1, 0x09, 0x23, 0x33, 0x52, 0xf0,+ 0x15, 0x62, 0x72, 0xd1, 0x0a, 0x16, 0x24, 0x34, 0xe1, 0x25, 0xf1, 0x17, 0x18, 0x19, 0x1a, 0x26,+ 0x27, 0x28, 0x29, 0x2a, 0x35, 0x36, 0x37, 0x38, 0x39, 0x3a, 0x43, 0x44, 0x45, 0x46, 0x47, 0x48,+ 0x49, 0x4a, 0x53, 0x54, 0x55, 0x56, 0x57, 0x58, 0x59, 0x5a, 0x63, 0x64, 0x65, 0x66, 0x67, 0x68,+ 0x69, 0x6a, 0x73, 0x74, 0x75, 0x76, 0x77, 0x78, 0x79, 0x7a, 0x82, 0x83, 0x84, 0x85, 0x86, 0x87,+ 0x88, 0x89, 0x8a, 0x92, 0x93, 0x94, 0x95, 0x96, 0x97, 0x98, 0x99, 0x9a, 0xa2, 0xa3, 0xa4, 0xa5,+ 0xa6, 0xa7, 0xa8, 0xa9, 0xaa, 0xb2, 0xb3, 0xb4, 0xb5, 0xb6, 0xb7, 0xb8, 0xb9, 0xba, 0xc2, 0xc3,+ 0xc4, 0xc5, 0xc6, 0xc7, 0xc8, 0xc9, 0xca, 0xd2, 0xd3, 0xd4, 0xd5, 0xd6, 0xd7, 0xd8, 0xd9, 0xda,+ 0xe2, 0xe3, 0xe4, 0xe5, 0xe6, 0xe7, 0xe8, 0xe9, 0xea, 0xf2, 0xf3, 0xf4, 0xf5, 0xf6, 0xf7, 0xf8,+ 0xf9, 0xfa,+];++/** IJG quality scaling for the quantization tables (quality 1-100). */+function scaledQuantTable(std, quality) {+ const q = Math.max(1, Math.min(100, Math.round(quality)));+ const scale = q < 50 ? Math.floor(5000 / q) : 200 - q * 2;+ return std.map((v) => Math.max(1, Math.min(255, Math.floor((v * scale + 50) / 100))));+}++/** symbol → {code, length} from a [lengths, symbols] table. */+function buildHuffmanCodes(lengths, symbols) {+ const total = lengths.reduce((a, b) => a + b, 0);+ if (total !== symbols.length) {+ throw new Error(`Huffman table mismatch: lengths sum ${total} ≠ ${symbols.length} symbols`);+ }+ const codes = new Map();+ let code = 0;+ let k = 0;+ for (let len = 1; len <= 16; len++) {+ for (let i = 0; i < lengths[len - 1]; i++) {+ codes.set(symbols[k], { code, length: len });+ code++;+ k++;+ }+ code <<= 1;+ }+ return codes;+}++function dhtSegment(tableClass, tableId, lengths, symbols) {+ const body = Buffer.alloc(1 + 16 + symbols.length);+ body.writeUInt8((tableClass << 4) | tableId, 0);+ for (let i = 0; i < 16; i++) body.writeUInt8(lengths[i], 1 + i);+ Buffer.from(symbols).copy(body, 17);+ return Buffer.concat([Buffer.from([0xff, 0xc4, (body.length + 2) >> 8, (body.length + 2) & 0xff]), body]);+}++class BitWriter {+ constructor() {+ this.buf = Buffer.alloc(256 * 1024);+ this.len = 0;+ this.bitBuf = 0;+ this.bitCount = 0;+ }+ ensure(n) {+ if (this.len + n + 2 > this.buf.length) {+ let size = this.buf.length * 2;+ while (size < this.len + n + 2) size *= 2;+ const next = Buffer.alloc(size);+ this.buf.copy(next, 0, 0, this.len);+ this.buf = next;+ }+ }+ writeBits(value, length) {+ if (length <= 0) return;+ this.bitBuf = (this.bitBuf << length) | (value & ((1 << length) - 1));+ this.bitCount += length;+ while (this.bitCount >= 8) {+ this.bitCount -= 8;+ const byte = (this.bitBuf >>> this.bitCount) & 0xff;+ this.ensure(2);+ this.buf[this.len++] = byte;+ if (byte === 0xff) this.buf[this.len++] = 0; // byte stuffing+ }+ this.bitBuf &= (1 << this.bitCount) - 1;+ }+ /** Pad the final byte with 1-bits (JPEG spec) and return the bytes. */+ flush() {+ if (this.bitCount > 0) {+ const padBits = 8 - this.bitCount;+ this.writeBits((1 << padBits) - 1, padBits);+ }+ return this.buf.subarray(0, this.len);+ }+}++const COS = (() => {+ const t = new Float64Array(8 * 8);+ for (let u = 0; u < 8; u++) {+ for (let x = 0; x < 8; x++) t[u * 8 + x] = Math.cos(((2 * x + 1) * u * Math.PI) / 16);+ }+ return t;+})();++/** Separable forward DCT; input/output in natural (row-major) order. */+function fdct(block) {+ const tmp = new Float64Array(64);+ const out = new Float64Array(64);+ for (let u = 0; u < 8; u++) {+ const cu = u === 0 ? Math.SQRT1_2 / 2 : 0.5;+ for (let x = 0; x < 8; x++) {+ let s = 0;+ for (let y = 0; y < 8; y++) s += block[y * 8 + x] * COS[u * 8 + y];+ tmp[u * 8 + x] = s * cu;+ }+ }+ for (let u = 0; u < 8; u++) {+ for (let v = 0; v < 8; v++) {+ const cv = v === 0 ? Math.SQRT1_2 / 2 : 0.5;+ let s = 0;+ for (let x = 0; x < 8; x++) s += tmp[u * 8 + x] * COS[v * 8 + x];+ out[u * 8 + v] = s * cv;+ }+ }+ return out;+}++/** Bit size category of a DC/AC value (JPEG additional-bits convention). */+function category(value) {+ let v = Math.abs(value);+ let n = 0;+ while (v) { n++; v >>= 1; }+ return n;+}++/** Encode one 8x8 block; returns the new DC predictor. Quant table is in+ * NATURAL order, as are the DCT coefficients; the RLE walks zigzag. */+function encodeBlock(coeffs, quant, dcCodes, acCodes, w, lastDC) {+ let dc = 0;+ {+ let v = coeffs[0] / quant[0];+ dc = v > 0 ? Math.round(v) : -Math.round(-v);+ }+ const diff = dc - lastDC;+ const cat = category(diff);+ const dcH = dcCodes.get(cat);+ if (!dcH) throw new Error(`no DC code for category ${cat}`);+ w.writeBits(dcH.code, dcH.length);+ if (cat > 0) {+ const bits = diff < 0 ? diff + (1 << cat) - 1 : diff;+ w.writeBits(bits, cat);+ }+ // Quantize in natural order, then RLE in zigzag order.+ const q = new Map();+ let lastNonZeroZ = 0;+ for (let i = 1; i < 64; i++) {+ const z = ZIGZAG[i];+ let v = coeffs[z] / quant[z];+ v = v > 0 ? Math.round(v) : -Math.round(-v);+ if (v !== 0) {+ q.set(i, v);+ lastNonZeroZ = i;+ }+ }+ let run = 0;+ for (let i = 1; i <= lastNonZeroZ; i++) {+ const v = q.get(i) ?? 0;+ if (v === 0) { run++; continue; }+ while (run > 15) {+ const zrl = acCodes.get(0xf0);+ w.writeBits(zrl.code, zrl.length);+ run -= 16;+ }+ const vcat = category(v);+ const sym = (run << 4) | vcat;+ const ac = acCodes.get(sym);+ if (!ac) throw new Error(`missing AC Huffman code for symbol 0x${sym.toString(16)}`);+ w.writeBits(ac.code, ac.length);+ const bits = v < 0 ? v + (1 << vcat) - 1 : v;+ w.writeBits(bits, vcat);+ run = 0;+ }+ if (lastNonZeroZ < 63) {+ const eob = acCodes.get(0x00);+ w.writeBits(eob.code, eob.length);+ }+ return dc;+}++function segment(markerByte, body) {+ return Buffer.concat([Buffer.from([0xff, markerByte, (body.length + 2) >> 8, (body.length + 2) & 0xff]), body]);+}++/**+ * Encode interleaved RGB rows as baseline JPEG (4:2:0, JFIF).+ * @param {Uint8Array|Buffer} rgb+ * @param {number} width+ * @param {number} height+ * @param {number} [stride] bytes per row (default width*3)+ * @param {number} [quality] 1-100+ * @returns {Buffer}+ */+export function encodeJPEG(rgb, width, height, stride = width * 3, quality = 80) {+ const lumQuant = scaledQuantTable(STD_LUM_Q, quality);+ const chromaQuant = scaledQuantTable(STD_CHROMA_Q, quality);+ const dcLumCodes = buildHuffmanCodes(DC_LUM_LENGTHS, DC_LUM_SYMBOLS);+ const acLumCodes = buildHuffmanCodes(AC_LUM_LENGTHS, AC_LUM_SYMBOLS);+ const dcChromaCodes = buildHuffmanCodes(DC_CHROMA_LENGTHS, DC_CHROMA_SYMBOLS);+ const acChromaCodes = buildHuffmanCodes(AC_CHROMA_LENGTHS, AC_CHROMA_SYMBOLS);++ const mcusX = Math.ceil(width / 16);+ const mcusY = Math.ceil(height / 16);+ const yW = mcusX * 16, yH = mcusY * 16;+ const cW = mcusX * 8, cH = mcusY * 8;+ const yPlane = new Float64Array(yW * yH);+ const cbAccum = new Float64Array(cW * cH);+ const crAccum = new Float64Array(cW * cH);+ const chromaCount = new Uint16Array(cW * cH);++ // RGB → YCbCr (BT.601 full range); chroma box-averaged 2x2.+ for (let row = 0; row < height; row++) {+ const src = row * stride;+ const yRow = row * yW;+ const cRow = (row >> 1) * cW;+ for (let col = 0; col < width; col++) {+ const o = src + col * 3;+ const r = rgb[o], g = rgb[o + 1], b = rgb[o + 2];+ yPlane[yRow + col] = 0.299 * r + 0.587 * g + 0.114 * b;+ const co = cRow + (col >> 1);+ cbAccum[co] += 128 - 0.168736 * r - 0.331264 * g + 0.5 * b;+ crAccum[co] += 128 + 0.5 * r - 0.418688 * g - 0.081312 * b;+ chromaCount[co]++;+ }+ }+ const cbPlane = new Float64Array(cW * cH);+ const crPlane = new Float64Array(cW * cH);+ for (let i = 0; i < cbAccum.length; i++) {+ const n = Math.max(1, chromaCount[i]);+ cbPlane[i] = cbAccum[i] / n;+ crPlane[i] = crAccum[i] / n;+ }++ const w = new BitWriter();+ let lastY = 0, lastCB = 0, lastCR = 0;+ const block = new Float64Array(64);++ function encodePlaneBlock(plane, planeW, bx, by, quant, dcCodes, acCodes, lastDC) {+ const baseY = by * 8;+ const baseX = bx * 8;+ for (let y = 0; y < 8; y++) {+ const src = (baseY + y) * planeW + baseX;+ for (let x = 0; x < 8; x++) block[y * 8 + x] = plane[src + x] - 128;+ }+ return encodeBlock(fdct(block), quant, dcCodes, acCodes, w, lastDC);+ }++ for (let my = 0; my < mcusY; my++) {+ for (let mx = 0; mx < mcusX; mx++) {+ for (let i = 0; i < 4; i++) {+ lastY = encodePlaneBlock(yPlane, yW, mx * 2 + (i % 2), my * 2 + (i >> 1), lumQuant, dcLumCodes, acLumCodes, lastY);+ }+ lastCB = encodePlaneBlock(cbPlane, cW, mx, my, chromaQuant, dcChromaCodes, acChromaCodes, lastCB);+ lastCR = encodePlaneBlock(crPlane, cW, mx, my, chromaQuant, dcChromaCodes, acChromaCodes, lastCR);+ }+ }+ const scan = w.flush();++ const app0 = segment(0xe0, Buffer.from([0x4a, 0x46, 0x49, 0x46, 0x00, 0x01, 0x01, 0x00, 0, 1, 0, 1, 0, 0]));+ // DQT payloads carry values in ZIGZAG order (the tables here are natural).+ const zig = (table) => Buffer.from(ZIGZAG.map((z) => table[z]));+ const dqt = Buffer.concat([+ segment(0xdb, Buffer.concat([Buffer.from([0x00]), zig(lumQuant)])),+ segment(0xdb, Buffer.concat([Buffer.from([0x01]), zig(chromaQuant)])),+ ]);+ const sof0Body = Buffer.alloc(15); // 1+2+2+1 + 3×3-byte component entries+ sof0Body.writeUInt8(8, 0);+ sof0Body.writeUInt16BE(height, 1);+ sof0Body.writeUInt16BE(width, 3);+ sof0Body.writeUInt8(3, 5);+ sof0Body.writeUInt8(1, 6); sof0Body.writeUInt8(0x22, 7); sof0Body.writeUInt8(0, 8);+ sof0Body.writeUInt8(2, 9); sof0Body.writeUInt8(0x11, 10); sof0Body.writeUInt8(1, 11);+ sof0Body.writeUInt8(3, 12); sof0Body.writeUInt8(0x11, 13); sof0Body.writeUInt8(1, 14);+ const sof0 = segment(0xc0, sof0Body);+ const dht = Buffer.concat([+ dhtSegment(0, 0, DC_LUM_LENGTHS, DC_LUM_SYMBOLS),+ dhtSegment(1, 0, AC_LUM_LENGTHS, AC_LUM_SYMBOLS),+ dhtSegment(0, 1, DC_CHROMA_LENGTHS, DC_CHROMA_SYMBOLS),+ dhtSegment(1, 1, AC_CHROMA_LENGTHS, AC_CHROMA_SYMBOLS),+ ]);+ const sosBody = Buffer.alloc(10);+ sosBody.writeUInt8(3, 0);+ sosBody.writeUInt8(1, 1); sosBody.writeUInt8(0x00, 2);+ sosBody.writeUInt8(2, 3); sosBody.writeUInt8(0x11, 4);+ sosBody.writeUInt8(3, 5); sosBody.writeUInt8(0x11, 6);+ sosBody.writeUInt8(0, 7); sosBody.writeUInt8(63, 8); sosBody.writeUInt8(0, 9);+ const sos = segment(0xda, sosBody);++ return Buffer.concat([+ Buffer.from([0xff, 0xd8]), app0, dqt, sof0, dht, sos, scan, Buffer.from([0xff, 0xd9]),+ ]);+}diff --git a/src/desktop/linux.js b/src/desktop/linux.jsnew file mode 100644index 0000000..80c7bf7--- /dev/null+++ b/src/desktop/linux.js@@ -0,0 +1,806 @@+/**+ * axuse — Linux desktop driver over the zero-dependency X11 client.+ *+ * Lineage: Windows uses a resident PowerShell + SendInput driver and macOS+ * a JXA/CoreGraphics one; both shell out to the OS's scriptable surface.+ * Linux has no guaranteed scripting surface (xdotool/wmctrl are extra apt+ * packages), so this driver speaks X11 directly through src/desktop/x11.js+ * — same contract as the Windows driver (coordinate+vision mode):+ *+ * - input: XTEST FakeInput (trusted, server-synthesized events — the X+ * equivalent of SendInput), strictly ordered like the Windows driver+ * - screenshots: GetImage (ZPixmap) → baseline JPEG (src/desktop/jpeg.js)+ * with the same budget/quality retry contract as GDI+ on Windows+ * - window list/activate: EWMH properties on the root window+ * (_NET_CLIENT_LIST / _NET_WM_NAME / _NET_WM_PID / _NET_ACTIVE_WINDOW)+ *+ * No OS accessibility tree on X11 v1 (AT-SPI is a bus protocol out of scope+ * here) — the ax_* browser tools remain the structure-first surface, and+ * this driver is the fallback for native apps, exactly like Windows today.+ */+import { AxError, Codes } from "../core/errors.js";+import { KEYS, parseCombo } from "../core/actuate.js";+import { DEFAULT_BUDGET_BYTES } from "../core/screenshot.js";+import { X11Client, FakeEventType } from "./x11.js";+import { encodeJPEG } from "./jpeg.js";+import { spawn } from "node:child_process";++/** Shared KEYS-table names → X keysyms (Latin-1 + the standard ranges). */+const KEY_NAME_TO_KEYSYM = {+ enter: 0xff0d, tab: 0xff09, escape: 0xff1b, esc: 0xff1b,+ backspace: 0xff08, delete: 0xffff, del: 0xffff,+ up: 0xff52, down: 0xff54, left: 0xff51, right: 0xff53,+ home: 0xff50, end: 0xff57, pageup: 0xff55, pagedown: 0xff56,+ space: 0x20,+ ctrl: 0xffe3, control: 0xffe3, shift: 0xffe1, alt: 0xffe9,+ meta: 0xffeb, cmd: 0xffeb, command: 0xffeb,+};+for (let i = 1; i <= 12; i++) KEY_NAME_TO_KEYSYM["f" + i] = 0xffbe + (i - 1);++/** keysym for a KEYS-table name, or null. */+function keysymFor(name) {+ if (name.length === 1) {+ const c = name.charCodeAt(0);+ if ((c >= 0x61 && c <= 0x7a) || (c >= 0x30 && c <= 0x39)) return c; // a-z, 0-9+ if (c >= 0x21 && c <= 0xff) return c; // printable latin-1 punctuation etc.+ }+ return KEY_NAME_TO_KEYSYM[name] ?? null;+}++const BUTTON_DETAIL = { left: 1, middle: 2, right: 3 };++/** X11/EWMH driver. Ops are serialized (input ordering must never+ * interleave — the same discipline as the resident Windows driver). */+export class LinuxDriver {+ /** @param {string} [display] @param {{atspiSessionBus?: string}} [opts] */+ constructor(display, opts = {}) {+ this.client = new X11Client(display);+ this.atspiSessionBus = opts.atspiSessionBus;+ this.chain = Promise.resolve();+ }++ /** Serialized op execution; the X11 connection opens lazily on first use. */+ op(fn) {+ const run = async () => {+ if (!this.client.connected) await this.client.connect();+ return fn(this.client);+ };+ this.chain = this.chain.then(run, run);+ return this.chain;+ }++ /* ------------------------------------------------------------- capture */++ async screenInfo() {+ return this.op(async (c) => {+ const g = await c.getGeometry(c.rootWindow);+ const monitors = await c.getMonitors();+ return {+ ok: true,+ x: 0,+ y: 0,+ w: g.w,+ h: g.h,+ coordinate_space: "X11 root window pixels (spans the monitors of the display's default screen)",+ note: "X11/XWayland driver (XTEST input, GetImage capture, EWMH windows)",+ ...(monitors.length ? { monitors } : {}),+ };+ });+ }++ /** GetImage → RGB rows honoring bpp/stride/byte-order. */+ async grabRGB(c, x, y, w, h) {+ const img = await c.getImage(x, y, w, h);+ const { bytesPerPixel: bpp, stride, data, byteOrder } = img;+ if (bpp !== 4 && bpp !== 3) {+ throw new AxError(Codes.DESKTOP_OP_FAILED,+ `Unsupported X11 pixmap format (${bpp} bytes/pixel, depth ${img.depth})`,+ { retry: "never" });+ }+ const rgb = Buffer.alloc(w * h * 3);+ for (let row = 0; row < h; row++) {+ const src = row * stride;+ const dst = row * w * 3;+ if (bpp === 4) {+ if (byteOrder === 0) { // LSBFirst: [B, G, R, X]+ for (let col = 0; col < w; col++) {+ const o = src + col * 4;+ rgb[dst + col * 3] = data[o + 2];+ rgb[dst + col * 3 + 1] = data[o + 1];+ rgb[dst + col * 3 + 2] = data[o];+ }+ } else { // MSBFirst: [X, R, G, B]+ for (let col = 0; col < w; col++) {+ const o = src + col * 4;+ rgb[dst + col * 3] = data[o + 1];+ rgb[dst + col * 3 + 1] = data[o + 2];+ rgb[dst + col * 3 + 2] = data[o + 3];+ }+ }+ } else { // 3 bytes: LSB [B, G, R], MSB [R, G, B]+ if (byteOrder === 0) {+ for (let col = 0; col < w; col++) {+ const o = src + col * 3;+ rgb[dst + col * 3] = data[o + 2];+ rgb[dst + col * 3 + 1] = data[o + 1];+ rgb[dst + col * 3 + 2] = data[o];+ }+ } else {+ data.copy(rgb, dst, src, src + w * 3);+ }+ }+ }+ return rgb;+ }++ /** 2× box downscale of packed RGB rows (vision agents don't need 1080p,+ * and each halving quarters the JPEG cost). */+ static downscale2x(rgb, w, h) {+ const nw = Math.max(1, w >> 1);+ const nh = Math.max(1, h >> 1);+ const out = Buffer.alloc(nw * nh * 3);+ for (let y = 0; y < nh; y++) {+ const r0 = (y * 2) * w * 3;+ const r1 = (y * 2 + 1 < h ? y * 2 + 1 : y * 2) * w * 3;+ const d = y * nw * 3;+ for (let x = 0; x < nw; x++) {+ const c0 = (x * 2) * 3;+ const c1 = (x * 2 + 1 < w ? x * 2 + 1 : x * 2) * 3;+ out[d + x * 3] = (rgb[r0 + c0] + rgb[r0 + c1] + rgb[r1 + c0] + rgb[r1 + c1]) >> 2;+ out[d + x * 3 + 1] = (rgb[r0 + c0 + 1] + rgb[r0 + c1 + 1] + rgb[r1 + c0 + 1] + rgb[r1 + c1 + 1]) >> 2;+ out[d + x * 3 + 2] = (rgb[r0 + c0 + 2] + rgb[r0 + c1 + 2] + rgb[r1 + c0 + 2] + rgb[r1 + c1 + 2]) >> 2;+ }+ }+ return { rgb: out, w: nw, h: nh };+ }++ async screenshot({ x, y, w, h, budgetBytes = DEFAULT_BUDGET_BYTES, quality = 80 } = {}) {+ return this.op(async (c) => {+ let rx = Math.max(0, Math.floor(x ?? 0));+ let ry = Math.max(0, Math.floor(y ?? 0));+ let rw = w === undefined ? c.screenWidth - rx : Math.floor(w);+ let rh = h === undefined ? c.screenHeight - ry : Math.floor(h);+ // Clamp: GetImage rejects regions hanging off the root.+ rw = Math.min(rw, c.screenWidth - rx);+ rh = Math.min(rh, c.screenHeight - ry);+ if (rw < 1 || rh < 1) {+ throw new AxError(Codes.COMMAND_REJECTED,+ `Capture region ${JSON.stringify({ x: rx, y: ry, w: rw, h: rh })} is outside the screen (${c.screenWidth}x${c.screenHeight})`,+ { retry: "reobserve" });+ }+ // Budget ladder, cheapest first: full quality, then q45, then halving+ // the resolution twice (each halving also quarters the encode cost).+ // Vision workflows keep a usable image instead of a typed dead end.+ let { rgb: pixels, w: iw, h: ih } = { rgb: null, w: rw, h: rh };+ const grabFull = async () => {+ pixels = await this.grabRGB(c, rx, ry, rw, rh);+ iw = rw; ih = rh;+ };+ await grabFull();+ let jpg = encodeJPEG(pixels, iw, ih, iw * 3, quality);+ let usedQuality = quality;+ if (jpg.length > budgetBytes && quality > 45) {+ jpg = encodeJPEG(pixels, iw, ih, iw * 3, 45);+ usedQuality = 45;+ }+ for (const _ of [1, 2]) {+ if (jpg.length <= budgetBytes) break;+ if (iw <= 160 || ih <= 160) break; // keep text legible+ const small = LinuxDriver.downscale2x(pixels, iw, ih);+ pixels = small.rgb; iw = small.w; ih = small.h;+ jpg = encodeJPEG(pixels, iw, ih, iw * 3, 70);+ usedQuality = 70;+ }+ const bytes = jpg.length;+ if (bytes > budgetBytes) {+ throw new AxError(Codes.SCREENSHOT_BUDGET_EXCEEDED,+ `Desktop screenshot is ${bytes} bytes, over the ${budgetBytes} byte budget.`,+ { hint: "Pass a crop region ({x,y,w,h} in pixels from screen_info).", retry: "never" });+ }+ return {+ base64: jpg.toString("base64"),+ mime: "image/jpeg",+ bytes,+ budget_bytes: budgetBytes,+ quality: usedQuality,+ ...(iw !== rw || ih !== rh ? { downscaled: `${rw}x${rh} → ${iw}x${ih}` } : {}),+ };+ });+ }++ /* --------------------------------------------------------------- input */++ async move(x, y) {+ await this.op((c) => c.warpPointer(x, y));+ return { action_sent: "sent" };+ }++ async click({ x, y, button = "left", double = false } = {}) {+ const detail = BUTTON_DETAIL[button];+ if (!detail) throw new AxError(Codes.COMMAND_REJECTED, `button must be left|middle|right, got "${button}"`);+ await this.op(async (c) => {+ await c.warpPointer(x, y);+ await c.fakeInput({ type: FakeEventType.ButtonPress, detail });+ await c.fakeInput({ type: FakeEventType.ButtonRelease, detail });+ if (double) {+ await c.fakeInput({ type: FakeEventType.ButtonPress, detail });+ await c.fakeInput({ type: FakeEventType.ButtonRelease, detail });+ }+ });+ return { action_sent: "sent" };+ }++ async down(button = "left") {+ const detail = BUTTON_DOWN[button] ?? 1;+ await this.op((c) => c.fakeInput({ type: FakeEventType.ButtonPress, detail }));+ return { action_sent: "sent" };+ }++ async up(button = "left") {+ const detail = BUTTON_DOWN[button] ?? 1;+ await this.op((c) => c.fakeInput({ type: FakeEventType.ButtonRelease, detail }));+ return { action_sent: "sent" };+ }++ /** Wheel: X buttons 4/5 (vertical) and 6/7 (horizontal); one FakeInput+ * click per notch (120 delta per notch, matching the Windows driver's+ * SendInput WHEEL_DELTA convention). */+ async wheel(dx = 0, dy = 0) {+ await this.op(async (c) => {+ const notches = (d) => Math.max(1, Math.min(50, Math.round(Math.abs(d) / 120) || 1));+ if (dy < 0) for (let i = 0; i < notches(dy); i++) await c.fakeInput({ type: FakeEventType.ButtonPress, detail: 4 }), await c.fakeInput({ type: FakeEventType.ButtonRelease, detail: 4 });+ if (dy > 0) for (let i = 0; i < notches(dy); i++) await c.fakeInput({ type: FakeEventType.ButtonPress, detail: 5 }), await c.fakeInput({ type: FakeEventType.ButtonRelease, detail: 5 });+ if (dx < 0) for (let i = 0; i < notches(dx); i++) await c.fakeInput({ type: FakeEventType.ButtonPress, detail: 6 }), await c.fakeInput({ type: FakeEventType.ButtonRelease, detail: 6 });+ if (dx > 0) for (let i = 0; i < notches(dx); i++) await c.fakeInput({ type: FakeEventType.ButtonPress, detail: 7 }), await c.fakeInput({ type: FakeEventType.ButtonRelease, detail: 7 });+ });+ return { action_sent: "sent" };+ }++ async keyTapKeysym(keysym) {+ await this.op(async (c) => {+ const key = c.keyForKeysym(keysym);+ if (!key) throw untypeableKeysym(keysym);+ if (key.shift) {+ await c.fakeInput({ type: FakeEventType.KeyPress, detail: c.keyForKeysym(0xffe1).keycode });+ await c.fakeInput({ type: FakeEventType.KeyPress, detail: key.keycode });+ await c.fakeInput({ type: FakeEventType.KeyRelease, detail: key.keycode });+ await c.fakeInput({ type: FakeEventType.KeyRelease, detail: c.keyForKeysym(0xffe1).keycode });+ } else {+ await c.fakeInput({ type: FakeEventType.KeyPress, detail: key.keycode });+ await c.fakeInput({ type: FakeEventType.KeyRelease, detail: key.keycode });+ }+ });+ return { action_sent: "sent" };+ }++ async keyDownKeysym(keysym) {+ await this.op(async (c) => {+ const key = c.keyForKeysym(keysym);+ if (!key) throw untypeableKeysym(keysym);+ if (key.shift) await c.fakeInput({ type: FakeEventType.KeyPress, detail: c.keyForKeysym(0xffe1).keycode });+ await c.fakeInput({ type: FakeEventType.KeyPress, detail: key.keycode });+ });+ return { action_sent: "sent" };+ }++ async keyUpKeysym(keysym) {+ await this.op(async (c) => {+ const key = c.keyForKeysym(keysym);+ if (!key) throw untypeableKeysym(keysym);+ await c.fakeInput({ type: FakeEventType.KeyRelease, detail: key.keycode });+ if (key.shift) await c.fakeInput({ type: FakeEventType.KeyRelease, detail: c.keyForKeysym(0xffe1).keycode });+ });+ return { action_sent: "sent" };+ }++ /** Unicode text at the caret. Two paths, chosen per payload:+ * - short, keymap-typeable text → one XTEST keysym press per char (the+ * same per-char discipline as the Windows KEYEVENTF_UNICODE driver)+ * - long (≥48 chars) or non-keymap text with xclip available →+ * clipboard set + trusted ctrl+v paste (fast, layout-proof; the+ * clipboard side effect is reported honestly). */+ async type(text) {+ const s = String(text);+ // Load the server keymap (lazy connect) before probing typeability —+ // an empty map must not push short text onto the clipboard path.+ await this.op(async () => {});+ const chars = [...s];+ const typeable = (ch) => {+ if (ch === "\n" || ch === "\r") return true; // mapped to Return below+ if (ch === "\t") return true; // mapped to Tab below+ let keysym = ch.codePointAt(0);+ if (keysym > 0xff) keysym = 0x01000000 | keysym;+ return !!this.client.keysymToKey.get(keysym);+ };+ const wantsClipboard = chars.length >= 48 || chars.some((ch) => !typeable(ch));+ if (wantsClipboard && await LinuxDriver.clipboardAvailable()) {+ await this.setClipboard(s);+ await this.press("ctrl+v");+ return { action_sent: "sent", via: "clipboard-paste", clipboard_replaced: true };+ }++ await this.op(async (c) => {+ for (const ch of chars) {+ let keysym;+ if (ch === "\n" || ch === "\r") keysym = 0xff0d; // Return+ else if (ch === "\t") keysym = 0xff09; // Tab+ else {+ keysym = ch.codePointAt(0);+ if (keysym > 0xff) keysym = 0x01000000 | keysym; // Unicode range+ }+ const key = c.keyForKeysym(keysym);+ if (!key) {+ throw new AxError(Codes.COMMAND_REJECTED,+ `Character ${JSON.stringify(ch)} has no keysym/keycode on this keyboard layout (XTEST types through the server keymap).`,+ { hint: "Install xclip for the automatic clipboard-paste path, or fill this text via the browser tools.", retry: "never" });+ }+ if (key.shift) await c.fakeInput({ type: FakeEventType.KeyPress, detail: c.keyForKeysym(0xffe1).keycode });+ await c.fakeInput({ type: FakeEventType.KeyPress, detail: key.keycode });+ await c.fakeInput({ type: FakeEventType.KeyRelease, detail: key.keycode });+ if (key.shift) await c.fakeInput({ type: FakeEventType.KeyRelease, detail: c.keyForKeysym(0xffe1).keycode });+ }+ });+ return { action_sent: "sent", via: "keysym" };+ }++ /** xclip presence (the clipboard helper), memoized. */+ static async clipboardAvailable() {+ if (LinuxDriver._clipboardHelper === undefined) {+ LinuxDriver._clipboardHelper = await new Promise((resolve) => {+ const probe = spawn("which", ["xclip"], { stdio: "ignore" });+ probe.on("exit", (code) => resolve(code === 0));+ probe.on("error", () => resolve(false));+ });+ }+ return LinuxDriver._clipboardHelper;+ }++ /** Set the CLIPBOARD selection through xclip (waits until xclip owns it).+ * The driver's OWN display is passed explicitly — never the ambient one. */+ async setClipboard(text) {+ const ok = await new Promise((resolve) => {+ const child = spawn("xclip", ["-display", this.client.display || ":0", "-selection", "clipboard", "-in"], { stdio: ["pipe", "ignore", "ignore"] });+ child.on("error", () => resolve(false));+ child.on("exit", (code) => resolve(code === 0));+ child.stdin.on("error", () => {});+ child.stdin.end(String(text), "utf8");+ });+ if (!ok) {+ throw new AxError(Codes.DESKTOP_OP_FAILED,+ "xclip failed while setting the clipboard (is it installed and is an X server reachable?).",+ { retry: "retry" });+ }+ return true;+ }++ /** Combo press ("ctrl+s") mapped through the shared KEYS table. */+ async press(combo) {+ const { mods } = parseCombo(combo); // validates; throws typed errors+ const mainName = comboMainName(combo);+ const modKeysyms = [];+ for (const m of mods) {+ const ks = keysymFor(m);+ if (!ks) throw new AxError(Codes.COMMAND_REJECTED, `Unknown modifier "${m}" in combo "${combo}"`);+ modKeysyms.push(ks);+ }+ const mainKeysym = keysymFor(mainName);+ if (!mainKeysym) {+ throw new AxError(Codes.COMMAND_REJECTED, `Unknown key "${mainName}" in combo "${combo}"`);+ }+ await this.op(async (c) => {+ for (const ks of modKeysyms) {+ const key = c.keyForKeysym(ks);+ if (!key) throw untypeableKeysym(ks);+ await c.fakeInput({ type: FakeEventType.KeyPress, detail: key.keycode });+ }+ const mainKey = c.keyForKeysym(mainKeysym);+ if (!mainKey) throw untypeableKeysym(mainKeysym);+ if (mainKey.shift && !modKeysyms.length) {+ await c.fakeInput({ type: FakeEventType.KeyPress, detail: c.keyForKeysym(0xffe1).keycode });+ }+ await c.fakeInput({ type: FakeEventType.KeyPress, detail: mainKey.keycode });+ await c.fakeInput({ type: FakeEventType.KeyRelease, detail: mainKey.keycode });+ if (mainKey.shift && !modKeysyms.length) {+ await c.fakeInput({ type: FakeEventType.KeyRelease, detail: c.keyForKeysym(0xffe1).keycode });+ }+ for (let i = modKeysyms.length - 1; i >= 0; i--) {+ const key = c.keyForKeysym(modKeysyms[i]);+ await c.fakeInput({ type: FakeEventType.KeyRelease, detail: key.keycode });+ }+ });+ return { action_sent: "sent", combo };+ }++ /** Drag via move/down/move/up (interpolated like both sibling drivers). */+ async drag({ from, to, steps = 12 } = {}) {+ await this.op(async (c) => {+ await c.warpPointer(from.x, from.y);+ await c.fakeInput({ type: FakeEventType.ButtonPress, detail: 1 });+ for (let i = 1; i <= steps; i++) {+ const x = Math.round(from.x + ((to.x - from.x) * i) / steps);+ const y = Math.round(from.y + ((to.y - from.y) * i) / steps);+ await c.fakeInput({ type: FakeEventType.MotionNotify, detail: 0, x, y });+ }+ await c.fakeInput({ type: FakeEventType.ButtonRelease, detail: 1 });+ });+ return { action_sent: "sent" };+ }++ /* ------------------------------------------------------------- windows */++ async windows() {+ return this.op(async (c) => {+ const ids = await c.getPropertyU32s(c.rootWindow, "_NET_CLIENT_LIST");+ let hiddenAtom = 0;+ try { hiddenAtom = await c.internAtom("_NET_WM_STATE_HIDDEN"); } catch {}+ const rows = [];+ for (const win of ids) {+ const row = { hwnd: win, window_id: win };+ try {+ if (hiddenAtom) {+ const state = await c.getPropertyU32s(win, "_NET_WM_STATE").catch(() => []);+ if (state.includes(hiddenAtom)) continue; // minimized windows are not visible+ }+ const nameProp = await c.getProperty(win, "_NET_WM_NAME").catch(() => null);+ let title = "";+ if (nameProp?.value?.length) title = nameProp.value.toString("utf8");+ if (!title) {+ const wmName = await c.getProperty(win, "WM_NAME").catch(() => null);+ if (wmName?.value?.length) title = wmName.value.toString("latin1");+ }+ row.title = title.replace(/\s+/g, " ").trim();+ const pidProp = await c.getProperty(win, "_NET_WM_PID").catch(() => null);+ if (pidProp?.format === 32 && pidProp.value.length >= 4) row.pid = pidProp.value.readUInt32LE(0);+ const classProp = await c.getProperty(win, "WM_CLASS").catch(() => null);+ if (classProp?.value?.length) {+ const parts = classProp.value.toString("latin1").split("\0").filter(Boolean);+ row.application = parts[parts.length - 1] || undefined;+ }+ try {+ const geom = await c.getGeometry(win);+ const origin = await c.translateCoordinates(win);+ row.bounds = { x: origin.x, y: origin.y, w: geom.w, h: geom.h };+ } catch {}+ } catch {}+ rows.push(row);+ }+ return { windows: rows };+ });+ }++ async activate({ title, hwnd, pid, windowId } = {}) {+ return this.op(async (c) => {+ let target = hwnd ?? windowId ?? null;+ if (!target && pid !== undefined) {+ const ids = await c.getPropertyU32s(c.rootWindow, "_NET_CLIENT_LIST");+ for (const win of ids) {+ const p = await c.getProperty(win, "_NET_WM_PID").catch(() => null);+ if (p?.format === 32 && p.value.length >= 4 && p.value.readUInt32LE(0) === pid) { target = win; break; }+ }+ }+ if (!target && title) {+ const needle = String(title).toLowerCase();+ const ids = await c.getPropertyU32s(c.rootWindow, "_NET_CLIENT_LIST");+ for (const win of ids) {+ const nameProp = await c.getProperty(win, "_NET_WM_NAME").catch(() => null);+ const t = nameProp?.value?.length ? nameProp.value.toString("utf8") : "";+ if (t.toLowerCase().includes(needle)) { target = win; break; }+ }+ }+ if (!target) {+ throw new AxError(Codes.DESKTOP_OP_FAILED,+ "Window not found (activate needs a title substring, pid, or window_id from ax_desktop_windows).",+ { hint: "List windows with ax_desktop_windows and retry.", retry: "retry" });+ }+ const atom = await c.internAtom("_NET_ACTIVE_WINDOW");+ // Source indication 1 (application); the target window rides data.l[2].+ await c.sendClientMessage(target, atom, [1, 0, target, 0, 0]);+ // WM-less fallback (plain Xvfb): focus the window directly so+ // subsequent XTEST key events have a target.+ try { await c.setInputFocus(target); } catch {}+ return { activated: true };+ });+ }++ /* -------------------------------------------------- AT-SPI structure ops */++ /** Lazy AT-SPI connection (structure-first layer, like macOS AX). */+ async ensureAtspi() {+ if (!this.atspi || !this.atspi.connected) {+ const { AtspiClient } = await import("./atspi.js");+ this.atspi = await new AtspiClient({ sessionBusAddress: this.atspiSessionBus }).connect();+ }+ return this.atspi;+ }++ /** Is the AT-SPI structure layer actually usable (tools mount only when+ * true — matching the macOS driver's AX gating, where tools exist only+ * when the platform AX tree answers)? Cached and BOUNDED: activating a+ * dead at-spi registry can take seconds, so a slow/dead probe unmounts+ * the ax tools instead of stalling startup — and its connection attempt+ * is closed when it (eventually) lands, so nothing dangles on the event+ * loop. */+ async axCapable() {+ if (this.#axCapableCache !== null) return this.#axCapableCache;+ const probe = this.ensureAtspi()+ .then(async (atspi) => (await atspi.apps()).some((a) => a.name))+ .catch(() => false);+ const verdict = await Promise.race([+ probe,+ new Promise((resolve) => setTimeout(() => resolve(null), 2_500)),+ ]);+ this.#axCapableCache = verdict === true;+ if (verdict !== true) {+ // Close a connection that already landed; a late one closes itself.+ if (this.atspi) {+ try { this.atspi.close(); } catch {}+ this.atspi = null;+ } else {+ probe.then(() => {+ if (this.#axCapableCache === false) {+ try { this.atspi?.close(); } catch {}+ this.atspi = null;+ }+ }).catch(() => {});+ }+ }+ return this.#axCapableCache;+ }++ #axCapableCache = null;++ /** Resolve the target app root (pid via EWMH cross-reference, exact name,+ * or the frontmost app). Returns the AT-SPI ref. */+ async resolveAtspiApp({ pid, name, front } = {}) {+ const atspi = await this.ensureAtspi();+ const apps = await atspi.apps();+ if (!apps.length) {+ throw new AxError(Codes.DESKTOP_OP_FAILED, "No AT-SPI applications are registered.",+ { hint: "Structure-first desktop control needs apps exposing AT-SPI (GTK/Qt apps do).", retry: "reobserve" });+ }+ if (name) {+ const hit = apps.find((a) => a.name === name);+ if (!hit) {+ throw new AxError(Codes.REF_NOT_FOUND, `No AT-SPI application named "${name}".`,+ { hint: `Available: ${apps.map((a) => a.name).filter(Boolean).join(", ") || "none"}.`, retry: "reobserve" });+ }+ return hit.ref;+ }+ if (pid !== undefined) {+ // AT-SPI app roots carry the app's accessible name; map the EWMH pid+ // → WM_CLASS instance → AT-SPI app name (windows() already knows pids).+ const target = String(pid);+ const rows = (await this.windows()).windows.filter((w) => String(w.pid) === target);+ const candidates = new Set(rows.map((w) => (w.application ?? "").toLowerCase()));+ const hit = apps.find((a) => candidates.has(a.name.toLowerCase()));+ if (!hit) {+ throw new AxError(Codes.REF_NOT_FOUND, `No AT-SPI application matches pid ${pid} (window class: ${[...candidates].join(", ") || "unknown"}).`,+ { hint: `Available: ${apps.map((a) => a.name).filter(Boolean).join(", ") || "none"}.`, retry: "reobserve" });+ }+ return hit.ref;+ }+ // front (default): the app owning the ACTIVE top-level window.+ for (const app of apps) {+ try {+ const wins = await atspi.children(app.ref);+ for (const w of wins) {+ const states = await atspi.states(w);+ if (states.includes("active") || states.includes("focused")) return app.ref;+ }+ } catch {}+ }+ // No active window found (minimized desktop?) — fall back to the first+ // app that actually has UI children.+ for (const app of apps) {+ try {+ if ((await atspi.childCount(app.ref)) > 0) return app.ref;+ } catch {}+ }+ throw new AxError(Codes.DESKTOP_OP_FAILED, "No active AT-SPI application found.",+ { hint: `Available: ${apps.map((a) => a.name).filter(Boolean).join(", ") || "none"}.`, retry: "reobserve" });+ }++ /** Re-resolve an "ax0.3.2"-style ref (child-index path from the app root;+ * "ax0" is the root itself, suffixes walk children). Refs are stale by+ * construction — this fails typed, like macOS. */+ async resolveAtspiRef(ref, appRef) {+ const m = /^ax(\d+(?:\.\d+)*)$/.exec(String(ref ?? ""));+ if (!m) {+ throw new AxError(Codes.REF_NOT_FOUND, `Not an AT-SPI ref: ${JSON.stringify(ref)} (expected "ax0.3.2" from ax_desktop_ax_snapshot).`,+ { retry: "reobserve" });+ }+ const indices = m[1].split(".").map(Number);+ if (indices[0] !== 0) {+ throw new AxError(Codes.REF_NOT_FOUND, `AT-SPI refs are rooted at ax0 (got ${ref}).`,+ { retry: "reobserve" });+ }+ const atspi = await this.ensureAtspi();+ let node = appRef;+ for (const idx of indices.slice(1)) {+ const [child] = await atspi.bus.call(node[0], node[1], "org.a11y.atspi.Accessible", "GetChildAtIndex", "i", [idx]);+ // Out-of-range indices answer with the /org/a11y/atspi/null sentinel.+ if (!child || !child[1] || /\/null$/.test(String(child[1]))) {+ throw new AxError(Codes.REF_NOT_FOUND, `Ref ${ref} went stale (no child at index ${idx}).`,+ { hint: "Take a fresh ax_desktop_ax_snapshot.", retry: "reobserve" });+ }+ node = child;+ }+ return node;+ }++ async axSnapshot({ pid, name, front, maxNodes = 250, maxDepth = 10 } = {}) {+ const atspi = await this.ensureAtspi();+ const appRef = await this.resolveAtspiApp({ pid, name, front });+ const appName = await atspi.name(appRef);+ const nodes = [];+ let truncated = false;+ let frontier = [{ ref: appRef, depth: 0, refPath: "ax0" }];+ const TEXT_ROLES = new Set(["text", "entry", "text box", "spin button", "paragraph", "document text", "terminal"]);+ while (frontier.length) {+ if (nodes.length >= maxNodes) { truncated = true; break; }+ // Pipelined per level: role+name+interfaces for the whole frontier.+ const level = frontier;+ frontier = [];+ const infos = await Promise.all(level.map(async (item) => {+ const [role, nameText, ifaces] = await Promise.all([+ atspi.role(item.ref).catch(() => ""),+ atspi.name(item.ref).catch(() => ""),+ atspi.interfaces(item.ref).catch(() => []),+ ]);+ return { ...item, role, nameText, ifaces };+ }));+ for (const info of infos) {+ if (nodes.length >= maxNodes) { truncated = true; break; }+ const row = { ref: info.refPath, role: info.role || "unknown", name: info.nameText || "" };+ const has = (i) => info.ifaces.some((f) => String(f).endsWith(i));+ // Bounds and text/value fetches are best-effort and only when the+ // interface exists — keeps the walk's round trips bounded.+ if (has("Component")) {+ try {+ const b = await atspi.bounds(info.ref);+ // GNOME reports INT32_MIN sentinels for unplaced actors — drop.+ if (b.w > 0 && b.h > 0 && b.x > -1e9 && b.y > -1e9) row.bounds = b;+ } catch {}+ }+ if (has("Text") && TEXT_ROLES.has((info.role || "").toLowerCase())) {+ try {+ const n = await atspi.charCount(info.ref);+ if (n >= 0 && n <= 4000) row.text = await atspi.text(info.ref, 0, Math.min(n, 2000));+ } catch {}+ }+ if (has("Value")) {+ try { row.value = await atspi.value(info.ref); } catch {}+ }+ nodes.push(row);+ if (info.depth < maxDepth) {+ try {+ const kids = await atspi.children(info.ref);+ frontier.push(...kids.map((k, i) => ({ ref: k, depth: info.depth + 1, refPath: `${info.refPath}.${i}` })));+ } catch {}+ }+ }+ }+ return {+ app: appName,+ nodes,+ truncated,+ node_count: nodes.length,+ note: "AT-SPI snapshot (Linux structure-first): refs are child-index paths from the app root and re-resolve live at action time.",+ };+ }++ async axRead({ ref, pid, name, front } = {}) {+ const atspi = await this.ensureAtspi();+ const appRef = await this.resolveAtspiApp({ pid, name, front });+ const node = await this.resolveAtspiRef(ref, appRef);+ const [role, nameText, description, states, ifaces] = await Promise.all([+ atspi.role(node),+ atspi.name(node).catch(() => ""),+ atspi.description(node).catch(() => ""),+ atspi.states(node).catch(() => []),+ atspi.interfaces(node).catch(() => []),+ ]);+ const has = (i) => ifaces.some((f) => String(f).endsWith(i));+ const out = { ref, role, name: nameText, description, states, interfaces: ifaces };+ if (has("Component")) {+ try {+ const b = await atspi.bounds(node);+ if (b.w > 0 && b.h > 0 && b.x > -1e9 && b.y > -1e9) out.bounds = b;+ } catch {}+ }+ if (has("Text")) {+ try {+ const n = await atspi.charCount(node);+ // Some bridges reject a -1 end offset — clamp to the real count.+ out.text = n >= 0 && n <= 4000 ? await atspi.text(node, 0, Math.min(n, 2000)) : `(text of ${n} chars)`;+ } catch {}+ }+ if (has("Value")) {+ try { out.value = await atspi.value(node); } catch {}+ }+ if (has("Action")) {+ // Name lookups only — GetActions would crash GTK's atk-bridge.+ try {+ const count = await atspi.actionCount(node).catch(() => 0);+ const names = [];+ for (let i = 0; i < Math.min(count, 10); i++) {+ names.push(await atspi.actionName(node, i).catch(() => `action ${i}`));+ }+ if (count > 0) out.actions = names;+ } catch {}+ }+ return out;+ }++ async axClick({ ref, pid, name, front, fallbackToCoords = true } = {}) {+ const atspi = await this.ensureAtspi();+ const appRef = await this.resolveAtspiApp({ pid, name, front });+ const node = await this.resolveAtspiRef(ref, appRef);+ const ifaces = await atspi.interfaces(node).catch(() => []);+ const has = (i) => ifaces.some((f) => String(f).endsWith(i));+ if (has("Action")) {+ // NOTE: never call GetActions here — GTK's atk-bridge (2.4x) CRASHES+ // marshalling its a(sss) reply, killing the app's bus connection.+ // DoAction(i) directly is the safe universal path; the name lookup is+ // best-effort (missing on GTK, present on Qt's bridge).+ try {+ const done = await atspi.doAction(node, 0);+ if (done) {+ const name = await atspi.actionName(node, 0).catch(() => null);+ return { action_sent: "sent", via: "atspi-action", action_index: 0, ...(name ? { action: name } : {}) };+ }+ } catch {+ // Action interface present but refused → coordinate fallback below.+ }+ }+ if (!fallbackToCoords) {+ throw new AxError(Codes.COMMAND_REJECTED,+ `Element ${ref} (${await atspi.role(node)}) exposes no AT-SPI action and the coordinate fallback is disabled.`,+ { retry: "never" });+ }+ const b = await atspi.bounds(node);+ const x = b.x + (b.w >> 1);+ const y = b.y + (b.h >> 1);+ await this.click({ x, y });+ return { action_sent: "sent", via: "coordinate-fallback", clicked: { x, y } };+ }++ async axSet({ ref, value, pid, name, front } = {}) {+ const atspi = await this.ensureAtspi();+ const appRef = await this.resolveAtspiApp({ pid, name, front });+ const node = await this.resolveAtspiRef(ref, appRef);+ const ifaces = await atspi.interfaces(node).catch(() => []);+ const has = (i) => ifaces.some((f) => String(f).endsWith(i));+ if (!has("EditableText")) {+ throw new AxError(Codes.COMMAND_REJECTED,+ `Element ${ref} (${await atspi.role(node)}) is not AT-SPI-editable (interfaces: ${ifaces.join(", ") || "none"}).`,+ { hint: "Use ax_desktop_ax_click and type, or the browser tools for web fields.", retry: "reobserve" });+ }+ const ok = await atspi.setTextContents(node, String(value));+ if (!ok) {+ throw new AxError(Codes.DESKTOP_OP_FAILED, "EditableText.SetTextContents returned false.", { retry: "retry" });+ }+ const readBack = has("Text") ? await atspi.text(node, 0, -1).catch(() => null) : null;+ return { value_set: String(value), via: "atspi-editabletext", ...(readBack !== null ? { read_back_matches: readBack === String(value) } : {}) };+ }++ dispose() {+ try { this.client.close(); } catch {}+ try { this.atspi?.close(); } catch {}+ }+}++const BUTTON_DOWN = { left: 1, middle: 2, right: 3 };++function untypeableKeysym(keysym) {+ return new AxError(Codes.COMMAND_REJECTED,+ `Key with keysym 0x${keysym.toString(16)} has no keycode on this keyboard layout.`,+ { hint: "Prefer named keys (enter, tab, arrows, f1-f12) or the browser tools for text.", retry: "never" });+}++/** The non-modifier part of a combo string ("ctrl+shift+p" → "p"). */+function comboMainName(combo) {+ const parts = String(combo).toLowerCase().split("+").map((s) => s.trim()).filter(Boolean);+ const main = parts.find((p) => !(p in KEY_NAME_TO_KEYSYM && KEY_NAME_TO_KEYSYM[p] >= 0xffe0));+ return main ?? parts[parts.length - 1] ?? "";+}diff --git a/src/desktop/x11.js b/src/desktop/x11.jsnew file mode 100644index 0000000..0115215--- /dev/null+++ b/src/desktop/x11.js@@ -0,0 +1,800 @@+/**+ * axuse — minimal zero-dependency X11 protocol client (Linux desktop driver).+ *+ * Lineage: the Windows driver scripts PowerShell + SendInput, the macOS+ * driver scripts JXA + CoreGraphics. Linux has no comparable "scriptable+ * surface" guaranteed present — xdotool/wmctrl/scrot are extra apt packages+ * — but every X11 server speaks one wire protocol over a Unix socket, and+ * Node ships everything needed to speak it directly:+ *+ * - connection setup + Xauthority cookie (MIT-MAGIC-COOKIE-1) with an+ * empty-auth retry (SI:localuser grants local same-user clients)+ * - core requests: WarpPointer, QueryPointer, GetImage, GetGeometry,+ * TranslateCoordinates, InternAtom, GetProperty, SendEvent,+ * GetKeyboardMapping+ * - XTEST FakeInput — trusted server-side input synthesis, the X+ * equivalent of SendInput (wire layout per the XTEST 2.2 encoding)+ * - EWMH properties for the client window list + _NET_ACTIVE_WINDOW+ *+ * Scope notes (v1):+ * - X11/XWayland only; native Wayland sessions refuse with a typed error.+ * - The root window of the display's default screen is the coordinate+ * space. On the standard single-X-screen layout the root spans every+ * monitor, so warps and captures work across monitors.+ */+import { connect as netConnect } from "node:net";+import { readFileSync } from "node:fs";+import { homedir } from "node:os";+import { join } from "node:path";+import { AxError, Codes } from "../core/errors.js";++const NO_WINDOW = 0;++/** X core request opcodes used here. */+export const Op = Object.freeze({+ GetGeometry: 14, QueryTree: 15, InternAtom: 16, GetProperty: 20, SendEvent: 25,+ QueryPointer: 38, TranslateCoordinates: 40, WarpPointer: 41,+ GetImage: 73, QueryExtension: 98, GetKeyboardMapping: 101,+});++/** FAKE_EVENT_TYPE values (same as the core event codes). */+export const FakeEventType = Object.freeze({+ KeyPress: 2, KeyRelease: 3, ButtonPress: 4, ButtonRelease: 5, MotionNotify: 6,+});++/** Parse a display string ":1", ":1.0", "unix:1", "host:1" → parts. */+export function parseDisplay(display = process.env.DISPLAY ?? "") {+ const m = /^([a-zA-Z0-9._-]*):([0-9]+)(?:\.([0-9]+))?$/.exec(String(display).trim());+ if (!m) return null;+ return { host: m[1], display: Number(m[2]), screen: m[3] === undefined ? null : Number(m[3]) };+}++/** Xauthority records (little-endian u16 lengths, no padding between fields):+ * family(u16) addrLen addr | numLen num | nameLen name | dataLen data */+export function readXauthority(path) {+ let buf;+ try {+ buf = readFileSync(path);+ } catch {+ return [];+ }+ const out = [];+ let o = 0;+ const u16 = () => { const v = buf.readUInt16LE(o); o += 2; return v; };+ const blob = () => { const n = u16(); const b = buf.subarray(o, o + n); o += n; return b; };+ while (o + 2 <= buf.length) {+ const start = o;+ try {+ const family = u16();+ const addr = blob();+ const num = blob();+ const name = blob();+ const data = blob();+ out.push({ family, addr, num: num.toString("latin1"), name: name.toString("latin1"), data });+ } catch {+ o = start;+ break; // truncated record+ }+ }+ return out;+}++/** Best MIT-MAGIC-COOKIE-1 for a display: exact display-number match first,+ * then a wildcard-family record (FamilyWild=65535, FamilyLocal=256). */+function cookieFor(display) {+ const path = process.env.XAUTHORITY || join(homedir(), ".Xauthority");+ const cookies = readXauthority(path).filter((r) => r.name === "MIT-MAGIC-COOKIE-1");+ return cookies.find((r) => r.num === String(display))+ ?? cookies.find((r) => r.family === 65535 || r.family === 256)+ ?? null;+}++function describeXError(code) {+ const names = {+ 1: "BadRequest", 2: "BadValue", 3: "BadWindow", 4: "BadPixmap", 5: "BadAtom",+ 6: "BadCursor", 7: "BadFont", 8: "BadMatch", 9: "BadDrawable", 10: "BadAccess",+ 11: "BadAlloc", 12: "BadColor", 13: "BadGC", 14: "BadIDChoice", 15: "BadName",+ 16: "BadLength", 17: "BadImplementation",+ };+ return names[code] ?? `code ${code}`;+}++/** Static evdev keymap (keysym → [keycode = KEY_* + 8, shift]) for servers+ * whose core keyboard map is empty (XKB-owned). US-layout punctuation; the+ * server map wins whenever it is populated. */+const EVDEV_KEYMAP = (() => {+ const rows = [];+ const push = (keysym, keycode, shift) => rows.push([keysym, keycode, shift]);+ // Letters in QWERTY PHYSICAL order (evdev keycodes; X keycode = code + 8):+ // q w e r t y u i o p → 24..33, a s d f g h j k l → 38..46,+ // z x c v b n m → 52..58. Alphabetical order would be wrong — the map is+ // positional, not alphabetical.+ const letterRows = [+ ["qwertyuiop", 24],+ ["asdfghjkl", 38],+ ["zxcvbnm", 52],+ ];+ for (const [letters, base] of letterRows) {+ for (let i = 0; i < letters.length; i++) {+ const c = letters.charCodeAt(i);+ push(c, base + i, false); // lowercase+ push(c - 32, base + i, true); // uppercase (shift)+ }+ }+ // Digits row (KEY_1=2 → keycode 10 … KEY_0=11 → 19)+ const digits = "1234567890";+ const shifted = "!@#$%^&*()";+ for (let i = 0; i < 10; i++) {+ push(digits.charCodeAt(i), 10 + i, false);+ push(shifted.charCodeAt(i), 10 + i, true);+ }+ const named = [+ // [keysym, keycode, shift]+ [0xff0d, 36, false], [0xff09, 23, false], [0xff1b, 9, false], [0xff08, 22, false],+ [0xffff, 119, false], [0xff63, 118, false],+ [0xff52, 111, false], [0xff54, 116, false], [0xff51, 113, false], [0xff53, 114, false],+ [0xff50, 110, false], [0xff57, 115, false], [0xff55, 112, false], [0xff56, 117, false],+ [0x20, 65, false],+ [0xffe1, 50, false], [0xffe2, 62, false], // Shift_L/R+ [0xffe3, 37, false], [0xffe4, 105, false], // Control_L/R+ [0xffe9, 64, false], [0xffea, 108, false], // Alt_L/R+ [0xffeb, 133, false], [0xffec, 134, false], // Super_L/R+ [0xffe5, 66, false], [0xff7f, 77, false], // Caps_Lock, Num_Lock+ // US punctuation (unshifted, shifted)+ [0x2d, 20, false], [0x5f, 20, true], // - _+ [0x3d, 21, false], [0x2b, 21, true], // = ++ [0x5b, 34, false], [0x7b, 34, true], // [ {+ [0x5d, 35, false], [0x7d, 35, true], // ] }+ [0x5c, 51, false], [0x7c, 51, true], // \ |+ [0x3b, 47, false], [0x3a, 47, true], // ; :+ [0x27, 48, false], [0x22, 48, true], // ' "+ [0x60, 49, false], [0x7e, 49, true], // ` ~+ [0x2c, 59, false], [0x3c, 59, true], // , <+ [0x2e, 60, false], [0x3e, 60, true], // . >+ [0x2f, 61, false], [0x3f, 61, true], // / ?+ ];+ rows.push(...named);+ for (let i = 0; i < 12; i++) push(0xffbe + i, 67 + i, false); // F1-F12+ // Numeric keypad: KP_0=90, KP_1=87 … KP_9=81 (evdev + 8).+ const kpDigits = [[0xffb0, 90], [0xffb1, 87], [0xffb2, 88], [0xffb3, 89], [0xffb4, 83],+ [0xffb5, 84], [0xffb6, 85], [0xffb7, 79], [0xffb8, 80], [0xffb9, 81]];+ for (const [keysym, keycode] of kpDigits) push(keysym, keycode, false);+ push(0xffab, 86, false); // KP_Add+ push(0xffad, 82, false); // KP_Subtract+ push(0xaa, 63, false); // KP_Multiply+ push(0xaf, 106, false); // KP_Divide+ push(0xff8b, 104, false); // KP_Enter+ push(0xff9e, 91, false); // KP_Decimal+ return rows;+})();++/**+ * A small synchronous-request-at-a-time X11 client. Desktop input must be+ * strictly ordered anyway (the driver serializes ops like WindowsDriver),+ * so a sequence-keyed pending map is sufficient — no concurrent pipelines.+ */+export class X11Client {+ /** @param {string} [display] */+ constructor(display) {+ this.display = String(display ?? process.env.DISPLAY ?? "");+ this.displaySpec = parseDisplay(this.display);+ this.socket = null;+ this.seq = 0;+ this.pending = new Map(); // seq → {resolve, reject}+ this.buffer = Buffer.alloc(0);+ this.setupWaiters = []; // pre-handshake readers (waitFor)+ this.rootWindow = 0;+ this.rootCandidates = [];+ this.screenWidth = 0;+ this.screenHeight = 0;+ this.minKeycode = 0;+ this.maxKeycode = 0;+ this.imageByteOrder = 0; // 0 LSBFirst, 1 MSBFirst+ this.pixmapFormats = new Map(); // depth → bits-per-pixel+ this.xtestOpcode = 0;+ this.atoms = new Map();+ this.keysymToKey = new Map(); // keysym → {keycode, shift}+ this.dead = false;+ }++ get connected() {+ return !!this.socket && !this.dead;+ }++ /* ----------------------------------------------------------- connection */++ async connect() {+ this.dead = false; // a disposed client may reconnect (driver respawn)+ if (!this.displaySpec) {+ throw new AxError(Codes.DESKTOP_DRIVER_UNAVAILABLE,+ process.env.WAYLAND_DISPLAY+ ? "This looks like a native Wayland session with no usable DISPLAY; the Linux desktop driver speaks X11 (XWayland covers most desktop apps — set DISPLAY, usually :0, :1 or similar)."+ : "DISPLAY is not set; the Linux desktop driver needs an X11 session (run the harness inside the desktop session, or under xvfb-run for a virtual display).",+ { hint: "Desktop control on Linux targets X11/XWayland; browser tools work everywhere.", retry: "never" });+ }+ const { host, display } = this.displaySpec;+ await new Promise((resolve, reject) => {+ const sock = (host === "" || host === "unix")+ ? netConnect(`/tmp/.X11-unix/X${display}`)+ : netConnect({ host, port: 6000 + display });+ const fail = (e) => reject(new AxError(Codes.DESKTOP_DRIVER_UNAVAILABLE,+ `Cannot open X display ${process.env.DISPLAY ?? ""}: ${e.message}`,+ { hint: "Run inside the X session (XAUTHORITY may need to point at the session cookie), or under xvfb-run.", retry: "never" }));+ sock.once("error", fail);+ sock.once("connect", () => {+ sock.off("error", fail);+ this.socket = sock;+ resolve();+ });+ });++ try {+ await this.handshake(cookieFor(display));+ } catch (e) {+ // A mismatched cookie is the common cause; SI:localuser access control+ // admits same-user clients without any cookie — retry cookie-less.+ if (e instanceof AxError && e.code === Codes.DESKTOP_DRIVER_UNAVAILABLE) {+ this.teardownSocket();+ await this.reopenSocket();+ await this.handshake(null);+ } else {+ throw e;+ }+ }+ this.socket.on("data", (d) => this.routeData(d));+ this.socket.on("close", () => this.failAll("X11 connection closed"));+ this.socket.on("error", () => this.failAll("X11 connection error"));++ if (!(await this.resolveRoot())) {+ throw new AxError(Codes.INTERNAL, "No candidate root window answered GetGeometry.");+ }+ this.xtestOpcode = (await this.queryExtension("XTEST")).majorOpcode;+ if (!this.xtestOpcode) {+ throw new AxError(Codes.DESKTOP_DRIVER_UNAVAILABLE,+ "The X server does not support the XTEST extension; trusted input synthesis is unavailable.",+ { retry: "never" });+ }+ await this.loadKeyboardMap();+ return this;+ }++ async reopenSocket() {+ const { host, display } = this.displaySpec;+ await new Promise((resolve, reject) => {+ const sock = (host === "" || host === "unix")+ ? netConnect(`/tmp/.X11-unix/X${display}`)+ : netConnect({ host, port: 6000 + display });+ sock.once("error", reject);+ sock.once("connect", () => { this.socket = sock; resolve(); });+ });+ this.buffer = Buffer.alloc(0);+ this.dead = false;+ }++ teardownSocket() {+ try { this.socket?.destroy(); } catch {}+ this.socket = null;+ }++ /** Send the connection setup with an (optional) auth cookie and parse it. */+ async handshake(cookie) {+ const name = "MIT-MAGIC-COOKIE-1";+ const data = cookie?.data ?? Buffer.alloc(0);+ const pad4 = (n) => (4 - (n % 4)) % 4;+ const head = Buffer.alloc(12);+ head.writeUInt8(0x6c, 0); // 'l' — little-endian+ head.writeUInt8(0, 1);+ head.writeUInt16LE(11, 2); // protocol 11.0+ head.writeUInt16LE(0, 4);+ head.writeUInt16LE(name.length, 6);+ head.writeUInt16LE(data.length, 8);+ head.writeUInt16LE(0, 10);+ this.socket.write(Buffer.concat([+ head,+ Buffer.from(name, "latin1"), Buffer.alloc(pad4(name.length)),+ data, Buffer.alloc(pad4(data.length)),+ ]));++ const first = await this.waitFor(8);+ // The setup length field counts 4-byte units AFTER the 8-byte header,+ // INCLUDING the 32-byte fixed part (release/ids/keycodes…): the whole+ // packet is 8 + length*4 bytes (verified against Xorg 1.21 wire).+ const total = 8 + first.readUInt16LE(6) * 4;+ const full = Buffer.concat([first, await this.waitFor(total - 8)]);+ if (full[0] === 2) {+ // SetupFailed: [2][reasonLen][major][minor][len16][reason…]+ const reason = full.subarray(8).toString("utf8").replace(/\0.*$/s, "").trim();+ throw new AxError(Codes.DESKTOP_DRIVER_UNAVAILABLE,+ `The X server refused the connection${reason ? `: ${reason}` : ""}.`,+ { hint: "Auth mismatch — run inside the X session or point XAUTHORITY at the session cookie.", retry: "never" });+ }+ if (full[0] !== 1) throw new AxError(Codes.INTERNAL, `Unexpected X11 setup reply type ${full[0]}`);+ // Fixed-field layout verified byte-by-byte against Xorg 1.21.1 wire+ // (Ubuntu 22.04–26.04 family; LSB connection):+ // vlen@24, maxReqLen@26, pad@28, nformats@29, imageByteOrder@30,+ // bitmapBitOrder@31, scanlineUnit@32, scanlinePad@33, minKey@34,+ // maxKey@35, [email protected], vendor@40, formats (8·nformats),+ // then screens. Single-byte fields are junk-guarded (some builds+ // leave "unused" slots uninitialized).+ this.imageByteOrder = full[30] <= 1 ? full[30] : 0;+ this.minKeycode = full[34];+ this.maxKeycode = full[35];+ if (!(this.minKeycode < this.maxKeycode)) {+ // Fallback: the canonical evdev range every mainstream server uses.+ this.minKeycode = 8;+ this.maxKeycode = 255;+ }+ const vendorLen = full.readUInt16LE(24);+ const afterVendor = 40 + vendorLen + ((4 - (vendorLen % 4)) % 4);+ // Pixmap formats ([depth, bits-per-pixel, scanline-pad, 5 junk] × 8):+ // Xorg maps depth 24 → 32 bpp — GetImage rows must use this, not depth/8.+ this.pixmapFormats = new Map();+ const fmtCount = full[29] >= 1 && full[29] <= 16 ? full[29] : 6;+ for (let i = 0; i < fmtCount && afterVendor + i * 8 + 2 <= full.length; i++) {+ const depth = full[afterVendor + i * 8];+ const bitsPerPixel = full[afterVendor + i * 8 + 1];+ if (depth > 0 && bitsPerPixel > 0) this.pixmapFormats.set(depth, bitsPerPixel);+ }+ // Screen 0 root candidates: the declared pixmap-format count when it+ // parses sanely, else a scan — the true root is confirmed against the+ // live server by GetGeometry in resolveRoot().+ this.rootCandidates = [];+ const nformats = full[29] >= 1 && full[29] <= 16 ? full[29] : null;+ for (const formats of nformats !== null ? [nformats] : [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 14, 16]) {+ const s = afterVendor + formats * 8;+ if (s + 24 > full.length) break;+ const root = full.readUInt32LE(s);+ // Screen record: root(4) colormap(4) white(4) black(4) masks(4)+ // then width/height in pixels at s+20/s+22.+ const w = full.readUInt16LE(s + 20);+ const h = full.readUInt16LE(s + 22);+ if (root !== 0 && w > 0 && h > 0 && w <= 32767 && h <= 32767) {+ this.rootCandidates.push({ root, w, h });+ }+ }+ if (!this.rootCandidates.length) {+ throw new AxError(Codes.INTERNAL, "Could not parse the X11 connection setup screen list.");+ }+ }++ /** Pick the real root window: the candidate whose GetGeometry round-trips+ * on the live connection (formats/visual bytes can masquerade as screens+ * to a heuristic, but only a real window answers GetGeometry). */+ async resolveRoot() {+ for (const cand of this.rootCandidates ?? []) {+ try {+ const g = await this.getGeometry(cand.root);+ if (g.w > 0 && g.h > 0) {+ this.rootWindow = cand.root;+ this.screenWidth = g.w;+ this.screenHeight = g.h;+ return true;+ }+ } catch {+ // BadWindow → wrong candidate, keep scanning.+ }+ }+ return false;+ }++ /** Accumulate `n` bytes during setup (sole consumer of socket data). */+ waitFor(n) {+ if (this.buffer.length >= n) {+ const out = this.buffer.subarray(0, n);+ this.buffer = this.buffer.subarray(n);+ return Promise.resolve(out);+ }+ return new Promise((resolve, reject) => {+ const w = { n, resolve, reject };+ this.setupWaiters.push(w);+ const onChunk = (d) => { this.buffer = Buffer.concat([this.buffer, d]); this.pumpSetup(); };+ const onClose = () => {+ this.setupWaiters.splice(this.setupWaiters.indexOf(w), 1);+ reject(new AxError(Codes.DESKTOP_DRIVER_UNAVAILABLE, "X11 connection closed during handshake"));+ };+ this.socket.on("data", onChunk);+ this.socket.once("close", onClose);+ w.cleanup = () => { this.socket.off("data", onChunk); this.socket.off("close", onClose); };+ this.pumpSetup();+ });+ }++ pumpSetup() {+ while (this.setupWaiters.length && this.buffer.length >= this.setupWaiters[0].n) {+ const w = this.setupWaiters.shift();+ const out = this.buffer.subarray(0, w.n);+ this.buffer = this.buffer.subarray(w.n);+ w.cleanup?.();+ w.resolve(out);+ }+ }++ /* ------------------------------------------------------- reply routing */++ routeData(chunk) {+ this.buffer = Buffer.concat([this.buffer, chunk]);+ for (;;) {+ if (this.buffer.length < 32) return;+ const first = this.buffer[0];+ if (first === 0) {+ // X error: [0][code][seq][value(4)][minor(2)][major(1)][pad(21)]+ const code = this.buffer[1];+ const seq = this.buffer.readUInt16LE(2);+ const value = this.buffer.readUInt32LE(4);+ const minor = this.buffer.readUInt16LE(8);+ const major = this.buffer[10];+ this.buffer = this.buffer.subarray(32);+ const p = this.pending.get(seq);+ if (p) {+ this.pending.delete(seq);+ p.reject(new AxError(Codes.DESKTOP_OP_FAILED,+ `X11 ${describeXError(code)} on request ${major}.${minor} (value ${value})`, { retry: "retry" }));+ }+ continue;+ }+ if (first === 1) {+ const seq = this.buffer.readUInt16LE(2);+ const extra = this.buffer.readUInt32LE(4) * 4;+ const total = 32 + extra;+ if (this.buffer.length < total) return;+ const reply = Buffer.from(this.buffer.subarray(0, total)); // copy: buffer window gets sliced+ this.buffer = this.buffer.subarray(total);+ const p = this.pending.get(seq);+ if (p) {+ this.pending.delete(seq);+ p.resolve(reply);+ }+ continue;+ }+ // Event (2..64): v1 routes nothing — skip one 32-byte frame.+ this.buffer = this.buffer.subarray(32);+ }+ }++ failAll(reason) {+ this.dead = true;+ for (const [, p] of this.pending) p.reject(new AxError(Codes.DESKTOP_OP_FAILED, reason));+ this.pending.clear();+ }++ /** Send a fully-assembled request buffer (length slot at bytes 2..4 is+ * filled in here from the buffer size). Resolves on the reply when+ * `wantReply`, else immediately after the write. */+ send(buf, wantReply = false) {+ if (!this.socket || this.dead) {+ return Promise.reject(new AxError(Codes.DESKTOP_OP_FAILED, "X11 connection is not open"));+ }+ if (buf.length % 4 !== 0) throw new Error("X11 request must be 4-byte aligned");+ buf.writeUInt16LE(buf.length / 4, 2);+ // The server numbers EVERY request (replies AND no-reply requests);+ // the counter must advance even when we expect nothing back.+ const seq = (this.seq = (this.seq + 1) & 0xffff);+ if (!wantReply) {+ this.socket.write(buf);+ return Promise.resolve();+ }+ return new Promise((resolve, reject) => {+ this.pending.set(seq, { resolve, reject });+ this.socket.write(buf);+ });+ }++ /* ------------------------------------------------------------- requests */++ async queryExtension(name) {+ const n = Buffer.byteLength(name);+ const buf = Buffer.alloc(4 + 4 + n + ((4 - (n % 4)) % 4));+ buf.writeUInt8(Op.QueryExtension, 0);+ buf.writeUInt16LE(2 + (4 + n) / 4, 2);+ buf.writeUInt16LE(n, 4);+ buf.write(name, 8, "latin1");+ const rep = await this.send(buf, true);+ // Field layout verified on the wire (Xorg 1.21 LSB): present BOOL at+ // byte 8, major opcode / first event / first error as single bytes.+ return rep[8] === 1 ? { present: true, majorOpcode: rep[9] } : { present: false, majorOpcode: 0 };+ }++ async internAtom(name) {+ const cached = this.atoms.get(name);+ if (cached) return cached;+ const n = Buffer.byteLength(name);+ const buf = Buffer.alloc(4 + 4 + n + ((4 - (n % 4)) % 4));+ buf.writeUInt8(Op.InternAtom, 0);+ buf.writeUInt8(1, 1); // only if exists+ buf.writeUInt16LE(2 + (4 + n) / 4, 2);+ buf.writeUInt16LE(n, 4);+ buf.write(name, 8, "latin1");+ const rep = await this.send(buf, true);+ const atom = rep.readUInt32LE(8);+ this.atoms.set(name, atom);+ return atom;+ }++ async getProperty(window, name, { longLength = 4096 } = {}) {+ const prop = typeof name === "string" ? await this.internAtom(name) : name;+ if (!prop) {+ // Atom not present (e.g. EWMH atoms without a running window manager):+ // the property cannot exist — report empty rather than BadAtom.+ return { format: 0, typeAtom: 0, bytesAfter: 0, valueLen: 0, value: Buffer.alloc(0) };+ }+ const buf = Buffer.alloc(24);+ buf.writeUInt8(Op.GetProperty, 0);+ buf.writeUInt8(0, 1); // delete = false+ buf.writeUInt16LE(6, 2);+ buf.writeUInt32LE(window, 4);+ buf.writeUInt32LE(prop, 8);+ buf.writeUInt32LE(0, 12); // type = AnyPropertyType+ buf.writeUInt32LE(0, 16); // long-offset+ buf.writeUInt32LE(longLength, 20);+ const rep = await this.send(buf, true);+ return {+ format: rep[1],+ typeAtom: rep.readUInt32LE(8),+ bytesAfter: rep.readUInt32LE(12),+ valueLen: rep.readUInt32LE(16),+ value: rep.subarray(32, 32 + rep.readUInt32LE(16)),+ };+ }++ /** GetProperty value as a CARD32 list (window ids, pids…). */+ async getPropertyU32s(window, name) {+ const { format, value } = await this.getProperty(window, name);+ if (format !== 32) return [];+ const out = [];+ for (let o = 0; o + 4 <= value.length; o += 4) out.push(value.readUInt32LE(o));+ return out;+ }++ async getGeometry(drawable) {+ const buf = Buffer.alloc(8);+ buf.writeUInt8(Op.GetGeometry, 0);+ buf.writeUInt16LE(2, 2);+ buf.writeUInt32LE(drawable, 4);+ const rep = await this.send(buf, true);+ return {+ depth: rep[1],+ root: rep.readUInt32LE(8),+ x: rep.readInt16LE(12),+ y: rep.readInt16LE(14),+ w: rep.readUInt16LE(16),+ h: rep.readUInt16LE(18),+ };+ }++ async queryPointer() {+ const buf = Buffer.alloc(8);+ buf.writeUInt8(Op.QueryPointer, 0);+ buf.writeUInt16LE(2, 2);+ buf.writeUInt32LE(this.rootWindow, 4);+ const rep = await this.send(buf, true);+ return {+ sameScreen: rep[1] === 1,+ child: rep.readUInt32LE(12),+ rootX: rep.readInt16LE(16),+ rootY: rep.readInt16LE(18),+ };+ }++ async warpPointer(x, y) {+ // 24 bytes (length 6): dst-x/dst-y are INT16 per the protocol encoding+ // (verified against Xorg 1.21 — a 28-byte form draws BadLength).+ const buf = Buffer.alloc(24);+ buf.writeUInt8(Op.WarpPointer, 0);+ buf.writeUInt16LE(6, 2);+ buf.writeUInt32LE(NO_WINDOW, 4); // src = None+ buf.writeUInt32LE(this.rootWindow, 8); // dst = root+ buf.writeInt16LE(0, 12); // src x+ buf.writeInt16LE(0, 14); // src y+ buf.writeUInt16LE(0, 16); // src width 0 = whole+ buf.writeUInt16LE(0, 18); // src height+ buf.writeInt16LE(Math.round(x), 20); // dst x+ buf.writeInt16LE(Math.round(y), 22); // dst y+ await this.send(buf);+ }++ /** XTEST FakeInput (36 bytes, length 9): keys use server keycodes, buttons+ * 1-7 (4/5 = wheel up/down, 6/7 = wheel left/right), motion with+ * detail 0 = absolute rootX/rootY. delay 0 = immediate. */+ async fakeInput({ type, detail = 0, delayMs = 0, x, y } = {}) {+ if (!this.xtestOpcode) throw new AxError(Codes.DESKTOP_OP_FAILED, "XTEST extension not available");+ const buf = Buffer.alloc(36);+ buf.writeUInt8(this.xtestOpcode, 0);+ buf.writeUInt8(2, 1); // FakeInput+ buf.writeUInt8(type, 4);+ buf.writeUInt8(detail, 5);+ buf.writeUInt32LE(delayMs, 8);+ // root (0 = current root) at 12; 8 unused; rootX/rootY at 24/26.+ if (x !== undefined) buf.writeInt16LE(Math.round(x), 24);+ if (y !== undefined) buf.writeInt16LE(Math.round(y), 26);+ await this.send(buf);+ }++ /** ZPixmap grab of the root window (clamped to bounds by the caller).+ * Returns raw rows + layout facts for the RGB conversion step. */+ async getImage(x, y, w, h) {+ const buf = Buffer.alloc(20); // 5 units: opcode, format, drawable, x/y/w/h, plane-mask+ buf.writeUInt8(Op.GetImage, 0);+ buf.writeUInt8(2, 1); // format = ZPixmap+ buf.writeUInt32LE(this.rootWindow, 4);+ buf.writeInt16LE(Math.round(x), 8);+ buf.writeInt16LE(Math.round(y), 10);+ buf.writeUInt16LE(w, 12);+ buf.writeUInt16LE(h, 14);+ buf.writeUInt32LE(0xffffffff, 16); // plane mask = all+ const rep = await this.send(buf, true);+ const depth = rep[1];+ const data = rep.subarray(32);+ const bitsPerPixel = this.pixmapFormats?.get(depth) ?? (depth >= 24 ? 32 : depth);+ const bytesPerPixel = Math.max(1, bitsPerPixel / 8);+ const stride = Math.ceil((w * bytesPerPixel) / 4) * 4;+ return { depth, bytesPerPixel, stride, w, h, data, byteOrder: this.imageByteOrder };+ }++ /** Fill keysym → {keycode, shift} from the server keyboard map. Servers+ * whose keyboard is fully owned by XKB (Xvfb, some minimal setups) answer+ * the core query with an empty map — then fall back to the static evdev+ * table (X keycode = Linux KEY_* + 8, stable across distros). */+ async loadKeyboardMap() {+ const count = this.maxKeycode - this.minKeycode + 1;+ if (count > 0) {+ const buf = Buffer.alloc(8);+ buf.writeUInt8(Op.GetKeyboardMapping, 0);+ buf.writeUInt8(this.minKeycode, 1);+ buf.writeUInt16LE(2, 2);+ buf.writeUInt8(count, 4);+ const rep = await this.send(buf, true);+ const perKeycode = rep[1] || 2;+ const nKeysyms = rep.readUInt32LE(4); // reply length in CARD32 units+ this.keysymToKey.clear();+ for (let k = 0; k < count; k++) {+ for (let col = 0; col < perKeycode; col++) {+ const idx = k * perKeycode + col;+ if (idx >= nKeysyms) break;+ const keysym = rep.readUInt32LE(32 + idx * 4);+ if (!keysym) continue;+ // Column parity encodes Shift on the primary group; register the+ // first mapping for each keysym (group-2 columns only add).+ if (!this.keysymToKey.has(keysym)) {+ this.keysymToKey.set(keysym, { keycode: this.minKeycode + k, shift: col % 2 === 1 });+ }+ }+ }+ if (this.keysymToKey.size > 0) return this.keysymToKey.size;+ }+ this.applyEvdevMap();+ return this.keysymToKey.size;+ }++ /** The static evdev/XKB fallback map: [keysym, keycode, needsShift]. */+ applyEvdevMap() {+ this.keysymToKey.clear();+ for (const [keysym, keycode, shift] of EVDEV_KEYMAP) {+ this.keysymToKey.set(keysym, { keycode, shift: !!shift });+ }+ }++ keyForKeysym(keysym) {+ return this.keysymToKey.get(keysym) ?? null;+ }++ /** A child window's origin in root coordinates. */+ async translateCoordinates(window) {+ const buf = Buffer.alloc(16);+ buf.writeUInt8(Op.TranslateCoordinates, 0);+ buf.writeUInt16LE(4, 2);+ buf.writeUInt32LE(window, 4);+ buf.writeUInt32LE(this.rootWindow, 8);+ buf.writeInt16LE(0, 12);+ buf.writeInt16LE(0, 14);+ const rep = await this.send(buf, true);+ return { child: rep.readUInt32LE(8), x: rep.readInt16LE(12), y: rep.readInt16LE(14) };+ }++ /** SetInputFocus — the WM-less fallback so XTEST keys have a target. */+ async setInputFocus(window) {+ const buf = Buffer.alloc(12);+ buf.writeUInt8(42, 0); // SetInputFocus+ buf.writeUInt8(1, 1); // revert-to: PointerRoot+ buf.writeUInt16LE(3, 2);+ buf.writeUInt32LE(window, 4);+ buf.writeUInt32LE(0, 8); // CurrentTime+ await this.send(buf);+ }++ /* ------------------------------------------------------------ RandR ---- */++ #randrOpcode = 0;+ #randrConfigTimestamp = 0;++ async randrOpcode() {+ if (this.#randrOpcode === 0) {+ const { present, majorOpcode } = await this.queryExtension("RANDR");+ this.#randrOpcode = present ? majorOpcode : -1;+ }+ return this.#randrOpcode > 0 ? this.#randrOpcode : 0;+ }++ /** Active monitor layout via RandR 1.2 (RRGetScreenResourcesCurrent ++ * RRGetCrtcInfo + RRGetOutputPrimary). Returns [] without RandR. */+ async getMonitors() {+ const ext = await this.randrOpcode();+ if (!ext) return [];+ try {+ // RRGetScreenResourcesCurrent (minor 25, per randrproto encoding)+ const req = Buffer.alloc(8);+ req.writeUInt8(ext, 0);+ req.writeUInt8(25, 1);+ req.writeUInt16LE(2, 2);+ req.writeUInt32LE(this.rootWindow, 4);+ const res = await this.send(req, true);+ this.#randrConfigTimestamp = res.readUInt32LE(12);+ const numCrtcs = res.readUInt16LE(16);+ const crtcs = [];+ for (let i = 0; i < numCrtcs; i++) {+ crtcs.push(res.readUInt32LE(32 + i * 4));+ }+ // RRGetOutputPrimary (minor 31) → xid+ const pReq = Buffer.alloc(8);+ pReq.writeUInt8(ext, 0);+ pReq.writeUInt8(31, 1);+ pReq.writeUInt16LE(2, 2);+ pReq.writeUInt32LE(this.rootWindow, 4);+ let primary = 0;+ try {+ const p = await this.send(pReq, true);+ primary = p.readUInt32LE(8);+ } catch {}++ const monitors = [];+ for (const crtc of crtcs) {+ // RRGetCrtcInfo (minor 20): reply carries timestamp, x/y/w/h, mode…+ const cReq = Buffer.alloc(12);+ cReq.writeUInt8(ext, 0);+ cReq.writeUInt8(20, 1);+ cReq.writeUInt16LE(3, 2);+ cReq.writeUInt32LE(crtc, 4);+ cReq.writeUInt32LE(this.#randrConfigTimestamp, 8);+ const info = await this.send(cReq, true);+ const mode = info.readUInt32LE(20);+ if (!mode) continue; // disabled crtc+ const numOutputs = info.readUInt16LE(28);+ const firstOutput = numOutputs ? info.readUInt32LE(32) : 0;+ monitors.push({+ x: info.readInt16LE(12),+ y: info.readInt16LE(14),+ w: info.readUInt16LE(16),+ h: info.readUInt16LE(18),+ primary: firstOutput === primary,+ });+ }+ return monitors;+ } catch {+ return []; // RandR shapes vary; the root union stays the source of truth+ }+ }++ /** SendEvent of a format-32 ClientMessage to the root window (EWMH).+ * 44-byte request: SendEvent header (12) + 32-byte event, where the+ * ClientMessage event packs type(1) format(1) seq(2) window(4)+ * message-type(4) data.l[5](20) with no pad. */+ async sendClientMessage(window, messageTypeAtom, data = [0, 0, 0, 0, 0]) {+ const buf = Buffer.alloc(44);+ buf.writeUInt8(Op.SendEvent, 0);+ buf.writeUInt8(1, 1); // propagate+ buf.writeUInt16LE(11, 2);+ buf.writeUInt32LE(this.rootWindow, 4); // destination+ buf.writeUInt32LE((1 << 19) | (1 << 20), 8); // SubstructureNotify|SubstructureRedirect+ buf.writeUInt8(33, 12); // ClientMessage+ buf.writeUInt8(32, 13); // format 32+ buf.writeUInt32LE(window, 16);+ buf.writeUInt32LE(messageTypeAtom, 20);+ for (let i = 0; i < 5; i++) buf.writeUInt32LE((data[i] ?? 0) >>> 0, 24 + i * 4);+ await this.send(buf);+ }++ close() {+ this.dead = true;+ this.failAll("X11 client closed");+ this.teardownSocket();+ }+}diff --git a/src/mcp/manifest.js b/src/mcp/manifest.jsindex 57740e9..b4737d7 100644--- a/src/mcp/manifest.js+++ b/src/mcp/manifest.js@@ -178,7 +178,7 @@ export function desktopTools(driverAvailable, axCapable = false) {},...((axCapable) ? [{name: "ax_desktop_ax_snapshot",- description: "STRUCTURE-FIRST desktop snapshot (macOS): the app's accessibility tree with roles, titles, values, bounds, and live refs — the desktop twin of ax_observe. Pick the app by pid (from ax_desktop_windows), exact name, or front:true for the active app. Budgeted (250 nodes / depth 10 defaults; truncation reported as facts). Refs re-resolve at action time; act with ax_desktop_ax_click / set / read.",+ description: "STRUCTURE-FIRST desktop snapshot (macOS System Events / Linux AT-SPI): the app's accessibility tree with roles, titles, values, bounds, and live refs — the desktop twin of ax_observe. Pick the app by pid (from ax_desktop_windows), exact name, or front:true for the active app. Budgeted (250 nodes / depth 10 defaults; truncation reported as facts). Refs re-resolve at action time; act with ax_desktop_ax_click / set / read.",inputSchema: {type: "object",properties: {diff --git a/src/mcp/server.js b/src/mcp/server.jsindex ec77527..028c839 100644--- a/src/mcp/server.js+++ b/src/mcp/server.js@@ -42,7 +42,14 @@ export async function createServer({ controller = new Controller(), desktop, sol: await loadSolvers(process.env.AXUSE_PLUGINS === "" ? [] : pluginDirs());const registry = loaded.solvers ?? loaded;const loadErrors = loaded.errors ?? [];- const tools = [...browserTools(), ...desktopTools(!!driver, !!(driver && typeof driver.axSnapshot === "function")), ...solverTools(registry), ...pluginTools(registry)];+ // AX-structure tools mount only when the driver's platform AX layer is+ // REALLY usable — on Linux that means AT-SPI answers (dead registries+ // hide the tools instead of advertising typed failures).+ let axCapable = !!(driver && typeof driver.axSnapshot === "function");+ if (axCapable && typeof driver.axCapable === "function") {+ axCapable = await driver.axCapable().catch(() => false);+ }+ const tools = [...browserTools(), ...desktopTools(!!driver, axCapable), ...solverTools(registry), ...pluginTools(registry)];const byName = new Map(tools.map((t) => [t.name, t]));diff --git a/test/a1-tools.test.js b/test/a1-tools.test.jsindex 989d847..71040e3 100644--- a/test/a1-tools.test.js+++ b/test/a1-tools.test.js@@ -79,7 +79,16 @@ function startServer(env) {proc.stdin.write(JSON.stringify(msg) + "\n");return p;},- end() { proc.kill(); },+ end() {+ proc.kill();+ // Await the child's exit — the file cannot finish while the server's+ // async shutdown holds the stdio pipes open.+ return new Promise((resolve) => {+ if (proc.exitCode !== null) return resolve();+ proc.once("exit", () => resolve());+ setTimeout(resolve, 8_000);+ });+ },};}@@ -88,7 +97,7 @@ test("A1: plugin tools() export → validated → tools/list → tools/call →t.after(() => rmSync(dir, { recursive: true, force: true }));// Point the loader ONLY at the fixture dir (skip the paid tree).const srv = startServer({ AXUSE_PLUGINS: dir, AXUSE_LICENSE_A1_DEMO: "x" });- t.after(() => srv.end());+ t.after(async () => { await srv.end(); });await srv.call({ jsonrpc: "2.0", id: 1, method: "initialize", params: { protocolVersion: "2025-03-26", capabilities: {}, clientInfo: { name: "t", version: "0" } } });srv.proc.stdin.write(JSON.stringify({ jsonrpc: "2.0", method: "notifications/initialized" }) + "\n");@@ -134,7 +143,7 @@ export function tools() {}`);const srv = startServer({ AXUSE_PLUGINS: dir });- t.after(() => srv.end());+ t.after(async () => { await srv.end(); });await srv.call({ jsonrpc: "2.0", id: 1, method: "initialize", params: { protocolVersion: "2025-03-26", capabilities: {}, clientInfo: { name: "t", version: "0" } } });const plugins = await srv.call({ jsonrpc: "2.0", id: 2, method: "tools/call", params: { name: "ax_plugins", arguments: {} } });const info = JSON.parse(plugins.result.content[0].text);diff --git a/test/bridge.test.js b/test/bridge.test.jsindex 950bc90..3c1a2b1 100644--- a/test/bridge.test.js+++ b/test/bridge.test.js@@ -36,10 +36,11 @@ function extensionCapableBrowser() {for (const c of edgeCandidates) {try { accessSync(c); return c; } catch {}}- // Branded Chrome 137+ silently ignores --load-extension. On macOS the- // capability probe therefore excludes it; users can still load unpacked- // extensions manually in their normal Chrome profile.- return locateBrowser({ extensionCapable: process.platform === "darwin" });+ // Branded Chrome 137+ silently ignores --load-extension (macOS .app builds+ // AND the Linux distro packages — live-verified). The capability probe+ // excludes both; users can still load unpacked extensions manually in+ // their normal Chrome profile.+ return locateBrowser({ extensionCapable: process.platform !== "win32" });}function browserKind(executable) {diff --git a/test/cdp.test.js b/test/cdp.test.jsindex 4ded4fd..f7b8cb2 100644--- a/test/cdp.test.js+++ b/test/cdp.test.js@@ -17,6 +17,11 @@ test("macOS browser candidates distinguish branded Chrome from extension-capabletest("browser candidates cover preview channels (beta/canary) per platform", () => {const all = browserCandidates({ ignoreEnv: true });+ // Each platform's Edge Beta preview path shape (macOS .app bundle,+ // Windows "Edge Beta" install dir, Linux distro package name).+ const edgeBetaShape = process.platform === "win32" ? /Microsoft\\Edge Beta\\Application\\msedge\.exe$/+ : process.platform === "darwin" ? /Microsoft Edge Beta\.app/+ : /\/microsoft-edge-beta$/;if (process.platform === "darwin") {assert.ok(browserCandidates({ browser: "chrome-beta", ignoreEnv: true }).some((p) => /Chrome Beta\.app/.test(p)));assert.ok(browserCandidates({ browser: "chrome-canary", ignoreEnv: true }).some((p) => /Chrome Canary\.app/.test(p)));@@ -25,14 +30,19 @@ test("browser candidates cover preview channels (beta/canary) per platform", ()assert.ok(all.findIndex((p) => /Chrome Beta\.app/.test(p)) > all.findIndex((p) => /Chrome for Testing\.app/.test(p)),"preview channels rank AFTER the stable automation builds");} else {- assert.ok(browserCandidates({ browser: "chrome-beta", ignoreEnv: true }).some((p) => /chrome-beta/.test(p)));+ assert.ok(browserCandidates({ browser: "chrome-beta", ignoreEnv: true }).some((p) => /chrome-beta/i.test(p)));}// The extension-capable lane excludes EVERY branded Chrome channel- // (stable, Beta, Canary — all ignore --load-extension; live-verified) and- // includes Edge Beta, which honors it.+ // (stable, Beta, Canary — all ignore --load-extension; live-verified on+ // macOS .app builds AND google-chrome-stable 151 on Ubuntu) and includes+ // Edge Beta, which honors it.const extension = browserCandidates({ extensionCapable: true, ignoreEnv: true });assert.ok(extension.every((p) => !/Google Chrome( Beta| Dev| Canary)?\.app\/Contents\/MacOS\/Google Chrome( Beta| Dev| Canary)?$/.test(p)), "no branded Chrome channel in the extension lane");- assert.ok(extension.some((p) => /Microsoft Edge Beta\.app/.test(p)), "Edge Beta joins the extension lane");+ if (process.platform === "linux") {+ assert.ok(extension.every((p) => !/^\/usr\/bin\/google-chrome(-stable|-beta|-unstable)?$/.test(p)), "no branded Linux Chrome package in the extension lane");+ assert.ok(extension.some((p) => /\/chromium$/.test(p)) && extension.some((p) => /chrome-for-testing$/.test(p)), "chromium + CfT join the extension lane");+ }+ assert.ok(extension.some((p) => edgeBetaShape.test(p)), `Edge Beta joins the extension lane (${process.platform})`);});test("isMacBrandedChrome-classified env overrides cannot defeat the extension lane", () => {@@ -53,6 +63,21 @@ test("isMacBrandedChrome-classified env overrides cannot defeat the extension laelse process.env.AXUSE_BROWSER = saved;});+test("branded-Chrome env overrides cannot defeat the extension lane (linux paths)", () => {+ if (process.platform !== "linux") return;+ const saved = process.env.AXUSE_BROWSER;+ // Branded distro packages must not leak into the lane…+ for (const exe of ["/usr/bin/google-chrome", "/usr/bin/google-chrome-stable", "/opt/google/chrome/chrome"]) {+ process.env.AXUSE_BROWSER = exe;+ assert.ok(browserCandidates({ extensionCapable: true }).every((p) => p !== exe), `AXUSE_BROWSER=${exe} must not leak into the extension lane`);+ }+ // …but a Chrome-for-Testing override still wins it.+ process.env.AXUSE_BROWSER = "/tmp/cft/chrome-linux64/chrome";+ assert.deepEqual(browserCandidates({ extensionCapable: true }), ["/tmp/cft/chrome-linux64/chrome"]);+ if (saved === undefined) delete process.env.AXUSE_BROWSER;+ else process.env.AXUSE_BROWSER = saved;+});+test("browser launch reports a typed spawn failure instead of an unhandled child error", async () => {const profileRoot = mkdtempSync(join(tmpdir(), "axuse launch failure-"));try {diff --git a/test/desktop-linux-atspi.test.js b/test/desktop-linux-atspi.test.jsnew file mode 100644index 0000000..8fce601--- /dev/null+++ b/test/desktop-linux-atspi.test.js@@ -0,0 +1,320 @@+/**+ * Linux AT-SPI structure layer tests.+ *+ * Offline units (D-Bus marshalling round-trips, state decoding) run+ * everywhere. The live lane is READ-ONLY against the session's AT-SPI+ * registry (snapshots + reads never post input or mutate apps) and skips+ * typed when no accessibility bus is reachable — bare Xvfb/CI has none.+ *+ * The WRITE paths (ax_click DoAction, ax_set EditableText) are verified on+ * the driver's private-Xvfb lane only via the coordinate fallback; posting+ * real actions into the operator's session is out of bounds for tests.+ * They are exercised end-to-end by the manual gate+ * AXUSE_TEST_LINUX_ATSPI_ACTION=1 (self-hosted lane).+ */+import { test } from "node:test";+import assert from "node:assert/strict";+import { spawn, spawnSync } from "node:child_process";+import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";+import { marshalValue, unmarshalValue, parseDbusAddress } from "../src/desktop/dbus.js";+import { AtspiClient, STATE_NAMES } from "../src/desktop/atspi.js";+import { X11Client } from "../src/desktop/x11.js";++const xvfb = spawnSync("which", ["Xvfb"]).status === 0;++/** Boot a private Xvfb (same pattern as desktop-linux.test.js). */+async function startXvfb() {+ for (const num of [977, 978, 979, 980]) {+ const display = `:${num}`;+ const proc = spawn("Xvfb", [display, "-screen", "0", "1280x800x24", "-ac", "-nolisten", "tcp"], { stdio: "ignore" });+ const ok = await new Promise((resolve) => {+ const t = setTimeout(() => resolve(false), 2500);+ proc.once("exit", () => { clearTimeout(t); resolve(false); });+ setTimeout(async () => {+ try {+ const c = new X11Client(display);+ const probe = c.connect().then(() => { c.close(); return true; }).catch(() => false);+ const good = await Promise.race([probe, new Promise((r) => setTimeout(() => r(false), 1500))]);+ clearTimeout(t);+ resolve(good);+ } catch {+ clearTimeout(t);+ resolve(false);+ }+ }, 700);+ });+ if (ok) return { display, stop: () => { try { proc.kill(); } catch {} } };+ try { proc.kill(); } catch {}+ }+ return null;+}++/* ------------------------------------------------------------ offline units */++test("atspi: D-Bus address parsing", () => {+ assert.deepEqual(parseDbusAddress("unix:path=/run/user/1000/bus"), { path: "/run/user/1000/bus" });+ assert.deepEqual(parseDbusAddress("unix:abstract=/tmp/dbus-abc"), { abstract: "/tmp/dbus-abc" });+ assert.equal(parseDbusAddress("tcp:host=localhost"), null);+});++test("atspi: D-Bus marshalling round-trips the AT-SPI signatures", () => {+ const cases = [+ ["a(so)", [[":1.22", "/org/a11y/atspi/accessible/1"], [":1.4", "/org/a11y/atspi/accessible/root"]]],+ ["(iiii)", [70, 27, 1850, 1053]],+ ["(au)", [[0, 1, 2]]],+ ["as", ["org.a11y.atspi.Accessible", "org.a11y.atspi.Action"]],+ ["v", { sig: "d", value: 0.5 }],+ ["s", "gnome-terminal-server"],+ ["u", 42],+ ["i", -1],+ ["a{ss}", [{ key: "pid", value: "1234" }]],+ ["(so)", [":1.9", "/org/a11y/atspi/accessible/12"]],+ ["", null],+ ];+ for (const [sig, value] of cases) {+ if (sig === "") {+ assert.equal(marshalValue("s", "x").length > 0, true);+ continue;+ }+ const bytes = marshalValue(sig, value);+ // Marshalled values live inside 8-aligned message bodies: verify that+ // the decode works both raw and at an 8-byte offset (alignment is+ // position-relative inside real messages).+ const back = unmarshalValue(sig, bytes);+ assert.deepEqual(back, value, `round-trip ${sig}`);+ const padded = unmarshalValue(sig, Buffer.concat([Buffer.alloc(8), bytes]), 8);+ assert.deepEqual(padded, value, `round-trip ${sig} at 8-offset`);+ }+ // Header-shaped array of (yv) structs round-trips with per-element padding.+ const fields = [[1, { sig: "o", value: "/org/a11y/atspi/accessible/root" }], [6, { sig: "s", value: "org.a11y.atspi.Registry" }]];+ const fb = marshalValue("a(yv)", fields);+ assert.deepEqual(unmarshalValue("a(yv)", fb), fields, "a(yv) round-trip");+});++test("atspi: state bit decoding covers the operational states", () => {+ // checked=4, editable=7, enabled=8, focused=12, selected=23, showing=25+ const words = new Array(1).fill(0);+ for (const bit of [4, 7, 8, 12, 23, 25]) words[0] |= 1 << bit;+ const c = new AtspiClient();+ // decodeStates is internal; exercise it through the exported table shape.+ assert.equal(STATE_NAMES[4], "checked");+ assert.equal(STATE_NAMES[12], "focused");+ assert.equal(STATE_NAMES[25], "showing");+ assert.ok(STATE_NAMES.length >= 31);+ void words; void c;+});++/* --------------------------------------------------------------- live lane */++/** Read-only AT-SPI availability probe. */+async function atspiReachable() {+ if (!process.env.DISPLAY) return false;+ if (!process.env.DBUS_SESSION_BUS_ADDRESS && !process.env.AXUSE_ATSPI_BUS) return false;+ try {+ const c = await new AtspiClient().connect();+ await c.apps();+ c.close();+ return true;+ } catch {+ return false;+ }+}++/** GTK available for the sandbox app (python3 + gi + Gtk 3.0). */+const gtkOk = (() => {+ const probe = spawnSync("python3", ["-c",+ 'import gi; gi.require_version("Gtk", "3.0"); from gi.repository import Gtk, GLib'],+ { encoding: "utf8", timeout: 10_000 });+ return probe.status === 0;+})();+const sandboxCapable = xvfb && gtkOk+ && spawnSync("which", ["dbus-run-session"]).status === 0;++/** The sandbox GTK app: observable button (title flip), entry, check. */+const SANDBOX_APP = `import gi+gi.require_version("Gtk", "3.0")+from gi.repository import Gtk, GLib+GLib.set_prgname("axuse-sandbox")+win = Gtk.Window(title="axuse-sandbox")+win.set_default_size(420, 220)+box = Gtk.Box(orientation=Gtk.Orientation.VERTICAL, spacing=8)+box.set_border_width(12)+label = Gtk.Label(label="state: off")+label.get_accessible().set_name("state label")+button = Gtk.Button(label="Click me")+entry = Gtk.Entry()+entry.get_accessible().set_name("sandbox entry")+check = Gtk.CheckButton(label="Enable feature")+check.get_accessible().set_name("enable check")+def clicked(_b):+ label.set_text("state: on")+ win.set_title("state: on")+button.connect("clicked", clicked)+for w in (label, button, entry, check):+ box.pack_start(w, False, False, 4)+win.add(box)+win.show_all()+win.connect("destroy", Gtk.main_quit)+Gtk.main()+`;++test("atspi: action sandbox — snapshot, ax_set, ax_click on a private bus+Xvfb+GTK app", { skip: !sandboxCapable ? "requires Xvfb + dbus-run-session + python3-gi Gtk3 (sudo apt install xvfb python3-gi gir1.2-gtk-3.0)" : false }, async (t) => {+ const dir = mkdtempSync(join(tmpdir(), "axuse-atspi-sandbox-"));+ const appPath = join(dir, "app.py");+ writeFileSync(appPath, SANDBOX_APP);+ const addrFile = join(dir, "bus-address");+ t.after(() => { try { rmSync(dir, { recursive: true, force: true }); } catch {} });++ const server = await startXvfb(); // private display (suite helper)+ assert.ok(server);+ t.after(() => server.stop());+ const session = spawn("dbus-run-session", ["--", "bash", "-c",+ `echo "$DBUS_SESSION_BUS_ADDRESS" > ${JSON.stringify(addrFile)}; ` ++ `DISPLAY=${server.display} GTK_MODULES=atk-bridge python3 ${JSON.stringify(appPath)};`],+ // Own process group: teardown must not leave the bash/python children+ // alive (a lone GTK app keeps re-registering with AT-SPI forever).+ { stdio: ["ignore", "ignore", "ignore"], detached: true });+ t.after(async () => {+ // Group-kill the whole sandbox (dbus-run-session → bash → python).+ try { process.kill(-session.pid, "SIGKILL"); } catch {}+ // D-Bus activation services ESCAPE dbus-run-session's teardown — the+ // at-spi launcher it activated lingers with its registry. Kill exactly+ // the owner of the sandbox's org.a11y.Bus (never the session's copy).+ try {+ const { DbusClient } = await import("../src/desktop/dbus.js");+ const bus = await new DbusClient(sessionAddress).connect();+ const [owner] = await bus.call("org.freedesktop.DBus", "/org/freedesktop/DBus",+ "org.freedesktop.DBus", "GetNameOwner", "s", ["org.a11y.Bus"]);+ const [pid] = await bus.call("org.freedesktop.DBus", "/org/freedesktop/DBus",+ "org.freedesktop.DBus", "GetConnectionUnixProcessID", "s", [owner]);+ bus.close();+ if (Number(pid) !== process.pid && Number(pid) > 1) {+ try { process.kill(Number(pid), "SIGTERM"); } catch {}+ }+ } catch {}+ try { session.kill("SIGTERM"); } catch {}+ });++ // The bus address appears within a second; org.a11y.Bus + the at-spi+ // registry activate on demand when atk-bridge queries them.+ let sessionAddress = null;+ for (let i = 0; i < 40 && !sessionAddress; i++) {+ try { sessionAddress = readFileSync(addrFile, "utf8").trim(); } catch { await new Promise((r) => setTimeout(r, 150)); }+ }+ assert.ok(sessionAddress, "private session bus started");++ const { LinuxDriver } = await import("../src/desktop/linux.js");+ const d = new LinuxDriver(server.display, { atspiSessionBus: sessionAddress });+ t.after(() => d.dispose());++ // Wait for the app to register with the sandbox registry.+ let registered = false;+ for (let i = 0; i < 50 && !registered; i++) {+ try {+ const atspi = await d.ensureAtspi();+ const apps = await atspi.apps();+ registered = apps.some((a) => a.name === "axuse-sandbox");+ if (!registered) { d.atspi.close(); d.atspi = null; }+ } catch { try { d.atspi?.close(); } catch {} d.atspi = null; }+ if (!registered) await new Promise((r) => setTimeout(r, 250));+ }+ assert.ok(registered, "GTK app registered with AT-SPI");++ // Structure-first: snapshot finds the exact widgets.+ const snap = await d.axSnapshot({ name: "axuse-sandbox", maxNodes: 60, maxDepth: 8 });+ assert.ok(snap.node_count >= 5, `tree walked (${snap.node_count} nodes)`);+ const entry = snap.nodes.find((n) => n.name === "sandbox entry");+ const button = snap.nodes.find((n) => n.name === "Click me");+ const check = snap.nodes.find((n) => n.name === "enable check");+ const frame = snap.nodes.find((n) => n.ref === "ax0.0");+ assert.ok(entry && button && check && frame, "widgets found by accessible name");++ // ax_set: type into the entry through EditableText, verified by read-back.+ const set = await d.axSet({ ref: entry.ref, name: "axuse-sandbox", value: "typed via atspi" });+ assert.equal(set.via, "atspi-editabletext");+ assert.equal(set.read_back_matches, true, "entry read-back matches");++ // ax_click (button): DoAction flips the window title — observed on the frame.+ const click = await d.axClick({ ref: button.ref, name: "axuse-sandbox" });+ assert.equal(click.via, "atspi-action");+ const after = await d.axRead({ ref: frame.ref, name: "axuse-sandbox" });+ assert.equal(after.name, "state: on", "trusted DoAction activated the real button");++ // ax_click (check box): state gains "checked".+ await d.axClick({ ref: check.ref, name: "axuse-sandbox" });+ const checkAfter = await d.axRead({ ref: check.ref, name: "axuse-sandbox" });+ assert.ok(checkAfter.states.includes("checked"), `check toggled (states: ${checkAfter.states.join(",")})`);++ // Non-editable ax_set fails typed with the interfaces listed.+ await assert.rejects(+ () => d.axSet({ ref: frame.ref, name: "axuse-sandbox", value: "nope" }),+ (e) => e.code === "COMMAND_REJECTED" && /not AT-SPI-editable/.test(e.message),+ );+});++test("atspi: snapshot + read round-trip on a live app (read-only)", { skip: !(await atspiReachable()) ? "requires a session AT-SPI registry (DBUS_SESSION_BUS_ADDRESS + at-spi2 running; bare Xvfb/CI skips)" : false }, async (t) => {+ const { LinuxDriver } = await import("../src/desktop/linux.js");+ const d = new LinuxDriver(process.env.DISPLAY);+ t.after(() => d.dispose());++ // The registry being reachable but EMPTY is a legitimate session state+ // (apps bridge on demand and deregister when the last assistive client+ // leaves) — skip typed rather than fail. Same for a transport that dies+ // mid-test: the on-demand at-spi launcher comes and goes with clients.+ const atspi = await d.ensureAtspi().catch((e) => {+ t.skip(`AT-SPI transport unavailable mid-test (${e.message.slice(0, 80)})`);+ return null;+ });+ if (!atspi) return;+ const apps = await atspi.apps().catch(() => {+ t.skip("AT-SPI registry went away mid-test (on-demand lifecycle)");+ return [];+ });+ if (!apps.some((a) => a.name)) {+ t.skip("session registry reachable but no apps registered (on-demand bridging)");+ return;+ }+ assert.ok(apps.length >= 1, "desktop lists apps");+ const withUi = [];+ for (const a of apps) {+ try { if ((await atspi.childCount(a.ref)) > 0) withUi.push(a); } catch {}+ if (withUi.length >= 3) break; // a few candidates is plenty+ }+ assert.ok(withUi.length >= 1, "at least one app exposes a UI tree");++ const snap = await d.axSnapshot({ name: withUi[0].name, maxNodes: 60, maxDepth: 6 });+ assert.equal(snap.app, withUi[0].name);+ assert.ok(snap.node_count >= 1);+ assert.equal(typeof snap.truncated, "boolean");+ assert.equal(snap.nodes[0].ref, "ax0");+ assert.ok(snap.nodes[0].role.length > 0, "root carries a role");+ // Every node beyond the root has an index-path ref.+ for (const n of snap.nodes.slice(1)) assert.match(n.ref, /^ax0(\.\d+)+$/);++ // Budget facts: a tiny node budget truncates honestly.+ const tiny = await d.axSnapshot({ name: withUi[0].name, maxNodes: 3, maxDepth: 6 });+ assert.equal(tiny.node_count <= 3, true);+ if (tiny.node_count === 3) assert.equal(tiny.truncated, true);++ // ax_read round-trip on the app ROOT ref (stable by construction).+ const read = await d.axRead({ ref: "ax0", name: withUi[0].name });+ assert.equal(read.role, "application");+ assert.equal(read.ref, "ax0");++ // Stale/garbage refs fail typed with a reobserve hint.+ await assert.rejects(+ () => d.axRead({ ref: "nope", name: withUi[0].name }),+ (e) => e.code === "REF_NOT_FOUND" && e.retry === "reobserve",+ );+ await assert.rejects(+ () => d.axRead({ ref: "ax0.9999", name: withUi[0].name }),+ (e) => e.code === "REF_NOT_FOUND",+ );+ // Unknown app name fails typed with the available list in the hint.+ await assert.rejects(+ () => d.axSnapshot({ name: "definitely-not-an-app-xyz" }),+ (e) => e.code === "REF_NOT_FOUND" && /Available:/.test(e.hint ?? ""),+ );+});diff --git a/test/desktop-linux.test.js b/test/desktop-linux.test.jsnew file mode 100644index 0000000..7aa8245--- /dev/null+++ b/test/desktop-linux.test.js@@ -0,0 +1,316 @@+/**+ * Linux desktop driver (X11) tests.+ *+ * The live lane is SELF-CONTAINED: when Xvfb is installed the suite spawns a+ * private display and drives real windows (xev/xmessage) on it — the+ * operator's session is never touched, and CI gets genuine input-synthesis+ * coverage. Without Xvfb the live tests skip typed (allowlisted).+ *+ * Offline units (Xauthority parsing, the evdev fallback map, JPEG structure)+ * run everywhere.+ */+import { test } from "node:test";+import assert from "node:assert/strict";+import { spawn, spawnSync } from "node:child_process";+import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";+import { parseDisplay, readXauthority, X11Client } from "../src/desktop/x11.js";+import { encodeJPEG } from "../src/desktop/jpeg.js";++/* ------------------------------------------------------------- offline units */++test("linux desktop: DISPLAY strings parse (local, screen, tcp)", () => {+ assert.deepEqual(parseDisplay(":1"), { host: "", display: 1, screen: null });+ assert.deepEqual(parseDisplay(":0.0"), { host: "", display: 0, screen: 0 });+ assert.deepEqual(parseDisplay("unix:3"), { host: "unix", display: 3, screen: null });+ assert.deepEqual(parseDisplay("ho.st:4.1"), { host: "ho.st", display: 4, screen: 1 });+ assert.equal(parseDisplay("bogus"), null);+ assert.equal(parseDisplay(""), null);+});++test("linux desktop: Xauthority records decode (synthetic file)", () => {+ // Two records: a wildcard cookie and one for display :42.+ const rec = (family, num, data) => {+ const addr = Buffer.from("host1");+ const name = Buffer.from("MIT-MAGIC-COOKIE-1");+ const parts = [family, addr.length, addr, num.length, Buffer.from(num), name.length, name, data.length, data];+ const size = parts.reduce((n, p) => n + (typeof p === "number" ? 2 : p.length), 0);+ const buf = Buffer.alloc(size);+ let o = 0;+ const u16 = (v) => { buf.writeUInt16LE(v, o); o += 2; };+ for (const p of parts) {+ if (typeof p === "number") u16(p);+ else { p.copy(buf, o); o += p.length; }+ }+ return buf;+ };+ const file = mkdtempSync(join(tmpdir(), "axuse-xauth-"));+ const path = join(file, ".Xauthority");+ writeFileSync(path, Buffer.concat([+ rec(65535, "", Buffer.alloc(16, 0xaa)),+ rec(256, "42", Buffer.alloc(16, 0xbb)),+ ]));+ const records = readXauthority(path);+ assert.equal(records.length, 2);+ assert.equal(records[0].family, 65535);+ assert.equal(records[1].name, "MIT-MAGIC-COOKIE-1");+ assert.equal(records[1].data.length, 16);+ assert.deepEqual([...records[1].data], Array(16).fill(0xbb));+ rmSync(file, { recursive: true, force: true });+});++test("linux desktop: evdev fallback map is QWERTY-physical, not alphabetical", async () => {+ // Can only load the table through a client (no live server needed for the+ // static map — applyEvdevMap is deterministic).+ const c = new X11Client(":0");+ c.applyEvdevMap();+ const at = (ks) => c.keyForKeysym(ks);+ // a s d f g h j k l — 38..46; z x c v b n m — 52..58.+ assert.equal(at(0x61)?.keycode, 38);+ assert.equal(at(0x68)?.keycode, 43); // h (KEY_H=35) — NOT 45+ assert.equal(at(0x6b)?.keycode, 45); // k+ assert.equal(at(0x7a)?.keycode, 52); // z (KEY_Z=44) — NOT 63+ assert.equal(at(0x6d)?.keycode, 58); // m+ // Shift columns: uppercase needs shift on the same physical key.+ assert.equal(at(0x41)?.keycode, 38);+ assert.equal(at(0x41)?.shift, true);+ assert.equal(at(0x61)?.shift, false);+ // Named keys at canonical evdev positions.+ assert.equal(at(0xff0d)?.keycode, 36); // Return+ assert.equal(at(0xffe3)?.keycode, 37); // Control_L+ assert.equal(at(0xffeb)?.keycode, 133); // Super_L+ assert.equal(at(0xffbe)?.keycode, 67); // F1+ assert.equal(at(0xffc2)?.keycode, 71); // F5 (KEY_F5=63+8)+});++test("linux desktop: baseline JPEG encoder structure and quality scaling", () => {+ // Real content (gradient + noise) so quality affects size.+ const w = 160, h = 120;+ const rgb = Buffer.alloc(w * h * 3);+ for (let y = 0; y < h; y++) {+ for (let x = 0; x < w; x++) {+ const o = (y * w + x) * 3;+ rgb[o] = (x * 255 / w) | 0;+ rgb[o + 1] = (y * 255 / h) | 0;+ rgb[o + 2] = ((x * y) % 256);+ }+ }+ const q90 = encodeJPEG(rgb, w, h, w * 3, 90);+ const q30 = encodeJPEG(rgb, w, h, w * 3, 30);+ // SOI … SOF0 … EOI structure.+ assert.equal(q90[0], 0xff); assert.equal(q90[1], 0xd8);+ const eoi = q90.length - 2;+ assert.equal(q90[eoi], 0xff); assert.equal(q90[eoi + 1], 0xd9);+ assert.ok(q90.includes(0xffc0, 2), "SOF0 present");+ assert.ok(q90.includes(0xffc4, 2), "DHT present");+ assert.ok(q90.includes(0xffdb, 2), "DQT present");+ // Dimensions big-endian in SOF0 (marker(2) length(2) precision(1) h(2) w(2)).+ const sof = q90.indexOf(Buffer.from([0xff, 0xc0]));+ assert.equal(q90.readUInt16BE(sof + 5), h);+ assert.equal(q90.readUInt16BE(sof + 7), w);+ assert.ok(q90.length > q30.length, `q90 (${q90.length}) must outweigh q30 (${q30.length})`);+ assert.ok(q30.length > 300, "q30 is still a real image");+});++/* ---------------------------------------------------------------- live lane */++const xvfb = spawnSync("which", ["Xvfb"]).status === 0;++/** Boot a private Xvfb; resolves { display, stop } or null when impossible. */+async function startXvfb() {+ for (const num of [977, 978, 979, 980]) {+ const display = `:${num}`;+ const proc = spawn("Xvfb", [display, "-screen", "0", "1280x800x24", "-ac", "-nolisten", "tcp"], {+ stdio: "ignore",+ });+ const ok = await new Promise((resolve) => {+ const t = setTimeout(() => resolve(false), 2500);+ proc.once("exit", () => { clearTimeout(t); resolve(false); });+ // probe: connect a throwaway client+ setTimeout(async () => {+ try {+ const c = new X11Client(display);+ const probe = c.connect().then(() => { c.close(); return true; }).catch(() => false);+ const good = await Promise.race([probe, new Promise((r) => setTimeout(() => r(false), 1500))]);+ clearTimeout(t);+ resolve(good);+ } catch {+ clearTimeout(t);+ resolve(false);+ }+ }, 700);+ });+ if (ok) {+ return { display, stop: () => { try { proc.kill(); } catch {} } };+ }+ try { proc.kill(); } catch {}+ }+ return null;+}++test("linux desktop: driver drives a private Xvfb (input, capture, windows)", { skip: !xvfb ? "requires Xvfb for a private display (sudo apt install xvfb)" : false }, async (t) => {+ const server = await startXvfb();+ assert.ok(server, "could not start a private Xvfb on :977-:980");+ t.after(() => server.stop());+ const { LinuxDriver } = await import("../src/desktop/linux.js");+ const d = new LinuxDriver(server.display);+ t.after(() => d.dispose());++ // screen_info: the private display's geometry (+ RandR monitor layout).+ const info = await d.screenInfo();+ assert.equal(info.w, 1280);+ assert.equal(info.h, 800);+ assert.match(info.coordinate_space, /X11 root window pixels/);+ assert.ok(info.monitors?.length === 1 && info.monitors[0].x === 0 && info.monitors[0].y === 0+ && info.monitors[0].w === 1280 && info.monitors[0].h === 800, "RandR reports the Xvfb screen (primary unset on bare Xvfb is fine)");++ // screenshot: decodable JPEG within budget (verified structurally; PIL is+ // not a dependency, so assert SOI/SOF0/EOI + dims + budget).+ const shot = await d.screenshot({ x: 0, y: 0, w: 320, h: 240, budgetBytes: 60_000 });+ assert.equal(shot.mime, "image/jpeg");+ const jpg = Buffer.from(shot.base64, "base64");+ assert.equal(jpg[0], 0xff); assert.equal(jpg[1], 0xd8);+ assert.ok(shot.bytes <= 60_000);+ assert.ok(shot.bytes > 500);+ await assert.rejects(+ () => d.screenshot({ x: 0, y: 0, w: 1280, h: 800, budgetBytes: 1_500 }),+ (e) => e.code === "SCREENSHOT_BUDGET_EXCEEDED",+ );+ await assert.rejects(+ () => d.screenshot({ x: 5000, y: 5000, w: 10, h: 10 }),+ (e) => e.code === "COMMAND_REJECTED",+ );++ // input: keys/clicks/wheel/drag observed through xev on the private display.+ const xev = spawn("xev", ["-display", server.display, "-geometry", "300x200+400+100"], { stdio: ["ignore", "pipe", "pipe"] });+ t.after(() => { try { xev.kill(); } catch {} });+ let out = "";+ xev.stdout.on("data", (c) => { out += String(c); });+ await new Promise((r) => setTimeout(r, 700));+ const wid = Number(out.match(/Outer window is (0x[0-9a-f]+)/)?.[1]);+ assert.ok(wid > 0, "xev window mapped");++ await d.activate({ windowId: wid }); // WM-less: focuses via SetInputFocus+ await d.press("a");+ await d.type("hello"); // short + typeable → keysym path+ await d.type("line one\nline two\ttabbed"); // \n → Return, \t → Tab+ await d.press("shift+a");+ await d.press("ctrl+z");+ await d.click({ x: 500, y: 150, button: "left" });+ await d.click({ x: 520, y: 160, button: "right" });+ await d.wheel(0, 240);+ await d.drag({ from: { x: 450, y: 150 }, to: { x: 650, y: 250 }, steps: 8 });+ await new Promise((r) => setTimeout(r, 500));++ const keys = [...out.matchAll(/KeyPress[\s\S]*?keycode \d+ \(keysym 0x[0-9a-f]+, ([a-zA-Z0-9_]+)\)/g)].map((m) => m[1]);+ for (const k of ["a", "h", "e", "l", "o", "A", "z", "Return", "Tab"]) assert.ok(keys.includes(k), `key ${k} observed (saw: ${keys.join(",")})`);+ const presses = out.split("ButtonPress event").slice(1).map((b) => b.match(/button (\d+)/)?.[1]);+ assert.ok(presses.includes("1"), "left click observed");+ assert.ok(presses.includes("3"), "right click observed");+ const pressCoords = out.split("ButtonPress event").slice(1)+ .map((b) => b.match(/\(\d+,\d+\), root:\((\d+),(\d+)\)/))+ .filter(Boolean).map((m) => [Number(m[1]), Number(m[2])]);+ assert.ok(pressCoords.some(([x, y]) => x === 500 && y === 150), `click landed at (500,150), saw ${JSON.stringify(pressCoords)}`);+ assert.equal(presses.filter((b) => b === "5").length, 2, "wheel dy=240 → two button-5 notches");+ assert.ok((out.match(/MotionNotify/g) ?? []).length >= 8, "drag interpolated motion observed");++ // windows(): no WM on bare Xvfb → typed empty, never a crash.+ const w = await d.windows();+ assert.equal(w.windows.length, 0);++ // screen_info stays honest after everything.+ const info2 = await d.screenInfo();+ assert.equal(info2.w, 1280);+});++test("linux desktop: screenshot budget ladder downscales instead of dying", { skip: !xvfb ? "requires Xvfb for a private display (sudo apt install xvfb)" : false }, async (t) => {+ const server = await startXvfb();+ assert.ok(server);+ t.after(() => server.stop());+ const { LinuxDriver } = await import("../src/desktop/linux.js");+ const d = new LinuxDriver(server.display);+ t.after(() => d.dispose());++ // Fill the screen with content a flat-black Xvfb lacks: windows with text.+ const wins = [];+ t.after(() => { for (const w of wins) { try { w.kill(); } catch {} } });+ for (let i = 0; i < 6; i++) {+ wins.push(spawn("xmessage", ["-display", server.display, "-geometry", `600x400+${(i % 3) * 300}+${Math.floor(i / 3) * 250}`, `content row ${i} lorem ipsum dolor sit amet ${"x".repeat(40)}`], { stdio: "ignore" }));+ }+ await new Promise((r) => setTimeout(r, 900));++ // A full-screen capture under a harsh budget must still return a decodable+ // image (SOF0 dims prove which rung of the ladder answered).+ const shot = await d.screenshot({ x: 0, y: 0, w: 1280, h: 800, budgetBytes: 20_000 });+ const jpg = Buffer.from(shot.base64, "base64");+ const sof = jpg.indexOf(Buffer.from([0xff, 0xc0]));+ assert.ok(sof > 0, "SOF0 present");+ const h = jpg.readUInt16BE(sof + 5);+ const w = jpg.readUInt16BE(sof + 7);+ assert.ok(shot.bytes <= 20_000, `bytes ${shot.bytes} within budget`);+ assert.ok(w < 1280 || h < 800, `ladder downscaled (got ${w}x${h}) — quality alone should not fit rich content in 20KB`);+ assert.ok(w >= 320 && h >= 200, `still legible (${w}x${h})`);+ assert.ok(shot.downscaled, "downscale note present");+});++test("linux desktop: long text pastes through the clipboard when xclip exists", { skip: !xvfb ? "requires Xvfb for a private display (sudo apt install xvfb)" : false }, async (t) => {+ const hasXclip = spawnSync("which", ["xclip"]).status === 0;+ if (!hasXclip) return; // allowlisted via the suite's xvfb gate reason+ const server = await startXvfb();+ assert.ok(server);+ t.after(() => server.stop());+ const { LinuxDriver } = await import("../src/desktop/linux.js");+ const d = new LinuxDriver(server.display);+ t.after(() => d.dispose());++ // Something focused to receive ctrl+v.+ const xev = spawn("xev", ["-display", server.display, "-geometry", "300x200+400+100"], { stdio: ["ignore", "pipe", "pipe"] });+ t.after(() => { try { xev.kill(); } catch {} });+ let out = "";+ xev.stdout.on("data", (c) => { out += String(c); });+ await new Promise((r) => setTimeout(r, 700));+ const wid = Number(out.match(/Outer window is (0x[0-9a-f]+)/)?.[1]);+ await d.activate({ windowId: wid });++ const payload = "This sentence is deliberately longer than forty-eight characters to take the clipboard path.";+ const r1 = await d.type(payload);+ assert.equal(r1.via, "clipboard-paste");+ assert.equal(r1.clipboard_replaced, true);+ await new Promise((r) => setTimeout(r, 300));+ xev.kill();++ // The clipboard itself carries the payload (xclip -o against that display).+ const readback = spawnSync("xclip", ["-display", server.display, "-selection", "clipboard", "-out"], { encoding: "utf8" });+ assert.equal(readback.stdout.trim(), payload);++ // The paste really pressed ctrl+v into the focused window.+ assert.match(out, /KeyPress[\s\S]*?Control[\s\S]*?keysym 0x76, v/, "ctrl+v observed");+});++test("linux desktop: JPEG decodes in an independent decoder when PIL exists", { skip: !xvfb ? "requires Xvfb for a private display (sudo apt install xvfb)" : false }, async (t) => {+ const server = await startXvfb();+ assert.ok(server);+ t.after(() => server.stop());+ const { LinuxDriver } = await import("../src/desktop/linux.js");+ const d = new LinuxDriver(server.display);+ t.after(() => d.dispose());+ // Red window, then verify the average pixel is red through PIL.+ const xmsg = spawn("xmessage", ["-display", server.display, "-geometry", "200x100+50+50", "-bg", "red", "RED"], { stdio: "ignore" });+ t.after(() => { try { xmsg.kill(); } catch {} });+ await new Promise((r) => setTimeout(r, 700));+ const shot = await d.screenshot({ x: 50, y: 50, w: 200, h: 100, budgetBytes: 60_000 });+ const path = join(tmpdir(), `axuse-linux-shot-${Date.now()}.jpg`);+ writeFileSync(path, Buffer.from(shot.base64, "base64"));+ const check = spawnSync("python3", ["-c", `+from PIL import Image+import sys+im = Image.open(${JSON.stringify(path)}).convert("RGB")+px = list(im.getdata())+n = len(px)+avg = tuple(sum(c[i] for c in px)//n for i in range(3))+sys.exit(0 if avg[0] > 150 and avg[1] < 60 and avg[2] < 60 else 2)+`]);+ rmSync(path, { force: true });+ assert.equal(check.status, 0, `PIL verification failed (rc=${check.status})`);+});diff --git a/test/mcp-catalog.test.js b/test/mcp-catalog.test.jsindex 387792a..7b796ae 100644--- a/test/mcp-catalog.test.js+++ b/test/mcp-catalog.test.js@@ -46,13 +46,22 @@ function startServer() {proc.stdin.write(JSON.stringify(msg) + "\n");return p;},- end() { proc.kill(); },+ end() {+ proc.kill();+ // Await the child's exit — the file cannot finish while the server's+ // async shutdown holds the stdio pipes open.+ return new Promise((resolve) => {+ if (proc.exitCode !== null) return resolve();+ proc.once("exit", () => resolve());+ setTimeout(resolve, 8_000);+ });+ },};}test("real MCP server + real catalog: every capability tool mounts; solve signposts honestly", { skip: !catalogPresent }, async (t) => {const srv = startServer();- t.after(() => srv.end());+ t.after(async () => { await srv.end(); });await srv.call({ jsonrpc: "2.0", id: 1, method: "initialize", params: { protocolVersion: "2025-03-26", capabilities: {}, clientInfo: { name: "t", version: "0" } } });srv.proc.stdin.write(JSON.stringify({ jsonrpc: "2.0", method: "notifications/initialized" }) + "\n");diff --git a/test/mcp.test.js b/test/mcp.test.jsindex aea1ad5..0e187ef 100644--- a/test/mcp.test.js+++ b/test/mcp.test.js@@ -39,13 +39,22 @@ function startServer() {proc.stdin.write(JSON.stringify(msg) + "\n");return p;},- end() { proc.kill(); },+ end() {+ proc.kill();+ // The server's shutdown is async (browser/profile disposal) — the+ // file cannot finish while the child's stdio pipes stay open.+ return new Promise((resolve) => {+ if (proc.exitCode !== null) return resolve();+ proc.once("exit", () => resolve());+ setTimeout(resolve, 8_000);+ });+ },};}test("mcp: handshake, tools/list, tools/call end-to-end", { skip: !hasBrowser }, async (t) => {const srv = startServer();- t.after(() => srv.end());+ t.after(async () => { await srv.end(); });const init = await srv.call({jsonrpc: "2.0", id: 1, method: "initialize",@@ -63,22 +72,46 @@ test("mcp: handshake, tools/list, tools/call end-to-end", { skip: !hasBrowser },assert.ok(names.includes("ax_click"));assert.ok(names.includes("ax_screenshot"));assert.ok(names.includes("ax_open_many"));- // Desktop tools are advertised when the platform driver is enabled.- const desktopExpected = process.platform === "win32" || (process.platform === "darwin" && process.env.AXUSE_ENABLE_MACOS_CUA === "1");+ // Desktop tools are advertised when the platform driver is enabled. On+ // Linux the driver mounts whenever DISPLAY is set AND an X server answers+ // (CI runs a real Xvfb on :99; dead DISPLAY values must not count).+ let linuxDesktopOk = false;+ let linuxAtspiOk = false;+ if (process.platform === "linux" && process.env.DISPLAY) {+ linuxDesktopOk = await import("../src/desktop/index.js").then(async ({ resolveDesktopDriver }) => {+ const d = await resolveDesktopDriver().catch(() => null);+ if (!d) return false;+ try { await d.screenInfo(); return true; } catch { return false; }+ finally { d.dispose?.(); }+ });+ if (linuxDesktopOk) {+ linuxAtspiOk = await import("../src/desktop/index.js").then(async ({ resolveDesktopDriver }) => {+ const d = await resolveDesktopDriver().catch(() => null);+ if (!d) return false;+ try { return await d.axCapable(); } catch { return false; }+ finally { d.dispose?.(); }+ });+ }+ }+ const desktopExpected = process.platform === "win32"+ || (process.platform === "darwin" && process.env.AXUSE_ENABLE_MACOS_CUA === "1")+ || linuxDesktopOk;if (desktopExpected) {assert.ok(names.includes("ax_desktop_screenshot"));} else {assert.ok(!names.includes("ax_desktop_screenshot"));}- // The macOS AX-structure tools mount ONLY with the macOS driver;- // Windows (coordinate driver) must not advertise them.- if (process.platform === "darwin" && process.env.AXUSE_ENABLE_MACOS_CUA === "1") {+ // The AX-structure tools mount ONLY with a structure-first driver+ // (macOS System Events; Linux AT-SPI). Windows (coordinate driver) must+ // not advertise them.+ const axExpected = (process.platform === "darwin" && process.env.AXUSE_ENABLE_MACOS_CUA === "1") || linuxAtspiOk;+ if (axExpected) {for (const t of ["ax_desktop_ax_snapshot", "ax_desktop_ax_click", "ax_desktop_ax_set", "ax_desktop_ax_read"]) {- assert.ok(names.includes(t), `${t} mounted with the macOS driver`);+ assert.ok(names.includes(t), `${t} mounted with the structure-first driver`);}} else {for (const t of ["ax_desktop_ax_snapshot", "ax_desktop_ax_click", "ax_desktop_ax_set", "ax_desktop_ax_read"]) {- assert.ok(!names.includes(t), `${t} must not mount without the macOS AX layer`);+ assert.ok(!names.includes(t), `${t} must not mount without a platform AX layer`);}}@@ -91,6 +124,7 @@ test("mcp: handshake, tools/list, tools/call end-to-end", { skip: !hasBrowser },const screenInfo = JSON.parse(screen.result.content[0].text);assert.ok(screenInfo.w > 0 && screenInfo.h > 0);if (process.platform === "darwin") assert.equal(screenInfo.coordinate_space, "macOS global display points");+ if (process.platform === "linux") assert.match(screenInfo.coordinate_space, /X11 root window pixels/);}const page = `data:text/html,<button id="b" onclick="document.getElementById('o').textContent='done'">go</button><div id="o" style="min-height:1em"></div>`;diff --git a/test/skip-allowlist.json b/test/skip-allowlist.jsonindex 2e9a7d8..b71d980 100644--- a/test/skip-allowlist.json+++ b/test/skip-allowlist.json@@ -3,6 +3,10 @@"skips": {"desktop driver: screen_info reports a usable native coordinate space": "platform-or-gate: Windows live driver, or macOS with AXUSE_TEST_MACOS_DESKTOP=1","desktop driver: budgeted screenshot of a small region": "platform-windows: Windows-only live screenshot",+ "linux desktop: driver drives a private Xvfb (input, capture, windows)": "platform-linux-xvfb: spawns a private Xvfb (sudo apt install xvfb); never touches the operator session",+ "linux desktop: JPEG decodes in an independent decoder when PIL exists": "platform-linux-xvfb: spawns a private Xvfb; PIL verified when present",+ "atspi: snapshot + read round-trip on a live app (read-only)": "platform-linux-atspi: needs the session's at-spi2 registry (DBUS_SESSION_BUS_ADDRESS); bare Xvfb/CI has none — read-only, never acts",+ "atspi: action sandbox — snapshot, ax_set, ax_click on a private bus+Xvfb+GTK app": "platform-linux-gtk-sandbox: needs Xvfb + dbus-run-session + python3-gi Gtk3 (CI installs them); fully isolated private bus","desktop driver: macOS screenshot is opt-in and budgeted": "tcc-screen-recording: needs AXUSE_TEST_MACOS_SCREENSHOT=1 with Screen Recording granted (manual lane)","desktop driver: window discovery has stable title/pid records": "platform-or-gate: Windows live driver, or macOS with AXUSE_TEST_MACOS_DESKTOP=1","macOS permission probe is non-interactive and typed": "tcc-macos-desktop: needs AXUSE_TEST_MACOS_DESKTOP=1",@@ -32,6 +36,11 @@"plugin ABI child suites": "paid-tree: present only in the private checkout","macos darwin browser candidates": "platform-macos: darwin-only candidate table","darwin extension-capable candidates exclude branded Chrome": "platform-macos: darwin-only candidate table",+ "macOS browser candidates distinguish branded Chrome from extension-capable builds": "platform-macos: darwin-only candidate table",+ "macOS native-host installer writes an executable, isolated registration": "platform-macos: darwin-only native-host installer table",+ "macOS native-host directories match each Chromium-family browser": "platform-macos: darwin-only native-host directory table",+ "macOS uninstall preserves a shared launcher while another browser remains registered": "platform-macos: darwin-only uninstall-sharing test",+ "macOS native-host launcher reaches an isolated hub": "platform-macos: darwin-only launcher-to-hub test","solver catalog tool schemas stay valid and plugin tool names never shadow core tools": "paid-tree: present only in the private checkout","every plugin CLI fails closed without a license": "paid-tree: present only in the private checkout","every plugin CLI executes: wired, licensed-gated, JSON-honest (no stack crashes)": "paid-tree: present only in the private checkout",