Voice-to-text dictation app with local (Nvidia Parakeet/Whisper) and cloud models (BYOK). Privacy-first and available cross-platform.
JavaScript aianthropiccross-platformgeminigroqlinuxmacosnvidiaopen-sourceopenaiparakeetspeech-to-texttranscribewhisperwindows
fix(assistant): handle dormant macOS selection capture safely (#1952)
Gabriel Steinea122650006f
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 4bcf61fd9..8144e8c05 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0## [Unreleased]+### Fixed++- **Assistant selection capture handles dormant macOS accessibility trees.** When accessibility exposes only a browser or Electron window, selection capture now tries a synthetic copy instead of assuming nothing is selected. The caret probe also waits long enough for its native retries to finish. Automatic answer delivery still requires a verified writable field at capture and delivery time: an unknown target keeps the answer in the panel and, with Auto-Paste enabled, copies it for manual paste. Dormant inputs still require manual paste because an empty copy cannot distinguish them from an integrated terminal, a page without an input, or some live selections. (#1952)+## [1.10.2] - 2026-09-15A hotfix for 1.10.1. Generate AI Summary with your own API key or an enterprise provider no longer times out on a real transcript, a recording with nothing substantive in it gets a one-line summary instead of a blank page and an error, and OpenRouter models can read screenshots again.diff --git a/CLAUDE.md b/CLAUDE.mdindex fee16f8a9..3701623df 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -673,7 +673,7 @@ Detects meetings via three independent sources, orchestrated by `MeetingDetectio### 17. Voice Assistant Hotkey-A dedicated global hotkey that starts a dictation whose transcript is sent straight to the voice assistant as a command — no wake word ("Hey [AgentName]") needed — and that always bypasses the cleanup model. Standalone commands never type at an unverified cursor: with auto-paste enabled and a writable caret verified at capture time (an opaque `caret` session in `selectionManager.js`, revalidated before pasting; terminals and fields with a live selection excluded — the native probes check the focused element's own selection state, closing the clipboard-capture blind spots), the completed answer pastes at that caret and the pill returns to idle. Otherwise the answer streams into a floating assistant panel attached to the dictation pill (there is no separate assistant window) and, with auto-paste enabled, is also copied to the clipboard for manual paste (the Copy button confirms for six seconds). The pill window is content-protected while the panel is open (the panel never appears in screen shares).+A dedicated global hotkey that starts a dictation whose transcript is sent straight to the voice assistant as a command — no wake word ("Hey [AgentName]") needed — and that always bypasses the cleanup model. With auto-paste enabled, a caret verified as writable at capture time (an opaque `caret` session in `selectionManager.js`, revalidated before pasting) receives the completed answer and the pill returns to idle. A dormant macOS accessibility tree reports "unknown": synthetic copy can still recover selected text, but cannot establish a safe caret destination. Unknown targets therefore stay on the panel route, since they can also be integrated terminals, pages without inputs, or fields with selections hidden by clipboard capture. A verified caret whose accessibility becomes unknown before delivery falls back the same way. Terminal and secure-field checks still apply to verified targets. Otherwise the answer streams into a floating assistant panel attached to the dictation pill (there is no separate assistant window) and, with auto-paste enabled, is also copied to the clipboard for manual paste (the Copy button confirms for six seconds). The pill window is content-protected while the panel is open (the panel never appears in screen shares).**Flow**:diff --git a/resources/macos-text-monitor.swift b/resources/macos-text-monitor.swiftindex 1048acdf3..bec23d687 100644--- a/resources/macos-text-monitor.swift+++ b/resources/macos-text-monitor.swift@@ -1,3 +1,4 @@+import Carbonimport Cocoaimport Foundationimport Darwin@@ -31,6 +32,16 @@ func writeTextOutput(_ prefix: String, _ value: String) {writeOutput("\(prefix):\(truncated)")}+func elementRole(_ element: AXUIElement) -> String? {+ var roleValue: AnyObject?+ guard AXUIElementCopyAttributeValue(+ element,+ kAXRoleAttribute as CFString,+ &roleValue+ ) == .success else { return nil }+ return roleValue as? String+}+func isEditableTextElement(_ element: AXUIElement) -> Bool {// A live selection means an editable verdict would let generated text// paste over the user's highlighted text (--editable-target mode probes@@ -62,12 +73,7 @@ func isEditableTextElement(_ element: AXUIElement) -> Bool {return false}- var roleValue: AnyObject?- guard AXUIElementCopyAttributeValue(- element,- kAXRoleAttribute as CFString,- &roleValue- ) == .success, let role = roleValue as? String,+ guard let role = elementRole(element),["AXTextField", "AXTextArea", "AXComboBox"].contains(role)else {return false@@ -201,16 +207,35 @@ for attempt in 1...maxRetries {}}+// The editable probe answers in three states: UNKNOWN is the dormant-tree+// signature (Chromium-family apps never resolve a focused element, or resolve+// only the bare window), telling the caller accessibility cannot see the field+// — distinct from a resolved non-text element, a confirmed NOT_EDITABLE.+if editableTargetMode {+ // Secure input is a session-global flag set while a password field holds+ // keyboard focus, covering the fields a dormant tree hides from the+ // AXSecureTextField subrole check.+ if IsSecureEventInputEnabled() {+ writeOutput("NOT_EDITABLE")+ exit(0)+ }+ guard let element = focusedElement else {+ writeOutput("UNKNOWN")+ exit(0)+ }+ if isEditableTextElement(element) {+ writeOutput("EDITABLE")+ } else {+ writeOutput(elementRole(element) == "AXWindow" ? "UNKNOWN" : "NOT_EDITABLE")+ }+ exit(0)+}+guard let resolvedElement = focusedElement else {writeOutput("NO_ELEMENT")exit(1)}-if editableTargetMode {- writeOutput(isEditableTextElement(resolvedElement) ? "EDITABLE" : "NOT_EDITABLE")- exit(0)-}-// Selection capture is a short-lived, read-only mode used before an AI edit.// It deliberately uses the same AX client as the monitor so a packaged app's// trusted native helper does not need to rely on System Events automation.@@ -232,7 +257,11 @@ if selectionReadMode {}if selectionResult == .noValue || selectionResult == .attributeUnsupported {- writeOutput("NONE:")+ // A dormant Chromium tree reports the window itself as the focused+ // element; its unsupported AXSelectedText is not evidence of an empty+ // selection, so surface it as UNKNOWN and let the synthetic-copy path+ // read the real selection state.+ writeOutput(elementRole(resolvedElement) == "AXWindow" ? "UNKNOWN:" : "NONE:")exit(0)}diff --git a/src/helpers/selectionManager.js b/src/helpers/selectionManager.jsindex 0396efb26..a1d897c69 100644--- a/src/helpers/selectionManager.js+++ b/src/helpers/selectionManager.js@@ -336,9 +336,10 @@ class SelectionManager {};}// Chromium browsers (and any app whose accessibility tree stays dormant)- // never resolve a focused element, so the read above cannot tell a selection- // from an empty field. A synthetic copy still can — the same route Windows- // and Linux take by default.+ // never resolve a focused element — or resolve only the bare window (the+ // native read's "unknown" state) — so the read above cannot tell a+ // selection from an empty field. A synthetic copy still can — the same+ // route Windows and Linux take by default.return this._readMacSelectionViaClipboard(pid, expectedTarget, probeEditable);}@@ -498,8 +499,24 @@ class SelectionManager {// to match — so resolve the executable name before a Wayland terminal's// empty prompt can read as a writable caret.if (target.kind === "atspi-pid" && (await this._isTerminalPid(target.id))) return capture;- const editable = await this.textEditMonitor?.isFocusedEditable?.(target);- return editable ? { status: "editable", target } : capture;+ // macOS targets normally carry the copier's NSWorkspace app name, already+ // matched above; only an unnamed target needs its executable resolved so a+ // terminal cannot become a caret destination. Never resolve a+ // named one — `ps` reports bundle paths like "Visual Studio Code" whose+ // "st" substring would misread editors as terminals.+ if (+ target.kind === "mac-pid" &&+ !this._targetSignature(target) &&+ (await this._isTerminalPid(target.pid))+ ) {+ return capture;+ }+ const verdict = await this.textEditMonitor?.isFocusedEditable?.(target);+ if (verdict === "editable") return { status: "editable", target };+ // An empty copy cannot distinguish a caret from an integrated terminal or+ // a page without an input; it can also hide a selection matching the clipboard.+ // Unknown accessibility must keep the panel fallback, including on revalidation.+ return capture;}_isTerminalTarget(...targets) {@@ -630,9 +647,8 @@ class SelectionManager {// Wayland) still holds pre-copy content; snapshot it so stale text can't// be mistaken for the copied selection. Known limitation: a clipboard that// already held exactly the selected text reads as "no selection". The- // command then falls back to the Assistant panel — never to a caret paste,- // because the editable probe reads the focused element's own selection- // state and refuses a field with a live selection.+ // editable probe must independently verify an empty writable field before+ // caret delivery; a live selection or unknown accessibility keeps the panel.const baseline = new Set([...beforeWrite, ...this.clipboardManager._readClipboardTextAll()]);const copyResult = await sendCopy();diff --git a/src/helpers/textEditMonitor.js b/src/helpers/textEditMonitor.jsindex a36d443dc..d92c9e317 100644--- a/src/helpers/textEditMonitor.js+++ b/src/helpers/textEditMonitor.js@@ -226,6 +226,13 @@ class TextEditMonitor extends EventEmitter {resolve({ state: "none" });return;}+ // The focused element resolved but is the bare window — the+ // dormant-Chromium signature. The selection state is unreadable,+ // not empty; the caller's synthetic-copy fallback can still read it.+ if (output === "UNKNOWN:") {+ resolve({ state: "unknown" });+ return;+ }if (output.startsWith("SELECTED_B64:")) {try {resolve({@@ -297,14 +304,18 @@ class TextEditMonitor extends EventEmitter {);}- async isFocusedEditable(target, timeoutMs = 1000) {+ // Resolves "editable" | "not_editable" | "unknown". "unknown" is macOS-only:+ // the dormant-Chromium signature, where accessibility cannot inspect the+ // focused field at all. The default timeout reuses SELECTED_TEXT_TIMEOUT_MS+ // because a dormant tree's verdict only arrives after the same retry ladder.+ async isFocusedEditable(target, timeoutMs = SELECTED_TEXT_TIMEOUT_MS) {const resolved = this.resolveBinary();- if (!resolved) return false;+ if (!resolved) return "not_editable";let args;if (process.platform === "darwin") {const pid = target?.kind === "mac-pid" ? target.pid : this.lastTargetPid;- if (!pid) return false;+ if (!pid) return "not_editable";args = [...resolved.args, "--editable-target", String(pid)];} else {args = [...resolved.args, "--probe-editable"];@@ -315,6 +326,8 @@ class TextEditMonitor extends EventEmitter {// then emits its monitor output and keeps running — closing stdin and// resolving on that first line keeps the stale case a fast "not editable"// rather than a hang until the timeout.+ const toVerdict = (line) =>+ line === "EDITABLE" ? "editable" : line === "UNKNOWN" ? "unknown" : "not_editable";return new Promise((resolve) => {const child = spawn(resolved.command, args, { stdio: ["pipe", "pipe", "ignore"] });let buffered = "";@@ -328,16 +341,16 @@ class TextEditMonitor extends EventEmitter {} catch {}resolve(verdict);};- const timer = setTimeout(() => settle(false), timeoutMs);+ const timer = setTimeout(() => settle("not_editable"), timeoutMs);child.stdin.on("error", () => {});child.stdin.end();child.stdout.on("data", (data) => {buffered += data.toString();const newline = buffered.indexOf("\n");- if (newline !== -1) settle(buffered.slice(0, newline).trim() === "EDITABLE");+ if (newline !== -1) settle(toVerdict(buffered.slice(0, newline).trim()));});- child.on("error", () => settle(false));- child.on("close", () => settle(buffered.trim() === "EDITABLE"));+ child.on("error", () => settle("not_editable"));+ child.on("close", () => settle(toVerdict(buffered.trim())));});}diff --git a/test/helpers/selectionManager.test.js b/test/helpers/selectionManager.test.jsindex a646842c6..4b0e1b7f9 100644--- a/test/helpers/selectionManager.test.js+++ b/test/helpers/selectionManager.test.js@@ -142,7 +142,7 @@ test("a terminal-flagged target is refused as a caret destination without probintextEditMonitor: {isFocusedEditable: async (target) => {probes.push(target);- return true;+ return "editable";},},platform: "win32",@@ -182,7 +182,7 @@ test("a Linux AT-SPI terminal pid never becomes a caret delivery target", asynctextEditMonitor: {isFocusedEditable: async (target) => {probes.push(target);- return true;+ return "editable";},},platform: "linux",@@ -562,7 +562,7 @@ test("a terminal target reads as no selection", async () => {};const manager = new SelectionManager({clipboardManager,- textEditMonitor: { isFocusedEditable: async () => true },+ textEditMonitor: { isFocusedEditable: async () => "editable" },platform: "linux",now: () => 1000,});@@ -648,6 +648,122 @@ test("a macOS line copy in a line-copy editor reads as no selection", async () =assert.equal((await manager.captureSelectedText()).status, "none");});+test("an unverifiable macOS field with no selection stays on the panel route", async () => {+ const { manager } = makeMacClipboardHarness({ copied: null });+ manager.textEditMonitor.isFocusedEditable = async () => "unknown";++ const result = await manager.captureSelectedText({ probeEditable: true });+ assert.equal(result.status, "none");+ assert.equal(result.sessionId, undefined);+});++test("a confirmed non-editable macOS focus stays on the panel route", async () => {+ const { manager } = makeMacClipboardHarness({ copied: null });+ manager.textEditMonitor.isFocusedEditable = async () => "not_editable";++ assert.equal((await manager.captureSelectedText({ probeEditable: true })).status, "none");+});++test("an unknown verdict in a terminal is still refused as a caret", async () => {+ const { manager } = makeMacClipboardHarness({ copyOutput: "COPY_OK 42 Ghostty", copied: null });+ manager.textEditMonitor.isFocusedEditable = async () => "unknown";++ assert.equal((await manager.captureSelectedText({ probeEditable: true })).status, "none");+});++// COPY_OK reports the app name best-effort; when it is missing, the executable+// must be resolved before an unnamed terminal can be trusted as a caret.+test("an unnamed macOS target resolves the executable before trusting a caret", async () => {+ const { manager } = makeMacClipboardHarness({ copyOutput: "COPY_OK 42", copied: null });+ manager.textEditMonitor.isFocusedEditable = async () => "editable";+ manager._readExecutablePath = async () => "/Applications/Ghostty.app/Contents/MacOS/ghostty";++ assert.equal((await manager.captureSelectedText({ probeEditable: true })).status, "none");+});++test("a caret verified after clipboard capture delivers the assistant response", async () => {+ const { manager } = makeMacClipboardHarness({ copied: null });+ const pastes = [];+ manager.textEditMonitor.isFocusedEditable = async () => "editable";+ manager.clipboardManager._pasteText = async (text, options) => {+ pastes.push({ text, options });+ return { restoreComplete: Promise.resolve() };+ };++ const capture = await manager.captureSelectedText({ probeEditable: true });+ assert.deepEqual(await manager.pasteAtCapturedTarget(capture.sessionId, "Agent response"), {+ success: true,+ });+ assert.equal(pastes[0].text, "Agent response");+});++for (const { name, appName, copied } of [+ { name: "selection matches the clipboard", appName: "Dia", copied: "user clipboard" },+ { name: "selection is a full code line", appName: "Code", copied: "const value = 1;\n" },+ { name: "focus moves to an integrated terminal", appName: "Code", copied: null },+ { name: "focus moves to a browser page", appName: "Chrome", copied: null },+]) {+ test(`a verified caret stops delivery when AX becomes unknown and ${name}`, async () => {+ const { manager } = makeMacClipboardHarness({ copyOutput: `COPY_OK 42 ${appName}`, copied });+ const pastes = [];+ manager.textEditMonitor.getSelectedText = async () => ({ state: "none", editable: true });+ manager._readExecutablePath = async () => appName;+ manager.clipboardManager._pasteText = async (text) => {+ pastes.push(text);+ return { restoreComplete: Promise.resolve() };+ };+ const capture = await manager.captureSelectedText({ probeEditable: true });+ assert.equal(capture.status, "editable");++ // The app/PID is unchanged, but the focused field is no longer inspectable.+ manager.textEditMonitor.getSelectedText = async () => ({ state: "unknown" });+ manager.textEditMonitor.isFocusedEditable = async () => "unknown";++ assert.deepEqual(await manager.pasteAtCapturedTarget(capture.sessionId, "Agent response"), {+ success: false,+ code: "target_changed",+ });+ assert.deepEqual(pastes, []);+ });+}++test("an assistant answer remains available when a dormant browser has no input", async () => {+ const { createAssistantResponseDelivery, deliverAssistantResponse } =+ await import("../../src/helpers/assistantResponseDelivery.ts");+ const { manager } = makeMacClipboardHarness({ copyOutput: "COPY_OK 42 Chrome" });+ const pastes = [];+ const writes = [];+ manager.textEditMonitor.getSelectedText = async () => ({ state: "unknown" });+ manager.textEditMonitor.isFocusedEditable = async () => "unknown";+ // Posting Cmd+V succeeds even when there is no field to receive it.+ manager.clipboardManager._pasteText = async (text) => {+ pastes.push(text);+ return { restoreComplete: Promise.resolve() };+ };++ const capture = await manager.captureSelectedText({ probeEditable: true });+ const delivery = createAssistantResponseDelivery({+ autoPasteEnabled: true,+ deliverySessionId: capture.sessionId,+ restoreClipboard: true,+ allowClipboardFallback: false,+ });+ const result = await deliverAssistantResponse(delivery, "Agent response", {+ electronAPI: {+ pasteAtCapturedTarget: (...args) => manager.pasteAtCapturedTarget(...args),+ writeClipboard: async (text) => {+ writes.push(text);+ return { success: true };+ },+ },+ clipboard: {},+ });++ assert.deepEqual(result, { pasted: false, copied: true });+ assert.deepEqual(pastes, []);+ assert.deepEqual(writes, ["Agent response"]);+});+test("a failed macOS copy stays non-fatal so the command still runs", async () => {const { manager } = makeMacClipboardHarness({ copied: null });manager._runCopyHelper = async () => ({ success: false, stdout: "", stderr: "" });diff --git a/test/helpers/textEditMonitorSelection.test.js b/test/helpers/textEditMonitorSelection.test.jsindex d706d73c1..449a7cb36 100644--- a/test/helpers/textEditMonitorSelection.test.js+++ b/test/helpers/textEditMonitorSelection.test.js@@ -99,17 +99,25 @@ test("getSelectedText identifies an empty writable field", darwinOnly, async ()assert.deepEqual(await m.getSelectedText(42), { state: "none", editable: true });});-test("isFocusedEditable accepts only an explicit native editable verdict", darwinOnly, async () => {- const editable = new TextEditMonitor();- editable.resolveBinary = () => ({ command: "/bin/sh", args: ["-c", 'echo "EDITABLE"'] });- const readOnly = new TextEditMonitor();- readOnly.resolveBinary = () => ({- command: "/bin/sh",- args: ["-c", 'echo "NOT_EDITABLE"'],- });-- assert.equal(await editable.isFocusedEditable({ kind: "mac-pid", pid: 42 }), true);- assert.equal(await readOnly.isFocusedEditable({ kind: "mac-pid", pid: 42 }), false);+test("isFocusedEditable maps the native verdicts, including the dormant-tree UNKNOWN", darwinOnly, async () => {+ const withOutput = (line) => {+ const m = new TextEditMonitor();+ m.resolveBinary = () => ({ command: "/bin/sh", args: ["-c", `echo "${line}"`] });+ return m;+ };++ assert.equal(+ await withOutput("EDITABLE").isFocusedEditable({ kind: "mac-pid", pid: 42 }),+ "editable"+ );+ assert.equal(+ await withOutput("NOT_EDITABLE").isFocusedEditable({ kind: "mac-pid", pid: 42 }),+ "not_editable"+ );+ assert.equal(+ await withOutput("UNKNOWN").isFocusedEditable({ kind: "mac-pid", pid: 42 }),+ "unknown"+ );});test("isFocusedEditable answers from a stale binary's first monitor line", darwinOnly, async () => {@@ -123,10 +131,20 @@ test("isFocusedEditable answers from a stale binary's first monitor line", darwi});const startedAt = Date.now();- assert.equal(await m.isFocusedEditable({ kind: "mac-pid", pid: 42 }), false);+ assert.equal(await m.isFocusedEditable({ kind: "mac-pid", pid: 42 }), "not_editable");assert.ok(Date.now() - startedAt < 900, "must resolve on the first output line");});+// The bare window as the focused element is the dormant-Chromium signature —+// the selection state is unreadable, not empty, so the caller must fall to the+// synthetic-copy read instead of treating it as "no selection".+test("getSelectedText surfaces the native UNKNOWN verdict", darwinOnly, async () => {+ const m = new TextEditMonitor();+ m.resolveBinary = () => ({ command: "/bin/sh", args: ["-c", 'echo "UNKNOWN:"'] });++ assert.deepEqual(await m.getSelectedText(42), { state: "unknown" });+});+test("a non--25212 native failure is not cached", darwinOnly, async () => {const m = new TextEditMonitor();m.resolveBinary = () => ({